sds is a command line program that can perform serveral basic operations on a DataSet. Scientific DataSet installer adds a path to
sds to the environment variable PATH, so you can run it from a command line just by typing a command
sds.
Structure of a DataSet
sds or
sds list commands take an SDS file path or
DataSet URI and display a list of variables in the DataSet.
ExampleFile air.mon.mean.nc can be downloaded (126 MB) from ftp://ftp.cdc.noaa.gov/Datasets/ncep.reanalysis2.derived/pressure/air.mon.mean.nc.
>sds air.mon.mean.nc
[6] air of type Int16 (time:372) (level:17) (lat:73) (lon:144)
[5] time_bnds of type Double (time:372) (nbnds:2)
[4] time of type Double (time:372)
[3] lon of type Single (lon:144)
[2] lat of type Single (lat:73)
[1] level of type Single (level:17)
The tool displays a list of variables in the DataSet opened from the URI. Each variable summary shows the Variable ID, Name, TypeOfData, and shape. Variable shape includes one dimension for vectors, two dimensions for matrices, and so on. Variable summary shows both name and length for each of the dimensions.
Reading metadata
Command
sds meta prints metadata of a DataSet. In particular,
sds meta <DataSet URI> prints the global metadata for a DataSet.
sds meta <DataSet URI> <var> ... prints the metadata for individual variables. Each
<var> is either a name of a variable or ID.
Example
>sds meta air.mon.mean.nc
Name = air.mon.mean.nc
Conventions = CF-1.0
title = Monthly NCEP/DOE Reanalysis 2
history = created 2002/03 by Hoop (netCDF2.3)
comments = Data is from
NCEP/DOE AMIP-II Reanalysis (Reanalysis-2)
(4x/day). It consists of most variables interpolated to
pressure surfaces from model (sigma) surfaces.
platform = Model
source = NCEP/DOE AMIP-II Reanalysis (Reanalysis-2) Model
institution = National Centers for Environmental Prediction
references = http://wesley.wwb.noaa.gov/reanalysis2/
http://www.cdc.noaa.gov/cdc/data.reanalysis2.html
>sds meta air.mon.mean.nc air
[6] air of type Int16 (time:372) (level:17) (lat:73) (lon:144)
Name = air
long_name = Monthly Air Temperature on Pressure Levels
valid_range = -32765 -10260
unpacked_valid_range = 137.5 362.5
actual_range = 179.4077 315.7219
units = degK
add_offset = 465.15
scale_factor = 0.01
missing_value = 32766
_FillValue = -32767
precision = 2
least_significant_digit = 1
GRIB_id = 11
GRIB_name = TMP
var_desc = Air temperature
dataset = NCEP/DOE AMIP-II Reanalysis (Reanalysis-2) Monthly Averages
level_desc = Pressure Levels
statistic = Individual Obs
parent_stat = Other
standard_name = air_temperature
cell_methods = time: mean (monthly from 6-hourly values)
Updating metadata
Command
sds update lets you update the metadata for a DataSet:
sds update <DataSet URI> [/g|<var>] [/t:<Type>] <key> <value> [<value> ...]
- You must specify either the /g flag or <var> after update. The /g flag updates global metadata; <var> is the name or ID of a variable to update.
- The optional /t flag specifies the type of the attribute. Possible values are the names of any .NET types that the MetadataDictionary supports (see Supported Types). Namespace System can be omited. Examples: Int32, Double, DateTime[]. If the flag is missing, sdsutil tries to infer one of the following types from the actual <value> string: Double, DateTime, Boolean, String, Double[], DateTime[], Boolean[], String[].
- <key> is a name of an attribute to set.
- <value> [<value> ...] is a single value or an array of values to set for the attribute.
ExampleSets the attribute "avg" of variable "air" from DataSet "air.nc" equal to 12.5. Inferred type is "Double".
sds update air.nc air avg 12.5
Sets the attribute "descr" of variable "air" from DataSet "air.nc" equal to string "100.0".
sds update air.nc air /t:String descr 100.0
Sets the attribute "range" of variable "air" from DataSet "air.nc" equal to array { -50, 50 }. Inferred type is "Double[]".
sds update air.nc air range -50 50
Sets the global attribute "range" of DataSet "air.nc" equal to array of Int32 { -50, 50 }.
sds update air.nc /g /t:Int32[] range -50 50
Reading data
Command
sds data displays the data of a variable or its subset.
sds data <DataSet URI> [ <varstride> ... ], where
<varstride> == <var>[<range>,...]{<cellsize>:<dataformat>}
- Each <varstride> specifies a variable and its data subset to display. If <varstride> is omitted, sdsutil displays data for every variable in the DataSet.
- <var> is either a variable name or ID.
- Optional [<range>,...] element specifies a subset of data. If you omit the [<range>,...], sdsutil displays all data for the variable. The number of <range> elements must be equal to the rank of the variable. Thus, each <range> is start:stop or start:step:stop, where all start,step,stop are indices by the related dimension and also can be omitted. If start is omitted, the subset starts with zero; if step is missing, it is 1; if stop is omitted, the subset ends with the last element in that dimension.
- Optional {<cellsize>:<dataformat>} defines the ouput format for every data element. Both <cellsize> and <dataformat> are optional. For details about the syntax, seethe MSDN Library page Composite Formatting.
Example
>sds data air.mon.mean.nc level
[1] level of type Single (level:17)
Name = level
units = millibar
actual_range = 1000 10
long_name = Level
positive = down
GRIB_id = 100
GRIB_name = hPa
axis = z
coordinate_defines = point
[0] 1000 925 850 700 600 500 400 300
[8] 250 200 150 100 70 50 30 20
[16] 10
>sds data air.mon.mean.nc level{:E1}
[1] level of type Single (level:17)
Name = level
units = millibar
actual_range = 1000 10
long_name = Level
positive = down
GRIB_id = 100
GRIB_name = hPa
axis = z
coordinate_defines = point
[0] 1.0E+003 9.3E+002 8.5E+002 7.0E+002 6.0E+002 5.0E+002 4.0E+002 3.0E+002
[8] 2.5E+002 2.0E+002 1.5E+002 1.0E+002 7.0E+001 5.0E+001 3.0E+001 2.0E+001
[16] 1.0E+001
>sds data air.mon.mean.nc air[0,1,:10:,:20:]
[6] air of type Int16 (time:372) (level:17) (lat:73) (lon:144)
Name = air
long_name = Monthly Air Temperature on Pressure Levels
valid_range = -32765 -10260
unpacked_valid_range = 137.5 362.5
actual_range = 179.4077 315.7219
units = degK
add_offset = 465.15
scale_factor = 0.01
missing_value = 32766
_FillValue = -32767
precision = 2
least_significant_digit = 1
GRIB_id = 11
GRIB_name = TMP
var_desc = Air temperature
dataset = NCEP/DOE AMIP-II Reanalysis (Reanalysis-2) Monthly Averag
es
level_desc = Pressure Levels
statistic = Individual Obs
parent_stat = Other
standard_name = air_temperature
cell_methods = time: mean (monthly from 6-hourly values)
[0,1,0,0] -21660 -21660 -21660 -21660 -21660 -21660 -21660
[0,1,0,140] -21660
[0,1,10,0] -19522 -20852 -23004 -21973 -19889 -21282 -20365
[0,1,10,140] -19773
[0,1,20,0] -18500 -18964 -19141 -19464 -18560 -19163 -18683
[0,1,20,140] -18500
[0,1,30,0] -16681 -17362 -16848 -17181 -17433 -17239 -17335
[0,1,30,140] -16768
[0,1,40,0] -17312 -17059 -17065 -16824 -16989 -17404 -16984
[0,1,40,140] -17427
[0,1,50,0] -17906 -17604 -18021 -17105 -17743 -17845 -17180
[0,1,50,140] -17847
[0,1,60,0] -19466 -19302 -19369 -19257 -19105 -19132 -19026
[0,1,60,140] -19420
[0,1,70,0] -19884 -20185 -20376 -20062 -19860 -19730 -19706
[0,1,70,140] -19832
Copying DataSet
Command
sds copy copies DataSet into another DataSet.
sds copy <DataSet URI 1> <DataSet URI 2>
- First <DataSet URI 1> defines the DataSet to copy, and the second defines the output DataSet.
- If an output DataSet must be new and it supports "openMode" parameter, specify the extra URI parameter "openMode=create" or "openMode=createNew".
- Providers for the input and output DataSets can be different. Thus, the command can convert a DataSet from one format to another.
- If the output DataSet is a NetCDFDataSet, specifiy the extra URI parameter "enableRollback=false" to increase performance.
- If the input DataSet is too large to fit in memory, sdsutil copies it by parts.
Example
>sds copy air.mon.mean.nc?openMode=open air.mon.mean.csv?openMode=create
Copying . . .
Creating structure and copying global metadata and scalar variables... Done.
Total memory capacity: 200.00 Mb
Deltas for the dimensions adjusted (max iteration capacity: 126.81 Mb):
Dimension time: 372
Dimension level: 17
Dimension lat: 73
Dimension lon: 144
Dimension nbnds: 2
Copying data... 100%
(00:01:55.5712749) Done.
Printing information
Command
sds info displays DataSet Core version, used by sdsutil, and list of available providers.
>sds info
DataSet Core version 1.2.5657.0
Provider as: class ASGridDataSet [1.2.5657.0]
Provider csv: class CsvDataSet [1.2.5657.0]
Provider asciigrid: class EsriGridDataSet [1.2.5657.0]
Provider memory: class MemoryDataSet [1.2.5657.0]
Provider memory2: class MemoryDataSet [1.2.5657.0]
Provider nc: class NetCDFDataSet [1.2.5657.0]
Provider wcf: class WcfDataSetFactory [1.2.5657.0]
Provider remoting: class RemotingDataSetFactory [1.2.5657.0]
Extension .csv: CsvDataSet
Extension .tsv: CsvDataSet
Extension .asc: EsriGridDataSet
Extension .nc: NetCDFDataSet