netCDF utilities

New features in Igor 9 now allow the correct reading and writing of netCDF-4 files directly from within Igor. Igor 8 already added native netCDF-3 loading ability and Igor 7 added support for the extended data-types of netCDF-4 e.g. uint64 etc. Here are two sets of utilities to do both reading and writing using all in-built operations and user defined functions.

QLA netCDF reader

  • Loads netCDF-3 and netCDF-4 files such that the original type is transparent to the user 
  • Parse all netCDF metadata nicely into CDL syntax attached to the WaveNote (c.f. ncdump or Panoply)
  • Set the units and dimension names of the dataset
  • Optionally apply scale factor, offset, fill value, valid range to data
  • Functions to list, get, or set the attributes of a dataset
  • Menu is provided for convenience, inline is also perfectly fine 

QLA netCDF4 library

  • Functions to create netCDF-4 or netCDF-4 'classic model' files directly from Igor
  • Switch between types easily without changing any code
  • Keeps track of dimension information
  • Keeps track of _Netcdf4Dimid and _Netcdf4Coordinates required hidden attributes
  • Keeps track of unlimited dimensions, layout, and chunking
  • Keeps track of the hierarchy, duplicate names, dimensions size matching etc.
  • Provides some basic help for keeping CF compliance
  • Provides some hopefully helpful error/information messages in the history area
  • Compression and shuffling is available
  • Functions follow similar syntax to netCDF C library functions
  • Functions vaguely named after netCDF functions e.g. NC_def_var becomes QLA_def_var

A sample .ipf is included which uses almost all the functions available that you can use to make your own netCDF-4 / netCDF-4 classic and figure out how it all works, test the files using ncdump, nccopy, or the included loader etc.

Limitations:
  • Only HDF5 based netCDF-4 files can be created directly. To create a netCDF-3 use the netCDF API nccopy.exe to  convert your netCDF-4 classic model into a netCDF-3 i.e.
// convert a netCDF-4 classic into a netCDF-3
ExecuteScriptText /B /Z "cmd.exe /C nccopy -3 myNC4classic.nc myNC3.nc
  • It is only possible to create scalar 'double' variables
  • More exotic datatypes like ENUM are probably possible but aren't currently included in the library 

All credit goes to Howard Rodstein for all his effort with changes and additions to the HDF5operations within Igor. These include but are probably not limited to:

  1. New HDF5DimensionScales operation - this is what makes creating netCDF-4 dimensions possible
  2. HDF5 attribute creation order tracking and indexing
  3. Fixed length HDF5 string attributes
  4. Loading of VLEN type data needed for reading netCDF-4 dimensions 
  5. HDF5SaveData /STRF flag for explicit control of text formatting

Images:

Dimension information Global attributesinformation Variable attribute information

Project Details

Current Project Release

Release File: QLA_netCDF_utilities_140.zip
Version: IGOR_9.00x_v140
Version Date: Sun, 08/01/2021 - 07:13 am
Version Major: 1
Version Extra: 0
Version Patch Level: 4
OS Compatibility: Windows Mac-Intel
Release Notes:

QLA_netCDF4_library:

  • Added features:
  1. It is no longer necessary to define a group using QLA_def_grp before creating datasets or attributes of that group. If a none-existant group is referenced by any of QLA_def_dim_var, QLA_def_unlim_dim_var, QLA_def_dim, QLA_def_unlim_dim, QLA_def_var, QLA_put_glob_att, QLA_put_glob_att_text, QLA_put_glob_att_list, QLA_put_glob_att_wave then the group will be created automatically. 
  • Changed behaviour:
  1. QLA_def_dim and QLA_def_unlim_dim by default no longer write any data to file irrespective of what is passed to them. As a result delfate and shuffle also have no effect. This behaviour is controlled by the writeDim library constant, to restore writing of data fo file set writeDim=1.    
  • Performance enhancements:
  1. The time required to determine whether a string variable is really char has been reduced.

QLA_netCDF_reader:

  • Added features:
  1. Added illegalPreStr and illegalSuffStr strings which should be prepended or appended to any netCDF variable which has a name that conflics with any function or operation. One or both of either illegalPreStr or illegalSuffStr can be defined, or if both are zero length strings ("") then the default behaviour of appending a number to the variable name is restored.
  • Performance enhancements:
  1. The time required to unpack a variable has been reduced by ~⅓ through the use of multithreading.
  2. The time required to determine whether a string variable is really char has been reduced.
  • Other:
  1. The documentation for the unpack option was fixed to reflect that unpack=0 not only range checks and scales data where appropriate, but also sets the fill value to NaN in floating point variables (float/real & double).

Included files:

  1. QLA_netCDF4_sample.ipf
  2. QLA_netCDF4_library.ipf
  3. QLA_netCDF4_library_help.ihf
  4. QLA_netCDF_reader.ipf
  5. QLA_netCDF_reader_help.ihf
  6. string2text2char2byte.ipf 
View All Releases

Forum

Support

Gallery

Igor Pro 8

Learn More

Igor XOP Toolkit

Learn More

Igor NIDAQ Tools MX

Learn More