Waves of records (or structs)
cfneese
Tue, 07/05/2011 - 02:58 pm
It is all too easy to destroy the database table via a InsertPoints, DeletePoints, Redimension, or Sort that doesn't include all of the waves that comprise the table.
A wave of records could easily solve this problem. If the fields of the records were internally stored as waves, then the changes to Igor could probably be limited to just a few operations.
The syntax could look like:
Make/REC={TextField/T,DoubleField/D,ComplexField/C} wave0
AddField wave0, NewTextField/T, NewDoubleField/D, NewComplexField/C
DeleteField[/Z] wave0, FieldToDelete
AddField wave0, NewTextField/T, NewDoubleField/D, NewComplexField/C
DeleteField[/Z] wave0, FieldToDelete
You may want to consider using a wave containing wave references as a sort of aggregate. The difficulty here is that your data are of different types, which probably precludes this option.
Igor has only very limited support for polymorphism. As such it is unlikely that you could use Sort, or most other operations for that matter, on an nontrivial aggregate data type (if such a possibility were to be introduced in a future version of Igor).
July 5, 2011 at 07:02 pm - Permalink
I'm not sure if this is exactly what you're looking for, but you could do something like the following (requires at least Igor 6.10):
Make/T textwave
Make/C complexwave
Make/N=3/WAVE/O databaseWave
databaseWave[0] = doublewave
databaseWave[1] = textwave
databaseWave[2] = complexwave
print/D databaseWave
databaseWave[0]= {51149788,51149812,51149816}
As 741 touched on, there isn't a great way in Igor to store related pieces of data of different types. Structures are sometimes useful, but they aren't global and so the data they contain must be stored in some other way outside of when a function is executing.
If you need to do things with the data that would usually be done in a database, such as queries, you may be able to use the Extract operation, but if your queries are complex you may run into problems. For example, you could probably accomodate a statement like
SELECT name FROM students WHERE age=21<br />
But if you need to do something like a JOIN, that would take more coding.
If your data is really structured like data in a typical SQL database, you may consider storing it that way and using the SQL XOP, which ships with Igor, to access and manipulate the data.
If you need further assistance it would be helpful if you could provide more details on your data and what you need to do with it.
July 6, 2011 at 08:12 am - Permalink
Reading aclight's reply, I just remembered something John posted here a while ago: in a related problem he proposed using StructPut and StructGet to get persistent storage of structs in waves. While not entirely straightforward, it could be a useful workaround for the above.
July 6, 2011 at 01:59 pm - Permalink
Just to be a bit more clear, 741 is talking about storing structures as compressed strings in a text wave (a text wave is actually an array of strings), where each text wave element contains one compressed structure.
If you take this approach, make sure to define your structure(s) with a version field that you use to verify that the content is what you expect. This can be used to protect yourself against incompatible future revisions.
--Jim Prouty
Software Engineer, WaveMetrics, Inc.
July 29, 2011 at 02:27 pm - Permalink
StructPut and StructGet do not work with structures containing pointer-like fields such as WAVE, NVAR and SVAR fields. This is because these pointers are not valid after the object they are pointing to has been killed and possibly recreated, such as when you close and reopen an experiment.
There is currently no straightforward solution to the "wave of structures". Though you may be able to come up with a facsimile of a solution through heroic efforts, I think this would be putting a square peg into a round hole and you would wind up creating something fragile.
July 31, 2011 at 09:46 am - Permalink
Since Igor does not support global record types, one has to maintain record type data in waves where each wave represents one field. This works fine, but there are a few Igor operations that can destroy the structure if used incorrectly. Let's call these operations the SMOPS. To my knowledge, the SMOPS are DeletePoints, IndexSort, InsertPoints, Redimension, Reverse, Rotate, and Sort. I think it would be a tremendous feature if waves could be flagged as belonging to a set. I will call such a set a DataSet. A DataSet could also simply be a flag on the DataFolder containing the waves.
The idea would be to modify the SMops so that they complain if a wave is in a DataSet, and extend these operations with new syntax so that they can operate on a DataSet collectively. If DataSets were built on top of folders, then you would need to extend Make as well. If DataSets were a new construct you would need AddToDataSet and RemoveFromDataSet ops.
Another option would be to protect waves in a DataSet that are in a set by extending SetWaveLock so that one can choose how to protect the wave. Currently SetWaveLock complains if lockVal is not 0 or 1. So maybe it would be possible to add some more bits to lockVal? With an appropriate modification of SetWaveLock, everything else could be done in a package. I could already write a DataSet package to do a good portion of the work, but the only thing that I have available to protect the member waves is SetWaveLock which effectively makes them read only and is too restrictive for my purposes.
July 18, 2016 at 08:19 am - Permalink
Make/WAVEintroduced a long time ago.Or why not serialize a structure to a datafolder?
A structure can hold the following data types:
Variable, String, WAVE, NVAR, SVAR, DFREF, FUNCREF, or STRUCT and PODs like char, uchar...
Serializing variable/string/wave/PODs is straight forward.
NVAR/SVAR would need to be serialized to a string (Igor internally probably knows the name of the global you are refering to)
DFREF can be stored in a /DF wave.
FUNCREF would need to be serialized to a string as well.
STRUCTS itself are done recursively.
And because a free data folder is also a datafolder you could in that way store the serialized struct in a wave.
So much for the night-time-smart-assing ;)
PS: There seems to be no way to implement this in a XOP for arbitrary structures. Even "Extended Structure Parameters" don't allow the XOP to iterate over the structure members. Only Igor itself seems to be able to do that (e.g. print).
July 29, 2016 at 04:30 pm - Permalink