I would like to be able to use longer wave names
seanokeefe
Tue, 05/25/2010 - 07:36 am
The Igor WAVENAME limit is 32 characters, and I run into problems with this all the time.
If there is a simple solution to this problem, then could someone please point me there?? If not, then please keep reading to see exactly what my problem is.
Like many of you, I take lots of data, and I like to be able to identify it by inserting a time-stamp somewhere. I like to put it in the filename so that I rely upon the filename, not the Microsoft stamp.
For example, a filename might be
O27_995_31_53_38B_I_13_0520_1640.txt
Each group of text means something to me, and they are already shortened.
For example, the last two sets are May 20th at 4:40 pm. The YEAR and SECONDS have been removed.
(losing the seconds causes minor problems already)
The other numbers are also 2-digits instead of 4-digits for 3 of the fields, etc...
The underscores eat up valuable WAVENAME characters, but I'd like to be able to understand what is there.
I use the "autoname and go" feature for loading these text files, and my column titles are text like
filename_AAA, filename_BBB, etc...
in this case:
O27_995_31_53_38B_I_13_0520_1640_AAA, O27_995_31_53_38B_I_13_0520_1640_BBB, etc...
This is already adding up to a bunch of characters.
(Yes, I can reduce the "_AAA" to "_A", etc. It gets me 2 characters back, but it still might not be enough...)
If I add a "fit_" prefix or a "fitX_" prefix, I add 4 or 5 characters to the beginning, then I only have 27 or 28 characters to play with.
(Yes, I can rename the fit waves to "f_", and "fX_", which saves 2 more, but it still might not be enough...)
I get all sorts of errors when I run macros and the macro adds 5 characters to a 28-character long wavename. The result has the last character truncated as a wavename and then the string that I create doesn't match the name of the wave that was created.
Is there any reasonable workaround to this problem???
I know where the problem comes from - the definition of the Igor Binary file. It has space allocated for a 32 character wavename, and that's it. Changing the .itx header would cause a massive cascade of programming confusion.
Thank you in advance!!
Sean
(a dedicated user since ~1991!)
root:O27:995:31:53:38B:I:13:0520:1640
or some other arrangement that meant that you could have the waves AAA, BBB, etc in the datafolder. Of course you only need as many directories as make sense. The trouble is that you need to be able to drill down to those folders quickly, so you make have to write some code to quickly change between folders.
May 25, 2010 at 08:33 am - Permalink
For now, you can do what Andy suggests above and use data folders to provide some hierarchy so that you don't need all information about a wave to be encoded in the wave name itself. Another approach would be to store information about the wave in the wave note. If your acquisition application is writing .ibw files, you could write the wave note into the .ibw file itself. Otherwise, you could set the wave's note when you load the data in Igor. I believe that Igor can load files that have file names longer than 32 characters, but of course if you're using the auto name and go feature the name of the waves within Igor won't be the same as the original file names. However you could write your own Igor code that loads the files and sets the name to something that is somewhat descriptive and then store additional information in the wave note.
May 25, 2010 at 09:21 am - Permalink
I have used Wave Notes in the past, but as you mentioned, it also means that almost a completely-new front end has to be written in order to plot something.
In some ways, this is nice, and has been very useful in the past, but it is definitely an evolutionary sort of thing. I just got this Igor and am starting from scratch.
I really don't like the folders... Again, too much work to plot things...
I would put all of my "AAA" into one folder and my "BBB" into another.
I would then use a custom GUI to search one folder only, and then graph, fit, etc... assuming that the mating waves existed.
I've thought about this in the past, but have never implemented it.
--> I only save 2 characters this way, a lot of work for 2 chars...
I don't need to use autoname and go, but even if I were to build an auto-naming routine, it would still run out of characters.
So, I'd like to keep on hearing suggestions. Maybe someone has another trick.
My first tricks are
1) make the "AAA" and "BBB" into "A" and "B" (easy to do)
2) it requires a couple extra steps, but I can remove the extra characters in "fit_"
duplicate the wave into a dummy-named-wave,
use that one to do the fit so that "fit_" and fitX_" won't confuse anything
after the fit, duplicate the "fit_" wave into "f_", so that I won't over-run 31 characters during the operations
3) write a custom loading routine that checks the length of the wavenames, and prompts for shorter names
Any other quick ideas??
Because this desire seems common, is there any good WaveNote-sorting code out there?
Tricks like 'make each line look like "label;value" so that the listsort/search/match functions work' are welcome!
Thanks!
Sean
May 25, 2010 at 10:07 am - Permalink
I would by comparison find such excessively long file names as "too much work" measured against the ease afforded by being concise in file naming convention, verbose in my lab notebook records, and correspondingly consequential in setting up the analysis to handle what all was important. Perhaps that comes from the days of having to work against 6-8 character limits on file names.
Be that as it may ...
You might consider, when reading in the files, to collapse the generated wave names by storing the date+time string from the file name into the wave note and replacing those digits with a set of two numbers that run sequentially. That would give you 100 files to cover a given set of experiments in a given year. If you need 1000 to cover a given year, then make it a three digit replacement. You would get back six or seven characters (drop nine for the date+time set and add a two or three digit or letter consecutive sequence in place) on your wave names this way.
Then, you might consider storing your _AAA and _BBB waves into a two-column matrix, thereby avoiding the need for the such suffixes. That gives you back four characters.
Finally, you might consider which of the numbers designate something that dominates a "set" (for example, the 027 "set"), and store data from the same "set" in folders (root:set027:....). I do agree that, after about the first sub-folder level and certainly after a second sub-folder level, keeping track of where data waves are located for plotting can be rather tedious.
--
J. J. Weimer
Chemistry / Chemical & Materials Engineering, UAHuntsville
May 25, 2010 at 03:00 pm - Permalink
I don't want to sound like a 'whiner', but there is a 'pro' and a 'con' to all of these ideas/schemes. I'm just trying to get some ideas on how others have solved this problem. (I really wish Wavemetrics would update this!) I like many of the proposed ideas and will see how I can integrate them into a system that works for me and my data.
I have also been around in the days of 6-8 character filenames. (We used PDP-11's in college.) We decided that Roman numerals used fewer characters than Arabic most of the time! (We were testing semiconductor laser diodes of varying lengths and widths, and we just chose lengths at nice Roman increments!)
One thing about these long filenames is that I can make my data-collection system auto-name these files.
(referencing the list below, I can sequence some of those experiments and auto-name the data) (LabVIEW knows all of those #'s already, so the filename is basically automatic, and then it adds a timestamp.)
(I also take s-parameter data, so I can sequence a set of s-parameters and put the S11, S12, ... into the filename so that it is easy to sort later. Yes, I could put 'A', 'B', ... but that seems like too much to remember.)
Changing the datestamp to an indexing count saves characters but I'll have to get my head around how to reference the wavename to the file on the drive. The wave will have the code and the timestamp buried in the WaveNote, and the HD will have the timestamp. I'll try this one for a while. I'm easily going to be over 100 files, but probably not over a 1000 on the R&D side. The production side is handled by a database.
Actually, I have to admit that I do very little writing in my notebook these days. Most of the "story" is in the data, and the evolution that can be seen in the sequence in which it is taken.
The 2-D array idea is also potentially a nice solution. The AAA, BBB goes away for all of the waves (I've already truncated to "_A", so I'd gain 2, but 2 is good!)
The original filename was actually quite descriptive :
O27_995_31_53_38B_I_13_0520_1640.txt
OSNR of 27dB (can't easily start with a number in IGOR!!)
9.95Gb/s (other values are 10.7Gb/s and 11.3Gb/s -- yes I could use just 2 chars)
31 = 2^31-1 PRBS sequence (the other relevant value for this work would be '7')
53 = one of the styles of die that we are using (already a short form)
38B = one of the other styles (already a short form, could drop one character)
I = a coded wire-bond configuration (map is in my notebook)
13 = tested with a separate evaluation board with '13' designating the die style (could turn this into a single character)
0520 = MMDD (again, the preceeding YY has already been removed)
1640 = HHMM (again, if I try to save 2 files in the same minute, then there is a "file already exists" error)
And, I could get rid of some of the "_" characters.
But... in a week, I'll have a different set of dies, etc... and I'm sure that I'll get totally confused if I drop too many more characters.
In theory, I like the idea putting all of the "O27" data into a folder. There will be lots of other sets of data like this --> "O115" (11.5), "O11", etc... But, there will be the issue that if I use this system to strip off the "O115_", then I'll just be storing the rest of the fields in the name. Then, if I measure the same die under different conditions, I'll have all of the same fields (except the timestamp), and I'll have to be very careful about keeping some reference to the folder (in the WaveNote probably). I'll only be able to tell the difference by looking at the coded timestamp an the wavenote.
But, this set of ideas has promise, so I'll at least start sticking "stuff" into the WaveNote and start writing code to look at it.
Thank you again!!
Sean
May 26, 2010 at 04:57 am - Permalink
S
May 26, 2010 at 05:57 am - Permalink
No problem. That was not how you came across, and my response was only to show how taking a different perspective might solve the problem in place of waiting for WaveMetrics to change the wave name length limits. I can appreciate the PDP 11 reference -- exactly where I was some many years ago as well.
I see how your file naming scheme is beneficial. I would however suggest that you consider which parameters from the name are designated as the independent ones and which are the "statistical repeats". For example, are you analyzing trends versus Gb/s measured a few times in a given day, or are you analyzing trends on a daily (hourly) basis in Gb/s? In the former case, collapse out the date+time stamp from the file names into sequence numbers. In the latter, collapse out the Gb/s tag (and load different Gb/s data sets into different data folders).
Good luck, however you end up tackling this!
--
J. J. Weimer
Chemistry / Chemical & Materials Engineering, UAHuntsville
May 26, 2010 at 09:34 am - Permalink
No, as of, hm..., at least 6.0 and maybe 5? you can do fits using sub-ranges. It's just not supported in the dialog. It would look like this:
FuncFit myfitfunc, coefwave, matrixwave[][0]/X=matrixwave[][1]/D=matrixwave[][2] ...John Weeks
WaveMetrics, Inc.
support@wavemetrics.com
May 26, 2010 at 03:05 pm - Permalink
When I try the Matrix-load idea, all sorts of programming subtleties get in my way.
I'm also using the auto-generate and to-full-extent-of-xaxis options.
From the Procedure Window:
I use TRACENAMELIST("",";",1) to get all of the names of the waves on the graph.
I delete the previous fits
Then for each of the waves, I assign it's string-name to a STRING and then use that in the fit routine:
FuncFit/X=1/NTHR=0 fiterfc W_coef $first /X=XWaveRefFromTrace("", first) /W=$w_first /I=1 /D
The subtlety of the matrix approach is that the result of the tracenamelist call is not WAVE0[][3], it is WAVE0 (i.e., the whole matrix)
When I use Xwavereffromtrace. I get the matrix, not the column.
When I duplicate $first to make a weighting curve, I get a matrix, not a column.
I understand the difference and how to get around it - but it means that the routine has to check the # of dimensions of WAVE, pick which column is the correct one, and then duplicate into a "first" wave that can be used to do all of the fitting (and the same for the X-wave).
I expect that similar things happen internally when the "fit_" and "fitX_" waves are generated.
I have decided against this matrix-loading option because of these coding subtleties.
These challenges are all surmountable, but I get paid to design telecommunications modules, not write all-encompassing code.
I'll just pick another way to skin this cat.
Thank you,
Sean
May 27, 2010 at 05:13 am - Permalink