Help with Wave References/Other stuff (newbie)

Share icon

Thu, 06/17/2010 - 06:39 am

Right, so I'm new to this program but not to coding in general. The "Wave" concept seems totally alien to me and they don't seem to always work as if they should from the documentation.

So here's the current problem: I'm attempting to write functions to automize a process that takes n data files, loads them as 3-D plots, takes a slice of these plots as 2-D (which is a previously coded function that seems to be causing a huge part of my grief), runs this data through a linear convolution, and then plots a double exponential curve fit through said convolutions.

So I've pastebinned the code here for you, both the code for the original function which I did not write (but attempted to make a small modification to return a Wave variable) and the code which I did write. The original function takes the topmost image from the topmost graph in igor pro, and outputs the 2-D graph as wavename_ProfX where X is incremented if the current name is taken. I'm trying to make the program handle any possible number of waves by using a reference wave (declared with /WAVE) to hold other waves, but when used in a for loop with a counter this reference wave doesn't seem to react as I'd expect. I can't call wave functions on referenceWave[i] for example, the compiler refuses to compile.

One of the main problems is that the previously created function WMCreateImageLineProfileGraph() takes no parameters and returns no variables, so it's a lot of guesswork to hope that it's working on the correct graph when the program runs. However the code is long and convoluted and I can't see an easy fix. It pops up a GUI window that lets you pick several parameters. Before making the 2-D plot, and the output is generally wavename_Prof1.

Generally I'm trying to automate this process as much as possible, so a pause at this point to play with the settings should be fine. I'm more concerned about how I can use a reference wave in a for loop to call various functions on the waves it contains. I'm used to arrays being variable independent, that is, you don't need to know variable names within the array because the variables aren't actually IN the array, but it seems that a reference wave containers implicit pointers to the old waves rather than the waves themselves.

Honestly I'm very close to just attempting to code the entire thing using strings to keep track of incremental variable names, which while it goes against everything I've ever learned coding, I find this incredibly frustrating to work with.

Any thoughts would be greatly helpful.

My code:
http://www.pastebin.ca/1885051

WMCreateLineProfileGraph()
http://www.pastebin.ca/1885049
caaok wrote:
I can't call wave functions on referenceWave[i] for example, the compiler refuses to compile.

You're getting confused by WAVE reference variables and wave names. You are not the first and will likely not be the last :)

This is something that trips up a lot of users. I recommend that you do a little reading on this by executing the following commands in Igor. They'll bring up the parts of the help relevant to the topic.
DisplayHelpTopic "Accessing Waves In Functions"
DisplayHelpTopic "Converting a String into a Reference Using $"
DisplayHelpTopic "Accessing Global Variables And Waves"
DisplayHelpTopic "WAVE"


I was able to get your code to compile by making the following two changes. Hopefully you can figure out what lines you need to change.
AppendImage $(GetWavesDataFolder(RefWave[i], 2))
Duplicate/O RefProfWave[i],$(GetWavesDataFolder(RefConvWave[i], 2))


On a side note, you can delete all the calls to DelayUpdate if you want. That has no effect in functions. Igor adds that line when you do things using the dialogs because in that case the commands are interpreted, but when you copy from the command window into a function you can get rid of the DelayUpdate calls so that your code is a little easier to read.

caaok wrote:
One of the main problems is that the previously created function WMCreateImageLineProfileGraph() takes no parameters and returns no variables, so it's a lot of guesswork to hope that it's working on the correct graph when the program runs.

The WMCreateImageLineProfileGraph() function is designed to provide some GUI user interface to the built in ImageLineProfile operation. If you're trying to do an automated analysis of your data, you might be better off just using the ImageLineProfile operation directly.

I recommend that, if you're using the latest beta version of Igor 6.2, you change #pragma rtGlobals=2 at the top of the file to #pragma rtGlobals=3. This will enforce stricter checking when compiling, which will mean that your code as written (even with my two changes above) won't compile. There are places in your code where you use wave names. This is OK as long as those waves actually exist when the code runs. Forcing yourself to declare WAVE references to those waves results in cleaner code that is less likely to break.

Write back if you still run into problems.

June 17, 2010 at 10:49 am - Permalink

Right so I've tried to rewrite the thing using ImageLineProfile instead. The problem is that I'm still having issues with wave references v.s. wave names I think. Why can't I call RefWave[i] = someWave in a for loop... that makes perfect sense to me. I've rewritten the main function, perhaps you could take a look and see where I'm still going wrong.

As for the documentation, it's quite sparse for the wave reference waves, and the examples they give don't seem too relevant to what I'm doing.

http://www.pastebin.ca/1888018

Changes:

So instead of WMCreateImageLineProfileGraph, I'm now using ImageLineProfile. The parameters being passed reflect the fact that it should be using a Width of 200 with a midpoint of roughly 789 for the data I'm processing, i.e. should contain the points from 689-889, using a horizontal profile. I'm not sure if I'm doing that in the correct fashion.

Ideally, I want to:

Read n waves, put them into RefWave from i=0 to n-1, display each wave on a separate graph.
Call ImageLineProfile on each of those graphs, putting the resultant waves into RefProfWave, with the parameters I specified earlier. Display the profiles on a single plot.
Convolve those waves into RefConvWave, plot them on a single plot.
Plot a double exponential fit through the convolved waves, plotting those fits on the convolved plot.

And ideally have some sort of colour change for each wave.

June 21, 2010 at 01:48 pm - Permalink


Why can't I call RefWave[i] = someWave in a for loop


This should work. Your code compiles for me, once I comment out the functions I don't have (GetInputWaves and GetCanc).

I recommend you change this:
$(GetWavesDataFolder(RefWave[i], 2))


to this

RefWave[i]


in the two places it occurs.

I can't test further because I don't have your GetInputWaves function so refWave is uninitialized.

If you still have problems, it is best to boil the question down to a simple function and include the function right here in the IgorExchange forum window, using <igor> and </igor> to bracket the code. You can also attach a file right here in IgorExchange.

June 21, 2010 at 02:57 pm - Permalink

Hm...I'm not sure why I thought you needed to change the AppendImage line originally. I must have goofed with that line. However the Duplicate line still doesn't compile for me as you originally wrote it (in your original post).

I suspect that your problem is now in your GetInputWaves() function. I assume you haven't changed that from the first post. If so, I'd recommend you change it to this:
Function GetInputWaves(rWave1)
    // This function pops up an "Open File" dialog for multiple waves.
    // At this point this function is built to handle as many input files as you want.
    // The program, however, is built to deal with two waves and an IRF wave, so don't exceed that.
    // In the context of this program it's used to select the two actual data files.

    // Declares some variables.
    Wave/WAVE rWave1
    Variable refNum
    String message = "Select two electron scatter data files:"
    String outputPaths
    String fileFilters = "Data Files (*.txt,*.dat,*.csv):.txt,.dat,.csv;"
    fileFilters += "All Files:.*;"
    // Here's the dialog box again. The filename output is a delimted list of the chosen files.
    Open /D /R /MULT=1 /F=fileFilters /M=message refNum
    outputPaths = S_fileName
 
    // Checks to see if it was cancelled, otherwise performs a LoadWave function per file.
    // The output will be wave0, wave1, wave2, ..., wave(n-1).
    // In this case the output waves will be wave0 and wave1.
    if (strlen(outputPaths) == 0)
        Print "Cancelled"
        return 0
    else
        Variable numFilesSelected = ItemsInList(outputPaths, "\r")
        Variable i
        for(i=0; i<numFilesSelected; i+=1)
            String path = StringFromList(i, outputPaths, "\r")
            Printf "%d: %s\r", i, path
            LoadWave/G/M/D/N=fileWave path
           
            // Check that a wave was successfully loaded.
            // <----
            if (V_flag == 1)
                WAVE loadedWave = $(StringFromList(0, S_waveNames, ";"))
                rWave1[i]=loadedWave
            else
                // You might report an error here
            endif
            // <----
        endfor
    endif
 
    // Returns the number of wave files for possible use in expanding the program.
    // Could be used to make the program work with 3, 4, 5 different waves, etc.
    return numFilesSelected
End


Look for the // <--- marks to see what I changed. I think your problem was that you were expecting the wave name to always be fileWave0, but that isn't the case because the /N=baseName flag will cause waves to be named fileWave0, fileWave1, fileWave2, etc.

June 21, 2010 at 05:18 pm - Permalink

It's not that the code refuses to compile now, it's more that the code seems to call waves filled with null references. I think there's a problem in the main function with filling the matrix with W_ImageLineProfile, or perhaps the call to ImageLineProfile(). The graphs load fine but the resultant profile waves are just filled with zeroes (as are, naturally, the convolution waves). I've attached my prospective data files in a .zip.

Pastebin of the current code:

http://www.pastebin.ca/1888727
data.zip

June 22, 2010 at 12:35 pm - Permalink

I was about to look into this again, but right now pastebin.ca seems to be down, so I can't see your code. I'm with Howard that it's best if you post it here within igor tags so that it gets the right syntax coloring.

My guess is that one of your problems is that, if I recall correctly, at one point you were doing something like:

Make/WAVE waveOfWaveReferences

// and then later
For(....)
     ImageLineProfile ...arguments...
     waveOfWaveReferences[n] = W_ImageLineProfile
EndFor


This probably doesn't do what you think it does. What should actually happen here is that after the loop completes, waveOfWaveReferences will contain n rows, each of which contains a wave reference to the exact same wave (and therefore the exact same data).

You said you're familiar with programming, so think of a wave reference as being similar to a pointer. You've filled up a wave with pointers that all point to the same location in memory.

What you probably want instead is to do something like this:

Make/WAVE waveOfWaveReferences.
String newWaveName

// and then later
For(....)
     ImageLineProfile ...arguments...
     newWaveName = "newwave" + num2istr(n)
     Duplicate/O W_ImageLineProfile, $(newWaveName)
     WAVE newWaveRef = $(newWaveName)
     waveOfWaveReferences[n] = newWaveRef
EndFor


There may also be other problems with your code, but until I can get the code again I can't execute it locally to help you troubleshoot more.

June 23, 2010 at 08:55 am - Permalink

Ok, I've sorted out the code to the point where it now seems to run fine. The main problem now is that the Line Profiles aren't created correctly, mostly due to the fact that the WMCreateImageLineProfileGraph() function has a lot going on, and even searching through it, I'm finding it hard to figure out how to create the x and y waves to be passed correctly. As such, I've attached the current code (no worries about pastebin) and I've uploaded a save file to Rapidshare of what the final product *should* look like, perhaps that will help you visualize the problem. The input files are those that were included in the .zip in the earlier post.

http://rapidshare.com/files/403571599/2ns_ZnO_RmT_Zn_Edge.pxp.html
IgorPro Functions2.txt

June 28, 2010 at 08:46 am - Permalink

Based on your last comment it's not at all clear to me what your question is. I'm not in the same field as you, so saying that the profiles aren't created correctly doesn't give me enough information to solve your problem.

I think it would be easier if you sent your question to us at support@wavemetrics.com. Send us an Igor experiment with the code and any external data files necessary for us to reproduce your problem. You can delete any large waves from the experiment that aren't necessary for us to have to reduce the size. Just zip everything together and sent that as an attachment if your email server will allow it. Otherwise we can give you information for you to upload it to our FTP server.

Please include step by step instructions for using your code (eg. run this command, choose this file, enter this value, etc.) and provide a clear description of what result you expect and what result you are getting, including any error messages Igor is giving you. If you have narrowed down the problem to a certain line in your code or a certain function please also mention that.

By the way, if you haven't already, you might try using the Igor debugger to step through your code. For more information, execute:
DisplayHelpTopic "The Debugger"

June 28, 2010 at 09:35 am - Permalink

I'm sorry, let me try to clarify: The code now compiles and when executed, will actually reach CurveFit right at the end of the main function. However, CurveFit throws the error that there are no Y values on the plots to be fitted. When looking at the graph output generated, I see the following:

My initial waves display fine, one per graph.

My profile waves do not display, leaving a blank graph with a legend appended to it. Afterwards of course, the convolutions of those profiles aren't displayed since the initial functions are treated as zeroes, so the graph of convolutions is also blank.

So I guess the only thing I still need help with is the ImageLineProfile function. Currently I'm feeding it the initial waves, each of which are a three dimensional scatter plot.

I'm looking to take a horizontal profile with a width of 200, roughly from the Y axis points 689-889 on the initial plots (so a midpoint of ~789). As I understand, ImageLineProfile should output the result in W_ImageLineProfile which I then want to save into a reference wave for further use (but I think that part's fixed now with your last suggestion). If you look at the .pxp I posted (I know it's rather large), you'll see a visual representation of each graph since the WMCreateImageLineProfileGraph() function actually draws two lines through the initial plots so you can better see which portion is being analyzed. You can also see exactly what was done in the command line I'm trying to emulate. To save you the trouble of grabbing that huge file, here's the image output:

The initial 3-D scatter plot (there will be "n" of these) (this displays correctly for each file when I run my code):
http://img.photobucket.com/albums/v300/ca_aok/IgorPro%20Pics/InitialPlo…

WMCreateImageLineProfileGraph() in action (working on the graph shown above):
http://img.photobucket.com/albums/v300/ca_aok/IgorPro%20Pics/WMProfile…

The profile created by the above (there will be "n" of these) (I skipped this in my file and appended them all to a single graph):
http://img.photobucket.com/albums/v300/ca_aok/IgorPro%20Pics/Profile.jpg

The profiles appended to one graph, the green plot is the single inputted IRF reference file (this is blank shown with a legend in my file):
http://img.photobucket.com/albums/v300/ca_aok/IgorPro%20Pics/Profile2.j…

The final graph, convolutions with double exponential curve fits (this is blank with a legend in my file):
http://img.photobucket.com/albums/v300/ca_aok/IgorPro%20Pics/Conv.jpg

June 28, 2010 at 11:19 am - Permalink

As I said above, if I can't reproduce your problem, I can't help you. And I can't reproduce your problem, because you aren't telling me what you are doing. Please provide a step by step list of instructions that I can use to reproduce your problem. Tell me what functions you are calling from the command line, what values you are inputting into the prompts your code creates, and what file(s) you are selecting at each step. I get roughly the same results you described when I execute RunThisProgram(), but since I have no idea what steps you do after that, I can't help you.

June 28, 2010 at 01:59 pm - Permalink

The Steps:

RunThisProgram()

A file dialog box appears, select all regular files with CTRL+click (in the case of the zipped file these are ZnO_NN_14.dat and ZnO_NN_15.dat). n = 2

Another open dialog appears, this time select the IRF data file (ZnO_NN_IRF_15.dat). This is the standard against which the other waves are convolved.

A dialog box appears prompting for eV values. These are only relevant for the legends of the graphs. I'm thinking about changing this part later but I'd rather worry about making the graphs work first rather than worrying about their legends. In this case the values are 1090 and 1000, respectively (it'll appear twice).

A box appears prompting for truncation values. This truncates the linear profile of ZnO_NN_IRF_15.dat, which reduces or eliminates the noise in the far ranges of the convolution graphs. Again, this can be tweaked later, it's likely I'd simply have the user call the truncation function by hand. I'm using the values 224 and 800 here.

The eV prompts appear again, same values. Again, not too fussed about this part but it's for the legends on the Convolution plot.

What I want to appear:

-Three (n+1) electron scatter plots, which currently show up fine. One for each .dat file.

-A single plot of linear profiles, with (n+1) profiles. The IRF profile only really matches the initial gaussian part of the profile curves for the real data, but it's still useful to see the correlation. Some of the graphs need multiplier offsets for comparison purposes, but that can be done by hand or added later.

-A single plot of convolved waves, with (n) convolutions. Then (n) double exponential curve fits should be added to this plot.

Currently the profile and convolution plots appear blank. Here's the command line of the original file if you want to see what I'm attempting to emulate. I put in some comments in bold when I was originally working through the file.
Wave Process Documentation.doc

June 29, 2010 at 06:19 am - Permalink

Ok..I think I finally see your problem. If I'm right, you should be getting a run time out of memory error, which I don't see you having mentioned before. So maybe you are having a different problem.

When I do what you said above, the call to ImageLineProfile gives a run time out of memory error. This makes sense because the x and y waves you are giving it contain bogus values:
•print lineprofilex LineProfileX[0]= {-inf,inf,NaN,-inf,inf} •print lineprofiley LineProfileY[0]= {889,889,NaN,689,689}

I suspect you are doing that because of this bit of code in WMUpdatePositionAndWidth() in Image Line Profile.ipf:
            LineProfileX= {position+w2,position+w2,NaN,position-w2,position-w2}
            LineProfileY={-INF,INF,NaN,-INF,INF}


However, if you use the Igor debugger, and set a breakpoint before ImageLineProfile is actually called in the WMDoLineProfile() function, and then inspect the contents of the xWave and yWave waves that are used as parameters to ImageLineProfile, you'll see that they look very different:
xwave[0]= {0,1024} ywave[0]= {788,788}

I can understand how you might have gotten confused by reading the Image Line Profile.ipf file. It was written a while back and has been modified many times by several different people, so it's a bit um....convoluted.

You can find my changes in the attached procedure file by searching for "ACL". I also fixed the code that was creating your legends incorrectly. This seems to solve your problems with creating the Profiles graph. Your Convolutions graph still looks messed up, but I think that's because your call to Convolve looks like this:
Convolve waveIRF0_Prof1, newConvRef

waveIRF0_Prof1 contains 225 points, while newConvRef contains 1249 points. I suspect this mismatch between points is giving unexpected results. You'll have to figure out how you want to fix that, assuming that this is the actual problem.
caaok.ipf

June 29, 2010 at 10:39 am - Permalink

Using the data browser I figured out that the problem with the Convolve function was the presence of a blank value field at the end of each profile wave. A simple call to DeletePoints fixed that up. Everything seems to be working now, thanks again for all your help :)

July 7, 2010 at 06:54 am - Permalink