LoadWave Cut data into multiple columns.

Share icon

Thu, 08/20/2015 - 03:39 am

Hi Igor!

I have a txt file format that has three columns: Spectrum Number, xdata, ydata and after the data from spectrum1 has finished the next row goes straight into the spectrum2. (I have attached a small part of the text file to illustrate my point, only 2 spectra).

The data I acquire can have as few as 10 consecutive spectra to 120.

I ideally want to load a 2D matrix of waves that such that column0=spectrum1(S1); column1=spectrum2(s2)....columnN(SN) that are interpolated with xdata and keep one column of xdata for reference.

I read the "FReadLine" help topic and I am not entirely sure how to execute it. As using the /T flag the terminator number could be included in previous data and the /N flag the number of characters to read varies depending on data set. I am a terribly bad programmer but learning!

Function LoadAndGraphSpectrum(fileName)
    String fileName // Name of file to load or "" to get dialog
    String pathName // Name of path or "" to get dialog
    String columnInfoStr
   
    // Load the waves and set the local variables.
    LoadWave/A/J/D/O fileName
    if (V_flag==0) // No waves loaded. Perhaps user canceled.
        return -1
    endif
            // Put the names of the three waves into string variables

    String s0, s1, s2
    s0 = StringFromList(0, S_waveNames)
    s1 = StringFromList(1, S_waveNames)
    s2 = StringFromList(2,S_waveNames)

       
    Wave identifier = $s0 // Create wave references.
    Wave wavenumber = $s1
    Wave counts=$s2
   
    Interpolate2   wavenumber, counts
   
    Display counts  // Create a new graph
    // Annotate graph
    Textbox/N=TBFileName/A=LT "Waves loaded from " + S_fileName
    return 0 // Signifies success.
   
End


I have a multiple load function that uses this as its core function...clearly this is very crude and I am at a loss as to how to proceed.

I really appreciate the help i receive on the Igor exchange and look forward to the day I may be able to return the favour!!

Thanks in advance,
Nicola

PS Edited my script to be more specific as for this type of data I will never have more than 3 columns still chopping the waves up is hard and if possible it would be nice once cut to name each column S1, S2 etc Rennie2SpectraTestdata.txt
Hi Nicola,

You could try adding something like
    Variable nSpec=WaveMax(identifier)  //find maximum spectrum
       
    Variable i
       
    For(i=0; i<(nSpec+1); i+=1)
        newName1 = "wn_" + num2str(i)
        newName2 = "c_" + num2str(i)
       
        Duplicate/O wavenumber $newName1
        Wave w2=$newName1
        w2 = (identifier==i) ? wavenumber : NaN
        WaveTransform zapnans w2
        Duplicate/O counts $newName2
        Wave w3=$newName2
        w3 = (identifier==i) ? counts : NaN
        WaveTransform zapnans w3
        If(numpnts(w2)==0)
            Killwaves w2
            Killwaves w3
        Else
        //do what you like here
        Endif
    EndFor


This solution works by filtering your wavenumber and counts waves by the identifier status. It makes a copy of them named according to the identifier. Apologies, it's not well-written but it should help. Your Interpolate2 stuff would go after Else.

August 20, 2015 at 06:04 am - Permalink

Thanks sjr51,

I think your way might be a bit more efficient and I have been reading the helptopic on wavetransform to try and better understand what is going on.

I have been working all morning on the code I had initially started and transformed it into this

Function LoadAndGraphSpectrum(fileName)
    String fileName // Name of file to load or "" to get dialog
    String newDF
   
       
    // Load the waves and set the local variables.
    LoadWave/A/J/D/O fileName
    if (V_flag==0) // No waves loaded. Perhaps user canceled.
        return -1
    endif
            // Put the names of the three waves into string variables

    String s0, s1, s2
    s0 = StringFromList(0, S_waveNames)
    s1 = StringFromList(1, S_waveNames)
    s2 = StringFromList(2,S_waveNames)

    Wave identifier = $s0 // Create wave references.
    Wave wavenumber = $s1
    Wave counts=$s2
   
    Variable result=Counting(identifier,0)
    Variable numPoints = numpnts(counts)    
    Variable numSpectra = numPoints/result  //number of individual spectra in data set
    make/D/N=(result) Sp
   
    Variable i
    Variable j
    string newName1
    for (i=0;i<numPoints;i+=result)
        for(j=0;j<numSpectra;j+=1)
            newName1 = "c_" + num2str(j)
            Duplicate/O/R=[0+i,result+i] Sp, $newName1
        endfor
    endfor
   
    Duplicate/O/R=[0,result] wavenumber, wnum

    //Interpolate2 wavenumber, counts
End


so the i loop goes through every point and the j loop goes through the number of spectra in data, i think I will adopt the wavemax function though.

I have a problem with the duplicate in the loop. I seem to create waves with increasing names but I can't seem to extract data to insert in the wave.

Any ideas?

Thanks for your help!

Best,
N

Edit: I can't use the wavemax because the identifier doesn't go up in 1,2,3etc which is annoying!

August 20, 2015 at 09:40 am - Permalink

I think I misunderstood a few things...
1) have you managed to load the three waves (I assumed you had looking at your OP, I'm now not sure)?
2) are the numbers in identifier column all integers? If they are not monotonic it won't matter for the code I wrote.

Edit: in fact I definitely misunderstood, you want a 2D matrix not a bunch of individual waves. Let me know about the above and I'll try to help more.

August 20, 2015 at 11:37 am - Permalink

Hi sjr51,

I'm sorry that I didn't explain myself well. I really appreciate your help though!! =)

1) yes I can load my three original waves (which are there really for a reality check)

I decided to essentially load all the waves separately and concatenate at the end...which i'm sure is not the most efficient way to do this...I finally came up with this code(after cornering a software engineer who had never seen Igor before for help):

Function LoadAndGraphSpectrum(fileName)
    String fileName // Name of file to load or "" to get dialog
//  String newDF
   
       
    // Load the waves and set the local variables.
    LoadWave/A/J/D/O fileName
    if (V_flag==0) // No waves loaded. Perhaps user canceled.
        return -1
    endif
            // Put the names of the three waves into string variables

    String s0, s1, s2
    s0 = StringFromList(0, S_waveNames)
    s1 = StringFromList(1, S_waveNames)
    s2 = StringFromList(2, S_waveNames)

    Wave identifier = $s0 // Create wave references.
    Wave wavenumber = $s1
    Wave counts=$s2
   
    Variable result=Counting(identifier,0)
    Variable numPoints = numpnts(counts)    
   
    Variable i
    String newName1
    Wave bawbag
    for (i=0;i<numPoints;i+=result)

        newName1 = "c_" + num2str(i)
   
        Duplicate/O/R=[0+i,result-1+i] counts, $newName1
    endfor
   
    string waves = wavelist ("c_*", ";", "")   
    concatenate waves, '2Dbawbag'
    Duplicate/O/R=[0,result-1] wavenumber, wnum

    //Interpolate2 wavenumber, counts
End


I ideally would like use this as a base function to pass to for a multiload and pass each file in a folder to its own unique datafolder.

This structure is good for my file type but maybe not great for adapting for someone else...but hopefully it can give someone a starting point if wanting to do something similar.

If anyone can add comments for improvements it would be much appreciated.

Best wishes,
N

August 20, 2015 at 02:28 pm - Permalink