Customized data import

Share icon

Tue, 09/23/2014 - 02:05 am

I'm pretty good with the basic commands and have automated data importation before, but not when the format is complex. Any tips on how to go about the following? If I use the load data dialogue (ctrl-L), and then choose general text, I can get the blocks loaded easily enough with the load columns into matrix option, but the "chi = xxxxx" is lost, althought the "z=xxx" is easy enough to extract from the base names.

Would add that it is really unfortunate that choosing 'data/load waves/load general text' does not include the 2D option (?!). Also some ability to tell Igor where to start numbering sequential files would be nice, I ended up writing a little script to rename everything (z_0 -> z_1) so that the file numbers matched the z. I imagine there is probably an easier way tho?

Anyhow, here is what the .dat file looks like, the exact format is not important regarding how the waves are saved, just need to see how to extract all the data in any fashion, if its possible. Thanks.

Z= 1, chisq= 0.170190
3.55221981e-002 2.25354459e-001 2.62782423e-002 2.25354636e-001
3.52695173e-002 2.25355749e-001 6.77755867e-002 4.38850114e+000
3.56601775e-003 4.03881150e-001 2.76131055e-002 1.44488619e+000

Z= 2, chisq= 0.013113
3.73696013e-002 5.36479656e-001 3.60228922e-002 5.36480729e-001
1.22544155e-001 5.36493547e-001 5.06400486e-003 2.70930915e-001
2.03620941e-002 8.51962305e-001 2.74236552e-002 2.33542971e+000

Z= 3, chisq= 0.258875
1.07960369e-001 9.23258728e-001 1.06777482e-001 9.23233431e-001
7.71392003e-002 4.74123946e-002 2.75244208e-003 1.69761557e-001
1.08034077e+000 1.90675572e+001 3.46769903e-001 6.28398894e+000

Oh and I have a problem with rounding as well, 3.56601775e-003 becomes 0.00356602 (i clicked double precision), is it possible not to round here?
I would write a function to open the file, read one line at a time, and process the data accordingly.

Here is a quick and dirty piece of code that you can start with:
Function LoadMatrixBlockData()
    // loads into waves in current datafolder
    variable vRefNum
    string sLine, sHeaderLine,sDataName
    variable vColumns,vRow
    Open/R vRefNum
    do
        FReadLine vRefNum,sLine
        if(strLen(sLine)==0)
            break
        endif
        // look for Z= at start of line
        if(cmpStr(sLine[0,1],"Z=")==0)
            sHeaderLine=sLine
            sDataName=stringFromList(0,sLine,",")
            FReadLine vRefNum,sLine
            vColumns=ItemsInList(sLine," ")
            Make/D/O/N=(1,vColumns) $sDataName/wave=wData
            note/K wData,sHeaderLine // store Z and chisq in wave note
            vRow=0
            wData[vRow][]=str2num(stringFromList(q,sLine," "))
            do
                FReadLine vRefNum,sLine
                if(strLen(sLine)==0)
                    break
                endif
                if(cmpStr(sLine,"\r")==0)
                    break
                endif
                vRow+=1
                ReDimension/N=(vRow+1,-1) wData
                wData[vRow][]=str2num(stringFromList(q,sLine," "))
            while(1)
        endif
    while(1)
    Close vRefNum
End

Hope this helps,
Kurt

September 23, 2014 at 03:14 am - Permalink

Thank you Kurt, that is exactly what I am looking for. Will figure it out from here =)

September 23, 2014 at 03:20 am - Permalink

Quote:
Oh and I have a problem with rounding as well, 3.56601775e-003 becomes 0.00356602 (i clicked double precision), is it possible not to round here?

That's just the display in the table. Right-click on the column and select Digits-> [something bigger than 6]

John Weeks
WaveMetrics, Inc.
support@wavemetrics.com

September 23, 2014 at 10:34 am - Permalink