Customized data import

Share icon

I'm pretty good with the basic commands and have automated data importation before, but not when the format is complex. Any tips on how to go about the following? If I use the load data dialogue (ctrl-L), and then choose general text, I can get the blocks loaded easily enough with the load columns into matrix option, but the "chi = xxxxx" is lost, althought the "z=xxx" is easy enough to extract from the base names.

Would add that it is really unfortunate that choosing 'data/load waves/load general text' does not include the 2D option (?!). Also some ability to tell Igor where to start numbering sequential files would be nice, I ended up writing a little script to rename everything (z_0 -> z_1) so that the file numbers matched the z. I imagine there is probably an easier way tho?

Anyhow, here is what the .dat file looks like, the exact format is not important regarding how the waves are saved, just need to see how to extract all the data in any fashion, if its possible. Thanks.

Z= 1, chisq= 0.170190
3.55221981e-002 2.25354459e-001 2.62782423e-002 2.25354636e-001
3.52695173e-002 2.25355749e-001 6.77755867e-002 4.38850114e+000
3.56601775e-003 4.03881150e-001 2.76131055e-002 1.44488619e+000

Z= 2, chisq= 0.013113
3.73696013e-002 5.36479656e-001 3.60228922e-002 5.36480729e-001
1.22544155e-001 5.36493547e-001 5.06400486e-003 2.70930915e-001
2.03620941e-002 8.51962305e-001 2.74236552e-002 2.33542971e+000

Z= 3, chisq= 0.258875
1.07960369e-001 9.23258728e-001 1.06777482e-001 9.23233431e-001
7.71392003e-002 4.74123946e-002 2.75244208e-003 1.69761557e-001
1.08034077e+000 1.90675572e+001 3.46769903e-001 6.28398894e+000

Oh and I have a problem with rounding as well, 3.56601775e-003 becomes 0.00356602 (i clicked double precision), is it possible not to round here?
I would write a function to open the file, read one line at a time, and process the data accordingly.

Here is a quick and dirty piece of code that you can start with:

Function LoadMatrixBlockData()
	// loads into waves in current datafolder
	variable vRefNum
	string sLine, sHeaderLine,sDataName
	variable vColumns,vRow
	Open/R vRefNum
	do
		FReadLine vRefNum,sLine
		if(strLen(sLine)==0)
			break
		endif
		// look for Z= at start of line
		if(cmpStr(sLine[0,1],"Z=")==0)
			sHeaderLine=sLine
			sDataName=stringFromList(0,sLine,",")
			FReadLine vRefNum,sLine
			vColumns=ItemsInList(sLine," ")
			Make/D/O/N=(1,vColumns) $sDataName/wave=wData
			note/K wData,sHeaderLine // store Z and chisq in wave note
			vRow=0
			wData[vRow][]=str2num(stringFromList(q,sLine," "))
			do
				FReadLine vRefNum,sLine
				if(strLen(sLine)==0)
					break
				endif
				if(cmpStr(sLine,"\r")==0)
					break
				endif
				vRow+=1
				ReDimension/N=(vRow+1,-1) wData
				wData[vRow][]=str2num(stringFromList(q,sLine," "))
			while(1)
		endif
	while(1)
	Close vRefNum
End

Hope this helps,
Kurt

September 23, 2014 at 03:14 am - Permalink
Thank you Kurt, that is exactly what I am looking for. Will figure it out from here =)

September 23, 2014 at 03:20 am - Permalink
Oh and I have a problem with rounding as well, 3.56601775e-003 becomes 0.00356602 (i clicked double precision), is it possible not to round here?

That's just the display in the table. Right-click on the column and select Digits-> [something bigger than 6]

John Weeks
WaveMetrics, Inc.
support@wavemetrics.com

September 23, 2014 at 10:34 am - Permalink