LoadWave error - how to determine number of lines in a file before opening?

Share icon

I'll paste in the problem code and then explain the problem:


	if(WaveExists(SPP200Paths))
		sort /A SPP200Paths, SPP200Paths
		tmpWaveList = ""
		for(i =0;i<dimsize(SPP200Paths,0);i+=1)
			wfprintf currentFile, "%s" /R=(i,i) SPP200Paths
			LoadWave/J/M/U={0,0,1,0}/D/A=SPP200tmp/E=0/K=0/V={"\t,"," $",0,1}/L={19,20,0,0,0}/Q currentFile
			tmpWaveList += "SPP200tmp" + num2str(i) + ";"
		endfor
			Concatenate /NP=0 /KILL tmpWaveList, SPP200
			SetScale /P y, 0,1,SPP200
			ModifyControl CAPS_Analyse_CAS disable=0
//	columnLabel = "SPP200_Ch1 [counts]"
//	display SPP200[][(FindDimLabel(SPP200,1,columnLabel))]		
	endif


The file paths contained in the text wave SPP200Paths can refer to files which contain header data, but nothing else. I don't want to read the header data in this part of the function, - that's taken care of elsewhere. When the loop hits a file containing nothing but header information, I get the error "LoadWave error: "no data was found in file"". If I try to get rid of this warning using the /C flag to LoadWave, the Concatenate command generates errors (Concatenate error: "expected wave name").

So at last to the question - is there an elegant way to determine the number of rows in these delimited text files before calling LoadWave? If I could put some sort of IF statement in the loop it'd fix the problem i.e.


		if (LinesInCurrentFile > 20)
		        Loadwave ...
		endif


But how would one go about determining LinesInCurrentFile?
First, try calling GetRTError(1) after calling LoadWave to see if there was an error. If so, you can assume the file contained no data.

If that is not good enough, you will have to parse the file yourself. Here is an example of a function that looks for a line that starts with two numbers:



static Function FindFirstDataLine(refNum)
	Variable refNum
	
	Variable lineNumber = 0
	do
		String text
		FReadLine refNum, text
		if (strlen(text) == 0)
			// No more lines
			return -1			// Signifies data not found
		endif
		
		// This looks for a line with blank space followed by a float followed by blank space followed by a float.
		String regExp = "^[[:blank:]]*[[:digit:]]+\\.[[:digit:]]+[[:blank:]]*[[:digit:]]+\\.[[:digit:]]+"
		if (GrepString(text, regExp) != 0)
			break				// Found it
		endif
			
		lineNumber += 1
	while(1)
	
	return lineNumber
End


This is called like this:


	Open/R/P=$pathName refNum as fileName
	firstDataLine = FindFirstDataLine(refNum)	
	Close refNum

October 5, 2010 at 08:16 am - Permalink
Thanks for the comments. I wrote some quick code which seems to suggest that if I use the /C flag to LoadWave then GetRTError doesn't pick up a runtime error. I'll have to look into either parsing the headers before reading, or checking the length of the wave returned by LoadWave tomorrow.

October 5, 2010 at 09:46 am - Permalink
if I use the /C flag to LoadWave then GetRTError doesn't pick up a runtime error


You should not use /C. It is intended for experiment recreation procedures only.

Omit /C and use GetRTError.

October 5, 2010 at 12:59 pm - Permalink
hrodstein wrote:
You should not use /C. It is intended for experiment recreation procedures only.
Omit /C and use GetRTError.

Thanks for your help. I got so annoyed by the debugger popping up during expected occurrences (empty files) that I went with your other suggestion.


Static Function FindFirstDataLine(currentFile)
	String currentFile
	Variable refNum
	Variable lineNumber = 0
	String text = ""
	Open/R refNum as currentFile
	do
		FReadLine refNum, text
		FStatus refNum
		if (!GrepString(text, "[(?!)a-z*][^NaN]") && strlen(text) > 3)
			break
		endif
		if (V_filePos == V_logEOF)
			return -1
		endif
		lineNumber += 1
	while(1)
	Close refNum
	return lineNumber
End

October 7, 2010 at 08:04 am - Permalink
This may not work in your situation, but if the files that contain only header data are quite small relative to files that contain actual data, you could use 
GetFileFolderInfo
and check the output variable V_logEOF to see if it's above some threshold.

October 8, 2010 at 06:43 am - Permalink