Load several data files

Function LoadSelectedNephFiles() Variable refNum String message = "Select one or more .dat files" String outputPaths String fileFilters = "Data Files (*.dat):.dat;" SVAR workdir = root:workdir fileFilters += "All Files:.*;" Open /D /R /MULT=1 /F=fileFilters /M=message refNum outputPaths = S_fileName if (strlen(outputPaths) == 0) Print "Cancelled" return -1 endif Variable numFilesSelected = ItemsInList(outputPaths, "\r") NVAR numberofAnalFiles = root:numberofAnalFiles numberofAnalFiles = numFilesSelected Variable i for(i=0; i<numFilesSelected; i+=1) String path = StringFromList(i, outputPaths, "\r") // Printf "%d: %s\r", i, path // For debugging only Variable err = LoadNephFile("", path) // It is OK to pass "" for pathName because we are passing a full path for fileName if (err != 0) return err endif endfor return 0 // Success End

hrodstein

How should I deal with the first block and put that in a loop?

I think you mean "How can I loop through all of the files in a folder". Use IndexedFile. For and example, see http://www.igorexchange.com/node/5843

Can I use LoadWave function?

LoadWave can only load a regular block of numbers. You can tell LoadWave to load a block of lines of text using the /L flag. So you can tell LoadWave to load lines 1, 2 and 3 (zero-based). You can then parcel the data into your 3D output wave and repeat for the next block of RGB lines. To do this you need to be able to calculate the line numbers in each RGB block. If the files are regular, that is doable. However you also need to know when to stop. For this you need to know the total number of lines in the file, which can note be determined except by examining the file line-by-line.

Because of this last issue, I think I would dispense with LoadWave and use FReadLine instead. There is an example at http://www.igorexchange.com/node/880. You would then use sscanf to parse a given line and store the values in the appropriate place in the output wave.

March 31, 2015 at 10:16 am - Permalink

tooprock

Thanks for the comment. I think I did exactly what you suggested. This function can extract the Date/Time for each parcel. I am now thinking of applying the same for RGB data.

 Function LoadNephFile(pathName, fileName)
	String pathName				// Igor symbolic path name or "" for dialog.
	String fileName				// file name, partial path and file name, or full path or "" for dialog.
	
	Variable err

	// This puts up a dialog if the pathName and fileName do not specify a file.
	String message = "Select a data file"
	Variable refNum
	String fileFilters = "Data Files (*.dat):.dat;"
	Open /R /Z=2 /P=$pathName /F=fileFilters /M=message refNum as fileName
	
	// Save outputs from Open in a safe place.
	err = V_Flag
	String fullPath = S_fileName

	SVAR workdir = root:workdir
	workdir = S_fileName
	if (err != 0)
		return err			// -1 means user canceled.
	endif
	
	String fileNameFromFile		// File name as read from the first line of the file
	
	// Create data folder
	String dfName = ParseFilePath(3, fullPath, ":", 0, 0)	// Get file name without extension
	dfName = CleanupName(dfName, 0)
	NewDataFolder /O /S $dfName
	
	String targetText
	String format
	Variable year, month, day, hour, minute, second
	Variable foundDateTime
	Make/D/O/N=0 TimeWave
	Variable thRow = 0
	
	do
		FReadLine refNum, targetText
		if(stringmatch(targetText,"T,*"))
			format = "T,%d,%d,%d,%d,%d,%d"		
			sscanf targetText, format, year, month, day, hour, minute, second
			foundDateTime = Date2Secs(year,month,day)		// Convert year. month and day into seconds
			foundDateTime += second + 60*minute + 60*60*hour
			InsertPoints thRow, 1, TimeWave
			TimeWave[thRow] = foundDateTime
			thRow += 1
		endif	
	
	while(strlen(targetText)!=0)
	
	SetScale d, 0, 0, "dat", TimeWave
	Close refNum
	
	SetDataFolder ::
	return 0
End

Cheers,
Emre

March 31, 2015 at 10:28 am - Permalink