Loading a text file with space and tabs

Hello, 

I would like to load a series of .csv files that unfortunately contain a mixture of "tabs" and spaces. The procedure that I have to load the data works well for the data that is separated with tabs but not at all for the spaces. I have included an example of my data files plus the IGOR procedure used for loading. 

Any help is appreciated

Function Loadsmps()	// Loads smps data

// Prompts to select folder into which data is loaded
String PathToFldr="",CtrlName="",PromptStr="Select folder to load smps data into:",CurrentFldr=getdatafolder(1)	// variables for Browse4Folder
Browse4Folder(PathToFldr,CtrlName,PromptStr,CurrentFldr)	// prompts user to select an experiment

// Get folder with data in it
	NewPath/M="Select folder containing smps data:"/O/Q pathname	// prompts user to select path to data folder
	PathInfo pathname
// Abort if cancel is pressed
	If(V_flag==0)		
	Abort "proceedure cancelled"	
	EndIf
// Make matrix for smps data
	Make/o/n=(0,175) smpsDataMatrix
// Strings and Variables 
Variable numfolders,n, subdirs, numfiles, filenum, refnum	// variables
String folderlist, filename, name, currentfolder			//    strings
// Counts number of files in folder			
NumFiles=ItemsInList(IndexedFile(pathname,-1,".csv")) 								// # files in folder	, specify extensions e.g. ".txt"
	
	For(filenum=0;filenum<numfiles;filenum+=1) // loops through each file name	
		filename=s_path+StringFromList(filenum,IndexedFile(pathname,-1,".csv"))	// get full path to data file	, specify extensions e.g. ".txt"
		If(Stringmatch(FileName,"*dmps*")==1)
			Open /R/P=pathname/Z refNum as filename	// open
			FStatus refNum	// check there is data in file
				If(V_logEOF>10)	// specify minimum file size to load e.g. >10bytes
					Load_smps_File(filename, refnum)	// calls function to load file	// unique function for specific file format
					Wave  Temp_smpsDataMatrix0	
					//	add loaded waves to loaded data
					InsertPoints DimSize(smpsDataMatrix,0),DimSize(Temp_smpsDataMatrix0,0), smpsDataMatrix
					smpsDataMatrix[DimSize(smpsDataMatrix,0)-DimSize(Temp_smpsDataMatrix0,0),*][]=Temp_smpsDataMatrix0[p-DimSize(smpsDataMatrix,0)+DimSize(Temp_smpsDataMatrix0,0)][q]
					currentfolder=GetDataFolder(1)			// root folder for experiment	
				Endif	// end data to folder allocation
		EndIf
		
	EndFor			// end loop through files of a dp
	
	Close/A 			// close open files
		
	Killwaves Temp_smpsDataMatrix0
//	Do additional analytsis on smps data
	Processsmps()	// adds a time wave

End



Function Load_smps_File(filename, refnum)	// load single smps file
	String filename
	Variable refnum
	print "Loading: " + filename
	LoadWave/V={"\t, ","",0,0}/q/a/n=Temp_smpsDataMatrix/m/o/j/k=1/L={0,3,0,0,0} filename

End

Function Processsmps()		// Additional processing of smps data
	Wave smpsDataMatrix
	Make/o/d/n=(DimSize(smpsDataMatrix,0)) smpsTime
	Make/o/d/n=(DimSize(smpsDataMatrix,0)) smpsdate
	SetScale d,0,0,"dat" smpsTime 
		SetScale d,0,0,"dat" smpsdate
	smpsDate=Date2Secs(smpsDataMatrix[p][0],smpsDataMatrix[p][1],smpsDataMatrix[p][2])+smpsDataMatrix[p][3]*3600+smpsDataMatrix[p][4]*60+smpsDataMatrix[p][5]
smpsTime=smpsDataMatrix[p][3]*3600+smpsDataMatrix[p][4]*60+smpsDataMatrix[p][5]
End
End

e#pragma rtGlobals=1		// Use modern global access method.

Function Browse4Folder(PathToFldr,CtrlName,PromptStr,CurrentFldr)	// based on mg code from htdma toolkit
String PathToFldr	// path to data folder
String PromptStr	// Prompt text in data browser
String CurrentFldr // current data folder
String CtrlName

variable exit=0	// stops trying when exit reaches 6
do
	string/g BrowserList
	string /g TempFldr=CurrentFldr
	variable/g Flag
	execute "CreateBrowser prompt=\""+PromptStr+"\", select=tempfldr"	// prompt for file
	killstrings/z tempfldr
	svar BrowserList=s_browserlist
	nvar Flag=V_Flag
	// print browser_list
	if (!Flag)
			PathToFldr="cancelled"		
			break
		endif
		
		if (itemsinlist(BrowserList)==1)
			if (stringmatch("root:ROOT;", BrowserList))		//data browser returns strange path if root is selected
				BrowserList="root"
			endif
			PathToFldr=stringfromlist(0,BrowserList)+":"
			if(DataFolderExists(PathToFldr))
				break
			else
				doalert 0, "you must select a data folder, not a wave, string or variable! => try again"
			endif
		else
			doalert 0, "none or multiple data folders were selected => try again"
		endif
		exit+=1		
	while (exit<6)		//maximum six attempts
End

 

20190115-dmps_cpc.csv.csv (18.23 KB) 20190116-dmps_cpc.csv.csv (27.47 KB)

Since your file does not contain string date or date/time data, and since it does not contain blank data as far as I can tell, you can use LoadWave/G (Load General Text) instead of LoadWave/J (Load Delimited Text).

Load General Text automatically accepts spaces or a tab as a delimiter. It also automatically skips the header to find the numeric data, so you don't need the /L or /K flags.

This worked for me to load the files you attached:

LoadWave/G/O/M/N=Temp_smpsDataMatrix/Q

For more information, execute:

DisplayHelpTopic "Loading General Text Files"

DisplayHelpTopic "Loading Delimited Text Files"

Hello hrodstein, 

Thank you very much for getting back to me and apologies for the delay in my reply. However, I am having the same problem. The first 14 columns load correctly and then the 14th column is copied for the remainder of the matrix. 

When I check the raw files it seems that there are tabs at the start and then these change to spaces afterwards.

Thanks again for your help.

 

 

In reply to by LaMP

LaMP wrote: The first 14 columns load correctly and then the 14th column is copied for the remainder of the matrix.

I don't see that. For me, the waves load correctly using the command the I listed above.

I am using Igor Pro 8.04.

I will attach a zip archive containing my test experiment and the result of executing the LoadWave command above as well as the files that I loaded. The files are your files renamed for simplicity.

If you need further help, please specify your Igor version and include a self-contained example like the one I am attaching that reproduces the problem.

LAMP1_0.zip (30.4 KB)

Hello again, 

Problem solved. I had copied the new code snipet into the wrong load procedure :/, so that is why I didnt see any change! Thanks again for your help. 

For info, I am still working in IGOR 6. 

Forum

Support

Gallery

Igor Pro 10

Learn More

Igor XOP Toolkit

Learn More

Igor NIDAQ Tools MX

Learn More