multi general text loading

Share icon

hello
I want to write a proc by which I can load all the text files in a folder. the files are categorised by name, for example basename_001, basename_002 and so on.
for loadindg the files simultanously I have to have a flexible path, which changes automaticaly after it imports the first file.
I wrote a test proc in which it uses the loadwave function. after I execute it it ask for the path, I give the path by browsing and choosing the first file, so it store the full path to the s_path string, then I want to use a loop in which there is another loadwave function except the path is coming from the s_path+index.
index goes up one by one and it continues until loading all the waves. but in any form I used the path,the procedure doesnt accept it while compiling, and mention: ill formed path,
would any body help please
If you show us the code, we can help you better.

To include igor code in these forum postings, put <igor> and </igor> tags around the text, like this:

<igor>
Function/S ListFilesOfType(dataFolderStr,type,fileTypeStr)
String dataFolderStr
Variable type // popup,"Igor Binary;Igor Text;General Text;Delimited Text;"
String fileTypeStr // usually "TEXT" for *.txt files on Windows

String files=""
if( type == 1 )
files= IndexedFile($dataFolderStr,-1,fileTypeStr) // list of all files with matching type
else
if( type == 2 ) // Igor Text, either "TEXT" or "IGTX" (.txt or .itx)
// open each TEXT or IGTX file, and if the first line is "IGOR", add it to files
files= IgorOrTextFiles(dataFolderStr,fileTypeStr,1) // on Windows, *.txt (or user's file type)
if( CmpStr(fileTypeStr,"IGTX") != 0 ) // avoid listing a file twice
files += IgorOrTextFiles(dataFolderStr,"IGTX",1) // on Windows, *.itx,
endif
else
files= IgorOrTextFiles(dataFolderStr,fileTypeStr,0) // on Windows, *.txt
endif
endif
return files
End

</igor>

and it will be nicely formatted:


Function/S ListFilesOfType(dataFolderStr,type,fileTypeStr)
	String dataFolderStr
	Variable type			// popup,"Igor Binary;Igor Text;General Text;Delimited Text;"
	String fileTypeStr		// usually "TEXT" for *.txt files on Windows

	String files=""
	if( type == 1 )
		files= IndexedFile($dataFolderStr,-1,fileTypeStr)	// list of all files with matching type
	else
		if( type == 2 )	// Igor Text, either "TEXT" or "IGTX" (.txt or .itx)
			// open each TEXT or IGTX file, and if the first line is "IGOR", add it to files
			files= IgorOrTextFiles(dataFolderStr,fileTypeStr,1)		// on Windows, *.txt (or user's file type)
			if( CmpStr(fileTypeStr,"IGTX") != 0 )					// avoid listing a file twice
				files +=  IgorOrTextFiles(dataFolderStr,"IGTX",1)	// on Windows, *.itx,
			endif
		else
			files= IgorOrTextFiles(dataFolderStr,fileTypeStr,0)		// on Windows, *.txt
		endif
	endif
	return files
End



--Jim Prouty
Software Engineer, WaveMetrics, Inc.

November 12, 2010 at 02:10 pm - Permalink
thank you for reply
I found two functions from Igor examples which do the thing that I want, but there still exists a problem

Function LoadAndGraph(fileName, pathName)
String fileName // Name of file to load or "" to get dialog
String pathName // Name of path or "" to get dialog
// Load the waves and set the variables.
LoadWave/g/D/O/P=$pathName fileName
if (V_flag==0) // No waves loaded. Perhaps user canceled.
return -1
endif 
End
Function LoadAndGraphAll(pathName)
String pathName // Name of symbolic path or "" to get dialog
String fileName
Variable index=0
if (strlen(pathName)==0) // If no path specified, create one
NewPath/O temporaryPath // This will put up a dialog
if (V_flag != 0)
return -1 // User cancelled
endif
pathName = "temporaryPath"
endif
Variable result
do // Loop through each file in folder
fileName = IndexedFile($pathName, index, ".txt")
if (strlen(fileName) == 0) // No more files?
break // Break out of loop
endif
result = LoadAndGraph(fileName, pathName)
index += 1
while (1)
if (Exists("temporaryPath")) // Kill temp path if it exists
KillPath temporaryPath
endif
return 0 // Signifies success.
End

suppose that I have a folder with 100 text files in it, when I call this function the window of loading waves apears and I have to
click on load button 100 times. if I want to cancel now I have to click cancel button 100 times again. I am not satisfied with this, is there any way to load 100 text files at once.
thank you in advance

November 14, 2010 at 01:37 pm - Permalink
I would rewrite it this way, but you should note that if pathName and fileName are valid, there won't be any dialog other than the one initial NewPath dialog for the user to cancel out of, so you shouldn't ever get into the situation where there are many dialogs to cancel out of.


#pragma rtGlobals=1		// Use modern global access method.
Function LoadAndGraph(fileName, pathName)
	String fileName // Name of file to load or "" to get dialog
	String pathName // Name of path or "" to get dialog
	// Load the waves and set the variables.
	LoadWave/g/D/O/P=$pathName fileName		// no dialog if pathName and fileName are valid
	if (V_flag==0)
		return -1 // No waves loaded (if pathName and file name are right, user doesn't have opportunity to cancel).
	endif 
	return 0 // 0 signifies success
End
Function LoadAndGraphAll(pathName)
	String pathName // Name of symbolic path or "" to get dialog
	String fileName
	Variable index=0
	if (strlen(pathName)==0) // If no path specified, create one
		NewPath/O temporaryPath // This will put up a dialog
		if (V_flag != 0)
			return -1 // User cancelled
		endif
		pathName = "temporaryPath"
	endif
	Variable result=0
	do // Loop through each file in folder
		fileName = IndexedFile($pathName, index, ".txt")
		if (strlen(fileName) == 0) // No more files?
			break // Break out of loop
		endif
		result = LoadAndGraph(fileName, pathName)
		if( result == -1 )
			break	// user cancelled or no waves in this file (which we take to be an error)
		endif
		index += 1
	while (1)
	if (Exists("temporaryPath")) // Kill temp path if it exists
		KillPath temporaryPath
	endif
	return result // 0 Signifies success.
End

--Jim Prouty
Software Engineer, WaveMetrics, Inc.

November 13, 2010 at 04:23 pm - Permalink
You need to add /A to the LoadWave command. Otherwise it will put up a dialog asking you to confirm or change the wave names.

November 13, 2010 at 06:47 pm - Permalink
thank you
I excecuted it and it worked properly. both advices were so helpful.
I learned a lot here. I will continue learning and discussing in this nice forum
thanks again

November 14, 2010 at 05:44 am - Permalink
hello again
I merged two functions and edited it a little. now it works nicely for me so I put it
here for whom it may be useful to import all the general text files in a folder at once.

function loadwaves(filename, pathname)
string filename
string pathname
if (strlen(pathname)==0)
newpath/o temporaryPath
if (v_flag!=0)
return -1
endif
pathname= "temporarypath"
endif
variable index = 0
do
filename= indexedfile ($pathname, index, ".txt")
if (strlen (filename)== 0)
break
endif
loadwave/a/g/o/d/p=$pathname filename
if (v_flag==0)
break
endif
index+=1
while(1)
end
menu "macros"
submenu "Load Waves"
"multi general text load", loadwaves("","")
end
end

November 14, 2010 at 01:27 pm - Permalink
Hi,

I've copied your program to Igor and tried to use it.
I got an error message that said "A non-existent data folder was referenced while accessing a child data folder".
My command line was: "LoadAndGraphAll(C:\Users\Aviv\Documents\Aviv\flex\Flex52 _Pani_90_30\)"

I'd appreciate any assistance on the matter, since you guys wrote that this program works for you.

Thanks you,
Aviv

JimProuty wrote: I would rewrite it this way, but you should note that if pathName and fileName are valid, there won't be any dialog other than the one initial NewPath dialog for the user to cancel out of, so you shouldn't ever get into the situation where there are many dialogs to cancel out of.


#pragma rtGlobals=1		// Use modern global access method.
Function LoadAndGraph(fileName, pathName)
	String fileName // Name of file to load or "" to get dialog
	String pathName // Name of path or "" to get dialog
	// Load the waves and set the variables.
	LoadWave/g/D/O/P=$pathName fileName		// no dialog if pathName and fileName are valid
	if (V_flag==0)
		return -1 // No waves loaded (if pathName and file name are right, user doesn't have opportunity to cancel).
	endif 
	return 0 // 0 signifies success
End
Function LoadAndGraphAll(pathName)
	String pathName // Name of symbolic path or "" to get dialog
	String fileName
	Variable index=0
	if (strlen(pathName)==0) // If no path specified, create one
		NewPath/O temporaryPath // This will put up a dialog
		if (V_flag != 0)
			return -1 // User cancelled
		endif
		pathName = "temporaryPath"
	endif
	Variable result=0
	do // Loop through each file in folder
		fileName = IndexedFile($pathName, index, ".txt")
		if (strlen(fileName) == 0) // No more files?
			break // Break out of loop
		endif
		result = LoadAndGraph(fileName, pathName)
		if( result == -1 )
			break	// user cancelled or no waves in this file (which we take to be an error)
		endif
		index += 1
	while (1)
	if (Exists("temporaryPath")) // Kill temp path if it exists
		KillPath temporaryPath
	endif
	return result // 0 Signifies success.
End

--Jim Prouty
Software Engineer, WaveMetrics, Inc.

December 22, 2011 at 02:04 am - Permalink
A literal string input to a function should be quoted:

 LoadAndGraphAll("C:\Users\Aviv\Documents\Aviv\flex\Flex52 _Pani_90_30\")


You also need to use double backslashes to have a backslash included in the path. Igor uses backslash to included special characters (such as tabs) in strings. You can also substitue a colon for a backslash in the path. Internally, Igor can use this Mac path separator convention. If you use double backslashes you command should look like this:

 LoadAndGraphAll("C:\\Users\\Aviv\\Documents\\Aviv\\flex\\Flex52 _Pani_90_30\\")

December 22, 2011 at 05:59 am - Permalink
Great, thank you very much.
I have another question on this part, if I want to use the filename variable and rename every wave loaded to it's filename + an extention, such as:
wave0 = filename_loaded
filename includes the file type "aviv.txt" instead of just "aviv".
I've tried using strcpy, strncpy, strcat, etc, but none of these commands work in Igor.
How can I shorten the filename?

Thank you again,
Aviv

December 29, 2011 at 02:50 am - Permalink
For example:

String str1 = "aviv.txt"
String str2 = RemoveEnding(str1, ".txt"); Print str2
String str3 = str1[0,strlen(str1)-5]; Print str3

December 29, 2011 at 05:30 am - Permalink
If the file extension can vary, then you might want to use ParseFilePath instead of RemoveEnding. To remove the extension use the form:

ParseFilePath(3, filename, ":", 0, 0)

where filename is your input file name with extension (with or without the path). If filename includes the path then the ":" above specifies the path separator. If you get the path from Igor, regardless of whether you are working on Mac or Windows, a colon will be the path separator. In effect this function removes everything after the final ".".

For example:

print ParseFilePath(3, "gabbagabba.hey", ":", 0, 0)


prints:

gabbagabba

December 29, 2011 at 01:44 pm - Permalink
Hello

I have also been attempting to write a macro which I can use to load all text files within a folder, unfortunately these are not categorised the same way as fairman's and are named as shown in the following example:

p100kimage545836.tif.dat
p100kimage545837.tif.dat
....etc. etc.......
p100kimage545936.tif.dat

Ideally I would also like the macro to append the new waves into seperate graph windows as they are loaded, as demonstrated using the following macro for an individual 2D intensity plot, unfortunately there are >100 scans make in each scan so it's not really feasible to do this individually!:


Macro Load2DImage(imagename)
	Variable imagename
	LoadWave/G/M/D/A=Image/E=0
	Display;AppendImage "Image0"
	ModifyImage "Image0" ctab= {0,50,Grays,1}
End


Initially I tried to accomplish this by using a do loop within a do loop, however, due to my lack of knowledge w.r.t. Igor syntax I have not found a means by which to read every file within a directory and allocate systematic wavenames to these files (wave0, wave1 etc.). Therefore after reading Fariman's original query I attempted to use the solutions that were suggested.

Unfortunately the following macro was only capable of loading a single wave and I'm unable to see how it differentiates from that suggested by JimProuty


macro Load2DImage(filename, pathname)
	string filename
	string pathname
	LoadWave/G/M/D/o/a/E=0/P=$pathname filename
	if (V_flag !=0)
		return -1
	endif
	return 0
end

macro Load2DImageAll(pathname)
	string pathname
	string filename
	variable index=0
	
	if (strlen(pathname)==0)
		newpath/o/m="Select a scan directory" temppath
		if (V_flag !=0)
			return -1
		endif
		pathname = "temppath"
	endif
	variable result=0
		
	do
		filename = indexedfile($pathname, index, ".tif.dat")
		if (strlen(filename) == 0)
			break
		endif
		result = Load2DImage(filename, pathname)
		if( result == -1 )
			break
		endif
		index += 1
	while (1)
	if (exists("temppath"))
		killpath temppath
	endif
	return result
end


Whereas the following macro failed to load any waves whatsoever and I'm also lost as to why.


Macro Load2DImage(filename, pathname)
string filename
string pathname
if (strlen(pathname)==0)
newpath/o/m="Select a scan directory" thepath
if (v_flag!=0)
return -1
endif
pathname= "thepath"
endif
variable index = 0
	do
		filename= indexedfile ($pathname, index, ".tif.dat")
		if (strlen (filename)== 0)
		break
		endif
		LoadWave/G/M/D/a=Image/E=0/P=$pathname filename
		Display;AppendImage $pathname filename
		ModifyImage $pathname filename ctab= {0,50,Grays,1}  
		if (V_flag==0)
		break
		endif
		index+=1
	while(1)
end


As you can probably establish I am fairly new to script writing as a whole so any suggestions, no matter how condescending you feel they may be, will be much appreciated.

Jon

February 18, 2012 at 09:58 am - Permalink
Igor considers the file name extension to be the text from the last dot to the end of the file name. So you need to use ".dat" with IndexedFile, not ".tif.dat".

Next, in Load2DImage, this statement:

	if (V_flag != 0)

should be:

	if (V_flag == 0)	// No waves loaded?


Finally, all new code should be written as functions, not as macros. To understand why, execute:

DisplayHelpTopic "Macros and Functions"


Here are the procedures rewritten as functions:


Function Load2DImage(filename, pathname)
	string filename
	string pathname
	
	LoadWave/G/M/D/o/a/E=0/P=$pathname filename
	if (V_flag == 0)		// No waves loaded?
		return -1		// Error
	endif
	return 0			// Success
End
 
Function Load2DImageAll(pathname)
	string pathname
 
	if (strlen(pathname)==0)
		newpath/o/m="Select a scan directory" temppath
		if (V_flag !=0)
			return -1
		endif
		pathname = "temppath"
	endif

	variable result=0
 
	Variable index=0
	do
		String filename = IndexedFile($pathname, index, ".dat")
		if (strlen(filename) == 0)
			break
		endif
		result = Load2DImage(filename, pathname)
		if (result != 0)
			Print "Error from Load2DImage, filename=", filename
			break
		endif
		index += 1
	while (1)
	
	if (exists("temppath"))
		killpath/Z temppath
	endif
	
	return result
End


February 18, 2012 at 11:22 am - Permalink
Hello Hrodstein,

Thank you very much for the quick reply, unfortunately when attempting to execute the functions posted I was met with "syntax error: expected string variable or string function".

Could this be effected in anyway by my using Igor 5.04B (on a mac)

Regards
Jon

February 19, 2012 at 09:47 am - Permalink
It works for me in Igor Pro 5.04B on Macintosh.

when attempting to execute the functions posted I was met with "syntax error: expected string variable or string function".


You need to provide all of the parameters when calling a user-defined function, like this:

Load2DImageAll("")


If you omit the parameter (an empty string in this case), you will get the "expected string" error.

February 19, 2012 at 10:14 am - Permalink