Extracting metadata from .dm3 files

Share icon

So Ive seen some scripts for doing this in imageJ, but Im terrible at java and I love the Igor environment, so I am curious how to go about doing it here, if its possible? Ive tried loading the files thru the various menu options, but the metadata waves get garbled every time so clearly I am doing it wrong so far.

Advice/tips much appreciated. Thanks.


EDIT: dm3 is a file format used by Gatan Digital Micrograph and are common in the microscopy world. Its basically a tiff with some metadata tucked in. I threw in an example file (its soot imaged with HRTEM if anybody is curious), need to rename it from .jpg to .dm3 as I couldnt upload it otherwise. 44-82969s1-87000X.jpg (4.16 MB)
So you were able to load at least the image data?

Here igor always complains with "out of memory" if I try to load your file (renamed to tiff before).

November 20, 2014 at 03:51 am - Permalink
I'm interested in an answer to this. I can't offer any help except to say that BioFormats in ImageJ displays the metadata for the file. As you know, you can extract parts of that using a script, which we do for files in other microscopy formats. We tend to export this extraction to Igor, but it would be great to do it direct.

November 20, 2014 at 04:59 am - Permalink
A quick look at the file suggests that it does not comply with TIFF standard (missing signature in the first few bytes) so perish the thought of using the ImageLoad operation to read these images. If you have sufficient information about the this file format you should be able to extract the relevant information using FBinRead (actually Open, FBinRead and Close).

A.G.
WaveMetrics, Inc.

November 20, 2014 at 12:45 pm - Permalink
I have knocked up a quick reader, based on the format description here (http://www.er-c.org/cbb/info/dmformat/#dm3) and here (http://rsb.info.nih.gov/ij/plugins/DM3Format.gj.html).
No guarantees that this is right.. but it should help you get something. Maybe someone can tidy up the code and make it more robust.
The meta data is put into a Notebook. I have suppressed the image data itself being put there.
Hope this helps,
Kurt

Code as follows:

 #pragma rtGlobals=3		// Use modern global access method and strict wave access.
Function LoadDM3()
	// User selects DM3 file
	// opens DM3 file
	// starts to read data from the file
	variable vRefNum
	string sFileOpenMessage="Select a DM3 file"
	string sFileFilter="DM3 files (*.dm3):.dm3;All Files:.*.*;"
	Open/R/Z=2/F=sFileFilter/M=sFileOpenMessage vRefNum
	// keep data from file open
	variable vErr=V_flag
	string sFullName=S_fileName
	string sCleanFileName=CleanUpName(ParseFilePath(3, sFullName, ":", 0, 0),0)
	if (vErr == -1)
		return -1			// User cancelled.
	endif
	if (vErr != 0)
		return vErr
	endif
	variable/G gvLayer=0 // indent or nested group layer
	variable/G gvPrintData=1 // =1 to allow printing of data, 0 to forbid (eg image data itself)
	NewNotebook/F=0/N=NBFileData
	LoadDM3NBSetData("Loading data from "+sFullName)
	string sData
	variable vData
	// start reading data from the file
	do
		// image file version
		FBinRead/B=2/F=3 vRefNum,vData // big-endian, 4 bytes
		if(vData!=3) // should be 3
			LoadDM3NBSetData("Error in file format (version)")
			break
		endif
		// number of bytes in file
		FBinRead/B=2/F=3 vRefNum,vData // big-endian, 4 bytes
		variable vNumBytes = vData
		LoadDM3NBSetData("Root tag directory size: "+num2str(vData))
		// byte order of tag data
		FBinRead/B=2/F=3 vRefNum,vData // big-endian, 4 bytes
		variable/G gvByteOrder // this will be the code in FBinRead/B=(gvByteOrder)
		if(vData==0) // big-endian
			gvByteOrder=2
			LoadDM3NBSetData("Data byte order: big-endian")
		elseif(vData==1) // little-endian
			gvByteOrder=3
			LoadDM3NBSetData("Data byte order: little-endian")
		else
			LoadDM3NBSetData("Error in file format (byte order)")
			break
		endif
		// Tag Group
		LoadDM3TagGroup(vRefNum)
		break
	while(1)
	FStatus vRefNum
	if(V_flag!=0) // is valid
		Close vRefNum		// Close the file.
	endif
	return vErr			// Zero signifies no error.	
End

Function LoadDM3NBSetData(sData)
	string sData
	NVAR gvLayer
	NVAR gvPrintData
	if(gvPrintData)
		sData=padstring("",gvLayer*2,0x20)+sData+"\r"
		Notebook NBFileData setData=sData
	endif
End

Function LoadDM3TagGroup(vRefNum)
	variable vRefNum
	// loads a DM3 file Tag Group
	NVAR gvLayer
	variable vFlag
	variable vData
	gvLayer+=1
	// is this group sorted?
	FBinRead/F=1 vRefNum,vData //  1 byte
	variable vGroupSorted
	vGroupSorted=vData
	LoadDM3NBSetData("Tag Group Sorted = "+num2str(vGroupSorted))
	// is this group open?
	FBinRead/F=1 vRefNum,vData //  1 byte
	variable vGroupOpen
	vGroupOpen=vData
	LoadDM3NBSetData("Tag Group Open = "+num2str(vGroupOpen))
	// number of tags in group
	FBinRead/B=2/F=3 vRefNum,vData //  big-endian, 4 bytes
	variable vNumTags
	vNumTags=vData
	LoadDM3NBSetData("Number of tags in this group = "+num2str(vNumTags))
	// tag entries
	variable vTags
	for(vTags=0;vTags<vNumTags;vTags+=1)
		vFlag=LoadDM3TagEntry(vRefNum)
		if(vFlag!=0) // had an error
			return -1
		endif
	endfor // vTags
	gvLayer-=1
	return 0
End

Function LoadDM3TagEntry(vRefNum)
	variable vRefNum
	// loads a DM3 file Tag Entry
	NVAR gvLayer
	variable vFlag
	variable vData
	string sData
	// Tag type
	FBinRead/F=1 vRefNum,vData //  1 byte
	variable vTagType
	vTagType=vData // 21 = data, 20 = group
	if(vTagType==21) // data - TagType
		LoadDM3NBSetData("Tag type: 21 (Data)")
	elseif(vTagType==20) // Group
		LoadDM3NBSetData("Tag type: 20 (Group)")
	elseif(vTagType==0) // end of file
		LoadDM3NBSetData("End of file.")
		return -1
	else
		LoadDM3NBSetData("Error: Tag type.")
		return -1		
	endif
	// Tag label length
	FBinRead/B=2/F=2 vRefNum,vData //  2 bytes
	variable vTagLabelLength
	vTagLabelLength=vData
	// Tag label
	if(vTagLabelLength==0) // zero length
		sData="_no label_"
	else
		FReadLine/N=(vTagLabelLength) vRefNum, sData
	endif
	LoadDM3NBSetData("Tag Label: "+sData)
	NVAR gvPrintData
	if(cmpStr(sData,"Data")==0)
		gvPrintData=0 // forbid printing data
	else
		gvPrintData=1 // allow printing data
	endif
	// Tag instance
	if(vTagType==21) // data - TagType
		vFlag=LoadDM3TagType(vRefNum)
		if(vFlag!=0)
			return -1
		endif
	elseif(vTagType==20) // Group
		vFlag=LoadDM3TagGroup(vRefNum) // note: this is recursive
		if(vFlag!=0)
			return -1
		endif
	endif
	return 0
End

Function LoadDM3TagType(vRefNum) // tag data
	variable vRefNum
	string sData
	variable vData
	NVAR gvLayer
	FReadLine/N=(4) vRefNum, sData
	if(cmpstr(sData,"%%%%")!=0)
		LoadDM3NBSetData("Error in Tag (expected %%%%).")
		return -1
	endif
	// length of encoded type definition
	FBinRead/B=2/F=3 vRefNum,vData //  4 bytes
    		// for a simple type this will = 1,
    		// for a string this will = 2,
    		// an array of a simple type will = 3,
    		// structs have 1+2*f where f=number of fields in struct
    	variable vDefinitionLength=vData    	
    	// encoded type info
	make/FREE/O/I/N=(vDefinitionLength) wEncodingType // 4-byte wave
	FBinRead/B=2/F=3 vRefNum,wEncodingType
	
	variable vIndex,vIndex2,vDataType,vSize
	switch(vDefinitionLength)
		case 1: // simple type
			LoadDM3EncodedType(vRefNum,wEncodingType[0],wEncodingType)
			break
		case 2:  // string type
			FBinRead/B=2/F=3 vRefNum,vData //  4 bytes
			LoadDM3EncodedType(vRefNum,vData,wEncodingType)
			break
		case 3: // array of simple type
			// wEncodingType[0]=20 - array
			// wEncodingType[1]= tag data type for array members
			// wEncodingType[2]= size of array
			LoadDM3NBSetData("Array Data ("+num2str(wEncodingType[2])+")")
			for(vIndex=0;vIndex<wEncodingType[2];vIndex+=1) // loop round fields
				LoadDM3EncodedType(vRefNum,wEncodingType[1],wEncodingType)
			endfor // vIndex
			break
		default: // struct or array type
			switch(wEncodingType[0])
				case 15: // struct type
					// wEncodingType[0]=15 - group
					// wEncodingType[1]=0 - usually
					// wEncodingType[2]= number fields
					// wEncodingType[3+field*2]= length field name =0
					// wEncodingType[4+field*2]= tag data type for value[field]
					LoadDM3NBSetData("Struct Data ("+num2str(wEncodingType[2])+")")
					for(vIndex=0;vIndex<wEncodingType[2];vIndex+=1) // loop round fields
						LoadDM3EncodedType(vRefNum,wEncodingType[4+vIndex*2],wEncodingType)
					endfor // vIndex
					break
				case 20: // array of structs type
					// wEncodingType[0]=20 - array
					// wEncodingType[1]=15 - group
					// wEncodingType[2]=0 - (group name) usually
					// wEncodingType[3]=number of entries in struct (field)
					// wEncodingType[4+field*2]= length field name =0
					// wEncodingType[5+field*2]= tag data type for value[field]
					// wEncodingType[...]
					// wEncodingType[last]= size of array (i.e. number of repeats of struct)
					LoadDM3NBSetData("Array Data ("+num2str(wEncodingType[2])+")")
					for(vIndex2=0;vIndex2<wEncodingType[DimSize(wEncodingType,0)-1];vIndex2+=1) // loop round array
						for(vIndex=0;vIndex<wEncodingType[3];vIndex+=1) // loop round fields
							LoadDM3EncodedType(vRefNum,wEncodingType[5+vIndex*2],wEncodingType)
						endfor // vIndex
					endfor // vIndex2
					break
			endSwitch
	endSwitch
	return 0
End

Function LoadDM3EncodedType(vRefNum,vEncodingType,wEncodingType)
	variable vRefNum
	variable vEncodingType
	wave wEncodingType
	variable vData, vIndex
	string sData
	NVAR gvByteOrder
	NVAR gvLayer
	switch(vEncodingType)
		case 2: // short
			FBinRead/B=(gvByteOrder)/f=2 vRefNum,vData
			LoadDM3NBSetData("Short data: "+num2str(vData))
			break
		case 3: // long
			FBinRead/B=(gvByteOrder)/f=3 vRefNum,vData
			LoadDM3NBSetData("Long data: "+num2str(vData))
			break
		case 4: // ushort
			FBinRead/B=(gvByteOrder)/f=2/U vRefNum,vData
			LoadDM3NBSetData("UShort data: "+num2str(vData))
			break
		case 5: // ulong
			FBinRead/B=(gvByteOrder)/f=3/U vRefNum,vData
			LoadDM3NBSetData("ULong data: "+num2str(vData))
			break
		case 6: // float
			FBinRead/B=(gvByteOrder)/f=4 vRefNum,vData
			LoadDM3NBSetData("Float data: "+num2str(vData))
			break
		case 7: //double
			FBinRead/B=(gvByteOrder)/f=5 vRefNum,vData
			LoadDM3NBSetData("Double data: "+num2str(vData))
			break
		case 8: // boolean
			FBinRead/F=1 vRefNum,vData //  1 byte
			LoadDM3NBSetData("Boolean data = "+num2str(vData))
			break
		case 9: // char
			FReadLine/N=(1) vRefNum, sData
			LoadDM3NBSetData("Char data = "+sData)
			break
		case 10: // octet
			FBinRead/F=1 vRefNum,vData //  1 byte
			LoadDM3NBSetData("Octet data = "+num2str(vData))
			break
		case 15: // struct
			
			break
		case 18: // string
			for(vIndex=0;vIndex<wEncodingType[1];vIndex+=1) // loop round string length
				FBinRead/B=(gvByteOrder)/f=2 vRefNum,vData
				LoadDM3NBSetData("Unicode string data = "+num2str(vData)+ " ("+num2char(vData)+")")
			endfor
			break
		case 20: // array
		
			break
	endSwitch
End

November 21, 2014 at 01:16 am - Permalink
Wow guys, thanks!

I had almost given up on this, and got distracted by work for some time, only to come back and find all this great help. I was otherwise looking at doing this manually for several hundred images (and that is just one week's work).

I love these forums, really.

EDIT: Acquisition time is given as the ushort data (all the meta data is provided in the raw format), I've decoded the following

0-9 = 48-57
: = 58
space = 32
P = 80
M = 77
A should follow from this to be equal to 65 (77 - 12)

Now to hijack this script so that it pulls this out from a group of files, translates it and writes it to a wave. Fun fun =)

May 5, 2015 at 05:20 am - Permalink
All right, I got it running so that you select a folder, and then it will open each file, pull the time stamp, and put that data into a wave entry for easy use. Problem is just that it runs _real_ slow. I check to see if some of the metadata (like timestamps) had permanent positions in the .dm3 file, but no such luck. So I am forced to let the script read through the file piece by piece (with notebook disabled), until it finds the correct header, at which point it extracts the data.

Is it possible to search the entire open file for a specific string and then get that string position? This would be significantly faster, I imagine.

Here is the modified code if anybody wants to use it. Its ugly after I got done with it. 

 #pragma rtGlobals=3		// Use modern global access method and strict wave access.
//Notebook function now commented out
Function extract_timestamps()

	string list_dm3_files, prompt_alert,file_name
	variable num_files
	variable/G i
	//prompt for path to data folder
	newpath/o dm3_data_folder
	pathinfo dm3_data_folder
	
	// load file names into string, sorting appropriately
	list_dm3_files = IndexedFile(dm3_data_folder, -1, ".dm3")
	
	// Sort using combined alpha and numeric sort
	list_dm3_files = SortList(list_dm3_files, ";", 16)

	

	//print list_dm3_files
	num_files = itemsinlist(list_dm3_files)
	
	
	
	if(num_files == 0)
		prompt_alert = "Could not find any DM3 data files in the chosen folder"
		doAlert 0, prompt_alert
		return 0
	endif
	
	//make an output wave to store timestamps
	make/o/d/n=(num_files) timestamps
	
	
	// cycle through each file, one at a time	
	for(i=0;i<num_files;i+=1)
	file_name = stringfromlist(i,list_dm3_files)
	LoadDM3(file_name)
	endfor
	appendtotable timestamps
	modifytable format(timestamps) = 7
	SetScale d 0,0,"dat", timestamps
	
End



Function LoadDM3(file_name)
	string file_name
	variable vRefNum
	Open/R/Z=2/P=dm3_data_folder vRefNum as file_name
	variable vErr=V_flag
	



	// User selects DM3 file
	// opens DM3 file
	// starts to read data from the file
	//variable vRefNum
	//string sFileOpenMessage="Select a DM3 file"
	//string sFileFilter="DM3 files (*.dm3):.dm3;All Files:.*.*;"
	//Open/R/Z=2/F=sFileFilter/M=sFileOpenMessage vRefNum
	// keep data from file open
	//variable vErr=V_flag
	//string sFullName=S_fileName
	//string sCleanFileName=CleanUpName(ParseFilePath(3, sFullName, ":", 0, 0),0)
	//if (vErr == -1)
	//	return -1			// User cancelled.
	//endif
	//if (vErr != 0)
	//	return vErr
	//endif
	variable/G gvLayer=0 // indent or nested group layer
	variable/G gvPrintData=1 // =1 to allow printing of data, 0 to forbid (eg image data itself)
	//NewNotebook/F=0/N=NBFileData
	//LoadDM3NBSetData("Loading data from "+sFullName)
	string sData
	variable vData
	// start reading data from the file
	do
		// image file version
		FBinRead/B=2/F=3 vRefNum,vData // big-endian, 4 bytes
		if(vData!=3) // should be 3
			//LoadDM3NBSetData("Error in file format (version)")
			break
		endif
		// number of bytes in file
		FBinRead/B=2/F=3 vRefNum,vData // big-endian, 4 bytes
		variable vNumBytes = vData
		//LoadDM3NBSetData("Root tag directory size: "+num2str(vData))
		// byte order of tag data
		FBinRead/B=2/F=3 vRefNum,vData // big-endian, 4 bytes
		variable/G gvByteOrder // this will be the code in FBinRead/B=(gvByteOrder)
		if(vData==0) // big-endian
			gvByteOrder=2
			//LoadDM3NBSetData("Data byte order: big-endian")
		elseif(vData==1) // little-endian
			gvByteOrder=3
			//LoadDM3NBSetData("Data byte order: little-endian")
		else
			//LoadDM3NBSetData("Error in file format (byte order)")
			break
		endif
		// Tag Group
		LoadDM3TagGroup(vRefNum)
		break
	while(1)
	FStatus vRefNum
	if(V_flag!=0) // is valid
		Close vRefNum		// Close the file.
	endif
	return vErr			// Zero signifies no error.	
	
End
 
Function LoadDM3NBSetData(sData)
	string sData
	NVAR gvLayer
	NVAR gvPrintData
	if(gvPrintData)
		sData=padstring("",gvLayer*2,0x20)+sData+"\r"
		//Notebook NBFileData setData=sData
	endif
End
 
Function LoadDM3TagGroup(vRefNum)
	variable vRefNum
	// loads a DM3 file Tag Group
	NVAR gvLayer
	variable vFlag
	variable vData
	gvLayer+=1
	// is this group sorted?
	FBinRead/F=1 vRefNum,vData //  1 byte
	variable vGroupSorted
	vGroupSorted=vData
	//LoadDM3NBSetData("Tag Group Sorted = "+num2str(vGroupSorted))
	// is this group open?
	FBinRead/F=1 vRefNum,vData //  1 byte
	variable vGroupOpen
	vGroupOpen=vData
	//LoadDM3NBSetData("Tag Group Open = "+num2str(vGroupOpen))
	// number of tags in group
	FBinRead/B=2/F=3 vRefNum,vData //  big-endian, 4 bytes
	variable vNumTags
	vNumTags=vData
	//LoadDM3NBSetData("Number of tags in this group = "+num2str(vNumTags))
	// tag entries
	variable vTags
	for(vTags=0;vTags<vNumTags;vTags+=1)
		vFlag=LoadDM3TagEntry(vRefNum)
		if(vFlag!=0) // had an error
			return -1
		endif
	endfor // vTags
	gvLayer-=1
	return 0
End
 
Function LoadDM3TagEntry(vRefNum)
	variable vRefNum
	// loads a DM3 file Tag Entry
	NVAR gvLayer
	variable vFlag
	variable vData
	string sData
	// Tag type
	FBinRead/F=1 vRefNum,vData //  1 byte
	variable vTagType
	vTagType=vData // 21 = data, 20 = group
	if(vTagType==21) // data - TagType
		//LoadDM3NBSetData("Tag type: 21 (Data)")
	elseif(vTagType==20) // Group
		//LoadDM3NBSetData("Tag type: 20 (Group)")
	elseif(vTagType==0) // end of file
		//LoadDM3NBSetData("End of file.")
		return -1
	else
		//LoadDM3NBSetData("Error: Tag type.")
		return -1		
	endif
	// Tag label length
	FBinRead/B=2/F=2 vRefNum,vData //  2 bytes
	variable vTagLabelLength
	vTagLabelLength=vData
	// Tag label
	if(vTagLabelLength==0) // zero length
		sData="_no label_"
	else
		FReadLine/N=(vTagLabelLength) vRefNum, sData
	endif
	
	// EDIT of original code, introducing case for exporting data directly into IGOR
	
	if(cmpStr(sData, "Acquisition Time")==0)
	//read 5x to move to correct position
	FReadLine/N=(4) vRefNum, sData
	FBinRead/B=2/F=3 vRefNum,vData
	FBinRead/B=2/F=3 vRefNum,vData
	FBinRead/B=2/F=3 vRefNum,vData
	FBinRead/B=2/F=3 vRefNum,vData

	//read actual timestamp data
	wave timestamps
	Nvar gvByteOrder,i
	variable byte_num,hours,minutes,seconds,total_seconds
	variable timestampLength
	string timepiece
	string timestamp

	FBinRead/B=(gvByteOrder)/f=2/U vRefNum, vData  // read the first piece
	//print vData
	vData -= 48 // translates from unsigned short data format
	timepiece = num2str(vData)
	timestamp = timepiece
	
	FBinRead/B=(gvByteOrder)/f=2/U vRefNum, vData  // read the second piece
	//print vData
	vData -= 48
	if (vData != 10) // check for the semi-colon and describe length of stamp
	timepiece = num2str(vData)
	timestampLength = 6
	else
	timepiece = ":"
	timestampLength = 5
	endif 
	timestamp = timestamp + timepiece

	
	for(byte_num=0;byte_num<timestampLength;byte_num+=1)
		FBinRead/B=(gvByteOrder)/f=2/U vRefNum, vData  // read the rest of it
		//print vData
		vData -= 48
		if (vData != 10) // check for the semi-colon
			timepiece = num2str(vData)
		else
			timepiece = ":"
		endif 
	timestamp = timestamp + timepiece
	endfor
	print "The timestamp is :", timestamp 
	hours = str2num(StringFromList(0, timestamp ,":"))
	// set 1 pm - 7 pm to military hours
	if (hours < 8)
		hours += 12
	endif
	total_seconds = hours * 3600
	minutes = str2num(StringFromList(1,timestamp,":"))
	total_seconds += minutes*60
	seconds = str2num(StringFromList(2,timestamp,":"))
	total_seconds += seconds
	timestamps[i] = total_seconds
	


	// continue with normal code
      else
	LoadDM3NBSetData("Tag Label: "+sData)
	NVAR gvPrintData
	if(cmpStr(sData,"Data")==0)
		gvPrintData=0 // forbid printing data
	else
		gvPrintData=1 // allow printing data
	endif
	// Tag instance
	if(vTagType==21) // data - TagType
		vFlag=LoadDM3TagType(vRefNum)
		if(vFlag!=0)
			return -1
		endif
	elseif(vTagType==20) // Group
		vFlag=LoadDM3TagGroup(vRefNum) // note: this is recursive
		if(vFlag!=0)
			return -1
		endif
	endif

	endif // End of the introduced case see above
	return 0
End
 
Function LoadDM3TagType(vRefNum) // tag data
	variable vRefNum
	string sData
	variable vData
	NVAR gvLayer
	FReadLine/N=(4) vRefNum, sData
	if(cmpstr(sData,"%%%%")!=0)
		//LoadDM3NBSetData("Error in Tag (expected %%%%).")
		return -1
	endif
	// length of encoded type definition
	FBinRead/B=2/F=3 vRefNum,vData //  4 bytes
    		// for a simple type this will = 1,
    		// for a string this will = 2,
    		// an array of a simple type will = 3,
    		// structs have 1+2*f where f=number of fields in struct
    	variable vDefinitionLength=vData    	
    	// encoded type info
	make/FREE/O/I/N=(vDefinitionLength) wEncodingType // 4-byte wave
	FBinRead/B=2/F=3 vRefNum,wEncodingType
 
	variable vIndex,vIndex2,vDataType,vSize
	switch(vDefinitionLength)
		case 1: // simple type
			LoadDM3EncodedType(vRefNum,wEncodingType[0],wEncodingType)
			break
		case 2:  // string type
			FBinRead/B=2/F=3 vRefNum,vData //  4 bytes
			LoadDM3EncodedType(vRefNum,vData,wEncodingType)
			break
		case 3: // array of simple type
			// wEncodingType[0]=20 - array
			// wEncodingType[1]= tag data type for array members
			// wEncodingType[2]= size of array
			//LoadDM3NBSetData("Array Data ("+num2str(wEncodingType[2])+")")
			for(vIndex=0;vIndex<wEncodingType[2];vIndex+=1) // loop round fields
				LoadDM3EncodedType(vRefNum,wEncodingType[1],wEncodingType)
			endfor // vIndex
			break
		default: // struct or array type
			switch(wEncodingType[0])
				case 15: // struct type
					// wEncodingType[0]=15 - group
					// wEncodingType[1]=0 - usually
					// wEncodingType[2]= number fields
					// wEncodingType[3+field*2]= length field name =0
					// wEncodingType[4+field*2]= tag data type for value[field]
					//LoadDM3NBSetData("Struct Data ("+num2str(wEncodingType[2])+")")
					for(vIndex=0;vIndex<wEncodingType[2];vIndex+=1) // loop round fields
						LoadDM3EncodedType(vRefNum,wEncodingType[4+vIndex*2],wEncodingType)
					endfor // vIndex
					break
				case 20: // array of structs type
					// wEncodingType[0]=20 - array
					// wEncodingType[1]=15 - group
					// wEncodingType[2]=0 - (group name) usually
					// wEncodingType[3]=number of entries in struct (field)
					// wEncodingType[4+field*2]= length field name =0
					// wEncodingType[5+field*2]= tag data type for value[field]
					// wEncodingType[...]
					// wEncodingType[last]= size of array (i.e. number of repeats of struct)
					//LoadDM3NBSetData("Array Data ("+num2str(wEncodingType[2])+")")
					for(vIndex2=0;vIndex2<wEncodingType[DimSize(wEncodingType,0)-1];vIndex2+=1) // loop round array
						for(vIndex=0;vIndex<wEncodingType[3];vIndex+=1) // loop round fields
							LoadDM3EncodedType(vRefNum,wEncodingType[5+vIndex*2],wEncodingType)
						endfor // vIndex
					endfor // vIndex2
					break
			endSwitch
	endSwitch
	return 0
End
 
Function LoadDM3EncodedType(vRefNum,vEncodingType,wEncodingType)
	variable vRefNum
	variable vEncodingType
	wave wEncodingType
	variable vData, vIndex
	string sData
	NVAR gvByteOrder
	NVAR gvLayer
	switch(vEncodingType)
		case 2: // short
			FBinRead/B=(gvByteOrder)/f=2 vRefNum,vData
			//LoadDM3NBSetData("Short data: "+num2str(vData))
			break
		case 3: // long
			FBinRead/B=(gvByteOrder)/f=3 vRefNum,vData
			//LoadDM3NBSetData("Long data: "+num2str(vData))
			break
		case 4: // ushort
			FBinRead/B=(gvByteOrder)/f=2/U vRefNum,vData
			//LoadDM3NBSetData("UShort data: "+num2str(vData))
			break
		case 5: // ulong
			FBinRead/B=(gvByteOrder)/f=3/U vRefNum,vData
			//LoadDM3NBSetData("ULong data: "+num2str(vData))
			break
		case 6: // float
			FBinRead/B=(gvByteOrder)/f=4 vRefNum,vData
			//LoadDM3NBSetData("Float data: "+num2str(vData))
			break
		case 7: //double
			FBinRead/B=(gvByteOrder)/f=5 vRefNum,vData
			//LoadDM3NBSetData("Double data: "+num2str(vData))
			break
		case 8: // boolean
			FBinRead/F=1 vRefNum,vData //  1 byte
			//LoadDM3NBSetData("Boolean data = "+num2str(vData))
			break
		case 9: // char
			FReadLine/N=(1) vRefNum, sData
			//LoadDM3NBSetData("Char data = "+sData)
			break
		case 10: // octet
			FBinRead/F=1 vRefNum,vData //  1 byte
			//LoadDM3NBSetData("Octet data = "+num2str(vData))
			break
		case 15: // struct
 
			break
		case 18: // string
			for(vIndex=0;vIndex<wEncodingType[1];vIndex+=1) // loop round string length
				FBinRead/B=(gvByteOrder)/f=2 vRefNum,vData
				//LoadDM3NBSetData("Unicode string data = "+num2str(vData)+ " ("+num2char(vData)+")")
			endfor
			break
		case 20: // array
 
			break
	endSwitch
End

May 13, 2015 at 05:44 am - Permalink
daggaz wrote:
Is it possible to search the entire open file for a specific string and then get that string position? This would be significantly faster, I imagine.

Just a thought - you could use 
FStatus
to find the length of the file, then use 
PadString
and 
FBinRead
to read the entire file into a string variable. Then finally search this for your specific string using 
strsearch
.
I don't know whether this would be faster.
HTH,
Kurt

May 13, 2015 at 11:43 pm - Permalink
Thanks Kurt, I'll play around with that. Its got to be faster than calling multiple recursive functions to search the file.

May 18, 2015 at 12:13 am - Permalink