But the problem is not fixed. Now the code is running well but I am getting the concatenated waves without any values. I am attaching a screenshot of it. Could you have a look of this and let me know if you have any ideas? Thanks.
--
J. J. Weimer
Chemistry / Chemical & Materials Engineering, UAH
Well, I used a different designation but the problem is not fixed. If I change like that, the code again doesn't run. I am working on it for hours to get the expected concatenated waves with correct length but each time having some issues. Anyways, thanks a lot.
I just looked again at the code and how it works. Forget my earlier answer. I see now how it is supposed to run (I think).
When I run your code, it does the first two iterations (loads the 1st and 2nd dat files) and does the concatenation, on the next iteration it says that it loads the file but no waves have been created in the temporary data folder. This means that the wave references to ProtonEnergyFor_SpecAmpNew etc are NULL, when the concatenate command runs it gives Error: Expected Wave Reference for this reason. I didn't investigate why no waves are loaded on subsequent iterations. I did check if it was something to do with the file, but the code just fails on the next file if I take that particular file out of the directory. Sorry I haven't fixed the problem, but hopefully this brings you closer to a solution.
Switch the debugger on and check out the state of the tempdatafolder when it throws the error. Note that in the history window it says:
path: "Macintosh HD:Users:xxx:Downloads:ONeDatFile:"
Reading from file "s_0340_r_000001_1.dat". Number of Event is: 326
Loaded file 0: "s_0340_r_000001_1.dat"
Reading from file "s_0340_r_000003_1.dat". Number of Event is: 270
Loaded file 1: "s_0340_r_000003_1.dat"
Loaded file 2: "s_0340_r_000001_7.dat"
It looks like it doesn't read from s_0340_r_000001_7.dat so my guess is that the bug is with
sjr51 wrote: I just looked again at the code and how it works. Forget my earlier answer. I see now how it is supposed to run (I think).
When I run your code, it does the first two iterations (loads the 1st and 2nd dat files) and does the concatenation, on the next iteration it says that it loads the file but no waves have been created in the temporary data folder. This means that the wave references to ProtonEnergyFor_SpecAmpNew etc are NULL, when the concatenate command runs it gives Error: Expected Wave Reference for this reason. I didn't investigate why no waves are loaded on subsequent iterations. I did check if it was something to do with the file, but the code just fails on the next file if I take that particular file out of the directory. Sorry I haven't fixed the problem, but hopefully this brings you closer to a solution.
Switch the debugger on and check out the state of the tempdatafolder when it throws the error. Note that in the history window it says:
path: "Macintosh HD:Users:xxx:Downloads:ONeDatFile:"
Reading from file "s_0340_r_000001_1.dat". Number of Event is: 326
Loaded file 0: "s_0340_r_000001_1.dat"
Reading from file "s_0340_r_000003_1.dat". Number of Event is: 270
Loaded file 1: "s_0340_r_000003_1.dat"
Loaded file 2: "s_0340_r_000001_7.dat"
It looks like it doesn't read from s_0340_r_000001_7.dat so my guess is that the bug is with
LoadOneNeutronDataFile
Hi,
Thank you very much for your explanation.
I got your point. I tried to modify the code in a different way. For your attention, my target is to read only _1.dat files of series 340. Since Each series has almost 2500 files, so I switched from "Concatenate [/DL /KILL /NP[=dim ] /O] [typeFlags ] {wave1, wave2, wave3,...}, destWave" command to "Concatenate [ /DL /KILL /NP[=dim ] /O] [typeFlags ] waveListStr, destWave" command, since according to Igor manual, the first one is limited to concatenate 100 waves or less, but there is no limit when using waveListStr. Now the code is looping over all files but I am facing another problem, it is not giving the correct length of the concatenated wave. For example, If I concatenate wave1 with N points to wave2 with M points, then I should have a concatenated wave with points (N+M). To check it, I took one file in a different folder (file "s_0340_r_000000_1.dat") and then ran the code. It gave me the correct number of points (274) which is equal to the number of events in that file. Then I add another file (file "s_0340_r_000001_1.dat") to that folder and ran the code again over these two files. Before that, I ran the code for the second file "s_0340_r_000001_1.dat" separately and got the correct number of data points and values in the table which is 326, the number of events in that file. So after running the code over these two files, I must have (274+326)=600 points in my concatenated wave. But instead of that, I am getting 652 points. I guess something wrong is happening when it is overwriting the existing waves, or something else, though not sure. I faced the same problem before also and tried to figure out the issue but couldn't. I hope now you will understand what is going on with the code. Any help will save my so many evening hours working on it.
Again thanks for your help.
#pragma TextEncoding = "MacRoman"
#pragma rtGlobals=3 // Use modern global access method and strict wave access.
//At first read a single file and then concatenate the data from each file
//Use the following command in the commmand window to read all the files
//LoadAndConcateNateAllFiles("")
//**************************
//READING A SINGLE FILE//
//*************************
Menu "Load Waves"
"Load One File.....", LoadOneNeutronDatFile("","")
"Load and Concatenate All Files in Folder.....", LoadAndConcateNateAllFiles("")
"Make Histogram for Proton Energy and Proton Time...", MakeHist()
"Make Two Dim Hisotgram....", MakeTwoDimHist()
End
static strConstant FileNameExtension=".dat"
//LoadOneNeutronDatFile(pathName, fileName)
//Creates the following wave: TimeStampForSpecAmp, TimeStampForPreAmp, ProtonEnergyFor_SpecAmp, ProtonTime_Spec, ProtonEnergyFor_PreAmp, ProtonTime_PreAmp
Function LoadOneNeutronDatFile(pathName, fileName)
String pathName //Name of an Igor symbolic path or ""
String fileName //Name of file or full path to file
//First we have to get a valid reference to a file
if (strlen(pathName) == 0 ||strlen(fileName) == 0)
//Display dialog looking for file
variable refNum
Open/D=2/R/F=FileNameExtension/P=$pathName refNum as fileName
filename=S_fileName //S_filename is set by Open/D
If (strlen(fileName)==0) //User Cancelled?
Print "LoadOneNeutronDatFile was Cancelled"
return -1
endif
endif
//String fullpath = S_filename
Variable i=0, j=0, k
Variable Vnam1, Vnam2, NumEvent, SpecMax, PreAmpMax, SpecTiMax, PreAmpTiMax, SpecAmpAvg, PreAmpAvg, Numerator, Denominator, ProtonEnergy
//String fullpath = S_filename
Open/P=$pathName/R refNum as fileName // fullpath
Printf "Reading from file \"%s\". Number of Event is: ", fileName//fullpath // full path is basically the s_filename
FStatus refNum // Used to read the file size
NumEvent = V_logEOF/8208 // Gives the total Number of Events where 8208 is the total byte of a single event
Print NumEvent
//Making Waves
Make/O/N=(NumEvent) TimeStampForSpecAmp
Make/O/N=(NumEvent) TimeStampForPreAmp
Make/O/N=(NumEvent) ProtonEnergyFor_SpecAmp
Make/O/N=(Numevent) ProtonTime_SpecAmp
Make/O/N=(NumEvent) ProtonEnergyFor_PreAmp
Make/O/N=(NumEvent) ProtonTime_PreAmp
for (k=0; k Specmax)
Specmax = Vnam2
SpecTiMax = j*10^-7
endif
Numerator = Numerator + Vnam2
endfor
//SpecAmpAvg = Numerator / Denominator
SpecAmpAvg = Numerator / 2048
ProtonEnergy = Specmax - SpecAmpAvg
ProtonEnergyFor_SpecAmp[k] = ProtonEnergy
ProtonTime_SpecAmp[k] = SpecTimax
//Reading 8 byte time stamp for PreAmp start
Fbinread/B=3/F=5 refNum, Vnam1
TimeStampForPreAmp[k] = Vnam1
//Reading 2 byte for PreAmp
for (j = 0; j < 2048; j+=1)
Fbinread/B=3/F=2 refNum, Vnam2
if (Vnam2 > PreAmpmax)
PreAmpmax = Vnam2
PreAmpTimax = j*10^-7
endif
Numerator = Numerator + Vnam2
endfor
PreAmpAvg = Numerator / 2048
ProtonEnergy = PreAmpmax - PreAmpAvg
ProtonenergyFor_PreAmp[k] = ProtonEnergy
ProtonTime_PreAmp[k] = PreAmpTiMax
//*********End OF ONE EVENT**********
endfor
Close refNum
return 0 //Signifies Success
End
//*************************************//
//READING ALL FILES IN SERIES 340//
//************************************//
Function LoadAndConCateNateAllFiles(pathName)
String pathName // Name of symbolic path or " " to get dialog
String fileName
Variable index=0
if (strlen(pathName) == 0) // If no path specified, create one
NewPath/O temporaryPath // This will put up a dialog. Also NewPath sets the variable V_flag to zero if the operation succeeded or to nonzero if it failed
If (V_flag != 0)
return -1 // User Cancelled
endif
pathName = "temporaryPath"
endif
String ProtonEnerFor_SpecAmpWaveList = ""
String ProtonTimeFor_SpecAmpWaveList = ""
Variable result
do // Loop through each file in the folder
fileName = IndexedFile($pathName, index, FileNameExtension) // If index is greater than or equal to zero, IndexedFile returns a STRING containing the name of the index th file in the folder specified by pathName which matches the file type or extension specified by fileTypeOrExtStr
If (strlen(fileName) == 0) // No more files?
break //Break out of the loop
endif
// Create data folder from file Name
String dfName = ParseFilePath(0, fileName, ":", 1, 0) // Provides the ability to manipulate file paths & to extract sections of file paths and gives the file name eg s_0340_r_000000_1.dat
//Print dfName // Prints the file name
dfName = RemoveEnding(dfName, ".dat")
//print dfName // Prints the file name without extension i.e. s_0340_r_000000_1
Variable LengthOfFileNameString = strlen(dfName) // Determines the length of the file name string
String FirstPart = dfName[3,5] // For the purpose of reading series 340 only
//Print FirstPart = dfName[3,5] i.e. 340
String LastPart = dfName[LengthOfFileNameString - 1, LengthOfFileNameString - 1] // To split the file name for the purpose of reading only _1.dat files
//Print LastPart = dfName[LengthOfFileNameString - 1, LengthOfFileNameString - 1] i.e. 1
if (stringmatch(FirstPart, "*340*") && stringmatch(LastPart, "1"))
result = LoadOneNeutronDatFile(pathName, fileName)
if (result == 0)
// Create wave reference for the wave named "ProtonEnergyFor_SpecAmp" & "ProtonTime_SpecAmp
Wave/Z TargetWavProEngFor_SpecAmp = ProtonEnergyFor_SpecAmp
if (WaveExists(TargetWavProEngFor_SpecAmp)) // The WaveExists function returns one if wave reference is valid or zero if the wave reference is null
ProtonEnerFor_SpecAmpWaveList = AddListItem(GetWavesDataFolder(TargetWavProEngFor_SpecAmp, 2), ProtonEnerFor_SpecAmpWaveList) // AddListItem function Returns listStr after adding itemStr. GetWavesDataFolder function is used when we want to create a wave in the data folder containing a wave passed as a parameter
else
Print "Couldn't find a wave named \"ProtonEnergyFor_SpecAmp\" in the current data folder."
endif
Wave/Z TargetWavProTimFor_SpecAmp = ProtonTime_SpecAmp
if (WaveExists(TargetWavProTimFor_SpecAmp))
ProtonTimeFor_SpecAmpWaveList = AddListItem(GetWavesDataFolder(TargetWavProTimFor_SpecAmp, 2), ProtonTimeFor_SpecAmpWaveList)
else
Print "Couldn't find a wave named \"ProtonTime_SpecAmp\" in the current data folder"
endif
if (ItemsInList(ProtonEnerFor_SpecAmpWaveList) >= 0)
Concatenate/O/NP ProtonEnerFor_SpecAmpWaveList, ProtonEnergyFor_SpecAmpConcat // Concatenates source waves to the destinated waves
endif
if (ItemsInList(ProtonTimeFor_SpecAmpWaveList) >= 0)
Concatenate/O/NP ProtonTimeFor_SpecAmpWaveList, ProtonTime_SpecAmpConcat
endif
endif
endif
// if (DataFolderExists(dfName))
// String prompt
// Sprintf prompt, "Data folder name '%s' exists. Click Yes to overwrite the existing data, No to cancel.", dfName
// DoAlert 1, prompt
// if (V_flag == 2)
// Print "Load cancelled. You can rename the existing data folder and try again."
// return -1 // User Cancelled
// endif
// SetDataFolder $dfName
// else
// NewDataFolder/O/S $dfName
// endif
//SetDataFolder :: //Back to the parent data folder
//KilldataFolder $dfName
Printf "Loaded file %d: \"%s\"\r", index, fileName
index += 1
while (1)
if (Exists("temporaryPath"))
Killpath temporaryPath
endif
Wave ProtonEnergyFor_SpecAmp, ProtonTime_Spec, ProtonEnergyFor_PreAmp, ProtonTime_PreAmp
// Sort ProtonEnergyFor_SpecAmp, ProtonTime_Spec, ProtonEnergyFor_PreAmp, ProtonTime_PreAmp, ProtonEnergyFor_SpecAmp
return 0 // Signifies success
End
//***************************************************************//
//MAKING HISTOGRAM FOR PROTON ENERGY AND PROTON TIME FOR SPECAMP//
//***************************************************************//
Function MakeHistogram()
WAVE ProtonEnergyFor_SpecAmpConCat, ProtonTime_SpecAmpConcat
// Histogram for proton energy for SpecAmp
Make/N=100/O ProtEnrgyFor_SpecAmpConCat_His; DelayUpdate
Histogram/C/B={0,300,100} ProtonEnergyFor_SpecAmpConCat, ProtEnrgyFor_SpecAmpConcat_His; DelayUpdate // binstatrt, binwidth, num of bins
Display ProtEnrgyFor_SpecAmpConcat_His
Modifygraph mode=5
ModifyGraph standoff=0
Label bottom "proton energy (a.u.)"
Label left "frequency of proton energ"
TextBox/C/N=text0/A=MC "proton energy histogram"
// Histogram for proton time for SpecAmp
Make/N=100/O ProtTime_SpecAmpConcat_His; DelayUpdate
Histogram/C/B={0,1.2e-6,100} ProtonTime_SpecAmpConcat, ProtTime_SpecAmpConcat_His; DelayUpdate
Display ProtTime_SpecAmpConcat_His
Modifygraph mode=5
ModifyGraph standoff=0
Label left "frequency of proton time"
Label bottom "proton time (s)"
TextBox/C/N=text0/A=MC "histogram for proton time"
End
//********************************
//MAKING TWO DIM JOINT HISTOGRAM
//********************************
Function TwoDimJointHist()
Wave ProtonEnergyFor_SpecAmpConcat, ProtonTime_SpecAmpConcat
JointHistogram/DEST=M_JointHistogram/BINS={100,100 } ProtonEnergyFor_SpecAmpConcat, ProtonTime_SpecAmpConcat
NewImage M_JointHistogram
End
I had another look at this. The original code as you have it works, I think the reason it fails is to do with reading from the file. So, I think it works because if you take the dat files ending 1_1 or 3_1 and make copies giving them other names e.g. 2_1, 4_1, and just have those in a directory. The code works. The concatenated waves are the right length and all is good.
So why doesn't it work on the whole folder? If you run LoadOneNeutronDatFile("","") and point it at 1_7, it fails to make the waves (I think), hence the load from directory option is failing.
I don't know enough about the files or the data structure to help further. The error is because of this part:
TimeStampForSpecAmp[k] = Vnam1
k is out of range. This is because:
NumEvent = V_logEOF/8208
which sets the size of the waves, is not a whole integer for this particular file.
Setting it to ceil(V_logEOF/8208) causes the read to crash because it reaches the end of file unexpectedly and throws a different error (ceil takes the integer above).
So, if you can fix this error I think the whole code will run on all files in your directory.
sjr51 wrote: I had another look at this. The original code as you have it works, I think the reason it fails is to do with reading from the file. So, I think it works because if you take the dat files ending 1_1 or 3_1 and make copies giving them other names e.g. 2_1, 4_1, and just have those in a directory. The code works. The concatenated waves are the right length and all is good.
So why doesn't it work on the whole folder? If you run LoadOneNeutronDatFile("","") and point it at 1_7, it fails to make the waves (I think), hence the load from directory option is failing.
I don't know enough about the files or the data structure to help further. The error is because of this part:
TimeStampForSpecAmp[k] = Vnam1
k is out of range. This is because:
NumEvent = V_logEOF/8208
which sets the size of the waves, is not a whole integer for this particular file.
Setting it to ceil(V_logEOF/8208) causes the read to crash because it reaches the end of file unexpectedly and throws a different error (ceil takes the integer above).
So, if you can fix this error I think the whole code will run on all files in your directory.
Hi,
Thank you very much for your explanations and suggestions. I have figured out the problem with the code. Now it is running well and giving the concatenated wave with the correct length.
Again thanks for your time.
The second part is the destwave.
January 9, 2018 at 01:40 pm - Permalink
Thank you very much for your reply. I switched the source and destination wave position as you said:
But the problem is not fixed. Now the code is running well but I am getting the concatenated waves without any values. I am attaching a screenshot of it. Could you have a look of this and let me know if you have any ideas? Thanks.
January 9, 2018 at 02:33 pm - Permalink
... where theConcatenatedWave is the wave that should contain the concatenated results.
Alternatively, you need to specify a different designation like this ...
--
J. J. Weimer
Chemistry / Chemical & Materials Engineering, UAH
January 9, 2018 at 02:41 pm - Permalink
Well, I used a different designation but the problem is not fixed. If I change like that, the code again doesn't run. I am working on it for hours to get the expected concatenated waves with correct length but each time having some issues. Anyways, thanks a lot.
January 9, 2018 at 03:07 pm - Permalink
When I run your code, it does the first two iterations (loads the 1st and 2nd dat files) and does the concatenation, on the next iteration it says that it loads the file but no waves have been created in the temporary data folder. This means that the wave references to ProtonEnergyFor_SpecAmpNew etc are NULL, when the concatenate command runs it gives Error: Expected Wave Reference for this reason. I didn't investigate why no waves are loaded on subsequent iterations. I did check if it was something to do with the file, but the code just fails on the next file if I take that particular file out of the directory. Sorry I haven't fixed the problem, but hopefully this brings you closer to a solution.
Switch the debugger on and check out the state of the tempdatafolder when it throws the error. Note that in the history window it says:
path: "Macintosh HD:Users:xxx:Downloads:ONeDatFile:"
Reading from file "s_0340_r_000001_1.dat". Number of Event is: 326
Loaded file 0: "s_0340_r_000001_1.dat"
Reading from file "s_0340_r_000003_1.dat". Number of Event is: 270
Loaded file 1: "s_0340_r_000003_1.dat"
Loaded file 2: "s_0340_r_000001_7.dat"
It looks like it doesn't read from s_0340_r_000001_7.dat so my guess is that the bug is with
January 10, 2018 at 12:08 am - Permalink
January 10, 2018 at 01:44 pm - Permalink
So why doesn't it work on the whole folder? If you run LoadOneNeutronDatFile("","") and point it at 1_7, it fails to make the waves (I think), hence the load from directory option is failing.
I don't know enough about the files or the data structure to help further. The error is because of this part:
k is out of range. This is because:
which sets the size of the waves, is not a whole integer for this particular file.
Setting it to ceil(V_logEOF/8208) causes the read to crash because it reaches the end of file unexpectedly and throws a different error (ceil takes the integer above).
So, if you can fix this error I think the whole code will run on all files in your directory.
January 10, 2018 at 11:39 pm - Permalink
Hi,
Thank you very much for your explanations and suggestions. I have figured out the problem with the code. Now it is running well and giving the concatenated wave with the correct length.
Again thanks for your time.
January 11, 2018 at 09:03 am - Permalink