Checking memory usage by Igor
euaruksakul
Sat, 11/24/2018 - 08:13 pm
Hello
We are using Igor for analysis of hyperspectral data, each set being about 300-400 MB in size. After loading many scans, the size of an Igor experiment can easily go up to a few GB, and that can cause problems with the computer sometimes. Is there a command in Igor to check for the current size of Igor experiment, so that our function automatically warns users when it becomes larger than a set limit?
We have a function to 'delete raw data' to lighten the RAM usage after finishing data reduction of each scan, but that has to be done manually (after users decide the data reduction is successful).
Thank you!
Best
Chanan
It would be helpful to know what version of Igor you are using. My guess is Igor Pro 6.xx, 32-bit version. If you were running the 64-bit version of Igor Pro 8, you would probably not need to do any memory checking.
The only reliable way to check this is to try to create a wave of a certain size. For example:
Variable numberOfMegabytes
Variable numBytes = 1048576*numberOfMegabytes
Make/B/U/O/N=(numBytes)/FREE temp
Variable err = GetRTError(1) // Get error if any, and clear it
return err
End
November 25, 2018 at 04:36 am - Permalink
I would check the free memory at good points, and if that is too low do your cleanup. I've posted https://www.wavemetrics.com/code-snippet/get-free-memory-gb for getting the free memory.
November 26, 2018 at 08:31 am - Permalink
In Igor 7 and later, the string returned by WaveInfo contains the size of the wave in the SIZEINBYTES keyword. You could walk your data tree to calculate the total amount of data used by waves in your experiment. Or, if the experiment is saved on disk, you could look at the size of the file itself with GetFileFolderInfo. Igor experiments don't compress the data contained in the experiment, so the size of an experiment file should roughly correlate with the amount of memory used when an experiment is loaded.
November 26, 2018 at 09:13 am - Permalink
With SIZEINBYTES keyword, I made a code for folder size calculations.
//CREATED: 09.02.2021
//DESCRIPTIONs: Calculate folder size (all waves combined) including wave in subfolders.
String FolderName //Full path of a folder to check **Without ":" at the end
Variable Print_Flg //Show size of each subfolders
String List_FoldersInFolder=""
String CurrentFolder=GetDataFolder(1)
SetDataFolder $(FolderName+":")
List_FoldersInFolder=StringByKey("FOLDERS",DataFolderDir(1),":")
String SubFolder_FName=""
Variable i=0
Variable TotalSize=SLP_Util_FolderWaveSizeMBytes(FolderName,0)
If (Print_Flg)
Print "Waves in "+FolderName+": "+num2str(TotalSize)+" MBytes" //Show combined size of all waves in the specified folder.
EndIf
Variable SubFolderSize=0
For (i=0;i<itemsinlist(List_FoldersInFolder,",");i+=1)
SubFolder_FName=FolderName+":"+StringFromList(i,List_FoldersInFolder,",")
//TotalSize+=SLP_Util_FolderWaveSizeMBytes(SubFolder_FName,0) //All waves in this Subfolder
SubFolderSize=SLP_Util_FolderTotalSizeMBytes(SubFolder_FName,0) //Recursive - look inside the subfolder
TotalSize+=SubFolderSize
If (Print_Flg)
Print SubFolder_FName+": "+num2str(SubFolderSize)+" MBytes"
EndIf
EndFor
SetDataFolder $CurrentFolder
If (Print_Flg)
Print "Total size = "+num2str(TotalSize)+" MBytes"
EndIf
Return TotalSize
End
Function SLP_Util_FolderWaveSizeMBytes(FolderName,Print_Flg)
//CREATED: 09.02.2021
//DESCRIPTIONs: Return the total size of all waves in a folder in MBytes (not including waves in subfolders)
String FolderName //Full path of a folder **Without ":" at the end
Variable Print_Flg //Flag to print out results
String List_WavesInFolder=""
String CurrentFolder=GetDataFolder(1)
setdatafolder $(FolderName+":")
List_WavesInFolder=StringByKey("WAVES",DataFolderDir(2),":")
Variable i=0
String Name_Wave
String Info_Wave
Variable Size_Wave
Variable Size_Sum=0
For (i=0;i<itemsinlist(List_WavesInFolder,",");i+=1)
Name_Wave=StringFromList(i,List_WavesInFolder,",")
Info_Wave=WaveInfo($Name_Wave,0)
Size_Wave=str2num(StringByKey("SIZEINBYTES",Info_Wave,":"))/10^6 //Size in MBytes
If (Print_Flg)
Print Name_Wave+" Size = "+num2str(Size_Wave)+" MBytes"
EndIF
Size_Sum+=Size_Wave
EndFor
If (Print_Flg)
Print "Total size = "+num2str(Size_Sum)+" MBytes"
EndIF
SetDataFolder $CurrentFolder
Return Size_SUM
End
February 12, 2021 at 11:38 am - Permalink