Find Big Waves

In order to clear memory and reduce file sizes, it helps to be able to find the big waves, wherever they are.

// Returns the name of any waves exceeding 'minSize' points wherever they are found in any directory.  
Function FindBigWaves(minSize[,df,depth,noShow])
    Variable minSize // A minimum number of points, e.g. 100000
    variable depth // Used by the function recursion.  Ignore.  
    variable noShow // Don't show the table at the end.  
    dfref df // A folder to use as the top level of the search.  Default is root:  
   
    if(paramisdefault(df))
        dfref df=root:
    endif
    if(depth==0)
        NewDataFolder /O root:Packages
        NewDataFolder /O root:Packages:FindBigWaves
        dfref packageDF=root:Packages:FindBigWaves
        Make /o/T/n=0 packageDF:names
        Make /o/n=0 packageDF:sizes
    else
        dfref packageDF=root:Packages:FindBigWaves
    endif
    variable i
    wave /T/sdfr=packageDF names
    wave /sdfr=packageDF sizes
    variable points=numpnts(names)
    for(i=0;i<CountObjectsDFR(df,1);i+=1)
        wave w=df:$getindexedobjnamedfr(df,1,i)
        if(numpnts(w)>minSize)
            names[points]={GetWavesDataFolder(w,2)}
            sizes[points]={numpnts(w)}
            points+=1
        endif
    endfor
    i=0
    Do
        string folder=GetIndexedObjNamedfr(df,4,i)
        if(strlen(folder))
            dfref subDF=df:$folder
            FindBigWaves(minSize,df=subDF,depth=depth+1)
        else
            break
        endif
        i+=1
    While(1)
    if(depth==0)
        sort /R sizes,sizes,names
        if(!noShow)
            if(wintype("BigWaves"))
                dowindow /f BigWaves
            else
                edit /K=1 /N=BigWaves names,sizes as "Big Waves"
            endif
        endif
    endif
End
Don't forget the imagetransform compress operation, which will prevent you having to lose those big waves.
I am interested in finding the size in bytes and needed to account for multidimensional as well as different wave types. So I took the code posted above (thank you) and some code posted by Howard for file type sizing and added in multidimensional sizes.


// Returns the name of any waves exceeding 'minSize' points wherever they are found in any directory.  Modified from RGerkin's code.
Function FindBigWaves(minSize[,df,depth,noShow])
    Variable minSize // A minimum number of points, e.g. 100000
    variable depth // Used by the function recursion.  Ignore.  
    variable noShow // Don't show the table at the end.  
    dfref df // A folder to use as the top level of the search.  Default is root:  
 
    if(paramisdefault(df))
        dfref df=root:
    endif
    if(depth==0)
        NewDataFolder /O root:Packages
        NewDataFolder /O root:Packages:FindBigWaves
        dfref packageDF=root:Packages:FindBigWaves
        Make /o/T/n=0 packageDF:names
        Make /o/n=0 packageDF:sizes
    else
        dfref packageDF=root:Packages:FindBigWaves
    endif
    variable i
    wave /T/sdfr=packageDF names
    wave /sdfr=packageDF sizes
    variable points=numpnts(names)
    for(i=0;i<CountObjectsDFR(df,1);i+=1)
        wave w=df:$getindexedobjnamedfr(df,1,i)
        if(numpnts(w)>minSize)
            names[points]={GetWavesDataFolder(w,2)}
            sizes[points]={sizeOfWave(w)}
            points+=1
        endif
    endfor
    i=0
    Do
        string folder=GetIndexedObjNamedfr(df,4,i)
        if(strlen(folder))
            dfref subDF=df:$folder
            FindBigWaves(minSize,df=subDF,depth=depth+1)
        else
            break
        endif
        i+=1
    While(1)
    if(depth==0)
        sort /R sizes,sizes,names
        if(!noShow)
            if(wintype("BigWaves"))
                dowindow /f BigWaves
            else
                edit /K=1 /N=BigWaves names,sizes as "Big Waves"
            endif
        endif
    endif
End


// posted by Howard in the forums
Function sizeOfType(inType)
    Variable inType
 
    Variable size=1
    if(inType & 0x01)
        size*=2
    endif
 
    if(inType & 0x02)
        size*=4
    elseif(inType & 0x04)
        size*=8
    elseif(inType & 0x10)
        size*=2
    elseif(inType & 0x20)
        size*=4
    elseif(inType==0)
        size=nan
    endif
    return size
End

//My addition to account for multidimensional waves

Function SizeOfWave(w)
    wave w
   
    variable index, wavesize
    wavesize =1
    For(index=0;index<wavedims(w);index +=1)
        wavesize *=DimSize(w,index)
    endfor
    wavesize *=sizeOfType(wavetype(w))
    return wavesize
End
@hegedus:

Your snipped includes this function:
Function SizeOfWave(w)
    wave w
 
    variable index, wavesize
    wavesize =1
    For(index=0;index<wavedims(w);index +=1)
        wavesize *=DimSize(w,index)
    endfor
    wavesize *=sizeOfType(wavetype(w))
    return wavesize
End


This can be rewritten as:
Function SizeOfWave(w)
    wave w
    wavesize =numpnts(w) * sizeOfType(wavetype(w))
    return wavesize
End


Also, note that this will not really work for text waves, data folder reference waves, or wave reference waves. Each point of a text wave can take up a different number of bytes, depending on the length of the string that is stored in that point. You could call strlen() on each point of a text wave to get a pretty good estimate of the number of bytes the wave will take up. The SizeOfWave function also doesn't account for the size of the wave's header, but every wave header is the same size and is usually relatively small compared to the size of the wave's data.
Here is a version for IP7 which uses the SIZEINBYTES value from WaveInfo and works with wave reference waves as well.
I also changed the first parameter to mean size in MB instead of number of points.

/// Returns the name of any waves exceeding 'minSize' points wherever they are found in any directory.
///
/// Modified from RGerkin's and hegedus code.
Function FindBigWaves(minSizeInMB[,df,depth,noShow])
    Variable minSizeInMB // Minimum size in MB, e.g. 100
    variable depth // Used by the function recursion.  Ignore.  
    variable noShow // Don't show the table at the end.  
    dfref df // A folder to use as the top level of the search.  Default is root:  
 
    if(paramisdefault(df))
        dfref df=root:
    endif
    if(depth==0)
        NewDataFolder /O root:Packages
        NewDataFolder /O root:Packages:FindBigWaves
        dfref packageDF=root:Packages:FindBigWaves
        Make /o/T/n=0 packageDF:names
        Make /o/n=0 packageDF:sizes
    else
        dfref packageDF=root:Packages:FindBigWaves
    endif
    variable i
    wave /T/sdfr=packageDF names
    wave /sdfr=packageDF sizes
    variable points=numpnts(names)
    for(i=0;i<CountObjectsDFR(df,1);i+=1)
        wave w=df:$getindexedobjnamedfr(df,1,i)
        variable size = sizeOfWave(w)
        if(size > minSizeInMB)
            names[points]={GetWavesDataFolder(w,2)}
            sizes[points]={size}
            points+=1
        endif
    endfor
    i=0
    Do
        string folder=GetIndexedObjNamedfr(df,4,i)
        if(strlen(folder))
            dfref subDF=df:$folder
            FindBigWaves(minSizeInMB,df=subDF,depth=depth+1)
        else
            break
        endif
        i+=1
    While(1)
    if(depth==0)
        sort /R sizes,sizes,names
        if(!noShow)
            if(wintype("BigWaves"))
                dowindow /f BigWaves
            else
                edit /K=1 /N=BigWaves names,sizes as "Big Waves"
            endif
        endif
    endif
End

Function SizeOfWave(wv)
    wave/Z wv

    variable i, numEntries
   
    variable total = NumberByKey("SIZEINBYTES", WaveInfo(wv, 0))

    if(WaveType(wv, 1) == 4)
        WAVE/WAVE temp = wv
        numEntries = numpnts(wv)
        for(i = 0; i < numEntries; i += 1)
            WAVE/Z elem = temp[i]
            if(!WaveExists(elem))
                continue
            endif
            total += SizeOfWave(elem)
        endfor
    endif

    return total / 1024 / 1024
End

Forum

Support

Gallery

Igor Pro 8

Learn More

Igor XOP Toolkit

Learn More

Igor NIDAQ Tools MX

Learn More