Add WaveMinMax

In many of my code bases I call both WaveMin and WaveMax to calculate the minimum and maximum of a wave. As that is usually faster than calling WaveStats.

But from an algorithmic point of view if you want to calculate both you would only have to traverse the wave elements once. Also C++ learned that a while ago, see std::minmax.

So my request is to add a function WaveMinMax which uses multiple return value syntax to return both the minimum and the maximum of a wave.

You know that WaveStats/M=0 does essentially that, yes? /M=0 skips the slower calculations.

Yes I do know that. But my feeling was that this is still slower.

 

And some testing with

 

Function MeasureStatsVsExtrema()

    SetRandomSeed 1

    variable maxExp = 24
    variable numRuns = 10
    variable i, j, elapsedTime
    variable refTime, ret, wmin, wmax

    Make/D/O/N=(maxExp, 3) result = NaN
   
    SetDimLabel 1, 0, stats, result
    SetDimLabel 1, 1, extrema, result
    SetDimLabel 1, 2, mtrxop, result
   
    SetScale/I x, 0, 2^maxExp*4, "byte", result
    SetScale d, 0, 0, "s", result

    for(i = 1; i < maxExp; i += 1)
        Make/FREE/N=(2^i)/R data = trunc(enoise(1e6))
       
        refTime = stopmstimer(-2)
        for(j = 0; j < numRuns; j += 1)
            WaveStats/Q/M=0 data
        endfor
        elapsedTime = stopmstimer(-2) - refTime
        result[i][%stats] = elapsedTime/1e6

        refTime = stopmstimer(-2)
        for(j = 0; j < numRuns; j += 1)
            wmin = WaveMin(data)
            wmax = WaveMax(data)
        endfor
        elapsedTime = stopmstimer(-2) - refTime
        result[i][%extrema] = elapsedTime/1e6

        refTime = stopmstimer(-2)
        for(j = 0; j < numRuns; j += 1)
            matrixop/O mmin = minVal(data)
            matrixop/O mmax = maxVal(data)
        endfor
        elapsedTime = stopmstimer(-2) - refTime
        result[i][%mtrxop] = elapsedTime/1e6
       
        AbortOnValue !(V_min == wmin), 125
        AbortOnValue !(V_max == wmax), 125
        AbortOnValue !(V_min == mmin[0]), 125
        AbortOnValue !(V_max == mmax[0]), 125

    endfor
   
    Display/K=1
    AppendToGraph result[][0]/TN=stats
    AppendToGraph result[][1]/TN=extrema
    AppendToGraph result[][2]/TN=matrixOP
    ModifyGraph mode=4
    ModifyGraph log=1
    ModifyGraph rgb(extrema)=(1,39321,19939)
    ModifyGraph rgb(matrixOP)=(0,0,65535)
    Legend
End

 

gives the attached graph.

 

So WaveMin/WaveMax is the fastest.

 

Graph1.png

That's interesting. Your bottom axis label says, "Number of Elements" but the tick label says, "10 MByte". Which is it?

What sort of hardware and OS are you using for this test?

I just looked at our source code, and the underlying code actually gets the min and max together, then WaveMin and WaveMax return the one that is asked for. So I started implementing WaveMinMax for Igor 9 and immediately ran into a problem. How do I return the result? The result has two values... I suppose we could return a complex number, but that's an ugly hack.

I've adapted the code and graph. The x units are bytes.

I'm on Windows 7 with an i7-3930K@3.2GHz and 64GB RAM.

How do I return the result? The result has two values... I suppose we could return a complex number, but that's an ugly hack.

So mutiple return syntax is not available for builtin functions?

[Note: I wrote this response before John's response but forgot to post it]

First of all, it's technically WaveStats /M=1, not /M=0, though it appears that /M=0 gives the same results as /M=1. But that's not how the flag is documented.

Also, the documentation for WaveMin says "For a floating-point wave, WaveMin runs about three times faster than getting the same information using WaveStats. For an integer wave, WaveMin runs about ten times faster than WaveStats. The advantage may not hold for short waves.". A similar statement is present for WaveMax. I'm not sure when that was written, but it's not completely accurate at least for large waves. I believe that's because WaveStats is automatically multithreaded and for large waves that improves its performance substantially. WaveMin and WaveMax are not automatically multithreaded.

I don't believe that adding a built-in function that uses the new multi-return syntax is technically possible, and I suspect that it would require a lot of work to make it possible. As far as I understand, built-in functions can't take parameters by reference, and that's how user-defined multi-return functions are implemented under the hood.

By the way, you're not handling the X axis correctly. You need to do a y vs. x plot because the increment between your x values is not actually uniform. Here's corrected code that I used:

Function MeasureStatsVsExtrema()

    SetRandomSeed 1

    variable maxExp = 24
    variable numRuns = 10
    variable i, j, elapsedTime
    variable refTime, ret, wmin, wmax, wsum

    Make/D/O/N=(maxExp, 3) result = NaN
    Make/D/O/N=(maxExp-1) numElements
   
    SetDimLabel 1, 0, stats, result
    SetDimLabel 1, 1, extrema, result
    SetDimLabel 1, 2, mtrxop, result
   
    SetScale d, 0, 0, "s", result

    for(i = 1; i < maxExp; i += 1)
          numElements[i-1] = 2^i
        Make/FREE/N=(2^i)/R data = trunc(enoise(1e6))
       
        refTime = stopmstimer(-2)
        for(j = 0; j < numRuns; j += 1)
            WaveStats/Q/M=1 data
        endfor
        elapsedTime = stopmstimer(-2) - refTime
        result[i][%stats] = elapsedTime/1e6

        refTime = stopmstimer(-2)
        for(j = 0; j < numRuns; j += 1)
            wmin = WaveMin(data)
            wmax = WaveMax(data)
            wsum = sum(data)
        endfor
        elapsedTime = stopmstimer(-2) - refTime
        result[i][%extrema] = elapsedTime/1e6

        refTime = stopmstimer(-2)
        for(j = 0; j < numRuns; j += 1)
            matrixop/O mmin = minVal(data)
            matrixop/O mmax = maxVal(data)
        endfor
        elapsedTime = stopmstimer(-2) - refTime
        result[i][%mtrxop] = elapsedTime/1e6
       
        AbortOnValue !(V_min == wmin), 125
        AbortOnValue !(V_max == wmax), 125
        AbortOnValue !(V_min == mmin[0]), 125
        AbortOnValue !(V_max == mmax[0]), 125

    endfor
   
    Display/K=1
    AppendToGraph result[][0]/TN=stats vs numElements
    AppendToGraph result[][1]/TN=extrema vs numElements
    AppendToGraph result[][2]/TN=matrixOP vs numElements
    ModifyGraph mode=4
    ModifyGraph log=1
    ModifyGraph rgb(extrema)=(1,39321,19939)
    ModifyGraph rgb(matrixOP)=(0,0,65535)
    ModifyGraph tickUnit(bottom)=1
    Label bottom, "Number of elements"
    Legend
End

Running the above gives Graph9.png.

The strange shape of the red line is due to the fact that WaveStats is automatically multithreaded. If I first execute

MultiThreadingControl setMode=0

and the I rerun the test, I get results in Graph10.png.

Graph9.png Graph10.png

Adam: You are right, I did not handle the x axis correctly.

So If I really want to speed that up I would need to use a paralell implementation for minmax. This starts to get serious work.

For now can we get the sentence
 

The advantage may not hold for short waves.

from WaveMin/WaveMax removed as that is cleary not the case?

Thanks Jim, John and Adam.

Are you actually experiencing performance problems? I was under the impression that you were looking for cleaner code and not so much performance improvements. Yes, it would of course be faster if we had a WaveMinMax that returned both the minimum and maximum, but using WaveMin followed by WaveMax (for short waves) is still quite fast.

The directive is to get it as fast as possible with reasonable effort. Having cleaner code would be a nice side effect as well.