Calculation of 10th Percentile and Insert Value to Wave

Share icon

Thu, 06/02/2016 - 10:48 pm

Hi everyone,

I am trying to calculate the 10th percentile for each set of hourly data (Igor's Percentile and Box Plot procedure does not calculate 10th percentile).

I use the following procedure to first insert all the data from the same hour to the wave 'TempConcWave', then use the function fWavePercentile("TempConcWave", "10", "Q", 0, 0, 0) to get the 10th percentile for each hour and insert the hourly value to the wave 'Quartile_10' and ultimately insert the values to the second column of 'ConcWaveStatsTemp' table.

However, I am stuck at the wave 'Quartile_10' as I got 174 values instead of 24 values (one for each hour). Can you please take a look at what is wrong? Thank you so much.

<br />
        <br />
        Function CalPercentile(ConcWave, HourWave)<br />
    Wave ConcWave<br />
    Wave HourWave<br />
    Variable HourNum<br />
    <br />
    <br />
    For (HourNum = 0; HourNum < 24; HourNum += 1)<br />
<br />
        Make/O/D/N = (24, 5) ConcWaveStatsTemp // make a matrix to start the stats number<br />
<br />
        Make/O/D/N=0 TempConcWave  //refresh the temp wave<br />
        <br />
        Make/O/D/N=(24,1) Quartile_10<br />
        <br />
        Variable i<br />
<br />
        For (i = 0; i < (numpnts(Concwave)); i += 1)<br />
<br />
            If (HourWave[i] == HourNum)  // for the number of hour, put the concentration values to the temp wave<br />
                Insertpoints 0, 1, TempConcWave<br />
                TempConcWave[0] = ConcWave[i]       <br />
            <br />
            fWavePercentile("TempConcWave", "10", "Q", 0, 0, 0)<br />
            Insertpoints 0, 1, Quartile_10<br />
            Quartile_10[0] = ConcWave[i]<br />
            <br />
            Endif<br />
            <br />
        Endfor<br />
        <br />
        <br />
        WaveStats/Q TempConcWave //perform calculation needed and assign the values to the waves<br />
<br />
        ConcWaveStatsTemp[HourNum][0] = V_avg<br />
<br />
        StatsQuantiles/Q TempConcWave<br />
                                <br />
        ConcWaveStatsTemp[HourNum][2] = V_Q25   <br />
        <br />
        ConcWaveStatsTemp[HourNum][3] = V_Median    <br />
                <br />
        ConcWaveStatsTemp[HourNum][4] = V_Q75   <br />
    <br />
            <br />
            <br />
    Endfor<br />
    <br />
    //Duplicate wave with name indicating species and kill temp wave<br />
    Duplicate ConcWaveStatsTemp, $"ConcWaveStats" + "_" + NameOfWave(ConcWave)      //The NameOfWave function returns a string containing the name of the specified wave.<br />
    KillWaves ConcWaveStatsTemp     //The KillWaves operation destroys the named waves<br />
    <br />
        End<br />
    <br />
<pre><code class="language-igor"></span>
I may not be understanding your function clearly, but.... Don't you want

fWavePercentile("TempConcWave", "10", "Q", 0, 0, 0)
Insertpoints 0, 1, Quartile_10
Quartile_10[0] = ConcWave[i]


to be placed outside of the inner for/endfor loop? It seems that the percentile should be calculated only once when all data for the given hour have been collected in TempConcWave.

Additionally, Quartile_10 should get the return value from fWavePercentile not ConcWave[i]. Finally, you have already defined Quartile_10 to have 24 rows. I see no need to insert additional points into it. Finally-finally, you define Quartile_10 to be a 2D wave, but with only 1 column with the flag "\N=(24,1)", perhaps you only need "\N=24"?

June 3, 2016 at 05:31 am - Permalink

wkial wrote:
(Igor's Percentile and Box Plot procedure does not calculate 10th percentile).

Just enter "10;" in the list of percentiles.

But in fact, fWavePercentile() isn't designed for computing percentiles for a single wave, it's designed for a situation where you have a bunch of waves and you need to calculate statistics on all the waves together. On the other hand, you *can* use it for a single wave, and you'll get a one-point wave for each percentile in the list.

I made a wave with "random" numbers: make/n=1000 junk=enoise(1), then I set up the Wave Percentiles panel with Select Waves by Name, Name template: Junk, Replicates are Columns, and this list of percentiles:
0;10;25;50;75;90;100
I got a single one-point wave for each of the percentiles in the list.

With your call to fWavePercentile(), you should get a single one-point wave called "Q_10" containing the tenth-percentile value.

John Weeks
WaveMetrics, Inc.
support@wavemetrics.com

June 3, 2016 at 04:58 pm - Permalink

johnweeks wrote:


With your call to fWavePercentile(), you should get a single one-point wave called "Q_10" containing the tenth-percentile value.



Hi johnweeks,

Thanks for your reply. Regarding your comment, I do get the single one-point wave called "Q_10", but what I wanted to do is to calculate the 10th percentile of TempConcWave for each hour of the day, and then insert the 24 values to the wave "Quartile_10". However, I have been getting "0" values for the "Quartile_10" wave.

June 3, 2016 at 06:58 pm - Permalink

jtigor wrote:
I may not be understanding your function clearly, but.... Don't you want

fWavePercentile("TempConcWave", "10", "Q", 0, 0, 0)
Insertpoints 0, 1, Quartile_10
Quartile_10[0] = ConcWave[i]


to be placed outside of the inner for/endfor loop? It seems that the percentile should be calculated only once when all data for the given hour have been collected in TempConcWave.

Additionally, Quartile_10 should get the return value from fWavePercentile not ConcWave[i]. Finally, you have already defined Quartile_10 to have 24 rows. I see no need to insert additional points into it. Finally-finally, you define Quartile_10 to be a 2D wave, but with only 1 column with the flag "\N=(24,1)", perhaps you only need "\N=24"?


Hi jtigor,

Thanks for your suggestions. I have changed the code as below, but Igor did not seem to understand that I want to put the fWavePercentile values for each hour of the day into the wave "Quartile_10", as I keep getting "0" values for the wave "Quartile_10".

   For (HourNum = 0; HourNum < 24; HourNum += 1)<br />
<br />
        Make/O/D/N = (24, 5) ConcWaveStatsTemp // make a matrix to start the stats number<br />
<br />
        Make/O/D/N=0 TempConcWave  //refresh the temp wave<br />
        <br />
        Make/O/D/N=24 Quartile_10<br />
        <br />
        Variable i<br />
<br />
        For (i = 0; i < (numpnts(Concwave)); i += 1)<br />
<br />
            If (HourWave[i] == HourNum)  // for the number of hour, put the concentration values to the temp wave<br />
                Insertpoints 0, 1, TempConcWave<br />
                TempConcWave[0] = ConcWave[i]       //?<br />
            <br />
            Endif       <br />
        <br />
        Endfor<br />
        <br />
        fWavePercentile("TempConcWave", "10", "Q", 0, 0, 0)<br />
        Insertpoints 0, 1, Quartile_10<br />
        <br />
            <br />
            Endfor<br />
    </span>

June 3, 2016 at 07:10 pm - Permalink

The calculated percentile value from wave Q_10 was not being transferred to Quartile_10. See code below; I think this may work for you.

Try this:
Function CalPercentile(ConcWave, HourWave)
    Wave ConcWave
    Wave HourWave
    Variable HourNum


    Make/O/D/N=24 Quartile_10
    Make/O/D/N = (24, 5) ConcWaveStatsTemp // make a matrix to start the stats number

 For (HourNum = 0; HourNum < 24; HourNum += 1)
 
        Make/O/D/N=0 TempConcWave  //refresh the temp wave
 
        Variable i
 
        For (i = 0; i < (numpnts(Concwave)); i += 1)
 
            If (HourWave[i] == HourNum)  // for the number of hour, put the concentration values to the temp wave
                Insertpoints 0, 1, TempConcWave
                TempConcWave[0] = ConcWave[i]       //This is ok
 
            Endif      
 
        Endfor
 
        fWavePercentile("TempConcWave", "10", "Q", 0, 0, 0)
        Wave Q_10
        Quartile_10[HourNum] = Q_10[0]

    Endfor

End

June 6, 2016 at 05:49 am - Permalink