Using StatsInvQCDF to interpret results of StatsNPMCTest

Share icon

Tue, 03/19/2024 - 02:03 pm

Hello.  Any Igor stats experts out there?  I'm using StatsNPMCTest/DHW to compare three waves using Dunn's test for multiple comparisons.  This operation returns a wave M_NPMCDHWResults, which  reports differences in ranks between each pair of waves and the estimated standard error.  If I understand right, this yields a Q statistic (diff/SE) for each comparison.  Statistical significance is determined by comparing Q against a criterion level Qc.  If Q < Qc, then there is a significant difference between that pair of groups.

My problem is that I really want a P value, which should be a matter of calculating where the Q statistic value lies on the Q distribution using StatsQCDF, but I am having trouble reconciling the values from these two functions.  Right now, I'm trying to use the inverse function StatsInvQCDF to verify the value for Qc.  This is because I doubt my understanding, not because I doubt Igor.

For example, for one pair of waves, M_NPMCDHWResults reports Q = 2.52039, and Qc = 2.39398.  The value of Q is confirmed by a calculator I found online, and the value of Qc matches a table I have of the Q distribution for k = 3 groups for alpha = 0.05 (Glantz's Primer of Biostatistics, 4th Ed, 1991, table 10-10).  So, StatsNPMCTest produces values that make sense for my dataset.  Glantz's table doesn't mention degrees of freedom.

The problem is that StatsInvQCDF reports a very different value from this Qc:

print StatsInvQCDF (1 - 0.05, 1, 3, 62) // cdf, r = # groups, c = # treatments, df = degrees of freedom

This statement prints 3.39589, which is not even close.  I thought I might have misunderstood how to represent alpha, so I also tried alpha = 0.025 and 0.1, and those are also way off (Q = 3.7939 or 2.95704, respectively). I also tried modifying DF to no avail.  The closest I can get is if I set alpha to 0.21, which isn't recognizably similar to 0.05.  StatsQCDF is consistent with StatsInvQCDF, so I don't want to proceed till I figure out what I'm doing wrong.

Is anyone out there able to help me sort this out?  I would like to make sure that StatsNPMCTest, StatsQCDF, and StatsInvQCDF all agree with each other.  Thanks for your help.

-Matthew

Hello Matthew:

To get an idea about the inverse and the CDF execute:

•make/o ddd
•SetScale/I x 0,5,"", ddd
•ddd=StatsQCDF(x,1, 3, 62)
•display ddd
•findlevel ddd,0.95
  V_Flag= 0; V_LevelX= 3.3961; V_rising= 1;

So the CDF and InvCDF do match.

Now the critical value reported in the DWH case is different because it is computed for normal deviates therefore based on the inverseERF().  Since you have access to the explicit CDF I recommend using it directly.

Note that it is usually helpful if you can send a copy of the experiment file to support@wavemetrics.com so that we can see exactly what you are doing and the corresponding code is fact executing.  In the case of StatsNPMCTest the calculation may use different distributions depending on the exact flag combination.  In particular, some of the tests use the Q distribution and some the Q' distribution (see the differences between statsQCDF() and statsQpCDF().

 

AG

 

March 19, 2024 at 04:29 pm - Permalink

Sorry to be thick.  If I want to turn the Q statistic from M_NPMCDHWResults into a P value, which explicit CDF should I be using?  It doesn't seem to be StatsQCDF, because that gives me:

•print 1 - statsqcdf (2.52039, 1, 3, 62)
  0.183964

I'm expecting a result less than 0.05, because the Conclusion in M_NPMCDHWResults was 0, i.e. significant.

Similarly, if I use Qc,

•print 1 - statsqcdf (2.39398, 1, 3, 62)
  0.216035

I'm expecting that to evaluate to 0.05. This means to me that either I'm using the wrong cdf, or I'm using the cdf incorrectly.  I'm grateful for your help.

-Matthew

March 19, 2024 at 06:39 pm - Permalink

Problem solved!  It was indeed my lack of understanding that StatsQCDF was not the right function to use for this purpose.  I got a rare, behind-the-scenes tour of an annotated version of Zar's Biostatistics led by AG, that had the formula I needed.  I have now turned that into a function that translates the reported values of Q (for Dunn's test for multiple comparisons = StatsNPMCTest/DHW) to a P value.  While I was at it, I also figured out the equivalent function for Q' (for Dunn's test for multiple comparisons against a single control = StatsNPMCTest/CIDX).

function StatsDunnQ2P (variable DunnQ, variable numgroups)
    // converts Q statistic from Dunn's test to a P value (k = number of groups)
    // Q is for all multiple comparisons
    return (numgroups * (numgroups - 1) / 2 * (1 - erf (DunnQ / sqrt (2))))
end
function StatsDunnQp2P (variable DunnQp, variable numgroups)
    // converts Q' statistic from Dunn's test to a P value (k = number of groups)
    // Q' is for multiple comparisons against a single control
    return ((numgroups - 1) * (1 - erf (DunnQp / sqrt (2))))
end

I haven't actually used StatsNPMCTest/CIDX yet, so I haven't double-checked how it works or reports Q'.  I'm saving that fun for another day.

Thanks very much to AG for taking the time to help, and for figuring these out in 2006!  I have verified the functions' accuracy by recapitulating tables 10-10 (StatsDunnQ2P) and 10-11 (StatsDunnQp2P) in Glantz's Primer of Biostatistics 4th Ed.  If anyone spots an error, please let me know.

-Matthew

March 21, 2024 at 08:36 am - Permalink

Yes!  In all seriousness, I understand that statistics has limitations, but some journals expect exact P values.  I gather there is also a saying around there about selling rope, and my original request probably fits in that category.

-Matthew

March 21, 2024 at 09:45 am - Permalink