Wilcoxon Signed-Rank Test - Validated by other implementations?

[DonRoth]

I am using the Wilcoxon Signed-Rank Test for Hypothesis testing on paired samples.  I am having some issues with this function, which is new to LabVIEW 8.2.

First, I wanted to know if someone has used this LabVIEW implementation and has been able to get the same answers (reject null hypothesis & p-value answers) using implementations either self-coded or in other language subVIs.  I guess another way to check on this is to do the calculations manually and look in the appropriate statistical tables.  I have had a few instances where I am selecting nearly identical regions of images (and thus the means are nearly identical), using signficance level of 0.05, and still getting a rejection of null hypothesis (null hypothesis = the two variable populations have a common mean).  Additionally, the help on this VI states that "If p value is greater than significance level, reject null hypothesis returns FALSE. "  I have many times been getting p-value > significance level (and also at times much greater than 1) while the reject null hypothesis returns TRUE.

Another issue that I have seen is that changing the signficance level to 0.1 from 0.05 would change the p-value, but then changing it back to 0.05 did not change it back to the original p-value.  I am running my implementation in an event-driven subVI with a value-change event on the signficance level and have verified the event to execute the Wilcoxon function when it is supposed to, and also confirmed no errors coming out of the function.  It is also not possible it seems to get into the Wilcoxon function diagram while running the overall code since when one selects the pair-samples polymorphic instance, the function front panel and diagram do not indicate running.

Any help is appreciated.

Sincerely,

Don

[altenbach]

Sorry, I never used this function and I am not familiar with the math.

---

@DonRoth wrote:
It is also not possible it seems to get into the Wilcoxon function diagram while running the overall code since when one selects the pair-samples polymorphic instance, the function front panel and diagram do not indicate running.

---

I have no problem opening the diagram, see it running if the calling program is running, and see all values appear on its front panel, and even place working probes deep inside subVIs. (Make sure you are opening the right instance: First select the polymorphic instance, then open the front panel).

---

@DonRoth wrote:
Another issue that I have seen is that changing the signficance level to 0.1 from 0.05 would change the p-value, but then changing it back to 0.05 did not change it back to the original p-value. 

---

I cannot find anything in the function code that would cause this (e.g. no uninitialized shift registers, etc.) All code is pretty linear and deterministic. Make sure you are reading the new value from a terminal within the event structure, else you might get a stale value for p.

Can you make a small demo containing some sample data that shows the behavior?

Message Edited by altenbach on 10-29-2006 10:31 AM

[DonRoth]

Well here is a very quick test program.

It turns out that I the p-values I saw were less than 1, I just did not have my indicator lengthened sufficiently to see the scientific formatting exponent.  So I think we are fine with that.

Also, as you stated, I must not have had the proper instance open for the Wilcoxon test function because today I could see the front panel and block diagram running during main run.

However, and maybe it is something obvious since we all have our days, I still do not see a change in p-value after the FIRST CHANGE in confidence level.  Changing from 0.05 --> 0.1, a change is seen.  Changing back to 0.05, I do not see a change back to the original p-value.

Check it out.

Sincerely,

Don

[DonRoth]

ps. you can substitute and two distributions (images) of your choice (although for testing value change on confidence level this should not make a difference.

[Chris_Cilino]

Hey DonRoth,

I noticed that you were passing your data to two separate instances of the Test vi. You're also running reshape in both instances. I don't know if the data is equivalent so I decided to use a shift register and pass the data from the "distribution statists boolean" case, controls 1 and 2 after they'd already gone through the sgl to dbl conversion to the significance level case and bypass the local variables and calculations. The result was that the p-value never changes regardles of the significance level set.

So something  changes from the initial computation to the subsequent computations in the "significance leve" case.

Chris C

Message Edited by Chris_C. on 10-30-2006 06:19 PM

Message Edited by Chris_C. on 10-30-2006 06:19 PM

[DonRoth]

Yesterday I just threw together a quick example with no regard for proper programming style and in my haste interchanged sample sets X and Y in the two different instances of the Wilcoxon Signed Rank Test in the Wilcoxon_test.VI. (In fact this was the problem in my original code as well). This example should be easier to follow and just sends you directly back to the "Distribution Statistics Boolean" case after changing the Significance Level, so that only one instance executes. 

Digging deeper into the NI_Gmath.lvlib:st_Wilcoxon diff test.vi block diagram, we can see that 'p-value' should not depend on the signficance level, it is only dependent on calculation from 'Normal CDF.vi'. Only the 'Reject Null Hypothesis' will be dependent upon the relationship between significance level and p-value. Therefore, to make a long story short, I believe it was my mistake in interchanging sample sets between the two instances of the Wilcoxon Signed Rank Test function that caused my confusion (and probably yours). I do now think the Wilcoxon code is executing correctly, although I will look into validation against statistical tables.

Sorry for taking your time on this one - just could not see the obvious until I put it down for 24 hours and looked at it again.

Sincerely,

Don