From: Duncan Murdoch <murdoch_at_stats.uwo.ca>

Date: Mon 22 May 2006 - 23:15:21 EST

*>
*

> We see this misunderstanding worryingly often. Worrying because it

*> reveals that a fundamental aspect of statistical inference has not
*

*> been grasped: that p-values are designed to be (approximately)
*

*> uniformly distributed and fall below any given level with the stated
*

*> probability, when the null hypothesis is true.
*

R-help@stat.math.ethz.ch mailing list

https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html Received on Mon May 22 23:33:02 2006

Date: Mon 22 May 2006 - 23:15:21 EST

On 5/22/2006 5:25 AM, Peter Dalgaard wrote:

> Uwe Ligges <ligges@statistik.uni-dortmund.de> writes:

*>
**>
*

>>> A = 0.1846, p-value = 0.9059 >>> >>>> ad.test(rnorm(100,mean=5,sd=3)) >>> ... >>> A = 0.5138, p-value = 0.1887 >>> >>> I mistakenly had thought the p-values would be more stable since I >>> am artificially creating a random normal distribution. Is this expected >>> for a normality test or is this an issue with how rnorm is producing >>> random numbers? I guess if I run it many times, I would find that I >>> would get many large values for the p-value? >> Well, as many large values as small values, 5% significant differences >> for the 5% level.... >> >> The following looks alright: >> >> hist(replicate(1000, ad.test(rnorm(100,mean=5,sd=3))$p.value))

> We see this misunderstanding worryingly often. Worrying because it

I think it's the fallacious belief that the p-value measures the probability that the null hypothesis is true. This is currently misunderstanding #1 in the Wikipedia entry for P-values.

(Google had me worried: I searched for "probability that the null hypothesis is true" and found

*> P-value - Wikipedia, the free encyclopedia
*

> The p-value is the probability that the null hypothesis is true, justifying the "rule" of considering as significant p-values closer to 0 (zero). ...

> en.wikipedia.org/wiki/P-value - 17k - Cached - Similar pages

This quote is preceded by: "All of the following [...] statements are false:" Context is important! :-)

The vast majority of hits to that search also pointed out that this interpretation was incorrect. A couple of counterexamples were a "research methods" page at a department of psychology, and another at a medical school. I'll send a copy of this note to people there.

Duncan Murdoch

*>
*

> There is no mechanism to give you "fewer significant" or "more stable"

*> p-values, and a p-value close to one is no better an indication of a
**> true null hypothesis than one of 0.5 or 0.25.
*

>

R-help@stat.math.ethz.ch mailing list

https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html Received on Mon May 22 23:33:02 2006

Archive maintained by Robert King, hosted by
the discipline of
statistics at the
University of Newcastle,
Australia.

Archive generated by hypermail 2.1.8, at Tue 23 May 2006 - 00:10:14 EST.

*
Mailing list information is available at https://stat.ethz.ch/mailman/listinfo/r-help.
Please read the posting
guide before posting to the list.
*