Re: [R] normality testing with nortest

From: Duncan Murdoch <>
Date: Mon 22 May 2006 - 23:15:21 EST

On 5/22/2006 5:25 AM, Peter Dalgaard wrote:
> Uwe Ligges <> writes:

>>> A = 0.1846, p-value = 0.9059
>>>> ad.test(rnorm(100,mean=5,sd=3))
>>> ...
>>> A = 0.5138, p-value = 0.1887
>>>     I mistakenly had thought the p-values would be more stable since I 
>>> am artificially creating a random normal distribution.  Is this expected 
>>> for a normality test or is this an issue with how rnorm is producing 
>>> random numbers?  I guess if I run it many times, I would find that I 
>>> would get many large values for the p-value?
>> Well, as many large values as small values, 5% significant differences 
>> for the 5% level....
>> The following looks alright:
>> hist(replicate(1000,  ad.test(rnorm(100,mean=5,sd=3))$p.value))

> We see this misunderstanding worryingly often. Worrying because it
> reveals that a fundamental aspect of statistical inference has not
> been grasped: that p-values are designed to be (approximately)
> uniformly distributed and fall below any given level with the stated
> probability, when the null hypothesis is true.

I think it's the fallacious belief that the p-value measures the probability that the null hypothesis is true. This is currently misunderstanding #1 in the Wikipedia entry for P-values.

(Google had me worried: I searched for "probability that the null hypothesis is true" and found

> P-value - Wikipedia, the free encyclopedia
> The p-value is the probability that the null hypothesis is true, justifying the "rule" of considering as significant p-values closer to 0 (zero). ...
> - 17k - Cached - Similar pages

This quote is preceded by: "All of the following [...] statements are false:" Context is important! :-)

The vast majority of hits to that search also pointed out that this interpretation was incorrect. A couple of counterexamples were a "research methods" page at a department of psychology, and another at a medical school. I'll send a copy of this note to people there.

Duncan Murdoch

> There is no mechanism to give you "fewer significant" or "more stable"
> p-values, and a p-value close to one is no better an indication of a
> true null hypothesis than one of 0.5 or 0.25.
> mailing list PLEASE do read the posting guide! Received on Mon May 22 23:33:02 2006

Archive maintained by Robert King, hosted by the discipline of statistics at the University of Newcastle, Australia.
Archive generated by hypermail 2.1.8, at Tue 23 May 2006 - 00:10:14 EST.

Mailing list information is available at Please read the posting guide before posting to the list.