# Re: [R] normality testing with nortest

From: Duncan Murdoch <murdoch_at_stats.uwo.ca>
Date: Mon 22 May 2006 - 23:15:21 EST

On 5/22/2006 5:25 AM, Peter Dalgaard wrote:
> Uwe Ligges <ligges@statistik.uni-dortmund.de> writes:
>
>

```>>> A = 0.1846, p-value = 0.9059
>>>
>>> ...
>>> A = 0.5138, p-value = 0.1887
>>>
>>>     I mistakenly had thought the p-values would be more stable since I
>>> am artificially creating a random normal distribution.  Is this expected
>>> for a normality test or is this an issue with how rnorm is producing
>>> random numbers?  I guess if I run it many times, I would find that I
>>> would get many large values for the p-value?
>> Well, as many large values as small values, 5% significant differences
>> for the 5% level....
>>
>> The following looks alright:
>>
```

>
> We see this misunderstanding worryingly often. Worrying because it
> reveals that a fundamental aspect of statistical inference has not
> been grasped: that p-values are designed to be (approximately)
> uniformly distributed and fall below any given level with the stated
> probability, when the null hypothesis is true.

I think it's the fallacious belief that the p-value measures the probability that the null hypothesis is true. This is currently misunderstanding #1 in the Wikipedia entry for P-values.

(Google had me worried: I searched for "probability that the null hypothesis is true" and found

> P-value - Wikipedia, the free encyclopedia
> The p-value is the probability that the null hypothesis is true, justifying the "rule" of considering as significant p-values closer to 0 (zero). ...
> en.wikipedia.org/wiki/P-value - 17k - Cached - Similar pages

This quote is preceded by: "All of the following [...] statements are false:" Context is important! :-)

The vast majority of hits to that search also pointed out that this interpretation was incorrect. A couple of counterexamples were a "research methods" page at a department of psychology, and another at a medical school. I'll send a copy of this note to people there.

Duncan Murdoch

>
> There is no mechanism to give you "fewer significant" or "more stable"
> p-values, and a p-value close to one is no better an indication of a
> true null hypothesis than one of 0.5 or 0.25.
>

R-help@stat.math.ethz.ch mailing list