[Rd] cor.test(method = spearman, exact = TRUE) not exact (PR#14095)

From: <dsimcha_at_gmail.com>
Date: Mon, 30 Nov 2009 04:00:12 +0100 (CET)

Full_Name: David Simcha
Version: 2.10
OS: Windows XP Home
Submission from: (NULL) (

> a <- c(1:10)
> b <- c(1:10)
> cor.test(a, b, method = "spearman", alternative = "greater", exact = TRUE)

        Spearman's rank correlation rho

data: a and b
S = 0, p-value < 2.2e-16
alternative hypothesis: true rho is greater than 0 sample estimates:

> 1 / factorial(10)

[1] 2.755732e-07

Since we have perfect rank correlation and only one permutation out of 10! could give this for N = 10, the p-value should be 1/10!. Reading the code in prho.c, it appears that the "exact" calculation uses the Edgeworth approximation for N > 9. This makes sense because, for similar examples with N <= 9, the results are as expected (1 / N!).

The "exact" p-value calculation is good enough for most practical purposes, but is clearly not exact. Some informal testing I've done indicates that it can even be less accurate than the "approximate" p-value calculation in some cases. I think it's absurd to call these p-values "exact" when they are clearly based on an asymptotic approximation that can be off by orders of magnitude in some cases.

R-devel_at_r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel Received on Mon 30 Nov 2009 - 11:24:29 GMT

This archive was generated by hypermail 2.2.0 : Mon 30 Nov 2009 - 13:20:53 GMT