Re: [R] Rank-based p-value on large dataset

From: Deepayan Sarkar <deepayan_at_stat.wisc.edu>
Date: Fri 04 Mar 2005 - 09:32:38 EST

On Thursday 03 March 2005 16:22, Sean Davis wrote:
> I have a fairly simple problem--I have about 80,000 values (call them
> y) that I am using as an empirical distribution and I want to find
> the p-value (never mind the multiple testing issues here, for the
> time being) of 130,000 points (call them x) from the empirical
> distribution. I typically do that (for one-sided test) something like
>
> loop over i in x
> p.val[i] = sum(y>x[i])/length(y)
>
> and repeat for all i. However, length(x) is large here as is
> length(y), so this process takes quite a long time. Any suggestions?

The obvious thing to do would be

p.val = 1 - ecdf(x)(y)

wouldn't it? On a 1.1 GHz Athlon, I get

> x <- rnorm(130000)
> y <- rnorm(80000)

> system.time(p.val <- 1 - ecdf(y)(x))


[1] 1.03 0.03 1.06 0.00 0.00

-Deepayan



R-help@stat.math.ethz.ch mailing list
https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html Received on Fri Mar 04 10:03:13 2005

This archive was generated by hypermail 2.1.8 : Fri 03 Mar 2006 - 03:30:40 EST