From: Dennis Murphy <djmuser_at_gmail.com>

Date: Mon, 03 Jan 2011 21:41:05 -0800

library(plyr)

# and then to recombine them with cbind() into a 4-column matrix => each # row is then ready to be reshaped into a 2 x 2 matrix for input into fisher.test()

+

+ } )

> I have 4 1000*1000 matrix A,B,C,D. I want to use the corresponding element

*> of the 4 matrices. Using the "for loop" as follow:
*

*> E<-o
*

> for (i in 1:1000)

*> {for (j in 1:1000)
*

*> {
*

*> E<-fisher.test(matrix(c(A[i][j],B[i][j],C[i][j],D[i][j]),2))#call
*

*> fisher.test for every element
*

*> }
*

*> }
*

> It is so time-consuming

*> Need vectorization
*

> Yours sincerely

> ZhaoXing

*> Department of Health Statistics
*

*> West China School of Public Health
*

*> Sichuan University
*

*> No.17 Section 3, South Renmin Road
*

*> Chengdu, Sichuan 610041
*

*> P.R.China
*

*> 8O?lW"2aQE;"3,4sH]A?Cb7QSJOd?
*

*> R-help_at_r-project.org mailing list
*

*> https://stat.ethz.ch/mailman/listinfo/r-help
*

*> PLEASE do read the posting guide
*

*> http://www.R-project.org/posting-guide.html
*

*> and provide commented, minimal, self-contained, reproducible code.
*

>

R-help_at_r-project.org mailing list

https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code. Received on Tue 04 Jan 2011 - 05:44:11 GMT

Date: Mon, 03 Jan 2011 21:41:05 -0800

Hi:

The idea is as follows:

- string your four matrices into vectors, cbinding them so that the
columns correspond to what you want as the (1,1), (1, 2), (2, 1) and (2, 2)
elements, respectively, of the matrix/table to be used for the Fisher test;
- Operate row-wise on the constructed matrix, running fisher.test() in
each row, producing a list of model objects;
- use the ldply() function in package plyr to pick off pieces of output (or alternatively, do.call(rbind, <function applied to list>))

- Operate row-wise on the constructed matrix, running fisher.test() in
each row, producing a list of model objects;

Before I give an example, note that an object returned by fisher.test() has the following components, each of which can be extracted individually; w is the output object of a single run of fisher.test(): names(w)

[1] "p.value" "conf.int" "estimate" "null.value" "alternative" [6] "method" "data.name" #-----------------

library(plyr)

# generate some matrices:

m1 <- matrix(rpois(100, 7), nrow = 10) m2 <- matrix(rpois(100, 7), nrow = 10) m3 <- matrix(rpois(100, 7), nrow = 10) m4 <- matrix(rpois(100, 7), nrow = 10) # I want m1 to represent the (1, 1) elements, m2 the (1, 2) elements,# m3 the (2, 1) elements and m4 the (2, 2) elements of the table used as # input to fisher.test. The idea is to string each of the matrices into vectors

# and then to recombine them with cbind() into a 4-column matrix => each # row is then ready to be reshaped into a 2 x 2 matrix for input into fisher.test()

v <- cbind(as.vector(m1), as.vector(m2), as.vector(m3), as.vector(m4))

# Each row of v is reshaped into a 2 x 2 matrix and fisher.test() is
# applied. The result is a list of model objects from fisher.test(), one
component

# for each row of v. [In this example, 100 model objects are generated.]

ll <- vector('list', nrow(v))

for(i in seq_along(nrow(v))) ll[[i]] <- fisher.test(matrix(v[i, ], nrow =
2))

# Now ll can be used to extract individual pieces of output for each run;
e.g.,

# to get all the p-values and confidence intervals,

pvals <- ldply(ll, function(x) x$p.value) confints <- ldply(ll, function(x) x$conf.int)

The time it takes to generate the data and produce the 1000000 model objects
is:

> system.time( {

+ m1 <- matrix(rpois(1000000, 7), nrow = 1000) + m2 <- matrix(rpois(1000000, 7), nrow = 1000) + m3 <- matrix(rpois(1000000, 7), nrow = 1000) + m4 <- matrix(rpois(1000000, 7), nrow = 1000) + v <- cbind(as.vector(m1), as.vector(m2), as.vector(m3), as.vector(m4)) + ll <- vector('list', nrow(v)) + for(i in seq_along(nrow(v))) ll[[i]] <- fisher.test(matrix(v[i, ], nrow= 2))

+

+ } )

user system elapsed

0.43 0.00 0.43

> length(ll)

[1] 1000000

A second run in a fresh R session took 0.73 seconds elapsed.

When running a lot of tests like this, it's important to try out different alternatives. My initial approach to this problem was to use apply() on the rows of v. Testing it on my system when the input matrices were 100 x 100, it took 11.23 seconds, so for the 1000 x 1000 input matrices you're using, use of apply() would have taken around 20 minutes. In this particular case, use of a loop to fill in the components of a pre-allocated list object with the output of fisher.test() was much more efficient; in fact, the actual data generation and reshaping took up a significant amount of the time!

Data generation and reshaping:

> system.time( {

+ m1 <- matrix(rpois(1000000, 7), nrow = 1000) + m2 <- matrix(rpois(1000000, 7), nrow = 1000) + m3 <- matrix(rpois(1000000, 7), nrow = 1000) + m4 <- matrix(rpois(1000000, 7), nrow = 1000) + + v <- cbind(as.vector(m1), as.vector(m2), as.vector(m3), as.vector(m4)) + })

user system elapsed

0.27 0.00 0.27

**HTH,
**

Dennis

On Mon, Jan 3, 2011 at 6:57 AM, zhaoxing731 <zhaoxing731_at_yahoo.com.cn>wrote:

> Hello

>

> I have 4 1000*1000 matrix A,B,C,D. I want to use the corresponding element

>

> for (i in 1:1000)

>

> It is so time-consuming

>

> Yours sincerely

> > > >

> ZhaoXing

>

> [[alternative HTML version deleted]]

> >

> __________________________________________________

>

> ______________________________________________

>

[[alternative HTML version deleted]]

R-help_at_r-project.org mailing list

https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code. Received on Tue 04 Jan 2011 - 05:44:11 GMT

Archive maintained by Robert King, hosted by
the discipline of
statistics at the
University of Newcastle,
Australia.

Archive generated by hypermail 2.2.0, at Tue 04 Jan 2011 - 05:50:04 GMT.

*
Mailing list information is available at https://stat.ethz.ch/mailman/listinfo/r-help.
Please read the posting
guide before posting to the list.
*