[R] can the matrix size limit be increased?

From: Robert Leach <rwleach_at_ccr.buffalo.edu>
Date: Fri, 29 Feb 2008 14:25:36 -0500


Hi there,

I'm brand new to R, so let me know if this question is not appropriate for this list. I've been reading through the documentation and have tried a number of things, but am pretty much stuck so far. Here's the session info:

> sessionInfo()

R version 2.6.2 (2008-02-08)
i386-apple-darwin8.10.1

locale:
C

attached base packages:
[1] stats graphics grDevices utils datasets methods base

loaded via a namespace (and not attached): [1] rcompgen_0.1-17

So I seem to be hitting a limit on matrix size. First I read in my data into a list and it's OK:

> mz <- scan("data.column3.txt", list(0))
Read 158991 records
>mz
> mz

[[1]]

     [1] 0.000000e+00 0.000000e+00 1.003393e+01 3.651888e+00 0.000000e +00

     [6] 0.000000e+00 3.067042e+00 1.277249e+00 1.984366e+00 3.644203e +01

    [11] 1.172925e+02 1.933753e+02 2.020940e+02 1.570501e+02 8.990829e +01
...

But when I try to put it into a matrix like this, I don't get an error, but the matrix appears empty...

> MZ=matrix(mz[[1]],nrow=1)
> MZ

      [,1] [,2] [,3] [,4] [,5] [,6] [,7] [,8] [, 9] [,10]

         [,11] [,12] [,13] [,14] [,15] [,16] [, 17] [,18]

         [,19] [,20] [,21] [,22] [,23] [,24] [, 25] [,26]
...

When I did a subset of my data, it was fine. I did a manual binary search and determined the cutoff to be 100000 elements. So if I do just 99,999 elements, it looks as I would expect:

> mz <- scan("data.column3.txt", list(0), 99999)
Read 99999 records
> tst <- matrix(mz[[1]],nrow=1)
> tst

      [,1] [,2] [,3] [,4] [,5] [,6] [,7] [,8] [, 9] [,10]
[1,] 0 0 10.03393 3.651888 0 0 3.067042 1.277249 1.984366 36.44203

         [,11] [,12] [,13] [,14] [,15] [,16] [, 17] [,18]
[1,] 117.2925 193.3753 202.0940 157.0501 89.9083 26.44127 17.05373 53.40315

         [,19] [,20] [,21] [,22] [,23] [,24] [, 25] [,26]
[1,] 65.20086 37.33463 17.71247 27.37268 41.83289 48.46916 58.94969 76.05099
...

If I do 100,000, I get the same empty appearance. I've assumed that there must be a limitation on the number of elements in a matrix. Is that right? If so, how do I increase the maximum number of elements? I tried another machine's installation of R and it apparently doesn't have a 99,999 element limit. I've tried using:

R --max-mem-size=2G
R --max-vsize=200000
R --max-nsize=200000
R --max-vsize=200000 --max-nsize=200000 --max-ppsize=200000
R --max-vsize=10M

I still end up with the empty-looking matrix when I try these. How do I get my installation to work like the installation on another computer I tried where I was able to have larger matrices?

Oh yeah, I also tried this, just to rule out problems with my data:

> tst <- matrix(seq(1,158991),nrow=1,ncol=158991)
> tst

      [,1] [,2] [,3] [,4] [,5] [,6] [,7] [,8] [,9] [,10] [,11] [,12]  
[,13] [,14]
      [,15] [,16] [,17] [,18] [,19] [,20] [,21] [,22] [,23] [,24] [, 
25] [,26]
      [,27] [,28] [,29] [,30] [,31] [,32] [,33] [,34] [,35] [,36] [, 
37] [,38]
...

Thanks,
Rob

Robert W. Leach
Scientific Programmer
Center for Computational Research
Center of Excellence in Bioinformatics
University at Buffalo
http://www.ccr.buffalo.edu/



R-help_at_r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code. Received on Fri 29 Feb 2008 - 19:29:43 GMT

Archive maintained by Robert King, hosted by the discipline of statistics at the University of Newcastle, Australia.
Archive generated by hypermail 2.2.0, at Sat 01 Mar 2008 - 01:30:17 GMT.

Mailing list information is available at https://stat.ethz.ch/mailman/listinfo/r-help. Please read the posting guide before posting to the list.

list of date sections of archive