Re: [R] can the matrix size limit be increased?

From: jim holtman <jholtman_at_gmail.com>
Date: Fri, 29 Feb 2008 19:36:22 -0500

You only have 1 row in your matrix, so what you are getting printed out is not an empty matrix, but the header. If you print the transpose you get:

> head(t(tst))

     [,1]

[1,]    1
[2,]    2
[3,]    3
[4,]    4
[5,]    5
[6,]    6

>

The default is to only print out 100000 values. Your data is there, you just have to wait for the headers to print out, or at least make a matrix with more than one row.

On Fri, Feb 29, 2008 at 2:25 PM, Robert Leach <rwleach_at_ccr.buffalo.edu> wrote:
> Hi there,
>
> I'm brand new to R, so let me know if this question is not
> appropriate for this list. I've been reading through the
> documentation and have tried a number of things, but am pretty much
> stuck so far. Here's the session info:
>
> > sessionInfo()
> R version 2.6.2 (2008-02-08)
> i386-apple-darwin8.10.1
>
> locale:
> C
>
> attached base packages:
> [1] stats graphics grDevices utils datasets methods base
>
> loaded via a namespace (and not attached):
> [1] rcompgen_0.1-17
>
>
> So I seem to be hitting a limit on matrix size. First I read in my
> data into a list and it's OK:
>
> > mz <- scan("data.column3.txt", list(0))
> Read 158991 records
> >mz
> > mz
> [[1]]
> [1] 0.000000e+00 0.000000e+00 1.003393e+01 3.651888e+00 0.000000e
> +00
> [6] 0.000000e+00 3.067042e+00 1.277249e+00 1.984366e+00 3.644203e
> +01
> [11] 1.172925e+02 1.933753e+02 2.020940e+02 1.570501e+02 8.990829e
> +01
> ...
>
> But when I try to put it into a matrix like this, I don't get an
> error, but the matrix appears empty...
>
> > MZ=matrix(mz[[1]],nrow=1)
> > MZ
> [,1] [,2] [,3] [,4] [,5] [,6] [,7] [,8] [,
> 9] [,10]
> [,11] [,12] [,13] [,14] [,15] [,16] [,
> 17] [,18]
> [,19] [,20] [,21] [,22] [,23] [,24] [,
> 25] [,26]
> ...
>
> When I did a subset of my data, it was fine. I did a manual binary
> search and determined the cutoff to be 100000 elements. So if I do
> just 99,999 elements, it looks as I would expect:
>
> > mz <- scan("data.column3.txt", list(0), 99999)
> Read 99999 records
> > tst <- matrix(mz[[1]],nrow=1)
> > tst
> [,1] [,2] [,3] [,4] [,5] [,6] [,7] [,8] [,
> 9] [,10]
> [1,] 0 0 10.03393 3.651888 0 0 3.067042 1.277249 1.984366
> 36.44203
> [,11] [,12] [,13] [,14] [,15] [,16] [,
> 17] [,18]
> [1,] 117.2925 193.3753 202.0940 157.0501 89.9083 26.44127 17.05373
> 53.40315
> [,19] [,20] [,21] [,22] [,23] [,24] [,
> 25] [,26]
> [1,] 65.20086 37.33463 17.71247 27.37268 41.83289 48.46916 58.94969
> 76.05099
> ...
>
> If I do 100,000, I get the same empty appearance. I've assumed that
> there must be a limitation on the number of elements in a matrix. Is
> that right? If so, how do I increase the maximum number of
> elements? I tried another machine's installation of R and it
> apparently doesn't have a 99,999 element limit. I've tried using:
>
> R --max-mem-size=2G
> R --max-vsize=200000
> R --max-nsize=200000
> R --max-vsize=200000 --max-nsize=200000 --max-ppsize=200000
> R --max-vsize=10M
>
> I still end up with the empty-looking matrix when I try these. How
> do I get my installation to work like the installation on another
> computer I tried where I was able to have larger matrices?
>
> Oh yeah, I also tried this, just to rule out problems with my data:
>
> > tst <- matrix(seq(1,158991),nrow=1,ncol=158991)
> > tst
> [,1] [,2] [,3] [,4] [,5] [,6] [,7] [,8] [,9] [,10] [,11] [,12]
> [,13] [,14]
> [,15] [,16] [,17] [,18] [,19] [,20] [,21] [,22] [,23] [,24] [,
> 25] [,26]
> [,27] [,28] [,29] [,30] [,31] [,32] [,33] [,34] [,35] [,36] [,
> 37] [,38]
> ...
>
> Thanks,
> Rob
>
> Robert W. Leach
> Scientific Programmer
> Center for Computational Research
> Center of Excellence in Bioinformatics
> University at Buffalo
> http://www.ccr.buffalo.edu/
>
> ______________________________________________
> R-help_at_r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>

-- 
Jim Holtman
Cincinnati, OH
+1 513 646 9390

What is the problem you are trying to solve?  Tell me what you want to
do, not how you want to do it.

______________________________________________
R-help_at_r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Received on Sat 01 Mar 2008 - 00:39:17 GMT

Archive maintained by Robert King, hosted by the discipline of statistics at the University of Newcastle, Australia.
Archive generated by hypermail 2.2.0, at Sat 01 Mar 2008 - 01:30:17 GMT.

Mailing list information is available at https://stat.ethz.ch/mailman/listinfo/r-help. Please read the posting guide before posting to the list.

list of date sections of archive