Re: [R] Memory allocation

From: Prof Brian Ripley <ripley_at_stats.ox.ac.uk>
Date: Thu 07 Sep 2006 - 14:44:41 GMT

On Thu, 7 Sep 2006, alex lam (RI) wrote:

> Dear list,
>
> I have been trying to run the function "qvalue" under the package qvalue
> on a vector with about 20 million values.
>
> > asso_p.qvalue<-qvalue(asso_p.vector)

> Error: cannot allocate vector of size 156513 Kb
> > sessionInfo()
> Version 2.3.1 (2006-06-01)
> i686-pc-linux-gnu

>
> attached base packages:
> [1] "methods" "stats" "graphics" "grDevices" "utils"
> "datasets"
> [7] "base"
>
> other attached packages:
> qvalue
> "1.1"
> > gc()
> used (Mb) gc trigger (Mb) max used (Mb)
> Ncells 320188 8.6 23540643 628.7 20464901 546.5
> Vcells 101232265 772.4 294421000 2246.3 291161136 2221.4
>
> I have been told that the linux box has 4Gb of RAM, so it should be able
> to do better than this.

But it also has a 4Gb/process address space, and of that some (1Gb?) is reserved for the system. So it is quite possible that with 2.2Gb used you are unable to find any large blocks.

> I searched the FAQ and found some tips on increasing memory size, but
> they seem to be windows specific, such as memory.size() and the
> -max-mem-size flag. On my linux box R didn't recognise them.

?"Memory-limits" is the key

     Error messages beginning 'cannot allocate vector of size' indicate
     a failure to obtain memory, either because the size exceeded the
     address-space limit for a process or, more likely, because the
     system was unable to provide the memory.  Note that on a 32-bit OS
     there may well be enough free memory available, but not a large
     enough contiguous block of address space into which to map it.

> I don't understand the meaning of max-vsize, max-nsize and max-ppsize.
> Any help on how to increase the memory allocation on linux is much
> appreciated.

Get a 64-bit OS.

-- 
Brian D. Ripley,                  ripley@stats.ox.ac.uk
Professor of Applied Statistics,  http://www.stats.ox.ac.uk/~ripley/
University of Oxford,             Tel:  +44 1865 272861 (self)
1 South Parks Road,                     +44 1865 272866 (PA)
Oxford OX1 3TG, UK                Fax:  +44 1865 272595

______________________________________________
R-help@stat.math.ethz.ch mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Received on Fri Sep 08 00:50:35 2006

Archive maintained by Robert King, hosted by the discipline of statistics at the University of Newcastle, Australia.
Archive generated by hypermail 2.1.8, at Thu 07 Sep 2006 - 15:50:41 GMT.

Mailing list information is available at https://stat.ethz.ch/mailman/listinfo/r-help. Please read the posting guide before posting to the list.