Re: [R] memory issues

From: Prof Brian Ripley <ripley_at_stats.ox.ac.uk>
Date: Wed, 16 Apr 2008 22:41:31 +0100 (BST)

I think the clue is that the message you quote comes from gstat, which does not use R's memory allocator. It is gstat and not R that has failed to allocate memory.

Try re-reading the help page for memory.size. 'max=T' does not indicate the limit (that is the job of memory.limit()), but the maximum 'used' (acquired from the OS) in that session. So 19Mb looks reasonable for R's usage.

I don't understand the message from memory.limit() (and the formatting is odd). memory.limit() does not call max() (it is entirely internal), so I wonder if that really is the output from that command. (If you can reproduce it, please let us have precise reproduction instructions.)

There isn't much point in increasing the memory limit from the default 1.5Gb on a 2Gb XP machine. The problem is that the user address space limit is 2Gb and fragmentation means that you are unlikely to be able to get over 1.5Gb unless you have very many small objects, in which case R will run very slowly. In any case, that is not the issue here.

On Wed, 16 Apr 2008, Dave Depew wrote:

> Hi all,
> I've read the R for windows FAQ and am a little confused re:
> memory.limit and memory.size
>
> to start using R 2.6.2 on WinXP, 2GB RAM, I have the command line "sdi
> --max-mem-size=2047M"
> Once the Rgui is open, memory.limit() returns 2047, memory.size()
> returns 11.315, and memory.size(max=T) returns 19.615
>
> Shouldn't memory.size(max=T) return 2047?
>
> Upon running several operations involving kriging (gstat package,
> original data file 3 variables, 12000 observations)
> the program runs out of memory
>
> "memory.c", line 57: can't allocate memory in function m_get()
> Error in predict.gstat(fit.uk, newdata = EcoSAV.grid.clip.spxdf,
> debug.level = -2, :
> m_get
>
> Immediately following this,
>
> memory.limit() returns [1] -Inf
> Warning message:
> In memory.limit() : no non-missing arguments
> to max; returning -Inf
>
> memory.size() returns 24.573.
>
> memory.size(max=T) returns 46.75
>
> To my untrained eye, it appears that R is not being allowed access to
> the full memory limit specified in the cmd line....if this is the case,
> how does one ensure that R is getting access to the full allotment of RAM?
> Any insight is appreciated...
>
>
> > sessionInfo()
> R version 2.6.2 (2008-02-08)
> i386-pc-mingw32
>
> locale:
> LC_COLLATE=English_United States.1252;LC_CTYPE=English_United
> States.1252;LC_MONETARY=English_United
> States.1252;LC_NUMERIC=C;LC_TIME=English_United States.1252
>
> attached base packages:
> [1] stats graphics grDevices datasets tcltk utils
> methods base
>
> other attached packages:
> [1] maptools_0.7-7 foreign_0.8-23 gstat_0.9-43 rgdal_0.5-24
> lattice_0.17-4 sp_0.9-23 svSocket_0.9-5 svIO_0.9-5
> R2HTML_1.58 svMisc_0.9-5 svIDE_0.9-5
>
> loaded via a namespace (and not attached):
> [1] grid_2.6.2 tools_2.6.2
>
> ______________________________________________
> R-help_at_r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>

-- 
Brian D. Ripley,                  ripley_at_stats.ox.ac.uk
Professor of Applied Statistics,  http://www.stats.ox.ac.uk/~ripley/
University of Oxford,             Tel:  +44 1865 272861 (self)
1 South Parks Road,                     +44 1865 272866 (PA)
Oxford OX1 3TG, UK                Fax:  +44 1865 272595

______________________________________________
R-help_at_r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Received on Wed 16 Apr 2008 - 21:50:50 GMT

Archive maintained by Robert King, hosted by the discipline of statistics at the University of Newcastle, Australia.
Archive generated by hypermail 2.2.0, at Wed 16 Apr 2008 - 23:30:28 GMT.

Mailing list information is available at https://stat.ethz.ch/mailman/listinfo/r-help. Please read the posting guide before posting to the list.

list of date sections of archive