Re: [R] memory managment under Windows XP

From: Prof Brian Ripley <ripley_at_stats.ox.ac.uk>
Date: Fri 24 Feb 2006 - 09:01:14 EST

On Thu, 23 Feb 2006, roger bos wrote:

> And of course using rm(...) to clean up objects you no longer need.  No
> amount of physical memory can save you from grossly inefficient code and
> large memory leaks.  For example, lets say I have a large testMat object
> that I use time period.  I loop though each month using for loops.  Even
> though the object has the same name each month, and thus gets overwritten,
> the memory management seems to go better by manually removing the object at
> the end of each loop.  Also, I sometimes call gc() at the end of each loop,
> but I don't know if that actually helps or not.  I figure it can't hurt.

It does help to rm() and then gc() at a point when you know that the number of objects in use is minimal. R will gc() repeatedly when it starts to run out of address space, but this does not help if the address space is already fragmented.

The main problem on 32-bit OSes is (virtual) memory fragmentation, and 3Gb is not really much address space for objects in 100s of Mb.

I've now only got a 64-bit desktop and servers (plus a 32-bit Windows laptop). It is a shame for Windows users that a 64-bit Open Source toolchain* is nowhere in sight, but I suspect a sufficiently determined user of Win64 could build a 64-bit port of R with commercial compilers.

*You need a compiler, assembler, linker and runtime, and the latter may well be the most problematic.

> On 2/23/06, Liaw, Andy <andy_liaw@merck.com> wrote:

>>
>> There are plenty in the the list archive, one of which is to switch to a
>> run
>> 64-bit R on a 64-bit platform with lots of physical RAM. Such hardware is
>> quite affordable these days (certainly cheaper than most commercial
>> software
>> that you'd have to buy if you didn't have R).
>>
>> Andy
>>
>> From: r user
>>>
>>> I am using R 2.2.1 in a Windowes XP environment.
>>>
>>> I work with very large datasets, and occassionally run
>>> out of memory.
>>>
>>> I have modified my boot.ini file to use the "/3gb
>>> switch".
>>>
>>> I also run the following line after I launch R ( I am
>>> unsure if it is helpful).
>>>
>>> "memory.limit(size = 4095)"
>>>
>>> Please point me to useful references on how to better
>>> manage memory, or suggestother actions.
>>>
>>> ______________________________________________
>>> R-help@stat.math.ethz.ch mailing list
>>> https://stat.ethz.ch/mailman/listinfo/r-help
>>> PLEASE do read the posting guide!
>>> http://www.R-project.org/posting-guide.html
>>>
>>>
>>
>> ______________________________________________
>> R-help@stat.math.ethz.ch mailing list
>> https://stat.ethz.ch/mailman/listinfo/r-help
>> PLEASE do read the posting guide!
>> http://www.R-project.org/posting-guide.html
>>
>
> 	[[alternative HTML version deleted]]
>
> ______________________________________________
> R-help@stat.math.ethz.ch mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html
>

-- 
Brian D. Ripley,                  ripley@stats.ox.ac.uk
Professor of Applied Statistics,  http://www.stats.ox.ac.uk/~ripley/
University of Oxford,             Tel:  +44 1865 272861 (self)
1 South Parks Road,                     +44 1865 272866 (PA)
Oxford OX1 3TG, UK                Fax:  +44 1865 272595

______________________________________________
R-help@stat.math.ethz.ch mailing list
https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html Received on Fri Feb 24 09:10:45 2006

This archive was generated by hypermail 2.1.8 : Fri 24 Feb 2006 - 12:08:49 EST