Re: [R] Maximum amount of memory

From: Prof Brian Ripley <ripley_at_stats.ox.ac.uk>
Date: Tue 22 Mar 2005 - 19:26:04 EST

On Mon, 21 Mar 2005, Thomas Lumley wrote:

> On Mon, 21 Mar 2005, Tim Cutts wrote:
>
>>
>> On 21 Mar 2005, at 4:42 pm, marvena@tin.it wrote:
>>
>>> Hi,
>>> I have a problem:I need to use the maximum amount of memory in order to
>>> perform a very tough analysis. By purchasing the suitable computer, what's
>>> the maximum amount of memory obtainable in R?
>>
>> Assuming that R is happy to use 64-bit memory pointers, the limit will be
>> your wallet.

(It has been for several years.)

> I believe there are still some limits on sizes of individual objects, such as
> C and Fortran code that uses int or INTEGER to hold dimensions.
>
> Many packages will definitely have problems: for example, the survival
> package cannot correctly handle a design matrix with more than 2^31-1
> elements, no matter how much memory it has. I don't know how much of the
> internal R code would also break when vectors have more than 2^31-1 entries.

For the record: R limits the length of vectors to 2^31 - 1, even on 64-bit machines. We have discussed changing this, but the use of Fortran for e.g. matrix algebra (which does not have a longer integer type) means that a lot of work would be needed to raise the limit.

> Now, 2^31-1 entries in a numeric matrix is 16Gb in one object, so your wallet
> is still likely to be the practical limit.

Indeed, that is why we have postponed changing the internal limit until nearer the time as machines with say 64Gb of RAM become commonplace. (You need to be able to make copies to do anything useful with R objects.) Moore's Law suggests that will not happen until the early 2010s.

-- 
Brian D. Ripley,                  ripley@stats.ox.ac.uk
Professor of Applied Statistics,  http://www.stats.ox.ac.uk/~ripley/
University of Oxford,             Tel:  +44 1865 272861 (self)
1 South Parks Road,                     +44 1865 272866 (PA)
Oxford OX1 3TG, UK                Fax:  +44 1865 272595

______________________________________________
R-help@stat.math.ethz.ch mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html
Received on Tue Mar 22 19:47:08 2005

This archive was generated by hypermail 2.1.8 : Fri 03 Mar 2006 - 03:30:52 EST