Re: [Rd] R 2.14.1 memory management under Windows

From: Prof Brian Ripley <ripley_at_stats.ox.ac.uk>
Date: Thu, 22 Mar 2012 12:02:08 +0000

On 22/03/2012 06:11, Peter Meilstrup wrote:
> My guess would be that it's a matter of having swap space be a dedicated
> partition or fixed-size file (Linux, usually) versus swapping to a regular
> file that grows as needed (Windows and OS X, usually.) So if you
> defragmented your drive and set Windows to have a fixedsize swap file, it
> would probably behave more like your Linux machine.

There is far more to the topic than that, but the answer here appears to be a complete failure to supply the relevant information.

We haven't even been told the 'at a minumum' information required by the posting guide, so we do not know what architectures are in use. The messages suggest that 'Linux' is 32-bit and 'Windows' is 64-bit, in which case the tasks are simply not comparable. On 32-bit R on Windows I got the message about 3.4GB after 0.05 sec. Conversely, with 64-bit R on an 8GB Linux box with 16GB swap it swapped away for about 10 minutes.   On a 32GB box it succeeded after 270s, typically using 8-14GB. The object SG tried to create is a bit over 7GB.

But Windows' memory management is notoriously slow, and R actually adds a layer on top to make it tolerable for routine use of R.

I have no idea why this was posted on R-devel: it did not involve R development nor programming, just a basic understanding of 32- vs 64-bit R.

>
> Peter
>
> On Wed, Mar 21, 2012 at 10:14 PM, Spencer Graves<
> spencer.graves_at_prodsyse.com> wrote:
>
>> I computed "system.time(diag(30000))" with R 2.12.0 on Fedora 13 Linux
>> with 4 GB RAM and with R 2.14.1 on Windows 7 with 8 GB RAM:
>>
>>
>> Linux (4 GB RAM): 0, 0.21, 0.21 -- a fifth of a second
>>
>>
>> Windows 7 (8 GB RAM): 11.37 7.47 93.19 -- over 1.5 minutes. Moreover,
>> during most of that time, I could not switch windows or get any response
>> from the system. When I first encountered this, I thought Windows was hung
>> permanently and the only way out was a hard reset and reboot.
>>
>>
>> On both systems, diag(30000) generated, "Error: cannot allocate
>> vector of size ___ Gb", with "___" = 3.4 for Linux with 4 GB RAM and 6.7
>> for Windows with 8 GB RAM. Linux with half the RAM and an older version of
>> R was done with this in 0.21 seconds. Windows 7 went into suspension for
>> over 93 seconds -- 1.5 minutes before giving an error message.
>>
>>
>> I don't know how easy this would be to fix under Windows, but I felt
>> a need to report it.
>>
>>
>> Best Wishes,
>> Spencer
>>
>>
>> --
>> Spencer Graves, PE, PhD
>> President and Chief Technology Officer
>> Structure Inspection and Monitoring, Inc.
>> 751 Emerson Ct.
>> San Josť, CA 95126
>> ph: 408-655-4567
>> web: www.structuremonitoring.com
>>
>> ______________________________**________________
>> R-devel_at_r-project.org mailing list
>> https://stat.ethz.ch/mailman/**listinfo/r-devel<https://stat.ethz.ch/mailman/listinfo/r-devel>
>>
>
> [[alternative HTML version deleted]]
>
>
>
>
> ______________________________________________
> R-devel_at_r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel

-- 
Brian D. Ripley,                  ripley_at_stats.ox.ac.uk
Professor of Applied Statistics,  http://www.stats.ox.ac.uk/~ripley/
University of Oxford,             Tel:  +44 1865 272861 (self)
1 South Parks Road,                     +44 1865 272866 (PA)
Oxford OX1 3TG, UK                Fax:  +44 1865 272595

______________________________________________
R-devel_at_r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Received on Thu 22 Mar 2012 - 12:04:32 GMT

This quarter's messages: by month, or sorted: [ by date ] [ by thread ] [ by subject ] [ by author ]

All messages

Archive maintained by Robert King, hosted by the discipline of statistics at the University of Newcastle, Australia.
Archive generated by hypermail 2.2.0, at Thu 22 Mar 2012 - 13:50:33 GMT.

Mailing list information is available at https://stat.ethz.ch/mailman/listinfo/r-devel. Please read the posting guide before posting to the list.

list of date sections of archive