Re: [R] growing a list sequentially -- memory management

From: erwann rogard <erwann.rogard_at_gmail.com>
Date: Fri, 14 Nov 2008 12:33:33 -0500

Thank you all for your answers. I think I have enough to keep going:

As pointed out it is probably not growing a list of objects each of size 6824 bytes that is the source of the problem per se (even if the list is not pre-allocated), but rather a "memory leak" with the function fun within the loop. I'm not providing the full example here because the code spans several files.

Nonetheless the body of fun itself contains deeply nested lists, some which are temporary and others which are part of the return value. So i'm now trying to see what to do about these nested lists instead. About memory leak: my assumption was that the memory needed for a temporary object (i.e. defined within the body of a function) would be redeemed after the function call, but perhaps i'm wrong about that (no automatic garbage collection)?

On Fri, Nov 14, 2008 at 6:37 AM, Patrick Burns <pburns_at_pburns.seanet.com>wrote:

> The recommended technique is to create objects
> as their final size and then subscript into them
> with your data.
>
> My intuition (which is often brutally wrong) tells
> me that your case should not be overly traumatic.
> So I'm suspicious that you are fragmenting memory
> in other ways as well.
>
> Patrick Burns
> patrick_at_burns-stat.com
> +44 (0)20 8525 0696
> http://www.burns-stat.com
> (home of S Poetry and "A Guide for the Unwilling S User")
>
>
> erwann rogard wrote:
>
>> hello,
>>
>> i have something like:
>>
>> out<-list()
>>
>> for(i in 1:n){
>> data<-gen(...) #fixed size data
>> out[[i]]<- fun(data)
>> }
>>
>>
>>
>>> object.size(out[[1]])
>>>
>>>
>> 6824
>>
>> In principle 1 GB should allow
>>
>> n = 1024^3/6824 = 157347?
>>
>> i have about 2GB are not taken by other processes. however, I can see the
>> memory shrinking quite rapidly on my system monitor and have to stop the
>> simulation after only n=300. why such a discrepancy? any remedy?
>>
>> x86_64-pc-linux/RKWard/R2.8.0/ 4GB
>>
>> thanks.
>>
>> [[alternative HTML version deleted]]
>>
>> ______________________________________________
>> R-help_at_r-project.org mailing list
>> https://stat.ethz.ch/mailman/listinfo/r-help
>> PLEASE do read the posting guide
>> http://www.R-project.org/posting-guide.html
>> and provide commented, minimal, self-contained, reproducible code.
>>
>>
>>
>>
>

        [[alternative HTML version deleted]]



R-help_at_r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code. Received on Fri 14 Nov 2008 - 17:35:28 GMT

Archive maintained by Robert King, hosted by the discipline of statistics at the University of Newcastle, Australia.
Archive generated by hypermail 2.2.0, at Fri 14 Nov 2008 - 18:30:24 GMT.

Mailing list information is available at https://stat.ethz.ch/mailman/listinfo/r-help. Please read the posting guide before posting to the list.

list of date sections of archive