Re: [Rd] [R] Suspected memory leak with R v.2.5.x and large matrices with dimnames set

From: Peter Waltman <waltman_at_cs.nyu.edu>
Date: Sat, 18 Aug 2007 17:13:46 -0400

Hi Seth -

Thanks for the follow up. I'll definitely check out the devel version at some point since while I've come up with a workaround, this is causing problems for me as it uses up so much memory on some systems that R starts throwing malloc errors and has to be killed from the command line. The machine I'm thinking of in particular is a MacOS machine with 8 gigs of memory.

Also, having the row and column names set to alphanumeric names causes the processing to slow down significantly - as much as by a power of 10 (or more).

As for you speculation that the memory released by R may not be recognized as being free'd by the OS, as a further test, I re-ran my code snippet three consecutive times w/in the same R interpreter window. In theory, if there were a memory leak, after the first run (resulting in a memory stamp of 2 gig), the subsequent runs would further increase R's memory stamp, i.e. up to 4 after the second, and 6 for the 3rd.

This didn't happen, and R's stamp remained at 2 gig, so I can only assume that you're correct and I was wrong about a leak.

Still, it's quite the memory hog when using dimnames, so I'll have to avoid those for now and will try the devel version you mentioned.

Thanks and have a good weekend,

Peter

Seth Falcon wrote:
> Hi Peter,
>
> Peter Waltman <waltman_at_cs.nyu.edu> writes:
>
>> Admittedly, this may not be the most sophisticated memory profiling
>> performed, but when using unix's top command, I'm noticing a notable
>> memory leak when using R with a large matrix that has dimnames
>> set.
>>
>
> I'm not sure I understand what you are reporting. One thing to keep
> in mind is that how memory released by R is handled is OS dependent
> and one will often observe that after R frees some memory, the OS does
> not report that amount as now free.
>
> Is what you are observing preventing you from getting things done, or
> just a concern that there is a leak that needs fixing? It is worth
> noting that the internal handling of character vectors has changed in
> R-devel and so IMO testing there would make sense before persuing this
> further, I suspect your results will be different.
>
> + seth
>
>



R-devel_at_r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel Received on Mon 20 Aug 2007 - 06:39:18 GMT

Archive maintained by Robert King, hosted by the discipline of statistics at the University of Newcastle, Australia.
Archive generated by hypermail 2.2.0, at Mon 20 Aug 2007 - 20:39:56 GMT.

Mailing list information is available at https://stat.ethz.ch/mailman/listinfo/r-devel. Please read the posting guide before posting to the list.