Re: [R] dyn.load, dyn.unload -memory management

From: Thomas Lumley <tlumley_at_u.washington.edu>
Date: Sat 29 Jul 2006 - 10:05:21 EST

On Fri, 28 Jul 2006, johan Faux wrote:

> hello everybody,
>
> I have some code which looks like:
>
> dyn.load("lpSolve.so")
> res <- lp(some.parameters)
>
> and everything runs fine.
>
> What lp() does, it's just calling a C function which is in "lpSolve.so"
>
> If I call lp() a large number of times:
>
> for(1 in 1:5000){
> gc(verbose=TRUE)
> res <- lp(some.parameters)
> }
>
> then gc() is showing that the memory used by R process remain almost constant all the time. But the system memory used is going up _very fast_ and the above code never succeed because the memory used rich the limit.
>> From this I'm drawing the conclusion that there is some memory leak on the C code called by lp() which gc() cannot report since this is a C process.
>
> Does all this makes any sense or I'm wrong?

gc() does track some C memory usage, but not memory obtained directly from the C implementation by malloc(). The symptoms are consistent with the C code allocating via malloc()/calloc() and then not free()ing the memory.

>
> The next thing I tried was
>
> for(1 in 1:5000){
> gc(verbose=TRUE)
> dyn.load("lpSolve.so")
> res <- lp(some.parameters)
> dyn.unload("lpSolve.so")
> }
>
> But the problem is still there. The memory used is going up really fast.
> Now my other question is: When I call dyn.unload() doesnt it terminate
> the previous C process? Next time in the loop, a new C process starts,
> and I will not have any problem with memory.

No, this won't help at all. dyn.unload() may or may not do anything on your system, but it certainly won't affect the way the C implementation handles memory. The only ways to return memory allocated by malloc() are to do a free() for every allocated chunk or to quit R.

> P.S. lpSolve.so above is from package "lpSolve". I thought that getting
> "lpSolve.so" and doing the above trick will solve my problem, but I have
> been surely wrong.
>

Indeed.

You need to either report this to the lpsolve package maintainers (with a reproducible example) and hope that they will fix it or look at the source code to find out which allocations are not being freed and fix them.

         -thomas



R-help@stat.math.ethz.ch mailing list
https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code. Received on Sat Jul 29 10:08:46 2006

Archive maintained by Robert King, hosted by the discipline of statistics at the University of Newcastle, Australia.
Archive generated by hypermail 2.1.8, at Sat 29 Jul 2006 - 12:16:45 EST.

Mailing list information is available at https://stat.ethz.ch/mailman/listinfo/r-help. Please read the posting guide before posting to the list.