Re: [Rd] External pointers and an apparent memory leak

From: Matt Shotwell <matt_at_biostatmatt.com>
Date: Thu, 15 Sep 2011 09:05:35 -0500

James,

Works for me!

In a file, 'test.c', I have:

void h5R_allocate_finalizer(SEXP eptr) {

    char* vector = (char*) R_ExternalPtrAddr(eptr);     free(vector);
    printf("memory freed at address %lu\n", (long unsigned) vector);     R_ClearExternalPtr(eptr);
}

SEXP h5R_allocate(SEXP size) {

    int i = INTEGER(size)[0];
    char* vector = (char*) malloc(i*sizeof(char));     printf("memory allocated at address %lu\n", (long unsigned) vector);     SEXP e_ptr = R_MakeExternalPtr(vector, R_NilValue, R_NilValue);     R_RegisterCFinalizerEx(e_ptr, h5R_allocate_finalizer, TRUE);     return e_ptr;
}

I generated the shared library with R CMD SHLIB. Then, at the R prompt:

R> dyn.load('test.so')
R> system(paste('ps -eo pid,vsz | grep', Sys.getpid())) 10961 77160
R> v <- .Call('h5R_allocate', as.integer(2^24)) memory allocated at address 140127935406096 R> system(paste('ps -eo pid,vsz | grep', Sys.getpid())) 10961 93548
R> rm(v); gc()
memory freed at address 140127935406096

         used (Mb) gc trigger (Mb) max used (Mb)
Ncells 132769  7.1     350000 18.7   350000 18.7
Vcells 129312  1.0     786432  6.0   555216  4.3
R> system(paste('ps -eo pid,vsz | grep', Sys.getpid())) 10961 77160

It appears that the external pointer is behaving as expected. However, I did find that gc() may not immediately collect the removed object 'v'. Sometimes it took more than one call. In any case, the memory was freed when R exits.

Best,
Matt

On Wed, 2011-09-14 at 21:21 +0000, James Bullard wrote:
> I'm using external pointers and seemingly leaking memory. My determination of a memory leak is that the R process continually creeps up in memory as seen by top while the usage as reported by gc() stays flat. I have isolated the C code:
>
> void h5R_allocate_finalizer(SEXP eptr) {
> Rprintf("Calling the finalizer\n");
> void* vector = R_ExternalPtrAddr(eptr);
> free(vector);
> R_ClearExternalPtr(eptr);
> }
>
> SEXP h5R_allocate(SEXP size) {
> int i = INTEGER(size)[0];
> char* vector = (char*) malloc(i*sizeof(char));
> SEXP e_ptr = R_MakeExternalPtr(vector, R_NilValue, R_NilValue);
> R_RegisterCFinalizerEx(e_ptr, h5R_allocate_finalizer, TRUE);
> return e_ptr;
> }
>
>
> If I run an R program like this:
>
> v <- replicate(100000, {
> .Call("h5R_allocate", as.integer(1000000))
> })
> rm(v)
> gc()
>
> Then you can see the problem (top reports that R still has a bunch of memory, but R doesn't think it does). I have tried using valgrind and it says I have memory left on the table at the end lest you think it is because top. Also, I have tried Free/Calloc as well and this doesn't make a difference. Finally, I see this in both R-2-12 (patched) and R-2-13 - I think it is more an understanding issue on my part.
>
> thanks much in advance, to me it really resembles the connection.c code, but what am I missing?
>
> thanks, jim
>
>
> [[alternative HTML version deleted]]
>
> ______________________________________________
> R-devel_at_r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel



R-devel_at_r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel Received on Thu 15 Sep 2011 - 14:10:34 GMT

This quarter's messages: by month, or sorted: [ by date ] [ by thread ] [ by subject ] [ by author ]

All messages

Archive maintained by Robert King, hosted by the discipline of statistics at the University of Newcastle, Australia.
Archive generated by hypermail 2.2.0, at Thu 15 Sep 2011 - 17:40:31 GMT.

Mailing list information is available at https://stat.ethz.ch/mailman/listinfo/r-devel. Please read the posting guide before posting to the list.

list of date sections of archive