Re: [Rd] avoid copying big object passed into optimize()

From: Matt Shotwell <matt_at_biostatmatt.com>
Date: Wed, 09 Mar 2011 22:16:07 -0500

On Wed, 2011-03-09 at 17:15 -0900, Zepu Zhang wrote:
> Hello list,
>
> I have the following scenario:
>
> f1 <- function(a)
> {
> .... # doing things; may need 'a', but does not change 'a'.
>
> g <- function(x)
> {
> sum(x + a) # Say. Use 'a'; does not change 'a'.

The expression 'x + a' causes 'a' to be duplicated; 'x' is added to each element of the duplicated vector, then returned. The sum occurs afterward. To avoid this use an expression like: 'length(a) * x + sum(a)'. Also, please see this recent thread regarding the pass-by-value / pass-by-reference issue: http://tolstoy.newcastle.edu.au/R/e13/help/11/03/6632.html

> }
>
> optimize(f = g, lower = 0, upper = 1)
> }
>
>
> f2 <- function()
> {
> b <- runif(100000000000) # Create big object.
>
> f1(a = b)
> }
>
>
> My main concern is to reduce copying of the big object 'a'. Questions:
>
> (1) In f1, 'a' never appears on the LHS of assignment. Is it passed by value
> or by reference? Say the situation is simpler and more general: no
> optimization call in f1.

'a' is passed by value, but not necessarily copied in memory.

> (2) Is there any difference, as far as copying of the big 'a' is concerned,
> if 'g' is changed to
> g <- function(x, b) { sum(x + b) }
> and called by
> optimize(f = g, lower = 0, upper = 1, b = a)

No.

> (3) Is 'a' passed into the C optimization function one-off, or again and
> again across the C-R interface?

I don't think either is completely correct. But more to your point, 'a' is not necessarily copied repeatedly. If you make the substitution I suggested above for 'g', then 'a' is not repeatedly copied.

> (4) Does it help if I remove the argument 'a' of 'f1', and let 'g' look for
> it (of course it should be referred to as 'b' now) directly in the
> environment of 'f2'?

No. 'g' would then search and find 'a' farther down the environment tree.

> (5) Any suggestions?

Avoid operations that necessitate a copy. Compile R with --enable-memory-profiling and use the tracemem function to help in this.

> Many thanks for your help!
>
> Zepu
>
> [[alternative HTML version deleted]]
>
> ______________________________________________
> R-devel_at_r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel



R-devel_at_r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel Received on Thu 10 Mar 2011 - 03:20:17 GMT

Archive maintained by Robert King, hosted by the discipline of statistics at the University of Newcastle, Australia.
Archive generated by hypermail 2.2.0, at Fri 18 Mar 2011 - 07:20:33 GMT.

Mailing list information is available at https://stat.ethz.ch/mailman/listinfo/r-devel. Please read the posting guide before posting to the list.

list of date sections of archive