Re: [Rd] avoid copying big object passed into optimize()

From: Zepu Zhang <zepu.zhang_at_gmail.com>
Date: Thu, 17 Mar 2011 18:51:45 -0800

Thanks! I found, say, exp(x) causes 2 duplications whereas sum(x) 0 duplication. Is there any document to learn from about this?

(first time list user. sorry if anything about the posting procedure is wrong)

On Wed, Mar 9, 2011 at 6:16 PM, Matt Shotwell <matt_at_biostatmatt.com> wrote:

> On Wed, 2011-03-09 at 17:15 -0900, Zepu Zhang wrote:
> > Hello list,
> >
> > I have the following scenario:
> >
> > f1 <- function(a)
> > {
> > .... # doing things; may need 'a', but does not change 'a'.
> >
> > g <- function(x)
> > {
> > sum(x + a) # Say. Use 'a'; does not change 'a'.

>

> The expression 'x + a' causes 'a' to be duplicated; 'x' is added to each
> element of the duplicated vector, then returned. The sum occurs
> afterward. To avoid this use an expression like: 'length(a) * x +
> sum(a)'. Also, please see this recent thread regarding the
> pass-by-value / pass-by-reference issue:
> http://tolstoy.newcastle.edu.au/R/e13/help/11/03/6632.html
>

> > }
> >
> > optimize(f = g, lower = 0, upper = 1)
> > }
> >
> >
> > f2 <- function()
> > {
> > b <- runif(100000000000) # Create big object.
> >
> > f1(a = b)
> > }
> >
> >
> > My main concern is to reduce copying of the big object 'a'. Questions:
> >
> > (1) In f1, 'a' never appears on the LHS of assignment. Is it passed by
> value
> > or by reference? Say the situation is simpler and more general: no
> > optimization call in f1.
>

> 'a' is passed by value, but not necessarily copied in memory.
>

> > (2) Is there any difference, as far as copying of the big 'a' is
> concerned,
> > if 'g' is changed to
> > g <- function(x, b) { sum(x + b) }
> > and called by
> > optimize(f = g, lower = 0, upper = 1, b = a)
>

> No.
>

> > (3) Is 'a' passed into the C optimization function one-off, or again and
> > again across the C-R interface?
>

> I don't think either is completely correct. But more to your point, 'a'
> is not necessarily copied repeatedly. If you make the substitution I
> suggested above for 'g', then 'a' is not repeatedly copied.
>

> > (4) Does it help if I remove the argument 'a' of 'f1', and let 'g' look
> for
> > it (of course it should be referred to as 'b' now) directly in the
> > environment of 'f2'?
>

> No. 'g' would then search and find 'a' farther down the environment
> tree.
>

> > (5) Any suggestions?
>

> Avoid operations that necessitate a copy. Compile R with
> --enable-memory-profiling and use the tracemem function to help in this.
>

> > Many thanks for your help!
> >
> > Zepu
> >
> > [[alternative HTML version deleted]]
> >
> > ______________________________________________
> > R-devel_at_r-project.org mailing list
> > https://stat.ethz.ch/mailman/listinfo/r-devel
>
>
>

	[[alternative HTML version deleted]]

______________________________________________
R-devel_at_r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel Received on Fri 18 Mar 2011 - 03:01:31 GMT

Archive maintained by Robert King, hosted by the discipline of statistics at the University of Newcastle, Australia.
Archive generated by hypermail 2.2.0, at Fri 18 Mar 2011 - 14:15:34 GMT.

Mailing list information is available at https://stat.ethz.ch/mailman/listinfo/r-devel. Please read the posting guide before posting to the list.

list of date sections of archive