Re: [Rd] relist, an inverse operator to unlist

From: Andrew Clausen <clausen_at_econ.upenn.edu>
Date: Sat, 19 May 2007 11:36:36 -0400

Hi all,

I've written a new version of relist that includes the suggestions from Gabor and Martin:

        http://www.econ.upenn.edu/~clausen/computing/relist.R

The leading example now looks like this:

	initial.param <- list(mean=c(0, 1), vcov=cbind(c(1, 1), c(1, 0)))
	initial.param <- as.relistable(initial.param)

	ll <- function(param.vector)
	{
		param <- relist(initial.param)
		-sum(dnorm(x, mean=param$mean, vcov=param$vcov, log=TRUE))
		# note: dnorm doesn't do vcov... but I hope you get the point
	}

	optim(unlist(initial.param), ll)

One thing that concerns me is that relist() needs to count how many items are in a structued object. In this version, it does that by repeatedly calling

        length(unlist(obj))

which is quite inefficient (O(n^2) time, where n is the depth of the structure).

Is there a clean way of making this faster? I suppose relist() could return attach a "length" attribute to the object it returns.

Apart from that, I suppose I should do these things before inclusion:

What's the usual process? Email a patch to R-core_at_r-project.org?

Cheers,
Andrew

On Mon, May 14, 2007 at 09:53:31AM +0200, Martin Maechler wrote:
> Nice ideas, Gabor and Andrew.
>
> While I agree with Andrew that such a utility makes for nicer
> and considerably better maintainable code in examples like his,
> and I do like to provide "inverse operator functions" in R
> whenever sensible,
> OTOH, we have strived to keep R's "base" package as lean and
> clean as possible, so I think this had to go to "utils".
>
> One further small proposal: I'd use class name "relistable"
> since that's what the object of this class are
> and hence as.relistable().
>
> What do other R-develers think?
> Martin
>
> >>>>> "GaGr" == Gabor Grothendieck <ggrothendieck_at_gmail.com>
> >>>>> on Mon, 14 May 2007 02:54:22 -0400 writes:
>
> GaGr> unlist would not attach a skeleton to every vector it
> GaGr> returns, only the relist method of unlist would.
> GaGr> That way just that method needs to be added and no
> GaGr> changes to unlist itself are needed.
>
> GaGr> Before applying unlist to an object you would coerce
> GaGr> the object to class "relist" to force the relist
> GaGr> method of unlist to be invoked.
>
> GaGr> Here is an outline of the code:
>
> GaGr> as.relist <- function(x) {
> GaGr> if (!inherits(x, "relist")) class(x) <- c("relist", class(x))
> GaGr> x
> GaGr> }
>
> GaGr> unlist.relist <- function(x, ...) {
> GaGr> y <- x
> GaGr> cl <- class(y)
> GaGr> class(y) <- cl[- grep("relist", cl)]
> GaGr> z <- unlist(y)
> GaGr> attr(z, "relist") <- y
> GaGr> as.relist(z)
> GaGr> }
>
> GaGr> relist <- function(x, skeleton = attr(x, "relist")) {
> GaGr> # simpler version of relist so test can be executed
> GaGr> skeleton
> GaGr> }
>
> GaGr> # test
> GaGr> x <- list(a = 1:2, b = 3)
> GaGr> class(as.relist(x))
> GaGr> unlist(as.relist(x))
> GaGr> relist(unlist(as.relist(x)))
>
>
> GaGr> On 5/14/07, Andrew Clausen <clausen_at_econ.upenn.edu> wrote:
> >> Hi GaGr,
> >>
> >> Thanks for the interesting suggestion. I must confess I got lost -- is
> >> it something like this?
> >> * unlist() could attach skeleton to every vector it returns.
> >> * relist() could then use the skeleton attached to the vector to reconstruct
> >> the object. The interface might be
> >>
> >> relist <- function(flesh, skeleton=attributes(flesh)$skeleton)
> >>
> >> For example:
> >>
> >> par <- list(mean=c(0, 0), vcov(rbind(c(1, 1), c(1, 1))))
> >> vector.for.optim <- unlist(par)
> >> print(attributes(vector.optim)$skeleton) # the skeleton is stored!
> >> converted.back.again <- relist(par)
> >>
> >> Some concerns:
> >> * the metadata might get lost in some applications -- although it seems
> >> to work fine with optim(). But, if we provide both interfaces (where
> >> skeleton=flesh$skeleton is the default), then there should be no problem.
> >> * would there be any bad side-effects of changing the existing unlist
> >> interface? I suppose an option like "save.skeleton" could be added to unlist.
> >> I expect there would be some objections to enabling this as default behaviour,
> >> as it would significantly increase the storage requirements of the output.
> >>
> >> Cheers,
> >> Andrew
> >>
> >> On Sun, May 13, 2007 at 07:02:37PM -0400, GaGr Grothendieck wrote:
> >> > I suggest you define a "relist" class and then define an unlist
> >> > method for it which stores the skeleton as an attribute. Then
> >> > one would not have to specify skeleton in the relist command
> >> > so
> >> >
> >> > relist(unlist(relist(x))) === x
> >> >
> >> > 1. relist(x) is the same as x except it gets an additional class "relist".
> >> > 2. unlist(relist(x)) invokes the relist method of unlist on relist(x)
> >> > returning another relist object
> >> > 3. relist(unlist(relist(x))) then recreates relist(x)
> >> >
> >> >
> >> > On 5/13/07, Andrew Clausen <clausen_at_econ.upenn.edu> wrote:
> >> > >Hi all,
> >> > >
> >> > >I wrote a function called relist, which is an inverse to the existing
> >> > >unlist function:
> >> > >
> >> > > http://www.econ.upenn.edu/~clausen/computing/relist.R
> >> > >
> >> > >Some functions need many parameters, which are most easily represented in
> >> > >complex structures. Unfortunately, many mathematical functions in R,
> >> > >including optim, nlm, and grad can only operate on functions whose domain
> >> > >is
> >> > >a vector. R has a function to convert complex objects into a vector
> >> > >representation. This file provides an inverse operation called "unlist" to
> >> > >convert vectors back to the convenient structural representation.
> >> > >Together,
> >> > >these functions allow structured functions to have simple mathematical
> >> > >interfaces.
> >> > >
> >> > >For example, a likelihood function for a multivariate normal model needs a
> >> > >variance-covariance matrix and a mean vector. It would be most convenient
> >> > >to
> >> > >represent it as a list containing a vector and a matrix. A typical
> >> > >parameter
> >> > >might look like
> >> > >
> >> > > list(mean=c(0, 1), vcov=cbind(c(1, 1), c(1, 0)))
> >> > >
> >> > >However, optim can't operate on functions that take lists as input; it
> >> > >only likes vectors. The solution is conversion:
> >> > >
> >> > > initial.param <- list(mean=c(0, 1), vcov=cbind(c(1, 1), c(1, 0)))
> >> > >
> >> > > ll <- function(param.vector)
> >> > > {
> >> > > param <- relist(initial.param, param.vector)
> >> > > -sum(dnorm(x, mean=param$mean, vcov=param$vcov, log=TRUE))
> >> > > # note: dnorm doesn't do vcov... but I hope you get the
> >> > > point
> >> > > }
> >> > >
> >> > > optim(unlist(initial.param), ll)
> >> > >
> >> > >"relist" takes two parameters: skeleton and flesh. Skeleton is a sample
> >> > >object that has the right "shape" but the wrong content. "flesh" is a
> >> > >vector
> >> > >with the right content but the wrong shape. Invoking
> >> > >
> >> > > relist(skeleton, flesh)
> >> > >
> >> > >will put the content of flesh on the skeleton.
> >> > >
> >> > >As long as "skeleton" has the right shape, it should be a precise inverse
> >> > >of unlist. These equalities hold:
> >> > >
> >> > > relist(skeleton, unlist(x)) == x
> >> > > unlist(relist(skeleton, y)) == y
> >> > >
> >> > >Is there any easy way to do this without my new relist function? Is there
> >> > >any
> >> > >interest in including this in R's base package? (Or anywhere else?) Any
> >> > >comments on the implementation?
> >> > >
> >> > >Cheers,
> >> > >Andrew



R-devel_at_r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel Received on Sat 19 May 2007 - 15:40:40 GMT

Archive maintained by Robert King, hosted by the discipline of statistics at the University of Newcastle, Australia.
Archive generated by hypermail 2.2.0, at Sat 19 May 2007 - 19:34:22 GMT.

Mailing list information is available at https://stat.ethz.ch/mailman/listinfo/r-devel. Please read the posting guide before posting to the list.