Re: [Rd] Arrays Partial unserialization

From: Duncan Murdoch <murdoch.duncan_at_gmail.com>
Date: Fri, 31 Aug 2012 10:41:22 -0400

On 31/08/2012 9:47 AM, Damien Georges wrote:
> Hi all,
>
> I'm working with some huge array in R and I need to load several ones to
> apply some functions that requires to have all my arrays values for each
> cell...
>
> To make it possible, I would like to load only a part (for example 100
> cells) of all my arrays, apply my function, delete all cells loaded,
> loaded following cells and so on.
>
> Is it possible to unserialize (or load) only a defined part of an R array ?
> Do you know some tools that might help me?

I don't know of any tools to do that, but there are tools to maintain large objects in files, and load only parts of them at a time, e.g. the ff package. Or you could simply use readBin and writeBin to do the same yourself.
>
> Finally, I did lot of research to find the way array (and all other R
> object) are serialized into binary object, but I found nothing
> explaining really algorithms involved. If someone has some information
> on this topic, I'm interesting in.

You can read the source for this; it is in src/main/serialize.c.

Duncan Murdoch
>
> Hoping my request is understandable,
>
> All the best,
>
> Damien.G
>
> ______________________________________________
> R-devel_at_r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel



R-devel_at_r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel Received on Fri 31 Aug 2012 - 14:44:17 GMT

This quarter's messages: by month, or sorted: [ by date ] [ by thread ] [ by subject ] [ by author ]

All messages

Archive maintained by Robert King, hosted by the discipline of statistics at the University of Newcastle, Australia.
Archive generated by hypermail 2.2.0, at Sat 01 Sep 2012 - 00:50:41 GMT.

Mailing list information is available at https://stat.ethz.ch/mailman/listinfo/r-devel. Please read the posting guide before posting to the list.

list of date sections of archive