Re: [R] load ing and saving R objects

From: Prof Brian Ripley <ripley_at_stats.ox.ac.uk>
Date: Wed 15 Jun 2005 - 00:41:10 EST

On Tue, 14 Jun 2005, Barry Rowlingson wrote:

> Richard Mott wrote:
>> Does anyone know a way to do the following:
>>
>> Save a large number of R objects to a file (like load() does) but then
>> read back only a small named subset of them . As far as I can see,
>> load() reads back everything.
>
> Save them to individual files when you generate them?
>
> for(i in 1:15000){
>
> m=generateBigMatrix(i)
>
> filename=paste("BigMatrix-",i,".Rdata",sep='')
> save(m,file=filename)
> }
>
> Note that load will always overwrite 'm', so to load a sample of them in
> you'll need to do something like this:
>
> bigSamples=list()
>
> for(i in sample(15000,N)){
> filename=paste("BigMatrix-",i,".Rdata",sep='')
> load(filename)
> bigSamples[[i]]=m
> }
>
> But there may be a more efficient way to string up a big list like
> that, I can never remember - get it working, then worry about optimisation.

(Yes, use bigSamples <- vector("list", 15000) first.)

> I hope your filesystem is happy with 15000 objects in it. I would
> dedicate a folder or directory for just these objects' files, since it
> then becomes near impossible to see anything other than the big matrix
> files...

.readRDS/.saveRDS might be a better way to do this, and avoids always restoring to "m".

If your file system does not like 15000 files you can always save in a DBMS. I did once look into restoring just some of the objects in a save()ed file, but it is not really possible to do so efficiently due to sharing between objects.

-- 
Brian D. Ripley,                  ripley@stats.ox.ac.uk
Professor of Applied Statistics,  http://www.stats.ox.ac.uk/~ripley/
University of Oxford,             Tel:  +44 1865 272861 (self)
1 South Parks Road,                     +44 1865 272866 (PA)
Oxford OX1 3TG, UK                Fax:  +44 1865 272595

______________________________________________
R-help@stat.math.ethz.ch mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html
Received on Wed Jun 15 00:50:01 2005

This archive was generated by hypermail 2.1.8 : Fri 03 Mar 2006 - 03:32:36 EST