Re: [R] Memory question

From: Prof Brian Ripley <ripley_at_stats.ox.ac.uk>
Date: Thu 14 Jul 2005 - 21:29:25 EST

On Thu, 14 Jul 2005, Kenneth Cabrera wrote:

> Thank you Dr. Spencer Graves for your answer.
>
> What kind of matrices? They come form an image of about 3000x5000, and
> I need to generate arround 1024 matrices of the same size, they are not
> sparse
> matrices.
>
> What function can I use to, once generated one matrix, I could save into disk
> and then use the same space for the following, and so on.

You can use either save or .saveRDS/serialize followed by rm() and gc(). You cannot use the same space, but you can free up the space.

Then when you need the data again, load/.readRDS/unserialize can pull the object back. (If you arrange this right the object will only go into a temporary frame and so only be needed one at a time.)

>
> Thank you very much for your help
>
> Kenneth
>
> Spencer Graves wrote:
>
>> What kinds of matrices? There are facilities in the Matrix and
>> SparseM packages that might help for sparse matrices. If they are N x k
>> where N is large and k is not, can you compute something like the QR
>> decomposition and get away with keeping only the R part for most of your
>> matrices?
>>
>> One could potentially define a class of matrices that are only kept
>> in memory only when needed; I think S-Plus may do that. It would take a
>> lot of work to make that work generally, but you might be able to
>> accomplish what you need with a much smaller effort.
>>
>> spencer graves
>>
>> Kenneth Roy Cabrera Torres wrote:
>>
>>
>>> Hi R users and developers:
>>>
>>> I want to know how can I save memory in R
>>> for example:
>>> - saving on disk a matrix.
>>> - using again the matrix (changing their values)
>>> - saving again the matrix on disk in a different file.
>>>
>>> The idea is that I have a process that generate several
>>> matrices, but if I keep them all in memory it will overflow.
>>>
>>> How can I save them in different files, so I use the same
>>> amount of memory for each processed matrix?
>>>
>>> Thank you for your help.
>>>
>>>
>>
>>
>

-- 
Brian D. Ripley,                  ripley@stats.ox.ac.uk
Professor of Applied Statistics,  http://www.stats.ox.ac.uk/~ripley/
University of Oxford,             Tel:  +44 1865 272861 (self)
1 South Parks Road,                     +44 1865 272866 (PA)
Oxford OX1 3TG, UK                Fax:  +44 1865 272595

______________________________________________
R-help@stat.math.ethz.ch mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html
Received on Thu Jul 14 21:36:33 2005

This archive was generated by hypermail 2.1.8 : Fri 03 Mar 2006 - 03:33:40 EST