RE: [Rd] R's IO speed

From: Vadim Ogranovich <vograno_at_evafunds.com>
Date: Tue 04 Jan 2005 - 08:02:13 EST


A technical question here: how does one measure the memory overhead mentioned below? I have a set of functions of my own and would like to profile them.

Thanks,
Vadim

> -----Original Message-----
> From: r-devel-bounces@stat.math.ethz.ch
> [mailto:r-devel-bounces@stat.math.ethz.ch] On Behalf Of Prof
> Brian Ripley
> Sent: Sunday, December 26, 2004 2:04 AM
> To: R-devel@r-project.org
> Subject: [Rd] R's IO speed
>
> R-devel now has some improved versions of read.table and write.table.
>
> For a million-row data frame containing one number, one
> factor with few levels and one logical column, a 56Mb object.
>
> generating it takes 4.5 secs.
>
> calling summary() on it takes 2.2 secs.
>
> writing it takes 8 secs and an additional 10Mb.
>
> saving it in .rda format takes 4 secs.
>
> reading it naively takes 28 secs and an additional 240Mb
>
> reading it carefully (using nrows, colClasses and
> comment.char) takes 16 secs and an additional 150Mb (56Mb of
> which is for the object read in).
> (The overhead of read.table over scan was about 2 secs,
> mainly in the conversion back to a factor.)
>
> loading from .rda format takes 3.4 secs.
>
> [R 2.0.1 read in 23 secs using an additional 210Mb, and wrote
> in 50 secs using an additional 450Mb.]
>
>
> Will Frank Harrell or someone else please explain to me a
> real application
> in which this is not fast enough?
>
> --
> Brian D. Ripley, ripley@stats.ox.ac.uk
> Professor of Applied Statistics, http://www.stats.ox.ac.uk/~ripley/
> University of Oxford, Tel: +44 1865 272861 (self)
> 1 South Parks Road, +44 1865 272866 (PA)
> Oxford OX1 3TG, UK Fax: +44 1865 272595
>
> ______________________________________________
> R-devel@stat.math.ethz.ch mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
>



R-devel@stat.math.ethz.ch mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel Received on Tue Jan 04 07:13:23 2005

This archive was generated by hypermail 2.1.8 : Fri 18 Mar 2005 - 09:02:27 EST