RE: [R] R for large data sets

About this list Date view Thread view Subject view Author view Attachment view

From: wei, xiaoyan (wei_xiaoyan@emc.com)
Date: Wed 16 Jan 2002 - 08:11:36 EST


Message-id: <FA2F59D0E55B4B4892EA076FF8704F55380028@srgraham.eng.emc.com>

Where is R-patched? I didnot see it in index of /R/CRAN/src/base today. I
remeber seeing it a few days ago.

Thanks.

-----Original Message-----
From: Prof Brian Ripley [mailto:ripley@stats.ox.ac.uk]
Sent: Tuesday, January 15, 2002 2:43 PM
To: wei, xiaoyan
Cc: r-help@stat.math.ethz.ch
Subject: Re: [R] R for large data sets

On Tue, 15 Jan 2002, wei, xiaoyan wrote:

> As a part of our regular data analysis, I have to read in large data sets
> with six columns and about a million rows. In Splus, this usually take a
> couple of minutes. I just tried R, it seems take forever to use
read.table()
> to read in the data frame! It did not help much even though I specified
> colClasses and nrows in read.table().
>
> How is R's ability to analyze large data sets? I used R on solaris 2.6 and
I
> used all default compilation flags when building the R package. Will it
help
> if I use some compilation flags with higher optimization level?

It will help to use R-patched, since I guess you are using 1.4.0.
Also, look in the list archives, as I answered this more fully earlier
today.

In either S-PLUS or R, scan would be a better choice for such a dataset.

-- 
Brian D. Ripley,                  ripley@stats.ox.ac.uk
Professor of Applied Statistics,  http://www.stats.ox.ac.uk/~ripley/
University of Oxford,             Tel:  +44 1865 272861 (self)
1 South Parks Road,                     +44 1865 272860 (secr)
Oxford OX1 3TG, UK                Fax:  +44 1865 272595
-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-
r-help mailing list -- Read http://www.ci.tuwien.ac.at/~hornik/R/R-FAQ.html
Send "info", "help", or "[un]subscribe"
(in the "body", not the subject !)  To: r-help-request@stat.math.ethz.ch
_._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._


About this list Date view Thread view Subject view Author view Attachment view

This archive was generated by hypermail 2.1.3 : Thu 17 Jan 2002 - 11:10:11 EST