Re: [R] Can R handle large dataset?

From: Richard Rowe <richard.rowe_at_jcu.edu.au>
Date: Thu, 15 May 2008 09:39:43 +1000

Mingjun Huang wrote:
> Hello,
>
> I am new to R, can anyone give me an idea of how R handle a large dataset
> (e.g. couple of Gbytes)? Thanks a lot!
>
>
In answer to your subject line: yes, not gigantic (multi-terrabyte), but large is likely to be OK.

How depends on what you are trying to do. If you are running a 64 bit OS with a terrabyte or more of RAM you shouldn't notice unless you are doing something messy (all bets are off if you are attempting to list all permutations of the data!); if you are running 32bit with a max size set by the OS of 2Gb then the answer is with much care and considerable cunning and possibly modification of your intermediate goals. Q1 is always - why are you dealing with such a big dataset? Is all the data equally informative? Because you can collect data doesn't mean you have to, or if you do insist on collecting it (presumably automatically), that it will be useful.

-- 
Dr Richard Rowe
Zoology & Tropical Ecology
School of Marine & Tropical Biology
James Cook University
Townsville 4811
AUSTRALIA

ph +61 7 47 81 4851
fax +61 7 47 25 1570
JCU has CRICOS Provider Code 00117J

______________________________________________
R-help_at_r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Received on Thu 15 May 2008 - 02:20:28 GMT

Archive maintained by Robert King, hosted by the discipline of statistics at the University of Newcastle, Australia.
Archive generated by hypermail 2.2.0, at Thu 15 May 2008 - 05:30:38 GMT.

Mailing list information is available at https://stat.ethz.ch/mailman/listinfo/r-help. Please read the posting guide before posting to the list.

list of date sections of archive