Re: [R] How to read HUGE data sets?

From: Gabor Grothendieck <ggrothendieck_at_gmail.com>
Date: Thu, 28 Feb 2008 17:16:35 -0500

read.table's colClasses= argument can take a "NULL" for those columns that you want
ignored. Also see the skip= argument. ?read.table .

The sqldf package can read a subset of rows and columns (actually any sql operation)
from a file larger than R can otherwise handle. It will automatically set up a temporary
SQLite database for you, load the file into the database without going through R and
extract just the data you want into R and then automatically delete the database. All this
can be done in 2 lines of code. See example 6 on the home page: http://sqldf.googlecode.com

On Thu, Feb 28, 2008 at 12:03 AM, Jorge Iván Vélez <jorgeivanvelez_at_gmail.com> wrote:
> Dear R-list,
>
> Does somebody know how can I read a HUGE data set using R? It is a hapmap
> data set (txt format) which is around 4GB. After read it, I need to delete
> some specific rows and columns. I'm running R 2.6.2 patched over XP SP2
> using a 2.4 GHz Core 2-Duo processor and 4GB RAM. Any suggestion would be
> appreciated.
>
> Thanks in advance,
>
> Jorge
>
> [[alternative HTML version deleted]]
>
> ______________________________________________
> R-help_at_r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>



R-help_at_r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code. Received on Thu 28 Feb 2008 - 23:09:37 GMT

Archive maintained by Robert King, hosted by the discipline of statistics at the University of Newcastle, Australia.
Archive generated by hypermail 2.2.0, at Fri 29 Feb 2008 - 14:00:18 GMT.

Mailing list information is available at https://stat.ethz.ch/mailman/listinfo/r-help. Please read the posting guide before posting to the list.

list of date sections of archive