[R] Odp: reading a big file

From: Petr PIKAL <petr.pikal_at_precheza.cz>
Date: Thu, 24 May 2007 10:22:44 +0200


Hi

One possibility is to use scan with combination of skip and nlines parameters in a body of for cycle and adding read portion to some suitable object.

Regards

Petr

petr.pikal_at_precheza.cz

r-help-bounces_at_stat.math.ethz.ch napsal dne 23.05.2007 19:38:33:

> Dear All,
>
> I am on WindowsXP with 512 MB of RAM, R 2.4.0, and I want to read in a
> big file mln100.txt. The file is 390MB big, it contains a column of 100
> millions integers.
>
> > mln100=scan("mln100.txt")
> Error: cannot allocate vector of size 512000 Kb
> In addition: Warning messages:
> 1: Reached total allocation of 511Mb: see help(memory.size)
> 2: Reached total allocation of 511Mb: see help(memory.size)
>
> In fact, I would be quite happy if I could read, say, every tenth
> integer (line) of the file. Is it possible to do this?
>
> Cheers,
> Rem
>
> ______________________________________________
> R-help_at_stat.math.ethz.ch mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide
http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.



R-help_at_stat.math.ethz.ch mailing list
https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code. Received on Thu 24 May 2007 - 08:33:44 GMT

Archive maintained by Robert King, hosted by the discipline of statistics at the University of Newcastle, Australia.
Archive generated by hypermail 2.2.0, at Thu 24 May 2007 - 09:31:21 GMT.

Mailing list information is available at https://stat.ethz.ch/mailman/listinfo/r-help. Please read the posting guide before posting to the list.