[R] Can we do GLM on 2GB data set with R?

From: WILLIE, JILL <JILWIL_at_SAFECO.com>
Date: Sun 21 Jan 2007 - 01:26:50 GMT


We are wanting to use R instead of/in addition to our existing stats package because of it's huge assortment of stat functions. But, we routinely need to fit GLM models to files that are approximately 2-4GB (as SQL tables, un-indexed, w/tinyint-sized fields except for the response & weight variables). Is this feasible, does anybody know, given sufficient hardware, using R? It appears to use a great deal of memory on the small files I've tested.

I've read the data import, memory.limit, memory.size & general documentation but can't seem to find a way to tell what the boundaries are & roughly gauge the needed memory...other than trial & error. I've started by testing the data.frame & run out of memory on my PC. I'm new to R so please be forgiving if this is a poorly-worded question.

Jill Willie
Open Seas
Safeco Insurance

jilwil@safeco.com
206-545-5673



R-help@stat.math.ethz.ch mailing list
https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code. Received on Sun Jan 21 12:33:05 2007

Archive maintained by Robert King, hosted by the discipline of statistics at the University of Newcastle, Australia.
Archive generated by hypermail 2.1.8, at Sun 21 Jan 2007 - 09:30:24 GMT.

Mailing list information is available at https://stat.ethz.ch/mailman/listinfo/r-help. Please read the posting guide before posting to the list.