Re: [R] advice requested re: building "good" system (R, SQL db) for handling large datasets

From: Rainer M Krug <r.m.krug_at_gmail.com>
Date: Wed, 6 Feb 2008 14:34:52 +0200

With databases, one issue that might be relevant is whether you want to
> store data in tables (e.g. one table to store one data.frame) that can
> subsequently be manipulated in the DB, or to store R objects as R
> objects (e.g. as BLOBs). My situation is likely to be the later case,
> and one of my concerns is that many DBs have an upper limit of 2GB on
> BLOBs, and I might potentially have objects that are larger than this.

R objects in blobs - I never thought about that. Could you elaborate on how to do something like that (I am using RMySQL)?

Thanks

Rainer

Finally, you might get more response on database issues from R-sig-db
> than R-help.
>
> Best wishes
>
> Richard.
>
>
> Thomas Pujol wrote:
> > R-community,
> > Sometime during the next 12-months, I plan on configuring a new computer
> system on which I will primarily run "R" and a SQL database (Microsoft SQL
> Server, MySQL, Oracle, etc). My primary goal is to "optimize" the system
> for R, and for passing data to and from R and the database.
> >
> > I work with large datasets, and therefore I "think" one of my most
> important goals should be to maximize the amount of RAM that R can utilize
> effectively.
> >
> > I am seeking advice concerning the version of R, OS, processor,
> hard-drive/storage configuration, database, etc. that I should consider. (I
> am guessing that I should build a system with lots of RAM, and a Linux OS,
> but am seeking advice from the R community.) If I choose Linux, does it
> matter which version I use? Any opinion regarding implementing a
> commercially supported version from a vendor such as Red Hat, Sun, etc? Is
> any database particularly better at "exchanging" data with R?
> >
> > While cost is of course a consideration, it is probably a secondary
> consideration to overall performance, reliability, and ease of ongoing
> maintenance/support.
> >
> > Thanks!
> >
> >
> > ---------------------------------
> >
> > [[alternative HTML version deleted]]
> >
> > ______________________________________________
> > R-help_at_r-project.org mailing list
> > https://stat.ethz.ch/mailman/listinfo/r-help
> > PLEASE do read the posting guide
> http://www.R-project.org/posting-guide.html
> > and provide commented, minimal, self-contained, reproducible code.
> >
> >
>
> ______________________________________________
> R-help_at_r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide
> http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>

-- 

-- 
Rainer M. Krug, Dipl. Phys. (Germany), MSc Conservation Biology (UCT)

Plant Conservation Unit Department of Botany
University of Cape Town
Rondebosch 7701
South Africa

	[[alternative HTML version deleted]]

______________________________________________
R-help_at_r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Received on Wed 06 Feb 2008 - 12:37:03 GMT

Archive maintained by Robert King, hosted by the discipline of statistics at the University of Newcastle, Australia.
Archive generated by hypermail 2.2.0, at Wed 06 Feb 2008 - 16:30:12 GMT.

Mailing list information is available at https://stat.ethz.ch/mailman/listinfo/r-help. Please read the posting guide before posting to the list.

list of date sections of archive