Re: [R] importing a VERY LARGE database from Microsoft SQL into R

From: Sean Davis <sdavis2_at_mail.nih.gov>
Date: Wed 25 Jan 2006 - 08:19:13 EST

On 1/24/06 3:28 PM, "r user" <ruser2006@yahoo.com> wrote:

> I am using R 2.1.1 in a Windows Xp environment.
>
> I need to import a large database from Microsoft SQL
> into R.
>
> I am currently using the ³sqlQuery² function/command.
>
> This works, but I sometimes run out of memory if my
> database is too big, or it take quite a long time for
> the data to import into R.
>
> Is there a better way to bring a large SQL database
> into R?

How are you using sqlQuery? The power of SQL is that you can fetch records on demand. If you don't need this functionality, then you may want to just dump to a text file from Microsoft SQL and read into R.

> IS there an efficient way to convert the data into R
> format prior to bringing it into R? (E.g. directly
> from Microsoft SQL?)

You could probably check into how Microsoft SQL dumps a table.

Also, if you haven't read this:

http://cran.r-project.org/doc/manuals/R-data.html

it is probably a good time to look at it--it is full of tips for doing data import from various sources including SQL databases.

Sean



R-help@stat.math.ethz.ch mailing list
https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html Received on Wed Jan 25 10:30:10 2006

This archive was generated by hypermail 2.1.8 : Wed 25 Jan 2006 - 14:08:12 EST