Re: [R] Error in vector("double", length) : vector size specified is too large....VLDs

From: Tom Colson <tom_colson_at_ncsu.edu>
Date: Fri 16 Sep 2005 - 06:15:55 EST


Yes, using geoR.

I can interpolate the DEM quite easily in Grass (v.surf.rst, kriging) and block kriging in ArcInfo. What we need, though, is to be able to "estimate" or even nail down the variogram for these data sets. Where am I going with this? I'm guessing that variables such as slope, ruggedness, etc.. are going to produce different sill, range, and nugget values, which I can then use to fine tune the interpolation process, rather than using the same spline or kriging parameters on say, a whole state boundary worth of Lidar data. And yes, I can estimate the variogram in ArcInfo (limited to 10000 points) and can also import the DEM from grass into R using spgrass....but the point is to analyze the point data BEFORE I make the DEM.

So I'm guessing the geoR isn't ever going to handle this size data, and I need to be using gstat? (As I write this, gstat(variogram) is plugging away for last 10 minute with no errors.....)

Thanks for quick replies

Thomas Colson
North Carolina State University
Department of Forestry and Environmental Resources (919) 673 8023
tom_colson@ncsu.edu

Calendar:
www4.ncsu.edu/~tpcolson  

-----Original Message-----
From: Roger Bivand [mailto:Roger.Bivand@nhh.no] Sent: Thursday, September 15, 2005 3:28 PM To: Peter Dalgaard
Cc: Tom Colson; r-help@stat.math.ethz.ch Subject: Re: [R] Error in vector("double", length) : vector size specified is too large....VLDs

On 15 Sep 2005, Peter Dalgaard wrote:

> "Tom Colson" <tom_colson@ncsu.edu> writes:
>
> >
> > > rm(data1)
> > > variog.1.b <- variog(raw.data)
> > variog: computing omnidirectional variogram Error in
> > vector("double", length) : vector size specified is too large
> >
> > Turns out I was wrong re: # of rows...it's 304,000
> >
> >
> > Same problem. Version is 2.1.1, hardware is Dual Xeon 3.6 4 GB RAM,
> > XP Pro
> > 64 Bit. Can reproduce the problem with 64Bit R 2.1.1 running on
> > Fedora 4, same hardware.
> >
>
> Variograms involve the differences between all pairs of points which
> can become a rather large number of values. 304000*303999/2 in your
> case, about 344GB by my reckoning. And the distances between them
> makes for a similar quantity.
>
> Now, some algorithms may be smarter than to keep all values in memory,
> but you haven't even told us where you got the variog() from. It
> doesn't seem to be in the standard packages, although we do have
> variogram() and Variogram() in spatial and nlme.

Right, this is from geoR, which uses full matrices. I think both fields and gstat can work with larger data sets. Whether model-based geostatistics is what you need for interpolating a digital elevation model is another question.

>
>

-- 
Roger Bivand
Economic Geography Section, Department of Economics, Norwegian School of
Economics and Business Administration, Helleveien 30, N-5045 Bergen,
Norway. voice: +47 55 95 93 55; fax +47 55 95 95 43
e-mail: Roger.Bivand@nhh.no

______________________________________________
R-help@stat.math.ethz.ch mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html
Received on Wed Sep 21 02:44:32 2005

This archive was generated by hypermail 2.1.8 : Sun 23 Oct 2005 - 17:23:22 EST