[R] loess function takes long to estimate

From: Robert A'gata <rhelpacc_at_gmail.com>
Date: Fri, 04 Mar 2011 21:52:41 -0500


I have 2 questions regarding using loess function in stats package.

  1. I have about 1 million points. I did loess with alpha=0.5 and degree=1 and 2 regressors. It has run for more than 5 hrs on 64bit 16CPU and 64GB server (with no other process running). I am wondering if this is usual? And if there is anyway to make it run faster?
  2. I noticed a difference in RAM consumption when specifying loess function differently. The first case is I ran loess(z ~ x+y,data=X,degree=1,alpha=.5) where my X has about 50 columns. The second case I trimmed X to only contain x,y and z. Then run loess(z ~ x+y, data=SmallX, degree=1, alpha=.5). I find that the second case consumes only 4% of RAM whereas the first case uses up to 33%. I'd like to know how loess handles input data under the hood? It seems to me it attaches the whole data into memory and hence results in what I observed. Is my understanding correct?

Thank you in advance.



R-help_at_r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code. Received on Sat 05 Mar 2011 - 02:57:12 GMT

Archive maintained by Robert King, hosted by the discipline of statistics at the University of Newcastle, Australia.
Archive generated by hypermail 2.2.0, at Sat 05 Mar 2011 - 03:10:19 GMT.

Mailing list information is available at https://stat.ethz.ch/mailman/listinfo/r-help. Please read the posting guide before posting to the list.

list of date sections of archive