Re: [R] Loess with more than 4 predictors / offsets

From: Louisell, Paul <ploua_at_allstate.com>
Date: Tue 23 Jan 2007 - 21:51:41 GMT


I apologize for clogging up inboxes, but I realized I needed to amend what I said in my last comment below:

In fact, I'd like to specify that it be unconditionally linear, but with estimated coefficients, _both an intercept and a slope_.

If the "offset" were only multiplied by a nonzero constant c, this would have the effect of moving the whole response surface -log(c) units parallel to the response axis in the scenario I outline below. This would effectively give me the same thing I already have.

Paul Louisell
650-833-6254
ploua@allstate.com
Research Associate (Statistician)
Modeling & Data Analytics
ARPC -----Original Message-----
From: Louisell, Paul
Sent: Tuesday, January 23, 2007 12:40 PM To: 'Prof Brian Ripley'
Cc: r-help@stat.math.ethz.ch
Subject: RE: [R] Loess with more than 4 predictors / offsets

In response to your questions:

I asked about including the offset for convenience; I currently put the offset in by subtracting it from the response, just as you suggest. The reason for including them is that I'm doing something slightly unusual with loess:

I'm fitting loess to log((response+1)/offset) because the response is actually a vector of counts. This is intended to give a rough approximation to a Poisson regression; the reason for using loess is that the mean response should be approximated by a Poisson process with 4 predictor variables which can be divided into 2 pairs, each pair of which are geographic location coordinates. The two location pairs are expected to exhibit strong interaction; hence, the reason for fitting loess to all 4 predictors.

I'm aware of the curse of dimensionality, but I have a very large dataset--over 600,000 observations. Since each pair of predictors represents a point on a grid, I think Euclidean distance is probably a good choice. And this brings me to the motivation for wanting to fit with 5 predictors:

The offset is not _really_ an offset; it's just an approximation to what the real offset should be. Hence, I'd rather include it as a predictor than artificially force it to be included linearly with a coefficient of 1. I'm less concerned with linearity than I am with forcing the coefficient. In fact, I'd like to specify that it be unconditionally linear, but with an estimated coefficient.

Thanks,

Paul Louisell
650-833-6254
ploua@allstate.com
Research Associate (Statistician)
Modeling & Data Analytics
ARPC -----Original Message-----
From: Prof Brian Ripley [mailto:ripley@stats.ox.ac.uk] Sent: Monday, January 22, 2007 11:01 PM
To: Louisell, Paul
Cc: r-help@stat.math.ethz.ch
Subject: Re: [R] Loess with more than 4 predictors / offsets

On Mon, 22 Jan 2007, Louisell, Paul wrote:

> Hello,
>
> Does anyone know of an R version of loess that allows more than 4
> predictors and/or allows the specification of offsets? For that
matter,
> does anyone know of _any_ version of loess that does either of the
> things I mention?

Why would you want offsets in a regression?: just subtract them from the

lhs. (R's lm has gained offsets by analogy with glm, but the S original

did not have them). If you would be more comfortable working with them,

it would be very easy to create a modified version that supports them.

Also, have you heard of the 'curse of dimensionality'? Localization even
to 4 dimensions is no longer really an appropriate term, and Euclidean distance will be the main determinant of 'local' and is quite arbitrary.

> Thanks,
>
> Paul Louisell
> 650-833-6254
> ploua@allstate.com
> Research Associate (Statistician)
> Modeling & Data Analytics
> ARPC
>
>
>
> [[alternative HTML version deleted]]
>
> ______________________________________________
> R-help@stat.math.ethz.ch mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide
http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>

-- 
Brian D. Ripley,                  ripley@stats.ox.ac.uk
Professor of Applied Statistics,  http://www.stats.ox.ac.uk/~ripley/
University of Oxford,             Tel:  +44 1865 272861 (self)
1 South Parks Road,                     +44 1865 272866 (PA)
Oxford OX1 3TG, UK                Fax:  +44 1865 272595

______________________________________________
R-help@stat.math.ethz.ch mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Received on Wed Jan 24 08:58:16 2007

Archive maintained by Robert King, hosted by the discipline of statistics at the University of Newcastle, Australia.
Archive generated by hypermail 2.1.8, at Tue 23 Jan 2007 - 22:30:29 GMT.

Mailing list information is available at https://stat.ethz.ch/mailman/listinfo/r-help. Please read the posting guide before posting to the list.