Re: [R] Question about variable selection

From: John Fox <jfox_at_mcmaster.ca>
Date: Sun 19 Feb 2006 - 06:35:05 EST


Dear Wensui and Andy,

When the explanatory variables are correlated it's perfectly possible for the marginal relationship between and X and Y to be zero and a partial relationship nonzero (even in the absence of interactions) -- this is simply a reflection of the more general point that partial and marginal relationships can differ.

Regards,
 John



John Fox
Department of Sociology
McMaster University
Hamilton, Ontario
Canada L8S 4M4
905-525-9140x23604
http://socserv.mcmaster.ca/jfox

> -----Original Message-----
> From: r-help-bounces@stat.math.ethz.ch
> [mailto:r-help-bounces@stat.math.ethz.ch] On Behalf Of Wensui Liu
> Sent: Saturday, February 18, 2006 2:03 PM
> To: Liaw, Andy
> Cc: r-help@stat.math.ethz.ch
> Subject: Re: [R] Question about variable selection
>
> Thank you so much for your reply, Andy.
>
> But what if I am only interesed in main effects instead of
> interactions?
>
>
>
> On 2/18/06, Liaw, Andy <andy_liaw@merck.com> wrote:
> >
> > That depends on whether the IV could have some significant
> > interactions with other Ivs not considered in the bivariate
> analysis.
> > E.g.,
> >
> > > iv <- expand.grid(-2:2, -2:2)
> > > y <- 3 + iv[,1] * iv[,2] + rnorm(nrow(iv), sd=0.1) summary(lm(y ~
> > > iv[,1]))
> >
> > Call:
> > lm(formula = y ~ iv[, 1])
> >
> > Residuals:
> > Min 1Q Median 3Q Max
> > -4.06259 -1.06048 -0.02377 1.05901 4.04315
> >
> > Coefficients:
> > Estimate Std. Error t value Pr(>|t|)
> > (Intercept) 3.01908 0.41482 7.278 2.09e-07 ***
> > iv[, 1] 0.01417 0.29332 0.048 0.962
> > ---
> > Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
> >
> > Residual standard error: 2.074 on 23 degrees of freedom Multiple
> > R-Squared: 0.0001014, Adjusted R-squared: -0.04337
> > F-statistic: 0.002333 on 1 and 23 DF, p-value: 0.9619
> >
> > > summary(lm(y ~ iv[,1] * iv[,2]))
> >
> > Call:
> > lm(formula = y ~ iv[, 1] * iv[, 2])
> >
> > Residuals:
> > Min 1Q Median 3Q Max
> > -0.22390 -0.08894 -0.01279 0.13525 0.17608
> >
> > Coefficients:
> > Estimate Std. Error t value Pr(>|t|)
> > (Intercept) 3.019083 0.026330 114.665 <2e-16 ***
> > iv[, 1] 0.014167 0.018618 0.761 0.455
> > iv[, 2] -0.005486 0.018618 -0.295 0.771
> > iv[, 1]:iv[, 2] 0.992865 0.013165 75.418 <2e-16 ***
> > ---
> > Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
> >
> > Residual standard error: 0.1316 on 21 degrees of freedom
> > Multiple R-Squared: 0.9963, Adjusted R-squared: 0.9958
> > F-statistic: 1896 on 3 and 21 DF, p-value: < 2.2e-16
> >
> >
> >
> >
> > Andy
> >
> > From: Wensui Liu
> > >
> > > Dear Lister,
> > >
> > > I have a question about variable selection for regression.
> > >
> > > if the IV is not significantly related to DV in the bivariate
> > > analysis, does it make sense to include this IV into the
> full model
> > > with multiple IVs?
> > >
> > > Thank you so much!
> > >
> > > [[alternative HTML version deleted]]
> > >
> > > ______________________________________________
> > > R-help@stat.math.ethz.ch mailing list
> > >
https://stat.ethz.ch/mailman/listinfo/r-help
> > > PLEASE do read the posting guide!
> > > http://www.R-project.org/posting-guide.html
> > >
> > >
> >
> >
> >
> >
> ----------------------------------------------------------------------
> > --------
> > Notice: This e-mail message, together with any
> > attachment...{{dropped}}
>
> ______________________________________________
> R-help@stat.math.ethz.ch mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide!
> http://www.R-project.org/posting-guide.html



R-help@stat.math.ethz.ch mailing list
https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html Received on Sun Feb 19 06:41:10 2006

This archive was generated by hypermail 2.1.8 : Fri 03 Mar 2006 - 03:42:35 EST