Re: [R] How to compare linear models with intercept and those withoutintercept using minimizing adjs R^2 strategy

From: 李俊杰 <klijunjie_at_gmail.com>
Date: Tue, 22 May 2007 00:34:21 +0800

So when I am using the adjusted R2 and as a penalized optimality criterion, and I have to compare models with intercept and those without intercept to decide the final model selected, does my crierion in my first email make sense?

Because we know that in leaps(leaps), if we want to select a model by the adjusted R2 criterion, we have to decide whether the intercept should be added in advance. But with my adjusted R2 criterion, we don't have to decide that in advance.

Thank you so much for your patient clarification.

2007/5/22, Lucke, Joseph F <Joseph.F.Lucke_at_uth.tmc.edu>:
>
> You don't have to embed model selection as hypothesis testing. You are
> using the adjusted R2 and as a penalized optimality criterion.
>
> ------------------------------
> *From:* 李俊杰 [mailto:klijunjie_at_gmail.com]
> *Sent:* Monday, May 21, 2007 10:43 AM
> *To:* Lucke, Joseph F
> *Cc:* r-help_at_stat.math.ethz.ch
> *Subject:* Re: [R] How to compare linear models with intercept and those
> withoutintercept using minimizing adjs R^2 strategy
>
>
> I have a question about what you've wrote in your pdf file. Why must we
> view my problem in the viewpoint of hypothesis testing? Is testing the
> original philosophy of maximizing Fisher's A-statistic to choose a optimum
> model?
>
> Thanks.
>
>
> 2007/5/21, Lucke, Joseph F <Joseph.F.Lucke_at_uth.tmc.edu>:
> >
> > I taken the conversation offline and used a pdf file to better display
> > equations.
> >
> > ------------------------------
> > *From:* 李俊杰 [mailto:klijunjie_at_gmail.com]
> > *Sent: *Monday, May 21, 2007 10:14 AM
> > *To:* Lucke, Joseph F
> > *Cc:* r-help_at_stat.math.ethz.ch
> > *Subject:* Re: [R] How to compare linear models with intercept and those
> > withoutintercept using minimizing adjs R^2 strategy
> >
> >
> >
> >
> > 2007/5/21, Lucke, Joseph F <Joseph.F.Lucke_at_uth.tmc.edu>:
> > >
> > > One issue is whether you want your estimators to be based on central
> > > moments (covariances) or on non-central moments. Removing the
> > > intercept
> > > changes the statistics from central to non-central moments. The
> > > adjusted R2, by which I think you mean Fisher's adjusted R2, is based
> > > on
> > > central moments (ratio of unbiased estimators of variances---central
> > > moments). So if you remove the intercept, you must re-derive the
> > > adjusted R2 for non-central moments --- you can't just plug in the
> > > number of independent variables as zero.
> >
> >
> > I have consulted A.J. Miller's Subset Selection in Regression(1990), and
> > I found what I was talking about adjusted R^2 was exactly as you
> > said--Fisher's A-statisitc. The formula of adjusted R^2 without the
> > intercept in that book was also the same as what summary(lm)$adj.r.squared
> > does in R. I guess what you want me to derive is the formula in that book.
> >
> > Though I know the formula of adjusted R2 for non-central moments, I
> > still want to know whether I am in the right way to compare *linear
> > models with intercept and those without intercept using maximizing adjs R^2
> > strategy. *
> > **
> > Actually, I consider the left column consisted of all 1 in predictor
> > matrix Z as the intercept term. Then I apply maximizing adjs R^2
> > strategy to decide which variables to select. Z is the term in the model:
> > Y=Zb+e.
> >
> > Thanks for your suggestion, and I am looking forward for your reply.
> >
> >
> >
> > -----Original Message-----
> > > From: r-help-bounces_at_stat.math.ethz.ch
> > > [mailto:r-help-bounces_at_stat.math.ethz.ch] On Behalf Of ???
> > > Sent: Sunday, May 20, 2007 8:53 PM
> > > To: r-help_at_stat.math.ethz.ch
> > > Subject: [R] How to compare linear models with intercept and those
> > > withoutintercept using minimizing adjs R^2 strategy
> > >
> > > Dear R-list,
> > >
> > > I apologize for my many emails but I think I know how to desctribe my
> > > problem differently and more clearly.
> > >
> > > My question is how to compare linear models with intercept and those
> > > without intercept using maximizing adjusted R^2 strategy.
> > >
> > > Now I do it like the following:
> > >
> > > > library(leaps)
> > > > n=20
> > > > x=matrix(rnorm(n*3),ncol=3)
> > > > b=c(1,2,0)
> > > > intercept=1
> > > > y=x%*%b+rnorm(n,0,1)+intercept
> > > >
> > > > var.selection=leaps(cbind(rep(1,n),x),y,int=F,method="adjr2")
> > > > ##### Choose the model with maximum adjr2
> > > > var.selection$which[var.selection$adjr2==max(var.selection$adjr2),]
> > > 1 2 3 4
> > > TRUE TRUE TRUE FALSE
> > >
> > >
> > > Actually, I use the definition of R-square in which the model is
> > > without
> > > a intercept term.
> > >
> > > Is what I am doing is correct?
> > >
> > > Thanks for any suggestion or correction.
> > > --
> > > Junjie Li, klijunjie_at_gmail.com
> > > Undergranduate in DEP of Tsinghua University,
> > >
> > > [[alternative HTML version deleted]]
> > >
> > > ______________________________________________
> > > R-help_at_stat.math.ethz.ch mailing list
> > > https://stat.ethz.ch/mailman/listinfo/r-help
> > > PLEASE do read the posting guide
> > > http://www.R-project.org/posting-guide.html<http://www.r-project.org/posting-guide.html>
> > > and provide commented, minimal, self-contained, reproducible code.
> > >
> >
> >
> >
> > --
> > Junjie Li, klijunjie_at_gmail.com
> > Undergranduate in DEP of Tsinghua University,
> >
> >
>
>
> --
> Junjie Li, klijunjie_at_gmail.com
> Undergranduate in DEP of Tsinghua University,
>

-- 
Junjie Li,                  klijunjie_at_gmail.com
Undergranduate in DEP of Tsinghua University,

	[[alternative HTML version deleted]]

______________________________________________ R-help_at_stat.math.ethz.ch mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.

Received on Mon 21 May 2007 - 17:08:47 GMT

Archive maintained by Robert King, hosted by the discipline of statistics at the University of Newcastle, Australia.
Archive generated by hypermail 2.2.0, at Mon 21 May 2007 - 17:31:14 GMT.

Mailing list information is available at https://stat.ethz.ch/mailman/listinfo/r-help. Please read the posting guide before posting to the list.