Re: [R] Comparing non nested models with correlation coefficients (inspired from Lorch and Myers )

From: Brian S Cade <cadeb_at_usgs.gov>
Date: Wed, 23 Mar 2011 13:26:56 -0600

As a follow-up to Greg's suggested graphical presentation, it seems like the Vuong test is sometimes used to compare fits of non nested models.

Brian

Brian S. Cade, PhD

U. S. Geological Survey
Fort Collins Science Center
2150 Centre Ave., Bldg. C
Fort Collins, CO 80526-8818

email: brian_cade_at_usgs.gov
tel: 970 226-9326

From:
Greg Snow <Greg.Snow_at_imail.org>
To:
Boris New <new.boris_at_gmail.com>, "r-help_at_r-project.org" <r-help_at_r-project.org>
Date:
03/23/2011 01:14 PM
Subject:
Re: [R] Comparing non nested models with correlation coefficients (inspired from Lorch and Myers )
Sent by:
r-help-bounces_at_r-project.org

If you are interested in the fits, then I would just plot the fits. Plot the fitted/predicted values from the 1st model as the x-values and the fitted/predicted values from the second model as the y-values. It is best to plot on a square plotting region and use asp=1, probably also doing abline(0,1) after. This gives a nice visual of how the predictions from the models compare. You could take it a step further by taking the 2 sets of predictions and doing a Tukey mean-difference plot, or a Bland-Altman plot.

I am not sure that the test you suggest would be meaningful. What null hypothesis is it testing and what does it mean if it is not rejected? (what is the power to reject?). Do you really need a formal test? If not then the plot is probably more meaningful.

-- 
Gregory (Greg) L. Snow Ph.D.
Statistical Data Center
Intermountain Healthcare
greg.snow_at_imail.org
801.408.8111



> -----Original Message-----
> From: r-help-bounces_at_r-project.org [mailto:r-help-bounces_at_r-
> project.org] On Behalf Of Boris New
> Sent: Wednesday, March 23, 2011 7:32 AM
> To: r-help_at_r-project.org
> Subject: [R] Comparing non nested models with correlation coefficients
> (inspired from Lorch and Myers )
>
> Hi,
>
> I would like to compare two models in R with the same dependant
> variable but
> different predictors (two different types of frequency and reaction
> times as
> RT).
> An editor told me to have a look at Lorch and Myers 1990.
>
> Lorch and Myers use the following technique:
> 1) they perform regressions on individual subjects' data
> 2) they extract the beta weights
> 3) they run t-test on these beta weights.
>
> The point is that I don't want to compare the "size effect" from the
> different models but how well the two models fit. So my idea was to
> extract
> the correlation coefficients instead of betas and doing t-tests on
> these.
> I checked and my correlation coefficients are normally distributed...
>
> Is it ok to do that?
>
> Best regards,
>
> [[alternative HTML version deleted]]
>
> ______________________________________________
> R-help_at_r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-
> guide.html
> and provide commented, minimal, self-contained, reproducible code.
______________________________________________ R-help_at_r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code. [[alternative HTML version deleted]] ______________________________________________ R-help_at_r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Received on Wed 23 Mar 2011 - 20:40:23 GMT

Archive maintained by Robert King, hosted by the discipline of statistics at the University of Newcastle, Australia.
Archive generated by hypermail 2.2.0, at Thu 24 Mar 2011 - 02:50:24 GMT.

Mailing list information is available at https://stat.ethz.ch/mailman/listinfo/r-help. Please read the posting guide before posting to the list.

list of date sections of archive