[R] Re: HOWTO compare univariate binomial glm lrm models which are not nested

From: Prof Brian Ripley <ripley_at_stats.ox.ac.uk>
Date: Sun 17 Apr 2005 - 01:52:45 EST

Compare them by `goodness for purpose': you have not told us the purpose. Please do read some of the extensive literature on model comparison.

On Sat, 16 Apr 2005, Jan Verbesselt wrote:

> Thanks a lot for the input!
>
> I forgot to add family=binomial, for a binomial glm. Now the AIC's are
> positive!
>
> I was planning to use AIC (from the binomial glm) and c-index (lrm) to
> compare and rank different uni-variate (one continue explanatory variable)
> logistic models to evaluate the 'performance' of the different explanatory
> variables in the different models.
>
> What is the best technique to compare these lrm.models, which are not
> nested? I found in literature that ranking based on different parameters
> (goodness of fit and predictability) that these can be used to compare
> uni-variate models.
>
> Thanks in advance,
> Regards,
> Jan-
>
>
> _______________________________________________________________________
> ir. Jan Verbesselt
> Research Associate
> Lab of Geomatics Engineering K.U. Leuven
> Vital Decosterstraat 102. B-3000 Leuven Belgium
> Tel: +32-16-329750 Fax: +32-16-329760
> http://gloveg.kuleuven.ac.be/
> _______________________________________________________________________
>
> -----Original Message-----
> From: Prof Brian Ripley [mailto:ripley@stats.ox.ac.uk]
> Sent: Friday, April 15, 2005 5:06 PM
> To: Jan Verbesselt
> Cc: r-help@stat.math.ethz.ch
> Subject: Re: [R] negetative AIC values: How to compare models with negative
> AIC's
>
> AICs (like log-likelihoods) can be positive or negative.
> However, you fitted a Gaussian and not a binomial glm (as lrm does if
> m.arson is binary).
>
> For a discrete response with the usual dominating measure (counting
> measure) the log-likelihood is negative and hence the AIC is positive,
> but not in general (and it is matter of convention even there).
>
> In any case, Akaike only suggested comparing AIC for nested models, no one
> suggests comparing continuous and discrete models.
>
> On Fri, 15 Apr 2005, Jan Verbesselt wrote:
>
>>
>> Dear,
>>
>> When fitting the following model
>> knots <- 5
>> lrm.NDWI <- lrm(m.arson ~ rcs(NDWI,knots)
>>
>> I obtain the following result:
>>
>> Logistic Regression Model
>>
>> lrm(formula = m.arson ~ rcs(NDWI, knots))
>>
>>
>> Frequencies of Responses
>> 0 1
>> 666 35
>>
>> Obs Max Deriv Model L.R. d.f. P C
> Dxy
>> Gamma Tau-a R2 Brier
>> 701 5e-07 34.49 4 0 0.777
> 0.553
>> 0.563 0.053 0.147 0.045
>>
>> Coef S.E. Wald Z P
>> Intercept -4.627 3.188 -1.45 0.1467
>> NDWI 5.333 20.724 0.26 0.7969
>> NDWI' 6.832 74.201 0.09 0.9266
>> NDWI'' 10.469 183.915 0.06 0.9546
>> NDWI''' -190.566 254.590 -0.75 0.4541
>>
>> When analysing the glm fit of the same model
>>
>> Call: glm(formula = m.arson ~ rcs(NDWI, knots), x = T, y = T)
>>
>> Coefficients:
>> (Intercept) rcs(NDWI, knots)NDWI rcs(NDWI, knots)NDWI'
>> rcs(NDWI, knots)NDWI'' rcs(NDWI, knots)NDWI'''
>> 0.02067 0.08441 -0.54307
>> 3.99550 -17.38573
>>
>> Degrees of Freedom: 700 Total (i.e. Null); 696 Residual
>> Null Deviance: 33.25
>> Residual Deviance: 31.76 AIC: -167.7
>>
>> A negative AIC occurs!
>>
>> How can the negative AIC from different models be compared with each
> other?
>> Is this result logical? Is the lowest AIC still correct?
>
> --
> Brian D. Ripley, ripley@stats.ox.ac.uk
> Professor of Applied Statistics, http://www.stats.ox.ac.uk/~ripley/
> University of Oxford, Tel: +44 1865 272861 (self)
> 1 South Parks Road, +44 1865 272866 (PA)
> Oxford OX1 3TG, UK Fax: +44 1865 272595
>
>
>

-- 
Brian D. Ripley,                  ripley@stats.ox.ac.uk
Professor of Applied Statistics,  http://www.stats.ox.ac.uk/~ripley/
University of Oxford,             Tel:  +44 1865 272861 (self)
1 South Parks Road,                     +44 1865 272866 (PA)
Oxford OX1 3TG, UK                Fax:  +44 1865 272595

______________________________________________
R-help@stat.math.ethz.ch mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html
Received on Sun Apr 17 01:58:00 2005

This archive was generated by hypermail 2.1.8 : Fri 03 Mar 2006 - 03:31:13 EST