Re: [R] multinom() residual deviance

From: <Bill.Venables_at_csiro.au>
Date: Sat, 09 Apr 2011 23:51:45 +1000

The residual deviance from a multinomial model is numerically equal (up to round-off error) to that you would get had you fitted the model as a surrogate Poisson generalized linear model. Here is a short demo building on your example

> set.seed(101)
> df <- data.frame(f = sample(letters[1:3], 500, rep = TRUE),

+                  x = rnorm(500))

> library(nnet)
> mm <- multinom(f ~ x, df)

# weights: 9 (4 variable)
initial value 549.306144
final value 548.246724
converged
>
> X <- with(df, outer(f, levels(f), "==")+0)
>
> DF <- within(expand.grid(a = 1:500, b = letters[1:3]),
+ a <- factor(a))
> DF <- cbind(DF, f = as.vector(X), x = df$x)
>
>
> gm <- glm(f ~ a + b/x, poisson, DF) ## surrogate Poisson equivalent model
>
> deviance(gm) ## surrogate Poisson

[1] 1096.493
> deviance(mm) ## multinomial from nnet package
[1] 1096.493
>

Bill Venables



From: Sascha Vieweg [saschaview_at_gmail.com] Sent: 09 April 2011 17:06
To: Venables, Bill (CMIS, Dutton Park)
Cc: saschaview_at_gmail.com; r-help_at_r-project.org Subject: RE: [R] multinom() residual deviance

Hello

I am aware of the differences between the two models, excuse me for being imprecise on that in my first posting. My question only regards the "nature" or "structure" of the deviance, and thus whether the residual deviance of the multinomial model is the same residual deviance as reported by, say, glm. This is particularly important, since I want to calculate a pseudo R-Squared as follows (data from below):

library("nnet")
m <- multinom(y ~ ., data=df)

(llnull <- deviance(update(m, . ~ 1, trace=F)))
(llmod <- deviance(m))
(RMcFadden <- 1 - (llmod/llnull))

(I know that many statisticians here highly discourage people from using the pseudo R-Squareds, however, not many editors read this mailing list and still insist.)

The "MASS" book warning alerted me that the multinom residual deviance may be of principally different structure/nature than the glm one. Thus, while the calculation above holds for a glm, it does not for a multinom model. Am I right? And how to fix?

On 11-04-09 05:43, Bill.Venables_at_csiro.au wrote:

> The two models you fit are quite different. The first is a binomial model equivalent to
>
> fm <- glm(I(y == "a") ~ x, binomial, df)
>
> which you can check leads to the same result. I.e. this model amalgamates classes "b" and "c" into one.
>
> The second is a multivariate logistic model that considers all three classes defined by your factor y, (and has twice the number of parameters, among other things). The three clases, "a", "b" and "c" remain separate in the model.
>
> Hence the two models are not directly comparable, so why should the deviance be?
>
> Bill Venables.
> ________________________________________
> From: r-help-bounces_at_r-project.org [r-help-bounces_at_r-project.org] On Behalf Of Sascha Vieweg [saschaview_at_gmail.com]
> Sent: 09 April 2011 01:14
> To: r-help_at_r-project.org
> Subject: [R] multinom() residual deviance
>
> Running a binary logit model on the data
>
> df <- data.frame(y=sample(letters[1:3], 100, repl=T),
> x=rnorm(100))
>
> reveals some residual deviance:
>
> summary(glm(y ~ ., data=df, family=binomial("logit")))
>
> However, running a multinomial model on that data (multinom, nnet)
> reveals a residual deviance:
>
> summary(multinom(y ~ ., data=df))
>
> On page 203, the MASS book says that "here the deviance is
> comparing with the model that correctly predicts each person, not
> the multinomial response for each cell of the mininal model",
> followed by and instruction how to compare with the saturated
> model.
>
> For me as a beginner, this sounds like an important warning,
> however, I don't know what the warning exactly means and hence do
> not know what the difference between the residual deviance of the
> former (binary) and the latter (multinomial) model is.
>
> (I need the deviances to calculate some of the pseudo R-squares
> with function pR2(), package "pscl".)
>
> Could you give good advice?
>
> Thanks
> *S*
>
> --
> Sascha Vieweg, saschaview_at_gmail.com
>
> ______________________________________________
> R-help_at_r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>

--
Sascha Vieweg, saschaview_at_gmail.com

______________________________________________
R-help_at_r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Received on Sat 09 Apr 2011 - 13:55:39 GMT

Archive maintained by Robert King, hosted by the discipline of statistics at the University of Newcastle, Australia.
Archive generated by hypermail 2.2.0, at Sat 09 Apr 2011 - 16:30:27 GMT.

Mailing list information is available at https://stat.ethz.ch/mailman/listinfo/r-help. Please read the posting guide before posting to the list.

list of date sections of archive