From: Spencer Graves <spencer.graves_at_pdf.com>

Date: Thu 18 Aug 2005 - 13:20:26 EST

Andrew Robinson wrote:

p.s. It looks like fm@bVars is a list containing vectors of length 29 and 6 in your example. I don't know what they are, but I don't see how they can be standard errors in the usual sense.

Date: Thu 18 Aug 2005 - 13:20:26 EST

I don't have Pinheiro and Bates handy, but as I recall, it's somthing like chapter 3. See "simulate.lme" in package nlme; with luck, it will also be in the index to Pinheiro and Bates. If not, it's still not hard to find. Their chapter on this includes some marvelous figures presenting simulation results presumably produced with a version of "simulate.lme".

The paper that seemed most insightful to me is Crainiceanu, Ruppert
and Vogelsang (2003) Some Properties of Likelihood Ratio Tests in Linear
Mixed Models" Cornell U.

(http://www.orie.cornell.edu/~davidr/papers/zeroprob_rev01.pdf).
However, I suggest you also examine Ota et al. (2000) "Approximate
Likelihood Ratio Tests and Marginal Distributions for Evolutionary Tree
Models with Constraints on Parameters", Molecular Biology and Evolution
17:798-803.

Just now I found other references by Googling for "likelihood ratio with parameter at a boundary". My literature search on this issue is far from exhaustive, and I would be pleased to hear from others who have other articles to recommend.

spencer graves

#####################

Andrew Robinson wrote:

* > Hi Spencer,
** >
** > thanks for this note to the r-users list. I wonder if I could ask ...
** >
** >
*

>> 4. With parameters at a boundary as with variance components, the

* >>best procedure seems to double the p-value from a nested anova (unless
** >>the reported p-value is already large). This is because the
** >>2*log(likelihood ratio) in such cases is roughly a 50-50 mixture of 0
** >>and chi-square(1) [if testing only 1 variance component parameter].
** >>This is supported by a substantial amount of research, including
** >>simulations discussed in a chapter in Pinheiro and Bates (2000)
** >>Mixed-Effects Models in S and S-Plus (Springer). The may be more
** >>accurate procedures available in the literature, but none so simple as
** >>this as far as I know.
** >
** >
** > Could you provide a page reference or references in the literature?
** > I've heard similar things before, but never seen them myself. Don't
** > worry if they're not immediately available.
** >
** > Thanks,
** >
** > Andrew
*

###################################### Is there some reason you are NOT using "anova", as in "Examples"section of "?lmer"?

Permit me to summarize what I know about this, and I'll be pleased if someone else who thinks they know different would kindly enlighten me and others who might otherwise be misled if anything I say is inconsistent with the best literature available at the moment:

- Doug Bates in his PhD dissertation and later in his book with Don Watts (1988) Nonlinear Regression Analysis and Its Applications (Wiley) split approximation errors in nonlinear least squares into "intrinsic curvature" and "parameter effects curvature". He quantified these two problems in the context of roughly three dozen published examples, if my memory is correct, and found that in not quite all cases, the parameter effects were at least an order of magnitude greater than the intrinsic curvature.
- In nonnormal situations, maximum likelihood is subject to more approximation error -- intrinsic curvature -- than "simple" nonlinear least squares. However, I would expect this comparison to still be fairly accurate, even if the differences may not be quite as stark.
- The traditional use of "standard errors" to judge statistical significance is subject to both intrinsic and parameter effects errors, while likelihood ratio procedures such as anova are subject only to the intrinsic curvature (assuming there are no substantive problems with nonconvergence). Consequently, to judge statistical significance of an effect, anova is usually substantially better than the so-called Wald procedure using approximate standard errors, and is almost never worse. If anyone knows of a case where this is NOT true, I'd like to know.
- With parameters at a boundary as with variance components, the best procedure seems to double the p-value from a nested anova (unless the reported p-value is already large). This is because the 2*log(likelihood ratio) in such cases is roughly a 50-50 mixture of 0 and chi-square(1) [if testing only 1 variance component parameter]. This is supported by a substantial amount of research, including simulations discussed in a chapter in Pinheiro and Bates (2000) Mixed-Effects Models in S and S-Plus (Springer). The may be more accurate procedures available in the literature, but none so simple as this as far as I know.

Comments? spencer graves

p.s. It looks like fm@bVars is a list containing vectors of length 29 and 6 in your example. I don't know what they are, but I don't see how they can be standard errors in the usual sense.

Doran, Harold wrote:

> These are the posterior variances of the random effects (I think more > properly termed "empirical" posteriors). Your model apparently includes > three levels of random variation (commu, bcohort, residual). The first > are the variances associated with your commu random effect and the > second are the variances associated with the bcohort random effect. > > Accessing either one would require > > fm@bVar$commu or fm@bVar$bcohort > > Obviously, replace "fm" with the name of your fitted model. > > -----Original Message----- > From: r-help-bounces@stat.math.ethz.ch > [mailto:r-help-bounces@stat.math.ethz.ch] On Behalf Of Shige Song > Sent: Wednesday, August 17, 2005 7:50 AM > To: r-help@stat.math.ethz.ch > Subject: Re: [R] How to assess significance of random effect in lme4 > > Hi Harold, > > Thanks for the reply. I looked at my outputs using str() as you > suggested, here is the part you mentioned: > > ..@ bVar :List of 2 > .. ..$ commu : num [1, 1, 1:29] 5e-10 5e-10 5e-10 5e-10 5e-10 ... > .. ..$ bcohort: num [1, 1, 1:6] 1.05e-05 7.45e-06 6.53e-06 8.25e-06 > 7.11e-06 ... > > where commu and bcohort are the two second-level units. Are these > standard errors? Why the second vector contains a series of different > numbers? > > Thanks! > > Shige > > On 8/17/05, Doran, Harold <HDoran@air.org> wrote: > >> >> >>You can extract the posterior variance of the random effect from the >>bVar slot of the fitted lmer model. It is not a hidden option, but a >>part of the fitted model. It just doesn't show up when you use > > summary(). > >> >> Look at the structure of your object to see what is available using > > str(). > >> >> However, your comment below seems to imply that it is incorrect for >>lmer to report SDs instead of the standard error, which is not true. >>That is a quantity of direct interest. >> >> Other multilevel programs report the same exact statistics (for the >>most part). For instance, HLM reports the variances as well. If you >>want the posterior variance of an HLM model you need to extract it. >> >> >> >> -----Original Message----- >> From: r-help-bounces@stat.math.ethz.ch on behalf of >>Shige Song >> Sent: Wed 8/17/2005 6:30 AM >> To: r-help@stat.math.ethz.ch >> Cc: >> Subject: [R] How to assess significance of random effect in > > lme4 > >> >> Dear All, >> >> With kind help from several friends on the list, I am getting close. >> Now here are something interesting I just realized: for random >>effects, lmer reports standard deviation instead of standard error! Is > > >>there a hidden option that tells lmer to report standard error of >>random effects, like most other multilevel or mixed modeling software, > > >>so that we can say something like "randome effect for xxx is >>significant, while randome effect for xxx is not significant"? Thanks! >> >> Best, >> Shige >> >> ______________________________________________ >> R-help@stat.math.ethz.ch mailing list >>https://stat.ethz.ch/mailman/listinfo/r-help >> PLEASE do read the posting guide! >>http://www.R-project.org/posting-guide.html >> >> >> >> >> > > > ______________________________________________ > R-help@stat.math.ethz.ch mailing list > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide! > http://www.R-project.org/posting-guide.html > > ______________________________________________ > R-help@stat.math.ethz.ch mailing list > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html

-- Spencer Graves, PhD Senior Development Engineer PDF Solutions, Inc. 333 West San Carlos Street Suite 700 San Jose, CA 95110, USA spencer.graves@pdf.com www.pdf.com <http://www.pdf.com> Tel: 408-938-4420 Fax: 408-280-7915 ______________________________________________ R-help@stat.math.ethz.ch mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide! http://www.R-project.org/posting-guide.htmlReceived on Fri Aug 19 20:27:49 2005

*
This archive was generated by hypermail 2.1.8
: Sun 23 Oct 2005 - 15:33:22 EST
*