From: Ted Harding <Ted.Harding_at_manchester.ac.uk>

Date: Mon, 11 Feb 2008 16:18:50 +0000 (GMT)

E-Mail: (Ted Harding) <Ted.Harding_at_manchester.ac.uk> Fax-to-email: +44 (0)870 094 0861

https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code. Received on Mon 11 Feb 2008 - 16:27:36 GMT

Date: Mon, 11 Feb 2008 16:18:50 +0000 (GMT)

On 11-Feb-08 15:07:37, Douglas Bates wrote:

> [...]

*> Except that Doug Bates doesn't use the EM algorithm for fitting mixed
**> models any more. The lme4 package previously had an option for
**> starting with EM (actually ECME, which is a variant of EM) iterations
**> but I have since removed it. For large data sets and especially for
**> models with non-nested random effects, the EM iterations just slowed
**> things down relative to direct optimisation of the log-likelihood.
**> [...]
*

The "raw" EM Algorithm can be slow. I have had good success using Aitken Acceleration for it.

The basic principle is that, once an interative algorithm gets to a stage where (approximately)

[A] (X[n+1] - X) = k*(X[n] -X)

where X[n] is the result at the n-th iteration, -1 < k < 1, and X is the limit, then you can use recent results to predict the limit. Taking the above equation literally, along with its analogue for the next step:

[B] (X[n+2] - X) = k*(X[n+1] -X)

from which k = (X[n+2] - X[[n+1])/(X[n+1] - X[n])

and then

[C] X = (X[n+1] - X[n])/(1 - k).

If X is multidimensional (say dimension = p), then k is a pxp matrix, and you want all its eigenvalues to be less than 1 in modulus. Then you use the matrix analogues of the above equations, based it on (p+1) successive iterations X[n], X[n+1], ... , X[n+p+1]), i.e. on the p-vector

c(X[n+1]-X[n], X[n+1]-X[n+1], ... , X[n+p+1]-X[n+p])

I have had good experience with this too!

The best method of proceeding is:

Stage 1: Monitor the sequence {X[n]} until it seems that equation [A] is beginning to be approximately true;

Stage 2: Apply equations [A], [B], [C] to estimate X.

Stage 3: Starting at this X, run a few more iterations so that you get a better (later) estimate of k, and then apply [A], [B], [C] aqain to re-estimate X.

Repeat stage 3 until happy (or bored).

The EM Algorithm, in most cases, falls into the class of procedures to which Aitken Acceleration is applicable.

Best wishes to all,

Ted.

E-Mail: (Ted Harding) <Ted.Harding_at_manchester.ac.uk> Fax-to-email: +44 (0)870 094 0861

Date: 11-Feb-08 Time: 16:18:45 ------------------------------ XFMail ------------------------------ ______________________________________________R-help_at_r-project.org mailing list

https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code. Received on Mon 11 Feb 2008 - 16:27:36 GMT

Archive maintained by Robert King, hosted by
the discipline of
statistics at the
University of Newcastle,
Australia.

Archive generated by hypermail 2.2.0, at Mon 11 Feb 2008 - 17:30:12 GMT.

*
Mailing list information is available at https://stat.ethz.ch/mailman/listinfo/r-help.
Please read the posting
guide before posting to the list.
*