Re: [R] MLE maximum number of parameters

From: Albyn Jones <jones_at_reed.edu>
Date: Tue 20 Jun 2006 - 04:47:57 EST

I regularly optimize functions of over 1000 parameters for posterior mode computations using a variant of newton-raphson. I have some favorable conditions: the prior is pretty good, the posterior is smooth, and I can compute the gradient and hessian.

albyn

On Mon, Jun 19, 2006 at 06:53:00PM +0100, Patrick Burns wrote:
> Seagulls have a very different perspective to ballparks
> than ants. Nonetheless, there is something that can be
> said.
>
> There are several variables in addition to the number of
> parameters that are important. These include:
>
> * The complexity of the likelihood
>
> * The number of observations in the dataset
>
> * How close to the optimum is close enough
>
> * Your patience
>
> The latter is undoubtedly the most important of all. It
> matters a lot whether you think a minute is a long time
> or only periods measured in weeks.
>
> The optimization strategy can also have a big effect. If
> you are using a derivative-based optimizer, then the number
> of parameters can have a big impact. Typically one iteration
> in such algorithms requires p+1 function calls, where p is the
> number of parameters. Since more iterations are generally
> required with more parameters, the speed can decrease
> rapidly as the number of parameters increases.
>
> One strategy to deal with a large number of parameters is to
> start with something like a genetic algorithm. Once the genetic
> algorithm has a pretty good solution, then switch to a derivative-
> based algorithm to finish. The amount to run the initial
> algorithm before switching depends on the problem, the quality
> of the two optimizers, and probably other things.
>
> With this switching strategy and at least a modicum of patience,
> problems with thousands of parameters may be feasible to solve.
>
> Patrick Burns
> patrick@burns-stat.com
> +44 (0)20 8525 0696
> http://www.burns-stat.com
> (home of S Poetry and "A Guide for the Unwilling S User")
>
> Federico Calboli wrote:
>
> >Hi All,
> >
> >I would like to know, is there a *ballpark* figure for how many
> >parameters the minimisation routines can cope with?
> >
> >I'm asking because I was asked if I knew.
> >
> >Cheers,
> >
> >Federico
> >
> >--
> >Federico C. F. Calboli
> >Department of Epidemiology and Public Health
> >Imperial College, St. Mary's Campus
> >Norfolk Place, London W2 1PG
> >
> >Tel +44 (0)20 75941602 Fax +44 (0)20 75943193
> >
> >f.calboli [.a.t] imperial.ac.uk
> >f.calboli [.a.t] gmail.com
> >
> >______________________________________________
> >R-help@stat.math.ethz.ch mailing list
> >https://stat.ethz.ch/mailman/listinfo/r-help
> >PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html
> >
> >
> >
> >
>
> ______________________________________________
> R-help@stat.math.ethz.ch mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html



R-help@stat.math.ethz.ch mailing list
https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html Received on Tue Jun 20 04:51:31 2006

Archive maintained by Robert King, hosted by the discipline of statistics at the University of Newcastle, Australia.
Archive generated by hypermail 2.1.8, at Tue 20 Jun 2006 - 06:12:17 EST.

Mailing list information is available at https://stat.ethz.ch/mailman/listinfo/r-help. Please read the posting guide before posting to the list.