Re: [R] nonlinear least squares fitting Trust-Region"

From: RAVI VARADHAN <rvaradhan_at_jhmi.edu>
Date: Sat 02 Sep 2006 - 23:36:02 GMT

I think the idea of parameter and intrinsic nonlinearity is due to Beale (JRSSB 1960). Was he Doug Bates' thesis advisor?

Ravi.

> May I also suggest Bates and Watts (1988) Nonlinear
> Regression
> Analysis and Its Applications (Wiley). This book carefully
> explains the
> difference between "parameter effects" and "intrinsic" curvature
> in
> non-linear fitting. I don't know if this idea was original with
> Bates or
> Watts, but I believe that Bates' PhD dissertation made important,
> original contributions to our understanding of it -- and it helped
> get
> him the faculty position in Statistics at the University of
> Wisconsin,
> where he still is. Bates is also a leading contributor to R.
>
> hope this helps.
> spencer graves
>
> RAVI VARADHAN wrote:
> > As suggested by Prof. Ripley, you should read a good book in the
> optimization area. One that I would highly recommend is the book
> by Dennis and Schnabel (1983) - Numerical methods for
> unconstrained optimization, which does a great job of explaining
> both "line-search" and "trust-region" approaches for achieving
> globally-convergent versions of a fast numerical scheme such as
> Gauss-Newton.
> >
> > Best,
> > Ravi.
> >
> > ----- Original Message -----
> > From: Prof Brian Ripley <ripley@stats.ox.ac.uk>
> > Date: Saturday, September 2, 2006 5:51 am
> > Subject: Re: [R] nonlinear least squares fitting Trust-Region"
> > To: Martin Ivanov <tramni@abv.bg>
> > Cc: r-help@stat.math.ethz.ch
> >
> >
> >> I believe people (including me) did not reply because you
> appeared
> >> not to
> >> have done your homework. The help page for ?nls _does_ have a
> >> reference
> >> to the 'port' documentation, and RSiteSearch("trust region") is
> >> informative and leads to an R package that does trust-region
> >> optimization.
> >> (So would looking in the R FAQ.)
> >>
> >> You say:
> >>
> >>
> >>> Since I am not an expert in the field of optimization, I am
> just
> >>> conforming to what matlab documentation
> >>>
> >> Please note that some of the R developers are really expert in
> >> that area,
> >> and their advice (in the R documentation) should be taken as
> >> seriously as
> >> that in some commercial package that is merely commenting about
> >> the very
> >> sparse choice it offers. Or if R is not in your personal trust
> >> region,
> >> just use 'matlab'.
> >>
> >> Please
> >>
> >> 1) do not shout at your helpers: using all caps is regarded as
> >> shouting.
> >> 2) study and follow the posting guide. People are much more
> >> likely to
> >> help you if you demonstrate you have made efforts to help yourself.
> >>
> >> 3) read the literature. The R FAQ leads to books that cover
> >> fitting
> >> non-linear models in S/R in considerable detail.
> >>
> >>
> >> On Sat, 2 Sep 2006, Martin Ivanov wrote:
> >>
> >>
> >>> Dear Mr Graves,
> >>>
> >>> Thank you very much for your response. Nobody else from this
> >>>
> >> mailing
> >>
> >>> list ventured to reply to me for the two weeks since I posted
> my
> >>> question. "nlminb" and "optim" are just optimization
> procedures.
> >>>
> >> What I
> >>
> >>> need is not just optimization, but a nonlinear CURVE FITTING
> >>>
> >> procedure.
> >> Which is just optimization: usually by least squares (although
> you
> >> have
> >> not actually specified that and there are better modern
> >> statistical
> >> ideas).
> >>
> >>
> >>> If there is some way to perform nonlinear curve fitting with
> the
> >>> "Trust-Region" algorithm using any of these functions, I would
> >>>
> >> me much
> >>
> >>> obliged to you if you suggest to me how to achieve that. You
> >>>
> >> asked me
> >>
> >>> why I do not want Gauss-Newton. Since I am not an expert in
> the
> >>>
> >> field of
> >>
> >>> optimization, I am just conforming to what matlab
> documentation
> >>> suggests, namely: "Algorithm used for the fitting procedure:
> >>> Trust-Region -- This is the default algorithm and must be used
> >>>
> >> if you
> >>
> >>> specify coefficient constraints. Levenberg-Marquardt -- If the
> >>> trust-region algorithm does not produce a reasonable fit, and
> >>>
> >> you do not
> >>
> >>> have coefficient constraints, you should try the Levenberg-
> >>>
> >> Marquardt
> >>
> >>> algorithm. Gauss-Newton --THIS ALGORITHM IS POTENTIALLY FASTER
> >>>
> >> THAN THE
> >>
> >>> OTHER ALGORITHMS, BUT IT ASSUMES THAT THE RESIDUALS ARE CLOSE
> TO
> >>>
> >> ZERO.
> >>
> >>> IT IS INCLUDED FOR PEDAGOGICAL REASONS AND SHOULD BE THE LAST
> >>>
> >> CHOICE FOR
> >>
> >>> MOST MODELS AND DATA SETS. I browsed some literature about the
> >>>
> >> garchfit
> >>
> >>> function, but I did not see the "Trust-Region" algorithm there
> >>>
> >> either:
> >>
> >>> algorithm = c("sqp", "nlminb", "lbfgsb", "nlminb+nm",
> >>>
> >> "lbfgsb+nm"),
> >>
> >>> control = list(), title = NULL, description = NULL, ...)
> >>>
> >>> Thank you for your attention. I am looking forward to your reply.
> >>> Regards,
> >>> Martin
> >>>
> >>> ---------------------------------------------------------------
> --
> >>> vbox7.com - ??????? ????? ???????!
> >>>
> >>> ______________________________________________
> >>> R-help@stat.math.ethz.ch mailing list
> >>> https://stat.ethz.ch/mailman/listinfo/r-help
> >>> PLEASE do read the posting guide http://www.R-
> >>>
> >> project.org/posting-guide.html
> >>
> >>> and provide commented, minimal, self-contained, reproducible code.
> >>>
> >>>
> >> --
> >> Brian D. Ripley, ripley@stats.ox.ac.uk
> >> Professor of Applied Statistics,
> http://www.stats.ox.ac.uk/~ripley/>> University of Oxford,
> Tel: +44 1865 272861 (self)
> >> 1 South Parks Road, +44 1865 272866 (PA)
> >> Oxford OX1 3TG, UK Fax: +44 1865 272595
> >>
> >> ______________________________________________
> >> R-help@stat.math.ethz.ch mailing list
> >> https://stat.ethz.ch/mailman/listinfo/r-help
> >> PLEASE do read the posting guide http://www.R-
> project.org/posting-
> >> guide.htmland provide commented, minimal, self-contained,
> >> reproducible code.
> >>
> >>
> >
> > ______________________________________________
> > R-help@stat.math.ethz.ch mailing list
> > https://stat.ethz.ch/mailman/listinfo/r-help
> > PLEASE do read the posting guide http://www.R-
> project.org/posting-guide.html
> > and provide commented, minimal, self-contained, reproducible code.
> >
>



R-help@stat.math.ethz.ch mailing list
https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code. Received on Sun Sep 03 09:43:43 2006

Archive maintained by Robert King, hosted by the discipline of statistics at the University of Newcastle, Australia.
Archive generated by hypermail 2.1.8, at Thu 07 Sep 2006 - 07:51:17 GMT.

Mailing list information is available at https://stat.ethz.ch/mailman/listinfo/r-help. Please read the posting guide before posting to the list.