Re: [Rd] [R] How to improve the "OPTIM" results

From: Spencer Graves <spencer.graves_at_pdf.com>
Date: Sun, 06 Apr 2008 08:37:50 -0700

<moved to R-devel>

      What would people suggest someone do to improve the optimization capabilities in R? A few ideas are mentioned below. I just got 578 hits from RSiteSearch("optimization", "fun"), and I wonder if there's a better way to get a reasonable overview of what's currently available in R -- and outside R? My initial thought was to develop an enhanced version of 'optim' that would check for negative curvature (via indefinite Hessians), do auto-scaling (see http://finzi.psych.upenn.edu/R/Rhelp02a/archive/125518.html), and output an object of a class like 'mle' or a new class like 'optimMLE' for which appropriate methods could be written. I have to often encountered problems with overparameterization, which could be fairly easily diagnosed via a singular or indefinite hessian. It would help me, and I believe other users, if functions like 'optim' and 'nls' would include simple diagnostics like this -- rather than returning a cryptic error or warning message.

      However, the comment from Hans Borchers (below) raises a related question: What might be the best way to build a collaboration with existing optimization initiatives, possibly making R a platform of choice for making it easy for users to access and compare alternative optimization methods?

      Thanks,
      Spencer         

Hans W Borchers wrote:
>> MORE GENERAL OPTIM ISSUES
>>
>> I'm considering creating a package 'optimMLE' that would automate
>> some of this and package it with common 'methods' that would assume that
>> sum(fn(...)) was either a log(likelihood) or the negative of a
>> log(likelihood), etc. However, before I do, I need to make more
>> progress on some of my other commitments, review RSiteSearch("optim",
>> "fun") to make sure I'm not duplicating something that already exists,
>> etc. If anyone is interested in collaborating on this, please contact
>> me off-line.
>>
>> Hope this helps.
>> Spencer
>>
>
> Thanks for your tips on using `optim()'. I believe `optim' is a reasonable good
> implementation of numerical optimization techniques such as quasi-Newton BFGS or
> conjugate gradients. Maybe methods using modern line searches or trust regions
> can follow someday.
>
> Everybody applying `optim' should be aware that it is a *Local Optimization*
> (LO) approach. What you describe appears to be a step towards *Global
> Optimization* (GO) in R. And indeed more and more requests in to the R-help list
> are about `optim' as a tool for global optimization, not always being fully
> aware of the differences.
>
> I am wondering whether it would be more useful to provide one or two global
> optimization approaches to the R community. And as this is an active research
> area, there are many and with different advantages and drawbacks.
>
> I would like to propose IPOPT as one of he newer and more advanced methods for
> global optimization. This powerful software package is open source and available
> through the COIN-OR initiative and its Web pages:
>
> http://www.coin-or.org/Ipopt/documentation/
>
> ``Ipopt (Interior Point OPTimizer, pronounced I-P-Opt) is a software package
> for large-scale nonlinear optimization. Ipopt is written in C++ and is
> released as open source code under the Common Public License (CPL). It is
> available from the COIN-OR initiative. The code has been written by Carl
> Laird (Carnegie Mellon University) and Andreas Wachter (IBM's T.J. Watson
> Research Center), who is the COIN project leader for Ipopt.''
>
> PERHAPS the COIN project would agree for IPOPT to be integrated into the open
> source project R as a package. For testing it right now there is an AMPL-based
> interface to IPOPT at the NEOS solver:
>
> http://neos.mcs.anl.gov/neos/solvers/nco:Ipopt/AMPL.html
>
> There may be other rewarding projects in the COIN-OR initiative, such as
> `SYMPHONY' for solving mixed-integer linear programs (MILP, stronger than `glpk'
> and `lp-solve'), or the BONMIN open source *non-linear* mixed integer
> programming (MINLP) solver. I could imagine R to be a good platform for
> integrating some of these algorithms.
>
> ------------
> Hans W. Borchers
> Control and Optimization Group
> ABB Corporate Research Germany
> hans-werner.borchers_at_de.abb.com
>
> ______________________________________________
> R-help_at_r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>
>



R-devel_at_r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel Received on Sun 06 Apr 2008 - 15:41:51 GMT

Archive maintained by Robert King, hosted by the discipline of statistics at the University of Newcastle, Australia.
Archive generated by hypermail 2.2.0, at Sun 06 Apr 2008 - 17:31:03 GMT.

Mailing list information is available at https://stat.ethz.ch/mailman/listinfo/r-devel. Please read the posting guide before posting to the list.

list of date sections of archive