Re: [Rd] non-differentiable evaluation points in nlminb(), follow-up of PR#15052

From: Ravi Varadhan <ravi.varadhan_at_jhu.edu>
Date: Thu, 27 Sep 2012 14:27:37 +0000

Can you provide a correct/sensible example that illustrates the problem?

Your gradient function is wrong. So, how do you expect the algorithms to work?

Why is the gradient Inf when |x| < 1.e-5? It should be 0.

Here the following works fine:

require(optimx)

f <- function (x) {
if(is.na(x)) Inf else abs(x)
}

gr <- function (x) if (abs(x) < 1e-5) 0 else sign(x)

hess <- function (x) matrix(if (abs(x) < 1e-5) 0 else 0, 1L, 1L)

nlminb(5, f, gr, hess, control=list(eval.max=30, trace=1))

ans <- optimx(par=5, fn=f, gr=gr, control=list(all.methods=TRUE))

Ravi



From: r-devel-bounces_at_r-project.org [r-devel-bounces_at_r-project.org] on behalf of Sebastian Meyer [Sebastian.Meyer_at_ifspm.uzh.ch] Sent: Wednesday, September 26, 2012 5:13 AM To: r-devel_at_r-project.org
Subject: [Rd] non-differentiable evaluation points in nlminb(), follow-up of PR#15052

This is a follow-up question for PR#15052 <http://bugs.r-project.org/bugzilla3/show_bug.cgi?id=15052>

There is another thing I would like to discuss wrt how nlminb() should proceed with NAs. The question is: What would be a successful way to deal with an evaluation point of the objective function where the gradient and the hessian are not well defined?

If the gradient and the hessian both return NA values (assuming R < r60789, e.g. R 2.15.1), and also if both return +Inf values, nlminb steps to an NA parameter vector.
Here is a really artificial one-dimensional example for demonstration:

f <- function (x) {
  cat("evaluating f(", x, ")\n")
  if(is.na(x)) {Inf # to prevent an infinite loop for R < r60789   } else abs(x)
}

gr <- function (x) if (abs(x) < 1e-5) Inf else sign(x) hess <- function (x) matrix(if (abs(x) < 1e-5) Inf else 0, 1L, 1L) trace(gr)
trace(hess)
nlminb(5, f, gr, hess, control=list(eval.max=30, trace=1))

Thus, if nlminb reaches a point where the derivatives are not defined, optimization is effectively lost. Is there a way to deal with such points in nlminb? Otherwise, the objective function is doomed to emergency stop() if it receives NA parameters because nlminb won't pick up courage - regardless of the following return value of the objective function.
As far as I would assess the situation, nlminb is currently not capable of optimizing objective functions with non-differentiable points.

Best regards,
  Sebastian Meyer

--
Sebastian Meyer
Division of Biostatistics
Institute of Social and Preventive Medicine
University of Zurich

______________________________________________
R-devel_at_r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
______________________________________________
R-devel_at_r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Received on Fri 28 Sep 2012 - 10:32:36 GMT

This quarter's messages: by month, or sorted: [ by date ] [ by thread ] [ by subject ] [ by author ]

All messages

Archive maintained by Robert King, hosted by the discipline of statistics at the University of Newcastle, Australia.
Archive generated by hypermail 2.2.0, at Fri 28 Sep 2012 - 19:10:45 GMT.

Mailing list information is available at https://stat.ethz.ch/mailman/listinfo/r-devel. Please read the posting guide before posting to the list.

list of date sections of archive