Re: [R] L-BFGS-B needs finite values of 'fn'

From: Paul Smith <phhs80_at_gmail.com>
Date: Thu, 03 Apr 2008 21:00:08 +0100

Thanks, Ravi, for your insight. My initial experiments were conducted with b = 0.7, and my procedure (similar to yours) worked reasonably well. Then I wanted to check the quality of my procedure with lower b's, and the weak result motivated my initial post here.

I have meanwhile got a reasonably accurate solution with R through the DEoptim package. The number of iterations was 50000 and the code the following:

library(DEoptim)

k <- 10000
b <- 0.3

f <- function(x) {

 n <- length(x)

 if (sum(x) > k)
   r <- Inf
 else r <- -sum((b^(0:(n-1)))*log(x))

 return(r)

}

nvar <- 10
(sols <- DEoptim(f, lower=rep(0.000000001,nvar), upper=rep(k,nvar), control=list(itermax=50000)))
summary(sols)

The result was:

I am still struggling to find a fast way of getting the solution to my problem at least as accurate as the one returned by DEoptim. With so many iterations, DEoptim takes longer than 15 minutes to complete the job.

Paul

On Thu, Apr 3, 2008 at 4:38 PM, Ravi Varadhan <rvaradhan_at_jhmi.edu> wrote:
> Paul,
>
> After looking at your objective function and the penalty, I realized that
> since the contribution of each term to the objective function decreases
> geometrically, the later terms contribute relatively less to the overall
> maximum. Hence the numerical estimation of those terms is much less precise
> compared to the dominant terms. This explains why the ratios of x[k]/x[k-1]
> is not close to b, for small values of b.
>
> Now let us try a larger value of b, say, b=0.8.
>
> Here are the results from optim, with "BFGS".
>
> > k <- 10000
> > b <- 0.8
> >
> > f <- function(x, pen, k, b) {
> + n <- length(x)
> + r <- sum((b^(0:(n-1)))*log(x)) - pen*(sum(x)-k)^2
> + return(r)
> + }
> >
> > gr <- function(x, pen, k, b) {
> + n <- length(x)
> + r <- (b^(0:(n-1)))*(1/x) - 2*pen*(sum(x)-k)
> + return(r)
> + }
> >
> >
> > nvar <- 10
> > p0 <- runif(nvar, 0, 20)
> > sols <- optim(p0, f, gr, method="BFGS", control=list(fnscale=-1), k=k,
> b=b, pen=1)
> > sum(sols$par)
> [1] 10000
> > sols$par[2:nvar] / sols$par[1:(nvar-1)]
> [1] 0.8038902 0.8021216 0.7974323 0.7999681 0.7983918 0.8006497 0.8013940
> [8] 0.8044702 0.7879365
> >
> >
>
> As you can see, things are much better!
>
> Hope this helps,
>
> Ravi.
> ----------------------------------------------------------------------------
> -------
>
> Ravi Varadhan, Ph.D.
>
> Assistant Professor, The Center on Aging and Health
>
> Division of Geriatric Medicine and Gerontology
>
> Johns Hopkins University
>
> Ph: (410) 502-2619
>
> Fax: (410) 614-9625
>
> Email: rvaradhan_at_jhmi.edu
>
> Webpage: http://www.jhsph.edu/agingandhealth/People/Faculty/Varadhan.html
>
>
>
> ----------------------------------------------------------------------------
> --------
>
> -----Original Message-----
> From: r-help-bounces_at_r-project.org [mailto:r-help-bounces_at_r-project.org] On
>
>
> Behalf Of Ravi Varadhan
> Sent: Wednesday, April 02, 2008 2:03 PM
> To: 'Paul Smith'; 'R-help'
> Subject: Re: [R] L-BFGS-B needs finite values of 'fn'
>
> Yes, that is very important. If you look at the ratios x[k]/x[k-1], they
> are very close to 0.3 for the first few components, and then they start
> slowly diverging (ratio becomes smaller than 0.3) from that.
>
> So, optim is indeed finding a correct solution to the problem that you
> "actually" posed. You could increase your penalty to get a solution that is
> closer to the analytical solution you are expecting.
>
> Ravi.
>
> ----------------------------------------------------------------------------
> -------
>
> Ravi Varadhan, Ph.D.
>
> Assistant Professor, The Center on Aging and Health
>
> Division of Geriatric Medicine and Gerontology
>
> Johns Hopkins University
>
> Ph: (410) 502-2619
>
> Fax: (410) 614-9625
>
> Email: rvaradhan_at_jhmi.edu
>
> Webpage: http://www.jhsph.edu/agingandhealth/People/Faculty/Varadhan.html
>
>
>
> ----------------------------------------------------------------------------
> --------
>
>
> -----Original Message-----
> From: r-help-bounces_at_r-project.org [mailto:r-help-bounces_at_r-project.org] On
> Behalf Of Paul Smith
> Sent: Wednesday, April 02, 2008 1:57 PM
> To: R-help
> Subject: Re: [R] L-BFGS-B needs finite values of 'fn'
>
> But let me add the following: the part
>
> - 2000000*(sum(x)-k)^2
>
> of my function is a penalty. In truth, I want to maximize
>
> sum((b^(0:(n-1)))*log(x))
>
> s.t.
>
> sum(x) = k.
>
> Paul
>
>
> On Wed, Apr 2, 2008 at 6:48 PM, Paul Smith <phhs80_at_gmail.com> wrote:
> > Thanks, Ravi. The analytical solution, (x_1,x_2,...,x_10), should
> > satisfy this equality:
> >
> > x_t / x_(t-1) = 0.3.
> >
> > Unfortunately, the procedure that you suggest does not lead to a
> > solution that satisfies such an equality.
> >
> > Paul
> >
> >
> >
> >
> >
> > On Wed, Apr 2, 2008 at 5:12 PM, Ravi Varadhan <rvaradhan_at_jhmi.edu> wrote:
> > > Paul,
> > >
> > > Have you tried using "BFGS" without bounds?
> > >
> > > sols <- optim(rep(20,nvar), f, gr, method="BFGS",
> control=list(fnscale=-1))
> > >
> > > This converges to a solution, although I don't know if the converged
> > > solution is what you want.
> > >
> > > Ravi.
> > >
> > >
> ----------------------------------------------------------------------------
> > > -------
> > >
> > > Ravi Varadhan, Ph.D.
> > >
> > > Assistant Professor, The Center on Aging and Health
> > >
> > > Division of Geriatric Medicine and Gerontology
> > >
> > > Johns Hopkins University
> > >
> > > Ph: (410) 502-2619
> > >
> > > Fax: (410) 614-9625
> > >
> > > Email: rvaradhan_at_jhmi.edu
> > >
> > > Webpage:
> http://www.jhsph.edu/agingandhealth/People/Faculty/Varadhan.html
> > >
> > >
> > >
> > >
> ----------------------------------------------------------------------------
> > > --------
> > >
> > >
> > >
> > > -----Original Message-----
> > > From: r-help-bounces_at_r-project.org
> [mailto:r-help-bounces_at_r-project.org] On
> > > Behalf Of Paul Smith
> > >
> > > Sent: Monday, March 31, 2008 2:25 PM
> > > To: R-help
> > >
> > >
> > > Subject: Re: [R] L-BFGS-B needs finite values of 'fn'
> > >
> > > On Mon, Mar 31, 2008 at 2:57 PM, Zaihra T <zaihra_at_uwindsor.ca> wrote:
> > > > try something like this before wrapping up your function else i
> guess
> > > u'll
> > > > have to stick to Prof Brian Ripley suggestion his suggestions are
> usually
> > > > best bet .
> > > >
> > > > f <- function(x) {
> > > >
> > > > n <- length(x)
> > > >
> > > > r <- sum((b^(0:(n-1)))*log(x)) - 2000000*(sum(x)-k)^2
> > > > if(!is.finite(r))
> > > >
> > > > r<-1e+20 return(r)
> > > >
> > > > }
> > > >
> > > > have a nice day.
> > > >
> > > >
> > > >
> > > >
> > > > On Mon, 31 Mar 2008 12:24:09 +0100 "Paul Smith" wrote:
> > > > > Dear All,
> > > > >
> > > > > I am trying to solve the optimization problem below, but I am
> always
> > > > > getting the following error:
> > > > >
> > > > > Error in optim(rep(20, nvar), f, gr, method = "L-BFGS-B", lower =
> rep(0,
> > > :
> > > > > L-BFGS-B needs finite values of 'fn'
> > > > >
> > > > > Any ideas?
> > > > >
> > > > > Thanks in advance,
> > > > >
> > > > > Paul
> > > > >
> > > > > -----------------------------------------! ------
> > > > >
> > > > > k <- 10000
> > > > > b <- 0.3
> > > > >
> > > > > f <- function(x) {
> > > > >
> > > > > n <- length(x)
> > > > >
> > > > > r <- sum((b^(0:(n-1)))*log(x)) - 2000000*(sum(x)-k)^2
> > > > >
> > > > > return(r)
> > > > >
> > > > > }
> > > > >
> > > > > gr <- function(x) {

> > > > >
> > > > > n <- length(x)
> > > > >
> > > > > r <- (b^(0:(n-1)))*(1/x) - 4000000*(sum(x)-k)
> > > > >
> > > > > return(r)
> > > > >
> > > > > }
> > > > >
> > > > > nvar <- 10
> > > > > (sols <-
> > > > >
> > > >
> > >
> optim(rep(20,nvar),f,gr,method="L-BFGS-B",lower=rep(0,nvar),upper=rep(k,nvar
> > >
> ),control=list(fnscale=-1,parscale=rep(2000,nvar),factr=1e-300,pgtol=1e-300)
> > > ))
> > >
> > > Not much progress, Zaihra. Unfortunately! I am wondering whether one
> > > can transform the original problem into an equivalent one and solvable
> > > with optim.
> > >
> > > I know the analytical solution; I am just trying to check how far can
> > > R go regarding optimization problems.
> > >
> > > Paul
> > >
> > > ______________________________________________
> > >
> > > R-help_at_r-project.org mailing list
> > > https://stat.ethz.ch/mailman/listinfo/r-help
> > > PLEASE do read the posting guide
> http://www.R-project.org/posting-guide.html
> > >
> > >
> > > and provide commented, minimal, self-contained, reproducible code.
> > >
> >
>
> ______________________________________________
> R-help_at_r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>
> ______________________________________________
> R-help_at_r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>



R-help_at_r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code. Received on Thu 03 Apr 2008 - 20:07:33 GMT

Archive maintained by Robert King, hosted by the discipline of statistics at the University of Newcastle, Australia.
Archive generated by hypermail 2.2.0, at Thu 03 Apr 2008 - 22:30:26 GMT.

Mailing list information is available at https://stat.ethz.ch/mailman/listinfo/r-help. Please read the posting guide before posting to the list.

list of date sections of archive