Re: [R] Effect of multiplying the parscale vector by a scalar

From: Prof Brian Ripley <ripley_at_stats.ox.ac.uk>
Date: Fri 12 May 2006 - 16:50:05 EST

Almost certainly your objective function has multiple local minima. Scaling affects the path taken away from the initial value.

On Fri, 12 May 2006, Fernando Saldanha wrote:

> I noticed that the result of running optim() varies if the vector
> passed as the parscale argument is multiplied by a scalar. For
> example, only difference between the two code fragments below is that
> in the second one parscale is ten times larger than in the first. The
> optimal value and the optimal solution are quite different, though (in
> the second case a much better solution was found, as the function is
> being minimized).
>
> Of course, multiplication by a very small scalar could lead to
> problems related to rounding errors. But this is not what is happening
> here, I believe. In the first example the rescaled optimal variables
> are all around 1, within one order of magnitude.
>
> I have also noticed that if one varies one (and only one) of the third
> or fourth parameters from their original optimal values (0.9046640
> and 0.9050617) towards 1 the function increases, but if one moves them
> simultaneously and linearly towards (1, 1) then the function
> decreases, even for very small steps. But optim does not seem to be
> able to discover that direction of descent in the first example.
>
> Any insights will be greatly appreciated.
>
> Thanks.
>
> FS
>
> ######################## First example ##########################
>> phi.init <- c(0.002, 0.0002, 0.05, 0.9, 0.9, -0.3, 0.3, 0.3, -1)
>> lo <- c(0, 0, 0, 0, 0, -Inf, 0, 0, -Inf)
>> hi <- c(Inf, Inf, Inf, 1, 1, 0, Inf, Inf, 0)
>> phi_ <- phi.init
>>
>> opt.time <- system.time(phi_opt <- optim(phi_, model_lik, NULL, method = "L-BFGS-B", lower = lo, upper = hi, control = list(maxit = 1000, parscale = c(0.005, 0.0001, 0.05, 1, 1, 0.1, 0.1, 0.3, 1), trace = 0, REPORT = 3), hessian = FALSE))[3]
>>
>> phi_opt

> $par
> [1] 0.0065395 0.0002001 0.0475511 0.9046640 0.9050617 -0.3166011
> 0.3050438 0.2868558 -0.9515073
>
> $value
> [1] -4.901
>
> $counts
> function gradient
> 21 21
>
> $convergence
> [1] 0
>
> $message
> [1] "CONVERGENCE: REL_REDUCTION_OF_F <= FACTR*EPSMCH"
>
> ######################## Second example ##########################
>
>>
>> phi.init <- c(0.002, 0.0002, 0.05, 0.9, 0.9, -0.3, 0.3, 0.3, -1)
>> lo <- c(0, 0, 0, 0, 0, -Inf, 0, 0, -Inf)
>> hi <- c(Inf, Inf, Inf, 1, 1, 0, Inf, Inf, 0)
>> phi_ <- phi.init
>>
>> opt.time <- system.time(phi_opt <- optim(phi_, model_lik, NULL, method = "L-BFGS-B", lower = lo, upper = hi, control = list(maxit = 1000, parscale = 10 * c(0.005, 0.0001, 0.05, 1, 1, 0.1, 0.1, 0.3, 1), trace = 0, REPORT = 3), hessian = FALSE))[3]
>>
>> phi_opt

> $par
> [1] 0.001167 0.000200 0.000000 1.000000 1.000000 -0.296163
> 0.295126 0.280790 0.000000
>
> $value
> [1] -91.94
>
> $counts
> function gradient
> 48 48
>
> $convergence
> [1] 0
>
> $message
> [1] "CONVERGENCE: REL_REDUCTION_OF_F <= FACTR*EPSMCH"
>
>>
>
> ______________________________________________
> R-help@stat.math.ethz.ch mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html
>

-- 
Brian D. Ripley,                  ripley@stats.ox.ac.uk
Professor of Applied Statistics,  http://www.stats.ox.ac.uk/~ripley/
University of Oxford,             Tel:  +44 1865 272861 (self)
1 South Parks Road,                     +44 1865 272866 (PA)
Oxford OX1 3TG, UK                Fax:  +44 1865 272595

______________________________________________
R-help@stat.math.ethz.ch mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html
Received on Fri May 12 16:54:04 2006

Archive maintained by Robert King, hosted by the discipline of statistics at the University of Newcastle, Australia.
Archive generated by hypermail 2.1.8, at Fri 12 May 2006 - 18:10:06 EST.

Mailing list information is available at https://stat.ethz.ch/mailman/listinfo/r-help. Please read the posting guide before posting to the list.