Re: [Rd] Numerical optimisation and "non-feasible" regions

From: Patrick Burns <pburns_at_pburns.seanet.com>
Date: Thu, 07 Aug 2008 17:57:25 +0100

If I understand your proposal correctly, then it probably isn't a good idea.

A derivative-based optimization algorithm is going to get upset whenever it sees negative infinity. Genetic algorithms, simulated annealing (and I think Nelder-Mead) will be okay when they see infinity but if all infeasible solutions have value negative infinity, then you are not giving the algorithm a clue about what direction to go.

Pat

Mathieu Ribatet wrote:
> Dear Patrick (and other),
>
> Well I used the Sylvester's criteria (which is equivalent) to test for
> this. But unfortunately, this is not the only issue!
> Well, to sum up quickly, it's more or less like geostatistics.
> Consequently, I have several unfeasible regions (covariance, margins
> and others).
> The problem seems that the unfeasible regions may be large and
> sometimes lead to optimization issues - even when the starting values
> are well defined.
> This is the reason why I wonder if setting by myself a $-\infty$ in
> the composite likelihood function is appropriate here.
>
> However, you might be right in setting a tolerance value 'eps' instead
> of the theoretical bound eigen values > 0.
> Best,
> Mathieu
>
>
> Patrick Burns a écrit :
>> If the positive definiteness of the covariance
>> is the only issue, then you could base a penalty on:
>>
>> eps - smallest.eigen.value
>>
>> if the smallest eigen value is smaller than eps.
>>
>> Patrick Burns
>> patrick_at_burns-stat.com
>> +44 (0)20 8525 0696
>> http://www.burns-stat.com
>> (home of S Poetry and "A Guide for the Unwilling S User")
>>
>> Mathieu Ribatet wrote:
>>
>>> Thanks Ben for your tips.
>>> I'm not sure it'll be so easy to do (as the non-feasible regions
>>> depend on the model parameters), but I'm sure it's worth giving a try.
>>> Thanks !!!
>>> Best,
>>>
>>> Mathieu
>>>
>>> Ben Bolker a écrit :
>>>
>>>> Mathieu Ribatet <mathieu.ribatet <at> epfl.ch> writes:
>>>>
>>>>
>>>>
>>>>> Dear list,
>>>>>
>>>>> I'm currently writing a C code to compute the (composite)
>>>>> likelihood -
>>>>> well this is done but not really robust. The C code is wrapped in
>>>>> an R
>>>>> one which call the optimizer routine - optim or nlm. However, the
>>>>> fitting procedure is far from being robust as the parameter space
>>>>> depends on the parameter - I have a covariance matrix that should
>>>>> be a
>>>>> valid one for example.
>>>>>
>>>>>
>>>> One reasonably straightforward hack to deal with this is
>>>> to add a penalty that is (e.g.) a quadratic function of the
>>>> distance from the feasible region, if that is reasonably
>>>> straightforward to compute -- that way your function will
>>>> get gently pushed back toward the feasible region.
>>>>
>>>> Ben Bolker
>>>>
>>>> ______________________________________________
>>>> R-devel_at_r-project.org mailing list
>>>> https://stat.ethz.ch/mailman/listinfo/r-devel
>>>>
>>>>
>

R-devel_at_r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel Received on Thu 07 Aug 2008 - 17:00:58 GMT

Archive maintained by Robert King, hosted by the discipline of statistics at the University of Newcastle, Australia.
Archive generated by hypermail 2.2.0, at Thu 07 Aug 2008 - 17:36:13 GMT.

Mailing list information is available at https://stat.ethz.ch/mailman/listinfo/r-devel. Please read the posting guide before posting to the list.