Re: [R] (Off topic.) Observed Fisher information.

From: Duncan Murdoch <murdoch_at_stats.uwo.ca>
Date: Tue 07 Jun 2005 - 20:42:26 EST

Rolf Turner wrote:
> I have been building an R function to calculate the ***observed***
> (as opposed to expected) Fisher information matrix for parameter
> estimates in a rather complicated setting. I thought I had it
> working, but I am getting a result which is not positive definite.
> (One negative eigenvalue. Out of 10.)
>
> Is it the case that the observed Fisher information must be positive
> definite --- thereby indicating for certain that there are errors in
> my code --- or is it possible for such a matrix not to be pos. def.?

If you are at the maximum, it should be at least positive indefinite (or nonnegative definite). Numerical errors could make zero (or small positive) eigenvalues look negative. It's also possible that your optimization has missed the maximum by a bit, and then it could have truly negative eigenvalues.

In either case I'd expect the negative eigenvalues to be small.
>
> It seems to me that if the log likelihood surface is ***not*** well
> approximated by a quadratic in a neighbourhood of the maximum, then
> it might well be that case that the observed information could fail
> to be positive definite. Is this known/understood? Can anyone point
> me to appropriate places in the literature?

If there is a true negative eigenvalue, then moving along that eigenvector should increase the likelihood, so I don't think even irregular problems could have true negative eigenvalues at the MLE. The problem there would be that a zero score and a positive definite observed information matrix don't necessarily imply you're at even a local maximum.

Duncan Murdoch



R-help@stat.math.ethz.ch mailing list
https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html Received on Tue Jun 07 20:47:01 2005

This archive was generated by hypermail 2.1.8 : Fri 03 Mar 2006 - 03:32:26 EST