Re: [R] function censReg in panel data setting

From: Arne Henningsen <arne.henningsen_at_googlemail.com>
Date: Wed, 14 Sep 2011 00:36:02 +0200

Hi Igors

On 13 September 2011 13:27, Igors <igors.lahanciks_at_gmail.com> wrote:
> Any success in finding possible solutions for my problem?

Somewhat. The calculation of the log-likelihood values is numerically much more robust/stable now. The log-likelihood contributions of some individuals became minus infinity in your model. This was caused by rounding errors as illustrated in the following simplified example:

log( exp( a ) + exp( b ) )

If a and b become smaller than approximately -800, exp( a ) and exp( b ) are rounded to zero and the log of their sum (zero) is minus infinity.
I have solved this problem by replacing the above calculation by

log( exp( a - c ) + exp( b - c ) ) + c
with c = max( a, b )

The source code of the improved censReg package is available on R-Forge [1]; R packages will be available on R-Forge [2] probably within one day.

[1] https://r-forge.r-project.org/scm/?group_id=256 [2] https://r-forge.r-project.org/R/?group_id=256

Unfortunately, the calculation of the gradients is still not robust but I expect that I can solve this problem in a similar way as I used to solve the problem with the likelihood function itself. I will continue working on this.

> I have tried to experiment with size of sample and I get really bad picture.
> I can't get it work even if sample is ~ 1000 obs. And it is way less than I
> would like to see working, taking into account my full sample size ~ 540 000
> obs.

I hope that you have a very fast computer -- or a lot of time for waiting many days or even a few weeks.

/Arne

-- 
Arne Henningsen
http://www.arne-henningsen.name

______________________________________________
R-help_at_r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Received on Tue 13 Sep 2011 - 22:48:48 GMT

This quarter's messages: by month, or sorted: [ by date ] [ by thread ] [ by subject ] [ by author ]

All messages

Archive maintained by Robert King, hosted by the discipline of statistics at the University of Newcastle, Australia.
Archive generated by hypermail 2.2.0, at Wed 14 Sep 2011 - 08:40:21 GMT.

Mailing list information is available at https://stat.ethz.ch/mailman/listinfo/r-help. Please read the posting guide before posting to the list.

list of date sections of archive