Re: [R] nonlinear model: pseudo-design matrix

From: Spencer Graves <>
Date: Sat 18 Feb 2006 - 09:32:19 EST

Hi, Murray:

          I just got 54 hits from RSiteSearch("numerical differentiation"), the first of which mentioned a function "numericDeriv" WITH a warning (

	  hope this helps.
	  spencer graves

Hi Spencer,

I will try some of the ways you suggest and thank you for the suggestions. I still think that performing a score test is a sensible thing to do in my situation, though.


Spencer Graves wrote:

> Hi, Murray:
> When I have problems with nonconvergence of nls, I often
> move the problem to "optim(..., hessian=TRUE)". Even if the larger
> model is overparameterized and the hessian is singular, I optim usually
> returns an answer from which I can then compute 2*log(likelihood ratio).
> Moreover, the hessian will help me diagnose the problem. If it were my
> problem today, I'd try the following:
> (1) If there are substantial differences in the diagonal elements
> of the hessian, it suggests the scaling should be adjusted. Not too
> long ago, someone else suggested that this could be done within optim
> via the argument control = list(parscale=...). I have yet to try that,
> but I think it should work fine.
> (2) If the diagonal elements of the hessian do not differ by more
> than a couple orders of magnitude, then I'd try eigen(fit$hessian,
> symmetric=TRUE). The relative magnitudes of the eigenvalues will expose
> the effective numer of paramaters that can be estimated, and the
> eigenvectors associated with the smallest eigenvalues can help one
> diagnose the problem.
> hope this helps.
> spencer graves
> Murray Jorgensen wrote:

>> Hi Spencer,
>> you were the only one to reply. Yes I am aware of the intrinsic / 
>> parameter effects distinction and the advantages of LR tests and 
>> profiling over Wald tests based on the local curvature of the 
>> loglikelihood surface at the larger of two models being compared. My 
>> situation is that I am comparing two nested models both of which have 
>> uncomfortably many parameters for the amount of data available. I am 
>> able to fit the smaller of the two models but not the larger. In this 
>> situation neither the the Wald nor the LR test is available to me but 
>> the score test (a.k.a. the Lagrange Multiplier test) is available to 
>> me because it is based on the loglikelihood gradient at the smaller 
>> model.
>> I have been able to carry out the test by extracting
>> X <- smaller.nls$m$gradient()
>> and obtaining the extra columns of X for the parameters in larger but 
>> not in smaller by numerical differentiation. It seems that there 
>> should be some way of obtaining the extra columns without recourse to 
>> numerical differentiation, though.
>> Cheers,  Murray Jorgensen
>> Spencer Graves wrote:
>>>       There doubtless is a way to extract the gradient information 
>>> you desire, but have you considered profiling instead?  Are you 
>>> familiar with the distinction between intrinsic and parameter effects 
>>> curvature?  In brief, part of the nonlinearities involved in 
>>> nonlinear least squares are intrinsic to the problem, and part are 
>>> due to the how the problem is parameterized.  If you change the 
>>> parameterization, you change the parameter effects curvature, but the 
>>> intrinsic curvature remains unchanged.  Roughly 30 years ago, Doug 
>>> Bates and Don Watts reanalized a few dozen published nonlinear 
>>> regression fits, and found that in all but perhaps one or two, the 
>>> parameter effects were dominant and the intrinsic curvature was 
>>> negligible.  See Bates and Watts (1988) Nonlinear Regression Analysis 
>>> and Its Applications (Wiley) or Seber and Wild (1989) Nonlinear 
>>> Regression (Wiley).
>>>       Bottom line:
>>>       1.  You will always get more accurate answers from profiling 
>>> than from the Wald "pseudodesign matrix" approach.  Moreover, often 
>>> the differences are dramatic.
>>>       2.  I just did RSiteSearch("profiling with nls").  The first 
>>> hit was 
>>> "". 
>>> If this is not satisfactory, please explain why.
>>>       hope this helps.
>>>       spencer graves
>>> Murray Jorgensen wrote:
>>>> Given a nonlinear model formula and a set of values for all the
>>>> parameters defining a point in parameter space, is there a neat way to
>>>> extract the pseudodesign matrix of the model at the point? That is the
>>>> matrix of partial derivatives of the fitted values w.r.t. the 
>>>> parameters
>>>> evaluated at the point.
>>>> (I have figured out how to extract the gradient information from an 
>>>> nls fitted model using the nlsModel part, but I wish to implement a 
>>>> score test, so I need to be able to extract the information at 
>>>> points other than the mle.)
>>>> Thanks, Murray Jorgensen

______________________________________________ mailing list PLEASE do read the posting guide! Received on Sat Feb 18 09:34:54 2006

This archive was generated by hypermail 2.1.8 : Mon 20 Feb 2006 - 14:08:40 EST