Re: [R] Weird LM behaviour

From: Thomas Lumley <>
Date: Sat 20 May 2006 - 07:39:02 EST

On Fri, 19 May 2006, Jason Barnhart wrote:

> No, not weird.
> Think of it this way. As you move point (0,2) to (1,2) the slope which was
> 0 is moving towards infinity. Eventually the 3 points are perfectly
> vertical and so must have infinite slope.
> Your delta-x is not sufficiently granular to show the slope change for
> x-values very close to 1 but not yet 1, like 0.999999999. Note lm returns
> NA when x=1.

This turns out not to be the case. Worked to infinite precision the mean of y is 2 at x and at 1, so the infinite-precision slope is exactly zero for all x!=1 and undefined for x=1.

Now, we are working to finite precision and the slope is obtained by solving a linear system that gets increasingly poorly conditioned as x approaches 1. This means that for x not close to 1 the answer should be 0 to withing a small multiple of machine epsilon (and it is) and that for x close to 1 the answer should be zero to within an increasingly large multiple of machine epsilon (and it is).

Without a detailed error analysis of the actual algorithm being used, you can't really predict whether the answer will follow a more-or-less consistent trend or oscillate violently. You can estimate a bound for the error: it should be a small multiple of the condition number of the design matrix times machine epsilon.

As an example of how hard it is to predict exactly what answer you get, if R used the textbook formula for linear regression the bound would be a lot worse, but in this example the answer is slightly closer to zero done that way.

Unless you really need to know, trying to understand why the fourteenth decimal place of a result has the value it does is not worth the effort.

         -thomas mailing list PLEASE do read the posting guide! Received on Sat May 20 07:46:34 2006

Archive maintained by Robert King, hosted by the discipline of statistics at the University of Newcastle, Australia.
Archive generated by hypermail 2.1.8, at Sat 20 May 2006 - 10:10:13 EST.

Mailing list information is available at Please read the posting guide before posting to the list.