# Re: [R] Standard error of coefficient in linear regression

From: Dimitrios Rizopoulos <Dimitris.Rizopoulos_at_med.kuleuven.be>
Date: Sun 17 Sep 2006 - 19:13:02 GMT

these standard errors and other quantities are calculated as by products of the QR decomposition used in lm.fit(). A simple way (but not efficient) to obtain them is:

exped <- c(4.2, 6.1, 3.9, 5.7, 7.3, 5.9) sales <- c(27.1, 30.4, 25.0, 29.7, 40.1, 28.8) S <- data.frame(exped, sales)
lmfit <- lm(sales ~ exped, data = S)

X <- model.matrix(lmfit)
sigma2 <- sum((sales - fitted(lmfit))^2) / (nrow(X) - ncol(X))

sqrt(sigma2)
sqrt(diag(solve(crossprod(X))) * sigma2)

I hope it helps.

Best,
Dimitris

Dimitris Rizopoulos
Ph.D. Student
Biostatistical Centre
School of Public Health
Catholic University of Leuven

Tel: +32/(0)16/336899
Fax: +32/(0)16/337015
Web: http://med.kuleuven.be/biostat/
http://www.student.kuleuven.be/~m0390867/dimitris.htm



Quoting Maciej Bliziński <m.blizinski@wit.edu.pl>:

> Hello R users,
>
> I have a substantial question about statistics, not about R itself, but
> I would love to have an answer from an R user, in form of an example in
> R syntax. I have spent whole Sunday searching in Google and browsing the
> books. I've been really close to the answer but there are at least three
> standard errors you can talk about in the linear regression and I'm
> really confused. The question is:
>
> How exactly are standard errors of coefficients calculated in the linear
> regression?
>
> Here's an example from a website I've read [1]. A company wants to know
> if there is a relationship between its advertising expenditures and its
> sales volume.
>
> ========================================================
>> exped <- c(4.2, 6.1, 3.9, 5.7, 7.3, 5.9)
>> sales <- c(27.1, 30.4, 25.0, 29.7, 40.1, 28.8)
>> S <- data.frame(exped, sales)
>> summary(lm(sales ~ exped, data = S))
>
> Call:
> lm(formula = sales ~ exped, data = S)
>
> Residuals:
> 1 2 3 4 5 6
> 1.7643 -1.9310 0.7688 -1.1583 3.3509 -2.7947
>
> Coefficients:
> Estimate Std. Error t value Pr(>|t|)
> (Intercept) 9.8725 5.2394 1.884 0.1326
> exped 3.6817 0.9295 3.961 0.0167 *
> ---
> Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
>
> Residual standard error: 2.637 on 4 degrees of freedom
> Multiple R-Squared: 0.7968, Adjusted R-squared: 0.7461
> F-statistic: 15.69 on 1 and 4 DF, p-value: 0.01666
> ========================================================
>
> I can calculate the standard error of the estimate, according to the
> equation [2]...
>
>> S.m <- lm(sales ~ exped, data = S)
>> S$pred <- predict(S.m) >> S$ye <- S$sales - S$pred
>> S$ye2 <- S$ye ^ 2
>> Se <- sqrt(sum(S$ye2)/(length(S$sales) - 1 - 1))
>> Se
> [1] 2.636901
>
> ...which matches the "Residual standard error" and I'm on the right
> track. Next step would be to use the equation [3] to calculate the
> standard error of the regression coefficient (here: exped). The equation
> [3] uses two variables, meaning of which I can't really figure out. As
> the calculated value Sb is scalar, all the parameters need also to be
> scalars. I've already calculated Se, so I'm missing x and \bar{x}. The
> latter could be the estimated coefficient. What is x then?
>
> Regards,
> Maciej
>
> [1] http://www.statpac.com/statistics-calculator/correlation-regression.htm
> [2]
>
> --
> Maciej Bliziński <m.blizinski@wit.edu.pl>
> http://automatthias.wordpress.com
>
> ______________________________________________
> R-help@stat.math.ethz.ch mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help