Re: [R] Standard error of coefficient in linear regression

From: Charles Annis, P.E. <Charles.Annis_at_statisticalengineering.com>
Date: Sun 17 Sep 2006 - 19:23:52 GMT


An easier way is to use summary()

summary(lmfit)

or

> summary(lmfit)$coefficients

            Estimate Std. Error t value Pr(>|t|) (Intercept) 9.872541 5.2394254 1.884279 0.13262386 exped 3.681715 0.9294818 3.961040 0.01666313

or

> summary(lmfit)$coefficients[,2]

(Intercept) exped
  5.2394254 0.9294818

Charles Annis, P.E.

Charles.Annis@StatisticalEngineering.com phone: 561-352-9699
eFax: 614-455-3265
http://www.StatisticalEngineering.com  

-----Original Message-----
From: r-help-bounces@stat.math.ethz.ch
[mailto:r-help-bounces@stat.math.ethz.ch] On Behalf Of Dimitrios Rizopoulos Sent: Sunday, September 17, 2006 3:13 PM To: Maciej Bliziński
Cc: R - help
Subject: Re: [R] Standard error of coefficient in linear regression

these standard errors and other quantities are calculated as by products of the QR decomposition used in lm.fit(). A simple way (but not efficient) to obtain them is:

exped <- c(4.2, 6.1, 3.9, 5.7, 7.3, 5.9) sales <- c(27.1, 30.4, 25.0, 29.7, 40.1, 28.8) S <- data.frame(exped, sales)
lmfit <- lm(sales ~ exped, data = S)

X <- model.matrix(lmfit)
sigma2 <- sum((sales - fitted(lmfit))^2) / (nrow(X) - ncol(X))

sqrt(sigma2)
sqrt(diag(solve(crossprod(X))) * sigma2)

I hope it helps.

Best,
Dimitris



Dimitris Rizopoulos
Ph.D. Student
Biostatistical Centre
School of Public Health
Catholic University of Leuven

Address: Kapucijnenvoer 35, Leuven, Belgium

Tel: +32/(0)16/336899
Fax: +32/(0)16/337015
Web: http://med.kuleuven.be/biostat/
      http://www.student.kuleuven.be/~m0390867/dimitris.htm


Quoting Maciej Bliziński <m.blizinski@wit.edu.pl>:

> Hello R users,
>
> I have a substantial question about statistics, not about R itself, but
> I would love to have an answer from an R user, in form of an example in
> R syntax. I have spent whole Sunday searching in Google and browsing the
> books. I've been really close to the answer but there are at least three
> standard errors you can talk about in the linear regression and I'm
> really confused. The question is:
>
> How exactly are standard errors of coefficients calculated in the linear
> regression?
>
> Here's an example from a website I've read [1]. A company wants to know
> if there is a relationship between its advertising expenditures and its
> sales volume.
>
> ========================================================
>> exped <- c(4.2, 6.1, 3.9, 5.7, 7.3, 5.9)
>> sales <- c(27.1, 30.4, 25.0, 29.7, 40.1, 28.8)
>> S <- data.frame(exped, sales)
>> summary(lm(sales ~ exped, data = S))
>
> Call:
> lm(formula = sales ~ exped, data = S)
>
> Residuals:
> 1 2 3 4 5 6
> 1.7643 -1.9310 0.7688 -1.1583 3.3509 -2.7947
>
> Coefficients:
> Estimate Std. Error t value Pr(>|t|)
> (Intercept) 9.8725 5.2394 1.884 0.1326
> exped 3.6817 0.9295 3.961 0.0167 *
> ---
> Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
>
> Residual standard error: 2.637 on 4 degrees of freedom
> Multiple R-Squared: 0.7968, Adjusted R-squared: 0.7461
> F-statistic: 15.69 on 1 and 4 DF, p-value: 0.01666
> ========================================================
>
> I can calculate the standard error of the estimate, according to the
> equation [2]...
>
>> S.m <- lm(sales ~ exped, data = S)
>> S$pred <- predict(S.m)
>> S$ye <- S$sales - S$pred
>> S$ye2 <- S$ye ^ 2
>> Se <- sqrt(sum(S$ye2)/(length(S$sales) - 1 - 1))
>> Se
> [1] 2.636901
>
> ...which matches the "Residual standard error" and I'm on the right
> track. Next step would be to use the equation [3] to calculate the
> standard error of the regression coefficient (here: exped). The equation
> [3] uses two variables, meaning of which I can't really figure out. As
> the calculated value Sb is scalar, all the parameters need also to be
> scalars. I've already calculated Se, so I'm missing x and \bar{x}. The
> latter could be the estimated coefficient. What is x then?
>
> Regards,
> Maciej
>
> [1]
http://www.statpac.com/statistics-calculator/correlation-regression.htm
> [2] http://www.answers.com/topic/standard-error-of-the-estimate
> [3]
http://www.answers.com/topic/standard-error-of-the-regression-coefficient
>
> --
> Maciej Bliziński <m.blizinski@wit.edu.pl>
> http://automatthias.wordpress.com
>
> ______________________________________________
> R-help@stat.math.ethz.ch mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide
http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>

Disclaimer: http://www.kuleuven.be/cwis/email_disclaimer.htm



R-help@stat.math.ethz.ch mailing list
https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.

R-help@stat.math.ethz.ch mailing list
https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code. Received on Mon Sep 18 05:28:01 2006

Archive maintained by Robert King, hosted by the discipline of statistics at the University of Newcastle, Australia.
Archive generated by hypermail 2.1.8, at Sun 17 Sep 2006 - 20:30:15 GMT.

Mailing list information is available at https://stat.ethz.ch/mailman/listinfo/r-help. Please read the posting guide before posting to the list.