Re: [R] meta analysis with repeated measure-designs?

From: Mike Cheung <mikewlcheung_at_gmail.com>
Date: Wed, 16 Jun 2010 16:47:29 +0800

Dear Gerrit,

If the correlations of the dependent effect sizes are unknown, one approach is to conduct the meta-analysis by assuming that the effect sizes are independent. A robust standard error is then calculated to adjust for the dependence. You may refer to Hedges et. al., (2010) for more information. I have coded it here for reference.

Hedges, L. V., Tipton, E., & Johnson, M. C. (2010). Robust variance estimation in meta-regression with dependent effect size estimates. Research Synthesis Methods, 1(1), 39-65. doi:10.1002/jrsm.5

Regards,
Mike

-- 
---------------------------------------------------------------------
 Mike W.L. Cheung               Phone: (65) 6516-3702
 Department of Psychology       Fax:   (65) 6773-1843
 National University of Singapore
 http://courses.nus.edu.sg/course/psycwlm/internet/
---------------------------------------------------------------------

library(metafor)

#### Robust SE based on Hedges et al., (2010) Eq. 6
#### rma.obj: object fitted by metafor()
#### cluster: indicator for clusters of studies
robustSE <- function(rma.obj, cluster=NULL, CI=.95) {
  # m: no. of clusters; assumed independent if not specified
  if (is.null(cluster)) {
      m=nrow(rma.obj$X)
  } else {
      m=nlevels(unique(as.factor(cluster)))
  }
  res2 <- diag(residuals(rma.obj)^2)
  X <- rma.obj$X
  b <- rma.obj$b
  W <- diag(1/(rma.obj$vi+rma.obj$tau2))     # Use vi+tau2
  meat <- t(X) %*% W %*% res2 %*% W %*% X    # W is symmetric
  bread <- solve( t(X) %*% W %*% X)
  V.R <- bread %*% meat %*% bread            # Robust sampling covariance matrix
  p <- length(b)                             # no. of predictors
including intercept
  se.R <- sqrt( diag(V.R)*m/(m-p) )          # small sample adjustment (Eq.7)
  t.R <- b/se.R
  p.R <- 2*(1-pt(abs(t.R),df=(m-p)))
  crit <- qt( 1-CI/2, df=(m-p) )
  ci.lb <- b-crit*se.R
  ci.ub <- b+crit*se.R
  data.frame(estimate=b, se.R=se.R, t.R=t.R, p.R=p.R, ci.lb=ci.lb, ci.ub=ci.ub)
}



## Example: calculate log relative risks and corresponding sampling variances
data(dat.bcg) dat <- escalc(measure="RR", ai=tpos, bi=tneg, ci=cpos, di=cneg, data=dat.bcg) dat <- cbind(dat.bcg, dat)
## random-effects model
fit1 <- rma(yi, vi, data=dat, method="DL") summary(fit1) estimate se zval pval ci.lb ci.ub -0.7141 0.1787 -3.9952 <.0001 -1.0644 -0.3638 ***
## Robust SE
robustSE(fit1) estimate se.R t.R p.R ci.lb ci.ub intrcpt -0.7141172 0.1791445 -3.986265 0.001805797 -0.725588 -0.7026465
## mixed-effects model with two moderators (absolute latitude and
publication year) fit2 <- rma(yi, vi, mods=cbind(ablat, year), data=dat, method="DL") summary(fit2) estimate se zval pval ci.lb ci.ub intrcpt -1.2798 25.7550 -0.0497 0.9604 -51.7586 49.1990 ablat -0.0288 0.0090 -3.2035 0.0014 -0.0464 -0.0112 ** year 0.0008 0.0130 0.0594 0.9526 -0.0247 0.0262 ### Robust SE robustSE(fit2) estimate se.R t.R p.R ci.lb intrcpt -1.2797914381 22.860353022 -0.05598301 0.956458098 -2.749670e+00 ablat -0.0287644840 0.007212163 -3.98832970 0.002566210 -2.922821e-02 year 0.0007720798 0.011550188 0.06684565 0.948022174 2.942412e-05 ci.ub intrcpt 0.190086881 ablat -0.028300755 year 0.001514735 On Mon, Jun 14, 2010 at 4:58 PM, Gerrit Hirschfeld <Gerrit.Hirschfeld_at_uni-muenster.de> wrote: > Hi, > > thanks for the references I will try the sensitivitiy-analysis in R and try out winbugs if that does not work (little afraid of switching programmes). > > I also had an idea for a reasonable estimate of the correlations. Some studies report both results from paired t-tests and means and SDs, and thus allow to calculate two estimates for d one based on M and SD alone the other on t. The difference between the two estimates should be systematically related to the correlations of measures. > > I will keep you posted, if I have a solution or hit a wall. > > efachristo and dank je wel! > > Gerrit > > > On 12.06.2010, at 15:59, Viechtbauer Wolfgang (STAT) wrote: > >> Dear Gerrit, >> >> the most appropriate approach for data of this type would be a proper multivariate meta-analytic model (along the lines of Kalaian & Raudenbush, 1996). Since you do not know the correlations of the reaction time measurements across conditions for the within-subject designs, a simple solution is to "guestimate" those correlations and then conduct sensitivity analyses to make sure your conclusions do not depend on those guestimates. >> >> Best, >> >> -- >> Wolfgang Viechtbauer                        http://www.wvbauer.com/ >> Department of Methodology and Statistics    Tel: +31 (0)43 388-2277 >> School for Public Health and Primary Care   Office Location: >> Maastricht University, P.O. Box 616         Room B2.01 (second floor) >> 6200 MD Maastricht, The Netherlands         Debyeplein 1 (Randwyck) >> >> >> ----Original Message---- >> From: r-help-bounces_at_r-project.org [mailto:r-help-bounces_at_r-project.org] On >> Behalf Of Gerrit Hirschfeld Sent: Saturday, June 12, 2010 12:45 >> To: r-help_at_r-project.org >> Subject: [R] meta analysis with repeated measure-designs? >> >>> Dear all, >>> >>> I am trying to run a meta analysis of psycholinguistic reaction-time >>> experiments with the meta package. The problem is that most of the >>> studies have a within-subject designs and use repeated measures ANOVAs to >>> analyze their data. So at present it seems that there are three >>> non-optimal ways to run the analysis. >>> >>> 1. Using metacont() to estimate effect sizes and standard errors. But as >>> the different sores are dependent this would result in biased estimators >>> (Dunlap, 1996). Suppose I had the correlations of the measures (which I >>> do not) would there by an option to use them in metacont() ? >>> >>> 2. Use metagen() with an effect size that is based on the reported F for >>> the contrasts but has other disadvantages (Bakeman, 2005). The problem I >>> am having with this is that I could not find a formular to compute the >>> standard error of partial eta squared. Any Ideas? >>> >>> 3. Use metagen() with r computed from p-values (Rosenthal, 1994) as >>> effect size with the problem that sample-size affects p as much as effect >>> size. >>> >>> Is there a fourth way, or data showing that correlations can be neglected >>> as long as they are assumed to be similar in the studies? >>> Any ideas are much apprecciated. >>> >>> best regards >>> Gerrit >>> >>> ______________________________ >>> Gerrit Hirschfeld, Dipl.-Psych. >>> >>> Psychologisches Institut II >>> Westfälische Wilhelms-Universität >>> Fliednerstr. 21 >>> 48149 Münster >>> Germany >>> >>> psycholinguistics.uni-muenster.de >>> GerritHirschfeld.de >>> Fon.: +49 (0) 251 83-31378 >>> Fon.: +49 (0) 234 7960728 >>> Fax.: +49 (0) 251 83-34104 >>> >>> ______________________________________________ >>> R-help_at_r-project.org mailing list >>> https://stat.ethz.ch/mailman/listinfo/r-help >>> PLEASE do read the posting guide >>> http://www.R-project.org/posting-guide.html >>> and provide commented, minimal, self-contained, reproducible code. > > ______________________________ > Gerrit Hirschfeld, Dipl.-Psych. > > Psychologisches Institut II > Westfälische Wilhelms-Universität > Fliednerstr. 21 > 48149 Münster > Germany > > psycholinguistics.uni-muenster.de > GerritHirschfeld.de > Fon.: +49 (0) 251 83-31378 > Fon.: +49 (0) 234 7960728 > Fax.: +49 (0) 251 83-34104 > > ______________________________________________ > R-help_at_r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide http://www.R-project.org/posting-guide.html > and provide commented, minimal, self-contained, reproducible code. > ______________________________________________ R-help_at_r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Received on Wed 16 Jun 2010 - 08:52:05 GMT

Archive maintained by Robert King, hosted by the discipline of statistics at the University of Newcastle, Australia.
Archive generated by hypermail 2.2.0, at Fri 18 Jun 2010 - 09:40:33 GMT.

Mailing list information is available at https://stat.ethz.ch/mailman/listinfo/r-help. Please read the posting guide before posting to the list.

list of date sections of archive