Re: [R] meta-analysis on diagnostic tests (bivariate approach)

From: Emmanuel Charpentier <>
Date: Fri, 30 Nov 2007 22:08:35 +0100

Pedro Emmanuel Alvarenga Americano do Brasil a écrit :
> Hello friends of R list,
> Im a physician and Im not that good in statistics. I have posted similar
> email in the epi-sig list but no one aswered so far. Im cunducting a
> systematic review on two diagnostic test for a particular tropical disease.
> I found a free software that make most of the analysis called MetaDiSc.
> However there is a paticular analysis that I wuould like to try that it is
> not implemented in this software. I looked in R for meta-analysis functions
> but the packeges available dont work with diagnostic tests.
> Im trying for a while to adapt in R a function develped in SAS and published
> in "J.B. Reitsma et al. / Journal of Clinical Epidemiology 58 (2005)
> 982?990", wich is a bivariate approach to meta-analysis of diagnotic tests.

lmer (in package lme4, IIRC), which *does* treat logistic regression (among other GLMs) directly, should do the trick. Probably with more ease...

> Since I do not have any experience with SAS, Im having a hard time trying to
> do so.
> There is an appendix at the original text with some step by step (in SAS
> syntax) on how to do it but Im stuck on a more advanced stuff which I cant
> find a solution. In a step it is mentioned a Van Houwelingen (Statist. Med.
> 2002; 21:589–624) approach to incorporate between and within study variances
> and using the restricted maximum likelihood estimation.

This is a large tutorial paper, which goes well beyond your goals.

> This is the script (or function) so far....
> bivar<-function(TP,FP,FN,TN){
> # find a way to sum 0.5 to each null cell if any
> Se<-TP/(TP+FN)
> Sp<-TN/(TN+FP)
> logSe<-log(Se/(1-Se))
> logSp<-log(Sp/(1-Sp))
> vlogSe<-1/(Se*(1-Se)*(TP+FN))
> vlogSp<-1/(Sp*(1-Sp)*(TN+FP))
> bsvlogSp<-0
> cblog<-0
> bsvlogSp<-0
> sdata<-rbind(bsvlogSp,cblog,bsvlogSp,vlogSe,vlogSp)
> Does anyone know if there is any package or function with this bivariate
> random effect model that I cuold try to use? Neither a I could find any
> apckage or function to aply the restricted maximum likelihood estimation.
> Any tip that helps me go on or hand in the development of this script
> (function) would be most welcome.

I'll try to have a look on this problem one of these days (I already have a serious backlog in daylight life...).

Out of the top of (what I use as) my mind (assuming that technique and study are factors correctly identifying studies and compared techniques (and forgetting +0.5 in empty cells which can be done along the lines of TP<-TP+0.5*(TP==0), etc ...)) :

data<-rbind(data.frame(suc=TP, fai=FN,

                       study=study, technique=technique),
            data.frame(suc=TN, fai=FP,
                       study=study, technique=technique))


model<-lmer(cbind(suc,fai)~situation:technique-1+(1|situation %in% study), data=data, family=binomial(link=logit))

should be close to what you're aiming at. But asking for Douglas Bates' advice (in the R-SIG mixed model list) would be much better...

However, I am not a sanguine as Reitsma & al about the whole ROC methodology. It has sound foundations for one specific problem : modeling (a simplified type of) signal perception. Its use in medical diagnostic evaluation is perfecly justified for problems of this kind (e. g. choosing, for example, a cutoff point to interpret a biological measurement in clinical terms, or comparing two numerical diagnostic indices).

However, this approach ignores the "costs" and "utilities" of a false positive and a false negative, which may lead to very different choices and judgement criteria (I think that Frank Harrell has much better credentials than mine to explain this point).

Furthermore, I have reservations about the (ab?)use of this model in situation when there is no numerical measure for which a cutoff point is to be choosen (e. g. ROC methodology in radiology) : while this is curent practice, I am not sure what "real world" meaning such a ROC curve has : as far as I can tell, a physician does *not* choose (reliably) his/her sensitivity level, and therefore cannot "move" his/her operating point along this hypothetical ROC curve.

If one accepts the hypothesis that sensitivities and specificities are the only relevant data, the DOR estimation makes more sense (less untestable hypotheses...).

                                        Emmanuel Charpentier mailing list PLEASE do read the posting guide and provide commented, minimal, self-contained, reproducible code. Received on Fri 30 Nov 2007 - 21:06:13 GMT

Archive maintained by Robert King, hosted by the discipline of statistics at the University of Newcastle, Australia.
Archive generated by hypermail 2.2.0, at Fri 30 Nov 2007 - 21:30:16 GMT.

Mailing list information is available at Please read the posting guide before posting to the list.