Re: [R] Reproducibility of experiment

From: Marc Schwartz <>
Date: Mon, 17 Dec 2007 11:02:40 -0600

On Mon, 2007-12-17 at 16:27 +0100, wrote:
> Dear Marc and R-list,
> thanks for your help. I
> have checked Bland-Altman help
> page about repeatability, and I learnt that instead of
> reproducibility,
> I was talking about repeatability. Although I am not sure whether they
> only
> focuse on agreement of two different measurement methods, and not
> on
> repeatability of one single method.
> To explain further on my topic, I have repeated ten times an
> experiment involving protein quantification(i.e. how much protein I
> have),
> giving me ten continuous values. All experimental settings are similar
> so there should be no variability due to day of experiment, operator
> or any batch effect. My aim is to know whether these ten observations
> are good enough so that I can conclude that the repeatability of my
> detection technique is good. But as I have learnt from Altmanīs page,
> it is not possible to set a threshold to the repeatability score to
> say my experiment is "repeatable".
> I guess I can obtain a 95% confidence interval for the protein
> quantification values,
> but I am not sure this will show how well my experiment performs.
> Putting it differently,
> something I would like to know is whether I can estimate
> beforehand how many times I need to run an experiment in order to be
> confident that it is "repeatable".
> Thanks for your comments
> David



There is information on Prof. Bland's pages pertaining to the questions you ask. If you have not reviewed his FAQ, please do so as it covers issues such as sample size calculations, etc.

If the 10 measures are all of the same quantity, then a simple one sample t-test is all you need to determine whether or not the measured values are significantly different than a presumably known correct value and to get confidence intervals for the mean measurement.

However, if all 10 values are of the same quantity, you will not answer the questions as to whether or not any measurement error is constant/linear over the range of possible values and whether that error is within "acceptable limits". This is what the Bland-Altman methods address.

My recommendation would be to solicit local expertise in the design of such studies, as in reality, all of this should have been specified a priori.

In addition, both Profs. Bland and Altman participate in the MedStats group and that would be a better forum for your queries. More information here:

HTH, Marc Schwartz mailing list PLEASE do read the posting guide and provide commented, minimal, self-contained, reproducible code. Received on Mon 17 Dec 2007 - 17:09:26 GMT

Archive maintained by Robert King, hosted by the discipline of statistics at the University of Newcastle, Australia.
Archive generated by hypermail 2.2.0, at Mon 17 Dec 2007 - 18:30:20 GMT.

Mailing list information is available at Please read the posting guide before posting to the list.