Re: [R] adjusted p-values with TukeyHSD?

From: Sander Oom <slist_at_oomvanlieshout.net>
Date: Wed 18 May 2005 - 00:14:27 EST

Shame I can not get hold of Hsu, J. C. and M. Peruggia (1994) just now. I am quite curious to see what their graphs look like. Would you be able to give an example in R.....? ;-)

The graph I put forward is typically used by ecologists to summarize data. It comes down to a simple means plot with error bars. Significant differences of multiple comparisons are then added using the letters a, b, c etc. If two bars have the same letter, they are not significantly different. It can become quite complicated when mean one is different from mean three but not from mean two and mean two is different from mean three but not mean one. You then get: a, ab, c for mean one, two and three respectively.

Of course what is often used does not constitute the best way of doing it.

Sander.

Liaw, Andy wrote:
>>From: Sander Oom
>>
>>Hi Chris and Chris,
>>
>>I was keeping my eye on this thread as I have also been discovering
>>multiple comparisons recently. Your instructions are very
>>clear! Thanks.

> 
> One thing to note, though:  Multcomp does not do Dunnett's or 
> Tukey's multiple comparisons per se.  Those names in multcomp 
> refer to the contrasts being used (comparison to a control for 
> Dunnett and all pairwise comparison for Tukey).  The actual 
> methods used are as described in the references of the help
> pages.
> 
>  

>>Now I would love to see an R boffin write a nifty function to
>>produce a
>>graphical representation of the multiple comparison, like this one:
>>
>>http://www.theses.ulaval.ca/2003/21026/21026024.jpg
>>
>>Should not be too difficult.....[any one up for the challenge?]
> 
> I beg to differ:  That's probably as bad a way as one can use to 
> graphically show multiple comparison.  The shaded bars serve no 
> purpose.
> 
> Two alternatives that I'm aware of are 
> 
> - Multiple comparison circles, due to John Sall, and not 
>   surprisingly, implemented in JMP and SAS/Insight.  See:
>  
> http://support.sas.com/documentation/onlinedoc/v7/whatsnew/insight/sect4.htm
> 
> 
> - The mean-mean display proposed by Hsu and Peruggia:
>   Hsu, J. C. and M. Peruggia (1994). 
>   Graphical representations of Tukey's multiple comparison method.
>   Journal of Computational and Graphical Statistics 3, 143{161
> 
> Andy
>  

>>I came across more multiple comparison info here;
>>
>>http://www.agr.kuleuven.ac.be/vakken/statisticsbyR/ANOVAbyRr/m
>>ultiplecomp.htm
>>
>>Cheers,
>>
>>Sander.
>>
>>Christoph Buser wrote:
>>>Dear Christoph
>>>
>>>You can use the multcomp package. Please have a look at the
>>>following example:
>>>
>>>library(multcomp)
>>>
>>>The first two lines were already proposed by Erin Hodgess:
>>>
>>>summary(fm1 <- aov(breaks ~ wool + tension, data = warpbreaks))
>>>TukeyHSD(fm1, "tension", ordered = TRUE)
>>>
>>> Tukey multiple comparisons of means
>>> 95% family-wise confidence level
>>> factor levels have been ordered
>>>
>>>Fit: aov(formula = breaks ~ wool + tension, data = warpbreaks)
>>>
>>>$tension
>>> diff lwr upr
>>>M-H 4.722222 -4.6311985 14.07564
>>>L-H 14.722222 5.3688015 24.07564
>>>L-M 10.000000 0.6465793 19.35342
>>>
>>>
>>>By using the functions simtest or simint you can get the
>>>p-values, too:
>>>
>>>summary(simtest(breaks ~ wool + tension, data = warpbreaks,
>>whichf="tension",
>>> type = "Tukey"))
>>>
>>> Simultaneous tests: Tukey contrasts
>>>
>>>Call:
>>>simtest.formula(formula = breaks ~ wool + tension, data =
>>warpbreaks,
>>> whichf = "tension", type = "Tukey")
>>>
>>> Tukey contrasts for factor tension, covariable: wool
>>>
>>>Contrast matrix:
>>> tensionL tensionM tensionH
>>>tensionM-tensionL 0 0 -1 1 0
>>>tensionH-tensionL 0 0 -1 0 1
>>>tensionH-tensionM 0 0 0 -1 1
>>>
>>>
>>>Absolute Error Tolerance: 0.001
>>>
>>>Coefficients:
>>> Estimate t value Std.Err. p raw p Bonf p adj
>>>tensionH-tensionL -14.722 -3.802 3.872 0.000 0.001 0.001
>>>tensionM-tensionL -10.000 -2.582 3.872 0.013 0.026 0.024
>>>tensionH-tensionM -4.722 -1.219 3.872 0.228 0.228 0.228
>>>
>>>
>>>
>>>or if you prefer to get the confidence intervals, too, you can
>>>use:
>>>
>>>summary(simint(breaks ~ wool + tension, data = warpbreaks,
>>whichf="tension",
>>> type = "Tukey"))
>>>
>>> Simultaneous 95% confidence intervals: Tukey contrasts
>>>
>>>Call:
>>>simint.formula(formula = breaks ~ wool + tension, data =
>>warpbreaks,
>>> whichf = "tension", type = "Tukey")
>>>
>>> Tukey contrasts for factor tension, covariable: wool
>>>
>>>Contrast matrix:
>>> tensionL tensionM tensionH
>>>tensionM-tensionL 0 0 -1 1 0
>>>tensionH-tensionL 0 0 -1 0 1
>>>tensionH-tensionM 0 0 0 -1 1
>>>
>>>Absolute Error Tolerance: 0.001
>>>
>>> 95 % quantile: 2.415
>>>
>>>Coefficients:
>>> Estimate 2.5 % 97.5 % t value Std.Err.
>>p raw p Bonf p adj
>>>tensionM-tensionL -10.000 -19.352 -0.648 -2.582 3.872
>>0.013 0.038 0.034
>>>tensionH-tensionL -14.722 -24.074 -5.370 -3.802 3.872
>>0.000 0.001 0.001
>>>tensionH-tensionM -4.722 -14.074 4.630 -1.219 3.872
>>0.228 0.685 0.447
>>>-----------------------------------------------------------------
>>>Please be careful: The resulting confidence intervals in
>>>simint are not associated with the p-values from 'simtest' as it
>>>is described in the help page of the two functions.
>>>-----------------------------------------------------------------
>>>
>>>I had not the time to check the differences in the function or
>>>read the references given on the help page.
>>>If you are interested in the function you can check those to
>>>find out which one you prefer.
>>>
>>>Best regards,
>>>
>>>Christoph Buser
>>>
>>>--------------------------------------------------------------
>>>Christoph Buser <buser@stat.math.ethz.ch>
>>>Seminar fuer Statistik, LEO C13
>>>ETH (Federal Inst. Technology) 8092 Zurich SWITZERLAND
>>>phone: x-41-44-632-4673 fax: 632-1228
>>>http://stat.ethz.ch/~buser/
>>>--------------------------------------------------------------
>>>
>>>
>>>Christoph Strehblow writes:
>>> > hi list,
>>> >
>>> > i have to ask you again, having tried and searched for
>>several days...
>>> >
>>> > i want to do a TukeyHSD after an Anova, and want to get
>>the adjusted
>>> > p-values after the Tukey Correction.
>>> > i found the p.adjust function, but it can only correct
>>for "holm",
>>> > "hochberg", bonferroni", but not "Tukey".
>>> >
>>> > Is it not possbile to get adjusted p-values after
>>Tukey-correction?
>>> >
>>> > sorry, if this is an often-answered-question, but i
>>didn´t find it on
>>> > the list archive...
>>> >
>>> > thx a lot, list, Chris
>>> >
>>> >
>>> > Christoph Strehblow, MD
>>> > Department of Rheumatology, Diabetes and Endocrinology
>>> > Wilhelminenspital, Vienna, Austria
>>> > chrisxe@gmx.at
>>> >
>>> > ______________________________________________
>>> > R-help@stat.math.ethz.ch mailing list
>>> > https://stat.ethz.ch/mailman/listinfo/r-help
>>> > PLEASE do read the posting guide!
>>http://www.R-project.org/posting-guide.html
>>>
>>______________________________________________
>>>R-help@stat.math.ethz.ch mailing list
>>>https://stat.ethz.ch/mailman/listinfo/r-help
>>>PLEASE do read the posting guide!
>>http://www.R-project.org/posting-guide.html
>>
>>--
>>
>>--------------------------------------------
>>Dr Sander P. Oom
>>Animal, Plant and Environmental Sciences,
>>University of the Witwatersrand
>>Private Bag 3, Wits 2050, South Africa
>>Tel (work) +27 (0)11 717 64 04
>>Tel (home) +27 (0)18 297 44 51
>>Fax +27 (0)18 299 24 64
>>Email sander@oomvanlieshout.net
>>Web www.oomvanlieshout.net/sander
>>
>>______________________________________________
>>R-help@stat.math.ethz.ch mailing list
>>https://stat.ethz.ch/mailman/listinfo/r-help
>>PLEASE do read the posting guide!
>>http://www.R-project.org/posting-guide.html
>>
>>
>>
> 
> ______________________________________________
> R-help@stat.math.ethz.ch mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html
> 


-- 
--------------------------------------------
Dr Sander P. Oom
Animal, Plant and Environmental Sciences,
University of the Witwatersrand
Private Bag 3, Wits 2050, South Africa
Tel (work)      +27 (0)11 717 64 04
Tel (home)      +27 (0)18 297 44 51
Fax             +27 (0)18 299 24 64
Email   sander@oomvanlieshout.net
Web     www.oomvanlieshout.net/sander

______________________________________________
R-help@stat.math.ethz.ch mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html
Received on Wed May 18 00:23:30 2005

This archive was generated by hypermail 2.1.8 : Fri 03 Mar 2006 - 03:31:48 EST