Re: [R] Rating R Helpers

From: John Sorkin <>
Date: Sun, 02 Dec 2007 22:51:24 -0500

I believe we need to know the following about packages:
(1) Does the package do what it purports to do, i.e. are the results valid?
(2) Have the results generated by the package been validate against some other statistical package, or hand-worked example?
(3) Are the methods used in the soundly based?
(4) Does the package documentation refer to referred papers or textbooks?
(5) In addition to the principle result, does the package return ancillary values that allow for proper interpretation of the main result, (e.g. lm gives estimates of the betas and their SEs, but also generates residuals)?.
(6) Is the package easy to use, i.e. do the parameters used when invoking the package chosen so as to allow the package to be flexible?
(7) Are the error messages produced by the package helpful?
(8) Does the package conform to standards of R coding and good programming principles in general?
(9) Does the package interact will with the larger R environment, e.g. does it have a plot method etc.?
(10) Is the package well documented internally, i.e. is the code easy to follow, are the comments in the code adequate?
(11) Is the package well documented externally, i.e. through man pages and perhaps other documentation (e.g. MASS and its associated textbook)?

In addition to package evaluation and reviews, we also need some plan for the future of R. Who will maintain, modify, and extend packages after the principle author, or authors, retire? Software is never "done". Errors need to be corrected, programs need to be modified to accommodate changes in software and hardware. I have reasonable certainty that commercial software (e.g. SAS) will be available in 10-years (and that PROC MIXED will still be a part of SAS). I am far less sanguine about any number of R packages. John

John Sorkin M.D., Ph.D.
Chief, Biostatistics and Informatics

University of Maryland School of Medicine Division of Gerontology Baltimore VA Medical Center
10 North Greene Street
Baltimore, MD 21201-1524
(Phone) 410-605-7119
(Fax) 410-605-7913 (Please call phone number above prior to faxing)

>>> <> 12/1/2007 2:21 AM >>> This seems a little impractical to me. People respond so much at random and most only tackle questions with which they feel comfortable. As it's not a competition in any sense, it's going to be hard to rank people in any effective way. But suppose you succeed in doing so, then what?

To me a much more urgent initiative is some kind of user online review system for packages, even something as simple as that used by has for customer review of books.

I think the need for this is rather urgent, in fact. Most packages are very good, but I regret to say some are pretty inefficient and others downright dangerous. You don't want to discourage people from submitting their work to CRAN, but at the same time you do want some mechanism that allows users to relate their experience with it, good or bad.

Bill Venables
CSIRO Laboratories
PO Box 120, Cleveland, 4163

Office Phone (email preferred): +61 7 3826 7251
Fax (if absolutely necessary):  +61 7 3826 7304
Mobile:                         +61 4 8819 4402
Home Phone:                     +61 7 3286 7700

-----Original Message-----
From: [] On Behalf Of Doran, Harold
Sent: Saturday, 1 December 2007 6:13 AM
To: R Help
Subject: [R] Rating R Helpers

Since R is open source and help may come from varied levels of experience on R-Help, I wonder if it might be helpful to construct a method that can be used to "rate" those who provide help on this list.

This is something that is done on other comp lists, like

I think some of the reasons for this are pretty transparent, but I suppose one reason is that one could decide to implement the advise of those with "superior" or "expert" levels. In other words, you can trust the advice of someone who is more experienced more than someone who is not. Currently, there is no way to discern who on this list is really an R expert and who is not. Of course, there is R core, but most people don't actually know who these people are (at least I surmise that to be true).

If this is potentially useful, maybe one way to begin the development of such ratings is to allow the original poster to "rate" the level of help from those who responded. Maybe something like a very simple questionnaire on a likert-like scale that the original poster would respond to upon receiving help which would lead to the accumulation of points for the responders. Higher points would result in higher levels of expertise (e.g., novice, ..., wizaRd).

Just a random thought. What do others think?


        [[alternative HTML version deleted]] mailing list PLEASE do read the posting guide and provide commented, minimal, self-contained, reproducible code. mailing list PLEASE do read the posting guide and provide commented, minimal, self-contained, reproducible code.

Confidentiality Statement:
This email message, including any attachments, is for th...{{dropped:6}} mailing list PLEASE do read the posting guide and provide commented, minimal, self-contained, reproducible code. Received on Mon 03 Dec 2007 - 03:54:42 GMT

Archive maintained by Robert King, hosted by the discipline of statistics at the University of Newcastle, Australia.
Archive generated by hypermail 2.2.0, at Wed 05 Dec 2007 - 02:30:16 GMT.

Mailing list information is available at Please read the posting guide before posting to the list.