Re: [Rd] Brainstorm: Alpha and Beta testing of R versions

From: Martin Maechler <>
Date: Mon 07 Nov 2005 - 21:09:46 GMT

Thanks a lot,
Andrew, for your input!

A general point about your suggestions: You seem to assume that bug reports are typically entered via the R-bugs web interface (which is down at the moment and for a few more dozen hours probably}, rather than via R's builtin function or the simple e-mail to [[which will also not properly work for the moment, as long as the bug repository is suffering from a fiber cable cut in Kopenhagen]].

For some dinosaurs like me, having to fill a web page rather than sending e-mail would be quite a loss of comfort, but actually, it might not be a bad idea to require a unique bug-entry interface -- actually we have been thinking of moving to bugzilla -- if only Peter Dalgaard could find a smart enough person (even to be paid) who'd port all the old bug reports into the new format..

>>>>> "Andrew" == Andrew Robinson <> >>>>> on Sun, 6 Nov 2005 11:01:30 +1100 writes:

    Andrew> Hi Martin, On Fri, Nov 04, 2005 at 09:58:47AM +0100,     Andrew> Martin Maechler wrote:
>> [Mainly for R-foundation members; but kept in public for
>> general brainstorming...]

    Andrew> I'll take up the invitation to brainstorm.

good, thank

    Andrew> As a user of R for a number of years, I'd really
    Andrew> like to perform some useful service.  I use a
    Andrew> relatively obscure platform (FreeBSD) and I can
    Andrew> compile code.  I'd like to think that I'm in the
    Andrew> target market for beta testing :).  


    Andrew> But, I'm timid. I do not feel, in general, that R core welcomes bug     Andrew> reports.

I think that's a partly wrong feeling; understandibly nourished by some of our reactions about some "bug reports" that stemmed from user misconceptions. As you've remarked below, I've expressed gratitude more than once for helpful bug reports.

    Andrew> I think that there are several things that could be
    Andrew> tried to encourage more, and more useful, bug
    Andrew> reports.

    Andrew> 1) Put the following text on the *front page* of the
    Andrew> tracking system, so that it is seen before the     Andrew> reader clicks on "New Bug Report":
    Andrew> "Before submitting a bug report, please read Chapter
    Andrew> `R Bugs' of `The R FAQ'. It describes what a bug is
    Andrew> and how to report a bug.

    Andrew> If you are not sure whether you have observed a bug
    Andrew> or not, it is a good idea to ask on the mailing list
    Andrew> R-Help by sending an e-mail to
    Andrew> rather than submitting a     Andrew> bug report."

    Andrew> (BTW is this true also for alpha/beta testing?)

Yes, in principile. The only thing to be changed would be

   sub("-help", "-devel", <the above text>)

    Andrew> 2) Try to use the structure of the reporting page to
    Andrew> prompt good reporting.  On the report page,
    Andrew> summarize the key points of identifying and
    Andrew> reporting a bug in a checklist format.  Maybe even
    Andrew> insist that the boxes be checked before allowing
    Andrew> submission.  Include seperate text boxes for
    Andrew> description and sample code, to suggest that sample
    Andrew> code is valued.

    Andrew> 3) On either or both pages (and in FAQ), explain
    Andrew> that thoughtful bug reports are valued and
    Andrew> appreciated.  Further, explain that bug reports that
    Andrew> do not follow the protocol are less valuable, and     Andrew> take more time.
    Andrew> 4) Add checkboxes to the report page for alpha/beta.
    Andrew> (I suggest this for the purposes of marketing, not
    Andrew> organization.)

    Andrew> 5) On the report page, include hyperlinks to
    Andrew> archived bug reports that were good. Do likewise     Andrew> with some artificial bug reports that are bad.
    Andrew> 6) Add an intermediate, draft step for bug
    Andrew> submission, to allow checking.  If possible, include
    Andrew> as part of this step an automated pattern matching
    Andrew> call that identifies similarly texted bug reports,
    Andrew> provides links to the reports, and invites a
    Andrew> last-minute cross-check.

    Andrew> 7) Keep a list of people who report useful bugs in
    Andrew> alpha/beta phase on the website. Many academics     Andrew> could point to it as evidence of community service.

>> In order to discourage an increased number of non-bug
>> reports we may have to also open a "hall of shame"
>> though...

    Andrew> 8) I'm sure that you're being ironic!

indeed I was, partly. The point was just that if the bug reporting will be something like a challenge with prizes, we had to discourage too many entries {which would be made just to try to win (a|the) prize}.

    Andrew> 8) I'm sure that you're being ironic!  But I will
    Andrew> take the point seriously, for what it's worth.  I
    Andrew> think that humiliating submitters who haven't
    Andrew> followed the protocol is deleterious.  It seems like
    Andrew> almost every month we see someone get slapped on the
    Andrew> wrist for not doing something the right way.  Of
    Andrew> course, it's frustrating that people aren't
    Andrew> following the posting guide.  But, why is that?
    Andrew> Where is the breakdown?  It might be interesting to
    Andrew> try some follow-up (an exit interview!). If someone
    Andrew> has failed to follow the protocol, perhaps we should
    Andrew> try to find out why it was confusing, or if they
    Andrew> just ignored it.

    Andrew>    The R-core is surrounded by, and serves, a
    Andrew> community that comprises people who are not
    Andrew> sufficiently good at what R-core does to be invited
    Andrew> in to R-core. But, we're clearly interested in what
    Andrew> R-core produces.  Please don't assume that bug
    Andrew> submissions that do not follow the R protocol are     Andrew> the consequence of deliberate malfeasance.
    Andrew>    To paraphrase Ian Fleming: Once is happenstance.
    Andrew> Twice is incompetence.  The third time, Mr. Bond, is
    Andrew> enemy action. So, ...

    Andrew> 9) Publicly thank bug reporters whether their
    Andrew> reports are useful or not. I just googled 'R-devel     Andrew> thank' and you figure prominently, Martin :).

Thanks again, Andrew,
for your useful input!

Martin mailing list Received on Tue Nov 08 08:37:25 2005

This archive was generated by hypermail 2.1.8 : Mon 20 Feb 2006 - 03:21:33 GMT