Re: [R] Unit Testing Frameworks: summary and brief discussion

From: hadley wickham <>
Date: Thu, 10 May 2007 11:03:23 +0200

On 5/10/07, Paul Murrell <> wrote:
> Hi
> Paul Gilbert wrote:
> > Tony
> >
> > Thanks for the summary.
> >
> > My ad hoc system is pretty good for catching flagged errors, and
> > numerical errors when I have a check. Could you (or someone else)
> > comment on how easy it would be with one of these more formal frameworks
> > to do three things I have not been able to accomplish easily:
> >
> > - My code gives error and warning messages in some situations. I want to
> > test that the errors and warnings work, but these flags are the correct
> > response to the test. In fact, it is an error if I don't get the flag.
> > How easy is it to set up automatic tests to check warning and error
> > messages work?
> >
> > - For some things it is the printed format that matters. How easy is it
> > to set up a test of the printed output? (Something like the Rout files
> > used in R CMD check.) I think this is what Tony Plate is calling
> > transcript file tests, and I guess it is not automatically available. I
> > am not really interested in something I would have to change with each
> > new release of R, and I need it to work cross-platform. I want to know
> > when something has changed, in R or my own code, without having to
> > examine the output carefully.
> >
> > - (And now the hard one.) For some things it is the plotted output that
> > matters. Is it possible to set up automatic tests of plotting? I can
> > already test that plots run. I want to know if they "look very
> > different". And no, I don't have a clue where to start on this one.
> For text-based graphics formats, you can just use diff; for raster
> formats, you can do per pixel comparisons. These days there is
> ImageMagick to do a compare and it will even produce an image of the
> difference. I have an old package called graphicsQC (not on CRAN) that
> implemented some of these ideas (there was a talk at DSC 2003, see
> A student worked on a
> much better approach more recently, but I haven't put that up on the web
> yet. Let me know if you'd like to take a look at the newer package (it
> would help to have somebody nagging me to get it finished off).

I'd be interested!

Hadley mailing list PLEASE do read the posting guide and provide commented, minimal, self-contained, reproducible code. Received on Thu 10 May 2007 - 09:08:25 GMT

Archive maintained by Robert King, hosted by the discipline of statistics at the University of Newcastle, Australia.
Archive generated by hypermail 2.2.0, at Thu 10 May 2007 - 09:31:46 GMT.

Mailing list information is available at Please read the posting guide before posting to the list.