Re: [Rd] Parallel R CMD check?

From: Paul Gilbert <pgilbert902_at_gmail.com>
Date: Fri, 17 Feb 2012 14:46:30 -0500

On 12-02-17 01:19 PM, Martin Morgan wrote:
> On 02/17/2012 01:42 AM, Prof Brian Ripley wrote:
>> On 17/02/2012 06:50, Martin Morgan wrote:
>>> Running R CMD check on a package can take quite a lot of time. Checks
>>> seem like they could be run in parallel (separate processes for, e.g.,
>>> codoc, examples, tests, ...). Is there a way to do this? My current
>>> usage is typically
>>>
>>> R CMD build <pkg>
>>> R CMD check pkg_x.y.z.tar.gz
>>>
>>> Thanks for any hints,
>>
>> Not at present. It would need a lot of re-organization of the check.R
>> code to collect output and present it in a reasonable order. I rather
>> doubt is worth the effort: most of us with machines with large numbers
>> of cores are not just checking one package at a time, and for many
>> packages with long check times it is one aspect of the check which takes
>> most of the time.
>
> OK thank you. My own 'issue' is that the package takes ~ 20s to load
> (because of dependencies). The check process seems to load the package
> at least 10 times, so 200s in package loading. I'm thinking of this from
> a package developer perspective, rather than checking many packages.
> Obviously from my end I can work to reduce the dependencies (and their
> loading times).

For my own package development testing I use a make target that does something like:

        R CMD BATCH --vanilla tests/$(notdir $@) $@.tmp and then make -j does all the tests in parallel. Of course, when the package is actually working this means that I run the tests twice, once individually with this target, and another time when I do R CMD check (then again on R-forge, and then on CRAN Kurt catches something I missed.) So, when you are really in development mode the above target is useful, because you get to the failure point quickly, but if everything is working then it is slower.

Happy to share more details off-line if you want.

Paul

> Martin
>
>
>>
>> We have considered running separate tests and vignette R code in
>> parallel: it was one of the motivations of having package 'parallel'.
>> But even then people complained when we batched up all the test output
>> and only reported it when all the tests had been run.
>>
>>>
>>> Martin
>>
>>
>
>



R-devel_at_r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel Received on Fri 17 Feb 2012 - 19:50:44 GMT

This quarter's messages: by month, or sorted: [ by date ] [ by thread ] [ by subject ] [ by author ]

All messages

Archive maintained by Robert King, hosted by the discipline of statistics at the University of Newcastle, Australia.
Archive generated by hypermail 2.2.0, at Fri 17 Feb 2012 - 21:00:18 GMT.

Mailing list information is available at https://stat.ethz.ch/mailman/listinfo/r-devel. Please read the posting guide before posting to the list.

list of date sections of archive