Re: [Rd] R cmd check and multicore foreach loop

From: Brian G. Peterson <brian_at_braverock.com>
Date: Wed, 17 Aug 2011 07:56:53 -0500

On Wed, 2011-08-17 at 04:50 -0700, Tim Triche, Jr. wrote:
> I'll see if I can put together a self-contained example. Primarily,
> the times that I use multicore (and attempted to use doSMP, mostly
> because one of my users refuses to ditch Windows) are when I am
> reading a ton of binary files, none of which depend on the others.
> This is a blindingly obvious use-case for e.g. doMC and doSMP, yet
> what typically happens is that the entire operation wedges. I'm told
> that doSMP really only works well with Revolution R, but per above, I
> will try to put together a working self-contained example to show
> how.

Remember that physical I/O can bind up the processes too. Having a bunch of processes all trying to read from local disk at the same time (especially when they are all trying to read the same file(s), a problem it seems you may not have) is a recipe for I/O locks that can seize up your processes.

So, if the problem only occurs with physical I/O, the first thing I would try is to move that storage to a storage device on another machine that is tuned for that level of disk I/O.

Regards,

-- 
Brian G. Peterson
http://braverock.com/brian/
Ph: 773-459-4973
IM: bgpbraverock

______________________________________________
R-devel_at_r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Received on Wed 17 Aug 2011 - 12:59:45 GMT

This quarter's messages: by month, or sorted: [ by date ] [ by thread ] [ by subject ] [ by author ]

All messages

Archive maintained by Robert King, hosted by the discipline of statistics at the University of Newcastle, Australia.
Archive generated by hypermail 2.2.0, at Wed 17 Aug 2011 - 16:20:19 GMT.

Mailing list information is available at https://stat.ethz.ch/mailman/listinfo/r-devel. Please read the posting guide before posting to the list.

list of date sections of archive