Re: [Rd] How to execute R scripts simultaneously from multiple threads

From: Vladimir Dergachev <vdergachev_at_rcgardis.com>
Date: Thu 04 Jan 2007 - 17:37:29 GMT

On Thursday 04 January 2007 4:54 am, Erik van Zijst wrote:
> Vladimir Dergachev wrote:
> > On Wednesday 03 January 2007 3:47 am, Erik van Zijst wrote:
> >>Appearantly the R C-API does not provide a mechanism for parallel
> >>execution..
> >>
> >>It is preferred that the solution is not based on multi-processing (like
> >>C/S), because that would introduce IPC overhead.
> >
> > One thing to keep in mind is that IPC is very fast in Linux. So unless
> > you are making lots of calls to really tiny functions this should not be
> > an issue.
>
> Using pipes or shared memory to pass things around to other processes on
> the same box is very fast indeed, but if we base our design around
> something like RServe which uses TCP it could be significantly slower.
> Our R-based system will be running scripts in response to high-volume
> real-time stock exchange data, so we expect lots of calls to many tiny
> functions indeed.

Very interesting :)

If you are running RServe on the other box you will need to send data over ethernet anyway (and will probably use TCP). If it is on the same box and you use "localhost" the packets will go over loopback - which would be significantly faster.

At some point (years ago) there was even an argument on some mailiing list (xfree86-devel ?) about whether Xserver should support shared memory as unix socket was "fast enough" - with the other side arguing that when you pass megabyte images around (as in DVD playback) there is non-negligible overhead.

                       best

                           Vladimir Dergachev

______________________________________________
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel Received on Fri Jan 05 04:42:58 2007

Archive maintained by Robert King, hosted by the discipline of statistics at the University of Newcastle, Australia.
Archive generated by hypermail 2.1.8, at Sat 06 Jan 2007 - 00:31:06 GMT.

Mailing list information is available at https://stat.ethz.ch/mailman/listinfo/r-devel. Please read the posting guide before posting to the list.