Re: [Rd] How to execute R scripts simultaneously from multiple threads

From: Erik van Zijst <r_at_erik.prutser.cx>
Date: Fri 05 Jan 2007 - 10:29:48 GMT

Vladimir Dergachev wrote:

>>Using pipes or shared memory to pass things around to other processes on
>>the same box is very fast indeed, but if we base our design around
>>something like RServe which uses TCP it could be significantly slower.
>>Our R-based system will be running scripts in response to high-volume
>>real-time stock exchange data, so we expect lots of calls to many tiny
>>functions indeed.

>
> Very interesting :)
>
> If you are running RServe on the other box you will need to send data over
> ethernet anyway (and will probably use TCP). If it is on the same box and you
> use "localhost" the packets will go over loopback - which would be
> significantly faster.

I'd like to do as little IO as possible, so I'm looking at using a single, very powerfull box and keep everything local.

> At some point (years ago) there was even an argument on some mailiing list
> (xfree86-devel ?) about whether Xserver should support shared memory as unix
> socket was "fast enough" - with the other side arguing that when you pass
> megabyte images around (as in DVD playback) there is non-negligible overhead.

We're currently doing performance tests with the RServe-approach where we measure the actual evaluation time of a function. I'm interested in the evaluation-time versus overhead ratio. Loopback TCP might work as long as this ratio is sufficiently high.

Erik

-- 
Practice is the best of all instructors.
		-- Publilius

______________________________________________
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Received on Sat Jan 06 10:34:23 2007

Archive maintained by Robert King, hosted by the discipline of statistics at the University of Newcastle, Australia.
Archive generated by hypermail 2.1.8, at Mon 08 Jan 2007 - 18:31:10 GMT.

Mailing list information is available at https://stat.ethz.ch/mailman/listinfo/r-devel. Please read the posting guide before posting to the list.