Re: [R] loading .RData on distributed system

About this list Date view Thread view Subject view Author view Attachment view

From: Luke Tierney (luke@nokomis.stat.umn.edu)
Date: Sat 21 Jul 2001 - 04:01:47 EST


Message-id: <20010720130147.A12352@nokomis.stat.umn.edu>

As far as I can tell from a quick look the file is opened with fopen
and read with xdr. So unless the xdr implementation R is linked
against is doing something very weird, the input should be buffered.
On my RedHat 7.1 system strace output seems to suggest that the read
is buffered with a buffer size of 4096.

luke

On Fri, Jul 20, 2001 at 12:56:18PM -0400, Matthew Wiener wrote:
> Hi, all.
>
> I'm running R on a beowulf system, with software that can take a list of
> single-processor jobs and parcel them out over the system. (In case it's
> of interest, they're dual pentium 866 machines connected by fast
> ethernet.) The administrators of the system found that I was causing a
> huge load on the file server, and when they investigated it turned out
> that each process was reading in .RData 4 bytes at a time. Not kilobytes,
> bytes. The .RData in question is several megabytes, and more than one
> process is starting up at a time, so apparently it really adds up.
>
> I looked through the documentation, and a little bit in some of the
> source, and haven't been able to find how exactly .RData is read. Is this
> part of the program? Is it something I can set? Is it something strange
> happening with the file system on the beowulf cluster?
>
> Any help appreciated.
>
> Matt Wiener
>
>
> -.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-
> r-help mailing list -- Read http://www.ci.tuwien.ac.at/~hornik/R/R-FAQ.html
> Send "info", "help", or "[un]subscribe"
> (in the "body", not the subject !) To: r-help-request@stat.math.ethz.ch
> _._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._

-- 
Luke Tierney
University of Minnesota                      Phone:           612-625-7843
School of Statistics                         Fax:             612-624-8868
313 Ford Hall, 224 Church St. S.E.           email:      luke@stat.umn.edu
Minneapolis, MN 55455 USA                    WWW:  http://www.stat.umn.edu
-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-
r-help mailing list -- Read http://www.ci.tuwien.ac.at/~hornik/R/R-FAQ.html
Send "info", "help", or "[un]subscribe"
(in the "body", not the subject !)  To: r-help-request@stat.math.ethz.ch
_._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._


About this list Date view Thread view Subject view Author view Attachment view

This archive was generated by hypermail 2.1.3 : Wed 10 Mar 2004 - 08:18:21 EST