Re: [Rd] serialize() takes too long when serializing to a raw vector

From: Hin-Tak Leung <hin-tak.leung_at_cimr.cam.ac.uk>
Date: Thu 25 Jan 2007 - 13:03:15 GMT

Ashish Kulkarni wrote:
> Hello,
>
> R version 2.4.1 (2006-12-18)
> i386-pc-mingw32
>
> Calling serialize() with a NULL connection serializes it to a raw vector. However, when the object to be serialized is large, it takes a very long time:
>

>> system.time( serialize(matrix(0, 1000, 1000), NULL) )

> [1] 38.25 40.73 81.54 NA NA
>
>> system.time( serialize(matrix(0, 2000, 2000), NULL) )
> [1] 609.72 664.75 1318.57 NA NA

<snipped>

>> system.time( .mpi.quick.serialize(matrix(0, 1000, 1000) ) )

> [1] 0.2500000000000000 0.0499999999999545 0.3000000000001819
> [4] NA NA
>
>> system.time( .mpi.quick.serialize(matrix(0, 2000, 2000) ) )

> [1] 1.059999999999945 0.220000000000027 1.289999999999964
> [4] NA NA
>
> Does anyone have an idea why the performance difference is so
> large? Also, I was wondering if there is a better way -- the
> above solution feels like a quick fix rather than a correct
> approach.

It might be interesting to know get some details on your hardware.

On my box, linux native seems to be a little slower than your quick.serialize times:

 > system.time( serialize(matrix(0, 1000, 1000), NULL) ) [1] 0.372 0.288 0.692 0.000 0.000
 > system.time( serialize(matrix(0, 2000, 2000), NULL) ) [1] 1.237 1.195 2.501 0.000 0.000

running R 2.4.1 windows under wine (same box) is a good deal s lower, but is not anywhere nearly as slow as yours.

 > system.time( serialize(matrix(0, 1000, 1000), NULL) ) [1] 0.00 0.00 6.08 NA NA
 > system.time( serialize(matrix(0, 2000, 2000), NULL) ) [1] 0.01 0.01 78.00 NA NA

 >

Since you mentioned that you are using Rmpi, there is a possibility that you might be calling a different serialize() than base::serialize all together???



R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel Received on Fri Jan 26 00:11:49 2007

Archive maintained by Robert King, hosted by the discipline of statistics at the University of Newcastle, Australia.
Archive generated by hypermail 2.1.8, at Thu 25 Jan 2007 - 13:31:14 GMT.

Mailing list information is available at https://stat.ethz.ch/mailman/listinfo/r-devel. Please read the posting guide before posting to the list.