Re: [Rd] serialization regression in 2.15.0 beta

From: Ben Goodrich <bg2382_at_columbia.edu>
Date: Tue, 27 Mar 2012 17:01:01 -0400

In case anyone is concerned that this regression will affect them, the code was reverted to the 2.14.x behavior by



r58842 | ripley | 2012-03-26 08:12:43 -0400 (Mon, 26 Mar 2012) | 1 line Changed paths:
   M /branches/R-2-15-branch/doc/NEWS.Rd
   M /branches/R-2-15-branch/src/library/parallel/R/unix/forkCluster.R
   M /branches/R-2-15-branch/src/library/parallel/R/unix/mcfork.R

revert to XDR serialization for 2.15.0


Thanks,
Ben

> I am experiencing a problem related to serialization behavior in
> 2.15.0 beta (binary installed from Debian unstable) and 2.16.0 (from
> svn) that is not present in 2.14.2 (binary from Debian testing).
>
> I don't fully understand the problem. Also, I tried but have not yet
> been able to create a small, self-contained example that reproduces
> the problem. However, I do have a large, not self-contained example,
> which requires an alpha version (not yet on CRAN) of the mi package
> (the mi package on CRAN would not exhibit this issue). Anyone
> interested in reproducing the problem can follow the readme.txt file
> in this directory:
>
> http://www.columbia.edu/~bg2382/mi/serialization/
>
> I track r-devel with git-svn and was able to git bisect to svn commit r58219
>
> commit 799102bd9d0266fe89c3120981decf0b1f17ef11
> Author: ripley <ripley at 00db46b3-68df-0310-9c12-caf00c1e9a41>
> Date: Sat Jan 28 15:02:34 2012 +0000
>
> make use of non-xdr serialization;.
>
> although this commit could merely expose the problem rather than cause it.
>
> The problem occurs when the FUN called by mclapply() in the parallel
> package returns a S4 object that contains a slot (called X) that is a
> large matrix, specifically a "model matrix" similar to that produced
> by glm(). Some columns of this matrix get corrupted with wrong values
> (usually zero, but sometimes NaN or 10^300ish), which can be seen by
> examining X right before FUN returns (to mclapply()'s environment) and
> comparing to the "same" X after mclapply() returns to the calling
> environment.
>
> Part of svn commit r58219 is this hunk
>
> diff --git a/src/library/parallel/R/unix/mcfork.R
> b/src/library/parallel/R/unix/mcfork.R
> index 8e27534..4f92193 100644
> --- a/src/library/parallel/R/unix/mcfork.R
> +++ b/src/library/parallel/R/unix/mcfork.R
> @@ -82,7 +82,8 @@ mckill <- function(process, signal = 2L)
> ## used by mcparallel, mclapply
> sendMaster <- function(what)
> {
> - if (!is.raw(what)) what <- serialize(what, NULL, FALSE)
> + # This is talking to the same machine, so no point in using xdr.
> + if (!is.raw(what)) what <- serialize(what, NULL, xdr = FALSE)
> .Call(C_mc_send_master, what, PACKAGE = "parallel")
> }
>
> Contrary to the comment, I have found that if I specify xdr = TRUE, I
> get the expected (non-corrupted X slot) behavior in 2.16.0, even
> though it is forking locally on my 64bit Debian laptop with a little
> endian i7 processor, whose specs are
>
> goodrich at CYBERPOWERPC:/tmp/serialization$ cat /proc/cpuinfo
> processor : 0
> vendor_id : GenuineIntel
> cpu family : 6
> model : 42
> model name : Intel(R) Core(TM) i7-2630QM CPU @ 2.00GHz
> stepping : 7
> microcode : 0x17
> cpu MHz : 800.000
> cache size : 6144 KB
> physical id : 0
> siblings : 8
> core id : 0
> cpu cores : 4
> apicid : 0
> initial apicid : 0
> fpu : yes
> fpu_exception : yes
> cpuid level : 13
> wp : yes
> flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge
> mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe
> syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl
> xtopology nonstop_tsc aperfmperf pni pclmulqdq dtes64 monitor ds_cpl
> vmx est tm2 ssse3 cx16 xtpr pdcm pcid sse4_1 sse4_2 x2apic popcnt
> tsc_deadline_timer xsave avx lahf_lm ida arat epb xsaveopt pln pts dts
> tpr_shadow vnmi flexpriority ept vpid
> bogomips : 3990.83
> clflush size : 64
> cache_alignment : 64
> address sizes : 36 bits physical, 48 bits virtual
> power management:
>
> ...
>
> processor : 7
> [same as processor 0]
>
> So, to summarize I get the good behavior on R 2.14.2 when using
> mclapply(), on 2.15.0 beta when using lapply(), and on 2.16.0 using
> mclapply() iff I patch in xdr = TRUE in sendMaster(). I get the bad
> behavior on 2.15.0 beta and unpatched 2.16.0 when using mclapply().
>
> My session info:
>

>> sessionInfo()

> R version 2.15.0 beta (2012-03-16 r58769)
> Platform: x86_64-pc-linux-gnu (64-bit)
>
> locale:
> [1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C
> [3] LC_TIME=en_US.UTF-8 LC_COLLATE=en_US.UTF-8
> [5] LC_MONETARY=en_US.UTF-8 LC_MESSAGES=en_US.UTF-8
> [7] LC_PAPER=C LC_NAME=C
> [9] LC_ADDRESS=C LC_TELEPHONE=C
> [11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C
>
> attached base packages:
> [1] stats4 stats graphics grDevices utils datasets methods
> [8] base
>
> other attached packages:
> [1] mi_0.9-83 bigmemory_4.2.11 arm_1.5-03 foreign_0.8-49
> [5] abind_1.4-0 R2WinBUGS_2.1-18 coda_0.14-5 lme4_0.999375-42
> [9] Matrix_1.0-4 lattice_0.20-0 MASS_7.3-17
>
> loaded via a namespace (and not attached):
> [1] grid_2.15.0 nlme_3.1-103
>
> Thanks,
> Ben


R-devel_at_r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel Received on Tue 27 Mar 2012 - 21:20:22 GMT

This quarter's messages: by month, or sorted: [ by date ] [ by thread ] [ by subject ] [ by author ]

All messages

Archive maintained by Robert King, hosted by the discipline of statistics at the University of Newcastle, Australia.
Archive generated by hypermail 2.2.0, at Wed 28 Mar 2012 - 09:50:35 GMT.

Mailing list information is available at https://stat.ethz.ch/mailman/listinfo/r-devel. Please read the posting guide before posting to the list.

list of date sections of archive