Re: [Rd] conflict between lme4 and RMySQL packages (PR#9753)

From: Martin Maechler <maechler_at_stat.math.ethz.ch>
Date: Tue, 26 Jun 2007 15:07:25 +0200

>>>>> "BDR" == Prof Brian Ripley <ripley_at_stats.ox.ac.uk> >>>>> on Tue, 26 Jun 2007 10:51:03 +0100 (BST) writes:

    BDR> You have three summary() functions here:
>> library(RMySQL)

    BDR> Loading required package: DBI
>> library(lme4)

    BDR> Loading required package: Matrix     BDR> Loading required package: lattice
>> find("summary")

    BDR> [1] "package:Matrix" "package:DBI" "package:base"

    BDR> and this is simply a namespace issue (note the environments):

>> summary

    BDR> standardGeneric for "summary" defined from package "base"

    BDR> function (object, ...)
    BDR> standardGeneric("summary")
    BDR> <environment: 0x15122d0>
    BDR> Methods may be defined for arguments: object

>> Matrix:::summary

    BDR> standardGeneric for "summary" defined from package "base"

    BDR> function (object, ...)
    BDR> standardGeneric("summary")
    BDR> <environment: 0x26c6640>
    BDR> Methods may be defined for arguments: object

>> DBI:::summary

    BDR> standardGeneric for "summary" defined from package "base"

    BDR> function (object, ...)
    BDR> standardGeneric("summary")
    BDR> <environment: 0x15122d0>
    BDR> Methods may be defined for arguments: object

    BDR> so the visible summary() from package:Matrix is a copy of that in the DBI     BDR> namespace, not that in the Matrix namespace.

    BDR> Since lme4 imports all of 
    BDR> Matrix, it imports its take-over of summary.  So the issue is not *S4* 
    BDR> dispatch per se, but how the 'methods' package copes with two or more 
    BDR> packages presenting the same generic.

yes. And that's what I meant.

The problem is --- not for the first time --- that in this case, from an S4 point of view, you would want to keep only one version of summary(), namely ``the generic'', and all packages / namespaces loaded and all other   setMethod(summary, ...)
should ``attach'' their methods to that generic. {and of also get an error or at least a loud warning when  re-setting a method for a signature that already has a method  set ``with the generic''. }

I think we should provide useRs and package writers a way to achieve that --- which I think is pretty close to having the S4 generic behave as if there were no namespaces.

The other approach, that some of the bioconductor folks advocate, is for each package to define it's own version of the generic explicitly, by setGeneric(..).
But that also breaks functionality, namely that of all toplevel calls {of summary() in this case} apart from the ones of the last package loaded, where as the namespace-internal calls {e.g., the summary(.) inside printMer(.) in the given case} will continue to work.

    BDR> If you dig a bit deeper,

>> ls(environment(summary)$".AllMTable")

    BDR> [1] "ANY"             "DBIObject"       "MySQLConnection" "MySQLDriver"
    BDR> [5] "MySQLResult"     "lmer2"           "mer"             "sparseMatrix"
    BDR> [9] "summary.lmer2"   "summary.mer"

>> ls(environment(summary)$".AllMTable")
BDR> [1] "ANY" "DBIObject" "MySQLConnection" "MySQLDriver" BDR> [5] "MySQLResult" "lmer2" "mer" "sparseMatrix"
    BDR> [9] "summary.lmer2" "summary.mer"
>> ls(environment(Matrix:::summary)$".AllMTable")
    BDR> [1] "ANY" "sparseMatrix"
>> ls(environment(DBI:::summary)$".AllMTable")
    BDR> [1] "ANY"             "DBIObject"       "MySQLConnection" "MySQLDriver"
    BDR> [5] "MySQLResult"     "lmer2"           "mer"             "sparseMatrix"
    BDR> [9] "summary.lmer2"   "summary.mer"

    BDR> shows the issue: the methods do not get stored on the version in the 
    BDR> Matrix namespace, but rather on the version in package:Matrix which was     BDR> copied from the existing export and not Matrix:::summary.

    BDR> Clearly there is going to be a namespace problem one     BDR> way round: what seems undocumented is which.

Hmm, haven't we been "here" before and come to the conclusion that we'd like a special namespace, ("base4" IIRC ??), for all those "global S4 generics" ?

Martin

    BDR> On Tue, 26 Jun 2007, maechler_at_stat.math.ethz.ch wrote:

>> Thank you, Dale.
>>
>> I think you've revealed a bug in the method dispatch mechanism.
>> I'm adding here a script which confirms your finding and add
>> slightly more insight:
>>
>> library(RMySQL)
>> library(lme4)
>>
>> data(sleepstudy)
>> fm1 <- lmer(Reaction ~ Days + (Days|Subject), sleepstudy)
>>
>> ## MM: First save, then print -- the error is in "print" i.e. show() :
>> s1 <- summary(fm1)
>> s1 #-> error in printMer() which is hidden :
>>
>> debug(lme4:::printMer)
>> s1 # --> printMer(x, ..) calls so <- summary(x) at the very beginning
>> ## and indeed, summary(x) -- *inside* printMer() dispatches wrongly !!!!
>> ## whereas top-level, there is no problem :
>>
>> ## maybe here, or then later
>> undebug(lme4:::printMer)
>>
>> ## This works fine top-level, but gives non-sense when inside printMer() !!
>> ##
>> so <- summary(s1) ## does nothing, since s1 *is* already summary.lmer :
>> stopifnot(identical(so, s1)) # ok
>>
>> ## one other symptom of the same problem:
>> showMethods(summary) ## is fine top-level
>>
>> ## whereas it does not see the correct methods when used from inside printMer()
>> ## i.e., also when debugging
>>
>>
>>

    >>>>>>> "db" == dale barr <dale.barr_at_ucr.edu>
    >>>>>>> on Tue, 26 Jun 2007 00:57:22 +0200 (CEST) writes:

>>
    db> Full_Name: Dale Barr
    db> Version: 2.5.1 (patched)
    db> OS: Ubuntu linux x86_64
    db> Submission from: (NULL) (138.23.70.108)

>>
>>

    db> When RMySQL is loaded in before lme4, the summary() function for lmer objects in     db> the lme4 packages produces the following error:
>>

    db> Error in printMer(object) : no slot of name "status" for this object of class     db> "table"
>>

    db> When RMySQL is loaded AFTER lme4, however, no such error arises. For example,     db> the following code gives the error:
>>
>> >> library(RMySQL)

    db> Loading required package: DBI
>> >> library(lme4)

    db> Loading required package: Matrix     db> Loading required package: lattice
>> >> data(sleepstudy)
>> >> fm1 <- lmer(Reaction ~ Days + (Days|Subject), sleepstudy)
>> >> summary(fm1)

    db> Error in printMer(object) : no slot of name "status" for this object of class     db> "table"
>>

    db> Now, here is the exact same code except that lme4 is loaded before RMySQL. The     db> summary function works properly.
>>
>> >> library(lme4)
    db> Loading required package: Matrix
    db> Loading required package: lattice
    db> library(RMySQL)> library(RMySQL)
    db> Loading required package: DBI

>> >> data(sleepstudy)
>> >> fm1 <- lmer(Reaction ~ Days + (Days|Subject), sleepstudy)
>> >> summary(fm1)
    db> Linear mixed-effects model fit by REML
    db> Formula: Reaction ~ Days + (Days | Subject)
    db> Data: sleepstudy
    db> AIC  BIC logLik MLdeviance REMLdeviance
    db> 1754 1770 -871.8       1752         1744
    db> Random effects:
    db> Groups   Name        Variance Std.Dev. Corr
    db> Subject  (Intercept) 610.835  24.7151
    db> Days         35.056   5.9208  0.067
    db> Residual             655.066  25.5943
    db> number of obs: 180, groups: Subject, 18

>>
    db> Fixed effects:
    db> Estimate Std. Error t value
    db> (Intercept)  251.405      6.820   36.86
    db> Days          10.467      1.546    6.77

>>
    db> Correlation of Fixed Effects:
    db> (Intr)
    db> Days -0.137

>>
>>
    db> MY SESSION INFO:
    db> R version 2.5.1 RC (2007-06-22 r42030)
    db> x86_64-unknown-linux-gnu

>>

    db> locale:
    db> LC_CTYPE=en_US.UTF-8;LC_NUMERIC=C;LC_TIME=en_US.UTF-8;LC_COLLATE=en_US.UTF-8;LC_MONETARY=en_US.UTF-8;LC_MESSAGES=en_US.UTF-8;LC_PAPER=en_US.UTF-8;LC_NAME=C;LC_ADDRESS=C;LC_TELEPHONE=C;LC_MEASUREMENT=en_US.UTF-8;LC_IDENTIFICATION=C
>>
    db> attached base packages:
    db> [1] "stats"     "graphics"  "grDevices" "utils"     "datasets"  "methods"
    db> [7] "base"

>>
    db> other attached packages:
    db> RMySQL         DBI        lme4      Matrix     lattice
    db> "0.6-0"     "0.2-3" "0.99875-2" "0.99875-2"   "0.15-11"

>>
    db> ______________________________________________
    db> R-devel_at_r-project.org mailing list     db> https://stat.ethz.ch/mailman/listinfo/r-devel
>>
>> ______________________________________________
>> R-devel_at_r-project.org mailing list
>> https://stat.ethz.ch/mailman/listinfo/r-devel
>>
    BDR> -- 
    BDR> Brian D. Ripley,                  ripley_at_stats.ox.ac.uk
    BDR> Professor of Applied Statistics,  http://www.stats.ox.ac.uk/~ripley/
    BDR> University of Oxford,             Tel:  +44 1865 272861 (self)
    BDR> 1 South Parks Road,                     +44 1865 272866 (PA)
    BDR> Oxford OX1 3TG, UK                Fax:  +44 1865 272595

    BDR> ______________________________________________
    BDR> R-devel_at_r-project.org mailing list     BDR> https://stat.ethz.ch/mailman/listinfo/r-devel

R-devel_at_r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel Received on Tue 26 Jun 2007 - 13:10:15 GMT

Archive maintained by Robert King, hosted by the discipline of statistics at the University of Newcastle, Australia.
Archive generated by hypermail 2.2.0, at Tue 26 Jun 2007 - 17:35:43 GMT.

Mailing list information is available at https://stat.ethz.ch/mailman/listinfo/r-devel. Please read the posting guide before posting to the list.