Re: [Rd] S4 and Namespaces problems {was "error message from lmer"}

From: John Chambers <jmc_at_r-project.org>
Date: Thu, 19 Jul 2007 09:19:36 -0400

Generic functions include a "package" slot. The concept is that only one definition of the generic _for a particular package_ should exist. For various reasons, there may be "copies" of that generic in multiple places (from the way namespaces work, e.g.) but the model is that they are really the same generic and dispatch the same methods.

Changes now partly in place should enforce this model for 2.6.0. To each (non-generic) function in a package, there corresponds an implicit generic version, either by default or by the owner of the package having used a mechanism that's being added (function implicitGeneric() and friends). The mechanism allows arguments to be excluded from method dispatch or to prohibit a generic version entirely.

As Robert says, there can be other generic functions of the same name, but they will have different package slots, and conceptually will be unrelated to each other.

I think we are all looking for much the same solution. The proposed mechanism is to ensure that all the methods defined for a function foo from package A will be consistent, and use the same definition of the generic. So far as is evident at this point, the mechanism can be implemented with the implicit generic code, and probably some extra code invoked when packages/namespaces are attached/loaded, to catch new "copies" of a given generic.

Extra tinkering for functions in base (other than establishing implicit generics) seems a different issue: The model for generics by package doesn't need to single out functions in base.

John

Prof Brian Ripley wrote:
> On Wed, 18 Jul 2007, Robert Gentleman wrote:
>
>
>> Hi,
>>
>> Martin Maechler wrote:
>>
>>> Here is a reproducible example for the Bug that both Sebastian
>>> and Dale Barr found.
>>>
>>> As Brian mentioned in an another thread,
>>> the problem is in the interaction of Namespaces and S4 generics
>>> and which S4 generic should keep which methods.
>>>
>>> We know there are workarounds, but till now they seem either
>>> ugly or very much against the idea that conceptually there
>>> should be only one generic which may have methods defined in
>>> many different packages / namespaces.
>>>
>> There should *not* be one generic. Generics are no different than any
>> other function. Package A can have a generic named foo, and so can
>> package B. Other packages that want to add methods to a generic named
>> foo need to know which one they would like to add to. These generics
>> can be masked. If Package A is first on the search path then that is the
>> foo that is found first (and if Package B is first then that is the foo,
>> users that specifically want foo from B should use B::foo).
>>
>
> I do think base is a special case here, given its privileged position in
> the namespace hierarchy. I'd take Martin's position as rather that there
> should be only one generic derived by take-over from a function, at least
> for those in base.
>
> I believe that if you are allowed to make a function in base S4-generic
> (and it is not obvious that you should be in many cases), then the derived
> generic should conceptually be in base and thus always be found in place
> of that in base. Since base is not a normal environment or namespace,
> that is not straightforward to implement but it should be possible by
> (e.g.) C-level hacking and a environment to store such derived generics.
> Primitives are rather different, and we now have a fairly efficient way to
> allow designated primitives to have S4 methods.
>
> I believe the registration model for S3 methods is pretty much the right
> one: methods are set on the first 'foo' on the relevant search path, and
> that does not cause another 'foo' to be created elsewhere that may or may
> not precede the original in the relevant search path (since there are
> multiple search paths once namespaces enter the mix).
>
> The new implied generics model helps here: taking over a function in base
> unaltered (the same as the implied generic) could be a derived generic,
> and any alteration would create a new generic in your package. That
> protects code in other namespaces from your alterations.
>
>
>>> I would like us (R-core, mostly) to resolve this as quickly as
>>> possible.
>>>
>
> It seems tricky enough to need debate what we want for a while and aim for
> 2.6.0. But we have been discussing it on and off for at least two years.
>
> Brian
>
>
>>> -------------------------------------------------------------------------
>>>
>>> ### Do this in a fresh R session:
>>>
>>> summary # S3 generic
>>> find("summary") # base
>>>
>>> library(stats4)
>>> summary # S4 generic
>>> find("summary") # stats4 , base
>>>
>>> library(lme4)
>>> ## -> loads Matrix (and lattice)
>>> find("summary") # lme4, Matrix, stats4 , base --- 4 times ! ---
>>>
>> Have they all defined generics? If that is the case then there are 4.
>>
>> We did discuss, and I hope to make progress on the following
>> proposal. For functions in base that have an S4 method defined for them
>> (and hence we need a generic function), that we create a new package
>> that lives slightly above base (and potentially other recommended
>> packages) where these generics will live. Developers can then rely on
>> finding the generic there, and using it - if their intention is to
>> extend the base generic. Note that they may want to have their own
>> generic with the same name as the one from base, and that is fine, it
>> will mask the one in base.
>>
>> best wishes
>> Robert
>>
>>> fm1 <- lmer(Reaction ~ Days + (Days|Subject), sleepstudy)
>>> ## -->
>>> ## Error in lmer(Reaction ~ Days + (Days | Subject), sleepstudy) :
>>> ## cannot get a slot ("Dim") from an object of type "NULL"
>>>
>>> -------------------------------------------------------------------------
>>> Martin Maechler
>>>
>>>
>>>
>>>>>>>> "BDR" == Prof Brian Ripley <ripley_at_stats.ox.ac.uk>

>>>>>>>> on Thu, 28 Jun 2007 06:08:45 +0100 (BST) writes:
>>>>>>>>
>>> BDR> See the thread starting
>>> BDR> https://stat.ethz.ch/pipermail/r-devel/2007-June/046157.html
>>> BDR> https://stat.ethz.ch/pipermail/r-devel/2007-June/046160.html
>>>
>>> BDR> I can't reproduce this without knowing what is in your
>>> BDR> startup files: it should work with --vanilla, so please
>>> BDR> try that and try to eliminate whatever is in your
>>> BDR> .Rprofile etc that is causing the problem.
>>>
>>> BDR> Incidentally, using rcompletion is counterproductive in
>>> BDR> R 2.5.1 RC: the base functionality using rcompgen is a
>>> BDR> more sophisticated version.
>>>
>>> BDR> On Wed, 27 Jun 2007, Sebastian P. Luque wrote:
>>>
>>> >> Hi,
>>> >>
>>> >> I've begun to use the lme4 package, rather than nlme, for more flexibility
>>> >> during modelling, and running the examples in lmer I receive this error
>>> >> message:
>>> >>
>>> >> ---<---------------cut here---------------start-------------->---
>>> R> (fm1 <- lmer(Reaction ~ Days + (Days|Subject), sleepstudy))
>>> >> Error in printMer(object) : no slot of name "status" for this object of class "table"
>>> >>
>>> R> sessionInfo()
>>> >> R version 2.5.1 RC (2007-06-25 r42057)
>>> >> x86_64-pc-linux-gnu
>>> >>
>>> >> locale:
>>> >> LC_CTYPE=en_CA.UTF-8;LC_NUMERIC=C;LC_TIME=en_CA.UTF-8;LC_COLLATE=en_CA.UTF-8;LC_MONETARY=en_CA.UTF-8;LC_MESSAGES=en_CA.UTF-8;LC_PAPER=en_CA.UTF-8;LC_NAME=C;LC_ADDRESS=C;LC_TELEPHONE=C;LC_MEASUREMENT=en_CA.UTF-8;LC_IDENTIFICATION=C
>>> >>
>>> >> attached base packages:
>>> >> [1] "stats4" "stats" "graphics" "grDevices" "utils" "datasets"
>>> >> [7] "methods" "base"
>>> >>
>>> >> other attached packages:
>>> >> lme4 Matrix rcompletion rcompgen lattice diveMove
>>> >> "0.99875-2" "0.99875-2" "0.1-2" "0.1-13" "0.15-11" "0.7-9"
>>> >> reshape
>>> >> "0.7.4"
>>> >> ---<---------------cut here---------------end---------------->---
>>> >>
>>> >> Since this is happening in a fresh session, and with code from examples
>>> >> help file, this looks like a potential bug. Any thoughts?
>>> >>
>>> >>
>>> >> Cheers,
>>>
>>> ______________________________________________
>>> R-devel_at_r-project.org mailing list
>>> https://stat.ethz.ch/mailman/listinfo/r-devel
>>>
>>>
>>
>
>

        [[alternative HTML version deleted]]



R-devel_at_r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel Received on Thu 19 Jul 2007 - 13:55:42 GMT

Archive maintained by Robert King, hosted by the discipline of statistics at the University of Newcastle, Australia.
Archive generated by hypermail 2.2.0, at Thu 19 Jul 2007 - 22:36:45 GMT.

Mailing list information is available at https://stat.ethz.ch/mailman/listinfo/r-devel. Please read the posting guide before posting to the list.