Ok, I was not wrong with the above: from the docs
to quote (about __*__)
System-defined names. These names are defined by the interpreter and its implementation (including the standard library). (...) More will likely be defined in future versions of Python. Any use of
__*__names, in any context, that does not follow explicitly documented use, is subject to breakage without warning.
_function is not good and __function__ is not good either.Then there is the practical solution:
- having the pattern "public-calls-private", documenting that well (like scipy, tensorflow probability does) and saying explicitly that this is part of the internal api is fine
- the same goes for "fancy enough names"
__*__methods (as e.g. numpy does): python won't have (most likely) a__array_function__and if so, they slightly rename it (given the importance of numpy). But realistically, this won't happen anyway
Although the surrounding context for this quote is talking about what modules import, and therefore names defined in module scope. Above, we're talking about names defined in a class's static scope (a nested namespace).
Not really. a) it's basically the same in Python :) and b), the one leading underscore below talks explicitliy about class private names
_* methods are stable) instead of using a non-guaranteed feature. But realistically, I agree: adding the name of the library (__*__) really makes things "safe" again.
__*__ might be a rule defined by the Python docs but disregarded by the community, but _* is in agreement: it's for internal APIs with no guarantee of stability.
__*__. If I can't think of any examples beyond the standard library and Numpy (__array__, __array_interface__, __array_ufunc__, __array_function__!), then perhaps Numpy has special status, it's "extended standard library..."
Thanks for pointing me to the documentation about
__*__. If I can't think of any examples beyond the standard library and Numpy (__array__,__array_interface__,__array_ufunc__,__array_function__!), then perhaps Numpy has special status, it's "extended standard library..."
Yes, numpy special status could go on... I don't know any other and remember to had long discussions already about it
_*, it's not "golden"
_pdf etc
The links get you directly there: Scipy and TensorFlow Probability (and TensorFlow itself)
E.g. (TFP): prob (_prob), log_prob,...
(scipy): _logpdf, _cdf, _logcdf, _ppf, _rvs, _isf, _sf, _logsf
The advantage of this method is for the case where you need "preprocessing":
- the naming is clear. You could have another convention (
user_pdfor whatever), but why not_*? - it is private in the class API. So it is clear from the outside not to call it.
I think the question of _* boils down to whom should it be private to. The subclasser or the invoker. Saying that a user can only override public methods implies that a users method is always exposed (except you add another layer of indirection of course). The question that remains: is there a practical, equivalent way to do it?
(Not talking about e.g. registering integrals etc., this mechanisms are complementary to me)
The links get you directly there: Scipy and TensorFlow Probability (and TensorFlow itself)
exactly. The official documentation is following the link above
So SciPy and TensorFlow really do require "people who are defining new probability distributions" (one kind of user) to override methods starting with a single underscore so that "people who use those probability distributions" (another kind of user) know that they're not supposed to call them directly. It's exactly the pattern I was advocating the __*__ for, but there's a rule against that in the central Python documentation.
I stand corrected.
_* can't change for the same reason that protocols defined by __*__ can't change.
It's a use of "privateness" that is unrelated to stability of API: the protocols defined by
_*can't change for the same reason that protocols defined by__*__can't change.
Fully agree!
_awkward_serialize_.
__*__ name matching the one they are using; but that’s only becuase they are numpy - some parts of the python syntax, like @ and …, were added for Numpy. Other users should never add a name like that (at least they are not supposed to)
_*_ might work for what you want to do.
__*__ rule could be relaxed to allow libraries to implement __*_*__
Tbh, I am not convinced of the __*_*__ in general. So for the numpy or uproot use case, that looks good, but for the general subclassing case (as scipy, zfit,....), this seems too (unnecessary) cumbersome for a user, where overriding by the end user is a usual case. _*_ sounds better to me and is, AFAIK, not taken. To be defined as "private but stable" to make a distinction to "_*".
The _*_ is also independent of any library name and better for any parser tool (autodocs?)
But yes, it seems worth to standardise
_*_ has the virtue of being completely outside Python's reserved namespace (whereas __*_*__ is only out of the likely namespace) and it connotes the right thing: it resembles the __*__ that users (for some special definition of "user") are accustomed to overriding. _* looks like a protected method, which looks odd, even when SciPy and TensorFlow do it. With IPython's _*_ methods, the intent is clear.
Crazy, it is possible to replace a base class dynamically in Python:
class Bar(object):
def __init__(self):
self.bar_init = 1
def __str__(self):
return "Bar %i" % self.bar_init
class Baz(object):
def __init__(self):
self.baz_init = 2
def __str__(self):
return "Baz %i" % self.baz_init
class Foo(Bar):
def __init__(self, use_baz):
if use_baz:
Foo.__bases__ = (Baz,)
super(Foo, self).__init__()
print(Foo(0))
print(Foo(1))Output:
Bar 1
Baz 2>>> a = Foo(0)
>>> b = Foo(1)
>>> print(a)
Baz 2
>>> print(b)
Baz 2self.__class__ to dynamically change a class's type after it has been created. Classes in Python mean much less than they do in a statically compiled language: they're really just namespaces for methods and "static" member data if Python fails to find a name on the object itself.