Nice, hadn’t read this from the discussion:
I agree with Henry's point of view, default capping is a bad idea. To be specific, the worst thing about defaulting to a cap is that there is no way to distinguish between affirmative knowledge that a package fails above a certain version (I'm thinking pydantic and python 3.10) and a cap based on untested presumptions.
It also forces the library to update rapidly, as the they have to do a new release as soon as a new major version of any dependency comes out. Often, it’s beyond the resources of most open-source projects. It’s really a disaster.
This is a two-sided sword however: In general I agree to let it free if fine (and definitely restrict if needed, I had multiple releases of old versions that just broke, I very well agree with @HDembinski point here and am restricting always a couple of dependencies, recently even Jinja broke the doc builds...). But if letting it freely float is the norm, libraries also may tend less to do breaking changes, and that would sometimes be a good thing while also sometimes problematic (Python 2 vs 3 is definitely a good thing but yeah, problematic transition...). Expecting in general version compatibility for all versions inherently limits the ability to get reasonably rid of legacy code or behavior.
Just to say: I really like the uproot approach here! That is a pretty explicit pinnig of version but allowing the co-existance of multiple versions to avoid any dependency conflict, like having the good of both worlds. If this can be formalised better and we see a import iminuit1, iminuit2, that seems quite favorable to me.
25 replies
3 replies
starts[i] == stops[i] is defined to mean that list i has zero length. Okay, but what if either starts[i] or stops[i] is negative or greater than or equal to the length of the content? In any other case, that would be an error. Should it be an error when starts and stops are equal? I decided that it should not be, and therefore I have to include tests where they're equal but greater than any value in the content. If I hadn't thought of it, no line coverage or even extreme value tester like hypothesis would have caught it, because it involves a dependency between two array elements that the automated tools don't know might be related. It requires human attention and can't be fully guaranteed by any automated system. THAT'S the sort of thing that can slip though version changes despite the vigilance of the testers.
12 replies
monolens :star: s or traffic in the coming month as monolens was mentioned on the PythonBytes podcast. :)
3 replies
@matthewfeickert in what has become a bit of a trend for me at the moment, I'm re-evaluating all of this again myself.
I prefer writing relative imports but I enjoy reading absolute imports (where only the modules are imported rather than their contents)
build. Some require relative imports (often due to vendorability, I think, though also redability). Importing objects from a package instead of the package itself can cause circular dependency issues, so I’m leaning more toward importing modules if I write something new. Either as relative imports or absolute.
from . import module or import package.module. For things in the stdlib, I’m fine to pull objects in, as you can’t get into a circular sitation there, and the names are commonly known. It makes typing so much more readable, so I’m a bit torn on importing local classes.
pyhf has some circular import warnings that haven't seemed to bite us yet, but similar to @agoose77:matrix.org I'm thinking of just switching everything over to absolute imports now that the repository structure isn't undergoing much change anymore
@chernals Before you get too excited about Aghast, it's a nearly deprecated project (just haven't mustered the courage to pull the trigger yet). It hasn't been touched in a long time. The intention was to unify histograms into a common serialization, and histograms still do need better interoperability on a serialized level (we want to be able to save Boost histograms without losing their special features by converting them into ROOT v6 histograms, etc.), but the Aghast model won't allow for irregular data, such as you might get in sparse or categorical histograms. Nick made a good point here: scikit-hep/aghast#10 which could be solved by replacing FlatBuffers as the serialization with Awkward Array or Arrow. Since that issue was in 2019, Awkward Array needed more development to make that happen (Arrow, too, that long ago). Now it could be done.
I'm only warning you against getting too attached to Aghast—Boost histogram and hist are very active, mature, up-to-date projects.
That's a bit sad indeed but now I'm prepared.
The features I'm mostly interested in are :
- adding histograms like aghast was providing
- reading root histograms from a root file and then manipulating them as pyboost histograms or "hist" histograms in Python
@chernals Aghast was intended as a superset of all histogram features, but that's nearly equal to boost-histogram's features, so your first bullet point should be satisfied by that.
As for zero-copy, the bin contents are either a zero-copy view or a direct memcpy-style copy, but this doesnt
