yeah, we're still doing so much refactoring that it doesn't make sense to use semantic versioning as intended
and, when we need to, we can always start maintaining distinct version branches
Does anyone use a Mac by any chance? I’m still having trouble running pytest on my laptop… I wrote a comment in aesara-devs/aesara#934 about my attempts but I have yet to figure out something that works well on my local machine.
1 reply
version 2.6.5 made a bunch of flaky tests fail in
aemcmc, aehmc. Time for a good clean.
1 reply
they relied on a specific seed value / implementation of the rng
the relevant Aesara changes were probably the
RandomStream seeding ones then
Ricardo Vieira, after looking over those
MergeFeature/MergeOptimizer updates, I'm brought back to a complete reassessment of the merging process in Aesara
(moved the comment over here)
I can't shake the feeling that the entire setup/framework for merging is just wrong
for instance, why aren't we performing merges before constructing a graph/updating a
FunctionGraph?
we're basically allowing ourselves to create many little graphs that will need to be merged later on
I've assumed that the reason(s) might have to do with some "efficiency" claim regarding follow-up rewrites
e.g. don't perform merges until a bunch of other rewrites have been performed, just in case those other rewrites simplify the "merge landscape"
(i.e. remove subgraphs that would've been merged had merging been performed more eagerly)
claims like that absolutely require solid studies
(and/or, when possible, some clear, guiding mathematical justifications)
one could also consider bringing the merging further upstream
so that duplicate graphs are never even produced
anyway, someone has surely done some (published) work on this, so it would be good to have an informed position
:point_up: Edit: e.g. caching in
Apply.__[init|new]__
:point_up: Edit: e.g.
Op and inputs-based weakref.WeakKeyDictionary caching in [Variable|Apply].__[init|new]__
now that I think about it, the
Variable/Apply-based caching might be the easiest and most "amortized" approach
while it has an obvious hashing/
dict lookup + storage cost, those are generally very cheap for the relevant object types (i.e. Variables and Ops)
we would be adding that cost to every
Op.make_node call
and, worst case scenario, our baseline is the current cost of
Op.make_node construction plus the caching cost
but, best case scenario, we only pay the caching cost
since the
Op.make_node can easily be much larger than the cache lookup cost, there's the solid potential for a cost reduction with this approach
there's a "long-term" memory/storage cost for the cache, but also a lot of "short-term" memory costs for
Op.make_node calls that would be removed
plus, I'm not sure that the "long-term" memory cost for such a cache would even be relevant
every cached term would be session-dependent/relevant
and, if a term is being used in a given session (i.e. isn't in a user-owned graph object or something), then the object will be garbage collected and the weak reference invalidated
in this case, if the same inputs and
Op are combined again later on within the same session, a new unique representative term is constructed and cached
yeah, I'm really not sure why we shouldn't be doing this
:point_up: Edit: and, if a
Variable/Apply node is no longer being used in a given session (i.e. isn't in a user-owned graph object or something), then the object will be garbage collected and the weak reference invalidated
the rewriting logic in
FunctionGraph would need to be change so that Apply nodes are no longer updated in-place, which we've already discussed and prototyped
plus, we would have a much clearer foundation for Equality Saturation-like "rewrite-space" considerations
because all of the rewritten graphs would exist within this cache
one need only keep separate collections connecting rewritten
Apply nodes to have a usable record of all equivalent graphs under rewrites
Will need to reread this tomorrow, thanks for writing it up
Are you suggesting that
Op.make_node calls would be cached and the same output returned?
not exactly that, though