Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
    brandonwillard
    @brandonwillard:matrix.org
    [m]
    yeah, we're still doing so much refactoring that it doesn't make sense to use semantic versioning as intended
    and, when we need to, we can always start maintaining distinct version branches
    larryshamalama
    @larryshamalama
    Does anyone use a Mac by any chance? I’m still having trouble running pytest on my laptop… I wrote a comment in aesara-devs/aesara#934 about my attempts but I have yet to figure out something that works well on my local machine.
    1 reply
    remilouf
    @remilouf:matrix.org
    [m]
    version 2.6.5 made a bunch of flaky tests fail in aemcmc, aehmc. Time for a good clean.
    1 reply
    remilouf
    @remilouf:matrix.org
    [m]
    library ones most likely
    they relied on a specific seed value / implementation of the rng
    brandonwillard
    @brandonwillard:matrix.org
    [m]
    the relevant Aesara changes were probably the RandomStream seeding ones then
    brandonwillard
    @brandonwillard:matrix.org
    [m]
    @room meeting in progress
    Ricardo Vieira
    @ricardov94:matrix.org
    [m]
    @channel meeting in progress
    brandonwillard
    @brandonwillard:matrix.org
    [m]
    Ricardo Vieira, after looking over those MergeFeature/MergeOptimizer updates, I'm brought back to a complete reassessment of the merging process in Aesara
    (moved the comment over here)
    I can't shake the feeling that the entire setup/framework for merging is just wrong
    for instance, why aren't we performing merges before constructing a graph/updating a FunctionGraph?
    we're basically allowing ourselves to create many little graphs that will need to be merged later on
    I've assumed that the reason(s) might have to do with some "efficiency" claim regarding follow-up rewrites
    e.g. don't perform merges until a bunch of other rewrites have been performed, just in case those other rewrites simplify the "merge landscape"
    brandonwillard
    @brandonwillard:matrix.org
    [m]
    (i.e. remove subgraphs that would've been merged had merging been performed more eagerly)
    claims like that absolutely require solid studies
    (and/or, when possible, some clear, guiding mathematical justifications)
    one could also consider bringing the merging further upstream
    so that duplicate graphs are never even produced
    brandonwillard
    @brandonwillard:matrix.org
    [m]
    e.g. Apply.__[init|new]__
    anyway, someone has surely done some (published) work on this, so it would be good to have an informed position
    brandonwillard
    @brandonwillard:matrix.org
    [m]
    :point_up: Edit: (i.e. rewrites could remove subgraphs that would've been merged had merging been performed more eagerly, resulting in wasted merges)
    :point_up: Edit: e.g. caching in Apply.__[init|new]__
    :point_up: Edit: e.g. Op and inputs-based caching in Apply.__[init|new]__
    brandonwillard
    @brandonwillard:matrix.org
    [m]
    :point_up: Edit: e.g. Op and inputs-based weakref.WeakKeyDictionary caching in Apply.__[init|new]__
    :point_up: Edit: e.g. Op and inputs-based weakref.WeakKeyDictionary caching in [Variable|Apply].__[init|new]__
    now that I think about it, the Variable/Apply-based caching might be the easiest and most "amortized" approach
    brandonwillard
    @brandonwillard:matrix.org
    [m]
    while it has an obvious hashing/dict lookup + storage cost, those are generally very cheap for the relevant object types (i.e. Variables and Ops)
    we would be adding that cost to every Op.make_node call
    and, worst case scenario, our baseline is the current cost of Op.make_node construction plus the caching cost
    but, best case scenario, we only pay the caching cost
    since the Op.make_node can easily be much larger than the cache lookup cost, there's the solid potential for a cost reduction with this approach
    brandonwillard
    @brandonwillard:matrix.org
    [m]
    there's a "long-term" memory/storage cost for the cache, but also a lot of "short-term" memory costs for Op.make_node calls that would be removed
    plus, I'm not sure that the "long-term" memory cost for such a cache would even be relevant
    every cached term would be session-dependent/relevant
    and, if a term is being used in a given session (i.e. isn't in a user-owned graph object or something), then the object will be garbage collected and the weak reference invalidated
    in this case, if the same inputs and Op are combined again later on within the same session, a new unique representative term is constructed and cached
    brandonwillard
    @brandonwillard:matrix.org
    [m]
    yeah, I'm really not sure why we shouldn't be doing this
    brandonwillard
    @brandonwillard:matrix.org
    [m]
    :point_up: Edit: and, if an Apply node is no longer being used in a given session (i.e. isn't in a user-owned graph object or something), then the object will be garbage collected and the weak reference invalidated
    :point_up: Edit: and, if a Variable/Apply node is no longer being used in a given session (i.e. isn't in a user-owned graph object or something), then the object will be garbage collected and the weak reference invalidated
    the rewriting logic in FunctionGraph would need to be change so that Apply nodes are no longer updated in-place, which we've already discussed and prototyped
    plus, we would have a much clearer foundation for Equality Saturation-like "rewrite-space" considerations
    because all of the rewritten graphs would exist within this cache
    one need only keep separate collections connecting rewritten Apply nodes to have a usable record of all equivalent graphs under rewrites
    Ricardo Vieira
    @ricardov94:matrix.org
    [m]
    Will need to reread this tomorrow, thanks for writing it up
    Ricardo Vieira
    @ricardov94:matrix.org
    [m]
    Are you suggesting that Op.make_node calls would be cached and the same output returned?
    brandonwillard
    @brandonwillard:matrix.org
    [m]
    something like that
    not exactly that, though