Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
    brandonwillard
    @brandonwillard:matrix.org
    [m]
    yeah, we could have a generalized Elemwise
    but the implementation/computation details are important
    remilouf
    @remilouf:matrix.org
    [m]
    This would be very useful for instance to advance several chains in aehmc
    brandonwillard
    @brandonwillard:matrix.org
    [m]
    I think there are directly analogous Numba vectorizations
    so that backend might be covered easily
    and, obviously, we can map to jax.vmap for the JAX backend
    and numpy.vectorize/for-loops in the Python case
    I just don't want to deal with an implementation in the current C backend
    although that would only involve some straightforward for-loops as well
    per usual, I've been avoiding these kinds of things until we replace/remove the C backend
    brandonwillard
    @brandonwillard:matrix.org
    [m]
    anyway, as a basic graph abstraction it's a good thing
    it's a generalization that has been motivated a few times by our RandomVariable work
    I think we're actually talking about gufuncs: https://numpy.org/doc/stable/reference/c-api/generalized-ufuncs.html#
    and since Elemwise is equivalent to ufunc, we need an equivalent Op for gufunc
    brandonwillard
    @brandonwillard:matrix.org
    [m]
    yeah, we need an issue for that
    brandonwillard
    @brandonwillard:matrix.org
    [m]
    perhaps we could call it Blockwise
    and at.vectorize would construct those
    actually, this probably would be worth doing sooner than later
    especially since there are some optimizations we could write
    brandonwillard
    @brandonwillard:matrix.org
    [m]
    we could probably replace the size parameter with this and generalize a lot of the RandomVariable logic
    (although the actual NumPy/SciPy size parameter would be used when computed)
    we might also want to create a general protocol for specifying signature-like information per Op
    because that's valuable information across the board
    and, when/if that information isn't available at the Op-level, it can always be provided to a Blockwise
    e.g. via the at.vectorize helper/constructor function
    Ricardo Vieira
    @ricardov94:matrix.org
    [m]
    Is the C backend somehow in the way of any of this? Or it can happen independently?
    brandonwillard
    @brandonwillard:matrix.org
    [m]
    it can happen independently
    i.e. we can provide only a Python implementation
    Ricardo Vieira
    @ricardov94:matrix.org
    [m]
    Maybe this could also help with the elemwise gradients that involve scans
    brandonwillard
    @brandonwillard:matrix.org
    [m]
    I don't know the issue very well, but it could definitely simplify/generalize certain types of gradients
    in a sense, this proposed Blockwise and Scan are very related
    brandonwillard
    @brandonwillard:matrix.org
    [m]
    yeah, Ricardo Vieira, this was a great thing to address
    do you want to create an issue for a Blockwise Op?
    1 reply
    Kaustubh
    @kc611:matrix.org
    [m]
    import aesara.tensor as aet
    import copy
    
    x = aet.dscalar("x")
    z = copy.deepcopy(x)
    
    assert x==z # Fails
    Is this behavior of Aesara variables intended/serves some purpose or is it simply a bug ?
    brandonwillard
    @brandonwillard:matrix.org
    [m]
    it's expected, but undesirable
    for == to work like it generally should, we would need to implement consistent __eq__s
    the end result would be that == effectively does what aesara.graph.basic.equal_computations does
    this would make g_1 == g_2 a little expensive when they're both large graphs that are very similar except in the "leaf" nodes/inputs
    it would also require consistent __hash__ implementations
    brandonwillard
    @brandonwillard:matrix.org
    [m]
    and computing hashes for large graphs would take some effort
    and, since we put graphs in sets and dicts all the time, it would surely have an effect
    now, if we made sure that graph objects were immutable, these hashes could be cached
    and that problem would be largely mitigated
    ultimately, I would like to take things down this path
    (or a better one)
    you can find this idea--and all the surrounding ones--in some Aesara issues
    and a full implementation in the symbolic-pymc meta objects
    Kaustubh
    @kc611:matrix.org
    [m]

    This seems like a whole other problem in itself.

    As of now, I was looking for ways to have a copy of a nested structure (for instance a dict with list of TensorVariables), I wanted to track changes within such structures, by comparing old to new modified structures. But ran into this issue when I deepcopied the nested structure.

    brandonwillard
    @brandonwillard:matrix.org
    [m]
    it doesn't work for the reason described above; in other words, equality is by identity
    that's why we need to clone graphs and track/use maps between old and cloned variables, as well