Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
    brandonwillard
    @brandonwillard:matrix.org
    [m]
    one of the biggest differences is that FunctionGraph.replace is an in-place update of Apply.inputs that operates on lambdas-like objects and has some lambda-relevant considerations
    it also keeps track of term relationships within the lambda
    and uses those
    the other approaches don't
    they simply walk the graph in topological order and make replacements sequentially
    and the substitutions have little to no "awareness" of each other
    e.g. one substitution can re-introduce a substituted variable
    brandonwillard
    @brandonwillard:matrix.org
    [m]
    that's a result of the substitution order
    Ricardo Vieira
    @ricardov94:matrix.org
    [m]
    Yeah, I am finding that. It would be really nice to document this a bit more in detail. I was really missing a doc page with a non trivial example of how to manipulate graphs and how the different ways differ and gotchas. This was my semi-systematic way of looking at these: https://colab.research.google.com/drive/1jmrkyYiYP_Z0IsERCpN_tdX-GddpNgZU?usp=sharing
    clone_replace also does something weird, where it first replaces the keys of the replacements to dummy variable in a graph and then replaces those dummy variables by the desired "new values". This seems to be a work around to allow for updates that use variables that depend on the variable being replaced
    But this seems to break multiple replacements as fair as I can understand, as in my last example above with sin and cos replacements
    Ricardo Vieira
    @ricardov94:matrix.org
    [m]
    Also, some of these seem still very relevant so it might be useful to discuss them: Theano/Theano#5483
    brandonwillard
    @brandonwillard:matrix.org
    [m]
    most of those aren't actually bugs
    just confusions caused by a lack of documentation and understanding
    with some attemps to help via more warnings/messages
    i.e. not really solutions to the underlying problems
    the closest one to a solution is the request for documentation
    the recursion limit is a real limitation that can be fixed, though
    Ricardo Vieira
    @ricardov94:matrix.org
    [m]
    Documentation seems to be really critical. I personally have been going over this for a day and still couldn't guess what the outcome of most of these methods would be.
    brandonwillard
    @brandonwillard:matrix.org
    [m]
    this is largely due to the need to describe the entire replacement process itself
    Ricardo Vieira
    @ricardov94:matrix.org
    [m]
    Having the memo dictionary returned from clone_replace also feels like it would be useful, because from my experiments successive calls to clone_replace seems to be doing what I would expect it to do in the first place.
    brandonwillard
    @brandonwillard:matrix.org
    [m]
    i.e. the requisite documentation is essentially meta code
    Ricardo Vieira
    @ricardov94:matrix.org
    [m]
    More than the alternatives
    brandonwillard
    @brandonwillard:matrix.org
    [m]
    as I recall, that's simply due to poor library design
    e.g. these clone functions are all small alterations of each other
    so they have a lot of overlap and differing features that are uncomplementary
    it might not be a good idea for us to build around/on top of them
    brandonwillard
    @brandonwillard:matrix.org
    [m]
    we should start from the beginning instead
    Ricardo Vieira
    @ricardov94:matrix.org
    [m]
    I have to say that does sound appealing to me at the moment xD
    Also those early commits/PRs on Theano are so void of context that it seems difficult to understand the reasoning behind some of the features / changes
    brandonwillard
    @brandonwillard:matrix.org
    [m]
    I think a lot of the code was written in isolation and later cobbled together
    and there wasn't much interest/effort toward cohesion
    you can see this all over the scan codebase
    one of those clone functions comes from that
    and started to be used all throughout the library
    I believe it created one of the bad dependency chains we had to fix early on
    that issue is very related to all of this
    brandonwillard
    @brandonwillard:matrix.org
    [m]
    especially the excessive need to clone entire graphs and manually remap the cloned values
    we could/should separate the non-in-place replacement logic from FunctionGraph
    when that's implemented
    that would be an answer to a lot of clone-like use cases
    Ricardo Vieira
    @ricardov94:matrix.org
    [m]
    Trying to see what you mean by non-inplace replacement.
    a = at.scalar('a')
    b = a + 1; b.name = 'b'
    c = b + 1; c.name = 'c'
    fg = FunctionGraph([a], [c], clone=False)
    fg.replace(b, at.sin(a))
    Would anything change in that toy example under the hood?
    brandonwillard
    @brandonwillard:matrix.org
    [m]
    yeah, you've gotta test the bad and the good
    if only to set expectations
    Ricardo Vieira
    @ricardov94:matrix.org
    [m]
    Totally unrelated this is interesting. They are suggesting there is no need for R_Ops at all: Theano/Theano#6400
    brandonwillard
    @brandonwillard:matrix.org
    [m]
    yeah, I remember that one
    Ricardo Vieira
    @ricardov94:matrix.org
    [m]
    This is the blogpost where they presented it: https://j-towns.github.io/2017/06/12/A-new-trick.html