Yeah it is. And why is the first case different? Is the introduced x treated differently than the original one?
because
x is an input to the FunctionGraph, and the checks surrounding that are fairly strict
the first case is probably not treating the
x as an input that's distinct from an output
It's also an input in the other cases. Is the difference that it is also an output?
so the checks are different
I see. I should try the example with 2 intermediate variables then
The other thing is that I struggle with coming up with non recursive replacements. Unless all inputs are new won't the replacement variable always contain the variable it is trying to replace as one of it's inputs (maybe with some extra nodes in between)?
Or in other words what is a valid replacement for tbe original graph which is just
exp(x)
valid substitions (in the untyped lambda calculus) usually have a formal rename rule as part of their definition
e.g.
(λx.P)[x := N] == λz.(P'[x := N]) where P' has y renamed to z
otherwise a naive substitution like
λy.yx[x := xy] would have the free y in x := xy bound/captured by the lambda
I will probably have to read some examples somewhere to understand the concept and "algorithm"
well, that's just a good reference for how it should work
how it does work is a whole other story
def create_fgraph():
a = at.scalar('a')
b = at.exp(a); b.name = 'b'
c = at.log(b); c.name = 'c'
d = c + 5; d.name = 'd'
e = at.log(d); e.name = 'e'
f = e - 3; f.name = 'f'
nodes = dict(a=a, b=b, c=c, d=d, e=e, f=f)
fg = FunctionGraph(inputs=[a], outputs=[f], clone=False)
return nodes, fg
# Can always replace one variable by an earlier one
nodes, fg = create_fgraph()
fg.replace_validate(nodes['c'], nodes['b'])
nodes, fg = create_fgraph()
fg.replace_validate(nodes['e'], nodes['c'])
# Can never replace one variable by a later one
nodes, fg = create_fgraph()
# This would enter an inifinite loop!
# fg.replace_validate(nodes['b'], nodes['c'])
# Unless it is the output variable, but then it gives
# an invalid cyclical graph
nodes, fg = create_fgraph()
fg.replace_validate(nodes['e'], nodes['f'])
aesara.dprint(fg)
# Elemwise{sub,no_inplace} [id A] 'f' 0
# |Elemwise{sub,no_inplace} [id A] 'f' 0
# |TensorConstant{3} [id B]:point_up: Edit: ```python
def create_fgraph():
a = at.scalar('a')
b = at.exp(a); b.name = 'b'
c = at.log(b); c.name = 'c'
d = c + 5; d.name = 'd'
e = at.log(d); e.name = 'e'
f = e - 3; f.name = 'f'
nodes = dict(a=a, b=b, c=c, d=d, e=e, f=f)
fg = FunctionGraph(inputs=[a], outputs=[f], clone=False)
return nodes, fg
Can always replace one variable by an earlier one
nodes, fg = create_fgraph()
fg.replace(nodes['c'], nodes['b'])
nodes, fg = create_fgraph()
fg.replace(nodes['e'], nodes['c'])
Can never replace one variable by a later one
nodes, fg = create_fgraph()
fg.replace(nodes['b'], nodes['c'])
Unless it is the output variable, but then it gives
an invalid cyclical graph
nodes, fg = create_fgraph()
fg.replace_validate(nodes['e'], nodes['f'])
aesara.dprint(fg)
Elemwise{sub,no_inplace} [id A] 'f' 0
|Elemwise{sub,no_inplace} [id A] 'f' 0
|TensorConstant{3} [id B]
```
:point_up: Edit: ```python
def create_fgraph():
a = at.scalar('a')
b = at.exp(a); b.name = 'b'
c = at.log(b); c.name = 'c'
d = c + 5; d.name = 'd'
e = at.log(d); e.name = 'e'
f = e - 3; f.name = 'f'
nodes = dict(a=a, b=b, c=c, d=d, e=e, f=f)
fg = FunctionGraph(inputs=[a], outputs=[f], clone=False)
return nodes, fg
Can always replace one variable by an earlier one
nodes, fg = create_fgraph()
fg.replace(nodes['c'], nodes['b'])
nodes, fg = create_fgraph()
fg.replace(nodes['e'], nodes['c'])
Can never replace one variable by a later one
Produces an invalid cyclical graph
nodes, fg = create_fgraph()
fg.replace(nodes['b'], nodes['c'])
Unless it is the output variable, but then it gives
an invalid cyclical graph
nodes, fg = create_fgraph()
fg.replace(nodes['e'], nodes['f'])
aesara.dprint(fg.outputs)
Elemwise{sub,no_inplace} [id A] 'f' 0
|Elemwise{sub,no_inplace} [id A] 'f' 0
|TensorConstant{3} [id B]
```
Last code dump I promise
#%%
a = at.scalar('a')
b = at.exp(a); b.name = 'b'
c = at.log(b); c.name = 'c'
d = c + 5; d.name = 'd'
fg = FunctionGraph(inputs=[a], outputs=[c], clone=False)
fg.replace(b, d)
aesara.dprint(fg.outputs)
# Elemwise{log,no_inplace} [id A] 'c'
# |Elemwise{add,no_inplace} [id B] 'd'
# |Elemwise{log,no_inplace} [id A] 'c' <-- CYCLICAL
# |TensorConstant{5} [id C]
#%%
a = at.scalar('a')
b = at.exp(a); b.name = 'b'
c = at.log(b); c.name = 'c'
d = c + 5; d.name = 'd'
fg = FunctionGraph(inputs=[a], outputs=[b], clone=False)
fg.replace(b, d)
aesara.dprint(fg.outputs)
# Elemwise{add,no_inplace} [id A] 'd'
# |Elemwise{log,no_inplace} [id B] 'c'
# | |Elemwise{exp,no_inplace} [id C] 'b'
# | |a [id D]
# |TensorConstant{5} [id E]Does it make sense that the replacement of b -> d works when b is the output of the FunctionGraph (second half) but not when it is not (first half)?
Is this just a corner case I am hitting, and they should both fail/ or succeed?
:point_up: Edit: Last code dump I promise
#%%
a = at.scalar('a')
b = at.exp(a); b.name = 'b'
c = at.log(b); c.name = 'c'
d = c + 5; d.name = 'd'
fg = FunctionGraph(inputs=[a], outputs=[c], clone=False)
fg.replace(b, d)
aesara.dprint(fg.outputs)
# Elemwise{log,no_inplace} [id A] 'c'
# |Elemwise{add,no_inplace} [id B] 'd'
# |Elemwise{log,no_inplace} [id A] 'c' <-- CYCLICAL
# |TensorConstant{5} [id C]
#%%
a = at.scalar('a')
b = at.exp(a); b.name = 'b'
c = at.log(b); c.name = 'c'
d = c + 5; d.name = 'd'
fg = FunctionGraph(inputs=[a], outputs=[b], clone=False)
fg.replace(b, d)
aesara.dprint(fg.outputs)
# Elemwise{add,no_inplace} [id A] 'd'
# |Elemwise{log,no_inplace} [id B] 'c'
# | |Elemwise{exp,no_inplace} [id C] 'b'
# | |a [id D]
# |TensorConstant{5} [id E]Does it make sense that the replacement of b -> d "works" when b is the output of the FunctionGraph (second half) but not when it is not (first half)?
By works I mean that it produces an acyclical graph
:point_up: Edit: Is this just a corner case I am hitting, and should both fail/ or succeed?
when you replaced the output
b in the latter case, it basically cleared the entire FunctionGraph and replaced it with another graph/output that also happened to reference b
so no issue there
remember, these steps need to happen in a fixed sequence of events
and the actual replacement steps are different depending on whether or not the replaced term is an output
in that case, the thing being changed is
FunctionGraph.outputs
when a replacement is made on something that is not just a
FunctionGraph output, like in the former case, an Apply node is updated in-place
specifically, I believe it's
Apply.inputs that's updated in-place
anyway, there's no real cycle detection at this level, so it's completely up to the caller to not introduce them
Okay, now things are clicking a bit more for me. The (aesara) problem is not with the recursive expression, but somehow its identity?
a = at.scalar('a')
b = at.exp(a); b.name = 'b'
c = at.log(b); c.name = 'c'
d = c + 5; d.name = 'd'
fg = FunctionGraph(inputs=[a], outputs=[d], clone=False)
# Cannot do this
# fg.replace(b, c)
# But can do this
new_c = c.owner.op(*c.owner.inputs); new_c.name = 'new_c'
fg.replace(b, new_c)
aesara.dprint(fg.outputs)
Is there a reason why
fg.replace(b, c) should behave differently than fg.replace(b, new_c)?
:point_up: Edit: Things are still not clicking entirely. The (aesara) problem I was facing before is not with the recursive expression, but somehow its identity?
a = at.scalar('a')
b = at.exp(a); b.name = 'b'
c = at.log(b); c.name = 'c'
d = c + 5; d.name = 'd'
fg = FunctionGraph(inputs=[a], outputs=[d], clone=False)
# Cannot do this
# fg.replace(b, c)
# But can do this
new_c = c.owner.op(*c.owner.inputs); new_c.name = 'new_c'
fg.replace(b, new_c)
aesara.dprint(fg.outputs)
:point_up: Edit: Things are still not clicking entirely. The (aesara) problem I was facing before is not with the recursive expression, but somehow its identity?
a = at.scalar('a')
b = at.exp(a); b.name = 'b'
c = at.log(b); c.name = 'c'
d = c + 5; d.name = 'd'
fg = FunctionGraph(inputs=[a], outputs=[d], clone=False)
# Cannot do this
# fg.replace(b, c)
# But can do this
new_c = c.owner.op(*c.owner.inputs); new_c.name = 'new_c'
fg.replace(b, new_c)
aesara.dprint(fg.outputs)
# Elemwise{add,no_inplace} [id A] 'd'
# |Elemwise{log,no_inplace} [id B] 'c'
# | |Elemwise{log,no_inplace} [id C] 'new_c'
# | |Elemwise{exp,no_inplace} [id D] 'b'
# | |a [id E]
# |TensorConstant{5} [id F]
:point_up: Edit: Things are still not clicking entirely. The (aesara) problem I was facing before is not with the recursive expression, but somehow its identity?
a = at.scalar('a')
b = at.exp(a); b.name = 'b'
c = at.log(b); c.name = 'c'
d = c + 5; d.name = 'd'
fg = FunctionGraph(inputs=[a], outputs=[d], clone=False)
# Cannot do this
# fg.replace(b, c)
# But can do this
# new_c = at.log(b); new_c.name = 'new_c'
new_c = c.owner.op(*c.owner.inputs); new_c.name = 'new_c'
fg.replace(b, new_c)
aesara.dprint(fg.outputs)
# Elemwise{add,no_inplace} [id A] 'd'
# |Elemwise{log,no_inplace} [id B] 'c'
# | |Elemwise{log,no_inplace} [id C] 'new_c'
# | |Elemwise{exp,no_inplace} [id D] 'b'
# | |a [id E]
# |TensorConstant{5} [id F]
1 reply
Yeah, I know that, I am trying to understand what are the commonalities. Between fgraph.replace, function.givens, and clone_replace, each one seems to follow it's own unique logic
clone_replace seems completely broken for more than one replacement
a = at.scalar('a')
b = a + 1; b.name = 'b'
c = b + 1; c.name = 'c'
d = c + 2; d.name = 'd'
e = d + 2; e.name = 'e'
f = e + 1; f.name = 'f'
out = aesara.clone_replace(f, replace={b: at.cos(a), d: at.sin(c)})
aesara.dprint(out)
# Elemwise{add,no_inplace} [id A] 'f'
# |Elemwise{add,no_inplace} [id B] 'e'
# | |Elemwise{sin,no_inplace} [id C] ''
# | | |Elemwise{add,no_inplace} [id D] 'c'
# | | |Elemwise{add,no_inplace} [id E] 'b'
# | | | |a [id F]
# | | | |TensorConstant{1} [id G]
# | | |TensorConstant{1} [id H]
# | |TensorConstant{2} [id I]
# |TensorConstant{1} [id J]
I understand
d: at.sin(c) erases the first update, but then clone_replace with multiple updates only works in very specific cases where the replacements are not nested at all
one of the biggest differences is that
FunctionGraph.replace is an in-place update of Apply.inputs that operates on lambdas-like objects and has some lambda-relevant considerations
it also keeps track of term relationships within the lambda
and uses those
the other approaches don't