Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
    Sam Hodge
    @samhodge
    I might be able to look into it on my lunch break here at work.
    Sam Hodge
    @samhodge
    I have updated the issue apache/incubator-mxnet#9813
    Just training now, I will let it cook until the morning CPU only
    Sam Hodge
    @samhodge
    @anirudh2290 Thanks for all of your help, if I get a decent trained model out of this I will put in a pull request.
    Anirudh Subramanian
    @anirudh2290
    no worries! will be happy to look at your PR!
    Sam Hodge
    @samhodge
    OK nearly there, it seems to be training a model OK, but when I go to save the model the issue is in the Inspiration(HybridBlock) the current code which was (poorly) converted from Block to HybridBlock, is not using a Symbol in the args for return value from HybridForward, it is a NDArray, the errors tell me this, but now I am stuck with the issue that the infer_shapes code keeps givng me back and error
    terminate called after throwing an instance of 'std::logic_error'
    what(): basic_string::_S_construct null not valid
    so basically the question comes back to
    Sam Hodge
    @samhodge
    class Inspiration(HybridBlock):
        """ Inspiration Layer (from MSG-Net paper)
        tuning the featuremap with target Gram Matrix
        ref https://arxiv.org/abs/1703.06953
        """
        def __init__(self, C, B=1):
            super(Inspiration, self).__init__()
            # B is equal to 1 or input mini_batch
            self.C = C
            self.weight = self.params.get('weight', shape=(1,C,C),
                                          init=mx.initializer.Uniform(),
                                          allow_deferred_init=True)
            self.gram = self.params.get('gram', shape=(B,C,C),
                                        init=mx.initializer.Uniform(),
                                        allow_deferred_init=True,
                                        lr_mult=0)
    
        def setTarget(self, target):
            self.gram.set_data(target)
    
        def forward(self, X):
            # input X is a 3D feature map
            self.P = F.batch_dot(F.broadcast_to(self.weight.data(), shape=(self.gram.shape)), self.gram.data())
            return F.batch_dot(F.SwapAxis(self.P,1,2).broadcast_to((X.shape[0], self.C, self.C)), X.reshape((0,0,X.shape[2]*X.shape[3]))).reshape(X.shape)
    
        def __repr__(self):
            return self.__class__.__name__ + '(' \
                + 'N x ' + str(self.C) + ')'
    How do you implement the hybrid forward function for this block, given X.shape is not a method of a Symbol and X.infer_shape(Y), needs an argument of Y as a shape and returns args_shapes,output_shapes,input_shapes
    Anyone want to throw me a hand?
    I have been able to train on GPU and CPU, but I am not able to save the symbolic moel
    I will be able to commit my changes to repro the error in about 7 hours from now.
    Sam Hodge
    @samhodge
    OK apache/incubator-mxnet#9813 has been updated @anirudh2290 if you could help me out that would be great
    Sam Hodge
    @samhodge
    Does anybody have experience with how to get the shape of a Symbol using infer_shape?
    Anirudh Subramanian
    @anirudh2290
    hi @samhodge sorry i have been busy and havent gotten a chance to look at your changes. Have you looked at infer_shape documentation ?
    Anirudh Subramanian
    @anirudh2290
    i think you should be able to use infer_shape without any arguments
    the shape should be inferred when data is bound to the symbolic inputs to the model
    Sam Hodge
    @samhodge
    I will try again, I looked
    Anirudh Subramanian
    @anirudh2290
    its not very clear from the documentation. maybe we should add an example to the documentation.
    Sam Hodge
    @samhodge
    /home/samh/dev/MXNet-Gluon-Style-Transfer/net.py:332: UserWarning: Cannot decide shape for the following arguments (0s in shape means unknown dimensions). Consider providing them as input:
    data: ()
    in_shapes,out_shapes,arg_shapes= X.infer_shape()
    that is not a good start
    With this
    `class Inspiration(HybridBlock):
    """ Inspiration Layer (from MSG-Net paper)
    tuning the featuremap with target Gram Matrix
    ref https://arxiv.org/abs/1703.06953
    """
    def init(self, C, B=1, ctx=mx.cpu(0)):
    super(Inspiration, self).init()
        # B is equal to 1 or input mini_batch
        self.C = C
        self.B = B
    
        self.weight = self.collect_params().get('weight', shape=(1,self.C,self.C),
                                      init=mx.initializer.Uniform(),
                                      allow_deferred_init=True)
        self.gram = self.collect_params().get('gram', shape=(self.B,self.C,self.C),
                                    init=mx.initializer.Uniform(),
                                    allow_deferred_init=True,
                                    lr_mult=0)
        self.weight.initialize(ctx=ctx)
        self.gram.initialize(ctx=ctx)
    def setTarget(self, target):
        self.gram.set_data(target)
    def hybrid_forward(self, F, X, gram, weight):
    
        P = F.batch_dot(F.broadcast_to(weight, shape=(self.gram.shape)), gram) 
        if not isinstance(X,symbol.Symbol):   
            return F.batch_dot(F.SwapAxis(P,1,2).broadcast_to((X.shape[0], self.C, self.C)), X.reshape((0,0,X.shape[2]*X.shape[3]))).reshape(X.shape)
        else:
            #print "Hooppla", interals
            #for i in dir(interals):
            #    print "kk:", i
            in_shapes,out_shapes,arg_shapes= X.infer_shape(self.gram.shape)
            #print out_shapes
            #raise Exception
            #arg_shapes, out_shapes, aux_shapes = interals.infer_shape(self.gram.shape)
            #print "A", arg_shapes, "O", out_shapes, "AU", aux_shapes
            return F.batch_dot(F.SwapAxis(P,1,2).broadcast_to((in_shapes[0], self.C, self.C)), X.reshape((0,0,in_shapes[2]*in_shapes[3]))).reshape(in_shapes)
    
    def __repr__(self):
        return self.__class__.__name__ + '(' \
            + 'N x ' + str(self.C) + ')'`
    i get the following error
    Sam Hodge
    @samhodge
    terminate called after throwing an instance of 'std::logic_error'
    what(): basic_string::_S_construct null not valid
    Abort
    Sam Hodge
    @samhodge
    OK I will update the repository and the ticket
    Anirudh Subramanian
    @anirudh2290
    @samhodge i dont have much clue at this point. i will try to reproduce in the evening. when you use infer_shape without argument it just fails ?
    Sam Hodge
    @samhodge
    I have updated the ticket in my lunch break
    Anirudh Subramanian
    @anirudh2290
    very positive news @samhodge . dont you have a validation dataset to test your model with ?
    thanks for taking time out to work on this !
    Sam Hodge
    @samhodge
    No thank you for your help. I can test that the C++ evaluates to the same result as the Python model with a pixel diff.
    Anirudh Subramanian
    @anirudh2290
    btw i realized that you are using arg_shapes instead of out_shape. dont you want to infer_shape of X ?
    Sam Hodge
    @samhodge
    I think I do, I will change it to out_shap
    out_shape is an empty tuple in a list
    how can I use that?
    Sam Hodge
    @samhodge
    @anirudh2290
    Anirudh Subramanian
    @anirudh2290
    whatsup ?
    Sam Hodge
    @samhodge
    Do you think it is possible to port the network to resolution independent
    In C++
    Anirudh Subramanian
    @anirudh2290
    i am not sure i understand. when you call the hybridize you basically get all the advantages of running a symbolic model in C++ backend
    you are able to hybridize your model correct ?
    Sam Hodge
    @samhodge
    Well yes you can
    but the model needs to know the resolution of the input data
    before you can serialise it
    let me show you what I mean
    this commit kind of spells it out: samhodge/incubator-mxnet@1d72d60
    Anirudh Subramanian
    @anirudh2290
    looking
    Sam Hodge
    @samhodge
    I think I may know the solution to my problem
    I need to save parts of the model and put the pieces together in C++
    Anirudh Subramanian
    @anirudh2290
    i know that for backend symbolic api one uses mx.sym.variable and then binds it to the model
    Sam Hodge
    @samhodge
    that way I can have a resolution independent model