Oh I guess I'm a little lucky.. My deployed models don't need to be trained in the field... So, I just train them on pyTorch, and then copy the weights to my re-written tensorflow models... So I had to rewrite a few to make them pytorch friendly... I think bidirectionalLSTM was one, which had subtle differences between TF's implementation and Torch's... But, man development on pyTorch is just so much easier all this is still worth it...
That might not always work unfortunately since there are still TF-specific things like feature normalization which makes concrete TF model work while Torch doesn't care much.
tensorboardXand use it on non-gpu ec2 instances.. but doesn't seem to work on p2
pytorch_p36environment successfully, but Jupyter notebook shows
conda_pytorch_p36as the environment - is this the issue? all environment have the conda prefix in jupyter but not in the terminal
I’m new to Torch and I’m using it for my master thesis project.
I am trying to integrate to an existing implementation a personalized loss function like below :
sum_loss = lambda * coral_loss + classification_loss
where coral_loss is an unsupervised loss that measures the “distance” between a source and a target batch.
Basically I want to minimize the classification loss on the source data while jointly minimizing this distance between source and target data distributions.
In PyTorch that is very easy since I can just call sum_loss.backward() to compute the gradients. However in Torch there are predefined Criterions in which I can only call backward(input,target) to compute the gradients of the loss function associated to that specific criterion.
How can I implement the new loss function and backpropagate to compute the gradients?
This is the loss that I would like to implement in Torch :
def CORAL(source, target): d = source.data.shape # source covariance xm = torch.mean(source, 0, keepdim=True) - source xc = xm.t() @ xm # target covariance xmt = torch.mean(target, 0, keepdim=True) - target xct = xmt.t() @ xmt # frobenius norm between source and target loss = torch.mean(torch.mul((xc - xct), (xc - xct))) loss = loss/(4*d*d) return loss