it is not used later
but we name it, so that we can find it by traversing the graph
I guess if it is output of nn.gModule, then we should manage its backpropagation with gradients too.
Hi, I am using OpenNMT0.9 Lua version on an AWS server with Tesla K80 GPU and Ubuntu 16.04 OS. I ran two trainings in the background and they seem to have been working fine (I used -log_file option, and checked if the log is growing), but once I ran a release model command, at first, I got a segmentation fault core dump message, and when I ran it the second time, the two trainings I mentioned exited. The error log says
THCudaCheck FAIL file=/tmp/luarocks_cutorch-scm-1-1460/cutorch/lib/THC/generic/THCStorage.c line=32 error=39 : uncorrectable ECC error encountered
/torch/install/bin/luajit: cuda runtime error (39) : uncorrectable ECC error encountered at /tmp/luarocks_cutorch-scm-1-1460/cutorch/lib/THC/generic/THCStorage.c:32. Do I have to be cautious with GPU usage? Is there any documentation/known ways to handle multiple processes using a GPU?
Ok, I found http://opennmt.net/OpenNMT/issues/ page, and it seems I can try to disable the caching CUDA memory allocator, but won't it make trainings last much longer (as on CPU)? As for the reducing network size, if my corpus is 1,760K segments, aren't the default values OK ( word embeddings size: 500, structure: cell = LSTM; layers = 2; rnn_size = 500; dropout = 0.3)?
With input feeding, the output of the attention layer at
The
t is concatenated to the word embedding input at t + 1. cf. https://arxiv.org/abs/1508.04025The
brnn_merge option only applies to bidirectional layer, which the rnn encoded type does not have.
@stribizhev Disabling the CUDA caching memory allocator will have a small impact on training time, at most 10% I would say (I did not measure it though). Regarding the network size, people frequently add these options :
-encoder_type brnn -layers 4 -rnn_size 1000.
@guillaumekln Hi Guillaume, thank you for feedback. I have kept the CUDA CMA enabled and the training completed in 1.5 days. The frequently used options are use frequently, I think, because these are the ones used in WMT16 training tutorial. The main problem for me with the network settings is I can't find any guidelines on how to calculate the size. I see that people mostly rely on gut feeling here. http://forum.opennmt.net/t/issues-when-running-the-english-german-wmt15-training/228/19?u=wiktor.stribizew and http://forum.opennmt.net/t/how-should-i-choose-parameters/994 prove that it is not so evident. I used
-layers 4 -rnn_size 1000 -encoder_type brnn -word_vec_size 600 for the next training (ENJA), with 4M segments, and Epoch 1 has been training for ~2 days. Maybe it is OK, but it also takes a lot of GPU memory, and it constantly crashes out with "out of memory" errors.
when I am trying to startup the rest server
Hi, I was hoping to add context vectors for each source sentence to condition the decoder output in the baseline Encoder-Decoder Architecture for ONMT. The task to integrate these in the TextDataset is seemingly non-trivial. Does anyone have ideas on what would be a simplified design choice to achieve this?
Thanks a lot
@guillaumekln In regards to target side word features (factored MT), the docs for OpenNMT (lua) say the decoder predicts the features of the decoded sentence. So, is the prediction loss being backpropagated along with translation loss, and hence eventually improve both the translation and feature prediction tasks? OR does the translation in any way benefit from target side features in the current implementation? Thanks!
@guillaumekln @jsenellart Is there a way to combine two BPE models? The reason is I have a generic model with its BPE model, then I train an incremental model using new corpora for which I create a new BPE model. When running incremental training, I use
-update_vocab merge. If the vocabulary is merged, I guess I should also merge the BPE models to use when translating, right?
Hello, I am interested in learning more about Machine Learning and OpenNMT has me captivated. I have an idea for a project to learn more with OpenNMT, but I require more information on preparing the data source and target files. I've looked through the documentation, but I seem to be having trouble finding any resources about pre-preprocessing the data. As in, are there best practices in preparing the data in source.txt and target.txt before preprocessing? Are there guidelines on the corpus of text required (number of sentences/size of sentences/theme/etc)? Any available reading materials or other resources I could learn more about preparing the data would be greatly appreciated. Thank you very much for the help!
To add to the above question: I plan on using a corpus of text where one body is English and the other is written in Canadian Aboriginal syllabics. Are there considerations I should have with a latin to non-latin characterset? It is possible to convert the syallbics to roman orthography.
@rauschmerscen
Can OpenNMT help me achieve a tool like https://spinbot.com
I have a theory that if I do English->German(or any another language)->English, I can get an article spinner
?
Can OpenNMT help me achieve a tool like https://spinbot.com
I have a theory that if I do English->German(or any another language)->English, I can get an article spinner
?
It is possible?
@rauschmerscen - the idea is good, but I am not sure it will work since the models are generally trained on same dataset, and EN>DE>EN will be very close to the original. on spinbot they appararently have list of paraphrases they use to generate these alternatives. You can find some good review on paraphrase building and novel approaches here: http://aclweb.org/anthology/E17-1083
*very
Can you suggest me the easiest way to achieve spinbot, i can't find any repo.
How can I generate/where can i get a list of paraphrases like spin bot has
