Hi @guillaumekln , thanks so much for your response, it's very helpful!! Just a couple of quick follow-up questions: 1. I think the maximum_decoding_length and length_penalty are parameters affecting both training and inferencing, am I correct? 2. does the 'length' here refer to the number of characters, the number or subwords, or the number of words?
@guillaumekln for another parameter 'maximum_features_length', does the length here also refer to the number of tokens, as defined by the tokenization?
Hi @guillaumekln I've trained a Transformer model using onmt and have also run infer command to do the inference. all went good. now I have a requirement to bring the model into production which supports tensorflow, so I exported the model into TF SavedModel format (https://opennmt.net/OpenNMT-tf/serving.html). However for the inferencing part, some beam or decoder related parameters may affect the inference latency, such as beam_width, n_best, maximum_decoding_length, how I could incorporate these onmt parameters given an exported TF SavedModel package, in the tensorflow production environment? onmt inferencing refers to a config file where we specify those parameters, would a similar config file needed for doing inference given a SavedModel on a machine where only tensorflow is supported?
Thanks in advance!
One additional question @guillaumekln :) I read the comment for the parameter 'sampling_topk': # (optional) Sample predictions from the top K most likely tokens (requires beam_width to 1). If 0, sample from the full output distribution (default: 1).
My question is: With greedy search model, how should I set "sampling_topk" to maximize the speed?
My question is: With greedy search model, how should I set "sampling_topk" to maximize the speed?
Just should be the default value 1, right?
Hi. See this code example: https://forum.opennmt.net/t/problem-in-using-extract-embeddings-lua/3779/2
Thank you @guillaumekln ! Got it.
Not sure though about the one in the image. I printed the size of my vocab and the embedding shape. What could be this difference? 24000 and 24001
I would like to treat unknown words like Names as UNKs and they should be replaced by their sources. I do set replace_unknown_target to True. However, the result seems not like what I expects, all Names replaces by some word which are not correct. Am I missing something? I don't see <UNK> in my vocab as well as the prediction. Is it normal ? Thank you
replace_unknown_target uses the model attention to select the corresponding source token. However, it is well known that Transformer attention usually can not be used as target-source alignments. You should either constrain the attention to be an alignment or use subword tokenization (like SentencePiece) to avoid UNK. Note that the UNK token does not appear in the vocab but is automatically added when starting the training.
You probably need to increase
max_step in the training parameters. There should be a warning about this somewhere in the logs. We just improved that for the next version: a more visible error message will be shown, see OpenNMT/OpenNMT-tf@21df1c7
Ive tried both with normal pip install and with downloading and pip installing the wheels available on pypi
Hello
I'm using SetentencePiece as tokenizer to train an OpenNMT model
I will like that when I ask the model to translate something in upper case "HELLO" is able to preserve the case in the translation.
I was expecting this to be a configuration of the Tokenizer but I have not been able to found it. Any help or hint is appreciated.
I'm using SetentencePiece as tokenizer to train an OpenNMT model
I will like that when I ask the model to translate something in upper case "HELLO" is able to preserve the case in the translation.
I was expecting this to be a configuration of the Tokenizer but I have not been able to found it. Any help or hint is appreciated.
Hi. You can either add training examples in uppercase or look into the
case_markup option from the Tokenizer.
Hello
I updated from OpenNMT 2.40 to 2.70 and "replace_unknown_target=True" has stop working. And now I get <unk> tag instead of the source tags for words of vocabulary. Is possible that a regression has been introduced after 2.40? Thanks
(also fails for me with the latest version 2.13)
I updated from OpenNMT 2.40 to 2.70 and "replace_unknown_target=True" has stop working. And now I get <unk> tag instead of the source tags for words of vocabulary. Is possible that a regression has been introduced after 2.40? Thanks
(also fails for me with the latest version 2.13)
I am trying to understand exactly how
effective_batch_size works. The auto config for a Transformer model is effective_batch_size: 25000 and batch_size: 3072. This means that 9 iterations are required to accumulate the gradients to reach a batch size of 25000 on a single GPU. So does that mean that the actual effective batch size is 3072 * 9 = 27648? If this is true, then I would expect that if I set batch_size to 8192, the actual effective batch size would be 8192 * 4 = 32768. This feels like enough of a difference in effective batch size that it would have an impact on training. Is this accurate?
We typically want to avoid changing the user provided
batch_size since increasing it would result in OOM and decreasing it would result in under utilization of compute resources.
Hello. We'd like to experiment with guided alignments for TransformerBig model. Is the expected beneficial impact of training the model with guided alignment related to efficiency, performance or quality? Having read https://opennmt.net/OpenNMT-tf/alignments.html#alignments, https://forum.opennmt.net/t/guided-alignment-and-weighted-datasets/4084 and https://forum.opennmt.net/t/opennmt-tf-how-to-use-alignments-and-phares-tables/2209/5, we understand that guided alignment constraints the attention to be an alignment, that it could be used by decoding features for replacing unk (however since we use subword encoding it doesn't seem needed for this). and that alignment information can be retrieved for additional postprocessing. However, we do not yet have a clear understanding of the expected benefits. Thanks in advance.
Hi. The primary benefit is that one attention head can now be used as alignment. Guided alignment does not improve efficiency (it's actually less efficient) and I don't think there is evidence that it improves quality in any way. If you don't need the model to produce alignments, then you probably don't need to use guided alignment.
Thanks. Indeed we had experimented with guided alignment in some older models, without achieving any added-value. But we wanted to test it with our current, improved models as well. For the primary benefit you mentioned, I understand that by using an attention head as alignment, and as explicit alignment may help in determining which target words were generated by which source words, this could result to better attention weights and therefore better performance, no?