by

Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
    Yanay Lehavi
    @Lehavi
    @Lehavi
    Hi, I have an install problem and would appreciate any help:
    I'm on a Mac Catalina, installed TensorFlow, and it looks fine. Installed t2t using pip install tensor2tensor, no complaint from the OS.
    Trying to run t2t-trainer I get:
    AttributeError: module 'tensorflow' has no attribute 'variable_scope'
    Anyone can help? Many thanks!
    Lukasz Kaiser
    @lukaszkaiser
    @emesha92 : for trax please use this room: https://gitter.im/trax-ml/community
    @Lehavi : I think you have TF 2. For Tensor2Tensor you need TF1, e.g., pip install tensorflow==1.15 ; TF2 is the reason we have trax :)
    tavanduy
    @tavanduy
    Hello, has anyone used t2t transformer for punctuation restoration? My inputs are unpunctuated sentences and outputs are labels. But the longer the model trains, the less COMMA labels appear in decoded test set which is weird.
    Regalia
    @RegaliaXYZ
    @lukaszkaiser would you happen to know which versions of tensorflow/tensor2tensor etc do I need to resume training on the ASR english pre trained model that you made available? Because with my current versions I get a weight_0 not found in checkpoint error that is probably due to a version mismatch
    Bedapudi Praneeth
    @bedapudi6788

    Hi all, I am experimenting with a speech to text model trained using tensor2tensor (not by me). I only have access to saved_model format generated using t2t-export and the vocab for decoding.

    Is it possible to change the decoding params of the exported model?
    Is there some way to see all the beam outputs instead of the top scoring one?

    Any help/suggestions are appreciated. Thank you.

    py4_
    @py4
    @lukaszkaiser I guess there is a dimension mismatch problem for img2img transformer when the mode is PREDICT (it works on EVAL or TRAIN): tensorflow/tensor2tensor#1797
    Kapil Gupta
    @daemonslayer
    How do I retrain a trained model using tensor2tensor?
    HaibaraAi
    @AiHaibara
    I have a question for why setting the ALPHA = 0.1 or 0.3 hparams.learning_rate = ALPHA and train it, but in tensorboard learning_rate is only 2e-4
    HaibaraAi
    @AiHaibara
    image.png
    image.png
    James Hirschorn
    @quantitative-technologies

    Hello,

    I am trying to define by own Text2TextProblem with vocab_type TOKEN. Are there any existing problems of this type to help me figure out how to get it working?

    James Hirschorn
    @quantitative-technologies
    The diaglog_* problems are Text2TextProblem with TOKEN vocabulary.
    Lukasz Kaiser
    @lukaszkaiser
    @AiHaibara : I think you may need to set hparams.learning_rate_constant -- sorry for the confusion!~
    @quantitative-technologies : I think some LM problems like PTB may be TOKEN too? It should be very similar to SUBWORD...
    James Hirschorn
    @quantitative-technologies
    Yes, that is true. But it was easy to figure out how to use TOKEN by looking at the dialog problem. I'm actually not sure what SUBWORD is, but for my problem with symbolic reasoning TOKEN was definitely what I needed.
    Bedapudi Praneeth
    @bedapudi6788
    Hi all, does anyone know if getting token level scores is possible in tensor2tensor.
    I am working on a machine translation task, where I want to mark the words with low confidence.
    James Hirschorn
    @quantitative-technologies

    When I train my transformer using t2t-trainer, I see the targets/accuracy go up to around 65% on tensorboard. But then when I try:

    t2t-decoder --data_dir=$DATA_DIR --problem=$PROBLEM --model=$MODEL --hparams_set=$HPARAMS --output_dir=$TRAIN_DIR --t2t_usr_dir='.'

    It looks like none of the OUTPUTs matches the corresponding TARGETs. Am I missing something obvious?

    Lukasz Kaiser
    @lukaszkaiser
    @bedapudi6788 : are you asking for sth like the TransformerScorer model? https://github.com/tensorflow/tensor2tensor/blob/master/tensor2tensor/models/transformer.py#L1302
    @quantitative-technologies : Transformer is autoregressive, so at 65% accuracy it may not produce any fully identical sequence at decoding time; but normally it should give some reasonable output...?
    James Hirschorn
    @quantitative-technologies

    @lukaszkaiser Yes, the output is "reasonable" about 2/3 of the time. But then what does the 65% refer to?

    I am still learning about the transformer architecture so maybe it will become clear with further reading.

    Lukasz Kaiser
    @lukaszkaiser
    The 65% is per-symbol accuracy of eval; at eval, you do get the ground truth (so all correct symbols) upto time t, and you predict symbol at t+1; at infer, there are no "correct" symbols so you predict t+1 from what you generated upto time t -- so it's usually less accurate than eval
    Bedapudi Praneeth
    @bedapudi6788
    @lukaszkaiser Thank you! I think that's exactly what I need. I will try to figure out how to use that class.
    James Hirschorn
    @quantitative-technologies
    @lukaszkaiser Thanks! I should have guessed that. So I assume that targets/accuracy_per_sequence is the accuracy of predicting the entire target sequence. How about targets/accuracy_top5? My guess is that it is either the accuracy of getting one of the top 5 symbols correct, or perhaps the accuracy of getting all of the top 5 symbols correct. I will have to check the code...
    Lukasz Kaiser
    @lukaszkaiser
    You're right! top_5 is getting the correct one in the top5 (so it's always higher than accuracy)
    Giovanni-Alzetta
    @Giovanni-Alzetta
    Hi everyone! I have a question about the hparam "num_trainable_top_decoder_layers"; I have found it into the evolved transformer. I am expecting it to freeze a certain number of layers for transfer learning, but reading the code I cannot fully grasp what it does when it calls the function "tf.stop_gradient"...could someone please help me? Thank you!
    Anton Kretov
    @anton_kretov_twitter
    Hi everyone! What T2T model has the same architecture as trax Transformer model? Seems like they are all different basing on their parameters number.
    Lukasz Kaiser
    @lukaszkaiser
    @anton_kretov_twitter : as said above, it may be tricky to get that...
    @Giovanni-Alzetta : stop_gradient(x) is just x on the forward pass, but stops all gradient propagation through x on the backward pass... sorry if that's tricky, it is a bit...
    KosayJabre
    @KosayJabre
    Hi, anyone here?
    I'm trying to use a pretrained character level language model, but I keep getting the last character of my input as the output
    I've submitted an issue here tensorflow/tensor2tensor#1808
    Any help or tips would be appreciated
    Lukasz Kaiser
    @lukaszkaiser
    Hi @KosayJabre - I don't think I can help with that, as tensor2tensor is now in maintenance mode... we've switched to Trax, that's where we help and debug.
    Giovanni-Alzetta
    @Giovanni-Alzetta
    @lukaszkaiser Thank you for your answer! Indeed that's a bit tricky...
    Yongqiang Cheng
    @ForeverStrongCheng
    @lukaszkaiser
    Yongqiang Cheng
    @ForeverStrongCheng
    TIMIT dataset - audio_timit_characters_test - speech recognition
    for i in xrange(self._model_hparams.audio_compression):
    AttributeError: 'HParams' object has no attribute 'audio_compression'
    Yongqiang Cheng
    @ForeverStrongCheng
    tensorflow/tensor2tensor#1821 Any help or tips would be appreciated。
    Lukasz Kaiser
    @lukaszkaiser
    @ForeverStrongCheng : T2T is deprecated by now, we're using Trax and TFDS (tensorflow-datasets) for data. I think TFDS may offer more help with data.
    dinosaxon
    @dinosaxon
    Hi all, I have a question. On a 1-GPU machine, I have several mt models and I want to translate using the same model batches of files. I am using t2t-decoder but this way the model is loaded from scratch for each batch. Is there any way to keep the model loaded and translate each batch separately? I tried to merge the batches in one single file but the system crashes due to OoM so this is why I am splitting it into smaller files. I don't want to use the servering. Any help would be appreciated
    Giovanni-Alzetta
    @Giovanni-Alzetta
    I would be interested into training a transformer with multiple targets (from a string generate a list containing two different strings). I have seen this tensorflow/tensor2tensor#183 but the link is broken. Has anything changed in this direction?
    Jiajia Zhao
    @snowbabyjia
    would it be possible to acquire a docker image of the previously working version of SimPLe?
    dinosaxon
    @dinosaxon
    Hi, I am facing an issue training an Engine on an 8GPU google instance. However, when I train on a single GPU, the training starts with no issues. I also tried to use smaller batch size but no luck. Any suggestions? Here is the command I use
    t2t-trainer \
    --data_dir=$DATA_DIR \
    --problem=translate_enfr_wmt_small8k\
    --model=transformer \
    --hparams='max_length=100,eval_drop_long_sequences=True'\
    --hparams='batch_size=1024'\
    --worker_gpu=8 \
    --train_steps=350000 \
    --hparams_set=$HPARAMS \
    --eval_steps=5000 \
    --output_dir=$TRAIN_DIR \
    --schedule=continuous_train_and_eval
    I am getting an Out of Memory error or an allocation error. I am using Tesla K80 with cuda 10.1 and tensorflow 2
    it crashes even if I use batch size 400 but no with one GPU. I also tried to CUDA_VISIBLE_DEVICES
    Regalia
    @RegaliaXYZ
    @lukaszkaiser Hello, it seems like every PR on tensor2tensor fails during the Travis build with the same error:
    The command "./oss_scripts/oss_pip_install.sh" failed and exited with 1 during .