Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
    Waqar Ali
    @aliw77
    any thoughts on why the xhtml parser is not extracting the Level 1 bullets in a multi-level ul-li structure
    image.png
    Igor Afanasyev
    @iafan
    @aliw77 because there's a localizable content inside (another list). Wrap the underlined strings with <span lang="en">...</span> to extract them
    Edgar Kalinovski
    @edgar.kalinovski_gitlab

    Good day, need community help. In smartcat my translation files look like:

    en.json_ru
    en.json_uk
    But after sync I see in smartcat project a lot of new empty translation files like:

    et.json_ru
    hu.json_be

    Output during sync:

    Pushing translation files to external translation service...

    Use of uninitialized value in string ne at /serge/bin/../lib/Serge/Sync/Plugin/TranslationService/smartcat_v2.pm line 135.
    Running 'push' command for project '111111111111111111111111111111111' and translation files from '/data/ts/test'...
    Getting project '11111111111111111111111111111111111'...
    Updating project '11111111111111111111111111111111111' with 'source:Serge' external tag...
    State:
    Upload [516]
    'hu.json (uk)', 'hu.json (be)', 'en-GB.json (ja)', 'en-AU.json (ms)', 'hu.json (es)', 'en-AU.json (en-GB)', 'en-AU.json (de)', 'et.json (ru)', 'en-AU.json (zh-Hans-SG)', 'de.json (en-GB)', 'en-US.json (cs)', 'cs.json (en-GB)', 'en-AU.json (fr)', 'en-CA.json (tr)', 'be.json (tr)', 'fi.json (no)', 'hr.json (pl)', 'en-US.json (bg)', 'et.json (lt)', 'en-US.json (id)', 'be.json (it)', 'et.json (zh-Hans-MY)', 'hu.json (en-AU)', 'fi.json (cs)', 'en-CA.json (es)', 'da.json (en-GB)', 'hr.json (zh-Hans-SG)', 'hu.json (bg)', 'fi.json (da)', 'hu.json (en-CA)', 'en-AU.json (ko)', 'be.json (fi)', 'es.json (en-GB)', 'en-US.json (zh-Hans)', 'es.json (it)', 'de.json (tr)', 'cs.json (tr)', 'da.json (it)', 'en-CA.json (en-AU)', 'en-GB.json (ko)', 'et.json (nl)', 'fr.json (fr)', 'fr.json (et)', 'en-CA.json (pl)', 'be.json (en-GB)', 'es.json (fi)', 'bg.json (no)', 'en-CA.json (da)', 'da.json (fi)', 'et.json (hu)', 'en-GB.json (fr)', 'de.json (fi)', 'fr.json (ms)', 'bg.json (cs)', 'cs.json (fi)', 'hu.json (sv)', 'en-AU.json (ja)', 'en-GB.json (ms)', 'es.json (tr)', 'cs.json (it)', 'de.json (it)', 'et.json (de)', 'en-GB.json (zh-Hans-SG)', 'da.json (tr)', 'en-GB.json (de)', 'en-GB.json (en-GB)', 'hu.json (ro)', 'bg.json (da)', 'en-GB.json (sv)', 'en-AU.json (fi)', 'et.json (ja)', 'be.json (ms)', 'en-AU.json (nl)', 'et.json (ko)', 'en-AU.json (ru)', 'bg.json (zh-Hans-SG)', 'en-US.json (en-AU)', 'id.json (cs)', 'be.json (et)', 'et.json (zh-Hans)', 'be.json (fr)', 'et.json (en-US)', 'en-US.json (da)', 'fr.json (en-GB)', 'hu.json (id)', 'hu.json (lv)', 'en-US.json (pl)', 'bg.json (pl)', 'id.json (da)', 'hr.json (cs)', 'es.json (fr)', 'es.json (et)', 'en-CA.json (bg)', 'fr.json (it)', 'da.json (et)', 'en-CA.json (id)', 'da.json (fr)', 'en-CA.json (zh-Hans)', 'de.json (ms)', 'fr.json (fi)', 'cs.json (ms)', 'hr.json (da)', 'en-US.json (es)', 'fi.json (zh-Hans-SG)', 'et.json (hr)', 'en-GB.json (ru)', 'en-CA.json (cs)', 'da.json (ms)', 'es.json (ms)', 'cs.json (fr)', 'hr.json (no)', 'cs.json (et)', 'de.json (fr)', 'fi.json (pl)', 'en-US.json (tr)', 'de.json (et)', 'en-GB.json (fi)', 'en-AU.json (sv)', 'fr.json (tr)', 'en-GB.json (nl)', 'hu.json (hr)', ................

    Taras Semenenko
    @ta2-1
    Hi @edgar.kalinovski_gitlab Could you share your serge config and project file structure in your repo?
    Edgar Kalinovski
    @edgar.kalinovski_gitlab

    sync
    {
    ts
    {
    @inherit common.serge.inc#sync-smartcat

        data
        {
            project_id              11111111111111111111111111111111
            #project_translation_files_path ../ts/test
        }
    }
    
    vcs
    {
        @inherit                    common.serge.inc#sync-git
    
        data
        {
            local_path              ../vcs/test
            remote_path             git@git.name.eu:user/test.git
        }
    }

    }

    jobs
    {
    {
    @inherit common.serge.inc#json-common

        name                        .json processing
        id                                 test
        db_namespace                test.json
    
        destination_languages       lt lv et fi no sv uk ru pl en-AU en-CA en-GB en-US cs fr nl it ja ko hu hr ro bg de da be id ms es tr zh-Hans zh-Hans-MY zh-Hans-SG
    
        source_dir                  ../vcs/test/
        source_match                \.json$
    
        output_file_path            ../vcs/test/%LANG%.json
    
        ts_file_path                ../ts/test/%LANG%/%FILE%.po
    
        parser
        {
            plugin               parse_json
        }
    
        output_lang_rewrite {
               zh-Hans zh-CN
        }
    }

    Project I use: https://github.com/smartcatai/smartcat-serge-bootstrap

    common.serge.inc file:

    sync-git
    {
    plugin git

    data
    {
        add_unversioned         YES
        email                   user@domain.com
        name                    User
    }

    }

    sync-smartcat
    {
    plugin smartcat_v2

    data
    {
        base_url                https://smartcat.ai
    
        account_id              1111111111111111111111111111111
    
        token                   1111111111111111111111111111111
    }

    }

    job-common
    {
    active YES

    reuse_orphaned              NO
    reuse_as_fuzzy_default      NO
    reuse_translations          NO
    
    source_language             en
    source_process_subdirs      NO
    normalize_strings           NO
    db_source                   DBI:SQLite:dbname=../db/translations.db3
    output_lang_files           YES
    output_default_lang_file    NO
    output_encoding             UTF-8
    output_bom                  NO
    trademarks                  company

    }

    android-common
    {
    source_match strings.xml$

    parser
    {
        plugin                  parse_android
    }
    
    normalize_strings           YES

    }

    json-common
    {
    source_match .json$

    parser
    {
         plugin                  parse_json
    }

    }

    Igor Afanasyev
    @iafan
    @edgar.kalinovski_gitlab here's the problem:
        source_dir                  ../vcs/test/
        source_match                \.json$
    
        output_file_path            ../vcs/test/%LANG%.json
    you scan all the files with the .json extension in your ../vcs/test/ folder, localize them, generate more .json files in the same folder, and on the next sync cycle, scan them again as a source, so the number of files will explode with each sync run.
    you need to make sure that you're selecting only the source language files in your source_match mask
    it can be as easy as specifying ^en\.json$
    Edgar Kalinovski
    @edgar.kalinovski_gitlab

    Thank You, @iafan it works! :)

    Now I'm trying to "serge localize test.serge" project, but get this:

    [Serge Shell] /data/configs $ serge localize test.serge

    /data/configs/test.serge

    Running the translation framework...

    [test] ".json processing"

    OPTIMIZATIONS DISABLED: job definition has changed (or job is ran for the first time)
    Source dir: [/data/configs/../vcs/test]
    DB source: [DBI:SQLite:dbname=../db/translations.db3]
    TS file path: [/data/configs/../ts/test/%LANG%/%FILE%.po]
    Output path: [/data/configs/../vcs/test/%LANG%.json]
    Destination languages: [be,bg,cs,da,de,en-AU,en-CA,en-GB,en-US,es,et,fi,fr,hr,hu,id,it,ja,ko,lt,lv,ms,nl,no,pl,ro,ru,sv,tr,uk,zh-Hans,zh-Hans-MY,zh-Hans-SG]
    Modified languages: [be,bg,cs,da,de,en-AU,en-CA,en-GB,en-US,es,et,fi,fr,hr,hu,id,it,ja,ko,lt,lv,ms,nl,no,pl,ro,ru,sv,tr,uk,zh-Hans,zh-Hans-MY,zh-Hans-SG]
    Preloading cache for job 'test' in namespace 'test.json'...

    Updating database from source files...

    Scanning directory structure...
    Scanned in 0.000608 sec, 0 files match the criteria
    Exception occurred while processing job 'test': No files match the search criteria. Please reconfigure your job at /serge/bin/../lib/Serge/Engine.pm line 473.

    'localize' step took 0.144194 seconds

    Sync complete. 1 job in 1 configuration file was processed
    1 job ended abnormally

    What is wrong? Because of this I can not pull-ts translation files from Smartcat.

    Igor Afanasyev
    @iafan
    @edgar.kalinovski_gitlab well, this means that source_dir + source_match parameters don't match any files. If you're talking about the same project we talked earlier, then maybe the pattern I suggested, ^en\.json$ needs some adjustments.
    Edgar Kalinovski
    @edgar.kalinovski_gitlab
    Thanks, @iafan :)
    Konstantin Sipunin
    @vintagentleman

    Hi! I need some help adapting Serge to my needs.
    The standard parse_js parser only seems to work properly on simple key-value pair objects, and localized strings are apparently supposed to be stored in files different than the source content.
    In my case every piece of localized content is stored in a single file like this:

    const localization = {
        'all': {
            ru: 'Все',
            en: 'EN localization Все',
            cn: 'CN localization Все',
        },
        // ...
    };

    Here, Russian is the source language, and strings that need localization contain the untranslated content with self-explanatory prefixes.
    Is there a way to parse a structure like this properly other than via a crude metaparser with regexes?

    Igor Afanasyev
    @iafan
    @vintagentleman to avoid merge conflicts, I'd recommend keeping every language in its own file, then require your language files to make it a bundle. This gives a clean separation between files, where a developer commits changes to source language files only, and localized files are generated by Serge.
    Igor Afanasyev
    @iafan

    so your source file (ru.js) could become:

    export default {
      all: 'Все',
      //...
    }

    and then in an outer code you can do something like:

    export loc {
      ru: require('ru'),
      en: require('en'),
      zhCN: require('zh-CN'),
      //...
    }

    so that then you can use loc['ru'].all or loc['en'].all

    Konstantin Sipunin
    @vintagentleman
    @iafan Thanks for the advice, this does seem the right way to do it.
    Alas, with this approach loc.all.ru will have to be changed to loc.ru.all all across the existing code base, and likewise for everything else.
    Also, I suppose some proxy will have to be implemented so that references to loc.en.all return a placeholder string like EN localization Все when no localization has been supplied yet.
    Igor Afanasyev
    @iafan
    @vintagentleman you won’t need to implement a fallback mechanism: Serge will ensure you have all the keys in all locales. Until a translation is provided, the English string will be used as a value.
    Konstantin Sipunin
    @vintagentleman
    @iafan As I understand it, Serge will ensure this only upon sync.
    But supposing I'm a just a frontend developer who knows nothing about localization nor has Serge installed locally. I create a feature branch and update the file in the default locale only (e.g. en.js). If I now build my project from this branch and send it to QA like this, Serge won't have synced anything since, well, at least the way I see it, continuous localization is intended to be run against the master/release branch only. So I have to run serge localize manually so that placeholders in the default locale are generated.
    Igor Afanasyev
    @iafan
    @vintagentleman not upon sync, but upon serge localize, which you can run as often as you want to. You can also run localization against multiple branches (see https://serge.io/docs/plugins/callback/feature_branch/ and https://github.com/evernote/serge/tree/master/bin/tools/feature-branch-config-generator)
    but regardless of using or not using branches, you need to figure out how to separate resources into per-locale files. You can implement a fallback mechanism if you want to (might be a good thing regardless, as it will likely make your code cleaner)
    Konstantin Sipunin
    @vintagentleman
    @iafan Thanks for this discussion! I'll try looking into the config generator tool.
    Waqar Ali
    @aliw77
    output_file_path seems to remember prior file output pattern, e.g. i had it set to %NAME%.%LANG%.%EXT% and then changed it to have an underscore instead %NAME%__%LANG%.%EXT% but the sync cycle keeps generating the BOTH output file patterns. What could be causing this.
    Igor Afanasyev
    @iafan
    @aliw77 Serge doesn't clean up old localization files. Try removing the files you don't need, then run serge localize <config.serge> and see if it re-generates them.
    Waqar Ali
    @aliw77
    @iafan - thank you for the tip, I figured out my mistake was using sync in a loop and it was just downloading the previous output files... rookie mistake, need to get a hang of the sync cycle... eventually ;)
    Kardo Kallis
    @kkallis_gitlab

    Hello, I'm setting up smartcat_serge_bootstrap and I have a problem with parse_json and the output of the .po files for Smartcat.

    en.json file structure

    [
        {
            "term": "app.logout",
            "definition": "Log Out",
            "context": "Logout text, displayed on button or in a tooltip",
            "term_plural": "",
            "reference": "",
            "comment": ""
        },
        {
            "term": "app.login.dismiss",
            "definition": "Dismiss",
            "context": "Text on a button to dismiss error message. \"Oh, well...\"",
            "term_plural": "",
            "reference": "",
            "comment": ""
        },

    en-US/en.json.po file

    msgid ""
    msgstr ""
    "Content-Type: text/plain; charset=UTF-8\n"
    "Content-Transfer-Encoding: 8bit\n"
    "Language: en_US\n"
    "Generated-By: Serge 1.4\n
    
    . [0]/context
    : File: en.json
    : ID: cd58f1b123bd677d3de77935fa6cf04e
    msgid "Logout text, displayed on button or in a tooltip"
    msgstr ""
    
    . [0]/definition
    : File: en.json
    : ID: c2412a7a82f557d9cce54381bca114cc
    msgid "Log Out"
    msgstr ""
    
    . [0]/term
    : File: en.json
    : ID: 03c29e74c13151068f484bbe2dad9e24
    msgid "app.logout"
    msgstr ""
    
    . [1]/context
    : File: en.json
    : ID: e678f89e8b02311b0041cfa1f36e4387
    msgid "Text on a button to dismiss error message. \"Oh, well...\""
    msgstr ""
    
    . [1]/definition
    : File: en.json
    : ID: e4092ae6de95a1a2ef4bff834c8f649d
    msgid "Dismiss"
    msgstr ""
    
    . [1]/term
    : File: en.json
    : ID: d91df5e730f741ee789371d9063a36db
    msgid "app.login.dismiss"
    msgstr ""

    But in older setup we have the en.json.po file looks like this:
    en.json.po

    Igor Afanasyev
    @iafan
    @kkallis_gitlab for the format that you described you will need a custom parser. I recommend looking at parse_chrome_json plugin (https://serge.io/docs/plugins/parser/parse_chrome_json/) if you want to create your own, since it's closer to the format you have.
    Kristina Popova
    @kr111kr
    image.png
    Hello!
    Is it possible to make line translations unloaded from the Smartcat only if they are in a certain status, for example "Done"? I can't found any parameters for this using Serge with Smartcat. The standard behavior unloads any changed value, even if that appears without changing the status. But this behavior doesn't suit us because our translators write translations into the system, then they themselves double-check each other, make corrections and only then change the status. I would like to unload the value after all changings (i.e. after changing this status)
    Taras Semenenko
    @ta2-1
    Hi Kristina, there is a pull_params section in the serge config file, you can set “mode=complete” there.
    Brian Caruso
    @carusology

    Hi there! :wave:

    I recently got some pushback from our translators about the optional "approved" annotation for <trans-unit> elements of XLIFF interchange files. It looks like it is incorrectly being set to "yes" when a translation is still needed due to fuzzy being set to undef in that state. This attribute value doesn't make sense. There's no translation that would be needing final approval because one doesn't exist yet. It seems the "approved" annotation should be absent when a translation is also absent.

    I filed this bug on the issue: evernote/serge#136

    If I submit a MR that avoids setting any "approved" annotation when a translation does not exist, would you accept it?

    Igor Afanasyev
    @iafan
    @carusology hi! I responded in the ticket. Thanks for the detailed exaplanation. Feel free to provide a PR. Please make sure to run engine.t --init in the t/ folder to re-generate test data, then git diff to see if there are not changes that shouldn't be there (and the changes that are there make sense are are a direct result of your code changes), then submit changes to test data as a part of the PR.
    Brian Caruso
    @carusology

    Hi Igor, thanks for the prompt response. :bow:

    I'll get feedback from our translators (Powerling) on "approved=no" vs approved omitted entirely (it should be logically the same) and follow up with a MR once I hear back.

    Kristina Popova
    @kr111kr
    Hi Taras! Thanks for the quick response!
    Setting the pull_params --mode = complete option really helped. It would be great if you could add this information to the documentation. Now it can only be found in smartcat https://smartcat.com/api/methods/#!/DocumentExport/DocumentExport_RequestExport. And what is needed there "--" only in the smartcat-cli source
    Brian Caruso
    @carusology
    @iafan alright, MR up and ready for review: evernote/serge#137. Thanks!
    Brian Caruso
    @carusology
    Hey @iafan :wave: - do you happen to know when would have some time to review this MR? --^ We can certainly use a fork if you're overloaded, but I figured I'd ask.
    Igor Afanasyev
    @iafan
    @carusology just merged your PR. thanks!
    Brian Caruso
    @carusology
    Thanks! 🙇‍♂️
    Andrzej Jordanow
    @ajordanow
    Hi @iafan - I think I've found a bug in run_command plugin (probably elsewhere :). I have many files that match source_match but the run_command always get input parameters for the first matching file (e.g. command /bin/echo %OUTFILE% %LANGNAME% %FILE% returns FILE value that is the first matching file). Could you confirm this? Is there any quick fix around this? Or maybe you could point me at the source code where I should look to investigate this issue further?
    Igor Afanasyev
    @iafan
    @ajordanow can you share (privately) some minimal Serge configuration file with this plugin enabled, and a bunch of resource files that you process with it? That would make it easier to reproduce the issue on my side and identify the problem
    1 reply
    Vera
    @VeraVes
    @iafan Hi! Could you please help me? How can I use parse_json_keyvalue plugin, but turn off space_after option?
    Igor Afanasyev
    @iafan

    @VeraVes this particular plugin doesn't have any configuration options, but do you mean that instead of

    "key1": "string",

    you would prefer to have no space after the colon, like below?

    "key1":"string",

    If that's the case, you can post-process the resulting localization file by adding a replace_strings callback plugin tied to before_save_localized_file phase, exactly as shown in the documentation: https://serge.io/docs/plugins/callback/replace_strings/#sel:I2NvZGVfMC9zcGFuLjI1L0AtI2NvZGVfMC9zcGFuLjkxMy9AOjE

    There you can define your own replace regular expression that would postprocess the content on the fly, removing this space symbol.

    Vera
    @VeraVes
    @iafan Thank you, got it!
    Waqar Ali
    @aliw77
    serge is not matching all files in the folder for some reason (picking up 32 files, but there are 35)
    CleanShot 2021-10-11 at 21.05.09.png
    CleanShot 2021-10-11 at 21.07.23.png
    regexp match is confirmed, file exists, but for some reason i get:
    Scanning directory structure...
    Scanned in 0.036914 sec, 32 files match the criteria
    other folders/files are new, and are locally visible in the server where serge is running
    Mahmoud Almasri
    @Mhm0ud

    Hi,
    I'm trying to use srege with smartcat, and I have an issue:
    I need to use vcs to pull content from gitub repo, but I need to pull specefic directory only not the all project, is there anyway to do that?
    this is my serge config file:

    vcs
        {
            plugin                                  git
    
            data
            {
                add_unversioned                     YES
    
                clone_params                        --depth 1
                email                               l10n@mycompany.com
                name                                L10N Robot
    
                local_path                          data
    
                # Set this parameter to be your Git clone URL. If your repository
                # main branch is not called 'master', you need to specify it 
                # explicitly at the end of the URL, like this: #main
                remote_path                         git@github.com:Mhm0ud/smartcut-project-a.git#main
            }
        }

    the folder that I need to use it in serge is under main branch /test/test1/