Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
    Aaron Radzinski
    @aradzinski
    all default tool chain must work/build/compile without any Python-related installation or requirements
    Let's not lose the sight of that as we are improving our Python support
    Rahul Padmanabhan
    @rahul3

    Thanks for clarifying. It may be a good idea to have a build switch property that enables/disables python component builds in the future.

    On a différent note, if you could point me to any easy/medium complexity Scala tickets to work on, I would appreciate it.

    Aaron Radzinski
    @aradzinski
    how about NLPCRAFT-361
    skhdl
    @skhdl

    I still didn't do detailed code review.
    I just tried to review how it works and want to shortly describe my remarks which should be fixed in current PR.
    After we 'll do detailed code review code, etc

    If somebody has another opinion - please let's discuss.
    I can move this discussion to devList, @aradzinski - what do you think?

    1) python part should be optional.

    • required libs for project are just: java 11 and maven.
    • mvn clean package/verify/install should work as in master branch without python stuff.
    • Apache actions should work https://github.com/apache/incubator-nlpcraft/actions
    • ctx server and spacy are disabled by default.
      So, by default everything should be same as in master branch and requires installed java and maven only.

    2) python stuff usage.
    there are few python stuff installations ways:

    • via scripts,
    • via maven to simplify OS depended issues.
      if maven way is prefered - it should be processed with some parameter - like mvn clean package/verify/install -some_parameter_use_python (if it is possible with maven)
      • required libs for project: java 11, maven, python, conda (? -is it possible to install it on backstage?)

    (@rahul3, -Dauto-rm-existing-python-env= - is good thinks I guess, but it is about general approach)

    3) main current remarks about implementation.
    3.1) We can't keep such progress bar in out installation output
    (Look please at attached file)
    How to avoid it - another question, but our final installation product can't print it I guess

    3.2) Also we have to try to find some way to avoid warning like

    python/fasttext_module/fasttext/pybind/fasttext_pybind.cc:346:35: warning: comparison of integers of different signs: 'int32_t' (aka 'int') and 'std::__1::vector<long long>::size_type' (aka 'unsigned long') [-Wsign-compare]
        for (int32_t i = 0; i < vocab_freq.size(); i++) {

    I see that some of these things are out of our control - but we have to do some workarounds, change our approaches etc - we can't provide it to our users (Especially progressbar)

    3.3) start_server.sh
    Duplicated output - should be fixed (it is still here)

    CUDA is not available for Torch.
    CUDA is not available for Torch.
    [2021-08-23 12:52:42]: 80055 DEBUG Registering ctxserver blueprint
    [2021-08-23 12:52:42]: 80057 DEBUG Registering ctxserver blueprint

    3.3) server starts too slowly comparing to previous version

    I think 3.3 and 3.4 - significant issues and should be resolved in this ticket (we can't merge implementations with such degradation)

    Conclusions:
    I guess after these 6 points (1, 2, 3.1-3.4) resolving, we can start deep code review and detailed tests on various OS.
    Other improvements etc can be moved on next tickets.

    skhdl
    @skhdl
    current progress bar output.png
    skhdl
    @skhdl
    above - current progress bar output
    Aaron Radzinski
    @aradzinski
    we need to move this discussion to dev list - otherwise too many places to look
    I'm already getting lost
    skhdl
    @skhdl
    done, @rahul3 - do you receive devList email messages ?
    Rahul Padmanabhan
    @rahul3
    Yes I do
    Will check it in detail after work today . Say in 24/48 hours
    Aaron Radzinski
    @aradzinski
    👌
    Rahul Padmanabhan
    @rahul3
    Sorry. Need more time. Slammed at work. Will check by this weekend
    Aaron Radzinski
    @aradzinski
    no worry! Free time always ebbs and flows…
    Rahul Padmanabhan
    @rahul3
    FYI : I’m slammed with schoolwork as semester started (apart from my job) . I’ll resume this task and also contribute more when I have free time. Likely December but may be before.
    Aaron Radzinski
    @aradzinski
    N/p
    We are embarking on rework on core APIs- so it is a good timing
    Philip D'Souza
    @PhilipAD

    After downloading Apache NLPCraft library (https://nlpcraft.apache.org/download.html) and running the command bin\nlpcraft.cmd I get the following error:

    "\apache-nlpcraft\build was unexpected at this time."

    what's going wrong here?

    @skhdl
    skhdl
    @skhdl
    Hi. It is apache release, you can't start shell scripts with these source files. You should build it before etc. Please look at https://nlpcraft.apache.org/installation.html
    Rahul Padmanabhan
    @rahul3
    Is gitter still active for this project?
    I was looking for the list of changes the project is going through . If someone can point me to it, it would be great. Thanks.
    Aaron Radzinski
    @aradzinski
    Hi Rahul - nice to have you back
    changes are significant
    Let's ask Sergey if we have any write up on the new architecture...
    @skhdl Sergey?
    skhdl
    @skhdl
    Hi! Tomorrow I am sending email on dev-list, describing last changes.
    Rahul Padmanabhan
    @rahul3
    Thanks!
    skhdl
    @skhdl
    Rahul Padmanabhan
    @rahul3
    I can work on them
    We don't have existing idl examples any more right?
    Rahul Padmanabhan
    @rahul3
    I think it will be good, if the examples are (primarily) in Scala. Like you said in your email, I think it is the more preferred language for NLP developers compared to Java/Kotlin/Groovy
    skhdl
    @skhdl

    We don't have existing idl examples any more right?

    We have few examples, where all intents definitions are based on IDL (please look at light-switch, light-switch-ru, light-switch-fr, pizzeria, time) We just need mores examples,
    maybe especialy very similar to existing in Google Dialogflow and Amazon Alexa to be able to compare approaches between them and Nlpcraft,
    compare usability of the products, strong and week sides.

    Rahul Padmanabhan
    @rahul3
    Feel free to assign this to me: https://issues.apache.org/jira/browse/NLPCRAFT-497
    I'm working on an example for pizza delivery
    skhdl
    @skhdl
    done
    Rahul Padmanabhan
    @rahul3
    Question: Where is nlpcraft_idl_idea_settings.zip located?
    image.png
    From the IDL documentation:
    I can't seem to find the file in the project.
    skhdl
    @skhdl
    It is accessible in master-0.9.1 branch (it is not fixed and moved yet). IDL was changed, so this file can not be 100% actual.
    Rahul Padmanabhan
    @rahul3
    Thanks
    I could not find a clear defintion of 'system token'. Can someone clarify what that is?
    image.png
    The link points to the table with NlpCraft tokens
    skhdl
    @skhdl
    the notion System token left in last version (before we had 2 kinds of tokens: user defined and system based on built enrichers like nlpcraft:date etc)
    Rahul Padmanabhan
    @rahul3
    We dont have anymore nlpcraft:xxxx etc?
    I wasn't aware of that if that is the case.
    skhdl
    @skhdl
    Yes, we drop support of built NERs at least for now.
    You can
    • implement your own NER (implement NCEntityParser) and add with NCPipelineBuilder (look at any test or example)
    • use predefined wrappers for stanford (NCStanfordNLPEntityParser) and opennlp (NCOpenNLPEntityParser) NERS - look at org.apache.nlpcraft.examples.time.TimeModel#TimeModel
    • Also use can use one custom implementation - NCSemanticEntityParser (look at any examples, 'time', 'light-switch', etc.)
      RU and FR versions - org.apache.nlpcraft.examples.lightswitch.nlp.entity.parser.NCRuSemanticEntityParser and org.apache.nlpcraft.examples.lightswitch.nlp.entity.parser.NCFrSemanticEntityParser