Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
    Luke Hollis
    @lukehollis
    The simplest possible solution (maybe not optimal?) that I could imagine if we wanted to keep sorting on the keys would just be to do something like this:
    {  
          "1":{
                        "speaker" : "Actor 1",
                         "text" : "This is the first line"
           },
          "2":{
                        "speaker" : "Actor 2", 
                         "text" : "This is the second line"
          },
          "3":{
                        "speaker" : "Actor 2", 
                         "text" : "This is the third line"
          },
    }
    Luke Hollis
    @lukehollis
    But, we can ingest data from either data format--whatever makes the most sense for the document conversion.
    When we move things to Text objects in the database, I think we'll just have to add a speaker property that we can use across genres when appropriate
    I was thinking about this back in the day with the speakers in the Eclogues
    Sameer Chaudhari
    @sameeriitkgp
    I agree with Luke.
    Sameer Chaudhari
    @sameeriitkgp
    @kylepjohnson -- If you agree, we can create an issue for this actor-line
    I think I can take care of the document conversion and the database part.
    What do you think @lukehollis ?
    Kyle P. Johnson
    @kylepjohnson
    @SameerIITKGP and @lukehollis I agree, too. I like you solution better much than mine
    @SameerIITKGP I think you can go for it. I thank you both for being so helpful. I knew this would be a problem and had avoided it
    Luke Hollis
    @lukehollis
    sounds good! I think we should just add the speaker parameter to the Text objects collection schema and add the check for ingesting "actor-line" documents in the text sync
    Kyle P. Johnson
    @kylepjohnson
    Wonderful! Seeing drama in the frontend will be powerful
    Luke Hollis
    @lukehollis
    totally! that'll be good
    Was wondering about involving the cltk core more in the API
    So when you GET api.cltk.org:5000/core/stemmer?input="quid faciat laetas segetes quo sidere...etc."
    it will run something like this:
    def stem_input(input):
       s = Stemmer()
       return s.stem(input)
    and you will get back "Quid fac laet seget quo sid ... etc."
    Seems like pretty low-hanging fruit if we could keep it simple.
    Idk about the path /core/stemmer... maybe we could just make that match the directory structure of the cltk core package
    Luke Hollis
    @lukehollis
    This is in my head as I look more at getting scansion and text reuse on the frontend
    Kyle P. Johnson
    @kylepjohnson
    @lukehollis In and of itself, this isn't wrong.
    If you are OK testing out some ideas now in the API and don't mind if it gets redone, definitely go for implementing the routing you're thinking of.
    I think I'll need to sit down with pen and paper to come to a good API proposal that will be flexible enough for all processing and all texts
    Comment/criticism about your URI in particular: I think we'll be happier posting text as data objects, then calling the processing type in URI
    Kyle P. Johnson
    @kylepjohnson
    Because there will come a time when we could be posting very large data packets to the API
    Luke Hollis
    @lukehollis
    Ah sure, that sounds great! well, I'll create some issues for this
    makes sense with the long uris sometimes breaking things in the future
    Luke Hollis
    @lukehollis
    Okay, created an issue to explore the stemmer and cltk api so that we can start moving forward there: cltk/cltk_api#20
    Luke Hollis
    @lukehollis
    There have been a few requests for more comprehensive web application architecture diagramming, so here's what I propose:
    I'm not married to any of this, but I think that this solves the problem of leveraging the CLTK advanced NLP functionality to render data to the frontend interface.
    Kyle P. Johnson
    @kylepjohnson
    This looks very good
    I'll point people here too if they need
    Manvendra Singh
    @manu-chroma
    @lukehollis what is the use of mongoDB database here ?
    can you explain it's function briefly
    can't the meteor application query api in realtime and serve whatever is required rather than getting the whole set of data and storing it the DB
    Kyle P. Johnson
    @kylepjohnson
    @manu-chroma querying in real time might be too slow, so Luke is planning on pre-processing some text
    We may move away from this, but it's the best way to get started for us
    Luke Hollis
    @lukehollis
    Yes, that's all
    We have to keep the text somewhere so for each Text object created we can request definitions, commentary, translations, etc. from the API.
    It'd be awesome if we could just get it all at once!
    And in the future, we may be able to, but that seems like it would require the Flask API application also having a database
    Also, Meteor needs a Mongo database for User account, user notes/comments, and other things that are only on the frontend
    Manvendra Singh
    @manu-chroma
    And in the future, we may be able to, but that seems like it would require the Flask API application also having a database @lukehollis I had this doubt in my mind. Thanks for your response!
    Luke Hollis
    @lukehollis
    Ah, sure--well, I think there's a lot of ways that we can do it..
    Manvendra Singh
    @manu-chroma
    To retrieve data faster, we can use a very simple hack of querying various data from API async fashion. I don't know if that's implemented already in the frontend.
    Luke Hollis
    @lukehollis
    The nice part about only rendering from MongoDb is we can leverage the Meteor websocket DDP connection
    and then put our queries in our react .jsx templates
    Manvendra Singh
    @manu-chroma
    That sure will make site more responsive while the data is being fetched in real-time
    Luke Hollis
    @lukehollis
    yeah! and should be easy to program