Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
    Jordi Masip
    @masipcat_gitlab
    @vangheem can you port https://github.com/plone/guillotina/pull/811/ to master?
    Nathan Van Gheem
    @vangheem
    @masipcat_gitlab in master, ahh you want default to be True again?
    Jordi Masip
    @masipcat_gitlab

    @masipcat_gitlab in master, ahh you want default to be True again?

    Yes, I'm not confident that our application will continue to work the same way with this change (I haven't defined the required=true explicitly on all fields)

    @vangheem ^
    Nathan Van Gheem
    @vangheem
    sounds good! I'll try and do it today!
    Jordi Masip
    @masipcat_gitlab
    thank you :)
    Nathan Van Gheem
    @vangheem
    @masipcat_gitlab @bloodbare okay rescheduling guillotina meeting to next week? I'm pretty busy this week...
    Ramon Navarro Bosch
    @bloodbare
    @vangheem sounds good
    Jordi Masip
    @masipcat_gitlab
    @vangheem yess
    Nathan Van Gheem
    @vangheem
    thank you!
    Jordi Collell
    @jordic
    @vangheem @bloodbare there is a big braking change between G4 and G5, that affects mostly guillotina_elasticsearch: On elastic catalog.query was used as default raw query method, but on G5, the catalog.query was used to build querys (following the catalog api)... That makes super hard to be able to provide compativility for guillotina_react...
    guillotina_elasticsearch (it's absolutly broken.. es7, query (broken), anyway, thought anyone is using it.
    We need elasticsearch on guillotina_react
    pgcatalog doesn't scale.. (also we are planning to move some more objects (around 1M to guillotina) and obviously pgcatalog will not scale..
    We need to discuss a bit, how we can solve the situation. I can work moving elasticsearch and the apis.
    That's one side.
    On the other side, I'm considering a fork of elastic
    Jordi Collell
    @jordic
    We need something simple, without the complexity of indexer tasks, and all those bits. just an event that indexes an object, (mostly, it sends to queue to later index it).
    amyway, all these bits around indexer tasks, are on core (that doesn't make any sense)
    Just to try to explain it a bit better: If I put an object on guillotina, I need to know if the indexer had failed)
    And I can retry it on the client side.
    There is also an issue with backpressure..
    There is no proper way of measuring how busy is your event loop, neither how many tasks are waiting, and how many job is sending to elastic a guillotina instance.
    On the end, a "blocking" request, waiting elastic, is a nice way of handling backpressure.
    Jordi Collell
    @jordic
    What do you think about it? @vangheem @bloodbare @masipcat_gitlab ?
    Ramon Navarro Bosch
    @bloodbare
    @jordic i think its necessary to provide an out of the box sync pg catalog experience. For extra async/queue/es solutions guillotina_elasticsearch its the solution. About the API, i think that the proposal i did fullfills most of use cases so it can be used for search. On top of this what is the exact issue you are pointing? Guillotina_react should talk the search API that is deliverable by pg catalog and es
    Jordi Collell
    @jordic
    that the api from G4 to G5 had been broken
    Ramon Navarro Bosch
    @bloodbare
    In order to provide ensurance that something has been indexed, u can use Nats-streaming (what i use at flaps.io) or kafka (there is a guillotina_kafka)
    Jordi Collell
    @jordic
    on G4, elastic was using the query method, to build RAW querys
    Ramon Navarro Bosch
    @bloodbare
    Tha api from 4 to 5 changed, its clearly defined. The search endpoint was not designed and we did it at g5
    Jordi Collell
    @jordic
    Yes I know...
    also I can use rabbitmq
    but, on the end, all these wired bits around indexers, tasks, and other are there, without not making too much sense
    Ramon Navarro Bosch
    @bloodbare
    If u need an endpoint for raw es queries u can add but its not an out of box solution if u need es
    Jordi Collell
    @jordic
    that's not the problem
    the problem is that we have ES working
    but with the semantics of G4
    and right, it's so hard to move forward...
    Ramon Navarro Bosch
    @bloodbare
    I dont understand what is hard
    Btw, rabbit is not a streaming system, insnt it?
    Jordi Collell
    @jordic
    we don't need a streaming sustem, we will not do eventsourcing...
    so sorry, it's too much hard to evolve...
    no worry, we can fix these for ourselves :)
    but thought the idea of Opensource is to build things around colaboration
    Ramon Navarro Bosch
    @bloodbare
    Jordi what do u propose exactly?
    Jordi Collell
    @jordic
    No worry, it's just that when things are too much hard, sometimes doesn't make too much sense..
    We can just go with python and elastic, and this is enought
    Jordi Collell
    @jordic
    On the end, for guillotina react, perhaps the easy path is to just go with the POST @search to elastic
    Ramon Navarro Bosch
    @bloodbare
    @masipcat_gitlab what about a call to discuss #852 ?
    Jordi Masip
    @masipcat_gitlab
    @bloodbare sound good. I'll call you in about 20 min