Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
    Aaron Christianson
    @ninjaaron
    :thumbsup:
    Kurt Griffiths
    @kgriffs
    it's good feedback... terms can mean different things to different people and perhaps we can tweak our language a bit
    I'm probably dating myself, but "bare metal" used to be used commonly with software that was lower level / fewer layers between the software and the machine.
    but it is used commonly these days now to mean non-virtualized servers, so that is at least one direction the confusion might be coming from
    ianbeck
    @ianbeck
    Hey Falcon folks; any tentative idea about how far out the 3.0 release and ASGI support likely is (days, months?)? My employer is going to be pivoting to an ASGI server in the near future and I was very intrigued to see that ASGI is coming to Falcon because you're one of the more stable and responsibly-developed frameworks around as far as I can tell (Starlette/FastAPI look really good feature-wise, but are quiet young and I'm not confident I can look my supervisor in the eye and promise that they are likely to be supported long-term).
    Nick Zaccardi
    @nZac
    Hi @ianbeck! Thanks for the compliments it is really humbling to think people like what Falcon represents and all the hard work the team has put in. The short answer is "it will ship when it is ready" which is incredibly unsatisfying I am sure. There are a host of PR's for 3.0, but the ASGI one can be found here: falconry/falcon#1573. Review comments / other eyes would be really helpful in moving the backlog along, that list has really grown. We have been making a push to get the ball rolling for 3.0 and ASGI is the big hitter for that (along with multi-part). Please pull it down and take a look if you like what you see, we would love to hear how you are using it.
    Kurt would be the person to answer how far away 1573 actually is... my attention has been on some of the other PR's in the backlog.
    ianbeck
    @ianbeck
    Hey @nZac; no worries, I know how big releases typically work. :-) I've been keeping an eye on the ASGI PR for a few weeks but haven't tested it out or tried to do any code review because I don't have any prior experience with Falcon so wasn't sure I'd be able to provide any actually-useful feedback. Is there a particular branch or similar where the 3.0 work is consolidating, or is this stuff being rolled out into master as it completes?
    Nick Zaccardi
    @nZac
    Thanks for understanding! So far as I know #1573 has everything... if I am missing something, hopefully someone else can correct me. I know at one point there were gists floating around that had example code, but I believe those were for earlier renditions.
    ianbeck
    @ianbeck
    Okay, I'll try to make some time to play with it and see if I can offer any constructive feedback.
    Nick Zaccardi
    @nZac
    Thanks Ian! Here is the bulk of the code (and the branch for the PR) if you just want to give it a cursory glance to start: https://github.com/kgriffs/falcon/tree/asgi-final/falcon/asgi
    "bulk" is probably a bad word there considering that PR hits 90+ files :)
    We do have a falconry/dev channel which is a good place to discuss the details of that PR, should you want to discuss outside the formal PR.
    ianbeck
    @ianbeck
    Sounds good!
    Kurt Griffiths
    @kgriffs
    Thanks @ianbeck. The more eyes we get on this the better. We'd like to cut an alpha as soon as we can land the form and ASGI PRs. The ASGI PR does not have much in the way of docs (yet) but you can see this gist for usage examples: https://gist.github.com/kgriffs/4f99da6dde2266201ddddc42784e5aee
    Also, I haven't benchmarked anything yet but we've written it to be lean and mean, so I have hopes. :)
    We've been trying to get caught up with some other things in the PR backlog but I hope to soon circle back on that PR and get it rebased and ready for final review.
    ianbeck
    @ianbeck
    Is there a best practice for handling SQLAlchemy sessions with Falcon? I found a suggestion for using a middleware that stores the instantiated session object in req.context but that's a little conceptually weird (database access has nothing to do with a request). Particularly given that I'm working with an ASGI instance at the moment, I'm not sure what else would be safe, though
    Kurt Griffiths
    @kgriffs
    @ianbeck for per-request sessions, req.context would be a reasonable choice. In a way, the session is associated with a given request, but I can understand that not everyone might see it that way. Alternatively you could use thread-local storage I suppose, but that arguably makes session lifetimes a tiny bit harder to reason about. There is also another school of thought that recommends not using a per-request session at all, but simply setting up and committing your transactions as locally as possible to the associated business logic. It can be a bit more tedious but also may make it easier to track down errors if/when they occur. I know @jmvrbanac is a fan of this last approach and may have some thoughts.
    Nick Zaccardi
    @nZac
    @ianbeck I would highly suggest using a scoped session. See the docs here: https://docs.sqlalchemy.org/en/13/orm/contextual.html#contextual-thread-local-sessions. I have not personally done anything async with SQLAlchemy, so your mileage may vary there. For WSGI though, scoped_session should do everything you need. Storing it on the context is reasonable, but you can also make the session a global and use it from there by importing it. The docs use the example of a web request as the basis for the scoped session.
    ianbeck
    @ianbeck
    @nZac Yeah, I was looking at scoped sessions, but wasn't sure what the implications of having a single session per thread would be, particularly in the context of an ASGI architecture. Our particular application communicates with IOT devices, so it's seems more likely to me for weirdness to occur if session objects are shared (say an endpoint is accessed that results in device communication; while that's awaiting, the thread goes to work on a second request for the same device that results in another IOT message. If one of the IOT communications times out but the other completes, I can imagine pending changes to the device SQLAlchemy object getting flushed to the database from a non-related request if they are not committed prior to contacting the IOT device in the other request. That would be a very difficult bug to identify and fix, and one that wouldn't get caught by automated testing)
    Vytautas Liuolia
    @vytas7
    @ianbeck I'm no expert on ASGI, but my understanding would be that thread-local scoped session is incompatible with things async. You probably need to use the async flavour of Sqlalchemy altogether.
    ianbeck
    @ianbeck
    @vytas7 Good to know there are likely issues with scoped sessions and async. As far as I'm aware, there is no async version of SQLAlchemy at the moment (barring things like GINO that use SQLAlchemy Core and eschew the ORM), though if you have a link I'd love to be proven wrong; I'm currently planning to just use SQLAlchemy synchronously because for us database communication is practically instantaneous, while IOT commands can sit idle for 15-30 seconds
    Vytautas Liuolia
    @vytas7
    I saw something on PyPi, but as I've never used that myself, no clue how legit that is :slight_smile:
    ianbeck
    @ianbeck
    Ha, fair enough
    Kurt Griffiths
    @kgriffs
    Good catch. I wasn't thinking about async when I mentioned thread-local
    Vytautas Liuolia
    @vytas7
    Kurt, you remember Flask's David Lord was telling they were probably going to tread carefully when it comes to async, as their thread local stuff actually needed to be async local, or coroutine local, w/e the proper expression is. And that comes only in 3.7 or so.
    Kurt Griffiths
    @kgriffs
    Yeah, that's right. There is an async flavor there that is the equiv. of thread-local
    It's all evil though. ;)
    Vytautas Liuolia
    @vytas7
    Programming is always easier with global variables -- you have access to anything, any time :see_no_evil:
    ianbeck
    @ianbeck
    I'll probably stick with automatically storing and cleaning up an instantiated session in req.context via middleware for the moment. It's a little weird (since the database session isn't coming in with the request), but req.context.session isn't that much worse than self.session or similar in practice
    Vytautas Liuolia
    @vytas7
    I think it is not unreasonable to think that your session is actually bound to request, if your api is database-centric.
    @ianbeck you might also have issues with non-async sqlalchemy blocking I/O if it is using non-cooperative sockets. But I'm probably out of date, don't know what people use for that purpose. In the gevent world, you could at least monkey patch.
    ianbeck
    @ianbeck
    Thanks for the heads up; I'll do a little digging
    Vytautas Liuolia
    @vytas7

    @ianbeck Re the stuff I've seen -- at least this seems to be legit https://github.com/encode/databases . From the same authors as Starlette etc. But that gives roughly the equivalent of SQLAlchemy Core, so probably it's the same thing you mentioned

    barring things like GINO that use SQLAlchemy Core and eschew the ORM

    Hmm...

    GINO - GINO Is Not ORM

    At least they get some bonus points for using a recursive acronym :wink:
    ianbeck
    @ianbeck
    Yeah, I've experimented with encode/databases and I wasn't terribly impressed. It's super young, so it's not particularly well polished and has some inconsistencies. Plus at that point you're basically writing SQL lightly translated into Python, so the only benefit from SQLAlchemy we'd be getting is Alembic migrations. Given our architecture, it would almost make more sense to just go bare metal, write the SQL directly without worrying about model classes (or table definitions, really, since it's just Core), and leave migrations to be handled by the Django monstrosity that we're trying to migrate away from
    But I'm not quite that much of a masochist... :-D
    jakegatsby
    @jakegatsby
    Hi. Just wondering if there's a way to send the response and then continue with normal python code in an on_get method. My scenario is I have a long running endpoint that I simply want to reply with a job ID and then continue processing on the worker.
    Vytautas Liuolia
    @vytas7
    @jakegatsby I think there will be something like that in the upcoming ASGI variant. For WSGI, you should spawn a thread (or a greenlet, if you are using gevent), or fork a subprocess, or, conversely, have a worker process or thread/greenlet running all the time, and grabbing jobs from some form of queue.
    ianbeck
    @ianbeck
    @jakegatsby I don't know if the setup complexity would be worth it (particularly if you only have one background task you ever need to run), but Celery is fantastic for handling this sort of thing. Otherwise, yeah continuing to process in the same thread after returning is only possible under ASGI (though I don't know if it's specifically supported in the initial ASGI rollout for Falcon 3.0 or not)
    Vytautas Liuolia
    @vytas7
    I don't think it is too strange to spawn follow-up processes/threads/greenlets in the WSGI world either though. We use that in some services at work. Furthermore, the ubiquitous uWSGI even has mule functionality (i.e. you spawn some workers which do not handle requests, but only do background processing), although I have never used that in production setting though.
    One needs to be mindful how to control the lifetime of those extra threads, so yes, if the whole thing is growing in complexity, something like Celery (or another async task system) is worth investing in.
    ianbeck
    @ianbeck
    Agreed, spawning a new process or thread makes total sense in a lot of simple cases and is the only way to handle this under WSGI that I know of, outside of workers outside the web process
    Hrobjartur Thorsteinsson
    @thorsteinssonh_gitlab
    hi, it looks like req.get_param_as_list('myparam') has changed between 1.4 and 2.0 version
    I cant find any discussion on it, basically I have things like this myparam=(something),(other) and I used to get [something,other] out of this op
    now I just get ['(something),(other)']
    Hrobjartur Thorsteinsson
    @thorsteinssonh_gitlab
    ok found the issue now, its been reported, falconry/falcon#1547
    dodumosu
    @dodumosu
    hi. is there a recommended Falcon/SQLA boilerplate i can study? a couple of the links on the wiki are a bit dated