Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
    giorgosp
    @giorgosp

    Hello.

    I have just found out about graceful and would like to learn more about it.

    I am already using Falcon for an API project (backed by database), however I am pissed that I have to reinvent the wheel and have already regretted not posing stronger objections against using it.

    I have ended up using sqlalchemy, marshmallow and some custom mixins for wrapping common get/post/... logic with db models but still there are many little things that are needed like pagination and, from a time efficiency perspective, I wish I won't have to implement them from scratch. I am happy however that I ve found out about Graceful and I considering using in it my project as well as contributing to it.

    If Falcon is to become usable, then frameworks like Graceful should emerge. On the other hand, if we add a lot of functionality on top on Falcon, won't this be against its “fast and lightweight” purpose?

    On a more technical note regarding Graceful specifically, may I ask what deterred you from using some ready package for serializing, like marshmallow? Or Serpy, which aims to be fast and only validation would need to be added.

    Also, what are your current thoughts about an ORM? Building a new one from scratch? SqlAlchemy?

    Maybe you wouldn't mind sharing some motives behind the development of Graceful.

    I hope this wasn't so long a message.

    Best,
    George

    Michał Jaworski
    @swistakm

    Hi George,
    At the beginning I need to start with explanation why I choosed Falcon over othes frameworks/micro-frameworks.

    reasons for falcon

    The first time I worked with falcon was few years ago at my first company. We have tried to use different tools to build both simple and complex APIs. We iterated over few popular solutions: django-tastypie, vanilla flask, Django REST framework. Each of them in different versions, also working on different versions of base framework (for Django it was everything from Django 1.2 to Django 1.7). We finally settled on falcon due to its simplicity and lightweighness lighweight. And its eficciency was never a pure reason we used it!
    Falcon's design simply makes applications cleaner and more readable.

    reasons for falcon - why not Django

    As many agree django is not the best solution for API-only applications or small services. Yes, DRF is great. If I had to find a reason to use Django the DRF would be at very top. It allows to make API design explicit and very modular. But after sometimes it started to become more tied to django models and this was already main drawback of TastyPie in the time DRF emerged. I don't follow the development of the latter project. Maybe people behind it solved main problems and make it usable again.
    Anyway Django does not encurage you to make really composable applications. Yes, there is a notion of apps but I have not seen many closed-source project that distribute every app they use as a separate packages (with setuptools, distribute). Also distributing whole package is usually great PITA in Django due to how settings are implemented and need to rely on management commands. Also often we had to make APIs that were not depending on classical RDMS that Django is so thightly coupled to.

    reasons for falcon - why not flask

    Flask is great for simple things. People build even complex software using it and I've seen few times how this usually ends. Usually in complicated codebases that are hard to maintain. The main conventions for flask are great: you keep routes close to views so it's easier to figure out how to reach resources. But way how views are implemented is very similar to old Django's way - as functions not classes. And it is not the best style for REST where every HTTP method has rather strictly defined semantics. Yes, there is flask.views.MethodView that is simmilar to falcon's style of building resources but it is still optional and you can't constantly keep an eye on every coworker that may break basic design principles of your API's code. Also I never understood how this magic access to request object in flask worked. But that's only because I did not want too. Anyway it is pretty scary.
    Maybe I'm nitpicking and flask is as good as falcon for REST style applications.

    reasons for falcon - composability

    I mentioned composability few times but did not explain what I understand by that. So now I will give example.

    Falcon resources are very basic custom classes that does not inherit from any falcon's base class. It's very Pythonic - if it has compatible interface it will work. This allows to keep every resources as single module or even package distributed through your own Python package repository (it's a good thing to have BTW).
    Use case where this approach really shines is APIs/services that are both exposed publicly and used internally. It's not uncommon that such APIs have some protected resources available only for internal services. The natural way (for some) is to implement additional application-level authorization layer that will restrict access. But why do this? Falcon way is way easier. Just deploy your project in two environments and for internal use register additional resources that are not registered for public API. You can now solve problem of access on network level withour increasing complecity of your app.

    Michał Jaworski
    @swistakm

    reasons for falcon - composability cont.
    API versioning is also easier that way. Why to keep codebase for old API versions (e.g. accessed with vX/ URI prefixes) when you can just distribute WSGI app object definition as versioned package with strictly set dependencies (to versioned resource packages) with setuptools. Code layout will be more flat and also you don't need to keep legacy stuff in latest developent code branch. If you need to serve multiple versions just deploy every required version of app and eventually add some additional glue with good reverse proxy like nginx. Your API does not even need to have any notion of version prefixes. This is a thing that should be handled by reverse proxy.

    why graceful

    As you already noticed - you need to do a lot by yourself. Parameter access/validation is very basic in falcon and it is hard to:

    • make it consistent in whole project
    • make it easy to document
    • do not repeat yourself

    The most important part of graceful are:

    • "documentability" through OPTIONS request
    • parameter classes

    Serializers classes and generic APIs are just nice addition but they don't need to be used always and ever. In the APIs we build in my company we very often use basic resource class with mixins but without serializers. It really depends on what you build.

    graceful/falcon and ORMs

    Yeah, this is not something I worked a lot with. If I had to choose something I would use SQLAlchemy. It's good and battle-tested. Falcon does not have any tools to integrate with ORMs but should it? With every ORM what you will probably do is using some session object that abstracts database connection. What you need to do is to make this session object available during request processing and unique for that request for thread/async safety. If you need this for every HTTP method in every resource you can easily create middleware class that will put it into Request.context object. In graceful this is trickier becasue you usually don't work directly with req/resp instances but the 0.3.0 allows to have context-aware resource classes. See #32 issue and #33 PR for more information.

    Michał Jaworski
    @swistakm

    graceful and Serpy/Marshmallow

    I have not used them but I heard something about Marshmallow. They may be good solution. I need to test them. Maybe they can be good replacement for built-in graceful serializers. This is good idea for 0.4.0 release or maybe 1.0.0 depending on complexity of required solution.

    Anyway, as I already said - serializers are optional. They are designed to make extending easy and also to encurage the "self-descriptive" style of resources. This is why you cannot create any field without details argument that is going to be included in OPTIONS response. The same rule is for param objects. I don't know how fast they are but they are so simple that I expect they are not slower than those of Serpy/Marshmallow. It is good idea to test/benchmark it.

    Michał Jaworski
    @swistakm
    I took a look on Serpy and no wonder it is so fast. It has nothing to offer. I don't see reason in using such kind of "serialization". Especially if it does not have any validation built in.
    giorgosp
    @giorgosp

    Hi Michal, thanks for the long reply. I like that you provided concrete examples like api versioning where the simplicity of Falcon would shine.

    For Serpy, its true that it does not offer much. I have suggested marshmallow because why reinvent the wheel? Also, there is the marshmallow-sqlalchemy package which does the integration with sqlalchemy, so it seems that the serialization and orm problems are solved for graceful.

    I couldn't find the time to read the graceful docs in detail, but I will consider utilizing grateful in my current falcon project. I currently use marshmallow and sqlalchemy but I am going to need pagination and authentication as well. I see that graceful supports pagination.

    As for authentication, here are some requirements that I ve compiled for my project without much thought:

    • login/logout resources. I need the user to send a username/pass to /login and get back a jwt token. But I guess this should be generalized if it is to implemented inside graceful.
    • Simple way to check inside resource a method whether a user is authenticated and ideally who that user is. This can utilize your concept of having a request context in each resource method, so the context can include some info about the current user provided by some middleware.

    These are some quick thoughts, perhaps I can help you with #17.

    Also, regarding falcon and graceful in general, I dont know if you see it this way, but I could see graceful become the goto framework for microservices in python. Falcon can also be run with gevent, so there will not be any argument that it cant utilize and event loop. Although the workers vs event loop debate will end, in my opinion.
    Michał Jaworski
    @swistakm

    That's great! Any thoughts on #17 will be useful. I think that key is to separate authentication from authorization. In one project of ours we had implemented an authentication layer as falcon middleware class that simply allows/denies access depending on request content (mainly headers but we have few ways to pass/store "credentials"). In our simple use case just authenticating the user (identifying him) was enough to implicitly grant access to all resources but we are now hitting situation where we need to tell which user/group has access to which resources/methods. Additional authorization layer on top of that would be great. Context aware resources are step towards this goal because allow the authentication layer to set user object in context that could be later retrieved in authorization layer. Still, I think that the best would be if we could find a nice way to make retrieve/list/create/etc. methods somehow compatible with falcon hooks (issue #31) . Right now you can only attach hook to whole resource.

    Regarding microservices: It would be great if people started using graceful to build them :)

    Regarding event loops: we at Opera TV already use graceful with gunicorn and eventlet. We had in past some issues with SSL with gevent in Python3 but it is possible that they were already fixed.

    Michał Jaworski
    @swistakm
    The login/logout resources would be specific for some authentication methods. For example in OAuth (in both versions) there must be an endpoint that grants access tokens. Some API creators might decide to use session in cookies (so login/logout maybe makes sense). Other users may not even need auth-related resources because they use simple authentication scheme with API keys in headers or GET params. I think that it would be hard to make something that is generic enough but maybe I'm wrong. It may be good idea to implement authentication layer in the simplest way possible that allows to easily customize backends/flows.
    Michał Jaworski
    @swistakm
    Generally the hardest part in designing such features is that we don't now anything about the application being created. Django app devs have easier life. They can just assume that there is RDBMS available and can create some models in ORM to store authentication data. User does not need to implement anything. He can just install and import new app.
    But we need to design pluggable interfaces that assume only as little as possible, and user will still have to implement his own storage backends and probably most of auth logic.
    Michał Jaworski
    @swistakm
    I made some progress in #17
    Michał Jaworski
    @swistakm
    ANNOUNCEMENT: New 0.3.0 version released.
    Michał Jaworski
    @swistakm
    ANNOUNCEMENT: New 0.4.0 version released.
    Michał Jaworski
    @swistakm
    ANNOUCEMENT: New 0.4.1 version released (changelog).
    Michał Jaworski
    @swistakm
    ANNOUCEMENT: New 0.5.0 version released (changelog)