Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
  • Aug 24 23:19
    prkumar edited #172
  • Aug 24 23:18
    prkumar edited #172
  • Aug 24 23:14

    prkumar on master

    Fix broken Travis CI build The… (compare)

  • Aug 24 23:12
    prkumar edited #172
  • Aug 24 23:12
    prkumar opened #172
  • Aug 24 23:01
    prkumar commented #170
  • Aug 24 23:00
    prkumar commented #170
  • Aug 24 22:59
    prkumar commented #170
  • Aug 24 22:57
    prkumar labeled #170
  • Aug 24 21:24

    prkumar on master

    Fix broken Travis CI build Bui… (compare)

  • Aug 21 09:44
    asmodehn closed #171
  • Aug 21 09:44
    asmodehn commented #171
  • Aug 21 09:24
    asmodehn opened #171
  • Aug 21 09:19
    asmodehn opened #170
  • Aug 18 17:58

    prkumar on master

    Fix uplink.Url by checking url … Update arguments.Url to use uti… Build relative URL behind Reque… and 1 more (compare)

  • Aug 18 17:58
    prkumar closed #165
  • Aug 18 17:58
    prkumar closed #164
  • Aug 18 17:57
    codecov[bot] commented #165
  • Aug 18 17:56
    codecov[bot] commented #165
  • Aug 18 17:55
    codecov[bot] commented #165
myslak71
@myslak71
class = CustomerSchema(Schema):
id = fields.Integer()
name = fields.String()
customer_type = fields.String()

class MyClass(Consumer):
@json
@post('/sort_clients')
def filter_clients(self, customer_type: Field) -> CustomerSchema:
What I'd like to achieve is to pass customer_type value to the CustomerSchema because it is not returned by the /sort_clients endpoint and I would like to include somehow
P. Raj Kumar
@prkumar

@myslak71 - Unfortunately, there isn't an easy way to do this. As of v0.9, the simplest way is to wrap the request method and add the customer_type value to the return object. The caveats here are: (1) the CustomerSchema won't get a chance to validate the customer_type value, and (2) if you want your client to work with both sync and async I/O, this approach may make that more difficult to achieve:

class = CustomerSchema(Schema):
    id = fields.Integer()
    name = fields.String()
    customer_type = fields.String()

class MyClass(Consumer):
    @json
    @post('/sort_clients')
    def _filter_clients(self, customer_type: Field) -> CustomerSchema:
        pass

    def filter_clients(self, customer_type):
        customer = self._filter_clients(customer_type)
        customer.customer_type = customer_type
        return customer

Let me know if this works with your use case! If you'd like to see better support for this , we could log an issue on GitHub :smiley:

myslak71
@myslak71
I had already come up with a solution you have provided :) but thanks!
P. Raj Kumar
@prkumar
@myslak71 no problem! :)
Jacob Floyd
@cognifloyd
What order are annotations appllied? ie I have a class level annotation that raises errors when status >= 400. But, I just discovered that one of the APIs also returns 404 when it should be returning an empty list, so I want to check for 404 in a method-level annotation before the class-level 400+ check happens.
Jacob Floyd
@cognifloyd
Look at this ugly handler:
@response_handler
def returns_empty_list_on_404(response):
    # ugly hack to handle stupid api
    if response.status_code == 404:
        response.status_code = 200
        response._content = '[]'
    return response
P. Raj Kumar
@prkumar
@cognifloyd - Currently, class-level annotations run before method-level annotations. Seems like this is related to #120: we want to provide a way for users to override the order, such as in your case above.
Jacob Floyd
@cognifloyd
Does it work to have two different @retry configs?
ie one at class level to retry three times for 5xx errors on all endpoints + another retry on a special method that has much more aggressive retry characteristics?
Jacob Floyd
@cognifloyd
Ah. I think I misidentified my problem, because I dropped the class level retry, and it was still not seemingly using my method-level retry.
I want to raise an exception in a @response_handler to trigger the jittered exponential retry of the request.

But retries run before the response_handlers...

Response and error handlers (see here) are invoked after the retry condition breaks or after all retry attempts are exhausted, whatever comes first.
https://uplink.readthedocs.io/en/stable/user/quickstart.html#retrying

Jacob Floyd
@cognifloyd
    @retry(
        when=retry.when.raises(FutureAgain) | retry.when.status_5xx(),
        stop=retry.stop.after_delay(240),
    )
    @returns_results_or_future
    @get(args=(Url,))
    def _check_async_results(self, location):
        pass
And the FutureAgain exception gets raised in @returns_results_or_future - ultimately, it is inspecting the json results to see if the transaction completed (return results) or not (raise FutureAgain).
@prkumar So, I guess I'm trying to move the retry logic from your Pizza sample's while True into @retry because hounding the service so persistently caused issues.
# Poll until order is successful
while True:
    if pizza_client.is_order_ready(order):
        # Order was created!
        break
Jacob Floyd
@cognifloyd
Hmm. It looks like I need to move the inspection out of the response_handler and into a should_retry_after_response method on a new RetryPredicate.
Am I going down the rabbit hole on this one?
Jacob Floyd
@cognifloyd
Success!
That worked. OK - well, sorry for the late night spam ;)
It helped to explain it even if no one was listening :)
P. Raj Kumar
@prkumar
@cognifloyd nice!! Sorry that I’m late to the action, but I’m glad that you were able to resolve this! From my understanding of the problem, this may be relevant to #162; what do you think?
Jacob Floyd
@cognifloyd
Yeah. I didn't notice for a couple weeks that my retry wasn't really doing anything. Seeing counts/metrics would have been helpful in figuring out why my requests were apparently timing out (I assumed the retry stop condition was met).
Making the retry easier to debug sounds superb :)
Max Davila
@MaxDavila
Hi, is there a way to enable global debug logging for the library?
Jacob Floyd
@cognifloyd
Is there a way to do validation on a path param before making a request? ie something like an enum where the path param has to be one of two strings.
Or Carmi
@liiight
@MaxDavila there aren't any logs in uplink yet, I think theres an issue on that
@cognifloyd maybe you should handle that with marshmallow or pydantic and have uplink load that object into the path? I think that's possible
I think you can set Path(type=PathEnum) or something of the likes. Not sure though
Ratul Minhaz
@ratulotron
Hi guys! Do you have any idea how I can forcefully insert few keys in the request body? I am making a Pocket client for my Django project and it seems I need to send the "consumer_key" and
"access_token" each time with the post requests.
Max Davila
@MaxDavila
Has anyone ran into any resource problems using this in a medium throughput env? ~200 external rps. Looking at the source and issues, I don't see anything that jumps at me since it looks we always reuse the same session. We shouldn't be running into problems like running out of file descriptors etc
P. Raj Kumar
@prkumar
@MaxDavila - what sort of resource problems are you running into (e.g., is it a possible memory leak)?
Max Davila
@MaxDavila
@prkumar sorry I wasn't clear. This is an exploratory question. We are not experiencing any problems since we are still trying to decide whether to use this library or not. Just trying to see if anyone here has run into any resource related problems. The most common one would be running out of file descriptors because of not closing connections but I am not seeing that being a problem since we always reuse the session. We have experience using retrofit and really enjoy this pattern so it'd be nice to integrate it in our stack moving forward.
P. Raj Kumar
@prkumar
@MaxDavila Thanks for clarifying! Yes, keep-alive connections should mitigate such failure modes as long as you reuse the session across Consumer objects. And, when the session object is destroyed, it should close any remaining open connections, too; this is all standard stuff that should be handled directly by the underlying HTTP client you are using with uplink (e.g., requests or aiohttp). Glad to hear that you're considering adding uplink to your stack. Let me know if you have any concerns or questions that come up. Happy to help where I can.
Max Davila
@MaxDavila
awesome, thanks @prkumar!
manzella
@manzella_gitlab
hi all! i'm wondering how to make posts through a proxy service using uplink. i see we can use MultiAuth to auth on a proxy service but when actually defining endpoint calls, is there a predefined way to make those calls using the proxy?
Sérgio
@sergioisidoro
Hello :) I was wondering if there's any way to somehow sign / checksum the request made with uplink. I was looking at the inject method, but I'm unsure if that's the way to go...
Sérgio
@sergioisidoro
Specifically, something to the likes of this: https://github.com/pyauth/requests-http-signature
P. Raj Kumar
@prkumar
@manzella_gitlab - Hi! If I understand your question correctly, you need Uplink to target your proxy service instead of your backend? If your proxy exposes the same API as your backend, then you should be able to define you client by following the quickstart and just point the client to your proxy by setting the base_url appropriately.
P. Raj Kumar
@prkumar
@sergioisidoro - Hello! One way to accomplish what your want is to provide uplink with a custom requests.Session
Here's a sample snippet that should work (haven't tested it):
import requests
from requests_http_signature import HTTPSignatureAuth

preshared_key_id = 'squirrel'
preshared_secret = 'monorail_cat'

session = requests.Session()
session.auth = HTTPSignatureAuth(key=preshared_secret, key_id=preshared_key_id)

# You uplink defined consumer
api_client = MyApi(BASE_URL, client=session)
^ This is assuming that you want to use the auth on all requests
Sérgio
@sergioisidoro
Thanks a lot! I'll try this out
btw, Thanks a lot for Uplink. I'm basically using it in every personal project I do :)
P. Raj Kumar
@prkumar
Welcome! Glad to hear that you're finding the library useful :)
AlexV
@asmodehn
Hi, I just discovered uplink, after doing a quick research, because I was thinking of building something similar :) so thanks for the work !
I had however a question/concern... I am a fan of https://sans-io.readthedocs.io/ approach, and I think REST API fit in this category, so I am a bit concerned to see that uplink embeds various kinds of IO stuff in it.
My main concern is about testing, like what if I want to test my client implementation against mock data like with vcrpy, how is that doable ?
AlexV
@asmodehn
Well I ran some tests and vcrpy plugs itself in aiohttp, so things seem to work fine for my usecase so far... I'll be playing with that for the next few days/weeks ;-)
P. Raj Kumar
@prkumar
@asmodehn - Thanks for the link to the sans-io docs! I agree that this is the right approach here, too. For the most part, because uplink needs to work across different HTTP Clients, we try to decouple much of the underlying I/O already: the uplink.clients.io subpackage exposes abstractions for the different I/O models (i.e., asycio, twisted, and blocking I/O), then the uplink.clients subpackage provides an adapter abstraction for the HTTP client (i.e., requests, aiohttp, and twisted), which in turn leverages an appropriate I/O implementation.
I think we have a lot of room for improvement, though! For one, our abstractions for I/O and the HTTP clients can definitely be improved on by leveraging the learnings of the Sans I/O approach you outlined. I'll create a GitHub issue to capture the additional work that needs to be done to align ourselves with Sans I/O. We also have an existing GitHub Project for uplink that may be suitable for this effort. Thanks again for the tip!