Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
  • Jan 31 21:20
    sigmavirus24 commented #1531
  • Jan 31 21:17
    sethmlarson commented #1531
  • Jan 31 18:55
    sigmavirus24 commented #1531
  • Jan 31 18:44
    sethmlarson commented #1531
  • Jan 31 18:34
    sigmavirus24 commented #1531
  • Jan 31 18:27
    sethmlarson commented #1531
  • Jan 31 18:26
    sethmlarson commented #1531
  • Jan 31 18:26
    sethmlarson commented #1531
  • Jan 31 18:24
    sethmlarson commented #1531
  • Jan 31 18:00
    sethmlarson commented #1531
  • Jan 31 15:52
    sigmavirus24 commented #1531
  • Jan 31 15:32
    codecov-io commented #1531
  • Jan 31 15:32
    sethmlarson synchronize #1531
  • Jan 31 15:31
    codecov-io commented #1531
  • Jan 31 15:31
    sethmlarson synchronize #1531
  • Jan 31 15:00
    sethmlarson commented #1531
  • Jan 31 14:54
    sigmavirus24 commented #1531
  • Jan 31 14:52
    sigmavirus24 commented #1531
  • Jan 31 14:49
    sethmlarson commented #1531
  • Jan 31 14:49
    sethmlarson commented #1531
Quentin Pradet
@pquentin
@SethMichaelLarson ok, thanks for the info!
Nathaniel J. Smith
@njsmith
When lines in global scope aren't covered, that usually means that someone is importing your package early, before coverage starts running
If you're using pytest + pytest-cov, this could happen if some other plugin imports urllib3 before pytest-cov starts (and unfortunately pytest imports plugins in arbitrary order)
Seth M. Larson
@SethMichaelLarson
Coverage is reporting 100% coverage on urllib3/__init__.py though, would that scenario have that behavior as well?
Nathaniel J. Smith
@njsmith
I dunno, I was just going off of
if you look at the downloaded coverage files you can see that the line 30 of urllib3/init.py is actually not listed as covered.
Which made it sound like you don't have 100% coverage on urllib3/__init__.py :-)
Seth M. Larson
@SethMichaelLarson
Haha I think there;s been a little misunderstanding because of the module being named "coverage" :P basically coverage report -m reports the file as 100% covered but codecov reports the file as not 100% covered with global scope being marked as missing
Quentin Pradet
@pquentin
@SethMichaelLarson thank you for the travis + 3.8 fight. :)
Zach H
@ZeldaZach
Hi, all!
I believe I found a bug with the library, but didn't seem to find it on GitHub. Thought I'd raise it here to see if I was just searching poorly
I create a session, multiprocess a function, and call the session within those processes. Results in a Connection broken: IncompleteRead(0 bytes read)', IncompleteRead(0 bytes read) error from requests which calls to urllib3. A crashlog can be found at https://pastebin.com/zXfGQc4S
Nathaniel J. Smith
@njsmith
Sharing a session across multiple processes is definitely not supported
Zach H
@ZeldaZach
Is it a bug, an error on my part, or something not defined yet?
Nathaniel J. Smith
@njsmith
Just make a new session inside each process
How are you passing the session object between the processes? It's possible the library could catch things at that point and give a more informative error, but that's not always possible either depending on how you're using multiprocessing
Zach H
@ZeldaZach
I'm using contextvars to access it
Nathaniel J. Smith
@njsmith
So you set the contextvar in the parent, then access it in the child, and you're doing this on Linux/macOS?
Zach H
@ZeldaZach
Correct
And the parent spawns 8 children, if that matters
When spawning one child, it doesn't have any negative effects
Nathaniel J. Smith
@njsmith
yeah, in that case the object is getting copied by fork(), so it's not easy for urllib3 to even detect what's happened
but this is definitely not intended to work, and trying to make it work would probably just make urllib3 more fragile without actually being useful. Just create a new session inside each process.
Zach H
@ZeldaZach
Thanks for the help, appreciated!
Boris Staletic
@bstaletic
Hello, is this room active? I'd like to ask for some help with moving away from requests in favour of urllib3.
Zach H
@ZeldaZach
Really now? What’s wrong with Req
Boris Staletic
@bstaletic
We're using requests as a git submodule. Recently we had to update it and, since requests isn't vendoring its dependencies any more we had to include urllib3, chardet, idna and one more thing in our submodules.
Then it completely broke, because of the leftover *pyc files and we had to rearrange submodules to make sure that when a user runs git pull it invalidates python's *pyc, because requests wanted to be "clever".
Finally, everything works now, but if a user wants a shallow clone, even though it is not officially supported, due to storage saving, requests emits a warning when sourcing its __init__.py, creating a terrible trace back that explodes in user's face, because the urllib3 and chardet are newer than requests expects.
So dropping requests and just using urllib3 would be nice for the following reasons:
  • Shallow clones work.
  • Less dependencies, less stuff to do on start up, faster startup
    • Vim users are crazy about their startup times, so shaving off a couple of milliseconds would be praised
Quentin Pradet
@pquentin
well, it really depends on what you're doing with requests -- for simple requests, you should be fine
sorry if that's not helpful, but... what's wrong with installing requests/urllib3 the normal way?
Boris Staletic
@bstaletic
By normal way, you mean using pip? That blew up in our face when we tried to use regex module. Apparently, tons of users don't have pip installed.
I've tried asking reddit for help, but got no response. There should be all information that is relevant to my issue. If something is missing, please ask.
https://old.reddit.com/r/learnpython/comments/ar06wc/help_moving_away_from_requests_in_favour_of/
Zach H
@ZeldaZach
I figured pip is default for most
Boris Staletic
@bstaletic
For example, Ubuntu 16.04, apt install python-pip is necessary for pip, but then it also doesn't install wheel, so pip runs into some weird errors and users also need apt install python-wheel. In general, linux distros ship pip as a separate package and we were flooded with issue reports when users didn't have a working pip available.
Quentin Pradet
@pquentin
I see, that makes sense
Boris Staletic
@bstaletic
macOS also ships pip as a separate package, I think. That's why we shy away from assuming pip is available on users' machines.
Zach H
@ZeldaZach
I wish I knew why they split them up
They feel bundled to me
Boris Staletic
@bstaletic
That's what we assumed when we tried to use pip to install regex into our local directory. :/
Zach H
@ZeldaZach
Pre install it?
Boris Staletic
@bstaletic
What exactly do you mean by that? We are using git submodules, so a git submoudle update will fetch the sources of all our dependencies. This works most of the time, but occasionally, a mess like requests update comes.
Boris Staletic
@bstaletic
Anyway, if I want to use urllib3, is this the correct patch to apply?
@@ -212,11 +212,12 @@ def _Request( self, method, handler, data = None, params = None ):
     headers = self._ExtraHeaders( method,
                                   request_uri,
                                   data )
-    response = requests.request( method,
-                                 request_uri,
-                                 headers = headers,
-                                 data = data,
-                                 params = params )
+    if not PY2:
+      request_uri = ToUnicode( request_uri )
+    response = urllib3.PoolManager().request( method,
+                                              request_uri,
+                                              headers = headers,
+                                              fields = data if data else params )
     return response
Mujahid Alex
@mujahidalkausari
Hey guys
Good morning #I'mNewHere
Got an error
need support
I'm working on urllib3
raise MaxRetryError(_pool, url, error or ResponseError(cause))
MaxRetryError: HTTPSConnectionPool(host='authoraditiagarwal.com', port=443): Max retries exceeded with url: / (Caused by SSLError(SSLError("bad handshake: Error([('SSL routines', 'tls_process_server_certificate', 'certificate verify failed')],)",),))
Please check the above error and let me know the cause
Thank you