Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
  • Jan 31 2019 19:45
    dcecile commented #503
  • Jan 31 2019 19:40
    blueyed commented #526
  • Jan 31 2019 13:29
    codecov[bot] commented #526
  • Jan 31 2019 13:26
    codecov[bot] commented #526
  • Jan 31 2019 11:59
    codecov[bot] commented #526
  • Jan 31 2019 11:59
    popravich synchronize #526
  • Jan 31 2019 11:59

    popravich on travis_pypy

    .travis.yml: chaching pip packa… Makefile: build/install only re… .travis.yml: cache redis-server… (compare)

  • Jan 31 2019 11:44
    codecov[bot] commented #526
  • Jan 31 2019 11:36
    codecov[bot] commented #526
  • Jan 31 2019 11:35
    codecov[bot] commented #526
  • Jan 31 2019 11:34
    codecov[bot] commented #526
  • Jan 31 2019 11:24
    codecov[bot] commented #526
  • Jan 31 2019 11:23
    codecov[bot] commented #526
  • Jan 31 2019 10:04
    gyermolenko commented #503
  • Jan 31 2019 09:31
    gyermolenko commented #431
  • Jan 31 2019 09:23
    Natim closed #444
  • Jan 31 2019 09:23
    Natim commented #444
  • Jan 31 2019 09:09
    gyermolenko commented #444
  • Jan 31 2019 05:32
    codecov[bot] commented #539
  • Jan 31 2019 05:32
    vir-mir synchronize #539
matrixbot
@matrixbot
wraptile posted an image: image.png
wraptile the docs here are incorrect as dict is invalid type
wraptile ValueError: dictionary update sequence element #0 has length 9; 2 is required
matrixbot
@matrixbot
wraptile man, client cookies are extremely confusing here 😐️
Justin Turner Arthur
@JustinTArthur
wraptile, we can’t see your screenshot on the Gitter side of your Matrix bridge unfortunately
matrixbot
@matrixbot
wraptile yeah I've noticed that as well 😅
wraptile what's the de facto way to serialize/deserialize session? For cookies it seems to be awfully complicated

wraptile ```
async def test_cookies():
async with ClientSession() as session:
session.cookie_jar.update_cookies(SimpleCookie('login=jogn; Path=/; Domain=httpbin.org'))
session.cookie_jar.update_cookies(SimpleCookie('pass=snow; Path=/; Domain=httpbin.org'))
session.cookie_jar.update_cookies(SimpleCookie('foo=bar; Path=/;'))
cookies = pickle.dumps(session.cookie_jar._cookies)
resp = await session.get('http://httpbin.org/cookies')
print(await resp.text())

async with ClientSession() as session:
    session.cookie_jar._cookies = pickle.loads(cookies)
    print(await resp.text())

```

wraptile this is kinda ugly, but I guess it should work fine? 😬
wraptile I see the bridge doesn't format the code snippet well either, ugh. Maybe now that Matrix bought gitter they can finally fix this
Justin Turner Arthur
@JustinTArthur
What is the serialization for? Are you sending this pickle down the wire to subprocesses?
matrixbot
@matrixbot
wraptile no I want the session to be persistent so that cronjobs can pick up a saved session details and resume some work
Justin Turner Arthur
@JustinTArthur
Gotchya. Well, two questions A) Did you get past your ValueError you mentioned above? B) Have you looked at session.cookie_jar.save(…) and session.cookie_jar.load(…)?
matrixbot
@matrixbot
wraptile e.g. job 1: log in to a website; job 2 few minutes later: scrape some stuff.
Job1 should save cookies to db; job2 should be able to restore ClientSession object from that db pickle.
wraptile yeah, the code snippet is absed of save() and load()
wraptile except to memory rather than file
wraptile I'm kinda disappointed that none of python's http libs implement session serialization/deserialization :|
wraptile would be great if the API supported ClientSession().dump() and ClientSession.load(pickle_obj) or something like that.
Justin Turner Arthur
@JustinTArthur
Which line does the exception happen on?
matrixbot
@matrixbot
wraptile oh, I solved the exception. I happened on dict(ClientSession().cookie_jar) I think.
Justin Turner Arthur
@JustinTArthur
I would also definitely recommend the HIGHEST_PROTOCOL param aiohttp uses internally. A lot of coroutine magic wasn’t part of Python 3.4, which the default protocol is based on
Justin Turner Arthur
@JustinTArthur

And yea, that would be good, re the API for in-memory. I also wish it used something more portable like JSON or CBOR instead of pickle.

I wish I had a better recommendation for you, but based on what’s officially available, the only other thing I can think of is if you don’t actually need session-based cookie management, you can store cookies however you want and send down a dict of your own with the cookies= arg to individual requests.

matrixbot
@matrixbot
wraptile I'm confused by ClientSession(cookies=) parameter. What's the actual expected type there? list of SimpleCookie objects? Can you plop a dictionary in there? e.g. ClientSession(cookies={'user': 'foo', 'pass': 'bar'}) ?
Martin Kopta
@mkopta_gitlab

Hello, I have odd issue with typing in aiohttp 3.6.2 (latest). I have this piece of code:

import aiohttp

with aiohttp.MultipartWriter('form-data') as mpwriter:
    mpwriter.append_json(
        {'a': 5},
        headers={'content-disposition': 'form-data;name=value_of_a'})

and I have these versions

# python            3.9.0
# aiohttp           3.6.2
# mypy              0.782

and I am getting this error from mypy:

append.py:6: error: Argument "headers" to "append_json" of "MultipartWriter" has inco
mpatible type "Dict[str, str]"; expected "Optional[MultiMapping[str]]"
Found 1 error in 1 file (checked 1 source file)

Looking at the code of aiohttp, and the documentation examples, I see a mismatch in typing. The documentation uses regular python dict, while the typing in the implementation expects a multidict. I would be ok with passing a multidict instead of dict, but I don't have this dependency and I don't want to add another dependency to my project because of this. Is the documentation wrong or the implementation wrong? Or am I wrong? Thanks!

4 replies
Justin Turner Arthur
@JustinTArthur
I could use a code review on the new implementation of asyncio.as_completed if anyone wants to chime in: python/cpython#22491 for bpo-33533
wuyuanyi135
@wuyuanyi135
Hello, the optional flag quote_fields of aiohttp.FormData() is not documented. The default escaping behavior makes it troublesome to work with array field (e.g., client_files[0]) since the bracket is escaped. Do you think we should highlight the usage of quote_fields?
Andres Hermosilla
@rezen
Has anyone here deployed an aiohttp server in AWS Lambda? It's easy to find adapters for Flask for example, but not so much with aiohttp
Justin Turner Arthur
@JustinTArthur

Has anyone here deployed an aiohttp server in AWS Lambda? It's easy to find adapters for Flask for example, but not so much with aiohttp

Usually in Lambda, I’m doing straight functions, going into asyncio if I need to.

Andres Hermosilla
@rezen
Ah, I see. With Flask I like using API Gateway as a proxy to pass off the handling to my handler which converts the the request to WSGI and then have flask handle it. I wasn't sure the best way to implement that sort of pattern with aiohttp
Justin Turner Arthur
@JustinTArthur
Could be a fun project to implement such a layer for aiohttp and Lambda Proxy. I haven’t seen how the Flask adapters do it.
Kay Khan
@kaykhancheckpoint

Friends does anyone else use gunicorn to run their aiohttp web server?

I want to be able to setup json logs

Nikhil Pareek
@NikhilPareek88
Hello,
Hope this is right channel to ask about aiohttp python 3.7.
I am requesting 5000 API calls through aiohttp, but need to wait until end to get the response of all calls together as it is mentioned in aiohttp.. it does in that way.
So query : Is there any way to read the response of individual API response, once it is done.
using code:
async with session.delete("{url}"), headers="") as response:
if response.status != 200:
if response.status != 404:
print("Failed")
else:
self.deleted += 1
Nikhil Pareek
@NikhilPareek88
async with session.delete("{url}"), headers="") as response:
            if response.status != 200:
                if response.status != 404:
                    print("Failed")
            else:
                self.deleted += 1
Alex Grönholm
@agronholm
@NikhilPareek88 that code just makes one call
so what is the problem?
I assume you don't literally put "{url}" in the call
Kay Khan
@kaykhancheckpoint
what are people using to log http request to json
Nikhil Pareek
@NikhilPareek88
@agronholm it it in a loop, and every time url is changed with some id.
the problem is, is it possible to avoid all data store in-memory
Andres Hermosilla
@rezen
Here is a my first pass at making aiohttp server work in lambda. It's a bit hacky, but it does work https://gist.github.com/rezen/3f8294c04d96eadca49515cb392f00e7
1 reply
Aviram Hassan
@aviramha
Hi All,
I'm having issues with aio kafka spamming this message: Heartbeat failed for group 0 because it is rebalancing
RES
@madkote

=== === ===
Hello Here!

a bit of topic question, but I still try it here:

in setup.py i neeed the keras-contrib@https://github.com/keras-team/keras-contrib.git as REQUIRES_INSTALL package.
Everyting works fine when installing with pip -r ..., BUT when tha package is built as wheel, then

ERROR: Could not find a version that satisfies the requirement keras-contrib (from superduperpack==0.3.5) (from versions: none)
ERROR: No matching distribution found for keras-contrib (from superduperpack==0.3.5)

Help appreciated!

Taras Voinarovskyi
@tvoinarovskyi
@aviramha if you are not yet on 0.6.0 aiokafka, please update, there were a few issues fixed
Those errors are usually fine if a few are thrown. Do you have a lot of partitions, I think the error may be printed for each one, which may be the reason of spamming.
If you are actually seeing constant spamming of this error, than you need to check your brokers, maybe Kafka can't elect a new coordinator. If nothing helps feel free to open an issue in aiokafka's bug tracker
Logs of how exactly it's spamming would be helpful in that case
Aviram Hassan
@aviramha
@tvoinarovskyi It seems to be a false alarm. I had an issue in my microservice and it was sending it every 3 secs so I thought it's related. Thanks for the help!
manideepl
@manideepl
Is there any good video course or book other than the polls tutorial available on the site?
manideepl
@manideepl
where do I store the salt generated by bcrypt? Is there a way I can persist it other than db?