These are chat archives for evhub/coconut

7th
Jul 2016
Constantine Molchanov
@moigagoo
Jul 07 2016 08:08

parallel_map is super convenient to speed up execution of CPU-bound tasks (like calculating hashes). It also gives a significant performance boost for IO-bound tasks (like sending HTTP requests), but not as significant as running the tasks asynchronously would give. It would be great to have something like concurrent_map that would utilize asyncio module or even Python 3's async and await syntax. For the first implementation, using a thread pool instead of a process pool would probably do the trick well enough.

What do you guys think? Would that be doable or useful?

Evan Hubinger
@evhub
Jul 07 2016 17:18
@moigagoo Great idea. Just added a concurrent_map in develop.
Constantine Molchanov
@moigagoo
Jul 07 2016 19:03
BTW I tried adding it myself: copied all invocations of parallel_map replacing processes with threads etc. However, I failed to run the shell. The error was that _coconut_concurrent_map was not found. The compiler.py code is a bit hairy, so I couldn't dig deeply into the issue.
Nevermind, I just looked into your commit and understood where I had made a mistake.
Evan Hubinger
@evhub
Jul 07 2016 20:13
@moigagoo Yeah, compiler.py is gigantic and rather confusing. I've been meaning to clean it up for a while as I was telling @NickSeagull.