These are chat archives for cherrypy/cherrypy

19th
Dec 2016
David Klaftenegger
@davidklaftenegger
Dec 19 2016 16:39
hello. is anyone here willing to explain to me how multithreading is used in cherrypy? (I am not interested in using my own threads, but rather in understanding how cherrypy is using threads itself)
Tiago Rodrigues
@TiagoMRodrigues
Dec 19 2016 17:13
anyone used cherrypy with keras ? (tensorflow backend) I have the same error described here fchollet/keras#2397
if anyone sorted this out it would be apreciated some tips
Jason R. Coombs
@jaraco
Dec 19 2016 18:11
@davidklaftenegger: CherryPy keeps a pool of threads for handling requests (typically 10 or 20 threads). As each request is answered by the main listening thread, it’s handed off to a thread in the pool to fulfill the request. In that way, each request runs in exactly one thread, so CherryPy can use ‘thread-local’ constructs to store state for that each request.
David Klaftenegger
@davidklaftenegger
Dec 19 2016 18:40
are these python-threads, or is there some parallelism involved?
my instrumentation shows some calls to pthreads_create which I wonder where they are coming from
@jaraco: I see 11 calls to pthread_create and 11 calls to mprotect. Do you happen to know what mprotect() is used for?
Jason R. Coombs
@jaraco
Dec 19 2016 19:16
@davidklaftenegger They are Python threads, which are OS threads, which are inherently parallel in concept as code execution can be arbitrarily interrupted in any thread for code in another thread to run. Only one OS process is involved. I’m guessing that pthread_create and mprotect are system calls in your OS implementing the creation of those threads, probably 10 for handling requests and an additional one to monitor or accept requests.
David Klaftenegger
@davidklaftenegger
Dec 19 2016 19:25
@jaraco: pthread_create is the OS call for creating a thread, yes. mprotect has nothing to do with threads, and instead is used for setting access privileges on memory regions.
@jaraco: I am not sure about the implementation of python threads, as the global interpreter lock issue is something that makes an implementation without OS threads possible / potentially more performant. But I have not checked.
David Klaftenegger
@davidklaftenegger
Dec 19 2016 21:23
@jaraco: It appears to work even in my environment where mprotect has no function, but I don't fully understand what it is doing there. Thanks for your explanation of how the threads are used.
Jason R. Coombs
@jaraco
Dec 19 2016 23:03
@davidklaftenegger Yeah, mprotect doesn’t appear anywhere in the CherryPy code, so it’s probably the Python implementation of threads (https://docs.python.org/3/library/threading.html) that’s calling mprotect. You might verify that by creating your own Python threads and see if the calls you’re observing in CherryPy are also seen in simple Python thread creation.