cherrypy/cherrypy

CherryPy is a pythonic, object-oriented HTTP framework. | post your issues to: github.com/cherrypy/cherrypy/issues | FAQ: https://stackoverflow.com/questions/tagged/cheroot+or+cherrypy

Nathaniel J. Smith

@njsmith

@webknjaz done

Sviatoslav Sydorenko

@webknjaz

@njsmith thank you! Mind cutting a release?

bmxp

@bmxp

So I just want to give everyone a followup from my post from August 18, 2019 where I asked for help with a very basic problem.

Can anyone explain why using <!DOCTYPE html> results in missing css within the browser?
I prepared a small test where the only distinction between two html files is the first line. If <!DOCTYPE html>is omitted then the background looks fine in green and we have a pleasing Hello world
If it is included then css won't be used for display. Maybe one of you cracks can help me?
And BTW: I am using Cherrypy 18.1.2 on Windows

Finally it turned out to be a combination of two problems:
First: If starting a document with only <html> ... then this seems to be considered to be not really a valid html file.
Thus an enclosed link to a css which is delivered with text/plain as mimetype will display fine.
If starting correctly with <!DOCTYPE html> ... then it is not forgiven that the mimetype is essentially wrong.

But why was cherrypy delivering text/plain as mimetype for my css file? I tried it on another windows computer and it did not show the problem. I tried another older computer which I used earlier for some development and it showed the same buggy behaviour...
Hm. Linux never showed this failure not with any flavour I tried out.

So I investigated deeper: Cherrypy uses the python standard mimetype module or I can pass a dedicated Content-Type for css when including 'tools.staticdir.content_types': { 'css': 'text/css' } in my \static config. When passing the dedicated Content-Type the failure was gone.

Now why seems there always to be a wrong association on just certain windows systems? I found out that winreg is used to directly read mimetypes out of the registry. Inspecting Computer\HKEY_CLASSES_ROOT\.css showed an association with text\plain. So this caused my headaches! After changing Content Type to text/css the while beast worked as expected.
So a big thank you @webknjaz and @cyraxjoe for trying to help me

Sviatoslav Sydorenko

@webknjaz

@bmxp interesting! I think this information may be useful to others in your shoes. Could you please document it on StackOverflow? Simply file a question and while doing it, use the checkbox saying Answer your own question here https://stackoverflow.com/questions/ask. Don't forget to add cherrypy tag. Ref https://stackoverflow.blog/2011/07/01/its-ok-to-ask-and-answer-your-own-questions/.

bmxp

@bmxp

https://stackoverflow.com/questions/63213763/why-using-html-file-starting-with-doctype-html-served-with-cherrypy-on-window/63213853#63213853

Please have a look if something is missing as I am not a professional programmer...

Sviatoslav Sydorenko

@webknjaz

Perfect, looks great!

Farooq Karimi Zadeh

@farooqkz

cherrypy.tools.caching does not work or I am misusing it: https://gist.github.com/farooqkz/8fafae050bd1bd57bff4d3f70f252005

Sviatoslav Sydorenko

@webknjaz

define "does not work"

Farooq Karimi Zadeh

@farooqkz

Instead of output of index() being updated every 30secs, I see the output being updated everytime I send a request.

Sviatoslav Sydorenko

@webknjaz

And how exactly do you that? What's the server env? What's the client?

Farooq Karimi Zadeh

@farooqkz

server env?Client if Firefox 79.0

Farooq Karimi Zadeh

@farooqkz

is*

I just refresh the page in Firefox and I see that cherrypy does not cache the response.

ajyoung

@ajyoung

I've added repeatable reproduction steps to cherrypy/cheroot#210 and a proposed fix cherrypy/cheroot#310. I hope it is ok. There was a windows test suite failure that has been happening on master for the last ten days as well. But I'm not sure yet about the ci/circleci: macos-build and continuous-integration/appveyor/pr failures.

Tomás Lund Petersen

@kotogadekiru

hello

i have a couple of questions about cherrypy and the docs havent been helpfull

one: is it posible to serve more than one request at the same time in cherrypy? how

two: where can i read about available tools in cherrypy. i mean like @cherrypy.tools.json_in()

Tomás Lund Petersen

@kotogadekiru

im trying to use _cp_config={'response.stream':True} with @cherrypy.tools.json_in() but when i add _cp_config={'response.stream':True} to the method it throws Request' object has no attribute 'json' at input_json = cherrypy.request.json

Sviatoslav Sydorenko

@webknjaz

@farooqkz I did some testing and figured out that caching works by my browser sends Cache-Control: max-age=0 header that instructs the server not to send the cached response. Try doing it in your terminal via curl 'http://127.0.0.1:8080/' and you'll see that it works. So you problem is that the client doesn't want cache, not the server.

@ajyoung I'll try to review that when I have time. Have you added the tests there?

@kotogadekiru what do you mean by "one request at a time"? The HTTP part is implemented in Cheroot and is thread-based.

wrt json_in + stream response, I'm not sure these are compatible. why do you need such a combo, anyway?

https://docs.cherrypy.org/en/latest/advanced.html#how-streaming-output-works-with-cherrypy

Farooq Karimi Zadeh

@farooqkz

@webknjaz Thank you :)

Tomás Lund Petersen

@kotogadekiru

@webknjaz > @kotogadekiru what do you mean by "one request at a time"? The HTTP part is implemented in Cheroot and is thread-based.
i'll try explain. my api makes calls to 3rd party apis so it takes a lot of time waiting for the third party to respond. i believe this is bloking the thread. so i can't serve multiple requests at the same time (while the other request is waiting). i'm deploying cherrypy on a heroku web dyno. i have to check this but i believe each dyno is single thread.

Tomás Lund Petersen

@kotogadekiru

@webknjaz > wrt json_in + stream response, I'm not sure these are compatible. why do you need such a combo, anyway?
firt i take my parameters in json format so its easyer to just use json_in. unless there is a problem with that i don't care for parsing them myself
second as i explained before my api makes calls to third party apis and this can take some time heroku router timeouts the calls if you don't respond before 30 seconds so i would like to first send the headers as the doc you provided explains and serve the data as it is processed thus keeping the connection alive and avoiding timeout problems.
i have also tried _cp_config = {"request.methods_with_bodies": ("POST")} wich also seems to interfere with json_in. maybe theres a diferente way of doing this.
thankyou for your time.

Sviatoslav Sydorenko

@webknjaz

@kotogadekiru Heroku dyno runs one instance of the process that you specify but then, when setting up CherryPy, you may specify the number of threads. It runs a handler for each request in a separate thread. You may set server.thread_pool to 100 or however many threads you need and you'll have more capacity.

@kotogadekiru could you please share some MCVE for me to test?

Sviatoslav Sydorenko

@webknjaz

_cp_config = {"request.methods_with_bodies": ("POST")}

I believe this syntax is invalid because you set the value to a "POST" string rather than an iterable, like a tuple. Try adding a comma to see if that helps:

_cp_config = {"request.methods_with_bodies": ("POST", )}

P.S. FYI you can use backticks to highlight code snippets that you send here in chat.

Tomás Lund Petersen

@kotogadekiru

@kotogadekiru could you please share some MCVE for me to test?
hi, i've come up with this exaple i hope it is what you needed
`
import os
import time
import json
import sys

import cherrypy

class App(object):
@cherrypy.expose
def index(self):
return "hello world"

@cherrypy.tools.json_in()
@cherrypy.expose
def takes_json_no_stream(self):
    input_json = cherrypy.request.json

    def generator():
        for i in range(31):
            time.sleep(1)
            yield '{} {}\n'.format(time.asctime(), i + 1)
            print('{} {}\n'.format(time.asctime(), i ))

        time.sleep(1)
        yield '{} Done'.format(time.asctime())

    return generator()

@cherrypy.tools.json_in()
@cherrypy.expose
def fails_json(self):
    input_json = cherrypy.request.json

    def generator():
        for i in range(31):
            time.sleep(1)
            yield '{} {}\n'.format(time.asctime(), i + 1)
            print('{} {}\n'.format(time.asctime(), i))

        time.sleep(1)
        yield '{} Done'.format(time.asctime())

    return generator()
fails_json._cp_config = {'response.stream': True}

@cherrypy.expose
def no_json_stream(self):
    body = cherrypy.request.body.read()
    json_in = json.loads(body)
    print(body)
    print(json_in)
    def generator():
        for i in range(31):
            time.sleep(1)
            yield '{} {}\n'.format(time.asctime(), i + 1)
            print('{} {}\n'.format(time.asctime(), i))
        time.sleep(1)
        yield '{} Done'.format(time.asctime())
    return generator()
no_json_stream._cp_config = {'response.stream': True}

config = {
'global': {
'server.socket_host': '0.0.0.0',
'server.socket_port': int(os.environ.get('PORT', 5001)),
'server.thread_pool': 100,
},
'robots.txt':{
'tools.staticfile.on': True,
'tools.staticfile.filename':'/html/robots.txt'
},
}
if name == 'main':
cherrypy.quickstart(App(),'/',config)
`

sorry i cant seem to master the backticks yet.

Tomás Lund Petersen

@kotogadekiru

the idea is that i expose 3 methods:
one that takes json in but won't stream
one that should stream but fails on json_in
and one that streams by parsing the content with json.loads

Sviatoslav Sydorenko

@webknjaz

@kotogadekiru to send multiline blocks, use 3 backticks at the beginning on a separate line and 3 backticks at the end, that's it :)

oh and at the beginning, it didn't work because if you don't have an empty line after the blockquote, it'll merge that line with whatever is under ">"

Sviatoslav Sydorenko

@webknjaz

I've formatted it a bit:

import os
import time
import json
import sys

import cherrypy

class App(object):
    @cherrypy.expose
    def index(self):
        return "hello world"

    @cherrypy.tools.json_in()
    @cherrypy.expose
    def takes_json_no_stream(self):
        input_json = cherrypy.request.json

        def generator():
            for i in range(31):
                time.sleep(1)
                yield '{} {}\n'.format(time.asctime(), i + 1)
                print('{} {}\n'.format(time.asctime(), i ))

            time.sleep(1)
            yield '{} Done'.format(time.asctime())

        return generator()

    @cherrypy.tools.json_in()
    @cherrypy.expose
    def fails_json(self):
        input_json = cherrypy.request.json

        def generator():
            for i in range(31):
                time.sleep(1)
                yield '{} {}\n'.format(time.asctime(), i + 1)
                print('{} {}\n'.format(time.asctime(), i))

            time.sleep(1)
            yield '{} Done'.format(time.asctime())

        return generator()
    fails_json._cp_config = {'response.stream': True}

    @cherrypy.expose
    def no_json_stream(self):
        body = cherrypy.request.body.read()
        json_in = json.loads(body)
        print(body)
        print(json_in)
        def generator():
            for i in range(31):
                time.sleep(1)
                yield '{} {}\n'.format(time.asctime(), i + 1)
                print('{} {}\n'.format(time.asctime(), i))
            time.sleep(1)
            yield '{} Done'.format(time.asctime())
        return generator()
    no_json_stream._cp_config = {'response.stream': True}
config = {
    'global': {
        'server.socket_host': '0.0.0.0',
        'server.socket_port': int(os.environ.get('PORT', 5001)),
        'server.thread_pool': 100,
    },
    'robots.txt':{
        'tools.staticfile.on': True,
        'tools.staticfile.filename':'/html/robots.txt'
    },
}
if name == '__main__':
    cherrypy.quickstart(App(), '/', config)

Sviatoslav Sydorenko

@webknjaz

I'm not sure if all of the normal machinery kicks in when using streaming

OTOH need to check if you tried methods_with_bodies setting correctly after all.

Sviatoslav Sydorenko

@webknjaz

@kotogadekiru I did some testing and I think you're sending the requests wrong

It's reproducible if I don't specify a proper content-type request header but works with it in place

Sviatoslav Sydorenko

@webknjaz

#! /usr/bin/env python3

import time

import cherrypy


class App:
    @cherrypy.tools.json_in()
    @cherrypy.config(**{'response.stream': True})
    @cherrypy.expose
    def input_json_with_streaming_response(self):
        input_json = cherrypy.request.json
        print(input_json)

        def stream_chunks():
            for i in range(31):
                time.sleep(1)
                yield '{} {} {}\n'.format(time.asctime(), i + 1, input_json)
                print('{} {} {}\n'.format(time.asctime(), i, input_json))

            time.sleep(1)
            yield '{} Done'.format(time.asctime())

        return stream_chunks()


if __name__ == '__main__':
    cherrypy.quickstart(App())

@kotogadekiru ^ this is a minimal example that works

All you need to do is to make sure you send Content-Type: application/json header in your requests because json_in tool only attempts to read and parse the body and then assign it to cherrypy.request.json if the client actually says that it sends JSON.

So this fails:

$ curl 127.0.0.1:8080/input_json_with_streaming_response -d '{"some": "data"}'

And this succeeds:

$ curl 127.0.0.1:8080/input_json_with_streaming_response -d '{"some": "data"}' -H 'Content-Type: application/json'

Tomás Lund Petersen

@kotogadekiru

hi it's working now. i believe the problem was with fails_json._cp_config = {'response.stream': True} at the botom of the declaration. it seems to be overwriting the other configurations. but with @cherrypy.config(**{'response.stream': True}) everything seems to work.

I was sending the correct header all along so I don't think this was the problem

this is my postman configuration for tests

{
    "info": {
        "_postman_id": "1cd4f1a6-fec2-43d3-a07f-5ee5136e0541",
        "name": "puma",
        "schema": "https://schema.getpostman.com/json/collection/v2.1.0/collection.json"
    },
    "item": [
        {
            "name": "localhost stream test",
            "request": {
                "method": "POST",
                "header": [
                    {
                        "key": "Content-Type",
                        "name": "Content-Type",
                        "value": "application/json",
                        "type": "text"
                    }
                ],
                "body": {
                    "mode": "raw",
                    "raw": "{\r\n\"polygons\":\"[[[[-58.5507026615934,-37.6724720338776],[-58.5505008132247,-37.6727103305258],[-58.5504727912878,-37.6728886065142],[-58.5504637325053,-37.6730400092927],[-58.5504489088968,-37.6732877592815],[-58.5504406734803,-37.6734253981575],[-58.5504316144574,-37.6735768009156],[-58.5505751091014,-37.6737891505271],[-58.5506509855884,-37.6739713539652],[-58.5506452209221,-37.6740677011943],[-58.5506361621052,-37.6742191039781],[-58.5505702121887,-37.67445112633],[-58.5504407602637,-37.6745841829903],[-58.5502610051408,-37.6746877478088],[-58.5500507344831,-37.6747211835971],[-58.5498932373105,-37.6747428192149],[-58.5497398586315,-37.67469563526],[-58.5496211134049,-37.6746497605856],[-58.5494899942528,-37.6745206478493],[-58.5493794871783,-37.6743371342979],[-58.5493506204981,-37.6742394777134],[-58.5492145597478,-37.6741929479211],[-58.5490933278044,-37.6738986684914],[-58.548981174874,-37.6737426823926],[-58.5489209714632,-37.6735886606821],[-58.548878084594,-37.6734352937781],[-58.5488690066109,-37.6732970003742],[-58.5488665192461,-37.6730485961011],[-58.5488491871003,-37.6727582454277],[-58.548863191936,-37.6725242598164],[-58.5487856734394,-37.6723695832816],[-58.5487757717439,-37.6722450537311],[-58.5487938955813,-37.6719422488237],[-58.5488450033796,-37.6716682812241],[-58.5488713808781,-37.6715175335412],[-58.5488994057751,-37.6713392581165],[-58.5489274304952,-37.6711609826727],[-58.5489408000303,-37.670937594539],[-58.5457777973257,-37.6681826608375],[-58.5440719123915,-37.6694480399521],[-58.5431921478343,-37.6694989943634],[-58.5423016544411,-37.6701529062093],[-58.552000522243,-37.6816421312368],[-58.5549192596306,-37.6794408695454],[-58.5550601996307,-37.6792830395455],[-58.5550439996307,-37.6788537095455],[-58.5549901596306,-37.6785925895455],[-58.5548103596305,-37.6784711795454],[-58.5519765967334,-37.6759829060069],[-58.5526359798996,-37.6755346162678],[-58.5528817433976,-37.6757805048908],[-58.5537391737234,-37.6751163914454],[-58.5507026615934,-37.6724720338776]]]]\",\r\n \"end\": \"2018-01-29\", \r\n \"begin\": \"2018-01-15\",\r\n \"token\":\"testToken\"\r\n }",
                    "options": {
                        "raw": {
                            "language": "json"
                        }
                    }
                },
                "url": {
                    "raw": "http://localhost:5001/fails_json",
                    "protocol": "http",
                    "host": [
                        "localhost"
                    ],
                    "port": "5001",
                    "path": [
                        "fails_json"
                    ]
                },
                "description": "test request para ndvi puma"
            },
            "response": []
        }
    ],
    "protocolProfileBehavior": {}
}

Sviatoslav Sydorenko

@webknjaz

There shouldn't be any difference, but yes, it's possible that the config was overridden somehow

Where communities thrive

People

Repo info

Activity