Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
    Erus Lawliet
    @eruslawliet_twitter

    Hi everyone, I recently implement schemathesis in a project that I'm working on and really like it, I use the cli to run the tests, but not sure how to write the tests in code to be able to run the same parameters that I use in the cli.
    For example I run in the cli: schemathesis run --hypothesis-deadline 2000 -c all --hypothesis-max-examples 1000 --validate-schema true --show-errors-tracebacks http://localhost:8000/swagger.json and in the code I only have:

    import pytest
    import schemathesis
    from hypothesis import settings
    
    schema = schemathesis.from_uri('http://127.0.0.1:8000/swagger.json')
    
    
    @pytest.mark.slow
    @schema.parametrize(validate_schema=True)
    @settings(max_examples=1000)
    def test_api(case):
        response = case.call()
        case.validate_response(response)

    Not sure if that's okay and not sure how to add the others parameters, I will appreciate all of your help :D thanks in advance!

    3 replies
    Giulio Ungaretti
    @giulioungaretti
    Tips on how to use UUIDS from the command line
    I tried adding a module in the pre-run but it does not seem to do much :D
    Giulio Ungaretti
    @giulioungaretti
    ah no sorry please ignore, I had made a typo in the spec :(
    Alexander Hultnér
    @Hultner
    @Stranger6667 I submitted a talk about Schemathesis for EuroPython, it was accepted :)
    Dmitry Dygalo
    @Stranger6667
    @Hultner
    This is amazing :) I'll buy a ticket :)
    Alexander Hultnér
    @Hultner
    Cool! If you’re intrested it would be awesome if you wanted to look through my content once my first draft is ready and see if there’s anything I should change, include or exclude
    Cuong Luu
    @cluu12
    Hi @Stranger6667 , we are evaluating Schemathesis and I am very impressed with the tool. We already found issues with our API and I would like to customize the individual endpoints to control certain parameters, such as uploading a specific file, but leave the id to be set by Schemathesis, or set the bearer token before the call, etc... Do you have any documentation on the construct of a hook? Such as the available options for context, and required parameters in a hook function? If you have any link as a suggested reading, I greatly appreciate any pointers you have.
    Dmitry Dygalo
    @Stranger6667
    @Hultner
    Sorry, I missed your message :) I'd be happy to take a look and provide feedback
    Dmitry Dygalo
    @Stranger6667

    @cluu12
    I am glad to hear that! We have documentation on hooks here - https://schemathesis.readthedocs.io/en/latest/customization.html
    But from what I understand you'll require some more hooks - for modifying a request, for example. We don't have it at the moment, but we had some discussions on this matter and there is some ongoing work:

    Feel free to open an issue for things that are missing :) There is a list of them already in kiwicom/schemathesis#458 but I didn't manage to create separate issues for that

    Cuong Luu
    @cluu12
    @Stranger6667 , thank you for your response. I greatly appreciate it. I look forward to the future changes and will explore ways to utilize Schemathesis for our testing efforts.
    Cuong Luu
    @cluu12
    @Stranger6667, another question, is it possible to parameterize to specific response code? For example, I have a login endpoint with RC 201, 400, and 401. The examples will not be able to test 201 response without a valid userame/password. If make any changes to any parameter in the test, it will be used to test for all responses of the endpoint. Please do let me know if my conclusion is incorrect.
    3 replies
    Origin1227
    @Origin1227

    Hey @Stranger6667 great to see all these awesome updates :)
    is deepObject style supported in query param?

    parameters
           [
              {
                "name": "abcd",
                "in": "query",
                "description": "Parameters",
                "explode": false,
                "style": "deepObject",
                "schema": {
                  "type": "object",
                  "properties": {
                    "a": {
                      "type": "number"
                    },
                    "b": {
                      "type": "number"
                    }
                  }
                }
              }
            ]

    This is my schema for a query param
    i need query param to be generated of the form /users?abcd[a]=123
    but the generated ones are /users?abcd=123

    4 replies
    Alexander Hultnér
    @Hultner
    @Stranger6667 Schemathesis is featured in this weeks python bytes :smile:
    https://podcasts.apple.com/se/podcast/python-bytes/id1173690032?l=en&i=1000476214729
    1 reply
    Dmitry Dygalo
    @Stranger6667

    Hello there!
    Schemathesis 1.7.0 is out!🙂

    • Stateful testing via Open API links
    • add_case hook
    • Parameters serialization support. E.g. pipeDelimited or deepObject. More complex scenarios with content.<media-type> are supported as well
    • Support for YAML files in references

    Some info about stateful testing. Let's say you have a CRUD API for orders:

    • POST /api/orders
    • GET /api/orders/{order_id}
    • PATCH /api/orders/{order_id}
    • DELETE /api/orders/{order_id}

    With Open API links, you can say to Schemathesis - after you successfully create a new order, use the returned id as order_id parameter to GET /api/orders/{order_id} endpoint and run tests there for all created orders. This procedure will be applied recursively for all defined links (it is limited by 5 levels by default).
    In this way, you are much more likely to discover defects, because Schemathesis will know what data to use in tests to avoid 404 responses.

    You can enable it with --stateful=links CLI option.

    Friends, I'd be glad if you'll try this feature and share your feedback🙂

    Cheers!

    karusya
    @karusya
    hi guys, sorry for the stupid question - I just can't find this in the documentation.
    I want to use schemathesis in my pytest tests.
    what if I need to pass certain predefined BasicAuth username password if I want to use strategy like here - how that parameters should be passed ?
    strategy = schema["/{language}/api/v2/account/info"]["GET"].as_strategy()
    
    @given(case=strategy)
    def test_api(case):
        # some assertions
    18 replies
    Dmitry Dygalo
    @Stranger6667
    Folks, I added a PR for adding Schemathesis to the awesome-python repo and will really appreciate the support on this PR. If you agree with adding Schemathesis there, please add :+1: to the PR message
    vinta/awesome-python#1555
    Richard Boon
    @RichardB9
    Hey, I was wondering if it is possible to apply function-hooks in combination with lazy loading the schema with from_pytest_fixture?
    3 replies
    Stephen Mizell
    @Stephen_Mizell_twitter
    This is a great library! I've just started trying it out. I was wondering, what would be a good way to test optional response properties? I want to add a property to a response schema and see a failing test
    4 replies
    Francesco Bartoli
    @francbartoli

    Hey all, I like very much this library. I’m trying to add it a CI on travis for a project. However, I get an error for a nested route and I’m not able to dig out what is the parameters that is responsible for that. I get this in the stdout:

    _________________________________________________________________________________________ GET: /collections/obs/items __________________________________________________________________________________________
    Traceback (most recent call last):
      File "/Users/geobart/.pyenv/versions/3.7.6/envs/pygeoapi/lib/python3.7/site-packages/schemathesis/runner/impl/core.py", line 160, in run_test
        test(checks, targets, result, headers=headers, **kwargs)
      File "/Users/geobart/.pyenv/versions/3.7.6/envs/pygeoapi/lib/python3.7/site-packages/schemathesis/runner/impl/core.py", line 248, in network_test
        case: Case,
      File "/Users/geobart/.pyenv/versions/3.7.6/envs/pygeoapi/lib/python3.7/site-packages/hypothesis/core.py", line 1141, in wrapped_test
        raise the_error_hypothesis_found
      File "/Users/geobart/.pyenv/versions/3.7.6/envs/pygeoapi/lib/python3.7/site-packages/schemathesis/specs/openapi/serialization.py", line 17, in <lambda>
        return functools.reduce(lambda f, g: lambda x: f(g(x)), functions, noop)
      File "/Users/geobart/.pyenv/versions/3.7.6/envs/pygeoapi/lib/python3.7/site-packages/schemathesis/specs/openapi/serialization.py", line 162, in _map
        func(item, name, **kwargs)
      File "/Users/geobart/.pyenv/versions/3.7.6/envs/pygeoapi/lib/python3.7/site-packages/schemathesis/specs/openapi/serialization.py", line 182, in delimited
        item[name] = delimiter.join(item[name])
    TypeError: sequence item 0: expected str instance, float found
    
    =================================================================================================== SUMMARY ====================================================================================================
    
    Performed checks:
        not_a_server_error                             725 / 725 passed          PASSED
        status_code_conformance                        725 / 725 passed          PASSED
        content_type_conformance                       725 / 725 passed          PASSED
        response_schema_conformance                    725 / 725 passed          PASSED
    
    ======================================================================================== 7 passed, 2 errored in 12.05s =========================================================================================

    Do you have any hint?

    9 replies
    Francesco Bartoli
    @francbartoli
    Francesco Bartoli
    @francbartoli
    Many thanks
    Francesco Bartoli
    @francbartoli
    Thanks for your prompt reply and fix
    Dmitry Dygalo
    @Stranger6667
    @francbartoli You are very welcome! :) The new 2.2.0 release should be available in a few minutes and will contain that fix :)
    Francesco Bartoli
    @francbartoli
    Great @Stranger6667!
    Lloyd N.
    @Kulu-M
    Hello, I have an GET endpoint which has no parameters. However when I test it, it says "AssertionError: Passing a wrong parameter should result in a 4XX status code". I don't know how I can react to such a case. Thanks for any answers, I tried it for hours.
    Lloyd N.
    @Kulu-M
    Or how could I just ignore one endpoint?
    Dmitry Dygalo
    @Stranger6667
    Hi! Could you please share the schema and your test? CLI command or Python code
    Unfortunately we don’t have a feature to exclude endpoints, only including
    Rakesh1111
    @Rakesh1111
    Hello! I am new to schemathesis, I am trying to explore more on it. I am trying to use the schemathesis runner from AWS lambda to execute results and send an email notification. When I am trying to use the Json schema file from s3 location, it says me internal error ; failed to load the schema.. instead of file from s3, if I give any open swagger file from internet, it gives the results. Could some one help or guide me what I am missing. I have allowed or given all the permission for lambda to access the objects in s3 location.
    Rakesh1111
    @Rakesh1111

    from schemathesis import runner

    I m using this approach : events = runner.prepare("http://127.0.0.1:8080/swagger.json")
    for event in events:

    # do something with event
    Rakesh1111
    @Rakesh1111
    I have placed my client swagger schema.json in AWS S3 bucket. I have tried running just the schemathesis run command with the file uri as the file object url from AWS S3 bucket location. Even it says same error “internal error : Failed to load scheme 400 ... “. Will schemathesis support to run the files from AWS S3 bucket locations ?
    Michael Okoko
    @idoko_gitlab
    From your error, it seems it can find the and read the file correctly. The problem could be with your swagger file
    Dmitry Dygalo
    @Stranger6667
    Hi ! @Rakesh1111 could you please share the complete error message without traceback? As I see it starts with 400 - messages on this stage are usually prefixed with HTTP response codes. It could be a missing header or something like this (not sure about AWS S3 behavior)
    If the error comes from the headers issue , then you can pass headers to ‘from_uri’ function as a dictionary
    Rakesh1111
    @Rakesh1111
    @idoko_gitlab : if I place the same file in the local file system under my home directory in computer, it works with same headers. If I try to place the swagger file in the AWS S3 location, with the same headers which works in above case .. it doesn’t work here
    @idoko_gitlab : in the above two cases I am using the schemathesis run command from terminal
    Rakesh1111
    @Rakesh1111
    @Stranger6667 : Failed to load schema, code 400 was returned from https://<<bucketname>>.s3-<<regionname>>.amazonaws.com/swagger2.json
    Dmitry Dygalo
    @Stranger6667
    Thanks for sharing the message. Re local file I think that if you load the schema from your file system, then the headers don’t matter - it is a different use case. The code under the hood boils down to requests.get(“...address...”) - my assumption is that you need to pass an additional Authorization header to access your bucket
    Or some query parameters
    Rakesh1111
    @Rakesh1111
    Programatically using the below approach:
    events = runner.prepare(url,
                            headers=headers,
                            checks=[not_a_server_error, response_schema_conformance], base_url=baseUrl)
    Dmitry Dygalo
    @Stranger6667
    8 replies
    Rakesh1111
    @Rakesh1111
    url Is the location of s3 file, header include accept-version, x-api-key, "Authorization": "Bearer " + token.
    Dmitry Dygalo
    @Stranger6667
    You could try to run requests.get with your url and headers, then check the response content - probably it will contain some reassigning for 400 response status. At the moment I am not able to reproduce this behavior locally - it will require some more time
    1 reply
    partizan
    @last-partizan

    Hi all.
    I'm trying to use schemathesis with pytest, making asgi_call instead of call. Should i populate my test database with data before running tests? or can i use pytest fixtures?

    I found before_process_path hook, and there is database access for populating data, but how i can use pytest fixtures there?

    3 replies
    Michael Okoko
    @idoko_gitlab
    Hi, I'm running running my tests against the all checks, but I was wondering if there's a way for it to not fail on 500 errors as long as the HTTP 500 error is documented
    4 replies
    samuel guesnier
    @samuelguesnier
    Hello everyone :)
    Can someone help me pleae ? I'd like to save all requests in cassette, but I don't know how to do that
    import requests
    import schemathesis
    from hypothesis import settings
    schema = schemathesis.from_path(path="sam.yaml",validate_schema=False,base_url="https://baseurl.com")
    
    
    
    
    
    @schema.parametrize( endpoint="/subscriptions/{subscriptionUid}/devices", method="POST")
    @settings(max_examples=1)
    def test_no_server_errors(case):
        case.query['sessionToken'] = 'ad3414b8becbb1e9'
        case.path_parameters["subscriptionUid"] = 1384
        response = case.call()  
        case.validate_response(response)
    1 reply