Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
    Dmitry Dygalo
    @Stranger6667

    Hello there!
    Schemathesis 1.7.0 is out!🙂

    • Stateful testing via Open API links
    • add_case hook
    • Parameters serialization support. E.g. pipeDelimited or deepObject. More complex scenarios with content.<media-type> are supported as well
    • Support for YAML files in references

    Some info about stateful testing. Let's say you have a CRUD API for orders:

    • POST /api/orders
    • GET /api/orders/{order_id}
    • PATCH /api/orders/{order_id}
    • DELETE /api/orders/{order_id}

    With Open API links, you can say to Schemathesis - after you successfully create a new order, use the returned id as order_id parameter to GET /api/orders/{order_id} endpoint and run tests there for all created orders. This procedure will be applied recursively for all defined links (it is limited by 5 levels by default).
    In this way, you are much more likely to discover defects, because Schemathesis will know what data to use in tests to avoid 404 responses.

    You can enable it with --stateful=links CLI option.

    Friends, I'd be glad if you'll try this feature and share your feedback🙂

    Cheers!

    karusya
    @karusya
    hi guys, sorry for the stupid question - I just can't find this in the documentation.
    I want to use schemathesis in my pytest tests.
    what if I need to pass certain predefined BasicAuth username password if I want to use strategy like here - how that parameters should be passed ?
    strategy = schema["/{language}/api/v2/account/info"]["GET"].as_strategy()
    
    @given(case=strategy)
    def test_api(case):
        # some assertions
    18 replies
    Dmitry Dygalo
    @Stranger6667
    Folks, I added a PR for adding Schemathesis to the awesome-python repo and will really appreciate the support on this PR. If you agree with adding Schemathesis there, please add :+1: to the PR message
    vinta/awesome-python#1555
    Richard Boon
    @RichardB9
    Hey, I was wondering if it is possible to apply function-hooks in combination with lazy loading the schema with from_pytest_fixture?
    3 replies
    Stephen Mizell
    @Stephen_Mizell_twitter
    This is a great library! I've just started trying it out. I was wondering, what would be a good way to test optional response properties? I want to add a property to a response schema and see a failing test
    4 replies
    Francesco Bartoli
    @francbartoli

    Hey all, I like very much this library. I’m trying to add it a CI on travis for a project. However, I get an error for a nested route and I’m not able to dig out what is the parameters that is responsible for that. I get this in the stdout:

    _________________________________________________________________________________________ GET: /collections/obs/items __________________________________________________________________________________________
    Traceback (most recent call last):
      File "/Users/geobart/.pyenv/versions/3.7.6/envs/pygeoapi/lib/python3.7/site-packages/schemathesis/runner/impl/core.py", line 160, in run_test
        test(checks, targets, result, headers=headers, **kwargs)
      File "/Users/geobart/.pyenv/versions/3.7.6/envs/pygeoapi/lib/python3.7/site-packages/schemathesis/runner/impl/core.py", line 248, in network_test
        case: Case,
      File "/Users/geobart/.pyenv/versions/3.7.6/envs/pygeoapi/lib/python3.7/site-packages/hypothesis/core.py", line 1141, in wrapped_test
        raise the_error_hypothesis_found
      File "/Users/geobart/.pyenv/versions/3.7.6/envs/pygeoapi/lib/python3.7/site-packages/schemathesis/specs/openapi/serialization.py", line 17, in <lambda>
        return functools.reduce(lambda f, g: lambda x: f(g(x)), functions, noop)
      File "/Users/geobart/.pyenv/versions/3.7.6/envs/pygeoapi/lib/python3.7/site-packages/schemathesis/specs/openapi/serialization.py", line 162, in _map
        func(item, name, **kwargs)
      File "/Users/geobart/.pyenv/versions/3.7.6/envs/pygeoapi/lib/python3.7/site-packages/schemathesis/specs/openapi/serialization.py", line 182, in delimited
        item[name] = delimiter.join(item[name])
    TypeError: sequence item 0: expected str instance, float found
    
    =================================================================================================== SUMMARY ====================================================================================================
    
    Performed checks:
        not_a_server_error                             725 / 725 passed          PASSED
        status_code_conformance                        725 / 725 passed          PASSED
        content_type_conformance                       725 / 725 passed          PASSED
        response_schema_conformance                    725 / 725 passed          PASSED
    
    ======================================================================================== 7 passed, 2 errored in 12.05s =========================================================================================

    Do you have any hint?

    9 replies
    Francesco Bartoli
    @francbartoli
    Francesco Bartoli
    @francbartoli
    Many thanks
    Francesco Bartoli
    @francbartoli
    Thanks for your prompt reply and fix
    Dmitry Dygalo
    @Stranger6667
    @francbartoli You are very welcome! :) The new 2.2.0 release should be available in a few minutes and will contain that fix :)
    Francesco Bartoli
    @francbartoli
    Great @Stranger6667!
    Lloyd N.
    @Kulu-M
    Hello, I have an GET endpoint which has no parameters. However when I test it, it says "AssertionError: Passing a wrong parameter should result in a 4XX status code". I don't know how I can react to such a case. Thanks for any answers, I tried it for hours.
    Lloyd N.
    @Kulu-M
    Or how could I just ignore one endpoint?
    Dmitry Dygalo
    @Stranger6667
    Hi! Could you please share the schema and your test? CLI command or Python code
    Unfortunately we don’t have a feature to exclude endpoints, only including
    Rakesh1111
    @Rakesh1111
    Hello! I am new to schemathesis, I am trying to explore more on it. I am trying to use the schemathesis runner from AWS lambda to execute results and send an email notification. When I am trying to use the Json schema file from s3 location, it says me internal error ; failed to load the schema.. instead of file from s3, if I give any open swagger file from internet, it gives the results. Could some one help or guide me what I am missing. I have allowed or given all the permission for lambda to access the objects in s3 location.
    Rakesh1111
    @Rakesh1111

    from schemathesis import runner

    I m using this approach : events = runner.prepare("http://127.0.0.1:8080/swagger.json")
    for event in events:

    # do something with event
    Rakesh1111
    @Rakesh1111
    I have placed my client swagger schema.json in AWS S3 bucket. I have tried running just the schemathesis run command with the file uri as the file object url from AWS S3 bucket location. Even it says same error “internal error : Failed to load scheme 400 ... “. Will schemathesis support to run the files from AWS S3 bucket locations ?
    Michael Okoko
    @idoko_gitlab
    From your error, it seems it can find the and read the file correctly. The problem could be with your swagger file
    Dmitry Dygalo
    @Stranger6667
    Hi ! @Rakesh1111 could you please share the complete error message without traceback? As I see it starts with 400 - messages on this stage are usually prefixed with HTTP response codes. It could be a missing header or something like this (not sure about AWS S3 behavior)
    If the error comes from the headers issue , then you can pass headers to ‘from_uri’ function as a dictionary
    Rakesh1111
    @Rakesh1111
    @idoko_gitlab : if I place the same file in the local file system under my home directory in computer, it works with same headers. If I try to place the swagger file in the AWS S3 location, with the same headers which works in above case .. it doesn’t work here
    @idoko_gitlab : in the above two cases I am using the schemathesis run command from terminal
    Rakesh1111
    @Rakesh1111
    @Stranger6667 : Failed to load schema, code 400 was returned from https://<<bucketname>>.s3-<<regionname>>.amazonaws.com/swagger2.json
    Dmitry Dygalo
    @Stranger6667
    Thanks for sharing the message. Re local file I think that if you load the schema from your file system, then the headers don’t matter - it is a different use case. The code under the hood boils down to requests.get(“...address...”) - my assumption is that you need to pass an additional Authorization header to access your bucket
    Or some query parameters
    Rakesh1111
    @Rakesh1111
    Programatically using the below approach:
    events = runner.prepare(url,
                            headers=headers,
                            checks=[not_a_server_error, response_schema_conformance], base_url=baseUrl)
    Dmitry Dygalo
    @Stranger6667
    8 replies
    Rakesh1111
    @Rakesh1111
    url Is the location of s3 file, header include accept-version, x-api-key, "Authorization": "Bearer " + token.
    Dmitry Dygalo
    @Stranger6667
    You could try to run requests.get with your url and headers, then check the response content - probably it will contain some reassigning for 400 response status. At the moment I am not able to reproduce this behavior locally - it will require some more time
    1 reply
    partizan
    @last-partizan

    Hi all.
    I'm trying to use schemathesis with pytest, making asgi_call instead of call. Should i populate my test database with data before running tests? or can i use pytest fixtures?

    I found before_process_path hook, and there is database access for populating data, but how i can use pytest fixtures there?

    3 replies
    Michael Okoko
    @idoko_gitlab
    Hi, I'm running running my tests against the all checks, but I was wondering if there's a way for it to not fail on 500 errors as long as the HTTP 500 error is documented
    4 replies
    samuel guesnier
    @samuelguesnier
    Hello everyone :)
    Can someone help me pleae ? I'd like to save all requests in cassette, but I don't know how to do that
    import requests
    import schemathesis
    from hypothesis import settings
    schema = schemathesis.from_path(path="sam.yaml",validate_schema=False,base_url="https://baseurl.com")
    
    
    
    
    
    @schema.parametrize( endpoint="/subscriptions/{subscriptionUid}/devices", method="POST")
    @settings(max_examples=1)
    def test_no_server_errors(case):
        case.query['sessionToken'] = 'ad3414b8becbb1e9'
        case.path_parameters["subscriptionUid"] = 1384
        response = case.call()  
        case.validate_response(response)
    1 reply
    samuel guesnier
    @samuelguesnier
    Maybe I must use prepare function ?
    1 reply
    samuel guesnier
    @samuelguesnier
    Another question: is it possible to pass query paramers / body on CLI ?
    Ho, I found an answer :p kiwicom/schemathesis#695
    Ramesh
    @Rameshpvg_gitlab
    Hi,
    I am new here, I just started looking at schemathesis.. Looks like a great library. I would like to explore more.
    Could somebody please show me where can I find more examples on from_file, from_path, from_dict usage.
    5 replies
    samuel guesnier
    @samuelguesnier
    Hello eveyone ! :)
    Is Schematesis is reliable enough to be used all alone for testing my API or is it only used to find corner case ?
    Another question: Schemathesis generates right data according to the schema and also wrong data ? If so, is it possible to know when Schemathesis is generating right datas ? For example: If case contain wrong data && response_code == 200 then Fail .
    4 replies
    panix
    @alex-pancho
    Question: How can I get an extended test report? So that I could run the schemathesis on my api, he did 4,000 tests, wrote that everything was ok, but what exactly was it for the tests?
    7 replies
    vkutepov
    @vkutepov
    This message was deleted
    1 reply
    vkutepov
    @vkutepov

    and one more question
    i understand that schemathesis
    not made for this
    But if I want to use it for an integration test
    if I want to call one service and then another in the same test
    can I create schema1 and schema2?
    if yes then two case fixtures should be created

    like that
    or maybe it can be split into separate fixtures

    schema1 = schemathesis.from_uri(
        "http://my_service1.stg.com:84/swagger.json"
    )
    schema2 = schemathesis.from_uri(
        "http://my_service2.stg.com:84/swagger.json"
    )
    
    
    def test_1(case1, case2):
        case1.endpoint = "/get_result"
        case1.method = "GET"
        response1 = case1.call()
    
        case2.endpoint = "/send_result"
        case2.method = "POST"
        case2.body = response1.json()
        response2 = case2.call()
    12 replies
    panix
    @alex-pancho
    вопросы
    10 replies
    Dmitry Dygalo
    @Stranger6667
    Hi all! We are gathering feedback on Schemathesis - https://forms.gle/arTf6yQEcQH8ShxHA . It will help me to make improvements to it and prioritize new features! :) It will take 5 minutes. The results are anonymous.
    Janus Troelsen
    @ysangkok
    i am having a problem where i call validate_response but it doesn't run the check that the response conforms to the schema. how do i programmatically add that check?
    matrixbot
    @matrixbot
    @janus.troelsen:matrix.org (this is also me, please tag this matrix user, i use it more)
    matrixbot
    @matrixbot
    @janus.troelsen:matrix.org oh, seems like the check wasn't running because the content-type attribute was missing... but i had to single-step through schemathesis to find out
    matrixbot
    @matrixbot
    @janus.troelsen:matrix.org reported schemathesis/schemathesis#816
    1 reply
    arvidj
    @arvidj
    Hi, I'm trying out schemathesis in pytest. Seems useful! Is there an API documentation available somewhere? More specifically, I'm trying to add a timeout to each case generated by @schema.parametrize().
    1 reply