Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
    Dmitry Dygalo
    @Stranger6667
    If the error comes from the headers issue , then you can pass headers to ‘from_uri’ function as a dictionary
    Rakesh1111
    @Rakesh1111
    @idoko_gitlab : if I place the same file in the local file system under my home directory in computer, it works with same headers. If I try to place the swagger file in the AWS S3 location, with the same headers which works in above case .. it doesn’t work here
    @idoko_gitlab : in the above two cases I am using the schemathesis run command from terminal
    Rakesh1111
    @Rakesh1111
    @Stranger6667 : Failed to load schema, code 400 was returned from https://<<bucketname>>.s3-<<regionname>>.amazonaws.com/swagger2.json
    Dmitry Dygalo
    @Stranger6667
    Thanks for sharing the message. Re local file I think that if you load the schema from your file system, then the headers don’t matter - it is a different use case. The code under the hood boils down to requests.get(“...address...”) - my assumption is that you need to pass an additional Authorization header to access your bucket
    Or some query parameters
    Rakesh1111
    @Rakesh1111
    Programatically using the below approach:
    events = runner.prepare(url,
                            headers=headers,
                            checks=[not_a_server_error, response_schema_conformance], base_url=baseUrl)
    Dmitry Dygalo
    @Stranger6667
    8 replies
    Rakesh1111
    @Rakesh1111
    url Is the location of s3 file, header include accept-version, x-api-key, "Authorization": "Bearer " + token.
    Dmitry Dygalo
    @Stranger6667
    You could try to run requests.get with your url and headers, then check the response content - probably it will contain some reassigning for 400 response status. At the moment I am not able to reproduce this behavior locally - it will require some more time
    1 reply
    partizan
    @last-partizan

    Hi all.
    I'm trying to use schemathesis with pytest, making asgi_call instead of call. Should i populate my test database with data before running tests? or can i use pytest fixtures?

    I found before_process_path hook, and there is database access for populating data, but how i can use pytest fixtures there?

    3 replies
    Michael Okoko
    @idoko_gitlab
    Hi, I'm running running my tests against the all checks, but I was wondering if there's a way for it to not fail on 500 errors as long as the HTTP 500 error is documented
    4 replies
    samuel guesnier
    @samuelguesnier
    Hello everyone :)
    Can someone help me pleae ? I'd like to save all requests in cassette, but I don't know how to do that
    import requests
    import schemathesis
    from hypothesis import settings
    schema = schemathesis.from_path(path="sam.yaml",validate_schema=False,base_url="https://baseurl.com")
    
    
    
    
    
    @schema.parametrize( endpoint="/subscriptions/{subscriptionUid}/devices", method="POST")
    @settings(max_examples=1)
    def test_no_server_errors(case):
        case.query['sessionToken'] = 'ad3414b8becbb1e9'
        case.path_parameters["subscriptionUid"] = 1384
        response = case.call()  
        case.validate_response(response)
    1 reply
    samuel guesnier
    @samuelguesnier
    Maybe I must use prepare function ?
    1 reply
    samuel guesnier
    @samuelguesnier
    Another question: is it possible to pass query paramers / body on CLI ?
    Ho, I found an answer :p kiwicom/schemathesis#695
    Ramesh
    @Rameshpvg_gitlab
    Hi,
    I am new here, I just started looking at schemathesis.. Looks like a great library. I would like to explore more.
    Could somebody please show me where can I find more examples on from_file, from_path, from_dict usage.
    5 replies
    samuel guesnier
    @samuelguesnier
    Hello eveyone ! :)
    Is Schematesis is reliable enough to be used all alone for testing my API or is it only used to find corner case ?
    Another question: Schemathesis generates right data according to the schema and also wrong data ? If so, is it possible to know when Schemathesis is generating right datas ? For example: If case contain wrong data && response_code == 200 then Fail .
    4 replies
    panix
    @alex-pancho
    Question: How can I get an extended test report? So that I could run the schemathesis on my api, he did 4,000 tests, wrote that everything was ok, but what exactly was it for the tests?
    7 replies
    vkutepov
    @vkutepov
    This message was deleted
    1 reply
    vkutepov
    @vkutepov

    and one more question
    i understand that schemathesis
    not made for this
    But if I want to use it for an integration test
    if I want to call one service and then another in the same test
    can I create schema1 and schema2?
    if yes then two case fixtures should be created

    like that
    or maybe it can be split into separate fixtures

    schema1 = schemathesis.from_uri(
        "http://my_service1.stg.com:84/swagger.json"
    )
    schema2 = schemathesis.from_uri(
        "http://my_service2.stg.com:84/swagger.json"
    )
    
    
    def test_1(case1, case2):
        case1.endpoint = "/get_result"
        case1.method = "GET"
        response1 = case1.call()
    
        case2.endpoint = "/send_result"
        case2.method = "POST"
        case2.body = response1.json()
        response2 = case2.call()
    12 replies
    panix
    @alex-pancho
    вопросы
    10 replies
    Dmitry Dygalo
    @Stranger6667
    Hi all! We are gathering feedback on Schemathesis - https://forms.gle/arTf6yQEcQH8ShxHA . It will help me to make improvements to it and prioritize new features! :) It will take 5 minutes. The results are anonymous.
    Janus Troelsen
    @ysangkok
    i am having a problem where i call validate_response but it doesn't run the check that the response conforms to the schema. how do i programmatically add that check?
    matrixbot
    @matrixbot
    @janus.troelsen:matrix.org (this is also me, please tag this matrix user, i use it more)
    matrixbot
    @matrixbot
    @janus.troelsen:matrix.org oh, seems like the check wasn't running because the content-type attribute was missing... but i had to single-step through schemathesis to find out
    matrixbot
    @matrixbot
    @janus.troelsen:matrix.org reported schemathesis/schemathesis#816
    1 reply
    arvidj
    @arvidj
    Hi, I'm trying out schemathesis in pytest. Seems useful! Is there an API documentation available somewhere? More specifically, I'm trying to add a timeout to each case generated by @schema.parametrize().
    1 reply
    arvidj
    @arvidj
    More questions:
    • is it possible to exclude certain endpoints (in pytest or using the schematest binary)
    • for one of my apis, the schematest binary just sits there with no output. I've played a around with most options (and all related to verbosity), but still no output. I tried running schematest in strace. It seems like it is stuck in an allocation loop. It's allocating more than 1gig of memory.
    5 replies
    Any ideas on how to debug that?
    matrixbot
    @matrixbot
    @janus.troelsen:matrix.org @Dimitry is there a way to totally override the strategy for an endpoint? because i found that it is spending too much time fuzzing the first endpoint. there is too much randomness, i wanna write a custom strategy that will be quickly exhausted. there is Endpoint.make_case but that is not for replacing the strategy
    1 reply
    DanielXu77
    @DanielXu77
    Hi, new to schemathesis here. I'm just wondering is there a way to provide custom TLS certificate(e.g. https://urllib3.readthedocs.io/en/latest/advanced-usage.html#custom-ssl-certificates) or choose not to verify server's TLS certificate(e.g. https://2.python-requests.org/en/master/api/#requests.Session.verify) ?
    16 replies
    matrixbot
    @matrixbot
    @janus.troelsen:matrix.org DanielXu77 (Gitter): you could probably use get_call_kwargs to pass something to the requests library that would make it ignore the certificate
    1 reply
    DanielXu77
    @DanielXu77
    Also another related question: Does schemathesis support HTTP/2 ? Is there any way to enable or force using HTTP/2 ?
    matrixbot
    @matrixbot
    @janus.troelsen:matrix.org DanielXu77 (Gitter): schemathesis works with a few HTTP providers, i believe requests is the default. so you should figure out if requests uses HTTP/2
    DanielXu77
    @DanielXu77
    Thanks all! That really helps a lot
    Artem Surov
    @artemsurov
    Hi! Thank you for schemathesis, it's make life much easier. But I have problems with a few tests:
    1. hypothesis.errors.DeadlineExceeded: Test took 1025.33ms, which exceeds the deadline of 500.00ms
    2. hypothesis.errors.FailedHealthCheck: It looks like your strategy is filtering out a lot of data.
    3. hypothesis.errors.FailedHealthCheck: Data generation is extremely slow: Only produced 0 valid
      What can I do to fix this tests?
    4 replies
    Artem Surov
    @artemsurov
    And one more error!
    hypothesis.errors.Unsatisfiable: Unable to satisfy assumptions of hypothesis test_api.
    3 replies
    Artem Surov
    @artemsurov
    And one more thing:
    When copy/paste suggested by schemathesis requests code, i.e
    """
    Run this Python code to reproduce this response:
    requests.post('http://localhost/api/users/', headers={'Authorization': 'KÂ', 'User-Agent': 'schemathesis/2.7.1', 'Cookie': 'Session=', 'Content-Length': '48', 'Content-Type': 'application/json'}, body=b'{"id": 22489, "is_staff": true, "&": {"": null}}')
    """
    Requests return error: TypeError: request() got an unexpected keyword argument 'body'
    3 replies
    vlade11115
    @vlade11115
    Hi all!
    Thank you all for the great tool, I very much love it.
    Is there any way to completely disable hypothesis.errors.DeadlineExceeded: and all errors related to slow or Flaky test cases?
    In my organization, we want to test AWS Lambda endpoints, on a fresh sandbox account. In our case, we will hit the "cold start" problem, when response time is unpredictable. For tests on sandbox this is acceptable for us, but sadly not for schemathesis.
    For example:
    schemathesis run --base-url=http://0.0.0.0:5000/ --workers 8 --hypothesis-suppress-health-check=too_slow --checks all requirements/openapi.json
    As I understand from https://schemathesis.readthedocs.io/en/stable/cli.html#tests-configuration --hypothesis-suppress-health-check=too_slow should completly disable slow test check. But, output of command contains:
    =================================================================================== ERRORS ==================================================================================
    ___________________________________________________________________ GET: /assessment/{env}/{assessment_id} __________________________________________________________________
    hypothesis.errors.DeadlineExceeded: Test took 533.43ms, which exceeds the deadline of 500.00ms
    5 replies
    R. Tyler Croy
    @rtyler
    I feel like I must be doing something foolish here, when I specify a requestBody to send along with a value, schemathesis appears to be putting the value value in double-quotes before sending it along
    R. Tyler Croy
    @rtyler
    aha, it looks like schemathesis thinks my value is json, depite the example being text/plain
    The code to reproduce via requests uses the json="my value" keyword rather than data="my value"
    R. Tyler Croy
    @rtyler
    went ahead and filed schemathesis/schemathesis#850 since I'm pretty sure this is a bug
    3 replies
    Francis Le Roy
    @GrandChaman_gitlab

    Hi ! First, the tool is amazing, thank you !
    I've noticed a bug (don't know if it's my fault or not) but can't seem to pinpoint where it's coming from.

    I get some

    hypothesis.errors.Flaky: Tests on this endpoint produce unreliable results: 
    Falsified on the first call but did not on a subsequent one

    on a random basis, even with the same hypothesis seed.
    When looking at the network log, the request and the responses are exactly the same the first and second time. I don't know what triggers that exception and the lack of verbosity doesn't help me here :/

    14 replies
    Øyvind Grimnes
    @Oyvindkg

    Hey, Dmitry! I'm trying out the CLI and I'm impressed. I have one, somewhat essential, issue though – when running with --stateful=links, it doesn't seem to pick up on links inside referenced responses.

    E.g.:

    ...
        '200':
            $ref: '#/components/responses/MyResponseWithLinks'
    ...

    The links in MyResponseWithLinks are not followed.
    Aren't links in referenced responses supported, or am I missing something? :)

    10 replies
    Øyvind Grimnes
    @Oyvindkg

    We have an endpoint for uploading files. We're using externalValue (https://swagger.io/docs/specification/adding-examples/) to provide relevant example binaries. At the moment it causes a fatal error because the more common value property doesn't exist. I realise that it's probably a less common use case, but would it be feasible to add support for external values?

    ...
      File "/usr/local/Caskroom/miniconda/base/envs/ahtohallan/lib/python3.7/site-packages/schemathesis/specs/openapi/examples.py", line 58, in get_request_body_examples
        "examples": [example["value"] for example in schema.get(examples_field, {}).values()],
      File "/usr/local/Caskroom/miniconda/base/envs/ahtohallan/lib/python3.7/site-packages/schemathesis/specs/openapi/examples.py", line 58, in <listcomp>
        "examples": [example["value"] for example in schema.get(examples_field, {}).values()],
    KeyError: 'value'

    I can try to cobble together a PR myself, if that helps :)

    5 replies
    vlade11115
    @vlade11115

    Hi all.
    I hit some strange issue, schemathesis generating invalid JSON payload, and producing SyntaxError.
    For example:
    Schema (relevant part of actual docs).

    Collection:
          title: Collection
          description: Collection is set of datapoints, connected to transaction
          type: object
          properties:
            transaction_id:
              type: string
              format: uuid
              description: "Transaction id"
              example: "6b19feeb-7433-448e-b285-b758d19a4fe6"
            collection_id:
              type: string
              format: uuid
              description: "Collection id"
              example: "8109c79a-e50b-46b8-8032-0d6e556dfaa0"
            data:
              type: object
              description: Data in  language schema
              additionalProperties: true
            metadata:
              type: object
              description: Metadata about collection, f.e version of used  shema

    And schemathesis produced this line: (I remove sensitive data lile URL and x-api-key)

    requests.put('', headers={'x-api-key': '', 'User-Agent': 'schemathesis/2.8.0'}, json={'data': {'!ù\x05': [{'A\U000b5b33À`\x99\uf555ü\x02': '\x94', 'Ø\U0009648b\x89è\x92\x9c\x80': {}, '\x1fÏüÞ\x80_\tÒõì': []}, {}], 'Ë\x93': 2.637258487788318e+43, '': [{}, [[None], True, [-9007199254740992.0]], {}], '\U0003ff6b5\U00109318ã\x1f\uf68f': [[]], 'ì\x1a\x9a': {}, '\U0006e270í¤Ü\U00104b40§ÍÈ𘚿\U00071eacPB9k\U000a2c16': {'\U00102941dü1\U0009e7b1!\U000434bc\x08\r\U000f5c7c\U000aed63\U00035d6a\U000f9359': -78, 'Y\U0003b158©\x7f#\tâ\U000970af\U00074f1c': {}}, '\x9b\x9eÿ': {}, 'õÒè<\x05Ö2B¤v\U00065c6eù¤\U00104e66\U000ced18%\xa0"dw\x17ª\x92ÏR\U00074d07\U000a5eed': {'Çñ¼¶þë´\x87': {}, '': -16687}, 'Z\x87\x8c]5\x0fyR\x86\U001081cb𬑢ã\U000e478f&': [{}, {}], '\x00\x87jzI ': [False, [] {'': None, 'ç': 0.0}], '\x93<': {'\n\uf00d': [{'¥醆§Þ\x13³\U0008df5d\U000628e9êÔ': -1.5, '\x935Áé\U0004c8ad§#|': '\x9b|¯[\x9c\U0003b32dB\x07«', '06\x9e*\U0002fd6d\U000bef91¿·Ý)©ù': -24940}], 'íZ\000f7c8a\x95\U0004715f\x8d+-': {}, 'íT)¬': -6.997965566385589e+16}}, '\U00092506\U000d42a5¸µ*\U0007446b]': {'õñ\x1c>x¼\U000f43b8': False, '\U00083dee': {}, 'l\U00057240\U000b80f9': {'ÌO\x11': [None, '\x85', -2.854204201425003e+203]}}, 'metadata': {'ôW\U0009287d\U000b3e18\x17\x02ÑDO\U0001e216)æ': [[None, [False]]]}})

    Output missing comma, [False, [] {'': None, 'ç': 0.0}],.
    Should I open issuse about it? If we copy this line and paste it to python terminal, it will produce SyntaxError.

    3 replies
    Dmitry Dygalo
    @Stranger6667
    Folks, we are doing research about Schemathesis's effectiveness. If you have an API and you'd like to have it extensively tested by different Property-based testing tools - let me know :) It could be open/closed source, Python, or any other language, what matters is that it should have Open API 2/3 or GraphQL schema available :)
    9 replies