Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
    Brian Okken
    @okken
    [Ilesh, Test Podcast] :wave: I’m here! What’d I miss?
    Brian Okken
    @okken
    [Pax, Test Podcast] How would one approach testing that “when a connection is lost, the API should stop streaming”? I want to create a test this issue, https://github.com/encode/starlette/issues/20#issuecomment-586304354 --- I checked starlette’s tests, then looked at their testclient module. I see there is a WebSocketSession.Close (but I’m thinking this is the server closing the connection) whereas I need the client to be the one closing the connection. TestClient seems to not have a ‘close’ method or anything similar. Any tips for me?
    Brian Okken
    @okken

    [Tony Cappellini, Test Podcast] Should there be unusual behavior when running a simple program in a subprocess, when using pytest?

    I have a pretty simple loop which counts from 1-10, sleeping 1 second in between each iteration. The loop should take 10 seconds (plus minor overhead). pytest thinks it takes < 1 second to execute. When I run that loop on the cmd line (outside of pytest), the loop runs as expected

    ```from time import sleep

    finished = False
    max_loops = 10
    loop_count = 1

    while not finished:
    print(f'Subprocess running: loop {loop_count} of {max_loops}')
    sleep(1.0)
    loop_count += 1
    if loop_count > max_loops:
    finished = True

    print('\nLoop complete, subprocess will exit\n')```

    Brian Okken
    @okken
    [Tony Cappellini, Test Podcast] The other issue I have with running a subprocess from pytest is I can't make the -s option work so that I can see the output of the subprocess.
    Hal Wine
    @hwine
    Question on using Sphinx to document test code. Using the sphinx-apidoc approach, there are a number of import errors during rendering. (The tests execute just fine.) Does the autodoc approach tend to be more successful? AIUI, that method does not import the modules, but I've never used it.
    Brian Okken
    @okken
    [Israel Fruchter, Test Podcast] Hi guys, I have streaming API, I need to test.
    It's basically identical to dynamodb streams api https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/Streams.html,
    any oracles as for what are the things streams might be problematic with ? and worth testing ?
    Brian Okken
    @okken
    [Dima Spivak, Test Podcast] @fruch well obviously the way the API is intended to be used
    [Dima Spivak, Test Podcast] and then a whole bunch of negative cases of what happens when people give it the wrong parameters
    [Israel Fruchter, Test Podcast] LOL
    [Dima Spivak, Test Podcast] :)
    [Israel Fruchter, Test Podcast] I was looking for something more general for streaming data
    [Israel Fruchter, Test Podcast] I.e. there are lots of those types of API all around trying to handling streams of a data, just thought to ask maybe someone already has kind of experience like that
    [Israel Fruchter, Test Podcast] the guys at AWS didn't picked the phone :)
    [Dima Spivak, Test Podcast] I mean that it's streaming at the backend or not doesn't really affect how you approach it if you're testing an API
    [Israel Fruchter, Test Podcast] (or at least didn't shared they tests, or test plans :))
    [Dima Spivak, Test Podcast] you'd obviously want to do things with parallelism to make sure stuff doesn't fall over with many users or whatnot
    [Dima Spivak, Test Podcast] but maybe give us a sense of what you're thinking and we can critique it?
    [Israel Fruchter, Test Podcast] basically it a implementation of this exact API ontop of out dynamodb drop in replacement.
    Brian Okken
    @okken
    [Dima Spivak, Test Podcast] yep, got that part. I mean what sort of cases or scenarios do you have in mind to cover?
    [Israel Fruchter, Test Podcast] and out plan is to write a tool using https://github.com/awslabs/dynamodb-streams-kinesis-adapter, to duplicate the a table in realtime
    [Israel Fruchter, Test Podcast] while using YCSB to populated that table.
    [Dima Spivak, Test Podcast] gotcha. so that's more of a load test?
    [Israel Fruchter, Test Podcast] and then we could compare both, source and target versions
    [Israel Fruchter, Test Podcast] while we are stopping/killing nodes of our DB
    [Israel Fruchter, Test Podcast] i.e. with tandam of Chaos Monkey style, which we already have in place
    [Israel Fruchter, Test Podcast] we also have more specific functional tests
    [Dima Spivak, Test Podcast] yep, all seems reasonable. so this is less about testing the API than testing the behavior of your database?
    [Israel Fruchter, Test Podcast] that would cover the APIs
    [Dima Spivak, Test Podcast] (when I see "API test" I imagine more things around correctness and error handling than load in the face of fault injection)
    [Israel Fruchter, Test Podcast] I'm planning both testing the APIs in a more controlled and specific environment
    [Israel Fruchter, Test Podcast] but also in a more close to reality environment, with bigger amount of traffic going into the system
    [Dima Spivak, Test Podcast] gotcha
    [Dima Spivak, Test Podcast] so yeah, for APIs, the things most people forget are around malformed data
    [Dima Spivak, Test Podcast] payloads that are too big
    [Dima Spivak, Test Podcast] wrong datatypes being passed
    [Dima Spivak, Test Podcast] that sorta thing
    [Israel Fruchter, Test Podcast] FYI, those are the "unit-tests" the development guys were doing so far
    Brian Okken
    @okken
    [Dima Spivak, Test Podcast] oh man, so nice to see pretty pytest work =D
    [Israel Fruchter, Test Podcast] yeah, it is
    [Israel Fruchter, Test Podcast] it's not mine, but I do approve most of it :)
    [Israel Fruchter, Test Podcast] whom wrote it, it's the first time he's touching pytest (or python for that matter)...
    [Israel Fruchter, Test Podcast] but he's really experienced developer, like a 20x one :)
    [Dima Spivak, Test Podcast] :) seeing error != None I thought "this guy might be a little newer to Python"
    [Dima Spivak, Test Podcast] there's that muscle memory of having read pep-8 90 times that shows in code looking "pythonic"
    [Israel Fruchter, Test Podcast] his day job, he's doing C++... so yeah :)
    [Israel Fruchter, Test Podcast] So one take I have from you @dima is approaching from the angle of a "naive" developer angle. (and make it "idiot" proof) :)
    Brian Okken
    @okken
    [Dima Spivak, Test Podcast] absolutely
    [Dima Spivak, Test Podcast] the most common issues that surface are from people who don't rtfm
    [Dima Spivak, Test Podcast] so having your application respond to that gracefully is important