People
Activity
    John Erickson
    @johnterickson
    3) I'd like to avoid wrapping cargo with another executable, though I think it would make sense for a "cargo test" build task in Pipelines to also publish the test results to Azure DevOps
    4) I'm relatively new and not running Rust in production, so I'm open to hearing other thoughts and also do take my opinions with a grain of salt :)
    Nick Babcock
    @nickbabcock
    Thanks @johnterickson , you guys have really gone above and beyond
    Andrey Cherkashin
    @andoriyu
    @epage Yeah, I can do that. Just point to master?
    btw, I've submitted PR to junit-report-rs that should make it usable in suity.
    @johnterickson I think it should emit JUnit. It's commonly supported and has very clear schema.
    Ed Page
    @epage
    Good timing @johnterickson , I've been meaning to create an issue for discussing testing's needs
    crate-ci/resources#9
    Andrey Cherkashin
    @andoriyu
    funny, originally suity started as cargo json -> junit xml, but then I wanted automate multiple workflows and get away from sh scripts.
    Shame cargo's json missing more detailed information on what this testcase is (doc test, unit test, etc)
    Ed Page
    @epage
    Yeah, that is unfortunate. The best we have is the file name
    Maybe post on rust-lang/rust#49359 about it?
    Andrey Cherkashin
    @andoriyu
    @epage I'm trying you use CargoTest and a bit confused how to use it.
    Andrey Cherkashin
    @andoriyu
    There is no way to run unit tests with this?
    Ed Page
    @epage
    sigh Is cargo build --tests not reporting a valid executable for unit tests?
    Andrey Cherkashin
    @andoriyu
    nope
    --tests here implies integration tests.
    I honestly have no clue how to run non-integration tests other than cargo test
    Ed Page
    @epage
    I thought I've seen --tests report compiler errors for unit tests
    I'll need to look at this more later tonight when I'm off work.
    Andrey Cherkashin
    @andoriyu
    It compiles them, but unit tests are part of the binary.
    As in it's not a separate binary for tests, it's right there in library/bin binary.
    Oh, also, I wasn't able to pass features after .tests()
    I don't have an error message handy, but I get that for you when I get off work.
    Christopher Durham
    @CAD97
    Unit tests are generated in a new binary, but they're cleaned up immediately IIRC
    Andrey Cherkashin
    @andoriyu
    ah
    Ah, right.
    So the right way to compile them would be cargo test --no-run
    @epage I think you better off mimicking cargo test, rather than build tests and them run them.
    Ed Page
    @epage
    Fun
    Hopefully that works
    Ed Page
    @epage
    So looking further, I'm seeing the same output between cargo build and cargo test --no-run. The problem in escargot is that the unit tests have the kind set to lib. I assumed the kind would be marked as bin. It'll need a couple tweaks but should provide the results
    As for doc tests, I'm not seeing anything messages for those. I suspect there are extra interactions happening to make those work
    Andrey Cherkashin
    @andoriyu
    doc tests are really weird
    I don't thing there is a code coverage tool that supports them neither.
    @epage have you though about json schema proposal for libtest?
    I guess I will fork junit-report at the end of week
    Andrey Cherkashin
    @andoriyu
    @epage it's probably depends if unit test is in the binary or library?
    Ed Page
    @epage
    Yeah, the kind is the root klind is came from, not the kind of compile-artifact that was generated.
    I just need to set aside time to change the filtering logic for cargo test
    And I have ideas on how to address doc tests.
    As for schema proposal for libtest, I've not taken the time to get that specific; just the high level direction I'd like to see it go
    Ed Page
    @epage
    @andoriyu got unit test support in a PR
    crate-ci/escargot#26
    I suspect we don't need libtest telling us what kind of test is being run. We could instead augment CargoTest to capture the kind and allow the user to check it. This will be bin or lib for unit tests and test for integration tests
    Ed Page
    @epage
    I've taken a quick glance at doctests and have ideas recorded at crate-ci/escargot#27 but I think in the mean time, if someone is meant to exclusive run cargo-suity, it might be good to just wrap cargo test --docs in the most naive way possible.
    Andrey Cherkashin
    @andoriyu
    @epage do tests inside file for binary have lib kind as well?
    Ed Page
    @epage
    tests from a main.rs have a bin kind. tests from a lib.rs have a lib kind.. Tests from integration tests have a test kind.
    That clarify?
    Andrey Cherkashin
    @andoriyu
    Yup
    I want to work on suity this weekend, but accidentally triggered TPM lockdown on my second laptop that my wife use to edit photos and never backup the results... So all of my machines now brute forcing that damn recovery key in case IT that set it up doesn't have key.
    Ed Page
    @epage
    Ouch. Good luck with that. btw a crate I should probably make soon is clap-cargo, a crate of cargo CLI flags to make it easier for cargo plugins to get a consistent experience (documentation, etc). Pretty trivial for people to implement the flags on their own but I suspect it will be helpful.
    Andrey Cherkashin
    @andoriyu
    nice