by

Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
    Marcel Lanz
    @marcellanz
    Screenshot 2020-09-17 at 16.35.59.png
    The in-official "TCK Nightly Verification" this night for the first time failed.
    😃 awesome
    Adriano Santos
    @sleipnir
    I think a new tck has been released and some things have broken
    Marcel Lanz
    @marcellanz
    cool... so things are happening
    Adriano Santos
    @sleipnir
    This report should definitely be included directly in the main project. It is very good to have this visibility
    Marcel Lanz
    @marcellanz
    There is a ticket to get structured output from the TCK. The nightly TCK run maintains a report available here:
    https://gist.githubusercontent.com/marcellanz/a9cf0dffb01d1a83ba2bbf2ac52166fe/raw/cloudstate_go-support-tck-run.json
    errors included.
    I think LB has itself some quality checks with their CI running. Unfortunately I had not enough time to go further with this for the Community project.
    Adriano Santos
    @sleipnir
    Yes, I imagine but this report is the best
    Marcel Lanz
    @marcellanz
    If anyone likes to pick up; I think a report via e-mail would be nice. It runs a travis job defined here:
    https://github.com/marcellanz/cloudstate/blob/nightly/tck_run/.travis.yml
    Adriano Santos
    @sleipnir
    cool
    great work
    Guy Youansi
    @ralphlaude
    @marcellanz good work!
    Marcel Lanz
    @marcellanz
    thanks guys
    … old stuff :)
    It also seems to have a bug.
    Adriano Santos
    @sleipnir
    I haven't been able to look at the logs yet, any suspicions?
    Marcel Lanz
    @marcellanz
    This one seems to fail:
    ""verify get cart, add item, remove item, and failures"
    https://github.com/cloudstateio/cloudstate/blob/master/tck/src/main/scala/io/cloudstate/tck/CloudStateTCK.scala#L456
    those are all new tests by the work on the "new" TCK. Don't yet see why it fails.
    Adriano Santos
    @sleipnir
    hummmm...
    Thank's I'll evaluate later
    Marcel Lanz
    @marcellanz
    Sure, actually I expect more issues, the original TCK was also kind of simple. The new tests are quite exhaustive. Great work by Peter.
    Adriano Santos
    @sleipnir
    \0/
    Thank you @pvlugter for breaking everything :D
    That's good, now we have more robust tests
    Marcel Lanz
    @marcellanz
    Also, I think he works on the CRDT TCK tests; they are even more comprehensive from what I saw with the functional tests being done.
    Adriano Santos
    @sleipnir
    awesome
    Marcel Lanz
    @marcellanz
    I would have really liked to work on the TCK for the CRDT model, but never got into Scala so far (I think I can read ir, but not write). Did all of them then in Go 😅
    Adriano Santos
    @sleipnir
    Scala is difficult to read, but humanly possible, writing is another story. I don't think even skilled programmers in Scala know what they do for sure :D
    Calm it was a joke
    Marcel Lanz
    @marcellanz
    I just never learned it, can't say anything about it.
    Peter Vlugter
    @pvlugter
    I'll take a look at the TCK failures. It looks like it's just against the native image proxy that it fails. This is tested in the cloudstate main repo integration tests. Interesting looking at the nightly build matrix: https://observablehq.com/@marcellanz/cloudstate-go-tck-verification as it has cloudstate-proxy-native-dev-mode:0.5.1 passing but cloudstate-proxy-native-dev-mode:latest failing. These are actually the same image (we don't have CI builds published of the native image versions): https://hub.docker.com/r/cloudstateio/cloudstate-proxy-native-dev-mode/tags — so it's weird that it would consistently pass on 0.5.1 but not on the same image with latest tag.
    Adriano Santos
    @sleipnir
    Strange very strange
    Thanks @pvlugter
    Peter Vlugter
    @pvlugter
    Okay, I think I see what the issue is. It should actually fail for both the native and non-native proxy for 0.5.1. The TCK now tests failures in the shopping cart (that's the only change to that test) and the failure handling in the proxy was broken earlier so that it didn't return failures properly (but reset the connection). The TCK test uses the failure message from the client verified against what the proxy saw from the user function (we don't know what the exact failure message will be, varies for shopping cart implementations, but it checks these are the same).
    @marcellanz, you may want to check the scripts, as I think it should be failing more in that TCK verification actually.
    Adriano Santos
    @sleipnir
    Makes sense
    Peter Vlugter
    @pvlugter
    So, to address this, we can either make the latest TCK handle the old proxy as well. Or we could always only verify TCK 0.5.1 and proxy 0.5.1 together, and TCK latest and proxy latest together (and I'll get a new native proxy published for latest).
    Adriano Santos
    @sleipnir
    I think it makes sense to use the same version between proxy and tck
    Peter Vlugter
    @pvlugter
    Yeah, agree
    Adriano Santos
    @sleipnir
    Cool
    Peter Vlugter
    @pvlugter
    I've manually published a new cloudstate-proxy-native-dev-mode:latest and added docker image publishing to the circleci build for the native-image proxies.
    Peter Vlugter
    @pvlugter
    @marcellanz, and ignore what I said. The verification scripts do, of course, already match proxy and TCK version — that was just the extra combinations I ran locally, not the script. With the updated native-image proxy for latest, should all be good now.
    Marcel Lanz
    @marcellanz
    cool, thanks for looking into it @pvlugter
    Peter Vlugter
    @pvlugter
    So for the new TCK tests, it checks which test services are implemented, and marks tests as pending if they're not implemented. So it could be useful to include the pending tests in the TCK verification visualisation as well. Tried it out, with marking pending as orange:
    tck-verification-with-pending.png
    Marcel Lanz
    @marcellanz
    nice 👍
    Adriano Santos
    @sleipnir
    Thanks for the work @pvlugter
    Now let's work to make all the tests green
    Viktor Klang (√)
    @viktorklang
    :+1: on that suggestion, Adriano! :D