by

Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
    Dmitry S.
    @zilm13
    doesn't match in our client
    Dmitry S.
    @zilm13
    btw, I want to add signed_root tests to eth2.0-test-generator's. Is it ok? Nobody doing this thing already?
    Chih Cheng Liang
    @ChihChengLiang
    Afaik, signed_roothaven’t been implemented. Feel free to do it :thumbsup:
    Dmitry S.
    @zilm13
    Cool, thx!
    Mamy Ratsimbazafy
    @mratsim
    @zilm13 I'm curious about what doesn't match, for now I just focused on parsing the YAML files which work fine.
    jannikluhn
    @jannikluhn
    @zilm13 if you do it in Python, feel free to reuse code from the SSZ test generator (in particular "render_type_definition" and "render_value" may be useful)
    jannikluhn
    @jannikluhn
    if not, I'd suggest to still use the same format for values/types (see ethereum/eth2.0-test-generators#13), to make life easier for the test runners
    Dmitry S.
    @zilm13
    @jannikluhn ok, I will check it
    @mratsim signed_root doesn't match, but I'm going to add generators for it, so it would be simpler for us and other clients to verify that we have the same logics for hashing. Next, for example, test_proposer_slashing doesn't match. I guess, there will be different results for get_crosslink_committees_at_slot (slashing values match, but validator order differs) etc
    Mamy Ratsimbazafy
    @mratsim
    ah right, yeah I asked to add hashes of the block, but afaik we need more fine-grained ssz/treehashes to guide implementations
    Dmitry S.
    @zilm13
    we just need more low level unit tests. state tests is higher level, we have not many tests at lower levels - ssz only ints, no even containers, shuffling, bls, and that's all.
    Mamy Ratsimbazafy
    @mratsim
    that should improve with the progress of executable specs and trinity
    jannikluhn
    @jannikluhn
    @zilm13 agree that we need much more testing. You can check out ethereum/eth2.0-test-generators#32 for a more complete SSZ test suite. Tree hashing tests are on the way as well, hope to be able to push something tomorrow.
    Dmitry S.
    @zilm13
    :+1: great PR, ssz containers are here! What is empty containers? Containers with no fields?
    jannikluhn
    @jannikluhn
    yes. I originally thought they should be valid (the spec doesn't say they're not), but in fact they should not be
    they're still in the tests though, which is why some of them fail
    will fix them once there's confirmation in the spec issue
    Dmitry S.
    @zilm13
    Btw, could we add name to every test case and require that it should be unique?
    You could always make something simple, say "ssz" + counter in generator if you don't have any, but it's more simple to point out to exact test case when it's failing, when you have some kind of id.
    jannikluhn
    @jannikluhn
    i guess we could
    in py-ssz we just use line numbers for this
    if we add this, we should probably add this to the general testing format, not only SSZ tests
    Dmitry S.
    @zilm13
    yeah, sure
    some tests are already with unique names, but others not
    jannikluhn
    @jannikluhn
    Dmitry S.
    @zilm13
    @jannikluhn when you are going to release py-ssz with Vector?
    jannikluhn
    @jannikluhn
    @zilm13 the current release and the one before include Vector already
    Dmitry S.
    @zilm13
    @jannikluhn this one not https://github.com/ethereum/eth2.0-test-generators/pull/34/files#diff-10153377b1e735f1ff7066fd8b51f0b7R2
    ohhh I see 2 new releases, need just to change versions
    Dmitry S.
    @zilm13
    As I understand https://github.com/ethereum/eth2.0-test-generators/ is abandoned now and all generators will be in spec.
    What about generated tests https://github.com/ethereum/eth2.0-tests ? closing too?
    Mamy Ratsimbazafy
    @mratsim
    The big tests will be put in git LFS because github is not really suitable for 1 million lines YAML files anyway
    Eth2.0-tests will stay for issues, project management and maybe handcrafted tests of some edge cases.
    Dmitry S.
    @zilm13
    is there an url for git LFS already?
    we are going to move to the new tests, but I don't understand, should we build them from generators or there will be generated like in the past
    Mamy Ratsimbazafy
    @mratsim
    Not yet, I suppose it will be at the same time as 0.6
    Dmitry S.
    @zilm13
    Got it. Thank you!
    Gregory Markou
    @GregTheGreek
    Git LFS doesn't change the repo url
    It just allows the current repo to support really large files
    Dmitry S.
    @zilm13
    So everything will be pushed to eth2.0-tests? Just a matter of time for CI setup
    Gregory Markou
    @GregTheGreek
    I would assume so
    But as mamy said theyre planning to release the new tests with 0.6
    Mamy Ratsimbazafy
    @mratsim
    @protolambda, you said to contact you on Monday to use the test generators in eth2.0 specs, was off yesterday but available now and the rest of the week. Any tests you would like to target in particular?
    If it was up to me I'd like the SSZ/SOS PR merged to generates some test vectors from there.
    Diederik Loerakker
    @protolambda
    Hey
    At airport now, will be back online for eth2 things in ~12h or so
    Goal is to get more input/contributions from implementers in spec testing (others are working on network testing etc).
    We have some tooling in place to write tests. A POC spec based deposits test. (There's an ongoing effort to unify setup of test scenarios between pytests and yaml gen, do others are on hold for a bit), and then we have updated shuffling + ssz tests
    Diederik Loerakker
    @protolambda
    Also, looking to deprecate old tests repo, to just have LFS from the start, and not have non-LFS yaml files in git history forever.
    And there's a good chance the spec-generators focus on spec tests, and we'll have network, execution, and other test types coming from elsewhere in the future. So it makes sense to not force a single tests repo
    Mamy Ratsimbazafy
    @mratsim
    sounds good to me
    specs tests is quite largeanyway:
    • state tests
    • BLS
    • shuffling
    • SSZ
    • fork choice