by

Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
    Gregory Markou
    @GregTheGreek
    :thumbsup:
    Danny Ryan
    @djrtwo
    Proposed state test format for review/discussion ethereum/eth2.0-test-generators#21
    Dmitry S.
    @zilm13
    @ChihChengLiang @mratsim @djrtwo I want to update shuffling test generators to be up-to-date with spec. Does it make any sense?
    there are new generators with permutated_index which doesn't reflects in compiled tests repo, but current shuffling tests are useless and outdated. We'd either remove them at all and leave only permutated tests or update shuffling, so it will duplicate permutated but with a bit of shuffling logic a-top.
    Mamy Ratsimbazafy
    @mratsim
    they changed again? :P
    Chih Cheng Liang
    @ChihChengLiang
    Suggest updating shuffling, merging permutated index test cases into shuffling, and removing permutated_index suite.
    Dmitry S.
    @zilm13
    Yeah, not much sense to have both permutated_index tests and shuffling. I will try to cook some pr thus
    Dmitry S.
    @zilm13
    here it is ethereum/eth2.0-test-generators#24
    Please, review
    Dmitry S.
    @zilm13
    @terenc3t Hi! I was told that you have some consensus tests, could you, please, provide some info?
    terence tsao
    @terenc3t
    Hmm I don't think we have consensus test at this point, we are still working on fork choice :)
    Dmitry S.
    @zilm13
    @terenc3t fork choice sounds cool too.
    any link?
    Mamy Ratsimbazafy
    @mratsim
    Dmitry S.
    @zilm13
    ahhh format only, no fixtures yet, got it
    Danny Ryan
    @djrtwo
    I am actively working on vectors that will be released for v0.5.0
    probably won’t have vectors for v0.4.0 because our spec was not executable for v0.4.0
    Dmitry S.
    @zilm13
    :+1: got it
    Dmitry S.
    @zilm13
    How could I run eth2.0-tests (not generators) with Python (Trinity, I guess)? I couldn't find any info neither in first repo nor in Trinity. Could you help me?
    Chih Cheng Liang
    @ChihChengLiang
    We haven’t integrate the tests to the Trinity yet. Is there a specific issue you are investigating into?
    Dmitry S.
    @zilm13
    yeah I want to debug current state tests
    the result doesn't match for me and I need some middle results
    doesn't match in our client
    Dmitry S.
    @zilm13
    btw, I want to add signed_root tests to eth2.0-test-generator's. Is it ok? Nobody doing this thing already?
    Chih Cheng Liang
    @ChihChengLiang
    Afaik, signed_roothaven’t been implemented. Feel free to do it :thumbsup:
    Dmitry S.
    @zilm13
    Cool, thx!
    Mamy Ratsimbazafy
    @mratsim
    @zilm13 I'm curious about what doesn't match, for now I just focused on parsing the YAML files which work fine.
    jannikluhn
    @jannikluhn
    @zilm13 if you do it in Python, feel free to reuse code from the SSZ test generator (in particular "render_type_definition" and "render_value" may be useful)
    jannikluhn
    @jannikluhn
    if not, I'd suggest to still use the same format for values/types (see ethereum/eth2.0-test-generators#13), to make life easier for the test runners
    Dmitry S.
    @zilm13
    @jannikluhn ok, I will check it
    @mratsim signed_root doesn't match, but I'm going to add generators for it, so it would be simpler for us and other clients to verify that we have the same logics for hashing. Next, for example, test_proposer_slashing doesn't match. I guess, there will be different results for get_crosslink_committees_at_slot (slashing values match, but validator order differs) etc
    Mamy Ratsimbazafy
    @mratsim
    ah right, yeah I asked to add hashes of the block, but afaik we need more fine-grained ssz/treehashes to guide implementations
    Dmitry S.
    @zilm13
    we just need more low level unit tests. state tests is higher level, we have not many tests at lower levels - ssz only ints, no even containers, shuffling, bls, and that's all.
    Mamy Ratsimbazafy
    @mratsim
    that should improve with the progress of executable specs and trinity
    jannikluhn
    @jannikluhn
    @zilm13 agree that we need much more testing. You can check out ethereum/eth2.0-test-generators#32 for a more complete SSZ test suite. Tree hashing tests are on the way as well, hope to be able to push something tomorrow.
    Dmitry S.
    @zilm13
    :+1: great PR, ssz containers are here! What is empty containers? Containers with no fields?
    jannikluhn
    @jannikluhn
    yes. I originally thought they should be valid (the spec doesn't say they're not), but in fact they should not be
    they're still in the tests though, which is why some of them fail
    will fix them once there's confirmation in the spec issue
    Dmitry S.
    @zilm13
    Btw, could we add name to every test case and require that it should be unique?
    You could always make something simple, say "ssz" + counter in generator if you don't have any, but it's more simple to point out to exact test case when it's failing, when you have some kind of id.
    jannikluhn
    @jannikluhn
    i guess we could
    in py-ssz we just use line numbers for this
    if we add this, we should probably add this to the general testing format, not only SSZ tests
    Dmitry S.
    @zilm13
    yeah, sure
    some tests are already with unique names, but others not
    jannikluhn
    @jannikluhn
    Dmitry S.
    @zilm13
    @jannikluhn when you are going to release py-ssz with Vector?
    jannikluhn
    @jannikluhn
    @zilm13 the current release and the one before include Vector already
    Dmitry S.
    @zilm13
    @jannikluhn this one not https://github.com/ethereum/eth2.0-test-generators/pull/34/files#diff-10153377b1e735f1ff7066fd8b51f0b7R2
    ohhh I see 2 new releases, need just to change versions
    Dmitry S.
    @zilm13
    As I understand https://github.com/ethereum/eth2.0-test-generators/ is abandoned now and all generators will be in spec.
    What about generated tests https://github.com/ethereum/eth2.0-tests ? closing too?
    Mamy Ratsimbazafy
    @mratsim
    The big tests will be put in git LFS because github is not really suitable for 1 million lines YAML files anyway