I'm looking into the shuffling tests and realized that they were not in sync with the latest spec as they used get_permuted_index instead of get_shuffled_index: https://github.com/ethereum/eth2.0-specs/search?q=get_permuted_index&unscoped_q=get_permuted_index
It seems like ethereum/eth2.0-specs#1045 was not complete @protolambda
Some feedback on the presets. On Nim side it's very easy to use them at compile-time. We pass a compile-time switch to the compiler to tell it to import one preset or the other: status-im/nim-beacon-chain#269.
We are passing the shuffling official shuffling minimal and mainnet shuffling tests for example
We've finished adoption of latest github tests harmony-dev/beacon-chain-java#144 . There are still two cases which we ignore, but first is definitely minor, the other one is more about infrastructure and isn't associated with particular test case, so I could say we are passing all tests from https://github.com/ethereum/eth2.0-spec-tests now.
I've found several issues with our spec implementation during adoption of tests, so the core goal of any tests is reached here! Thanks @protolambda for a big amount of work in tests development and building skeletons/core classes to simplify tests development.
5. Running tests in bitvector/invalid/bitvec_2_zero_3... FAILED 6. Running tests in bitvector/invalid/bitvec_4_zero_5... FAILED 11. Running tests in bitvector/invalid/bitvec_1_zero_2... FAILED 14. Running tests in bitvector/invalid/bitvec_3_zero_4... FAILED 16. Running tests in bitvector/invalid/bitvec_4_random_5... FAILED 22. Running tests in bitvector/invalid/bitvec_5_zero_6... FAILED