Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
  • Oct 21 2017 21:24
    @jpitts banned @Musk55
Andrei Maiboroda
@gumb0
I'd say it's very important to execute all the previous fork rules, especially when client developers make changes in consensus-critical code (which is all the time during new fork development)
winsvega
@winsvega
the sealed test suite is available for execution. not available for changes
so not to break any of the previously created tests
Danno Ferrin
@shemnon
So would the older test suites be capped at a specific fork? i.e. when balances change EIPs are written do a new suite of tests need to be written for that fork and that fork alone?
winsvega
@winsvega
yes. will be capped at current ConstantinopleFix
all the test cases remain to be executed on new forks.
new tests could still be added for previous forks
Danno Ferrin
@shemnon
what about baseline “dose your evm function sensibly” tests, will there need to be new ones for each new fork or will some subset of the tests be updated for each fork? Or will stuff like balance checks be removed for that set of tests?
winsvega
@winsvega

all tests will be updated for each new fork. but not for previous forks. (right now it's updated for all forks with each new fork and new change)

postConditions for balance better be changed to smth more reliable and not conflicting if possible. although the fork interference might be resolved with Fortname + EIP + EIP chain params scheme where a specific test is executed only on a specific eip applied on top of fork. But this also brings up a question how a particular test case functions when many EIPs applied simultaneously

Danno Ferrin
@shemnon
I think the fork+EIP model is fine for development but I have serious doubts about the scalability for released forks.
James Hancock
@MadeofTin
The original proposal was to have fork+eip1+eip2 respect the ordering
So fork+eip2+eip1 would execute a different order of tests
rain
@rainbreak
hey, what's the story with these tests: RevertPrecompiledTouch_storage_d0g0v0|RevertPrecompiledTouch_storage_d3g0v0? (for petersburg) I can see that geth has them as known failures, but not sure why they're still in the test suite.
rain
@rainbreak
i'm guessing it's to do with special casing of ripemd, following https://github.com/ethereum/go-ethereum/pull/3341/files#diff-2433aa143ee4772026454b8abd76b9dd, but not totally sure
Andrei Maiboroda
@gumb0
@rainbreak correct, it's about that case, though I think _storage_ ones are later added cases similar to what happened on the mainnet, but with non-empty storage
geth should unmark these cases as known failures, because the tests were recently changed to reflect the result produced by geth
rain
@rainbreak
thanks. I still don't understand why the tests exist for subsequent hardforks. in the case of ConstantinopleFix, is the client expected to specifically special case ripemd?
Andrei Maiboroda
@gumb0
It should if it wants to pass the tests, but it can't happen on the mainnet
(geth has it as a special case regardless of block number, and aleth now, too)
rain
@rainbreak
ah ok, it's because the tests follow behaviour of the clients
well, apart from these two tests, hevm is now passing all of the constantinople fix GSTs :)
Martin Holst Swende
@holiman
Re the fork+eip model, I have now added more tests of 1884 and one also a test for 1344at https://github.com/holiman/IstanbulTests
and yes, it's for development -- however, the idea is to take them and turn them into Istanbul blockchain tests later
And I'm also trying to generate the same thing (minus traces and dumps) using retesteth, but haven't gotten it going quite yet
Martin Holst Swende
@holiman
@shemnon do you have the inputs to the blake tests somewhere in a more concise format? As in, only the inputs and the expected outputs?
thats for the 20 main vectors. The rest are derivitive in the json files.
Martin Holst Swende
@holiman
Thanks!
Ah, they're from teh keep-network PR. They just nuked those, due to changes in the endian-ness
That's why I was looking for other sources of tests....
Danno Ferrin
@shemnon
Makes sense. The main input was just increasing bytes. The message state hash however looks to be internal state from an iteration.
Martin Holst Swende
@holiman
After we 'align' the pantheon/geth implementation, would be good to generate some random inputs, at least as pure json input/output tests (we don't have to make blockchain tests out of every single vector)
Danno Ferrin
@shemnon
Is this the current spec? - ethereum/EIPs#2136 - I’m not seeing the f zero/non-zero or zero/one claifications, unless I missed it.
Martin Holst Swende
@holiman
hmm.., no that looks like an old version, to me
Martin Holst Swende
@holiman
Are there any new transaction tests made for eip 2028 yet?
GuthL
@GuthL
Not as far as I know (just joined the chat). We reached out to various people to see how as champions, we can help on this front.
Martin Holst Swende
@holiman
Hm, wait, it may not be possible -- transaction tests don't have the concept of a balance... so what we need are statetests where the balance is low enough that they wouldn't work pre-istanbul, but are sufficient for 2028.
Should be fairly simple to add a couple of such tests, really
Oh no wait, transaction tests will do fine, since balance doesn't matter, what matters is the gas
Martin Holst Swende
@holiman
I continued generating Istanbul-tests with geth. If anyone wants to validate it, filled transaction tests for EIP 2028 are available at https://github.com/holiman/IstanbulTests/tree/master/TransactionTests/ttEIP2028
James Hancock
@MadeofTin
👏👏
Tomasz Kajetan Stańczak
@tkstanczak
I may load these tomorrow, no promise yet but I am just looking at testing now
Danno Ferrin
@shemnon
@holiman do you have fillers for the state tests you published?
Martin Holst Swende
@holiman
@shemnon Yes and no. I generated it using native go, here: https://github.com/holiman/go-ethereum/blob/generate_statetests/tests/statetestgenerator_test.go#L43 . However, I did also convert it to filler-form, in order to run them through the retesteth-process and get that working (see what's needed for retesteth to support base+eip form). I ran into some problems, and haven't had time to finish that
Is it possible for you to run the tests themselves and check if they seem to pass / are on correct form?
Danno Ferrin
@shemnon
I would need to hand load them and adjust the network target. It’s been on my to-do list but I haven’t been able to get that far down my list in the last few days.
However the new “from a file in a directory” chain params method may make the +EIP form a bit verbose.
Danno Ferrin
@shemnon
I adapted your GeneralStateTests into filler form for retesteth: ethereum/tests#627
let me know what you think of how I re-worked the gas tests. I think it’s easier to compare to the spec since we are getting the net gas cost of one operaton.
Danno Ferrin
@shemnon
retesteth will need this patch - shemnon/retesteth@e7eb725