Metropolis Testing Guide - https://gist.github.com/Souptacular/fd197b1fac7c6d2660b0bef27a33ed40
<Dimitry Khokhlov (tests)> if this is too much we have the docker container. but it might be outdated
<Dimitry Khokhlov (tests)> http://retesteth.ethdevops.io/release/0.0.9-berlin/dretesteth-0.0.9-berlin.tar
<Dimitry Khokhlov (tests)> it has geth version
geth evm version 1.9.24-unstable-b63bffe8-20201030
<Dimitry Khokhlov (tests)> and you need dretesteth.sh script to run this docker. its in retesteth repo
<Dimitry Khokhlov (tests)> also can generate custom tests
<Dimitry Khokhlov (tests)> More docker instructions
<Dimitry Khokhlov (tests)> https://ethereum-tests.readthedocs.io/en/latest/retesteth-tutorial.html
<Jochem Brouwer> http://retesteth.ethdevops.io/web/index.php?gtests=stSpecialTest&singletest=JUMPDEST_Attack&clients=t8ntool&snetwork=Berlin&vmtrace=vmtrace&verbosity=none&trd=&trg=&trv= Gives an error @Dimitry
Very cool that we have this online!
<Jochem Brouwer> I am having this problem on my pc:
./retesteth -t GeneralStateTests/stSpecialTest -- --testpath /Volumes/Ethereum/ethereumjs-vm/packages/ethereum-tests --singletest JUMPDEST_Attack/JUMPDEST_Attack_d0g0v0_Berlin --clients t8ntool
Running 1 test case...
Running tests using path: /Volumes/Ethereum/ethereumjs-vm/packages/ethereum-tests
WARNING: Retesteth configs version is different (running: '0.1.0-accesslist' vs config '0.0.9-berlin')! Redeploy the configs by deleting the folder ~/.retesteth!
Active client configurations: 't8ntool '
Filter: 'JUMPDEST_Attack'
Running tests for config 'Ethereum GO on StateTool' 2
Test Case "stSpecialTest":
100%
flag provided but not defined: -input.alloc1
Error: The command '/Users/jochem/.retesteth/t8ntool/start.sh --input.alloc /var/folders/40/14b4jfr51f3dnkytndsf3xh00000gn/T/413e89a5-a394-4743-a529-46a94ebf7c8e/alloc.json --input.txs /var/folders/40/14b4jfr51f3dnkytndsf3xh00000gn/T/413e89a5-a394-4743-a529-46a94ebf7c8e/txs.json --input.env /var/folders/40/14b4jfr51f3dnkytndsf3xh00000gn/T/413e89a5-a394-4743-a529-46a94ebf7c8e/env.json --state.fork YOLOv2 --output.basedir /var/folders/40/14b4jfr51f3dnkytndsf3xh00000gn/T/413e89a5-a394-4743-a529-46a94ebf7c8e --output.result out.json --output.alloc outAlloc.json' exited with 256 code. (stSpecialTest/JUMPDEST_Attack, step: GeneralStateTest)
--------
TestOutputHelper detected 1 errors during test execution!
/Volumes/Ethereum/retesteth/retesteth/TestOutputHelper.cpp:208: error: in "GeneralStateTests/stSpecialTest":
*** 1 failure is detected in the test module "Master Test Suite"
*** Total Tests Run: 0
--------
*** TOTAL ERRORS DETECTED: 1 errors during all test execution!
--------
info: (stSpecialTest/JUMPDEST_Attack, step: GeneralStateTest)
<Jochem Brouwer> Thanks so much 😄
<Dimitry Khokhlov (tests)> yay. the tool usecase got useful
<Jochem Brouwer> Yep this is extremely helpful!!
<Dimitry Khokhlov (tests)> 359 depth
<Dimitry Khokhlov (tests)> Test regeneration and fix update:
https://github.com/ethereum/tests/releases/tag/8.0.1
All state tests now have an rlp txbytes field. So no need to parse transaction arrays and do the signing.
<Dimitry Khokhlov (tests)> Retesteth release:
https://github.com/ethereum/retesteth/releases/tag/v0.1.0-accesslist
https://github.com/ethereum/retesteth/releases/tag/v0.1.0-accesslist
<carver, Jason Carver> The retesteth traces look super helpful. I poked around the web UI but didn't understand how to load an arbitrary test like this:
(I tried uploading the file, but got
How can I get the trace for that?
BlockchainTests/GeneralStateTests/VMTests/vmIOandFlowOperations/codecopy.json:codecopy_d0g0v0_Berlin(I tried uploading the file, but got
Could not locate custom test file: '/data/web/web/uploads/codecopy.json')How can I get the trace for that?
<carver, Jason Carver> I decided it's probably time to just use the docker version, but I thought you might want to know about ^ issue anyway
<Dimitry Khokhlov (tests)> What size is the file? Send it to me to reproduce and fix
<carver, Jason Carver>
BlockchainTests/GeneralStateTests/VMTests/vmIOandFlowOperations/jump.json
<Dimitry Khokhlov (tests)> the limit was 500
<Dimitry Khokhlov (tests)> I fixed the docker as well
<Dimitry Khokhlov (tests)> 2mb
<Dimitry Khokhlov (tests)> Why you try the bc version?
<Dimitry Khokhlov (tests)> You can access that one from the selector as a state test
<carver, Jason Carver> Hah, thanks
<carver, Jason Carver> I don't really understand which subset of tests are already available vs needing upload, so I just got in the habit of uploading
<Dimitry Khokhlov (tests)> in blockchain tests we are working on selectors for vm trace of a specific block/transactions