ethereum/tests

Metropolis Testing Guide - https://gist.github.com/Souptacular/fd197b1fac7c6d2660b0bef27a33ed40

adria0.eth

@adria0

Finally I figured out the sha3_memSize stuff :)

Buttt.. about the https://github.com/ethereum/tests/blob/develop/BlockchainTests/ValidBlocks/VMTests/vmSha3Test/sha3_memSizeQuadraticCost64.json, take a look in the prestate and poststate of 0x0f572e5295c57f15886f9b263e2f6d2d6c7b5ec6 (balance is untouched if I understand the test well)

        "pre" : {
            "0x0f572e5295c57f15886f9b263e2f6d2d6c7b5ec6" : {
                "balance" : "0xffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff",
                "code" : "0x60016107e02060005500",
                "nonce" : "0x00",
                "storage" : {
                }
            },
        }
        post {
        "0x0f572e5295c57f15886f9b263e2f6d2d6c7b5ec6" : {
                "code" : "0x60016107e02060005500",
                "nonce" : "0x00",
                "balance" : "0x7b",
                "storage" : {
                    "0x00" : "0xbc36789e7a1e281436464229828f817d6612f7b477d66591ff96a9e064bcc98a"
             }
        }
    }

is there any overflow? Is time to implement is some way EIP 1985: Sane limits for certain EVM parameters?

Martin Holst Swende

@holiman

Finally I figured out the sha3_memSize stuff :)

@adria0 oh, do share!

"balance" : "0x7b", -- when you get too rich, you've won ethereum already, and you have to start over. didn't you know :? :)

Danno Ferrin

@shemnon

This is something Besu is seeing too. We clamp Value/Ether to 2^256. If this clamp is valid or invalid it should be the subject of it’s own test, not the ones where we are trying to blow up the memory stack. The tests that set balances to 2^256-1 should knock down the initial balance, 2^250 should easily grant enough gas to exhaust memory. Such an amount of ether is out of reach of the current network parameters.

adria0.eth

@adria0

Finally I figured out the sha3_memSize stuff :)

@adria0 oh, do share!

and keeping 2^-247% of your funds for lolipops :)

Martin Holst Swende

@holiman

So what was the point of that test? Is it a memory exploder thingy?

adria0.eth

@adria0

@holiman afais, is to check the gas of the allocated memoryC_memory(a)=G_memory x a + (a²/512), but I do not know why this U256(-1) in the balance.

@holiman I'm running the ForkStressTest_Istanbul test and works ok for me, and the error in hive is request failed (500): client container terminated due to unresponsive RPC, https://hivetests.ethdevops.io/?suite=1588004896-c5692a8369b2107c8ad1f101bb574bad.json . Do you have any idea about what could happen? A timeout?

Eth-Gitter-Bridge

@Eth-Gitter-Bridge

<M H Swende (holiman)> yes: ethereum/hive#279

Martin Holst Swende

@holiman

(The change above has been done in config via ansible, but I haven't yet redeployed/configured the actual server)

I you were to look at the simulator log (the simulator is responsible for spinning up nodes with the rlp blocks, and then verifying the poststate), you'd see it saying that it's still unresponsive after 60s, and tears it down.

Eth-Gitter-Bridge

@Eth-Gitter-Bridge

<Dimitry> Fork Stress test is chain reorg. Where many blocks imported but only one chain has largest total difficulty. So the client has to reorg a lot

adria0.eth

@adria0

thanks for the info, dimitry

Eth-Gitter-Bridge

@Eth-Gitter-Bridge

<Péter Szilágyi (karalabe)> @Dimitry @M H Swende (holiman) What's the purpose of this test? https://github.com/ethereum/tests/blob/develop/src/GeneralStateTestsFiller/stStaticCall/static_Call1MB1024CalldepthFiller.json#L106

<Péter Szilágyi (karalabe)> It consumes ~900 billion gas for its execution

<Péter Szilágyi (karalabe)> and allocates over 1GB RAM

<Péter Szilágyi (karalabe)> This test is causing us all kinds of CI failures due to the insane alloc

<Péter Szilágyi (karalabe)> Do we really need this huge test?

<Péter Szilágyi (karalabe)> (especially since we run this for all forks, so it's 1GB x 8?)

<Péter Szilágyi (karalabe)> Can't we maybe make it use 10MB total? What's the purpose of 1MB/stack depth. what does this aim to test?

Eth-Gitter-Bridge

@Eth-Gitter-Bridge

<Dimitry> This one is trying to make 1024 subcalls. (in depth)
So much gas is set because sstore is used on every call.

<Dimitry> 1 gb of ram because each call allocate 1 mb for memory.

Eth-Gitter-Bridge

@Eth-Gitter-Bridge

<M H Swende (holiman)> Is the 1mb at every depth needed though?

<M H Swende (holiman)> @karalabe we do already disable some very resource-intense tests on CI, we could add this one too

<Dimitry> I guess 1 mb per depth was to stress test the client. This is one of the earliest tests created by Christoph

Paweł Bylica

@chfast

Memory consumption itself is ~23M

Maybe lower that by 2x or 4x. And restrict to single fork?

Eth-Gitter-Bridge

@Eth-Gitter-Bridge

<Péter Szilágyi (karalabe)> Why would it be 23MB?

<Péter Szilágyi (karalabe)> It allocates a new 1MB memory area at every call

Eth-Gitter-Bridge

@Eth-Gitter-Bridge

<Dimitry> This particular test executed only on Istanbul

<Dimitry> It allocates but not use. There is a room for client optimization.

Paweł Bylica

@chfast

~23M gas

Eth-Gitter-Bridge

@Eth-Gitter-Bridge

<Péter Szilágyi (karalabe)> Waht room for optimization

<Péter Szilágyi (karalabe)> You allocase 1GB ram

<Péter Szilágyi (karalabe)> how am I supposed to optimize that

<Péter Szilágyi (karalabe)> guess that you allocate junk?

<Dimitry> > @karalabe we do already disable some very resource-intense tests on CI, we could add this one too
@holiman

Also can bundle memory alloc tests into resource consuming suite

Martin Holst Swende

@holiman

Hey look, it's already tagged as 'slow': https://github.com/ethereum/go-ethereum/blob/master/tests/state_test.go#L40

No wait that isn't it

Eth-Gitter-Bridge

@Eth-Gitter-Bridge

<Péter Szilágyi (karalabe)> slow still runs

<M H Swende (holiman)> Oh yes it is

<Péter Szilágyi (karalabe)> at least for me locally

<M H Swende (holiman)> Yeah, slow ones still run, but we could disable them on travis

<M H Swende (holiman)> by adding the -short - flag

SebastianDremo

@SebastianDremo

Hey, new etherum tester here, working with Hive right now and got a question.
Is there a way to run only specific test from tests suit lets say v4014 from devp2p?

Right now I'm just removing available tests from devp2p_test.go but I suppose there is a faster way and I don't see any clues on GH docs

Martin Holst Swende

@holiman

@SebastianDremo for particular tests within a suite, it's not possible to whitelist only one of them (right now). Are you on Nethermind?

SebastianDremo

@SebastianDremo

Okay ( maybe when I find enough time, will try to make a PR for this feature ), yes I am on Nethermind

lightclient

@lightclient

i'm having trouble understanding why this test doesn't have an "out" key: https://github.com/ethereum/tests/blob/develop/VMTests/vmIOandFlowOperations/return1.json

could anyone shed some light on this?

Holger Drewes

@holgerd77

Have compiled down some release notes with the changes in the tests repo from the last half a year, this has now been released as a v7.0.0 tagged release on GitHub, also did a post on Reddit along.

This is a last stable pre-Berlin HF release and can be used by clients to have a reference point to update their test suites against, also the release notes should be helpful during the upgrade process.

Where communities thrive

People

Repo info

Activity