Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
    Evgeny Medvedev
    @medvedev1088
    @rraina just created this issue blockchain-etl/ethereum-etl-airflow#8 :)
    @evgeniuz the issue has been funded blockchain-etl/ethereum-etl-airflow#2. I invited you to the team. I will assign you to the task so other people don't apply.
    bb324930
    @bb324930
    I haven't been able to track down a specific block, but it appears there is a bug that fails to extract some ERC20 transfers.
    -- amount flowing into address
    select
    sum((cast(t.value as numeric))) as total
    from `bigquery-public-data.ethereum_blockchain.token_transfers` t
    where t.token_address = '0xc02aaa39b223fe8d0a0e5c4f27ead9083c756cc2'
    and t.to_address = '0x4969358e80cdc3d74477d7447bffa3b2e2acbe92'
    
    UNION ALL
    
    -- amount flowing out of address (CANNOT be greater than the above)
    select
    -sum((cast(t.value as numeric))) as total
    from `bigquery-public-data.ethereum_blockchain.token_transfers` t
    where t.token_address = '0xc02aaa39b223fe8d0a0e5c4f27ead9083c756cc2'
    and t.from_address = '0x4969358e80cdc3d74477d7447bffa3b2e2acbe92'
    The debits are larger than the credits, which is impossible.
    The token address is a well known ERC20 contract, WETH
    I have confirmed on my local version of ethereum-etl that this is also the case, not just Google's ETL
    bb324930
    @bb324930
    The other explanation is that other contracts have allowed the movement of this token. For example, old contracts replaced by newer ones
    Evgeny Medvedev
    @medvedev1088
    @bb324930 you should also include Deposit and Withdrawal events in your double entry book. You can query them from the logs table. token_transfers only includes the Transfer event.
    bb324930
    @bb324930

    @medvedev1088 do you mean the transferFrom call? this also calls the Transfer() event:

    function transferFrom(address from, address to, uint tokens) public returns (bool success) { balances[from] = balances[from].sub(tokens); allowed[from][msg.sender] = allowed[from][msg.sender].sub(tokens); balances[to] = balances[to].add(tokens); Transfer(from, to, tokens); return true; }

    so Transfer logs should encompass this as well
    Evgeny Medvedev
    @medvedev1088
    @bb324930 I meant deposit() and withdraw() methods. They change the balances also:
    function deposit() public payable {
        balanceOf[msg.sender] += msg.value;
        Deposit(msg.sender, msg.value);
      }
    bb324930
    @bb324930
    Interesting - where are you finding this? I don't see it in the reference:
    https://github.com/ethereum/EIPs/blob/master/EIPS/eip-20.md
    Evgeny Medvedev
    @medvedev1088
    Rahul
    @rraina
    Happy New Years folks!
    Rahul
    @rraina
    Congrats!!
    GuthL
    @GuthL
    @medvedev1088 Congrats for the good work!
    I’m looking into withdrawing the public key from the transaction data for all active addresses. Is there a way to do it with ETL?
    Rahul
    @rraina
    Yes.
    Just write a SQL query to get unique addresses and union the from and to addresses
    chandanthundercore
    @chandanthundercore
    How do we get new EVM compliant chain added to this ETL? ThunderCore just released their mainnet. Is there a way to have ETL analyze their blockchain?
    GuthL
    @GuthL

    Just write a SQL query to get unique addresses and union the from and to addresses

    @rraina I’m not sure we are talking about the same things here. The address is not the public key. To extract a public key associated with a specific address, I need to extract it from an emitted transaction. I don’t know if ETL stores them.

    Rahul
    @rraina
    ETL stores every transaction including traces. Have you had a chance to look at the dataset on BigQuery?
    GuthL
    @GuthL
    Thanks a lot! I did but could not really make completely sense of it. From what you told me, it looks like what I’m looking for is under input in the transactions table.
    Rahul
    @rraina
    You want to look at the from_address and to_address I think...
    Evgeny Medvedev
    @medvedev1088
    @GuthL the JSON RPC api also returns v, r, s values of the ECDSA signature of a transaction https://github.com/ethereum/wiki/wiki/JSON-RPC#eth_gettransactionbyhash, from which it should be possible to recover the public key. Ethereum ETL doesn't export these values though. Feel free to create a PR. btw why are you interested in public keys and not addresses as Rahul suggested?
    Evgeny Medvedev
    @medvedev1088

    How do we get new EVM compliant chain added to this ETL? ThunderCore just released their mainnet. Is there a way to have ETL analyze their blockchain?

    @chandanthundercore if you want to to add ThunderCore support to the ethereum-etl tool you can create a PR. If you want to add it to BigQuery public datasets we'll need to discuss it with a Google representative.

    Eric Price
    @EricPrice2_twitter
    has anyone experienced getting this error when running "export_all" using geth via RPC?
    INFO [04-03|04:12:19.947] Served eth_getFilterLogs conn=192.168.1.10:54074 reqid=1 t=251.823µs err="context canceled"
    get this python error when it reaches token_transfers
    ValueError: {'code': -32000, 'message': 'context canceled'}
    Evgeny Medvedev
    @medvedev1088
    @EricPrice2_twitter is it reproduced consistently or from time to time?
    Eric Price
    @EricPrice2_twitter
    I can reproduce this every time
    Evgeny Medvedev
    @medvedev1088
    @EricPrice2_twitter I created an issue here blockchain-etl/ethereum-etl#165
    Veikko Eeva
    @veikkoeeva
    This appears the be a channel to ask if anyone knows if it possible to save transaction traces while any of the clients are syncing a full node? I mean I'd like to avoid the issue of saving an archive node but would like to record the transactions to a separate persistence system while syncing and then maintain just pruned full node and record transactions from the tip as they appear. I wonder if this is wishful thinking or possible without taking one the clients and modifying the actual code?
    Maybe one option would be to download the BigQuery dataset as the baseline.
    Saravana Kumar
    @saravana87
    Can anyone please tell me if you fixed this issue.
    "symbolic execution not available no module named 'mythril.ether'"
    Evgeny Medvedev
    @medvedev1088

    Can anyone please tell me if you fixed this issue.
    "symbolic execution not available no module named 'mythril.ether'"

    That's a warning from one of the dependencies. You can ignore it.

    Haaroon Yousaf
    @Haaroon
    how big is the export-all?
    jyfocus
    @wjy7551376
    Hello
    Everyone
    Evgeny Medvedev
    @medvedev1088
    Hello

    how big is the export-all?

    do you mean the size of files in GB? All entities will be more than 1 TB of data. You can check the size in BigQuery

    YituoHe
    @HaveYouTall

    When I tried to excuate this ethereumetl extract_csv_column --input transactions.csv --column hash --output transaction_hashes.txt, I got the wrong message bellow

    Symbolic Execution not available: No module named 'mythril.ether'

    And the transaction_hashes.txt file is empty! Can any one help me with this? Thanks!

    And by the way, I got the transactions.csv from this
    ethereumetl export_blocks_and_transactions --start-block 0 --end-block 5000 \
    
    --provider-uri file:///toshiba/*******/ethereum/eth-mainnet-dataset/fast-dataset/geth.ipc \
    
    --blocks-output blocks.csv --transactions-output transactions.csv
    peterfranzo
    @peterfranzo
    Hello, I want to do a full export of all erc20 transactions. Do I have to run a geth archive node or can I get away with just a full node?
    Evgeny Medvedev
    @medvedev1088

    Hello, I want to do a full export of all erc20 transactions. Do I have to run a geth archive node or can I get away with just a full node?

    Just a full node will work

    peterfranzo
    @peterfranzo
    Cool, thank you!