Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
  • 19:21
    tkstanczak synchronize #2348
  • 19:21

    tkstanczak on pluginPaths

    add plugin paths (compare)

  • 17:40

    LukaszRozmej on TxPriority

    Updating Fast Sync config files… Initialize chain from rlp file … Manual dispatch of posdao tests… and 4 more (compare)

  • 16:05
    tkstanczak synchronize #2348
  • 16:05

    tkstanczak on pluginPaths

    loader dirs fix (#2347) * load… Merge branch 'master' into plug… (compare)

  • 15:58

    tkstanczak on master

    loader dirs fix (#2347) * load… (compare)

  • 15:58
    tkstanczak closed #2347
  • 15:58
    tkstanczak opened #2348
  • 15:58

    tkstanczak on pluginPaths

    plugin paths (compare)

  • 15:51
    tkstanczak synchronize #2347
  • 15:51

    tkstanczak on cliPluginsPath

    fix namespace (compare)

  • 15:48
    tkstanczak synchronize #2347
  • 15:48

    tkstanczak on cliPluginsPath

    no api (compare)

  • 15:39
    tkstanczak synchronize #2347
  • 15:39

    tkstanczak on cliPluginsPath

    added api to cli (compare)

  • 15:35
    tkstanczak synchronize #2347
  • 15:35

    tkstanczak on cliPluginsPath

    Fixed type. (#2342) No cli (#2345) * remove CLI re… Merge branch 'master' into cliP… (compare)

  • 15:29
    tkstanczak synchronize #2347
  • 15:29

    tkstanczak on cliPluginsPath

    cli load self (compare)

  • 15:23
    tkstanczak synchronize #2347
Gitter Bridge
@nethermind-bridge
[discord] <tkstanczak> @schwabo just finishing the Plugin project where you will be able to create any subscribers and redirect the stream of transactions / blocks to websocket / kafka / grpc
[discord] <tkstanczak> do you need exactly eth_subscribe for compatibility?
Gitter Bridge
@nethermind-bridge
[discord] <schwabo> @tkstanczak thanks for coming back at me - yes, i would need eth_subscribe
Gitter Bridge
@nethermind-bridge
[discord] <tjayrush> Hello. I really like the formatting of your RPC documentation (https://docs.nethermind.io/nethermind/ethereum-client/json-rpc/json-rpc-modules/), but in some places it seems to be incomplete.
[discord] <tjayrush> I've been working on the TurboGeth RPC, and I'd really like to link over to some of your documentation. I think this would tend to move things closer to a day where more of the non-eth_ routines can be standardized.
[discord] <tjayrush> Is there something I can do to help get your docs moving forward? I could copy and paste or something. That would actually help me with my work on TurboGeth. Let me know how to get started.
Gitter Bridge
@nethermind-bridge
This message was deleted
This message was deleted
[discord] <MysticRyuujin> Deleted my reply because you're looking at the new docs haha
Gitter Bridge
@nethermind-bridge
[discord] <tkstanczak> @tjayrush we have the docs code generation written based on the attributes in the RPC modules
[discord] <tkstanczak> let me show you an example:
[discord] <tkstanczak> these are in code attributes:
[discord] <tkstanczak> so improving them is a matter of creating a code PR
[discord] <tkstanczak> as for the code generator (where it is buggy)
[discord] <tkstanczak> it is here:
[discord] <tkstanczak> at the moment we have an issue where we cannot fully automate docs regenerator on each commit (which we would love to have)
Gitter Bridge
@nethermind-bridge
[discord] <tkstanczak> if there is anyone who wants to finish this work I would be willing to fund it.
[discord] <tkstanczak> @SebastianDremo was working on it but there is some work required for RPC + CLI + Config docs
Gitter Bridge
@nethermind-bridge
[discord] <tjayrush> Hey. I was just checking out your docs and came across this: https://github.com/NethermindEth/docs/blob/master/guides-and-helpers/custom-analytic-tools.md, and I thought "Wow" I just wrote an article about exactly that (https://medium.com/coinmonks/ethereums-issuance-minerreward-3cad5b9a72ff). Five or six different people have been collecting this data here: https://github.com/lastmjs/eth-total-supply#reconciliation. Perhaps you'll add your results.
Gitter Bridge
@nethermind-bridge
[discord] <Electririlla> I was also just glancing over the "Custom Analytics" tool section 🙂 Does anyone know if this way of accessing blockchain data is significantly faster than going through RPC calls? We scan token transfer events via RPC and for example for Tether it's millions of events/RPC calls, would using the C# interfaces be significantly faster (e.g. how much overhead does the RPC call really add)?
Gitter Bridge
@nethermind-bridge
[discord] <tkstanczak> @Electririlla - it is significantly faster and there will be plugins available from next week
Gitter Bridge
@nethermind-bridge
[discord] <schwabo> is there any recommendations on the db config to get more performance on most recent data? (have around 24gb of ram available on 8 core)
Gitter Bridge
@nethermind-bridge
[discord] <Electririlla> we just run another fast sync'd node and hit it with 16 rpc request threads in parallel and it's fast for our purpose and recent data 🙂
Gitter Bridge
@nethermind-bridge
[discord] <Electririlla> it always seems to be io limited, I've never seen cpu at more than 20% or so on all cores even when hitting the node with 16 rpc request threads in parallel, memory also didn't make a huge difference, just some rough observations, don't have measurements, but io is pretty much the most important
[discord] <luk> there is a way to get 100% cpu utilization through rpc, but its a secret 😉
[discord] <schwabo> Yeah i was thinking that too that it will most likely be i/o limiting me a bit
[discord] <Electririlla> we do mostly get events from block 0 to current block, that seems to be purely io restricted, I guess cache is not so useful for block 0 to current stuff as it can't possibly cache all of that
[discord] <luk> @Electririlla what is your cpu utilization on that? Why do you often go all the way back from block 0?
[discord] <luk> seems like waste of resources
[discord] <Electririlla> yean not block 0, from contract creation, but like to get all transfers of tether for example, thats a few million events
[discord] <Electririlla> cpu utilization is like below 20%
[discord] <Electririlla> on a ryzen 3600
[discord] <Electririlla> or ryzen 3700, forgot, something like that 🙂
Gitter Bridge
@nethermind-bridge
[discord] <luk> but consecutive call are also from start point (contract creation), or just incemental? (or some filter?)
[discord] <Electririlla> just incremental, but we're trying to scan all events for all tokens, we've done that now
[discord] <Electririlla> but now all uniswap events etc, although those are less in amount compared to tether 🙂
[discord] <Electririlla> our use case is basically decoding smart contract calls en masse, like duneanalytics or thegraph for example
[discord] <luk> what do you do with all this data later?
[discord] <Electririlla> we dump it into an ungodly db 🙂 🙂
[discord] <luk> ah thought so
[discord] <luk> index it there as you need it
[discord] <Electririlla> yep, simple example is historical balance chart of all your token holdings for example
[discord] <luk> just FYI in latest release eth_getLogs got default timeout, you might want to increase it for your massive calls 🙂
Gitter Bridge
@nethermind-bridge
[discord] <Electririlla> basically nethermind is fast enough for this purpose, it takes time but we can get all tether transfers (token with most transfers i think) in 6 minutes or so (and that includes our time to dump these few million records into the db as well)
[discord] <Electririlla> ah ok, we'll keep that in mind 🙂
[discord] <luk> yeah youll definitly need to configure that then
[discord] <luk> JsonRpcConfig.Timeout
[discord] <Electririlla> hence when schwabo asked about memory optimization, we basically just started another fast sync'd node (next to the archive node) and that serves us well and gives us basically twice the throughput (for the stuff that the fast sync node can do), and it's cheap and easy to get another fast sync'd node running