These are chat archives for ethersphere/orange-lounge

18th
Apr 2018
Viktor Trón
@zelig
Apr 18 03:28
@justelad @homotopycolimit ethersphere/go-ethereum#187 found this
Elad
@justelad
Apr 18 07:09
morning :) :+1: @zelig
Elad
@justelad
Apr 18 08:15
pss documentation chapter revised here: http://swarm-docs.s3-eu-west-1.amazonaws.com/userguide.html#pss @zelig @nolash
lash
@nolash
Apr 18 08:31
Cheers @justelad I'll have a look through!
Balint Gabor
@gbalint
Apr 18 08:32
The android job of our build is always failing due to some java download error. Anybody experienced this earlier?
Connecting to download.oracle.com (download.oracle.com)|23.72.155.205|:80... connected.
HTTP request sent, awaiting response... 404 Not Found
2018-04-18 06:59:56 ERROR 404: Not Found.
lash
@nolash
Apr 18 09:32
@zelig I got the test to pass with 32 nodes and 4 restarts. First time since refactor: https://github.com/ethersphere/go-ethereum/pull/370/commits/1723e754030e2b01ca89c4618e9f415693f1c74e
Viktor Trón
@zelig
Apr 18 09:32
awesome
lash
@nolash
Apr 18 09:33
But there are still intermittent timeouts :/
Re: Your question on unsubscribe: Will the pending sends be flushed?
Viktor Trón
@zelig
Apr 18 09:33
doublechecked suggestPeer changes?
lash
@nolash
Apr 18 09:33
what changes?
Viktor Trón
@zelig
Apr 18 09:34
ones that seeemed missing yesterday from the branch
lash
@nolash
Apr 18 09:34
They didn't seem to make a difference
Viktor Trón
@zelig
Apr 18 09:34
?
they surely do
lash
@nolash
Apr 18 09:35
... for the result of the test.
Viktor Trón
@zelig
Apr 18 09:35
pls talk to @janos for the latest
im flying in 10 mins
lash
@nolash
Apr 18 09:35
Anyway, I didn't get a reply on whether the lines I posted were the lines you meant
Viktor Trón
@zelig
Apr 18 09:37
@nolash there is this thing with the raw msg in pss i asked to consult twice but it was merged nonetheless, can we talk. abuot it?
lash
@nolash
Apr 18 09:38
You mean the topic thing?
Viktor Trón
@zelig
Apr 18 09:38
yep
lash
@nolash
Apr 18 09:38
@janos you know more about what @zelig is referring to above? ^
Well you didn't answer my reply on your question ;)
But sure, we can talk about it.
Viktor Trón
@zelig
Apr 18 09:40
im referring to a discussion yesterday when @janos found some change didnt make it to the branch/got lost in the merge
in kademlia suggestPeer
Janoš Guljaš
@janos
Apr 18 09:49
@nolash @zelig: @justelad and I went through the diff and found only one missing change that is now fixed in swarm-network-rewrite
all required changes that we did for kademila-fixes are now in swarm-network-rewrite branch
lash
@nolash
Apr 18 09:51
+       if e.retries == 0 && bytes.Compare(e.Address(), k.BaseAddr()) < 0 {
+               e.retries++
+               return nil
+       }
I copied these lines from a branch from @justelad I believe. It's not in swarm-network-rewrite. Should they be? @janos ?
They are in swarm/network/kademlia.go function callable
Janoš Guljaš
@janos
Apr 18 09:56
I do not think that they should be @nolash. I do not recall that we needed the change in callable function.
lash
@nolash
Apr 18 09:56
@justelad The docs are they on git? For changes, can I branch?
ok thanks @janos I will remove them
Wow, the test seems to pass every time now <3
Janoš Guljaš
@janos
Apr 18 09:58
:+1: :)
lash
@nolash
Apr 18 09:59
I'll commence with the pruning test then, @gbalint ?
Elad
@justelad
Apr 18 10:05
@nolash they are on the ethersphere repo, branch docs-overhaul :)
push directly to it please
lash
@nolash
Apr 18 10:12
ok
lash
@nolash
Apr 18 10:50
@justelad shuffled around the words a bit in the pss doc.
... and pushed
lash
@nolash
Apr 18 11:39
@janos @gbalint concerning the errors topic:
$ grep SetErrored swarm/ -r
swarm/network/stream/delivery.go:                chunk.SetErrored(storage.ErrChunkForward)
swarm/network/stream/delivery.go:                chunk.SetErrored(storage.ErrChunkTimeout)
swarm/network/stream/delivery.go:            chunk.SetErrored(nil)
swarm/storage/localstore.go:        chunk.SetErrored(ErrChunkInvalid)
swarm/storage/types.go:func (c *Chunk) SetErrored(err error) {
swarm/storage/netstore.go:                chunk.SetErrored(ErrChunkUnavailable)
swarm/storage/netstore.go:        chunk.SetErrored(ErrChunkNotFound)
swarm/storage/netstore.go:    chunk.SetErrored(nil)
and swarm/storage/localstore.go:
        if err == ErrFetching && chunk.GetErrored() == nil {
                log.Trace(fmt.Sprintf("LocalStore.GetOrRetrieve: %v hit on an existing request %v", key, chunk.ReqC))
                return chunk, false
        }
Seems ErrFetching is never set anywhere
Oh wait
Argh
It's the "other" error type
Let's extend the test timeout, send it a SIGQUIT (or panic) and see if there are clues in the stack trace on deadlocks?
Janoš Guljaš
@janos
Apr 18 11:51
yes, ErrFetching is sent by LocalStore.get, it is a different error
Balint Gabor
@gbalint
Apr 18 12:41
@nolash we are back in the hangout if you would like to join
lash
@nolash
Apr 18 12:43
Ok I'll join you in a bit
holisticode
@holisticode
Apr 18 13:18
@nolash did you push the latest changes on discovery-dynamic
lash
@nolash
Apr 18 13:26
ya
@holisticode
Janoš Guljaš
@janos
Apr 18 13:28
a short gist how to replicate is issue with TestSwarmNetwork test https://gist.github.com/janos/e5ac5556384e3ba5252240c1b06ccc3a
lash
@nolash
Apr 18 15:34
@janos @gbalint @holisticode sadly fixing the races doesn't make the test pass
Janoš Guljaš
@janos
Apr 18 15:34
at least it is improvement, for sure :)
lash
@nolash
Apr 18 15:35
But now the same edits are in both my branch and this. Maybe I'll avoid pushing...
@janos did you uncover anything in the meantime?
Janoš Guljaš
@janos
Apr 18 15:38
set/get errored change is not the problem
trying to separate the validation from params change
Janoš Guljaš
@janos
Apr 18 15:56
@nolash it appears that PssEnabled is not checked in NewSwarm https://github.com/ethersphere/go-ethereum/pull/337/files#diff-1213d35c0c5fafb97a1ae8769549f85eR223, is there a reason for that?
it does not solve the issue, but it may be relevant
I am disabling pss in TestSwarmNetwork test
Balint Gabor
@gbalint
Apr 18 15:58
@janos nice catch
Janoš Guljaš
@janos
Apr 18 15:59
thanks
lash
@nolash
Apr 18 16:08
The flag should be disabled
I'll PR that later.
Thanks tho
Does it change anything?
Balint Gabor
@gbalint
Apr 18 16:09
no
Janoš Guljaš
@janos
Apr 18 16:09
unfortunately, it does not improve the situation
lash
@nolash
Apr 18 16:09
It would have been thoroughly weird if it did
Balint Gabor
@gbalint
Apr 18 16:09
I disabled pss manually to check it ,but the test still fails
Viktor Trón
@zelig
Apr 18 17:06
any news on the frontline?
lash
@nolash
Apr 18 17:06
No luck on my side :/
Viktor Trón
@zelig
Apr 18 17:15

go test -v -timeout 15m -count 1 github.com/ethereum/go-ethereum/swarm -run TestSwarmNetwork
# github.com/ethereum/go-ethereum/swarm
swarm/network_test.go:198:3: undefined: "math/rand".Shuffle
swarm/network_test.go:387:2: undefined: "math/rand".Shuffle
FAIL    github.com/ethereum/go-ethereum/swarm [build failed]
go version
go version go1.9.2
lash
@nolash
Apr 18 17:17
Yeah, shouldn't be there.
@janos @zelig If I wanted to incrementally test propagation of chunks from lowest to highest level, what would be the tests I would run?
Viktor Trón
@zelig
Apr 18 17:19
TestSyncSim and TestDeliverySim then fabios snapshot tests
if i understand u correctly
lash
@nolash
Apr 18 17:23
I think you do
Balint Gabor
@gbalint
Apr 18 17:59
@zelig you need to upgrade to go 1.10 to eliminate this compile error
lash
@nolash
Apr 18 18:38

@justelad alerted me to the fact that there is currently no way to specify a multihash resource update through the swarm/api layer. Of course, there should be.

Currently, a raw resource update is made by POSTing data to bzz-resource://somename.eth. Anyone have a good idea on how we should implement distinguishing between "raw" and multihash updates?

Maybe bzz-resource://somename.eth for multihash updates and bzz-resource://somename.eth/raw for "raw" ones?

lash
@nolash
Apr 18 19:03
I added an issue for it here: ethersphere/go-ethereum#400
holisticode
@holisticode
Apr 18 19:48
@zelig you need to upgrade to go 1.10 to eliminate this compile error
one of the builds still runs on go 1.9.x though, isn't that a problem?
Balint Gabor
@gbalint
Apr 18 19:49
it would be, but this is just a development branch, not ready for merge
holisticode
@holisticode
Apr 18 19:49
:+1:
holisticode
@holisticode
Apr 18 20:01
is @nonsense not around today?
holisticode
@holisticode
Apr 18 20:06
@homotopycolimit do you know where the logs of single swarm nodes in the test cluster are?