Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
  • Feb 08 19:45
    pieceofchum commented #1420
  • Feb 08 19:42

    mikeb01 on master

    [Java] Add toString methods. (compare)

  • Feb 08 02:54

    mikeb01 on master

    [Java] Update documentation. (compare)

  • Feb 08 02:14

    mikeb01 on master

    [CodeQL] Disable integer as enu… [Java] Make null checks clearer. (compare)

  • Feb 08 00:18

    mikeb01 on master

    [C] Remove unused media driver … (compare)

  • Feb 08 00:03
    mikeb01 commented on 7b0e917
  • Feb 07 23:48
    mikeb01 closed #1420
  • Feb 07 23:48
    mikeb01 commented #1420
  • Feb 07 21:52
    mikeb01 updated the wiki
    Cluster-Tutorial
  • Feb 07 21:50

    mikeb01 on 1.40.0_tutorial_patch

    [Java] Fix basic auction clust… (compare)

  • Feb 07 21:20
    vyazelenko commented on 7b0e917
  • Feb 07 20:35

    mikeb01 on master

    [Java] Add deprecated tags to d… (compare)

  • Feb 07 20:16

    mikeb01 on master

    [Java] Deprecate dynamic join. (compare)

  • Feb 07 19:26

    mikeb01 on master

    [C] Use int64 when parsing uint… (compare)

  • Feb 07 18:18

    mikeb01 on master

    Set max of resource free queue … (compare)

  • Feb 07 18:18
    mikeb01 closed #1421
  • Feb 07 18:18
    mikeb01 commented #1421
  • Feb 07 16:53

    mjpt777 on master

    [Java] Flip buffer for writing. (compare)

  • Feb 07 16:51

    mjpt777 on master

    [Java] Add defaults to switch s… (compare)

  • Feb 07 16:16

    mjpt777 on master

    [Java] Use Object streams in tr… (compare)

Carlo
@entangled90
not with UDP?
I know it's stupid, but topology of services is not "strict"
or I need to add "spy" to the subscription?
Dmitry Vyazelenko
@vyazelenko
Communication between io.aeron.Aeron and MediaDriver is always IPC. Subscriptions/Publications can be UDP or IPC.
Carlo
@entangled90
yeah, sorry misunderstood your comment
Basically what we were seeing (in aeron 1.30.0) is that an application received a frame with unexpected data, which was even different from what was written in the log buffer files of the media driver
sometimes the unexpected data is written in the media driver as well
for publishing, using the method that thakes 2 buffers
and the inconsistency is between these two buffers
Dmitry Vyazelenko
@vyazelenko
So you are using io.aeron.Publication#offer(org.agrona.DirectBuffer, int, int, org.agrona.DirectBuffer, int, int)? Do you have a reproducer (test case) that you can share?
Carlo
@entangled90
not yet
i'm trying upgrading right now
I'll try to see if I can reproduce it
Dmitry Vyazelenko
@vyazelenko
ok
Carlo
@entangled90
another doubt I had was if I can use ExclusivePublication from differen threads (each one with it's own Exclusive publication)
the publication is not shared between threads
Dmitry Vyazelenko
@vyazelenko
ExclusivePublication cannot be shared between threads. If you have multiple threads publishing then you need to use ConcurrentPublication.
Carlo
@entangled90
ah ok! that's probably it then..
Carlo
@entangled90
thanks very much
Carlo
@entangled90
actually that change didn't fix, as every thread used a local (per thread) ExclusivePublication
I entered the media driver machine and I found strange data from AeronStat
NAKs are consistently increasing, +1 every 20 sec or so
It doesn't sound normal for two processes in the same rack
This message was deleted
Martin Thompson
@mjpt777
@entangled90 NAKs can happen even over loopback when buffer sizes are not correct given sending rate or congestion.
Carlo
@entangled90
ok thanks
What's the best way to investigate missing "messages" in a stream? I tried the LossReport but the timestamps don't match
Martin Thompson
@mjpt777
@entangled90 Messages will not go missing due to loss. Loss gets recovered. Without spending time understanding your application it is difficult to guess what can be wrong.
Mostly when people say messages have gone missing when we have investigated it was a bug in their app.
Are you registering image unavailable handlers to see if you have connections dropping out and reconnecting?
Carlo
@entangled90
yes, but I don't explicitly reconnect
Martin Thompson
@mjpt777
Aeron will reconnect if the publication and subscription are still active unless rejoin=false is set on the URI for the subscription.
Carlo
@entangled90
ok, so in fact It reconnects normally
I don't understand one thing about the LossReport:
I have lines where LAST_OBSERVATION = 11:02:19 & FIRST_OBSERVATION = 13:19:01
Martin Thompson
@mjpt777
Not only should exclusive publications not be shared across threads, subscriptions should never be shared across threads.
Carlo
@entangled90
shouldn't first_observation be before last_observation?
Martin Thompson
@mjpt777
Which media driver?
Carlo
@entangled90

Not only should exclusive publications not be shared across threads, subscriptions should never be shared across threads.

Now i'm using ConcurrentPublication and a single thread with a Subscription

C Media driver
Martin Thompson
@mjpt777
There is a bug in the recording of loss observations in the C media driver. It is writing the last and first observations the wrong way round. I'll fix it.
Carlo
@entangled90
ah ok, thanks
btw, is it normal that the loss report is not empty?
Martin Thompson
@mjpt777
It depends on congestion and how you size your buffers. A little loss is common.
Carlo
@entangled90
ok..Is there a way to check if NAK sent by one media driver are received by the other?
I suppose that the output of AeronStat is enough: NAKs received & Retransmits sent should mean it's receiving NAK from the other media driver
Martin Thompson
@mjpt777
Correct.
Carlo
@entangled90
I still there is something strange happening. Yesterday I restarted all the services in the afternoon and no loss was reported for the remaining time. Today I log in and I see that application is not receiving some messages. It startied at 07:42 on machine 1 (machine 2 reports no losses). LossReport is filled with rows starting from 07:42. Around 280 rows for that stream id.
pulisher publishes around 3k msgs/s of about 700 bytes each
Carlo
@entangled90
is there a "golden rule"for sizing buffer in the media driver given a throughput?
Martin Thompson
@mjpt777
@entangled90 You need to read up on Bandwidth Delay Product and queuing theory.