Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
  • 10:47

    vyazelenko on master

    [Java] Upgrade to ByteBuddy 1.1… (compare)

  • 10:47

    vyazelenko on master

    [Java] Upgrade to ByteBuddy 1.1… (compare)

  • 10:37

    vyazelenko on master

    [Java] Do not delete root direc… (compare)

  • 10:05

    vyazelenko on master

    [Java] Use @TempDir for the med… (compare)

  • Feb 04 20:21
    pieceofchum edited #1420
  • Feb 04 20:19
    pieceofchum opened #1420
  • Feb 03 15:12

    mjpt777 on master

    Number format parsing exception… (compare)

  • Feb 03 15:09

    vyazelenko on master

    [Java] Upgrade to versions plug… (compare)

  • Feb 03 15:06

    vyazelenko on master

    [Java] Upgrade to versions plug… (compare)

  • Feb 03 15:05

    mjpt777 on master

    Add test for hex string convers… (compare)

  • Feb 03 14:45

    mjpt777 on master

    Change toString formation for b… Change toString formation for b… (compare)

  • Feb 03 14:38
    mjpt777 commented on 347671c
  • Feb 03 14:30
    vyazelenko commented on 347671c
  • Feb 03 14:26

    mjpt777 on master

    Warnings cleanup. (compare)

  • Feb 03 14:14

    mjpt777 on master

    Address toString on raw array. (compare)

  • Feb 03 14:05

    mjpt777 on master

    Remove unnecessary test. (compare)

  • Feb 03 12:09

    mjpt777 on master

    Change algorithm for rehash so … (compare)

  • Feb 03 12:05

    mjpt777 on master

    Remove lgtm suppression which d… (compare)

  • Feb 03 10:51

    vyazelenko on master

    [Java] Use a dedicated subclass… (compare)

  • Feb 03 10:19

    mjpt777 on master

    [C] Remove comments causing war… (compare)

Carlo
@entangled90
sometimes the unexpected data is written in the media driver as well
for publishing, using the method that thakes 2 buffers
and the inconsistency is between these two buffers
Dmitry Vyazelenko
@vyazelenko
So you are using io.aeron.Publication#offer(org.agrona.DirectBuffer, int, int, org.agrona.DirectBuffer, int, int)? Do you have a reproducer (test case) that you can share?
Carlo
@entangled90
not yet
i'm trying upgrading right now
I'll try to see if I can reproduce it
Dmitry Vyazelenko
@vyazelenko
ok
Carlo
@entangled90
another doubt I had was if I can use ExclusivePublication from differen threads (each one with it's own Exclusive publication)
the publication is not shared between threads
Dmitry Vyazelenko
@vyazelenko
ExclusivePublication cannot be shared between threads. If you have multiple threads publishing then you need to use ConcurrentPublication.
Carlo
@entangled90
ah ok! that's probably it then..
Carlo
@entangled90
thanks very much
Carlo
@entangled90
actually that change didn't fix, as every thread used a local (per thread) ExclusivePublication
I entered the media driver machine and I found strange data from AeronStat
NAKs are consistently increasing, +1 every 20 sec or so
It doesn't sound normal for two processes in the same rack
This message was deleted
Martin Thompson
@mjpt777
@entangled90 NAKs can happen even over loopback when buffer sizes are not correct given sending rate or congestion.
Carlo
@entangled90
ok thanks
What's the best way to investigate missing "messages" in a stream? I tried the LossReport but the timestamps don't match
Martin Thompson
@mjpt777
@entangled90 Messages will not go missing due to loss. Loss gets recovered. Without spending time understanding your application it is difficult to guess what can be wrong.
Mostly when people say messages have gone missing when we have investigated it was a bug in their app.
Are you registering image unavailable handlers to see if you have connections dropping out and reconnecting?
Carlo
@entangled90
yes, but I don't explicitly reconnect
Martin Thompson
@mjpt777
Aeron will reconnect if the publication and subscription are still active unless rejoin=false is set on the URI for the subscription.
Carlo
@entangled90
ok, so in fact It reconnects normally
I don't understand one thing about the LossReport:
I have lines where LAST_OBSERVATION = 11:02:19 & FIRST_OBSERVATION = 13:19:01
Martin Thompson
@mjpt777
Not only should exclusive publications not be shared across threads, subscriptions should never be shared across threads.
Carlo
@entangled90
shouldn't first_observation be before last_observation?
Martin Thompson
@mjpt777
Which media driver?
Carlo
@entangled90

Not only should exclusive publications not be shared across threads, subscriptions should never be shared across threads.

Now i'm using ConcurrentPublication and a single thread with a Subscription

C Media driver
Martin Thompson
@mjpt777
There is a bug in the recording of loss observations in the C media driver. It is writing the last and first observations the wrong way round. I'll fix it.
Carlo
@entangled90
ah ok, thanks
btw, is it normal that the loss report is not empty?
Martin Thompson
@mjpt777
It depends on congestion and how you size your buffers. A little loss is common.
Carlo
@entangled90
ok..Is there a way to check if NAK sent by one media driver are received by the other?
I suppose that the output of AeronStat is enough: NAKs received & Retransmits sent should mean it's receiving NAK from the other media driver
Martin Thompson
@mjpt777
Correct.
Carlo
@entangled90
I still there is something strange happening. Yesterday I restarted all the services in the afternoon and no loss was reported for the remaining time. Today I log in and I see that application is not receiving some messages. It startied at 07:42 on machine 1 (machine 2 reports no losses). LossReport is filled with rows starting from 07:42. Around 280 rows for that stream id.
pulisher publishes around 3k msgs/s of about 700 bytes each
Carlo
@entangled90
is there a "golden rule"for sizing buffer in the media driver given a throughput?
Martin Thompson
@mjpt777
@entangled90 You need to read up on Bandwidth Delay Product and queuing theory.
@entangled90 If you need more detailed help we can provide consulting.
Carlo
@entangled90
ok I'll take a look
@entangled90 If you need more detailed help we can provide consulting.
atm we are still assessing aeron, I'll let you know in the future
Ghost
@ghost~5fab2f45d73408ce4ff3c0e0
Hello I had a quick question about how aeron cluster nodes behave. This is not a support question, I am just trying to understand how the nodes interact and what impact network latency would have. My questions is If you have nodes in different regions where there is much more latency for one region than another is it expected that the slower region would slow down all nodes? Is there communications that are happening that would have that type of effect possibly in how the RAFT is communicating with each node? Thanks for any help in understanding performance characteristics of the cluster.
Martin Thompson
@mjpt777
@pieceofchum We only answer cluster questions to those on a support contract.
Ghost
@ghost~5fab2f45d73408ce4ff3c0e0
ok np