Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
  • Nov 24 17:32

    mjpt777 on master

    [Java] Simplify LogReplay. (compare)

  • Nov 24 16:08

    mjpt777 on master

    [C++] Formatting. (compare)

  • Nov 24 15:24

    mjpt777 on master

    [Java] Fix test so line numbers… (compare)

  • Nov 24 15:19

    mjpt777 on master

    [Java] Record errors with diffe… (compare)

  • Nov 24 15:08

    vyazelenko on master

    [C++] Wait for ArchivingMediaDr… (compare)

  • Nov 24 14:49

    vyazelenko on master

    [C++] Wait for the ArchivingMed… (compare)

  • Nov 24 11:24

    vyazelenko on master

    [Java] Make static. (compare)

  • Nov 24 11:22

    vyazelenko on master

    [Java] Add Java 16-ea to the bu… (compare)

  • Nov 24 11:03

    vyazelenko on master

    [Java] Always use Java 8 to run… (compare)

  • Nov 23 18:33

    vyazelenko on master

    [Java] Add Java `16-ea` to the … (compare)

  • Nov 23 18:02

    mjpt777 on master

    [Java] Clarify logic on when lo… (compare)

  • Nov 23 16:53

    vyazelenko on master

    [Java] Use `buildJavaVersion` t… [Java] Use BUILD_JAVA_HOME in t… (compare)

  • Nov 23 16:34

    vyazelenko on master

    [Java] Remove print statement f… (compare)

  • Nov 23 12:41

    vyazelenko on master

    Revert "[Java] Use Java to run … (compare)

  • Nov 23 11:24

    vyazelenko on master

    [Java] Use Java to run Gradle s… (compare)

  • Nov 23 11:04

    vyazelenko on master

    [Java] Support Java versions wi… (compare)

  • Nov 23 10:49

    vyazelenko on master

    [Java] Configure toolchain to c… (compare)

  • Nov 21 21:45
    vyazelenko synchronize #1091
  • Nov 21 21:13
    vyazelenko synchronize #1091
  • Nov 21 16:33
    vyazelenko synchronize #1091
Welly Tambunan
@welly87

hi all, i’ve posting a question in stack overflow regarding the messaging pattern in aeron
https://stackoverflow.com/questions/63528428/aeron-messaing-pattern-for-pub-sub-one-way-and-competing-consumer

thanks a lot

Erik Lensherr
@erik-lensherr_gitlab
Greetings! I am trying to grasp the mechanics of ReplayMerge since I have a scenario in which an app starts a replaying an existing recording and then joins the live channel seamlessly using ReplayMerge class. What I can't figure out is why ReplayMerge, after successfully merging with live, does not reconnect automatically if the publisher goes away and returns (I am aware of the 10 seconds liveness timeout). This scenario is not the same if I have a Subscription instead of a ReplayMerge, in this situation the communication is re-established automatically . Is this an expected scenario?
Todd L. Montgomery
@tmontgomery
@erik-lensherr_gitlab ReplayMerge does a single cycle of merging. Then is complete. You could just use the subscription at that point. And if there is a need for another merge later, create a new ReplayMerge.
but, ReplayMerge itself is not something the expects to toggle back and forth. It merges and is done.
Martin Thompson
@mjpt777
@erik-lensherr_gitlab As the name suggests it is a one shot replay and merge. Not a ReplayMergeRepeatedly.
Erik Lensherr
@erik-lensherr_gitlab
yeah, my doubts were not about the replay/merge part but the following one in which in order to continue processing switches to livestream. Once ReplayMerge is in MERGED state. I guess the cleanest way to go is to start a new replayMerge session.
Thanks @tmontgomery
Todd L. Montgomery
@tmontgomery
:thumbsup:
Noah Cornwell
@ncornwell
:wave: I had a 3 second network bonding failover (interface 1 was down, detected was down, failed over to interface 2). On my app side my aeron connection dropped but didn't recover for about 90 seconds. Any thoughts on the where I can go in aeron config etc to try to enable a more graceful or faster re-connect? Seems like I must be doing something wrong here.
Andrew Cowper
@bloat
Hi there - is there any information anywhere about the effect of running AeronStat (and friends) on a running mediadriver directory? Will it cause any delays? Alternatively, is it possible to copy a running mediadriver directory for analysis elsewhere? Thanks a lot!
Martin Thompson
@mjpt777
@bloat AeronStat has a minimal impact on a running driver. Subsequent writes to the counters will require cache invalidation traffic but nothing blocking.
The media driver directory can be copied but you cannot guarantee you will not see partial writes of data during the copy.
Andrew Cowper
@bloat
Thank you
Jussi Virtanen
@jvirtanen
A quick question regarding channels. (I tried to find a definitive answer to this on the wiki but, alas, couldn't.) If I have two publications identified by the same channel and different streams, 10 and 11, on host A, and I subscribe to the channel and stream 11 on host B, will this always result in all messages on both streams 10 and 11 being transported over the transmission media from host A to host B?
Jussi Virtanen
@jvirtanen
Rephrasing the above. Let's say I want to build an application that publishes multiple feeds. To conserve network bandwidth, I want to make the entire system work in such a way that subscribing to a feed from another host results in only that feed being transmitted over the network. Can I achieve this by making the feeds separate streams in a single channel or should I make each feed a separate channel?
Martin Thompson
@mjpt777
@jvirtanen Streams are multiplexed within the channel they belong to. Create channels for the routes you want data to flow. Then stream ids within the channels when you want to avoid head of line blocking. Sessions identify individual sources for stream and channel pairs.
maxuustalu
@maxuustalu

Hi, I'm new to Aeron and I'm looking to implement subscribers that can catch up from the archive before joining a live stream.

I've added archiving in the publisher (aeron:udp?control=localhost:40456|control-mode=dynamic). The subscribers use ReplayMerge. The issue I'm having is that ReplayMerge never merges the live stream. It alternates between CATCHUP and ATTEMPT_LIVE_JOIN. I narrowed it down to shouldStopAndRemoveReplay method where image.activeTransportCount() >= 2 never evaluates to true (image.activeTransportCount() == 1). I would be grateful if someone could advise what may be causing this.

Martin Thompson
@mjpt777
@maxuustalu That means it is not seeing the two streams merge so it cannot catch up to live. Is this a very fast moving stream? Do you have sufficient CPU resource and bandwidth? Lots could be going on. What version of Aeron, language, and JVM/OS platform are you using?
maxuustalu
@maxuustalu
@mjpt777 thanks, I've reduced the number of messages to 1 per 2 secs, but the issue remains.
Martin Thompson
@mjpt777
That is not a fast rate. Something is not right in how it is being used.
@maxuustalu You can see the test in CI goes at a much higher rate.
maxuustalu
@maxuustalu
Hi, I'm looking to build a high-availabilty sequencer with Aeron cluster and I'd like the cluster clients to be able to catch up with the live stream from the sequencer archive. I was wondering how well AeronCluster mix with ReplayMerge.
Martin Thompson
@mjpt777
@maxuustalu We only offer support on Cluster to those on a commercial agreement.
maxuustalu
@maxuustalu
Thanks. Just noticed in the doc it is described as preview feature. What is the plan to make it complete?
Martin Thompson
@mjpt777
A number of our clients are in production with Cluster. It is pretty solid and will have features added in time. We have it marked as preview as we reserve the right to change the API if we agree that with our customers on support.
tedvash
@tedvash
Hi, I am trying to understand the overhead of my Aeron setup, is there any config mode or setting I can enable to plot a histogram, or in general some metrics as to how much time it us taking to hand over messages from a producer to a consumer? I am using ipc, one to one setup.
Martin Thompson
@mjpt777
@tedvash You can see the overhead of Aeron in these benchmarks https://github.com/real-logic/benchmarks
For IPC on the same server socket you can expect well under 100ns one-way. Those benchmarks produce histograms.
tedvash
@tedvash
Thanks for this, I see the ideal times in those benchmarks but looking to check my own Aeron latency if possible, because I might have misconfigured something.
Martin Thompson
@mjpt777
@tedvash We can offer consultancy or commercial support for that.
Ivan Zemlyanskiy
@QIvan

hi! I was exploring your benchmarks and I wonder what do you think about running benchmarks with docker containers? I found it very useful and very convenient for everybody else to reproduce the results on their machines, but I guess you might have concerns about affecting the results.
My experience tells it barely noticeable:

thanks

Martin Thompson
@mjpt777
@QIvan Are you using loopback in your benchmarks? Try around machines and see if there is a difference.
Ivan Zemlyanskiy
@QIvan
do you mean network=host or just localhost?
with network=host should be no difference from network point of view, the program should use core directly for sending frames
Ivan Zemlyanskiy
@QIvan
at least it how I understand it now
Martin Thompson
@mjpt777
@QIvan I mean what endpoint are using using in the Aeron channel?
If source and destination are on the same machine then loopback will be used on Linux regardless.
Ivan Zemlyanskiy
@QIvan
yes, it's localhost by default
Martin Thompson
@mjpt777
Not a real test of Docker then.
You need to run on different machines.
Ivan Zemlyanskiy
@QIvan
got it, I'll try to find ones
thanks for the pointer
Martin Thompson
@mjpt777
You're welcome ;-)
feyyaz91
@feyyaz91
Hi, i have a recorded publication on host A. I would like to replicate this to a destination archive on host B. When attempting to call replicate() on host B to begin the replication, how can i obtain the srcRecordingId from from host A that i require? Its not clear to me how i would get this. Any pointers appreciated. Cheers, Feyyaz
Todd L. Montgomery
@tmontgomery
feyyaz91
@feyyaz91
Thank you, will take a look
VadimMolodyh
@VadimMolodyh
Hi,
I'm looking for a low-latency messaging system supporting strict ordering of messages with mutliple writters and readers for the same queue (i.e. with resolved distributed consensus issue). I see that Kafka provides it but has quite high latency (~15ms for p95 end-to-end latency). It looks like Aeron Cluster provides this features as well, but I do not see actual end-to-end benchmarks.
Plus, I could not find C++ API, is it in plans?
Thanks, Vadim.
Martin Thompson
@mjpt777
@VadimMolodyh Aeron Cluster can beat Kafka by a few orders of magnitude on latency. At present there are Java and C# APIs for Cluster. We would be interested in doing a C++ API if someone was willing to sponsor the development.
VadimMolodyh
@VadimMolodyh
Got it. Thanks!