real-logic/Aeron

Efficient reliable UDP unicast, UDP multicast, and IPC message transport

Martin Thompson

@mjpt777

I've not needed to use the Disruptor for many years. The Disruptor was designed to help work around the issues in other messaging products. Aeron was designed to avoid those issues.

Welly Tambunan

@welly87

right.. thanks a lot for clarification.

Martin Thompson

@mjpt777

Those issues include batching support, and allowing multiple consumers of a stream without having to copy the data.

Welly Tambunan

@welly87

btw, i see your presentation about high performance managed language. you said that C# is fastest without any tuning. and the API for CLR is good because it give more control than JVM. is that still true today? i think JVM has support Unsafe? but still no struct etc...

Martin Thompson

@mjpt777

Java has Unsafe but it is an unsupported API plus it does not have any memory layout control yet.

The optimiser and GC tends to be better for Java so it comes out as a wash for general applications.

Welly Tambunan

@welly87

right.. is that mean that JVM is have more mature than CLR?

Martin Thompson

@mjpt777

Not so much about maturity, more about investment in different areas.

Welly Tambunan

@welly87

right...

Welly Tambunan

@welly87

hi all, i’ve read article from Martin Fowler about LMAX. Is this still valid given that the Aeron replace the Disruptor? https://martinfowler.com/articles/lmax.html

I like clean domain model in DDD + Event Sourcing. I think Aeron is very perfect for that kind of architecture. thought?

Martin Thompson

@mjpt777

@welly87 We like to keep this chat room for Aeron design discussions. Aeron Cluster is a good fit for that type of architecture. We can provide commercial training and support if you would like to know more.

Welly Tambunan

@welly87

sure. is there any email i can sent to for consulting?

Martin Thompson

@mjpt777

sales@real-logic.co.uk

Welly Tambunan

@welly87

thanks

Welly Tambunan

@welly87

hi all, i’ve posting a question in stack overflow regarding the messaging pattern in aeron
https://stackoverflow.com/questions/63528428/aeron-messaing-pattern-for-pub-sub-one-way-and-competing-consumer

thanks a lot

Erik Lensherr

@erik-lensherr_gitlab

Greetings! I am trying to grasp the mechanics of ReplayMerge since I have a scenario in which an app starts a replaying an existing recording and then joins the live channel seamlessly using ReplayMerge class. What I can't figure out is why ReplayMerge, after successfully merging with live, does not reconnect automatically if the publisher goes away and returns (I am aware of the 10 seconds liveness timeout). This scenario is not the same if I have a Subscription instead of a ReplayMerge, in this situation the communication is re-established automatically . Is this an expected scenario?

Todd L. Montgomery

@tmontgomery

@erik-lensherr_gitlab ReplayMerge does a single cycle of merging. Then is complete. You could just use the subscription at that point. And if there is a need for another merge later, create a new ReplayMerge.

but, ReplayMerge itself is not something the expects to toggle back and forth. It merges and is done.

Martin Thompson

@mjpt777

@erik-lensherr_gitlab As the name suggests it is a one shot replay and merge. Not a ReplayMergeRepeatedly.

Erik Lensherr

@erik-lensherr_gitlab

yeah, my doubts were not about the replay/merge part but the following one in which in order to continue processing switches to livestream. Once ReplayMerge is in MERGED state. I guess the cleanest way to go is to start a new replayMerge session.

Thanks @tmontgomery

Todd L. Montgomery

@tmontgomery

:thumbsup:

Noah Cornwell

@ncornwell

:wave: I had a 3 second network bonding failover (interface 1 was down, detected was down, failed over to interface 2). On my app side my aeron connection dropped but didn't recover for about 90 seconds. Any thoughts on the where I can go in aeron config etc to try to enable a more graceful or faster re-connect? Seems like I must be doing something wrong here.

Andrew Cowper

@bloat

Hi there - is there any information anywhere about the effect of running AeronStat (and friends) on a running mediadriver directory? Will it cause any delays? Alternatively, is it possible to copy a running mediadriver directory for analysis elsewhere? Thanks a lot!

Martin Thompson

@mjpt777

@bloat AeronStat has a minimal impact on a running driver. Subsequent writes to the counters will require cache invalidation traffic but nothing blocking.

The media driver directory can be copied but you cannot guarantee you will not see partial writes of data during the copy.

Andrew Cowper

@bloat

Thank you

Jussi Virtanen

@jvirtanen

A quick question regarding channels. (I tried to find a definitive answer to this on the wiki but, alas, couldn't.) If I have two publications identified by the same channel and different streams, 10 and 11, on host A, and I subscribe to the channel and stream 11 on host B, will this always result in all messages on both streams 10 and 11 being transported over the transmission media from host A to host B?

Jussi Virtanen

@jvirtanen

Rephrasing the above. Let's say I want to build an application that publishes multiple feeds. To conserve network bandwidth, I want to make the entire system work in such a way that subscribing to a feed from another host results in only that feed being transmitted over the network. Can I achieve this by making the feeds separate streams in a single channel or should I make each feed a separate channel?

Martin Thompson

@mjpt777

@jvirtanen Streams are multiplexed within the channel they belong to. Create channels for the routes you want data to flow. Then stream ids within the channels when you want to avoid head of line blocking. Sessions identify individual sources for stream and channel pairs.

maxuustalu

@maxuustalu

Hi, I'm new to Aeron and I'm looking to implement subscribers that can catch up from the archive before joining a live stream.

I've added archiving in the publisher (aeron:udp?control=localhost:40456|control-mode=dynamic). The subscribers use ReplayMerge. The issue I'm having is that ReplayMerge never merges the live stream. It alternates between CATCHUP and ATTEMPT_LIVE_JOIN. I narrowed it down to shouldStopAndRemoveReplay method where image.activeTransportCount() >= 2 never evaluates to true (image.activeTransportCount() == 1). I would be grateful if someone could advise what may be causing this.

Martin Thompson

@mjpt777

@maxuustalu That means it is not seeing the two streams merge so it cannot catch up to live. Is this a very fast moving stream? Do you have sufficient CPU resource and bandwidth? Lots could be going on. What version of Aeron, language, and JVM/OS platform are you using?

maxuustalu

@maxuustalu

@mjpt777 thanks, I've reduced the number of messages to 1 per 2 secs, but the issue remains.

Martin Thompson

@mjpt777

That is not a fast rate. Something is not right in how it is being used.

@maxuustalu You can see the test in CI goes at a much higher rate.

maxuustalu

@maxuustalu

Hi, I'm looking to build a high-availabilty sequencer with Aeron cluster and I'd like the cluster clients to be able to catch up with the live stream from the sequencer archive. I was wondering how well AeronCluster mix with ReplayMerge.

Martin Thompson

@mjpt777

@maxuustalu We only offer support on Cluster to those on a commercial agreement.

maxuustalu

@maxuustalu

Thanks. Just noticed in the doc it is described as preview feature. What is the plan to make it complete?

Martin Thompson

@mjpt777

A number of our clients are in production with Cluster. It is pretty solid and will have features added in time. We have it marked as preview as we reserve the right to change the API if we agree that with our customers on support.

tedvash

@tedvash

Hi, I am trying to understand the overhead of my Aeron setup, is there any config mode or setting I can enable to plot a histogram, or in general some metrics as to how much time it us taking to hand over messages from a producer to a consumer? I am using ipc, one to one setup.

Martin Thompson

@mjpt777

@tedvash You can see the overhead of Aeron in these benchmarks https://github.com/real-logic/benchmarks

For IPC on the same server socket you can expect well under 100ns one-way. Those benchmarks produce histograms.

tedvash

@tedvash

Thanks for this, I see the ideal times in those benchmarks but looking to check my own Aeron latency if possible, because I might have misconfigured something.

Martin Thompson

@mjpt777

@tedvash We can offer consultancy or commercial support for that.

Ivan Zemlyanskiy

@QIvan

hi! I was exploring your benchmarks and I wonder what do you think about running benchmarks with docker containers? I found it very useful and very convenient for everybody else to reproduce the results on their machines, but I guess you might have concerns about affecting the results.
My experience tells it barely noticeable:

thanks

Martin Thompson

@mjpt777

@QIvan Are you using loopback in your benchmarks? Try around machines and see if there is a difference.

Ivan Zemlyanskiy

@QIvan

do you mean network=host or just localhost?

this is how I run https://github.com/easy-logic/transport-benchmarks/blob/master/aeron-docker-compose.yml

with network=host should be no difference from network point of view, the program should use core directly for sending frames

Ivan Zemlyanskiy

@QIvan

at least it how I understand it now

Where communities thrive

People

Repo info

Activity