real-logic/Aeron

Efficient reliable UDP unicast, UDP multicast, and IPC message transport

Lorenzo Nicora

@nicusX

Thanks. I read that.
Re-reading "The socket address to which publications will send messages and from which subscriptions will receive messages" now I understand the subtlety. But it wasn't massively obvious.
(I don't want to criticise. Please, just get this as a newbie feedback)

Todd L. Montgomery

@tmontgomery

thanks! If you have any suggestions on where that info should be, feel free to let us know

i.e. where would you have expected it, etc.

Lorenzo Nicora

@nicusX

The endpoint paragraph on the wiki is fine. Clarifying the "socket address" is the subscriber (recipient) machine would have helped

Martin Thompson

@mjpt777

@nicusX I've just added that.

Lorenzo Nicora

@nicusX

Cool

Martin Thompson

@mjpt777

@zaradai Thanks, I've pushed a fix.

Mendel Monteiro-Beckerman

@MendelMonteiro

Hi, has there ever been any interest in being able to set QoS traffic classes on the sockets created by Aeron?

Todd L. Montgomery

@tmontgomery

@MendelMonteiro nothing that anyone has been willing to sponsor development on. But if interested in sponsoring it, we could look into it.

Mendel Monteiro-Beckerman

@MendelMonteiro

@tmontgomery thanks for the reply. I'll get in touch if sponsorship is an option.

William

@ilove7412369_twitter

i look at the stats:
sessionId=-1385188173 streamId=1001 channel=aeron:ipc?term-length=268435456 : pub-pos (sampled):251:1694624000 pub-lmt:251:1694623552

is that when it publish reach the pub-lmt, it will take tons of time for the archive to create new memory mapped file and zero them out causing such 10 s?

any ways to avoid?

William

@ilove7412369_twitter

sorry problem solved

Ivan Zemlyanskiy

@QIvan

hi! a quick question if you don't mind:
should in theory the taggedReplicate work with C Media Driver?
I'm getting a timeout exception on my even basic tests, while with Java MD everything is just fine.
thanks.

Martin Thompson

@mjpt777

This room is not being used to answer Aeron questions. Questions can go to our support channels or Stackoverflow which we may eventually get round to.

This room is being reserved for discussions with public contributors.

Martin Thompson

@mjpt777

We are considering shutting down this room. For now we will keep it open but will no longer be giving out free support.

Ivan Zemlyanskiy

@QIvan

oh, cool! glad to read it! It would be awesome if Aeron tag would become more popular on SO

I hope that'd be a good start https://stackoverflow.com/questions/65780369/aeron-should-in-the-taggedreplicate-work-with-c-media-driver =)

Martin Thompson

@mjpt777

@QIvan Please stop posting here if it is not related to contributions.

Ivan Zemlyanskiy

@QIvan

ok, sorry. Speaking about contributions, a little feedback about reading with Aeron:
why do we need this errorHandler in TermReader https://github.com/real-logic/aeron/blob/master/aeron-client/src/main/java/io/aeron/logbuffer/TermReader.java#L82 ?
I found it very inconveniet, because when I read with my own FH and I throw an exception, I want to use my own try-catch block around a poll, not the global errorHandler which I provided for Aeron and excpect handle aeron related exceptions.

Martin Thompson

@mjpt777

@QIvan This is really a question and not a contribution. I'll allow grace just this once. Look at the following: https://github.com/real-logic/aeron/blob/master/aeron-client/src/main/java/io/aeron/Aeron.java#L1124

William

@ilove7412369_twitter

Aeron archive replay seems quite slow when remote replicate from say Tokyo to SG. Is it not a right use of that? Any way should be better way to do for a higher throughput?

Martin Thompson

@mjpt777

@ilove7412369_twitter As previous stated we will not be offering free consultancy in the room any longer. This room is reserved for public contributor discussions. If you need support we have a commercial offering.

rinkigoyal

@rinkigoyal

I am trying to identify source stream Id from image unavailable && unavailable events.

Say we have 3 services running in 3 separate JVM && all of communicates via IPC channel.

Service one subscribes on streamId - 101
Service two subscribes on streamId - 102
Service three subscribes on streamId - 103

Service two creates publication for stream 101 && publish few messages

Service three creates publication for stream 101 && publish few messages

Service one receives two image available events with two different session Id. Is there any way to determine whether service two or service three has become available ?

Now service two is stopped && service one receives image unavailable , Is there any way to determine whether service two or service three has become un
available ?

Martin Thompson

@mjpt777

@rinkigoyal We do not offer free support in this room. It is for contributor discussions.

Ghost

@ghost~5fab2f45d73408ce4ff3c0e0

Is there a way for getting the sequence number of a message with Aeron? So if throughout the day I send 1 million messages can I know that I am dealing with say message #42 versus message #2010?

Martin Thompson

@mjpt777

@pieceofchum I'm getting bored of saying we are not offering support in this room. This is for contributor conversations.

Ghost

@ghost~5fab2f45d73408ce4ff3c0e0

Ok sorry I see that in my emails now no worries.

Eduard Silantyev

@eliquinox

Set up this beauty at home: https://twitter.com/eliquinox/status/1355517372749246464

Todd L. Montgomery

@tmontgomery

sweet!

hisitepu

@hisitepu

some type in aeron_stat.c, getopt processing -p but in case capturing 'd'. Thanks

i mean typo, not type

Martin Thompson

@mjpt777

@hisitepu If you are pointing out an issue then please raise one with a detailed example or test to illustrate.

bartlaw

@bartlaw

Seems that cluster egress messages are lost when FOLLOWER business logic is faster than LEADER.

By slowing down LEADER in BasicAuctionClusteredService.onSessionMessage() with

if (role == Cluster.Role.LEADER) {
  Thread.sleep(1000);
}

we get LEADER business logic to lag behind FOLLOWER. When LEADER goes down egress messages which were produced by FOLLOWER but not yet by LEADER are lost.

Maybe I'm talking nonsense but when CLIENT switches to new LEADER shouldn't it remember his egress position and ask for retransmission when gap is detected.

Martin Thompson

@mjpt777

@bartlaw This channel is for contributor conversations. We can provide commercial support to where needed.

Jeffrey Mazar

@jmazar

how does the MaxMulticastFlowControlSupplier decide what the "fastest" consumer is? Lets say I have 5 consumers and consumer 1 is the "fastest" but starts falling behind and then #2 is further along. Will that be reflected in the flow control or is it possible for a consumer which was the fastest to hang for some reason and then slow down the other 4? if that makes sense

Michael Barker

@mikeb01

Hi @jmazar we've changed the role of this room to be specific to public contributor discussions. For support-like queries the best place is: https://stackoverflow.com/questions/tagged/aeron

dantaras

@dantaras

Hi guys! Not sure if this is the right place to post, let me know if not. I'm working on a Julia version of an Aeron client using the C++ client as a model and had a question about the position limit mechanism.

In the offer methods in https://github.com/real-logic/aeron/blob/master/aeron-client/src/main/cpp/Publication.h position is checked against a limit (eg: https://github.com/real-logic/aeron/blob/master/aeron-client/src/main/cpp/Publication.h#L358) and if smaller publishing can continue. Shouldn't that check take into account how much data there is to be published? If, for example, the position limit is higher by one than the current position and we want to publish 100 bytes, then we will be allowed to publish, go over the limit and in extreme cases we might overwrite data which subscribers haven't read.

Michael Barker

@mikeb01

@dantaras This is handled within the TermAppender (TermAppender.h:164). If you are writing a client for another language, I would recommend looking at the C++ wrapper over the C API. There is quite a lot of complexity in the API and writers of clients for different languages have found it difficult to keep up with feature changes. Hence we've created the C API implementation to enable the development of APIs in other languages more quickly.

dantaras

@dantaras

Thanks for the response, Michael!

What is the link between position limit here https://github.com/real-logic/aeron/blob/master/aeron-client/src/main/cpp/Publication.h#L344 and term length here https://github.com/real-logic/aeron/blob/master/aeron-client/src/main/cpp/concurrent/logbuffer/TermAppender.h#L158? I was under the impression that position limit can be different and could limit publication within the same term.

I will have a look at the C API as well. Does the shared memory driver API change often?

Michael Barker

@mikeb01

@dantaras data won't be overwritten as it will only allow that message if it fits within a term and there is another cleaned term between the one being published to and the one being read from. If message is larger than the remaining space in the term then the term will be padded, the offer rejected with ADMIN_ACTION and the position will then be higher (or equal) to the limit and offer will be blocked with back pressure.

dantaras

@dantaras

I get it now, thanks @mikeb01 !

Valto Vipul

@ValtoVipul_twitter

hi guys, is there a way to fall back to the previous mode of segment file naming convention? 10-0,10-1,10-2, etc?

I have some scripts which used that so would like to see if possible to use the old behaviour

oops it seems no more help in this room

michaelredman

@michaelredman

Hi, I am trying to persist the egress with aeron cluster, the clustered driver already has an archive and various channels setup, is it a bad idea to use ClusteredMediaDriver.archive().startRecording(my_log) and try connect to it over udp? will the cluster client get confused?

Martin Thompson

@mjpt777

@michaelredman This channel is for contributor discussions only. If you require support we have commercial offerings.

William

@ilove7412369_twitter

I believe i encounter an Aeron Bug.
i publish multidestintaion cast from Host A to HostA and HostB, big message as big as 300k bytes. at rate say 10 times per second.
After 1-2 hours, one of the host , say host B, will get corrupted message. We detect that as it is SBE message with wrong data. While the same message properly processed correctly and successfully.
And then, we will not able to receive any message in host B.
I think bottom line we got exception instead of corrupt message as that make me worry the integrity of the data via aeron.

Where communities thrive

People

Repo info

Activity