These are chat archives for IndySockets/Indy

1st
Sep 2017
Remy Lebeau
@rlebeau
Sep 01 2017 01:31
@DelphiWorlds no work has been started yet to support 1.1.0: IndySockets/Indy#183
DelphiWorlds
@DelphiWorlds
Sep 01 2017 01:32
Thanks for the update
robert010
@robert010
Sep 01 2017 01:56
Looking for help on udp performance with delphi and Rad studio with wireless link
Remy Lebeau
@rlebeau
Sep 01 2017 01:56
@robert010 what kind of help?
robert010
@robert010
Sep 01 2017 01:58
Performance. I am using Udp so I can broadcast to multiple devices. I have fairly small messages, 96 bytes every 77 ms.. everything works fine over wired lan, but it doesn't work reliably over wireless
I found the best performance by setting the client (sender) buffer to 0, and flat out sending the output message twice.
Wondering what I can tweak and what status I can look at to see what's happening.
Remy Lebeau
@rlebeau
Sep 01 2017 02:00
@robert010 define "doesn't work reliably". You do know that UDP is not reliable in general to begin with, right? Are you designing any reliability into your protocol? Sequence numbering, ACKs, resends, etc?
robert010
@robert010
Sep 01 2017 02:02
I am sending a waveform, and instead of moving smoothly, e.g. drawing at the same rate it is sent. it is jumpy while on wireless, and it skips some
I check the integrity of what I rcv.. it is good.
Remy Lebeau
@rlebeau
Sep 01 2017 02:03
@robert010 integrity of each individual packet doesn't matter if packets are dropped, received out of order, repeated, etc.
@robert010 are you sending the raw wave data as-is, or are you wrapping it in something like RTP?
robert010
@robert010
Sep 01 2017 02:03
Out of order isn't the problem it is missing and delayed.
not using any rtp type protocol,, very simple slow stuff, just an index per increment.
Remy Lebeau
@rlebeau
Sep 01 2017 02:05
@robert010 that is a defining charactertictic of UDP, you can't guarantee packet delivery, or guarantee timing. Packets may be dropped, packets may take different network routes to the target, etc. You have to be prepared to handle that. Put an incrementing sequence number in the packets, require receivers to ACK packets received, re-send any packets that are not ACKed, reorder received pacekts, etc. Of course, this only applies if you require data integrity.
robert010
@robert010
Sep 01 2017 02:06
yeah but this is a lot of losses,,, and there are only two or three devices on the network and the network is isolated from the world.
it is a visual wave,, so you see what is happening.
Remy Lebeau
@rlebeau
Sep 01 2017 02:07
@robert010 if the data is important, why are you using UDP?
robert010
@robert010
Sep 01 2017 02:07
I may have to switch to a tcp socket.
Data is for visualization only so occasional glitch would be ok
I know the disadvantages of udp.
Remy Lebeau
@rlebeau
Sep 01 2017 02:08
@robert010 have you monitored the network traffic on the sending side to make sure that packets are being sent correctly and in the desired time? Have you monitored the network traffic on the receiving side to make sure packets are being received?
robert010
@robert010
Sep 01 2017 02:09
But it is the only simple way of broadcasting.. yes.. and no.. i have checked the sending side.
Remy Lebeau
@rlebeau
Sep 01 2017 02:10
@robert010 and? were outgoing packets being sent correctly and in time? And no, there are other options, like multicasting
robert010
@robert010
Sep 01 2017 02:10
My suspicion is that indy is not sending the packets out evenly spaced and the wireless router is throwing away udps when it needs to.
explain multicasting
Remy Lebeau
@rlebeau
Sep 01 2017 02:11
@robert010 Indy passes the packets to the network stack as soon as you send them in code. Any delays woudl have to be on the OS layer
robert010
@robert010
Sep 01 2017 02:11
could be.
is there a feedback on the os stack status out of indy
Remy Lebeau
@rlebeau
Sep 01 2017 02:14
@robert010 no. Have you clocked the outgoing packet sends to see if any delays are occuring? Have you queried/tweaked the socket's outgoing buffer size at all? Is your sending machine doing other things at the time of sending that might take CPU time away from handling network requests?
robert010
@robert010
Sep 01 2017 02:16
The problem is not in the sender or receiver, it is in the router with wireless. If I hook up one recv hardwired and the second wireless, and the sender is hardwired and that is all that are on the router, then the wireless one looses and jumps the wired one works.
Remy Lebeau
@rlebeau
Sep 01 2017 02:17
@robert010 then get a faster router. Or maybe the issue is in the receiver's wireless, not the router's wireless (again, you should monitor the network traffic on the receiving side). Not much you can do about it in code, unless you play around with QOS settings to give your packets higher priority on the network.
robert010
@robert010
Sep 01 2017 02:19
thats a good idea thanks
if you have a simple way of doing multiple tcp connections, I am all ears. Best I can think is to use a initial UDP to establish the active port number for a connection.
Remy Lebeau
@rlebeau
Sep 01 2017 02:23
@robert010 why? Cant the sender act as a server and have the devices connect to it? You don't need UDP unless you are trying to discover a port dynamically, and you won't need that if the sender is a server and the devices are configured where to connect to.
In any case, do try multicasting first, that is better on network traffic than broadcasting since it is opt-in (receivers actively register with the network to receive packets) rather than "anyone want it?" (dumping packets on the network and anyone who happens to be listening can receive). That might make a difference in reducing router overhead and prioritizing packets, before you resort to switching to TCP
robert010
@robert010
Sep 01 2017 02:28
when I look up multi-casting I can't find any good examples
Remy Lebeau
@rlebeau
Sep 01 2017 02:30
@robert010 Indy has multicasting components - TIdIPMCastClient and TIdIPMCastServer
Remy Lebeau
@rlebeau
Sep 01 2017 02:36
@robert010 you might want to read up on what multicasting is from an API perspective: Multicast Programming, Multipoint and Multicast Semantics, Protocol-Independent Multicast and Multipoint, etc. There is a lot involved beyond simple UDP, but the jist is that multicasting allows you to send a packet once (to a multicast group IP) and it is delivered only to recipients who have joined the same multicast group (have notified the network that packets directed to the group IP should be delivered to the receiver's listening socket).
robert010
@robert010
Sep 01 2017 02:38
yeah but is it documented, first web searches aren't very encouraging. I will give it a try... thanks for the helpl
Remy Lebeau
@rlebeau
Sep 01 2017 02:40
@robert010 of course it is documented. A web search finds all kinds of articles on it.
robert010
@robert010
Sep 01 2017 02:40
Yeah one of the first is why is it a secret.
Remy Lebeau
@rlebeau
Sep 01 2017 02:42
@robert010 it is not a secret. it is a standardized (set of) protocol(s) used for all kinds of things. Especially in IPv6, which doesn't even support broadcasting (only IPv4 does)
DelphiWorlds
@DelphiWorlds
Sep 01 2017 03:14
good luck making IPv6 multicasting work on iOS.. and perhaps also Android
robert010
@robert010
Sep 01 2017 22:58
I need help implementing a client reading thread.. I have an example from http://borland.public.delphi.internet.winsock.narkive.com/MDQ9vtPV/how-do-i-use-tidtcpclient-in-nonblocking-mode-using-d7. that I am trying to use as a template
FConn.ReadFromStack(True, 1000, False);
if FConn.InputBuffer.Size > 0 then needs to be changed as TIdTCPConnection no longer has readfromstack
I assume that it should be something like fconn.socket.read??
Remy Lebeau
@rlebeau
Sep 01 2017 23:05
@robert010 You are using an example that was written for Indy 9. If you read the rest of the conversation, it does say ReadFromStack() is only for Indy 9, and to use ReadFromSource() instead for Indy 10. For example:
FConn.IOHandler.ReadFromSource(True, 1000, False);
if not FConn.IOHandler.InputBufferIsEmpty then ...
robert010
@robert010
Sep 01 2017 23:09
I wish I had an indy10 reference manual is it online
Remy Lebeau
@rlebeau
Sep 01 2017 23:13
@robert010 http://www.indyproject.org/Sockets/Docs/index.aspx. It is not up-to-date, but it should get you started.
robert010
@robert010
Sep 01 2017 23:15
ok how do I declare readfromSource e.g what unit or class
Remy Lebeau
@rlebeau
Sep 01 2017 23:16
@robert010 it is in the TIdIOHandler class in the IdIOHandler unit. IOHandler is a property of TIdTCPConnection. Most of the old reading/writing methods in TIdTCPConnection were moved to TIdIOHandler in Indy 10, and some were also renamed
robert010
@robert010
Sep 01 2017 23:19
fconn.iohandler.readfromsource is not defined ,,, I am being stupid
I am not sure what that line in the sample code does anyway... the fconn.iohandler.inputbuffer.size >0 should trigger the reads??
Remy Lebeau
@rlebeau
Sep 01 2017 23:22
@robert010 my bad,ReadFromSource() is protected, use CheckForDataOnSource() instead:
FConn.IOHandler.CheckForDataOnSource(1000);
FConn.IOHandler.CheckForDisconnect;
if not FConn.IOHandler.InputBufferIsEmpty then ...
robert010
@robert010
Sep 01 2017 23:24
how do I parse out my packets... is there a size associated with each send from the server>
Remy Lebeau
@rlebeau
Sep 01 2017 23:24
@robert010 CheckForDataOnSource is the actual read, the InputBuffer is what it reads into. But in most cases, you shouldn't be reading data this way to begin with, even in Indy 9. This approach should only be used when you don't know what you are reading and have to read arbitrary chunks of raw bytes. Most network protocols have structure to them, you should be coding to that structure instead. TIdIOHandler has many methods that deal with structured data (integers, strings, string lists, streams, etc). What do the sent packets actually look like? What kind of protocol are you dealing with?
robert010
@robert010
Sep 01 2017 23:27

fixed record structures : TMessageID = record
MessageType:Byte;
DataStartIndex:int16;
DataLength:int16;
CheckSum:Int32;
end;
TsampleSendData = array [1..samples_per_send] of int16;

TWaveFormSample = record
MessageID:TmessageID;
WaveformIndex:int16;
WaveformLength:int16;
WaveformData:TsampleSendData;
EndofData: int32;

end;

the last record is what is sent twaveformsample
Remy Lebeau
@rlebeau
Sep 01 2017 23:28
@robert010 since you are dealing with fixed-sized data, I would use TIdIOHandler.ReadBytes() to read the data, or even ReadByte, ReadInt16, and ReadInt32 to read the individual values
robert010
@robert010
Sep 01 2017 23:29
that was my thought but do I have to parse for the EOD or is there a definitive start?
Remy Lebeau
@rlebeau
Sep 01 2017 23:31
@robert010 the MessageType is the start, look for that. Is WaveFormData variable-length based on WaveformLength? Or is it always samples_per_send * 2 bytes in size, but only filled with WaveFormLength bytes? And what about DataLength, what does that apply to? And what is with the Index fields?
robert010
@robert010
Sep 01 2017 23:32
always a fixed size right now... the size change is for future
index are local fields
Remy Lebeau
@rlebeau
Sep 01 2017 23:33
@robert010 making the change for variable length data will be harder later if you lock it into fixed sized data today.
@robert010 local to what, though? What do they represent? Indexes don't usually have much meaning in network protocols
robert010
@robert010
Sep 01 2017 23:35
iindex is used downstream in the parsing.
I will be happy with it working fixed size right now
Is there a timeout on readbytes?
Remy Lebeau
@rlebeau
Sep 01 2017 23:35
@robert010 if the data is fixed sized, why do you have EndOfData, and why is it an Int32?
robert010
@robert010
Sep 01 2017 23:36
I used int32 so I could have 4 bytes all compliments of each other.
Remy Lebeau
@rlebeau
Sep 01 2017 23:36
@robert010 I/O operations are blocking by default, using an infinite timeout. TIdIOHandler does have a ReadTimeout property, though
robert010
@robert010
Sep 01 2017 23:36
$F0F0
$f00F
Remy Lebeau
@rlebeau
Sep 01 2017 23:37
@robert010 that is not necessary for a binary protocol that specifies data lengths
robert010
@robert010
Sep 01 2017 23:37
yeah I know.
how do I gracefully recover from out sync reads if there is no timeout
is there a buffer clear?
Remy Lebeau
@rlebeau
Sep 01 2017 23:39
@robert010 don't put wasted stuff like that in your protocol. If you get a timeout, you should just abort the connection. If you absolutely must resync (and I don't think you should), a unique start identifer at the front of valid packets is all you really need. Then you can keep reading until you find it (and yes, there is a way to clear the buffer - its called reading :-) No, seriously, there is a memory buffer clear, too).
robert010
@robert010
Sep 01 2017 23:40
is there a timeout on readbytes or does it hang forever
Remy Lebeau
@rlebeau
Sep 01 2017 23:40
@robert010 I already asnwered that
robert010
@robert010
Sep 01 2017 23:43
sorry i missed it .
robert010
@robert010
Sep 01 2017 23:48
how do I sync this thread back with the main ui after I read the block of data?
Remy Lebeau
@rlebeau
Sep 01 2017 23:50
@robert010 There are many ways to do that, depending on your needs. TThread.Synchronize(), TThread.Queue(), TIdSync, TIdNotify, (Post|Send)Message(), queues, etc. Inter-thread communication is a very broad topic.
robert010
@robert010
Sep 01 2017 23:57
in the android world I am not sure about the post and send messages,, ideally I would 'call' an event from this thread with the data? Your suggestion?