These are chat archives for IndySockets/Indy

8th
Aug 2016
fan_tangshan
@sainimu78
Aug 08 2016 18:09
blob
A bug in Indy TCP, it losses data!!!
In TIdTCPServer.OnExecute
SetLength(arrBuf, 0);
argContext.Connection.IOHandler.ReadBytes(arrBuf, -1);
OnReceive(@(arrBuf[0]), Length(arrBuf));
In the callback function OnReceive, counts the "Length(arrBuf)", the total number of bytes received. After 10000 times sending, there are about 9000~16000 bytes losed.
fan_tangshan
@sainimu78
Aug 08 2016 18:14
In the client app, it sends buffers with random size
blob
Remy Lebeau
@rlebeau
Aug 08 2016 18:20
ReadBytes() does not lose data (unless you are performing reads across multiple threads at the same time and corrupting the IOHandler.InputBuffer). You are asking ReadBytes() to return whatever arbitrary bytes are currently available at the time it is being called. However, when AByteCount is -1, ReadBytes() always reads from the socket first, even if there are pending bytes still in the InputBuffer. If there are no new bytes on the socket at that moment, it waits for the IOHandler's ReadTimeout to elapse before then returning whatever is in the InputBuffer. And ReadTimeout is set to infinite by default. So, if there is pending bytes in the InputBuffer, not no new bytes on the socket, ReadBytes() will not return until the next client send. You could check IOHandler.InputBufferIsEmpty() first, and if False then call IOHandler.InputBuffer.ExtractToBytes() directly, otherwise call ReadBytes() instead.
However, that is actually not necessary, because your server code is wrong to begin with. Your client is sending a data packet that has a fixed header on it that includes the data size in it. You server code is completely ignoring that header. You need to have the server read the header, then read the specified number of bytes, then you can pass the completed packet to OnReceive. Then there is no problem.
SetLength(arrBuf, 8); argContext.Connection.IOHandler.ReadBytes(arrBuf, 8); argContext.Connection.IOHandler.ReadBytes(arrBuf, PInteger(@arrBuf[4])^, True); OnReceive(@arrBuf[0], Length(arrBuf));
Remy Lebeau
@rlebeau
Aug 08 2016 18:26
Note that you are sending integers in host byte order, which means you are subject to endian differences between the client and server. You really should be using network byte order instead. If you are not going to use the IOHandler's Read/Write methods for integer values, at least use Indy's GStack.HostToNetwork() function before sending the data, and the GStack.NetworkToHost() function after receiving the data, to make sure the integers are in a consistent endian on the wire.
PUInt32(@ sendBuf [0])^ := GStack.HostToNetwork(UInt32(CMDCOMMU_HEADER)); PUInt32(@ sendBuf [4])^ := GStack.HostToNetwork(UInt32(bufsize)); ... PUInt32(@ arrBuf [0])^ := GStack.NetworkToHost(PUInt32(@ arrBuf [0])^); PUInt32(@ arrBuf [4])^ := GStack.NetworkToHost(PUInt32(@ arrBuf [4])^);
Remy Lebeau
@rlebeau
Aug 08 2016 18:36
Otherwise, use the IOHandler's reading/writing functions: with cmdClient.fIdClient.IOHandler do begin Write(Int32(CMDCOMMU_HEADER)); Write(Int32(bufsize)); Write(buf); end; ... with argContext.Connection.IOHandler do begin PktType := ReadInt32; PktSize := ReadInt32; ReadBytes(arrBuf, PktSize); end;
fan_tangshan
@sainimu78
Aug 08 2016 18:40
@rlebeau Telling me so much, thanks, I will try InputBufferIsEmpty.
Are there simple ways to handle non-fixed size large buffer receiving and to receive buffer with no sticky packs?
Do uses GStack.HostToNetwork to prevent some network device reading error?
Remy Lebeau
@rlebeau
Aug 08 2016 18:59
What is the point of putting a fixed header on the packets if your server is not going to use the header for reading the packet? You really shouldn't be doing arbitrary reading in this situation. You know the exact data size up front, so just ask Indy to read that many bytes. Problem solved.
fan_tangshan
@sainimu78
Aug 08 2016 18:59
packet splicing
Remy Lebeau
@rlebeau
Aug 08 2016 18:59
What are you talking about?
fan_tangshan
@sainimu78
Aug 08 2016 19:02
I don't know what it calls, it's just two or more packets received at one time when using TIdTCPServer
Remy Lebeau
@rlebeau
Aug 08 2016 19:05
TCP is a byte stream, it doesn't matter if the data is tranmitted on the wire in 1 packet or 100 packets. TCP guarantees reliable delivery. The bytes will be accurate on the receiving end, no matter how many packets it has to read. But, if you want to reduce the number of TCP packets transmitted, you can use Indy's write buffering when sending a packet, eg: with cmdClient.fIdClient.IOHandler do begin WriteBufferOpen; try Write(...); Write(...); Write(...); WriteBufferClose; except WriteBufferCancel; raise; end end;
fan_tangshan
@sainimu78
Aug 08 2016 19:11
Then my need is call packet based TCP?
Freq, client sends two times, server gets just one time.
Some real time requirement
Remy Lebeau
@rlebeau
Aug 08 2016 19:15
TIdTCPClient and TIdTCPServer use TCP, hence their names. Do you understand how TCP actually works? Do you understand that there is no 1-to-1 relationship between sends and reads in TCP (unlike in UDP)? You could sent 100 bytes 1 time, but read 1 byte 100 times. TCP guarantees the bytes are delivered, and in the right order, but how you send the bytes does not affect how you read the bytes and vice versa.
In any case, regarding the 2-sends 1-read issue, it likely has to do with how ReadBytes(-1) interacts with the InputBuffer. After the OnExecute event exits, the server calls IOHandler.Connected before firing OnExecute again. Connected performs a read, so it is possible that bytes are getting read into the InputBuffer and a subsequent ReadBytes(-1) is not checking for them before waiting for more bytes to arrive on the socket. If you were reading the packet header, like you should be, you wouldn't have that issue.
fan_tangshan
@sainimu78
Aug 08 2016 19:26
I see, I know how to do