These are chat archives for IndySockets/Indy

27th
Sep 2018
code4tips
@code4tips
Sep 27 2018 18:55

Hi, i am looking for suggestions in reducing the code inside IdTCPServer.Execute event. It works great, but as you can see how busy it is in the below pseudo-code with just a few of the functions I am executing within it. What is the recommended way to farm out some of the processing into other functions without violating the threading model?

pCode begin
ServerExecute(AContext: TIDContext)
if Authenticated (persist as multiple files can be sent for the same authenticated session)
If Command is SENDINGFILE then
Get Attributes (FileName and expected bytes, dates)
Post file attributes to DB - set download start time to NOW
Create the FileStream in the download folder
Get the File
Track number of bytes received for verification (Implement CRC?)
Decompress File
If decompression successful then send number of bytes received - this is confirmation (client to track bytes sent = byte received)
if decompression fails send 0 bytes as bytes received - cleanup
Send File received - 200 message - (do not send a 500 as we want to continue with the next file)
end SENDINGFILE
end
else
If Process Login Credentials is successful Send Greeting - 200 Welcome xxx
else If Authentication Failed
Log the event - record IP, time, and UserName
Send Login Failed - 400 Login Failed
Disconnect the connection (Force?)
end Authentication Failed
end ProcessLogin
end Authenticated

pCode end

Thank you

Remy Lebeau
@rlebeau
Sep 27 2018 19:41
@code4tips is authentication always the first command? If so, you could perform authentication in the OnConnect event instead, and then cache the results in the TIdContext for OnExecute to use. Also, if your protocol consists of textual commands, then depending on the particular formatting, you might have a look at TIdCmdTCPServer and its CommandHandlers collection. Then you can define separate OnCommandevent handlers for each of your individual commands. You might also consider decompressing the file while you are downloading it, to save time and disk space. If the compression is using deflate or gzip, Indy has streaming classes for that.
code4tips
@code4tips
Sep 27 2018 20:55
@rlebeau 1 - Authentication will always be first - Can a connection be rejected if the auth fails? I will look at TidCmdTCPServer . I am avoiding streaming classes as some of the expected files are huge , more than 4 gb. I tried streams (outside of Indy) and ran into issues with different file sizes. The client is expected to compress using ComponentACE ECLCompressHugeFile routine, so I will check if there is compatibility with gzip or deflate. The fallback (in case performance is degraded) is to fire off a separate thread to do the decompression and let the main thread continue to download the next file from the client queue. At some point system would need to communicate back the result of the decompression so I will be having fun trying persist data across socket sessions. Thinking along the lines of a manifest, until the client receives a manifest of all the files transferred it should not disconnect so the context is maintained. Client will review the manifest and resend files as needed.
Remy Lebeau
@rlebeau
Sep 27 2018 21:21
@code4tips yes, you can reject the connection if auth fails. Just close the connection, or raise an exception. I would defintely recommend using streams, ESPECIALLY if you are dealing with such large files. You don't want to download a 4GB file and then have to decompress it separately, that will take way too much time and waste way too much disk space. You can decompress the data in chunks as the file is being downloaded, writing only the uncompressed bytes to disk. That will also allow the client to get a reply immediately after finishin the transfer and not have to wait to get it at some unknown future time. If you were having problems with streaming then you likely just weren't doing it correctly.
Marcos Douglas B. Santos
@mdbs99
Sep 27 2018 22:33
Very good explanation
Kudzu
@czhower
Sep 27 2018 22:35
+`1