These are chat archives for IndySockets/Indy

2nd
Oct 2018
Marcos Douglas B. Santos
@mdbs99
Oct 02 2018 00:17
@rlebeau ok, I'll try - feedback as soon as possible - thanks!
Marcos Douglas B. Santos
@mdbs99
Oct 02 2018 00:50
Would be better if TIdAttachment has a property to give the content as RawByteString - I need to instantiate a temp stream just to get the content, and it takes time
Remy Lebeau
@rlebeau
Oct 02 2018 00:54
@mdbs99 TIdAttachment is an abstract base class, it doesn't handle content directly, descendent classes do. Indy has 2 descendants by default, TIdAttachmentFile and TIdAttachmentMemory. TIdAttachmentFile stores its content in a file on disk, TIdAttachmentMemory stores its content in an internal TMemoryStream. Use the virtual TIdAttachment.OpenLoadStream() method to access the underlying data as a TStream, then you can read its content as needed. If you want direct access to the raw data, for instance, using TIdAttachmentMemory, you can call OpenLoadStream() and then access the TMemoryStream.Memory property. Using TIdAttachmentFile, you can call OpenLoadStream() and then access the TFileStream.Handle property to use when creating a Win32 memory mapped view of the file (or, use the TIdAttachmentFile.StoredPathName property to get the full path and filename so you can open your own handle to the file).
Marcos Douglas B. Santos
@mdbs99
Oct 02 2018 00:59
@rlebeau but I didn't understand how Indy will choose between these classes to instantiate an attachment
To create, Ok, I can pick one - but for reading...
Remy Lebeau
@rlebeau
Oct 02 2018 01:00
@mdbs99 when loading an email, TIdMessage uses TIdAttachmentFile by default, unless overridden by an OnCreateAttachment event handler, which lets you create any TIdAttachment-derived class you want
Marcos Douglas B. Santos
@mdbs99
Oct 02 2018 01:01
(do you have a help online for all classes?)
Remy Lebeau
@rlebeau
Oct 02 2018 01:02
@mdbs99 https://www.indyproject.org/documentation but the material is a bit dated, it hasn't been updated for a long time (despite the recent website look&feel update), but the majority of the content is still relavant
Marcos Douglas B. Santos
@mdbs99
Oct 02 2018 01:03

when loading an email, TIdMessage uses TIdAttachmentFile by default, unless overridden by an OnCreateAttachment event handler

Got it

Ops! sorry, when I saw I thought it was just to download - but there is online
So, TIdAttachmentMemory has property DataString: string - however, I don't know if I may have trouble in Lazarus
I guess this should be RawByteString, what do you think @rlebeau ?
code4tips
@code4tips
Oct 02 2018 01:15

@rlebeau
@code4tips yes, you can reject the connection if auth fails. Just close the connection, or raise an exception. I would defintely recommend using streams, ESPECIALLY if you are dealing with such large files. You don't want to download a 4GB file and then have to decompress it separately, that will take way too much time and waste way too much disk space. You can decompress the data in chunks as the file is being downloaded, writing only the uncompressed bytes to disk. That will also allow the client to get a reply immediately after finishin the transfer and not have to wait to get it at some unknown future time. If you were having problems with streaming then you likely just weren't doing it correctly.

Thanks for the tips. Much easier managing code in CmdTcpServer. I am seeing one issue - When I do TCPClient.Sendcmd('TEST1', 200). The TEST1 handler in the TCPCmdServer appears to be returning two responses back. The second one just has '.' (period) in it. Is this normal? To work around it I had to code the client to execute two ReadLn, one before the command and one after. The one before retrieves the '.' (period) and the one after the actual response.

Got both the client and the server working, including progress bar on Client during deflate (do not need it on inflate). Thanks a bunch for all your help. I could not figure out how to inflate while receiving so I receive the data and then send the TMemoryStream to the Inflate routine, and recover the file.

Remy Lebeau
@rlebeau
Oct 02 2018 01:22
@mdbs99 no, it is intended to be a string, in whatever the compiler's native string type is, hence the name DataString. For FreePascal, that will be AnsiString or UnicodeString, depending on the {$mode} used. Lazarus is an IDE, not a compiler. It uses FreePascal for compiling, but it maps string to UTF8String for Lazarus projects. So, unless you actually compile Indy in Lazarus itself, it will end up assigning Indy's (Ansi|Unicode)String values to its own UTF8String variables, performing runtime conversions accordingly. Internally, the DataString getter reads the TMemoryStream using ReadStringFromStream() without any IIdTextEncoding, so it will be subject to Indy's GIdDefaultTextEncoding to process the stream bytes, and then output in whatever string type FreePascal is set to use. If you want to access the raw bytes, you need to use OpenLoadStream() instead of DataString.
Remy Lebeau
@rlebeau
Oct 02 2018 01:28
@mdbs99 actually, in Lazarus, string is not actually UTF8String, it is just AnsiString with the RTL's codepage set to UTF-8 at runtime (see http://wiki.freepascal.org/Unicode_Support_in_Lazarus#Technical_implementation and http://wiki.freepascal.org/Unicode_Support_in_Lazarus#Why_not_use_UTF8String_in_Lazarus.3F ). But when dealing with AnsiString in Indy, many classes and routines have extra parameters/properties to specify AnsiString encodings, so you would just set those to UTF-8 as well. Not an issue when using UnicodeString instead
Marcos Douglas B. Santos
@mdbs99
Oct 02 2018 13:43

actually, in Lazarus, string is not actually UTF8String, it is just AnsiString

@rlebeau I'll tell you that, but you were faster :)

Lazarus is an IDE, not a compiler.

As a programmer that compiles the compiler and Lazarus - without any installation - I'm pretty aware on that :)

If you want to access the raw bytes, you need to use OpenLoadStream() instead of DataString

Got it.

...you would just set those to UTF-8 as well

I guess that all string that comes from Indy are UTF-8 and I don't need to translate.

mezen
@mezen
Oct 02 2018 13:58
Hm, i was using TIdAttachmentFile.SaveToFile and handle the stored file on harddisk^^
Marcos Douglas B. Santos
@mdbs99
Oct 02 2018 15:04
@mezen I'm storing in a database
Remy Lebeau
@rlebeau
Oct 02 2018 17:03

@mdbs99

I guess that all string that comes from Indy are UTF-8 and I don't need to translate.

Actually, no. When Indy is compiled to use AnsiString instead of UnicodeString, Indy's default encoding is US-ASCII for most things, because most Internet protocols that Indy implements are originally based on US-ASCII with support for Unicode added in later specs. You can change the GIdDefaultTextEncoding variable at runtime to switch to UTF-8 by default. Though, in the case of email specifically, received text will usually be in whatever charset the email specifies, falling back to raw 8bit if the charset is unknown/unsupported. So you do need to pay attention to charsets in Ansi mode. In Indy 11, we're dropping support for AnsiString and going full Unicode.

Marcos Douglas B. Santos
@mdbs99
Oct 02 2018 20:21

@rlebeau

hough, in the case of email specifically, received text will usually be in whatever charset the email specifies

Well, for now that is my case.

In Indy 11, we're dropping support for AnsiString and going full Unicode.

I'm not sure nowadays about Delphi, but AFAIK, string=UnicodeString in these last versions, right? So, I believe all components interfaces will be the same, ie, continue using string all over places. Am I right?

Kudzu
@czhower
Oct 02 2018 20:30
Yes. Ansistring is bascially pre D2009. 2009 and later AnsiString is used only when explicitly wanted.
and generally for interop reasons to old DLLs etc.
Remy Lebeau
@rlebeau
Oct 02 2018 21:36
@mdbs99 In the latest FPC, string is still AnsiString by default, unless you use {$MODESWITCH UnicodeStrings} (or {$MODE DelphiUnicode}). In Delphi 2009+, string is UnicodeString unconditionally. Indy 11 will only support Delphi/C++Builder/RADStudio 2009+ and FPC 3+, and for FPC will use {$MODE DelphiUnicode} (currently {$MODE Delphi} is used), this way UnicodeString is used universally in all compilers for everything (except where AnsiString is explicitly needed, like for DLL interop, like Chad mentioned).