Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
  • Oct 16 04:22
    rlebeau labeled #269
  • Oct 16 04:22
    rlebeau opened #269
  • Oct 08 19:00

    Fulgan on Restructure

    Bug fix for a typo in TIdIMAP4.… (compare)

  • Oct 08 19:00

    Fulgan on master

    Bug fix for a typo in TIdIMAP4.… (compare)

  • Oct 02 21:00

    Fulgan on Restructure

    Updating TIdIMAP4's InternalSea… (compare)

  • Oct 02 21:00

    Fulgan on master

    Updating TIdIMAP4's InternalSea… (compare)

  • Sep 20 21:50

    Fulgan on master

    Embarcadero patch for race cond… (compare)

  • Sep 20 21:50

    Fulgan on Restructure

    Embarcadero patch for race cond… (compare)

  • Sep 10 18:50
    rlebeau closed #268
  • Sep 10 18:50
    rlebeau commented #268
  • Sep 10 18:50

    Fulgan on Restructure

    Fix for TIdResponseHeaderInfo.S… (compare)

  • Sep 10 18:50

    Fulgan on master

    Fix for TIdResponseHeaderInfo.S… (compare)

  • Sep 10 18:49
    rlebeau labeled #268
  • Sep 10 18:49
    rlebeau labeled #268
  • Sep 10 18:49
    rlebeau assigned #268
  • Sep 10 18:49
    rlebeau review_requested #268
  • Sep 09 13:15
    gjdoornink opened #268
  • Aug 28 21:00

    Fulgan on Restructure

    Setting TIdSSLIOHandlerSocketBa… (compare)

  • Aug 28 21:00

    Fulgan on master

    Setting TIdSSLIOHandlerSocketBa… (compare)

  • Aug 28 19:42
    rlebeau milestoned #183
jimakoz
@jimakoz
Hi guys, my set up is Delphi 10.1 and Indy 10.6.2.5341 and basically I’ve got an issue with the TIdFTPServer when clients abruptly disconnect during a data transfer. So, the problem is really the fact that when an abruptly disconnect occurs the server doesn’t pick it up and never triggers the OnDisconenct() event. I’ve introduced a mechanism that periodically checks for timed out connections but I cannot find a way to completely kick out the connection. Any ideas?
jimakoz
@jimakoz
That’s the code I’m using to clear idle connections, but unfortunately doesn’t work.
with ftpServer.Contexts.LockList do
begin
try
for i := Count - 1 downto 0 do
begin
Context := TidContext(List[i]);
if Context = nil then Continue;
Context.Connection.IOHandler.WriteBufferClear;
Context.Connection.IOHandler.InputBuffer.Clear;
Context.Connection.IOHandler.Close;
if Context.Connection.Connected then Context.Connection.Disconnect;
end;
finally
ftpServer.Contexts.UnlockList;
end;
end;
jimakoz
@jimakoz
Hi, I think I have found some sort of a work around to this issue. Instead of getting the TIdContext context of a connection I get the TIdFTPServerContext instead. Then by calling the KillDataChannel method I can disconnect fully the connection. Yes, it produces a couple of exception but the OnException Event will trap all of those, so no problem!
Remy Lebeau
@rlebeau
@jimakoz abnormal disconnects take time for the OS to detect, they are not immediate. Only graceful disconnects are. What you are doing is VERY dangerous code, because you are manipulating connections that may be actively busy doing things, like processing commands or transferring files. You are not doing anything to validate the current state of the connections. Each client runs in its own thread, you can't just wipe the buffers, or rip out the data channel, from behind the thread's back. If you really want to kill idle connections, just set a timeout on each connection in the OnConnect event, and let the client thread raise an exception if the timeout elapses while waiting for new data from the client. You can do the same thing for the data channel conection during each transfer. Let the server handle any raised exception and it will close the connection(s) for you. You can use an IOHandler's own ReadTimeout property, or you can enable TCP layer keep-alives using the IOHandler's Binding.SetKeepAliveValues() method.
jimakoz
@jimakoz
@rlebeau many thanks for your reply. I have tried indeed setting both ReadTimeout and SetKeepAliveValues in the OnConnect event as part of a solution, but nothing is happening when the client disconnects abruptly. For example when a client uploads a file and the network cable gets unplugged the server will never trigger the disconnect event. It will release the connections only when the server gets deactivated, with errors simi!ar to the ones i get with the above solution. How can i set timeouts on the data channel connection?
Remy Lebeau
@rlebeau
@jimakoz Let me say it again - "abnormal disconnects take time for the OS to detect" You are NOT going to get an immediate reaction from the OS, it needs time to timeout internally, and that can take a LONG time, but it will happen EVENTUALLY. Until that happens, socket operations will not report failures. TCP is designed to recover connections after short network outages, so the OS has to wait awhile before it kills a lost connection for good. If you don't want to wait that long, you have to use your own timeout in your own code. TCP keepalives help with that, as do reading timeouts. You might also consider using Binding.SetSockOpt(SO_SNDTIMEO) and Binding.SetSockOpt(SO_RCVTIMEO) on platforms that support those options (like Windows).
@jimakoz as for setting a data channel timeout, there does not appear to be a specific event that is appropriate for that, but TIdFTPServer.OnDataPortAfterBind might work, at least in Active mode transfers (probably not for Passive mode transfers since an inbound connection is not accepted yet).
jimakoz
@jimakoz
@rlebeau , i see what you are saying, but unfortunately the connection will NEVER timeout (even if i set the keepalive and readtimeout values) , we're talking about days here. I've seen cases that the connection was still "active" even after a month! Anyhow, i might give it another go with the SetSockOpt option but i doubt is going to make any difference. I think the issue is on the data channel side, that never gets released...thanks for your support anyway.
Remy Lebeau
@rlebeau
@jimakoz The OS will certainly never wait THAT long, so the socket code is either deadlocked, or probably stuck in an endless loop somewhere. Rather than rip the connection out, you should debug the server to find out where the code flow is going to when the cable is pulled out and then patch the code to address that.
jimakoz
@jimakoz
Thanks @rlebeau , I'll try that and if i find something i'll let you know
Justin
@klsyzzz
hi got another question, The property TIdSMTP.Port, is it true that if useTLS then TLS will select a different port for connection? for example, if I assign port 25 to the TIdSMTP, when start TLS, does it using 25 or use different port like 587?
Remy Lebeau
@rlebeau
@klsyzzz it depends on what you set UseTLS to. If you set UseTLS=utUseImplicitTLS and the Port is currently 25 or 587, the Port is changed to 465. If you set UseTLS=utUseExplicitTLS and the Port is currently 25 or 465, the Port is changed to 587. If you set UseTLS=utNoTLSSupport and the Port is currently 465 or 587, the Port is changed to 25. If you want to use a specific Port, set UseTLS first, then set the Port afterwards
Justin
@klsyzzz
thanks @rlebeau, i'm using utUseExplicitTLS, so if the port currently is 587, it will use 587 no change, right?
Remy Lebeau
@rlebeau
@klsyzzz yes
Justin
@klsyzzz
thank you very much @rlebeau
irawancepu
@irawancepu
Using Indy 10.6.2.0, i got warning that TidNotify is deprecated. What the replacement is? How to use it?
Remy Lebeau
@rlebeau
@irawancepu the answer is in the deprecated warning message: 'Use static TThread.Queue()', which is all TIdNotify calls, so you may as well just call it directly. That is why TIdNotify is deprecated.
Sergey
@icegood

Hi, Remy. After migration from Indy9 to Indy10 i found one more issue:
our web server always requires Content-Length in header of HTTP Post method. It doesn't matter whether ContentStream is assigned or not.
In Indy10 code i see that in case when ASource is not assigned then header is not added:
if Assigned(ARequest.Source) then begin
ARequest.ContentLength := ARequest.Source.Size;
end else begin
ARequest.ContentLength := -1; => leads to omitting of 'Content-Length'
end;
For this moment i have workaround via adding 'Content-Length' to CustomHeaders (value is '0', works great!). Moreover, seems Indy9 always added 'Content-Length'!

Please, consider adding 'Content-Length' always in library as well. At least i see it is a widespread issue:
http://stackoverflow.com/questions/19227142/http-status-code-411-length-required

perverez
@perverez
Hi, I have problem with GStack.LocalAddress. When I call this function on Windows Server 2016 (ver. 1607, build 14393, virtualized), error occured. Function Stub_GetAdaptersAddresses() returns any pointer, but GetAdaptersAddresses() return error 127. Some solution?
Remy Lebeau
@rlebeau
@icegood Indy 9 does not allow the source stream to be nil. If the request is POST or PUT, it assumes a valid stream and will crash accessing the Size if the stream is nil. Indy 9 also omitted the Content-Length for all requests other than POST and PUT. Indy 10 allows a nil source stream, but the real question is - why are you posting a nil source stream to begin with? You really shouldn't be. That being said, I have checked in a fix to generate a 'Content-Length: 0' header when the source stream is nil for POST and PUT requests.
Remy Lebeau
@rlebeau
@perverez error 127 is ERROR_PROC_NOT_FOUND. GetAdaptersAddresses() should not be returning that error, it comes from GetProcAddress() instead. Did you check to make sure Indy is able to load iphlpapi.dll correctly before calling GetAdaptersAddresses()?
perverez
@perverez
Hi, I check all - DLL is loaded (hIpHlpApi > 0), GetAdaptersAddresses is assigned, function Stub_GetAdaptersAddresses not calling Impl_GetAdaptersAddresses. The problem does not occur in the physical network interface cards, only in Hyper-V.
Remy Lebeau
@rlebeau
@perverez Then that is an OS issue, not an Indy issue. Ask Microsoft why GetAdaptersAddresses() returns 127 in Hyper-V (apparently you have already tried to?). It is likely a driver bug or something like that.
Remy Lebeau
@rlebeau
@perverez does GetAdaptersInfo() exhibit the same problem? If I have Indy treat ERROR_PROC_NOT_FOUND the same as ERROR_NOT_SUPPORTED when calling GetAdaptersAddresses(), Indy would then fallback to GetAdaptersInfo().
Sergey
@icegood

Hi, Remy. I wonder, what Indy10 means by 'Content-Length' for TIdMultipartFormDataStream?

I have next code:

AMultiPartFormData := TIdMultipartFormDataStream.Create; // Size=-1, OK

AField := AMultiPartFormData.AddFormField(AId, '', '', 'application/octet-stream'); // size = 222, might be OK

DataStream.CopyFrom(AStream, ALength); // ALength = 100500 bytes
ADataStream.Position := 0;
AField.FieldStream := ADataStream; // after that size became even smaller 220 bytes! why? i expect to see it >= ALength

moreover i must also sent 'Content-MD5' header and have no idea what should be there... In previous version of code with Alcinoe library we had there
CalculateMD5ForStream(ADataStream); and everything works well!

Sergey
@icegood
Should be ADataStream.Position := 0; before Post. But question about decreasing of size remains...
Remy Lebeau
@rlebeau
@icegood The Content-Length is the total bytes of the entire generated MIME content, from opening boundary to closing boundary, as one would expect. AMultiPartFormData.Size is not calculated until TIdHTTP requests it during posting, at which time the current field content is counted, but there is no way the size could be calculated so low, it takes the full MIME headers and FieldStream.Size into account (see TIdFormDataField.GetFieldSize()). How exactly are you determining the size is decreasing? You can use TIdHashMessageDigest5 to generate an MD5 hash, but there is currently no option to add custom headers to TIdMultipartFormDataStream fields, sorry.
Sergey
@icegood

@rlebeau, i've just checked change of size via Debug Watch . As i said above it happens only in case when my stream position is at the end.

I have another question, my HTTP protocol has KeepAlive=True and i see that in this case no disconnection happens:

if not Response.KeepAlive then begin
// TODO: do not disconnect if hoNoReadMultipartMIME is in effect
// TODO: do not disconnect if hoNoReadChunked is in effect
Disconnect;
end;

but i wonder whether is it a good idea to disconnect in case when socket error occurs? E.g. 'connection reset by peer'?

Sergey
@icegood
@rlebeau : "but there is currently no option to add custom headers to TIdMultipartFormDataStream fields, sorry."
we don't need that , do we? I've added MD5 header directly to TIdHTTP.CustomHeader and it works for me
irawancepu
@irawancepu
Using Delphi 2007, Indy 10 version 10.6.2.0, i got : E2251: ambiguous overloaded call to 'Post'. How to resolve this?
irawancepu
@irawancepu
Whew, there are 8 overloaded functions in idHttp.pas What i wanto to do actually is simply read response from portchecker.co. From there i can decide my port forwarding correctly or not. Many combination on Post methods are possible. Depend on STRING_IS_ANSI directive too. Too hard for me.
davidpn
@davidpn
you could just read the return value of the version of Post that returns a value
you can use code completion to determine which overloads are applicable
I mean code insight
irawancepu
@irawancepu
Here is my fragment code procedure TfMain.btnPostClick(Sender: TObject);
var ms: TMemoryStream; s: string;
begin
mmoLog.Clear; s:= '';
ms := TMemoryStream.Create;
try
try
s := idhttpX.Post(edtHost.Text,nil);
mmoLog.Lines.Text := s;
except on E: Exception do ShowMessage(e.ClassName + ':' + e.Message);
end; // try..except
finally ms.Free;
end; // try..finally
end;
davidpn
@davidpn
does the server support only Post for that URL? Are you able to use a Get?
Post usually includes sending content, and you're passing nil, so the compiler does not know whether to use the overload with TStream or with TIdMultiPartFormDataStream
thus the "ambiguous" message
irawancepu
@irawancepu
This error raise up on compile process. From html text the server produces, yes this server support POST method.
davidpn
@davidpn
I asked if it also supports Get
I know it will raise a compile error; I explained why
irawancepu
@irawancepu
Coz, i never pass compile process, err: E2251, i never know whether this site (yougetsignal.com and portchecker.co) is really support POST method or not. I only evaluate their html i got using Curl, curl.haxx.se. Using web browser, Torch and Firefox , everything is fine, so i assume it support GET method.
davidpn
@davidpn
ok.. so use Get instead
irawancepu
@irawancepu
But this site wat POST method, not GET method. Can i use GET method to emulate POST method? Here is a part of html code from portchecker.com. <form class="row" id="mainForm" action="/check" method="post">
<div class="large-7 columns">
<div class="row">
<div class="large-7 columns">
<label>Your IP Address</label>
<input type="text" name="target_ip" id="targetIP" value="36.78.221.82" disabled />
</div>
<div class="large-5 columns">
<label>Port Number</label>
<input type="text" name="port" id="portNumber" value="80" required/>
</div>
</div>
<div class="common-ports-list" id="labels-wrapper">
<span class="label info" data-port="80" >HTTP - 80</span>
<span class="label success hidden-on-mobile" data-port="115" >SFTP - 115</span>
<span class="label alert" data-port="25">SMTP - 25</span>
<span class="label info" data-port="8080" >HTTP - 8080</span>
<span class="label alert" data-port="21" >FTP - 21</span>
<span class="label hidden-on-mobile" data-port="22" >SSH - 22</span>
<span class="label warning" data-port="143">IMAP - 143</span>
<span class="label success hidden-on-mobile" data-port="443" >HTTPS - 443</span>
<span class="label alert hidden-on-mobile" data-port="873" >RSYNC - 873</span>
<span class="label info hidden-on-mobile" data-port="110" >POP3 - 110</span>
<span class="label success hidden-on-mobile" data-port="993">IMAPS - 993</span>
</div>
</div>
<div class="large-5 columns btn-wrapper">
<button type="submit" class="button">Check</button>
</div>
</f
davidpn
@davidpn
yes, it will require Post. It also means you'll need to pass the content it expects when you call Post. You're passing nil, which means no content
irawancepu
@irawancepu
problem is, i never pass compile process. how can i send the data it needs then? i will try using other delphi version. maybe D2007 is too old. very hard to keep many versions.
davidpn
@davidpn
it has been a while since I have done that with Indy, perhaps @rlebeau can help when he is around
I might be able to find an example
davidpn
@davidpn
there's an example here which is similar to what you need to do:
http://stackoverflow.com/questions/7762584/post-problems-with-indy-tidhttp
Although you can ignore the parts about SSL. Refer to the accepted answer
the main thing is that you need to pass the content that the URL expects, in the 2nd parameter where you are passing nil instead