rlebeau on master
#395 Adding hcPATCH to THTTPCom… Updating TIdFTP.ExtListDir() to… When compiling for FPC, now def… and 2 more (compare)
procedure TForm1.SendBroadcast(const AServer: TIdIPMCastServer; const ABroadcast: string); var I: Integer; begin for I := 0 to FLocalAddresses.Count - 1 do begin if FLocalAddresses[I].IPVersion = AServer.IPVersion then begin AServer.Active := False; AServer.BoundIP := FLocalAddresses[I].IPAddress; AServer.BoundPort := AServer.Port; AServer.Active := True; AServer.Send(ABroadcast); end; end; end;
setsockopt()when adding a local IP to the multicast group, which is done after the socket is bound locally. The local IP that is being passed in is
in6addr_any), which comes from the
TIdSocketHandle.IPproperty, which is updated after binding. If the socket is bound to a specific local IP and not to in6addr_any, the
TIdSocketHandle.IPproperty should not be all zeros. That implies a possible failure in
TIdSocketHandle.UpdateBindingLocal(). But no matter, because the local IP is not currently being passed to
setsockopt(IPV6_ADD_MEMBERSHIP)in IPv6, it is set to the default multicast interface instead. Maybe that is what iOS is complaining about.
I have still problems with HTTP and NTLM Proxy :( IdAuthenticationNTLM, IdAuthenticationSSPI, IdAuthenticationDigest are used in the interface uses section
For debugging I created event handler for every event of TIdHTTP with logging all method parameters.
OnStatus (with hsConnecting) will be fired, then
OnStatus again (with hsConnected),
OnWorkBegin (AWorkCountMax = 2609),
OnWork (AWorkCount = 2609) and
OnWorkEnd as last... No
OnSelectProxyAuthorization is called!
The received header is
Proxy-Authenticate: Negotiate Proxy-Authenticate: NTLM Date: Tue, 06 Jun 2017 09:07:12 GMT Cache-Control: no-cache Pragma: no-cache Content-Type: text/html; charset="UTF-8" Content-Length: 2609 Accept-Ranges: none Proxy-Connection: keep-alive
The component will be created via
LHttp := TIdHTTP.Create(nil); LHttp.ProxyParams.ProxyServer := LProxy.ProxyServer; LHttp.ProxyParams.ProxyPort := LProxy.ProxyPort; if LProxy.UseAuthentification then begin LHttp.ProxyParams.BasicAuthentication := True; LHttp.ProxyParams.ProxyUsername := LProxy.Username; LHttp.ProxyParams.ProxyPassword := LProxy.Password; end; LHttp.Request.UserAgent := 'My Client ' + CVersion; // WebServer will always accept this user agent LHttp.OnAuthorization := HTTP1Authorization; LHttp.OnChunkReceived := HTTP1ChunkReceived; LHttp.OnConnected := HTTPConnected; LHttp.OnDisconnected := HTTPDisconnected; LHttp.OnHeadersAvailable := HTTP1HeadersAvailable; LHttp.OnProxyAuthorization := HTTP1ProxyAuthorization; LHttp.OnRedirect := HTTP1Redirect; LHttp.OnSelectAuthorization := HTTP1SelectAuthorization; LHttp.OnSelectProxyAuthorization := HTTP1SelectProxyAuthorization; LHttp.OnStatus := HTTP1Status; LHttp.OnWork := HTTP1Work; LHttp.OnWorkBegin := HTTP1WorkBegin; LHttp.OnWorkEnd := HTTP1WorkEnd; LHttp.Get(LUrl, LFileStream, [CAccepted_WebServerResponses]);
Did I miss something?
I also have Wireshark logs of the Get-Trys. The first Get from Indy sends already a Proxy-Authorization in Basic, is that correct? But Answer is always
HTTP/1.1 407 Proxy Authorization required and there is no more traffic, no second attampt is maded. My biggest problem is, I have no test enviremont. Every change I do, I have to create a new binary and give it to a customer (which knows that they are experimental and, for now, is happy to be a alpha tester).
MauxAuthRetries= 42. No success :(
BasicAuthenticationhas to be set before you send a request, since it helps govern the type of authentication sent in the request. The only way to determine the server's desired authentication type(s) is to send a request, even if just
HEAD, and see how the server responds. The authentication(s) are specified in the response's
Proxy-Authenticateheaders. If you plan on having TIdHTTP handle authentication,
hoInProcessAuthshould always be set. At the very least, it needs to be set before the response is parsed, so you could set it dynamically in the
OnHeadersAvailableevent, for instance. If
hoInProcessAuthis not set, you have to handle authentication manually by checking the
TIdHTTP.ResponseCodeand then setting the
TIdHTTP.Request.CustomHeaders.Values['Authorization']on the next request.
hoInProcessAuth = Trueand
BasicAuthentication = Falseand retry the GET request.
MaxAuthRetriesis still on default = 3) request instead of 1. Every request still has a basic proxy-authorization set.
BasicAuthentication=True, or the server actually requests
Proxy-Authenticate, or you force
TIdBasicAuthenticationdirectly. When TIdHTTP receives 407, the only ways that
OnSelectProxyAuthorizationwould not be triggered are either 1)
MaxAuthRetries, or 2)
ProxyParams.Authenticationis already assigned an auth class. After an auth class has been assigned, the only ways that
OnProxyAuthorizationwould not be triggered are either 1)
ProxyParams.ProxyPasswordis blank, or 2) the authentication class doesn't request user input (
TIdAuthentication.Next()does not return
wnAskTheProgram). You are going to have to step into Indy's souce code with the debugger to figure out what TIdHTTP is really doing during its 407 processing. TIdHTTP should be handling the retry requests for you. Make sure the credentials you are using are accurate. If they are not working, maybe the NTLM request is being malformed, so the proxy keeps rejecting it.
var LIdHTTP : TIdHTTP; LSSLIOHandler : TIdSSLIOHandlerSocketOpenSSL; LRequestStr, LResult : String; begin LRequestStr := 'https://www.google.com'; LIdHTTP := TIdHTTP.Create(nil); try // Setup of TidHTTP that supposedly works with Squid SSPINTLM authentication. LSSLIOHandler := TIdSSLIOHandlerSocketOpenSSL.Create(LIdHTTP); LSSLIOHandler.SSLOptions.Method := TIdSSLVersion.sslvTLSv1_2; LIdHTTP.IOHandler := LSSLIOHandler; LIdHTTP.HandleRedirects := True; LIdHTTP.AllowCookies := True; LIdHTTP.ConnectTimeout := 10000; LIdHTTP.ReadTimeout := 10000; LIdHTTP.Request.BasicAuthentication := True; LIdHTTP.HTTPOptions := LIdHTTP.HTTPOptions + [hoKeepOrigProtocol] + [hoInProcessAuth] + [hoNoProtocolErrorException] + [hoWantProtocolErrorContent]; LIdHTTP.ProtocolVersion := pv1_1; LIdHTTP.ProxyParams.ProxyServer := 'proxy'; LIdHTTP.ProxyParams.ProxyPort := 3128; LResult := LIdHTTP.Get(LRequestStr); CheckEquals(200, LIdHTTP.ResponseCode); finally LIdHTTP.Free; end; end;
TIdHTTP.OnSelectProxyAuthorizationevent if the
TIdHTTP.ProxyParams.Authenticationproperty is not assigned, and then the
TIdHTTP.OnProxyAuthorizationevent to ask for new credentials if needed. Neither of which you have assigned event handlers to. If authentication cannot be performed at all (unsupported auth scheme, no credentials provided, retry limit reached, etc), TIdHTTP should exit if
hoNoProtocolErrorExceptionis set, otherwise it will raise an exception. If authentication can proceed, TIdHTTP will try to perform it if
hoInProcessAuthis set, otherwise it should exit and you will have to send a new request with proxy authentication added.
CONNECTverb. And yes, it is looped, because it may take multiple HTTP requests to satisfy authentication, HTTP redirects, etc. The loop is hard-coded to "False" because the number of iterations needed is unknown. The inner body of the loop needs to decide when to exit the loop.
hoNoProtocolErrorExceptionwas never intended to be used with proxy handling. The loop in question does not look at the return value of
LLocalHTTP.ProcessResponseto break the loop if needed. A similar loop is in
TIdCustomHTTP.DoRequest(), but it does look at the return value and act accordingly. So similar logic will have to be added to
TIdCustomHTTP.ConnectToHost()when communicating with a proxy
hoWantProtocolErrorContent, and then catch the raised
EIdHTTPProtocolException. The response code will be in the exception's
ErrorCodeproperty, and any body content will be in the
Thanks for the response, very much appreciated. I only added
hoWantProtocolErrorContent) this morning as otherwise the code below the
Get() would never run etc. and when it failed, the raised exception didn't make it particularly obvious initially what had failed where (whereas having the Check like fail would've been rather more direct) so I thought I'd rather just stop the whole exception thing and check the result code, whatever it may be.
Anyway given what you've said however I'll remove these again and make do with catching the exception instead as suggested for now.
Aside from this I'm still puzzled about the test failing now with a 407 in the first place. I'm trying to understand and reconcile your explanation with the fact that this code was working previously through this proxy, and to add which works (still) with NTLMSSP via Chrome and Firefox with no problems.
So I'm slightly puzzled regarding the auth, how this worked at all in the past then, seemingly by some fluke, since if I understand you correctly you're implying that the above code could not auth properly against an NTLM proxy due to missing event handlers/properties? Or am I misunderstanding you? (I've added IdHTTPSSPI and IdHTTPNTLM units to the above code, which was previously missing to ensure these were registered but hasn't made a difference... shouldn't the presence of these units register the necessary handlers so that
hoInProcessAuth makes Indy deal with the authentication? I'm obviously being stupid about something... sorry!)