These are chat archives for IndySockets/Indy

26th
Jun 2017
Walter Prins
@ByteJuggler
Jun 26 2017 14:08

@rlebeau Thanks. The proxy was definitely not just allowing unauthenticated access previously. However there has been some changes to the proxy config I've been told to improve security so this adds an unknown, which apparently may result in 407 errors on some sites. Google however is not one of these.

Leaving that aside, I've however since discovered another variable, which is that Google seems to be processing the request differently to before.

That, is, it turns out that https://www.google.com now returns a redirect (HTTP 302) and not a direct HTTP 200; It looks like, with Indy, this is being somehow associated to the tail end of Authentication processing and ends up in effect appearing as a 407 failure. (As opposed to with Chrome/Firefox where it redirects and works as expected.)

If I however change the code to directly fetch the HTTP redirected URL, then Indy succeeds to fetch the page via the proxy (with NTLMSSP) without issue and the test passes.

So, there still seems to be something strange going on. All else being equal I should not arguably be seeing different behaviour between the browser and an Indy client, on the same PC, using the same Windows login, working through the same proxy that only allows NTLM authentication, I'd say, in the case of an immediate redirect, wouldn't you agree? (Again, both Indy and browser works as expected if no immediate redirect is present and you just fetch the URL directly.)

As an aside, I traced the code and Indy is detecting NTLM as the (only requested/allowable) auth protocol from the proxy and automatically opting to use TIdSSPINTLMAuthentication class, due to the registration of same in unit initialization. As you'll see I did start out adding handlers etc for OnSelectProxyAuthorization() and OnProxyAuthorization() but it turned out to be unnecessary.

Here is the modified code:
type TAuth = class
  procedure DoSelectAuthorization(Sender: TObject; var AuthenticationClass: TIdAuthenticationClass; AuthInfo: TIdHeaderList);
  procedure DoProxyAuthorization (Sender: TObject; Authentication: TIdAuthentication; var Handled: Boolean);
  end;

procedure TAuth.DoSelectAuthorization(Sender: TObject; var AuthenticationClass: TIdAuthenticationClass; AuthInfo: TIdHeaderList);
begin
  //It turns out AuthenticateClass is already set to TIdSSPINTLMAuthentication by
  //TIdCustomHTTP.DoOnProxyAuthorization when this event is called.
  //
  //It does this by inspecting AResponse.ProxyAuthenticate
  //list to see what protocols are requested/supported by the proxy and then
  //trying to look up a suitable registered class to use to handle the requirement.
  //
  //In our case this of course contains 'NTLM' which results in TIdSSPINTLMAuthentication
  //being looked up as appropriate class to use.

  //TIdSSPINTLMAuthentication gets looked up because this class was previously
  //automatically registered by idAuthenticationSSPI unit initialization
  //via line 1320 call to RegisterAuthenticationMethod('NTLM', TIdSSPINTLMAuthentication);
  //which registers it into global "AuthList".  It is registered by inclusion in uses clause
  //via IdAuthenticationSSPI (or IdAllAuthentications).

  //Consequently we don't have to provide anything here and I instead just
  //assert here that we're using the expected auth class instead:

  Assert(AuthenticationClass = TIdSSPINTLMAuthentication);
end;

procedure TAuth.DoProxyAuthorization (Sender: TObject; Authentication: TIdAuthentication; var Handled: Boolean);
begin
  //This is never called? (Presumably since not applicable because with SSP (single sign-on protocol)
  //the NTLM response is generated automatically.)
  ShowMessage(Authentication.Authentication);
  Assert(Authentication.Authentication <> '');
end;

procedure TTextMagicTests.test_SSL_proxy_access_to_google_via_NTLMSSP;
var
  LIdHTTP : TIdHTTP;
  LSSLIOHandler : TIdSSLIOHandlerSocketOpenSSL;
  LRequestStr, LResult : String;
  LAuth : TAuth;
begin
  //Using 'https://www.google.com' as LRequestStr results in HTTP 302 from google
  //which seems to result in what ends up looking like a 407 Indy Exception to
  //the Delphi code...
  //
  //Why is this? It seems incorrect behaviour.  Indy should just deal with the
  //redirect, like a browser does faced with the same situation.
  //
  //Probably not the right diagnosis, but currently it seems Indy is seemingly
  //conflating the 302 as the tail-end "error" of proxy auth protocol (maybe?)
  //and is then surfacing the whole lot as the original 407 exception/response.
  //
  //However when using the redirected URL instead, the proxy auth works as expected
  //and no problems occur, and the test passes. So there the SSPINTLM auth works
  //correctly, aside from this specfic situation, it seems.

  LRequestStr := 'https://www.google.co.uk/?gfe_rd=cr&ei=aeJQWc2JM6nHXpaklfgG';

  LIdHTTP := TIdHTTP.Create(nil);
  LAuth := TAuth.Create;
  try
    LSSLIOHandler := TIdSSLIOHandlerSocketOpenSSL.Create(LIdHTTP);
    LIdHTTP.IOHandler := LSSLIOHandler;
    LSSLIOHandler.SSLOptions.Method := TIdSSLVersion.sslvTLSv1_2;

    //The following 2 lines are unnecesary as SSPI is used automatically due to
    //registration of TIdSSPINTLMAuthentication as handler for NTLM.
    //The proxy below only supports NTLM as well. They are kept in for context.
    LIdHTTP.OnSelectProxyAuthorization := LAuth.DoSelectAuthorization;
    LIdHTTP.OnProxyAuthorization := LAuth.DoProxyAuthorization;

    LIdHTTP.HandleRedirects := True;
    LIdHTTP.AllowCookies := True;
    LIdHTTP.ConnectTimeout := 10000;
    LIdHTTP.ReadTimeout := 10000;
    LIdHTTP.HTTPOptions := LIdHTTP.HTTPOptions + [hoKeepOrigProtocol] + [hoInProcessAuth];
    LIdHTTP.ProtocolVersion := pv1_1;
    LIdHTTP.ProxyParams.Clear;
    LIdHTTP.ProxyParams.BasicAuthentication := False;
    LIdHTTP.ProxyParams.ProxyServer := 'proxy';
    LIdHTTP.ProxyParams.ProxyPort := 3128;
    try
      LResult := LIdHTTP.Get(LRequestStr);
    except
      on E:EIdHTTPProtocolException do
      begin
        Fail('HTTP Error code: ' + IntToStr(E.ErrorCode));
      end;
    end;

    CheckEquals(200, LIdHTTP.ResponseCode);
  finally
    LAuth.Free;
    LIdHTTP.Free;
  end;
end;
Walter Prins
@ByteJuggler
Jun 26 2017 16:05
Just to add, I'll try to look into this a bit more if I get some time with sniffers like Wireshark or Fiddler etc. and post back if I find anything.
Remy Lebeau
@rlebeau
Jun 26 2017 18:33
@ByteJuggler HTTP redirects are not part of authentication. And you can't even get an HTTP redirect in the first place if a proxy is used and you havn't authenticated with the proxy yet. Proxies don't generally use redirects, a request to connect to a server either succeeds or fails, with or without authentication. There is no way Indy would treat 302 as a 407 failure. Look at the code in TIdHTTPProtocol.ProcessResponse(). The very first thing it does after validating the response headers is check for a 3xx redirect and handle it, otherwise it checks for 4xx authentication and handle it, otherwise it checks for 2xx success and handles it, otherwise it fails.