Several customers have inquired about how our UEM solution deals with HTTP applications that use NTLM authentication. How does our UEM solution (Foglight Experience Monitor, Foglight Experience Viewer) replay behave with the Windows-based NTLM authentication? What can be seen?
To the typical web application user who integrates with NTLM, they may get automatically logged in from their user credentials from their Windows operating system. For those who are not automatically authenticated, they may be presented with a dialog box asking for credentials like this one:
Since that dialog is handled by the browser, and not actual HTML, this cannot be replayed. What gets displayed instead in the browser during an FxV replay is quite different:
Why? Short answer: it's pretty ugly what goes over the http conversation during an NTLM handshake. Plain and simple.
What can you do? Well, in FxV you can create a simple hit filter that matches on the 401 http code and simply discard it. This doesn't mean the data didn't go over the wire, you are just choosing not to store that hit in the FxV database, and therefore squelch it from replay.
What about FxM? Glad you asked. FxM is going to drop these by default. There is a setting in "Advanced Options" seen below where NTLM is dealt with simply by discarding it.
If you're still interested why NTLM is so chatty over http, here is a great article referencing it...http://davenport.sourceforge.net/ntlm.html#respondingToTheChallenge :
Appendix B: Application Protocol Usage of NTLM
This section examines the use of NTLM authentication within some of Microsoft's network protocol implementations.
NTLM HTTP Authentication
Microsoft has established the proprietary "NTLM" authentication scheme for HTTP to provide integrated authentication to IIS web servers. This authentication mechanism allows clients to access resources using their Windows credentials, and is typically used within corporate environments to provide single sign-on functionality to intranet sites. Historically, NTLM authentication was only supported by Internet Explorer; recently, however, support has been added to various other user agents.
The NTLM HTTP authentication mechanism works as follows:
1. The client requests a protected resource from the server:
2. GET /index.html HTTP/1.1
3. The server responds with a 401 status, indicating that the client must authenticate. "NTLM" is presented as a supported authentication mechanism via the " WWW-Authenticate " header. Typically, the server closes the connection at this time:
4. HTTP/1.1 401 Unauthorized
5. WWW-Authenticate: NTLM
6. Connection: close
Note that Internet Explorer will only select NTLM if it is the first mechanism offered; this is at odds with RFC 2616, which states that the client must select the strongest supported authentication scheme.
7. The client resubmits the request with an " Authorization " header containing a Type 1 message parameter. The Type 1 message is Base-64 encoded for transmission. From this point forward, the connection is kept open; closing the connection requires reauthentication of subsequent requests. This implies that the server and client must support persistent connections, via either the HTTP 1.0-style "Keep-Alive" header or HTTP 1.1 (in which persistent connections are employed by default). The relevant request headers appear as follows (the line break in the " Authorization " header below is for display purposes only, and is not present in the actual message):
8. GET /index.html HTTP/1.1
9. Authorization: NTLM TlRMTVNTUAABAAAABzIAAAYABgArAAAACwALACAAAABXT1
10. JLU1RBVElPTkRPTUFJTg==
11. The server replies with a 401 status containing a Type 2 message in the " WWW-Authenticate " header (again, Base-64 encoded). This is shown below (the line breaks in the " WWW-Authenticate " header are for editorial clarity only, and are not present in the actual header).
12. HTTP/1.1 401 Unauthorized
13. WWW-Authenticate: NTLM TlRMTVNTUAACAAAADAAMADAAAAABAoEAASNFZ4mrze8
14. AAAAAAAAAAGIAYgA8AAAARABPAE0AQQBJAE4AAgAMAEQATwBNAEEASQBOAAEADABTA
15. EUAUgBWAEUAUgAEABQAZABvAG0AYQBpAG4ALgBjAG8AbQADACIAcwBlAHIAdgBlAHI
16. ALgBkAG8AbQBhAGkAbgAuAGMAbwBtAAAAAAA=
17. The client responds to the Type 2 message by resubmitting the request with an " Authorization " header containing a Base-64 encoded Type 3 message (again, the line breaks in the " Authorization " header below are for display purposes only):
18. GET /index.html HTTP/1.1
19. Authorization: NTLM TlRMTVNTUAADAAAAGAAYAGoAAAAYABgAggAAAAwADABAAA
20. AACAAIAEwAAAAWABYAVAAAAAAAAACaAAAAAQIAAEQATwBNAEEASQBOAHUAcwBlAHIA
21. VwBPAFIASwBTAFQAQQBUAEkATwBOAMM3zVy9RPyXgqZnr21CfG3mfCDC0+d8ViWpjB
22. wx6BhHRmspst9GgPOZWPuMITqcxg==
23. Finally, the server validates the responses in the client's Type 3 message and allows access to the resource.
24. HTTP/1.1 200 OK
This scheme differs from most "normal" HTTP authentication mechanisms, in that subsequent requests over the authenticated connection are not themselves authenticated; NTLM is connection-oriented, rather than request-oriented. So a second request for " /index.html " would not carry any authentication information, and the server would request none. If the server detects that the connection to the client has been dropped, a request for " /index.html " would result in the server reinitiating the NTLM handshake.
A notable exception to the above is the client's behavior when submitting a POST request (typically employed when the client is sending form data to the server). If the client determines that the server is not the local host, the client will initiate reauthentication for POST requests over the active connection. The client will first submit an empty POST request with a Type 1 message in the " Authorization " header; the server responds with the Type 2 message (in the " WWW-Authenticate " header as shown above). The client then resubmits the POST with the Type 3 message, sending the form data with the request.
The NTLM HTTP mechanism can also be used for HTTP proxy authentication. The process is similar, except:
- The server uses the 407 response code (indicating proxy authentication required) rather than 401.
- The client's Type 1 and 3 messages are sent in the " Proxy-Authorization " request header, rather than the " Authorization " header.
- The server's Type 2 challenge is sent in the " Proxy-Authenticate " response header (instead of " WWW-Authenticate ").
With Windows 2000, Microsoft introduced the "Negotiate" HTTP authentication mechanism. While primarily aimed at providing a means of authenticating the user against Active Directory via Kerberos, it is backward-compatible with the NTLM scheme. When the Negotiate mechanism is used in "legacy" mode, the headers passed between the client and server are identical, except "Negotiate" (rather than "NTLM") is indicated as the mechanism name.