Finding peace of mind with end user traffic analysis

Foglight is not just a forensics tool - it delivers operational intelligence every day on your web traffic. I’ve seen our customers use it to monitor far more than just the user experience. For example, on numerous occasions it has helped operations managers identify attempts to compromise their sites. One of our Foglight users recently caught an issue with a low level set of traffic hitting at a steady rate trying to do SQL injection and cross-site scripting. Normally, such attempts wouldn’t have visibility unless they were denial-of-service attacks. That peace of mind represents the real value of Foglight - it helps you protect your business in all aspects.

I know a number of you are concerned about this kind of unwanted traffic on your sites, especially with respect to exposure to cyber attacks. Many of you are addressing the risk of exposure manually, and by using tools such as Google Analytics. I’d like to walk you through an analysis I recently did for a client using Foglight's out-of-the-box ability to perform geographic analysis of end user traffic. In what follows, I've modified some of the images slightly in order to protect my client's intellectual property.

A Foglight APM appliance had been set up by the client's network administrator. The admin had connected our appliance to a port mirror that provided copies of all of the incoming and outgoing http and https traffic against their sites, and then assigned an IP address to the appliance. This setup took less than an hour of the admin's time and, shortly thereafter, Foglight started to show us every hit, page, and end user session in the monitored traffic:




Foglight lets you analyze end user traffic using any of the rich set of data and metadata that comes along for the ride with HTTP, but our initial use case was pretty simple: find slow traffic. It's easy to create a Hit Analyzer in Foglight that does this:




When you create a Hit Analyzer in Foglight, you can instantly assign multidimensional analytics directly to the filter. These built-in analytics are called Pivots, and there are a rich set of them built into Foglight. For example, you can set it up so that a filter for Slow Hits or Slow Pages automatically partitions the matched traffic by browser type or OS platform or geographic location. When we assigned a geographic region Pivot to our Slow Pages filter, we saw something interesting:




My client had no idea there was any traffic against their websites originating in China but the Map view confirmed the traffic origin:




Foglight instantly provided deeper information on this traffic:




The Hit Details view provides all of the HTTP request and response data and metadata, and a click on the Client Details button provided actionable details:




It seems that there was a web crawler or spider accessing my client's site. This had been happening for some time without their knowledge. I created a Hit Analyzer to focus specifically on this traffic:




Foglight quickly provided all of the contextual client-side information my client needed to take action and filter out this traffic. When I checked in on this client a few days later, the traffic from China was nonexistent:





We discovered this information about an hour after I started installing Foglight and, less than a day later, the client had updated their firewall policy to prevent further intrusions of unwanted traffic. The sensitive nature of their business is such that any vulnerability is potentially serious for them, and the peace of mind they will get from Foglight as their “watchdog” will be invaluable.


We addressed a similar situation with another client recently where the network admin was certain he had blocked all international traffic, but pulling up the Foglight interface showed little balloons popping up all over China. As he commented to me, “You don’t know what you don’t know.”


I hope you found this interesting, and I hope you continue having fun with Foglight.