Data Collection for Foglight End User Appliances

Hi all, Rick Tenison here Principal Solutions Architect primarily focused on the End User solutions here at Quest software. There have been many posts and blogs by various individuals on data capture methodology and I wanted to talk to a couple of points today.


  • The cleanest and simplest way to get data from your network to the Foglight Experience Monitor appliance is a network TAP. This is a device that goes in-line between say a firewall and a load balancer; the data is copied and sent out to the appliance over two connections one for ingress and one for egress. This is usually the least expensive solution as well.


  • A second type of TAP is called a Regeneration TAP. This type of TAP is installed just as in the above simple tap however the hardware has the ability to regenerate the traffic to more than one device, so in essence a one to two tap would be connected in-line and then the data feed goes out four ports to two devices that being two outbound ingress and two outbound egress. This type of TAP is a little more costly but can add flexibility.


  • A third type of TAP is called an Aggregation TAP. This type of TAP is again in-line however it combines both ingress and egress traffic into a single stream that is output to the appliance. This is a less desirable method of data collection as some packets may be dropped especially if the bandwidth limit of the TAP output is reached.


  • Another method that is commonly discussed is using a SPAN port off of your switch or load balancer, while this is convenient and almost no cost it is notorious for dropping packets and poor data quality, since the Foglight Experience Viewer is recording for playback the data stream if a packet is lost or the Foglight Experience Monitor cannot decrypt a session because of lost packets then the quality of the replay can and in most cases will be compromised. It is highly suggested that this method only be used in an extreme case where no other methodology can be used.


  • Newer on the market today are aggregation switches. These are switches that are designed to take in TAP or SPAN traffic and allow you to output that data to multiple devices. These switches also allow you to filter that data that you output up to layer three. So in essence you could output the data going to host through host and only ports 80, 443 and 3306 for example. These aggregation switches bring a lot of flexibility to your datacenter in that appliances such the Quest appliances, intrusion detection systems, network sniffer as well as other products no longer have to each have their own TAP or SPAN. There is a caveat to using these solutions. Since the data is only aggregated and regenerated onto your configured ports the switch does not know of data it never gets. That said if you feed these devices with SPAN or aggregation TAPs the data that goes to the aggregation switch will in most cased be of poor quality. To insure the best quality data for your monitoring, security and troubleshooting needs, use a TAP to send data to the devices. Some of the vendors in this space have facilities for TAP modules right in the aggregation switch, others rely on external feeds. I am a big supporter of these devices but only when the data feed is clean to begin with.


Thanks all and we will be talking to you again in the future. Please let me know if you have any questions or comments.