Real-time monitoring of a collection of heterogeneous systems operating within a production environment, and doing so in such a manner that does not impact the behaviors of those systems, has, and continues to be, a challenge. Some traditional reporting systems require expensive, time-delayed processing of log files. Other systems provide real-time reporting but can only offer limited visibility into the internal processes. In short, while several systems and techniques in the prior art address some aspects of reporting, each has its limitations.
One attempted technique is Logfile Analysis. This typically involves post-processing standard transaction files produced by a web server to generate reports. The logfiles reside on the company's servers. This method worked well in the early days of the web when the primary statistic was a hit (an HTTP get request). In order to more accurately measure human interaction page views (a single count for a page that may have required several HTTP requests) and visits (sessions) emerged as more meaningful analytics.
Logfile analytics packages were extended to consolidate requests into page views and pages into higher level groupings (e.g., shopping pages, checkout pages, purchase pages and support pages).
Some problems with logfile analysis include the following. A visit to a previously displayed page may result in the page being retrieved from the client browser's cache—The page view would not appear in the web log and therefore the user's path through the site would not be accurately reflected. Only limited information can be captured—This includes basic characteristics (e.g., browser type, client IP and cookies) and the requested URL. Important information that may be contained in the text of the page (e.g., product selected, user zipcode or error message) can not be captured—Inconsistencies are to be expected as the logfiles may be spread across several servers. The files from the multiple servers must be collected and combined together for reporting. During this process one may expect slight differences—For example, it may be difficult to precisely reconstruct a user's path through a site because the clock settings on the servers may be slightly different. This approach does not lend itself to real-time reporting—Logfiles are typically processed as batch jobs during off-hours.
Another prior art technique is Page Tagging. Deficiencies with logfile analysis, along with the desire to perform analytics as an outsourced offering, led to a page tagging approach. Page tagging involves dropping small pieces of code on each page. This code, typically written in JavaScript, runs each time a page is loaded and therefore avoids the problems with caching. With this approach it is easier to add additional information, including details displayed on the page, which will be collected by the reporting server.
Some potential problems with page tagging include the following: Page tagging could be disabled by the client's browser (such as turning off JavaScript); Page tagging only records successful pages, it can not record error conditions; Page tagging reporting will measure page displays or refreshes, it does not capture the underlying event (for example, a page may be redisplayed as a result of hitting the browser back button or the browser refresh); Page tagging only works in environments that support HTML; and Reports could be manipulated by someone simulating the functionality of the JavaScript code.
Web Bug-based systems represent one specialization of prior art page-tagging systems. Its use is sufficiently broad that it merits discussion. A web bug is an invisible object (usually a single pixel gif) on a page. Alternate names include web beacon, tracking bug, pixel tag and clear .gif. When the browser renders the page it performs an HTTP get for the image, thus recording the event to the reporting server.
Web bug-based systems are limited in several ways including the following: Limited information can be passed to a reporting system such as the URL of the page containing the Web Bug and the URL of the Web Bug itself—Since a Web Bug is requested as a standard HTTP request, all typical HTTP request data (e.g., client IP address, client browser type and any cookies for the domain of the Web Bug) will also be available; Since the Web Bug URL is the only mechanism to relay information from the server to the reporting system, embedding multiple data elements in this one string can be difficult to manage and the amount of data is limited by the size of the URL; There is no guarantee that the client browser will ever request the Web Bug URL; Security features and user preferences (such as don't allow 3rd party images or don't load images) can effectively defeat this reporting; Web Bug reporting will measure page displays or refreshes, it does not capture the underlying event (for example, a page may be redisplayed as a result of hitting the browser back button); Reports can be easily manipulated by a process that simply generates HTTP gets based on the form of the URL which is clearly visible in the source of any page; This mechanism is limited in that it can only work on browser-based applications using HTTP as the communication vehicle.
Another method emerged that relies on intercepting or sniffing network traffic (e.g. stream interceptors). It typically includes a device that sits between the client browser and the company server. This device could be trained to identify and capture specific types of events in the traffic stream. A major advantage of such a system is that it resides on the company side and is not impacted by client browser settings. Another advantage is that it does not require modifications to any existing code in order to generate basic reports. Additionally, this method can form the foundation for a more completed screen capture and replay system.
However, some problems with stream interception systems include the following: Without modifying the system (and perhaps adding tracers) it could be difficult to identify the events of interest; As with page tagging, only successful events will be recorded; The level of information captured is limited to that included on the displayed pages; Recording the secure portions of the site will require special machine reconfiguration or separate encryption/decryption hardware; If the entire stream (or a large portion of it) is captured, the amount of information could quickly grow to an unmanageable size.
All of the above systems focus on the user experience side of reporting. That is, they rely on analysis of HTTP requests and responses to count page displays and visits at an aggregate level or to perform path analysis at a more granular level. None of these systems provide measurements on the health of the underlying system. This has been the domain of system monitoring tools.
Traditionally, system monitoring tools report on key parameters such as CPU utilization, network utilization, memory usage, page faults and disk transfers. All of these provide some measure of the responsiveness of a system but impact of these variables on a user experience is hard to quantify. Many of these tools require substantial resources so in the process of collecting observations on system performance the tool is itself changing system performance.
The most general reporting mechanism is placement of print statements in specific areas of a system. However, its effectiveness is limited since creating the data is only the first step in the process. The data must be collected and aggregated. This could be challenging in an environment in which applications may be spread across several machines.
As shown above, the current approaches have several limitations including the following: Reliance on a specific technologies such as HTTP or proprietary system monitoring protocols; Limitation on the amount and type of information collected; Vulnerabilities in generating and managing data which leads to potential inaccuracies; Difficulty in collecting the data and mining through it to pull out meaningful information; Lack of uniform methods for handling system health and customer experience information
Some prior art references that relate to the above and other current approaches include U.S. Pat. No. 6,856,983, U.S. Pat. No. 7,099,932, U.S. Pat. No. 6,754,181, U.S. Patent Pub. No. 20040015579, U.S. Patent Pub. No. 20040261116, U.S. Patent Pub. No. 20060179064, U.S. Patent Pub. No. 20060142011, and U.S. Patent Pub. No. 20020049608. However, a need remains for a system and method that can overcome the above-identified limitations and problems, among others.