Effective management of event messages is the cornerstone of high quality information technology (IT) service delivery.
Intense competition among IT service providers to demonstrate high quality service management (e.g., low response times, high availability) has led to very aggressive goals for IT-based services. Realizing these goals requires proactive management processes which provide early detection and isolation of IT event messages signaling service delivery problems. As IT service providers are forced by an extremely competitive market to aggressively control cost of service delivery, the automation of these processes becomes increasingly critical. This capability of automated event detection, problem isolation and resolution is a key aspect of an autonomic computing strategy. This is especially the case for complex IT systems comprising distributed, heterogeneous components.
As is known, “autonomic computing” is a comprehensive and holistic approach to self-managed computing systems with a minimum of human interference, e.g., see P. Horn, “Autonomic Computing: IBM's Perspective on the State of Information Technology,” IBM Research, October 2001, the disclosure of which is incorporated by reference herein.
Real-time, high-performance event management systems universally require transformation of the incoming event data to a common format prior to application of event processing logic. This transformation from unique formats to a common format is controlled by parsing rules.
Creation of parsing rules that transform event data into a unified format has traditionally been a very time consuming exercise that requires technology domain experts to develop unique parsing rules for all event messages. In the past, parsing has often been addressed manually by creating ad-hoc parsers directed to event logs of specific technologies and applications.
Several problems exist with such an approach. First, the manual approach involves a time-consuming, error prone process. Second, the manual approach requires a user to have both: (1) domain knowledge in understanding data formats; and (2) programming knowledge in translating domain knowledge into event data parsing rules.
In addition, the manual approach has been rendered ineffective by significant challenges emerging from the present day IT environment.
A critical challenge in the deployment of autonomic event management methods and systems is the need for the solution to address very large numbers of events in real-time, support a broadening spectrum of event message formats, and recognize and process individually thousands of unique event messages.
The most onerous issue is event volume. Many IT operations centers report volumes of one million or more events per day. More IT users are reaching that plateau each month. Unfortunately, users lack a process for collection, parsing and extraction of pertinent event data which effectively addresses this scaling issue.
The IT industry has introduced a broad range of proprietary and standardized event protocols, log file formats, and (even within a single protocol) syntax. The variety of formats assumed by event messages adds considerable complexity to the event data environment of the user. Viewed from a practical data management perspective, the variety in event formats will add significantly to the effort the customer will be required to invest in development of data parsing rules.
Further, the torrent of events generated across the IT environment of the user is composed of thousands of unique event types, each containing potentially important management information and each, potentially, requiring unique parsing rules.
To summarize, many users contend with more than a million event messages per day. Their event streams contain a multitude of differing data protocols and formats. The individual events within these event streams represent thousands of unique event types. Traditional labor intensive approaches to the parsing analysis of this mass of event data are inadequate.
Thus, a need exists for parsing rule creation techniques that are supported with automated facilities such that the above-mentioned and other limitations may be overcome.