e organized in some fashion for classification. The preprocessing will help determine the format the data are put into, which is usually some canonical format or could be a structured database. Once the data are formatted, they are broken down further into classifications.
These classifications can depend on the analysis schemes being used. For example, if rule-based detection is being used, the classification will involve rules and pattern descriptors. If anomaly detection is used, you will usually have a statistical profile based on different algorithms in which the user behavior is baselined over time and any behavior that falls outside of that classification is flagged as an anomaly. Both pattern matching and anomaly detection are covered later in this chapter.
Upon completion of the classification process, the data is concatenated and put into a defined version or detection template of some object by replacing variables with values. These detection templates populate the knowledgebase which are stored in the core analysis engine:
Once the prepossessing is completed, the analysis stage begins. The data record is compared to the knowledge base, and the data record will either be logged as an intrusion event or it will be dropped. Then the next data record is analyzed.
The next phase, response, is one of the differentiating factors between IDS and IPS. With IDS, you typically have limited prevention abilities—you are getting the information passively after the fact, so you will have an alert after the fact. Once information has been logged as an intrusion, a response can be initiated. With IPS, the sensor is inline and it can provide real-time prevention through an automated response. This is the essential difference between reactive security and proactive security.
Either way, the response is specific to the nature of the intrusion or the different analysis schemes used. The response can be set to be automatically performed, or it can be done manually after someone has manually analyzed the situation. For example, Network Flight Recorder (a commercial IDS) offers a feature that can send aTCPRST packet and kill a session.
The final phase is the refinement stage. This is where the fine-tuning of the IDS or IPS system can be done, based on previous usage and detected intrusions. This gives the security professional a chance to reduce false-positive levels and to have a more accurate security tool. This is a very critical stage for getting the most from your IDS or IPS system. The system must be fine-tuned for your environment to get any real value from it. There are tools, like Cisco Threat Response (CTR), that will help with the refining stage by actually making sure that an alert is valid by checking whether you are vulnerable to that attack or not.
Rule-based detection, also referred to as signature detection, pattern matching and misuse detection, is the first scheme that was used in early intrusion-detection systems. Rule-based detection uses pattern matching to detect known attack patterns.
Let’s look at how the four phases of the analysis process are applied in a rule-based detection system:
The pattern descriptors are typically either content-based signatures, which examine the payload and header of a packet, or context-based signatures that evaluate only the packet headers to identify an alert. Note that pattern descriptors can be atomic (single) or composite (multiple) descriptors. An atomic descriptor requires only one packet to be inspected to identify an alert, while a composite descriptor requires multiple packets to be inspected to identify an alert. The pattern descriptors are then put into a knowledge base that contains the criteria for analysis.
An anomaly is something that is different from the norm or that cannot be easily classified. Anomaly detection, also referred to as profile-based detection, creates a profile system that flags any events that strays from a normal pattern and passes this information on to output routines.
One key distinction between anomaly detection and other analysis schemes is