A Survey of Physics-Based Attack Detection in Cyber-Physical Systems <s> Towards Better Evaluation Metrics <s> In this paper we survey a number of methods for the detection of abrupt changes (such as failures) in stochastic dynamical systems. We concentrate on the class of linear systems, but the basic concepts, if not the detailed analyses, carry over to other classes of systems. The methods surveyed range from the design of specific failure-sensitive filters, to the use of statistical tests on filter innovations, to the development of jump process formulations. Tradeoffs in complexity vs performance are discussed. <s> BIB001 </s> A Survey of Physics-Based Attack Detection in Cyber-Physical Systems <s> Towards Better Evaluation Metrics <s> A relatively new trend in Critical Infrastructures (e.g., power plants, nuclear plants, energy grids, etc.) is the massive migration from the classic model of isolated systems, to a system-of-systems model, where these infrastructures are intensifying their interconnections through Information and Communications Technology (ICT) means. The ICT core of these industrial installations is known as Supervisory Control And Data Acquisition Systems (SCADA). Traditional ICT security countermeasures (e.g., classic firewalls, anti-viruses and IDSs) fail in providing a complete protection to these systems since their needs are different from those of traditional ICT. This paper presents an innovative approach to Intrusion Detection in SCADA systems based on the concept of Critical State Analysis and State Proximity. The theoretical framework is supported by tests conducted with an Intrusion Detection System prototype implementing the proposed detection approach. <s> BIB002 </s> A Survey of Physics-Based Attack Detection in Cyber-Physical Systems <s> Towards Better Evaluation Metrics <s> Advanced false data injection attack in targeted malware intrusion is becoming an emerging severe threat to the Supervisory Control And Data Acquisition SCADA system. Several intrusion detection schemes have been proposed previously [1, 2]. However, designing an effective real-time detection system for a resource-constraint device is still an open problem for the research community. In this paper, we propose a new relation-graph-based detection scheme to defeat false data injection attacks at the SCADA system, even when injected data may seemly fall within a valid/normal range. To balance effectiveness and efficiency, we design a novel detection model, alternation vectors with state relation graph. Furthermore, we propose a new inference algorithm to infer the injection points, i.e., the attack origin, in the system. We evaluate SRID with a real-world power plant simulator. The experiment results show that SRID can detect various false data injection attacks with a low false positive rate at 0.0125%. Meanwhile, SRID can dramatically reduce the search space of attack origins and accurately locate most of attack origins. <s> BIB003 </s> A Survey of Physics-Based Attack Detection in Cyber-Physical Systems <s> Towards Better Evaluation Metrics <s> Off-the-shelf intrusion detection systems prove an ill fit for protecting industrial control systems, as they do not take their process semantics into account. Specifically, current systems fail to detect recent process control attacks that manifest as unauthorized changes to the configuration of a plant's programmable logic controllers (PLCs). In this work we present a detector that continuously tracks updates to corresponding process variables to then derive variable-specific prediction models as the basis for assessing future activity. Taking a specification-agnostic approach, we passively monitor plant activity by extracting variable updates from the devices' network communication. We evaluate the capabilities of our detection approach with traffic recorded at two operational water treatment plants serving a total of about one million people in two urban areas. We show that the proposed approach can detect direct attacks on process control, and we further explore its potential to identify more sophisticated indirect attacks on field device measurements as well. <s> BIB004
One of the differences between detecting attacks in control systems when compared to detecting attacks in general IT systems is that researchers do not have readily available data from attacks in the wild. Even if we test our algorithms on the few known examples (like Stuxnet), they are domain specific, and it is not clear they will give insights into the evaluation other than to show that we can detect Stuxnet (which can be easily detected ex post). For that reason, researchers need to generate novel attacks in their papers, and the question we would like to address in this section is how to create attacks that are general enough to be applicable across multiple industrial control domains but that will also allow us to define an evaluation metric that is fair (and that is not biased to detect the specific attacks from the researchers). To motivate the need of a new metric, we now discuss the challenges and limitations of previously used metrics. Measuring the True Positive Rate is Misleading. To obtain the true positive rate of a detection algorithm, we need to generate an attack that will be detected. It is not clear if there can be a principled way of justifying the generation of an attack that will be detected as this implies our attacker is not adaptive and will not attempt to evade our detection algorithms. Publications using the true positive rate BIB002 BIB003 generate their attacks as random signals (e.g., a sensor reporting random values instead of reporting the true state of the physical system). This type of non-strategic random failure is precisely what the fault-detection community has been working on for over 40 years BIB001 ; with those attacks, we are not advancing the state of the art on attack-detection, but rather reinforcing the fact that fault-detection works when sensor or control signals fail in a non-malicious way. Model Fidelity is an Incomplete Metric. One of the first papers to articulate why measuring in a meaningful way the true positive rate for control systems is hard is the work of Hadziosmanovic et al. BIB004 . Having summarized the reasons why measuring the true positive rate can be misleading, they focus instead on understanding how accurately their AR system models the real-world system and identifying the cases where it fails. They are more interested in understanding the model fidelity than in specific true-/false-alarm rates. However, understanding the model fidelity is implicitly looking at the potential of false alarms, because deviations between predictions and observations during normal operations are indicators of false alarms. While this is a good approach for the exploratory data analysis done in the paper, it might be misunderstood or applied incorrectly by future researchers. The anomaly detection rule of "never raise an alert" will have zero false alarms-i.e., perfect fidelity-but it never detects any attack.