Secure Sensor Design Against Undetected Infiltration: Minimum Impact-Minimum Damage

We propose a new defense mechanism against undetected infiltration into controllers in cyber-physical systems. To this end, we cautiously design the outputs of the sensors that monitor the state of the system. Different from the defense mechanisms that seek to detect infiltration, the proposed approach seeks to minimize the damage of possible attacks before they have been detected. Controller of a cyber-physical system could have been infiltrated into by an undetected attacker at any time of the operation. Disregarding such a possibility and disclosing system's state without caution benefits the attacker in his/her malicious objective. Therefore, secure sensor design can improve the security of cyber-physical systems further when incorporated along with other defense mechanisms. We, specifically, consider a controlled Gauss-Markov process, where the controller could have been infiltrated into at any time within the system's operation. In the sense of game-theoretic hierarchical equilibrium, we provide a semi-definite programming based algorithm to compute the optimal linear secure sensor outputs and analyze the performance for various scenarios numerically.

Secure Sensor Design Against Undetected Infiltration: Minimum Impact-Minimum Damage Muhammed

O. Sayin and Tamer Başar, Life Fellow, IEEE
Abstract-We propose a new defense mechanism against undetected infiltration into controllers in cyber-physical systems. To this end, we cautiously design the outputs of the sensors that monitor the state of the system. Different from the defense mechanisms that seek to detect infiltration, the proposed approach seeks to minimize the damage of possible attacks before they have been detected. Controller of a cyber-physical system could have been infiltrated into by an undetected attacker at any time of the operation. Disregarding such a possibility and disclosing system's state without caution benefits the attacker in his/her malicious objective. Therefore, secure sensor design can improve the security of cyber-physical systems further when incorporated along with other defense mechanisms. We, specifically, consider a controlled Gauss-Markov process, where the controller could have been infiltrated into at any time within the system's operation. In the sense of game-theoretic hierarchical equilibrium, we provide a semi-definite programming based algorithm to compute the optimal linear secure sensor outputs and analyze the performance for various scenarios numerically.

I. INTRODUCTION
C YBER-PHYSICAL systems, incorporating both physical and cyber parts, e.g., process control systems, robotics, smart grid, and autonomous vehicles, have resulted in new and distinct challenges for control system design, e.g., specifically, security-related challenges due to cyber attacks [1], [2]. Different from external random disturbances, cyber attacks can be very target specific and persistent by attacking stealthily for long term benefits. Recently in 2014, cyber-physical, e.g., process control, systems in energy and pharmaceutical industries have been infiltrated into by Dragonfly Malware, which intervened in the systems over a long period of time without being detected [3]. However, isolation from the cyber networks are also not perfectly sufficient any more. In 2010, StuxNet Worm caused substantial damage on certain "isolated" supervisory control and data acquisition (SCADA) systems [4]. Therefore, developing novel formal security mechanisms against advanced and persistent threats that can cause substantial damage without being detected plays a vital role in the security of cyber-physical systems.
Beyond exploiting uncertainties in the systems, e.g., due to random disturbances, advanced and persistent attackers can also seek to deceive the detection mechanisms by manipulating monitoring signals used by the detectors. For example, in [5], the authors have introduced false data injection attacks, where the attackers can inject data into the sensor outputs, in the context of state estimation, and characterized undetectable attacks. Based on the deceptive attacker model in [5], the existing studies mainly focus on characterizing the vulnerabilities of control systems against such undetectable attacks (which can deceive the detectors) and designing counter measures to be able to detect them.
In discrete-time linear-quadratic-Gaussian (LQG) systems, in [6], the authors have introduced replay attacks and proposed a defense mechanism against such attacks. Replay attacks take place during the steady state of the system, and the attacker records and replays the sensor outputs so that the detectors using those signals cannot detect any anomalies. Note that the signals are expected to be similar at steady state. As a defense mechanism, the authors have proposed to inject an independent signal into the control input to detect such attacks in the expense of degraded control performance. An optimal defense strategy with respect to the probability of detection has been formulated in [7]. Again in LQG systems, in [8], the same set of authors have introduced integrity attacks, where the attacker can inject data into sensor outputs and control inputs, and characterized the reachable set that the attacker can drive the system to without being detected (via innovation based failure detectors [9]). They have also provided necessary conditions for unbounded reachable set, i.e., conditions where the attacker can destabilize the system. Within deterministic control scenarios, in [10], the authors have analyzed zero-dynamics attacks that do not depend on online information, i.e., open-loop stealthy attacks, and provided an algorithm to reveal all such attacks by adding new measurements, similar to [6]. Again within deterministic control scenarios, in [11], the authors have provided a unified framework for false data injection and replay attacks, and formulated the limitations of monitoring-based detection mechanisms. In [12], the authors have analyzed tolerance of control systems to false data injection attacks on a subset of sensors in the deterministic settings and proposed decoding schemes to estimate the state via corrupted measurements. They have also introduced a secure control loop that can enhance the decoding performance and, correspondingly, the resilience of the system.
The attackers can also have adversarial control objectives. In [13], [14], the authors have analyzed such attacks, where the attacker both seeks to be undetected and drive the state of the system according to his/her adversarial goal by manipulating both sensor outputs and control inputs together. Recently, [15] has analyzed optimal attack strategies to maximize the quadratic cost of a system with linear Gaussian dynamics without being detected, where the stealthiness is measured in terms of the Kullback-Leibler distance between the realized and the desired state behaviors. In the optimal attack, the attacker injects independent Gaussian noise having certain variance into the control input. In another recent study [16], the authors have proposed linear encoding schemes for sensor outputs of an LQG system in order to enhance detectability of false data injection attacks while the coding matrix is assumed to be unknown by the attackers, which can be mitigated via time-varying coding matrices. In spite of these extensive studies, we still have significant and yet unexplored problems about how to enhance security against undetected attacks, i.e., impact of attacks before detection.
In this paper, we address primarily the following two questions: "If we have already designed the sensor outputs, to what extent would we have secured the system against undetected infiltration into the controllers?" Further, "what would be the best linear sensor outputs that can lead to both minimum impact and minimum damage on the ordinary operations of the systems?" The damage due to inconspicuous (undetectable, or difficult to detect) attacks with long term control objectives is our main concern in this paper. We can classify such attacks as "advanced and persistent threats", since they are advanced by being very target specific and persistent by being inconspicuous. Therefore, we propose to be cautious while disclosing the state information to the controller due to the possibility of undetected infiltration. However, as a system designer, we should not take precautions as if the cyber part of the system is compromised due to just a possibility, since that would impact the ordinary operations of the system substantially. Combining these seemingly opposing goals all together, we seek to design sensor outputs cautiously with minimum impact and minimum damage on the system's operations.
To obtain explicit results, we specifically consider systems with linear Gaussian dynamics and quadratic control objectives, which have various applications in industry [15] from manufacturing processes to aerospace control. We consider the possibility of infiltration into the controller of the system by various attackers at any stage within the time horizon. The attackers have long term control objectives and attack stealthily. To this end, they include soft constraints on the energy of the state and the deviation of the constructed control inputs from system-desired ones, which would have been constructed if the attacker rather had a friendly objective. Such constraints are, especially, against the detection mechanisms that take actions when such deviations exceed certain thresholds.
We note that the sensors could also be infiltrated into by the attackers, which can cancel the proposed approach via a shortcut to the state if the sensors have access to the state realizations. To mitigate that, we consider the scenarios where the sensors do not have access to the actual state realizations. All the sensor strategies, defining the relation between the state and the sensor output, are selected beforehand to minimize the expected loss and fixed (can be time-variant, yet not controlled) during the operation. Therefore, we can design the sensor outputs off-line, i.e., in advance, which leads to a hierarchical structure between the sensor and the controller of the system (even when he/she is adversarial).
Due to the stochastic nature of the problem, i.e., due to the state noise, any open-loop control strategy of an attacker could not drive the system in his/her desired path effectively [17]. Therefore, regardless of whether the controller has an adversarial objective or not, he/she needs to construct a closedloop control input based on the designed sensor outputs while knowing the relationship between the sensor output and the state. This implies that the interaction between the sensor and the controller of the system could be analyzed as a gametheoretic hierarchical equilibrium, where the sensor leads the game by announcing his/her strategies beforehand. Therefore, while designing the sensor outputs, we should consider both adversarial and friendly control outputs and the possibility of infiltration over the time horizon.
Particularly, we seek to formulate the best linear sensor strategies for controlled Gauss-Markov processes. We consider a different time scale for the infiltration into the system and we formulate the optimal sensor strategies in a Bayesian setting based on given infiltration statistics. Note that since the sensor strategies are set to be linear, the problem is an LQG control problem and correspondingly the optimal, friendly, control policy is linear in the conditional estimate of the state given all the sensor outputs. We first compute the best control inputs of friendly and adversarial controllers for any given linear sensor strategies and any time when they become in charge of the controller. Corresponding to these optimal strategies, we provide a semi-definite programming (SDP) based algorithm to design the optimal (memoryless) linear secure sensor strategies. Furthermore, we analyze the sensitivity of the design against inaccurate perception of the underlying statistics numerically. We note that in [18], we have introduced secure sensor design against advanced and persistent threats in cyber-physical systems, but have not completely solved the problem. Here, we consider different attack models that have soft constraints on the energy of the state and the deviation of the constructed control inputs from the system-desired ones, and we consider more comprehensive scenarios, where the controller can be infiltrated into or an adversarial infiltration could be detected at any time within the time horizon.
In the design of secure cyber-physical systems, each additional security layer leads to new monetary and computational costs [19]. In particular, there are fundamental trade-offs in terms of investment on the security mechanisms and the value of the protected assets or securing the system and maintaining the ordinary operations. In order to offer better trade-offs, we aim to propose a defense mechanism that does not require any additional online computational load with minimum impact and minimum damage on the ordinary operations of the system. To summarize, we can list the main contributions of this paper as follows: • We introduce secure sensor design against various inconspicuous attackers with control objectives, which can infiltrate into the controller of a cyber-physical system at any time during the operation.
• Given any linear sensor strategies and the underlying linear quadratic Gaussian dynamics, we compute the optimal attack strategies depending on the infiltration time. • We provide a practical algorithm to compute the optimal linear memoryless sensor strategies in the sense of gametheoretic hierarchical equilibrium. • We also analyze sensitivity of the proposed algorithm against inaccurate perception of attack statistics. The paper is organized as follows: In Section II, we provide the secure sensor design framework. In Section III, we formulate the associated multi-stage Bayesian Stackelberg game. In Section IV, we characterize the optimal controller response strategies for given sensor strategies. We compute the corresponding optimal sensor strategies in Section V. In Section VI, we examine the performance of the proposed scheme under various scenarios numerically. We conclude the paper in Section VII with several remarks and possible research directions. Notations: For an ordered set of parameters, e.g., x 1 , · · · , x n , we define x [k,l] := x k , · · · , x l , where 1 ≤ k ≤ l ≤ n. N(0, .) denotes the multivariate Gaussian distribution with zero mean and designated covariance. We denote random variables by bold lower case letters, e.g., x x x. For a random variable x x x,x xx is another random variable corresponding to its posterior belief conditioned on certain other random variables that will be apparent from the context. For a vector x and a matrix A, x and A denote their transposes, and x denotes the Euclidean (L 2 ) norm of the vector x. For a matrix A, tr{A} denotes its trace. We denote the identity and zero matrices with the associated dimensions by I and O, respectively, while 1 (or 0) denotes a vector whose entries are all 1 (or 0). For positive semi-definite matrices A and B, A B means that A − B is also a positive semi-definite matrix. A ⊗ B denotes the Kronecker product of the matrices A and B.

II. PROBLEM FORMULATION
Consider a controlled stochastic system described by the following equations: for k = 1, 2, . . . , n, where 1 A ∈ R m×m , B ∈ R m×r , D ∈ R r×r , and x x x k ∼ N(0, Σ k ), k = 1, . . . , n. The additive state and control input noise sequences {v v v k } and {w w w k } are white Gaussian vector processes, i.e., v v v k ∼ N(0, Σ v ) and w w w k ∼ N(0, Σ w ); and are independent of the initial state x x x 1 and of each other. We assume that the matrix A is non-singular, and the auto-covariance matrices Σ 1 and Σ v are positive definite while Σ w is positive semi-definite. The closed loop control vector u u u k ∈ R r is given by u u u k = γ k (s s s [1,k]   where γ k (·) can be any Borel measurable function from R mk to R r . The sensor output s s s k ∈ R m is given by where η k (·) can be any linear function from R m to R m . And y y y k ∈ R r , given by (2), denotes the noisy observation of the control input u u u k . We have two separate agents: Sensor (S) and Controller (C). At each stage k = 1, . . . , n, the agents construct s s s k and u u u k according to their own objectives. In particular, S chooses η k (·) from the strategy space ϒ, which, for each k, is the set of all linear functions from R m to R m , i.e., η k ∈ ϒ and s s s k = η k (x x x k ). This implies that for each η k ∈ ϒ, there exists a matrix L k ∈ R m×m such that almost surely on R m . C chooses γ k (·) from the strategy space Γ k , which is the set of all Borel measurable functions from R mk to R r , i.e., γ k ∈ Γ k and u u u k = γ k (s s s [1,k] ). Sensor and Controller Objective. As in a stochastic control scenario [17], S and C can have a common finite horizon 2 quadratic cost function: almost surely. Correspondingly, S could disclose the state x x x k directly so that C could drive the state in their commonly desired path [17], [20]. However, in a cyber physical system, the system is vulnerable to adversarial infiltration attacks that seek to drive the state of the system away from the system's desired target as seen in Fig. 1. We call such attacks "advanced persistent threats", which are advanced by being very target specific, i.e., the attacker knows the underlying state recursion, and persistent by avoiding infiltration detection. Therefore, S, i.e., the sensor designer, should anticipate the likelihood of adversarial infiltration into C, i.e., the possibility that C can be an adversary, and select η k ∈ ϒ accordingly. Remark 1. We note that the sensor output s s s k only depends on the current state x x x k , i.e., η k is memoryless. Otherwise, S would need to have access to the state information in order to store and to be able to use them in the future stages. However, similar to C, S can also be infiltrated into by the attackers, which would neutralize S's effort to design sensor outputs strategically via a shortcut to the state information. Remark 2. In control system design, sensors are designed and implemented in advance, and system engineers design the controllers knowing the relation between the sensor output and the underlying state. Correspondingly, an attacker that has infiltrated into the system can be aware of how the sensor outputs have been constructed and can design his/her attack accordingly. Therefore, there exists a hierarchy between S and C such that C can have access to S's strategies. Infiltration Detection. We note that if the control inputs could have been monitored perfectly, then any deviation of the control input from the system-desired one could have been detected instantly since both sensor outputs and control inputs will be accessible. Therefore, in this paper, we address the scenarios where the control input cannot be monitored perfectly. As an example of such scenarios, the infiltration detection mechanism can have access to noisy control input observation y y y k ∈ R r and the state x x x k as seen in Fig. 1. In the scope of this work, we will not consider the details of how the infiltration detector operates except that the advanced and persistent attackers are aware of the presence of an infiltration detector that can have access to the state (correspondingly the sensor outputs) and noisy versions of the control inputs.
Inconspicuous Infiltration Attacks into C. As seen in Fig.  1, C is under infiltration attacks by the (advanced) attackers A 1 , . . . , A t over the time horizon k = 1, . . . , n and such attacks may be successful or not in infiltrating into C. As mentioned earlier, being advance refers to being target specific with knowledge about underlying system dynamics while also avoiding detection mechanisms by attacking inconspicuously. Therefore, C can be a friend or an adversary within the time horizon while S may not know C's type surely until an infiltration detection takes place, which may be less likely due to inconspicuousness of the attacks. C observes s s s [1,k] , knows S's strategies η [1,k] due to a hierarchy between the agents, and, via a strategy γ k ∈ Γ k , can construct a closed-loop control input u u u k , yet the state and the control input can be monitored by the infiltration detector. Therefore, as an attack model, we consider the situation where the attacker A i , i = 1, . . . ,t, selects γ A i ,k ∈ Γ k , k = κ, . . . , n, where κ denotes the infiltration time, to minimize the cost function: where λ i ≥ 0, "·" as an argument of the cost function (7) refers to the C's strategies γ 1 , . . . , γ κ−1 , which are not selected Infiltration by Infiltration by into the system already under attack Exfiltration of from the system e.g., due to detection Fig. 2: Possible transitions among the agents F, A 1 , and A 2 to be in charge of C within the time horizon. by A i ; and z i ∈ R m is the desired state that the adversary seeks to drive the system to. The matrices 4 Q A i ∈ R m×m are positive semi-definite, and R A i ∈ R r×r are positive definite. Here, x x x A i ,k denotes the state driven by the adversarial control input u u u A i ,k ; while u u u F,k denotes the control input that would have been constructed if C was a friend.
Particularly, the last two terms in (7) are soft constraints to avoid infiltration detection by being close to the expected behavior of the system, e.g., small energy of the state x x x A i ,k+1 , and small deviations of u u u A i ,k from u u u F,k . Note that λ i ≥ 0 can also be zero. We also note that deviation of the state x x x A i ,k+1 from the system-desired x x x F,k+1 is equivalent to Furthermore, deviation of the observed control input 5 y y y A i ,k from the system desired y y y F,k is equivalent to Therefore, via R A i ∈ R r×r , the attacker can take precautions against the thresholding-based detection mechanisms that check the deviation of the state or the control input from the system-desired ones. We also note that u u u F would have been constructed by C if there were no infiltration. Correspondingly, the attacker can construct u u u A i ,k by adding another signal on top of u u u F,k such that u u u A i ,k = u u u F,k + δu u u A i ,k as in [13], [14]. Therefore, the last term in (7) also corresponds to a soft energy constraint on δu u u A i ,k .
In the following section, we provide a game theoretical formulation to analyze the interaction between the agents.

III. HIERARCHICAL EQUILIBRIUM FORMULATION
Cyber-physical systems are vulnerable to infiltration attacks. Within the time horizon various attackers can infiltrate into the system as well as defense mechanism can detect infiltration and take appropriate actions accordingly. As an example, Fig. 2 demonstrates possible transitions among the agents, F, A 1 , and A 2 , to be in charge of C. When an attacker, e.g., A 1 , infiltrates into the system, A 1 becomes in charge of C and can construct the control input according to his/her adversarial objective. Furthermore, when the attackers are not communicating with each other, an attacker, e.g., A 2 , may not know whether the system is already under attack or not and correspondingly may infiltrate into the system while A 1 is still in charge of C. Furthermore, there can be infiltration detection due to the active defense mechanisms monitoring the control and sensor inputs as seen in Fig. 1. Remark 3. We point out that when infiltration has been detected, the system could prefer to use different sensor outputs since the uncertainty about C has been removed. As an example, if the attacker cannot be forced to exfiltrate from the system immediately, the system can prefer to use sensor outputs designed specifically against the adversarial objectives of the attacker. Or if F becomes in charge of C after detection, as long as the system ensures F is in charge, direct state disclosure can be preferred. Therefore, we consider scenarios where the system switches his/her operation mode and uses different sensor outputs, once an infiltration has been detected.
In order to model uncertainty of the transitions between the agents to be in charge of C explicitly, we consider a jump process {θ θ θ j ∈ Θ}, where Θ := {F, A 1 , . . . , A t , T} and transitions can occur on a different time scale called transition time, e.g., κ ∈ {δ , 2δ , . . . , Nδ }, where δ ∈ Z and N := n/δ . The state T denotes the switch to a different operation mode, e.g., due to infiltration detection. Therefore, when the process jumps to the state T, the horizon practically terminates for S. Correspondingly, for a given sequence of the jump process: θ 1 , . . . , θ N , if there is a jump to T, we let N T denote the index of the last state before the jump, and otherwise N T = N +1, and leth(θ [1,N] ) be an ordered set including the non-terminating transition times, and 1 and min{N T δ , n + 1}. As an illustrative example, let n = 100 and δ = 30 and consider the situation where F is in charge of C initially while A 1 infiltrates into C at k = 60 and becomes in charge until detection at k = 90. Therefore, we haveh(θ [1,N] ) = {1, 60, 90}, which implies that while θ 1 = θ 2 = F is in charge during the interval [1, 60), θ 3 = A 1 becomes in charge during the interval [60, 90) and the system switches its mode at k = 90.
The underlying state recursion is common knowledge to both S and C (even if C can be an adversary). The type of C and, if C is an adversary, his/her objective are not known by S. However, S knows the statistics of the jump process {θ θ θ j }. As also noted in Remark 1, there is also a hierarchy [21] between the agents in the announcement of the strategies such that S leads the game by announcing and sticking to his/her strategies in advance, i.e., C knows η [1,n] in advance. Therefore, we can model such a scheme as a multi-stage Bayesian Stackelberg game, in which S is the leader. Remark 4. When the infiltration detector has access to the sensor outputs s s s k but not the states x x x k , even though the attacker can also inject false data into the sensor outputs in order to avoid detection as in integrity attacks, e.g., [8], [13], [14], due to the "stochastic" state recursion (1), the attacker still needs the actual sensor outputs, which are designed by the system designer in advance. Therefore, secure sensor design framework can also play a crucial role for the security of the systems against integrity attacks.
In the following sections, we analyze the equilibrium achieving strategies.

A. Optimal Agent-F Reaction
Based on Remark 3, secure sensor designer is only interested in the reaction of F when he/she is in charge of C starting from time k = 1. We note that given linear memoryless S strategies, the problem is an LQG control problem for F [17]. In the following, for completeness, we will derive the corresponding optimal control inputs.
In order to facilitate the subsequent analysis, we can rewrite the state equations (1)-(4) and the cost function (6) without altering the optimization problem. Particularly, after completing the squares [17], [23], the friendly objective (6) is equivalent to: where 6 The sequence {Q F,k } is defined through the following discretetime Riccati equation: Then, through a change of variables [23], friendly type C's objective (11) can be written as subject to (12)-(13) and for k = 1, · · · , n, Note that, now, the process {x x x o k } is independent of how the control inputs u u u F,k (and u u u o F,k ) are constructed while the sensor outputs by depending on the current state x x x F,k also depend on the previous control inputs.
Applying the Principle of Optimality to (14), in view of (5), leads to the result that the last stage optimal transformed control input is given by However, by (15a), we have where the underbraced term is σ -s s s [1,k−1] measurable, for k = 1, . . . , n. Therefore, (16) is equivalent to and does not depend on previous control inputs. Therefore, by induction, we can conclude that the problem entails classical information and for k = 1, . . . , n, the optimal transformed control input is given by almost everywhere on R r , for k = 1, . . . , n, which would also imply the uniqueness of the best F reactions and singleton reaction set Π F,1 . Then, by (15b) and (19), the optimal control inputs are given by wherex xx o k := E{x x x o k |s s s [1,k] }, and equivalently:

B. Optimal Adversarial Reaction (Attack)
Here, we compute the optimal attack strategies with control objective (7). Different from Subsection IV-A, now, we also consider the scenarios where A i can infiltrate into C at k > 1. That would imply that an attacker A j can infiltrate into C that has already been infiltrated into by another attacker A i . Furthermore, the attacker may not know the underlying statistics of the jump process corresponding to the transitions among the agents that can be in charge of C. Correspondingly, the attacker may not know how the state has been driven until he/she has infiltrated into. However, as also noted in Remark 4, consecutive succesful infiltration by different attackers can be considered as a rare event relative to the infiltration of the attacker into C while F was in charge. Therefore, as an attack model, we consider the scenarios where the attackers can assume that F was in charge of C before they have infiltrated into.
Next, we aim to rewrite the state equations and the cost functions as in (11) and (14) for the minimization of the adversarial objectives (7). For k = κ, . . . , n, let Then, instead of (1), consider the following recursion: which can be written in compact form as Correspondingly, the objective (7) can be rewritten as whereQ We point out the resemblance between (24) and (6). As in (11), we can rewrite the cost function (24) without altering the optimization problem. After completing the squares, the adversarial objective (24) is equivalent to: where [κ,n] , and instead depends on γ F, [1,κ−1] . The sequence {Q A i ,k } is defined through the following discrete-time Riccati equation: We emphasize that K A i ,k , ∆ A i ,k , andQ A i ,k do not depend on the infiltration time κ. And corresponding to (14), the adversarial objective (26) can be written as subject to (27)-(28) and for k = 1, . . . , n, Note that in (29), G A i ,κ does not depend on the adversary's optimization arguments even though it depends on u u u F due tō Σ A i ,κ in (27c). However, F does not consider the impact of u u u F on G A i ,κ while selecting γ F, [1,n] since (29) is the cost function of A i , and not of F. Note also that if F is in charge of C before A i has infiltrated into C, for k = 1, . . . , κ − 1, we have Even though the process {x xx o A i ,k } is independent of how the control inputs δu u u A i ,k (and δu u u o A i ,k ) are constructed, the sensor outputs s s s k , k = κ, . . . , n, depend on the taken actions, i.e., u u u A i ,k . Similar to (16), the Principle of Optimality yields Then, irrespective of (30a) and (23), we have where x x x o k evolves according to (15a) and the underbraced terms are σ -s s s [1,k−1] measurable. Therefore, (32) is equivalent to and does not depend on the previous control inputs u u u A i ,[κ,n−1] . Remark 7. We emphasize that if A i does not know which agent was in charge of C before κ, then the relation between u u u [1,κ−1] and the corresponding sensor outputs s s s [1,κ−1] would not be known by A i and correspondingly (34) would not be equivalent of (32). Furthermore, the equivalence does also not hold if another attacker had already infiltrated when A i infiltrates since A i has assumed that F was in charge before the infiltration. Note also that S is only interested in the scenarios where θ θ θ [1,N] ∈ Ω, i.e., there is no successful consecutive infiltration by different attackers. A detailed analysis of the consecutive successful infiltration and attacks with partial information is left as future work. By induction, we can conclude that the optimal transformed control inputs of A i are given by for k = κ, . . . , n, almost everywhere on R r . This implies the uniqueness of the best A i reactions and singleton reaction set which can also be written as Next, we seek to compute E{x xx o A i ,k |s s s [1,k] } in (38) in terms of xx x o . To this end, let us take a closer look at (30a): where we introducex xx k , which is given by Then, we have if l ≥ k; and the second case, i.e., l < k, follows since E{x x x o l |s s s [1,l] } is σ -s s s [1,k] measurable if l < k. Therefore, (41) can be written as E{u u u F |s s s [1,k] where the middle block is the kth block column from the right. Hence, we can rewrite (40) as (44) Therefore, (44) and (37) lead to Note that by (31) and (21), we obtain In the following theorem, we recap these results. Theorem 1. Given S's strategies η [1,n] , C's optimal reactions γ (κ) θ ,k (η [1,k] ), where θ ∈ {F, A 1 , . . . , A t }, are given by (21) or (45) depending on whether C is a friend or an adversary, respectively. These reaction strategies are unique.
In the following section, we formulate S's optimal strategies.

V. OPTIMAL LEADER (SENSOR) ACTIONS
For any given η [1,n] , Theorem 1 provides the unique optimal reactions of friendly and adversarial agents. Now, for each sequence θ [1,N] ∈ Ω, we aim to compute the optimal S strategies that minimize ∑ θ θ θ [1,N] ∈Ω µ(θ θ θ [1,N] ) ∑ κ∈h(θ θ θ [1,N] ) To this end, we first seek to write u u u (κ) * θ ,k , θ ∈ {F, A 1 , . . . , A t }, derived in (21) and (45), in the same form. By (21), we have Correspondingly, (45) leads to Let Z (1) F = 0 be a zero vector, then for θ ∈ {F, A 1 , . . . , A t }, we obtain u u u where T (κ) θ is a (n−κ +1)r ×nm matrix and Z (κ) θ ∈ R (n−κ+1)r . However, in (46), only u u u (κ) * θ θ θ j ,k for k = κ, . . . , κ + −1 are included. Let κ T be the last non-terminating state transition time, and n T denote the termination time corresponding to a jump to state T or end of horizon. Then, by (48), for a given realization of the process {θ θ θ j }, e.g., θ [1,N] ∈ Ω, we have Therefore, we obtain Even though S constructs a single set of strategies {η k ∈ ϒ} without knowing C's type, the resulting sensor outputs {s s s k = η k (x x x k )} depend on the state x x x k , hence C's type and correspondingly θ [1,N] . However, as shown in Section IV, the problem entails classical information andx xx o does not depend on θ [1,N] . Therefore, let u u u * θ [1,N] ,k be the corresponding control input at time k according to (49) for a given realization θ [1,N] . Then, the objective function (46) can be written as (50) Note that the inner summation can be written as where Similar to (13), the sequence {Q S,k } is defined through the following discrete-time Riccati equation: Then, by (14), (51) leads to subject to (52a), where x x x o k evolves according to (15a), and u u u o * θ θ θ [1,N] ,k = u u u * θ θ θ [1,N] ,k + K S,k A k−2 Bu u u * θ θ θ [1,N] ,1 . We point out that due to time-invariant Q F and R F , we havě whereQ F,k , K F,k , and ∆ F,k are defined in (14). Therefore, the summation can be written as Furthermore, in terms of u u u * θ θ θ [1,N] , u u u o * θ θ θ [1,N] is given by . . .

SDP Problem:
Solve the SDP problem on the left hand side of (70) through a numerical toolbox, e.g., CVX [25], [26], and obtain the solutions S * k , for k = 1, . . . , n. Set S * 0 = O. Equilibrium achieving sensor strategies: Compute the corresponding idempotent matrices P k ,∀k, by using S * k , ∀k, and (71). Compute the eigen decompositions: P k = U k Λ k U k .
which are independent of the optimization arguments. Hence, the optimization problem (66) faced by S can be written as an affine function of H k 's as follows: for certain symmetric deterministic matrices 7 V k ∈ S m , k = 1, . . . , n, which are given by and Π k,l is the corresponding m × m sub-block of Π. As a secure sensor designer, we seek to solve this nonlinear optimization problem (68). Note that [1,k] }. However, as pointed out in Remark 6, a brute force approach is computationally expensive. To this end, we employ the approach in [24], which considers another optimization problem that bounds the original one from below, and then, compute strategies for the original problem, which optimize the lower bound. Based on this, the following theorem characterizes equilibrium achieving secure sensor strategies. Theorem 2. The optimal linear secure sensor strategies can be computed via Algorithm 1, described in Table I. Proof. Based on Lemma 3 in [24], by characterizing necessary conditions on H k 's, we have min S k ∈S m , k=1,...,n ∑ n k=1 tr{V k S k } ≤ min η k ∈ϒ, k=1,...,n ∑ n k=1 tr{V k H k }, Note that the left hand side of (70) is an SDP problem. By invoking Theorem 4 in [24], we can characterize the solution, S * 1 , . . . , S * n , as 71) for k = 1, . . . , n, where S * 0 = O and P k ∈ S m is a certain symmetric idempotent matrix. Note that by solving the SDP problem numerically, we can compute the corresponding P k 's.
Next, say that S employs memoryless linear policies s s s k = η k (x x x k ) = L k x x x k . Then, by (17) and (33), we havê which can also be written aŝ for k = 1, . . . , n,x xx o −1 := 0 and H 0 := O. Therefore, We emphasize the resemblance between (71) and (73). In particular, if we setL k : yields whereL k (L kL k ) †L k is also a symmetric idempotent matrix just like P k in (71). Therefore, given P k 's, let P k = U k Λ k U k be the eigen decomposition, and setL k = U k Λ k , i.e., set Then, we obtain H k = S * k , which implies that S's optimal strategies are given by (75) while the optimal control inputs for both friendly and adversarial C are given by (21) or (45), respectively. Remark 8. We have considered that S knows the underlying state transition probabilities, i.e., the statistics of the jump process {θ θ θ j }. However, for a robust design against inaccurate perception of the statistics, S can design the sensor outputs by being cautious for the worst case scenario. For all possible measures of θ θ θ [1,N] ∈ Ω, we can recompute Π and Π o in (67). Let V denote the set of the corresponding (V 1 , . . . ,V n ) tuples, which can be computed via (69). Then, by (70), the worst case scenario is equivalent to subject to Σ j S j AS j−1 A , j = 1, . . . , n and S 0 = O. If V is compact and convex, there exists a saddle point equilibrium by the Minimax theorem [21]. Otherwise, we can consider the convex hull of V , denoted by V c for computational simplicity in addition to robustness. A detailed analysis in that direction is left as future work.
In the following section, we provide several numerical examples examining the performance of secure sensor design.

VI. ILLUSTRATIVE EXAMPLES
As numerical illustrations, we consider two different scenarios: Scenario 1, where we compare the performance of the proposed secure sensors with classical sensors that disclose x x x k to C directly, and Scenario 2, where we analyze the robustness of the proposed scheme against inaccurate perception of the state transition statistics, i.e., µ P (θ [1,N] ) = µ(θ [1,N] ) for θ [1,N] ∈ Ω, where µ P denotes the perceived statistics while µ denotes the actual ones. We set time horizon n = 100, the state's dimension m = 8, and the control input's dimension r = 2. The matrices in the state recursion (1) are set randomly according to uniform distribution such that A is not a singular matrix and scaled by 0.1 in order to avoid computational instability. In order to construct auto-covariance matrices Σ 1 and Σ v , we draw a number from the uniform distribution on the unit interval [0, 1] for each entry of a matrix D. Then, we can construct a positive-definite covariance matrix by (D + D )/2 + 2m I, where the last term ensures that the constructed matrix is diagonally dominant, and therefore, positive-definite. Then, we have scaled Σ v by 10 since the sensor outputs play more essential role for C when the state noise variance is larger.
Scenario 1 -Performance Comparison. We specifically consider the scenario where there are two adversaries, who seek to regularize state x x x k around z 1 , z 2 ∈ R m that are drawn from a multivariate Gaussian distribution. The positive semidefinite weight matrices in the cost functions are set such that and R F = R A 1 = R A 2 = I r while λ 1 = λ 2 = 0.1. The state transition interval is set δ = 35 and Fig. 3 shows the possible state transitions, e.g., θ θ θ [1,N] ∈ Ω, within the time horizon. We consider the situation where the normalized measure of noinfiltration case, i.e., θ [1,N] = {F, F, F}, referred to as Case-1, is 0.7 while all the other cases, i.e., Cases 2-13, have the same measure, as tabulated in Table II.
In Table II, we compare the performances of three different schemes for the cases shown in Fig. 3. The classical scheme refers to a sensor, who discloses x x x k directly to C, while no sensor output refers to open-loop control of the system, i.e., s s s k = 0 almost everywhere on R m . As a performance measure for each θ θ θ [1,N] ∈ Ω, we consider (51), where the last term G [n T ] S , which does not depend on S's strategies η [1,n] , is excluded, i.e., µ(θ θ θ [1,N] )Ĵ S (θ θ θ [1,N] ).
In Table II, we have observed that open-loop control of the system leads to inferior performance compared to the other schemes in all the cases. The classical scheme outperforms the proposed secure sensor scheme only in Case 1 yet slightly. Note that Case 1 is the best case scenario, where there is no infiltration into C. In all the other cases, the proposed scheme outperforms the classical scheme substantially. Even though the best case scenario is relatively likely compared to the other cases, on the average, the proposed scheme outperforms the classical scheme also substantially and achieves 40% enhancement in the performance. Furthermore, in Fig. 4, we have plotted the time evolution of the singular values of the normalized gain matrix L k L k . Note that the gain matrix L k and the normalized gain matrix L k L k leads to the same performance, while the normalized one is preferred in Fig. 4 for better demonstration. We also note that while L k ∈ R m×m , all the gain matrices have rank less than m = 8, i.e., L 1 , . . . , L n−1 have rank 2 and the last stage gain matrix L n has rank 4.
Scenario 2 -Robustness Analysis. We examine the robustness of the proposed scheme for inaccurate perception of the underlying state transition statistics. To this end, we consider the setup in Scenario 1; however, now S perceives the statistics  as tabulated in Table III. Particularly, S is not aware of the attacker A 2 , and designs the secure sensor outputs accordingly even though the actual underlying statistics are as in Scenario 1, which is also provided in Table III. We have observed that the performance degrades due to inaccurate perception of the statistics; however, the proposed scheme still outperforms classical scheme, whose performance is tabulated in Table II, in individual cases, Cases 2 − 13, and on the average. We note that since, in the inaccurate perception of the statistics, Case 1 has relatively higher probability, i.e., 0.85 > 0.70, we have observed slight improvement in the performance in that case.

VII. CONCLUSION
In this paper, we have proposed and addressed secure sensor design problem for cyber-physical systems with linear Gaussian dynamics against the advanced persistent threats with quadratic control objectives. By designing sensor outputs cautiously in advance, we have sought to minimize the damage that can be caused by undetected target-specific threats. To this end, we have modeled the problem formally in a gametheoretical setting. We have determined the optimal control inputs for both friendly and adversarial objectives for given linear sensor strategies. Then, we have provided an algorithm to compute the optimal linear secure sensor strategies that lead to the equilibrium. We note that without linearity assumption on the sensor outputs, the problem entails non-classical information model due to distinct objectives of the agents. Furthermore, for general, e.g., nonlinear, sensor outputs, the corresponding optimal control policies could not be unique and could not even be expressed in closed form [17].
Some future directions of research on this topic include: • Formulation of secure sensor design strategies when the sensor has access to noisy observations, or for, e.g., robust control or feedback stability of the systems. • Here, we have considered the scenarios, where the attackers have perfect knowledge about the underlying state recursion. Another interesting extension would be to analyze the scenarios, where the attackers can only have partial knowledge. In such scenarios, intuitively, sensor outputs could play relatively more powerful roles since, without caution, sensors might help the attackers to recover the unknown part of the system dynamics.