Partially Observed Decision Making and Output Feedback Control.
In many problems [1]}, [2]}, only specific outputs but not the full system states are available for the decision maker. Therefore, memory in the controller is required to recover the full system states [3]}. Control of these partially observed systems is often referred to as output feedback control [4]}, and has been studied extensively from both control and optimization perspectives [5]}, [6]}. Under the setting with convexifiable objectives (e.g., \(H_{\infty }\)  or \(H_2\)  performances), the optimal linear dynamic (i.e.  with memory) controller can be obtained by using a change of variables or solving algebraic Riccati equations [7]}, [8]}. However, for more sophisticated settings with unknown and/or flexibly defined cost functions, the problems become intractable for the aforementioned traditional methods, and RL techniques
are proposed to reduced the computation cost and improve overall performance at test time, including the ones [9]}, [10]} with static neural network controllers, and the ones [11]}, [12]}, [13]} with dynamic controllers, represented by RNNs/long short-term memory neural networks.
