We review the problem of defining and inferring a "state" for a control system based on complex, high-dimensional, highly uncertain measurement streams such as videos. Such a state, or representation, should contain all and only the information needed for control, and discount nuisance variability in the data. It should also have finite complexity, ideally modulated depending on available resources. This representation is what we want to store in memory in lieu of the data, as it "separates" the control task from the measurement process. For the trivial case with no dynamics, a representation can be inferred by minimizing the Information Bottleneck Lagrangian in a function class realized by deep neural networks. The resulting representation has much higher dimension than the data, already in the millions, but it is smaller in the sense of information content, retaining only what is needed for the task. This process also yields representations that are invariant to nuisance factors and having maximally independent components. We extend these ideas to the dynamic case, where the representation is the posterior density of the task variable given the measurements up to the current time, which is in general much simpler than the prediction density maintained by the classical Bayesian filter. Again this can be finitely-parametrized using a deep neural network, and already some applications are beginning to emerge. No explicit assumption of Markovianity is needed; instead, complexity trades off approximation of an optimal representation, including the degree of Markovianity.
The trade-off we seek between complexity of the representation and sufficiency for future prediction is also closely related to the Minimum-Information LQG Control of @cite_41 , which explicitly accounts for the agent having a representation of bounded complexity. @cite_41 only addresses the linear-quadratic Gaussian case, but gives a complete account of that. Similarly @cite_8 @cite_20 deal with capacity costs, but in continuous time. The general principles are laid out in @cite_0 .
@cite_41 @cite_0 @cite_20 @cite_8
2015893577 2588872972 2541696016 2225272567
Motivated in part by the “rational inattention” framework of information-constrained decision-making by economic agents, we have recently introduced a general model for average-cost optimal control of Markov processes subject to mutual information constraints [1]. The optimal information-constrained control problem reduces to an infinite-dimensional convex program and admits a decomposition based on the Bellman error, which is the object of study in approximate dynamic programming. In this paper, we apply our general theory to an information-constrained variant of the scalar linear-quadratic-Gaussian (LQG) control problem. We give an upper bound on the optimal steady-state value of the quadratic performance objective and present explicit constructions of controllers that achieve this bound. We show that the obvious certainty-equivalent control policy is suboptimal when the information constraints are very severe, and exhibit another policy that performs better in this low-information regime. In the two extreme cases of no information (open-loop) and perfect information, these two policies coincide with the optimum. Consider a control problem with a communication channel connecting the observer of a linear stochastic system to the controller. The goal of the controller is to minimize a quadratic cost function in the state variables and control signal, known as the linear quadratic regulator (LQR). We study the fundamental tradeoff between the communication rate @math bits sec and the expected cost @math . We obtain a lower bound on a certain rate-cost function, which quantifies the minimum directed mutual information between the channel input and output that is compatible with a target LQR cost. The rate-cost function has operational significance in multiple scenarios of interest: among others, it allows us to lower-bound the minimum communication rate for fixed and variable length quantization, and for control over noisy channels. We derive an explicit lower bound to the rate-cost function, which applies to the vector, non-Gaussian, and partially observed systems, thereby extending and generalizing an earlier explicit expression for the scalar Gaussian system, due to Tatikonda el al. The bound applies as long as the system noise has a probability density function. It can be closely approached by a simple lattice quantization scheme that only quantizes the innovation, that is, the difference between the controller's belief about the current state and the true state. Via a separation principle between control and communication, similar results hold for causal lossy compression of additive noise Markov sources. Apart from standard dynamic programming arguments, our technical approach leverages the Shannon lower bound, develops new estimates for data compression with coding memory, and uses some recent results on high resolution variable-length vector quantization to prove that the new converse bounds are tight. This paper is about allocation of an infinitely divisible good to several rational and strategic agents. The allocation is done by a social planner who has limited information because the agents’ valuation functions are taken to be private information known only to the respective agents. We allow only a scalar signal, called a bid, from each agent to the social planner. Yang and Hajek [Yang, S., Hajek, B., 2007. “VCG-Kelly mechanisms for allocation of divisible goods: Adapting VCG mechanisms to one-dimensional signals”, IEEE Journal on Selected Areas in Communications 25 (6), 1237–1243.] and Johari and Tsitsiklis [Johari, R., Tsitsiklis, J. N., 2009. “Efficiency of scalar-parameterized mechanisms”, Operations Research 57 (4), 823–839.] proposed a scalar strategy Vickrey–Clarke–Groves (SSVCG) mechanism with efficient Nash equilibria. We consider a setting where the social planner desires minimal budget surplus. Example situations include fair sharing of Internet resources and auctioning of certain public goods where revenue maximization is not a consideration. Under the SSVCG framework, we propose a mechanism that is efficient and comes close to budget balance by returning much of the payments back to the agents in the form of rebates. We identify a design criterion for almost budget balance, impose feasibility and voluntary participation constraints, simplify the constraints, and arrive at a convex optimization problem to identify the parameters of the rebate functions. The convex optimization problem has a linear objective function and a continuum of linear constraints. We propose a solution method that involves a finite number of constraints, and identify the number of samples sufficient for a good approximation. We study the complexity of motivating time-inconsistent agents to complete long term projects in a graph-based planning model as proposed by Kleinberg and Oreni¾?[5]. Given a task graph G with n nodes, our objective is to guide an agent towards a target node t under certain budget constraints. The crux is that the agent may change its strategy over time due to its present-bias. We consider two strategies to guide the agent. First, a single reward is placed at t and arbitrary edges can be removed from G. Secondly, rewards can be placed at arbitrary nodes of G but no edges must be deleted. In both cases we show that it is NP-complete to decide if a given budget is sufficient to guide the agent. For the first setting, we give complementing upper and lower bounds on the approximability of the minimum required budget. In particular, we devise a @math 1+n-approximation algorithm and prove NP-hardness for ratios greater than @math n 3. Finally, we argue that the second setting does not permit any efficient approximation unless @math P=NP.