The tendency to overestimate immediate utility is a common cognitive bias. As a result people behave inconsistently over time and fail to reach long-term goals. Behavioral economics tries to help affected individuals by implementing external incentives. However, designing robust incentives is often difficult due to imperfect knowledge of the parameter @math quantifying a person's present bias. Using the graphical model of Kleinberg and Oren, we approach this problem from an algorithmic perspective. Based on the assumption that the only information about @math is its membership in some set @math , we distinguish between two models of uncertainty: one in which @math is fixed and one in which it varies over time. As our main result we show that the conceptual loss of efficiency incurred by incentives in the form of penalty fees is at most @math in the former and @math in the latter model. We also give asymptotically matching lower bounds and approximation algorithms.
The graphical model is of particular interest to us as it provides a natural framework for a design problem frequently encountered in behavioral economics. Given a certain social or economic setting, the problem is to improve a time-inconsistent person's performance via various sorts of incentives , such as monetary rewards, deadlines or penalty fees, see e.g. @cite_7 . Using the graphical model, Kleinberg and Oren demonstrate how a strategic choice reduction can incentivize people to reach predefined goals @cite_4 . To implement their incentives, they simply remove the corresponding edges from the task graph. However, there is a computational drawback to this approach. As we have shown in previous work, an optimal set of edges to remove from a task graph with @math nodes is NP-hard to approximate within a factor less than @math @cite_8 . A more general form of incentives avoiding these harsh complexity theoretic limitations are penalty fees. In the graphical model penalty fees are at least as powerful as choice reduction and admit a polynomial time @math -approximation @cite_3 .
@cite_3 @cite_4 @cite_7 @cite_8
2225272567 2080430334 2594830350 2964268978
We study the complexity of motivating time-inconsistent agents to complete long term projects in a graph-based planning model as proposed by Kleinberg and Oreni¾?[5]. Given a task graph G with n nodes, our objective is to guide an agent towards a target node t under certain budget constraints. The crux is that the agent may change its strategy over time due to its present-bias. We consider two strategies to guide the agent. First, a single reward is placed at t and arbitrary edges can be removed from G. Secondly, rewards can be placed at arbitrary nodes of G but no edges must be deleted. In both cases we show that it is NP-complete to decide if a given budget is sufficient to guide the agent. For the first setting, we give complementing upper and lower bounds on the approximability of the minimum required budget. In particular, we devise a @math 1+n-approximation algorithm and prove NP-hardness for ratios greater than @math n 3. Finally, we argue that the second setting does not permit any efficient approximation unless @math P=NP. Graphical models (e.g., Bayesian and constraint networks, influence diagrams, and Markov decision processes) have become a central paradigm for knowledge representation and reasoning in both artificial intelligence and computer science in general. These models are used to perform many reasoning tasks, such as scheduling, planning and learning, diagnosis and prediction, design, hardware and software verification, and bioinformatics. These problems can be stated as the formal tasks of constraint satisfaction and satisfiability, combinatorial optimization, and probabilistic inference. It is well known that the tasks are computationally hard, but research during the past three decades has yielded a variety of principles and techniques that significantly advanced the state of the art. In this book we provide comprehensive coverage of the primary exact algorithms for reasoning with such models. The main feature exploited by the algorithms is the model's graph. We present inference-based, message-passing schemes (e.g., variable-elimination) and search-based, conditioning schemes (e.g., cycle-cutset conditioning and AND OR search). Each class possesses distinguished characteristics and in particular has different time vs. space behavior. We emphasize the dependence of both schemes on few graph parameters such as the treewidth, cycle-cutset, and (the pseudo-tree) height. We believe the principles outlined here would serve well in moving forward to approximation and anytime-based schemes. The target audience of this book is researchers and students in the artificial intelligence and machine learning area, and beyond. Table of Contents: Preface Introduction What are Graphical Models Inference: Bucket Elimination for Deterministic Networks Inference: Bucket Elimination for Probabilistic Networks Tree-Clustering Schemes AND OR Search Spaces and Algorithms for Graphical Models Combining Search and Inference: Trading Space for Time Conclusion Bibliography Author's Biography People tend to behave inconsistently over time due to an inherent present bias. As this may impair performance, social and economic settings need to be adapted accordingly. Common tools to reduce the impact of time-inconsistent behavior are penalties and prohibition. Such tools are called commitment devices. In recent work Kleinberg and Oren [EC, 2014] connect the design of a prohibition-based commitment device to a combinatorial problem in which edges are removed from a task graph G with n nodes. However, this problem is NP-hard to approximate within a ratio less than n^(1 2) 3 [Albers and Kraft, WINE, 2016]. To address this issue, we propose a penalty-based commitment device that does not delete edges, but raises their cost. The benefits of our approach are twofold. On the conceptual side, we show that penalties are up to 1 beta times more efficient than prohibition, where 0 < beta <= 1 parameterizes the present bias. On the computational side, we improve approximability by presenting a 2-approximation algorithm for allocating penalties. To complement this result, we prove that optimal penalties are NP-hard to approximate within a ratio of 1.08192. As a new way of training generative models, Generative Adversarial Net (GAN) that uses a discriminative model to guide the training of the generative model has enjoyed considerable success in generating real-valued data. However, it has limitations when the goal is for generating sequences of discrete tokens. A major reason lies in that the discrete outputs from the generative model make it difficult to pass the gradient update from the discriminative model to the generative model. Also, the discriminative model can only assess a complete sequence, while for a partially generated sequence, it is nontrivial to balance its current score and the future one once the entire sequence has been generated. In this paper, we propose a sequence generation framework, called SeqGAN, to solve the problems. Modeling the data generator as a stochastic policy in reinforcement learning (RL), SeqGAN bypasses the generator differentiation problem by directly performing gradient policy update. The RL reward signal comes from the GAN discriminator judged on a complete sequence, and is passed back to the intermediate state-action steps using Monte Carlo search. Extensive experiments on synthetic data and real-world tasks demonstrate significant improvements over strong baselines.