It is well known that sequential decision making may lead to information cascades. That is, when agents make decisions based on their private information, as well as observing the actions of those before them, then it might be rational to ignore their private signal and imitate the action of previous individuals. If the individuals are choosing between a right and a wrong state, and the initial actions are wrong, then the whole cascade will be wrong. This issue is due to the fact that cascades can be based on very little information. We show that if agents occasionally disregard the actions of others and base their action only on their private information, then wrong cascades can be avoided. Moreover, we study the optimal asymptotic rate at which the error probability at time @math can go to zero. The optimal policy is for the player at time @math to follow their private information with probability @math , leading to a learning rate of @math , where the constants @math and @math are explicit.
The framework of this paper also fits into the broader field of social learning. In particular, there is a large literature on learning in social networks. Acemoglu, Dahleh, Lobel, and Ozdaglar @cite_5 consider a model of sequential decision making where agents act only once, but each agent can only observe a of previous actions, based on a stochastic social network. One of their results is that asymptotic learning occurs even if private signals have bounded informativeness if there are sufficiently many individuals whose neighborhoods are non-persuasive and hence whose action will necessarily be influenced by their private signal. Similar to revealers in the model described in , these individuals provide a sufficient amount of information for those coming after them to lead to asymptotic learning.
@cite_5
2110152576 1970930040 2804072087 2068143054
We study the perfect Bayesian equilibrium of a model of learning over a general social network. Each individual receives a signal about the underlying state of the world, observes the past actions of a stochastically-generated neighborhood of individuals, and chooses one of two possible actions. The stochastic process generating the neighborhoods defines the network topology (social network). The special case where each individual observes all past actions has been widely studied in the literature. We characterize pure-strategy equilibria for arbitrary stochastic and deterministic social networks and characterize the conditions under which there will be asymptotic learning -- that is, the conditions under which, as the social network becomes large, individuals converge (in probability) to taking the right action. We show that when private beliefs are unbounded (meaning that the implied likelihood ratios are unbounded), there will be asymptotic learning as long as there is some minimal amount of "expansion in observations". Our main theorem shows that when the probability that each individual observes some other individual from the recent past converges to one as the social network becomes large, unbounded private beliefs are sufficient to ensure asymptotic learning. This theorem therefore establishes that, with unbounded private beliefs, there will be asymptotic learning an almost all reasonable social networks. We also show that for most network topologies, when private beliefs are bounded, there will not be asymptotic learning. In addition, in contrast to the special case where all past actions are observed, asymptotic learning is possible even with bounded beliefs in certain stochastic network topologies. We study the stochastic multi-armed bandit (MAB) problem in the presence of side-observations across actions. In our model, choosing an action provides additional side observations for a subset of the remaining actions. One example of this model occurs in the problem of targeting users in online social networks where users respond to their friends's activity, thus providing information about each other's preferences. Our contributions are as follows: 1) We derive an asymptotic (with respect to time) lower bound (as a function of the network structure) on the regret (loss) of any uniformly good policy that achieves the maximum long term average reward. 2) We propose two policies - a randomized policy and a policy based on the well-known upper confidence bound (UCB) policies, both of which explore each action at a rate that is a function of its network position. We show that these policies achieve the asymptotic lower bound on the regret up to a multiplicative factor independent of network structure. The upper bound guarantees on the regret of these policies are better than those of existing policies. Finally, we use numerical examples on a real-world social network to demonstrate the significant benefits obtained by our policies against other existing policies. In a wide variety of applications, humans interact with a complex environment by means of asynchronous stochastic discrete events in continuous time. Can we design online interventions that will help humans achieve certain goals in such asynchronous setting? In this paper, we address the above problem from the perspective of deep reinforcement learning of marked temporal point processes, where both the actions taken by an agent and the feedback it receives from the environment are asynchronous stochastic discrete events characterized using marked temporal point processes. In doing so, we define the agent's policy using the intensity and mark distribution of the corresponding process and then derive a flexible policy gradient method, which embeds the agent's actions and the feedback it receives into real-valued vectors using deep recurrent neural networks. Our method does not make any assumptions on the functional form of the intensity and mark distribution of the feedback and it allows for arbitrarily complex reward functions. We apply our methodology to two different applications in personalized teaching and viral marketing and, using data gathered from Duolingo and Twitter, we show that it may be able to find interventions to help learners and marketers achieve their goals more effectively than alternatives. We consider a new problem in multiterminal source coding motivated by the following decentralized communication estimation task. A firm's Chief Executive Officer (CEO) is interested in the data sequence X(t) sub t=1 sup spl infin which cannot be observed directly, perhaps because it represents tactical decisions by a competing firm. The CEO deploys a team of L agents who observe independently corrupted versions of X(t) sub t=1 sup spl infin . Because X(t) is only one among many pressing matters to which the CEO must attend, the combined data rate at which the agents may communicate information about their observations to the CEO is limited to, say, R bits per second. If the agents were permitted to confer and pool their data, then in the limit as L spl rarr spl infin they usually would be able to smooth out their independent observation noises entirely. Then they could use their R bits per second to provide the CEO with a representation of X(t) with fidelity D(R), where D( spl middot ) is the distortion-rate function of X(t) . In particular, with such data pooling D can be made arbitrarily small if R exceeds the entropy rate H of X(t) . Suppose, however, that the agents are not permitted to convene, Agent i having to send data based solely on his own noisy observations Y sub i (t) . We show that then there does not exist a finite value of R for which even infinitely many agents can make D arbitrarily small. Furthermore, in this isolated-agents case we determine the asymptotic behavior of the minimal error frequency in the limit as L and then R tend to infinity.