Contextual bandits with linear payoffs, which are also known as linear bandits, provide a powerful alternative for solving practical problems of sequential decisions, e.g., online advertisements. In the era of big data, contextual data usually tend to be high-dimensional, which leads to new challenges for traditional linear bandits mostly designed for the setting of low-dimensional contextual data. Due to the curse of dimensionality, there are two challenges in most of the current bandit algorithms: the first is high time-complexity; and the second is extreme large upper regret bounds with high-dimensional data. In this paper, in order to attack the above two challenges effectively, we develop an algorithm of Contextual Bandits via RAndom Projection ( CBRAP ) in the setting of linear payoffs, which works especially for high-dimensional contextual data. The proposed CBRAP algorithm is time-efficient and flexible, because it enables players to choose an arm in a low-dimensional space, and relaxes the sparsity assumption of constant number of non-zero components in previous work. Besides, we provide a linear upper regret bound for the proposed algorithm, which is associated with reduced dimensions.
The total sequential rounds of playing bandits is @math . For each round @math with @math , a learner receives contextual information from the set of @math , where @math can be an extremely large integer representing a high-dimensional space. In this work, high-dimensional contextual data precisely mean that @math or even @math , which is the case mentioned in @cite_6 . Let @math be the number of arms and @math the reward of arm @math on round @math with @math and @math . We adopt @math to denote the @math norm of a vector @math , and @math to denote the identity matrix with dimensions of @math . For a positive definite matrix @math , the weighted norm of vector @math is defined as @math . The inner product is represented as @math , and the weighted inner product is @math .
@cite_6
2950816192
Motivated by online recommendation and advertising systems, we consider a causal model for stochastic contextual bandits with a latent low-dimensional confounder. In our model, there are @math observed contexts and @math arms of the bandit. The observed context influences the reward obtained through a latent confounder variable with cardinality @math ( @math ). The arm choice and the latent confounder causally determines the reward while the observed context is correlated with the confounder. Under this model, the @math mean reward matrix @math (for each context in @math and each arm in @math ) factorizes into non-negative factors @math ( @math ) and @math ( @math ). This insight enables us to propose an @math -greedy NMF-Bandit algorithm that designs a sequence of interventions (selecting specific arms), that achieves a balance between learning this low-dimensional structure and selecting the best arm to minimize regret. Our algorithm achieves a regret of @math at time @math , as compared to @math for conventional contextual bandits, assuming a constant gap between the best arm and the rest for each context. These guarantees are obtained under mild sufficiency conditions on the factors that are weaker versions of the well-known Statistical RIP condition. We further propose a class of generative models that satisfy our sufficient conditions, and derive a lower bound of @math . These are the first regret guarantees for online matrix completion with bandit feedback, when the rank is greater than one. We further compare the performance of our algorithm with the state of the art, on synthetic and real world data-sets. We consider settings in which we wish to incentivize myopic agents (such as Airbnb landlords, who may emphasize short-term profits and property safety) to treat arriving clients fairly, in order to prevent overall discrimination against individuals or groups. We model such settings in both classical and contextual bandit models in which the myopic agents maximize rewards according to current empirical averages, but are also amenable to exogenous payments that may cause them to alter their choices. Our notion of fairness asks that more qualified individuals are never (probabilistically) preferred over less qualified ones []. We investigate whether it is possible to design inexpensive subsidy or payment schemes for a principal to motivate myopic agents to play fairly in all or almost all rounds. When the principal has full information about the state of the myopic agents, we show it is possible to induce fair play on every round with a subsidy scheme of total cost @math (for the classic setting with @math arms, @math , and for the @math -dimensional linear contextual setting @math ). If the principal has much more limited information (as might often be the case for an external regulator or watchdog), and only observes the number of rounds in which members from each of the @math groups were selected, but not the empirical estimates maintained by the myopic agent, the design of such a scheme becomes more complex. We show both positive and negative results in the classic and linear bandit settings by upper and lower bounding the cost of fair subsidy schemes. We present a near-optimal distributed algorithm for @math -approximation of single-commodity maximum flow in undirected weighted networks that runs in @math communication rounds in the model. Here, @math and @math denote the number of nodes and the network diameter, respectively. This is the first improvement over the trivial bound of @math , and it nearly matches the @math round complexity lower bound. The development of the algorithm contains two results of independent interest: (i) A @math -round distributed construction of a spanning tree of average stretch @math . (ii) A @math -round distributed construction of an @math -congestion approximator consisting of the cuts induced by @math virtual trees. The distributed representation of the cut approximator allows for evaluation in @math rounds. All our algorithms make use of randomization and succeed with high probability. We show a parallel repetition theorem for the entangled value @math of any two-player one-round game @math where the questions @math to Alice and Bob are drawn from a product distribution on @math . We show that for the @math -fold product @math of the game @math (which represents the game @math played in parallel @math times independently), @math , where @math and @math represent the sets from which the answers of Alice and Bob are drawn. Let @math be the maximal value such that the product of an @math matrix by an @math matrix can be computed with @math arithmetic operations. In this paper we show that @math , which improves the previous record @math by Coppersmith (Journal of Complexity, 1997). More generally, we construct a new algorithm for multiplying an @math matrix by an @math matrix, for any value @math . The complexity of this algorithm is better than all known algorithms for rectangular matrix multiplication. In the case of square matrix multiplication (i.e., for @math ), we recover exactly the complexity of the algorithm by Coppersmith and Wino grad (Journal of Symbolic Computation, 1990). These new upper bounds can be used to improve the time complexity of several known algorithms that rely on rectangular matrix multiplication. For example, we directly obtain a @math -time algorithm for the all-pairs shortest paths problem over directed graphs with small integer weights, where @math denotes the number of vertices, and also improve the time complexity of sparse square matrix multiplication. We consider the problem of selecting the best subset of exactly @math columns from an @math matrix @math . We present and analyze a novel two-stage algorithm that runs in @math time and returns as output an @math matrix @math consisting of exactly @math columns of @math . In the first (randomized) stage, the algorithm randomly selects @math columns according to a judiciously-chosen probability distribution that depends on information in the top- @math right singular subspace of @math . In the second (deterministic) stage, the algorithm applies a deterministic column-selection procedure to select and return exactly @math columns from the set of columns selected in the first stage. Let @math be the @math matrix containing those @math columns, let @math denote the projection matrix onto the span of those columns, and let @math denote the best rank- @math approximation to the matrix @math . Then, we prove that, with probability at least 0.8, @math This Frobenius norm bound is only a factor of @math worse than the best previously existing existential result and is roughly @math better than the best previous algorithmic result for the Frobenius norm version of this Column Subset Selection Problem (CSSP). We also prove that, with probability at least 0.8, @math This spectral norm bound is not directly comparable to the best previously existing bounds for the spectral norm version of this CSSP. Our bound depends on @math , whereas previous results depend on @math ; if these two quantities are comparable, then our bound is asymptotically worse by a @math factor. Generalized Linear Bandits (GLBs), a natural extension of the stochastic linear bandits, has been popular and successful in recent years. However, existing GLBs scale poorly with the number of rounds and the number of arms, limiting their utility in practice. This paper proposes new, scalable solutions to the GLB problem in two respects. First, unlike existing GLBs, whose per-time-step space and time complexity grow at least linearly with time @math , we propose a new algorithm that performs online computations to enjoy a constant space and time complexity. At its heart is a novel Generalized Linear extension of the Online-to-confidence-set Conversion (GLOC method) that takes online learning algorithm and turns it into a GLB algorithm. As a special case, we apply GLOC to the online Newton step algorithm, which results in a low-regret GLB algorithm with much lower time and memory complexity than prior work. Second, for the case where the number @math of arms is very large, we propose new algorithms in which each next arm is selected via an inner product search. Such methods can be implemented via hashing algorithms (i.e., "hash-amenable") and result in a time complexity sublinear in @math . While a Thompson sampling extension of GLOC is hash-amenable, its regret bound for @math -dimensional arm sets scales with @math , whereas GLOC's regret bound scales with @math . Towards closing this gap, we propose a new hash-amenable algorithm whose regret bound scales with @math . Finally, we propose a fast approximate hash-key computation (inner product) with a better accuracy than the state-of-the-art, which can be of independent interest. We conclude the paper with preliminary experimental results confirming the merits of our methods. Contextual bandits are widely used in Internet services from news recommendation to advertising, and to Web search. Generalized linear models (logistical regression in particular) have demonstrated stronger performance than linear models in many applications where rewards are binary. However, most theoretical analyses on contextual bandits so far are on linear bandits. In this work, we propose an upper confidence bound based algorithm for generalized linear contextual bandits, which achieves an @math regret over @math rounds with @math dimensional feature vectors. This regret matches the minimax lower bound, up to logarithmic terms, and improves on the best previous result by a @math factor, assuming the number of arms is fixed. A key component in our analysis is to establish a new, sharp finite-sample confidence bound for maximum-likelihood estimates in generalized linear models, which may be of independent interest. We also analyze a simpler upper confidence bound algorithm, which is useful in practice, and prove it to have optimal regret for certain cases. Let @math be an @math real random matrix whose elements are independent and identically distributed standard normal random variables, and let @math be the 2-norm condition number of @math . We prove that, for any @math , @math , and @math , @math satisfies @math where @math and @math @math are universal positive constants independent of @math , @math , and @math . Moreover, for any @math and @math , @math A similar pair of results for complex Gaussian random matrices is also established. Given an @math matrix @math , its Schatten- @math norm, @math , is defined as @math , where @math is the @math -th largest singular value of @math . These norms have been studied in functional analysis in the context of non-commutative @math -spaces, and recently in data stream and linear sketching models of computation. Basic questions on the relations between these norms, such as their embeddability, are still open. Specifically, given a set of matrices @math , suppose we want to construct a linear map @math such that @math for each @math , where @math and @math , and further, @math for a given approximation factor @math and real number @math . Then how large do @math and @math need to be as a function of @math ? We nearly resolve this question for every @math , for the case where @math can be expressed as @math , where @math and @math are arbitrary matrices that are allowed to depend on @math , that is, @math can be implemented by left and right matrix multiplication. Namely, for every @math , we provide nearly matching upper and lower bounds on the size of @math and @math as a function of @math . Importantly, our upper bounds are oblivious , meaning that @math and @math do not depend on the @math , while our lower bounds hold even if @math and @math depend on the @math . As an application of our upper bounds, we answer a recent open question of about space-approximation trade-offs for the Schatten @math -norm, showing in a data stream it is possible to estimate the Schatten- @math norm up to a factor of @math using @math space. We study the asymptotic behavior of the persistent homology of i.i.d. samples from a @math -Ahlfors regular measure --- one that satisfies uniform bounds of the form for some @math all @math in the support of @math and all sufficiently small @math Our main result is that if @math are sampled from a @math -Ahlfors regular measure on @math and @math denotes the @math -weight of the minimal spanning tree on @math [E_ (x_1, ,x_n )= e T (x_1, ,x_n ) |e|^ ] then [E_ (x_1, ,x_n ) n^ d- d ] with high probability as @math We also prove theorems about the asymptotic behavior of weighted sums defined in terms of higher-dimensional persistent homology. As an application, we exhibit hypotheses under which the fractal dimension of a measure can be computed from the persistent homology of i.i.d. samples from that space, in a manner similar to that proposed in the experimental work of (2018). We consider a linear stochastic bandit problem where the dimension @math of the unknown parameter @math is larger than the sampling budget @math . In such cases, it is in general impossible to derive sub-linear regret bounds since usual linear bandit algorithms have a regret in @math . In this paper we assume that @math is @math sparse, i.e. has at most @math non-zero components, and that the space of arms is the unit ball for the @math norm. We combine ideas from Compressed Sensing and Bandit Theory and derive algorithms with regret bounds in @math . We present a simple distributed @math -approximation algorithm for maximum weight independent set (MaxIS) in the @math model which completes in @math rounds, where @math is the maximum degree, @math is the number of rounds needed to compute a maximal independent set (MIS) on @math , and @math is the maximum weight of a node. Whether our algorithm is randomized or deterministic depends on the MIS algorithm used as a black-box. Plugging in the best known algorithm for MIS gives a randomized solution in @math rounds, where @math is the number of nodes. We also present a deterministic @math -round algorithm based on coloring. We then show how to use our MaxIS approximation algorithms to compute a @math -approximation for maximum weight matching without incurring any additional round penalty in the @math model. We use a known reduction for simulating algorithms on the line graph while incurring congestion, but we show our algorithm is part of a broad family of for which we describe a mechanism that allows the simulation to run in the @math model without an additional overhead. Next, we show that for maximum weight matching, relaxing the approximation factor to ( @math ) allows us to devise a distributed algorithm requiring @math rounds for any constant @math . For the unweighted case, we can even obtain a @math -approximation in this number of rounds. These algorithms are the first to achieve the provably optimal round complexity with respect to dependency on @math . Let @math be a cyclic @math -algebra of dimension @math with finite dimensional cohomology only in dimension one and two. By transfer theorem there exists a cyclic @math -algebra structure on the cohomology @math . The inner product plus the higher products of the cyclic @math -algebra defines a superpotential function @math on @math . We associate with an analytic Milnor fiber for the formal function @math and define the Euler characteristic of @math is to be the Euler characteristic of the 'et ale cohomology of the analytic Milnor fiber. In this paper we prove a Thom-Sebastiani type formula for the Euler characteristic of cyclic @math -algebras. As applications we prove the Joyce-Song formulas about the Behrend function identities for semi-Schur objects in the derived category of coherent sheaves over Calabi-Yau threefolds. A motivic Thom-Sebastiani type formula and a conjectural motivic Joyce-Song formulas for the motivic Milnor fiber of cyclic @math -algebras are also discussed. Many real-world problems face the dilemma of choosing best @math out of @math options at a given time instant. This setup can be modelled as combinatorial bandit which chooses @math out of @math arms at each time, with an aim to achieve an efficient tradeoff between exploration and exploitation. This is the first work for combinatorial bandit where the reward received can be a non-linear function of the chosen @math arms. The direct use of multi-armed bandit requires choosing among @math -choose- @math options making the state space large. In this paper, we present a novel algorithm which is computationally efficient and the storage is linear in @math . The proposed algorithm is a divide-and-conquer based strategy, that we call CMAB-SM. Further, the proposed algorithm achieves a regret bound of @math for a time horizon @math , which is sub-linear in all parameters @math , @math , and @math . The evaluation results on different reward functions and arm distribution functions show significantly improved performance as compared to standard multi-armed bandit approach with @math choices. We show a new way to round vector solutions of semidefinite programming (SDP) hierarchies into integral solutions, based on a connection between these hierarchies and the spectrum of the input graph. We demonstrate the utility of our method by providing a new SDP-hierarchy based algorithm for constraint satisfaction problems with 2-variable constraints (2-CSP's). More concretely, we show for every 2-CSP instance I a rounding algorithm for r rounds of the Lasserre SDP hierarchy for I that obtains an integral solution that is at most worse than the relaxation's value (normalized to lie in [0,1]), as long as r > k ( ) ( ) ;, where k is the alphabet size of I, @math , and @math denotes the number of eigenvalues larger than @math in the normalized adjacency matrix of the constraint graph of @math . In the case that @math is a instance, the threshold @math is only a polynomial in @math , and is independent of the alphabet size. Also in this case, we can give a non-trivial bound on the number of rounds for instance. In particular our result yields an SDP-hierarchy based algorithm that matches the performance of the recent subexponential algorithm of Arora, Barak and Steurer (FOCS 2010) in the worst case, but runs faster on a natural family of instances, thus further restricting the set of possible hard instances for Khot's Unique Games Conjecture. Our algorithm actually requires less than the @math constraints specified by the @math level of the Lasserre hierarchy, and in some cases @math rounds of our program can be evaluated in time @math . Let G=(V,E) be an n-vertex graph and Md a d-vertex graph, for some constant d. Is Md a subgraph of G? We consider this problem in a model where all n processes are connected to all other processes, and each message contains up to @math bits. A simple deterministic algorithm that requires @math communication rounds is presented. For the special case that Md is a triangle, we present a probabilistic algorithm that requires an expected @math rounds of communication, where t is the number of triangles in the graph, and @math with high probability. We also present deterministic algorithms that are specially suited for sparse graphs. In graphs of maximum degree Δ, we can test for arbitrary subgraphs of diameter D in @math rounds. For triangles, we devise an algorithm featuring a round complexity of @math , where A denotes the arboricity of G. The goal of (stable) sparse recovery is to recover a @math -sparse approximation @math of a vector @math from linear measurements of @math . Specifically, the goal is to recover @math such that @math for some constant @math and norm parameters @math and @math . It is known that, for @math or @math , this task can be accomplished using @math non-adaptive measurements CRT06:Stable-Signal and that this bound is tight DIPW, FPRU, PW11 . In this paper we show that if one is allowed to perform measurements that are adaptive , then the number of measurements can be considerably reduced. Specifically, for @math and @math we show A scheme with @math measurements that uses @math rounds. This is a significant improvement over the best possible non-adaptive bound. A scheme with @math measurements that uses two rounds. This improves over the best possible non-adaptive bound. To the best of our knowledge, these are the first results of this type. Many data analysis applications deal with large matrices and involve approximating the matrix using a small number of “components.” Typically, these components are linear combinations of the rows and columns of the matrix, and are thus difficult to interpret in terms of the original features of the input data. In this paper, we propose and study matrix approximations that are explicitly expressed in terms of a small number of columns and or rows of the data matrix, and thereby more amenable to interpretation in terms of the original data. Our main algorithmic results are two randomized algorithms which take as input an @math matrix @math and a rank parameter @math . In our first algorithm, @math is chosen, and we let @math , where @math is the Moore-Penrose generalized inverse of @math . In our second algorithm @math , @math , @math are chosen, and we let @math . ( @math and @math are matrices that consist of actual columns and rows, respectively, of @math , and @math is a generalized inverse of their intersection.) For each algorithm, we show that with probability at least @math , @math , where @math is the “best” rank- @math approximation provided by truncating the SVD of @math , and where @math is the Frobenius norm of the matrix @math . The number of columns of @math and rows of @math is a low-degree polynomial in @math , @math , and @math . Both the Numerical Linear Algebra community and the Theoretical Computer Science community have studied variants of these matrix decompositions over the last ten years. However, our two algorithms are the first polynomial time algorithms for such low-rank matrix approximations that come with relative-error guarantees; previously, in some cases, it was not even known whether such matrix decompositions exist. Both of our algorithms are simple and they take time of the order needed to approximately compute the top @math singular vectors of @math . The technical crux of our analysis is a novel, intuitive sampling method we introduce in this paper called “subspace sampling.” In subspace sampling, the sampling probabilities depend on the Euclidean norms of the rows of the top singular vectors. This allows us to obtain provable relative-error guarantees by deconvoluting “subspace” information and “size-of- @math ” information in the input matrix. This technique is likely to be useful for other matrix approximation and data analysis problems. We give an approximation algorithm for non-uniform sparsest cut with the following guarantee: For any @math , given cost and demand graphs with edge weights @math respectively, we can find a set @math with @math at most @math times the optimal non-uniform sparsest cut value, in time @math provided @math . Here @math is the @math 'th smallest generalized eigenvalue of the Laplacian matrices of cost and demand graphs; @math (resp. @math ) is the weight of edges crossing the @math cut in cost (resp. demand) graph and @math is the sparsity of the optimal cut. In words, we show that the non-uniform sparsest cut problem is easy when the generalized spectrum grows moderately fast. To the best of our knowledge, there were no results based on higher order spectra for non-uniform sparsest cut prior to this work. Even for uniform sparsest cut, the quantitative aspects of our result are somewhat stronger than previous methods. Similar results hold for other expansion measures like edge expansion, normalized cut, and conductance, with the @math 'th smallest eigenvalue of the normalized Laplacian playing the role of @math in the latter two cases. Our proof is based on an l1-embedding of vectors from a semi-definite program from the Lasserre hierarchy. The embedded vectors are then rounded to a cut using standard threshold rounding. We hope that the ideas connecting @math -embeddings to Lasserre SDPs will find other applications. Another aspect of the analysis is the adaptation of the column selection paradigm from our earlier work on rounding Lasserre SDPs [GS11] to pick a set of edges rather than vertices. This feature is important in order to extend the algorithms to non-uniform sparsest cut.