This paper considers a problem where multiple users make repeated decisions based on their own observed events. The events and decisions at each time step determine the values of a utility function and a collection of penalty functions. The goal is to make distributed decisions over time to maximize time average utility subject to time average constraints on the penalties. An example is a collection of power constrained sensor nodes that repeatedly report their own observations to a fusion center. Maximum time average utility is fundamentally reduced because users do not know the events observed by others. Optimality is characterized for this distributed context. It is shown that optimality is achieved by correlating user decisions through a commonly known pseudorandom sequence. An optimal algorithm is developed that chooses pure strategies at each time step based on a set of time-varying weights.
The framework of partial knowledge at each user is similar in spirit to a @cite_9 @cite_14 . There, the goal is to design competitive strategies that lead to a Nash equilibrium. This is significantly different from the goal of the current paper. The current paper is not concerned with competition or equilibrium. Rather, there is a single utility function that all users desire to maximize. Distributed algorithms are developed to maximize time average utility subject to time average penalty constraints.
@cite_9 @cite_14
2109100253 1528676759
This paper considers a time-varying game with N players. Every time slot, players observe their own random events and then take a control action. The events and control actions affect the individual utilities earned by each player. The goal is to maximize a concave function of time average utilities subject to equilibrium constraints. Specifically, participating players are provided access to a common source of randomness from which they can optimally correlate their decisions. The equilibrium constraints incentivize participation by ensuring that players cannot earn more utility if they choose not to participate. This form of equilibrium is similar to the notions of Nash equilibrium and correlated equilibrium, but is simpler to attain. A Lyapunov method is developed that solves the problem in an online max-weight fashion by selecting actions based on a set of time-varying weights. The algorithm does not require knowledge of the event probabilities. A similar method can be used to compute a standard correlated equilibrium, albeit with increased complexity. A game-theoretic model for studying power control in multicarrier code-division multiple-access systems is proposed. Power control is modeled as a noncooperative game in which each user decides how much power to transmit over each carrier to maximize its own utility. The utility function considered here measures the number of reliable bits transmitted over all the carriers per joule of energy consumed and is particularly suitable for networks where energy efficiency is important. The multidimensional nature of users' strategies and the nonquasi-concavity of the utility function make the multicarrier problem much more challenging than the single-carrier or throughput-based-utility case. It is shown that, for all linear receivers including the matched filter, the decorrelator, and the minimum-mean-square-error detector, a user's utility is maximized when the user transmits only on its "best" carrier. This is the carrier that requires the least amount of power to achieve a particular target signal-to-interference-plus-noise ratio at the output of the receiver. The existence and uniqueness of Nash equilibrium for the proposed power control game are studied. In particular, conditions are given that must be satisfied by the channel gains for a Nash equilibrium to exist, and the distribution of the users among the carriers at equilibrium is characterized. In addition, an iterative and distributed algorithm for reaching the equilibrium (when it exists) is presented. It is shown that the proposed approach results in significant improvements in the total utility achieved at equilibrium compared with a single-carrier system and also to a multicarrier system in which each user maximizes its utility over each carrier independently Dynamic spectrum access can significantly improve the spectrum utilization. For wireless service providers, the emergence of dynamic spectrum access brings new opportunities and challenges. The flexible spectrum acquisition gives a particular provider the chance to easily adapt its system capacity to fit end users' demand. However, the competition among several providers for both spectrum and end users complicates the situation. In this paper, we propose a general three-layer spectrum market model for the future dynamic spectrum access system, in which the interaction among spectrum holder, wireless service providers and end users are considered. We study a duopoly situation, where two wireless service providers participate in bandwidth competition in spectrum purchasing and price competition to attract end users, with the aim of maximizing their own profit. We believe we are the first one to explicitly study the relation of these two competitions in dynamic spectrum market. We formulate the wireless service providers' competition as a non-cooperative two-stage game. We first analyze the static game when full information is available for providers. Under general assumptions about the price and demand functions, a unique pure Nash equilibrium is identified as the outcome of the game, which shows the stability of the market. We further evaluate the market efficiency of the equilibrium in a symmetric case, and show that the gap with the social optimal is bounded within a small constant ratio. When the market information is limited, we provide myopically optimal adjustment algorithms for the providers. With such strategies, short term price updating converges to the Nash equilibrium of the given subgame, while long term bandwidth updating converges to a point close to the Nash equilibrium of the full game. We study in this paper a noncooperative approach for sharing resources of a common pool among users, wherein each user strives to maximize its own utility. The optimality notion is then a Nash equilibrium. First, we present a general framework of systems wherein a Nash equilibrium is Pareto inefficient, which are similar to the 'tragedy of the commons' in economics. As examples that fit in the above framework, we consider noncooperative flow-control problems in communication networks where each user decides its throughput to optimize its own utility. As such a utility, we first consider the power which is defined as the throughput divided by the expected end-to-end packet delay, and then consider another utility of additive costs. For both utilities, we establish the non-efficiency of the Nash equilibria. Power allocation across users in two adjacent cells is studied for a code-division multiple access (CDMA) data service. The forward link is considered and cells are modeled as one-dimensional with uniformly distributed users and orthogonal signatures within each cell. Each user is assumed to have a utility function that describes the user's received utility, or willingness to pay, for a received signal-to-interference-plus-noise ratio (SINR). The objective is to allocate the transmitted power to maximize the total utility summed over all users subject to power constraints in each cell. It is first shown that this optimization can be achieved by a pricing scheme in which each base station announces a price per unit transmitted power to the users, and each user requests power to maximize individual surplus (utility minus cost). Setting prices to maximize total revenue over both cells is also considered, and it is shown that, in general, the solution is different from the one obtained by maximizing total utility. Conditions are given for which independent optimization in each cell, which leads to a Nash equilibrium (NE), is globally optimal. It is shown that, in general, coordination between the two cells is needed to achieve the maximum utility or revenue. We consider a multi-cell wireless network with a large number of users. Each user selfishly chooses the Base Station (BS) that gives it the best throughput (utility), and each BS allocates its resource by some simple scheduling policy. First we consider two cases: (1) BS allocates the same time to its users; (2) BS allocates the same throughput to its users. It turns out that, combined with users' selfish behavior, case (1) results in a single Nash Equilibrium (NE), which achieves system-wide Proportional Fairness. On the other hand, case (2) results in many possible Nash Equilibria, some of which are very inefficient. Next, we extend (1) to the case where the users have general concave utility functions. It is shown that the if each BS performs intra- cell optimization, the total utility of all users is maximized at NE. This suggests that under our model, the task of joining the ";correct"; BS can be left to individual users, leading to a distributed solution. In this paper, we study a make-to-stock manufacturing system where two firms compete through dynamic pricing and inventory control. Our goal is to address competition (in particular a duopoly setting) together with the presence of demand uncertainty. We consider a dynamic setting where multiple products share production capacity. We introduce a demand-based fluid model where the demand is a linear function of the price of the supplier and of her competitor, the inventory and production costs are quadratic, and all coefficients are time dependent. A key part of the model is that no backorders are allowed and the strategy of a supplier depends on her competitor's strategy. First, we reformulate the robust problem as a fluid model of similar form to the deterministic one and show existence of a Nash equilibrium in continuous time. We then discuss issues of uniqueness and address how to compute a particular Nash equilibrium, i.e., the normalized Nash equilibrium. We consider a wireless collision channel, shared by a finite number of users who transmit to a common base station. Users are self-optimizing, and each wishes to minimize its average transmission rate (or power investment), subject to minimum- throughput demand. The channel quality between each user and the base station is time-varying, and partially observed by the user in the form of channel state information (CSI) signals. We assume that each user can transmit at a fixed power level and that its transmission decision at each time slot is stationary in the sense that it can depend only on the current CSI. We are interested in properties of the Nash equilibrium of the resulting game between users. We define the feasible region of user's throughput demands, and show that when the demands are within this region, there exist exactly two Nash equilibrium points, with one strictly better than the other (in terms of invested power) for all users. We further provide some lower bounds on the channel capacity that can be obtained, both in the symmetric and non-symmetric case. Finally, we show that a simple greedy mechanism converges to the best equilibrium point without requiring any coordination between the users. The existence of Nash equilibria in noncooperative flow control in a general product-form network shared by K users is investigated. The performance objective of each user is to maximize its average throughput subject to an upper bound on its average time-delay. Previous attempts to study existence of equilibria for this flow control model were not successful, partly because the time-delay constraints couple the strategy spaces of the individual users in a way that does not allow the application of standard equilibrium existence theorems from the game theory literature. To overcome this difficulty, a more general approach to study the existence of Nash equilibria for decentralized control schemes is introduced. This approach is based on directly proving the existence of a fixed point of the best reply correspondence of the underlying game. For the investigated flow control model, the best reply correspondence is shown to be a function, implicitly defined by means of K interdependent linear programs. Employing an appropriate definition for continuity of the set of optimal solutions of parameterized linear programs, it is shown that, under appropriate conditions, the best reply function is continuous. Brouwer's theorem implies, then, that the best reply function has a fixed point. We address the problem of spectrum pricing in a cognitive radio network where multiple primary service providers compete with each other to offer spectrum access opportunities to the secondary users. By using an equilibrium pricing scheme, each of the primary service providers aims to maximize its profit under quality of service (QoS) constraint for primary users. We formulate this situation as an oligopoly market consisting of a few firms and a consumer. The QoS degradation of the primary services is considered as the cost in offering spectrum access to the secondary users. For the secondary users, we adopt a utility function to obtain the demand function. With a Bertrand game model, we analyze the impacts of several system parameters such as spectrum substitutability and channel quality on the Nash equilibrium (i.e., equilibrium pricing adopted by the primary services). We present distributed algorithms to obtain the solution for this dynamic game. The stability of the proposed dynamic game algorithms in terms of convergence to the Nash equilibrium is studied. However, the Nash equilibrium is not efficient in the sense that the total profit of the primary service providers is not maximized. An optimal solution to gain the highest total profit can be obtained. A collusion can be established among the primary services so that they gain higher profit than that for the Nash equilibrium. However, since one or more of the primary service providers may deviate from the optimal solution, a punishment mechanism may be applied to the deviating primary service provider. A repeated game among primary service providers is formulated to show that the collusion can be maintained if all of the primary service providers are aware of this punishment mechanism, and therefore, properly weight their profits to be obtained in the future. We study the demand side management (DSM) problem when customers are equipped with energy storage devices. Two games are discussed: the first is a non-cooperative one played between the residential energy consumers, while the second is a Stackelberg game played between the utility provider and the energy consumers. We introduce a new cost function applicable to the case of users selling back stored energy. The non-cooperative energy consumption game is played between users who schedule their energy use to minimize energy cost. The game is shown to have a unique Nash equilibrium, that is also the global system optimal point. In the Stackelberg game, the utility provider sets the prices to maximize its profit knowing that users will respond by minimizing their cost. We provide existence and uniqueness results for the Stackelberg equilibrium. The Stackelberg game is shown to be the general case of the minimum Peak-to-Average power ratio (PAR) problem. Two algorithms, centralized and distributed, are presented to solve the Stackelberg game. We present results that elucidate the interplay between storage capacity, energy requirements, number of users and system performance measured in total cost and peak-to-average power ratio (PAR). We consider the following class of problems. The value of an outcome to a society is measured via a submodular utility function (submodularity has a natural economic interpretation: decreasing marginal utility). Decisions, however, are controlled by non-cooperative agents who seek to maximise their own private utility. We present, under basic assumptions, guarantees on the social performance of Nash equilibria. For submodular utility functions, any Nash equilibrium gives an expected social utility within a factor 2 of optimal, subject to a function-dependent additive term. For non-decreasing, submodular utility functions, any Nash equilibrium gives an expected social utility within a factor 1+ spl delta of optimal, where 0 spl les spl delta spl les 1 is a number based upon discrete curvature of the function. A condition under which all sets of social and private utility functions induce pure strategy Nash equilibria is presented. The case in which agents themselves make use of approximation algorithms in decision making is discussed and performance guarantees given. Finally we present specific problems that fall into our framework. These include competitive versions of the facility location problem and k-median problem, a maximisation version of the traffic routing problem studied by Roughgarden and Tardos (2000), and multiple-item auctions. We introduce take-it-or-leave-it auctions (TLAs) as an allocation mechanism that allows buyers to retain much of their private valuation information, yet generates close-to-optimal expected utility for the seller. We show that if each buyer receives at most one offer, each buyer's dominant strategy is to act truthfully. In more general TLAs, the buyers' optimal strategies are more intricate, and we derive the perfect Bayesian equilibrium for the game. We develop algorithms for finding the equilibrium and also for optimizing the offers so as to maximize the seller's expected utility. In several example settings we show that the seller's expected utility already is close to optimal for a small number of offers. As the number of buyers increases, the seller's expected utility increases, and becomes increasingly (but not monotonically) more competitive with Myerson's expected utility maximizing auction. Demand Response Management (DRM) is a key component of the future smart grid that helps to reduce power peak load and variation. Different from most existing studies that focus on the scenario with a single utility company, this paper studies DRM with multiple utility companies. First, the interaction between utility companies and residential users is modeled as a two-level game. That is, the competition among the utility companies is formulated as a non-cooperative game, while the interaction among the residential users is formulated as an evolutionary game. Then, we prove that the proposed strategies are able to make both games converge to their own equilibrium. In addtion, the strategies for the utility companies and the residential users are implemented by distributed algorithms. Illustrative examples show that the proposed scheme is able to significantly reduce peak load and demand variation. "Cognitive radio" is an emerging technique to improve the utilization of radio frequency spectrum in wireless networks. In this paper, we consider the problem of spectrum sharing among a primary user and multiple secondary users. We formulate this problem as an oligopoly market competition and use a noncooperative game to obtain the spectrum allocation for secondary users. Nash equilibrium is considered as the solution of this game. We first present the formulation of a static game for the case where all secondary users have the current information of the adopted strategies and the payoff of each other. However, this assumption may not be realistic in some cognitive radio systems. Therefore, we consider the case of bounded rationality in which the secondary users gradually and iteratively adjust their strategies based on the observations on their previous strategies. The speed of adjustment of the strategies is controlled by the learning rate. The stability condition of the dynamic behavior for this spectrum sharing scheme is investigated. The numerical results reveal the dynamics of distributed dynamic adaptation of spectrum sharing strategies. Market-based mechanisms offer promising approaches for spectrum access in cognitive radio networks. In this paper, we focus on two market models, one with a monopoly primary user (PU) market and the other with a multiple PU market, where each PU sells its temporarily unused spectrum to secondary users (SUs). We propose a pricing-based spectrum trading mechanism that enables SUs to contend for channel usage by random access, in a distributed manner, which naturally mitigates the complexity and time overhead associated with centralized scheduling. For the monopoly PU market model, we first consider SUs contending via slotted Aloha. The revenue maximization problems here are nonconvex. We first characterize the Pareto optimal region, and then obtain a Pareto optimal solution that maximizes the SUs' throughput subject to the SUs' budget constraints. To mitigate the spectrum underutilization due to the “price of contention,” we revisit the problem where SUs contend via CSMA, and show that spectrum utilization is enhanced, resulting in higher revenue. When the PU's unused spectrum is a control parameter, we study further the tradeoff between the PU's utility and its revenue. For the multiple PU market model, we cast the competition among PUs as a three-stage Stackelberg game, where each SU selects a PU's channel to maximize its throughput. We characterize the Nash equilibria, in terms of access prices and the spectrum offered to SUs. Our findings reveal that the number of equilibria exhibits a phase transition phenomenon, in the sense that when the number of PUs is greater than a threshold, there exist infinitely many equilibria; otherwise, there exists a unique Nash equilibrium, where the access prices and spectrum opportunities are determined by the budgets elasticity of SUs and the utility level of PUs. Most of the existing demand-side management programs focus primarily on the interactions between a utility company and its customers users. In this paper, we present an autonomous and distributed demand-side energy management system among users that takes advantage of a two-way digital communication infrastructure which is envisioned in the future smart grid. We use game theory and formulate an energy consumption scheduling game, where the players are the users and their strategies are the daily schedules of their household appliances and loads. It is assumed that the utility company can adopt adequate pricing tariffs that differentiate the energy usage in time and level. We show that for a common scenario, with a single utility company serving multiple customers, the global optimal performance in terms of minimizing the energy costs is achieved at the Nash equilibrium of the formulated energy consumption scheduling game. The proposed distributed demand-side energy management strategy requires each user to simply apply its best response strategy to the current total load and tariffs in the power distribution system. The users can maintain privacy and do not need to reveal the details on their energy consumption schedules to other users. We also show that users will have the incentives to participate in the energy consumption scheduling game and subscribing to such services. Simulation results confirm that the proposed approach can reduce the peak-to-average ratio of the total energy demand, the total energy costs, as well as each user's individual daily electricity charges. We consider the problem of dynamic spectrum access for network utility maximization in multichannel wireless networks. The shared bandwidth is divided into @math orthogonal channels. In the beginning of each time slot, each user selects a channel and transmits a packet with a certain transmission probability. After each time slot, each user that has transmitted a packet receives a local observation indicating whether its packet was successfully delivered or not (i.e., ACK signal). The objective is a multi-user strategy for accessing the spectrum that maximizes a certain network utility in a distributed manner without online coordination or message exchanges between users. Obtaining an optimal solution for the spectrum access problem is computationally expensive, in general, due to the large-state space and partial observability of the states. To tackle this problem, we develop a novel distributed dynamic spectrum access algorithm based on deep multi-user reinforcement leaning. Specifically, at each time slot, each user maps its current state to the spectrum access actions based on a trained deep-Q network used to maximize the objective function. Game theoretic analysis of the system dynamics is developed for establishing design principles for the implementation of the algorithm. The experimental results demonstrate the strong performance of the algorithm. Demand Response Management (DRM) is a key component in the smart grid to effectively reduce power generation costs and user bills. However, it has been an open issue to address the DRM problem in a network of multiple utility companies and consumers where every entity is concerned about maximizing its own benefit. In this paper, we propose a Stackelberg game between utility companies and end-users to maximize the revenue of each utility company and the payoff of each user. We derive analytical results for the Stackelberg equilibrium of the game and prove that a unique solution exists. We develop a distributed algorithm which converges to the equilibrium with only local information available for both utility companies and end-users. Though DRM helps to facilitate the reliability of power supply, the smart grid can be succeptible to privacy and security issues because of communication links between the utility companies and the consumers. We study the impact of an attacker who can manipulate the price information from the utility companies. We also propose a scheme based on the concept of shared reserve power to improve the grid reliability and ensure its dependability. A mixed manna contains goods (that everyone likes), bads (that everyone dislikes), as well as items that are goods to some agents, but bads or satiated to others. If all items are goods and utility functions are homothetic, concave (and monotone), the Competitive Equilibrium with Equal Incomes maximizes the Nash product of utilities: hence it is welfarist (determined utility-wise by the feasible set of pro les), single-valued and easy to compute. We generalize the Gale-Eisenberg Theorem to a mixed manna. The Competitive division is still welfarist and related to the product of utilities or disutilities. If the zero utility pro le (before any manna) is Pareto dominated, the competitive pro le is unique and still maximizes the product of utilities. If the zero pro le is unfeasible, the competitive pro les are the critical points of the product of disutilities on the eciency frontier, and multiplicity is pervasive. In particular the task of dividing a mixed manna is either good news for everyone, or bad news for everyone. We re ne our results in the practically important case of linear preferences, where the axiomatic comparison between the division of goods and that of bads is especially sharp. When we divide goods and the manna improves, everyone weakly bene ts under the competitive rule; but no reasonable rule to divide bads can be similarly Resource Monotonic. Also, the much larger set of Non Envious and Ecient divisions of bads can be disconnected so that it will admit no continuous selection.