Another set of related work on multi-user channel allocation has approached it from the angle of game theoretic and congestion control ( [1]}, [2]}, [3]}, [4]}, [5]}, [6]}, [7]}, [8]}, [9]}, [10]}, [11]} and references therein), hidden channel states[12]}, and graph coloring ([13]}, [14]}, [15]}, [16]} and references therein).
The game theoretic aspects of the problem have been investigated from both non-cooperative (i.e., each user aims at maximizing an individual utility) [2]}, [3]}, [8]}, [9]}, [21]}, and cooperative (i.e., each user aims at maximizing a system-wide global utility) [1]}, [23]}, [10]}, [25]} settings. Model-free learning strategies were developed in [26]}, [27]} for orthogonal channels, compact models [28]}, and multiple access channel strategies were developed in
[29]}, [30]}. Graph coloring formulations have dealt with modeling the spectrum access problem as a graph coloring problem, in which users and channels are represented by vertices and colors, respectively (see [13]}, [14]}, [16]}, [15]} and references therein for related studies). Finally, none of these studies have considered the problem of achieving provable stable strategies in the learning context under unknown restless Markovian dynamics, as considered in this paper.
