As explained above, by contrast to [1]}, [2]}, [3]}, [4]}, [5]}, [6]}, in this paper we first solve the channel allocation and access problem under general unknown restless Markovian channel model. Handling this model adds significant challenges in algorithm design and regret analysis. Due to the restless nature of the channels and potential reward loss due to transient effects as compared to steady state when switching channels, learning the Markovian channel characteristics requires that the channels be accessed in a judicious consecutive manner for a period of time. This is reflected in a novel algorithm design that guarantees efficient learning, as detailed next.
