A Survey of Client-Controlled HetNets for 5G <s> G. PROBABILISTIC ANALYSIS <s> The architecture for the Beyond 3rd Generation (B3G) or 4th Generation (4G) wireless networks aims at integrating various heterogeneous wireless access networks. One of the major design issues is the support of vertical handoff. Vertical handoff occurs when a mobile terminal switches from one network to another (e.g., from wireless local area network to code-division multiple-access 1x radio transmission technology). The objective of this paper is to determine the conditions under which vertical handoff should be performed. The problem is formulated as a Markov decision process with the objective of maximizing the total expected reward per connection. The network resources that are utilized by the connection are captured by a link reward function. A signaling cost is used to model the signaling and processing load incurred on the network when vertical handoff is performed. The value iteration algorithm is used to compute a stationary deterministic policy. For performance evaluation, voice and data applications are considered. The numerical results show that our proposed scheme performs better than other vertical handoff decision algorithms, namely, simple additive weighting, the technique for order preference by similarity to ideal solution, and Grey relational analysis. <s> BIB001 </s> A Survey of Client-Controlled HetNets for 5G <s> G. PROBABILISTIC ANALYSIS <s> This paper addresses the problem of radio access technology (RAT) selection in heterogeneous multi-access/multi-service scenarios. For such purpose, a Markov model is proposed to compare the performance of various RAT selection policies within these scenarios. The novelty of the approach resides in the embedded definition of the aforementioned RAT selection policies within the Markov chain. In addition, the model also considers the constraints imposed by those users with terminals that only support a subset of all the available RATs (i.e. multi-mode terminal capabilities). Furthermore, several performance metrics may be measured to evaluate the behaviour of the proposed RAT selection policies under varying offered traffic conditions. In order to illustrate the validation and suitability of the proposed model, some examples of operative radio access networks are provided, including the GSM/EDGE Radio Access Network (GERAN) and the UMTS Radio Access Network (UTRAN), as well as several service-based, load-balancing and terminal-driven RAT selection strategies. The flexibility exhibited by the presented model enables to extend these RAT selection policies to others responding to diverse criteria. The model is successfully validated by means of comparing the Markov model results with those of system-level simulations. <s> BIB002 </s> A Survey of Client-Controlled HetNets for 5G <s> G. PROBABILISTIC ANALYSIS <s> In this paper, we deal with a cost minimization problem for a multihomed mobile terminal that downloads and plays a video-on-demand (VoD) stream. The cost consists of the user's dissatisfaction due to playback disruptions and communication cost for downloading the VoD stream. There are three components in our approach: parameter estimation, threshold adjustment, and threshold compensation. Since we do not assume any a priori knowledge about underlying random variables, necessary parameter values are estimated online. Using the resultant estimates, we formulate the problem as a Markov decision process (MDP) problem considering as if the random variables are exponentially distributed. To solve the MDP problem efficiently, we propose a threshold adjustment algorithm that exploits some structural properties of any optimal solution that are specific to our problem. Finally, to handle the cases where the random variables are not exponentially distributed, we propose a threshold compensation algorithm to compensate for the modeling error. Through extensive simulations, we compare the performance of our scheme with those of static threshold schemes. <s> BIB003 </s> A Survey of Client-Controlled HetNets for 5G <s> G. PROBABILISTIC ANALYSIS <s> The recently proposed 3-Tier access model for Whitespace by the Federal Communications Commission (FCC) mandates certain classes of devices to share frequency bands in space and time. These devices are envisioned to be a heterogeneous mixture of licensed (Tier-1 and Tier-2) and unlicensed, opportunistic devices (Tier-3). The hierarchy in accessing the channel calls for superior adaptation of Tier-3 devices with varying spectral opportunity. While policies are being ratified for efficient sharing, it also calls for redesigning many common applications to adapt to this novel paradigm. In this paper, we focus on the ever-increasing demand for video streaming and present a methodology suitable for Tier-3 devices in the shared access model. Our analysis begins with a stress test of commonly adopted video streaming methods under the new sharing model. This is followed by the design of a robust MDP-based solution that proactively adapts to fast-varying channel conditions, providing better user quality of experience when compared to existing solutions, such as MPEG-DASH. We evaluate our solution on an experimental testbed and find that our MDP-based algorithm outperforms DASH, and partial information of Tier-2 dynamics improves video quality. <s> BIB004
Markov Decision Processes (MDP) have also been used to model the HetNets RAT selection problem at the client side BIB001 - BIB004 . Clients may internally store empirical knowledge on the rewards (e.g. throughputs) obtained in each state (e.g. BS) that are accessible, as well as their transition probabilities. Every time period, the client must make a decision on which action to take-which RAT or BS to associate with, in order to maximize some expected total reward. [80] develop a MDP model for vertical handoff between different types of RATs, that considers the link reward for connecting to a BS, the signaling load and processing load when the handoff is performed. The algorithm relies solely on implicit feedback from the network in the link reward obtained after connecting to the BS, and shows improvement over several other algorithms for vertical handoff. Markov chains can also be used to explicitly model ongoing voice and data sessions on individual RATs BIB002 . With the assumption that calls and data sessions begin and end sequentially (they arrive/depart individually) and that the total traffic offered to both networks are known, this work develops a 4D Markov Chain to model two TDMA and WCMDA networks in order to simulate the performance of several RAT selection policies. Application-specific models can also be used, such as for Video-on-Demand BIB003 . In this work, multihomed clients optimize their choice of RATs for the minimization of video playback disruption costs and the communication cost of receiving a video chunk over a given RAT. The MDP is used to determine which RAT to send a chunk request to at a given time, and the resulting adaptive ATAC policy is shown to have lower costs than policies with static thresholds for request allocation.