An Overview for Markov Decision Processes in Queues and Networks <s> (5) MDPs of different server queues <s> In this paper we consider a queueing system having n exponential servers, each with its own queue and service rate. Customers arrive according to a Poisson process with rate λ, and upon arrival each customer must be routed to some server's queue. No jockeying amongst the queues is allowed and each server serves its queue according to a first-come-first-served discipline. Each server i , 1 ≦ i ≦ n , provides service with a state-dependent rate μ ( i ) ( k ), k = 0, 1, …. In addition, at every queue i , there is a deterministic holding cost which occurs at rate h ( i ) ( k ) while there are k customers at the queue. An admissible routing policy is a policy that assigns each arriving customer to one of the queues. A decision at time t may be randomized and dependent on the queue lengths and decisions till time t . An optimal routing policy is an admissible policy that minimizes the long-run average holding cost. In this study, we bound the optimal cost from below, by considering an ideal system, where each server optimally selects a given proportion of customers, irrespective of other servers' selections. From this ideal system we construct a class of admissible routing policies, the overflow routing class , that approximates the ideal situation for each server. Finally, we evaluate the policies and compare them to the lower bound. <s> BIB001 </s> An Overview for Markov Decision Processes in Queues and Networks <s> (5) MDPs of different server queues <s> This paper considers a queueing model with batch Poisson input and two heterogeneous servers, where the service times are exponentially distributed. The faster server is always on, but the slower server is only used when the queue length exceeds a certain level. Activating the slower server involves fixed set-up costs. Also there are linear operating costs and linear holding costs. The class of two-level hysteretic control rules is considered. Rather than proving the overall average cost optimality of a hysteretic rule, the purpose of this paper is to develop a tailor-made policy iteration algorithm for computing the optimal switch-on and switch-off levels for the slower server. An embedding method is used that is generally applicable to structured Markovian control problems with an infinitely large state space. <s> BIB002 </s> An Overview for Markov Decision Processes in Queues and Networks <s> (5) MDPs of different server queues <s> A multi-server controllable queueing system with heterogeneous servers is considered. Several monotonicity properties of optimal policies for such a system are proved. <s> BIB003 </s> An Overview for Markov Decision Processes in Queues and Networks <s> (5) MDPs of different server queues <s> An optimal policy to minimize the queue length in a multi-server controllable queueing system with heterogeneous servers has a threshold property, and it uses the fastest server if necessary (see [8] and [17]). This study gives a numerical description of optimal policies that minimize the operational cost for such a system. <s> BIB004 </s> An Overview for Markov Decision Processes in Queues and Networks <s> (5) MDPs of different server queues <s> This paper deals with optimal control points of M(t)/M/c/c queues with periodic arrival rates and two levels of the number of servers. We use the results of this model to build a Markov decision process (MDP). The problem arose from a case study in the Kelowna General Hospital (KGH). The KGH uses surge beds when the emergency room is overcrowded which results in having two levels for the number of the beds. The objective is to minimize a cost function. The findings of this work are not limited to the healthcare; They may be used in any stochastic system with fluctuation in arrival rates and/or two levels of the number of servers, i.e., call centers, transportation, and internet services. We model the situation and define a cost function which needs to be minimized. In order to find the cost function we need transient solutions of the M(t)/M/c/c queue. We modify the fourth-order Runge–Kutta to calculate the transient solutions and we obtain better solutions than the existing Runge–Kutta method. We show that the periodic variation of arrival rates makes the control policies time-dependent and periodic. We also study how fast the policies converge to a periodic pattern and obtain a criterion for independence of policies in two sequential cycles. <s> BIB005
Rosberg and Kermani BIB001 , Nobel and Tijms BIB002 , Rykov BIB003 , Rykov and Efrosinin BIB004 and Tirdad et al. BIB005 .