An Overview for Markov Decision Processes in Queues and Networks <s> MDPs of Single-Server Queues <s> We consider an M/M/1 queue with two types of customers: priority customers and regular customers. They arrive at the service facility according to two independent Poisson streams and form a single queue according to the order in which they arrive. The two types of customers are distinguished by the holding costs charged per unit time that each of them resides in the queue. The server can either serve customers according to the order in which they arrive or pay a fixed fee R and promote a priority customer, bypassing the customers ahead of him. The server selects the customers to be served so as to minimize the expected average cost per unit of time of operating the system. We show that whenever the number of regular customers bypassed in a promotion, times the expected holding costs per priority customer per service period, is greater than or equal to R, promotion is strictly optimal. Moreover, for each state there exists a value of R—with R exceeding the number of regular customers bypassed in a promotio... <s> BIB001 </s> An Overview for Markov Decision Processes in Queues and Networks <s> MDPs of Single-Server Queues <s> A method is developed for determining the optimal policy for entry of customers from many independent classes of Poisson arrivals to a first-come, first-serve (for customers admitted to the queue) single-server queue with exponential service times. The solution technique utilizes a semi-Markov formulation or the decision problem. <s> BIB002
In this section, we provide an overview for MDPs of single-server queues, including the M/M/1 queues, the M/M/c queues, the M/G/1 queues, the GI/M/1 queues and others. In the early research on MDPs of queues and networks, the single-server queues have been an active topic for many years. (1) MDPs of M/M/1 queues Kofman and Lippman BIB001 , Rue and Rosenshine BIB002  