Content caching at the edge nodes is a promising technique to reduce the data traffic in next-generation wireless networks. Inspired by the success of Deep Reinforcement Learning (DRL) in solving complicated control problems, this work presents a DRL-based framework with Wolpertinger architecture for content caching at the base station. The proposed framework is aimed at maximizing the long-term cache hit rate, and it requires no knowledge of the content popularity distribution. To evaluate the proposed framework, we compare the performance with other caching algorithms, including Least Recently Used (LRU), Least Frequently Used (LFU), and First-In First-Out (FIFO) caching strategies. Meanwhile, since the Wolpertinger architecture can effectively limit the action space size, we also compare the performance with Deep Q-Network to identify the impact of dropping a portion of the actions. Our results show that the proposed framework can achieve improved short-term cache hit rate and improved and stable long-term cache hit rate in comparison with LRU, LFU, and FIFO schemes. Additionally, the performance is shown to be competitive in comparison to Deep Q-learning, while the proposed framework can provide significant savings in runtime.
Content caching has attracted much interest recently. For instance, the authors in @cite_4 analyzed proactive content caching from the aspect of big data analytics and demonstrated that with limited cache size, proactive caching can provide $100 As seen in previous studies, content popularity distribution is always the key to solve the content caching problem. Considering the large scale of contents and the changing popularities, we note that deep reinforcement learning is a particularly attractive strategy to tackle this problem. In @cite_6 , @cite_3 and @cite_5 , reinforcement learning methods are addressed. For instance, the proposed reinforcement learning agent in @cite_6 is a typical multi-step return actor-critic architecture. The deep deterministic policy gradient proposed in @cite_3 describes how the network is to be updated. And the Wolpertinger policy proposed in @cite_5 provides a solution for large discrete action spaces.
@cite_5 @cite_3 @cite_4 @cite_6
2963805280 2963008907 2078905979 2278115387
Content caching at the edge nodes is a promising technique to reduce the data traffic in next-generation wireless networks. Inspired by the success of Deep Reinforcement Learning (DRL) in solving complicated control problems, this work presents a DRL-based framework with Wolpertinger architecture for content caching at the base station. The proposed framework is aimed at maximizing the long-term cache hit rate, and it requires no knowledge of the content popularity distribution. To evaluate the proposed framework, we compare the performance with other caching algorithms, including Least Recently Used (LRU), Least Frequently Used (LFU), and First-In First-Out (FIFO) caching strategies. Meanwhile, since the Wolpertinger architecture can effectively limit the action space size, we also compare the performance with Deep Q-Network to identify the impact of dropping a portion of the actions. Our results show that the proposed framework can achieve improved short-term cache hit rate and improved and stable long-term cache hit rate in comparison with LRU, LFU, and FIFO schemes. Additionally, the performance is shown to be competitive in comparison to Deep Q-learning, while the proposed framework can provide significant savings in runtime. Mobile cellular networks are becoming increasingly complex to manage while classical deployment optimization techniques and current solutions (i.e., cell densification, acquiring more spec­trum, etc.) are cost-ineffective and thus seen as stopgaps. This calls for development of novel approaches that leverage recent advances in storage memory, context-awareness, edge cloud computing, and falls into framework of big data. However, the big data by itself is yet another complex phenomena to handle and comes with its notorious 4V: Velocity, voracity, volume, and variety. In this work, we address these issues in optimization of 5G wireless networks via the notion of proactive caching at the base stations. In particular, we investigate the gains of proactive caching in terms of backhaul of-floadings and request satisfactions, while tackling the large-amount of available data for content popularity estimation. In order to estimate the content popularity, we first collect users' mobile traffic data from a Turkish telecom operator from several base stations in hours of time interval. Then, an analysis is carried out locally on a big data platform and the gains of proactive caching at the base stations are investigated via numerical simulations. It turns out that several gains are possible depending on the level of available information and storage size. For instance, with 10 of content ratings and 15.4 Gbyte of storage size (87 of total catalog size), proactive caching achieves 100 of request satisfaction and offloads 98 of the backhaul when considering 16 base stations. We introduce the concept of resource management for in-network caching environments. We argue that in Information-Centric Networking environments, deterministically caching content messages at predefined places along the content delivery path results in unfair and inefficient content multiplexing between different content flows, as well as in significant caching redundancy. Instead, allocating resources along the path according to content flow characteristics results in better use of network resources and therefore, higher overall performance. The design principles of our proposed in-network caching scheme, which we call ProbCache, target these two outcomes, namely reduction of caching redundancy and fair content flow multiplexing along the delivery path. In particular, ProbCache approximates the caching capability of a path and caches contents probabilistically to: 1) leave caching space for other flows sharing (part of) the same path, and 2) fairly multiplex contents in caches along the path from the server to the client. We elaborate on the content multiplexing fairness of ProbCache and find that it sometimes behaves in favor of content flows connected far away from the source, that is, it gives higher priority to flows travelling longer paths, leaving little space to shorter-path flows. We introduce an enhanced version of the main algorithm that guarantees fair behavior to all participating content flows. We evaluate the proposed schemes in both homogeneous and heterogeneous cache size environments and formulate a framework for resource allocation in in-network caching environments. The proposed probabilistic approach to in-network caching exhibits ideal performance both in terms of network resource utilization and in terms of resource allocation fairness among competing content flows. Finally, and in contrast to the expected behavior, we find that the efficient design of ProbCache results in fast convergence to caching of popular content items. While 5G wireless networks are expected to handle ever growing data avalanche, classical deployment optimization approaches such as hyper- dense deployment of base stations or having more bandwidth are cost-inefficient, therefore seen as stopgaps. In this regard, context-aware approaches which exploits human predictability, recent advances in storage, edge cloud computing and big data analytics are needed. In this article, we approach to this problem from a proactive caching perspective where gains of cache-enabled base stations in 5G wireless are studied. In particular, huge amount of real data from a telecom operator in Turkey is collected processed on a big data platform, and an analysis is carried out for content popularity estimation for caching, aiming to improve users' experience in terms of request satisfactions and offload the backhaul. Subsequently, with this mobile traffic data collected from many base stations within several hours of time interval and estimation of content popularity via machine learning tools, we investigate the gains of the proactive caching via numerical simulations. The results show that proactive caching fulfils 100 of user request satisfaction and offloads 98 of the backhaul, in a setting of 16 base station with 15.4 Gbyte of storage size (87 of the total catalog size) and 10 of content ratings.