In this paper we analyze and extend the neural network based associative memory proposed by Gripon and Berrou. This associative memory resembles the celebrated Will- shaw model with an added partite cluster structure. In the literature, two retrieving schemes have been proposed for the network dynamics, namely SUM-OF-SUM and SUM-OF-MAX. They both offer considerably better performance than Willshaw and Hopfield networks, when comparable retrieval scenarios are considered. Former discussions and experiments concentrate on the erasure scenario, where a partial message is used as a probe to the network, in the hope of retrieving the full message. In this regard, SUM-OF-MAX outperforms SUM-OF-SUM in terms of retrieval rate by a large margin. However, we observe that when noise and errors are present and the network is queried by a corrupt probe, SUM-OF-MAX faces a severe limitation as its stringent activation rule prevents a neuron from reviving back into play once deactivated. In this manuscript, we categorize and analyze different error scenarios so that both the erasure and the corrupt scenarios can be treated consistently. We make an amendment to the network structure to improve the retrieval rate, at the cost of an extra scalar per neuron. Afterwards, five different approaches are proposed to deal with corrupt probes. As a result, we extend the network capability, and also increase the robustness of the retrieving procedure. We then experimentally compare all these proposals and discuss pros and cons of each approach under different types of errors. Simulation results show that if carefully designed, the network is able to preserve both a high retrieval rate and a low running time simultaneously, even when queried by a corrupt probe.
Gripon and Berrou propose the network structure in @cite_4 . @cite_8 , they show that using the same amount of storage, CSAMs outperform Hopfield networks in diversity (the number of patterns a network can store for a targeted performance), capacity (the maximum amount of stored information in bits for a targeted performance) and efficiency (the ratio between capacity and the amount of information in bits consumed by the network when capacity reaches its maximum) simultaneously. They later interpret CSAMs using the formalism of error correcting codes @cite_21 and propose a new decoding scheme called , which significantly decreases retrieval error. Jiang al @cite_20 modify CSAMs to store long sequences by incorporating directed links. Aboudib al @cite_11 extend the structure so that messages of different lengths can be stored in the same network. They also summarize criteria to build possible retrieving schemes and study the number of iterations required by each scheme. Yao al @cite_22 discover a previously overlooked problem that the network may converge to a bogus fixed point and propose heuristics to mitigate the issue. A novel post-processing algorithms is also developed, customized to the partite structure of CSAMs, which brings notably better retrieval rates than the standard scheme.
@cite_4 @cite_22 @cite_8 @cite_21 @cite_20 @cite_11
1666036116 2602199281 2892998444 2136340918
The Gripon-Berrou neural network (GBNN) is a recently invented recurrent neural network embracing a LDPC-like sparse encoding setup which makes it extremely resilient to noise and errors. A natural use of GBNN is as an associative memory. There are two activation rules for the neuron dynamics, namely sum-of-sum and sum-of-max. The latter outperforms the former in terms of retrieval rate by a huge margin. In prior discussions and experiments, it is believed that although sum-of-sum may lead the network to oscillate, sum-of-max always converges to an ensemble of neuron cliques corresponding to previously stored patterns. However, this is not entirely correct. In fact, sum-of-max often converges to bogus fixed points where the ensemble only comprises a small subset of the converged state. By taking advantage of this overlooked fact, we can greatly improve the retrieval rate. We discuss this particular issue and propose a number of heuristics to push sum-of-max beyond these bogus fixed points. To tackle the problem directly and completely, a novel post-processing algorithm is also developed and customized to the structure of GBNN. Experimental results show that the new algorithm achieves a huge performance boost in terms of both retrieval rate and run-time, compared to the standard sum-of-max and all the other heuristics. In this paper, a novel framework is proposed to jointly optimize user association and resource allocation in the uplink cognitive femtocell network (CFN). In the considered CFN, femtocell base stations (FBSs) are deployed to serve a set of femtocell user equipments (FUEs) by reusing subchannels used in a macrocell base station (MBS). The problem of joint user association, subchannel assignment, and power allocation is formulated as an optimization problem, in which the goal is to maximize the overall uplink throughput while guaranteeing FBSs overloading avoidance, data rate requirements of the served FUEs, and MBS protection. To solve this problem, a distributed framework based on the matching game is proposed to model and analyze the interactions between the FUEs and FBSs. Using this framework, distributed algorithms are developed to enable the CFN to make decisions about user association, subchannel allocation, and transmit power. The algorithms are then shown to converge to a stable matching and exhibit a low computational complexity. Simulation results show that the proposed approach yields a performance improvement in terms of the overall network throughput and outage probability, with a small number of iterations to converge. The performance of single image super-resolution has achieved significant improvement by utilizing deep convolutional neural networks (CNNs). The features in deep CNN contain different types of information which make different contributions to image reconstruction. However, most CNN-based models lack discriminative ability for different types of information and deal with them equally, which results in the representational capacity of the models being limited. On the other hand, as the depth of neural networks grows, the long-term information coming from preceding layers is easy to be weaken or lost in late layers, which is adverse to super-resolving image. To capture more informative features and maintain long-term information for image super-resolution, we propose a channel-wise and spatial feature modulation (CSFM) network in which a sequence of feature-modulation memory (FMM) modules is cascaded with a densely connected structure to transform low-resolution features to high informative features. In each FMM module, we construct a set of channel-wise and spatial attention residual (CSAR) blocks and stack them in a chain structure to dynamically modulate multi-level features in a global-and-local manner. This feature modulation strategy enables the high contribution information to be enhanced and the redundant information to be suppressed. Meanwhile, for long-term information persistence, a gated fusion (GF) node is attached at the end of the FMM module to adaptively fuse hierarchical features and distill more effective information via the dense skip connections and the gating mechanism. Extensive quantitative and qualitative evaluations on benchmark datasets illustrate the superiority of our proposed method over the state-of-the-art methods. We consider the problem of resource allocation in downlink OFDMA systems for multi service and unknown environment. Due to users' mobility and intercell interference, the base station cannot predict neither the Signal to Noise Ratio (SNR) of each user in future time slots nor their probability distribution functions. In addition, the traffic is bursty in general with unknown arrival. The probability distribution functions of the SNR, channel state and traffic arrival density are then unknown. Achieving a multi service Quality of Service (QoS) while optimizing the performance of the system (e.g. total throughput) is a hard and interesting task since it depends on the unknown future traffic and SNR values. In this paper we solve this problem by modeling the multiuser queuing system as a discrete time linear dynamic system. We develop a robust H∞ controller to regulate the queues of different users. The queues and Packet Drop Rates (PDR) are controlled by proposing a minimum data rate according to the demanded service type of each user. The data rate vector proposed by the controller is then fed as a constraint to an instantaneous resource allocation framework. This instantaneous problem is formulated as a convex optimization problem for instantaneous subcarrier and power allocation decisions. Simulation results show small delays and better fairness among users.