Recently, neural networks have achieved great success on sentiment classification due to their ability to alleviate feature engineering. However, one of the remaining challenges is to model long texts in document-level sentiment classification under a recurrent architecture because of the deficiency of the memory unit. To address this problem, we present a Cached Long Short-Term Memory neural networks (CLSTM) to capture the overall semantic information in long texts. CLSTM introduces a cache mechanism, which divides memory into several groups with different forgetting rates and thus enables the network to keep sentiment information better within a recurrent unit. The proposed CLSTM outperforms the state-of-the-art models on three publicly available document-level sentiment analysis datasets.
The other one tries to use multiple time-scales to distinguish different states @cite_8 @cite_32 @cite_13 . They partition the hidden states into several groups and each group is activated and updated at different frequencies (e.g. one group updates every 2 time-step, the other updates every 4 time-step). In these methods, different memory groups are not fully interconnected, and the information is transmitted from faster groups to slower ones, or vice versa.
@cite_13 @cite_32 @cite_8
2099641086 2081564414 2949343062 61705880
Suppose a discrete-time signal S(t), 0 spl les t<N, is a superposition of atoms taken from a combined time-frequency dictionary made of spike sequences 1 sub t= spl tau and sinusoids exp 2 spl pi iwt N spl radic N. Can one recover, from knowledge of S alone, the precise collection of atoms going to make up S? Because every discrete-time signal can be represented as a superposition of spikes alone, or as a superposition of sinusoids alone, there is no unique way of writing S as a sum of spikes and sinusoids in general. We prove that if S is representable as a highly sparse superposition of atoms from this time-frequency dictionary, then there is only one such highly sparse representation of S, and it can be obtained by solving the convex optimization problem of minimizing the l sup 1 norm of the coefficients among all decompositions. Here "highly sparse" means that N sub t +N sub w < spl radic N 2 where N sub t is the number of time atoms, N sub w is the number of frequency atoms, and N is the length of the discrete-time signal. Underlying this result is a general l sup 1 uncertainty principle which says that if two bases are mutually incoherent, no nonzero signal can have a sparse representation in both bases simultaneously. For the above setting, the bases are sinusoids and spikes, and mutual incoherence is measured in terms of the largest inner product between different basis elements. The uncertainty principle holds for a variety of interesting basis pairs, not just sinusoids and spikes. The results have idealized applications to band-limited approximation with gross errors, to error-correcting encryption, and to separation of uncoordinated sources. Related phenomena hold for functions of a real variable, with basis pairs such as sinusoids and wavelets, and for functions of two variables, with basis pairs such as wavelets and ridgelets. In these settings, if a function f is representable by a sufficiently sparse superposition of terms taken from both bases, then there is only one such sparse representation; it may be obtained by minimum l sup 1 norm atomic decomposition. The condition "sufficiently sparse" becomes a multiscale condition; for example, that the number of wavelets at level j plus the number of sinusoids in the jth dyadic frequency band are together less than a constant times 2 sup j 2 . We report results of a numerical analysis of the memory effects in two-dimensional Abelian sandpiles. It is found that a sandpile forgets its instantaneous configuration in two distinct stages: a fast stage and a slow stage, whose durations roughly scale as N and N2 respectively, where N is the linear size of the sandpile. We confirm the presence of the longer time-scale by an independent diagnostic based on analysing emission probabilities of a hidden Markov model applied to a time-averaged sequence of avalanche sizes. The application of hidden Markov modelling to the output of sandpiles is novel. It discriminates effectively between a sandpile time series and a shuffled control time series with the same time-averaged event statistics and hence deserves further development as a pattern-recognition tool for Abelian sandpiles. In this paper, collocated and distributed space-time block codes (DSTBCs) which admit multi-group maximum likelihood (ML) decoding are studied. First the collocated case is considered and the problem of constructing space-time block codes (STBCs) which optimally tradeoff rate and ML decoding complexity is posed. Recently, sufficient conditions for multi-group ML decodability have been provided in the literature and codes meeting these sufficient conditions were called Clifford Unitary Weight (CUW) STBCs. An algebraic framework based on extended Clifford algebras is proposed to study CUW STBCs and using this framework, the optimal tradeoff between rate and ML decoding complexity of CUW STBCs is obtained for few specific cases. Code constructions meeting this tradeoff optimally are also provided. The paper then focuses on multi-group ML decodable DSTBCs for application in synchronous wireless relay networks and three constructions of four-group ML decodable DSTBCs are provided. Finally, the OFDM based Alamouti space-time coded scheme proposed by Li-Xia for a 2 relay asynchronous relay network is extended to a more general transmission scheme that can achieve full asynchronous cooperative diversity for arbitrary number of relays. It is then shown how differential encoding at the source can be combined with the proposed transmission scheme to arrive at a new transmission scheme that can achieve full cooperative diversity in asynchronous wireless relay networks with no channel information and also no timing error knowledge at the destination node. Four-group decodable DSTBCs applicable in the proposed OFDM based transmission scheme are also given. Research in recent years has provided some evidence of temporal non-stationarity of functional connectivity in resting state fMRI. In this paper, we present a novel methodology that can decode connectivity dynamics into a temporal sequence of hidden network "states" for each subject, using a Hidden Markov Modeling (HMM) framework. Each state is characterized by a unique covariance matrix or whole-brain network. Our model generates these covariance matrices from a common but unknown set of sparse basis networks, which capture the range of functional activity co-variations of regions of interest (ROIs). Distinct hidden states arise due to a variation in the strengths of these basis networks. Thus, our generative model combines a HMM framework with sparse basis learning of positive definite matrices. Results on simulated fMRI data show that our method can effectively recover underlying basis networks as well as hidden states. We apply this method on a normative dataset of resting state fMRI scans. Results indicate that the functional activity of a subject at any point during the scan is composed of combinations of overlapping task-positive negative pairs of networks as revealed by our basis. Distinct hidden temporal states are produced due to a different set of basis networks dominating the covariance pattern in each state.