The Search for Extra-terrestrial Intelligence (SETI) aims to find technological signals of extra-solar origin. Radio frequency SETI is characterized by large unlabeled datasets and complex interference environment. The infinite possibilities of potential signal types require generalizable signal processing techniques with little human supervision. We present a generative model of self-supervised deep learning that can be used for anomaly detection and spatial filtering. We develop and evaluate our approach on spectrograms containing narrowband signals collected by Breakthrough Listen at the Green Bank telescope. The proposed approach is not meant to replace current narrowband searches but to demonstrate the potential to generalize to other signal types.
Recurrent neural network (RNN) and long short-term memory (LSTM) models ( @cite_11 @cite_17 @cite_1 @cite_13 @cite_16 @cite_7 @cite_14 ) have driven many of the recent advances in predictive video generation. The LSTM encoder-decoder models proposed in works such as @cite_16 and @cite_13 provide a general framework for sequence-to-sequence learning problems by training two separate LSTM models, one to map input sequence to a vector of fixed dimension and another to extract output sequence from that vector, thereby predicting future sequences or reconstructing past sequences. Predictive anomaly detection in radio frequency data has challenges of high thermal noise and large variations of signal strength. Time domain anomaly detection has first been explored in @cite_19 . The authors used a recurrent network, and show effective detection of a range of synthetic anomalies. Most recently @cite_3 explores predictive anomaly detection in spectrogram and spectral density functions (SDF). There the authors experimented with simulated signals and anomalies and concluded that there was a greater probability of detection in the SDF domain than in spectrograms.
@cite_11 @cite_14 @cite_7 @cite_1 @cite_3 @cite_19 @cite_16 @cite_13 @cite_17
2227316993 2743945814 2785979806 1489125746
Recurrent Neural Network (RNN) and one of its specific architectures, Long Short-Term Memory (LSTM), have been widely used for sequence labeling. In this paper, we first enhance LSTM-based sequence labeling to explicitly model label dependencies. Then we propose another enhancement to incorporate the global information spanning over the whole input sequence. The latter proposed method, encoder-labeler LSTM, first encodes the whole input sequence into a fixed length vector with the encoder LSTM, and then uses this encoded vector as the initial state of another LSTM for sequence labeling. Combining these methods, we can predict the label sequence with considering label dependencies and information of whole input sequence. In the experiments of a slot filling task, which is an essential component of natural language understanding, with using the standard ATIS corpus, we achieved the state-of-the-art F1-score of 95.66 . Recurrent neural networks (RNNs), such as long short-term memory networks (LSTMs), serve as a fundamental building block for many sequence learning tasks, including machine translation, language modeling, and question answering. In this paper, we consider the specific problem of word-level language modeling and investigate strategies for regularizing and optimizing LSTM-based models. We propose the weight-dropped LSTM which uses DropConnect on hidden-to-hidden weights as a form of recurrent regularization. Further, we introduce NT-ASGD, a variant of the averaged stochastic gradient method, wherein the averaging trigger is determined using a non-monotonic condition as opposed to being tuned by the user. Using these and other regularization strategies, we achieve state-of-the-art word level perplexities on two data sets: 57.3 on Penn Treebank and 65.8 on WikiText-2. In exploring the effectiveness of a neural cache in conjunction with our proposed model, we achieve an even lower state-of-the-art perplexity of 52.8 on Penn Treebank and 52.0 on WikiText-2. The Long Short-Term Memory Recurrent Neural Network (LSTM-RNN) has been demonstrated successful in handwritten text recognition of Western and Arabic scripts. It is totally segmentation free and can be trained directly from text line images. However, the application of LSTM-RNNs (including Multi-Dimensional LSTM-RNN (MDLSTM-RNN)) to Chinese text recognition has shown limited success, even when training them with large datasets and using pre-training on datasets of other languages. In this paper, we propose a handwritten Chinese text recognition method by using Separable MDLSTMRNN (SMDLSTM-RNN) modules, which extract contextual information in various directions, and consume much less computation efforts and resources compared with the traditional MDLSTMRNN. Experimental results on the ICDAR-2013 competition dataset show that the proposed method performs significantly better than the previous LSTM-based methods, and can compete with the state-of-the-art systems. We have recently shown that deep Long Short-Term Memory (LSTM) recurrent neural networks (RNNs) outperform feed forward deep neural networks (DNNs) as acoustic models for speech recognition. More recently, we have shown that the performance of sequence trained context dependent (CD) hidden Markov model (HMM) acoustic models using such LSTM RNNs can be equaled by sequence trained phone models initialized with connectionist temporal classification (CTC). In this paper, we present techniques that further improve performance of LSTM RNN acoustic models for large vocabulary speech recognition. We show that frame stacking and reduced frame rate lead to more accurate models and faster decoding. CD phone modeling leads to further improvements. We also present initial results for LSTM RNN models outputting words directly.