<jupyter_start><jupyter_text>BeatGAN: Anomalous Rhythm Detection using Adversarially Generated Time AbstractGiven a large-scale rhythmic time series containing mostly normal data segments (or ‘beats’), can we learn how to detect anomalous beats in an effective yet efficient way? For example, how can we detect anomalous beats from electrocardiogram (ECG) readings? Existing approaches either require excessively high amounts of labeled and balanced data for classification, or rely on less regularized reconstructions, resulting in lower accuracy in anomaly detection. Therefore, we propose BeatGAN, an unsupervised anomaly detection algorithm for time series data. BeatGAN outputs explainable results to pinpoint the anomalous time ticks of an input beat, by comparing them to adversarially generated beats. Its robustness is guaranteed by its regularization of reconstruction error using an adversarial generation approach, as well as data augmentation using time series warping. Experiments show that BeatGAN accurately and efficiently detects anomalous beats in ECG time series, and routes doctors’ attention to anomalous time ticks, achieving accuracy of nearly 0.95 AUC, and very fast inference (2.6 ms per beat). In addition, we show that BeatGAN accurately detects unusual motions from multivariate motion-capture time series data, illustrating its generality. You can configure the backend to use GPU or CPU only. \Default is using backend cpu.<jupyter_code>import sys
sys.path.append('..')
import spartan as st<jupyter_output><empty_output><jupyter_text>```loadTensor``` function automatically read data from file and ```toDTensor``` function extract time and value separately from the tensor.```Timeseries``` class is designed to construct time tensor.<jupyter_code>time, value = st.loadTensor(path = "inputData/example_time.tensor", col_types = [float, float]).toDTensor(hastticks=True)
time_series = st.Timeseries(value, time)<jupyter_output><empty_output><jupyter_text>Run RPeak to segment ECG series data. This example use ECG series data as example. Before we train the model, we need RPeak to preprocess data.Default parameters are:{'sampling_rate': 360, 'left_size': 120, 'right_size': 136, 'out_path': None}<jupyter_code>rpeak = st.RPeak(time_series)
segments = rpeak.run()<jupyter_output><empty_output><jupyter_text>Run BeatGAN as a single model BeatGAN has two kinds of structure, CNN and RNN which is controled by parameter ```network```.Default parameters are: {'seq_len': 256, 'stride': 32, 'input_size': 1, 'rep_size': 20, 'batch_size': 64, 'max_epoch':5, 'lr': 0.01, 'lambda': 1'net_type':'gru', 'layers':1, 'hidden_size':100} You can change them be passing = as the doc shows.<jupyter_code>beatgan_CNN = st.BeatGAN(segments, "my_beatgan_model", network="CNN")
trained_model = beatgan_CNN.fit()<jupyter_output><empty_output><jupyter_text>Use PipeLine to finish the work mentioned above.<jupyter_code>rpeak_params = {'sampling_rate': 360, 'left_size': 120, 'right_size': 136}
beatgan_params = {'network': 'CNN', 'seq_len': 256, 'stride': 32, 'input_size': 1, 'rep_size': 20, 'model_name': 'my_beatgan_model'}
pipeline = st.PipeLine(time_series, [(st.RPeak, rpeak_params), ((st.Train, st.TrainPolicy.BeatGAN), beatgan_params)])<jupyter_output><empty_output><jupyter_text>We will get a trained model after calling ```run``` function of ```pipeline```.<jupyter_code>pipeline.run()<jupyter_output><empty_output><jupyter_text>Save and load trained models Function ```save_model_to``` saves model to assigned path.<jupyter_code>trained_model.save_model_to(path='../spartan/model/beatgan/model/test_cnn.pth')<jupyter_output><empty_output><jupyter_text>Parameter ```model_path``` loads model from assigned path.<jupyter_code>beatgan_CNN = st.BeatGAN(segments, "my_beatgan_model", network="CNN", model_path="../spartan/model/beatgan/model/ecg_cnn.pth")
rec_diff, ori_ts, rec_ts, rec_err = beatgan_CNN.predict()<jupyter_output><empty_output><jupyter_text>Sort by max window reconstruction error and get the sorted indices.<jupyter_code>import numpy as np
new_score=[]
window_size=32
for scores in rec_err:
    tmp_score=0
    for i in range(0,len(scores),window_size):
        tmp_score=max(tmp_score, np.mean(scores[i:i+window_size]))
    new_score.append(tmp_score)
ind=np.argsort(new_score)<jupyter_output><empty_output><jupyter_text>Draw plots of segments which deviate the least and the most with ```st.plot```.<jupyter_code>st.plot(st.BeatGAN, ori_ts[ind[0]], rec_ts[ind[0]], rec_err)
st.plot(st.BeatGAN, ori_ts[ind[-1]], rec_ts[ind[-1]], rec_err)<jupyter_output><empty_output><jupyter_text>Deviate the least | Deviate the most:-------------------------:|:-------------------------:  |   Segment with maximum reconstruction error. |  Segment with minimum reconstruction error.  Run BeatGAN from train task to get a trained model<jupyter_code>train_task = st.Train.create(segments, st.TrainPolicy.BeatGAN, 'my_beatgan_model', network='CNN')
train_task.run()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.11441693290734824, 'fraction_numerical': 0.0115814696485623, 'mean_word_length': 6.647328244274809, 'pattern_counts': {'":': 0, '<': 45, '<?xml version=': 0, '>': 45, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '11', 'dir': 'jupyter-structured-clean-dedup', 'id': '21992', 'license': 'BSD-3-Clause', 'n_tokens_mistral': 1621, 'n_tokens_neox': 1480, 'n_words': 480, 'path': 'live-tutorials/BeatGAN.ipynb', 'repo_name': 'sunxiaobing1999/spartan2', 'size': '5008'}