<jupyter_start><jupyter_text>TorchLife> Survival Analysis using pytorch This library takes a deep learning approach to Survival Analysis. What is Survival AnalysisA lot of classification problems are actually survival analysis problems and haven't been tackled as such. For example, consider a cancer patient and you take X-ray data from that patient. Over time, patients will **eventually** die from cancer (lets ignore the case where people will die from other diseases). The usual approach is to say here is the X-ray (x) and will the patient die in the next 30 days or not (y).Survival analysis instead asks the question given the input (x) and a time(t), what is the probability that a patient will survive for a time greater than t. Considering the training dataset if a patient is still alive, in the classification case it would be thought of as y = 0. In survival analysis we say that it is a censored observation since the patient will die at a certain time in the future when the experiment is not being conducted.The above analogy can be thought of in other scenarios such as churn prediction as well.A proper dive into theory can be seen [here](./SAT). What's with the name?Well, if you torch a life... you probability wouldn't survive. 😬 How to use this libraryThere are 3 models in here that can be used.1. [Kaplan Meier Model]2. [Proportional Hazard Models]3. [Accelerated Failure Time Models]All 3 models require you to input a pandas dataframe with the columns `"t", "e"` indicating time elapsed and a binary variable, event if a death (1) or live (0) instance is observed respectively. They are all capable of doing `fit` and `plot_survival_function`. Kaplan Meier ModelThis is the most simplistic model. Proportional Hazard ModelThis model attempts to model the instantaneous hazard of an instance given time. It does this by binning time and finding the cumulative hazard upto a given point in time. It's extention the cox model, takes into account other variables that are not time dependant such that the above mentioned hazard can grow or shrink proportional to the risk associated with non-temporal feature based hazard.```pythonfrom torchlife.model import ModelHazardmodel = ModelHazard('cox')model.fit(df)inst_hazard, surv_probability = model.predict(df)``` Accelerated Failure Time ModelsThis model attempts to model (the mode and not average) time such that it is a function of the non-temporal features, x.You are free to choose the distribution of the error, however, `Gumbel` distribution is a popular option in SA literature. The distribution needs to be over the real domain and not just positive since we are modelling log time.```pythonfrom torchlife.model import ModelAFTmodel = ModelAFT('Gumbel')model.fit(df)surv_prob = model.predict(df)mode_time = model.predict_time(df)``` KudosSpecial thanks to the following libraries and resources.- [lifelines](https://lifelines.readthedocs.io/en/latest/) and especially Cameron Davidson-Pilon- [pymc3 survival analysis examples](https://docs.pymc.io/nb_examples/index.html)- [nbdev](https://nbdev.fast.ai/)- [pytorch lightning](https://pytorch-lightning.readthedocs.io/)- [Generalised Linear Models by <NAME>](https://data.princeton.edu/wws509/notes/c7s1), Chapter 7.  Install `pip install torchlife`  How to useWe need a dataframe that has a column named 't' indicating time, and 'e' indicating a death event.<jupyter_code># hide
%load_ext autoreload
%autoreload 2
%matplotlib inline
import pandas as pd
import numpy as np
url = "https://raw.githubusercontent.com/CamDavidsonPilon/lifelines/master/lifelines/datasets/rossi.csv"
df = pd.read_csv(url)
df.rename(columns={'week':'t', 'arrest':'e'}, inplace=True)
df.head()
from torchlife.model import ModelHazard

model = ModelHazard('cox', lr=0.5)
model.fit(df)
λ, S = model.predict(df)<jupyter_output>GPU available: False, used: False
TPU available: None, using: 0 TPU cores

  | Name | Type               | Params
--------------------------------------------
0 | base | ProportionalHazard | 12    
--------------------------------------------
12        Trainable params
0         Non-trainable params
12        Total params
Epoch 0:  75%|███████▌  | 3/4 [00:00<00:00, 25.43it/s, loss=nan, v_num=49]
Epoch 0: 100%|██████████| 4/4 [00:00<00:00, 16.39it/s, loss=nan, v_num=49]
Epoch 1:  75%|███████▌  | 3/4 [00:00<00:00, 23.74it/s, loss=nan, v_num=49]
Epoch 1: 100%|██████████| 4/4 [00:00<00:00, 15.25it/s, loss=nan, v_num=49]
Epoch 2:  75%|███████▌  | 3/4 [00:00<00:00, 22.98it/s, loss=nan, v_num=49]
Epoch 2: 100%|██████████| 4/4 [00:00<00:00, 15.53it/s, loss=nan, v_num=49]
Epoch 3:  75%|███████▌  | 3/4 [00:00<00:00, 20.23it/s, loss=nan, v_num=49]
Validating: 0it [00:00, ?it/s][A
Epoch 3: 100%|██████████| 4/4 [00:00<00:00, 13.30it/s, loss=nan, v_num=49]
                                          [...]<jupyter_text>Let's plot the survival function for the 4th element in the dataframe:<jupyter_code>x = df.drop(['t', 'e'], axis=1).iloc[2]
t = np.arange(df['t'].max())
model.plot_survival_function(t, x)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.12771417291748915, 'fraction_numerical': 0.038097118041847615, 'mean_word_length': 4.261682242990654, 'pattern_counts': {'":': 0, '<': 17, '<?xml version=': 0, '>': 10, 'https://': 6, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '2', 'dir': 'jupyter-structured-clean-dedup', 'id': '429815', 'license': 'Apache-2.0', 'n_tokens_mistral': 1737, 'n_tokens_neox': 1532, 'n_words': 622, 'path': '00_index.ipynb', 'repo_name': 'sachinruk/torchlife2', 'size': '5076'}