<jupyter_start><jupyter_text>ExtrapolationIn this notebook you will find:- How to get a survival curve using xgbse- How to extrapolate your predicted survival curve using the `xgbse.extrapolation` module  MetrabicWe will be using the Molecular Taxonomy of Breast Cancer International Consortium (METABRIC) dataset from [pycox](https://github.com/havakv/pycoxdatasets) as base for this example.<jupyter_code>from xgbse.converters import convert_to_structured
from pycox.datasets import metabric
import numpy as np

# getting data
df = metabric.read_df()

df.head()<jupyter_output><empty_output><jupyter_text>Split and create Time BinsSplit the data in train and test, using sklearn API. We also setup the TIME_BINS arange, which will be used to fit the survival curve<jupyter_code>from xgbse.converters import convert_to_structured
from sklearn.model_selection import train_test_split

# splitting to X, T, E format
X = df.drop(['duration', 'event'], axis=1)
T = df['duration']
E = df['event']
y = convert_to_structured(T, E)

# splitting between train, and validation 
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=1/3, random_state = 0)
TIME_BINS = np.arange(15, 315, 15)
TIME_BINS<jupyter_output><empty_output><jupyter_text>Fit model and predict survival curvesThe package follows `scikit-learn` API, with a minor adaptation to work with time and event data. The model outputs the probability of survival, in a `pd.Dataframe` where columns represent different times.<jupyter_code>from xgbse import XGBSEDebiasedBCE

# fitting xgbse model
xgbse_model = XGBSEDebiasedBCE()
xgbse_model.fit(X_train, y_train, time_bins=TIME_BINS)

# predicting
survival = xgbse_model.predict(X_test)
survival.head()<jupyter_output><empty_output><jupyter_text>Survival curves visualization<jupyter_code>import matplotlib.pyplot as plt

plt.figure(figsize=(12,4), dpi=120)

plt.plot(
    survival.columns,
    survival.iloc[42],
    'k--',
    label='Survival'
)

plt.title('Sample of predicted survival curves - $P(T>t)$')
plt.legend()<jupyter_output><empty_output><jupyter_text>Notice that this predicted survival curve does not end at zero (cure fraction due to censored data). In some cases it might be useful to extrapolate our survival curves using specific strategies. `xgbse.extrapolation` implements a constant risk extrapolation strategy.  Extrapolation<jupyter_code>from xgbse.extrapolation import extrapolate_constant_risk

# extrapolating predicted survival
survival_ext = extrapolate_constant_risk(survival, 450, 11)
survival_ext.head()
# plotting extrapolation #

plt.figure(figsize=(12,4), dpi=120)

plt.plot(
    survival.columns,
    survival.iloc[42],
    'k--',
    label='Survival'
)

plt.plot(
    survival_ext.columns,
    survival_ext.iloc[42],
    'tomato',
    alpha=0.5,
    label='Extrapolated Survival'
)

plt.title('Extrapolation of survival curves')
plt.legend()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.09131326949384405, 'fraction_numerical': 0.012311901504787962, 'mean_word_length': 5.98090692124105, 'pattern_counts': {'":': 0, '<': 21, '<?xml version=': 0, '>': 22, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '5', 'dir': 'jupyter-structured-clean-dedup', 'id': '207112', 'license': 'Apache-2.0', 'n_tokens_mistral': 1022, 'n_tokens_neox': 957, 'n_words': 273, 'path': 'examples/extrapolation_example.ipynb', 'repo_name': 'gdmarmerola/xgboost-survival-embeddings', 'size': '2924'}