<jupyter_start><jupyter_text>Train and deploy on Kubeflow from NotebooksThis notebook introduces you to using Kubeflow Fairing to train and deploy a model to Kubeflow on Google Kubernetes Engine (GKE), and Google Cloud ML Engine. This notebook demonstrate how to: * Train an XGBoost model in a local notebook,* Use Kubeflow Fairing to train an XGBoost model remotely on Kubeflow,* Use Kubeflow Fairing to train an XGBoost model remotely on Cloud ML Engine,* Use Kubeflow Fairing to deploy a trained model to Kubeflow, and* Call the deployed endpoint for predictions.To learn more about how to run this notebook locally, see the guide to [training and deploying on GCP from a local notebook][gcp-local-notebook].[gcp-local-notebook]: https://kubeflow.org/docs/fairing/gcp-local-notebook/  Set up your notebook for training an XGBoost modelImport the libraries required to train this model.<jupyter_code>!pip3 install --user joblib
!pip3 install --user sklearn
import argparse
import logging
import os
import joblib
import sys
from pathlib import Path
import pandas as pd
from sklearn.metrics import mean_absolute_error
from sklearn.model_selection import train_test_split
from sklearn.impute import SimpleImputer
from xgboost import XGBRegressor
import ames
from importlib import reload

reload(ames)
fairing_code = os.path.join(Path.home(), "git_jlewi-kubecon-demo", "fairing")

if os.path.exists(fairing_code):    
    logging.info("Adding %s to path", fairing_code)
    sys.path = [fairing_code] + sys.path
logging.basicConfig(format='%(message)s')
logging.getLogger().setLevel(logging.INFO)<jupyter_output><empty_output><jupyter_text>Define a class for your model, with methods for training and prediction.<jupyter_code>class HousingServe(object):    
    def __init__(self):
        self.train_input = "ames_dataset/train.csv"
        self.n_estimators = 50
        self.learning_rate = 0.1
        self.model_file = "trained_ames_model.dat"
        self.model = None

    def train(self):
        (train_X, train_y), (test_X, test_y) = ames.read_input(self.train_input)
        model = ames.train_model(train_X,
                                 train_y,
                                 test_X,
                                 test_y,
                                 self.n_estimators,
                                 self.learning_rate)

        ames.eval_model(model, test_X, test_y)
        ames.save_model(model, self.model_file)

    def predict(self, X, feature_names):
        """Predict using the model for given ndarray."""
        if not self.model:
            self.model = joblib.load(self.model_file)
        # Do any preprocessing
        prediction = self.model.predict(data=X)
        # Do any postprocessing
        return [[prediction.item(0), prediction.item(0)]]<jupyter_output><empty_output><jupyter_text>Train an XGBoost model in a notebookCall `HousingServe().train()` to train your model, and then evaluate and save your trained model.<jupyter_code>HousingServe().train()<jupyter_output>[0]	validation_0-rmse:177514
Will train until validation_0-rmse hasn't improved in 40 rounds.
[1]	validation_0-rmse:161858
[2]	validation_0-rmse:147237
[3]	validation_0-rmse:134132
[4]	validation_0-rmse:122224
[5]	validation_0-rmse:111538
[6]	validation_0-rmse:102142
[7]	validation_0-rmse:93392.3
[8]	validation_0-rmse:85824.6
[9]	validation_0-rmse:79667.6
[10]	validation_0-rmse:73463.4
[11]	validation_0-rmse:68059.4
[12]	validation_0-rmse:63350.5
[13]	validation_0-rmse:59732.1
[14]	validation_0-rmse:56260.7
[15]	validation_0-rmse:53392.6
[16]	validation_0-rmse:50770.8
[17]	validation_0-rmse:48107.8
[18]	validation_0-rmse:45923.9
[19]	validation_0-rmse:44154.2
[20]	validation_0-rmse:42488.1
[21]	validation_0-rmse:41263.3
[22]	validation_0-rmse:40212.8
[23]	validation_0-rmse:39089.1
[24]	validation_0-rmse:37691.1
[25]	validation_0-rmse:36875.2
[26]	validation_0-rmse:36276.2
[27]	validation_0-rmse:35444.1
[28]	validation_0-rmse:34831.5
[29]	validation_0-rmse:34205.4
[30]	validation_0-rmse[...]<jupyter_text>Set up Kubeflow Fairing for training and predictionsImport the `fairing` library and configure the environment that your training or prediction job will run in.<jupyter_code>import os
import fairing

# Setting up google container repositories (GCR) for storing output containers
# You can use any docker container registry istead of GCR
GCP_PROJECT = fairing.cloud.gcp.guess_project_name()
DOCKER_REGISTRY = 'gcr.io/{}/fairing-job'.format(GCP_PROJECT)
PY_VERSION = ".".join([str(x) for x in sys.version_info[0:3]])
BASE_IMAGE = 'python:{}'.format(PY_VERSION)<jupyter_output><empty_output><jupyter_text>Train an XGBoost model remotely on KubeflowImport the `TrainJob` and `KubeflowGKEBackend` classes. Kubeflow Fairing packages the `HousingServe` class, the training data, and the training job's software prerequisites as a Docker image. Then Kubeflow Fairing deploys and runs the training job on Kubeflow.<jupyter_code>rerun_training = True
from fairing import TrainJob
from fairing.backends import KubeflowGKEBackend
train_job = TrainJob(HousingServe, BASE_IMAGE, input_files=['ames_dataset/train.csv', "ames.py", 
                                                            "requirements.txt"],
                     docker_registry=DOCKER_REGISTRY, backend=KubeflowGKEBackend())

if rerun_training:
    train_job.submit()
else:
    # Print out cached output. This is solely for demo purposes since training can take a while
    with open("train_output.txt") as hf:        
        print(hf.read())
!kubectl get jobs -o yaml fairing-job-j5bh6<jupyter_output>apiVersion: batch/v1
kind: Job
metadata:
  creationTimestamp: "2019-05-17T01:25:23Z"
  generateName: fairing-job-
  labels:
    fairing-deployer: job
    fairing-id: a483e1be-7842-11e9-85d6-0a580a00012d
  name: fairing-job-j5bh6
  namespace: kubeflow
  resourceVersion: "13095243"
  selfLink: /apis/batch/v1/namespaces/kubeflow/jobs/fairing-job-j5bh6
  uid: a48de4d8-7842-11e9-8964-42010a8e00ff
spec:
  backoffLimit: 6
  completions: 1
  parallelism: 1
  selector:
    matchLabels:
      controller-uid: a48de4d8-7842-11e9-8964-42010a8e00ff
  template:
    metadata:
      creationTimestamp: null
      labels:
        controller-uid: a48de4d8-7842-11e9-8964-42010a8e00ff
        fairing-deployer: job
        fairing-id: a483e1be-7842-11e9-85d6-0a580a00012d
        job-name: fairing-job-j5bh6
      name: fairing-deployer
    spec:
      containers:
      - command:
        - python
        - /app/function_shim.py
        - --serialized_fn_file
        - /app/p[...]<jupyter_text>Deploy the trained model to Kubeflow for predictionsImport the `PredictionEndpoint` and `KubeflowGKEBackend` classes. Kubeflow Fairing packages the `HousingServe` class, the trained model, and the prediction endpoint's software prerequisites as a Docker image. Then Kubeflow Fairing deploys and runs the prediction endpoint on Kubeflow.<jupyter_code>rerun_deploy = False
from fairing import PredictionEndpoint
from fairing.backends import KubeflowGKEBackend
endpoint = PredictionEndpoint(HousingServe, BASE_IMAGE, input_files=['trained_ames_model.dat', "requirements.txt"],
                              docker_registry=DOCKER_REGISTRY, backend=KubeflowGKEBackend())

if rerun_deploy:
    endpoint.create()
else:
    endpoint.url = "http://fairing-service-p7zjs.kubeflow.svc.cluster.local:5000/predict"
    # Print out cached output. This is solely for demo purposes since training can take a while
    with open("deploy_output.txt") as hf:        
        print(hf.read())
!kubectl get deploy -o yaml fairing-deployer-gqpq8<jupyter_output>apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  annotations:
    deployment.kubernetes.io/revision: "1"
  creationTimestamp: "2019-05-17T01:32:33Z"
  generateName: fairing-deployer-
  generation: 1
  labels:
    fairing-deployer: serving
    fairing-id: a4bd1e10-7843-11e9-85d6-0a580a00012d
  name: fairing-deployer-gqpq8
  namespace: kubeflow
  resourceVersion: "13097036"
  selfLink: /apis/extensions/v1beta1/namespaces/kubeflow/deployments/fairing-deployer-gqpq8
  uid: a4be5a12-7843-11e9-8964-42010a8e00ff
spec:
  progressDeadlineSeconds: 600
  replicas: 1
  revisionHistoryLimit: 10
  selector:
    matchLabels:
      fairing-deployer: serving
      fairing-id: a4bd1e10-7843-11e9-85d6-0a580a00012d
  strategy:
    rollingUpdate:
      maxSurge: 25%
      maxUnavailable: 25%
    type: RollingUpdate
  template:
    metadata:
      creationTimestamp: null
      labels:
        fairing-deployer: serving
        fairing-id: a4bd1e10-7843-11e9-85d6-0[...]<jupyter_text>Call the prediction endpointCreate a test dataset, then call the endpoint on Kubeflow for predictions.<jupyter_code>(train_X, train_y), (test_X, test_y) = read_input("ames_dataset/train.csv")
import pprint
test_X
result = endpoint.predict_nparray(test_X)
pprint.pprint(result)<jupyter_output>'{"data":{"names":["t:0","t:1"],"tensor":{"shape":[1,2],"values":[165164.875,165164.875]}},"meta":{}}\n'<jupyter_text>Clean up the prediction endpointDelete the prediction endpoint created by this notebook.<jupyter_code># endpoint.delete()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.09517106972715461, 'fraction_numerical': 0.06258120398440883, 'mean_word_length': 3.8692672641012122, 'pattern_counts': {'":': 6, '<': 29, '<?xml version=': 0, '>': 29, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 7, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '8', 'dir': 'jupyter-structured-clean-dedup', 'id': '88293', 'license': 'Apache-2.0', 'n_tokens_mistral': 3526, 'n_tokens_neox': 3133, 'n_words': 736, 'path': 'xgboost-high-level-apis.ipynb', 'repo_name': 'jlewi/kubecon-demo', 'size': '9236'}