<jupyter_start><jupyter_text>Deploying Machine Learning Models Minikube with RBAC using kubectlThis demo shows how you can interact directly with kubernetes using kubectl to create and manage runtime machine learning models. It uses Minikube as the target Kubernetes cluster.  PrerequistesYou will need - [Git clone of Seldon Core](https://github.com/SeldonIO/seldon-core) - [Helm](https://github.com/kubernetes/helm) - [Minikube](https://github.com/kubernetes/minikube) version v0.24.0 or greater - [python grpc tools](https://grpc.io/docs/quickstart/python.html)  Create ClusterStart minikube and ensure custom resource validation is activated and there is 5G of memory. **2018-06-13** : At present we find the most stable version of minikube across platforms is 0.25.2 as there are issues with 0.26 and 0.27 on some systems. We also find the default VirtualBox driver can be problematic on some systems to we suggest using the [KVM2 driver](https://github.com/kubernetes/minikube/blob/master/docs/drivers.mdkvm2-driver).Your start command would then look like:```minikube start --vm-driver kvm2 --memory 4096 --feature-gates=CustomResourceValidation=true --extra-config=apiserver.Authorization.Mode=RBAC```  Setup<jupyter_code>!kubectl create namespace seldon
!kubectl config set-context $(kubectl config current-context) --namespace=seldon
!kubectl create clusterrolebinding kube-system-cluster-admin --clusterrole=cluster-admin --serviceaccount=kube-system:default<jupyter_output>clusterrolebinding "kube-system-cluster-admin" created<jupyter_text>Install Helm<jupyter_code>!kubectl -n kube-system create sa tiller
!kubectl create clusterrolebinding tiller --clusterrole cluster-admin --serviceaccount=kube-system:tiller
!helm init --service-account tiller<jupyter_output>serviceaccount "tiller" created
clusterrolebinding "tiller" created
$HELM_HOME has been configured at /home/clive/.helm.

Tiller (the Helm server-side component) has been installed into your Kubernetes Cluster.

Please note: by default, Tiller is deployed with an insecure 'allow unauthenticated users' policy.
To prevent this, run `helm init` with the --tiller-tls-verify flag.
For more information on securing your installation see: https://docs.helm.sh/using_helm/#securing-your-helm-installation
Happy Helming!<jupyter_text>Label the node to allow load testing to run on it<jupyter_code>!kubectl label nodes `kubectl get nodes -o jsonpath='{.items[0].metadata.name}'` role=locust --overwrite<jupyter_output>node "minikube" labeled<jupyter_text>Start seldon-core Install the custom resource definition<jupyter_code>!helm install ../helm-charts/seldon-core-crd --name seldon-core-crd --set usage_metrics.enabled=true
!helm install ../helm-charts/seldon-core --name seldon-core --namespace seldon<jupyter_output>NAME:   seldon-core
LAST DEPLOYED: Sat Sep  8 18:19:30 2018
NAMESPACE: seldon
STATUS: DEPLOYED

RESOURCES:
==> v1/Pod(related)
NAME                                               READY  STATUS             RESTARTS  AGE
seldon-core-seldon-apiserver-68f5984b49-grlg9      0/1    ContainerCreating  0         0s
seldon-core-seldon-cluster-manager-cfdbbccf-b425q  0/1    ContainerCreating  0         0s
seldon-core-redis-575979b79b-94qwv                 0/1    ContainerCreating  0         0s

==> v1/ServiceAccount
NAME    SECRETS  AGE
seldon  1        0s

==> v1beta1/ClusterRole
NAME        AGE
seldon-crd  0s

==> v1/ClusterRoleBinding
NAME    AGE
seldon  0s

==> v1beta1/Role
NAME          AGE
seldon-local  0s

==> v1/RoleBinding
NAME    AGE
seldon  0s

==> v1/Service
NAME                          TYPE       CLUSTER-IP      EXTERNAL-IP  PORT(S)                        AGE
seldon-core-seldon-apiserver  NodePort   10.99.37.242    <none>       8080:31791/TCP,5000:31403/TCP  0s
seldon-core-redis    [...]<jupyter_text>Install prometheus and grafana for analytics<jupyter_code>!helm install ../helm-charts/seldon-core-analytics --name seldon-core-analytics \
    --set grafana_prom_admin_password=password \
    --set persistence.enabled=false \
    --namespace seldon<jupyter_output>NAME:   seldon-core-analytics
LAST DEPLOYED: Sat Sep  8 18:19:34 2018
NAMESPACE: seldon
STATUS: DEPLOYED

RESOURCES:
==> v1/Service
NAME                      TYPE       CLUSTER-IP     EXTERNAL-IP  PORT(S)       AGE
alertmanager              ClusterIP  10.101.168.86  <none>       80/TCP        1s
grafana-prom              NodePort   10.106.43.18   <none>       80:30388/TCP  1s
prometheus-node-exporter  ClusterIP  None           <none>       9100/TCP      1s
prometheus-seldon         ClusterIP  10.105.60.175  <none>       80/TCP        1s

==> v1beta1/ClusterRole
NAME        AGE
prometheus  1s

==> v1/Job
NAME                            DESIRED  SUCCESSFUL  AGE
grafana-prom-import-dashboards  1        0           1s

==> v1/ServiceAccount
NAME        SECRETS  AGE
prometheus  1        1s

==> v1beta1/ClusterRoleBinding
NAME        AGE
prometheus  1s

==> v1beta1/Deployment
NAME                     DESIRED  CURRENT  UP-TO-DATE  AVAILABLE  AGE
alertmanager-deployment  1        1        1   [...]<jupyter_text>Check all services are running before proceeding.<jupyter_code>!kubectl rollout status deploy/seldon-core-seldon-cluster-manager
!kubectl rollout status deploy/seldon-core-seldon-apiserver<jupyter_output>deployment "seldon-core-seldon-cluster-manager" successfully rolled out
deployment "seldon-core-seldon-apiserver" successfully rolled out<jupyter_text>Set up REST and gRPC methods**Ensure you port forward the seldon api-server REST and GRPC ports**:REST:```kubectl port-forward $(kubectl get pods -n seldon -l app=seldon-apiserver-container-app -o jsonpath='{.items[0].metadata.name}') -n seldon 8002:8080```GRPC:```kubectl port-forward $(kubectl get pods -n seldon -l app=seldon-apiserver-container-app -o jsonpath='{.items[0].metadata.name}') -n seldon 8003:5000``` Install gRPC modules for the prediction protos.<jupyter_code>!cp ../proto/prediction.proto ./proto
!python -m grpc.tools.protoc -I. --python_out=. --grpc_python_out=. ./proto/prediction.proto<jupyter_output><empty_output><jupyter_text>Integrating with Kubernetes API  Validation Using OpenAPI Schema certain basic validation can be done before the custom resource is accepted.<jupyter_code>!kubectl create -f resources/model_invalid1.json -n seldon<jupyter_output>The SeldonDeployment "seldon-deployment-example" is invalid: []: Invalid value: map[string]interface {}{"apiVersion":"machinelearning.seldon.io/v1alpha2", "kind":"SeldonDeployment", "metadata":map[string]interface {}{"clusterName":"", "labels":map[string]interface {}{"app":"seldon"}, "name":"seldon-deployment-example", "namespace":"seldon", "creationTimestamp":"2018-09-08T17:28:16Z", "uid":"92581429-b38c-11e8-8d19-f8fff9063696", "selfLink":""}, "spec":map[string]interface {}{"annotations":map[string]interface {}{"deployment_version":"v1", "project_name":"FX Market Prediction"}, "name":"test-deployment", "oauth_key":1234, "oauth_secret":"oauth-secret", "predictors":[]interface {}{map[string]interface {}{"annotations":map[string]interface {}{"predictor_version":"v1"}, "componentSpecs":[]interface {}{map[string]interface {}{"spec":map[string]interface {}{"containers":[]interface {}{map[string]interface {}{"image":"seldonio/mean_classifier:0.6", "imagePullPolicy":22, "name":"mean-classifie[...]<jupyter_text>Normal OperationA simple example is shown below we use a single prepacked model for illustration. The spec contains a set of predictors each of which contains a ***componentSpec*** which is a Kubernetes [PodTemplateSpec](https://kubernetes.io/docs/api-reference/v1.9/podtemplatespec-v1-core) alongside a ***graph*** which describes how components fit together.<jupyter_code>!pygmentize resources/model.json<jupyter_output>{
    [34;01m"apiVersion"[39;49;00m: [33m"machinelearning.seldon.io/v1alpha2"[39;49;00m,
    [34;01m"kind"[39;49;00m: [33m"SeldonDeployment"[39;49;00m,
    [34;01m"metadata"[39;49;00m: {
        [34;01m"labels"[39;49;00m: {
            [34;01m"app"[39;49;00m: [33m"seldon"[39;49;00m
        },
        [34;01m"name"[39;49;00m: [33m"seldon-model"[39;49;00m
    },
    [34;01m"spec"[39;49;00m: {
        [34;01m"name"[39;49;00m: [33m"test-deployment"[39;49;00m,
        [34;01m"oauth_key"[39;49;00m: [33m"oauth-key"[39;49;00m,
        [34;01m"oauth_secret"[39;49;00m: [33m"oauth-secret"[39;49;00m,
        [34;01m"predictors"[39;49;00m: [
            {
                [34;01m"componentSpecs"[39;49;00m: [{
                    [34;01m"spec"[39;49;00m: {
                        [34;01m"containers"[39;49;00m: [
                            {
                                [34;01m"image"[39;49;00m: [33m"seldonio/mock_classifier:1.0"[39;[...]<jupyter_text>Create Seldon Deployment Deploy the runtime graph to kubernetes.<jupyter_code>!kubectl apply -f resources/model.json -n seldon<jupyter_output>seldondeployment "seldon-model" created<jupyter_text>Get the status of the SeldonDeployment. **When ready the replicasAvailable should be 1**.<jupyter_code>!kubectl get seldondeployments seldon-model -o jsonpath='{.status}' -n seldon<jupyter_output>map[predictorStatus:[map[replicas:1 replicasAvailable:1 name:test-deployment-example-svc-orch] map[replicasAvailable:1 name:test-deployment-example-classifier-0 replicas:1]]]<jupyter_text>Get predictions<jupyter_code>from seldon_utils import *
API_GATEWAY_REST="localhost:8002"
API_GATEWAY_GRPC="localhost:8003"<jupyter_output><empty_output><jupyter_text>REST Request<jupyter_code>rest_request_api_gateway("oauth-key","oauth-secret",API_GATEWAY_REST)<jupyter_output>{"access_token":"<KEY>","token_type":"bearer","expires_in":43199,"scope":"read write"}
{
  "meta": {
    "puid": "urvtkn3uh34bguifguqiihd3vd",
    "tags": {
    },
    "routing": {
    }
  },
  "data": {
    "names": ["proba"],
    "tensor": {
      "shape": [1, 1],
      "values": [0.07598352200745992]
    }
  }
}<jupyter_text>gRPC Request<jupyter_code>grpc_request_api_gateway("oauth-key","oauth-secret",API_GATEWAY_REST,API_GATEWAY_GRPC)<jupyter_output>{"access_token":"<KEY>","token_type":"bearer","expires_in":43197,"scope":"read write"}
meta {
  puid: "6s3l23rl05qasv9cdkhkq12hrh"
}
data {
  names: "proba"
  tensor {
    shape: 1
    shape: 1
    values: 0.07028283389333956
  }
}<jupyter_text>Update deployment with canary We will change the deployment to add a "canary" deployment. This illustrates: - Updating a deployment with no downtime - Adding an extra predictor to run alongside th exsting predictor.  You could manage different traffic levels by controlling the number of replicas of each.<jupyter_code>!pygmentize resources/model_with_canary.json
!kubectl apply -f resources/model_with_canary.json -n seldon<jupyter_output>seldondeployment "seldon-model" configured<jupyter_text>Check the status of the deployments. Note: **Might need to run several times until replicasAvailable is 1 for both predictors**.<jupyter_code>!kubectl get seldondeployments seldon-model -o jsonpath='{.status}' -n seldon<jupyter_output>map[predictorStatus:[map[name:test-deployment-example-svc-orch replicas:1 replicasAvailable:1] map[name:test-deployment-example-classifier-0 replicas:1 replicasAvailable:1] map[name:test-deployment-canary-svc-orch replicas:1 replicasAvailable:1] map[name:test-deployment-canary-mean-classifier-0 replicas:1 replicasAvailable:1]]]<jupyter_text>REST Request<jupyter_code>rest_request_api_gateway("oauth-key","oauth-secret",API_GATEWAY_REST)<jupyter_output>{"access_token":"<KEY>","token_type":"bearer","expires_in":43117,"scope":"read write"}
{
  "meta": {
    "puid": "35fc3sdl0jj0m8s6lco0kq9bt5",
    "tags": {
    },
    "routing": {
    }
  },
  "data": {
    "names": ["proba"],
    "tensor": {
      "shape": [1, 1],
      "values": [0.07515448234459429]
    }
  }
}<jupyter_text>gRPC request<jupyter_code>grpc_request_api_gateway("oauth-key","oauth-secret",API_GATEWAY_REST,API_GATEWAY_GRPC)<jupyter_output>{"access_token":"<KEY>","token_type":"bearer","expires_in":43115,"scope":"read write"}
meta {
  puid: "t7b9vir0mcvfoa9llvi6jbvs8h"
}
data {
  names: "proba"
  tensor {
    shape: 1
    shape: 1
    values: 0.08294586370978486
  }
}<jupyter_text>Load test Start a load test which will post REST requests at 10 requests per second.<jupyter_code>!helm install seldon-core-loadtesting --name loadtest  \
    --set locust.host=http://seldon-core-seldon-apiserver:8080 \
    --set oauth.key=oauth-key \
    --set oauth.secret=oauth-secret \
    --namespace seldon \
    --repo https://storage.googleapis.com/seldon-charts<jupyter_output>NAME:   loadtest
LAST DEPLOYED: Sat Sep  8 18:32:36 2018
NAMESPACE: seldon
STATUS: DEPLOYED

RESOURCES:
==> v1/ReplicationController
NAME             DESIRED  CURRENT  READY  AGE
locust-slave-1   1        1        0      0s
locust-master-1  1        1        0      0s

==> v1/Service
NAME             TYPE      CLUSTER-IP     EXTERNAL-IP  PORT(S)                                       AGE
locust-master-1  NodePort  10.111.138.47  <none>       5557:31357/TCP,5558:31829/TCP,8089:31564/TCP  0s

==> v1/Pod(related)
NAME                   READY  STATUS             RESTARTS  AGE
locust-slave-1-pc6mc   0/1    ContainerCreating  0         0s
locust-master-1-8g9jr  0/1    ContainerCreating  0         0s<jupyter_text>You should port-foward the grafana dashboard```bashkubectl port-forward $(kubectl get pods -n seldon -l app=grafana-prom-server -o jsonpath='{.items[0].metadata.name}') -n seldon 3000:3000```You can then view an analytics dashboard inside the cluster at http://localhost:3000/dashboard/db/prediction-analytics?refresh=5s&orgId=1. Your IP address may be different. get it via minikube ip. Login with: - Username : admin - password : password (as set when starting seldon-core-analytics above)  The dashboard should look like below:     Tear down<jupyter_code>!helm delete loadtest --purge
!kubectl delete -f resources/model_with_canary.json -n seldon
!helm delete seldon-core-analytics --purge
!helm delete seldon-core --purge
!helm delete seldon-core-crd --purge<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.12517670342097822, 'fraction_numerical': 0.05188012439920837, 'mean_word_length': 4.117179023508138, 'pattern_counts': {'":': 61, '<': 74, '<?xml version=': 0, '>': 90, 'https://': 8, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 5, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 3, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '20', 'dir': 'jupyter-structured-clean-dedup', 'id': '36019', 'license': 'Apache-2.0', 'n_tokens_mistral': 5439, 'n_tokens_neox': 4854, 'n_words': 1011, 'path': 'notebooks/kubectl_demo_minikube_rbac.ipynb', 'repo_name': 'NunoEdgarGFlowHub/seldon-core', 'size': '14272'}