<jupyter_start><jupyter_text>Custom Header Routing with Seldon and AmbassadorThis notebook shows how you can deploy Seldon Deployments which can have custom routing via Ambassador's custom header routing.  PrerequistesYou will need - [Git clone of Seldon Core](https://github.com/SeldonIO/seldon-core) running this notebook - A running Kubernetes cluster with kubectl authenticated - [Helm client](https://helm.sh/)  Creating a Kubernetes ClusterFollow the [Kubernetes documentation to create a cluster](https://kubernetes.io/docs/setup/).Once created ensure ```kubectl``` is authenticated against the running cluster.  Setup<jupyter_code>!kubectl create namespace seldon
!kubectl config set-context $(kubectl config current-context) --namespace=seldon
!kubectl create clusterrolebinding kube-system-cluster-admin --clusterrole=cluster-admin --serviceaccount=kube-system:default<jupyter_output><empty_output><jupyter_text>Install Helm<jupyter_code>!kubectl -n kube-system create sa tiller
!kubectl create clusterrolebinding tiller --clusterrole cluster-admin --serviceaccount=kube-system:tiller
!helm init --service-account tiller
!kubectl rollout status deploy/tiller-deploy -n kube-system<jupyter_output><empty_output><jupyter_text>Start seldon-core<jupyter_code>!helm install ../../../helm-charts/seldon-core-crd --name seldon-core-crd --set usage_metrics.enabled=true
!helm install ../../../helm-charts/seldon-core --name seldon-core --namespace seldon  --set ambassador.enabled=true
!kubectl rollout status deploy/seldon-core-seldon-cluster-manager
!kubectl rollout status deploy/seldon-core-seldon-apiserver
!kubectl rollout status deploy/seldon-core-ambassador<jupyter_output><empty_output><jupyter_text>Set up REST and gRPC methods**Ensure you port forward ambassador**:```kubectl port-forward $(kubectl get pods -n seldon -l service=ambassador -o jsonpath='{.items[0].metadata.name}') -n seldon 8003:8080```  Launch main modelWe will create a very simple Seldon Deployment with a dummy model image `seldonio/mock_classifier:1.0`. This deployment is named `example`.<jupyter_code>!pygmentize model.json
!kubectl create -f model.json
!kubectl rollout status deploy/production-model-single-7cd068f<jupyter_output><empty_output><jupyter_text>Get predictions<jupyter_code>import sys
sys.path.append("../../../notebooks")
from seldon_utils import *
API_AMBASSADOR="localhost:8003"<jupyter_output><empty_output><jupyter_text>REST Request<jupyter_code>r = rest_request_ambassador("example","seldon",API_AMBASSADOR)
print(r.text)<jupyter_output><empty_output><jupyter_text>gRPC Request<jupyter_code>grpc_request_ambassador("example","seldon",API_AMBASSADOR)<jupyter_output><empty_output><jupyter_text>Launch Model with Custom RoutingWe will now create a new graph for our Canary with a new model `seldonio/mock_classifier_rest:1.1`. To make it a canary of the original `example` deployment we add two annotations```"annotations": {	    "seldon.io/ambassador-header":"location:london"	    "seldon.io/ambassador-service-name":"example"	    	},	```The first annotation says we want to route traffic that has the header `location:london`. The second says we want to use `example` as our service endpoint rather than the default which would be our deployment name - in this case `example-canary`. This will ensure that this Ambassador setting will apply to the same prefix as the previous one.<jupyter_code>!pygmentize model_with_header.json
!kubectl create -f model_with_header.json
!kubectl rollout status deploy/header-model-single-4c8805f<jupyter_output><empty_output><jupyter_text>Check a request without a header goes to the existing model.<jupyter_code>r = rest_request_ambassador("example","seldon",API_AMBASSADOR)
print(r.text)<jupyter_output>{
  "meta": {
    "puid": "n3r9g0frjb7l98k21oamov0il",
    "tags": {
    },
    "routing": {
    },
    "requestPath": {
      "classifier": "seldonio/mock_classifier:1.0"
    },
    "metrics": []
  },
  "data": {
    "names": ["proba"],
    "tensor": {
      "shape": [1, 1],
      "values": [0.07517603988385778]
    }
  }
}<jupyter_text>Check a REST request with the required header gets routed to the new model.<jupyter_code>r = rest_request_ambassador("example","seldon",API_AMBASSADOR,headers={"location":"london"})
print(r.text)<jupyter_output>{
  "meta": {
    "puid": "j7kuirsnel00kbi8qrhbpq2eb5",
    "tags": {
    },
    "routing": {
    },
    "requestPath": {
      "classifier": "seldonio/mock_classifier_rest:1.1"
    },
    "metrics": []
  },
  "data": {
    "names": ["proba"],
    "tensor": {
      "shape": [1, 1],
      "values": [0.08067696978751894]
    }
  }
}<jupyter_text>Now do the same checks with gRPC<jupyter_code>grpc_request_ambassador("example","seldon",API_AMBASSADOR)
grpc_request_ambassador("example","seldon",API_AMBASSADOR,headers={"location":"london"})<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.12245736593384014, 'fraction_numerical': 0.017464557222108074, 'mean_word_length': 5.686813186813187, 'pattern_counts': {'":': 29, '<': 43, '<?xml version=': 0, '>': 43, 'https://': 3, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '11', 'dir': 'jupyter-structured-clean-dedup', 'id': '650245', 'license': 'Apache-2.0', 'n_tokens_mistral': 1701, 'n_tokens_neox': 1610, 'n_words': 368, 'path': 'examples/ambassador/headers/ambassador_headers.ipynb', 'repo_name': 'naseemkullah/seldon-core', 'size': '4867'}