<jupyter_start><jupyter_text>Module 4: Inference Patterns- endpoint based feature look up**This notebook uses the feature groups created in `module 1` and `module 2` and model trained in `module 3` to show how we can look up features from online feature store in realtime from an endpoint****Note:** Please set kernel to `Python 3 (Data Science)` and select instance to `ml.t3.medium` Contents1. [Background](Background)2. [Setup](Setup)3. [Load feature groups names](Load-feature-groups-names)4. [Load the traind model to use for inference](Load-the-traind-model-to-use-for-inference)5. [Deploy the model](Deploy-the-model)6. [Make inferences using the deployed model](Make-inferences-using-the-deployed-model)7. [Cleanup](Cleanup)  BackgroundIn this notebook, we demonstrate how to retreive features from two online feature groups within an endpoint. First we use the feature sets derived and ingested to online features stores in Modules 1 and 2 as well as the model trained in module 3 that was a SageMaker XGBoost binary classifier algorithm predicting whether a given product is added by a user to their basket.In this notebook, we will  deploy the already trained model from the model artifact on to a SageMaker endpoint for real-time inference. Our endpoint will get the features from two online features groups (customers and products feature groups created in Module 2). By sending the request body as customer id and product id we retreive the associated features from customer and product feature groups from low latency online feature stores and send them to the model endpoint for real-time inference.For this we will create a custom inference script, and specify the feature groups as well as features. We Utilise a custom library (helper.py) that faciliates returning the resutls from featurestore using some helper functions. The returned result is then fed into the model for inference.Take a few minutes reviewing the following architecture:![Inference endpoint lookup](../images/m4_nb2_inference_pattern.png "Inference endpoint feature look up")  Setup<jupyter_code>from sagemaker.serializers import CSVSerializer
from sagemaker.inputs import TrainingInput
from sagemaker.predictor import Predictor
from sagemaker import get_execution_role
import pandas as pd
import numpy as np
import sagemaker
import logging
import json
import os
import sys
sys.path.append('..')
#if required upgrade Boto3
#!pip install boto3 --upgrade
import boto3<jupyter_output><empty_output><jupyter_text>Essentials<jupyter_code>logger = logging.getLogger('__name__')
logger.setLevel(logging.DEBUG)
logger.addHandler(logging.StreamHandler())
sagemaker_execution_role = get_execution_role()
logger.info(f'Role = {sagemaker_execution_role}')
session = boto3.Session()
sagemaker_session = sagemaker.Session()
sagemaker_client = session.client(service_name="sagemaker")
default_bucket = sagemaker_session.default_bucket()
prefix = 'sagemaker-featurestore-workshop'
s3 = session.resource('s3')<jupyter_output><empty_output><jupyter_text>Load feature groups names<jupyter_code>%store -r customers_feature_group_name
%store -r products_feature_group_name<jupyter_output><empty_output><jupyter_text>Load the traind model to use for inferenceWe will use the model artificats we trained in module 3 and use it for making inference- We will customise the inference of the model as described below<jupyter_code>%store -r training_jobName

from sagemaker.xgboost.model import XGBoostModel

training_job_info = sagemaker_client.describe_training_job(
    TrainingJobName=training_jobName
)
xgb_model_data = training_job_info["ModelArtifacts"]["S3ModelArtifacts"]<jupyter_output><empty_output><jupyter_text>Deploy the modelWe deploy our model into an real-time endpoint and customise that by passing an inference script as well as our helper python library.First we prepare an inference.py entry script. If you pay attention inside the script, you will see few changes. Pay attention to the following details in the script:1. Firstly, you will see that we have defined a list that includes feature group names along side feature names, where it allows to select for a pre-defined features by specifying their names or all the features by specifying '*'. For example:To get selected features from users and product featurs stores:**`feature_list=['fscw-products-10-18-00-12: age_60-69, age_70-plus',              'fscw-customers-10-18-00-12: category_packaged_cheese']`**To get all features from users and product features stroes:  **`feature_list=['fscw-products-10-18-00-12:*',              'fscw-customers-10-18-00-12:*']`**2. Our model was built using all the features from the two feature groups and therefore we will exctract all the features.3. We are defining a custom input_fn function. Our request body is in a form of customer id and product id that calls the feature store client and loop up the feature values, using the helper library.4. You can take note of the time it takes for the features to be returned from the feature stores within the cloudwatch logs.<jupyter_code>%%writefile custom_library/inference_entry.py

import json
from io import StringIO
import os
import pickle as pkl
import joblib
import time
import sys
import subprocess
import numpy as np


import pandas as pd
import boto3
import sagemaker
import helper
import sagemaker_xgboost_container.encoder as xgb_encoders
import argparse
import os
import json
import os
import pickle as pkl
import numpy as np
import ast

boto_session = boto3.Session()
region= boto_session.region_name

#The feature list is passed as an environemnt variable to the script- feature list is defined by the client.
feature_list=os.environ['feature_list']
feature_list=ast.literal_eval(feature_list)



def model_fn(model_dir):
    """
    Deserialize and return fitted model.
    """
    model_file = "xgboost-model"
    booster = pkl.load(open(os.path.join(model_dir, model_file), "rb"))
    return booster

def input_fn(request_body, request_content_type):
    print(request_content_type)
    """
    The SageMaker XGBoost model server receives the request data body and the content type,
    and invokes the `input_fn`.
    Return a DMatrix (an object that can be passed to predict_fn).
    """
    print(request_content_type)
    if request_content_type == "text/csv":
        params =request_body.split(',')
        id_dict={'customer_id':params[0].strip(), 'product_id':params[1].strip()}
        
        start = time.time()
        records= helper.get_latest_featureset_values(id_dict, feature_list, verbose=True)
        end= time.time()
        duration= end-start
        print ("time to lookup features from two feature stores:", duration)
        
        records= ",".join([str(e) for e in records.values()])
        return xgb_encoders.csv_to_dmatrix(records)
    else:
        raise ValueError("{} not supported by script!".format(request_content_type))<jupyter_output><empty_output><jupyter_text>Prepare the featuregroup names and list of features to be retreived from the online featurestore defined by the client and passed on to the script as an environemnt variable<jupyter_code>%store -r customers_feature_group_name
%store -r products_feature_group_name

customers_fg = sagemaker_client.describe_feature_group(
    FeatureGroupName=customers_feature_group_name)

products_fg = sagemaker_client.describe_feature_group(
    FeatureGroupName=products_feature_group_name)


'''select all features from the feature group using '*' OR selected a list from the complete list of features, you can get via the following code.
customers_feats='*'
products_feats='*'

OR

customers_feats=','.join(i['FeatureName'] for i in customers_fg['FeatureDefinitions'])
products_feats=','.join(i['FeatureName'] for i in products_fg['FeatureDefinitions'])
'''

customers_feats='*'
products_feats='*'

customer_feats_desc=customers_fg["FeatureGroupName"]+ ":"+customers_feats
products_feats_desc=products_fg["FeatureGroupName"]+ ":"+products_feats

feature_list=str([customer_feats_desc,products_feats_desc])
print(feature_list)

from sagemaker.xgboost.model import XGBoostModel
from sagemaker.serializers import CSVSerializer

instance_type = "ml.m4.xlarge"

env={"feature_list": feature_list}

xgboost_inference_feature = XGBoostModel(
    model_data= xgb_model_data,
    role=sagemaker_execution_role,
    source_dir= './custom_library',
    entry_point="inference_entry.py",
    framework_version="1.2-2",
    env=env,
)

predictor_feature = xgboost_inference_feature.deploy(
    initial_instance_count=1, instance_type=instance_type)<jupyter_output><empty_output><jupyter_text>Make inferences using the deployed model<jupyter_code>predictor_feature.serializer = CSVSerializer()

cust_id='C50'
prod_id='P2'
test_data= f'{cust_id}, {prod_id}'
print(test_data)

print(predictor_feature.predict(test_data))<jupyter_output><empty_output><jupyter_text>*Optional*- Use the input_fn utilised in the inference script to evaluate how fast features are returned from the two features stores within your inference endpoint<jupyter_code>import numpy as np
import sagemaker
import time
import pandas as pd
from custom_library import helper
import ast

feature_list=ast.literal_eval(feature_list)


def input_fn(request_data, request_content_type):
    if request_content_type == "text/csv":
        params =request_data.split(',')
        id_dict={'customer_id':params[0].strip(), 'product_id':params[1].strip()}
        start = time.time()
        records= helper.get_latest_featureset_values(id_dict, feature_list)
        end= time.time()
        duration= end-start
        print("time to lookup features from two feature stores:", duration)
        records= ",".join([str(e) for e in records.values()])
        return records, duration
    else:
        raise ValueError("{} not supported by script!".format(request_content_type))

cust_id='C45'
prod_id='P26'
payload= f'{cust_id},{prod_id}'
print(payload)

input_fn(payload, "text/csv")<jupyter_output><empty_output><jupyter_text>Cleanup<jupyter_code>endpoint_name = predictor_feature.endpoint_name
print(endpoint_name)
response = sagemaker_client.describe_endpoint_config(EndpointConfigName=endpoint_name)
model_name = response['ProductionVariants'][0]['ModelName']
model_name
sagemaker_client.delete_model(ModelName=model_name) 
sagemaker_client.delete_endpoint(EndpointName=endpoint_name)
sagemaker_client.delete_endpoint_config(EndpointConfigName=endpoint_name)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.07958774692241626, 'fraction_numerical': 0.008397747876705792, 'mean_word_length': 5.7265725288831835, 'pattern_counts': {'":': 5, '<': 37, '<?xml version=': 0, '>': 37, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '9', 'dir': 'jupyter-structured-clean-dedup', 'id': '253617', 'license': 'MIT-0', 'n_tokens_mistral': 3097, 'n_tokens_neox': 3002, 'n_words': 997, 'path': '04-module-working-with-online-store/m4_nb2_Inference_endpoint_lookup.ipynb', 'repo_name': 'RominaUQ/amazon-sagemaker-feature-store-end-to-end-workshop', 'size': '10479'}