<jupyter_start><jupyter_text>Deploy US Current Population Survey Analysis Model Package from AWS Marketplace 


 The Current Population Survey (CPS) is a monthly survey of households conducted by the United States Census Bureau. It provides a comprehensive data, across geography and demographics, on the labor force, employment, wages, and job characteristics. This solution provides descriptive analysis and easy-to-understand visualization of the Basic Monthly CPS dataset: number of households, members per household, unemployment rate, employment population ratio, labor force participation, family income, non-response rate, employment details (hourly/weekly earnings, major occupation, industry) across geography (state, CSA, CBSA, county) and demographics (education, gender, race, age). This solution can assist government departments, policy consultants, and businesses to quickly generate key insights relevant to understand the U.S. economy. 


This sample notebook shows you how to deploy [US Current Population Survey Analysis](https://aws.amazon.com/marketplace/pp/prodview-2ys62bfulqtr6) using Amazon SageMaker.

> **Note**: This is a reference notebook and it cannot run unless you make changes suggested in the notebook.

 Pre-requisites:
1. **Note**: This notebook contains elements which render correctly in Jupyter interface. Open this notebook from an Amazon SageMaker Notebook Instance or Amazon SageMaker Studio.
1. Ensure that IAM role used has **AmazonSageMakerFullAccess**
1. To deploy this ML model successfully, ensure that:
    1. Either your IAM role has these three permissions and you have authority to make AWS Marketplace subscriptions in the AWS account used: 
        1. **aws-marketplace:ViewSubscriptions**
        1. **aws-marketplace:Unsubscribe**
        1. **aws-marketplace:Subscribe**  
    2. or your AWS account has a subscription to [US Current Population Survey Analysis](https://aws.amazon.com/marketplace/pp/prodview-2ys62bfulqtr6). If so, skip step: [Subscribe to the model package](1.-Subscribe-to-the-model-package)

 Contents:
1. [Subscribe to the model package](1.-Subscribe-to-the-model-package)
2. [Create an endpoint and perform real-time inference](2.-Create-an-endpoint-and-perform-real-time-inference)
   1. [Create an endpoint](A.-Create-an-endpoint)
   2. [Create input payload](B.-Create-input-payload)
   3. [Perform real-time inference](C.-Perform-real-time-inference)
   4. [Visualize output](D.-Visualize-output)
   5. [Delete the endpoint](E.-Delete-the-endpoint)
3. [Perform batch inference](3.-Perform-batch-inference) 
4. [Clean-up](4.-Clean-up)
    1. [Delete the model](A.-Delete-the-model)
    2. [Unsubscribe to the listing (optional)](B.-Unsubscribe-to-the-listing-(optional))
    

 Usage instructions
You can run this notebook one cell at a time (By using Shift+Enter for running a cell).  1. Subscribe to the model package To subscribe to the model package:
1. Open the model package listing page [US Current Population Survey Analysis](https://aws.amazon.com/marketplace/pp/prodview-2ys62bfulqtr6).
1. On the AWS Marketplace listing, click on the **Continue to subscribe** button.
1. On the **Subscribe to this software** page, review and click on **"Accept Offer"** if you and your organization agrees with EULA, pricing, and support terms. 
1. Once you click on **Continue to configuration button** and then choose a **region**, you will see a **Product Arn** displayed. This is the model package ARN that you need to specify while creating a deployable model using Boto3. Copy the ARN corresponding to your region and specify the same in the following cell.<jupyter_code>model_package_arn='arn:aws:sagemaker:us-east-2:786796469737:model-package/monthly-cps'
import base64
import json 
import uuid
from sagemaker import ModelPackage
import sagemaker as sage
from sagemaker import get_execution_role
from sagemaker import ModelPackage
from urllib.parse import urlparse
import boto3
from IPython.display import Image
from PIL import Image as ImageEdit
import urllib.request
import numpy as np
role = get_execution_role()

sagemaker_session = sage.Session()

bucket=sagemaker_session.default_bucket()
bucket<jupyter_output><empty_output><jupyter_text>2. Create an endpoint and perform real-time inference If you want to understand how real-time inference with Amazon SageMaker works, see [Documentation](https://docs.aws.amazon.com/sagemaker/latest/dg/how-it-works-hosting.html).<jupyter_code>model_name='monthly-cps'

content_type='application/zip'

real_time_inference_instance_type='ml.m5.large'
batch_transform_inference_instance_type='ml.m5.large'<jupyter_output><empty_output><jupyter_text>A. Create an endpoint<jupyter_code>def predict_wrapper(endpoint, session):
    return sage.predictor.Predictor(endpoint, session,content_type)

#create a deployable model from the model package.
model = ModelPackage(role=role,
                    model_package_arn=model_package_arn,
                    sagemaker_session=sagemaker_session,
                    predictor_cls=predict_wrapper)

#Deploy the model
predictor = model.deploy(1, real_time_inference_instance_type, endpoint_name=model_name)<jupyter_output>----!<jupyter_text>Once endpoint has been created, you would be able to perform real-time inference.  B. Create input payload<jupyter_code>file_name = './Input/input.zip'<jupyter_output><empty_output><jupyter_text>C. Perform real-time inference<jupyter_code>!aws sagemaker-runtime invoke-endpoint \
    --endpoint-name $model_name \
    --body fileb://$file_name \
    --content-type $content_type \
    --region $sagemaker_session.boto_region_name \
    output.zip<jupyter_output>{
    "ContentType": "application/zip",
    "InvokedProductionVariant": "AllTraffic"
}<jupyter_text>D. Visualize output Download zip and extract excel file to view the results  E. Delete the endpoint Now that you have successfully performed a real-time inference, you do not need the endpoint any more. You can terminate the endpoint to avoid being charged.<jupyter_code>predictor=sage.predictor.Predictor(model_name, sagemaker_session,content_type)
predictor.delete_endpoint(delete_endpoint_config=True)<jupyter_output><empty_output><jupyter_text>3. Perform batch inference In this section, you will perform batch inference using multiple input payloads together. If you are not familiar with batch transform, and want to learn more, see these links:1. [How it works](https://docs.aws.amazon.com/sagemaker/latest/dg/ex1-batch-transform.html)2. [How to run a batch transform job](https://docs.aws.amazon.com/sagemaker/latest/dg/how-it-works-batch.html)<jupyter_code>#upload the batch-transform job input files to S3
transform_input_folder = "Input"
transform_input = sagemaker_session.upload_data(transform_input_folder, key_prefix=model_name) 
print("Transform input uploaded to " + transform_input)
#Run the batch-transform job
transformer = model.transformer(1, batch_transform_inference_instance_type,max_payload=100)
transformer.transform(transform_input, content_type=content_type)
transformer.wait()
transformer.output_path
import os
s3_conn = boto3.client("s3")
with open('Output/output.zip', 'wb') as f:
    s3_conn.download_fileobj(bucket, os.path.basename(transformer.output_path)+'/input.zip.out', f)
    print("Output file loaded from bucket")<jupyter_output>Output file loaded from bucket<jupyter_text>Download zip and extract excel file to view the results  4. Clean-up  A. Delete the model<jupyter_code>model.delete_model()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.08679046490188332, 'fraction_numerical': 0.010009219017516134, 'mean_word_length': 5.084935897435898, 'pattern_counts': {'":': 2, '<': 30, '<?xml version=': 0, '>': 31, 'https://': 6, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 3, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '8', 'dir': 'jupyter-structured-clean-dedup', 'id': '413703', 'license': 'Apache-2.0', 'n_tokens_mistral': 2212, 'n_tokens_neox': 2148, 'n_words': 723, 'path': 'US Current Population Survey Analysis.ipynb', 'repo_name': 'Mphasis-ML-Marketplace/US-Current-Population-Survey-Analysis', 'size': '7593'}