<jupyter_start><jupyter_text>Amazon Fraud Detector - Send Events ExampleOnce you’ve defined an event, you can start to ingest your transactions to fraud detector. This is an example notebook shows you how to stream your events from a file stored in S3 to Amazon Fraud Detector. Setup permissions------First, setup your AWS credentials so that Fraud Detector can store and access training data. See more details: https://docs.aws.amazon.com/frauddetector/latest/ug/set-up.htmlTo use Amazon Fraud Detector, you have to set up permissions that allow access to the Amazon Fraud Detector console and API operations. You also have to allow Amazon Fraud Detector to perform tasks on your behalf and to access resources that you own. We recommend creating an AWS Identify and Access Management (IAM) user with access restricted to Amazon Fraud Detector operations and required permissions. You can add other permissions as needed. If you are using SageMaker Notebook Instance, attach the two policies __*AmazonFraudDetectorFullAccessPolicy*__ and __*AmazonS3FullAccess*__ to the Instance's IAM role and restart your kernel. Preparation-----Before this step, you should have:1. created a S3 bucket and upload the csv file you want to send to Amazon Fraud Detector https://docs.aws.amazon.com/frauddetector/latest/ug/step-1-get-s3-data.html2. defined your Entity and Event on Amazon Fraud Detector console  https://docs.aws.amazon.com/frauddetector/latest/ug/define-event.html 💡  Concepts - ENTITY represents the "what or who" that is performing the event you want to evaluate- EVENT is a business activity that you want evaluated for fraud risk<jupyter_code># -- Import packages -- 
import os
import logging
import boto3
import pandas as pd
import numpy as np
from datetime import datetime
from multiprocessing import Pool

# -- Display --
from IPython.core.display import display, HTML
from IPython.display import clear_output
display(HTML("<style>.container { width:90% }</style>"))
# -- initialize the AFD client 
client = boto3.client('frauddetector')<jupyter_output><empty_output><jupyter_text>Setup Notebook-----1. Plug in your S3 Bucket and CSV File     2. Plug in the major components of Fraud Detector you have defined on the console   3. Plug in the column names of the major components 💡  Plug in Column Names of the Major Components:     - ENTITY_ID_COL is the column name of entity id, e.g. "customer_id"- EVENT_ID_COL is the column name of event id, e.g. "transaction_id"- TIMESTAMP_COL is the column name of event timestamp (when the event happens), e.g. "EVENT_TIMESTAMP"- LABEL_COL is the column name of the target variable, e.g. "EVENT_LABEL"<jupyter_code># -- this is all you need to fill out, once complete simply interactively run each code cell --  
S3_BUCKET      = "your-s3-bucket"                
S3_FILE        = "path-to-your-file"             

EVENT_TYPE     = "event-type-defined-on-console"

ENTITY_ID_COL  = "column-name-of-entity-id"      
EVENT_ID_COL   = "column-name-of-event-id"       
TIMESTAMP_COL  = "column-name-of-event-timestamp"
LABEL_COL      = "column-name-of-event-label"<jupyter_output><empty_output><jupyter_text>Load Your Dataset from S3-----<jupyter_code># -- connect to S3, snag file, and convert to a panda's dataframe --
s3   = boto3.resource('s3')
obj  = s3.Object(S3_BUCKET, S3_FILE)
body = obj.get()['Body']
df   = pd.read_csv(body, dtype=object)

# -- convert the event timestamp to standard format --
df[TIMESTAMP_COL] = pd.to_datetime(df[TIMESTAMP_COL]).apply(lambda x: x.strftime("%Y-%m-%dT%H:%M:%SZ"))

print("Number of events to send:", df.shape[0])<jupyter_output>Number of events to send: 116715<jupyter_text>Send Events-----The following section will turn on the eventIngestion and automatically stream your data from s3 bucket to Amazon Fraud Detector._**Note**: the following code use multiprocessing package to parallelize the_ `send_event` _call_<jupyter_code>#record_count = 5000           # -- send 5000 records for testing -- 
record_count = df.shape[0]     # -- uncomment this to run all of the records -- 

# -- check the eventType, if eventIngestion is disabled, enable it before send events
response = client.get_event_types(name = EVENT_TYPE)
event_variables = response['eventTypes'][0]['eventVariables']
event_labels = response['eventTypes'][0]['labels']
entity_type  = response['eventTypes'][0]['entityTypes'][0]

if response['eventTypes'][0]['eventIngestion'] != 'ENABLED':
    response = client.put_event_type (
        name           = EVENT_TYPE,
        eventVariables = event_variables,
        labels         = event_labels,
        eventIngestion = 'ENABLED',
        entityTypes    = [entity_type])
    response = client.get_event_types(name = EVENT_TYPE)
print('Event Ingestion of eventType "{0}" has been {1}.'.format(EVENT_TYPE, response['eventTypes'][0]['eventIngestion']))
%%time
def _send_event(record):
    event_id = str(record[0])
    entity_id = str(record[1])
    event_timestamp = str(record[2])
    event_label = record[3]
    
    try:
        rec_content = {event_variables[i]: record[4:][i] for i in range(len(event_variables)) if pd.isnull(record[4+i])==False} 
        if pd.isnull(event_label):
            # If event label is missing, do not assign label 
            response = client.send_event(
                eventId        = event_id,
                eventTypeName  = EVENT_TYPE,
                eventTimestamp = event_timestamp,
                eventVariables = rec_content,
                entities = [
                    {
                        'entityType': entity_type,
                        'entityId': entity_id
                    },
                ]
            )
        else:
            response = client.send_event(
                eventId        = event_id,
                eventTypeName  = EVENT_TYPE,
                eventTimestamp = event_timestamp,
                eventVariables = rec_content,
                assignedLabel  = str(event_label),
                labelTimestamp = event_timestamp,
                entities = [
                    {
                        'entityType': entity_type,
                        'entityId': entity_id
                    },
                ]
            )
        http_status_code = response['ResponseMetadata']['HTTPStatusCode']
        if http_status_code != 200:
            logging.error('EventId: {0}, httpStatusCode: {1}'.format(event_id, str(http_status_code)))
    except Exception as e:
        logging.error('EventId: {0}, Failed to send_event: {1}'.format(event_id, str(e)))

        
# -- send events in parallel --
cols_keep = [EVENT_ID_COL, ENTITY_ID_COL, TIMESTAMP_COL, LABEL_COL] + event_variables
df_list = df.iloc[0:record_count,:]
df_list = df_list[cols_keep].values.tolist()
with Pool(processes = 8) as p:
    result = p.map(_send_event, df_list)<jupyter_output>CPU times: user 1.67 s, sys: 334 ms, total: 2.01 s
Wall time: 14min 57s
{'fraction_non_alphanumeric': 0.0947155645394167, 'fraction_numerical': 0.01140629511983829, 'mean_word_length': 3.0819092516205067, 'pattern_counts': {'":': 0, '<': 17, '<?xml version=': 0, '>': 17, 'https://': 3, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '4', 'dir': 'jupyter-structured-clean-dedup', 'id': '569729', 'license': 'MIT-0', 'n_tokens_mistral': 2060, 'n_tokens_neox': 1991, 'n_words': 648, 'path': 'Fraud_Detector_Send_Event.ipynb', 'repo_name': 'SheliaXin/aws-fraud-detector-samples', 'size': '6926'}