<jupyter_start><jupyter_text>Automated MLTODO: Import Dependencies. In the cell below, import all the dependencies that you will need to complete the project.<jupyter_code>from azureml.core import Workspace, Experiment
from azureml.core.compute import ComputeTarget, AmlCompute
from azureml.core.compute_target import ComputeTargetException
from azureml.data.dataset_factory import TabularDatasetFactory
import pandas as pd
from azureml.train.automl import AutoMLConfig
from azureml.widgets import RunDetails
from azureml.core.model import Model
from azureml.core.model import InferenceConfig
from azureml.core import Workspace, Environment
from azureml.core import Model
from azureml.core.webservice import AciWebservice, Webservice
import json
import joblib
# Check core SDK version number
print("SDK version:", azureml.core.VERSION)
ws = Workspace.from_config()
experiment = Experiment(workspace=ws, name="Detection")

print('Workspace name: ' + ws.name, 
      'Azure region: ' + ws.location, 
      'Subscription id: ' + ws.subscription_id, 
      'Resource group: ' + ws.resource_group, sep = '\n')

run = experiment.start_logging()
# TODO: Create compute cluster

# Choose a name for your CPU cluster
cpu_cluster_name = "auto-ml-cluster"

# Verify that cluster does not exist already
try:
    cpu_cluster = ComputeTarget(workspace=ws, name=cpu_cluster_name)
    print('Found existing cluster, use it.')
except ComputeTargetException:
    compute_config = AmlCompute.provisioning_configuration(vm_size='STANDARD_DS12_V2',
                                                           max_nodes=4)
    cpu_cluster = ComputeTarget.create(ws, cpu_cluster_name, compute_config)

cpu_cluster.wait_for_completion(show_output=True, min_node_count = 1, timeout_in_minutes = 2)<jupyter_output>Found existing cluster, use it.
Succeeded
AmlCompute wait for completion finished

Minimum number of nodes requested have been provisioned<jupyter_text>Dataset OverviewThe original dataset is in Kaggle Datasets. The original data is licensed by Open Database License (ODbL) 1.0.Open Database License (ODbL) 1.0.This data is about fraud detection in credit card transactions. The data was made by credit cards in September 2013 by European cardholders. The dataset is highly unbalanced, the positive class which depicts fraudulent transactions (frauds) account for 0.17% of all transactions.It contains only numerical input variables which are the result of a PCA transformation. Unfortunately, due to confidentiality issues, we do not have the original features and more background information about the data. Features V1, V2, ... V28 are the principal components obtained with Principal component analysis (PCA), the only features which have not been transformed with PCA are 'Time' and 'Amount'. Feature 'Time' contains the seconds elapsed between each transaction and the first transaction in the dataset. The feature 'Amount' is the transaction Amount, this feature can be used for example-dependant cost-senstive learning. Feature 'Class' is the response variable and it takes value 1 in case of fraud and 0 otherwiseThis project aims to detect potential fraud cases is credit card transactions and the task here is to differentiate between them. My ultimate intent is to tackle this situation by building classification models to classify and distinguish fraud transactions.<jupyter_code>path = 'https://media.githubusercontent.com/media/Tekhunt/Creditcard-fraud-detection/master/fraud-data.csv'
data = TabularDatasetFactory.from_delimited_files(path = path)
data.to_pandas_dataframe().head()<jupyter_output><empty_output><jupyter_text>AutoML Settings and Configuration Explained n_cross-validationHow many cross validations to perform when user validation data is not specified.enable_early_stoppingWhether to enable early termination if the score is not improving in the short term. The default is False but it is set to True here. experiment_timeout_minutessMaximum amount of time in minutes that all iterations combined can take before the experiment terminates. It is set to 15 minutes here. verbosityThis is the verbosity level for writing to the log file and it is set to logging.INFO training_dataThis can be any of these: DataFrame or Dataset or DatasetDefinition or TabularDataset The training data to be used within the experiment. It should contain both training features and a label column (optionally a sample weights column). If training_data is specified, then the label_column_name parameter must also be specified. label_column_nameThis is the name of the label column. If the input data is from a pandas.DataFrame which doesn't have column names, column indices can be used instead, expressed as integers. Here we have column headers and our arget column is the Class column which we aim to predict in the project. max_cores_per_iterationThe maximum number of threads to use for a given training iteration. Acceptable values: Equal to -1, which means to use all the possible cores per iteration per child-run. max_concurrent_iterationsRepresents the maximum number of iterations that would be executed in parallel. The value used here is 4 compute_targetThe Azure Machine Learning compute target to run the Automated Machine Learning experiment on. primary_metricThe metric that Automated Machine Learning will optimize for model selection. Accuracy is the primary_metric here. taskThe type of task to run. Values the here is 'classification'<jupyter_code># TODO: Put your automl settings here
automl_settings = {
       "n_cross_validations": 3,
       "enable_early_stopping": True,
       "experiment_timeout_minutes" :5,
       "max_concurrent_iterations": 4,
       "max_cores_per_iteration": -1,
       "verbosity": logging.INFO,
   }

# TODO: Puyour experiment name here


automl_config = AutoMLConfig(
    compute_target=cpu_cluster,
    task='classification',
    primary_metric= 'accuracy',
    training_data= data,
    label_column_name= 'Class',
    n_cross_validations=5)
# TODo your experiment name here
#remote_run = experiment.submit(automl_config)

remote_run = experiment.submit(automl_config, show_output=True)
remote_run.wait_for_completion()<jupyter_output>Running on remote.
No run_configuration provided, running on auto-ml-cluster with default configuration
Running on remote compute: auto-ml-cluster
Parent Run ID: AutoML_26986b18-0c16-4670-b009-817d60da1c62

Current status: FeaturesGeneration. Generating features for the dataset.
Current status: DatasetBalancing. Performing class balancing sweeping
Current status: DatasetCrossValidationSplit. Generating individually featurized CV splits.
Current status: ModelSelection. Beginning model selection.

****************************************************************************************************
DATA GUARDRAILS: 

TYPE:         Class balancing detection
STATUS:       ALERTED
DESCRIPTION:  To decrease model bias, please cancel the current run and fix balancing problem.
              Learn more about imbalanced data: https://aka.ms/AutomatedMLImbalancedData
DETAILS:      Imbalanced data can lead to a falsely perceived positive effect of a model's accuracy because the input data has bias t[...]<jupyter_text>Run DetailsOPTIONAL: Write about the different models trained and their performance. Why do you think some models did better than others?TODO: In the cell below, use the `RunDetails` widget to show the different experiments.<jupyter_code>RunDetails(remote_run).show()<jupyter_output><empty_output><jupyter_text>Best ModelTODO: In the cell below, get the best model from the automl experiments and display all the properties of the model.<jupyter_code>best_run, fitted_model = remote_run.get_output()
parameter_values = best_run.get_details()['runDefinition']['arguments']

# Retrieve the best automl model.

print(best_run)
print(fitted_model)

#Register the model
description ='this model predicts whether a transaction is fraudulent or not'
model_name='creditcard-fraud-detection-model'
model_path='./'
tags = None
model = remote_run.register_model(model_name = model_name, description = description , tags = tags)
print(remote_run.model_id)
print('Best Run Id: ', best_run.id)
#TODO: Save the best model<jupyter_output><empty_output><jupyter_text>Model DeploymentRemember you have to deploy only one of the two models you trained.. Perform the steps in the rest of this notebook only if you wish to deploy this model.TODO: In the cell below, register the model, create an inference config and deploy the model as a web service.<jupyter_code>#Save the best model for the deployement
import os
os.makedirs('./remote_model', exist_ok=True)

best_run.download_file('/outputs/model.pkl',os.path.join('./remote_model','best_remote_model.pkl'))

for f in best_run.get_file_names():
    if f.startswith('outputs'):
        output_file_path = os.path.join('./remote_model', f.split('/')[-1])
        print(f'Downloading from {f} to {output_file_path} ...')
        best_run.download_file(name=f, output_file_path=output_file_path)
#Register the best model for the deployement

model=best_run.register_model(
            model_name = 'creditcard-fraud-detection-model', 
            model_path = './outputs/model.pkl',
            model_framework=Model.Framework.SCIKITLEARN,
            description='The model detects whether a card transaction is fraudulent or not'
)
# Download the conda environment file and define the environement
best_run.download_file('outputs/conda_env_v_1_0_0.yml', 'conda_env.yml')
myenv = Environment.from_conda_specification(name = 'myenv',
                                             file_path = 'conda_env.yml')
# download the scoring file produced by AutoML
best_run.download_file('outputs/scoring_file_v_1_0_0.py', 'score_remote_run.py')

# set inference config
inference_config = InferenceConfig(entry_script= 'score_remote_run.py',
                                    environment=myenv)
# set Aci Webservice config
aci_config = AciWebservice.deploy_configuration(cpu_cores=1, memory_gb=1, auth_enabled=False)
# deploye the model as a web service
service_name =  'automl-prediction'
service = Model.deploy(workspace=ws, 
                       name= service_name,
                       models=[model], 
                       inference_config=inference_config,
                       deployment_config=aci_config,
                       overwrite=True)<jupyter_output><empty_output><jupyter_text>TODO: In the cell below, send a request to the web service you deployed to test it.<jupyter_code>service
# wait for deployment to finish and display the scoring uri and swagger uri
service.wait_for_deployment(show_output=True)

print('Service state:')
print(service.state)

print('Scoring URI:')
print(service.scoring_uri)

print('Swagger URI:')
print(service.swagger_uri)
import requests
import json

# URL for the web service
scoring_uri = 'http://2beb2e57-4e0a-4beb-889f-4da2184753a5.southcentralus.azurecontainer.io/score'


data = {"data":
        [
            [
                0.0,
                1.359807,
                -0.072781,
                2.536347,
                1.378155,
                -0.338321,
                0.462388,
                0.239599,
                0.098698,
                0.363787,
                0.090794172,
                -0.551599533,
                -0.617800856,
                -0.991389847,
                -0.311169354,
                1.468176972,
                -0.470400525,
                0.207971242,
                0.02579058,
                0.40399296,
                0.251412098,
                -0.018306778,
                0.277837576,
                -0.11047391,
                0.066928075,
                0.128539358,
                -0.189114844,
                0.133558377,
                -0.021053053,
                149.62
            ],
            [
                0.0,
                1.191857111,
                0.266150712,
                0.166480113,
                0.448154078,
                0.060017649,
                -0.082360809,
                -0.078802983,
                0.085101655,
                -0.255425128,
                -0.166974414,
                1.612726661,
                1.065235311,
                0.489095016,
                -0.143772296,
                0.635558093,
                0.463917041,
                -0.114804663,
                -0.18336127,
                -0.145783041,
                -0.069083135,
                -0.225775248,
                -0.638671953,
                0.101288021,
                -0.339846476,
                0.167170404,
                0.125894532,
                -0.008983099,
                0.014724169,
                2.69
            ]
        ]
        }
        
# Convert to JSON string
input_data = json.dumps(data)

# Set the content type
headers = {'Content-Type': 'application/json'}
# If authentication is enabled, set the authorization header
#headers['Authorization'] = f'Bearer {key}'

# Make the request and display the response
resp = requests.post(scoring_uri, input_data, headers=headers)
print(resp.text)<jupyter_output>"{\"result\": [0, 0]}"<jupyter_text>TODO: In the cell below, print the logs of the web service and delete the service<jupyter_code>print(service.get_logs())

service.delete()
cpu_cluster.delete()<jupyter_output>Current provisioning state of AmlCompute is "Deleting"
{'fraction_non_alphanumeric': 0.07745638884747279, 'fraction_numerical': 0.04681675861040704, 'mean_word_length': 3.2519809825673534, 'pattern_counts': {'":': 7, '<': 29, '<?xml version=': 0, '>': 29, 'https://': 2, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 12, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '8', 'dir': 'jupyter-structured-clean-dedup', 'id': '181793', 'license': 'OML', 'n_tokens_mistral': 4092, 'n_tokens_neox': 3642, 'n_words': 1226, 'path': 'automl_run1.ipynb', 'repo_name': 'Tekhunt/Creditcard-fraud-dictection', 'size': '13414'}