<jupyter_start><jupyter_text>The police department of your city has received lots of complaints about its stop and search policy. Every time a car is stopped, the police officers have to decide whether or not to search the car for contraband. According to critics, these searches have a bias against people of certain backgrounds.Your company has been hired to help with this situation, and you were assigned with creating a service to decide whether or not to search a car, based on objective data. This service will be used by police officers to request authorization to search, and your service will return a Yes or No answer.These exercises will guide you on your task, step by step.  1. Meet the dataStart by getting familiar with the dataset, in file `connecticut_dataset_small.csv`.Each row in the dataset is about one car search operation, and has 4 fields.3 features:- **StatuteReason**: what is the reason why the car was stopped- **SubjectAge**: the driver's age- **InterventionDateTime**: when the car search operation occured; date presented in the American way: month/day/yearAnd the target:- **ContrabandIndicator**: whether contraband and/or evidence were foundWe'll consider that a search was successful if contraband and/or evidence were found.<jupyter_code>df = pd.read_csv(os.path.join("data", "connecticut_dataset_small.csv"))
df.head(2)
# Create a DataFrame with the 3 features: StatuteReason, SubjectAge, InterventionDateTime
# Keep them in this order
# X = ...

# YOUR CODE HERE
raise NotImplementedError()


# Create a series with the target: ContrabandIndicator
# y = ...
# YOUR CODE HERE
raise NotImplementedError()
assert isinstance(X, pd.DataFrame)
assert X.columns.tolist() == ["StatuteReason", "SubjectAge", "InterventionDateTime"]

assert isinstance(y, pd.Series)<jupyter_output><empty_output><jupyter_text>2. Build a modelBuild a scikit model that predicts whether a car search was successful, based on the features that you have available. Your model should be delivered as a scikit [Pipeline](https://scikit-learn.org/stable/modules/generated/sklearn.pipeline.Pipeline.htmlsklearn.pipeline.Pipeline.predict_proba).Don't worry too much about the model's performance, anything better than random works! We'll focus on model performance in the next BLUs.<jupyter_code># Create the pipeline with your model
# pipeline = ...
# YOUR CODE HERE
raise NotImplementedError()
assert isinstance(pipeline, Pipeline)
# Use cross validation with 5 folds and ROC_AUC as metric, to check your model's performance
# roc_aucs = cross_val_score(...)
# YOUR CODE HERE
raise NotImplementedError()
assert roc_aucs.mean() > 0.5
# Now fit the pipeline to all the training data
# YOUR CODE HERE
raise NotImplementedError()
assert pipeline.predict_proba(X.head(1)).shape == (1, 2)<jupyter_output><empty_output><jupyter_text>3. Serialize all the things!Now we need to serialize three things:1. The column names in the correct order1. The fitted pipeline1. The dtypes of the columns of the training set<jupyter_code># This is a temporary directory where your serialized files will be saved
# You can change it while working on the exercises locally,
# but change it back to TMP_DIR = '/tmp' before submitting the exercises,
# otherwise grading will fail
TMP_DIR = '/tmp'
# Serialize the column names from the X DataFrame into a file named columns.json
# YOUR CODE HERE
raise NotImplementedError()
with open(os.path.join(TMP_DIR, "columns.json"), 'r') as fh:
    columns = json.load(fh)
    
assert columns == X.columns.tolist()
# Pickle the dtypes of the columns from the X DataFrame into a file named dtypes.pickle
# YOUR CODE HERE
raise NotImplementedError()
with open(os.path.join(TMP_DIR, "dtypes.pickle"), 'rb') as fh:
    dtypes = pickle.load(fh)
    
assert dtypes.equals(X.dtypes)
# Pickle the fitted pipeline into a file named pipeline.pickle
# YOUR CODE HERE
raise NotImplementedError()
pipeline_recovered = joblib.load(os.path.join(TMP_DIR, "pipeline.pickle"))

assert isinstance(pipeline_recovered, Pipeline)
assert pipeline_recovered.predict_proba(X.head(1)).shape == (1, 2)<jupyter_output><empty_output><jupyter_text>4. Create a new repo for your serviceNow it's time to create a new repo for your service. As you learned in the README of the [heroku-model-deploy repository](https://github.com/LDSSA/heroku-model-deploy), duplicate the heroku-model-deploy repo.From this point on, you should code on the new repo. The remaining exercises in this notebook are questions meant to check if your service is working as expected.After you've setup your new repo, copy the following things over there:- `columns.json` file- `dtypes.pickle` file- `pipeline.pickle` file- the package containing custom code in your model (only if you've used it, of course!).  5. Build your flask app /predictAt this point, you can either edit the `app.py` file that's in the repo, or start a new file from scratch.My advice is that you start one from scratch, as it will probably be a better learning experience.Start by creating a `predict` endpoint, that should receive POST requests, and a JSON payload with:- id- observation, which has 3 fields: StatuteReason, SubjectAge, and InterventionDateTime.This endpoint should return the proba returned by your model for this observation.Make sure that each field is in the correct format before passing it to the scikit model. If you receive an observation with an invalid value, return an appropriate error message.When a request is received, you should update your local sqlite database with the following:- id- observation- proba- true_class (which is null for now)In case your app has received an observation with an id that it has seen before, it should return an error message, the correspondent proba, and don't store anything.Try the following commands to check that everything is working as expected. **Command**```bash~ > curl -X POST http://localhost:5000/predict -d '{"id": 0, "observation": {"StatuteReason": "Registration", "SubjectAge": 22.0, "InterventionDateTime": "11/08/2014 11:38:00 AM"}}' -H "Content-Type:application/json"```**Expected output**```json{  "proba": 0.3192823701310136}```(any proba value works, it depends on your model, of course!) **Command**```bash~ > curl -X POST http://localhost:5000/predict -d '{"id": 0, "observation": {"StatuteReason": "Registration", "SubjectAge": 22.0, "InterventionDateTime": "11/08/2014 11:38:00 AM"}}' -H "Content-Type:application/json"```**Expected output**```json{  "error": "Observation ID: \"0\" already exists",  "proba": 0.3192823701310136}``` **Command**```bashcurl -X POST http://localhost:5000/predict -d '{"id": 1, "observation": {"StatuteReason": "Registration", "SubjectAge": "hello", "InterventionDateTime": "11/08/2014 11:38:00 AM"}}' -H "Content-Type:application/json"```**Expected output**```json{  "error": "Observation is invalid!"}```<jupyter_code># When the predict endpoint of your flask app is working as expected,
# set variable predict_endpoint_working_fine to True
predict_endpoint_working_fine = False

# YOUR CODE HERE
raise NotImplementedError()
assert predict_endpoint_working_fine<jupyter_output><empty_output><jupyter_text>/updateThe update endpoint should receive POST requests, and a JSON payload with:- id- true_classIf there is an observation with `id` in your database, you should update the `true_class` value with the value in the request. The response should be the observation, with the updated true_class value.Otherwise, you should return an appropriate error message.Try the following commands to check that everything is working as expected. **Command**```bash~ > curl -X POST http://localhost:5000/update -d '{"id": 0, "true_class": 1}'  -H "Content-Type:application/json"```**Expected output**```json{  "id": 1,  "observation": "{\"id\": 0, \"observation\": {\"StatuteReason\": \"Registration\", \"SubjectAge\": 22.0}}",  "observation_id": 0,  "proba": 0.3192823701310136,  "true_class": 1}``` **Command**```bash~ > curl -X POST http://localhost:5000/update -d '{"id": 3, "true_class": 1}'  -H "Content-Type:application/json"```**Expected output**```json{  "error": "Observation ID: \"3\" does not exist"}```<jupyter_code># When the predict endpoint of your flask app is working as expected,
# set variable update_endpoint_working_fine to True
update_endpoint_working_fine = False

# YOUR CODE HERE
raise NotImplementedError()
assert update_endpoint_working_fine<jupyter_output><empty_output><jupyter_text>6. Deploy your app to herokuFollow the instructions on the Learning part of this BLU to deploy your app to heroku.In order to check that your app is working correctly on heroku, re-run the previous commands, but replacing the `localhost` with the URL of your heroku app (like `https://.herokuapp.com`). For instance, the first command would be:**Command**```bash~ > curl -X POST https://.herokuapp.com/predict -d '{"id": 0, "observation": {"StatuteReason": "Registration", "SubjectAge": 22.0, "InterventionDateTime": "11/08/2014 11:38:00 AM"}}' -H "Content-Type:application/json"```**Expected output**```json{  "proba": 0.3192823701310136}```<jupyter_code># In this test, we will call your app to check if it's working as expected
# Assign the variable APP_NAME to the name of your heroku app
# APP_NAME = ...

# YOUR CODE HERE
raise NotImplementedError()
# Testing the /predict endpoint

url = f"http://{APP_NAME}.herokuapp.com/predict"
payload = {
    "id": 0,
    "observation": {
        "StatuteReason": "Registration",
        "SubjectAge": 22.0,
        "InterventionDateTime": "11/08/2014 11:38:00 AM"
    }
}

r = requests.post(url, json=payload)

assert isinstance(r, requests.Response)
assert r.ok
assert "proba" in r.json()
assert isinstance(r.json()["proba"], float)
assert 0 <= r.json()["proba"] <= 1
# Testing the /update endpoint

url = f"http://{APP_NAME}.herokuapp.com/update"
payload = {
    "id": 0,
    "true_class": 1
}

r = requests.post(url, json=payload)

assert isinstance(r, requests.Response)
assert r.ok
assert "observation" in r.json()
assert "proba" in r.json()
assert "true_class" in r.json()
assert r.json()["true_class"] == 1<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.10964437001280662, 'fraction_numerical': 0.021278691754506945, 'mean_word_length': 5.297766749379653, 'pattern_counts': {'":': 46, '<': 27, '<?xml version=': 0, '>': 31, 'https://': 4, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '6', 'dir': 'jupyter-structured-clean-dedup', 'id': '189359', 'license': 'MIT', 'n_tokens_mistral': 3092, 'n_tokens_neox': 2851, 'n_words': 1189, 'path': 'S06 - DS in the Real World/BLU13 - Basic Model Deployment/Exercise notebook.ipynb', 'repo_name': 'LDSSA/batch4-students', 'size': '10151'}