<jupyter_start><jupyter_text>After model.fit, before you deploy: Prototype with FastAPI in Jupyter!_By <NAME>, for JupyterCon 2020_You want to deploy your scikit-learn model. Now what? You can make an API for your model in Jupyter!You’ll learn [FastAPI](https://fastapi.tiangolo.com/), a Python web framework with automatic interactive docs. We’ll validate inputs with type hints, and convert to a dataframe, to make new predictions with your model. You’ll have a working API prototype, running from a notebook and ready to deploy! This talk is for people who feel comfortable in notebooks and can fit scikit-learn models. It’s about the technical process in-between developing your model and deploying it. Maybe you’ve never deployed an API before, or maybe you’ve tried Flask but you’re curious about FastAPI.  Part 0, model.fit  We'll use the [Palmer Penguins](https://github.com/allisonhorst/palmerpenguins) dataset. It's an alternative to [Iris](https://en.wikipedia.org/wiki/Iris_flower_data_set). Instead of using Iris flower measurements to predict one of three species, we'll use penguin measurements to predict one of three species. Artwork by [@allison_horst](https://twitter.com/allison_horst) First, load and explore the data:<jupyter_code>import seaborn as sns
penguins = sns.load_dataset('penguins')
sns.pairplot(data=penguins, hue='species')<jupyter_output><empty_output><jupyter_text>Looks like Adelie penguins have less bill length. Gentoo penguins have less bill depth, more flipper length, and more body mass.So we can classify the three species using two features: bill length and another numeric feature, such as bill depth. Artwork by [@allison_horst](https://twitter.com/allison_horst) We'll select `bill_length_mm` and `bill_depth_mm` for our features, and `species` is our target. We'll use scikit-learn to fit a Logistic Regression model. Scikit-learn's implementation of Logistic Regression is regularized. We'll use cross-validation to automate the amount of regularization, after scaling the features. We can combine the scaler transformation and the model into a scikit-learn pipeline. We'll also use cross-validation to estimate how accurately the model generalizes.<jupyter_code>from sklearn.linear_model import LogisticRegressionCV
from sklearn.preprocessing import StandardScaler
from sklearn.model_selection import cross_val_score
from sklearn.pipeline import make_pipeline

features = ['bill_length_mm', 'bill_depth_mm']
target = 'species'

penguins.dropna(subset=features, inplace=True)
X = penguins[features]
y = penguins[target]

classifier = make_pipeline(
    StandardScaler(), 
    LogisticRegressionCV()
)

classifier.fit(X, y)

scores = cross_val_score(classifier, X, y)
avg_acc = scores.mean() * 100
std_acc = scores.std() * 100
print(f'Cross-Validation Accuracy: {avg_acc:.0f}% +/- {2*std_acc:.0f}%')<jupyter_output>Cross-Validation Accuracy: 96% +/- 4%<jupyter_text>So, our model seems to classify penguins nearly perfectly.Next, we'll deploy this model in a FastAPI app.  Web apps aren't usually served from notebooks, especially temporary cloud notebooks like Binder. But it can be useful for rapid prototyping. Here's a helper function to make it possible:<jupyter_code>def enable_cloud_notebook(port=8000):
    """
    Enables you to run a FastAPI app from a cloud notebook.
    Useful for rapid prototyping if you like notebooks!
    Not needed when you develop in a local IDE or deploy "for real."
    """

    # Prevent "RuntimeError: This event loop is already running"
    import nest_asyncio
    nest_asyncio.apply()

    # Get a public URL to the localhost server 
    from pyngrok import ngrok
    print('Public URL:', ngrok.connect(port=port))<jupyter_output><empty_output><jupyter_text>Part 1, random penguins, GET request Let's back up and begin with something like "Hello World." Before we make real predictions, we’ll make random guesses.<jupyter_code>import random

def random_penguin():
    """Return a random penguin species"""
    return random.choice(['Adelie', 'Chinstrap', 'Gentoo'])<jupyter_output><empty_output><jupyter_text>Run this function and you'll get random penguin species.<jupyter_code>random_penguin()<jupyter_output><empty_output><jupyter_text>In the next cell, you'll see that we add a half-dozen lines of code to turn this function into a FastAPI app.These lines create a FastAPI app instance:```pythonfrom fastapi import FastAPIapp = FastAPI()```This decorator tells FastAPI to call the function whenever the app receives a request to the `/` path using the HTTP GET method.```python@app.get('/')def random_penguin():    ...```This line enables running FastAPI from a cloud notebook:```pythonenable_cloud_notebook()```These lines run the app with Uvicorn, the recommended web server for FastAPI:```pythonimport uvicornuvicorn.run(app)```The code below puts it all together. Run the cell. You'll see a "Public URL" that ends in "ngrok.io". Click the link to open it in a new tab. You'll see a random penguin species. Refresh the tab to get another random penguin.<jupyter_code>import random

from fastapi import FastAPI
import uvicorn

app = FastAPI()

@app.get('/')
def random_penguin():
    """Return a random penguin species"""
    species = random.choice(['Adelie', 'Chinstrap', 'Gentoo'])
    return species

enable_cloud_notebook()
uvicorn.run(app)<jupyter_output><empty_output><jupyter_text>Every time you refresh you see it in the web logs above. The app is up on the public internet for anyone to access, but only while this cell in this notebook is running. In this notebook, stop the cell from running now. Next we'll add an app `title`, change the `docs_url` parameter, and change the path to `/random` for the `random_penguin` function.Run the cell and click the new Public URL.<jupyter_code>import random

from fastapi import FastAPI
import uvicorn

app = FastAPI(
    title='🐧 Penguin predictor API',
    docs_url='/'
)

@app.get('/random')
def random_penguin():
    """Return a random penguin species"""
    species = random.choice(['Adelie', 'Chinstrap', 'Gentoo'])
    return species

enable_cloud_notebook()
uvicorn.run(app)<jupyter_output><empty_output><jupyter_text>Now you'll see automatically generated documentation. It's interactive too! Click on "/random", then the "Try It Out" button, then the "Execute" button. Scroll down to the "Server response." You'll see "Response body" with a penguin species, and "Code" with 200 which is a successful [status code](https://developer.mozilla.org/en-US/docs/Web/HTTP/Status). Or change the end of the URL to `/random` and you can use the API directly. You access your API with code like this, from another notebook or Python shell. Replace the url with your own dynamically generated ngrok URL.```pythonimport requestsurl = 'http://9571e5899f73.ngrok.io/random'response = requests.get(url)print(response.status_code, response.text)``` Then stop the cell above from running like before.  Part 2, real predictions, POST request Okay, now let's work on adding our model to make real predictions. To make a prediction, we need penguin measurements, which we'll receive as [JSON](https://developer.mozilla.org/en-US/docs/Learn/JavaScript/Objects/JSON): > JavaScript Object Notation (JSON) is a standard text-based format for representing structured data based on JavaScript object syntax. It is commonly used for transmitting data in web applications (e.g., sending some data from the server to the client, so it can be displayed on a web page, or vice versa). You'll come across it quite often ... it can be used independently from JavaScript, and many programming environments feature the ability to read (parse) and generate JSON. JSON looks a lot like a Python dictionary, like this example:<jupyter_code>gary_gentoo = {"bill_length_mm": 45, "bill_depth_mm": 15}<jupyter_output><empty_output><jupyter_text>How do we go from JSON / dictionary format to something our model can use?We need a Numpy array or a Pandas dataframe, with two columns (for our two features) and one row (for our one observation that we want to predict). We can make a dataframe from a list of dicts, like this:<jupyter_code>import pandas as pd
pd.DataFrame([gary_gentoo])<jupyter_output><empty_output><jupyter_text>When we use this dataframe with our classifier's predict method, we get the correct result.<jupyter_code>import pandas as pd
df = pd.DataFrame([gary_gentoo])
classifier.predict(df)<jupyter_output><empty_output><jupyter_text>The predict method returns a Numpy array with all our predictions. But we're just making a single prediction, so we want the "zeroeth" item from the array. Putting it all together, we could write a function like this:<jupyter_code>def predict_species(penguin: dict):
    """Predict penguin species"""
    df = pd.DataFrame([penguin])
    species = classifier.predict(df)
    return species[0]
predict_species(gary_gentoo)<jupyter_output><empty_output><jupyter_text>Here's another example.<jupyter_code>amy_adelie = {"bill_length_mm": 35, "bill_depth_mm": 18}
predict_species(amy_adelie)<jupyter_output><empty_output><jupyter_text>We'll add the function to our FastAPI app using a decorator. The decorator tells FastAPI to call the function whenever the app receives a request to the `/predict` path using the [HTTP POST method](https://developer.mozilla.org/en-US/docs/Web/HTTP/Methods/POST). FastAPI will automatically parse the request body's JSON to a Python dict named `penguin`.```python@app.post('/predict')def predict_species(penguin: dict):    ...```We'll also add a more descriptive `description` parameter to the app. Putting it all together:<jupyter_code>import random

from fastapi import FastAPI
import pandas as pd
import uvicorn

app = FastAPI(
    title='🐧 Penguin predictor API', 
    description='Deploys a logistic regression model fit on the [Palmer Penguins](https://github.com/allisonhorst/palmerpenguins) dataset.', 
    docs_url='/'
)


@app.post('/predict')
def predict_species(penguin: dict):
    """Predict penguin species"""
    df = pd.DataFrame([penguin])
    species = classifier.predict(df)
    return species[0]


@app.get('/random')
def random_penguin():
    """Return a random penguin species"""
    species = random.choice(['Adelie', 'Chinstrap', 'Gentoo'])
    return species


enable_cloud_notebook()
uvicorn.run(app)<jupyter_output>INFO:     Started server process [16616]
INFO:     Waiting for application startup.
INFO:     Application startup complete.
INFO:     Uvicorn running on http://127.0.0.1:8000 (Press CTRL+C to quit)<jupyter_text>Run the cell above, then try an example:- Click the "Try it out" button.- The "Request body" text field becomes editable. Copy-paste <NAME>'s measurements into the field: `{"bill_length_mm": 45, "bill_depth_mm": 15}`- Click the "Execute" button, then scroll down to the "Server response." You should see the species "Gentoo" correctly classified.Try another example:- Copy-paste <NAME>'s measurements into the "Request body" text field: `{"bill_length_mm": 35, "bill_depth_mm": 18}`- Click the "Execute" button. You should see the species "Adelie" correctly classified.But what happens if you change your "Request body" to something unexpected?- What if your input doesn't have exactly two keys, `bill_length_mm` and `bill_depth_mm`, in that order?- What if your input values are zero? Huge numbers? Negative numbers? Not a number?We aren't validating input yet. We just assume the API users give valid input. That's a dangerous assumption. When the inputs aren't valid, the app may respond with a Server Error instead of helpful warnings. Or worse, the app seems to work and returns a response, but because the inputs were flawed, the output is flawed too. "Garbage in, garbage out." Stop the cell above from running. Next we'll add data validation.  Part 3, Data validation Look at the type annotation for the `predict_species` function's argument. The function accepts any `dict`. ```python@app.post('/predict')def predict_species(penguin: dict):    ...```We'll change this so the function expects an argument of type `Penguin`.```pythonclass Penguin:    """Parse & validate penguin measurements"""    ...@app.post('/predict')def predict_species(penguin: Penguin):    ...``` We'll create a `Penguin` [data class](https://docs.python.org/3/library/dataclasses.html) with [type annotations](https://docs.python.org/3/library/typing.html) to define what attributes we expect our input to have. We'll use [Pydantic](https://pydantic-docs.helpmanual.io/), a data validation library integrated with FastAPI. It sounds complex, but it's just a few lines of code!<jupyter_code>from pydantic import BaseModel

class Penguin(BaseModel):
    """Parse & validate penguin measurements"""
    bill_length_mm: float
    bill_depth_mm: float<jupyter_output><empty_output><jupyter_text>We can instantiate a penguin object like this:<jupyter_code>Penguin(bill_length_mm=45, bill_depth_mm=15)<jupyter_output><empty_output><jupyter_text>Or like this, by unpacking our dictionary into parameters:<jupyter_code>Penguin(**gary_gentoo)<jupyter_output><empty_output><jupyter_text>Now let's see what happens with missing input:<jupyter_code>missing_input = {"bill_length_mm": 45}
Penguin(**missing_input)<jupyter_output><empty_output><jupyter_text>We automatically get a `ValidationError` with a helpful, descriptive error message. That's what we want in this situation! Next we'll try a misnamed input (`bill_depth` instead of `bill_depth_mm`)<jupyter_code>wrong_name = {"bill_length_mm": 45, "bill_depth": 15}
Penguin(**wrong_name)<jupyter_output><empty_output><jupyter_text>Again, we get a `ValidationError`, which is want we want here. Let's try an input with the wrong type, such as a string instead of a number.<jupyter_code>wrong_type = {"bill_length_mm": 45, "bill_depth_mm": "Hello Penguins!"}
Penguin(**wrong_type)<jupyter_output><empty_output><jupyter_text>We get a different `ValidationError` because the value is not a valid float. Let's try a different string:<jupyter_code>convertable_type = {"bill_length_mm": 45, "bill_depth_mm": "15"}
Penguin(**convertable_type)<jupyter_output><empty_output><jupyter_text>This works because the string can be converted to a float.  If we add an extra input ...<jupyter_code>extra_input = {"bill_length_mm": 45, "bill_depth_mm": 15, "extra_feature": "will be ignored"}
Penguin(**extra_input)<jupyter_output><empty_output><jupyter_text>... it will be ignored. If we flip the order of inputs ...<jupyter_code>flipped_order = {"bill_depth_mm": 15, "bill_length_mm": 45}
Penguin(**flipped_order)<jupyter_output><empty_output><jupyter_text>... they'll be flipped back. What about penguin measurements that are implausibly large or small? We can use "constrained floats" to catch this.We'll set constraints that each input must be greater than (`gt`) some minimum and less than (`lt`) some maximum.<jupyter_code>from pydantic import confloat
help(confloat)<jupyter_output><empty_output><jupyter_text>First, let's look at the minimum and maximum measurements from our training data:<jupyter_code>X.describe()<jupyter_output><empty_output><jupyter_text>Then, set some reasonable constraints:<jupyter_code>from pydantic import BaseModel, confloat

class Penguin(BaseModel):
    """Parse & validate penguin measurements"""
    bill_length_mm: confloat(gt=32, lt=60)
    bill_depth_mm: confloat(gt=13, lt=22)<jupyter_output><empty_output><jupyter_text>Now when inputs are too large or small, we get a `ValidationError` with descriptive messages.<jupyter_code>huge_penguin = {"bill_depth_mm": 1500, "bill_length_mm": 4500}
Penguin(**huge_penguin)
zero_penguin = {"bill_depth_mm": 0, "bill_length_mm": 0}
Penguin(**zero_penguin)
negative_penguin = {"bill_depth_mm": -45, "bill_length_mm": -15}
Penguin(**negative_penguin)<jupyter_output><empty_output><jupyter_text>One more thing. Let's add a helpful method to our class:<jupyter_code>from pydantic import BaseModel, confloat

class Penguin(BaseModel):
    """Parse & validate penguin measurements"""
    bill_length_mm: confloat(gt=32, lt=60)
    bill_depth_mm: confloat(gt=13, lt=22)

    def to_df(self):
        """Convert to pandas dataframe with 1 row."""
        return pd.DataFrame([dict(self)])<jupyter_output><empty_output><jupyter_text>Now we can validate JSON input and convert it to a pandas dataframe with one line of code.<jupyter_code>Penguin(**gary_gentoo).to_df()<jupyter_output><empty_output><jupyter_text>Let's put this all together in our FastAPI code.- Add the `Penguin` class.- Change the type annotation for the `predict_species` function argument. Instead of `dict`, the type is now `Penguin`.- When a POST request is made to the `/predict` path, then FastAPI will automatically validate and parse the request body's JSON into a `Penguin` object.- Use the penguin's `to_df` method to convert into a dataframe for our model.<jupyter_code>import random

from fastapi import FastAPI
import pandas as pd
from pydantic import BaseModel, confloat
import uvicorn

app = FastAPI(
    title='🐧 Penguin predictor API', 
    description='Deploys a logistic regression model fit on the [Palmer Penguins](https://github.com/allisonhorst/palmerpenguins) dataset.', 
    docs_url='/'
)


class Penguin(BaseModel):
    """Parse & validate penguin measurements"""
    bill_length_mm: confloat(gt=32, lt=60)
    bill_depth_mm: confloat(gt=13, lt=22)

    def to_df(self):
        """Convert to pandas dataframe with 1 row."""
        return pd.DataFrame([dict(self)])


@app.post('/predict')
def predict_species(penguin: Penguin):
    """Predict penguin species from bill length & depth
    
    Parameters
    ----------
    bill_length_mm : float, greater than 32, less than 60  
    bill_depth_mm : float, greater than 13, less than 22  

    Returns
    -------
    str "Adelie", "Chinstrap", or "Gentoo"  
    """
    species = classifier.predict(penguin.to_df())
    return species[0]


@app.get('/random')
def random_penguin():
    """Return a random penguin species"""
    species = random.choice(['Adelie', 'Chinstrap', 'Gentoo'])
    return species


enable_cloud_notebook()
uvicorn.run(app)<jupyter_output><empty_output><jupyter_text>Test the app, then stop the cell from running.  Part -1, Deploy Let's save the model so you can use it without retraining. This is sometimes called "pickling." See [scikit-learn docs on "model persistence."](https://scikit-learn.org/stable/modules/model_persistence.html)<jupyter_code>from joblib import dump
dump(classifier, 'classifier.joblib', compress=True)<jupyter_output><empty_output><jupyter_text>Now even if we delete the object from memory ...<jupyter_code>del classifier<jupyter_output><empty_output><jupyter_text>We can reload from our file ...<jupyter_code>from joblib import load
classifier = load('classifier.joblib')<jupyter_output><empty_output><jupyter_text>... and it's back, ready to use:<jupyter_code>from sklearn import set_config
set_config(display='diagram')
classifier<jupyter_output><empty_output><jupyter_text>If you're using a cloud notebook, you can get a link to download the file using code like this:<jupyter_code>from IPython.display import FileLink
FileLink('classifier.joblib')<jupyter_output><empty_output><jupyter_text>This last code cell has 3 changes from the previous iteration:- Loads the model with joblib- Adds image HTML tags in the app's description- Configures [CORS (Cross-Origin Resource Sharing)](https://fastapi.tiangolo.com/tutorial/cors/) so your API could be called by apps on different domains.<jupyter_code>import random

from fastapi import FastAPI
from fastapi.middleware.cors import CORSMiddleware
from joblib import load
import pandas as pd
from pydantic import BaseModel, confloat
import uvicorn

description = """
Deploys a logistic regression model fit on the [Palmer Penguins](https://github.com/allisonhorst/palmerpenguins) dataset.

<img src="https://raw.githubusercontent.com/allisonhorst/palmerpenguins/master/man/figures/lter_penguins.png" width="40%" /> <img src="https://raw.githubusercontent.com/allisonhorst/palmerpenguins/master/man/figures/culmen_depth.png" width="30%" />

Artwork by [@allison_horst](https://twitter.com/allison_horst)
"""

app = FastAPI(
    title='🐧 Penguin predictor API',
    description=description, 
    docs_url='/'
)

app.add_middleware(
    CORSMiddleware,
    allow_origins=['*'],
    allow_methods=['*']
)

classifier = load('classifier.joblib')


class Penguin(BaseModel):
    """Parse & validate penguin measurements"""
    bill_length_mm: confloat(gt=32, lt=60)
    bill_depth_mm: confloat(gt=13, lt=22)

    def to_df(self):
        """Convert to pandas dataframe with 1 row."""
        return pd.DataFrame([dict(self)])


@app.post('/predict')
def predict_species(penguin: Penguin):
    """Predict penguin species from bill length & depth
    
    Parameters
    ----------
    bill_length_mm : float, greater than 32, less than 60  
    bill_depth_mm : float, greater than 13, less than 22  

    Returns
    -------
    str "Adelie", "Chinstrap", or "Gentoo"  
    """
    species = classifier.predict(penguin.to_df())
    return species[0]


@app.get('/random')
def random_penguin():
    """Return a random penguin species"""
    species = random.choice(['Adelie', 'Chinstrap', 'Gentoo'])
    return species


enable_cloud_notebook()
uvicorn.run(app)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.10052910052910052, 'fraction_numerical': 0.00791309640867163, 'mean_word_length': 5.386961722488039, 'pattern_counts': {'":': 26, '<': 144, '<?xml version=': 0, '>': 145, 'https://': 19, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 4, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '35', 'dir': 'jupyter-structured-clean-dedup', 'id': '499120', 'license': 'MIT', 'n_tokens_mistral': 6829, 'n_tokens_neox': 6332, 'n_words': 2257, 'path': 'notebook/Prototype with FastAPI in Jupyter.ipynb', 'repo_name': 'DS-Buildweek-Spotify/SpotifyBD', 'size': '21369'}