<jupyter_start><jupyter_text>Basic Mechanics-------------------------Let's get an overivew of how authentication/etc work with Planet's library and set the groundwork for how we're going to interact with the Planet Data API.Note that we're trying to stay relatively language agnostic, so we're still going to build things up mostly from scratch.  We'll make use of Python's standard library functionality and we'll use the `requests` library to simplify communicating with the api a bit.  We're not going to focus heavily on Planet's Python client library or other high-level ways of accessing the data.  Our goal here is to explore and understand the underlying REST api.To start with, let's make a request against the top-level endpoint for the Planet data api. It returns some information about the api and doesn't require authentication (we'll cover that soon).<jupyter_code># Just for nicer display of response dicts -- This is a standard library module
from pprint import pprint  

# Not part of the standard library, but makes life much easier when working with http APIs
import requests

# Let's do this properly and raise an exception if we don't get a 200 (ok) response code
response = requests.get('https://api.planet.com/data/v1')
response.raise_for_status()
body = response.json()

# Nicer display than a bare print
pprint(body)<jupyter_output><empty_output><jupyter_text>Authentication----------------------Planet's APIs handle authentication primarily through API keys. It's a bit more secure than using your username and password, as the key only has access to our api (and not, say, deleting your account).  You can find your api key by logging in to https://www.planet.com/account  You should already have had an account created for you as part of the workshop signup.  Your account will have access to a variety of data in California for the next month.The python client reads from the `PL_API_KEY` environment variable if it exists. We'll read the api key from that env variable rather than hard-coding it in these exercises.  Let's go ahead and set that now:<jupyter_code>import os

# If you're following along with this notebook, you can enter your API Key on the following line, and uncomment it:
#os.environ['PL_API_KEY'] = "YOUR API KEY HERE"

# Setup the API Key from the `PL_API_KEY` environment variable
PLANET_API_KEY = os.getenv('PL_API_KEY')
assert PLANET_API_KEY is not None, 'PL_API_KEY not found'<jupyter_output><empty_output><jupyter_text>To start with, let's look at what happens when we don't authenticate:<jupyter_code># We'll explain this url later -- This is just a demo of something you can't access without auth
scene_url = 'https://api.planet.com/data/v1/item-types/PSScene4Band/items/20191010_183406_0f28'

response = requests.get(scene_url)
response.raise_for_status()<jupyter_output><empty_output><jupyter_text>You can use your api key to authenticate through basic http authentication:<jupyter_code># Basic auth for our API expects api key as the username and no password
response = requests.get(scene_url, auth=(PLANET_API_KEY, ''))
response.raise_for_status()<jupyter_output><empty_output><jupyter_text>We can also use a session in the `requests` library that will store our auth and use it for all requests:<jupyter_code>session = requests.Session()
session.auth = (PLANET_API_KEY, '')

response = session.get(scene_url)
response.raise_for_status()<jupyter_output><empty_output><jupyter_text>Rate Limiting and Retries--------------------------------------Planet's services limit the number of requests you can make in a short amount of time to avoid unintentional and intentional denial of service attacks.  A 429 response code indicates that the service won't deliver results until you slow down.As an example, let's do something that will trigger a "slow down" response (429):<jupyter_code># We'll use multiple threads to really hammer things... Otherwise it's unlikely to trigger.
from concurrent.futures import ThreadPoolExecutor

def make_request(ignored):
    requests.get(scene_url, auth=(PLANET_API_KEY, '')).raise_for_status()
    
nthreads = 8
with ThreadPoolExecutor(nthreads) as executor:
    for _ in executor.map(make_request, range(100)):
        pass<jupyter_output><empty_output><jupyter_text>We ask that you follow an exponential backoff pattern when this occurs -- E.g. on the first occurence, wait `n` seconds, on the second `n**2` seconds, on the third `n**3`, etc.I'm sure a lot of you have probably implemented similar retry functionality or have a library you often use to do so.  There are also methods for this in our python client.However, to keep with the spirit of staying relatively low-level and using widely available tools, we'll set up the `requests` library to do this for us.<jupyter_code>import requests
from urllib3.util.retry import Retry
from requests.adapters import HTTPAdapter

session = requests.Session()
session.auth = (PLANET_API_KEY, '')

retries = Retry(total=5,
                backoff_factor=0.2, # Exponential backoff scaled by 0.2
                status_forcelist=[429]) # In practice, you may want other codes there too, e.g. 500s...

session.mount('https://', HTTPAdapter(max_retries=retries))<jupyter_output><empty_output><jupyter_text>And just to demonstrate that we can now hammer the service in parallel and properly back off and retry when it asks us to:<jupyter_code>def make_request(ignored):
    # Note that we're using the session we set up above
    session.get(scene_url, auth=(PLANET_API_KEY, '')).raise_for_status()
    
nthreads = 8
with ThreadPoolExecutor(nthreads) as executor:
    for _ in executor.map(make_request, range(100)):
        pass<jupyter_output><empty_output><jupyter_text>Setup for Future Exercises----------------------------------------Taking what we've just walked through, we'll use the following as a bit of boilerplate to set up our later exercises.  For actual use cases, you can see how this could form the start of an `APIClient` class. We're going to keep things as mininimal as possible in this tutorial, though.<jupyter_code>import os
from pprint import pprint
from urllib3.util.retry import Retry

import requests
from requests.adapters import HTTPAdapter

PLANET_API_URL = 'https://api.planet.com/data/v1'

def setup_session(api_key=None):
    """
    Initialize a requests.Session that handles Planet api key auth and retries.
    
    :param str api_key:
        A Planet api key. Will be read from the PL_API_KEY env var if not specified.
    
    :returns requests.Session session:
        A Session instance optimized for use with Planet's api.
    """
    if api_key is None:
        api_key = os.getenv('PL_API_KEY')

    session = requests.Session()
    session.auth = (api_key, '')

    retries = Retry(total=5,
                    backoff_factor=0.2,  
                    status_forcelist=[429])

    session.mount('https://', HTTPAdapter(max_retries=retries))
    return session

session = setup_session() # Or pass in an api key if the environment variable isn't set<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.08825612693016008, 'fraction_numerical': 0.008358124380223828, 'mean_word_length': 4.233506300963676, 'pattern_counts': {'":': 0, '<': 37, '<?xml version=': 0, '>': 37, 'https://': 6, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '9', 'dir': 'jupyter-structured-clean-dedup', 'id': '270637', 'license': 'Apache-2.0', 'n_tokens_mistral': 2051, 'n_tokens_neox': 1944, 'n_words': 837, 'path': 'notebooks/1_BasicMechanics.ipynb', 'repo_name': 'joferkington/planet_api_workshop', 'size': '7059'}