<commit_before>{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Planet API Python Client\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This tutorial is an introduction to [Planet](https://www.planet.com)'s Data API using the official [Python client](https://github.com/planetlabs/planet-client-python), the `planet` module.\n",
    "\n",
    "## Requirements\n",
    "\n",
    "This tutorial assumes familiarity with the [Python](https://python.org) programming language throughout. Python modules used in this tutorial are:\n",
    "* [IPython](https://ipython.org/) and [Jupyter](https://jupyter.org/)\n",
    "* [planet](https://github.com/planetlabs/planet-client-python)\n",
    "* [geojsonio](https://pypi.python.org/pypi/geojsonio)\n",
    "\n",
    "You should also have an account on the Planet Platform and retrieve your API key from your [account page](https://www.planet.com/account/).\n",
    "\n",
    "## Useful links \n",
    "* [Documentation](https://planetlabs.github.io/planet-client-python/index.html)\n",
    "* [Planet Data API reference](https://www.planet.com/docs/reference/data-api/)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This tutorial will cover the basic operations possible with the Python client, particularly those that interact with the Data API."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The basic workflow for interacting with the Data API is:\n",
    "1. search item types based on filters\n",
    "1. activate assets\n",
    "1. download assets"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Set up\n",
    "\n",
    "In order to interact with the Planet API using the client, we need to import the necessary packages & define helper functions."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from datetime import datetime\n",
    "from planet import Auth\n",
    "from planet import Session, DataClient, OrdersClient, data_filter\n",
    "import json\n",
    "import requests\n",
    "import time\n",
    "import os\n",
    "\n",
    "# We will also create a small helper function to print out JSON with proper indentation.\n",
    "def p(data):\n",
    "    print(json.dumps(data, indent=2))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We next need to create a `client` object registered with our API key. The API key will be automatically read from the `PL_API_KEY` environment variable if it exists. If not, you can provide it below. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# if your Planet API Key is not set as an environment variable, you can paste it below\n",
    "if os.environ.get('PL_API_KEY', ''):\n",
    "    API_KEY = os.environ.get('PL_API_KEY', '')\n",
    "else:\n",
    "    API_KEY = 'PASTE_YOUR_API_KEY_HERE'\n",
    "\n",
    "client = Auth.from_key(API_KEY)\n",
    "\n",
    "# Setup the session\n",
    "session = requests.Session()\n",
    "\n",
    "# Authenticate\n",
    "session.auth = (API_KEY, \"\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's also read in a GeoJSON geometry into a variable so we can use it during testing."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "with open(\"data/san-francisco.json\") as f:\n",
    "    geom = json.loads(f.read())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Searching\n",
    "\n",
    "We can search for items that are interesting by using the `quick_search` member function. Searches, however, always require a proper request that includes a filter that selects the specific items to return as seach results."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Filters"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The possible filters include `and_filter`, `date_filter`, `range_filter` and so on, mirroring the options supported by the Planet API.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Define the filters we'll use to find our data\n",
    "\n",
    "item_types = [\"REOrthoTile\", \"PSOrthoTile\"]\n",
    "\n",
    "geom_filter = data_filter.geometry_filter(geom)\n",
    "clear_percent_filter = data_filter.range_filter('clear_percent', 90)\n",
    "date_range_filter = data_filter.date_range_filter(\"acquired\", datetime(month=1, day=1, year=2017))\n",
    "cloud_cover_filter = data_filter.range_filter('cloud_cover', None, 0.1)\n",
    "\n",
    "combined_filter = data_filter.and_filter([geom_filter, clear_percent_filter, date_range_filter, cloud_cover_filter])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "p(combined_filter)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now let's build the request:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "async with Session() as sess:\n",
    "    cl = DataClient(sess)\n",
    "    request = await cl.create_search(name='planet_client_demo',search_filter=combined_filter, item_types=item_types)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "p(request)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Search the Data API\n",
    "async with Session() as sess:\n",
    "    cl = DataClient(sess)\n",
    "    items = await cl.run_search(search_id=request['id'])\n",
    "    item_list = [i async for i in items]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, we can iterate through our search results."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "for item in item_list:\n",
    "    print(item['id'], item['properties']['item_type'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If the number of items requested is more than 250, the client will automatically fetch more pages of results in order to get the exact number requested."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "with open('output/results.json','w') as f:\n",
    "    jsonStr = json.dumps(item_list)\n",
    "    f.write(jsonStr)\n",
    "    f.close()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This GeoJSON file can be opened and viewed in any compatible application."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "## Assets and downloads\n",
    "\n",
    "After a search returns results, the Python client can be used to check for assets and initiate downloads.\n",
    "\n",
    "The list of assets for an item that a user has access to can be retrieved with `permissions`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "print(item['id'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "item['_permissions']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Get the assets link for the item\n",
    "assets_url = item[\"_links\"][\"assets\"]\n",
    "\n",
    "# Print the assets link\n",
    "print(assets_url)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Send a GET request to the assets url for the item (Get the list of available assets for the item)\n",
    "res = session.get(assets_url)\n",
    "\n",
    "# Assign a variable to the response\n",
    "assets = res.json()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "# Assign a variable to the visual asset from the item's assets\n",
    "analytic_asset = assets[\"analytic\"]\n",
    "\n",
    "# Print the visual asset data\n",
    "p(analytic_asset)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Setup the activation url for a particular asset (in this case an analytic asset)\n",
    "activation_url = analytic_asset[\"_links\"][\"activate\"]\n",
    "\n",
    "# Send a request to the activation url to activate the item\n",
    "res = session.get(activation_url)\n",
    "\n",
    "# Print the response from the activation request\n",
    "p(res.status_code)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A response of 202 means that the request has been accepted and the activation will begin shortly. A 204 code indicates that the asset is already active and no further action is needed. A 401 code means the user does not have permissions to download this file.\n",
    "\n",
    "Below, we are polling the API until the item is done activation. This may take awhile."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "asset_activated = False\n",
    "\n",
    "while asset_activated == False:\n",
    "    # Send a request to the item's assets url\n",
    "    res = session.get(assets_url)\n",
    "\n",
    "    # Assign a variable to the item's assets url response\n",
    "    assets = res.json()\n",
    "\n",
    "    # Assign a variable to the analytic asset from the response\n",
    "    visual = assets[\"analytic\"]\n",
    "\n",
    "    asset_status = analytic_asset[\"status\"]\n",
    "    print(asset_status)\n",
    "    \n",
    "    # If asset is already active, we are done\n",
    "    if asset_status == 'active':\n",
    "        asset_activated = True\n",
    "        print(\"Asset is active and ready to download\")\n",
    "\n",
    "# Print the ps3b_analytic asset data    \n",
    "p(analytic_asset)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": false<commit_msg>documentation updates to planet client python intro notebook<commit_after>{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Planet API Python Client\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This tutorial is an introduction to [Planet](https://www.planet.com)'s Data API using the official [Python client](https://github.com/planetlabs/planet-client-python), the `planet` module.\n",
    "\n",
    "## Requirements\n",
    "\n",
    "This tutorial assumes familiarity with the [Python](https://python.org) programming language throughout. Python modules used in this tutorial are:\n",
    "* [IPython](https://ipython.org/) and [Jupyter](https://jupyter.org/)\n",
    "* [planet](https://github.com/planetlabs/planet-client-python)\n",
    "* [geojsonio](https://pypi.python.org/pypi/geojsonio)\n",
    "\n",
    "You should also have an account on the Planet Platform and retrieve your API key from your [account page](https://www.planet.com/account/).\n",
    "\n",
    "## Useful links \n",
    "* [Documentation](https://planetlabs.github.io/planet-client-python/index.html)\n",
    "* [Planet Data API reference](https://developers.planet.com/docs/apis/data/)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This tutorial will cover the basic operations possible with the Python client, particularly those that interact with the Data API."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The basic workflow for interacting with the Data API is:\n",
    "1. search item types based on filters\n",
    "1. activate assets\n",
    "1. download assets"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Set up\n",
    "\n",
    "In order to interact with the Planet API using the client, we need to import the necessary packages & define helper functions."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from datetime import datetime\n",
    "from planet import Auth\n",
    "from planet import Session, DataClient, OrdersClient, data_filter\n",
    "import json\n",
    "import requests\n",
    "import time\n",
    "import os\n",
    "\n",
    "# We will also create a small helper function to print out JSON with proper indentation.\n",
    "def p(data):\n",
    "    print(json.dumps(data, indent=2))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We next need to create a `client` object registered with our API key. The API key will be automatically read from the `PL_API_KEY` environment variable if it exists. If not, you can provide it below. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# if your Planet API Key is not set as an environment variable, you can paste it below\n",
    "if os.environ.get('PL_API_KEY', ''):\n",
    "    API_KEY = os.environ.get('PL_API_KEY', '')\n",
    "else:\n",
    "    API_KEY = 'PASTE_YOUR_API_KEY_HERE'\n",
    "\n",
    "client = Auth.from_key(API_KEY)\n",
    "\n",
    "# Setup the session\n",
    "session = requests.Session()\n",
    "\n",
    "# Authenticate\n",
    "session.auth = (API_KEY, \"\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's also read in a GeoJSON geometry into a variable so we can use it during testing."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "with open(\"data/san-francisco.json\") as f:\n",
    "    geom = json.loads(f.read())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Searching\n",
    "\n",
    "We can search for items that are interesting by using the `quick_search` member function. Searches, however, always require a proper request that includes a filter that selects the specific items to return as seach results."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Filters"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The possible filters include `and_filter`, `date_range_filter`, `range_filter` and so on, mirroring the options supported by the Planet API.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Define the filters we'll use to find our data\n",
    "\n",
    "item_types = [\"REOrthoTile\", \"PSOrthoTile\"]\n",
    "\n",
    "geom_filter = data_filter.geometry_filter(geom)\n",
    "clear_percent_filter = data_filter.range_filter('clear_percent', 90)\n",
    "date_range_filter = data_filter.date_range_filter(\"acquired\", datetime(month=1, day=1, year=2017))\n",
    "cloud_cover_filter = data_filter.range_filter('cloud_cover', None, 0.1)\n",
    "\n",
    "combined_filter = data_filter.and_filter([geom_filter, clear_percent_filter, date_range_filter, cloud_cover_filter])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "p(combined_filter)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now let's build the request:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "async with Session() as sess:\n",
    "    cl = DataClient(sess)\n",
    "    request = await cl.create_search(name='planet_client_demo',search_filter=combined_filter, item_types=item_types)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "p(request)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Search the Data API\n",
    "async with Session() as sess:\n",
    "    cl = DataClient(sess)\n",
    "    items = await cl.run_search(search_id=request['id'])\n",
    "    item_list = [i async for i in items]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, we can iterate through our search results."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "for item in item_list:\n",
    "    print(item['id'], item['properties']['item_type'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If the number of items requested is more than 250, the client will automatically fetch more pages of results in order to get the exact number requested."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "with open('output/results.json','w') as f:\n",
    "    jsonStr = json.dumps(item_list)\n",
    "    f.write(jsonStr)\n",
    "    f.close()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This GeoJSON file can be opened and viewed in any compatible application."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "## Assets and downloads\n",
    "\n",
    "After a search returns results, the Python client can be used to check for assets and initiate downloads.\n",
    "\n",
    "The list of assets for an item that a user has access to can be retrieved with `permissions`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "print(item['id'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "item['_permissions']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Get the assets link for the item\n",
    "assets_url = item[\"_links\"][\"assets\"]\n",
    "\n",
    "# Print the assets link\n",
    "print(assets_url)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Send a GET request to the assets url for the item (Get the list of available assets for the item)\n",
    "res = session.get(assets_url)\n",
    "\n",
    "# Assign a variable to the response\n",
    "assets = res.json()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "# Assign a variable to the analytic asset from the item's assets\n",
    "analytic_asset = assets[\"analytic\"]\n",
    "\n",
    "# Print the analytic asset data\n",
    "p(analytic_asset)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Setup the activation url for a particular asset (in this case an analytic asset)\n",
    "activation_url = analytic_asset[\"_links\"][\"activate\"]\n",
    "\n",
    "# Send a request to the activation url to activate the item\n",
    "res = session.get(activation_url)\n",
    "\n",
    "# Print the response from the activation request\n",
    "p(res.status_code)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A response of 202 means that the request has been accepted and the activation will begin shortly. A 204 code indicates that the asset is already active and no further action is needed. A 401 code means the user does not have permissions to download this file.\n",
    "\n",
    "Below, we are polling the API until the item is done activation. This may take awhile."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "asset_activated = False\n",
    "\n",
    "while asset_activated == False:\n",
    "    # Send a request to the item's assets url\n",
    "    res = session.get(assets_url)\n",
    "\n",
    "    # Assign a variable to the item's assets url response\n",
    "    assets = res.json()\n",
    "\n",
    "    asset_status = analytic_asset[\"status\"]\n",
    "    print(asset_status)\n",
    "    time.sleep(15) \n",
    "    \n",
    "    # If asset is already active, we are done\n",
    "    if asset_status == 'active':\n",
    "        asset_activated = True\n",
    "        print(\"Asset is active and ready to download\")\n",
    "\n",
    "# Print the analytic asset data    \n",
    "p(analytic_asset)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": false
{'fraction_non_alphanumeric': 0.17624912851498956, 'fraction_numerical': 0.0025563560306762724, 'mean_word_length': 2.79136563876652, 'pattern_counts': {'":': 290, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 20, 'lorem ipsum': 0, 'www.': 5, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 10, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '7062020', 'n_tokens_mistral': 7483, 'n_tokens_neox': 7296, 'n_words': 1923}