<jupyter_start><jupyter_text>JSON and APIs JSONWhat is JSON? From JSON.org:    JSON (JavaScript Object Notation) is a lightweight data-interchange format. It is easy for humans to read and write. It is easy for machines to parse and generate. It is based on a subset of the JavaScript Programming Language, Standard ECMA-262 3rd Edition - December 1999. JSON is a text format that is completely language independent but uses conventions that are familiar to programmers of the C-family of languages, including C, C++, C, Java, JavaScript, Perl, Python, and many others. These properties make JSON an ideal data-interchange language.    But that isn't exactly helpful is it? JSON is a string format that allows you to store dictionaries, lists, strings, and numbers in a way that you can pass it from one source to another. You can take a Python dictionary and pass it to a perl program by printing it in JSON format and loading it or you can pull data from the web and create a python dictionary or list from that. Even if you don't understand now, after you use it, JSON will become more clear.You have in this folder a json file called `shapes.json`. Take a look at it and then we can talk about JSON format.  JSON FormatJSON is a subset of python. You have a top level object that is either a list or a dictionary. You can have values and keys in the top level object be any of the following: strings, floats, ints, lists, boolean, null, or dictonary. To see how to represent these refer to the documentation www.json.org  JSON in PythonTo use json in python we use the module `json`. It is part of the standard library so you don't need to install anything.    import json<jupyter_code>import json<jupyter_output><empty_output><jupyter_text>Loading dataYou can load data from json format into python from either a string using the `loads` method or a file handle using the `load` method.     my_list = json.loads('[1, 2, 3]')    with open(my_file.json) as fh:        my_dict = json.load(fh)<jupyter_code># Load from file
with open('shapes.json') as fh:
    shapes = json.load(fh)
print(shapes)
# Load from string
complex_shapes_string = '["pentagon", "spiral", "double helix"]'
complex_shapes = json.loads(complex_shapes_string)
print(complex_shapes)<jupyter_output><empty_output><jupyter_text>TRY ITCreate a string called three_d_json which cotains the string '["cube", "sphere"]' and then load that data into a python list using json.load.  Using JSON dataOnce you load data from python format, you can now use the data like you would any other python dictionary or list.<jupyter_code>for shape in shapes:
    title_shape = shape.title()
    area_formula = shapes[shape]['area']
    print("{}'s area can be calculated using {}".format(title_shape, area_formula))<jupyter_output><empty_output><jupyter_text>TRY ITfor each shape in `complex_shapes` print "`shape` is hard to find the area of".  Dumping JSON DataIf you want to store data from your python program into JSON format, it is as simple as loading it. To dump to a string use `json.dumps` and to dump to a file use `json.dump`. Make sure that you are using only valid json values in your list or dictionary.    json_string = json.dumps(my_list)    with open('json_file.json', 'w') as fh:        json.dump(my_dict, fh)<jupyter_code># Dumping to string
favorite_shapes = ['hexagon', 'heart']
fav_shapes_json = json.dumps(favorite_shapes)
print(fav_shapes_json)
# Dumping to a file
with open('fav_shapes.json', 'w') as fh:
    json.dump(favorite_shapes, fh)<jupyter_output><empty_output><jupyter_text>TRY ITcreate a list of 4 sided shapes and store in a variable called `quads`, dump `quad` to json and store the result in a variable called `quads_json`.  Web APIsWeb APIs are a way to retreive and send data to and from a url. The urls have a pattern so that you can retreive data programtically. With REST APIs specifically, you build a url putting data in the correct places to retreive the data you need. Many Web APIs (the best ones) return their data in JSON format. There are many free api's available, most require that you sign up to recieve an API key. You will need to read the API docs for any specific api to figure out how to get the data you want.Here are some fun APIs to try out:* Dropbox: https://www.dropbox.com/developers* Google Maps: https://developers.google.com/maps/web/* Twitter: https://dev.twitter.com/docs* YouTube: https://developers.google.com/youtube/v3/getting-started* Soundcloud: http://developers.soundcloud.com/docs/api/guideplaying* Stripe: https://stripe.com/docs/tutorials/checkout* Instagram: http://instagram.com/developer/* Twilio: https://www.twilio.com/docs* Yelp: http://www.yelp.com/developers/getting_started* Facebook: https://developers.facebook.com/docs/facebook-login/login-flow-for-web* Etsy: https://www.etsy.com/developers/documentationWe are going to use the steam api because certain endpoints don't require an app id (and who has time to sign up for one when there is python to learn?)The endpoint we will use is one that will get us metadata info about a specific game:    http://store.steampowered.com/api/appdetails?appids=        If the game doesn't exist it returns json that looks like this:        {"1":{"success":false}}        If the game does exist it returns json that looks like this:    "100":{      "success":true,    "data":{        "type":"game",      "name":"Counter-Strike: Condition Zero",      "steam_appid":80,      "required_age":0,      "is_free":false,      "detailed_description":"With its extensive Tour of Duty campaign, a near-limitless number of skirmish modes, updates and new content for Counter-Strike's award-winning multiplayer game play, plus over 12 bonus single player missions, Counter-Strike: Condition Zero is a tremendous offering of single and multiplayer content.",      "about_the_game":"With its extensive Tour of Duty campaign, a near-limitless number of skirmish modes, updates and new content for Counter-Strike's award-winning multiplayer game play, plus over 12 bonus single player missions, Counter-Strike: Condition Zero is a tremendous offering of single and multiplayer content.",      "supported_languages":"English, French, German, Italian, Spanish, Simplified Chinese, Traditional Chinese, Korean",      "header_image":"http:\/\/cdn.akamai.steamstatic.com\/steam\/apps\/80\/header.jpg?t=1447889920",      "website":null,      "pc_requirements":{          "minimum":"\r\n\t\t\tMinimum: 500 mhz processor, 96mb ram, 16mb video card, Windows XP, Mouse, Keyboard, Internet Connection\r\n\t\t\tRecommended: 800 mhz processor, 128mb ram, 32mb+ video card, Windows XP, Mouse, Keyboard, Internet Connection\r\n\t\t\t"      },      "mac_requirements":[        ],      "linux_requirements":[        ],      "developers":[          "Valve"      ],      "publishers":[          "Valve"      ],      "price_overview":{          "currency":"USD",        "initial":999,        "final":999,        "discount_percent":0      },      "packages":[          7      ],      "package_groups":[          {            "name":"default",          "title":"Buy Counter-Strike: Condition Zero",          "description":"",          "selection_text":"Select a purchase option",          "save_text":"",          "display_type":0,          "is_recurring_subscription":"false",          "subs":[              {                "packageid":7,              "percent_savings_text":"",              "percent_savings":0,              "option_text":"Counter-Strike: Condition Zero $9.99",              "option_description":"",              "can_get_free_license":"0",              "is_free_license":false,              "price_in_cents_with_discount":999            }          ]        }      ],      "platforms":{          "windows":true,        "mac":true,        "linux":true      },      "metacritic":{          "score":65,        "url":"http:\/\/www.metacritic.com\/game\/pc\/counter-strike-condition-zero?ftag=MCD-06-10aaa1f"      },      "categories":[          {            "id":2,          "description":"Single-player"        },        {            "id":1,          "description":"Multi-player"        },        {            "id":8,          "description":"Valve Anti-Cheat enabled"        }      ],      "genres":[          {            "id":"1",          "description":"Action"        }      ],      "screenshots":[          {            "id":0,          "path_thumbnail":"http:\/\/cdn.akamai.steamstatic.com\/steam\/apps\/80\/0000002528.600x338.jpg?t=1447889920",          "path_full":"http:\/\/cdn.akamai.steamstatic.com\/steam\/apps\/80\/0000002528.1920x1080.jpg?t=1447889920"        },        {            "id":1,          "path_thumbnail":"http:\/\/cdn.akamai.steamstatic.com\/steam\/apps\/80\/0000002529.600x338.jpg?t=1447889920",          "path_full":"http:\/\/cdn.akamai.steamstatic.com\/steam\/apps\/80\/0000002529.1920x1080.jpg?t=1447889920"        },        {            "id":2,          "path_thumbnail":"http:\/\/cdn.akamai.steamstatic.com\/steam\/apps\/80\/0000002530.600x338.jpg?t=1447889920",          "path_full":"http:\/\/cdn.akamai.steamstatic.com\/steam\/apps\/80\/0000002530.1920x1080.jpg?t=1447889920"        },        {            "id":3,          "path_thumbnail":"http:\/\/cdn.akamai.steamstatic.com\/steam\/apps\/80\/0000002531.600x338.jpg?t=1447889920",          "path_full":"http:\/\/cdn.akamai.steamstatic.com\/steam\/apps\/80\/0000002531.1920x1080.jpg?t=1447889920"        },        {            "id":4,          "path_thumbnail":"http:\/\/cdn.akamai.steamstatic.com\/steam\/apps\/80\/0000002532.600x338.jpg?t=1447889920",          "path_full":"http:\/\/cdn.akamai.steamstatic.com\/steam\/apps\/80\/0000002532.1920x1080.jpg?t=1447889920"        },        {            "id":5,          "path_thumbnail":"http:\/\/cdn.akamai.steamstatic.com\/steam\/apps\/80\/0000002533.600x338.jpg?t=1447889920",          "path_full":"http:\/\/cdn.akamai.steamstatic.com\/steam\/apps\/80\/0000002533.1920x1080.jpg?t=1447889920"        },        {            "id":6,          "path_thumbnail":"http:\/\/cdn.akamai.steamstatic.com\/steam\/apps\/80\/0000002534.600x338.jpg?t=1447889920",          "path_full":"http:\/\/cdn.akamai.steamstatic.com\/steam\/apps\/80\/0000002534.1920x1080.jpg?t=1447889920"        },        {            "id":7,          "path_thumbnail":"http:\/\/cdn.akamai.steamstatic.com\/steam\/apps\/80\/0000002535.600x338.jpg?t=1447889920",          "path_full":"http:\/\/cdn.akamai.steamstatic.com\/steam\/apps\/80\/0000002535.1920x1080.jpg?t=1447889920"        }      ],      "recommendations":{          "total":6647      },      "release_date":{          "coming_soon":false,        "date":"Mar 1, 2004"      },      "support_info":{          "url":"http:\/\/steamcommunity.com\/app\/80",        "email":""      },      "background":"http:\/\/cdn.akamai.steamstatic.com\/steam\/apps\/80\/page_bg_generated_v6b.jpg?t=1447889920"    }  }} You can actually use the url in a browser. Try that and see if you hit on any interesting games by entering an id number  Accessing API data with pythonThere are many options for getting data from a url with python: httplib, urllib, urllib2, requests. This isn't limited to JSON data from a web api, you can get the raw html from any website. We are going to use urllib2 because it is part of the standard library and it is easy to use.First, as with any library, we import it    import urllib2    Then you open a url using the method `urlopen`    connection = urllib2.urlopen('url')    Then you can read the data    data = connection.read()<jupyter_code>import urllib.request, urllib.error, urllib.parse
game_id = str(251990)
connection = urllib.request.urlopen('http://store.steampowered.com/api/appdetails?appids=' + game_id)
data = connection.read()
type(data)<jupyter_output><empty_output><jupyter_text>Now the result is a string, but it is valid json and we know how to turn a json string into a python dictionary: json.loads()<jupyter_code>game_data = json.loads(data)
print(type(game_data))<jupyter_output><empty_output><jupyter_text>Finally you can use this data just like you would any python dictionary.<jupyter_code>print(game_data[game_id]['data']['name'])
print(game_data[game_id]['data']['about_the_game'])
print(game_data[game_id]['data']['price_overview']['final'])<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.12247546435234001, 'fraction_numerical': 0.04753021331819288, 'mean_word_length': 3.2458677685950414, 'pattern_counts': {'":': 95, '<': 29, '<?xml version=': 0, '>': 29, 'https://': 8, 'lorem ipsum': 0, 'www.': 6, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 3, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '7', 'dir': 'jupyter-structured-clean-dedup', 'id': '657793', 'license': 'MIT', 'n_tokens_mistral': 4134, 'n_tokens_neox': 3653, 'n_words': 1123, 'path': 'Lesson11_JSONandAPIs/JSONandAPIs.ipynb', 'repo_name': 'WomensCodingCircle/CodingCirclePython', 'size': '12329'}