<commit_before>{
  "metadata": {
    "name": ""
  }, 
  "nbformat": 3, 
  "nbformat_minor": 0, 
  "worksheets": [
    {
      "cells": [
        {
          "source": [
            "# Homework 5: Networks and Congress\n", 
            "\n", 
            "*Due Friday, November 15, 11:59pm*\n", 
            "\n", 
            "<img src=\"http://img.washingtonpost.com/rf/image_1024w/2010-2019/WashingtonPost/2011/08/05/National-Politics/Images/uscap.JPG\">\n", 
            "\n", 
            "<br>"
          ], 
          "cell_type": "markdown", 
          "metadata": {}
        }, 
        {
          "cell_type": "code", 
          "language": "python", 
          "outputs": [], 
          "collapsed": false, 
          "prompt_number": 10, 
          "input": [
            "%matplotlib inline\n", 
            "\n", 
            "import json\n", 
            "\n", 
            "import numpy as np\n", 
            "import networkx as nx\n", 
            "import requests\n", 
            "from pattern import web\n", 
            "import matplotlib.pyplot as plt\n", 
            "\n", 
            "# set some nicer defaults for matplotlib\n", 
            "from matplotlib import rcParams\n", 
            "\n", 
            "#these colors come from colorbrewer2.org. Each is an RGB triplet\n", 
            "dark2_colors = [(0.10588235294117647, 0.6196078431372549, 0.4666666666666667),\n", 
            "                (0.8509803921568627, 0.37254901960784315, 0.00784313725490196),\n", 
            "                (0.4588235294117647, 0.4392156862745098, 0.7019607843137254),\n", 
            "                (0.9058823529411765, 0.1607843137254902, 0.5411764705882353),\n", 
            "                (0.4, 0.6509803921568628, 0.11764705882352941),\n", 
            "                (0.9019607843137255, 0.6705882352941176, 0.00784313725490196),\n", 
            "                (0.6509803921568628, 0.4627450980392157, 0.11372549019607843),\n", 
            "                (0.4, 0.4, 0.4)]\n", 
            "\n", 
            "rcParams['figure.figsize'] = (10, 6)\n", 
            "rcParams['figure.dpi'] = 150\n", 
            "rcParams['axes.color_cycle'] = dark2_colors\n", 
            "rcParams['lines.linewidth'] = 2\n", 
            "rcParams['axes.grid'] = False\n", 
            "rcParams['axes.facecolor'] = 'white'\n", 
            "rcParams['font.size'] = 14\n", 
            "rcParams['patch.edgecolor'] = 'none'\n", 
            "\n", 
            "def remove_border(axes=None, top=False, right=False, left=True, bottom=True):\n", 
            "    \"\"\"\n", 
            "    Minimize chartjunk by stripping out unnecessary plot borders and axis ticks\n", 
            "    \n", 
            "    The top/right/left/bottom keywords toggle whether the corresponding plot border is drawn\n", 
            "    \"\"\"\n", 
            "    ax = axes or plt.gca()\n", 
            "    ax.spines['top'].set_visible(top)\n", 
            "    ax.spines['right'].set_visible(right)\n", 
            "    ax.spines['left'].set_visible(left)\n", 
            "    ax.spines['bottom'].set_visible(bottom)\n", 
            "    \n", 
            "    #turn off all ticks\n", 
            "    ax.yaxis.set_ticks_position('none')\n", 
            "    ax.xaxis.set_ticks_position('none')\n", 
            "    \n", 
            "    #now re-enable visibles\n", 
            "    if top:\n", 
            "        ax.xaxis.tick_top()\n", 
            "    if bottom:\n", 
            "        ax.xaxis.tick_bottom()\n", 
            "    if left:\n", 
            "        ax.yaxis.tick_left()\n", 
            "    if right:\n", 
            "        ax.yaxis.tick_right()"
          ], 
          "metadata": {}
        }, 
        {
          "source": [
            "The website [govtrack.us](http://www.govetrack.us) collects data on activities in the Senate and House of Representatives. It's a great source of information for making data-driven assessments about Congress."
          ], 
          "cell_type": "markdown", 
          "metadata": {}
        }, 
        {
          "source": [
            "### Problem 1.\n", 
            "\n", 
            "The directories at http://www.govtrack.us/data/congress/113/votes/2013 contain JSON information about every vote cast for the current (113th)  Congress. Subdirectories beginning with \"S\" correspond to Senate votes, while subdirectories beginning with \"H\" correspond to House votes.\n", 
            "\n", 
            "Write two functions: one that downloads and parses a single Senate vote page given the vote number, and another that repeatedly calls this function to build a full collection of Senate votes from the 113th Congress."
          ], 
          "cell_type": "markdown", 
          "metadata": {}
        }, 
        {
          "cell_type": "code", 
          "language": "python", 
          "outputs": [], 
          "collapsed": false, 
          "prompt_number": 11, 
          "input": [
            "\"\"\"\n", 
            "Function\n", 
            "--------\n", 
            "get_senate_vote\n", 
            "\n", 
            "Scrapes a single JSON page for a particular Senate vote, given by the vote number\n", 
            "\n", 
            "Parameters\n", 
            "----------\n", 
            "vote : int\n", 
            "   The vote number to fetch\n", 
            "   \n", 
            "Returns\n", 
            "-------\n", 
            "vote : dict\n", 
            "   The JSON-decoded dictionary for that vote\n", 
            "   \n", 
            "Examples\n", 
            "--------\n", 
            ">>> get_senate_vote(12)['bill']\n", 
            "{u'congress': 113,\n", 
            " u'number': 325,\n", 
            " u'title': u'A bill to ensure the complete and timely payment of the obligations of the United States Government until May 19, 2013, and for other purposes.',\n", 
            " u'type': u'hr'}\n", 
            "\"\"\"\n", 
            "#your code here\n"
          ], <commit_msg>fixed vote number typo in get_senate_vote function spec<commit_after>{
  "metadata": {
    "name": ""
  }, 
  "nbformat": 3, 
  "nbformat_minor": 0, 
  "worksheets": [
    {
      "cells": [
        {
          "source": [
            "# Homework 5: Networks and Congress\n", 
            "\n", 
            "*Due Friday, November 15, 11:59pm*\n", 
            "\n", 
            "<img src=\"http://img.washingtonpost.com/rf/image_1024w/2010-2019/WashingtonPost/2011/08/05/National-Politics/Images/uscap.JPG\">\n", 
            "\n", 
            "<br>"
          ], 
          "cell_type": "markdown", 
          "metadata": {}
        }, 
        {
          "cell_type": "code", 
          "language": "python", 
          "outputs": [], 
          "collapsed": false, 
          "prompt_number": 10, 
          "input": [
            "%matplotlib inline\n", 
            "\n", 
            "import json\n", 
            "\n", 
            "import numpy as np\n", 
            "import networkx as nx\n", 
            "import requests\n", 
            "from pattern import web\n", 
            "import matplotlib.pyplot as plt\n", 
            "\n", 
            "# set some nicer defaults for matplotlib\n", 
            "from matplotlib import rcParams\n", 
            "\n", 
            "#these colors come from colorbrewer2.org. Each is an RGB triplet\n", 
            "dark2_colors = [(0.10588235294117647, 0.6196078431372549, 0.4666666666666667),\n", 
            "                (0.8509803921568627, 0.37254901960784315, 0.00784313725490196),\n", 
            "                (0.4588235294117647, 0.4392156862745098, 0.7019607843137254),\n", 
            "                (0.9058823529411765, 0.1607843137254902, 0.5411764705882353),\n", 
            "                (0.4, 0.6509803921568628, 0.11764705882352941),\n", 
            "                (0.9019607843137255, 0.6705882352941176, 0.00784313725490196),\n", 
            "                (0.6509803921568628, 0.4627450980392157, 0.11372549019607843),\n", 
            "                (0.4, 0.4, 0.4)]\n", 
            "\n", 
            "rcParams['figure.figsize'] = (10, 6)\n", 
            "rcParams['figure.dpi'] = 150\n", 
            "rcParams['axes.color_cycle'] = dark2_colors\n", 
            "rcParams['lines.linewidth'] = 2\n", 
            "rcParams['axes.grid'] = False\n", 
            "rcParams['axes.facecolor'] = 'white'\n", 
            "rcParams['font.size'] = 14\n", 
            "rcParams['patch.edgecolor'] = 'none'\n", 
            "\n", 
            "def remove_border(axes=None, top=False, right=False, left=True, bottom=True):\n", 
            "    \"\"\"\n", 
            "    Minimize chartjunk by stripping out unnecessary plot borders and axis ticks\n", 
            "    \n", 
            "    The top/right/left/bottom keywords toggle whether the corresponding plot border is drawn\n", 
            "    \"\"\"\n", 
            "    ax = axes or plt.gca()\n", 
            "    ax.spines['top'].set_visible(top)\n", 
            "    ax.spines['right'].set_visible(right)\n", 
            "    ax.spines['left'].set_visible(left)\n", 
            "    ax.spines['bottom'].set_visible(bottom)\n", 
            "    \n", 
            "    #turn off all ticks\n", 
            "    ax.yaxis.set_ticks_position('none')\n", 
            "    ax.xaxis.set_ticks_position('none')\n", 
            "    \n", 
            "    #now re-enable visibles\n", 
            "    if top:\n", 
            "        ax.xaxis.tick_top()\n", 
            "    if bottom:\n", 
            "        ax.xaxis.tick_bottom()\n", 
            "    if left:\n", 
            "        ax.yaxis.tick_left()\n", 
            "    if right:\n", 
            "        ax.yaxis.tick_right()"
          ], 
          "metadata": {}
        }, 
        {
          "source": [
            "The website [govtrack.us](http://www.govetrack.us) collects data on activities in the Senate and House of Representatives. It's a great source of information for making data-driven assessments about Congress."
          ], 
          "cell_type": "markdown", 
          "metadata": {}
        }, 
        {
          "source": [
            "### Problem 1.\n", 
            "\n", 
            "The directories at http://www.govtrack.us/data/congress/113/votes/2013 contain JSON information about every vote cast for the current (113th)  Congress. Subdirectories beginning with \"S\" correspond to Senate votes, while subdirectories beginning with \"H\" correspond to House votes.\n", 
            "\n", 
            "Write two functions: one that downloads and parses a single Senate vote page given the vote number, and another that repeatedly calls this function to build a full collection of Senate votes from the 113th Congress."
          ], 
          "cell_type": "markdown", 
          "metadata": {}
        }, 
        {
          "cell_type": "code", 
          "language": "python", 
          "outputs": [], 
          "collapsed": false, 
          "prompt_number": 11, 
          "input": [
            "\"\"\"\n", 
            "Function\n", 
            "--------\n", 
            "get_senate_vote\n", 
            "\n", 
            "Scrapes a single JSON page for a particular Senate vote, given by the vote number\n", 
            "\n", 
            "Parameters\n", 
            "----------\n", 
            "vote : int\n", 
            "   The vote number to fetch\n", 
            "   \n", 
            "Returns\n", 
            "-------\n", 
            "vote : dict\n", 
            "   The JSON-decoded dictionary for that vote\n", 
            "   \n", 
            "Examples\n", 
            "--------\n", 
            ">>> get_senate_vote(11)['bill']\n", 
            "{u'congress': 113,\n", 
            " u'number': 325,\n", 
            " u'title': u'A bill to ensure the complete and timely payment of the obligations of the United States Government until May 19, 2013, and for other purposes.',\n", 
            " u'type': u'hr'}\n", 
            "\"\"\"\n", 
            "#your code here\n"
          ], 
{'fraction_non_alphanumeric': 0.15245307869608166, 'fraction_numerical': 0.07029963780046099, 'mean_word_length': 1.3756355103637075, 'pattern_counts': {'":': 56, '<': 7, '<?xml version=': 0, '>': 13, 'https://': 0, 'lorem ipsum': 0, 'www.': 4, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 2, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '7598354', 'n_tokens_mistral': 4393, 'n_tokens_neox': 3558, 'n_words': 787}