<commit_before>{
 "cells": [
  {
   "cell_type": "raw",
   "metadata": {},
   "source": [
    "<script>\n",
    "code_show=true; \n",
    "function code_toggle() {\n",
    " if (code_show){\n",
    " $('div.input').hide();\n",
    " } else {\n",
    " $('div.input').show();\n",
    " }\n",
    " code_show = !code_show\n",
    "} \n",
    "$( document ).ready(code_toggle);\n",
    "</script>\n",
    "<form action=\"javascript:code_toggle()\"><input type=\"submit\" value=\"Click here to toggle on/off the raw code.\"></form>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src=\"https://raw.githubusercontent.com/jbwhit/svds-style/master/figures/logo-medium.png\" alt=\"SVDS\" width=\"590\" align=\"left\">"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Presentation Template\n",
    "<div class=\"alert alert-info\">*This is a template for SVDS presentations. Blue boxes in the template convey information about using the template. Remove these before sharing with clients*</div>\n",
    "<div class=\"alert alert-warning\">Yellow boxes hold important messages to keep in mind. Remove these before sharing with clients as well.</div>\n",
    "\n",
    "Use markdown cells to begin each section with a heading, and subsections with subheadings, using # and ## and ### in descending hierarchy.
     \n",
    "\n",
    " \n",
    "*The template is not intended to be an overly rigid document. Violations of the template are acceptable as long as they substantively improve the presentation and have a valid purpose. However, if the template is regularly being modified in a specific way then the template itself should be forked or modified.*\n",
    "\n",
    "\n",
    "<div class=\"alert alert-warning\">Recognize that violations of the template come at a cost to the reader who has to adjust to the changes and come at a cost, so **modify with care**</div>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Introduction"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Purpose"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div class=\"alert alert-info\">This section describes what this notebook is for, and why the reader should care about what they are reading</div>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Guiding Questions:\n",
    "1. **Question 1**\n",
    "2. **Question 2**\n",
    "3. **Question 3**\n",
    "4. ...."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div class=\"alert alert-info\">*This part of the document is an explanation of the key hypothesis and questions of interest. The notebook should explore either a single Agile story broken into subsections. If multiple stories need to be discussed it would generally be better to use two notebooks so the thematic flow is retained.*</div>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Key Findings\n",
    "\n",
    "1. **Finding 1**\n",
    "2. **Finding 2**\n",
    "3. **Finding 3**\n",
    "4. ..."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div class=\"alert alert-info\">*This part of the document is an explanation of the key results that will be shared below. The idea is to inform the reader what they will be seeing in the document below. This establishes the messages of the analysis below and helps the users understand what they will be seeing.*</div>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Notes\n",
    "<div class=\"alert alert-warning\">Optional, Remove if Unused</div>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div class=\"alert alert-info\">*This section should include any notes to reader, such as major issues with data quality or other points of concern they should be aware of. This section is optional and depends on the needs of the document*</div>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "######\n",
    "# \n",
    "# Library/Package Loads\n",
    "#\n",
    "# This section should contain all library/package loads required for the code.\n",
    "#\n",
    "######\n",
    "\n",
    "%matplotlib inline\n",
    "%config InlineBackend.figure_format='retina'\n",
    "# %load_ext autoreload\n",
    "# %autoreload 1\n",
    "\n",
    "from __future__ import absolute_import, division, print_function\n",
    "import matplotlib as mpl\n",
    "from matplotlib import pyplot as plt\n",
    "from matplotlib.pyplot import GridSpec\n",
    "import seaborn as sns\n",
    "import mplsvds\n",
    "import mpld3\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "import os, sys\n",
    "from tqdm import tqdm\n",
    "import warnings\n",
    "\n",
    "######\n",
    "#\n",
    "# Configurations\n",
    "#\n",
    "# This section contains specific configurations required by the code \n",
    "# going forward, including formatting options.\n",
    "#\n",
    "######\n",
    "\n",
    "def define_paths():\n",
    "    paths = {}\n",
    "    paths['working'] = os.path.abspath(os.path.curdir)\n",
    "    main = os.path.split(os.path.split(paths['working'])[0])[0]\n",
    "    strike = os.path.split(os.path.split(main)[0])[0]\n",
    "    for directory in ['source','data','develop','report']:\n",
    "        paths[directory] = os.path.join(main, directory)\n",
    "    paths['sql'] = os.path.join(paths['source'], directory)\n",
    "    paths['src'] = os.path.join(strike, 'src')\n",
    "    return paths\n",
    "\n",
    "paths = define_paths()\n",
    "\n",
    "# Notebook variables and settings\n",
    "user = os.path.expanduser('~').split('/')[-1]\n",
    "os.system(\"kinit -k -t /home/{u}/.keytabs/{u}.keytab -p {u}\".format(u=user))\n",
    "sys.path.append(paths['src'])\n",
    "\n",
    "######\n",
    "#\n",
    "# Source Files\n",
    "#\n",
    "# This section should contain other source files loaded before the code runs.\n",
    "# Preferably some note of the features of the source files would be appropriate.\n",
    "#\n",
    "######"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Analysis Sections"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "<div class=\"alert alert-info\">*These sections should contain the core results of the analysis.*</div>\n",
    "\n",
    "*Some general rules:*\n",
    "\n",
    "\n",
    "<div class=\"alert alert-warning\">***Not everything you have done in your work should be shown.***\n",
    "\n",
    "*Focus on the main points. Have the work available if needed but someone looking at your work has not completed \n",
    "every step, nor are they likely to have time to do so.*</div>\n",
    "\n",
    "<div class=\"alert alert-warning\">***Develop a thematic message***\n",
    "\n",
    "*Make it easy for the person looking at your presentation to understand the major points. Be prepared to discuss minor points, or address specific questions, but prepare the topic so that the key takeaways and important efforts are clear.*</div>\n",
    "\n",
    "<div class=\"alert alert-warning\">***Emphasize the key takeaways, strive for clarity***\n",
    "\n",
    "*This is a presentation, not a raw document. The person seeing your work has not gone through the rigorous evaluation you have. They don't know every nook and cranny. They need you to point out things that became obvious to you as part of your analysis. Provide strong takeaways.*</div>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Conclusions"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "<div class=\"alert alert-info\">This section should wrap up the main ideas expressed above, and should contain a basic repeat of the findings addressed above along with suggested action items. The findings in this section should be slightly more detailed, in that at this point the user should have seen the work and you can reference specifics.</div>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Key Findings\n",
    "\n",
    "1. **Finding 1**\n",
    "2. **Finding 2**\n",
    "3. **Finding 3**\n",
    "4. ..."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "### Action Items\n",
    "1. **Item 1**\n",
    "2. **Item 2**\n",
    "3. **Item 3**"
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {},
   "source": [
    "<center><h2>© <a href=\"http://www.svds.com\">2016 Silicon Valley Data Science LLC</a></h2></center>"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}
<commit_msg>Update template<commit_after>{
 "cells": [
  {
   "cell_type": "raw",
   "metadata": {},
   "source": [
    "<script>\n",
    "code_show=true; \n",
    "function code_toggle() {\n",
    " if (code_show){\n",
    " $('div.input').hide();\n",
    " } else {\n",
    " $('div.input').show();\n",
    " }\n",
    " code_show = !code_show\n",
    "} \n",
    "$( document ).ready(code_toggle);\n",
    "</script>\n",
    "<form action=\"javascript:code_toggle()\"><input type=\"submit\" value=\"Click here to toggle on/off the raw code.\"></form>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src=\"https://raw.githubusercontent.com/jbwhit/svds-style/master/figures/logo-medium.png\" alt=\"SVDS\" width=\"590\" align=\"left\">"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Presentation Template\n",
    "<div class=\"alert alert-info\">*This is a template for SVDS presentations. Blue boxes in the template convey information about using the template. Remove these before sharing with clients*</div>\n",
    "<div class=\"alert alert-warning\">Yellow boxes hold important messages to keep in mind. Remove these before sharing with clients as well.</div>\n",
    "\n",
    "Use markdown cells to begin each section with a heading, and subsections with subheadings, using # and ## and ### in descending hierarchy.\n",
    "\n",
    "*The template is not intended to be an overly rigid document. Violations of the template are acceptable as long as they substantively improve the presentation and have a valid purpose. However, if the template is regularly being modified in a specific way then the template itself should be forked or modified.*\n",
    "\n",
    "\n",
    "<div class=\"alert alert-warning\">Recognize that violations of the template come at a cost to the reader who has to adjust to the changes and come at a cost, so **modify with care**</div>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Introduction"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Purpose"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div class=\"alert alert-info\">This section describes what this notebook is for, and why the reader should care about what they are reading</div>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Guiding Questions:\n",
    "1. **Question 1**\n",
    "2. **Question 2**\n",
    "3. **Question 3**\n",
    "4. ...."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div class=\"alert alert-info\">*This part of the document is an explanation of the key hypothesis and questions of interest. The notebook should explore either a single Agile story broken into subsections. If multiple stories need to be discussed it would generally be better to use two notebooks so the thematic flow is retained.*</div>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Key Findings\n",
    "\n",
    "1. **Finding 1**\n",
    "2. **Finding 2**\n",
    "3. **Finding 3**\n",
    "4. ..."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div class=\"alert alert-info\">*This part of the document is an explanation of the key results that will be shared below. The idea is to inform the reader what they will be seeing in the document below. This establishes the messages of the analysis below and helps the users understand what they will be seeing.*</div>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Notes\n",
    "<div class=\"alert alert-warning\">Optional, Remove if Unused</div>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div class=\"alert alert-info\">*This section should include any notes to reader, such as major issues with data quality or other points of concern they should be aware of. This section is optional and depends on the needs of the document*</div>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "######\n",
    "# \n",
    "# Library/Package Loads\n",
    "#\n",
    "# This section should contain all library/package loads required for the code.\n",
    "#\n",
    "######\n",
    "\n",
    "%matplotlib inline\n",
    "%config InlineBackend.figure_format='retina'\n",
    "# %load_ext autoreload\n",
    "# %autoreload 1\n",
    "\n",
    "from __future__ import absolute_import, division, print_function\n",
    "import matplotlib as mpl\n",
    "from matplotlib import pyplot as plt\n",
    "from matplotlib.pyplot import GridSpec\n",
    "import seaborn as sns\n",
    "import mplsvds\n",
    "import mpld3\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "import os, sys\n",
    "from tqdm import tqdm\n",
    "import warnings\n",
    "\n",
    "######\n",
    "#\n",
    "# Configurations\n",
    "#\n",
    "# This section contains specific configurations required by the code \n",
    "# going forward, including formatting options.\n",
    "#\n",
    "######\n",
    "\n",
    "def define_paths():\n",
    "    paths = {}\n",
    "    paths['working'] = os.path.abspath(os.path.curdir)\n",
    "    main = os.path.split(os.path.split(paths['working'])[0])[0]\n",
    "    strike = os.path.split(os.path.split(main)[0])[0]\n",
    "    for directory in ['source','data','develop','report']:\n",
    "        paths[directory] = os.path.join(main, directory)\n",
    "    paths['sql'] = os.path.join(paths['source'], directory)\n",
    "    paths['src'] = os.path.join(strike, 'src')\n",
    "    return paths\n",
    "\n",
    "paths = define_paths()\n",
    "\n",
    "# Notebook variables and settings\n",
    "user = os.path.expanduser('~').split('/')[-1]\n",
    "os.system(\"kinit -k -t /home/{u}/.keytabs/{u}.keytab -p {u}\".format(u=user))\n",
    "sys.path.append(paths['src'])\n",
    "\n",
    "######\n",
    "#\n",
    "# Source Files\n",
    "#\n",
    "# This section should contain other source files loaded before the code runs.\n",
    "# Preferably some note of the features of the source files would be appropriate.\n",
    "#\n",
    "######"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Analysis Sections"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "<div class=\"alert alert-info\">*These sections should contain the core results of the analysis.*</div>\n",
    "\n",
    "*Some general rules:*\n",
    "\n",
    "\n",
    "<div class=\"alert alert-warning\">***Not everything you have done in your work should be shown.***\n",
    "\n",
    "*Focus on the main points. Have the work available if needed but someone looking at your work has not completed \n",
    "every step, nor are they likely to have time to do so.*</div>\n",
    "\n",
    "<div class=\"alert alert-warning\">***Develop a thematic message***\n",
    "\n",
    "*Make it easy for the person looking at your presentation to understand the major points. Be prepared to discuss minor points, or address specific questions, but prepare the topic so that the key takeaways and important efforts are clear.*</div>\n",
    "\n",
    "<div class=\"alert alert-warning\">***Emphasize the key takeaways, strive for clarity***\n",
    "\n",
    "*This is a presentation, not a raw document. The person seeing your work has not gone through the rigorous evaluation you have. They don't know every nook and cranny. They need you to point out things that became obvious to you as part of your analysis. Provide strong takeaways.*</div>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Conclusions"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "<div class=\"alert alert-info\">This section should wrap up the main ideas expressed above, and should contain a basic repeat of the findings addressed above along with suggested action items. The findings in this section should be slightly more detailed, in that at this point the user should have seen the work and you can reference specifics.</div>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Key Findings\n",
    "\n",
    "1. **Finding 1**\n",
    "2. **Finding 2**\n",
    "3. **Finding 3**\n",
    "4. ..."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "### Action Items\n",
    "1. **Item 1**\n",
    "2. **Item 2**\n",
    "3. **Item 3**"
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {},
   "source": [
    "<center><h2>© <a href=\"http://www.svds.com\">2016 Silicon Valley Data Science LLC</a></h2></center>"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}

{'fraction_non_alphanumeric': 0.18403464684809923, 'fraction_numerical': 0.005667539966850238, 'mean_word_length': 2.8652614176482745, 'pattern_counts': {'":': 168, '<': 79, '<?xml version=': 0, '>': 79, 'https://': 2, 'lorem ipsum': 0, 'www.': 2, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '795229', 'n_tokens_mistral': 6177, 'n_tokens_neox': 6098, 'n_words': 1889}