<commit_before>{
 "metadata": {
  "name": "",
  "signature": "sha256:4568f95942be4f0ef8e7e7eaa7ffa3da3fe9c7b1cd33afabddf58d3520d725a6"
 },
 "nbformat": 3,
 "nbformat_minor": 0,
 "worksheets": [
  {
   "cells": [
    {
     "cell_type": "raw",
     "metadata": {},
     "source": [
      ".. _dataset_exploration:\n",
      "\n",
      ".. currentmodule:: seaborn"
     ]
    },
    {
     "cell_type": "heading",
     "level": 1,
     "metadata": {},
     "source": [
      "Visual dataset exploration"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "%matplotlib inline"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": null
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "import numpy as np\n",
      "import pandas as pd\n",
      "import seaborn as sns\n",
      "import matplotlib.pyplot as plt"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": null
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "iris = sns.load_dataset(\"iris\")\n",
      "flights = sns.load_dataset(\"flights\")\n",
      "networks = sns.load_dataset(\"brain_networks\", index_col=0, header=[0, 1, 2])"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": null
    },
    {
     "cell_type": "raw",
     "metadata": {},
     "source": [
      ".. _heatmap:\n",
      "\n",
      "Visualizing matrices with :func:`heatmap`\n",
      "-----------------------------------------\n",
      "\n",
      "Often the easiest thing to do to visualize a reasonably large table of data is to encode the value in each cell with a color and plot a heatmap. This can be accomplished with the :func:`heatmap` function. Note that unlike many other seaborn functions, :func:`heatmap` expects the input data to be a table of values with one variable in the rows and one variable in the columns. Your dataset may be in tidy format, as with the ``flights`` example."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "flights.head()"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": null
    },
    {
     "cell_type": "raw",
     "metadata": {},
     "source": [
      "Fortunately, it's easy to pivot a dataframe out into rectangular format. Note that this sorts the index by default, so if your index has an ordering that doesn't correspond to its alphabetical order, you may need to reorder things after the pivot operation. It's also possible to collapse over a third variable (using an aggregation like a mean or sum) with the ``pivot_table`` function."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "flights_rect = flights.pivot(\"month\", \"year\", \"passengers\")\n",
      "flights_rect = flights_rect.ix[flights.month.iloc[:12]]"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": null
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "flights_rect.head()"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": null
    },
    {
     "cell_type": "raw",
     "metadata": {},
     "source": [
      "Data in this format can be passed to :func:`heatmap` to produce an easily-interpretable visualization."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "sns.heatmap(flights_rect);"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": null
    },
    {
     "cell_type": "raw",
     "metadata": {},
     "source": [
      "If you do particularly care about the precise numeric values, you can annotate each cell."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "sns.heatmap(flights_rect, annot=True, fmt=\"d\");"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": null
    },
    {
     "cell_type": "raw",
     "metadata": {},
     "source": [
      "Because the color is an encoding of the data, it's very important to use an appropriate colormap. (See the :link:`palette tutorial <palette_tutorial>` for more information about different kinds of color palettes and how to choose one that is appropriate for your data). :func:`heatmap` tries to choose good defaults for your data based off some heuristics about it. For example, if your dataset spans 0, it is assumed that a diverging colormap is more appropriate."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "network_corr = networks.iloc[:, :12].corr()\n",
      "sns.heatmap(network_corr);"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": null
    },
    {
     "cell_type": "raw",
     "metadata": {},
     "source": [
      "Note that this might not always be the case! For example, you might be plotting temperature over time in several cities, for which 0 isn't really a meaningful midpoint value. A sequential colormap would be better in that case.\n",
      "\n",<commit_msg>Fix broken reference in dataset_exploration.ipynb<commit_after>{
 "metadata": {
  "name": "",
  "signature": "sha256:4568f95942be4f0ef8e7e7eaa7ffa3da3fe9c7b1cd33afabddf58d3520d725a6"
 },
 "nbformat": 3,
 "nbformat_minor": 0,
 "worksheets": [
  {
   "cells": [
    {
     "cell_type": "raw",
     "metadata": {},
     "source": [
      ".. _dataset_exploration:\n",
      "\n",
      ".. currentmodule:: seaborn"
     ]
    },
    {
     "cell_type": "heading",
     "level": 1,
     "metadata": {},
     "source": [
      "Visual dataset exploration"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "%matplotlib inline"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": null
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "import numpy as np\n",
      "import pandas as pd\n",
      "import seaborn as sns\n",
      "import matplotlib.pyplot as plt"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": null
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "iris = sns.load_dataset(\"iris\")\n",
      "flights = sns.load_dataset(\"flights\")\n",
      "networks = sns.load_dataset(\"brain_networks\", index_col=0, header=[0, 1, 2])"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": null
    },
    {
     "cell_type": "raw",
     "metadata": {},
     "source": [
      ".. _heatmap:\n",
      "\n",
      "Visualizing matrices with :func:`heatmap`\n",
      "-----------------------------------------\n",
      "\n",
      "Often the easiest thing to do to visualize a reasonably large table of data is to encode the value in each cell with a color and plot a heatmap. This can be accomplished with the :func:`heatmap` function. Note that unlike many other seaborn functions, :func:`heatmap` expects the input data to be a table of values with one variable in the rows and one variable in the columns. Your dataset may be in tidy format, as with the ``flights`` example."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "flights.head()"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": null
    },
    {
     "cell_type": "raw",
     "metadata": {},
     "source": [
      "Fortunately, it's easy to pivot a dataframe out into rectangular format. Note that this sorts the index by default, so if your index has an ordering that doesn't correspond to its alphabetical order, you may need to reorder things after the pivot operation. It's also possible to collapse over a third variable (using an aggregation like a mean or sum) with the ``pivot_table`` function."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "flights_rect = flights.pivot(\"month\", \"year\", \"passengers\")\n",
      "flights_rect = flights_rect.ix[flights.month.iloc[:12]]"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": null
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "flights_rect.head()"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": null
    },
    {
     "cell_type": "raw",
     "metadata": {},
     "source": [
      "Data in this format can be passed to :func:`heatmap` to produce an easily-interpretable visualization."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "sns.heatmap(flights_rect);"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": null
    },
    {
     "cell_type": "raw",
     "metadata": {},
     "source": [
      "If you do particularly care about the precise numeric values, you can annotate each cell."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "sns.heatmap(flights_rect, annot=True, fmt=\"d\");"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": null
    },
    {
     "cell_type": "raw",
     "metadata": {},
     "source": [
      "Because the color is an encoding of the data, it's very important to use an appropriate colormap. (See the :ref:`palette tutorial <palette_tutorial>` for more information about different kinds of color palettes and how to choose one that is appropriate for your data). :func:`heatmap` tries to choose good defaults for your data based off some heuristics about it. For example, if your dataset spans 0, it is assumed that a diverging colormap is more appropriate."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "network_corr = networks.iloc[:, :12].corr()\n",
      "sns.heatmap(network_corr);"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [],
     "prompt_number": null
    },
    {
     "cell_type": "raw",
     "metadata": {},
     "source": [
      "Note that this might not always be the case! For example, you might be plotting temperature over time in several cities, for which 0 isn't really a meaningful midpoint value. A sequential colormap would be better in that case.\n",
      "\n",
{'fraction_non_alphanumeric': 0.16418190342240976, 'fraction_numerical': 0.0090014064697609, 'mean_word_length': 2.409846547314578, 'pattern_counts': {'":': 190, '<': 5, '<?xml version=': 0, '>': 5, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '5309861', 'n_tokens_mistral': 3474, 'n_tokens_neox': 3327, 'n_words': 981}