<commit_before>{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Machine Learning Applications for Astronomy\n",
    "========\n",
    "\n",
    "(or, but seriously - how does a machine learn?)\n",
    "\n",
    "#### Version 0.1\n",
    "\n",
    "***\n",
    "By <NAME> 2018 Nov 05"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## What is Machine Learning?\n",
    "\n",
    "Artificial Intelligence, clustering, data mining, pattern recognition, ...\n",
    "\n",
    "Lots of hype, lots of buzz, and lots and lots of \\$\\$\\$\\$."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "Summarizing the excitement for machine learning with one figure: \n",
    "\n",
    "<img style=\"display: block; margin-left: auto; margin-right: auto\" src=\"images/ImageNet.jpg\" align=\"middle\">\n",
    "\n",
    "<div align=\"right\"> <font size=\"-3\">(credit: https://www.researchgate.net/Winner-results-of-the-ImageNet-large-scale-visual-recognition-challenge-LSVRC-of-the_fig7_324476862) </font></div>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "So... what is it?\n",
    "\n",
    "Bishop's text book provides a useful 4 word summary: \n",
    "\n",
    "<img style=\"display: block; margin-left: auto; margin-right: auto\" src=\"images/Pattern-Recognition-and-Machine-Learning-Information-Science-and-Statistics.jpg\" align=\"middle\">\n",
    "\n",
    "<div align=\"right\"> <font size=\"-3\">(credit: Springer-Verlag New York) </font></div>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "Briefly, machine learning algorithms build highly complex, non-linear models (i.e., these things are black boxes) to map outcomes from input information.\n",
    "\n",
    "Ultimately, machine learning is fundamentally concerned with classification (i.e. predicting outcomes). "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "As astronomers and astrophysicists –– this is great news!\n",
    "\n",
    "The history of astronomy is a long story of classification. Essentially, point a telescope at some new location in the sky, and there is a decent chance you might find something that has never been seen before. Now, figure out how this new thing relates to all the things that you already know (classification).  "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "Astronomy is (and has been) an observational/experimental led field. The typical pattern is obsservers find some weird thing, and then the theorists try to explain what is going on (there are obviously exceptions, predictions for kilonovae prior to the LIGO detection of a neutron star-neutron star merger are a recent example).\n",
    "\n",
    "This makes us very different from physics, where theory generates predictions that then lead the observations (e.g., Higgs boson, general relativity, etc.)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "Thus, if machine learning fundamentally is about classification, and astronomers spend all their time classifying objects, that must mean that machine learning and astronomy are a match made in heaven.\n",
    "\n",
    "\n",
    "\n",
    "Right?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "This is where I say – not so fast.$^\\dagger$\n",
    "\n",
    "Even though astronomy is an observationally-led, classification-concerned field, ultimately, like physicists, we care about the development of a physical understanding of how the Universe works. And this is not what machine learning is fundamentally built to do.\n",
    "\n",
    "$^\\dagger$ A long list of people would dispute this assertion."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "In other words, \n",
    "\n",
    "machine learning $\\longleftrightarrow$ prediction\n",
    "\n",
    "astronomy $\\longleftrightarrow$ inference\n",
    "\n",
    "And thus, astronomy and machine learning may not be a match made in heaven. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Basic Concepts of Machine Learning\n",
    "\n",
    "Broadly, there are 2 types of machine learning: \n",
    "\n",
    "  *  There are no known outcomes for the data $\\longleftrightarrow$ unsupervised machine learning\n",
    "  *  Outcomes are known for a subset of the data $\\longleftrightarrow$ supervised machine learning\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Unsupervised Machine Learning\n",
    "\n",
    "(aka clustering, data mining)\n",
    "\n",
    "1. Algorithm is designed to cluster data into groups\n",
    "2. No natural metric for measuring the quality of clustering (3, 4, or 17 clusters)\n",
    "3. Can be very useful for data exploration"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Supervised Machine Learning\n",
    "\n",
    "1. Algorithm builds a map between data and (known) outcomes for a subset of the data\n",
    "2. Algorithm is optimized to maximize accuracy\n",
    "3. Useful for both classification and regression"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Terminology\n",
    "\n",
    "Features – measured properties (numerical or categorical)\n",
    "\n",
    "Labels – outcomes, i.e., thing the algorithm predicts\n",
    "\n",
    "Training set – subset of the full data set where features and labels are known (used to train the model)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "Adam's 1 slide summary of supervised machine learning: \n",
    "\n",
    "<img style=\"display: block; margin-left: auto; margin-right: auto\" src=\"images/ML_diagram.png\" align=\"middle\">"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "True positive (TP) = + classified as +\n",
    "\n",
    "False positive (FP) = + classified as - (type I error)\n",
    "\n",
    "True negative (TN) = - classified as -\n",
    "\n",
    "False negative (FN) = - classified as + (type II error)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Problem 1\n",
    "\n",
    "What is more detrimental when building a model: false positives or false negatives?"
   ]<commit_msg>MAINT: fix FP and FN typo<commit_after>{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Machine Learning Applications for Astronomy\n",
    "========\n",
    "\n",
    "(or, but seriously - how does a machine learn?)\n",
    "\n",
    "#### Version 0.1\n",
    "\n",
    "***\n",
    "By <NAME> 2018 Nov 05"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## What is Machine Learning?\n",
    "\n",
    "Artificial Intelligence, clustering, data mining, pattern recognition, ...\n",
    "\n",
    "Lots of hype, lots of buzz, and lots and lots of \\$\\$\\$\\$."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "Summarizing the excitement for machine learning with one figure: \n",
    "\n",
    "<img style=\"display: block; margin-left: auto; margin-right: auto\" src=\"images/ImageNet.jpg\" align=\"middle\">\n",
    "\n",
    "<div align=\"right\"> <font size=\"-3\">(credit: https://www.researchgate.net/Winner-results-of-the-ImageNet-large-scale-visual-recognition-challenge-LSVRC-of-the_fig7_324476862) </font></div>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "So... what is it?\n",
    "\n",
    "Bishop's text book provides a useful 4 word summary: \n",
    "\n",
    "<img style=\"display: block; margin-left: auto; margin-right: auto\" src=\"images/Pattern-Recognition-and-Machine-Learning-Information-Science-and-Statistics.jpg\" align=\"middle\">\n",
    "\n",
    "<div align=\"right\"> <font size=\"-3\">(credit: Springer-Verlag New York) </font></div>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "Briefly, machine learning algorithms build highly complex, non-linear models (i.e., these things are black boxes) to map outcomes from input information.\n",
    "\n",
    "Ultimately, machine learning is fundamentally concerned with classification (i.e. predicting outcomes). "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "As astronomers and astrophysicists –– this is great news!\n",
    "\n",
    "The history of astronomy is a long story of classification. Essentially, point a telescope at some new location in the sky, and there is a decent chance you might find something that has never been seen before. Now, figure out how this new thing relates to all the things that you already know (classification).  "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "Astronomy is (and has been) an observational/experimental led field. The typical pattern is obsservers find some weird thing, and then the theorists try to explain what is going on (there are obviously exceptions, predictions for kilonovae prior to the LIGO detection of a neutron star-neutron star merger are a recent example).\n",
    "\n",
    "This makes us very different from physics, where theory generates predictions that then lead the observations (e.g., Higgs boson, general relativity, etc.)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "Thus, if machine learning fundamentally is about classification, and astronomers spend all their time classifying objects, that must mean that machine learning and astronomy are a match made in heaven.\n",
    "\n",
    "\n",
    "\n",
    "Right?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "This is where I say – not so fast.$^\\dagger$\n",
    "\n",
    "Even though astronomy is an observationally-led, classification-concerned field, ultimately, like physicists, we care about the development of a physical understanding of how the Universe works. And this is not what machine learning is fundamentally built to do.\n",
    "\n",
    "$^\\dagger$ A long list of people would dispute this assertion."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "In other words, \n",
    "\n",
    "machine learning $\\longleftrightarrow$ prediction\n",
    "\n",
    "astronomy $\\longleftrightarrow$ inference\n",
    "\n",
    "And thus, astronomy and machine learning may not be a match made in heaven. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Basic Concepts of Machine Learning\n",
    "\n",
    "Broadly, there are 2 types of machine learning: \n",
    "\n",
    "  *  There are no known outcomes for the data $\\longleftrightarrow$ unsupervised machine learning\n",
    "  *  Outcomes are known for a subset of the data $\\longleftrightarrow$ supervised machine learning\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Unsupervised Machine Learning\n",
    "\n",
    "(aka clustering, data mining)\n",
    "\n",
    "1. Algorithm is designed to cluster data into groups\n",
    "2. No natural metric for measuring the quality of clustering (3, 4, or 17 clusters)\n",
    "3. Can be very useful for data exploration"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Supervised Machine Learning\n",
    "\n",
    "1. Algorithm builds a map between data and (known) outcomes for a subset of the data\n",
    "2. Algorithm is optimized to maximize accuracy\n",
    "3. Useful for both classification and regression"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Terminology\n",
    "\n",
    "Features – measured properties (numerical or categorical)\n",
    "\n",
    "Labels – outcomes, i.e., thing the algorithm predicts\n",
    "\n",
    "Training set – subset of the full data set where features and labels are known (used to train the model)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "Adam's 1 slide summary of supervised machine learning: \n",
    "\n",
    "<img style=\"display: block; margin-left: auto; margin-right: auto\" src=\"images/ML_diagram.png\" align=\"middle\">"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "True positive (TP) = + classified as +\n",
    "\n",
    "False positive (FP) = - classified as + (type I error)\n",
    "\n",
    "True negative (TN) = - classified as -\n",
    "\n",
    "False negative (FN) = + classified as - (type II error)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Problem 1\n",
    "\n",
    "What is more detrimental when building a model: false positives or false negatives?"
   ]
{'fraction_non_alphanumeric': 0.16290777285541683, 'fraction_numerical': 0.004564371056517654, 'mean_word_length': 2.6733234714003946, 'pattern_counts': {'":': 172, '<': 27, '<?xml version=': 0, '>': 27, 'https://': 2, 'lorem ipsum': 0, 'www.': 2, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '970283', 'n_tokens_mistral': 5011, 'n_tokens_neox': 4785, 'n_words': 1493}