<commit_before>{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Intro. to Snorkel: Extracting Spouse Relations from the News"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Part V: Evaluating the Model on the Test Set"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this final part of the tutorial, we will reload the model developed in Part IV and test it on the test `CandidateSet`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "%load_ext autoreload\n",
    "%autoreload 2\n",
    "%matplotlib inline\n",
    "\n",
    "import numpy as np\n",
    "from snorkel import SnorkelSession\n",
    "session = SnorkelSession()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We repeat our definition of the `Spouse` `Candidate` subclass, and load the test set:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "from snorkel.models import candidate_subclass\n",
    "\n",
    "Spouse          = candidate_subclass('Spouse', ['person1', 'person2'])\n",
    "test_candidates = session.query(Spouse).filter(Spouse.split == 2).all()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Automatically Creating Features\n",
    "We also generate features for the test set; note that we specify `create_new_keyset=False` to use the features from the training set.  For further details see _Intro Tutorial 4_."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },<commit_msg>Dumb error in tutorial 5<commit_after>{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Intro. to Snorkel: Extracting Spouse Relations from the News"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Part V: Evaluating the Model on the Test Set"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this final part of the tutorial, we will reload the model developed in Part IV and test it on the test `CandidateSet`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "%load_ext autoreload\n",
    "%autoreload 2\n",
    "%matplotlib inline\n",
    "import os\n",
    "\n",
    "# TO USE A DATABASE OTHER THAN SQLITE, USE THIS LINE\n",
    "# Note that this is necessary for parallel execution amongst other things...\n",
    "# os.environ['SNORKELDB'] = 'postgres:///snorkel-intro'\n",
    "\n",
    "import numpy as np\n",
    "from snorkel import SnorkelSession\n",
    "session = SnorkelSession()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We repeat our definition of the `Spouse` `Candidate` subclass, and load the test set:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "from snorkel.models import candidate_subclass\n",
    "\n",
    "Spouse          = candidate_subclass('Spouse', ['person1', 'person2'])\n",
    "test_candidates = session.query(Spouse.id).filter(Spouse.split == 2).all()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Automatically Creating Features\n",
    "We also generate features for the test set; note that we specify `create_new_keyset=False` to use the features from the training set.  For further details see _Intro Tutorial 4_."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
{'fraction_non_alphanumeric': 0.1686091686091686, 'fraction_numerical': 0.002849002849002849, 'mean_word_length': 2.865865865865866, 'pattern_counts': {'":': 64, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '6256520', 'n_tokens_mistral': 1364, 'n_tokens_neox': 1324, 'n_words': 354}