<commit_before>{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Homework 5: Semantic date parsing"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "__author__ = \"<NAME>\"\n",
    "__version__ = \"CS224u, Stanford, Spring 2018 term\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Contents\n",
    "\n",
    "0. [Submission](#Submission)\n",
    "0. [Set-up](#Set-up)\n",
    "0. [Goals](#Goals)\n",
    "0. [Dataset](#Dataset)\n",
    "0. [Core interpretive functionality](#Core-interpretive-functionality)\n",
    "0. [Questions 1–6: Build the crude grammar [6 points]](#Questions-1–6:-Build-the-crude-grammar-[6-points])\n",
    "0. [Check oracle accuracy](#Check-oracle-accuracy)\n",
    "0. [Question 7: Timezone sensitivity [2 points]](#Question-7:-Timezone-sensitivity-[2-points])\n",
    "0. [Question 8: Year-string length [2 points]](#Question-8:-Year-string-length-[2-points])\n",
    "0. [Putting the feature functions together](#Putting-the-feature-functions-together)\n",
    "0. [Train a model](#Train-a-model)\n",
    "0. [Evaluate the trained model](#Evaluate-the-trained-model)\n",
    "0. [A rule-based baseline](#A-rule-based-baseline)\n",
    "0. [Inspect the trained model](#Inspect-the-trained-model)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Submission\n",
    "\n",
    "For this homework, the submission procedure is a bit different than before:\n",
    "    \n",
    "1. Run the complete notebook, or at least up to the section 'A rule-based baseline'. (It's probably easier to just run the whole notebook.) Leave all the output embedded.\n",
    "1. Save the notebook and upload the notebook file to Canvas: \n",
    "\n",
    "It's fine to add cells for your work, but don't delete any, so that we're sure we have the output we need for evaluation.\n",
    "\n",
    "As usual, evaluating your work is not really our central goal. In fact, we've included various tests and checks to help ensure that you do everything correctly. Our goal is to give you the experience of designing and optimizing a semantic parser that could have real-world applicability."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Set-up\n",
    "\n",
    "0. Make sure the `sys.path.append` value is the path to your local [SippyCup repository](https://github.com/wcmac/sippycup). (Alternatively, you can add SippyCup to your Python path; see one of the teaching team if you'd like to do that but aren't sure how.)\n",
    "\n",
    "0. Make sure that `semparse_dateparse_data.py` is in the current directory (or available via your Python path)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import sys\n",
    "sys.path.append('../sippycup')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},<commit_msg>Adds the HW5 Canvas submission URL<commit_after>{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Homework 5: Semantic date parsing"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "__author__ = \"<NAME>\"\n",
    "__version__ = \"CS224u, Stanford, Spring 2018 term\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Contents\n",
    "\n",
    "0. [Submission](#Submission)\n",
    "0. [Set-up](#Set-up)\n",
    "0. [Goals](#Goals)\n",
    "0. [Dataset](#Dataset)\n",
    "0. [Core interpretive functionality](#Core-interpretive-functionality)\n",
    "0. [Questions 1–6: Build the crude grammar [6 points]](#Questions-1–6:-Build-the-crude-grammar-[6-points])\n",
    "0. [Check oracle accuracy](#Check-oracle-accuracy)\n",
    "0. [Question 7: Timezone sensitivity [2 points]](#Question-7:-Timezone-sensitivity-[2-points])\n",
    "0. [Question 8: Year-string length [2 points]](#Question-8:-Year-string-length-[2-points])\n",
    "0. [Putting the feature functions together](#Putting-the-feature-functions-together)\n",
    "0. [Train a model](#Train-a-model)\n",
    "0. [Evaluate the trained model](#Evaluate-the-trained-model)\n",
    "0. [A rule-based baseline](#A-rule-based-baseline)\n",
    "0. [Inspect the trained model](#Inspect-the-trained-model)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Submission\n",
    "\n",
    "For this homework, the submission procedure is a bit different than before:\n",
    "    \n",
    "1. Run the complete notebook, or at least up to the section 'A rule-based baseline'. (It's probably easier to just run the whole notebook.) Leave all the output embedded.\n",
    "1. Save the notebook and upload the notebook file to Canvas: https://canvas.stanford.edu/courses/83399/assignments/135084\n",
    "\n",
    "It's fine to add cells for your work, but don't delete any, so that we're sure we have the output we need for evaluation.\n",
    "\n",
    "As usual, evaluating your work is not really our central goal. In fact, we've included various tests and checks to help ensure that you do everything correctly. Our goal is to give you the experience of designing and optimizing a semantic parser that could have real-world applicability."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Set-up\n",
    "\n",
    "0. Make sure the `sys.path.append` value is the path to your local [SippyCup repository](https://github.com/wcmac/sippycup). (Alternatively, you can add SippyCup to your Python path; see one of the teaching team if you'd like to do that but aren't sure how.)\n",
    "\n",
    "0. Make sure that `semparse_dateparse_data.py` is in the current directory (or available via your Python path)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import sys\n",
    "sys.path.append('../sippycup')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
{'fraction_non_alphanumeric': 0.1735403525796539, 'fraction_numerical': 0.014879508329289989, 'mean_word_length': 3.1840324763193504, 'pattern_counts': {'":': 52, '<': 5, '<?xml version=': 0, '>': 5, 'https://': 3, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '1040699', 'n_tokens_mistral': 2159, 'n_tokens_neox': 2023, 'n_words': 615}