<commit_before>{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "V8k9buNKTfud"
   },
   "source": [
    "# Label and feature engineering\n",
    "\n",
    "This lab is *optional*. It demonstrates advanced SQL queries for time-series engineering. For real-world problems, this type of feature engineering code is essential. If you are pursuing a time-series project for open project week, feel free to use this code as a template. \n",
    "\n",
    "---\n",
    "\n",
    "Learning objectives:\n",
    "\n",
    "1. Learn how to use BigQuery to build time-series features and labels for forecasting\n",
    "2. Learn how to visualize and explore features.\n",
    "3. Learn effective scaling and normalizing techniques to improve our modeling results\n",
    "\n",
    "Now that we have explored the data, let's start building our features, so we can build a model.\n",
    "\n",
    "<h3><font color=\"#4885ed\">Feature Engineering</font> </h3>\n",
    "\n",
    "Use the `price_history` table, we can look at past performance of a given stock, to try to predict it's future stock price. In this notebook we will be focused on cleaning and creating features from this table. \n",
    "\n",
    "There are typically two different approaches to creating features with time-series data. \n",
    "\n",
    "**One approach** is aggregate the time-series into \"static\" features, such as \"min_price_over_past_month\" or \"exp_moving_avg_past_30_days\". Using this approach, we can use a deep neural network or a more \"traditional\" ML model to train. Notice we have essentially removed all sequention information after aggregating. This assumption can work well in practice. \n",<commit_msg>Added note to run dependent notebook<commit_after>{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "V8k9buNKTfud"
   },
   "source": [
    "# Label and feature engineering\n",
    "\n",
    "This lab is *optional*. It demonstrates advanced SQL queries for time-series engineering. For real-world problems, this type of feature engineering code is essential. If you are pursuing a time-series project for open project week, feel free to use this code as a template. \n",
    "\n",
    "**Note: In the previous lab we explored the data, if you haven’t run the previous notebooks, go back to [optional_1_data_exploration.ipynb](../solutions/optional_1_data_exploration.ipynb) and run it.**\n",
    "\n",
    "---\n",
    "\n",
    "Learning objectives:\n",
    "\n",
    "1. Learn how to use BigQuery to build time-series features and labels for forecasting\n",
    "2. Learn how to visualize and explore features.\n",
    "3. Learn effective scaling and normalizing techniques to improve our modeling results\n",
    "\n",
    "Now that we have explored the data, let's start building our features, so we can build a model.\n",
    "\n",
    "<h3><font color=\"#4885ed\">Feature Engineering</font> </h3>\n",
    "\n",
    "Use the `price_history` table, we can look at past performance of a given stock, to try to predict it's future stock price. In this notebook we will be focused on cleaning and creating features from this table. \n",
    "\n",
    "There are typically two different approaches to creating features with time-series data. \n",
    "\n",
    "**One approach** is aggregate the time-series into \"static\" features, such as \"min_price_over_past_month\" or \"exp_moving_avg_past_30_days\". Using this approach, we can use a deep neural network or a more \"traditional\" ML model to train. Notice we have essentially removed all sequention information after aggregating. This assumption can work well in practice. \n",
{'fraction_non_alphanumeric': 0.11305048335123523, 'fraction_numerical': 0.007518796992481203, 'mean_word_length': 3.4292508917954816, 'pattern_counts': {'":': 12, '<': 11, '<?xml version=': 0, '>': 11, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '7326260', 'n_tokens_mistral': 1068, 'n_tokens_neox': 1039, 'n_words': 482}