<jupyter_start><jupyter_text>Create TensorFlow model This notebook illustrates: Creating a model using the high-level Estimator API<jupyter_code># change these to try this notebook out
BUCKET = 'qwiklabs-gcp-<KEY>'
PROJECT = 'qwiklabs-gcp-37b9fafbd24bf385'
REGION = 'us-central1'
import os
os.environ['BUCKET'] = BUCKET
os.environ['PROJECT'] = PROJECT
os.environ['REGION'] = REGION
%%bash
if ! gsutil ls | grep -q gs://${BUCKET}/; then
  gsutil mb -l ${REGION} gs://${BUCKET}
fi<jupyter_output><empty_output><jupyter_text>Create TensorFlow model using TensorFlow's Estimator API First, write an input_fn to read the data. Lab Task 1Verify that the headers match your CSV output<jupyter_code>import shutil
import numpy as np
import tensorflow as tf
# Determine CSV, label, and key columns
CSV_COLUMNS = 'weight_pounds,is_male,mother_age,plurality,gestation_weeks,key'.split(',')
LABEL_COLUMN = 'weight_pounds'
KEY_COLUMN = 'key'

# Set default values for each CSV column
DEFAULTS = [[0.0], ['null'], [0.0], ['null'], [0.0], ['nokey']]
TRAIN_STEPS = 1000<jupyter_output><empty_output><jupyter_text>Lab Task 2Fill out the details of the input function below<jupyter_code># Create an input function reading a file using the Dataset API
# Then provide the results to the Estimator API
def read_dataset(filename_pattern, mode, batch_size = 512):
  def _input_fn():
    def decode_csv(line_of_text):
      # TODO #1: Use tf.decode_csv to parse the provided line
      columns = tf.decode_csv(line_of_text, record_defaults=DEFAULTS)
      
      # TODO #2: Make a Python dict.  The keys are the column names, the values are from the parsed data
      features = dict(zip(CSV_COLUMNS, columns))
      
      # TODO #3: Return a tuple of features, label where features is a Python dict and label a float
      label = features.pop(LABEL_COLUMN)
      return features, label
    
    # TODO #4: Use tf.gfile.Glob to create list of files that match pattern
    file_list = tf.gfile.Glob(filename_pattern)

    # Create dataset from file list
    dataset = (tf.data.TextLineDataset(file_list)  # Read text file
                 .map(decode_csv))  # Transform each elem by applying decode_csv fn
    
    # TODO #5: In training mode, shuffle the dataset and repeat indefinitely
    #                (Look at the API for tf.data.dataset shuffle)
    #          The mode input variable will be tf.estimator.ModeKeys.TRAIN if in training mode
    #          Tell the dataset to provide data in batches of batch_size 
    if mode == tf.estimator.ModeKeys.TRAIN:
      epochs = None # Repeat indefinitely
      dataset = dataset.shuffle(buffer_size = 10 * batch_size)
    else:
      epochs = 1
    
    dataset = dataset.repeat(epochs).batch(batch_size)
    
    # This will now return batches of features, label
    return dataset
  return _input_fn<jupyter_output><empty_output><jupyter_text>Lab Task 3Use the TensorFlow feature column API to define appropriate feature columns for your raw features that come from the CSV. Bonus:  Separate your columns into wide columns (categorical, discrete, etc.) and deep columns (numeric, embedding, etc.)<jupyter_code># Define feature columns
# Define feature columns
def get_categorical(name, values):
  return tf.feature_column.indicator_column(
    tf.feature_column.categorical_column_with_vocabulary_list(name, values))

def get_cols():
  # Define column types
  return [\
          get_categorical('is_male', ['True', 'False', 'Unknown']),
          tf.feature_column.numeric_column('mother_age'),
          get_categorical('plurality',
                      ['Single(1)', 'Twins(2)', 'Triplets(3)',
                       'Quadruplets(4)', 'Quintuplets(5)','Multiple(2+)']),
          tf.feature_column.numeric_column('gestation_weeks')
      ]<jupyter_output><empty_output><jupyter_text>Lab Task 4To predict with the TensorFlow model, we also need a serving input function (we'll use this in a later lab). We will want all the inputs from our user.Verify and change the column names and types here as appropriate. These should match your CSV_COLUMNS<jupyter_code># Create serving input function to be able to serve predictions later using provided inputs
def serving_input_fn():
    feature_placeholders = {
        'is_male': tf.placeholder(tf.string, [None]),
        'mother_age': tf.placeholder(tf.float32, [None]),
        'plurality': tf.placeholder(tf.string, [None]),
        'gestation_weeks': tf.placeholder(tf.float32, [None])
    }
    features = {
        key: tf.expand_dims(tensor, -1)
        for key, tensor in feature_placeholders.items()
    }
    return tf.estimator.export.ServingInputReceiver(features, feature_placeholders)<jupyter_output><empty_output><jupyter_text>Lab Task 5Complete the TODOs in this code:<jupyter_code># Create estimator to train and evaluate
def train_and_evaluate(output_dir):
  EVAL_INTERVAL = 300
  run_config = tf.estimator.RunConfig(save_checkpoints_secs = EVAL_INTERVAL,
                                      keep_checkpoint_max = 3)
  # TODO #1: Create your estimator
  estimator = tf.estimator.DNNRegressor(
                        model_dir = output_dir,
                        feature_columns = get_cols(),
                        hidden_units = [64, 32],
                        config = run_config)
  train_spec = tf.estimator.TrainSpec(
                       # TODO #2: Call read_dataset passing in the training CSV file and the appropriate mode
                       input_fn = read_dataset('train.csv', mode = tf.estimator.ModeKeys.TRAIN),
                       max_steps = TRAIN_STEPS)
  exporter = tf.estimator.LatestExporter('exporter', serving_input_fn)
  eval_spec = tf.estimator.EvalSpec(
                       # TODO #3: Call read_dataset passing in the evaluation CSV file and the appropriate mode
                       input_fn = read_dataset('eval.csv', mode = tf.estimator.ModeKeys.EVAL),
                       steps = None,
                       start_delay_secs = 60, # start evaluating after N seconds
                       throttle_secs = EVAL_INTERVAL,  # evaluate every N seconds
                       exporters = exporter)
  tf.estimator.train_and_evaluate(estimator, train_spec, eval_spec)<jupyter_output><empty_output><jupyter_text>Finally, train!<jupyter_code># Run the model
shutil.rmtree('babyweight_trained', ignore_errors = True) # start fresh each time
train_and_evaluate('babyweight_trained')<jupyter_output>INFO:tensorflow:Using config: {'_num_worker_replicas': 1, '_save_summary_steps': 100, '_model_dir': 'babyweight_trained', '_keep_checkpoint_every_n_hours': 10000, '_is_chief': True, '_evaluation_master': '', '_keep_checkpoint_max': 3, '_task_id': 0, '_global_id_in_cluster': 0, '_train_distribute': None, '_task_type': 'worker', '_session_config': None, '_tf_random_seed': None, '_log_step_count_steps': 100, '_service': None, '_cluster_spec': <tensorflow.python.training.server_lib.ClusterSpec object at 0x7f2680a1c710>, '_save_checkpoints_steps': None, '_save_checkpoints_secs': 300, '_master': '', '_num_ps_replicas': 0}
INFO:tensorflow:Running training and evaluation locally (non-distributed).
INFO:tensorflow:Start train and evaluate loop. The evaluate will happen after 300 secs (eval_spec.throttle_secs) or training is finished.
INFO:tensorflow:Calling model_fn.
INFO:tensorflow:Done calling model_fn.
INFO:tensorflow:Create CheckpointSaverHook.
INFO:tensorflow:Graph was finalized.
INFO:tens[...]<jupyter_text>When I ran it, the final lines of the output (above) were:INFO:tensorflow:Saving dict for global step 1000: average_loss = 1.2693067, global_step = 1000, loss = 635.9226INFO:tensorflow:Restoring parameters from babyweight_trained/model.ckpt-1000INFO:tensorflow:Assets added to graph.INFO:tensorflow:No assets to write.INFO:tensorflow:SavedModel written to: babyweight_trained/export/exporter/temp-1517899936/saved_model.pbThe exporter directory contains the final model and the final RMSE (the average_loss) is 1.2693067  Monitor and experiment with training<jupyter_code>from google.datalab.ml import TensorBoard
TensorBoard().start('./babyweight_trained')
for pid in TensorBoard.list()['pid']:
  TensorBoard().stop(pid)
  print('Stopped TensorBoard with pid {}'.format(pid))<jupyter_output>Stopped TensorBoard with pid 3769
{'fraction_non_alphanumeric': 0.09345568703211785, 'fraction_numerical': 0.01690412943733398, 'mean_word_length': 3.9244946492271104, 'pattern_counts': {'":': 0, '<': 33, '<?xml version=': 0, '>': 33, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '8', 'dir': 'jupyter-structured-clean-dedup', 'id': '2539', 'license': 'MIT', 'n_tokens_mistral': 2637, 'n_tokens_neox': 2490, 'n_words': 721, 'path': 'week2/3_tensorflow_dnn.ipynb', 'repo_name': 'isacmoura/machine-learning-tf-google-cloud', 'size': '8293'}