<jupyter_start><jupyter_text>EstimatorsThe actual model is contained in a `tf.estimator.Estimator` instance. In this notebook we will work with the premade TensorFlow estimators. Estimators have a Scikit-Learn style api, where there are methods for `train`, `evaluate`, and `predict`. Estimators automatically provide many features that would have to be hand-coded in otherwise. Two features worth highlighting are:1. Logging and checkpointing as the model trains2. Distributing the computation graph for training on the cloud.The idea behind working with Estimators is that the only things about the computation graph that need to change between the three different phases of machine learning are all about how and what data is fed into the model. For example, during training we iterate for many epochs, while evaluation and prediction we don't; for training and evaluation we have labels in addition to features while for prediction we only have features.The way TensorFlow handles this is that each method takes as input an `input_fn` which is a function that takes no inputs and returns ops to be added to the graph. **Note** when training for example, the `input_fn` is not called on each batch of data or anything like that. It is called only once, and used to add nodes to the computation graph.<jupyter_code>import datetime
import pathlib
import shutil

import numpy as np
import pandas as pd
import tensorflow as tf
from matplotlib import pyplot as plt
plt.style.use('seaborn')<jupyter_output><empty_output><jupyter_text>0. Prepare dataWe need to split our data into train and evaluation datasets. A good design pattern with sharded data is to make separate folders for training and evaluation data. A bad design pattern is to not randomize what is train data and what is test data, but none-the-less we will just set aside `boston-0.csv` as test data and the rest as train.<jupyter_code>data_dir = pathlib.Path('../data/sharded_data')
train_dir = data_dir.parent / 'train'
test_dir = data_dir.parent / 'test'

if not train_dir.exists():
    train_dir.mkdir()
if not test_dir.exists():
    test_dir.mkdir()
    
for f in data_dir.glob('*.csv'):
    if f.stem.endswith('0'):
        shutil.copyfile(f, test_dir / f.name)
    else:
        shutil.copyfile(f, train_dir / f.name)<jupyter_output><empty_output><jupyter_text>1. Estimator setupEstimators take feature columns as one of the inputs to their constructors, so let's start off by instantiating the feature columns we will be using in our model. Since all of our csv files have the same schema, we can load in a few rows of one using pandas to get some useful information about the data we're ingesting into out model.<jupyter_code>data_dir = pathlib.Path('data/sharded_data')
file = (data_dir / 'boston-0.csv').as_posix()

df = pd.read_csv(file, nrows=10)
df.head()
csv_columns = list(df.columns)
label_col = 'target'
# Decoding the csv requires a list of default values to use for each tensor
# produced. The defaults are passed as a list of lists.
default_values = [[0.0]] * 14
default_values[3] = ['_UNKNOWN']; default_values[8] = 0
# Get columns different dtypes
feature_cols = [c for c in csv_columns if c != label_col]
byte_cols = ['chas']
int64_cols = ['rad']
float_cols = [c for c in feature_cols if c not in int64_cols and c not in byte_cols]
# make feature columns
byte_cols = [
    tf.feature_column.indicator_column(
            tf.feature_column.categorical_column_with_vocabulary_list(
            name, ['Y', 'N']
        )
    )
    for name in byte_cols
]
int64_cols = [
    tf.feature_column.numeric_column(name, dtype=tf.int64)
    for name in int64_cols
]
float_cols = [
    tf.feature_column.numeric_column(name, dtype=tf.float32)
    for name in float_cols
]
feature_columns = byte_cols + int64_cols + float_cols<jupyter_output><empty_output><jupyter_text>There is one more optional component to setup for our model, namely the configuration details of the model. To evaluate the model's performance we will want to examine the its train and test error. This requires us to set some configurations in advance.As TensorFlow models train, they write checkpoints after however many steps or hours of training. By default, `Estimator`s are configured to only write a checkpoint once when training is complete. However, evaluations metrics, such as test error, are only computed during checkpoints. Thus to get a test loss curve, we need to set our model to save multiple checkpoints.Setting runtime configuration for our the model is done by passing a `tf.estimator.RunConfig` object to the estimator. Below we set the `RunConfig` to save multiple checkpoints.**Aside:** One of the things that is a bit strange about working in TensorFlow is that a lot configuration and parameters are set by passing various objects which hold configuration and specification details. This is a drastic contrast from, say, Scikit Learn where configurations are set by passing strings or numeric values to an estimator/transformer or whatever. Moreover in Scikit Learn, there are only a few base classes one works with, and a standardized API. With TensorFlow estimators, there is a lot more flexibility in how things are configured, and accordingly, more complicated objects used to configure them. These configuration objects are mostly glorified holders for key/value pairs and don't have methods that a user would typically access.<jupyter_code>model_dir = pathlib.Path('models')
if not model_dir.exists():
    model_dir.mkdir()

model_dir = model_dir / datetime.datetime.now().strftime('model_%Y%m%d_%H%M%S/')

run_config = tf.estimator.RunConfig(
    save_checkpoints_steps=100,
    keep_checkpoint_max=20,
)

estimator = tf.estimator.LinearRegressor(
    feature_columns,
    model_dir=model_dir,
    config=run_config
)<jupyter_output>INFO:tensorflow:Using default config.
INFO:tensorflow:Using config: {'_model_dir': 'models\\model_20190302_144436', '_tf_random_seed': None, '_save_summary_steps': 100, '_save_checkpoints_steps': None, '_save_checkpoints_secs': 600, '_session_config': allow_soft_placement: true
graph_options {
  rewrite_options {
    meta_optimizer_iterations: ONE
  }
}
, '_keep_checkpoint_max': 5, '_keep_checkpoint_every_n_hours': 10000, '_log_step_count_steps': 100, '_train_distribute': None, '_device_fn': None, '_protocol': None, '_eval_distribute': None, '_experimental_distribute': None, '_service': None, '_cluster_spec': <tensorflow.python.training.server_lib.ClusterSpec object at 0x000001F53E8876A0>, '_task_type': 'worker', '_task_id': 0, '_global_id_in_cluster': 0, '_master': '', '_evaluation_master': '', '_is_chief': True, '_num_ps_replicas': 0, '_num_worker_replicas': 1}<jupyter_text>Before moving on, let's recap what the dataset we produced before is doing. In particular, we use defined a function `parse_row` which takes a rstring tensor and returns a tuple of tensors, one for each column of the csv.<jupyter_code>data_dir = pathlib.Path('data')
files = (data_dir / 'train' / 'boston-*.csv').as_posix()

# Training parameters
n_epochs = 10
batch_size = 2

def parse_row(row):
    return tf.decode_csv(row, record_defaults=default_values)

# Build data set
dataset = tf.data.Dataset.list_files(files)
dataset = dataset.flat_map(lambda f: tf.data.TextLineDataset(f).skip(1))
dataset = dataset.map(parse_row)
# Repeat the dataset
dataset = dataset.repeat(n_epochs)
# Shuffle data
dataset = dataset.shuffle(buffer_size=1024)
# Get a batch of data
dataset = dataset.batch(batch_size)
# Preload next batch to speed up training
dataset = dataset.prefetch(buffer_size=batch_size)
iterator = dataset.make_one_shot_iterator()
batch = iterator.get_next()

with tf.Session() as sess:
    batch1 = sess.run(batch)

# Just getting a dataset of individual file names
print(batch1)<jupyter_output>(array([ 2.37857, 14.4208 ], dtype=float32), array([0., 0.], dtype=float32), array([18.1, 18.1], dtype=float32), array([b'Y', b'Y'], dtype=object), array([0.583, 0.74 ], dtype=float32), array([5.871, 6.461], dtype=float32), array([41.9, 93.3], dtype=float32), array([3.724 , 2.0026], dtype=float32), array([24, 24]), array([666., 666.], dtype=float32), array([20.2, 20.2], dtype=float32), array([370.73,  27.49], dtype=float32), array([13.34, 18.05], dtype=float32), array([20.6,  9.6], dtype=float32))<jupyter_text>2. TrainingDuring training and evaluation, data is fed to the estimator as a tuple of features and labels. The features should be in a dict, where the key matches the name of the corresponding `tf.feature_column` feature column the training instance gets sent to. The main thing we need to change about our curent dataset implementation is that instead of `parse_row` returning a tuple of all of the csv folumns, it needs to return these tuples. We also need to only take in training data.<jupyter_code>def parse_row(row):
    # Get tuple of csv row
    parsed = tf.decode_csv(row, record_defaults=default_values)
    # Get dict of col_name: value pairs
    features = dict(zip(csv_columns, parsed))
    # Remove label from features
    label = features.pop(label_col)
    return features, label

# File for training
files = (data_dir / 'train' / 'boston-*.csv').as_posix()

# Build data set
dataset = tf.data.Dataset.list_files(files)
dataset = dataset.flat_map(lambda f: tf.data.TextLineDataset(f).skip(1))
dataset = dataset.map(parse_row)
# Repeat the dataset
dataset = dataset.repeat(n_epochs)
# Shuffle data
dataset = dataset.shuffle(buffer_size=1024)
# Get a batch of data
dataset = dataset.batch(batch_size)
# Preload next batch to speed up training
dataset = dataset.prefetch(buffer_size=batch_size)
iterator = dataset.make_one_shot_iterator()
batch = iterator.get_next()

with tf.Session() as sess:
    batch1 = sess.run(batch)

# Now we get a (dict, value) pair
print(batch1) 
# Wrap in a train input_fn
n_epochs = 5
batch_size = 32

def train_input_fn():
    # Files for training
    files = (data_dir / 'train' / 'boston-*.csv').as_posix()
    # Build data set
    dataset = tf.data.Dataset.list_files(files)
    dataset = dataset.flat_map(lambda f: tf.data.TextLineDataset(f).skip(1))
    dataset = dataset.map(parse_row)
    # Repeat the dataset
    dataset = dataset.repeat(n_epochs)
    # Shuffle data
    dataset = dataset.shuffle(buffer_size=1024)
    # Get a batch of data
    dataset = dataset.batch(batch_size)
    # Preload next batch to speed up training
    dataset = dataset.prefetch(buffer_size=batch_size)
    iterator = dataset.make_one_shot_iterator()
    features, label = iterator.get_next()
    return features, label
# Train that bad boy on all of the data
estimator.train(input_fn=train_input_fn)<jupyter_output>WARNING:tensorflow:From c:\users\stephen.hermes\desktop\tensorflow\venv\lib\site-packages\tensorflow\python\framework\op_def_library.py:263: colocate_with (from tensorflow.python.framework.ops) is deprecated and will be removed in a future version.
Instructions for updating:
Colocations handled automatically by placer.
INFO:tensorflow:Calling model_fn.
WARNING:tensorflow:From c:\users\stephen.hermes\desktop\tensorflow\venv\lib\site-packages\tensorflow\python\feature_column\feature_column_v2.py:2703: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use tf.cast instead.
WARNING:tensorflow:From c:\users\stephen.hermes\desktop\tensorflow\venv\lib\site-packages\tensorflow\python\ops\lookup_ops.py:1137: to_int64 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use tf.cast instead.
WARNING:tensorflow:From c:\users\stephen.hermes\desktop\tensorf[...]<jupyter_text>3. EvaluateSimilar deal, but we don't need to shuffle the data or run for multiple epochs. We also need to take our data from a different directory.<jupyter_code># Wrap in a train input_fn
n_epochs = 1
batch_size = 32

def eval_input_fn():
    # Files for training
    files = (data_dir / 'test' / 'boston-*.csv').as_posix()
    # Build data set
    dataset = tf.data.Dataset.list_files(files)
    dataset = dataset.flat_map(lambda f: tf.data.TextLineDataset(f).skip(1))
    dataset = dataset.map(parse_row)
    # Repeat the dataset
    dataset = dataset.repeat(n_epochs)
    # Shuffle data
    # dataset = dataset.shuffle(buffer_size=1024)
    # Get a batch of data
    dataset = dataset.batch(batch_size)
    # Preload next batch to speed up training
    dataset = dataset.prefetch(buffer_size=batch_size)
    iterator = dataset.make_one_shot_iterator()
    features, label = iterator.get_next()
    return features, label
metrics = estimator.evaluate(input_fn=eval_input_fn)
for k, v in metrics.items():
    print('{0}: {1}'.format(k, v))<jupyter_output>average_loss: 83.72891998291016
label/mean: 23.3217830657959
loss: 2114.1552734375
prediction/mean: 23.391040802001953
global_step: 64<jupyter_text>4. PredictGuess. The output of the `predict` method is an iterator of the results, with each result being a dict with a `'predictions'` key and value the predicted value.<jupyter_code># Wrap in a train input_fn
n_epochs = 1
batch_size = 32

def pred_input_fn():
   # Files for training
    files = (data_dir / 'test' / 'boston-*.csv').as_posix()
    # Build data set
    dataset = tf.data.Dataset.list_files(files)
    dataset = dataset.flat_map(lambda f: tf.data.TextLineDataset(f).skip(1))
    dataset = dataset.map(parse_row)
    # Repeat the dataset
    dataset = dataset.repeat(n_epochs)
    # Shuffle data
    # dataset = dataset.shuffle(buffer_size=1024)
    # Get a batch of data
    dataset = dataset.batch(batch_size)
    # Preload next batch to speed up training
    dataset = dataset.prefetch(buffer_size=batch_size)
    iterator = dataset.make_one_shot_iterator()
    features, label = iterator.get_next()
    # We could actually also return the labels here; they would get ignored.
    # The commented out returns all work
    
    # return features, label
    # return features, None
    return features
    
predictions = estimator.predict(input_fn=pred_input_fn)
for _ in range(5):
    print(next(predictions))
# Examining the results
file = data_dir / 'test' / 'boston-0.csv'
df = pd.read_csv(file)
# Need to re-predict since we used up some of the iterator...
predictions = estimator.predict(input_fn=pred_input_fn)

df['predicted'] = np.array(list(x['predictions'][0] for x in predictions))
df.head()
# Note this is basically the same value as what the eval did.
mse = ((df['target'] - df['predicted']) ** 2).mean()
print('mse:', mse)<jupyter_output>mse: 83.7289246990946<jupyter_text>5. Better design patternsClearly all of the different `input_fn` that needed to be made differ superficially. A better pattern is to make a genric function that can return different dataset ops depending on what we want to do with it.<jupyter_code>def generic_input_fn(file, batch_size=32, n_repeat=1, shuffle=False, return_labels=False):
    # Build data set
    dataset = tf.data.Dataset.list_files(file)
    dataset = dataset.flat_map(lambda f: tf.data.TextLineDataset(f).skip(1))
    dataset = dataset.map(parse_row)
    dataset = dataset.repeat(n_repeat)
    if shuffle:
        dataset = dataset.shuffle(buffer_size=1024)
    dataset = dataset.batch(batch_size)
    dataset = dataset.prefetch(buffer_size=batch_size)
    iterator = dataset.make_one_shot_iterator()
    features, label = iterator.get_next()
    if not return_labels:
        label = None
    return features, label<jupyter_output><empty_output><jupyter_text>Now we can either use `lambda` function to make the desired `input_fn`, or wrap the above function in a function factory.<jupyter_code># Lambda example:
file = (data_dir / 'test' / 'boston-*.csv').as_posix()
estimator.evaluate(input_fn=lambda: generic_input_fn(file, return_labels=True))
# Function factory example:
def get_input_fn(mode):
    if mode == tf.estimator.ModeKeys.TRAIN:
        files = (data_dir / 'train' / 'boston-*.csv').as_posix()
        return lambda: generic_input_fn(
            files, 
            batch_size=32, 
            n_repeat=100, 
            shuffle=True, 
            return_labels=True
        )
    elif mode == tf.estimator.ModeKeys.EVAL:
        files = (data_dir / 'test' / 'boston-*.csv').as_posix()
        return lambda: generic_input_fn(
            files, 
            batch_size=32, 
            n_repeat=20, 
            shuffle=False, 
            return_labels=True
        )
    elif mode == tf.estimator.ModeKeys.PREDICT:
        files = (data_dir / 'test' / 'boston-*.csv').as_posix()
        return lambda: generic_input_fn(
            files, 
            batch_size=32, 
            n_repeat=1, 
            shuffle=False, 
            return_labels=False
        )
estimator.evaluate(input_fn=get_input_fn(tf.estimator.ModeKeys.EVAL))<jupyter_output>INFO:tensorflow:Calling model_fn.
INFO:tensorflow:Done calling model_fn.
INFO:tensorflow:Starting evaluation at 2019-03-02T19:44:44Z
INFO:tensorflow:Graph was finalized.
INFO:tensorflow:Restoring parameters from models\model_20190302_144436\model.ckpt-64
INFO:tensorflow:Running local_init_op.
INFO:tensorflow:Done running local_init_op.
INFO:tensorflow:Finished evaluation at 2019-03-02-19:44:44
INFO:tensorflow:Saving dict for global step 64: average_loss = 83.728905, global_step = 64, label/mean = 23.321775, loss = 2642.6936, prediction/mean = 23.391035
INFO:tensorflow:Saving 'checkpoint_path' summary for global step 64: models\model_20190302_144436\model.ckpt-64<jupyter_text>In the function factory implementation we used the `ModeKeys` object, which is basically just an object whose attributes are standardized names for training, evaluation, and prediction phases (to avoid things like one person's code using `'eval'` vs. another person's code using `'test'`, etc.) 6. Train and evaluateLastly, TensorFlow provides a utility function `tf.estimator.train_and_evaluate` which trains the model, then evaluates it. Why not just do those two steps one after the other as above you ask? According to the TensorFlow documentation, `train_and_evaluate` provides consistent behavior for both local and distributed training/evaluation. In particular, it let's us test out our model training and evaluation pipelines locally, and get the same behavior when we move to the cloud to train with Cloud ML Engine.**Note** In the olden days, CMLE worked by running an instance of the TensorFlow `Experiment` class, which is now deprecated. This class was replaced by the `train_and_evaluate` function. You will run across examples using the `Experiment` class online. Ignore them.In order to use `train_and_evaluate` we need to pass to it specifications for how training and evaluation should run. These are controlled by `TrainSpec` and `EvalSpec` objects, respectively. It is best to think of these objects as just packages of parameters and not worry about them too much. They don't functionality as things in their own right.Both `TrainSpec` and `EvalSpec` take as their primary argument the corresponding `input_fn`. For `TrainSpec`, there is another argument for `max_steps` which specifies the maximum number of steps for which to train. If set to `None`, the TensorFlow documentation says that the model will train forever, but this is wrong. It trains until the dataset iterator throws an `OutOfRangeError`.<jupyter_code>train_spec = tf.estimator.TrainSpec(
    input_fn=get_input_fn(tf.estimator.ModeKeys.TRAIN),
    max_steps=None
)
eval_spec = tf.estimator.EvalSpec(
    input_fn=get_input_fn(
        tf.estimator.ModeKeys.EVAL
    )
)
tf.estimator.train_and_evaluate(estimator, train_spec, eval_spec)<jupyter_output>INFO:tensorflow:Not using Distribute Coordinator.
INFO:tensorflow:Running training and evaluation locally (non-distributed).
INFO:tensorflow:Start train and evaluate loop. The evaluate will happen after every checkpoint. Checkpoint frequency is determined based on RunConfig arguments: save_checkpoints_steps None or save_checkpoints_secs 600.
INFO:tensorflow:Calling model_fn.
INFO:tensorflow:Done calling model_fn.
INFO:tensorflow:Create CheckpointSaverHook.
INFO:tensorflow:Graph was finalized.
INFO:tensorflow:Restoring parameters from models\model_20190302_144436\model.ckpt-64
WARNING:tensorflow:From c:\users\stephen.hermes\desktop\tensorflow\venv\lib\site-packages\tensorflow\python\training\saver.py:1070: get_checkpoint_mtimes (from tensorflow.python.training.checkpoint_management) is deprecated and will be removed in a future version.
Instructions for updating:
Use standard file utilities to get mtimes.
INFO:tensorflow:Running local_init_op.
INFO:tensorflow:Done running local_init_op.[...]<jupyter_text>We can also export the model for serving while using `train_and_evaluate`. To do so, we need to pass an `Exporter` to our `EvalSpec`. Just like how the `TrainSpec` and `EvalSpec` need an `input_fn` to know what nodes to add to the model's computation graph to prepare it for training and evaluation, the `Exporter` needs a `serving_input_fn` to know what nodes to add to the graph for serving predictions. There are a few different types of exportersYou might think that we would just use the `prediction_input_fn` above for this `serving_input_fn`, but this isn't quite the case. Our `prediction_input_fn` is designed for ingesting in data from a file we have access to. When we serve the model on the cloud, predictions will be made by sending json requests to the server serving our model. Thus we need to add `tf.placeholder` nodes for input to the model, and some logic to parse a json request. This additional logic is handled by a `ServingInputReceiver` object. *I need to look into these more...*OKAY SO : Eval steps only called at checkpoints, so in tensorboard, you will have an eval metric point for each checkpoint. In the model_dir there is an initial checkpoint, which doesnt count and the rest are when the eval steps are called. so if ya want to print off an actual eval loss curve you need to set somethign to checkpoint more frequentlyAccording to Tensorflow documentdation:Checkpointing FrequencyBy default, the Estimator saves checkpoints in the model_dir according to the following schedule:Writes a checkpoint every 10 minutes (600 seconds).Writes a checkpoint when the train method starts (first iteration) and completes (final iteration).Retains only the 5 most recent checkpoints in the directory.You may alter the default schedule by taking the following steps:Create a tf.estimator.RunConfig object that defines the desired schedule.When instantiating the Estimator, pass that RunConfig object to the Estimator's config argument.For example, the following code changes the checkpointing schedule to every 20 minutes and retains the 10 most recent checkpoints:<jupyter_code>def get_dtype(col):
    # Not a great solution, but whatever
    if col == 'chas':
        return tf.string
    elif col == 'rad':
        return tf.int64
    else:
        return tf.float32

def serving_input_fn():
    # Expected json input
    feature_placeholders = dict(
        (col, tf.placeholder(get_dtype(col), (None))) for col in feature_cols
    )
    # Can do aditional transformations to prepare for model if needed
    features = feature_placeholders
    return tf.estimator.export.ServingInputReceiver(features, feature_placeholders)   
# Reinstantiating model so that logs aren't polluted by previous runs
model_dir = pathlib.Path('models')
if not model_dir.exists():
    model_dir.mkdir()

model_dir = model_dir / datetime.datetime.now().strftime('model_%Y%m%d_%H%M%S/')

run_config = tf.estimator.RunConfig(
    save_checkpoints_steps=100,
    keep_checkpoint_max=20,
)

estimator = tf.estimator.LinearRegressor(
    feature_columns,
    model_dir=model_dir,
    config=run_config
)

train_spec = tf.estimator.TrainSpec(
    input_fn=get_input_fn(tf.estimator.ModeKeys.TRAIN),
    max_steps=None
)

# Add exporter to EvalSpec
exporter = tf.estimator.LatestExporter('exporter', serving_input_fn)

eval_spec = tf.estimator.EvalSpec(
    input_fn=get_input_fn(tf.estimator.ModeKeys.EVAL),
#     exporters=exporter,
)

tf.estimator.train_and_evaluate(estimator, train_spec, eval_spec)<jupyter_output>INFO:tensorflow:Using config: {'_model_dir': 'models\\model_20190302_144601', '_tf_random_seed': None, '_save_summary_steps': 100, '_save_checkpoints_steps': 100, '_save_checkpoints_secs': None, '_session_config': allow_soft_placement: true
graph_options {
  rewrite_options {
    meta_optimizer_iterations: ONE
  }
}
, '_keep_checkpoint_max': 20, '_keep_checkpoint_every_n_hours': 10000, '_log_step_count_steps': 100, '_train_distribute': None, '_device_fn': None, '_protocol': None, '_eval_distribute': None, '_experimental_distribute': None, '_service': None, '_cluster_spec': <tensorflow.python.training.server_lib.ClusterSpec object at 0x000001F544274668>, '_task_type': 'worker', '_task_id': 0, '_global_id_in_cluster': 0, '_master': '', '_evaluation_master': '', '_is_chief': True, '_num_ps_replicas': 0, '_num_worker_replicas': 1}
INFO:tensorflow:Not using Distribute Coordinator.
INFO:tensorflow:Running training and evaluation locally (non-distributed).
INFO:tensorflow:Start train and eval[...]
{'fraction_non_alphanumeric': 0.08728615091493669, 'fraction_numerical': 0.021791767554479417, 'mean_word_length': 4.809084620705557, 'pattern_counts': {'":': 0, '<': 43, '<?xml version=': 0, '>': 43, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '12', 'dir': 'jupyter-structured-clean-dedup', 'id': '60035', 'license': 'MIT', 'n_tokens_mistral': 7911, 'n_tokens_neox': 7425, 'n_words': 2594, 'path': 'notebooks/3. Estimators.ipynb', 'repo_name': 'stephenhermes/tensorflow-tutorials', 'size': '25193'}