<jupyter_start><jupyter_text>Time Series Exercise -  Follow along with the instructions in bold. Watch the solutions video if you get stuck!  The Data** Source: https://datamarket.com/data/set/22ox/monthly-milk-production-pounds-per-cow-jan-62-dec-75!ds=22ox&display=line ****Monthly milk production: pounds per cow. Jan 62 - Dec 75** ** Import numpy pandas and matplotlib ** ** Use pandas to read the csv of the monthly-milk-production.csv file and set index_col='Month' ** ** Check out the head of the dataframe** ** Make the index a time series by using: **    milk.index = pd.to_datetime(milk.index) ** Plot out the time series data. ** ___  Train Test Split** Let's attempt to predict a year's worth of data. (12 months or 12 steps into the future) **** Create a test train split using indexing (hint: use .head() or tail() or .iloc[]). We don't want a random train test split, we want to specify that the test set is the last 12 months of data is the test set, with everything before it is the training. **  Scale the Data** Use sklearn.preprocessing to scale the data using the MinMaxScaler. Remember to only fit_transform on the training data, then transform the test data. You shouldn't fit on the test data as well, otherwise you are assuming you would know about future behavior!**  Batch Function** We'll need a function that can feed batches of the training data. We'll need to do several things that are listed out as steps in the comments of the function. Remember to reference the previous batch method from the lecture for hints. Try to fill out the function template below, this is a pretty hard step, so feel free to reference the solutions! **<jupyter_code>def next_batch(training_data,batch_size,steps):
    """
    INPUT: Data, Batch Size, Time Steps per batch
    OUTPUT: A tuple of y time series results. y[:,:-1] and y[:,1:]
    """
    
    # STEP 1: Use np.random.randint to set a random starting point index for the batch.
    # Remember that each batch needs have the same number of steps in it.
    # This means you should limit the starting point to len(data)-steps
    
    # STEP 2: Now that you have a starting index you'll need to index the data from
    # the random start to random start + steps + 1. Then reshape this data to be (1,steps+1)
    
    # STEP 3: Return the batches. You'll have two batches to return y[:,:-1] and y[:,1:]
    # You'll need to reshape these into tensors for the RNN to .reshape(-1,steps,1)<jupyter_output><empty_output><jupyter_text>Setting Up The RNN Model ** Import TensorFlow **  The Constants** Define the constants in a single cell. You'll need the following (in parenthesis are the values I used in my solution, but you can play with some of these): *** Number of Inputs (1)* Number of Time Steps (12)* Number of Neurons per Layer (100)* Number of Outputs (1)* Learning Rate (0.03)* Number of Iterations for Training (4000)* Batch Size (1) ** Create Placeholders for X and y. (You can change the variable names if you want). The shape for these placeholders should be [None,num_time_steps-1,num_inputs] and [None, num_time_steps-1, num_outputs] The reason we use num_time_steps-1 is because each of these will be one step shorter than the original time steps size, because we are training the RNN network to predict one point into the future based on the input sequence.**   ** Now create the RNN Layer, you have complete freedom over this, use tf.contrib.rnn and choose anything you want, OutputProjectionWrappers, BasicRNNCells, BasicLSTMCells, MultiRNNCell, GRUCell etc... Keep in mind not every combination will work well! (If in doubt, the solutions used an Outputprojection Wrapper around a basic LSTM cell with relu activation.** ** Now pass in the cells variable into tf.nn.dynamic_rnn, along with your first placeholder (X)**  Loss Function and Optimizer** Create a Mean Squared Error Loss Function and use it to minimize an AdamOptimizer, remember to pass in your learning rate. ** ** Initialize the global variables ** ** Create an instance of tf.train.Saver() **  Session** Run a tf.Session that trains on the batches created by your next_batch function. Also add an a loss evaluation for every 100 training iterations. Remember to save your model after you are done training. **<jupyter_code>with tf.Session() as sess:
    # CODE HERE!
    
    # Save Model for Later
    saver.save(sess, "./ex_time_series_model")<jupyter_output>0 	MSE: 0.0628359
100 	MSE: 0.00854151
200 	MSE: 0.00699567
300 	MSE: 0.0156167
400 	MSE: 0.00777238
500 	MSE: 0.00864684
600 	MSE: 0.0159645
700 	MSE: 0.00656524
800 	MSE: 0.0076439
900 	MSE: 0.006401
1000 	MSE: 0.00369383
1100 	MSE: 0.00988994
1200 	MSE: 0.00803645
1300 	MSE: 0.00575964
1400 	MSE: 0.0151093
1500 	MSE: 0.00752775
1600 	MSE: 0.00542804
1700 	MSE: 0.00162975
1800 	MSE: 0.00230503
1900 	MSE: 0.00416592
2000 	MSE: 0.00369024
2100 	MSE: 0.00397327
2200 	MSE: 0.00235241
2300 	MSE: 0.00472639
2400 	MSE: 0.00418429
2500 	MSE: 0.00693244
2600 	MSE: 0.00375631
2700 	MSE: 0.00236074
2800 	MSE: 0.00268888
2900 	MSE: 0.00708326
3000 	MSE: 0.00418036
3100 	MSE: 0.00486205
3200 	MSE: 0.00659863
3300 	MSE: 0.00621194
3400 	MSE: 0.00150676
3500 	MSE: 0.0050875
3600 	MSE: 0.00395521
3700 	MSE: 0.00200348
3800 	MSE: 0.00386259
3900 	MSE: 0.00360108<jupyter_text>______  Predicting Future (Test Data) ** Show the test_set (the last 12 months of your original complete data set) **<jupyter_code># CODE HERE<jupyter_output><empty_output><jupyter_text>** Now we want to attempt to predict these 12 months of data, using only the training data we had. To do this we will feed in a seed training_instance of the last 12 months of the training_set of data to predict 12 months into the future. Then we will be able to compare our generated 12 months to our actual true historical values from the test set! **  Generative Session NOTE: Recall that our model is really only trained to predict 1 time step ahead, asking it to generate 12 steps is a big ask, and technically not what it was trained to do! Think of this more as generating new values based off some previous pattern, rather than trying to directly predict the future. You would need to go back to the original model and train the model to predict 12 time steps ahead to really get a higher accuracy on the test data. (Which has its limits due to the smaller size of our data set)** Fill out the session code below to generate 12 months of data based off the last 12 months of data from the training set. The hardest part about this is adjusting the arrays with their shapes and sizes. Reference the lecture for hints.**<jupyter_code>with tf.Session() as sess:
    
    # Use your Saver instance to restore your saved rnn time series model
    saver.restore(sess, "./ex_time_series_model")

    # CODE HERE!<jupyter_output>INFO:tensorflow:Restoring parameters from ./ex_time_series_model
{'fraction_non_alphanumeric': 0.07032615026208504, 'fraction_numerical': 0.08299359347699475, 'mean_word_length': 4.133781763826607, 'pattern_counts': {'":': 0, '<': 15, '<?xml version=': 0, '>': 15, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '4', 'dir': 'jupyter-structured-clean-dedup', 'id': '491272', 'license': 'MIT', 'n_tokens_mistral': 2462, 'n_tokens_neox': 2049, 'n_words': 1016, 'path': '04-Recurrent-Neural-Networks/02-Time-Series-Exercise.ipynb', 'repo_name': 'joemmetry/tf-dl-notes', 'size': '6868'}