<jupyter_start><jupyter_text>Modified HDF5 LoaderThe HDF5 loader now has 3 new parameters: flip_enable, rot90_enable, and crop_enable.When true, these will cause the HDF5 iterator to randomly flip, rotate (90 degrees only), and/or randomly crop (specified with the border_size value). The random crop is border_size:-border_size along each dimension (except the channel). There's a random offset from the center (0 to border_size) with the crop. Note: I haven't figured out how to properly do the mean subtraction when there is a crop. Ideally, the mean substraction should be done before the crop, but I think the way it is implemented is that the subtraction happens on the GPU. For now, I just don't allow mean subtraction when cropping.<jupyter_code>from neon.data import HDF5Iterator  # Neon's HDF5 data loader
from neon.backends import gen_backend

import itertools
be = gen_backend(backend='cpu', batch_size=24)  
#train_set = HDF5Iterator('/Users/ganthony/Desktop/luna16_roi_subset0_augmented.h5')
#train_set_flipped = HDF5Iterator('/Users/ganthony/Desktop/luna16_roi_subset0_augmented.h5', 
#                                 flip_enable=False, rot90_enable=False, crop_enable=True, border_size=7)
train_set = HDF5Iterator('/Users/ganthony/Desktop/mnist_test.h5')
train_set_flipped = HDF5Iterator('/Users/ganthony/Desktop/mnist_test.h5', flip_enable=True, rot90_enable=True, crop_enable=False, border_size=6)
from matplotlib import pyplot as plt
import numpy as np

%matplotlib inline
j = 0
for x,y in itertools.izip(train_set, train_set_flipped):
    
    for i in range(x[0].shape[1]):
        plt.subplot(1,2,1)
        plt.imshow(x[0].get()[:,i].reshape(train_set.lshape)[0], cmap='bone')
        plt.subplot(1,2,2)
        plt.imshow(y[0].get()[:,i].reshape(train_set_flipped.crop_shape)[0], cmap='bone')
        plt.show()
        
    j += 1
    if (j >= 1):
        break<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.10015649452269171, 'fraction_numerical': 0.025039123630672927, 'mean_word_length': 4.527377521613833, 'pattern_counts': {'":': 0, '<': 5, '<?xml version=': 0, '>': 6, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '1', 'dir': 'jupyter-structured-clean-dedup', 'id': '533876', 'license': 'Apache-2.0', 'n_tokens_mistral': 659, 'n_tokens_neox': 634, 'n_words': 181, 'path': 'Modified_HDF5_With_Augmentation.ipynb', 'repo_name': 'tonyreina/neon', 'size': '1917'}