<jupyter_start><jupyter_text>Image data can stay in multiple ways. If one needs to use tensorflow2 for image classification, then they can utilize the high performance dataset API that are introduced in tensorflow. Images can come with labels in different ways. Here are all the techniques that can be possible with tensorflow dataset API.<jupyter_code>import tensorflow as tf
from tensorflow import keras
from tensorflow.keras.preprocessing.image import ImageDataGenerator
tf.__version__<jupyter_output><empty_output><jupyter_text>Tesnsorflow version should be atleast 2.3   When class labels are in the folder names<jupyter_code>!tree -L 1 datasets/plantseed/train/<jupyter_output><empty_output><jupyter_text>Define the path where all the files would exist for training<jupyter_code>BASE_PATH = './datasets/plantseed/train/'
img_height = 28
img_width = 28<jupyter_output><empty_output><jupyter_text>Define the dataset for both validation and training<jupyter_code>train_dataset = keras.preprocessing.image_dataset_from_directory(
    directory=BASE_PATH,
    labels='inferred',
    label_mode='int',
    color_mode='rgb',
    batch_size=16,
    image_size=(img_height, img_width),
    shuffle=True,
    seed=42,
    validation_split=0.2,
    subset="training",
)

valid_dataset = keras.preprocessing.image_dataset_from_directory(
    directory=BASE_PATH,
    labels='inferred',
    label_mode='int',
    color_mode='rgb',
    batch_size=16,
    image_size=(img_height, img_width),
    shuffle=True,
    seed=42,
    validation_split=0.2,
    subset="validation",
)<jupyter_output><empty_output><jupyter_text>Using some functions for data augmentation<jupyter_code>def augment(X, y):
    """
    This augments the image data on the fly.
    """
    X = tf.image.random_brightness(X, max_delta=0.05)
    X = tf.image.random_contrast(X, 0.2, 0.5)
    X = tf.image.flip_up_down(X)
    return X, y<jupyter_output><empty_output><jupyter_text>Apply the augmentation and other functions<jupyter_code>train_dataset = train_dataset.map(augment)<jupyter_output><empty_output><jupyter_text>Now loop over the dataset and fit it with the model<jupyter_code>EPOCHS = 10
# Using custom loops
for epochs in range(EPOCHS):
    for X, y in train_dataset:
        # train here
        print(X.shape)
        print(y.shape)
        break
    break
# Or Keras functional API
model.compile(
    optimizer=keras.optimizers.Adam(),
    loss=[keras.losses.SparseCategoricalCrossentropy(from_logits=True),],
    metrics=["accuracy"],
)

model.fit(train_dataset, epochs=EPOCHS, verbose=2)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.0933641975308642, 'fraction_numerical': 0.011574074074074073, 'mean_word_length': 5.115566037735849, 'pattern_counts': {'":': 0, '<': 29, '<?xml version=': 0, '>': 29, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '7', 'dir': 'jupyter-structured-clean-dedup', 'id': '301985', 'license': 'MIT', 'n_tokens_mistral': 872, 'n_tokens_neox': 841, 'n_words': 200, 'path': 'tfdataImage.ipynb', 'repo_name': 'MayukhSobo/Awesome-Snippets', 'size': '2592'}