<jupyter_start><jupyter_text>**Objective : To collect data from the data source and process all images to make them suitable for training.**The dataset consist of 23479 different face images with their respective age and gender [male(0) and female(1)] values. Dataset link : https://www.kaggle.com/nipunarora8/age-gender-and-ethnicity-face-data-csv<jupyter_code>import numpy as np
import pandas as pd
import seaborn as sns
import matplotlib.pyplot as plt
import cv2
dataset_path = r"/content/drive/MyDrive/age_gender.csv"

# loading the dataset from drive
df = pd.read_csv(dataset_path)
df.head(10)<jupyter_output><empty_output><jupyter_text>**Exploratory Data Analysis (EDA)**<jupyter_code>df.info()
# Lets find how many images do we have for each age group

plt.figure(figsize=(25,8))
sns.countplot(x='age', data=df) #age distribution
# Lets find how many images do we have for each gender group

plt.figure(figsize=(8,6))
sns.countplot(x='gender', data=df) #gender distribution
plt.figure(figsize=(8,6))
sns.countplot(x='ethnicity', data=df) #ethnicity distribution<jupyter_output><empty_output><jupyter_text>**Processing the data**Currently all images are in 1D format in "pixels" column,we'll have to reshape them to 3D format so that we can train models.<jupyter_code># Highest age in our dataset
print("Highest age in our dataset : ",max(df["age"].unique()))<jupyter_output>Highest age in our dataset :  116<jupyter_text>**Currently our dataset consists of face images of people between age of (1-116).We'll remove all images with age above 60,so that our dataset size reduces and it also makes it easy for model training.**<jupyter_code># Removing all rows with age>60
df = df[df["age"] <= 60]
df.reset_index(inplace=True,drop=True)

# Highest age in our dataset
print("Highest age in our dataset : ",max(df["age"].unique()))

print("Total samples : ",len(df))
# Lets find how many images do we have for each age group

plt.figure(figsize=(25,8))
sns.countplot(x='age', data=df) #age distribution
from tqdm import tqdm

# converting data to numpy array
x = df['pixels'].apply(lambda x: np.array(x.split(), dtype="float32"))

X_t = []
for i in tqdm(range(x.shape[0])):
    # reshaping the data to (n,48,48,1)
    X_t.append(x[i].reshape(48, 48, 1))
#normalization
X_t=np.array(X_t)/255.0

print(X_t[0])
# plotting images 

for i in range(10):
  plt.imshow(cv2.cvtColor(X_t[i+1500], cv2.COLOR_BGR2RGB))
  plt.show()

# seperating x and y 
x = X_t
y = df[["age","gender"]]
x.shape
y.shape
y.head(100)
# print images with their ground truth
for i in range(10):
  plt.imshow(cv2.cvtColor(x[i*800], cv2.COLOR_BGR2RGB))
  print(y.iloc[i*800])
  plt.show()
# saving images as numpy file for next notebook
np.save("x.npy",x)
# saving y as numpy file for next notebook
np.save("y.npy",y)
# Creating a subplot of images

plt.figure(figsize=(15,15))

for i in range(13):
  if i==0:
    continue
  plt.subplot(3,4,i)
  plt.tick_params(left = False, right = False , labelleft = False ,
                labelbottom = False, bottom = False)
  plt.imshow(cv2.cvtColor(x[i*800], cv2.COLOR_BGR2RGB))

# Saving the figure.
plt.savefig("output.jpg")
plt.tight_layout(pad=0.7,h_pad=0.2)
plt.show()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.13196389666977903, 'fraction_numerical': 0.029878618113912233, 'mean_word_length': 5.314341846758349, 'pattern_counts': {'":': 0, '<': 17, '<?xml version=': 0, '>': 17, 'https://': 1, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '4', 'dir': 'jupyter-structured-clean-dedup', 'id': '420979', 'license': 'MIT', 'n_tokens_mistral': 1224, 'n_tokens_neox': 1151, 'n_words': 334, 'path': 'Colab Notebooks/Data_Collection_&_Processing.ipynb', 'repo_name': 'deepeshdm/Age-Gender-Detection', 'size': '3213'}