<jupyter_start><jupyter_text>Populate the DB with CSV dataset  Modules needed and configMake sure you have a valid .env file to load the correct MONGO_URL<jupyter_code>import os
from dotenv import load_dotenv
load_dotenv()

import pandas as pd

from pymongo import MongoClient
client = MongoClient(os.getenv("MONGO_URL"))

from datetime import datetime<jupyter_output><empty_output><jupyter_text>Import Dataset<jupyter_code>df_lter = pd.read_csv("./source/penguins_lter.csv")<jupyter_output><empty_output><jupyter_text>Setup MongoDB client and collection for data<jupyter_code>db = client["palmer-penguins"]
collection_kpl = db["kaggle-penguins-lter"]<jupyter_output><empty_output><jupyter_text>Dataset cleaning and formatting<jupyter_code>df_lter["Date Egg"]=df_lter["Date Egg"].apply(lambda e: datetime.strptime(e,'%m/%d/%y'))
df_lter = df_lter.drop("Comments", 1)
df_lter = df_lter.rename(columns=lambda x: x.split(
    "(")[0].strip().replace(" ", "_").lower())
df_lter["clutch_completion"] = df_lter["clutch_completion"].apply(
    lambda x: True if x == "Yes" else False)<jupyter_output><empty_output><jupyter_text>Transform DataFrame to MongoDB documents and insertOutput is the documents inserted on the database<jupyter_code>documents = df_lter.to_dict("records")
collection_kpl.drop()
result = collection_kpl.insert_many(documents)
len(result.inserted_ids)
client.close()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.12119064493267187, 'fraction_numerical': 0.001417434443656981, 'mean_word_length': 8.605442176870747, 'pattern_counts': {'":': 0, '<': 21, '<?xml version=': 0, '>': 21, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '5', 'dir': 'jupyter-structured-clean-dedup', 'id': '443571', 'license': 'MIT', 'n_tokens_mistral': 511, 'n_tokens_neox': 494, 'n_words': 98, 'path': 'data/Data to MongoDB.ipynb', 'repo_name': 'teresaromero/palmer-penguins', 'size': '1411'}