<jupyter_start><jupyter_text>Codebook **Authors**: <NAME>Documenting existing data files of DaanMatch with information about location, owner, "version", source etc.<jupyter_code>import pandas as pd
import numpy as np
import boto3
client = boto3.client('s3')
resource = boto3.resource('s3')
my_bucket = resource.Bucket('daanmatchdatafiles')<jupyter_output><empty_output><jupyter_text>File Name Here  TOC:<jupyter_code># Lists out the column names in TOC format
def toc_maker(dataset):
    counter = 1
    for column in dataset.columns:
        print("* ["+column+"](4."+str(counter)+")")
        counter +=1
#toc_maker(Final_Data_ngodarpan)<jupyter_output><empty_output><jupyter_text>**About this dataset** Data provided by: Unknown Source:  Type: xlsx Last Modified:  Size:<jupyter_code>#Read into file<jupyter_output><empty_output><jupyter_text>**What's in this dataset?**<jupyter_code>df = 
print("Shape:", df.shape)
print("Rows:", df.shape[0])
print("Columns:", df.shape[1])
print("Each row is a .")<jupyter_output><empty_output><jupyter_text>**Codebook**<jupyter_code>dataset_columns = [column for column in df.columns]
dataset_desc = []
dataset_dtypes = [dtype for dtype in dataset.dtypes]

data = {"Column Name": dataset_columns, "Description": dataset_desc, "Type": dataset_dtypes}
codebook = pd.DataFrame(data)
codebook<jupyter_output><empty_output><jupyter_text>**Missing Values**<jupyter_code>df.isnull().sum()<jupyter_output><empty_output><jupyter_text>**Summary Statistics**<jupyter_code>df.describe()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.14193130265716136, 'fraction_numerical': 0.0064808813998703824, 'mean_word_length': 8.414634146341463, 'pattern_counts': {'":': 3, '<': 30, '<?xml version=': 0, '>': 30, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '7', 'dir': 'jupyter-structured-clean-dedup', 'id': '386323', 'license': 'MIT', 'n_tokens_mistral': 548, 'n_tokens_neox': 535, 'n_words': 108, 'path': '[DIR] from Shekhar/Final_Data_givingtuesdayindia.org/.ipynb_checkpoints/Template-checkpoint.ipynb', 'repo_name': 'GeLi2001/Codebook', 'size': '1545'}