<jupyter_start><jupyter_text>Transcribed from FOIA Doc ID: 6689695https://archive.org/details/comp3321  Module: Pandas (U) This modules covers the Pandas package in Python, for working with dataframes.**Note:** This module does not include portion markings in the source PDF after this point and did not include an overall classification. It is from the same FOIA source as the other modules included in this collection. I will call out my changes if I make any.  Pandas Resource & Examples (Note: this was modified from the Pandamonium notebook by _redacted_ on nbGallery.)This resource should help people who are new to Pandas and need to explore capabilities or learn the syntax. We'll provide a few examples for each command we introduce. It's important to mention that these are not all the commands available! If you prefer video tutorials, here's a Safari series => [Data Analysis with Python and Pandas](https://learning.oreilly.com/videos/data-analysis-with/100000006A0408)Also note that Pandas documentation is available [here](https://pandas.pydata.org/pandas-docs/stable/). First we'll import and install all necessary modules<jupyter_code>import ipydeps 
modules = ['pandas', 'xlrd', 'bokeh', 'numpy', 
           'requests', 'openpyxl'] 
ipydeps.pip(modules)<jupyter_output><empty_output><jupyter_text>`pd` is the standard abbreviation for pandas, and `np` for numpy. It's a standard convention if you're writing a program which uses pandas or numpy to import them as `pd` or `np`. You don't have to do this, but if you don't other programmers reviewing your code might get confused.<jupyter_code>import math
import pandas as pd
# import numpy as np

# This is only included to give us a sample dataframe to work with
from bokeh.sampledata.autompg import autompg as df<jupyter_output><empty_output><jupyter_text>Creating a DataFrame The very basics of creating your own DataFrame. You might not be creating them from scratch often but you probably will create empty DataFrames like you'll see a few times further down in the guide.<jupyter_code># Create Empty DataFrame Object
df1 = pd.DataFrame()
# This is the very basic method, create empty DataFrame but specify 4 columns and their names 
# You can also specify datatypes, index, and many other advanced things here 
df1 = pd.DataFrame(columns=('Column1', 'Column2', 'Column3', 'Column4'))
df1
# Create testing DataFrames (a, b, c), always useful for evaluating merge/join/concat/append operations. 
a = pd.DataFrame([[1,2,3],[3,4,5]], columns=list('ABC'))
b = pd.DataFrame([[5,2,3],[7,4,5]], columns=list('BDE'))
c = pd.DataFrame([[11,12,13],[17,14,15]], columns=list('XYZ'))
a 
b 
c<jupyter_output><empty_output><jupyter_text>Reading from and Writing To Files Super easy in Pandas   CSVLet's write our autompg dataframe out to csv first so we have one to work this. Note: if you leave the `index` parameter set to `True`, you'll get an extra column called "Unnamed: 0" in your CSV.<jupyter_code>df.to_csv("autompg.csv", index=False)<jupyter_output><empty_output><jupyter_text>Now reading it in is super easy.<jupyter_code>df1 = pd.read_csv("autompg.csv")
df1.head()<jupyter_output><empty_output><jupyter_text>If the file contains special encoding (if it's not English for example) you can look up the encoding you need for your language or text and include that when you read the file.<jupyter_code>df1 = pd.read_csv('autompg.csv', encoding='utf-8-sig')<jupyter_output><empty_output><jupyter_text>You can also specify which columns you'd like to read in (if you'd prefer a subset).<jupyter_code>df2 = pd.read_csv('autompg.csv', usecols=['name', 'mpg']) 
df2.head()<jupyter_output><empty_output><jupyter_text>If your file is not a csv, and uses alternative seperators, you can specify that when you read it in. Your file does not need to have a ".csv" extension to be read by this function, but should be a text file that represents data._For Example,_ if you have a ".tsv", or tab-delimited file you can specify that to pandas when reading the file in.<jupyter_code>df1.to_csv("autompg.tsv", index=False, sep='\t') 
df1 = pd.read_csv('autompg.tsv', sep='\t') 
df1.head()<jupyter_output><empty_output><jupyter_text>Chunking on Large CSVsOften times, when working with very large CSVs you will run into errors. There are a few methods to work around these errors without installing more memory.If you don't have enough memory to directly open an entire CSV, as when they start going above 500MB-1GB+, you can _sometimes_ alleviate the problem by chunking the in-read (opening them in smaller pieces). **Note:** your numeric index will be reset each time.<jupyter_code># first we'll create a large DataFrame for an example 
large_df = pd.DataFrame() 
for i in range(100): 
    # ignore_index prevents the index from being reset with each DataFrame added 
    large_df = large_df.append(df1, ignore_index=True) 
large_df.to_csv("large_file.csv", index=False) 
# chunk becomes the temporary dataframe containing the data of that chunk size 
for chunk in pd.read_csv('large_file.csv', chunksize=1000): 
    print(chunk.head(1))<jupyter_output><empty_output><jupyter_text>Another chunking variation If you still need to load a very large CSV into memory for deduplication or other processing reasons, there are ways to do it. This method uses a temporary DataFrame for appending, which gets dumped into a master DataFrame after 200 chunks have been processed. Clearing the temporary DataFrame every 200 chunks reduces memory overhead and improves speed during the append process. You can improve efficiency by adjusting chunksize and the interval that it dumps data into the master DataFrame. There may be more efficient ways to do this, but this is effective. At the end of the cell, we have a DataFrame `df1` which has all the data that we couldn't read all at once. **Notes:** We use ignore_index in order to have unique index values, since append will automatically preserve index values.<jupyter_code>df1 = pd.DataFrame()
df2 = pd.DataFrame()

for counter, chunk in enumerate(pd.read_csv('large_file.csv', chunksize=1000)):
    # Every 200 chunks, append df2 to df1, clear memory, start an empty df2
    if (counter % 200) == 0:
        df2 = df2.append(chunk, ignore_index=True)
        df1 = df1.append(df2, ignore_index=True)
        df2 = pd.DataFrame()
    else:
        df2 = df2.append(chunk, ignore_index=True)

# Anything leftover gets appended to master dataframe (df1)
df1 = df1.append(df2, ignore_index=True)

# remove the temporary DataFrame

del df2

print(f"There are {len(df1)} rows in this DataFrame.")

df1.head()<jupyter_output><empty_output><jupyter_text>Excel Use `Excelwriter` to write a `DataFrame` or multiple `DataFrames` to an Excel workbook.<jupyter_code>df2 = pd.DataFrame([
    {'Name': 'Po', 'Occupation': 'Dragon Warrior'},
    {'Name': 'Shifu', 'Occupation': 'Sensei'}
])
# this just initializes the workbook
writer = pd.ExcelWriter("test_workbook.xlsx")
# write as many DataFrames as sheets as you want
df.to_excel(writer, "Sheet1")
df2.to_excel(writer, "Sheet2")
writer.save() # ,save() finishes the operation and saves the workbook<jupyter_output><empty_output><jupyter_text>When reading from an Excel workbook, Pandas assumes you want just the first sheet of the workbook by default.<jupyter_code>df1 = pd.read_excel('test_workbook.xlsx')
df1.head()<jupyter_output><empty_output><jupyter_text>To read a specific sheet, simply include the name of the sheet in the read command.<jupyter_code>df1 = pd.read_excel('test_workbook.xlsx', sheet_name='Sheet2') 
df1.head()<jupyter_output><empty_output><jupyter_text>Loading from JSON/APIThis is just a very simple example to show that it's very easy for JSON or API payloads to be converted to a DataFrame, as long as the payload has a structured format that can be interpreted. Pandas can write a DataFrame to a JSON file, and also read in from a JSON file.<jupyter_code>df.to_json("json_file.json")
from_json = pd.read_json("json_file.json")
from_json.head()<jupyter_output><empty_output><jupyter_text>The same can be done for JSON objects instead of files.<jupyter_code>json_object = df.to_json() # don't specify a file and it will create a JSON object
from_json = pd.read_json(json_object)
from_json.head()<jupyter_output><empty_output><jupyter_text>DataFrame Information Summaries Now that your data is imported, we can get down to business. To retrieve basic information about your DataFrame, like the shape (column and row numbers), index values (row identifiers), DataFrame info (attributes of the object), and the count (number of values in the columns).<jupyter_code>df.shape
df.index
df.info()
df.count()<jupyter_output><empty_output><jupyter_text>Describe DataFrame Summary Statistics - `DataFrame.describe()` will try to process numeric columns by running: (count, mean, standard deviation (std), min, 25%, 50%, 75%, max) output will be that summary.<jupyter_code>df.describe()<jupyter_output><empty_output><jupyter_text>Checking Head and Foot of DataFrame **Note:** You can use this on most operations (especially in this guide) to get a small preview of the output instead of the entire DataFrame.<jupyter_code># Show first 5 rows of DataFrame 
df.head()
# Specify the number of rows to preview 
df.head(10)
# Show Last 5 rows of DataFrame 
df.tail()
# Or Specify 
df.tail(10)<jupyter_output><empty_output><jupyter_text>Checking DataTypes It's important to know how your DataFrame will treat the data contained in specific columns, and how it will read in the columns. Pandas will attempt to automatically parse numbers as `int` or `float`, and can be asked to parse dates as datetime objects. Understanding where it succeeded and where an explicit parse statement will be needed is important, the dataframe can provide this information. **Note:** Pandas automatically uses numpy objects.<jupyter_code># View column names and their associated datatype 
df.dtypes
# Select columns where the datatype is float64 using numpy (a decimal number) 
df.select_dtypes([np.float64])
# Select columns where the datatype is a numpy object (like a string) 
df.select_dtypes([np.object])
# Change the data type of a column 
df2 = df.copy()
df2['mpg'] = df2['mpg'].astype(str)
df2['mpg'].unique()
df2.dtypes<jupyter_output><empty_output><jupyter_text>Modifying DataFrames Modifications only work on assignment or when using `inplace=True`, which instructs the DataFrame to make the change without reassignment.See examples below.  Change by assignment<jupyter_code>df2 = df.drop('cyl',axis=1)
df2.head()<jupyter_output><empty_output><jupyter_text>Change in place<jupyter_code>df2.drop('hp',axis=1, inplace=True) # inplace 
df2.head()<jupyter_output><empty_output><jupyter_text>View and Rename Columns Check all column names or Rename specific columns<jupyter_code># Check Column Names 
df.columns
# Store column names as a list 
x = list(df.columns)<jupyter_output><empty_output><jupyter_text>Batch renaming columns requires a dictionary of the old values mapped to the new ones.<jupyter_code>df2 = df.rename(columns={'mpg': 'miles_per_gallon', 
                         'cyl': 'cylinders'}) 
df2.head()<jupyter_output><empty_output><jupyter_text>Create New Columns Similar to a dictionary, if a column doesn't exist, this will automatically create it<jupyter_code># Will populate entire column with value specified 
df2 = df.copy()
df2['year'] = '2017' 
df2.head()<jupyter_output><empty_output><jupyter_text>Accessing Index and Columns Access a specific column by name or row by indexChange the column placeholders below to actually see working Columns Available in Practice DataFrame: (mpg, cyl, displ, hp, weight, accel, yr, origin, name)<jupyter_code># By Column 
df['name'].head()
# Alternatively and equivalent to above, this won't work if there are spaces in the column name 
df.name.head()
# By Numeric Index, below is specifying 2nd and 3rd rows of vaLues 
df.iloc[2:4]
# 8y Index + Column 
df.loc[[1], ['name']]<jupyter_output><empty_output><jupyter_text>Remove Duplicates Important operation for reducing a DataFrame! Change the column placeholders below to actually see working Columns Available in Practice DataFrame: (mpg, cyl, displ, hp, weight, accel, yr, origin, name)<jupyter_code>len(df)
#first let's create some duplicates 
df2 = df.append(df, ignore_index=True) 
print("There are {} rows in the DataFrame.".format(len(df2))) 
# Remove any rows which contain dupiicates of another row 
df2.drop_duplicates(inplace=True) 
print ("There are now {} rows in the DataFrame.".format (len(df2)))
# or specify columns to reduce the number of cells in a row that must match to be dropped 
df2 = df2.drop_duplicates(subset=['mpg']) 

print("There are now {} rows in the DataFrame.".format(len(df2)))<jupyter_output><empty_output><jupyter_text>Filtering on Columns Filter a DataFrame based on specific column & value parameters. In the example below, we are creating a new DataFrame (df2) from our filter specifications against the sample DataFrame (df).<jupyter_code># Created new dataframe where 'cyl' vaLue == 6 
df2 = df.loc[df['cyl'] == 6] 
df2.head()
# use reset_index to re-number the index values 
df2 = df.loc[df['cyl'] == 6].reset_index(drop=True) 
df2.head()
df[df['name'] == '<NAME>']
# not that we don't need .loc for these operations 
df2 = df[df['mpg'] >= 16].reset_index(drop=True) 
df2.head()<jupyter_output><empty_output><jupyter_text>Fill or Drop the NaN or null Values Repair Empty Values or 'NaN' across DataFrame or Columns Change the column placeholders below to actually see working Columns Available in Practice DataFrame: (mpg, cyl, displ, hp, weight, accel, yr, origin, name) **Note:** `df.dropna` & `df.fillna` are modifications and will modify the sample DataFrame. Remove "inplace=True" from entries to prevent modification<jupyter_code>help(df.reindex)
# first we'll add some empty values 
df3 = pd.DataFrame([{'name': '<NAME>'}, {'mpg': 18.0}]) 
df2 = df.append(df3, ignore_index=True) 
# check for NaN values 
df2.loc[df2['mpg'].isnull()]
df2.loc[df2['name'].isnull()]
# True/False Output on if columns contain null values 
df2.isnull().any()
# Sum of all missing values by column 
df2.isnull().sum()
# Sum of all missing values across all columns 
df2.isnull().sum().sum()
# Locate all missing values 
df2.loc[df2.isnull().transpose().any()]
# Fill NaN values 
df2.fillna(0).tail()
df2.fillna(0).tail().info()
# Drop NaN values 
df2.dropna().tail()
# Alternatively target a column 
df2['cyl'].fillna(0).tail()
# Drop row only if all columns are NaN 
df2.dropna(how='all').tail()
# Drop if a specific number of columns are NaN 
df2.dropna(thresh=2).tail()
# Drop if specific columns are NaN 
df2.dropna(subset=['displ', 'hp']).tail()<jupyter_output><empty_output><jupyter_text>Simple Operations<jupyter_code># All Unique values in column 
df['mpg'].unique()
# Count of Unique Values in column 
df['cyl'].value_counts() 
# Count of all entries in column 
df['hp'].count()
# sum of all column values 
df['hp'].sum()
# mean of all column values 
df['cyl'].mean()
# median of all column values 
df['cyl'].median()
# min (Lowest numeric vaLue) of all column values 
df['cyl'].min()
# max (highest numeric vaLue) of all column values 
df['cyl'].max()
# Standard Deviation of all column values 
df['cyl'].std()<jupyter_output><empty_output><jupyter_text>Sorting Columns **Note:** These are just the very basic sort operations. There are many other advanced methods (multi-column sort, index sort, etc) that include multiple arguments.<jupyter_code># Sort dataframe by column values 
df.sort_values('mpg', ascending=False).head()
# Multi-Column Sort 
df.sort_values(['mpg', 'displ']).head()<jupyter_output><empty_output><jupyter_text>Merging DataFrames While many of these are similar, there are specifics and numerous arguments that can be used in conjunction that truly customize the type of DataFrame joining/merging/appending/concating you're trying to accomplish. **Note:** We've provided more sample DataFrames (a, b, c) to help illustrate the various methods. Join/Merge act similar to SQL joins. This Wikipedia entry might help but it can take some time to learn and get comfortable with using them all.<jupyter_code># example df's 
a = pd.DataFrame([[1,2,3], [3,4,5]], columns=list('ABC')) 
b = pd.DataFrame([[5,2,3],[7,4,5]], columns=list('BDE')) 
c = pd.DataFrame([[11,12,13],[17,14,15]], columns=list('XYZ')) 
print(a) 
print(b) 
print(c)<jupyter_output><empty_output><jupyter_text>Append DataFrames Merges 2+ DataFrames, Does not care if dissimilar or similar. Can also use a list of DataFrames.<jupyter_code>ab = a.append(b) 
ab<jupyter_output><empty_output><jupyter_text>Concatenate DataFrames Simlar to append, but handles large lists of dataframes well.<jupyter_code>abc = pd.concat([a,b,c]) 
abc<jupyter_output><empty_output><jupyter_text>Join DataFrames SQL-ish join operations (Inner/Outer etc), can specify join on index, similar columns may require specification<jupyter_code>joined_df = a.join(b,how='left',lsuffix="_a",rsuffix="_b") 
joined_df<jupyter_output><empty_output><jupyter_text>Merge DataFrames Merges 2+ DataFrames with overlapping columns, Very similar to join.<jupyter_code>merged_df = a.merge(b, left_on='B', right_on='D') 
merged_df<jupyter_output><empty_output><jupyter_text>Iterate across DataFramesIterating is only good for small dataframes, larger dataframes generally require apply/map and functions for efficiency.You will inevitably use these methods at one point or another, but it's important to remember that dataframes aren't like most of the objects you've used before now. If you think you have to iterate across your dataframe to get something done there is probably a better way to do what you're trying to do.  Iter Rows Access to values is done by indexrows[0] = Index rows[1] = values as pandas series (similar to a diet) rows[1][0] = First column value of row, can specify column rows[1]['Column']<jupyter_code>counter = 0 
for row in df.iterrows(): 
    counter += 1
    if counter > 15: 
        break 
    print(row[1].keys()[0]) 
    print(row[1]['name']) 
    print(row[0], row[1][0])<jupyter_output><empty_output><jupyter_text>IterTuples Faster and more efficient, access to values is slightly different from iterrows (Index is not nested). rowtuples[0] = Index rowtuples[1] = First column value rowtuples[2] = Second column value<jupyter_code>counter = 0 
for rowtuples in df.itertuples(): 
    counter += 1 
    if counter > 15: 
        break 
    print(rowtuples[0], rowtuples[1],rowtuples[2],rowtuples[3], rowtuples[4], rowtuples[5], rowtuples[6], rowtuples[7], rowtuples[8])<jupyter_output><empty_output><jupyter_text>Pivoting on DataFrame Create Excel style pivot tables based on specified criteria<jupyter_code># Basic Pivot 
df.pivot_table(index=['mpg','name'])
# Specify for a more complex pivot table 
df.pivot_table(values=['weight'], index=['cyl','name'], aggfunc=np.mean).head()<jupyter_output><empty_output><jupyter_text>Boolean Indexing Filter DataFrame on Multiple Columns and Values using Boolean index **Note:** The '&' in this example represents 'and' which might cause confusion. The explanation for this can also be a bit confusing, at least it caught the author off guard the first few times. The '&' will create a boolean array (of True/False which is used by the filtering operation to construct the output. When all 3 statements below return true for a row, pandas knows that we want that row in our output. The 'and' comparator functions differently than '&' and will throw a 'the truth value for the array is ambiguous' exception.<jupyter_code>df.loc[(df['cyl'] < 6) & 
       (df['mpg'] > 35)]
# the same thing can be done with .query for a more SQL-esque way to do it 
# just beware that you can run into issues with string formatting when using this method 
df.query("cyl < 6 & mpg > 35")<jupyter_output><empty_output><jupyter_text>Crosstab Viewing Contingency table (also known as a cross tabulation or crosstab) is a type of table in a matrix format that displays the (multivariate) frequency distribution of the variables<jupyter_code>pd.crosstab(df['cyl'],df['yr'],margins=True)<jupyter_output><empty_output><jupyter_text>Example using multiple options **Note:** This is an example using a combination of techniques seen above. We've also introduced a new method `.nlargest`<jupyter_code># Top Number of Column1 Unique Values based on the Mean of NumColumn Unique Values using .nlargest 
df.cyl.value_counts().nlargest(math.ceil(df.mpg.value_counts().mean()))<jupyter_output><empty_output><jupyter_text>Create a New Column with simple logic Useful technique for simple operations<jupyter_code># Using.astype(str) we can treat the float64 df['mpg'] column as a string and merge it with other strings 
df2 = df.copy()
df2['mpg_str'] = df2['name'] + ' Has MPG ' + df2['mpg'].astype(str)
df2.head()<jupyter_output><empty_output><jupyter_text>Functions on DataFrames The fastest and most effecient method of running calculations against an entire dataframe. This will become your new method of 'iterating' over the data and doing analytics. axis = 0 means function will be applied to each column axis = 1 means function will be applied to each row **Note:** This is a step into more advanced techniques. Map/Apply/Applymap are the most efficient Pandas method of iterating and running functions across a DataFrame.   Map Map applys a function to each element in a series, very much like iterating.<jupyter_code>def concon(x): 
    return 'Adding this String to all values: ' + str(x)

df['name'].map(concon).head()<jupyter_output><empty_output><jupyter_text>Apply Apply runs a function against the axis specified. We are creating hp_and_mpg based on results of adding.We are creating a New_Column based on the results of summing Column1 + Column2<jupyter_code>df2['hp_and_mpg'] = df2[['hp', 'mpg']].apply(sum,axis=1) 
df2.loc[:, ['hp', 'mpg', 'hp_and_mpg', 'name']].head()<jupyter_output><empty_output><jupyter_text>ApplyMap Runs a function against each element in a dataframe (each 'cell')<jupyter_code>df.applymap(concon).head()<jupyter_output><empty_output><jupyter_text>More Function Examples<jupyter_code>def num_missing(x): 
    return sum(x.isnull()) 

# Check how many missing values in each column 
df.apply(num_missing, axis=0)
# Check how many missing values in each row 
df.apply(num_missing, axis=1).head()<jupyter_output><empty_output><jupyter_text>Python 3 and Map **Note:** Similar to zip, map can return an object (instead of a value) depending on how it's configured. For both zip and map, you can use list() to get the values.<jupyter_code>def Example1(stuff):
    return stuff + ' THINGS'

# Try this without List, obverse the NewColumn values which are returned as objects 
df2 = df.copy()
df2['NewColumn'] = map(Example1, df2['name'])
df2.head() 
# Now try with a list, problem solved when using this syntax 
df2 = df.copy() 
df2['NewColumn'] = list(map(Example1, df2['name']))
df2.head()<jupyter_output><empty_output><jupyter_text>Advanced Multi-Column Functions **Note:** This is a technique to modify or create multiple columns based on a function that outputs multiple values in a tuple. We've written this to work directly with the sample DataFrame imported at the beginning of this resource guide. Example2 outputs a tuple of (x, y, z) which we unpack from map using * and then zip inline.<jupyter_code>def Example2(one, two, three):
    text = ' Text '
    x = ''.join([str(one), text, str(two), text, str(three)])
    y = sum([one, two, three]) 
    z = 'Poptarts' 
    return x, y, z 

df2 = df.copy() 

df2['StrColumn'], df2['SumColumn'], df2['PopColumn'] = zip(*map(
    Example2, df2['mpg'], df2['cyl'], df2['hp'])) 

df2.head()<jupyter_output><empty_output><jupyter_text>Conditionally Updating Values Use `.loc` to update values where a certain condition has been met. This is analogous to `SET ... WHERE ...` syntax in SQL.<jupyter_code>df2 = df.copy()
df2['efficiency'] = ""
# in SQL, "UPDATE <tablename> SET efficiency = 'poor' WHERE mpg < 10"
df2.loc[(df2.mpg < 10), 'efficiency'] = "poor"
df2.loc[(df2.mpg >= 10) & (df2.mpg < 30), 'efficiency'] = "medium"
df2.loc[(df2.mpg >= 30), 'efficiency'] = "high"
df2.tail()<jupyter_output><empty_output><jupyter_text>GroupBy and Aggregate Pandas makes it pretty simply to group your dataframe on a value or values, and then aggregate the other results. It's a little less flexible than SQL in some ways, but still pretty powerful. There's a lot you can do in Pandas with GroupBy objects, so definitely check the documentation.<jupyter_code># setting as_index to False will keep the grouped values as 
# regular columns values rather than indices 
grouped_df = df.groupby(by=['cyl'])

# use.agg to aggregate the values and run specified functions 
# note that Me can't create new columns here 
aggregated = grouped_df.agg({
    'mpg': np.mean,
    'displ' : np.mean,
    'hp' : np.mean,
    'yr': np.max,
    'accel': 'mean'
}) 
aggregated.head()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.10259688373951259, 'fraction_numerical': 0.013503795445465441, 'mean_word_length': 5.523586135001303, 'pattern_counts': {'":': 0, '<': 209, '<?xml version=': 0, '>': 212, 'https://': 3, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 1, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '50', 'dir': 'jupyter-structured-clean-dedup', 'id': '454806', 'license': 'Unlicense', 'n_tokens_mistral': 8010, 'n_tokens_neox': 7642, 'n_words': 2908, 'path': 'Module - Pandas.ipynb', 'repo_name': 'thundercats-py/python_training', 'size': '25040'}