<jupyter_start><jupyter_text>EXAM 2 ANSWERS  PROBLEM 1In order to prove that both files are in the same sequence, my idea is to retrieve the Locus column from both files and compare them. To do so, I opened the files as csv files and employed the DictReader function to read the files (as they are .tsv, it must be denoted \t as the delimiter). Following this, the csv.writer function allows me to write on a new csv file, wich I named Ger.csv for the Germplasm.tsv file. FIrstly, I wrote "Germplasm" (as a header) file and closed the file. After that, I started a loop for item in agicode (where agicode is the file after the DictReader object transformation) in order to extract every row from the column "Locus", which after extracting, I wrote that information on the Germ.csv filw below the header.Did the same thing for LousGene.tsv, only changing the header.After both new files are created, in order to compare them is necessary to list the contents of the file, so I could extract them one by one. It was alwo necessary to enable a second loop for item to change the data type in a format readable and callable by *menuwriter.writerow([a[i], b[i]])*The outcome (both "Locus" AGI codes set as a two column file) is stored on the file Juntos.csv and printed to show the perfect correlation in the order of the sequences.<jupyter_code>import csv 
import io



with open("Germplasm.tsv") as csvfile:
        agicode = csv.DictReader(csvfile, delimiter='\t', quotechar='"')
        with open('Germ.csv', 'a', newline='') as csvfile:
            menuwriter = csv.writer(csvfile, delimiter="\n", quotechar='"')
            menuwriter.writerow(["Germplasm"])
        csvfile.close
        for item in agicode:
           
           with open('Germ.csv', 'a', newline='') as csvfile:
                menuwriter = csv.writer(csvfile, delimiter="\n", quotechar='"')
               
                menuwriter.writerow([item["Locus"]])
           csvfile.close

with open("LocusGene.tsv") as csvfile:
        agicode = csv.DictReader(csvfile, delimiter='\t', quotechar='"')
        with open('Locus.csv', 'a', newline='') as csvfile:
                menuwriter = csv.writer(csvfile, delimiter="\n", quotechar='"')
                menuwriter.writerow(["Locus"])
        csvfile.close
        for item in agicode:
           
           with open('Locus.csv', 'a', newline='') as csvfile:
                menuwriter = csv.writer(csvfile, delimiter="\n", quotechar='"')
                menuwriter.writerow([item["Locus"]])
           csvfile.close      
        
i=()
with open('Germ.csv', 'r', newline='') as csvfile: 
    Unoread = list(csv.DictReader(csvfile))
with open('Locus.csv', 'r', newline='') as csvfile: 
    Dosread = list(csv.DictReader(csvfile))
with open('Juntos.csv', 'a', newline='') as csvfile:
    menuwriter = csv.writer(csvfile)
    a =[item["Germplasm"] for item in Unoread]
    b =[item["Locus"] for item in Dosread]
    for i in range(0,len(a)):
        menuwriter.writerow([a[i], b[i]]) 
csvfile.close


csvfile = io.open("Juntos.csv")
for item in csvfile:
    print(csvfile.read())
csvfile.close()<jupyter_output><empty_output><jupyter_text>Problem 2To crate the database, pymsql will be used. As only two tables are needed, and the .tsv files that will be used as data have the "Locus" column equal (as proven on Problem 1) the linkage is already 1:1, so there is no nedd for a third table. Anyway, another table will be made as a possible linkage in case it is needed.First, the connection to MYSQL is stablished from a python enviroment (docker containing mysql was started previously). mysql is called and ordered to create a new database amed Exam_2  and to show the databases list in order to confirm the newly created Exam_2. As we are not working in SQL directly, the outcomes must be selected by a *cursor.fetchall()* function and printed. Finally, the connection is closed. To have a more comprehensive explanation of each part, the code has been fragmented into sections that will be explained in detail, but technically, onece we have stablished the connection we could be ordering SQL all commands in a single (longer) code.Finally, connection is closed.<jupyter_code>%sql drop database Exam_2
#Execute before any other code.
%load_ext sql
#%config SqlMagic.autocommit=False
%sql mysql+pymysql://root:root@127.0.0.1:3306/mysql
import pymysql.cursors


connection = pymysql.connect(host='localhost',
                             user='root',
                             password='<PASSWORD>',
                             db='germplasm',
                             charset='utf8mb4',  
                             cursorclass=pymysql.cursors.DictCursor)
try:
    with connection.cursor() as cursor:
        
        sql = "create database Exam_2"
        cursor.execute(sql)
        sql = "show databases"
        cursor.execute(sql)
        results = cursor.fetchall()
        print(results)

finally:
    print("")
    connection.close()<jupyter_output><empty_output><jupyter_text>The first table is created, wich will be te previously mentioned (and not required) linkage table. The order in the creation of the tables is usually motivated for the "link" we have stablished. In this particular case the link of this column is the Gen_id AUTO_INCREMENT column that gives a ID to every Locus. With this assignation, other tables with the ID or the Locus can be linked to this and so establish the kind of relationship required for a database.The table Gen_id will have two columns, the primary key and AUTO_INCREMENT Gene_id and the Locus (where the AGI codes are) as a varchar of no more than 20 characters.<jupyter_code>connection = pymysql.connect(host='localhost',
                             user='root',
                             password='<PASSWORD>',
                             db='Exam_2',
                             charset='utf8mb4',  
                             cursorclass=pymysql.cursors.DictCursor)
try:
    with connection.cursor() as cursor:
       
        sql = "create table Gene_id(Gene_id INTEGER NOT NULL AUTO_INCREMENT PRIMARY KEY, Locus VARCHAR(20) NOT NULL)"
        cursor.execute(sql)
        
        
        results = cursor.fetchall()
        print(results)

finally:
    print("")
    connection.close()<jupyter_output><empty_output><jupyter_text>Then, the two other tables are created following the same commands (but now and in the code piece avobe, the database selected when stablishing the connection must be Exam_2) they will have the same headers as in the .tsv files, and one more in the case of he LocusGene table, as it will be linked to the Gene_id table by the Gene_id column. It is extremelly important to remember to adjust the VARCHAR () number of characters on the column phenotype, as it might be very long. I set it to 2000 to avoid any problems.The databas is then finally created, with every column accordingly described (depending on the data stored) and set.<jupyter_code>connection = pymysql.connect(host='localhost',
                             user='root',
                             password='<PASSWORD>',
                             db='Exam_2',
                             charset='utf8mb4',  
                             cursorclass=pymysql.cursors.DictCursor)
try:
    with connection.cursor() as cursor:
        
        sql = "create table Germplasm(Locus VARCHAR(20) NOT NULL PRIMARY KEY, Germplasm VARCHAR(100) NOT NULL, Phenotype VARCHAR(2000) NOT NULL, Pubmed INTEGER NOT NULL)"
        cursor.execute(sql)
        sql = "create table LocusGene(Locus VARCHAR(20) NOT NULL PRIMARY KEY, Gene VARCHAR(20) NOT NULL, Protein_length INTEGER NOT NULL, Gene_id INTEGER NOT NULL)"
        cursor.execute(sql)
       
        
        results = cursor.fetchall()
        print(results)

finally:
    print("")
    connection.close()<jupyter_output><empty_output><jupyter_text>Problem 3To better understand the code, it is fragmented.  To fill the database I kept using pymysql, where mysql is called by python. the INSERT INTO function of mysql allows to introduce data into the databases, more preciselly into the tables. The code below refers to the filling of the Germplasm table, with the data of the Germplasm.tsv. In order to do so, the Germplasm.tsv file is opened and readed by a DictReader function and the output stored on variable _proj_. Starting a loop for row, we call for every row in the Germplasm.tsv file, and print it. Inside that same loop, we INSERT INTO (mysql function) the table Germplasm the _*row.values()_ (all row values from the Germplasm.tsv file stored in _proj_) by means of a .format. We are able to do this because we have used DictReader, which is capable of determin what is the header (keys) and what is the values of the file. So when I stablished the columns of the table where the data was going, I was cautious to make it match the keys of the file.tsv.  After this, we simply close the connection.<jupyter_code>connection = pymysql.connect(host='localhost',
                            user='root',
                            password='<PASSWORD>',
                            db='Exam_2',
                            charset='utf8mb4', 
                            cursorclass=pymysql.cursors.DictCursor)
connection.autocommit = False

try:
    import csv
    with connection.cursor() as cursor:
        with open('Germplasm.tsv') as csvfile:
            proj = csv.DictReader(csvfile, delimiter="\t", quotechar='"')
            for row in proj:
                    print(row)
                    sql = """INSERT INTO Germplasm (Locus, Germplasm, Phenotype, Pubmed) values
                     ('{}', '{}', '{}', {})""".format(*row.values())
                    cursor.execute(sql)
        connection.commit() 

    results = cursor.fetchall()
    print(results)

finally:
    print("")
    connection.close()<jupyter_output><empty_output><jupyter_text>_(This code below is just a "safe point" that drops the tables Germplasm and LocusGene to make sure no overwritting errors or doublewritting errors happen before testing any code)_<jupyter_code>connection = pymysql.connect(host='localhost',
                             user='root',
                             password='<PASSWORD>',
                             db='Exam_2',
                             charset='utf8mb4',
                             cursorclass=pymysql.cursors.DictCursor)                    
try:
    with connection.cursor() as cursor:
        
        sql ="drop table Germplasm"
        cursor.execute(sql)
        sql = "drop table LocusGene"
        cursor.execute(sql)
       
        
        results = cursor.fetchall()
        print(results)

finally:
    print("")
    connection.close()<jupyter_output><empty_output><jupyter_text>Now, I created the table Gene_id, and linked it to the Germplasm table. This was made taking from the Germplasm.tsv the same "Locus" data that was introduced into Germplasm. To achieve this, all that data opened with a DictReader is stored in a _genid_ variable and .fotmat-ed into the sql call command.<jupyter_code>connection = pymysql.connect(host='localhost',
                            user='root',
                            password='<PASSWORD>',
                            db='Exam_2',
                            charset='utf8mb4', 
                            cursorclass=pymysql.cursors.DictCursor)
connection.autocommit = False
try:
    with connection.cursor() as cursor:
        with open('Germplasm.tsv') as csvfile:
            ge = csv.DictReader(csvfile, delimiter="\t", quotechar='"')
            for row in ge:
                print(row)
                genid = row["Locus"]
                genid = genid.strip()
                print(genid)
                sql = """INSERT INTO Gene_id (Locus) values
                     ('{}')""".format(genid)
                cursor.execute(sql)
                results = cursor.fetchall()
                print(results)
            
            
    connection.commit()
  

finally:
    print("")
    connection.close()<jupyter_output><empty_output><jupyter_text>To properlly see if the table was done right, another call is made to sql to SELECT command  and  show the table.<jupyter_code>connection = pymysql.connect(host='localhost',
                             user='root',
                             password='<PASSWORD>',
                             db='Exam_2',
                             charset='utf8mb4',  
                             cursorclass=pymysql.cursors.DictCursor)
try:
    with connection.cursor() as cursor:
       
        sql ="SELECT * FROM Gene_id"
        cursor.execute(sql)
       
       
        
        results = cursor.fetchall()
        print(results)

finally:
    print("")
    connection.close()<jupyter_output><empty_output><jupyter_text>Last table to be made is the LocusGene. This one is linked to the Gene_id column by a more sophisticated method. First, all "Locus" sequences are opened by DictRead from the LocusGene.tsv file and stored on the variable _project_ (which is the "Locus" row data extracted from _loc_ variable). As all AGI codes are the same and in the same sequence, a SELECT query is made on the Gene_id table to extrac every Gene id number associated to every AGI code that has been extracted from the file .tsv (all in variable _project_). These Gene ids are printed and cursor.fetchall()-ed and afterwards inserted into the LocusGene table as the variable _projid_ in a .format() manner. This all is donde inside a for row loop.<jupyter_code>connection = pymysql.connect(host='localhost',
                            user='root',
                            password='<PASSWORD>',
                            db='Exam_2',
                            charset='utf8mb4', 
                            cursorclass=pymysql.cursors.DictCursor)
connection.autocommit = False
try:
    with connection.cursor() as cursor:
        with open('LocusGene.tsv') as csvfile:
            loc = csv.DictReader(csvfile, delimiter="\t", quotechar='"' )
            for row in loc:
                print(row)
                project = row["Locus"]
                project = project.strip()
                sql = "SELECT Gene_id FROM Gene_id WHERE Locus='{}'".format(project)
                cursor.execute(sql)
                results = cursor.fetchall()
                print(results)
                projid = results[0]["Gene_id"]
                sql = """INSERT INTO LocusGene (Locus, Gene, Protein_length, Gene_id) values
                ('{}', '{}',{},{})""".format(row["Locus"], row ["Gene"], row ["ProteinLength"], projid)

                cursor.execute(sql)
    connection.commit() 

    flipflop = cursor.fetchall()
    print(flipflop)

finally:
    print("")
    connection.close()<jupyter_output><empty_output><jupyter_text>In order to inspect the complete output of the process of filling th database, a last call is made to SQL, to show all three tables.<jupyter_code>connection = pymysql.connect(host='localhost',
                             user='root',
                             password='<PASSWORD>',
                             db='Exam_2',
                             charset='utf8mb4',  
                             cursorclass=pymysql.cursors.DictCursor)
try:
    with connection.cursor() as cursor:
       
        sql ="SELECT * FROM Gene_id"
        cursor.execute(sql)
        results = cursor.fetchall()
        print(results)
        sql ="SELECT * FROM Germplasm"
        cursor.execute(sql)
        results = cursor.fetchall()
        print(results)
        sql ="SELECT * FROM LocusGene"
        cursor.execute(sql)
        results = cursor.fetchall()
        print(results)
       
finally:
    print("")
    connection.close()<jupyter_output><empty_output><jupyter_text>Problem 4The first question is about making a JOIN-ed report. It was at this moment that i realized that the third table (Gen_id) was almost useless, as the 1:1 linkage between the Germplasm and LocusGene tables was more than enough to accomplish the database queries and the Problem 4s' questions.For this reason, I avoided to show the Gene_id table on this JOIN report.Connection with SQL is made, and a SELECT... query is made, followed by a WHERE condition that refers to the column Locus in both tables. This, apparently gave some problems, as the columns were named the same and when the joined table is created one of those columns changes its name to Locus_1.After that, I created a csv.file where to store the report as required in the excercise by means of the DictWriter function. The fieldnames are determined by the *results[0].keys()* data transformation, which simply makes the callable elements for the DictWriter object.After this, the information of the tables is totally written into the "Report.csv" file in a \t delimited way and by means of the function .writerow.<jupyter_code>rm "Report.csv" # Execute before any other code
%load_ext sql
#%config SqlMagic.autocommit=False
%sql mysql+pymysql://root:root@127.0.0.1:3306/mysql
import pymysql.cursors
connection = pymysql.connect(host='localhost',
                             user='root',
                             password='<PASSWORD>',
                             db='Exam_2',
                             charset='utf8mb4', 
                             cursorclass=pymysql.cursors.DictCursor)
try:
    with connection.cursor() as cursor:
        
        sql ="SELECT * FROM Germplasm, LocusGene WHERE \
      Germplasm.Locus = LocusGene.Locus;"
        cursor.execute(sql)
        results = cursor.fetchall()
        print(results[0].keys())
      
        import csv
        import io
        with open('Report.csv', 'a', newline='') as csvfile:
                dumbwriter = csv.writer(csvfile, delimiter=" ", quotechar='"')
                dumbwriter.writerow("New Report")
                reportwriter = csv.DictWriter(csvfile, fieldnames=results[0].keys(), delimiter= '\t')
                reportwriter.writeheader()
                for row in results:
                    reportwriter.writerows([row])
                    csvfile.close
        
finally:
    print("")
    connection.close()
    
csvfile = io.open("Report.csv")
print(csvfile.read())
csvfile.close()<jupyter_output><empty_output><jupyter_text>Almost every call to SQL made during the elaboration of this code was tried and perfectioned outside the pymsql enviroment, and by direct %sql calls to SQL. For comprobation purposes, most of those calls are still visible.<jupyter_code>%sql use Exam_2;
%sql SELECT * FROM Germplasm, Gene_id, LocusGene WHERE\
      Germplasm.Locus = LocusGene.Locus AND\
LocusGene.Gene_id = Gene_id.Gene_id;<jupyter_output><empty_output><jupyter_text>To accomplish the DRY rule, I defined the function _salsa()_ to load every new report into the Repor.csv file with the parameters decided. First, a "New report" message is print by means of _csv.writer_ before the headedr of the new report, with a delimiter " " (space) for the components. Next, employing _csv.DictWriter_ the contents of the report stored in the variable _results_ are written on the csv file respecting the headers and with a \t delimiter. This funcion was employed from now until the last question of Problem 4 to store the resuts in a single file Reports.csv.<jupyter_code>def salsa():
        import csv
        import io
        with open('Report.csv', 'a', newline='') as csvfile:
            dumbwriter = csv.writer(csvfile, delimiter=" ", quotechar='"')
            dumbwriter.writerow("New Report")
            reportwriter = csv.DictWriter(csvfile, fieldnames=results[0].keys(), delimiter= '\t')
            reportwriter.writeheader()
            for row in results:
                reportwriter.writerows([row])
                csvfile.close<jupyter_output><empty_output><jupyter_text>To siolate two single rows (the genes SKOR and MAA3), two independent queries were performed on the tables LocusGene and Germplasm searching for the names of the genes (that appear as a column on LocusGene) and the rows of both tables linked by the common column Locus. The command UNION mades sure that the output of the queries is only one, as it joins queries. Finally, the function _salsa()_ is executed to store the data as previously mentioned.<jupyter_code>connection = pymysql.connect(host='localhost',
                             user='root',
                             password='<PASSWORD>',
                             db='Exam_2',
                             charset='utf8mb4',  
                             cursorclass=pymysql.cursors.DictCursor)
try:
    with connection.cursor() as cursor:
       
        sql ="SELECT *  FROM LocusGene, Germplasm WHERE\
        LocusGene.Gene = 'SKOR' AND\
        LocusGene.Locus = Germplasm.Locus\
        UNION SELECT *  FROM LocusGene, Germplasm WHERE\
        LocusGene.Gene = 'MAA3' AND\
        LocusGene.Locus = Germplasm.Locus"
        cursor.execute(sql)
        results = cursor.fetchall()
        salsa()
        
finally:
    print("")
    connection.close()
    
csvfile = io.open("Report.csv")
print(csvfile.read())
csvfile.close()
       
     
        
%sql SELECT *  FROM LocusGene, Germplasm WHERE\
        LocusGene.Gene = 'SKOR' AND\
        LocusGene.Locus = Germplasm.Locus\
        UNION SELECT *  FROM LocusGene, Germplasm WHERE\
        LocusGene.Gene = 'MAA3' AND\
        LocusGene.Locus = Germplasm.Locus<jupyter_output><empty_output><jupyter_text>To query for a number of entries, the search command SELECT COUNT (* ) which refers to everything, counts rhe number of rows or entries indicated. Also, a LIKE command is set to stablish the text search and the % indicates that the characters to be searched are at the beginning of the row. In this very case, as the Locus column is equal en both Germplasm and LocusGene tables, the number of entries of that column is asked for and given as the "Number of entries". Correctly, the output is a total of 32, as there were 32 AGI codes. There are 4 for chromosome 1 and 2, 9 for chromosome 3, 8 for chromosome 4 and 7 for chromoosome 5. salsa() function is also employed. Remember all outputs are stored on the Report.csv file.<jupyter_code>connection = pymysql.connect(host='localhost',
                             user='root',
                             password='<PASSWORD>',
                             db='Exam_2',
                             charset='utf8mb4',  
                             cursorclass=pymysql.cursors.DictCursor)
try:
    with connection.cursor() as cursor:
       
        sql =""" SELECT COUNT(*) AS "Number of entries for 1" FROM Germplasm WHERE Locus LIKE 'AT1G%'\
                UNION SELECT COUNT(*) AS "Number of entries for 2" FROM Germplasm WHERE Locus LIKE 'AT2G%'\
                UNION SELECT COUNT(*) AS "Number of entries for 3" FROM Germplasm WHERE Locus LIKE 'AT3G%'\
                UNION SELECT COUNT(*) AS "Number of entries for 4" FROM Germplasm WHERE Locus LIKE 'AT4G%'\
                UNION SELECT COUNT(*) AS "Number of entries for 5" FROM Germplasm WHERE Locus LIKE 'AT5G%';"""
        cursor.execute(sql)
        results = cursor.fetchall()
        salsa()
        
finally:
    print("")
    connection.close()
    
csvfile = io.open("Report.csv")
print(csvfile.read())
csvfile.close()
       
     
        
%sql SELECT COUNT(*) AS "Number of entries for CHROMOSOME" FROM Germplasm WHERE Locus LIKE 'AT1G%'\
                UNION SELECT COUNT(*) FROM Germplasm WHERE Locus LIKE 'AT2G%'\
                UNION SELECT COUNT(*) FROM Germplasm WHERE Locus LIKE 'AT3G%'\
                UNION SELECT COUNT(*) FROM Germplasm WHERE Locus LIKE 'AT4G%'\
                UNION SELECT COUNT(*) FROM Germplasm WHERE Locus LIKE 'AT5G%';
%load_ext sql
#%config SqlMagic.autocommit=False
%sql mysql+pymysql://root:root@127.0.0.1:3306/mysql<jupyter_output><empty_output><jupyter_text>The final question asks for the average protein size, which is denoted on the column Protein_length in the table LocusGene. AVG is a command that can be set on any SELECT query on SQL, so simply asking for the specific column (Locus) and the LIKE UNION query commands to obtain the average for every chromoosome, inmediately gives us the answer as an average of all INTEGERS on that column (is important to define that column as a in INTEGER column). Finally, salsa() is executed and the connection is closed.<jupyter_code>connection = pymysql.connect(host='localhost',
                             user='root',
                             password='<PASSWORD>',
                             db='Exam_2',
                             charset='utf8mb4',  
                             cursorclass=pymysql.cursors.DictCursor)
try:
    with connection.cursor() as cursor:
        sql ="""SELECT AVG(LocusGene.Protein_length) FROM LocusGene WHERE LocusGene.Locus LIKE 'AT1G%'\
UNION SELECT AVG(LocusGene.Protein_length) FROM LocusGene WHERE LocusGene.Locus LIKE 'AT2G%'\
UNION SELECT AVG(LocusGene.Protein_length) FROM LocusGene WHERE LocusGene.Locus LIKE 'AT3G%'\
UNION SELECT AVG(LocusGene.Protein_length) FROM LocusGene WHERE LocusGene.Locus LIKE 'AT4G%'\
UNION SELECT AVG(LocusGene.Protein_length) FROM LocusGene WHERE LocusGene.Locus LIKE 'AT5G%';"""
        cursor.execute(sql)
        results = cursor.fetchall()
        salsa()
        
finally:
    print("")
    connection.close()
    
csvfile = io.open("Report.csv")
print(csvfile.read())
csvfile.close()
       
     
%sql SELECT AVG(LocusGene.Protein_length) FROM LocusGene WHERE LocusGene.Locus LIKE 'AT1G%'\
UNION SELECT AVG(LocusGene.Protein_length) FROM LocusGene WHERE LocusGene.Locus LIKE 'AT2G%'\
UNION SELECT AVG(LocusGene.Protein_length) FROM LocusGene WHERE LocusGene.Locus LIKE 'AT3G%'\
UNION SELECT AVG(LocusGene.Protein_length) FROM LocusGene WHERE LocusGene.Locus LIKE 'AT4G%'\
UNION SELECT AVG(LocusGene.Protein_length) FROM LocusGene WHERE LocusGene.Locus LIKE 'AT5G%';<jupyter_output><empty_output><jupyter_text>If I wanted to make a more 'readable' content for the report, instead of a csv file entirely made as a database, I would simply write the code (and change it on the required places) as:<jupyter_code>import csv
import io
with open('Report.csv', 'r', newline='') as csvfile:
            reportread = csv.DictReader(csvfile, delimiter= '\t' ,fieldnames=("Locus", "Germplasm", "Phenotype", "Pubmed", "LocusGene.Locus","Gene","Protein_length", "Gene_id"))
            print(results[0])
            for row in reportread:
                
                normaldict = dict(row)
                
                arch = "Locus: {}, Germplasm: {}, Phenotype description: {}, Pubmed id: {}, Locus: {}, Gene name: {}, Protein length: {}, Gene id number: {} ".format(*row.values())
                print(arch)
#"Name: {} with age {}".format(*row.values()))
#sentence = str(''.join(["Name: ", name, " \tAge: ", age, "\tDo we know this person? ", known]))<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.08818073898204482, 'fraction_numerical': 0.005787208784686155, 'mean_word_length': 2.568100595632032, 'pattern_counts': {'":': 0, '<': 82, '<?xml version=': 0, '>': 82, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 3, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '17', 'dir': 'jupyter-structured-clean-dedup', 'id': '635886', 'license': 'MIT', 'n_tokens_mistral': 7794, 'n_tokens_neox': 7358, 'n_words': 2717, 'path': 'Exam_2_Answers.ipynb', 'repo_name': 'Mariogver/Accelerated_Intro_WilkinsonExams', 'size': '26878'}