<jupyter_start><jupyter_text>Datastores---**Text File Format**- CSV (Comma Separated Values)- JSON (JavaScript Object Notation)**Binary File Format**- Sqlite (Relational Database)- HDF5 (Object Store)  CSV - Comma Separated Calues---    Station, "Maximum Temp.", "Minimum Temp.", Rainfall    Dipayal, 34.5, 24.6, 1.6    Dadeldhura, 25.2, 17.3, 30.1    Dhangadi, 34.0, 24.2, 99.8    Birendranagar, 32.0, 23.0, 9.3*data.csv* json - Javascript Object Notation    {        "Dipayal": {            "maximum": 34.5,            "minimim": 24.6,            "rainfall": 1.6        },        "Dadeldhura": {            "maximum": 34.5,            "minimim": 24.6,            "rainfall": 1.6        },    }*data.json*  File Handling---<jupyter_code>csvfile = open('../Data/Pokemon.csv', 'r')
csvfile
csvfile.read()
csvfile.read()
csvfile.seek(0)
csvfile.read(500)
csvfile.read(500)
csvfile.seek(0)
csvfile.readlines()
csvfile.seek(0)
csvfile.readline()
csvfile.readline()
csvfile.readline()
line = csvfile.readline()
line
line.strip().split(',')
csvfile.close()<jupyter_output><empty_output><jupyter_text>CSV File---<jupyter_code>import csv
with open('../Data/Pokemon.csv', 'r') as csvfile:
    reader = csv.reader(csvfile)
    for line in reader:
        print(line[1], line[3])
fieldnames = ['#', 'Name', 'Type 1', 'Type 2', 'Total', 'HP', 'Attack', 'Defense',
              'Sp. Atk', 'Sp. Def', 'Speed', 'Generation', 'Legendary']
with open('../Data/Pokemon.csv', 'r') as csvfile:
    reader = csv.DictReader(csvfile, fieldnames=fieldnames)
    # next(reader)
    for line in reader:
        print(line['Name'], line['Type 2'])<jupyter_output>Name Type 2
Bulbasaur Poison
Ivysaur Poison
Venusaur Poison
VenusaurMega Venusaur Poison
Charmander 
Charmeleon 
Charizard Flying
CharizardMega Charizard X Dragon
CharizardMega Charizard Y Flying
Squirtle 
Wartortle 
Blastoise 
BlastoiseMega Blastoise 
Caterpie 
Metapod 
Butterfree Flying
Weedle Poison
Kakuna Poison
Beedrill Poison
BeedrillMega Beedrill Poison
Pidgey Flying
Pidgeotto Flying
Pidgeot Flying
PidgeotMega Pidgeot Flying
Rattata 
Raticate 
Spearow Flying
Fearow Flying
Ekans 
Arbok 
Pikachu 
Raichu 
Sandshrew 
Sandslash 
Nidoran♀ 
Nidorina 
Nidoqueen Ground
Nidoran♂ 
Nidorino 
Nidoking Ground
Clefairy 
Clefable 
Vulpix 
Ninetales 
Jigglypuff Fairy
Wigglytuff Fairy
Zubat Flying
Golbat Flying
Oddish Poison
Gloom Poison
Vileplume Poison
Paras Grass
Parasect Grass
Venonat Poison
Venomoth Poison
Diglett 
Dugtrio 
Meowth 
Persian 
Psyduck 
Golduck 
Mankey 
Primeape 
Growlithe 
Arcanine 
Poliwag 
Poliwhirl 
Poliwrath Fighting
Abra 
Kadabra 
Alakazam 
AlakazamMega Alakazam 
Machop 
M[...]<jupyter_text>Working with JSON---<jupyter_code>import json
_jsn = {'Name': 'Wartortle'}
_jsn
json.dumps(_jsn)
json.loads('{"Name": "Wartortle"}')
data = {
    'pokemons': []
}
with open('../Data/Pokemon.csv', 'r') as csvfile:
    reader = csv.DictReader(csvfile, fieldnames=fieldnames)
    next(reader)
    for line in reader:
        data['pokemons'].append(line)

with open('../Data/Pokemon.json', 'w') as jsonfile:
    json.dump(data, jsonfile)
_jsonfile = open('../Data/Pokemon.json', 'r')
_json = json.load(_jsonfile)
_json<jupyter_output><empty_output><jupyter_text>Sqlite---<jupyter_code>import sqlite3
conn = sqlite3.connect('../Data/db.sqlite3')
conn
cursor = conn.cursor()
cursor
sql = """CREATE TABLE pokemons(id INTEGER primary key, attack INTEGER, defense INTEGER,
                               generation INTEGER, hp INTEGER, legendary BOOLEAN, 
                               name TEXT, sp_atk INTEGER, sp_def INTEGER, speed INTEGER,
                               total INTEGER, type1 TEXT, type2 TEXT)"""
cursor.execute(sql)
conn.commit()
sql = """INSERT INTO pokemons(id, attack, defense, generation, hp, legendary, name, sp_atk,
                              sp_def, speed, total, type1, type2) 
                              VALUES(1, 49, 49, 1, 45, 0, 'Bulbasaur', 65, 65, 45, 318,
                                     'Grass', 'Poison')"""
cursor.execute(sql)
conn.commit()
_jsonfile.close()
sql = """INSERT INTO pokemons(id, attack, defense, generation, hp, legendary, name, sp_atk,
                              sp_def, speed, total, type1, type2) 
                              VALUES(?,?,?,?,?,?,?,?,?,?,?,?,?)"""
counter = 1
with open('../Data/Pokemon.json', 'r') as _jsonfile:
    _json = json.load(_jsonfile)
    for each in _json['pokemons']:
        _leg = 0 if each['Legendary'] == 'False' else 1
        values = (counter, int(each['Attack']), int(each['Defense']),
                  int(each['Generation']), int(each['HP']), _leg, 
                  each['Name'], int(each['Sp. Atk']), int(each['Sp. Def']),
                  int(each['Speed']), int(each['Total']), each['Type 1'], each['Type 2'])
        print(values)
        cursor.execute(sql, values)
        counter += 1
conn.commit()
cursor.execute("SELECT * FROM pokemons")
cursor.fetchone()
cursor.fetchone()
cursor.fetchall()
cursor.execute("SELECT * FROM pokemons WHERE type2=?", ('Poison',))
cursor.fetchall()
cursor.execute("SELECT * FROM pokemons WHERE type2=? AND type1=?", ('Poison', 'Bug',))
cursor.fetchall()
cursor.execute("SELECT id, name FROM pokemons WHERE type2=? AND type1=?", ('Poison', 'Bug',))
cursor.fetchall()<jupyter_output><empty_output><jupyter_text>**H/W** *practice more from [https://www.w3schools.com/SQL/sql_create_table.asp](https://www.w3schools.com/SQL/sql_create_table.asp)*  HDF5---<jupyter_code>import h5py
hf = h5py.File('../Data/datasets.h5', 'w')
g1 = hf.create_group('ktm')
g1
hf
g1.create_group('substation1')
sg2 = g1.create_group('substation2')
sg2
dt = sg2.create_dataset('statellite_image', (100,), dtype='i')
dt
dt.ndim
dt.shape
dt.value
import numpy as np
dt2 = sg2.create_dataset('random', data=np.arange(0, 12).reshape(2, 6))
dt2
dt2.ndim
dt2.value<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.1465620201330831, 'fraction_numerical': 0.022521753966899847, 'mean_word_length': 3.5057647963105305, 'pattern_counts': {'":': 9, '<': 20, '<?xml version=': 0, '>': 20, 'https://': 2, 'lorem ipsum': 0, 'www.': 2, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '5', 'dir': 'jupyter-structured-clean-dedup', 'id': '516879', 'license': 'CC-BY-4.0', 'n_tokens_mistral': 2296, 'n_tokens_neox': 2138, 'n_words': 494, 'path': 'Sessions/Session-4.ipynb', 'repo_name': 'rprasai/datascience', 'size': '5861'}