<jupyter_start><jupyter_text>DatatableTon    💯 datatable exercises                                                             This is Set 2: Files and Formats (Exercises 11-20) of DatatableTon: 💯 datatable exercises        You can find all the exercises and solutions on GitHub **Prerequisites*** The `datatable` package should be upgraded to the latest version (or v1.0.0+).* Additional packages `numpy`, `pandas` and `pyarrow` should be installed.* Small example values along with the sample dataset [datatableton_sample.zip](https://github.com/vopani/datatableton/blob/main/data/datatableton_sample.zip) will be used for the exercises.<jupyter_code>!python3 -m pip install -U pip
!python3 -m pip install -U datatable
!python3 -m pip install numpy
!python3 -m pip install pandas
!python3 -m pip install pyarrow
!wget https://raw.githubusercontent.com/vopani/datatableton/main/data/datatableton_sample.zip
import datatable as dt
import numpy as np
import pandas as pd
import pyarrow as pa

dtable = dt.Frame(f1=range(10), f2=['Y', 'O', 'U', 'C', 'A', 'N', 'D', 'O', 'I', 'T'])
dframe = pd.DataFrame({'v1': range(11), 'v2': ['N', 'E', 'V', 'E', 'R', 'G', 'I', 'V', 'E', 'U', 'P']})
nparray = np.array([[0, 'C'], [1, 'O'], [2, 'D'], [3, 'E']])
patable = pa.Table.from_pandas(dframe)
dlist = [range(4), ['D', 'A', 'T', 'A']]
ddict = {'x': range(6), 'y': ['P', 'Y', 'T', 'H', 'O', 'N']}
dtup = [(0, 'L'), (1, 'E'), (2, 'A'), (3, 'R'), (4, 'N')]<jupyter_output><empty_output><jupyter_text>**Exercise 11: Write `dtable` to a csv named `data.csv`, to a compressed gz csv named `data.gz` and to a jay named `data.jay`**<jupyter_code>dtable.to_csv('data.csv')
dtable.to_csv('data.gz', compression='gzip')
dtable.to_jay('data.jay')<jupyter_output><empty_output><jupyter_text>**Exercise 12: Read `data.csv` and assign it to `data_csv`, read `data.gz` and assign it to `data_gz`, read `data.jay` and assign it to `data_jay`**<jupyter_code>data_csv = dt.fread('data.csv')
data_csv
data_gz = dt.fread('data.gz')
data_gz
data_jay = dt.fread('data.jay')
data_jay<jupyter_output><empty_output><jupyter_text>**Exercise 13: Read data from this URL: [https://raw.githubusercontent.com/vopani/datatableton/main/data/datatableton_sample.csv](https://raw.githubusercontent.com/vopani/datatableton/main/data/datatableton_sample.csv) and assign it to `data_url`**<jupyter_code>data_url = dt.fread(url='https://raw.githubusercontent.com/vopani/datatableton/main/data/datatableton_sample.csv')
data_url<jupyter_output><empty_output><jupyter_text>**Exercise 14: Read `users.csv` from `datatableton_sample.zip` and assign it to `data_zip`**<jupyter_code>data_zip = dt.fread('datatableton_sample.zip/users.csv')
data_zip<jupyter_output><empty_output><jupyter_text>**Exercise 15: Create a dt.Frame `data_pd` from the pd.DataFrame `dframe` and create a pd.DataFrame `pd_data` from the dt.Frame `data_pd`**<jupyter_code>data_pd = dt.Frame(dframe)
data_pd
pd_data = data_pd.to_pandas()
pd_data<jupyter_output><empty_output><jupyter_text>**Exercise 16: Create a dt.Frame `data_np` from the np.array `nparray` and create a np.array `np_data` from the dt.Frame `data_np`**<jupyter_code>data_np = dt.Frame(nparray)
data_np
np_data = data_np.to_numpy()
np_data<jupyter_output><empty_output><jupyter_text>**Exercise 17: Create a dt.Frame `data_ls` from the list `dlist` and create a list `ls_data` from the dt.Frame `data_ls`**<jupyter_code>data_ls = dt.Frame(dlist)
data_ls
ls_data = data_ls.to_list()
ls_data<jupyter_output><empty_output><jupyter_text>**Exercise 18: Create a dt.Frame `data_dc` from the dictionary `ddict` and create a dictionary `dc_data` from the dt.Frame `data_dc`**<jupyter_code>data_dc = dt.Frame(ddict)
data_dc
dc_data = data_dc.to_dict()
dc_data<jupyter_output><empty_output><jupyter_text>**Exercise 19: Create a dt.Frame `data_tp` from the tuples `dtup` and create tuples `tp_data` from the dt.Frame `data_tp`**<jupyter_code>data_tp = dt.Frame(dtup)
data_tp
tp_data = data_tp.to_tuples()
tp_data<jupyter_output><empty_output><jupyter_text>**Exercise 20: Create a dt.Frame `data_pa` from the pyarrow.Table `patable` and create a pyarrow.Table `pa_data` from the dt.Frame `data_pa`**<jupyter_code>data_pa = dt.Frame(patable)
data_pa
pa_data = data_pa.to_arrow()
pa_data<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.1553896406906206, 'fraction_numerical': 0.012132524498366775, 'mean_word_length': 6.027868852459016, 'pattern_counts': {'":': 0, '<': 45, '<?xml version=': 0, '>': 45, 'https://': 5, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 4, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '11', 'dir': 'jupyter-structured-clean-dedup', 'id': '281266', 'license': 'Apache-2.0', 'n_tokens_mistral': 1683, 'n_tokens_neox': 1640, 'n_words': 391, 'path': 'notebooks/02_files_and_formats_solutions.ipynb', 'repo_name': 'akhilmaddineni/datatableton', 'size': '4286'}