<jupyter_start><jupyter_text>Using `tables_io.TableDict` The class `tables_io.TableDict` is just an Ordered Dictionary of Tables.The Tables can be in any of the formats that `tables_io` supports, see more on that in the notebook below.Let's have a look<jupyter_code># Standard imports
import os
import numpy as np
import tables_io
from tables_io.testUtils import make_test_data<jupyter_output><empty_output><jupyter_text>Some test data.Ok, lets make some test data and have a look at it<jupyter_code>data = make_test_data()
data<jupyter_output><empty_output><jupyter_text>Building a table dictWe can using any Mapping (i.e., something that allows use to iterate over key-value pairs) to build a `TableDict`.So lets make a `TableDict`<jupyter_code>td = tables_io.TableDict(data)
td<jupyter_output><empty_output><jupyter_text>`TableDict` inherits from the `collections.OrderedDict` class, so it has the standard interface for python dictionaries<jupyter_code>td.keys()
td['data']
td['md']<jupyter_output><empty_output><jupyter_text>`TableDict` will not take non tables<jupyter_code>try:
    td['bad'] = 'a'
except TypeError as msg:
    print("Caught attempt to add non table to TableDict: %s" % msg)<jupyter_output><empty_output><jupyter_text>Supported Table types and converting between them`TableDict` supports several different types of tables.  These include:1. astropy Tables: `astropy.table.Table` objects2. Mapping of `str`, `numpy.array`3. pandas DataFrames: `pandas.DataFrame` objectsLet's convert to each of these<jupyter_code>td_ap = td.convert(tables_io.types.AP_TABLE)
td_ap
td_np = td.convert(tables_io.types.NUMPY_DICT)
td_np
td_pd = td.convert(tables_io.types.PD_DATAFRAME)
td_pd<jupyter_output><empty_output><jupyter_text>File IO with `TableDict`We can write tables into several different formats.  These include:1. fits:  Writing `astropy.table.Table` objects to FITS files (with the suffix 'fits')2. hf5: Writing `astropy.table.Table` objects to HDF5 files (with the suffix 'hf5')3. hfd5: Writing `numpy.array` objects to HDF5 files (with the suffix 'hdf5')4. h5: Writing `pandas.DataFrame` objects to HDF5 files (with the suffix 'h5')5. pq: Writing `pandas.DataFrame` objects to parquet files (with the suffix 'pq')Also, each table type has a 'native' format that we use as a default.  Setting the `fmt` to `None` in function calls will typically use the 'native' format.<jupyter_code>all_fmts = list(tables_io.types.FILE_FORMAT_SUFFIXS.keys()) + [None]
print(all_fmts)<jupyter_output><empty_output><jupyter_text>Ok let's write the data to different files<jupyter_code>for fmt in all_fmts:
    if fmt is None:
        basename = 'test_native'
    else:
        basename = 'test_out'
    print("Writing to %s using format %s" % (basename, fmt))
    try:
        os.unlink('%s.%s' % (basename, fmt))
    except:
        pass
    try:
        td.write(basename, fmt)
    except ImportError as msg:
        print("Skipping format %s because %s" % (fmt, msg))
! ls test_*<jupyter_output><empty_output><jupyter_text>Ok, now let's read things back<jupyter_code>td_r_fits = tables_io.TableDict.read("test_out.fits")
td_r_fits
td_r_hdf5 = tables_io.TableDict.read("test_out.hdf5")
td_r_hdf5
td_r_hf5 = tables_io.TableDict.read("test_out.hf5")
td_r_hf5
td_r_pq = tables_io.TableDict.read("test_out.pq", keys=list(td.keys()))
td_r_pq
td_r_h5 = tables_io.TableDict.read("test_out.h5")
td_r_h5
td_native = tables_io.TableDict.read("test_out.hf5")
td_native<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.10541392151246061, 'fraction_numerical': 0.007734173589229447, 'mean_word_length': 5.793774319066148, 'pattern_counts': {'":': 0, '<': 37, '<?xml version=': 0, '>': 37, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '9', 'dir': 'jupyter-structured-clean-dedup', 'id': '26027', 'license': 'MIT', 'n_tokens_mistral': 1252, 'n_tokens_neox': 1224, 'n_words': 335, 'path': 'nb/tableDict_example.ipynb', 'repo_name': 'LSSTDESC/tables_io', 'size': '3491'}