<jupyter_start><jupyter_text>Exploring tabular dataWhen working with data in tables, the ability of quickly getting an overview about the data is key.<jupyter_code>import pandas as pd<jupyter_output><empty_output><jupyter_text>Loading CSV files from diskTo ensure compatility beween different software for processing tabular data the [CSV file format](https://en.wikipedia.org/wiki/Comma-separated_values) is commonly used. We can open those files using [pandas.read_csv](https://pandas.pydata.org/docs/reference/api/pandas.read_csv.html).<jupyter_code>data = pd.read_csv('../../data/Results.csv', index_col=0, delimiter=';')
data<jupyter_output><empty_output><jupyter_text>Viewing the dataViewing data can be tricky, especially when working with large tables.<jupyter_code>data.head(10) # top 10 rows
data.tail(10) # bottom 10 rows<jupyter_output><empty_output><jupyter_text>Overview descriptive statisticsTo get a glimpse of the range of values which exist in the given table, we can ask the DateFrame to _describe_ itself using [`DataFrame.describe()`](https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.describe.html). It will display count, mean, standard deviation and other descriptive statistics for each column in our table.<jupyter_code>data.describe()<jupyter_output><empty_output><jupyter_text>Sorting in tablesIn many cases, we are interested in table rows that contain the maximum value, e.g. in the `area` column we can find the largest object:<jupyter_code>data.sort_values(by = "Area", ascending=False)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.0955128205128205, 'fraction_numerical': 0.0057692307692307696, 'mean_word_length': 6.844221105527638, 'pattern_counts': {'":': 0, '<': 21, '<?xml version=': 0, '>': 21, 'https://': 3, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '5', 'dir': 'jupyter-structured-clean-dedup', 'id': '258448', 'license': 'CC-BY-4.0', 'n_tokens_mistral': 490, 'n_tokens_neox': 465, 'n_words': 144, 'path': 'docs/40_tabular_data_wrangling/explore_dataset.ipynb', 'repo_name': 'rayanirban/BioImageAnalysisNotebooks', 'size': '1560'}