<jupyter_start><jupyter_text>Modify the report's structureIn this notebook we have a look at two use cases in which we modify the existing report structure: splitting up large reports and reordering the report sections. Both use cases are based on actual user inquiries. The datasets used in this notebook are obtained using the `kaggle` api. If you haven't done so already, you should set up the [api credentials](https://github.com/Kaggle/kaggle-apiapi-credentials). The autoreload instruction reloads modules automatically before code execution, which is helpful for the update below.<jupyter_code>%load_ext autoreload
%autoreload 2<jupyter_output><empty_output><jupyter_text>Make sure that we have the latest version of pandas-profiling.<jupyter_code>import sys

!{sys.executable} -m pip install -U pandas-profiling[notebook]
!jupyter nbextension enable --py widgetsnbextension<jupyter_output><empty_output><jupyter_text>You might want to restart the kernel now.<jupyter_code>from pathlib import Path

import kaggle

from pandas_profiling.utils.common import extract_zip

kaggle.api.authenticate()<jupyter_output><empty_output><jupyter_text>Reorder SectionsWe can leverage the same approach to reorder sections. First, we need to generate a profile report.<jupyter_code># We are using the Craigslist Carstrucks Data
vehicles_dataset = Path("craigslist-carstrucks-data/vehicles.csv")

# Download and extract (~228M)
if not vehicles_dataset.exists():
    kaggle.api.dataset_download_files(
        "austinreese/craigslist-carstrucks-data",
        path="craigslist-carstrucks-data",
        quiet=False,
    )

    extract_zip(
        "craigslist-carstrucks-data/craigslist-carstrucks-data.zip",
        "craigslist-carstrucks-data/",
    )
import pandas as pd

from pandas_profiling import ProfileReport

# For our demonstration, we only take a fraction of the dataset
df = pd.read_csv(vehicles_dataset, nrows=100)

# Generate the profile report
vehicles_report = ProfileReport(df)<jupyter_output><empty_output><jupyter_text>The structure of the report is stored in the `report` attribute. The report is essentially a tree object. We inspect the root of the report.<jupyter_code>print(repr(vehicles_report.report))<jupyter_output><empty_output><jupyter_text>We can see that the root object is of the type "Sequence". Sequence types have at least the attributes `name` and `items`.<jupyter_code>print(vehicles_report.report.content)<jupyter_output><empty_output><jupyter_text>In this example, we would like to pull up the samples section, so that the reordered sequence items are:- Overview- Samples- Missing- Variables- Interactions- Correlations<jupyter_code># Reorder the sections
vehicles_report.report.content["items"] = [
    vehicles_report.report.content["items"][i] for i in [0, 5, 1, 2, 3, 4]
]<jupyter_output><empty_output><jupyter_text>Finally, we can render the report and see that the changes have taken place:<jupyter_code>vehicles_report.to_notebook_iframe()<jupyter_output><empty_output><jupyter_text>Split Profile ReportsWhen profiling large datasets, a monolithic HTML file can become enormous. Using the report structure generated by `pandas-profiling`, we create a modular report. In this notebook we demonstrate how to split up a profile report in multiple different titles. We start with generating the report's structure in the usual way. The `minimal` mode is set to `True`. This step may take a few minutes.<jupyter_code># We are using the IEEE Fraud Detection transaction training data
ieee_dataset = Path("ieee-fraud-detection/train_transaction.csv")

# Download and extract (~118M)
if not ieee_dataset.exists():
    kaggle.api.competition_download_files(
        "ieee-fraud-detection", path="ieee-fraud-detection", quiet=False
    )

    extract_zip(
        "ieee-fraud-detection/ieee-fraud-detection.zip",
        "ieee-fraud-detection/",
    )
import pandas as pd

from pandas_profiling import ProfileReport

# Read the dataset
df = pd.read_csv(ieee_dataset)

# Generate the profile report
ieee_fraud_report = ProfileReport(df, minimal=True)
print(repr(ieee_fraud_report.report))
print(repr(ieee_fraud_report.report.content))
from copy import deepcopy

# Make a copy for the original report structure
original_report_structure = deepcopy(ieee_fraud_report.report)
# Loop over each section
for section in original_report_structure.content["items"]:
    # Only consider sections that contain items
    #     if len(section.content['items']) > 0:
    # Set the report structure
    ieee_fraud_report.report = deepcopy(original_report_structure)
    # Overwrite the section lists with the section we would like to print
    ieee_fraud_report.report.content["items"] = [section]
    # Output the report to HTML
    ieee_fraud_report.to_file(f"ieee_fraud_report_section_{section.name.lower()}.html")<jupyter_output><empty_output><jupyter_text>Paginate variablesWe can use the same approach to paginate variables:<jupyter_code>ieee_fraud_report.report = original_report_structure

# Number of variables per page
page_size = 25

# The Root node, which is a sequence of sections
print(repr(ieee_fraud_report.report.content["items"]))

# The variables
variable_section = ieee_fraud_report.report.content["items"][1]
variables = variable_section.content["items"]
variable_count = len(variables)
print(f"Number of variables: {variable_count}")

# Reset the report structure
ieee_fraud_report.report = deepcopy(original_report_structure)

# Only keep the variables section
ieee_fraud_report.report.content["items"] = [
    ieee_fraud_report.report.content["items"][1]
]

for page_num, variable_page in enumerate(
    [variables[i : i + page_size] for i in range(0, variable_count, page_size)]
):
    print(f"Write page {page_num}")
    # Set the report title
    ieee_fraud_report.title = (
        f"IEEE Fraud Detection Dataset, Variables, Page {page_num}"
    )

    # Overwrite the variables lists with the variables we would like to print
    ieee_fraud_report.report.content["items"][0].content["items"] = variable_page

    # Output the report to HTML
    ieee_fraud_report.to_file(f"ieee_fraud_report_variables_page_{page_num}.html")<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.08379076525670003, 'fraction_numerical': 0.003713270907329674, 'mean_word_length': 5.5625, 'pattern_counts': {'":': 0, '<': 41, '<?xml version=': 0, '>': 42, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 4, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '10', 'dir': 'jupyter-structured-clean-dedup', 'id': '272321', 'license': 'MIT', 'n_tokens_mistral': 1957, 'n_tokens_neox': 1865, 'n_words': 558, 'path': 'examples/tutorials/modify_report_structure.ipynb', 'repo_name': 'timgates42/pandas-profiling', 'size': '6194'}