<jupyter_start><jupyter_text>Enterprise Deep Learning with TensorFlow: openSAP  SAP Innovation Center Network```Licensed under the Apache License, Version 2.0 (the "License");you may not use this file except in compliance with the License.You may obtain a copy of the License at    http://www.apache.org/licenses/LICENSE-2.0Unless required by applicable law or agreed to in writing, softwaredistributed under the License is distributed on an "AS IS" BASIS,WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.See the License for the specific language governing permissions andlimitations under the License.```  Visualizing Machine Learning DatasetsFacets allows for easy visualization. For using Facets, we first need to clone the git repository:> git clone https://github.com/PAIR-code/facets.gitTo use the visualization capabilities, we have to add an nbextension. Therefore, find the path to the facets-dist directory in the cloned git repo and execute the following line of code:> jupyter nbextension install facets-dist/ --userIn which case 'facets-dist' is the path to the respective folder. If the above command still does not show the visualizations on the notebook, copy the file called facets-jupyter.html in 'facets/facets-dist' folder your local anaoconda file path '[anaconda_path]/share/jupyter/nbextensions/'. This is a known issue https://github.com/PAIR-code/facets/issues/41You might need to restart jupyter after this and proceed with the vizualisation. For a more detailed installation guide and updates, have a look at:> https://github.com/PAIR-code/facets<jupyter_code># Add the facets overview python code to the python path
import sys
# FACETS_PATH is the full path to the python file in the clonde github repo of Facets.
# It should look similar to this: ".../facets/facets_overview/python"
# If you have cloned the facets repo to your current working directory, you can proceed.
# If you have chosen another location, just add it here.

FACETS_PATH = './facets/facets_overview/python'
sys.path.append(FACETS_PATH)
# Load UCI census train and test data into dataframes. 
# Pandas can directly read URL, download the files in the URLs below
# https://archive.ics.uci.edu/ml/machine-learning-databases/adult/adult.data"
# https://archive.ics.uci.edu/ml/machine-learning-databases/adult/adult.test"
import pandas as pd
features = ["Age", "Workclass", "fnlwgt", "Education", "Education-Num", "Marital Status",
            "Occupation", "Relationship", "Race", "Sex", "Capital Gain", "Capital Loss",
            "Hours per week", "Country", "Target"]
train_data = pd.read_csv(
    "./data/adult.data",
    names=features,
    sep=r'\s*,\s*',
    engine='python',
    na_values="?")
test_data = pd.read_csv(
    "./data/adult.test",
    names=features,
    sep=r'\s*,\s*',
    skiprows=[0],
    engine='python',
    na_values="?")
# Calculate the feature statistics proto from the datasets and stringify it for use in 
# facets overview
from generic_feature_statistics_generator import GenericFeatureStatisticsGenerator
import base64

gfsg = GenericFeatureStatisticsGenerator()
proto = gfsg.ProtoFromDataFrames([{'name': 'train', 'table': train_data},
                                  {'name': 'test', 'table': test_data}])
protostr = base64.b64encode(proto.SerializeToString()).decode("utf-8")
# Display the facets overview visualization for this data
from IPython.core.display import display, HTML

HTML_TEMPLATE = """<link rel="import" href="/nbextensions/facets-jupyter.html" >
        <facets-overview id="elem"></facets-overview>
        <script>
          document.querySelector("#elem").protoInput = "{protostr}";
        </script>"""

html = HTML_TEMPLATE.format(protostr=protostr)
display(HTML(html))<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.10214342418629267, 'fraction_numerical': 0.0037046837787774543, 'mean_word_length': 4.510204081632653, 'pattern_counts': {'":': 0, '<': 10, '<?xml version=': 0, '>': 13, 'https://': 5, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '1', 'dir': 'jupyter-structured-clean-dedup', 'id': '54439', 'license': 'Apache-2.0', 'n_tokens_mistral': 1115, 'n_tokens_neox': 1069, 'n_words': 392, 'path': 'Week_01_Unit_03_Facets.ipynb', 'repo_name': 'DGCoach/jupyter', 'size': '3779'}