<jupyter_start><jupyter_text>Print the version to check that Pandas is available.If the module is not found, install pandas with `pip install pandas`.<jupyter_code>import pandas as pd
print(pd.__version__)<jupyter_output>1.2.4<jupyter_text>Import the Json file with read_json. This will make a Pandas [*DataFrame*](https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.html) (a table).<jupyter_code>df = pd.read_json("Application_Log.json")<jupyter_output><empty_output><jupyter_text>Print the first three rows to see what the table looks like.<jupyter_code>df.head(5)<jupyter_output><empty_output><jupyter_text>We can use Pandas to filter and process the structured logs. Use Unique to find all the event types seen during the session.<jupyter_code>df.type.unique()<jupyter_output><empty_output><jupyter_text>Pandas can perform vector comparisons, and filter DataFrames by row indices. Select all the SpawnObject events.<jupyter_code>df[df.event == "SpawnObject"].head(5)<jupyter_output><empty_output><jupyter_text>The Shape member shows the size of the result<jupyter_code>df[df.event == "SpawnObject"].shape<jupyter_output><empty_output><jupyter_text>The Pandas [*merge*](https://pandas.pydata.org/pandas-docs/stable/user_guide/merging.html) method is used to perform inner and outer joins to relate different log events. We merge the `Awake` events with the `SpawnObject` events based on the `NetworkScene` Ids (`sceneid` in from `ContextEventLogger` for `SpawnObject`, and `arg1` for `Awake`).<jupyter_code>spawn = df[df.event == "SpawnObject"]
awake = df[df.event == "Awake"]
f = pd.merge(spawn,awake,how="left",left_on="sceneid",right_on="arg1")
f.head(5)<jupyter_output><empty_output><jupyter_text>We can perform arithmetic operations too. We use [string](https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.Series.str.contains.html) operations, boolean arrays and [size](https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.size.html) to find the number of distinct objects spawned by the Oculus Quest.<jupyter_code>is_quest = f.arg2_y.astype(str).str.contains("Quest")
is_owner = f.arg3_x.astype(bool)
spawned_ids = f[is_quest & is_owner].arg2_x
spawned_ids.unique().size<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.12056737588652482, 'fraction_numerical': 0.004875886524822695, 'mean_word_length': 7.267399267399267, 'pattern_counts': {'":': 0, '<': 32, '<?xml version=': 0, '>': 32, 'https://': 4, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '8', 'dir': 'jupyter-structured-clean-dedup', 'id': '581971', 'license': 'Apache-2.0', 'n_tokens_mistral': 794, 'n_tokens_neox': 779, 'n_words': 176, 'path': 'Documentation/docs/files/Application_Log.ipynb', 'repo_name': 'lucidbard/ubiq', 'size': '2256'}