<jupyter_start><jupyter_text>Stats Quality for 2016 College NationalsAs one of the biggest tournaments hosted by USAU, the Club Nationals is one of the few tournaments where player statistics are relatively reliably tracked. For each tournament game, each player's aggregate scores, assists, Ds, and turns are counted, although its quite possible the definition of a "D" or a "Turn" could differ across stat-keepers.Data below was scraped from the [USAU website](http://play.usaultimate.org/events/). First we'll set up some imports to be able to load this data.<jupyter_code>import usau.reports
import usau.fantasy
from IPython.display import display, HTML
import pandas as pd
pd.options.display.width = 200
pd.options.display.max_colwidth = 200
pd.options.display.max_columns = 200
def display_url_column(df):
  """Helper for formatting url links"""
  df.url = df.url.apply(lambda url: "<a href='{base}{url}'>Match Report Link</a>"
                        .format(base=usau.reports.USAUResults.BASE_URL, url=url))
  display(HTML(df.to_html(escape=False)))<jupyter_output><empty_output><jupyter_text>Since we should already have the data downloaded as csv files in this repository, we will not need to re-scrape the data. Omit this cell to directly download from the USAU website (may be slow).<jupyter_code># Read data from csv files
usau.reports.club_nats_men_2016.load_from_csvs()
usau.reports.club_nats_mixed_2016.load_from_csvs()
usau.reports.club_nats_women_2016.load_from_csvs()<jupyter_output><empty_output><jupyter_text>Let's take a look at the games for which the sum of the player goals/assists is less than the final score of the game:<jupyter_code>missing_tallies = pd.concat([usau.reports.club_nats_men_2016.missing_tallies,
                             usau.reports.club_nats_mixed_2016.missing_tallies,
                             usau.reports.club_nats_women_2016.missing_tallies,
                            ])
display_url_column(missing_tallies[["Score", "Gs", "As", "Ds", "Ts", "Team", "Opponent", "url"]])<jupyter_output>/spare/local/azhu/venv_el6-usau-stonelib/lib/python2.7/site-packages/pandas/core/generic.py:2698: SettingWithCopyWarning: 
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead

See the caveats in the documentation: http://pandas.pydata.org/pandas-docs/stable/indexing.html#indexing-view-versus-copy
  self[name] = value<jupyter_text>There are a total of 69 unreported scorers and 86 unreported assisters (although its possible some of those 17 scores were callahans). At a quick glance a lot of these missing results are from less important games, such as the Machine-Madison Club placement game.<jupyter_code>(missing_tallies["Score"] - missing_tallies["Gs"]).sum(), (missing_tallies["Score"] - missing_tallies["As"]).sum()<jupyter_output><empty_output><jupyter_text>All games had reported turnovers:<jupyter_code>men_matches = usau.reports.club_nats_men_2016.match_results
mixed_matches = usau.reports.club_nats_mixed_2016.match_results
women_matches = usau.reports.club_nats_women_2016.match_results
display_url_column(pd.concat([men_matches[(men_matches.Ts == 0) & (men_matches.Gs > 0)],
                              mixed_matches[(mixed_matches.Ts == 0) & (mixed_matches.Gs > 0)],
                              women_matches[(women_matches.Ts == 0) & (women_matches.Gs > 0)]])
                   [["Score", "Gs", "As", "Ds", "Ts", "Team", "Opponent", "url"]])<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.10401823881447705, 'fraction_numerical': 0.01937874038187518, 'mean_word_length': 4.616, 'pattern_counts': {'":': 0, '<': 22, '<?xml version=': 0, '>': 25, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '5', 'dir': 'jupyter-structured-clean-dedup', 'id': '456705', 'license': 'MIT', 'n_tokens_mistral': 1217, 'n_tokens_neox': 1097, 'n_words': 297, 'path': 'notebooks/2016-Club_Nationals_Data_Quality.ipynb', 'repo_name': 'azjps/usau-py', 'size': '3509'}