<jupyter_start><jupyter_text>Analyzing Ice Dance Scoring by Team and Judge CountryThis analysis builds off BuzzFeed News' prior work analzying figure skating scores, which you can find [here](../README.md). The code below calculates the percentage of times judges from any given country have scored each ice dance team above or below the average of the other judges for that performance at 17 high-level competitions between Oct. 2016 and Dec. 2017.<jupyter_code>import pandas as pd<jupyter_output><empty_output><jupyter_text>Load Judge and Score Data*Note:* This section is a reproduction of the setup steps in the [`home-country-preference` notebook](./home-country-preference.ipynb). Please see that notebook for more details about the process.<jupyter_code>all_judges = pd.read_csv("../data/processed/judges.csv")
judge_nat = pd.read_csv("../data/processed/judge-country.csv")
judges = pd.merge(
    all_judges,
    judge_nat,
    on="clean_judge_name"
)
def clean_judge_number(role):
    return "J" + role.strip()[-1]
judges["clean_role"] = judges["role"].apply(clean_judge_number)<jupyter_output><empty_output><jupyter_text>Load score data<jupyter_code>performances = pd.read_csv("../data/raw/performances.csv")
print("{:,} performances".format(len(performances)))

aspects = pd.read_csv("../data/raw/judged-aspects.csv")
print("{:,} aspects".format(len(aspects)))

scores = pd.read_csv("../data/raw/judge-scores.csv")
print("{:,} scores".format(len(scores)))
judge_goe = pd.read_csv("../data/processed/judge-goe.csv")
scores_with_context = scores.pipe(
    pd.merge,
    aspects,
    on = "aspect_id",
    how = "left"
).pipe(
    pd.merge,
    performances,
    on = "performance_id",
    how = "left"
).pipe(
    pd.merge,
    judge_goe,
    on = [ "aspect_id", "judge" ],
    how = "left"
).assign(
    is_junior = lambda x: x["program"].str.contains("JUNIOR"),
    program_type = lambda x: x["program"]\
        .apply(lambda x: "short" if "SHORT" in x else "free")
)
assert len(scores) == len(scores_with_context)<jupyter_output><empty_output><jupyter_text>Set Up Data for Analysis<jupyter_code>senior_scores = scores_with_context[
    (scores_with_context["is_junior"] == False) &
    # Because we are only analyzing Ice Dance in this notebook
    # we limit the scope of the scoring data to those programs
    (scores_with_context["program"].str.contains("ICE DANCE"))
].copy()

senior_scores["performance_id"].nunique()
def total_points(row):
    if row["section"] == "elements":
        return round(row["base_value"] + row["judge_goe"], 2)
    
    elif row["section"] == "components":
        return round(row["factor"] * row["score"], 2)
    
    else:
        print("Unknown section: {}".format(row["section"]))
        return None
senior_scores["total_points"] = senior_scores.apply(total_points, axis=1)
perf_judge_grps = senior_scores[
    ~senior_scores["total_points"].isnull()
].groupby(["performance_id", "judge"])
points_by_judge = pd.DataFrame({
    "points": perf_judge_grps["total_points"].sum(),
    "deductions": perf_judge_grps["total_deductions"].first(),
    "name": perf_judge_grps["name"].first(),
    "nation": perf_judge_grps["nation"].first(),
    "program": perf_judge_grps["program"].first(),
    "program_type": perf_judge_grps["program_type"].first(),
    "competition": perf_judge_grps["competition"].first()
}).reset_index()
points_by_judge["final_score"] = points_by_judge["points"] - points_by_judge["deductions"]

points_by_judge.head()
perf_grps = points_by_judge.groupby(["performance_id"])
perfs = pd.DataFrame({
    "total_points": perf_grps["final_score"].sum(),
    "total_judges": perf_grps.size()
}).reset_index()
points_with_comparison = pd.merge(
    points_by_judge,
    perfs,
    how = "left",
    on = "performance_id"
)

points_with_comparison.head()
points_with_comparison["avg_without_judge"] = points_with_comparison\
    .apply(lambda x: (x["total_points"] - x["final_score"]) / (x["total_judges"] - 1), axis=1)
points_with_comparison["points_vs_avg"] = points_with_comparison["final_score"] - \
    points_with_comparison["avg_without_judge"]
judge_points = pd.merge(
    points_with_comparison,
    judges[[
        "program", "competition", "segment_category",
        "clean_judge_name", "judge_country", "clean_role"
    ]],
    left_on=[ "program", "competition", "judge" ],
    right_on=[ "program", "competition", "clean_role" ],
    how="left"
).dropna(subset=["judge_country"])

judge_points.head()<jupyter_output><empty_output><jupyter_text>Analyze Ice Dance Competition<jupyter_code>def compare_scores_by_judge_country(skater_name):
    
    skater_scores = judge_points[
        judge_points["name"] == skater_name
    ]
    
    country_grps = skater_scores.groupby("judge_country")
    
    country_df = pd.DataFrame({
        "scores_above_average": country_grps.apply(lambda x: len(x[x["points_vs_avg"] > 0])),
        "scores_below_average": country_grps.apply(lambda x: len(x[x["points_vs_avg"] < 0])),
        "scores_exactly_average": country_grps.apply(lambda x: len(x[x["points_vs_avg"] == 0])),
        "total_scores": country_grps.size(),
        "pct_above_average": round(country_grps.apply(lambda x: len(x[x["points_vs_avg"] > 0])) / \
            country_grps.size() * 100, 2),
        "pct_below_average": round(country_grps.apply(lambda x: len(x[x["points_vs_avg"] < 0])) / \
            country_grps.size() * 100, 2)
    })
    
    return country_df[
        country_df["total_scores"] >= 5
    ].sort_values("pct_above_average", ascending=False)<jupyter_output><empty_output><jupyter_text>Gold Medal Contenders **Results for Virtue/Moir (Canada)**<jupyter_code>compare_scores_by_judge_country("<NAME> / <NAME>")<jupyter_output><empty_output><jupyter_text>**Results for Papadakis/Cizeron (France)**<jupyter_code>compare_scores_by_judge_country("<NAME> / <NAME>")<jupyter_output><empty_output><jupyter_text>**Comparing results for the two teams**(Includes only countries whose judges scored, in the dataset, at least five performances of each team.)<jupyter_code>compare_scores_by_judge_country("<NAME> / <NAME>")\
    .join(
        compare_scores_by_judge_country("<NAME> / <NAME>"), 
        lsuffix="_canada", 
        rsuffix="_france")\
    .dropna()[["pct_above_average_canada", "pct_above_average_france"]]<jupyter_output><empty_output><jupyter_text>Bronze Medal Contenders **Results for Bobrova/Soloviev (Russia)**<jupyter_code>compare_scores_by_judge_country("<NAME> / <NAME>")<jupyter_output><empty_output><jupyter_text>**Results for Chock/Bates (USA)**<jupyter_code>compare_scores_by_judge_country("<NAME> / <NAME>")<jupyter_output><empty_output><jupyter_text>**Results for Shibutani/Shibutani (USA)**<jupyter_code>compare_scores_by_judge_country("<NAME> / <NAME>")<jupyter_output><empty_output><jupyter_text>**Results for Hubbell/Donohue (USA)**<jupyter_code>compare_scores_by_judge_country("<NAME> / <NAME>")<jupyter_output><empty_output><jupyter_text>**Results for Cappellini/Lanotte (Italy)**<jupyter_code>compare_scores_by_judge_country("<NAME> / <NAME>")<jupyter_output><empty_output><jupyter_text>**Comparing results for the four teams**<jupyter_code>team_scores = judge_points[
    (
        judge_points["name"].isin([
            "<NAME> / <NAME>",
            "<NAME> / <NAME>",
            "<NAME> / <NAME>",
            "<NAME> / <NAME>"
        ])) &
    (
        judge_points["judge_country"].isin([
            "ITA", "RUS", "USA", "JPN"
        ]))
]
    
grps = team_scores.groupby(["judge_country","name"])
    
team_df = pd.DataFrame({
    "scores_above_average": grps.apply(lambda x: len(x[x["points_vs_avg"] > 0])),
    "total_scores": grps.size(),
    "pct_above_average": round(grps.apply(lambda x: len(x[x["points_vs_avg"] > 0])) / \
        grps.size() * 100, 2)
}).unstack()
    
team_df.sort_values(("pct_above_average", "<NAME> / <NAME>"), ascending=False)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.15638794299407238, 'fraction_numerical': 0.004540295119182747, 'mean_word_length': 5.1141094834232845, 'pattern_counts': {'":': 20, '<': 87, '<?xml version=': 0, '>': 90, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '14', 'dir': 'jupyter-structured-clean-dedup', 'id': '90635', 'license': 'CC-BY-4.0', 'n_tokens_mistral': 2887, 'n_tokens_neox': 2711, 'n_words': 461, 'path': 'notebooks/ice-dance-team-scoring.ipynb', 'repo_name': 'BuzzFeedNews/2018-02-figure-skating-analysis', 'size': '8159'}