<jupyter_start><jupyter_text>Initial data exploration<jupyter_code>star_wars.head()
star_wars.columns<jupyter_output><empty_output><jupyter_text>Data cleaning and normalization of column values<jupyter_code># Remove row with null values
print(star_wars.shape)
star_wars = star_wars[star_wars['RespondentID'].notnull()]
print(star_wars.shape)
yes_no = {"Yes": True, "No": False}

star_wars['Have you seen any of the 6 films in the Star Wars franchise?'] = star_wars['Have you seen any of the 6 films in the Star Wars franchise?'].map(yes_no)
star_wars['Do you consider yourself to be a fan of the Star Wars film franchise?'] = star_wars['Do you consider yourself to be a fan of the Star Wars film franchise?'].map(yes_no)
# Rename each of the seen columns so the names are more intuitive.
star_wars.rename(columns={'Which of the following Star Wars films have you seen? Please select all that apply.': 'seen_1',
'Unnamed: 4':'seen_2',
'Unnamed: 5':'seen_3',
'Unnamed: 6':'seen_4',
'Unnamed: 7':'seen_5',
'Unnamed: 8':'seen_6',
}, inplace=True)
# Convert each of the seen columns so that it only contains the values True and False.
seen_cols = ['seen_1', 'seen_2', 'seen_3', 'seen_4', 'seen_5', 'seen_6']

seen_yes_no = {'Star Wars: Episode I  The Phantom Menace':True,
'Star Wars: Episode II  Attack of the Clones':True,
'Star Wars: Episode III  Revenge of the Sith':True,
'Star Wars: Episode IV  A New Hope':True,
'Star Wars: Episode V The Empire Strikes Back':True,
'Star Wars: Episode VI Return of the Jedi':True,
np.NaN: False}

for col in seen_cols:
    star_wars[col] = star_wars[col].map(seen_yes_no)
# Convert each of the ranking columns to a float type.
star_wars[star_wars.columns[9:15]] = star_wars[star_wars.columns[9:15]].astype(float)
# Give each column a more descriptive name.
star_wars.rename(columns={'Please rank the Star Wars films in order of preference with 1 being your favorite film in the franchise and 6 being your least favorite film.': 'ranking_1',
'Unnamed: 10':'ranking_2',
'Unnamed: 11':'ranking_3',
'Unnamed: 12':'ranking_4',
'Unnamed: 13':'ranking_5',
'Unnamed: 14':'ranking_6',
}, inplace=True)
ep_abbrev = ['Ep.I', 'Ep.II', 'Ep.III', 'Ep.IV', 'Ep.V','Ep.VI']<jupyter_output><empty_output><jupyter_text>Analyze average movie rankings<jupyter_code>plt.bar(range(6), star_wars[star_wars.columns[9:15]].mean())
plt.ylabel('mean_ranking')
plt.title('Mean ranking of Star Wars movings (Lower is Better)')<jupyter_output><empty_output><jupyter_text>ObservationsThe original trilogy has a better average ranking then the prequel trilogy.The average ranking of Star Wars movies from most favorite to least favorite is:1. Episode V: The Empire Strikes Back2. Episode VI: Return of the Jedi3. Episode IV: A New Hope4. Episode I: The Phantom Menace5. Episode II: Attack of the Clones6. Episode III: Revenge of the Sith  Analyze movie viewers<jupyter_code>plt.bar(range(6), star_wars[star_wars.columns[3:9]].sum(), align='center')
plt.ylabel('# Seen')
plt.xticks(range(6), ep_abbrev)
plt.title('# of Respondents that have seen each Star Wars movie')<jupyter_output><empty_output><jupyter_text>ObservationsMore respondents have seen the original trilogy than the prequel trilogy.The Star Wars movies from most viewed to least viewed is:1. Episode V: The Empire Strikes Back2. Episode VI: Return of the Jedi3. Episode I: The Phantom Menace4. Episode IV: A New Hope5. Episode II: Attack of the Clones6. Episode III: Revenge of the Sith  Analyze by segment<jupyter_code># Split the data into two groups based on binary column
males = star_wars[star_wars["Gender"] == "Male"]
females = star_wars[star_wars["Gender"] == "Female"]<jupyter_output><empty_output><jupyter_text>Segment: Analyze average movie rankings<jupyter_code>plt.bar(range(6), males[males.columns[9:15]].mean(), align='center')
plt.ylabel('mean_ranking')
plt.xticks(range(6), ep_abbrev)
plt.title('Males - Ranking of Star Wars movings (Lower is Better)')
plt.show()

plt.bar(range(6), females[females.columns[9:15]].mean(), align='center')
plt.ylabel('mean_ranking')
plt.xticks(range(6), ep_abbrev)
plt.title('Females - Ranking of Star Wars movings (Lower is Better)')
plt.show()<jupyter_output><empty_output><jupyter_text>Segment: Analyze movie viewers<jupyter_code>plt.bar(range(6), males[males.columns[3:9]].mean(), align='center')
plt.ylabel('# Seen')
plt.xticks(range(6), ep_abbrev)
plt.title('Males - Star Wars movings seen')
plt.show()

plt.bar(range(6), females[females.columns[3:9]].mean(), align='center')
plt.ylabel('# Seen')
plt.xticks(range(6), ep_abbrev)
plt.title('Females - Star Wars movings seen')
plt.show()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.1317662281647617, 'fraction_numerical': 0.017468190640500324, 'mean_word_length': 6.653465346534653, 'pattern_counts': {'":': 2, '<': 29, '<?xml version=': 0, '>': 29, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 18, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '7', 'dir': 'jupyter-structured-clean-dedup', 'id': '603944', 'license': 'MIT', 'n_tokens_mistral': 1692, 'n_tokens_neox': 1577, 'n_words': 485, 'path': 'analyzing-star-wars-movies-survey.ipynb', 'repo_name': 'awhlam/analyzing-star-wars-movies-survey', 'size': '4637'}