<jupyter_start><jupyter_text>Biodiversity in National Parks<jupyter_code>from matplotlib import pyplot as plt 
import pandas as pd 
import numpy as np

species = pd.read_csv("species_info.csv")
species.head()<jupyter_output><empty_output><jupyter_text>What is the distribution of conservation_status for animals?<jupyter_code># Find number of species in the species DataFrame.
num_species = species.scientific_name.nunique()
print("Number of species: ", num_species)
#Find there is what kind of category for the species.
different_category = species.category.unique()
num_category = species.category.nunique()
print("There is {} kinds of categories which are: {}".format(num_category,different_category))
# Find the different values of conservation status.
value_of_conserfvation_status = species.conservation_status.unique()
print("Different values of conservation status: ", value_of_conserfvation_status)
# Count the number of species of each conservation status:
count_conservation_status = species.groupby(['conservation_status']).scientific_name.nunique().reset_index()
print(count_conservation_status)<jupyter_output>conservation_status  scientific_name
0          Endangered               15
1         In Recovery                4
2  Species of Concern              151
3          Threatened               10<jupyter_text>As we calculated the number of species before, there is 5541 number of species, but we find that only less than 200 of species are in conservation status. The rest of species have no intervention.<jupyter_code>#Fill the rest species conservation status as No Intervention
species.fillna('No Intervention', inplace=True)
species
# Count the number of species of each conservation status again:
count_conservation_status = species.groupby(['conservation_status']).scientific_name.nunique().reset_index()
print(count_conservation_status)
#Order the counted status:
protection_counts = species.groupby('conservation_status').scientific_name.nunique().reset_index()

protection_counts = protection_counts.sort_values(by='scientific_name')
    
print(protection_counts)
# Use bar chart to see the distribution of the conservation status.
plt.figure(figsize=(10,4))
ax = plt.subplot(111)
plt.bar(protection_counts.conservation_status, protection_counts.scientific_name)
ax.set_xticks(range(len(protection_counts)))
ax.set_xticklabels(protection_counts.conservation_status)
plt.ylabel('Number of Species')
plt.title('Conservation Status by Species')
plt.show()<jupyter_output><empty_output><jupyter_text>Are certain types of species more likely to be endangered?Are the differences between species and their conservation status significant?<jupyter_code># Create a new column at species call is_protected. True if conservation_status is not 'No intervention', otherwise, False
species['is_protected'] = species.conservation_status.apply(lambda x: True if x != "No Intervention" else False)
species
# Group the species dataframe with category and count whether the type is protected
category_counts = species.groupby(['category', 'is_protected']).scientific_name.nunique().reset_index()
category_counts
# Pivot the table to have better looking and understanding for manipulate
category_pivot = category_counts.pivot(columns = 'is_protected', index = 'category', values= 'scientific_name').reset_index()
category_pivot
# Rename the column of False as not_protected and True as protected
category_pivot.columns = ['category', 'not_protected', 'protected']
category_pivot
# Add a new column called percent_protected
category_pivot['percent_protected'] = category_pivot.protected / (category_pivot.protected + category_pivot.not_protected)
category_pivot
#Use chi square test to see whether the species in category Mammel are more likely to be endangered than species in Bird.
# To use chi square test, create a contingency table for Bird and Mammal for protect or not protected first
contingency = [[30,146],[75,413]]
contingency

#Apply the chi square test
from scipy.stats import chi2_contingency
chi2, pval, dof, expected = chi2_contingency(contingency)
print(pval)<jupyter_output>0.6875948096661336<jupyter_text>The p-value is not less than the significant threshold 0.05.The difference isn't significant. We can not conclude that the species in category Mammel are more likely to be endangered than species in Bird.<jupyter_code># See the differece between category Reptile and Mammal.
contingency = [[5,73],[30,146]]
print(contingency)
chi2, pval, dof, expected = chi2_contingency(contingency)
print(pval)<jupyter_output>[[5, 73], [30, 146]]
0.03835559022969898<jupyter_text>Which species were spotted the most at each park? The p-value is less than the significant threshold 0.05. The difference between Reptile and Mammal is significant.<jupyter_code>observations = pd.read_csv('observations.csv')
observations.head()
# Merge species and obseervations
all_observations = species.merge(observations)
all_observations
# Group the dataframe by park_name and species
all_observations_grouped = all_observations.groupby(['park_name','scientific_name']).observations.sum().reset_index()
all_observations_grouped
# Find the most species at Bryce National Park
bnp_obs = all_observations_grouped[all_observations_grouped.park_name == 'Bryce National Park']
bnp_obs.loc[bnp_obs['observations'].idxmax()]<jupyter_output><empty_output><jupyter_text>The species columba livia was spotted the most at Bryce National Park.<jupyter_code># Find the most species at Great Smoky Mountains National Park
gsmanp_obs = all_observations_grouped[all_observations_grouped.park_name == 'Great Smoky Mountains National Park']
gsmanp_obs.loc[gsmanp_obs['observations'].idxmax()]<jupyter_output><empty_output><jupyter_text>Streptopelia decaocto was spotted the most at Great Smoky Mountains National Park.<jupyter_code># Find the most species at Yosemite National Park
yosnp_obs = all_observations_grouped[all_observations_grouped.park_name == 'Yosemite National Park']
yosnp_obs.loc[yosnp_obs['observations'].idxmax()]<jupyter_output><empty_output><jupyter_text>Hypochaeris radicata was spotted the most at Yosemite National Park.<jupyter_code># Find the most species at Yellowstone National Park
yellownp_obs = all_observations_grouped[all_observations_grouped.park_name == 'Yellowstone National Park']
yellownp_obs.loc[yellownp_obs['observations'].idxmax()]<jupyter_output><empty_output><jupyter_text>Holcus lanatus was spotted the most at Yellowstone National Park.  Further analyze for sheep mammel<jupyter_code>#Create a new column in species called is_sheep, True if the common_name contains 'Sheep', False otherwise.
species['is_sheep'] = species.common_names.apply(lambda x: True if 'Sheep' in x else False)
# Show the row with is_sheep is True and the animal category is mammal.
sheep_species = species[(species.is_sheep ==True) & (species.category == 'Mammal')]
sheep_species
# Merge sheep_species with observations to get a dataframe with observations of sheep
sheep_observations = sheep_species.merge(observations)
sheep_observations

# There is three species of sheep were observed in 4 parks. Use groupby to get the sum of observations for each park_name.
obs_by_park = sheep_observations.groupby('park_name').observations.sum().reset_index()
obs_by_park
# Use barchart to display the obserivations of sheep per week for all 4 parks
plt.figure(figsize=(16,4))
ax = plt.subplot(111)
plt.bar(obs_by_park.park_name, obs_by_park.observations)
ax.set_xticks(range(len(obs_by_park)))
ax.set_xticklabels(obs_by_park.park_name)
plt.ylabel('Number of Observations')
plt.title('Observations of Sheep per Week')
plt.show()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.08126469819702116, 'fraction_numerical': 0.013718317219754377, 'mean_word_length': 6.973958333333333, 'pattern_counts': {'":': 0, '<': 38, '<?xml version=': 0, '>': 38, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '10', 'dir': 'jupyter-structured-clean-dedup', 'id': '626086', 'license': 'Unlicense', 'n_tokens_mistral': 2397, 'n_tokens_neox': 2242, 'n_words': 684, 'path': 'Project 3 biodiversity/biodiversity.ipynb', 'repo_name': 'louischoiccl/Project-3-biodiversity', 'size': '7654'}