<jupyter_start><jupyter_text>WeatherPy---- Note* Instructions have been included for each segment. You do not have to follow them exactly, but they are included to help you think through the steps.<jupyter_code># Dependencies and Setup
import matplotlib.pyplot as plt
import pandas as pd
import numpy as np
import requests
import time
from scipy.stats import linregress
from ratelimit import limits
from datetime import datetime

# Import API key
from api_keys import weather_api_key
from api_keys import weather_api_key2

# Incorporated citipy to determine city based on latitude and longitude
from citipy import citipy

# Output File (CSV)
output_data_file = 'output_data/cities.csv'

# Range of latitudes and longitudes
lat_range = (-90, 90)
lng_range = (-180, 180)<jupyter_output><empty_output><jupyter_text>Generate Cities List<jupyter_code># List for holding lat_lngs and cities
lat_lngs = []
cities = []

# Create a set of random lat and lng combinations
lats = np.random.uniform(lat_range[0], lat_range[1], size=1500)
lngs = np.random.uniform(lng_range[0], lng_range[1], size=1500)
lat_lngs = zip(lats, lngs)

# Identify nearest city for each lat, lng combination
for lat_lng in lat_lngs:
    city = citipy.nearest_city(lat_lng[0], lat_lng[1]).city_name
    
    # If the city is unique, then add it to a our cities list
    if city not in cities:
        cities.append(city)

# Print the city count to confirm sufficient count
len(cities)
# cities_final = cities
# len(cities_final)
# For Testing Only - examine json return
# city = "Budapest"

# response = requests.get(query_url + city + "&units=Imperial").json()
# response
# # For Testing Only - Short
# # List for holding lat_lngs and cities
# lat_lngs = []
# cities = []

# # Create a set of random lat and lng combinations
# lats = np.random.uniform(lat_range[0], lat_range[1], size=60)
# lngs = np.random.uniform(lng_range[0], lng_range[1], size=60)
# lat_lngs = zip(lats, lngs)

# # Identify nearest city for each lat, lng combination
# for lat_lng in lat_lngs:
#     city = citipy.nearest_city(lat_lng[0], lat_lng[1]).city_name
    
#     # If the city is unique, then add it to a our cities list
#     if city not in cities:
#         cities.append(city)

# # Print the city count to confirm sufficient count
# len(cities)
# # cities_final = cities
# # len(cities_final)
# For Testing Only - Very Short
# cities = ["Quebradillas", "Arecibo", "Aguada", "Aguadilla"]<jupyter_output><empty_output><jupyter_text>Perform API Calls* Perform a weather check on each city using a series of successive API calls.* Include a print log of each city as it'sbeing processed (with the city number and city name).<jupyter_code>url = "http://api.openweathermap.org/data/2.5/weather?"
query_url = f"{url}appid={weather_api_key2}&q="
query_url

city_list = []
lat = []
lng = []
max_temp = []
humidity = []
cloudiness = []
wind_speed = []
country = []
date = []
temp = []
count = 0


for city in cities:
    try: 
        time.sleep(1)
        response = requests.get(query_url + city + '&units=Imperial').json()
        lat.append(response['coord']['lat'])
        lng.append(response['coord']['lon'])
        max_temp.append(response['main']['temp_max'])
        humidity.append(response['main']['humidity'])
        cloudiness.append(response['clouds']['all'])
        wind_speed.append(response['wind']['speed'])
        country.append(response['sys']['country'])
        date.append(datetime.fromtimestamp(response['dt']).strftime('%m/%d/%y')) 
        temp.append(response['main']['temp'])
        city_list.append(city)
        count = count + 1
        print(f"Processing record {count} of Set 1 | {city}")
    except KeyError:
        print("Oops, that key doesn't exist.")

print(f"The latitude information received is: {lat}")
print(f"The temperature information received is: {lng}")
print(f"The city is: {city_list}")<jupyter_output>Processing record 1 of Set 1 | morehead
Processing record 2 of Set 1 | busselton
Processing record 3 of Set 1 | ponta do sol
Processing record 4 of Set 1 | kailua
Processing record 5 of Set 1 | lebu
Processing record 6 of Set 1 | hermanus
Processing record 7 of Set 1 | mataura
Processing record 8 of Set 1 | sinazongwe
Processing record 9 of Set 1 | andenes
Processing record 10 of Set 1 | punta arenas
Processing record 11 of Set 1 | tezu
Processing record 12 of Set 1 | eregli
Processing record 13 of Set 1 | wairoa
Processing record 14 of Set 1 | albany
Processing record 15 of Set 1 | hilo
Processing record 16 of Set 1 | arraial do cabo
Processing record 17 of Set 1 | puerto ayora
Processing record 18 of Set 1 | rikitea
Processing record 19 of Set 1 | ushuaia
Processing record 20 of Set 1 | avarua
Processing record 21 of Set 1 | longyearbyen
Processing record 22 of Set 1 | nagato
Processing record 23 of Set 1 | murgab
Processing record 24 of Set 1 | atuona
Processing record 25 of Set 1 |[...]<jupyter_text>Convert Raw Data to DataFrame* Export the city data into a .csv.* Display the DataFrame<jupyter_code>#####################
print(f"city: {len(city_list)}")
print(f"lat: {len(lat)}")
print(f"lng: {len(lng)}")
print(f"max_temp: {len(max_temp)}")
print(f"humidity: {len(humidity)}")
print(f"cloudiness: {len(cloudiness)}")
print(f"wind_speed: {len(wind_speed)}")
print(f"country: {len(country)}")
print(f"date: {len(date)}")
print(f"temp: {len(temp)}")
#####################
# create a data frame from cities, lat, and temp

weather_dict = {
    "City": city_list,
    "Latitude": lat,
    "Longitude": lng,
    "Max Temp": max_temp,
    "Humidity": humidity,
    "Cloudiness": cloudiness,
    "Wind Speed": wind_speed,
    "Country": country,
    "Date": date,
    "Temp": temp}

weather_data = pd.DataFrame(weather_dict)
weather_data.head()<jupyter_output><empty_output><jupyter_text>Inspect the data and remove the cities where the humidity > 100%.----Skip this step if there are no cities that have humidity > 100%.<jupyter_code>#  Get the indices of cities that have humidity over 100%.

##############
hum_ov_100 = weather_data[weather_data['Humidity']>100]
# weather_data = weather_data.drop(weather_data.loc[weather_data['Humidity']>100].index, inplace=True)
hum_ov_100.head()
# Make a new DataFrame equal to the city data to drop all humidity outliers by index.
# Passing "inplace=False" will make a copy of the city_data DataFrame, which we call "clean_city_data".

# Extract relevant fields from the data frame
# output_data_file = "output_data/cities.csv"
weather_data_csv = weather_data
weather_data_csv


# Export the City_Data into a csv
weather_data.to_csv(r'../output_data/weather_data.csv')

# Visualize to confirm
weather_data.head(10)<jupyter_output><empty_output><jupyter_text>Plotting the Data* Use proper labeling of the plots using plot titles (including date of analysis) and axes labels.* Save the plotted figures as .pngs.  Latitude vs. Temperature Plot<jupyter_code>##################
# Gather and plot relevant variables from dataset
date = weather_data['Date'][0]
date
plt.scatter(weather_data['Latitude'],weather_data['Max Temp'],\
            edgecolors='black',facecolors='steelblue')

# Add labels and title
plt.title(f'City Latitude vs. Max Temperature {date}')
plt.xlabel('Latitude')
plt.ylabel('Max Temperature (F)')

# Add grid lines
plt.grid (b=True,which='major',axis='both',linestyle='-',color='lightgrey')

# Save Plot
plt.savefig('../output_data/Lat_vs_Max_Temp.png')

# This plot shows the relationship between Latitude on the X-axis, and 
# Maximum temperature (in degrees, Farenheit) on the Y-axis.  We can 
# see that there is a parabolic, non-linear relationship between the 
# variables.<jupyter_output><empty_output><jupyter_text>Latitude vs. Humidity Plot<jupyter_code>##################
# Gather and plot relevant variables from dataset
plt.scatter(weather_data['Latitude'],weather_data['Humidity'],\
            edgecolors='black',facecolors='steelblue')


# Add labels and title
title = (f'City Latitude vs. Humidity {date}')
plt.title(title)
plt.xlabel('Latitude')
plt.ylabel('Humidity (%)')

# Add grid lines
plt.grid (b=True,which='major',axis='both',linestyle='-',color='lightgrey')

# Save Plot
plt.savefig('../output_data/Lat_vs_Humidity.png')
# This plot shows the relationship between Latitude on the X-axis, and 
# Humidity (%) on the Y-axis.  We can that there is not much of a clear
# relationship between the X and Y variables, but most of the obserbvations
#  seem to cluster in the upper righthand quadrant, so there might be a
#  week linear relationship between the variables.<jupyter_output><empty_output><jupyter_text>Latitude vs. Cloudiness Plot<jupyter_code>##################
# Gather and plot relevant variables from dataset
plt.scatter(weather_data['Latitude'],weather_data['Cloudiness'],\
            edgecolors='black',facecolors='steelblue')

# Add labels and title
plt.title(f'City Latitude vs. Cloudiness {date}')
plt.xlabel('Latitude')
plt.ylabel('Cloudiness (%)')

# Add grid lines
plt.grid (b=True,which='major',axis='both',linestyle='-',color='lightgrey')

# Save Plot
plt.savefig('../output_data/Lat_vs_Cloudiness.png')
# This plot shows the relationship between Latitude on the X-axis, and 
# Cloudiness (%) on the Y-axis.  We can that there is not much of a clear
# relationship between the X and Y variables as there are observations 
# scatter all over the plot.<jupyter_output><empty_output><jupyter_text>Latitude vs. Wind Speed Plot<jupyter_code>##################
# Gather and plot relevant variables from dataset
plt.scatter(weather_data['Latitude'],weather_data['Wind Speed'],\
            edgecolors='black',facecolors='steelblue')

# Add labels and title
plt.title(f'City Latitude vs. Wind Speed {date}')
plt.xlabel('Latitude')
plt.ylabel('Wind Speed (mph)')

# Add grid lines
plt.grid (b=True,which='major',axis='both',linestyle='-',color='lightgrey')

# Save Plot
plt.savefig('../output_data/Lat_vs_Wind_Speed.png')
# This plot shows the relationship between Latitude on the X-axis, and 
# Wind Speed (mph) on the Y-axis.  Since most of the observations are 
# clustered in the lower half of the quadrant, we can say there is probably 
# not much of a strong relationship between the variables.<jupyter_output><empty_output><jupyter_text>Linear Regression<jupyter_code># OPTIONAL: Create a function to create Linear Regression plots

##################
# Add the linear regression equation and line to plot
def lin_reg_plot(x_values,y_values, hemisphere, text_coordinates):
    (slope, intercept, rvalue, pvalue, stderr) = linregress(x_values, y_values)
    regress_values = x_values * slope + intercept  
    line_eq = 'y = ' + str(round(slope,2)) + 'x + ' + str(round(intercept,2))

    # New Scatter plot with regression line
    plt.scatter(x_values,y_values,edgecolors='black')
    plt.plot(x_values,regress_values,'r-')
    plt.annotate(line_eq,text_coordinates,fontsize=14,color="red")
    
    # Add R-squared annotation
    print(f'The r-squared is: {rvalue}')
    print(line_eq)


# Create Northern and Southern Hemisphere DataFrames
# store the boolean criteria in a variable to pass to the dataframe indexing function
crit_north = weather_data.Latitude >= 0
crit_south = weather_data.Latitude < 0

# Create the north and south hemisphere dataframes using boolean indexing from the criteria from above 
north_weather = weather_data[crit_north]
south_weather = weather_data[crit_south]

# The indexes will not be continuous so they need to be reset with the drop=True argument so we don't make
# the prior index as a column
north_weather = north_weather.reset_index(drop=True)
south_weather = south_weather.reset_index(drop=True)
#north_weather.head()<jupyter_output><empty_output><jupyter_text>Northern Hemisphere - Max Temp vs. Latitude Linear Regression<jupyter_code>##################
x_values = north_weather['Latitude']
y_values = north_weather['Max Temp']
hemisphere = 'Northern Hemisphere'
lin_reg_plot(x_values,y_values, title, (10, 50))

# Title and labels
plt.title(f'{hemisphere} City Latitude vs. Max Temp {date}')
plt.xlabel('Latitude')
plt.ylabel('Max Temp')

# Save Plot
plt.savefig('../output_data/Reg_NH_LvMT.png')
plt.show()
################## 
# With an R-squared value of about -.74, we van say there is a strong 
# negative correlation between latitude and the Maximum Temperature.  
# This means that nearly 74% of the variance in the Maximum Temperature 
# can be explained by latitude alone.<jupyter_output><empty_output><jupyter_text>Southern Hemisphere - Max Temp vs. Latitude Linear Regression<jupyter_code>##################
x_values = south_weather['Latitude']
y_values = south_weather['Max Temp']
hemisphere = 'Southern Hemisphere'
lin_reg_plot(x_values,y_values, title, (-50,90))

# Title and labels
plt.title(f'{hemisphere} City Latitude vs. Max Temp {date}')
plt.xlabel('Latitude')
plt.ylabel('Max Temp')

# Save Plot
plt.savefig('../output_data/Reg_SH_LvMT.png')
plt.show()
################## 
# With an R-squared value of about .80, we van say there is a strong 
# positive correlation between latitude and the Maximum Temperature.  
# This means that nearly 80% of the variance in the Maximum Temperature 
# can be explained by latitude alone.<jupyter_output><empty_output><jupyter_text>Northern Hemisphere - Humidity (%) vs. Latitude Linear Regression<jupyter_code>##################
x_values = north_weather['Latitude']
y_values = north_weather['Humidity']
hemisphere = 'Northern Hemisphere'
lin_reg_plot(x_values,y_values, title, (49, 8))

# Title and labels
plt.title(f'{hemisphere} City Latitude vs. Humidity {date}')
plt.xlabel('Latitude')
plt.ylabel('Humidity (%)')

# Save Plot
plt.savefig('../output_data/Reg_NH_LvH.png')
plt.show()
################## 
# With an R-squared value of about .16, we van say there is a very weak  
# positive correlation between latitude and the Humidity.  This means 
# that only about 16% of the variance in the Humidity can be explained by 
# latitude alone.<jupyter_output><empty_output><jupyter_text>Southern Hemisphere - Humidity (%) vs. Latitude Linear Regression<jupyter_code>##################
x_values = south_weather['Latitude']
y_values = south_weather['Humidity']
hemisphere = 'Southern Hemisphere'
lin_reg_plot(x_values,y_values, title, (-50,25))

# Title and labels
plt.title(f'{hemisphere} City Latitude vs. Humidity {date}')
plt.xlabel('Latitude')
plt.ylabel('Humidity (%)')

# Save Plot
plt.savefig('../output_data/Reg_SH_LvH.png')
plt.show()
################## 
# With an R-squared value of about -.04, we van say there is a relatively 
# weak negative correlation between latitude and the Humidity.  This 
# means that only about 4% of the variance in the Humidity can be 
# explained by latitude alone.<jupyter_output><empty_output><jupyter_text>Northern Hemisphere - Cloudiness (%) vs. Latitude Linear Regression<jupyter_code>##################
x_values = north_weather['Latitude']
y_values = north_weather['Cloudiness']
hemisphere = 'Northern Hemisphere'
lin_reg_plot(x_values,y_values, title, (47, 52))

# Title and labels
plt.title(f'{hemisphere} City Latitude vs. Cloudiness {date}')
plt.xlabel('Latitude')
plt.ylabel('Cloudiness (%)')

# Save Plot
plt.savefig('../output_data/Reg_NH_LvC.png')
plt.show()
################## 
# With an R-squared value of about -.01, we van say there is almost no 
# correlation between latitude and the Cloudiness.  This means that only 
# about 1% of the variance in the Cloudiness can be explained by latitude 
# alone, which is effectively meaningless.<jupyter_output><empty_output><jupyter_text>Southern Hemisphere - Cloudiness (%) vs. Latitude Linear Regression<jupyter_code>##################
x_values = south_weather['Latitude']
y_values = south_weather['Cloudiness']
hemisphere = 'Southern Hemisphere'
lin_reg_plot(x_values,y_values, title, (-50,25))

# Title and labels
plt.title(f'{hemisphere} City Latitude vs. Cloudiness {date}')
plt.xlabel('Latitude')
plt.ylabel('Cloudiness (%)')

# Save Plot
plt.savefig('../output_data/Reg_SH_LvC.png')
plt.show()
################## 
# With an R-squared value of about -.14, we van say there is a very weak
# negative correlation between latitude and the Cloudiness.  This means 
# that only about 14% of the variance in the Cloudiness can be explained 
# by latitude alone.<jupyter_output><empty_output><jupyter_text>Northern Hemisphere - Wind Speed (mph) vs. Latitude Linear Regression<jupyter_code>##################
x_values = north_weather['Latitude']
y_values = north_weather['Wind Speed']
hemisphere = 'Northern Hemisphere'
lin_reg_plot(x_values,y_values, title, (40, 33))

# Title and labels
plt.title(f'{hemisphere} City Latitude vs. Wind Speed {date}')
plt.xlabel('Latitude')
plt.ylabel('Wind Speed')

# Save Plot
plt.savefig('../output_data/Reg_NH_LvH.png')
plt.show()
################## 
# With an R-squared value of about -.09, we van say there is a very weak
# negative correlation between latitude and the Wind Speed.  This means 
# that only about 9% of the variance in the Wind Speed can be explained 
# by latitude alone.<jupyter_output><empty_output><jupyter_text>Southern Hemisphere - Wind Speed (mph) vs. Latitude Linear Regression<jupyter_code>##################
x_values = south_weather['Latitude']
y_values = south_weather['Wind Speed']
hemisphere = 'Southern Hemisphere'
lin_reg_plot(x_values,y_values, title, (-50,35))

# Title and labels
plt.title(f'{hemisphere} City Latitude vs. Wind Speed {date}')
plt.xlabel('Latitude')
plt.ylabel('Wind Speed')

# Save Plot
plt.savefig('../output_data/Reg_SH_LvW.png')
plt.show()
################## 
# With an R-squared value of about -.09, we van say there is a very weak
# negative correlation between latitude and the Wind Speed.  This means 
# that only about 9% of the variance in the Wind Speed can be explained 
# by latitude alone.<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.1317898985369135, 'fraction_numerical': 0.011043219911430013, 'mean_word_length': 5.270650263620387, 'pattern_counts': {'":': 10, '<': 73, '<?xml version=': 0, '>': 77, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '18', 'dir': 'jupyter-structured-clean-dedup', 'id': '561844', 'license': 'MIT', 'n_tokens_mistral': 6094, 'n_tokens_neox': 5641, 'n_words': 1784, 'path': '06-Python-APIs/Instructions/starter_code/WeatherPy - V6 - 8-7-20.ipynb', 'repo_name': 'cce24/Homework', 'size': '17839'}