<jupyter_start><jupyter_text>WeatherPy---- Note* Instructions have been included for each segment. You do not have to follow them exactly, but they are included to help you think through the steps.  Trends based on the data - The only strong inference that can be made from the data is that the closer we are from the latitude zero the higher the temperature is going to be.- A weaker trend is that wind speed decreases as long as we approach to the latitude zero.- Another trend that was seen is that humidity and cloudiness decrease toward south pole.<jupyter_code># Dependencies and Setup
import matplotlib.pyplot as plt
import pandas as pd
import numpy as np
import requests
import time
from scipy.stats import linregress
import scipy.stats as st
from datetime import date

# Import API key
from api_keys import weather_api_key

# Incorporated citipy to determine city based on latitude and longitude
from citipy import citipy
def Mbox(title, text, style):
    return ctypes.windll.user32.MessageBoxW(0, text, title, style)

def build_scatter_plot(df, colx, coly, title, xlabel, ylabel, file):
    # Build a scatter plot
    plt.scatter(df[colx], df[coly], marker="o")
    # Incorporate the other graph properties
    plt.title(title)
    plt.ylabel(ylabel)
    plt.xlabel(xlabel)
    plt.grid(True)
    # Save the figure
    plt.savefig(file)
    # Show plot
    plt.show()
    
def build_linear_regression(df, colx, coly, title, xlabel, ylabel, file, le_x, le_y, r_x, r_y):
    # Perform a linear regression on coly vs. colx
    x_values = df[colx]
    y_values = df[coly]
    # Calculate the line equation using linear regression function
    (slope, intercept, rvalue, pvalue, stderr) = linregress(x_values, y_values)
    regress_values = x_values * slope + intercept
    line_eq = "y = " + str(round(slope,2)) + "x + " + str(round(intercept,2))
    # Plot original data using Scatter type
    plt.scatter(x_values,y_values, label='original data')
    # Plot fitted line using Line type
    plt.plot(x_values,regress_values,"r-", label='fitted line')
    # Add line equation to plot
    plt.annotate(line_eq,(le_x,le_y),fontsize=15,color="red")
    # Set title, xlabel, ylabel and legend
    plt.title(title)
    plt.xlabel(xlabel)
    plt.ylabel(ylabel)
    plt.legend()
    # Show r-squared value
    r2_latex = r'$r^2 = $'
    r2_value = f'{r2_latex}{round(rvalue**2,4)}'
    plt.annotate(r2_value,(r_x,r_y),fontsize=15,color="green")
    print(f"The r-value is: {rvalue}")
    # Save the figure
    plt.savefig(file)
    # Show plot
    plt.show()
# Output File (CSV)
output_data_file = "../output_data/cities.csv"

today = date.today().strftime("%m/%d/%Y")

# Range of latitudes and longitudes
lat_range = (-90, 90)
lng_range = (-180, 180)<jupyter_output><empty_output><jupyter_text>Generate Cities List<jupyter_code># List for holding lat_lngs and cities
lat_lngs = []
cities = []

# Create a set of random lat and lng combinations
lats = np.random.uniform(lat_range[0], lat_range[1], size=1500)
lngs = np.random.uniform(lng_range[0], lng_range[1], size=1500)
lat_lngs = zip(lats, lngs)

# Identify nearest city for each lat, lng combination
for lat_lng in lat_lngs:
    city = citipy.nearest_city(lat_lng[0], lat_lng[1]).city_name
    
    # If the city is unique, then add it to a our cities list
    if city not in cities:
        cities.append(city)

# Print the city count to confirm sufficient count
len(cities)<jupyter_output><empty_output><jupyter_text>Perform API Calls* Perform a weather check on each city using a series of successive API calls.* Include a print log of each city as it'sbeing processed (with the city number and city name).<jupyter_code># Save config information.
base_url = "http://api.openweathermap.org/"
endpoint = "data/2.5/weather?"
units = "imperial"
url = base_url + endpoint

# Build partial query URL
query_url = f"{url}appid={weather_api_key}&units={units}&q="
# set up lists to hold reponse info
checked_cities = []
latitudes = []
longitudes = []
max_temperatures = []
humidities = []
cloudiness = []
wind_speeds = []
countries = []
dates = []
# Loop through the list of cities and perform a request for data on each
rec = 0
print("Beginning Data Retrieval")
print("-----------------------------")
for city in cities:
    rec += 1
    time.sleep(1.05)
    response = requests.get(query_url + city).json()
    if response['cod'] == '404':
        print("City not found. Skipping...")
    elif int(response['cod']) >= 400:
        print("Error. Skipping...")
    else:
        print(f"Processing Record {rec} of {len(cities)} | {response['name']}")
        checked_cities.append(response['name'])
        latitudes.append(response['coord']['lat'])
        longitudes.append(response['coord']['lon'])
        max_temperatures.append(response['main']['temp_max'])
        humidities.append(response['main']['humidity'])
        cloudiness.append(response['clouds']['all'])
        wind_speeds.append(response['wind']['speed'])
        countries.append(response['sys']['country'])
        dates.append(response['dt'])     

# create a data frame from previuos values
weather_dict = {
    "City": checked_cities,
    "Lat": latitudes,
    "Lng": longitudes,
    "Max Temp": max_temperatures,
    "Humidity": humidities,
    "Cloudiness": cloudiness,
    "Wind Speed": wind_speeds,
    "Country": countries,
    "Date": dates
}

print("-----------------------------")
print("Data Retrieval Complete")
print("-----------------------------")
weather_data_df = pd.DataFrame(weather_dict)<jupyter_output>Beginning Data Retrieval
-----------------------------
Processing Record 1 of 658 | Mattru
Processing Record 2 of 658 | Hobart
Processing Record 3 of 658 | Hermanus
Processing Record 4 of 658 | Atuona
Processing Record 5 of 658 | Klaksvík
Processing Record 6 of 658 | Tashtyp
Processing Record 7 of 658 | Coruripe
Processing Record 8 of 658 | Saint Paul Harbor
Processing Record 9 of 658 | Rikitea
Processing Record 10 of 658 | Dobryanka
Processing Record 11 of 658 | Taksimo
Processing Record 12 of 658 | Goderich
Processing Record 13 of 658 | Saint-Louis
City not found. Skipping...
Processing Record 15 of 658 | Roald
Processing Record 16 of 658 | Cape Town
Processing Record 17 of 658 | Porosozero
Processing Record 18 of 658 | Tandur
Processing Record 19 of 658 | Qaanaaq
City not found. Skipping...
Processing Record 21 of 658 | Iranshahr
Processing Record 22 of 658 | Kasongo-Lunda
Processing Record 23 of 658 | Mahébourg
Processing Record 24 of 658 | Hilo
Processing Record 25 of 658 | Kounda[...]<jupyter_text>Convert Raw Data to DataFrame* Export the city data into a .csv.* Display the DataFrame<jupyter_code># Export file as a CSV
weather_data_df.to_csv(output_data_file, index=False, header=True)
weather_data_df.head()<jupyter_output><empty_output><jupyter_text>Inspect the data and remove the cities where the humidity > 100%.----Skip this step if there are no cities that have humidity > 100%.<jupyter_code>#  Get the indices of cities that have humidity over 100%.
humidity_threshold = 100
over100 = weather_data_df.loc[weather_data_df['Humidity'] > humidity_threshold].reset_index()['index'].tolist()
over100
# Make a new DataFrame equal to the city data to drop all humidity outliers by index.
# Passing "inplace=False" will make a copy of the city_data DataFrame, which we call "clean_city_data".
if len(over100)>0:
    clean_city_data = weather_data_df.drop(over100, inplace=False)
else:
    clean_city_data = weather_data_df.copy()
clean_city_data.head()<jupyter_output><empty_output><jupyter_text>Plotting the Data* Use proper labeling of the plots using plot titles (including date of analysis) and axes labels.* Save the plotted figures as .pngs.  Latitude vs. Temperature Plot<jupyter_code>build_scatter_plot(clean_city_data, "Lat", "Max Temp", f"City Latitude vs Max Temperature ({today})", 
                   "Latitude", "Max Temperature (F)", "../output_data/MaxTemperatureInCities.png")<jupyter_output><empty_output><jupyter_text>Here it can be noticed that as the cities approach to the equator at latitude zero, the temperature rises.  Latitude vs. Humidity Plot<jupyter_code>build_scatter_plot(clean_city_data, "Lat", "Humidity", f"City Latitude vs Humidity ({today})", 
                   "Latitude", "Humidity (%)", "../output_data/HumidityInCities.png")<jupyter_output><empty_output><jupyter_text>In this case, a clear pattern of humidity is not noticeable. Maybe it drops a little around latitude 20, at the level of the Caribbean Sea.  Latitude vs. Cloudiness Plot<jupyter_code>build_scatter_plot(clean_city_data, "Lat", "Cloudiness", f"City Latitude vs Cloudiness ({today})", 
                   "Latitude", "Cloudiness (%)", "../output_data/CloudinessInCities.png")<jupyter_output><empty_output><jupyter_text>In this cloudiness graph, the data is totally spread out with no pattern. It seems to have nothing to do with latitude.  Latitude vs. Wind Speed Plot<jupyter_code>build_scatter_plot(clean_city_data, "Lat", "Wind Speed", f"City Latitude vs Wind Speed ({today})", 
                   "Latitude", "Wind Speed (mph)", "../output_data/WindSpeedInCities.png")<jupyter_output><empty_output><jupyter_text>Here we can see a uniform pattern of low wind speeds in a wide band around the equator with some outliers over 20 mph.  Linear Regression<jupyter_code># Northern Hemisphere
hemisphere_threshold = 0
north_hem = clean_city_data.loc[clean_city_data['Lat'] >= hemisphere_threshold]
north_hem['Lat']
# Southern Hemisphere
south_hem = clean_city_data.loc[clean_city_data['Lat'] < hemisphere_threshold]
south_hem['Lat']<jupyter_output><empty_output><jupyter_text>Northern Hemisphere - Max Temp vs. Latitude Linear Regression<jupyter_code>build_linear_regression(north_hem, "Lat", "Max Temp",
                        "Northern Hemisphere - Max Temp vs. Latitude \nLinear Regression\n",
                        "Latitude", "Max Temperature (F)", "../output_data/NH_MT_vs_L_LinearRegression.png",
                        4,-40,4,-10)<jupyter_output>The r-value is: -0.8854357868563709<jupyter_text>Southern Hemisphere - Max Temp vs. Latitude Linear Regression<jupyter_code>build_linear_regression(south_hem, "Lat", "Max Temp",
                        "Southern Hemisphere - Max Temp vs. Latitude \nLinear Regression\n",
                        "Latitude", "Max Temperature (F)", "../output_data/SH_MT_vs_L_LinearRegression.png",
                        -25,50,-25,45)<jupyter_output>The r-value is: 0.5831756951126643<jupyter_text>Maximum temperature gets higher as we approach to the equator. However, the correlation is stronger in the north hemisphere for this dataset.   Northern Hemisphere - Humidity (%) vs. Latitude Linear Regression<jupyter_code>build_linear_regression(north_hem, "Lat", "Humidity",
                        "Northern Hemisphere - Humidity vs. Latitude \nLinear Regression\n",
                        "Latitude", "Humidity (%)", "../output_data/NH_H_vs_L_LinearRegression.png",
                        45,35,50,55)<jupyter_output>The r-value is: 0.40862229477942336<jupyter_text>Southern Hemisphere - Humidity (%) vs. Latitude Linear Regression<jupyter_code>build_linear_regression(south_hem, "Lat", "Humidity",
                        "Southern Hemisphere - Humidity vs. Latitude \nLinear Regression\n",
                        "Latitude", "Humidity (%)", "../output_data/SH_H_vs_L_LinearRegression.png",
                        -30,49,-20,30)<jupyter_output>The r-value is: 0.22321663756386295<jupyter_text>The trend is heading downward as we head to the south pole. But the correlation is not strong enough to conclude that.   Northern Hemisphere - Cloudiness (%) vs. Latitude Linear Regression<jupyter_code>build_linear_regression(north_hem, "Lat", "Cloudiness",
                        "Northern Hemisphere - Cloudiness vs. Latitude \nLinear Regression\n",
                        "Latitude", "Cloudiness (%)", "../output_data/NH_C_vs_L_LinearRegression.png",
                        32,32,12,62)<jupyter_output>The r-value is: 0.3774026256884425<jupyter_text>Southern Hemisphere - Cloudiness (%) vs. Latitude Linear Regression<jupyter_code>build_linear_regression(south_hem, "Lat", "Cloudiness",
                        "Southern Hemisphere - Cloudiness vs. Latitude \nLinear Regression\n",
                        "Latitude", "Cloudiness (%)", "../output_data/SH_C_vs_L_LinearRegression.png",
                        -55,20.5,-25,10)<jupyter_output>The r-value is: 0.23980465689907637<jupyter_text>Similar to the previous chart, there is a slight trend to increase cloudiness as long as we go up to the north pole. However, correlation is still too low for a conclusion.   Northern Hemisphere - Wind Speed (mph) vs. Latitude Linear Regression<jupyter_code>build_linear_regression(north_hem, "Lat", "Wind Speed",
                        "Northern Hemisphere - Wind Speed vs. Latitude \nLinear Regression\n",
                        "Latitude", "Wind Speed (mph)", "../output_data/NH_WS_vs_L_LinearRegression.png",
                        0,21.5,40,20)<jupyter_output>The r-value is: 0.1107354105088309<jupyter_text>Southern Hemisphere - Wind Speed (mph) vs. Latitude Linear Regression<jupyter_code>build_linear_regression(south_hem, "Lat", "Wind Speed",
                        "Southern Hemisphere - Wind Speed vs. Latitude \nLinear Regression\n",
                        "Latitude", "Wind Speed (mph)", "../output_data/SH_WS_vs_L_LinearRegression.png",
                        -57,25,-20,19.5)<jupyter_output>The r-value is: -0.3002593666713614
{'fraction_non_alphanumeric': 0.11216814159292035, 'fraction_numerical': 0.028908554572271386, 'mean_word_length': 3.771639690358902, 'pattern_counts': {'":': 9, '<': 65, '<?xml version=': 0, '>': 70, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '18', 'dir': 'jupyter-structured-clean-dedup', 'id': '655158', 'license': 'ADSL', 'n_tokens_mistral': 4616, 'n_tokens_neox': 4166, 'n_words': 1253, 'path': 'WeatherPy/WeatherPy.ipynb', 'repo_name': 'luiserojas74/python-api-challenge', 'size': '13560'}