<jupyter_start><jupyter_text>WeatherPy---- Note* Instructions have been included for each segment. You do not have to follow them exactly, but they are included to help you think through the steps.<jupyter_code># Dependencies and Setup
import matplotlib.pyplot as plt
import pandas as pd
import numpy as np
import requests
import time
from scipy.stats import linregress

# Import API key
from api_keys import weather_api_key



# Incorporated citipy to determine city based on latitude and longitude
from citipy import citipy

# Output File (CSV)
output_data_file = "output_data/cities.csv"

# Range of latitudes and longitudes
lat_range = (-90, 90)
lng_range = (-180, 180)<jupyter_output><empty_output><jupyter_text>Generate Cities List<jupyter_code># List for holding lat_lngs and cities
lat_lngs = []
cities = []

# Create a set of random lat and lng combinations
lats = np.random.uniform(lat_range[0], lat_range[1], size=1500)
lngs = np.random.uniform(lng_range[0], lng_range[1], size=1500)
lat_lngs = zip(lats, lngs)

# Identify nearest city for each lat, lng combination
for lat_lng in lat_lngs:
    city = citipy.nearest_city(lat_lng[0], lat_lng[1]).city_name
    
    # If the city is unique, then add it to a our cities list
    if city not in cities:
        cities.append(city)
        
# Print the city count to confirm sufficient count
len(cities)
city_name_list = []
cloudiness_list = []
country_list = []
date_list= []
humidity_list = []
lat_list = []
lng_list = []
max_temp_list = []
wind_speed_list = []
index_counter = 0
set_counter = 1<jupyter_output><empty_output><jupyter_text>Perform API Calls* Perform a weather check on each city using a series of successive API calls.* Include a print log of each city as it'sbeing processed (with the city number and city name).<jupyter_code>print("Beginning Data Retrieval ")
print("-----------------------------")

base_url = "http://api.openweathermap.org/data/2.5/weather?"
units = "imperial"
query_url = f"{base_url}appid={weather_api_key}&units={units}&q="


# For each city name in cities list, do below things...
for index, city in enumerate(cities, start = 1):
    try:
        response = requests.get(query_url + city).json()
        city_name_list.append(response["name"])
        cloudiness_list.append(response["clouds"]["all"])
        country_list.append(response["sys"]["country"])
        date_list.append(response["dt"])
        humidity_list.append(response["main"]["humidity"])
        lat_list.append(response["coord"]["lat"])
        lng_list.append(response["coord"]["lon"])
        max_temp_list.append(response['main']['temp_max'])
        wind_speed_list.append(response["wind"]["speed"])
        if index_counter > 49:
            index_counter = 0
            set_counter = set_counter + 1
    
        else:
            index_counter = index_counter + 1
            
        print(f"Processing Record {index_counter} of Set {set_counter} : {city}") 
  
    except(KeyError, IndexError):
        print("City not found.")

print("-----------------------------")
print("Data Retrieval Complete")
print("-----------------------------")<jupyter_output>Beginning Data Retrieval 
-----------------------------
Processing Record 1 of Set 1 : puerto ayora
Processing Record 2 of Set 1 : kruisfontein
Processing Record 3 of Set 1 : hasaki
Processing Record 4 of Set 1 : ushuaia
Processing Record 5 of Set 1 : georgetown
Processing Record 6 of Set 1 : trinidad
Processing Record 7 of Set 1 : hobart
City not found.
Processing Record 8 of Set 1 : rikitea
Processing Record 9 of Set 1 : borovoy
Processing Record 10 of Set 1 : katsuura
Processing Record 11 of Set 1 : karratha
Processing Record 12 of Set 1 : qaanaaq
Processing Record 13 of Set 1 : umm lajj
Processing Record 14 of Set 1 : manokwari
City not found.
Processing Record 15 of Set 1 : sao joao da barra
Processing Record 16 of Set 1 : castanos
Processing Record 17 of Set 1 : lata
Processing Record 18 of Set 1 : webster
City not found.
Processing Record 19 of Set 1 : port hedland
Processing Record 20 of Set 1 : chara
Processing Record 21 of Set 1 : pangnirtung
City not found.
Processing Record[...]<jupyter_text>Convert Raw Data to DataFrame* Export the city data into a .csv.* Display the DataFrame<jupyter_code># Create a panda data frame using data retrieved
weather_df = pd.DataFrame({ 
                "City" : city_name_list,
                "Lat" : lat_list,
                "Lng" : lng_list,
                "Max Temp" : max_temp_list,
                "Humidity" : humidity_list,
                "Cloudiness" : cloudiness_list,
                "Wind Speed" : wind_speed_list,
                "Country" : country_list,
                "Date" : date_list
               
                
                
                
               
})

#Exporting data to csv
weather_df.to_csv("../output_data/city.csv")

#Displaying dataframe
weather_df.head()
weather_df.count()
weather_df<jupyter_output><empty_output><jupyter_text>Inspect the data and remove the cities where the humidity > 100%.----Skip this step if there are no cities that have humidity > 100%.<jupyter_code># --- find cities that have humidity over 100% ---
humidity_over_100 = weather_df.loc[weather_df["Humidity"] > 100, :]

if humidity_over_100.empty == True:
    print("There are no cities with humidity values over 100%")
    print("In this case continue with plotting...")
    
    # --- extract relevant fields from the data frame for plotting ---
    latitudes = weather_df["Lat"]
    temperatures = weather_df["Max Temp"]
    humidity = weather_df["Humidity"]
    cloudiness = weather_df["Cloudiness"]
    windspeed = weather_df["Wind Speed"]
    
else:
    # -- get the indices of cities that have humidity over 100% ---
    Humidityover100Index = humidity_over_100.index
    
    # --- create a clean DataFrame by dropping all humidity outliers by index found above ---
    clean_weather_data = weather_df.drop(Humidityover100Index).reset_index(drop=True)
    clean_weather_data.head()
    
    # --- extract relevant fields from the data frame for plotting ---
    latitudes = clean_weather_data["Lat"]
    temperatures = clean_weather_data["Max Temp"]
    humidity = clean_weather_data["Humidity"]
    cloudiness = clean_weather_data["Cloudiness"]
    windspeed = clean_weather_data["Wind Speed"]

    # --- export the new dataframe into a csv ---
    clean_weather_data.to_csv("../output_data/clean_cities_data.csv", index=True, index_label="City ID", header=True)
#  Get the indices of cities that have humidity over 100%

test_stats = weather_df["Humidity"].describe(include = "all")
print("By inspection, the maximum humidity value is 100%, so there are no values greater than 100% in this data")
test_stats
#  Get the indices of cities that have humidity over 100%.
humidity_over_100.index<jupyter_output><empty_output><jupyter_text>Plotting the Data* Use proper labeling of the plots using plot titles (including date of analysis) and axes labels.* Save the plotted figures as .pngs.  Latitude vs. Temperature Plot<jupyter_code>#Converting Date for title of plot
plt.scatter(weather_df['Lat'], weather_df['Max Temp'], marker = "o")

#Creating plot
plt.scatter(weather_df["Lat"], weather_df["Max Temp"], marker = "o", color = "green",edgecolor = "black")
plt.grid()
plt.xlabel("Latitude")
plt.ylabel("Max Temperature")
plt.title(f"City Latitude vs. Max Temperature")
plt.savefig("output_data/Latitude vs. Max Temperature Plot.png")
plt.show()<jupyter_output><empty_output><jupyter_text>Latitude vs. Humidity Plot<jupyter_code>plt.scatter(weather_df["Lat"], weather_df["Humidity"], marker = "o", color = "green",edgecolor = "black")
plt.grid()
plt.ylim(0, weather_df["Humidity"].max()+50)
plt.xlabel("Latitude")
plt.ylabel("Humidity")
plt.title(f"City Latitude vs. Humidity")
plt.savefig("output_data/Lat_vs_humidity_plot")
plt.show()<jupyter_output><empty_output><jupyter_text>Latitude vs. Cloudiness Plot<jupyter_code>plt.scatter(weather_df["Lat"], weather_df["Cloudiness"], marker = "o", color = "green",edgecolor = "black")
plt.grid()
plt.xlabel("Latitude")
plt.ylabel("Cloudiness(%)")
plt.title(f"City Latitude vs. Cloudiness")
plt.savefig("output_data/Lat_vs_cloudiness_plot")
plt.show()<jupyter_output><empty_output><jupyter_text>Latitude vs. Wind Speed Plot<jupyter_code>plt.scatter(weather_df["Lat"], weather_df["Wind Speed"], marker = "o", color = "green",edgecolor = "black")
plt.grid()
plt.xlabel("Latitude")
plt.ylabel("Wind Speed")
plt.title(f"City Latitude vs. Wind Speed")
plt.savefig("output_data/Lat_vs_windSpeed_plot")
plt.show()<jupyter_output><empty_output><jupyter_text>Linear Regression<jupyter_code>def plotLinearRegression(xdata,ydata,xlbl,ylbl,lblpos,ifig):
    (slope, intercept, rvalue, pvalue, stderr) = linregress(xdata, ydata)
    print(f"The r-squared is: {rvalue}")
    regress_values = xdata * slope + intercept
    line_eq = "y = " + str(round(slope,2)) + "x + " + str(round(intercept,2))

    plt.scatter(xdata,ydata)
    plt.plot(xdata,regress_values,"r-")
    plt.annotate(line_eq,lblpos,fontsize=15,color="red")
    plt.xlabel(xlbl)
    plt.ylabel(ylbl)
    plt.savefig(f"output_data/fig{ifig}.png")
    plt.show()
# Create Northern and Southern Hemisphere DataFrames
Northern_hemisphere = weather_df.loc[weather_df["Lat"]>0]
Southern_hemisphere = weather_df.loc[weather_df["Lat"]<0]

Northern_hemisphere.head()

x_values = Northern_hemisphere["Lat"]
y_values = Northern_hemisphere["Max Temp"]

(slope, intercept, rvalue, pvalue, stderr) = linregress(x_values, y_values)
regress_values = x_values * slope + intercept
line_eq = "y = " + str(round(slope,2)) + "x + " + str(round(intercept,2))
correlation = (x_values, y_values)

plt.scatter(x_values,y_values)
plt.title('Temperatures at Latitude Positions (Northern Hemisphere)')
plt.plot(x_values,regress_values,"r-")
plt.annotate(line_eq,(10,20),fontsize=12,color="red")
plt.xlabel('Latitude')
plt.ylabel('Temperature (F)')
plt.ylim(-5, 100)
plt.xlim(0, 80)
plt.minorticks_on()
plt.grid(which='major', linestyle='-')
plt.grid(which='minor', linestyle=':')
plt.tight_layout()
plt.show()
plt.savefig("output_data/nh_lat_temp_regression.png")<jupyter_output><empty_output><jupyter_text>Northern Hemisphere - Max Temp vs. Latitude Linear Regression<jupyter_code>xlbl = "Lat"
ylbl = "Max Temp"
lblpos = (0,25)
plotLinearRegression(Northern_hemisphere[xlbl],Northern_hemisphere[ylbl],xlbl,ylbl,lblpos,6)
plt.savefig("output_data/Linear_Regresion_plot")<jupyter_output>The r-squared is: -0.8386581475402424<jupyter_text>Southern Hemisphere - Max Temp vs. Latitude Linear Regression<jupyter_code>xlbl = "Lat"
ylbl = "Max Temp"
lblpos = (-55,90)
plotLinearRegression(Southern_hemisphere[xlbl],Southern_hemisphere[ylbl],xlbl,ylbl,lblpos,7)
plt.savefig("output_data/Linear_Regresion_plot")<jupyter_output>The r-squared is: 0.5986839205783634<jupyter_text>Northern Hemisphere - Humidity (%) vs. Latitude Linear Regression<jupyter_code>xlbl = "Lat"
ylbl = "Humidity"
lblpos = (45,10)
plotLinearRegression(Northern_hemisphere[xlbl],Northern_hemisphere[ylbl],xlbl,ylbl,lblpos,8)
plt.savefig("output_data/Linear_Regresion_plot")<jupyter_output>The r-squared is: 0.431879440360338<jupyter_text>Southern Hemisphere - Humidity (%) vs. Latitude Linear Regression<jupyter_code>xlbl = "Lat"
ylbl = "Humidity"
lblpos = (-55,15)
plotLinearRegression(Southern_hemisphere[xlbl],Southern_hemisphere[ylbl],xlbl,ylbl,lblpos,9)
plt.savefig("output_data/Linear_Regresion_plot")<jupyter_output>The r-squared is: 0.19802558447774485<jupyter_text>Northern Hemisphere - Cloudiness (%) vs. Latitude Linear Regression<jupyter_code>xlbl = "Lat"
ylbl = "Cloudiness"
lblpos = (20,40)
plotLinearRegression(Northern_hemisphere[xlbl],Northern_hemisphere[ylbl],xlbl,ylbl,lblpos,10)
plt.savefig("output_data/Linear_Regresion_plot")<jupyter_output>The r-squared is: 0.14507084598311096<jupyter_text>Southern Hemisphere - Cloudiness (%) vs. Latitude Linear Regression<jupyter_code>xlbl = "Lat"
ylbl = "Cloudiness"
lblpos = (-55,50)
plotLinearRegression(Southern_hemisphere[xlbl],Southern_hemisphere[ylbl],xlbl,ylbl,lblpos,11)
plt.savefig("output_data/Linear_Regresion_plot")<jupyter_output>The r-squared is: 0.44171741721691743<jupyter_text>Northern Hemisphere - Wind Speed (mph) vs. Latitude Linear Regression<jupyter_code>xlbl = "Lat"
ylbl = "Wind Speed"
lblpos = (0,30)
plotLinearRegression(Northern_hemisphere[xlbl],Northern_hemisphere[ylbl],xlbl,ylbl,lblpos,12)
plt.savefig("output_data/Linear_Regresion_plot")<jupyter_output>The r-squared is: 0.15300827961560898<jupyter_text>Southern Hemisphere - Wind Speed (mph) vs. Latitude Linear Regression<jupyter_code>xlbl = "Lat"
ylbl = "Wind Speed"
lblpos = (-25,33)
plotLinearRegression(Southern_hemisphere[xlbl],Southern_hemisphere[ylbl],xlbl,ylbl,lblpos,13)
plt.savefig("output_data/Linear_Regresion_plot")<jupyter_output>The r-squared is: -0.31755609938002033
{'fraction_non_alphanumeric': 0.12581496429680222, 'fraction_numerical': 0.026544551381558524, 'mean_word_length': 4.8197831978319785, 'pattern_counts': {'":': 0, '<': 65, '<?xml version=': 0, '>': 69, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '18', 'dir': 'jupyter-structured-clean-dedup', 'id': '440423', 'license': 'ADSL', 'n_tokens_mistral': 4759, 'n_tokens_neox': 4308, 'n_words': 1023, 'path': 'WeatherPy.ipynb', 'repo_name': 'studam/Python_APIs', 'size': '12884'}