<jupyter_start><jupyter_text>WeatherPy---- Note* Instructions have been included for each segment. You do not have to follow them exactly, but they are included to help you think through the steps.<jupyter_code># Dependencies and Setup
import matplotlib.pyplot as plt
import pandas as pd
import numpy as np
import requests
import time
from scipy.stats import linregress

# Import API key
from api_keys import weather_api_key

# Incorporated citipy to determine city based on latitude and longitude
from citipy import citipy

# Output File (CSV)
output_data_file = "output_data/cities.csv"

# Range of latitudes and longitudes
lat_range = (-90, 90)
lng_range = (-180, 180)<jupyter_output><empty_output><jupyter_text>Generate Cities List<jupyter_code># List for holding lat_lngs and cities
lat_lngs = []
cities = []

# Create a set of random lat and lng combinations
lats = np.random.uniform(lat_range[0], lat_range[1], size=1500)
lngs = np.random.uniform(lng_range[0], lng_range[1], size=1500)
lat_lngs = zip(lats, lngs)

# Identify nearest city for each lat, lng combination
for lat_lng in lat_lngs:
    city = citipy.nearest_city(lat_lng[0], lat_lng[1]).city_name
    
    # If the city is unique, then add it to a our cities list
    if city not in cities:
        cities.append(city)

# Print the city count to confirm sufficient count
len(cities)<jupyter_output><empty_output><jupyter_text>Perform API Calls* Perform a weather check on each city using a series of successive API calls.* Include a print log of each city as it'sbeing processed (with the city number and city name).<jupyter_code># Save config information
url = "http://api.openweathermap.org/data/2.5/weather?"

# Set the unit type
units = "imperial"

# Counter to count the number of calls to API
count = 1 

# Counter to count the number of call sets
sets = 1

# Index for the dataframe 
index = 0

# Create empty lists
city_list = []
cloudiness = []
country = []
date = []
maxtemp = []
windspeed = []
lat = []
lng = []
humidity = [] 

# Get current time
t0 =time.time()

print(f'Beginning Data Retrieval \n ........................')

for city in cities:
    try:
        city1 = city.replace(" ", "%20")
        print(f'Processing Record {count} of Set {sets} | {city}' )
        query_url = f"{url}appid={weather_api_key}&units={units}&q={city1}"
        weather_response = requests.get(query_url)
        city_stats = weather_response.json()
        city_list.append(city_stats["name"])
        cloudiness.append(city_stats["clouds"]["all"])
        country.append(city_stats["sys"]["country"])
        date.append(city_stats["dt"])
        maxtemp.append(city_stats["main"]["temp_max"])
        windspeed.append(city_stats["wind"]["speed"])
        lat.append(city_stats["coord"]["lat"])
        lng.append(city_stats["coord"]["lon"])
        humidity.append(city_stats["main"]["humidity"])
        index = index + 1
    except KeyError:
        print('City not found. Skipping...')
    count = count+1
    
    # Loop to limit 50 API calls per minute. The OpenWeatherMap API limit is 60 calls/min
    if count == 51:
        t1=time.time()
        count = 1
        sets = sets + 1
        time.sleep(60-(t1-t0))
        t0=time.time()

print('-----------------------------')
print('Data Retrieval Complete')
print('-----------------------------')<jupyter_output>Beginning Data Retrieval 
 ........................
Processing Record 1 of Set 1 | kloulklubed
Processing Record 2 of Set 1 | semey
Processing Record 3 of Set 1 | mataura
Processing Record 4 of Set 1 | cape town
Processing Record 5 of Set 1 | kingaroy
Processing Record 6 of Set 1 | oktyabrskoye
Processing Record 7 of Set 1 | upernavik
Processing Record 8 of Set 1 | lavrentiya
Processing Record 9 of Set 1 | barentsburg
City not found. Skipping...
Processing Record 10 of Set 1 | georgetown
Processing Record 11 of Set 1 | puerto colombia
Processing Record 12 of Set 1 | vila franca do campo
Processing Record 13 of Set 1 | rikitea
Processing Record 14 of Set 1 | taolanaro
City not found. Skipping...
Processing Record 15 of Set 1 | hobart
Processing Record 16 of Set 1 | ushuaia
Processing Record 17 of Set 1 | new norfolk
Processing Record 18 of Set 1 | vao
Processing Record 19 of Set 1 | thompson
Processing Record 20 of Set 1 | port moresby
Processing Record 21 of Set 1 | qaanaaq
Processing [...]<jupyter_text>Convert Raw Data to DataFrame* Export the city data into a .csv.* Display the DataFrame<jupyter_code># Store data to DataFrame
WeatherPy_df = pd.DataFrame({"City": city_list,
                             "Lat": lat,
                             "Lng": lng,
                             "Max Temp": maxtemp,
                             "Humidity": humidity,
                             "Cloudiness": cloudiness,
                             "Wind Speed": windspeed,
                             "Country": country,
                             "Date": date})

# Save as cvs file
WeatherPy_df.to_csv("outputs/WeatherPy.csv", encoding='utf-8', index=False)
WeatherPy_df<jupyter_output><empty_output><jupyter_text>Inspect the data and remove the cities where the humidity > 100%.----Skip this step if there are no cities that have humidity > 100%.<jupyter_code># Condition if cleanning up dataframe is needed.
if WeatherPy_df['Humidity'].max() > 100:
    print('There is a city has Humidity value > 100%. Need to cleanup DataFrame.')
else:
    print('No humidity value > 100%. Move on to next step.')<jupyter_output>No humidity value > 100%. Move on to next step.<jupyter_text>Plotting the Data* Use proper labeling of the plots using plot titles (including date of analysis) and axes labels.* Save the plotted figures as .pngs.  Latitude vs. Temperature Plot<jupyter_code>plt.scatter(WeatherPy_df['Lat'], WeatherPy_df['Max Temp'],
            marker="o", facecolors="steelblue", edgecolors="black")
plt.title('City Latitude vs. Max Temperature (%s)' % time.strftime('%x'))
plt.xlabel('Latitude')
plt.ylabel('Max Temperature (F)')
plt.grid()
plt.savefig('outputs/LatVsTemp.png')
plt.show()<jupyter_output><empty_output><jupyter_text>* The plot above show the temperature as we go from South (-60) to North (80). The highest max temperature values are focus around the Equator (0)  Latitude vs. Humidity Plot<jupyter_code>plt.scatter(WeatherPy_df['Lat'], WeatherPy_df['Humidity'],
            marker="o", facecolors="steelblue", edgecolors="black")
plt.title('City Latitude vs. Humidity (%s)' % time.strftime('%x'))
plt.xlabel('Latitude')
plt.ylabel('Humidity (%)')
plt.grid()
plt.savefig('outputs/LatVsHumidity.png')
plt.show()<jupyter_output><empty_output><jupyter_text>* The plot above show the humidity as we go from South (-60) to North (80). The humidity values are spread out almost evenly with high values (mostly greater than 60%)  Latitude vs. Cloudiness Plot<jupyter_code>plt.scatter(WeatherPy_df['Lat'], WeatherPy_df['Cloudiness'],
            marker="o", facecolors="steelblue", edgecolors="black")
plt.title('City Latitude vs. Cloudiness (%s)' % time.strftime('%x'))
plt.xlabel('Latitude')
plt.ylabel('Cloudiness (%)')
plt.grid()
plt.savefig('outputs/LatVsCloudiness.png')
plt.show()<jupyter_output><empty_output><jupyter_text>* The plot above shows the cloudiness as we go from South (-60) to North (80). The lowest values focus in the latitude of -60 to -20 and 4 to 70, while the highest values focus in the latitude of -30 to 3 and 40 to 75  Latitude vs. Wind Speed Plot<jupyter_code>plt.scatter(WeatherPy_df['Lat'], WeatherPy_df['Wind Speed'],
            marker="o", facecolors="steelblue", edgecolors="black")
plt.title('City Latitude vs. Wind Speed (%s)' % time.strftime('%x'))
plt.xlabel('Latitude')
plt.ylabel('Wind Speed (mph)')
plt.grid()
plt.savefig('outputs/LatVsWindSpeed.png')
plt.show()<jupyter_output><empty_output><jupyter_text>* The plot above show the wind speed as we go from South (-60) to North (80). The values are spread out evenly with the values mostly below 7.5 mph  Linear Regression<jupyter_code># Create 2 DataFrame for each hemispheres
north_df = WeatherPy_df[WeatherPy_df.Lat >= 0]
south_df = WeatherPy_df[WeatherPy_df.Lat < 0]

north_df
south_df<jupyter_output><empty_output><jupyter_text>Northern Hemisphere - Max Temp vs. Latitude Linear Regression<jupyter_code>x_values = north_df['Lat']
y_values = north_df['Max Temp']
(slope, intercept, rvalue, pvalue, stderr) = linregress(x_values, y_values)
regress_values = x_values * slope + intercept
line_eq = "y = " + str(round(slope,2)) + "x + " + str(round(intercept,2))
plt.scatter(x_values,y_values)
plt.plot(x_values,regress_values,"r-")
plt.annotate(line_eq,(5,-20),fontsize=15,color="red")
plt.xlabel('Latitude')
plt.ylabel('Max Temp')
print(f"The r-squared is: {rvalue**2}")
plt.show()<jupyter_output>The r-squared is: 0.7583108964162665<jupyter_text>Southern Hemisphere - Max Temp vs. Latitude Linear Regression<jupyter_code>x_values = south_df['Lat']
y_values = south_df['Max Temp']
(slope, intercept, rvalue, pvalue, stderr) = linregress(x_values, y_values)
regress_values = x_values * slope + intercept
line_eq = "y = " + str(round(slope,2)) + "x + " + str(round(intercept,2))
plt.scatter(x_values,y_values)
plt.plot(x_values,regress_values,"r-")
plt.annotate(line_eq,(-30,55),fontsize=15,color="red")
plt.xlabel('Latitude')
plt.ylabel('Max Temp')
print(f"The r-squared is: {rvalue**2}")
plt.show()<jupyter_output>The r-squared is: 0.3239042932630443<jupyter_text>* Comparing between the two plots above, the data shows to have higher value around the Latitude 0 or the Equator   Northern Hemisphere - Humidity (%) vs. Latitude Linear Regression<jupyter_code>x_values = north_df['Lat']
y_values = north_df['Humidity']
(slope, intercept, rvalue, pvalue, stderr) = linregress(x_values, y_values)
regress_values = x_values * slope + intercept
line_eq = "y = " + str(round(slope,2)) + "x + " + str(round(intercept,2))
plt.scatter(x_values,y_values)
plt.plot(x_values,regress_values,"r-")
plt.annotate(line_eq,(40,20),fontsize=15,color="red")
plt.xlabel('Latitude')
plt.ylabel('Humidity')
print(f"The r-squared is: {rvalue**2}")
plt.show()<jupyter_output>The r-squared is: 0.13511375201746273<jupyter_text>Southern Hemisphere - Humidity (%) vs. Latitude Linear Regression<jupyter_code>x_values = south_df['Lat']
y_values = south_df['Humidity']
(slope, intercept, rvalue, pvalue, stderr) = linregress(x_values, y_values)
regress_values = x_values * slope + intercept
line_eq = "y = " + str(round(slope,2)) + "x + " + str(round(intercept,2))
plt.scatter(x_values,y_values)
plt.plot(x_values,regress_values,"r-")
plt.annotate(line_eq,(-50,20),fontsize=15,color="red")
plt.xlabel('Latitude')
plt.ylabel('Humidity')
print(f"The r-squared is: {rvalue**2}")
plt.show()<jupyter_output>The r-squared is: 0.10138828578413896<jupyter_text>* There is more data in the Northern Hemisphere, therefore, more cities in the North are more humid than the South   Northern Hemisphere - Cloudiness (%) vs. Latitude Linear Regression<jupyter_code>x_values = north_df['Lat']
y_values = north_df['Cloudiness']
(slope, intercept, rvalue, pvalue, stderr) = linregress(x_values, y_values)
regress_values = x_values * slope + intercept
line_eq = "y = " + str(round(slope,2)) + "x + " + str(round(intercept,2))
plt.scatter(x_values,y_values)
plt.plot(x_values,regress_values,"r-")
plt.annotate(line_eq,(40,25),fontsize=15,color="red")
plt.xlabel('Latitude')
plt.ylabel('Cloudiness')
print(f"The r-squared is: {rvalue**2}")
plt.show()<jupyter_output>The r-squared is: 0.07006977329267004<jupyter_text>Southern Hemisphere - Cloudiness (%) vs. Latitude Linear Regression<jupyter_code>x_values = south_df['Lat']
y_values = south_df['Cloudiness']
(slope, intercept, rvalue, pvalue, stderr) = linregress(x_values, y_values)
regress_values = x_values * slope + intercept
line_eq = "y = " + str(round(slope,2)) + "x + " + str(round(intercept,2))
plt.scatter(x_values,y_values)
plt.plot(x_values,regress_values,"r-")
plt.annotate(line_eq,(-55,55),fontsize=15,color="red")
plt.xlabel('Latitude')
plt.ylabel('Cloudiness')
print(f"The r-squared is: {rvalue**2}")
plt.show()<jupyter_output>The r-squared is: 0.06853462548654811<jupyter_text>* The data on both plot is spread out evenly and no correlation bewteen the two plot   Northern Hemisphere - Wind Speed (mph) vs. Latitude Linear Regression<jupyter_code>x_values = north_df['Lat']
y_values = north_df['Wind Speed']
(slope, intercept, rvalue, pvalue, stderr) = linregress(x_values, y_values)
regress_values = x_values * slope + intercept
line_eq = "y = " + str(round(slope,2)) + "x + " + str(round(intercept,2))
plt.scatter(x_values,y_values)
plt.plot(x_values,regress_values,"r-")
plt.annotate(line_eq,(0,25),fontsize=15,color="red")
plt.xlabel('Latitude')
plt.ylabel('Wind Speed')
print(f"The r-squared is: {rvalue**2}")
plt.show()<jupyter_output>The r-squared is: 0.016006128855772617<jupyter_text>Southern Hemisphere - Wind Speed (mph) vs. Latitude Linear Regression<jupyter_code>x_values = south_df['Lat']
y_values = south_df['Wind Speed']
(slope, intercept, rvalue, pvalue, stderr) = linregress(x_values, y_values)
regress_values = x_values * slope + intercept
line_eq = "y = " + str(round(slope,2)) + "x + " + str(round(intercept,2))
plt.scatter(x_values,y_values)
plt.plot(x_values,regress_values,"r-")
plt.annotate(line_eq,(-55,15),fontsize=15,color="red")
plt.xlabel('Latitude')
plt.ylabel('Wind Speed')
print(f"The r-squared is: {rvalue**2}")
plt.show()<jupyter_output>The r-squared is: 0.0662299050094148
{'fraction_non_alphanumeric': 0.13588850174216027, 'fraction_numerical': 0.02772629550003707, 'mean_word_length': 4.720949957591179, 'pattern_counts': {'":': 9, '<': 64, '<?xml version=': 0, '>': 70, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '18', 'dir': 'jupyter-structured-clean-dedup', 'id': '216394', 'license': 'ADSL', 'n_tokens_mistral': 5020, 'n_tokens_neox': 4564, 'n_words': 1232, 'path': 'WeatherPy.ipynb', 'repo_name': 'tnguyen0306/python-api-challenge', 'size': '13489'}