<jupyter_start><jupyter_text>Importing libraries<jupyter_code>import matplotlib.pyplot as plt
import pandas as pd
import numpy as np
import math
from scipy.stats import kde

%matplotlib inline<jupyter_output><empty_output><jupyter_text>Task 1<jupyter_code>data = pd.read_csv("fire_nrt_V1_96617.csv")
proportion = 0.10
data.head()
x = data["longitude"]
y = data["latitude"]

num_samples = math.floor(proportion * len(x))
idx = np.random.choice(np.arange(len(x)), num_samples)

x_subsample = x[idx]
y_subsample = y[idx]
fig, axes = plt.subplots(2, 4 , figsize=(20, 10))
# axes.titlesize : "x-large"   
# axes.labelsize : "large"  
axes[0,0].plot(x, y, 'o')
axes[1,0].plot(x_subsample, y_subsample, 'o')

axes[0,1].scatter(x, y, alpha = 0.01)
axes[1,1].scatter(x_subsample, y_subsample, alpha = 0.01)

axes[0,2].scatter(x, y, s = 3, alpha = 0.1)
axes[1,2].scatter(x_subsample, y_subsample, s = 3, alpha = 0.1)

im = axes[0,3].hexbin(x,y, bins = "log")
im = axes[1,3].hexbin(x_subsample,y_subsample, bins = "log")

axes[0,0].set_xlabel('Longitude')
axes[0,0].set_ylabel('Latitude')
axes[0,0].set_title("Scatter plot with matplotlib defaults")
axes[1,0].set_xlabel('Longitude')
axes[1,0].set_ylabel('Latitude')
axes[1,0].set_title("Scatter plot with matplotlib defaults \n (Subsample of 0.10)")

axes[0,1].set_xlabel('Longitude')
axes[0,1].set_ylabel('Latitude')
axes[0,1].set_title("Scatter plot with alpha = 0.01")
axes[1,1].set_xlabel('Longitude')
axes[1,1].set_ylabel('Latitude')
axes[1,1].set_title("Scatter plot with alpha = 0.01 \n (Subsample of 0.10)")

axes[0,2].set_xlabel('Longitude')
axes[0,2].set_ylabel('Latitude')
axes[0,2].set_title("Scatter plot with (s) = 3 and alpha = 0.1")
axes[1,2].set_xlabel('Longitude')
axes[1,2].set_ylabel('Latitude')
axes[1,2].set_title("Scatter plot with (s) = 3 and alpha = 0.1 \n (Subsample of 0.10)")

axes[0,3].set_xlabel('Longitude')
axes[0,3].set_ylabel('Latitude')
axes[0,3].set_title("Hexbin Plot")
axes[1,3].set_xlabel('Longitude')
axes[1,3].set_ylabel('Latitude')
axes[1,3].set_title("Hexbin Plot \n (Subsample of 0.10)")

plt.tight_layout()
fig.colorbar(im, ax=axes.ravel().tolist())
plt.show()<jupyter_output><empty_output><jupyter_text>- From the plots it can be seen that the anomalies are spread out across the borders and aren't located in the central area of the continent. The scatter plot with specified alpha value also helps in determining that the central part of the continent has fewer anomalies. - The Heatmap further shows that the anomalies in the bottom-right (i.e) the South-East is having higher concentration for the anomalies.- The anomalies seem to be spreading out from the borders towards the central part of the continent. - The plots in general show the prominanace of anomalies in the South-East region and somewhat on the top-left (i.e) the North-west part of the continent.  Task 2<jupyter_code>temp_i4 = data["bright_ti4"]
plt.hist(temp_i4,bins=50)
plt.xlabel('Brightness temperature I-4')
plt.ylabel('Count')
plt.show()
temp = temp_i4[data["bright_ti4"]>=360]
unique_temp_i4 = temp.unique()[0]
print(unique_temp_i4)<jupyter_output>367.0<jupyter_text>- As visible in the histogram, the point at 367 is having a peek and hence we can identify that 367 is the value of Brightness temperature I-4 which corresponds to the value which is saturated as visible from the histogram.<jupyter_code>temp_i4_saturated = temp_i4[data["bright_ti4"] == unique_temp_i4]
temp_i4_not_saturated = temp_i4[data["bright_ti4"] != unique_temp_i4]
data_saturated = data.loc[data["bright_ti4"].isin(temp_i4_saturated)]
data_not_saturated = data.loc[data["bright_ti4"].isin(temp_i4_not_saturated)]
long_saturated = data_saturated["longitude"]
lat_saturated = data_saturated["latitude"]

long_not_saturated = data_not_saturated["longitude"]
lat_not_saturated = data_not_saturated["latitude"]
fig, ax = plt.subplots(1, 2, figsize=(16, 4))
ax[0].scatter(long_saturated, lat_saturated, s=3, alpha = 0.1, color = "red")
ax[1].scatter(long_not_saturated, lat_not_saturated, s=3, alpha = 0.1, color = "blue")

ax[0].set_xlabel('Longitude')
ax[0].set_ylabel('Latitude')

ax[1].set_xlabel('Longitude')
ax[1].set_ylabel('Latitude')

ax[0].set_title("Longitude-Latitude points corresponding to Saturated brightness temperature       ")
ax[1].set_title("      Longitude-Latitude points corresponding to Non-Saturated brightness temperature")

plt.show()<jupyter_output><empty_output><jupyter_text>The above two plots differentiate the points having Saturated Brightness temperature (i.e) the one we are certain of a fire from the non-saturated ones.The difference that are visible from the above plots - The non-saturated brightness points seems to be more concentrated in the South-East Region.- The concentration of non-saturated points in general is more than that of the saturated ones. However, the concentration of the saturated ones seems to be more in the western side of the central part of the continent.<jupyter_code>fig, ax = plt.subplots(1, 2, figsize=(16, 4))

ax[0].scatter(long_saturated, lat_saturated, s=5, alpha = 0.01, color = "red" , label = "Sarturated")
ax[0].scatter(long_not_saturated, lat_not_saturated, s=5, alpha = 0.01, color = "blue", label = "Not Saturated")
leg = ax[0].legend()
ax[0].set_xlabel('Longitude')
ax[0].set_ylabel('Latitude')

for lh in leg.legendHandles: 
    lh.set_alpha(1)

ax[1].scatter(long_not_saturated, lat_not_saturated, s=5, alpha = 0.01, color = "blue", label = "Not Saturated")
ax[1].scatter(long_saturated, lat_saturated, s=5, alpha = 0.01, color = "red" , label = "Sarturated")
leg = ax[1].legend()
ax[1].set_xlabel('Longitude')
ax[1].set_ylabel('Latitude')

ax[0].set_title("Latitude vs Longitude Scatter plot with Non-Saturated over Saturated points")
ax[1].set_title("Latitude vs Longitude Scatter plot with Saturated over Non-Saturated points")
for lh in leg.legendHandles: 
    lh.set_alpha(1)
plt.show()<jupyter_output><empty_output><jupyter_text>The two subplots formed by interchanging the order of plotting impacts the results a lot and it really is dependent on the order of plotting to get the information out from the plots. - When the saturated were plotted before non-saturated ones they were getting overlapped and it was not quite possible to differentiate the plots. When the order was changed the points came out and it was possible to notice that the non-saturated points are surrounding the saturated ones. Thus, it can be said that the saturated ones might be the epicenter for the fire points which spread out and become non saturated as it further propogates.<jupyter_code>nbins = 50
x,y = long_saturated, lat_saturated
k = kde.gaussian_kde([x,y])
xi, yi = np.mgrid[x.min():x.max():nbins*1j, y.min():y.max():nbins*1j]
zi = k(np.vstack([xi.flatten(), yi.flatten()]))

xj,yj = long_not_saturated, lat_not_saturated
kj = kde.gaussian_kde([xj,yj])
xii, yii = np.mgrid[xj.min():xj.max():nbins*1j, yj.min():yj.max():nbins*1j]
zii = kj(np.vstack([xii.flatten(), yii.flatten()]))

fig, ax = plt.subplots(1, 2, figsize=(16, 4))

ax[0].contour(xi, yi, zi.reshape(xi.shape))
ax[0].set_xlabel('Longitude')
ax[0].set_ylabel('Latitude')
 
ax[1].contour(xii, yii, zii.reshape(xii.shape))
ax[1].set_xlabel('Longitude')
ax[1].set_ylabel('Latitude')

ax[0].set_title("Longitude-Latitude points corresponding to Saturated brightness temperature      ")
ax[1].set_title("       Longitude-Latitude points corresponding to Non-Saturated brightness temperature")

plt.show()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.13589128697042366, 'fraction_numerical': 0.030242472688515855, 'mean_word_length': 6.204414587332054, 'pattern_counts': {'":': 0, '<': 24, '<?xml version=': 0, '>': 25, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '6', 'dir': 'jupyter-structured-clean-dedup', 'id': '357075', 'license': 'MIT', 'n_tokens_mistral': 2920, 'n_tokens_neox': 2679, 'n_words': 706, 'path': 'Matplotlib Visualizations/pds2136_HW1_submission.ipynb', 'repo_name': 'Prasham8897/Applied_Machine_Learning', 'size': '7506'}