<jupyter_start><jupyter_text>**AUTHOR : <NAME>** Data visualization and analysis of temperature dataset 1. Compute the normalized histogram of Tmin and Tmax for the above data. Compute andplot cumulative distribution function (cdf) from the histograms.2. Calculate Pearson correlation coefficient (r) between Tmax and Tmin .3. Interpret the r2value you got.4. Plot the scatter plot from of Tmax → Tmin.5. Now plot the smoothed scatter plot without sorting and observe the plot.6. Plot smoothed scatter plot with sorting and observe the difference with the plot of above.State how smoothed scatter plot is in some ways better than usual scatter plot. Importing libraries and datset<jupyter_code>import matplotlib.pyplot as plt
import pandas as pd
import numpy as np 
import scipy.stats as st

data = pd.read_csv('https://raw.githubusercontent.com/Piyush9323/Data-Analysis-and-visualization/main/Adelaide%20Temperature.csv')
TMAX = data['TMAX']
TMIN = data['TMIN']
data.head(5)
#print(len(TMAX))
#print(len(TMIN))<jupyter_output><empty_output><jupyter_text>**Detecting unit of temperature :**Mostly temperature datapoints are ranging from 100 to 400 except the outliners.So we can conclude that the temperature unit is:   **KELVIN** ** DATA CLEANING :**  Detecting and Discarding the inconsistencies in the data. **DETECTION :**  In kelvin scale , the negative values of temperature are not practicle. these observations are inconsistent with the dataset.**DISCARDING :** I am discarding them by setting their values to 0.<jupyter_code># This function detects negative temp. observations and # set them to 0.
def discard_error(A,B):
  for i in range(len(A)) :
    if A[i]<0 :
      B.append((i,A[i]))
      A[i] = 0        

C = []
D = []
discard_error(TMAX,C)   
discard_error(TMIN,D)
#print(f"The following are detected faulty observations in the provided TMAX data.\n",C)
#print(f"The following are detected faulty observations in the provided TMIN data.\n",D)
print(f"There are {len(C)} inconsistent data points in the TMAX data.")
print(f"There are {len(D)} inconsistent data points in the TMIN data.")
# Finding the limits of observations in TMAX and TMIN.
print(max(TMAX), max(TMIN), min(TMAX), min(TMIN))<jupyter_output>464 315 0 0<jupyter_text>PLOTTING THE NORMALIZED HISTOGRAMS FOR TMAX AND TMIN.<jupyter_code>bins = [0,50,100,150,200,250,300,350,400,450,500]
fig, (ax1, ax2) = plt.subplots(1,2, figsize = (14,4))

counts_1, bins_1, _ = ax1.hist(TMAX, bins = bins,
   density = True, edgecolor = 'black') 
ax1.set_xlabel('TMAX')
ax1.set_title('Max. Temperature')

counts_2, bins_2, _ = ax2.hist(TMIN, bins = bins, 
   density = True, edgecolor = 'black')
ax2.set_xlabel('TMIN')
ax2.set_title('Min. Temperature')
plt.show()<jupyter_output><empty_output><jupyter_text>**PLOTTING THE PDF AND CDF FROM THE HISTROGRAMS.**<jupyter_code>pdf_TMAX = counts_1/(sum(counts_1))
pdf_TMIN = counts_2/(sum(counts_2))
#print(pdf_TMAX)
#print(bins_1)
#compute cdf
cdf_TMAX = np.cumsum(pdf_TMAX)
cdf_TMIN = np.cumsum(pdf_TMIN)
fig, (ax1, ax2) = plt.subplots(1,2, figsize = (14,4))
ax1.plot(bins_1[1:],pdf_TMAX, label = 'pdf_TMAX')
ax1.plot(bins_1[1:],cdf_TMAX, label = 'cdf_TMAX')
ax1.set_xlabel('TMAX')
ax1.set_title('Max. Temperature')
ax2.plot(bins_2[1:],pdf_TMIN, label = 'pdf_TMIN')
ax2.plot(bins_2[1:],cdf_TMIN, label = 'cdf_TMIN')
ax2.set_xlabel('TMIN')
ax2.set_title('Min. Temperature')
plt.legend()
plt.show()<jupyter_output><empty_output><jupyter_text>CALCULATION of Pearson correlation coefficient (r) between Tmax and Tmin .<jupyter_code>r,s = st.pearsonr(TMAX,TMIN)
#data.corr()
print("The pearson coorelation coefficient is : ", r)
x = r**2
print("The value of r square is = : ","{:.12f}".format(x))<jupyter_output>The pearson coorelation coefficient is :  0.6872063457523527
The value of r square is = :  0.472252561642<jupyter_text>**Interpreting the r square value** :We are getting r square value approx. = 0.47"It shows a medium level positive linear relationship between TMAX and TMIN." **PLOTTING THE SCATTER PLOT (TMAX TO TMIN).**<jupyter_code>plt.style.use('seaborn')
plt.scatter(TMIN, TMAX) 
#data.plot.scatter(x = 'TMIN', y = 'TMAX')
# To show the plot 
#plt.show()<jupyter_output><empty_output><jupyter_text>**PLOTTING THE SMOOTHED SCATTER PLOT WITHOUT SORTING.**<jupyter_code># Slicing the data into deciles.
TMIN_deciles = np.array_split(TMIN, 10)
TMAX_deciles = np.array_split(TMAX, 10)
#print(len(TMIN_deciles))
sm_x = []
sm_y = []
# Computing the central tendencies
for i in range(10):
  sm_x.append(np.mean(TMIN_deciles[i]))
  sm_y.append(np.mean(TMAX_deciles[i]))

#print(sm_x,"\n",sm_y)
plt.style.use('seaborn')
plt.scatter(sm_x, sm_y) 
plt.plot(sm_x,sm_y)
plt.show()<jupyter_output><empty_output><jupyter_text>**PLOTTING THE SMOOTHED SCATTER PLOT WITH SORTING.**<jupyter_code>plt.style.use('seaborn')
plt.scatter(np.sort(sm_x), np.sort(sm_y)) 
plt.plot(np.sort(sm_x),np.sort(sm_y))
plt.show()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.13078633515261776, 'fraction_numerical': 0.03011926420052557, 'mean_word_length': 5.949438202247191, 'pattern_counts': {'":': 0, '<': 33, '<?xml version=': 0, '>': 32, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '8', 'dir': 'jupyter-structured-clean-dedup', 'id': '170610', 'license': 'MIT', 'n_tokens_mistral': 1978, 'n_tokens_neox': 1832, 'n_words': 479, 'path': 'Data_analysis_and_visualization_Temperature_Dataset.ipynb', 'repo_name': 'Piyush9323/Data-Analysis-and-visualization', 'size': '4954'}