<jupyter_start><jupyter_text>IOT Demo Notebook Topics covered1. Reading data physically from serial port 2. Reading data via bluetooth3. Using APIs for getting data via requests4. Introduction to numpy     1. Numpy arrays basics    2. Numpy Functions    3. Coding perceptrons using numpy (***)5. Introduction to pandas    1. Reading xlsx , csv datafiles using pandas    2. Slicing and indexing using pandas    3. Processing data using pandas         1. Fining NaN datapoints        2. Dropping Nan points        3. Finding out statistical parameters of the dataset {mean , median , ...}6. Introduction to matplotlib   1. Plotting data using plot()   2. Plotting data using scatter()   3. Plotting data using hist()   4. How to decide which plot function is needed ?   5. Adding subplots and adjusting layouts   6. Adding labels , title and grid   7. Introduction to regression and classification models on sklearn    1. What is regression ?   2. Linear regression using sklearn   3. Poly regression using sklearn   4. Support Vector Regression using sklearn   5. What is classification ?   6. Suppot Vector Machine(linear and RBF kernel) for classification problems  Importing the required libraries<jupyter_code>import serial
import matplotlib.pyplot as plt
import requests 
import json
import pandas as pd
import numpy as np
import sklearn
import numpy
from sklearn.tree import DecisionTreeRegressor
from sklearn.linear_model import LogisticRegression<jupyter_output><empty_output><jupyter_text>Reading data from Serial Port<jupyter_code>port = serial.Serial('COM6' , 9600)
port.readline().decode('utf-8')[:3]<jupyter_output><empty_output><jupyter_text>File for datalogging<jupyter_code>log = open("data_det.csv" , "w")<jupyter_output><empty_output><jupyter_text>Getting data via requests<jupyter_code>import requests

url = "https://weatherapi-com.p.rapidapi.com/forecast.json"

querystring = {"q":"Hyderabad , India","days":"1"}

headers = {
    'x-rapidapi-key': "<KEY>",
    'x-rapidapi-host': "weatherapi-com.p.rapidapi.com"
    }

response = requests.request("GET", url, headers=headers, params=querystring)

import json
data_weth = json.loads(response.content.decode('utf-8'))<jupyter_output><empty_output><jupyter_text>Creating the dataset<jupyter_code>while True:
    data = port.readline().decode('utf-8')[:3]
    log.write(data)
    log.write(",")
    log.write(str(data_weth["current"]["temp_c"]))
    log.write(",")
    log.write((data_weth["current"]["wind_kph"]))
    log.write(",")
    log.write((data_weth["current"]["humidity"]))
    log.write(",")
    log.write((data_weth["current"]["condition"]["text"]) + "\r\n")<jupyter_output><empty_output><jupyter_text>Reading the data<jupyter_code>dataset = pd.read_csv("data_det.csv")
dat = dataset.dropna() #Dropping NaN points
dat
dat.isna() #Checking for NaN points after transformation<jupyter_output><empty_output><jupyter_text>Analysing data from a weather station  Steps for making machine learning models:1. Collect the data from source2. Clean the data i.e    1. Drop all NaN values as we cannot apply statistical models on Nan or missing data    2. If the data values are too large look for transformations    3. If needed apply normalization4. Visualize the data and decide the type of problem i.e regression , classification , clustering , ...5. Engineer the feature vectors6. Identify the target vector7. Decide the model to be used8. Apply data pre-processing needed9. Train the model10. Look at evaluation parameters like confusion matrices or R2 scores11. If the required performance is achieved export the model to your application and use the inference in the application<jupyter_code>data = pd.read_csv("feed.csv")
new_dat = data.drop(columns=['created_at' , 'entry_id'])
plt.plot(data["field1"])
plt.grid("on")
plt.xlabel("Sample number -->")
plt.ylabel("Temperature -->")
plt.title("Temperature plot")
plt.plot(data["field2"])
plt.grid("on")
plt.xlabel("Sample number -->")
plt.ylabel("Humidity -->")
plt.title("Humidity plot")
target = []
for i in range(len(data["field1"])):
    if data["field1"][i] > 32.4:
        target.append("Hot")
    else:
        target.append("OK")
        
target1 = []
for i in range(len(data["field1"])):
    if data["field1"][i] > 32.4:
        target1.append(1)
    else:
        target1.append(0)
data["field1"]
data = data.drop(columns = ["created_at" , "entry_id"])
data
data["Target"] = target
data
#Classification problem Hot or Normal ?
featurevector = np.concatenate((np.array(data["field1"]).reshape(-1,1) , np.array(data["field2"]).reshape(-1,1)) , axis = 1)
featurevector[:10]
targetvector = np.array(data["Target"]).reshape(-1,1)
target1vector = np.array(target1).reshape(-1,1)
target1vector[:10]
t = targetvector.reshape(-1,1)
t.shape
#Classifier
model = LogisticRegression()
model.fit(featurevector , target1vector)
model.predict(np.array([32.8, 43.2]).reshape(1 , -1))
plt.plot(target1vector)
plt.plot(model.predict(featurevector))<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.1252003205128205, 'fraction_numerical': 0.02063301282051282, 'mean_word_length': 4.860328638497653, 'pattern_counts': {'":': 2, '<': 30, '<?xml version=': 0, '>': 36, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '7', 'dir': 'jupyter-structured-clean-dedup', 'id': '379879', 'license': 'Apache-2.0', 'n_tokens_mistral': 1620, 'n_tokens_neox': 1562, 'n_words': 454, 'path': 'Requests_101/IOT_Demo.ipynb', 'repo_name': 'srimanthtenneti/IOT_Demo', 'size': '5037'}