<jupyter_start><jupyter_text>Tasks These are my solutions to the Tasks assessment. The author is <NAME>. (<EMAIL>). ---  Task 1: Calculating a square rootWe can calculate the square root of a number by using  Newton's method [1, 2].To find the square root $z$ of a number $x$, we can iterate using the following equation.$$ z_{next} = z - \frac{z^2 - x}{2z} $$[1] A Tour of Go; Excersize: Loops and Functions; https://tour.golang.org/flowcontrol/8 [2] Newton's method; https://en.wikipedia.org/wiki/Newton%27s_method[3] 1.14.3. String Formats for Float Precision; http://anh.cs.luc.edu/python/hands-on/3.1/handsonHtml/float.html<jupyter_code># importing libraries
import numpy as np
import matplotlib.pyplot as plt
# Better sized plots.
plt.rcParams['figure.figsize'] = (12, 8)
# Nicer colours and styles for plots.
plt.style.use("fivethirtyeight")
x = np.linspace(0.0, 100.0, 1000)
y = np.sqrt(x)
plt.plot(x,y)
# setting x-axis
plt.xlabel("X")
# setting y-axis
plt.ylabel("Y")
plt.title("square root of a number")


def sqrt(x):
    """"
    A function to calculate the square root of a number x.
    """
    # Initial guess for the square root z
    z = x 
    
    # precision, we want the precision to be to 100
    precision = 10**(-10)

    # Loop until we're happy with the accuracy.
    while abs(2 - (z * z)) > precision:
        # Calculate a better guess for the square root.
        z = (z + x / z) / 2
    # Return the (approximate) square root of x.
    return z<jupyter_output><empty_output><jupyter_text>**Tests of the function**<jupyter_code>sqrt(2)
# increasing the precision of the floating point by using string formats [3]
format(sqrt(2), ".100f")<jupyter_output><empty_output><jupyter_text>***  Task 2: The Chi-Square Test of Independence. Definition:The Chi-Square Test of Independence determines whether there is an association between categorical variables (i.e., whether the variables are independent or related). It is a nonparametric test.This test utilizes a contingency table to analyze the data. A contingency table (also known as a cross-tabulation, crosstab, or two-way table) is an arrangement in which data is classified according to two categorical variables. The categories for one variable appear in the rows, and the categories for the other variable appear in columns. Each variable must have two or more categories. Each cell reflects the total count of cases for a specific pair of categories. [3][1] towards data science: https://towardsdatascience.com/gentle-introduction-to-chi-square-test-for-independence-7182a7414a95[2] Wikipedia; Chi-squared test: https://en.wikipedia.org/wiki/Chi-squared_test[3] Kent State University: https://libguides.library.kent.edu/spss/chisquare:~:text=The%20Chi%2DSquare%20Test%20of%20Independence%20determines%20whether%20there%20is,Chi%2DSquare%20Test%20of%20Association.<jupyter_code># importing necessary libraries.
from scipy.stats import chi2_contingency
import numpy as np
import pandas as pd
# generating the data frame.
df = pd.DataFrame([
    
    [90, 60, 104, 95], [30, 50, 51, 20], [30, 40, 45, 35]
], 
    index = ["white collar", "blue collar", "no collar"],
    columns = ["A", "B", "C", "D"])
df

# columns in the data frame
df.columns
# rows in the data frame
df.index
chi2_contingency(df)
# Chisquare value
chisquare = chi2_contingency(df)[0]
chisquare
# p-value
pvalue = chi2_contingency(df)[1]
pvalue
# Degree of freedom
dof = chi2_contingency(df)[2]
dof<jupyter_output><empty_output><jupyter_text>---  Task 3: Standard Deviation __Definitions__:In `EXCEL`;**`STDEV.P`**: is a function used when data represents the entire population. In `STDEV.P` function, the squared deviation is divided by the total number of arguments, mostly represented as N.- The `STDEV.P` function considers entire data, and some factors may dominate the result standard deviation. And since it will be taken as the standard deviation for everyone in data, even for minorities, this is called **Biased Analysis**. This is why this standard deviation is recommended to use only when an analysis is non-destructive.- The `STDEV.P` is just a newer version of STDEVP and STDEV functions of excel. There is no significant difference between them.- The `STDEV.P` function only recognises numbers. Any other value is ignored.The equivalent calculation for the **`STDEV.P`** using **`NumPy`** is : __np.sqrt(np.sum((x - np.mean(x))**2)/len(x))__.**`STDEV.S`**: is a function used when  data is a sample of the entire population. In `STDEV.S` or `STDEV`, the squared deviation is divided by the total number of sample -1. It is represented as N-1.- The `STDEV.S` or `STDEV` function is used on a small sample of the entire population, and we subtract one from the denominator (number of samples arguments). This is called non-biassed analysis of standard deviation. This is used when an analysis is destructive. [1]The equivalent calculation for the `STDEV.S` using `NumPy` is : __np.sqrt(np.sum((x - np.mean(x))**2)/(len(x)-1))__.[1] EXCELTIP; https://www.exceltip.com/statistical-formulas/how-to-use-excel-stdev-p-function.html:~:text=Let's%20Explore.-,The%20STDEV.,sample%20of%20the%20entire%20population.[2] Wikipedia ; Standard Deviation : https://en.wikipedia.org/wiki/Standard_deviation[3] NumPy; numpy.std : https://numpy.org/doc/stable/reference/generated/numpy.std.html[4] seaborn: statistical data visualization; https://seaborn.pydata.org/<jupyter_code># importing libraries
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
# Better sized plots.
plt.rcParams['figure.figsize'] = (12, 8)
# Nicer colours and styles for plots.
plt.style.use("fivethirtyeight")

url = "https://raw.githubusercontent.com/ianmcloughlin/2020A-machstat-project/master/dataset/powerproduction.csv"
df = pd.read_csv(url, error_bad_lines=False)

df.transpose()
# we can notice that in the below functions that ddof (delta degrees of freedom) with ddof = 1 as a standard is applied
df.describe()
# Considering the data resembling the whole population ; np.sqrt(np.sum((x - np.mean(x))**2)/len(x)).
STDVP = np.std(df)
STDVP
# Considering the data resembling a SAMPLE of the whole population np.sqrt(np.sum((x - np.mean(x))**2)/(len(x)-1)).
STDVS = np.std(df, ddof = 1)
STDVS
sns.scatterplot(x="speed", y="power", data=df, ci = STDVS)
# Scatter plotting.
sns.scatterplot(x="speed", y="power", data=df, ci = STDVS)
ax = sns.barplot(x="speed", y="power", data=df, ci="sd")
ax<jupyter_output><empty_output><jupyter_text>---  Task 4: KNN Iris Data Set[1] DataCamp; KNN Classification using Scikit-learn: https://www.datacamp.com/community/tutorials/k-nearest-neighbor-classification-scikit-learn[2] Scikit-learn; Nearest Neighbors: https://scikit-learn.org/stable/modules/neighbors.htmlclassification[3] Stack Abuse; K-Nearest Neighbors Algorithm in Python and Scikit-Learn: https://stackabuse.com/k-nearest-neighbors-algorithm-in-python-and-scikit-learn/  _Libraries_<jupyter_code># importing libraries

# data frames
import pandas as pd

# Visualisation
import seaborn as sns

# Moddelling, Classification and Prediction
import sklearn.neighbors as nei
import sklearn.model_selection as mod<jupyter_output><empty_output><jupyter_text>_Data_<jupyter_code># load the iris data set from a URL.

df = pd.read_csv("https://raw.githubusercontent.com/G00387867/pands-project/master/iris.data")
variables =['sepal_length','sepal_width','petal_length','petal_width','class'] 
df = pd.read_csv("https://raw.githubusercontent.com/G00387867/pands-project/master/iris.data", names = variables)
df<jupyter_output><empty_output><jupyter_text>_Visualise_<jupyter_code># Plot the Iris data set with a pair plot using seaborn.
sns.pairplot(df, hue="class")<jupyter_output><empty_output><jupyter_text>_Inputs_ and _Outputs_<jupyter_code># Create inputs and outputs of the model.
inputs = df[['sepal_length', 'sepal_width', 'petal_length', 'petal_width']]
outputs = df['class']<jupyter_output><empty_output><jupyter_text>_Classification_<jupyter_code># Create the Classifier
knn = nei.KNeighborsClassifier(n_neighbors=5)<jupyter_output><empty_output><jupyter_text>_Fitting_<jupyter_code>knn.fit(inputs, outputs)<jupyter_output><empty_output><jupyter_text>_Prediction_<jupyter_code>df.loc[0]
df.iloc[0]
knn.predict([[5.1, 3.5, 1.4,0.2]])<jupyter_output><empty_output><jupyter_text>_Evaluation_Now we need to evaluate the accuracy of the prediction model and optimise it. In order to do so, we need to split the data set into `train` and `test`. The thought ratio for testing is **0.33** which is about a third of the total size of the data set.<jupyter_code># Checking the current accuracy by comparing inputs and outputs of the current model.
(knn.predict(inputs)==outputs).sum()
# Total count.
df["class"].count()
# creating the test sample.
inputs_train, inputs_test, outputs_train, outputs_test = mod.train_test_split(inputs, outputs, test_size=0.33)
knn = nei.KNeighborsClassifier(n_neighbors=5)
knn.fit(inputs_train, outputs_train)
# number of the test samples.
(knn.predict(inputs_test) == outputs_test).sum()
# Number of train samples.
(knn.predict(inputs_train) == outputs_train).sum()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.1191512513601741, 'fraction_numerical': 0.020674646354733407, 'mean_word_length': 5.286593707250342, 'pattern_counts': {'":': 0, '<': 51, '<?xml version=': 0, '>': 52, 'https://': 15, 'lorem ipsum': 0, 'www.': 2, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '12', 'dir': 'jupyter-structured-clean-dedup', 'id': '172401', 'license': 'MIT', 'n_tokens_mistral': 3206, 'n_tokens_neox': 2987, 'n_words': 935, 'path': 'Tasks.ipynb', 'repo_name': 'G00387867/MLS', 'size': '9200'}