<jupyter_start><jupyter_text>1. Import the dataset iris_dirty.csv from here into a Pandas dataframe. Check the Pandas documentation of read_csv() and use the column names sepal_length, sepal_width, petal_length, petal_width and class when you read in the data.<jupyter_code>import pandas as pd
data = pd.read_csv('iris_dirty.csv',names= ['sepal_length', 'sepal_width', 'petal_length', 'petal_width','class'])<jupyter_output><empty_output><jupyter_text>2. Explore the data with the known Pandas functions head(), info(), describe(), count(), value_counts() (for all columns). Watch for missing values, duplicates, spelling errors, non-uniform units, columns with datatypes they shouldn't have etc.<jupyter_code>data.head()
data.info()
data.describe()
data.count()<jupyter_output><empty_output><jupyter_text>* sepal_width has a Null<jupyter_code>data.value_counts()
data['class'].unique()<jupyter_output><empty_output><jupyter_text>* Typo in class Iris-setsoa is Iris-setosa<jupyter_code>data['sepal_length'].unique()
data['sepal_length'].value_counts()<jupyter_output><empty_output><jupyter_text>* outlier 58<jupyter_code>data['sepal_width'].unique()
data['petal_width'].unique()<jupyter_output><empty_output><jupyter_text>* the unit is always mm, so can be deleted and the values turned into integer 3. Substitute missing values with the column mean. Check/remove duplicates. Correct possible spelling errors.<jupyter_code>#remove duplicates
data = data.drop_duplicates()
data.count()
#substitue missing values with column mean
data= data.fillna(data.mean())
data['sepal_width'].unique()
data.isna().sum()
#Correct possible spelling errors
def flower_spell(x):
    if x in ['Iris-setsoa']:
        return 'Iris-setosa'
    else:
        return x
data['class']=data['class'].apply(flower_spell)
data['class'].unique()<jupyter_output><empty_output><jupyter_text>4. Bring all columns to centimeter length. Before, transform any column to numeric if necessary.<jupyter_code>#first delete ' mm'
data['petal_width'] = data['petal_width'].map(lambda x: x.rstrip(' mm'))
#now transform to numeric
data['petal_width']=pd.to_numeric(data['petal_width'])
#all columns to centimeter length
data['petal_width']=(data['petal_width'])/10
data['petal_width'].unique()<jupyter_output><empty_output><jupyter_text>4. Check for outliers with .describe() or means of plotting. Think about the appropriate way to correct for possible outliers and perform that correction.<jupyter_code>data.describe()<jupyter_output><empty_output><jupyter_text>* the outlier is in sepal length. It can be assumed that it was a typo and is supposed to be 5.8cm and not 58cm.<jupyter_code>import matplotlib.pyplot as plt
import seaborn as sns
sns.displot(data['sepal_length'])
#correcting the outlier
def outlier_corr(x):
    if x in [58]:
        return 5.8
    else:
        return x
data['sepal_length']=data['sepal_length'].apply(outlier_corr)<jupyter_output><empty_output><jupyter_text>6. Plot a correlation matrix with a heatmap between the numeric features.<jupyter_code>corr_matrix = data.corr()
type(corr_matrix)
sns.heatmap(corr_matrix,
            cmap='seismic',
            center=0,
            annot=True)<jupyter_output><empty_output><jupyter_text>7. Model the relationship between our features (the four numeric flower metrics) and the label (class) with Logistic Regression using LogisticRegression() from the sklearn.linear_model module. For the purpose of model validation, use the known train-test-split method with a size parameter of 0.3. Calculate the predicted labels, the accuracy score of y_test and y_predicted and plot a confusion matrix of the two.<jupyter_code>#class sklearn.linear_model.LinearRegression(*, fit_intercept=True, normalize=False, copy_X=True, n_jobs=None, positive=False)[source]¶
import numpy as np
from sklearn.linear_model import LogisticRegression
X=data.select_dtypes(include =np.number)
y=data['class']
from sklearn.model_selection import train_test_split
X_train,X_test,y_train,y_test=train_test_split(X,y,test_size=0.3,random_state=0)
# create an (untrained) model "instance"
logreg = LogisticRegression(random_state=0, solver='lbfgs',max_iter=400,
                  multi_class='ovr')

# train the model
logreg.fit(X_train,y_train)

#predict target data from test features , that the model has never seen
y_pred=logreg.predict(X_test)
from sklearn.metrics import accuracy_score 
predictions = logreg.predict(X_test)
predictions
accuracy_score(y_test, predictions)
type(y_test)
from sklearn.metrics import confusion_matrix
from sklearn.metrics import plot_confusion_matrix
confusion_matrix= confusion_matrix(y_test,predictions,  labels=['Iris-virginica', 'Iris-versicolor', 'Iris-setosa'])

plot_confusion_matrix(y_test,y_pred)
plt.show()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.10853199498117942, 'fraction_numerical': 0.006064408197406943, 'mean_word_length': 6.840983606557377, 'pattern_counts': {'":': 0, '<': 45, '<?xml version=': 0, '>': 45, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '11', 'dir': 'jupyter-structured-clean-dedup', 'id': '440515', 'license': 'Apache-2.0', 'n_tokens_mistral': 1597, 'n_tokens_neox': 1537, 'n_words': 406, 'path': 'Predict_the_flower.ipynb', 'repo_name': 'IsabelJabs/Logistic_regression_flower_model', 'size': '4782'}