<jupyter_start><jupyter_text>Dataset Anaylsis<jupyter_code>%%html
<style>
body {
    font-family: "Apple Script", cursive, sans-serif;
}
</style><jupyter_output><empty_output><jupyter_text>_importing necessary libraries of Data Science_<jupyter_code>import numpy as np
import pandas as pd
import cv2
from matplotlib import pyplot as plt
import os<jupyter_output><empty_output><jupyter_text>_making a function for showing image through open cv_<jupyter_code>def imshow(img):
    cv2.imshow('image',img)
    cv2.waitKey(0)
    cv2.destroyAllWindows()<jupyter_output><empty_output><jupyter_text>_________ _read one data out of the dataset and viewing the image_<jupyter_code># reading data using opencv
img = cv2.imread(r"PatientSpiral\sp1-P1.jpg")
grey_img = cv2.imread(r"PatientSpiral\sp1-P1.jpg", cv2.IMREAD_GRAYSCALE)
# viewing data using matplotlib
plt.imshow(img)
# plt.imshow(grey_img)<jupyter_output><empty_output><jupyter_text>_As we can see from above image, there is two diffrent pattern. one is question and one is answer, and they both overlap each other, we need to decompose them into __question__ and __answer__ to get accurate comparision across the dataset._<jupyter_code>Q_img = np.zeros(np.shape(grey_img))
A_img = np.zeros(np.shape(grey_img))<jupyter_output><empty_output><jupyter_text>extracting question Setting bias as 30 for black colour<jupyter_code>for i,e in enumerate(grey_img):
    for j,f in enumerate(e):
        if f > 30:
            Q_img[i][j] = 255         
        else:
            Q_img[i][j] = 0<jupyter_output><empty_output><jupyter_text>If we consider that the darkness of shade is also a factor in diagnosing the Parkinson's disease, as it may reflect the muscle strength of the Person thendon't use the else statement in above code__Points to consider:__- This may increase the ambiguity as the lightning conditions do not remain constant during image capture- Muscle strength also depends on the person irrespective of disease (not the age,weight factor)- Variance of darkness due to age, height, weight factor will be handled later by the model- Camera quality may also increase ambiguity- Pen and paper quality also contribute to ambiguity, but has nothing to do with the disease itself.- not considering the darkness of ink may leave out variables that represent the deterioration of muscle control that is a significant part of Diagnosis<jupyter_code>imshow(Q_img)<jupyter_output><empty_output><jupyter_text>extracting answer Setting +ve bias as __60__ for _removing black colour_ and -ve bias as __160__ for _removing white noise_<jupyter_code>for i,e in enumerate(grey_img):
    for j,f in enumerate(e):
        if f < 60 or f > 160:
            A_img[i][j] = 255
        else:
            A_img[i][j] = 0<jupyter_output><empty_output><jupyter_text>Again don't use the else statement in the above code if considering darkness of ink as a variable<jupyter_code>imshow(A_img)
plt.imshow(img)
plt.imshow(Q_img)
plt.imshow(A_img)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.07656458055925433, 'fraction_numerical': 0.011318242343541944, 'mean_word_length': 5.208677685950414, 'pattern_counts': {'":': 0, '<': 40, '<?xml version=': 0, '>': 41, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 1, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '9', 'dir': 'jupyter-structured-clean-dedup', 'id': '3054', 'license': 'MIT', 'n_tokens_mistral': 1000, 'n_tokens_neox': 943, 'n_words': 322, 'path': 'Parkinsons/Dataset Analysis.ipynb', 'repo_name': 'jayanta-banik/DataScience', 'size': '3004'}