<jupyter_start><jupyter_text>[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/Clinical-Informatics-Interest-Group/CLiC.notebooks/blob/main/notebooks/notebook1.ipynb)  1. Hospital Readmission DataAn original publication with the dataset we will explore investigated the impact of HbA1c measurement in hospital readmission rates by modeling the relationship with a multivariable logistic regression.The open dataset is provided by the University of California Irvine herehttps://archive.ics.uci.edu/ml/datasets/diabetes+130-us+hospitals+for+years+1999-2008The original publication of data sourcehttps://www.hindawi.com/journals/bmri/2014/781670/Publication authors description of the data.https://www.hindawi.com/journals/bmri/2014/781670/tab1/<jupyter_code># The first cell is reserved for importing useful libraries that we
# will need for machine learning and working with lots of data.
import pandas as pd
import numpy as np
import zipfile, requests, io #We'll use these to get and extract the data from its repository
from pathlib import Path
# Run this block to download the data into the working directory.
# In Colab, this is a directory on the cloud within your google drive.
# The 'get_data' function checks to see if the data has already been download
# and downloads it if not.
def get_data():
    if Path('./dataset_diabetes/diabetic_data.csv').is_file():
        pass
    else:
        r = requests.get('https://archive.ics.uci.edu/ml/machine-learning-databases/00296/dataset_diabetes.zip')
        z = zipfile.ZipFile(io.BytesIO(r.content))
        z.extractall()
        
get_data() 
# Assign the variable "df" to the pandas DataFrame containing our data.
df = pd.read_csv('./dataset_diabetes/diabetic_data.csv')
df.head() # Calling the first 5 rows of the data to have a first look
# Ask pandas if it knows any of the data samples are duplicates
df.duplicated().sum()
# See the class labels to decide if any na-values should be dropped
for col in df.columns[2:]:
    print(col, 'labels:', np.unique(df[col]), "\n")

# We see some of the data entries are '?'. Let's encode this so that python knows these are missing data. 
df = pd.read_csv('./dataset_diabetes/diabetic_data.csv', na_values=['?'])<jupyter_output>/home/justin/anaconda3/envs/dsmed/lib/python3.9/site-packages/IPython/core/interactiveshell.py:3165: DtypeWarning: Columns (10) have mixed types.Specify dtype option on import or set low_memory=False.
  has_raised = await self.run_ast_nodes(code_ast.body, cell_name,<jupyter_text>Last block threw an interesting error. We'll investigate this shortly with "df.dtypes()"<jupyter_code># Now we can find the number of samples missing data for each feature
df.isnull().sum()<jupyter_output><empty_output><jupyter_text>We already knew the data was missing some values. See https://www.hindawi.com/journals/bmri/2014/781670/tab1/We will have to consider whether to drop these feature columns all together. Just to be sure how many "samples" we have, let's check the total number of rows.<jupyter_code>len(df)
df.dtypes<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.09595154606311763, 'fraction_numerical': 0.01976410583359898, 'mean_word_length': 4.843575418994414, 'pattern_counts': {'":': 0, '<': 12, '<?xml version=': 0, '>': 12, 'https://': 7, 'lorem ipsum': 0, 'www.': 3, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 3, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '3', 'dir': 'jupyter-structured-clean-dedup', 'id': '650375', 'license': 'MIT', 'n_tokens_mistral': 1005, 'n_tokens_neox': 930, 'n_words': 322, 'path': 'notebooks/notebook1.ipynb', 'repo_name': 'Clinical-Informatics-Interest-Group/CLiC.notebooks', 'size': '3137'}