<jupyter_start><jupyter_text>Introduction The first step in any machine learning endeavor is to get the raw data into our system.The raw data might be a logfile, dataset file, or database. Furthermore, often we willwant to retrieve data from multiple sources.  1. Loading a Sample Data<jupyter_code># we want to load preexisting sample dataset

## scikit-learn comes with number of popular dataset for you
## These datasets are often called “toy” datasets because they are far smaller and
## cleaner than a dataset we would see in the real world.

# Load scikit-learn's datasets
from sklearn import datasets
# Load digits dataset
digits = datasets.load_digits()
# Create Feature matrix
features = digits.data
# Create Target vector
target = digits.target
# View First Observation

## Note that each row called observation and each element in the observation
## considered as a single feature. specificly in this dataset each single feature
## represent a pixel and each observation represent 8x8 image
## which represent a single digit

features[0]
# View the dimentional of dataset
features.shape<jupyter_output><empty_output><jupyter_text>2. Creating a Simulated Dataset<jupyter_code>we need to generate a dataset of simulated data

scikit-learn offers many methods for creating simulated data.
Of those, three methods are particularly useful.
creating a simulated dataset to be used with linear regression ~> [make_regression]
creating a simulated dataset for classification ~> [make_classification]
creating a simulated dataset to work well with clustering techniques ~> [make_blobs]<jupyter_output><empty_output><jupyter_text>2.1 Generate Data For Linear Regression<jupyter_code># Generate a dataset to work with linear regression

# Load library
from sklearn.datasets import make_regression
# Generate feature matrix, target vector, and the row coefficients
features, target, coefficient = make_regression(n_samples     = 100,  # num of observations
                                                n_features    = 3,    # num of features
                                                n_informative = 3,    # num features that affect on the target
                                                n_targets     = 1,    # num labels (output column)
                                                noise         = 0.0,  # ratio of noise
                                                coef          = True, # If True, the coefficients of the underlying linear model are returned.
                                                random_state  = 1)    # generate same output every time run the same code
# View feature matrix
features[:3] # same as features[:3, :] and features[:3,]
# View target vector
target[:3]<jupyter_output><empty_output><jupyter_text>2.2 Generate Data For Classification<jupyter_code># Generate a dataset to work with classification

# Load library
from sklearn.datasets import make_classification
# Generate features matrix and target vector
features, target = make_classification(n_samples     = 100,
                                       n_features    = 3,
                                       n_informative = 3,
                                       n_redundant   = 0,          # These features are generated as random linear combinations of the informative features.
                                       n_classes     = 2,          # he number of classes (or labels) of the classification problem.
                                       weights       = [.25, .75], # The proportions of samples assigned to each class. this is imbalanced classes.
                                       random_state  = 1)

## If n_informative is less than the total number of features ( n_features ),
## the resulting dataset will have redundant features that can be
## identified through feature selection techniques.
# view feature matrix
features[:3]
# view target vector
# note that we have two classes (0, 1).
target[:3]<jupyter_output><empty_output><jupyter_text>2.3 Generating Data For Clustering<jupyter_code># Generate a dataset to work with clustering

# Load library
from sklearn.datasets import make_blobs
# Generate feature matrix and target vector
features, target = make_blobs(n_samples    = 100,
                              n_features   = 2,
                              centers      = 3,    # The number of centers to generate, or the fixed center locations.
                              cluster_std  = 0.5,  # The standard deviation of the clusters.
                              shuffle      = True, # Shuffle the samples
                              random_state = 1)

##  Shuffling data serves the purpose of reducing variance and making sure
## that models remain general and overfit less.
# View Features Matrix
features[:3]
# View Target Vector
target[:3]
# Visualize the clusters
import matplotlib.pyplot as plt
%matplotlib inline

plt.scatter(features[:,0], features[:,1], c=target) # scatter(x, y, c ~> color)
plt.show()<jupyter_output><empty_output><jupyter_text>3. Loading a Data from CSV File<jupyter_code># we need to import a comma-separated values (CSV) file.
# Use the pandas library’s read_csv to load a local or hosted CSV file.

# Load library
import pandas as pd

# load dataset
# we will load boston house price dataset
dataframe = pd.read_csv("housing.csv")

dataframe.head(5) # head, load first five observation<jupyter_output><empty_output><jupyter_text>4. Loading an Excel File<jupyter_code># we need to import an Excel spreadsheet.
# Use the pandas library’s read_excel to load an Excel spreadsheet.

## This solution is similar to our solution for reading CSV files. The main difference is
## the additional parameter, sheet_name , that specifies which sheet in the Excel file we
## wish to load. sheet_name can accept both strings containing the name of the sheet and
## integers pointing to sheet positions (zero-indexed).

# create url
url = "iris.xls"

# Load data
dataframe = pd.read_excel(url, sheet_name=0, header=1)

# show first five observations
dataframe.head(5)<jupyter_output><empty_output><jupyter_text>5. Loading From a JSON File<jupyter_code># we need to load a JSON file for data preprocessing.
# The pandas library provides read_json to convert a JSON file into a pandas object.

# create url
url = "dummy.json"

# Load Data
dataframe = pd.read_json(url, orient='columns')

# View first three observation
dataframe.head(3)<jupyter_output><empty_output><jupyter_text>6. Creating a SQL Database<jupyter_code># Before we can load SQL Database let us create one

# load library
import sqlite3

# create iris table
conn = sqlite3.connect("iris.db") # You can create a new database
                                  # by changing the name within the quotes.
    
c = conn.cursor() # The database will be saved in the location
                  # where your 'py' file is saved

    
# create table
c.execute('''CREATE TABLE IRIS
             ([Sepal Length (cm)] floate,
             [Sepal Width (cm)] floate,
             [Petal Length (cm)] floate,
             [Petal Width (cm)] floate,
             [Class] text)''')

conn.commit()
# Insert the values from the excel file into the table 'IRIS'
read_iris = pd.read_excel("iris.xls", sheet_name=0, header=1)
read_iris.to_sql('IRIS', conn, if_exists='append', index = False)
conn.close()<jupyter_output><empty_output><jupyter_text>7. Querying a SQL Database<jupyter_code># we need to load data from a database using the structured query language (SQL).
# pandas’ read_sql_query allows us to make a SQL query to a database and load it.

# load library
from sqlalchemy import create_engine
# create connection to database
database_connection = create_engine('sqlite:///iris.db')
# Load data
dataframe = pd.read_sql_query('SELECT * FROM IRIS', database_connection)
# View fist five columns
dataframe.head(5)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.06787847972543536, 'fraction_numerical': 0.00851658828015762, 'mean_word_length': 2.9557566616390147, 'pattern_counts': {'":': 0, '<': 41, '<?xml version=': 0, '>': 45, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 19, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '10', 'dir': 'jupyter-structured-clean-dedup', 'id': '101227', 'license': 'MIT', 'n_tokens_mistral': 2131, 'n_tokens_neox': 2024, 'n_words': 856, 'path': 'CH02-Loading_Data.ipynb', 'repo_name': 'ahmedmohamedeid98/MachineLearningPractice', 'size': '7867'}