<jupyter_start><jupyter_text>Neural Networks Tutorial - the first step to Deep Learning  sigmoid functionA common activation function, **a non-linear function applied on top linear combination**, used in Machine Learning and Deep Learning is the **sigmoid function**, which has a "switch on" character, simulating the "turning on" feature of a biological neuron.The need for non-linearity comes from the fact, that connect neurons together and **the linear function on top of linear function is itself a linear function**. So, if didn’t have non-linear function applied in each neuron, the neural network would be a linear function, thus not more powerful than a single neuron. Want a number between 0 and 1 as an output from out neural network is treated as a probability.&nbsp;&nbsp;&nbsp;&nbsp;$sigmoid(x) = \frac{1}{1+e^{-x}}$The code written in Python looks like this:<jupyter_code>def sigmoid(x):
    return 1 / (1 + np.exp(-x))<jupyter_output><empty_output><jupyter_text>sigmoid function drawn by matplot<jupyter_code>import matplotlib.pylab as plt
import numpy as np

x = np.arange(-10, 10, 0.1)

plt.plot(x, sigmoid(x))

plt.xlabel('x')
plt.ylabel('sigmoid(x)')

plt.show()<jupyter_output><empty_output><jupyter_text>sigmoid function drawn by plotlyplotly configuration files under user HOME directory:```sh𝜆 cat .plotly/.config{    "plotly_domain": "https://plot.ly",    "plotly_streaming_domain": "stream.plot.ly",    "plotly_api_domain": "https://api.plot.ly",    "plotly_ssl_verification": true,    "plotly_proxy_authorization": false,    "world_readable": true,    "sharing": "public",    "auto_open": true}𝜆 cat .plotly/.credentials{    "username": "TerrenceMiao",    "api_key": "abracadbra",    "proxy_username": "",    "proxy_password": "",    "stream_ids": []}```<jupyter_code>import pandas as pd
import numpy as np
import scipy as sp
import plotly.plotly as py
import plotly.graph_objs as go

size = 50

trace = go.Scatter(
    x = [i for i in np.arange(-10, 10, 0.1)],
    y = sigmoid(x),
    mode = 'markers',
    name='sigmoid func',
    marker = dict(
        color = '#1E64F0',
        size = 2
    )
)
    
data = [trace]
    
py.iplot(data, filename='sigmoid')<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.1213898916967509, 'fraction_numerical': 0.01128158844765343, 'mean_word_length': 4.584382871536524, 'pattern_counts': {'":': 13, '<': 13, '<?xml version=': 0, '>': 13, 'https://': 2, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '3', 'dir': 'jupyter-structured-clean-dedup', 'id': '157014', 'license': 'MIT', 'n_tokens_mistral': 751, 'n_tokens_neox': 720, 'n_words': 236, 'path': 'notebooks/Neural Network Tutorial.ipynb', 'repo_name': 'TerrenceMiao/Data-Science-and-A.I.', 'size': '2216'}