<jupyter_start><jupyter_text>The perceptron The perceptron is (one of?) the very first algorithms for artificial neural networks, it was developed in 1957 at a [Cornell lab](https://en.wikipedia.org/wiki/Calspan) funded by the US Navy, by Rosenblatt, a psychologist who was a pioneer in the field of Artificial Intelligence, and published in [[1]](rosenblatt). It was immediately advertised with lots of fanfare and great expectations (the NY Times at that time published an article [[2]](nyt), boldly describing it as a machine which could think and learn on its own), and then the interest in it declined due to the unfeasability of many tasks it was supposed to tackle. Have a look a [[3]](article) for a nice outline of the history of this algorithm. Rosenblatt's book   How it works    Image from [[the M Nielsen book]](book), which inspired most of the writing here.The way a perceptron works is rather simple, yet quite ingenious. You have the representation of a neuron as per figure, where a series of binary input values $(x_1, x_2, \ldots, x_n)$ come in and an output is out. Inputs can be weighted differently, so that weights $(w_1, w_2, \ldots, w_n)$ are given. The output will depend on inputs and weights, the inputs being the values which determine a decision, the weights the how important each value is.  The neuron is equipped with a threshold value $t$, or alternatively a bias value $b=-t$, such that the output $o$ follows rules$$o = \begin{cases}0 \ \text{ if } \ w \cdot x + b \leq 0\\1 \ \text{ if } \ w \cdot x + b > 0\end{cases}$$which is to say that the neuron "fires" when the dot product of input and weights plus its bias pass 0 (or aleternatively when the dot product of weights and inputs passes threshold), a representation of what happens to real neurons when stimulated and producing electrical activity. The bias values makes things such that the larger it is, the easier it will be for the neuron to fire. So the perceptron is pretty much an artificial neuron with the output function given by the [Heaviside step](../maths/functions.ipynbHeaviside-step), displayed in figure.<jupyter_code>x = np.arange(-10, 10, 1)
y = []
for item in x:
    value = 0 if item < 0 else 1
    y.append(value)

plt.step(x, y)
plt.xlim(-10, 10)
plt.xlabel('$x$')
plt.ylabel('$y$')
plt.title('The Heaviside step function')
plt.show();<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.08144416456759027, 'fraction_numerical': 0.011754827875734676, 'mean_word_length': 4.3792325056433405, 'pattern_counts': {'":': 0, '<': 6, '<?xml version=': 0, '>': 6, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '1', 'dir': 'jupyter-structured-clean-dedup', 'id': '501257', 'license': 'MIT', 'n_tokens_mistral': 694, 'n_tokens_neox': 648, 'n_words': 364, 'path': 'neural-nets/types/perceptron.ipynb', 'repo_name': 'walkenho/tales-science-data', 'size': '2382'}