<jupyter_start><jupyter_text>Phasor activation functions MVN activation functionThis code explains the logic of mvn activation function for an easy understanding.For further information refer to the original papers of <NAME>: - [Multivalued threshold functions](https://link.springer.com/article/10.1007%2FBF01068667) - [A GENERALIZATION OF THE THRESHOLD FUNCTIONS](https://www.jstor.org/stable/43667575?casa_token=QrhCiytSlCUAAAAA%3ARRwNRt0TRmfddbOwMP_Si5cqYg6FzCa44WtvX2Ac2yPK_xkA6n2ycfuyLzvWvKFdh-L9JfShlDV6elRcjgtiuDgfKEKFOfA6fqxCIUX5r9SG4Ou7hETN&seq=1metadata_info_tab_contents) According to these works: *A multi-valued neuron (MVN) is a neural element with n inputs and one output lying on the unit circle, and with complex-valued weights.*<jupyter_code># We first import everything
import matplotlib.pyplot as plt
from cvnn.activations import mvn_activation, georgiou_cdbp
import tensorflow as tf
import numpy as np<jupyter_output><empty_output><jupyter_text>For a start we will create complex valued points to use as an example.<jupyter_code>x = tf.constant([-2, 1.0, 0.0, 1.0, -3], dtype=tf.float32)
y = tf.constant([-2.5, -1.5, 0.0, 1.0, 2], dtype=tf.float32)
z = tf.complex(x, y)<jupyter_output><empty_output><jupyter_text>MVN function divides the phase into `k` sections and cast the input phase to the closest of those k values while also fizing the amplitude to 1.The equation would be$$f(z) = \exp^{\frac{i 2 \pi a}{ k } }$$with $a$ so that $$\frac{i 2 \pi a}{ k }  \leq arg(z) \le \frac{i 2 \pi (a+1)}{ k } $$<jupyter_code>k = 3
result = mvn_activation(z, k=k)
# cnums = np.arange(5) + 1j * np.arange(6, 11)]
ax = plt.axes()
ax.scatter(tf.math.real(z), tf.math.imag(z), color='red')
ax.scatter(tf.math.real(result), tf.math.imag(result), color='blue')

# Plot arrows of the mapping road
for x, y, dx, dy in zip(tf.math.real(z), tf.math.imag(z),
                        tf.math.real(result) - tf.math.real(z),
                        tf.math.imag(result) - tf.math.imag(z)):
    ax.arrow(x, y, dx, dy, length_includes_head=True, head_width=0.1)
# PLot unit circle
t = np.linspace(0, np.pi * 2, 100)
ax.plot(np.cos(t), np.sin(t), linewidth=1)

yabs_max = abs(max(ax.get_ylim(), key=abs))
xabs_max = abs(max(ax.get_xlim(), key=abs))
axis_max = max(yabs_max, xabs_max)

# Divide map on the different zones
ax.pie(np.ones(k) / k, radius=4, wedgeprops={'alpha': 0.3})

ax.set_ylim(ymin=-axis_max, ymax=axis_max)
ax.set_xlim(xmin=-axis_max, xmax=axis_max)
plt.show()<jupyter_output><empty_output><jupyter_text>Continous valuesIf `k` is not given, it will use $k \to \infty$ making it an equivalence of just mapping the input to the unitary circle (keeps the phase but changes the amplitude to 1).This is mathematically $$ f(z) = \exp^{i arg(z)} .$$For $z \neq 0$ this is also $$ f(z) = \frac{z}{|z|} .$$<jupyter_code>result = mvn_activation(z)

ax = plt.axes()
ax.scatter(tf.math.real(z), tf.math.imag(z), color='red')
ax.scatter(tf.math.real(result), tf.math.imag(result), color='blue')
for x, y, dx, dy in zip(tf.math.real(z), tf.math.imag(z),
                        tf.math.real(result) - tf.math.real(z),
                        tf.math.imag(result) - tf.math.imag(z)):
    ax.arrow(x, y, dx, dy, length_includes_head=True, head_width=0.1)
t = np.linspace(0,np.pi*2,100)
ax.plot(np.cos(t), np.sin(t), linewidth=1)

yabs_max = abs(max(ax.get_ylim(), key=abs))
xabs_max = abs(max(ax.get_xlim(), key=abs))
axis_max = max(yabs_max, xabs_max)

ax.set_ylim(ymin=-axis_max, ymax=axis_max)
ax.set_xlim(xmin=-axis_max, xmax=axis_max)<jupyter_output><empty_output><jupyter_text>Georgiou CDBPActivation function proposed by [<NAME> and <NAME>](https://ieeexplore.ieee.org/abstract/document/142037)There are a few differences with MVN:- You can choose the circle radius with the `r` parameter.- Zero stays at zero.<jupyter_code>x = tf.constant([-2, 1.0, 0.0, 1.0, -3, 0.8, 0.1], dtype=tf.float32)
y = tf.constant([-2.5, -1.5, 0.0, 1.0, 2, 0.4, -0.4], dtype=tf.float32)
z = tf.complex(x, y)
result = georgiou_cdbp(z)

ax = plt.axes()
ax.scatter(tf.math.real(z), tf.math.imag(z), color='red')
ax.scatter(tf.math.real(result), tf.math.imag(result), color='blue')
for x, y, dx, dy in zip(tf.math.real(z), tf.math.imag(z),
                        tf.math.real(result) - tf.math.real(z),
                        tf.math.imag(result) - tf.math.imag(z)):
    ax.arrow(x, y, dx, dy, length_includes_head=True, head_width=0.1)
t = np.linspace(0, np.pi * 2, 100)
ax.plot(np.cos(t), np.sin(t), linewidth=1)

yabs_max = abs(max(ax.get_ylim(), key=abs))
xabs_max = abs(max(ax.get_xlim(), key=abs))
axis_max = max(yabs_max, xabs_max)

ax.set_ylim(ymin=-axis_max, ymax=axis_max)
ax.set_xlim(xmin=-axis_max, xmax=axis_max)
plt.show()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.1610865445356917, 'fraction_numerical': 0.02884817856390819, 'mean_word_length': 4.516840882694542, 'pattern_counts': {'":': 0, '<': 24, '<?xml version=': 0, '>': 24, 'https://': 3, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '5', 'dir': 'jupyter-structured-clean-dedup', 'id': '192128', 'license': 'MIT', 'n_tokens_mistral': 2009, 'n_tokens_neox': 1911, 'n_words': 432, 'path': 'docs/_build/html/activations/mvn_activation.ipynb', 'repo_name': 'saugatkandel/cvnn', 'size': '4774'}