<jupyter_start><jupyter_text>Soft_max_function Neural networks are available for both regression and classificationsIn general, using identity function for Regression and soft max function for classificationidentity function outputs the input as it is. The expression for soft max function is: ![soft_max_function1](./images/soft_max_function1.png) exp means exponential function OK, Let's implement a soft max function.<jupyter_code>import numpy as np
a = np.array([0.3, 2.9, 4.0])
exp_a = np.exp(a) 
print(exp_a)

sum_exp_a = np.sum(exp_a)
print(sum_exp_a)

y = exp_a / sum_exp_a
print(y)<jupyter_output>[ 1.34985881 18.17414537 54.59815003]
74.1221542101633
[0.01821127 0.24519181 0.73659691]<jupyter_text>pretty easy isn't it?, Let's recreate it as a function.<jupyter_code>def softmax(a):
    exp_a = np.exp(a)
    sum_exp_a = np.sum(exp_a)
    return exp_a / sum_exp_a<jupyter_output><empty_output><jupyter_text>it's perfect showing soft max function expression but there is one problem, overflow !As you can see, softmax uses exponential functions, but exponential functions outputs very large numbers.so, we need recreate it ![soft_max_function2](./images/soft_max_function2.png)i will show you one of example with the problem<jupyter_code>a = np.array([1010., 1000., 990.])
np.exp(a) / np.sum(np.exp(a))<jupyter_output>C:\Users\che99\Anaconda3\lib\site-packages\ipykernel_launcher.py:2: RuntimeWarning: overflow encountered in exp
  
C:\Users\che99\Anaconda3\lib\site-packages\ipykernel_launcher.py:2: RuntimeWarning: invalid value encountered in true_divide<jupyter_text>you see? there is no number because overflow you wanna fix it ?<jupyter_code>a = np.array([1010., 1000., 990.])
a = a - np.max(a)
np.exp(a) / np.sum(np.exp(a))<jupyter_output><empty_output><jupyter_text>it's so easy , you just subtract the maximum number of inputs.Let's reimplement a soft max function.<jupyter_code>def softmax(a):
    a = a - np.max(a)
    exp_a = np.exp(a)
    sum_exp_a = np.sum(exp_a)
    return exp_a / sum_exp_a
test = np.array([1010,1000,990])
softmax(test)<jupyter_output><empty_output><jupyter_text>Let's look at the characteristics of soft maxfunction.<jupyter_code>a = np.array([3.3, 2.2, 4.8, 7.7, 1,5])
y = softmax(a)
print('after soft max :', y)

sum = 0
for i in y:
    sum += i
print('sum of outputs : ', sum)<jupyter_output>after soft max : [0.01077126 0.00358544 0.04827345 0.87732862 0.00107991 0.05896132]
sum of outputs :  1.0<jupyter_text>you realized? 'Soft maximum function' always outputs between 0 and 1and the sum of all the outputs is always 1we can think outpus are percentage because that characteristics you can interpret it as follows.<jupyter_code>a = np.array([8.4, 6.8, 8.9])
result = softmax(a)

for i in range(3):
    print('{}: {:.3f} %'.format(i, result[i]* 100 ))<jupyter_output>0: 35.080 %
1: 7.083 %
2: 57.837 %
{'fraction_non_alphanumeric': 0.11816269284712483, 'fraction_numerical': 0.07713884992987377, 'mean_word_length': 5.135483870967742, 'pattern_counts': {'":': 0, '<': 25, '<?xml version=': 0, '>': 25, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '7', 'dir': 'jupyter-structured-clean-dedup', 'id': '91551', 'license': 'MIT', 'n_tokens_mistral': 1188, 'n_tokens_neox': 1033, 'n_words': 292, 'path': '4_Soft_max_function.ipynb', 'repo_name': 'che9992/DeepLeaningFromScratch', 'size': '2852'}