<jupyter_start><jupyter_text>Softmax and SigmoidTask: more practice using the `softmax` function, and connect it with the `sigmoid` function.  Setup<jupyter_code>import torch
from torch import tensor
import matplotlib.pyplot as plt
%matplotlib inline
def softmax(x):
    return torch.softmax(x, axis=0)<jupyter_output><empty_output><jupyter_text>Task Try this example:<jupyter_code>x1 = tensor([0.1, 0.2, 0.3])
x2 = tensor([0.1, 0.2, 100])
softmax(x1)<jupyter_output><empty_output><jupyter_text>1. Write a block of code that assigns `p = softmax(x1)` then evaluates `p.sum()`. **Before you run it**, predict what the output will be.<jupyter_code># your code here<jupyter_output><empty_output><jupyter_text>2. Write a block of code that evaluates `p2 = softmax(x2)` and displays the result. **Before you run it**, predict what it will output.<jupyter_code># your code here<jupyter_output><empty_output><jupyter_text>3. Evaluate `torch.sigmoid(tensor(0.1))`. Write an expression that uses `softmax` to get the same output. *Hint*: Give `softmax` a two-element `tensor([num1, num2])`, where one of the numbers is 0.<jupyter_code># your code here<jupyter_output><empty_output><jupyter_text>Analysis 1. A valid probability distribution has no negative numbers and sums to 1. Is `softmax(x)` a valid probability distribution? Why or why not? *your answer here* 2. Jargon alert: sometimes `x` is called the "logits" and `x.softmax(axis=0).log()` (or `x.log_softmax(axis=0)`) is called the "logprobs", short for "log probabilities". Complete the following expressions for `x1` (from the example above).<jupyter_code>logits = ...
logprobs = ...
probabilities = ...<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.1228486646884273, 'fraction_numerical': 0.020178041543026708, 'mean_word_length': 6.113924050632911, 'pattern_counts': {'":': 0, '<': 25, '<?xml version=': 0, '>': 25, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '6', 'dir': 'jupyter-structured-clean-dedup', 'id': '270653', 'license': 'MIT', 'n_tokens_mistral': 586, 'n_tokens_neox': 569, 'n_words': 184, 'path': 'static/fundamentals/u5n2-softmax-2.ipynb', 'repo_name': 'VelSharon/cs344', 'size': '1685'}