<jupyter_start><jupyter_text>Lab 8-3: XOR with Wide and Deep Neural Network**<NAME> 2021****[Deep Learning By Torch] End to End study scripts of Deep Learning by implementing code practice with Pytorch.**If you have an any issue, please PR below.[[Deep Learning By Torch] - Github @JonyChoi](https://github.com/jonychoi/Deep-Learning-By-Torch)Here, we are going to learn about how the wide and deep neural network can change the process of learning and the reducing cost of XOR problem.  Imports<jupyter_code>import torch
import torch.nn as nn
import torch.nn.functional as F
import torch.optim as optim
device = 'cuda' if torch.cuda.is_available() else 'cpu'

torch.manual_seed(1)
if device == 'cuda':
    torch.cuda.manual_seed_all(1)<jupyter_output><empty_output><jupyter_text>Define XOR Problem<jupyter_code>X = torch.FloatTensor([[0, 0], [0, 1], [1, 0], [1, 1]]).to(device)
Y = torch.FloatTensor([[0], [1], [1], [0]]).to(device)<jupyter_output><empty_output><jupyter_text>Create the Wide and Deep Model<jupyter_code># nn layers
linear1 = nn.Linear(2, 10)
linear2 = nn.Linear(10, 10)
linear3 = nn.Linear(10, 10)
linear4 = nn.Linear(10, 1)
sigmoid = nn.Sigmoid()
model = nn.Sequential(linear1, sigmoid, linear2, sigmoid, linear3, sigmoid, linear4, sigmoid).to(device)
criterion = nn.BCELoss().to(device)
optimizer = optim.SGD(model.parameters(), lr =1)
for step in range(10001):

    #prediction
    pred = model(X)

    #cost
    cost = criterion(pred, Y)

    #Reduce the cost
    optimizer.zero_grad()
    cost.backward()
    optimizer.step()

    if step % 1000 == 0:
        result = sigmoid(pred).squeeze().detach().cpu().numpy()
        print('Epoch: {:4d}/10000, Result: {}, Cost: {:.6f}'.format(step, result, cost.item()))
# Accuracy computation
# True if hypothesis > 0.5 else False
with torch.no_grad():
    pred = model(X)
    predicted = (pred > 0.5).float()
    accuracy = (predicted == Y).float().mean() *100
    print('Prediction: {}, \nPredicted: {}, \nAccuracy: {}'.format(pred.detach().cpu().numpy(), predicted.detach().cpu().numpy(), accuracy))<jupyter_output>Prediction: [[1.19966135e-04]
 [9.99865890e-01]
 [9.99826610e-01]
 [1.66115104e-04]], 
Predicted: [[0.]
 [1.]
 [1.]
 [0.]], 
Accuracy: 100.0<jupyter_text>High-Level Implementation with ```nn.Module```<jupyter_code>class XOR_Wide_Deep(nn.Module):
    def __init__(self):
        super().__init__()
        self.sq = nn.Sequential(
            nn.Linear(2, 10),
            nn.Sigmoid(),
            nn.Linear(10, 10),
            nn.Sigmoid(),
            nn.Linear(10, 10),
            nn.Sigmoid(),
            nn.Linear(10, 1),
            nn.Sigmoid()
        )
    
    def forward(self, x):
        return self.sq(x)
model = XOR_Wide_Deep().to(device)
optimizer = optim.SGD(model.parameters(), lr=1)
nb_epochs = 10001

for epoch in range(nb_epochs):

    #prediction
    pred = model(X)

    #cost
    cost = F.binary_cross_entropy(pred, Y).to(device)

    #Reduce cost
    optimizer.zero_grad()
    cost.backward()
    optimizer.step()

    if epoch % 1000 == 0:
        result = torch.sigmoid(pred).squeeze().detach().cpu().numpy()
        print('Epoch: {:2d}/10000, result: {}, cost: {:.6f}'.format(epoch, result, cost.item()))
# Accuracy computation
# True if hypothesis > 0.5 else False
with torch.no_grad():
    pred = model(X)
    predicted = (pred > 0.5).float()
    accuracy = (predicted == Y).float().mean() *100
    print('Prediction: {}, \nPredicted: {}, \nAccuracy: {}'.format(pred.detach().cpu().numpy(), predicted.detach().cpu().numpy(), accuracy))<jupyter_output>Prediction: [[0.00111703]
 [0.99814284]
 [0.9980995 ]
 [0.00196971]], 
Predicted: [[0.]
 [1.]
 [1.]
 [0.]], 
Accuracy: 100.0
{'fraction_non_alphanumeric': 0.17014359252235167, 'fraction_numerical': 0.05445678677865077, 'mean_word_length': 3.6323713927227104, 'pattern_counts': {'":': 0, '<': 16, '<?xml version=': 0, '>': 20, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '4', 'dir': 'jupyter-structured-clean-dedup', 'id': '95973', 'license': 'MIT', 'n_tokens_mistral': 1471, 'n_tokens_neox': 1334, 'n_words': 306, 'path': '09. XOR to Neural Network/9-4. XOR with Wide and Deep Neural Network.ipynb', 'repo_name': 'jonychoi/Deep-Learning', 'size': '3698'}