<jupyter_start><jupyter_text>Federated Learning - SMS spam prediction with a GRU model **NOTE**: At the time of running this notebook, we were running the grid components in background mode.**NOTE**: Components:* Grid Gateway (http://localhost:8080)* Grid Node Bob (http://localhost:3000)* Grid Node Anne (http://localhost:3001) To **start the gateway**:* ```cd gateway```* ```python gateway.py --start_local_db --port=8080```To **start one grid node**:* ```cd app/websocket/```* ```python websocket_app.py --start_local_db --id=anne --port=3001 --gateway_url=http://localhost:8080```This notebook was made based on [Federated SMS Spam prediction](https://github.com/OpenMined/PySyft/tree/master/examples/tutorials/advanced/Federated%20SMS%20Spam%20prediction).Authors:* <NAME>: Github: [@andrelmfarias](https://github.com/andrelmfarias) | Twitter: [@andrelmfarias](https://twitter.com/andrelmfarias)* <NAME>: Githib [@gmuraru](https://github/com/gmuraru) | Twitter: [@georgemuraru](https://twitter.com/georgemuraru) | Facebook: [@<NAME>](https://www.facebook.com/georgecmuraru)  Useful imports<jupyter_code>import numpy as np

import torch

import syft as sy
import grid as gr<jupyter_output><empty_output><jupyter_text>Setup configInit hook, connect with grid nodes, etc...<jupyter_code>hook = sy.TorchHook(torch)

# Connect directly to grid nodes
nodes = ["ws://localhost:3000/",
         "ws://localhost:3001/"]

compute_nodes = []
for node in nodes:
    compute_nodes.append(gr.WebsocketGridClient(hook, node))<jupyter_output>WARNING:root:Torch was already hooked... skipping hooking process<jupyter_text>Load Dataset  1) Download (if not present) and preprocess dataset<jupyter_code>import os
import urllib.request
import pathlib
from zipfile import ZipFile

URL = "https://archive.ics.uci.edu/ml/machine-learning-databases/00228/smsspamcollection.zip"
DATASET_NAME = "smsspamcollection"

def dataset_exists():
    return os.path.isfile('./data/inputs.npy') and \
    os.path.isfile('./data/labels.npy')
    
if not dataset_exists():
    #If the dataset does not already exist, let's download the dataset directly from the URL where it is hosted
    print('Downloading the dataset with urllib2 to the current directory...')
    pathlib.Path("data").mkdir(exist_ok=True)
    urllib.request.urlretrieve(URL, './data/data.zip')
    print("The dataset was successfully downloaded")
    print("Unzipping the dataset...")
    with ZipFile('./data/data.zip', 'r') as zipObj:
       # Extract all the contents of the zip file in current directory
       zipObj.extractall("./data")
    print("Dataset successfully unzipped")
    
    from preprocess import preprocess_spam

    preprocess_spam()
else:
    print("Not downloading the dataset because it was already downloaded")<jupyter_output>Not downloading the dataset because it was already downloaded<jupyter_text>2) Loading data As we are most interested in the usage of PySyft and Federated Learning, I will skip the text-preprocessing part of the project. If you are interested in how I performed the preprocessing of the raw dataset you can take a look on the script [preprocess.py](https://github.com/OpenMined/PyGrid/tree/master/examples/data/SMS-spam/preprocess.py).Each data point of the `inputs.npy` dataset correspond to an array of 30 tokens obtained form each message (padded at left or truncated at right)The `label.npy` dataset has the following unique values: `1` for `spam` and `0` for `non-spam`<jupyter_code>inputs = np.load('./data/inputs.npy')
labels = np.load('./data/labels.npy')
datasets_spam = torch.split(torch.tensor(inputs), int(len(inputs) / len(compute_nodes)), dim=0 ) #tuple of chunks (dataset / number of nodes)
labels_spam = torch.split(torch.tensor(labels), int(len(labels) / len(compute_nodes)), dim=0 )  #tuple of chunks (labels / number of nodes)<jupyter_output><empty_output><jupyter_text>3) Tagging tensorsThe code below will add a tag (of your choice) to the data that will be sent to grid nodes. This tag is important as the gateway will need it to retrieve this data later.<jupyter_code>tag_img = []
tag_label = []


for i in range(len(compute_nodes)):
    tag_img.append(datasets_spam[i].tag("#X", "#spam", "#dataset").describe("The input datapoints to the SPAM dataset."))
    tag_label.append(labels_spam[i].tag("#Y", "#spam", "#dataset").describe("The input labels to the SPAM dataset."))<jupyter_output><empty_output><jupyter_text>4) Sending our tensors to grid nodes<jupyter_code># NOTE: For some reason, there is strange behavior when trying to send within a loop.
# Ex : tag_x[i].send(compute_nodes[i])
# When resolved, this should be updated.

for i in range(len(compute_nodes)):
    shared_x = tag_img[i].send(compute_nodes[i], garbage_collect_data=False)
    shared_y = tag_label[i].send(compute_nodes[i], garbage_collect_data=False)
    print("X tensor pointers: ", shared_x, shared_y)<jupyter_output>X tensor pointers:  (Wrapper)>[PointerTensor | me:57842565258 -> bob:81837175413]
	Tags: #spam #dataset #X 
	Shape: torch.Size([2786, 30])
	Description: The input datapoints to the SPAM dataset.... (Wrapper)>[PointerTensor | me:39027710159 -> bob:41763767662]
	Tags: #Y #dataset #spam 
	Shape: torch.Size([2786])
	Description: The input labels to the SPAM dataset....
X tensor pointers:  (Wrapper)>[PointerTensor | me:37603922321 -> anne:44834606227]
	Tags: #spam #dataset #X 
	Shape: torch.Size([2786, 30])
	Description: The input datapoints to the SPAM dataset.... (Wrapper)>[PointerTensor | me:46472599434 -> anne:46155488760]
	Tags: #Y #dataset #spam 
	Shape: torch.Size([2786])
	Description: The input labels to the SPAM dataset....<jupyter_text>Disconnect Nodes<jupyter_code>for i in range(len(compute_nodes)):
    compute_nodes[i].close()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.12748659401487633, 'fraction_numerical': 0.02802283341982356, 'mean_word_length': 5.305343511450381, 'pattern_counts': {'":': 0, '<': 29, '<?xml version=': 0, '>': 37, 'https://': 8, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 9, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '7', 'dir': 'jupyter-structured-clean-dedup', 'id': '29883', 'license': 'Apache-2.0', 'n_tokens_mistral': 1960, 'n_tokens_neox': 1814, 'n_words': 533, 'path': 'examples/tutorials/advanced/Federated_SMS_Spam_prediction/Federated_SMS_Spam_prediction [Part 1] - Populate a Grid Network (Dataset).ipynb', 'repo_name': 'gmuraru/PyGrid', 'size': '5817'}