<jupyter_start><jupyter_text>Training an XOR network using PySyftIn this notebook we'll be training an XOR network using PySyft. What's special about this is that the training data will be divided between two domain nodes that are owned by different parties.<jupyter_code>import syft as sy
import numpy as np
from syft.core.adp.entity import DataSubject
from syft.core.tensor.smpc.mpc_tensor import MPCTensor
sy.logger.remove()<jupyter_output><empty_output><jupyter_text>Party 1 - OpenMined's team in the UK<jupyter_code># Party 1 logs into their domain node
p1_domain_node = sy.login(email="<EMAIL>",password="<PASSWORD>",port=8081)
# They take their training data for the XOR network
training_data_p1 = np.array(
    [[0,0,1],
    [0,1,1]], dtype=np.int64
)

# rasswanth = DataSubject(name="Rasswanth")

training_data_p1 = sy.Tensor(training_data_p1).private(
    min_val=0,
    max_val=1,
    entities=["Rasswanth"] * training_data_p1.shape[0],
    ndept=True
)

training_targets_p1 = np.array([[0,1]]).T

training_targets_p1 = sy.Tensor(training_targets_p1).private(
    min_val=0,
    max_val=1,
    entities=["Rasswanth"] * training_data_p1.shape[0],
    ndept=True
)

p1_domain_node.load_dataset(
    assets={
        "training_data":training_data_p1,
        "training_targets":training_targets_p1
    },
    name="Our training data for XOR networks!",
    description="Collected on Jan 27 2022"
)

# This is just used to check our results afterwards.
# t1 = sy.Tensor(np.array([1,2,3])).send(p1)<jupyter_output><class 'syft.core.tensor.tensor.Tensor'>
<class 'syft.core.tensor.autodp.ndim_entity_phi.PhiTensor'>
Dataset is uploaded successfully !!! 🎉                                                                                                                                       

Run `<your client variable>.datasets` to see your new dataset loaded into your machine!<jupyter_text>Let's check and make sure our dataset  was uploaded!<jupyter_code>p1_domain_node.datasets<jupyter_output><empty_output><jupyter_text>Looks good!Now the last thing this team needs to do is to create an account for the Data Scientist who will be training the network using their data.<jupyter_code>p1_domain_node.users.create(
    **{
        "name": "<NAME>",
        "email": "<EMAIL>",
        "password": "<PASSWORD>",
        "budget": 100
    }
)
p1_domain_node.users<jupyter_output><empty_output><jupyter_text>Party 2 - Facebook's team in Menlo Park<jupyter_code>DOMAIN2_PORT = 8082
p2_domain_node = sy.login(email="<EMAIL>",password="<PASSWORD>",port=DOMAIN2_PORT)
training_data_p2 = np.array([
    [1,0,1],
    [1,1,1]]
)

training_data_p2 = sy.Tensor(training_data_p2).private(
    min_val=0,
    max_val=1,
    entities=["Rasswanth"] * training_data_p2.shape[0],
    ndept=True
)


training_targets_p2 = np.array([[1,0]]).T
training_targets_p2 = sy.Tensor(training_targets_p2).private(
    min_val=0,
    max_val=1,
    entities=["Rasswanth", "Rasswanth"],
    ndept=True
)

p2_domain_node.load_dataset(
    assets={
        "training_data":training_data_p2,
        "training_targets":training_targets_p2
    },
    name="Our training data for XOR networks!",
    description="Collected on Jan 27 2022"
)

# Used to check our results afterwards
# t2 = sy.Tensor(np.array([1,2,3])).send(p2)
p2_domain_node.datasets<jupyter_output><empty_output><jupyter_text>Looks good!Now the last thing this team needs to do is to create an account for the Data Scientist who will be training the network using their data.<jupyter_code>p2_domain_node.users.create(
    **{
        "name": "<NAME>",
        "email": "<EMAIL>",
        "password": "<PASSWORD>",
        "budget": 100
    }
)
p2_domain_node.users<jupyter_output><empty_output><jupyter_text>Data Scientists<jupyter_code>ds_domain1 = sy.login(email="<EMAIL>", password="<PASSWORD>", port=8081)
ds_domain1.datasets
ds_domain2 = sy.login(email="<EMAIL>", password="<PASSWORD>", port=8082)
ds_domain2.datasets
ds_domain1.privacy_budget
ds_domain2.privacy_budget
X, y = sy.concatenate(p1, p2, X, y)

## This replaces the following code:
# d1 = X.send(p1)
# X = MPCTensor(secret=d1,parties=parties,shape=X.shape)
# d2 = y.send(p1)
# y = MPCTensor(secret=d2,parties=parties,shape=y.shape)
def relu(x,deriv=False):
    if deriv==True:
        return x>0
    return x*(x>0)
layer0_weights = 2*np.random.random((3,4)) - 1
layer1_weights = 2*np.random.random((4,1)) - 1
for j in range(1):
    # Forward propagation
    layer1_inputs = relu(X @ layer0_weights)  ; layer1_inputs.block
    layer2_inputs = relu(layer1_inputs @ layer1_weights) ; layer2_inputs.block 
    
    # Calculate errors
    layer2_inputs_delta = (y - layer2_inputs)* relu(layer2_inputs,deriv=True) ; layer2_inputs_delta.block
    layer1_inputs_delta = (layer2_inputs_delta@(layer1_weights.T)) * relu(layer1_inputs,deriv=True) ; layer1_inputs_delta.block
    
    # Update weights
    layer1_weights  = layer1_weights + layer1_inputs.T @ layer2_inputs_delta ; layer1_weights.block
    layer0_weights =  layer0_weights + X.T @ layer1_inputs_delta ; layer0_weights.block<jupyter_output>Computing...:   0%|                                                                                                                                                    | 0/64 [00:00<?, ?it/s]/home/e/PycharmProjects/PySyft/packages/syft/src/syft/core/tensor/smpc/utils.py:72: RuntimeWarning: overflow encountered in multiply
  res = op(np.empty(x_shape), np.empty(y_shape)).shape
Computing...: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 64/64 [00:52<00:00,  1.21it/s]<jupyter_text>Let's see how our privacy budget changed as a result of training for a single epoch:<jupyter_code>ds_domain1.privacy_budget
ds_domain2.privacy_budget<jupyter_output><empty_output><jupyter_text>And voila! We've trained a neural network using PySyft's adversaril differential privacy system and its secure multiparty computation system working in tandem. Demo ends above- the cells below are for temporary Debugging:<jupyter_code>X
layer0_weights
layer1_inputs = relu(X @ layer0_weights)  ; layer1_inputs.block
layer2_inputs = relu(layer1_inputs @ layer1_weights) ; layer2_inputs.block 
layer2_inputs_delta = (y - layer2_inputs)* relu(layer2_inputs,deriv=True) ; layer2_inputs_delta.block
layer1_inputs_delta = (layer2_inputs_delta@(layer1_weights.T)) * relu(layer1_inputs,deriv=True) ; layer1_inputs_delta.block
layer1_weights  = layer1_inputs.T @ layer2_inputs_delta  + layer1_weights; layer1_weights.block
layer0_weights =  X.T @ layer1_inputs_delta  + layer0_weights; layer0_weights.block
res = X.T @ layer1_inputs_delta
v1 = X.T
v2 = layer1_inputs_delta
def get_val(val):
    t1 = val.child[0].get_copy()
    t2 = val.child[0].get_copy()
    return t1+t2
v1 = get_val(v1)
v2 = get_val(v2)
v1.child.min_vals.to_numpy() @ v2.child.min_vals.to_numpy()
v2.child.min_vals
layer0_weights_dp = layer0_weights.publish(sigma=1e4)
layer1_weights_dp = layer1_weights.publish(sigma=1e4)
print(layer0_weights_dp.get_copy())
print(layer1_weights_dp.get_copy())<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.13186199189830983, 'fraction_numerical': 0.03380360385528705, 'mean_word_length': 4.295857988165681, 'pattern_counts': {'":': 12, '<': 54, '<?xml version=': 0, '>': 54, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 14, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '9', 'dir': 'jupyter-structured-clean-dedup', 'id': '502578', 'license': 'Apache-2.0', 'n_tokens_mistral': 2695, 'n_tokens_neox': 2544, 'n_words': 483, 'path': 'notebooks/smpc/ModelTraining.ipynb', 'repo_name': 'Noob-can-Compile/PySyft', 'size': '7233'}