<jupyter_start><jupyter_text>This notebook shows how runtime performance of the `tfe.protocol.pond` protocol can be improved by splitting the computation into an offline and an online phase. It assumes some understanding of how the protocol works at a cryptographic layer, at the very least an understanding of the notion of triples, and is intended for intermediate to advanced users.  Setup<jupyter_code>import tensorflow as tf
import tf_encrypted as tfe<jupyter_output><empty_output><jupyter_text>Turn on TFE profling flags since we want to inspect with TensorBoard below.<jupyter_code>%load_ext tensorboard.notebook

TENSORBOARD_DIR = "/tmp/tensorboard"

tfe.setTFETraceFlag(True)
tfe.setMonitorStatsFlag(True)
tfe.setLogDirectory(TENSORBOARD_DIR)<jupyter_output>INFO:tf_encrypted:Writing trace files for every session.run() call with a tag
INFO:tf_encrypted:Writing event files for every session.run() call with a tag
INFO:tf_encrypted:Writing event and trace files to '/tmp/tensorboard'<jupyter_text>ComputationAs a first step in defining our computation we have to select which triple source we want to use for the Pond protocol. By default this is `OnlineTripleSource` but here we want to used the queued version instead. Below we leave the option to select either.<jupyter_code>tf.Session.reset('')
tf.reset_default_graph()
# triple_source = tfe.protocol.pond.OnlineTripleSource("server2")
# tfe.set_protocol(tfe.protocol.Pond("server0", "server1", triple_source=triple_source))
triple_source = tfe.protocol.pond.QueuedOnlineTripleSource("server0", "server1", "server2", capacity=10)
tfe.set_protocol(tfe.protocol.Pond("server0", "server1", triple_source=triple_source))<jupyter_output><empty_output><jupyter_text>For our actual computation we are interested in two values, `y` and `w`, as defined next.Note that we optionally cache `c` as a useful way to speed up computations that repeatedly make use of the same fixed values, for instance the weights in private predictions.<jupyter_code>c = tfe.define_private_input("coefficients-provider", lambda: tf.constant([1,2,3,4,5,6,7,8,9,10], shape=[10, 1]))
c_updater, c = tfe.cache(tfe.mask(c))
# c_updater, c = tfe.cache(c)

x = tfe.define_private_input("data-provider", lambda: tf.fill([1, 10], 1))
y = tfe.matmul(x, c).reveal()

v = tfe.define_private_input("data-provider", lambda: tf.fill([1, 10], 2))
w = tfe.matmul(v, c).reveal()<jupyter_output>WARNING:tensorflow:From /usr/local/miniconda3/envs/tfe-dev/lib/python3.5/site-packages/tensorflow/python/framework/op_def_library.py:263: colocate_with (from tensorflow.python.framework.ops) is deprecated and will be removed in a future version.
Instructions for updating:
Colocations handled automatically by placer.<jupyter_text>Offline and Online Execution<jupyter_code>!rm -rf {TENSORBOARD_DIR}

sess = tfe.Session(disable_optimizations=True)
sess.run(tf.global_variables_initializer())
def print_triple_status():

    if not hasattr(triple_source, "queues"):
        return

    for queue in triple_source.queues:
        print("{:40}: {:>2} / {:>2}".format(
            queue.name,
            sess.run(queue.size()),
            triple_source.capacity))<jupyter_output><empty_output><jupyter_text>With the queued triple source the computation is split into two phases:- an *offline* phase where triples are generated and distributed by a trusted third-party- an *online* phase where the actual values are computed by consuming triples previously generated, and which does not involve the trusted third-partyThe most important thing it to always maintain a correspondance between these two phases, so that when we run an online computation we are consuming triples that where made to match for every node in the subgraph that defines it.To get started let us first update our cache. To generate triples for this we simply ask the triple source, passing in the fetch we intend to run later. This in itself does not run anything, it simply figures out how to generate triples for the particular fetch.<jupyter_code>c_updater_triples = triple_source.generate_triples(c_updater)

print_triple_status()<jupyter_output>mask/triple-store-0/fifo_queue          :  0 / 10
mask/triple-store-1/fifo_queue          :  0 / 10
mask_1/triple-store-0/fifo_queue        :  0 / 10
mask_1/triple-store-1/fifo_queue        :  0 / 10
matmul/triple-store-0/fifo_queue        :  0 / 10
matmul/triple-store-1/fifo_queue        :  0 / 10
mask_2/triple-store-0/fifo_queue        :  0 / 10
mask_2/triple-store-1/fifo_queue        :  0 / 10
matmul_1/triple-store-0/fifo_queue      :  0 / 10
matmul_1/triple-store-1/fifo_queue      :  0 / 10<jupyter_text>Next we then run the new fetch to actually generate triples.<jupyter_code>sess.run(c_updater_triples)

print_triple_status()<jupyter_output>mask/triple-store-0/fifo_queue          :  1 / 10
mask/triple-store-1/fifo_queue          :  1 / 10
mask_1/triple-store-0/fifo_queue        :  0 / 10
mask_1/triple-store-1/fifo_queue        :  0 / 10
matmul/triple-store-0/fifo_queue        :  0 / 10
matmul/triple-store-1/fifo_queue        :  0 / 10
mask_2/triple-store-0/fifo_queue        :  0 / 10
mask_2/triple-store-1/fifo_queue        :  0 / 10
matmul_1/triple-store-0/fifo_queue      :  0 / 10
matmul_1/triple-store-1/fifo_queue      :  0 / 10<jupyter_text>And finally we run our online computation, consuming what we just generated.<jupyter_code>sess.run(c_updater)

print_triple_status()<jupyter_output>mask/triple-store-0/fifo_queue          :  0 / 10
mask/triple-store-1/fifo_queue          :  0 / 10
mask_1/triple-store-0/fifo_queue        :  0 / 10
mask_1/triple-store-1/fifo_queue        :  0 / 10
matmul/triple-store-0/fifo_queue        :  0 / 10
matmul/triple-store-1/fifo_queue        :  0 / 10
mask_2/triple-store-0/fifo_queue        :  0 / 10
mask_2/triple-store-1/fifo_queue        :  0 / 10
matmul_1/triple-store-0/fifo_queue      :  0 / 10
matmul_1/triple-store-1/fifo_queue      :  0 / 10<jupyter_text>One way of maintaining the correspondance between the offline and online phase is to alternate between them as we have just done, first running the offline computation and then immediately running the online computation.But we can also work with larger sequences as shown next, generating triples for several fetches before running any online computation. Be careful not to overdo this though, as TensorFlow will block when the queues fill up and exceeds their capacity.<jupyter_code>y_triples = triple_source.generate_triples(y)
w_triples = triple_source.generate_triples(w)
sess.run(y_triples, tag='y_triples')
sess.run(w_triples, tag='w_triples')

print_triple_status()
res_y = sess.run(y, tag='y')
res_w = sess.run(w, tag='w')
print(res_y, res_w)

print_triple_status()<jupyter_output>[[55.]] [[110.]]
mask/triple-store-0/fifo_queue          :  0 / 10
mask/triple-store-1/fifo_queue          :  0 / 10
mask_1/triple-store-0/fifo_queue        :  0 / 10
mask_1/triple-store-1/fifo_queue        :  0 / 10
matmul/triple-store-0/fifo_queue        :  0 / 10
matmul/triple-store-1/fifo_queue        :  0 / 10
mask_2/triple-store-0/fifo_queue        :  0 / 10
mask_2/triple-store-1/fifo_queue        :  0 / 10
matmul_1/triple-store-0/fifo_queue      :  0 / 10
matmul_1/triple-store-1/fifo_queue      :  0 / 10<jupyter_text>Instead of running a sequence of fetches as above, one might be tempted to use either `sess.run([y_triples, w_triples])` or `sess.run([y, w])`. This is fine as long as it does not introduce non-determinism into the computation, which could break the correspondance we have to maintain.For instance, if we had used `tfe.cache(c)` instead of `tfe.cache(tfe.mask(c))` then there is an overlap in the computation of `y` and `w` that can cause different evaluation orders to break the correspondance and give wrong results. To force this to happen you can modify the computation earlier and change the evaluation order of `y` and `w` as done below: offline is for sequence `[y, w]` but online is for `[w, y]`.<jupyter_code># sess.run(y_triples)
# sess.run(w_triples)

# res_w = sess.run(w)
# res_y = sess.run(y)
# print(res_y, res_w)<jupyter_output><empty_output><jupyter_text>However, we can safely streamline our process and generate triples for the next run while executing the current. The reason this works is because of the `tf.queue.FIFOQueue`s used by the triple source.<jupyter_code>sess.run(y_triples)

res, _ = sess.run([y, y_triples], tag='streamlined')
print(res)

res, _ = sess.run([y, y_triples], tag='streamlined')
print(res)

res = sess.run(y)
print(res)<jupyter_output>[[55.]]
[[55.]]
[[55.]]<jupyter_text>Inspection<jupyter_code>%tensorboard --logdir {TENSORBOARD_DIR}<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.09866451761455215, 'fraction_numerical': 0.027285286668201705, 'mean_word_length': 4.223692122669874, 'pattern_counts': {'":': 0, '<': 42, '<?xml version=': 0, '>': 44, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '12', 'dir': 'jupyter-structured-clean-dedup', 'id': '321383', 'license': 'Apache-2.0', 'n_tokens_mistral': 3072, 'n_tokens_neox': 2943, 'n_words': 821, 'path': 'examples/notebooks/Queued triples.ipynb', 'repo_name': 'capeprivacy/tf-encrypted', 'size': '8686'}