<jupyter_start><jupyter_text>Exercise 1 1.1 Given a Bloom Filter with these parameters: size=10, function count=5, tweak=99, which bits are set after adding these items?  * `b'Hello World'` * `b'Goodbye!'` 1.2. Make [this test](/edit/session8/bloomfilter.py) pass.<jupyter_code># Exercise 1.1

field_size = 10
function_count = 5
tweak = 99
items = (b'Hello World',  b'Goodbye!')

# bit_field_size is 8 * field_size
bit_field_size = field_size * 8
# create a bit field with the appropriate size
bit_field = [0] * bit_field_size

# for each item you want to add to the filter
for item in items:
    # iterate function_count number of times
    for i in range(function_count):
        # BIP0037 spec seed is i*BIP37_CONSTANT + tweak
        seed = i * BIP37_CONSTANT + tweak
        # get the murmur3 hash given that seed
        h = murmur3(item, seed=seed)
        # set the bit to be h mod the bit_field_size
        bit = h % bit_field_size
        # set the bit_field at the index bit to be 1
        bit_field[bit] = 1
# print the bit field converted to bytes using bit_field_to_bytes in hex
print(bit_field_to_bytes(bit_field).hex())
# Exercise 1.2

reload(bloomfilter)
run(bloomfilter.BloomFilterTest('test_add'))<jupyter_output>.
----------------------------------------------------------------------
Ran 1 test in 0.001s

OK<jupyter_text>Exercise 2 2.1. Make [this test](/edit/session8/bloomfilter.py) pass.```bloomfilter.py:BloomFilterTest:test_filterload``` 2.2. Do the following:* Connect to a testnet node* Load a filter for your testnet address* Send a request for transactions from the block which had your previous testnet transaction* Receive the merkleblock and tx messages.<jupyter_code># Exercise 2.1

reload(bloomfilter)
run(bloomfilter.BloomFilterTest('test_filterload'))
# Exercise 2.2

from bloomfilter import BloomFilter

block_hash = bytes.fromhex('0000000053787814ed9dd8c029d0a0a9af4ab8ec0591dc31bdc4ab31fae88ce9')  # FILL THIS IN
passphrase = b'<PASSWORD>'  # FILL THIS IN
secret = little_endian_to_int(hash256(passphrase))
private_key = PrivateKey(secret=secret)
addr = private_key.point.address(testnet=True)
print(addr)
filter_size = 30
filter_num_functions = 5
filter_tweak = 90210  # FILL THIS IN

# get the hash160 of the address using decode_base58
h160 = decode_base58(addr)
# create a bloom filter using the filter_size, filter_num_functions and filter_tweak above
bf = BloomFilter(filter_size, filter_num_functions, filter_tweak)
# add the h160 to the bloom filter
bf.add(h160)

# connect to tbtc.programmingblockchain.com in testnet mode, logging True
node = SimpleNode('tbtc.programmingblockchain.com', testnet=True, logging=True)
# complete the handshake
node.handshake()
# send the filterload message
node.send(bf.filterload())
# create a getdata message
getdata = GetDataMessage()
# add_data (FILTERED_BLOCK_DATA_TYPE, block_hash) to request the block
getdata.add_data(FILTERED_BLOCK_DATA_TYPE, block_hash)
# send the getdata message
node.send(getdata)
# wait for the merkleblock command
mb = node.wait_for(MerkleBlock)
# wait for the tx command
tx_obj = node.wait_for(Tx)
# print the envelope payload in hex
print(tx_obj.serialize().hex())<jupyter_output>mwJn1YPMq7y5F8J3LkC5Hxg9PHyZ5K4cFv
sending: version: 7f1101000000000000000000bed4f05b00000000000000000000000000000000000000000000ffff000000008d20000000000000000000000000000000000000ffff000000008d208bb81ad3dbdec79c1b2f70726f6772616d6d696e67626c6f636b636861696e3a302e312f0000000001
receiving: version: 7f1101000d04000000000000bfd4f05b00000000000000000000000000000000000000000000ffff4830fd33fd6a0d040000000000000000000000000000000000000000000000000c15ae81a23209e9102f5361746f7368693a302e31372e302f3a08160001
sending: verack: 
receiving: verack: 
sending: filterload: 1e000000000448000000000004000000000200000000000000000000000000050000006260010001
sending: getdata: 0103000000e98ce8fa31abc4bd31dc9105ecb84aafa9a0d029c0d89ded1478785300000000
receiving: sendheaders: 
receiving: sendcmpct: 000200000000000000
receiving: sendcmpct: 000100000000000000
receiving: ping: 4276cd7f35d6d8ce
sending: pong: 4276cd7f35d6d8ce
receiving: addr: 01bfd4f05b0d0400000000000000000000000000000000ffffa2d49eac479d
receiving[...]<jupyter_text>Exercise 3 3.1. Make [this test](/edit/session8/merkleblock.py) pass.<jupyter_code># Exercise 3.1

reload(merkleblock)
run(merkleblock.MerkleBlockTest('test_is_valid'))<jupyter_output>.
----------------------------------------------------------------------
Ran 1 test in 0.001s

OK<jupyter_text>Exercise 4 4.1. You have been sent some unknown amount of testnet bitcoins to your address. Send all of it back (minus fees) to `mwJn1YPMq7y5F8J3LkC5Hxg9PHyZ5K4cFv` using only the networking protocol.<jupyter_code># Exercise 4.1

from merkleblock import MerkleBlock, MerkleTree

last_block_hex = '000000000d65610b5af03d73ed67704713c9b734d87cf4b970d39a0416dd80f9'  # FILL THIS IN
last_block = bytes.fromhex(last_block_hex)
passphrase = b'<PASSWORD>'  # FILL THIS IN
secret = little_endian_to_int(hash256(passphrase))
private_key = PrivateKey(secret=secret)
addr = private_key.point.address(testnet=True)
print(addr)
h160 = decode_base58(addr)
target_address = 'mwJn1YPMq7y5F8J3LkC5Hxg9PHyZ5K4cFv'
filter_size = 30
filter_num_functions = 5
filter_tweak = 90210  # FILL THIS IN
target_h160 = decode_base58(target_address)
target_script = p2pkh_script(target_h160)
fee = 5000  # fee in satoshis

# connect to tbtc.programmingblockchain.com in testnet mode, logging True
node = SimpleNode('tbtc.programmingblockchain.com', testnet=True, logging=True)
# create a bloom filter using variables above
bf = BloomFilter(filter_size, filter_num_functions, filter_tweak)
# add the h160 to the bloom filter
bf.add(h160)
# complete the handshake
node.handshake()
# send the 'filterload' message
node.send(bf.filterload())

# create GetHeadersMessage with the last_block as the start_block
getheaders = GetHeadersMessage(start_block=last_block)
# send a getheaders message
node.send(getheaders)

# wait for the headers message
headers = node.wait_for(HeadersMessage)

# initialize the GetDataMessage
getdata = GetDataMessage()
# loop through the headers in the headers message
for header in headers.headers:
    # check that the proof of work on the block is valid
    if not header.check_pow():
        raise RuntimeError
    # check that this block's prev_block is the last block
    if last_block is not None and header.prev_block != last_block:
        raise RuntimeError
    # set the last block to the current hash
    last_block = header.hash()
    # add_data(FILTERED_BLOCK_DATA_TYPE, last_block) to get_data_message
    getdata.add_data(FILTERED_BLOCK_DATA_TYPE, last_block)
# send the getdata message
node.send(getdata)

# initialize prev_tx to None
prev_tx = None
# while prev_tx is None 
while prev_tx is None:
    # wait for the merkleblock or tx commands
    message = node.wait_for(MerkleBlock, Tx)
    # if we have the merkleblock command
    if message.command == b'merkleblock':
        # check that the MerkleBlock is valid
        if not message.is_valid():
            raise RuntimeError
    # else we have the tx command
    else:
        # set message.testnet=True
        message.testnet = True
        # loop through the enumerated tx outs (enumerate(message.tx_outs))
        for i, tx_out in enumerate(message.tx_outs):
            # if our output has the same address as our address (addr) we found it
            if tx_out.script_pubkey.address(testnet=True) == addr:
                # we found our utxo. set prev_tx, prev_index, prev_amount
                prev_tx = message.hash()
                prev_index = i
                prev_amount = tx_out.amount
                # break
                break
# create tx_in
tx_in = TxIn(prev_tx, prev_index)
# calculate the output amount (prev_amount - fee)
output_amount = prev_amount - fee
# create tx_out
tx_out = TxOut(output_amount, target_script)
# create transaction on testnet
tx_obj = Tx(1, [tx_in], [tx_out], 0, testnet=True)
# sign the one input we have
tx_obj.sign_input(0, private_key)
# serialize and hex to see what it looks like
print(tx_obj.serialize().hex())
# send this signed transaction on the network
node.send(tx_obj)
# wait a sec so this message goes through to the other node sleep(1) 
sleep(1)
# now ask for this transaction from the other node
# create a GetDataMessage
getdata = GetDataMessage()
# add_data (TX_DATA_TYPE, tx_obj.hash()) to get data message
getdata.add_data(TX_DATA_TYPE, tx_obj.hash())
# send the GetDataMessage
node.send(getdata)
# now wait for a response
got = node.wait_for(Tx)
if got.id() == tx_obj.id():
    # yes! we got to what we wanted
    print('success!')
    print(tx_obj.id())<jupyter_output>mwJn1YPMq7y5F8J3LkC5Hxg9PHyZ5K4cFv
sending: version: 7f1101000000000000000000e6d5f05b00000000000000000000000000000000000000000000ffff000000008d20000000000000000000000000000000000000ffff000000008d20648cac6d855295461b2f70726f6772616d6d696e67626c6f636b636861696e3a302e312f0000000001
receiving: version: 7f1101000d04000000000000e8d5f05b00000000000000000000000000000000000000000000ffff4830fd33feb30d040000000000000000000000000000000000000000000000004c53a609f7915de7102f5361746f7368693a302e31372e302f3b08160001
sending: verack: 
receiving: verack: 
sending: filterload: 1e000000000448000000000004000000000200000000000000000000000000050000006260010001
sending: getheaders: 7f11010001f980dd16049ad370b9f47cd834b7c913477067ed733df05a0b61650d000000000000000000000000000000000000000000000000000000000000000000000000
receiving: sendheaders: 
receiving: sendcmpct: 000200000000000000
receiving: sendcmpct: 000100000000000000
receiving: ping: 86cb5f28d057c9d4
sending: pong: 86cb5f28d057c9d4
receiving: addr: 01e8d[...]
{'fraction_non_alphanumeric': 0.08329905388728918, 'fraction_numerical': 0.15106951871657753, 'mean_word_length': 5.0819262038774236, 'pattern_counts': {'":': 0, '<': 15, '<?xml version=': 0, '>': 15, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 35, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '4', 'dir': 'jupyter-structured-clean-dedup', 'id': '503069', 'license': 'BSD-2-Clause', 'n_tokens_mistral': 4303, 'n_tokens_neox': 3197, 'n_words': 832, 'path': 'session8/complete/session8.ipynb', 'repo_name': 'robbaltzer/pb-exercises', 'size': '9770'}