<jupyter_start><jupyter_text>Practical 3: Proof of Work - Mining  This jupyter notebook can be downloaded and run locally with anaconda. Jupyter and anaconda should be installed in all AUT engineering and computer science labs. The benefit of using jupyter is that code snippets can be run live (python is running in the background).A static version can be found on github at https://github.com/millecodex/COMP842/. All code can be copied and pasted into your favourite text editior or IDE and *should* run with Python 3.x.You are encouraged to use any programming language you feel comfortable with, this is simply an example using python (and jupyter is designed for python demonstrations). AUT lab computers also have a java interpreter, and visual basic. For scientific computing you may also want to consider matlab or R.*** **_Proof of work_** is the unique contribution to the bitcoin protocol that achieves two aims:1. allows nodes to vote with their hashpower to achieve consensus2. keeps the network secure by aligning incentives.We have seen in tutorial 1 some of the uses of hashing with the `hashlib` library. Because a standard hash function like `sha256` is random, meaning there is no way to game the system or shortcut the output, it allows equal participation from anyone in the network. If one participant chooses to dedicate all of their computers and resources to hashing, then they are allowed more "votes" in the network and will receive proportionally more rewards for their effort.This whole process is referred to as _mining_ because it is analagous to gold mining.So lets get mining!  Prepare a block to be mined<jupyter_code># we will need pickle to turn our block into a byte object
# sha256 operates one byte at a time
import hashlib
import pickle

# create a new block
block = {   'timestamp': 1553200425.597771,
            'transactions': ['a','b','c'],
            'prev_hash': '00574faf29dbb37a3b12c2f8f5c05cf03c708313d4ad1d5d968cd2438beea104',
            'merkle_root': 'aa0522a507b6499cb3512494908ac9220',
        }

# pickle the block
pickled_block = pickle.dumps(block)
print(pickled_block)<jupyter_output>b'\x80\x03}q\x00(X\t\x00\x00\x00timestampq\x01GA\xd7$\xfdJfA\xe1X\x0c\x00\x00\x00transactionsq\x02]q\x03(X\x01\x00\x00\x00aq\x04X\x01\x00\x00\x00bq\x05X\x01\x00\x00\x00cq\x06eX\t\x00\x00\x00prev_hashq\x07X@\x00\x00\x0000574faf29dbb37a3b12c2f8f5c05cf03c708313d4ad1d5d968cd2438beea104q\x08X\x0b\x00\x00\x00merkle_rootq\tX!\x00\x00\x00aa0522a507b6499cb3512494908ac9220q\nu.'<jupyter_text>This block is in byte form and can now be hashed. Lets see what it looks like.<jupyter_code># hash the block and output a digest of hexidecimal digits
hashed_block = hashlib.sha256(pickled_block).hexdigest()
print(hashed_block)<jupyter_output>6c257095d5b9eac35f9f0f6503c49f54e336f79c5943d3bb12e3ba4589b7639e<jupyter_text>Every time we hash this same block with sha256 we will get the same unique output. In this case repeated hashing will not change the output. Lets introduce a variable that is added to the block. This variable doesn't change the data in the block, but allows for a different hash output. It is a **number** we will use **once**, then hash, test, update, and hash again. This is called a **nonce**.This simple example shows how a nonce changes the hash:<jupyter_code># choose 16 because a hex output has 16 digits
for nonce in range(16):
    print(str(nonce)+'_block_hash')<jupyter_output>0_block_hash
1_block_hash
2_block_hash
3_block_hash
4_block_hash
5_block_hash
6_block_hash
7_block_hash
8_block_hash
9_block_hash
10_block_hash
11_block_hash
12_block_hash
13_block_hash
14_block_hash
15_block_hash<jupyter_text>Now output the hashed of the above strings:<jupyter_code>for nonce in range(16):  
    print(hashlib.sha256((str(nonce)+'block_hash').encode()).hexdigest())<jupyter_output>f19542b4bfdfbad0c3d5645a01f046eb917d0ba23c4285b3b80e772c8e639b44
ec9727e33ae1f00f13ac2f24477e343ec0fe1586790dfdd8c1da0c40bf97dbad
0e228fa6afeaea338240378c1aaceda5d302bcf38d54d914e85893bfb7f96a39
4c5e814ff68c5beb662af2b1d0bb36c249bf3bc7d1f87048bdf8bd8f523de5da
3daa55e5d73e3db39294560cac88312405361940a8c82dc21cdd816ee1602cd1
e33dd12d4bef2ae199f468aaf9f061e9225fa6ce123c08f5a278a73268645dd4
6a49d5532ca475de8c0d0bcd04c40e19ac0317f8d5307bf6ad4335244ca70456
e31ba32365fbb59be5bba6bcb88edf878fee4e0411566238c5a9504331f2763d
fde106023507eb299435d62d7343cd4aec1805bb965a1492e305cd7c8e2a8b1a
b0649b7b974aa9d6d888c8a5068604127c7686ffb2b19d36a11f8601a86c7cf7
0ee472786259f438e2a89ddad826dafd7ea1c8c1486082dd83292f4d814ae58b
3f73ba333475c4fc54dd2b23b4e79ea7cfbf3b36a7b8c2f4299992d035d391db
3f21ef090c8481a1d0bbcd2458b5415333a9740b3e63bd3c6d675ce280fb6093
a1f1a1151379c671e6f70ab1b2be404b54d8fb7bd8bcadd72855fa47ee276b35
d68c9cc0a853050ab401bebcded574be884394de13333af075c77fb30f20ad9f
cf7b9b326fcca7b47d1b11be2[...]<jupyter_text>There should be no pattern to the hashed values above. All that was changed was the nonce. Since a hexidecial number has 16 possible digits (0-9,a-f), there should be a one-sixteenth probability that the first digit of the digest is a `0`.What is the probability that the first two digits are `00`? $P($ `00` $)=\left(\frac{1}{16}\right)^2=\frac{1}{256}=0.39\%$$\vdots$$P($ `0` $^n)=\frac{1}{16^n}$As the number of zeroes increase, the probability of finding a hash with those leading zeroes goes down. If we are looking for a hash with `00` at the beginning this is called the _target difficulty_. Mining a successful block means finding a hash that is below the target value. One way to do this is to take the hash string and compare the first digit.<jupyter_code># set the number of leading zeroes
difficulty = 1

# loop through the hashes checking each one
for nonce in range(16):  
    hash = hashlib.sha256((str(nonce)+'block_hash').encode()).hexdigest()
    print(str(hash))
    
    # check the string up to the index 'difficulty' and compare
    # note python string manipulation: '0'*5 = 00000; this is string multiplication
    if hash[:difficulty] == '0'*difficulty:
        print('Block Found!')
        break
        
    # end of range condition
    if nonce == 15: print('Block not found')<jupyter_output>f19542b4bfdfbad0c3d5645a01f046eb917d0ba23c4285b3b80e772c8e639b44
ec9727e33ae1f00f13ac2f24477e343ec0fe1586790dfdd8c1da0c40bf97dbad
0e228fa6afeaea338240378c1aaceda5d302bcf38d54d914e85893bfb7f96a39
Block Found!<jupyter_text>Our loop breaks after three iterations because we've satisfied the condition.***Up to this point we have just been hashing a string `'_block_hash'`. Lets now use our block.<jupyter_code># !!caution!! - setting difficulty too high could result in waiting until the heat death of the universe to find your block
difficulty = 4
nonce = 1
block_found = False

# this will loop until a block is found
while block_found == False:
    # add a nonce value to the block, here it is prepended
    work = hashlib.sha256(bytes(nonce) + pickled_block)
    
    # check to see if the hash meets the target
    if work.hexdigest()[:difficulty] == '0'*difficulty:
        block_found = True
        
        # output some data
        print('Difficulty: ' + str(difficulty))
        print('Nonce: ' + str(nonce))
        print('Proof-of-work: ' + work.hexdigest())
        
    # update the nonce before looping
    # this ensures the block is different before it is hashed again    
    nonce += 1
    
# you may have to wait 10-20 seconds for output to appear below<jupyter_output>Difficulty: 4
Nonce: 102591
Proof-of-work: 0000ee2830b3999eec469cc16a81f0eb6a163e84ece694803adaf3392b74804b<jupyter_text>Add a system timer for analysis Change the difficulty value above and compare the different outputs. Adding a timer can help with the comparison.<jupyter_code>difficulty = 4
nonce = 1
block_found = False

# access the system clock and record the start time
import time
start = time.time()
while block_found == False:
    work = hashlib.sha256(bytes(nonce) + pickled_block)
    if work.hexdigest()[:difficulty] == '0'*difficulty:
        end = time.time()
        block_found = True
        print('Difficulty: ' + str(difficulty))
        print('Nonce: ' + str(nonce))
        print('Proof-of-work: ' + work.hexdigest())
        
        # calculate and print the time; this will output in seconds
        print('Elapsed time: ' + str(end - start))
    nonce += 1<jupyter_output>Difficulty: 4
Nonce: 102591
Proof-of-work: 0000ee2830b3999eec469cc16a81f0eb6a163e84ece694803adaf3392b74804b
Elapsed time: 22.482012510299683<jupyter_text>Generate random blocks Re-running this will take similar times because we keep hashing the same block with the same nonce. In practice, once a block is found, the miners start mining the next block which contains different transactions. Introducing some random bytes into our block can help you with generating "new" blocks to analyse mining and difficulty performance.<jupyter_code># secrets provides access to a secure random number generator. 
# Refer to documentation for what this means. 
# Always be cautious when generating random data for important purposes like keys.
import secrets
# generate random data to append to a block
random_data = secrets.token_bytes(4)

difficulty = 4
nonce = 1
block_found = False
start = time.time()
while block_found == False:
    # the block now contains 4 random bytes of data
    work = hashlib.sha256(bytes(nonce) + pickled_block + random_data)
    if work.hexdigest()[:difficulty] == '0'*difficulty:
        end = time.time()
        block_found = True
        print('Difficulty: '+ str(difficulty))
        print('Nonce: ' + str(nonce))
        print('Proof-of-work: ' + work.hexdigest())
        print('Elapsed time: ' + str(end - start))
    nonce += 1<jupyter_output>Difficulty: 4
Nonce: 22947
Proof-of-work: 0000f9b8371941856efe3f94f09b36bbc9580de5451ab058f04c0cb53580154a
Elapsed time: 0.6624038219451904
{'fraction_non_alphanumeric': 0.06886773344136116, 'fraction_numerical': 0.1292282762811424, 'mean_word_length': 5.065724815724816, 'pattern_counts': {'":': 0, '<': 25, '<?xml version=': 0, '>': 25, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '8', 'dir': 'jupyter-structured-clean-dedup', 'id': '483137', 'license': 'MIT', 'n_tokens_mistral': 4120, 'n_tokens_neox': 3426, 'n_words': 1016, 'path': 'Tutorial 3 - Proof of work.ipynb', 'repo_name': 'millecodex/Delta2019', 'size': '9874'}