<jupyter_start><jupyter_text>Deadlocks Deadlocks are yet another problematic condition that might arise as the result of poorly synchronized code.  A simple example Let's start by analyzing a simple example: movement between two "bank accounts":<jupyter_code>ACCOUNTS = {
    'a1': 1000,
    'a2': 1000
}
ITERATIONS = {
    'a1': 0,
    'a2': 0,
}
def move_funds(_from, _to, expected):
    name = threading.current_thread().name

    while True:
        amount = random.randint(0, 100)
        ACCOUNTS[_from] -= amount
        ACCOUNTS[_to] += amount
        total = sum(ACCOUNTS.values())
        if total != expected:
            print(f"{name} found an inconsistent balance: ${total}")
            break
        ITERATIONS[_from] += 1
t1 = threading.Thread(target=move_funds, name='Thread-1', args=('a1', 'a2', 2000))
t2 = threading.Thread(target=move_funds, name='Thread-2', args=('a2', 'a1', 2000))
t1.start()
t2.start()<jupyter_output>Thread-2 found an inconsistent balance: $1930<jupyter_text>Fixing it with Locks We've already learned about Locks, so we can use those to try synchronizing the access to the accounts. We'll create 2 locks, on for each account:<jupyter_code>ACCOUNTS = {
    'a1': 1000,
    'a2': 1000
}
lock_1 = threading.Lock()
lock_2 = threading.Lock()
def move_funds(_from, _to, lock_from, lock_to, expected):
    name = threading.current_thread().name
    iterations = 0
    while True:
        amount = random.randint(0, 100)
        with lock_from:
            with lock_to:
                ACCOUNTS[_from] -= amount
                ACCOUNTS[_to] += amount
                total = sum(ACCOUNTS.values())
                if total != expected:
                    print(f"{name} found an inconsistent balance: ${total}")
                    break
        iterations += 1
        if iterations > 1_000_000:
            print(f'{name} reached iteration limit. Stopping...')
            return
t1 = threading.Thread(target=move_funds, name='Thread-1', args=('a1', 'a2', lock_1, lock_2, 2000))
t2 = threading.Thread(target=move_funds, name='Thread-2', args=('a2', 'a1', lock_1, lock_2, 2000))
t1.start()
t2.start()
[t.join() for t in (t1, t2)];
sum(ACCOUNTS.values())
lock_1.locked(), lock_2.locked()<jupyter_output><empty_output><jupyter_text>Worked! Now the access to the accounts is protected by the locks. But there's a very dangerous potential situation hidden in our code. If we make just 1 tiny change, altering the order of the locks that are passed to our threads, we'll find ourselves deadlocked:<jupyter_code>ACCOUNTS = {
    'a1': 1000,
    'a2': 1000
}
# t1 = threading.Thread(target=move_funds, name='Thread-1', args=('a1', 'a2', lock_1, lock_2, 2000))
# t2 = threading.Thread(target=move_funds, name='Thread-2', args=('a2', 'a1', lock_2, lock_1, 2000))
# t1.start()
# t2.start()

# [t.join() for t in (t1, t2)];<jupyter_output><empty_output><jupyter_text>What are Deadlocks and when do they happen? a deadlock is a state in which each member of a group is waiting for another member, including itself, to take action, such as sending a message or more commonly releasing a lock ![title](deadlock.gif) In our code, the issue is the ordering in which the locks are acquired. This is a VERY difficult thing to spot. ![title](deadlocks.png)  How to prevent Deadlock  It's very hard to prevent Deadlocks. One simple technique is to always use timeouts when trying to acquire locks. If you're trying to acquire N shared locks, if you can't acquire all N, you can release them all and start over. Let's see an example:<jupyter_code>ACCOUNTS = {
    'a1': 1000,
    'a2': 1000
}

LOCK_TIMEOUT = .001

lock_1 = threading.Lock()
lock_2 = threading.Lock()
def move_funds(_from, _to, lock_from, lock_to, expected):
    name = threading.current_thread().name
    iterations = 0
    while True:
        amount = random.randint(0, 100)
        locked = False
        while not locked:
            res1 = lock_from.acquire(timeout=LOCK_TIMEOUT)
            res2 = lock_to.acquire(timeout=LOCK_TIMEOUT)
            if all([res1, res2]):
                # Success! We acquired both locks
                locked = True
            else:
                # Release locks "partially" acquired
                if res1:
                    lock_from.release()
                if res2:
                    lock_to.release()
        # locked is True, we're safe
        ACCOUNTS[_from] -= amount
        ACCOUNTS[_to] += amount
        total = sum(ACCOUNTS.values())
        lock_from.release()
        lock_to.release()
        if total != expected:
            print(f"{name} found an inconsistent balance: ${total}")
            break
        iterations += 1
        if iterations > 100_000:
            print(f'{name} reached iteration limit. Stopping...')
            return
t1 = threading.Thread(target=move_funds, name='Thread-1', args=('a1', 'a2', lock_1, lock_2, 2000))
t2 = threading.Thread(target=move_funds, name='Thread-2', args=('a2', 'a1', lock_2, lock_1, 2000))
t1.start()
t2.start()
[t.join() for t in (t1, t2)];
sum(ACCOUNTS.values())
lock_1.locked(), lock_2.locked()
As you can see, we've just prevented the deadlock. The key code piece that prevents the deadlock is the following:
# locked = False
# while not locked:
#     res1 = lock_from.acquire(timeout=LOCK_TIMEOUT)
#     res2 = lock_to.acquire(timeout=LOCK_TIMEOUT)
#     if all([res1, res2]):
#         # Success! We acquired both locks
#         locked = True
#     else:
#         # Release locks "partially" acquired
#         if res1:
#             lock_from.release()
#         if res2:
#             lock_to.release()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.11800141743444366, 'fraction_numerical': 0.03437278525868179, 'mean_word_length': 2.9365411436541144, 'pattern_counts': {'":': 1, '<': 16, '<?xml version=': 0, '>': 18, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '4', 'dir': 'jupyter-structured-clean-dedup', 'id': '18390', 'license': 'Apache-2.0', 'n_tokens_mistral': 1924, 'n_tokens_neox': 1769, 'n_words': 551, 'path': '3 Deadlocks.ipynb', 'repo_name': 'khannadheeraj/Concurrency-in-Python', 'size': '5644'}