<jupyter_start><jupyter_text>pyeither demoSource: https://github.com/SegFaultAX/pyeither IntroductionThis is a basic introduction into error handling and pyeither. Some key takeaways are:* There are several popular methods for handling errors in large programs (exceptions, return codes, etc.) each with their own benefits and drawbacks* The common ways of handling errors don't compose well and/or incur lots of boilerplate (boilerplate = bugs)* Building a new error handling pattern from scratch is easy (and fun!)* pyeither is an alternative to the "normal" way we construct programs which favors composition and equational reasoning<jupyter_code>!pip install pyeither>=0.0.3

import attr
import either<jupyter_output><empty_output><jupyter_text>Handling errorsBroadly speaking, there are 2 styles of managing failures in code. The most popular in the Python world is via `Exception`s. Let's see a couple of examples of how that's typically done.<jupyter_code>def catch_locally(a, b):
    """Handle errors locally, propagate values only"""
    
    try:
        return a / b
    except ZeroDivisionError:
        return None

def main_v1():
    a, b = int(input("Number 1: ")), int(input("Number 2: "))
    result = catch_locally(a, b)
    if result is None:
        print("Something bad happened")
    else:
        print("And the answer is: {}".format(result))

main_v1()
def catch_nonlocally(a, b):
    """Propagate exceptions and values to my caller"""
    
    return a / b

def main_v2():
    a, b = int(input("Number 1: ")), int(input("Number 2: "))
    try:
        result = catch_nonlocally(a, b)
        print("And the answer is: {}".format(result))
    except ZeroDivisionError:
        print("Something bad happened")
        
main_v2()<jupyter_output>Number 1: 10
Number 2: 0
Something bad happened<jupyter_text>For our purposes the only interesting distinction between these examples is whether the function we're depdending on catches exceptions locally and propagates only values back to us, or expects us to handle both exceptions and successful results. Something worth reflecting on, particularly in dynamic languages, is the inability for the programmer to discover which exceptions a piece of code they're calling into is likely to raise. Languages like Java (via checked exceptions) do have a mechanism for statically notifying calling code of possible exceptions, but these features have other dubious drawbacks (namely, exception creep in types).Another common technique for propagating errors is to use multiple return with an error part and a result part. This is more common in langauges like Go, but it's still occasionally pops up in Python. Here's an example of what that might look like.<jupyter_code>def div(a, b):
    try:
        return True, a / b
    except ZeroDivisionError:
        return False, None

def main_v3():
    a, b = int(input("Number 1: ")), int(input("Number 2: "))
    success, result = div(a, b)
    
    if success:
        print("And the answer is: {}".format(result))
    else:
        print("Something bad happened")

main_v3()<jupyter_output>Number 1: 10
Number 2: 0
Something bad happened<jupyter_text>Both styles of error handling and propagation have their strengths. Exceptions in particular allow you to be relatively dynamic in the way you handle errors by allowing them to unwind the callstack until a relevant receiver is found, or else crashing the application if there isn't one. Sometimes failing hard and immediately is the best option, but sometimes otherwise recoverable errors propagate in unexpected or undesirable ways.Return codes and success flags have the great benefit of being highly explicit and obvious. If most (or all) functions return a result and an indication of success, then it's clear from reading the code where the error is handled because handling and/or propagation is done locally and immediately. The main drawback is the significant amount of repetition that return codes cause, and the incredible level of care the programmer needs to take in refactoring around that boilerplate.As a slightly hyperbolic example, consider a function that chains a computation through a series of functions that can fail. Without some other mechanism for propagating failures, you might end up with functions that looks like this:```pythondef m():    success, result1 = f()    if not success:        return False, None        success, result2 = g(result1)    if not success:        return False, None        success, result3 = h(result2)    if not success:        return False, None        return True, result3``` An alternative approach: first-class failure effects!Whether you agree or not that the above solutions are subpar, I'd like to present to you and alternative way you can tackle this problem with concepts borrowed from functional programming. To demonstrate the technique, we are going to build a basic library for generically modeling failure in our applications. Step 1: The `Perhaps` typeAt a high level, what we'd like to have is a way to represent a possible result of some computation. That result can be in one of two states: it either failed (in which case we have nothing to work with), or we have a successful result which can be fed into the next part of our program. So let's start by implementing a simple type that represents exactly that, which we'll call `Perhaps`.*Why "perhaps"? Because Perhaps we have a value, or Perhaps we don't.*<jupyter_code>@attr.s(frozen=True, repr=False)
class Perhaps:
    """A class that represents a possibly-failed computation"""
    
    value = attr.ib()
    success = attr.ib()
    
    @property
    def failure(self):
        return not success
    
    def __repr__(self):
        if self.success:
            return "Success({})".format(self.value)
        else:
            return "Failure"
        
fail = Perhaps(None, False)

def succeed(v):
    """Lift a value into Perhaps"""
    
    return Perhaps(v, True)
# A constant indicating a failed computation
print(fail)

# A value representing a successful computation of 3
print(succeed(3))
def div(a, b):
    return fail if b == 0 else succeed(a / b)

print(div(10, 2))
print(div(10, 0))<jupyter_output>Success(5.0)
Failure<jupyter_text>Step 2: A function `inside``Perhaps` represents a very simple concept with 2 possible states:* It's a value such as `Success(3)`, or* It's a `Failure`As you can see from the `div` example, it's very easy to re-implement functions using this notion of success/failure. Either you return a value via `succeed(x)` or you return `fail` to indicate that something has gone wrong.Unfortunately, `Perhaps` values are a little hard to work with. Specifically, if I have a function like `div`, how do I use the successful result of `div(10, 2)`? Calling that function returns `Success(5.0)`, not `5.0`, so a simple expression like this will not work:```pythondiv(10, 2) * 100  Won't work, the result of div(10, 2) is Success(5.0)```What we'd like to be able to do is have a way to apply a normal function **inside** of a successful result. "Normal" in this context is a function that does not use `Perhaps` in any way, for example:```pythondef times_10(n):    return n * 10```For the same reason `div(10, 2) * 100` won't work, neither will `times_10(div(10, 2))` because again, `times_10` is working on normal values instead of values wrapped in `Perhaps`. The solution to this problem is simple: create a method `inside` that, given a function, does nothing on `Failure` or else applies the function to the value inside of a `Success`.<jupyter_code>@attr.s(frozen=True, repr=False)
class Perhaps:
    """A class that represents a possibly-failed computation"""
    
    value = attr.ib()
    success = attr.ib()
    
    @property
    def failure(self):
        return not success
    
    ### NEW CODE ###
    
    def inside(self, f):
        """Apply `f` to my value if I have one, or else fail"""
        
        if self.success:
            return succeed(f(self.value))
        else:
            return fail

    ### NEW CODE ###
    
    def __repr__(self):
        if self.success:
            return "Success({})".format(self.value)
        else:
            return "Failure"
        
fail = Perhaps(None, False)
def times_10(n):
    return n * 10

a = div(10, 2)
print(a)

b = a.inside(times_10)
print(b)

print(fail.inside(times_10))<jupyter_output>Success(5.0)
Success(50.0)
Failure<jupyter_text>Step 3: `collapse` the resultsOk, now our `Perhaps` type is really starting to shape up. To re-iterate what we've done so far:* A value of `Perhaps` is either a successful result, or a failure* We can run normal (non-`Perhaps`) functions on a successful result using `inside`So the next question is: if we can use `inside` to make normal functions work with `Perhaps`, what about functions that **do** use `Perhaps` to indicate their own success or failure? Consider this function:```pythondef even(n):    if n % 2 == 0:        return succeed(n)    else:        return fail```This function works exactly like we'd expect:* An even value `N` will return `succeed(N)`* An odd value will return `Failure`So what happens if we try to use `inside` with `even`?<jupyter_code>def even(n):
    if n % 2 == 0:
        return succeed(n)
    else:
        return fail

print(succeed(2).inside(even))
print(succeed(3).inside(even))<jupyter_output>Success(Success(2))
Success(Failure)<jupyter_text>Clearly something has gone a little weird here. Remember that `inside` just applies the function to the possible value inside of the `Perhaps`. In the first case, `even(2)` returns `Success(2)`, therefore `succeed(2).inside(even)` returns `Success(Success(2))`, a success nested within a success. The latter case is also fairly obvious: `even(3)` returns `Failure` and therefore the expression `succeed(3).inside(even)` results in `Success(Failure)`, a failure nested within a success.So let's create a method `collapse` that will reduce 1 layer of nesting within a `Perhaps` value. In other words, a value of `Success(Success(X))` when collapsed will simply be `Success(X)`. Likewise, a failure inside of a success such as `Success(Failure)` will collapse to just `Failure`. Note that because `Failure`s cannot be further nested, collapsing a `Failure` is just `Failure`.<jupyter_code>@attr.s(frozen=True, repr=False)
class Perhaps:
    """A class that represents a possibly-failed computation"""
    
    value = attr.ib()
    success = attr.ib()
    
    @property
    def failure(self):
        return not success
    
    def inside(self, f):
        """Apply `f` to my value if I have one, or else fail"""
        
        if self.success:
            return succeed(f(self.value))
        else:
            return fail
        
    ### NEW CODE ###
    
    def collapse(self):
        if self.success:
            return self.value
        else:
            return fail

    ### NEW CODE ###
    
    def __repr__(self):
        if self.success:
            return "Success({})".format(self.value)
        else:
            return "Failure"

fail = Perhaps(None, False)
print(succeed(2).inside(even))
print(succeed(3).inside(even))

print(succeed(2).inside(even).collapse())
print(succeed(3).inside(even).collapse())
print(fail.collapse())<jupyter_output>Success(Success(2))
Success(Failure)
Success(2)
Failure
Failure<jupyter_text>Step 4: Do this `and_then` thatOnce again, let's recap what we've actually created so far:* A value of `Perhaps` is either a successful result, or a failure* We can run normal (non-`Perhaps`) functions on a successful result using `inside`* We can reduce one layer of nesting within a `Perhaps` using `collapse`Let's look at a larger example of a processing pipeline. This pipeline will have 2 important characteristics:1. Each step in the pipeline can fail or succeed using `Perhaps`2. The output of one step in the pipeline is the input to the next<jupyter_code>## Sample config files used for the next couple sections

# Create a valid config file
with open("example.yaml", "w") as fd:
    fd.write("""---\nname: mkbernard\nage: 30\n""")

# Create an invalid config file
with open("example_invalid.yaml", "w") as fd:
    fd.write("""---\n  name: mkbernard\nage: 30\n""")
import os
import yaml

@attr.s(frozen=True)
class Person:
    name = attr.ib()
    age = attr.ib()
    
    @classmethod
    def from_config(cls, config):
        return Person(config["name"], config["age"])

def attempt(f, *args, **kwargs):
    try:
        return succeed(f(*args, **kwargs))
    except:
        return fail

def ensure_path(path):
    return succeed(path) if os.path.isfile(path) else fail
    
def read_content(path):
    return succeed(open(path).read())

def parse_yaml(content):
    return attempt(yaml.safe_load, content)

def load_person(config):
    return attempt(Person.from_config, config)

def process(path):
    return (succeed(path)
        .inside(ensure_path).collapse()
        .inside(read_content).collapse()
        .inside(parse_yaml).collapse()
        .inside(load_person).collapse())
        
print(process("example.yaml"))
print(process("example_invalid.yaml"))
print(process("not a path"))<jupyter_output>Success(Person(name='mkbernard', age=30))
Failure
Failure<jupyter_text>This is a much more realistic example of something you're likely to do with `Perhaps`. You can trivially decompose a complex pipeline into individual pieces. If you look at the `process` function, something should immediately jump out at you: `inside(f).collapse()` is repeated over and over. This is because every time you use a `Perhaps`-returning function, you need to `collapse` the resulting nested value. This pattern is so common, that we can create a helper method on `Perhaps` called `and_then` that will automatically collapse the nesting.<jupyter_code>@attr.s(frozen=True, repr=False)
class Perhaps:
    """A class that represents a possibly-failed computation"""
    
    value = attr.ib()
    success = attr.ib()
    
    @property
    def failure(self):
        return not success
    
    def inside(self, f):
        """Apply `f` to my value if I have one, or else fail"""
        
        if self.success:
            return succeed(f(self.value))
        else:
            return fail
    
    def collapse(self):
        if self.success:
            return self.value
        else:
            return fail

    ### NEW CODE ###
    
    def and_then(self, f):
        return self.inside(f).collapse()
    
    ### NEW CODE ###
    
    def __repr__(self):
        if self.success:
            return "Success({})".format(self.value)
        else:
            return "Failure"

fail = Perhaps(None, False)
def process(path):
    return (succeed(path)
        .and_then(ensure_path)
        .and_then(read_content)
        .and_then(parse_yaml)
        .and_then(load_person))
        
print(process("example.yaml"))
print(process("example_invalid.yaml"))
print(process("not a path"))<jupyter_output>Success(Person(name='mkbernard', age=30))
Failure
Failure<jupyter_text>Step 5: pyeither and beyondAnd now our `Perhaps` type is complete! Let's recap one last time what we've created:* A value of `Perhaps` is either a successful result, or a failure* We can run normal (non-`Perhaps`) functions on a successful result using `inside`* We can reduce one layer of nesting within a `Perhaps` using `collapse`* We can apply a `Perhaps`-returning function and collapse the result automatically using `and_then`In just a few lines of code, we've built a very simple abstraction that allows us to model failure in our programs in a highly composable way. The key insight was to represent successful and unsuccessful results in a type, and then provide some simple functions for chaining additional actions off of those possibly-present results.---By now you're probably wondering what pyeither is, and why you should use it over something like `Perhaps`. Indeed, pyeither works almost identically to `Perhaps` with one extremely important difference: `Either` values can carry context about **why** they failed! As a final example, let's look at our above code written using `Either` from pyeither.<jupyter_code>import os
import yaml

@attr.s(frozen=True)
class Person:
    name = attr.ib()
    age = attr.ib()
    
    @classmethod
    def from_config(cls, config):
        return Person(config["name"], config["age"])

def ensure_path(path):
    return either.should(os.path.isfile(path), path, "is not a valid file")
    
def read_content(path):
    return either.succeed(open(path).read())

def parse_yaml(content):
    return either.lmap(
        lambda e: "invalid yaml: " + str(e),
        either.attempt(yaml.safe_load, content))

def load_person(config):
    return either.lmap(
        lambda e: "invalid user config: " + str(e),
        either.attempt(Person.from_config, config))

def process(path):
    return (either.succeed(path)
            .chained()
            .bind(ensure_path)
            .bind(read_content)
            .bind(parse_yaml)
            .bind(load_person)
            .unchain())
        
print(process("example.yaml"))
print(process("example_invalid.yaml"))
print(process("not a path"))<jupyter_output>Right(result=Person(name='mkbernard', age=30))
Left(error='invalid yaml: expected \'<document start>\', but found \'<block mapping start>\'\n  in "<unicode string>", line 3, column 1:\n    age: 30\n    ^')
Left(error='is not a valid file')
{'fraction_non_alphanumeric': 0.0955355607771219, 'fraction_numerical': 0.00790876267980973, 'mean_word_length': 3.6545745532141907, 'pattern_counts': {'":': 0, '<': 35, '<?xml version=': 0, '>': 36, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '10', 'dir': 'jupyter-structured-clean-dedup', 'id': '351427', 'license': 'MIT', 'n_tokens_mistral': 5011, 'n_tokens_neox': 4750, 'n_words': 2064, 'path': 'pyeither_demo.ipynb', 'repo_name': 'SegFaultAX/pyeither', 'size': '17449'}