<jupyter_start><jupyter_text>Iterating Collections We saw how sequence types support iteration by being able to access elements by index. We could even write our custom sequence types by implementing the `__getitem__` method. But there are some limitations:* items must be numerically indexable, with indexing starting at `0`* cannot be used with unordered collections, such as sets If we think about iterating over a collection, what we really need is a way to request the **next** item in the collection.If we can do that, our collection does not require being indexable, nor does it need to be ordered (i.e. we don't need the notion of relative positions of elements in the container). This is exactly what iterables are in general - they provide a method that returns the "next" element in the collection. This approach works equally well with sequence type collections, as well as unordered collection types such as sets.Of course, the order in which **next** returns items from an unordered colllection is not known in advance - and we see that when we iterate over a set for example:<jupyter_code>s = {'x', 'y', 'b', 'c', 'a'}
for item in s:
    print(item)<jupyter_output>x
a
c
y
b<jupyter_text>As you can see the order in which the elements of the set was returned, did not match the order in which we added elements to the set. Furthermore, we cannot use indexing to access elements in a set:<jupyter_code>s[0]<jupyter_output><empty_output><jupyter_text>Rolling our own Next method Let's go ahead and define a kind of iterable ourselves. What we'll want to do is to have a container type of class that implements a `next` method, instead of that `__getitem__` method. Every time we call `next`, it should return the next element in the collection - so we'll have to keep track of where we are in the iteration somehow.Since `next` is a built-in function, which we'll look at in a bit, we'll use `next_` instead.<jupyter_code>class Squares:
    def __init__(self):
        self.i = 0
    
    def next_(self):
        result = self.i ** 2
        self.i += 1
        return result
sq = Squares()
sq.next_()
sq.next_()
sq.next_()<jupyter_output><empty_output><jupyter_text>How do we re-start the iteration from the beginning?We can't - we have to create a new instance of `Squares`:<jupyter_code>sq = Squares()
for i in range(10):
    print(sq.next_())<jupyter_output>0
1
4
9
16
25
36
49
64
81<jupyter_text>We even are able to iterate over the squares.But you'll notice that we essentially have an **infinite** number of items.We can fix that easily enough - by specifying a length when we create the collection, and raise an exception if `next_()` goes beyond the number of elements in the collection - we'll raise a `StopIteration` exception -- that's a built-in exception Python provides us specifically for this kind of scenario!!We'll even implement a `__len__` method to support the `len()` function:<jupyter_code>class Squares:
    def __init__(self, length):
        self.length = length
        self.i = 0
    
    def next_(self):
        if self.i >= self.length:
            raise StopIteration
        else:
            result = self.i ** 2
            self.i += 1
            return result           
        
    def __len__(self):
        return self.length
sq = Squares(3)
len(sq)
sq.next_()
sq.next_()
sq.next_()
sq.next_()<jupyter_output><empty_output><jupyter_text>So now, we can essentially loop over the collection in a very similar way to how we did it with sequences and the `__getitem__` method:<jupyter_code>sq = Squares(5)
while True:
    try:
        print(sq.next_())
    except StopIteration:
        # reached end of iteration
        # stop looping
        break<jupyter_output>0
1
4
9
16<jupyter_text>There are two issues here.The first is that the "iterable" `sq` has been exhausted - we can't just "re-start" the iteration:<jupyter_code>sq.next_()<jupyter_output><empty_output><jupyter_text>The second problem is that we can't use a `for` loop - Python does not know about our `next_()` method:<jupyter_code>for i in Squares(10):
    print(i)<jupyter_output><empty_output><jupyter_text>Of course if we had a `__getitem__` method, everything would work again - but remember that `__getitem__` means we have a sequence type. Although our Squares is actually a sequence, we want to look at a more general way of creating containers that are not necessarily sequences. Much like Python's `len()` function and the `__len__()` method, Python has a built-in `next()` function - it calls the `__next__()` method in our class if there is one.Let's see this:<jupyter_code>class Squares:
    def __init__(self, length):
        self.length = length
        self.i = 0
    
    def __next__(self):
        if self.i >= self.length:
            raise StopIteration
        else:
            result = self.i ** 2
            self.i += 1
            return result   
    
    def __len__(self):
        return self.length
sq = Squares(3)
next(sq)
next(sq)
next(sq)
next(sq)<jupyter_output><empty_output><jupyter_text>So that's nice, makes typing a bit easier - our loop we wrote earlier would look something like this now:<jupyter_code>sq = Squares(5)
while True:
    try:
        print(next(sq))
    except StopIteration:
        break<jupyter_output>0
1
4
9
16<jupyter_text>Does this mean Python can now iterate over an instance of Squares?<jupyter_code>for i in Squares(10):
    print(i)<jupyter_output><empty_output><jupyter_text>Nope, Python still does not recognize our class as an iterable collection.We need to do a little bit more work to get there.We also are going to need to look at how to "reset" the iteration without having to create a whole new object. You'll notice that technically our `Squares` class could be built as a sequence type - it was just a very simple example.Instead, let's build another collection that is a container of random numbers, but in no particular order.<jupyter_code>import random
class RandomNumbers:
    def __init__(self, length, *, range_min=0, range_max=10):
        self.length = length
        self.range_min = range_min
        self.range_max = range_max
        self.num_requested = 0
        
    def __len__(self):
        return self.length
    
    def __next__(self):
        if self.num_requested >= self.length:
            raise StopIteration
        else:
            self.num_requested += 1
            return random.randint(self.range_min, self.range_max)<jupyter_output><empty_output><jupyter_text>We can now iterate over instances of this object:<jupyter_code>numbers = RandomNumbers(10)
len(numbers)
while True:
    try:
        print(next(numbers))
    except StopIteration:
        break<jupyter_output>2
5
8
7
5
2
6
4
7
5<jupyter_text>We still cannot use a `for` loop, and if we want to 'restart' the iteration, we have to create a new object every time.<jupyter_code>numbers = RandomNumbers(10)
for item in numbers:
    print(item)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.07687886190544618, 'fraction_numerical': 0.00977151889639316, 'mean_word_length': 3.6031746031746033, 'pattern_counts': {'":': 0, '<': 52, '<?xml version=': 0, '>': 55, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '14', 'dir': 'jupyter-structured-clean-dedup', 'id': '57650', 'license': 'Apache-2.0', 'n_tokens_mistral': 2122, 'n_tokens_neox': 2032, 'n_words': 877, 'path': 'python-tuts/1-intermediate/02 - Iterables and Iterators/01 - Iterating Collections.ipynb', 'repo_name': 'timgates42/Artificial-Intelligence-Deep-Learning-Machine-Learning-Tutorials', 'size': '6959'}