<jupyter_start><jupyter_text>Chapter 01 -- Data Structures Topics Covered:	list 	indexing 	tuple 	dictionary 	sequences 	set 	Resources This chapter briefly touches on Python's data structures.  As the name suggests, data structures are containers for data and other objects.  We introduce these objects since they form the building-blocks for other structures discussed in subsequent chapters.A cursory examination is needed, so a quick review is in order.  Later, you will want to refer back to these since they aid the creation of DataFrames and other useful objects.  DataFrames are built on top of Python and Numpy modules which are the predominate structures discussed in Chapter 4 onwards.  Python has four main data structures.  There are:        1. list    2. tuple    3. set    4. dictionary  listA list contains an ordered collection of items.  You determine the order.  In Python, items in a list are separarated by commas and enclosed in square brackets.  Lists are mutable, meaning you can add,  remove, or alter items.<jupyter_code>a_list = ['ale', 'lager', 'stout', 'hefeweizen', 'stout']
print('There are', len(a_list), 'items in "a_list"')<jupyter_output>There are 5 items in "a_list"<jupyter_text>In the example below, the parameter:    end=' 'for the print() method suppresses the default new-line character (CR/LF).<jupyter_code>print("A list of beers include:")
for i in a_list:
    print(i, end=' ')<jupyter_output>A list of beers include:
ale lager stout hefeweizen stout<jupyter_text>IndexingPython provides indexing methods for a number of objects, including lists.  SAS has a similiar construct \_n\_ for the Data Step.  Both indexing methods act as keys providing access to an individual item (or set of items).  In Python's case, the default start index position is 0, and for SAS it is 1.The SAS code example in the cell below is an imperfect analogy for a Python list, since the SAS logic uses a variable to hold the beer_type values.  Nonetheless, each program illustrates access to an array 'item' by indexing.Chapter 05 -- Understanding Indexes" has more details. Python uses zero as its index start position, in contrast to SAS' \_n\_ which uses a start position of 1.  Almost every example of SAS DO loops, by convention, use a start position of 1.<jupyter_code>print('Value for beer type is:', a_list[0])<jupyter_output>Value for beer type is: ale<jupyter_text>````    /******************************************************/    /* c02_retrieve_list_item_by_index_position.sas       */    /******************************************************/    4         data beers;    5         length beer_type $ 10;    6         input beer_type $ @@;    7             8         if _n_ = 1 then    9            put 'Value for beer type is: ' beer_type;    10            11        list;    12        datalines;    Value for beer type is: ale    RULE:     ----+----1----+----2----+----3----+----4----+----5----+----6----+----7----+----8----+-    13        ale lager stout hefeweizen stout ````        The next two examples illustrate the .append() and .sort() attributes for a list.  They produce no visible output, so use the print() method to view results.<jupyter_code>a_list.append('malt')<jupyter_output><empty_output><jupyter_text><jupyter_code>a_list.sort(reverse=True)<jupyter_output><empty_output><jupyter_text>A for statement used to interate along the items in the list.  For statements are documented  here.<jupyter_code>print("A list of beers include:")
for i in a_list:
    print(i, end=' ')<jupyter_output>A list of beers include:
stout stout malt lager hefeweizen ale<jupyter_text>The .count() attribute returns the number of items in an object.  Here it is used to return the number of occurrences of a value by having the list element as an argument.<jupyter_code>print('a_list count for stout is:', a_list.count('stout'))<jupyter_output>a_list count for stout is: 2<jupyter_text>The example below illustrates Python's flexibility.  Since nearly everything in Python is an object, you can have a list containing other lists.  The built-in len() method provides a method to count the number of items in a list.<jupyter_code>b_list = ['ales', 23, a_list]
print(b_list)
print('Item count for b_list is:', len(b_list))<jupyter_output>['ales', 23, ['stout', 'stout', 'malt', 'lager', 'hefeweizen', 'ale']]
Item count for b_list is: 3<jupyter_text>tupleA tuple is similar to a list, but unlike lists, are immutable.   Tuples are defined by a list of comma-separated items inside a set of optional parentheses.  Legibility of code demands their use, however.A common use case for tuples is where Python statements or user-defined functions can assume that the items will not change, for example the names of the months.<jupyter_code>dishes = ('eggs', 'green ham', 'biscuits', 'grits', 'steak')
print('The breakfast menu has:', len(dishes), 'items')<jupyter_output>The breakfast menu has: 5 items<jupyter_text><jupyter_code>more_dishes = ('pancakes', 'cupcakes', 'twinkies', 'dishes')
for i in more_dishes:
    print(i, end=' ')<jupyter_output>pancakes cupcakes twinkies dishes<jupyter_text>The tuple within a tuple remains one that is indexed<jupyter_code>len(more_dishes)<jupyter_output><empty_output><jupyter_text>How to count all of the items in the tuple.      1. get the number of items in the tuple: more_dishes.      2. subtract 1 since the 'tuple-within-a-tuple' item is not counted.     3. add the length of the tuple more_dishes starting at position [3]<jupyter_code>print('The number of items in more_dishes is:', len(more_dishes)-1+len(more_dishes[3]))<jupyter_output>The number of items in more_dishes is: 9<jupyter_text>dictionaryA dictionary provides a look-up method through key/value pairs.  Keys must be unique.  Keys must be immutable objects such as lists, however, values can be either mutable or immutable objects.Key/value pairs are specified as:x = {key1: value1, key2: value2, key_n: value_n}  Create a dictionary<jupyter_code>capital = {'Oregon' : 'Salem',
           'Washington' : 'Olympia',
           'California' : 'Sacrament',
           'Nevada' : 'Carson City'
          }
print(type(capital))<jupyter_output><class 'dict'><jupyter_text>Print a dictionary value by key<jupyter_code>print('The capital of Nevada is', capital['Nevada'])<jupyter_output>The capital of Nevada is Carson City<jupyter_text>Add a key/value pair to the dictionary<jupyter_code>capital['Colorado'] = 'Denver'<jupyter_output><empty_output><jupyter_text>Printing key/value pairs.  String formatting is covered in Chapter 03 -- Data Types and Formatting<jupyter_code>print('Number of value/pairs in the dictionary capital is {}:'.format(len(capital)))<jupyter_output>Number of value/pairs in the dictionary capital is 5:<jupyter_text>Delete a value.<jupyter_code>del capital['California']<jupyter_output><empty_output><jupyter_text>Many Python operations are silent unless an error is raised.  Check the length of the dictionary following the delete operation.<jupyter_code>print('Number of value/pairs in the dictionary capital is {}:'.format(len(capital)))<jupyter_output>Number of value/pairs in the dictionary capital is 4:<jupyter_text>sequencesThere are three basic sequence types:    list     tuple      range    A feature for sequences is membership testing.  Using operators such as in, not in, and concatenation, you can test for the presence or absences of values.  Implicit in these examples are boolean values(True False) being returned.  Boolean operators are discussed in Chapter 03, Boolean operators.The precedence order for sequence operations is found here.   The example below does an item membership test for the a_list list object.<jupyter_code>item = 'ale'
if (item in a_list):
    print('found')
else:
    print('not found')<jupyter_output>found<jupyter_text>Item membership test in a tuple using the boolean and operator.<jupyter_code>item1 = 'eggs' 
item2 = 'cupcakes'
if (item1 and item2 in more_dishes):
    print('found')
else:
    print('not found')<jupyter_output>found<jupyter_text>A for statement to iterate over the key value pairs in the 'capital' dictionary.<jupyter_code>for state, capital in capital.items():
    print('The capital of {} is {}'.format(state, capital))<jupyter_output>The capital of Washington is Olympia
The capital of Nevada is Carson City
The capital of Colorado is Denver
The capital of Oregon is Salem<jupyter_text>if logic used for membership testing.  The if/elif statements are documented  here.<jupyter_code>if 'Nevada' in capital:
    print("Nevada's capital:", capital['Nevada'])<jupyter_output><empty_output><jupyter_text>Using the boolean not operator for membership test<jupyter_code>'California' not in capital<jupyter_output><empty_output><jupyter_text>setA set object is an unordered collection of distinct objects. They are used for membership testing, removing duplicates from a sequence, and computing mathematical operations such as intersection, union, difference, and symmetric differences.  Another common use case is when the existence of an object is more important than order or obtaining a count of occurrences of items.    Membership test using the in operator for a set<jupyter_code>months1 = set(['January', 'February', 'March', 'April', 'May', 'June'])

'Jan' in months1<jupyter_output><empty_output><jupyter_text>copy() method for sets<jupyter_code>months2 = months1.copy()<jupyter_output><empty_output><jupyter_text>The remove() method for sets to delete an item<jupyter_code>months1.remove('February')<jupyter_output><empty_output><jupyter_text>The add() method for adding an item to a set<jupyter_code>months2.add('July')<jupyter_output><empty_output><jupyter_text>The three examples above do not produce any visible output.  The sets Month1 and Month2 are displayed below.<jupyter_code>print('The set months1 contains:', months1)
print('The set months2 contains:', months2)<jupyter_output>The set months1 contains: {'January', 'June', 'May', 'April', 'March'}
The set months2 contains: {'June', 'March', 'May', 'April', 'July', 'January', 'February'}<jupyter_text>Now we can find the intersection of the sets 'months1' and 'months2'<jupyter_code>months1 & months2<jupyter_output><empty_output><jupyter_text>Test if the set months2 is a super-set of the set months1<jupyter_code>months2.issuperset(months1)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.10166875663161956, 'fraction_numerical': 0.008970772643966432, 'mean_word_length': 4.972350230414746, 'pattern_counts': {'":': 0, '<': 105, '<?xml version=': 0, '>': 105, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '30', 'dir': 'jupyter-structured-clean-dedup', 'id': '336212', 'license': 'MIT', 'n_tokens_mistral': 3173, 'n_tokens_neox': 3019, 'n_words': 1197, 'path': '.ipynb_checkpoints/Chapter01 - Data Structures-checkpoint.ipynb', 'repo_name': 'ivenkwan/skillmodernization', 'size': '10367'}