<jupyter_start><jupyter_text>Selecting values from an array<jupyter_code>import numpy as np<jupyter_output><empty_output><jupyter_text>We use arrays all the time, in data science.One of the most common tasks we have to do on arrays, is to select values.We do this with *array slicing*.We do array slicing when we follow the array variable name with`[` (open square brackets), followed by something to specifywhich elements we want, followed by `]` (close square brackets.The simplest case is when we want a single element from a one-dimensional array.  In that case the thing between the `[` and the `]` is the *index* of the value that we want.The *index* of the first value is 0, the index of the second value is 2, and so on.We start by loading data from a Comma Separated Value file (CSVfile).This is summary data that [<NAME>](https://asrosen.com) downloaded from for an analysis he reported in [a 2018paper](https://asrosen.com/wp-content/uploads/2018/07/postprint_rmp-1.pdf). Thedata file here is from the [supplementarymaterial](https://www.tandfonline.com/doi/suppl/10.1080/02602938.2016.1276155);it has the average rating across academic discipline for all professors ina particular discipline who have more than 20 ratings.See [the dataset page]({{ site.baseurl }}/data/rate_my_professors) for moredetail.If you are running on your laptop, you should download the[rate_my_course.csv]({{ site.baseurl }}/data/rate_my_course.csv) file to thesame directory as this notebook.Don't worry about the next cell.  It just loads the data we need from a data file.  We will come on the machinery to do this very soon.<jupyter_code># This code we have not covered yet.  We will soon.
# It loads the data file, and makes some arrays.
# Load the library for reading data files.
import pandas as pd
# Read the file.
courses = pd.read_csv('rate_my_course.csv')
# Sort the courses by number of rated teachers, most first.
courses_by_n = courses.sort_values(
    'Number of Professors', ascending=False)
# Select the top eight easiest courses.
big_courses = courses_by_n.head(8)
big_courses
# Again - don't worry about this code - we will cover it later.
# Put the columns into arrays
disciplines = big_courses['Discipline'].values
easiness = big_courses['Easiness'].values
quality = big_courses['Overall Quality'].values<jupyter_output><empty_output><jupyter_text>We now have the names of the disciplines with the largest number of professors.<jupyter_code>disciplines<jupyter_output><empty_output><jupyter_text>Here we get the first value.  This value is at index 0.<jupyter_code># Get the first value (at index position 0)
disciplines[0]<jupyter_output><empty_output><jupyter_text>Notice that this is another way of writing:<jupyter_code>disciplines.item(0)
# Get the second value (at index position 1)
disciplines[1]
# Get the third value (at index position 2)
disciplines[2]<jupyter_output><empty_output><jupyter_text>At first this will take some time to get used to, that the firstvalue is at index position 0.  There are good reasons for this,and many programming languages have this convention, but it does a while to get this habit of mind. Notice too that we use square brackets for indexing.We have seen square brackets before, when we create lists.  For example, we can create a list with `my_list = [1, 2, 3]`. (We usually then create an array with something like `my_array = np.array(my_list)`.  This is *a different use of square brackets*. When we create a list, the square brackets tell Python that the elements between the brackets should be the elements of the list.  This use is called a *list literal* expression.  The square brackets follow an equal sign, or some other operator, or start the line.When we use square brackets for indexing, the square brackets always follow an expression that returns an array.  For example, consider this line:<jupyter_code>disciplines[2]<jupyter_output><empty_output><jupyter_text>`disciplines` is an expression that returns an array.  Therefore the open square brackets follows this expression, and therefore, Python can tell that this use of square brackets means *select an element or elements from the array*.So we have seen different uses of square brackets:* Creating a list (a *list literal*); (often uses in making arrays).* Indexing into an array.Python can tell which of these two we mean, from the context.  Index with negative numbers If we know how many elements the array has, then we can get the last element by using the number of elements, minus one (why?).Here the number of elements is:<jupyter_code>len(disciplines)<jupyter_output><empty_output><jupyter_text>So, the last element of the array is at index position 7:<jupyter_code>disciplines[7]<jupyter_output><empty_output><jupyter_text>In fact, there is a short cut for getting elements at the end of the array, and that is to use an offset with a minus in front.  The number is then the offset from one past the last item.  For example, here is another way to get the last element:<jupyter_code>disciplines[-1]<jupyter_output><empty_output><jupyter_text>The last but one element:<jupyter_code>disciplines[-2]<jupyter_output><empty_output><jupyter_text>Index with slices Sometimes we want more than one element from the array.  For example, we might want the first 4 elements from the array.  We can get these using an array *slice*.  It looks like this:<jupyter_code># All the elements from offset zero up to (not including) 4
disciplines[0:4]
# All the elements from offset 4 up to (not including) 8
disciplines[5:11]<jupyter_output><empty_output><jupyter_text>You can omit the first number, if you mean to start at offset 0:<jupyter_code>disciplines[:4]<jupyter_output><empty_output><jupyter_text>You can omit the last number if you mean to end at the last element of the array:<jupyter_code>disciplines[4:]<jupyter_output><empty_output><jupyter_text>Indexing with Boolean arraysWe often want to select several elements from an array according to some criterion.The most common way to do this, is to do array slicing, usinga Boolean array between the square brackets.It can be easier to understand this by example than by description. Here are the "Overall Quality" scores for the ten largest courses:<jupyter_code>quality<jupyter_output><empty_output><jupyter_text>These are the quality ratings corresponding to the `disciplines` we saw earlier.  The top (largest) discipline is:<jupyter_code>disciplines[0]<jupyter_output><empty_output><jupyter_text>The Overall Quality rating for that course is:<jupyter_code>quality[0]<jupyter_output><empty_output><jupyter_text>You have already come across Boolean arrays.These are arrays of `True` and `False` values.Here is a Boolean array, created from applying a comparison to an array:<jupyter_code>greater_than_3 = easiness > 3
greater_than_3<jupyter_output><empty_output><jupyter_text>This has a `True` value at the positions of elements > 3, and `False` otherwise. As you have already seen, we can do things like count the numberof `True` values in the Boolean array:<jupyter_code>np.count_nonzero(greater_than_3)<jupyter_output><empty_output><jupyter_text>Now let us say that we wanted to get the elements from `easiness`that are greater than 4.   That is, we want to get the elementsin `easiness` for which the corresponding element in`greater_than_3` is `True`.We can do this with *Boolean array indexing*.  The Boolean array goes between the square brackets, like this:<jupyter_code>easiness
greater_than_3
easiness[greater_than_3]<jupyter_output><empty_output><jupyter_text>We have selected the numbers in `easiness` that are greater than 3. We can use this same Boolean array to index into another array.  For example, here we show the discipline *names* corresponding to the courses with Easiness scores greater than 3:<jupyter_code>disciplines[greater_than_3]<jupyter_output><empty_output><jupyter_text>Just to confirm that these are the disciplines with "Easiness" ratings greater than 4, here are the first ten rows of the table.  These are the rows corresponding to numbers in the arrays.<jupyter_code>big_courses.head(10)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.06896975657732973, 'fraction_numerical': 0.010695844602901402, 'mean_word_length': 5.762261014131338, 'pattern_counts': {'":': 0, '<': 86, '<?xml version=': 0, '>': 88, 'https://': 3, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '21', 'dir': 'jupyter-structured-clean-dedup', 'id': '468817', 'license': 'CC-BY-4.0', 'n_tokens_mistral': 2434, 'n_tokens_neox': 2301, 'n_words': 1038, 'path': 'notebooks/03/array_indexing.ipynb', 'repo_name': 'matthew-brett/cfd-uob', 'size': '8140'}