<jupyter_start><jupyter_text>**CST2312 Class19 - Regex and APIs** *This notebook includes CST2312 homework Assignment 6 which includes four exercises to be completed ⚪ 6.1,  ⚪ 6.2, ⚪ 6.3, and  ⚪ 6.4.* --- Raw strings    First, a reminder about *raw strings* which are delineated with the `r` character preceding the first quotation (single or double) mark of a *string*.  When a string is denoted as a *raw string* then the contents inside the quaotes are interpreted exactly as they are.  That is, special control characters such as `\n`, `\t`, etc.  are interpreted as the text characters '\' and 'n' or '\' and 't' respectively and not as their control characters (new line, or tab).<jupyter_code>print('This string\nprints two lines.')
print(r'This string\n does not print two lines.')<jupyter_output><empty_output><jupyter_text>---  ***Regular Expressions******Regex*** **Reading** from the required textbook: ( [https://www.py4e.com/lessons/](https://www.py4e.com/lessons/))* [Regular Expressions](https://www.py4e.com/lessons/regex) (Chapter 12)* [Data Science Cheat Sheet Python Regular Expressions](https://www.dataquest.io/wp-content/uploads/2019/03/python-regular-expressions-cheat-sheet.pdf) So far we have been using methods like `split` and `find` to extract portions of strings or to answer a question of whether a particular item / string is part of a list-set-tuple-dictionary / longer string.Regular Expressions-------------------Regular expressions (regexes or re’s) constitute an extremely powerful, flexible and concise language for matching elements in text ranging from a few characters to complex patterns. While mastering the syntax of the regular expression language does require climbing a learning curve, this learning curve is not particularly steep, and a newcomer can find herself performing useful tasks with regular expressions almost immediately. Efforts spent learning regular expressions quickly pay off--tasks that are well suited for regular expressions abound. Indeed, regular expressions are one of the most useful computer skills, and an absolutely critical tool for data scientists.This document will present basic regular expression syntax and cover common use cases for regular expressions: pattern matching, filtering, data extraction, and string replacement. We will present examples using python’s standard [re regular expression library](http://docs.python.org/library/re.html).We will discuss Python libraries in detail later.Many examples from from this [Google tutorial](https://developers.google.com/edu/python/regular-expressions). Pay attention that tutorial itself uses a 2.x version of Python, thus, several statements (for example, print()) look differently than in the later versions of Python. In this notebook, the examples from the Google tutorial are converted to the current Python version.   Searching strings using regexes The regular expression library `re` must be imported into your program before you can use it. The simplest use of the regular expression library is the `search()` function.<jupyter_code># first import the library
import re
# First let's create a string variable and try to use `split` to find 'cat'
inputStr = 'An example word:cat!!'
input_list = inputStr.split(' ')
for word in inputStr :
    print(word)

inputStr = 'An example word:cat!!'
if re.search(r'cat', inputStr):
  print(inputStr)

print ("Done with the example")
inputStr = 'An example word: cat!!'
if re.search(r'dog', inputStr):
  print(inputStr)

print ("Done with the example")
inputStr = 'An example word: cat!!'
if re.search(r'word: \w\w\w', inputStr):
  print(inputStr)
  
print ("Done with the example")
inputStr = 'An example word:dog, anda, cat!!'
if re.search(r'word:\w\w\w', inputStr):
  print(inputStr)

print ("Done with the example")
the_text = 'there are many words here some with three characters and also this phrase "stuffH dog", plus cat!!'
if re.search(r'stuffH \w\w\w', the_text):
  print("I found the pattern you asked about.")

print ("Done with the example")<jupyter_output><empty_output><jupyter_text>We can store the result of `re.search(pat, str)` in a variable.In Python a regular expression search is typically written as:`match = re.search(pat, str)`The code `match = re.search(pat, str)` stores the search result in a variable named "match". The `re.search()` method takes a regular expression pattern and a string and searches for that pattern within the string. If the search is successful, `search()` returns a match object or `None` otherwise.Then the `if`-statement tests the match -- if `True` the search succeeded and `match.group()` is the matching text (e.g. 'word: cat'). Otherwise if the match is `False` (`None` to be more specific), then the search did not succeed, and there is no matching text.The 'r' at the start of the pattern string designates a python "raw" string which passes through backslashes without change which is very handy for regular expressions. It is recommended that you always write pattern strings with the 'r' just as a habit.<jupyter_code>inputStr = 'An example word: cat!!'
match = re.search(r'word: \w\w\w', inputStr) ## both 'cat' and 'dog' are 3-letter words
match
print(match)<jupyter_output><re.Match object; span=(11, 20), match='word: cat'><jupyter_text>See Geeks for Geeks on the use of arguments in the group() function for re.search() at ▶ https://www.geeksforgeeks.org/re-matchobject-group-function-in-python-regex/ *Here is the example used by Geeks for Geeks*<jupyter_code>"""We create a re.MatchObject and store it in
   match_object variable
   the '()' parenthesis are used to define a
   specific group"""
 
match_object = re.match(r'(\w+)@(\w+)\.(\w+)', '<EMAIL>')
 
""" w in above pattern stands for alphabetical character
    + is used to match a consecutive set of characters
    satisfying a given condition
    so w+ will match a consecutive set of alphabetical characters"""
 
# for entire match
print(match_object.group())
# also print(match_object.group(0)) can be used
 
# for the first parenthesized subgroup
print(match_object.group(1))
 
# for the second parenthesized subgroup
print(match_object.group(2))
 
# for the third parenthesized subgroup
print(match_object.group(3))
 
# for a tuple of all matched subgroups
print(match_object.group(1, 2, 3))<jupyter_output><empty_output><jupyter_text>---<jupyter_code>inputStr = 'an example word: dog!!'
match = re.search(r'word: \w\w\w', inputStr)
# If-statement after search() tests if it succeeded
if match:
  print ('found', match.group()) ## 'found word:cat'
else:
  print ('did not find')
print ("Done with the example")
line = "Cats are smarter than dogs"

matchObj = re.match( r'(.*) are (.*?) .*', line) # regExs within a regEx

if matchObj:
   print ("matchObj.group() : ", matchObj.group(0))
   print ("matchObj.group(0) : ", matchObj.group(0))
   print ("matchObj.group(1) : ", matchObj.group(1))
   print ("matchObj.group(2) : ", matchObj.group(2))
else:
   print ("No match!!")<jupyter_output><empty_output><jupyter_text>Basic Patternssee [Python Documentation](https://docs.python.org/3/library/re.html)The power of regular expressions is that they can specify patterns, not just fixed characters. Here are the most basic patterns which match single chars:* a, X, 9, < -- ordinary characters just match themselves exactly. The meta-characters which do not match themselves because they have special meanings are: . ^ $ * + ? { [ ] \ | ( ) (details below)* . (a period) -- matches any single character except newline '\n'* \w -- (lowercase w) matches a "word" character: a letter or digit or underbar [a-zA-Z0-9_]. Note that although "word" is the mnemonic for this, it only matches a single word char, not a whole word. \W (upper case W) matches any non-word character.* \b -- boundary between word and non-word* \s -- (lowercase s) matches a single whitespace character -- space, newline, return, tab, form [ \n\r\t\f]. \S (upper case S) matches any non-whitespace character.* \t, \n, \r -- tab, newline, return* \d -- decimal digit [0-9] (some older regex utilities do not support but \d, but they all support \w and \s)* ^ = start, $ = end -- match the start or end of the string* \ -- inhibit the "specialness" of a character. So, for example, use \. to match a period or \\ to match a slash. If you are unsure if a character has special meaning, such as '@', you can put a slash in front of it, \@, to make sure it is treated just as a character.  Basic ExamplesThe basic rules of regular expression search for a pattern within a string are:* The search proceeds through the string from start to end, stopping at the first match found* All of the pattern must be matched, but not all of the string* If `match = re.search(pat, str)` is successful, match is not `None` and in particular `match.group()` is the matching text<jupyter_code>## Search for pattern 'iii' in string 'piiig'.
## All of the pattern must match, but it may appear anywhere.
## On success, match.group() is matched text.

matched_string = re.search(r'iii', 'piiig')
if matched_string:
  print ('found, matched_string.group() == "iii"')
  print (matched_string.group())
match = re.search(r'igs', 'piiig')
if not match:  
  print ('not found, match == None')
  print (re.search(r'igs', 'piiig'))


## . = any char but \n
match = re.search(r'..g', 'piiig') 
if match:
  print ('found, match.group() == "iig"')
  print (match.group())

print("All done.")
## \d = digit char, \w = word char
match = re.search(r'\d\d\d', 'p123g')
if match:
  print ('found, match.group() == "123"')
  print (match.group())


match = re.search(r'\w\w\w', '@@abcd!!')
if match:
  print ('found, match.group() == "abc"')
  print (match.group())
match = re.search(r'\w\w\w\w', '@@abcd!!')
if match:
  print ('found, match.group() == "abcd"')
  print (match.group())<jupyter_output><empty_output><jupyter_text>Email Example 1Extract the account name and the domain from the email address<jupyter_code>email = '<EMAIL>'

# your code here<jupyter_output><empty_output><jupyter_text>***don't look here*** ``` a solution to exercise 1 email_match=re.search(r'\S+@\S+', email)print('found email address ' + email_match.group(0))match_list=re.split(r"@", email_match.group(0))print('the name and domain are: ', match_list)```<jupyter_code><jupyter_output><empty_output><jupyter_text>Email Example 2Suppose you want to find the email address inside the string 'xyz <EMAIL> data  science'. Here's an attempt using the pattern r'\w+@\w+':<jupyter_code>emailText = 'xyz <EMAIL> data science'
match = re.search(r'\w+@\w+', emailText)
if match:
  print (match.group())<jupyter_output><empty_output><jupyter_text>The search does not get the whole email address in this case because the `\w` does not match the `'-'` or `'.'` in the address. We'll fix this using the regular expression features below.**Square Brackets**Square brackets can be used to indicate a set of chars, so `[abc]` matches `'a'` or `'b'` or `'c'`. The codes `\w`, `\s` etc. work inside square brackets too with the one exception that dot (`.`) just means a literal dot. For the emails problem, the square brackets are an easy way to add `'.'` and `'-'` to the set of chars which can appear around the `@` with the pattern `r'[\w.-]+@[\w.-]+'` to get the whole email address:<jupyter_code>emailText = 'xyz <EMAIL> data science'
match = re.search(r'[\w.-]+@[\w.-]+', emailText)
if match:
  print (match.group())<jupyter_output><empty_output><jupyter_text>We will use `group()` function to extract the account name and the domain from the email address<jupyter_code>emailText = 'xyz <EMAIL> data science'
match = re.search(r'([\w.-]+)@([\w.-]+)', emailText)
if match:
  print ("email address:\t", match.group())
  print ("email account:\t", match.group(1))
  print ("email domain:\t", match.group(2))<jupyter_output><empty_output><jupyter_text>Iteration using regular expressionsIf we want to extract data from a string in Python we can use the `findall()` or `finditer()` methods to extract all of the substrings which match a regular expression. These two methods produce resutls of different types. Let’s use the example of wanting to extract anything that looks like an email address from any line regardless of format. For example, we want to pull the email addresses from each of the following lines:<jupyter_code>emailText = '''xyz <EMAIL> data science or 
<EMAIL>, also we can mention <EMAIL>'''
match = re.search(r'([\w.-]+)@([\w.-]+)', emailText)
if match:
  print ("email address:\t", match.group())
  print ("email account:\t", match.group(1))
  print ("email domain:\t", match.group(2))
matches = re.finditer(r'[\w.-]+@[\w.-]+', emailText)
for match in matches:
    print(match.group())
matches = re.finditer(r'([\w.-]+)@([\w.-]+)', emailText)
for match in matches:
    print(match.group(),"\t", match.group(1),"\t", match.group(2))
print (matches)  
matches = re.findall(r'([\w\.-]+)@([\w\.-]+)', emailText)
print (matches)  
for match in matches:
  print (match[0])  ## username
  print (match[1])  ## host
matches = re.findall(r'[\w\.-]+@[\w\.-]+', emailText)
print (matches)  
for match in matches:
  print (match)<jupyter_output><empty_output><jupyter_text>--- Some [practice examples](https://www.programiz.com/python-programming/regex) using different Regex patterns. --- Additional reading on an 'almost perfect' regular expression approach to finding email addresses ▶ [Almost Perfect Email Regex](https://emailregex.com/)  Additional reading on the complexities of thorough regex rules for email addresses ▶ [O'Relly Regular Expressions Cookbook 4.1, Validate Email Addresses](https://www.oreilly.com/library/view/regular-expressions-cookbook/9781449327453/ch04s01.html) by <NAME> and <NAME>. ---  Create a variable containing regular expression`re.compile()` method.<jupyter_code># We are looking for binary numbers
regex = re.compile(r'[10]+')
text = "asddf1101110100011abd1111panos0000"
matches = regex.finditer(text)
for match in matches:
    print(match.group())
emailText = '''xyz <EMAIL> data science or 
<EMAIL>, also we can mention <EMAIL>'''
# We are looking for email addresses
regex = re.compile(r'[\w\.-]+@[\w\.-]+')
matches = regex.finditer(emailText)
for match in matches:
    print(match.group())
# We look for money figures, either integers, or with 1 or 2 decimal
# digits
regex = re.compile(r'\$\d+(\.\d\d?)?')
text = '$1200.23 is the price today. $1400 was the price yesterday'
matches = regex.finditer(text)
for match in matches:
    print(match.group())
# This code is going to generate no matches
regex = re.compile(r'Ra*nd.*m R[egex]')
text = "CUNY, Citytech, Information and Data Management, <EMAIL>"
matches = regex.finditer(text)
for match in matches:
    print(match.group())

print ("The end")<jupyter_output><empty_output><jupyter_text>Regular expressions are typically case-sensitive.<jupyter_code># Regular expressions are compiled into pattern objects
# Regular expressions are case-sensitive
regex = re.compile(r'in.*on')
text = "CUNY, Citytech, Information and Data Management, <EMAIL>"
matches = regex.finditer(text)
for match in matches:
    print(match.group())
    
print('The end')<jupyter_output><empty_output><jupyter_text>But we can specify that they are case-insensitive, using the flag re.IGNORECASE<jupyter_code># Unless we specify that they are case-insensitive, using the flag re.IGNORECASE
regex = re.compile('in.*on',re.IGNORECASE)
text = "CUNY, Citytech, Information and Data Management, <EMAIL>"
matches = regex.finditer(text)
for match in matches:
    print(match.group())
    
print('The end')<jupyter_output><empty_output><jupyter_text>Greedy vs. Non-GreedySuppose you have text with tags in it: `foo and so on`Suppose you are trying to match each tag with the pattern '()' -- what does it match first?<jupyter_code>text = "<b>foo</b> and <i>so on</i>"
regex = re.compile(r'(<.*>)')
match = regex.search(text)
print (match.group())

print('The end')<jupyter_output><empty_output><jupyter_text>The result is a little surprising, but the greedy aspect of the `.*` causes it to match the whole `foo and so on` as one big match. The problem is that the `.*` goes as far as is it can, instead of stopping at the first `>` (aka it is "greedy").There is an extension to regular expression where you add a `?` at the end, such as `.*?` or `.+?`, changing them to be non-greedy. Now they stop as soon as they can. So the pattern `'()'` will get just `''` as the first match, and `''` as the second match, and so on getting each  pair in turn. The style is typically that you use a `.*?`, and then immediately its right look for some concrete marker (> in this case) that forces the end of the `.*?` run.<jupyter_code>text = "<b>foo</b> and <i>so on</i>"
regex = re.compile(r'(<.?>)')
match = regex.search(text)
print (match.group())

print('The end')
text = "<b>foo</b> and <i>so on</i>"
regex = re.compile(r'(<.?>)')
matches = regex.finditer(text)
for match in matches:
    print(match.group())

print('The end')
text = "<b>foo</b> and <i>so on</i>"
regex = re.compile(r'(<.+?>)')
matches = regex.finditer(text)
for match in matches:
    print(match.group())

print('The end')<jupyter_output><empty_output><jupyter_text>--- The `re.match()` function returns a match object on success, `None` on failure. We use `group(num)` function of match object to get matched expression.`group(num=0)`: This method returns entire match (or specific subgroup num)<jupyter_code>inputStr = 'An example word: cat!!'
# If-statement after search() tests if it succeeded
if re.search(r'word: \w\w\w', inputStr):
  print ('found', match.group()) ## 'found word:cat'
else:
  print ('did not find')
print ("Done with the example")<jupyter_output>found word: cat
Done with the example<jupyter_text>---  Regular Expression Functions  Analyzing* `.match()`* `.search()`* `.finditer()`* `.findall()`* `.compile()`[https://docs.python.org/3/library/re.html](https://docs.python.org/3/library/re.html)Let me separate these functions into three groups: 1. `.compile()`2. `.finditer()`, `.findall()`3. `.match()`, `.search()`  1. `re.compile()`  Regular expression compilationRegular expression compilation produces a Python object that can be used to do all sort of regular expression operations. What is the benefit of that as long as we can use `re.match` and `re.search` directly? This technique is convenient in case we want to use a regular expression more than once. It makes our code efficient and more readable.  `re.compile(pattern, flags=0)`Compile a regular expression pattern into a regular expression object, which can be used for matching using its `match()`, `search()` and other methods. This sequence<jupyter_code>inStr = 'This is my 123 example string'
prog = re.compile(r'\d+')
result = prog.search(inStr)
result<jupyter_output><empty_output><jupyter_text>is equivalent to<jupyter_code>result = re.search(r'\d+', inStr)
result<jupyter_output><empty_output><jupyter_text>By using `re.compile()` and saving the resulting regular expression object for reuse you can make your code more efficient when the expression will be used several times in a single program.  2. `re.finditer()`, `re.findall()`  `.finditer()` VS `.findall()`* `.findall()` returns a **list** of all matches of a regex in a string.* `.finditer()` returns an iterator that yields regex matches.https://realpython.com/regex-python/https://realpython.com/regex-python-part-2/https://www.8bitavenue.com/difference-between-re-search-and-re-match-in-python/ `re.findall(, )` returns a list of all non-overlapping matches of `` in ``. It scans the search string from left to right and returns all matches in the order found:<jupyter_code>re.findall(r'#(\w+)#', '#foo#.#bar#.#baz#')<jupyter_output><empty_output><jupyter_text>In this case, the specified regex is `(\w+)`. The matching strings are `'foo'`, `'bar'`, and `'baz'`. But the hash (``) characters don’t appear in the return list because they’re outside the grouping parentheses.If `` contains more than one capturing group, then `re.findall()` returns a list of tuples containing the captured groups. The length of each tuple is equal to the number of groups specified:<jupyter_code>re.findall(r'(\w+),(\w+)', 'foo,bar,baz,qux,quux,corge')
re.findall(r'(\w+),(\w+),(\w+)', 'foo,bar,baz,qux,quux,corge')<jupyter_output><empty_output><jupyter_text>`re.finditer(, )` scans `` for non-overlapping matches of `` and returns an iterator that yields the match objects from any it finds. It scans the search string from left to right and returns matches in the order it finds them:<jupyter_code>it = re.finditer(r'\w+', '...foo,,,,bar:%$baz//|')
it
for i in re.finditer(r'\w+', '...foo,,,,bar:%$baz//|'):
  print(i.group())<jupyter_output><empty_output><jupyter_text>`re.findall()` and `re.finditer()` are very similar, but they differ in two respects:1. `re.findall()` returns a **list**, whereas `re.finditer()` returns an **iterator**.The items in the list that `re.findall()` returns are the actual matching strings, whereas the items yielded by the iterator that `re.finditer()` returns are match objects.Any task that you could accomplish with one, you could probably also manage with the other. Which one you choose will depend on the circumstances. However, a lot of useful information can be obtained from a match object. If you need that information, then `re.finditer()` will probably be the better choice. For example, you can use the `.group()` method to return Match object.  Match objectsIf a match is found when using `re.finditer()`, `re.match` or `re.search`, we can use some useful methods provided by the match object. Here is a short list of such methods, you may check the reference section for more details…* `group()` returns the part of the string matched by the entire regular expression* `group(1)` returns the text matched by the second capturing group* `start()` and `end()` return the indices of the start and end of the substring matched by the capturing group.  3. `re.match()`, `re.search()`  .match() VS .search()This is the trickiest part. A good explanation is provided in this tutorial: https://www.8bitavenue.com/difference-between-re-search-and-re-match-in-python/ which is paraphrased here.Additional information can also be found at ▶https://realpython.com/regex-python/https://realpython.com/regex-python-part-2/ When searching or matching, regular expression operations can take optional flags or modifiers. The following two modifiers are the most used ones…* `re.M multiline`* `re.I ignore case`  .match()`re.match` matches an expression at the beginning of a string. If a match is found, a match object is returned, otherwise `None` is returned. If the input is a multiline string (i.e. starts and ends with three double quotes) that does not change the behavior of the match operation. `re.match` always tries to match the beginning of the string. In regular expressions syntax, the control character (^) is used to match the beginning of a string. If this character is used with re.match, it has no effect. The syntax for `re.match` operation is as follows… `mobj = re.match(pat, str, flag)`* pat: regular expression pattern to match* str: string in which to search for the pattern* flags: one or more modifiers, for example re.M|re.I  `.match(pattern, string, flags=0)`Looks for a regex match at the beginning of a string.If zero or more characters at the beginning of *string* match the regular expression *pattern*, return a corresponding match object. Return `None` if the string does not match the pattern; note that this is different from a zero-length match.This is identical to `re.search()`, except that `re.search()` returns a match if the regular expression *pattern* matches anywhere in *string*, whereas `re.match()` returns a match only if the regular expression *pattern* matches at the beginning of *string*.  .search()`re.search` attempts to find the first occurrence of the pattern anywhere in the input string as opposed to the beginning. If the search is successful, `re.search` returns a match object, otherwise it returns `None`. The syntax for re.search operation is as follows… `mobj = re.search(pat, str, flag)`* pat: regular expression pattern to search for* str: string in which to search for the pattern* flags: one or more modifiers, for example re.M|re.I  `.search(pattern, string, flags=0)`Scans a string for a regex match.Scan through *string* looking for the **first** location where the regular expression *pattern* produces a match, and return a corresponding match object. Return `None` if no position in the string matches the pattern.You can use this function in a Boolean context like a conditional statement that checkes if a pattern is found in the input string (`True`) or not (`False`).  Substitution The `re.sub(pat, replacement, str)` function searches for all the instances of pattern in the given string, and replaces them. The replacement string can include `'\1'`, `'\2'` which refer to the text from `group(1)`, `group(2)`, and so on from the original matching text.Here's an example which searches for all the email addresses, and changes them to keep the user `(\1)` but have `nyc.gov` as the host.<jupyter_code>text = '''xyz <EMAIL> data science or 
<EMAIL>, also we can mention <EMAIL>'''
## re.sub(pat, replacement, str) -- returns new string with all replacements,
## \1 is group(1), \2 group(2) in the replacement
print (re.sub(r'([\w\.-]+)@([\w\.-]+)', r'\<EMAIL>', text))<jupyter_output><empty_output><jupyter_text>--- Another explanation of the basics of regular expressions with examples of different functions and their uses as well as descriptions of pattern-matching can be found at Geeks for Geeks post on Regular Expressions for Python here ▶ https://www.geeksforgeeks.org/regular-expression-python-examples-set-1/ ---  CST2312 - Homework Assignment 6.1Imagine we want to conceal all phone numbers in a document.Read the below multiline string and substitute all phone numbers with 'XXX-XXX-XXXX'<jupyter_code>raw_text = """512-234-5234
foo
bar
124-512-5555
biz
125-555-5785
679-397-5255
2126660921
212-998-0902
888-888-2222
801-555-1211
802 555 1212
803.555.1213
(804) 555-1214
1-805-555-1215
1(806)555-1216
807-555-1217-1234
808-555-1218x1234
809-555-1219 ext. 1234
work 1-(810) 555.1220 #1234
"""<jupyter_output><empty_output><jupyter_text>Solution for Assignment 6.1<jupyter_code>## your solution here<jupyter_output><empty_output><jupyter_text>*solution provided after assignments are submitted* ---  **Web APIs** ---  **Reading** from the required textbook: ( [https://www.py4e.com/lessons/](https://www.py4e.com/lessons/))* [Web APIs](https://www.py4e.com/lessons/servces) (Chapter 14)  Interacting with Web APIs We are going to examine now the concept of a Web API. A web API is similar to a function call, but the "function" that we call is located in another machine, and we submit the parameters of the function through the web. ---  First Example: GeoIP resolution We will start with an example that is doing a "geoIP" resolution: it takes the IP of a computer and returns back its location. Students, you will need to get an API here from here: https://ipstack.com/<jupyter_code># We first import the requests library
import requests
api_key = "?"  # replace with ? for general users
url = 'http://api.ipstack.com/check?access_key='+api_key
resp = requests.get(url)
print(resp)
# the straightforward version of the last snippet
import requests
response = requests.get('http://api.ipstack.com/check?access_key=<KEY>')
print(response)
# The resp object encapsulates the "response" of the server
# Notice the status code that is displayed. 
# Code 200 means that things went fine
# Code 404 means that the URL was not found
# Codes 5xx mean that something went wrong
resp
if (resp.status_code == 200):
    print("Everything was ok:", resp.status_code)
else:
    print("There was a problem:", resp.status_code)
# Let's see the content of the response
# As you can see, it contain the JSON response
data = resp.text
data
# We want to transform the JSON file into a Python dictionary object
# We use the response.json() command to get back a dictionary
data = resp.json()
# Now data is a Python dictionary
data
# And we can access the fields of the JSON as we normally access Python dictionary entries
print("Lon:", data["longitude"], "Lat:", data["latitude"])
other_data = response.json()
other_data<jupyter_output><empty_output><jupyter_text>And in one piece:<jupyter_code>import requests
api_key="?"  # replace the ? with your API key
url = 'http://api.ipstack.com/check?access_key='+api_key
resp = requests.get(url)
data = resp.json()
print("Lon:", data["longitude"], "Lat:", data["latitude"])<jupyter_output><empty_output><jupyter_text>Using Parameters with API Calls The first API call that we tried was very simple. We just fetched a URL. Now let's see a URL that accepts as input a set of **parameters**. We have already seen this concept with functions; the parameters of the API calls are the exact equivalent but for Web APIs, which are, at their core, functions that we call over the web.   Example: OpenWeatherMap Let's try to query OpenWeatherMap now, to get data about the weather. [Documentation](http://openweathermap.org/currentgeo). Below you can find the URL that you can copy and paste in your browser, to get the weather for New York. You will notice that it contains parameters as part of the URL, including an `appid` which is a key that is used to limit the number of calls that can be issued by a single application. Try the URL in your browser. Also try to change the query parameter `q` and change it from `New%20York,NY` to something different. (Note: The `%20` is a transformation for the space (` `) character in URLs.) Students, you will need to get an API key from here: https://openweathermap.org/price Below you can find the same code, but now we have a Python dictionary to organize and list the parameters.<jupyter_code>import requests

openweathermap_url = "http://api.openweathermap.org/data/2.5/weather"
parameters = {
    'q'     : 'New York, NY, USA',
    'units' : 'metric',
    'mode'  : 'json',
    'appid' : API_key
}
resp = requests.get(openweathermap_url, params=parameters)
data = resp.json()
data
resp = requests.get("http://api.openweathermap.org/data/2.5/weather?q=New%20York,NY,USA&units=imperial&mode=json&appid="+API_key")
data = resp.json()
data<jupyter_output><empty_output><jupyter_text>CST2312 - Homework Assignment 6.2 a. Extract the current temperature from the returned JSON response.<jupyter_code># your code here
data['main']['feels_like']<jupyter_output><empty_output><jupyter_text>b. Extract the description of the current weather<jupyter_code># your code here
data['weather'][0]['description']<jupyter_output><empty_output><jupyter_text>c. Try to change the units to `metric` and repeat<jupyter_code># your code here<jupyter_output><empty_output><jupyter_text>d. Get the weather for San Francisco, CA  Solution for Assignment 6.2<jupyter_code># your code here<jupyter_output><empty_output><jupyter_text>*solution provided after assignments are submitted*  CST2312 - Homework Assignment 6.3 * Study the documentation of the API ([Documentation](http://openweathermap.org/currentgeo)). Change the API call to use the longitute and latitude.  Solution for Assignment 6.3<jupyter_code># your code here<jupyter_output><empty_output><jupyter_text>*solution provided after assignments are submitted*  CST2312 - Homework Assignment 6.4 Read the location of your computer using the GeoIP API. Then use the OpenWeatherMap to query the API and fetch the temperature for the location returned by the GeoIP API. For this exercise, you will need to learn to read variables from a Web API (geoip) and use them as input in another (openweathermap)  Solution for Assignment 6.4<jupyter_code>#your code here<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.12044150387256593, 'fraction_numerical': 0.012762221316358848, 'mean_word_length': 4.875460574797347, 'pattern_counts': {'":': 0, '<': 183, '<?xml version=': 0, '>': 184, 'https://': 24, 'lorem ipsum': 0, 'www.': 13, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '35', 'dir': 'jupyter-structured-clean-dedup', 'id': '204936', 'license': 'MIT', 'n_tokens_mistral': 9810, 'n_tokens_neox': 9331, 'n_words': 3912, 'path': 'CST2312_Class19.ipynb', 'repo_name': 'ProfessorPatrickSlatraigh/CST2312', 'size': '32206'}