<jupyter_start><jupyter_text>Working with RESTFul APIs in Python --- <NAME>**CTO at Top20Talent**---* Twitter/Github: [@littlepea](https://twitter.com/littlepea12)* Email: [<EMAIL>](mailto:<EMAIL>)---![](https://www.top20talent.com/wp-content/uploads/2017/09/logotype-1.png)  Outline* What are RESTFul APIs?* Consuming APIs* Exposing APIs* Best Practices  What are RESTFul APIs? An architectural style called **REST** (**RE**presentational **S**tate **T**ransfer) advocates that web applications should use HTTP as it was **originally envisioned**. Lookups should use `GET` requests. `PATCH`, `POST`, and `DELETE` requests should be used for **mutation**, **creation**, and **deletion** respectively.  An API can be considered “RESTful” if it has the following features*:* **Client–server**    - The client handles the front end the server handles the backend and can both be replaced independently of each other.* **Stateless**    - No client data is stored on the server between requests and session state is stored on the client.* **Cacheable**    - Clients can cache response (just like browsers caching static elements of a web page) to improve performance.    ---[*] not a complete list just the main ones  Example: [OpenSkills API from DataAtWork](https://github.com/workforce-data-initiative/skills-api/wiki/API-Overview)This API can be used to search Jobs and Skills. These are the two main resources of http://api.dataatwork.org/v1/ :* /jobs/    * /jobs/autocomplete    * /jobs/:id:        * /jobs/:id:/related_skills* /skills/    * /skills/autocomplete    * /skills/:id:        * /skills/:id:/related_jobs* etc...  Consuming APIs Let's try to consume the OpenSkills API from our Python code.We'll try to find top skills most relevant to any job title (ex: `Software Architect`).  Using requests libraryWe'll try to make an HTTP GET request to the `/jobs/autocomplete` endpoint:<jupyter_code>import requests

response = requests.get(
    'http://api.dataatwork.org/v1/jobs/autocomplete', 
    {
        'begins_with': 'Software Architect'
    }).json()<jupyter_output><empty_output><jupyter_text>Now let's see the response:<jupyter_code>response<jupyter_output><empty_output><jupyter_text>Then let's search for related skills by `job_id`:<jupyter_code>first_job = response[0]
job_id = first_job['uuid']

response = requests.get(f'http://api.dataatwork.org/v1/jobs/{job_id}/related_skills').json()<jupyter_output><empty_output><jupyter_text>Now let's see the response:<jupyter_code>response<jupyter_output><empty_output><jupyter_text>Let's print the top 10 skills from the response:<jupyter_code>skills = response['skills']
top_skills = [skill['skill_name'] for skill in skills][:10]
top_skills<jupyter_output><empty_output><jupyter_text>Adding busines logic Let's say we want to have a custom weighted formula for scoring the top skills:* importance is on a scale 0-5* level is on a scale 0-10* level has 50% more weight than importanceWe can add a scoring function and use it for forting the skills from the API response:<jupyter_code>def skill_score(skill):
    importance = skill['importance'] * 2
    level = skill['level']
    score = importance * 0.4 + level * 0.6
    return score


top_skills = [skill['skill_name'] for skill in reversed(sorted(skills, key=skill_score))][:10]<jupyter_output><empty_output><jupyter_text>As a result we have an improved set of top skills:<jupyter_code>top_skills<jupyter_output><empty_output><jupyter_text>Putting it togetherIn real-world applications, you might put all this code in one service function which will give you the same result:<jupyter_code>def get_top_skills_for_job_title(title):
    jobs = requests.get('http://api.dataatwork.org/v1/jobs/autocomplete', {'begins_with': title}).json()
    job = jobs[0]
    job_id = job['uuid']

    skills = requests.get(f'http://api.dataatwork.org/v1/jobs/{job_id}/related_skills').json()['skills']
    top_skills = [skill['skill_name'] for skill in reversed(sorted(skills, key=skill_score))][:10]

    return top_skills


get_top_skills_for_job_title('Software Engineer')<jupyter_output><empty_output><jupyter_text>Separate the API logic from the Application logic  Create "Skill" entity and API adapter<jupyter_code>from collections import namedtuple


Skill = namedtuple('Skill', ['name', 'score'])
class DataAtWorkAPIAdapter:
    API_ROOT = 'http://api.dataatwork.org/v1/'

    @classmethod
    def _build_url(cls, collection=None, id=None, related=None):
        detail_path = f'/{id}' if id else ''
        related_path = f'/{related}' if related else ''
        return f'{cls.API_ROOT}{collection}{detail_path}{related_path}'

    @classmethod
    def _request(cls, collection=None, id=None, related=None, data=None):
        url = cls._build_url(collection, id, related)
        return requests.get(url, data).json()

    @classmethod
    def jobs(cls, title):
        return cls._request('jobs/autocomplete', data={'begins_with': title})

    @classmethod
    def job(cls, job_id):
        return cls._request('jobs', job_id)

    @classmethod
    def job_by_title(cls, title):
        jobs = cls.jobs(title)
        return jobs[0]

    @classmethod
    def related_skills(cls, job_id):
        return cls._request('jobs', job_id, 'related_skills')['skills']

    @classmethod
    def related_skills_by_title(cls, title):
        job_id = cls.job_by_title(title)['uuid']
        related_skills = cls.related_skills(job_id)
        return [Skill(s['skill_name'], skill_score(s)) for s in related_skills]<jupyter_output><empty_output><jupyter_text>Inject the adapter as "skills_service"<jupyter_code>def get_top_skills_for_job_title(title, skills_service=DataAtWorkAPIAdapter):
    skills = skills_service.related_skills_by_title(title)
    top_skills = [skill for skill in reversed(sorted(skills, key=lambda x: x.score))][:10]
    return top_skills


get_top_skills_for_job_title('Software Engineer')<jupyter_output><empty_output><jupyter_text>Best practices for consuming APIs* Your application shouldn't know about the API request/response structure* Write adapters to abstract external APIs* Make APIs easily swappable with other APIs or even a database* Use core entities shared between your application and adapters  Exposing RESTful APIs  Why build your own APIs? * Expose integration points into your application for other developers* Build distributed systems / microsevices* Create clear client-server separation  Tools for building APIs * Django REST Framework* Flask-RESTful* HUG* etc...  Simple API using HUG frameworkLet's expose our top skills by title with custom scoring via an API:<jupyter_code>"""API to get skills by job title"""
import hug


@hug.get()
def top_skills(title: hug.types.text):
    """Get skills by a job title"""
    return {'skills': [
        {
            'name': s.name,
            'score': s.score
        } for s in get_top_skills_for_job_title(title)
    ]}<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.11341691886438968, 'fraction_numerical': 0.006340971321516069, 'mean_word_length': 4.807531380753138, 'pattern_counts': {'":': 0, '<': 48, '<?xml version=': 0, '>': 48, 'https://': 3, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '11', 'dir': 'jupyter-structured-clean-dedup', 'id': '258971', 'license': 'MIT', 'n_tokens_mistral': 2257, 'n_tokens_neox': 2149, 'n_words': 629, 'path': 'talk.ipynb', 'repo_name': 'littlepea/python-rest-api-talk', 'size': '6987'}