<commit_before># cobaltuoft

[![PyPI version](https://badge.fury.io/py/cobaltuoft.svg)](https://badge.fury.io/py/cobaltuoft)

A Python library for interfacing with publicly available UofT open data through [Cobalt](http://cobalt.qas.im).

## Requirements

* Python 3

## Installation

`pip3 install cobaltuoft`

## Usage

### Classes

* [Cobalt](#cobaltuoftcobalt)
* [Datasets](#cobaltuoftdatasets)

### API

#### `cobaltuoft.Cobalt`

Directly interface with [Cobalt](http://cobalt.qas.im) APIs. Data is returned as a native Python dictionary.
  
  * Initialize the class, a key is required to use this class (get yours [here](https://cobalt.qas.im/signup)).

    ```python
    from cobaltuoft import Cobalt
    cobalt = Cobalt('API_KEY')
    ```

  * Each [active API](https://cobalt.qas.im/#apis) can be accessed through a method of the same name.
  
  * Method parameters:
    
    * `endpoint` –  The endpoint to be accessed for the respective API. One of `None` (used for [list](https://cobalt.qas.im/documentation/courses/list) and [show](https://cobalt.qas.im/documentation/courses/show)), [`search`](https://cobalt.qas.im/documentation/courses/search), or [`filter`](https://cobalt.qas.im/documentation/courses/filter).
    
    * `params` – The URL parameters for this query. Possible parameter keys (read more about these parameters in the [Cobalt docs](https://cobalt.qas.im/documentation/courses/search)):
    
      | Parameter     | Endpoints                  | Description                                                                                                      |
      |---------------|----------------------------|------------------------------------------------------------------------------------------------------------------|
      | `limit`       | All (except `show`)        | The number of results to return.                                                                                 |
      | `skip`        | All (except `show`)        | The number of results to skip.                                                                                   |
      | `sort`        | All (except `show`)        | The sorting procedure to be used on the returned list.                                                           |
      | `q`           | Only `search` or `filter`  | The string to perform a text search with. For filter queries, refer to [this](#filter-queries).                  |
      | `id` / `date` | Only `show`                | The `:id` or `:date` endpoint value for the [`show`](https://cobalt.qas.im/documentation/courses/show) endpoint. |

  * Examples:
    * `/courses?skip=10`
    
      ```python
      courses = cobalt.courses(params={'skip': 10})
      ```
    
    * `/buildings/134`
    
      ```python
      buildings = cobalt.courses(params={'id': 134})
      ```
   
    * `/food/search?q="pizza"&limit=2`
    
      ```python
      food = cobalt.food(endpoint='search', params={
          'q': '"pizza"',
          'limit': 2
      })
      ```
   
    * `/textbooks/filter?price:>=500 OR author="Queen"`
      
      ```python
      textbooks = cobalt.textbooks(endpoint='filter', params={
          'q': [
              [
                  ('price', '>=500'),
                  ('author', '"Queen"')
              ]
          ]
      })

      # equivalently:
      textbooks = cobalt.textbooks(endpoint='filter', params={
          'q': 'price:>=500 OR author:"Queen"'
      })
      ```

###### Filter Queries

* Parameter value for `q` can be passed as a [Cobalt-filter string](https://cobalt.qas.im/documentation/courses/filter) or a nested list of key-value pairs.
* Outer lists are joined by `"AND"` and inner lists are joined by `"OR"`.
* Use a tuple for the key-value pair.
* Examples:

  * `instructor:"<NAME>" AND level:<=200`
  
    ```python
    [
        [('instructor', '"<NAME>"')], 
        [('level', '<=200')]
    ]
    ```
  
  * `breadth:!2 OR code:"CSC"`
  
    ```python
    [
        [('breadth', '!2'), ('code', '"CSC"')]
    ]
    ```
  
  * `instructor:"<NAME>" AND level:<=200 AND level:<=200 AND breadth:!2 OR code:"CSC"`
  
    ```python
    [
        [('instructor', '"<NAME>"')], 
        [('level', '<=200')],
        [('breadth', '!2'), ('code', '"CSC"')]
    ]
    ```

--------------------------------------------------------------------------------

#### `cobaltuoft.Datasets`

  * Interface with full [Cobalt datasets](https://github.com/cobalt-uoft/datasets). Data is returned as a native Python dictionary.
  * This class doesn't need to be instantiated, instead call it's methods directly.
  * Currently only has a single method, `run`, parameters:

    | Name     | Value | Description                                                                                                                                                                                                  |
    |----------|-------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
    | `tag`    | `str` | The release tag for which to download data from. Defaults to `'latest'`. For a list of available tags, refer to [this](https://api.github.com/repos/cobalt-uoft/datasets/tags) or run `Datasets._get_tags()` |

  ```python
  from cobaltuoft import Datasets

  datasets = Datasets.run()
  ```

## Contribute

This project is completely open source. Feel free to open an [issue](https://github.com/kshvmdn/cobalt-uoft-python/issues) for questions/requests or submit a [pull request](https://github.com/kshvmdn/cobalt-uoft-python/pulls)<commit_msg>Update examples & wording<commit_after># cobaltuoft [![PyPI version](https://badge.fury.io/py/cobaltuoft.svg)](https://badge.fury.io/py/cobaltuoft)

A Python wrapper for interfacing with [Cobalt](http://github.com/cobalt-uoft), open data [APIs](http://cobalt.qas.im) and [datasets](http://github.com/cobalt-uoft/datasets) for the University of Toronto.

## Requirements

* Python 3

## Installation

```sh
pip3 install cobaltuoft
```

## Usage

### Classes

* [Cobalt](#cobaltuoftcobalt)
* [Datasets](#cobaltuoftdatasets)

### API

#### `cobaltuoft.Cobalt`

  * Directly interface with [Cobalt](http://cobalt.qas.im) APIs. Data is returned as a native Python object (`[dict]` or `dict`).

  * Initialize the class (a key is required, get one [here](https://cobalt.qas.im/signup)).

    ```py
    >>> from cobaltuoft import Cobalt
    >>> cobalt = Cobalt('API_KEY')
    ```

  * Each [active API](https://cobalt.qas.im/#apis) can be accessed through a method of the same name.

  * Method parameters:

    * `endpoint` –  The endpoint to be accessed for the respective API. Any of the following values are accepted (for most of the APIs at least, refer to the docs for exceptions):

      * `None` (used for [list](https://cobalt.qas.im/documentation/courses/list) and [show](https://cobalt.qas.im/documentation/courses/show))
      * [`search`](https://cobalt.qas.im/documentation/courses/search)
      * [`filter`](https://cobalt.qas.im/documentation/courses/filter)

    * `params` – The URL parameters for this query. Accepts `None` or a `dict` of key-value pairs. Any of the following keys are accepted (read more about these parameters [here](https://cobalt.qas.im/documentation/courses/search)):

      | Parameter     | Endpoints                  | Description                                                                                             |
      |---------------|----------------------------|---------------------------------------------------------------------------------------------------------|
      | `limit`       | All except `show`          | The number of results to return.                                                                        |
      | `skip`        | All except `show`          | The number of results to skip.                                                                          |
      | `sort`        | All except `show`          | The sorting procedure to be used on the returned list.                                                  |
      | `q`           | Only `search` or `filter`  | The string to perform a text search with. For filter queries, refer to [this](#filter-queries).         |
      | `id` / `date` | Only `show`                | The `:id` or `:date` value for the [`show`](https://cobalt.qas.im/documentation/courses/show) endpoint. |

  * Examples:
    * `/courses?skip=10`

      ```python
      >>> cobalt.courses(params={'skip': 10})
      [{...}]
      ```

    * `/buildings/134`

      ```python
      >>> cobalt.buildings(params={'id': 134})
      {...} # not a list!
      ```

    * `/food/search?q="pizza"&limit=2`

      ```python
      >>> cobalt.food(endpoint='search', params={
      ...     'q': '"pizza"',
      ...     'limit': 2
      ... })
      [{...}]
      ```

    * `/textbooks/filter?price:>=500 OR author="Queen"`

      ```python
      >>> cobalt.textbooks(endpoint='filter', params={
      ...     'q': [
      ...         [
      ...             ('price', '>=500'),
      ...             ('author', '"Queen"')
      ...         ]
      ...     ]
      ... })
      [{...}]

      # equivalently
      >>> cobalt.textbooks(endpoint='filter', params={
      ...     'q': 'price:>=500 OR author:"Queen"'
      ... })
      [{...}]
      ```

###### Filter Queries

* The `q` value can either be a [Cobalt-filter string](https://cobalt.qas.im/documentation/courses/filter) or a nested list of key-value pairs.
* Outer lists are joined by `"AND"` and inner lists are joined by `"OR"`. Use a `tuple` for the key-value pair.
* You can use [`Cobalt._process_filter`](https://github.com/kshvmdn/cobalt-uoft-python/blob/master/cobaltuoft/endpoints/__init__.py#L17) to test your filters.
* Examples:

  ```python
  >>> [
  ...     [('instructor', '"<NAME>"')],
  ...     [('level', '<=200')]
  ... ]
  'instructor:"<NAME>" AND level:<=200'
  ```

  ```python
  >>> [
  ...    [('breadth', '!2'), ('code', '"CSC"')]
  ... ]
  'breadth:!2 OR code:"CSC"'
  ```

  ```python
  >>> [
  ...     [('instructor', '"<NAME>"')],
  ...     [('level', '<=200')],
  ...     [('breadth', '!2'), ('code', '"CSC"')]
  ... ]
  'instructor:"<NAME>" AND level:<=200 AND breadth:!2 OR code:"CSC"'
  ```

--------------------------------------------------------------------------------

#### `cobaltuoft.Datasets`
  * Interface with full [Cobalt datasets](https://github.com/cobalt-uoft/datasets). Data is returned as a native Python object (`[dict]` or `dict`).
  * This class doesn't need to be instantiated, instead call it's methods directly.
  * `Datasets.run`

    | Parameter  | Type   | Description                                                                                                                                                                                                  |
    |------------|--------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
    | `tag`      |`str`   | The release tag for which to download data from. Defaults to `'latest'`. For a list of available tags, refer to [this](https://api.github.com/repos/cobalt-uoft/datasets/tags) or run `Datasets._get_tags()` |

    ```python
    >>> from cobaltuoft import Datasets
    >>> Datasets.run('2016-2017')
    [{...}]
    ```

## Contribute

This project is completely open source. Feel free to open an [issue](https://github.com/kshvmdn/cobalt-uoft-python/issues) for questions/requests or submit a [pull request](https://github.com/kshvmdn/cobalt-uoft-python/pulls)!
{'fraction_non_alphanumeric': 0.22503814205797593, 'fraction_numerical': 0.007713171724021021, 'mean_word_length': 1.544533103299547, 'pattern_counts': {'":': 0, '<': 20, '<?xml version=': 0, '>': 53, 'https://': 31, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 44, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '2904232', 'n_tokens_mistral': 3413, 'n_tokens_neox': 3226, 'n_words': 819}