<commit_before># Broker REST(ful) API

Resource URL will depend on your broker location. We report only the common suffixes.


## Client Endpoints

### POST /register/system
  - Request body a JSON object containing:`{"groupid":"trec group identifier","alias" : "system alias"}`
    + used for validation

Register a client system using the group identifier provided in the request body and aliased with the given
name for simplicity in TREC reporting.
Returns a JSON object of the client identifier to be used by the system when making requests,
e.g., `{"clientid":"foobar123"}`.


### GET /topics/:clientid
  - **:clientid** is the client's identifier that was returned when the system was registered
    + This is used for bookkeeping/rate-limiting purposes

Client is requesting the current topics to process. This is a JSON array of pairs of topic identifier (topid), the 
TREC title (the information need), and the body (which may be the description and/or narrative in a traditional TREC topic).
For example, `[{"topid":"test","title":"birthday","body" : "Tweets pertaining to birthdays, including wishing happy birthday."}]`.

### POST /tweet/:topid/:tweetid/:clientid
  - **:topid** specifies the topic identifier
  - **:tweetid** specifies the tweet identifier supplied by the Twitter Streaming API 
  - **:clientid** is the client's identifier that was returned when the system was registered
    + This is used for bookkeeping/rate-limiting purposes

Submit a (tweet,topic) pair for assessment. Returns a 204 status code on success.

### POST /tweets/:topid/:clientid
  - **:topid** specifies the topic identifier
  - **:clientid** is the client's identifier that was returned when the system was registered

The body of the request should contain a JSON object with a field denoted 'tweets' with a JSON array of tweetid strings,
e.g., '{"tweets":["123","456"]}'.

This submits a system's daily digest set of tweets. Can be done in batches but only the first X will be used for evaluation (X is given in the guidelines).

 Returns a 204 status code on success.

### GET /judge/:topid/:tweetid/:clientid
  - **:topid** specifies the topic identifier
  - **:tweetid** specifies the tweet identifier supplied by the Twitter Streaming API 
  - **:clientid** is the client's identifier that was returned when the system was registered
    + This is used for bookkeeping/rate-limiting purposes

Request the relevance assessment for  a (tweet,topic) pair. Returns a 200 status code and a JSON object containing:
`{"tweetid":tweetid,"topid":topid,"rel":rel}`, where rel is either -1 (not relevant), 1 (relevance), and 0 (unjudged).
The tweet identifier and topic identifier are repeated for verification purposes.

## Mobile Assessor Endpoints

### POST /register/mobile
  - Request body a JSON object containing:`{"regid":"Push notification id","partid" : "The participant id", "deviceid" : "iOS and/or Android"}`
    + used for pushing tweets to the device

Register an instance of the mobile app so that we can push topics and tweets to the mobile phone.
Users should specify their participant id as a means of authentication to receive push notifications. 
The deviceid field is used to determine which style of push notification to use.

### POST /tweet/:topid/:tweetid/:rel/:partid
  - **:topid** specifies the topic identifier
  - **:tweetid** specifies the tweet identifier supplied by the Twitter Streaming API 
  - **:rel** is the relevance assessment (-1 or 1) of tweet to the topic
  - **:partid** is the participant identifier used to track assessments by each mobile assessor

Mobile app has submited a relevance assessment for a (tweet,topic) pair for assessment. 
Returns a 204 status code on success.

<commit_msg>Updated documentation.<commit_after># TREC RTS Evaluation Broker REST(ful) API

## Client Endpoints

### POST /register/system

- Request body should be a JSON object as follows: `{"groupid":"trec group identifier","alias":"system alias"}`
- The `groupid` will be provided by the organizers.
- The `alias` is a human-friendly identifier that you specify to help identify the client, e.g., `WaterlooBaseline`. We will use this alias as the "runid" to report results (e.g., in the track overview paper), so usual TREC rules apply (e.g., no spaces).

Use this API endpoint to register a client with the evaluation broker.
The API call will return a JSON object of the client id to be used by the system when making requests, e.g., `{"clientid":"abcdefghijk"}`.

Note that each call to this API endpoint will return a unique client id. Therefore, for each individual experimental run, you should request a separate client id.

You can manually test this API with `curl` as follows:

```
curl -H 'Content-Type: application/json' --data '{"groupid":"mygroup","alias":"MyBaseline"}' hostname.com/register/system
```

### GET /topics/:clientid

- **:clientid** is the client id that was returned by the broker above. This is used for bookkeeping/rate-limiting purposes.

Use this API endpoint to request the topics to be evaluated.
This is a JSON array of pairs of topic identifier (topid), the 
TREC title (the information need), and the body (which may be the description and/or narrative in a traditional TREC topic).
For example, `[{"topid":"test","title":"birthday","body" : "Tweets pertaining to birthdays, including wishing happy birthday."}]`.

You can manually test this API with `curl` as follows:

```
curl -H 'Content-Type: application/json' hostname.com/topics/abcdefghijk
```

### POST /tweet/:topid/:tweetid/:clientid

- **:topid** specifies the topic identifier.
- **:tweetid** specifies the id of the tweet from the Twitter Streaming API that was identified as being relevant to the topic.
- **:clientid** is the client's identifier that was returned when the system was registered. This is used for bookkeeping/rate-limiting purposes.

Use this API endpoint to submit a tweet for assessment with respect to a particular topic. Returns a 204 status code on success.

You can manually test this API with `curl` as follows:

```
curl -X POST -H 'Content-Type: application/json' hostname.com/tweet/MBXXX/738418531520352258/abcdefghijk
```

## Mobile Assessor Endpoints

Note that these API endpoints are documented for completeness only. System participants will not need to use these APIs.

### POST /register/mobile
  - Request body a JSON object containing:`{"regid":"Push notification id","partid" : "The participant id", "deviceid" : "iOS and/or Android"}`
    + used for pushing tweets to the device

Register an instance of the mobile app so that we can push topics and tweets to the mobile phone.
Users should specify their participant id as a means of authentication to receive push notifications. 
The deviceid field is used to determine which style of push notification to use.

### POST /tweet/:topid/:tweetid/:rel/:partid
  - **:topid** specifies the topic identifier
  - **:tweetid** specifies the tweet identifier supplied by the Twitter Streaming API 
  - **:rel** is the relevance assessment (-1 or 1) of tweet to the topic
  - **:partid** is the participant identifier used to track assessments by each mobile assessor

Mobile app has submited a relevance assessment for a (tweet,topic) pair for assessment. 
Returns a 204 status code on success.


{'fraction_non_alphanumeric': 0.08964007116463665, 'fraction_numerical': 0.007116463664978787, 'mean_word_length': 4.465968586387435, 'pattern_counts': {'":': 17, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '5183765', 'n_tokens_mistral': 2015, 'n_tokens_neox': 1899, 'n_words': 986}