task_categories:text-classification task_categories:question-answering size_categories:100K<n<1M language:zh license:cc-by-sa-4.0 fact-checking crowd-sourcing region:us
---
license: cc-by-sa-4.0
language:
- zh
pretty_name: Cofacts archive for reported messages and crowd-sourced fact-check replies
tags:
- fact-checking
- crowd-sourcing
size_categories:
- 100K<n<1M
extra_gated_prompt: >-
  To access this repository, you agree to follow the [Cofacts Data User Agreement](https://github.com/cofacts/opendata/blob/master/LEGAL.md).
  This is vital to sustain a crowd-sourced database like Cofacts to attribute the fact-checking community that contributed to this dataset.

  欲存取此資料集，需同意[Cofacts 真的假的 資料使用者條款](https://github.com/cofacts/opendata/blob/master/LEGAL.md)。
  彰顯查核社群對此資料集之貢獻，對協作型資料庫如 Cofacts 的永續發展至關重要。

  It would be great if you share with us who you are and your planned usage of the Cofacts data. Your cooperation is greatly appreciated.
  If you have no specific details to share with us, please simply enter "n/a."
  
  若方便的話，希望您可以與 Cofacts 工作小組分享您的單位以及預計會怎麼運用這個資料，感謝您！若不方便，可輸入「n/a」。
extra_gated_fields:
  'I agree to follow the Data User Agreement and promise to attribute Cofacts as specified 我同意遵守資料使用者條款並承諾按規定彰顯 Cofacts': checkbox
  'Anything to share with us 有什麼想要與我們分享的嗎': text
configs:
- config_name: analytics
  data_files: analytics.csv.zip
- config_name: article_categories
  data_files: article_categories.csv.zip
- config_name: article_hyperlinks
  data_files: article_hyperlinks.csv.zip
  lineterminator: |+

- config_name: article_replies
  data_files: article_replies.csv.zip
- config_name: article_reply_feedbacks
  data_files: article_reply_feedbacks.csv.zip
  lineterminator: |+

- config_name: articles
  data_files: articles.csv.zip
  lineterminator: |+

  default: true
- config_name: categories
  data_files: categories.csv.zip
  lineterminator: |+

- config_name: replies
  data_files: replies.csv.zip
  lineterminator: |+

- config_name: reply_hyperlinks
  data_files: reply_hyperlinks.csv.zip
  lineterminator: |+

- config_name: reply_requests
  data_files: reply_requests.csv.zip
  lineterminator: |+

task_categories:
- text-classification
- question-answering
---

# Cofacts Archive for Reported Messages and Crowd-Sourced Fact-Check Replies

[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/1qdE-OMJTi6ZO68J6KdzGdxNdheW4ct6T?usp=sharing)

The Cofacts dataset encompasses instant messages that have been reported by users of the [Cofacts chatbot](https://line.me/R/ti/p/@cofacts) and the replies provided by the [Cofacts crowd-sourced fact-checking community](https://www.facebook.com/groups/cofacts/).

## Attribution to the Community

This dataset is a result of contributions from both Cofacts LINE chatbot users and the community fact checkers.
To appropriately attribute their efforts, please adhere to the rules outlined in the [Cofacts 真的假的 資料使用者條款 (Cofacts Data User Agreement)](https://github.com/cofacts/opendata/blob/master/LEGAL.md).

Unless stated otherwise, when redistributing Cofacts data outside the LINE application, the attribution specified by the Cofacts Working Group is as follows:

> This data by Cofacts message reporting chatbot and crowd-sourced fact-checking community is licensed under CC BY-SA 4.0. To provide more info, please visit Cofacts LINE bot https://line.me/ti/p/@cofacts

除非以其他方式議定，否則 Cofacts 真的假的工作小組，針對在 LINE 之外的地方散布的 Cofacts 所提供資料，所指定的中文顯名聲明為：

> 本編輯資料取自「Cofacts 真的假的」訊息回報機器人與查證協作社群，採 CC BY-SA 4.0 授權提供。若欲補充資訊請訪問 Cofacts LINE bot https://line.me/ti/p/@cofacts

For more detailed information, please refer to [Cofacts 真的假的 資料使用者條款](https://github.com/cofacts/opendata/blob/master/LEGAL.md).

## How to Access Cofacts Data

To access Cofacts data, you should first register on Hugging Face and accept the Cofacts Data User Agreement. Afterward, you can preview the data on the Hugging Face website.

You can access Cofacts data through the following methods:

1. Load `cofacts/line-msg-fact-check-tw` with Hugging Face's `load_dataset('Cofacts/line-msg-fact-check-tw', TABLE_NAME)`.
2. Download individual zipped CSV files in the `Files` tab on the Hugging Face website.

If you plan to process the data using Python, `load_dataset()` is the simpler solution.
Please refer to [Example on Google Colab](https://colab.research.google.com/drive/1qdE-OMJTi6ZO68J6KdzGdxNdheW4ct6T?usp=sharing) to get started.

## Data Formats

Cofacts data comprises multiple normalized tables, with some tables containing foreign keys to other tables' IDs.

If you have manually downloaded the data, the tables are distributed as zipped CSV files. These files use `\n` as the line terminator, and quotes are used around multi-line contents.
The [`csv-stringify`](https://www.npmjs.com/package/csv-stringify) library is employed to perform escaping and handle quotes and multi-line contents.

### Fields in All Tables

* `userIdsha` (string) Hashed user identifier.
* `appId` (string) Possible values include:
    * `LEGACY_APP`: Articles collected before 2017-03.
    * `RUMORS_LINE_BOT`: Articles collected with the current LINE bot client after 2017-03.

These two fields together uniquely identify a user across different CSV files. For example, if one row (reply) in `replies.csv` and another row (feedback) in `article_reply_feedbacks.csv` have identical `userIdsha` and `appId`, it indicates that the reply and the feedback were submitted by the same user.

## Tables and their fields

### `articles`

The instant messages LINE bot users submitted into the database.

| Field            | Data type | Description |
| ----------------------- | -------- | ---- |
| `id`                      | String     |  |
| `references`              | Enum string     | Where the message is from. Currently the only possible value is `LINE`. |
| `userIdsha`                  | String     | Author of the article.|
| `appId`                   | String     |  |
| `normalArticleReplyCount` | Integer     | The number of replies are associated to this article, excluding the deleted reply associations. |
| `text`                    | Text     | The instant message text |
| `createdAt`               | ISO time string     | When the article is submitted to the database. |
| `updatedAt`               | ISO time string     | Preserved, currently identical to `createdAt` |
| `lastRequestedAt`         | ISO time string     | The submission time of the last `reply_request` is sent on the article, before the article is replied.  |

### `article_hyperlinks`

Parsed hyperlink contents in each instant messages, parsed using [cofacts/url-resolver](https://github.com/cofacts/url-resolver/).
The data is used in Cofacts system for indexing and retrieving messages.

| Field            | Data type | Description |
| ---------------- | -------- | ---- |
| `articleId`      | String     |                                    |
| `url`            | String     | The URL string detected in article |
| `normalizedUrl`  | String     | Canonical URL after normalization process including unfolding shortened URLs |
| `title`          | String     | Title of the scrapped web content |

Note: Scrapped contents do not belong to Cofacts and are redistributed under research purposes.
The scrapping mechanism is not reliable either.
Researchers may need to implement their own scrapper if content is important in their research.

### `article_categories`

Categories linked to this article.

| Field            | Data type  | Description |
| ---------------- | ---------- | ---- |
| `articleId`      | String     |                                    |
| `categoryId`     | String     |
| `aiConfidence`   | Number     | Confidence level by AI marking this category. Empty for crowd-sourced labels. |
| `aiModel` .      | String     | Name of the AI model marking this cateogry. Empty for crowd-sourced labels. |
| `userIdsha` .    | String     | The person that connected article and category. |
| `appId` .        | String     |  |
| `negativeFeedbackCount` | Integer     | Number of `article_category_feedbacks` that has score `-1` |
| `positiveFeedbackCount` | Integer     | Number of `article_category_feedbacks` that has score `1` |
| `status`                | Enum string     | `NORMAL`: The category and article are connected. `DELETED`: The category does not connect to the article anymore. |
| `createdAt`      | ISO time string     | The time when the reply is connected to the article |
| `updatedAt`             | ISO time string     | The latest date when the category's status is updated |

### `categories`

| Field         | Data type | Description |
| ------------- | --------- | ----------- |
| `id`          | String    |             |
| `title`       | String    | Name of the category |
| `description` | Text      | Definition of the category |
| `createdAt`   | ISO time string     |   |
| `updatedAt`   | ISO time string     |   |

### `article_replies`

Articles and replies are in has-and-belongs-to-many relationship. That is, an article can have multiple replies, and a reply can be connected to multiple similar articles.

`article_replies` is the "join table" between `articles` and `replies`, bringing `articleId` and `replyId` together, along with other useful properties related to this connection between an article and a reply.

One pair of `articleId`, `replyId` will map to exactly one `article_reply`.


| Field            | Data type | Description |
| --------------------- | -------- | - |
| `articleId`             | String     | Relates to `id` field of `articles` |
| `replyId`               | String     | Relates to `id` field of `replies` |
| `userId`                | String     | The user connecting the reply with the article |
| `negativeFeedbackCount` | Integer     | Number of `article_reply_feedbacks` that has score `-1` |
| `positiveFeedbackCount` | Integer     | Number of `article_reply_feedbacks` that has score `1` |
| `replyType`             | Enum string     | Duplicated from `replies`'s type. |
| `appId`                 | String     | |
| `status`                | Enum string     | `NORMAL`: The reply and article are connected. `DELETED`: The reply does not connect to the article anymore. |
| `createdAt`             | ISO time string     | The time when the reply is connected to the article |
| `updatedAt`             | ISO time string     | The latest date when the reply's status is updated |


### `replies`

Editor's reply to the article.

| Field            | Data type | Description |
| --------- | -------- | - |
| `id`        | String     | |
| `type`      | Enum string     | Type of the reply chosen by the editor. `RUMOR`: The article contains rumor. `NOT_RUMOR`: The article contains fact. `OPINIONATED`: The article contains personal opinions. `NOT_ARTICLE`: The article should not be processed by Cofacts. |
| `reference` | Text     | For `RUMOR` and `NOT_RUMOR` replies: The reference to support the chosen `type` and `text`. For `OPINIONATED` replies: References containing different perspectives from the `article`. For `NOT_ARTICLE`: empty string. |
| `userId`    | String     | The editor that authored this reply. |
| `appId`     | String     | |
| `text`      | Text     | Reply text writtern by the editor |
| `createdAt` | ISO Time string     | When the reply is written |

### `reply_hyperlinks`

Parsed hyperlink contents in reply text and references, parsed using [cofacts/url-resolver](https://github.com/cofacts/url-resolver/).
The data is used in Cofacts system for URL previews.

| Field            | Data type | Description |
| ---------------- | -------- | ---- |
| `replyId`      | String     |                                    |
| `url`            | String     | The URL string detected in article |
| `normalizedUrl`  | String     | Canonical URL after normalization process including unfolding shortened URLs |
| `title`          | String     | Title of the scrapped web content |

Note: Scrapped contents do not belong to Cofacts and are redistributed under research purposes.
The scrapping mechanism implementation is not reliable either.
Researchers may need to implement their own scrapper if content is important in their research.

### `reply_requests`

Before an article is replied, users may submit  `reply_requests` to indicate that they want this article to be answered.

When an article is first submitted to the article, an reply request is also created. Any further queries to the same article submits new `reply_requests`.

An user can only submit one reply request to an article.

| Field            | Data type | Description |
| --------- | -------- | - |
| `articleId`        | String     | The target of the request |
| `reason`        | Text     | The reason why the user wants to submit this reply request |
| `positiveFeedbackCount`        | Text     | Number of editors think the reason is reasonable |
| `negativeFeedbackCount`        | Text     | Number of editors think the reason is nonsense |
| `createdAt` | ISO Time string     | When the reply request is issued |

### `article_reply_feedbacks`

Editors and LINE bot users can express if a reply is useful by submitting `article_reply_feedbacks` toward a `article_reply` with score `1` or `-1`.

The feedback is actually submitted toward an `article_reply`, the connection between an article and a reply. This is because a reply can be connected to multiple articles. A reply that makes sense in one article does not necessarily mean that it is useful in answering another article. Therefore, the feedback count for a reply connecting to different articles are counted separately.

| Field            | Data type | Description |
| --------- | -------- | - |
| `articleId`        | String     | Relates to `articleId` of the target `article_reply` |
| `replyId`        | String     | Relates to `replyId` of the target `article_reply` |
| `score`        | Integer     | `1`: Useful. `-1`: Not useful. |
| `comment`        | Text     | Why the user chooses such score for this article reply |
| `createdAt` | ISO Time string     | When the feedback is submitted |

### `analytics`

Usage (visit / show) statistics of website and Cofacts LINE bot.

LINE bot data starts from April 2nd, 2018; website data starts from May 3rd, 2017.

| Field       | Data type       | Description |
| ----------- | --------------- | ----------- |
| `type`      | Enum string     | Either `article` or `reply` |
| `docId`     | String          | Article ID or Reply ID that is being visited / shown |
| `date`      | ISO Time string | The date of usage, represented by start of the day (0:00:00+08:00) |
| `lineUser`  | Integer         | The number of LINE users who inspected this article / reply in Cofacts LINE bot in this date. May be empty if no such users |
| `lineVisit` | Integer         | The number of times this article / reply is inspected in Cofacts LINE bot in this date. May be empty if no visits |
| `webUser`   | Integer         | The number of web users who visited this article page (`/article/<docId>`) / reply page (`/reply/<docId>`) in Cofacts website in this date. May be empty if no such users |
| `webVisit`  | Integer         | The number of page views of this article page (`/article/<docId>`) / reply page (`/reply/<docId>`) in Cofacts website in this date. May be empty if no page views |

## ⚠ [NOTICE] Caveats of using this data ⚠

The methodology we use to collect these data (i.e. [how Cofacts works](https://beta.hackfoldr.org/cofacts/https%253A%252F%252Fhackmd.io%252Fs%252FBJSdbUMpZ))
could have some impact on the data credibility.

![How cofacts work](https://i.imgur.com/e3Awc50.png)

Please keep in mind that all data in this dataset are user-generated,
thus is not free from noise and sampling bias coming from these sources:
- The distribution Cofacts' users may not reflect the real distribution of all LINE users in Taiwan.
- Users may not use Cofacts in the same way we want them to be.
  Some `articles` may not be actual messages circulating in LINE network.
- `replies` may contain factual error.
  All replies should be merely regarded as "responses to the original message (`article`) to provide different point of view".
  They are neither the "truth" nor the editor's personal opinion.
- There may also exist malicious users sending garbage `articles` into the database. [(Previous incident report)](https://hackmd.io/@cofacts/incidents)
- The program to collect data and to generate dataset may contain error.
  The dataset may be inaccurate systematically in this way.

Lastly, the dataset is provided without warrenty.

THE DATASET IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE DATASET OR THE USE OR OTHER DEALINGS IN THE DATASET.