task_categories:text-classification task_categories:text-generation task_categories:text-to-image task_categories:text-to-video language:ja license:apache-2.0 region:us
---
license: apache-2.0
task_categories:
- text-classification
- text-generation
- text-to-image
- text-to-video
language:
- ja
pretty_name: MissingKeys
---

# Dataset Card for MissingKeys

## Dataset Description

- **Homepage:** Here!
- **Repository:** N/A
- **Paper:** N/A
- **Leaderboard:** N/A
- **Point of Contact:** KaraKaraWitch

### Dataset Summary

MissingKeys is a raw dataset archive of the misskey.io network.

### Supported Tasks and Leaderboards

This dataset is primarily intended for unsupervised training of text generation models; however, it may be useful for other purposes.

- text-classification
- text-generation

### Languages

Primarily japanese, however there are also english as well.

## Dataset Structure

All the files are located in jsonl files that has been compressed into .7z archives by date.

### Data Instances

Here is a sample with all the potential fields:

```json
{
    "id": "9hh9iux6al",
    "createdAt": "2023-07-22T07:38:17.994Z",
    "userId": "9grv7htulz",
    "user": {
        "uid": "9grv7htulz#chikusa_nao@misskey.backspace.fm",
        "name": "千種ナオ(ばすキー)",
        "avatarUrl": "https://proxy.misskeyusercontent.com/avatar.webp?url=https%3A%2F%2Fs3.isk01.sakurastorage.jp%2Fbackspacekey%2Fmisskey%2Fca098593-5c2f-4488-8b82-18961149cf92.png&avatar=1",
        "avatarBlurhash": "eGD8ztEK0KVb-=4TtSXm-jf4B7Vs~CEND*Fy%2Mct7%Lx.M{xcS0bv",
        "states": "bot,nyaa~",
        "hostInfo": "misskey@13.13.2#e4d440"
        "emojis": {},
        "onlineStatus": "unknown"
    },
    "text": "パソコン工房などのユニットコム系列だと、マザボ売るときにドライバディスクがないと30%買取金額が下がるという知見を得た",
    "cw": null,
    "visibility": "public",
    "localOnly": false,
    "renoteCount": 0,
    "repliesCount": 0,
    "reactions": {},
    "reactionEmojis": {},
    "emojis": {},
    "fileIds": [],
    "files": [],
    "replyId": null,
    "renoteId": null,
    "uri": "https://misskey.backspace.fm/notes/9hh9iux6p7"
}
```

If the value is "Falsey" in python, it has been removed to save on space.

`states` is a comma seperated string that either includes: `bot` or `nyaa~` (Indicates they enabled cat mode) or both.

### Data Fields

Refer to the sample above. I'll drop in some additional notes:

`uid` in `user` follows this specific format:

`user_id#username@user_host`

### Data Splits

Each jsonl file is split at 100000 notes.

## Dataset Creation

### Curation Rationale

Because we need a SNS dataset, and since twitter appears to be quite reluctant, we went for the alternative.

### Source Data

#### Initial Data Collection and Normalization

None. No normalization is performed as this is a raw dump of the dataset. However we have removed empty and null fields to conserve on space.

#### Who are the source language producers?

The related users of misskey.io network.

### Annotations

#### Annotation process

No Annotations are present.

#### Who are the annotators?

No human annotators.

### Personal and Sensitive Information

We are certain there is no PII included in the dataset.

## Considerations for Using the Data

### Social Impact of Dataset

[More Information Needed]

### Discussion of Biases

Misskey.io tends to be NSFW for images and is focused on Japanese culture.

### Other Known Limitations

N/A

## Additional Information

### Dataset Curators

KaraKaraWitch

### Licensing Information

Apache 2.0, for all parts of which KaraKaraWitch may be considered authors. All other material is distributed under fair use principles.  

Ronsor Labs additionally is allowed to relicense the dataset as long as it has gone through processing.

### Citation Information

```
@misc{missingkeys,
  title         = {MissingKeys: A SNS dataset on misskey.io network},
  author        = {KaraKaraWitch},
  year          = {2023},
  howpublished  = {\url{https://huggingface.co/datasets/RyokoExtra/MissingKeys}},
}
```

### Name Etymology

N/A

### Contributions

- [@KaraKaraWitch (Twitter)](https://twitter.com/KaraKaraWitch) for gathering this dataset.