task_categories:text-generation task_categories:conversational language:en license:apache-2.0 arxiv:2306.09200 region:us
---
license: apache-2.0
task_categories:
- text-generation
- conversational
language:
- en
---

# The Chess Dataset

## Dataset Description

- **Paper:** [ChessGPT: Bridging Policy Learning and Language Modeling](https://arxiv.org/abs/2306.09200)

### Dataset Summary

The dataset consists of three sources of dataset described in the paper, including:

- **ChessCLIP dataset**: Annotated PGNs for training CLIP.
- **ChessGPT Base dataset**: Game dataset, language dataset and mixed dataset for training ChessGPT-Base.
- **ChessGPT Chat dataset**: Conversational dataset for training ChessGPT-Chat.

Because of the legal issue, for ChessGPT dataset, we do not open-source the chess-book, chess-forum, chess-blog, and Youtube transcript datasets.
And for ChessCLIP dataset, we do not open-source two commercial annotated datasets we use.

### Languages

The language of the data is primarily English.

## Dataset Structure

- **ChessCLIP dataset**: Annotated PGNs for training CLIP.
- **ChessGPT Base dataset**: Game dataset: ccrl, pro_player, lichess_db_37, chess_puzzles, chess_modeling. Language dataset: redpajama, oscar, c4, pile, wikipedia, and stackexchange, and mixed dataset: annotated_pgn.
- **ChessGPT Chat dataset**: Chess-related conversation dataset: 

### Data Instances

- **ChessCLIP dataset**:
  ```python
  [Event "GMA, Wijk aan Zee NED"]
  [Site "?"]
  [Date "2003.??.??"]
  [Round "1"]
  [White "Anand,V"]
  [Black "Radjabov,T"]
  [Result "1/2"]
  [WhiteElo "2750"]
  [BlackElo "2620"]
  [ECO "C12"]
  [PlyCount "55"]
  [Annotator "Hathaway"]
  
  1. e4 e6
  { I'm not terribly familiar with the style of Radjabov, so I don't know if this is his usual opening. }
  2. d4 d5 3. Nc3 Nf6 (3...Bb4 
  { The Winawer Variation is probably best, though not as easy to play. }) 4. Bg5
  { threatens e4-e5xf6 }
   (4. e5 
  { keeps pieces on the board and avoids ...dxe4 }) 4...Bb4 (4...Be7 
  { is more common and aims to trade dark-square bishops to ease Black's cramp }) (4...dxe4 
  { aims to avoid any cramp by bringing pieces into alignment for trading, though White does get at least one very good piece (Ne4 or Bg5) and an easier time castling queen-side, to stir up king-side threats } ...
  ```
- **ChessGPT dataset**:
  ```python
  {"metadata": {"id": "65268522",
   "title": "1923 USSR Chess Championship",
   "url": "https://en.wikipedia.org/wiki/1923%20USSR%20Chess%20Championship"},
   "text": "The 1923 USSR Chess Championship was the second edition of USSR Chess Championship. Held from 8 to 24 July in Petrograd. The tournament was won by Peter Romanovsky.
   "pipeline_key": "wiki-squad-train.jsonl.zst-10549"
  }
  ```
- **ChessGPT Chat dataset**:
  ```python
  {"conversations": [
   {"author": "Human 0", "text": "Analyze the given sentence and identify its grammatical errors, if any.\nThese type of problems are difficult to solve.", "url": "gpt-4-alpaca", "score": 0},
   {"author": "Human 1", "text": "The sentence is incorrect. The error is in the usage of \"these\". Since \"these\" is a plural determiner, it should be used with a plural noun. Similarly, \"type\" is singular noun and it is used with the singular determiner, \"this\".\n\nThe correct sentence should be \"This type of problem is difficult to solve.\" or \"These types of problems are difficult to solve.\"",
   "url": "gpt-4-alpaca", "score": 0}
   ]
  }
  ```
  
### Data Splits

The data is unsplit.

## Dataset Creation

Check [ChessGPT: Bridging Policy Learning and Language Modeling](https://arxiv.org/abs/2306.09200) for more details.

### Licensing Information

**Annotated PGN**: [PGNlib](https://www.angelfire.com/games3/smartbridge/), [lichess](https://lichess.org/terms-of-service), [GameKnot](https://gameknot.com/pg/pol_eula.htm), [pathtomaster](https://www.pathtochessmastery.com/)

**Game Dataset**: [Lichess dataset](https://www.tldrlegal.com/license/creative-commons-cc0-1-0-universal), [CCRL](https://ccrl.chessdom.com/ccrl/), [pro-player](https://www.pgnmentor.com/files.html), [puzzle](https://www.tldrlegal.com/license/creative-commons-cc0-1-0-universal), Modeling data(Apache-2.0).

**Language Dataset** [Wikipedia](https://huggingface.co/datasets/wikipedia#licensing-information), [Redpajama](https://github.com/togethercomputer/RedPajama-Data#license), [Oscar](https://huggingface.co/datasets/oscar-corpus/OSCAR-2301#licensing-information), [Pile](https://github.com/EleutherAI/the-pile/blob/master/LICENSE), [StackExchange](https://archive.org/details/stackexchange), [C4](https://huggingface.co/datasets/allenai/c4#license)

**Conversatoinal Datset**: [Chessable forums](https://www.chessable.com/terms), [Reddit](https://www.redditinc.com/policies/data-api-terms), [gpt-4](https://openai.com/policies/terms-of-use), [sharegpt](https://chrome.google.com/webstore/detail/sharegpt-share-your-chatg/daiacboceoaocpibfodeljbdfacokfjb), oasst1(Apache-2.0), dolly-v2(MIT)

### Citation Information
```bash
@article{feng2023chessgpt,
  title={ChessGPT: Bridging Policy Learning and Language Modeling},
  author={Feng, Xidong and Luo, Yicheng and Wang, Ziyan and Tang, Hongrui and Yang, Mengyue and Shao, Kun and Mguni, David and Du, Yali and Wang, Jun},
  journal={arXiv preprint arXiv:2306.09200},
  year={2023}
}
```