region:us
---
dataset_info:
  features:
  - name: id
    dtype: string
  - name: question
    dtype: string
  - name: question_concept
    dtype: string
  - name: choices
    sequence:
    - name: label
      dtype: string
    - name: text
      dtype: string
  - name: answerKey
    dtype: string
  - name: retrieved
    list:
    - name: answerKey
      dtype: string
    - name: choices
      struct:
      - name: label
        sequence: string
      - name: text
        sequence: string
    - name: passage
      dtype: string
    - name: rank
      dtype: int64
    - name: score
      dtype: float64
  splits:
  - name: validation
    num_bytes: 2646054
    num_examples: 1221
  download_size: 755467
  dataset_size: 2646054
---
# Dataset Card for "commonsense_qa_retrieved_by_colbert"

This is the validation set of [CSQA](https://huggingface.co/datasets/commonsense_qa/viewer/default/validation), with each question retrieved from [ColBERT v2](https://github.com/stanford-futuredata/ColBERT/tree/main) trained on MS MARCO Passage Ranking (`https://downloads.cs.stanford.edu/nlp/data/colbert/colbertv2/colbertv2.0.tar.gz`).

We index the question part of the train set using doc_maxlen=30, nbits=2. We search each question of validation set with k=10 and put the results in the `retrieved` column.