task_categories:translation annotations_creators:found language_creators:found multilinguality:multilingual size_categories:100K<n<1M source_datasets:original language:aa language:ab language:ae language:aeb language:af language:ak language:am language:an language:ar language:arq language:arz language:as language:ase language:ast language:av language:ay language:az language:ba language:be language:ber language:bg language:bh language:bi language:bm language:bn language:bnt language:bo language:br language:bs language:bug language:ca language:ce language:ceb language:ch language:cho language:cku language:cnh language:co language:cr language:cs language:cu language:cv language:cy language:da language:de language:dv language:dz language:ee language:efi language:el language:en language:eo language:es language:et language:eu language:fa language:ff language:fi language:fil language:fj language:fo language:fr language:ga language:gd language:gl language:gn language:gu language:ha language:hai language:haw language:haz language:hch language:he language:hi language:ho language:hr language:ht language:hu language:hup language:hus language:hy language:hz language:ia language:id language:ie language:ig language:ik language:inh language:io language:iro language:is language:it language:iu language:ja language:jv language:ka language:kar language:ki language:kj language:kk language:kl language:km language:kn language:ko language:kr language:ksh language:ku language:kv language:kw language:ky language:la language:lb language:lg language:li language:lkt language:lld language:ln language:lo language:lt language:ltg language:lu language:luo language:luy language:lv language:mad language:mfe language:mg language:mi language:mk language:ml language:mn language:mni language:moh language:mos language:mr language:ms language:mt language:mus language:my language:nb language:nci language:nd language:ne language:nl language:nn language:nso language:nv language:ny language:oc language:om language:or language:pa language:pam language:pap language:pi language:pl language:pnb language:prs language:ps language:pt language:qu language:rm language:rn language:ro language:ru language:rup language:rw language:sa language:sc language:scn language:sco language:sd language:sg language:sgn language:sh language:si language:sk language:sl language:sm language:sn language:so language:sq language:sr language:st language:sv language:sw language:szl language:ta language:te language:tet language:tg language:th language:ti language:tk language:tl language:tlh language:to language:tr language:ts language:tt language:tw language:ug language:uk language:umb language:ur language:uz language:ve language:vi language:vls language:vo language:wa language:wo language:xh language:yaq language:yi language:yo language:za language:zam language:zh language:zu license:unknown region:us
The QCRI Educational Domain Corpus (formerly QCRI AMARA Corpus) is an open multilingual collection of subtitles for educational videos and lectures collaboratively transcribed and translated over the AMARA web-based platform.
Developed by: Qatar Computing Research Institute, Arabic Language Technologies Group
The QED Corpus is made public for RESEARCH purpose only.
The corpus is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. Copyright Qatar Computing Research Institute. All rights reserved.
225 languages, 9,291 bitexts
total number of files: 271,558
total number of tokens: 371.76M
total number of sentence fragments: 30.93M
A. Abdelali, F. Guzman, H. Sajjad and S. Vogel, "The AMARA Corpus: Building parallel language resources for the educational domain", The Proceedings of the 9th International Conference on Language Resources and Evaluation (LREC'14). Reykjavik, Iceland, 2014. Pp. 1856-1862. Isbn. 978-2-9517408-8-4.
---
annotations_creators:
- found
language_creators:
- found
language:
- aa
- ab
- ae
- aeb
- af
- ak
- am
- an
- ar
- arq
- arz
- as
- ase
- ast
- av
- ay
- az
- ba
- be
- ber
- bg
- bh
- bi
- bm
- bn
- bnt
- bo
- br
- bs
- bug
- ca
- ce
- ceb
- ch
- cho
- cku
- cnh
- co
- cr
- cs
- cu
- cv
- cy
- da
- de
- dv
- dz
- ee
- efi
- el
- en
- eo
- es
- et
- eu
- fa
- ff
- fi
- fil
- fj
- fo
- fr
- ga
- gd
- gl
- gn
- gu
- ha
- hai
- haw
- haz
- hch
- he
- hi
- ho
- hr
- ht
- hu
- hup
- hus
- hy
- hz
- ia
- id
- ie
- ig
- ik
- inh
- io
- iro
- is
- it
- iu
- ja
- jv
- ka
- kar
- ki
- kj
- kk
- kl
- km
- kn
- ko
- kr
- ksh
- ku
- kv
- kw
- ky
- la
- lb
- lg
- li
- lkt
- lld
- ln
- lo
- lt
- ltg
- lu
- luo
- luy
- lv
- mad
- mfe
- mg
- mi
- mk
- ml
- mn
- mni
- moh
- mos
- mr
- ms
- mt
- mus
- my
- nb
- nci
- nd
- ne
- nl
- nn
- nso
- nv
- ny
- oc
- om
- or
- pa
- pam
- pap
- pi
- pl
- pnb
- prs
- ps
- pt
- qu
- rm
- rn
- ro
- ru
- rup
- rw
- sa
- sc
- scn
- sco
- sd
- sg
- sgn
- sh
- si
- sk
- sl
- sm
- sn
- so
- sq
- sr
- st
- sv
- sw
- szl
- ta
- te
- tet
- tg
- th
- ti
- tk
- tl
- tlh
- to
- tr
- ts
- tt
- tw
- ug
- uk
- umb
- ur
- uz
- ve
- vi
- vls
- vo
- wa
- wo
- xh
- yaq
- yi
- yo
- za
- zam
- zh
- zu
license:
- unknown
multilinguality:
- multilingual
size_categories:
- 100K<n<1M
source_datasets:
- original
task_categories:
- translation
task_ids: []
paperswithcode_id: null
pretty_name: QedAmara
dataset_info:
- config_name: ar-ko
  features:
  - name: id
    dtype: string
  - name: translation
    dtype:
      translation:
        languages:
        - ar
        - ko
  splits:
  - name: train
    num_bytes: 79605277
    num_examples: 592589
  download_size: 23410393
  dataset_size: 79605277
- config_name: de-fr
  features:
  - name: id
    dtype: string
  - name: translation
    dtype:
      translation:
        languages:
        - de
        - fr
  splits:
  - name: train
    num_bytes: 75861416
    num_examples: 407224
  download_size: 26579871
  dataset_size: 75861416
- config_name: es-it
  features:
  - name: id
    dtype: string
  - name: translation
    dtype:
      translation:
        languages:
        - es
        - it
  splits:
  - name: train
    num_bytes: 80650321
    num_examples: 447369
  download_size: 28344317
  dataset_size: 80650321
- config_name: en-ja
  features:
  - name: id
    dtype: string
  - name: translation
    dtype:
      translation:
        languages:
        - en
        - ja
  splits:
  - name: train
    num_bytes: 86731218
    num_examples: 497531
  download_size: 29836171
  dataset_size: 86731218
- config_name: he-nl
  features:
  - name: id
    dtype: string
  - name: translation
    dtype:
      translation:
        languages:
        - he
        - nl
  splits:
  - name: train
    num_bytes: 51448732
    num_examples: 273165
  download_size: 16642865
  dataset_size: 51448732
---

# Dataset Card for QedAmara

## Table of Contents
- [Dataset Description](#dataset-description)
  - [Dataset Summary](#dataset-summary)
  - [Supported Tasks and Leaderboards](#supported-tasks-and-leaderboards)
  - [Languages](#languages)
- [Dataset Structure](#dataset-structure)
  - [Data Instances](#data-instances)
  - [Data Fields](#data-fields)
  - [Data Splits](#data-splits)
- [Dataset Creation](#dataset-creation)
  - [Curation Rationale](#curation-rationale)
  - [Source Data](#source-data)
  - [Annotations](#annotations)
  - [Personal and Sensitive Information](#personal-and-sensitive-information)
- [Considerations for Using the Data](#considerations-for-using-the-data)
  - [Social Impact of Dataset](#social-impact-of-dataset)
  - [Discussion of Biases](#discussion-of-biases)
  - [Other Known Limitations](#other-known-limitations)
- [Additional Information](#additional-information)
  - [Dataset Curators](#dataset-curators)
  - [Licensing Information](#licensing-information)
  - [Citation Information](#citation-information)
  - [Contributions](#contributions)

## Dataset Description

- **Homepage:** http://opus.nlpl.eu/QED.php
- **Repository:** None
- **Paper:** https://www.aclweb.org/anthology/L14-1675/
- **Leaderboard:** [More Information Needed]
- **Point of Contact:** [More Information Needed]

### Dataset Summary


To load a language pair which isn't part of the config, all you need to do is specify the language code as pairs.
You can find the valid pairs in Homepage section of Dataset Description: http://opus.nlpl.eu/QED.php
E.g.

`dataset = load_dataset("qed_amara", lang1="cs", lang2="nb")`


### Supported Tasks and Leaderboards

[More Information Needed]

### Languages

The languages in the dataset are:
- aa
- ab
- ae
- aeb
- af
- aka: `ak`
- amh: `am`
- an
- ar
- arq
- arz
- as
- ase
- ast
- av
- ay
- az
- ba
- bam: `bm`
- be
- ber
- bg
- bh
- bi
- bn
- bnt
- bo
- br
- bs
- bug
- ca
- ce
- ceb
- ch
- cho
- cku
- cnh
- co
- cr
- cs
- cu
- cv
- cy
- da
- de
- dv
- dz
- ee
- efi
- el
- en
- eo
- es
- et
- eu
- fa
- ff
- fi
- fil
- fj
- fo
- fr
- ful: `ff`
- ga
- gd
- gl
- gn
- gu
- hai
- hau: `ha`
- haw
- haz
- hb: ?
- hch
- he
- hi
- ho
- hr
- ht
- hu
- hup
- hus
- hy
- hz
- ia
- ibo: `ig`
- id
- ie
- ik
- inh
- io
- iro
- is
- it
- iu
- ja
- jv
- ka
- kar
- kau: `kr`
- kik: `ki`
- kin: `rw`
- kj
- kk
- kl
- km
- kn
- ko
- ksh
- ku
- kv
- kw
- ky
- la
- lb
- lg
- li
- lin: `ln`
- lkt
- lld
- lo
- lt
- ltg
- lu
- luo
- luy
- lv
- mad
- mfe
- mi
- mk
- ml
- mlg: `mg`
- mn
- mni
- mo: Moldavian (deprecated tag; preferred value: Romanian; Moldavian; Moldovan (`ro`))
- moh
- mos
- mr
- ms
- mt
- mus
- my
- nb
- nci
- nd
- ne
- nl
- nn
- nso
- nv
- nya: `ny`
- oc
- or
- orm: `om`
- pam
- pan: `pa`
- pap
- pi
- pl
- pnb
- prs
- ps
- pt
- que: `qu`
- rm
- ro
- ru
- run: `rn`
- rup
- ry: ?
- sa
- sc
- scn
- sco
- sd
- sg
- sgn
- sh
- si
- sk
- sl
- sm
- sna: `sn`
- som: `so`
- sot: `st`
- sq
- sr
- srp: `sr`
- sv
- swa: `sw`
- szl
- ta
- te
- tet
- tg
- th
- tir: `ti`
- tk
- tl
- tlh
- to
- tr
- ts
- tt
- tw
- ug
- uk
- umb
- ur
- uz
- ve
- vi
- vls
- vo
- wa
- wol: `wo`
- xh
- yaq
- yi
- yor: `yo`
- za
- zam
- zh
- zul: `zu`

## Dataset Structure

### Data Instances

Here are some examples of questions and facts:


### Data Fields

[More Information Needed]

### Data Splits

[More Information Needed]

## Dataset Creation

### Curation Rationale

[More Information Needed]

### Source Data

[More Information Needed]

#### Initial Data Collection and Normalization

[More Information Needed]

#### Who are the source language producers?

[More Information Needed]

### Annotations

[More Information Needed]

#### Annotation process

[More Information Needed]

#### Who are the annotators?

[More Information Needed]

### Personal and Sensitive Information

[More Information Needed]

## Considerations for Using the Data

### Social Impact of Dataset

[More Information Needed]

### Discussion of Biases

[More Information Needed]

### Other Known Limitations

[More Information Needed]

## Additional Information

### Dataset Curators

[More Information Needed]

### Licensing Information

[More Information Needed]

### Citation Information

[More Information Needed]

### Contributions

Thanks to [@abhishekkrthakur](https://github.com/abhishekkrthakur) for adding this dataset.