region:us
# Dataset Card for Bokmål-Nynorsk Language Detection (main_train_split)

## Dataset Summary

This dataset is intended for language detection for Bokmål to Nynorsk and vice versa. It contains 800,000 sentence pairs, sourced from Språkbanken and pruned to avoid overlap with the NorBench dataset. The data comes from translations of news text from Norsk telegrambyrå (NTB), performed by Nynorsk pressekontor (NPK). In addition the dev and test set has 1000 entries.

## Data Collection

- **Period**: February 2011 to December 2022
- **Source**: [Språkbanken](https://www.nb.no/sprakbanken/)
- **Size**: 800,000 sentence pairs
- **Format**: JSON-lines (with `text` , `label`, `label_full` fields)

### Processing Steps

1. Pruned to avoid overlap with NorBench
2. Deduplicated
3. Shuffled with a fixed seed (42)

## Usage

Intended for training Bokmål-Nynorsk detection models. For more details, refer to the repository where the dataset preparation script and the actual dataset reside.