task_categories:token-classification annotations_creators:expert-generated multilinguality:monolingual size_categories:1K<n<10K language:en license:cc-by-4.0 astronomy region:us
---
annotations_creators:
- expert-generated
license: cc-by-4.0
task_categories:
- token-classification
language:
- en
multilinguality:
- monolingual
size_categories:
- 1K<n<10K
tags:
- astronomy
---

# Function Of Citation in Astrophysics Literature (FOCAL): Dataset and Task
*Can you explain why the authors made a given citation?*

This dataset was created as a [shared task](https://ui.adsabs.harvard.edu/WIESP/2023/shared_task_1) for [WIESP @ AACL-IJCNLP 2023](https://ui.adsabs.harvard.edu/WIESP/2023/).

## Dataset Description
Datasets are in JSON Lines format (each line is a json dictionary).  

Each entry consists of a dictionary with the following keys:
- `"Identifier"`: unique string to identify the entry
- `"Paragraph"`: text string from an astrophysics paper 
- `"Citation Text"`: list of strings forming the citation (most often a single string, but sometimes the citation text is split up)
- `"Citation Start End"`: list of integer pairs denoting where the citation starts and end in `"Paragraph"` (most often a single pair, sometimes the citation text is split up, if so follows the order in `"Citation Text"`)
- `"Functions Text"`: list of strings highlighting parts of the paragraph that explain the function of the citation
- `"Functions Label"`: list of strings with the label for each text element in `"Functions Text"` (in same order)
- `"Functions Start End"`: list of integer pairs denoting where the elements in `"Functions Text"` start and end in `"Paragraph"`(in same order)

start and end are defined by the character position in the `"Paragraph"` string.


## Instructions for Workshop Participants:
How to load the data using the Huggingface library:
```python
from datasets import load_dataset
dataset = load_dataset("adsabs/FOCAL") # !!! Only loads the training split. Validation and testing splits will be added after the shared task of [WIESP-2023](https://ui.adsabs.harvard.edu/WIESP/2023/) has ended.
```

How to load the data if you cloned the repository locally:  
(assuming `./FOCAL-TRAINING.jsonl` is in the current directory, change as needed)
- python (as list of dictionaries):
```python
import json
with open("./FOCAL-TRAINING.jsonl", 'r') as f:
    focal_training_from_json = [json.loads(l) for l in list(f)]
```
 - into Huggingface (as a Huggingface Dataset):
```python
from datasets import Dataset
focal_training_from_json = Dataset.from_json(path_or_paths="./FOCAL-TRAINING.jsonl")
```

## File List
``` 
├── FOCAL-TRAINING.jsonl (2421 samples for training)
├── FOCAL-VALIDATION-NO-LABELS.jsonl (606 samples for validation without the labels. Used during the shared task of [WIESP-2023](https://ui.adsabs.harvard.edu/WIESP/2023/)  
├── FOCAL-TESTING-NO-LABELS.jsonl (821 samples for testing without the labels. Used during the shared task of [WIESP-2023](https://ui.adsabs.harvard.edu/WIESP/2023/)
├── /scoring_scripts/score_focal_seqeval.py (scoring scripts used during the shared task of [WIESP-2023](https://ui.adsabs.harvard.edu/WIESP/2023/)
├── /scoring_scripts/score_focal_labels_only.py (scoring scripts used during the shared task of [WIESP-2023](https://ui.adsabs.harvard.edu/WIESP/2023/)
├── /data/train.parquet (train split of FOCAL)
├── README.MD (this file)  
└──  
```

Maintainer: Felix Grezes (ORCID: 0000-0001-8714-7774)  
Data annotator: Tom Allen (ORCID: 0000-0002-5532-4809)