task_categories:image-to-text size_categories:1K<n<10K language:en medical region:us
---
dataset_info:
  features:
  - name: image
    dtype: image
  - name: text
    dtype: string
  splits:
  - name: train
    num_bytes: 1108945726.54
    num_examples: 6060
  download_size: 1108991167
  dataset_size: 1108945726.54
task_categories:
- image-to-text
language:
- en
tags:
- medical
size_categories:
- 1K<n<10K
---
# Indiana University Chest Xray Dataset Card

## Data sources:
This is a converted and processed version of the open access pneumonia chest x-ray dataset provided by the indiana university.<br>
You can see its information page [here](https://openi.nlm.nih.gov/faq).<br>
The compressed images in the png format were downloaded from [here](https://openi.nlm.nih.gov/imgs/collections/NLMCXR_png.tgz) and the corresponding reports from [here](https://openi.nlm.nih.gov/imgs/collections/NLMCXR_reports.tgz).

## Data fields:
There are two fields: image and text.
The images are the x-rays and the texts are their associated findings.

## Preprocessing done:

1. **Make all text lowercase**: Convert all text to lowercase to ensure consistent and case-insensitive processing.

2. **Remove all punctuation**: Eliminate any punctuation marks (e.g., periods, commas, exclamation marks) from the text to avoid interference in language analysis.

3. **Remove all numbers**: Eliminate all numeric characters from the text since they might not be relevant for certain natural language processing tasks.

4. **Remove all words with 2 or more Xs in a row**: Remove any words that contain two or more consecutive occurrences of the letter "X" as they may not contribute meaningful information.

5. **Remove the bottom and top 2% of text by length**: Discard the shortest and longest text samples, removing the bottom 2% and top 2% of the text's length, respectively. This step is aimed at reducing the impact of outliers and ensuring a more balanced dataset.