task_categories:conversational task_categories:question-answering task_categories:image-to-text size_categories:100K<n<1M license:cc-by-sa-4.0 medical region:us
---
license: cc-by-sa-4.0
task_categories:
- conversational
- question-answering
- image-to-text
tags:
- medical
size_categories:
- 100K<n<1M
---

# Dataset Card for Dataset Name
CXR for medical multimodal LLMs

## Dataset Description

- **Homepage:** 
- **Repository:** 
- **Paper:** 
- **Leaderboard:** 
- **Point of Contact:** 

### Dataset Summary

This dataset aims to provide medical conversations with contextual images.  \
Most of the questions ensure the model understands what anomalies are present within the CXR images. \ 
There are also follow-up questions to teach the LLM how to follow up after identifying the anomaly.

There is a total of 104892 human-bot conversations with contextual images \

50,021 images from chexpert \
5,229 images from https://www.kaggle.com/datasets/paultimothymooney/chest-xray-pneumonia \
44,714 images from vinbigdata \
9 images from vqarad \
4,919 images from slake \

The labels include a image token which should be removed or changed depending on your model architecture/training

## Dataset Structure

Each conversation has the following structure:



### Annotation process
Annotations were mostly done by gpt4, some labels were written manually while referring to medical school texts


### Personal and Sensitive Information
All data have already underwent de-identification with accordance to HIPAA requirements

## Limitations & Considerations for Using the Data
Labels may not be sound medical advice, labels were scanned thru by medical students and not licensed doctors. \
There are many questions asking "What does this image show" with "The image shows a chest X-ray." as answer, this may cause overfitting if only this dataset is using

## Additional Information
This dataset was created with Llava in mind, so it can be used straightaway with Llava's trainer. Note that the prompt should be changed if you wish to train Llava on this dataset


### Dataset Curators

