size_categories:100K<n<1M language:en art music text-to-music music-to-text arxiv:2307.16372 region:us
---
language:
- en
tags:
- art
- music
- text-to-music
- music-to-text
pretty_name: LP-MusicCaps-MSD
size_categories:
- 100K<n<1M
---


======================================

**!important**: Be careful when using `caption_attribute_prediction` (We don't recommend to use)!

======================================

# Dataset Card for LP-MusicCaps-MSD

## Dataset Description

- **Repository:** [LP-MusicCaps repository](https://github.com/seungheondoh/lp-music-caps)
- **Paper:** [ArXiv](https://arxiv.org/abs/2307.16372)

## Dataset Summary

**LP-MusicCaps** is a Large Language Model based Pseudo Music Caption dataset for `text-to-music` and `music-to-text` tasks. We construct the music-to-caption pairs with tag-to-caption generation (using three existing multi-label tag datasets and four task instructions). The data sources are MusicCaps, Magnatagtune, and Million Song Dataset ECALS subset.

- **LP-MusicCaps MSD (This Repo)**: 0.5M Audio with 2.2M Caption. We utilize 1054 unique tags in the [MSD-ECALS](https://github.com/SeungHeonDoh/msd-subsets) to perform tag-to-caption generation through LLM.
- [LP-MusicCaps MTT](https://huggingface.co/datasets/seungheondoh/LP-MusicCaps-MTT): 22k Audio with 88k Caption
- [LP-MusicCaps MC](https://huggingface.co/datasets/seungheondoh/LP-MusicCaps-MC): 6k Audio with 22k Caption.


## Data Instances

Each instance in LP-MusicCaps MSD (This Repo) represents multiple image-text pair information with meta-attributes:

```
{
  'track_id': 'TRIHXPZ128F1466744',
  'title': 'In The Sunshine',
  'artist_name': 'ARRESTED DEVELOPMENT',
  'release': 'Zingalamaduni',
  'year': 1994,
  'tag': ['laid back mellow',
    'hip hop',
    'rnb',
    'amiable good natured',
    'rap',
    'urban',
    'gentle',
    'political rap',
    'soul',
    'calm peaceful',
    'summery',
    'cheerful',
    'alternative rap'
  ],
  'caption_writing': 'An amiable and laid back alternative rap tune, this summery and cheerful song blends elements of soul and R&B with a gentle, mellow rap flow to create a calm and peaceful urban vibe that is both hip hop and political in its message.',
  'caption_summary': 'This summery, alternative rap song is a mellow and gentle blend of hip hop, RnB, and political rap with a cheerful and amiable good natured vibe.',
  'caption_paraphrase': 'This laid back mellow rap song infuses soulful and urban elements while showcasing a gentle and amiable good natured vibe, perfect for a summery day. With hints of cheerful R&B and hip hop, the alternative political rap lyrics bring balance to this peaceful and calming tune.',
  'caption_attribute_prediction': 'This mellow, soulful tune is a perfect blend of rap and RnB, with a gentle beat and smooth flow that will transport you to the laid-back urban vibes of a sunny summertime day. The amiable good-natured lyrics touch on political themes, while the alternative rap style adds a cheerful, upbeat twist to the message. Overall, this is a hip-hop gem thats sure to put you in a peaceful, calm state of mind.',
  'path': '3/0/303545.clip.mp3'
}
```

## Pseudo Caption Example:

Input Tags: 

*"video game theme, no singer, instrumental, analog sounding, small keyboard, beatboxing, playful, cheerful, groovy"*

Output Pseudo Captions

*"instrumental track has a joyful and playful vibe, perfect for a video game theme. With no singer, the analog-sounding music features a small keyboard and beatboxing, creating a groovy and cheerful atmosphere"*

[More Information for pseudo caption generation](https://github.com/seungheondoh/lp-music-caps/blob/main/lpmc/llm_captioning/generate.py)
 

## Data Fields

| Name                         | Type            | Description                                                          |
|------------------------------|-----------------|----------------------------------------------------------------------|
| track_id                     | string          | Unique identifier for the track                                      |
| title                        | string          | Title of the song                                                    |
| artist_name                  | string          | Name of the artist performing the song                               |
| release                      | string          | Release name or album name of the song                               |
| year                         | integer         | Year of the song's release                                           |
| tag                          | list of strings | List of tags associated with the song                                |
| caption_writing              | string          | Pseudo caption generated through a writing instruction               |
| caption_summary              | string          | Pseudo caption generated through a summary instruction               |
| caption_paraphrase           | string          | Pseudo caption generated through a paraphrase instruction            |
| caption_attribute_prediction | string          | Pseudo caption generated through an attribute_prediction instruction |
| path                         | string          | File path or location of the audio clip                              |

## Data Splits

- train: 444865
- valid: 34481
- test: 34631
    
## Considerations for Using the Data

The LP-MusicCaps dataset is recommended to be used for research purposes. Due to the wrong labeling issue, we recommend not using caption_attribute_prediction and pseudo_attribute unless it is specifically for large-scale pretraining. Additionally, the field "is_crawled" indicates the samples used in the reference paper mentioned below.

## Discussion of Biases

It will be described in a paper to be released soon.

## Other Known Limitations

It will be described in a paper to be released soon.