Update README.md with HF implementation

README.md

@@ -6,9 +6,9 @@ license: cc-by-nc-4.0
 
 # MusicGen - Small - 300M
 
-Audiocraft provides the code and models for MusicGen, a simple and controllable model for music generation. 
-MusicGen is a single stage auto-regressive Transformer model trained over a 32kHz EnCodec tokenizer with 4 codebooks sampled at 50 Hz. 
-Unlike existing methods like MusicLM, MusicGen doesn't not require a self-supervised semantic representation, and it generates all 4 codebooks in one pass. 
+MusicGen is a text-to-music model capable of genreating high-quality music samples conditioned on text descriptions or audio prompts.
+It is is a single stage auto-regressive Transformer model trained over a 32kHz EnCodec tokenizer with 4 codebooks sampled at 50 Hz. 
+Unlike existing methods, like MusicLM, MusicGen doesn't not require a self-supervised semantic representation, and it generates all 4 codebooks in one pass. 
 By introducing a small delay between the codebooks, we show we can predict them in parallel, thus having only 50 auto-regressive steps per second of audio.
 
 MusicGen was published in [Simple and Controllable Music Generation](https://arxiv.org/abs/2306.05284) by *Jade Copet, Felix Kreuk, Itai Gat, Tal Remez, David Kant, Gabriel Synnaeve, Yossi Adi, Alexandre Défossez*.
@@ -21,48 +21,64 @@ Four checkpoints are released:
 
 ## Example
 
-Try out MusicGen yourself!
+Try out MusicGen yourself! 
 
-- <a target="_blank" href="https://colab.research.google.com/drive/1fxGqfg96RBUvGxZ1XXN07s3DthrKUl4-?usp=sharing">
-  <img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Colab"/>
+<a target="_blank" href="https://colab.research.google.com/github/sanchit-gandhi/notebooks/blob/main/MusicGen.ipynb"> 
+  <img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Colab"/> 
 </a>
 
-- <a target="_blank" href="https://huggingface.co/spaces/facebook/MusicGen">
-  <img src="https://huggingface.co/datasets/huggingface/badges/raw/main/open-in-hf-spaces-sm.svg" alt="Open in HugginFace"/>
+<a target="_blank" href="https://huggingface.co/spaces/facebook/MusicGen">
+  <img src="https://huggingface.co/datasets/huggingface/badges/raw/main/open-in-hf-spaces-sm.svg" alt="Open in HuggingFace"/>
 </a>
 
-- You can run MusicGen locally as well:
+### Usage
 
-1. First install the [`audiocraft` library](https://github.com/facebookresearch/audiocraft)
-```
-pip install git+https://github.com/facebookresearch/audiocraft.git
-```
+You can run MusicGen locally with the 🤗 Transformers library from version 4.31.0 onwards.
+
+1. First install the 🤗 [Transformers library](https://github.com/huggingface/transformers) from main:
 
-2. Make sure to have [`ffmpeg`](https://ffmpeg.org/download.html) installed:
 ```
-apt get install ffmpeg
+pip install git+https://github.com/huggingface/transformers.git
 ```
 
-3. Run the following Python code:
+2. Run the following Python code to generate text-conditional audio samples:
 
 ```py
-import torchaudio
+from transformers import AutoProcessor, MusicgenForConditionalGeneration
 
-from audiocraft.models import MusicGen
-from audiocraft.data.audio import audio_write
 
-model = MusicGen.get_pretrained('small')
-model.set_generation_params(duration=8)  # generate 8 seconds.
+processor = AutoProcessor.from_pretrained("facebook/musicgen-small")
+model = MusicgenForConditionalGeneration.from_pretrained("facebook/musicgen-small")
 
-descriptions = ['happy rock', 'energetic EDM', 'sad jazz']
+inputs = processor(
+    text=["80s pop track with bassy drums and synth", "90s rock song with loud guitars and heavy drums"],
+    padding=True,
+    return_tensors="pt",
+)
 
-wav = model.generate(descriptions)  # generates 3 samples.
+audio_values = model.generate(**inputs, do_sample=True, guidance_scale=3, max_new_tokens=256)
+```
+
+3. Listen to the audio samples either in an ipynb notebook:
+
+```py
+from IPython.display import Audio
 
-for idx, one_wav in enumerate(wav):
-    # Will save under {idx}.wav, with loudness normalization at -14 db LUFS.
-    audio_write(f'{idx}', one_wav.cpu(), model.sample_rate, strategy="loudness")
+sampling_rate = model.config.audio_encoder.sampling_rate
+Audio(audio_values[0].numpy(), rate=sampling_rate)
 ```
 
+Or save them as a `.wav` file using a third-party library, e.g. `scipy`:
+
+```py
+import scipy
+
+sampling_rate = model.config.audio_encoder.sampling_rate
+scipy.io.wavfile.write("musicgen_out.wav", rate=sampling_rate, data=audio_values[0, 0].numpy())
+```
+
+For more details on using the MusicGen model for inference using the 🤗 Transformers library, refer to the [MusicGen docs](https://huggingface.co/docs/transformers/model_doc/musicgen).
+
 ## Model details
 
 **Organization developing the model:** The FAIR team of Meta AI.