ChatGPT und Co.: Diese KI-Tools erleichtern Ihren Arbeitsalltag
Die rasante Entwicklung der Künstlichen Intelligenz (KI) hat längst Einzug in den Alltag vieler Unternehmen gefunden. Große Konzerne nutzen KI-Technologien ...
Github
Ob für persönliche Notizen oder berufliche Zwecke, diese KI-Transkriptionstools sparen Zeit und Aufwand. Whisper steht in fünf verschiedenen Größen auf <anchor>Github</anchor> zur Verfügung, wobei nur die größte Modellversion mit anderen Sprachen umgehen kann: Die anderen Versionen laufen ausschließlich in englischer Sprache.
GitHub - openai/whisper: Robust Speech Recognition via Large-Scale Weak Supervision
## Navigation Menu

Toggle navigation

Explore

By size

By industry

By use case

Topics

Explore

Repositories

Available add-ons

Search or jump to...

# Search code, repositories, users, issues, pull requests...

Search

Clear

# Provide feedback

We read every piece of feedback, and take your input very seriously.

Include my email address so I can be contacted

Cancel
Submit feedback

# Saved searches

## Use saved searches to filter your results more quickly

Name

Query

To see all available qualifiers, see our documentation.

Cancel
Create saved search

You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert

{{ message }}

Robust Speech Recognition via Large-Scale Weak Supervision

### License

Notifications You must be signed in to change notification settings

Additional navigation options

# openai/whisper

This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.

main

Go to file

Code

## Folders and files

NameName

Last commit message

Last commit date

## Latest commit

## History

View all files

## Repository files navigation

- README
- MIT license

# Whisper

Whisper is a general-purpose speech recognition model. It is trained on a large dataset of diverse audio and is also a multitasking model that can perform multilingual speech recognition, speech translation, and language identification.

## Approach

A Transformer sequence-to-sequence model is trained on various speech processing tasks, including multilingual speech recognition, speech translation, spoken language identification, and voice activity detection. These tasks are jointly represented as a sequence of tokens to be predicted by the decoder, allowing a single model to replace many stages of a traditional speech-processing pipeline. The multitask training format uses a set of special tokens that serve as task specifiers or classification targets.

## Setup

We used Python 3.9.9 and PyTorch 1.10.1 to train and test our models, but the codebase is expected to be compatible with Python 3.8-3.11 and recent PyTorch versions. The codebase also depends on a few Python packages, most notably OpenAI's tiktoken for their fast tokenizer implementation. You can download and install (or update to) the latest release of Whisper with the following command:

Alternatively, the following command will pull and install the latest commit from this repository, along with its Python dependencies:

To update the package to the latest version of this repository, please run:

It also requires the command-line tool `ffmpeg` to be installed on your system, which is available from most package managers:

You may need `rust` installed as well, in case tiktoken does not provide a pre-built wheel for your platform. If you see installation errors during the `pip install` command above, please follow the Getting started page to install Rust development environment. Additionally, you may need to configure the `PATH` environment variable, e.g. `export PATH=&quot;$HOME/.cargo/bin:$PATH&quot;`. If the installation fails with `No module named 'setuptools_rust'`, you need to install `setuptools_rust`, e.g. by running:

## Available models and languages

There are five model sizes, four with English-only versions, offering speed and accuracy tradeoffs. Below are the names of the available models and their approximate memory requirements and inference speed relative to the large model; actual speed may vary depending on many factors including the available hardware.

SizeParametersEnglish-only modelMultilingual modelRequired VRAMRelative speedtiny39 M`tiny.en``tiny`~1 GB~32xbase74 M`base.en``base`~1 GB~16xsmall244 M`small.en``small`~2 GB~6xmedium769 M`medium.en``medium`~5 GB~2xlarge1550 MN/A`large`~10 GB1x

The `.en` models for English-only applications tend to perform better, especially for the `tiny.en` and `base.en` models. We observed that the difference becomes less significant for the `small.en` and `medium.en` models.

Whisper's performance varies widely depending on the language. The figure below shows a performance breakdown of `large-v3` and `large-v2` models by language, using WERs (word error rates) or CER (character error rates, shown in Italic) evaluated on the Common Voice 15 and Fleurs datasets. Additional WER/CER metrics corresponding to the other models and datasets can be found in Appendix D.1, D.2, and D.4 of the paper, as well as the BLEU (Bilingual Evaluation Understudy) scores for translation in Appendix D.3.

## Command-line usage

The following command will transcribe speech in audio files, using the `medium` model:

The default setting (which selects the `small` model) works well for transcribing English. To transcribe an audio file containing non-English speech, you can specify the language using the `--language` option:

Adding `--task translate` will translate the speech into English:

Run the following to view all available options:

See tokenizer.py for the list of all available languages.

## Python usage

Transcription can also be performed within Python:

Internally, the `transcribe()` method reads the entire file and processes the audio with a sliding 30-second window, performing autoregressive sequence-to-sequence predictions on each window.

Below is an example usage of `whisper.detect_language()` and `whisper.decode()` which provide lower-level access to the model.

## More examples

Please use the 🙌 Show and tell category in Discussions for sharing more example usages of Whisper and third-party extensions such as web demos, integrations with other tools, ports for different platforms, etc.

## License

Whisper's code and model weights are released under the MIT License. See LICENSE for further details.

## About

Robust Speech Recognition via Large-Scale Weak Supervision

### Resources

### License

### Stars

### Watchers

### Forks

## Releases\  10

## Contributors\  69

## Languages

## Footer

GitHub
© 2024 GitHub, Inc.

### Footer navigation

You can’t perform that action at this time.