Spaces:
Running
on
CPU Upgrade
Running
on
CPU Upgrade
whisper-leaderboard
#28
by
Steveeeeeeen
HF staff
- opened
What does this PR do?
This PR adds a dedicated Whisper Model Leaderboard tab to compare different Whisper model implementations across various backends and hardware configurations.
Key Changes:
- Added new environment variables for Whisper-specific repository and paths
- Extended
load_all_info_from_dataset_hub()
to handle both original and Whisper evaluation results - Created a new "🔄 Whisper Model Leaderboard" tab with:
- Additional columns for Backend and Device information
- Interactive filters to compare results by backend type and hardware
- Same evaluation metrics (WER, RTFx) as the main leaderboard
Why?
This dedicated view will help users:
- Compare performance across different Whisper implementations
- Understand speed/accuracy tradeoffs on different hardware
- Make informed decisions about which backend to use for their specific needs
Testing
- Verified that both original and Whisper leaderboards load correctly
- Tested filtering functionality with multiple backend/device combinations
- Confirmed that all WER and RTFx metrics display properly
TODO
- Add secrets in the Space settings:
QUEUE_REPO_WHISPER: Path to the repository containing Whisper evaluation results
QUEUE_PATH_WHISPER: Local path where Whisper results will be stored
Screenshots
Steveeeeeeen
changed pull request status to
open