Error loading file audio when downloading the Common Voice dataset directly from the Hub
{'login': 'aliceinland', 'id': 30385910, 'node_id': 'MDQ6VXNlcjMwMzg1OTEw', 'avatar_url': 'https://avatars.githubusercontent.com/u/30385910?v=4', 'gravatar_id': '', 'url': 'https://api.github.com/users/aliceinland', 'html_url': 'https://github.com/aliceinland', 'followers_url': 'https://api.github.com/users/aliceinland/followers', 'following_url': 'https://api.github.com/users/aliceinland/following{/other_user}', 'gists_url': 'https://api.github.com/users/aliceinland/gists{/gist_id}', 'starred_url': 'https://api.github.com/users/aliceinland/starred{/owner}{/repo}', 'subscriptions_url': 'https://api.github.com/users/aliceinland/subscriptions', 'organizations_url': 'https://api.github.com/users/aliceinland/orgs', 'repos_url': 'https://api.github.com/users/aliceinland/repos', 'events_url': 'https://api.github.com/users/aliceinland/events{/privacy}', 'received_events_url': 'https://api.github.com/users/aliceinland/received_events', 'type': 'User', 'site_admin': False}
[{'id': 1935892857, 'node_id': 'MDU6TGFiZWwxOTM1ODkyODU3', 'url': 'https://api.github.com/repos/huggingface/datasets/labels/bug', 'name': 'bug', 'color': 'd73a4a', 'default': True, 'description': "Something isn't working"}]
[]
['Hi ! It could an issue with torchaudio, which version of torchaudio are you using ? Can you also try updating `datasets` to 2.0.0 and see if it works ?'
 'I _might_ have a similar issue. I\'m trying to use the librispeech_asr dataset and read it with soundfile.\r\n\r\n```python\r\nfrom datasets import load_dataset, load_metric\r\nfrom transformers import Speech2TextForConditionalGeneration, Speech2TextProcessor\r\nimport soundfile as sf\r\n\r\nlibrispeech_eval = load_dataset("librispeech_asr", "clean", split="test")  # change to "other" for other test dataset\r\nwer = load_metric("wer")\r\n\r\nmodel = Speech2TextForConditionalGeneration.from_pretrained("facebook/s2t-small-librispeech-asr").to("cuda")\r\nprocessor = Speech2TextProcessor.from_pretrained("facebook/s2t-small-librispeech-asr", do_upper_case=True)\r\n\r\ndef map_to_array(batch):\r\n    speech, _ = sf.read(batch["file"])\r\n    batch["speech"] = speech\r\n    return batch\r\n\r\nlibrispeech_eval = librispeech_eval.map(map_to_array)\r\n\r\ndef map_to_pred(batch):\r\n    features = processor(batch["speech"], sampling_rate=16000, padding=True, return_tensors="pt")\r\n    input_features = features.input_features.to("cuda")\r\n    attention_mask = features.attention_mask.to("cuda")\r\n\r\n    gen_tokens = model.generate(input_ids=input_features, attention_mask=attention_mask)\r\n    batch["transcription"] = processor.batch_decode(gen_tokens, skip_special_tokens=True)\r\n    return batch\r\n\r\nresult = librispeech_eval.map(map_to_pred, batched=True, batch_size=8, remove_columns=["speech"])\r\n\r\nprint("WER:", wer(predictions=result["transcription"], references=result["text"]))\r\n```\r\n\r\nThe code is taken directly from "https://huggingface.co/facebook/s2t-small-librispeech-asr".\r\n\r\nThe short error code is "RuntimeError: Error opening \'6930-75918-0000.flac\': System error." (it can\'t find the first file), and I agree, I can\'t find the file either. The dataset has downloaded correctly (it says), but on the location, there are only ".arrow" files, no ".flac" files.\r\n\r\n**Error message:**\r\n\r\n```python\r\nRuntimeError                              Traceback (most recent call last)\r\nInput In [15], in <cell line: 16>()\r\n     13     batch["speech"] = speech\r\n     14     return batch\r\n---> 16 librispeech_eval = librispeech_eval.map(map_to_array)\r\n     18 def map_to_pred(batch):\r\n     19     features = processor(batch["speech"], sampling_rate=16000, padding=True, return_tensors="pt")\r\n\r\nFile C:\\ProgramData\\Miniconda3\\envs\\noise_cancel\\lib\\site-packages\\datasets\\arrow_dataset.py:1953, in Dataset.map(self, function, with_indices, with_rank, input_columns, batched, batch_size, drop_last_batch, remove_columns, keep_in_memory, load_from_cache_file, cache_file_name, writer_batch_size, features, disable_nullable, fn_kwargs, num_proc, suffix_template, new_fingerprint, desc)\r\n   1950 disable_tqdm = not logging.is_progress_bar_enabled()\r\n   1952 if num_proc is None or num_proc == 1:\r\n-> 1953     return self._map_single(\r\n   1954         function=function,\r\n   1955         with_indices=with_indices,\r\n   1956         with_rank=with_rank,\r\n   1957         input_columns=input_columns,\r\n   1958         batched=batched,\r\n   1959         batch_size=batch_size,\r\n   1960         drop_last_batch=drop_last_batch,\r\n   1961         remove_columns=remove_columns,\r\n   1962         keep_in_memory=keep_in_memory,\r\n   1963         load_from_cache_file=load_from_cache_file,\r\n   1964         cache_file_name=cache_file_name,\r\n   1965         writer_batch_size=writer_batch_size,\r\n   1966         features=features,\r\n   1967         disable_nullable=disable_nullable,\r\n   1968         fn_kwargs=fn_kwargs,\r\n   1969         new_fingerprint=new_fingerprint,\r\n   1970         disable_tqdm=disable_tqdm,\r\n   1971         desc=desc,\r\n   1972     )\r\n   1973 else:\r\n   1975     def format_cache_file_name(cache_file_name, rank):\r\n\r\nFile C:\\ProgramData\\Miniconda3\\envs\\noise_cancel\\lib\\site-packages\\datasets\\arrow_dataset.py:519, in transmit_tasks.<locals>.wrapper(*args, **kwargs)\r\n    517     self: "Dataset" = kwargs.pop("self")\r\n    518 # apply actual function\r\n--> 519 out: Union["Dataset", "DatasetDict"] = func(self, *args, **kwargs)\r\n    520 datasets: List["Dataset"] = list(out.values()) if isinstance(out, dict) else [out]\r\n    521 for dataset in datasets:\r\n    522     # Remove task templates if a column mapping of the template is no longer valid\r\n\r\nFile C:\\ProgramData\\Miniconda3\\envs\\noise_cancel\\lib\\site-packages\\datasets\\arrow_dataset.py:486, in transmit_format.<locals>.wrapper(*args, **kwargs)\r\n    479 self_format = {\r\n    480     "type": self._format_type,\r\n    481     "format_kwargs": self._format_kwargs,\r\n    482     "columns": self._format_columns,\r\n    483     "output_all_columns": self._output_all_columns,\r\n    484 }\r\n    485 # apply actual function\r\n--> 486 out: Union["Dataset", "DatasetDict"] = func(self, *args, **kwargs)\r\n    487 datasets: List["Dataset"] = list(out.values()) if isinstance(out, dict) else [out]\r\n    488 # re-apply format to the output\r\n\r\nFile C:\\ProgramData\\Miniconda3\\envs\\noise_cancel\\lib\\site-packages\\datasets\\fingerprint.py:458, in fingerprint_transform.<locals>._fingerprint.<locals>.wrapper(*args, **kwargs)\r\n    452             kwargs[fingerprint_name] = update_fingerprint(\r\n    453                 self._fingerprint, transform, kwargs_for_fingerprint\r\n    454             )\r\n    456 # Call actual function\r\n--> 458 out = func(self, *args, **kwargs)\r\n    460 # Update fingerprint of in-place transforms + update in-place history of transforms\r\n    462 if inplace:  # update after calling func so that the fingerprint doesn\'t change if the function fails\r\n\r\nFile C:\\ProgramData\\Miniconda3\\envs\\noise_cancel\\lib\\site-packages\\datasets\\arrow_dataset.py:2318, in Dataset._map_single(self, function, with_indices, with_rank, input_columns, batched, batch_size, drop_last_batch, remove_columns, keep_in_memory, load_from_cache_file, cache_file_name, writer_batch_size, features, disable_nullable, fn_kwargs, new_fingerprint, rank, offset, disable_tqdm, desc, cache_only)\r\n   2316 if not batched:\r\n   2317     for i, example in enumerate(pbar):\r\n-> 2318         example = apply_function_on_filtered_inputs(example, i, offset=offset)\r\n   2319         if update_data:\r\n   2320             if i == 0:\r\n\r\nFile C:\\ProgramData\\Miniconda3\\envs\\noise_cancel\\lib\\site-packages\\datasets\\arrow_dataset.py:2218, in Dataset._map_single.<locals>.apply_function_on_filtered_inputs(inputs, indices, check_same_num_examples, offset)\r\n   2216 if with_rank:\r\n   2217     additional_args += (rank,)\r\n-> 2218 processed_inputs = function(*fn_args, *additional_args, **fn_kwargs)\r\n   2219 if update_data is None:\r\n   2220     # Check if the function returns updated examples\r\n   2221     update_data = isinstance(processed_inputs, (Mapping, pa.Table))\r\n\r\nFile C:\\ProgramData\\Miniconda3\\envs\\noise_cancel\\lib\\site-packages\\datasets\\arrow_dataset.py:1913, in Dataset.map.<locals>.decorate.<locals>.decorated(item, *args, **kwargs)\r\n   1909 decorated_item = (\r\n   1910     Example(item, features=self.features) if not batched else Batch(item, features=self.features)\r\n   1911 )\r\n   1912 # Use the LazyDict internally, while mapping the function\r\n-> 1913 result = f(decorated_item, *args, **kwargs)\r\n   1914 # Return a standard dict\r\n   1915 return result.data if isinstance(result, LazyDict) else result\r\n\r\nInput In [15], in map_to_array(batch)\r\n     11 def map_to_array(batch):\r\n---> 12     speech, _ = sf.read(batch["file"])\r\n     13     batch["speech"] = speech\r\n     14     return batch\r\n\r\nFile C:\\ProgramData\\Miniconda3\\envs\\noise_cancel\\lib\\site-packages\\soundfile.py:256, in read(file, frames, start, stop, dtype, always_2d, fill_value, out, samplerate, channels, format, subtype, endian, closefd)\r\n    170 def read(file, frames=-1, start=0, stop=None, dtype=\'float64\', always_2d=False,\r\n    171          fill_value=None, out=None, samplerate=None, channels=None,\r\n    172          format=None, subtype=None, endian=None, closefd=True):\r\n    173     """Provide audio data from a sound file as NumPy array.\r\n    174 \r\n    175     By default, the whole file is read from the beginning, but the\r\n   (...)\r\n    254 \r\n    255     """\r\n--> 256     with SoundFile(file, \'r\', samplerate, channels,\r\n    257                    subtype, endian, format, closefd) as f:\r\n    258         frames = f._prepare_read(start, stop, frames)\r\n    259         data = f.read(frames, dtype, always_2d, fill_value, out)\r\n\r\nFile C:\\ProgramData\\Miniconda3\\envs\\noise_cancel\\lib\\site-packages\\soundfile.py:629, in SoundFile.__init__(self, file, mode, samplerate, channels, subtype, endian, format, closefd)\r\n    626 self._mode = mode\r\n    627 self._info = _create_info_struct(file, mode, samplerate, channels,\r\n    628                                  format, subtype, endian)\r\n--> 629 self._file = self._open(file, mode_int, closefd)\r\n    630 if set(mode).issuperset(\'r+\') and self.seekable():\r\n    631     # Move write position to 0 (like in Python file objects)\r\n    632     self.seek(0)\r\n\r\nFile C:\\ProgramData\\Miniconda3\\envs\\noise_cancel\\lib\\site-packages\\soundfile.py:1183, in SoundFile._open(self, file, mode_int, closefd)\r\n   1181 else:\r\n   1182     raise TypeError("Invalid file: {0!r}".format(self.name))\r\n-> 1183 _error_check(_snd.sf_error(file_ptr),\r\n   1184              "Error opening {0!r}: ".format(self.name))\r\n   1185 if mode_int == _snd.SFM_WRITE:\r\n   1186     # Due to a bug in libsndfile version <= 1.0.25, frames != 0\r\n   1187     # when opening a named pipe in SFM_WRITE mode.\r\n   1188     # See http://github.com/erikd/libsndfile/issues/77.\r\n   1189     self._info.frames = 0\r\n\r\nFile C:\\ProgramData\\Miniconda3\\envs\\noise_cancel\\lib\\site-packages\\soundfile.py:1357, in _error_check(err, prefix)\r\n   1355 if err != 0:\r\n   1356     err_str = _snd.sf_error_number(err)\r\n-> 1357     raise RuntimeError(prefix + _ffi.string(err_str).decode(\'utf-8\', \'replace\'))\r\n\r\nRuntimeError: Error opening \'6930-75918-0000.flac\': System error.\r\n```\r\n\r\n**Package versions:**\r\n```python\r\npython: 3.9\r\ntransformers: 4.17.0\r\ndatasets: 2.0.0\r\nSoundFile: 0.10.3.post1\r\n```\r\n'
 'Hi ! In `datasets` 2.0 can access the audio array with `librispeech_eval[0]["audio"]["array"]` already, no need to use `map_to_array`. See our documentation on [how to process audio data](https://huggingface.co/docs/datasets/audio_process) :)\r\n\r\ncc @patrickvonplaten we will need to update the readme at [facebook/s2t-small-librispeech-asr](https://huggingface.co/facebook/s2t-small-librispeech-asr) as well as https://huggingface.co/docs/transformers/model_doc/speech_to_text'
 'Thanks!\r\n\r\nAnd sorry for posting this problem in what turned on to be an unrelated thread.\r\n\r\nI rewrote the code, and the model works. The WER is 0.137 however, so I\'m not sure if I have missed a step. I will look further into that at a later point. The transcriptions look good through manual inspection.\r\n\r\nThe rewritten code:\r\n```python\r\nfrom datasets import load_dataset, load_metric\r\nfrom transformers import Speech2TextForConditionalGeneration, Speech2TextProcessor, Wav2Vec2Processor\r\n\r\nlibrispeech_eval = load_dataset("librispeech_asr", "clean", split="test")  # change to "other" for other test dataset\r\nwer = load_metric("wer")\r\n\r\nmodel = Speech2TextForConditionalGeneration.from_pretrained("facebook/s2t-small-librispeech-asr").to("cuda")\r\nprocessor = Speech2TextProcessor.from_pretrained("facebook/s2t-small-librispeech-asr", do_upper_case=True)\r\n\r\ndef map_to_pred(batch):\r\n    audio = batch["audio"]\r\n    features = processor(audio["array"], sampling_rate=audio["sampling_rate"], padding=True, return_tensors="pt")\r\n    input_features = features.input_features.to("cuda")\r\n    attention_mask = features.attention_mask.to("cuda")\r\n\r\n    gen_tokens = model.generate(input_features=input_features, attention_mask=attention_mask)\r\n    batch["transcription"] = processor.batch_decode(gen_tokens, skip_special_tokens=True)\r\n    return batch\r\n\r\nresult = librispeech_eval.map(map_to_pred)#, batched=True, batch_size=8)\r\n\r\nprint("WER:", wer.compute(predictions=result["transcription"], references=result["text"]))\r\n```'
 'I think the issue comes from the fact that you set `batched=False` while `map_to_pred` still returns a list of strings for "transcription". You can fix it by adding `[0]` at the end of this line to get the string:\r\n```python\r\nbatch["transcription"] = processor.batch_decode(gen_tokens, skip_special_tokens=True)[0]\r\n```'
 'Updating as many model cards now as I can find'
 'https://github.com/huggingface/transformers/pull/16611']
## Describe the bug
When loading the Common_Voice dataset, by downloading it directly from the Hugging Face hub, some files can not be opened. 

## Steps to reproduce the bug
```python
import torch
import torchaudio
from datasets import load_dataset, load_metric
from transformers import Wav2Vec2ForCTC, Wav2Vec2Processor
import re

test_dataset = load_dataset("common_voice", "it", split="test")
#test_dataset = load_dataset('csv', data_files = {'test': '/workspace/Dataset/Common_Voice/cv-corpus80/it/test.csv'})
wer = load_metric("wer")

processor = Wav2Vec2Processor.from_pretrained("joorock12/wav2vec2-large-xlsr-italian")
model = Wav2Vec2ForCTC.from_pretrained("joorock12/wav2vec2-large-xlsr-italian")
model.to("cuda")

chars_to_ignore_regex = '[\,\?\.\!\-\;\:\"\“\'\�]'
resampler = torchaudio.transforms.Resample(48_000, 16_000)
```
## Expected results
The common voice dataset downloaded and correctly loaded whit the use of the hugging face datasets library. 

## Actual results
The error is:
```python
0ex [00:00, ?ex/s]
---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
<ipython-input-48-ef87f4129e6e> in <module>
      7     return batch
      8 
----> 9 test_dataset = test_dataset.map(speech_file_to_array_fn)

/opt/conda/lib/python3.8/site-packages/datasets/arrow_dataset.py in map(self, function, with_indices, with_rank, input_columns, batched, batch_size, drop_last_batch, remove_columns, keep_in_memory, load_from_cache_file, cache_file_name, writer_batch_size, features, disable_nullable, fn_kwargs, num_proc, suffix_template, new_fingerprint, desc)
   2107 
   2108         if num_proc is None or num_proc == 1:
-> 2109             return self._map_single(
   2110                 function=function,
   2111                 with_indices=with_indices,

/opt/conda/lib/python3.8/site-packages/datasets/arrow_dataset.py in wrapper(*args, **kwargs)
    516             self: "Dataset" = kwargs.pop("self")
    517         # apply actual function
--> 518         out: Union["Dataset", "DatasetDict"] = func(self, *args, **kwargs)
    519         datasets: List["Dataset"] = list(out.values()) if isinstance(out, dict) else [out]
    520         for dataset in datasets:

/opt/conda/lib/python3.8/site-packages/datasets/arrow_dataset.py in wrapper(*args, **kwargs)
    483         }
    484         # apply actual function
--> 485         out: Union["Dataset", "DatasetDict"] = func(self, *args, **kwargs)
    486         datasets: List["Dataset"] = list(out.values()) if isinstance(out, dict) else [out]
    487         # re-apply format to the output

/opt/conda/lib/python3.8/site-packages/datasets/fingerprint.py in wrapper(*args, **kwargs)
    411             # Call actual function
    412 
--> 413             out = func(self, *args, **kwargs)
    414 
    415             # Update fingerprint of in-place transforms + update in-place history of transforms

/opt/conda/lib/python3.8/site-packages/datasets/arrow_dataset.py in _map_single(self, function, with_indices, with_rank, input_columns, batched, batch_size, drop_last_batch, remove_columns, keep_in_memory, load_from_cache_file, cache_file_name, writer_batch_size, features, disable_nullable, fn_kwargs, new_fingerprint, rank, offset, disable_tqdm, desc, cache_only)
   2465                 if not batched:
   2466                     for i, example in enumerate(pbar):
-> 2467                         example = apply_function_on_filtered_inputs(example, i, offset=offset)
   2468                         if update_data:
   2469                             if i == 0:

/opt/conda/lib/python3.8/site-packages/datasets/arrow_dataset.py in apply_function_on_filtered_inputs(inputs, indices, check_same_num_examples, offset)
   2372             if with_rank:
   2373                 additional_args += (rank,)
-> 2374             processed_inputs = function(*fn_args, *additional_args, **fn_kwargs)
   2375             if update_data is None:
   2376                 # Check if the function returns updated examples

/opt/conda/lib/python3.8/site-packages/datasets/arrow_dataset.py in decorated(item, *args, **kwargs)
   2067                 )
   2068                 # Use the LazyDict internally, while mapping the function
-> 2069                 result = f(decorated_item, *args, **kwargs)
   2070                 # Return a standard dict
   2071                 return result.data if isinstance(result, LazyDict) else result

<ipython-input-48-ef87f4129e6e> in speech_file_to_array_fn(batch)
      3 def speech_file_to_array_fn(batch):
      4     batch["sentence"] = re.sub(chars_to_ignore_regex, '', batch["sentence"]).lower()
----> 5     speech_array, sampling_rate = torchaudio.load(batch["path"])
      6     batch["speech"] = resampler(speech_array).squeeze().numpy()
      7     return batch

/opt/conda/lib/python3.8/site-packages/torchaudio/backend/sox_io_backend.py in load(filepath, frame_offset, num_frames, normalize, channels_first, format)
    150                 filepath, frame_offset, num_frames, normalize, channels_first, format)
    151         filepath = os.fspath(filepath)
--> 152     return torch.ops.torchaudio.sox_io_load_audio_file(
    153         filepath, frame_offset, num_frames, normalize, channels_first, format)
    154 

RuntimeError: Error loading audio file: failed to open file common_voice_it_17415776.mp3  ```


## Environment info

- `datasets` version: 1.18.4
- Platform: Linux-5.4.0-x86_64-with-glibc2.10
- Python version: 3.8.5
- PyArrow version: 7.0.0
{'url': 'https://api.github.com/repos/huggingface/datasets/issues/3909/reactions', 'total_count': 0, '+1': 0, '-1': 0, 'laugh': 0, 'hooray': 0, 'confused': 0, 'heart': 0, 'rocket': 0, 'eyes': 0}