codingutf8 2023 the huggingface inc team and the librosa torchaudio s licensed under the apache license version 2 0 the license you may not use this file except in compliance with the license you may obtain a copy of the license at http www apache orglicenseslicense2 0 unless required by applicable law or agreed to in writing software distributed under the license is distributed on an as is basis without warranties or conditions of any kind either express or implied see the license for the specific language governing permissions and limitations under the license audio processing functions to extract features from audio waveforms this code is pure numpy to support all frameworks and remove unnecessary dependencies convert frequency from hertz to mels args freq float or np ndarray the frequency or multiple frequencies in hertz hz melscale str optional defaults to htk the mel frequency scale to use htk kaldi or slaney returns float or np ndarray the frequencies on the mel scale convert frequency from mels to hertz args mels float or np ndarray the frequency or multiple frequencies in mels melscale str optional htk the mel frequency scale to use htk kaldi or slaney returns float or np ndarray the frequencies in hertz creates a triangular filter bank adapted from torchaudio and librosa args fftfreqs np ndarray of shape numfrequencybins discrete frequencies of the fft bins in hz filterfreqs np ndarray of shape nummelfilters center frequencies of the triangular filters to create in hz returns np ndarray of shape numfrequencybins nummelfilters creates a frequency bin conversion matrix used to obtain a mel spectrogram this is called a mel filter bank and various implementation exist which differ in the number of filters the shape of the filters the way the filters are spaced the bandwidth of the filters and the manner in which the spectrum is warped the goal of these features is to approximate the nonlinear human perception of the variation in pitch with respect to the frequency different banks of mel filters were introduced in the literature the following variations are supported mfcc fb20 introduced in 1980 by davis and mermelstein it assumes a sampling frequency of 10 khz and a speech bandwidth of 0 4600 hz mfcc fb24 htk from the cambridge hmm toolkit htk 1995 uses a filter bank of 24 filters for a speech bandwidth of 0 8000 hz this assumes sampling rate 16 khz mfcc fb40 from the auditory toolbox for matlab written by slaney in 1998 assumes a sampling rate of 16 khz and speech bandwidth of 133 6854 hz this version also includes area normalization hfcce fb29 human factor cepstral coefficients of skowronski and harris 2004 assumes a sampling rate of 12 5 khz and speech bandwidth of 0 6250 hz this code is adapted from torchaudio and librosa note that the default parameters of torchaudio s melscalefbanks implement the htk filters while librosa uses the slaney implementation args numfrequencybins int number of frequencies used to compute the spectrogram should be the same as in stft nummelfilters int number of mel filters to generate minfrequency float lowest frequency of interest in hz maxfrequency float highest frequency of interest in hz this should not exceed samplingrate 2 samplingrate int sample rate of the audio waveform norm str optional if slaney divide the triangular mel weights by the width of the mel band area normalization melscale str optional defaults to htk the mel frequency scale to use htk kaldi or slaney triangularizeinmelspace bool optional defaults to false if this option is enabled the triangular filter is applied in mel space rather than frequency space this should be set to true in order to get the same results as torchaudio when computing mel filters returns np ndarray of shape numfrequencybins nummelfilters triangular filter bank matrix this is a projection matrix to go from a spectrogram to a mel spectrogram center points of the triangular mel filters frequencies of fft bins in hz but filters triangularized in mel space frequencies of fft bins in hz slaneystyle mel is scaled to be approx constant energy per channel finds the best fft input size for a given windowlength this function takes a given window length and if not already a power of two rounds it up to the next power or two the fft algorithm works fastest when the length of the input is a power of two which may be larger than the size of the window or analysis frame for example if the window is 400 samples using an fft input size of 512 samples is more optimal than an fft size of 400 samples using a larger fft size does not affect the detected frequencies it simply gives a higher frequency resolution i e the frequency bins are smaller returns an array containing the specified window this window is intended to be used with stft the following window types are supported boxcar a rectangular window hamming the hamming window hann the hann window povey the povey window args windowlength int the length of the window in samples name str optional defaults to hann the name of the window function periodic bool optional defaults to true whether the window is periodic or symmetric framelength int optional the length of the analysis frames in samples provide a value for framelength if the window is smaller than the frame length so that it will be zeropadded center bool optional defaults to true whether to center the window inside the fft buffer only used when framelength is provided returns np ndarray of shape windowlength or framelength containing the window todo this method does not support batching yet as we are mainly focused on inference calculates a spectrogram over one waveform using the shorttime fourier transform this function can create the following kinds of spectrograms amplitude spectrogram power 1 0 power spectrogram power 2 0 complexvalued spectrogram power none log spectrogram use logmel argument mel spectrogram provide melfilters logmel spectrogram provide melfilters and logmel how this works 1 the input waveform is split into frames of size framelength that are partially overlapping by framelength hoplength samples 2 each frame is multiplied by the window and placed into a buffer of size fftlength 3 the dft is taken of each windowed frame 4 the results are stacked into a spectrogram we make a distinction between the following blocks of sample data each of which may have a different lengths the analysis frame this is the size of the time slices that the input waveform is split into the window each analysis frame is multiplied by the window to avoid spectral leakage the fft input buffer the length of this determines how many frequency bins are in the spectrogram in this implementation the window is assumed to be zeropadded to have the same size as the analysis frame a padded window can be obtained from windowfunction the fft input buffer may be larger than the analysis frame typically the next power of two note this function is not optimized for speed yet it should be mostly compatible with librosa stft and torchaudio functional transforms spectrogram although it is more flexible due to the different ways spectrograms can be constructed args waveform np ndarray of shape length the input waveform this must be a single realvalued mono waveform window np ndarray of shape framelength the windowing function to apply including zeropadding if necessary the actual window length may be shorter than framelength but we re assuming the array has already been zeropadded framelength int the length of the analysis frames in samples with librosa this is always equal to fftlength but we also allow smaller sizes hoplength int the stride between successive analysis frames in samples fftlength int optional the size of the fft buffer in samples this determines how many frequency bins the spectrogram will have for optimal speed this should be a power of two if none uses framelength power float optional defaults to 1 0 if 1 0 returns the amplitude spectrogram if 2 0 returns the power spectrogram if none returns complex numbers center bool optional defaults to true whether to pad the waveform so that frame t is centered around time t hoplength if false frame t will start at time t hoplength padmode str optional defaults to reflect padding mode used when center is true possible values are constant pad with zeros edge pad with edge values reflect pads with mirrored values onesided bool optional defaults to true if true only computes the positive frequencies and returns a spectrogram containing fftlength 2 1 frequency bins if false also computes the negative frequencies and returns fftlength frequency bins preemphasis float optional coefficient for a lowpass filter that applies preemphasis before the dft melfilters np ndarray of shape numfreqbins nummelfilters optional the mel filter bank if supplied applies a this filter bank to create a mel spectrogram melfloor float optional defaults to 1e10 minimum value of mel frequency banks logmel str optional how to convert the spectrogram to log scale possible options are none don t convert log take the natural logarithm log10 take the base10 logarithm db convert to decibels can only be used when power is not none reference float optional defaults to 1 0 sets the input spectrogram value that corresponds to 0 db for example use np maxspectrogram to set the loudest part to 0 db must be greater than zero minvalue float optional defaults to 1e10 the spectrogram will be clipped to this minimum value before conversion to decibels to avoid taking log0 for a power spectrogram the default of 1e10 corresponds to a minimum of 100 db for an amplitude spectrogram the value 1e5 corresponds to 100 db must be greater than zero dbrange float optional sets the maximum dynamic range in decibels for example if dbrange 80 the difference between the peak value and the smallest value will never be more than 80 db must be greater than zero removedcoffset bool optional subtract mean from waveform on each frame applied before preemphasis this should be set to true in order to get the same results as torchaudio compliance kaldi fbank when computing mel filters dtype np dtype optional defaults to np float32 data type of the spectrogram tensor if power is none this argument is ignored and the dtype will be np complex64 returns nd array containing a spectrogram of shape numfrequencybins length for a regular spectrogram or shape nummelfilters length for a mel spectrogram center pad the waveform promote to float64 since np fft uses float64 internally split waveform into frames of framelength size rfft is faster than fft note is much faster than np power converts a power spectrogram to the decibel scale this computes 10 log10spectrogram reference using basic logarithm properties for numerical stability the motivation behind applying the log function on the mel spectrogram is that humans do not hear loudness on a linear scale generally to double the perceived volume of a sound we need to put 8 times as much energy into it this means that large variations in energy may not sound all that different if the sound is loud to begin with this compression operation makes the mel spectrogram features match more closely what humans actually hear based on the implementation of librosa powertodb args spectrogram np ndarray the input power mel spectrogram note that a power spectrogram has the amplitudes squared reference float optional defaults to 1 0 sets the input spectrogram value that corresponds to 0 db for example use np maxspectrogram to set the loudest part to 0 db must be greater than zero minvalue float optional defaults to 1e10 the spectrogram will be clipped to this minimum value before conversion to decibels to avoid taking log0 the default of 1e10 corresponds to a minimum of 100 db must be greater than zero dbrange float optional sets the maximum dynamic range in decibels for example if dbrange 80 the difference between the peak value and the smallest value will never be more than 80 db must be greater than zero returns np ndarray the spectrogram in decibels converts an amplitude spectrogram to the decibel scale this computes 20 log10spectrogram reference using basic logarithm properties for numerical stability the motivation behind applying the log function on the mel spectrogram is that humans do not hear loudness on a linear scale generally to double the perceived volume of a sound we need to put 8 times as much energy into it this means that large variations in energy may not sound all that different if the sound is loud to begin with this compression operation makes the mel spectrogram features match more closely what humans actually hear args spectrogram np ndarray the input amplitude mel spectrogram reference float optional defaults to 1 0 sets the input spectrogram value that corresponds to 0 db for example use np maxspectrogram to set the loudest part to 0 db must be greater than zero minvalue float optional defaults to 1e5 the spectrogram will be clipped to this minimum value before conversion to decibels to avoid taking log0 the default of 1e5 corresponds to a minimum of 100 db must be greater than zero dbrange float optional sets the maximum dynamic range in decibels for example if dbrange 80 the difference between the peak value and the smallest value will never be more than 80 db must be greater than zero returns np ndarray the spectrogram in decibels deprecated functions below this line in order to compute the short time fourier transform the waveform needs to be split in overlapping windowed segments called frames the window length windowlength defines how much of the signal is contained in each frame while the hop length defines the step between the beginning of each new frame args waveform np array of shape samplelength the raw waveform which will be split into smaller chunks hoplength int optional defaults to 160 step between each window of the waveform fftwindowsize int optional defaults to 400 defines the size of the window center bool defaults to true whether or not to center each frame around the middle of the frame centering is done by reflecting the waveform on the left and on the right return framedwaveform np array of shape waveform shape hoplength fftwindowsize the framed waveforms that can be fed to np fft calculates the complex shorttime fourier transform stft of the given framed signal should give the same results as torch stft args frames np array of dimension numframes fftwindowsize a framed audio signal obtained using audioutils framwav windowingfunction np array of dimension nbfrequencybins nbmelfilters a array reprensenting the function that will be used to reduces the amplitude of the discontinuities at the boundaries of each frame when computing the stft each frame will be multiplied by the windowingfunction for more information on the discontinuities called spectral leakage refer to this tutorialhttps download ni comevaluationpxiunderstanding20ffts20and20windowing pdf fftwindowsize int optional size of the window om which the fourier transform is applied this controls the frequency resolution of the spectrogram 400 means that the fourrier transform is computed on windows of 400 samples the number of frequency bins nbfrequencybins used to divide the window into equal strips is equal to 1fftwindowsize2 an increase of the fftwindowsize slows the calculus time proportionnally example python from transformers audioutils import stft framwave import numpy as np audio np random rand50 fftwindowsize 10 hoplength 2 framedaudio framwaveaudio hoplength fftwindowsize spectrogram stftframedaudio np hanningfftwindowsize 1 returns spectrogram np ndarray a spectrogram of shape numframes nbfrequencybins obtained using the stft algorithm number of fft bins to store coding utf 8 2023 the huggingface inc team and the librosa torchaudio s licensed under the apache license version 2 0 the license you may not use this file except in compliance with the license you may obtain a copy of the license at http www apache org licenses license 2 0 unless required by applicable law or agreed to in writing software distributed under the license is distributed on an as is basis without warranties or conditions of any kind either express or implied see the license for the specific language governing permissions and limitations under the license audio processing functions to extract features from audio waveforms this code is pure numpy to support all frameworks and remove unnecessary dependencies convert frequency from hertz to mels args freq float or np ndarray the frequency or multiple frequencies in hertz hz mel_scale str optional defaults to htk the mel frequency scale to use htk kaldi or slaney returns float or np ndarray the frequencies on the mel scale convert frequency from mels to hertz args mels float or np ndarray the frequency or multiple frequencies in mels mel_scale str optional htk the mel frequency scale to use htk kaldi or slaney returns float or np ndarray the frequencies in hertz creates a triangular filter bank adapted from torchaudio and librosa args fft_freqs np ndarray of shape num_frequency_bins discrete frequencies of the fft bins in hz filter_freqs np ndarray of shape num_mel_filters center frequencies of the triangular filters to create in hz returns np ndarray of shape num_frequency_bins num_mel_filters creates a frequency bin conversion matrix used to obtain a mel spectrogram this is called a mel filter bank and various implementation exist which differ in the number of filters the shape of the filters the way the filters are spaced the bandwidth of the filters and the manner in which the spectrum is warped the goal of these features is to approximate the non linear human perception of the variation in pitch with respect to the frequency different banks of mel filters were introduced in the literature the following variations are supported mfcc fb 20 introduced in 1980 by davis and mermelstein it assumes a sampling frequency of 10 khz and a speech bandwidth of 0 4600 hz mfcc fb 24 htk from the cambridge hmm toolkit htk 1995 uses a filter bank of 24 filters for a speech bandwidth of 0 8000 hz this assumes sampling rate 16 khz mfcc fb 40 from the auditory toolbox for matlab written by slaney in 1998 assumes a sampling rate of 16 khz and speech bandwidth of 133 6854 hz this version also includes area normalization hfcc e fb 29 human factor cepstral coefficients of skowronski and harris 2004 assumes a sampling rate of 12 5 khz and speech bandwidth of 0 6250 hz this code is adapted from torchaudio and librosa note that the default parameters of torchaudio s melscale_fbanks implement the htk filters while librosa uses the slaney implementation args num_frequency_bins int number of frequencies used to compute the spectrogram should be the same as in stft num_mel_filters int number of mel filters to generate min_frequency float lowest frequency of interest in hz max_frequency float highest frequency of interest in hz this should not exceed sampling_rate 2 sampling_rate int sample rate of the audio waveform norm str optional if slaney divide the triangular mel weights by the width of the mel band area normalization mel_scale str optional defaults to htk the mel frequency scale to use htk kaldi or slaney triangularize_in_mel_space bool optional defaults to false if this option is enabled the triangular filter is applied in mel space rather than frequency space this should be set to true in order to get the same results as torchaudio when computing mel filters returns np ndarray of shape num_frequency_bins num_mel_filters triangular filter bank matrix this is a projection matrix to go from a spectrogram to a mel spectrogram center points of the triangular mel filters frequencies of fft bins in hz but filters triangularized in mel space frequencies of fft bins in hz slaney style mel is scaled to be approx constant energy per channel finds the best fft input size for a given window_length this function takes a given window length and if not already a power of two rounds it up to the next power or two the fft algorithm works fastest when the length of the input is a power of two which may be larger than the size of the window or analysis frame for example if the window is 400 samples using an fft input size of 512 samples is more optimal than an fft size of 400 samples using a larger fft size does not affect the detected frequencies it simply gives a higher frequency resolution i e the frequency bins are smaller returns an array containing the specified window this window is intended to be used with stft the following window types are supported boxcar a rectangular window hamming the hamming window hann the hann window povey the povey window args window_length int the length of the window in samples name str optional defaults to hann the name of the window function periodic bool optional defaults to true whether the window is periodic or symmetric frame_length int optional the length of the analysis frames in samples provide a value for frame_length if the window is smaller than the frame length so that it will be zero padded center bool optional defaults to true whether to center the window inside the fft buffer only used when frame_length is provided returns np ndarray of shape window_length or frame_length containing the window todo this method does not support batching yet as we are mainly focused on inference calculates a spectrogram over one waveform using the short time fourier transform this function can create the following kinds of spectrograms amplitude spectrogram power 1 0 power spectrogram power 2 0 complex valued spectrogram power none log spectrogram use log_mel argument mel spectrogram provide mel_filters log mel spectrogram provide mel_filters and log_mel how this works 1 the input waveform is split into frames of size frame_length that are partially overlapping by frame_length hop_length samples 2 each frame is multiplied by the window and placed into a buffer of size fft_length 3 the dft is taken of each windowed frame 4 the results are stacked into a spectrogram we make a distinction between the following blocks of sample data each of which may have a different lengths the analysis frame this is the size of the time slices that the input waveform is split into the window each analysis frame is multiplied by the window to avoid spectral leakage the fft input buffer the length of this determines how many frequency bins are in the spectrogram in this implementation the window is assumed to be zero padded to have the same size as the analysis frame a padded window can be obtained from window_function the fft input buffer may be larger than the analysis frame typically the next power of two note this function is not optimized for speed yet it should be mostly compatible with librosa stft and torchaudio functional transforms spectrogram although it is more flexible due to the different ways spectrograms can be constructed args waveform np ndarray of shape length the input waveform this must be a single real valued mono waveform window np ndarray of shape frame_length the windowing function to apply including zero padding if necessary the actual window length may be shorter than frame_length but we re assuming the array has already been zero padded frame_length int the length of the analysis frames in samples with librosa this is always equal to fft_length but we also allow smaller sizes hop_length int the stride between successive analysis frames in samples fft_length int optional the size of the fft buffer in samples this determines how many frequency bins the spectrogram will have for optimal speed this should be a power of two if none uses frame_length power float optional defaults to 1 0 if 1 0 returns the amplitude spectrogram if 2 0 returns the power spectrogram if none returns complex numbers center bool optional defaults to true whether to pad the waveform so that frame t is centered around time t hop_length if false frame t will start at time t hop_length pad_mode str optional defaults to reflect padding mode used when center is true possible values are constant pad with zeros edge pad with edge values reflect pads with mirrored values onesided bool optional defaults to true if true only computes the positive frequencies and returns a spectrogram containing fft_length 2 1 frequency bins if false also computes the negative frequencies and returns fft_length frequency bins preemphasis float optional coefficient for a low pass filter that applies pre emphasis before the dft mel_filters np ndarray of shape num_freq_bins num_mel_filters optional the mel filter bank if supplied applies a this filter bank to create a mel spectrogram mel_floor float optional defaults to 1e 10 minimum value of mel frequency banks log_mel str optional how to convert the spectrogram to log scale possible options are none don t convert log take the natural logarithm log10 take the base 10 logarithm db convert to decibels can only be used when power is not none reference float optional defaults to 1 0 sets the input spectrogram value that corresponds to 0 db for example use np max spectrogram to set the loudest part to 0 db must be greater than zero min_value float optional defaults to 1e 10 the spectrogram will be clipped to this minimum value before conversion to decibels to avoid taking log 0 for a power spectrogram the default of 1e 10 corresponds to a minimum of 100 db for an amplitude spectrogram the value 1e 5 corresponds to 100 db must be greater than zero db_range float optional sets the maximum dynamic range in decibels for example if db_range 80 the difference between the peak value and the smallest value will never be more than 80 db must be greater than zero remove_dc_offset bool optional subtract mean from waveform on each frame applied before pre emphasis this should be set to true in order to get the same results as torchaudio compliance kaldi fbank when computing mel filters dtype np dtype optional defaults to np float32 data type of the spectrogram tensor if power is none this argument is ignored and the dtype will be np complex64 returns nd array containing a spectrogram of shape num_frequency_bins length for a regular spectrogram or shape num_mel_filters length for a mel spectrogram center pad the waveform promote to float64 since np fft uses float64 internally split waveform into frames of frame_length size rfft is faster than fft note is much faster than np power converts a power spectrogram to the decibel scale this computes 10 log10 spectrogram reference using basic logarithm properties for numerical stability the motivation behind applying the log function on the mel spectrogram is that humans do not hear loudness on a linear scale generally to double the perceived volume of a sound we need to put 8 times as much energy into it this means that large variations in energy may not sound all that different if the sound is loud to begin with this compression operation makes the mel spectrogram features match more closely what humans actually hear based on the implementation of librosa power_to_db args spectrogram np ndarray the input power mel spectrogram note that a power spectrogram has the amplitudes squared reference float optional defaults to 1 0 sets the input spectrogram value that corresponds to 0 db for example use np max spectrogram to set the loudest part to 0 db must be greater than zero min_value float optional defaults to 1e 10 the spectrogram will be clipped to this minimum value before conversion to decibels to avoid taking log 0 the default of 1e 10 corresponds to a minimum of 100 db must be greater than zero db_range float optional sets the maximum dynamic range in decibels for example if db_range 80 the difference between the peak value and the smallest value will never be more than 80 db must be greater than zero returns np ndarray the spectrogram in decibels converts an amplitude spectrogram to the decibel scale this computes 20 log10 spectrogram reference using basic logarithm properties for numerical stability the motivation behind applying the log function on the mel spectrogram is that humans do not hear loudness on a linear scale generally to double the perceived volume of a sound we need to put 8 times as much energy into it this means that large variations in energy may not sound all that different if the sound is loud to begin with this compression operation makes the mel spectrogram features match more closely what humans actually hear args spectrogram np ndarray the input amplitude mel spectrogram reference float optional defaults to 1 0 sets the input spectrogram value that corresponds to 0 db for example use np max spectrogram to set the loudest part to 0 db must be greater than zero min_value float optional defaults to 1e 5 the spectrogram will be clipped to this minimum value before conversion to decibels to avoid taking log 0 the default of 1e 5 corresponds to a minimum of 100 db must be greater than zero db_range float optional sets the maximum dynamic range in decibels for example if db_range 80 the difference between the peak value and the smallest value will never be more than 80 db must be greater than zero returns np ndarray the spectrogram in decibels deprecated functions below this line in order to compute the short time fourier transform the waveform needs to be split in overlapping windowed segments called frames the window length window_length defines how much of the signal is contained in each frame while the hop length defines the step between the beginning of each new frame args waveform np array of shape sample_length the raw waveform which will be split into smaller chunks hop_length int optional defaults to 160 step between each window of the waveform fft_window_size int optional defaults to 400 defines the size of the window center bool defaults to true whether or not to center each frame around the middle of the frame centering is done by reflecting the waveform on the left and on the right return framed_waveform np array of shape waveform shape hop_length fft_window_size the framed waveforms that can be fed to np fft calculates the complex short time fourier transform stft of the given framed signal should give the same results as torch stft args frames np array of dimension num_frames fft_window_size a framed audio signal obtained using audio_utils fram_wav windowing_function np array of dimension nb_frequency_bins nb_mel_filters a array reprensenting the function that will be used to reduces the amplitude of the discontinuities at the boundaries of each frame when computing the stft each frame will be multiplied by the windowing_function for more information on the discontinuities called spectral leakage refer to this tutorial https download ni com evaluation pxi understanding 20ffts 20and 20windowing pdf fft_window_size int optional size of the window om which the fourier transform is applied this controls the frequency resolution of the spectrogram 400 means that the fourrier transform is computed on windows of 400 samples the number of frequency bins nb_frequency_bins used to divide the window into equal strips is equal to 1 fft_window_size 2 an increase of the fft_window_size slows the calculus time proportionnally example python from transformers audio_utils import stft fram_wave import numpy as np audio np random rand 50 fft_window_size 10 hop_length 2 framed_audio fram_wave audio hop_length fft_window_size spectrogram stft framed_audio np hanning fft_window_size 1 returns spectrogram np ndarray a spectrogram of shape num_frames nb_frequency_bins obtained using the stft algorithm number of fft bins to store
import warnings
from typing import Optional, Union

import numpy as np


def hertz_to_mel(freq: Union[float, np.ndarray], mel_scale: str = "htk") -> Union[float, np.ndarray]:
    

    if mel_scale not in ["slaney", "htk", "kaldi"]:
        raise ValueError('mel_scale should be one of "htk", "slaney" or "kaldi".')

    if mel_scale == "htk":
        return 2595.0 * np.log10(1.0 + (freq / 700.0))
    elif mel_scale == "kaldi":
        return 1127.0 * np.log(1.0 + (freq / 700.0))

    min_log_hertz = 1000.0
    min_log_mel = 15.0
    logstep = 27.0 / np.log(6.4)
    mels = 3.0 * freq / 200.0

    if isinstance(freq, np.ndarray):
        log_region = freq >= min_log_hertz
        mels[log_region] = min_log_mel + np.log(freq[log_region] / min_log_hertz) * logstep
    elif freq >= min_log_hertz:
        mels = min_log_mel + np.log(freq / min_log_hertz) * logstep

    return mels


def mel_to_hertz(mels: Union[float, np.ndarray], mel_scale: str = "htk") -> Union[float, np.ndarray]:
    

    if mel_scale not in ["slaney", "htk", "kaldi"]:
        raise ValueError('mel_scale should be one of "htk", "slaney" or "kaldi".')

    if mel_scale == "htk":
        return 700.0 * (np.power(10, mels / 2595.0) - 1.0)
    elif mel_scale == "kaldi":
        return 700.0 * (np.exp(mels / 1127.0) - 1.0)

    min_log_hertz = 1000.0
    min_log_mel = 15.0
    logstep = np.log(6.4) / 27.0
    freq = 200.0 * mels / 3.0

    if isinstance(mels, np.ndarray):
        log_region = mels >= min_log_mel
        freq[log_region] = min_log_hertz * np.exp(logstep * (mels[log_region] - min_log_mel))
    elif mels >= min_log_mel:
        freq = min_log_hertz * np.exp(logstep * (mels - min_log_mel))

    return freq


def _create_triangular_filter_bank(fft_freqs: np.ndarray, filter_freqs: np.ndarray) -> np.ndarray:
    
    filter_diff = np.diff(filter_freqs)
    slopes = np.expand_dims(filter_freqs, 0) - np.expand_dims(fft_freqs, 1)
    down_slopes = -slopes[:, :-2] / filter_diff[:-1]
    up_slopes = slopes[:, 2:] / filter_diff[1:]
    return np.maximum(np.zeros(1), np.minimum(down_slopes, up_slopes))


def mel_filter_bank(
    num_frequency_bins: int,
    num_mel_filters: int,
    min_frequency: float,
    max_frequency: float,
    sampling_rate: int,
    norm: Optional[str] = None,
    mel_scale: str = "htk",
    triangularize_in_mel_space: bool = False,
) -> np.ndarray:
    
    if norm is not None and norm != "slaney":
        raise ValueError('norm must be one of None or "slaney"')

    
    mel_min = hertz_to_mel(min_frequency, mel_scale=mel_scale)
    mel_max = hertz_to_mel(max_frequency, mel_scale=mel_scale)
    mel_freqs = np.linspace(mel_min, mel_max, num_mel_filters + 2)
    filter_freqs = mel_to_hertz(mel_freqs, mel_scale=mel_scale)

    if triangularize_in_mel_space:
        
        fft_bin_width = sampling_rate / (num_frequency_bins * 2)
        fft_freqs = hertz_to_mel(fft_bin_width * np.arange(num_frequency_bins), mel_scale=mel_scale)
        filter_freqs = mel_freqs
    else:
        
        fft_freqs = np.linspace(0, sampling_rate // 2, num_frequency_bins)

    mel_filters = _create_triangular_filter_bank(fft_freqs, filter_freqs)

    if norm is not None and norm == "slaney":
        
        enorm = 2.0 / (filter_freqs[2 : num_mel_filters + 2] - filter_freqs[:num_mel_filters])
        mel_filters *= np.expand_dims(enorm, 0)

    if (mel_filters.max(axis=0) == 0.0).any():
        warnings.warn(
            "At least one mel filter has all zero values. "
            f"The value for `num_mel_filters` ({num_mel_filters}) may be set too high. "
            f"Or, the value for `num_frequency_bins` ({num_frequency_bins}) may be set too low."
        )

    return mel_filters


def optimal_fft_length(window_length: int) -> int:
    
    return 2 ** int(np.ceil(np.log2(window_length)))


def window_function(
    window_length: int,
    name: str = "hann",
    periodic: bool = True,
    frame_length: Optional[int] = None,
    center: bool = True,
) -> np.ndarray:
    
    length = window_length + 1 if periodic else window_length

    if name == "boxcar":
        window = np.ones(length)
    elif name in ["hamming", "hamming_window"]:
        window = np.hamming(length)
    elif name in ["hann", "hann_window"]:
        window = np.hanning(length)
    elif name in ["povey"]:
        window = np.power(np.hanning(length), 0.85)
    else:
        raise ValueError(f"Unknown window function '{name}'")

    if periodic:
        window = window[:-1]

    if frame_length is None:
        return window

    if window_length > frame_length:
        raise ValueError(
            f"Length of the window ({window_length}) may not be larger than frame_length ({frame_length})"
        )

    padded_window = np.zeros(frame_length)
    offset = (frame_length - window_length) // 2 if center else 0
    padded_window[offset : offset + window_length] = window
    return padded_window



def spectrogram(
    waveform: np.ndarray,
    window: np.ndarray,
    frame_length: int,
    hop_length: int,
    fft_length: Optional[int] = None,
    power: Optional[float] = 1.0,
    center: bool = True,
    pad_mode: str = "reflect",
    onesided: bool = True,
    preemphasis: Optional[float] = None,
    mel_filters: Optional[np.ndarray] = None,
    mel_floor: float = 1e-10,
    log_mel: Optional[str] = None,
    reference: float = 1.0,
    min_value: float = 1e-10,
    db_range: Optional[float] = None,
    remove_dc_offset: Optional[bool] = None,
    dtype: np.dtype = np.float32,
) -> np.ndarray:
    
    window_length = len(window)

    if fft_length is None:
        fft_length = frame_length

    if frame_length > fft_length:
        raise ValueError(f"frame_length ({frame_length}) may not be larger than fft_length ({fft_length})")

    if window_length != frame_length:
        raise ValueError(f"Length of the window ({window_length}) must equal frame_length ({frame_length})")

    if hop_length <= 0:
        raise ValueError("hop_length must be greater than zero")

    if waveform.ndim != 1:
        raise ValueError(f"Input waveform must have only one dimension, shape is {waveform.shape}")

    if np.iscomplexobj(waveform):
        raise ValueError("Complex-valued input waveforms are not currently supported")

    
    if center:
        padding = [(int(frame_length // 2), int(frame_length // 2))]
        waveform = np.pad(waveform, padding, mode=pad_mode)

    
    waveform = waveform.astype(np.float64)
    window = window.astype(np.float64)

    
    num_frames = int(1 + np.floor((waveform.size - frame_length) / hop_length))

    num_frequency_bins = (fft_length // 2) + 1 if onesided else fft_length
    spectrogram = np.empty((num_frames, num_frequency_bins), dtype=np.complex64)

    
    fft_func = np.fft.rfft if onesided else np.fft.fft
    buffer = np.zeros(fft_length)

    timestep = 0
    for frame_idx in range(num_frames):
        buffer[:frame_length] = waveform[timestep : timestep + frame_length]

        if remove_dc_offset:
            buffer[:frame_length] = buffer[:frame_length] - buffer[:frame_length].mean()

        if preemphasis is not None:
            buffer[1:frame_length] -= preemphasis * buffer[: frame_length - 1]
            buffer[0] *= 1 - preemphasis

        buffer[:frame_length] *= window

        spectrogram[frame_idx] = fft_func(buffer)
        timestep += hop_length

    
    if power is not None:
        spectrogram = np.abs(spectrogram, dtype=np.float64) ** power

    spectrogram = spectrogram.T

    if mel_filters is not None:
        spectrogram = np.maximum(mel_floor, np.dot(mel_filters.T, spectrogram))

    if power is not None and log_mel is not None:
        if log_mel == "log":
            spectrogram = np.log(spectrogram)
        elif log_mel == "log10":
            spectrogram = np.log10(spectrogram)
        elif log_mel == "dB":
            if power == 1.0:
                spectrogram = amplitude_to_db(spectrogram, reference, min_value, db_range)
            elif power == 2.0:
                spectrogram = power_to_db(spectrogram, reference, min_value, db_range)
            else:
                raise ValueError(f"Cannot use log_mel option '{log_mel}' with power {power}")
        else:
            raise ValueError(f"Unknown log_mel option: {log_mel}")

        spectrogram = np.asarray(spectrogram, dtype)

    return spectrogram


def power_to_db(
    spectrogram: np.ndarray,
    reference: float = 1.0,
    min_value: float = 1e-10,
    db_range: Optional[float] = None,
) -> np.ndarray:
    
    if reference <= 0.0:
        raise ValueError("reference must be greater than zero")
    if min_value <= 0.0:
        raise ValueError("min_value must be greater than zero")

    reference = max(min_value, reference)

    spectrogram = np.clip(spectrogram, a_min=min_value, a_max=None)
    spectrogram = 10.0 * (np.log10(spectrogram) - np.log10(reference))

    if db_range is not None:
        if db_range <= 0.0:
            raise ValueError("db_range must be greater than zero")
        spectrogram = np.clip(spectrogram, a_min=spectrogram.max() - db_range, a_max=None)

    return spectrogram


def amplitude_to_db(
    spectrogram: np.ndarray,
    reference: float = 1.0,
    min_value: float = 1e-5,
    db_range: Optional[float] = None,
) -> np.ndarray:
    
    if reference <= 0.0:
        raise ValueError("reference must be greater than zero")
    if min_value <= 0.0:
        raise ValueError("min_value must be greater than zero")

    reference = max(min_value, reference)

    spectrogram = np.clip(spectrogram, a_min=min_value, a_max=None)
    spectrogram = 20.0 * (np.log10(spectrogram) - np.log10(reference))

    if db_range is not None:
        if db_range <= 0.0:
            raise ValueError("db_range must be greater than zero")
        spectrogram = np.clip(spectrogram, a_min=spectrogram.max() - db_range, a_max=None)

    return spectrogram





def get_mel_filter_banks(
    nb_frequency_bins: int,
    nb_mel_filters: int,
    frequency_min: float,
    frequency_max: float,
    sample_rate: int,
    norm: Optional[str] = None,
    mel_scale: str = "htk",
) -> np.array:
    warnings.warn(
        "The function `get_mel_filter_banks` is deprecated and will be removed in version 4.31.0 of Transformers",
        FutureWarning,
    )
    return mel_filter_bank(
        num_frequency_bins=nb_frequency_bins,
        num_mel_filters=nb_mel_filters,
        min_frequency=frequency_min,
        max_frequency=frequency_max,
        sampling_rate=sample_rate,
        norm=norm,
        mel_scale=mel_scale,
    )


def fram_wave(waveform: np.array, hop_length: int = 160, fft_window_size: int = 400, center: bool = True):
    
    warnings.warn(
        "The function `fram_wave` is deprecated and will be removed in version 4.31.0 of Transformers",
        FutureWarning,
    )
    frames = []
    for i in range(0, waveform.shape[0] + 1, hop_length):
        if center:
            half_window = (fft_window_size - 1) // 2 + 1
            start = i - half_window if i > half_window else 0
            end = i + half_window if i < waveform.shape[0] - half_window else waveform.shape[0]
            frame = waveform[start:end]
            if start == 0:
                padd_width = (-i + half_window, 0)
                frame = np.pad(frame, pad_width=padd_width, mode="reflect")

            elif end == waveform.shape[0]:
                padd_width = (0, (i - waveform.shape[0] + half_window))
                frame = np.pad(frame, pad_width=padd_width, mode="reflect")

        else:
            frame = waveform[i : i + fft_window_size]
            frame_width = frame.shape[0]
            if frame_width < waveform.shape[0]:
                frame = np.lib.pad(
                    frame, pad_width=(0, fft_window_size - frame_width), mode="constant", constant_values=0
                )
        frames.append(frame)

    frames = np.stack(frames, 0)
    return frames


def stft(frames: np.array, windowing_function: np.array, fft_window_size: int = None):
    
    warnings.warn(
        "The function `stft` is deprecated and will be removed in version 4.31.0 of Transformers",
        FutureWarning,
    )
    frame_size = frames.shape[1]

    if fft_window_size is None:
        fft_window_size = frame_size

    if fft_window_size < frame_size:
        raise ValueError("FFT size must greater or equal the frame size")
    
    nb_frequency_bins = (fft_window_size >> 1) + 1

    spectrogram = np.empty((len(frames), nb_frequency_bins), dtype=np.complex64)
    fft_signal = np.zeros(fft_window_size)

    for f, frame in enumerate(frames):
        if windowing_function is not None:
            np.multiply(frame, windowing_function, out=fft_signal[:frame_size])
        else:
            fft_signal[:frame_size] = frame
        spectrogram[f] = np.fft.fft(fft_signal, axis=0)[:nb_frequency_bins]
    return spectrogram.T