The invention relates to signal processing and, more particularly, to a mixed-mode very-large-scale integration (VLSI) architecture and methods for real-time classification of acoustic transients.
Acoustic transients, short, impulsive bursts of acoustic energy that last between 10 and 100 ms, are a rich source of information in the natural world, and the ability to process them in real time provides a competitive advantage to species. As a result, humans, like other animals, have evolved the ability to quickly and economically process acoustic transients.
In the digital world of algorithms and computers, analogous evolutionary forces have caused engineers to develop powerful digital signal processing (DSP) algorithms for classification of acoustic signals on fast DSP engines. Using modern signal processing techniques to recognize acoustic transients in real time is straightforward on modern processors. The challenge of extracting information from signals has been met by powerful mathematical techniques such as wavelet analysis and hidden Markov models. The need for real-time performance has been met by fast and powerful central processing units (CPUs) and special-purpose DSP chips.
Despite the above, a closer look at the DSP solutions reveals that the burden of real-time processing is borne by increasingly powerful digital processors. The price for success is measured in terms of power dissipation and complexity. Power dissipation scales linearly with the processor""s clock rate. Thus, all else being equal, a 100-MHz processor dissipates 1000 times more power than a 100-kHz processor. Each bump up in clock rate requires increasingly heroic methods to control power dissipation.
Complexity can be measured by the number of cycles required to perform a calculation and by the surface area of the chip. Increasingly complex algorithms create pressure to increase the complexity of the processor and thus the area of a chip. The problem of greater chip area can be overcome by scaling down the feature size of the fabrication process, but scaling also has physical limits. Moreover, as the feature size scales down, the fabrication process itself becomes more difficult and exacting.
All of this contrasts sharply with nature""s solution. The characteristics and advantages of nature""s acoustic processing algorithms are well documented. Natural systems process acoustic information in real time, with precision and reliability, while dissipating minuscule amounts of energy. Nature accomplishes this with slow and unreliable devices, i.e., neurons. Biological hardware has no clock but typical time scales are measured in fractions of milliseconds. In effect, biological hardware runs at a 1- to 10-kHz clock rate.
If it were possible to engineer acoustic processors with biological levels of performance and power requirements, a large number of new applications would become feasible. Intelligence based on acoustic pattern recognition could be built into appliances, telephones, and credit cards. Cellular phones could take spoken commands. Smart credit cards could recognize not only passwords, but also the speaker. Digital watches and calculators that run for years on button cells could understand a small vocabulary of spoken words. Self-diagnosing machines could recognize acoustic transients caused by state changes and wear.
Motivated by the observation that biological systems perform very sophisticated tasks while making low demands on power consumption and component precision, artificial devices can be developed that perform as competently as biological systems while requiring minimal resources. The long-term goal is to build pattern recognition engines whose performance characteristics rival those of biological systems. To be more specific, acoustic processing engines with the following characteristics can be built:
Real-time operation, so that typical transients are recognized in about 100 ms or less.
High correct classification rates (near 95%) on hundreds of transient classes while achieving low false alarm rates.
Implementation of such engines with the highly mismatched metal-oxide-silicon (MOS) transistors that are typical in modern analog VLSI fabrication processes (feature size  less than 1.2 xcexcm).
Power dissipation on the order of a milliwatt or less. This requires subthreshold current-mode circuits. Currents in such circuits are in the 0.1- to 10-nA range, while voltage swings are in the 100-mV range. Clock rates will be tens of kilohertz or less.
One solution to the above would be a practical architecture for performing real-time recognition of acoustic transients by means of a correlation-based algorithm. In other words, the algorithm would perform pattern recognition by correlating an incoming signal with a stored template. However, correlation-based algorithms are generally believed to be so computationally intensive that they cannot be used for real-time applications except in conjunction with fast DSP chips.
Traditionally, correlation in analog VLSI poses two fundamental implementation challenges: first, the problem of template storage; second, the problem of accurate analog multiplication. Both problems can be solved by building sufficiently complex circuits. For example, analog values can be stored by sample-and-hold circuits or by storing digital values and converting them into analog values via digital-to-analog converters. These solutions are generally inferior to digital correlation algorithms because they lead to analog processors that are large compared with their digital counterparts.
Another, more compact solution to the template storage problem is to employ the recently developed floating gate devices. Presently, such devices can store precise analog values for years without significant degradation. Moreover, this approach can result in very compact devices. Unfortunately, programming floating gate devices is not particularly easy. It is relatively slow and requires high voltage. Furthermore, the floating gate degrades each time it is reprogrammed. The fabrication of high-quality floating gates also requires advanced fabrication processes that may not be compatible with circuits for other kinds of on-chip processing.
Finally, even if the analog storage problem could be solved effectively, the problem of building accurate analog-analog multipliers remains. High-quality analog multipliers are notoriously difficult to build. Effective solutions require considerable area on the chip.
One solution to the above problems is to sidestep them completely and to develop an algorithm and architecture that require neither analog storage nor analog multiplication. One instance of this approach is to binarize the input and then to correlate it with a binary template. Thus, the correlations can be performed by simple xe2x80x9cXORxe2x80x9d gates. This approach is compact and fast. Thus, there remains a need for analog VLSI devices for real-time classification of acoustic transients that provide a high level of classification and are small and relatively simple to build.
The above problems are solved by the invention, a hybrid approach that replaces analog-analog multiplication with analog-binary multiplication. In mixed-mode hardware this operation corresponds to simple binary multiplexing. The algorithm and architecture of the invention are expected to perform a correlation calculation on a special-purpose parallel analog VLSI chip, using a slow clock (about 10 kHz) and with just a few milliwatts of power dissipation.
Many time-based classification systems compute the correlation of an incoming discrete-time signal or signals with a predetermined template. While for speech and other complex long-term signals it is necessary to perform dynamic time warping (DTW) or similar weighting of the incoming signal, for transients, a simple correlation in the time-frequency domain yields accurate classification results.
A general form of the simple correlation is                                           c            z                    ⁡                      [            t            ]                          =                              ∑                          m              =              1                        M                    ⁢                      xe2x80x83                    ⁢                                    ∑                              n                =                1                            N                        ⁢                          xe2x80x83                        ⁢                                          x                ⁡                                  [                                                            t                      -                      n                                        ,                    m                                    ]                                            ⁢                                                p                  z                                ⁡                                  [                                      n                    ,                    m                                    ]                                                                                        (        1        )            
where M is the number of frequency channels of the input, N is the maximum number of time bins in the window, x is the array of input signals split into frequency bands, pz is the matrix of template pattern values for pattern z, and t is the current time. This formula produces a running correlation cz(t) of the input array with the template z.
For large M and N, this algorithm can be expensive to execute on a DSP in terms of speed and power requirements. However, the approach of the invention lends itself elegantly to low-power parallel analog computation in the form of MOS transistor circuits operating primarily in the subthreshold mode.
The algorithm of the invention is a set of modifications to the algorithm of Equation (1). First, the input is normalized, which is essential for the steps which follow. Next, the input and template are transformed into a zero-mean representation by taking the difference of successive samples, the difference between neighboring channels, or some combination thereof. In this form, the template values can be made binary without significantly increasing classification error rates as determined experimentally. Finally, the differencing operation is moved to the output, yielding the simplest possible form of the architecture.
Assume the input y to the system is a sampled, continuous-valued acoustic signal, split into M frequency bands. The (rectified) energy envelope for each band, denoted x, is computed and then these system inputs are normalize by the function                                           x            ⁡                          [                              t                ,                m                            ]                                =                                    y              ⁡                              [                                  t                  ,                  m                                ]                                                    θ              +                                                ∑                                      k                    =                    1                                    M                                ⁢                                  xe2x80x83                                ⁢                                  y                  ⁡                                      [                                          t                      ,                      k                                        ]                                                                                      ,                            (        2        )            
where xcex8 is a threshold value included to suppress noise during quiet intervals in the input. The normalized input representation is essential to significantly simplify the pattern classifier algorithm for analog inputs, greatly reducing the size and complexity of the hardware implementation but not degrading the classification result.
Several steps are necessary to make the computation less expensive. First, zero-mean transformed input is compared directly to the zero-mean transformed template. This has the effect of subtracting a constant from the result but otherwise does not affect the computation. The template is replaced by the sign of the transformed template. Thus,                                           c            z                    ⁡                      [            t            ]                          =                              ∑                          m              =              1                        M                    ⁢                      xe2x80x83                    ⁢                                    ∑                              n                =                1                            N                        ⁢                          xe2x80x83                        ⁢                                                            x                  xe2x80x2                                ⁡                                  (                                                            t                      -                      n                                        ,                    m                                    )                                            ⁢                                                p                  z                  xe2x80x2                                ⁡                                  [                                      n                    ,                    m                                    ]                                                                                        (        3        )            
where
xxe2x80x2(txe2x88x92n,m)=x(txe2x88x92n,m)xe2x88x92x(txe2x88x92n,mxe2x88x921) for channel differencing
and
pxe2x80x2z[n,m]=sign(pz[n,m]xe2x88x92pz[n,mxe2x88x921]).xe2x80x83xe2x80x83(4)
It has been shown that binarization of the template has a negligible effect on classification performance. Note that this result does not hold if the input and template are not transformed into a zero-mean representation.
With this major simplification of the algorithm, the multiplication can be reduced to a multiplexing function. The required normalization of the input allows a further simplification that does not affect system performance, in which the template values are made binary [0,1] rather than binary [xe2x88x921,1].
If time differencing is used for the zero-mean representation, another simplification is possible, noting that the time difference commutes with the summation, so that Equation (1) can be written                                           c            z                    ⁡                      [            t            ]                          =                                            ∑                              m                =                1                            M                        ⁢                          xe2x80x83                        ⁢                                          ∑                                  n                  =                  1                                N                            ⁢                              xe2x80x83                            ⁢                                                x                  ⁡                                      [                                                                  t                        -                        n                                            ,                      m                                        ]                                                  ⁢                                                      p                    z                    xe2x80x2                                    ⁡                                      [                                          n                      ,                      m                                        ]                                                                                -                                    ∑                              m                =                1                            M                        ⁢                          xe2x80x83                        ⁢                                          ∑                                  n                  =                  1                                N                            ⁢                              xe2x80x83                            ⁢                                                x                  ⁡                                      [                                                                                            (                                                      t                            -                            1                                                    )                                                -                        n                                            ,                      m                                        ]                                                  ⁢                                                                            p                      z                      xe2x80x2                                        ⁡                                          [                                              n                        ,                        m                                            ]                                                        .                                                                                        (        5        )            
If we let                                                         c              z              xe2x80x2                        ⁡                          [              t              ]                                =                                    ∑                              m                =                1                            M                        ⁢                          xe2x80x83                        ⁢                                          ∑                                  n                  =                  1                                N                            ⁢                              xe2x80x83                            ⁢                                                x                  ⁡                                      [                                                                  t                        -                        n                                            ,                      m                                        ]                                                  ⁢                                                      p                    z                    xe2x80x2                                    ⁡                                      [                                          n                      ,                      m                                        ]                                                                                      ,                            (        6        )            
then
cz[t]=cxe2x80x2z[t]xe2x88x92cxe2x80x2z[txe2x88x921].xe2x80x83xe2x80x83(7)
In this forms the time difference can be computed at the output of the correlator rather than at the inputs, yielding three advantages:
1. Only one difference rather than M differences need to be computed.
2. Architecturally, the algorithm is less affected by device mismatch when computing the output based on the difference of successive outputs rather than the absolute value of the output.
3. Most importantly, since the inputs x are rectified, the product xpxe2x80x2 (when pxe2x80x2 is binary [0,1] is always positive and equals either x or zero, which allows us to conveniently implement the entire convolution as an array of simple on/off current switches carrying current in one direction only.
With the invention, a high level of classification performance on real-world data can be achieved with no measurable loss of performance in comparison with a traditional, computationally intensive correlation algorithm. Moreover, the mixed-mode architecture of the invention is not significantly harder to implement than the binary-binary correlation. In the acoustic case, the approach requires neither digital-to-analog nor analog-to-digital converters nor the storage of analog values. The algorithm leads to a correlator whose computing surface bears a strong resemblance to conventional dynamic random access memory (RAM).