BACKGROUND OF THE INVENTION

Field of the Invention

The present invention relates to audio recording and more specifically to transducer systems and methods for making Virtual Reality (“VR”) audio-visual recordings having ambient soundscapes with an aural perspective which is substantially constant and fixed in relation to a contemporaneous video recording.

Discussion of the Prior Art

Audiovisual recordings with multichannel (e.g., stereo) sound are very common and can be entertaining, but do not provide an immersive experience in which the audience member or viewer feels immersed in the recorded environment. The principal problem with prior art methods and systems for recording (e.g., stereo or binaural) audio was and remains that when listening to the recording on headphones or with earbuds, the sound appears to be trapped inside the listener's head.

Microphones are transducers which transform variations in sound pressure into an electrical signal with two dimensions: pitch and amplitude. These are the same two dimensions a listener hears with one ear. For humans, sensitivity to pitch covers a range of 10 octaves starting at a frequency of 20 Hz in the low bass and extending to 20,000 Hz in the upper harmonics and sensitivity to amplitude exceeds a range of 100,000 to 1. That's a range that begins with a quiet whisper and builds in intensity to the painful noise of a jackhammer. Humans have two ears connected to the brain. This combination enables the listener's sense of hearing to tell more about sounds than just the pitch and amplitude. The listener in an ambient sound field can locate sounds in three-dimensional space.

Sound Localization

Over one hundred years ago, Lord Rayleigh in his treatise “Duplex Theory of Sound Localization,” described the basic principles of how listeners hear sounds coming from different directions. There were two main principles (hence the word “duplex”). The first principle is time difference. When a sound originates from a source directly to the listener's right, it is heard first with the right ear and then, a fraction of a second later, with the left ear. The time difference is miniscule, about 600 millionths of a second, but the listener's mind can detect it. The scientific terminology for this time difference is: Interaural Time Difference (ITD).

The second principle is level difference. Because there's a head located between listeners' ears, the sound coming from the source on the listener's right will be louder in the right ear and softer when it arrives at the left ear. That's because the listener's head blocks the sound creating a level difference or shadow. This level difference is not so simple to understand as the time difference. The listener's head is almost spherically shaped. Its interaction with sound waves creates level differences that are frequency dependent and quite complex. The listener's mind is amazingly sensitive to these level differences. The scientific terminology is: Interaural Level Difference (ILD).

There is a vast body of scientific literature written during the last century analyzing the interaction of waves with rigid bodies. The applicant has studied these works to gain an understanding of how sound waves behave when they encounter a spherical object, specifically the listener's head. Sound waves create pressure zones as they impinge upon and pass around the head. These effects influence the listener's sense of direction for sounds, creating a sense of spaciousness and presence. Understanding the literature requires an understanding of the anatomy of the human ear, as illustrated inFIGS. 1A and 1B.FIG. 1Ais a diagram excerpt from Gray's Anatomy showing the anatomical features of the outer ear including the Pinna's Helix and Scapha.FIG. 2Bis a diagram excerpt showing the anatomical features of the outer (external) ear, the middle ear (including the ear canal or external auditory meatus) and the internal ear. Those terms will be part of the nomenclature of the method of the present invention, as described and illustrated below.

Current Stereo Techniques

Stereo sound recordings became widespread in the late 1950s, although their invention dates to the 1930s. In concept, stereo recording is supposed to capture sounds and reproduce them in a way that recreates the “live experience” as if the listener were there. The listener has two ears, so the theory behind stereo says that two channels of sound should be satisfactory. “Stereo” is usually defined as a method for recording sound with two channels using two microphones and reproducing the sound with two earphones or loudspeakers. Ideally, one microphone captures sounds originating from the left, directing them towards the left ear, and the second microphone captures sounds on the right directing them to the right ear. This is the theory, but it's not what happens in real world recordings.

There are two traditional types of microphones: omnidirectional and unidirectional. An omnidirectional microphone is a pressure transducer. It senses variations in sound pressure. It is very nearly equally sensitive to sounds coming from all directions. The unidirectional microphone is a velocity transducer. It senses the difference in sound pressure as a soundwave passes by. Unidirectional microphones are more sensitive to sounds coming from one direction.

Sound engineers have many options for positioning traditional microphones to make stereo sound recordings. Author Stanley Lipshitz described many of the options in his paper “Stereo Microphone Techniques”. In summary, the combination of microphone spacing and directional angles produce variations in the ITD and ILD. The basis for all these stereo recording techniques is Rayleigh's 100-year-old Duplex Theory of Sound Localization. Unfortunately, 50 years of refining traditional stereo microphone techniques have not produced sound recordings that are close enough to realizing the goal of creating a “live experience”, so listeners often note that something is missing.

Binaural Recording

Binaural recording methods introduce effects of the human head into the sound recording process. Microphones are inserted into the ears, positioned as close as physically possible to the eardrums. Playback uses headphones that are also inserted into the ears. Since most humans find these intrusions into their ears uncomfortable, binaural recordings are usually made with a dummy head and artificial ears. Author Francis Rumsey summarized recent research in binaural methods in his report titled, “Whose head is it anyway?” In theory, a recording made at the eardrums should contain all the sonic effects caused by a listener's head so the Head Related Transfer Function (“HRTF”), the ILD and the ITD should all be incorporated in a binaural recording together with the effects of the pinnae and the inner ear. When reproduced, the sound at the eardrums should be identical to the original. Unfortunately, the theory doesn't hold up in practice. Listeners perceive flaws.

One flaw perceived in binaural recording playback is the fuzziness of sounds located in front of the listener. These sounds seem distant, while sounds on the left and right seem too close. It's as if the soloist in front of the listener is further away than the surrounding musicians. The phrase “Hole in the Middle” describes this effect. On closer listening one realizes that the sound in front of the listener may not be in front at all. It may be behind the listener. It's not at all like the sound image the listener hears in life. There are several reasons for the difference. Foremost is the lack of visual cues. Our eyes work together with our sense of hearing to help our mind locate sounds. Visual cues tell listeners whether a sound is in front. Listeners also constantly move their heads ever so slightly. By doing so listeners are subconsciously altering the ITD. Our mind senses these microsecond differences in time, processing them like radar to locate sounds precisely. Using this slight head motion, listeners can tell whether a sound is in front, behind and in some cases above the head. A dummy head cannot do this because it is stationary.

Another problem with binaural is the resonances introduced by the ear canal and the pinnae which cause colorations to the sound that are uniquely individual. Listeners each hear their own resonances naturally. However, with binaural the effect is doubled. First the microphone in the dummy head's ear canal embeds the resonances in the recording. Then on playback listeners hear the dummy's resonances added to those of the listeners' own ears. This doubling of resonances produces harshness in mid to high frequency sounds and confuses listeners' minds. The prior art includes efforts to create stereo or binaural recordings using microphones inserted into a live listener's ear canals. For example,FIG. 1Cis a drawing taken from U.S. Pat. No. 3,969,583, Griese et al showing an early attempt to make stereo recordings using microphones27anchored behind the tragus24in the ear canals22of a listener11by inserting “Mounting projection32” into the listener's ear canals. For a more modern example,FIG. 1Dis a drawing taken from another U.S. Pat. No. 9,967,668, Mattana showing another attempt to make recordings with an earpiece set using “binaural” microphones which are also inserted in the ear E within the ear canal of a user, behind the Tragus T. These approaches have not been widely adopted, possibly because physicians ask their patients not to insert foreign bodies into their ears, or perhaps because of poor comfort or poor audio quality in the resulting recordings.

Listening to binaural recordings on loudspeakers instead of headphones produces a sonic cauldron. The major problem is cross talk. Sound from the left channel that's intended only for the left ear can now be heard by the right ear. Similarly, the right ear hears left channel sound from the left speaker. This mixing together of channels collapses the binaural sound stage. Recent developments in digital processing are improving loudspeaker listening by introducing cross talk cancelling signals. The Jambox™ brand speaker product is a commercial implementation of this technology. The three-dimensional quality of the sound is quite impressive, but only for one listener positioned precisely in front of the speakers.

There is a need, therefore, for an improved method and system for capturing a sound field or the sense actually being present with practical sound recording instruments and methods which address many of the flaws of traditional stereo and binaural microphone techniques.

SUMMARY OF THE INVENTION

In the present invention, improved sensors, systems and methods for capturing a sound field (or the sense actually being present) overcome many of the flaws of traditional stereo and binaural microphone techniques. In the present invention, traditional microphones are replaced with a pair of small acoustic pressure sensors that are carried or worn, preferably by a recording user attending an event. A recording user wears the paired sensors in novel and carefully selected positions which capture sound and the sonic image or sound-field a way which enables playback simulating the way the user hears that sound field, when present. A configuration of paired (preferably spherically shaped) acoustic pressure sensors are incorporated into a system which effectively encodes the Head Related Transfer Function (“HRTF”) into an audio recording file while making a recording.

The sound field recording system of the present invention uses paired (i.e., left side and right side) acoustic pressure transducers or sensors, carried by or mounted on opposing sides of the head, attached to left and right side ear hook supports made of a malleable material, so the user/wearer can shape them to fit his or her ears. The paired acoustic pressure transducer assembly, once molded or shaped by the user, is comfortable to wear and visually discrete (meaning others in the vicinity won't likely notice the user is wearing and operating a sound field recording device).

In contrast to the binaural systems of the prior art (which position sensors in the ear canal of a user or stationary dummy head, the paired acoustic pressure transducer assembly of the present invention places sound field recording spherical acoustic pressure sensors or microphones in front of the recording user's ears, in front of the tragus, near or on the recording user's left and right temples. The applicant has discovered that shape of human head and the acoustic shadow is much more uniform (from person to person) in this area, making the HRTF similar for a wide variety of individuals. The applicant's early development work demonstrated that the prior art systems (e.g., as illustrated inFIGS. 1C and 1D) provided poor audio quality in the resulting recordings, because each of these prior art approaches was necessarily adversely affected by the individual listener's external ear anatomy (which can vary significantly from user to user). By moving the new Sonic Sphere sensors of the present invention in front of the ears, the acoustic effects of the ear's pinnae (whose shape differs widely between individuals) is minimized. Moving the sensors forward also reduces the recording angle, which enhances the center image and fills in the sonic soundstage ‘hole in the middle’ in recordings made with the system and method of the present invention. The hole in the middle is the chronic problem for binaural recording systems, as noted above. The sonic sphere spatial microphone sensors of the present invention are not inserted into the ear canals like binaural microphones, so there is no ear canal resonance or physical discomfort. The user's ears are not blocked, so the user/wearer can enjoy the sound while making a recording.

The design inherent in the sound field recording or capture system (with the paired acoustic pressure transducer assembly) and the method of the present invention capture sound with three-dimensional realism because the applicant has re-examined the effects of the Head Related Transfer Function (HRTF) and its associated time and level differences, which are critical during listening or playback for cueing the mind's auditory perception. Traditional microphones, and the complicated techniques for using them, do not adequately capture the HRTF. The Sonic Presence™ system and method of the present invention replace traditional microphones with the paired spherical sensors which are worn to capture sound the way a listener hears it, essentially encoding the HRTF into a recorded audio file while making a recording.

When the listener listens to a recording captured with the sound field recording system of the present invention, his or her mind detects the embedded spatial cues. The sound image expands outside the listener's head and beyond. Left, right, in front, and behind—the listener hears the full 360-degree soundstage all around. The system's spherical sensors are pressure transducers which transform variations in sound pressure into an electrical signal with two dimensions: pitch and amplitude. The system's pair of sensors encode audio which, when played back, provides a three-dimensional quality of sound which test listeners have indicated is quite impressive. When in use by recording users or wearers (recording an event's sound field), the recording user or wearer fits and then dons the labelled left and right spherical acoustic pressure sensors so they are supported next to the correct designated left and right ears, and so becomes the sound engineer, supporting and aiming the paired sensor array for the duration of the recording session.

The paired spherical acoustic pressure sensors are suspended upon the distal end of elongated flexible members made of a malleable material, so the wearer can readily shape the flexible members to fit over his or her ears. Once fitted, the slip-on design is comfortable to wear, very discrete, shockproof and waterproof, and the paired spherical acoustic pressure sensors plug directly into the wearers mobile device's charging port, for power and to communicate the transduced audio signals from each sensor or transducer.

The sensors, system and method of the present invention provide an economical and effective way to make Virtual Reality (“VR”) audio-visual recordings having ambient soundscapes with an aural perspective which is substantially constant and fixed in relation to a contemporaneous video recording. In the method for creating immersive VR recordings of an environment, performance or event of the present invention, the recording user's audio and video recording (“AVR”) instrument (e.g., a smartphone such as an iPhone™ or a portable recorder such as a GoPro™ camera) has at least one lens aimed along a lens central axis and has audio signal inputs for a left channel signal and a right channel signal. The recording user employs a spatial microphone audio recording system (with the left spatial microphone sensor configured to be worn in front of the left ear over (and preferably resting against) the left temple and the right spatial microphone sensor in front of the right ear over (and preferably resting against) the right temple). Once the recording user gathers these components, the components are worn, held or mounted (e.g., upon the recording user's body) with the AVR (or smartphone) in an orientation which aligns the AVR's lens central axis toward a target person, place or thing to be recorded (e.g., while the AVR is carried or worn in front of the recording user's chest, aimed forwardly).

Next, the recording user dons the spatial microphone recording system with the labelled left sensor over the left ear and the labelled right sensor over the right ear so that they are (preferably) symmetrically oriented and more or less equally spaced from an imaginary vertical plane bisecting the left and right sides of the recording user or wearer's head. Next, the AVR is oriented and aligned so that the AVR lens central (aiming) axis is very nearly in substantial alignment with the vertical plane bisecting the left and right sides of the wearer's head such that the AVR lens is preferably substantially equidistant from the left spatial microphone sensor and the right spatial microphone sensor. Preferably, the three elements (i.e., left spatial microphone sensor, right spatial microphone sensor and the AVR) are configured in a triangle with the spatial microphone sensors just a bit wider than head-width apart (e.g., 9 inches apart) and the AVR preferably equally spaced from the spatial microphone sensors and in front of the recording user's sternum (perhaps worn in a pocket or hanging from a chain worn around the neck) or chin (when handheld, in front of the face), so the AVR is preferably about 10-14 inches away from each spatial microphone sensor.

At the moment the recording user initiates a VR recording or begins a VR recording of an environment (e.g., a performance, event, target person, place or thing), the recording user maintains the triangle configuration as constantly as possible for the duration of the VR recording. It is important that for the selected duration of the VR recording, the recording user (or, alternatively, a fixture) maintains the relative positions of the AVR lens central axis to the left spatial microphone sensor and the right spatial microphone sensor such that there is substantially no change in the direction or distances between the AVR lens, the AVR lens central axis, the distance from the AVR lens to the left spatial microphone sensor and the distance from the AVR lens to the right spatial microphone sensor. This configuration, if substantially maintained, provides a VR recording which has, for the entire duration of the recording, a substantially constant and fixed aural perspective which an audience member viewing and hearing the VR recording will recognize as placing seen objects in a sound-field such that (a) moving objects in the VR recording's image are aurally tracked in the VR recording's audio playback and (b) moving (e.g., panning right) perspectives seen in the VR recording's image are continuously aurally tracked in the VR recording's audio playback (e.g., so something audible which was seen as straight ahead initially, upon panning right is heard moving continuously into the audience member's left ear's hearing and away from the right ear).

Applicant's development work with the system and method of the present invention has revealed that these VR recordings, upon playback, provide the substantially constant and fixed aural perspective which audience members recognize as placing seen objects in an immersive sound-field such that moving objects in the VR recording's image are aurally tracked in the VR recording's audio playback when the objects move out of the visual frame. Those objects, now heard but not seen, move into an imagined space which is to the left, or to the right, or overhead or behind the audience member so that the audience member experiences a substantially continuous immersive VR audio-video playback experience.

The above and still further features and advantages of the present invention will become apparent upon consideration of the following detailed description of a specific embodiment thereof, particularly when taken in conjunction with the accompanying drawings, wherein like reference numerals in the various figures are utilized to designate like components.

