CROSS REFERENCE TO RELATED APPLICATION

This application is the National Stage of PCT/CN2018/109348 filed on Oct. 8, 2018, which claims priority under 35 U.S.C. § 119 of Chinese Application No. 201710994756.6 filed on Oct. 23, 2017, the disclosure of which is incorporated by reference.

TECHNICAL FIELD

Embodiments of the present disclosure relate to acquisition equipment, a sound acquisition method, a sound source tracking system and a sound source tracking method.

BACKGROUND

Sound localization technology is a technique for determining spatial location information of one or more sound sources. The sound localization technology has the characteristics of passive detection mode, non-contact measurement, no communication condition interference, strong concealment and no visibility limitation. Therefore, the sound localization technology can be widely used in medical research, traffic detection, video conference, military field, robot speech recognition and other fields.

SUMMARY

At least one embodiment of the disclosure provides an acquisition equipment, comprising: an audio acquisition device, an image acquisition device, an information processing device and an angle control device, wherein the audio acquisition device is configured to acquire sound of a target object; the image acquisition device is configured to acquire an optical image including an acquisition object; the information processing device is configured to process the optical image to determine position information of the target object; and the angle control device is configured to receive the position information of the target object sent by the information processing device, and control a sound pick-up angle of the audio acquisition device according to the position information of the target object.

For example, in the acquisition equipment according to an embodiment of the disclosure, the image acquisition device and the audio acquisition device are integrated into a whole; the information processing device is also configured to determine image position information of the acquisition object in the optical image, and determine a horizontal azimuth of the acquisition object according to the image position information, in which the position information of the target object includes the horizontal azimuth of the acquisition object; and the angle control device is further configured to receive the horizontal azimuth of the acquisition object and control the sound pick-up angle of the audio acquisition device according to the horizontal azimuth of the acquisition object.

For example, in the acquisition equipment according to an embodiment of the disclosure, the information processing device is also configured to establish a camera coordinate system based on the image acquisition device, and establish an audio acquisition coordinate system based on the audio acquisition device, determine camera position information of the acquisition object in the camera coordinate system, transform the camera position information of the acquisition object into audio acquisition position information in the audio acquisition coordinate system, and determine an azimuth of the acquisition object in the audio acquisition coordinate system according to the audio acquisition position information, in which the position information of the target object includes the azimuth of the acquisition object; and the angle control device is further configured to receive the azimuth of the acquisition object and control the sound pick-up angle of the audio acquisition device according to the azimuth of the acquisition object.

For example, in the acquisition equipment according to an embodiment of the disclosure, the target object includes a sound source; the image acquisition device and the audio acquisition device are integrated into a whole; the information processing device is further configured to determine image position information of the acquisition object in the optical image, determine first image relative position information between the acquisition object and the sound source, and determine a horizontal azimuth of the sound source according to the image position information of the acquisition object and the first image relative position information, in which the position information of the target object includes the horizontal azimuth of the sound source; and the angle control device is further configured to receive the horizontal azimuth of the sound source and control the sound pick-up angle of the audio acquisition device according to the horizontal azimuth of the sound source.

For example, in the acquisition equipment according to an embodiment of the disclosure, the information processing device is further configured to determine image reference position information of the acquisition object in the optical image, determine second image relative position information between the image position information and the image reference position information of the acquisition object in the optical image, acquire third image relative position information between the sound source and the image reference position information, and determine the first image relative position information according to the third image relative position information and the second image relative position information.

For example, in the acquisition equipment according to an embodiment of the disclosure, the target object includes a sound source; the information processing device is further configured to establish a camera coordinate system based on the image acquisition device, establish an audio acquisition coordinate system based on the audio acquisition device, determine camera position information of the acquisition object in the camera coordinate system, determine first camera relative position information between the acquisition object and the sound source in the camera coordinate system, determine camera sound source position information of the sound source in the camera coordinate system according to the camera position information of the acquisition object and the first camera relative position information, transform the camera sound source position information into audio acquisition sound source position information in the audio acquisition coordinate system, and determine an azimuth of the sound source in the audio acquisition coordinate system according to the audio acquisition sound source position information, in which the position information of the target object includes the azimuth of the sound source; and the angle control device is further configured to receive the azimuth of the sound source and control the sound pick-up angle of the audio acquisition device according to the azimuth of the sound source.

For example, in the acquisition equipment according to an embodiment of the disclosure, the information processing device is further configured to determine camera reference position information of the acquisition object in the camera coordinate system, determine second camera relative position information between the camera position information and the camera reference position information of the acquisition object in the camera coordinate system, acquire third camera relative position information between the sound source and the camera reference position information, and determine first camera relative position information according to the third camera relative position information and the second camera relative position information.

At least one embodiment of the disclosure provides a sound source tracking system, comprising: the acquisition object and the acquisition equipment according to any one of the above items, the acquisition object including a luminous element, wherein the luminous element is configured to emit light when the acquisition object is in a tracking area of the acquisition equipment; and the angle control device of the acquisition equipment is further configured to control the sound pick-up angle of the audio acquisition device according to the position information of the target object so as to track the target object.

For example, in the sound source tracking system according to an embodiment of the disclosure, the image acquisition device is configured to acquire the optical image of the tracking area of the acquisition equipment when the acquisition object is in the tracking area of the acquisition equipment.

For example, in the sound source tracking system according to an embodiment of the disclosure, the acquisition object further includes a controller and an inertial measurement unit; the inertial measurement unit is configured to detect motion information of the acquisition object; and the controller or the information processing device is configured to establish a camera coordinate system based on the image acquisition device, acquire type information of the acquisition object, acquire the motion information of the acquisition object, and determine camera reference position information of the acquisition object in the camera coordinate system or determine image reference position information of the acquisition object in the optical image according to the motion information of the acquisition object and the type information of the acquisition object.

For example, the sound source tracking system according to an embodiment of the disclosure comprises a plurality of acquisition objects; and luminous parameters of the luminous elements in the plurality of acquisition objects have different change modes.

At least one embodiment of the disclosure provides a sound acquisition method, comprising: acquiring an optical image of an acquisition object; processing the optical image to determine position information of a target object; controlling a sound pick-up angle of an audio acquisition device of an acquisition equipment according to the position information of the target object; and acquiring sound of the target object through the audio acquisition device.

For example, in the sound acquisition method according to an embodiment of the disclosure, the acquisition equipment further includes an image acquisition device; the image acquisition device is configured to acquire the optical image; the image acquisition device and the audio acquisition device are integrated into a whole; processing the optical image to determine the position information of the target object includes: determining image position information of the acquisition object in the optical image, and determining a horizontal azimuth of the acquisition object according to the image position information, in which the position information of the target object includes the horizontal azimuth of the acquisition object; and controlling the sound pick-up angle of the audio acquisition device of the acquisition equipment according to the position information of the target object includes: controlling the sound pick-up angle of the audio acquisition device according to the horizontal azimuth of the acquisition object.

For example, in the sound acquisition method according to an embodiment of the disclosure, the acquisition equipment further includes an image acquisition device, the image acquisition device is configured to acquire the optical image; processing the optical image to determine the position information of the target object includes: establishing a camera coordinate system based on the image acquisition device, establishing an audio acquisition coordinate system based on the audio acquisition device, determining camera position information of the acquisition object in the camera coordinate system, transforming the camera position information of the acquisition object into audio acquisition position information in the audio acquisition coordinate system, and determining an azimuth of the acquisition object in the audio acquisition coordinate system according to the audio acquisition position information, in which the position information of the target object includes the azimuth of the acquisition object; and controlling the sound pick-up angle of the audio acquisition device of the acquisition equipment according to the position information of the target object includes: controlling the sound pick-up angle of the audio acquisition device according to the azimuth of the acquisition object.

For example, in the sound acquisition method according to an embodiment of the disclosure, the acquisition equipment further includes an image acquisition device, the image acquisition device is configured to acquire the optical image, the target object includes a sound source, the image acquisition device and the audio acquisition device are integrated into a whole; processing the optical image to determine the position information of the target object includes: determine an image position information of the acquisition object in the optical image, determine first image relative position information between the acquisition object and the sound source in the optical image, and determine a horizontal azimuth of the sound source according to the image position information of the acquisition object and the first image relative position information, in which the position information of the target object includes the horizontal azimuth of the sound source; and controlling the sound pick-up angle of the audio acquisition device of the acquisition equipment according to the position information of the target object includes: controlling the sound pick-up angle of the audio acquisition device according to the horizontal azimuth of the sound source.

For example, in the sound acquisition method according to an embodiment of the disclosure, determining the first image relative position information between the acquisition object and the sound source in the optical image includes: determining image reference position information of the acquisition object in the optical image; determining second image relative position information between the image position information and the image reference position information of the acquisition object in the optical image; acquiring third image relative position information between the sound source and the image reference position information; and determining the first image relative position information according to the third image relative position information and the second image relative position information.

For example, in the sound acquisition method according to an embodiment of the disclosure, the acquisition equipment further includes an image acquisition device, the image acquisition device is configured to acquire the optical image, the target object includes a sound source; processing the optical image to determine the position information of the target object includes establishing a camera coordinate system based on the image acquisition device, establishing an audio acquisition coordinate system based on the audio acquisition device, determining camera position information of the acquisition object in the camera coordinate system, determining first camera relative position information between the acquisition object and the sound source in the camera coordinate system, determining camera sound source position information of the sound source in the camera coordinate system according to the camera position information of the acquisition object and the first camera relative position information, transforming the camera sound source position information into audio acquisition sound source position information in the audio acquisition coordinate system, and determining an azimuth of the sound source in the audio acquisition coordinate system according to the audio acquisition sound source position information, in which the position information of the target object includes the azimuth of the sound source; and controlling the sound pick-up angle of the audio acquisition device of the acquisition equipment according to the position information of the target object includes: controlling the sound pick-up angle of the audio acquisition device according to the azimuth of the sound source.

For example, in the sound acquisition method according to an embodiment of the disclosure, determining the first camera relative position information between the acquisition object and the sound source in the camera coordinate system includes: determining camera reference position information of the acquisition object in the camera coordinate system; determining second camera relative position information between the camera position information and the camera reference position information of the acquisition object in the camera coordinate system; acquiring third camera relative position information between the sound source and the camera reference position information; and determining the first camera relative position information according to the third camera relative position information and the second camera relative position information.

At least one embodiment of the disclosure provides a sound source tracking method, comprising: controlling a luminous element of an acquisition object to emit light when determining that the acquisition object is in a tracking area of an acquisition equipment; acquiring an optical image including the acquisition object; processing the optical image to determine position information of a target object; and controlling a sound pick-up angle of the audio acquisition device of the acquisition equipment according to the position information of the target object so as to track the target object.

For example, in the sound source tracking method according to an embodiment of the disclosure, wherein the acquisition equipment includes an image acquisition device; the image acquisition device is configured to acquire the optical image; the acquisition object further includes an inertial measurement unit; the sound source tracking method further comprises: detecting motion information of the acquisition object through the inertial measurement unit; acquiring type information of the acquisition object; and determining camera reference position information of the acquisition object in the camera coordinate system or determining image reference position information of the acquisition object in the optical image, according to the motion information of the acquisition object and the type information of the acquisition object.

