TECHNICAL FIELD

The invention relates to a method of operating a radar sensor system for detecting an occupancy in an interior of a vehicle with vital sign monitoring, to a radar sensor system configured for automatically executing such method and a software module enabling automatically executing the method.

BACKGROUND

It is known in the art to use radar technology to detect objects of interest within an occupancy zone. For instance, US 2010/0225764 A1 describes a detection system and a method for such occupancy detection. The detection system can be applied to monitor vehicles or objects that could be used to transport unwanted materials into an area. The presence of objects of interest can include pedestrians, bicycles, automobile bumpers, glass surfaces, and vehicle wheel frames. The detection system comprises a plurality of sensors, which may be formed as frequency-modulated continuous-wave multi-beam radar sensors, including an entry sensor having a field of view oriented to detect object entry with respect to an entry portion of an occupancy volume, and an exit sensor having a field of view oriented to detect object exit with respect to an exit portion of the occupancy volume, the exit portion being different than the entry portion. The detection system further includes at least one approach sensor having a field of view oriented to detect object approach relative to the entry portion of the occupancy volume, and at least one retreat sensor having a field of view oriented to detect object retreat relative to the exit portion of the occupancy volume.

It is also known in the art to use radar technology for automotive seat occupant detection systems. Occupancy sensors based on radar technology offer advantages in comparison to other occupancy detection methods as their operation is contact-free and unnoticeable for vehicle occupants. Moreover, radar sensors can easily be integrated in the vehicle interior, for example behind plastic, plastic covers and diverse materials such as textiles, wood, stone, concrete, glass, and so forth.

Vehicle seat occupancy detection systems are nowadays widely used in vehicles, in particular in passenger cars, for instance for detection of left-behind pets and/or children, vital sign monitoring, vehicle seat occupancy detection for seat belt reminder (SBR) systems, or anti-theft alarm. Such vehicle seat occupancy detection systems can be employed for providing a seat occupancy signal for various appliances, for instance for the purpose of a seat belt reminder (SBR) system or an activation control for an auxiliary restraint system (ARS).

An output signal of a seat occupant detection and/or classification system is usually transferred to an electronic control unit of the vehicle to serve, for instance, as a means of assessing a potential activation of an installed vehicle passenger restraint system.

Further valuable information, usable as important input for Advanced Driver Assistance Systems (ADAS) could be provided by monitoring a vital sign of the detected person.

In U.S. Pat. No. 7,036,390 B2, a method for detecting a human body in a vehicle is described. A synthetic wave is obtained, which represents the synthesis (i.e. the superposition) of a transmitted wave radiated from a sensor and a reflected wave returned from a breathing human body, and the presence or absence of a human in the vehicle is detected from the envelope of the synthetic wave. The frequency of the transmitted wave is 4.7 GHz or higher. When the presence of a human is detected continuously for a predetermined length of time, it is determined that a human is present in the vehicle. When the interior of the vehicle reaches a temperature dangerous to a human, the sensor is activated. A device for monitoring temperature gradient inside the vehicle is provided, and the sensor is activated in advance when it is predicted by the device that the interior of the vehicle is likely to reach a dangerous temperature. When the rate of change of temperature inside the vehicle is mild, the sensor is operated at random timing for a predetermined length of time. When the interior of the vehicle reaches a dangerous temperature, and when the presence of a human is detected, an alarm is issued.

Furthermore, from U.S. Pat. No. 6,753,780 B2 a motion sensing system and method is known for detecting an occupant in a vehicle with enhanced sensitivity to detect small movement, such as movement caused by heartbeat and breathing of an occupant. The system includes a radar motion sensor located in a compartment of the vehicle. The radar sensor transmits and receives signals within the compartment and generates sensed signals. The system has a controller for converting the sensed signals to a frequency domain. The controller further processes the frequency domain of sensed signals and determines the presence of movement of an occupant due to one of heartbeat and breathing of the occupant. To this end, the controller compares the frequency domain of sensed signals within the frequency range to a predetermined frequency characteristic, and further identifies a frequency of the sensed signals indicative of movement of an occupant due to one of heartbeat and breathing.

WO 2015/140333 A1 describes a method for ascertaining whether an unattended child is present within an automotive vehicle. The method uses a radar sensor system comprising a transmitter, and at least one sensor and processing circuitry, and exploits a breathing motion detected by radar signals, for instance by applying autocorrelation and peak finding. The method comprises:illuminating at least one occupiable position within the vehicle with radiation, the radiation exhibiting multiple frequencies;generating radar sensor signals from radiation reflected as a result of the transmitted radiation, a plurality of the radar sensor signals corresponding to different frequencies;operating the processing circuitry for generating, based on the radar sensor signals, a first indicator value, for instance R-value, the first indicator value indicating a degree of motion associated with the occupiable position;determining whether the first indicator value satisfies a first predetermined criterion;if the first indicator value satisfies the first predetermined criterion, generating, based on radar sensor signals, a second indicator value, the second indicator value indicating a degree of repetitive pattern within the radar sensor signals, for instance breathing pattern; anddetermining that an unattended child is present within the automotive vehicle if the second indicator value satisfies a second predetermined criterion.

The second indicator value may comprise a breathing signature representing the extent to which the sensor signals indicate that motion indicative of infant breathing child is detected.

Further, WO 2015/022358 A1 describes a method for sensing an occupancy status within an automotive vehicle using a radar sensor system. The radar sensor system comprises an antenna system, at least one sensor and a processing circuitry. The method comprises illuminating, using the antenna system, at least one occupiable position within the vehicle with continuous wave (CW signals), wherein the CW signals can be frequency modulated in time. At least one sensor signal reflected as a result of the CW signals is received using at least one sensor, wherein the at least one sensor is defining a plurality of receive channels (1, . . . i). Each receive channel has a different frequency (f1, . . . , fi). Processing circuitry is operable for applying, for each receive channel (1, . . . , i), DC offset removal to the corresponding sensor signal to generate a modified signal; and generating, based on the modified signals, one or more occupancy status signals. In case of an employed FMCW radar sensor system, FFT processing is used to obtain the range of a detected occupant, and the vital signs of the occupant are obtained from the phase of the range FFT. The occupancy status signal indicate a property related to the at least one occupiable position. A system for carrying out the method is also disclosed. The techniques provide for in-vehicle occupant detection and classification (airbag suppression), for passenger presence detection and passenger's vital sign monitoring and for seat-belt reminder functionality (SBR). The radar system may consist of one or two (or four) outputs (I/Q) per channel. The received signal may be mixed down with a single channel demodulator or an I/Q demodulator into the baseband. An advanced signal processing including an auto calibration routine and clutter removal may be used to detect occupants and their vital sign (breathing and heart beat) and to determine their distance to the radar system.

In the article “Non-Contact Estimation at60GHz for Human Vital Signs Monitoring Using a Robust Optimization Algorithm” by Ting Zhang et al., Conference IEEE APS 2016, June 2016, Fajardo (Porto-Rico), United States, 2016, AP-S/URSI 2016. <hal-01340613>, an approach to estimate body movements related to vital activities by means of a 60 GHz Doppler radar is described, using robust optimization algorithms including signal autocorrelation analysis in order to extract heart-rate and breathing information from the radar signals.

It is therefore known in the art that a presence of a single vehicle passenger can be detected by conducting electromagnetic measurements such as Doppler radar techniques to measure, for instance, the passenger's breathing or heartbeat. Unfortunately, received radar waves are corrupted with noise if, for instance, a vehicle is moving over a rough surface, in the presence of strong wind gusts, in the presence of engine vibrations or in a carwash. This noise, which could be of high amplitude compared to the signal of a passenger that is desired to detect will inevitably lead to an increased number of false alarms, i.e. falls positives or false negatives.

SUMMARY

It is therefore an object of the invention to provide a method of robust vehicle occupancy detection with vital sign monitoring using radar technology, which is capable of reliably and robustly distinguishing between signals related to an occupant positioned in a specific region of interest in the vehicle interior and disturbing signals caused by vehicle interior events and/or events outside the vehicle, such as vehicle or sensor shaking induced by rough road, strong wind gusts, engine vibrations or car wash.

In one aspect of the present invention, the object may be achieved by a method of operating a radar sensor system for detecting an occupancy in an interior of a vehicle. The radar system comprises a radar transmitting unit that is configured for transmitting radar waves towards a scene within the vehicle interior, and a radar receiving unit that is configured for receiving radar waves that have been transmitted by the radar transmitting unit and have been reflected by an object within the scene.

Further, the radar sensor system includes a signal processing and control unit that is at least configured to derive range information, Doppler information and angle-of-arrival information from the received radar waves. Methods for deriving these types of information are well known to those skilled in the art and therefore need not to be described in more detail herein. Any method that appears suitable to those skilled in the art may be applied.

The phrases “configured to” and “configured for”, as used in this application, shall in particular be understood as being specifically programmed, laid out, furnished or arranged. Further, in this application the terms “radar wave” and “radar signal” are used as synonyms, wherein the term “radar wave” may be used to more specifically describe an electromagnetic wave of a radar frequency traveling in free air, and the term “radar signal” shall be understood to relate either to an electromagnetic wave of a radar frequency that is carried by a transmission line or to a wire-bound signal that has been obtained by processing a received radar wave, for instance an IF (intermediate frequency) signal obtained by mixing a received radar wave with a local oscillator signal and/or demodulation.

The method includes at least the following steps, which are to be carried out in an iterative manner:operating the radar transmitting unit for transmitting a radar wave towards a scene within the vehicle interior,operating the radar receiving unit for receiving at least one radar wave that has been generated by reflection of the transmitted radar wave,operating the signal processing and control unit for:decomposing the received at least one radar wave into range, Doppler and angular information, anddetecting and monitoring vital signs of occupants in each region of interest of a plurality of predefined different regions of interest.

According to an aspect of the invention, the step of operating the signal processing and control unit further includes:quantifying and tracking a movement in each region of interest of a plurality of predefined different regions of interest by angular gating and range gating, andfor each region of interest, determining whether quantified and tracked movements in the scene are related to an occupant or to external or internal disturbances, based on a fulfillment of at least one predefined condition concerning the quantified and tracked movements and/or the detected and monitored vital signs, wherein the at least one predefined condition includes that if strong movement is present in all or several range regions of interest, a strong disturbance is likely to be present, and if movement is detected only in a region with a seating position, and vital signs are found, it is quite certain to assume that the seat is occupied.

The phrase “vital signs of an occupant”, as used in this application, shall in particular be understood to encompass at least one out of heartbeat and breathing of the occupant. The term “occupant”, as used in this application, shall in particular be understood to encompass human beings and pets.

It is a feature of at least some embodiments of the invention that by using a combination of range, Doppler and angle-of-arrival information, a robust vehicle occupancy detection method can be provided. The method enables to detect, quantify and track different movements in different regions of the scene. Internal and external motion can then be detected, separated and rejected or suppressed, for instance by range gating. The disclosed method is thus capable of reliably and robustly distinguishing between signals related to an occupant positioned in a specific region of interest in the vehicle interior and disturbance signals caused by vehicle interior events and/or events outside the vehicle.

Further, a presence of several occupants in the vehicle interior can be detected and their positions with respect to the radar sensor system be determined, and respective vital signs can be monitored simultaneously. Strong signals or disturbances that are present only outside a region of interest, e.g. a region including a seat row, do not affect any vital sign signals of occupants. As a consequence, a number of false alarms can be reduced and a robustness of the radar sensor system can be increased.

In preferred embodiments of the method, the step of determining whether quantified and tracked movements in the scene are related to an occupant or to external or internal disturbances is based on a temporal development of fulfillments of the at least one predefined condition concerning the quantified and tracked movements and the detected and monitored vital signs. In this way, a robustness of the method can be increased.

Preferably, the at least one predefined condition concerning the quantified and tracked movements and the detected and monitored vital signs includes consideration of a constraint regarding a number of passengers in the vehicle interior to be constant while the vehicle is driving. By use of this consideration, a plurality of potential occupancies in the interior of the vehicle can be ruled out, and a faster reactivity of the method can be achieved without compromising a sensitivity to false alarms.

Preferably, the step of decomposing the received radar waves into range, Doppler and angular information includes applying a fast Fourier transform to an intermediate frequency (IF) signal of the received radar waves for converting the IF signal of the received radar waves from the time domain into the range domain. In suitable embodiments, the range information can readily be obtained in this way.

In preferred embodiments of the method, the step of decomposing the received radar waves further includes removing a static scene from the fast Fourier transform of the IF signal of the received radar waves by making use of a fast Fourier transform of an IF signal of at least one previously received radar wave. By removing the static scene, movements in the balance of the fast Fourier transform of the IF signals of the received radar waves can be detected more easily.

The static scene may be removed by, without being limited to,subtracting the fast Fourier transform of IF signals of the received radar waves computed in the previous iteration from the fast Fourier transform of IF signals of the currently received radar waves, orsubtracting an average of a plurality of fast Fourier transforms of IF signals of received radar waves, computed at previous iterations, or byapplying any other static scene removal method that appears to be suitable to those skilled in the art, such as applying a high pass filter or the like.

In preferred embodiments of the method, the step of quantifying and tracking a movement in each region of interest includes comparing an amount representing the quantified movement with at least one predefined detection threshold value for the quantity. This can allow for a fast and simple determination of a location of a maximum quantity within the gating.

Preferably, the at least one predefined detection threshold value for the quantity is a function of at least one out of vehicle speed and vehicle acceleration. In this way, the at least one predefined detection threshold value can be adjusted to match different driving situations and, by that, can reduce a number of false alarms. For instance, the at least one predefined detection threshold value may increase with increasing vehicle speed, at which larger disturbing signals by vehicle shaking induced by rough road driving are to be expected.

In preferred embodiments, determining a number of occupants in the vehicle interior can be supported by providing a signal from a wheel pressure sensor or sensors, or by providing analysis data from a wheel pressure sensor device. The wheel pressure information can be combined with information about the vehicle door activities for further support of determining the number of occupants.

In preferred embodiments of the method, the step of detecting and monitoring vital signs of occupants comprises calculating a phase signal from the fast Fourier transform of IF signals of the received radar waves at determined movements in each region of interest, and processing each phase signal by reconstructing the frequency domain signal into a phase-coherent time domain signal. In this way, the vital signs can readily be obtained from the received radar waves.

Preferably, the step of quantifying and tracking a movement in each region of interest includes calculating power integrals in predefined regions for each region of interest. In this way, a robust and reliable tracking of a movement in a specific region of interest can readily be accomplished.

In preferred embodiments of the method, the plurality of predefined different regions of interest includes at least one out of a range-azimuth, range-elevation or azimuth-elevation region of interest. In this way, a robustness of the method can be increased even more. In suitable embodiments of the radar sensor system, occupancy detection can be enabled for car interiors with larger cabin volume, which may, for instance, include several rows of seats. Further, detection of several occupants at different range-angle regions of interest can be enabled, and simultaneous monitoring of vital signs of these occupants can be accomplished.

Preferably, the method comprises a step of operating the signal processing and control unit for generating an output signal that is indicative of a determined occupancy of the vehicle interior. In this way, information concerning the determined occupancy of the vehicle interior can be transferred and assessed in a fast manner, and appropriate measures, such as warnings by an advanced driver assistance system (ADAS) can quickly be taken if applicable.

In another aspect of the invention, a radar sensor system for detecting an occupancy in an interior of the vehicle is provided. The radar sensor system comprises a radar transmitting unit that is configured for transmitting radar waves towards a scene within the vehicle interior, and a radar receiving unit that is configured for receiving radar waves that have been transmitted by the radar transmitting unit and have been reflected by an object within the scene.

Further, the radar sensor system includes a signal processing and control unit that is at least configured to derive range information, Doppler information and angle-of-arrival information from the received radar waves, to control operation of the radar transmitting unit and the radar receiving unit and to automatically execute steps of the method disclosed herein.

The benefits described in context with the disclosed method of operating a radar sensor system for detecting an occupancy in an interior of a vehicle apply to the radar sensor system to the full extent.

In preferred embodiments of the radar sensor system, the radar transmitting unit includes a plurality of radar transmitting antennas and the radar receiving unit includes a plurality of radar receiving antennas, and the radar transmitting antennas and the radar receiving antennas are configured to operate in a multiple-input and multiple-output (MIMO) configuration. In this way, an angular resolution of the radar sensor system can be improved. Finer angular resolution facilitates better distinguishing ability between signals related to an occupant positioned in a specific region of interest in the vehicle interior and disturbing signals caused by vehicle interior events and/or events outside the vehicle.

In one contemplated MIMO configuration with a plurality of transmitting antennas, each transmitting antenna is understood to be able to transmit radar waves in an independent manner that represent mutually orthogonal codes. Each receiving antenna is further understood to be able to receive radar waves that have been transmitted by any of the transmitting antennas and that have been reflected by an object without any cross talk disturbance.

Another contemplated MIMO configuration is the Time Division Multiplexing (TDM)-MIMO, which does not require orthogonal codes. Also, Binary Phase Modulated (BPM)-MIMO configurations are contemplated for use with this invention.

MIMO radar configurations provide benefits regarding enlarged size of virtual aperture, improved spatial resolution and less sensitivity to interference signals, as is well known in the art.

Preferably, the radar transmitting unit and the radar receiving unit form an integral unit and comprise a plurality of at least two transceiver antenna units, which are configured to work in a MIMO configuration. In this way, hardware parts can be saved and a more compact design of the radar sensor system can be achieved.

In preferred embodiments of the radar sensor system, the radar transmitting unit is configured to transmit frequency-modulated continuous radar waves. Frequency-modulated continuous wave (FMCVV) radar sensor systems are well known in the art to be able to provide received radar waves from which range information, Doppler information and angle-of-arrival information can be derived.

In yet another aspect of the invention, a software module is provided for controlling automatic execution of the method disclosed herein.

The method steps to be conducted are converted into a program code of the software module, wherein the program code is stored on a non-transitory computer-readable medium, such as a digital memory unit of the radar sensor system, and is executable by a processor unit of the radar sensor system. Preferably, the digital memory unit and/or processor unit may be a digital memory unit and/or a processing unit of the signal processing and control unit of the radar sensor system. The processor unit may, alternatively or supplementary, be another processor unit that is especially assigned to execute at least some of the method steps.

The software module can enable a robust and reliable execution of the method and can allow for a fast modification of method steps.

These and other aspects of the invention will be apparent from and elucidated with reference to the embodiments described hereinafter.

It shall be pointed out that the features and measures detailed individually in the preceding description can be combined with one another in any technically meaningful manner and show further embodiments of the invention. The description characterizes and specifies at least one embodiment of the invention in particular in connection with the figures.

