CROSS-REFERENCE TO RELATED APPLICATION

This application claims the priority, under 35 U.S.C. § 119, of European patent application EP 19 170 808, filed Apr. 24, 2019; the prior application is herewith incorporated by reference in its entirety.

BACKGROUND OF THE INVENTION

Field of the Invention

The present invention relates to a method and a system for analyzing the control of a vehicle containing an autonomous driving unit.

Autonomous driving on public roads has progressed into a real part of the future automotive landscape. Reflective of its still ongoing evolution, autonomous driving systems are currently grouped into at least five incremental levels of capability, from level 1 where the human driver must be ready to take over any time, to level 5 corresponding to complete automated driving, while humans are just along for the ride and to tell the vehicle where to go.

Autonomous driving requires extremely capable software not only for driving the car but for recognizing the surrounding objects and for correct prediction of their potential behavior with regard to the current traffic scenario, i.e. the current driving situation of the autonomously driven car. Such recognition and prediction is extremely complex given the number and different kinds of objects, situations and possibilities of behavior. Any combination of objects, situations and possibilities of behavior which is not correctly interpreted by the software may lead to an accident, resulting in damage to persons, costs and loss of confidence in the car and its driving abilities. This not only places a huge burden on quality management, but also implies questions regarding liability issues. For example, it will be necessary to think about how vehicles capable of driving autonomously are insured, e.g. whether the driver or the software will be held responsible, or whether the liability will be shared among driver and software.

Thus, in order to provide the necessary safety, the software and its reaction to certain traffic situations is usually extensively tested. Such testing is generally done in millions of kilometers of test driving in completely or at least partly autonomous driving modes. Further, autonomous driving software may be programmed such that it recognizes a potentially overtaxing situation in advance and alerts the driver to take over control of the vehicle. This functionality is usually realized in autonomous driving systems of levels 1 to 3.

SUMMARY OF THE INVENTION

It is an object of the present invention to provide an improved method and system for assessing the capability of an autonomous driving unit.

This object is solved by a method and a system for analyzing the control of a vehicle containing an autonomous driving unit according the independent claims.

The method for analyzing the control of a vehicle containing an autonomous driving unit according to a first aspect of the invention contains the steps of: (i) detecting a change in the driving mode from autonomous driving to manual driving; (ii) monitoring at least one driving parameter before and/or after detecting the change; and (iii) based on driving values obtained by the monitoring with respect to the detected change in driving mode, determining at least one driving quantity quantifying the quality of interplay between the autonomous driving unit and a human driver.

A driving quantity quantifying the quality of interplay between the autonomous driving unit and a human driver in the sense of the present invention particularly represents a quantity describing the overall or combined driving quality of a vehicle, wherein the vehicle is temporarily controlled by the autonomous driving unit and temporarily controlled by the human driver. In particular, the at least one driving quantity is a measure for the driving behavior of both the autonomous driving unit and the human driver, wherein the effect of the driving of the autonomous driving unit on the driving of the human driving and/or vice versa is considered.

An aspect of the invention is based on the approach that for comprehensive testing of an autonomous driving unit, e.g. an autonomous driving system or at least its software, it is advantageous to not only consider and possibly rate its reactions in a variety of traffic scenarios, i.e. driving situations, but also to consider and possibly rate the reactions of the human driver in traffic scenarios into which he is thrown after taking over vehicle control from the driving unit. Analyzing e.g. driving values obtained by monitoring driving parameters as speed, acceleration or deceleration immediately before and/or after a change in driving mode from autonomous driving by the autonomous driving unit to manual driving by the human driver may allow to more precisely assess the capability of the autonomous driving unit to recognize and/or handle particularly complex and demanding situations. In particular, based on these driving values, at least one driving quantity may be derived which quantifies the quality of man-machine interactions between the autonomous driving unit and human driver. In other words, it has been realized that there is a human component which should be factored into the testing and/or rating of a, in particular level 1-3, autonomous driving unit.

For example, an autonomous driving unit may be capable of handling most of the usually occurring traffic scenarios. Thus, if only purely autonomous driving is tested, the driving unit will rate exceptionally well, although its ability to assess a scenario too complex in advance and warn the driver accordingly may be only poor. This will result in situations where the driver might have insufficient time to take over control and react accordingly to prevent an accident. Such failure of the driving unit will generally not be recognized by the usual testing methods, because the driving mode will have changed to manual control and thus the human driver will be incorrectly held liable for the accident.

Thus, by detecting the change of the driving mode from autonomous driving to manual driving and taking it into account when e.g. determining quantities reflective of the quality of combined driving of the driving unit and the human driver, the autonomous driving unit may be tested more extensively. In particular, by this means a weak point of an autonomous driving unit can be detected such that e.g. the corresponding software may be enhanced to improve the control takeover procedure. Further, it becomes possible to compare driving quantities obtained with regard to a change in driving mode to augment the autonomous driving software, e.g. feeding data corresponding to driving quantities indicative of high quality driving to a neural network in order to train the autonomous driving unit.

For example, quantifying the quality of interplay between the autonomous driving unit and human driver may be based on quantities quantifying the quality of driving of the human driver directly after he has taken over control of the vehicle. Alternatively or additionally, quantifying the quality of interplay between the autonomous driving unit and human driver may be based on quantities quantifying the quality of driving of the autonomous driving unit even before the autonomous driving unit outputs a control takeover signal due to a complex traffic scenario. This way, e.g. the capability of the autonomous driving unit to assess complex traffic scenarios and potentially its effect on the driving behavior of the human driver may be taken into account, for example be rated. In particular, if these quantities are reflective for a high quality of driving, in particular better than average, this may indicate an effective design of the critical man-machine aspects of the driving unit with respect to alerting the driver for imminent transfer of control.

A further aspect is that the human driver and the autonomous driving unit may be understood as one entity. As such, the entity containing the autonomous driving unit, in particular the corresponding control software, and a specific human driver can be rated with regard to their combined driving quality. I.e. the autonomous driving unit and the human driver may not only be rated separately with a score reflective of the driving quality, but together. This rating score may be a useful measure not only for software improvement or redesign, but also for calculating an insurance risk or a premium, respectively, for an insurance company insuring vehicles which comprise this of driving unit or make use of this software, respectively. The rating score of the insurance risk is a fine indicator for usage based product liability for autonomous vehicle insurances.

In the sense of the present invention, obtaining driving values by monitoring with regard to the change in driving mode can be understood as grouping the obtained driving values, in particular processing the obtained driving modes depending on the respective group. For example, a driving value obtained immediately before the change in driving mode may be used to determine a driving quantity which is combined or at least compared to with a driving quantity determined from driving value obtained immediately after the change in driving mode to determine the at least one driving quantity quantifying the quality of interplay between the autonomous driving unit and the human driver. In particular, the at least one driving quantity may be determined based on the temporal relationship, e.g. the temporal distance, of the respective points in time at which the driving values relative to the change in driving mode, e.g. relative to a point in time at which the human driver takes over control of the vehicle.

An autonomous vehicle in the sense of the present invention can be a vehicle with software for controlling the driving of the vehicle completely, e.g. levels 4 and 5, or only partly, e.g. level 1 to 3. Therein, the software may be also referred to as an artificial intelligence (AI).

A driving parameter in the sense of the present invention can be the speed of the vehicle, an acceleration of the vehicle, either positive or negative, in driving direction, or sideward, a combination of speed and acceleration and/or a combination of different accelerations, e.g. during a predefined time span, a steering wheel turning parameter, e.g. a turning angle, turning rate per time, subsequent turning angles in different directions, a pedal activation parameter, e.g. pressing force, pressing velocity, or the like.

A driving quantity can be a one-dimensional quantity, e.g. a single value, or a multi-dimensional quantity, e.g. a vector with a multitude of values, which quantifies a driving quality of the driving of the autonomous driving unit or the human driver, respectively. A driving quality may be quantified by a comparison of values of a driving parameter, like speed, acceleration or turning of the steering wheel, with a geostationary parameter value, like a speed limit or a parameter value of other driving occurring at the same location. If the autonomous driving unit or the human driver drives faster than a speed limit or a prevailing driving speed of surrounding vehicles, the resulting value, like the speed difference or another value derived from the comparison, is an indicator of the driving quality. Generally, the faster the driving, the higher the resulting value, the worse is the driving, and the other way around.

In general:

a) Values of a driving parameter, like speed, acceleration or steering wheel turning angle, or the like, may be determined during the driving of the autonomous driving unit and/or the human driver. These values are called driving values.

b) Each driving value may be compared to a geostationary value of the same parameter, e.g. a speed limit, a prevailing speed, or another parameter value of other vehicles at the same location. The comparison result is preferably a value. Alternatively, a plurality of driving values may be combined to a resulting value.
c) A plurality of such resulting values may be consolidated to a driving quality value, which can be the mean, median or other mathematical resume of the resulting values. For example, the resulting values determined from driving during a driving period are consolidated to a driving quality value.
d) A driving quality value may be used as a driving quantity. Or a plurality of driving quality values collected during a superior driving period encompassing a corresponding plurality of driving values are consolidated to a driving quantity, which can be the mean, median or other mathematical resume of the driving quality values. The superior driving period may allow to generally characterize the combined driving of the autonomous driving unit and the human driver, e.g. during an extended period of time or/and a significant accrual of mileage—that is at least over one year, or at least over 10,000 km recorded driving distance, possibly in varied locales such as: urban, rural or highway, respectively.

A driving quantity is preferably based on driving values of a particular type of driving parameter, e.g. from values obtained by monitoring the vehicle speed or the vehicle acceleration. Preferably, each driving value of the same driving parameter is generated in the same way, e.g. all acceleration values are obtained by means of the same accelerometer. Alternatively, a driving quantity may be based on driving values of multiple driving parameters, in particular a combination of driving parameters. For example, a driving quantity may be based on a measured time interval and measured vehicle speed.

Preferably, the driving quantity is a driving quality metric. A driving quality metric can be a driving quantity which is determined for a plurality of autonomous driving units or human drivers, respectively, in the same way, so that the quantities are comparable and rankable, preferably for at least 1,000 autonomous driving units or human drivers. A superior driving period of a metric is called metric period in the following.

A driving period can be a single or a plurality of journeys of the vehicle, or all journeys of the vehicle during a predetermined time span, like ten days, a month or a quarter of a year. A driving period can further be a sub-period of a single or a plurality of journeys, e.g. grouped by day/night, or weather.

Driving quantities may be determined in different categories, each category using different types of driving values, as for example speed, acceleration, deceleration, and the like or a combination thereof. In particular, from one or more driving parameters being monitored during autonomous or manual driving a plurality of quantities of different categories can be determined.

To distinguish between different driving areas in which the driving risk may differ due to area characteristics, it is advantageous if the categories are a plurality of different locale categories where the driving parameter is monitored. The locale categories can be urban, rural and highway, or other specific driving surroundings such as “suburban” or “urban-business”.

As outlined above, a plurality of different categories of driving quantities can be generated during a driving period of one vehicle. The categories can be differentiated with regard to type, e.g. speed and acceleration events. Further, the categories can be differentiated into different locales where the driving values are gathered. Types and locales may be independent from one another. Accordingly, during one time period, more than one driving quantity can be determined. In other words: Preferably, a plurality of quantities are determined for different categories of driving quantities, in particular for different locales and/or for different types of driving parameters.

To obtain a rating corresponding to a determined driving quantity quantifying the interplay between the autonomous driving unit and the human driver, e.g. quantifying the quality of driving of the entity containing the autonomous driving unit and the human driver, the driving quantity can be compared with other driving quantities, preferably of the same type and locale. A suitable rating parameter is the quantity as such. This rating of a quantity against a plurality of quantities contained e.g. in a database may result in a rating value, like 43%, which means, that 57% of all metric values of the database show a statistically better driving and 43% show a worse driving.

Alternatively or additionally, it is possible to rate quantities with regard to the locale of the driving quantity and/or use the respective locale for a driving risk assessment. This is particularly advantageous because usually, driving on rural roads is considered more dangerous as driving on expressways, and driving in downtown traffic is more dangerous than driving on a well-built and wide rural road. In other words: A driving risk strongly depends on the location where the driving occurs, or more general: the area category. It is advantageous, therefore, if the rating is categorized into multiple different area categories where the driving event—from which the quantities are calculated—occurred.

In the following, preferred embodiments of the invention and their modifications will be described which may be combined with each other as well as with further aspects of the invention as far as its not explicitly excluded.

In a preferred embodiment, detecting the change in driving mode includes determining the point in time a control takeover request is outputted by the autonomous driving unit and determining the point in time the human driver takes over control. By this means, the driving values may be grouped at least into a period prior to the outputting of the control takeover request, a period in between the outputting of the control takeover request and the human driver taking over control, and a period after the human driver takes over control. Further, the at least one driving quantity may be determined based on a, in particular specific, combination of driving values from these different groups or periods, respectively. For example, the driving values from different periods may be differently weighted. This allows for a particular differentiated analysis of vehicle control, e.g. rating of the combined driving of the autonomous driving unit and the human driver. The rating may be suitable e.g. with regard to a user based insurance, for example to determine premiums. Further, this differentiated analysis of vehicle control may provide information based on which a global wear of the vehicle may be estimated, the information being usable e.g. to assess the vehicles value.

In particular, detecting the change driving mode may include identifying different stages of the transfer of control, wherein in each of the different stages, the autonomous driving unit or the human driver may exhibit different driving behavior, respectively.

For example, in a first stage, the control may be solely associated with the autonomous driving unit. The first stage preferably reaches up to the point in time where the autonomous driving unit outputs the control takeover request, which may be referred to as the control takeover request event. In other words, the first stage preferably covers the usual autonomous driving mode where responsibility for controlling the vehicle lies exclusively with the autonomous driving unit. The first stage may thus also be referred to “AI-assisted control”, wherein a metric based on driving values allows for rating the performance of the autonomous driving unit in basically all usually occurring traffic scenarios.

In a second stage, for example, the control of the vehicle may still be solely associated with the autonomous driving unit. However, the second stage may reach from the control takeover request event up to the point in time the human driver takes over control, which may be referred to as the control takeover event. During this stage, the focus may be not only on how the autonomous driving unit performs in the given situation, but rather how the driving will affect the situation the human driver finds himself in when he takes over control. Although control of the vehicle still lies with the autonomous driving unit, a metric based on driving values obtained in the second stage allows for rating the performance of the autonomous driving unit with respect to particularly complex traffic scenarios. The second stage may also be referred to as the “transfer of control”.

In a third stage, for example, the control of the vehicle may be associated with the human driver. The third stage preferably reaches from the control takeover event up to a point in time at which the human driver has adjusted to the current traffic scenario and no immediate consequences of his recent takeover of control are to be expected. In other words, the third stage may cover the time interval in which the human driver has to resolve the complex traffic scenario the autonomous driving unit has gotten him into. Preferably, the duration of the third stage corresponds to a predefined second time interval, e.g. at least 30 seconds, in particular at least one minute, after the takeover control event. The third stage may be referred to as the “default to manual control”, wherein a metric, even though based on driving values solely obtained during manual control, allows for rating the control handover performance of the autonomous driving unit.

For example, if the human driver has to brake hard directly after taking over control from the autonomous driving unit, this could be interpreted as negatively affecting the rating of the autonomous driving unit by increasing e.g. a driving quality value associated with the driving quantity (as the hard breaking corresponds to strong deceleration), because the autonomous driving unit e.g. failed to output the control takeover request early enough or did not drive defensively enough during the second stage.

In a fourth stage, for example, the control of the vehicle may be associated with the human driver. The fourth stage may cover all the manual driving performed by the human driver after he has adjusted to the traffic scenario in which he is taking over control, e.g. starting from the end of the predetermined second time interval after the control takeover event. During this stage, a metric based on driving values obtained during manual control relates solely to the driver and can be e.g. used to compare the quality of driving of the human driver to the quality of driving of the autonomous driving unit. The fourth stage may be referred to as “manual control”.

In another preferred embodiment, monitoring the at least one driving parameter includes determining the time interval between the autonomous driving unit outputting a control takeover request and the human driver taking over control, e.g. determining the time interval between the control takeover request event and the control takeover event. In other words, monitoring the at least one driving parameter includes measuring the time needed for the human driver to take over control. The time lag determined in this manner may be used to quantify how effective the takeover mechanism implemented by the autonomous driving unit is in getting the required human control and/or how attentive the human driver-to-be is. In particular, by determining the driving quantity based on the time interval, the intertwined impact of both hand over alert design and driver alertness may be captured. In this case, the driving quantity may also referred to as “time lag metric”.

In another preferred embodiment, the at least one driving quantity is determined based on the time interval between the autonomous driving unit outputting a control takeover request and the human driver taking over control, and a vehicle dynamic variable during the time interval. In particular, determining the at least one driving quantity may include weighting the time interval between the autonomous driving unit outputting a control takeover request and the human driver taking over control with the vehicle dynamic variable. In other words the time lag metric then includes a vehicle dynamic variable. By this means, the impact of the time lag between the control takeover request event and the control takeover event, during which period the control of the autonomous driving unit may be insufficient or at least suffer from the incapability of the autonomous driving unit to comprehensively evaluate the complex traffic scenario and/or the ineffectiveness of the method of signaling the driver for take-over, on the rating of the autonomous driving unit may be easily adapted to reflect the safety of the vehicle during this period.

Preferably, the at least one driving quantity determined based on the time interval and the vehicle dynamic variable is compared to driving quantities determined previously for the same autonomous driving unit and/or the same driver, in particular in similar environments and/or traffic scenarios. If, for example, this comparison shows that previously better time-lag values or earlier control takeover timings of the driver were determined, a statistically or otherwise determined significantly extended takeover time-lag may be interpreted as suspicious and likely due to driver inattention. Analyzing the driving quantity determined based on the time interval and the vehicle dynamic variable may thus be helpful in pinpointing deficiencies e.g. in the signaling and eliminate them accordingly. Further, the analysis may also be suitable to assess liability e.g. in case of an accident during the second stage of the change in driving mode.

A vehicle dynamic variable can, for example, be the vehicle speed, the acceleration, the deceleration, the yaw rate, the rotational frequency, and/or the like. Because during the time interval from the control takeover request event to the control takeover event the vehicle dynamic variable can change, e.g. by acceleration of the vehicle, the average of the vehicle dynamic variable may be used to determine the at least one driving quantity, in particular to weight the determined time interval.

In an example, the time interval or duration for the human driver to take over control after being requested so by the autonomous driving unit is multiplied by the average vehicle speed during this period. Weighting the time interval with the (average) vehicle speed effectively indicates a distance the vehicle travels in between the control takeover request event and the control takeover event. Hence, the magnitude of the time lag has a radically different impact on the driving quantity if the same time lag occurs at 120 km/h or at 20 km/h.

In particular, the at least one driving quantity may be determined based on a distribution or at least a part of a distribution of the time interval weighted by the dynamic driving variable, wherein the distribution is preferably generated by monitoring the time interval and the dynamic driving variable during a driving period, e.g. a plurality of journeys of the vehicle. Preferably, the distribution or at least a part of the distribution may be mathematically processed to obtain the at least one driving quantity, for example by averaging, determining the median and/or the like.

The time lag metric obtained this way may be easily ranked, e.g. by comparing it to an average time lag metric obtained from multiple different vehicles with autonomous driving units and different drivers.

In another preferred embodiment, the at least one driving quantity is determined based on the product of the time interval and the vehicle dynamic variable during the time interval, wherein the product reaches or exceeds a, in particular predefined, threshold. In particular, the weighted time interval is evaluated, i.e. processed for determining the at least one driving quantity, only if it reaches or exceeds a predefined threshold. By this means, it is possible to focus on control changes associated with a high accident risk, e.g. where it takes the human driver particularly long to take over control from the autonomous driving unit and/or where the autonomous driving unit transfers control to the human driver at high speeds.

For example, it is possible to only consider, i.e. evaluate, only those transfers of control, i.e. those second phases, in which the product of the vehicle dynamic variable and the time interval, in particular the weighted time interval, reaches or exceeds an average time lag metric obtained from multiple different vehicles with autonomous driving units and different drivers. It is particularly preferable to only consider those transfers of control in which the product of the vehicle dynamic variable and the time interval, in particular the weighted time interval, lies in the fourth quartile. Given that during a driving period, not too many changes of control are to be expected, considering only e.g. the 25% worst cases or scores, respectively, allows to focus on the most meaningful cases in terms of causation of AI assisted driving accident events.

In another preferred embodiment, the driving values obtained by the monitoring are stored for a predetermined storage time interval spanning at least from a point in time before a control takeover request is outputted by the autonomous driving unit to the point in time the autonomous driving unit outputs the control takeover request. Preferably, the predetermined storage time interval spans at least from the point in time before a control takeover request is outputted to the point in time the human driver takes control. This not only allows to determine the quality metric retrospectively, i.e. with the knowledge that the change in driving mode occurred, but the driving values stored prior to the control takeover request event can also be evaluated in view of the deterioration of the autonomous driving when e.g. a complex traffic scenario comes up.

For example, the stored driving values or the at least one driving quantity derived thereof, respectively, can be used to check whether the driving stabilizes e.g. after the control takeover request was outputted or whether it deteriorates further. From this analysis, the liability e.g. in case on an accident may be assessed. Further, this analysis may assist in pinpointing deficiencies e.g. in the software of the autonomous driving unit and enhance it accordingly.

In particular, by storing driving values already obtained before the control takeover request event, e.g. in the first stage, information is available on the driving behavior of the autonomous driving unit leading into the complex traffic scenario which the autonomous driving unit may not be able to safely resolve. Preferably, this information is considered in the determination of the at least one driving quantity, in particular in a driving quantity quantifying the quality of driving of the autonomous driving unit during the second stage, e.g. during the transfer of control. This allows a particularly valuable insight into the capability of the autonomous driving unit to assess complex traffic scenarios and interact accordingly with the driver.

For example, the at least one driving quantity determined from the stored driving values is compared to a driving quantity quantifying the unimpaired driving of the autonomous driving unit during the first stage. By this it can be tested whether the quantity determined from the stored driving values differs significantly from a previously determined quantity for the autonomous driving prior to the predetermined storage time interval, e.g. whether a difference between these two exceeds a predefined threshold. Thus, it can be assessed whether or how much earlier the control takeover request could have been outputted. Alternatively or additionally, the observed or measured lag between a significant deviation of the at least one driving quantity determined based on the stored driving values and quantity determined during the first stage can be compared with that observed from other autonomous navigation solution providers. In other words, this allows to determine systematically whether there is an earlier point in time to request control of the driver, because a degradation in autonomous driving was accessible earlier on. That information is may be used to enhance the autonomous driving unit. Further, that information may be helpful for a forensic understanding e.g. of an accident.

To ensure sufficient information regarding the driving behavior of the autonomous driving unit prior to the control takeover request event, the predetermined storage time interval may extend for at least one minute, preferably at least two minutes, in particular at least four minutes. Advantageously, the driving values are stored for at least 10 minutes. This allows for recording sufficient data, i.e. driving values, to capture dynamic events leading up to the autonomous driving unit outputting the control takeover request and/or transpiring during the second stage, i.e. before the control takeover event. This enables critical insight into this highly accident-prone period.

Preferably, the driving values are stored in a storage unit, e.g. a data buffer, which may advantageously be a centralized storage unit and accessible via a network, e.g. the Internet. To this end, the vehicles, in particular the autonomous driving unit, may be connected to the storage unit via the network. Alternatively, the vehicle, in particular the autonomous driving unit, may contain the storage unit which is configured to accumulate data which may then be evaluated e.g. after the current driving period.

The storage unit is preferably configured as a FIFO (first in, first out) buffer, in particular circular buffer. From the storage unit, the data, e.g. the driving values, can then be incrementally extracted to determine the at least one driving quantity.

In another preferred embodiment, the at least one driving quantity is a composite quantity. Preferably, the at least one driving quantity is determined based on driving quantities quantifying the quality of driving, in particular the driving behavior, of both the autonomous driving unit and the human driver. In particular, the at least one driving quantity may be determined based on an autonomous driving quantity quantifying the quality of driving of the autonomous driving unit during the first and/or second stage of the change in driving mode, in particular during a first time interval spanning from a point in time prior to the autonomous driving unit outputting a control takeover request event to the point in time the request is outputted, and/or on a manual driving quantity quantifying the quality of driving of the human driver during the third and/or fourth stage of the changing control. Considering the driving behavior during the different stages, e.g. of both the autonomous driving unit and the human driver, may provide a particularly precise assessment of the interplay between autonomous driving unit and human driver and thus the overall combined driving of the vehicle.

Additionally or alternatively, the at least one driving quantity is determined based on driving values obtained by monitoring more than one driving parameter. By this means, different driving situations may be the assessed adequately. For example, the at least one driving quantity may be determined based on vehicle speed and acceleration or deceleration, respectively. Given that strong acceleration can occur both at low speeds, e.g. when resuming driving after stopping at a traffic light which can be associated with safe driving, and at high speeds, e.g. when overtaking on rural road which can be associated with risky driving, this allows for a particularly nuanced rating of the overall driving quality.

In another preferred embodiment, the at least one driving quantity is determined based on an autonomous driving quantity quantifying the quality of the driving of the autonomous driving unit before the change in driving mode and on a manual driving quantity quantifying the quality of the driving of the human driver after the change in driving mode. In particular, the at least one driving quantity is determined based on an autonomous driving quantity during the first stage of the change in driving mode, i.e. before the control takeover request event, and on a manual driving quantity during the fourth stage of the change in driving mode, i.e. well after the control takeover event. By this means, the at least one driving quantity may represent the overall quality of driving of the vehicle in a particularly straightforward fashion and allows e.g. for a comprehensive and global rating.

Preferably, the at least one driving quantity is a mathematical function of the autonomous driving quantity and the manual driving quantity. The mathematical function may be a statistical description of the distribution of driving quality values obtained from e.g. a comparison of driving values with a predetermined value, e.g. a statutory speed limit or an average speed. Preferably, the mathematical function is based on one or more statistical moments of this distribution, in particular the variance. Using a statistical description for determining the at least one driving quantity may provide a particular robust and reliable rating of the overall or combined driving behavior of both autonomous driving unit and human driver.

In another preferred embodiment, the at least one driving quantity is further based on, in particular relative, mileage accumulated in the respective driving mode. In particular, the autonomous driving quantity and the manual driving quantity may be weighted by the mileage, in particular a ratio of the mileages. This enables robust scaling of the at least one driving quantity with e.g. the preference of the human driver to operate the vehicle predominantly in manual or autonomous driving mode.

In another preferred embodiment, the at least one driving quantity is determined based on an autonomous driving quantity quantifying the quality of the driving of the autonomous driving unit after a control takeover request is outputted by the autonomous driving unit and on a manual driving quantity quantifying the quality of the driving of the human driver subsequently to him taking over control. In other words, the at least one driving quantity is determined based on an autonomous driving quantity during the second stage of the change in driving mode, i.e. after the control takeover request event and before the control takeover event, and on a manual driving quantity during the third stage of the change in driving mode, i.e. after the control takeover event and before the human driver has adjusted to the current traffic scenario. In particular, the at least one driving quantity is determined based on a manual driving quantity quantifying the quality of manual driving during the predetermined second time interval subsequently to the human driver taking over control. This allows to closely monitor the dynamics of control takeover, in particular with focusing on the man-machine interaction between the autonomous driving unit and human driver, e.g. on the impact of the autonomous driving prior to complex traffic scenarios on the subsequent manual driving, and accordingly rate the entity of autonomous driving unit and human driver in order to pinpointing deficiencies e.g. in the man-machine interaction and enhance them accordingly. Further, its imaginable to utilize such rating for a user based insurance.

In another preferred embodiment, at least one driving quantity is further based on accident rate values for autonomous driving or manual driving after a takeover control request is outputted by the autonomous driving unit, in particular during the second stage of the change in driving mode, and subsequently to the human driver taking over control, in particular during the third stage of the change in driving mode, respectively. In particular, the autonomous driving quantity and the manual driving quantity may be weighted by the respective accident rate values. This allows for a particularly nuanced rating of the combined autonomous and manual driving of the vehicle during the change in driving mode.

Taking into account the respective accident rate values, i.e. the nominal percentage of accidents under autonomous control during the second stage and the nominal percentage of accident under manual control during the third stage, instead of e.g. the mileage accumulated during these stages may ensure sufficient computational impact, in particular a realistic scaling, on the determined driving quality.

The accident rate value for manual control, i.e. during the third stage, can e.g. be determined from insurer records. The accident rate value for autonomous control, i.e. during the second stage, may be determined based on an association of accident rate values for manual driving to an according autonomous driving. In particular, accident rate values obtained for a particular driving quality of manual driving may be associated with autonomous driving of a corresponding driving quality. To this end, an autonomous driving quantity may be associated to a plurality of manual driving quantities which have been determined e.g. from the same driving parameter during manual driving periods of different drivers such that from accident rate values associated to those manual driving quantities an autonomous driving accident rate value may be determined.

Alternatively, the accident rate value for autonomous control may be determined as a nominal performance parameter from Autonomous Navigation Solution Providers, at least as long no reliable insurer records in this regard exist. Additionally, this value may be updated as the percentage of autonomously driven vehicles increases and credible records start to emerge.

In another preferred embodiment, the at least one driving quantity is determined based on driving values obtained by the monitoring during a predetermined first time interval prior to the autonomous driving unit outputting a control takeover request. In other words, data indicative of the driving behavior of the autonomous driving unit immediately prior to the control takeover request event, i.e. during a part of the first stage, may be taken into account. In particular, this data may be used to determine the autonomous driving quantity quantifying the autonomous driving during the second stage of the change in driving mode, preferably in addition to data obtained during the second stage. By this means, the capability of the autonomous driving unit to assess traffic scenarios and recognize potentially overtaxing situations may be rated, potentially allowing to pinpoint and/or eliminate deficiencies e.g. in the software of the autonomous driving unit may or even determine an insurance premium.

Preferably, the predetermined first time interval corresponds to the predetermined storage time interval or at least part of it. By this means, the at least one driving quantity may be retrospectively determined, e.g. even after the change in driving mode occurred. Also, this allows to analyze the deterioration of the autonomous driving prior to a control takeover request being outputted.

In another preferred embodiment, the at least one determined driving quantity is interrelated to a quantity distribution of the same driving quantity. The quantities forming the quantity distribution are preferably obtained for a plurality of other vehicles, in particular reference vehicles, with an autonomous driving unit and different human drivers. Therein, the autonomous driving units of the other vehicles may be the same or those of other autonomous driving solution providers. Alternatively, the quantities forming the quantity distribution may be obtained from records for the same vehicle, for example from a databank which contains all driving quantities determined for that vehicle e.g. during a superior driving period. This allows to assess relative performance, in particular relative to other vehicles driven by different drivers and/or autonomous driving units, or relative to a usual driving behavior, i.e. driving style, of the present autonomous driving unit and/or driver.

In particular, the at least one determined driving quantity may be compared to a property of the quantity distribution, e.g. the average, the median or another mathematical measure characterizing the quantity distribution. This interrelation, in particular comparison, may be advantageously used to investigate the role of autonomous driving or human driving in the case of accidents, respectively. This may assist in pinpointing deficiencies e.g. in the software of the autonomous driving unit and enhance it accordingly. Further, this information may prove invaluable for e.g. insurance companies, as it might allow to determine whether liability lies with the autonomous driving unit or its manufacturer, respectively, or the human driver.

Preferably, interrelating the at least one driving quantity with the quantity distribution is part of a null hypothesis testing, wherein the null hypothesis may assume e.g. responsibility for an accident lies with the autonomous driving unit.

For example, if an accident occurs after the control takeover request event, the human driver may be held responsible if the time lag metric is larger than an average time lag metric obtained from a plurality of other vehicles or their own prior response characteristics. Alternatively or additionally, if the time lag metric is less than the average time lag metric, the autonomous driving unit may be held responsible. In this regard, “less than” or “more than” or “equal” can be assess in a statistical or other manner.

Additionally, it may be possible to determine a partial responsibility based on the magnitude of difference between the determined driving quantity and the corresponding average obtained from other vehicles. In other words, the more the determined driving quantity deviates from the norm defined by the average (or the median or another mathematical measure characterizing the quantity distribution, for that matter), e.g. relative to global or personal driver related norms, the more responsibility may be assigned to the autonomous driving unit or the human driver, respectively.

In another preferred embodiment, the at least one driving quantity is determined based on a quantity distribution, the quantity distribution being determined based on driving value data provided by a plurality of vehicles. Therein, the driving value data may characterize the driving behavior of the plurality of vehicles, in particular of autonomous driving units and/or human drivers thereof. Accordingly, the driving value data provided by the plurality of vehicles may be used for benchmarking the driving behavior of the autonomous driving unit and/or the human driver of the vehicle. Therefore, the plurality of vehicles may also be referred to as reference vehicles.

In other words, by determining the at least one driving quantity on the basis of the quantity distribution, the determined at least one driving quantity can correspond to a measure of the quality of the driving of the autonomous driving unit and/or the human driver of the vehicle with respect to the driving of autonomous driving units and/or human drivers of the plurality of (reference) vehicles.

Preferably, for determining the at least one driving quantity on the basis of the quantity distribution, the quantity distribution or the corresponding driving value data is read from a databank. The driving behavior of the autonomous driving unit and/or the human driver of the vehicle can then be compared to the average driving behavior of the driving units and/or human drivers of the reference vehicles, i.e. to a norm, in order to derive the at least one driving quantity.

If such databank does not exist, the method may comprise accumulating the driving value data. In particular, the driving value data may be obtained from monitoring at least one driving parameter of the plurality of vehicles during a predetermined driving period. In other words, driving quality metrics, based on which the quantity distribution may be derived or computed, respectively, may be compiled from a set of reference vehicles.

Therein, the reference vehicles or their drivers, respectively, may be selected from vehicles or drivers listed with an insurance company providing conventional automobile insurance, i.e. where premiums are fixed e.g. based on vehicle type, drivers age/driving experience, and/or the like. To reward such selected drivers for providing the driving values of their vehicles or allowing for a monitoring of the driving parameters of their vehicles, respectively, some inducement like a fee per mile driven may be offered.

Preferably, the reference vehicles are selected from vehicles or their drivers, respectively, who have mid-range to very good driving liability records with the insurance company. In other words, the driving value data may be obtained from reference vehicles based on the liability history with an insurance company. That is because it can be assumed that the autonomous driving unit of the vehicle will at least match the driving behavior of average drivers with respect to safety or accident risk, respectively.

In another preferred embodiment, the driving value data is generated by driving value statistic units provided in each of the plurality of vehicles. Preferably, the driving value statistic units monitor at least one driving parameter of each of the plurality of vehicles and transmit corresponding driving values to a server. The server may then compile the databank from which the quantity distribution can be read or derived, respectively.

The driving value statistic units, for example GPS sensors which are capable of determining, in particular tracking, the current position (and therefore speed, acceleration and/or deceleration), may be provided for reference vehicles or their drivers, respectively, selected from vehicles or drivers registered with the insurance company providing conventional automobile insurance. Alternatively or additionally, such driving value statistic units may be provided for vehicles or their drivers, respectively, who are willing to install these units in the vehicles. Such vehicles or drivers may be found for example through the media, e.g. social media. In this case, it is conceivable that these drivers need to fill in pertinent driver record details, for example accident record, mileage driven per year, and/or the like, in order to filter good candidates for compilation of a driving quality metric/driving value databank. These details may be, at least if an according waiver is provided, verified by their respective automobile insurer.

Preferably, the driving value statistic units are configured as a dongle, e.g. in form of an USB device, which may be connected to a control system, e.g. an engine control unit (ECU), of the vehicle, in order to determine and/or transmit the driving values of the at least one monitored driving parameter. Alternatively, the driving value statistic units are integral with the vehicle, i.e. provided as a built-in unit or a software module, respectively, and configured to determine and/or transmit the driving values of the at least one monitored driving parameter. In particular, the driving value statistics units may be provided by installing such a software module or performing a software update, respectively, such that control systems such as ECUs of the plurality of vehicles are able to determine and/or transmit the driving values of the at least one monitored driving parameter.

In another preferred embodiment, monitoring at least one driving parameter includes monitoring at least the following: (i) vehicle speed; (ii) vehicle acceleration; and (iii) vehicle deceleration. Preferably, these driving parameters may be combined and/or considered for a defined time interval to form other driving parameters, for example erratic driving which is described by the frequency of acceleration and deceleration in a defined time interval and wherein the corresponding driving quantity is indicative of e.g. predictive driving.

In particular, the at least one driving quality may be determined as follows:

a) During operation of the vehicle the driving speed of the vehicle is determined a plurality of times, e.g. regularly or at random instants of time. The driving speed is thus the monitored driving parameter.

b) The location of the vehicle is determined for those time points where the speed is or was determined.

c) Each vehicle speed value is compared with a speed value associated to the respective locations. Such associated speed value can be the statutory speed or prevailing speed at those locations at the time point at which the vehicle drove there.

d) Then a speed difference between the driving speed value and the associated speed value is determined. This speed difference can be seen as a resulting value.

e) From a plurality of such resulting values a speed quantity is determined, e.g. by averaging the resulting values or by forming the median of those resulting values. A speed difference is a resulting value, wherein from a plurality of such resulting values determined during a driving period, a driving quality value is determined, e.g. by averaging or other indicative valuation technique. Eventually, from a plurality of such driving quality values, determined e.g. over a month or three months, a driving quantity or driving quality metric is determined.

Alternatively or additionally, the at least one driving quantity is determined as follows:

a) During operation of the vehicle a positive acceleration of the vehicle, i.e. a speedup acceleration, is detected by an acceleration sensor in the vehicle. The forward acceleration is thus the monitored driving parameter.

b) If the vehicle acceleration is above a predetermined threshold, a maximum speed is stored as an acceleration event value. The maximum speed can occur during or without deceleration after an acceleration event in which the vehicle acceleration is above a predetermined threshold.
c) The acceleration event value is compared with a geostationary speed value of the place where the acceleration event value occurred. The geostationary speed value can be the statutory speed or prevailing speed at the location, area or locale where the acceleration event occurred.
d) This speed difference is a resulting value analog to the speed metric description.
e) From a plurality of such resulting values an acceleration quantity is determined, e.g. by averaging the resulting values or by forming the median of those resulting values.

Alternatively or additionally, the at least one driving quantity is determined as follows:

a) During operation of the vehicle a deceleration of the vehicle is detected by an acceleration sensor in the vehicle. In the following, the terms deceleration and negative acceleration are used analogously.

b) A speed difference between two speeds occurring at the beginning and at the end of an acceleration event, in which the vehicle deceleration is above a predetermined threshold, thus stronger than the threshold, is determined.

c) From this speed difference a resulting value is determined. The resulting value may be the speed difference as such, or the speed difference divided by a vehicle speed. The vehicle speed can be the speed at the beginning or end of the respective acceleration event, or it is the average speed over the acceleration event. The resulting value is stored as an acceleration event value.
d) From a plurality of such resulting values a deceleration quantity is determined, e.g. by averaging the resulting values or by forming the median of those resulting values or other indicative measures.

Alternatively or additionally, the at least one driving quantity is determined as follows:

a) During operation of the vehicle a positive and negative acceleration of the vehicle, each above a predetermined threshold, are detected by an acceleration sensor in the vehicle.

b) A time span between the positive and negative acceleration is stored as an erratic driving event value, or resulting value, respectively.

c) From a plurality of such resulting values an erratic driving quantity or erratic driving metric value is determined, e.g. by averaging the resulting values or by forming the median of those resulting values.

A system for analyzing the control of a vehicle comprising an autonomous driving unit according to a second aspect of the invention contains a detection unit configured to detect a change in the driving mode from autonomous driving to manual driving and a sensor unit configured to monitor at least one driving parameter before and/or after the change was detected. Further, an evaluation unit is configured to determine, based on driving values provided by the sensor unit upon monitoring with respect to the detected change in driving mode, at least one driving quantity quantifying the quality of interplay between the autonomous driving unit and a human driver.

The described features and advantages described with respect to the first aspect of the invention and its preferred embodiments are, if not explicitly stated otherwise and at least where technically meaningful, valid also for the second aspect of the invention and its preferred embodiments and vice versa.

Other features which are considered as characteristic for the invention are set forth in the appended claims.

Although the invention is illustrated and described herein as embodied in a method and a system for analyzing the control of a vehicle, it is nevertheless not intended to be limited to the details shown, since various modifications and structural changes may be made therein without departing from the spirit of the invention and within the scope and range of equivalents of the claims.

The construction and method of operation of the invention, however, together with additional objects and advantages thereof will be best understood from the following description of specific embodiments when read in connection with the accompanying drawings.

