TECHNICAL FIELD

The invention relates to a computing device for authenticating a user of the computing device, a method of authenticating a user of the computing device, a corresponding computer program, and a corresponding computer program product.

BACKGROUND

Known solutions for authenticating a user, e.g., for granting access to a computing device such as a mobile phone, or to a service such as online banking, oftentimes rely on shared secrets such as passwords. These solutions suffer from various security weaknesses including data breaches and are susceptible to brute-force guessing. Moreover, passwords are easily forgotten by the user, which introduces additional security issues including repeated use of the same passwords, use of easy-to-remember passwords which are based on information related to the user, such as the partner's name or the user's date of birth, insecure storage of passwords, and problems arising from the use of password-reset or -recovery solutions. The rise of small and simple keyboards on smartphones and portable devices also reinforces the use of simple passwords, as users favor brevity due to an increased effort in typing on small keyboards.

Alternative solutions which rely on biometric information, such as fingerprint sensors and iris scanners, have the advantage that users do not need to remember any password or other data. Instead, such solutions rely on unique biological information of the user, e.g., fingerprints or iris patterns, which are matched against stored reference information. However, such solutions are vulnerable to data breaches and have the disadvantage that, once the biometric data has been compromised, it is difficult to change and the compromised biometric information can accordingly no longer be used for authentication. In addition, some biometric information may also be stolen or faked, e.g., by means of a voice recording or a face mask for authentication schemes based on voice or face recognition, respectively.

Also known are authentication schemes which are based on recognition of songs by the user (see, e.g., “Musipass: authenticating me softly with “my” song”, by M. Gibson, K. Renaud, M. Conrad, and C. Maple, Proceedings of the 2009 workshop on New security paradigms (NSPW '09), pages 85-100, ACM New York, 2009).

SUMMARY

It is an object of the invention to provide an improved alternative to the above techniques and prior art.

More specifically, it is an object of the invention to provide improved solutions for authenticating a user of a computing device, such as a mobile phone, a mobile terminal, a smartphone, a tablet, a personal computer, a computer display, a television, a media player, or a laptop. In particular, it is an object of the invention to provide improved solutions for authenticating a user based on a melody generated by the user.

These and other objects of the invention are achieved by means of different aspects of the invention, as defined by the independent claims. Embodiments of the invention are characterized by the dependent claims.

According to a first aspect of the invention, a computing device for authenticating a user of the computing device is provided. The computing device comprises processing means operative to acquire a representation of a melody generated by the user, and authenticate the user in response to determining that the acquired representation of the melody generated by the user and a representation of a reference melody fulfil a similarity condition.

According to a second aspect of the invention, a method of authenticating a user of a computing device is provided. The method is performed by the computing device and comprises acquiring a representation of a melody generated by the user, and authenticating the user in response to determining that the acquired representation of the melody generated by the user and a representation of a reference melody fulfil a similarity condition.

According to a third aspect of the invention, a computer program is provided. The computer program comprises computer-executable instructions for causing a device to perform the method according to an embodiment of the second aspect of the invention, when the computer-executable instructions are executed on a processing unit comprised in the device.

According to a fourth aspect of the invention, a computer program product is provided. The computer program product comprises a computer-readable storage medium which has the computer program according to the third aspect of the invention embodied therein.

The invention makes use of an understanding that music, e.g., in the form of sung songs, or hummed or whistled melodies, offer a number of advantages over passwords and other conventional user authentication methods. This is the case since people tend of have a large internal database of known songs/melodies which they are regularly adding to. The number of songs/melodies a person can readily remember is large and varied, meaning that their use as a response in a challenge-response authentication scheme leads to a solution which is relatively difficult to crack using brute-force methods.

To this end, the user is authenticated if it is determined that a representation of a melody generated by the user, which may either be vocalized, e.g., by singing, humming, or whistling, or subvocalized, is sufficiently similar to a representation of a reference melody. The user may either be authenticated to access the computing device, or to access a separate device or a service. In the latter scenarios, an embodiment of the computing device is used to facilitate authentication to another device or a service in a multi-factor authentication scheme.

There exist a variety of definitions for what a “melody” is:The succession of single tones in musical compositions, as distinguished from harmony and rhythm.The principal part in a harmonic composition.The “strongest” pitch (i.e., frequency) in the likely melody range at a given time.The single (monophonic) pitch sequence that a listener might reproduce if asked to whistle or hum a piece of polyphonic music, and which the listener would recognize as being the “essence” of that music when heard in comparison.

For the purpose of elucidating embodiments of the invention, the latter definition is adopted throughout this disclosure.

Since it is clear that what constitutes the actual melody of a piece of music is somewhat subjective, two people, or users of a computing device, may have different views as to what the essence of a piece of music is. For instance, two sections of a melody may have a joining sequence, which some may consider to be an essential part of the melody, and others not. Considering the song “Yesterday” by The Beatles as an example, some people may consider the melody to be “Yesterday . . . all my troubles seemed so far away”, whilst others might consider it to be “Yesterday da dum all my troubles seemed so far away”. It will also be appreciated that rhythm, in addition to pitch, may be considered in determining that an acquired representation of a user-generated melody and a representation of a reference melody fulfil a similarity condition.

Hence, any scheme for comparing and judging whether a user-generated melody has been reproduced sufficiently accurately in comparison with the reference melody is to be based on a confidence score, or degree of similarity, rather than an absolute comparison. In practice, this means that the similarity condition is considered fulfilled if the user-generated melody and the reference melody are in sufficient agreement, as is described in further detail below. This is similar to authentication schemes based on biometric authentication, e.g., involving fingerprints, voice recognition, or iris patterns, which all rely on comparing a sample obtained from the user during the authentication process, and assessing whether the obtained sample is in agreement with a reference sample to sufficient extent, oftentimes defined by a threshold value to which a measure of similarity is compared. For instance, this may be achieved by identifying characteristic features in the user sample (user-generated melody) and the reference sample (reference melody), and assessing whether a sufficiently large fraction of characteristic features is present in both samples.

In general, embodiments of the invention are not limited to melodies of what is considered to be a piece of music or a song. Rather, a melody may also be present in any utterance which a person generates, either vocalized or subvocalized, e.g., by speaking or thinking a word or a sentence.

In the present context, a representation of a melody is considered to be any type of data representing a melody, either generated by the user or a stored reference melody, and which may be used for comparison with a corresponding representation of another melody. Such comparison may, e.g., be based on the melody itself, a signature of the melody, a pitch sequence or pitch contours of the melody, a hash of the melody, or the like, as is elucidated further below. Depending on the type of representation, comparison can be made between different types of melodies, including original music, i.e., a recorded song which may be stored, downloaded, or streamed, vocalized melodies which are audibly generated by the user, e.g., by singing, humming or whistling, and melodies which are subvocalized by the user.

Subvocalization is the “internal speech” while reading or listening to music and is characterized by minute movements of the larynx and other speech muscles, as well as other body parts, which are imperceptible to the human eye but detectable by technical means.

According to an embodiment of the invention, the representation of the melody generated by the user is acquired by acquiring audio data of the user vocalizing the melody, i.e., audibly generating the melody, e.g., by singing, humming, or whistling, and deriving a representation of the vocalized melody as a representation of the acquired audio data. The audio data may be acquired from a microphone operatively connected to the computing device, such as a built-in microphone or an external microphone, e.g., a headset, which is connected to the computing device.

According to a further embodiment of the invention, the representation of the melody generated by the user is acquired by acquiring nerve signals captured from a throat of the user subvocalizing the melody, and deriving a representation of the subvocalized melody as a representation of the nerve signals. The nerve signals are acquired from one or more electromyographical sensors which are operatively connected to the computing device via an interface. In this context, subvocalization is used for capturing a melody which the user is thinking of, or sings internally. As a person internally sings a melody, without vocalizing it audibly, they still make a variety of internal and external motions which are triggered by nerve signals controlling the muscles of the person. Both the nerve signals and the resulting motions, which typically are imperceptible to the human eye, are correlated with the melody and/or rhythm of the subvocalized music, and can be detected. These motions include muscle movements which are related to the ones the person would make to audibly create the sounds, such as the movement of vocal chords, and may also include associated movements the person would make, e.g., sway due to the music, modify breathing patterns, and so forth. Examples of these motions are motions of vocal chords, lips, tongue, jaw, neck, other body parts such as heads and shoulders, and motions related to breathing.

According to another embodiment of the invention, the representation of the melody generated by the user is acquired by acquiring a video sequence of the user subvocalizing the melody, magnifying motions of one or more body parts of the user, which motions are correlated with the subvocalized melody, and deriving a representation of the subvocalized melody as a representation of the magnified motions, e.g., using one or more metrics quantifying the motions of the lips, throat, or other body parts, of the user. The video sequence is acquired from a camera operatively connected to the computing device, e.g., a camera which is comprised in the computing device, such as a front-facing camera in a smartphone or tablet. Alternatively, an external camera, such as a web cam, may be connected to the computing device via an interface. The camera captures motions of one or more body parts of the user, such as the user's lips, throat, mouth, nose, face, neck, or shoulders, which motions are correlated with the subvocalized melody. The motions may, e.g., be magnified by means of Eulerian Video Magnification (EVM) or Lagrangian techniques which are described in further detail below. Acquiring the representation of the melody subvocalized by the user by means of a video capturing the user subvocalizing the melody, and processing the video to magnify motions which are correlated with the subvocalized melody, is an alternative to utilizing electromyographical sensors for capturing nerve signals.

Advantageously, embodiments of the invention which rely on authentication based on a melody subvocalized by the user enable a silent user authentication technique. Thereby, potential privacy issues are addressed where the entering of a password or audible exchange of information, such as audibly vocalizing a melody or a password, may present a risk.

In practice, a representation of a melody may, e.g., be an analog or digitized signal capturing a melody in the form of an audio signal (if the melody is vocalized or an original piece of music), motions of the user's body parts (if the melody is subvocalized and captured by means of video), or muscle activity of the user's body parts (if the melody is subvocalized and captured by means of electromyographical sensors).

According to an embodiment of the invention, the processing means is operative to determine that the acquired representation of the melody generated by the user and the representation of the reference melody fulfil a similarity condition by calculating a degree of similarity between the acquired representation of the melody generated by the user and the representation of the reference melody. Preferably, the two representations are of the same type, or at least compatible, so as to facilitate calculating a degree of similarity. The processing means is further operative to determine that the acquired representation of the melody generated by the user and the representation of the reference melody fulfil a similarity condition if the calculated degree of similarity exceeds a threshold value.

According to another embodiment of the invention, the processing means is operative to determine that the acquired representation of the melody generated by the user and the representation of the reference melody fulfil a similarity condition by deriving a melody signature from the acquired representation of the melody generated by the user, and calculating a degree of similarity between the melody signature derived from the melody generated by the user and a melody signature derived from the representation of the reference melody. This is particularly advantageous if the two representations are of different types and cannot be directly compared, e.g., if the melody generated by the user is subvocalized, and a representation of motions of the user's body parts, or nerve signals, are compared to a recorded original song. The melody signature may, e.g., be obtained based on a hashed time-frequency constellation analysis of a melody, aka as a “fingerprint” of a song, or by categorizing the pitch sequence of a melody in terms of pitch contours, as is explained in further detail below. The processing mans is further operative to determine that the acquired representation of the melody generated by the user and the representation of the reference melody fulfil a similarity condition if the calculated degree of similarity exceeds a threshold value.

The threshold values used in evaluating the similarity condition may, e.g., be generic threshold values which may be set by the user of the computing device, a manufacturer of the computing device, or by a provider of a software or a service relying on the authentication. Optionally, the threshold values may depend on a required level of security associated with the authentication, where a higher value of similarity is required for an increased level of security. For instance, a company may require a higher level of security for computing devices which is used by its employees, whereas a privately used device may be configured with a threshold level corresponding to a lower security level.

According to an embodiment of the invention, the representation of the reference melody is a previously acquired representation of the reference melody generated by the user. That is, the reference melody is recorded by the user for the purpose of authentication. This is similar to setting a conventional password or PIN code, which is typically done during configuring a computing device, such as a smartphone, before first use, or when enrolling for a service provided over the Internet, such as online banking or a music streaming service.

According to another embodiment of the invention, the reference melody is a piece of original music, i.e., a recoding, stored or retrieved in a digital format, of a melody or song. The reference melody may, e.g., be stored on the computing device as part of a music collection, or alternatively be part of a playlist. In the latter case, the reference melody may be retrieved, i.e., downloaded or streamed, by the computing device either prior to or during an authentication attempt.

According to an embodiment of the invention, the computing device further comprises a user-interface, e.g., a display, a loudspeaker, or an interface for a headset or earphones. The processing means is further operative to select the reference melody from a list comprising a plurality of melodies, such as a music collection, a playlist, or a list of reference melodies which is maintained for the purpose of authentication. In addition, the processing means is operative to request the user to generate the selected reference melody. The user is requested to generate the selected reference melody through the user-interface. This may, e.g., be achieved by audibly rendering at least a part of the selected reference melody, or by visually or audibly rendering a name of, or a clue pertaining to, the selected reference melody, through the user interface. For instance, if the reference melody is an original piece of music, i.e., a song, a part of the piece of music, e.g., the first few bars or seconds, may be played to the user as a challenge. Alternatively, the name or title of the selected reference melody may be rendered, either audibly by means of speech synthesis, or visually, as a message displayed on a display, e.g., a touchscreen comprised in the computing device. As a further alternative, rather than providing the user with the selected reference melody or its name or title, a clue may be presented to the user as a challenge to which the user responds by generating the melody. The clue may, e.g., be a picture which the user associates with the melody, such as a picture showing the members of a band performing the song. Alternatively, the clue may be displayed as a text message, such as “Anna's favorite song”, where Anna is a person known to the user.

According to an embodiment of the invention, the reference melody may be selected from a list comprising a plurality of melodies, such as a music collection, a playlist, or a dedicated list of reference melodies, based on any one, or a combination of, one or more metrics. These metrics may, e.g., pertain to a user ranking of the melodies, a popularity of the melodies, a familiarity of the user with the melodies, a rate of successful previous authentication attempts with the melodies, a duration of time since the last time the user has listened to the melodies, or a duration of time since the last successful authentication attempt with the melodies. This is advantageous in that the reference melody may be selected so as to better match the user's preference for certain melodies or songs. For instance, melodies which the user is more likely to remember, or generate with a sufficient accuracy, may be preferred, so as increase the likelihood of a successful authentication attempt.

According to an embodiment of the invention, a melody may be removed from, or added to, such a list based on any one, or a combination of, the one or more metrics listed hereinbefore. This is advantageous in that the set of melodies from which a reference melody is selected for authentication comprises melodies which better match the user's preference for certain melodies or songs. For instance, melodies which the user is more likely to remember, or generate with a sufficient accuracy, may be preferred, so as increase the likelihood of a successful authentication attempt.

Even though advantages of the invention have in some cases been described with reference to embodiments of the first aspect of the invention, corresponding reasoning applies to embodiments of other aspects of the invention.

Further objectives of, features of, and advantages with, the invention will become apparent when studying the following detailed disclosure, the drawings and the appended claims. Those skilled in the art realize that different features of the invention can be combined to create embodiments other than those described in the following.

