COPYRIGHT NOTICE

A portion of the disclosure of this patent document contains material which is subject to copyright protection. The copyright owner has no objection to the facsimile reproduction by anyone of the patent document or the patent disclosure, as it appears in the Patent and Trademark Office patent file or records, but otherwise reserves all copyright rights whatsoever.

BACKGROUND

Biometrics offer the opportunity for identity assurance and identity validation. Many conventional uses for biometrics currently exist for identity and validation. These conventional approaches suffer from many flaws. For example, the IPHONE facial recognition service limits implementation to a one to one match. This limitation is due to the inability to perform one to many searching on the biometric, let alone on a secure encrypted biometric. Other potential issues include faked biometric or replayed biometric signals that can be used to trick many conventional security systems.

SUMMARY

According to one aspect, it is realized that there is a need for a solution that provides one to many searching, and that provides for operations on encrypted biometric information. There is a further need to establish such searches that accomplish one to many matching in polynomial time. Various embodiments of the privacy-enabled biometric system provide for scanning of authentication credentials (e.g., one or more or multiple biometrics and/or one or more user behavioral (e.g., physical or logical) characteristics) to determine matches or closeness. Further embodiments can provide for search and matching across multiple types of encrypted authentication (e.g., biometric or behavioral, among other examples) information improving accuracy of validation over many conventional approaches, while improving the security over the same approaches.

According to another aspect, a private authentication system can invoke multiple authentication methodologies, for example, to speed initial enrollment for users. For example, a distance metric store can be used in an initial enrollment phase, that permits quick establishment of user authentication credentials (e.g., encrypted feature vectors) that can be examined to determine distance between a subsequent encrypted feature vector(s) and encrypted feature vectors in the distance store. Where the distance is within a certain threshold, the user can be authenticated. According to various aspects, authentication credentials can be based on identifying characteristics (e.g., user's fingerprint, retina scan, physical properties, facial characteristics, etc., and may also include physical characteristics of objects or other digitally capturable information of real world objects, things, persons, animals, etc.) and/or behavioral characteristics (e.g., behavior authentication information indicative of at least one of physical behavior, information indicative of at least one logical behavior, among other options). Any authentication credential can be used in conjunction with the first and second neural network architecture disclosed below, and any combination of authentication credentials can be used to identify/authenticate while preserving the privacy of the underlying information.

In various embodiments, the distance store is used as a rough or coarse authentication approach that can be quickly executed for authentication. During the initial authentication phase, a more sophisticated authentication approach can be trained—i.e. a DNN can be trained on encrypted feature vectors (e.g., Euclidean measurable feature vectors, distance measurable feature vectors, homomorphic encrypted feature vectors, etc., which can be derived from any one or more biometric measurement and/or from any one or more behavioral measurement) and identification labels, so that upon input of an encrypted feature vector the DNN can return an identification label (or unknown result, where applicable). According to further aspects, a privacy preserving authentication system can execute hybrid authentication schemes, a fast authentication approach (e.g., distance evaluations of encrypted authentication information (e.g., biometrics and/or behavioral information) coupled with a more robust trained DNN approach that takes longer to establish. Once ready, the system can use either authentication approach (e.g., switch over to the trained DNN approach (e.g., neural network accepts encrypted feature vector as input and returns an identification label or unknown result)). In yet further embodiments, the system is configured to leverage a fast authentication approach for new enrollments and/or updates to authentication information and use, for example, multiple threads for distance authentication and deep learning authentication (e.g., with the trained DNN) once the DNN trained on encrypted feature vectors is ready.

According to another aspect, conventional approaches are significantly burdened not only in that authentication credentials (e.g., biometric data and/or behavioral information) is to be searched in the clear but also by key management overhead that is needed for securing those authentication credentials (e.g., biometrics) in storage. Using APPLE as an example, a secure enclave is provided on the IPHONE with encryption keys only available to the secure enclave such that facial biometrics never leave a respective device or the secure enclave. Various embodiments described herein completely change this paradigm by fully encrypting the reference biometric, and executing comparisons on the encrypted biometrics (e.g., encrypted feature vectors of the biometric).

In further aspects, conventional approaches to passive authentication credential (e.g., biometric and/or behavioral) collection and authentication have been shown to be vulnerable to faked credentials and/or simply not useable for authentication. Some other approaches have attempted to resolve these issues with active authentication (e.g., biometric and/or behavioral) collection, but similar flaws are still present. For example, in gesture based authentication systems, requests are made of a user based on a set of gestures, and the set of gestures itself can become a vulnerability. Even random gesture authentication can be tricked with pre-recorded gestures that are played in response to random requests. The inventors have realized that there is a need for a solution that provides biometric identification coupled with randomized biometric liveness detection. According to one aspect, coupling a liveness factor into identity assurance and validation (e.g., with liveness with biometric identity) resolves problems with conventional security, closing security holes that allow replay or faked biometric signals.

Further embodiments incorporate liveness checks (e.g., with random biometric requests (e.g., voice identification coupled with identification of random words or syllables)) as part of a multi-factor authentication. According to one embodiment, imaging and facial recognition is executed in conjunction with random liveness testing of a separate biometric (e.g., voice identification with random word requests) to complete authentication. In other embodiments, the system can implement random behavioral information checks to determine liveness, and which can be done separately and/or in conjunction with liveness testing of random biometric requests. In still other embodiments, liveness testing/validation is the culmination of many dimensions. For example, liveness determination can be based an ensemble model of many authentication credential dimensions.

In further embodiments, privacy enabled authentication credentials (e.g., biometrics (e.g., privacy enabled facial recognition and/or voice identification)) can be used in conjunction with the liveness augmented authentication. In further embodiments, various authentication systems can incorporate fast enrollment authentication approaches (e.g., compare encrypted values for distance) coupled with neural networks trained on encrypted values (e.g., neural networks that subsequently accept encrypted input to return identification labels (or unknown as a result, wherein appropriate).

According to one aspect, an authentication system can test liveness and test identity using fully encrypted reference authentication credentials (e.g., biometrics and/or behavioral information). According to various embodiments, the system is configured to execute comparisons directly on the encrypted credentials (e.g., biometrics (e.g., encrypted feature vectors of the biometric or encrypted embeddings derived from unencrypted biometrics) and/or behavioral information (e.g., encrypted feature vectors of behavioral measurements)) to determine authenticity with a learning neural network. In further embodiments, one or more first neural networks are used to process unencrypted biometric inputs and/or unencrypted behavioral information and generate distance measurable encrypted feature vectors or encrypted embeddings (e.g., Euclidean measurable encrypted values)—referred to as a generation network. The encrypted feature vectors are used to train a classification deep neural network. Multiple learning networks (e.g., deep neural networks—which can be referred to as classification networks) can be trained and used to predict matches on different types of authentication credential input (e.g., biometric input (e.g., facial/feature biometrics, voice biometrics, health/biologic data biometrics, etc.) and/or user behavioral information inputs/measurements. Typically each authentication credential is processed by its own generation network and its own classification neural network. Although some authentication credentials have enough properties in common that the same type of generation network can be used (e.g., facial recognition uses images as does retinal scans). In various embodiments, the operation of the respective generation network (e.g., outputs encrypted authentication credentials), and the respective classification network (e.g., predicts identity on encrypted authentication inputs) is tailored specifically to an individual authentication credential (e.g., face image, eye image, voice, each user behavioral characteristic (e.g., including physical behavior, and/or logical behavior instances).

In some examples, multiple biometric types can be processed into an authentication system to increase accuracy of identification (and may have associated first and second network pairs for processing each). In another example, a first neural network is used to process user behavioral information inputs and generate distance measurable encrypted feature vectors reflecting the user's behavioral characteristics, which for example can include Euclidean measurable encrypted feature vectors. The output encrypted features vectors can then be used by the system to train a second network on the output of the first network (e.g., distance measurable encrypted feature vectors of biometric and/or behavioral information) with associated labels. Once trained, the second network can be used to determine identity (or unknown) based on an encrypted input generated on user behavioral information.

According to one embodiment, a set of encrypted feature vectors or encrypted embeddings can be derived from any biometric data (e.g., using a first pre-trained neural network) and/or user behavioral information using a corresponding generation network, and then using a corresponding deep neural network (“DNN”) on, for example, the resulting distance measurable encryptions (i.e., each biometrics' feature vector, each biometrics embedding values, or each behavioral information measurement, etc.) a system can determine matches or execute searches on the encrypted data. Each of the behavioral/biometric encrypted feature vectors/embeddings can then be stored and/or used in conjunction with respective classifications for use in subsequent comparisons without fear of compromising the original data. In various embodiments, any unencrypted or original identifying data is discarded responsive to generating the encrypted values, and in some examples, passing validation testing on the encrypted outputs.

According to one embodiment, distance measurable or homomorphic encryption enables computations and comparisons on cypher text without decryption. This improves security over conventional approaches. For example, searching biometrics in the clear on any system, represents a significant security vulnerability. In various examples described herein, only the one-way encrypted biometric data is available on a given device. Various embodiments restrict execution to occur on encrypted biometrics for any matching or searching. In other embodiments, a first phase uses encrypted values to make distance comparisons and authenticate (or not) based on a threshold distance between encrypted values, and a second phase is executed to train a DNN on the encrypted values while the first phase handles authentication. Once ready, the DNN can take over authentication operation. In various implementations, the system can accept or enroll new users by triggering the first phase of operation while the second phase trains at least one DNN on the new authentication information (e.g., encrypted feature vectors).

According to another aspect, an authentication system can also analyze an assurance factor while processing biometric input to ensure that the biometric input is generated by the individual seeking authentication (i.e., not pre-recorded or faked biometric signaling). In some embodiments, the authentication system is configured to request randomly selected instances (e.g., system random selection) of a biometric input or behavioral information (e.g., randomly selected words and/or actions by the user). The system as part of one process can evaluate the received voice information or user action information to determine an identity match, while processing the received voice information or action information to ensure that received voice information matches the randomly selected words. In various embodiments, the authentication system is able to validate that an identity match (e.g., neural network prediction of identity) was supplied at the time requested and by the entity trying to confirm their identity (i.e. liveness testing) based on matching the input to the requested random words. In further embodiments, the system and/or connected devices can collect biometric information of multiple types (e.g., facial features and voice, among other options) to increase accuracy of identity matching, which can be further augmented with liveness detection to prevent spoofing or fraud.

According to one aspect, a privacy-enabled biometric system is provided. The system comprises at least one processor operatively connected to a memory, the at least one processor configured to determine an authentication mode, trigger one or both of a first machine learning (“ML”) process or a second ML process responsive to determining the authentication mode, execute the first ML process, wherein the first ML process when executed by the at least one processor is configured to accept distance measurable encrypted feature vector (e.g., reflective of biometric and/or behavioral measurements) and label inputs during training of a first classification neural network and classify distance measurable encrypted feature vector inputs as part of authentication using the first classification network once trained, execute the second ML process, wherein the second ML process when executed by the at least one processor is configured to accept plain text biometric inputs during training of a generation neural network (e.g., creates encrypted feature vectors) to generate distance measurable encrypted feature vectors, and compare distances between distance measurable encrypted feature vectors during authentication.

According to one embodiment, one of the first ML process or the second ML process is configured to determine one or more distances between encrypted feature vectors produced by the generation neural network, exclude encrypted feature vectors having one or more distances exceeding a threshold distance for subsequent training processes, and include encrypted feature vectors having distances within the threshold distance for subsequent training processes. According to one embodiment, the at least one processor is configured to determine the authentication mode includes an enrollment mode for establishing a new entity (e.g., user, object, behavior, animal, etc.) for subsequent authentication. According to one embodiment, at least one processor is configured to trigger at least the second classification ML process responsive to determining a current authentication mode includes the enrollment mode. According to one embodiment, at least one processor is configured to trigger at least training operations of both the first and second classification ML processes responsive to determining that the current authentication mode includes the enrollment mode.

According to one embodiment, at least one processor is configured to execute the at least part of the second classification process to authenticate the new user until at least a period of time required for training the first classification network expires. According to one embodiment, at least one processor is configured to execute the at least part of the first classification process to authenticate the new user responsive to completing training of the first classification network. According to one embodiment, the first classification network comprises a deep neural network (“DNN”), wherein the DNN is configured to generate an array of values in response to the input of the at least one unclassified encrypted feature vector during authentication, and determine a label or unknown result based on analyzing the generate array of values. According to one embodiment, determining the label or the unknown includes analyzing a position of values within the array and analyzing a respective value at a respective position. According to one embodiment, the embedding network comprises a learning network configured to accept plain text biometric as input and generate distance measurable encrypted feature vectors as output. According to one embodiment, the first classification network is configured to return a label for identification or an unknown result, responsive to input of encrypted feature vector input for authentication. According to one embodiment, at least one processor is configured to determine a probability of match using the first classification neural network is below a threshold value, and validate an unknown result output by the first classification network based on distance analysis of a highest probability match compared to the input feature vectors.

According to one aspect, a computer implemented method for privacy enabled authentication is provided. The method comprises determining, by at least one processor, an authentication mode, triggering, by the at least one processor, one or both of a first machine learning (“ML”) process or a second ML process responsive to determining the authentication mode, executing, by the at least one processor, the first ML process, wherein executing the first ML process includes accepting distance measurable encrypted feature vector and label inputs during training of a first classification neural network and classifying distance measurable encrypted feature vector inputs as part of authentication using the first classification network once trained, executing, by the at least one processor, the second ML process, wherein executing the second ML process includes accepting plain text biometric inputs during training of a generation neural network (e.g., creates encrypted feature vectors) to generate distance measurable encrypted feature vectors, and comparing distances between distance measurable encrypted feature vectors during authentication.

According to one embodiment, the method further comprises determining one or more distances between encrypted feature vectors produced by the generation neural network, excluding encrypted feature vectors having one or more distances exceeding a threshold distance for subsequent training processes, and including encrypted feature vectors having distances within the threshold distance for subsequent training processes. According to one embodiment, the method further comprises determining the authentication mode includes an enrollment mode for establishing a new entity for subsequent authentication. According to one embodiment, the method further comprises triggering at least the second classification ML process responsive to determining a current authentication mode includes the enrollment mode. According to one embodiment, the method further comprises triggering at least training operations of both the first and second classification ML processes responsive to determining that the current authentication mode includes the enrollment mode.

According to one embodiment, the method further comprises executing the at least part of the second classification process to authenticate the new user until at least a period of time required for training the first classification network expires. According to one embodiment, the method further comprises executing the at least part of the first classification process to authenticate the new user responsive to completing training of the first classification network. According to one embodiment, the method further comprises generating, by a deep learning neural network (“DNN”) an array of values in response to the input of the at least one unclassified encrypted feature vector during authentication, and determining a label or unknown result based on analyzing the generate array of values. According to one embodiment, determining the label or the unknown includes analyzing a position of values within the array and analyzing a respective value at a respective position. According to one embodiment, the method further comprises accepting plain text biometric as input and generating distance measurable encrypted feature vectors as output. According to one embodiment, the method further comprises returning a label for identification or an unknown result, responsive to input of encrypted feature vector input for authentication. According to one embodiment, the method further comprises analyzing a user input set of instances of a first biometric data type, and validating an authentication request responsive to determining a match between the user input set of instances and a set of biometric instances randomly generated for the authentication request.

According to one aspect, an authentication system for evaluating privacy-enabled biometrics and validating contemporaneous input of biometrics is provided. The system comprises at least one processor operatively connected to a memory; an interface, executed by the at least one processor configured to: receive a candidate set of instances of a first biometric data type input by a user requesting authentication; a classification component executed by the at least one processor, configured to: analyze a liveness threshold, wherein analyzing the liveness threshold includes processing the candidate set of instances to determine that the candidate set of instances matches a random set of instances; the classification component further comprising at least a first deep neural network (“DNN”), the classification component configured to: accept encrypted feature vectors (e.g., voice feature vectors, etc.), generated from a first neural network, the first neural network configured to process an unencrypted input of the first data type into the encrypted feature vectors; classify with the first DNN the encrypted feature vectors of the first biometric type during training, based on training the first DNN with encrypted feature vector and label inputs; return a label for person identification or an unknown result during prediction responsive to analyzing encrypted feature vectors with the first DNN; and confirm authentication based at least on the label and the liveness threshold.

According to one embodiment, the classification component is configured to: determine for values above the liveness threshold that the input matches the random set of instances; and determine for values below the threshold that a current authentication request is invalid. According to one embodiment, the system further comprises a liveness component, executed by the at least one processor, configured to generate a random set of instances of a first biometric type in response to an authentication request. According to one embodiment, the system is configured to request a user provide the candidate set of instances of the first biometric data type based on the generated random set of instances. According to one embodiment, the interface is configured to prompt user input of the randomly selected instances of the first biometric input to establish a threshold volume of biometric information confirmed at validation.

According to one embodiment, the classification component further comprises at least a second deep neural network (“DNN”) configured to: accept encrypted feature vectors (e.g., face feature vectors, etc.), generated from a second neural network, the second neural network configured to process an unencrypted input of the second data type into the encrypted feature vectors; return a label for person identification or an unknown result during prediction responsive to analyzing encrypted feature vectors; and wherein the classification component is configured to confirm identification based on matching the label for person identification from the first and second DNNs.

According to one embodiment, the second DNN is configured to classify the encrypted feature vectors of the second biometric type during training, based on training the second DNN with encrypted feature vector and label inputs. According to one embodiment, the system further comprises the first neural network configured to process an unencrypted input of the first data type into the encrypted feature vectors. According to one embodiment, the system further comprises a pre-processing component configured to reduce a volume of unencrypted input biometric information for input into the first neural network. According to one embodiment, the classification component is configured to incrementally update the first DNN with new person labels and new persons feature vectors, based on updating null or undefined elements defined in the first DNN at training, and maintaining the network architecture and accommodating the unknown result for subsequent predictions without requiring full retraining of the first DNN. According to one embodiment, the system is configured to analyze the output values from the first DNN and based on positioning of the output values in an array and the values in those positions, determine the label or unknown.

According to one aspect, a computer implemented method or evaluating privacy-enabled biometrics and validating contemporaneous input of biometrics is provided. The method comprises: receiving, by at least one processor, a candidate set of instances of a first biometric data type input by a user requesting authentication; analyzing, by the at least one processor, a liveness threshold, wherein analyzing the liveness threshold includes processing the candidate set of instances to determine that the candidate set of instances matches a random set of instances; accepting, by a first deep neural network (“DNN”) executed by the at least one processor, encrypted feature vectors (e.g., voice feature vectors, etc.), generated from a first neural network, the first neural network configured to process an unencrypted input of the first data type into the encrypted feature vectors; classifying, by the first DNN, the encrypted feature vectors of the first biometric type during training, based on training the first DNN with encrypted feature vector and label inputs; returning, by the first DNN, a label for person identification or an unknown result during prediction responsive to analyzing encrypted feature vectors; and confirming authentication based at least on the label and the liveness threshold.

According to one embodiment, the method further comprises: determining for values above the liveness threshold that the input matches the random set of instances; and determining for values below the threshold that a current authentication request is invalid. According to one embodiment, the method further comprises generating a random set of instances of a first biometric type in response to an authentication request. According to one embodiment, the method further comprises requesting a user provide the candidate set of instances of the first biometric data type based on the generated random set of instances.

According to one embodiment, the method further comprises prompting user input of the randomly selected instances of the first biometric input to establish a threshold volume of biometric information confirmed at validation. According to one embodiment, the method further comprises: accepting, by at least a second deep neural network, encrypted feature vectors (e.g., face feature vectors, etc.), generated from a second neural network, the second neural network configured to process an unencrypted input of the second data type into the encrypted feature vectors; returning, by the second DNN a label for person identification or an unknown result during prediction responsive to analyzing encrypted feature vectors; and confirming identification based on matching the label for person identification from the first and second DNNs.

According to one embodiment, the second DNN is configured to classify the encrypted feature vectors of the second biometric type during training, based on training the second DNN with encrypted feature vector and label inputs. According to one embodiment, the method further comprises processing, by the first neural network, an unencrypted input of the first data type into the encrypted feature vectors. According to one embodiment, the method further comprises incrementally updating the first DNN with new person labels and new persons feature vectors, based on updating null or undefined elements established in the first DNN at training, and maintaining the architecture of the first DNN and accommodating the unknown result for subsequent predictions without requiring full retraining of the first DNN.

According to one aspect, an authentication system for evaluating privacy-enabled biometrics and contemporaneous input of biometrics for processing is provided. The system comprises at least one processor operatively connected to a memory, the at least one processor configured to generate in response to an authentication request, a random set of instances of a first biometric input of a first biometric data type (e.g., random words), an interface, executed by the at least one processor configured to: receive a candidate set of instances of a first biometric data type input by a user requesting authentication, for example, wherein the interface is configured to prompt a user to submit the first biometric input according to the randomly selected set of instances (e.g., display random words); a classification component executed by the at least one processor, configured to: analyze a liveness threshold; determine for values above the liveness threshold that the user is submitting the biometric information concurrent with or responsive to the authentication request; determine for values below the threshold that a current authentication request is unacceptable (e.g., invalid or incorrect, etc.), wherein analyzing the liveness threshold includes processing the candidate set of instances to determine a confidence value that the candidate set of instances matches the random set of instances generated by the at least one processor; the classification component further comprising at least a first deep neural network (“DNN”), the classification component configured to: accept encrypted embeddings (e.g., feature vectors) generated with a first neural network (“NN”) for processing the first data type of an unencrypted biometric input (e.g., pre-trained NN to classify the biometric input (e.g., your custom trained NN for voice, etc.)); classify embeddings (e.g., feature vectors) of the first type and label inputs during training based on processing the encrypted embeddings (e.g., feature vectors) obtained from the first neural network using the first DNN, return a label for person identification or an unknown result during prediction responsive to processing the encrypted embeddings from the candidate set of instances of the first biometric with the first DNN; and confirm authentication based on the person identification and the liveness threshold.

According to one embodiment, the system further comprises a feature vector generation component comprising a pre-trained neural network configured to generate Euclidean measurable encrypted feature vectors as an output of a least one layer in the neural network responsive to input of an unencrypted biometric input.

According to one aspect, an authentication system for evaluating privacy-enabled biometrics and liveness, the system comprising: at least one processor operatively connected to a memory; an interface configured to: accept a first biometric input associated with a first biometric data type (e.g., video or imaging); accept a second biometric input associated with a second biometric type, wherein the interface is configured to prompt a user to provide the second biometric input according to randomly selected instances of the second biometric input (e.g., the second biometric input providing voice and the randomly selected instances providing liveness); a classification component executed by the at least one processor, comprising at least a first and second deep neural network (“DNN”), the classification component configured to: accept encrypted feature vectors generated with a first classification neural network for processing a first type of an unencrypted biometric (e.g., pre-trained NN to classify the biometric input (e.g., FACENET, etc.)); accept encrypted feature vectors generated with a second classification neural network for processing a second type of an unencrypted biometric (e.g., custom pre-trained NN to classify voice identity—i.e. generate Euclidean measurable feature vectors); classify feature vector of the first type and label inputs during training based on processing the encrypted feature vectors from the first classification neural network using the first DNN, and return a label for person identification or an unknown result during prediction responsive to processing an unclassified encrypted biometric input of the first type with the first DNN; classify feature vector of the second type and label inputs during training based on processing the encrypted feature vectors from the second classification neural network using the second DNN, and return a label for person identification or an unknown result during prediction responsive to processing an unclassified encrypted biometric input of the second type with the second DNN; analyze an assurance factor derived from the randomly selected instances of the second biometric input, to determine that the input biometric information matches the randomly selected instances of the second biometric input, and to determine the input of the first and second biometric is contemporaneous with the authentication request; and confirm authentication based on the person identification resulting from the prediction executed by the first and second DNN and the assurance factor.

According to another aspect, encrypted search can be executed on the system in polynomial time, even in a one to many use case. This feature enables scalability that conventional systems cannot perform and enables security/privacy unavailable in many conventional approaches.

According to one aspect a privacy-enabled biometric system is provided. The system comprises at least one processor operatively connected to a memory; a classification component executed by the at least one processor, comprising a classification network having a deep neural network (“DNN”) configured to classify feature vector inputs during training and return a label for person identification or an unknown result during prediction; and the classification component is further configured to accept as an input feature vectors that are Euclidean measurable and return the unknown result or the label as output.

According to one embodiment, a set of biometric feature vectors is used for training in the DNN neural network for subsequent prediction. According to one embodiment, biometrics are morphed a finite number of times to create additional biometrics for training of the second (classification) neural network. The second neural network is loaded with the label and a finite number of feature vectors based on an input biometric. According to one embodiment, the classification component is configured to accept or extract from another neural network Euclidean measurable feature vectors. According to one embodiment, the another neural network comprises a pre-trained neural network. According to one embodiment, this network takes in a plaintext biometric and returns a Euclidean measurable feature vector that represents a one-way encrypted biometric. According to one embodiment, the classification neural network comprises a classification based deep neural network configured for dynamic training with label and feature vector input pairs to training. According to one embodiment, a feature vector is input for prediction.

According to one embodiment, the system further comprises a preprocessing component configured to validate plaintext biometric input. According to one embodiment, only valid images are used for subsequent training after the preprocessing. According to one embodiment, the classification component is configured with a plurality of modes of execution, including an enrollment mode configured to accept, as input, a label and feature vectors on which to train the classification network for subsequent prediction. According to one embodiment, the classification component is configured to predict a match, based on a feature vector as input, to an existing label or to return an unknown result. According to one embodiment, the classification component is configured to incrementally update an existing model, maintaining the network architecture (e.g., weighting values, loss function values, etc.) and accommodating the unknown result for subsequent predictions. In various embodiments, incremental updating the existing model avoids re-training operations that conventional approaches require. According to one embodiment, the system is configured to analyze the output values and based on their position and the values, determine the label or unknown.

According to one embodiment, the classification network further comprises an input layer for accepting feature vectors of a number of dimensions, the input layer having a number of classes at least equal to the number of dimensions of the feature vector input, first and second hidden layers, and an output layer that generates an array of values. According to one embodiment, the fully connected neural network further comprises an input layer for accepting feature vectors of a number of dimensions, the input layer having a number of nodes at least equal to the number of dimensions of the feature vector input, a first hidden layer of at least 500 dimensions, a second hidden layer of at least twice the number of input dimensions, and an output layer that generates an array of values—that based on their position in the array and the values at respective positions, determine the label or an unknown. According to one embodiment, a set of biometric feature vectors is used for training the DNN neural network for subsequent prediction.

According to one aspect a computer implemented method for executing privacy-enabled biometric training is provided. The method comprises instantiating, by at least one processor, a classification component comprising classification network having a deep neural network (“DNN”) configured to classify feature vector inputs during training and return a label for person identification or an unknown result during prediction; accepting, by the classification component, as an input feature vectors that are Euclidean measurable and a label for training the classification network; and Euclidean measurable feature vectors for prediction functions with the classification network; and classifying, by a classification component executed on at least one processor, the feature vector inputs and the label during training.

According to one embodiment, the method further comprises accepting or extracting, by the classification component, from another neural network the Euclidean measurable feature vectors. According to one embodiment, the another neural network comprises a pre-trained neural network. According to one embodiment, the classification neural network comprises a classification based deep neural network configured for dynamic training with label and feature vector input pairs. According to one embodiment, the method further comprises an act of validating input biometrics used to generate a feature vector. According to one embodiment, the method further comprises an act of triggering a respective one of a plurality of modes of operation, including an enrollment mode configured to accept a label and feature vectors for an individual. According to one embodiment, the method further comprises an act of predicting a match to an existing label or returning an unknown result responsive to accepting a biometric feature vector as input.

According to one embodiment, the method further comprises an act of updating the classification network with respective vectors for use in subsequent predictions. To handle the case of a person's looks changing over time, the input for prediction, may be used to re-train the individual. According to one embodiment, the method further comprises an act of updating, incrementally, an existing node in the classification network and maintaining the network architecture to accommodate the feature vector for subsequent predictions. According to one embodiment, the classification network further comprises an input layer for accepting feature vectors of a number of dimensions, the input layer having a number of nodes at least equal to the number of dimensions of the feature vector input, a first and second hidden layer and an output layer that generates an array of values.

According to one aspect a non-transitory computer readable medium containing instructions when executed by at least one processor cause a computer system to execute a method for executing privacy-enabled biometric analysis, the method is provided. A method comprises an instantiating, a classification component comprising a classification network having a deep neural network (“DNN”) configured to classify feature vector and label inputs during training and return a label for person identification or an unknown result during prediction; accepting, by the classification component, as an input feature vectors that are Euclidean measurable as an input and a label for training the classification network, and Euclidean measurable feature vectors for prediction functions with the classification network; and classifying, by a classification component executed on at least one processor, the feature vector inputs and the label during training.

According to one embodiment, the method further comprises an act of accepting or extracting, by the classification component, from another neural network Euclidean measurable feature vectors. According to one embodiment, the another neural network comprises a pre-trained neural network. According to various embodiments, the computer readable medium contains instructions to perform any of the method steps above, individually, in combination, or in any combination.

According to one aspect a privacy-enabled biometric system is provided. The system comprises a classification means comprising a classifying deep neural network (“DNN”) executed by at least one processor the FCNN configured to: classify feature vector inputs and return a label for person identification or an unknown result as a prediction; and accept as an input, feature vectors that are Euclidean measurable and a label as an instance of training.

According to one aspect, a privacy-enabled biometric system is provided. The system comprises at least one processor operatively connected to a memory; a classification component executed by the at least one processor, including a classification network having a deep neural network (“DNN”) configured to classify feature vector inputs during training and return a label for person identification or an unknown result during prediction, wherein the classification component is further configured to accept as an input feature vectors that are Euclidean measurable; a feature vector generation component comprising a pre-trained neural network configured to generate Euclidean measurable feature vectors as an output of a least one layer in the neural network responsive to input of an unencrypted biometric input.

According to one embodiment, the classification component is further configured to accept one way homomorphic, Euclidean measurable vectors, and labels for person identification as input for training. According to one embodiment, the classification component is configured to accept or extract from the pre-trained neural network the feature vectors. According to one embodiment, the pre-trained neural network includes an output generation layer which provides Euclidean measurable feature vectors. According to one embodiment, the classification network comprises a deep neural network suitable for training and, for prediction, output of a list of values allowing the selection of labels or unknown as output. According to one embodiment, the pre-trained network generates feature vectors on a first biometric type (e.g., image, voice, health data, iris, etc.); and the classification component is further configured to accept feature vectors from another neural network that generates Euclidean measurable feature vectors on a another biometric type.

According to one embodiment, the system is configured to instantiate multiple classification networks each associated with at least one different biometric type relative to another classification network, and classify input feature vectors based on executing at least a first or second classification network. According to one embodiment, the system is configured to execute a voting procedure to increase accuracy of identification based, for example, on multiple biometric inputs or multiple types of biometric input. According to one embodiment, the system is configured to maintain at least an executing copy of the classifying network and an updatable copy of classification network that can be locked or put in an offline state to enable retraining operations while the executing copy of the classifying network handles any classification requests. According to one embodiment, the classification component is configured with a plurality of modes of execution, including an enrollment mode configured to accept a label for identification and the input feature vectors for an individual from the feature vector generation component.

According to one embodiment, the classification component is configured to predict a match to an existing label or to return an unknown result based on feature vectors enrolled in the classification network. According to one embodiment, the classification component is configured to incrementally update an existing node in the neural network maintaining the network architecture and accommodating the unknown result for subsequent predictions. According to one embodiment, the classification network further comprises an input layer for accepting feature vectors of a number of dimensions, the input layer having a number of nodes at least equal to the number of dimensions of the feature vector input, a first hidden layer, a second hidden layer, and an output layer that generates an array of values that based on their position and the values, determine the label or unknown. According to one embodiment, the classification network further comprises a plurality of layers including two hidden layers and an output layer having a number of nodes at least equal to the number of dimensions of the feature vector input.

According to one aspect a computer implemented method for executing privacy-enabled biometric analysis, the method is provided. The method further comprises instantiating, by at least one processor, a classification component comprising a deep neural network (“DNN”) configured to classify feature vector inputs during training and return a label for person identification or an unknown result during prediction, and a feature vector generation component comprising a pre-trained neural network; generating, by the feature vector generation component Euclidean measurable feature vectors as an output of a least one layer in the pre-trained neural network responsive to input of an unencrypted biometric input; accepting, by the classification component, as an input feature vectors that are Euclidean measurable generated by the feature vector generation component and a label for training the classification network, and Euclidean measurable feature vectors for prediction functions with the classification network; and classifying, by a classification component executed on at least one processor, the feature vector inputs and the label during training.

According to one embodiment, the method further comprises accepting or extracting, by the classification network the Euclidean measurable feature vectors from the pre-trained neural network. According to one embodiment, the second neural network comprises a pre-trained neural network. According to one embodiment, the method further comprises an act of validating input feature vectors as Euclidean measurable. According to one embodiment, the method further comprises generating, by the classification component feature vectors on a first biometric type (e.g., image, voice, health data, iris, etc.); and accepting, by the classification component, feature vectors from another neural network that generates Euclidean measurable feature vectors on a second biometric type.

According to one embodiment, the method further comprises: instantiating multiple classification networks each associated with at least one different biometric type relative to another classification network, and classifying input feature vectors based on applying at least a first or second classification network. According to one embodiment, the method further comprises executing a voting procedure to increase accuracy of identification based on multiple biometric inputs or multiple types of biometric input and respective classifications. According to one embodiment, for a biometric to be considered a match, it must receive a plurality of votes based on a plurality of biometrics. According to one embodiment, the method further comprises instantiating multiple copies of the classification network to enable at least an executing copy of the classification network, and an updatable classification network that can be locked or put in an offline state to enable retraining operations while the executing copy of the classification network handles any classification requests. According to one embodiment, the method further comprises predicting a match to an existing label or to return an unknown result based, at least in part, on feature vectors enrolled in the classification network. According to one embodiment, the method further comprises updating, incrementally, an existing model in the classification network maintaining the network architecture and accommodating the unknown result for subsequent predictions.

According to one aspect a non-transitory computer readable medium containing instructions when executed by at least one processor cause a computer system to execute a method for executing privacy-enabled biometric analysis, the method is provided. The method comprises instantiating a classification component comprising a deep neural network (“DNN”) configured to classify feature vector and label inputs during training and return a label for person identification or an unknown result during prediction, and a feature vector generation component comprising a pre-trained neural network; generating, by the feature vector generation component Euclidean measurable feature vectors as an output of a least one layer in the pre-trained neural network responsive to input of an unencrypted biometric input; accepting, by the classification component, as an input feature vectors that are Euclidean measurable generated by the feature vector generation component and a label for training the classification network, and Euclidean measurable feature vectors for prediction functions with the classification network; and classifying, by a classification component executed on at least one processor, the feature vector inputs and the label during training. According to various embodiments, the computer readable medium contains instructions to perform any of the method steps above, individually, in combination, or in any combination.

According to one aspect a privacy-enabled biometric system is provided. The system comprises a feature vector generation means comprising a pre-trained neural network configured to generate Euclidean measurable feature vectors responsive to an unencrypted biometric input; a classification means comprising a deep neural network (“DNN”) configured to: classify feature vector and label inputs and return a label for person identification or an unknown result for training; and accept feature vectors that are Euclidean measurable as inputs and return a label for person identification or an unknown result for prediction.

According to one aspect a privacy-enabled biometric system is provided. The system comprises at least one processor operatively connected to a memory; a classification component executed by the at least one processor, including a classification network having a deep neural network (“DNN”) configured to classify feature vector and label inputs during training and return a label for person identification or an unknown result during prediction, wherein the classification component is further configured to accept as an input feature vectors that are Euclidean measurable; the classification network having an architecture comprising a plurality of layers: at least one layer comprising nodes associated with feature vectors, the at least one layer having an initial number of identification nodes and a subset of the identification nodes that are unassigned; the system responsive to input of biometric information for a new user is configured to trigger an incremental training operation for the classification network integrating the new biometric information into a respective one of the unallocated identification nodes usable for subsequent matching.

According to one embodiment, the system is configured to monitor allocation of the unallocated identification nodes and trigger a full retraining of the classification network responsive to assignment of the subset of unallocated nodes. According to one embodiment, the system is configured to execute a full retraining of the classification network to include additional unallocated identification nodes for subsequent incremental retraining of the DNN. According to one embodiment, the system iteratively fully retrains the classification network upon depletion of unallocated identification nodes with additional unallocated nodes for subsequent incremental training. According to one embodiment, the system is further configured to monitor matching of new biometric information to existing identification nodes in the classification network.

According to one embodiment, the system is further configured trigger integration of new biometric information into existing identification nodes responsive to exceeding a threshold associated with matching new biometric information. According to one embodiment, the pre-trained network is further configured to generate one way homomorphic, Euclidean measurable, feature vectors for the individual. According to one embodiment, the classification component is further configured to return a set of probabilities for matching a set of existing labels. According to one embodiment, the classification component is further configured to predict an outcome based on a trained model, a set of inputs for the prediction and a result of a class or unknown (all returned values dictating UNKNOWN).

According to one embodiment, the classification component is further configured to accept the feature vector inputs from a neural network model that generates Euclidean measurable feature vectors. According to one embodiment, the classification component is further configured to extract the feature vectors from the neural network model from layers in the model. According to one embodiment, the system further comprising a feature vector component executed by the at least one processor comprising a neural network. According to one embodiment, the feature vector component is configured to extract the feature vectors during execution of the neural network from layers. According to one embodiment, the neural network comprises of a set of layers wherein one layer outputs Euclidean Measurable Feature Vectors. According to one embodiment, the system further comprising a retraining component configured to monitor a number of new input feature vectors or matches of new biometric information to a label and trigger retraining by the classification component on the new biometric information for the label. This can be additional training on a person, using predict biometrics, that continues training as a biometric changes over time. The system may be configured to do this based on a certain number of consecutive predictions or may do it chronologically, say once every six months.

According to one embodiment, the classification component is configured to retrain the neural network on addition of new feature vectors. According to one embodiment, the neural network is initially trained with unallocated people classifications, and the classification component is further configured to incrementally retrain the neural network to accommodate new people using the unallocated classifications. According to one embodiment, the system further comprises a retraining component configured to: monitor a number of incremental retraining; trigger the classifier component to fully retrain the neural network responsive to allocation of the unallocated classifications. According to one embodiment, the classification component is configured to fully retrain the neural network to incorporate unallocated people classifications, and incrementally retrain for new people using the unallocated classifications. According to one embodiment, the classification component further comprises multiple neural networks for processing respective types of biometric information. According to one embodiment, the classification component is further configured to generate an identity of a person responsive to at least two probable biometric indicators that may be used simultaneously or as part of a “voting” algorithm.

According to one aspect a computer implemented method for privacy-enabled biometric analysis is provided. The method comprises instantiating, by at least one processor, a classification component comprising a classification network having a deep neural network (“DNN”) configured to classify feature vector and label inputs during training and return a label for person identification or an unknown result during prediction, and wherein the classification component is further configured to accept as an input feature vectors that are Euclidean measurable and return the unknown result or the label as output; instantiating the classification component includes an act of allocating within at least one layer of the classification network, an initial number of classes and having a subset of the class slots that are unassigned; triggering responsive to input of biometric information for a new user incremental training operation for the classification network integrating the new biometric information into a respective one of the unallocated class slots usable for subsequent matching.

According to one embodiment, the method further comprises acts of accepting, by the classification component, as an input feature vectors that are Euclidean measurable generated by a feature vector generation component; classifying, by the classification component executed on at least one processor, the feature vector inputs; and returning, by the classification component, a label for person identification or an unknown result. According to one embodiment, the method further comprises acts of instantiating a feature vector generation component comprising a pre-trained neural network; and generating, by the feature vector generation component Euclidean measurable feature vectors as an output of a least one layer in the pre-trained neural network responsive to input of an unencrypted biometric input. According to one embodiment, the method further comprises an act of monitoring, by the at least one processor, allocation of the unallocated identification classes and triggering an incremental retraining of the classification network responsive to assignment of the subset of unallocated nodes to provide additional unallocated classes.

According to one embodiment, the method further comprises an act of monitoring, by the at least one processor, allocation of the unallocated identification nodes and triggering a full retraining or incremental of the classification network responsive to assignment of the subset of unallocated nodes. According to one embodiment, the method further comprises an act of executing a full retraining of the classification network to include additional unallocated classes for subsequent incremental retraining of the DNN. According to one embodiment, the method further comprises an act of fully retraining the classification network iteratively upon depletion of unallocated identification nodes, the full retraining including an act of allocating additional unallocated nodes for subsequent incremental training. According to one embodiment, the method further comprises an act of monitoring matching of new biometric information to existing identification nodes. According to one embodiment, the method further comprises an act of triggering integration of new biometric information into existing identification nodes responsive to exceeding a threshold associated with matching new biometric information. According to one embodiment, the method further comprises an act of generating one way homomorphic, Euclidean measurable, labels for person identification responsive to input of Euclidean measurable feature vectors for the individual by the classification component.

According to one aspect a non-transitory computer readable medium containing instructions when executed by at least one processor cause a computer system to execute a method instantiating a classification component comprising a classification network having a deep neural network (“DNN”) configured to classify feature vector and label inputs during training and return a label for person identification or an unknown result during prediction, and wherein the classification component is further configured to accept as an input feature vectors that are Euclidean measurable and return the unknown result or the label as output; instantiating the classification component includes an act of allocating within at least one layer of the classification network, an initial number of classes and having a subset of additional classes that are unassigned; triggering responsive to input of biometric information for a new user incremental training operation for the classification network integrating the new biometric information into a respective one of the unallocated identification nodes usable for subsequent matching. According to various embodiments, the computer readable medium contains instructions to perform any of the method steps above, individually, in combination, or in any combination.

According to one aspect a privacy-enabled biometric system is provided. The system comprises at least one processor operatively connected to a memory; a classification component executed by the at least one processor, comprising classification network having a deep neural network configured to classify Euclidean measurable feature vectors and label inputs for person identification during training, and accept as an input feature vectors that are Euclidean measurable and return an unknown result or the label as output; and an enrollment interface configured to accept biometric information and trigger the classification component to integrate the biometric information into the classification network.

According to one embodiment, the enrollment interface is accessible via uri, and is configured to accept unencrypted biometric information and personally identifiable information (“PII”). According to one embodiment, the enrollment interface is configured to link the PII to a one way homomorphic encryption of an unencrypted biometric input. According to one embodiment, the enrollment interface is configured to trigger deletion of the unencrypted biometric information. According to one embodiment, the system is further configured to enroll an individual for biometric authentication; and the classification component is further configured to accept input of Euclidean measurable feature vectors for person identification during prediction. According to one embodiment, the classification component is further configured to return a set of probabilities for matching a feature vector. According to one embodiment, the classification component is further configured to predict an outcome based on a trained model, a set of inputs for the prediction and a result of a class (persons) or UNKNOWN (all returned values dictating UNKNOWN).

According to one embodiment, the system further comprises an interface configured to accept a biometric input and return and indication of known or unknown to a requesting entity. According to one embodiment, requesting entity includes any one or more of: an application, a mobile application, a local process, a remote process, a method, and a business object. According to one embodiment, the classification component further comprising multiple classification networks for processing different types of biometric information. According to one embodiment, the classification component is further configured to match an identity of a person responsive to at least two probable biometric indicators that may be used simultaneously or as part of a voting algorithm. According to one embodiment, the classification network further comprises an input layer for accepting feature vectors of a number of dimensions, the input layer having a number of classes at least equal to the number of dimensions of the feature vector input, a first and second hidden layer, and an output layer that generates an array of values.

According to one aspect a computer implemented method for privacy-enabled biometric analysis, the method is provided. The method comprises instantiating, by at least one processor, a classification component comprising a full deep neural network configured to classify feature vectors that are Euclidean measurable and a label inputs for person identification during training, and accept as an input feature vectors that are Euclidean measurable and return an unknown result or the label as output during prediction, and an enrollment interface; accepting, by the enrollment interface, biometric information associated with a new individual; triggering the classification component to train the classification network on feature vectors derived from the biometric information and a label for subsequent identification; and return the label through for subsequent identification.

According to one embodiment, instantiating the enrollment interface includes hosting a portal accessible via uri, and the method includes accepting biometric information and personally identifiable information (“PII”) through the portal. According to one embodiment, the method further comprises linking the PII to a one way homomorphic encryption of an unencrypted biometric input. According to one embodiment, the method further comprises triggering deletion of unencrypted biometric information on a submitting device. According to one embodiment, the method further comprises enrolling individuals for biometric authentication; and mapping labels and respective feature vectors for person identification, responsive to input of Euclidean measurable feature vectors and a label for the individual. According to one embodiment, the method further comprises returning a set of probabilities for matching a set of existing labels.

According to one embodiment, the method further comprises predicting an outcome based on a trained model, a set of inputs for the prediction and a result of a class (e.g., persons) or unknown (e.g., all returned values dictating UNKNOWN). According to one embodiment, the method further comprises accepting via an authentication interface a biometric input and returning and indication of known or unknown to a requesting entity. According to one embodiment, the requesting entity includes any one or more of: an application, a mobile application, a local process, a remote process, a method, and a business object. According to one embodiment, the method further comprises processing different types of biometric information using multiple classification networks. According to one embodiment, the method further comprises generating an identity of a person responsive to at least two probable biometric indicators that may be used simultaneously or as part of a voting algorithm.

According to one embodiment, the classification network further comprises an input layer for accepting feature vectors of a number of dimensions, the input layer having a number of classes at least equal to the number of dimensions of the feature vector input, a second hidden layer of at least twice the number of input dimensions, and an output layer that generates an array of values. According to one embodiment, the fully connected neural network further comprises an input layer for accepting feature vectors of a number of dimensions, the input layer having a number of nodes at least equal to the number of dimensions of the feature vector input, a first hidden layer of at least 500 dimensions, a second hidden layer of at least twice the number of input dimensions, and an output layer that generates an array of values that based on their position and the values, determine the label or unknown.

Still other aspects, examples, and advantages of these exemplary aspects and examples, are discussed in detail below. Moreover, it is to be understood that both the foregoing information and the following detailed description are merely illustrative examples of various aspects and examples, and are intended to provide an overview or framework for understanding the nature and character of the claimed aspects and examples. Any example disclosed herein may be combined with any other example in any manner consistent with at least one of the objects, aims, and needs disclosed herein, and references to “an example,” “some examples,” “an alternate example,” “various examples,” “one example,” “at least one example,” “this and other examples” or the like are not necessarily mutually exclusive and are intended to indicate that a particular feature, structure, or characteristic described in connection with the example may be included in at least one example. The appearances of such terms herein are not necessarily all referring to the same example.

