Vector based topological fingerprint matching
REFERENCE TO RELATED APPLICATION 
     This application is a continuation application of my application U.S. Ser. No. 08/248,012, filed May 24, 1994, and entitled &#34;VECTOR BASED TOPOLOGICAL FINGERPRINT MATCHING, now abandoned. 
    
    
     A portion of the disclosure of this patent document contains material which is subject to copyright protection. The copyright owner has no objection to the facsimile reproduction by anyone of the patent disclosure, as it appears in the Patent and Trademark Office patent files or records, but otherwise reserves all copyright rights whatsoever. 
     BACKGROUND AND BRIEF DESCRIPTION OF THE INVENTION 
     The invention relates to fingerprint coding and recognition and retrieval systems based on generally invarying topological irregularities, characteristics or minutiae (which terms are used interchangeably herein) of fingerprints. The term &#34;fingerprint&#34; or &#34;print&#34; is used in reference to the epidermal ridge lines of the ten fingers of the human hand, palm prints, toe and sole prints of humans wherein such epidermal ridge lines and characteristic features thereof are in patterns unique to a particular individual. 
     In my U.S. Pat. Nos. 4,747,147 and 4,817,183 and my papers entitled &#34;Digital Coding of Single Fingerprints--A New Approach for the Computer Age&#34;, Journal of Police Science and Administration, Vol. X, No. 2, June 1982, (1) &#34;A TOPOLOGICAL APPROACH TO THE MATCHING OF SINGLE FINGERPRINTS: DEVELOPMENT OF ALGORITHMS FOR USE ON ROLLED IMPRESSIONS&#34;, NBS Special Publication 500-124, May 1985, and (2) &#34;A TOPOLOGICAL APPROACH TO THE MATCHING OF SINGLE FINGERPRINTS: DEVELOPMENT OF ALGORITHMS FOR USE ON LATENT FINGERMARKS&#34;, NBS Special Publication 500-126, October, 1985. I show that the soft elastic nature of human skin causes substantial variation of the spatial descriptions of successive impressions of the same fingerprints. Consequently, spatially based coding schemes used for forming machine searchable databases have inherent inaccuracies due to the fact that the spatial based coordinate system typically used for coding purposes could not take into account the wide variations in spatial distortions making the match or identification between two rolled prints on the same finger somewhat problematical particularly where the prints are taken at substantially different times or pressures. 
     Topological coding schemes provide concise digital codes that provide a more reliable basis for fingerprint identification systems. In my above referred to patents and paper, I disclose comparison methods based on topological coding of prints in which a topology base coding system for recording and comparing minutiae used a single vector array generated from topologically based coding of a fingerprint. 
     In the topological vector extraction part of the system disclosed in my above referred to patents, 1982 Paper and my NBS Special Publication 500-124, the core of the fingerprint is centrally located at a reference point and a horizontal line is projected through the core to intersect ridge lines to each side of the centrally located reference point. In the case of an arch a vertical line is drawn through successive ridge summits. From the points of crossing of the ridges with the projected horizontal or vertical line, the ridges are traced to the first significant irregularity and a type code (T) is assigned to the irregularity. In my patents, the distance (D) from the reference line is measured and these data are recorded in a predetermined order to constitute a topological vector for the print which then is recorded in a machine searchable database. Comparison of vectors takes the form of a sequence of array operations. Comparison of good quality rolled prints is performed extremely rapidly on this basis. In my NBS Special Publication 500-126 I disclose a system for topologically coding fingerprints wherein each topological code would be a vector generated by exploration from short straight lines orthogonal to ridge flow direction and drawn offset from a given characteristic (bifurcations suggested). 
     THE PRESENT INVENTION 
     An object of the present invention is to provide an improved fingerprint recognition and retrieval system. 
     A further object of the invention is to provide an improved topological matching system in which topological code vectors are generated from the fingerprint with one vector being generated for each minutiae revealed on the print. 
     Another object of the invention is to provide an improved topological event code incorporating the reference number of any other minutiae point encountered through ridge explorations or tracing from each vector generating line. 
     Another object of the invention is to provide a more versatile matching algorithm. 
     According to the present invention, topological code vectors are generated directly from the fingerprint (i.e. not via any topological coordinate representation as in my U.S. Pat. No. 4,747,147), one vector being generated for each minutiae revealed on the print. 
     As is common in the art, incoming fingerprint images are electronically processed to provide a skeleton image of ridge lines and minutiae points constituted by ridge endings and bifurcations with the spatial positions of the minutiae points and an reference or index number (N) for each minutiae point being recorded in a storage medium. 
     The local ridge flow directions in the vicinity of each minutiae point is determined and a generating line is projected at a predetermined angle transverse to the local ridge flow direction at each minutiae point (the predetermined angle being preferably orthogonal to a local ridge flow direction). The generating line has a length sufficient to span a predetermined number of ridges to each side of each minutiae point. For each minutiae point, from the point of crossing of the projected generating line with the ridges within the span, a tracing is made in two directions along each ridge line crossed by the generating line to the first occurring topological event. 
     A type code (T) is assigned to each topological event encountered by the ridge tracings. Then, from these data, a topological event code vector is generated for each minutiae with the topological event code vector being comprised of the topological type code for any minutiae encountered during the tracing and the reference or index number (N) for each associated minutiae. These data are stored as vectors in a machine searchable database. It is not necessary to make reference to any central core point or delta and all dependence on the presence of arbitrary selection of a central reference point within the print is therefore eliminated. 
     A matching algorithm is arranged in a sequence of graded stages with the computational intensity of each stage being controlled by the parameters. This enables the algorithm to be adapted easily for different purposes (from high resolution identification of poor quality latent images at one extreme, to rapid pass filtering of good quality scanned or rolled fingerprint images at the other) simply by changing the parameters set. 
     Moreover, the matching algorithm systematically develops &#34;topologically connected clusters&#34; within any two prints under comparison. It therefore seeks to develop a maximal topologically connected cluster within one print, for which there exists a matching cluster within the other print (and where the topological relationships between minutiae within the clusters are replicated). A final score is derived for this optimal cluster which reflects both the size of the cluster and the density of the topological connectivity data which is replicated within the other print. 
     Thus, an important feature of the invention is that the matching algorithm identifies the maximal topologically connected cluster of minutiae points within one print for which there exists a matching cluster of minutiae points within the print under comparison and where the topological relationships among the set of minutiae within the matching clusters are identical. 
    
    
     BRIEF DESCRIPTION OF THE DRAWINGS 
     The above and other objects, advantages and features of the invention will become more apparent when considered with the following specification and accompanying drawings wherein: 
     FIG. 1A is a block diagram of a fingerprint recording and recognition system incorporating the invention, 
     FIG. 1B is a diagrammatic illustration showing how generating lines are off-set from minutiae points and oriented orthogonal to a local ridge flow direction (the selection of lines being shown only for purposes of illustration), 
     FIGS. 2A and 2B show the detail of how the generating line is oriented and, in a preferred embodiment, off-set in the two principle cases, namely, ridge endings and ridge bifurcations, 
     FIGS. 3A and 3B show how the vector entries correspond to the different ridge exploration events, 
     FIG. 4 is a table illustrating the ridge exploration event codes for use in the topological event code vector formations, 
     FIG. 5A shows a portion of a print around a minutiae point (minutiae reference No. 20) and spanning 6 ridge lines to each side of the minutiae point and, FIG. 5B illustrates the extracted vector centered on minutiae 20 of FIG. 5A, 
     FIG. 6 shows a sample part of a composite data record, 
     FIG. 7 shows a schematic of the stages of the matching algorithm, 
     FIG. 8 illustrates returning of a pairs list from the full vector matching (level 3) of FIG. 7, 
     FIG. 9 shows a detailed matcher output of two prints under comparison, and the final version of Table 3 is printed-out in full, 
     FIG. 10 shows a print-out of Table 4 revealing a number of different clusters with the cluster score in each case being calculated as the sum of the second order neighborhood scores for each search/file minutiae pairing within the cluster, 
     FIG. 11 shows the final combined cluster produced during the print comparison following through from FIGS. 9 and 10, and 
     FIG. 12 shows a full table of matcher parameters. 
    
    
     DETAILED DESCRIPTION OF THE INVENTION 
     In a fully automated system, rolled prints are received as grey-scale images, and pass through binarization 10 and skeletonization 11 stages (image processing steps). 
     Latent images will normally be photographed, placed under a scanner, and the resulting image displayed on a screen. A fingerprint technician then traces the ridge pattern on the computer screen using either a mouse or a light pen of some kind. The resulting tracing (albeit there is no physical tracing) acts as the skeleton image. 
     Either process produces a skeleton image of the ridge pattern. The difference is only that, in the case of latents, technicians intervene in interpreting and editing the ridge pattern; and with good quality prints (rolled, or live-scan) no human intervention would be required. 
     From the skeleton image onwards, the encoding processes are the same. 
     (1) All minutiae points (ridge-endings and bifurcations) are detected and their spatial positions noted in block 12. This process is well understood in the art. 
     (2) Blurred areas (where the ridge structure is unclear) are established and their positions and boundaries recorded (in what is commonly called a &#34;blur map&#34;) in 13. 
     (3) The local ridge flow direction is determined in the vicinity of each minutiae point in 14. 
     (4) In block 15, a vector &#34;generating line&#34; is placed orthogonal to the ridge flow direction, preferably slightly offset from the minutiae point, at each minutiae. The generating line is drawn long enough to cover a predetermined number of ridges either side of the central minutiae point. The number of ridges covered each side is a parameter of the encoding and matching process, which we will call &#34;SPAN&#34;. 
     FIG. 1B shows a skeletal print with such generating lines superimposed transverse to the ridge flow direction, where SPAN is assumed to be 6. A selection of lines only is displayed, for purposes of illustration. 
     FIGS. 2A and 2B shows the detail of how the generating line 20 is positioned according to local ridge flow direction and preferably oriented and offset OS in the two cases--ridge ending (FIG. 2A) or bifurcation (FIG. 2B). 
     (5) A topological code vector is then generated in 16 through the process of ridge exploration. 
     FIGS. 3A and 3B diagrammatically show how the vector entries correspond to the different ridge exploration events. The arbitrary terms &#34;up&#34; and &#34;down&#34; are merely terms of reference for the directions shown in FIGS. 3A and 3B. 
     The final position (14 in FIG. 3B) is &#34;padded&#34; both up and down in the case of a bifurcation. 
     FIG. 4 shows the selection of hexadecimal event codes according to different possible ridge-traced events. These are only slightly different from the original set disclosed in my above-reference patents and papers. 
     (6) For any ridge traced event that corresponds to another identified minutiae point (i.e. where the ridge-traced event is 2, 3, 4, 6, 7 or 8) the reference number of that other minutiae point is stored along with the topological exploration event code, as an &#34;associated minutiae reference number&#34;. Otherwise a &#34;0&#34; or blank entry is made for the associated minutiae reference number. 
     FIG. 5 shows an extracted vector EV (centered on minutiae 20), complete with associated minutiae reference numbers RN, for a selected minutiae point (minutiae No. 20 in this case). 
     (7) For each minutiae on the print, the topological code vectors are recorded in a storage medium 17 (e.g. optical or magnetic), together with the following additional pieces of data: 
     (a) a reference or index number (an arbitrary ordering of the minutiae points), 
     (b) spatial coordinates (according to some arbitrary global referencing scheme), 
     (c) the minutiae type code (`e` for ridge ending, or `b` for bifurcation), 
     (d) an angular orientation of the minutiae point (per se known in the art). 
     (8) These sets of information are then combined for each minutiae visible on the print, producing a composite data record for the whole print which is stored in 17. 
     FIG. 6 shows a sample part of such a composite data record. 
     FIG. 7 shows a schematic of the stages of the matching algorithm (Appendix A). They are in order: 
     (1) Read in search print data and file print data for comparison. 
     (2) Calculate &#34;information content&#34; for the search print and for the file print: This is essentially a count of the number of valid topological connections between minutiae which are represented within the topological code vectors. In practice it would be calculated once when a print was first introduced to the system, and the result stored as part of the data file. The &#34;information content&#34; measure is used only as part of the final score normalization procedure. 
     (3) Level 1: initial selection of search/file minutiae pairings: Level 1 provides a fast and efficient mechanism for generating search-file minutiae pairs. The large majority of these pairs will be false and are eliminated at subsequent stages of processing. All possible search/file minutiae pairings are evaluated against the following simple rules: 
     (a) do both the search and file minutiae vectors have sufficient non-trivial information content (i.e. the number of non-trivial topological connections represented exceeds a predetermined minimum threshold--&#34;Filter 1&#34;), 
     (b) are the search and file minutiae angular orientations the same, within some predetermined tolerance parameter--&#34;minimum angular tolerance&#34;. 
     Further rules could be added to speed processing in certain applications. For instance, global coordinates could be used to require that the search and file minutiae pairings came from essentially similar regions of the print. Another possible filtration rule would require that they be of the same type (i.e. both ridge-endings or both bifurcations). 
     Search/file minutiae pairings that pass the level 1 rules are passed to level 2. 
     (4) Level 2: &#34;FASTMATCH&#34; on topological code vectors: For each minutiae pair passing level 1, the topological code vectors for the search and file minutiae are compared. This comparison is designed to operate fast and within minimal computational load, so no use is made at this stage of the accompanying &#34;associated minutiae reference numbers&#34;. The hexadecimal code vectors are essentially overlaid, and a count made of the number of times matching event codes appear in similar positions. This count can be referred to as the &#34;hex overlay count&#34;. 
     Search/file minutiae pairs pass level 2 if and only if their &#34;hex overlay count&#34;, as returned from subroutine &#34;FULLMATCH&#34; exceeds a predetermined threshold--&#34;Filter 2&#34;. 
     The combinations passing the level 2 tests are stored in a table for subsequent processing. The file print index and search print index are stored, together with a &#34;shift condition indicator&#34; which records the precise alignment under which the vectors were best matched. 
     The shift condition indicator assumes the values `O`, `f`, or `s`. Condition `O` shows that the score was obtained with the natural alignment of the search and file vectors. Condition `f` shows a shift of one position (i.e. one ridge), where the file minutiae is type `e` and the search minutiae is type `b`. The search vector is effectively shifted one place to the right by adjusting all the FILE print references. Condition `s` is the converse, that is, the file minutiae is `b` while the search minutiae is `e`. &#34;Shifted&#34; scores can arise only if the central minutiae are of different types. If they are, and both the aligned and shifted matches given sufficiently high totals, then there is no objection to BOTH matches, that is, with and without a shift, forming entries in the Level 2 score table, and being passed on for further processing. If no minutiae pairs pass the Level 2 threshold score, processing is terminated and a zero score returned. 
     (5) Level 3: Full vector matching: Level 3 examines each of the search/file minutiae pairings (call them &#34;base pairings&#34;) that passed level 2 in turn. For each pair the corresponding pair of topological code vectors are passed to subroutine &#34;FULLMATCH&#34;. FULLMATCH incorporates a more detailed and sophisticated topological vector matching process, including: 
     (a) the use of crude distance tests, 
     (b) score reduction for dependent pairings. 
     The FULLMATCH scoring procedure is very similar to that disclosed in my above-identified patents within the vector matching routines of MATCH 4 and LM6. 
     However, FULLMATCH performs one additional function here. For each vector match undertaken FULLMATCH returns a list of further search/file minutiae pairings, derived from the vector match through reference to the &#34;associated minutiae reference numbers&#34;. These further pairings represent other minutiae on the search and file prints which are topologically connected to the base pairing in the same way, and through corresponding ridge explorations. These further search/file minutiae pairings are denoted &#34;pairs connected to the base minutiae pair&#34;. 
     FIG. 8 illustrates the returning of a pairs list from FULLMATCH. 
     After the use of FULLMATCH the vector comparisons that score above a threshold--Filter 3--are stored in a temporary table (Temporary Table 3), and the associated lists of connected pairs are retained for further use. 
     For each minutiae pair appearing in Temporary Table 3, a &#34;1st order neighborhood&#34; score is calculated by adding to its own vector match score the vector match scores for all the connected pairs returned from FULLMATCH. The &#34;1st order neighborhood&#34; score thus reflects not only how well the base vectors matched up, but also reflects how similar the vectors corresponding to all connected pairs were. 
     Then &#34;2nd order neighborhood&#34; scores are calculated for each base pair, simply adding together the &#34;1st order neighborhood&#34; scores for all the connected minutiae pairs. 
     The final version of Table 3 is constructed by reordering this information in descending order of the 2nd order neighborhood scores. A high composite neighborhood score provides a strong indication not only that the underlying vector match was a good one, but also that the minutiae in this pair are in fact topologically connected, in a manner consistent on both file and search prints, with a dense cluster of other high scoring minutiae pairs. 
     FIG. 9 shows the detailed matcher output of two prints under comparison, and the final version of Table 3 is printed out in full. 
     (6) Level 4: Exhaustive cluster formation: Minutiae pairs from Table 3 are lifted in sequence and used to generate exhaustively connected topological clusters. These clusters are recorded in sequence in &#34;Table 4&#34;. The conceptual approach is to establish, in clusters, search/file minutiae pairings which are all connected together topologically. 
     The rules under which this general procedure is carried out are: 
     (a) A new entry from Table 3 is first checked against existing entries in Table 4 (from previously generated clusters) and forms the basis for generation of a new cluster if, and only if, it has not already appeared. The object of this rule is to prevent duplication of effort that would result from starting a new cluster from a pairing already included within some previous cluster. 
     (b) Once a cluster has been opened, no file index or search index is allowed to be repeated within the cluster. That avoids going over ground already covered within that cluster, and also avoids any file or search minutiae being matched against two different minutiae in the other print. 
     (c) In order for the clusters to be topologically exhaustive, pairings can be added whether or not they appeared in Table 3 (i.e. had a vector match score above the Level 3 threshold). However, only pairings that appeared in level 3 will be used to start the generation of a new cluster. 
     (d) Once a cluster is exhausted, its length is checked against a run time parameter--Filter 4--which specifies the minimum number of minutiae pairs to form a reportable cluster. If it is not large enough the cluster is deleted. 
     FIG. 10 shows a printout of Table 4, revealing a number of different clusters. The cluster score in each case is calculated as the sum of the second order neighborhood scores for each search/file minutiae pairing within the cluster. 
     (7) Cluster combination routine: A cluster combination routine is included to boost match scores in cases where two or more portions of the prints may match up quite well, but where these regions are topologically disconnected (e.g. by scars running across the fingers). 
     Cluster combination follows the following rules. First the best cluster from Table 4 is copied (into &#34;Final&#34; Table). Then each other cluster (if there are any) is examined in turn. These subsequent clusters are deemed to be either: 
     (a) Disjoint: None of their file indices or search indices appear in the current (possibly already extended) version of &#34;final&#34;. 
     (b) Overlapping and non-contradictory: Some of the search/file minutiae pairings already appear in &#34;final&#34; (overlapping); but no file or search pairings appear with different pairings (i.e. non-contradictory). 
     (c) Contradictory: At least one of the search or file minutiae appears in &#34;final&#34; with a different mate. 
     Depending on which of these cases arises, &#34;final&#34; may be extended. The rules are: 
     (a) Disjoint cluster: add the whole cluster into &#34;final&#34; if the root minutiae pairing is distance compatible with all the existing entries in &#34;final&#34;. 
     (b) Overlapping and non-contradictory: add in all minutiae pairings not already in &#34;final&#34;. 
     (c) Contradictory: discard whole cluster. 
     FIG. 11 shows the final combined cluster produced during the print comparison following through from FIGS. 9 and 10. 
     (8) Final score calculation: The &#34;second order neighborhood&#34; scores are then summed for the best cluster, which may have been extended through the cluster combination stage. This composite score is then normalized to take account of the information content of both the search and file prints. 
     The effect and purpose of normalization, of course, is to correct high raw scores which resulted from information rich mismatches and which might have beaten down mate scores from very poor quality prints. 
     FIG. 12 shows a full table of matcher parameters, with reasonable (but not necessarily optimal) values. Optimal parameter tuning will depend heavily on the operational context and performance criteria. 
     The matching algorithm described herein is attached hereto as Appendix A, which is proprietary and protected by copyright. 
     While preferred embodiments of the invention have been illustrates and described, it will be appreciated that various other embodiments, adaptations and modifications of the invention will be readily apparent to those skilled in the art. ##SPC1##