Our work is motivated both by the study of the link between genotype and the human brain phenotype from large datasets [1]}, and by practical applications such as maintaining accurate patient records in hospital Picture Archiving and Communication Systems (PACS).
We adopted a memory-based learning model where all data are stored in memory and accessed via highly-specific keypoint queries. Memory-based learning requires no explicit training procedure [2]}, and approaches Bayes optimal performance as the number of data \(N\)  becomes large [3]}. 3D SIFT keypoints [4]} derived from Gaussian scale-space theory [5]}, [6]} are invariant to global similarity transforms and contrast variations, and thus account for misalignment and scanner variability between images. Efficient indexing reduces the quadratic \(O(N^2)\)  complexity of nearest neighbor (NN) keypoint lookup to \(O(N~\log ~N)\) , e.g. via \(O(\log N)\)  KD-tree indexing [7]}, ensuring that the method scales gracefully to large datasets, e.g. 7500 brain MRIs [8]} or 20000 lung CTs [9]}.
