Our primary contribution is a new pairwise distance measure for 3D keypoint sets, generalizing the Jaccard index to account for soft set equivalence (SSE) between keypoint elements. SSE is achieved via a kernel framework for keypoint similarity, and we introduce a new kernel that normalizes pairwise keypoint displacement by keypoint scale, accounting for localization uncertainty in scale-space. Our work extends and compares against keypoint-based neuroimaging analysis methods [1]}, [2]}, [3]}, specifically the keypoint signatures approach [3]}, currently the only method reporting perfect accuracy at identifying scans of the same individual in large public neuroimage MRI datasets, e.g. OASIS [5]}, ADNI [6]} and HCP [7]}. Here, following preliminary work [8]}, we demonstrate that the soft Jaccard distance is specific enough to predict family relationships from medical images, and we present the first results for this new task using 1010 MRIs of 434 families from the HCP [7]} dataset. The entire method is based on highly efficient keypoint indexing, and scales to large datasets of volumetric image data.
