I have a Spark dataset containing a column of SparseVector types. Additionally, I have another SparseVector $X$ which is not a part of the dataset. I want to order my dataset according to the closest distance (or similarity) relative to $X$.Can anyone help me with how to implement this?
Order SparseVectors by the closest distance to given SparseVector
apache spark
it appears that problem was solved with BucketedRandomProjectionLSH. After fit && transformation, and approxNearestNeighbors resulted dataset contains distCol, which (as per LSH.scala):@param distCol Output column for storing the distance between each result row and the key.