Based on the proposed column encoders, we utilize the cosine similarity between column embeddings as the column unionability score and develop a bipartite matching based method to calculate the table unionability score accordingly.
We propose a filter-and-verification framework that enables the usage of different indexing and pruning techniques to avoid accessing unlikely candidates in the data lake and to reduce the number of computations of the expensive bipartite matching.
While most previous studies employed LSH index to improve the search performance, we explore another design choice of using HNSW [1]} to index the embedding vectors to accelerate query processing.
Experimental results show that HNSW can significantly improve the query time at the cost of only slightly reducing the MAP/recall scores.
Except for table union search, we also conduct two case studies to show that \(\mathsf {Starmie}\)  can also support other dataset discovery applications such as finding joinable tables and column clustering.
