In this paper, we propose \(\mathsf {Starmie}\) , an end-to-end framework for dataset discovery from data lakes with table union search as the main use case.
\(\mathsf {Starmie}\)  utilizes pre-trained language models (LMs) such as BERT [1]} to obtain semantics-aware representations for columns of data lake tables.
While pre-trained LMs have been shown to achieve state-of-the-art results in table understanding applications [2]}, [3]}, [4]}, their good performance heavily relies
on high-quality labeled training data.
Meanwhile, based on the problem setting of table union search [5]}, [6]}, we must come up with a fully unsupervised approach in order to apply pre-trained LMs to such applications, which is not yet supported by previous studies.
\(\mathsf {Starmie}\)  addresses this issue by leveraging contrastive representation learning [7]} to learn column representations in a self-supervision manner.
The framework defines a learning objective that connects the same or similar columns in the representation space while separating distinct columns.
As such, \(\mathsf {Starmie}\)  can apply the pre-trained representation model in downstream tasks such as table union search without requiring any labels.
We also propose to combine the learning algorithm with a novel multi-column table Transformer model to learn contextualized column embeddings, which capture the column semantics depending on not only the column values but also their context within a table.
While a recent study SANTOS  [8]} can reach a similar goal by employing a knowledge base, our proposed methods can automatically capture such contextual information from tables in an unsupervised manner without relying on any external knowledge or labels.
