\(\mathsf {SingleCol}\)  is a basic column encoder proposed in Section REF  that uses the encoder to only encode a single column.
For efficiency experiments, we aim at exploring the benefits brought by different design choices in the \(\mathsf {Starmie}\)  framework.
Thus we compare the performance of 4 methods: basic linear search (Linear), pruning based on estimated bounds (Pruning), search with an LSH index (LSH), and search with an HNSW index (HNSW).
Column encoder settings
We empirically choose the most suitable sampling method (Section REF ) and augmentation operator (introduced in Section REF  and more details in Appendix A).
For sampling methods, on the SANTOS Small and TUS Large benchmark, we find that \(\mathsf {Starmie}\)  achieves the best performance when pre-training it with the column-ordered sampling method with cell-level TF-IDF scoring function.
On TUS Small, we find that a special column-ordered sampling method, alphaHead, that sorts
tokens in alphabetical order performs the best.
For augmentation operators, we find that the drop_col operator performs the best on SANTOS Small while
drop_cell achieves the best performance on the two TUS benchmarks.

Main results
Table REF  reports the results of MAP@k and R@k on the three benchmarks for all methods.
Note that the results for SANTOS are unavailable for TUS Large because SANTOS, which requires the labeled query table intent columns [1]}, have not been evaluated on this benchmark due to the absence of annotated intent columns.
We run the experiments up to k=10 on SANTOS Small following [1]}, and up to k=60 on the TUS benchmarks, which is consistent with [3]}.
<TABLE>We can observe that \(\mathsf {Starmie}\)  outperforms the baselines across all three benchmarks.
On the SANTOS Small benchmark, \(\mathsf {Starmie}\)  achieves the highest MAP@10 of 99.3% and highest R@10 of 73.7%, outperforming \(\mathsf {SATO}\) , \(\mathsf {Sherlock}\) , SANTOS, \(D^3\) L baselines by large margins of 13%, 27%, 6.8%, and 90% respectively.
Specifically, \(\mathsf {Starmie}\)  outperforms the \(\mathsf {SingleCol}\)  variation by 11%, showing that a multi-column approach is necessary.
Similarly, on the TUS Small benchmark, \(\mathsf {Starmie}\)  outperforms the highest-achieving baseline, \(\mathsf {Sherlock}\) , by 0.7% and \(\mathsf {SingleCol}\)  variation by 4% in MAP@k.
On the TUS Large benchmark, \(\mathsf {Starmie}\)  outperforms \(\mathsf {SATO}\)  by 4% and \(\mathsf {SingleCol}\)  by 7% in MAP@k.
Thus, the \(\mathsf {Starmie}\)  approach, by capturing column context and leveraging contrastive learning in pre-training, is very effective in solving the table union search problem.
<FIGURE>Figure REF  shows the comparison of P@k and R@k \(\mathsf {Starmie}\)  with the baselines as k increases (with the same respective k's), on all benchmarks.
Throughout all values of k, \(\mathsf {Starmie}\)  outperforms all baselines for both P@k and R@k.
Since the recall might be limited to the available ground truth, we also plot the IDEAL lines denoting the maximum recall that a method can achieve on a dataset.
In Figures REF (b), (d), and (f), \(\mathsf {Starmie}\)  is closest to IDEAL, with R@10 only 1.8% below IDEAL on SANTOS Small, R@60 18.8% below IDEAL on TUS Small, and R@60 14.1% below IDEAL on TUS Large.
<FIGURE>To better understand the influence of datasets on the performance of \(\mathsf {Starmie}\) , we conduct in-depth analysis experiments to look at its performance for different settings of arity, cardinality, and percentage of numerical columns in query tables.
We evenly split the query tables into five groups, from one with the smallest numbers to that with the most.
Here we compare \(\mathsf {Starmie}\)  with alternative representation methods \(\mathsf {SATO}\)  and \(\mathsf {Sherlock}\)  that also encode columns into high-dimensional vectors.
As shown in Figure REF , as the number of rows increases, the results of \(\mathsf {Starmie}\)  remain consistently high while the performance of \(\mathsf {Sherlock}\)  and \(\mathsf {SATO}\)  decreases.
Since \(\mathsf {Starmie}\)  is unaffected by the increasing number of rows due to our efforts in exploring the table preprocessing techniques (Section REF ).
Besides, \(\mathsf {Starmie}\)  performs consistently well as the percentage of numerical columns increases, showing that \(\mathsf {Starmie}\)  can differentiate even between the semantics of numerical values.
The methods have similar trends on the TUS Small and TUS Large benchmarks as shown in Appendix B.

Scalability
<TABLE>Impacts on effectiveness. 
First of all, since some design choices might result in performance loss of effectiveness, we report their results of three evaluation metrics on the SANTOS Small benchmark.
As shown in Table REF , we compare the basic linear scan with another three design choices (above the horizontal line), as well as baselines \(\mathsf {SATO}\) , \(\mathsf {Sherlock}\) , and \(\mathsf {SingleCol}\)  (full experiment results are shown in Appendix C).
The main takeaway is that HSNW index preserves the effectivenss as much if not better than the LSH index that is widely used in previous studies about table union search, while having tremendous speed improvement.
This suggests a very promising direction for providing real-time search over massive data lakes.
Preprocessing time. 
Since \(\mathsf {Starmie}\)  requires model pre-training and model inference, in addition to possibly indexing, we provide some insights of such overhead by comparing its preprocessing time with existing systems \(D^3\) L and SANTOS that are not based on pre-trained LMs . The preprocessing time of \(\mathsf {Starmie}\)  consists of the following parts: pre-training taking 3.1 hours, model inference taking 4.4 min, and indexing taking 10-30 sec.
Meanwhile, \(D^3\) L takes 7.6 hours to create four indexes for each column feature and SANTOS takes 17 hours to create indexes using a knowledge base and the data lake.
Thus, pre-training a language model in \(\mathsf {Starmie}\)  does not incur too much overhead compared to existing systems.
<FIGURE>Time efficiency. 
We have observed that the employed design choices can speed up the online query time while sufficiently preserving the effectiveness scores.
Next we evaluate the scalability of different design choices.
In Figure REF (a), we first evaluate the four variations of \(\mathsf {Starmie}\)  on the SANTOS Large benchmark, as we increase the number of returned unionable tables k from 10 to 60.
We then evaluate their query times as the data lake size grows to its full size of \(\sim \) 11K tables / \(\sim \) 120K columns.
We also experiment on the WDC benchmark, specifically when the data lake grows to 1M tables / 5M columns (Figure REF (b)) to show the trend of each method , and when the data lake grows to 50M tables / 250M columns (Figure REF (c)).
For each method, if a data point's query time does not finish within 24 hours, then we consider it as timeout and omit the result from the corresponding figures.
Throughout all these experiments, we see that the design choice with the HNSW index leads to the best performance.
On the SANTOS Large benchmark in Figure REF (a), the k-scalability experiment shows that Pruning is 2X faster than Linear, while LSH index is 20X faster than Linear.
Meanwhile, HNSW index, which leads to an average query time of around 300 ms, is 220X faster than Linear and 11X faster than the popular LSH index.
As the data lake grows to its full size, there is a steady increase in query time of Linear and Pruning; while that of LSH index and HNSW index remain stable, with the query time of HNSW index remaining around 400 ms.
On the WDC benchmark in Figure REF (b), there is a similar trend as the data lake grows to 1M tables.
Here, the query time of HNSW index remains around 270 ms as the data lake grows, even as the other methods have a steady or slight increase in query time.
At the data lake size of 1M tables, Pruning is 2.5X faster than Linear while LSH index is 8X faster than Linear. Notably, HNSW index is 3000X faster than Linear, and 400X faster than the widely-used LSH index.
On the full WDC benchmark in Figure REF (c), Linear and Pruning time out after 1M tables, while LSH index times out after having an average query time of 2,520 sec on 10M tables.
Meanwhile, HNSW index stays consistent throughout the experiment, with the query time staying around 60 ms even as the data lake grows to its full size of 50M tables / 250M columns.
The reason is that the hierarchical graph-based structure of HNSW allows it to locate to the nearest neighbors much faster than hash-based indexes [4]}.
Overall, the design choices explored in this paper, especially HNSW index, show a great improvement in the average query time, even when the data lake grows to an immense size of 50M tables.
Meanwhile, to the best of our knowledge, the largest dataset that are evaluated by existing solutions of table union search is with only 5,000 tables / 1M columns [3]}, which has 250 times smaller number of columns.
Memory overhead. 
Lastly, we examine the relative memory overhead of \(\mathsf {Starmie}\)  with different design choices (No index denotes linear scan and pruning methods in Table REF ).
In Table REF , we report the memory usage of \(\mathsf {Starmie}\)  relative to the total data lake size (11 GB) of
SANTOS Large.
The results show that \(\mathsf {Starmie}\)  is not only scalable but also memory efficient: its variations take up around 3-7% space overhead.
The memory saving is mainly due to the condensed vector column representations of \(\mathsf {Starmie}\)  which take up only 3% of the original data lake size.
<TABLE>

Data discovery for ML tasks
Next, we conduct a case study to show that \(\mathsf {Starmie}\)  can be applied to another application scenario of dataset discovery, i.e. retrieving relevant tables to improve
the performance of downstream ML tasks.
For this case study, we consider a subset of 78k WDC tables
used in the evaluation of \(\mathsf {SATO}\)   [6]}, from which
we collect all the 4,130 tables of at least 50
rows as the data lake tables. Among these tables, we find that 25 tables of at least 200 rows
contain a numeric column called “Rating”. These 25 tables contain various types of ratings
including those for sportsmen, TV shows, US congress members, etc.
From these tables, we construct 25 regression tasks with the goal of training an ML model that predicts
“Rating” as the target column.
Since the ratings are from different domains, we normalize their values to the range \([0, 1]\) .
More details about the setting can be found in Appendix D.
For each task, we train a Gradient-Boosted Tree model [7]} with all non-target
columns as features. We featurize the textual columns using Sentence Transformers [8]}. We split each dataset into training and test sets at a ratio of 4:1. Note that the original dataset may not contain informative features.
Figure REF  shows such a dataset of US congress members.
Without useful features, the model only achieves a Mean Squared Error (MSE) of 0.1598.
To improve the model's performance on these downstream tasks, we leverage \(\mathsf {Starmie}\)  to retrieve relevant tables from the data lake to join with the datasets (i.e., the query tables) to provide additional features.
To showcase the effectiveness of \(\mathsf {Starmie}\) , we use \(\mathsf {Starmie}\) 's contextualized column embeddings to retrieve from the data lake table that contains a column having the highest cosine similarity with a non-target column of the query table.
Finally, we augment the query table by performing
a left-join with the retrieved table to ensure that the size of the augmented table stays unchanged.
We also consider two popular similarity methods for this task, Jaccard and Overlap [9]}, [10]}, as baselines
by replacing the cosine similarity scores with the corresponding similarity functions.
Table REF  summarizes the results of the 3 evaluated methods. While all 3 methods result in
performance improvement (i.e., reduction of MSE), \(\mathsf {Starmie}\)  achieves significantly better
overall improvements with a 14.75% MSE reduction, on 15/25 tasks improved, and by an average of 20.64%.
By inspecting the retrieved tables, we find that \(\mathsf {Starmie}\)  indeed retrieves qualitatively better candidate
tables. As Figure REF  shows, for the same US congress members table,
Jaccard similarity may retrieve an irrelevant table of dog competitions that also contains
a similar “State” column, but the two tables are not semantically relevant.
On the other hand, \(\mathsf {Starmie}\)  retrieves a table consisting of the amount of money raised from different
interest groups, which is a potentially relevant feature to “Rating”. Indeed, by joining
with the retrieved table by \(\mathsf {Starmie}\) , the MSE of the model drops from 0.1598 to 0.1198 (by \(>\) 25%).
<TABLE><FIGURE>
Case study: Column clustering
<TABLE>Finally we show another application scenario of \(\mathsf {Starmie}\)  in dataset discovery: column clustering.
Specifically, we apply \(\mathsf {Starmie}\)  as a column encoder to provide embeddings
for clustering all the 119,360 columns from the 78k WDC tables
used in the experiments of \(\mathsf {Sherlock}\) , \(\mathsf {SATO}\) , and others [11]}, [6]}, [13]}. These columns are annotated with 78 ground truth semantic types
such as population, city, name, etc.
The goal of column clustering is to discover clusters
of columns that are semantically relevant.
The task of semantic type detection has traditionally been solved as
a supervised multi-class classification problem
which requires significant annotated training data [13]}.
\(\mathsf {Starmie}\)  provides an unsupervised solution.
From the contextualized column embeddings, we can construct a similarity graph over all data lake columns as nodes.
We can then add undirected edges between all pairs of columns having cosine similarities above a
threshold \(\theta \)  (e.g., 0.6).
Next the column clusters can be generated via any graph clustering algorithm. We choose the connected component algorithm for efficiency and simplicity.
With \(\mathsf {Starmie}\) , the clustering algorithm generates 2,297 clusters with an average cluster size of 51.96.
We measure the quality of the clusters by the purity score, which
is
the percentage of columns assigned with the same semantic type as the majority ground truth
type of each cluster. The discovered clusters are generally of high quality as
they achieve a purity score of 51.19 while using baselines such as
\(\mathsf {Sherlock}\)  and \(\mathsf {SATO}\)  only achieves 30.5 or 37.36 purity scores when generating a similar number of clusters.
We further inspect the column values within each cluster and find that \(\mathsf {Starmie}\)  discovers
clusters of finer-grained semantic types not present in the original 78 types (more results in Appendix E).
Table REF  shows 3 such example clusters. The majority types (from the 78 original types) of columns in the 3 clusters
are “type”, “name”, and “artist” respectively. After inspecting the column values,
we can interpret the types of the 3 clusters as names of schools, names of food/grocery stores,
and names of songs.
It is difficult to discover such fine-grained types by existing methods based on supervised classification.

Related Work
Dataset Discovery
Dataset Discovery has been a hot topic in the data management community.
Earlier studies [15]}, [16]}, [17]} relied on keyword search over web tables to identify essential information.
Octopus [18]} and InfoGather [19]} focused on the problem of schema complement, an important topic in exploring web tables.
Aurum [20]}, S3D [21]} and Tableminer+ [22]}, [23]} utilized knowledge bases to identify relationship between datasets.
SemProp [24]} followed this route by leveraging ontologies and word embeddings, and Leva [25]} solved a similar problem with graph neural networks.
\(D^4\) [26]} addressed the problem of column clustering in data lake tables.
Valentine [27]} provided resources for evaluating column matching tasks.
DomainNet [28]} studied the problem of disambiguation in data lakes.
Finding related tables from data lakes is an essential task in dataset discovery.
There are two sub-tasks in this application, namely finding joinable tables and table union search [29]}.
To support finding joinable tables, earlier studies utilized syntactic similarity metrics that are widely used in the applications of string similarity search and join [30]}, [31]}, [32]}.
LSH Ensemble used containment (overlap) [33]} as the similarity metric and provided a high-dimensional similarity search based solution.
Josie [9]} employed overlap over tokens and developed an exact data-optimized solution.
PEXESO [10]} relied on cosine similarity over word embeddings and proposed indexing techniques to improve performance.
The table union search problem has been well explored recently.
Ling et al. [36]} and Lehmberg et al. [37]} illustrated the importance of finding unionable tables.
Nargesian et al. [3]} proposed the first definition and comprehensive solution for the table union search problem.
Bogatu et al. [39]} proposed the \(D^3\) L system by dividing columns into different categories.
The SANTOS  [1]} system utilized knowledge base along with binary relationships in the data lake to identify tables that share unionable columns and relationships, and it is the state-of-the-art approach in this field.
To the best of our knowledge, our work is the first solution to utilize contrastive learning techniques in table union search.

Representation Learning for Tables
Recently there are many efforts that use representation learning techniques to address problems related to tabular data.
Sherlock [11]} and Sato [6]} utilized a feature based approach to learn vector representations for tables and columns.
TURL [43]} proposed to use a pre-trained language model for web table related tasks and to come up with benchmark datasets for several tasks.
And pre-trained language models have been widely applied to different table-related applications, including entity matching [44]}, [45]}, [46]}, column type detection [47]}, [13]}, and question answering [49]}, [50]}.
Our work follows this line of study and proposes the first solution that employs a pre-trained language model in a fully unsupervised way for the problem of table union search.

Conclusion
In this paper, we mainly focused on the problem of table union search, an essential application in dataset discovery from data lakes.
We argued that it is crucial to utilize contextual information to determine whether two columns are unionable and proposed \(\mathsf {Starmie}\) , an end-to-end framework as the solution.
\(\mathsf {Starmie}\)  utilized pre-trained language model as the column encoder and employed contrastive representation learning to pre-train the model in a fully unsupervised manner.
We also developed a multi-column table Transformer encoder that can capture the contextual information from a table so as to learn contextualized column embeddings.
Moreover, we came up with a scalable search framework for aggregating column unionability scores into a table unionability score, which can make use of multiple design choices in filtering and indexing to accelerate query processing.
Experimental results on popular benchmark datasets demonstrated that \(\mathsf {Starmie}\)  significantly outperformed existing solutions for table union search.
Besides, we also showed that by leveraging efficient indexing techniques, our proposed framework can scale well to datasets that are 250 times larger than those can be supported by existing solutions so far.

Comparing different operators
Augmentation Operators
<FIGURE>To find the most effective augmentation operator used in pre-training (Section
REF ) on the SANTOS Small benchmark, we conduct experiments
comparing the MAP@k scores of different \(\mathsf {op}\) 's, shown in Figure
REF .
Specifically, we experiment with augmentation operators at different table levels, including some of the operators listed in Table REF :
Cell-Level: 

drop_cell: drops a random cell in a column

Row-Level: 

sample_row: samples a random percentage of the rows

sample_row_ordered: samples random percentage of the rows, while preserving the original order of the rows

shuffle_row: shuffles the row order

Column-Level: 

shuffle_col: shuffles the column order

drop_col: drops a random subset of column

drop_nan_col: drops columns consisting mostly of NaN's

drop_num_col: drops a random subset of numeric columns

From this ablation study, we find that the column-level operator drop_col leads to the highest MAP@k of 98%, and thus conduct the effectiveness experiments with the drop_col \(\mathsf {op}\) .

Sampling Methods
<FIGURE>We also conduct an empirical study comparing different sampling methods to find the method that best preserves the most meaningful tokens in table preprocessing. Specifically, we experiment with the following sampling methods, categorized by the level of the table. Note that all methods preserve the original order of the tokens/cells/rows, while taking unique samples:
Column-Based, Token-Level: 

head: sample first N tokens

random: randomly sample tokens

everyN: sample every Nth token

uniform: sample most frequently-occurring tokens

tfidf_token: sample tokens with highest TF-IDF

alphaHead: sample first N tokens sorted alphabetically scores

Column-Based, Cell-Level: 

tfidf_entity: sample cells in a column with highest average TF-IDF scores over its tokens

Row-Level: 

tfidf_row: samples rows with highest average TF-IDF scores over tokens in a row

row_ordered: sample and serialize tokens in a row

For the design space listed in Section REF , we reach the following conclusions from the results shown in Figure REF :
Row-ordered vs. column-ordered:  Out of all the sampling methods, the only row-ordered method is “row_ordered" (tfidf_row is column-ordered but selects cells based on the highest average TF-IDF score across the row). The column-ordered methods outperform row_ordered, with the highest column-ordered method tfidf_entity achieving a MAP@k of 99.2% while row_ordered has a MAP@k of 97.9%, thus confirming the original hypothesis.
Token/cell scoring functions:  So far we have experimented with simple scoring functions (e.g. head, random), with the most complex scoring function being TF-IDF. However, we can see that the TF-IDF-based methods, specifically tfidf_entity performs the best.
Deterministic vs. non-deterministic:  All methods except for “random" are deterministic. Since the best-performing deterministic method, tfidf_entity, outperforms the non-deterministic method “random" (which achieves a MAP@k of 97.3%) we conclude that deterministic methods are more effective.
Row alignment:  Methods tfidf_row and row-ordered preserve the row alignment. We can see that column alignment is still more effective, but this design space requires further experimentation.
All in all, this ablation study on the SANTOS Small benchmark shows that the sampling method tfidf_entity performs the best, with a MAP@k of 99.2%. Thus, we conduct our effectiveness experiments on the SANTOS Small benchmark with tfidf_entity as the sampling method.

In-depth Analyses on Effectiveness
<FIGURE>Similar to the in-depth analyses conducted for SANTOS Small benchmark, shown in Figure REF , we conduct experiments on both TUS benchmarks to analyze \(\mathsf {Starmie}\) 's performance compared to the baselines SATO and Sherlock. With the same three analyses per benchmark, and the same division of tables into 5 buckets, we first explore the MAP@k results on the TUS Small benchmark. In Figure REF , \(\mathsf {Starmie}\)  again outperforms the baselines and is robust to data containing large numbers of columns, rows, and high percentage of numeric columns. The baselines also show relatively consistent results across all buckets in the three analyses, which can be attributed to the fact that the TUS Small benchmark is derived from only 10 seed tables, and thus may not exhibit much heterogeneity. This hypothesis requires further analysis.
<FIGURE>Conducting the same analyses on the TUS Large benchmark, we see in Figure REF  that \(\mathsf {Starmie}\)  also outperforms the baselines and is consistent across all buckets. On this larger benchmark, even when the baselines' MAP@k performance drops as the number of rows increases or as the percentage of numeric columns increases, \(\mathsf {Starmie}\)  MAP@k remains consistently high, proving again that \(\mathsf {Starmie}\)  performs well across tables of various sizes and columns of different types.

Full results for Efficiency experiments
Efficiency Techniques impact on Performance
<TABLE>As we experiment with different efficiency techniques in Section REF , we also explore their effects on not only the runtimes but also the effectiveness scores. Table REF  explores the different efficiency techniques for the \(\mathsf {Starmie}\)  method, showing that they lead to great speedup while preserving the \(\mathsf {Starmie}\)  performance. In Table REF , we expand on this experiment and apply the efficiency techniques to other embeddings, specifically those of the baselines SATO, Sherlock, and SingleCol. We see that the Pruning technique speeds up the query time by 1.1-2X, while consistently preserving the performance scores perfectly. For indexing techniques, LSH index and HNSW index speed up the query times by 2.6-10X and 2-24X, respectively. Even with the fastest speedup from HNSW index, the baselines SATO and Sherlock are still slower than \(\mathsf {Starmie}\)  while having worse performance scores. As expected, SingleCol is faster as it does not have the cost from the table context. However, even with the fastest query times from the approximation technique HNSW index, \(\mathsf {Starmie}\)  still outperforms all baselines.

k-Scalability on WDC Benchmark
<FIGURE>For the scalability experiments, in Section REF  we experiment with the 4 efficiency techniques on \(\mathsf {Starmie}\)  on the SANTOS Large and WDC benchmarks. In Figure REF (a), we show the experiment on SANTOS Large as we increase k from 10 to 60. In Figure REF , we show the same experiment on 1M of the WDC tables. The trends across both figures are similar, with the HNSW index having the fastest query time, followed by LSH index, Pruning, then Linear. However here, HNSW index has a much more impressive performance with the query time remaining around 250 ms as k increases to 60, which is 3000X faster than Linear and 400X faster than LSH index. Thus, \(\mathsf {Starmie}\)  is generally robust in query time as the number of results to return increases, and is sped up the most with HNSW index.

Details of column clustering
Table REF  shows the full results
of column clustering. We use Sherlock, Sato, \(\mathsf {Starmie}\) ,
and its single-column version to generate the column
embeddings. After obtaining the column embeddings,
we construct a similarity graph by adding edges
between pairs of columns with similarity above a threshold
\(\tau = 0.6\) .
We then cluster the columns by computing their connected
components. Note that for fair comparison,
we restrict the size of clusters to be around 50
so that different methods generate similar numbers of
clusters. We measure the quality of clusters by their
purity scores, which measure how likely a column is
assigned to a cluster with the same majority semantic type
as that of the column.
Among the 4 methods, \(\mathsf {Starmie}\)  generates
clusters with the highest purity score of 51.19%.
<TABLE>
Full results for data discovery for ML
Table REF  shows the full results
of the 25 rating prediction tasks created from
4,130 WDC web tables of \(\ge \) 50 rows.
Each dataset is split into a training and a testing set
at a 4:1 ratio. The baseline methods are:
NoJoin:  Train a XGBoost model with numeric and
textual features from the original table \(S\)  only.
We featurize text attributes using
the Sentence Transformers
library [8]}.
Jaccard:  Perform an equal left-join with
a table that contains a column with the highest Jaccard
similarity with any column in the query table. Namely,
given a query table \(S = \lbrace s_1, \dots , s_n\rbrace \)  of \(n\) 
non-target columns and a data lake \(\mathcal {T}\) , we join \(S\) 
with the data lake table
\(\operatornamewithlimits{\arg \!\max }_{T \in \mathcal {T}}\left(\max _{s_i \in S, t_j \in T}(\mathsf {Jaccard}(s_i, t_j))\right) \) 
where \(\mathsf {Jaccard}(s_i ,t_j)\)  is the token-level Jaccard
similarity over tokens in query column \(s_i\)  and
data lake column \(t_j\) . Note that we exclude “rating” columns
from \(T\)  to avoid any potential label leakage.
Overlap:  In this baseline, we simply replace Jaccard
similarity from above with the overlap score, i.e.,
\(\mathsf {Overlap}(s_i, t_j) :=|\mathsf {tokens}(s_i) \cap \mathsf {tokens}(t_j)|\) .
Starmie:  For \(\mathsf {Starmie}\) , we use the learned contextualized
embeddings for measuring similarities of columns.
Since the embeddings capture the table context of each column,
we expect the resulting data tables to be semantically relevant
to the query table. More formally, let \(\mathcal {M}\)  be the learned
column encoder, \(\mathsf {Starmie}\)  joins \(S\)  with the table

*T T(si S, tj T(cos(M(si), M(tj))) +
tj T(cos(M(starget), M(tj))) ) .

Note that we use the second term with the source
target column \(s_\mathsf {target}\)  (i.e., “Rating”) to take into account
the similarity between the target column with columns from the data lake table
\(T\) .
Lastly, an important implementation detail is to make sure
that the join result has the exact same number of rows with
the query table \(S\) . This is done by properly left-joining with
the data lake table \(T\) . This is done via the pandas
DataFrame command:
  # de-duplicate table T on column t_j
  T = T.drop_duplicates(subset=[t_j]).set_index(t_j)
  # left-join on the column pair (s_i, t_j)
  S.join(T, on=s_i)
<TABLE>