We obtain the similarity scores matrices \(H_{s_i} \in R^{q \times g}\)  between the query and gallery for every single model as the results, where \(s_i\) , \(q\) , and \(g\)  are the \(i\) -th single model, the number of queries and the galleries respectively. Given \(H \in \lbrace H_{s_1}, H_{s_2},... , H_{s_n}\rbrace \) , we find the set of \(W \in \lbrace w_1, w_2, ..., w_n\rbrace \)  to obtain the weighted sum of the scores \(H\) . The Bayesian optimization provided by the hyperopt package is used for the task. First, \(n\)  uniform distribution (0, 1) was set as the weight search space for the ensemble given outputs of \(n\)  single models. The randomly sampled weights were fed to Tree-structured Parzen Estimator (TPE) algorithm [1]} that finds the optimal \(W\)  with the objective set to \(f(W) = (R@10 + R@50)/2\) . Finally, the output score in the ensemble is also a form of a score matrix \(H_{best} = w_1H_1 + w_2H_2 + ... w_nH_n \in R^{q \times g}\) . We treat the best score \(H_{best}\)  at the moment as an output of a single model and maximize the best score by including \(H_{best}\)  as a candidate for every trial of the ensemble. It is an iterative accumulation of the single model result. However, we found that the excessive accumulation breaks the tendency between the results on the test and the validation because the score gap between test and validation would be accumulated as well.
