Once we have our trained latent space, we then identify distinct segments of the image by clustering the embedding vectors. Because the pixels have been mapped to the embedding space such that they now form a locally Euclidean data manifold, we are able to use spectral k-means clustering to capture similar pixels [1]}. Notably, this relatively simple clustering strategy would not be sufficient on the ambient pixel data as is, but the learned embedding has these convenient properties like being locally Euclidean that the ambient data lacks. The choice of k for this clustering is naturally related to the complexity of the image and the desired segmentation, but heuristically we found simply using a fixed value of 10 achieved satisfactory results on all of the experiments considered in this work.
