Due to the lack of large-scale segmentation datasets, direct training of the reported network results in low segmentation accuracy (as validated in Fig. REF ). Considering that large-scale natural image datasets are available, we derived a two-stage training strategy that intrinsically corresponds to the transfer learning theory [1]}. In the first training stage, we apply large-scale natural image datasets to learn image-related prior features, by controlling that the gradient flow only goes back through the encoder and the feature extraction block. This helps better extract target features and improve sensing performance. In the second training stage, the gradient flow goes back through the entire network, namely that both the encoder and the decoder are updated at the same time. This operation transfers the learned features to the segmentation datasets, and optimizes the learning efficiency of the model. Once converged, the optimized filters of the encoder are set as light modulation patterns. Correspondingly, the single-pixel detector collects a sequence of coupled measurements that are input into the sensing decoder, which outputs the final segmentation results.
