In this work, we aim to learn efficient radiograph representations from scratch that train on fewer images. Recently, Desai et al. [1]} introduced the VirTex framework showing that pretraining an image encoder via captioning leads to high-quality representations of natural images, which match or exceed those learned through ImageNet-supervised and contrastive pretraining, despite requiring fewer images for training. Inspired by the VirTex model's data-efficiency, we investigate radiograph representations learned from an image-captioning pretraining task and their potential for transfer to downstream medical image classification tasks. Specifically, we examine downstream performance under conditions of limited labeled data, comparing our model (RadTex) to other pretrained models. We find that (A) RadTex pretraining outperforms ImageNet and in-domain supervised pretraining methods when fewer than 1000 examples are available for training, (B) decreases in performance of just 0.05 AUC and 0.01 AUCPR are observed for RadTex when training data is reduced from tens of thousands of examples to 100 examples, and (C) pretraining RadTex on in-domain image-text datasets with at least 100K examples is necessary to build transferable and interpretable representations.
