1. A computer-implemented process for compressing a medical image, comprising using a computer to perform the following process actions: computing a set of base images from a larger group of source medical images belonging to a same category as the medical image being compressed, wherein the base image set substantially captures the variance exhibited amongst the group of source images and wherein the base images are orthogonal to each other; computing a difference between the medical image being compressed and the base images; compressing the difference and producing difference data representing the compressed medical image; and storing or transferring the compressed difference data.