1. A method comprising: applying, by a processing device, a machine-learned model to audio-visual content features of first content item segments of a first content item, the machine-learned model trained based on user interaction signals and audio-visual content features of a training set of training segments of training content items, wherein the user interaction signals comprise scrubbing in the training segments; calculating, based on applying the machine-learned model, a salience score for each of the first segments of the first content item; and selecting, based on the calculated salience scores, one of the first content item segments as a starting point for playback of the first content item.