1. In a digital medium environment to control patch-usage in image synthesis based on matching errors associated with potential source-to-target patch assignments between a source image and a target image, a method comprising: determining an error budget using a curve fitted to a sorted set of the matching errors, the error budget being usable to identify feasible patch assignments from the potential source-to-target patch assignments; assigning source patches from the source image to target patches in the target image using the error budget and uniform patch-usage enforcement; assigning at least one of the source patches to at least one additional target patch in the target image based on the error budget; and synthesizing an image based on the source patches assigned to the target patches.