1. A video encoding system comprising: an input receiving video input data; one or more processors configured to implement a sample adaptive offset (SAO) video encoder comprising: a statistics collection (SC) block configured to receive a set of deblocked pixels and a set of original pixels, the SC block configured to: categorize each deblocked pixel of the set of deblocked pixels in at least one of a plurality of band categories and a plurality of edge categories; count a number of deblocked pixels in each category of the plurality of band categories and the plurality of edge categories; and estimate an error in each category from a difference between a deblocked pixel of the set of deblocked pixels and corresponding original pixel of the set of original pixels; a rate distortion optimization (RDO) block coupled to the SC block, the RDO block is configured to: determine a sign of the error in each category; determine a set of candidate offsets associated with each category, a value of a candidate offset of the set of candidate offsets is between a maximum candidate offset and a minimum candidate offset; estimate a rate distortion (RD) cost associated with each candidate offset of the set of candidate offsets determined for each category; and determine a minimum RD cost of the estimated RD costs for each category; wherein the SAO video encoder encodes the video input data based at least partially on the minimum RD cost.