1. A method of controlling a backlight module, comprising: receiving an image, wherein the image has a plurality of regions and the image is displayed by a plurality of color backlights provided by a backlight module; calculating a first average of a characteristic in a specific region of the image, wherein the specific region is one of the regions; calculating a characteristic weight according to the first average of the characteristic and a weight function; and adjusting one of the color backlights provided to the specific region according to the characteristic weight by calculating a weight sum of the one of the color backlights provided to the specific region and a specific value, wherein the weight sum equals to Lpcx(1−W)+L×W, Lpc is a characteristic of each of the color backlights of Lpc(R) for first primary color, Lpc(G) for second primary color and Lpc(B) for third primary color, provided to the specific region, L is the specific value, and W is the characteristic weight, wherein the specific value L is a maximum of Lpc(R), Lpc(G) and Lpc(B) provided to the specific region, or is an average of Lpc(R), Lpc(G) and Lpc(B) provided to the specific region.