1. A method comprising: generating, by a processor, a label map by labeling pixels in an image frame based on a closest color cluster center to the pixels; determining a percentage of a first type of texture pixels and a percentage of a second type of texture pixels based on the label map to generate a texture map of the image frame; updating the texture map by updating texture probability as a local average with an updating factor for a particular difference between the percentage of the first type of texture pixels and the percentage of the second type of texture pixels; generating a directional edge map based on a data correlation analysis by removing a correlation response in texture regions based on the texture map and the label map; generating a gain map with boundaries and smoothing using localized pixels based on information from one or more of the label map, the texture map, and the directional edge map; and generating an enhancement controlled upscaled image frame based on using one or more of the label map, the texture map, the directional edge map and the gain map as a weight factor.