Q:

Blend (smooth) edges for a textured polygon in OpenGL ES 2.0

If you have a complex shape (polygon) with a texture mapped to it with GL_REPEAT in OpenGL ES 2.0, and all the points that form the edges of the polygon, is it possible to do something that will make the edges blend with the background?
I mean something that will make the edges smoother, so the textured shape will not look jagged around the edges and corners.

A:

Well, I'm not sure if I understood what you are trying, but you can always give your texture an alpha channel to control the opacity of the polygon, by using alpha blending, thus reducing the alpha values near the edges of the polygon. Of course in this case the texture is strongly coupled to the polygon and it won't work for a texture that is repeated multiple times across the entire polygon. But in this case you can just use a second texture that only contains the alpha channel and is mapped onto the polygon differently (uses different texture coordinates).
Another option, which may be easier to achieve when you have the polygon tessellated finer, with some inner vertices and some boundary vertices, would be to just give each vertex an additional attribute, that contains the vertex' alpha value. So you give the inner vertices an alpha of 1 (totally opaque) and the boundary vertices an alpha of 0 (totally transparent). You then give this value from the vertex shader to the fragment shader as a varying variable and this way get the alpha (or opacity) interpolated across the triangle from the interior to the boundary and thus get a smooth transition from opaque to transparent (though only linearly interpolated, which may be sufficient in your case). In the fragment shader you just pass it through to the output color's alpha to be used in alpha blending.
These are just general ideas, which assume that you know how shaders work and how to use alpha blending. Feel free to ask a more specific question if this answer is not sufficient.

