In a small bowl, stir together confectioners' sugar and unsweetened cocoa.
In a large mixer bowl with mixer on MEDIUM, beat butter and vanilla extract until creamy, about 2 minutes.
Reduce mixer speed to LOW, gradually beat in confectioners' sugar mixture with milk until blended, about 1 minute. Increase speed to MEDIUM-HIGH, beat 1 minute until creamy. Add more milk if mixture is too thick.
Spread onto cupcakes or cake.