I suggest to reduce the gauge of the stitches by using a smaller hook size.
by reducing the hook size from 3mm to 2.5mm will reduce the hat size by 16.7%, with the assumption that the tension is about the same.