first_mask = np.concatenate(([True], ref_sort[:-1] != ref_sort[1:]))
first_idx, = np.nonzero(first_mask)
cnts = np.diff(np.concatenate((first_idx, [ref_sort.size])))
cnts
unq = ref_sort[first_mask]
unq
Fastest way of calculating mean values for each indices