Error due to 'hidden_size' and 'num_heads'
#1
by
noCOMET
- opened
ValueError: hidden_size must be divisible by num_heads (got hidden_size: 2048 and num_heads: 24).
'hidden_size' and 'num_heads' seem to be same across all of the families of OpenMoE model. However, error message above appears when running the models. Is there something to modify for 2 parameters or other parameters?