Edit model card

This model was derived from the bert-base-uncased checkpoint by replacing the GELU with ReLU activation function and continued pre-training to adapt it to the change of the activation function.

Downloads last month
9
Safetensors
Model size
110M params
Tensor type
I64
·
F32
·
Inference Examples
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social visibility and check back later, or deploy to Inference Endpoints (dedicated) instead.

Datasets used to train mpiorczynski/relu-bert-base-uncased