Update README.md
Browse files
README.md
CHANGED
@@ -16,6 +16,23 @@ tags:
|
|
16 |
- neural-compressor
|
17 |
---
|
18 |
# INT8 GPT-J 6B
|
|
|
|
|
19 |
GPT-J 6B is a transformer model trained using Ben Wang's [Mesh Transformer JAX](https://github.com/kingoflolz/mesh-transformer-jax/). "GPT-J" refers to the class of model, while "6B" represents the number of trainable parameters.
|
20 |
|
21 |
-
This int8 PyTorch model is generated by [
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
16 |
- neural-compressor
|
17 |
---
|
18 |
# INT8 GPT-J 6B
|
19 |
+
|
20 |
+
## Model Description
|
21 |
GPT-J 6B is a transformer model trained using Ben Wang's [Mesh Transformer JAX](https://github.com/kingoflolz/mesh-transformer-jax/). "GPT-J" refers to the class of model, while "6B" represents the number of trainable parameters.
|
22 |
|
23 |
+
This int8 PyTorch model is generated by [intel-extension-for-transformers](https://github.com/intel/intel-extension-for-transformers).
|
24 |
+
| Package | Version |
|
25 |
+
|----------------------|------------|
|
26 |
+
| intel-extension-for-transformers| a4aba8ddb07c9b744b6ac106502ec059e0c47960 |
|
27 |
+
| neural-compressor | 2.4.1 |
|
28 |
+
| torch | 2.1.0+cpu |
|
29 |
+
| intel-extension-for-pytorch | 2.1.0 |
|
30 |
+
| transformers | 4.32.0 |
|
31 |
+
|
32 |
+
## Evaluation results
|
33 |
+
Evaluating the accuracy of the optimized model of gpt-j-6b using the lambada_openai dataset in lm_eval.
|
34 |
+
|
35 |
+
| Dtype | Dataset | Precision |
|
36 |
+
|------ |--------|--------|
|
37 |
+
| FP32 |Lambada_openai | 0.6831 |
|
38 |
+
| INT8 |Lambada_openai | 0.6835 |
|