Update README.md
Browse files
README.md
CHANGED
@@ -144,23 +144,19 @@ We directly adapted this mechanism from Graves ([2016](#graves-2016)). At each i
|
|
144 |
The architecture is not yet directly included in the Transformers library. The code used for pre-training is available in the following [github repository](https://github.com/AntoineSimoulin/adaptive-depth-transformers). So you should install the code implementation first:
|
145 |
|
146 |
```bash
|
147 |
-
pip install git+https://github.com/AntoineSimoulin/adaptive-depth-transformers
|
148 |
```
|
149 |
|
150 |
Then you can use the model directly.
|
151 |
|
152 |
```python
|
153 |
-
import
|
154 |
-
sys.path.append('adaptative-depth-transformers')
|
155 |
-
|
156 |
-
from modeling_albert_act_tf import TFAlbertActModel
|
157 |
-
from modeling_albert_act import AlbertActModel
|
158 |
-
from configuration_albert_act import AlbertActConfig
|
159 |
from transformers import AlbertTokenizer
|
160 |
|
161 |
-
|
|
|
162 |
_ = model.eval()
|
163 |
-
|
164 |
inputs = tokenizer("a lump in the middle of the monkeys stirred and then fell quiet .", return_tensors="pt")
|
165 |
outputs = model(**inputs)
|
166 |
outputs.updates
|
|
|
144 |
The architecture is not yet directly included in the Transformers library. The code used for pre-training is available in the following [github repository](https://github.com/AntoineSimoulin/adaptive-depth-transformers). So you should install the code implementation first:
|
145 |
|
146 |
```bash
|
147 |
+
!pip install git+https://github.com/AntoineSimoulin/adaptive-depth-transformers$
|
148 |
```
|
149 |
|
150 |
Then you can use the model directly.
|
151 |
|
152 |
```python
|
153 |
+
from act import AlbertActConfig, AlbertActModel, TFAlbertActModel
|
|
|
|
|
|
|
|
|
|
|
154 |
from transformers import AlbertTokenizer
|
155 |
|
156 |
+
tokenizer = AlbertTokenizer.from_pretrained('asi/albert-act-base')
|
157 |
+
model = AlbertActModel.from_pretrained('asi/albert-act-base')
|
158 |
_ = model.eval()
|
159 |
+
|
160 |
inputs = tokenizer("a lump in the middle of the monkeys stirred and then fell quiet .", return_tensors="pt")
|
161 |
outputs = model(**inputs)
|
162 |
outputs.updates
|