SAELens
Tom Lieberum commited on
Commit
4cdad7f
1 Parent(s): 2232397

add embedding sae parameters

Browse files
.gitattributes CHANGED
@@ -33,3 +33,5 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
 
 
 
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
36
+ embedding/width_4k filter=lfs diff=lfs merge=lfs -text
37
+ embedding/width_4k/*.npz filter=lfs diff=lfs merge=lfs -text
README.md CHANGED
@@ -16,6 +16,7 @@ See our [landing page](https://huggingface.co/google/gemma-scope) for details on
16
  - `gemma-scope-`: See 1.
17
  - `2b-pt-`: These SAEs were trained on Gemma v2 2B base model.
18
  - `res`: These SAEs were trained on the model's residual stream.
 
19
 
20
 
21
  # 3. Which SAE is in the [Neuronpedia demo](https://www.neuronpedia.org/gemma-scope)?
@@ -37,4 +38,4 @@ https://huggingface.co/ArthurConmyGDM
37
 
38
  # Citation
39
 
40
- Paper: https://arxiv.org/abs/2408.05147
 
16
  - `gemma-scope-`: See 1.
17
  - `2b-pt-`: These SAEs were trained on Gemma v2 2B base model.
18
  - `res`: These SAEs were trained on the model's residual stream.
19
+ - We include experimental SAEs trained on token embeddings in the ./embedding folder.
20
 
21
 
22
  # 3. Which SAE is in the [Neuronpedia demo](https://www.neuronpedia.org/gemma-scope)?
 
38
 
39
  # Citation
40
 
41
+ Paper: https://arxiv.org/abs/2408.05147
embedding/width_4k/average_l0_111/params.npz ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:8da0e023ae3c2e8263a2ab090aca4956966835750b8c51a5b878608cb1154b7a
3
+ size 75540696
embedding/width_4k/average_l0_21/params.npz ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:91acd58b05d718a8ba19c82a456d7a34612b154c38cb76d90f927229994d2d70
3
+ size 75540696
embedding/width_4k/average_l0_44/params.npz ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a9d3e26b87d14ba1efa6c134fef90ce0d97e45ee362d124b7577c022a9b6b1d1
3
+ size 75540696
embedding/width_4k/average_l0_6/params.npz ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:9e019fbac1b4d4f50da03fc6c1461aad0020efabcab1fadf082d81861db44ce2
3
+ size 75540696