region:us
# Bible Embeddings

## Setup

```sh
direnv allow
just install-deps
```

## Usage

```sh
just embed bsb BAAI/bge-large-en
```

This will create a directory structure with json files like such:

```
embeddings
├── Exodus
│   ├── 001
│   │   └── bsb-bge-large-en.json
│   ├── 002
│   │   └── bsb-bge-large-en.json
│   ├── 003
│   │   └── bsb-bge-large-en.json
│   └── etc
├── Genesis
│   ├── 001
│   │   └── bsb-bge-large-en.json
│   ├── 002
│   │   └── bsb-bge-large-en.json
│   ├── 003
│   │   └── bsb-bge-large-en.json
│   └── etc
└── Leviticus
    ├── 001
    │   └── bsb-bge-large-en.json
    ├── 002
    │   └── bsb-bge-large-en.json
    ├── 003
    │   └── bsb-bge-large-en.json
    └── etc
```

## Models

For high-level information about these models, see the [MTEB Leaderboard](https://huggingface.co/spaces/mteb/leaderboard).

- [BAAI/bge-large-en](https://huggingface.co/BAAI/bge-large-en)
  - 1024 dimensions
  - BSB: 899.769 MB
- [BAAI/bge-base-en](https://huggingface.co/BAAI/bge-base-en)
  - 768 dimensions
  - BSB: 674.226 MB
- [BAAI/bge-small-en](https://huggingface.co/BAAI/bge-small-en)
  - 384 dimensions
  - BSB: 338.351 MB
- [thenlper/gte-large](https://huggingface.co/thenlper/gte-large)
  - 1024 dimensions
  - BSB: 899.583 MB
- [thenlper/gte-base](https://huggingface.co/thenlper/gte-base)
  - 768 dimensions
  - BSB: 674.492 MB
- [thenlper/gte-small](https://huggingface.co/thenlper/gte-small)
  - 384 dimensions
  - BSB: 338.211 MB
- [intfloat/e5-large-v2](https://huggingface.co/intfloat/e5-large-v2)
  - 1024 dimensions
  - BSB: 898.283 MB
- [intfloat/e5-base-v2](https://huggingface.co/intfloat/e5-base-v2)
  - 768 dimensions
  - BSB: 672.98 MB
- [intfloat/e5-small-v2](https://huggingface.co/intfloat/e5-small-v2)
  - 384 dimensions
  - BSB: 336.522 MB
- [sentence-transformers/all-MiniLM-L6-v2](https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2)
  - 384 dimensions
  - BSB: 336.741 MB
- [text-embedding-ada-002](https://platform.openai.com/docs/guides/embeddings)
  - 1536 dimensions
  - BSB: 1359.56 MB

## Query Examples

✅ denotes accurate result.

| Translation | Query                             | text-embedding-ada-002                                                 | all-MiniLM-L6-v2                                                      | e5-large-v2                                                           | e5-base-v2                                                               | e5-small-v2                                                             | gte-large                                                             | gte-base                                                            | gte-small                                                                   | bge-large-en                                                         | bge-base-en                                                                  | bge-small-en                                                         |
| ----------- | --------------------------------- | ---------------------------------------------------------------------- | --------------------------------------------------------------------- | --------------------------------------------------------------------- | ------------------------------------------------------------------------ | ----------------------------------------------------------------------- | --------------------------------------------------------------------- | ------------------------------------------------------------------- | --------------------------------------------------------------------------- | -------------------------------------------------------------------- | ---------------------------------------------------------------------------- | -------------------------------------------------------------------- |
| BSB         | God is love                       | 1 John 4:16 (0.8935) ✅, 1 John 4:8 (0.8930) ✅, 1 John 4:7 (0.8854)   | 1 John 4:16 (0.6814) ✅, 1 John 4:8 (0.6255) ✅, 1 John 4:12 (0.6056) | 1 John 4:8 (0.8575) ✅, 1 John 4:16 (0.8354) ✅, Psalm 115:3 (0.8270) | 1 John 4:8 (0.8625) ✅, 1 John 4:12 (0.8353), 1 John 4:7 (0.8301)        | 1 John 4:16 (0.8750) ✅, 1 John 4:8 (0.8741) ✅, 1 John 4:12 (0.8573)   | 1 John 4:8 (0.9222) ✅, 1 John 4:16 (0.8993) ✅, 1 John 4:10 (0.8941) | 1 John 4:8 (0.9417), 1 John 4:16 (0.9147), 1 John 4:7 (0.8977)      | 1 John 4:8 (0.9231) ✅, 1 John 4:16 (0.9065) ✅, 1 Corinthians 8:3 (0.8928) | 1 John 4:8 (0.9084) ✅, 1 John 4:16 (0.9073) ✅, 1 John 4:7 (0.8963) | 1 John 4:16 (0.9078) ✅, 1 John 4:8 (0.8906) ✅, 1 John 4:7 (0.8882)         | 1 John 4:16 (0.9193) ✅, Psalm 73:1 (0.9131), 1 John 4:8 (0.9069) ✅ |
| BSB         | They meant bad but God meant good | Genesis 50:20 (0.8718) ✅, Jonah 3:10 (0.8536), Hebrews 11:40 (0.8433) | Luke 18:19 (0.5357), Mark 10:18 (0.5457), 3 John 11 (0.5030)          | Psalm 106:21 (0.8293), Romans 12:21 (0.8288), Romans 1:25 (0.8284)    | Genesis 50:20 (0.8596) ✅, 1 Peter 3:17 (0.8507), Hebrews 11:16 (0.8484) | 1 Peter 3:17 (0.8570), Isaiah 5:20 (0.8536), Ecclesiastes 7:18 (0.8510) | Jonah 3:10 (0.8847), Luke 18:19 (0.8835), Mark 10:18 (0.8835)         | Jonah 3:10 (0.8689), Jeremiah 18:20 (0.8668), Romans 14:16 (0.8657) | Psalm 99:8 (0.8747), 3 John 1:11 (0.8747), Galatians 1:24 (0.8744)          | Amos 5:14 (0.8809), Proverbs 12:2 (0.8797), 3 John 1:11 (0.8793)     | 3 John 11 (0.8639), Acts 13:32 (0.8625), Titus 3:8 (0.8618)                  | 3 John 11 (0.8871), Psalm 73:1 (0.8860), Psalm 76:9 (0.8857)         |
| BSB         | God loves the world               | John 3:16 (0.8704) ✅, 1 John 4:19 (0.8662), Psalm 67:7 (0.8643)       | Psalm 33:5 (0.6514), John 3:16 (0.6381) ✅, Psalm 47:7 (0.5882)       | Psalm 33:5 (0.8530), John 3:16 (0.8503) ✅, Psalm 146:8 (0.8409)      | Psalm 33:5 (0.8802), John 3:16 (0.8565) ✅, Psalm 146:8 (0.8501)         | Psalm 33:5 (0.8835), Psalm 47:7 (0.8734), Genesis 1:17 (0.8585)         | John 3:16 (0.8905), Psalm 33:5 (0.8889), 1 John 2:15 (0.8839)         | Psalm 33:5 (0.9002), 1 Corinthians 8:3 (0.8939), John 3:16 (0.8747) | John 3:16 (0.8953) ✅, Psalm 47:7 (0.8923), Psalm 33:5 (0.8921)             | Psalm 33:5 (0.8973), Isaiah 12:5 (0.8839), Psalm 115:3 (0.8817)      | Psalm 33:5 (0.8953), 1 Chronicles 16:31 (0.8755), 1 Corinthians 8:3 (0.8755) | Psalm 47:7 (0.9070), Psalm 24:1 (0.9059), Psalm 76:9 (0.8996)        |
| BSB         | God loved the world               | John 3:16 (0.8751) ✅, John 1:2 (0.8720), 1 John 4:19 (0.8671)         | John 3:16 (0.6464) ✅, John 3:17 (0.5846), Genesis 1:10 (0.5638)      | John 3:16 (0.8627) ✅, Psalm 33:5 (0.8456), Psalm 47:7 (0.8388)       | John 3:16 (0.8685) ✅, Psalm 33:5 (0.8631), Psalm 136:6 (0.8472)         | Psalm 33:5 (0.8738), John 3:16 (0.8725) ✅, Psalm 47:7 (0.8676)         | John 3:16 (0.8965), 1 John 4:9 (0.8838), Jeremiah 10:12 (0.8798)      | John 3:16 (0.8950) ✅, Psalm 33:5 (0.8871), Psalm 136:6 (0.8722)    | John 3:16 (0.9107) ✅, John 3:17 (0.8872), Genesis 1:10 (0.8866)            | Psalm 33:5 (0.8850), John 3:16 (0.8824) ✅, Genesis 1:1 (0.8824)     | John 3:16 (0.8864) ✅, Psalm 33:5 (0.8853), Acts 2:47 (0.8765)               | Psalm 76:9 (0.9071), Genesis 1:1 (0.9053), John 3:16 (0.8979) ✅     |
