Update README.md
Browse files
README.md
CHANGED
|
@@ -121,14 +121,6 @@ model-index:
|
|
| 121 |
|
| 122 |
Quantized to 4 bpw precision and tested using the `mlx_lm` utility on a 64GiB URAM M1 Max.
|
| 123 |
|
| 124 |
-
## Notes on using:
|
| 125 |
-
|
| 126 |
-
Requires and optimized for Apple Silicon. Fast enough for rapid back-and-forth as long as it fits on your URAM.
|
| 127 |
-
|
| 128 |
-
I tried to serve this with `mlx_lm.serve` per usual, but I got python string indexing errors no matter what I did. It works fine with LM Studio in OpenAI mode.
|
| 129 |
-
|
| 130 |
-
I used this with SillyTavern, it worked well.
|
| 131 |
-
|
| 132 |
See [original model](https://huggingface.co/anthracite-org/magnum-v2-72b) for further details.
|
| 133 |
|
| 134 |
Larger, 8bpw quants available at [mlx-community](https://huggingface.co/mlx-community/magnum-v2-72b).
|
|
|
|
| 121 |
|
| 122 |
Quantized to 4 bpw precision and tested using the `mlx_lm` utility on a 64GiB URAM M1 Max.
|
| 123 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 124 |
See [original model](https://huggingface.co/anthracite-org/magnum-v2-72b) for further details.
|
| 125 |
|
| 126 |
Larger, 8bpw quants available at [mlx-community](https://huggingface.co/mlx-community/magnum-v2-72b).
|