llama.cpp: llama_model_load: llama_model_load: tensor 'tok_embeddings.weight' has wrong size in model file main
The fix is this:
diff --git a/main.cpp b/main.cpp
index 3321818..3fa487b 100644
--- a/main.cpp
+++ b/main.cpp
@@ -34,8 +34,8 @@ static const int EOS_TOKEN_ID = 2;
// determine number of model parts based on the dimension
static const std::map<int, int> LLAMA_N_PARTS = {
{ 4096, 1 },
- { 5120, 2 },
- { 6656, 4 },
+ { 5120, 1 },
+ { 6656, 1 },
{ 8192, 8 },
};
@@ -43,10 +43,10 @@ static const std::map<int, int> LLAMA_N_PARTS = {
struct llama_hparams {
int32_t n_vocab = 32000;
int32_t n_ctx = 512; // this is provided as user input?
- int32_t n_embd = 4096;
+ int32_t n_embd = 5120;
int32_t n_mult = 256;
- int32_t n_head = 32;
- int32_t n_layer = 32;
+ int32_t n_head = 40;
+ int32_t n_layer = 40;
int32_t n_rot = 64;
int32_t f16 = 1;
};
its work for me) thanks!
Glad you got it to work. However, just one little comment. It's not anything major that will break stuff.
llama_hparams doesn't need to be changed since that's the default Paramus for 7B. If you want to change it, here are the correct values for 30B:
int32_t n_embd: 6656
int32_t n_head: 52
int32_t n_layer: 60
how can I recompile the main.cpp once i make the corrections? The "npx dalai alpaca install" command takes the old main.cpp from a repository.
how can I recompile the main.cpp once i make the corrections? The "npx dalai alpaca install" command takes the old main.cpp from a repository.
Modify alpaca.js in Dalai and change https://github.com/candywrap/alpaca.cpp.git to https://github.com/ItsPi3141/alpaca.cpp.git