readme update

README.md

@@ -14,6 +14,8 @@ tags:
 - Model creator: [Maykeye](https://huggingface.co/Maykeye)
 - Original model: [TinyLLama-v0](https://huggingface.co/Maykeye/TinyLLama-v0)
 
+If interested in the internal content of this model you can check [Tinyllama-4.6M-v0.0-F16.dump.md](./Tinyllama-4.6M-v0.0-F16.dump.md) included in this repo.
+
 ## Description
 
 * This repo is targeted towards:
@@ -73,26 +75,12 @@ make: Nothing to be done for 'all'.
 make: Nothing to be done for 'all'.
 ~/huggingface/TinyLLama-v0-5M-F16-llamafile
 == What is our llamafile name going to be? ==
-We will be aiming to generate Tinyllama-5M-v0.2-F16.llamafile
+maykeye_tinyllama/Tinyllama-4.6M-v0.0-F16.gguf
+We will be aiming to generate Tinyllama-4.6M-v0.0-F16.llamafile
 == Convert from safetensor to gguf ==
 INFO:hf-to-gguf:Loading model: maykeye_tinyllama
-INFO:hf-to-gguf:gguf: loading model part 'model.safetensors'
-INFO:gguf.gguf_writer:gguf: Will write to maykeye_tinyllama/Tinyllama-5M-v0.2-F16.gguf
 INFO:gguf.gguf_writer:gguf: This GGUF file is for Little Endian only
-INFO:hf-to-gguf:Set meta model
-INFO:hf-to-gguf:Set model parameters
-INFO:hf-to-gguf:gguf: context length = 2048
-INFO:hf-to-gguf:gguf: embedding length = 64
-INFO:hf-to-gguf:gguf: feed forward length = 256
-INFO:hf-to-gguf:gguf: head count = 16
-INFO:hf-to-gguf:gguf: rms norm epsilon = 1e-06
-INFO:hf-to-gguf:gguf: file type = 1
-INFO:hf-to-gguf:Set model tokenizer
-INFO:gguf.vocab:Setting special token type bos to 1
-INFO:gguf.vocab:Setting special token type eos to 2
-INFO:gguf.vocab:Setting special token type unk to 0
-INFO:gguf.vocab:Setting special token type pad to 0
-INFO:hf-to-gguf:Exporting model to 'maykeye_tinyllama/Tinyllama-5M-v0.2-F16.gguf'
+INFO:hf-to-gguf:Exporting model...
 INFO:hf-to-gguf:gguf: loading model part 'model.safetensors'
 INFO:hf-to-gguf:output.weight,              torch.bfloat16 --> F16, shape = {64, 32000}
 INFO:hf-to-gguf:token_embd.weight,          torch.bfloat16 --> F16, shape = {64, 32000}
@@ -169,44 +157,64 @@ INFO:hf-to-gguf:blk.7.attn_output.weight,   torch.bfloat16 --> F16, shape = {64,
 INFO:hf-to-gguf:blk.7.attn_q.weight,        torch.bfloat16 --> F16, shape = {64, 64}
 INFO:hf-to-gguf:blk.7.attn_v.weight,        torch.bfloat16 --> F16, shape = {64, 64}
 INFO:hf-to-gguf:output_norm.weight,         torch.bfloat16 --> F32, shape = {64}
-Writing: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████| 9.24M/9.24M [00:00<00:00, 139Mbyte/s]
-INFO:hf-to-gguf:Model successfully exported to 'maykeye_tinyllama/Tinyllama-5M-v0.2-F16.gguf'
+INFO:hf-to-gguf:Set meta model
+INFO:hf-to-gguf:Set model parameters
+INFO:hf-to-gguf:gguf: context length = 2048
+INFO:hf-to-gguf:gguf: embedding length = 64
+INFO:hf-to-gguf:gguf: feed forward length = 256
+INFO:hf-to-gguf:gguf: head count = 16
+INFO:hf-to-gguf:gguf: rms norm epsilon = 1e-06
+INFO:hf-to-gguf:gguf: file type = 1
+INFO:hf-to-gguf:Set model tokenizer
+INFO:gguf.vocab:Setting special token type bos to 1
+INFO:gguf.vocab:Setting special token type eos to 2
+INFO:gguf.vocab:Setting special token type unk to 0
+INFO:gguf.vocab:Setting special token type pad to 0
+INFO:hf-to-gguf:Set model quantization version
+INFO:gguf.gguf_writer:Writing the following files:
+INFO:gguf.gguf_writer:maykeye_tinyllama/Tinyllama-4.6M-v0.0-F16.gguf: n_tensors = 75, total_size = 9.2M
+Writing: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 9.24M/9.24M [00:00<00:00, 83.7Mbyte/s]
+INFO:hf-to-gguf:Model successfully exported to maykeye_tinyllama/Tinyllama-4.6M-v0.0-F16.gguf
 == Generating Llamafile ==
-== Test Output ==
+== Test Output ./Tinyllama-4.6M-v0.0-F16.llamafile ==
 note: if you have an AMD or NVIDIA GPU then you need to pass -ngl 9999 to enable GPU offloading
-main: llamafile version 0.8.6
-main: seed  = 1717436617
-llama_model_loader: loaded meta data with 29 key-value pairs and 75 tensors from Tinyllama-5M-v0.2-F16.gguf (version GGUF V3 (latest))
+main: llamafile version 0.8.9
+main: seed  = 1721461448
+llama_model_loader: loaded meta data with 33 key-value pairs and 75 tensors from Tinyllama-4.6M-v0.0-F16.gguf (version GGUF V3 (latest))
 llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output.
 llama_model_loader: - kv   0:                       general.architecture str              = llama
-llama_model_loader: - kv   1:                               general.name str              = TinyLLama
-llama_model_loader: - kv   2:                             general.author str              = mofosyne
-llama_model_loader: - kv   3:                            general.version str              = v0.2
-llama_model_loader: - kv   4:                                general.url str              = https://huggingface.co/mofosyne/TinyL...
+llama_model_loader: - kv   1:                               general.type str              = model
+llama_model_loader: - kv   2:                               general.name str              = TinyLLama
+llama_model_loader: - kv   3:                             general.author str              = Maykeye
+llama_model_loader: - kv   4:                            general.version str              = v0.0
 llama_model_loader: - kv   5:                        general.description str              = This gguf is ported from a first vers...
-llama_model_loader: - kv   6:                            general.license str              = apache-2.0
-llama_model_loader: - kv   7:                         general.source.url str              = https://huggingface.co/Maykeye/TinyLL...
-llama_model_loader: - kv   8:      general.source.huggingface.repository str              = Maykeye/TinyLLama-v0
-llama_model_loader: - kv   9:             general.parameter_weight_class str              = 5M
-llama_model_loader: - kv  10:                          llama.block_count u32              = 8
-llama_model_loader: - kv  11:                       llama.context_length u32              = 2048
-llama_model_loader: - kv  12:                     llama.embedding_length u32              = 64
-llama_model_loader: - kv  13:                  llama.feed_forward_length u32              = 256
-llama_model_loader: - kv  14:                 llama.attention.head_count u32              = 16
-llama_model_loader: - kv  15:     llama.attention.layer_norm_rms_epsilon f32              = 0.000001
-llama_model_loader: - kv  16:                          general.file_type u32              = 1
-llama_model_loader: - kv  17:                           llama.vocab_size u32              = 32000
-llama_model_loader: - kv  18:                 llama.rope.dimension_count u32              = 4
-llama_model_loader: - kv  19:                       tokenizer.ggml.model str              = llama
-llama_model_loader: - kv  20:                         tokenizer.ggml.pre str              = default
-llama_model_loader: - kv  21:                      tokenizer.ggml.tokens arr[str,32000]   = ["<unk>", "<s>", "</s>", "<0x00>", "<...
-llama_model_loader: - kv  22:                      tokenizer.ggml.scores arr[f32,32000]   = [0.000000, 0.000000, 0.000000, 0.0000...
-llama_model_loader: - kv  23:                  tokenizer.ggml.token_type arr[i32,32000]   = [2, 3, 3, 6, 6, 6, 6, 6, 6, 6, 6, 6, ...
-llama_model_loader: - kv  24:                tokenizer.ggml.bos_token_id u32              = 1
-llama_model_loader: - kv  25:                tokenizer.ggml.eos_token_id u32              = 2
-llama_model_loader: - kv  26:            tokenizer.ggml.unknown_token_id u32              = 0
-llama_model_loader: - kv  27:            tokenizer.ggml.padding_token_id u32              = 0
-llama_model_loader: - kv  28:               general.quantization_version u32              = 2
+llama_model_loader: - kv   6:                       general.quantized_by str              = Mofosyne
+llama_model_loader: - kv   7:                         general.size_label str              = 4.6M
+llama_model_loader: - kv   8:                            general.license str              = apache-2.0
+llama_model_loader: - kv   9:                                general.url str              = https://huggingface.co/mofosyne/TinyL...
+llama_model_loader: - kv  10:                         general.source.url str              = https://huggingface.co/Maykeye/TinyLL...
+llama_model_loader: - kv  11:                               general.tags arr[str,5]       = ["text generation", "transformer", "l...
+llama_model_loader: - kv  12:                          general.languages arr[str,1]       = ["en"]
+llama_model_loader: - kv  13:                           general.datasets arr[str,2]       = ["https://huggingface.co/datasets/ron...
+llama_model_loader: - kv  14:                          llama.block_count u32              = 8
+llama_model_loader: - kv  15:                       llama.context_length u32              = 2048
+llama_model_loader: - kv  16:                     llama.embedding_length u32              = 64
+llama_model_loader: - kv  17:                  llama.feed_forward_length u32              = 256
+llama_model_loader: - kv  18:                 llama.attention.head_count u32              = 16
+llama_model_loader: - kv  19:     llama.attention.layer_norm_rms_epsilon f32              = 0.000001
+llama_model_loader: - kv  20:                          general.file_type u32              = 1
+llama_model_loader: - kv  21:                           llama.vocab_size u32              = 32000
+llama_model_loader: - kv  22:                 llama.rope.dimension_count u32              = 4
+llama_model_loader: - kv  23:                       tokenizer.ggml.model str              = llama
+llama_model_loader: - kv  24:                         tokenizer.ggml.pre str              = default
+llama_model_loader: - kv  25:                      tokenizer.ggml.tokens arr[str,32000]   = ["<unk>", "<s>", "</s>", "<0x00>", "<...
+llama_model_loader: - kv  26:                      tokenizer.ggml.scores arr[f32,32000]   = [0.000000, 0.000000, 0.000000, 0.0000...
+llama_model_loader: - kv  27:                  tokenizer.ggml.token_type arr[i32,32000]   = [2, 3, 3, 6, 6, 6, 6, 6, 6, 6, 6, 6, ...
+llama_model_loader: - kv  28:                tokenizer.ggml.bos_token_id u32              = 1
+llama_model_loader: - kv  29:                tokenizer.ggml.eos_token_id u32              = 2
+llama_model_loader: - kv  30:            tokenizer.ggml.unknown_token_id u32              = 0
+llama_model_loader: - kv  31:            tokenizer.ggml.padding_token_id u32              = 0
+llama_model_loader: - kv  32:               general.quantization_version u32              = 2
 llama_model_loader: - type  f32:   17 tensors
 llama_model_loader: - type  f16:   58 tensors
 llm_load_vocab: special tokens definition check successful ( 259/32000 ).
@@ -221,6 +229,7 @@ llm_load_print_meta: n_head           = 16
 llm_load_print_meta: n_head_kv        = 16
 llm_load_print_meta: n_layer          = 8
 llm_load_print_meta: n_rot            = 4
+llm_load_print_meta: n_swa            = 0
 llm_load_print_meta: n_embd_head_k    = 4
 llm_load_print_meta: n_embd_head_v    = 4
 llm_load_print_meta: n_gqa            = 1
@@ -282,14 +291,15 @@ CFG -> Penalties -> top_k -> tfs_z -> typical_p -> top_p -> min_p -> temperature
 generate: n_ctx = 512, n_batch = 2048, n_predict = -1, n_keep = 1
 
 
- hello world the gruff man said yes. She was very happy. The man waved goodbye to the little boy and said the little boy. It was the best day ever. 
-The little boy was so excited. He took off his special favorite toy and a beautiful dress. He gave it to the little boy and said "thank you" to the little girl. He said "Thank you for being so clever. The man and the little boy both smiled. [end of text]
+ hello world the gruff man said no. The man was very sad and wanted to see what was wrong. He asked the man if they could do it. But they did not like his way to the park.
+One day, the man decided to go in and he took off his own new home. He gave the bird a little bit of his friend. He said he had to find a way to hide it in his woods. The man was very happy, but he knew he needed to make it in the yard.
+The man was very sad and he could not find the bird. He didn't want to get to the park and his friend was very sad. They could not find the bird and his friend. But the man was too sad. He had no friends and no friends. [end of text]
 
 
-llama_print_timings:        load time =       9.88 ms
-llama_print_timings:      sample time =       3.83 ms /    89 runs   (    0.04 ms per token, 23249.74 tokens per second)
-llama_print_timings: prompt eval time =       1.61 ms /     8 tokens (    0.20 ms per token,  4968.94 tokens per second)
-llama_print_timings:        eval time =     214.13 ms /    88 runs   (    2.43 ms per token,   410.96 tokens per second)
-llama_print_timings:       total time =     237.74 ms /    96 tokens
+llama_print_timings:        load time =      10.26 ms
+llama_print_timings:      sample time =       6.03 ms /   156 runs   (    0.04 ms per token, 25879.23 tokens per second)
+llama_print_timings: prompt eval time =       2.16 ms /     8 tokens (    0.27 ms per token,  3696.86 tokens per second)
+llama_print_timings:        eval time =     748.08 ms /   155 runs   (    4.83 ms per token,   207.20 tokens per second)
+llama_print_timings:       total time =     800.80 ms /   163 tokens
 Log end
 ```