|
--- |
|
license: mit |
|
library_name: transformers |
|
datasets: |
|
- Severian/Internal-Knowledge-Map |
|
pipeline_tag: text-generation |
|
--- |
|
|
|
# New Fixed Version with extended training available now! |
|
|
|
<img src="https://cdn-uploads.huggingface.co/production/uploads/64740cf7485a7c8e1bd51ac9/GO4MY_3adP2G9EHKZbZpg.webp" width="500" height="500"> |
|
|
|
|
|
This model is the second trained with experimental 'Internal Knowledge Map' dataset. Developed with an aim to go beyond the scope of usual data processing capabilities, this model gets trained to build comprehensive understanding and reasoning in a wide range of knowledge domains with elaborate guidelines. It bases its reasoning on a specially selected dataset emphasizing the interrelations of the diverse disciplines which aim to synthesize, integrate, and apply complex information in ways that mimic humanly abstract reasoning and creative thought processes. |
|
|
|
At the very core of the development of this model is the desire to make sure that LLMs engage in a kind of cognitive activity not limited to memory but actually taking on abstract reasoning, problem-solving, and generation of new insights. To achieve this, 'Nexus-IKM-Mistral-7B' has been fine-tuned until convergance at ~15 Epochs on this unique dataset, which resulted in the model demonstrating greater capability for giving rise to insights and problem-solving in complex, multi-disciplinary settings. This involves improved ability in drawing links between different pieces of knowledge, reasoning through complex scenarios, and proposing innovative solutions that cut across various domains, including science, technology, environmental studies, and humanities. |
|
|
|
Test this out and see if you find anything interesting or intriguing. I will keep iterating more versions but this one seems like a fun and useful way to start. |
|
|
|
|
|
|
|
**If you'd like to train your own version, here is the full notebook to recreate the training on Unsloth yourself (https://colab.research.google.com/drive/1828t77iO2nLRXVfB8HoI11eFu-79-Oe7?usp=sharing). You'll just have to drop in the train.jsonl from the Dataset repo (https://huggingface.co/datasets/Severian/Internal-Knowledge-Map) into your Colab directory and rename it dataset.jsonl** |
|
|
|
|
|
## Training Snapshot |
|
|
|
``` |
|
|
|
Step Training Loss |
|
1 3.223000 |
|
2 3.221300 |
|
3 3.215900 |
|
4 3.210600 |
|
5 3.203000 |
|
6 3.193500 |
|
7 3.184000 |
|
8 3.173400 |
|
9 3.162400 |
|
10 3.151500 |
|
11 3.140500 |
|
12 3.128800 |
|
13 3.117600 |
|
14 3.106700 |
|
15 3.095500 |
|
16 3.084700 |
|
17 3.073700 |
|
18 3.062700 |
|
19 3.052300 |
|
20 3.041800 |
|
|
|
|
|
201 1.273200 |
|
202 1.257600 |
|
203 1.241900 |
|
204 1.226100 |
|
205 1.210800 |
|
206 1.195500 |
|
207 1.180800 |
|
208 1.166000 |
|
209 1.151200 |
|
210 1.136900 |
|
211 1.122000 |
|
212 1.106600 |
|
213 1.091200 |
|
214 1.075200 |
|
215 1.059200 |
|
216 1.042900 |
|
217 1.026600 |
|
218 1.010300 |
|
219 0.994200 |
|
|
|
416 0.041700 |
|
417 0.041700 |
|
418 0.041600 |
|
419 0.041600 |
|
420 0.041600 |
|
421 0.041600 |
|
422 0.041500 |
|
423 0.041500 |
|
424 0.041500 |
|
425 0.041400 |
|
426 0.041400 |
|
427 0.041400 |
|
428 0.041400 |
|
429 0.041300 |
|
430 0.041300 |
|
431 0.041300 |
|
432 0.041200 |
|
433 0.041200 |
|
434 0.041200 |
|
435 0.041100 |
|
436 0.041200 |
|
437 0.041100 |
|
438 0.041100 |
|
439 0.041100 |
|
440 0.041000 |
|
441 0.041000 |
|
442 0.041000 |
|
443 0.040900 |
|
444 0.040900 |
|
445 0.040900 |
|
|
|
668 0.035200 |
|
669 0.035100 |
|
670 0.035100 |
|
671 0.035100 |
|
672 0.035100 |
|
673 0.035000 |
|
674 0.035000 |
|
675 0.035000 |
|
676 0.035000 |
|
677 0.034900 |
|
678 0.034900 |
|
679 0.034900 |
|
680 0.034800 |
|
681 0.034800 |
|
682 0.034800 |
|
683 0.034800 |
|
684 0.034800 |
|
685 0.034700 |
|
686 0.034700 |
|
687 0.034700 |
|
688 0.034700 |
|
689 0.034600 |
|
690 0.034600 |
|
691 0.034600 |
|
692 0.034600 |
|
693 0.034500 |
|
694 0.034500 |
|
695 0.034500 |
|
696 0.034400 |
|
697 0.034400 |
|
698 0.034400 |
|
699 0.034400 |
|
700 0.034300 |
|
701 0.034300 |
|
702 0.034300 |
|
703 0.034300 |
|
704 0.034200 |
|
705 0.034200 |
|
706 0.034200 |
|
707 0.034200 |
|
708 0.034100 |
|
709 0.034100 |
|
710 0.034100 |
|
711 0.034100 |
|
712 0.034000 |
|
713 0.034000 |
|
714 0.034000 |
|
715 0.034000 |
|
716 0.033900 |
|
717 0.033900 |
|
718 0.033800 |
|
719 0.033800 |
|
720 0.033800 |
|
721 0.033800 |
|
|
|
1209 0.006600 |
|
1210 0.006500 |
|
1211 0.006300 |
|
1212 0.006200 |
|
1213 0.006100 |
|
1214 0.006000 |
|
1215 0.005800 |
|
1216 0.005700 |
|
1217 0.005600 |
|
1218 0.005500 |
|
1219 0.005400 |
|
1220 0.005300 |
|
1221 0.005100 |
|
1222 0.004900 |
|
1223 0.004800 |
|
1224 0.004700 |
|
1225 0.004600 |
|
1226 0.004500 |
|
1227 0.004400 |
|
1228 0.004300 |
|
1229 0.004200 |
|
1230 0.004000 |
|
1231 0.003900 |
|
1232 0.003800 |
|
1233 0.003700 |
|
1234 0.003500 |
|
1235 0.003400 |
|
1236 0.003300 |
|
1237 0.003200 |
|
1238 0.003000 |
|
1239 0.003000 |
|
1240 0.002900 |
|
1241 0.002800 |
|
1242 0.002700 |
|
1243 0.002600 |
|
1244 0.002500 |
|
1245 0.002400 |
|
1246 0.002300 |
|
1247 0.002200 |
|
1248 0.002100 |
|
1249 0.002000 |
|
1250 0.001900 |
|
1251 0.001800 |
|
1252 0.001800 |
|
1253 0.001700 |
|
1254 0.001600 |
|
1255 0.001600 |
|
1256 0.001500 |
|
1257 0.001400 |
|
1258 0.001300 |
|
1259 0.001300 |
|
1260 0.001200 |
|
1261 0.001200 |
|
1262 0.001100 |
|
1263 0.001100 |
|
1264 0.001000 |
|
1265 0.001000 |
|
1266 0.000900 |
|
1267 0.000900 |
|
1268 0.000800 |
|
1269 0.000800 |
|
1270 0.000800 |
|
1271 0.000800 |
|
1272 0.000700 |
|
1273 0.000700 |
|
1274 0.000700 |
|
1275 0.000600 |
|
1276 0.000600 |
|
1277 0.000600 |
|
1278 0.000600 |
|
1279 0.000500 |
|
1280 0.000500 |
|
1281 0.000500 |
|
1282 0.000500 |
|
1283 0.000500 |
|
1284 0.000500 |
|
1285 0.000500 |
|
1286 0.000400 |
|
1287 0.000400 |
|
1288 0.000400 |
|
1289 0.000400 |
|
1290 0.000400 |
|
1291 0.000400 |
|
1292 0.000400 |
|
1293 0.000400 |
|
1294 0.000400 |
|
1295 0.000400 |
|
1296 0.000400 |
|
1297 0.000300 |
|
1298 0.000300 |
|
``` |
