sometimesanotion/Lamarck-14B-v0.4-Qwenvergence

Lamarck 14B v0.4 Qwenvergence: it's a big step up for Lamarck in terms of quality. All the same ingredients are involved as in previous releases of Lamarck; they are more effectively combined. This model features decent wit and stronger reasoning than 0.3.

Merge Details

This model was initialized from model_stock, and refined from there. No fine-tuning, use of models apart from those listed or the contents of Qwenvergence, wild parties, or sacrifices to the unnamed deities were involved.

Models Merged

Top influences: These ancestors are in the Qwenvergence model_stock, reinforced in later steps:

• arcee-ai/Virtuoso-Small - A brand new model from Arcee, refined from the notable cross-architecture Llama-to-Qwen distillation arcee-ai/SuperNova-Medius. The first two layers are nearly exclusively from Virtuoso. It has proven to be a well-rounded performer, and contributes a noticeable boost to the model's prose quality.

• CultriX/SeQwence-14B-EvolMerge - A top contender on reasoning benchmarks.

• VAGOsolutions/SauerkrautLM-v2-14b-DPO - This model's influence is understated, but aids BBH and coding capability.

• v000000/Qwen2.5-Lumen-14B - A leading influence for prose quality.

Prose added:

The prose quality has taken a leap, no doubt also to the way EVA-UNIT-01/EVA-Qwen2.5-14B-v0.2, sthenno-com/miscii-14b-1028, oxyapi/oxy-1-small, and underwoods/medius-erebus-magnum-14b were applied.

Configuration

The following YAML configuration was used to finalize this model:

name:                Lamarck-14B-v0.4-Qwenvergence
merge_method:        ties
base_model:          sometimesanotion/lamarck-14b-base
tokenizer_source:    base
parameters:         
  density:           1.00
  weight:            1.00
  int8_mask:         true
  normalize:         true
  rescale:           false
models:
  - model:           merges/Qwen2.5-14B-Qwenvergence-slerp
    parameters:
      weight:        1.00
      density:       1.00
  - model:           arcee-ai/Virtuoso-Small
    parameters:
      weight:        1.00
      density:       1.00