MoE-StrangeMerges-2x7B / mergekit_moe_config.yml
Gille's picture
Upload folder using huggingface_hub
5a97a5e verified
raw
history blame
300 Bytes
base_model: Gille/StrangeMerges_9-7B-dare_ties
gate_mode: cheap_embed
dtype: float16
experts:
- source_model: Gille/StrangeMerges_9-7B-dare_ties
positive_prompts: ["science, logic, math"]
- source_model: Gille/StrangeMerges_8-7B-slerp
positive_prompts: ["reasoning, numbers, abstract"]