This model has the same metadata problem as the BPE fix model
#2
by
yehiaserag
- opened
Sorry for bringing this up again here, but this model is showing as 7B in LM Studio
Here is the metadata
{
"name": "SFR-Iterative-DPO-LLaMA-3-8B-R",
"arch": "llama",
"quant": "Q8_0",
"context_length": 8192,
"embedding_length": 4096,
"num_layers": 32,
"rope": {
"freq_base": 500000,
"dimension_count": 128
},
"head_count": 32,
"head_count_kv": 8,
"parameters": "7B"
}
Hey @yehiaserag no problem !
Yeah not sure why it's doing that, I'm not editing the metadata or anything.. do you know by chance if all other 8B models have this issue?
This metadata is coming from where, lmstudio or the actual GGUF metadata? Where do you see these values?
The original first gguf that had the tokenizer problem was showing as 8B.
Llama.cpp repo has a python script that edits this metadata, but if you are not explicitly setting it to 7B, maybe be it's quessed/infered in the quantization process somewhere...