mistralai

/

Mistral-7B-Instruct-v0.1

like 1.5k

"utrawat/Mistralai-7B-V1" is my trained model of "mistralai/Mistral-7B-Instruct-v0.1"

import logging
logging.basicConfig(level=logging.DEBUG)

import torch, transformers, accelerate, bitsandbytes
from transformers import AutoModelForCausalLM, AutoTokenizer

base_model_id = "utrawat/Mistralai-7B-V1"

bnb_config = transformers.BitsAndBytesConfig(
load_in_4bit=True,
bnb_4bit_use_double_quant=True,
bnb_4bit_quant_type="nf4",
bnb_4bit_compute_dtype=torch.bfloat16
)
trained_model = AutoModelForCausalLM.from_pretrained(base_model_id, quantization_config=bnb_config)

RuntimeError: Failed to import transformers.models.transfo_xl.configuration_transfo_xl because of the following error (look up to see its traceback):
No module named 'transformers.models.transfo_xl.configuration_transfo_xl'