RuntimeError: probability tensor contains either `inf`, `nan` or element < 0
#7
by
Satya93
- opened
I get this error using the newest transformers 4.31.0 on my RTX 4090. I am loading the model like this:
tokenizer = AutoTokenizer.from_pretrained(model_dir, use_fast=False)
model = AutoModelForCausalLM.from_pretrained(model_dir,torch_dtype=torch.float16, low_cpu_mem_usage=True, device_map="auto")
Any ideas?