ValueError: Tokenizer class LlamaTokenizer does not exist or is not currently imported. (while using Transformers)
#23
by
ShifraSec
- opened
Background:
Using OpenPlayGround to connection to the model and getting eternal server error 500, so started to investigate my connection to the model.
Steps to reproduce
run a python REPL (or while in code)...
>> from transformers import AutoTokenizer, AutoModelForCausalLM
fine then ...
>>> tokenizer = AutoTokenizer.from_pretrained("anon8231489123/gpt4-x-alpaca-13b-native-4bit-128g")
gives me this error:
ValueError: Tokenizer class LlamaTokenizer does not exist or is not currently imported.
Notes:
tried to:
pip install --upgrade transformers
still didn't work.
My system info:
OS: MacOS M1 13.3 Pip version: pip 23.0.1 python version: Python 3.11.0