Unable to download model tokenizer
I am using huggingface-hub-0.10.1 tokenizers-0.13.1 transformers-4.23.1 sentencepiece-0.1.97 on Google Colab.
While trying to download the tokenizer using: tokenizer = AutoTokenizer.from_pretrained("google/pegasus-x-base-arxiv")
I face the following key error:
KeyError: <class 'transformers.models.pegasus_x.configuration_pegasus_x.PegasusXConfig'>
Model download works fine but not the tokenizer. Could someone please share why am I unable to use the tokenizer?
Hi, the corresponding tokenizers can be found under the non-finetuned versions of the models.
Try:tokenizer = AutoTokenizer.from_pretrained("google/pegasus-x-base")
This works. Thanks!
Do you know whether any other fine tuned versions of pegasusx (bigpatent, pubmed) would be released soon on huggingface?
On a similar note, I would love to see a version of PegasusX fine-tuned with GovReport on Hugging Face. That is because I'm working on summarizing some government -related texts. Thanks :)