Using this model in text-generation-webui just results in a string of meaningless characters.
Also tried your [stable-vicuna-13B-GPTQ] including both the MAIN and LATEST quantised models. Got similar results, except that the word repetition was higher.
Also verified that it's not a WebUI display issue!
transformers=4.31.0
peft=0.5.0.dev0
cuda 11.8
The text-generation-webui for the same environment, my own merged LLaMA->Alpaca model, is able to generate text properly.
Now it is no longer clear if there is something wrong with the model or if there is something I am doing that is causing this. Also after searching the web, tried various parameters in text-generation-webui that correspond to stable vicuna. But the results did not change.
*As you can see in the screenshot above, the added_tokens.json for this model has "[PAD]": 32000
where the [PAD] part appears in the generated text with high frequency. I don't know what the principle is.