Spaces:
Running
500 Internal Server Error (on local machine)
When running the chat-ui locally and enter something in the prompt the UI just says "..." forever and "Too much traffic, please try again." for a very short term. However, the reason for this behaviour appears to be that it cannot find the random load-balanced endpoint. See the errors below.
In the inspector tab of the browser I can see this error:
POST http://localhost:5173/conversation/644bb4523d7ee3475d8c536c 500 (Internal Server Error)
+page.svelte:129 Error: Server response contains error: 500
at streamingRequest (index.mjs:212:11)
at async textGenerationStream (index.mjs:452:3)
at async getTextGenerationStream (+page.svelte:55:20)
at async writeMessage (+page.svelte:115:54)
In my VSCode terminaI can see this error:
Error: Invalid config, no endpoint found
at Module.modelEndpoint (modelEndpoint.ts:20:7)
at POST (+server.ts:36:24)
at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
at async Module.render_endpoint (/node_modules/@sveltejs/kit/src/runtime/server/endpoint.js:42:20)
at async resolve (/node_modules/@sveltejs/kit/src/runtime/server/respond.js:377:17)
at async Object.handle (hooks.server.ts:24:18)
at async Module.respond (/node_modules/@sveltejs/kit/src/runtime/server/respond.js:248:20)
at async file:///home/user/dev/chat-ui/node_modules/@sveltejs/kit/src/exports/vite/dev/index.js:513:22
Any ideas how to fix this?
The error message is improved in https://huggingface.co/spaces/huggingchat/chat-ui/commit/69830ae14c20364c45d08a3db02ff2f58b9508d2
As for the problem, you need to add something like this in your .env.local:
MODEL_ENDPOINTS=`[{"endpoint": "https://api-inference.huggingface.co/models/OpenAssistant/oasst-sft-4-pythia-12b-epoch-3.5", "authorization": "Bearer hf_<token>", "weight": 1}]`
Where you need to replace hf_<token> by your huggingface read access token (huggingface.co/settings/tokens)