huggingchat/chat-ui · Error with local inference server

May 7, 2023

•

edited May 7, 2023

Hi again @coyotte508 , I'm runing a local inference server and the chat-ui worked with the old .env structure.
Now I've upgraded to the new version with the new .env file and I've added my local endpoint according to the documentation in the README:
MODELS=[{"name": "...", "endpoints": [{"url": "127.0.0.1:8080/generate_stream"}]}]

When I just provide the model string above I'm getting this error below. It appears that the "url" requires "http://" before "127.0.0.1" and some other also some other information is required.

10:22:24 AM [vite] Error when evaluating SSR module /src/lib/server/models.ts:

10:22:24 AM [vite] Error when evaluating SSR module /src/routes/+layout.server.ts:

Internal server error: [
{
"code": "invalid_type",
"expected": "string",
"received": "undefined",
"path": [
0,
"userMessageToken"
],
"message": "Required"
},
{
"code": "invalid_type",
"expected": "string",
"received": "undefined",
"path": [
0,
"assistantMessageToken"
],
"message": "Required"
},
{
"validation": "url",
"code": "invalid_string",
"message": "Invalid url",
"path": [
0,
"endpoints",
0,
"url"
]
},
{
"code": "invalid_type",
"expected": "object",
"received": "undefined",
"path": [
0,
"parameters"
],
"message": "Required"
}
]
at get error [as error] (file:///home/robertodonpedro/dev/chat-ui/node_modules/zod/lib/index.mjs:537:31)
at ZodArray.parse (file:///home/robertodonpedro/dev/chat-ui/node_modules/zod/lib/index.mjs:636:22)
at eval (/src/lib/server/models.ts:42:3)
at async instantiateModule (file:///home/robertodonpedro/dev/chat-ui/node_modules/vite/dist/node/chunks/dep-79892de8.js:53996:9)
Internal server error: [
{
"code": "invalid_type",
"expected": "string",
"received": "undefined",
"path": [
0,
"userMessageToken"
],
"message": "Required"
},
{
"code": "invalid_type",
"expected": "string",
"received": "undefined",
"path": [
0,
"assistantMessageToken"
],
"message": "Required"
},
{
"validation": "url",
"code": "invalid_string",
"message": "Invalid url",
"path": [
0,
"endpoints",
0,
"url"
]
},
{
"code": "invalid_type",
"expected": "object",
"received": "undefined",
"path": [
0,
"parameters"
],
"message": "Required"
}
]
at get error [as error] (file:///home/robertodonpedro/dev/chat-ui/node_modules/zod/lib/index.mjs:537:31)
at ZodArray.parse (file:///home/robertodonpedro/dev/chat-ui/node_modules/zod/lib/index.mjs:636:22)
at eval (/src/lib/server/models.ts:42:3)
at async instantiateModule (file:///home/robertodonpedro/dev/chat-ui/node_modules/vite/dist/node/chunks/dep-79892de8.js:53996:9) (x2)
ZodError: [
{
"code": "invalid_type",
"expected": "string",
"received": "undefined",
"path": [
0,
"userMessageToken"
],
"message": "Required"
},
{
"code": "invalid_type",
"expected": "string",
"received": "undefined",
"path": [
0,
"assistantMessageToken"
],
"message": "Required"
},
{
"validation": "url",
"code": "invalid_string",
"message": "Invalid url",
"path": [
0,
"endpoints",
0,
"url"
]
},
{
"code": "invalid_type",
"expected": "object",
"received": "undefined",
"path": [
0,
"parameters"
],
"message": "Required"
}
]
at get error [as error] (file:///home/robertodonpedro/dev/chat-ui/node_modules/zod/lib/index.mjs:537:31)
at ZodArray.parse (file:///home/robertodonpedro/dev/chat-ui/node_modules/zod/lib/index.mjs:636:22)
at eval (/src/lib/server/models.ts:42:3)
at async instantiateModule (file:///home/robertodonpedro/dev/chat-ui/node_modules/vite/dist/node/chunks/dep-79892de8.js:53996:9) {
issues: [
{
code: 'invalid_type',
expected: 'string',
received: 'undefined',
path: [Array],
message: 'Required'
},
{
code: 'invalid_type',
expected: 'string',
received: 'undefined',
path: [Array],
message: 'Required'
},
{
validation: 'url',
code: 'invalid_string',
message: 'Invalid url',
path: [Array]
},
{
code: 'invalid_type',
expected: 'object',
received: 'undefined',
path: [Array],
message: 'Required'
}
],
addIssue: [Function (anonymous)],
addIssues: [Function (anonymous)],
errors: [
{
code: 'invalid_type',
expected: 'string',
received: 'undefined',
path: [Array],
message: 'Required'
},
{
code: 'invalid_type',
expected: 'string',
received: 'undefined',
path: [Array],
message: 'Required'
},
{
validation: 'url',
code: 'invalid_string',
message: 'Invalid url',
path: [Array]
},
{
code: 'invalid_type',
expected: 'object',
received: 'undefined',
path: [Array],
message: 'Required'
}
]
}
Internal server error: [
{
"code": "invalid_type",
"expected": "string",
"received": "undefined",
"path": [
0,
"userMessageToken"
],
"message": "Required"
},
{
"code": "invalid_type",
"expected": "string",
"received": "undefined",
"path": [
0,
"assistantMessageToken"
],
"message": "Required"
},
{
"validation": "url",
"code": "invalid_string",
"message": "Invalid url",
"path": [
0,
"endpoints",
0,
"url"
]
},
{
"code": "invalid_type",
"expected": "object",
"received": "undefined",
"path": [
0,
"parameters"
],
"message": "Required"
}
]
at get error [as error] (file:///home/robertodonpedro/dev/chat-ui/node_modules/zod/lib/index.mjs:537:31)
at ZodArray.parse (file:///home/robertodonpedro/dev/chat-ui/node_modules/zod/lib/index.mjs:636:22)
at eval (/src/lib/server/models.ts:42:3)
at async instantiateModule (file:///home/robertodonpedro/dev/chat-ui/node_modules/vite/dist/node/chunks/dep-79892de8.js:53996:9) (x3)
ZodError: [
{
"code": "invalid_type",
"expected": "string",
"received": "undefined",
"path": [
0,
"userMessageToken"
],
"message": "Required"
},
{
"code": "invalid_type",
"expected": "string",
"received": "undefined",
"path": [
0,
"assistantMessageToken"
],
"message": "Required"
},
{
"validation": "url",
"code": "invalid_string",
"message": "Invalid url",
"path": [
0,
"endpoints",
0,
"url"
]
},
{
"code": "invalid_type",
"expected": "object",
"received": "undefined",
"path": [
0,
"parameters"
],
"message": "Required"
}
]
at get error [as error] (file:///home/robertodonpedro/dev/chat-ui/node_modules/zod/lib/index.mjs:537:31)
at ZodArray.parse (file:///home/robertodonpedro/dev/chat-ui/node_modules/zod/lib/index.mjs:636:22)
at eval (/src/lib/server/models.ts:42:3)
at async instantiateModule (file:///home/robertodonpedro/dev/chat-ui/node_modules/vite/dist/node/chunks/dep-79892de8.js:53996:9) {
issues: [
{
code: 'invalid_type',
expected: 'string',
received: 'undefined',
path: [Array],
message: 'Required'
},
{
code: 'invalid_type',
expected: 'string',
received: 'undefined',
path: [Array],
message: 'Required'
},
{
validation: 'url',
code: 'invalid_string',
message: 'Invalid url',
path: [Array]
},
{
code: 'invalid_type',
expected: 'object',
received: 'undefined',
path: [Array],
message: 'Required'
}
],
addIssue: [Function (anonymous)],
addIssues: [Function (anonymous)],
errors: [
{
code: 'invalid_type',
expected: 'string',
received: 'undefined',
path: [Array],
message: 'Required'
},
{
code: 'invalid_type',
expected: 'string',
received: 'undefined',
path: [Array],
message: 'Required'
},
{
validation: 'url',
code: 'invalid_string',
message: 'Invalid url',
path: [Array]
},
{
code: 'invalid_type',
expected: 'object',
received: 'undefined',
path: [Array],
message: 'Required'
}
]
}

RobertoDonPedro

May 7, 2023

I got it to work by providing the whole MODEL configuration (like if I would use the inference API) but adding the endpoint string as follows: "endpoints": [{"url": "http://127.0.0.1:8080/generate_stream"}],

So it requires "http://" and (at least some other) information like in the API based version. Would appreciate the minimal working configuration string to be in the README @coyotte508

RobertoDonPedro changed discussion status to closed May 9, 2023