Function calling doesn't like conversation's histories
I implemented Function calling for Mixtral 8x22B.
It works quite well if I just asks a question (for example: question -> tool_call -> tool_result -> answer or question -> answer)
But if I add the historic of the conversation, the model doesn't know how to respond and get stuck on whether or not to use function calling:
- if I start with a question that needs a tool, the model will always answer with tools no matter if it's not relevant
- if I start with a question that can be answer without tools, the model will never use the tools for the entire conversation even though some question need it
For me iuse a clean slate for each new question !
so only the history of the ttransaction is avaiable to the model :
It does not need the whole of the chat history , you can keep a speperate history to be tokenized to rag :
so your local rag chat history and the model chat history are seperate :
you should use chroma or another on the fly rag db : to store your chat history : to look up history the model should use a tool, to acess the converation rag :
For its current tranasaction it should have only this list ! clearing this chat memeory each question/Task!