History management for long conversation

[qbey]

The project must be able to manage conversation with long history.

Current behavior :

• the whole history is provided to the LLM

Expected behavior :

• a compressed history must be provided to the LLM (yep...). Several solutions: resume previous conversation, only send last messages, ...

[eliott07]

I'm adding the generalisation tag for the part of this pr that limits the size of conversation that is sent in context

[eliott07]

#421

[eliott07]

For larger deployment, the main priority is conversations not falling into an error state when the conversation gets too long. This could be a rolling window.