Summary:
Set up a message queue system to hold the posts that are ready for chunking, so they can be processed asynchronously by the Triton instances.
Tasks:
Choose a message queue system (likely RabbitMQ or Redis).
Implement logic to push prepared text into the queue.
Acceptance Criteria:
A queue has been implemented and prepared text can be pushed to it.
Summary:
Set up a message queue system to hold the posts that are ready for chunking, so they can be processed asynchronously by the Triton instances.
Tasks:
Acceptance Criteria: