feature/implement-rabbitmq

[cedricve]

Description

Motivation

The primary motivation for this change is to integrate RabbitMQ into our project, providing robust and reliable message queuing capabilities. RabbitMQ is a widely used message broker that supports various messaging protocols and ensures efficient, scalable, and fault-tolerant communication between different parts of the system. By implementing RabbitMQ, we can enhance the project's ability to handle asynchronous tasks, improve system resilience, and support more complex workflows.

Improvements

1. Reliable Messaging: RabbitMQ ensures that messages are delivered reliably, even in the event of system failures. This is vital for maintaining data integrity and ensuring that critical tasks are completed.

2. Scalability: With RabbitMQ, we can easily scale our message processing capabilities by adding more consumers. This allows the system to handle increased load without significant changes to the architecture.

3. Flexibility: RabbitMQ supports various messaging patterns (e.g., pub/sub, work queues, routing) which can be leveraged to design more flexible and efficient communication workflows within the system.

4. Error Handling and Recovery: The implementation includes disaster recovery handlers and dead-letter queues to manage failed messages, ensuring that no messages are lost and that errors can be handled gracefully.

5. Testing and Mocking: The changes include a comprehensive mocking framework for RabbitMQ, allowing for thorough testing of the message queues without needing a live RabbitMQ instance. This improves the reliability of tests and speeds up development.

6. Metrics and Monitoring: The implementation includes hooks for Prometheus metrics, enabling better monitoring and observability of the message processing system. This helps in identifying bottlenecks and ensuring the system operates efficiently.

Summary of Changes

1. Dependency Updates:

   • Added github.com/rabbitmq/amqp091-go for RabbitMQ integration.
   • Updated github.com/uug-ai/models to version v1.2.26.

2. Mock Queue Implementation:

   • Enhanced the MockQueue with additional methods like Close, ReadMessages, RouteMessages, Publish, and various helper methods for testing.
   • Added comprehensive tests for the MockQueue.

3. RabbitMQ Implementation:

   • Added RabbitMQ struct with methods to connect, declare queues, read messages, route messages, publish messages, and handle disaster recovery.
   • Added support for consumer, dead-letter, and router queues.
   • Included methods for delayed publishing and disaster recovery handling.
   • Added tests for RabbitMQ configuration and functionality.

4. Configuration Enhancements:

   • Updated RabbitOptions to include separate configurations for consumer, dead-letter, and router queues.
   • Enhanced the RabbitOptionsBuilder for fluent configuration of RabbitMQ options.

By integrating RabbitMQ, this change significantly enhances the project's messaging capabilities, making it more robust, scalable, and flexible. The comprehensive testing framework ensures that the implementation is reliable and maintainable.

[codecov]

Codecov Report

❌ Patch coverage is 15.94203% with 232 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
pkg/queue/rabbitmq.go	10.93%	171 Missing ⚠️
pkg/queue/mock.go	27.38%	58 Missing and 3 partials ⚠️

📢 Thoughts on this report? Let us know!