feat: add message queue

[tituschewxj]

Summary

• Implemented RabbitMQ message queue, which should work with both docker-compose and running without docker.
• Added environment variable for RABBITMQ_URL=amqp://guest:guest@rabbitmq:5672/ , so updated test.yml and github repo variables

NOTE: This requires updating the deployment/kubernetes so that it supports a RabbitMQ message queue. It may require some more work, but unlikely to cause more issues since it should work similarly to Redis.

Motivation

Decoupling and Asynchronous Communication (Generated with the help of ChatGPT)

RabbitMQ facilitates asynchronous communication, meaning the sender (producer) does not need to wait for the receiver (consumer) to process the message. This allows services to be loosely coupled, as the producer only cares about placing the message in the queue, not when or how the consumer handles it.

REST-based communication is synchronous. The sender sends a request and waits for a response, which creates tight coupling between services. If the consumer is slow or unavailable, the producer has to handle the delay or failure directly.

RabbitMQ message queues offer better decoupling, fault tolerance, and scalability, making them ideal for distributed and event-driven microservice architectures. REST calls, however, are simpler and better suited for synchronous communication where immediate responses are necessary.

Setting up message queue with RabbitMQ (without docker-compose)

A message queue is used to pass submission results asynchronously from the execution-service to the history-service.

1. In order to do so, we can run the following command to set up a docker container for RabbitMQ:

docker run -d --name rabbitmq -p 5672:5672 -p 15672:15672 rabbitmq:3-management

2. Then we can run the execution-service:

go run main.go

3. We can run the history-service by changing our directory and running the same command:

cd ../history-service
go run main.go

To view more details on the RabbitMQ queue, we can go to localhost:15672, and login using guest as the username and password.

[bensohh]

LGTM 👍 Thank you for implementing the rabbitmq, it looks really interesting! Tested it and it works on local docker as well :)