Skip to content

Commit

Permalink
[ci skip][docs] update poller and jobs general docs
Browse files Browse the repository at this point in the history
  • Loading branch information
sescobb27 committed Jan 9, 2019
1 parent 274d558 commit 1b32789
Show file tree
Hide file tree
Showing 2 changed files with 27 additions and 0 deletions.
13 changes: 13 additions & 0 deletions apps/repo_jobs/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,3 +19,16 @@ Documentation can be generated with [ExDoc](https://github.com/elixir-lang/ex_do
and published on [HexDocs](https://hexdocs.pm). Once published, the docs can
be found at [https://hexdocs.pm/repo_jobs](https://hexdocs.pm/repo_jobs).

## General Overview

- BugsBunny creates a pool of connections to RabbitMQ
- each connection worker traps exits and links the connection process to it
- each connection worker creates a pool of channels and links them to it
- we spawn in the ConsumerSupervisor a given number of GenServers that are going to be our RabbitMQ consumers
- each consumer is going to get a channel out of the channel pool and is going to subscribe itself as a consumer via Basic.Consume using that channel

## High Level Architecture

it is really similar to the architecture of `RepoPoller` in the sense that we need a pool of connections and channels to RabbitMQ and a pool of workers, in this case, RabbitMQ Consumers; this is because we also use BugsBunny which is our connection layer that both apps uses.

![screen shot 2018-08-29 at 7 22 49 am](https://user-images.githubusercontent.com/1157892/44787334-7a2e5d80-ab5c-11e8-86c4-16f7e5de3275.png)
14 changes: 14 additions & 0 deletions apps/repo_poller/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,3 +19,17 @@ Documentation can be generated with [ExDoc](https://github.com/elixir-lang/ex_do
and published on [HexDocs](https://hexdocs.pm). Once published, the docs can
be found at [https://hexdocs.pm/repo_poller](https://hexdocs.pm/repo_poller).

## General Overview

- BugsBunny creates a pool of connections to RabbitMQ
- each connection worker traps exits and links the connection process to it
- each connection worker creates a pool of channels and links them to it
- when a client checks out a channel out of the pool the connection worker monitors that client to return the channel into it in case of a crash
- RepoPoller polls GitHub repos for new tags
- if there are new tags it publishes a message to RabbitMQ using the connection/channels pool

## High Level Architecture

when starting a connection worker we are going to start within it a pool of multiplexed channels to RabbitMQ and store them in its state (we can move this later to ets). Then, inside the connection worker we are going to trap exits and link each channel to it. this way if a channel crashes, the connection worker is going to be able to start another channel and if a connection to RabbitMQ crashes we are going to be able to restart that connection, remove all crashed channels and then restart them with a new connection; also we are going to be able to easily monitor client accessing channels, queue an dequeue channels from the pool in order to make them accessible by 1 client at a time making them race condition free.

![screen shot 2018-08-17 at 7 51 36 am](https://user-images.githubusercontent.com/1157892/44267068-71e83100-a1f2-11e8-8d73-2bc7a1914733.png)

0 comments on commit 1b32789

Please sign in to comment.