Skip to content

Commit

Permalink
Further README work, incl example config.
Browse files Browse the repository at this point in the history
  • Loading branch information
bittrance committed Mar 18, 2024
1 parent 95806b5 commit a3bad28
Show file tree
Hide file tree
Showing 4 changed files with 2,324 additions and 34 deletions.
75 changes: 41 additions & 34 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,44 @@ docker run --rm --name kitops bittrance/kitops \
--action 'echo "kitops was updated"'
```

For a full set of commandline options, try `--help`. If you want more advanced configuration (particularly if you wan to track multiple repositories), the [example.yaml](./example.yaml) configuration file explains how.

We also provide pre-built binaries for Linux, Windows and MacOS for scenarios where you want to run kitops on a virtual machine. You can download it from the [releases page](https://github.com/bittrance/kitops/releases).

## How does kitops work?

kitops is a statically compiled binary. It uses only pure Rust libraries and it therefore depends only on your platform's standard library (and an ssh binary where you want git+ssh support). It supports a wide variety of platforms and can be deployed in many different ways:

- as a long-running process on a VM
- as a periodic job
- as a long-running container
- as a CLI tool to perform a single run

kitops runs each task on a schedule:

![kitops workflow](images/flow.png)

kitops will retry the provided actions until they all succeed, thus providing a basis for eventual consistency. Actions given to kitops need to be idempotent (that is, safe to run multiple times). When kitops successfully executes all the actions of a task, it updates its state file. The state file acts as memory between executions so if kitops is run as a periodic job, you should point `--state-file` to persistent file storage.

kitops will clone repositories that are not already present in its `--repo-dir` so you can use ephemeral storage for this, but if your repositories are large, you may want to keep repositories on persistent storage too. This allows fetching only new changes, dramatically reducing network traffic.

## Configuration

### GitHub integration

To use kitops with private repositories on GitHub, you need to create a GitHub App and install it in the private repoisitories that you want kitops to track. You pass the App's ID with `--github-app-id` and the private key using `--github-private-key-file`. The app needs to have the `repo` scope.

If you set `--github-status-context`, kitops will update the status of the commit that triggered the task. This requires the `repo:status` scope.

```shell
docker run --rm --name kitops bittrance/kitops \
--url https://github.com/myorg/private-repo \
--action ./deploy.sh \
--github-app-id 12345 \
--github-private-key-file /path/to/private-key.pem \
--github-status-context deploy/production
```

## Rationale

### Why would you want to use kitops?
Expand All @@ -34,53 +72,22 @@ This model has several potential weaknesses:
- Called APIs have to be accessible over the Internet (or require a VPN or similar)
- It is hard to delegate responsibility for the target environment to a third party

kitops enables the environment to "pull" deployments from a git repository.
Instead, kitops enables the environment to "pull" deployments from a git repository.

![Pull-style continuous deployment](images/cd-pull-model.png)

This model:

- is scalable - only repository rate limiting caps the number of target environments
- adheres to the Principle of Least Privilege - the pipeline has no access to the environment and the environment only needs read access to the repository. This is particularly relevant in Azure, where granting permissions to a pipeline requires extensive AAD permissions, but creating a service principal for kitops can be delegated to developers via the `Application Developer` role.
- separates concerns - the actions taken on the repository content can be kept separate from the repository content itself
- is NAT-friendly - the environment only needs to be able to make outbound connections to the git server
- allows a third party to take responsibility for the target environment

### How does kitops work?

kitops is a statically compiled binary. It uses only pure Rust libraries and it therefore depends only on libc (and an ssh binary where you want git+ssh support). It supports a wide variety of platforms and can be deployed in many different ways:

- as a long-running process on a VM
- as a periodic job
- as a long-running container
- as a CLI tool to perform a single run

kitops runs each task on a schedule.

<picture>

Each time kitops successfully applies all the actions of a task, it updates its state file. The state file acts as memory between executions so if kitops is run as a periodic job, you should point --state-file to persistent file storage.

kitops will clone repositories that are not already present in --repo-dir so you can use ephemeral storage for this, but if your repositories are large, you may want to keep repositories on persistent storage too. This allows fetching only new changes, dramatically reducing network traffic.

## Example use cases

### Infrastructure-as-code continuous deployment

Use a serverless platform such as AWS Fargate or Azure Container Apps to run kitops as a periodic container job that applies infrastructure changes as they occur. Becuase kitops only takes a second to start and check for changes, this solution will typically cost a few dollars per month to run.

This scenario still requires , manually maintained infrastructure definition for kitops itself as it does not currently have special support to update itself.

### Roll your own container-based build servers

One use case for Kitops is to combine it with [act](...). Kitops pullsrepo changes and simply inokes act which will give you a local runner. Act will interact with the loval Docker daemon, much like you were using GiHub-hosted runners.

This use case requires a virtual machine, because there is currently no container orchestration platform that gives access to the local Docker socket, as required by `act`.

kitops cannot yet coordinate execution across multiple nodes, so you will have to balance your repositories across build servers manually.

## Alternatives

- [snare](https://tratt.net/laurie/src/snare/) - tool with similar design goals, but limited to GitHub webhooks (i.e. push-based).
- [GitHub Actions Runner](https://github.com/actions/runner) - the official runner application for GitHub Actions.

## Roadmap

Expand Down
45 changes: 45 additions & 0 deletions example.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
# The configuration file consists of a list of tasks. Each task represents
# a repository that kitops will monitor for changes.
tasks:
- name: Example task
# The interval at which to check for changes in the repository.
interval: 1m
# The maximum amount of time that the task is allowed to run,
# including all actions. When the timeout is reached, kitops will
# kill outstanding actions and mark the task as failed.
timeout: 60m
git:
# The URL of the Git repository to check for changes.
url: https://example.com/some/repository
# The branch to check for changes.
branch: main
# Settings specific to GitHub. This section is optional.
github:
# The App ID of the GitHub App that kitops uses to interact with
# this repository. See README for more information.
app_id: 123456
# Private key file for the GitHub App that you received when you
# created the app.
private_key_file: ./private-key.pem
# Context used to update the commit status on GitHub. Setting this
# value will instruct kitops to set commit status. See
# https://docs.github.com/en/rest/commits/statuses for more
# information.
status_context: example-task
# Actions for the task are run in sequence. kitops will abort the
# task if an action fails.
actions:
- name: echo
# The binary to execute. Note that there is no implicit shell.
entrypoint: /bin/bash
# Arguments to pass to the binary, as a list of strings.
args:
- -c
- echo Hello $NAME
# When true, kitops will seed the child process environment
# variabless with its own environment variables.
inherit_environment: false
# Environment variables to set for the child process. These will
# override inherited environment variables.
environment:
- NAME: World
Binary file added images/flow.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading

0 comments on commit a3bad28

Please sign in to comment.