forked from cortexproject/cortex-helm-chart
-
Notifications
You must be signed in to change notification settings - Fork 0
/
Copy pathREADME.md.gotmpl
172 lines (139 loc) · 5.39 KB
/
README.md.gotmpl
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
<!-- README.md is a generated file. Make any changes in README.md.gotmpl or values.yaml. -->
{{ template "chart.header" . }}
{{ template "chart.deprecationWarning" . }}
{{ template "chart.badgesSection" . }}
{{ template "chart.description" . }}
{{ template "chart.homepageLine" . }}
{{ template "chart.maintainersSection" . }}
## Dependencies
### Key-Value store
Cortex requires an externally provided key-value store, such as [etcd](https://etcd.io/) or [Consul](https://www.consul.io/).
Both services can be installed alongside Cortex, for example using helm charts available [here](https://github.com/bitnami/charts/tree/master/bitnami/etcd) and [here](https://github.com/helm/charts/tree/master/stable/consul).
### Storage
Cortex requires a storage backend to store metrics and indexes.
See [cortex documentation](https://cortexmetrics.io/docs/) for details on storage types and documentation
## Installation
[Helm](https://helm.sh) must be installed to use the charts.
Please refer to Helm's [documentation](https://helm.sh/docs/) to get started.
Once Helm is set up properly, add the repo as follows:
```bash
helm repo add cortex-helm https://cortexproject.github.io/cortex-helm-chart
```
Cortex can now be installed with the following command:
```bash
helm install cortex --namespace cortex cortex-helm/cortex
```
If you have custom options or values you want to override:
```bash
helm install cortex --namespace cortex -f my-cortex-values.yaml cortex-helm/cortex
```
Specific versions of the chart can be installed using the `--version` option, with the default being the latest release.
What versions are available for installation can be listed with the following command:
```bash
helm search repo cortex-helm
```
As part of this chart many different pods and services are installed which all
have varying resource requirements. Please make sure that you have sufficient
resources (CPU/memory) available in your cluster before installing Cortex Helm
chart.
## Upgrades
To upgrade Cortex use the following command:
```bash
helm upgrade cortex -f my-cortex-values.yaml cortex-helm/cortex
```
Note that it might be necessary to use `--reset-values` since some default values in the values.yaml might have changed or were removed.
Source code can be found [here](https://cortexmetrics.io/)
## Usage
### Rules and AlertManager configuration
Cortex can be configured to use a sidecar container in the Ruler and AlertManager to dynamically discover rules and AlertManager config/templates that are declared as ConfigMaps to allow easy and extensible configuration that avoids having to store state in the Cortex system itself (via config service, etc).
Put ConfigMaps into the specified namespace, and they are automatically detected and added as files to the Ruler and/or AlertManager containers, both of which are polling for changes on the filesystem and will make the new configurations go live dynamically.
This feature is disabled by default. Here is a simple example:
```yaml
ruler:
sidecar:
enabled: true
searchNamespace: cortex-rules
alertmanager:
sidecar:
enabled: true
searchNamespace: cortex-alertmanager
```
And here are the related configuration values in AlertManager and Ruler:
```yaml
config:
alertmanager:
data_dir: /data/
storage:
type : local
local:
path: /data
ruler:
rule_path: /data/rules
storage:
type : local
local:
directory: /tmp/rules
```
In AlertManager, the data_dir and local storage directory should be the same.
In the Ruler, there needs to be two separate volumes. One is read-only and serves as the location shared with the sidecar that contains all of the rules that were derived from configmaps (/tmp/rules). The other is read-write and used by the Ruler itself for its own management of rules, etc (/data).
Example ConfigMap containing a rule:
```yaml
kind: ConfigMap
metadata:
annotations:
k8s-sidecar-target-directory: /tmp/rules/fake
labels:
# Label cortex_rules must exist unless overridden by ruler.sidecar.label
cortex_rules: "1"
name: rules-cortex-9f99md47tc
namespace: cortex-rules
apiVersion: v1
data:
fake.yaml: |-
groups:
- name: fake-system-metrics
rules:
- alert: HighCPUusage
expr: avg(100 - rate(node_cpu_seconds_total{instance=~"qag1ge1l.+",mode="idle"}[5m]) * 100) by (instance) > 100
for: 3m
labels:
severity: warning
annotations:
description: Metrics from {{"{{ $labels.job }}"}} on {{"{{ $labels.instance }}"}} show CPU > 90% for 3m.
title: Node {{"{{ $labels.instance }}"}} has high CPU usage
```
Example ConfigMap containing an alertmanager-config:
```yaml
apiVersion: v1
kind: ConfigMap
metadata:
annotations:
k8s-sidecar-target-directory: /data/fake
labels:
cortex_alertmanager: "1"
name: alertmanager-example-config
data:
fake.yaml: |-
global:
resolve_timeout: 5m
http_config: {}
smtp_hello: localhost
smtp_require_tls: true
route:
receiver: team-X-mails
group_by:
- alertname
routes:
- receiver: "null"
match:
alertname: Watchdog
group_wait: 30s
group_interval: 5m
repeat_interval: 12h
receivers:
- name: 'team-X-mails'
email_configs:
- to: '[email protected]'
```
{{ template "chart.requirementsSection" . }}
{{ template "chart.valuesSection" . }}