Merge pull request #3905 from Yelp/u/ahaswell/perel-4183-smartstack-s…

…chema-alerting-extensions smartstack alert schema
Yelp · Jul 3, 2024 · d3d4338 · d3d4338
2 parents 7c82a62 + f88376c
commit d3d4338
Show file tree

Hide file tree

Showing 2 changed files with 63 additions and 1 deletion.
diff --git a/docs/source/yelpsoa_configs.rst b/docs/source/yelpsoa_configs.rst
@@ -854,7 +854,7 @@ See the `marathon-[clustername].yaml`_ section for details for each of these par
 ``smartstack.yaml``
 -------------------
 
-Configure service registration, discovery, and load balancing.
+Configure service registration, discovery, load balancing and error alerting.
 
 Here is an example smartstack.yaml::
 
@@ -1069,6 +1069,36 @@ internally at Yelp.
      delay. Note that due to Envoy's current definition of *percentage* as an
      integer, this cannot be specified as a floating-point number.
 
+Error Alerting
+``````````````
+
+These keys provide optional overrides for the default alerting behaviour.
+
+ * ``monitoring``: Override default alerting behaviour. For example: ::
+
+      main:
+        monitoring:
+          team: frontend
+          slack_channel: "notifications"
+          project: "FRONTEND"
+          ticket: true
+          page: false
+          page_nonprod: true
+          error_threshold_ratio: 0.02
+          minimum_error_rps: 10
+    - ``team``: Override the default team for alerting.
+    - ``slack_channel``: Error alerts notify the first channel in the monitoring.yaml slack_channels list.
+      Use this key if you prefer a different channel.
+    - ``project``: Override the default JIRA project for alerting.
+    - ``ticket``: Override the default ticketing behaviour. Error Alert ticketing defaults to **false** but also
+      respects the ticketing behaviour set in the monitoring.yaml file. Override that here if required.
+    - ``page``: Override the default paging behaviour. Error Alert paging defaults to **true** but also
+      respects the paging behaviour set in the monitoring.yaml file. Override that here if required.
+    - ``page_nonprod``: Override the default paging behaviour for non-production
+      environments. Defaults to **false**.
+    - ``error_threshold_ratio``: Error threshold ratio (0-1). Defaults to **0.01**.
+    - ``minimum_error_rps``: Minimum error rate per second, minimum is zero. Defaults to **5**.
+
 Moving a Service to a different location type
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 If you don’t care about dropping traffic you can just change ``discover`` and

diff --git a/paasta_tools/cli/schemas/smartstack_schema.json b/paasta_tools/cli/schemas/smartstack_schema.json
@@ -227,6 +227,38 @@
             "vip_upgrade": {
                 "type": "string",
                 "description": "The service namespace that this namespace will route to (via Envoy) if a request has the X-Ctx-Vip header."
+            },
+            "monitoring": {
+                "type": "object",
+                "properties": {
+                    "team": {
+                        "type": "string"
+                    },
+                    "slack_channel": {
+                        "type": "string"
+                    },
+                    "project": {
+                        "type": "string"
+                    },
+                    "ticket": {
+                        "type": "boolean"
+                    },
+                    "page": {
+                        "type": "boolean"
+                    },
+                    "page_nonprod": {
+                        "type": "boolean"
+                    },
+                    "error_threshold_ratio": {
+                        "type": "number",
+                        "minimum": 0,
+                        "maximum": 1
+                    },
+                    "minimum_error_rps": {
+                        "type": "integer",
+                        "minimum": 0
+                    }
+                }
             }
         },
         "required": [