The metricpipeline.telemetry.kyma-project.io CustomResourceDefinition (CRD) is a detailed description of the kind of data and the format used to filter and ship metric data in Kyma. To get the current CRD and show the output in the YAML format, run this command:
kubectl get crd metricpipeline.telemetry.kyma-project.io -o yamlThe following MetricPipeline object defines a pipeline that integrates into an OTLP backend:
apiVersion: telemetry.kyma-project.io/v1alpha1
kind: MetricPipeline
metadata:
name: otlp
generation: 1
spec:
input:
application:
prometheus:
enabled: false
istio:
enabled: false
runtime:
enabled: false
output:
otlp:
endpoint:
value: https://myBackend:4317
status:
conditions:
- lastTransitionTime: "2024-01-09T07:02:16Z"
message: "Metric agent DaemonSet is ready"
observedGeneration: 1
reason: AgentReady
status: "True"
type: AgentHealthy
- lastTransitionTime: "2024-01-08T10:40:18Z"
message: "Metric gateway Deployment is ready"
observedGeneration: 1
reason: GatewayReady
status: "True"
type: GatewayHealthy
- lastTransitionTime: "2023-12-28T11:27:04Z"
message: ""
observedGeneration: 1
reason: ConfigurationGenerated
status: "True"
type: ConfigurationGeneratedFor further examples, see the samples directory.
For details, see the MetricPipeline specification file.
Spec:
| Parameter | Type | Description |
|---|---|---|
| input | object | Configures different inputs to send additional metrics to the metric gateway. |
| input.istio | object | Configures istio-proxy metrics scraping. |
| input.istio.diagnosticMetrics | object | Configures diagnostic metrics scraping |
| input.istio.diagnosticMetrics.enabled | boolean | If enabled, diagnostic metrics are scraped. The default is false. |
| input.istio.enabled | boolean | If enabled, istio-proxy metrics are scraped from Pods that have the istio-proxy sidecar injected. The default is false. |
| input.istio.namespaces | object | Describes whether istio-proxy metrics from specific namespaces are selected. System namespaces are enabled by default. |
| input.istio.namespaces.exclude | []string | Exclude signals from the specified Namespace names only. |
| input.istio.namespaces.include | []string | Include signals from the specified Namespace names only. |
| input.otlp | object | Configures the collection of push-based metrics that use the OpenTelemetry protocol. |
| input.otlp.disabled | boolean | If disabled, push-based OTLP signals are not collected. The default is false. |
| input.otlp.namespaces | object | Describes whether push-based OTLP signals from specific namespaces are selected. System namespaces are enabled by default. |
| input.otlp.namespaces.exclude | []string | Exclude signals from the specified Namespace names only. |
| input.otlp.namespaces.include | []string | Include signals from the specified Namespace names only. |
| input.prometheus | object | Configures Prometheus scraping. |
| input.prometheus.diagnosticMetrics | object | Configures diagnostic metrics scraping |
| input.prometheus.diagnosticMetrics.enabled | boolean | If enabled, diagnostic metrics are scraped. The default is false. |
| input.prometheus.enabled | boolean | If enabled, Services and Pods marked with prometheus.io/scrape=true annotation are scraped. The default is false. |
| input.prometheus.namespaces | object | Describes whether Prometheus metrics from specific namespaces are selected. System namespaces are disabled by default. |
| input.prometheus.namespaces.exclude | []string | Exclude signals from the specified Namespace names only. |
| input.prometheus.namespaces.include | []string | Include signals from the specified Namespace names only. |
| input.runtime | object | Configures runtime scraping. |
| input.runtime.enabled | boolean | If enabled, runtime metrics are scraped. The default is false. |
| input.runtime.namespaces | object | Describes whether runtime metrics from specific namespaces are selected. System namespaces are disabled by default. |
| input.runtime.namespaces.exclude | []string | Exclude signals from the specified Namespace names only. |
| input.runtime.namespaces.include | []string | Include signals from the specified Namespace names only. |
| input.runtime.resources | object | Describes the Kubernetes resources for which runtime metrics are scraped. |
| input.runtime.resources.container | object | Configures container runtime metrics scraping. |
| input.runtime.resources.container.enabled | boolean | If enabled, the runtime metrics for the resource are scraped. The default is true. |
| input.runtime.resources.daemonset | object | Configures DaemonSet runtime metrics scraping. |
| input.runtime.resources.daemonset.enabled | boolean | If enabled, the runtime metrics for the resource are scraped. The default is false. |
| input.runtime.resources.deployment | object | Configures Deployment runtime metrics scraping. |
| input.runtime.resources.deployment.enabled | boolean | If enabled, the runtime metrics for the resource are scraped. The default is false. |
| input.runtime.resources.job | object | Configures Job runtime metrics scraping. |
| input.runtime.resources.job.enabled | boolean | If enabled, the runtime metrics for the resource are scraped. The default is false. |
| input.runtime.resources.node | object | Configures Node runtime metrics scraping. |
| input.runtime.resources.node.enabled | boolean | If enabled, the runtime metrics for the resource are scraped. The default is false. |
| input.runtime.resources.pod | object | Configures Pod runtime metrics scraping. |
| input.runtime.resources.pod.enabled | boolean | If enabled, the runtime metrics for the resource are scraped. The default is true. |
| input.runtime.resources.statefulset | object | Configures StatefulSet runtime metrics scraping. |
| input.runtime.resources.statefulset.enabled | boolean | If enabled, the runtime metrics for the resource are scraped. The default is false. |
| input.runtime.resources.volume | object | Configures Volume runtime metrics scraping. |
| input.runtime.resources.volume.enabled | boolean | If enabled, the runtime metrics for the resource are scraped. The default is false. |
| output | object | Configures the metric gateway. |
| output.otlp (required) | object | Defines an output using the OpenTelemetry protocol. |
| output.otlp.authentication | object | Defines authentication options for the OTLP output |
| output.otlp.authentication.basic | object | Activates Basic authentication for the destination providing relevant Secrets. |
| output.otlp.authentication.basic.password (required) | object | Contains the basic auth password or a Secret reference. |
| output.otlp.authentication.basic.password.value | string | The value as plain text. |
| output.otlp.authentication.basic.password.valueFrom | object | The value as a reference to a resource. |
| output.otlp.authentication.basic.password.valueFrom.secretKeyRef | object | Refers to the value of a specific key in a Secret. You must provide name and namespace of the Secret, as well as the name of the key. |
| output.otlp.authentication.basic.password.valueFrom.secretKeyRef.key (required) | string | The name of the attribute of the Secret holding the referenced value. |
| output.otlp.authentication.basic.password.valueFrom.secretKeyRef.name (required) | string | The name of the Secret containing the referenced value |
| output.otlp.authentication.basic.password.valueFrom.secretKeyRef.namespace (required) | string | The name of the Namespace containing the Secret with the referenced value. |
| output.otlp.authentication.basic.user (required) | object | Contains the basic auth username or a Secret reference. |
| output.otlp.authentication.basic.user.value | string | The value as plain text. |
| output.otlp.authentication.basic.user.valueFrom | object | The value as a reference to a resource. |
| output.otlp.authentication.basic.user.valueFrom.secretKeyRef | object | Refers to the value of a specific key in a Secret. You must provide name and namespace of the Secret, as well as the name of the key. |
| output.otlp.authentication.basic.user.valueFrom.secretKeyRef.key (required) | string | The name of the attribute of the Secret holding the referenced value. |
| output.otlp.authentication.basic.user.valueFrom.secretKeyRef.name (required) | string | The name of the Secret containing the referenced value |
| output.otlp.authentication.basic.user.valueFrom.secretKeyRef.namespace (required) | string | The name of the Namespace containing the Secret with the referenced value. |
| output.otlp.endpoint (required) | object | Defines the host and port (:) of an OTLP endpoint. |
| output.otlp.endpoint.value | string | The value as plain text. |
| output.otlp.endpoint.valueFrom | object | The value as a reference to a resource. |
| output.otlp.endpoint.valueFrom.secretKeyRef | object | Refers to the value of a specific key in a Secret. You must provide name and namespace of the Secret, as well as the name of the key. |
| output.otlp.endpoint.valueFrom.secretKeyRef.key (required) | string | The name of the attribute of the Secret holding the referenced value. |
| output.otlp.endpoint.valueFrom.secretKeyRef.name (required) | string | The name of the Secret containing the referenced value |
| output.otlp.endpoint.valueFrom.secretKeyRef.namespace (required) | string | The name of the Namespace containing the Secret with the referenced value. |
| output.otlp.headers | []object | Defines custom headers to be added to outgoing HTTP or GRPC requests. |
| output.otlp.headers.name (required) | string | Defines the header name. |
| output.otlp.headers.prefix | string | Defines an optional header value prefix. The prefix is separated from the value by a space character. |
| output.otlp.headers.value | string | The value as plain text. |
| output.otlp.headers.valueFrom | object | The value as a reference to a resource. |
| output.otlp.headers.valueFrom.secretKeyRef | object | Refers to the value of a specific key in a Secret. You must provide name and namespace of the Secret, as well as the name of the key. |
| output.otlp.headers.valueFrom.secretKeyRef.key (required) | string | The name of the attribute of the Secret holding the referenced value. |
| output.otlp.headers.valueFrom.secretKeyRef.name (required) | string | The name of the Secret containing the referenced value |
| output.otlp.headers.valueFrom.secretKeyRef.namespace (required) | string | The name of the Namespace containing the Secret with the referenced value. |
| output.otlp.path | string | Defines OTLP export URL path (only for the HTTP protocol). This value overrides auto-appended paths /v1/metrics and /v1/traces |
| output.otlp.protocol | string | Defines the OTLP protocol (http or grpc). Default is grpc. |
| output.otlp.tls | object | Defines TLS options for the OTLP output. |
| output.otlp.tls.ca | object | Defines an optional CA certificate for server certificate verification when using TLS. The certificate must be provided in PEM format. |
| output.otlp.tls.ca.value | string | The value as plain text. |
| output.otlp.tls.ca.valueFrom | object | The value as a reference to a resource. |
| output.otlp.tls.ca.valueFrom.secretKeyRef | object | Refers to the value of a specific key in a Secret. You must provide name and namespace of the Secret, as well as the name of the key. |
| output.otlp.tls.ca.valueFrom.secretKeyRef.key (required) | string | The name of the attribute of the Secret holding the referenced value. |
| output.otlp.tls.ca.valueFrom.secretKeyRef.name (required) | string | The name of the Secret containing the referenced value |
| output.otlp.tls.ca.valueFrom.secretKeyRef.namespace (required) | string | The name of the Namespace containing the Secret with the referenced value. |
| output.otlp.tls.cert | object | Defines a client certificate to use when using TLS. The certificate must be provided in PEM format. |
| output.otlp.tls.cert.value | string | The value as plain text. |
| output.otlp.tls.cert.valueFrom | object | The value as a reference to a resource. |
| output.otlp.tls.cert.valueFrom.secretKeyRef | object | Refers to the value of a specific key in a Secret. You must provide name and namespace of the Secret, as well as the name of the key. |
| output.otlp.tls.cert.valueFrom.secretKeyRef.key (required) | string | The name of the attribute of the Secret holding the referenced value. |
| output.otlp.tls.cert.valueFrom.secretKeyRef.name (required) | string | The name of the Secret containing the referenced value |
| output.otlp.tls.cert.valueFrom.secretKeyRef.namespace (required) | string | The name of the Namespace containing the Secret with the referenced value. |
| output.otlp.tls.insecure | boolean | Defines whether to send requests using plaintext instead of TLS. |
| output.otlp.tls.insecureSkipVerify | boolean | Defines whether to skip server certificate verification when using TLS. |
| output.otlp.tls.key | object | Defines the client key to use when using TLS. The key must be provided in PEM format. |
| output.otlp.tls.key.value | string | The value as plain text. |
| output.otlp.tls.key.valueFrom | object | The value as a reference to a resource. |
| output.otlp.tls.key.valueFrom.secretKeyRef | object | Refers to the value of a specific key in a Secret. You must provide name and namespace of the Secret, as well as the name of the key. |
| output.otlp.tls.key.valueFrom.secretKeyRef.key (required) | string | The name of the attribute of the Secret holding the referenced value. |
| output.otlp.tls.key.valueFrom.secretKeyRef.name (required) | string | The name of the Secret containing the referenced value |
| output.otlp.tls.key.valueFrom.secretKeyRef.namespace (required) | string | The name of the Namespace containing the Secret with the referenced value. |
Status:
| Parameter | Type | Description |
|---|---|---|
| conditions | []object | An array of conditions describing the status of the pipeline. |
| conditions.lastTransitionTime (required) | string | lastTransitionTime is the last time the condition transitioned from one status to another. This should be when the underlying condition changed. If that is not known, then using the time when the API field changed is acceptable. |
| conditions.message (required) | string | message is a human readable message indicating details about the transition. This may be an empty string. |
| conditions.observedGeneration | integer | observedGeneration represents the .metadata.generation that the condition was set based upon. For instance, if .metadata.generation is currently 12, but the .status.conditions[x].observedGeneration is 9, the condition is out of date with respect to the current state of the instance. |
| conditions.reason (required) | string | reason contains a programmatic identifier indicating the reason for the condition's last transition. Producers of specific condition types may define expected values and meanings for this field, and whether the values are considered a guaranteed API. The value should be a CamelCase string. This field may not be empty. |
| conditions.status (required) | string | status of the condition, one of True, False, Unknown. |
| conditions.type (required) | string | type of condition in CamelCase or in foo.example.com/CamelCase. |
The status of the MetricPipeline is determined by the condition types GatewayHealthy, AgentHealthy, ConfigurationGenerated, and TelemetryFlowHealthy:
| Condition Type | Condition Status | Condition Reason | Condition Message |
|---|---|---|---|
| GatewayHealthy | True | GatewayReady | Metric gateway Deployment is ready |
| GatewayHealthy | True | RolloutInProgress | Pods are being started/updated |
| GatewayHealthy | False | GatewayNotReady | No Pods deployed |
| GatewayHealthy | False | GatewayNotReady | Failed to list ReplicaSets: reason |
| GatewayHealthy | False | GatewayNotReady | Failed to fetch ReplicaSets: reason |
| GatewayHealthy | False | GatewayNotReady | Pod is not scheduled: reason |
| GatewayHealthy | False | GatewayNotReady | Pod is in the pending state because container: container name is not running due to: reason. Please check the container: container name logs. |
| GatewayHealthy | False | GatewayNotReady | Pod is in the failed state due to: reason |
| GatewayHealthy | False | GatewayNotReady | Deployment is not yet created |
| GatewayHealthy | False | GatewayNotReady | Failed to get Deployment |
| GatewayHealthy | False | GatewayNotReady | Failed to get latest ReplicaSets |
| AgentHealthy | True | AgentNotRequired | |
| AgentHealthy | True | AgentReady | Metric agent DaemonSet is ready |
| AgentHealthy | True | RolloutInProgress | Pods are being started/updated |
| AgentHealthy | False | AgentNotReady | No Pods deployed |
| AgentHealthy | False | AgentNotReady | DaemonSet is not yet created |
| AgentHealthy | False | AgentNotReady | Failed to get DaemonSet |
| AgentHealthy | False | AgentNotReady | Pod is in the pending state because container: container name is not running due to: reason |
| AgentHealthy | False | AgentNotReady | Pod is in the failed state due to: reason |
| ConfigurationGenerated | True | AgentGatewayConfigured | MetricPipeline specification is successfully applied to the configuration of Metric gateway |
| ConfigurationGenerated | True | TLSCertificateAboutToExpire | TLS (CA) certificate is about to expire, configured certificate is valid until YYYY-MM-DD |
| ConfigurationGenerated | False | EndpointInvalid | OTLP output endpoint invalid: reason |
| ConfigurationGenerated | False | MaxPipelinesExceeded | Maximum pipeline count limit exceeded |
| ConfigurationGenerated | False | ReferencedSecretMissing | One or more referenced Secrets are missing: Secret 'my-secret' of Namespace 'my-namespace' |
| ConfigurationGenerated | False | ReferencedSecretMissing | One or more keys in a referenced Secret are missing: Key 'my-key' in Secret 'my-secret' of Namespace 'my-namespace'" |
| ConfigurationGenerated | False | ReferencedSecretMissing | Secret reference is missing field/s: (field1, field2, ...) |
| ConfigurationGenerated | False | TLSCertificateExpired | TLS (CA) certificate expired on YYYY-MM-DD |
| ConfigurationGenerated | False | TLSConfigurationInvalid | TLS configuration invalid |
| ConfigurationGenerated | False | ValidationFailed | Pipeline validation failed due to an error from the Kubernetes API server |
| TelemetryFlowHealthy | True | FlowHealthy | No problems detected in the telemetry flow |
| TelemetryFlowHealthy | False | AllDataDropped | Backend is not reachable or rejecting metrics. All metrics are dropped. See troubleshooting: No Metrics Arrive at the Backend |
| TelemetryFlowHealthy | False | BufferFillingUp | Buffer nearing capacity. Incoming log rate exceeds export rate. See troubleshooting: Gateway Buffer Filling Up |
| TelemetryFlowHealthy | False | GatewayThrottling | Metric gateway is unable to receive metrics at current rate. See troubleshooting: Gateway Throttling |
| TelemetryFlowHealthy | False | SomeDataDropped | Backend is reachable, but rejecting metrics. Some metrics are dropped. See troubleshooting: Not All Metrics Arrive at the Backend |
| TelemetryFlowHealthy | False | ConfigurationNotGenerated | No metrics delivered to backend because MetricPipeline specification is not applied to the configuration of Metric gateway. Check the 'ConfigurationGenerated' condition for more details |
| TelemetryFlowHealthy | Unknown | ProbingFailed | Could not determine the health of the telemetry flow because the self monitor probing failed |