Skip to content

Commit

Permalink
Merge branch 'main' into enh/support-bundle-env
Browse files Browse the repository at this point in the history
  • Loading branch information
dehaansa committed Nov 16, 2024
2 parents 235d88b + df34fdb commit d1798ac
Show file tree
Hide file tree
Showing 109 changed files with 1,749 additions and 7,785 deletions.
12 changes: 11 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,15 +18,25 @@ Main (unreleased)

- Add `otelcol.exporter.splunkhec` allowing to export otel data to Splunk HEC (@adlotsof)

- Add `otelcol.receiver.solace` component to receive traces from a Solace broker. (@wildum)

### Enhancements

- Add second metrics sample to the support bundle to provide delta information (@dehaansa)

- Add relevant golang environment variables to the support bundle (@dehaansa)

### Bugfixes

- Fixed an issue in the `prometheus.exporter.postgres` component that would leak goroutines when the target was not reachable (@dehaansa)

v1.5.0-rc.1
- Fixed issue with reloading configuration and prometheus metrics duplication in `prometheus.write.queue`. (@mattdurham)

### Other changes

- Change the stability of the `livedebugging` feature from "experimental" to "generally available". (@wildum)

v1.5.0
-----------------

### Breaking changes
Expand Down
2 changes: 1 addition & 1 deletion Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -141,7 +141,7 @@ lint: alloylint
# final command runs tests for all other submodules.
test:
$(GO_ENV) go test $(GO_FLAGS) -race $(shell go list ./... | grep -v /integration-tests/)
$(GO_ENV) go test $(GO_FLAGS) ./internal/static/integrations/node_exporter ./internal/static/logs ./internal/component/otelcol/processor/tail_sampling ./internal/component/loki/source/file ./internal/component/loki/source/docker ./internal/component/prometheus/write/queue/serialization ./internal/component/prometheus/write/queue/network
$(GO_ENV) go test $(GO_FLAGS) ./internal/static/integrations/node_exporter ./internal/static/logs ./internal/component/otelcol/processor/tail_sampling ./internal/component/loki/source/file ./internal/component/loki/source/docker
$(GO_ENV) find . -name go.mod -not -path "./go.mod" -execdir go test -race ./... \;

test-packages:
Expand Down
13 changes: 0 additions & 13 deletions docs/developer/release/11-update-otel.md

This file was deleted.

1 change: 0 additions & 1 deletion docs/developer/release/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,6 @@ responsible for ownership of the following workflows:
5. [Update Helm Charts](./8-update-helm-charts.md)
6. [Update Homebrew](./9-update-homebrew.md)
7. [Announce Release](./10-announce-release.md)
8. [Update OTEL Contrib](./11-update-otel.md)

## Patch Release Publish - latest version (`1.15.1`, `1.15.2`...)
1. [Update the "main" and "release/" branches](./3-update-version-in-code.md)
Expand Down
102 changes: 102 additions & 0 deletions docs/sources/collect/ecs-openteletry-data.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,102 @@
---
canonical: https://grafana.com/docs/alloy/latest/collect/ecs-opentelemetry-data/
description: Learn how to collect Amazon ECS or AWS Fargate OpenTelemetry data and forward it to any OpenTelemetry-compatible endpoint
menuTitle: Collect ECS or Fargate OpenTelemetry data
title: Collect Amazon Elastic Container Service or AWS Fargate OpenTelemetry data
weight: 500
---

# Collect Amazon Elastic Container Service or AWS Fargate OpenTelemetry data

You can configure {{< param "FULL_PRODUCT_NAME" >}} to collect OpenTelemetry-compatible data from Amazon Elastic Container Service (ECS) or AWS Fargate and forward it to any OpenTelemetry-compatible endpoint.

There are three different ways you can use {{< param "PRODUCT_NAME" >}} to collect Amazon ECS or AWS Fargate telemetry data.

1. [Use a custom OpenTelemetry configuration file from the SSM Parameter store](#use-a-custom-opentelemetry-configuration-file-from-the-ssm-parameter-store).
1. [Create an ECS task definition](#create-an-ecs-task-definition).
1. [Run {{< param "PRODUCT_NAME" >}} directly in your instance, or as a Kubernetes sidecar](#run-alloy-directly-in-your-instance-or-as-a-kubernetes-sidecar).

## Before you begin

* Ensure that you have basic familiarity with instrumenting applications with OpenTelemetry.
* Have an available Amazon ECS or AWS Fargate deployment.
* Identify where {{< param "PRODUCT_NAME" >}} writes received telemetry data.
* Be familiar with the concept of [Components][] in {{< param "PRODUCT_NAME" >}}.

## Use a custom OpenTelemetry configuration file from the SSM Parameter store

You can upload a custom OpenTelemetry configuration file to the SSM Parameter store and use {{< param "PRODUCT_NAME" >}} as a telemetry data collector.

You can configure the AWS Distro for OpenTelemetry Collector with the `AOT_CONFIG_CONTENT` environment variable.
This environment variable contains a full collector configuration file and it overrides the configuration file used in the collector entry point command.
In ECS, you can set the values of environment variables from AWS Systems Manager Parameters.

### Update the task definition

1. Select the task definition.

1. Open the AWS Systems Manager console.
1. Select Elastic Container Service.
1. In the navigation pane, choose *Task definition*.
1. Choose *Create new revision*.

1. Add an environment variable.

1. Select the AWS Distro for OpenTelemetry Collector container and navigate to the Environment variables section.
1. Add an environment variable named `AOT_CONFIG_CONTENT`.
1. Select `ValueFrom` to tell ECS to get the value from the SSM Parameter, and set the value to `otel-collector-config`.

1. Finish updating the task definition and create your revision.

### Create the SSM parameter

1. Open the AWS Systems Manager console.
1. In the navigation pane, choose *Parameter Store*.
1. Choose *Create parameter*.
1. Create a parameter with the following values:

* `Name`: otel-collector-config
* `Tier`: Standard
* `Type`: String
* `Data type`: Text
* `Value`: Copy and paste your custom OpenTelemetry configuration file or [{{< param "PRODUCT_NAME" >}} configuration file][configure].

### Run your task

When you run a task with this Task Definition, it uses your custom OpenTelemetry configuration file from the SSM Parameter store.

Refer to [Running an application as an Amazon ECS task][run] for more information about running your application as a task.

## Create an ECS Task definition

To create an ECS Task Definition for AWS Fargate with an ADOT collector, complete the following steps.

1. Download the [ECS Fargate task definition template][template] from GitHub.
1. Edit the task definition template and add the following parameters.
* `{{region}}`: The region the data is sent to.
* `{{ecsTaskRoleArn}}`: The AWSOTTaskRole ARN.
* `{{ecsExecutionRoleArn}}`: The AWSOTTaskExcutionRole ARN.
* `command` - Assign a value to the command variable to select the path to the configuration file.
The AWS Collector comes with two configurations. Select one of them based on your environment:
* Use `--config=/etc/ecs/ecs-default-config.yaml` to consume StatsD metrics, OTLP metrics and traces, and X-Ray SDK traces.
* Use `--config=/etc/ecs/container-insights/otel-task-metrics-config.yaml` to use StatsD, OTLP, Xray, and Container Resource utilization metrics.
1. Follow the ECS Fargate setup instructions to [create a task definition][task] using the template.

## Run {{% param "PRODUCT_NAME" %}} directly in your instance, or as a Kubernetes sidecar

SSH or connect to the Amazon ECS or AWS Fargate-managed container. Refer to [9 steps to SSH into an AWS Fargate managed container][steps] for more information about using SSH with Amazon ECS or AWS Fargate.

You can also use your own method to connect to the Amazon ECS or AWS Fargate-managed container as long as you can pass the parameters needed to install and configure {{< param "PRODUCT_NAME" >}}.

### Install Grafana Alloy

After connecting to your instance, follow the {{< param "PRODUCT_NAME" >}} [installation][install], [configuration][configure] and [deployment][deploy] instructions.

[Components]: https://grafana.com/docs/alloy/<ALLOY_VERSION>/get-started/components
[template]: https://github.com/aws-observability/aws-otel-collector/blob/master/examples/ecs/aws-cloudwatch/ecs-fargate-sidecar.json
[configure]: https://grafana.com/docs/alloy/<ALLOY_VERSION>/configure/
[steps]: https://medium.com/ci-t/9-steps-to-ssh-into-an-aws-fargate-managed-container-46c1d5f834e2
[install]: https://grafana.com/docs/alloy/<ALLOY_VERSION>/set-up/install/linux/
[deploy]: https://grafana.com/docs/alloy/<ALLOY_VERSION>/set-up/deploy/
[task]: https://docs.aws.amazon.com/AmazonECS/latest/developerguide/task_definitions.html
[run]: https://docs.aws.amazon.com/AmazonECS/latest/developerguide/standalone-task-create.html
2 changes: 1 addition & 1 deletion docs/sources/collect/prometheus-metrics.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ This topic describes how to:

Before components can collect Prometheus metrics, you must have a component responsible for writing those metrics somewhere.

The [prometheus.remote_write][] component is responsible for delivering Prometheus metrics to one or Prometheus-compatible endpoints.
The [prometheus.remote_write][] component is responsible for delivering Prometheus metrics to one or more Prometheus-compatible endpoints.
After a `prometheus.remote_write` component is defined, you can use other {{< param "PRODUCT_NAME" >}} components to forward metrics to it.

To configure a `prometheus.remote_write` component for metrics delivery, complete the following steps:
Expand Down
1 change: 1 addition & 0 deletions docs/sources/reference/compatibility/_index.md
Original file line number Diff line number Diff line change
Expand Up @@ -359,6 +359,7 @@ The following components, grouped by namespace, _consume_ OpenTelemetry `otelcol
- [otelcol.receiver.opencensus](../components/otelcol/otelcol.receiver.opencensus)
- [otelcol.receiver.otlp](../components/otelcol/otelcol.receiver.otlp)
- [otelcol.receiver.prometheus](../components/otelcol/otelcol.receiver.prometheus)
- [otelcol.receiver.solace](../components/otelcol/otelcol.receiver.solace)
- [otelcol.receiver.vcenter](../components/otelcol/otelcol.receiver.vcenter)
- [otelcol.receiver.zipkin](../components/otelcol/otelcol.receiver.zipkin)
{{< /collapse >}}
Expand Down
10 changes: 9 additions & 1 deletion docs/sources/reference/components/local/local.file.md
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,9 @@ Name | Type | Description
----------|----------------------|---------------------------------------------------
`content` | `string` or `secret` | The contents of the file from the most recent read

The `content` field will have the `secret` type only if the `is_secret` argument was true.
The `content` field will have the `secret` type only if the `is_secret` argument was true.

You can use `local.file.LABEL.content` to access the contents of the file.

## Component health

Expand All @@ -66,9 +68,15 @@ The read error will be exposed as a log message and in the debug information for

## Example

The following example shows a simple `local.file` configuration that watches a passwords text file and uses the exported content field.

```alloy
local.file "secret_key" {
filename = "/var/secrets/password.txt"
is_secret = true
}
grafana_cloud.stack "receivers" {
stack_name = "mystack"
token = local.file.secret_key.content
}
```
Loading

0 comments on commit d1798ac

Please sign in to comment.