From ed20177c187cc2538af67e73b339dc32eec79322 Mon Sep 17 00:00:00 2001 From: mdbirnstiehl Date: Mon, 5 Aug 2024 11:02:12 -0500 Subject: [PATCH] review updates --- README.md | 14 +++++++------- docs/collector-components.md | 7 ++++--- docs/collector-limitations.md | 9 ++++++--- docs/configure-upstream-collector.md | 8 ++++---- docs/guided-onboarding.md | 8 ++++---- docs/manual-configuration.md | 2 +- 6 files changed, 26 insertions(+), 22 deletions(-) diff --git a/README.md b/README.md index 9b48d7c..d101e33 100644 --- a/README.md +++ b/README.md @@ -1,12 +1,12 @@ # Elastic Distributions for OpenTelemetry -[OpenTelemetry](https://opentelemetry.io/docs/) is a vendor- and tool-neutral observability framework for collecting, processing, and exporting telemetry data. -The Elastic Distribution for OpenTelemetry (Elastic OTel Collector) is a supported distribution of the [OpenTelemetry Collector](https://github.com/open-telemetry/opentelemetry-collector) made up of mostly upstream components. +[OpenTelemetry](https://opentelemetry.io/docs/) is a vendor-neutral observability framework for collecting, processing, and exporting telemetry data. +The Elastic Distribution for OpenTelemetry Collector (Elastic OTel Collector) is a supported, drop-in replacement distribution of the [OpenTelemetry Collector](https://github.com/open-telemetry/opentelemetry-collector) made up of mostly upstream components. You can send your telemetry data to Elastic Observability using OpenTelemetry the following ways: - Collect and send logs and host metrics to [Elastic Cloud](https://cloud.elastic.co/) using the Elastic OTel Collector. -- Instrument your applications and send logs, traces, and metrics to [Elastic Cloud](https://cloud.elastic.co/) using the **Elastic Distributions for OpenTelemetry** extensions of select [OpenTelemetry language SDKs](https://opentelemetry.io/docs/languages/). The following languages are currently supported: Java, .NET, Node.js, and Python. -- Configure an upstream [custom collector](https://opentelemetry.io/docs/collector/custom-collector/) or [contrib](https://github.com/open-telemetry/opentelemetry-collector-contrib) distribution for the collector to collect logs and metrics and send them to Elastic Observability. +- Instrument your applications and send logs, traces, and metrics to [Elastic Cloud](https://cloud.elastic.co/) using the Elastic Distributions for select [OpenTelemetry language SDKs](https://opentelemetry.io/docs/languages/). Currently, Elastic provides distributions for the following language SDKs: Java, .NET, Node.js, and Python. +- Build and configure a [custom collector](https://opentelemetry.io/docs/collector/custom-collector/) or extend the [OpenTelemetry Collector Contrib](https://github.com/open-telemetry/opentelemetry-collector-contrib) distribution to collect logs and metrics and send them to Elastic Observability. This diagram provides a quick overview on how the different components work together. Refer to the [components](docs/collector-components.md) for a more in-depth look. @@ -23,13 +23,13 @@ These pages detail the components and how to configure the Elastic OTel Collecto ## Collect application data with Elastic Distributions for OpenTelemetry language SDKs -Elastic offers several distributions that extend [OpenTelemetry language SDKs](https://opentelemetry.io/docs/languages/). The following languages are currently supported: +Elastic offers several Distributions that extend [OpenTelemetry language SDKs](https://opentelemetry.io/docs/languages/). The following languages are currently supported: * [Java](https://github.com/elastic/elastic-otel-java) * [.NET](https://github.com/elastic/elastic-otel-dotnet) * [Node.js](https://github.com/elastic/elastic-otel-node) * [Python](https://github.com/elastic/elastic-otel-python) -## Configure an upstream collector for Elastic +## Configure a custom collector or the OpenTelemetry Collector Contrib distribution -[Configure a custom collector or a contrib distribution of the OpenTelemetry collector](docs/configure-upstream-collector.md): Configure a [custom collector](https://opentelemetry.io/docs/collector/custom-collector/) or [contrib](https://github.com/open-telemetry/opentelemetry-collector-contrib) distribution for the OpenTelemetry collector to collect logs and metrics and send them to Elastic Observability. \ No newline at end of file +[Configure a custom collector or the OpenTelemetry Collector Contrib distribution](docs/configure-upstream-collector.md): Build and configure a [custom collector](https://opentelemetry.io/docs/collector/custom-collector/) or extend the [OpenTelemetry Collector Contrib](https://github.com/open-telemetry/opentelemetry-collector-contrib) distribution to collect logs and metrics and send them to Elastic Observability. \ No newline at end of file diff --git a/docs/collector-components.md b/docs/collector-components.md index 15e6656..0c26589 100644 --- a/docs/collector-components.md +++ b/docs/collector-components.md @@ -7,7 +7,7 @@ The OpenTelemetry Collector uses the following components to receive, process, a - [Exporters](collector-components.md#exporters): Send data to the backends or destinations. - [Extensions](collector-components.md#extensions): Provide additional functionalities and capabilities. -The default configurations of the Elastic OTel Collector follows these flows. +The default configurations of the Elastic Distribution for the OpenTelemetry Collector follows these flows. **MacOS and Linux Host metrics:** @@ -84,9 +84,9 @@ flowchart LR style six fill: #e6f9f7,stroke:#333,stroke-width:1px ``` -Refer to the following tables for more information on the components included in the Elastic Distribution for the OpenTelemetry Collector. +Refer to the following tables for more information on the components included in the Elastic OTel Collector. Follow the links for OpenTelemetry documentation with more configuration details for each component. -To set up the collector, go to the [guided onboarding](guided-onboarding.md) or [manual configuration](manual-configuration.md) to get started. +To set up the Elastic OTel Collector, get started using the [guided onboarding](guided-onboarding.md) docs or the [manual configuration](manual-configuration.md) docs. ## Receivers @@ -104,6 +104,7 @@ To set up the collector, go to the [guided onboarding](guided-onboarding.md) or | Component | Description | |---|---| +| [`elasticinframetricsprocessor`](https://github.com/elastic/opentelemetry-collector-components/blob/processor/elasticinframetricsprocessor/v0.7.1/processor/elasticinframetricsprocessor/README.md) | Processes infrastructure metrics to enhance and convert them for Elasticsearch. | | [`attributesprocessor`](https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/processor/attributesprocessor/v0.105.0/processor/attributesprocessor/README.md) | Modifies telemetry data attributes. | | [`filterprocessor`](https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/processor/filterprocessor/v0.105.0/processor/filterprocessor/README.md) | Filters telemetry data to include or exclude specific data points. | | [`k8sattributesprocessor`](https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/processor/k8sattributesprocessor/v0.105.0/processor/k8sattributesprocessor/README.md) | Enhances telemetry data with Kubernetes-specific metadata. | diff --git a/docs/collector-limitations.md b/docs/collector-limitations.md index ec62720..abfe3cd 100644 --- a/docs/collector-limitations.md +++ b/docs/collector-limitations.md @@ -1,8 +1,11 @@ # Elastic Distribution for OpenTelemetry Collector limitations -The Elastic OTel Collector has the following limitations: +The Elastic Distribution for the OpenTelemetry Collector has the following limitations: - Because of an upstream limitation, `host.network.*` metrics aren't present from the OpenTelemetry side. - `process.state` isn't present in the OpenTelemetry host metric. It's set to a dummy value of **Unknown** in the **State** column of the host processes table. -- The Elasticsearch exporter handles the metadata fields, but **Host OS version** and **Operating system** may show as "N/A" and **Host IP** may show different values. -- The CPU scraper needs to be enabled to collect the `systm.load.cores` metric, which affects the **Normalized Load** column in the **Hosts** table and the **Normalized Load** visualization on the host detailed view. \ No newline at end of file +- The Elasticsearch exporter handles the resource attributes, but **Host OS version** and **Operating system** may show as "N/A". +- The CPU scraper needs to be enabled to collect the `systm.load.cores` metric, which affects the **Normalized Load** column in the **Hosts** table and the **Normalized Load** visualization on the host detailed view. +- The [`hostmetrics receiver`](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/receiver/hostmetricsreceiver) doesn't support CPU and disk metrics on MacOS. These values will stay empty for collectors running on MacOS. +- The console shows error Log messages when the [`hostmetrics receiver`](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/receiver/hostmetricsreceiver) can't access some of the process information due to permission issues. +- The console shows mapping errors initially until mapping occurs. \ No newline at end of file diff --git a/docs/configure-upstream-collector.md b/docs/configure-upstream-collector.md index 6ba3234..2a76ca3 100644 --- a/docs/configure-upstream-collector.md +++ b/docs/configure-upstream-collector.md @@ -1,9 +1,9 @@ -# Configure an upstream collector +# Configure a custom collector or the OpenTelemetry Collector Contrib distribution -You can configure an upstream collector, like a [custom collector](https://opentelemetry.io/docs/collector/custom-collector/) or [contrib](https://github.com/open-telemetry/opentelemetry-collector-contrib) distribution for the collector, to collect logs and metrics and send them to Elastic Observability. +You can build and configure a [custom collector](https://opentelemetry.io/docs/collector/custom-collector/) or extend the [OpenTelemetry Collector Contrib ](https://github.com/open-telemetry/opentelemetry-collector-contrib) distribution to collect logs and metrics and send them to Elastic Observability. -For a more seamless experience, use the Elastic distribution for the OpenTelemetry collector. -Refer to the [guided onboarding](guided-onboarding.md) docs or the [manual configuration](manual-configuration.md) docs for more on configuring the Elastic distribution. +For a more seamless experience, use the Elastic Distribution for the OpenTelemetry collector. +Refer to the [guided onboarding](guided-onboarding.md) docs or the [manual configuration](manual-configuration.md) docs for more on configuring the Elastic OTel collector. ## Upstream collector configuration examples Refer to the OpenTelemetry documentation on [building a custom collector](https://opentelemetry.io/docs/collector/custom-collector/) for more on creating an upstream collector. diff --git a/docs/guided-onboarding.md b/docs/guided-onboarding.md index 240cc9f..491c148 100644 --- a/docs/guided-onboarding.md +++ b/docs/guided-onboarding.md @@ -1,6 +1,6 @@ # Collect logs and metrics using the guided onboarding -The guided onboarding in Kibana or in a serverless Observability project walks you through collecting logs and metrics using the Elastic OTel Collector. -For manual configuration, refer to [Manually configure the Elastic OTel Collector](manual-configuration.md). +The guided onboarding in Kibana or in a serverless Observability project walks you through collecting logs and metrics using the Elastic Distribution for OpenTelemetry Collector. +To configure the Elastic OTel Collector manually, refer to the [manual configuration](manual-configuration.md) docs. ## Before you begin The onboarding has the following requirements and limitations: @@ -12,14 +12,14 @@ The onboarding has the following requirements and limitations: ## Collect your logs and metrics -Follow these steps to collect logs and metrics using the Elastic OTel collector +Follow these steps to collect logs and metrics using the Elastic OTel collector: 1. Open an [Elastic Cloud](cloud.elastic.co) deployment or a serverless Observability project. 1. To open the guided onboarding, either: 1. In an Elastic Cloud deployment, open Kibana, and go to **Observability** → **Add Data**. 1. In a serverless Observability project, go to **Add Data**. 1. Select **Collect and analyze logs**, and then select **OpenTelemetry**. -1. Select the appropriate platform: +1. Select the appropriate platform, and complete the following: 1. For **MacOS and Linux**, copy the command, open a terminal on your host, and run the command to download and configure the OpenTelemetry collector. 1. For **Kubernetes**, download the manifest. 1. Copy the command under Step 2: diff --git a/docs/manual-configuration.md b/docs/manual-configuration.md index 3eb78d7..5f10db5 100644 --- a/docs/manual-configuration.md +++ b/docs/manual-configuration.md @@ -1,5 +1,5 @@ # Manually configure the Elastic Distribution for OpenTelemetry Collector -Collecting logs and host metrics with the Elastic OTel Collector without using the [guided onboarding](guided-onboarding.md) requires some manual configuration. +Collecting logs and host metrics with the Elastic Distribution for OpenTelemetry Collector without using the [guided onboarding](guided-onboarding.md) requires some manual configuration. ## Before you begin The Elastic OTel collector has the following requirements and limitations: