Create Elastic OpenTelemetry Collector docs #1
Conversation
There was a problem hiding this comment.
@mdbirnstiehl I'm leaving a preliminary review of the README with some questions/suggestions around terminology. Maybe we can get input from @mlunadia on terminology here and apply it throughout the files in this PR.
I'll move on to review the other docs now, but I won't clutter those with any more terminology suggestions at this point.
Co-authored-by: Colleen McGinnis <[email protected]>
Co-authored-by: Colleen McGinnis <[email protected]>
docs/configure-upstream-collector.md
Outdated
|
|
||
| You can configure an upstream collector, like a [custom collector](https://opentelemetry.io/docs/collector/custom-collector/) or [contrib](https://github.com/open-telemetry/opentelemetry-collector-contrib) distribution for the collector, to collect logs and metrics and send them to Elastic Observability. | ||
|
|
||
| <!-- We might want to add some pros and cons of using an upstream collector --> |
There was a problem hiding this comment.
@mlunadia would it be valuable to layout some pros and cons of using an upstream collector vs. the elastic distro?
| The Elastic Distribution for the OpenTelemetry Collector has the following limitations: | ||
|
|
||
| - `host.network.*` metrics aren't present from the OpenTelemetry side. | ||
| - `process.state` isn't present in the OpenTelemetry host metric. It's set to a dummy value of **Unknown** in the **State** column of the host processes table. |
There was a problem hiding this comment.
@ishleenk17 is this an upstream limitation?
There was a problem hiding this comment.
The process.state is not present in the Hostmetrics receiver. So, yes its an upstream limitation.
At Elastic, we show the state as Unknown for all processes because of this limitation.
| Refer to the [guided onboarding](guided-onboarding.md) docs or the [manual configuration](manual-configuration.md) docs for more on configuring the Elastic distribution. | ||
|
|
||
| ## Upstream collector configuration examples | ||
| Refer to the OpenTelemetry documentation on [building a custom collector](https://opentelemetry.io/docs/collector/custom-collector/) for more on creating an upstream collector. |
There was a problem hiding this comment.
we need more concrete steps on how exactly to build a custom collector that contains all the components required. Could we maybe even provide the config file for ocb? @ishleenk17 could you help with the concrete steps here?
| The Elastic Distribution for the OpenTelemetry Collector has the following limitations: | ||
|
|
||
| - `host.network.*` metrics aren't present from the OpenTelemetry side. | ||
| - `process.state` isn't present in the OpenTelemetry host metric. It's set to a dummy value of **Unknown** in the **State** column of the host processes table. |
There was a problem hiding this comment.
The process.state is not present in the Hostmetrics receiver. So, yes its an upstream limitation.
At Elastic, we show the state as Unknown for all processes because of this limitation.
There was a problem hiding this comment.
@mdbirnstiehl I'm good for this to be merged so we can start sharing it with people who can provide feedback when trying it out.
* docs: add onboarding onprem Operator values * fix: autogenerate level * docs: add Operator commands * enable Go auto instrumentation * exclude operator logs * add missing configuration to the onprem_kube_stack_values.yaml (#1) Signed-off-by: Tetiana Kravchenko <[email protected]> * chore: use OTEL_K8S_NODE_NAME env var * docs: add information about the Helm deployment * add namespace creation command * collector configurations enhancements (#2) * splite otel and ecs pipelines for logs and metrics Signed-off-by: Tetiana Kravchenko <[email protected]> * remove default k8sobjects config; add storeCheckpoints for logs; add clusterName Signed-off-by: Tetiana Kravchenko <[email protected]> * comment out tls.insecure_skip_verify Signed-off-by: Tetiana Kravchenko <[email protected]> * remove in deployment: elasticsearch/apm, remove in daemon: hostmetrics from metrics/node/otel pipeline Signed-off-by: Tetiana Kravchenko <[email protected]> --------- Signed-off-by: Tetiana Kravchenko <[email protected]> * pin Chart version * add Java, Go, Node.js, .NET and Python autoinstrumentation Co-authored-by: jackshirazi <[email protected]> * Update docs/onboarding/8_16/operator/values.yaml Co-authored-by: jackshirazi <[email protected]> * Update docs/onboarding/8_16/operator/README.md Co-authored-by: jackshirazi <[email protected]> * remove k8s_logs_dataset; add configmaps clusterRole rule (#3) * remove k8s_logs_dataset; add configmaps clusterRole rule Signed-off-by: Tetiana Kravchenko <[email protected]> * add a comment for clusterName Signed-off-by: Tetiana Kravchenko <[email protected]> --------- Signed-off-by: Tetiana Kravchenko <[email protected]> * docs: remove suggestion needed backslashes * bump chart version --------- Signed-off-by: Tetiana Kravchenko <[email protected]> Co-authored-by: Tetiana Kravchenko <[email protected]> Co-authored-by: jackshirazi <[email protected]>
This PR creates the docs for the Elastic distribution of the OpenTelemetry Collector.
I've created some draft diagrams just to get conversation going, but once they are finalized we can make them look better.
I still need to create documentation on configuring a custom collector (aka the brownfield issue) as well.
Please feel free to add any additional reviewers.