Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add reference to prefer-ilm setting for apm lifeycle known bug #4501

Merged
merged 6 commits into from
Nov 8, 2024
Merged
Changes from 5 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
68 changes: 16 additions & 52 deletions docs/en/observability/apm/known-issues.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -24,10 +24,11 @@ _Versions: XX.XX.XX, YY.YY.YY, ZZ.ZZ.ZZ_
[discrete]
== Upgrading to v8.15.x may cause ingestion to fail

_Elastic Stack versions: 8.15.0+_
_Elastic Stack versions: 8.15.0, 8.15.1, 8.15.2, 8.15.3_ +
_Fixed in Elastic Stack version 8.15.4_

// The conditions in which this issue occurs
The issue only occurs when _upgrading_ the {stack} from 8.12.2 or lower directly to any 8.15.x version.
The issue only occurs when _upgrading_ the {stack} from 8.12.2 or lower directly to any 8.15.x version prior to 8.15.4.
The issue does _not_ occur when creating a _new_ cluster using any 8.15.x version, or when upgrading
from 8.12.2 to 8.13.x or 8.14.x and then to 8.15.x.

Expand Down Expand Up @@ -84,63 +85,26 @@ stream lifecycle configuration, such existing data streams become unmanaged for
lifecycle configurations.

// How to fix it
Upgrading to 8.15.1 should fix any new indices created for the data stream. However,
indices created in version 8.15.0 would remain unmanaged if the default ILM policy is
used. One of the following approaches can be adopted to fix the unmanaged indices:

. Manually delete the indices when they are no longer needed.
. Explicitly configure APM data streams with the default data stream lifecycle config.
Using this approach would migrate all data streams to use data stream lifecycles,
which should be equivalent to the default ILM policies:
Upgrading to 8.15.1 resolves the lifecycle issue for any new indices created for APM data streams.
However, indices created in version 8.15.0 will remain unmanaged if the default ILM policy is in place.
To fix these unmanaged indices, consider one of the following approaches:

. Manually delete the unmanaged indices when they are no longer needed.
. Explicitly configure APM data streams to use the default data stream lifecycle configuration.
This approach migrates all affected data streams to use data stream lifecycles,
maintaining behavior equivalent to the default ILM policies.
Apply this fix only to data streams that are currently unmanaged due to default ILM policies.
bmorelli25 marked this conversation as resolved.
Show resolved Hide resolved
+
[source,txt]
----
PUT _data_stream/traces-apm-*/_lifecycle
PUT _data_stream/{{data_stream_name}}-{{data_stream_namespace}}/_lifecycle
{
"data_retention": "10d"
}

PUT _data_stream/traces-apm.rum*/_lifecycle
{
"data_retention": "90d"
}

PUT _data_stream/traces-apm.sampled*/_lifecycle
{
"data_retention": "1h"
}

PUT _data_stream/metrics-apm.*.1m-*/_lifecycle
{
"data_retention": "90d"
}

PUT _data_stream/metrics-apm.*.10m-*/_lifecycle
{
"data_retention": "180d"
}

PUT _data_stream/metrics-apm.*.60m-*/_lifecycle
{
"data_retention": "390d"
}

PUT _data_stream/metrics-apm.internal-*/_lifecycle
{
"data_retention": "90d"
}

PUT _data_stream/metrics-apm.app.*/_lifecycle
{
"data_retention": "90d"
}

PUT _data_stream/logs-apm.*/_lifecycle
{
"data_retention": "10d"
"data_retention": <data_retention_period>
}
----

Default `<data_retention_period>` for each data stream is available in {observability-guide}/apm-ilm-how-to.html[this guide].

// Link to fix if it exists
This issue is fixed in 8.15.1 (https://github.com/elastic/elasticsearch/pull/112432[elastic/elasticsearch#112432]).

Expand Down