Failed to index documents in 'traces-apm-default' (resource_not_found_exception) after recreating integration

[zvazquez]

Hi,

We are trying to configure one of our Elasticsearch to receive APM traces, the expectation is that after the configuration of APM integration it will create all the required data stream to ingest traces into this elasticsearch server.

Instead in the APM server we are receiving the following message:

{"log.level":"error","@timestamp":"2025-01-16T23:29:43.380Z","log.origin":{"function":"github.com/elastic/go-docappender/v2.(*Appender).flush","file.name":"[email protected]/appender.go","file.line":443},"message":"failed to index documents in 'traces-apm-default' (resource_not_found_exception): [require_data_stream] request flag is [true] and [traces-apm-default] is not a data stream","service.name":"apm-server","documents":5,"ecs.version":"1.6.0"}
{"log.level":"error","@timestamp":"2025-01-16T23:30:01.032Z","log.origin":{"function":"github.com/elastic/go-docappender/v2.(*Appender).flush","file.name":"[email protected]/appender.go","file.line":443},"message":"failed to index documents in 'traces-apm-default' (resource_not_found_exception): [require_data_stream] request flag is [true] and [traces-apm-default] is not a data stream","service.name":"apm-server","documents":7,"ecs.version":"1.6.0"}

We have delete and recreate the agent policies and integration multiple times, restarted the APM server, Elasticsearch and Kibana and it does not create the required artifacts that we can see in other Elasticsearch systems where we can ingest traces without a problem.

We have follow the comments mentioned here

This is an initial setup that we are doing in these systems based on version 8.16.2, the other systems were configured in older versions.

Thanks in advance for your help,

Zareh

[marclop]

@zvazquez I'm afraid that issues in this repository are for confirmed bug reports. The issue you're running into seems like a set up issue and not a bug. If you could provide the same information in https://discuss.elastic.co/c/observability/82, we'll look into it.

The issue you reference is no longer applicable in the way the APM assets are installed, since those are now shipped through Elasticsearch.

Looking at the error message:

'traces-apm-default' (resource_not_found_exception): [require_data_stream] request flag is [true] and [traces-apm-default] is not a data stream

That seems to indicate that a traces-apm-default index exists in the system, but isn't a data stream (was it created manually?). I recommend removing that index in the stack management UI and reopen this issue in our discuss forum if the issue persists.