entityanalytics_okta: split user and device data into their own data streams

[efd6]

Proposed commit message

entityanalytics_okta: split user and device data into their own data streams

The entityanalytics integrations obtain entity data from a variety of
reasonably consistent data sources relating to system entities. These include
users and devices, and their associated attributes. Initially the
implementations of these integration only reported and indexed user data, but
device data collection was added to the integrations whose data sources made
that data available. In the case of the entra_id integration the user and
device documents were separated into separated data streams, enabling a more
refined approach to entity-based risk assessment. In this integration this
was not done, resulting in user and device data being placed in a data stream
labelled as user data. This can lead to confusing results being presented to
analysts.

So this change separates user data from device data and presents them in
separate data streams, leaving any non-user, non-device data in the
generalised entity data stream. The change does not alter the behaviour of
the user data stream beyond ensuring that device data does not get included
and so is not a breaking change. However, the upgrade path is not transparent
so this is marked as a v2 and an upgrade note is added to the documentation.

Checklist

• I have reviewed tips for building integrations and this pull request is aligned with them.
• I have verified that all data streams collect metrics or logs.
• I have added an entry to my package's changelog.yml file.
• I have verified that Kibana version constraints are current according to guidelines.
• I have verified that any added dashboard complies with Kibana's Dashboard good practices

Author's Checklist

• [ ]

How to test this PR locally

Related issues

• Closes [entityanalytics_okta]: device assets mixed up with user assets #12657

Screenshots

[elastic-vault-github-plugin-prod]

🚀 Benchmarks report

To see the full report comment with /test benchmark fullreport

[elasticmachine]

Pinging @elastic/security-service-integrations (Team:Security-Service Integrations)

[efd6]

Upgrade path is bumpy. Investigating.

[andrewkroh]

On the Add Integration page, I see three separate inputs. I think that in this file we should only have one instance of the entity-analytics input on the entity data stream.

Suggested change

	inputs:
	- type: entity-analytics
	title: Collect device identities
	description: Collecting device identities from Okta.

[kcreddy]

Just a thought:
Should we rename v1 and v2 to 1.x and 2.x, or users infer them anyway?

[efd6]

I'm finding it hard to get my brain into a state where v1 is not 1.x and v2 is not 2.x, though I do see that v1 is used immediately above referring to the API, so I will add, "of the integration" to the end of the sentence.

[kcreddy]

Probably a link to https://www.elastic.co/guide/en/fleet/current/upgrade-integration.html#resolve-conflicts might be helpful

[efd6]

The issue is not that fields fail to match, it's that the configuration is no longer visible since the name of the policy that holds it has changed, but I will add that link.

[elasticmachine]

💚 Build Succeeded

• Buildkite Build
• Commit: 1c587ac

History

• 💚 Build #22829 succeeded f061396
• 💚 Build #22793 succeeded 545f770
• 💚 Build #22680 succeeded 17318f9
• 💚 Build #22669 succeeded d746cda
• 💔 Build #22666 failed 5691c6eb313d5315c2a918a82ef7a7e1e3061a34
• 💚 Build #22364 succeeded 9328a62

cc @efd6

[elastic-sonarqube]

Quality Gate failed

Failed conditions
75.1% Coverage on New Code (required ≥ 80%)

See analysis details on SonarQube

[elastic-vault-github-plugin-prod]

Package entityanalytics_okta - 2.0.0 containing this change is available at https://epr.elastic.co/package/entityanalytics_okta/2.0.0/