Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[auditd_manager] Update fields and sample_event.json #12541

Draft
wants to merge 5 commits into
base: main
Choose a base branch
from

Conversation

mrodm
Copy link
Contributor

@mrodm mrodm commented Jan 30, 2025

Proposed commit message

Update field definitions to be valid when elastic-package uses validation based on mappings.

Errors can be found in this PR: https://buildkite.com/elastic/integrations/builds/21222

test case failed: one or more errors found in mappings in logs-auditd_manager.auditd index template:
[0] field "auditd.data.a0" is undefined: field definition not found
[1] field "auditd.data.a1" is undefined: field definition not found
[2] field "auditd.data.a2" is undefined: field definition not found
[3] field "auditd.data.a3" is undefined: field definition not found

To solve these issues this PR :

  • changes flattened type by a dynamic template where all string fields are converted to keyword
  • removed the definition of auditd.data.a0-N since this is not taken into account.

Checklist

  • I have reviewed tips for building integrations and this pull request is aligned with them.
  • I have verified that all data streams collect metrics or logs.
  • I have added an entry to my package's changelog.yml file.
  • I have verified that Kibana version constraints are current according to guidelines.
  • I have verified that any added dashboard complies with Kibana's Dashboard good practices

Author's Checklist

How to test this PR locally

Run elastic-package with mappings validation enabled and run system tests:

cd packages/auditd_manager
elastic-package stack up -v -d --version <kibana_constraint>
export ELASTIC_PACKAGE_FIELD_VALIDATION_TEST_METHOD=mappings
# Add defer-cleanup to check the data ingested in the corresponding indices/data streams
elastic-package test system -v --defer-cleanup 900s

elastic-package stack down -v

Related issues

@mrodm mrodm self-assigned this Jan 30, 2025
Comment on lines +742 to +744
- name: auditd.data.*
description: Auditd related data
type: flattened
type: keyword
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If this field is declared as flattened, this is not created as mapping currently.

Probably, because there are other fields auditd.data.xxx.

If it set as a dynamic template, it keeps all the fields above plus the ones not able to match (auditd.data.a0-N)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If flattened is expected maybe this should be defined without the wildcard:

- name: auditd.data
  description: Auditd related data
  type: flattened

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If that definition is changed as:

- name: auditd.data
  description: Auditd related data
  type: flattened

This mapping is not created. I think this is caused because there are other definitions present for auditd.data.<field> fields. For instance auditd.data.action (as keyword).

If it is required to be flattened, I'm afraid it should be needed to remove all the auditd.data.<field> definitions.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This mapping is not created. I think this is caused because there are other definitions present for auditd.data. fields. For instance auditd.data.action (as keyword).

On what version of the stack? This looks like a case of elastic/kibana#204104

Comment on lines +626 to +630
# this mapping does not generate a dynamic template, and the expected fields do not match
# should it be kept for documentation purposes?
# - name: auditd.data.a0-N
# description: the arguments to a syscall
# type: keyword
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What should it be done for this field definition ? Just remove it?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would it work with a auditd.data.a* definition, specially to keep the documentation?

Suggested change
# this mapping does not generate a dynamic template, and the expected fields do not match
# should it be kept for documentation purposes?
# - name: auditd.data.a0-N
# description: the arguments to a syscall
# type: keyword
- name: auditd.data.a*
description: the arguments to a syscall
type: keyword

Though this would match also things that are not arguments.

So maybe this can be removed, yes.

"a3": "0",
"arch": "x86_64",
"audit_pid": "22501",
"audit_pid": 2532842,
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It was required to re-generate the sample_event since this field is now a long (as in the field definition).

@andrewkroh andrewkroh added bugfix Pull request that fixes a bug issue Integration:auditd_manager Auditd Manager labels Jan 30, 2025
@elastic-vault-github-plugin-prod

🚀 Benchmarks report

To see the full report comment with /test benchmark fullreport

@andrewkroh andrewkroh added Integration:1password 1Password Integration:abnormal_security Abnormal Security New Integration Issue or pull request for creating a new integration package. and removed Integration:auditd_manager Auditd Manager labels Feb 4, 2025
@mrodm mrodm force-pushed the update_auditd_manager_mappings branch from e32c560 to f54601d Compare February 4, 2025 10:04
@mrodm mrodm added Integration:auditd_manager Auditd Manager and removed New Integration Issue or pull request for creating a new integration package. Integration:1password 1Password Integration:abnormal_security Abnormal Security labels Feb 4, 2025
@mrodm mrodm force-pushed the update_auditd_manager_mappings branch 2 times, most recently from 1e8b1b3 to af08a22 Compare February 5, 2025 17:27
@mrodm mrodm force-pushed the update_auditd_manager_mappings branch from 6b08b22 to d2ba547 Compare February 6, 2025 12:53
@elasticmachine
Copy link

💚 Build Succeeded

History

cc @mrodm

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bugfix Pull request that fixes a bug issue Integration:auditd_manager Auditd Manager
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants