-
Notifications
You must be signed in to change notification settings - Fork 8.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Security Solution] Tests for prebuilt rule customization workflow #202068
Comments
Pinging @elastic/security-solution (Team: SecuritySolution) |
Pinging @elastic/security-detection-rule-management (Team:Detection Rule Management) |
Pinging @elastic/security-detections-response (Team:Detections and Resp) |
@dplumlee Please check this ticket and prioritize working on it after bugfixing. Before writing any tests and test plans, try to plan your work and share the plan with me and the team, so we could give feedback on the scope, approach, etc. |
cc @pborgonovi |
## Summary Addresses #202068 Adds test plan for rule customization features related to the milestone 3 prebuilt rule customization epic
…c#204888) ## Summary Addresses elastic#202068 Adds test plan for rule customization features related to the milestone 3 prebuilt rule customization epic (cherry picked from commit ded92cf)
…c#204888) ## Summary Addresses elastic#202068 Adds test plan for rule customization features related to the milestone 3 prebuilt rule customization epic
…part 1 (#211300) **Epic:** #174168 **Partially addresses:** #202068, #202078, #202079 ## Summary We're cleaning up and refactoring our existing test plans for prebuilt rule customization, upgrade, and export/import workflows. Specifically, in this PR we're consistently: - Changing the file names and test plan titles. - Adding or expanding test plan summaries. - Regenerating tables of contents using the [Markdown All in One](https://marketplace.visualstudio.com/items?itemName=yzhang.markdown-all-in-one) VS Code extension. No "functional" changes have been made to any test plans, such as adding, removing, or updating any scenarios. This refactoring prepares the test plans for being "functionally" changed and improved in follow-up PRs. For example, we're going to cover the logic described in #210358 and address any other gaps in coverage.
…part 1 (elastic#211300) **Epic:** elastic#174168 **Partially addresses:** elastic#202068, elastic#202078, elastic#202079 ## Summary We're cleaning up and refactoring our existing test plans for prebuilt rule customization, upgrade, and export/import workflows. Specifically, in this PR we're consistently: - Changing the file names and test plan titles. - Adding or expanding test plan summaries. - Regenerating tables of contents using the [Markdown All in One](https://marketplace.visualstudio.com/items?itemName=yzhang.markdown-all-in-one) VS Code extension. No "functional" changes have been made to any test plans, such as adding, removing, or updating any scenarios. This refactoring prepares the test plans for being "functionally" changed and improved in follow-up PRs. For example, we're going to cover the logic described in elastic#210358 and address any other gaps in coverage. (cherry picked from commit 9600de4)
…part 1 (elastic#211300) **Epic:** elastic#174168 **Partially addresses:** elastic#202068, elastic#202078, elastic#202079 ## Summary We're cleaning up and refactoring our existing test plans for prebuilt rule customization, upgrade, and export/import workflows. Specifically, in this PR we're consistently: - Changing the file names and test plan titles. - Adding or expanding test plan summaries. - Regenerating tables of contents using the [Markdown All in One](https://marketplace.visualstudio.com/items?itemName=yzhang.markdown-all-in-one) VS Code extension. No "functional" changes have been made to any test plans, such as adding, removing, or updating any scenarios. This refactoring prepares the test plans for being "functionally" changed and improved in follow-up PRs. For example, we're going to cover the logic described in elastic#210358 and address any other gaps in coverage. (cherry picked from commit 9600de4)
…part 1 (elastic#211300) **Epic:** elastic#174168 **Partially addresses:** elastic#202068, elastic#202078, elastic#202079 ## Summary We're cleaning up and refactoring our existing test plans for prebuilt rule customization, upgrade, and export/import workflows. Specifically, in this PR we're consistently: - Changing the file names and test plan titles. - Adding or expanding test plan summaries. - Regenerating tables of contents using the [Markdown All in One](https://marketplace.visualstudio.com/items?itemName=yzhang.markdown-all-in-one) VS Code extension. No "functional" changes have been made to any test plans, such as adding, removing, or updating any scenarios. This refactoring prepares the test plans for being "functionally" changed and improved in follow-up PRs. For example, we're going to cover the logic described in elastic#210358 and address any other gaps in coverage. (cherry picked from commit 9600de4)
…lans, part 1 (#211300) (#211307) # Backport This will backport the following commits from `main` to `8.x`: - [[Security Solution] Refactor prebuilt rule customization test plans, part 1 (#211300)](#211300) <!--- Backport version: 9.4.3 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sqren/backport) <!--BACKPORT [{"author":{"name":"Georgii Gorbachev","email":"[email protected]"},"sourceCommit":{"committedDate":"2025-02-14T18:22:45Z","message":"[Security Solution] Refactor prebuilt rule customization test plans, part 1 (#211300)\n\n**Epic:** https://github.com/elastic/kibana/issues/174168\r\n**Partially addresses:**\r\nhttps://github.com//issues/202068,\r\nhttps://github.com//issues/202078,\r\nhttps://github.com//issues/202079\r\n\r\n## Summary\r\n\r\nWe're cleaning up and refactoring our existing test plans for prebuilt\r\nrule customization, upgrade, and export/import workflows.\r\n\r\nSpecifically, in this PR we're consistently:\r\n\r\n- Changing the file names and test plan titles.\r\n- Adding or expanding test plan summaries.\r\n- Regenerating tables of contents using the [Markdown All in\r\nOne](https://marketplace.visualstudio.com/items?itemName=yzhang.markdown-all-in-one)\r\nVS Code extension.\r\n\r\nNo \"functional\" changes have been made to any test plans, such as\r\nadding, removing, or updating any scenarios.\r\n\r\nThis refactoring prepares the test plans for being \"functionally\"\r\nchanged and improved in follow-up PRs. For example, we're going to cover\r\nthe logic described in https://github.com/elastic/kibana/issues/210358\r\nand address any other gaps in coverage.","sha":"9600de4338edc123dbac7c2cb3403ede2f016752","branchLabelMapping":{"^v9.1.0$":"main","^v8.19.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","test-plan","v9.0.0","Team:Detections and Resp","Team: SecuritySolution","Team:Detection Rule Management","Feature:Prebuilt Detection Rules","backport:version","v8.18.0","v9.1.0","v8.19.0"],"title":"[Security Solution] Refactor prebuilt rule customization test plans, part 1","number":211300,"url":"https://github.com/elastic/kibana/pull/211300","mergeCommit":{"message":"[Security Solution] Refactor prebuilt rule customization test plans, part 1 (#211300)\n\n**Epic:** https://github.com/elastic/kibana/issues/174168\r\n**Partially addresses:**\r\nhttps://github.com//issues/202068,\r\nhttps://github.com//issues/202078,\r\nhttps://github.com//issues/202079\r\n\r\n## Summary\r\n\r\nWe're cleaning up and refactoring our existing test plans for prebuilt\r\nrule customization, upgrade, and export/import workflows.\r\n\r\nSpecifically, in this PR we're consistently:\r\n\r\n- Changing the file names and test plan titles.\r\n- Adding or expanding test plan summaries.\r\n- Regenerating tables of contents using the [Markdown All in\r\nOne](https://marketplace.visualstudio.com/items?itemName=yzhang.markdown-all-in-one)\r\nVS Code extension.\r\n\r\nNo \"functional\" changes have been made to any test plans, such as\r\nadding, removing, or updating any scenarios.\r\n\r\nThis refactoring prepares the test plans for being \"functionally\"\r\nchanged and improved in follow-up PRs. For example, we're going to cover\r\nthe logic described in https://github.com/elastic/kibana/issues/210358\r\nand address any other gaps in coverage.","sha":"9600de4338edc123dbac7c2cb3403ede2f016752"}},"sourceBranch":"main","suggestedTargetBranches":["9.0","8.18","8.x"],"targetPullRequestStates":[{"branch":"9.0","label":"v9.0.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"8.18","label":"v8.18.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"main","label":"v9.1.0","branchLabelMappingKey":"^v9.1.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/211300","number":211300,"mergeCommit":{"message":"[Security Solution] Refactor prebuilt rule customization test plans, part 1 (#211300)\n\n**Epic:** https://github.com/elastic/kibana/issues/174168\r\n**Partially addresses:**\r\nhttps://github.com//issues/202068,\r\nhttps://github.com//issues/202078,\r\nhttps://github.com//issues/202079\r\n\r\n## Summary\r\n\r\nWe're cleaning up and refactoring our existing test plans for prebuilt\r\nrule customization, upgrade, and export/import workflows.\r\n\r\nSpecifically, in this PR we're consistently:\r\n\r\n- Changing the file names and test plan titles.\r\n- Adding or expanding test plan summaries.\r\n- Regenerating tables of contents using the [Markdown All in\r\nOne](https://marketplace.visualstudio.com/items?itemName=yzhang.markdown-all-in-one)\r\nVS Code extension.\r\n\r\nNo \"functional\" changes have been made to any test plans, such as\r\nadding, removing, or updating any scenarios.\r\n\r\nThis refactoring prepares the test plans for being \"functionally\"\r\nchanged and improved in follow-up PRs. For example, we're going to cover\r\nthe logic described in https://github.com/elastic/kibana/issues/210358\r\nand address any other gaps in coverage.","sha":"9600de4338edc123dbac7c2cb3403ede2f016752"}},{"branch":"8.x","label":"v8.19.0","branchLabelMappingKey":"^v8.19.0$","isSourceBranch":false,"state":"NOT_CREATED"}]}] BACKPORT--> Co-authored-by: Georgii Gorbachev <[email protected]>
…lans, part 1 (#211300) (#211308) # Backport This will backport the following commits from `main` to `9.0`: - [[Security Solution] Refactor prebuilt rule customization test plans, part 1 (#211300)](#211300) <!--- Backport version: 9.4.3 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sqren/backport) <!--BACKPORT [{"author":{"name":"Georgii Gorbachev","email":"[email protected]"},"sourceCommit":{"committedDate":"2025-02-14T18:22:45Z","message":"[Security Solution] Refactor prebuilt rule customization test plans, part 1 (#211300)\n\n**Epic:** https://github.com/elastic/kibana/issues/174168\r\n**Partially addresses:**\r\nhttps://github.com//issues/202068,\r\nhttps://github.com//issues/202078,\r\nhttps://github.com//issues/202079\r\n\r\n## Summary\r\n\r\nWe're cleaning up and refactoring our existing test plans for prebuilt\r\nrule customization, upgrade, and export/import workflows.\r\n\r\nSpecifically, in this PR we're consistently:\r\n\r\n- Changing the file names and test plan titles.\r\n- Adding or expanding test plan summaries.\r\n- Regenerating tables of contents using the [Markdown All in\r\nOne](https://marketplace.visualstudio.com/items?itemName=yzhang.markdown-all-in-one)\r\nVS Code extension.\r\n\r\nNo \"functional\" changes have been made to any test plans, such as\r\nadding, removing, or updating any scenarios.\r\n\r\nThis refactoring prepares the test plans for being \"functionally\"\r\nchanged and improved in follow-up PRs. For example, we're going to cover\r\nthe logic described in https://github.com/elastic/kibana/issues/210358\r\nand address any other gaps in coverage.","sha":"9600de4338edc123dbac7c2cb3403ede2f016752","branchLabelMapping":{"^v9.1.0$":"main","^v8.19.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","test-plan","v9.0.0","Team:Detections and Resp","Team: SecuritySolution","Team:Detection Rule Management","Feature:Prebuilt Detection Rules","backport:version","v8.18.0","v9.1.0","v8.19.0"],"title":"[Security Solution] Refactor prebuilt rule customization test plans, part 1","number":211300,"url":"https://github.com/elastic/kibana/pull/211300","mergeCommit":{"message":"[Security Solution] Refactor prebuilt rule customization test plans, part 1 (#211300)\n\n**Epic:** https://github.com/elastic/kibana/issues/174168\r\n**Partially addresses:**\r\nhttps://github.com//issues/202068,\r\nhttps://github.com//issues/202078,\r\nhttps://github.com//issues/202079\r\n\r\n## Summary\r\n\r\nWe're cleaning up and refactoring our existing test plans for prebuilt\r\nrule customization, upgrade, and export/import workflows.\r\n\r\nSpecifically, in this PR we're consistently:\r\n\r\n- Changing the file names and test plan titles.\r\n- Adding or expanding test plan summaries.\r\n- Regenerating tables of contents using the [Markdown All in\r\nOne](https://marketplace.visualstudio.com/items?itemName=yzhang.markdown-all-in-one)\r\nVS Code extension.\r\n\r\nNo \"functional\" changes have been made to any test plans, such as\r\nadding, removing, or updating any scenarios.\r\n\r\nThis refactoring prepares the test plans for being \"functionally\"\r\nchanged and improved in follow-up PRs. For example, we're going to cover\r\nthe logic described in https://github.com/elastic/kibana/issues/210358\r\nand address any other gaps in coverage.","sha":"9600de4338edc123dbac7c2cb3403ede2f016752"}},"sourceBranch":"main","suggestedTargetBranches":["9.0","8.18","8.x"],"targetPullRequestStates":[{"branch":"9.0","label":"v9.0.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"8.18","label":"v8.18.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"main","label":"v9.1.0","branchLabelMappingKey":"^v9.1.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/211300","number":211300,"mergeCommit":{"message":"[Security Solution] Refactor prebuilt rule customization test plans, part 1 (#211300)\n\n**Epic:** https://github.com/elastic/kibana/issues/174168\r\n**Partially addresses:**\r\nhttps://github.com//issues/202068,\r\nhttps://github.com//issues/202078,\r\nhttps://github.com//issues/202079\r\n\r\n## Summary\r\n\r\nWe're cleaning up and refactoring our existing test plans for prebuilt\r\nrule customization, upgrade, and export/import workflows.\r\n\r\nSpecifically, in this PR we're consistently:\r\n\r\n- Changing the file names and test plan titles.\r\n- Adding or expanding test plan summaries.\r\n- Regenerating tables of contents using the [Markdown All in\r\nOne](https://marketplace.visualstudio.com/items?itemName=yzhang.markdown-all-in-one)\r\nVS Code extension.\r\n\r\nNo \"functional\" changes have been made to any test plans, such as\r\nadding, removing, or updating any scenarios.\r\n\r\nThis refactoring prepares the test plans for being \"functionally\"\r\nchanged and improved in follow-up PRs. For example, we're going to cover\r\nthe logic described in https://github.com/elastic/kibana/issues/210358\r\nand address any other gaps in coverage.","sha":"9600de4338edc123dbac7c2cb3403ede2f016752"}},{"branch":"8.x","label":"v8.19.0","branchLabelMappingKey":"^v8.19.0$","isSourceBranch":false,"state":"NOT_CREATED"}]}] BACKPORT--> Co-authored-by: Georgii Gorbachev <[email protected]>
…plans, part 1 (#211300) (#211306) # Backport This will backport the following commits from `main` to `8.18`: - [[Security Solution] Refactor prebuilt rule customization test plans, part 1 (#211300)](#211300) <!--- Backport version: 9.4.3 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sqren/backport) <!--BACKPORT [{"author":{"name":"Georgii Gorbachev","email":"[email protected]"},"sourceCommit":{"committedDate":"2025-02-14T18:22:45Z","message":"[Security Solution] Refactor prebuilt rule customization test plans, part 1 (#211300)\n\n**Epic:** https://github.com/elastic/kibana/issues/174168\r\n**Partially addresses:**\r\nhttps://github.com//issues/202068,\r\nhttps://github.com//issues/202078,\r\nhttps://github.com//issues/202079\r\n\r\n## Summary\r\n\r\nWe're cleaning up and refactoring our existing test plans for prebuilt\r\nrule customization, upgrade, and export/import workflows.\r\n\r\nSpecifically, in this PR we're consistently:\r\n\r\n- Changing the file names and test plan titles.\r\n- Adding or expanding test plan summaries.\r\n- Regenerating tables of contents using the [Markdown All in\r\nOne](https://marketplace.visualstudio.com/items?itemName=yzhang.markdown-all-in-one)\r\nVS Code extension.\r\n\r\nNo \"functional\" changes have been made to any test plans, such as\r\nadding, removing, or updating any scenarios.\r\n\r\nThis refactoring prepares the test plans for being \"functionally\"\r\nchanged and improved in follow-up PRs. For example, we're going to cover\r\nthe logic described in https://github.com/elastic/kibana/issues/210358\r\nand address any other gaps in coverage.","sha":"9600de4338edc123dbac7c2cb3403ede2f016752","branchLabelMapping":{"^v9.1.0$":"main","^v8.19.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","test-plan","v9.0.0","Team:Detections and Resp","Team: SecuritySolution","Team:Detection Rule Management","Feature:Prebuilt Detection Rules","backport:version","v8.18.0","v9.1.0","v8.19.0"],"title":"[Security Solution] Refactor prebuilt rule customization test plans, part 1","number":211300,"url":"https://github.com/elastic/kibana/pull/211300","mergeCommit":{"message":"[Security Solution] Refactor prebuilt rule customization test plans, part 1 (#211300)\n\n**Epic:** https://github.com/elastic/kibana/issues/174168\r\n**Partially addresses:**\r\nhttps://github.com//issues/202068,\r\nhttps://github.com//issues/202078,\r\nhttps://github.com//issues/202079\r\n\r\n## Summary\r\n\r\nWe're cleaning up and refactoring our existing test plans for prebuilt\r\nrule customization, upgrade, and export/import workflows.\r\n\r\nSpecifically, in this PR we're consistently:\r\n\r\n- Changing the file names and test plan titles.\r\n- Adding or expanding test plan summaries.\r\n- Regenerating tables of contents using the [Markdown All in\r\nOne](https://marketplace.visualstudio.com/items?itemName=yzhang.markdown-all-in-one)\r\nVS Code extension.\r\n\r\nNo \"functional\" changes have been made to any test plans, such as\r\nadding, removing, or updating any scenarios.\r\n\r\nThis refactoring prepares the test plans for being \"functionally\"\r\nchanged and improved in follow-up PRs. For example, we're going to cover\r\nthe logic described in https://github.com/elastic/kibana/issues/210358\r\nand address any other gaps in coverage.","sha":"9600de4338edc123dbac7c2cb3403ede2f016752"}},"sourceBranch":"main","suggestedTargetBranches":["9.0","8.18","8.x"],"targetPullRequestStates":[{"branch":"9.0","label":"v9.0.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"8.18","label":"v8.18.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"main","label":"v9.1.0","branchLabelMappingKey":"^v9.1.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/211300","number":211300,"mergeCommit":{"message":"[Security Solution] Refactor prebuilt rule customization test plans, part 1 (#211300)\n\n**Epic:** https://github.com/elastic/kibana/issues/174168\r\n**Partially addresses:**\r\nhttps://github.com//issues/202068,\r\nhttps://github.com//issues/202078,\r\nhttps://github.com//issues/202079\r\n\r\n## Summary\r\n\r\nWe're cleaning up and refactoring our existing test plans for prebuilt\r\nrule customization, upgrade, and export/import workflows.\r\n\r\nSpecifically, in this PR we're consistently:\r\n\r\n- Changing the file names and test plan titles.\r\n- Adding or expanding test plan summaries.\r\n- Regenerating tables of contents using the [Markdown All in\r\nOne](https://marketplace.visualstudio.com/items?itemName=yzhang.markdown-all-in-one)\r\nVS Code extension.\r\n\r\nNo \"functional\" changes have been made to any test plans, such as\r\nadding, removing, or updating any scenarios.\r\n\r\nThis refactoring prepares the test plans for being \"functionally\"\r\nchanged and improved in follow-up PRs. For example, we're going to cover\r\nthe logic described in https://github.com/elastic/kibana/issues/210358\r\nand address any other gaps in coverage.","sha":"9600de4338edc123dbac7c2cb3403ede2f016752"}},{"branch":"8.x","label":"v8.19.0","branchLabelMappingKey":"^v8.19.0$","isSourceBranch":false,"state":"NOT_CREATED"}]}] BACKPORT--> Co-authored-by: Georgii Gorbachev <[email protected]>
…part 2 (#211472) **Epic:** #174168 **Partially addresses:** #202068, #202078, #202079 **Follow-up to:** #211300 ## Summary We're cleaning up and refactoring our existing test plans for prebuilt rule customization, upgrade, and export/import workflows. Specifically, this PR: - Creates an "entrypoint" file that should help navigate all the test plans for prebuilt rules. - Creates a file for keeping common information about prebuilt rules that can be shared between the test plans. - Extracts duplicated terminology to the file with common information. - Extracts duplicated assumptions to the file with common information. - Extracts duplicated non-functional requirements to the file with common information. - Adds user stories to each test plan. - Updates links to tickets in every test plan. No "functional" changes have been made to any test plans, such as adding, removing, or updating any scenarios. This refactoring prepares the test plans for being "functionally" changed and improved in follow-up PRs. For example, we're going to cover the logic described in #210358 and address any other gaps in coverage. ## Review tip It might be easier to review this PR commit-by-commit as each of them contains logically cohesive changes.
…part 2 (elastic#211472) **Epic:** elastic#174168 **Partially addresses:** elastic#202068, elastic#202078, elastic#202079 **Follow-up to:** elastic#211300 ## Summary We're cleaning up and refactoring our existing test plans for prebuilt rule customization, upgrade, and export/import workflows. Specifically, this PR: - Creates an "entrypoint" file that should help navigate all the test plans for prebuilt rules. - Creates a file for keeping common information about prebuilt rules that can be shared between the test plans. - Extracts duplicated terminology to the file with common information. - Extracts duplicated assumptions to the file with common information. - Extracts duplicated non-functional requirements to the file with common information. - Adds user stories to each test plan. - Updates links to tickets in every test plan. No "functional" changes have been made to any test plans, such as adding, removing, or updating any scenarios. This refactoring prepares the test plans for being "functionally" changed and improved in follow-up PRs. For example, we're going to cover the logic described in elastic#210358 and address any other gaps in coverage. ## Review tip It might be easier to review this PR commit-by-commit as each of them contains logically cohesive changes. (cherry picked from commit 786df79)
…part 2 (elastic#211472) **Epic:** elastic#174168 **Partially addresses:** elastic#202068, elastic#202078, elastic#202079 **Follow-up to:** elastic#211300 ## Summary We're cleaning up and refactoring our existing test plans for prebuilt rule customization, upgrade, and export/import workflows. Specifically, this PR: - Creates an "entrypoint" file that should help navigate all the test plans for prebuilt rules. - Creates a file for keeping common information about prebuilt rules that can be shared between the test plans. - Extracts duplicated terminology to the file with common information. - Extracts duplicated assumptions to the file with common information. - Extracts duplicated non-functional requirements to the file with common information. - Adds user stories to each test plan. - Updates links to tickets in every test plan. No "functional" changes have been made to any test plans, such as adding, removing, or updating any scenarios. This refactoring prepares the test plans for being "functionally" changed and improved in follow-up PRs. For example, we're going to cover the logic described in elastic#210358 and address any other gaps in coverage. ## Review tip It might be easier to review this PR commit-by-commit as each of them contains logically cohesive changes. (cherry picked from commit 786df79)
…part 2 (elastic#211472) **Epic:** elastic#174168 **Partially addresses:** elastic#202068, elastic#202078, elastic#202079 **Follow-up to:** elastic#211300 ## Summary We're cleaning up and refactoring our existing test plans for prebuilt rule customization, upgrade, and export/import workflows. Specifically, this PR: - Creates an "entrypoint" file that should help navigate all the test plans for prebuilt rules. - Creates a file for keeping common information about prebuilt rules that can be shared between the test plans. - Extracts duplicated terminology to the file with common information. - Extracts duplicated assumptions to the file with common information. - Extracts duplicated non-functional requirements to the file with common information. - Adds user stories to each test plan. - Updates links to tickets in every test plan. No "functional" changes have been made to any test plans, such as adding, removing, or updating any scenarios. This refactoring prepares the test plans for being "functionally" changed and improved in follow-up PRs. For example, we're going to cover the logic described in elastic#210358 and address any other gaps in coverage. ## Review tip It might be easier to review this PR commit-by-commit as each of them contains logically cohesive changes. (cherry picked from commit 786df79)
…plans, part 2 (#211472) (#211649) # Backport This will backport the following commits from `main` to `8.18`: - [[Security Solution] Refactor prebuilt rule customization test plans, part 2 (#211472)](#211472) <!--- Backport version: 9.6.6 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sorenlouv/backport) <!--BACKPORT [{"author":{"name":"Georgii Gorbachev","email":"[email protected]"},"sourceCommit":{"committedDate":"2025-02-18T20:34:15Z","message":"[Security Solution] Refactor prebuilt rule customization test plans, part 2 (#211472)\n\n**Epic:** https://github.com/elastic/kibana/issues/174168\r\n**Partially addresses:**\r\nhttps://github.com//issues/202068,\r\nhttps://github.com//issues/202078,\r\nhttps://github.com//issues/202079\r\n**Follow-up to:** https://github.com/elastic/kibana/pull/211300\r\n\r\n## Summary\r\n\r\nWe're cleaning up and refactoring our existing test plans for prebuilt\r\nrule customization, upgrade, and export/import workflows.\r\n\r\nSpecifically, this PR:\r\n\r\n- Creates an \"entrypoint\" file that should help navigate all the test\r\nplans for prebuilt rules.\r\n- Creates a file for keeping common information about prebuilt rules\r\nthat can be shared between the test plans.\r\n- Extracts duplicated terminology to the file with common information.\r\n- Extracts duplicated assumptions to the file with common information.\r\n- Extracts duplicated non-functional requirements to the file with\r\ncommon information.\r\n- Adds user stories to each test plan.\r\n- Updates links to tickets in every test plan.\r\n\r\nNo \"functional\" changes have been made to any test plans, such as\r\nadding, removing, or updating any scenarios.\r\n\r\nThis refactoring prepares the test plans for being \"functionally\"\r\nchanged and improved in follow-up PRs. For example, we're going to cover\r\nthe logic described in https://github.com/elastic/kibana/issues/210358\r\nand address any other gaps in coverage.\r\n\r\n## Review tip\r\n\r\nIt might be easier to review this PR commit-by-commit as each of them\r\ncontains logically cohesive changes.","sha":"786df79f37642b8ddf214e086a3375204ec63a3b","branchLabelMapping":{"^v9.1.0$":"main","^v8.19.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","test-plan","v9.0.0","Team:Detections and Resp","Team: SecuritySolution","Team:Detection Rule Management","Feature:Prebuilt Detection Rules","backport:version","v8.18.0","v9.1.0","v8.19.0"],"title":"[Security Solution] Refactor prebuilt rule customization test plans, part 2","number":211472,"url":"https://github.com/elastic/kibana/pull/211472","mergeCommit":{"message":"[Security Solution] Refactor prebuilt rule customization test plans, part 2 (#211472)\n\n**Epic:** https://github.com/elastic/kibana/issues/174168\r\n**Partially addresses:**\r\nhttps://github.com//issues/202068,\r\nhttps://github.com//issues/202078,\r\nhttps://github.com//issues/202079\r\n**Follow-up to:** https://github.com/elastic/kibana/pull/211300\r\n\r\n## Summary\r\n\r\nWe're cleaning up and refactoring our existing test plans for prebuilt\r\nrule customization, upgrade, and export/import workflows.\r\n\r\nSpecifically, this PR:\r\n\r\n- Creates an \"entrypoint\" file that should help navigate all the test\r\nplans for prebuilt rules.\r\n- Creates a file for keeping common information about prebuilt rules\r\nthat can be shared between the test plans.\r\n- Extracts duplicated terminology to the file with common information.\r\n- Extracts duplicated assumptions to the file with common information.\r\n- Extracts duplicated non-functional requirements to the file with\r\ncommon information.\r\n- Adds user stories to each test plan.\r\n- Updates links to tickets in every test plan.\r\n\r\nNo \"functional\" changes have been made to any test plans, such as\r\nadding, removing, or updating any scenarios.\r\n\r\nThis refactoring prepares the test plans for being \"functionally\"\r\nchanged and improved in follow-up PRs. For example, we're going to cover\r\nthe logic described in https://github.com/elastic/kibana/issues/210358\r\nand address any other gaps in coverage.\r\n\r\n## Review tip\r\n\r\nIt might be easier to review this PR commit-by-commit as each of them\r\ncontains logically cohesive changes.","sha":"786df79f37642b8ddf214e086a3375204ec63a3b"}},"sourceBranch":"main","suggestedTargetBranches":["9.0","8.18","8.x"],"targetPullRequestStates":[{"branch":"9.0","label":"v9.0.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"8.18","label":"v8.18.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"main","label":"v9.1.0","branchLabelMappingKey":"^v9.1.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/211472","number":211472,"mergeCommit":{"message":"[Security Solution] Refactor prebuilt rule customization test plans, part 2 (#211472)\n\n**Epic:** https://github.com/elastic/kibana/issues/174168\r\n**Partially addresses:**\r\nhttps://github.com//issues/202068,\r\nhttps://github.com//issues/202078,\r\nhttps://github.com//issues/202079\r\n**Follow-up to:** https://github.com/elastic/kibana/pull/211300\r\n\r\n## Summary\r\n\r\nWe're cleaning up and refactoring our existing test plans for prebuilt\r\nrule customization, upgrade, and export/import workflows.\r\n\r\nSpecifically, this PR:\r\n\r\n- Creates an \"entrypoint\" file that should help navigate all the test\r\nplans for prebuilt rules.\r\n- Creates a file for keeping common information about prebuilt rules\r\nthat can be shared between the test plans.\r\n- Extracts duplicated terminology to the file with common information.\r\n- Extracts duplicated assumptions to the file with common information.\r\n- Extracts duplicated non-functional requirements to the file with\r\ncommon information.\r\n- Adds user stories to each test plan.\r\n- Updates links to tickets in every test plan.\r\n\r\nNo \"functional\" changes have been made to any test plans, such as\r\nadding, removing, or updating any scenarios.\r\n\r\nThis refactoring prepares the test plans for being \"functionally\"\r\nchanged and improved in follow-up PRs. For example, we're going to cover\r\nthe logic described in https://github.com/elastic/kibana/issues/210358\r\nand address any other gaps in coverage.\r\n\r\n## Review tip\r\n\r\nIt might be easier to review this PR commit-by-commit as each of them\r\ncontains logically cohesive changes.","sha":"786df79f37642b8ddf214e086a3375204ec63a3b"}},{"branch":"8.x","label":"v8.19.0","branchLabelMappingKey":"^v8.19.0$","isSourceBranch":false,"state":"NOT_CREATED"}]}] BACKPORT--> Co-authored-by: Georgii Gorbachev <[email protected]>
…lans, part 2 (#211472) (#211651) # Backport This will backport the following commits from `main` to `9.0`: - [[Security Solution] Refactor prebuilt rule customization test plans, part 2 (#211472)](#211472) <!--- Backport version: 9.6.6 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sorenlouv/backport) <!--BACKPORT [{"author":{"name":"Georgii Gorbachev","email":"[email protected]"},"sourceCommit":{"committedDate":"2025-02-18T20:34:15Z","message":"[Security Solution] Refactor prebuilt rule customization test plans, part 2 (#211472)\n\n**Epic:** https://github.com/elastic/kibana/issues/174168\r\n**Partially addresses:**\r\nhttps://github.com//issues/202068,\r\nhttps://github.com//issues/202078,\r\nhttps://github.com//issues/202079\r\n**Follow-up to:** https://github.com/elastic/kibana/pull/211300\r\n\r\n## Summary\r\n\r\nWe're cleaning up and refactoring our existing test plans for prebuilt\r\nrule customization, upgrade, and export/import workflows.\r\n\r\nSpecifically, this PR:\r\n\r\n- Creates an \"entrypoint\" file that should help navigate all the test\r\nplans for prebuilt rules.\r\n- Creates a file for keeping common information about prebuilt rules\r\nthat can be shared between the test plans.\r\n- Extracts duplicated terminology to the file with common information.\r\n- Extracts duplicated assumptions to the file with common information.\r\n- Extracts duplicated non-functional requirements to the file with\r\ncommon information.\r\n- Adds user stories to each test plan.\r\n- Updates links to tickets in every test plan.\r\n\r\nNo \"functional\" changes have been made to any test plans, such as\r\nadding, removing, or updating any scenarios.\r\n\r\nThis refactoring prepares the test plans for being \"functionally\"\r\nchanged and improved in follow-up PRs. For example, we're going to cover\r\nthe logic described in https://github.com/elastic/kibana/issues/210358\r\nand address any other gaps in coverage.\r\n\r\n## Review tip\r\n\r\nIt might be easier to review this PR commit-by-commit as each of them\r\ncontains logically cohesive changes.","sha":"786df79f37642b8ddf214e086a3375204ec63a3b","branchLabelMapping":{"^v9.1.0$":"main","^v8.19.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","test-plan","v9.0.0","Team:Detections and Resp","Team: SecuritySolution","Team:Detection Rule Management","Feature:Prebuilt Detection Rules","backport:version","v8.18.0","v9.1.0","v8.19.0"],"title":"[Security Solution] Refactor prebuilt rule customization test plans, part 2","number":211472,"url":"https://github.com/elastic/kibana/pull/211472","mergeCommit":{"message":"[Security Solution] Refactor prebuilt rule customization test plans, part 2 (#211472)\n\n**Epic:** https://github.com/elastic/kibana/issues/174168\r\n**Partially addresses:**\r\nhttps://github.com//issues/202068,\r\nhttps://github.com//issues/202078,\r\nhttps://github.com//issues/202079\r\n**Follow-up to:** https://github.com/elastic/kibana/pull/211300\r\n\r\n## Summary\r\n\r\nWe're cleaning up and refactoring our existing test plans for prebuilt\r\nrule customization, upgrade, and export/import workflows.\r\n\r\nSpecifically, this PR:\r\n\r\n- Creates an \"entrypoint\" file that should help navigate all the test\r\nplans for prebuilt rules.\r\n- Creates a file for keeping common information about prebuilt rules\r\nthat can be shared between the test plans.\r\n- Extracts duplicated terminology to the file with common information.\r\n- Extracts duplicated assumptions to the file with common information.\r\n- Extracts duplicated non-functional requirements to the file with\r\ncommon information.\r\n- Adds user stories to each test plan.\r\n- Updates links to tickets in every test plan.\r\n\r\nNo \"functional\" changes have been made to any test plans, such as\r\nadding, removing, or updating any scenarios.\r\n\r\nThis refactoring prepares the test plans for being \"functionally\"\r\nchanged and improved in follow-up PRs. For example, we're going to cover\r\nthe logic described in https://github.com/elastic/kibana/issues/210358\r\nand address any other gaps in coverage.\r\n\r\n## Review tip\r\n\r\nIt might be easier to review this PR commit-by-commit as each of them\r\ncontains logically cohesive changes.","sha":"786df79f37642b8ddf214e086a3375204ec63a3b"}},"sourceBranch":"main","suggestedTargetBranches":["9.0","8.18","8.x"],"targetPullRequestStates":[{"branch":"9.0","label":"v9.0.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"8.18","label":"v8.18.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"main","label":"v9.1.0","branchLabelMappingKey":"^v9.1.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/211472","number":211472,"mergeCommit":{"message":"[Security Solution] Refactor prebuilt rule customization test plans, part 2 (#211472)\n\n**Epic:** https://github.com/elastic/kibana/issues/174168\r\n**Partially addresses:**\r\nhttps://github.com//issues/202068,\r\nhttps://github.com//issues/202078,\r\nhttps://github.com//issues/202079\r\n**Follow-up to:** https://github.com/elastic/kibana/pull/211300\r\n\r\n## Summary\r\n\r\nWe're cleaning up and refactoring our existing test plans for prebuilt\r\nrule customization, upgrade, and export/import workflows.\r\n\r\nSpecifically, this PR:\r\n\r\n- Creates an \"entrypoint\" file that should help navigate all the test\r\nplans for prebuilt rules.\r\n- Creates a file for keeping common information about prebuilt rules\r\nthat can be shared between the test plans.\r\n- Extracts duplicated terminology to the file with common information.\r\n- Extracts duplicated assumptions to the file with common information.\r\n- Extracts duplicated non-functional requirements to the file with\r\ncommon information.\r\n- Adds user stories to each test plan.\r\n- Updates links to tickets in every test plan.\r\n\r\nNo \"functional\" changes have been made to any test plans, such as\r\nadding, removing, or updating any scenarios.\r\n\r\nThis refactoring prepares the test plans for being \"functionally\"\r\nchanged and improved in follow-up PRs. For example, we're going to cover\r\nthe logic described in https://github.com/elastic/kibana/issues/210358\r\nand address any other gaps in coverage.\r\n\r\n## Review tip\r\n\r\nIt might be easier to review this PR commit-by-commit as each of them\r\ncontains logically cohesive changes.","sha":"786df79f37642b8ddf214e086a3375204ec63a3b"}},{"branch":"8.x","label":"v8.19.0","branchLabelMappingKey":"^v8.19.0$","isSourceBranch":false,"state":"NOT_CREATED"}]}] BACKPORT--> Co-authored-by: Georgii Gorbachev <[email protected]>
…lans, part 2 (#211472) (#211650) # Backport This will backport the following commits from `main` to `8.x`: - [[Security Solution] Refactor prebuilt rule customization test plans, part 2 (#211472)](#211472) <!--- Backport version: 9.6.6 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sorenlouv/backport) <!--BACKPORT [{"author":{"name":"Georgii Gorbachev","email":"[email protected]"},"sourceCommit":{"committedDate":"2025-02-18T20:34:15Z","message":"[Security Solution] Refactor prebuilt rule customization test plans, part 2 (#211472)\n\n**Epic:** https://github.com/elastic/kibana/issues/174168\r\n**Partially addresses:**\r\nhttps://github.com//issues/202068,\r\nhttps://github.com//issues/202078,\r\nhttps://github.com//issues/202079\r\n**Follow-up to:** https://github.com/elastic/kibana/pull/211300\r\n\r\n## Summary\r\n\r\nWe're cleaning up and refactoring our existing test plans for prebuilt\r\nrule customization, upgrade, and export/import workflows.\r\n\r\nSpecifically, this PR:\r\n\r\n- Creates an \"entrypoint\" file that should help navigate all the test\r\nplans for prebuilt rules.\r\n- Creates a file for keeping common information about prebuilt rules\r\nthat can be shared between the test plans.\r\n- Extracts duplicated terminology to the file with common information.\r\n- Extracts duplicated assumptions to the file with common information.\r\n- Extracts duplicated non-functional requirements to the file with\r\ncommon information.\r\n- Adds user stories to each test plan.\r\n- Updates links to tickets in every test plan.\r\n\r\nNo \"functional\" changes have been made to any test plans, such as\r\nadding, removing, or updating any scenarios.\r\n\r\nThis refactoring prepares the test plans for being \"functionally\"\r\nchanged and improved in follow-up PRs. For example, we're going to cover\r\nthe logic described in https://github.com/elastic/kibana/issues/210358\r\nand address any other gaps in coverage.\r\n\r\n## Review tip\r\n\r\nIt might be easier to review this PR commit-by-commit as each of them\r\ncontains logically cohesive changes.","sha":"786df79f37642b8ddf214e086a3375204ec63a3b","branchLabelMapping":{"^v9.1.0$":"main","^v8.19.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","test-plan","v9.0.0","Team:Detections and Resp","Team: SecuritySolution","Team:Detection Rule Management","Feature:Prebuilt Detection Rules","backport:version","v8.18.0","v9.1.0","v8.19.0"],"title":"[Security Solution] Refactor prebuilt rule customization test plans, part 2","number":211472,"url":"https://github.com/elastic/kibana/pull/211472","mergeCommit":{"message":"[Security Solution] Refactor prebuilt rule customization test plans, part 2 (#211472)\n\n**Epic:** https://github.com/elastic/kibana/issues/174168\r\n**Partially addresses:**\r\nhttps://github.com//issues/202068,\r\nhttps://github.com//issues/202078,\r\nhttps://github.com//issues/202079\r\n**Follow-up to:** https://github.com/elastic/kibana/pull/211300\r\n\r\n## Summary\r\n\r\nWe're cleaning up and refactoring our existing test plans for prebuilt\r\nrule customization, upgrade, and export/import workflows.\r\n\r\nSpecifically, this PR:\r\n\r\n- Creates an \"entrypoint\" file that should help navigate all the test\r\nplans for prebuilt rules.\r\n- Creates a file for keeping common information about prebuilt rules\r\nthat can be shared between the test plans.\r\n- Extracts duplicated terminology to the file with common information.\r\n- Extracts duplicated assumptions to the file with common information.\r\n- Extracts duplicated non-functional requirements to the file with\r\ncommon information.\r\n- Adds user stories to each test plan.\r\n- Updates links to tickets in every test plan.\r\n\r\nNo \"functional\" changes have been made to any test plans, such as\r\nadding, removing, or updating any scenarios.\r\n\r\nThis refactoring prepares the test plans for being \"functionally\"\r\nchanged and improved in follow-up PRs. For example, we're going to cover\r\nthe logic described in https://github.com/elastic/kibana/issues/210358\r\nand address any other gaps in coverage.\r\n\r\n## Review tip\r\n\r\nIt might be easier to review this PR commit-by-commit as each of them\r\ncontains logically cohesive changes.","sha":"786df79f37642b8ddf214e086a3375204ec63a3b"}},"sourceBranch":"main","suggestedTargetBranches":["9.0","8.18","8.x"],"targetPullRequestStates":[{"branch":"9.0","label":"v9.0.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"8.18","label":"v8.18.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"main","label":"v9.1.0","branchLabelMappingKey":"^v9.1.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/211472","number":211472,"mergeCommit":{"message":"[Security Solution] Refactor prebuilt rule customization test plans, part 2 (#211472)\n\n**Epic:** https://github.com/elastic/kibana/issues/174168\r\n**Partially addresses:**\r\nhttps://github.com//issues/202068,\r\nhttps://github.com//issues/202078,\r\nhttps://github.com//issues/202079\r\n**Follow-up to:** https://github.com/elastic/kibana/pull/211300\r\n\r\n## Summary\r\n\r\nWe're cleaning up and refactoring our existing test plans for prebuilt\r\nrule customization, upgrade, and export/import workflows.\r\n\r\nSpecifically, this PR:\r\n\r\n- Creates an \"entrypoint\" file that should help navigate all the test\r\nplans for prebuilt rules.\r\n- Creates a file for keeping common information about prebuilt rules\r\nthat can be shared between the test plans.\r\n- Extracts duplicated terminology to the file with common information.\r\n- Extracts duplicated assumptions to the file with common information.\r\n- Extracts duplicated non-functional requirements to the file with\r\ncommon information.\r\n- Adds user stories to each test plan.\r\n- Updates links to tickets in every test plan.\r\n\r\nNo \"functional\" changes have been made to any test plans, such as\r\nadding, removing, or updating any scenarios.\r\n\r\nThis refactoring prepares the test plans for being \"functionally\"\r\nchanged and improved in follow-up PRs. For example, we're going to cover\r\nthe logic described in https://github.com/elastic/kibana/issues/210358\r\nand address any other gaps in coverage.\r\n\r\n## Review tip\r\n\r\nIt might be easier to review this PR commit-by-commit as each of them\r\ncontains logically cohesive changes.","sha":"786df79f37642b8ddf214e086a3375204ec63a3b"}},{"branch":"8.x","label":"v8.19.0","branchLabelMappingKey":"^v8.19.0$","isSourceBranch":false,"state":"NOT_CREATED"}]}] BACKPORT--> Co-authored-by: Georgii Gorbachev <[email protected]>
Epics: https://github.com/elastic/security-team/issues/1974 (internal), #174168
Summary
Write test plans (one or a few) and create a comprehensive suite of automated tests for the workflow of customizing prebuilt rules.
Features to cover:
is_customized
field and theModified
badge in the UIModified
in the Rule Updates tableis_customized
- per each rule field - in API and UIPlease cover both the features under the feature flag turned ON and OFF.
Related tickets
Related functional tickets to cover with tests:
ruleSource.isCustomized
in API endpoint handlers #180145ruleSource.isCustomized
in bulk edit API #187706Related bugs to cover with tests:
Related PRs
Test plans for diff algorithms
data_source
field diff algorithm test plan #189669query
fields diff algorithm #192529type
field diff algorithm #193372Test plans for prebuilt rule customization workflow
Test coverage for prebuilt rule customization workflow
The text was updated successfully, but these errors were encountered: