-
Notifications
You must be signed in to change notification settings - Fork 10
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add IBM Cloud Databases list and backups list fetchers #44
Conversation
61106ac
to
c512e07
Compare
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pausing...
It looks like you may be adding functionality, so it's not a straight up migration of these fetchers. If that's what you're doing then your README write up needs to be accurate as to this new behavior for these fetchers. Can you clean up the README and then we can resume review?
Thanks.
compliance reports and notifications using the [auditree-framework][]. They | ||
validate the configuration and ensure smooth execution of an auditree instance. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Remove this sentence: They validate the configuration and ensure smooth execution of an auditree instance.
@@ -0,0 +1,120 @@ | |||
# -*- coding:utf-8; mode:python -*- |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You're missing the copyright header here.
CHANGES.md
Outdated
# 0.8.0 (2021-11-01) | ||
|
||
- [ADDED] IBM Cloud Databases list and backups list fetchers added. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
# 0.8.0 (2021-11-01) | |
- [ADDED] IBM Cloud Databases list and backups list fetchers added. | |
# [0.8.0](https://github.com/ComplianceAsCode/auditree-arboretum/releases/tag/v0.8.0) | |
- [ADDED] IBM Cloud Databases list fetcher added. | |
- [ADDED] IBM Cloud Databases backups fetcher added. | |
- [ADDED] Folder hierarchy for IBM Cloud Databases fetchers, checks, and harvest reports added. |
|
||
* Class: [DatabasesFetcher][fetch-databases] | ||
* Purpose: Store list of account-specific IBM Cloud databases to the evidence locker. | ||
* Behavior: Access [IBM Cloud API][ibm-cloud-api] details of the databases as a JSON. TTL for evidence is set to 1 day. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
* Behavior: Access [IBM Cloud API][ibm-cloud-api] details of the databases as a JSON. TTL for evidence is set to 1 day. | |
* Behavior: Retrieve details about account specific IBM Cloud Databases. TTL for evidence is set to 1 day. |
* Configuration elements: | ||
* `org.icd.list.accounts` | ||
* Required | ||
* List of objects representing the IBL Cloud accounts |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
* List of objects representing the IBL Cloud accounts | |
* List of dictionaries representing the IBM Cloud accounts |
* `resource_group_id` - the resource group associated with the above account name, output of [running the ibmcloud resource groups command][ic-resource-groups] while logged in to an IBM Cloud account. | ||
``` | ||
$ ibmcloud resource groups | ||
Retrieving all resource groups under account fedcba1234567890abcdef0123456789 as dummy.user@ibm.com... | ||
OK | ||
Name ID Default Group State | ||
default bcdef0123456789fedcba1234567890a true ACTIVE | ||
``` |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'd remove the example and simplify the bullet content. You already provide the link to the command.
* `resource_group_id` - the resource group associated with the above account name, output of [running the ibmcloud resource groups command][ic-resource-groups] while logged in to an IBM Cloud account. | |
``` | |
$ ibmcloud resource groups | |
Retrieving all resource groups under account fedcba1234567890abcdef0123456789 as dummy.user@ibm.com... | |
OK | |
Name ID Default Group State | |
default bcdef0123456789fedcba1234567890a true ACTIVE | |
``` | |
* `resource_group_id` - the resource group associated with the account. | |
* To obtain the resource group execute the [IBM Cloud resource groups command][ic-resource-groups]. |
* Required | ||
* List of objects representing the IBL Cloud accounts | ||
* Each object must have the following fields | ||
* `account_name` - an arbitrary name identifying the IBM Cloud account, used to match to the token provided in the |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
* `account_name` - an arbitrary name identifying the IBM Cloud account, used to match to the token provided in the | |
* `account_name` - an arbitrary name as a string identifying the IBM Cloud account, used to match to the token provided in the |
* Each object must have the following fields | ||
* `account_name` - an arbitrary name identifying the IBM Cloud account, used to match to the token provided in the | ||
credentials file in order for the fetcher to retrieve content from IBM Cloud for that account. | ||
* `resource_group_id` - the resource group associated with the above account name, output of [running the ibmcloud resource groups command][ic-resource-groups] while logged in to an IBM Cloud account. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
* `resource_group_id` - the resource group associated with the above account name, output of [running the ibmcloud resource groups command][ic-resource-groups] while logged in to an IBM Cloud account. | |
* `resource_group_id` - the resource group as a string associated with the above account name, output of [running the ibmcloud resource groups command][ic-resource-groups] while logged in to an IBM Cloud account. |
"resource_group_id": [ | ||
"bcdef0123456789fedcba1234567890a", | ||
... | ||
] |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This isn't valid JSON and renders as such in GH. This example also has resource_group_id as a list of strings whereas the description of it is that it should be a string. Assuming string...
"resource_group_id": [ | |
"bcdef0123456789fedcba1234567890a", | |
... | |
] | |
"resource_group_id": "bcdef0123456789fedcba1234567890a" |
The legacy config format where each account has a single resource_group_id is also supported | ||
|
||
```json | ||
{ | ||
"org": { | ||
"icd": { | ||
"list": { | ||
"accounts": [ | ||
{ | ||
"account_name": "prod_1234567", | ||
"resource_group_id": "bcdef0123456789fedcba1234567890a" | ||
} | ||
] | ||
} | ||
} | ||
} | ||
} | ||
``` |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
OK now it seems like you're introducing new functionality here? If you're looking to support this new functionality and deprecate the old one where you have a single resource group as a string that's fine but you don't need to include the "backward compatibility" version in the docs.
c512e07
to
8d893ed
Compare
Local test results added to the top - it raises the question of whether 404 errors should abort creation of the backups evidence. |
What's the question exactly? |
When fetch_backups.py iterates through a list of databases, if it gets a 404 from any of the backup URL's |
Closing due to inactivity |
What
Migrate IBM Cloud Databases fetcher for account specific list of databases and backups list.
Why
Audits often request evidence to check whether databases are regularly being backed up.
How
REST api calls will be made to the resource controller and IBM Cloud databases end points.
Test
Local testing to be done to verify evidence being created.
fetch_databases.py
The evidence file was generated and its contents verified
fetch_backups.py
NOTE: For the fetch_backups test to run to completion, certain entries in the database evidence that were getting 404's had to be manually removed.
Context
#43