Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add table for cost by region monthly #2310

Merged

Conversation

razbne
Copy link

@razbne razbne commented Sep 23, 2024

Integration test logs

Logs
N/A

Example query results

Results
with ranked_costs as (
  select
    region,
    period_start,
    unblended_cost_amount :: numeric :: money,
    rank() over(
      partition by region
      order by
        unblended_cost_amount desc
    )
  from
    aws_cost_by_region_monthly
)
select
  *
from
  ranked_costs
where
  rank = 1;
+----------------+---------------------------+-----------------------+------+
| region         | period_start              | unblended_cost_amount | rank |
+----------------+---------------------------+-----------------------+------+
| NoRegion       | 2024-09-01T03:00:00+03:00 | $0.00                 | 1    |
| NoRegion       | 2024-08-01T03:00:00+03:00 | $0.00                 | 1    |
| ap-northeast-1 | 2024-09-01T03:00:00+03:00 | $0.00                 | 1    |
| ap-northeast-2 | 2024-09-01T03:00:00+03:00 | $0.00                 | 1    |
| ap-northeast-3 | 2024-09-01T03:00:00+03:00 | $0.00                 | 1    |
| ap-south-1     | 2024-09-01T03:00:00+03:00 | $0.00                 | 1    |
| ap-southeast-1 | 2024-09-01T03:00:00+03:00 | $0.00                 | 1    |
| ap-southeast-2 | 2024-09-01T03:00:00+03:00 | $0.00                 | 1    |
| ca-central-1   | 2024-09-01T03:00:00+03:00 | $0.00                 | 1    |
| eu-central-1   | 2024-09-01T03:00:00+03:00 | $0.00                 | 1    |
| eu-north-1     | 2024-09-01T03:00:00+03:00 | $0.00                 | 1    |
| eu-west-1      | 2024-09-01T03:00:00+03:00 | $0.00                 | 1    |
| eu-west-2      | 2024-09-01T03:00:00+03:00 | $0.00                 | 1    |
| eu-west-3      | 2024-09-01T03:00:00+03:00 | $0.00                 | 1    |
| global         | 2024-09-01T03:00:00+03:00 | $0.00                 | 1    |
| sa-east-1      | 2024-09-01T03:00:00+03:00 | $0.00                 | 1    |
| us-east-1      | 2024-09-01T03:00:00+03:00 | $0.00                 | 1    |
| us-east-2      | 2024-09-01T03:00:00+03:00 | $0.00                 | 1    |
| us-west-1      | 2024-09-01T03:00:00+03:00 | $0.00                 | 1    |
| us-west-2      | 2024-09-01T03:00:00+03:00 | $0.00                 | 1    |
| <null>         | 2023-10-01T03:00:00+03:00 | $0.00                 | 1    |
| <null>         | 2024-04-01T03:00:00+03:00 | $0.00                 | 1    |
| <null>         | 2024-07-01T03:00:00+03:00 | $0.00                 | 1    |
| <null>         | 2024-02-01T02:00:00+02:00 | $0.00                 | 1    |
| <null>         | 2023-09-23T03:00:00+03:00 | $0.00                 | 1    |
| <null>         | 2024-06-01T03:00:00+03:00 | $0.00                 | 1    |
| <null>         | 2024-05-01T03:00:00+03:00 | $0.00                 | 1    |
| <null>         | 2024-03-01T02:00:00+02:00 | $0.00                 | 1    |
| <null>         | 2023-11-01T02:00:00+02:00 | $0.00                 | 1    |
| <null>         | 2023-12-01T02:00:00+02:00 | $0.00                 | 1    |
| <null>         | 2024-01-01T02:00:00+02:00 | $0.00                 | 1    |
+----------------+---------------------------+-----------------------+------+

Copy link
Contributor

@ParthaI ParthaI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hello @razbne, left a few review comments, please take look, thanks!

{Name: "region", Operators: []string{"=", "<>"}, Require: plugin.Optional},
},
},
Columns: costExplorerColumns([]*plugin.Column{
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @razbne,

Could you please add the missing common columns to this table? These columns are shared across tables.

  • For an aggregator connection, the partition and account_id columns are important to identify the account associated with the displayed result.
  • I understand that we can't use the awsGlobalRegionColumns() function here since the region column will always show as global.
  • However, we can manually include it as follows:
Columns: costExplorerColumns([]*plugin.Column{
			{
				Name:        "region",
				Description: "The name of the AWS region.",
				Type:        proto.ColumnType_STRING,
				Transform:   transform.FromField("Dimension1"),
			},

			// Steampipe common columns
			{
				Name:        "partition",
				Type:        proto.ColumnType_STRING,
				Hydrate:     getCommonColumns,
				Description: "The AWS partition in which the resource is located (aws, aws-cn, or aws-us-gov).",
			},
			{
				Name:        "account_id",
				Type:        proto.ColumnType_STRING,
				Hydrate:     getCommonColumns,
				Description: "The AWS Account ID in which the resource is located.",
				Transform:   transform.FromCamel(),
			},
		}),

Comment on lines +72 to +97
for _, keyQual := range d.Table.List.KeyColumns {
filterQual := d.Quals[keyQual.Name]
if filterQual == nil {
continue
}
for _, qual := range filterQual.Quals {
if qual.Value != nil {
value := qual.Value
switch qual.Operator {
case "=":
filter := types.Expression{}
filter.Dimensions = &types.DimensionValues{}
filter.Dimensions.Key = types.Dimension(strings.ToUpper(keyQual.Name))
filter.Dimensions.Values = []string{value.GetStringValue()}
filters = append(filters, filter)
case "<>":
filter := types.Expression{}
filter.Not = &types.Expression{}
filter.Not.Dimensions = &types.DimensionValues{}
filter.Not.Dimensions.Key = types.Dimension(strings.ToUpper(keyQual.Name))
filter.Not.Dimensions.Values = []string{value.GetStringValue()}
filters = append(filters, filter)
}
}
}
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please update the local variable name to lowerCamelCase for consistency across the tables.

Suggested change
for _, keyQual := range d.Table.List.KeyColumns {
filterQual := d.Quals[keyQual.Name]
if filterQual == nil {
continue
}
for _, qual := range filterQual.Quals {
if qual.Value != nil {
value := qual.Value
switch qual.Operator {
case "=":
filter := types.Expression{}
filter.Dimensions = &types.DimensionValues{}
filter.Dimensions.Key = types.Dimension(strings.ToUpper(keyQual.Name))
filter.Dimensions.Values = []string{value.GetStringValue()}
filters = append(filters, filter)
case "<>":
filter := types.Expression{}
filter.Not = &types.Expression{}
filter.Not.Dimensions = &types.DimensionValues{}
filter.Not.Dimensions.Key = types.Dimension(strings.ToUpper(keyQual.Name))
filter.Not.Dimensions.Values = []string{value.GetStringValue()}
filters = append(filters, filter)
}
}
}
}
for _, keyQual := range d.Table.List.KeyColumns {
filterQual := d.Quals[keyQual.Name]
if filterQual == nil {
continue
}
for _, qual := range filterQual.Quals {
if qual.Value != nil {
value := qual.Value
switch qual.Operator {
case "=":
filter := types.Expression{}
filter.Dimensions = &types.DimensionValues{}
filter.Dimensions.Key = types.Dimension(strings.ToUpper(keyQual.Name))
filter.Dimensions.Values = []string{value.GetStringValue()}
filters = append(filters, filter)
case "<>":
filter := types.Expression{}
filter.Not = &types.Expression{}
filter.Not.Dimensions = &types.DimensionValues{}
filter.Not.Dimensions.Key = types.Dimension(strings.ToUpper(keyQual.Name))
filter.Not.Dimensions.Values = []string{value.GetStringValue()}
filters = append(filters, filter)
}
}
}
}

limit 10;
```


Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change

limit 10;
```


Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change

@misraved
Copy link
Contributor

Hi @razbne, did you get a chance to work on the review comments suggested by @ParthaI?

Please feel free to reach out if you have questions 👍!

Happy to help 😄!!

@misraved misraved changed the base branch from main to add_cost_by_region_table October 29, 2024 13:39
@misraved misraved merged commit 61204d0 into turbot:add_cost_by_region_table Oct 29, 2024
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants