Skip to content

Commit

Permalink
feat: add gcp_organization_project table
Browse files Browse the repository at this point in the history
  • Loading branch information
pdecat committed Oct 5, 2024
1 parent 5bbdb77 commit 0a3d182
Show file tree
Hide file tree
Showing 6 changed files with 214 additions and 16 deletions.
6 changes: 3 additions & 3 deletions docs/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -56,9 +56,9 @@ steampipe plugin install gcp

| Item | Description |
| ----------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| Credentials | When running locally, you must configure your [Application Default Credentials](https://cloud.google.com/sdk/gcloud/reference/auth/application-default). If you are running in Cloud Shell or Cloud Code, [the tool uses the credentials you provided when you logged in, and manages any authorizations required](https://cloud.google.com/docs/authentication/provide-credentials-adc#cloud-based-dev). |
| Permissions | Assign the `Viewer` role to your user or service account. You may also need additional permissions related to IAM policies, like `pubsub.subscriptions.getIamPolicy`, `pubsub.topics.getIamPolicy`, `storage.buckets.getIamPolicy`, since these are not included in the `Viewer` role. You can grant these by creating a custom role in your project. |
| Radius | Each connection represents a single GCP project. |
| Credentials | When running locally, you must configure your [Application Default Credentials](https://cloud.google.com/sdk/gcloud/reference/auth/application-default). If you are running in Cloud Shell or Cloud Code, [the tool uses the credentials you provided when you logged in, and manages any authorizations required](https://cloud.google.com/docs/authentication/provide-credentials-adc#cloud-based-dev). |
| Permissions | Assign the `Viewer` role to your user or service account. You may also need additional permissions related to IAM policies, like `pubsub.subscriptions.getIamPolicy`, `pubsub.topics.getIamPolicy`, `storage.buckets.getIamPolicy`, since these are not included in the `Viewer` role. You can grant these by creating a custom role in your project. |
| Radius | Each connection represents a single GCP project, except for some tables like `gcp_organization` and `gcp_organization_project` which return all resources the credentials attached to the connection have access to. |
| Resolution | 1. Credentials from the JSON file specified by the `credentials` parameter in your steampipe config.<br />2. Credentials from the JSON file specified by the `GOOGLE_APPLICATION_CREDENTIALS` environment variable.<br />3. Credentials from the default JSON file location (~/.config/gcloud/application_default_credentials.json). <br />4. Credentials from [the metadata server](https://cloud.google.com/docs/authentication/application-default-credentials#attached-sa) |

### Configuration
Expand Down
83 changes: 83 additions & 0 deletions docs/tables/gcp_organization_project.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,83 @@
---
title: "Steampipe Table: gcp_organization_project - Query Google Cloud Platform Projects using SQL"
description: "Allows users to query Projects in Google Cloud Platform, specifically providing details about the project's ID, name, labels, and lifecycle state."
---

# Table: gcp_organization_project - Query Google Cloud Platform Projects using SQL

**Note: this table is a variant of the `gcp_project` table which does not filter on the GCP project attached to connection, and thus, will return all projects that the credentials used by the connection have access to. Using this table in aggregator connections can produce unexpected duplicate results.**

A Google Cloud Platform Project acts as an organizational unit within GCP where resources are allocated. It is used to group resources that belong to the same logical application or business unit. Each project is linked to a billing account and can have users, roles, and permissions assigned to it.

## Table Usage Guide

The `gcp_organization_project` table provides insights into Projects within Google Cloud Platform. As a DevOps engineer, explore project-specific details through this table, including ID, name, labels, and lifecycle state. Utilize it to uncover information about projects, such as their associated resources, user roles, permissions, and billing details.

## Examples

### Basic info
Explore which Google Cloud Platform projects are active, by looking at their lifecycle state and creation time. This can help you manage resources effectively and keep track of ongoing projects.

```sql+postgres
select
name,
project_id,
project_number,
lifecycle_state,
create_time
from
gcp_organization_project;
```

```sql+sqlite
select
name,
project_id,
project_number,
lifecycle_state,
create_time
from
gcp_organization_project;
```

### Get access approval settings for all projects
Explore the access approval settings across your various projects. This can help you understand and manage permissions and approvals more effectively.

```sql+postgres
select
name,
jsonb_pretty(access_approval_settings) as access_approval_settings
from
gcp_organization_project;
```

```sql+sqlite
select
name,
access_approval_settings
from
gcp_organization_project;
```

### Get parent and organization ID for all projects
Get the organization ID across your various projects.

```sql+postgres
select
project_id,
parent ->> 'id' as parent_id,
parent ->> 'type' as parent_type,
case when jsonb_array_length(ancestors) > 1 then ancestors -> -1 -> 'resourceId' ->> 'id' else null end as organization_id
from
gcp_project;
```

```sql+sqlite
select
project_id,
parent ->> 'id' as parent_id,
parent ->> 'type' as parent_type,
case when json_array_length(ancestors) > 1 then ancestors -> -1 -> 'resourceId' ->> 'id' else null end as organization_id
from
gcp_project;
```
2 changes: 2 additions & 0 deletions docs/tables/gcp_project.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,8 @@ description: "Allows users to query Projects in Google Cloud Platform, specifica

# Table: gcp_project - Query Google Cloud Platform Projects using SQL

**Note: this table is a variant of the `gcp_organization_project` table which filters on the GCP project attached to connection, and thus, will only ever return details about that specific project.**

A Google Cloud Platform Project acts as an organizational unit within GCP where resources are allocated. It is used to group resources that belong to the same logical application or business unit. Each project is linked to a billing account and can have users, roles, and permissions assigned to it.

## Table Usage Guide
Expand Down
1 change: 1 addition & 0 deletions gcp/plugin.go
Original file line number Diff line number Diff line change
Expand Up @@ -122,6 +122,7 @@ func Plugin(ctx context.Context) *plugin.Plugin {
"gcp_monitoring_group": tableGcpMonitoringGroup(ctx),
"gcp_monitoring_notification_channel": tableGcpMonitoringNotificationChannel(ctx),
"gcp_organization": tableGcpOrganization(ctx),
"gcp_organization_project": tableGcpOrganizationProject(ctx),
"gcp_project": tableGcpProject(ctx),
"gcp_project_organization_policy": tableGcpProjectOrganizationPolicy(ctx),
"gcp_project_service": tableGcpProjectService(ctx),
Expand Down
121 changes: 121 additions & 0 deletions gcp/table_gcp_organization_project.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,121 @@
package gcp

import (
"context"

"github.com/turbot/steampipe-plugin-sdk/v5/grpc/proto"
"github.com/turbot/steampipe-plugin-sdk/v5/plugin"
"github.com/turbot/steampipe-plugin-sdk/v5/plugin/transform"
)

//// TABLE DEFINITION

func tableGcpOrganizationProject(_ context.Context) *plugin.Table {
return &plugin.Table{
Name: "gcp_organization_project",
Description: "GCP Organization Project",
List: &plugin.ListConfig{
Hydrate: listGCPOrganizationProjects,
},
Columns: []*plugin.Column{
{
Name: "name",
Description: "The name of the project.",
Type: proto.ColumnType_STRING,
},
{
Name: "project_id",
Description: "An unique, user-assigned ID of the Project.",
Type: proto.ColumnType_STRING,
},
{
Name: "self_link",
Description: "Server-defined URL for the resource.",
Type: proto.ColumnType_STRING,
Transform: transform.From(projectSelfLink),
},
{
Name: "project_number",
Description: "The number uniquely identifying the project.",
Type: proto.ColumnType_INT,
},
{
Name: "lifecycle_state",
Description: "Specifies the project lifecycle state.",
Type: proto.ColumnType_STRING,
},
{
Name: "create_time",
Description: "Creation time of the project.",
Type: proto.ColumnType_TIMESTAMP,
},
{
Name: "parent",
Description: "An optional reference to a parent Resource.",
Type: proto.ColumnType_JSON,
},
{
Name: "labels",
Description: "A list of labels attached to this project.",
Type: proto.ColumnType_JSON,
},
{
Name: "access_approval_settings",
Description: "The access approval settings associated with this project.",
Type: proto.ColumnType_JSON,
Hydrate: getProjectAccessApprovalSettings,
Transform: transform.FromValue(),
},
{
Name: "ancestors",
Description: "The ancestors of the project in the resource hierarchy, from bottom to top.",
Type: proto.ColumnType_JSON,
Hydrate: getProjectAncestors,
Transform: transform.FromValue(),
},

// Steampipe standard columns
{
Name: "title",
Description: ColumnDescriptionTitle,
Type: proto.ColumnType_STRING,
Transform: transform.FromField("Name"),
},
{
Name: "tags",
Description: ColumnDescriptionTags,
Type: proto.ColumnType_JSON,
Transform: transform.FromField("Labels"),
},
{
Name: "akas",
Description: ColumnDescriptionAkas,
Type: proto.ColumnType_JSON,
Hydrate: getProjectAka,
Transform: transform.FromValue(),
},
},
}
}

//// LIST FUNCTION

func listGCPOrganizationProjects(ctx context.Context, d *plugin.QueryData, h *plugin.HydrateData) (interface{}, error) {
// Create Service Connection
service, err := CloudResourceManagerService(ctx, d)
if err != nil {
return nil, err
}

// List projects
resp, err := service.Projects.List().Do()
if err != nil {
return nil, err
}

for _, project := range resp.Projects {
d.StreamListItem(ctx, project)
}

return nil, nil
}
17 changes: 4 additions & 13 deletions gcp/table_gcp_project.go
Original file line number Diff line number Diff line change
Expand Up @@ -151,14 +151,9 @@ func getProjectAccessApprovalSettings(ctx context.Context, d *plugin.QueryData,
}

// Get project details
projectId := h.Item.(*cloudresourcemanager.Project).ProjectId

projectId, err := getProject(ctx, d, h)
if err != nil {
return nil, err
}
project := projectId.(string)

resp, err := service.Projects.GetAccessApprovalSettings("projects/" + project + "/accessApprovalSettings").Do()
resp, err := service.Projects.GetAccessApprovalSettings("projects/" + projectId + "/accessApprovalSettings").Do()
if err != nil {
if strings.Contains(err.Error(), "404") {
return nil, nil
Expand All @@ -178,13 +173,9 @@ func getProjectAncestors(ctx context.Context, d *plugin.QueryData, h *plugin.Hyd
}

// Get project details
projectId, err := getProject(ctx, d, h)
if err != nil {
return nil, err
}
project := projectId.(string)
projectId := h.Item.(*cloudresourcemanager.Project).ProjectId

resp, err := service.Projects.GetAncestry(project, &cloudresourcemanager.GetAncestryRequest{}).Do()
resp, err := service.Projects.GetAncestry(projectId, &cloudresourcemanager.GetAncestryRequest{}).Do()
if err != nil {
if strings.Contains(err.Error(), "404") {
return nil, nil
Expand Down

0 comments on commit 0a3d182

Please sign in to comment.