Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[rubrik] Enable TSDB #12917

Merged
merged 14 commits into from
Mar 4, 2025
2 changes: 2 additions & 0 deletions packages/rubrik/_dev/build/docs/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -145,6 +145,8 @@ Please refer to the following [document](https://www.elastic.co/guide/en/ecs/cur

The `node_statistics` dataset provides metrics related to the performance of the Rubrik cluster nodes.

**IMPORTANT: Setting `interval` to more than `1h` may cause documents to be dropped if node statistics metrics fall outside the index time range.**

**ECS Field Reference**

Please refer to the following [document](https://www.elastic.co/guide/en/ecs/current/ecs-field-reference.html) for detailed information on ECS fields.
Expand Down
8 changes: 8 additions & 0 deletions packages/rubrik/changelog.yml
Original file line number Diff line number Diff line change
@@ -1,4 +1,12 @@
# newer versions go on top
- version: "0.8.0"
changes:
- description: |
Enable TSDB.
Added support for specifying a time range in `ClusterNodeStatistics` query.
Adjusted default interval from `24h` to `1h`.
type: enhancement
link: https://github.com/elastic/integrations/pull/12917
- version: "0.7.0"
changes:
- description: Add `sla_domains` data stream
Expand Down
2 changes: 2 additions & 0 deletions packages/rubrik/data_stream/drives/manifest.yml
Original file line number Diff line number Diff line change
Expand Up @@ -70,3 +70,5 @@ streams:
required: false
show_user: false
description: The request tracer logs requests and responses to the agent's local file-system for debugging configurations. Enabling this request tracing compromises security and should only be used for debugging. See [documentation](https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-input-httpjson.html#_request_tracer_filename) for details.
elasticsearch:
index_mode: time_series
2 changes: 2 additions & 0 deletions packages/rubrik/data_stream/filesets/manifest.yml
Original file line number Diff line number Diff line change
Expand Up @@ -63,3 +63,5 @@ streams:
default: "# filter:\n# - field: \"location\"\n# texts: \n# - \"prod-lab.local\"\n"
description: >-
Specify filters for refining the data. Filters must be defined as an array of Filter objects. Refer to the [Rubrik API documentation](https://rubrikinc.github.io/rubrik-api-documentation/schema/reference/filter.doc.html) for valid fields.
elasticsearch:
index_mode: time_series
Original file line number Diff line number Diff line change
Expand Up @@ -54,3 +54,5 @@ streams:
required: false
show_user: false
description: The number of results per page in Rubrik GraphQL queries. Smaller pages mean more API requests but can be more efficient in some cases.
elasticsearch:
index_mode: time_series
2 changes: 2 additions & 0 deletions packages/rubrik/data_stream/managed_volumes/manifest.yml
Original file line number Diff line number Diff line change
Expand Up @@ -70,3 +70,5 @@ streams:
required: false
show_user: false
description: The request tracer logs requests and responses to the agent's local file-system for debugging configurations. Enabling this request tracing compromises security and should only be used for debugging. See [documentation](https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-input-httpjson.html#_request_tracer_filename) for details.
elasticsearch:
index_mode: time_series
2 changes: 2 additions & 0 deletions packages/rubrik/data_stream/monitoring_jobs/manifest.yml
Original file line number Diff line number Diff line change
Expand Up @@ -54,3 +54,5 @@ streams:
required: false
show_user: false
description: The request tracer logs requests and responses to the agent's local file-system for debugging configurations. Enabling this request tracing compromises security and should only be used for debugging. See [documentation](https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-input-httpjson.html#_request_tracer_filename) for details.
elasticsearch:
index_mode: time_series
2 changes: 2 additions & 0 deletions packages/rubrik/data_stream/mssql_databases/manifest.yml
Original file line number Diff line number Diff line change
Expand Up @@ -78,3 +78,5 @@ streams:
required: false
show_user: false
description: The request tracer logs requests and responses to the agent's local file-system for debugging configurations. Enabling this request tracing compromises security and should only be used for debugging. See [documentation](https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-input-httpjson.html#_request_tracer_filename) for details.
elasticsearch:
index_mode: time_series
Original file line number Diff line number Diff line change
Expand Up @@ -35,17 +35,18 @@ processors:

state:
index: 0
period: {{interval}}
pageSize: {{pageSize}}
queries:
- |
query ClusterNodeStatistics($first: Int, $after: String) {
query ClusterNodeStatistics($first: Int, $after: String, $range: TimeRangeInput) {
allClusterConnection(first: $first, after: $after) {
count
nodes {
name
type
id
clusterNodeStats {
clusterNodeStats(timeRange: $range) {
nodeId
clusterPhysicalDataIngest
networkBytesReceived
Expand Down Expand Up @@ -81,6 +82,10 @@ program: |-
"variables": {
"first": has(state.pageSize) ? state.pageSize : null,
"after": has(state.cursor) ? state.cursor.after : null,
"range": {
"start": now - duration(state.period),
"end": now.format(time_layout.RFC3339),
}
}
}.encode_json()
}
Expand Down Expand Up @@ -132,7 +137,7 @@ program: |-
"index": body.data.allClusterConnection.pageInfo.hasNextPage ? int(state.index) : (int(state.index) + 1) % size(state.queries),
"cursor": {
"after": body.data.allClusterConnection.pageInfo.hasNextPage ? body.data.allClusterConnection.pageInfo.endCursor : null
}
},
}
)
:
Expand Down
4 changes: 3 additions & 1 deletion packages/rubrik/data_stream/node_statistics/manifest.yml
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ streams:
description: The interval at which the API is polled, supported in seconds, minutes, and hours.
show_user: true
required: true
default: 24h
default: 1h
- name: processors
type: yaml
title: Processors
Expand Down Expand Up @@ -54,3 +54,5 @@ streams:
required: false
show_user: false
description: The number of results per page in Rubrik GraphQL queries. Smaller pages mean more API requests but can be more efficient in some cases.
elasticsearch:
index_mode: time_series
2 changes: 2 additions & 0 deletions packages/rubrik/data_stream/physical_hosts/manifest.yml
Original file line number Diff line number Diff line change
Expand Up @@ -78,3 +78,5 @@ streams:
required: false
show_user: false
description: The request tracer logs requests and responses to the agent's local file-system for debugging configurations. Enabling this request tracing compromises security and should only be used for debugging. See [documentation](https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-input-httpjson.html#_request_tracer_filename) for details.
elasticsearch:
index_mode: time_series
2 changes: 2 additions & 0 deletions packages/rubrik/data_stream/tasks/manifest.yml
Original file line number Diff line number Diff line change
Expand Up @@ -47,3 +47,5 @@ streams:
type: bool
multi: false
default: false
elasticsearch:
index_mode: time_series
2 changes: 2 additions & 0 deletions packages/rubrik/data_stream/virtual_machines/manifest.yml
Original file line number Diff line number Diff line change
Expand Up @@ -70,3 +70,5 @@ streams:
required: false
show_user: false
description: The request tracer logs requests and responses to the agent's local file-system for debugging configurations. Enabling this request tracing compromises security and should only be used for debugging. See [documentation](https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-input-httpjson.html#_request_tracer_filename) for details.
elasticsearch:
index_mode: time_series
2 changes: 2 additions & 0 deletions packages/rubrik/docs/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -1058,6 +1058,8 @@ An example event for `global_cluster_performance` looks as following:

The `node_statistics` dataset provides metrics related to the performance of the Rubrik cluster nodes.

**IMPORTANT: Setting `interval` to more than `1h` may cause documents to be dropped if node statistics metrics fall outside the index time range.**

**ECS Field Reference**

Please refer to the following [document](https://www.elastic.co/guide/en/ecs/current/ecs-field-reference.html) for detailed information on ECS fields.
Expand Down
2 changes: 1 addition & 1 deletion packages/rubrik/manifest.yml
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
format_version: 3.0.2
name: rubrik
title: "Rubrik RSC Metrics"
version: 0.7.0
version: 0.8.0
source:
license: "Elastic-2.0"
description: "Collect Metrics from Rubrik RSC with Elastic Agent."
Expand Down