From 03963429a6e01bae2566297c8c201eb25aae486e Mon Sep 17 00:00:00 2001 From: "github-actions[bot]" <41898282+github-actions[bot]@users.noreply.github.com> Date: Fri, 22 Nov 2024 01:24:34 +0000 Subject: [PATCH] Update 'Metric' reference documentation. (#3496) Co-authored-by: kkraune --- .../clustercontroller-metrics-reference.html | 4 +- .../default-set-metrics-reference.html | 134 +-- en/reference/vespa-set-metrics-reference.html | 1018 ++++++++--------- 3 files changed, 578 insertions(+), 578 deletions(-) diff --git a/en/reference/clustercontroller-metrics-reference.html b/en/reference/clustercontroller-metrics-reference.html index 93ca5fd728..b4ab15c5c0 100644 --- a/en/reference/clustercontroller-metrics-reference.html +++ b/en/reference/clustercontroller-metrics-reference.html @@ -100,12 +100,12 @@

cluster-controller.resource_usage.memory_limit

- Disk space limit as a fraction of available disk space + Memory space limit as a fraction of available memory fraction

cluster-controller.resource_usage.disk_limit

- Memory space limit as a fraction of available memory + Disk space limit as a fraction of available disk space fraction diff --git a/en/reference/default-set-metrics-reference.html b/en/reference/default-set-metrics-reference.html index 550d732f29..9cb740f0dc 100644 --- a/en/reference/default-set-metrics-reference.html +++ b/en/reference/default-set-metrics-reference.html @@ -11,50 +11,50 @@

ClusterController Metrics

- + - + - + - + - + - + - + - +
NameDescriptionUnitSuffixes
NameUnitSuffixesDescription

cluster-controller.down.count

Number of content nodes down node last, maxNumber of content nodes down

cluster-controller.maintenance.count

Number of content nodes in maintenance node last, maxNumber of content nodes in maintenance

cluster-controller.up.count

Number of content nodes up node last, maxNumber of content nodes up

cluster-controller.is-master

1 if this cluster controller is currently the master, or 0 if not binary last, max1 if this cluster controller is currently the master, or 0 if not

cluster-controller.resource_usage.nodes_above_limit

The number of content nodes above resource limit, blocking feed node last, maxThe number of content nodes above resource limit, blocking feed

cluster-controller.resource_usage.max_memory_utilization

Current memory utilisation, for content node with highest value fraction last, maxCurrent memory utilisation, for content node with highest value

cluster-controller.resource_usage.max_disk_utilization

Current disk space utilisation, for content node with highest value fraction last, maxCurrent disk space utilisation, for content node with highest value
@@ -62,176 +62,176 @@

ClusterController Metrics

Container Metrics

- + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - +
NameDescriptionUnitSuffixes
NameUnitSuffixesDescription

http.status.1xx

Number of responses with a 1xx status response rateNumber of responses with a 1xx status

http.status.2xx

Number of responses with a 2xx status response rateNumber of responses with a 2xx status

http.status.3xx

Number of responses with a 3xx status response rateNumber of responses with a 3xx status

http.status.4xx

Number of responses with a 4xx status response rateNumber of responses with a 4xx status

http.status.5xx

Number of responses with a 5xx status response rateNumber of responses with a 5xx status

jdisc.gc.ms

Time spent in JVM garbage collection millisecond average, maxTime spent in JVM garbage collection

jdisc.thread_pool.work_queue.capacity

Capacity of the task queue thread maxCapacity of the task queue

jdisc.thread_pool.work_queue.size

Size of the task queue thread count, max, min, sumSize of the task queue

jdisc.thread_pool.size

Size of the thread pool thread maxSize of the thread pool

jdisc.thread_pool.active_threads

Number of threads that are active thread count, max, min, sumNumber of threads that are active

jdisc.application.failed_component_graphs

JDISC Application failed component graphs item rateJDISC Application failed component graphs

jdisc.singleton.is_active

JDISC Singleton is active item last, maxJDISC Singleton is active

jdisc.http.ssl.handshake.failure.missing_client_cert

JDISC HTTP SSL Handshake failures due to missing client certificate operation rateJDISC HTTP SSL Handshake failures due to missing client certificate

jdisc.http.ssl.handshake.failure.incompatible_protocols

JDISC HTTP SSL Handshake failures due to incompatible protocols operation rateJDISC HTTP SSL Handshake failures due to incompatible protocols

jdisc.http.ssl.handshake.failure.incompatible_chifers

JDISC HTTP SSL Handshake failures due to incompatible chifers operation rateJDISC HTTP SSL Handshake failures due to incompatible chifers

jdisc.http.ssl.handshake.failure.unknown

JDISC HTTP SSL Handshake failures for unknown reason operation rateJDISC HTTP SSL Handshake failures for unknown reason

mem.heap.free

Free heap memory byte averageFree heap memory

athenz-tenant-cert.expiry.seconds

Time remaining until Athenz tenant certificate expires second last, max, minTime remaining until Athenz tenant certificate expires

feed.operations

Number of document feed operations operation rateNumber of document feed operations

feed.latency

Feed latency millisecond count, sumFeed latency

queries

Query volume operation rateQuery volume

query_latency

The overall query latency as seen by the container millisecond average, count, max, sumThe overall query latency as seen by the container

failed_queries

The number of failed queries operation rateThe number of failed queries

degraded_queries

The number of degraded queries, e.g. due to some content nodes not responding in time operation rateThe number of degraded queries, e.g. due to some content nodes not responding in time

hits_per_query

The number of hits returned hit_per_query average, count, max, sumThe number of hits returned

docproc.documents

Number of processed documents document sumNumber of processed documents

totalhits_per_query

The total number of documents found to match queries hit_per_query average, count, max, sumThe total number of documents found to match queries

serverActiveThreads

Deprecated. Use jdisc.thread_pool.active_threads instead. thread averageDeprecated. Use jdisc.thread_pool.active_threads instead.
@@ -239,20 +239,20 @@

Container Metrics

Distributor Metrics

- + - + - +
NameDescriptionUnitSuffixes
NameUnitSuffixesDescription

vds.distributor.docsstored

Number of documents stored in all buckets controlled by this distributor document averageNumber of documents stored in all buckets controlled by this distributor

vds.bouncer.clock_skew_aborts

Number of client operations that were aborted due to clock skew between sender and receiver exceeding acceptable range operation countNumber of client operations that were aborted due to clock skew between sender and receiver exceeding acceptable range
@@ -260,20 +260,20 @@

Distributor Metrics

NodeAdmin Metrics

- + - + - +
NameDescriptionUnitSuffixes
NameUnitSuffixesDescription

endpoint.certificate.expiry.seconds

Time until node endpoint certificate expires second N/ATime until node endpoint certificate expires

node-certificate.expiry.seconds

Time until node certificate expires second N/ATime until node certificate expires
@@ -281,110 +281,110 @@

NodeAdmin Metrics

SearchNode Metrics

- + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - +
NameDescriptionUnitSuffixes
NameUnitSuffixesDescription

content.proton.documentdb.documents.total

The total number of documents in this documents db (ready + not-ready) document last, maxThe total number of documents in this documents db (ready + not-ready)

content.proton.documentdb.documents.ready

The number of ready documents in this document db document last, maxThe number of ready documents in this document db

content.proton.documentdb.documents.active

The number of active / searchable documents in this document db document last, maxThe number of active / searchable documents in this document db

content.proton.documentdb.disk_usage

The total disk usage (in bytes) for this document db byte lastThe total disk usage (in bytes) for this document db

content.proton.documentdb.memory_usage.allocated_bytes

The number of allocated bytes byte lastThe number of allocated bytes

content.proton.search_protocol.query.latency

Query request latency (seconds) second average, count, max, sumQuery request latency (seconds)

content.proton.search_protocol.docsum.latency

Docsum request latency (seconds) second average, count, max, sumDocsum request latency (seconds)

content.proton.search_protocol.docsum.requested_documents

Total requested document summaries document rateTotal requested document summaries

content.proton.resource_usage.disk

The relative amount of disk used by this content node (transient usage not included, value in the range [0, 1]). Same value as reported to the cluster controller fraction averageThe relative amount of disk used by this content node (transient usage not included, value in the range [0, 1]). Same value as reported to the cluster controller

content.proton.resource_usage.memory

The relative amount of memory used by this content node (transient usage not included, value in the range [0, 1]). Same value as reported to the cluster controller fraction averageThe relative amount of memory used by this content node (transient usage not included, value in the range [0, 1]). Same value as reported to the cluster controller

content.proton.resource_usage.feeding_blocked

Whether feeding is blocked due to resource limits being reached (value is either 0 or 1) binary last, maxWhether feeding is blocked due to resource limits being reached (value is either 0 or 1)

content.proton.transactionlog.disk_usage

The disk usage (in bytes) of the transaction log byte lastThe disk usage (in bytes) of the transaction log

content.proton.documentdb.matching.docs_matched

Number of documents matched document rateNumber of documents matched

content.proton.documentdb.matching.docs_reranked

Number of documents re-ranked (second phase) document rateNumber of documents re-ranked (second phase)

content.proton.documentdb.matching.rank_profile.query_latency

Total average latency (sec) when matching and ranking a query second average, count, max, sumTotal average latency (sec) when matching and ranking a query

content.proton.documentdb.matching.rank_profile.query_setup_time

Average time (sec) spent setting up and tearing down queries second average, count, max, sumAverage time (sec) spent setting up and tearing down queries

content.proton.documentdb.matching.rank_profile.rerank_time

Average time (sec) spent on 2nd phase ranking second average, count, max, sumAverage time (sec) spent on 2nd phase ranking
@@ -392,14 +392,14 @@

SearchNode Metrics

Sentinel Metrics

- + - +
NameDescriptionUnitSuffixes
NameUnitSuffixesDescription

sentinel.totalRestarts

Total number of service restarts done by the sentinel since the sentinel was started restart last, max, sumTotal number of service restarts done by the sentinel since the sentinel was started
@@ -407,26 +407,26 @@

Sentinel Metrics

Storage Metrics

- + - + - + - +
NameDescriptionUnitSuffixes
NameUnitSuffixesDescription

vds.filestor.allthreads.put.count

Number of requests processed. operation rateNumber of requests processed.

vds.filestor.allthreads.remove.count

Number of requests processed. operation rateNumber of requests processed.

vds.filestor.allthreads.update.count

Number of requests processed. request rateNumber of requests processed.
diff --git a/en/reference/vespa-set-metrics-reference.html b/en/reference/vespa-set-metrics-reference.html index 1c7e949741..2cf630d1cd 100644 --- a/en/reference/vespa-set-metrics-reference.html +++ b/en/reference/vespa-set-metrics-reference.html @@ -11,122 +11,122 @@

ClusterController Metrics

- + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - +
NameDescriptionUnitSuffixes
NameUnitSuffixesDescription

cluster-controller.down.count

Number of content nodes down node last, maxNumber of content nodes down

cluster-controller.initializing.count

Number of content nodes initializing node last, maxNumber of content nodes initializing

cluster-controller.maintenance.count

Number of content nodes in maintenance node last, maxNumber of content nodes in maintenance

cluster-controller.retired.count

Number of content nodes that are retired node last, maxNumber of content nodes that are retired

cluster-controller.stopping.count

Number of content nodes currently stopping node lastNumber of content nodes currently stopping

cluster-controller.up.count

Number of content nodes up node last, maxNumber of content nodes up

cluster-controller.nodes-not-converged

Number of nodes not converging to the latest cluster state version node maxNumber of nodes not converging to the latest cluster state version

cluster-controller.cluster-buckets-out-of-sync-ratio

Ratio of buckets in the cluster currently in need of syncing fraction maxRatio of buckets in the cluster currently in need of syncing

cluster-controller.busy-tick-time-ms

Time busy millisecond count, last, max, sumTime busy

cluster-controller.idle-tick-time-ms

Time idle millisecond count, last, max, sumTime idle

cluster-controller.work-ms

Time used for actual work millisecond count, last, sumTime used for actual work

cluster-controller.is-master

1 if this cluster controller is currently the master, or 0 if not binary last, max1 if this cluster controller is currently the master, or 0 if not

cluster-controller.remote-task-queue.size

Number of remote tasks queued operation lastNumber of remote tasks queued

cluster-controller.resource_usage.nodes_above_limit

The number of content nodes above resource limit, blocking feed node last, maxThe number of content nodes above resource limit, blocking feed

cluster-controller.resource_usage.max_memory_utilization

Current memory utilisation, for content node with highest value fraction last, maxCurrent memory utilisation, for content node with highest value

cluster-controller.resource_usage.max_disk_utilization

Current disk space utilisation, for content node with highest value fraction last, maxCurrent disk space utilisation, for content node with highest value

cluster-controller.resource_usage.memory_limit

Disk space limit as a fraction of available disk space fraction last, maxMemory space limit as a fraction of available memory

cluster-controller.resource_usage.disk_limit

Memory space limit as a fraction of available memory fraction last, maxDisk space limit as a fraction of available disk space

reindexing.progress

Re-indexing progress fraction last, maxRe-indexing progress
@@ -134,836 +134,836 @@

ClusterController Metrics

Container Metrics

- + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - +
NameDescriptionUnitSuffixes
NameUnitSuffixesDescription

http.status.1xx

Number of responses with a 1xx status response rateNumber of responses with a 1xx status

http.status.2xx

Number of responses with a 2xx status response rateNumber of responses with a 2xx status

http.status.3xx

Number of responses with a 3xx status response rateNumber of responses with a 3xx status

http.status.4xx

Number of responses with a 4xx status response rateNumber of responses with a 4xx status

http.status.5xx

Number of responses with a 5xx status response rateNumber of responses with a 5xx status

application_generation

The currently live application config generation (aka session id) version N/AThe currently live application config generation (aka session id)

jdisc.gc.count

Number of JVM garbage collections done operation average, last, maxNumber of JVM garbage collections done

jdisc.gc.ms

Time spent in JVM garbage collection millisecond average, last, maxTime spent in JVM garbage collection

jdisc.jvm

JVM runtime version version lastJVM runtime version

jdisc.memory_mappings

JDISC Memory mappings operation maxJDISC Memory mappings

jdisc.open_file_descriptors

JDISC Open file descriptors item maxJDISC Open file descriptors

jdisc.thread_pool.unhandled_exceptions

Number of exceptions thrown by tasks thread count, last, max, min, sumNumber of exceptions thrown by tasks

jdisc.thread_pool.work_queue.capacity

Capacity of the task queue thread count, last, max, min, sumCapacity of the task queue

jdisc.thread_pool.work_queue.size

Size of the task queue thread count, last, max, min, sumSize of the task queue

jdisc.thread_pool.rejected_tasks

Number of tasks rejected by the thread pool thread count, last, max, min, sumNumber of tasks rejected by the thread pool

jdisc.thread_pool.size

Size of the thread pool thread count, last, max, min, sumSize of the thread pool

jdisc.thread_pool.max_allowed_size

The maximum allowed number of threads in the pool thread count, last, max, min, sumThe maximum allowed number of threads in the pool

jdisc.thread_pool.active_threads

Number of threads that are active thread count, last, max, min, sumNumber of threads that are active

jdisc.deactivated_containers.total

JDISC Deactivated container instances item last, sumJDISC Deactivated container instances

jdisc.deactivated_containers.with_retained_refs.last

JDISC Deactivated container nodes with retained refs item lastJDISC Deactivated container nodes with retained refs

jdisc.application.failed_component_graphs

JDISC Application failed component graphs item rateJDISC Application failed component graphs

jdisc.application.component_graph.creation_time_millis

JDISC Application component graph creation time millisecond lastJDISC Application component graph creation time

jdisc.application.component_graph.reconfigurations

JDISC Application component graph reconfigurations item rateJDISC Application component graph reconfigurations

jdisc.singleton.is_active

JDISC Singleton is active item last, max, minJDISC Singleton is active

jdisc.singleton.activation.count

JDISC Singleton activations operation lastJDISC Singleton activations

jdisc.singleton.activation.failure.count

JDISC Singleton activation failures operation lastJDISC Singleton activation failures

jdisc.singleton.activation.millis

JDISC Singleton activation time millisecond lastJDISC Singleton activation time

jdisc.singleton.deactivation.count

JDISC Singleton deactivations operation lastJDISC Singleton deactivations

jdisc.singleton.deactivation.failure.count

JDISC Singleton deactivation failures operation lastJDISC Singleton deactivation failures

jdisc.singleton.deactivation.millis

JDISC Singleton deactivation time millisecond lastJDISC Singleton deactivation time

jdisc.http.ssl.handshake.failure.missing_client_cert

JDISC HTTP SSL Handshake failures due to missing client certificate operation rateJDISC HTTP SSL Handshake failures due to missing client certificate

jdisc.http.ssl.handshake.failure.expired_client_cert

JDISC HTTP SSL Handshake failures due to expired client certificate operation rateJDISC HTTP SSL Handshake failures due to expired client certificate

jdisc.http.ssl.handshake.failure.invalid_client_cert

JDISC HTTP SSL Handshake failures due to invalid client certificate operation rateJDISC HTTP SSL Handshake failures due to invalid client certificate

jdisc.http.ssl.handshake.failure.incompatible_protocols

JDISC HTTP SSL Handshake failures due to incompatible protocols operation rateJDISC HTTP SSL Handshake failures due to incompatible protocols

jdisc.http.ssl.handshake.failure.incompatible_chifers

JDISC HTTP SSL Handshake failures due to incompatible chifers operation rateJDISC HTTP SSL Handshake failures due to incompatible chifers

jdisc.http.ssl.handshake.failure.connection_closed

JDISC HTTP SSL Handshake failures due to connection closed operation rateJDISC HTTP SSL Handshake failures due to connection closed

jdisc.http.ssl.handshake.failure.unknown

JDISC HTTP SSL Handshake failures for unknown reason operation rateJDISC HTTP SSL Handshake failures for unknown reason

jdisc.http.request.prematurely_closed

HTTP requests prematurely closed request rateHTTP requests prematurely closed

jdisc.http.request.requests_per_connection

HTTP requests per connection request average, count, max, min, sumHTTP requests per connection

jdisc.http.request.uri_length

HTTP URI length byte count, max, sumHTTP URI length

jdisc.http.request.content_size

HTTP request content size byte count, max, sumHTTP request content size

jdisc.http.requests

HTTP requests request count, rateHTTP requests

jdisc.http.filter.rule.blocked_requests

Number of requests blocked by filter request rateNumber of requests blocked by filter

jdisc.http.filter.rule.allowed_requests

Number of requests allowed by filter request rateNumber of requests allowed by filter

jdisc.http.filtering.request.handled

Number of filtering requests handled request rateNumber of filtering requests handled

jdisc.http.filtering.request.unhandled

Number of filtering requests unhandled request rateNumber of filtering requests unhandled

jdisc.http.filtering.response.handled

Number of filtering responses handled request rateNumber of filtering responses handled

jdisc.http.filtering.response.unhandled

Number of filtering responses unhandled request rateNumber of filtering responses unhandled

jdisc.http.handler.unhandled_exceptions

Number of unhandled exceptions in handler request rateNumber of unhandled exceptions in handler

jdisc.tls.capability_checks.succeeded

Number of TLS capability checks succeeded operation rateNumber of TLS capability checks succeeded

jdisc.tls.capability_checks.failed

Number of TLS capability checks failed operation rateNumber of TLS capability checks failed

jdisc.http.jetty.threadpool.thread.max

Configured maximum number of threads thread count, last, max, min, sumConfigured maximum number of threads

jdisc.http.jetty.threadpool.thread.min

Configured minimum number of threads thread count, last, max, min, sumConfigured minimum number of threads

jdisc.http.jetty.threadpool.thread.reserved

Configured number of reserved threads or -1 for heuristic thread count, last, max, min, sumConfigured number of reserved threads or -1 for heuristic

jdisc.http.jetty.threadpool.thread.busy

Number of threads executing internal and transient jobs thread count, last, max, min, sumNumber of threads executing internal and transient jobs

jdisc.http.jetty.threadpool.thread.total

Current number of threads thread count, last, max, min, sumCurrent number of threads

jdisc.http.jetty.threadpool.queue.size

Current size of the job queue thread count, last, max, min, sumCurrent size of the job queue

serverNumOpenConnections

The number of currently open connections connection average, last, maxThe number of currently open connections

serverNumConnections

The total number of connections opened connection average, last, maxThe total number of connections opened

serverBytesReceived

The number of bytes received by the server byte count, sumThe number of bytes received by the server

serverBytesSent

The number of bytes sent from the server byte count, sumThe number of bytes sent from the server

handled.requests

The number of requests handled per metrics snapshot operation countThe number of requests handled per metrics snapshot

handled.latency

The time used for requests during this metrics snapshot millisecond count, max, sumThe time used for requests during this metrics snapshot

httpapi_latency

Duration for requests to the HTTP document APIs millisecond count, max, sumDuration for requests to the HTTP document APIs

httpapi_pending

Document operations pending execution operation count, max, sumDocument operations pending execution

httpapi_num_operations

Total number of document operations performed operation rateTotal number of document operations performed

httpapi_num_updates

Document update operations performed operation rateDocument update operations performed

httpapi_num_removes

Document remove operations performed operation rateDocument remove operations performed

httpapi_num_puts

Document put operations performed operation rateDocument put operations performed

httpapi_succeeded

Document operations that succeeded operation rateDocument operations that succeeded

httpapi_failed

Document operations that failed operation rateDocument operations that failed

httpapi_parse_error

Document operations that failed due to document parse errors operation rateDocument operations that failed due to document parse errors

httpapi_condition_not_met

Document operations not applied due to condition not met operation rateDocument operations not applied due to condition not met

httpapi_not_found

Document operations not applied due to document not found operation rateDocument operations not applied due to document not found

httpapi_failed_unknown

Document operations failed by unknown cause operation rateDocument operations failed by unknown cause

httpapi_failed_timeout

Document operations failed by timeout operation rateDocument operations failed by timeout

httpapi_failed_insufficient_storage

Document operations failed by insufficient storage operation rateDocument operations failed by insufficient storage

mem.heap.total

Total available heap memory byte averageTotal available heap memory

mem.heap.free

Free heap memory byte averageFree heap memory

mem.heap.used

Currently used heap memory byte average, maxCurrently used heap memory

mem.direct.total

Total available direct memory byte averageTotal available direct memory

mem.direct.free

Currently free direct memory byte averageCurrently free direct memory

mem.direct.used

Direct memory currently used byte average, maxDirect memory currently used

mem.direct.count

Number of direct memory allocations byte maxNumber of direct memory allocations

mem.native.total

Total available native memory byte averageTotal available native memory

mem.native.free

Currently free native memory byte averageCurrently free native memory

mem.native.used

Native memory currently used byte averageNative memory currently used

athenz-tenant-cert.expiry.seconds

Time remaining until Athenz tenant certificate expires second last, max, minTime remaining until Athenz tenant certificate expires

container-iam-role.expiry.seconds

Time remaining until IAM role expires second N/ATime remaining until IAM role expires

peak_qps

The highest number of qps for a second for this metrics snapshot query_per_second maxThe highest number of qps for a second for this metrics snapshot

search_connections

Number of search connections connection count, max, sumNumber of search connections

feed.operations

Number of document feed operations operation rateNumber of document feed operations

feed.latency

Feed latency millisecond count, max, sumFeed latency

feed.http-requests

Feed HTTP requests operation count, rateFeed HTTP requests

queries

Query volume operation rateQuery volume

query_container_latency

The query execution time consumed in the container millisecond count, max, sumThe query execution time consumed in the container

query_latency

The overall query latency as seen by the container millisecond count, max, sumThe overall query latency as seen by the container

query_timeout

The amount of time allowed for query execution, from the client millisecond count, max, min, sumThe amount of time allowed for query execution, from the client

failed_queries

The number of failed queries operation rateThe number of failed queries

degraded_queries

The number of degraded queries, e.g. due to some content nodes not responding in time operation rateThe number of degraded queries, e.g. due to some content nodes not responding in time

hits_per_query

The number of hits returned hit_per_query count, max, sumThe number of hits returned

query_hit_offset

The offset for hits returned hit count, max, sumThe offset for hits returned

documents_covered

The combined number of documents considered during query evaluation document countThe combined number of documents considered during query evaluation

documents_total

The number of documents to be evaluated if all requests had been fully executed document countThe number of documents to be evaluated if all requests had been fully executed

documents_target_total

The target number of total documents to be evaluated when when all data is in sync document countThe target number of total documents to be evaluated when when all data is in sync

jdisc.render.latency

The time used by the container to render responses nanosecond average, count, last, max, min, sumThe time used by the container to render responses

query_item_count

The number of query items (terms, phrases, etc) item count, max, sumThe number of query items (terms, phrases, etc)

docproc.proctime

Time spent processing document millisecond count, max, sumTime spent processing document

docproc.documents

Number of processed documents document count, max, min, sumNumber of processed documents

totalhits_per_query

The total number of documents found to match queries hit_per_query count, max, sumThe total number of documents found to match queries

empty_results

Number of queries matching no documents operation rateNumber of queries matching no documents

requestsOverQuota

The number of requests rejected due to exceeding quota operation count, rateThe number of requests rejected due to exceeding quota

relevance.at_1

The relevance of hit number 1 score count, sumThe relevance of hit number 1

relevance.at_3

The relevance of hit number 3 score count, sumThe relevance of hit number 3

relevance.at_10

The relevance of hit number 10 score count, sumThe relevance of hit number 10

error.timeout

Requests that timed out operation rateRequests that timed out

error.backends_oos

Requests that failed due to no available backends nodes operation rateRequests that failed due to no available backends nodes

error.plugin_failure

Requests that failed due to plugin failure operation rateRequests that failed due to plugin failure

error.backend_communication_error

Requests that failed due to backend communication error operation rateRequests that failed due to backend communication error

error.empty_document_summaries

Requests that failed due to missing document summaries operation rateRequests that failed due to missing document summaries

error.invalid_query_parameter

Requests that failed due to invalid query parameters operation rateRequests that failed due to invalid query parameters

error.internal_server_error

Requests that failed due to internal server error operation rateRequests that failed due to internal server error

error.misconfigured_server

Requests that failed due to misconfigured server operation rateRequests that failed due to misconfigured server

error.invalid_query_transformation

Requests that failed due to invalid query transformation operation rateRequests that failed due to invalid query transformation

error.results_with_errors

The number of queries with error payload operation rateThe number of queries with error payload

error.unspecified

Requests that failed for an unspecified reason operation rateRequests that failed for an unspecified reason

error.unhandled_exception

Requests that failed due to an unhandled exception operation rateRequests that failed due to an unhandled exception

serverRejectedRequests

Deprecated. Use jdisc.thread_pool.rejected_tasks instead. operation count, rateDeprecated. Use jdisc.thread_pool.rejected_tasks instead.

serverThreadPoolSize

Deprecated. Use jdisc.thread_pool.size instead. thread last, maxDeprecated. Use jdisc.thread_pool.size instead.

serverActiveThreads

Deprecated. Use jdisc.thread_pool.active_threads instead. thread count, last, max, min, sumDeprecated. Use jdisc.thread_pool.active_threads instead.

jrt.transport.tls-certificate-verification-failures

TLS certificate verification failures failure N/ATLS certificate verification failures

jrt.transport.peer-authorization-failures

TLS peer authorization failures failure N/ATLS peer authorization failures

jrt.transport.server.tls-connections-established

TLS server connections established connection N/ATLS server connections established

jrt.transport.client.tls-connections-established

TLS client connections established connection N/ATLS client connections established

jrt.transport.server.unencrypted-connections-established

Unencrypted server connections established connection N/AUnencrypted server connections established

jrt.transport.client.unencrypted-connections-established

Unencrypted client connections established connection N/AUnencrypted client connections established

embedder.latency

Time spent creating an embedding millisecond count, max, sumTime spent creating an embedding

embedder.sequence_length

Size of sequence produced by tokenizer byte count, max, sumSize of sequence produced by tokenizer
@@ -971,488 +971,488 @@

Container Metrics

Distributor Metrics

- + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - +
NameDescriptionUnitSuffixes
NameUnitSuffixesDescription

vds.idealstate.buckets_rechecking

The number of buckets that we are rechecking for ideal state operations bucket averageThe number of buckets that we are rechecking for ideal state operations

vds.idealstate.idealstate_diff

A number representing the current difference from the ideal state. This is a number that decreases steadily as the system is getting closer to the ideal state bucket averageA number representing the current difference from the ideal state. This is a number that decreases steadily as the system is getting closer to the ideal state

vds.idealstate.buckets_toofewcopies

The number of buckets the distributor controls that have less than the desired redundancy bucket averageThe number of buckets the distributor controls that have less than the desired redundancy

vds.idealstate.buckets_toomanycopies

The number of buckets the distributor controls that have more than the desired redundancy bucket averageThe number of buckets the distributor controls that have more than the desired redundancy

vds.idealstate.buckets

The number of buckets the distributor controls bucket averageThe number of buckets the distributor controls

vds.idealstate.buckets_notrusted

The number of buckets that have no trusted copies. bucket averageThe number of buckets that have no trusted copies.

vds.idealstate.bucket_replicas_moving_out

Bucket replicas that should be moved out, e.g. retirement case or node added to cluster that has higher ideal state priority. bucket averageBucket replicas that should be moved out, e.g. retirement case or node added to cluster that has higher ideal state priority.

vds.idealstate.bucket_replicas_copying_out

Bucket replicas that should be copied out, e.g. node is in ideal state but might have to provide data other nodes in a merge bucket averageBucket replicas that should be copied out, e.g. node is in ideal state but might have to provide data other nodes in a merge

vds.idealstate.bucket_replicas_copying_in

Bucket replicas that should be copied in, e.g. node does not have a replica for a bucket that it is in ideal state for bucket averageBucket replicas that should be copied in, e.g. node does not have a replica for a bucket that it is in ideal state for

vds.idealstate.bucket_replicas_syncing

Bucket replicas that need syncing due to mismatching metadata bucket averageBucket replicas that need syncing due to mismatching metadata

vds.idealstate.max_observed_time_since_last_gc_sec

Maximum time (in seconds) since GC was last successfully run for a bucket. Aggregated max value across all buckets on the distributor. second averageMaximum time (in seconds) since GC was last successfully run for a bucket. Aggregated max value across all buckets on the distributor.

vds.idealstate.delete_bucket.done_ok

The number of operations successfully performed operation rateThe number of operations successfully performed

vds.idealstate.delete_bucket.done_failed

The number of operations that failed operation rateThe number of operations that failed

vds.idealstate.delete_bucket.pending

The number of operations pending operation averageThe number of operations pending

vds.idealstate.merge_bucket.done_ok

The number of operations successfully performed operation rateThe number of operations successfully performed

vds.idealstate.merge_bucket.done_failed

The number of operations that failed operation rateThe number of operations that failed

vds.idealstate.merge_bucket.pending

The number of operations pending operation averageThe number of operations pending

vds.idealstate.merge_bucket.blocked

The number of operations blocked by blocking operation starter operation rateThe number of operations blocked by blocking operation starter

vds.idealstate.merge_bucket.throttled

The number of operations throttled by throttling operation starter operation rateThe number of operations throttled by throttling operation starter

vds.idealstate.merge_bucket.source_only_copy_changed

The number of merge operations where source-only copy changed operation rateThe number of merge operations where source-only copy changed

vds.idealstate.merge_bucket.source_only_copy_delete_blocked

The number of merge operations where delete of unchanged source-only copies was blocked operation rateThe number of merge operations where delete of unchanged source-only copies was blocked

vds.idealstate.merge_bucket.source_only_copy_delete_failed

The number of merge operations where delete of unchanged source-only copies failed operation rateThe number of merge operations where delete of unchanged source-only copies failed

vds.idealstate.split_bucket.done_ok

The number of operations successfully performed operation rateThe number of operations successfully performed

vds.idealstate.split_bucket.done_failed

The number of operations that failed operation rateThe number of operations that failed

vds.idealstate.split_bucket.pending

The number of operations pending operation averageThe number of operations pending

vds.idealstate.join_bucket.done_ok

The number of operations successfully performed operation rateThe number of operations successfully performed

vds.idealstate.join_bucket.done_failed

The number of operations that failed operation rateThe number of operations that failed

vds.idealstate.join_bucket.pending

The number of operations pending operation averageThe number of operations pending

vds.idealstate.garbage_collection.done_ok

The number of operations successfully performed operation rateThe number of operations successfully performed

vds.idealstate.garbage_collection.done_failed

The number of operations that failed operation rateThe number of operations that failed

vds.idealstate.garbage_collection.pending

The number of operations pending operation averageThe number of operations pending

vds.idealstate.garbage_collection.documents_removed

Number of documents removed by GC operations document count, rateNumber of documents removed by GC operations

vds.distributor.puts.latency

The latency of put operations millisecond count, max, sumThe latency of put operations

vds.distributor.puts.ok

The number of successful put operations performed operation rateThe number of successful put operations performed

vds.distributor.puts.failures.total

Sum of all failures operation rateSum of all failures

vds.distributor.puts.failures.notfound

The number of operations that failed because the document did not exist operation rateThe number of operations that failed because the document did not exist

vds.distributor.puts.failures.test_and_set_failed

The number of mutating operations that failed because they specified a test-and-set condition that did not match the existing document operation rateThe number of mutating operations that failed because they specified a test-and-set condition that did not match the existing document

vds.distributor.puts.failures.concurrent_mutations

The number of operations that were transiently failed due to a mutating operation already being in progress for its document ID operation rateThe number of operations that were transiently failed due to a mutating operation already being in progress for its document ID

vds.distributor.puts.failures.notconnected

The number of operations discarded because there were no available storage nodes to send to operation rateThe number of operations discarded because there were no available storage nodes to send to

vds.distributor.puts.failures.notready

The number of operations discarded because distributor was not ready operation rateThe number of operations discarded because distributor was not ready

vds.distributor.puts.failures.wrongdistributor

The number of operations discarded because they were sent to the wrong distributor operation rateThe number of operations discarded because they were sent to the wrong distributor

vds.distributor.puts.failures.safe_time_not_reached

The number of operations that were transiently failed due to them arriving before the safe time point for bucket ownership handovers has passed operation rateThe number of operations that were transiently failed due to them arriving before the safe time point for bucket ownership handovers has passed

vds.distributor.puts.failures.storagefailure

The number of operations that failed in storage operation rateThe number of operations that failed in storage

vds.distributor.puts.failures.timeout

The number of operations that failed because the operation timed out towards storage operation rateThe number of operations that failed because the operation timed out towards storage

vds.distributor.puts.failures.busy

The number of messages from storage that failed because the storage node was busy operation rateThe number of messages from storage that failed because the storage node was busy

vds.distributor.puts.failures.inconsistent_bucket

The number of operations failed due to buckets being in an inconsistent state or not found operation rateThe number of operations failed due to buckets being in an inconsistent state or not found

vds.distributor.removes.latency

The latency of remove operations millisecond count, max, sumThe latency of remove operations

vds.distributor.removes.ok

The number of successful removes operations performed operation rateThe number of successful removes operations performed

vds.distributor.removes.failures.total

Sum of all failures operation rateSum of all failures

vds.distributor.removes.failures.notfound

The number of operations that failed because the document did not exist operation rateThe number of operations that failed because the document did not exist

vds.distributor.removes.failures.test_and_set_failed

The number of mutating operations that failed because they specified a test-and-set condition that did not match the existing document operation rateThe number of mutating operations that failed because they specified a test-and-set condition that did not match the existing document

vds.distributor.removes.failures.concurrent_mutations

The number of operations that were transiently failed due to a mutating operation already being in progress for its document ID operation rateThe number of operations that were transiently failed due to a mutating operation already being in progress for its document ID

vds.distributor.updates.latency

The latency of update operations millisecond count, max, sumThe latency of update operations

vds.distributor.updates.ok

The number of successful updates operations performed operation rateThe number of successful updates operations performed

vds.distributor.updates.failures.total

Sum of all failures operation rateSum of all failures

vds.distributor.updates.failures.notfound

The number of operations that failed because the document did not exist operation rateThe number of operations that failed because the document did not exist

vds.distributor.updates.failures.test_and_set_failed

The number of mutating operations that failed because they specified a test-and-set condition that did not match the existing document operation rateThe number of mutating operations that failed because they specified a test-and-set condition that did not match the existing document

vds.distributor.updates.failures.concurrent_mutations

The number of operations that were transiently failed due to a mutating operation already being in progress for its document ID operation rateThe number of operations that were transiently failed due to a mutating operation already being in progress for its document ID

vds.distributor.updates.diverging_timestamp_updates

Number of updates that report they were performed against divergent version timestamps on different replicas operation rateNumber of updates that report they were performed against divergent version timestamps on different replicas

vds.distributor.removelocations.ok

The number of successful removelocations operations performed operation rateThe number of successful removelocations operations performed

vds.distributor.removelocations.failures.total

Sum of all failures operation rateSum of all failures

vds.distributor.gets.latency

The average latency of gets operations millisecond count, max, sumThe average latency of gets operations

vds.distributor.gets.ok

The number of successful gets operations performed operation rateThe number of successful gets operations performed

vds.distributor.gets.failures.total

Sum of all failures operation rateSum of all failures

vds.distributor.gets.failures.notfound

The number of operations that failed because the document did not exist operation rateThe number of operations that failed because the document did not exist

vds.distributor.visitor.latency

The average latency of visitor operations millisecond count, max, sumThe average latency of visitor operations

vds.distributor.visitor.ok

The number of successful visitor operations performed operation rateThe number of successful visitor operations performed

vds.distributor.visitor.failures.total

Sum of all failures operation rateSum of all failures

vds.distributor.visitor.failures.notready

The number of operations discarded because distributor was not ready operation rateThe number of operations discarded because distributor was not ready

vds.distributor.visitor.failures.notconnected

The number of operations discarded because there were no available storage nodes to send to operation rateThe number of operations discarded because there were no available storage nodes to send to

vds.distributor.visitor.failures.wrongdistributor

The number of operations discarded because they were sent to the wrong distributor operation rateThe number of operations discarded because they were sent to the wrong distributor

vds.distributor.visitor.failures.safe_time_not_reached

The number of operations that were transiently failed due to them arriving before the safe time point for bucket ownership handovers has passed operation rateThe number of operations that were transiently failed due to them arriving before the safe time point for bucket ownership handovers has passed

vds.distributor.visitor.failures.storagefailure

The number of operations that failed in storage operation rateThe number of operations that failed in storage

vds.distributor.visitor.failures.timeout

The number of operations that failed because the operation timed out towards storage operation rateThe number of operations that failed because the operation timed out towards storage

vds.distributor.visitor.failures.busy

The number of messages from storage that failed because the storage node was busy operation rateThe number of messages from storage that failed because the storage node was busy

vds.distributor.visitor.failures.inconsistent_bucket

The number of operations failed due to buckets being in an inconsistent state or not found operation rateThe number of operations failed due to buckets being in an inconsistent state or not found

vds.distributor.visitor.failures.notfound

The number of operations that failed because the document did not exist operation rateThe number of operations that failed because the document did not exist

vds.distributor.docsstored

Number of documents stored in all buckets controlled by this distributor document averageNumber of documents stored in all buckets controlled by this distributor

vds.distributor.bytesstored

Number of bytes stored in all buckets controlled by this distributor byte averageNumber of bytes stored in all buckets controlled by this distributor

vds.bouncer.clock_skew_aborts

Number of client operations that were aborted due to clock skew between sender and receiver exceeding acceptable range operation countNumber of client operations that were aborted due to clock skew between sender and receiver exceeding acceptable range
@@ -1460,14 +1460,14 @@

Distributor Metrics

Logd Metrics

- + - +
NameDescriptionUnitSuffixes
NameUnitSuffixesDescription

logd.processed.lines

Number of log lines processed item countNumber of log lines processed
@@ -1475,20 +1475,20 @@

Logd Metrics

NodeAdmin Metrics

- + - + - +
NameDescriptionUnitSuffixes
NameUnitSuffixesDescription

endpoint.certificate.expiry.seconds

Time until node endpoint certificate expires second N/ATime until node endpoint certificate expires

node-certificate.expiry.seconds

Time until node certificate expires second N/ATime until node certificate expires
@@ -1496,1052 +1496,1052 @@

NodeAdmin Metrics

SearchNode Metrics

- + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - +
NameDescriptionUnitSuffixes
NameUnitSuffixesDescription

content.proton.config.generation

The oldest config generation used by this search node version lastThe oldest config generation used by this search node

content.proton.documentdb.documents.total

The total number of documents in this documents db (ready + not-ready) document last, maxThe total number of documents in this documents db (ready + not-ready)

content.proton.documentdb.documents.ready

The number of ready documents in this document db document last, maxThe number of ready documents in this document db

content.proton.documentdb.documents.active

The number of active / searchable documents in this document db document last, maxThe number of active / searchable documents in this document db

content.proton.documentdb.documents.removed

The number of removed documents in this document db document last, maxThe number of removed documents in this document db

content.proton.documentdb.index.docs_in_memory

Number of documents in memory index document last, maxNumber of documents in memory index

content.proton.documentdb.disk_usage

The total disk usage (in bytes) for this document db byte lastThe total disk usage (in bytes) for this document db

content.proton.documentdb.memory_usage.allocated_bytes

The number of allocated bytes byte maxThe number of allocated bytes

content.proton.documentdb.heart_beat_age

How long ago (in seconds) heart beat maintenace job was run second last, minHow long ago (in seconds) heart beat maintenace job was run

content.proton.docsum.docs

Total docsums returned document rateTotal docsums returned

content.proton.docsum.latency

Docsum request latency millisecond count, max, sumDocsum request latency

content.proton.search_protocol.query.latency

Query request latency (seconds) second count, max, sumQuery request latency (seconds)

content.proton.search_protocol.query.request_size

Query request size (network bytes) byte count, max, sumQuery request size (network bytes)

content.proton.search_protocol.query.reply_size

Query reply size (network bytes) byte count, max, sumQuery reply size (network bytes)

content.proton.search_protocol.docsum.latency

Docsum request latency (seconds) second average, count, max, sumDocsum request latency (seconds)

content.proton.search_protocol.docsum.request_size

Docsum request size (network bytes) byte count, max, sumDocsum request size (network bytes)

content.proton.search_protocol.docsum.reply_size

Docsum reply size (network bytes) byte count, max, sumDocsum reply size (network bytes)

content.proton.search_protocol.docsum.requested_documents

Total requested document summaries document count, max, sumTotal requested document summaries

content.proton.executor.proton.queuesize

Size of executor proton task queue task count, max, sumSize of executor proton task queue

content.proton.executor.proton.accepted

Number of executor proton accepted tasks task rateNumber of executor proton accepted tasks

content.proton.executor.proton.wakeups

Number of times a executor proton worker thread has been woken up wakeup rateNumber of times a executor proton worker thread has been woken up

content.proton.executor.proton.utilization

Ratio of time the executor proton worker threads has been active fraction count, max, sumRatio of time the executor proton worker threads has been active

content.proton.executor.flush.queuesize

Size of executor flush task queue task count, max, sumSize of executor flush task queue

content.proton.executor.flush.accepted

Number of accepted executor flush tasks task rateNumber of accepted executor flush tasks

content.proton.executor.flush.wakeups

Number of times a executor flush worker thread has been woken up wakeup rateNumber of times a executor flush worker thread has been woken up

content.proton.executor.flush.utilization

Ratio of time the executor flush worker threads has been active fraction count, max, sumRatio of time the executor flush worker threads has been active

content.proton.executor.match.queuesize

Size of executor match task queue task count, max, sumSize of executor match task queue

content.proton.executor.match.accepted

Number of accepted executor match tasks task rateNumber of accepted executor match tasks

content.proton.executor.match.wakeups

Number of times a executor match worker thread has been woken up wakeup rateNumber of times a executor match worker thread has been woken up

content.proton.executor.match.utilization

Ratio of time the executor match worker threads has been active fraction count, max, sumRatio of time the executor match worker threads has been active

content.proton.executor.docsum.queuesize

Size of executor docsum task queue task count, max, sumSize of executor docsum task queue

content.proton.executor.docsum.accepted

Number of executor accepted docsum tasks task rateNumber of executor accepted docsum tasks

content.proton.executor.docsum.wakeups

Number of times a executor docsum worker thread has been woken up wakeup rateNumber of times a executor docsum worker thread has been woken up

content.proton.executor.docsum.utilization

Ratio of time the executor docsum worker threads has been active fraction count, max, sumRatio of time the executor docsum worker threads has been active

content.proton.executor.shared.queuesize

Size of executor shared task queue task count, max, sumSize of executor shared task queue

content.proton.executor.shared.accepted

Number of executor shared accepted tasks task rateNumber of executor shared accepted tasks

content.proton.executor.shared.wakeups

Number of times a executor shared worker thread has been woken up wakeup rateNumber of times a executor shared worker thread has been woken up

content.proton.executor.shared.utilization

Ratio of time the executor shared worker threads has been active fraction count, max, sumRatio of time the executor shared worker threads has been active

content.proton.executor.warmup.queuesize

Size of executor warmup task queue task count, max, sumSize of executor warmup task queue

content.proton.executor.warmup.accepted

Number of accepted executor warmup tasks task rateNumber of accepted executor warmup tasks

content.proton.executor.warmup.wakeups

Number of times a warmup executor worker thread has been woken up wakeup rateNumber of times a warmup executor worker thread has been woken up

content.proton.executor.warmup.utilization

Ratio of time the executor warmup worker threads has been active fraction count, max, sumRatio of time the executor warmup worker threads has been active

content.proton.executor.field_writer.queuesize

Size of executor field writer task queue task count, max, sumSize of executor field writer task queue

content.proton.executor.field_writer.accepted

Number of accepted executor field writer tasks task rateNumber of accepted executor field writer tasks

content.proton.executor.field_writer.wakeups

Number of times a executor field writer worker thread has been woken up wakeup rateNumber of times a executor field writer worker thread has been woken up

content.proton.executor.field_writer.utilization

Ratio of time the executor fieldwriter worker threads has been active fraction count, max, sumRatio of time the executor fieldwriter worker threads has been active

content.proton.executor.field_writer.saturation

Ratio indicating the max saturation of underlying worker threads. A higher saturation than utilization indicates a bottleneck in one of the worker threads. fraction count, max, sumRatio indicating the max saturation of underlying worker threads. A higher saturation than utilization indicates a bottleneck in one of the worker threads.

content.proton.documentdb.job.total

The job load average total of all job metrics fraction averageThe job load average total of all job metrics

content.proton.documentdb.job.attribute_flush

Flushing of attribute vector(s) to disk fraction averageFlushing of attribute vector(s) to disk

content.proton.documentdb.job.memory_index_flush

Flushing of memory index to disk fraction averageFlushing of memory index to disk

content.proton.documentdb.job.disk_index_fusion

Fusion of disk indexes fraction averageFusion of disk indexes

content.proton.documentdb.job.document_store_flush

Flushing of document store to disk fraction averageFlushing of document store to disk

content.proton.documentdb.job.document_store_compact

Compaction of document store on disk fraction averageCompaction of document store on disk

content.proton.documentdb.job.bucket_move

Moving of buckets between 'ready' and 'notready' sub databases fraction averageMoving of buckets between 'ready' and 'notready' sub databases

content.proton.documentdb.job.lid_space_compact

Compaction of lid space in document meta store and attribute vectors fraction averageCompaction of lid space in document meta store and attribute vectors

content.proton.documentdb.job.removed_documents_prune

Pruning of removed documents in 'removed' sub database fraction averagePruning of removed documents in 'removed' sub database

content.proton.documentdb.threading_service.master.queuesize

Size of threading service master task queue task count, max, sumSize of threading service master task queue

content.proton.documentdb.threading_service.master.accepted

Number of accepted threading service master tasks task rateNumber of accepted threading service master tasks

content.proton.documentdb.threading_service.master.wakeups

Number of times a threading service master worker thread has been woken up wakeup rateNumber of times a threading service master worker thread has been woken up

content.proton.documentdb.threading_service.master.utilization

Ratio of time the threading service master worker threads has been active fraction count, max, sumRatio of time the threading service master worker threads has been active

content.proton.documentdb.threading_service.index.queuesize

Size of threading service index task queue task count, max, sumSize of threading service index task queue

content.proton.documentdb.threading_service.index.accepted

Number of accepted threading service index tasks task rateNumber of accepted threading service index tasks

content.proton.documentdb.threading_service.index.wakeups

Number of times a threading service index worker thread has been woken up wakeup rateNumber of times a threading service index worker thread has been woken up

content.proton.documentdb.threading_service.index.utilization

Ratio of time the threading service index worker threads has been active fraction count, max, sumRatio of time the threading service index worker threads has been active

content.proton.documentdb.threading_service.summary.queuesize

Size of threading service summary task queue task count, max, sumSize of threading service summary task queue

content.proton.documentdb.threading_service.summary.accepted

Number of accepted threading service summary tasks task rateNumber of accepted threading service summary tasks

content.proton.documentdb.threading_service.summary.wakeups

Number of times a threading service summary worker thread has been woken up wakeup rateNumber of times a threading service summary worker thread has been woken up

content.proton.documentdb.threading_service.summary.utilization

Ratio of time the threading service summary worker threads has been active fraction count, max, sumRatio of time the threading service summary worker threads has been active

content.proton.documentdb.ready.lid_space.lid_bloat_factor

The bloat factor of this lid space, indicating the total amount of holes in the allocated lid space ((lid_limit - used_lids) / lid_limit) fraction averageThe bloat factor of this lid space, indicating the total amount of holes in the allocated lid space ((lid_limit - used_lids) / lid_limit)

content.proton.documentdb.ready.lid_space.lid_fragmentation_factor

The fragmentation factor of this lid space, indicating the amount of holes in the currently used part of the lid space ((highest_used_lid - used_lids) / highest_used_lid) fraction averageThe fragmentation factor of this lid space, indicating the amount of holes in the currently used part of the lid space ((highest_used_lid - used_lids) / highest_used_lid)

content.proton.documentdb.ready.lid_space.lid_limit

The size of the allocated lid space documentid last, maxThe size of the allocated lid space

content.proton.documentdb.ready.lid_space.highest_used_lid

The highest used lid documentid last, maxThe highest used lid

content.proton.documentdb.ready.lid_space.used_lids

The number of lids used documentid last, maxThe number of lids used

content.proton.documentdb.notready.lid_space.lid_bloat_factor

The bloat factor of this lid space, indicating the total amount of holes in the allocated lid space ((lid_limit - used_lids) / lid_limit) fraction averageThe bloat factor of this lid space, indicating the total amount of holes in the allocated lid space ((lid_limit - used_lids) / lid_limit)

content.proton.documentdb.notready.lid_space.lid_fragmentation_factor

The fragmentation factor of this lid space, indicating the amount of holes in the currently used part of the lid space ((highest_used_lid - used_lids) / highest_used_lid) fraction averageThe fragmentation factor of this lid space, indicating the amount of holes in the currently used part of the lid space ((highest_used_lid - used_lids) / highest_used_lid)

content.proton.documentdb.notready.lid_space.lid_limit

The size of the allocated lid space documentid last, maxThe size of the allocated lid space

content.proton.documentdb.notready.lid_space.highest_used_lid

The highest used lid documentid last, maxThe highest used lid

content.proton.documentdb.notready.lid_space.used_lids

The number of lids used documentid last, maxThe number of lids used

content.proton.documentdb.removed.lid_space.lid_bloat_factor

The bloat factor of this lid space, indicating the total amount of holes in the allocated lid space ((lid_limit - used_lids) / lid_limit) fraction averageThe bloat factor of this lid space, indicating the total amount of holes in the allocated lid space ((lid_limit - used_lids) / lid_limit)

content.proton.documentdb.removed.lid_space.lid_fragmentation_factor

The fragmentation factor of this lid space, indicating the amount of holes in the currently used part of the lid space ((highest_used_lid - used_lids) / highest_used_lid) fraction averageThe fragmentation factor of this lid space, indicating the amount of holes in the currently used part of the lid space ((highest_used_lid - used_lids) / highest_used_lid)

content.proton.documentdb.removed.lid_space.lid_limit

The size of the allocated lid space documentid last, maxThe size of the allocated lid space

content.proton.documentdb.removed.lid_space.highest_used_lid

The highest used lid documentid last, maxThe highest used lid

content.proton.documentdb.removed.lid_space.used_lids

The number of lids used documentid last, maxThe number of lids used

content.proton.documentdb.bucket_move.buckets_pending

The number of buckets left to move bucket last, max, sumThe number of buckets left to move

content.proton.resource_usage.disk

The relative amount of disk used by this content node (transient usage not included, value in the range [0, 1]). Same value as reported to the cluster controller fraction averageThe relative amount of disk used by this content node (transient usage not included, value in the range [0, 1]). Same value as reported to the cluster controller

content.proton.resource_usage.disk_usage.total

The total relative amount of disk used by this content node (value in the range [0, 1]) fraction maxThe total relative amount of disk used by this content node (value in the range [0, 1])

content.proton.resource_usage.disk_usage.total_utilization

The relative amount of disk used compared to the content node disk resource limit fraction maxThe relative amount of disk used compared to the content node disk resource limit

content.proton.resource_usage.disk_usage.transient

The relative amount of transient disk used by this content node (value in the range [0, 1]) fraction maxThe relative amount of transient disk used by this content node (value in the range [0, 1])

content.proton.resource_usage.memory

The relative amount of memory used by this content node (transient usage not included, value in the range [0, 1]). Same value as reported to the cluster controller fraction averageThe relative amount of memory used by this content node (transient usage not included, value in the range [0, 1]). Same value as reported to the cluster controller

content.proton.resource_usage.memory_usage.total

The total relative amount of memory used by this content node (value in the range [0, 1]) fraction maxThe total relative amount of memory used by this content node (value in the range [0, 1])

content.proton.resource_usage.memory_usage.total_utilization

The relative amount of memory used compared to the content node memory resource limit fraction maxThe relative amount of memory used compared to the content node memory resource limit

content.proton.resource_usage.memory_usage.transient

The relative amount of transient memory used by this content node (value in the range [0, 1]) fraction maxThe relative amount of transient memory used by this content node (value in the range [0, 1])

content.proton.resource_usage.memory_mappings

The number of memory mapped files file maxThe number of memory mapped files

content.proton.resource_usage.open_file_descriptors

The number of open files file maxThe number of open files

content.proton.resource_usage.feeding_blocked

Whether feeding is blocked due to resource limits being reached (value is either 0 or 1) binary last, maxWhether feeding is blocked due to resource limits being reached (value is either 0 or 1)

content.proton.resource_usage.malloc_arena

Size of malloc arena byte maxSize of malloc arena

content.proton.documentdb.attribute.resource_usage.address_space

The max relative address space used among components in all attribute vectors in this document db (value in the range [0, 1]) fraction maxThe max relative address space used among components in all attribute vectors in this document db (value in the range [0, 1])

content.proton.documentdb.attribute.resource_usage.feeding_blocked

Whether feeding is blocked due to attribute resource limits being reached (value is either 0 or 1) binary maxWhether feeding is blocked due to attribute resource limits being reached (value is either 0 or 1)

content.proton.resource_usage.cpu_util.setup

cpu used by system init and (re-)configuration fraction count, max, sumcpu used by system init and (re-)configuration

content.proton.resource_usage.cpu_util.read

cpu used by reading data from the system fraction count, max, sumcpu used by reading data from the system

content.proton.resource_usage.cpu_util.write

cpu used by writing data to the system fraction count, max, sumcpu used by writing data to the system

content.proton.resource_usage.cpu_util.compact

cpu used by internal data re-structuring fraction count, max, sumcpu used by internal data re-structuring

content.proton.resource_usage.cpu_util.other

cpu used by work not classified as a specific category fraction count, max, sumcpu used by work not classified as a specific category

content.proton.transactionlog.entries

The current number of entries in the transaction log record averageThe current number of entries in the transaction log

content.proton.transactionlog.disk_usage

The disk usage (in bytes) of the transaction log byte averageThe disk usage (in bytes) of the transaction log

content.proton.transactionlog.replay_time

The replay time (in seconds) of the transaction log during start-up second last, maxThe replay time (in seconds) of the transaction log during start-up

content.proton.documentdb.ready.document_store.disk_usage

Disk space usage in bytes byte averageDisk space usage in bytes

content.proton.documentdb.ready.document_store.disk_bloat

Disk space bloat in bytes byte averageDisk space bloat in bytes

content.proton.documentdb.ready.document_store.max_bucket_spread

Max bucket spread in underlying files (sum(unique buckets in each chunk)/unique buckets in file) fraction averageMax bucket spread in underlying files (sum(unique buckets in each chunk)/unique buckets in file)

content.proton.documentdb.ready.document_store.memory_usage.allocated_bytes

The number of allocated bytes byte averageThe number of allocated bytes

content.proton.documentdb.ready.document_store.memory_usage.used_bytes

The number of used bytes (<= allocated_bytes) byte averageThe number of used bytes (<= allocated_bytes)

content.proton.documentdb.ready.document_store.memory_usage.onhold_bytes

The number of bytes on hold byte averageThe number of bytes on hold

content.proton.documentdb.notready.document_store.disk_usage

Disk space usage in bytes byte averageDisk space usage in bytes

content.proton.documentdb.notready.document_store.disk_bloat

Disk space bloat in bytes byte averageDisk space bloat in bytes

content.proton.documentdb.notready.document_store.max_bucket_spread

Max bucket spread in underlying files (sum(unique buckets in each chunk)/unique buckets in file) fraction averageMax bucket spread in underlying files (sum(unique buckets in each chunk)/unique buckets in file)

content.proton.documentdb.notready.document_store.memory_usage.allocated_bytes

The number of allocated bytes byte averageThe number of allocated bytes

content.proton.documentdb.notready.document_store.memory_usage.used_bytes

The number of used bytes (<= allocated_bytes) byte averageThe number of used bytes (<= allocated_bytes)

content.proton.documentdb.notready.document_store.memory_usage.dead_bytes

The number of dead bytes (<= used_bytes) byte averageThe number of dead bytes (<= used_bytes)

content.proton.documentdb.notready.document_store.memory_usage.onhold_bytes

The number of bytes on hold byte averageThe number of bytes on hold

content.proton.documentdb.removed.document_store.disk_usage

Disk space usage in bytes byte averageDisk space usage in bytes

content.proton.documentdb.removed.document_store.disk_bloat

Disk space bloat in bytes byte averageDisk space bloat in bytes

content.proton.documentdb.removed.document_store.max_bucket_spread

Max bucket spread in underlying files (sum(unique buckets in each chunk)/unique buckets in file) fraction averageMax bucket spread in underlying files (sum(unique buckets in each chunk)/unique buckets in file)

content.proton.documentdb.removed.document_store.memory_usage.allocated_bytes

The number of allocated bytes byte averageThe number of allocated bytes

content.proton.documentdb.removed.document_store.memory_usage.used_bytes

The number of used bytes (<= allocated_bytes) byte averageThe number of used bytes (<= allocated_bytes)

content.proton.documentdb.removed.document_store.memory_usage.dead_bytes

The number of dead bytes (<= used_bytes) byte averageThe number of dead bytes (<= used_bytes)

content.proton.documentdb.removed.document_store.memory_usage.onhold_bytes

The number of bytes on hold byte averageThe number of bytes on hold

content.proton.documentdb.ready.document_store.cache.memory_usage

Memory usage of the cache (in bytes) byte averageMemory usage of the cache (in bytes)

content.proton.documentdb.ready.document_store.cache.hit_rate

Rate of hits in the cache compared to number of lookups fraction averageRate of hits in the cache compared to number of lookups

content.proton.documentdb.ready.document_store.cache.lookups

Number of lookups in the cache (hits + misses) operation rateNumber of lookups in the cache (hits + misses)

content.proton.documentdb.ready.document_store.cache.invalidations

Number of invalidations (erased elements) in the cache. operation rateNumber of invalidations (erased elements) in the cache.

content.proton.documentdb.notready.document_store.cache.memory_usage

Memory usage of the cache (in bytes) byte averageMemory usage of the cache (in bytes)

content.proton.documentdb.notready.document_store.cache.hit_rate

Rate of hits in the cache compared to number of lookups fraction averageRate of hits in the cache compared to number of lookups

content.proton.documentdb.notready.document_store.cache.lookups

Number of lookups in the cache (hits + misses) operation rateNumber of lookups in the cache (hits + misses)

content.proton.documentdb.notready.document_store.cache.invalidations

Number of invalidations (erased elements) in the cache. operation rateNumber of invalidations (erased elements) in the cache.

content.proton.documentdb.ready.attribute.memory_usage.allocated_bytes

The number of allocated bytes byte averageThe number of allocated bytes

content.proton.documentdb.ready.attribute.memory_usage.used_bytes

The number of used bytes (<= allocated_bytes) byte averageThe number of used bytes (<= allocated_bytes)

content.proton.documentdb.ready.attribute.memory_usage.dead_bytes

The number of dead bytes (<= used_bytes) byte averageThe number of dead bytes (<= used_bytes)

content.proton.documentdb.ready.attribute.memory_usage.onhold_bytes

The number of bytes on hold byte averageThe number of bytes on hold

content.proton.documentdb.ready.attribute.disk_usage

Disk space usage (in bytes) of the flushed snapshot of this attribute for this document type byte averageDisk space usage (in bytes) of the flushed snapshot of this attribute for this document type

content.proton.documentdb.notready.attribute.memory_usage.allocated_bytes

The number of allocated bytes byte averageThe number of allocated bytes

content.proton.documentdb.notready.attribute.memory_usage.used_bytes

The number of used bytes (<= allocated_bytes) byte averageThe number of used bytes (<= allocated_bytes)

content.proton.documentdb.notready.attribute.memory_usage.dead_bytes

The number of dead bytes (<= used_bytes) byte averageThe number of dead bytes (<= used_bytes)

content.proton.documentdb.notready.attribute.memory_usage.onhold_bytes

The number of bytes on hold byte averageThe number of bytes on hold

content.proton.index.cache.postinglist.memory_usage

Memory usage of the cache (in bytes). Contains disk index posting list files across all document types byte averageMemory usage of the cache (in bytes). Contains disk index posting list files across all document types

content.proton.index.cache.postinglist.hit_rate

Rate of hits in the cache compared to number of lookups. Contains disk index posting list files across all document types fraction averageRate of hits in the cache compared to number of lookups. Contains disk index posting list files across all document types

content.proton.index.cache.postinglist.lookups

Number of lookups in the cache (hits + misses). Contains disk index posting list files across all document types operation rateNumber of lookups in the cache (hits + misses). Contains disk index posting list files across all document types

content.proton.index.cache.postinglist.invalidations

Number of invalidations (erased elements) in the cache. Contains disk index posting list files across all document types operation rateNumber of invalidations (erased elements) in the cache. Contains disk index posting list files across all document types

content.proton.index.cache.bitvector.memory_usage

Memory usage of the cache (in bytes). Contains disk index bitvector files across all document types byte averageMemory usage of the cache (in bytes). Contains disk index bitvector files across all document types

content.proton.index.cache.bitvector.hit_rate

Rate of hits in the cache compared to number of lookups. Contains disk index bitvector files across all document types fraction averageRate of hits in the cache compared to number of lookups. Contains disk index bitvector files across all document types

content.proton.index.cache.bitvector.lookups

Number of lookups in the cache (hits + misses). Contains disk index bitvector files across all document types operation rateNumber of lookups in the cache (hits + misses). Contains disk index bitvector files across all document types

content.proton.index.cache.bitvector.invalidations

Number of invalidations (erased elements) in the cache. Contains disk index bitvector files across all document types operation rateNumber of invalidations (erased elements) in the cache. Contains disk index bitvector files across all document types

content.proton.documentdb.index.memory_usage.allocated_bytes

The number of allocated bytes for the memory index for this document type byte averageThe number of allocated bytes for the memory index for this document type

content.proton.documentdb.index.memory_usage.used_bytes

The number of used bytes (<= allocated_bytes) for the memory index for this document type byte averageThe number of used bytes (<= allocated_bytes) for the memory index for this document type

content.proton.documentdb.index.memory_usage.dead_bytes

The number of dead bytes (<= used_bytes) for the memory index for this document type byte averageThe number of dead bytes (<= used_bytes) for the memory index for this document type

content.proton.documentdb.index.memory_usage.onhold_bytes

The number of bytes on hold for the memory index for this document type byte averageThe number of bytes on hold for the memory index for this document type

content.proton.documentdb.index.io.search.read_bytes

Bytes read from disk index posting list and bitvector files as part of search for this document type byte count, sumBytes read from disk index posting list and bitvector files as part of search for this document type

content.proton.documentdb.index.io.search.cached_read_bytes

Bytes read from cached disk index posting list and bitvector files as part of search for this document type byte count, sumBytes read from cached disk index posting list and bitvector files as part of search for this document type

content.proton.documentdb.ready.index.disk_usage

Disk space usage (in bytes) of this index field in all disk indexes for this document type byte averageDisk space usage (in bytes) of this index field in all disk indexes for this document type

content.proton.documentdb.matching.queries

Number of queries executed query rateNumber of queries executed

content.proton.documentdb.matching.soft_doomed_queries

Number of queries hitting the soft timeout query rateNumber of queries hitting the soft timeout

content.proton.documentdb.matching.query_latency

Total average latency (sec) when matching and ranking a query second count, max, sumTotal average latency (sec) when matching and ranking a query

content.proton.documentdb.matching.query_setup_time

Average time (sec) spent setting up and tearing down queries second count, max, sumAverage time (sec) spent setting up and tearing down queries

content.proton.documentdb.matching.docs_matched

Number of documents matched document count, rateNumber of documents matched

content.proton.documentdb.matching.rank_profile.queries

Number of queries executed query rateNumber of queries executed

content.proton.documentdb.matching.rank_profile.soft_doomed_queries

Number of queries hitting the soft timeout query rateNumber of queries hitting the soft timeout

content.proton.documentdb.matching.rank_profile.soft_doom_factor

Factor used to compute soft-timeout fraction count, max, min, sumFactor used to compute soft-timeout

content.proton.documentdb.matching.rank_profile.query_latency

Total average latency (sec) when matching and ranking a query second count, max, sumTotal average latency (sec) when matching and ranking a query

content.proton.documentdb.matching.rank_profile.query_setup_time

Average time (sec) spent setting up and tearing down queries second count, max, sumAverage time (sec) spent setting up and tearing down queries

content.proton.documentdb.matching.rank_profile.grouping_time

Average time (sec) spent on grouping second count, max, sumAverage time (sec) spent on grouping

content.proton.documentdb.matching.rank_profile.rerank_time

Average time (sec) spent on 2nd phase ranking second count, max, sumAverage time (sec) spent on 2nd phase ranking

content.proton.documentdb.matching.rank_profile.docs_matched

Number of documents matched document count, rateNumber of documents matched

content.proton.documentdb.matching.rank_profile.limited_queries

Number of queries limited in match phase query rateNumber of queries limited in match phase

content.proton.documentdb.feeding.commit.operations

Number of operations included in a commit operation count, max, rate, sumNumber of operations included in a commit

content.proton.documentdb.feeding.commit.latency

Latency for commit in seconds second count, max, sumLatency for commit in seconds
@@ -2549,32 +2549,32 @@

SearchNode Metrics

Sentinel Metrics

- + - + - + - + - +
NameDescriptionUnitSuffixes
NameUnitSuffixesDescription

sentinel.restarts

Number of service restarts done by the sentinel restart countNumber of service restarts done by the sentinel

sentinel.totalRestarts

Total number of service restarts done by the sentinel since the sentinel was started restart last, max, sumTotal number of service restarts done by the sentinel since the sentinel was started

sentinel.uptime

Time the sentinel has been running second lastTime the sentinel has been running

sentinel.running

Number of services the sentinel has running currently instance count, lastNumber of services the sentinel has running currently
@@ -2582,20 +2582,20 @@

Sentinel Metrics

Slobrok Metrics

- + - + - +
NameDescriptionUnitSuffixes
NameUnitSuffixesDescription

slobrok.heartbeats.failed

Number of heartbeat requests failed request countNumber of heartbeat requests failed

slobrok.missing.consensus

Number of seconds without full consensus with all other brokers second countNumber of seconds without full consensus with all other brokers
@@ -2603,488 +2603,488 @@

Slobrok Metrics

Storage Metrics

- + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - +
NameDescriptionUnitSuffixes
NameUnitSuffixesDescription

vds.datastored.alldisks.buckets

Number of buckets managed bucket averageNumber of buckets managed

vds.datastored.alldisks.docs

Number of documents stored document averageNumber of documents stored

vds.datastored.alldisks.bytes

Number of bytes stored byte averageNumber of bytes stored

vds.visitor.allthreads.averagevisitorlifetime

Average lifetime of a visitor millisecond count, max, sumAverage lifetime of a visitor

vds.visitor.allthreads.averagequeuewait

Average time an operation spends in input queue. millisecond count, max, sumAverage time an operation spends in input queue.

vds.visitor.allthreads.queuesize

Size of input message queue. operation count, max, sumSize of input message queue.

vds.visitor.allthreads.completed

Number of visitors completed operation rateNumber of visitors completed

vds.visitor.allthreads.created

Number of visitors created. operation rateNumber of visitors created.

vds.visitor.allthreads.failed

Number of visitors failed operation rateNumber of visitors failed

vds.visitor.allthreads.averagemessagesendtime

Average time it takes for messages to be sent to their target (and be replied to) millisecond count, max, sumAverage time it takes for messages to be sent to their target (and be replied to)

vds.visitor.allthreads.averageprocessingtime

Average time used to process visitor requests millisecond count, max, sumAverage time used to process visitor requests

vds.filestor.queuesize

Size of input message queue. operation count, max, sumSize of input message queue.

vds.filestor.averagequeuewait

Average time an operation spends in input queue. millisecond count, max, sumAverage time an operation spends in input queue.

vds.filestor.active_operations.size

Number of concurrent active operations operation count, max, sumNumber of concurrent active operations

vds.filestor.active_operations.latency

Latency (in ms) for completed operations millisecond count, max, sumLatency (in ms) for completed operations

vds.filestor.throttle_window_size

Current size of async operation throttler window size operation count, max, sumCurrent size of async operation throttler window size

vds.filestor.throttle_waiting_threads

Number of threads waiting to acquire a throttle token thread count, max, sumNumber of threads waiting to acquire a throttle token

vds.filestor.throttle_active_tokens

Current number of active throttle tokens instance count, max, sumCurrent number of active throttle tokens

vds.filestor.allthreads.mergemetadatareadlatency

Time spent in a merge step to check metadata of current node to see what data it has. millisecond count, max, sumTime spent in a merge step to check metadata of current node to see what data it has.

vds.filestor.allthreads.mergedatareadlatency

Time spent in a merge step to read data other nodes need. millisecond count, max, sumTime spent in a merge step to read data other nodes need.

vds.filestor.allthreads.mergedatawritelatency

Time spent in a merge step to write data needed to current node. millisecond count, max, sumTime spent in a merge step to write data needed to current node.

vds.filestor.allthreads.merge_put_latency

Latency of individual puts that are part of merge operations millisecond count, max, sumLatency of individual puts that are part of merge operations

vds.filestor.allthreads.merge_remove_latency

Latency of individual removes that are part of merge operations millisecond count, max, sumLatency of individual removes that are part of merge operations

vds.filestor.allstripes.throttled_rpc_direct_dispatches

Number of times an RPC thread could not directly dispatch an async operation directly to Proton because it was disallowed by the throttle policy instance rateNumber of times an RPC thread could not directly dispatch an async operation directly to Proton because it was disallowed by the throttle policy

vds.filestor.allstripes.throttled_persistence_thread_polls

Number of times a persistence thread could not immediately dispatch a queued async operation because it was disallowed by the throttle policy instance rateNumber of times a persistence thread could not immediately dispatch a queued async operation because it was disallowed by the throttle policy

vds.filestor.allstripes.timeouts_waiting_for_throttle_token

Number of times a persistence thread timed out waiting for an available throttle policy token instance rateNumber of times a persistence thread timed out waiting for an available throttle policy token

vds.filestor.allthreads.put.count

Number of requests processed. operation rateNumber of requests processed.

vds.filestor.allthreads.put.failed

Number of failed requests. operation rateNumber of failed requests.

vds.filestor.allthreads.put.test_and_set_failed

Number of operations that were skipped due to a test-and-set condition not met operation rateNumber of operations that were skipped due to a test-and-set condition not met

vds.filestor.allthreads.put.latency

Latency of successful requests. millisecond count, max, sumLatency of successful requests.

vds.filestor.allthreads.put.request_size

Size of requests, in bytes byte count, max, sumSize of requests, in bytes

vds.filestor.allthreads.remove.count

Number of requests processed. operation rateNumber of requests processed.

vds.filestor.allthreads.remove.failed

Number of failed requests. operation rateNumber of failed requests.

vds.filestor.allthreads.remove.test_and_set_failed

Number of operations that were skipped due to a test-and-set condition not met operation rateNumber of operations that were skipped due to a test-and-set condition not met

vds.filestor.allthreads.remove.latency

Latency of successful requests. millisecond count, max, sumLatency of successful requests.

vds.filestor.allthreads.remove.request_size

Size of requests, in bytes byte count, max, sumSize of requests, in bytes

vds.filestor.allthreads.get.count

Number of requests processed. operation rateNumber of requests processed.

vds.filestor.allthreads.get.failed

Number of failed requests. operation rateNumber of failed requests.

vds.filestor.allthreads.get.latency

Latency of successful requests. millisecond count, max, sumLatency of successful requests.

vds.filestor.allthreads.get.request_size

Size of requests, in bytes byte count, max, sumSize of requests, in bytes

vds.filestor.allthreads.update.count

Number of requests processed. request rateNumber of requests processed.

vds.filestor.allthreads.update.failed

Number of failed requests. request rateNumber of failed requests.

vds.filestor.allthreads.update.test_and_set_failed

Number of requests that were skipped due to a test-and-set condition not met request rateNumber of requests that were skipped due to a test-and-set condition not met

vds.filestor.allthreads.update.latency

Latency of successful requests. millisecond count, max, sumLatency of successful requests.

vds.filestor.allthreads.update.request_size

Size of requests, in bytes byte count, max, sumSize of requests, in bytes

vds.filestor.allthreads.createiterator.count

Number of requests processed. request rateNumber of requests processed.

vds.filestor.allthreads.createiterator.latency

Latency of successful requests. millisecond count, max, sumLatency of successful requests.

vds.filestor.allthreads.visit.count

Number of requests processed. request rateNumber of requests processed.

vds.filestor.allthreads.visit.latency

Latency of successful requests. millisecond count, max, sumLatency of successful requests.

vds.filestor.allthreads.remove_location.count

Number of requests processed. request rateNumber of requests processed.

vds.filestor.allthreads.remove_location.latency

Latency of successful requests. millisecond count, max, sumLatency of successful requests.

vds.filestor.allthreads.splitbuckets.count

Number of requests processed. request rateNumber of requests processed.

vds.filestor.allthreads.joinbuckets.count

Number of requests processed. request rateNumber of requests processed.

vds.filestor.allthreads.deletebuckets.count

Number of requests processed. request rateNumber of requests processed.

vds.filestor.allthreads.deletebuckets.failed

Number of failed requests. request rateNumber of failed requests.

vds.filestor.allthreads.deletebuckets.latency

Latency of successful requests. millisecond count, max, sumLatency of successful requests.

vds.filestor.allthreads.remove_by_gid.count

Number of requests processed. request rateNumber of requests processed.

vds.filestor.allthreads.remove_by_gid.failed

Number of failed requests. request rateNumber of failed requests.

vds.filestor.allthreads.remove_by_gid.latency

Latency of successful requests. millisecond count, max, sumLatency of successful requests.

vds.filestor.allthreads.setbucketstates.count

Number of requests processed. request rateNumber of requests processed.

vds.mergethrottler.averagequeuewaitingtime

Time merges spent in the throttler queue millisecond count, max, sumTime merges spent in the throttler queue

vds.mergethrottler.queuesize

Length of merge queue instance count, max, sumLength of merge queue

vds.mergethrottler.active_window_size

Number of merges active within the pending window size instance count, max, sumNumber of merges active within the pending window size

vds.mergethrottler.estimated_merge_memory_usage

An estimated upper bound of the memory usage (in bytes) of the merges currently in the active window byte count, max, sumAn estimated upper bound of the memory usage (in bytes) of the merges currently in the active window

vds.mergethrottler.bounced_due_to_back_pressure

Number of merges bounced due to resource exhaustion back-pressure instance rateNumber of merges bounced due to resource exhaustion back-pressure

vds.mergethrottler.locallyexecutedmerges.ok

The number of successful merges for 'locallyexecutedmerges' instance rateThe number of successful merges for 'locallyexecutedmerges'

vds.mergethrottler.mergechains.ok

The number of successful merges for 'mergechains' operation rateThe number of successful merges for 'mergechains'

vds.mergethrottler.mergechains.failures.busy

The number of merges that failed because the storage node was busy operation rateThe number of merges that failed because the storage node was busy

vds.mergethrottler.mergechains.failures.total

Sum of all failures operation rateSum of all failures

vds.server.network.tls-handshakes-failed

Number of client or server connection attempts that failed during TLS handshaking operation countNumber of client or server connection attempts that failed during TLS handshaking

vds.server.network.peer-authorization-failures

Number of TLS connection attempts failed due to bad or missing peer certificate credentials failure countNumber of TLS connection attempts failed due to bad or missing peer certificate credentials

vds.server.network.client.tls-connections-established

Number of secure mTLS connections established connection countNumber of secure mTLS connections established

vds.server.network.server.tls-connections-established

Number of secure mTLS connections established connection countNumber of secure mTLS connections established

vds.server.network.client.insecure-connections-established

Number of insecure (plaintext) connections established connection countNumber of insecure (plaintext) connections established

vds.server.network.server.insecure-connections-established

Number of insecure (plaintext) connections established connection countNumber of insecure (plaintext) connections established

vds.server.network.tls-connections-broken

Number of TLS connections broken due to failures during frame encoding or decoding connection countNumber of TLS connections broken due to failures during frame encoding or decoding

vds.server.network.failed-tls-config-reloads

Number of times background reloading of TLS config has failed failure countNumber of times background reloading of TLS config has failed

vds.server.network.rpc-capability-checks-failed

Number of RPC operations that failed to due one or more missing capabilities failure countNumber of RPC operations that failed to due one or more missing capabilities

vds.server.network.status-capability-checks-failed

Number of status page operations that failed to due one or more missing capabilities failure countNumber of status page operations that failed to due one or more missing capabilities

vds.server.fnet.num-connections

Total number of connection objects connection countTotal number of connection objects