From 7374391042fe1ecb67e58e62ff4dccf77091ed11 Mon Sep 17 00:00:00 2001 From: nolouch Date: Wed, 15 Jan 2025 17:13:10 +0800 Subject: [PATCH 01/25] *: Add resource management topic and split docs Signed-off-by: nolouch --- TOC.md | 5 +- basic-features.md | 4 +- error-codes.md | 4 +- faq/sql-faq.md | 2 +- .../information-schema-runaway-watches.md | 2 +- releases/release-7.2.0.md | 2 +- releases/release-7.3.0.md | 2 +- releases/release-7.4.0.md | 2 +- releases/release-8.1.0.md | 2 +- releases/release-8.4.0.md | 4 +- .../sql-statement-alter-resource-group.md | 4 +- .../sql-statement-create-resource-group.md | 2 +- sql-statements/sql-statement-query-watch.md | 4 +- tidb-resource-control-background-tasks.md | 95 +++++++ tidb-resource-control-runaway-queries.md | 163 +++++++++++ tidb-resource-control.md | 255 +----------------- 16 files changed, 281 insertions(+), 271 deletions(-) create mode 100644 tidb-resource-control-background-tasks.md create mode 100644 tidb-resource-control-runaway-queries.md diff --git a/TOC.md b/TOC.md index 2a275b5dc1b69..f9dbc6c47494b 100644 --- a/TOC.md +++ b/TOC.md @@ -262,7 +262,10 @@ - [Primary-Secondary DR](/dr-secondary-cluster.md) - [Multi-Replica Cluster DR](/dr-multi-replica.md) - [BR-based DR](/dr-backup-restore.md) - - [Resource Control](/tidb-resource-control.md) + - Resource Manager + - [Achieve resource group limitation and flow control using Resource Control](/tidb-resource-control.md) + - [Runaway Queries with resource consumption exceeding expectations](/tidb-resource-control-runaway-queries.md) + - [Limit background task resource usage](/tidb-resource-control-background-tasks.md) - [Configure Time Zone](/configure-time-zone.md) - [Daily Checklist](/daily-check.md) - [Maintain TiFlash](/tiflash/maintain-tiflash.md) diff --git a/basic-features.md b/basic-features.md index 05aa8b0d0099c..c05e34734589b 100644 --- a/basic-features.md +++ b/basic-features.md @@ -259,8 +259,8 @@ You can try out TiDB features on [TiDB Playground](https://play.tidbcloud.com/?u | [Cross-cluster RawKV replication](/tikv-configuration-file.md#api-version-new-in-v610) | E | E | E | E | E | N | N | N | N | N | | [Green GC](/system-variables.md#tidb_gc_scan_lock_mode-new-in-v50) | E | E | E | E | E | E | E | E | E | E | | [Resource control](/tidb-resource-control.md) | Y | Y | Y | Y | N | N | N | N | N | N | -| [Runaway Queries management](/tidb-resource-control.md#manage-queries-that-consume-more-resources-than-expected-runaway-queries) | Y | Y | E | N | N | N | N | N | N | N | -| [Background tasks management](/tidb-resource-control.md#manage-background-tasks) | E | E | E | N | N | N | N | N | N | N | +| [Runaway Queries management](/tidb-resource-control-runaway-queries.md) | Y | Y | E | N | N | N | N | N | N | N | +| [Background tasks management](/tidb-resource-control-background-tasks.md) | E | E | E | N | N | N | N | N | N | N | | [TiFlash Disaggregated Storage and Compute Architecture and S3 Support](/tiflash/tiflash-disaggregated-and-s3.md) | Y | Y | Y | E | N | N | N | N | N | N | | [Selecting TiDB nodes for the Distributed eXecution Framework (DXF) tasks](/system-variables.md#tidb_service_scope-new-in-v740) | Y | Y | N | N | N | N | N | N | N | | PD Follower Proxy (controlled by [`tidb_enable_tso_follower_proxy`](/system-variables.md#tidb_enable_tso_follower_proxy-new-in-v530)) | Y | Y | Y | Y | Y | Y | Y | Y | N | N | diff --git a/error-codes.md b/error-codes.md index 3993d38f04893..7ee05f40212e2 100644 --- a/error-codes.md +++ b/error-codes.md @@ -508,11 +508,11 @@ TiDB is compatible with the error codes in MySQL, and in most cases returns the * Error Number: 8253 - The query stops because it meets the condition of a runaway query. See [Runaway Queries](/tidb-resource-control.md#manage-queries-that-consume-more-resources-than-expected-runaway-queries). + The query stops because it meets the condition of a runaway query. See [Runaway Queries](/tidb-resource-control-runaway-queries.md). * Error Number: 8254 - The query stops because it meets the quarantined watch condition of a runaway query. See [Runaway Queries](/tidb-resource-control.md#manage-queries-that-consume-more-resources-than-expected-runaway-queries). + The query stops because it meets the quarantined watch condition of a runaway query. See [Runaway Queries](/tidb-resource-control-runaway-queries.md). * Error Number: 8260 diff --git a/faq/sql-faq.md b/faq/sql-faq.md index 7ff41e470943c..50298bf695b55 100644 --- a/faq/sql-faq.md +++ b/faq/sql-faq.md @@ -32,7 +32,7 @@ In addition, you can also use the [SQL binding](/sql-plan-management.md#sql-bind ## How to prevent the execution of a particular SQL statement? -For TiDB v7.5.0 or later versions, you can use the [`QUERY WATCH`](/sql-statements/sql-statement-query-watch.md) statement to terminate specific SQL statements. For more details, see [Manage queries that consume more resources than expected (Runaway Queries)](/tidb-resource-control.md#query-watch-parameters). +For TiDB v7.5.0 or later versions, you can use the [`QUERY WATCH`](/sql-statements/sql-statement-query-watch.md) statement to terminate specific SQL statements. For more details, see [Manage queries that consume more resources than expected (Runaway Queries)](/tidb-resource-control-runaway-queries.md#query-watch-parameters). For versions earlier than TiDB v7.5.0, you can create [SQL bindings](/sql-plan-management.md#sql-binding) with the [`MAX_EXECUTION_TIME`](/optimizer-hints.md#max_execution_timen) hint to limit the execution time of a particular statement to a small value (for example, 1ms). In this way, the statement is terminated automatically by the threshold. diff --git a/information-schema/information-schema-runaway-watches.md b/information-schema/information-schema-runaway-watches.md index d3889e6b8fd4b..2777df7d078f1 100644 --- a/information-schema/information-schema-runaway-watches.md +++ b/information-schema/information-schema-runaway-watches.md @@ -5,7 +5,7 @@ summary: Learn the `RUNAWAY_WATCHES` INFORMATION_SCHEMA table. # RUNAWAY_WATCHES -The `RUNAWAY_WATCHES` table shows the watch list of runaway queries that consume more resources than expected. For more information, see [Runaway Queries](/tidb-resource-control.md#manage-queries-that-consume-more-resources-than-expected-runaway-queries). +The `RUNAWAY_WATCHES` table shows the watch list of runaway queries that consume more resources than expected. For more information, see [Runaway Queries](/tidb-resource-control-runaway-queries.md). > **Note:** > diff --git a/releases/release-7.2.0.md b/releases/release-7.2.0.md index af7949497d7eb..e0ffa205f982d 100644 --- a/releases/release-7.2.0.md +++ b/releases/release-7.2.0.md @@ -92,7 +92,7 @@ Quick access: [Quick start](https://docs.pingcap.com/tidb/v7.2/quick-start-with- Automatic management of queries that consume more resources than expected provides you with an effective means to quickly respond to unexpected query performance problems. This feature can reduce the impact of the problem on overall database performance, thereby improving database stability. - For more information, see [documentation](/tidb-resource-control.md#manage-queries-that-consume-more-resources-than-expected-runaway-queries). + For more information, see [documentation](/tidb-resource-control-runaway-queries.md). * Enhance the capability of creating a binding according to a historical execution plan [#39199](https://github.com/pingcap/tidb/issues/39199) @[qw4990](https://github.com/qw4990) diff --git a/releases/release-7.3.0.md b/releases/release-7.3.0.md index 258e7078012b8..abf376bd8a6c1 100644 --- a/releases/release-7.3.0.md +++ b/releases/release-7.3.0.md @@ -99,7 +99,7 @@ Quick access: [Quick start](https://docs.pingcap.com/tidb/v7.3/quick-start-with- This feature provides an effective intervention method for sudden performance issues in the database. For performance issues caused by queries, before identifying the root cause, this feature can quickly alleviate its impact on overall performance, thereby improving system service quality. - For more information, see [documentation](/tidb-resource-control.md#query-watch-parameters). + For more information, see [documentation](/tidb-resource-control-runaway-queries.md#query-watch-parameters). ### SQL diff --git a/releases/release-7.4.0.md b/releases/release-7.4.0.md index af131dc40c5e3..9643476f061f3 100644 --- a/releases/release-7.4.0.md +++ b/releases/release-7.4.0.md @@ -183,7 +183,7 @@ Quick access: [Quick start](https://docs.pingcap.com/tidb/v7.4/quick-start-with- By default, the task types that are marked as background tasks are empty, and the management of background tasks is disabled. This default behavior is the same as that of versions prior to TiDB v7.4.0. To manage background tasks, you need to manually modify the background task types of the `default` resource group. - For more information, see [documentation](/tidb-resource-control.md#manage-background-tasks). + For more information, see [documentation](/tidb-resource-control-background-tasks.md). * Lock statistics becomes generally available (GA) [#46351](https://github.com/pingcap/tidb/issues/46351) @[hi-rustin](https://github.com/Rustin170506) diff --git a/releases/release-8.1.0.md b/releases/release-8.1.0.md index 4266391f33755..6947bb6709839 100644 --- a/releases/release-8.1.0.md +++ b/releases/release-8.1.0.md @@ -103,7 +103,7 @@ Compared with the previous LTS 7.5.0, 8.1.0 includes new features, improvements, The automatic management capability of queries that consume more resources than expected provides users with an effective means to quickly mitigate the impact of query problems on overall performance before the root cause is identified, thereby improving the stability of the database. - For more information, see [documentation](/tidb-resource-control.md#manage-queries-that-consume-more-resources-than-expected-runaway-queries). + For more information, see [documentation](/tidb-resource-control-runaway-queries.md). ### SQL diff --git a/releases/release-8.4.0.md b/releases/release-8.4.0.md index 03dc85c02ad46..73e40d33abd02 100644 --- a/releases/release-8.4.0.md +++ b/releases/release-8.4.0.md @@ -150,7 +150,7 @@ Quick access: [Quick start](https://docs.pingcap.com/tidb/v8.4/quick-start-with- You can observe the corresponding fields (`RESOURCE_GROUP`, `MAX_REQUEST_UNIT_WRITE`, `MAX_REQUEST_UNIT_READ`, `MAX_PROCESSED_KEYS`) in the [Statement Summary Tables](/statement-summary-tables.md) to determine the condition values based on historical execution. - For more information, see [documentation](/tidb-resource-control.md#manage-queries-that-consume-more-resources-than-expected-runaway-queries). + For more information, see [documentation](/tidb-resource-control-runaway-queries.md). * Support switching resource groups for runaway queries [#54434](https://github.com/pingcap/tidb/issues/54434) @[JmPotato](https://github.com/JmPotato) @@ -170,7 +170,7 @@ Quick access: [Quick start](https://docs.pingcap.com/tidb/v8.4/quick-start-with- TiDB resource control can identify and lower the priority of background tasks. In certain scenarios, you might want to limit the resource consumption of background tasks, even when resources are available. Starting from v8.4.0, you can use the `UTILIZATION_LIMIT` parameter to set the maximum percentage of resources that background tasks can consume. Each node will keep the resource usage of all background tasks below this percentage. This feature enables precise control over resource consumption for background tasks, further enhancing cluster stability. - For more information, see [documentation](/tidb-resource-control.md#manage-background-tasks). + For more information, see [documentation](/tidb-resource-control-background-tasks.md). * Optimize the resource allocation strategy of resource groups [#50831](https://github.com/pingcap/tidb/issues/50831) @[nolouch](https://github.com/nolouch) diff --git a/sql-statements/sql-statement-alter-resource-group.md b/sql-statements/sql-statement-alter-resource-group.md index e94851e3c97d0..65bb0bbfcc948 100644 --- a/sql-statements/sql-statement-alter-resource-group.md +++ b/sql-statements/sql-statement-alter-resource-group.md @@ -85,8 +85,8 @@ TiDB supports the following `DirectResourceGroupOption`, where [Request Unit (RU | `RU_PER_SEC` | Rate of RU backfilling per second | `RU_PER_SEC = 500` indicates that this resource group is backfilled with 500 RUs per second | | `PRIORITY` | The absolute priority of tasks to be processed on TiKV | `PRIORITY = HIGH` indicates that the priority is high. If not specified, the default value is `MEDIUM`. | | `BURSTABLE` | If the `BURSTABLE` attribute is set, TiDB allows the corresponding resource group to use the available system resources when the quota is exceeded. | -| `QUERY_LIMIT` | When the query execution meets this condition, the query is identified as a runaway query and the corresponding action is executed. | `QUERY_LIMIT=(EXEC_ELAPSED='60s', ACTION=KILL, WATCH=EXACT DURATION='10m')` indicates that the query is identified as a runaway query when the execution time exceeds 60 seconds. The query is terminated. All SQL statements with the same SQL text will be terminated immediately in the coming 10 minutes. `QUERY_LIMIT=()` or `QUERY_LIMIT=NULL` means that runaway control is not enabled. See [Runaway Queries](/tidb-resource-control.md#manage-queries-that-consume-more-resources-than-expected-runaway-queries). | -| `BACKGROUND` | Configure the background tasks. For more details, see [Manage background tasks](/tidb-resource-control.md#manage-background-tasks). | `BACKGROUND=(TASK_TYPES="br,stats", UTILIZATION_LIMIT=30)` indicates that the backup and restore and statistics collection related tasks are scheduled as background tasks, and background tasks can consume 30% of the TiKV resources at most. | +| `QUERY_LIMIT` | When the query execution meets this condition, the query is identified as a runaway query and the corresponding action is executed. | `QUERY_LIMIT=(EXEC_ELAPSED='60s', ACTION=KILL, WATCH=EXACT DURATION='10m')` indicates that the query is identified as a runaway query when the execution time exceeds 60 seconds. The query is terminated. All SQL statements with the same SQL text will be terminated immediately in the coming 10 minutes. `QUERY_LIMIT=()` or `QUERY_LIMIT=NULL` means that runaway control is not enabled. See [Runaway Queries](/tidb-resource-control-runaway-queries.md). | +| `BACKGROUND` | Configure the background tasks. For more details, see [Manage background tasks](/tidb-resource-control-background-tasks.md). | `BACKGROUND=(TASK_TYPES="br,stats", UTILIZATION_LIMIT=30)` indicates that the backup and restore and statistics collection related tasks are scheduled as background tasks, and background tasks can consume 30% of the TiKV resources at most. | > **Note:** > diff --git a/sql-statements/sql-statement-create-resource-group.md b/sql-statements/sql-statement-create-resource-group.md index 196fd7b5f64a3..dd48f0a686b71 100644 --- a/sql-statements/sql-statement-create-resource-group.md +++ b/sql-statements/sql-statement-create-resource-group.md @@ -82,7 +82,7 @@ TiDB supports the following `DirectResourceGroupOption`, where [Request Unit (RU | `RU_PER_SEC` | Rate of RU backfilling per second | `RU_PER_SEC = 500` indicates that this resource group is backfilled with 500 RUs per second | | `PRIORITY` | The absolute priority of tasks to be processed on TiKV | `PRIORITY = HIGH` indicates that the priority is high. If not specified, the default value is `MEDIUM`. | | `BURSTABLE` | If the `BURSTABLE` attribute is set, TiDB allows the corresponding resource group to use the available system resources when the quota is exceeded. | -| `QUERY_LIMIT` | When the query execution meets this condition, the query is identified as a runaway query and the corresponding action is executed. | `QUERY_LIMIT=(EXEC_ELAPSED='60s', ACTION=KILL, WATCH=EXACT DURATION='10m')` indicates that the query is identified as a runaway query when the execution time exceeds 60 seconds. The query is terminated. All SQL statements with the same SQL text will be terminated immediately in the coming 10 minutes. `QUERY_LIMIT=()` or `QUERY_LIMIT=NULL` means that runaway control is not enabled. See [Runaway Queries](/tidb-resource-control.md#manage-queries-that-consume-more-resources-than-expected-runaway-queries). | +| `QUERY_LIMIT` | When the query execution meets this condition, the query is identified as a runaway query and the corresponding action is executed. | `QUERY_LIMIT=(EXEC_ELAPSED='60s', ACTION=KILL, WATCH=EXACT DURATION='10m')` indicates that the query is identified as a runaway query when the execution time exceeds 60 seconds. The query is terminated. All SQL statements with the same SQL text will be terminated immediately in the coming 10 minutes. `QUERY_LIMIT=()` or `QUERY_LIMIT=NULL` means that runaway control is not enabled. See [Runaway Queries](/tidb-resource-control-runaway-queries.md). | > **Note:** > diff --git a/sql-statements/sql-statement-query-watch.md b/sql-statements/sql-statement-query-watch.md index e2aae925084ff..2336428751692 100644 --- a/sql-statements/sql-statement-query-watch.md +++ b/sql-statements/sql-statement-query-watch.md @@ -54,7 +54,7 @@ DropQueryWatchStmt ::= ## Parameters -See [`QUERY WATCH` parameters](/tidb-resource-control.md#query-watch-parameters). +See [`QUERY WATCH` parameters](/tidb-resource-control-runaway-queries.md#query-watch-parameters). ## MySQL compatibility @@ -62,4 +62,4 @@ This statement is a TiDB extension to MySQL syntax. ## See also -* [Runaway Queries](/tidb-resource-control.md#manage-queries-that-consume-more-resources-than-expected-runaway-queries) +* [Runaway Queries](/tidb-resource-control-runaway-queries.md) diff --git a/tidb-resource-control-background-tasks.md b/tidb-resource-control-background-tasks.md new file mode 100644 index 0000000000000..8693834e092bc --- /dev/null +++ b/tidb-resource-control-background-tasks.md @@ -0,0 +1,95 @@ +--- +title: Limit background task resource usage +summary: Introduces how to control background tasks through Resource Control. +--- + +# Manage background tasks + +> **Warning:** +> +> This feature is experimental. It is not recommended that you use it in the production environment. This feature might be changed or removed without prior notice. If you find a bug, you can report an [issue](https://docs.pingcap.com/tidb/stable/support) on GitHub. +> +> The background task management in resource control is based on TiKV's dynamic adjustment of resource quotas for CPU/IO utilization. Therefore, it relies on the available resource quota of each instance. If multiple components or instances are deployed on a single server, it is mandatory to set the appropriate resource quota for each instance through `cgroup`. It is difficult to achieve the expected effect in deployment with shared resources like TiUP Playground. + +Background tasks, such as data backup and automatic statistics collection, are low-priority but consume many resources. These tasks are usually triggered periodically or irregularly. During execution, they consume a lot of resources, thus affecting the performance of online high-priority tasks. + +Starting from v7.4.0, the TiDB resource control feature supports managing background tasks. When a task is marked as a background task, TiKV dynamically limits the resources used by this type of task to avoid the impact on the performance of other foreground tasks. TiKV monitors the CPU and IO resources consumed by all foreground tasks in real time, and calculates the resource threshold that can be used by background tasks based on the total resource limit of the instance. All background tasks are restricted by this threshold during execution. + +## `BACKGROUND` parameters + +- `TASK_TYPES`: specifies the task types that need to be managed as background tasks. Use commas (`,`) to separate multiple task types. +- `UTILIZATION_LIMIT`: limits the maximum percentage (0-100) of resources that background tasks can consume on each TiKV node. By default, TiKV calculates the available resources for background tasks based on the total resources of the node and the resources currently occupied by the foreground tasks. If `UTILIZATION_LIMIT` is configured, the resource allocated to background tasks will not exceed this limit. + +TiDB supports the following types of background tasks: + + + +- `lightning`: perform import tasks using [TiDB Lightning](/tidb-lightning/tidb-lightning-overview.md). Both physical and logical import modes of TiDB Lightning are supported. +- `br`: perform backup and restore tasks using [BR](/br/backup-and-restore-overview.md). PITR is not supported. +- `ddl`: control the resource usage during the batch data write back phase of Reorg DDLs. +- `stats`: the [collect statistics](/statistics.md#collect-statistics) tasks that are manually executed or automatically triggered by TiDB. +- `background`: a reserved task type. You can use the [`tidb_request_source_type`](/system-variables.md#tidb_request_source_type-new-in-v740) system variable to specify the task type of the current session as `background`. + + + + + +- `lightning`: perform import tasks using [TiDB Lightning](https://docs.pingcap.com/tidb/stable/tidb-lightning-overview). Both physical and logical import modes of TiDB Lightning are supported. +- `br`: perform backup and restore tasks using [BR](https://docs.pingcap.com/tidb/stable/backup-and-restore-overview). PITR is not supported. +- `ddl`: control the resource usage during the batch data write back phase of Reorg DDLs. +- `stats`: the [collect statistics](/statistics.md#collect-statistics) tasks that are manually executed or automatically triggered by TiDB. +- `background`: a reserved task type. You can use the [`tidb_request_source_type`](/system-variables.md#tidb_request_source_type-new-in-v740) system variable to specify the task type of the current session as `background`. + + + +By default, the task types that are marked as background tasks are `""`, and the management of background tasks is disabled. To enable background task management, you need to manually modify the background task type of the `default` resource group. After a background task is identified and matched, Resource Control is automatically performed. This means that when system resources are insufficient, the background tasks are automatically reduced to the lowest priority to ensure the execution of foreground tasks. + +> **Note:** +> +> Currently, background tasks for all resource groups are bound to the `default` resource group. You can manage background task types globally through `default`. Binding background tasks to other resource groups is currently not supported. + +## Examples + +1. Modify the `default` resource group by marking `br` and `ddl` as background tasks and setting the resource limit of background tasks to 30%. + + ```sql + ALTER RESOURCE GROUP `default` BACKGROUND=(TASK_TYPES='br,ddl', UTILIZATION_LIMIT=30); + ``` + +2. Change the `default` resource group to revert the background task type to its default value. + + ```sql + ALTER RESOURCE GROUP `default` BACKGROUND=NULL; + ``` + +3. Change the `default` resource group to set the background task type to empty. In this case, all tasks of this resource group are not treated as background tasks. + + ```sql + ALTER RESOURCE GROUP `default` BACKGROUND=(TASK_TYPES=""); + ``` + +4. View the background task type of the `default` resource group. + + ```sql + SELECT * FROM information_schema.resource_groups WHERE NAME="default"; + ``` + + The output is as follows: + + ``` + +---------+------------+----------+-----------+-------------+-------------------------------------------+ + | NAME | RU_PER_SEC | PRIORITY | BURSTABLE | QUERY_LIMIT | BACKGROUND | + +---------+------------+----------+-----------+-------------+-------------------------------------------+ + | default | UNLIMITED | MEDIUM | YES | NULL | TASK_TYPES='br,ddl', UTILIZATION_LIMIT=30 | + +---------+------------+----------+-----------+-------------+-------------------------------------------+ + ``` + +5. To explicitly mark tasks in the current session as the background type, you can use `tidb_request_source_type` to explicitly specify the task type. The following is an example: + + ``` sql + SET @@tidb_request_source_type="background"; + /* Add background task type */ + ALTER RESOURCE GROUP `default` BACKGROUND=(TASK_TYPES="background"); + /* Execute LOAD DATA in the current session */ + LOAD DATA INFILE "s3://resource-control/Lightning/test.customer.aaaa.csv" + ``` diff --git a/tidb-resource-control-runaway-queries.md b/tidb-resource-control-runaway-queries.md new file mode 100644 index 0000000000000..39bf477722d32 --- /dev/null +++ b/tidb-resource-control-runaway-queries.md @@ -0,0 +1,163 @@ +--- +title: Managing Queries with Excessive Resource Consumption (Runaway Queries) +summary: Introduces how to control and degrade queries with excessive resource consumption (Runaway Queries) through resource management capabilities. +--- + +# Manage queries that consume more resources than expected (Runaway Queries) + +A runaway query is a query (`SELECT` statement only) that consumes more time or resources than expected. The term **runaway queries** is used in the following to describe the feature of managing the runaway query. + +- Starting from v7.2.0, the resource control feature introduces the management of runaway queries. You can set criteria for a resource group to identify runaway queries and automatically take actions to prevent them from exhausting resources and affecting other queries. You can manage runaway queries for a resource group by including the `QUERY_LIMIT` field in [`CREATE RESOURCE GROUP`](/sql-statements/sql-statement-create-resource-group.md) or [`ALTER RESOURCE GROUP`](/sql-statements/sql-statement-alter-resource-group.md). +- Starting from v7.3.0, the resource control feature introduces manual management of runaway watches, enabling quick identification of runaway queries for a given SQL statement or Digest. You can execute the statement [`QUERY WATCH`](/sql-statements/sql-statement-query-watch.md) to manually manage the runaway queries watch list in the resource group. + +## `QUERY_LIMIT` parameters + +If a query exceeds any of the following limits, it is identified as a runaway query: + +- `EXEC_ELAPSED`: checks whether the query execution time exceeds the limit. +- `PROCESSED_KEYS`: checks whether the number of keys processed by the Coprocessor exceeds the limit. +- `RU`: checks whether the total number of read and write RUs consumed by the statement exceeds the limit. + +Supported operations (`ACTION`): + +- `DRYRUN`: no action is taken. The records are appended for the runaway queries. This is mainly used to observe whether the condition setting is reasonable. +- `COOLDOWN`: the execution priority of the query is lowered to the lowest level. The query continues to execute with the lowest priority and does not occupy resources of other operations. +- `KILL`: the identified query is automatically terminated and reports an error `Query execution was interrupted, identified as runaway query`. +- `SWITCH_GROUP`: introduced in v8.4.0, this parameter switches the identified query to the specified resource group for continued execution. After this query completes, subsequent SQL statements are executed in the original resource group. If the specified resource group does not exist, the query remains in the original resource group. + +To avoid too many concurrent runaway queries that exhaust system resources, the resource control feature introduces a quick identification mechanism, which can quickly identify and isolate runaway queries. You can use this feature through the `WATCH` clause. When a query is identified as a runaway query, this mechanism extracts the matching feature (defined by the parameter after `WATCH`) of the query. In the next period of time (defined by `DURATION`), the matching feature of the runaway query is added to the watch list, and the TiDB instance matches queries with the watch list. The matching queries are directly marked as runaway queries and isolated according to the corresponding action, instead of waiting for them to be identified by conditions. The `KILL` operation terminates the query and reports an error `Quarantined and interrupted because of being in runaway watch list`. + +There are three methods for `WATCH` to match for quick identification: + +- `EXACT` indicates that only SQL statements with exactly the same SQL text are quickly identified. +- `SIMILAR` indicates all SQL statements with the same pattern are matched by Plan Digest, and the literal values are ignored. +- `PLAN` indicates all SQL statements with the same pattern are matched by Plan Digest. + +The `DURATION` option in `WATCH` indicates the duration of the identification item, which is infinite by default. + +After a watch item is added, neither the matching feature nor the `ACTION` is changed or deleted whenever the `QUERY_LIMIT` configuration is changed or deleted. You can use `QUERY WATCH REMOVE` to remove a watch item. + +The parameters of `QUERY_LIMIT` are as follows: + +| Parameter | Description | Note | +|---------------|--------------|--------------------------------------| +| `EXEC_ELAPSED` | When the query execution time exceeds this value, it is identified as a runaway query | EXEC_ELAPSED =`60s` means the query is identified as a runaway query if it takes more than 60 seconds to execute. | +| `PROCESSED_KEYS` | When the number of keys processed by the Coprocessor exceeds this value, the query is identified as a runaway query | `PROCESSED_KEYS = 1000` means the query is identified as a runaway query if the number of keys processed by the Coprocessor exceeds 1000. | +| `RU` | When the total number of read and write RUs consumed by the query exceeds this value, this query is identified as a runaway query | `RU = 1000` means the query is identified as a runaway query if the total number of read and write RUs consumed by the query exceeds 1000. | +| `ACTION` | Action taken when a runaway query is identified | The optional values are `DRYRUN`, `COOLDOWN`, `KILL`, and `SWITCH_GROUP`. | +| `WATCH` | Quickly match the identified runaway query. If the same or similar query is encountered again within a certain period of time, the corresponding action is performed immediately. | Optional. For example, `WATCH=SIMILAR DURATION '60s'`, `WATCH=EXACT DURATION '1m'`, and `WATCH=PLAN`. | + +> **Note:** +> +> If you want to strictly limit runaway queries to a specific resource group, it is recommended to use `SWITCH_GROUP` together with the [`QUERY WATCH`](/tidb-resource-control-runaway-queries.md#query-watch-parameters) statement. Because `QUERY_LIMIT` only triggers the corresponding `ACTION` operation when the query meets the criteria, `SWITCH_GROUP` might not be able to switch the query to the target resource group in a timely manner in such scenarios. + +## Examples + +1. Create a resource group `rg1` with a quota of 500 RUs per second, and define a runaway query as one that exceeds 60 seconds, and lower the priority of the runaway query. + + ```sql + CREATE RESOURCE GROUP IF NOT EXISTS rg1 RU_PER_SEC = 500 QUERY_LIMIT=(EXEC_ELAPSED='60s', ACTION=COOLDOWN); + ``` + +2. Change the `rg1` resource group to terminate the runaway queries, and mark the queries with the same pattern as runaway queries immediately in the next 10 minutes. + + ```sql + ALTER RESOURCE GROUP rg1 QUERY_LIMIT=(EXEC_ELAPSED='60s', ACTION=KILL, WATCH=SIMILAR DURATION='10m'); + ``` + +3. Change the `rg1` resource group to cancel the runaway query check. + + ```sql + ALTER RESOURCE GROUP rg1 QUERY_LIMIT=NULL; + ``` + +## `QUERY WATCH` parameters + +For more information about the synopsis of `QUERY WATCH`, see [`QUERY WATCH`](/sql-statements/sql-statement-query-watch.md). + +The parameters are as follows: + +- The `RESOURCE GROUP` specifies a resource group. The matching features of runaway queries added by this statement are added to the watch list of the resource group. This parameter can be omitted. If omitted, it applies to the `default` resource group. +- The meaning of `ACTION` is the same as `QUERY LIMIT`. This parameter can be omitted. If omitted, the corresponding action after identification adopts the `ACTION` configured by `QUERY LIMIT` in the resource group, and the action does not change with the `QUERY LIMIT` configuration. If there is no `ACTION` configured in the resource group, an error is reported. +- The `QueryWatchTextOption` parameter has three options: `SQL DIGEST`, `PLAN DIGEST`, and `SQL TEXT`. + - `SQL DIGEST` is the same as that of `SIMILAR`. The following parameters accept strings, user-defined variables, or other expressions that yield string result. The string length must be 64, which is the same as the Digest definition in TiDB. + - `PLAN DIGEST` is the same as `PLAN`. The following parameter is a Digest string. + - `SQL TEXT` matches the input SQL as a raw string (`EXACT`), or parses and compiles it into `SQL DIGEST` (`SIMILAR`) or `PLAN DIGEST` (`PLAN`), depending on the following parameter. + +- Add a matching feature to the runaway query watch list for the default resource group (you need to set `QUERY LIMIT` for the default resource group in advance). + + ```sql + QUERY WATCH ADD ACTION KILL SQL TEXT EXACT TO 'select * from test.t2'; + ``` + +- Add a matching feature to the runaway query watch list for the `rg1` resource group by parsing the SQL into SQL Digest. When `ACTION` is not specified, the `ACTION` option already configured for the `rg1` resource group is used. + + ```sql + QUERY WATCH ADD RESOURCE GROUP rg1 SQL TEXT SIMILAR TO 'select * from test.t2'; + ``` + +- Add a matching feature to the runaway query watch list for the `rg1` resource group by parsing the SQL into SQL Digest, and specify `ACTION` as `SWITCH_GROUP(rg2)`. + + ```sql + QUERY WATCH ADD RESOURCE GROUP rg1 ACTION SWITCH_GROUP(rg2) SQL TEXT SIMILAR TO 'select * from test.t2'; + ``` + +- Add a matching feature to the runaway query watch list for the `rg1` resource group using `PLAN DIGEST`, and specify `ACTION` as `KILL`. + + ```sql + QUERY WATCH ADD RESOURCE GROUP rg1 ACTION KILL PLAN DIGEST 'd08bc323a934c39dc41948b0a073725be3398479b6fa4f6dd1db2a9b115f7f57'; + ``` + +- Get the watch item ID by querying `INFORMATION_SCHEMA.RUNAWAY_WATCHES` and delete the watch item. + + ```sql + SELECT * from information_schema.runaway_watches ORDER BY id\G + ``` + + ```sql + *************************** 1. row *************************** + ID: 1 + RESOURCE_GROUP_NAME: default + START_TIME: 2024-09-09 03:35:31 + END_TIME: 2024-09-09 03:45:31 + WATCH: Exact + WATCH_TEXT: SELECT variable_name, variable_value FROM mysql.global_variables + SOURCE: 127.0.0.1:4000 + ACTION: Kill + RULE: ProcessedKeys = 666(10) + 1 row in set (0.00 sec) + ``` + + ```sql + QUERY WATCH REMOVE 1; + ``` + +#### Observability + +You can get more information about runaway queries from the following system tables and `INFORMATION_SCHEMA`: + ++ The `mysql.tidb_runaway_queries` table contains the history records of all runaway queries identified in the past 7 days. Take one of the rows as an example: + + ```sql + MySQL [(none)]> SELECT * FROM mysql.tidb_runaway_queries LIMIT 1\G + *************************** 1. row *************************** + resource_group_name: default + start_time: 2024-09-09 17:43:42 + repeats: 2 + match_type: watch + action: kill + sample_sql: select sleep(2) from t + sql_digest: 4adbc838b86c573265d4b39a3979d0a362b5f0336c91c26930c83ab187701a55 + plan_digest: 5d094f78efbce44b2923733b74e1d09233cb446318293492901c5e5d92e27dbc + tidb_server: 127.0.0.1:4000 + ``` + + Field description: + + - `start_time` indicates the time when the runaway query is identified. + - `repeats` indicates the number of times the runaway query has been identified since `start_time`. + - `match_type` indicates how the runaway query is identified. The value can be one of the following: + - `identify` means that it matches the condition of the runaway query. + - `watch` means that it matches the quick identification rule in the watch list. + ++ The `information_schema.runaway_watches` table contains records of quick identification rules for runaway queries. For more information, see [`RUNAWAY_WATCHES`](/information-schema/information-schema-runaway-watches.md). diff --git a/tidb-resource-control.md b/tidb-resource-control.md index c34a7671dca82..6650468c5ed4e 100644 --- a/tidb-resource-control.md +++ b/tidb-resource-control.md @@ -1,9 +1,9 @@ --- -title: Use Resource Control to Achieve Resource Isolation +title: Use Resource Control to Achieve resource group limitation summary: Learn how to use the resource control feature to control and schedule application resources. --- -# Use Resource Control to Achieve Resource Isolation +# Use Resource Control to Achieve resource group limitation > **Note:** > @@ -259,257 +259,6 @@ The following example binds the current statement to the resource group `rg1`. ```sql SELECT /*+ RESOURCE_GROUP(rg1) */ * FROM t limit 10; ``` - -### Manage queries that consume more resources than expected (Runaway Queries) - -A runaway query is a query (`SELECT` statement only) that consumes more time or resources than expected. The term **runaway queries** is used in the following to describe the feature of managing the runaway query. - -- Starting from v7.2.0, the resource control feature introduces the management of runaway queries. You can set criteria for a resource group to identify runaway queries and automatically take actions to prevent them from exhausting resources and affecting other queries. You can manage runaway queries for a resource group by including the `QUERY_LIMIT` field in [`CREATE RESOURCE GROUP`](/sql-statements/sql-statement-create-resource-group.md) or [`ALTER RESOURCE GROUP`](/sql-statements/sql-statement-alter-resource-group.md). -- Starting from v7.3.0, the resource control feature introduces manual management of runaway watches, enabling quick identification of runaway queries for a given SQL statement or Digest. You can execute the statement [`QUERY WATCH`](/sql-statements/sql-statement-query-watch.md) to manually manage the runaway queries watch list in the resource group. - -#### `QUERY_LIMIT` parameters - -If a query exceeds any of the following limits, it is identified as a runaway query: - -- `EXEC_ELAPSED`: checks whether the query execution time exceeds the limit. -- `PROCESSED_KEYS`: checks whether the number of keys processed by the Coprocessor exceeds the limit. -- `RU`: checks whether the total number of read and write RUs consumed by the statement exceeds the limit. - -Supported operations (`ACTION`): - -- `DRYRUN`: no action is taken. The records are appended for the runaway queries. This is mainly used to observe whether the condition setting is reasonable. -- `COOLDOWN`: the execution priority of the query is lowered to the lowest level. The query continues to execute with the lowest priority and does not occupy resources of other operations. -- `KILL`: the identified query is automatically terminated and reports an error `Query execution was interrupted, identified as runaway query`. -- `SWITCH_GROUP`: introduced in v8.4.0, this parameter switches the identified query to the specified resource group for continued execution. After this query completes, subsequent SQL statements are executed in the original resource group. If the specified resource group does not exist, the query remains in the original resource group. - -To avoid too many concurrent runaway queries that exhaust system resources, the resource control feature introduces a quick identification mechanism, which can quickly identify and isolate runaway queries. You can use this feature through the `WATCH` clause. When a query is identified as a runaway query, this mechanism extracts the matching feature (defined by the parameter after `WATCH`) of the query. In the next period of time (defined by `DURATION`), the matching feature of the runaway query is added to the watch list, and the TiDB instance matches queries with the watch list. The matching queries are directly marked as runaway queries and isolated according to the corresponding action, instead of waiting for them to be identified by conditions. The `KILL` operation terminates the query and reports an error `Quarantined and interrupted because of being in runaway watch list`. - -There are three methods for `WATCH` to match for quick identification: - -- `EXACT` indicates that only SQL statements with exactly the same SQL text are quickly identified. -- `SIMILAR` indicates all SQL statements with the same pattern are matched by Plan Digest, and the literal values are ignored. -- `PLAN` indicates all SQL statements with the same pattern are matched by Plan Digest. - -The `DURATION` option in `WATCH` indicates the duration of the identification item, which is infinite by default. - -After a watch item is added, neither the matching feature nor the `ACTION` is changed or deleted whenever the `QUERY_LIMIT` configuration is changed or deleted. You can use `QUERY WATCH REMOVE` to remove a watch item. - -The parameters of `QUERY_LIMIT` are as follows: - -| Parameter | Description | Note | -|---------------|--------------|--------------------------------------| -| `EXEC_ELAPSED` | When the query execution time exceeds this value, it is identified as a runaway query | EXEC_ELAPSED =`60s` means the query is identified as a runaway query if it takes more than 60 seconds to execute. | -| `PROCESSED_KEYS` | When the number of keys processed by the Coprocessor exceeds this value, the query is identified as a runaway query | `PROCESSED_KEYS = 1000` means the query is identified as a runaway query if the number of keys processed by the Coprocessor exceeds 1000. | -| `RU` | When the total number of read and write RUs consumed by the query exceeds this value, this query is identified as a runaway query | `RU = 1000` means the query is identified as a runaway query if the total number of read and write RUs consumed by the query exceeds 1000. | -| `ACTION` | Action taken when a runaway query is identified | The optional values are `DRYRUN`, `COOLDOWN`, `KILL`, and `SWITCH_GROUP`. | -| `WATCH` | Quickly match the identified runaway query. If the same or similar query is encountered again within a certain period of time, the corresponding action is performed immediately. | Optional. For example, `WATCH=SIMILAR DURATION '60s'`, `WATCH=EXACT DURATION '1m'`, and `WATCH=PLAN`. | - -> **Note:** -> -> If you want to strictly limit runaway queries to a specific resource group, it is recommended to use `SWITCH_GROUP` together with the [`QUERY WATCH`](/tidb-resource-control.md#query-watch-parameters) statement. Because `QUERY_LIMIT` only triggers the corresponding `ACTION` operation when the query meets the criteria, `SWITCH_GROUP` might not be able to switch the query to the target resource group in a timely manner in such scenarios. - -#### Examples - -1. Create a resource group `rg1` with a quota of 500 RUs per second, and define a runaway query as one that exceeds 60 seconds, and lower the priority of the runaway query. - - ```sql - CREATE RESOURCE GROUP IF NOT EXISTS rg1 RU_PER_SEC = 500 QUERY_LIMIT=(EXEC_ELAPSED='60s', ACTION=COOLDOWN); - ``` - -2. Change the `rg1` resource group to terminate the runaway queries, and mark the queries with the same pattern as runaway queries immediately in the next 10 minutes. - - ```sql - ALTER RESOURCE GROUP rg1 QUERY_LIMIT=(EXEC_ELAPSED='60s', ACTION=KILL, WATCH=SIMILAR DURATION='10m'); - ``` - -3. Change the `rg1` resource group to cancel the runaway query check. - - ```sql - ALTER RESOURCE GROUP rg1 QUERY_LIMIT=NULL; - ``` - -#### `QUERY WATCH` parameters - -For more information about the synopsis of `QUERY WATCH`, see [`QUERY WATCH`](/sql-statements/sql-statement-query-watch.md). - -The parameters are as follows: - -- The `RESOURCE GROUP` specifies a resource group. The matching features of runaway queries added by this statement are added to the watch list of the resource group. This parameter can be omitted. If omitted, it applies to the `default` resource group. -- The meaning of `ACTION` is the same as `QUERY LIMIT`. This parameter can be omitted. If omitted, the corresponding action after identification adopts the `ACTION` configured by `QUERY LIMIT` in the resource group, and the action does not change with the `QUERY LIMIT` configuration. If there is no `ACTION` configured in the resource group, an error is reported. -- The `QueryWatchTextOption` parameter has three options: `SQL DIGEST`, `PLAN DIGEST`, and `SQL TEXT`. - - `SQL DIGEST` is the same as that of `SIMILAR`. The following parameters accept strings, user-defined variables, or other expressions that yield string result. The string length must be 64, which is the same as the Digest definition in TiDB. - - `PLAN DIGEST` is the same as `PLAN`. The following parameter is a Digest string. - - `SQL TEXT` matches the input SQL as a raw string (`EXACT`), or parses and compiles it into `SQL DIGEST` (`SIMILAR`) or `PLAN DIGEST` (`PLAN`), depending on the following parameter. - -- Add a matching feature to the runaway query watch list for the default resource group (you need to set `QUERY LIMIT` for the default resource group in advance). - - ```sql - QUERY WATCH ADD ACTION KILL SQL TEXT EXACT TO 'select * from test.t2'; - ``` - -- Add a matching feature to the runaway query watch list for the `rg1` resource group by parsing the SQL into SQL Digest. When `ACTION` is not specified, the `ACTION` option already configured for the `rg1` resource group is used. - - ```sql - QUERY WATCH ADD RESOURCE GROUP rg1 SQL TEXT SIMILAR TO 'select * from test.t2'; - ``` - -- Add a matching feature to the runaway query watch list for the `rg1` resource group by parsing the SQL into SQL Digest, and specify `ACTION` as `SWITCH_GROUP(rg2)`. - - ```sql - QUERY WATCH ADD RESOURCE GROUP rg1 ACTION SWITCH_GROUP(rg2) SQL TEXT SIMILAR TO 'select * from test.t2'; - ``` - -- Add a matching feature to the runaway query watch list for the `rg1` resource group using `PLAN DIGEST`, and specify `ACTION` as `KILL`. - - ```sql - QUERY WATCH ADD RESOURCE GROUP rg1 ACTION KILL PLAN DIGEST 'd08bc323a934c39dc41948b0a073725be3398479b6fa4f6dd1db2a9b115f7f57'; - ``` - -- Get the watch item ID by querying `INFORMATION_SCHEMA.RUNAWAY_WATCHES` and delete the watch item. - - ```sql - SELECT * from information_schema.runaway_watches ORDER BY id\G - ``` - - ```sql - *************************** 1. row *************************** - ID: 1 - RESOURCE_GROUP_NAME: default - START_TIME: 2024-09-09 03:35:31 - END_TIME: 2024-09-09 03:45:31 - WATCH: Exact - WATCH_TEXT: SELECT variable_name, variable_value FROM mysql.global_variables - SOURCE: 127.0.0.1:4000 - ACTION: Kill - RULE: ProcessedKeys = 666(10) - 1 row in set (0.00 sec) - ``` - - ```sql - QUERY WATCH REMOVE 1; - ``` - -#### Observability - -You can get more information about runaway queries from the following system tables and `INFORMATION_SCHEMA`: - -+ The `mysql.tidb_runaway_queries` table contains the history records of all runaway queries identified in the past 7 days. Take one of the rows as an example: - - ```sql - MySQL [(none)]> SELECT * FROM mysql.tidb_runaway_queries LIMIT 1\G - *************************** 1. row *************************** - resource_group_name: default - start_time: 2024-09-09 17:43:42 - repeats: 2 - match_type: watch - action: kill - sample_sql: select sleep(2) from t - sql_digest: 4adbc838b86c573265d4b39a3979d0a362b5f0336c91c26930c83ab187701a55 - plan_digest: 5d094f78efbce44b2923733b74e1d09233cb446318293492901c5e5d92e27dbc - tidb_server: 127.0.0.1:4000 - ``` - - Field description: - - - `start_time` indicates the time when the runaway query is identified. - - `repeats` indicates the number of times the runaway query has been identified since `start_time`. - - `match_type` indicates how the runaway query is identified. The value can be one of the following: - - `identify` means that it matches the condition of the runaway query. - - `watch` means that it matches the quick identification rule in the watch list. - -+ The `information_schema.runaway_watches` table contains records of quick identification rules for runaway queries. For more information, see [`RUNAWAY_WATCHES`](/information-schema/information-schema-runaway-watches.md). - -### Manage background tasks - -> **Warning:** -> -> This feature is experimental. It is not recommended that you use it in the production environment. This feature might be changed or removed without prior notice. If you find a bug, you can report an [issue](https://docs.pingcap.com/tidb/stable/support) on GitHub. -> -> The background task management in resource control is based on TiKV's dynamic adjustment of resource quotas for CPU/IO utilization. Therefore, it relies on the available resource quota of each instance. If multiple components or instances are deployed on a single server, it is mandatory to set the appropriate resource quota for each instance through `cgroup`. It is difficult to achieve the expected effect in deployment with shared resources like TiUP Playground. - -Background tasks, such as data backup and automatic statistics collection, are low-priority but consume many resources. These tasks are usually triggered periodically or irregularly. During execution, they consume a lot of resources, thus affecting the performance of online high-priority tasks. - -Starting from v7.4.0, the TiDB resource control feature supports managing background tasks. When a task is marked as a background task, TiKV dynamically limits the resources used by this type of task to avoid the impact on the performance of other foreground tasks. TiKV monitors the CPU and IO resources consumed by all foreground tasks in real time, and calculates the resource threshold that can be used by background tasks based on the total resource limit of the instance. All background tasks are restricted by this threshold during execution. - -#### `BACKGROUND` parameters - -- `TASK_TYPES`: specifies the task types that need to be managed as background tasks. Use commas (`,`) to separate multiple task types. -- `UTILIZATION_LIMIT`: limits the maximum percentage (0-100) of resources that background tasks can consume on each TiKV node. By default, TiKV calculates the available resources for background tasks based on the total resources of the node and the resources currently occupied by the foreground tasks. If `UTILIZATION_LIMIT` is configured, the resource allocated to background tasks will not exceed this limit. - -TiDB supports the following types of background tasks: - - - -- `lightning`: perform import tasks using [TiDB Lightning](/tidb-lightning/tidb-lightning-overview.md). Both physical and logical import modes of TiDB Lightning are supported. -- `br`: perform backup and restore tasks using [BR](/br/backup-and-restore-overview.md). PITR is not supported. -- `ddl`: control the resource usage during the batch data write back phase of Reorg DDLs. -- `stats`: the [collect statistics](/statistics.md#collect-statistics) tasks that are manually executed or automatically triggered by TiDB. -- `background`: a reserved task type. You can use the [`tidb_request_source_type`](/system-variables.md#tidb_request_source_type-new-in-v740) system variable to specify the task type of the current session as `background`. - - - - - -- `lightning`: perform import tasks using [TiDB Lightning](https://docs.pingcap.com/tidb/stable/tidb-lightning-overview). Both physical and logical import modes of TiDB Lightning are supported. -- `br`: perform backup and restore tasks using [BR](https://docs.pingcap.com/tidb/stable/backup-and-restore-overview). PITR is not supported. -- `ddl`: control the resource usage during the batch data write back phase of Reorg DDLs. -- `stats`: the [collect statistics](/statistics.md#collect-statistics) tasks that are manually executed or automatically triggered by TiDB. -- `background`: a reserved task type. You can use the [`tidb_request_source_type`](/system-variables.md#tidb_request_source_type-new-in-v740) system variable to specify the task type of the current session as `background`. - - - -By default, the task types that are marked as background tasks are `""`, and the management of background tasks is disabled. To enable background task management, you need to manually modify the background task type of the `default` resource group. After a background task is identified and matched, Resource Control is automatically performed. This means that when system resources are insufficient, the background tasks are automatically reduced to the lowest priority to ensure the execution of foreground tasks. - -> **Note:** -> -> Currently, background tasks for all resource groups are bound to the `default` resource group. You can manage background task types globally through `default`. Binding background tasks to other resource groups is currently not supported. - -#### Examples - -1. Modify the `default` resource group by marking `br` and `ddl` as background tasks and setting the resource limit of background tasks to 30%. - - ```sql - ALTER RESOURCE GROUP `default` BACKGROUND=(TASK_TYPES='br,ddl', UTILIZATION_LIMIT=30); - ``` - -2. Change the `default` resource group to revert the background task type to its default value. - - ```sql - ALTER RESOURCE GROUP `default` BACKGROUND=NULL; - ``` - -3. Change the `default` resource group to set the background task type to empty. In this case, all tasks of this resource group are not treated as background tasks. - - ```sql - ALTER RESOURCE GROUP `default` BACKGROUND=(TASK_TYPES=""); - ``` - -4. View the background task type of the `default` resource group. - - ```sql - SELECT * FROM information_schema.resource_groups WHERE NAME="default"; - ``` - - The output is as follows: - - ``` - +---------+------------+----------+-----------+-------------+-------------------------------------------+ - | NAME | RU_PER_SEC | PRIORITY | BURSTABLE | QUERY_LIMIT | BACKGROUND | - +---------+------------+----------+-----------+-------------+-------------------------------------------+ - | default | UNLIMITED | MEDIUM | YES | NULL | TASK_TYPES='br,ddl', UTILIZATION_LIMIT=30 | - +---------+------------+----------+-----------+-------------+-------------------------------------------+ - ``` - -5. To explicitly mark tasks in the current session as the background type, you can use `tidb_request_source_type` to explicitly specify the task type. The following is an example: - - ``` sql - SET @@tidb_request_source_type="background"; - /* Add background task type */ - ALTER RESOURCE GROUP `default` BACKGROUND=(TASK_TYPES="background"); - /* Execute LOAD DATA in the current session */ - LOAD DATA INFILE "s3://resource-control/Lightning/test.customer.aaaa.csv" - ``` - ## Disable resource control From ef39b0888c15541a5906e5b690d168ad75da5b0c Mon Sep 17 00:00:00 2001 From: nolouch Date: Wed, 15 Jan 2025 18:23:52 +0800 Subject: [PATCH 02/25] update Signed-off-by: nolouch --- .../information-schema-resource-groups.md | 2 +- releases/release-8.4.0.md | 2 +- system-variable-reference.md | 10 +++++----- tidb-resource-control-runaway-queries.md | 8 ++++---- tiflash/tiflash-pipeline-model.md | 2 +- user-account-management.md | 2 +- 6 files changed, 13 insertions(+), 13 deletions(-) diff --git a/information-schema/information-schema-resource-groups.md b/information-schema/information-schema-resource-groups.md index 3d54953fd92d8..a17517ce61ce6 100644 --- a/information-schema/information-schema-resource-groups.md +++ b/information-schema/information-schema-resource-groups.md @@ -5,7 +5,7 @@ summary: Learn the `RESOURCE_GROUPS` information_schema table. # RESOURCE_GROUPS -The `RESOURCE_GROUPS` table shows the information about all resource groups. For more information, see [Use Resource Control to Achieve Resource Isolation](/tidb-resource-control.md). +The `RESOURCE_GROUPS` table shows the information about all resource groups. For more information, see [Use Resource Control to Achieve resource group limitation](/tidb-resource-control.md). > **Note:** > diff --git a/releases/release-8.4.0.md b/releases/release-8.4.0.md index 73e40d33abd02..af431cd2d3d7c 100644 --- a/releases/release-8.4.0.md +++ b/releases/release-8.4.0.md @@ -156,7 +156,7 @@ Quick access: [Quick start](https://docs.pingcap.com/tidb/v8.4/quick-start-with- Starting from TiDB v8.4.0, you can switch the resource group of runaway queries to a specific one. If the `COOLDOWN` mechanism fails to lower resource consumption, you can create a [resource group](/tidb-resource-control.md#create-a-resource-group), limit its resource size, and set the `SWITCH_GROUP` parameter to move identified runaway queries to this group. Meanwhile, subsequent queries within the same session will continue to execute in the original resource group. By switching resource groups, you can manage resource usage more precisely, and control the resource consumption more strictly. - For more information, see [documentation](/tidb-resource-control.md#query_limit-parameters). + For more information, see [documentation](/tidb-resource-control-runaway-queries.md#query_limit-parameters). * Support setting the cluster-level Region scattering strategy using the `tidb_scatter_region` system variable [#55184](https://github.com/pingcap/tidb/issues/55184) @[D3Hunter](https://github.com/D3Hunter) diff --git a/system-variable-reference.md b/system-variable-reference.md index 7f0643abb192e..a49bdf690b610 100644 --- a/system-variable-reference.md +++ b/system-variable-reference.md @@ -1895,7 +1895,7 @@ Referenced in: - [SET RESOURCE GROUP](/sql-statements/sql-statement-set-resource-group.md) - [System Variables](/system-variables.md#tidb_enable_resource_control-new-in-v660) - [TiKV Configuration File](/tikv-configuration-file.md) -- [Use Resource Control to Achieve Resource Isolation](/tidb-resource-control.md) +- [Use Resource Control to Achieve resource group limitation](/tidb-resource-control.md) - [`CALIBRATE RESOURCE`](/sql-statements/sql-statement-calibrate-resource.md) - [TiDB 7.4.0 Release Notes](/releases/release-7.4.0.md) - [TiDB 7.0.0 Release Notes](/releases/release-7.0.0.md) @@ -2479,7 +2479,7 @@ Referenced in: Referenced in: - [System Variables](/system-variables.md#tidb_last_query_info-new-in-v4014) -- [Use Resource Control to Achieve Resource Isolation](/tidb-resource-control.md) +- [Use Resource Control to Achieve resource group limitation](/tidb-resource-control.md) - [TiDB 7.6.0 Release Notes](/releases/release-7.6.0.md) - [TiDB 7.5.1 Release Notes](/releases/release-7.5.1.md) @@ -3353,7 +3353,7 @@ Referenced in: Referenced in: - [System Variables](/system-variables.md#tidb_request_source_type-new-in-v740) -- [Use Resource Control to Achieve Resource Isolation](/tidb-resource-control.md) +- [Use Resource Control to Achieve resource group limitation](/tidb-resource-control.md) - [TiDB 7.4.0 Release Notes](/releases/release-7.4.0.md) ### tidb_resource_control_strict_mode @@ -3364,7 +3364,7 @@ Referenced in: - [Privilege Management](/privilege-management.md) - [SET RESOURCE GROUP](/sql-statements/sql-statement-set-resource-group.md) - [System Variables](/system-variables.md#tidb_resource_control_strict_mode-new-in-v820) -- [Use Resource Control to Achieve Resource Isolation](/tidb-resource-control.md) +- [Use Resource Control to Achieve resource group limitation](/tidb-resource-control.md) - [TiDB 8.2.0 Release Notes](/releases/release-8.2.0.md) ### tidb_restricted_read_only @@ -4152,7 +4152,7 @@ Referenced in: - [Limited SQL features on TiDB Cloud](https://docs.pingcap.com/tidbcloud/limited-sql-features) - [System Variables](/system-variables.md#txn_scope) - [TiDB Configuration File](/tidb-configuration-file.md) -- [Use Resource Control to Achieve Resource Isolation](/tidb-resource-control.md) +- [Use Resource Control to Achieve resource group limitation](/tidb-resource-control.md) ### validate_password.check_user_name diff --git a/tidb-resource-control-runaway-queries.md b/tidb-resource-control-runaway-queries.md index 39bf477722d32..9c0e0bc2ffea5 100644 --- a/tidb-resource-control-runaway-queries.md +++ b/tidb-resource-control-runaway-queries.md @@ -5,7 +5,7 @@ summary: Introduces how to control and degrade queries with excessive resource c # Manage queries that consume more resources than expected (Runaway Queries) -A runaway query is a query (`SELECT` statement only) that consumes more time or resources than expected. The term **runaway queries** is used in the following to describe the feature of managing the runaway query. +A runaway query is a query that consumes more time or resources than expected. The term **runaway queries** is used in the following to describe the feature of managing the runaway query. - Starting from v7.2.0, the resource control feature introduces the management of runaway queries. You can set criteria for a resource group to identify runaway queries and automatically take actions to prevent them from exhausting resources and affecting other queries. You can manage runaway queries for a resource group by including the `QUERY_LIMIT` field in [`CREATE RESOURCE GROUP`](/sql-statements/sql-statement-create-resource-group.md) or [`ALTER RESOURCE GROUP`](/sql-statements/sql-statement-alter-resource-group.md). - Starting from v7.3.0, the resource control feature introduces manual management of runaway watches, enabling quick identification of runaway queries for a given SQL statement or Digest. You can execute the statement [`QUERY WATCH`](/sql-statements/sql-statement-query-watch.md) to manually manage the runaway queries watch list in the resource group. @@ -14,9 +14,9 @@ A runaway query is a query (`SELECT` statement only) that consumes more time or If a query exceeds any of the following limits, it is identified as a runaway query: -- `EXEC_ELAPSED`: checks whether the query execution time exceeds the limit. -- `PROCESSED_KEYS`: checks whether the number of keys processed by the Coprocessor exceeds the limit. -- `RU`: checks whether the total number of read and write RUs consumed by the statement exceeds the limit. +- `EXEC_ELAPSED`: checks whether the query execution time exceeds the limit, this rule applies to read and write DML. +- `PROCESSED_KEYS`: checks whether the number of keys processed by the Coprocessor exceeds the limit, this rule applies only to read statements. +- `RU`: checks whether the total number of read and write RUs consumed by the statement exceeds the limit, this rule applies only to read statements. Supported operations (`ACTION`): diff --git a/tiflash/tiflash-pipeline-model.md b/tiflash/tiflash-pipeline-model.md index 46b7cb5c7661e..643fc6874a975 100644 --- a/tiflash/tiflash-pipeline-model.md +++ b/tiflash/tiflash-pipeline-model.md @@ -10,7 +10,7 @@ This document introduces the TiFlash pipeline execution model. Starting from v7.2.0, TiFlash supports a new execution model, the pipeline execution model. - For v7.2.0 and v7.3.0: The pipeline execution model is experimental and is controlled by [`tidb_enable_tiflash_pipeline_model`](https://docs.pingcap.com/tidb/v7.2/system-variables#tidb_enable_tiflash_pipeline_model-introduced-since-v720). -- For v7.4.0 and later versions: The pipeline execution model becomes generally available. It is an internal feature of TiFlash and is tightly integrated with TiFlash resource control. When you enable TiFlash resource control, the pipeline execution model is automatically enabled. For more information about how to use TiFlash resource control, refer to [Use resource control to achieve resource isolation](/tidb-resource-control.md#parameters-for-resource-control). Additionally, starting from v7.4.0, the system variable `tidb_enable_tiflash_pipeline_model` is deprecated. +- For v7.4.0 and later versions: The pipeline execution model becomes generally available. It is an internal feature of TiFlash and is tightly integrated with TiFlash resource control. When you enable TiFlash resource control, the pipeline execution model is automatically enabled. For more information about how to use TiFlash resource control, refer to [Use Resource Control to Achieve resource group limitation](/tidb-resource-control.md#parameters-for-resource-control). Additionally, starting from v7.4.0, the system variable `tidb_enable_tiflash_pipeline_model` is deprecated. Inspired by the paper [Morsel-Driven Parallelism: A NUMA-Aware Query Evaluation Framework for the Many-Core Age](https://dl.acm.org/doi/10.1145/2588555.2610507), the TiFlash pipeline execution model provides a fine-grained task scheduling model, which is different from the traditional thread scheduling model. It reduces the overhead of operating system thread application and scheduling and provides a fine-grained scheduling mechanism. diff --git a/user-account-management.md b/user-account-management.md index 8af449abbe3b6..20f80c3260c60 100644 --- a/user-account-management.md +++ b/user-account-management.md @@ -146,7 +146,7 @@ TiDB creates the `'root'@'%'` default account during the database initialization ## Set account resource limits -TiDB can limit the resources consumed by users using resource groups. For more information, see [Use resource control to achieve resource isolation](/tidb-resource-control.md). +TiDB can limit the resources consumed by users using resource groups. For more information, see [Use Resource Control to Achieve resource group limitation](/tidb-resource-control.md). ## Assign account passwords From fa069313d8b7fc8bba0b4ee6674c7d331f66a0d1 Mon Sep 17 00:00:00 2001 From: nolouch Date: Wed, 15 Jan 2025 18:23:52 +0800 Subject: [PATCH 03/25] update Signed-off-by: nolouch --- TOC-tidb-cloud.md | 2 +- TOC.md | 2 +- basic-features.md | 2 +- dashboard/dashboard-intro.md | 2 +- dashboard/dashboard-resource-manager.md | 4 ++-- error-codes.md | 2 +- faq/sql-faq.md | 2 +- functions-and-operators/tidb-functions.md | 6 +++--- glossary.md | 2 +- grafana-resource-control-dashboard.md | 4 ++-- .../information-schema-resource-groups.md | 4 ++-- optimizer-hints.md | 2 +- pd-configuration-file.md | 4 ++-- pd-control.md | 2 +- releases/release-6.6.0.md | 4 ++-- releases/release-7.0.0.md | 14 +++++++------- releases/release-7.1.0.md | 2 +- releases/release-7.4.0.md | 4 ++-- releases/release-7.5.1.md | 4 ++-- releases/release-7.6.0.md | 2 +- releases/release-8.2.0.md | 2 +- releases/release-8.4.0.md | 4 ++-- .../sql-statement-alter-resource-group.md | 4 ++-- .../sql-statement-create-resource-group.md | 4 ++-- .../sql-statement-drop-resource-group.md | 2 +- sql-statements/sql-statement-explain-analyze.md | 2 +- sql-statements/sql-statement-insert.md | 2 +- sql-statements/sql-statement-overview.md | 2 +- sql-statements/sql-statement-replace.md | 2 +- sql-statements/sql-statement-select.md | 2 +- sql-statements/sql-statement-set-resource-group.md | 2 +- .../sql-statement-show-create-resource-group.md | 2 +- sql-statements/sql-statement-update.md | 2 +- statement-summary-tables.md | 2 +- system-variable-reference.md | 10 +++++----- system-variables.md | 8 ++++---- tidb-configuration-file.md | 4 ++-- ...ontrol.md => tidb-resource-control-ru-groups.md | 5 +++-- tidb-resource-control-runaway-queries.md | 10 +++++----- tiflash/tiflash-pipeline-model.md | 2 +- tikv-configuration-file.md | 2 +- tiproxy/tiproxy-load-balance.md | 2 +- tiproxy/tiproxy-traffic-replay.md | 2 +- user-account-management.md | 2 +- 44 files changed, 76 insertions(+), 75 deletions(-) rename tidb-resource-control.md => tidb-resource-control-ru-groups.md (99%) diff --git a/TOC-tidb-cloud.md b/TOC-tidb-cloud.md index 3c4e6036b93b2..ef557816c4de3 100644 --- a/TOC-tidb-cloud.md +++ b/TOC-tidb-cloud.md @@ -736,7 +736,7 @@ - [update](/tidb-cloud/ticloud-update.md) - [help](/tidb-cloud/ticloud-help.md) - [Table Filter](/table-filter.md) - - [Resource Control](/tidb-resource-control.md) + - [Resource Control](/tidb-resource-control-ru-groups.md) - [URI Formats of External Storage Services](/external-storage-uri.md) - [DDL Execution Principles and Best Practices](/ddl-introduction.md) - [Batch Processing](/batch-processing.md) diff --git a/TOC.md b/TOC.md index f9dbc6c47494b..6f254f4bd2193 100644 --- a/TOC.md +++ b/TOC.md @@ -263,7 +263,7 @@ - [Multi-Replica Cluster DR](/dr-multi-replica.md) - [BR-based DR](/dr-backup-restore.md) - Resource Manager - - [Achieve resource group limitation and flow control using Resource Control](/tidb-resource-control.md) + - [Achieve resource group limitation and flow control using Resource Control](/tidb-resource-control-ru-groups.md) - [Runaway Queries with resource consumption exceeding expectations](/tidb-resource-control-runaway-queries.md) - [Limit background task resource usage](/tidb-resource-control-background-tasks.md) - [Configure Time Zone](/configure-time-zone.md) diff --git a/basic-features.md b/basic-features.md index c05e34734589b..dfb8e228332ae 100644 --- a/basic-features.md +++ b/basic-features.md @@ -258,7 +258,7 @@ You can try out TiDB features on [TiDB Playground](https://play.tidbcloud.com/?u | [Global memory control](/configure-memory-usage.md#configure-the-memory-usage-threshold-of-a-tidb-server-instance) | Y | Y | Y | Y | Y | N | N | N | N | N | | [Cross-cluster RawKV replication](/tikv-configuration-file.md#api-version-new-in-v610) | E | E | E | E | E | N | N | N | N | N | | [Green GC](/system-variables.md#tidb_gc_scan_lock_mode-new-in-v50) | E | E | E | E | E | E | E | E | E | E | -| [Resource control](/tidb-resource-control.md) | Y | Y | Y | Y | N | N | N | N | N | N | +| [Resource control](/tidb-resource-control-ru-groups.md) | Y | Y | Y | Y | N | N | N | N | N | N | | [Runaway Queries management](/tidb-resource-control-runaway-queries.md) | Y | Y | E | N | N | N | N | N | N | N | | [Background tasks management](/tidb-resource-control-background-tasks.md) | E | E | E | N | N | N | N | N | N | N | | [TiFlash Disaggregated Storage and Compute Architecture and S3 Support](/tiflash/tiflash-disaggregated-and-s3.md) | Y | Y | Y | E | N | N | N | N | N | N | diff --git a/dashboard/dashboard-intro.md b/dashboard/dashboard-intro.md index e67034328b6f3..4eb006cf733e8 100644 --- a/dashboard/dashboard-intro.md +++ b/dashboard/dashboard-intro.md @@ -62,7 +62,7 @@ See [Search Logs Page](/dashboard/dashboard-log-search.md) for details. ## Estimate cluster capacity for resource control -To implement resource isolation using the [Resource Control](/tidb-resource-control.md) feature, cluster administrators can create resource groups and set quotas for each group. +To implement resource isolation using the [Resource Control](/tidb-resource-control-ru-groups.md) feature, cluster administrators can create resource groups and set quotas for each group. Before resource planning, you need to know the overall capacity of the cluster. For more details, see [Resource Manager page](/dashboard/dashboard-resource-manager.md). diff --git a/dashboard/dashboard-resource-manager.md b/dashboard/dashboard-resource-manager.md index cbf0658ebe766..c1af56c22f963 100644 --- a/dashboard/dashboard-resource-manager.md +++ b/dashboard/dashboard-resource-manager.md @@ -5,7 +5,7 @@ summary: TiDB Dashboard Resource Manager Page helps cluster administrators imple # TiDB Dashboard Resource Manager Page -To implement resource isolation using the [Resource Control](/tidb-resource-control.md) feature, cluster administrators can create resource groups and set quotas for each group. Before resource planning, you need to know the overall capacity of the cluster. This document helps you view the information about resource control, so you can estimate the cluster capacity before resource planning and allocate resources more effectively. +To implement resource isolation using the [Resource Control](/tidb-resource-control-ru-groups.md) feature, cluster administrators can create resource groups and set quotas for each group. Before resource planning, you need to know the overall capacity of the cluster. This document helps you view the information about resource control, so you can estimate the cluster capacity before resource planning and allocate resources more effectively. ## Access the page @@ -34,7 +34,7 @@ The Resource Manager page contains the following three sections: ## Estimate Capacity -Before resource planning, you need to know the overall capacity of the cluster. TiDB provides two methods to estimate the capacity of [Request Unit (RU)](/tidb-resource-control.md#what-is-request-unit-ru#what-is-request-unit-ru) in the current cluster: +Before resource planning, you need to know the overall capacity of the cluster. TiDB provides two methods to estimate the capacity of [Request Unit (RU)](/tidb-resource-control-ru-groups.md#what-is-request-unit-ru#what-is-request-unit-ru) in the current cluster: - [Estimate capacity based on hardware deployment](/sql-statements/sql-statement-calibrate-resource.md#estimate-capacity-based-on-hardware-deployment) diff --git a/error-codes.md b/error-codes.md index 7ee05f40212e2..17279c2e61e92 100644 --- a/error-codes.md +++ b/error-codes.md @@ -484,7 +484,7 @@ TiDB is compatible with the error codes in MySQL, and in most cases returns the * Error Number: 8249 - The resource group does not exist. This error is returned when you modify or bind a resource group that does not exist. See [Create a resource group](/tidb-resource-control.md#create-a-resource-group). + The resource group does not exist. This error is returned when you modify or bind a resource group that does not exist. See [Create a resource group](/tidb-resource-control-ru-groups.md#create-a-resource-group). * Error Number: 8250 diff --git a/faq/sql-faq.md b/faq/sql-faq.md index 50298bf695b55..bc33b9520ee9c 100644 --- a/faq/sql-faq.md +++ b/faq/sql-faq.md @@ -211,7 +211,7 @@ TiDB supports changing the priority on a [global](/system-variables.md#tidb_forc > **Note:** > -> Starting from v6.6.0, TiDB supports [Resource Control](/tidb-resource-control.md). You can use this feature to execute SQL statements with different priorities in different resource groups. By configuring proper quotas and priorities for these resource groups, you can gain better scheduling control for SQL statements with different priorities. When resource control is enabled, statement priority will no longer take effect. It is recommended that you use [Resource Control](/tidb-resource-control.md) to manage resource usage for different SQL statements. +> Starting from v6.6.0, TiDB supports [Resource Control](/tidb-resource-control-ru-groups.md). You can use this feature to execute SQL statements with different priorities in different resource groups. By configuring proper quotas and priorities for these resource groups, you can gain better scheduling control for SQL statements with different priorities. When resource control is enabled, statement priority will no longer take effect. It is recommended that you use [Resource Control](/tidb-resource-control-ru-groups.md) to manage resource usage for different SQL statements. You can combine the above two parameters with the DML of TiDB to use them. For example: diff --git a/functions-and-operators/tidb-functions.md b/functions-and-operators/tidb-functions.md index b967cb556b2c4..05db30eb8e5a6 100644 --- a/functions-and-operators/tidb-functions.md +++ b/functions-and-operators/tidb-functions.md @@ -11,7 +11,7 @@ The following functions are TiDB extensions, and are not present in MySQL: | Function name | Function description | | :-------------- | :------------------------------------- | -| [`CURRENT_RESOURCE_GROUP()`](#current_resource_group) | Returns the name of the resource group that the current session is bound to. See [using resource control to achieve resource isolation](/tidb-resource-control.md). | +| [`CURRENT_RESOURCE_GROUP()`](#current_resource_group) | Returns the name of the resource group that the current session is bound to. See [using resource control to achieve resource isolation](/tidb-resource-control-ru-groups.md). | | [`TIDB_BOUNDED_STALENESS()`](#tidb_bounded_staleness) | Instructs TiDB to read the most recent data within a specified time range. See [reading historical data using the `AS OF TIMESTAMP` clause](/as-of-timestamp.md). | | [`TIDB_CURRENT_TSO()`](#tidb_current_tso) | Returns the current [TimeStamp Oracle (TSO) in TiDB](/tso.md). | | [`TIDB_DECODE_BINARY_PLAN()`](#tidb_decode_binary_plan) | Decodes binary plans. | @@ -36,7 +36,7 @@ The following functions are TiDB extensions, and are not present in MySQL: | Function name | Function description | | :-------------- | :------------------------------------- | -| [`CURRENT_RESOURCE_GROUP()`](#current_resource_group) | Returns the resource group name that the current session is bound to. See [using resource control to achieve resource isolation](/tidb-resource-control.md). | +| [`CURRENT_RESOURCE_GROUP()`](#current_resource_group) | Returns the resource group name that the current session is bound to. See [using resource control to achieve resource isolation](/tidb-resource-control-ru-groups.md). | | [`TIDB_BOUNDED_STALENESS()`](#tidb_bounded_staleness) | Instructs TiDB to read most recent data within a specified time range. See [reading historical data using the `AS OF TIMESTAMP` clause](/as-of-timestamp.md). | | [`TIDB_CURRENT_TSO()`](#tidb_current_tso) | Returns the current [TimeStamp Oracle (TSO) in TiDB](/tso.md). | | [`TIDB_DECODE_BINARY_PLAN()`](#tidb_decode_binary_plan) | Decodes binary plans. | @@ -59,7 +59,7 @@ The following functions are TiDB extensions, and are not present in MySQL: ## CURRENT_RESOURCE_GROUP -The `CURRENT_RESOURCE_GROUP()` function is used to show the resource group name that the current session is bound to. When the [Resource control](/tidb-resource-control.md) feature is enabled, the available resources that can be used by SQL statements are restricted by the resource quota of the bound resource group. +The `CURRENT_RESOURCE_GROUP()` function is used to show the resource group name that the current session is bound to. When the [Resource control](/tidb-resource-control-ru-groups.md) feature is enabled, the available resources that can be used by SQL statements are restricted by the resource quota of the bound resource group. When a session is established, TiDB binds the session to the resource group that the login user is bound to by default. If the user is not bound to any resource groups, the session is bound to the `default` resource group. Once the session is established, the bound resource group will not change by default, even if the bound resource group of the user is changed via [modifying the resource group bound to the user](/sql-statements/sql-statement-alter-user.md#modify-basic-user-information). To change the bound resource group of the current session, you can use [`SET RESOURCE GROUP`](/sql-statements/sql-statement-set-resource-group.md). diff --git a/glossary.md b/glossary.md index 51e612e6fb759..584490c674f04 100644 --- a/glossary.md +++ b/glossary.md @@ -259,7 +259,7 @@ Remote Procedure Call (RPC) is a communication way between software components. ### Request Unit (RU) -Request Unit (RU) is a unified abstraction unit for the resource usage in TiDB. It is used with [Resource Control](/tidb-resource-control.md) to manage resource usage. +Request Unit (RU) is a unified abstraction unit for the resource usage in TiDB. It is used with [Resource Control](/tidb-resource-control-ru-groups.md) to manage resource usage. ### Restore diff --git a/grafana-resource-control-dashboard.md b/grafana-resource-control-dashboard.md index 3cca223262ba3..7eda64ecfeb22 100644 --- a/grafana-resource-control-dashboard.md +++ b/grafana-resource-control-dashboard.md @@ -9,7 +9,7 @@ If you use TiUP to deploy the TiDB cluster, the monitoring system (Prometheus & The Grafana dashboard is divided into a series of sub dashboards which include Overview, PD, TiDB, TiKV, Node\_exporter, Disk Performance, and Performance\_overview. -If your cluster has used the [Resource Control](/tidb-resource-control.md) feature, you can get an overview of the resource consumption status from the Resource Control dashboard. +If your cluster has used the [Resource Control](/tidb-resource-control-ru-groups.md) feature, you can get an overview of the resource consumption status from the Resource Control dashboard. TiDB uses the [token bucket algorithm](https://en.wikipedia.org/wiki/Token_bucket) for flow control. As described in the [RFC: Global Resource Control in TiDB](https://github.com/pingcap/tidb/blob/master/docs/design/2022-11-25-global-resource-control.md#distributed-token-buckets), a TiDB node might have multiple Resource Groups, which are flow controlled by GAC (Global Admission Control) on the PD side. The Local Token Buckets in each TiDB node periodically (5 seconds by default) communicate with the GAC on the PD side to reconfigure the local tokens. In TiDB, the Local Token Buckets are implemented as Resource Controller Clients. @@ -17,7 +17,7 @@ This document describes some key monitoring metrics displayed on the Resource Co ## Metrics about Request Unit -- RU: the [Request Unit (RU)](/tidb-resource-control.md#what-is-request-unit-ru) consumption information of each resource group, calculated in real time. `total` is the sum of the Request Units consumed by all Resource Groups. The Request Unit consumption of each resource group should be equal to the sum of its read consumption (Read Request Unit) and write consumption (Write Request Unit). +- RU: the [Request Unit (RU)](/tidb-resource-control-ru-groups.md#what-is-request-unit-ru) consumption information of each resource group, calculated in real time. `total` is the sum of the Request Units consumed by all Resource Groups. The Request Unit consumption of each resource group should be equal to the sum of its read consumption (Read Request Unit) and write consumption (Write Request Unit). - RU Per Query: the average number of Request Units consumed by each SQL statement per second. It is obtained by dividing the above RU metric by the number of SQL statements executed per second. - RRU: the Read Request Unit consumption information of each resource group, calculated in real time. `total` is the sum of the Read Request Units consumed by all Resource Groups. - RRU Per Query: the average number of Read Request Units consumed by each SQL statement per second. It is obtained by dividing the above RRU metric by the number of SQL statements executed per second. diff --git a/information-schema/information-schema-resource-groups.md b/information-schema/information-schema-resource-groups.md index 3d54953fd92d8..2916c058baa70 100644 --- a/information-schema/information-schema-resource-groups.md +++ b/information-schema/information-schema-resource-groups.md @@ -5,7 +5,7 @@ summary: Learn the `RESOURCE_GROUPS` information_schema table. # RESOURCE_GROUPS -The `RESOURCE_GROUPS` table shows the information about all resource groups. For more information, see [Use Resource Control to Achieve Resource Isolation](/tidb-resource-control.md). +The `RESOURCE_GROUPS` table shows the information about all resource groups. For more information, see [Use Resource Control to Achieve Resource Group Limitation](/tidb-resource-control-ru-groups.md). > **Note:** > @@ -79,7 +79,7 @@ SELECT * FROM information_schema.resource_groups WHERE NAME = 'rg1'; -- View the The descriptions of the columns in the `RESOURCE_GROUPS` table are as follows: * `NAME`: the name of the resource group. -* `RU_PER_SEC`: the backfilling speed of the resource group. The unit is RU/second, in which RU means [Request Unit](/tidb-resource-control.md#what-is-request-unit-ru). +* `RU_PER_SEC`: the backfilling speed of the resource group. The unit is RU/second, in which RU means [Request Unit](/tidb-resource-control-ru-groups.md#what-is-request-unit-ru). * `PRIORITY`: the absolute priority of tasks to be processed on TiKV. Different resources are scheduled according to the `PRIORITY` setting. Tasks with high `PRIORITY` are scheduled first. For resource groups with the same `PRIORITY`, tasks will be scheduled proportionally according to the `RU_PER_SEC` configuration. If `PRIORITY` is not specified, the default priority is `MEDIUM`. * `BURSTABLE`: whether to allow the resource group to overuse the available system resources. diff --git a/optimizer-hints.md b/optimizer-hints.md index d3468c85f89f7..4c26637579bcb 100644 --- a/optimizer-hints.md +++ b/optimizer-hints.md @@ -851,7 +851,7 @@ SELECT /*+ NTH_PLAN(3) */ count(*) from t where a > 5; ### RESOURCE_GROUP(resource_group_name) -`RESOURCE_GROUP(resource_group_name)` is used for [Resource Control](/tidb-resource-control.md) to isolate resources. This hint temporarily executes the current statement using the specified resource group. If the specified resource group does not exist, this hint will be ignored. +`RESOURCE_GROUP(resource_group_name)` is used for [Resource Control](/tidb-resource-control-ru-groups.md) to isolate resources. This hint temporarily executes the current statement using the specified resource group. If the specified resource group does not exist, this hint will be ignored. Example: diff --git a/pd-configuration-file.md b/pd-configuration-file.md index 0600b360b8d29..883e9c056dc9f 100644 --- a/pd-configuration-file.md +++ b/pd-configuration-file.md @@ -503,7 +503,7 @@ Configuration items related to the replication mode of all Regions. See [Enable ## controller -This section describes the configuration items that are built into PD for [Resource Control](/tidb-resource-control.md). +This section describes the configuration items that are built into PD for [Resource Control](/tidb-resource-control-ru-groups.md). ### `degraded-mode-wait-duration` @@ -513,7 +513,7 @@ This section describes the configuration items that are built into PD for [Resou ### `request-unit` -The following are the configuration items about the [Request Unit (RU)](/tidb-resource-control.md#what-is-request-unit-ru). +The following are the configuration items about the [Request Unit (RU)](/tidb-resource-control-ru-groups.md#what-is-request-unit-ru). #### `read-base-cost` diff --git a/pd-control.md b/pd-control.md index 07fc485c24906..f3a6b9efe2859 100644 --- a/pd-control.md +++ b/pd-control.md @@ -919,7 +919,7 @@ resource-manager config controller show } ``` -- `ltb-max-wait-duration`: the maximum waiting time of Local Token Bucket (LTB). The default value is `30s`, and the value range is `[0, 24h]`. If the estimated [Request Unit (RU)](/tidb-resource-control.md#what-is-request-unit-ru) consumption of the SQL request exceeds the current accumulated RU of LTB, the request needs to wait for a certain period of time. If the estimated waiting time exceeds this maximum value, an error message [`ERROR 8252 (HY000) : Exceeded resource group quota limitation`](/error-codes.md) is returned to the application in advance. Increasing this value can reduce the occurrence of encountering `ERROR 8252` in cases of sudden concurrency increase, large transactions, and large queries. +- `ltb-max-wait-duration`: the maximum waiting time of Local Token Bucket (LTB). The default value is `30s`, and the value range is `[0, 24h]`. If the estimated [Request Unit (RU)](/tidb-resource-control-ru-groups.md#what-is-request-unit-ru) consumption of the SQL request exceeds the current accumulated RU of LTB, the request needs to wait for a certain period of time. If the estimated waiting time exceeds this maximum value, an error message [`ERROR 8252 (HY000) : Exceeded resource group quota limitation`](/error-codes.md) is returned to the application in advance. Increasing this value can reduce the occurrence of encountering `ERROR 8252` in cases of sudden concurrency increase, large transactions, and large queries. - `enable-controller-trace-log`: control whether to enable the controller diagnostic log. #### Modify the controller configuration of Resource Control diff --git a/releases/release-6.6.0.md b/releases/release-6.6.0.md index 53587d626ee76..d9ce0c9baeac0 100644 --- a/releases/release-6.6.0.md +++ b/releases/release-6.6.0.md @@ -134,9 +134,9 @@ In v6.6.0-DMR, the key new features and improvements are as follows: In addition, the rational use of the resource control feature can reduce the number of clusters, ease the difficulty of operation and maintenance, and save management costs. - In v6.6, you need to enable both TiDB's global variable [`tidb_enable_resource_control`](/system-variables.md#tidb_enable_resource_control-new-in-v660) and the TiKV configuration item [`resource-control.enabled`](/tikv-configuration-file.md#resource-control) to enable resource control. Currently, the supported quota method is based on "[Request Unit (RU)](/tidb-resource-control.md#what-is-request-unit-ru)". RU is TiDB's unified abstraction unit for system resources such as CPU and IO. + In v6.6, you need to enable both TiDB's global variable [`tidb_enable_resource_control`](/system-variables.md#tidb_enable_resource_control-new-in-v660) and the TiKV configuration item [`resource-control.enabled`](/tikv-configuration-file.md#resource-control) to enable resource control. Currently, the supported quota method is based on "[Request Unit (RU)](/tidb-resource-control-ru-groups.md#what-is-request-unit-ru)". RU is TiDB's unified abstraction unit for system resources such as CPU and IO. - For more information, see [documentation](/tidb-resource-control.md). + For more information, see [documentation](/tidb-resource-control-ru-groups.md). * Binding historical execution plans is GA [#39199](https://github.com/pingcap/tidb/issues/39199) @[fzzf678](https://github.com/fzzf678) diff --git a/releases/release-7.0.0.md b/releases/release-7.0.0.md index db275cb89e8b0..315cac3020e4d 100644 --- a/releases/release-7.0.0.md +++ b/releases/release-7.0.0.md @@ -162,7 +162,7 @@ In v7.0.0-DMR, the key new features and improvements are as follows: - Session level. Set the resource group used by the current session via [`SET RESOURCE GROUP`](/sql-statements/sql-statement-set-resource-group.md). - Statement level. Set the resource group used by the current statement via [`RESOURCE_GROUP()`](/optimizer-hints.md#resource_groupresource_group_name). - For more information, see [documentation](/tidb-resource-control.md). + For more information, see [documentation](/tidb-resource-control-ru-groups.md). * Support a checkpoint mechanism for Fast Online DDL, improving fault tolerance and automatic recovery capability [#42164](https://github.com/pingcap/tidb/issues/42164) @[tangenta](https://github.com/tangenta) @@ -356,12 +356,12 @@ In v7.0.0-DMR, the key new features and improvements are as follows: | TiKV | [`resolved-ts.advance-ts-interval`](/tikv-configuration-file.md#advance-ts-interval) | Modified | The default value changes from `"1s"` to `"20s"`. This modification can increase the interval of the regular advancement of Resolved TS and reduce the traffic consumption between TiKV nodes. | | TiKV | [`resource-control.enabled`](/tikv-configuration-file.md#resource-control) | Modified | The default value changes from `false` to `true`. | | TiKV | [`raft-engine.prefill-for-recycle`](/tikv-configuration-file.md#prefill-for-recycle-new-in-v700) | Newly added | Controls whether to generate empty log files for log recycling in Raft Engine. The default value is `false`. | -| PD | [`degraded-mode-wait-duration`](/pd-configuration-file.md#degraded-mode-wait-duration) | Newly added | A [Resource Control](/tidb-resource-control.md)-related configuration item. It controls the waiting time for triggering the degraded mode. The default value is `0s`. | -| PD | [`read-base-cost`](/pd-configuration-file.md#read-base-cost) | Newly added | A [Resource Control](/tidb-resource-control.md)-related configuration item. It controls the basis factor for conversion from a read request to RU. The default value is `0.25`. | -| PD | [`read-cost-per-byte`](/pd-configuration-file.md#read-cost-per-byte) | Newly added | A [Resource Control](/tidb-resource-control.md)-related configuration item. It controls the basis factor for conversion from read flow to RU. The default value is `1/ (64 * 1024)`. | -| PD | [`read-cpu-ms-cost`](/pd-configuration-file.md#read-cpu-ms-cost) | Newly added | A [Resource Control](/tidb-resource-control.md)-related configuration item. It controls the basis factor for conversion from CPU to RU. The default value is `1/3`. | -| PD | [`write-base-cost`](/pd-configuration-file.md#write-base-cost) | Newly added | A [Resource Control](/tidb-resource-control.md)-related configuration item. It controls the basis factor for conversion from a write request to RU. The default value is `1`. | -| PD | [`write-cost-per-byte`](/pd-configuration-file.md#write-cost-per-byte) | Newly added | A [Resource Control](/tidb-resource-control.md)-related configuration item. It controls the basis factor for conversion from write flow to RU. The default value is `1/1024`. | +| PD | [`degraded-mode-wait-duration`](/pd-configuration-file.md#degraded-mode-wait-duration) | Newly added | A [Resource Control](/tidb-resource-control-ru-groups.md)-related configuration item. It controls the waiting time for triggering the degraded mode. The default value is `0s`. | +| PD | [`read-base-cost`](/pd-configuration-file.md#read-base-cost) | Newly added | A [Resource Control](/tidb-resource-control-ru-groups.md)-related configuration item. It controls the basis factor for conversion from a read request to RU. The default value is `0.25`. | +| PD | [`read-cost-per-byte`](/pd-configuration-file.md#read-cost-per-byte) | Newly added | A [Resource Control](/tidb-resource-control-ru-groups.md)-related configuration item. It controls the basis factor for conversion from read flow to RU. The default value is `1/ (64 * 1024)`. | +| PD | [`read-cpu-ms-cost`](/pd-configuration-file.md#read-cpu-ms-cost) | Newly added | A [Resource Control](/tidb-resource-control-ru-groups.md)-related configuration item. It controls the basis factor for conversion from CPU to RU. The default value is `1/3`. | +| PD | [`write-base-cost`](/pd-configuration-file.md#write-base-cost) | Newly added | A [Resource Control](/tidb-resource-control-ru-groups.md)-related configuration item. It controls the basis factor for conversion from a write request to RU. The default value is `1`. | +| PD | [`write-cost-per-byte`](/pd-configuration-file.md#write-cost-per-byte) | Newly added | A [Resource Control](/tidb-resource-control-ru-groups.md)-related configuration item. It controls the basis factor for conversion from write flow to RU. The default value is `1/1024`. | | TiFlash | [`mark_cache_size`](/tiflash/tiflash-configuration.md) | Modified | Change the default cache limit of the metadata for a data block in TiFlash from `5368709120` to `1073741824` to reduce unnecessary memory usage. | | TiFlash | [`minmax_index_cache_size`](/tiflash/tiflash-configuration.md) | Modified | Change the default cache limit of the min-max index for a data block in TiFlash from `5368709120` to `1073741824` to reduce unnecessary memory usage. | | TiFlash | [`flash.disaggregated_mode`](/tiflash/tiflash-disaggregated-and-s3.md) | Newly added | In the disaggregated architecture of TiFlash, it indicates whether this TiFlash node is a write node or a compute node. The value can be `tiflash_write` or `tiflash_compute`. | diff --git a/releases/release-7.1.0.md b/releases/release-7.1.0.md index d228392800fa8..e53525e28fc6e 100644 --- a/releases/release-7.1.0.md +++ b/releases/release-7.1.0.md @@ -152,7 +152,7 @@ Compared with the previous LTS 6.5.0, 7.1.0 not only includes new features, impr To improve user experience, TiDB Dashboard provides the [Resource Manager page](/dashboard/dashboard-resource-manager.md). You can view the resource group configuration on this page and estimate cluster capacity in a visual way to facilitate reasonable resource allocation. - For more information, see [documentation](/tidb-resource-control.md). + For more information, see [documentation](/tidb-resource-control-ru-groups.md). * Support the checkpoint mechanism for Fast Online DDL to improve fault tolerance and automatic recovery capability [#42164](https://github.com/pingcap/tidb/issues/42164) @[tangenta](https://github.com/tangenta) diff --git a/releases/release-7.4.0.md b/releases/release-7.4.0.md index 9643476f061f3..7d28922f9fd18 100644 --- a/releases/release-7.4.0.md +++ b/releases/release-7.4.0.md @@ -154,7 +154,7 @@ Quick access: [Quick start](https://docs.pingcap.com/tidb/v7.4/quick-start-with- To control whether to enable the TiFlash resource control feature, you can configure the TiFlash parameter `enable_resource_control`. After enabling this feature, TiFlash performs resource scheduling and management based on the resource group configuration of TiDB, ensuring the reasonable allocation and use of overall resources. - For more information, see [documentation](/tidb-resource-control.md). + For more information, see [documentation](/tidb-resource-control-ru-groups.md). * TiFlash supports the pipeline execution model (GA) [#6518](https://github.com/pingcap/tiflash/issues/6518) @[SeaRise](https://github.com/SeaRise) @@ -306,7 +306,7 @@ Quick access: [Quick start](https://docs.pingcap.com/tidb/v7.4/quick-start-with- | [`tidb_cloud_storage_uri`](/system-variables.md#tidb_cloud_storage_uri-new-in-v740) | Newly added | Specifies the cloud storage URI to enable [Global Sort](/tidb-global-sort.md). | | [`tidb_opt_enable_hash_join`](/system-variables.md#tidb_opt_enable_hash_join-new-in-v656-v712-and-v740) | Newly added | Controls whether the optimizer will select hash joins for tables. The value is `ON` by default. If set to `OFF`, the optimizer avoids selecting a hash join of a table unless there is no other execution plan available. | | [`tidb_opt_objective`](/system-variables.md#tidb_opt_objective-new-in-v740) | Newly added | This variable controls the objective of the optimizer. `moderate` maintains the default behavior in versions prior to TiDB v7.4.0, where the optimizer tries to use more information to generate better execution plans. `determinate` mode tends to be more conservative and makes the execution plan more stable. | -| [`tidb_request_source_type`](/system-variables.md#tidb_request_source_type-new-in-v740) | Newly added | Explicitly specifies the task type for the current session, which is identified and controlled by [Resource Control](/tidb-resource-control.md). For example: `SET @@tidb_request_source_type = "background"`. | +| [`tidb_request_source_type`](/system-variables.md#tidb_request_source_type-new-in-v740) | Newly added | Explicitly specifies the task type for the current session, which is identified and controlled by [Resource Control](/tidb-resource-control-ru-groups.md). For example: `SET @@tidb_request_source_type = "background"`. | | [`tidb_schema_version_cache_limit`](/system-variables.md#tidb_schema_version_cache_limit-new-in-v740) | Newly added | This variable limits how many historical schema versions can be cached in a TiDB instance. The default value is `16`, which means that TiDB caches 16 historical schema versions by default. | | [`tidb_service_scope`](/system-variables.md#tidb_service_scope-new-in-v740) | Newly added | This variable is an instance-level system variable. You can use it to control the service scope of TiDB nodes under the [TiDB Distributed eXecution Framework (DXF)](/tidb-distributed-execution-framework.md). When you set `tidb_service_scope` of a TiDB node to `background`, the DXF schedules that TiDB node to execute DXF tasks, such as [`ADD INDEX`](/sql-statements/sql-statement-add-index.md) and [`IMPORT INTO`](/sql-statements/sql-statement-import-into.md). | | [`tidb_session_alias`](/system-variables.md#tidb_session_alias-new-in-v740) | Newly added | Controls the value of the `session_alias` column in the logs related to the current session. | diff --git a/releases/release-7.5.1.md b/releases/release-7.5.1.md index e519214530e67..fe48fefc2a9bf 100644 --- a/releases/release-7.5.1.md +++ b/releases/release-7.5.1.md @@ -32,7 +32,7 @@ Quick access: [Quick start](https://docs.pingcap.com/tidb/v7.5/quick-start-with- - [Slow Queries](/identify-slow-queries.md): add the resource group name, resource unit (RU) consumption, and time for waiting for resources. - [Statement Summary Tables](/statement-summary-tables.md): add the resource group name, RU consumption, and time for waiting for resources. - - In the system variable [`tidb_last_query_info`](/system-variables.md#tidb_last_query_info-new-in-v4014), add a new entry `ru_consumption` to indicate the consumed [RU](/tidb-resource-control.md#what-is-request-unit-ru) by SQL statements. You can use this variable to get the resource consumption of the last statement in the session. + - In the system variable [`tidb_last_query_info`](/system-variables.md#tidb_last_query_info-new-in-v4014), add a new entry `ru_consumption` to indicate the consumed [RU](/tidb-resource-control-ru-groups.md#what-is-request-unit-ru) by SQL statements. You can use this variable to get the resource consumption of the last statement in the session. - Add database metrics based on resource groups: QPS/TPS, execution time (P999/P99/P95), number of failures, and number of connections. - Modify the `CANCEL IMPORT JOB` statement to a synchronous statement [#48736](https://github.com/pingcap/tidb/issues/48736) @[D3Hunter](https://github.com/D3Hunter) @@ -49,7 +49,7 @@ Quick access: [Quick start](https://docs.pingcap.com/tidb/v7.5/quick-start-with- + TiFlash - - Improve the calculation method for [Request Unit (RU)](/tidb-resource-control.md#what-is-request-unit-ru) to make RU values more stable [#8391](https://github.com/pingcap/tiflash/issues/8391) @[guo-shaoge](https://github.com/guo-shaoge) + - Improve the calculation method for [Request Unit (RU)](/tidb-resource-control-ru-groups.md#what-is-request-unit-ru) to make RU values more stable [#8391](https://github.com/pingcap/tiflash/issues/8391) @[guo-shaoge](https://github.com/guo-shaoge) - Reduce the impact of disk performance jitter on read latency [#8583](https://github.com/pingcap/tiflash/issues/8583) @[JaySon-Huang](https://github.com/JaySon-Huang) - Reduce the impact of background GC tasks on read and write task latency [#8650](https://github.com/pingcap/tiflash/issues/8650) @[JaySon-Huang](https://github.com/JaySon-Huang) diff --git a/releases/release-7.6.0.md b/releases/release-7.6.0.md index 2085468ad58da..85cba0a3b1e69 100644 --- a/releases/release-7.6.0.md +++ b/releases/release-7.6.0.md @@ -220,7 +220,7 @@ Quick access: [Quick start](https://docs.pingcap.com/tidb/v7.6/quick-start-with- * [Slow Queries](/identify-slow-queries.md): add the resource group name, resource unit (RU) consumption, and time for waiting for resources. * [Statement Summary Tables](/statement-summary-tables.md): add the resource group name, RU consumption, and time for waiting for resources. - * In the system variable [`tidb_last_query_info`](/system-variables.md#tidb_last_query_info-new-in-v4014), add a new entry `ru_consumption` to indicate the consumed [RU](/tidb-resource-control.md#what-is-request-unit-ru) by SQL statements. You can use this variable to get the resource consumption of the last statement in the session. + * In the system variable [`tidb_last_query_info`](/system-variables.md#tidb_last_query_info-new-in-v4014), add a new entry `ru_consumption` to indicate the consumed [RU](/tidb-resource-control-ru-groups.md#what-is-request-unit-ru) by SQL statements. You can use this variable to get the resource consumption of the last statement in the session. * Add database metrics based on resource groups: QPS/TPS, execution time (P999/P99/P95), number of failures, and number of connections. * Add the system table [`request_unit_by_group`](/mysql-schema/mysql-schema.md#system-tables-related-to-resource-control) to record the history records of daily consumed RUs of all resource groups. diff --git a/releases/release-8.2.0.md b/releases/release-8.2.0.md index 9d5caa6507687..ce44f4b5ee990 100644 --- a/releases/release-8.2.0.md +++ b/releases/release-8.2.0.md @@ -119,7 +119,7 @@ Quick access: [Quick start](https://docs.pingcap.com/tidb/v8.2/quick-start-with- To maintain compatibility, the original behavior is retained when upgrading from earlier versions to v8.2.0 or later versions. To enable the enhanced privilege control, set the new variable [`tidb_resource_control_strict_mode`](/system-variables.md#tidb_resource_control_strict_mode-new-in-v820) to `ON`. - For more information, see [user documentation](/tidb-resource-control.md#bind-resource-groups). + For more information, see [user documentation](/tidb-resource-control-ru-groups.md#bind-resource-groups). ### Observability diff --git a/releases/release-8.4.0.md b/releases/release-8.4.0.md index 73e40d33abd02..3038ff0b199c9 100644 --- a/releases/release-8.4.0.md +++ b/releases/release-8.4.0.md @@ -154,9 +154,9 @@ Quick access: [Quick start](https://docs.pingcap.com/tidb/v8.4/quick-start-with- * Support switching resource groups for runaway queries [#54434](https://github.com/pingcap/tidb/issues/54434) @[JmPotato](https://github.com/JmPotato) - Starting from TiDB v8.4.0, you can switch the resource group of runaway queries to a specific one. If the `COOLDOWN` mechanism fails to lower resource consumption, you can create a [resource group](/tidb-resource-control.md#create-a-resource-group), limit its resource size, and set the `SWITCH_GROUP` parameter to move identified runaway queries to this group. Meanwhile, subsequent queries within the same session will continue to execute in the original resource group. By switching resource groups, you can manage resource usage more precisely, and control the resource consumption more strictly. + Starting from TiDB v8.4.0, you can switch the resource group of runaway queries to a specific one. If the `COOLDOWN` mechanism fails to lower resource consumption, you can create a [resource group](/tidb-resource-control-ru-groups.md#create-a-resource-group), limit its resource size, and set the `SWITCH_GROUP` parameter to move identified runaway queries to this group. Meanwhile, subsequent queries within the same session will continue to execute in the original resource group. By switching resource groups, you can manage resource usage more precisely, and control the resource consumption more strictly. - For more information, see [documentation](/tidb-resource-control.md#query_limit-parameters). + For more information, see [documentation](/tidb-resource-control-runaway-queries.md#query_limit-parameters). * Support setting the cluster-level Region scattering strategy using the `tidb_scatter_region` system variable [#55184](https://github.com/pingcap/tidb/issues/55184) @[D3Hunter](https://github.com/D3Hunter) diff --git a/sql-statements/sql-statement-alter-resource-group.md b/sql-statements/sql-statement-alter-resource-group.md index 65bb0bbfcc948..005cd9e6b1c24 100644 --- a/sql-statements/sql-statement-alter-resource-group.md +++ b/sql-statements/sql-statement-alter-resource-group.md @@ -78,7 +78,7 @@ DirectBackgroundOption ::= | "UTILIZATION_LIMIT" EqOpt LengthNum ``` -TiDB supports the following `DirectResourceGroupOption`, where [Request Unit (RU)](/tidb-resource-control.md#what-is-request-unit-ru) is a unified abstraction unit in TiDB for CPU, IO, and other system resources. +TiDB supports the following `DirectResourceGroupOption`, where [Request Unit (RU)](/tidb-resource-control-ru-groups.md#what-is-request-unit-ru) is a unified abstraction unit in TiDB for CPU, IO, and other system resources. | Option | Description | Example | |---------------|-------------------------------------|------------------------| @@ -184,4 +184,4 @@ MySQL also supports [ALTER RESOURCE GROUP](https://dev.mysql.com/doc/refman/8.0/ * [DROP RESOURCE GROUP](/sql-statements/sql-statement-drop-resource-group.md) * [CREATE RESOURCE GROUP](/sql-statements/sql-statement-create-resource-group.md) -* [Request Unit (RU)](/tidb-resource-control.md#what-is-request-unit-ru) +* [Request Unit (RU)](/tidb-resource-control-ru-groups.md#what-is-request-unit-ru) diff --git a/sql-statements/sql-statement-create-resource-group.md b/sql-statements/sql-statement-create-resource-group.md index dd48f0a686b71..2916f30989c60 100644 --- a/sql-statements/sql-statement-create-resource-group.md +++ b/sql-statements/sql-statement-create-resource-group.md @@ -75,7 +75,7 @@ ResourceGroupRunawayActionOption ::= The resource group name parameter (`ResourceGroupName`) must be globally unique. -TiDB supports the following `DirectResourceGroupOption`, where [Request Unit (RU)](/tidb-resource-control.md#what-is-request-unit-ru) is a unified abstraction unit in TiDB for CPU, IO, and other system resources. +TiDB supports the following `DirectResourceGroupOption`, where [Request Unit (RU)](/tidb-resource-control-ru-groups.md#what-is-request-unit-ru) is a unified abstraction unit in TiDB for CPU, IO, and other system resources. | Option | Description | Example | |---------------|-------------------------------------|------------------------| @@ -145,4 +145,4 @@ MySQL also supports [CREATE RESOURCE GROUP](https://dev.mysql.com/doc/refman/8.0 * [DROP RESOURCE GROUP](/sql-statements/sql-statement-drop-resource-group.md) * [ALTER RESOURCE GROUP](/sql-statements/sql-statement-alter-resource-group.md) * [ALTER USER RESOURCE GROUP](/sql-statements/sql-statement-alter-user.md#modify-the-resource-group-bound-to-the-user) -* [Request Unit (RU)](/tidb-resource-control.md#what-is-request-unit-ru) +* [Request Unit (RU)](/tidb-resource-control-ru-groups.md#what-is-request-unit-ru) diff --git a/sql-statements/sql-statement-drop-resource-group.md b/sql-statements/sql-statement-drop-resource-group.md index ac4d380534293..eb1df28582c01 100644 --- a/sql-statements/sql-statement-drop-resource-group.md +++ b/sql-statements/sql-statement-drop-resource-group.md @@ -87,4 +87,4 @@ MySQL also supports [DROP RESOURCE GROUP](https://dev.mysql.com/doc/refman/8.0/e * [ALTER RESOURCE GROUP](/sql-statements/sql-statement-alter-resource-group.md) * [CREATE RESOURCE GROUP](/sql-statements/sql-statement-create-resource-group.md) -* [Request Unit (RU)](/tidb-resource-control.md#what-is-request-unit-ru) +* [Request Unit (RU)](/tidb-resource-control-ru-groups.md#what-is-request-unit-ru) diff --git a/sql-statements/sql-statement-explain-analyze.md b/sql-statements/sql-statement-explain-analyze.md index 1622e084a95a0..5e311878dbd5f 100644 --- a/sql-statements/sql-statement-explain-analyze.md +++ b/sql-statements/sql-statement-explain-analyze.md @@ -284,7 +284,7 @@ commit_txn: {prewrite:48.564544ms, wait_prewrite_binlog:47.821579, get_commit_ts ### RU (Request Unit) consumption -[Request Unit (RU)](/tidb-resource-control.md#what-is-request-unit-ru) is a unified abstraction unit of system resources, which is defined in TiDB resource control. The `execution info` of the top-level operator shows the overall RU consumption of this particular SQL statement. +[Request Unit (RU)](/tidb-resource-control-ru-groups.md#what-is-request-unit-ru) is a unified abstraction unit of system resources, which is defined in TiDB resource control. The `execution info` of the top-level operator shows the overall RU consumption of this particular SQL statement. ``` RU:273.842670 diff --git a/sql-statements/sql-statement-insert.md b/sql-statements/sql-statement-insert.md index 4c17e03119a10..21fb30783cfdd 100644 --- a/sql-statements/sql-statement-insert.md +++ b/sql-statements/sql-statement-insert.md @@ -44,7 +44,7 @@ OnDuplicateKeyUpdate ::= > **Note:** > -> Starting from v6.6.0, TiDB supports [Resource Control](/tidb-resource-control.md). You can use this feature to execute SQL statements with different priorities in different resource groups. By configuring proper quotas and priorities for these resource groups, you can gain better scheduling control for SQL statements with different priorities. When resource control is enabled, statement priority (`PriorityOpt`) will no longer take effect. It is recommended that you use [Resource Control](/tidb-resource-control.md) to manage resource usage for different SQL statements. +> Starting from v6.6.0, TiDB supports [Resource Control](/tidb-resource-control-ru-groups.md). You can use this feature to execute SQL statements with different priorities in different resource groups. By configuring proper quotas and priorities for these resource groups, you can gain better scheduling control for SQL statements with different priorities. When resource control is enabled, statement priority (`PriorityOpt`) will no longer take effect. It is recommended that you use [Resource Control](/tidb-resource-control-ru-groups.md) to manage resource usage for different SQL statements. ## Examples diff --git a/sql-statements/sql-statement-overview.md b/sql-statements/sql-statement-overview.md index 696918c9b6470..dc097058b9975 100644 --- a/sql-statements/sql-statement-overview.md +++ b/sql-statements/sql-statement-overview.md @@ -163,7 +163,7 @@ TiDB uses SQL statements that aim to follow ISO/IEC SQL standards, with extensio | SQL Statement | Description | |---------------|-------------| | [`ALTER RESOURCE GROUP`](/sql-statements/sql-statement-alter-resource-group.md) | Modifies a resource group. | -| [`CALIBRATE RESOURCE`](/sql-statements/sql-statement-calibrate-resource.md) | Estimates and outputs the [Request Unit (RU)](/tidb-resource-control.md#what-is-request-unit-ru) capacity of the current cluster. | +| [`CALIBRATE RESOURCE`](/sql-statements/sql-statement-calibrate-resource.md) | Estimates and outputs the [Request Unit (RU)](/tidb-resource-control-ru-groups.md#what-is-request-unit-ru) capacity of the current cluster. | | [`CREATE RESOURCE GROUP`](/sql-statements/sql-statement-create-resource-group.md) | Creates a new resource group. | | [`DROP RESOURCE GROUP`](/sql-statements/sql-statement-drop-resource-group.md) | Drops a resource group. | | [`QUERY WATCH`](/sql-statements/sql-statement-query-watch.md) | Manages the runaway query watch list. | diff --git a/sql-statements/sql-statement-replace.md b/sql-statements/sql-statement-replace.md index 4ef240fb580ff..9eb0dc2cf3c4c 100644 --- a/sql-statements/sql-statement-replace.md +++ b/sql-statements/sql-statement-replace.md @@ -35,7 +35,7 @@ InsertValues ::= > **Note:** > -> Starting from v6.6.0, TiDB supports [Resource Control](/tidb-resource-control.md). You can use this feature to execute SQL statements with different priorities in different resource groups. By configuring proper quotas and priorities for these resource groups, you can gain better scheduling control for SQL statements with different priorities. When resource control is enabled, statement priority (`PriorityOpt`) will no longer take effect. It is recommended that you use [Resource Control](/tidb-resource-control.md) to manage resource usage for different SQL statements. +> Starting from v6.6.0, TiDB supports [Resource Control](/tidb-resource-control-ru-groups.md). You can use this feature to execute SQL statements with different priorities in different resource groups. By configuring proper quotas and priorities for these resource groups, you can gain better scheduling control for SQL statements with different priorities. When resource control is enabled, statement priority (`PriorityOpt`) will no longer take effect. It is recommended that you use [Resource Control](/tidb-resource-control-ru-groups.md) to manage resource usage for different SQL statements. ## Examples diff --git a/sql-statements/sql-statement-select.md b/sql-statements/sql-statement-select.md index dc5ce183204bc..664e4d01162b9 100644 --- a/sql-statements/sql-statement-select.md +++ b/sql-statements/sql-statement-select.md @@ -107,7 +107,7 @@ TableSample ::= > **Note:** > -> Starting from v6.6.0, TiDB supports [Resource Control](/tidb-resource-control.md). You can use this feature to execute SQL statements with different priorities in different resource groups. By configuring proper quotas and priorities for these resource groups, you can gain better scheduling control for SQL statements with different priorities. When resource control is enabled, statement priority (`HIGH_PRIORITY`) will no longer take effect. It is recommended that you use [Resource Control](/tidb-resource-control.md) to manage resource usage for different SQL statements. +> Starting from v6.6.0, TiDB supports [Resource Control](/tidb-resource-control-ru-groups.md). You can use this feature to execute SQL statements with different priorities in different resource groups. By configuring proper quotas and priorities for these resource groups, you can gain better scheduling control for SQL statements with different priorities. When resource control is enabled, statement priority (`HIGH_PRIORITY`) will no longer take effect. It is recommended that you use [Resource Control](/tidb-resource-control-ru-groups.md) to manage resource usage for different SQL statements. ## Examples diff --git a/sql-statements/sql-statement-set-resource-group.md b/sql-statements/sql-statement-set-resource-group.md index bd61e8523fa3a..04f9dbe82f6b8 100644 --- a/sql-statements/sql-statement-set-resource-group.md +++ b/sql-statements/sql-statement-set-resource-group.md @@ -97,4 +97,4 @@ MySQL also supports [SET RESOURCE GROUP](https://dev.mysql.com/doc/refman/8.0/en * [CREATE RESOURCE GROUP](/sql-statements/sql-statement-create-resource-group.md) * [DROP RESOURCE GROUP](/sql-statements/sql-statement-drop-resource-group.md) * [ALTER RESOURCE GROUP](/sql-statements/sql-statement-alter-resource-group.md) -* [Resource Control](/tidb-resource-control.md) \ No newline at end of file +* [Resource Control](/tidb-resource-control-ru-groups.md) \ No newline at end of file diff --git a/sql-statements/sql-statement-show-create-resource-group.md b/sql-statements/sql-statement-show-create-resource-group.md index c63db125c8101..13831ef9a88dc 100644 --- a/sql-statements/sql-statement-show-create-resource-group.md +++ b/sql-statements/sql-statement-show-create-resource-group.md @@ -50,7 +50,7 @@ This statement is a TiDB extension for MySQL. ## See also -* [TiDB RESOURCE CONTROL](/tidb-resource-control.md) +* [TiDB RESOURCE CONTROL](/tidb-resource-control-ru-groups.md) * [CREATE RESOURCE GROUP](/sql-statements/sql-statement-alter-resource-group.md) * [ALTER RESOURCE GROUP](/sql-statements/sql-statement-alter-resource-group.md) * [DROP RESOURCE GROUP](/sql-statements/sql-statement-drop-resource-group.md) diff --git a/sql-statements/sql-statement-update.md b/sql-statements/sql-statement-update.md index 77f0d4676a89c..268c4ff0a5265 100644 --- a/sql-statements/sql-statement-update.md +++ b/sql-statements/sql-statement-update.md @@ -29,7 +29,7 @@ TableRefs ::= > **Note:** > -> Starting from v6.6.0, TiDB supports [Resource Control](/tidb-resource-control.md). You can use this feature to execute SQL statements with different priorities in different resource groups. By configuring proper quotas and priorities for these resource groups, you can gain better scheduling control for SQL statements with different priorities. When resource control is enabled, statement priority (`LOW_PRIORITY` and `HIGH_PRIORITY`) will no longer take effect. It is recommended that you use [Resource Control](/tidb-resource-control.md) to manage resource usage for different SQL statements. +> Starting from v6.6.0, TiDB supports [Resource Control](/tidb-resource-control-ru-groups.md). You can use this feature to execute SQL statements with different priorities in different resource groups. By configuring proper quotas and priorities for these resource groups, you can gain better scheduling control for SQL statements with different priorities. When resource control is enabled, statement priority (`LOW_PRIORITY` and `HIGH_PRIORITY`) will no longer take effect. It is recommended that you use [Resource Control](/tidb-resource-control-ru-groups.md) to manage resource usage for different SQL statements. ## Examples diff --git a/statement-summary-tables.md b/statement-summary-tables.md index d5a35ed66d23d..5e3b42eb7172d 100644 --- a/statement-summary-tables.md +++ b/statement-summary-tables.md @@ -87,7 +87,7 @@ The following is a sample output of querying `statements_summary`: > **Note:** > > - In TiDB, the time unit of fields in statement summary tables is nanosecond (ns), whereas in MySQL the time unit is picosecond (ps). -> - Starting from v7.5.1 and v7.6.0, for clusters with [resource control](/tidb-resource-control.md) enabled, `statements_summary` will be aggregated by resource group, for example, the same statements executed in different resource groups will be collected as different records. +> - Starting from v7.5.1 and v7.6.0, for clusters with [resource control](/tidb-resource-control-ru-groups.md) enabled, `statements_summary` will be aggregated by resource group, for example, the same statements executed in different resource groups will be collected as different records. ## `statements_summary_history` diff --git a/system-variable-reference.md b/system-variable-reference.md index 7f0643abb192e..55154af10257f 100644 --- a/system-variable-reference.md +++ b/system-variable-reference.md @@ -1895,7 +1895,7 @@ Referenced in: - [SET RESOURCE GROUP](/sql-statements/sql-statement-set-resource-group.md) - [System Variables](/system-variables.md#tidb_enable_resource_control-new-in-v660) - [TiKV Configuration File](/tikv-configuration-file.md) -- [Use Resource Control to Achieve Resource Isolation](/tidb-resource-control.md) +- [Use Resource Control to Achieve Resource Group Limitation](/tidb-resource-control-ru-groups.md) - [`CALIBRATE RESOURCE`](/sql-statements/sql-statement-calibrate-resource.md) - [TiDB 7.4.0 Release Notes](/releases/release-7.4.0.md) - [TiDB 7.0.0 Release Notes](/releases/release-7.0.0.md) @@ -2479,7 +2479,7 @@ Referenced in: Referenced in: - [System Variables](/system-variables.md#tidb_last_query_info-new-in-v4014) -- [Use Resource Control to Achieve Resource Isolation](/tidb-resource-control.md) +- [Use Resource Control to Achieve Resource Group Limitation](/tidb-resource-control-ru-groups.md) - [TiDB 7.6.0 Release Notes](/releases/release-7.6.0.md) - [TiDB 7.5.1 Release Notes](/releases/release-7.5.1.md) @@ -3353,7 +3353,7 @@ Referenced in: Referenced in: - [System Variables](/system-variables.md#tidb_request_source_type-new-in-v740) -- [Use Resource Control to Achieve Resource Isolation](/tidb-resource-control.md) +- [Use Resource Control to Achieve Resource Group Limitation](/tidb-resource-control-ru-groups.md) - [TiDB 7.4.0 Release Notes](/releases/release-7.4.0.md) ### tidb_resource_control_strict_mode @@ -3364,7 +3364,7 @@ Referenced in: - [Privilege Management](/privilege-management.md) - [SET RESOURCE GROUP](/sql-statements/sql-statement-set-resource-group.md) - [System Variables](/system-variables.md#tidb_resource_control_strict_mode-new-in-v820) -- [Use Resource Control to Achieve Resource Isolation](/tidb-resource-control.md) +- [Use Resource Control to Achieve Resource Group Limitation](/tidb-resource-control-ru-groups.md) - [TiDB 8.2.0 Release Notes](/releases/release-8.2.0.md) ### tidb_restricted_read_only @@ -4152,7 +4152,7 @@ Referenced in: - [Limited SQL features on TiDB Cloud](https://docs.pingcap.com/tidbcloud/limited-sql-features) - [System Variables](/system-variables.md#txn_scope) - [TiDB Configuration File](/tidb-configuration-file.md) -- [Use Resource Control to Achieve Resource Isolation](/tidb-resource-control.md) +- [Use Resource Control to Achieve Resource Group Limitation](/tidb-resource-control-ru-groups.md) ### validate_password.check_user_name diff --git a/system-variables.md b/system-variables.md index 97b0da03751ff..0a76db7f3df8d 100644 --- a/system-variables.md +++ b/system-variables.md @@ -2612,7 +2612,7 @@ mysql> SELECT job_info FROM mysql.analyze_jobs ORDER BY end_time DESC LIMIT 1; - Applies to hint [SET_VAR](/optimizer-hints.md#set_varvar_namevar_value): No - Default value: `ON` - Type: Boolean -- This variable is a switch for [the resource control feature](/tidb-resource-control.md). When this variable is set to `ON`, the TiDB cluster can isolate application resources based on resource groups. +- This variable is a switch for [the resource control feature](/tidb-resource-control-ru-groups.md). When this variable is set to `ON`, the TiDB cluster can isolate application resources based on resource groups. ### tidb_enable_reuse_chunk New in v6.4.0 @@ -2992,7 +2992,7 @@ For a system upgraded to v5.0 from an earlier version, if you have not modified > **Note:** > -> Starting from v6.6.0, TiDB supports [Resource Control](/tidb-resource-control.md). You can use this feature to execute SQL statements with different priorities in different resource groups. By configuring proper quotas and priorities for these resource groups, you can gain better scheduling control for SQL statements with different priorities. When resource control is enabled, statement priority will no longer take effect. It is recommended that you use [Resource Control](/tidb-resource-control.md) to manage resource usage for different SQL statements. +> Starting from v6.6.0, TiDB supports [Resource Control](/tidb-resource-control-ru-groups.md). You can use this feature to execute SQL statements with different priorities in different resource groups. By configuring proper quotas and priorities for these resource groups, you can gain better scheduling control for SQL statements with different priorities. When resource control is enabled, statement priority will no longer take effect. It is recommended that you use [Resource Control](/tidb-resource-control-ru-groups.md) to manage resource usage for different SQL statements. ### tidb_gc_concurrency New in v5.0 @@ -3496,7 +3496,7 @@ For a system upgraded to v5.0 from an earlier version, if you have not modified - `start_ts`: The start timestamp of the transaction. - `for_update_ts`: The `for_update_ts` of the previously executed DML statement. This is an internal term of TiDB used for tests. Usually, you can ignore this information. - `error`: The error message, if any. - - `ru_consumption`: Consumed [RU](/tidb-resource-control.md#what-is-request-unit-ru) for executing the statement. + - `ru_consumption`: Consumed [RU](/tidb-resource-control-ru-groups.md#what-is-request-unit-ru) for executing the statement. ### tidb_last_txn_info New in v4.0.9 @@ -5145,7 +5145,7 @@ SHOW WARNINGS; - Type: String - Default value: `""` - Possible values: `"ddl"`, `"stats"`, `"br"`, `"lightning"`, `"background"` -- This variable is used to explicitly specify the task type for the current session, which is identified and controlled by [Resource Control](/tidb-resource-control.md). For example: `SET @@tidb_request_source_type = "background"`. +- This variable is used to explicitly specify the task type for the current session, which is identified and controlled by [Resource Control](/tidb-resource-control-ru-groups.md). For example: `SET @@tidb_request_source_type = "background"`. ### tidb_resource_control_strict_mode New in v8.2.0 diff --git a/tidb-configuration-file.md b/tidb-configuration-file.md index da89565963059..85c3a9535a531 100644 --- a/tidb-configuration-file.md +++ b/tidb-configuration-file.md @@ -576,7 +576,7 @@ Configuration items related to performance. > **Note:** > -> Starting from v6.6.0, TiDB supports [Resource Control](/tidb-resource-control.md). You can use this feature to execute SQL statements with different priorities in different resource groups. By configuring proper quotas and priorities for these resource groups, you can gain better scheduling control for SQL statements with different priorities. When resource control is enabled, statement priority will no longer take effect. It is recommended that you use [Resource Control](/tidb-resource-control.md) to manage resource usage for different SQL statements. +> Starting from v6.6.0, TiDB supports [Resource Control](/tidb-resource-control-ru-groups.md). You can use this feature to execute SQL statements with different priorities in different resource groups. By configuring proper quotas and priorities for these resource groups, you can gain better scheduling control for SQL statements with different priorities. When resource control is enabled, statement priority will no longer take effect. It is recommended that you use [Resource Control](/tidb-resource-control-ru-groups.md) to manage resource usage for different SQL statements. ### `distinct-agg-push-down` @@ -936,7 +936,7 @@ Configuration items related to read isolation. > **Note:** > -> Starting from v6.6.0, TiDB supports [Resource Control](/tidb-resource-control.md). You can use this feature to execute SQL statements with different priorities in different resource groups. By configuring proper quotas and priorities for these resource groups, you can gain better scheduling control for SQL statements with different priorities. When resource control is enabled, statement priority will no longer take effect. It is recommended that you use [Resource Control](/tidb-resource-control.md) to manage resource usage for different SQL statements. +> Starting from v6.6.0, TiDB supports [Resource Control](/tidb-resource-control-ru-groups.md). You can use this feature to execute SQL statements with different priorities in different resource groups. By configuring proper quotas and priorities for these resource groups, you can gain better scheduling control for SQL statements with different priorities. When resource control is enabled, statement priority will no longer take effect. It is recommended that you use [Resource Control](/tidb-resource-control-ru-groups.md) to manage resource usage for different SQL statements. ### `max_connections` diff --git a/tidb-resource-control.md b/tidb-resource-control-ru-groups.md similarity index 99% rename from tidb-resource-control.md rename to tidb-resource-control-ru-groups.md index 6650468c5ed4e..2357a5e5d3a62 100644 --- a/tidb-resource-control.md +++ b/tidb-resource-control-ru-groups.md @@ -1,9 +1,9 @@ --- -title: Use Resource Control to Achieve resource group limitation +title: Use Resource Control to Achieve Resource Group Limitation summary: Learn how to use the resource control feature to control and schedule application resources. --- -# Use Resource Control to Achieve resource group limitation +# Use Resource Control to Achieve Resource Group Limitation > **Note:** > @@ -259,6 +259,7 @@ The following example binds the current statement to the resource group `rg1`. ```sql SELECT /*+ RESOURCE_GROUP(rg1) */ * FROM t limit 10; ``` + ## Disable resource control diff --git a/tidb-resource-control-runaway-queries.md b/tidb-resource-control-runaway-queries.md index 39bf477722d32..36a40ec55b331 100644 --- a/tidb-resource-control-runaway-queries.md +++ b/tidb-resource-control-runaway-queries.md @@ -5,7 +5,7 @@ summary: Introduces how to control and degrade queries with excessive resource c # Manage queries that consume more resources than expected (Runaway Queries) -A runaway query is a query (`SELECT` statement only) that consumes more time or resources than expected. The term **runaway queries** is used in the following to describe the feature of managing the runaway query. +A runaway query is a query that consumes more time or resources than expected. The term **runaway queries** is used in the following to describe the feature of managing the runaway query. - Starting from v7.2.0, the resource control feature introduces the management of runaway queries. You can set criteria for a resource group to identify runaway queries and automatically take actions to prevent them from exhausting resources and affecting other queries. You can manage runaway queries for a resource group by including the `QUERY_LIMIT` field in [`CREATE RESOURCE GROUP`](/sql-statements/sql-statement-create-resource-group.md) or [`ALTER RESOURCE GROUP`](/sql-statements/sql-statement-alter-resource-group.md). - Starting from v7.3.0, the resource control feature introduces manual management of runaway watches, enabling quick identification of runaway queries for a given SQL statement or Digest. You can execute the statement [`QUERY WATCH`](/sql-statements/sql-statement-query-watch.md) to manually manage the runaway queries watch list in the resource group. @@ -14,9 +14,9 @@ A runaway query is a query (`SELECT` statement only) that consumes more time or If a query exceeds any of the following limits, it is identified as a runaway query: -- `EXEC_ELAPSED`: checks whether the query execution time exceeds the limit. -- `PROCESSED_KEYS`: checks whether the number of keys processed by the Coprocessor exceeds the limit. -- `RU`: checks whether the total number of read and write RUs consumed by the statement exceeds the limit. +- `EXEC_ELAPSED`: checks whether the query execution time exceeds the limit, this rule applies to read and write DML. +- `PROCESSED_KEYS`: checks whether the number of keys processed by the Coprocessor exceeds the limit, this rule applies only to read statements. +- `RU`: checks whether the total number of read and write RUs consumed by the statement exceeds the limit, this rule applies only to read statements. Supported operations (`ACTION`): @@ -132,7 +132,7 @@ The parameters are as follows: QUERY WATCH REMOVE 1; ``` -#### Observability +# Observability You can get more information about runaway queries from the following system tables and `INFORMATION_SCHEMA`: diff --git a/tiflash/tiflash-pipeline-model.md b/tiflash/tiflash-pipeline-model.md index 46b7cb5c7661e..61a1184f4ad68 100644 --- a/tiflash/tiflash-pipeline-model.md +++ b/tiflash/tiflash-pipeline-model.md @@ -10,7 +10,7 @@ This document introduces the TiFlash pipeline execution model. Starting from v7.2.0, TiFlash supports a new execution model, the pipeline execution model. - For v7.2.0 and v7.3.0: The pipeline execution model is experimental and is controlled by [`tidb_enable_tiflash_pipeline_model`](https://docs.pingcap.com/tidb/v7.2/system-variables#tidb_enable_tiflash_pipeline_model-introduced-since-v720). -- For v7.4.0 and later versions: The pipeline execution model becomes generally available. It is an internal feature of TiFlash and is tightly integrated with TiFlash resource control. When you enable TiFlash resource control, the pipeline execution model is automatically enabled. For more information about how to use TiFlash resource control, refer to [Use resource control to achieve resource isolation](/tidb-resource-control.md#parameters-for-resource-control). Additionally, starting from v7.4.0, the system variable `tidb_enable_tiflash_pipeline_model` is deprecated. +- For v7.4.0 and later versions: The pipeline execution model becomes generally available. It is an internal feature of TiFlash and is tightly integrated with TiFlash resource control. When you enable TiFlash resource control, the pipeline execution model is automatically enabled. For more information about how to use TiFlash resource control, refer to [Use Resource Control to Achieve Resource Group Limitation](/tidb-resource-control-ru-groups.md#parameters-for-resource-control). Additionally, starting from v7.4.0, the system variable `tidb_enable_tiflash_pipeline_model` is deprecated. Inspired by the paper [Morsel-Driven Parallelism: A NUMA-Aware Query Evaluation Framework for the Many-Core Age](https://dl.acm.org/doi/10.1145/2588555.2610507), the TiFlash pipeline execution model provides a fine-grained task scheduling model, which is different from the traditional thread scheduling model. It reduces the overhead of operating system thread application and scheduling and provides a fine-grained scheduling mechanism. diff --git a/tikv-configuration-file.md b/tikv-configuration-file.md index b6eebfc245b05..224c81d4c4ee0 100644 --- a/tikv-configuration-file.md +++ b/tikv-configuration-file.md @@ -2448,7 +2448,7 @@ Configuration items related to resource control of the TiKV storage layer. ### `enabled` New in v6.6.0 -+ Controls whether to enable scheduling for user foreground read/write requests according to [Request Unit (RU)](/tidb-resource-control.md#what-is-request-unit-ru) of the corresponding resource groups. For information about TiDB resource groups and resource control, see [TiDB resource control](/tidb-resource-control.md). ++ Controls whether to enable scheduling for user foreground read/write requests according to [Request Unit (RU)](/tidb-resource-control-ru-groups.md#what-is-request-unit-ru) of the corresponding resource groups. For information about TiDB resource groups and resource control, see [TiDB resource control](/tidb-resource-control-ru-groups.md). + Enabling this configuration item only works when [`tidb_enable_resource_control](/system-variables.md#tidb_enable_resource_control-new-in-v660) is enabled on TiDB. When this configuration item is enabled, TiKV will use the priority queue to schedule the queued read/write requests from foreground users. The scheduling priority of a request is inversely related to the amount of resources already consumed by the resource group that receives this request, and positively related to the quota of the corresponding resource group. + Default value: `true`, which means scheduling based on the RU of the resource group is enabled. diff --git a/tiproxy/tiproxy-load-balance.md b/tiproxy/tiproxy-load-balance.md index d4ac60d04b6e5..80ef15596367f 100644 --- a/tiproxy/tiproxy-load-balance.md +++ b/tiproxy/tiproxy-load-balance.md @@ -40,7 +40,7 @@ Consider an application that handles both transaction and BI workloads. To preve 1. Set [`balance.label-name`](/tiproxy/tiproxy-configuration.md#label-name) to `"app"` in TiProxy, indicating that TiDB servers will be matched by the label name `"app"`, and connections will be routed to TiDB servers with matching label values. 2. Configure two TiProxy instances, adding `"app"="Order"` and `"app"="BI"` to their respective [`labels`](/tiproxy/tiproxy-configuration.md#labels) configuration items. 3. Divide TiDB instances into two groups, adding `"app"="Order"` and `"app"="BI"` to their respective [`labels`](/tidb-configuration-file.md#labels) configuration items. -4. Optional: For storage layer isolation, configure [Placement Rules](/configure-placement-rules.md) or [Resource Control](/tidb-resource-control.md). +4. Optional: For storage layer isolation, configure [Placement Rules](/configure-placement-rules.md) or [Resource Control](/tidb-resource-control-ru-groups.md). 5. Direct transaction and BI clients to connect to their respective TiProxy instance addresses. Label-based Load Balancing diff --git a/tiproxy/tiproxy-traffic-replay.md b/tiproxy/tiproxy-traffic-replay.md index 891de97ee60a8..ce52b5a4daa8d 100644 --- a/tiproxy/tiproxy-traffic-replay.md +++ b/tiproxy/tiproxy-traffic-replay.md @@ -185,7 +185,7 @@ For more information, see [`tiproxyctl traffic cancel`](/tiproxy/tiproxy-command - TiProxy only supports replaying traffic files captured by TiProxy and does not support other file formats. Therefore, make sure to capture traffic from the production cluster using TiProxy first. - TiProxy traffic replay does not support filtering SQL types and DML and DDL statements are replayed. Therefore, you need to restore the cluster data to its pre-replay state before replaying again. -- TiProxy traffic replay does not support testing [Resource Control](/tidb-resource-control.md) and [privilege management](/privilege-management.md) because TiProxy uses the same username to replay traffic. +- TiProxy traffic replay does not support testing [Resource Control](/tidb-resource-control-ru-groups.md) and [privilege management](/privilege-management.md) because TiProxy uses the same username to replay traffic. - TiProxy does not support replaying [`LOAD DATA`](/sql-statements/sql-statement-load-data.md) statements. ## More resources diff --git a/user-account-management.md b/user-account-management.md index 8af449abbe3b6..a8edb4152d5c7 100644 --- a/user-account-management.md +++ b/user-account-management.md @@ -146,7 +146,7 @@ TiDB creates the `'root'@'%'` default account during the database initialization ## Set account resource limits -TiDB can limit the resources consumed by users using resource groups. For more information, see [Use resource control to achieve resource isolation](/tidb-resource-control.md). +TiDB can limit the resources consumed by users using resource groups. For more information, see [Use Resource Control to Achieve Resource Group Limitation](/tidb-resource-control-ru-groups.md). ## Assign account passwords From 8ef4b25d3772ea37ef1219625dde77111c7906ee Mon Sep 17 00:00:00 2001 From: xixirangrang Date: Fri, 7 Feb 2025 14:58:07 +0800 Subject: [PATCH 04/25] Update tidb-resource-control-ru-groups.md --- tidb-resource-control-ru-groups.md | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/tidb-resource-control-ru-groups.md b/tidb-resource-control-ru-groups.md index 2357a5e5d3a62..5ae48bacab4e4 100644 --- a/tidb-resource-control-ru-groups.md +++ b/tidb-resource-control-ru-groups.md @@ -33,6 +33,11 @@ Starting from v7.4.0, the resource control feature supports controlling TiFlash +For information on how to manage background tasks and handle resource-intensive queries (Runaway Queries), see the following documents: + +- [Use Resource Control to Managing Background Tasks](/tidb-resource-control-background-tasks.md) +- [Managing Runaway Queries](/tidb-resource-control-runaway-queries.md) +- ## Scenarios for resource control The introduction of the resource control feature is a milestone for TiDB. It can divide a distributed database cluster into multiple logical units. Even if an individual unit overuses resources, it does not crowd out the resources needed by other units. From 07e0f333058d254028b2e44516b33f2bec6df843 Mon Sep 17 00:00:00 2001 From: ShuNing Date: Fri, 7 Feb 2025 15:08:45 +0800 Subject: [PATCH 05/25] Update TOC.md Co-authored-by: xixirangrang --- TOC.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/TOC.md b/TOC.md index 6f254f4bd2193..b2707711e9753 100644 --- a/TOC.md +++ b/TOC.md @@ -263,9 +263,9 @@ - [Multi-Replica Cluster DR](/dr-multi-replica.md) - [BR-based DR](/dr-backup-restore.md) - Resource Manager - - [Achieve resource group limitation and flow control using Resource Control](/tidb-resource-control-ru-groups.md) - - [Runaway Queries with resource consumption exceeding expectations](/tidb-resource-control-runaway-queries.md) - - [Limit background task resource usage](/tidb-resource-control-background-tasks.md) + - [Use Resource Control to Achieve Resource Group Limitation and Flow Control](/tidb-resource-control-ru-groups.md) + - [Manage Runaway Queries](/tidb-resource-control-runaway-queries.md) + - [Manage Background Tasks](/tidb-resource-control-background-tasks.md) - [Configure Time Zone](/configure-time-zone.md) - [Daily Checklist](/daily-check.md) - [Maintain TiFlash](/tiflash/maintain-tiflash.md) From 261516738f7c2d29e603d181a7bed4a8c2b12540 Mon Sep 17 00:00:00 2001 From: ShuNing Date: Fri, 7 Feb 2025 15:08:54 +0800 Subject: [PATCH 06/25] Update faq/sql-faq.md Co-authored-by: xixirangrang --- faq/sql-faq.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/faq/sql-faq.md b/faq/sql-faq.md index bc33b9520ee9c..1d1351dfd67ab 100644 --- a/faq/sql-faq.md +++ b/faq/sql-faq.md @@ -211,7 +211,7 @@ TiDB supports changing the priority on a [global](/system-variables.md#tidb_forc > **Note:** > -> Starting from v6.6.0, TiDB supports [Resource Control](/tidb-resource-control-ru-groups.md). You can use this feature to execute SQL statements with different priorities in different resource groups. By configuring proper quotas and priorities for these resource groups, you can gain better scheduling control for SQL statements with different priorities. When resource control is enabled, statement priority will no longer take effect. It is recommended that you use [Resource Control](/tidb-resource-control-ru-groups.md) to manage resource usage for different SQL statements. +> Starting from v6.6.0, TiDB supports [Resource Control](/tidb-resource-control-ru-groups.md). You can use this feature to execute SQL statements with different priorities in different resource groups. By configuring proper quotas and priorities for these resource groups, you can gain better scheduling control for SQL statements with different priorities. When resource control is enabled, statement priority will no longer take effect. It is recommended that you use Resource Control to manage resource usage for different SQL statements. You can combine the above two parameters with the DML of TiDB to use them. For example: From 359f8348c0caf2574e7a6a002dce43a765f83839 Mon Sep 17 00:00:00 2001 From: ShuNing Date: Fri, 7 Feb 2025 15:09:01 +0800 Subject: [PATCH 07/25] Update functions-and-operators/tidb-functions.md Co-authored-by: xixirangrang --- functions-and-operators/tidb-functions.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/functions-and-operators/tidb-functions.md b/functions-and-operators/tidb-functions.md index 05db30eb8e5a6..7f929c935a27b 100644 --- a/functions-and-operators/tidb-functions.md +++ b/functions-and-operators/tidb-functions.md @@ -11,7 +11,7 @@ The following functions are TiDB extensions, and are not present in MySQL: | Function name | Function description | | :-------------- | :------------------------------------- | -| [`CURRENT_RESOURCE_GROUP()`](#current_resource_group) | Returns the name of the resource group that the current session is bound to. See [using resource control to achieve resource isolation](/tidb-resource-control-ru-groups.md). | +| [`CURRENT_RESOURCE_GROUP()`](#current_resource_group) | Returns the name of the resource group that the current session is bound to. See [Use Resource Control to Achieve Resource Group Limitation and Flow Control](/tidb-resource-control-ru-groups.md). | | [`TIDB_BOUNDED_STALENESS()`](#tidb_bounded_staleness) | Instructs TiDB to read the most recent data within a specified time range. See [reading historical data using the `AS OF TIMESTAMP` clause](/as-of-timestamp.md). | | [`TIDB_CURRENT_TSO()`](#tidb_current_tso) | Returns the current [TimeStamp Oracle (TSO) in TiDB](/tso.md). | | [`TIDB_DECODE_BINARY_PLAN()`](#tidb_decode_binary_plan) | Decodes binary plans. | From b15c9eedf694c528dfbee55ae758329d55234c93 Mon Sep 17 00:00:00 2001 From: ShuNing Date: Fri, 7 Feb 2025 15:09:08 +0800 Subject: [PATCH 08/25] Update information-schema/information-schema-resource-groups.md Co-authored-by: xixirangrang --- information-schema/information-schema-resource-groups.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/information-schema/information-schema-resource-groups.md b/information-schema/information-schema-resource-groups.md index 2916c058baa70..0da7bb84235d6 100644 --- a/information-schema/information-schema-resource-groups.md +++ b/information-schema/information-schema-resource-groups.md @@ -5,7 +5,7 @@ summary: Learn the `RESOURCE_GROUPS` information_schema table. # RESOURCE_GROUPS -The `RESOURCE_GROUPS` table shows the information about all resource groups. For more information, see [Use Resource Control to Achieve Resource Group Limitation](/tidb-resource-control-ru-groups.md). +The `RESOURCE_GROUPS` table shows the information about all resource groups. For more information, see [Use Resource Control to Achieve Resource Group Limitation and Flow Control](/tidb-resource-control-ru-groups.md). > **Note:** > From 8dc8eee1ef6bb645ea021492467512294c92119e Mon Sep 17 00:00:00 2001 From: ShuNing Date: Fri, 7 Feb 2025 15:09:19 +0800 Subject: [PATCH 09/25] Update tidb-resource-control-background-tasks.md Co-authored-by: xixirangrang --- tidb-resource-control-background-tasks.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tidb-resource-control-background-tasks.md b/tidb-resource-control-background-tasks.md index 8693834e092bc..12e7ae0f7a846 100644 --- a/tidb-resource-control-background-tasks.md +++ b/tidb-resource-control-background-tasks.md @@ -1,5 +1,5 @@ --- -title: Limit background task resource usage +title: Use Resource Control to Manage Background Tasks summary: Introduces how to control background tasks through Resource Control. --- From 1d4095a62e94a6aaff9f9d4054155f4625a7febe Mon Sep 17 00:00:00 2001 From: ShuNing Date: Fri, 7 Feb 2025 15:09:30 +0800 Subject: [PATCH 10/25] Update tidb-resource-control-background-tasks.md Co-authored-by: xixirangrang --- tidb-resource-control-background-tasks.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tidb-resource-control-background-tasks.md b/tidb-resource-control-background-tasks.md index 12e7ae0f7a846..1b872de42b2d9 100644 --- a/tidb-resource-control-background-tasks.md +++ b/tidb-resource-control-background-tasks.md @@ -3,7 +3,7 @@ title: Use Resource Control to Manage Background Tasks summary: Introduces how to control background tasks through Resource Control. --- -# Manage background tasks +# Use Resource Control to Manage Background Tasks > **Warning:** > From 5595fcdff86352bad8dd906ce05a473203f35494 Mon Sep 17 00:00:00 2001 From: ShuNing Date: Fri, 7 Feb 2025 15:09:46 +0800 Subject: [PATCH 11/25] Update tidb-resource-control-ru-groups.md Co-authored-by: xixirangrang --- tidb-resource-control-ru-groups.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/tidb-resource-control-ru-groups.md b/tidb-resource-control-ru-groups.md index 5ae48bacab4e4..320da6e383b13 100644 --- a/tidb-resource-control-ru-groups.md +++ b/tidb-resource-control-ru-groups.md @@ -36,8 +36,8 @@ Starting from v7.4.0, the resource control feature supports controlling TiFlash For information on how to manage background tasks and handle resource-intensive queries (Runaway Queries), see the following documents: - [Use Resource Control to Managing Background Tasks](/tidb-resource-control-background-tasks.md) -- [Managing Runaway Queries](/tidb-resource-control-runaway-queries.md) -- +- [Manage Queries That Consume More Resources Than Expected (Runaway Queries)](/tidb-resource-control-runaway-queries.md) + ## Scenarios for resource control The introduction of the resource control feature is a milestone for TiDB. It can divide a distributed database cluster into multiple logical units. Even if an individual unit overuses resources, it does not crowd out the resources needed by other units. From cbbf7f13f6095b9b7c06c7c7491ad75cc446fabf Mon Sep 17 00:00:00 2001 From: ShuNing Date: Fri, 7 Feb 2025 15:09:57 +0800 Subject: [PATCH 12/25] Update tidb-resource-control-ru-groups.md Co-authored-by: xixirangrang --- tidb-resource-control-ru-groups.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tidb-resource-control-ru-groups.md b/tidb-resource-control-ru-groups.md index 320da6e383b13..5e5d486cd0a5c 100644 --- a/tidb-resource-control-ru-groups.md +++ b/tidb-resource-control-ru-groups.md @@ -1,5 +1,5 @@ --- -title: Use Resource Control to Achieve Resource Group Limitation +title: Use Resource Control to Achieve Resource Group Limitation and Flow Control summary: Learn how to use the resource control feature to control and schedule application resources. --- From 3e43c493bcf70ba90037f98a5b35f62c239759e5 Mon Sep 17 00:00:00 2001 From: ShuNing Date: Fri, 7 Feb 2025 15:10:07 +0800 Subject: [PATCH 13/25] Update tidb-resource-control-ru-groups.md Co-authored-by: xixirangrang --- tidb-resource-control-ru-groups.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tidb-resource-control-ru-groups.md b/tidb-resource-control-ru-groups.md index 5e5d486cd0a5c..57e0b104a8209 100644 --- a/tidb-resource-control-ru-groups.md +++ b/tidb-resource-control-ru-groups.md @@ -3,7 +3,7 @@ title: Use Resource Control to Achieve Resource Group Limitation and Flow Contro summary: Learn how to use the resource control feature to control and schedule application resources. --- -# Use Resource Control to Achieve Resource Group Limitation +# Use Resource Control to Achieve Resource Group Limitation and Flow Control > **Note:** > From 1c3d33f8986d92a52178f8e7bf425e8ca8875b27 Mon Sep 17 00:00:00 2001 From: ShuNing Date: Fri, 7 Feb 2025 15:10:16 +0800 Subject: [PATCH 14/25] Update tidb-resource-control-runaway-queries.md Co-authored-by: xixirangrang --- tidb-resource-control-runaway-queries.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tidb-resource-control-runaway-queries.md b/tidb-resource-control-runaway-queries.md index 36a40ec55b331..89702d94bba9b 100644 --- a/tidb-resource-control-runaway-queries.md +++ b/tidb-resource-control-runaway-queries.md @@ -1,5 +1,5 @@ --- -title: Managing Queries with Excessive Resource Consumption (Runaway Queries) +title: Manage Queries That Consume More Resources Than Expected (Runaway Queries) summary: Introduces how to control and degrade queries with excessive resource consumption (Runaway Queries) through resource management capabilities. --- From 0afdc6a7c760305fe2a59cc5c30dd63d648ac995 Mon Sep 17 00:00:00 2001 From: ShuNing Date: Fri, 7 Feb 2025 15:10:26 +0800 Subject: [PATCH 15/25] Update tidb-resource-control-runaway-queries.md Co-authored-by: xixirangrang --- tidb-resource-control-runaway-queries.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tidb-resource-control-runaway-queries.md b/tidb-resource-control-runaway-queries.md index 89702d94bba9b..02305457b48dd 100644 --- a/tidb-resource-control-runaway-queries.md +++ b/tidb-resource-control-runaway-queries.md @@ -132,7 +132,7 @@ The parameters are as follows: QUERY WATCH REMOVE 1; ``` -# Observability +## Observability You can get more information about runaway queries from the following system tables and `INFORMATION_SCHEMA`: From 9e9f9bc59497eea79311bbe0166218a1ee1505f2 Mon Sep 17 00:00:00 2001 From: ShuNing Date: Fri, 7 Feb 2025 15:11:42 +0800 Subject: [PATCH 16/25] Update tiflash/tiflash-pipeline-model.md Co-authored-by: xixirangrang --- tiflash/tiflash-pipeline-model.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tiflash/tiflash-pipeline-model.md b/tiflash/tiflash-pipeline-model.md index 61a1184f4ad68..37752c3862485 100644 --- a/tiflash/tiflash-pipeline-model.md +++ b/tiflash/tiflash-pipeline-model.md @@ -10,7 +10,7 @@ This document introduces the TiFlash pipeline execution model. Starting from v7.2.0, TiFlash supports a new execution model, the pipeline execution model. - For v7.2.0 and v7.3.0: The pipeline execution model is experimental and is controlled by [`tidb_enable_tiflash_pipeline_model`](https://docs.pingcap.com/tidb/v7.2/system-variables#tidb_enable_tiflash_pipeline_model-introduced-since-v720). -- For v7.4.0 and later versions: The pipeline execution model becomes generally available. It is an internal feature of TiFlash and is tightly integrated with TiFlash resource control. When you enable TiFlash resource control, the pipeline execution model is automatically enabled. For more information about how to use TiFlash resource control, refer to [Use Resource Control to Achieve Resource Group Limitation](/tidb-resource-control-ru-groups.md#parameters-for-resource-control). Additionally, starting from v7.4.0, the system variable `tidb_enable_tiflash_pipeline_model` is deprecated. +- For v7.4.0 and later versions: The pipeline execution model becomes generally available. It is an internal feature of TiFlash and is tightly integrated with TiFlash resource control. When you enable TiFlash resource control, the pipeline execution model is automatically enabled. For more information about how to use TiFlash resource control, refer to [Use Resource Control to Achieve Resource Group Limitation and Flow Control](/tidb-resource-control-ru-groups.md#parameters-for-resource-control). Additionally, starting from v7.4.0, the system variable `tidb_enable_tiflash_pipeline_model` is deprecated. Inspired by the paper [Morsel-Driven Parallelism: A NUMA-Aware Query Evaluation Framework for the Many-Core Age](https://dl.acm.org/doi/10.1145/2588555.2610507), the TiFlash pipeline execution model provides a fine-grained task scheduling model, which is different from the traditional thread scheduling model. It reduces the overhead of operating system thread application and scheduling and provides a fine-grained scheduling mechanism. From cee61e3975b161189ebce2cc3e500d4d3bd112c1 Mon Sep 17 00:00:00 2001 From: ShuNing Date: Fri, 7 Feb 2025 15:11:51 +0800 Subject: [PATCH 17/25] Update tidb-resource-control-runaway-queries.md Co-authored-by: xixirangrang --- tidb-resource-control-runaway-queries.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tidb-resource-control-runaway-queries.md b/tidb-resource-control-runaway-queries.md index 02305457b48dd..f3f2305929e3e 100644 --- a/tidb-resource-control-runaway-queries.md +++ b/tidb-resource-control-runaway-queries.md @@ -3,7 +3,7 @@ title: Manage Queries That Consume More Resources Than Expected (Runaway Queries summary: Introduces how to control and degrade queries with excessive resource consumption (Runaway Queries) through resource management capabilities. --- -# Manage queries that consume more resources than expected (Runaway Queries) +# Manage Queries That Consume More Resources Than Expected (Runaway Queries) A runaway query is a query that consumes more time or resources than expected. The term **runaway queries** is used in the following to describe the feature of managing the runaway query. From 45f98862684be8eedcc8d8ba1792245cb3b71438 Mon Sep 17 00:00:00 2001 From: ShuNing Date: Fri, 7 Feb 2025 15:12:05 +0800 Subject: [PATCH 18/25] Update tikv-configuration-file.md Co-authored-by: xixirangrang --- tikv-configuration-file.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tikv-configuration-file.md b/tikv-configuration-file.md index 224c81d4c4ee0..e4c3c4ebcec32 100644 --- a/tikv-configuration-file.md +++ b/tikv-configuration-file.md @@ -2448,7 +2448,7 @@ Configuration items related to resource control of the TiKV storage layer. ### `enabled` New in v6.6.0 -+ Controls whether to enable scheduling for user foreground read/write requests according to [Request Unit (RU)](/tidb-resource-control-ru-groups.md#what-is-request-unit-ru) of the corresponding resource groups. For information about TiDB resource groups and resource control, see [TiDB resource control](/tidb-resource-control-ru-groups.md). ++ Controls whether to enable scheduling for user foreground read/write requests according to [Request Unit (RU)](/tidb-resource-control-ru-groups.md#what-is-request-unit-ru) of the corresponding resource groups. For information about TiDB resource groups and resource control, see [Use Resource Control to Achieve Resource Group Limitation and Flow Control](/tidb-resource-control-ru-groups.md). + Enabling this configuration item only works when [`tidb_enable_resource_control](/system-variables.md#tidb_enable_resource_control-new-in-v660) is enabled on TiDB. When this configuration item is enabled, TiKV will use the priority queue to schedule the queued read/write requests from foreground users. The scheduling priority of a request is inversely related to the amount of resources already consumed by the resource group that receives this request, and positively related to the quota of the corresponding resource group. + Default value: `true`, which means scheduling based on the RU of the resource group is enabled. From 35ed4aef9ca2945c234c600d3a917c37ba727b7a Mon Sep 17 00:00:00 2001 From: ShuNing Date: Fri, 7 Feb 2025 15:12:12 +0800 Subject: [PATCH 19/25] Update user-account-management.md Co-authored-by: xixirangrang --- user-account-management.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/user-account-management.md b/user-account-management.md index a8edb4152d5c7..1652c15f8ba99 100644 --- a/user-account-management.md +++ b/user-account-management.md @@ -146,7 +146,7 @@ TiDB creates the `'root'@'%'` default account during the database initialization ## Set account resource limits -TiDB can limit the resources consumed by users using resource groups. For more information, see [Use Resource Control to Achieve Resource Group Limitation](/tidb-resource-control-ru-groups.md). +TiDB can limit the resources consumed by users using resource groups. For more information, see [Use Resource Control to Achieve Resource Group Limitation and Flow Control](/tidb-resource-control-ru-groups.md). ## Assign account passwords From b43489328df8d71debc0a78a4afbb3c71b34e921 Mon Sep 17 00:00:00 2001 From: ShuNing Date: Fri, 7 Feb 2025 15:12:19 +0800 Subject: [PATCH 20/25] Update functions-and-operators/tidb-functions.md Co-authored-by: xixirangrang --- functions-and-operators/tidb-functions.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/functions-and-operators/tidb-functions.md b/functions-and-operators/tidb-functions.md index 7f929c935a27b..14db38e2eac4b 100644 --- a/functions-and-operators/tidb-functions.md +++ b/functions-and-operators/tidb-functions.md @@ -36,7 +36,7 @@ The following functions are TiDB extensions, and are not present in MySQL: | Function name | Function description | | :-------------- | :------------------------------------- | -| [`CURRENT_RESOURCE_GROUP()`](#current_resource_group) | Returns the resource group name that the current session is bound to. See [using resource control to achieve resource isolation](/tidb-resource-control-ru-groups.md). | +| [`CURRENT_RESOURCE_GROUP()`](#current_resource_group) | Returns the resource group name that the current session is bound to. See [Use Resource Control to Achieve Resource Group Limitation and Flow Control](/tidb-resource-control-ru-groups.md). | | [`TIDB_BOUNDED_STALENESS()`](#tidb_bounded_staleness) | Instructs TiDB to read most recent data within a specified time range. See [reading historical data using the `AS OF TIMESTAMP` clause](/as-of-timestamp.md). | | [`TIDB_CURRENT_TSO()`](#tidb_current_tso) | Returns the current [TimeStamp Oracle (TSO) in TiDB](/tso.md). | | [`TIDB_DECODE_BINARY_PLAN()`](#tidb_decode_binary_plan) | Decodes binary plans. | From 34f09a8a2f90de8b066390295e0260b6b529e617 Mon Sep 17 00:00:00 2001 From: xixirangrang Date: Fri, 7 Feb 2025 15:49:59 +0800 Subject: [PATCH 21/25] Update system-variable-reference.md --- system-variable-reference.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/system-variable-reference.md b/system-variable-reference.md index 53cd9078d2002..6b753b842c0aa 100644 --- a/system-variable-reference.md +++ b/system-variable-reference.md @@ -3353,7 +3353,7 @@ Referenced in: Referenced in: - [System Variables](/system-variables.md#tidb_request_source_type-new-in-v740) -- [Use Resource Control to Achieve Resource Group Limitation and Flow Control](/tidb-resource-control-ru-groups.md) +- [Use Resource Control to Manage Background Tasks](/tidb-resource-control--background-tasks.md) - [TiDB 7.4.0 Release Notes](/releases/release-7.4.0.md) ### tidb_resource_control_strict_mode From dbcbd7ab0c4f27d257d830ce8f3575cb0b5c18ea Mon Sep 17 00:00:00 2001 From: xixirangrang Date: Fri, 7 Feb 2025 15:53:51 +0800 Subject: [PATCH 22/25] Update system-variable-reference.md --- system-variable-reference.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/system-variable-reference.md b/system-variable-reference.md index 6b753b842c0aa..f23dd23790a60 100644 --- a/system-variable-reference.md +++ b/system-variable-reference.md @@ -3353,7 +3353,7 @@ Referenced in: Referenced in: - [System Variables](/system-variables.md#tidb_request_source_type-new-in-v740) -- [Use Resource Control to Manage Background Tasks](/tidb-resource-control--background-tasks.md) +- [Use Resource Control to Manage Background Tasks](/tidb-resource-control-background-tasks.md) - [TiDB 7.4.0 Release Notes](/releases/release-7.4.0.md) ### tidb_resource_control_strict_mode From 1a7a01a0956ebae4b7ccdcc5d96d9c9be3444da5 Mon Sep 17 00:00:00 2001 From: xixirangrang Date: Fri, 7 Feb 2025 16:11:52 +0800 Subject: [PATCH 23/25] Update TOC-tidb-cloud.md --- TOC-tidb-cloud.md | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/TOC-tidb-cloud.md b/TOC-tidb-cloud.md index ef557816c4de3..3fe999793939f 100644 --- a/TOC-tidb-cloud.md +++ b/TOC-tidb-cloud.md @@ -736,7 +736,10 @@ - [update](/tidb-cloud/ticloud-update.md) - [help](/tidb-cloud/ticloud-help.md) - [Table Filter](/table-filter.md) - - [Resource Control](/tidb-resource-control-ru-groups.md) + - Resource Manager + - [Use Resource Control to Achieve Resource Group Limitation and Flow Control](/tidb-resource-control-ru-groups.md) + - [Manage Runaway Queries](/tidb-resource-control-runaway-queries.md) + - [Manage Background Tasks](/tidb-resource-control-background-tasks.md) - [URI Formats of External Storage Services](/external-storage-uri.md) - [DDL Execution Principles and Best Practices](/ddl-introduction.md) - [Batch Processing](/batch-processing.md) From 8b6a84bbd9168e9eac9b6872d9e2c26b6152bee5 Mon Sep 17 00:00:00 2001 From: houfaxin Date: Fri, 7 Feb 2025 16:41:54 +0800 Subject: [PATCH 24/25] Update sql-statement-calibrate-resource.md --- sql-statements/sql-statement-calibrate-resource.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/sql-statements/sql-statement-calibrate-resource.md b/sql-statements/sql-statement-calibrate-resource.md index 2081409d55aa4..24857eca56602 100644 --- a/sql-statements/sql-statement-calibrate-resource.md +++ b/sql-statements/sql-statement-calibrate-resource.md @@ -5,7 +5,7 @@ summary: An overview of the usage of CALIBRATE RESOURCE for the TiDB database. # `CALIBRATE RESOURCE` -The `CALIBRATE RESOURCE` statement is used to estimate and output the ['Request Unit (RU)`](/tidb-resource-control#what-is-request-unit-ru) capacity of the current cluster. +The `CALIBRATE RESOURCE` statement is used to estimate and output the ['Request Unit (RU)`](/tidb-resource-control-ru-groups.md#what-is-request-unit-ru) capacity of the current cluster. > **Note:** > From 58e8050ef05a4148777dc0ce06ccf1089a5f0c53 Mon Sep 17 00:00:00 2001 From: xixirangrang Date: Fri, 7 Feb 2025 17:45:37 +0800 Subject: [PATCH 25/25] Apply suggestions from code review Co-authored-by: Grace Cai --- system-variable-reference.md | 2 +- tidb-resource-control-background-tasks.md | 4 ++-- tidb-resource-control-runaway-queries.md | 10 ++++++---- 3 files changed, 9 insertions(+), 7 deletions(-) diff --git a/system-variable-reference.md b/system-variable-reference.md index f23dd23790a60..00e6eac1ee4c3 100644 --- a/system-variable-reference.md +++ b/system-variable-reference.md @@ -1895,7 +1895,7 @@ Referenced in: - [SET RESOURCE GROUP](/sql-statements/sql-statement-set-resource-group.md) - [System Variables](/system-variables.md#tidb_enable_resource_control-new-in-v660) - [TiKV Configuration File](/tikv-configuration-file.md) -- [Use Resource Control to Achieve Resource Group Limitation](/tidb-resource-control-ru-groups.md) +- [Use Resource Control to Achieve Resource Group Limitation and Flow Control](/tidb-resource-control-ru-groups.md) - [`CALIBRATE RESOURCE`](/sql-statements/sql-statement-calibrate-resource.md) - [TiDB 7.4.0 Release Notes](/releases/release-7.4.0.md) - [TiDB 7.0.0 Release Notes](/releases/release-7.0.0.md) diff --git a/tidb-resource-control-background-tasks.md b/tidb-resource-control-background-tasks.md index 1b872de42b2d9..537e010a8b48a 100644 --- a/tidb-resource-control-background-tasks.md +++ b/tidb-resource-control-background-tasks.md @@ -9,11 +9,11 @@ summary: Introduces how to control background tasks through Resource Control. > > This feature is experimental. It is not recommended that you use it in the production environment. This feature might be changed or removed without prior notice. If you find a bug, you can report an [issue](https://docs.pingcap.com/tidb/stable/support) on GitHub. > -> The background task management in resource control is based on TiKV's dynamic adjustment of resource quotas for CPU/IO utilization. Therefore, it relies on the available resource quota of each instance. If multiple components or instances are deployed on a single server, it is mandatory to set the appropriate resource quota for each instance through `cgroup`. It is difficult to achieve the expected effect in deployment with shared resources like TiUP Playground. +> The background task management in resource control is based on TiKV's dynamic adjustment of resource quotas for CPU/IO utilization. Therefore, it relies on the available resource quota of each instance. If multiple components or instances are deployed on a single server, it is mandatory to set the appropriate resource quota for each instance through `cgroup`. It is difficult to achieve the expected effect in deployment with shared resources such as TiUP Playground. Background tasks, such as data backup and automatic statistics collection, are low-priority but consume many resources. These tasks are usually triggered periodically or irregularly. During execution, they consume a lot of resources, thus affecting the performance of online high-priority tasks. -Starting from v7.4.0, the TiDB resource control feature supports managing background tasks. When a task is marked as a background task, TiKV dynamically limits the resources used by this type of task to avoid the impact on the performance of other foreground tasks. TiKV monitors the CPU and IO resources consumed by all foreground tasks in real time, and calculates the resource threshold that can be used by background tasks based on the total resource limit of the instance. All background tasks are restricted by this threshold during execution. +Starting from v7.4.0, the [TiDB resource control](/tidb-resource-control-ru-groups.md) feature supports managing background tasks. When a task is marked as a background task, TiKV dynamically limits the resources used by this type of task to avoid the impact on the performance of other foreground tasks. TiKV monitors the CPU and IO resources consumed by all foreground tasks in real time, and calculates the resource threshold that can be used by background tasks based on the total resource limit of the instance. All background tasks are restricted by this threshold during execution. ## `BACKGROUND` parameters diff --git a/tidb-resource-control-runaway-queries.md b/tidb-resource-control-runaway-queries.md index f3f2305929e3e..9a7d3a313bb19 100644 --- a/tidb-resource-control-runaway-queries.md +++ b/tidb-resource-control-runaway-queries.md @@ -10,13 +10,15 @@ A runaway query is a query that consumes more time or resources than expected. T - Starting from v7.2.0, the resource control feature introduces the management of runaway queries. You can set criteria for a resource group to identify runaway queries and automatically take actions to prevent them from exhausting resources and affecting other queries. You can manage runaway queries for a resource group by including the `QUERY_LIMIT` field in [`CREATE RESOURCE GROUP`](/sql-statements/sql-statement-create-resource-group.md) or [`ALTER RESOURCE GROUP`](/sql-statements/sql-statement-alter-resource-group.md). - Starting from v7.3.0, the resource control feature introduces manual management of runaway watches, enabling quick identification of runaway queries for a given SQL statement or Digest. You can execute the statement [`QUERY WATCH`](/sql-statements/sql-statement-query-watch.md) to manually manage the runaway queries watch list in the resource group. +For more information about the resource control feature, see [Use Resource Control to Achieve Resource Group Limitation and Flow Control](/tidb-resource-control-ru-groups.md). + ## `QUERY_LIMIT` parameters If a query exceeds any of the following limits, it is identified as a runaway query: -- `EXEC_ELAPSED`: checks whether the query execution time exceeds the limit, this rule applies to read and write DML. -- `PROCESSED_KEYS`: checks whether the number of keys processed by the Coprocessor exceeds the limit, this rule applies only to read statements. -- `RU`: checks whether the total number of read and write RUs consumed by the statement exceeds the limit, this rule applies only to read statements. +- `EXEC_ELAPSED`: checks whether the query execution time exceeds the limit. This rule applies to read and write DML statements. +- `PROCESSED_KEYS`: checks whether the number of keys processed by the Coprocessor exceeds the limit. This rule only applies to read statements. +- `RU`: checks whether the total number of read and write RUs consumed by the statement exceeds the limit. This rule only applies to read statements. Supported operations (`ACTION`): @@ -49,7 +51,7 @@ The parameters of `QUERY_LIMIT` are as follows: > **Note:** > -> If you want to strictly limit runaway queries to a specific resource group, it is recommended to use `SWITCH_GROUP` together with the [`QUERY WATCH`](/tidb-resource-control-runaway-queries.md#query-watch-parameters) statement. Because `QUERY_LIMIT` only triggers the corresponding `ACTION` operation when the query meets the criteria, `SWITCH_GROUP` might not be able to switch the query to the target resource group in a timely manner in such scenarios. +> If you want to strictly limit runaway queries to a specific resource group, it is recommended to use `SWITCH_GROUP` together with the [`QUERY WATCH`](#query-watch-parameters) statement. Because `QUERY_LIMIT` only triggers the corresponding `ACTION` operation when the query meets the criteria, `SWITCH_GROUP` might not be able to switch the query to the target resource group in a timely manner in such scenarios. ## Examples