mcs, client: support returning ru statistics in Get/ListResourceGroup response #7617

glorv · 2023-12-25T05:53:37Z

What problem does this PR solve?

Issue Number: Close #7616

What is changed and how does it work?

- persist resource group ru consumption in meta storage
- Add optional param `with_stats` to allow GetResourceGroup/ListResourceGroups return ru consumption data.

Check List

Tests

Unit test
Integration test

Code changes

Has HTTP API interfaces changed (Don't forget to add the declarative for the new API)
Has persistent data change

Side effects

Related changes

PR to update pingcap/docs/pingcap/docs-cn:
PR to update pingcap/tiup:
Need to cherry-pick to the release branch

Release note

None.

Signed-off-by: glorv <[email protected]>

ti-chi-bot · 2023-12-25T05:53:40Z

[REVIEW NOTIFICATION]

This pull request has been approved by:

CabinfeverB
nolouch

To complete the pull request process, please ask the reviewers in the list to review by filling /cc @reviewer in the comment.
After your PR has acquired the required number of LGTMs, you can assign this pull request to the committer in the list by filling /assign @committer in the comment to help you merge this pull request.

The full list of commands accepted by this bot can be found here.

Reviewer can indicate their review by submitting an approval review.
Reviewer can cancel approval by submitting a request changes review.

glorv · 2023-12-25T05:54:06Z

/cc @nolouch @CabinfeverB @HuSharp

Signed-off-by: glorv <[email protected]>

nolouch · 2023-12-26T06:33:56Z

pkg/mcs/resourcemanager/server/resource_group.go

+
+		if rg.RUConsumption != nil {
+			consumption := *rg.RUConsumption
+			group.RUStats = &consumption


peresit consumption both in ResourceGroup.persistSettings and SaveResourceGroupStates? I think we can only persist once in GroupState.

This is the side effect that both grpc and persistSettings uses IntoProtoResourceGroup to serialize resource group. The RUStats in group setting is not used.

So is RuStats useless? can we remove it from proto.ResourceGroup?

RuStats is used in the grpc response

CabinfeverB · 2023-12-26T08:53:01Z

go.mod

 	gotest.tools/gotestsum v1.7.0
 )

+require (
+	google.golang.org/genproto/googleapis/api v0.0.0-20231002182017-d307bd883b97 // indirect


need format

Signed-off-by: glorv <[email protected]>

client/resource_manager_client.go

bufferflies · 2023-12-26T10:07:31Z

pkg/mcs/resourcemanager/server/manager.go

 		}
+		m.RUnlock()


why needs to change this?

The original code is incorrect and unnecessary. 1) there is possible concurrency issue between RUnlock and Lock; 2) persistStates only requires read lock, thus no need to acquire Lock here.

Signed-off-by: glorv <[email protected]>

glorv · 2023-12-27T03:39:49Z

/retest

Signed-off-by: glorv <[email protected]>

codecov · 2023-12-27T04:11:25Z

Codecov Report

Merging #7617 (260570c) into master (4ed2605) will increase coverage by 0.07%.
The diff coverage is 97.91%.

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #7617      +/-   ##
==========================================
+ Coverage   74.81%   74.88%   +0.07%     
==========================================
  Files         459      459              
  Lines       50615    50647      +32     
==========================================
+ Hits        37866    37927      +61     
+ Misses       9405     9388      -17     
+ Partials     3344     3332      -12

Flag	Coverage Δ
unittests	`74.88% <97.91%> (+0.07%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

CabinfeverB · 2023-12-27T04:16:50Z

/merge

ti-chi-bot · 2023-12-27T04:16:53Z

@CabinfeverB: It seems you want to merge this PR, I will help you trigger all the tests:

/run-all-tests

You only need to trigger /merge once, and if the CI test fails, you just re-trigger the test that failed and the bot will merge the PR for you after the CI passes.

If you have any questions about the PR merge process, please refer to pr process.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

ti-chi-bot · 2023-12-27T04:16:54Z

This pull request has been accepted and is ready to merge.

Commit hash: 260570c

support returning ru statistics in Get/ListResourceGroup resp

04131a8

Signed-off-by: glorv <[email protected]>

ti-chi-bot bot added the release-note-none Denotes a PR that doesn't merit a release note. label Dec 25, 2023

ti-chi-bot bot requested review from JmPotato and rleungx December 25, 2023 05:53

ti-chi-bot bot added the size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. label Dec 25, 2023

ti-chi-bot bot assigned CabinfeverB, HuSharp and nolouch Dec 25, 2023

CabinfeverB requested review from nolouch, CabinfeverB and HuSharp and removed request for JmPotato and rleungx December 25, 2023 05:54

fix swagger

7f510a2

Signed-off-by: glorv <[email protected]>

glorv force-pushed the rg-ru-stats branch from a070358 to 7f510a2 Compare December 25, 2023 06:08

make tidy

7774148

Signed-off-by: glorv <[email protected]>

ti-chi-bot bot added size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. and removed size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. labels Dec 25, 2023

glorv added 6 commits December 25, 2023 14:25

fix tidy again

c13f21f

Signed-off-by: glorv <[email protected]>

fix go.mod

ad842db

Signed-off-by: glorv <[email protected]>

fix check

32299aa

Signed-off-by: glorv <[email protected]>

Merge branch 'master' into rg-ru-stats

353db0a

exclude grpc CallbackSerializer

22ff774

Signed-off-by: glorv <[email protected]>

fix function comment

5170aee

Signed-off-by: glorv <[email protected]>

nolouch requested a review from bufferflies December 26, 2023 04:16

nolouch reviewed Dec 26, 2023

View reviewed changes

nolouch approved these changes Dec 26, 2023

View reviewed changes

ti-chi-bot bot added the status/LGT1 Indicates that a PR has LGTM 1. label Dec 26, 2023

nolouch unassigned CabinfeverB and HuSharp Dec 26, 2023

CabinfeverB reviewed Dec 26, 2023

View reviewed changes

glorv added 3 commits December 26, 2023 16:59

Merge branch 'master' into rg-ru-stats

310c6f8

fmt go.mod

ffa76e9

Signed-off-by: glorv <[email protected]>

Merge branch 'rg-ru-stats' of ssh://github.com/glorv/pd into rg-ru-stats

91bd6d1

CabinfeverB approved these changes Dec 26, 2023

View reviewed changes

ti-chi-bot bot added status/LGT2 Indicates that a PR has LGTM 2. and removed status/LGT1 Indicates that a PR has LGTM 1. labels Dec 26, 2023

do clean up

13d79e5

Signed-off-by: glorv <[email protected]>

bufferflies reviewed Dec 26, 2023

View reviewed changes

client/resource_manager_client.go Show resolved Hide resolved

bufferflies reviewed Dec 26, 2023

View reviewed changes

bufferflies approved these changes Dec 26, 2023

View reviewed changes

replace restart with resign

b2b70a6

Signed-off-by: glorv <[email protected]>

ti-chi-bot bot added size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. and removed size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. labels Dec 27, 2023

Merge branch 'master' of https://github.com/tikv/pd into rg-ru-stats

c821664

glorv added 2 commits December 27, 2023 11:45

try fix

eca1d61

Signed-off-by: glorv <[email protected]>

retry test

260570c

Signed-off-by: glorv <[email protected]>

ti-chi-bot bot added the status/can-merge Indicates a PR has been approved by a committer. label Dec 27, 2023

ti-chi-bot bot merged commit d3551ea into tikv:master Dec 27, 2023

glorv deleted the rg-ru-stats branch December 27, 2023 05:00

glorv mentioned this pull request Dec 27, 2023

*: update pd client tikv/client-go#1092

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

mcs, client: support returning ru statistics in Get/ListResourceGroup response #7617

mcs, client: support returning ru statistics in Get/ListResourceGroup response #7617

glorv commented Dec 25, 2023

ti-chi-bot bot commented Dec 25, 2023 •

edited

Loading

glorv commented Dec 25, 2023 •

edited

Loading

nolouch Dec 26, 2023

glorv Dec 26, 2023

nolouch Dec 26, 2023

glorv Dec 26, 2023

CabinfeverB Dec 26, 2023

glorv Dec 26, 2023

bufferflies Dec 26, 2023

glorv Dec 26, 2023

glorv commented Dec 27, 2023

codecov bot commented Dec 27, 2023 •

edited

Loading

CabinfeverB commented Dec 27, 2023

ti-chi-bot bot commented Dec 27, 2023

ti-chi-bot bot commented Dec 27, 2023

mcs, client: support returning ru statistics in Get/ListResourceGroup response #7617

mcs, client: support returning ru statistics in Get/ListResourceGroup response #7617

Conversation

glorv commented Dec 25, 2023

What problem does this PR solve?

What is changed and how does it work?

Check List

Release note

ti-chi-bot bot commented Dec 25, 2023 • edited Loading

glorv commented Dec 25, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

glorv commented Dec 27, 2023

codecov bot commented Dec 27, 2023 • edited Loading

Codecov Report

CabinfeverB commented Dec 27, 2023

ti-chi-bot bot commented Dec 27, 2023

ti-chi-bot bot commented Dec 27, 2023

ti-chi-bot bot commented Dec 25, 2023 •

edited

Loading

glorv commented Dec 25, 2023 •

edited

Loading

codecov bot commented Dec 27, 2023 •

edited

Loading