Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Autoscale vtgate replicas using HPA #3945

Closed
wants to merge 11 commits into from
Closed

Autoscale vtgate replicas using HPA #3945

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
11 commits
Select commit Hold shift + click to select a range
c5a7b27
DREIMP-10951: autoscale vtgate deployments using HPA
siadat Aug 22, 2024
39b71f6
rename Scalable to Vtgate
siadat Aug 22, 2024
8758c28
Make it explicit which CRs update related objects
siadat Sep 3, 2024
bb1f149
Remove hard-coded HPA specs
siadat Sep 3, 2024
91411fb
Rename Vtgate to ScalableVtgate
siadat Sep 12, 2024
eb997b5
Define ScalableVtgate metadata in one place
siadat Sep 12, 2024
0daabb8
HPA name should be the same as the cell, not cluster
siadat Sep 12, 2024
b382e1f
Set owner_references for vtgate HPAs
siadat Sep 12, 2024
e0a8bc9
Fix error message
siadat Sep 12, 2024
7288c60
Fix mypy errors
siadat Sep 12, 2024
d078a30
Fix tests
siadat Sep 12, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 18 additions & 0 deletions paasta_tools/setup_kubernetes_cr.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,9 @@
from paasta_tools.utils import load_all_configs
from paasta_tools.utils import load_system_paasta_config
from paasta_tools.vitesscluster_tools import load_vitess_service_instance_configs
from paasta_tools.vitesscluster_tools import (
update_related_api_objects as update_vitess_cluster_related_api_objects,
)


log = logging.getLogger(__name__)
Expand All @@ -58,6 +61,11 @@
INSTANCE_TYPE_TO_CONFIG_LOADER = {"vitesscluster": load_vitess_service_instance_configs}


INSTANCE_TYPE_TO_RELATED_OBJECTS_UPDATER = {
"vitesscluster": update_vitess_cluster_related_api_objects,
}


class StdoutKubeClient:
"""Replace all destructive operations in Kubernetes APIs with
writing out YAML to stdout."""
Expand Down Expand Up @@ -362,6 +370,7 @@ def reconcile_kubernetes_resource(
instance=inst,
cluster=cluster,
soa_dir=DEFAULT_SOA_DIR,
kube_client=kube_client,
)
git_sha = get_git_sha_from_dockerurl(soa_config.get_docker_url(), long=True)
formatted_resource = format_custom_resource(
Expand Down Expand Up @@ -414,6 +423,15 @@ def reconcile_kubernetes_resource(
)
else:
log.info(f"{desired_resource} is up to date, no action taken")

if crd.file_prefix in INSTANCE_TYPE_TO_RELATED_OBJECTS_UPDATER:
INSTANCE_TYPE_TO_RELATED_OBJECTS_UPDATER[crd.file_prefix](
service=service,
instance=inst,
cluster=cluster,
kube_client=kube_client,
soa_dir=DEFAULT_SOA_DIR,
Comment on lines +427 to +433
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we'd want a log message right before this as in

paasta/paasta_tools/setup_kubernetes_job.py

Line 323 in d078a30

log.info(f"Ensuring related API objects for {app} are in sync")
re: updating additional objects

I think I also need to take a closer look at what {setup,cleanup}_kubernetes_cr() currently do and see if there's anything we need to worry about there re: a CRD that's not created with create_custom_resource()

@EvanKrall @Rob-Johnson I'm also a little unsure about what the best way to do this is - I'm not sure if we'd want to have this sort of conditional logic based on the instance type or if we'd want to to something at the base class that all these instance types use so that updating CRs that aren't directly tied to a CRD that paasta discovers

)
except Exception as e:
log.error(str(e))
succeeded = False
Expand Down
Loading
Loading