unable to see/list RDS databases exposed via teleport-kube-agents #33103
dmitry-mightydevops
started this conversation in
General
Replies: 5 comments 6 replies
-
|
Run |
Beta Was this translation helpful? Give feedback.
0 replies
-
using local tctl after tsh login➜ tsh login --proxy=teleport.project.net:443 --auth=github
If browser window does not open automatically, open it by clicking on the link:
http://127.0.0.1:34525/e713d1fa-e124-4c9f-95db-0feda6190eb2
> Profile URL: https://teleport.project.net:443
Logged in as: dmitry-mightydevops
Cluster: teleport.project.net
Roles: administrators
Logins: administrators
Kubernetes: enabled
Kubernetes groups: system:masters
Valid until: 2023-10-11 04:52:24 -0500 CDT [valid for 8h0m0s]
Extensions: login-ip, permit-port-forwarding, permit-pty, private-key-policy
Did you know? Teleport Connect offers the power of tsh in a desktop app.
Learn more at https://goteleport.com/docs/connect-your-client/teleport-connect/
➜ tctl get db_service | wc
0 0 0
➜ tctl get db_server | wc
0 0 0if run against agent pod (teleport-0):➜ kgpo
NAME READY STATUS RESTARTS AGE
teleport-0 1/1 Running 0 4d4h
teleport-auth-6c8d4d7d49-t2rm9 2/2 Running 0 4d4h
teleport-proxy-6497d88f87-n4nt8 1/1 Running 0 6d
✗ k exec -ti teleport-0 -- tctl get db_service
ERROR: Could not load Teleport host UUID file at /var/lib/teleport/host_uuid. Please make sure that Teleport is up and running prior to using tctl.
lstat /var/lib/teleport/host_uuid: no such file or directory
command terminated with exit code 1
✗ k exec -ti teleport-0 -- tctl get db_server
ERROR: Could not load Teleport host UUID file at /var/lib/teleport/host_uuid. Please make sure that Teleport is up and running prior to using tctl.
lstat /var/lib/teleport/host_uuid: no such file or directory
command terminated with exit code 1if run against teleport-auth pod:✗ k exec -ti teleport-auth-6c8d4d7d49-t2rm9 -- tctl get db_service
Defaulted container "teleport" out of: teleport, operator
kind: db_service
metadata:
expires: "2023-10-11T02:06:26Z"
id: 1696628071717071093
name: f03526de-dc86-49b2-99f3-f1c798261484
spec:
resources: null
version: v1
➜ k exec -ti teleport-auth-6c8d4d7d49-t2rm9 -- tctl get db_server
Defaulted container "teleport" out of: teleport, operator
kind: db_server
metadata:
expires: "2023-10-11T02:06:52Z"
id: 1696628086526007343
name: project-prod-backend-postgres-rds-us-east-1-111111111111
spec:
database:
kind: db
metadata:
description: RDS instance in us-east-1
labels:
account-id: "111111111111"
client: project
component: backend
created_at: 06/12/2023
endpoint-type: instance
engine: postgres
engine-version: "15.3"
environment: prod
owner: Saritasa
project: project
region: us-east-1
repo: [email protected]:organization/project-infra-aws
teleport.dev/cloud: AWS
teleport.dev/origin: cloud
teleport.internal/discovered-name: project-prod-backend-postgres
terraform: "true"
name: project-prod-backend-postgres-rds-us-east-1-111111111111
spec:
ad:
domain: ""
spn: ""
aws:
account_id: "111111111111"
elasticache: {}
iam_policy_status: IAM_POLICY_STATUS_UNSPECIFIED
memorydb: {}
opensearch: {}
rds:
iam_auth: true
instance_id: project-prod-backend-postgres
resource_id: db-RCDLL3GFPHFRFKNCO2PT7FD5MM
rdsproxy: {}
redshift: {}
redshift_serverless: {}
region: us-east-1
secret_store: {}
azure:
redis: {}
gcp: {}
mongo_atlas: {}
mysql: {}
oracle:
audit_user: ""
protocol: postgres
tls:
mode: 0
uri: project-prod-backend-postgres.chipb2whnj74.us-east-1.rds.amazonaws.com:5432
status:
aws:
account_id: "111111111111"
elasticache: {}
iam_policy_status: IAM_POLICY_STATUS_FAILED
memorydb: {}
opensearch: {}
rds:
iam_auth: true
instance_id: project-prod-backend-postgres
resource_id: db-RCDLL3GFPHFRFKNCO2PT7FD5MM
rdsproxy: {}
redshift: {}
redshift_serverless: {}
region: us-east-1
secret_store: {}
azure:
redis: {}
ca_cert: |
-----BEGIN CERTIFICATE-----
MIIEBjCCAu6gAwIBAgIJAMc0ZzaSUK51MA0GCSqGSIb3DQEBCwUAMIGPMQswCQYD
...
-----END CERTIFICATE-----
-----BEGIN CERTIFICATE-----
MIIEBzCCAu+gAwIBAgICJVUwDQYJKoZIhvcNAQELBQAwgY8xCzAJBgNVBAYTAlVT
MRAwDgYDVQQHDAdTZWF0dGxlMRMwEQYDVQQIDApXYXNoaW5ndG9uMSIwIAYDVQQK
...
-----END CERTIFICATE-----
-----BEGIN CERTIFICATE-----
MIID/zCCAuegAwIBAgIRAPVSMfFitmM5PhmbaOFoGfUwDQYJKoZIhvcNAQELBQAw
gZcxCzAJBgNVBAYTAlVTMSIwIAYDVQQKDBlBbWF6b24gV2ViIFNlcnZpY2VzLCBJ
bmMuMRMwEQYDVQQLDApBbWF6b24gUkRTMQswCQYDVQQIDAJXQTEwMC4GA1UEAwwn
....
-----END CERTIFICATE-----
-----BEGIN CERTIFICATE-----
MIIF/jCCA+agAwIBAgIQaRHaEqqacXN20e8zZJtmDDANBgkqhkiG9w0BAQwFADCB
lzELMAkGA1UEBhMCVVMxIjAgBgNVBAoMGUFtYXpvbiBXZWIgU2VydmljZXMsIElu
...
-----END CERTIFICATE-----
-----BEGIN CERTIFICATE-----
MIICrjCCAjSgAwIBAgIRAPAlEk8VJPmEzVRRaWvTh2AwCgYIKoZIzj0EAwMwgZYx
CzAJBgNVBAYTAlVTMSIwIAYDVQQKDBlBbWF6b24gV2ViIFNlcnZpY2VzLCBJbmMu
...
-----END CERTIFICATE-----
mysql: {}
version: v3
host_id: f03526de-dc86-49b2-99f3-f1c798261484
hostname: teleport-0
rotation:
current_id: ""
last_rotated: "0001-01-01T00:00:00Z"
schedule:
standby: "0001-01-01T00:00:00Z"
update_clients: "0001-01-01T00:00:00Z"
update_servers: "0001-01-01T00:00:00Z"
started: "0001-01-01T00:00:00Z"
version: 14.0.2
version: v3
|
Beta Was this translation helpful? Give feedback.
0 replies
-
|
Add |
Beta Was this translation helpful? Give feedback.
0 replies
-
|
@webvictim I have that already applied via teleport-operator: verified in GUI management/roles - and I see no DBs in the GUI. also logout/login - still only see kubernetes clusters, and nothing else. Expect to see - apps and DBs that got discovered in the teleport-kube-agent |
Beta Was this translation helpful? Give feedback.
1 reply
-
|
I didn't see Whenever you can't see a database, it's generally a permissions problem. I don't see a reason why your role shouldn't work, though. The database seems to be registered with the backend. |
Beta Was this translation helpful? Give feedback.
-
|
yes @webvictim I added db_server - later after you mentioned it, but it had no effect at all. ➜ tsh login --proxy=teleport.projectapp.net:443 --auth=github
> Profile URL: https://teleport.projectapp.net:443
Logged in as: dmitry-mightydevops
Cluster: teleport.projectapp.net
Roles: administrators
Logins: administrators
Kubernetes: enabled
Kubernetes groups: system:masters
Valid until: 2023-10-12 00:38:17 -0500 CDT [valid for 7h57m0s]
Extensions: login-ip, permit-port-forwarding, permit-pty, private-key-policy
on ☸ project-prod-eks (teleport) ~/Projects
➜ tctl db ls
Host Name Protocol URI Labels Version
---- ---- -------- --- ------ -------
on ☸ project-prod-eks (teleport) ~/Projects
➜ k exec -ti teleport-auth-6c8d4d7d49-t2rm9 -- tctl db ls
Defaulted container "teleport" out of: teleport, operator
Host Name Protocol URI Labels Version
---------- ----------------------------- -------- -------------------------------------------------- ------------------------------------------------------------------------------------------------------------------------------------------------- -------
teleport-0 project-prod-backend-postgres postgres project-prod-backend-postgres.chipb2whnj74.us-e... account-id=303629721857,client=project,component=backend,created_at=06/12/2023,endpoint-type=instance,engine-version=... 14.0.2
but in gui no DBs :(
|
Beta Was this translation helpful? Give feedback.
5 replies
-
|
Could you also humour me and share the output of |
Beta Was this translation helpful? Give feedback.
-
|
@webvictim I was travelling, sorry for the delayed resp. Here you go: |
Beta Was this translation helpful? Give feedback.
-
|
One thing to note - I create the role via kubernetes operator and it created |
Beta Was this translation helpful? Give feedback.
-
|
so it's definitely the problem with the teleport operator because if I apply the role this way: ➜ POD=$(kubectl get pods -n teleport --no-headers -o="custom-columns=NAME:.metadata.name" --selector=app.kubernetes.io/component="auth")
➜ kubectl exec -i ${POD?} -- tctl create -f < ./todo/teleport/role-v7.yaml
Defaulted container "teleport" out of: teleport, operator
role 'administrators' has been createdwith the role defined as v7: kind: role
version: v7
metadata:
name: administrators
spec:
allow:
logins:
- administrators
kubernetes_groups:
- system:masters
node_labels:
'*': '*'
kubernetes_labels:
'*': '*'
kubernetes_resources:
- kind: pod
name: '*'
namespace: ops
verbs:
- '*'
app_labels:
'*': '*'
db_service_labels:
'*': '*'
db_labels:
'*': '*'
db_users: ["*"]
db_names: ["*"]
rules:
- resources:
- access_list
- app
- assistant
- auth_connector
- billing
- cluster_alert
- cluster_audit_config
- cluster_auth_preference
- cluster_name
- cluster_networking_config
- connection_diagnostic
- database_certificate
- db
- github
- github_request
- installer
- integration
- lock
- login_rule
- node
- oidc
- oidc_request
- okta_assignment
- okta_import_rule
- plugin
- remote_cluster
- role
- saml
- saml_idp_service_provider
- saml_request
- session
- session_recording_config
- token
- trusted_cluster
- ui_config
- user
- user_group
verbs:
- create
- delete
- list
- read
- update
- use
- resources:
- device
verbs:
- create
- create_enroll_token
- delete
- enroll
- list
- read
- update
- resources:
- db_service
- db_server
verbs:
- list
- read
- use
- resources:
- instance
verbs:
- list
- read
- use
- resources:
- session
verbs:
- list
- read
options:
max_session_ttl: 8h
client_idle_timeout: 30m
ssh_file_copy: false
disconnect_expired_cert: true
enhanced_recording:
- command
- network
require_session_mfa: 'yes'
lock: strict
record_session:
default: strict
desktop: falseeverything worked and I see immediately the resources via discovery: |
Beta Was this translation helpful? Give feedback.
-
|
The There is a big issue for tracking improvements to the operator here: #29480 In the meantime, all I can advise is that you apply your roles using YAML or Terraform instead. Sorry! |
Beta Was this translation helpful? Give feedback.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
I login via github SSO connector
➜ tsh clusters Cluster Name Status Cluster Type Labels Selected ----------------------- ------ ------------ ------ -------- teleport.project.net online root * ➜ tsh kube ls Kube Cluster Name Labels Selected ----------------------- ------ -------- teleport.project.net project-prod-eks ➜ tsh db ls Name Description Allowed Users Labels Connect ---- ----------- ------------- ------ ------- ➜ tctl db ls Host Name Protocol URI Labels Version ---- ---- -------- --- ------ ------- ➜ tsh kubectl get pods NAME READY STATUS RESTARTS AGE teleport-0 1/1 Running 0 8m6s teleport-auth-6c8d4d7d49-t2rm9 2/2 Running 0 27m teleport-proxy-6497d88f87-n4nt8 1/1 Running 0 44hlogs of the teleport-0 (agent):
But I don't see anything in GUI (just 2 clusters) nor via tctl or tsh
my role applied via teleport-operator:
what am I missing?
Beta Was this translation helpful? Give feedback.
All reactions