Remove the subset of multiple yaml keys and list items

[vikas027]

My apologies in advance for the big thread. I am looking for a way to find a way of removing items from multiple lists and then add them back to the yaml. I understand this cannot be a one-line operation and may need a few lines of code. All I am looking to use dasel for the same, and trying to save myself a python script :P

I have two yaml files - big-clusterrole and small-clusterrole. Please note that

• file big-clusterrole is just an example. The actual file is quite big (hundreds of lines)
• list elements like apiGroups, resources can be in any order (which makes the task a bit difficult)

big-clusterrole

---
# This is just an example role
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: big-clusterrole
rules:
- apiGroups:
  - ""
  resources:
  - podtemplates
  - replicationcontrollers
  - resourcequotas
  - secrets
  - serviceaccounts
  verbs:
  - create
  - delete
  - get
  - list
  - patch
  - update
  - watch
- apiGroups:
  - apps
  - batch
  resources:
  - jobs
  - cronjobs
  verbs:
  - create
  - delete
  - get
  - list
  - patch
  - update
  - watch
- apiGroups:
  - vpcresources.k8s.aws
  resources:
  - securitygrouppolicies
  verbs:
  - create
  - delete
  - deletecollection
  - get
  - list

small-clusterrole (subset of the big-clusterrole)

---
# This role is a subset of the big-clusterrole
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: small-clusterrole
rules:
- apiGroups:
  - ""
  resources:
  - secrets
  verbs:
  - get
  - list
  - watch
- apiGroups:
  - batch
  resources:
  - jobs
  verbs:
  - create
  - delete
- apiGroups:
  - vpcresources.k8s.aws
  resources:
  - securitygrouppolicies
  verbs:
  - get
  - list

What I want is to do the below (as per my understanding) :)

• remove the resources (list items) from the present in the small-clusterrole from the big-clusterrole and create another yaml intended-clusterrole
• If there is only resource in the apiGroups key (like securitygrouppolicies in my example), then the whole apiGroups key need to be deleted
• append the rules of small-clusterrole to the intended-clusterrole (created in the previous step)

intended-clusterrole

---
# The resultant yaml will look something like this
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: intended-clusterrole
rules:
- apiGroups:
  - ""
  resources:
  - podtemplates
  - replicationcontrollers
  - resourcequotas
  - serviceaccounts
  verbs:
  - create
  - delete
  - get
  - list
  - patch
  - update
  - watch
- apiGroups:
  - apps
  resources:
  - jobs
  - cronjobs
  verbs:
  - create
  - delete
  - get
  - list
  - patch
  - update
  - watch
- apiGroups:
  - ""
  resources:
  - secrets
  verbs:
  - get
  - list
  - watch
- apiGroups:
  - batch
  resources:
  - jobs
  verbs:
  - create
  - delete
- apiGroups:
  - vpcresources.k8s.aws
  resources:
  - securitygrouppolicies
  verbs:
  - get
  - list

[TomWright]

Hey @vikas027,

Could I get a reduced example of what you'd like to happen here? It's a little hard to follow.

[vikas027]

Hey @TomWright ,

Yeah, I know a little hard to follow.

Let me try to shorten it up.

---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: big-clusterrole
rules:
- apiGroups:
  - ""
  resources:
  - podtemplates
  - replicationcontrollers
  - resourcequotas
  - secrets
  - serviceaccounts
  verbs:
  - create
  - delete
  - get
  - list
  - patch
  - update
  - watch
 
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: small-clusterrole
rules:
- apiGroups:
  - ""
  resources:
  - secrets
  verbs:
  - get
  - list
  - watch

I need to find a way to filter the resources listed in a yaml (the second one here), in my example it has secrets.
Once, I have the keyword secrets, I need to remove it from the other yaml (the first one).
And then finally, merge all the rules from the second yaml to the first one.

Sorry, the more I am thinking about it now, it looks like more of a programming gig. Don't worry if it is out of dasel scope, I can code it in Python easily.

[TomWright]

It's probably do-able with dasel and some bash script although I'd argue it would be easier to read in a python script.