Feature Request: CRD for Humio Groups

[jonnylangefeld]

We utilize the Humio Operator to configure Humio Views for different users across the org.
We have okta groups for the different teams accessing the different views.
Currently we're using the Humio Operator to create the Humio Views and then we still need to integrate with the Humio API to do the graphql mutation to add the Okta group and add the respective roles to the Humio Group.
Our ideal would be that we only need to integrate with the Kubernetes API that has the operator installed and not partially use the Kubernetes API and partially use the Humio API via our provisioner.

We think Humio Groups should be represented as CRD object via the Humio Operator. That way we can just create HumioView and HumioGroup custom resources and only need to integrate with the K8s API.

[CJNA]

Proposal

Add a HumioGroup resource as a CRD.

• Humio Group CRD will have fields
  `displayName: String!
  defaultQueryPrefix: String
  defaultRole: Role
  lookupName: String
  searchDomainRoles: []SearchDomainRole
  reconcileSearchDomainQueryPrefix: bool

struct SearchDomainRole {
name: String,
role: Role
} (Represent this as 'object')

struct Role {
displayName: String
viewPermissions: String[]
systemPermissions: String[]
organizationPermissions: String[]
} (Represent this as 'object')
`

• reconciler will issue following operations

-> Initially, reconciler will try to find a group by displayName or lookupName
-> If group not found, reconciler will issue a mutation { addGroup } with given displayName
-> Reconciler will try to set defaultQueryPrefix if group's defaultQueryPrefix is set differently than CustomResources via
``
-> Reconciler will try to set the defaultRole if group's defaultRole is set differently
-> Reconciler will look through searchDomainRoles and assign roles to Group per provided searchDomainRoles
-> If reconcileSearchDomainQueryPrefix is true, set the queryPrefix of searchDomain to defaultQueryPrefix of CR

[CJNA]

Been thinking about this further.

I think it makes sense to create separate CRD for Role as well.

HumioRole itself should be defined separate, so that it can be controlled by Reconciler.

EX:

struct Role { displayName: String viewPermissions: String[] systemPermissions: String[] organizationPermissions: String[] } (Represent this as 'object')

And
defaultRole: string field should take in the displayName of Group to search for group.

This also simplifies object like
struct SearchDomainRole { name: String, role: String }
as it can fetch from string object

Thoughts?

[CJNA]

#621

This is a draft PR posted on my Fork.
Would appreciate an eye

[jswoods]

Hi @CJNA - I appreciate you taking the time to look at this. One thing I'd like to bring up is that the pattern we have established is that the operator does not make direct graphql calls to humio, but rather it makes calls to the humio client api which then makes the graphql calls. The humio client api is part of https://github.com/humio/cli. As you see there, we are missing critical functionality such as managing roles, and adding, deleting or updating groups (though some functionality, like adding users to groups is there).

I think developing a guideline here will be important to be able to implement the requirements we need from the client api, but before we can do any real work on the operator we need this functionality on the client api.

From the operator perspective, splitting the resources into separate Roles and Groups sounds like a good plan. From there, we should start with some more high level logic:

A reconcile for roles, where we add/delete/update based on the defined Roles CRs.
A reconcile for groups, where we add/delete/update based on the defined Groups CRs.
A way to link a list of repos/views to a group, which also contains a link to a corresponding role.
A way to link a list of users to a group.

I suspect the way we would like to "link" these things is by referencing the names of the resources; however, it's possible it may not be that simple, or that there may be a built-in way to do this. For example, how the views manage connections to repos https://github.com/humio/humio-operator/blob/master/api/v1alpha1/humioview_types.go#L53-L54. There may be cases where we have to pass a generated ID as the link, but then first have to look up the resource by name to fetch the ID to make it human-usable.

Before deciding on the exact details of how to link these resources, I suspect some discovery needs to be done on the client api, which also relies on the graphql api. Once the base actions of managing roles and groups are established at the client api level, the relationships will become much clearer and as we'll be forced to some degree to abide by the structure presented by the graphql api for the individual resources, and perhaps also to the relationships between them.

Hopefully this makes sense. I have not looked deeply at how we manage roles or groups from the graphql api, but I suspect it wont be too different than what we do with other resources already managed by the client api.

[CJNA]

Hi @jswoods .
Makes all sense to me.

I'll look for adding a graphql call on humio/cli repo first for managing role/group resouces.

[CJNA]

One thing to note tho!
I've realized I can reduce the scope of this by eliminating structure of Role.

What I mean by is that, Group 'can' just look for existing Role by searching for it's name.

searchByViewQuery := fmt.Sprintf( "query { searchDomain(name:\"%v\") { id name } roles { id displayName } groupByDisplayName(displayName:\"%v\") { id displayName lookupName } }", viewName, oktaGroup)

And we could just get by running something like

assignRoleToGroupQuery := fmt.Sprintf( "mutation { assignRoleToGroup(input: {viewId: \"%v\", groupId: \"%v\", roleId: \"%v\"}) { clientMutationId } }", domainID, groupID, roleID)

on create/update Groups.

Next step of PRs would be to actually create/update/delete Roles, and 'link' roles, but this does not seem to be necessary to get what I need.
This is to makes sure I can wrap HumioGroup part this hackathon week, so that I can work on Roles later on

[CJNA]

Hi team. I've made a quick PR for Humio Group
https://github.com/humio/cli/compare/master...CJNA:cli:adding_client_logic?expand=1

However, this update feature lacks any real update, as I've listed in description.
PTAL and let me know if there's a way we can change grahpql Input for UpdateGroup mutation.

Also, I noticed Nam has pushed a PR for adding Humio Roles. PTAL!
humio/cli#114

[CJNA]

Hi @jswoods

Could you take a look at #621 and humio/cli#116

This is a pre-requisite to use the Humio Group on
#621

[CJNA]

This is a follow up to https://github.com/humio/cli/pull/114/files
PTAL!
#640

Will start writing logic for reconciler once this gets reviewed and merged