Merge pull request #4 from yashsharma127/PHEE-operator

[1] - Fix Importer-RDBMS Deployment Issues and Add Logging

PHEE-operator/ARCHITECTURE.md

@@ -1,43 +1,167 @@
-# Architecture
-This file contain the repo structure and information on each component operator include.
+# PHEE Operator Architecture
+
+This document provides an in-depth overview of the architecture of the PHEE Operator, detailing the design decisions, components, and their interactions.
 
 ## Repo Structure
 
+```
 PHEE-operator/
 ├── deploy/
+│   ├── cr/
+│   │   └── ph-ee-importer-rdbms-cr.yaml
 │   ├── crds/
-│   │   └── ph-ee-importer-rdbms-crd.yaml 
-│   ├── operator-deployment.yaml
-│   └── ph-ee-importer-rdbms-cr.yaml
+│   │   └── ph-ee-importer-rdbms-crd.yaml
+│   └── operator/
+│       └── operator.yaml
 ├── src/
 │   └── main/
 │       └── java/
 │           └── com/
 │               └── example/
-│                   └── operator/
-│                       ├── PhEeImporterRdbms.java
-│                       ├── PhEeImporterRdbmsController.java
-│                       ├── PhEeImporterRdbmsSpec.java 
-│                       └── OperatorMain.java
-├── pom.xml
-└── README.md
+│                   ├── customresource/ 
+│                   │   ├── PhEeImporterRdbms.java
+│                   │   ├── PhEeImporterRdbmsSpec.java
+│                   │   └── PhEeImporterRdbmsStatus.java
+│                   ├── utils/
+|                   │   ├── LoggingUtil.java 
+|                   │   └── StatusUpdateUtil.java
+│                   ├── OperatorMain.java
+|                   └── PhEeImporterRdbmsController.java
+└── pom.xml 
+```
+
+## Table of Contents
+
+1. [Introduction](#introduction)
+2. [Overview](#overview)
+3. [Components](#components)
+   - [Custom Resource Definition (CRD)](#custom-resource-definition-crd)
+   - [Custom Resource (CR)](#custom-resource-cr)
+   - [Operator](#operator)
+   - [Controller](#controller)
+   - [Utility Classes](#utility-classes)
+4. [Deployment](#deployment)
+5. [Design Decisions](#design-decisions) 
+
+## Introduction
+
+The PHEE Operator is designed to manage and automate the lifecycle of a Custom Resource within a Kubernetes cluster. This document outlines the key components and architectural decisions that shape the operator.
+
+## Overview
+
+The PHEE Operator comprises several key components:
+- Custom Resource Definitions (CRDs) that specify the schema for custom resources.
+- Custom Resources (CRs) that represent instances of the CRD.
+- The Operator which contains the logic for managing CRs.
+- The Controller which handles reconciliation loops to ensure the desired state of the cluster.
+- Utility classes that aid in logging and status updates.
+
+
+## Components
+
+### Custom Resource Definition (CRD)
+
+- **File**: `deploy/crds/ph-ee-importer-rdbms-crd.yaml`
+
+- **Purpose**: This file contains the Custom Resource Definition (CRD) for the operator. The CRD defines the schema for the Custom Resources that will be managed by the operator.
+
+- **Details**:
+  - Specifies fields such as `spec`, `status`, etc.
+  - Defines validation criteria for each field.
+
+### Custom Resource (CR)
+
+- **File**: `deploy/cr/ph-ee-importer-rdbms-cr.yaml`
+
+- **Purpose**: This file contains the Custom Resource (CR) script for the Importer RDBMS, which includes values for the fields defined in the CRD.
+
+- **Details**:
+  - Contains fields defined in the CRD.
+  - Specifies configuration parameters for the Importer RDBMS.
+
+### Operator
+
+- **Main File**: `src/main/java/com/example/operator/OperatorMain.java`
+
+- **Purpose**: This is the main file for the operator. It registers the operator controller and starts the operator to apply the reconciliation logic.
+
+- **Details**:
+  - Initializes the Kubernetes client.
+  - Registers custom resource schemas and controllers.
+
+### Controller
+
+- **File**: `src/main/java/com/example/operator/PhEeImporterRdbmsController.java`
+
+- **Purpose**: This file is the controller for the operator, containing the reconciliation logic for the Importer RDBMS deployment and its RBAC configurations. It uses the CR to create Kubernetes resource objects.
+
+- **Details**:
+  - Watches for changes to custom resources.
+  - Applies necessary changes to the cluster to match the desired state.
+  - Handles error states and retries.
+
+### Custom Resource Classes
+
+#### PhEeImporterRdbms.java
+
+- **File**: `src/main/java/com/example/customresource/PhEeImporterRdbms.java`
+
+- **Purpose**: Defines the Custom Resource class according to the specification used in the controller file.
+
+#### PhEeImporterRdbmsSpec.java
+
+- **File**: `src/main/java/com/example/customresource/PhEeImporterRdbmsSpec.java`
+
+- **Purpose**: Defines the specification for the operator, containing fields defined in the CRD and applied by the CRs.
+
+#### PhEeImporterRdbmsStatus.java
+
+- **File**: `src/main/java/com/example/customresource/PhEeImporterRdbmsStatus.java`
+
+- **Purpose**: Defines the status fields for the Custom Resource, allowing the operator to communicate the current state of the resource.
+
+### Utility Classes
+
+#### LoggingUtil.java
+
+- **File**: `src/main/java/com/example/utils/LoggingUtil.java`
+
+- **Purpose**: Provides utility methods for logging within the operator.
+
+#### StatusUpdateUtil.java
+
+- **File**: `src/main/java/com/example/utils/StatusUpdateUtil.java`
+
+- **Purpose**: Provides utility methods for updating the status of the Custom Resource.
+
+### pom.xml
+
+- **File**: `pom.xml`
 
-**ph-ee-importer-rdbms-crd.yaml:** This file contains the Custom Resource Definition (CRD) for the operator. The CRD defines the schema for the Custom Resources that will be managed by the operator.
+- **Purpose**: This file contains all the dependencies required for this project.
 
-**operator-deployment.yaml:** This file contains the deployment script for the operator itself, including its required Role-Based Access Control (RBAC) configurations.
+## Deployment
 
-**ph-ee-importer-rdbms-cr.yaml:** This file contains the Custom Resource (CR) script for the Importer RDBMS, which includes values for the fields defined in the CRD.
+The deployment of the PHEE Operator involves several steps:
 
-**OperatorMain.java:** This is the main file for the operator. It registers the operator controller and starts the operator to apply the reconciliation logic.
+- **CRD Deployment**: Apply the CRD to the cluster.
+- **Operator Deployment**: Deploy the operator using a Deployment resource. 
+- **CR Deployment**: Create custom resources as needed.
 
-**PhEeImporterRdbmsController.java:** This file is the controller for the operator, containing the reconciliation logic for the Importer RDBMS deployment and its RBAC configurations. It uses the CR to create Kubernetes resource objects.
+Deployment files:
 
-**PhEeImporterRdbmsSpec.java:** This file defines the specification for the operator, containing fields defined in the CRD and applied by the CRs.
+- **CRD**: `deploy/crds/ph-ee-importer-rdbms-crd.yaml`
+- **Operator Deployment**: `deploy/operator/operator.yaml` 
+- **Custom Resource**: `deploy/cr/ph-ee-importer-rdbms-cr.yaml`
 
-**PhEeImporterRdbms.java:** This file defines the Custom Resource class according to the specification used in the controller file.
+## Design Decisions
 
-**pom.xml:** This file contains all the dependencies required for this project.
+- **Language Choice**: The operator is implemented in Java due to its strong typing and extensive ecosystem.
+- **Framework**: Utilized the Java Operator SDK for streamlined development.
+- **CRD Structure**: Designed to be extensible and easy to validate.
+- **Controller Logic**: Focused on idempotency and robustness.
+
 
 
-## Note
+### Note
 This file is still in progress will be updated as the project progresses.

PHEE-operator/README.md

@@ -1,26 +1,46 @@
 # Local Setup
-
-Prerequisite
-- Kubernetes cluster setup
-- maven, jdk, kubectl and docker installed
 
-**Install the maven dependencies**
-`mvn clean install`
+This guide provides steps to set up the PHEE Operator in a k3s cluster. The current setup does not contain a Dockerfile. We use the Maven Jib plugin to build a Docker image, save that image to a .tar file, load it into the k3s cluster, and then apply the CRD, Operator, and CR.
 
-**Creating local docker image using maven jib**
-`mvn compile jib:dockerBuild -Dimage=ph-ee-importer-rdbms-operator:latest`
+## Prerequisites
 
-**Save the Docker image to a tar file**
-`docker save ph-ee-importer-rdbms-operator:latest -o ph-ee-importer-rdbms-operator.tar`
+- k3s cluster setup
+- Maven, JDK, kubectl, and Docker installed
 
-**Load the tar file into k3s**
-`sudo k3s ctr images import ph-ee-importer-rdbms-operator.tar`
+## Steps
 
-**Apply CRD, Operator and then Custom Resource**
-`kubectl apply -f deploy/crds/ph-ee-importer-rdbms-crd.yaml`
-`kubectl apply -f deploy/operator-deployment.yaml`
-`kubectl apply -f deploy/ph-ee-importer-rdbms-cr.yaml`
+### 1. Install the Maven dependencies
 
+```
+mvn clean install
+```
 
-## Note
+### 2. Creating local docker image using maven jib 
+
+```
+mvn compile jib:dockerBuild -Dimage=ph-ee-importer-rdbms-operator:latest
+```
+
+### 3. Save the Docker image to a tar file
+
+```
+docker save ph-ee-importer-rdbms-operator:latest -o ph-ee-importer-rdbms-operator.tar
+```
+
+### 4. Load the tar file into k3s
+
+```
+sudo k3s ctr images import ph-ee-importer-rdbms-operator.tar
+```
+
+### 5. Apply CRD, Operator and Custom Resource
+
+```
+kubectl apply -f deploy/crds/ph-ee-importer-rdbms-crd.yaml
+kubectl apply -f deploy/operator/operator.yaml
+kubectl apply -f deploy/cr/ph-ee-importer-rdbms-cr.yaml
+```
+
+### Note
 This file is still in progress will be updated as the project progresses.
+Also, currently operator is configured for deploying importer-rdbms only

PHEE-operator/deploy/ph-ee-importer-rdbms-cr.yaml → PHEE-operator/deploy/cr/ph-ee-importer-rdbms-cr.yaml

@@ -1,4 +1,4 @@
-apiVersion: example.com/v1
+apiVersion: my.custom.group/v1
 kind: PhEeImporterRdbms
 metadata:
   name: sample-ph-ee-importer-rdbms
@@ -19,9 +19,9 @@ spec:
   resources:
     limits:
       cpu: 500m
-      memory: 512M
+      memory: 512Mi
     requests:
       cpu: 100m
-      memory: 256M
+      memory: 256Mi
   javaToolOptions: "-Xmx256M"
   bucketName: paymenthub-ee-dev

PHEE-operator/deploy/crds/ph-ee-importer-rdbms-crd.yaml

@@ -1,13 +1,23 @@
 apiVersion: apiextensions.k8s.io/v1
 kind: CustomResourceDefinition
 metadata:
-  name: ph-ee-importer-rdbmses.example.com
+  name: pheeimporterrdbmses.my.custom.group # Use the current naming convention
 spec:
-  group: example.com
+  group: my.custom.group
+  names:
+    kind: PhEeImporterRdbms
+    listKind: PhEeImporterRdbmsList
+    plural: pheeimporterrdbmses
+    singular: pheeimporterrdbms
+    shortNames:  # Add short names if required
+      - peeirdbms
+  scope: Namespaced
   versions:
-    - name: v1
+    - name: v1 
       served: true
       storage: true
+      subresources:
+        status: {}  # Ensure status is defined as a subresource
       schema:
         openAPIV3Schema:
           type: object
@@ -62,11 +72,14 @@ spec:
                   type: string
                 bucketName:
                   type: string
-  scope: Namespaced
-  names:
-    plural: ph-ee-importer-rdbmses
-    singular: ph-ee-importer-rdbms
-    kind: PhEeImporterRdbms
-    shortNames:
-      - peeirdbms
-
+            status:
+              type: object
+              properties:
+                availableReplicas:
+                  type: integer
+                errorMessage:
+                  type: string
+                lastAppliedImage:
+                  type: string
+                ready:
+                  type: boolean