Skip to content

Commit

Permalink
Add document for Storage API resources
Browse files Browse the repository at this point in the history
  • Loading branch information
Rohit-0505 committed Jan 6, 2025
1 parent 80abe1e commit afd0d6c
Show file tree
Hide file tree
Showing 8 changed files with 601 additions and 106 deletions.
81 changes: 80 additions & 1 deletion docs/usage/installation.md
Original file line number Diff line number Diff line change
@@ -1 +1,80 @@
# Installing IronCore
# Installing IronCore

## Requirements

* `go` >= 1.20
* `git`, `make` and `kubectl`
* [Kustomize](https://kustomize.io/)
* Access to a Kubernetes cluster ([Minikube](https://minikube.sigs.k8s.io/docs/), [kind](https://kind.sigs.k8s.io/) or a real cluster)

## Clone the Repository

To bring up and install the `ironcore` project, you first need to clone the repository.

```shell
git clone [email protected]:ironcore-dev/ironcore.git
cd ironcore
```

## Install cert-manager

If there is no [cert-manager](https://cert-manager.io/docs/) present in the cluster it needs to be installed.

```shell
kubectl apply -f https://github.com/cert-manager/cert-manager/releases/download/v1.16.2/cert-manager.yaml
```

## Install APIs into the Cluster

Your Kubernetes API server needs to know about the APIs which come with the `ironcore` project. To install the APIs into your cluster, run

```shell
make install
```

**Note**: This requires the `APISERVER_IMG` (Makefile default set to `apiserver`) to be pullable from your kubernetes
cluster. For local development with `kind`, a make target that builds and loads the api server image and then applies
the manifests is available via

```shell
make kind-install
```

**Note**: In case that there are multiple environments running, ensure that `kind get clusters` is pointing to the
default kind cluster.

## Deploy the Controller Manager

The controller manager can be started via the following command

```shell
make kind-deploy
```
## validate

Make sure you have all the below pods running

```shell
$ kubectl get po -n ironcore-system
NAME READY STATUS RESTARTS AGE
ironcore-apiserver-85995846f9-47247 1/1 Running 0 136m
ironcore-controller-manager-84bf4cc6d5-l224c 2/2 Running 0 136m
ironcore-etcd-0 1/1 Running 0 143m
```

## Apply Sample Manifests

The `config/samples` folder contains samples for all APIs supported by this project. You can apply any of the samples by
running

```shell
kubectl apply -f config/samples/SOME_RESOURCE.yaml
```

## Cleanup

To remove the APIs from your cluster, simply run

```shell
make uninstall
```
47 changes: 46 additions & 1 deletion docs/usage/storage/bucket.md
Original file line number Diff line number Diff line change
@@ -1 +1,46 @@
# Bucket
# Bucket
A `bucket` in `ironcore` refers to a storage resource that organizes and manages data, similar to the concept of buckets in cloud storage services like Amazon S3. Buckets serve as containers for storing objects, such as files or data blobs, and is crucial for managing storage workloads.

# Example Bucket Resource
An example of how to define a `bucket` resource in `ironcore`
```
apiVersion: storage.ironcore.dev/v1alpha1
kind: Bucket
metadata:
name: bucket-sample
spec:
bucketClassRef:
name: bucketclass-sample
# bucketPoolRef:
# name: bucketpool-sample
```

# Key Fields:
- `bucketClassRef`(`string`):
- Mandatory field
- `BucketClassRef` is the BucketClass of a bucket

- `bucketPoolRef`(`string`):
- Optional field
- `bucketPoolRef` indicates which BucketPool to use for the bucket, if not specified the controller itself picks the available bucketPool


# Usage
- **Data Storage**: Use buckets to store and organize data blobs, files, or any object-based data.

- **Multi-Tenant Workloads**: Leverage buckets for isolated and secure data storage in multi-tenant environments by using separate BucketClass or BucketPool references.

- **Secure Access**: Buckets store access credentials securely in their status. Applications can retrieve access details via Kubernetes secrets.

# Reconciliation Process:
- The controller detects changes and fetches bucket details.

- Creation/Update ensures the backend bucket exists, metadata is synced, and credentials are updated.

- The bucket will automatically sync with the backend storage system, Updates Bucket's state (e.g., `Available`, `Pending`, or `Error`) in the bucket's status.

- Access details and credentials will be managed securely and updated in the bucket status.

- During deletion, resources will be cleaned up gracefully without manual intervention.

- If the bucket is not ready (e.g., backend issues), reconciliation will retry
39 changes: 38 additions & 1 deletion docs/usage/storage/bucketclass.md
Original file line number Diff line number Diff line change
@@ -1 +1,38 @@
# BucketClass
# BucketClass
A BucketClass is a concept used to define and manage different types of storage buckets, typically based on resource capabilities

# Example BucketClass Resource
An example of how to define a bucketClass resource

```
apiVersion: storage.ironcore.dev/v1alpha1
kind: BucketClass
metadata:
name: bucketclass-sample
capabilities:
tps: 100Mi
iops: 100
```

# Key Fields:
- capabilities: Capabilities has `tps` and `iops` fields which need to specified, it's a mandatory field,
- tps(`string`): The `tps` represents trasactions per second

- iops(`string`): `iops` is the number of input/output operations a storage device can complete per second.

# Usage
- **BucketClass Definition**: Create a BucketClass to set storage properties based on resource capibilities
- **Associate with buckets**: Link a BucketClass to a Bucket using a reference in the Bucket resource.
- **Dynamic configuration**: Update the BucketClass to modify storage properties for all its Buckets.

# Reconciliation Process:

- **Fetches & Validates**: Retrieves the BucketClass from the cluster and checks if it exists.

- **Synchronizes State**: Keeps the BucketClass resource updated with its current state and dependencies.

- **Monitors Dependencies**: Watches for changes in dependent Bucket resources and reacts accordingly.

- **Handles Errors**: Requeues the reconciliation to handle errors and ensure successful completion.

- **Handles Deletion**: Cleans up references, removes the finalizer, and ensures no dependent Buckets exist before deletion.
34 changes: 33 additions & 1 deletion docs/usage/storage/bucketpool.md
Original file line number Diff line number Diff line change
@@ -1 +1,33 @@
# BucketPool
# BucketPool
A `BucketPool` is a resource in `IronCore` that represents a pool of storage buckets managed collectively. It defines the infrastructure's storage configuration used to provision and manage buckets, ensuring resource availability and compatibility with associated `BucketClasses`.

# Example BucketPool Resource

```
apiVersion: storage.ironcore.dev/v1alpha1
kind: BucketPool
metadata:
name: bucketpool-sample
spec:
providerID: ironcore://shared
#status:
# state: Available
# availableBucketClasses:
# ironcore.dev/fast-class: 10Gi
# ironcore.dev/slow-class: 100Gi
```

# Key Fields:
- `ProviderID`(`string`): The `providerId` helps the controller identify and communicate with the correct storage system within the specific backened storage porvider.

for example `ironcore://shared`

# Reconciliation Process:

- **Bucket Type Discovery**: It constantly checks what kinds of buckets (BucketClasses) are available in the `Ironcore` Infrastructure.

- **Compatibility Check**: Evaluating whether the BucketPool can create and manage each bucket type based on its capabilities.

- **Status Update**: Updating the BucketPool's status to indicate the bucket types it supports, like a menu of available options.

- **Event Handling**: Watches for changes in BucketClass resources and ensures the associated BucketPool is reconciled when relevant changes occur.
45 changes: 44 additions & 1 deletion docs/usage/storage/volume.md
Original file line number Diff line number Diff line change
@@ -1 +1,44 @@
# Volume
# Volume
The `IronCore` `Volume` is a storage abstraction provided by the `IronCore Runtime Interface` `(IRI)` service, designed to integrate with external storage backend for managing persistent storage. It acts as a managed storage unit, ensuring consistency, scalability, and compatibility with Kubernetes workloads.
By integrating IronCore Volumes with Kubernetes, users benefit from seamless storage management, automation, and advanced features such as encryption and scalability, making it suitable for modern cloud-native and hybrid applications.

# Example Volume Resource
An example of how to define a `volume` resource in `ironcore`

```
apiVersion: storage.ironcore.dev/v1alpha1
kind: Volume
metadata:
name: volume-sample
spec:
volumeClassRef:
name: volumeclass-sample
# volumePoolRef:
# name: volumepool-sample
resources:
storage: 100Gi
```

# Key Fields:

- `volumeClassRef`(`string`): `volumeClassRef` refers to the name of an ironcore `volumeclass`( for eg: `slow`, `fast`, `super-fast` etc.) to create a volume,

- `volumePoolRef` (`string`): `VolumePoolRef` indicates which VolumePool to use for a volume. If unset, the scheduler will figure out a suitable `VolumePoolRef`.

- `resources`: Resources is a description of the volume's resources and capacity.

# Reconciliation Process:

- **Fetch Volume Resource**: Retrieve the `Volume` resource and clean up any orphaned `IRI` volumes if the resource is missing.

- **Add Finalizer**: Ensure a finalizer is added to manage cleanup during deletion.

- **Check IRI Volumes**: List and identify `IRI` volumes linked to the `Volume` resource.

- **Create or Update Volume**:
- Create a new IRI volume if none exist.
- Update existing IRI volumes if attributes like size or encryption need adjustments.

- **Sync Status**: Reflect the IRI volume's state (e.g., Pending, Available) in the Kubernetes Volume resource's status.

- **Handle Deletion**: Safely delete all associated IRI volumes and remove the finalizer to complete the resource lifecycle.
39 changes: 38 additions & 1 deletion docs/usage/storage/volumeclass.md
Original file line number Diff line number Diff line change
@@ -1 +1,38 @@
# VolumeClass
# VolumeClass
The `VolumeClass` in `IronCore` is a Kubernetes-like abstraction that defines a set of parameters or configurations for provisioning storage resources through the `IronCore Runtime Interface (IRI)`. It is conceptually similar to Kubernetes `StorageClass`, enabling users to specify the desired properties for an IronCore `Volume` resource creation.

# Example VolumeClass Resource
An example of how to define a volumeClass resource

```
apiVersion: storage.ironcore.dev/v1alpha1
kind: VolumeClass
metadata:
name: volumeclass-sample
capabilities:
tps: 100Mi
iops: 100
```

# Key Fields:
- capabilities: Capabilities has tps and iops fields which need to specified, it's a mandatory field,
- tps(`string`): The `tps` represents trasactions per second

- iops(`string`): `iops` is the number of input/output operations a storage device can complete per second.

# Usage
- **VolumeClass Definition**: Create a volumeClass to set storage properties based on resource capibilities
- **Associate with Volume**: Link a volumeClass to a Volume using a reference in the Volume resource.
- **Dynamic configuration**: Update the volumeClass to modify storage properties for all its Volumes.

# Reconciliation Process:

- **Fetches & Validates**: Retrieves the VolumeClass from the cluster and checks if it exists.

- **Synchronizes State**: Keeps the VolumeClass resource updated with its current state and dependencies.

- **Monitors Dependencies**: Watches for changes in dependent Volume resources and reacts accordingly.

- **Handles Errors**: Requeues the reconciliation to handle errors and ensure successful completion.

- **Handles Deletion**: Cleans up references, removes the finalizer, and ensures no dependent Volumes exist before deletion.
32 changes: 32 additions & 0 deletions docs/usage/storage/volumepool.md
Original file line number Diff line number Diff line change
@@ -1 +1,33 @@
# VolumePool
A `VolumePool` is a resource in `IronCore` that represents a pool of storage volume managed collectively. It defines the infrastructure's storage configuration used to provision and manage volumes, ensuring resource availability and compatibility with associated `VolumeClasses`.

# Example Volume Resource

```
apiVersion: storage.ironcore.dev/v1alpha1
kind: VolumePool
metadata:
name: volumepool-sample
spec:
providerID: ironcore://shared
#status:
# state: Available
# availableVolumeClasses:
# ironcore.dev/fast-class: 10Gi
# ironcore.dev/slow-class: 100Gi
```

# Key Fields:
- `ProviderID`(`string`): The `providerId` helps the controller identify and communicate with the correct storage system within the specific backened storage porvider.

for example `ironcore://shared`

# Reconciliation Process:

- **Volume Type Discovery**: It constantly checks what kinds of volumes (volumeClasses) are available in the `Ironcore` Infrastructure.

- **Compatibility Check**: Evaluating whether the volumePool can create and manage each volume type based on its capabilities.

- **Status Update**: Updating the VolumePool's status to indicate the volume types it supports, like a menu of available options.

- **Event Handling**: Watches for changes in VolumeClass resources and ensures the associated VolumePool is reconciled when relevant changes occur.
Loading

0 comments on commit afd0d6c

Please sign in to comment.