[Sandbox] CloudNativePG

[gbartolini]

Application contact emails

[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected]

Project Summary

CloudNativePG is a Kubernetes native database platform for PostgreSQL

Project Description

CloudNativePG is a comprehensive platform designed to streamline the management of PostgreSQL databases within cloud-native environments based on Kubernetes. The main component is the CloudNativePG operator. CloudNativePG automates the entire lifecycle of PostgreSQL clusters—from deployment to maintenance—while ensuring high availability, disaster recovery, and self-healing capabilities. Built with a "security-by-default" mindset, it integrates advanced security measures, along with robust observability features like Prometheus metrics and Grafana dashboards for monitoring.

By adhering to Infrastructure as Code (IaC) principles, CloudNativePG enables users to define and manage PostgreSQL clusters through declarative configuration, simplifying complex tasks like automated failover, scaling, and backups. The CloudNativePG operator leverages Kubernetes-native resources, such as volume snapshots, to ensure consistent and reliable database operations.

CloudNativePG also extends its functionality with additional components, including kubectl plugins, Helm charts, and OLM bundles, ensuring seamless integration into existing cloud-native workflows. Furthermore, its extensibility framework (CNPG-I) supports third-party plugins, enabling users to adapt the platform to specific needs. CloudNativePG meets the growing demand for scalable, secure, and highly available database solutions in modern cloud environments, serving the basis for AI, ML and Analytics applications.

Org repo URL (provide if all repos under the org are in scope of the application)

https://github.com/cloudnative-pg

Project repo URL in scope of application

https://github.com/cloudnative-pg/cloudnative-pg

Additional repos in scope of the application

• https://github.com/cloudnative-pg/governance
• https://github.com/cloudnative-pg/postgres-containers
• https://github.com/cloudnative-pg/charts

Website URL

https://cloudnative-pg.io/

Roadmap

https://github.com/orgs/cloudnative-pg/projects/1

Roadmap context

We currently maintain a concise list of features for implementation, preferring a just-in-time approach to selecting tasks for upcoming releases. This method allows us to adapt dynamically, choosing features based on community feedback and conversations with customers of our primary sponsoring organization.

Contributing Guide

https://github.com/cloudnative-pg/cloudnative-pg/blob/main/CONTRIBUTING.md

Code of Conduct (CoC)

https://github.com/cloudnative-pg/cloudnative-pg/blob/main/CODE_OF_CONDUCT.md

Adopters

https://github.com/cloudnative-pg/cloudnative-pg/blob/main/ADOPTERS.md

Contributing or Sponsoring Org

https://www.enterprisedb.com/

Maintainers file

https://github.com/cloudnative-pg/governance/blob/main/MAINTAINERS.md

IP Policy

• If the project is accepted, I agree the project will follow the CNCF IP Policy

Trademark and accounts

• If the project is accepted, I agree to donate all project trademarks and accounts to the CNCF

Why CNCF?

We believe that contributing our project to the CNCF will allow it to benefit from the CNCF’s extensive community support and expertise, especially in the realm of cloud-native technologies. The CNCF community offers diverse perspectives and a strong ecosystem of contributors that will help push Postgres' capabilities in cloud-native environments. This collaboration will enable Postgres to evolve further and excel in scenarios where Kubernetes and cloud-native deployments are integral, broadening its use cases in modern architectures.

The CNCF provides a powerful framework of open governance, and our project’s core values align with them. By integrating with CNCF, the CloudNativePG community will gain guidance and support in expanding our contributor base and ensuring long-term sustainability. CNCF’s model ensures that projects remain fully open-source, keeping the freedom and accessibility principles intact for users worldwide. Being part of CNCF guarantees that our project will be free and available to anyone, built on a stack composed of Kubernetes, Postgres, and CloudNativePG, fostering worldwide innovation.

We chose the CNCF because it provides not only an infrastructure for open governance but also a strong ecosystem of interconnected projects and working groups (SIGs and TAGs) within Kubernetes. The ability to collaborate with other CNCF projects, such as Prometheus, Fluent Bit/Fluentd, Helm, Open Policy Agent, and cert-manager, will help improve the overall user experience of running Postgres in cloud-native environments. Furthermore, given the criticality of storage in database systems, we are keen to improve Kubernetes storage and cooperate with CNCF storage-related projects such as Longhorn, Rook, and OpenEBS (currently archived). This synergy will strengthen both our project and the broader CNCF ecosystem by driving innovation in database management in Kubernetes-based deployments.

Benefit to the Landscape

CloudNativePG will bring a native, Kubernetes-friendly approach to managing PostgreSQL databases in cloud-native environments, which fills a critical gap in the CNCF ecosystem. PostgreSQL is one of the most widely used open-source relational databases, and CloudNativePG offers seamless integration with Kubernetes to manage its lifecycle, including high availability, disaster recovery, and backup/restore functionalities. By adding CloudNativePG to the CNCF landscape, the project addresses a growing need for database solutions that natively support cloud-native principles like declarative configuration, scalability, and automation.

CloudNativePG’s differentiating factor is its Kubernetes-native architecture, designed from the ground up for cloud-native PostgreSQL workloads. Unlike other database management tools, CloudNativePG is fully integrated with Kubernetes' operator pattern, leveraging its APIs and controllers for managing Postgres clusters in a reliable, automated, and self-healing manner. This approach significantly reduces operational complexity for running Postgres in Kubernetes, aligning with CNCF’s mission to simplify and promote cloud-native applications.

In terms of enhancements, CloudNativePG strengthens the CNCF landscape by integrating with existing CNCF projects like Prometheus for monitoring, cert-manager for managing certificates, and Kubernetes-native storage solutions such as Rook, Longhorn and OpenEBS (currently archived). This enhances the overall cloud-native ecosystem by offering a cohesive, unified solution for running PostgreSQL in Kubernetes, particularly in enterprise and large-scale production environments.

Cloud Native 'Fit'

CloudNativePG extends the Kubernetes API using custom resource definitions (CRDs) in an operator pattern. The Cluster CRD is key, managing highly available PostgreSQL clusters with scalability, self-healing, and automated failover features. It integrates seamlessly with standard Kubernetes resources (pods, services, persistent volume claims, etc.), promoting Kubernetes-native database management.

The project emphasizes declarative configuration, enabling users to manage PostgreSQL clusters as code through YAML manifests, Helm charts, or OLM bundles. This approach streamlines deployment and ensures consistent lifecycle management across environments, embodying the Infrastructure as Code (IaC) principle.

CloudNativePG also prioritizes observability, offering a customizable Prometheus exporter and exporting logs in JSON format. This allows integration with CNCF monitoring, alerting, and logging tools, providing real-time insights crucial for maintaining high availability in cloud-native environments.

Security is integral, following a "security by default" approach. It enforces best practices, including secure container policies and restricted PostgreSQL access, ensuring compliance in multi-cloud or hybrid setups.

Overall, CloudNativePG fits into the cloud-native landscape by providing a Kubernetes-native solution for PostgreSQL, emphasizing automation, observability, security, and declarative infrastructure.

Cloud Native 'Integration'

CloudNativePG depends:

1. Kubernetes
2. gRPC
3. Helm (as an optional deployment method)
4. Operator Framework (as an optional deployment method via OLM)

CloudNativePG possibly integrates with:

1. Rook
2. Longhorn
3. OpenEBS (currently archived)
4. Carina
5. CubeFS
6. ArgoCD
7. Flux
8. Prometheus
9. Cortex
10. Thanos
11. FluentBit/Fluentd
12. OpenTelemetry
13. cert-manager
14. Keycloak

Cloud Native Overlap

Currently, there is no overlap, as no CNCF project specifically manages PostgreSQL databases.

Similar projects

Vitess

Landscape

https://landscape.cncf.io/?item=app-definition-and-development--database--cloudnativepg

Business Product or Service to Project separation

This has already been addressed. EDB has donated the intellectual property (IP) to "The CloudNativePG Authors," and the project is licensed under Apache 2.0. Additionally, to avoid any potential trademark issues with the CNCF, the name "PostgreSQL" (a trademark of the PostgreSQL Community Association of Canada) has been removed from the project’s original name, which was initially "Cloud Native PostgreSQL."

Project presentations

• CNCF TAG Storage Meeting presentation scheduled on September 25, 2024
• March 21 2024, KubeCon Europe 2024 in Paris: "Scaling Heights: Mastering Postgres Database Vertical Scalability with Kubernetes Storage Magic" (Gari Singh, Google & Gabriele Bartolini, EDB)
• March 19 2024, Data on Kubernetes Day at KubeCon Europe 2024 in Paris: "From Zero to Hero: Scaling Postgres in Kubernetes Using the Power of CloudNativePG" (Gabriele Bartolini, EDB)
• 7 November 2023, KubeCon North America 2023 in Chicago: "Disaster Recovery with Very Large Postgres Databases (in Kubernetes)" (Michelle Au, Google & Gabriele Bartolini, EDB)
• 27 October 2022, KubeCon North America 2022 in Detroit: "Data On Kubernetes, Deploying And Running PostgreSQL And Patterns For Databases In a Kubernetes Cluster" (Chris Milsted, Ondat & Gabriele Bartolini, EDB)

Project champions

CNCF TAG Storage

Additional information

Running PostgreSQL in Kubernetes with CloudNativePG also contributes to the PostgreSQL project's growth and development. CloudNativePG enables seamless testing and integration of PostgreSQL's latest source code within Kubernetes, providing PostgreSQL developers with an efficient environment to test patches and new features. The project has already helped introduce a few patches and bug fixes to PostgreSQL, and the community actively supports DBAs looking to transition PostgreSQL deployments to Kubernetes.

[ianstanton]

Chiming in from https://tembo.io. We use CloudNativePG under the hood in our platform, and it's been excellent to work with. We're a small team, and we've been able to get a lot of value out of it. Working with @gbartolini and the team has been smooth, and they're always quick to respond to any questions we have.

We feel the admission of CNPG into the CNCF Sandbox will help fuel the growth of the project and the surrounding community. We're excited to see where it goes from here!

[pgodowski]

Chiming in from IBM. We use CloudNativePG as the embedded Postgres datastore for the wide portfolio of the IBM products running on Kubernetes/OpenShift platform across hybrid environments (customer-managed and running on Cloud). Working with the community, we appreciate CloudNativePG for its robustness and resiliency, delivering the value to the customers running Postgres database in thousands on Kubernetes clusters. The community is energetic, always helpful and always looking for the best way how to do things in the Kubernetes native way: @gbartolini @sxd @leonardoce and other fantastic community builders are great experts leading the efforts of making Postgres the first class citizen in the Kubernetes ecosystem.

Admission of CNPG into CNCF Sandbox would elevate this community to the next level. Happy to be part of this journey.

[xing-yang]

CloudNativePG team gave a presentation to TAG Storage on 9/25/2024. Here's the recording: https://www.youtube.com/watch?v=Nsu5Em8Ig88. cc @chira001 @raffaelespazzoli

[zekiyeaydemir]

Managing a Kubernetes cluster is difficult, but managing databases in Kubernetes can be even more challenging. CloudNativePG makes this easier with features like direct management of PVCs instead of StatefulSets, online resizing, and volume snapshot backup and recovery. It also supports PostgreSQL management across different regions. As a DBA, I am pleased to use CloudNativePG thanks to these features.
As a PostgreSQL DBA and Kubestronaut, I have helped many customers manage PostgreSQL in Kubernetes clusters, and we use CloudNativePG with great satisfaction. The CloudNativePG community consists of energetic, supportive, and kind people. Thanks to their encouragement, I became a contributor, and this is my first open-source project.
I believe that CloudNativePG’s acceptance into the CNCF Sandbox will help move the community forward. I am very happy to be part of this community and for their support in helping me contribute to open-source projects.

[smiyc]

I discovered CloudNativePG in late 2021, which was called CNP back then, while searching for a database solution that would satisfy our devops team needs.
At this time, I had never used kubectl or git before, so the journey was way more just comparing operators.
In 2022 the demand for an always on database solution to host our keycloak data increased.
There where a couple of PostgreSQL Kubernetes operators, so I decided start a project comparing them.
I had a look at CloudNativePG, the Zalando and Crunchy Operators.
In the end I decided to use CloudNativePG because they had a very good documentation, a roadmap and most of all a very welcoming and supportive community.

In 2023 I met the team at Kubecon EU and asked how to support them? They told me, even if you are not a programmer yourself, use the operator, raise issues and don't be afraid to submit a PR if you find something e.g. in the documentation.
With there support, I learned how to fork a repo and submit PR's.
And now I'm more then proud to be an adopter and the # 24 contributor 🙂

I believe that accepting the CNCF Sandbox application will boost CloudNativePG adoption and CloudNativePG will help even more people like me to start an amazing journey.

[rdegez]

At Enix.io, we are absolutely in love with CloudnativePG!
It's a very practical and efficient way to run a large number of - sometimes very large - PostgreSQL database clusters in a Kubernetes cluster.
Its "day 2 operation" capabilities (backup management, updates, replicas scalability, point-in-time recovery, pgbouncer pooling, metrics...) make it a perfectly "production ready" solution.
The documentation is very comprehensive and of high quality, and the entire project team is very accessible and communicates well about the project's developments.
We can't imagine going back and firmly believe that the project is a perfect candidate for inclusion in the CNCF landscape.

[thomasboussekey]

Thanks to cloudnative-pg, we have modernized our database hosting stack at Mirakl

Kubernetes has been used for 7+ years for stateless applications.
The POD creation is automated with many tools like Helm & ArgoCD.

We were looking for a new hosting solution for PostgreSQL databases.
After comparing Kubernetes operators, IaaS home-made, PostgreSQL managed instance implementation, we have quickly focused only on cloudnative-pg that complied to our pre-study expectations and proof-of-concept early tests:

• easy to deploy
• useful documentation
• 100% declarative, all the parameters are defined in the HELM manifest

The project went fast

• On November 2023, we have decided to continue our investigations on cloudnative-pg only, and we have started to define the manifest in order to implement the creation of our new instances.
• On February 2024, we have created the first production with cloudnative-pg.
• On June 2024, we have migrated the first existing database to cloudnative-pg, using the embedded declarative logical replication.
• We are now operating 250+ cloudnative-pg PostgreSQL clusters, containing 4+ TB of data.

On the last months, several releases of cloudnative-pg have been released and we have updated smoothlessly our manifest to stay up-to-date. We have also upgraded the kubernetes engine version. For each operation, a single instance switchover is observed.

The configuration of the cloudnative-pg is highly configurable:

• we are using a custom-made docker image based on postgis-container, adding extra required extensions for our needs.
• Backup is also configured thanks to the manifest, and DRP is also defined in our manifest.
• Thanks to cloudnative-pg, we have also easily implemented a connection pooler with YAML declarative statements.

This project has very interesting features and it could be nice to add this operator in the CNCF ecosystem.

[thomasboussekey]

Direct link to sandbox board