Deploying CloudNativePG on Civo Kubernetes

PostgreSQL is a powerful, open-source relational database that has been a cornerstone for modern applications across industries. With the rise of cloud-native architectures, organizations are increasingly seeking ways to run databases like PostgreSQL on Kubernetes, leveraging the scalability, portability, and automation provided by container orchestration platforms. However, while Kubernetes is well-suited for stateless applications, deploying and managing stateful services like databases comes with unique challenges.

This tutorial will guide you through deploying and managing a PostgreSQL cluster on the Civo Kubernetes service using the CloudNativePG operator. By adopting this approach, you can take advantage of Kubernetes’ ecosystem to simplify database lifecycle management, achieve high availability, and ensure data persistence in a dynamic, scalable environment.

Introduction to PostgreSQL on Kubernetes

Running PostgreSQL on Kubernetes is an increasingly popular choice for organizations aiming to combine the scalability and flexibility of container orchestration with robust database management. Kubernetes excels at handling stateless applications, but managing stateful services like databases introduces unique complexities that require thoughtful solutions.

Deploying PostgreSQL on Kubernetes involves addressing critical aspects such as persistent storage, high availability, and data integrity. Key Kubernetes resources like StatefulSets are essential for managing the lifecycle of PostgreSQL pods, ensuring each pod retains a stable, unique identity and access to persistent storage even after restarts.

Persistent Volumes (PVs) ensure data durability, while ConfigMaps and Secrets simplify managing database configurations and securely storing sensitive information like credentials. These tools collectively enhance the reliability, security, and manageability of PostgreSQL in a Kubernetes environment, making it a compelling solution for modern cloud-native applications.

Why Use Operators for PostgreSQL?

While Kubernetes provides foundational building blocks for deploying stateful applications, managing databases like PostgreSQL involves much more than simply running pods and attaching storage. Ensuring high availability, automating backups, handling failovers, and maintaining data integrity during scaling or infrastructure changes requires specialized expertise and tools.

This is where database operators, like CloudNativePG, come into play. Operators extend Kubernetes functionality to automate complex operational tasks that are otherwise manual and error-prone. By translating operational knowledge into software, operators reduce the burden on DevOps teams and allow them to focus on higher-value tasks.

Deployment of CloudNativePG

Prerequisites

While this tutorial assumes basic familiarity with Kubernetes concepts such as Pods, Services, and ConfigMaps, you will need the following to complete this tutorial:

• A Civo account
• Kubectl installed (to manage namespaces and secrets on the cluster)
• Create a kubernetes cluster

Installing the CloudNativePG Operator

CloudNativePG can be installed using Helm or by directly applying the YAML manifests.

kubectl apply --server-side -f https://raw.githubusercontent.com/cloudnative-pg/cloudnative-pg/release-1.24/releases/cnpg-1.24.1.yaml

Check if the operator is running correctly:

kubectl get pods -n cnpg-system

NAME                                       READY   STATUS    RESTARTS   AGE
cnpg-controller-manager-7fd67898bb-2lc5x   1/1     Running   1          53s

Deploying a PostgreSQL Cluster

A cluster in CloudNativePG is a set of PostgreSQL instances managed in Kubernetes, providing high availability and automatic replication. To install it, you apply the operator's manifests and create a PostgreSQLCluster resource using a YAML file, which configures and deploys the cluster automatically.

Create a file named pgcluster.yaml and add the following content:

apiVersion: postgresql.cnpg.io/v1
kind: Cluster
metadata:
  name: civo-postgres-cluster
spec:
  instances: 3  # number of replicas
  storage:
    size: 1Gi

Apply this manifest to deploy the cluster:

kubectl apply -f pgcluster.yaml 

cluster.postgresql.cnpg.io/civo-postgres-cluster created

After installing the PostgreSQL cluster using CloudNativePG, you can see the instances distributed as pods within your Kubernetes cluster. Each PostgreSQL instance runs in its own pod, and you can check them using the kubectl get pods command to view the status and distribution of the database instances across your cluster.

kubectl get pods

NAME                                 READY   STATUS      RESTARTS   AGE
install-cert-manager-cloudst-qg82f   0/1     Completed   0          20h
civo-postgres-cluster-1              1/1     Running     0          3m28s
civo-postgres-cluster-2              1/1     Running     0          114s
civo-postgres-cluster-3              1/1     Running     0          48s

If you want to configure the cluster during boot initialization, you can do so using the BootstrapConfiguration. This allows you to set custom configurations at the time of cluster creation, such as defining users and databases and applying specific initialization scripts.

For each instance created by the CloudNativePG, a Persistent Volume Claim (PVC) is generated in Kubernetes. In our demo, the volumes are securely stored in Civo Cloud. To view the created volumes, simply log into your Civo account, click on the "Volumes" option, and you'll see them listed there.

Connecting to the PostgreSQL Cluster

Now, it's time to connect to the database, which we can do through one of the services created by the cluster operator. Since I didn't specify a username and password for my cluster, the operator generated these credentials automatically and stored them in a Kubernetes Secret.

Let's take a look at the Secrets to retrieve the connection details:

kubectl get secrets

NAME                              TYPE                     DATA   AGE
civo-postgres-cluster-ca          Opaque                   2      16m
civo-postgres-cluster-server      kubernetes.io/tls        2      16m
civo-postgres-cluster-replication kubernetes.io/tls        2      16m
civo-postgres-cluster-app         kubernetes.io/basic-auth 9      16m

To retrieve the username and password that the operator created for you, run the following commands:

#Get the password
kubectl get secret civo-postgres-cluster-app \ -o=jsonpath='{.data.password}' | base64 -d

#Get the username
kubectl get secret civo-postgres-cluster-app \ -o=jsonpath='{.data.username}' | base64 -d

Expose the PostgreSQL service so that you can connect from your local machine:

kubectl port-forward svc/civo-postgres-cluster-rw 5432:5432

Connect via psql or pgadmin using the username and password obtained above. In my case, the username was “app” and the password was “ANe9uuSy1LFPnR0LqOFYziiHL4nJO6QnKI2c0l5hzDwGr3bupzuGrIxDNq2dAUob”. To log in using these credentials, run the following command:

psql -h localhost -p 5432 -U app

You will be asked to enter the password, and after that, you will be ready to use your database:

psql (14.13 (Ubuntu 14.13-0ubuntu0.22.04.1), server 17.0 (Debian 17.0-1.pgdg110+1))
WARNING: psql major version 14, server major version 17.
         Some psql features might not work.
SSL connection (protocol: TLSv1.3, cipher: TLS_AES_256_GCM_SHA384, bits: 256, compression: off)
Type "help" for help.

app=> 

It's the perfect time to dive into CloudNativePG! If you're eager to explore advanced features such as automated backups, scaling, and monitoring, check out some of these resources:

• PostgreSQL Docs on Civo
• Beginners guide to PersistentVolume and PersistentVolumeClaim
• Creating a Kubernetes Operator with Kubebuilder
• Tutorials on Backups