1 of 18

From CONF to YAML: Redefining Database

Management

Using CloudNativePG

Mohammad Zulfikar

DevOps Engineer

Muhammad Fikri A

DevOps Engineer

1

2 of 18

The constraint

2

KUBERNETES COMMUNITY DAYS INDONESIA 2026

3 of 18

Operator Pattern

3

https://cloudnative-pg.io/docs/1.30/operator_capability_levels

KUBERNETES COMMUNITY DAYS INDONESIA 2026

4 of 18

Before: the CONF era

Hand-edited: postgresql.conf and pg_hba.conf on every VM

Cron: backups and failover lived in shell scripts

Drift: every server a little different from the others

4

$ ssh db-vm-02

$ sudo vim /etc/postgresql/18/main/postgresql.conf

$ sudo vim /etc/postgresql/18/main/pg_hba.conf

$ sudo systemctl restart postgresql

$ crontab -l

0 1 * * * /opt/scripts/pg_backup.sh

* * * * * /opt/scripts/failover_check.sh

Same parameter, three servers:

db-vm-01

max_connections

200

db-vm-02

max_connections

400

db-vm-03

max_connections

1000

KUBERNETES COMMUNITY DAYS INDONESIA 2026

5 of 18

CloudNativePG changes Everything

5

KUBERNETES COMMUNITY DAYS INDONESIA 2026

6 of 18

7 of 18

How we “Ship” the Database

7

Argo CD syncs every database from Git
One Argo CD application per database
The DBA team changes a database by pull request

KUBERNETES COMMUNITY DAYS INDONESIA 2026

8 of 18

High availability

CNPG can easily scale to multiple pods using instances on spec

primaryUpdateMethod: switchover

kubectl cnpg promote picks the new primary

Apps use the -rw service, which follows the primary: the connection string never changes

Exposing database via LB possible by creating, spec.managed.services.additional on Cluster CRD

8

KUBERNETES COMMUNITY DAYS INDONESIA 2026

9 of 18

Backup and PITR

CNPG Uses several CRDs for defining backups

ObjectStore (Connectors)
ScheduledBackup (Cron)
Backup (On-Demand)

Best Practice:

Set the retention
Compression for WAL
Test restores regularly

9

KUBERNETES COMMUNITY DAYS INDONESIA 2026

10 of 18

11 of 18

Monitoring

PodMonitor: Prometheus scrapes every Postgres pod

The metrics exporter runs inside each pod, on port 9187
Keep the exporter in mind: it comes back on patch day

11

KUBERNETES COMMUNITY DAYS INDONESIA 2026

12 of 18

13 of 18

The day it hurt

13

14 of 18

The setup

The damage

The fix

1 instance,�Ceph storage min_size 1

Application Exception,�Zeroed pages, carried into the restores

Rebuilt with 2 instances

14

KUBERNETES COMMUNITY DAYS INDONESIA 2026

the day it hurts

"Before I begin this slide, a strict disclaimer: this is a completely hypothetical, purely simulated scenario. It definitely never happened in our live production environment at 6:00 AM on a weekday…�Before i begin, a disclaimer, this is just a ‘simulated’ scenario, it is absolutely not happening in our infrastructure, i guarantee you.�(Pause for laughs)

Here is the scenario: Our database ran as instances: 1, relying entirely on Ceph block storage with min_size: 1. During an OSD migration on the storage layer, one Ceph Placement Group was lost. Because min_size was 1, Ceph didn't stop writes—it wrote zeroed blocks directly into Postgres data files.

At 6:00 AM, the app started throwing exceptions. We inspected the pod, and as you can see in the middle terminal, PG_VERSION was literally all zeroes.

And here was the horrifying realization: our physical backups had backed up those exact zeroed pages. You cannot simply run REINDEX when your underlying heap and system catalogs are wiped out. CloudNativePG did its job, but it couldn't undo storage-level zeroing.

Lesson learned to always implement best practice for ceph itself, and we need to create backup physical before every changes in underlaying infrastructure of cnpg.

Our fix? We created a fresh cluster declaration (device-db-v2) with instances: 2, bootstrapped clean schemas via Flyway[cite: 6], salvaged healthy rows from an earlier restore point[cite: 6], and had the application back by 07:59[cite: 6].

The ultimate lesson: Never run instances: 1 in production, and never rely solely on storage replication to replace database-level high availability."

disclaimer!

this is just a ‘simulated’ scenario, it is absolutely not happening in our infrastructure, i guarantee you.

let’s say one day in 8.00 we noticed an increased error from an application, after inspection we found there’s a missing data

on the index.

why (?) because we migrated ceph osd on previous day and one of the pg is gone and we set min size to 1.

there are 2 ways to fix, either rebuild the index, or pitr to previous healthy backups in this case.

15 of 18

CNPG != Silver Bullet

15

KUBERNETES COMMUNITY DAYS INDONESIA 2026

16 of 18

Day-2 operations

Plugin commands run as: kubectl cnpg <command> <cluster>

16

1 · INSPECT

status

Health, replication and backups

logs cluster

Logs from every instance

psql

A shell on the primary

→

2 · PROTECT

backup

On demand, before a migration or any risky change

→

3 · CHANGE

restart

Rolling restart, replicas first

hibernate on|off

Stop the pods, keep the data

imageName

Major upgrade: the operator runs pg_upgrade

→

4 · REPAIR

destroy

Remove a broken instance; the operator rebuilds it from the others

KUBERNETES COMMUNITY DAYS INDONESIA 2026

17 of 18

Takeaways and Q&A

One Cluster spec in Git replaced conf files and cron scripts

The -rw service hides failover from apps

One instance != HA: run at least three

Test restores: a backup of corrupted DB can carry the damage

Maintenance: patch the operator like any tools you have

17

KUBERNETES COMMUNITY DAYS INDONESIA 2026

18 of 18

18

THANK YOU