Database operations

Examples use <name> and <namespace>. Status is trustworthy only after the controller has observed the current spec.

Check database health

Check generation freshness first, then conditions. status.phase is only a human-readable summary.

kubectl get questdbcluster <name> -n <namespace> \
  -o jsonpath='generation={.metadata.generation}{" observedGeneration="}{.status.observedGeneration}{" phase="}{.status.phase}{"\n"}'
kubectl get questdbcluster <name> -n <namespace> \
  -o jsonpath='{range .status.conditions[*]}{.type}{"="}{.status}{" reason="}{.reason}{" observed="}{.observedGeneration}{" message="}{.message}{"\n"}{end}'

Do not act on old conditions while .metadata.generation differs from .status.observedGeneration. For an ordinary writable cluster, writer readiness requires all four signals: current generation, Available=True/PrimaryReady, Progressing=False/Settled, and WriteHealthy=True/Healthy. Available=True alone can mean only that a replica or a read-serving primary is available. WriteHealthy=False/PrimarySuspended names the impaired WAL table or tables in its message; reads and writes to other tables can remain available. WriteHealthy=True reports the latest wal_tables() observation. It does not perform a synthetic write and does not guarantee free disk capacity, so retain application write probes and storage monitoring where those guarantees are required.

An intentional replica-only follower is the exception: no primary is correct, so WriteHealthy is omitted rather than reported healthy. Require current generation, phase=Following, status.replication.following=true, the expected readyInstances, and an appropriate ReplicationHealthy follower result. True/FollowingExternalSource reports observed progress; a quiet source may report Unknown/StreamNotDetermined, which is acceptable only after confirming the source identity and roots. Treat ReplicationHealthy=False as unhealthy.

kubectl get questdbcluster <name> -n <namespace> -o wide
kubectl get pods -n <namespace> -l questdb.io/cluster=<name> \
  -L questdb.io/instance,questdb.io/role -o wide
kubectl get pvc -n <namespace> -l questdb.io/cluster=<name> \
  -L questdb.io/role,questdb.io/bootstrap -o wide

Services and ports

Service	Purpose
<name>-rw	Current primary. Use for writes and administration.
<name>-ro	Created for store-backed replication. Routes to qualified replicas, but falls back to the primary when no replica qualifies. It is not strict replica isolation.
<name>	Headless, internal identity/DNS service. It publishes unready addresses and must not be used as a client availability endpoint.

Port	Protocol	Published
9000	HTTP and Web Console, including QWP over WebSocket	Always
8812	PostgreSQL wire protocol	Always
9009	InfluxDB Line Protocol over TCP	Always
9003	Minimal HTTP server for health and database metrics	Always
9007/UDP	QWP UDP receiver	Only when enabled

QWP over WebSocket has no port of its own. Ingestion (/write/v4) and streaming query results (/read/v1) are served by the HTTP server on 9000 and share its network settings, so they are available wherever 9000 is.

The QWP UDP receiver is off by default. Enable it with spec.protocols.qwp.udp.enabled; until then, port 9007 is neither opened on the pod nor published on any Service. It is the only port here that is not published on <name>-ro.

The operator owns these Services and reconciles them as ClusterIP (the identity Service is headless). For temporary access, use port-forwarding. For durable external access, create a separate customer-managed Ingress, Gateway, or LoadBalancer that targets the operator Service. Do not mutate the operator-owned Service type.

Connect

Find the admin Secret without printing its value:

ADMIN_SECRET="$(kubectl get questdbcluster <name> -n <namespace> \
  -o jsonpath='{.status.adminSecretName}')"
kubectl get secret "$ADMIN_SECRET" -n <namespace> \
  -o custom-columns='NAME:.metadata.name,CREATED:.metadata.creationTimestamp'

When an administrator explicitly needs the credential, load it into the environment without echoing it and clear it after use:

ADMIN_PASSWORD="$(kubectl get secret "$ADMIN_SECRET" -n <namespace> \
  -o jsonpath='{.data.password}' | base64 -d)"

kubectl port-forward -n <namespace> service/<name>-rw 8812:8812 \
  >/tmp/questdb-pgwire-port-forward.log 2>&1 &
PF_PID=$!
trap 'kill "$PF_PID" 2>/dev/null; unset ADMIN_PASSWORD' EXIT
sleep 2
PGPASSWORD="$ADMIN_PASSWORD" psql -h 127.0.0.1 -p 8812 -U admin -d qdb
kill "$PF_PID"
unset ADMIN_PASSWORD
trap - EXIT

Use service/<name>-ro only for read-only traffic that can tolerate primary fallback. To access the Web Console temporarily, forward service/<name>-rw from local port 9000 and stop the process after the session.

Make mutable changes safely

Changes to spec.image, spec.imagePullSecrets, spec.resources, spec.config, replication tuning, or scheduling recreate affected pods. A rolling change remains single-writer-safe, but recreating the primary can briefly interrupt writes.

Before you start

1. For an ordinary writable cluster, confirm current generation, Available=True/PrimaryReady, Progressing=False/Settled, and WriteHealthy=True/Healthy. Available=True alone is not writer readiness.
2. For an intentional follower, instead confirm current generation, phase=Following, status.replication.following=true, the expected ready count, and an appropriate non-failing follower ReplicationHealthy result as described above.
3. Save the current spec.
4. Make one logical change at a time.

kubectl get questdbcluster <name> -n <namespace> -o yaml \
  > /secure/path/<name>-before.yaml

Change and watch

Apply a reviewed manifest or a narrow patch. This example changes an engine setting:

kubectl patch questdbcluster <name> -n <namespace> --type merge \
  -p '{"spec":{"config":{"cairo.max.uncommitted.rows":"500000"}}}'

GEN="$(kubectl get questdbcluster <name> -n <namespace> \
  -o jsonpath='{.metadata.generation}')"
for _ in $(seq 1 60); do
  OBS="$(kubectl get questdbcluster <name> -n <namespace> \
    -o jsonpath='{.status.observedGeneration}')"
  [ "$OBS" = "$GEN" ] && break
  sleep 5
done
[ "$OBS" = "$GEN" ]
kubectl wait questdbcluster/<name> -n <namespace> \
  --for=condition=Available --timeout=5m

The kubectl wait above is only an availability gate. Before declaring the writer ready, re-read the current-generation conditions and require Available=True/PrimaryReady, Progressing=False/Settled, and WriteHealthy=True/Healthy. Then verify pod UIDs/restarts, connectivity, and the intended setting. If the change fails, inspect ConfigRejected, events, and pod logs. Recover by reverting the mutable spec to the saved value and repeat the bounded generation and full writer-health checks. Do not try to revert immutable fields; create a new cluster when required by the API Reference.

Scale out

A replica is born from a completed backup and then consumes object-store WAL. Before increasing spec.instances, the cluster needs:

• spec.objectStoreRef;
• enabled scheduled backup; and
• a completed seed in .status.replication.seed.

An explicit spec.replication block is optional; add it only for tuning.

BACKUP_STATUS="$(kubectl get questdbcluster <name> -n <namespace> \
  -o jsonpath='{.status.backup.lastBackup.status}')"
SEED="$(kubectl get questdbcluster <name> -n <namespace> \
  -o jsonpath='{.status.replication.seed.backupInstanceName}')"
printf 'backup=%s seed=%s\n' "$BACKUP_STATUS" "$SEED"
[ "$BACKUP_STATUS" = "completed" ] && [ -n "$SEED" ]

kubectl patch questdbcluster <name> -n <namespace> --type merge \
  -p '{"spec":{"instances":3}}'

Watch with bounded checks:

for _ in $(seq 1 90); do
  READY="$(kubectl get questdbcluster <name> -n <namespace> \
    -o jsonpath='{.status.readyInstances}')"
  [ "$READY" = "3" ] && break
  sleep 10
done
[ "$READY" = "3" ]
kubectl get questdbcluster <name> -n <namespace> \
  -o jsonpath='{range .status.conditions[?(@.type=="ReplicationHealthy")]}{.status}{" "}{.reason}{" "}{.message}{"\n"}{end}{range .status.replication.replicas[*]}{.instance}{" caughtUp="}{.caughtUp}{" caughtUpNow="}{.caughtUpNow}{" lagTxns="}{.lagTxns}{"\n"}{end}'

Verify readyInstances equals the requested count, ReplicationHealthy=True, and each current replica is healthy. caughtUp means it has caught up at least once; it is a latch. caughtUpNow is the live freshness reading. An absent caughtUpNow means freshness was not determined and must fail closed.

Scale in

Lowering spec.instances removes replicas. Instance serials are monotonic and are never reused, so names may not remain contiguous after scale or promotion.

spec.pvcRetentionPolicy	Removed replica PVC
Retain (default)	Kept for deliberate inspection or cleanup.
Delete	Deleted during replica scale-in. A later replica gets a new serial and seeds again.

The policy applies only to replicas removed by scale-in. It never deletes the current primary PVC and does not govern whole-cluster deletion. On every managed PVC, questdb.io/role is the instance's current role and changes across promotion; questdb.io/bootstrap records how the volume was born and remains ancestry/state. Do not infer the current primary from bootstrap.

Before scaling in, identify the primary and replicas, choose the retention policy, and inventory PVCs:

kubectl get questdbcluster <name> -n <namespace> -o wide
kubectl get pvc -n <namespace> -l questdb.io/cluster=<name>
kubectl patch questdbcluster <name> -n <namespace> --type merge \
  -p '{"spec":{"instances":1,"pvcRetentionPolicy":"Retain"}}'
GENERATION="$(kubectl get questdbcluster <name> -n <namespace> \
  -o jsonpath='{.metadata.generation}')"
OBSERVED=""
AVAILABLE_OBSERVED=""
for _ in $(seq 1 120); do
  OBSERVED="$(kubectl get questdbcluster <name> -n <namespace> \
    -o jsonpath='{.status.observedGeneration}')"
  AVAILABLE_OBSERVED="$(kubectl get questdbcluster <name> -n <namespace> \
    -o jsonpath='{.status.conditions[?(@.type=="Available")].observedGeneration}')"
  [ "$OBSERVED" = "$GENERATION" ] && \
    [ "$AVAILABLE_OBSERVED" = "$GENERATION" ] && break
  sleep 5
done
if [ "$OBSERVED" = "$GENERATION" ] && \
   [ "$AVAILABLE_OBSERVED" = "$GENERATION" ]; then
  kubectl wait questdbcluster/<name> -n <namespace> \
    --for=condition=Available=True --timeout=10m &&
  kubectl get questdbcluster <name> -n <namespace> -o wide &&
  kubectl get pods,pvc -n <namespace> -l questdb.io/cluster=<name> -o wide
else
  printf 'Timed out waiting for generation %s (status %s, Available %s)\n' \
    "$GENERATION" "$OBSERVED" "$AVAILABLE_OBSERVED" >&2
  false
fi

Verify the managed and ready counts, current primary, the full ordinary writer-health or separate follower contract, and the expected retained or deleted replica PVCs. Confirm each current primary/replica PVC's questdb.io/role label matches that role.

Grow storage

Storage is expand-only, and spec.storage.storageClassName is immutable.

Before you start

STORAGE_CLASS="$(kubectl get questdbcluster <name> -n <namespace> \
  -o jsonpath='{.spec.storage.storageClassName}')"
kubectl get storageclass "$STORAGE_CLASS" \
  -o jsonpath='allowVolumeExpansion={.allowVolumeExpansion}{"\n"}'
kubectl get pvc -n <namespace> -l questdb.io/cluster=<name>

Proceed only when allowVolumeExpansion=true and the CSI driver supports the requested expansion.

Change and verify

kubectl patch questdbcluster <name> -n <namespace> --type merge \
  -p '{"spec":{"storage":{"size":"200Gi"}}}'
GENERATION="$(kubectl get questdbcluster <name> -n <namespace> \
  -o jsonpath='{.metadata.generation}')"
OBSERVED=""
AVAILABLE_OBSERVED=""
for _ in $(seq 1 120); do
  OBSERVED="$(kubectl get questdbcluster <name> -n <namespace> \
    -o jsonpath='{.status.observedGeneration}')"
  AVAILABLE_OBSERVED="$(kubectl get questdbcluster <name> -n <namespace> \
    -o jsonpath='{.status.conditions[?(@.type=="Available")].observedGeneration}')"
  [ "$OBSERVED" = "$GENERATION" ] && \
    [ "$AVAILABLE_OBSERVED" = "$GENERATION" ] && break
  sleep 5
done
if [ "$OBSERVED" = "$GENERATION" ] && \
   [ "$AVAILABLE_OBSERVED" = "$GENERATION" ]; then
  kubectl wait questdbcluster/<name> -n <namespace> \
    --for=condition=Available=True --timeout=10m &&
  kubectl get questdbcluster <name> -n <namespace> -o wide &&
  kubectl get pvc -n <namespace> -l questdb.io/cluster=<name> \
    -o custom-columns='NAME:.metadata.name,REQUESTED:.spec.resources.requests.storage,CAPACITY:.status.capacity.storage,PHASE:.status.phase'
else
  printf 'Timed out waiting for generation %s (status %s, Available %s)\n' \
    "$GENERATION" "$OBSERVED" "$AVAILABLE_OBSERVED" >&2
  false
fi

The availability wait is not proof of writer health. Require the current-generation Available=True/PrimaryReady, Progressing=False/Settled, and WriteHealthy=True/Healthy conditions before closing the change. If StorageResizeBlocked=True, read its reason/message and fix the StorageClass or CSI limitation. A size reduction is rejected. To change StorageClass, restore into a new cluster.

Node maintenance and disruption budgets

The default PodDisruptionBudget (PDB) uses minAvailable: 1 for a singleton and instances-1 for a replicated cluster. A singleton therefore blocks voluntary eviction and can make kubectl drain wait indefinitely.

Before planned maintenance, choose one safe option:

1. scale out and wait for a healthy, current replica;
2. lower spec.scheduling.podDisruptionBudget.minAvailable; or
3. set spec.scheduling.podDisruptionBudget.enabled: false only after accepting database downtime.

spec:
  scheduling:
    podDisruptionBudget:
      enabled: true
      minAvailable: 0

Restore the normal PDB after maintenance and verify the current-generation writer contract: Available=True/PrimaryReady, Progressing=False/Settled, and WriteHealthy=True/Healthy.

A PDB protects only voluntary disruption. On node loss, the operator reports InstanceUnreachable and does not automatically promote a replica. It avoids recreating a pod while the old pod may still run on an unreachable node. Restore the node/volume or follow the explicit promotion and failover procedure.

Delete a database cluster

Deleting a QuestDBCluster removes operator-owned pods, Services, ConfigMaps, and related resources. Its data PVCs are deliberately unowned and remain, and object-store backup/WAL data also remains. pvcRetentionPolicy does not change this whole-cluster behavior.

Before you start

Inventory and export the CR, PVCs, PV reclaim policies, store reference, and effective object prefixes:

kubectl get questdbcluster <name> -n <namespace> -o yaml \
  > /secure/path/<name>-cluster.yaml
kubectl get pvc -n <namespace> -l questdb.io/cluster=<name> -o wide
kubectl get questdbobjectstore -n <namespace>

danger

Deleting the custom resource stops the database and removes its managed runtime objects. Confirm applications are stopped and that the CR export and data-retention inventory are complete.

kubectl delete questdbcluster <name> -n <namespace> --timeout=5m
kubectl get pvc -n <namespace> -l questdb.io/cluster=<name>

Decide separately whether to retain or delete each PVC/PV according to its StorageClass reclaim policy. Decide separately whether to retain or delete cloud objects using customer-owned cloud tooling and policy. The operator never cleans the object store.