Moving data from Heroku to the cluster

How the application's data is carried from a Heroku app to its counterpart on the Kubernetes cluster: the Postgres database and the Flipper feature flags.

This was executed end to end on staging on 2026-09-09, and every command below is one that actually ran. It exists so the production window is a repeat of a rehearsed procedure rather than a first attempt — the production cutover is where a surprise is expensive.

What is carried, and what is not

	Carried	Why
Postgres database	yes	the application's own data
Flipper feature flags	yes	flags are operational state, not code, and are not reproducible from a deploy
Sidekiq queues, sessions	production only	staging can start fresh; production copies them key by key during the freeze window
Rate limits, circuit breakers	no	ephemeral by design, rebuilt within minutes

Encrypted columns need no special handling provided the encryption keys were carried first. SECRET_KEY_BASE and the three ACTIVE_RECORD_ENCRYPTION_* values must already be present in the target environment's secrets before the restore, or the restored rows arrive unreadable. On the cluster they live in the SOPS payload the Terraform root feeds into the app's secret_env.

Feature flags

Use Flipper's own export/import API from a Rails console. Do not copy Redis keys: the adapter stores one hash per feature keyed by the bare feature name (magnificat, p24, …) with no namespace, alongside a flipper_features set, so there is no prefix to match on safely.

There is a second reason not to go through Redis. lib/reports/redis.rb collapses every logical database onto 0 when Reports.staging?, so on staging the Sidekiq queues, the flags, the sessions and the circuit-breaker state all share database 0 — a database-level copy would drag all of it across. In production they are split across 0–5 and the flags live on database 2. The programmatic export is indifferent to both layouts.

Export from Heroku

heroku run --no-tty -a <heroku-app> \
  "rails runner 'STDOUT.write(%q{===FLIPPER-BEGIN===} + Flipper.export.contents + %q{===FLIPPER-END===})'" \
  > flipper-raw.txt

Two things about that command shape, both learned the hard way:

The Ruby has to be one quoted string passed to heroku run. Passing it as separate arguments lets the dyno's bash see the parentheses unquoted and it fails with syntax error near unexpected token '('.
Standard output on a one-off dyno is not clean. The Datadog buildpack's prerun.sh writes a preamble and the tracer prints its own JSON configuration banner, so "take everything from the first {" picks up the tracer's banner instead of the export. Hence the explicit markers; slice between them.

Then extract the payload between the markers into flipper.json.

Import into the cluster

rails runner - reads the script from standard input, so the JSON cannot be piped in alongside it — embed it in the script instead:

B64=$(base64 < flipper.json | tr -d '\n')
{ echo "require 'flipper/exporters/json/export'"
  echo "require 'base64'"
  echo "json = Base64.decode64('$B64')"
  echo "Flipper.import(Flipper::Exporters::Json::Export.new(contents: json))"
  echo "puts Flipper.features.map { |f| [f.key, f.state] }.sort.inspect"
} | kubectl --context <cluster> -n <namespace> exec -i deploy/reports -- bin/rails runner -

Flipper.import runs through the Synchronizer: it makes the target identical to the source, removing flags the source does not have. On a fresh environment that is harmless. Against an environment that already has flags of its own, it is a replace, not a merge — read that line twice before running it on production.

The final puts prints the resulting flag states; compare it against the source.

Database

Restore as the application role

The single most important detail, and the one that broke the staging run:

Restore as the application role, not as postgres.

Running pg_restore -U postgres --no-owner leaves every table owned by postgres. The application connects as its own role, finds no privileges, and crash-loops at boot on PG::InsufficientPrivilege: ERROR: permission denied for table settings. --no-owner is still wanted — it discards Heroku's role names — but the role that runs the restore then becomes the owner, so it has to be the right one.

If a restore has already been run as postgres, ownership can be reassigned afterwards:

DO $$
DECLARE r record;
BEGIN
  FOR r IN SELECT tablename AS n FROM pg_tables WHERE schemaname='public' LOOP
    EXECUTE format('ALTER TABLE public.%I OWNER TO <app-role>', r.n);
  END LOOP;
  FOR r IN SELECT sequencename AS n FROM pg_sequences WHERE schemaname='public' LOOP
    EXECUTE format('ALTER SEQUENCE public.%I OWNER TO <app-role>', r.n);
  END LOOP;
  FOR r IN SELECT viewname AS n FROM pg_views WHERE schemaname='public' LOOP
    EXECUTE format('ALTER VIEW public.%I OWNER TO <app-role>', r.n);
  END LOOP;
END $$;

Verify that nothing is left behind — the count must be 0:

SELECT count(*) FROM pg_tables
WHERE schemaname = 'public' AND tableowner <> '<app-role>';

Where the restore runs

Inside the Postgres pod, not the application pod. The application image is Debian bookworm and ships the PostgreSQL 15 client, which cannot read a custom format archive produced by a PostgreSQL 17 server. The CloudNativePG instance pod has the matching major and local superuser access over the socket, so no password has to be passed around.

With a single instance, that pod is the primary by definition. On an HA cluster, target the -rw service instead of a pinned pod, which can be a read-only replica.

The dump is piped from the workstation rather than downloaded inside the cluster: 25 MB costs nothing to stream, and it avoids putting a signed URL into a pod spec.

Procedure

# 1. Fresh backup on Heroku
heroku pg:backups:capture -a <heroku-app>

# 2. Stop the writers — pg_restore --clean issues DROPs that block on the locks
#    held by live connections
kubectl --context <cluster> -n <namespace> scale deploy/reports deploy/reports-worker --replicas=0

# 3. Stream the dump into the database
curl -sL "$(heroku pg:backups:url -a <heroku-app>)" \
| kubectl --context <cluster> -n <namespace> exec -i <postgres-pod> -- \
    pg_restore --clean --if-exists --no-owner --no-acl -U <app-role> -d <database>

# 4. Bring the application back
kubectl --context <cluster> -n <namespace> scale deploy/reports deploy/reports-worker --replicas=1

Scaling to zero is drift: Terraform holds replicas = 1 in state, so an apply landing in the middle would scale it back up. Restore the count immediately, and avoid running this concurrently with a deploy.

Check the major versions match before starting — heroku pg:info against the postgresql:<major> image the CloudNativePG cluster runs. Staging was 17.9 against 17, so no dump/restore version gap.

Verification

In order, cheapest first:

Row counts against expectations: SELECT count(*) FROM pg_tables WHERE schemaname='public', then a couple of business tables.
GET /healthz — the full check, not ?providers[]=none. It exercises database, cache, Redis and Sidekiq in one request and returns 200 with every provider OK.
Flag states printed by the import, compared with the source.
The Playwright suite against the environment. Before DNS moves, point baseURL at the hostname and map it to the ALB with Chromium's --host-resolver-rules; TLS still validates, because SNI carries the real hostname and the load balancer serves the certificate for it.

After the restore

Every subsequent deploy re-runs the bootstrap Job, which is db:prepare && db:seed_fu against the restored data. That is safe because the seed fixtures are idempotent (seed_once, or writing back the current value) — but it is the same mechanism that will run against production data, so a fixture that stops being idempotent becomes a data-safety bug rather than an inconvenience.

What is carried, and what is not​

Feature flags​

Export from Heroku​

Import into the cluster​

Database​

Restore as the application role​

Where the restore runs​

Procedure​

Verification​

After the restore​