Skip to main content
Version: 0.6.0

Decryption Keys Backfill

Wraps credential decryption keys that are still stored in plaintext into encrypted envelopes under DATA_ENCRYPTION_KEY.

This is an operator-run backfill. It ships in the image but never runs on its own, because wrapping keys under the wrong DATA_ENCRYPTION_KEY produces values nobody can unwrap. A human confirms the key and takes a backup first.

Credentials issued before keys were encrypted at rest keep working without it: the read path still recognises a legacy plaintext key. Until it has run, encryption at rest covers only newly issued credentials.

Before running it​

  1. Preview what it would do with the read-only encryption audit, which decrypts every stored envelope under the active key and reports what this backfill would wrap, skip, or abort on.
  2. Back up the database, and record which key the backup pairs with (see Key Management and Recovery).
  3. Confirm DATA_ENCRYPTION_KEY is exactly the key the running application uses.
  4. Stop older application instances that might still write plaintext keys during a rolling upgrade. Alternatively, plan to run it again once they are gone, which is safe and converges.

Running it​

From a source checkout, in packages/reference-implementation:

pnpm backfill:decryption-keys

The published Docker image carries no package manifest for the Reference Implementation, so inside the image run the script directly:

docker compose run --rm -w /app ri node_modules/.bin/tsx scripts/backfill-decryption-keys.ts

The one-off docker compose run form is recognised as a maintenance command, so the entrypoint does not run the boot preflight; the backfill's own key and database checks still apply.

It needs DATA_ENCRYPTION_KEY and a database target. A pre-set RI_DATABASE_URL is honoured as given, and the RI_POSTGRES_* variables are used to construct one only when it is absent, which is the same rule the application follows.

How it protects itself​

Before writing anything, the run decrypts every encrypted value it can find in every store the key protects: credential keys, service instance configurations, the keys of credentials registered from third parties, and idempotent-retry response bodies. A response body that is damaged or does not open is reported, never blocks the run and never counts as proof that the key is right; the next key rotation clears it, and the claim and its credential stay. Any other failed decryption aborts the run, naming every row that could not be read. A wrong key therefore ends as a refusal rather than as damage.

Where nothing stored can prove the key and there are plaintext keys waiting to be wrapped, the run refuses rather than guessing. Both halves matter: a database with nothing to wrap completes without --force, because there is no risky write to gate. --force accepts the risk explicitly, and is only appropriate once you have verified the key out of band and hold a backup:

pnpm backfill:decryption-keys -- --force
docker compose run --rm -w /app ri node_modules/.bin/tsx scripts/backfill-decryption-keys.ts --force

--force covers only that one case. It does not bypass a failed decryption: a run whose preflight cannot read an existing service instance configuration or credential key aborts whether or not the flag is passed. A response body that will not open is the one exception, reported rather than blocking.

What it reports​

A completed run reports how many keys it wrapped and how many were already protected. Three cases get called out individually:

  • Suspect rows. A stored value that resembles a corrupted envelope is skipped rather than treated as plaintext and wrapped again. These rows are listed by id for manual inspection, and the run exits non-zero so an automated caller notices.
  • Rows deleted mid-run. A credential removed while the backfill was working is skipped and reported, which is a benign race rather than a failure.
  • An unproven key. Whenever a run completes without an envelope to verify the key against, whether it wrapped rows under --force or had nothing to wrap, it says so and asks you to confirm a wrapped key decrypts correctly.

After a run that wrapped keys, re-run the encryption audit. A clean report is the proof that DATA_ENCRYPTION_KEY unwraps the values this run wrapped; any envelope the audit cannot open is a failure. If the audit reports that nothing existed to verify, the key remains unproven.

As an optional per-record spot check, choose a known encrypted native record and request GET /api/v1/library/{id}.

  • A 200 response with a non-null decryptionKey means the key was revealed.
  • A 200 response with hasKey: true, decryptionKey: null and a DECRYPTION_KEY_UNAVAILABLE warning is a stored key the deployment key could not unwrap and is a failure.
  • hasKey: false is a record that holds no key.

A row whose stored key is still legacy plaintext is returned as-is by this read and proves nothing about the deployment key, so the audit is the proof.

A run that finishes its work exits 0, including one that reported deleted rows or proceeded under --force. It exits 1 when it skipped suspect rows, when it refused an unproven key, when the preflight could not decrypt a service instance configuration or a credential key, and when a write failed.

Re-running​

Re-running is safe and converges. Rows already holding an envelope are counted as already protected and left untouched. After a rolling upgrade has drained the old writers, the second run therefore only picks up what those writers added.

Rolling back afterwards​

Once any encrypted key has been written, whether by a newly issued credential or by this backfill, rolling back to an application version without an unwrap step returns the raw envelope JSON in place of those decryption keys. Treat the upgrade as forward-only once credentials have been issued, or restore the database backup that pairs with the key.