Back Up and Restore TheHive Flow#

TheHive Flow keeps its state in two Docker volumes: the PostgreSQL data and the blob store.

Docker Compose deployment

This page describes the Docker Compose deployment of TheHive Flow. On a Kubernetes deployment, see Configuration and operations on Kubernetes.

What's backed up#

Data	Backed up	Notes
The `orchestrator` database	Yes, by `backup.sh`	Workflows, runs, variables, webhook configuration, audit trail
Blob store contents	Yes, by `backup.sh`	Files ingested through webhooks or runs, and promoted activity outputs
Temporal workflow history	Not by `backup.sh`. Captured by Method 1 and Method 3 Option B	In-flight workflow state
`.env`	No	Keep a secure, encrypted copy off-host separately
TLS certificates	No	Regenerated by `init.sh` or supplied again by the operator
`orchestrator/secret/thehive-api-key`	No	Keep alongside `.env` in your secret store

The database and the blobs are one unit

Rows in the orchestrator database reference blobs by content hash. Restoring the database against a blob store that doesn't hold the referenced content leaves dangling references that the application can't detect: a workflow reading such a file fails at run time, not at restore time. Whichever method you use, capture both together.

Temporal history isn't backed up by backup.sh

The Temporal persistence, the temporal and temporal_visibility databases, isn't included in backup.sh. Restoring the orchestrator database while Temporal carries stale workflow history leaves dangling workflow handles: any workflow that was in-flight at the time of the backup won't resume correctly. This limitation doesn't apply to Method 1 snapshots and Method 3 Option B raw volume archives, which capture the whole PostgreSQL volume, Temporal persistence included.

Step 1: Choose a backup method#

Method	When	TheHive Flow downtime	Consistency across the two volumes
Method 1: Instance or volume snapshot	Single-node host on a platform that offers them, the common case	None: the stack keeps running	Guaranteed by the platform: both volumes are captured at one point in time
Method 2: Hot backup with `backup.sh`	No snapshot facility available, or you want a portable artifact restorable anywhere	Unavailable while the `orchestrator` and `s3-store` services are stopped: a few seconds, plus the time to compress the blob volume, proportional to its size	Guaranteed by the script: it stops both services before copying, so nothing writes during the copy
Method 3: Cold backup	Before a risky operation such as a major upgrade or a migration	The whole TheHive Flow stack is down for the duration	Guaranteed: everything is stopped

Step 2: Run the backup#

Run the method you chose in step 1.

Method 1: Instance or volume snapshot#

Use a snapshot when your host runs on a platform that offers them: it's the simplest and safest option. A snapshot taken while the stack runs captures orchestrator-postgres-data and orchestrator-s3-data at the same instant, so the database and the blobs can't drift apart. It also captures the Temporal persistence, which backup.sh doesn't.

Use your platform's own tooling, such as AWS EBS snapshots, hypervisor snapshots, or LVM. Nothing in the stack is needed, and nothing here interferes.

Two caveats:

A snapshot of a running PostgreSQL is a crash-consistent image, not a clean dump: PostgreSQL replays its write-ahead log on restore, which is supported but isn't the same guarantee as pg_dump. For a guaranteed-clean database artifact, take a hot or cold backup as well.
Snapshot size grows with the blob store volume, which only grows.

Method 2: Hot backup#

Use a hot backup when no snapshot facility is available, or when you want a portable pair of files restorable on any host. It briefly stops the orchestrator and s3-store services while PostgreSQL, Temporal, and nginx remain up:

./scripts/backup.sh

It produces a pair of files in ./backups/: backup-orchestrator-<utc_timestamp>.pgdump and backup-s3-<utc_timestamp>.tar.gz.

What the script does:

Stops the orchestrator and s3-store services.
Runs pg_dump -Fc on the orchestrator database.
Archives the blob store volume as a gzip-compressed tar archive.
Validates both artifacts, sets them to mode 600, then publishes them.
Restarts both services.

The two files are a pair: they share one timestamp, and restore.sh finds the archive from the dump's name. Keep them together and copy them together. They're written under a .partial suffix, with the archive published first and the dump last, so an interrupted publish can leave at worst an inert stray archive, never a lone dump advertising a complete backup whose blobs are absent.

To back up the database only, when the object storage is unreachable or broken and you still need the database out:

./scripts/backup.sh --no-blobs

Without the flag, the script aborts before pg_dump when no s3-store container exists, and there's no other scripted way to get a dump. Such a backup must be restored with restore.sh --no-blobs, and the restored data carries blob references pointing at content the store may not hold. It's an escape hatch, not a routine mode.

Expected downtime is a few seconds for the database, plus the time to compress the blob store volume, which is proportional to its size. On an installation handling large files, measure it once before scheduling: time ./scripts/backup.sh. The s3-store service is stopped for that whole window, because the store keeps a local index that a live copy could capture mid-write.

Runs in flight when the services stop don't fail: Temporal keeps their state and they resume once the orchestrator service restarts. A step still executing at that moment re-runs from the start if it has retries configured, and fails at its timeout otherwise.

Method 3: Cold backup#

Maintenance window required

This procedure involves changing configuration files and restarting services. Schedule a maintenance window to prevent service disruption.

Use a cold backup when you need a guaranteed consistent capture of everything, or before a risky operation such as a major upgrade or a migration.

Name the artifacts so restore.sh can pair them: the script derives the blob archive from the dump's name and accepts only backup-orchestrator-<key>.pgdump paired with backup-s3-<key>.tar.gz. A cold backup named anything else is still a perfectly good backup, but restore.sh refuses it, so use one shared key, exactly as backup.sh does.

Option A: Database dump and volume archive#

Portable and smaller. The pair is restorable by restore.sh:

docker compose down

KEY="cold-$(date -u +%Y%m%dT%H%M%SZ)"

docker compose up -d postgresql
docker compose exec postgresql \
  pg_dump -U postgres -Fc -d orchestrator \
  > "./backups/backup-orchestrator-${KEY}.pgdump"
docker compose down

docker run --rm \
  -v orchestrator-s3-data:/data:ro \
  alpine \
  tar czf - -C /data . > "./backups/backup-s3-${KEY}.tar.gz"

chmod 600 "./backups/backup-orchestrator-${KEY}.pgdump" \
          "./backups/backup-s3-${KEY}.tar.gz"

docker compose up -d

Restore the pair exactly like a backup.sh pair:

./scripts/restore.sh "./backups/backup-orchestrator-${KEY}.pgdump"

Option B: Raw volume archives#

Larger, and captures the Temporal persistence too. This option isn't restorable by restore.sh: the archives are filesystem images of the volumes, not a database dump. Restore them by stopping the stack and extracting each archive back over its volume, the same way restore.sh does for the blob volume.

docker compose down

for vol in orchestrator-postgres-data orchestrator-s3-data; do
  docker run --rm \
    -v "$vol":/data:ro \
    alpine \
    tar czf - -C /data . > "./backups/${vol}-$(date -u +%Y%m%dT%H%M%SZ).tar.gz"
done
chmod 600 ./backups/orchestrator-*-data-*.tar.gz

docker compose up -d

Both options stream the archive to stdout and redirect on the host rather than writing through a bind-mounted directory: the helper container runs as root, so a file it created directly would be root-owned. The chmod 600 isn't cosmetic either, because the blob volume carries the store's S3 secret key in cleartext. See TheHive Flow Security.

(Optional) Step 3: Ship the backups off-host#

This step applies to the file pairs produced by methods 2 and 3. Method 1 snapshots are stored by the platform and don't pass through ./backups/.

The backup.sh script writes its files to ./backups/. Ship them off-host using your existing tooling: the stack doesn't handle this automatically. Ship both halves of a pair, or neither, because a dump without its archive can't be fully restored.

Recommended options:

# rsync to a backup server
rsync -az ./backups/ <backup_server>:/srv/backups/orchestrator/

# Upload to S3
aws s3 cp ./backups/backup-orchestrator-<timestamp>.pgdump s3://<bucket>/orchestrator/
aws s3 cp ./backups/backup-s3-<timestamp>.tar.gz           s3://<bucket>/orchestrator/

# Encrypt before shipping
gpg --encrypt --recipient <recipient> ./backups/backup-orchestrator-<timestamp>.pgdump
gpg --encrypt --recipient <recipient> ./backups/backup-s3-<timestamp>.tar.gz

(Optional) Step 4: Schedule automated backups#

The cron example below schedules method 2. For method 1, configure the cadence and retention in your platform's snapshot tooling instead. Method 3 is typically run on demand before a risky operation rather than scheduled, but the retention job below purges its pairs too, because they land in ./backups/ with the same naming.

The recommended retention is daily backups for 30 days, with at least one weekly backup shipped to a geographically separate location.

Size the retention against the blob archive, not the dump. The dump is roughly proportional to workflow and run metadata, while the archive is roughly proportional to everything ever ingested, and the blob store has no garbage collection. Thirty daily pairs means thirty copies of a volume that only grows: on an installation handling large files, that dominates the disk budget by a wide margin. Measure one pair before committing to a retention window, and prefer snapshots if the platform offers them.

Example cron entries, running daily at 2 AM and keeping 30 days of backups:

0 2 * * * cd /opt/orchestrator && ./scripts/backup.sh >> /var/log/orchestrator-backup.log 2>&1
0 3 * * * find /opt/orchestrator/backups \( -name "*.pgdump" -o -name "*.tar.gz" \) -mtime +30 -delete

The retention job must match both patterns. Purging only *.pgdump files would leave the blob archives, by far the larger half, accumulating forever.

Step 5: Restore#

Maintenance window required

This procedure involves changing configuration files and restarting services. Schedule a maintenance window to prevent service disruption.

The restore.sh procedure below restores a file pair produced by method 2 or method 3 Option A. To restore a method 1 snapshot, use your platform's snapshot tooling instead. To restore method 3 Option B raw archives, stop the stack and extract each archive back over its volume.

The stack must be running with at least PostgreSQL and the s3-store service up. Start them if the stack is down:

docker compose up -d postgresql s3-store

Run the restore with an interactive confirmation:

./scripts/restore.sh ./backups/backup-orchestrator-<timestamp>.pgdump

For automation, skip the confirmation prompt:

./scripts/restore.sh --yes ./backups/backup-orchestrator-<timestamp>.pgdump

To restore the database only, on purpose, leaving the blob store untouched:

./scripts/restore.sh --no-blobs ./backups/backup-orchestrator-<timestamp>.pgdump

Only the dump is passed: the blob archive is the other half of the pair and is found by name. If it's missing, the script aborts before destroying anything rather than restoring half the state. Pass --no-blobs if that's what you actually want.

What the script does:

Verifies the dump file exists, and locates and validates the blob archive.
Prompts for confirmation, unless --yes is passed.
Stops the orchestrator and s3-store services.
Takes a safety copy of the current database and the current blobs, with mode 600.
Drops and recreates the orchestrator database, then runs pg_restore.
Empties the blob store volume and extracts the archive into it.
Restarts both services.

Steps 1 and 2 abort harmlessly. From step 5 on, a failure isn't harmless, and the script's behavior differs by window:

Before anything is destroyed, a failure restarts the services and says so.
After the database is dropped, or while the blob volume is being replaced, the orchestrator service is deliberately left stopped: restarting it against half-restored state would look healthy while serving incomplete data. The script prints the safety copies from step 4 and the exact command to roll back to them or to retry the pair. Those copies share one timestamp, so they obey the same pairing rule as a normal backup.
After both halves are in place but the restart fails, the script says exactly that: the data is fine, only the services are down, and it prints the command to bring them back up.

Restoring the blob volume also reinstates the S3 identity stored inside it, because the volume holds the store's own credentials. That doesn't strand TheHive Flow: the final restart runs init-s3-store again, which applies the identity from the current .env.

Step 6: After the restore#

This step applies after a restore.sh restore of a method 2 or method 3 Option A pair: those don't include the Temporal persistence, so the Temporal workflow history is out of sync with the restored orchestrator database. Skip it after restoring a method 1 snapshot or method 3 Option B archives, where Temporal was captured together with the database.

Restart Temporal to clear its in-memory state:

docker compose restart temporal

Any workflow run that was in-flight at backup time appears stuck. List the running executions with the Temporal admin CLI:

docker compose --profile admin run --rm temporal-admin \
  temporal workflow list \
    --query 'ExecutionStatus = "Running"' \
    --address temporal:7233

The stuck runs are those whose start time predates the backup. The backup timestamp is part of the dump's file name. Terminate each of them, using the workflow ID shown in the list output:

docker compose --profile admin run --rm temporal-admin \
  temporal workflow terminate \
    --workflow-id <workflow_id> \
    --reason "Restored from backup" \
    --address temporal:7233