Skip to main content

Minimal Setup

For a runnable development setup, use the self-hosting examples, which include Docker Compose files and configuration. For development and testing, use:
  1. One PowerSync container running in unified mode (API and replication): 1 vCPU, 512 MiB memory.
  2. A MongoDB or Postgres bucket storage database. For MongoDB, use one node in replica set mode: 1 vCPU, 2 GiB memory (M10+ on Atlas).
  3. A load balancer for TLS.
This setup has no redundancy. If bucket storage data is lost, re-replicate from the source database. All clients must then download their data again.

Production

Run API and replication in separate containers so you can scale them independently.

Storage Requirements

You can use MongoDB or Postgres for bucket storage, independently of your source database: With S3 object storage, S3 stores larger blocks of sync data while MongoDB retains smaller blocks and metadata.

Compute Sizing

Use these CPU and memory settings as a starting point. These may be scaled up or down depending on your specific load patterns.

Replication Container

Run start -r sync to replicate source changes into bucket storage. Only one process can replicate from the source database at a time. Running multiple replication containers provides failover, not more throughput. It is recommended to run 2 replication processes for failover purposes. During rolling deploys or failover, PSYNC_S1003 is expected briefly while another process holds the replication lock. Size replication for source database write volume, row size, and buckets per row. A row in ten buckets creates more replication work than a row in one bucket. Initial replication and catch-up after a large batch update can need more capacity than steady-state replication. Use replication lag, replication throughput, CPU, memory, and database metrics to assess capacity.

API Containers

Run at least two API containers in production for high availability. Scale horizontally by adding API containers as the concurrent client load grows. Run start -r api for client connections. Collect metrics separately from API and replication containers. The max_concurrent_connections config parameter determines the maximum number of concurrent client connections that each API process can handle. The default is 200. We recommend keeping autoscaling targets below the max_concurrent_connections limit (e.g. ~100 concurrent connections per API process) to allow for headroom.
Depending on the use case, handling up to 1,000 concurrent clients per API process is possible. Contact us to discuss your specific requirements and we will recommend an optimal setup.
Scale on both CPU utilization and concurrent connections per API process. Idle connections can keep CPU low, while initial syncs can saturate CPU with fewer connections. Add API containers before many clients are expected to run an initial sync at once, or before deployments that make clients download all their data again. Wait for the new containers to become healthy before those initial syncs or deployments begin. Adding containers does not immediately redistribute existing long-lived connections. Reduce capacity gradually after the download load subsides.

Compact Job

Run compact as a separate scheduled job, hourly for storage version 4 or daily for storage version 2. Compaction resources depend on bucket size, accumulated changes, and concurrency. Monitor completion, duration, CPU, and peak memory to size the job for your dataset. If jobs fail or overlap, check the compact job’s CPU and memory usage and bucket storage performance before changing resources or the schedule. See Compacting Buckets.

Memory Configuration

Set the environment variable NODE_OPTIONS=--max-old-space-size-percentage=80 for the API, replication, and compact containers. This allocates 80% of the container’s assigned memory to the Node.js heap. This avoids manually retuning when scaling resources up or down.

Load Balancer

Use a redundant load balancer to terminate TLS and distribute connections across API containers. Host the API on a dedicated subdomain. For HTTP streaming with nginx, add this proxy block to your TLS server configuration. Replace powersync:8080 with your PowerSync API hostname and port. Disabling response buffering lets sync data reach clients as it arrives.
For nginx Kubernetes ingress, use this annotation:

Health Checks

Configure load-balancer health checks to poll /probes/liveness. See Health Checks.

Migrations

Occasionally, new versions of the PowerSync Service image may require migrations on the underlying storage database. This is also specifically required the first time the service starts up on a new storage database. Replication containers run storage migrations on startup. Unified containers, which run both API and replication, also run them. API containers do not run migrations. Large migrations can delay startup during an update. To run migrations as a separate job before replacing the containers, disable automatic migrations:
service.yaml
Then run migrate up as a separate job before each deployment, including the first.

Backups

Keep non-secret configuration in version control and credentials in a secrets manager. None of the containers use any local storage, so no backups are required there. Recover PowerSync’s sync data by re-replicating from the source database. Allow for the time and capacity needed for full replication and for clients to download their data again.

High Availability and Disaster Recovery

The production setup covers the failure of a single container or node. A standby instance covers failures that affect the whole instance, such as the loss of bucket storage, a Sync Config deployment that syncs the wrong data, or a regional outage. A standby instance is a second PowerSync instance that replicates from the same source database and that you can switch clients to.

Standby Instance

Deploy the same components as the primary instance: replication containers, API containers, the compact job, a load balancer, and a bucket storage database.
  • Use the same source database. Both instances replicate from it independently.
  • Use separate bucket storage. Each instance needs its own bucket storage database and, if enabled, its own S3 bucket or prefix. Two instances cannot share bucket storage.
  • Keep the standby running. A running standby stays up to date with the source database, so clients can connect to it as soon as you switch. A standby that starts after the primary fails must complete its initial replication first.
  • Run the same Sync Config. Deploy Sync Config changes to one instance at a time. See Sync Config Deployments.
  • Accept the same tokens. Configure client authentication so that both instances accept the tokens that your backend issues: use the same keys and the same audience values on both. If your tokens use an instance URL as the audience, add both instance URLs to audience on each instance.
For a Postgres source database, each instance uses its own replication slots. Set a different slot_name_prefix for each instance so that you can identify which instance a slot belongs to.

Standby Region

Where you run the standby determines which failures it covers:
  • Same region as the primary. The standby covers failures of the primary instance, such as the loss of its bucket storage or a Sync Config deployment that syncs the wrong data.
  • Different region. The standby also covers an outage of the primary’s region. For example, if your primary instance runs in us-east-1, run the standby in us-west-2, with its bucket storage in the same region as the standby.
A standby in a different region keeps clients syncing during a regional outage only if the source database and your backend API also remain available. The standby replicates from the source database, and clients upload their writes through your backend. For a replicated Postgres source database, see High Availability / Replicated Databases.

Sync Config Deployments

A single instance already protects clients while a Sync Config deployment is in progress. The Service replicates the new Sync Config alongside the active one. Clients continue to sync with the active Sync Config until the new one completes initial replication, and then the Service makes the new one active. How the standby helps therefore depends on how the deployment goes wrong:
  • Replication of the new Sync Config fails. The new Sync Config never becomes active, so the instance continues to serve clients with the previous version. You do not need to switch clients. Correct the Sync Config and deploy it again.
  • The new Sync Config becomes active but syncs the wrong data. For example, a query syncs more rows than you intended, or leaves out rows that clients need. If you deploy the previous version to the same instance, it becomes active only after the instance replicates it again. A standby that still runs the previous version lets you revert sooner: switch clients to it.
To keep this option, deploy a Sync Config change to one instance first. Confirm that the instance replicates and syncs correctly before you deploy the change to the other instance.

Switching Clients to the Standby

Do not hardcode the URL of the primary instance in the app. Use one of these two options so that you can switch clients without releasing a new app version.

Return the Endpoint From Your Backend

Your fetchCredentials() implementation returns the PowerSync endpoint together with the token. Get the endpoint from your backend API in that function. To switch clients to the standby, change the endpoint that your API returns.
  • If fetchCredentials() already calls your backend for a token, return the endpoint in the same response.
  • If fetchCredentials() returns a JWT from client-side state, for example a token from your authentication provider, it makes no backend call. Add a call that gets the endpoint.
The client SDK caches credentials and fetches new ones when the token expires or the app restarts. A client that is already running therefore keeps trying the failed endpoint until its token expires. Use a short token lifetime, such as 5 to 15 minutes, to reduce this delay.

Reroute With an API Gateway

Give clients one stable URL that points to an API gateway, and configure the gateway to forward connections to the primary instance. To switch clients to the standby, change the gateway’s target. The app needs no endpoint lookup, and fetchCredentials() always returns the same URL. The client SDK retries a failed connection against the same URL, so a client reaches the standby on its next connection attempt and does not wait for its token to expire. A client with an open connection to a primary that still runs stays on the primary until that connection closes. The gateway must meet the same requirements as the load balancer: it must support long-lived streaming connections and must not buffer responses. With either option, host the backend API or gateway so that it remains available when the primary instance or its region fails. Do not load-balance clients between the two instances. A client that alternates between instances downloads all of its data again on every switch. See Mapping Users to PowerSync Endpoints.

What Clients Experience

After the switch, each client performs an initial sync again from the standby.
The client keeps its local data during the switch:
  • The app can read and write the local database for the whole process.
  • Pending writes stay in the upload queue. The client uploads them through your backend, which does not depend on the PowerSync instance.
  • New changes from the source database appear after the sync completes.
On its first connection after the switch, a client can log a checksum error such as Could not apply checkpoint, Checksums didn't match. The client’s local sync state comes from the previous instance and does not match the new instance. The client SDK then retries and downloads its data from the new instance. No action is needed.
Every client downloads all of its data from the standby at about the same time, which puts initial-sync load on the standby’s API containers and bucket storage. Add API containers to the standby before you switch.

After the Primary Recovers

For a Postgres source database, the replication slots of a stopped instance continue to retain WAL on the source database. Restore the instance so that it can catch up, or drop its replication slots. Switching back to the primary makes every client perform an initial sync again. Switch back only if you need to, and add API containers to the primary first to handle a surge of concurrent client reconnections.

Self-Hosted Architecture Diagram

The diagram omits S3 object storage.
Source database changes flow through replication and bucket storage to API containers, a TLS load balancer, and clients. Client writes go through the app backend.

PowerSync Deployment Architecture