`ps_data__`
This contains the data for each "table", in JSON format. Since JSON is being used, this table's schema does not change when columns are added, removed or changed in the Sync Streams/Rules and client-side schema.
`ps_data_local__`
Same as the previous point, but for [local-only](/client-sdks/advanced/local-only-usage) tables.
`` (`VIEW`)
These are views on the above `ps_data` tables, with each defined column in the client-side schema extracted from the JSON. For example, a `description` text column would be `CAST(data ->> '$.description' as TEXT)`.
`ps_untyped`
Any synced table that is not defined in the client-side schema is placed here. If the table is added to the schema at a later point, the data is then migrated to `ps_data__`.
`ps_oplog`
This is the operation history data as received from the [PowerSync Service](/architecture/powersync-service), grouped per bucket.
`ps_crud`
The client-side upload queue (see [Writing Data](#writing-data-via-sqlite-database-and-upload-queue) below)
`ps_buckets`
A small amount of metadata for each bucket.
`ps_migrations`
Table keeping track of Client SDK schema migrations.
Most rows are present in at least two tables: `ps_data__` and `ps_oplog`.
The copy of the row in `ps_oplog` may be newer than the one in `ps_data__`. This is because of the checkpoint system in PowerSync that gives the system its consistency properties. When a full [checkpoint](/architecture/consistency) has been downloaded, data is copied over from `ps_oplog` to the individual `ps_data__` tables.
It is possible for different [buckets](/architecture/powersync-service#bucket-system) to include overlapping data (for example, if multiple buckets contain data from the same table). If rows with the same table and ID have been synced via multiple buckets, it may be present multiple times in `ps_oplog`, but only one will be preserved in the `ps_data__` table (the one with the highest `op_id`).
**Raw Tables Instead of JSON-Backed SQLite Views**: If you run into limitations with the above JSON-based SQLite view system, check out the [Raw Tables feature](/client-sdks/advanced/raw-tables) which allows you to define and manage raw SQLite tables to work around some of the limitations of PowerSync's default JSON-backed SQLite views system.
## Writing Data (via SQLite Database and Upload Queue)
Any mutations on the SQLite database, namely updates, deletes and inserts, are immediately reflected in the SQLite database, and also automatically placed into an **upload queue** by the Client SDK.
The upload queue is a blocking [FIFO](https://en.wikipedia.org/wiki/FIFO_%28computing_and_electronics%29) queue.
The upload queue is automatically managed by the PowerSync Client SDK.
The Client SDK processes the upload queue by invoking an `uploadData()` function [that you define](/configuration/app-backend/client-side-integration) when you integrate the Client SDK. Your `uploadData()` function implementation should call your [backend application API](/configuration/app-backend/setup) to persist the mutations to the backend source database.
PowerSync is designed this way so that you can apply your own backend business logic, validations and authorization to any mutations going to your source database.
The PowerSync Client SDK automatically takes care of network failures and retries. If processing mutations in the upload queue fails (e.g. because the user is offline), it is automatically retried. For a detailed breakdown of when `uploadData()` is called, including throttling, retry behavior, and error handling, see [When `uploadData()` is Called](/configuration/app-backend/client-side-integration#when-uploaddata-is-called).
server , as well as being persisted into an [upload queue](/architecture/client-architecture#writing-data-via-sqlite-database-and-upload-queue).
While mutations are present in the upload queue, the client does not advance to a new checkpoint. This means the client never has to resolve conflicts locally.
Only once all the client-side mutations have been acknowledged by the server, and the data for that new checkpoint is downloaded by the client, does the client advance to the next checkpoint. This ensures that the operations are always ordered correctly on the client.
There is one nuanced case here, which is buckets with [Priority 0](/sync/advanced/prioritized-sync#special-case-priority-0) if you are using [Prioritized Syncing](/sync/advanced/prioritized-sync).
## Types of Client-Side Mutations/Operations
The client automatically records mutations to the client-side database as `PUT`, `PATCH` or `DELETE` operations — corresponding to `INSERT`, `UPDATE` or `DELETE` statements in SQLite. These are grouped together in a batch per client-side transaction.
Since the [developer has full control](/architecture/client-architecture#writing-data-via-sqlite-database-and-upload-queue) over how mutations are applied to the source database, more advanced operations can be modeled on top of these three. See [Custom Conflict Resolution](/handling-writes/custom-conflict-resolution) for examples.
## Validation and Conflict Handling
With PowerSync offering [full flexibility](/architecture/client-architecture#writing-data-via-sqlite-database-and-upload-queue) in how mutations are applied on the server , it is also the developer's responsibility to implement this correctly to avoid consistency issues.
Some scenarios to consider:
While the client was offline, a row was modified on the client-side. By the time the client is online again, that row has been deleted on the source database. Some options for handling the mutation in your backend:
* Discard the mutation.
* Discard the entire transaction.
* Re-create the row .
* Record the failed mutation elsewhere, potentially notifying the user and allowing the user to resolve the issue.
Some other examples include foreign-key or not-null constraints, maximum size of numeric fields, unique constraints, and access restrictions (such as row-level security policies).
In an online-only application, the user typically sees the error as soon as it occurs, and can correct the issue as required. In an offline-capable application that syncs asynchronously with the server, these errors may occur much later than when the mutation was made, so more care is required to handle these cases.
Special care must be taken so that issues such as those do not block the upload queue. The upload queue in the PowerSync Client SDK is a blocking [FIFO](https://en.wikipedia.org/wiki/FIFO_%28computing_and_electronics%29) queue, and the queue cannot advance if the backend does not acknowledge a mutation . And as mentioned above, if the queue cannot be cleared, the client does not move on to the next checkpoint of synced data.
There is no single correct choice for handling write failures like these. The best action depends on the specific application and scenario. However, we do have some suggestions for general approaches:
1. In general, consider relaxing constraints somewhat on the backend where they are not absolutely required. It may be better to accept data that is somewhat inconsistent (e.g. a client not applying all expected validations), rather than discarding the data completely.
2. If it is critical to preserve all client mutations and preserve the order of mutations:
1. Block the client's upload queue on unexpected errors (don't acknowledge the mutation in your backend API).
2. Implement error monitoring to be notified of issues, and resolve the issues as soon as possible.
3. If it is critical to preserve all client mutations, but the exact order may not be critical:
1. On a constraint error, persist the transaction in a separate queue on your backend, and acknowledge the change.
2. The backend queue can then be inspected and retried asynchronously, without blocking the client-side upload queue.
4. If it is acceptable to lose some mutations due to constraint errors:
1. Discard the mutation, or the entire transaction if the changes must all be applied together.
2. Implement error notifications to detect these issues.
See also:
* [Handling Update Conflicts](/handling-writes/handling-update-conflicts)
* [Custom Conflict Resolution](/handling-writes/custom-conflict-resolution)
## Questions?
If you have any questions about consistency, please [join our Discord](https://discord.gg/powersync) to discuss.
# PowerSync Protocol
Source: https://docs.powersync.com/architecture/powersync-protocol
Technical overview of the sync protocol between PowerSync clients and the Service.
This contains a broad overview of the sync protocol used between PowerSync clients and the [PowerSync Service](/architecture/powersync-service).
For details, see the implementation in the various PowerSync Client SDKs.
## Design
The PowerSync protocol is designed to efficiently sync changes to clients, while maintaining [consistency](/architecture/consistency) and integrity of data.
The same process is used for:
* Downloading the initial set of data
* Bulk downloading changes after being offline for a while
* Incrementally streaming changes while connected
## Concepts
### Buckets
All synced data is grouped into [buckets](/architecture/powersync-service#bucket-system). A bucket represents a collection of synced rows, synced to any number of users.
[Buckets](/architecture/powersync-service#bucket-system) are a core concept that allows PowerSync to efficiently scale to tens of thousands of concurrent clients per PowerSync Service instance, and incrementally sync changes to hundreds of thousands of rows (or even [a million or more](/resources/performance-and-limits#sync-powersync-service-→-client)) to each client.
Each bucket keeps an ordered list of changes to rows within the bucket (operation history) — generally as `PUT` or `REMOVE` operations.
* `PUT` is the equivalent of `INSERT OR REPLACE`
* `REMOVE` is slightly different from `DELETE`: a row is only deleted from the client if it has been removed from *all* buckets synced to the client.
As a practical example of how buckets manifest themselves, say you have a bucket named `user_todo_lists` that contains the to-do lists for a user, and that bucket utilizes a `user_id` parameter (which will be obtained from the JWT). Now suppose users with IDs `A` and `B` exist in the source database. PowerSync will then replicate data from the source database and create individual buckets with bucket IDs `user_todo_lists["A"]` and `user_todo_lists["B"]`.
Buckets are scoped by their parameters (`A` and `B` in this example), so they are always synced as a whole. For user `A` to receive only their relevant to-do lists, they would sync the entire contents of the bucket `user_todo_lists["A"]`.
### Checkpoints
A checkpoint is a sequential ID that represents a single point-in-time for consistency purposes. This is further explained in [Consistency](/architecture/consistency).
### Checksums for Verifying Data Integrity
For any checkpoint, the client and server compute a per-bucket checksum. This is essentially the sum of checksums of individual operations within the bucket, with each individual checksum being a hash of the operation data.
The checksum helps to ensure that the client has all the correct data. In the hypothetical scenario where the bucket data becomes corrupted on the PowerSync Service, the checksums will stop matching, and the client will re-download the entire bucket.
Checksums are not a cryptographically secure method to verify data integrity. Rather, they are designed to detect simple data mismatches, whether due to bugs, bucket data tampering, or other corruption issues.
### Compacting
To avoid indefinite growth in size of buckets, the operation history of a bucket can be [compacted](/maintenance-ops/compacting-buckets). Stale updates are replaced with marker entries, which can be merged together, while keeping the same checksums.
## Protocol
A client initiates a sync session using:
1. A JWT that typically contains the `user_id`, and additional parameters (optional).
2. A list of current buckets that the client has, and the latest operation ID in each.
The server then responds with a stream of:
1. **Checkpoint available**: A new checkpoint ID, with a checksum for each bucket in the checkpoint.
2. **Data**: New operations for the above checkpoint for each relevant bucket, starting from the last operation ID as sent by the client.
3. **Checkpoint complete**: Sent once all data for the checkpoint has been sent.
The server then waits until a new checkpoint is available, then repeats the above sequence.
The stream can be interrupted at any time, at which point the client will initiate a new session, resuming from the last point.
If a checksum validation fails on the client, the client will delete the bucket and start a new sync session.
Data for individual rows are represented [using JSON](/architecture/client-architecture#client-side-schema-and-sqlite-database-structure). The protocol itself is schemaless. The client is expected to use their own copy of the schema , and gracefully handle schema differences.
### Write Checkpoints
Write checkpoints are used to ensure clients have synced their own mutations back before applying downloaded data locally.
Creating a write checkpoint is a separate operation, which is performed by the client after all mutations have been uploaded (i.e. the client's [upload queue](/architecture/client-architecture#writing-data-via-sqlite-database-and-upload-queue) has been successfully fully processed and is empty). It is [important](/handling-writes/writing-client-changes#why-must-my-write-endpoint-be-synchronous) that this happens after the data has been written to the backend source database.
The server then keeps track of the current CDC stream position on the database (LSN in Postgres and SQL Server, resume token in MongoDB and GTID+Binlog Position in MySQL), and notifies the client when the data has been replicated, as part of checkpoint data in the normal data stream.
# PowerSync Service
Source: https://docs.powersync.com/architecture/powersync-service
Understand the PowerSync Service architecture and how it replicates data and delivers real-time sync.
When we say "PowerSync instance" we are referring to an instance of the [PowerSync Service](https://github.com/powersync-ja/powersync-service), which is the server-side component of the sync engine responsible for the *read path* from the source database to client-side SQLite databases. The primary purposes of the PowerSync Service are (1) replicating data from your source database (Postgres, MongoDB, MySQL, SQL Server or Convex), and (2) streaming data to clients. Both of these happen based on your [Sync Streams](/sync/streams/overview) (or legacy [Sync Rules](/sync/rules/overview)).
## Bucket System
The concept of *buckets* is core to PowerSync and its scalability.
*Buckets* are basically partitions of data that allow the PowerSync Service to efficiently query the correct data that a specific client needs to sync.
With [Sync Streams](/sync/streams/overview), buckets are created **implicitly** based on your stream definitions, their queries, and subqueries. You don't need to understand or manage buckets directly. The PowerSync Service handles this automatically.
For example, if you define a stream like:
```yaml theme={null}
streams:
user_lists:
auto_subscribe: true
query: SELECT * FROM lists WHERE owner_id = auth.user_id()
```
PowerSync automatically creates the appropriate buckets internally based on the query parameters.
With legacy [Sync Rules](/sync/rules/overview), you explicitly define the buckets using `bucket_definitions` and specify which [parameters](/sync/rules/overview#parameters) are used for each bucket.
### How Buckets Work
To understand how buckets enable efficient syncing, consider this example: say you have data scoped to users, such as the to-do lists for each user. Based on the data that exists in your source database, PowerSync will create individual buckets for each user. If users with IDs `1`, `2`, and `3` exist in your source database, PowerSync will create buckets with IDs `user_todo_lists["1"]`, `user_todo_lists["2"]`, and `user_todo_lists["3"]`.
When a user with `user_id=1` in their JWT connects to the PowerSync Service, PowerSync can very efficiently look up the appropriate bucket to sync, i.e. `user_todo_lists["1"]`.
With legacy Sync Rules, a bucket ID is formed from the bucket definition name and its parameter values, for example `user_todo_lists["1"]`. With Sync Streams, the bucket IDs are generated automatically based on your stream queries. You don't need to define and name buckets explicitly.
### Deduplication for Scalability
The bucket system also allows for high-scalability because it *deduplicates* data that is shared between different users.
For example, suppose that instead of `user_todo_lists`, we have `org_todo_lists` buckets, each containing the to-do lists for an *organization*, and we use an `organization_id` parameter from the JWT for this bucket. Now suppose that users with IDs `1` and `2` both belong to an organization with an ID of `1`. In this scenario, both users `1` and `2` will sync from a bucket with a bucket ID of `org_todo_lists["1"]`.
This also means that the PowerSync Service has to keep track of less state per user. As a result, server-side resource requirements don't scale linearly with the number of users/clients.
## Operation History
Each bucket stores the *recent history* of operations on each row , not just the latest state of the row.
This is another core part of the PowerSync architecture. The PowerSync Service can efficiently query the *operations* that each client needs to receive in order to be up to date. Tracking of operation history is also key to the data integrity and [consistency](/architecture/consistency) properties of PowerSync.
When a change occurs in the source database that affects a certain bucket (based on your Sync Streams/Rules), that change will be appended to the operation history in that bucket. Buckets are therefore treated as "append-only" data structures. That being said, to avoid an ever-growing operation history, the buckets can be [compacted](/maintenance-ops/compacting-buckets) (this is automatically done on PowerSync Cloud).
## Bucket Storage
The PowerSync Service persists the bucket state in durable storage: there is a pluggable storage layer for bucket data, and MongoDB and Postgres are currently supported as *bucket storage* databases. The *bucket storage* database is separate from the connection to your *source database* (Postgres, MongoDB, MySQL, SQL Server or Convex). Our cloud-hosting offering (PowerSync Cloud) uses MongoDB Atlas as the *bucket storage* database.
With [S3 object storage](/sync/advanced/storage-version-4#s3-object-storage) (Beta, requires storage version 4), larger blocks of prepared sync data are stored in S3 and synced to clients directly from there, while MongoDB keeps the metadata that locates them. Clients continue to connect only to the PowerSync Service.
Persisting the bucket state in a database is also part of how PowerSync achieves high scalability: it means that the PowerSync Service can have a low memory footprint even as you scale to very large volumes of synced data and users/clients.
The layout of that persisted data follows a storage version. That version can change when you deploy new Sync Streams or Sync Rules. That way, large internal collections can evolve without always requiring expensive upfront migrations across your entire dataset. In most cases you do not need to name the storage version in your Sync Config. The optional `config.storage_version` field is there for situations where you do need explicit control. Examples include preparing for a Service downgrade that only supports an older storage format, testing different storage formats in a non-production environment, or keeping bucket data on an older stable format while you change other parts of the Sync Config. For more details, see [Storage version](/sync/advanced/compatibility#storage-version).
## Replication from the Source Database
As mentioned above, one of the primary purposes of the PowerSync Service is replicating data from the source database, based on your Sync Streams/Rules:
The `@powersync/attachments` package (JavaScript/TypeScript) and `powersync_attachments_helper` package (Flutter/Dart) are deprecated. Attachment functionality is now built-in to the PowerSync SDKs. Please use the [built-in attachment helpers](#sdk-%26-demo-reference) instead, and see the [migration notes](#migrating-from-deprecated-packages).
While PowerSync excels at syncing structured data, storing large files (images, videos, PDFs) directly in SQLite is not recommended. Embedding files as base64-encoded data or binary blobs in database rows can lead to many issues.
Instead, PowerSync uses a **metadata + storage provider pattern**: sync small metadata records through PowerSync while storing actual files in purpose-built storage systems (S3, Supabase Storage, Cloudflare R2, etc.). This approach provides:
* **Optimal performance** - Database stays small and fast
* **Automatic queue management** - Background uploads/downloads with retry logic
* **Works offline** - Local files available immediately, sync happens in background
* **Cache management** - Automatic cleanup of unused files
* **Platform flexibility** - Works across web, mobile, and desktop
## SDK & Demo Reference
We provide attachment helpers for multiple platforms:
| SDK | Package | Min. SDK version | Demo App |
| ------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------- |
| **JavaScript/TypeScript** | [Built-in attachments (alpha)](https://github.com/powersync-ja/powersync-js/tree/main/packages/common/src/attachments) | Web v1.33.0, React Native v1.30.0, Node.js v0.17.0 | [React Native Todo](https://github.com/powersync-ja/powersync-js/tree/main/demos/react-native-supabase-todolist) |
| **Flutter** | [Built-in attachments (alpha)](https://pub.dev/documentation/powersync_core/latest/topics/attachments-topic.html) | v1.16.0 | [Flutter Todo](https://github.com/powersync-ja/powersync.dart/tree/main/demos/supabase-todolist) |
| **Swift** | [Built-in attachments (alpha)](https://github.com/powersync-ja/powersync-swift/blob/main/Sources/PowerSync/attachments/README.md) | v1.0.0 | [iOS Demo](https://github.com/powersync-ja/powersync-swift/tree/main/Demos/PowerSyncExample) |
| **Kotlin** | [Built-in attachments (alpha)](https://github.com/powersync-ja/powersync-kotlin/tree/main/common/src/commonNonWeb/kotlin/com/powersync/attachments) | v1.0.0 | [Android Todo](https://github.com/powersync-ja/powersync-kotlin/tree/main/demos/android-supabase-todolist) |
| **.NET** | [Built-in attachments (alpha)](https://github.com/powersync-ja/powersync-dotnet/tree/main/PowerSync/PowerSync.Common/Attachments/README.md) | v0.1.2 | [MAUI Todo](https://github.com/powersync-ja/powersync-dotnet/tree/main/demos/MAUITodo) |
Most demo applications use Supabase Storage as the storage provider, but the patterns are adaptable to any storage system.
## How It Works
On React Native, use a streaming transport instead of a remote storage adapter. You cannot control your users' devices or file sizes, and buffering large files can exhaust memory on low-end devices. Node.js supports the same setup. See [Transferring Large Files Without Buffering](#transferring-large-files-without-buffering).
For security reasons, client-side implementations should use signed URLs:
1. Request a signed upload/download URL from your backend
2. Your backend validates permissions and generates a temporary URL
3. Client uploads/downloads directly to storage using the signed URL
4. Never expose storage credentials to clients
### Local Storage Adapter
The **Local Storage Adapter** handles file persistence on the device. PowerSync provides implementations for common platforms and allows you to create custom adapters.
**Interface methods:**
* `initialize()` - Set up storage (create directories, etc.)
* `saveFile(path, data)` - Write file to storage
* `readFile(path)` - Read file from storage
* `deleteFile(path)` - Remove file from storage
* `fileExists(path)` - Check if file exists
* `getLocalUri(filename)` - Get full path for a filename
**Built-in adapters:**
* **IndexedDB** - For web browsers (`IndexDBFileSystemStorageAdapter`)
* **Node.js Filesystem** - For Node/Electron (`NodeFileSystemAdapter`)
* **React Native** - For React Native with Expo or bare React Native we have a dedicated package [(`@powersync/attachments-storage-react-native`)](https://github.com/powersync-ja/powersync-js/tree/main/packages/attachments-storage-react-native)
* **Native mobile storage** - For Flutter, Kotlin, Swift
The React Native local storage adapter requires Expo 54 or later.
### Attachment Queue
The **Attachment Queue** is the orchestrator that manages the entire attachment lifecycle. It:
* **Watches your data model** - You pass a `watchAttachments` function as a parameter that monitors which files your app references
* **Manages state transitions** - Automatically moves files through states (upload/download → synced → archive → delete)
* **Handles retries** - Failed operations are retried on the next sync interval
* **Performs cleanup** - Removes archived files that are no longer needed
* **Verifies integrity** - Checks local files exist and repairs inconsistencies
**Watched Attachments pattern:**
The queue needs to know which attachments exist in your data model. The `watchAttachments` function you provide monitors your data model and returns a list of attachment IDs that your app references. The queue compares this list with its internal attachment table to determine:
* **New attachments** - Download them
* **Missing attachments** - Upload them
* **Removed attachments** - Archive them
The `watchAttachments` queries are reactive and execute whenever the watched tables change, keeping the attachment queue in sync with your data model.
There are a few scenarios you might encounter:
* **Single attachment type** - Watch one table. For example, if users have profile photos: `SELECT photo_id FROM users WHERE photo_id IS NOT NULL`
* **Multiple attachment types, single queue** - Combine queries with SQL `UNION ALL` to watch attachments across different tables (e.g., `users.photo_id`, `documents.document_id`) in one queue
* **Multiple attachment types, multiple queues** - Create a separate queue per attachment type. Each queue watches its own table(s) with a simpler query, allowing independent configuration, at the cost of some extra memory
Implementation examples for all three are shown in the [Initialize Attachment Queue](#initialize-attachment-queue) section below.
## Implementation Guide
### Installation
```bash JavaScript/TypeScript theme={null}
# Built into @powersync/web, @powersync/node, and @powersync/react-native.
# React Native requires an additional storage adapter:
# npm install @powersync/attachments-storage-react-native
# Expo projects (v54+) also require:
# npm install expo-file-system
# Bare React Native projects also require:
# npm install @dr.pogodin/react-native-fs
```
```bash Dart/Flutter theme={null}
# Built into the powersync package (v1.16.0+). No additional installation needed.
```
```swift Swift theme={null}
// Built into the PowerSync Swift SDK (v1.0.0+). No additional installation needed.
```
```kotlin Kotlin theme={null}
// Built into the PowerSync Kotlin SDK (v1.0.0+). No additional installation needed.
```
```bash .NET theme={null}
# Built into PowerSync.Common (v0.1.2+). No additional installation needed.
```
### Setup: Add Attachment Table to Schema
```typescript JavaScript/TypeScript theme={null}
import { Schema, Table, column, AttachmentTable } from '@powersync/web';
const appSchema = new Schema({
users: new Table({
name: column.text,
email: column.text,
photo_id: column.text // References attachment ID
}),
// Add the attachment table
attachments: new AttachmentTable()
});
```
```dart Flutter theme={null}
import 'package:powersync/powersync.dart';
import 'package:powersync_core/attachments/attachments.dart';
final schema = Schema([
Table('users', [
Column.text('name'),
Column.text('email'),
Column.text('photo_id'), // References attachment ID
]),
AttachmentsQueueTable(),
]);
```
```swift Swift theme={null}
import PowerSync
let users = Table(
name: "users",
columns: [
Column.text("name"),
Column.text("email"),
Column.text("photo_id"), // References attachment ID
]
)
let schema = Schema(
tables: [
users,
// Add the local-only table which stores attachment states
createAttachmentTable(name: "attachments")
]
)
```
```kotlin Kotlin theme={null}
import com.powersync.attachments.createAttachmentsTable
import com.powersync.db.schema.Column
import com.powersync.db.schema.Schema
import com.powersync.db.schema.Table
val users = Table(
name = "users",
columns = listOf(
Column.text("name"),
Column.text("email"),
Column.text("photo_id") // References attachment ID
)
)
val schema = Schema(
users,
// Add the local-only table which stores attachment states
createAttachmentsTable("attachments")
)
```
```csharp .NET theme={null}
using PowerSync.Common.Attachments;
using PowerSync.Common.DB.Schema;
var users = new Table(
"users",
new Dictionary
{
["name"] = ColumnType.Text,
["email"] = ColumnType.Text,
["photo_id"] = ColumnType.Text, // References attachment ID
});
// new Table(typeof(Attachment)) uses the [Table("attachments", LocalOnly = true)] attribute
var schema = new Schema(users, new Table(typeof(Attachment)));
```
### Configure Storage Adapters
```typescript JavaScript/TypeScript theme={null}
// For web browsers (IndexedDB)
import { IndexDBFileSystemStorageAdapter } from '@powersync/web';
const localStorage = new IndexDBFileSystemStorageAdapter('my-app-files');
// For Node.js/Electron (filesystem)
// import { NodeFileSystemAdapter } from '@powersync/node';
// const localStorage = new NodeFileSystemAdapter('./user-attachments');
// For React Native (Expo or bare React Native)
// Need to install @powersync/attachments-storage-react-native
//
// For Expo projects, also install expo-file-system
// import { ExpoFileSystemStorageAdapter } from '@powersync/attachments-storage-react-native';
// const localStorage = new ExpoFileSystemStorageAdapter();
//
// For bare React Native, also install @dr.pogodin/react-native-fs
// import { ReactNativeFileSystemStorageAdapter } from '@powersync/attachments-storage-react-native';
// const localStorage = new ReactNativeFileSystemStorageAdapter();
// Remote storage adapter (example with signed URLs)
const remoteStorage = {
async uploadFile(fileData: ArrayBuffer, attachment: AttachmentRecord) {
// Request signed upload URL from your backend
const { uploadUrl } = await fetch('/api/attachments/upload-url', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({
filename: attachment.filename,
contentType: attachment.mediaType
})
}).then(r => r.json());
// Upload to cloud storage using signed URL
await fetch(uploadUrl, {
method: 'PUT',
body: fileData,
headers: {
'Content-Type': attachment.mediaType || 'application/octet-stream'
}
});
},
async downloadFile(attachment: AttachmentRecord): Promise {
// Request signed download URL from your backend
const { downloadUrl } = await fetch(
`/api/attachments/${attachment.id}/download-url`
).then(r => r.json());
// Download from cloud storage
const response = await fetch(downloadUrl);
return response.arrayBuffer();
},
async deleteFile(attachment: AttachmentRecord) {
// Delete via your backend
await fetch(`/api/attachments/${attachment.id}`, {
method: 'DELETE'
});
}
};
```
```dart Flutter theme={null}
import 'dart:io';
import 'dart:typed_data';
import 'package:path_provider/path_provider.dart';
import 'package:powersync_core/attachments/attachments.dart';
import 'package:powersync_core/attachments/io.dart';
import 'package:http/http.dart' as http;
// For Flutter (native platforms)
Future getLocalStorage() async {
final appDocDir = await getApplicationDocumentsDirectory();
final attachmentsDir = Directory('${appDocDir.path}/attachments');
return IOLocalStorage(attachmentsDir);
}
// Remote storage adapter (example with signed URLs)
class SignedUrlStorageAdapter implements RemoteStorage {
@override
Future uploadFile(
Stream> fileData,
Attachment attachment,
) async {
// Request signed upload URL from your backend
final response = await http.post(
Uri.parse('/api/attachments/upload-url'),
headers: {'Content-Type': 'application/json'},
body: jsonEncode({
'filename': attachment.filename,
'contentType': attachment.mediaType,
}),
);
final uploadUrl = jsonDecode(response.body)['uploadUrl'] as String;
// Collect stream data
final bytes = [];
await for (final chunk in fileData) {
bytes.addAll(chunk);
}
// Upload to cloud storage using signed URL
await http.put(
Uri.parse(uploadUrl),
body: Uint8List.fromList(bytes),
headers: {
'Content-Type': attachment.mediaType ?? 'application/octet-stream',
},
);
}
@override
Future>> downloadFile(Attachment attachment) async {
// Request signed download URL from your backend
final response = await http.get(
Uri.parse('/api/attachments/${attachment.id}/download-url'),
);
final downloadUrl = jsonDecode(response.body)['downloadUrl'] as String;
// Download from cloud storage
final httpResponse = await http.get(Uri.parse(downloadUrl));
return Stream.value(httpResponse.bodyBytes);
}
@override
Future deleteFile(Attachment attachment) async {
// Delete via your backend
await http.delete(
Uri.parse('/api/attachments/${attachment.id}'),
);
}
}
```
```swift Swift theme={null}
import Foundation
import PowerSync
// For iOS/macOS (FileManager)
func getAttachmentsDirectoryPath() throws -> String {
guard let documentsURL = FileManager.default.urls(
for: .documentDirectory,
in: .userDomainMask
).first else {
throw PowerSyncAttachmentError.attachmentError("Could not determine attachments directory path")
}
return documentsURL.appendingPathComponent("attachments").path
}
let localStorage = FileManagerStorageAdapter()
// Remote storage adapter (example with signed URLs)
class SignedUrlStorageAdapter: RemoteStorageAdapter {
func uploadFile(fileData: Data, attachment: Attachment) async throws {
// Request signed upload URL from your backend
struct UploadUrlResponse: Codable {
let uploadUrl: String
}
let requestBody = [
"filename": attachment.filename,
"contentType": attachment.mediaType ?? "application/octet-stream"
]
var request = URLRequest(url: URL(string: "/api/attachments/upload-url")!)
request.httpMethod = "POST"
request.setValue("application/json", forHTTPHeaderField: "Content-Type")
request.httpBody = try JSONSerialization.data(withJSONObject: requestBody)
let (data, _) = try await URLSession.shared.data(for: request)
let response = try JSONDecoder().decode(UploadUrlResponse.self, from: data)
// Upload to cloud storage using signed URL
var uploadRequest = URLRequest(url: URL(string: response.uploadUrl)!)
uploadRequest.httpMethod = "PUT"
uploadRequest.setValue(attachment.mediaType ?? "application/octet-stream", forHTTPHeaderField: "Content-Type")
uploadRequest.httpBody = fileData
let (_, uploadResponse) = try await URLSession.shared.data(for: uploadRequest)
guard let httpResponse = uploadResponse as? HTTPURLResponse,
(200...299).contains(httpResponse.statusCode) else {
throw PowerSyncAttachmentError.generalError("Upload failed")
}
}
func downloadFile(attachment: Attachment) async throws -> Data {
// Request signed download URL from your backend
struct DownloadUrlResponse: Codable {
let downloadUrl: String
}
let request = URLRequest(url: URL(string: "/api/attachments/\(attachment.id)/download-url")!)
let (data, _) = try await URLSession.shared.data(for: request)
let response = try JSONDecoder().decode(DownloadUrlResponse.self, from: data)
// Download from cloud storage
let downloadRequest = URLRequest(url: URL(string: response.downloadUrl)!)
let (fileData, _) = try await URLSession.shared.data(for: downloadRequest)
return fileData
}
func deleteFile(attachment: Attachment) async throws {
// Delete via your backend
var request = URLRequest(url: URL(string: "/api/attachments/\(attachment.id)")!)
request.httpMethod = "DELETE"
let (_, response) = try await URLSession.shared.data(for: request)
guard let httpResponse = response as? HTTPURLResponse,
(200...299).contains(httpResponse.statusCode) else {
throw PowerSyncAttachmentError.generalError("Delete failed")
}
}
}
let remoteStorage = SignedUrlStorageAdapter()
```
```kotlin Kotlin theme={null}
import com.powersync.attachments.LocalStorage
import com.powersync.attachments.RemoteStorage
import com.powersync.attachments.Attachment
import com.powersync.attachments.storage.IOLocalStorageAdapter
import kotlinx.coroutines.flow.Flow
import kotlinx.coroutines.flow.flowOf
import kotlinx.io.files.Path
// For local storage (uses IOLocalStorageAdapter by default)
// On Android: "${applicationContext.filesDir.canonicalPath}/attachments"
val attachmentsDirectory = Path("attachments").toString()
val localStorage: LocalStorage = IOLocalStorageAdapter()
// Remote storage adapter (example with signed URLs)
val remoteStorage = object : RemoteStorage {
override suspend fun uploadFile(
fileData: Flow,
attachment: Attachment
) {
// Request signed upload URL from your backend
val uploadUrl = // ... fetch from your API
// Upload to cloud storage using signed URL
// Collect the flow and upload
val bytes = mutableListOf()
fileData.collect { bytes.add(it) }
val allBytes = bytes.flatMap { it.toList() }.toByteArray()
// Upload allBytes to uploadUrl
// ... your HTTP upload implementation
}
override suspend fun downloadFile(attachment: Attachment): Flow {
// Request signed download URL from your backend
val downloadUrl = // ... fetch from your API
// Download from cloud storage
val response = // ... your HTTP download implementation
return flowOf(response) // or convert your ByteArray to Flow
}
override suspend fun deleteFile(attachment: Attachment) {
// Delete via your backend
// ... your HTTP delete implementation
}
}
```
```csharp .NET theme={null}
using PowerSync.Common.Attachments;
// Local storage backed by System.IO.File — works on Windows, macOS, Linux, iOS, and Android
var localStorage = new FileManagerLocalStorage(
Path.Combine(Environment.GetFolderPath(Environment.SpecialFolder.ApplicationData), "myapp", "attachments"));
// Remote storage adapter (example with signed URLs)
public sealed class MyRemoteStorage(HttpClient http) : IRemoteStorageAdapter
{
public async Task UploadFileAsync(Stream fileData, Attachment attachment)
{
// Request a signed upload URL from your backend
var resp = await http.PostAsJsonAsync("/api/attachments/upload-url",
new { filename = attachment.Filename });
var body = await resp.Content.ReadFromJsonAsync()
?? throw new InvalidOperationException("missing body");
// Upload directly to cloud storage using the signed URL
using var content = new StreamContent(fileData);
content.Headers.ContentType = new(attachment.MediaType ?? "application/octet-stream");
(await http.PutAsync(body.UploadUrl, content)).EnsureSuccessStatusCode();
}
public async Task DownloadFileAsync(Attachment attachment)
{
// Request a signed download URL from your backend
var url = await http.GetStringAsync($"/api/attachments/download-url/{attachment.Id}");
return await http.GetStreamAsync(url);
}
public async Task DeleteFileAsync(Attachment attachment)
{
// Delete via your backend; 404 is treated as success (file already gone)
var resp = await http.DeleteAsync($"/api/attachments/{attachment.Id}");
if (resp.StatusCode != HttpStatusCode.NotFound)
resp.EnsureSuccessStatusCode();
}
private sealed record UploadUrlResponse(string UploadUrl);
}
```
**Security Best Practice:** Always use your backend to generate signed URLs and validate permissions. Never expose storage credentials directly to clients.
### Initialize Attachment Queue
```typescript JavaScript/TypeScript theme={null}
import { AttachmentQueue } from '@powersync/web';
const attachmentQueue = new AttachmentQueue({
db: db, // PowerSync database instance
localStorage,
remoteStorage,
// Define which attachments exist in your data model
watchAttachments: (onUpdate) => {
db.watch(
`SELECT photo_id FROM users WHERE photo_id IS NOT NULL`,
[],
{
onResult: async (result) => {
const attachments = result.rows?._array.map(row => ({
id: row.photo_id,
// Names the file in local storage (.jpg). Either fileExtension or filename is required
fileExtension: 'jpg',
// MIME type for remote transfer (the Content-Type). Optional, never inferred
// from fileExtension, and preserved on the downloaded record (JS SDK only)
mediaType: 'image/jpeg'
})) ?? [];
await onUpdate(attachments);
}
}
);
},
// Optional configuration
syncIntervalMs: 30000, // Sync every 30 seconds
downloadAttachments: true, // Auto-download referenced files
archivedCacheLimit: 100 // Keep 100 archived files before cleanup
});
// Start the sync process
await attachmentQueue.startSync();
```
```dart Flutter theme={null}
import 'package:logging/logging.dart';
import 'package:powersync/powersync.dart';
import 'package:powersync_core/attachments/attachments.dart';
final logger = Logger('AttachmentQueue');
late AttachmentQueue attachmentQueue;
Future initializeAttachmentQueue(PowerSyncDatabase db) async {
attachmentQueue = AttachmentQueue(
db: db,
remoteStorage: SignedUrlStorageAdapter(),
localStorage: await getLocalStorage(),
// Define which attachments exist in your data model
watchAttachments: () => db.watch('''
SELECT photo_id as id
FROM users
WHERE photo_id IS NOT NULL
''').map(
(results) => [
for (final row in results)
WatchedAttachmentItem(
id: row['id'] as String,
fileExtension: 'jpg',
)
],
),
// Optional configuration
syncInterval: const Duration(seconds: 30), // Sync every 30 seconds
downloadAttachments: true, // Auto-download referenced files
archivedCacheLimit: 100, // Keep 100 archived files before cleanup
logger: logger,
);
// Start the sync process
await attachmentQueue.startSync();
}
```
```swift Swift theme={null}
let attachmentQueue = AttachmentQueue(
db: db, // PowerSync database instance
remoteStorage: remoteStorage,
attachmentsDirectory: try getAttachmentsDirectoryPath(),
// Define which attachments exist in your data model
watchAttachments: {
try db.watch(
sql: """
SELECT photo_id
FROM users
WHERE photo_id IS NOT NULL
""",
parameters: [],
mapper: { cursor in
try WatchedAttachmentItem(
id: cursor.getString(name: "photo_id"),
fileExtension: "jpg"
)
}
)
},
// Optional configuration
syncInterval: 30.0, // Sync every 30 seconds
downloadAttachments: true, // Auto-download referenced files
archivedCacheLimit: 100 // Keep 100 archived files before cleanup
)
// Start the sync process
try await attachmentQueue.startSync()
```
```kotlin Kotlin theme={null}
import com.powersync.attachments.AttachmentQueue
import com.powersync.attachments.WatchedAttachmentItem
import com.powersync.db.getString
import kotlinx.coroutines.flow.Flow
import kotlin.time.Duration.Companion.seconds
val attachmentQueue = AttachmentQueue(
db = db, // PowerSync database instance
remoteStorage = remoteStorage,
attachmentsDirectory = attachmentsDirectory,
localStorage = localStorage, // Optional, defaults to IOLocalStorageAdapter()
// Define which attachments exist in your data model
watchAttachments = {
db.watch(
sql = """
SELECT photo_id
FROM users
WHERE photo_id IS NOT NULL
""",
parameters = null
) { cursor ->
WatchedAttachmentItem(
id = cursor.getString("photo_id"),
fileExtension = "jpg"
)
}
},
// Optional configuration
syncInterval = 30.seconds, // Sync every 30 seconds
downloadAttachments = true, // Auto-download referenced files
archivedCacheLimit = 100 // Keep 100 archived files before cleanup
)
// Start the sync process
attachmentQueue.startSync()
```
```csharp .NET theme={null}
using PowerSync.Common.Attachments;
var queue = new AttachmentQueue(new AttachmentQueueOptions
{
Db = powersync,
LocalStorage = localStorage,
RemoteStorage = remoteStorage,
// Define which attachments exist in your data model
WatchAttachments = ct => WatchProfilePhotos(powersync, ct),
// Optional configuration
SyncInterval = TimeSpan.FromSeconds(30), // Retry failed transfers every 30 seconds
DownloadAttachments = true, // Auto-download referenced files
ArchivedCacheLimit = 100, // Keep 100 archived files before cleanup
});
// Start automatic syncing
await queue.StartSyncAsync();
static async IAsyncEnumerable WatchProfilePhotos(
PowerSyncDatabase db,
[EnumeratorCancellation] CancellationToken ct)
{
var stream = db.Watch(
"SELECT photo_id FROM users WHERE photo_id IS NOT NULL",
null,
new SQLWatchOptions { TriggerImmediately = true, Signal = ct });
await foreach (var rows in stream.WithCancellation(ct))
yield return [.. rows.Select(r => new WatchedAttachmentItem(r.photo_id, fileExtension: "jpg"))];
}
internal sealed class UserPhotoRow { public string photo_id { get; set; } = ""; }
```
The `watchAttachments` callback is crucial - it tells the queue which files your app needs based on your data model. The queue uses this to automatically download, upload, or archive files.
#### Watching Multiple Attachment Types
When watching multiple attachment types, you need to provide the `fileExtension` for each attachment. You can store this in your data model tables or derive it from other fields.
**Single Queue with UNION ALL**
Combining queries with `UNION ALL` lets one queue watch attachments across different tables. Use `UNION ALL` rather than `UNION`: attachment IDs should already be unique, so deduplication is unnecessary. The combined query executes whenever any of the watched tables change, which may have higher database overhead than watching a single table.
```typescript JavaScript/TypeScript theme={null}
// Example: Watching users.photo_id, documents.document_id, and videos.video_id
// Assuming your tables store file extensions
const attachmentQueue = new AttachmentQueue({
db: db,
localStorage,
remoteStorage,
watchAttachments: (onUpdate) => {
db.watch(
`SELECT photo_id as id, photo_file_extension as file_extension
FROM users
WHERE photo_id IS NOT NULL
UNION ALL
SELECT document_id as id, document_file_extension as file_extension
FROM documents
WHERE document_id IS NOT NULL
UNION ALL
SELECT video_id as id, video_file_extension as file_extension
FROM videos
WHERE video_id IS NOT NULL`,
[],
{
onResult: async (result) => {
const attachments = result.rows?._array.map(row => ({
id: row.id,
fileExtension: row.file_extension
})) ?? [];
await onUpdate(attachments);
}
}
);
},
// ... other options
});
await attachmentQueue.startSync();
```
```dart Flutter theme={null}
// Example: Watching users.photo_id, documents.document_id, and videos.video_id
// Assuming your tables store file extensions
attachmentQueue = AttachmentQueue(
db: db,
remoteStorage: SignedUrlStorageAdapter(),
localStorage: await getLocalStorage(),
watchAttachments: () => db.watch('''
SELECT photo_id as id, photo_file_extension as file_extension
FROM users
WHERE photo_id IS NOT NULL
UNION ALL
SELECT document_id as id, document_file_extension as file_extension
FROM documents
WHERE document_id IS NOT NULL
UNION ALL
SELECT video_id as id, video_file_extension as file_extension
FROM videos
WHERE video_id IS NOT NULL
''').map(
(results) => [
for (final row in results)
WatchedAttachmentItem(
id: row['id'] as String,
fileExtension: row['file_extension'] as String,
)
],
),
// ... other options
);
await attachmentQueue.startSync();
```
```swift Swift theme={null}
// Example: Watching users.photo_id, documents.document_id, and videos.video_id
// Assuming your tables store file extensions
let attachmentQueue = AttachmentQueue(
db: db,
remoteStorage: remoteStorage,
attachmentsDirectory: try getAttachmentsDirectoryPath(),
watchAttachments: {
try db.watch(
sql: """
SELECT photo_id as id, photo_file_extension as file_extension
FROM users
WHERE photo_id IS NOT NULL
UNION ALL
SELECT document_id as id, document_file_extension as file_extension
FROM documents
WHERE document_id IS NOT NULL
UNION ALL
SELECT video_id as id, video_file_extension as file_extension
FROM videos
WHERE video_id IS NOT NULL
""",
parameters: [],
mapper: { cursor in
try WatchedAttachmentItem(
id: cursor.getString(name: "id"),
fileExtension: cursor.getString(name: "file_extension")
)
}
)
},
// ... other options
)
try await attachmentQueue.startSync()
```
```kotlin Kotlin theme={null}
// Example: Watching users.photo_id, documents.document_id, and videos.video_id
// Assuming your tables store file extensions
val attachmentQueue = AttachmentQueue(
db = db,
remoteStorage = remoteStorage,
attachmentsDirectory = attachmentsDirectory,
localStorage = localStorage,
watchAttachments = {
db.watch(
sql = """
SELECT photo_id as id, photo_file_extension as file_extension
FROM users
WHERE photo_id IS NOT NULL
UNION ALL
SELECT document_id as id, document_file_extension as file_extension
FROM documents
WHERE document_id IS NOT NULL
UNION ALL
SELECT video_id as id, video_file_extension as file_extension
FROM videos
WHERE video_id IS NOT NULL
""",
parameters = null
) { cursor ->
WatchedAttachmentItem(
id = cursor.getString("id"),
fileExtension = cursor.getString("file_extension")
)
}
},
// ... other options
)
attachmentQueue.startSync()
```
```csharp .NET theme={null}
// Example: watching users.photo_id, documents.document_id, and videos.video_id
// Assuming your tables store file extensions
var queue = new AttachmentQueue(new AttachmentQueueOptions
{
Db = powersync,
LocalStorage = localStorage,
RemoteStorage = remoteStorage,
WatchAttachments = ct => WatchAllAttachments(powersync, ct),
// ... other options
});
await queue.StartSyncAsync();
static async IAsyncEnumerable WatchAllAttachments(
PowerSyncDatabase db,
[EnumeratorCancellation] CancellationToken ct)
{
var stream = db.Watch(
"""
SELECT photo_id AS id, photo_file_extension AS file_extension
FROM users
WHERE photo_id IS NOT NULL
UNION ALL
SELECT document_id AS id, document_file_extension AS file_extension
FROM documents
WHERE document_id IS NOT NULL
UNION ALL
SELECT video_id AS id, video_file_extension AS file_extension
FROM videos
WHERE video_id IS NOT NULL
""",
null,
new SQLWatchOptions { TriggerImmediately = true, Signal = ct });
await foreach (var rows in stream.WithCancellation(ct))
yield return [.. rows.Select(r => new WatchedAttachmentItem(r.id, fileExtension: r.file_extension))];
}
internal sealed class AttachmentRef
{
public string id { get; set; } = "";
public string file_extension { get; set; } = "";
}
```
**Multiple Queues**
Alternatively, create separate queues for different attachment types. Each queue watches its own table(s) with a simpler query, allowing independent configuration and management, at the cost of some extra memory.
```typescript JavaScript/TypeScript theme={null}
// Create separate queues for different attachment types
const photoQueue = new AttachmentQueue({
db: db,
localStorage,
remoteStorage,
watchAttachments: (onUpdate) => {
db.watch(
`SELECT photo_id FROM users WHERE photo_id IS NOT NULL`,
[],
{
onResult: async (result) => {
const attachments = result.rows?._array.map(row => ({
id: row.photo_id,
fileExtension: 'jpg',
mediaType: 'image/jpeg' // Optional: preserved on the downloaded record
})) ?? [];
await onUpdate(attachments);
}
}
);
},
});
const documentQueue = new AttachmentQueue({
db: db,
localStorage,
remoteStorage,
watchAttachments: (onUpdate) => {
db.watch(
`SELECT document_id FROM documents WHERE document_id IS NOT NULL`,
[],
{
onResult: async (result) => {
const attachments = result.rows?._array.map(row => ({
id: row.document_id,
fileExtension: 'pdf',
mediaType: 'application/pdf' // Optional: preserved on the downloaded record
})) ?? [];
await onUpdate(attachments);
}
}
);
},
});
await Promise.all([
photoQueue.startSync(),
documentQueue.startSync()
]);
```
```dart Flutter theme={null}
// Create separate queues for different attachment types
final photoQueue = AttachmentQueue(
db: db,
remoteStorage: SignedUrlStorageAdapter(),
localStorage: await getLocalStorage(),
watchAttachments: () => db.watch('''
SELECT photo_id as id
FROM users
WHERE photo_id IS NOT NULL
''').map(
(results) => [
for (final row in results)
WatchedAttachmentItem(
id: row['id'] as String,
fileExtension: 'jpg',
)
],
),
);
final documentQueue = AttachmentQueue(
db: db,
remoteStorage: SignedUrlStorageAdapter(),
localStorage: await getLocalStorage(),
watchAttachments: () => db.watch('''
SELECT document_id as id
FROM documents
WHERE document_id IS NOT NULL
''').map(
(results) => [
for (final row in results)
WatchedAttachmentItem(
id: row['id'] as String,
fileExtension: 'pdf',
)
],
),
);
await Future.wait([
photoQueue.startSync(),
documentQueue.startSync(),
]);
```
```swift Swift theme={null}
// Create separate queues for different attachment types
let photoQueue = AttachmentQueue(
db: db,
remoteStorage: remoteStorage,
attachmentsDirectory: try getAttachmentsDirectoryPath(),
watchAttachments: {
try db.watch(
sql: """
SELECT photo_id
FROM users
WHERE photo_id IS NOT NULL
""",
parameters: [],
mapper: { cursor in
try WatchedAttachmentItem(
id: cursor.getString(name: "photo_id"),
fileExtension: "jpg"
)
}
)
}
)
let documentQueue = AttachmentQueue(
db: db,
remoteStorage: remoteStorage,
attachmentsDirectory: try getAttachmentsDirectoryPath(),
watchAttachments: {
try db.watch(
sql: """
SELECT document_id
FROM documents
WHERE document_id IS NOT NULL
""",
parameters: [],
mapper: { cursor in
try WatchedAttachmentItem(
id: cursor.getString(name: "document_id"),
fileExtension: "pdf"
)
}
)
}
)
try await photoQueue.startSync()
try await documentQueue.startSync()
```
```kotlin Kotlin theme={null}
// Create separate queues for different attachment types
val photoQueue = AttachmentQueue(
db = db,
remoteStorage = remoteStorage,
attachmentsDirectory = attachmentsDirectory,
localStorage = localStorage,
watchAttachments = {
db.watch(
sql = """
SELECT photo_id
FROM users
WHERE photo_id IS NOT NULL
""",
parameters = null
) { cursor ->
WatchedAttachmentItem(
id = cursor.getString("photo_id"),
fileExtension = "jpg"
)
}
}
)
val documentQueue = AttachmentQueue(
db = db,
remoteStorage = remoteStorage,
attachmentsDirectory = attachmentsDirectory,
localStorage = localStorage,
watchAttachments = {
db.watch(
sql = """
SELECT document_id
FROM documents
WHERE document_id IS NOT NULL
""",
parameters = null
) { cursor ->
WatchedAttachmentItem(
id = cursor.getString("document_id"),
fileExtension = "pdf"
)
}
}
)
photoQueue.startSync()
documentQueue.startSync()
```
```csharp .NET theme={null}
// Create separate queues for different attachment types
var photoQueue = new AttachmentQueue(new AttachmentQueueOptions
{
Db = powersync,
LocalStorage = localStorage,
RemoteStorage = remoteStorage,
WatchAttachments = ct => WatchPhotos(powersync, ct),
});
var documentQueue = new AttachmentQueue(new AttachmentQueueOptions
{
Db = powersync,
LocalStorage = localStorage,
RemoteStorage = remoteStorage,
WatchAttachments = ct => WatchDocuments(powersync, ct),
});
await Task.WhenAll(
photoQueue.StartSyncAsync(),
documentQueue.StartSyncAsync());
static async IAsyncEnumerable WatchPhotos(
PowerSyncDatabase db,
[EnumeratorCancellation] CancellationToken ct)
{
var stream = db.Watch(
"SELECT photo_id FROM users WHERE photo_id IS NOT NULL",
null,
new SQLWatchOptions { TriggerImmediately = true, Signal = ct });
await foreach (var rows in stream.WithCancellation(ct))
yield return [.. rows.Select(r => new WatchedAttachmentItem(r.photo_id, fileExtension: "jpg"))];
}
static async IAsyncEnumerable WatchDocuments(
PowerSyncDatabase db,
[EnumeratorCancellation] CancellationToken ct)
{
var stream = db.Watch(
"SELECT document_id FROM documents WHERE document_id IS NOT NULL",
null,
new SQLWatchOptions { TriggerImmediately = true, Signal = ct });
await foreach (var rows in stream.WithCancellation(ct))
yield return [.. rows.Select(r => new WatchedAttachmentItem(r.document_id, fileExtension: "pdf"))];
}
internal sealed class DocumentRow { public string document_id { get; set; } = ""; }
```
### Upload an Attachment
```typescript JavaScript/TypeScript theme={null}
async function uploadProfilePhoto(imageBlob: Blob, userId: string) {
const arrayBuffer = await imageBlob.arrayBuffer();
const attachment = await attachmentQueue.saveFile({
data: arrayBuffer,
fileExtension: 'jpg',
mediaType: 'image/jpeg',
// updateHook runs in same transaction, ensuring atomicity
updateHook: async (tx, attachment) => {
await tx.execute(
'UPDATE users SET photo_id = ? WHERE id = ?',
[attachment.id, userId]
);
}
});
return attachment;
}
// The queue will:
// 1. Save file locally immediately
// 2. Create attachment record with state QUEUED_UPLOAD
// 3. Update user record in same transaction
// 4. Automatically upload file in background
// 5. Update state to SYNCED when complete
```
```dart Flutter theme={null}
import 'dart:io';
import 'dart:typed_data';
import 'package:powersync_core/attachments/attachments.dart';
Future uploadProfilePhoto(
File imageFile,
String userId,
) async {
final imageBytes = await imageFile.readAsBytes();
final attachment = await attachmentQueue.saveFile(
data: Stream.value(imageBytes),
mediaType: 'image/jpeg',
fileExtension: 'jpg',
// updateHook runs in same transaction, ensuring atomicity
updateHook: (context, attachment) async {
await context.execute(
'UPDATE users SET photo_id = ? WHERE id = ?',
[attachment.id, userId],
);
},
);
return attachment;
}
// The queue will:
// 1. Save file locally immediately
// 2. Create attachment record with state QUEUED_UPLOAD
// 3. Update user record in same transaction
// 4. Automatically upload file in background
// 5. Update state to SYNCED when complete
```
```swift Swift theme={null}
func uploadProfilePhoto(imageData: Data, userId: String) async throws -> Attachment {
let attachment = try await attachmentQueue.saveFile(
data: imageData,
mediaType: "image/jpeg",
fileExtension: "jpg",
// updateHook runs in same transaction, ensuring atomicity
updateHook: { tx, attachment in
try tx.execute(
sql: "UPDATE users SET photo_id = ? WHERE id = ?",
parameters: [attachment.id, userId]
)
}
)
return attachment
}
// The queue will:
// 1. Save file locally immediately
// 2. Create attachment record with state QUEUED_UPLOAD
// 3. Update user record in same transaction
// 4. Automatically upload file in background
// 5. Update state to SYNCED when complete
```
```kotlin Kotlin theme={null}
import kotlinx.coroutines.flow.flowOf
suspend fun uploadProfilePhoto(imageBytes: ByteArray, userId: String) {
val attachment = attachmentQueue.saveFile(
data = flowOf(imageBytes),
mediaType = "image/jpeg",
fileExtension = "jpg",
// updateHook runs in same transaction, ensuring atomicity
updateHook = { tx, attachment ->
tx.execute(
"UPDATE users SET photo_id = ? WHERE id = ?",
listOf(attachment.id, userId)
)
}
)
return attachment
}
// The queue will:
// 1. Save file locally immediately
// 2. Create attachment record with state QUEUED_UPLOAD
// 3. Update user record in same transaction
// 4. Automatically upload file in background
// 5. Update state to SYNCED when complete
```
```csharp .NET theme={null}
async Task UploadProfilePhotoAsync(Stream imageStream, string currentUserId)
{
var attachment = await queue.SaveFileAsync(
data: imageStream,
fileExtension: "jpg",
mediaType: "image/jpeg",
// updateHook runs in the same transaction, ensuring atomicity
updateHook: async (tx, attachment) =>
{
await tx.Execute(
"UPDATE users SET photo_id = ? WHERE id = ?",
[attachment.Id, currentUserId]);
});
// The queue will:
// 1. Save the file locally immediately
// 2. Create an attachment record with state QueuedUpload
// 3. Update the user record in the same transaction
// 4. Automatically upload the file in the background
// 5. Update the state to Synced when complete
}
```
The `updateHook` parameter is the recommended way to link attachments to your data model. It runs in the same database transaction, ensuring data consistency.
On React Native and Node.js, files already on disk (such as camera captures or recordings) can be queued without reading them into memory; see [Saving Files Already on Disk](#saving-files-already-on-disk).
### Download/Access an Attachment
```typescript JavaScript/TypeScript theme={null}
import { AttachmentState } from '@powersync/web';
// Downloads happen automatically when watchAttachments references a file
async function getProfilePhotoUri(userId: string): Promise {
const user = await db.get(
'SELECT photo_id FROM users WHERE id = ?',
[userId]
);
if (!user?.photo_id) {
return null;
}
const attachment = await db.get(
'SELECT * FROM attachments WHERE id = ?',
[user.photo_id]
);
if (!attachment) {
return null;
}
if (attachment.state === AttachmentState.SYNCED && attachment.local_uri) {
return attachment.local_uri;
}
return null;
}
// Example: display the image in React on web. On web, local_uri is an
// internal indexeddb:// reference, not a URL the browser can load, so
// passing it straight to (null);
useEffect(() => {
let objectUrl: string | null = null;
const watch = db.watch(
`SELECT a.local_uri, a.media_type, a.state
FROM users u
LEFT JOIN attachments a ON a.id = u.photo_id
WHERE u.id = ?`,
[userId],
{
onResult: async (result) => {
const row = result.rows?._array[0];
if (row?.state === AttachmentState.SYNCED && row?.local_uri) {
const buffer = await localStorage.readFile(row.local_uri);
const nextUrl = URL.createObjectURL(
new Blob([buffer], { type: row.media_type ?? 'image/jpeg' })
);
if (objectUrl) {
URL.revokeObjectURL(objectUrl);
}
objectUrl = nextUrl;
setPhotoUrl(nextUrl);
}
}
}
);
return () => {
watch.close();
if (objectUrl) {
URL.revokeObjectURL(objectUrl);
}
};
}, [userId]);
if (!photoUrl) {
return Loading photo...
;
}
return getProfilePhotoUri(
PowerSyncDatabase db,
String userId,
) async {
final user = await db.get(
'SELECT photo_id FROM users WHERE id = ?',
[userId],
);
if (user == null || user['photo_id'] == null) {
return null;
}
final attachment = await db.get(
'SELECT * FROM attachments_queue WHERE id = ?',
[user['photo_id']],
);
if (attachment == null) {
return null;
}
final state = AttachmentState.fromInt(attachment['state'] as int);
final localUri = attachment['local_uri'] as String?;
if (state == AttachmentState.synced && localUri != null) {
// Resolve full path from local storage
final appDocDir = await getApplicationDocumentsDirectory();
return '${appDocDir.path}/attachments/$localUri';
}
return null;
}
// Example: Display image in Flutter with StreamBuilder
StreamBuilder>>(
stream: db.watch('''
SELECT a.local_uri, a.state
FROM users u
LEFT JOIN attachments_queue a ON a.id = u.photo_id
WHERE u.id = ?
''').map((results) => results.toList()),
builder: (context, snapshot) {
if (!snapshot.hasData || snapshot.data!.isEmpty) {
return const CircularProgressIndicator();
}
final row = snapshot.data!.first;
final state = AttachmentState.fromInt(row['state'] as int);
final localUri = row['local_uri'] as String?;
if (state == AttachmentState.synced && localUri != null) {
// Load and display image
return Image.file(File(localUri));
}
return const Text('Loading photo...');
},
)
```
```swift Swift theme={null}
// Downloads happen automatically when watchAttachments references a file
func getProfilePhotoUri(userId: String) async throws -> String? {
guard let user = try await db.getOptional(
sql: "SELECT photo_id FROM users WHERE id = ?",
parameters: [userId],
mapper: { cursor in
try cursor.getStringOptional(name: "photo_id")
}
), let photoId = user else {
return nil
}
guard let attachment = try await db.getOptional(
sql: "SELECT * FROM attachments WHERE id = ?",
parameters: [photoId],
mapper: { cursor in
try Attachment.fromCursor(cursor)
}
) else {
return nil
}
if attachment.state == .synced, let localUri = attachment.localUri {
return localUri
}
return nil
}
// Example: Display image in SwiftUI with watch query
struct ProfilePhotoView: View {
let userId: String
@State private var photoUri: String?
var body: some View {
Group {
if let photoUri = photoUri {
AsyncImage(url: URL(fileURLWithPath: photoUri)) { image in
image.resizable()
} placeholder: {
ProgressView()
}
} else {
Text("Loading photo...")
}
}
.task {
do {
for try await results in try db.watch(
sql: """
SELECT a.local_uri, a.state
FROM users u
LEFT JOIN attachments a ON a.id = u.photo_id
WHERE u.id = ?
""",
parameters: [userId],
mapper: { cursor in
(
state: try AttachmentState.from(cursor.getInt(name: "state")),
localUri: try cursor.getStringOptional(name: "local_uri")
)
}
) {
if let first = results.first,
first.state == .synced,
let localUri = first.localUri {
photoUri = localUri
}
}
} catch {
print("Error watching photo: \(error)")
}
}
}
}
```
```kotlin Kotlin theme={null}
import com.powersync.attachments.AttachmentState
import com.powersync.db.getString
import com.powersync.db.getStringOptional
import kotlinx.coroutines.flow.Flow
import kotlinx.coroutines.flow.map
// Downloads happen automatically when watchAttachments references a file
suspend fun getProfilePhotoUri(userId: String): String? {
val user = db.get(
"SELECT photo_id FROM users WHERE id = ?",
listOf(userId)
) { cursor ->
cursor.getStringOptional("photo_id")
}
if (user == null) {
return null
}
val attachment = db.get(
"SELECT * FROM attachments WHERE id = ?",
listOf(user)
) { cursor ->
com.powersync.attachments.Attachment.fromCursor(cursor)
}
if (attachment == null) {
return null
}
if (attachment.state == AttachmentState.SYNCED && attachment.localUri != null) {
return attachment.localUri
}
return null
}
// Example: Watch attachment state in Compose/UI
fun watchProfilePhoto(userId: String): Flow {
return db.watch(
sql = """
SELECT a.local_uri, a.state
FROM users u
LEFT JOIN attachments a ON a.id = u.photo_id
WHERE u.id = ?
""",
parameters = listOf(userId)
) { cursor ->
val state = AttachmentState.fromLong(cursor.getLong("state"))
val localUri = cursor.getStringOptional("local_uri")
if (state == AttachmentState.SYNCED && localUri != null) {
localUri
} else {
null
}
}.map { results -> results.firstOrNull() }
}
```
```csharp .NET theme={null}
// Downloads happen automatically when WatchAttachments references a file
// Reactively yield the local file path whenever the attachment reaches the Synced state
static async IAsyncEnumerable WatchProfilePhotoUri(
PowerSyncDatabase db,
string userId,
[EnumeratorCancellation] CancellationToken ct = default)
{
var stream = db.Watch(
"""
SELECT a.local_uri, a.state
FROM users u
LEFT JOIN attachments a ON a.id = u.photo_id
WHERE u.id = ?
""",
[userId],
new SQLWatchOptions { TriggerImmediately = true, Signal = ct });
await foreach (var rows in stream.WithCancellation(ct))
{
var row = rows.FirstOrDefault();
yield return row?.state == (long)AttachmentState.Synced ? row.local_uri : null;
}
}
internal sealed class PhotoState
{
public string? local_uri { get; set; }
public long state { get; set; }
}
```
### Delete an Attachment
```typescript JavaScript/TypeScript theme={null}
async function deleteProfilePhoto(userId: string, photoId: string) {
await attachmentQueue.deleteFile({
id: photoId,
// updateHook ensures atomic deletion
updateHook: async (tx, attachment) => {
await tx.execute(
'UPDATE users SET photo_id = NULL WHERE id = ?',
[userId]
);
}
});
console.log('Photo queued for deletion');
// The queue will:
// 1. Delete from remote storage
// 2. Delete local file
// 3. Remove attachment record
}
// Alternative: Remove reference and let queue archive it automatically
async function removePhotoReference(userId: string) {
await db.execute(
'UPDATE users SET photo_id = NULL WHERE id = ?',
[userId]
);
// The watchAttachments callback will detect this change
// The queue will automatically archive the unreferenced attachment
// After reaching archivedCacheLimit, it will be deleted
}
```
```dart Flutter theme={null}
Future deleteProfilePhoto(
String userId,
String photoId,
) async {
await attachmentQueue.deleteFile(
attachmentId: photoId,
// updateHook ensures atomic deletion
updateHook: (context, attachment) async {
await context.execute(
'UPDATE users SET photo_id = NULL WHERE id = ?',
[userId],
);
},
);
print('Photo queued for deletion');
// The queue will:
// 1. Delete from remote storage
// 2. Delete local file
// 3. Remove attachment record
}
// Alternative: Remove reference and let queue archive it automatically
Future removePhotoReference(
PowerSyncDatabase db,
String userId,
) async {
await db.execute(
'UPDATE users SET photo_id = NULL WHERE id = ?',
[userId],
);
// The watchAttachments callback will detect this change
// The queue will automatically archive the unreferenced attachment
// After reaching archivedCacheLimit, it will be deleted
}
```
```swift Swift theme={null}
func deleteProfilePhoto(userId: String, photoId: String) async throws {
try await attachmentQueue.deleteFile(
attachmentId: photoId,
// updateHook ensures atomic deletion
updateHook: { tx, attachment in
try tx.execute(
sql: "UPDATE users SET photo_id = NULL WHERE id = ?",
parameters: [userId]
)
}
)
print("Photo queued for deletion")
// The queue will:
// 1. Delete from remote storage
// 2. Delete local file
// 3. Remove attachment record
}
// Alternative: Remove reference and let queue archive it automatically
func removePhotoReference(userId: String) async throws {
try await db.execute(
sql: "UPDATE users SET photo_id = NULL WHERE id = ?",
parameters: [userId]
)
// The watchAttachments callback will detect this change
// The queue will automatically archive the unreferenced attachment
// After reaching archivedCacheLimit, it will be deleted
}
```
```kotlin Kotlin theme={null}
suspend fun deleteProfilePhoto(userId: String, photoId: String) {
attachmentQueue.deleteFile(
attachmentId = photoId,
// updateHook ensures atomic deletion
updateHook = { tx, attachment ->
tx.execute(
"UPDATE users SET photo_id = NULL WHERE id = ?",
listOf(userId)
)
}
)
// The queue will:
// 1. Delete from remote storage
// 2. Delete local file
// 3. Remove attachment record
}
// Alternative: Remove reference and let queue archive it automatically
suspend fun removePhotoReference(userId: String) {
db.writeTransaction { tx ->
tx.execute(
"UPDATE users SET photo_id = NULL WHERE id = ?",
listOf(userId)
)
}
// The watchAttachments callback will detect this change
// The queue will automatically archive the unreferenced attachment
// After reaching archivedCacheLimit, it will be deleted
}
```
```csharp .NET theme={null}
async Task DeleteProfilePhotoAsync(string userId, string photoId)
{
await queue.DeleteFileAsync(
id: photoId,
// updateHook ensures the data model update and deletion are atomic
updateHook: async (tx, attachment) =>
{
await tx.Execute(
"UPDATE users SET photo_id = NULL WHERE id = ?",
[userId]);
});
// The queue will:
// 1. Delete from remote storage
// 2. Delete the local file
// 3. Remove the attachment record
}
// Alternative: remove the reference and let the queue archive it automatically
async Task RemovePhotoReferenceAsync(string userId)
{
await powersync.Execute(
"UPDATE users SET photo_id = NULL WHERE id = ?",
[userId]);
// The WatchAttachments callback detects this change.
// The queue archives the unreferenced attachment automatically.
// Once the cache limit is reached, archived attachments are deleted.
}
```
## Advanced Topics
### Error Handling
Implement custom error handling to control retry behavior:
```typescript JavaScript/TypeScript theme={null}
import { AttachmentErrorHandler } from '@powersync/web';
const errorHandler: AttachmentErrorHandler = {
async onDownloadError(attachment, error) {
console.error(`Download failed: ${attachment.filename}`, error);
// Return true to retry, false to archive
if (error.message.includes('404')) {
return false; // File doesn't exist, don't retry
}
return true; // Retry on network errors
},
async onUploadError(attachment, error) {
console.error(`Upload failed: ${attachment.filename}`, error);
return true; // Always retry uploads
},
async onDeleteError(attachment, error) {
console.error(`Delete failed: ${attachment.filename}`, error);
return true; // Retry deletes
}
};
const queue = new AttachmentQueue({
// ... other options
errorHandler
});
```
```dart Flutter theme={null}
import 'package:powersync_core/attachments/attachments.dart';
final errorHandler = AttachmentErrorHandler(
onDownloadError: (attachment, exception, stackTrace) async {
print('Download failed: ${attachment.filename}');
print('Error: $exception');
// Return true to retry, false to archive
if (exception.toString().contains('404')) {
return false; // File doesn't exist, don't retry
}
return true; // Retry on network errors
},
onUploadError: (attachment, exception, stackTrace) async {
print('Upload failed: ${attachment.filename}');
print('Error: $exception');
return true; // Always retry uploads
},
onDeleteError: (attachment, exception, stackTrace) async {
print('Delete failed: ${attachment.filename}');
print('Error: $exception');
return true; // Retry deletes
},
);
final queue = AttachmentQueue(
// ... other options
errorHandler: errorHandler,
);
```
```swift Swift theme={null}
class CustomErrorHandler: SyncErrorHandler {
func onDownloadError(attachment: Attachment, error: Error) async -> Bool {
print("Download failed: \(attachment.filename), error: \(error)")
// Return true to retry, false to archive
if let urlError = error as? URLError, urlError.code == .badServerResponse {
return false // File doesn't exist (404), don't retry
}
return true // Retry on network errors
}
func onUploadError(attachment: Attachment, error: Error) async -> Bool {
print("Upload failed: \(attachment.filename), error: \(error)")
return true // Always retry uploads
}
func onDeleteError(attachment: Attachment, error: Error) async -> Bool {
print("Delete failed: \(attachment.filename), error: \(error)")
return true // Retry deletes
}
}
let queue = AttachmentQueue(
db: db,
remoteStorage: remoteStorage,
attachmentsDirectory: try getAttachmentsDirectoryPath(),
watchAttachments: watchAttachments,
errorHandler: CustomErrorHandler()
)
```
```kotlin Kotlin theme={null}
import com.powersync.attachments.SyncErrorHandler
val errorHandler = object : SyncErrorHandler {
override suspend fun onDownloadError(
attachment: Attachment,
exception: Exception
): Boolean {
println("Download failed: ${attachment.filename}", exception)
// Return true to retry, false to archive
if (exception.message?.contains("404") == true) {
return false // File doesn't exist, don't retry
}
return true // Retry on network errors
}
override suspend fun onUploadError(
attachment: Attachment,
exception: Exception
): Boolean {
println("Upload failed: ${attachment.filename}", exception)
return true // Always retry uploads
}
override suspend fun onDeleteError(
attachment: Attachment,
exception: Exception
): Boolean {
println("Delete failed: ${attachment.filename}", exception)
return true // Retry deletes
}
}
val queue = AttachmentQueue(
// ... other options
errorHandler = errorHandler
)
```
```csharp .NET theme={null}
using PowerSync.Common.Attachments;
public sealed class MyErrorHandler(ILogger logger) : IAttachmentErrorHandler
{
public Task OnDownloadErrorAsync(Attachment attachment, Exception error)
{
logger.LogError(error, "Download failed for {Filename}", attachment.Filename);
// Return true to retry, false to archive
if (error is HttpRequestException { StatusCode: HttpStatusCode.NotFound })
return Task.FromResult(false); // File no longer exists
return Task.FromResult(true); // Retry on network errors
}
public Task OnUploadErrorAsync(Attachment attachment, Exception error)
{
logger.LogError(error, "Upload failed for {Filename}", attachment.Filename);
return Task.FromResult(true); // Always retry
}
public Task OnDeleteErrorAsync(Attachment attachment, Exception error)
{
logger.LogError(error, "Delete failed for {Filename}", attachment.Filename);
return Task.FromResult(true); // Retry
}
}
var queue = new AttachmentQueue(new AttachmentQueueOptions
{
// ... other options
ErrorHandler = new MyErrorHandler(logger),
});
```
### Transferring Large Files Without Buffering
This section applies to React Native, Expo and Node.js platforms only, and requires `@powersync/react-native` v2.1.0 or `@powersync/node` v0.21.0 or later. React Native also requires `@powersync/attachments-storage-react-native` v0.1.0 or later.
In the Dart and Kotlin SDKs, the remote storage interface is already stream-based (`Stream`/`Flow`), so transfers can avoid buffering. The Swift SDK currently receives files as `Data` and has no streaming equivalent yet.
By default, the queue transfers files by buffering them through JS memory: the entire file is read into an `ArrayBuffer` before it is handed to the remote storage adapter, and the reverse for downloads. This works well for small files but limits the practical attachment size, particularly in React Native, where a large video can exhaust the JS heap on lower-end devices.
To stream instead, configure the queue with a transport adapter in place of the remote storage adapter (you provide one or the other, not both; TypeScript enforces this). A transport owns all remote operations through the three methods of the [`AttachmentTransportAdapter`](https://powersync-ja.github.io/powersync-js/common/interfaces/AttachmentTransportAdapter.html) interface:
* `upload(attachment)` - Transfer the file at `attachment.localUri` to remote storage
* `download(attachment)` - Fetch the remote file into `attachment.localUri` (the queue assigns the destination path before the call)
* `delete(attachment)` - Remove the file from remote storage
You usually don't implement these methods yourself. The streaming-capable local storage adapters each create a ready-made transport through their `createTransportAdapter` method:
* `ExpoFileSystemStorageAdapter` (`@powersync/attachments-storage-react-native`) - The transport streams with Expo's native `File.upload`/`File.downloadFileAsync`. Using the transport requires Expo 56 or later; using only the storage adapter requires Expo 54
* `ReactNativeFileSystemStorageAdapter` (`@powersync/attachments-storage-react-native`) - The transport streams with `uploadFiles`/`downloadFile` from `@dr.pogodin/react-native-fs`, uploading as a raw binary `PUT` by default
* `NodeFileSystemAdapter` (`@powersync/node`) - The transport streams with `fetch` and Node.js filesystem streams
All three take the same options. `resolveUpload` and `resolveDownload` map an attachment to the HTTP request that transfers its bytes, typically a signed URL from your backend. `deleteFile` performs the remote delete, which is a plain remote call rather than a byte transfer.
```typescript theme={null}
import { AttachmentQueue } from '@powersync/react-native';
import { ExpoFileSystemStorageAdapter } from '@powersync/attachments-storage-react-native';
const localStorage = new ExpoFileSystemStorageAdapter();
// Streams bytes natively and owns upload/download/delete. No remoteStorage needed.
const transportAdapter = localStorage.createTransportAdapter({
resolveUpload: async (attachment) => ({
url: await getSignedUploadUrl(attachment.filename), // from your backend
mimeType: attachment.mediaType ?? 'application/octet-stream'
}),
resolveDownload: async (attachment) => ({
url: await getSignedDownloadUrl(attachment.filename)
}),
deleteFile: async (attachment) => {
await deleteFromStorage(attachment.filename); // your backend or storage SDK call
}
});
const attachmentQueue = new AttachmentQueue({
db,
localStorage,
transportAdapter, // owns all remote operations; used in place of remoteStorage
watchAttachments: (onUpdate) => {
// Same as in Initialize Attachment Queue
}
});
await attachmentQueue.startSync();
```
#### Saving Files Already on Disk
This section applies to React Native, Expo and Node.js platforms only.
For files your app produces on disk (camera captures, recordings, exports), `saveFileFromUri` queues the upload without reading the file into memory. `saveFile` would read the file into an `ArrayBuffer` just to write it back to disk; `saveFileFromUri` moves it into managed storage instead. This requires a streaming-capable local storage adapter: those adapters implement the `StreamingLocalStorageAdapter` subinterface, which adds `moveFile(sourceUri, targetUri)`. Combined with a transport adapter, the file is saved and uploaded without ever passing through JS memory:
```typescript theme={null}
async function attachRecording(localUri: string, recordingId: string) {
return attachmentQueue.saveFileFromUri({
localUri, // path to the existing file
fileExtension: 'm4a',
mediaType: 'audio/mp4',
updateHook: async (tx, attachment) => {
await tx.execute(
'UPDATE recordings SET audio_id = ? WHERE id = ?',
[attachment.id, recordingId]
);
}
});
}
```
### Custom Transport Adapters
The transport API requires `@powersync/web` v2.2.0, `@powersync/react-native` v2.1.0, or `@powersync/node` v0.21.0 or later. React Native also requires `@powersync/attachments-storage-react-native` v0.1.0 or later.
In the JavaScript SDKs, you can also write your own [transport adapter](#transferring-large-files-without-buffering). A custom remote storage adapter always receives the file as one full in-memory buffer. A custom transport receives the file's path instead. This makes the following possible:
* **Buffer-free transfers** - Let a native package transfer directly between the file system and the network, bypassing JS entirely, as the built-in transports do
* **Resumable transfers** - The queue retries a failed operation by calling the transport again on the next sync interval. A transport built on a resumable protocol such as [tus](https://tus.io) or S3 multipart upload can continue from the last confirmed offset instead of restarting from zero. Downloads can resume a partial file with HTTP `Range` requests
* **Encryption** - Encrypt files before upload and decrypt them after download for end-to-end encrypted attachments, without holding the whole file in memory
To build your own transport, implement the [`AttachmentTransportAdapter`](https://powersync-ja.github.io/powersync-js/common/interfaces/AttachmentTransportAdapter.html) interface. It has three methods: `upload(attachment)`, `download(attachment)`, and `delete(attachment)`. For a working reference, see the built-in [`NodeFileSystemTransportAdapter`](https://github.com/powersync-ja/powersync-js/blob/main/packages/node/src/attachments/NodeFileSystemTransportAdapter.ts), which streams with `fetch` and Node.js filesystem streams.
The queue retries failed operations on the next sync interval, subject to your [error handler](#error-handling).
### Custom Storage Adapters
The following is an example of how to implement a custom storage adapter for IPFS:
```typescript JavaScript/TypeScript theme={null}
import { LocalStorageAdapter, RemoteStorageAdapter } from '@powersync/web';
// Example: IPFS remote storage
class IPFSStorageAdapter implements RemoteStorageAdapter {
async uploadFile(fileData: ArrayBuffer, attachment: AttachmentRecord) {
// Upload to IPFS
const cid = await ipfs.add(fileData);
// Store CID in your backend for retrieval
await fetch('/api/ipfs-cids', {
method: 'POST',
body: JSON.stringify({ attachmentId: attachment.id, cid })
});
}
async downloadFile(attachment: AttachmentRecord): Promise {
// Retrieve CID from backend
const { cid } = await fetch(`/api/ipfs-cids/${attachment.id}`)
.then(r => r.json());
// Download from IPFS
return ipfs.cat(cid);
}
async deleteFile(attachment: AttachmentRecord) {
// IPFS is immutable, but you can unpin and remove from backend
await fetch(`/api/ipfs-cids/${attachment.id}`, { method: 'DELETE' });
}
}
```
```dart Flutter theme={null}
// Example: IPFS remote storage
class IPFSStorageAdapter implements RemoteStorage {
@override
Future uploadFile(
Stream> fileData,
Attachment attachment,
) async {
// Collect the stream
final bytes = [];
await for (final chunk in fileData) {
bytes.addAll(chunk);
}
// Upload to IPFS
final cid = await ipfs.add(Uint8List.fromList(bytes));
// Store CID in your backend for retrieval
await http.post(
Uri.parse('/api/ipfs-cids'),
body: jsonEncode({
'attachmentId': attachment.id,
'cid': cid,
}),
);
}
@override
Future>> downloadFile(Attachment attachment) async {
// Retrieve CID from backend
final response = await http.get(
Uri.parse('/api/ipfs-cids/${attachment.id}'),
);
final cid = jsonDecode(response.body)['cid'] as String;
// Download from IPFS
final data = await ipfs.cat(cid);
return Stream.value(data);
}
@override
Future deleteFile(Attachment attachment) async {
// IPFS is immutable, but you can unpin and remove from backend
await http.delete(
Uri.parse('/api/ipfs-cids/${attachment.id}'),
);
}
}
```
```swift Swift theme={null}
// Example: IPFS remote storage
class IPFSStorageAdapter: RemoteStorageAdapter {
func uploadFile(fileData: Data, attachment: Attachment) async throws {
// Upload to IPFS
// let cid = try await ipfs.add(fileData)
// Store CID in your backend for retrieval
struct CIDRequest: Codable {
let attachmentId: String
let cid: String
}
let requestBody = CIDRequest(attachmentId: attachment.id, cid: "your-cid-here")
var request = URLRequest(url: URL(string: "/api/ipfs-cids")!)
request.httpMethod = "POST"
request.setValue("application/json", forHTTPHeaderField: "Content-Type")
request.httpBody = try JSONEncoder().encode(requestBody)
_ = try await URLSession.shared.data(for: request)
}
func downloadFile(attachment: Attachment) async throws -> Data {
// Retrieve CID from backend
struct CIDResponse: Codable {
let cid: String
}
let request = URLRequest(url: URL(string: "/api/ipfs-cids/\(attachment.id)")!)
let (data, _) = try await URLSession.shared.data(for: request)
let response = try JSONDecoder().decode(CIDResponse.self, from: data)
// Download from IPFS
// let fileData = try await ipfs.cat(response.cid)
// return fileData
return Data() // Replace with actual IPFS download
}
func deleteFile(attachment: Attachment) async throws {
// IPFS is immutable, but you can unpin and remove from backend
var request = URLRequest(url: URL(string: "/api/ipfs-cids/\(attachment.id)")!)
request.httpMethod = "DELETE"
_ = try await URLSession.shared.data(for: request)
}
}
```
```kotlin Kotlin theme={null}
// Example: IPFS remote storage
class IPFSStorageAdapter : RemoteStorage {
override suspend fun uploadFile(
fileData: Flow,
attachment: Attachment
) {
// Collect the flow
val bytes = mutableListOf()
fileData.collect { bytes.add(it) }
val allBytes = bytes.flatMap { it.toList() }.toByteArray()
// Upload to IPFS
val cid = // ... upload to IPFS
// Store CID in your backend for retrieval
// ... your HTTP POST to store CID
}
override suspend fun downloadFile(attachment: Attachment): Flow {
// Retrieve CID from backend
val cid = // ... fetch CID from your API
// Download from IPFS
val data = // ... download from IPFS
return flowOf(data)
}
override suspend fun deleteFile(attachment: Attachment) {
// IPFS is immutable, but you can unpin and remove from backend
// ... your HTTP DELETE implementation
}
}
```
```csharp .NET theme={null}
// Example: IPFS remote storage
public sealed class IPFSStorageAdapter(HttpClient http) : IRemoteStorageAdapter
{
public async Task UploadFileAsync(Stream fileData, Attachment attachment)
{
// Upload the file to IPFS via your backend
using var content = new StreamContent(fileData);
var resp = await http.PostAsync("/api/ipfs/add", content);
resp.EnsureSuccessStatusCode();
var cid = await resp.Content.ReadAsStringAsync();
// Store the CID so it can be retrieved for downloads
await http.PostAsJsonAsync("/api/ipfs-cids",
new { attachmentId = attachment.Id, cid });
}
public async Task DownloadFileAsync(Attachment attachment)
{
// Retrieve the CID from your backend
var cid = await http.GetStringAsync($"/api/ipfs-cids/{attachment.Id}");
return await http.GetStreamAsync($"/api/ipfs/cat/{cid}");
}
public async Task DeleteFileAsync(Attachment attachment)
{
// IPFS is immutable; unpin and remove the record from your backend
var resp = await http.DeleteAsync($"/api/ipfs-cids/{attachment.Id}");
if (resp.StatusCode != HttpStatusCode.NotFound)
resp.EnsureSuccessStatusCode();
}
}
```
### Verification and Recovery
`verifyAttachments()` is always called internally during `startSync()`.
This method does the following:
1. Verifies local files exist at expected paths
2. Repairs broken `localUri` references
3. Archives attachments with missing files
4. Requeues downloads for synced files with missing local copies
```typescript JavaScript/TypeScript theme={null}
await attachmentQueue.verifyAttachments();
```
```swift Swift theme={null}
try await attachmentQueue.waitForInit()
```
```csharp .NET theme={null}
await queue.VerifyAttachmentsAsync();
```
In the Flutter and Kotlin SDKs, this method is not yet exposed publicly. It still runs automatically during `startSync()`.
### Cache Management
Control archived file retention:
```typescript JavaScript/TypeScript theme={null}
const queue = new AttachmentQueue({
// ... other options
archivedCacheLimit: 200 // Keep 200 archived files; oldest deleted when limit reached
});
// For manually expiring the cache
queue.expireCache()
```
```dart Flutter theme={null}
final queue = AttachmentQueue(
// ... other options
archivedCacheLimit: 200, // Keep 200 archived files; oldest deleted when limit reached
);
// For manually expiring the cache
await queue.expireCache();
```
```swift Swift theme={null}
let queue = AttachmentQueue(
db: db,
remoteStorage: remoteStorage,
attachmentsDirectory: try getAttachmentsDirectoryPath(),
watchAttachments: watchAttachments,
// ... other options
archivedCacheLimit: 200 // Keep 200 archived files; oldest deleted when limit reached
)
// For manually expiring the cache
try await queue.expireCache()
```
```kotlin Kotlin theme={null}
val queue = AttachmentQueue(
// ... other options
archivedCacheLimit = 200 // Keep 200 archived files; oldest deleted when limit reached
)
// For manually expiring the cache
queue.expireCache()
```
```csharp .NET theme={null}
var queue = new AttachmentQueue(new AttachmentQueueOptions
{
// ... other options
ArchivedCacheLimit = 200, // Keep 200 archived files; oldest deleted when limit reached
});
// Manually expire the cache
await queue.ExpireCacheAsync();
```
### Offline Behavior
The attachment queue keeps working in poor or no network conditions:
* **Local saves** - Files are saved locally immediately, synced later
* **Automatic retry** - Failed uploads/downloads retry when connection returns
* **Queue persistence** - Queue state survives app restarts
* **Conflict-free** - Files are immutable, identified by UUID
* **Bandwidth efficient** - Only syncs when needed, respects network conditions
## Migrating From Deprecated Packages
If you are migrating from the now deprecated attachment helpers for Dart or JavaScript, follow the notes below:
A fairly simple migration from `powersync_attachments_helper` to the new utilities would be to adopt the new library with a different Attachment Queue table name and drop the legacy package. This means existing attachments are lost, but will be re-downloaded automatically.
Import `AttachmentTable` and `AttachmentQueue` directly from your platform SDK (`@powersync/web`, `@powersync/node`, or `@powersync/react-native`), then remove `@powersync/attachments` from your dependencies.
**React Native only:** also install `@powersync/attachments-storage-react-native` plus either `expo-file-system` (Expo 54+) or `@dr.pogodin/react-native-fs`.
**What changed:**
| Before (`@powersync/attachments`) | After (platform SDK) |
| --------------------------------------------- | ------------------------------------------------------------------------ |
| `AbstractAttachmentQueue` subclass | `AttachmentQueue` instantiated directly |
| `onAttachmentIdsChange(ids: string[])` | `watchAttachments` — items must be `{ id, fileExtension }`, not just IDs |
| `newAttachmentRecord()` + `saveToQueue()` | `saveFile({ data, fileExtension, updateHook })` |
| `init()` | `startSync()` |
| Single `storage` adapter | `localStorage` + `remoteStorage` (two separate adapters) |
| `syncInterval` | `syncIntervalMs` |
| `cacheLimit` | `archivedCacheLimit` |
| `AttachmentTable` option: `name` | `viewName` |
| `AttachmentTable` option: `additionalColumns` | Removed — use the built-in `meta_data` column (JSON string) instead |
| Error handlers return `{ retry: boolean }` | Return `Promise`; `onDeleteError` is now also required |
**Tip:** use a different `viewName` (e.g. `attachment_queue`) to avoid a SQLite conflict with the old `attachments` table during the transition.
**Data on existing users:** the new local attachments table starts empty. Files already in remote storage will re-download automatically once referenced by your `watchAttachments` query. Files that were only ever stored locally and never uploaded have no remote copy and will not be recoverable.
## Related Resources
* **[An Implementation Walkthrough Using The Flutter/Dart Attachment Helpers](https://www.powersync.com/blog/building-offline-first-file-uploads-with-powersync-attachments-helper)** - Blog post on building offline-first uploads
***
# Background Syncing
Source: https://docs.powersync.com/client-sdks/advanced/background-syncing
Run PowerSync sync operations while your app is in the background to keep local data up to date.
Applications often need to sync data when they're not in active use. This document explains background syncing implementations with PowerSync.
## Platform Support
Background syncing has been tested in:
* **Flutter** - Using [workmanager](https://github.com/fluttercommunity/flutter_workmanager/)
* **React Native & Expo** - Using Expo's `BackgroundTask` API. See our [demo](https://github.com/powersync-ja/powersync-js/tree/main/demos/react-native-supabase-background-sync) and accompanying [blog post](https://www.powersync.com/blog/keep-background-apps-fresh-with-expo-background-tasks-and-powersync).
* **Kotlin - Android** - Implementation details in the [Supabase To-Do List demo](https://github.com/powersync-ja/powersync-kotlin/blob/main/demos/supabase-todolist/docs/BackgroundSync.md)
These examples can be adapted for other platforms/frameworks. For implementation questions or assistance, chat to us on [Discord](https://discord.gg/powersync).
## Flutter Implementation Guide
### Prerequisites
1. Complete the [workmanager platform setup](https://github.com/fluttercommunity/flutter_workmanager/#platform-setup)
2. Review the [Supabase To-Do List Demo](https://github.com/powersync-ja/powersync.dart/tree/main/demos/supabase-todolist) for context
### Configure the Background Task
In `main.dart`:
```dart theme={null}
void main() async {
// ... existing setup code ...
const simpleTaskKey = "com.domain.myapp.taskId";
// Mandatory if the App is obfuscated or using Flutter 3.1+
@pragma('vm:entry-point')
void callbackDispatcher() {
Workmanager().executeTask((task, inputData) async {
switch (task) {
case simpleTaskKey:
// Initialize PowerSync database and connection
final currentConnector = await openDatabase();
db.connect(connector: currentConnector!);
// Perform database operations
await TodoList.create('New background task item');
await currentConnector.uploadData(db);
await TodoList.create('testing1111');
await currentConnector.uploadData(db);
// print("$simpleTaskKey was executed. inputData = $inputData");
break;
}
// Close database when done
await db.close();
return Future.value(true);
});
}
// Initialize the workmanager with your callback
Workmanager().initialize(
callbackDispatcher,
// Shows notifications during task execution (useful for debugging)
isInDebugMode: true
);
// ... rest of your app initialization ...
}
```
Note specifically in the switch statement:
```dart theme={null}
// currentConnector is the connector to the remote DB
// openDatabase sets the db variable to the PowerSync database
final currentConnector = await openDatabase();
// connect PowerSync to the remote database
db.connect(connector: currentConnector!);
// a database write operation
await TodoList.create('Buy new shoes');
// Sync with the remote database
await currentConnector.uploadData(db);
```
1. Since WorkManager executes in a new process, you need to set up the PowerSync local database and connect to the remote database using your connector.
2. Run a write (in the case of this demo app, we create a 'todo list')
3. Make sure to run `currentConnector.uploadData(db);` so that the local write is uploaded to the remote database.
### Testing
Add a test button:
```dart theme={null}
ElevatedButton(
title: const Text("Start the Flutter background service"),
onTap: () async {
await Workmanager().cancelAll();
// print("RUN BACKGROUND TASK");
await Workmanager().registerOneOffTask(
simpleTaskKey,
simpleTaskKey,
initialDelay: Duration(seconds: 10),
inputData: {
int': 1,
},
);
},
),
```
Click the button, background the app, wait 10 seconds, then verify new records in the remote database.
### Platform Compatibility
#### Android
* Implementation works as expected.
#### iOS
* At the time of last testing this (January 2024), we were only able to get part of this to work using the branch for [this PR](https://github.com/fluttercommunity/flutter_workmanager/pull/511) into workmanager.
* While testing we were not able to get iOS background fetching to work, however this is most likely an
[issue](https://github.com/fluttercommunity/flutter_workmanager/issues/515) with the package.
# Sync Catch-Up (Checkpoint Requests)
Source: https://docs.powersync.com/client-sdks/advanced/checkpoint-requests
Wait for the local database to catch up with the latest server state.
PowerSync syncs continuously in the background and [SyncStatus](/client-sdks/usage-examples#accessing-powersync-connection-status-information) fields like `downloading` and `hasSynced` tell you that this is happening. However, none of these fields tell you whether the data on the device is up to date with the server *right now*.
Checkpoint requests let you catch up on demand: you request a marker of the current server state, then wait until the local database has caught up to it. Waiting covers uploads as well as downloads: a request created after local writes confirms that those writes have been uploaded and their results have synced back (see [Relationship to Local Writes](#relationship-to-local-writes)).
Checkpoint requests are an alpha API and may change. Support requires PowerSync Service version 1.24.0 or later. Checkpoint requests are not currently available for the Rust SDK.
Example use cases include:
* Critical operations: ensure that pending local writes have uploaded and the latest server data has downloaded before a user starts a work session, or before the app performs a sensitive operation.
* Pull-to-refresh: resolve the refresh indicator only once the device has caught up with the server, instead of hiding it after an arbitrary delay.
* Data availability: when a user opens a link or notification, wait until the data it refers to has synced before rendering the screen.
* Backend processing: after your backend finishes a job and writes the result to the source database, know when that result is available locally.
* App startup or foregrounding: show a "syncing latest changes" state that ends exactly when the device is caught up.
* On-demand sync: not every app needs an always-on connection. If eventual sync is enough, connect, wait for a checkpoint request to sync, then disconnect again.
Checkpoint requests are related to, but different from, [waitForFirstSync()](/client-sdks/usage-examples#wait-for-the-initial-sync-to-complete). `waitForFirstSync()` resolves once, when the first complete sync finishes. A checkpoint request can be created at any time after that, and lets you wait until the local database has caught up with the server as of the moment you created it.
## How Checkpoint Requests Work
During normal sync, the PowerSync Service groups changes from the source database into checkpoints, and the PowerSync Client SDK applies each checkpoint to the local database as a single consistent unit. This happens continuously in the background. Checkpoint requests add a way to point at a specific checkpoint: one that reflects server state at or after the moment you asked.
When you call `requestCheckpoint()`, the PowerSync Service records the source database's current replication position. `waitForSync()` then resolves once a checkpoint that covers that position has been completely synced and applied to the local database. At that point, everything the source database contained when the Service handled the request is present locally.
You cannot supply an arbitrary point in time. The PowerSync Service always captures the source database's position at the moment it handles the request.
See [Consistency](/architecture/consistency) for more information about how PowerSync applies complete checkpoints.
## Prerequisites
Before creating a checkpoint request:
1. Run PowerSync Service v1.24.0 or later.
2. Opt in to checkpoint requests with the `checkpointMode` option when calling `connect()`.
```typescript JavaScript/TypeScript theme={null}
// Requires @powersync/web 2.3.0, @powersync/react-native 2.2.0,
// @powersync/node 1.0.0, or @powersync/capacitor 0.9.0 or later
await db.connect(connector, { checkpointMode: 'requests' });
```
```dart Dart theme={null}
// Requires powersync 2.4.0 or later
await db.connect(
connector: connector,
options: SyncOptions(checkpointMode: .requests()),
);
```
```kotlin Kotlin theme={null}
// Requires PowerSync Kotlin SDK 1.15.0 or later
database.connect(
connector,
options = SyncOptions(
checkpointMode = CheckpointMode.Requests(),
)
)
```
```swift Swift theme={null}
// Requires PowerSync Swift SDK 1.16.0 or later
try await database.connect(
connector: connector,
options: ConnectOptions(checkpointMode: .requests())
)
```
```cs .NET theme={null}
// Requires PowerSync.Common 0.1.5 or later
await database.Connect(
connector: connector,
options: new PowerSyncConnectionOptions(checkpointMode: new CheckpointMode.Requests())
);
```
Checkpoint requests are currently opt-in while this feature is in alpha. Without this connect option, calling `requestCheckpoint()` throws an error. Checkpoint requests will be enabled by default in a future release.
## Waiting for the Latest Server Data
Create a checkpoint request, then wait for it to sync before reading the refreshed data:
```typescript JavaScript/TypeScript theme={null}
async function refreshLocalData() {
const checkpoint = await database.requestCheckpoint();
await checkpoint.waitForSync({ signal: AbortSignal.timeout(30_000) });
}
```
```dart Dart theme={null}
Future refreshLocalData() async {
final checkpoint = await database.requestCheckpoint();
await checkpoint.waitForSync(
abortTrigger: Future.pause(Duration(seconds: 30))
);
}
```
```kotlin Kotlin theme={null}
suspend fun refreshLocalData() {
val checkpoint = database.requestCheckpoint()
withTimeout(30.seconds) {
checkpoint.waitForSync()
}
}
```
```swift Swift theme={null}
func refreshLocalData() async throws {
let checkpoint = try await database.requestCheckpoint()
try await checkpoint.waitForSync(timeout: 30)
// Local queries now reflect server state from when the request was made.
}
```
```cs .NET theme={null}
async Task RefreshLocalData(CancellationToken cancellationToken = default)
{
var checkpoint = await database.RequestCheckpoint(cancellationToken);
await checkpoint.WaitForSync(cancellationToken);
}
```
`requestCheckpoint()` requires that the database is connected or connecting. The device must be online for the request to reach the PowerSync Service. If it is offline or the sync client is reconnecting, the call waits and continues once the Service is reachable.
Creating the request has no timeout of its own, so it can stay suspended while the sync client retries its connection. Cancel the calling task if you need to stop waiting.
Aborting a `waitForSync` call or configuring a timeout does not remove the checkpoint. It only limits how long you wait for the checkpoint to sync and apply locally.
## Handling Wait Failures
Handle request creation and waiting errors separately when your app needs different recovery behavior:
```typescript JavaScript/TypeScript theme={null}
const signal = AbortSignal.timeout(30_000);
try {
const checkpoint = await database.requestCheckpoint();
await checkpoint.waitForSync({ signal });
} catch (e) {
if (signal.aborted) {
showRefreshMessage('The refresh timed out. Try again.');
} else {
showRefreshMessage(`Could not wait for checkpoint: ${e}`);
}
}
```
```dart Dart theme={null}
try {
final checkpoint = await database.requestCheckpoint();
await checkpoint.waitForSync(
abortTrigger: Future.pause(Duration(seconds: 30))
);
} on AbortException {
showRefreshMessage('The refresh timed out. Try again.');
} catch (e) {
showRefreshMessage('Could not wait for checkpoint: $e');
}
```
```kotlin Kotlin theme={null}
try {
val checkpoint = database.requestCheckpoint()
withTimeout(30.seconds) {
checkpoint.waitForSync()
}
} catch (e: TimeoutCancellationException) {
showRefreshMessage("The refresh timed out. Try again.")
} catch (e: CheckpointRequestException.Disconnected) {
showRefreshMessage("Reconnect before refreshing again.")
} catch (e: Exception) {
// Other checkpoint exceptions
}
```
```swift Swift theme={null}
do {
let checkpoint = try await database.requestCheckpoint()
try await checkpoint.waitForSync(timeout: 30)
} catch CheckpointWaitError.timeout {
showRefreshMessage("The refresh timed out. Try again.")
} catch CheckpointWaitError.disconnected {
showRefreshMessage("Reconnect before refreshing again.")
} catch let error as any CheckpointError {
showRefreshMessage(error.localizedDescription)
}
```
```cs .NET theme={null}
try
{
var checkpoint = await database.RequestCheckpoint();
await checkpoint.WaitForSync().WaitAsync(TimeSpan.FromSeconds(30));
}
catch (TimeoutException)
{
ShowRefreshMessage("The refresh timed out. Try again.");
}
catch (CheckpointRequestException e) when (e.Message == CheckpointRequestException.Disconnected)
{
ShowRefreshMessage("Reconnect before refreshing again.");
}
catch (CheckpointRequestException e)
{
// Handle other checkpoint errors
ShowRefreshMessage($"Failed to request checkpoint: {e.Message}");
}
catch (Exception)
{
// Handle other errors
}
```
A request remains valid across a disconnect. After reconnecting with `.requests()`, you can call `waitForSync()` again on the same request. Discard existing request values after clearing the local PowerSync database because clearing it resets the persisted request state.
`waitForSync()` also fails if the sync client reports an upload or download error. Wait for sync to recover before retrying.
## Relationship to Local Writes
PowerSync never applies a checkpoint while local writes are waiting to upload, so sync cannot revert your own pending changes. When checkpoint requests are enabled, the PowerSync Client SDK maintains this guarantee with checkpoint requests: each time it finishes uploading the local write queue, it internally creates a request that captures a source position from after the upload completed. You do not need to call `requestCheckpoint()` for your own writes.
`waitForSync()` considers a request complete when the same or a newer checkpoint request has been applied locally. This makes explicit requests safe to combine with pending writes. If you create a request while local writes are waiting to upload, it is not applied while they are pending; once the upload queue empties, the SDK's newer internal request supersedes it and captures a source position from after the upload. If you create the request after the SDK's internal request instead, it captures an even later position.
In both cases, waiting on the request also waits for the pending upload and for its result to sync back. You can therefore write locally and wait for the uploaded result to return through sync:
```typescript JavaScript/TypeScript theme={null}
await database.execute('INSERT INTO tasks (id, description) VALUES (uuid(), ?)', ['Review the project plan']);
const checkpoint = await database.requestCheckpoint();
await checkpoint.waitForSync();
```
```dart Dart theme={null}
await database.execute(
'INSERT INTO tasks (id, description) VALUES (uuid(), ?)',
['Review the project plan'],
);
final checkpoint = await database.requestCheckpoint();
await checkpoint.waitForSync();
```
```kotlin Kotlin theme={null}
database.execute(
"INSERT INTO tasks (id, description) VALUES (uuid(), ?)",
listOf("Review the project plan")
)
val checkpoint = database.requestCheckpoint()
checkpoint.waitForSync()
```
```swift Swift theme={null}
try await database.execute(
sql: "INSERT INTO tasks (id, description) VALUES (uuid(), ?)",
parameters: ["Review the project plan"]
)
let checkpoint = try await database.requestCheckpoint()
try await checkpoint.waitForSync(timeout: 30)
// The pending write has uploaded and its source state has synced locally.
```
```cs .NET theme={null}
await database.Execute(
"INSERT INTO tasks (id, description) VALUES (uuid(), ?)",
["Review the project plan"]
);
var checkpoint = await database.RequestCheckpoint();
await checkpoint.WaitForSync();
```
This behavior relies on `uploadData()` returning only after your backend has committed the uploaded changes to the source database. See [Writing Client Changes](/handling-writes/writing-client-changes#why-must-my-write-endpoint-be-synchronous) for the reason your write endpoint must be synchronous.
The upload response remains the authority on whether your backend accepted, changed, or rejected a mutation. A synced checkpoint request only confirms that PowerSync and the local database have progressed through the source database position the request captured.
## Asynchronous Upload Backends
This feature is available for customers on [Team and Enterprise](https://www.powersync.com/pricing) plans.
The managed flow assumes that `uploadData()` returns only after your backend commits the uploaded changes to the source database. If your backend queues uploads for later processing, use custom checkpoint requests.
Follow [Setting Up Custom Checkpoint Requests](/handling-writes/custom-write-checkpoints#setting-up-custom-checkpoint-requests) for the source database, Sync Config, and backend changes. If older app versions still use legacy Custom Write Checkpoints, see [Migrating to Custom Checkpoint Requests](/handling-writes/custom-write-checkpoints#migrating-to-custom-checkpoint-requests).
The difference on the client is that the PowerSync Client SDK generates the checkpoint request ID and sends it to your backend through `CustomCheckpointRequestConnector`.
### Checkpoint Request IDs
The SDK persists an increasing checkpoint request ID in the local database. When connecting, it sends its current ID to the backend to reconcile the local counter with any state the backend still holds. The SDK uses the ID returned by the backend as the starting point before allocating later requests.
Store the greatest request ID received for each authenticated user and PowerSync client ID. When handling a request:
1. If the submitted ID is greater than the stored ID, record and process the submitted ID.
2. If the submitted ID is equal to or less than the stored ID, do not move the stored value backward.
3. Return the greater of the submitted and stored IDs.
This comparison makes repeated and stale requests idempotent. It also handles seeding after the local request counter has been reset. For example, if the client submits ID `1` while the backend still holds ID `42`, return `42`. The SDK then continues allocating IDs after `42`.
You can delete stored request records after an appropriate retention period. While a record exists, return its value during reconciliation so the SDK can resume from that value.
### Connector Changes
To make the PowerSync client SDK post checkpoint requests to your backend instead of to the PowerSync Service,
declare support for checkpoint requests on your backend connector:
* For JavaScript, implement the optional `postCheckpointRequest` method.
* For Dart, mix in the `CustomCheckpointRequestConnector` class.
* For Kotlin, implement `CustomCheckpointRequestConnector`.
* For Swift, make your connector conform to `CustomCheckpointRequestConnector`.
* For .NET, implement `ICustomCheckpointRequestConnector`.
Forward the request to your application backend:
```typescript JavaScript/TypeScript theme={null}
class MyBackendConnector implements PowerSyncBackendConnector {
// ... also implement fetchCredentials and uploadData
async postCheckpointRequest(clientId: string, requestId: string): Promise {
const response = await MyBackendConnector.createCheckpointRequest(clientId, requestId);
return response.checkpointRequestId;
}
}
```
```dart Dart theme={null}
final class MyBackendConnector extends PowerSyncBackendConnector
with CustomCheckpointRequestConnector {
// ... also implement fetchCredentials and uploadData
@override
Future postCheckpointRequest(
String clientId,
String requestId,
) async {
final response = await myBackend.createCheckpointRequest(
clientId,
requestId,
);
return response.checkpointRequestId;
}
}
```
```kotlin Kotlin theme={null}
class MyBackendConnector: PowerSyncBackendConnector(), CustomCheckpointRequestConnector {
override suspend fun fetchCredentials(): PowerSyncCredentials? = TODO()
override suspend fun uploadData(database: PowerSyncDatabase) = TODO()
override suspend fun postCheckpointRequest(
clientId: String,
requestId: Long
): Long {
val response = myBackend.createCheckpointRequest(clientId, requestId)
return response.checkpointRequestId
}
}
```
```swift Swift theme={null}
extension BackendConnector: CustomCheckpointRequestConnector {
func postCheckpointRequest(
_ checkpointRequestId: Int64,
clientId: String
) async throws -> Int64 {
let response = try await backendAPI.createCheckpointRequest(
checkpointRequestId: checkpointRequestId,
clientId: clientId
)
return response.checkpointRequestId
}
}
```
```cs .NET theme={null}
class BackendConnector : ICustomCheckpointRequestConnector
{
public async Task PostCheckpointRequest(string clientId, string requestId, CancellationToken ct)
{
var response = await BackendAPI.CreateCheckpointRequest(clientId, requestId, cancellationToken: ct);
return response.RequestId;
}
// ...also implement other IPowerSyncBackendConnector methods (FetchCredentials, UploadData)
}
```
The connector must use your application's own authentication for this call because `postCheckpointRequest()` does not receive the PowerSync sync token.
Checkpoint IDs are positive integers between 1 and 263 - 1 (inclusive). For compatibility with JavaScript numbers,
the JavaScript and Dart client SDKs represent requests as strings.
The .NET SDK also represents requests as strings for compatibility with older SDK APIs; this will change in a future release
of the .NET SDK.
# CRDT Data Structures
Source: https://docs.powersync.com/client-sdks/advanced/crdts
Use CRDT data structures from libraries like Yjs alongside PowerSync for conflict-free collaborative editing with offline sync support.
This may be useful for cases such as document editing, where last-write-wins is not sufficient for conflict resolution. PowerSync becomes the provider for CRDT data — both for local storage and for propagating changes to other clients.
### Example Implementations
For an example implementation, refer to the following demo built using the PowerSync Web SDK:
* [Yjs Document Collaboration Demo](https://github.com/powersync-ja/powersync-js/tree/main/demos/yjs-react-supabase-text-collab)
# JSON, Arrays and Custom Types
Source: https://docs.powersync.com/client-sdks/advanced/custom-types-arrays-and-json
Sync JSON, JSONB, arrays, and other custom Postgres types through PowerSync by serializing them to text.
PowerSync supports JSON/JSONB and array columns. They are synced as JSON text and can be queried with SQLite JSON functions on the client. Other custom Postgres types can be synced by serializing their values to text in the client-side schema. When updating client data, you have the option to replace the entire column value with a string or enable [advanced schema options](#advanced-schema-options-to-process-writes) to track more granular changes and include custom metadata.
## JSON and JSONB
The PowerSync Service treats JSON and JSONB columns as text and provides many helpers for working with JSON in [Sync Streams](/sync/streams/overview) (or legacy [Sync Rules](/sync/rules/overview)).
**Note:** Native Postgres arrays, JSON arrays, and JSONB arrays are effectively all equivalent in PowerSync.
### Postgres
JSON columns are represented as:
```sql theme={null}
ALTER TABLE todos
ADD COLUMN custom_payload json;
```
### Sync Streams
PowerSync treats JSON columns as text. Use `json_extract()` and other JSON functions in stream queries. Subscribe per list to sync only that list's todos:
```yaml theme={null}
config:
edition: 3
streams:
my_json_todos:
auto_subscribe: true
with:
owned_lists: SELECT id AS list_id FROM lists WHERE owner_id = auth.user_id()
query: SELECT * FROM todos WHERE json_extract(custom_payload, '$.json_list') IN owned_lists
```
The client subscribes once per list (e.g. `db.syncStream('my_json_todos', { list_id: listId }).subscribe()`).
PowerSync treats JSON columns as text and provides transformation functions in Sync Rules such as `json_extract()`.
```yaml theme={null}
bucket_definitions:
my_json_todos:
# Separate bucket per To-Do list
parameters: SELECT id AS list_id FROM lists WHERE owner_id = request.user_id()
data:
- SELECT * FROM todos WHERE json_extract(custom_payload, '$.json_list') = bucket.list_id
```
### Client SDK
**Schema**
Add your JSON column as a `text` column in your client-side schema definition. For advanced update tracking, see [Advanced Schema Options](#advanced-schema-options-to-process-writes).
```dart theme={null}
Table(
name: 'todos',
columns: [
Column.text('custom_payload'),
// ... other columns ...
],
// Optionally, enable advanced update tracking options (see details at the end of this page):
trackPreviousValues: true,
trackMetadata: true,
ignoreEmptyUpdates: true,
)
```
```javascript theme={null}
const todos = new Table(
{
custom_payload: column.text,
// ... other columns ...
},
{
// Optionally, enable advanced update tracking options (see details at the end of this page):
trackPrevious: true,
trackMetadata: true,
ignoreEmptyUpdates: true,
}
);
```
```csharp theme={null}
new Table
{
Name = "todos",
Columns =
{
["custom_payload"] = ColumnType.Text,
// ... other columns ...
},
// Optionally, enable advanced update tracking options (see details at the end of this page):
TrackPreviousValues = new TrackPreviousOptions(),
TrackMetadata = true,
IgnoreEmptyUpdates = true
}
```
Example not yet available.
**Writing Changes**
You can write the entire updated column value as a string, or, with `trackPreviousValues` enabled, compare the previous and new values to process only the changes you care about:
```dart theme={null}
// Full replacement (basic):
await db.execute('UPDATE todos set custom_payload = ?, _metadata = ? WHERE id = ?', [
'{"foo": "bar", "baz": 123}',
'op-metadata-example', // Example metadata value
'00000000-0000-0000-0000-000000000000'
]);
// Diffing columns in uploadData (advanced):
// See details about these advanced schema options at the end of this page
import 'dart:convert';
if (op.op == UpdateType.put && op.previousValues != null) {
var oldJson = jsonDecode(op.previousValues['custom_payload'] ?? '{}');
var newJson = jsonDecode(op.opData['custom_payload'] ?? '{}');
var metadata = op.metadata; // Access metadata here
// Compare oldJson and newJson to determine what changed
// Use metadata as needed as you process the upload
}
```
```javascript theme={null}
// Full replacement (basic):
await db.execute(
'UPDATE todos set custom_payload = ?, _metadata = ? WHERE id = ?',
['{"foo": "bar", "baz": 123}', 'op-metadata-example', '00000000-0000-0000-0000-000000000000']
);
// Diffing columns in uploadData (advanced):
// See details about these advanced schema options at the end of this page
if (op.op === UpdateType.PUT && op.previousValues) {
const oldJson = JSON.parse(op.previousValues['custom_payload'] ?? '{}');
const newJson = JSON.parse(op.opData['custom_payload'] ?? '{}');
const metadata = op.metadata; // Access metadata here
// Compare oldJson and newJson to determine what changed
// Use metadata as needed as you process the upload
}
```
```csharp theme={null}
// Full replacement (basic):
await db.Execute(
"UPDATE todos SET custom_payload = ?, _metadata = ? WHERE id = ?",
new object[] { "{\"foo\": \"bar\", \"baz\": 123}", "op-metadata-example", "00000000-0000-0000-0000-000000000000" }
);
// Diffing columns in UploadData (advanced):
// See details about these advanced schema options at the end of this page
using Newtonsoft.Json;
if (op.Op.ToString() == "PUT" && op.PreviousValues != null)
{
var oldJson = JsonConvert.DeserializeObject>(
op.PreviousValues.GetValueOrDefault("custom_payload", "{}")?.ToString() ?? "{}"
);
var newJson = JsonConvert.DeserializeObject>(
(op.OpData != null ? op.OpData.GetValueOrDefault("custom_payload", "{}")?.ToString() ?? "{}" : "{}") ?? "{}"
);
var metadata = op.Metadata; // Access metadata here
// Compare oldJson and newJson to determine what changed
// Use metadata as needed as you process the upload
}
```
Example not yet available.
## Arrays
PowerSync treats array columns as JSON text. This means that the SQLite JSON operators can be used on any array columns.
Additionally, array membership is supported in [Sync Streams](/sync/streams/overview) (or legacy [Sync Rules](/sync/rules/overview)) so you can sync rows based on whether a parameter value appears in an array column.
**Note:** Native Postgres arrays, JSON arrays, and JSONB arrays are effectively all equivalent in PowerSync.
### Postgres
Array columns are defined in Postgres using the following syntax:
```sql theme={null}
ALTER TABLE todos
ADD COLUMN unique_identifiers text[];
```
### Sync Streams
Array columns are converted to text by the PowerSync Service. A text array as defined above would be synced to clients as the following string:
`["00000000-0000-0000-0000-000000000000", "12345678-1234-1234-1234-123456789012"]`
**Array Membership**
Sync rows where a subscription parameter value is in the row's array column using `IN`:
```yaml theme={null}
config:
edition: 3
streams:
custom_todos:
query: SELECT * FROM todos WHERE subscription.parameter('list_id') IN unique_identifiers
```
The client subscribes per list (e.g. `db.syncStream('custom_todos', { list_id: listId }).subscribe()`).
It's possible to sync rows dynamically based on the contents of array columns using the `IN` operator:
```yaml theme={null}
bucket_definitions:
custom_todos:
# Separate bucket per To-Do list
parameters: SELECT id AS list_id FROM lists WHERE owner_id = request.user_id()
data:
- SELECT * FROM todos WHERE bucket.list_id IN unique_identifiers
```
See these additional details when using the `IN` operator: [Operators](/sync/supported-sql#operators)
### Client SDK
**Schema**
Add your array column as a `text` column in your client-side schema definition. For advanced update tracking, see [Advanced Schema Options](#advanced-schema-options-to-process-writes).
```javascript theme={null}
const todos = new Table(
{
unique_identifiers: column.text,
// ... other columns ...
},
{
// Optionally, enable advanced update tracking options (see details at the end of this page):
trackPrevious: true,
trackMetadata: true,
ignoreEmptyUpdates: true,
}
);
```
```dart theme={null}
Table(
name: 'todos',
columns: [
Column.text('unique_identifiers'),
// ... other columns ...
],
// Optionally, enable advanced update tracking options (see details at the end of this page):
trackPreviousValues: true,
trackMetadata: true,
ignoreEmptyUpdates: true,
)
```
```csharp theme={null}
new Table
{
Name = "todos",
Columns =
{
["unique_identifiers"] = ColumnType.Text,
// ... other columns ...
},
// Optionally, enable advanced update tracking options (see details at the end of this page):
TrackPreviousValues = new TrackPreviousOptions(),
TrackMetadata = true,
IgnoreEmptyUpdates = true
}
```
Example not yet available.
**Writing Changes**
You can write the entire updated column value as a string, or, with `trackPreviousValues` enabled, compare the previous and new values to process only the changes you care about:
```javascript theme={null}
// Full replacement (basic):
await db.execute(
'UPDATE todos set unique_identifiers = ?, _metadata = ? WHERE id = ?',
['["DEADBEEF-DEAD-BEEF-DEAD-BEEFDEADBEEF", "ABCDEFAB-ABCD-ABCD-ABCD-ABCDEFABCDEF"]', 'op-metadata-example', '00000000-0000-0000-0000-000000000000']
);
// Diffing columns in uploadData (advanced):
// See details about these advanced schema options at the end of this page
if (op.op === UpdateType.PUT && op.previousValues) {
const oldArray = JSON.parse(op.previousValues['unique_identifiers'] ?? '[]');
const newArray = JSON.parse(op.opData['unique_identifiers'] ?? '[]');
const metadata = op.metadata; // Access metadata here
// Compare oldArray and newArray to determine what changed
// Use metadata as needed as you process the upload
}
```
```dart theme={null}
// Full replacement (basic):
await db.execute('UPDATE todos set unique_identifiers = ?, _metadata = ? WHERE id = ?', [
'["DEADBEEF-DEAD-BEEF-DEAD-BEEFDEADBEEF", "ABCDEFAB-ABCD-ABCD-ABCD-ABCDEFABCDEF"]',
'op-metadata-example', // Example metadata value
'00000000-0000-0000-0000-000000000000'
]);
// Diffing columns in uploadData (advanced):
// See details about these advanced schema options at the end of this page
if (op.op == UpdateType put && op.previousValues != null) {
final oldArray = jsonDecode(op.previousValues['unique_identifiers'] ?? '[]');
final newArray = jsonDecode(op.opData['unique_identifiers'] ?? '[]');
final metadata = op.metadata; // Access metadata here
// Compare oldArray and newArray to determine what changed
// Use metadata as needed as you process the upload
}
```
```csharp theme={null}
// Full replacement (basic):
await db.Execute(
"UPDATE todos SET unique_identifiers = ?, _metadata = ? WHERE id = ?",
new object[] {
"[\"DEADBEEF-DEAD-BEEF-DEAD-BEEFDEADBEEF\", \"ABCDEFAB-ABCD-ABCD-ABCD-ABCDEFABCDEF\"]",
"op-metadata-example",
"00000000-0000-0000-0000-000000000000"
}
);
// Diffing columns in UploadData (advanced):
// See details about these advanced schema options at the end of this page
using Newtonsoft.Json;
if (op.Op.ToString() == "PUT" && op.PreviousValues != null)
{
var oldArray = JsonConvert.DeserializeObject>(
(op.PreviousValues != null ? op.PreviousValues.GetValueOrDefault("unique_identifiers", "[]")?.ToString() : "[]") ?? "[]"
);
var newArray = JsonConvert.DeserializeObject>(
(op.OpData != null ? op.OpData.GetValueOrDefault("unique_identifiers", "[]")?.ToString() : "[]") ?? "[]"
);
var metadata = op.Metadata; // Access metadata here
// Compare oldArray and newArray to determine what changed
// Use metadata as needed as you process the upload
}
```
Example not yet available.
**Attention Supabase users:** Supabase can handle writes with arrays, but you must convert from string to array using `jsonDecode` in the connector's `uploadData` function. The default implementation of `uploadData` does not handle complex types like arrays automatically.
## Custom Types
PowerSync respects Postgres custom types: DOMAIN types sync as their inner type, custom type columns as JSON objects, arrays of custom types as JSON arrays, and ranges (and multi-ranges) as structured JSON. This behavior is the default for Sync Streams. For configuration and legacy behavior, see [Compatibility](/sync/advanced/compatibility#custom_postgres_types). For type handling in queries, see [Types](/sync/types).
### Postgres
Postgres allows developers to create custom data types for columns. For example:
```sql theme={null}
create type location_address AS (
street text,
city text,
state text,
zip numeric
);
```
### Sync Streams
The custom type column is serialized as JSON and you can use `json_extract()` and other JSON functions in stream queries:
```yaml theme={null}
config:
edition: 3
streams:
todos_by_city:
query: SELECT * FROM todos WHERE json_extract(location, '$.city') = subscription.parameter('city')
```
Custom type columns are converted to text by the PowerSync Service.
Depending on whether the `custom_postgres_types` [compatibility option](/sync/advanced/compatibility) is enabled,
PowerSync would sync the row as:
* `{"street":"1000 S Colorado Blvd.","city":"Denver","state":"CO","zip":80211}` if the option is enabled.
* `("1000 S Colorado Blvd.",Denver,CO,80211)` if the option is disabled.
You can use regular string and JSON manipulation functions in Sync Rules. This means that individual values of the type
can be synced with `json_extract` if the `custom_postgres_types` compatibility option is enabled.
Without the option, the entire column must be synced as text.
### Client SDK
**Schema**
Add your custom type column as a `text` column in your client-side schema definition. For advanced update tracking, see [Advanced Schema Options](#advanced-schema-options-to-process-writes).
```javascript theme={null}
const todos = new Table(
{
location: column.text,
// ... other columns ...
},
{
// Optionally, enable advanced update tracking options (see details at the end of this page):
trackPrevious: true,
trackMetadata: true,
ignoreEmptyUpdates: true,
}
);
```
```dart theme={null}
Table(
name: 'todos',
columns: [
Column.text('location'),
// ... other columns ...
],
// Optionally, enable advanced update tracking options (see details at the end of this page):
trackPreviousValues: true,
trackMetadata: true,
ignoreEmptyUpdates: true,
)
```
```csharp theme={null}
new Table
{
Name = "todos",
Columns =
{
["location"] = ColumnType.Text,
// ... other columns ...
},
// Optionally, enable advanced update tracking options (see details at the end of this page):
TrackPreviousValues = new TrackPreviousOptions(),
TrackMetadata = true,
IgnoreEmptyUpdates = true
}
```
Example not yet available.
**Writing Changes**
You can write the entire updated column value as a string, or, with `trackPreviousValues` enabled, compare the previous and new values to process only the changes you care about:
```javascript theme={null}
// Full replacement (basic):
await db.execute(
'UPDATE todos set location = ?, _metadata = ? WHERE id = ?',
['("1234 Update Street",Denver,CO,80212)', 'op-metadata-example', 'faffcf7a-75f9-40b9-8c5d-67097c6b1c3b']
);
// Diffing columns in uploadData (advanced):
// See details about these advanced schema options at the end of this page
if (op.op === UpdateType.PUT && op.previousValues) {
const oldCustomType = op.previousValues['location'] ?? 'null';
const newCustomType = op.opData['location'] ?? 'null';
const metadata = op.metadata; // Access metadata here
// Compare oldCustomType and newCustomType to determine what changed
// Use metadata as needed as you process the upload
}
```
```dart theme={null}
// Full replacement (basic):
await db.execute('UPDATE todos set location = ?, _metadata = ? WHERE id = ?', [
'("1234 Update Street",Denver,CO,80212)',
'op-metadata-example', // Example metadata value
'faffcf7a-75f9-40b9-8c5d-67097c6b1c3b'
]);
// Diffing columns in uploadData (advanced):
// See details about these advanced schema options at the end of this page
if (op.op == UpdateType.put && op.previousValues != null) {
final oldCustomType = op.previousValues['location'] ?? 'null';
final newCustomType = op.opData['location'] ?? 'null';
final metadata = op.metadata; // Access metadata here
// Compare oldCustomType and newCustomType to determine what changed
// Use metadata as needed as you process the upload
}
```
```csharp theme={null}
// Full replacement (basic):
await db.Execute(
"UPDATE todos SET location = ?, _metadata = ? WHERE id = ?",
new object[] { "(\"1234 Update Street\",Denver,CO,80212)", "op-metadata-example", "faffcf7a-75f9-40b9-8c5d-67097c6b1c3b" }
);
// Diffing columns in UploadData (advanced):
// See details about these advanced schema options at the end of this page
if (op.Op.ToString() == "PUT" && op.PreviousValues != null)
{
var oldCustomType = op.PreviousValues.GetValueOrDefault("location", "null")?.ToString() ?? "null";
var newCustomType = op.OpData.GetValueOrDefault("location", "null")?.ToString() ?? "null";
var metadata = op.Metadata; // Access metadata here
// Compare oldCustomType and newCustomType to determine what changed
// Use metadata as needed as you process the upload
}
```
Example not yet available.
## Bonus: Mashup
What if we had a column defined as an array of custom types, where a field in the custom type was JSON? Consider the following Postgres schema:
```sql theme={null}
-- define custom type
CREATE TYPE extended_location AS (
address_label text,
json_address json
);
-- add column
ALTER TABLE todos
ADD COLUMN custom_locations extended_location[];
```
## Advanced Schema Options to Process Writes
With arrays and JSON fields, it's common for only part of the value to change during an update. To make handling these writes easier, you can enable advanced schema options that let you track exactly what changed in each row—not just the new state.
* `trackPreviousValues` (or `trackPrevious` in our JS SDKs): Access previous values for diffing JSON or array fields. Accessible later via `CrudEntry.previousValues`.
* `trackMetadata`: Adds a `_metadata` column for storing custom metadata. Value of the column is accessible later via `CrudEntry.metadata`.
* `ignoreEmptyUpdates`: Skips updates when no data has actually changed.
These advanced schema options were introduced in the following SDK versions:
* Flutter v1.13.0
* React Native v1.20.1
* JavaScript/Web v1.20.1
* Kotlin v1.1.0
* Swift v1.1.0
* Node.js v0.4.0
* .NET v0.0.6-alpha.1
# Data Encryption
Source: https://docs.powersync.com/client-sdks/advanced/data-encryption
Secure client-side data with encryption in transit via TLS and at rest for SQLite databases.
### In Transit Encryption
Data is always encrypted in transit using TLS — both between the client and PowerSync, and between PowerSync [and the source database](/configuration/source-db/security-and-ip-filtering#powersync-cloud-tls-with-postgres).
### At Rest Encryption
The client-side database can be encrypted at rest. This is currently available for:
The PowerSync Dart SDK provides multiple options for client-side encryption:
1. [SQLite3MultipleCiphers](https://utelle.github.io/SQLite3MultipleCiphers), which is available both on native platforms and the web.
2. [SQLCipher Community Edition](https://www.zetetic.net/sqlcipher/), available on native platforms only.
Setting up encryption has changed in version 2.0 of the PowerSync SDK. When upgrading, follow these steps and remove dependencies on `powersync_sqlcipher`.
To enable encryption, pass an instance of `EncryptionOptions` to a `PowerSyncDatabase` constructor:
```dart theme={null}
final db = PowerSyncDatabase(
schema: schema,
path: path,
encryption: EncryptionOptions(
key: 'my secret key',
),
);
```
Additionally, encryption needs to be enabled for your target platforms.
For native platforms, add this snippet to your `pubspec.yaml`:
```yaml theme={null}
hooks:
user_defines:
sqlite3:
# Bundle and load SQLite3MultipleCiphers instead of SQLite for encryption.
source: sqlite3mc # alternatively, pass sqlcipher instead.
```
If you're using [pub workspaces](https://dart.dev/tools/pub/workspaces), this needs to be added to the root `pubspec.yaml`, not the one for your app.
Not sure whether to use SQLCipher or SQLite3MultipleCiphers? Consider this:
1. SQLite3MultipleCiphers is the only option on the web, using it for native targets ensures consistency.
2. SQLCipher can be more performant because it links OpenSSL / `Security.framework` on Apple targets instead of using a builtin implementation.
Using operating system libraries also makes it easier to comply with [export regulations](https://developer.apple.com/documentation/security/complying-with-encryption-export-regulations).
On the web, enabling encryption requires SQLite3MultipleCiphers as well. [PowerSync releases](https://github.com/powersync-ja/powersync.dart/releases)
have a `sqlite3mc.wasm` file attached to them, which can be downloaded to `web/sqlite3.wasm` to add encryption
support.
You can also run `dart run powersync:setup_web --encryption` to download the correct binary into your `web/`
directory.
The React Native SDK uses OP-SQLite, which [supports encryption](https://op-engineering.github.io/op-sqlite/docs/installation/) through [SQLCipher](https://www.zetetic.net/sqlcipher/).
For usage details, see the README of `@powersync/react-native`.
The Web SDK uses the [ChaCha20 cipher algorithm by default](https://utelle.github.io/SQLite3MultipleCiphers/docs/ciphers/cipher_chacha20/). See usage details in the package README:
Encryption support is available for PowerSync's Node.js SDK using [`better-sqlite3-multiple-ciphers`](https://www.npmjs.com/package/better-sqlite3-multiple-ciphers). See usage details and code examples in the [Node.js SDK reference](/client-sdks/reference/node#encryption-and-custom-sqlite-drivers).
Encryption support is available for PowerSync's Kotlin SDK (since version 1.9.0) using [`SQLite3MultipleCiphers`](https://utelle.github.io/SQLite3MultipleCiphers/) via the [`com.powersync:sqlite3multipleciphers`](https://central.sonatype.com/artifact/com.powersync/sqlite3multipleciphers) package. This allows you to encrypt your local SQLite database with various cipher algorithms.
**Setup:**
1. Replace your dependency on `com.powersync:core` with `com.powersync:common` of the same version.
2. Add a dependency on `com.powersync:sqlite3multipleciphers`.
3. Since `:core` includes a Ktor client implementation, you'll need to [add one manually](https://ktor.io/docs/client-engines.html) if you're not already using Ktor:
* Android/JVM: `io.ktor:ktor-client-okhttp`
* Apple targets (Kotlin/Native): `io.ktor:ktor-client-darwin`
4. Use the appropriate encrypted database factory when creating your `PowerSyncDatabase`:
```kotlin theme={null}
// Android
val database = PowerSyncDatabase(
factory = AndroidEncryptedDatabaseFactory(
context,
Key.Passphrase("your encryption key")
),
schema = yourSchema,
dbFilename = "your_database"
)
// JVM
val database = PowerSyncDatabase(
factory = JavaEncryptedDatabaseFactory(
Key.Passphrase("your encryption key")
),
schema = yourSchema,
dbFilename = "your_database"
)
// Kotlin/Native (Apple targets)
val database = PowerSyncDatabase(
factory = NativeEncryptedDatabaseFactory(
Key.Passphrase("your encryption key")
),
schema = yourSchema,
dbFilename = "your_database"
)
```
Store encryption keys securely rather than hardcoding them in your code.
For more details, see the [`sqlite3multipleciphers` README](https://github.com/powersync-ja/powersync-kotlin/tree/main/sqlite3multipleciphers) in the PowerSync Kotlin SDK repository.
If you're using PowerSync with Room, see the [Room encrypted databases section](/client-sdks/orms/kotlin/room#encrypted-databases) for the recommended setup.
Encryption support is available for PowerSync's Swift SDK (since version 1.10.0) using [`SQLite3MultipleCiphers`](https://utelle.github.io/SQLite3MultipleCiphers/). Encryption keys are configured with the `initialStatements` parameter on `PowerSyncDatabase()` which allows running `PRAGMA key` statements.
**Setup requirements:**
The PowerSync Swift SDK depends on [CSQLite](https://github.com/powersync-ja/CSQLite) to build and link SQLite.
That package can be configured to optionally link SQLite3 Multiple Ciphers by enabling the `Encryption` trait. Due to SwiftPM limitations, we can't directly expose that trait on the Swift SDK.
Instead, we recommend directly depending on CSQLite with the encryption trait, which will enable the same for the SDK (since each package can only appear in a build once). Since Xcode doesn't support specifying package traits when adding dependencies, you first need to add a local Swift package as a workaround.
1. Create a local `Package.swift` in your project that depends on CSQLite with the `Encryption` trait:
```swift theme={null}
// swift-tools-version: 6.2
import PackageDescription
let package = Package(
name: "helper",
products: [
.library(name: "helper", targets: ["helper"]),
],
dependencies: [
.package(url: "https://github.com/powersync-ja/CSQLite.git", exact: "3.51.2", traits: ["Encryption"]),
],
targets: [
.target(name: "helper", dependencies: [.product(name: "CSQLite", package: "CSQLite")]),
]
)
```
2. Add a dependency to this local package from Xcode and resolve packages. This enables `sqlite3mc` for your entire app, including the PowerSync framework.
3. Configure encryption when opening the database:
```swift theme={null}
let db = PowerSyncDatabase(
schema: yourSchema,
initialStatements: ["pragma key = 'your encryption key'"]
)
```
Store encryption keys securely (e.g., in Keychain) rather than hardcoding them in your code.
For a complete working example, see the [SwiftEncryptionDemo](https://github.com/powersync-ja/powersync-swift/tree/main/Demos/SwiftEncryptionDemo) in the PowerSync Swift SDK repository.
Support for encryption on other platforms is planned. In the meantime, let us know your needs and use cases on [Discord](https://discord.gg/powersync).
### End-to-end Encryption
For end-to-end encryption, the encrypted data can be synced using PowerSync. The data can then either be encrypted and decrypted directly in memory by the application, or a separate local-only table can be used to persist the decrypted data — allowing querying the data directly.
[Raw SQLite Tables](/client-sdks/advanced/raw-tables) can be used for full control over the SQLite schema and managing tables for the decrypted data. We have a [React & Supabase example app](https://github.com/powersync-community/react-supabase-chat-e2ee) that demonstrates this approach. See also the accompanying [blog post](https://www.powersync.com/blog/building-an-e2ee-chat-app-with-powersync-supabase).
## See Also
* Database Setup → [Security & IP Filtering](/configuration/source-db/security-and-ip-filtering)
* Resources → [Security](/resources/security)
# GIS Data: PostGIS
Source: https://docs.powersync.com/client-sdks/advanced/gis-data-postgis
Sync PostGIS geographic data through PowerSync and query it on the client using SQLite spatial tools.
Custom types, arrays and [PostGIS](https://postgis.net/) are frequently presented together since geospatial data is often complex and multidimensional. It's therefore recommend to first quickly scan the content in [Custom Types, Arrays and JSON](/client-sdks/advanced/custom-types-arrays-and-json)
### PostGIS
In Supabase, the PostGIS extension needs to be added to your project to use this type. Run the following command in the SQL editor to include the PostGIS extension:
```sql theme={null}
CREATE extension IF NOT EXISTS postgis;
```
The `geography` and `geometry` types are now available in your Postgres.
## Supabase Configuration Example:
This example builds on the To-Do List demo app in our [Supabase integration guide](/integrations/supabase/guide).
### Add Custom Type, Array and PostGIS Columns to the `todos` Table
```sql theme={null}
--SQL command to update the todos table with 3 additional columns:
ALTER TABLE todos
ADD COLUMN address location_address null,
ADD COLUMN contact_numbers text [] null,
ADD COLUMN location geography (point) null
```
### Insert a Row of Data Into the Table
```sql theme={null}
-- Grab the id of a list object and a user id and create a new todos
INSERT INTO public.todos(description, list_id, created_by, address, location, contact_numbers) VALUES ('Bread', 'list_id', 'user_id', '("1000 S Colorado Blvd.","Denver","CO",80211)', st_point(39.742043, -104.991531), '{000-000-0000, 000-000-0000, 000-000-0000}');
```
Note the following:
**Custom type**: Specify the value for the `address` column by wrapping the value in single quotes and comma separate the different location\_address properties.
* `'("1000 S Colorado Blvd.","Denver","CO",80211)'`
**Array**: Specify the value of the `contact_numbers` column, by surrounding the comma-separated array items in curly braces.
* `'{000-000-0000, 000-000-0000, 000-000-0000}'`
**PostGIS**: Specify the value of the `location` column by using the `st_point` function and pass in the latitude and longitude
* `st_point(39.742043, -104.991531)`
### What This Data Looks Like in Postgres
Postgres' internal binary representation of the PostGIS type is as follows:
| location |
| -------------------------------------------------- |
| 0101000020E6100000E59CD843FBDE4340E9818FC18AC052C0 |
## On the Client
### AppSchema Example
```js theme={null}
const todos = new Table(
{
list_id: column.text,
created_at: column.text,
completed_at: column.text,
description: column.text,
completed: column.integer,
created_by: column.text,
completed_by: column.text,
address: column.text,
contact_numbers: column.text,
location: column.text
},
{ indexes: { list: ['list_id'] } }
);
const lists = new Table({
created_at: column.text,
name: column.text,
owner_id: column.text
});
export const AppSchema = new Schema({
todos,
lists
});
```
Note:
* The custom type, array and PostGIS type have been defined as `TEXT` in the AppSchema. The Postgres PostGIS capabilities are not available because the PowerSync SDK uses SQLite, which only has a limited number of types. This means that everything is replicated into the SQLite database as TEXT values.
* Depending on your application, you may need to implement functions in the client to parse the values and then other functions to write them back to the Postgres database.
### What Does the Data Look Like in SQLite?
The data looks exactly how it’s stored in the Postgres database i.e.
1. **Custom Type**: It has the same format as if you inserted it using a SQL statement, i.e.
1. `(1000 S Colorado Blvd.,Denver,CO,80211)`
2. **Array**: Array types act similar in that it shows the data in the same way it was inserted e.g
1. `{000-000-0000, 000-000-0000, 000-000-0000}`
3. **PostGIS**: The `geography` type is transformed into an encoded form of the value.
1. If you insert coordinates as `st_point(39.742043, -104.991531)` then it is shown as `0101000020E6100000E59CD843FBDE4340E9818FC18AC052C0`
## Sync Streams
### PostGIS
Example use case: Extract x (long) and y (lat) values from a PostGIS type, to use these values independently in an application.
PowerSync supports the following PostGIS functions in Sync Streams (or legacy Sync Rules): [Operators and Functions](/sync/supported-sql#functions)
1. `ST_AsGeoJSON`
2. `ST_AsText`
3. `ST_X`
4. `ST_Y`
IMPORTANT NOTE: These functions will only work if your Postgres instance has the PostGIS extension installed and you’re storing values as type `geography` or `geometry`.
```yaml theme={null}
config:
edition: 3
streams:
global:
queries:
- SELECT * FROM lists
- SELECT *, st_x(location) as longitude, st_y(location) as latitude FROM todos
```
```yaml theme={null}
bucket_definitions:
global:
data:
- SELECT * FROM lists
- SELECT *, st_x(location) as longitude, st_y(location) as latitude from todos
```
# Local-Only Usage
Source: https://docs.powersync.com/client-sdks/advanced/local-only-usage
Use PowerSync for local-only data persistence without backend sync or authentication.
In some of those cases, the user may want to register and start syncing data with other devices or users at a later point, while other users may keep on using the app without ever registering or going online."
PowerSync supports these scenarios. By default, all local changes will be stored in the upload queue, and will be uploaded to the backend server if the user registers at a later point.
A caveat is that if the user never registers, this queue will keep on growing in size indefinitely. For many applications this should be small enough to not be significant, but some data-intensive applications may want to avoid the indefinite queue growth.
There are two general approaches we recommend for this:
### 1. Local-Only Tables
```dart theme={null}
final table = Table.localOnly(
...
)
```
**Flutter + Drift users:** If you're using local-only tables with `viewName` overrides, Drift's watch streams may not update correctly. See the [troubleshooting guide](/client-sdks/orms/flutter-orm-support#troubleshooting-watch-streams-with-local-only-tables) for the solution.
```js theme={null}
const lists = new Table({
...
}, {
localOnly: true
});
```
```kotlin theme={null}
val Table = Table(
...
localOnly = true
)
```
```swift theme={null}
let table = Table(
...
localOnly: true
)
```
```csharp theme={null}
public static Table Todos = new Table
{
Name = "todos",
Columns =
{
// ... column definitions ...
},
LocalOnly = true
};
```
Example not yet available.
Use local-only tables until the user has registered or signed in. This would not store any data in the upload queue, avoiding any overhead or growth in database size.
Once the user registers, move the data over to synced tables, at which point the data would be placed in the upload queue.
The following example implementations are available:
| Client framework | Link |
| -------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------- |
| Flutter To-Do List App (with Supabase) | [supabase-todolist-optional-sync](https://github.com/powersync-ja/powersync.dart/tree/main/demos/supabase-todolist-optional-sync) |
| React To-Do List App (with Supabase) | [react-supabase-todolist-optional-sync](https://github.com/powersync-ja/powersync-js/tree/main/demos/react-supabase-todolist-optional-sync) |
### 2. Clearing the Upload Queue
The upload queue can be cleared periodically (for example on every app start-up), avoiding the growth in database size over time. This can be done using:
```sql theme={null}
DELETE FROM ps_crud
```
It is up to the application to then re-create the queue when the user registers, or upload data directly from the existing tables instead.
A small amount of metadata per row is also stored in the `ps_oplog` table. We do not recommend deleting this data, as it can cause or hide consistency issues when later uploading the data. If the overhead in `ps_oplog` is too much, rather use the local-only tables approach.
### Local-Only Columns on Synced Tables
If you need individual local-only columns on a table that is otherwise synced (rather than an entirely local-only table), this can be achieved with [raw tables](/client-sdks/advanced/raw-tables#local-only-columns).
# SQLite Performance Optimization
Source: https://docs.powersync.com/client-sdks/advanced/performance-optimization
Optimize query performance in PowerSync client SDKs
PowerSync client SDKs use SQLite, and apply reasonable defaults to optimize performance:
1. For all native targets, [write-ahead logging](https://www.sqlite.org/wal.html) (WAL) is enabled to support concurrent reads and writes.
PowerSync also dispatches queries to multiple threads for parallelism.
With the JavaScript web SDK, a WAL-like option is [available on Chromium browsers](/client-sdks/reference/javascript-web#2-opfs-based-alternatives).
2. SDKs also configure connections for performance by enabling a large page cache of around 50MB and setting `pragma synchronous = normal` to avoid frequent fsync operations.
## Debugging Query Performance
Slow queries can have two causes:
1. The query itself is slow, e.g. because it reads a lot of rows or uses unoptimized joins not backed by an index.
The [Fixing Slow Queries](#fixing-slow-queries) section describes how these queries can be optimized.
2. The database is blocked. There can only be a single writer at a time, and the number of concurrent readers is bound
by an option set when opening the database. So a query that appears slow might actually be fast, having run
at an unfortunate time when the database was busy.
To understand the problem, having a trace of query runtimes available helps.
Enabling the `debugMode` flag in the [Web SDK](/client-sdks/reference/javascript-web) logs all SQL queries on the Performance timeline in Chrome's Developer Tools (after recording). This can help identify slow-running queries.
With the Dart SDK, queries are logged to the [Performance View in DevTools](https://docs.flutter.dev/tools/devtools/performance) by default outside of release builds.
```typescript TypeScript theme={null}
const db = new PowerSyncDatabase({
schema,
database: {
dbFilename: 'my_database.db',
// Cache up to 64 prepared statements
preparedStatementsCache: 64,
},
});
```
```dart Dart theme={null}
final db = PowerSyncDatabase(
schema: Schema([...]),
path: 'my_database.db',
sqliteOptions: SqliteOptions(
// Cache up to 64 prepared statements
preparedStatementCacheSize: 64,
),
);
```
In both SDKs, each connection uses its own independent cache and the maximum size applies to those caches.
When the cache is full, the least-recently-used statement is evicted.
For more information, see the `preparedStatementsCache` option [for JavaScript](https://powersync-ja.github.io/powersync-js/web-sdk/globals#webspecificopenoptions)
or [for Dart](https://pub.dev/documentation/powersync/latest/sqlite_async/SqliteOptions/preparedStatementCacheSize.html).
# Pre-Seeding SQLite Databases
Source: https://docs.powersync.com/client-sdks/advanced/pre-seeded-sqlite
Speed up initial sync by pre-seeding SQLite databases with baseline data so users can start working immediately after app installation.
## Overview
When syncing large amounts of data to connected clients, it can be useful to pre-seed the SQLite database with an initial snapshot of the data. This can help to reduce the initial sync time and improve the user experience.
To achieve this, you can run server-side processes using the [PowerSync Node.js SDK](/client-sdks/reference/node) to pre-seed SQLite files. These SQLite files can then be uploaded to blob storage providers such as AWS S3, Azure Blob Storage, or Google Cloud Storage and downloaded directly by client applications. Client applications can then initialize the pre-seeded SQLite file, effectively bypassing the initial sync process.
### Demo App
If you're interested in seeing an end-to-end example, we've prepared a demo repo that can be used as a template for your own implementation. This repo covers all of the key concepts and code examples shown in this page.
Self-hosted PowerSync instance connected to a Postgres database, using the PowerSync Node.js SDK, React Native SDK and AWS S3 for storing the pre-seeded SQLite files.
## Main Concepts
### Generate a Scoped JWT Token
In most cases you'd want to pre-seed the SQLite database with user specific data and not all data from the source database, as you normally would when using PowerSync. For this you would need to generate JWT tokens that include the necessary properties to satisfy the conditions of the queries in your Sync Streams (or legacy Sync Rules).
Let's say we have the following Sync Config:
```yaml theme={null}
sync_config:
content: |
config:
edition: 3
streams:
store_products:
query: SELECT * FROM products WHERE store_id = auth.parameter('store_id')
```
```yaml theme={null}
sync_config:
content: |
bucket_definitions:
store_products:
parameters: SELECT id as store_id FROM stores WHERE id = request.jwt() ->> 'store_id'
data:
- SELECT * FROM products WHERE store_id = bucket.store_id
```
In the example above the `store_id` is part of the JWT payload and is used to filter products by store for a user. Given this we would want to do the following:
1. Query the source database, directly from the Node.js application, for all the store ids you'd want a pre-seeded SQLite database for.
2. Generate a JWT token for each store and include the `store_id` in the payload.
3. In the Node.js application which implements the PowerSync SDK, return the JWT token in the `fetchCredentials()` function.
This will ensure that only the data for a specific store is pre-seeded into the SQLite database.
Here's an example of a function that generates a JWT token based on the `store_id` using the [`jose`](https://github.com/panva/jose) library:
```typescript theme={null}
import * as jose from 'jose';
export const generateToken = async (subject: string, store_id: string) => {
return await new jose.SignJWT({store_id: store_id}) // Set the store_id in the payload
.setProtectedHeader({ alg: 'HS256', kid: "My Kid" })
.setSubject(subject)
.setIssuedAt(new Date())
.setAudience('powersync')
.setExpirationTime('1h')
.sign(Buffer.from("My Base64 Encoded Secret", 'base64url'));
};
```
### Pre-seeding Script
Once you've got a plan in place for generating the JWT tokens, you can write a simple script to connect to the PowerSync instance and pre-seed the SQLite database. Here's an example of a script that does this:
```typescript theme={null}
async function prepareDatabase (storeId: string) {
const backupPath = `/path/to/sqlite/${storeId}.sqlite`;
const connector = new Connector();
await powersync.connect(connector);
await powersync.waitForFirstSync();
const result = await powersync.execute("DELETE FROM ps_kv WHERE key = ?", ["client_id"]);
const vacuumResult = await powersync.execute(`VACUUM INTO ${backupPath}`);
await uploadFile(storeId, `${storeId}.sqlite`, backupPath);
await powersync.close();
await powersync.disconnect();
}
```
Some critical points to note:
* You will need to wait for the first sync to complete before deleting the `client_id` key and vacuuming the database. This makes sure all of the data is synced to the database before we proceed.
* The `client_id` key is used to identify the client device and is typically set when the client connects to the PowerSync instance. So when pre-seeding the database, we need to delete the `client_id` key to avoid conflicts when the client connects to the PowerSync instance.
* It's important to note that you will need to use the [`VACUUM INTO`](https://sqlite.org/lang_vacuum.html) command to create a clean, portable SQLite database file. This will help to reduce the size of the database file and provide an optimized version for the client to download.
* In this example the upload function is using AWS S3, but you can use any blob storage provider that you prefer.
#### Scheduling and Cleaning Up
To enhance the process you can consider doing the following:
* To keep the pre-seeded SQLite databases fresh, schedule a CRON jobs for periodic regeneration, ensuring that new clients always download the latest snapshot of the initial sync data.
* After each run, perform some environment cleanup to avoid disk bloat. This can be done by deleting the pre-seeded SQLite database files after they have been uploaded to the blob storage provider.
### Client-Side Usage
When the client application boots, before connecting to the PowerSync instance, check if a SQLite database exists in the application's permanent storage. If it does, use it, else download a pre-seeded SQLite database from the blob storage provider.
Here's an example of a function that checks if a file exists in the application's permanent storage:
```typescript theme={null}
import { File, Paths } from 'expo-file-system/next';
export const FilePath = `${Paths.document.uri}`;
export const fileExists = (storeId: string) => {
const file = new File(FilePath, `${storeId}.sqlite`);
return file.exists;
}
```
Here's an example of a function that downloads the pre-seeded SQLite database from the blob storage provider:
```typescript theme={null}
export const downloadFile = async (storeId: string) => {
// Retrieve a pre-signed URL from the server that allows the client to download the file.
const response = await fetch(`https://your-api-url.com/database?store_id=${storeId}`);
const { databaseUrl } = await response.json();
// Download the file to the permanent location on the device.
const newFile = new File(FilePath, `${storeId}.sqlite`);
await File.downloadFileAsync(databaseUrl, newFile);
}
```
It's important to note that when the client downloads the pre-seeded SQLite database that it's stored in a permanent location on the device. This means that the database will not be deleted when the app is restarted.
Depending on which PowerSync SDK you are using, you may need to use framework specific methods to store the file in a permanent location on the device. For example, with React Native + Expo you can use the [`expo-file-system`](https://docs.expo.dev/versions/latest/sdk/filesystem/) module to store the file in a permanent location on the device.
Once the database is downloaded, initialize the `PowerSyncDatabase` class with the file path and connect to the PowerSync instance.
```typescript theme={null}
import { PowerSyncDatabase } from '@powersync/react-native';
import { AppSchema } from './Schema';
// databasePath is the path to the pre-seeded SQLite database file on the device.
export const configureDatabase = async (storeId: string) => {
const powersync = new PowerSyncDatabase({
schema: AppSchema,
database: {
dbFilename: `${storeId}.sqlite`,
dbLocation: FilePath.replace('file://', '')
},
});
// Call init() first, this will ensure the database is initialized, but not connected to the PowerSync instance.
await powersync.init();
// Insert a new `client_id` key into the `ps_kv` table to avoid conflicts when the client connects to the PowerSync instance.
await powersync.execute("INSERT INTO ps_kv (key, value) VALUES (?, ?)", ["client_id", "1234567890"]);
// Connect to the PowerSync instance.
await powersync.connect(connector);
}
```
It's important that you insert a new `client_id` key into the `ps_kv` table to avoid conflicts when the client connects to the PowerSync instance.
At this point the client would connect to the PowerSync instance and sync the data from where the pre-seeded snapshot was created, bypassing the initial sync process.
# Querying JSON Data in SQLite
Source: https://docs.powersync.com/client-sdks/advanced/query-json-in-sqlite
How to query JSON data synced from your backend and stored as strings in SQLite using `json_extract` and JSON operators.
## Overview
When syncing data from your backend source database to PowerSync, JSON columns (whether from MongoDB documents, Postgres JSONB columns, or other JSON data types) are stored as `TEXT` in SQLite. See the [type mapping guide](/sync/types) for more details. This guide shows you how to effectively query and filter JSON data using SQLite's powerful JSON functions on the client.
### Understanding JSON Storage in PowerSync
Your backend source database might store structured data as JSON in various ways:
* **MongoDB**: Nested documents and arrays
* **Postgres**: JSONB, JSON, array, or custom types
* **MySQL**: JSON columns
* **SQL Server**: JSON columns
Regardless of the source, PowerSync syncs these JSON structures to SQLite as `TEXT` columns. On the client side, you can query this data using SQLite's built-in JSON functions without needing to parse it yourself. Learn more about [how PowerSync handles JSON, arrays, and custom types](/client-sdks/advanced/custom-types-arrays-and-json).
### Example Data Structure
Let's use a task management system where tasks have nested metadata:
```json theme={null}
{
"id": "task_123",
"title": "Redesign homepage",
"assignees": [
{
"user_id": "user_001",
"role": "designer",
"hours_allocated": 20
},
{
"user_id": "user_002",
"role": "developer",
"hours_allocated": 40
}
],
"tags": ["urgent", "frontend", "design"],
"metadata": {
"priority": 1,
"sprint": "2024-Q1",
"dependencies": ["task_100", "task_101"]
}
}
```
In SQLite, the `assignees`, `tags`, and `metadata` columns are stored as JSON strings. For details on how different backend types map to SQLite, see [database types and mapping](/sync/types).
### JSON Extraction Basics
#### Standard [`json_extract()`](https://sqlite.org/json1.html#jex) Function
Extract values from JSON using path expressions:
```sql theme={null}
SELECT
id,
title,
json_extract(metadata, '$.priority') AS priority,
json_extract(metadata, '$.sprint') AS sprint
FROM tasks;
```
**Path syntax:**
* `$` - root element
* `.` - object member access
* `[index]` - array element access
#### Shorthand: The -> and ->> Operators
SQLite provides convenient [shorthand operators](https://sqlite.org/json1.html#jptr) for JSON extraction:
```sql theme={null}
SELECT
id,
title,
metadata -> '$.dependencies' AS dependencies_array, -- maintains JSON array
metadata ->> '$.sprint' AS sprint
FROM tasks;
```
**The difference between [-> and ->>](https://sqlite.org/json1.html#jptr):**
* `->` returns JSON (preserves type information, quotes strings)
* `->>` extracts the value unquoted (strings as TEXT, numbers/booleans as their native types)
```sql theme={null}
-- Using -> (returns JSON)
SELECT metadata -> '$.priority' FROM tasks;
-- Result: 1 (as a SQLite TEXT value)
-- Using ->> (returns parsed value)
SELECT metadata ->> '$.priority' FROM tasks;
-- Result: 1 (as a SQLite INTEGER value)
-- For strings, the difference is clearer:
SELECT metadata -> '$.sprint' FROM tasks;
-- Result: "2024-Q1" (with quotes, as JSON)
SELECT metadata ->> '$.sprint' FROM tasks;
-- Result: 2024-Q1 (without quotes, as text)
```
**When to use which:**
* Use `->>` when extracting **final values** for display or comparison
* Use `->` when extracting **intermediate JSON** for further processing
* `->>` preserves data types (numbers stay numbers, not strings)
#### Nested Path Access
Access deeply nested values:
```sql theme={null}
-- All three are equivalent:
json_extract(metadata, '$.dependencies[0]')
metadata -> '$.dependencies[0]'
metadata -> '$.dependencies' -> '$[0]'
```
### Querying Arrays with [`json_each()`](https://sqlite.org/json1.html#jeach)
#### Flattening Simple Arrays
For the `tags` array, use `json_each()` to create one row per element:
```sql theme={null}
SELECT
t.id,
t.title,
tag.value AS tag
FROM tasks t,
json_each(t.tags) AS tag
WHERE tag.value = 'urgent';
```
**What's happening:**
* `json_each(t.tags)` creates a virtual table with one row per tag
* `tag.value` contains each individual tag string
* You can filter, join, or aggregate these expanded rows
#### Querying Nested Objects in Arrays
For complex objects like `assignees`:
```sql theme={null}
SELECT
t.id,
t.title,
assignee.value ->> '$.user_id' AS user_id,
assignee.value ->> '$.role' AS role,
assignee.value -> '$.hours_allocated' AS hours
FROM tasks t,
json_each(t.assignees) AS assignee
WHERE (assignee.value ->> '$.role') = 'developer';
```
**Key points:**
* Each `assignee.value` is a JSON object representing one assignee
* Use `->>` to extract text values for comparison
* Use `->` when you need numeric values for calculations
### Real-World Query Examples
#### Example 1: Finding Tasks by Assignee
**Use case:** Show all tasks assigned to a specific user.
```sql theme={null}
SELECT DISTINCT
t.id,
t.title,
t.metadata ->> '$.priority' AS priority
FROM tasks t,
json_each(t.assignees) AS assignee
WHERE (assignee.value ->> '$.user_id') = 'user_001'
ORDER BY t.metadata ->> '$.priority';
```
#### Example 2: Calculating Total Hours by Role
**Use case:** Aggregate hours across all tasks grouped by role.
```sql theme={null}
SELECT
assignee.value ->> '$.role' AS role,
SUM(assignee.value ->> '$.hours_allocated') AS total_hours,
COUNT(DISTINCT t.id) AS task_count
FROM tasks t,
json_each(t.assignees) AS assignee
GROUP BY role
ORDER BY total_hours DESC;
```
#### Example 3: Tasks with Specific Tags
**Use case:** Find tasks tagged with multiple specific tags.
```sql theme={null}
-- Tasks with BOTH 'urgent' AND 'frontend' tags
SELECT DISTINCT t.*
FROM tasks t
WHERE EXISTS (
SELECT 1 FROM json_each(t.tags)
WHERE value = 'urgent'
)
AND EXISTS (
SELECT 1 FROM json_each(t.tags)
WHERE value = 'frontend'
);
```
Or using a simpler approach for single tags:
```sql theme={null}
-- Tasks with 'urgent' tag
SELECT *
FROM tasks t,
json_each(t.tags) AS tag
WHERE tag.value = 'urgent';
```
#### Example 4: Filtering by Array Contents
**Use case:** Find tasks that depend on a specific task ID.
```sql theme={null}
SELECT *
FROM tasks t,
json_each(t.metadata -> '$.dependencies') AS dep
WHERE dep.value = 'task_100';
```
#### Example 5: Checking for Array Membership
**Use case:** Check if a task has any dependencies.
```sql theme={null}
SELECT
id,
title,
json_array_length(metadata -> '$.dependencies') AS dep_count
FROM tasks
WHERE json_array_length(metadata -> '$.dependencies') > 0;
```
### Working with Comma or Delimiter-Separated Values
Sometimes JSON strings contain delimiter-separated values (like `"NYC;LAX;MIA"`). Here's how to query them efficiently:
```sql theme={null}
-- Assume tasks have a field: "approved_by": "user_001;user_002;user_003"
-- Find tasks approved by a specific user
SELECT *
FROM tasks
WHERE instr(
';' || (metadata ->> '$.approved_by') || ';',
';user_001;'
) > 0;
```
**Why this pattern works:**
* Wraps the value: `";user_001;user_002;user_003;"`
* Searches for `;user_001;` ensuring exact delimiter-bounded match
* Prevents false matches (won't match "user\_0011" when searching for "user\_001")
**Avoid `LIKE` for delimited strings:**
```sql theme={null}
-- ❌ WRONG - can match partial values
WHERE (metadata ->> '$.approved_by') LIKE '%user_001%'
-- This would incorrectly match "user_0011" or "user_001_archive"
-- ✅ CORRECT - exact delimiter match
WHERE instr(';' || (metadata ->> '$.approved_by') || ';', ';user_001;') > 0
```
### Advanced Techniques
#### Using CTEs for Cleaner Queries
Common Table Expressions make complex JSON queries more readable:
```sql theme={null}
WITH task_assignees AS (
SELECT
t.id,
t.title,
assignee.value ->> '$.user_id' AS user_id,
assignee.value ->> '$.role' AS role,
assignee.value ->> '$.hours_allocated' AS hours
FROM tasks t,
json_each(t.assignees) AS assignee
)
SELECT
user_id,
role,
SUM(hours) AS total_hours,
COUNT(*) AS assignment_count
FROM task_assignees
WHERE hours > 10
GROUP BY user_id, role;
```
#### Combining Multiple JSON Arrays
Query across multiple nested arrays:
```sql theme={null}
SELECT DISTINCT
t.id,
t.title,
assignee.value ->> '$.user_id' AS assigned_to,
tag.value AS tag
FROM tasks t,
json_each(t.assignees) AS assignee,
json_each(t.tags) AS tag
WHERE tag.value IN ('urgent', 'high-priority')
AND assignee.value ->> '$.role' = 'developer';
```
**Cartesian product warning:** When using multiple `json_each()` calls, you create a Cartesian product. A task with 3 assignees and 4 tags creates 12 rows. Use `DISTINCT` when needed and filter early to minimize row expansion.
#### Checking for Key Existence
Verify if a JSON key exists:
```sql theme={null}
-- Check if 'sprint' key exists
SELECT *
FROM tasks
WHERE json_extract(metadata, '$.sprint') IS NOT NULL;
-- Or using shorthand
SELECT *
FROM tasks
WHERE metadata -> '$.sprint' IS NOT NULL;
```
### Performance Optimization
**Important Performance Considerations**
1. **Index JSON columns for better performance**: If you frequently query JSON fields, add indexes to the JSON string columns in your `AppSchema`:
```typescript theme={null}
const tasks = new Table(
{
id: column.text,
title: column.text,
metadata: column.text,
tags: column.text,
},
{
indexes: {
tagsIndex: ['tags']
}
}
);
```
2. **Minimize `json_each()` usage**: Each `json_each()` call expands rows. For a table with 10,000 tasks averaging 5 assignees each, you're processing 50,000 rows.
3. **Use EXISTS for membership checks**: More efficient than joining:
```sql theme={null}
-- ✅ BETTER for large datasets
SELECT * FROM tasks t
WHERE EXISTS (
SELECT 1 FROM json_each(t.tags) WHERE value = 'urgent'
);
-- vs joining which creates all row combinations
```
4. **Cache extracted values in CTEs**: Extract once, use multiple times:
```sql theme={null}
WITH task_metrics AS (
SELECT
t.id,
t.title,
t.metadata,
COUNT(assignee.value) AS assignee_count,
SUM(assignee.value ->> '$.hours_allocated') AS total_hours
FROM tasks t,
json_each(t.assignees) AS assignee
GROUP BY t.id, t.title, t.metadata
)
SELECT *
FROM task_metrics
WHERE metadata ->> '$.sprint' = '2024-Q1'
AND assignee_count > 1
ORDER BY total_hours DESC;
```
### Useful JSON Functions
Beyond extraction, SQLite offers many JSON utilities:
```sql theme={null}
-- Get array length
SELECT json_array_length(tags) FROM tasks;
-- Check JSON validity
SELECT json_valid(metadata) FROM tasks;
-- Get all object keys
SELECT json_each.key, json_each.value
FROM tasks,
json_each(tasks.metadata)
WHERE id = 'task_123';
-- Get JSON type of a value
SELECT json_type(metadata -> '$.priority') FROM tasks;
-- Returns: 'integer', 'text', 'array', 'object', 'null', etc.
-- Aggregate JSON arrays
SELECT json_group_array(tag.value)
FROM tasks t,
json_each(t.tags) AS tag
WHERE t.id = 'task_123';
```
### Common Gotchas
**Watch out for these common issues:**
1. **NULL vs missing keys**: `json_extract()` returns `NULL` for non-existent paths. Always check for NULL:
```sql theme={null}
WHERE COALESCE(metadata ->> '$.priority', 999) = 1
```
2. **Type mismatches**:
```sql theme={null}
-- ❌ String comparison (wrong!)
WHERE metadata -> '$.priority' > 5
-- ✅ BEST: Use ->> for direct numeric extraction
WHERE metadata ->> '$.priority' > 5
```
3. **Array index bounds**: Out-of-bounds array access returns NULL, not an error:
```sql theme={null}
SELECT metadata -> '$.dependencies[99]' -- Returns NULL if not enough elements
```
4. **Quotes in JSON strings**: Use `->>` to get unquoted text, not `->`:
```sql theme={null}
-- ❌ Returns: "2024-Q1" (with quotes)
WHERE metadata -> '$.sprint' = '2024-Q1'
-- ✅ Returns: 2024-Q1 (without quotes)
WHERE metadata ->> '$.sprint' = '2024-Q1'
```
5. **Performance on large arrays**: `json_each()` on arrays with thousands of elements can be slow. Consider data restructuring for such cases.
### Summary
Querying JSON data in SQLite effectively requires:
* Understanding that JSON is stored as strings but queryable with built-in functions
* Using `json_extract()` or the shorthand `->` and `->>` operators
* Leveraging `json_each()` to flatten arrays for filtering and aggregation
* Being mindful of type conversions and NULL handling
* Optimizing queries by filtering early and considering denormalization for critical paths
With these techniques, you can query complex nested data structures synced from your backend while maintaining good performance on mobile and edge devices.
For complete SQLite JSON function reference, see the [SQLite JSON documentation](https://www.sqlite.org/json1.html).
# Raw SQLite Tables to Bypass JSON View Limitations
Source: https://docs.powersync.com/client-sdks/advanced/raw-tables
Access raw SQLite tables to bypass JSON view limitations for native SQLite functionality and better query performance.
Raw tables are not currently compatible with the [High Performance Diffs](/client-sdks/high-performance-diffs) feature available in our JavaScript SDKs. We have a [public proposal](https://docs.google.com/document/d/12bvZDJF2aaTOqkxHyo3W-TYGQTaS9FQFdu-MX2yOuqI/edit) open to discuss this addition - your feedback is welcome!
By default, PowerSync uses a [JSON-based view system](/architecture/client-architecture#client-side-schema-and-sqlite-database-structure) where data is stored schemalessly in JSON format and then presented through SQLite views based on the client-side schema. Raw tables allow you to define native SQLite tables in the client-side schema, bypassing this.
This eliminates overhead associated with extracting values from the JSON data and provides access to advanced SQLite features like foreign key constraints and custom indexes.
**Availability**
Features describes on this page were introduced in the following versions of our client SDKs:
* **JavaScript** (Node: `0.18.0`, React-Native: `1.31.0`, Web: `1.35.0`)
* **Dart**: Version 1.18.0 of `package:powersync`.
* **Kotlin**: Version 1.11.0.
* **Swift**: Version 1.12.0.
* **Rust**: Version 0.0.4.
* This feature is not yet available on our .NET SDK.
## When to Use Raw Tables
Consider raw tables when you need:
* **Indexes** - PowerSync's default schema has basic support for indexes on columns, while raw tables give you complete control to create indexes on expressions, use `GENERATED` columns, etc.
* **Improved performance** for complex queries (e.g., `SELECT SUM(value) FROM transactions`) - raw tables more efficiently get these values directly from the SQLite column, instead of extracting the value from the JSON object on every row.
* **Reduced storage overhead** - eliminate JSON object overhead for each row in `ps_data__.data` column.
* **To manually create tables** - Sometimes you need full control over table creation, for example when implementing custom triggers.
**Advanced SQLite features** like `FOREIGN KEY` and `ON DELETE CASCADE` constraints need [special consideration](#using-foreign-keys).
## How Raw Tables Work
### Current JSON-Based System
Currently the sync system involves two general steps:
1. Download bucket operations from the PowerSync Service.
2. Once the client has a complete checkpoint and no pending local changes in the upload queue, sync the local database with the bucket operations.
The bucket operations use JSON to store the individual operation data. The local database uses tables with a simple schemaless `ps_data__` structure containing only an `id` (TEXT) and `data` (JSON) column.
PowerSync automatically creates views on that table that extract JSON fields to resemble standard tables reflecting your schema.
### Raw Tables Approach
When opting in to raw tables, you are responsible for creating the tables before using them - PowerSync will no longer create them automatically.
Because PowerSync takes no control over raw tables, you need to manually:
1. Define how PowerSync's [schemaless protocol](/architecture/powersync-protocol#protocol) maps to your raw tables — see [Define sync mapping for raw tables](#define-sync-mapping-for-raw-tables)
2. Define triggers that capture local writes from raw tables — see [Capture local writes with triggers](#capture-local-writes-with-triggers)
For the purpose of this example, consider a simple table like this:
```sql theme={null}
CREATE TABLE todo_lists (
id TEXT NOT NULL PRIMARY KEY,
created_by TEXT NOT NULL,
title TEXT NOT NULL,
content TEXT
) STRICT;
```
### Define Sync Mapping for Raw Tables
To sync into the raw `todo_lists` table instead of `ps_data__`, PowerSync needs the SQL statements extracting columns from the untyped JSON protocol used during syncing.
Internally, this involves two SQL statements:
1. A `put` SQL statement for upserts, responsible for creating a `todo_list` row or updating it based on its `id` and data columns.
2. A `delete` SQL statement responsible for deletions.
The PowerSync client as part of our SDKs will automatically run these statements in response to sync lines being sent from the PowerSync Service.
In most cases, these statements can be inferred automatically. However, the statements can also be given explicitly if customization is needed.
#### Inferring Sync Statements
In most cases, the `put` and `delete` statements are obvious when looking at the structure of the table.
With the `todo_list` example, a delete statement would `DELETE FROM todo_lists WHERE id = $row_id_to_delete`.
Similarly, a `put` statement would use a straightforward upsert to create or update rows.
When the SDK knows the name of the local table you're inserting into, it can infer statements automatically
by analyzing the `CREATE TABLE` structure.
The name of raw tables can be provided with the `RawTableSchema` type:
```javascript JavaScript theme={null}
// Raw tables are not included in the regular Schema() object.
// Instead, add them afterwards using withRawTables().
const mySchema = new Schema({
// Define your PowerSync-managed schema here
// ...
});
mySchema.withRawTables({
todo_lists: {
schema: {},
}
});
```
```dart Dart theme={null}
// Raw tables are not part of the regular tables list and can be defined with the optional rawTables parameter.
const schema = Schema([], rawTables: [
RawTable.inferred(
name: 'todo_lists',
schema: RawTableSchema(),
),
]);
```
```kotlin Kotlin theme={null}
// To define a raw table, include it in the list of tables passed to the Schema
val schema = Schema(listOf(
RawTable(
name = "todo_lists",
schema = RawTableSchema(),
)
))
```
```swift Swift theme={null}
// To define a raw table, include it in the list of tables passed to the Schema
let lists = RawTable(
name: "todo_lists",
schema: RawTableSchema()
)
let schema = Schema(lists)
```
```csharp .NET theme={null}
Unfortunately, raw tables are not yet available in the .NET SDK.
```
```rust Rust theme={null}
use powersync::schema::{RawTable, RawTableSchema, Schema};
pub fn app_schema() -> Schema {
let mut schema = Schema::default();
let table = RawTable::with_schema("todo_lists", RawTableSchema::default());
schema.raw_tables.push(table);
schema
}
```
**When to use inferred statements**
If you have a local table that directly corresponds to the schema of a synced output table,
inferred statements greatly simplify the schema setup.
You will need explicit sync statements if, for instance:
* you want to apply transformations on synced values before inserting them into your local database.
* you need custom default values for synced `NULL` values.
* you're using the [rest column pattern](#the-_extra-column-pattern) to help with migrations.
* you have a custom setup where a raw table stores data from multiple source tables.
If the name of the SQLite table and the name of the synced table aren't the same, the inferred
statements can be customized.
For instance, say you had a `local_users` table in your SQLite database and want to sync rows
from the `users` table in your backend.
Here, the name of the raw table must be `users` to match PowerSync definitions, but the `RawTableSchema`
type on every SDK has an optional `tableName` field that can be set to `local_users` in this case.
#### Explicit Sync Statements
To pass statements explicitly, use the `put` and `delete` parameters available in each SDK.
A statement consists of two parts:
1. An SQL string of the statement to run. It should use positional parameters (`?`) as placeholders for values from the synced row.
2. An array describing the instantiation of positional parameter.
`delete` statements can reference the id of the affected row, while `put` statements can also reference individual column values.
A `rest` parameter is also available, see [migrations](#the-_extra-column-pattern) for details on how that can be useful.
Declaring these statements and parameters happens as part of the schema passed to PowerSync databases:
```javascript JavaScript theme={null}
// Raw tables are not included in the regular Schema() object.
// Instead, add them afterwards using withRawTables().
// The values of parameters are described as a JSON array either containing:
// - the string 'Id' to reference the id of the affected row.
// - the object { Column: name } to reference the value of the column 'name'.
const mySchema = new Schema({
// Define your PowerSync-managed schema here
// ...
});
mySchema.withRawTables({
// The name here doesn't have to match the name of the table in SQL. Instead, it's used to match
// the table name from the backend source database as sent by the PowerSync Service.
todo_lists: {
put: {
sql: 'INSERT OR REPLACE INTO todo_lists (id, created_by, title, content) VALUES (?, ?, ?, ?)',
params: ['Id', { Column: 'created_by' }, { Column: 'title' }, { Column: 'content' }]
},
delete: {
sql: 'DELETE FROM lists WHERE id = ?',
params: ['Id']
}
}
});
// We will simplify this API after understanding the use-cases for raw tables better.
```
```dart Dart theme={null}
// Raw tables are not part of the regular tables list and can be defined with the optional rawTables parameter.
final schema = Schema(const [], rawTables: const [
RawTable(
// The name here doesn't have to match the name of the table in SQL. Instead, it's used to match
// the table name from the backend source database as sent by the PowerSync Service.
name: 'todo_lists',
put: PendingStatement(
sql: 'INSERT OR REPLACE INTO todo_lists (id, created_by, title, content) VALUES (?, ?, ?, ?)',
params: [
.id(),
.column('created_by'),
.column('title'),
.column('content'),
],
),
delete: PendingStatement(
sql: 'DELETE FROM todo_lists WHERE id = ?',
params: [
.id(),
],
),
),
]);
```
```kotlin Kotlin theme={null}
// To define a raw table, include it in the list of tables passed to the Schema
val schema = Schema(listOf(
RawTable(
// The name here doesn't have to match the name of the table in SQL. Instead, it's used to match
// the table name from the backend database as sent by the PowerSync Service.
name = "todo_lists",
put = PendingStatement(
"INSERT OR REPLACE INTO todo_lists (id, created_by, title, content) VALUES (?, ?, ?, ?)",
listOf(
PendingStatementParameter.Id,
PendingStatementParameter.Column("created_by"),
PendingStatementParameter.Column("title"),
PendingStatementParameter.Column("content")
)
),
delete = PendingStatement(
"DELETE FROM todo_lists WHERE id = ?", listOf(PendingStatementParameter.Id)
)
)
))
```
```swift Swift theme={null}
// To define a raw table, include it in the list of tables passed to the Schema
let lists = RawTable(
name: "todo_lists",
put: PendingStatement(
sql: "INSERT OR REPLACE INTO todo_lists (id, created_by, title, content) VALUES (?, ?, ?, ?)",
parameters: [.id, .column("created_by"), .column("title"), .column("content")]
),
delete: PendingStatement(
sql: "DELETE FROM todo_lists WHERE id = ?",
parameters: [.id],
),
)
let schema = Schema(lists)
```
```csharp .NET theme={null}
Unfortunately, raw tables are not yet available in the .NET SDK.
```
```rust Rust theme={null}
use powersync::schema::{PendingStatement, PendingStatementValue, RawTable, Schema};
pub fn app_schema() -> Schema {
let mut schema = Schema::default();
let lists = RawTable::with_statements(
"todo_lists",
PendingStatement {
sql: "INSERT OR REPLACE INTO todo_lists (id, created_by, title, content) VALUES (?, ?, ?, ?)".into(),
params: vec![
PendingStatementValue::Id,
PendingStatementValue::Column("created_by".into()),
PendingStatementValue::Column("title".into()),
PendingStatementValue::Column("content".into()),
]
},
PendingStatement {
sql: "DELETE FROM todo_lists WHERE id = ?".into(),
params:vec![PendingStatementValue::Id]
}
);
schema.raw_tables.push(lists);
schema
}
```
After adding raw tables to the schema, you're also responsible for creating them by executing the corresponding `CREATE TABLE` statement before `connect()`-ing the database.
### Capture Local Writes with Triggers
PowerSync uses an internal SQLite table to collect local writes. For PowerSync-managed views, a trigger for insertions, updates and deletions automatically forwards local mutations into this table. When using raw tables, defining those triggers is your responsibility.
The [PowerSync SQLite core extension](https://github.com/powersync-ja/powersync-sqlite-core) creates an insert-only virtual table named `powersync_crud` with these columns:
```sql theme={null}
-- This table is part of the PowerSync SQLite core extension
CREATE VIRTUAL TABLE powersync_crud(
-- The type of operation: 'PUT' or 'DELETE'
op TEXT,
-- The id of the affected row
id TEXT,
type TEXT,
-- optional (not set on deletes): The column values for the row
data TEXT,
-- optional: Previous column values to include in a CRUD entry
old_values TEXT,
-- optional: Metadata for the write to include in a CRUD entry
metadata TEXT,
);
```
The virtual table associates local mutations with the current transaction and ensures writes made during the sync process (applying server-side changes) don't count as local writes.
The role of triggers is to insert into `powersync_crud` to record writes on raw tables.
Like [with statements](#inferring-sync-statements), these triggers can usually be inferred from the schema of the table.
#### Inferred Triggers
The `powersync_create_raw_table_crud_trigger` SQL function is available in migrations to create triggers for
raw tables. It takes three arguments:
1. A JSON description of the raw table with options, which can be generated by PowerSync SDKs.
2. The name of the trigger to create.
3. The type of write for which to generate a trigger (`INSERT`, `UPDATE` or `DELETE`). Typically, you'd generate all three.
`powersync_create_raw_table_crud_trigger` parses the structure of tables from the database schema, so it
must be called *after* the raw table has been created.
```javascript JavaScript theme={null}
const table: RawTable = { name: 'todo_lists', schema: {} };
await database.execute("CREATE TABLE todo_lists (...)");
for (const write of ["INSERT", "UPDATE", "DELETE"]) {
await database.execute(
"SELECT powersync_create_raw_table_crud_trigger(?, ?, ?)",
[JSON.stringify(Schema.rawTableToJson(table)), `users_${write}`, write],
);
}
```
```dart Dart theme={null}
const table = RawTable.inferred(
name: 'todo_lists',
schema: RawTableSchema(),
);
await database.execute("CREATE TABLE todo_lists (...)");
for (final write in ["INSERT", "UPDATE", "DELETE"]) {
await database.execute(
"SELECT powersync_create_raw_table_crud_trigger(?, ?, ?)",
[json.encode(table), "users_$write", write],
);
}
```
```kotlin Kotlin theme={null}
// To define a raw table, include it in the list of tables passed to the Schema
val table = RawTable(
name = "todo_lists",
schema = RawTableSchema(),
)
database.execute("CREATE TABLE todo_lists (...)")
for (write in listOf("INSERT", "UPDATE", "DELETE)) {
database.execute(
"SELECT powersync_create_raw_table_crud_trigger(?, ?, ?)",
listOf(table.jsonDescription(), "users_$write", write),
)
}
```
```swift Swift theme={null}
let lists = RawTable(
// The name here specifies the name of the table in your backend database or sync configuration.
name: "todo_lists",
schema: RawTableSchema()
)
try await database.execute("CREATE TABLE todo_lists (...)")
for write in ["INSERT", "UPDATE", "DELETE"] {
try await database.execute(
sql: "SELECT powersync_create_raw_table_crud_trigger(?, ?, ?)",
parameters: [
lists.jsonDescription(),
"todo_lists_\(write)",
write,
]
)
}
```
```csharp .NET theme={null}
Unfortunately, raw tables are not yet available in the .NET SDK.
```
```rust Rust theme={null}
use powersync::schema::{RawTable, RawTableSchema};
pub async fn configure_raw_tables(db: &PowerSyncDatabase) -> Result<(), PowerSyncError> {
let raw_table = RawTable::with_schema("todo_lists", RawTableSchema::default());
let serialized_table = serde_json::to_string(&raw_table).unwrap();
let mut writer = db.writer().await?;
writer.execute("CREATE TABLE todo_lists (...);")?;
let mut trigger_stmt = writer.prepare("SELECT powersync_create_raw_table_crud_trigger(?, ?, ?)");
for write in &["INSERT", "UPDATE", "DELETE"] {
trigger_stmt.query_one(
params![serialized_table, format!("todo_lists_{write}", write)],
|_| Ok(()),
)?;
}
Ok(())
}
```
Note that these triggers are created just once! It is your responsibility to drop and re-create them after
altering the table.
Regular JSON-based tables include [advanced options](/client-sdks/advanced/custom-types-arrays-and-json#advanced-schema-options-to-process-writes).
These are also available on raw tables and they affect the generated trigger.
You can track previous values, mark a raw table as insert-only or configure the trigger to ignore
empty updates by passing an `options` parameter (Rust, Swift, Dart, Kotlin)
or set the options on the object literal when defining raw tables (JavaScript).
#### Explicit Triggers
Triggers on raw tables can also be defined explicitly instead of using `powersync_create_raw_table_crud_trigger`.
It is your responsibility to set up and migrate these triggers along with the table:
```sql theme={null}
CREATE TRIGGER todo_lists_insert
AFTER INSERT ON todo_lists
FOR EACH ROW
BEGIN
INSERT INTO powersync_crud (op, id, type, data) VALUES ('PUT', NEW.id, 'todo_lists', json_object(
'created_by', NEW.created_by,
'title', NEW.title,
'content', NEW.content
));
END;
CREATE TRIGGER todo_lists_update
AFTER UPDATE ON todo_lists
FOR EACH ROW
BEGIN
SELECT CASE
WHEN (OLD.id != NEW.id)
THEN RAISE (FAIL, 'Cannot update id')
END;
-- TODO: You may want to replace the json_object with a powersync_diff call of the old and new values, or
-- use your own diff logic to avoid marking unchanged columns as updated.
INSERT INTO powersync_crud (op, id, type, data) VALUES ('PATCH', NEW.id, 'todo_lists', json_object(
'created_by', NEW.created_by,
'title', NEW.title,
'content', NEW.content
));
END;
CREATE TRIGGER todo_lists_delete
AFTER DELETE ON todo_lists
FOR EACH ROW
BEGIN
INSERT INTO powersync_crud (op, id, type) VALUES ('DELETE', OLD.id, 'todo_lists');
END;
```
#### Using Foreign Keys
Raw tables support advanced table constraints including foreign keys. When enabling foreign keys however, you need to be aware of the following:
1. While PowerSync will always apply synced data in a transaction, there is no way to control the order in which rows get applied.
For this reason, foreign keys need to be configured with `DEFERRABLE INITIALLY DEFERRED`.
2. When using [stream priorities](/sync/advanced/prioritized-sync), you need to ensure you don't have foreign keys from high-priority
rows to lower-priority data. PowerSync applies data in one transaction per priority, so these foreign keys would not work.
3. As usual when using foreign keys, note that they need to be explicitly enabled with `pragma foreign_keys = on`.
## Local-Only Columns
Raw tables allow you to add columns that exist only on the client and are never synced to the backend. This is useful for client-specific state like user preferences, local notes, or UI flags that should persist across app restarts but have no equivalent in the backend database.
Local-only columns are not supported with PowerSync's default [JSON-based view system](/architecture/client-architecture#client-side-schema-and-sqlite-database-structure). Raw tables are required for this functionality.
Building on the `todo_lists` example above, you can add local-only columns such as `is_pinned` and `local_notes`:
```sql theme={null}
CREATE TABLE IF NOT EXISTS todo_lists (
id TEXT NOT NULL PRIMARY KEY,
-- Synced columns
created_by TEXT NOT NULL,
title TEXT NOT NULL,
content TEXT,
-- Local-only columns (not synced)
is_pinned INTEGER NOT NULL DEFAULT 0,
local_notes TEXT
) STRICT;
```
### With Inferred Statements and Triggers
Both the inferred `put` and `delete` statements as well as triggers generated by `powersync_create_raw_table_crud_trigger`
support local-only columns.
To configure this, include a `syncedColumns` array on the `RawTableSchema`:
```javascript JavaScript theme={null}
const table: RawTable = {
name: 'todo_lists',
schema: {
syncedColumns: ['created_by', 'title', 'content'],
},
};
```
```dart Dart theme={null}
const table = RawTable.inferred(
name: 'todo_lists',
schema: RawTableSchema(
syncedColumns: ['created_by', 'title', 'content'],
),
);
```
```kotlin Kotlin theme={null}
// To define a raw table, include it in the list of tables passed to the Schema
val table = RawTable(
name = "todo_lists",
schema = RawTableSchema(
syncedColumns = listOf("created_by", "title", "content"),
),
)
```
```swift Swift theme={null}
let lists = RawTable(
name: "todo_lists",
schema: RawTableSchema(
syncedColumns: ["created_by", "title", "content"]
)
)
```
```csharp .NET theme={null}
Unfortunately, raw tables are not yet available in the .NET SDK.
```
```rust Rust theme={null}
use powersync::schema::{RawTable, RawTableSchema};
let raw_table = RawTable::with_schema("todo_lists", {
let mut info = RawTableSchema::default();
// Columns not included in this list will not be synced.
info.synced_columns = Some(vec!["created_by", "title", "content"]);
info
});
```
### With Explicit Statements
The standard raw table setup requires modifications to support local-only columns:
#### Use Upsert Instead of INSERT OR REPLACE
The `put` statement must use `INSERT ... ON CONFLICT(id) DO UPDATE SET` instead of `INSERT OR REPLACE`. `INSERT OR REPLACE` deletes and re-inserts the row, which resets local-only columns to their defaults on every sync update. An upsert only updates the specified synced columns, leaving local-only columns intact.
Only synced columns should be referenced in the `put` params. Local-only columns are omitted entirely:
```javascript JavaScript theme={null}
schema.withRawTables({
todo_lists: {
put: {
sql: `INSERT INTO todo_lists (id, created_by, title, content)
VALUES (?, ?, ?, ?)
ON CONFLICT(id) DO UPDATE SET
created_by = excluded.created_by,
title = excluded.title,
content = excluded.content`,
params: ['Id', { Column: 'created_by' }, { Column: 'title' }, { Column: 'content' }]
},
delete: {
sql: 'DELETE FROM todo_lists WHERE id = ?',
params: ['Id']
}
}
});
```
```dart Dart theme={null}
final schema = Schema(const [], rawTables: const [
RawTable(
name: 'todo_lists',
put: PendingStatement(
sql: '''INSERT INTO todo_lists (id, created_by, title, content)
VALUES (?, ?, ?, ?)
ON CONFLICT(id) DO UPDATE SET
created_by = excluded.created_by,
title = excluded.title,
content = excluded.content''',
params: [
PendingStatementValue.id(),
PendingStatementValue.column('created_by'),
PendingStatementValue.column('title'),
PendingStatementValue.column('content'),
],
),
delete: PendingStatement(
sql: 'DELETE FROM todo_lists WHERE id = ?',
params: [
PendingStatementValue.id(),
],
),
),
]);
```
```kotlin Kotlin theme={null}
val schema = Schema(listOf(
RawTable(
name = "todo_lists",
put = PendingStatement(
"""INSERT INTO todo_lists (id, created_by, title, content)
VALUES (?, ?, ?, ?)
ON CONFLICT(id) DO UPDATE SET
created_by = excluded.created_by,
title = excluded.title,
content = excluded.content""",
listOf(
PendingStatementParameter.Id,
PendingStatementParameter.Column("created_by"),
PendingStatementParameter.Column("title"),
PendingStatementParameter.Column("content")
)
),
delete = PendingStatement(
"DELETE FROM todo_lists WHERE id = ?", listOf(PendingStatementParameter.Id)
)
)
))
```
```swift Swift theme={null}
let lists = RawTable(
name: "todo_lists",
put: PendingStatement(
sql: """
INSERT INTO todo_lists (id, created_by, title, content)
VALUES (?, ?, ?, ?)
ON CONFLICT(id) DO UPDATE SET
created_by = excluded.created_by,
title = excluded.title,
content = excluded.content
""",
parameters: [.id, .column("created_by"), .column("title"), .column("content")]
),
delete: PendingStatement(
sql: "DELETE FROM todo_lists WHERE id = ?",
parameters: [.id],
),
)
let schema = Schema(lists)
```
#### Exclude Local-Only Columns from Triggers
The `json_object()` in both the INSERT and UPDATE triggers should only reference synced columns. Local-only columns must not appear in the CRUD payload sent to the backend.
Additionally, the UPDATE trigger needs a `WHEN` clause that checks only synced columns. Without it, changes to local-only columns would fire the trigger and produce unnecessary CRUD entries that get uploaded. The `WHEN` clause must use `IS NOT` instead of `!=` for NULL-safe comparisons. `NULL != NULL` evaluates to `NULL` in SQLite, which would cause the trigger to skip legitimate changes to nullable synced columns.
```sql theme={null}
CREATE TRIGGER todo_lists_insert
AFTER INSERT ON todo_lists
FOR EACH ROW
BEGIN
INSERT INTO powersync_crud (op, id, type, data) VALUES ('PUT', NEW.id, 'todo_lists', json_object(
'created_by', NEW.created_by,
'title', NEW.title,
'content', NEW.content
));
END;
-- WHEN clause ensures this only fires for synced column changes.
-- Uses IS NOT instead of != for correct NULL handling.
CREATE TRIGGER todo_lists_update
AFTER UPDATE ON todo_lists
FOR EACH ROW
WHEN
OLD.created_by IS NOT NEW.created_by
OR OLD.title IS NOT NEW.title
OR OLD.content IS NOT NEW.content
BEGIN
INSERT INTO powersync_crud (op, id, type, data) VALUES ('PATCH', NEW.id, 'todo_lists', json_object(
'created_by', NEW.created_by,
'title', NEW.title,
'content', NEW.content
));
END;
CREATE TRIGGER todo_lists_delete
AFTER DELETE ON todo_lists
FOR EACH ROW
BEGIN
INSERT INTO powersync_crud (op, id, type) VALUES ('DELETE', OLD.id, 'todo_lists');
END;
```
With this setup, local-only columns can be queried and updated using standard SQL without affecting sync:
```sql theme={null}
-- Updating a local-only column does not produce a CRUD entry
UPDATE todo_lists SET is_pinned = 1 WHERE id = '...';
-- Local-only columns can be used in queries and ordering
SELECT * FROM todo_lists ORDER BY is_pinned DESC, title ASC;
```
## Migrations
In PowerSync's [JSON-based view system](/architecture/client-architecture#client-side-schema-and-sqlite-database-structure) the client-side schema is applied to the schemaless data, meaning no migrations are required. Raw tables however are excluded from this, so it is the developer's responsibility to manage migrations for these tables.
### Adding Raw Tables as a New Table
When you're adding new tables to your Sync Streams (or legacy Sync Rules), clients will start to sync data on those tables - even if the tables aren't mentioned in the client's schema yet. So at the time you're introducing a new raw table to your app, it's possible that PowerSync has already synced some data for that table, which would be stored in `ps_untyped`. When adding regular tables, PowerSync will automatically extract rows from `ps_untyped`. With raw tables, that step is your responsibility. To copy data, run these statements in a transaction after creating the table:
```
INSERT INTO my_table (id, my_column, ...)
SELECT id, data ->> 'my_column' FROM ps_untyped WHERE type = 'my_table';
DELETE FROM ps_untyped WHERE type = 'my_table';
```
This does not apply if you've been using the raw table from the beginning (and never called `connect()` without them) - you only need this for raw tables you already had locally.
Another workaround is to clear PowerSync data when changing raw tables and opt for a full resync.
### Migrating to Raw Tables
To migrate from PowerSync-managed tables to raw tables, first:
1. Open the database with the new schema mentioning raw tables. PowerSync will copy data from tables previously managed by PowerSync into `ps_untyped`.
2. Create raw tables.
3. Run the `INSERT FROM SELECT` statement to insert `ps_untyped` data into your raw tables.
### Migrations on Raw Tables
For JSON-based tables, migrations are trivial since all rows are stored as complete JSON objects.
Adding or removing columns only affects views over unchanged JSON data, making the schema a stateless structure.
For raw tables, the situation is different. When adding a new column for instance, existing rows would
not have a default value even if one could have been synced already.
Suppose a new column is added with a simple migration: `ALTER TABLE todo_list ADD COLUMN priority INTEGER`.
This adds the new column on the client, with null values for each existing row.
If the client updates the schema before the server and then syncs the changes, every row effectively
resyncs and reflects populated values for the new column. So clients observe a consistent state after the sync.
If new values have been synced before the client updates, existing rows may not receive the new column
until those rows are synced again! This is why special approaches are needed when migrating synced
tables.
#### Deleting Data on Migrations
One option that makes migrations safe (with obvious downsides) is to simply reset the database before
migrating: `await db.disconnectAndClear(soft: true)` deletes materialized sync rows while keeping
downloaded data active. Afterwards, migrations can migrate the schema in any way before you reconnect.
In a soft clear, data doesn't have to be downloaded again in most cases. This might reduce the downtime
in which no data is available, but a network connection is necessary for data to become
available again.
#### Triggering Resync on Migrations
An alternative to the approach of deleting data could be to trigger a re-sync *without* clearing tables.
For example:
```sql theme={null}
-- We need an (optimistic) default value for existing rows
ALTER TABLE todo_list ADD COLUMN priority INTEGER DEFAULT 1 NOT NULL;
SELECT powersync_trigger_resync(TRUE);
```
The optimistic default value would be overridden on the next completed sync (depending on when
the user is online again).
This means that the app is still usable offline after an update, but having optimistic state
on the client is a caveat because PowerSync normally has [stronger consistency guarantees](/architecture/consistency).
There may be cases where the approach of deleting data is a safer choice.
#### The `_extra` Column Pattern
Another option to avoid data inconsistencies in migrations is to ensure the raw table stores
a full row as expected by PowerSync.
To do that, you can introduce an extra column on your table designed to hold values from the backend
database that a client is not yet aware of:
```sql theme={null}
CREATE TABLE todo_lists (
id TEXT NOT NULL PRIMARY KEY,
created_by TEXT NOT NULL,
title TEXT NOT NULL,
content TEXT,
_extra TEXT
) STRICT;
```
The `_extra` column is not used in the app, but the sync service can be informed about it using
the `Rest` column source:
```javascript JavaScript theme={null}
mySchema.withRawTables({
// The name here doesn't have to match the name of the table in SQL. Instead, it's used to match
// the table name from the backend source database as sent by the PowerSync Service.
todo_lists: {
put: {
sql: 'INSERT OR REPLACE INTO todo_lists (id, created_by, title, content, _extra) VALUES (?, ?, ?, ?, ?)',
params: ['Id', { Column: 'created_by' }, { Column: 'title' }, { Column: 'content' }, 'Rest']
},
delete: ...
}
});
```
```dart Dart theme={null}
final schema = Schema(const [], rawTables: const [
RawTable(
name: 'todo_lists',
put: PendingStatement(
sql: 'INSERT OR REPLACE INTO todo_lists (id, created_by, title, content, _extra) VALUES (?, ?, ?, ?, ?)',
params: [
.id(),
.column('created_by'),
.column('title'),
.column('content'),
.rest(),
],
),
delete: PendingStatement(...),
),
]);
```
```kotlin Kotlin theme={null}
val schema = Schema(listOf(
RawTable(
name = "todo_lists",
put = PendingStatement(
"INSERT OR REPLACE INTO todo_lists (id, created_by, title, content, _extra) VALUES (?, ?, ?, ?, ?)",
listOf(
PendingStatementParameter.Id,
PendingStatementParameter.Column("created_by"),
PendingStatementParameter.Column("title"),
PendingStatementParameter.Column("content"),
PendingStatementParameter.Rest,
)
),
delete = PendingStatement(...)
)
))
```
```swift Swift theme={null}
let lists = RawTable(
name: "todo_lists",
put: PendingStatement(
sql: "INSERT OR REPLACE INTO todo_lists (id, created_by, title, content, _extra) VALUES (?, ?, ?, ?, ?)",
parameters: [.id, .column("created_by"), .column("title"), .column("content"), .rest]
),
delete: ...
)
```
```csharp .NET theme={null}
Unfortunately, raw tables are not yet available in the .NET SDK.
```
```rust Rust theme={null}
use powersync::schema::{PendingStatement, PendingStatementValue, RawTable, Schema};
let lists = RawTable::with_statements(
"todo_lists",
PendingStatement {
sql: "INSERT OR REPLACE INTO todo_lists (id, created_by, title, content, _extra) VALUES (?, ?, ?, ?, ?)".into(),
params: vec![
PendingStatementValue::Id,
PendingStatementValue::Column("created_by".into()),
PendingStatementValue::Column("title".into()),
PendingStatementValue::Column("content".into()),
PendingStatementValue::Rest,
]
},
...
);
```
If PowerSync then syncs a row like `{"created_by": "User", "title": "title", "content": "content", "tags": "Important"}`,
this put statement would set `_extra` to `{"tags":"Important"}`, ensuring that the entire source row
can be recovered from a row in the raw table.
This then allows writing migrations:
1. Adding new columns by using `json_extract(_extra, '$.newColumnName')` as a default value.
2. Removing existing columns by updating `_extra = json_set(_extra, '$.droppedColumnName', droppedColumnName)` before dropping
the column.
Don't forget to delete triggers before running these statements in migrations, since these updates
shouldn't result in `ps_crud` writes.
## Deleting Data and Raw Tables
APIs that clear an entire PowerSync database, like e.g. `disconnectAndClear()`, don't affect raw tables by default. You can use the `clear` parameter on the `RawTable` constructor to set an SQL statement to run when clearing the database. Typically, something like `DELETE FROM $tableName` would be a reasonable statement to run.
`clear` statements are not inferred automatically and must always be set explicitly.
Raw tables themselves are not managed by PowerSync and need to be dropped to delete them.
# Sequential ID Mapping
Source: https://docs.powersync.com/client-sdks/advanced/sequential-id-mapping
Map local UUIDs to server-assigned sequential IDs when your backend requires integer primary keys.
## Introduction
When auto-incrementing / sequential IDs are used on the backend source database, the ID can only be generated on the backend source database, and not on the client while offline.
To handle this, you can use a secondary UUID on the client, then map it to a sequential ID when performing an update on the backend source database.
This allows using a sequential primary key for each record, with a UUID as a secondary ID.
This mapping must be performed wherever the UUIDs are referenced, including for every foreign key column.
To illustrate this, we will use the [React To-Do List demo app](https://github.com/powersync-ja/powersync-js/tree/main/demos/react-supabase-todolist) and modify it to use UUIDs
on the client and map them to sequential IDs on the backend source database (Supabase in this case).
### Overview
Before we get started, let's outline the changes we will have to make:
Update the `lists` and `todos` tables
Add two triggers that will map the UUID to the integer ID and vice versa.
Update your Sync Streams (or legacy Sync Rules) to use the UUID column instead of the integer ID.
The following components/files will have to be updated:
* *Files*:
* `AppSchema.ts`
* `fts_setup.ts`
* `SupabaseConnector.ts`
* *Components*:
* `lists.tsx`
* `page.tsx`
* `SearchBarWidget.tsx`
* `TodoListsWidget.tsx`
## Schema
In order to map the UUID to the integer ID, we need to update the
* `lists` table by adding a `uuid` column, which will be the secondary ID, and
* `todos` table by adding a `uuid` column, and a `list_uuid` foreign key column which references the `uuid` column in the `lists` table.
```sql schema {3, 13, 21, 26} theme={null}
create table public.lists (
id serial,
uuid uuid not null unique,
created_at timestamp with time zone not null default now(),
name text not null,
owner_id uuid not null,
constraint lists_pkey primary key (id),
constraint lists_owner_id_fkey foreign key (owner_id) references auth.users (id) on delete cascade
) tablespace pg_default;
create table public.todos (
id serial,
uuid uuid not null unique,
created_at timestamp with time zone not null default now(),
completed_at timestamp with time zone null,
description text not null,
completed boolean not null default false,
created_by uuid null,
completed_by uuid null,
list_id int not null,
list_uuid uuid not null,
constraint todos_pkey primary key (id),
constraint todos_created_by_fkey foreign key (created_by) references auth.users (id) on delete set null,
constraint todos_completed_by_fkey foreign key (completed_by) references auth.users (id) on delete set null,
constraint todos_list_id_fkey foreign key (list_id) references lists (id) on delete cascade,
constraint todos_list_uuid_fkey foreign key (list_uuid) references lists (uuid) on delete cascade
) tablespace pg_default;
```
With the schema updated, we now need a method to sync and map the `list_id` and `list_uuid` in the `todos` table, with the `id` and `uuid` columns in the `lists` table.
We can achieve this by creating SQL triggers.
## Create SQL Triggers
We need to create triggers that can look up the integer ID for the given UUID and vice versa.
These triggers will maintain consistency between `list_id` and `list_uuid` in the `todos` table by ensuring that they remain in sync with the `id` and `uuid` columns in the `lists` table;
even if changes are made to either field.
We will create the following two triggers that cover either scenario of updating the `list_id` or `list_uuid` in the `todos` table:
1. `update_integer_id`, and
2. `update_uuid_column`
The `update_integer_id` trigger ensures that whenever a `list_uuid` value is inserted or updated in the `todos` table,
the corresponding `list_id` is fetched from the `lists` table and updated automatically. It also validates that the `list_uuid` exists in the `lists` table; otherwise, it raises an exception.
```sql theme={null}
CREATE OR REPLACE FUNCTION func_update_integer_id()
RETURNS TRIGGER AS $$
BEGIN
IF TG_OP = 'INSERT' THEN
-- Always update list_id on INSERT
SELECT id INTO NEW.list_id
FROM lists
WHERE uuid = NEW.list_uuid;
IF NOT FOUND THEN
RAISE EXCEPTION 'UUID % does not exist in lists', NEW.list_uuid;
END IF;
ELSIF TG_OP = 'UPDATE' THEN
-- Only update list_id if list_uuid changes
IF NEW.list_uuid IS DISTINCT FROM OLD.list_uuid THEN
SELECT id INTO NEW.list_id
FROM lists
WHERE uuid = NEW.list_uuid;
IF NOT FOUND THEN
RAISE EXCEPTION 'UUID % does not exist in lists', NEW.list_uuid;
END IF;
END IF;
END IF;
RETURN NEW;
END;
$$ LANGUAGE plpgsql;
CREATE TRIGGER update_integer_id
BEFORE INSERT OR UPDATE ON todos
FOR EACH ROW
EXECUTE FUNCTION func_update_integer_id();
```
The `update_uuid_column` trigger ensures that whenever a `list_id` value is inserted or updated in the todos table, the corresponding `list_uuid` is fetched from the
`lists` table and updated automatically. It also validates that the `list_id` exists in the `lists` table.
```sql update_uuid_column theme={null}
CREATE OR REPLACE FUNCTION func_update_uuid_column()
RETURNS TRIGGER AS $$
BEGIN
IF TG_OP = 'INSERT' THEN
-- Always update list_uuid on INSERT
SELECT uuid INTO NEW.list_uuid
FROM lists
WHERE id = NEW.list_id;
IF NOT FOUND THEN
RAISE EXCEPTION 'ID % does not exist in lists', NEW.list_id;
END IF;
ELSIF TG_OP = 'UPDATE' THEN
-- Only update list_uuid if list_id changes
IF NEW.list_id IS DISTINCT FROM OLD.list_id THEN
SELECT uuid INTO NEW.list_uuid
FROM lists
WHERE id = NEW.list_id;
IF NOT FOUND THEN
RAISE EXCEPTION 'ID % does not exist in lists', NEW.list_id;
END IF;
END IF;
END IF;
RETURN NEW;
END;
$$ LANGUAGE plpgsql;
CREATE TRIGGER update_uuid_column
BEFORE INSERT OR UPDATE ON todos
FOR EACH ROW
EXECUTE FUNCTION func_update_uuid_column();
```
We now have triggers in place that will handle the mapping for our updated schema and
can move on to updating your Sync Streams/Sync Rules to use the UUID column instead of the integer ID.
## Update Sync Streams
As sequential IDs can only be created on the backend source database, we need to use UUIDs in the client. The Sync Config is updated to use the `uuid` column as the `id` column for the `lists` and `todos` tables, explicitly defining which columns to select so that `list_id` (the integer ID) is no longer exposed to the client.
```yaml theme={null}
config:
edition: 3
streams:
user_lists:
auto_subscribe: true
with:
user_lists_param: SELECT id FROM lists WHERE owner_id = auth.user_id()
queries:
- "SELECT lists.uuid AS id, lists.created_at, lists.name, lists.owner_id FROM lists WHERE lists.id IN user_lists_param"
- "SELECT todos.uuid AS id, todos.created_at, todos.completed_at, todos.description, todos.completed, todos.created_by, todos.list_uuid FROM todos WHERE todos.list_id = user_lists_param"
```
```yaml sync-config.yaml {4, 7-8} theme={null}
bucket_definitions:
user_lists:
# Separate bucket per todo list
parameters: select id from lists where owner_id = request.user_id()
data:
# Explicitly define all the columns
- select uuid as id, created_at, name, owner_id from lists where id = bucket.id
- select uuid as id, created_at, completed_at, description, completed, created_by, list_uuid from todos where list_id = bucket.id
```
We can now move on to updating the client to use UUIDs.
## Update Client to Use UUIDs
With Sync Streams updated, we no longer have the `list_id` column in the `todos` table.
We start by updating `AppSchema.ts` and replacing `list_id` with `list_uuid` in the `todos` table.
```typescript AppSchema.ts {3, 11} theme={null}
const todos = new Table(
{
list_uuid: column.text,
created_at: column.text,
completed_at: column.text,
description: column.text,
created_by: column.text,
completed_by: column.text,
completed: column.integer
},
{ indexes: { list: ['list_uuid'] } }
);
```
The `uploadData` function in `SupabaseConnector.ts` needs to be updated to use the new `uuid` column in both tables.
```typescript SupabaseConnector.ts {13, 17, 20} theme={null}
export class SupabaseConnector extends BaseObserver implements PowerSyncBackendConnector {
// other code
async uploadData(database: CommonPowerSyncDatabase): Promise {
// other code
try {
for (const op of transaction.crud) {
lastOp = op;
const table = this.client.from(op.table);
let result: any;
switch (op.op) {
case UpdateType.PUT:
const record = { ...op.opData, uuid: op.id };
result = await table.upsert(record);
break;
case UpdateType.PATCH:
result = await table.update(op.opData).eq('uuid', op.id);
break;
case UpdateType.DELETE:
result = await table.delete().eq('uuid', op.id);
break;
}
}
} catch (ex: any) {
// other code
}
}
}
```
For the remaining files, we simply need to replace any reference to `list_id` with `list_uuid`.
```typescript fts_setup.ts {3} theme={null}
export async function configureFts(): Promise {
await createFtsTable('lists', ['name'], 'porter unicode61');
await createFtsTable('todos', ['description', 'list_uuid']);
}
```
```tsx page.tsx {4, 14} theme={null}
const TodoEditSection = () => {
// code
const { data: todos } = useQuery(
`SELECT * FROM ${TODOS_TABLE} WHERE list_uuid=? ORDER BY created_at DESC, id`,
[listID]
);
// code
const createNewTodo = async (description: string) => {
// other code
await powerSync.execute(
`INSERT INTO
${TODOS_TABLE}
(id, created_at, created_by, description, list_uuid)
VALUES
(uuid(), datetime(), ?, ?, ?)`,
[userID, description, listID!]
);
}
}
```
```tsx TodoListWidget.tsx {10, 18} theme={null}
export function TodoListsWidget(props: TodoListsWidgetProps) {
// hooks and navigation
const { data: listRecords, isLoading } = useQuery(`
SELECT
${LISTS_TABLE}.*, COUNT(${TODOS_TABLE}.id) AS total_tasks, SUM(CASE WHEN ${TODOS_TABLE}.completed = true THEN 1 ELSE 0 END) as completed_tasks
FROM
${LISTS_TABLE}
LEFT JOIN ${TODOS_TABLE}
ON ${LISTS_TABLE}.id = ${TODOS_TABLE}.list_uuid
GROUP BY
${LISTS_TABLE}.id;
`);
const deleteList = async (id: string) => {
await powerSync.writeTransaction(async (tx) => {
// Delete associated todos
await tx.execute(`DELETE FROM ${TODOS_TABLE} WHERE list_uuid = ?`, [id]);
// Delete list record
await tx.execute(`DELETE FROM ${LISTS_TABLE} WHERE id = ?`, [id]);
});
};
}
```
```tsx SearchBarWidget.tsx {8, 19} theme={null}
export const SearchBarWidget: React.FC = () => {
const handleInputChange = async (value: string) => {
if (value.length !== 0) {
let listsSearchResults: any[] = [];
const todoItemsSearchResults = await searchTable(value, 'todos');
for (let i = 0; i < todoItemsSearchResults.length; i++) {
const res = await powersync.get(`SELECT * FROM ${LISTS_TABLE} WHERE id = ?`, [
todoItemsSearchResults[i]['list_uuid']
]);
todoItemsSearchResults[i]['list_name'] = res.name;
}
if (!todoItemsSearchResults.length) {
listsSearchResults = await searchTable(value, 'lists');
}
const formattedListResults: SearchResult[] = listsSearchResults.map(
(result) => new SearchResult(result['id'], result['name'])
);
const formattedTodoItemsResults: SearchResult[] = todoItemsSearchResults.map((result) => {
return new SearchResult(result['list_uuid'], result['list_name'] ?? '', result['description']);
});
setSearchResults([...formattedTodoItemsResults, ...formattedListResults]);
}
};
}
```
# Using SQLite Extensions
Source: https://docs.powersync.com/client-sdks/advanced/sqlite-extensions
Extend capabilities in client-side databases with custom SQLite extensions
SQLite can be extended with [Run-Time Loadable Extensions](https://sqlite.org/loadext.html) contributing
new functions, virtual tables, full-text search tokenizers and more. Since PowerSync SDKs are built on top
of SQLite connections, most SQLite extensions also work with PowerSync.
Including custom SQLite extensions in your app is possible, but requires SDK and platform-specific setup steps.
This document collects examples showing how to use custom extensions on different PowerSync SDKs.
## Dart and Flutter
### Native
On native Dart and Flutter targets, custom native code can be called via `dart:ffi` and linked with
[build hooks](https://dart.dev/tools/hooks).
The `sqlite3` package used by PowerSync [contains a complete example](https://github.com/simolus3/sqlite3.dart/tree/main/sqlite3/example/custom_extension)
adopting build hooks to link SQLite extensions.
Using that example, you would call the `sqlite3.loadSqliteVectorExtension()` extension method before using PowerSync to
ensure the extension is loaded.
### Web
Because dynamic linking is not generally available with WebAssembly, loading extensions on the web requires a custom
`sqlite3.wasm` build linking both PowerSync and any additional extensions you may want to use.
This is an advanced topic, and not directly supported by PowerSync.
The `sqlite3` package provides [instructions](https://github.com/simolus3/sqlite3.dart/tree/main/sqlite3_wasm_build)
and cmake build scripts to compile `sqlite3.wasm`.
The PowerSync Dart SDK [adopts that](https://github.com/powersync-ja/powersync.dart/tree/main/packages/sqlite3_wasm_build)
with [a patch](https://github.com/powersync-ja/powersync.dart/blob/main/packages/sqlite3_wasm_build/patches/0001-Link-PowerSync-core-extension.patch)
to automatically load the PowerSync SQLite core extension when SQLite is loaded (in `sqlite3_os_init`).
To link additional extensions, you could add similar build script additions to call their entrypoint.
## JavaScript
### Web
Similar to Dart on the web, loading extensions with `@powersync/web` is only possible with a custom `wa-sqlite` WebAssembly
build.
PowerSync [forks wa-sqlite](https://github.com/powersync-ja/wa-sqlite) for this reason, loading custom extension requires patching
build definitions in that repository and adding an override from your npm package manager to patch `@journeyapps/wa-sqlite`.
### React Native
The recommended OP-SQLite library has builtin support for custom extensions, and [excellent documentation](https://op-engineering.github.io/op-sqlite/docs/api#loading-extensions)
on how to build and bundle them with your app.
### Node.js
Loading extensions is possible with `better-sqlite3` and the [`loadExtension`](https://github.com/WiseLibs/better-sqlite3/blob/HEAD/docs/api.md#loadextensionpath-entrypoint---this)
method on databases.
Because the database is created on a worker, a custom worker is necessary to customize it:
```TypeScript theme={null}
// custom.worker.ts
import Database from 'better-sqlite3';
import { startPowerSyncWorker } from '@powersync/node/worker.js';
async function resolveBetterSqlite3() {
class DatabaseWithUsedExtensions extends Database {
constructor(...args: any[]) {
super(...args);
this.loadExtension('libyourExtension.dylib');
}
}
return DatabaseWithUsedExtensions;
}
startPowerSyncWorker({ loadBetterSqlite3: resolveBetterSqlite3 });
```
This worker can then be used like this to load extension for a database:
```TypeScript theme={null}
const db = new PowerSyncDatabase({
schema: AppSchema,
database: {
dbFilename: 'test.db',
openWorker: (_, options) => {
return new Worker(new URL('./custom.worker.js', import.meta.url), options);
},
},
});
```
For another example using custom workers, see [this section](https://github.com/powersync-ja/powersync-js/tree/main/packages/node#encryption).
### Capacitor
The PowerSync Capacitor SDK uses [`@capacitor-community/sqlite`](https://github.com/capacitor-community/sqlite)
as a SQLite library, which doesn't support loading extensions directly.
On Android however, you can use the [`load_extension`](https://sqlite.org/lang_corefunc.html#load_extension)
SQL function to load extensions.
To build and bundle extensions with your app, follow the approach for [React Native](#react-native) or [Kotlin](#kotlin).
On iOS, you'd have to follow the approach for [Swift](#swift) and expose your Swift helper method loading the extension
in a way that can be called from JavaScript.
## Kotlin
On all platforms supported by the Kotlin SDK, a custom [`PersistentConnectionFactory`](https://powersync-ja.github.io/powersync-kotlin/common/com.powersync/-persistent-connection-factory/index.html)
can be used to customize how PowerSync opens SQLite connections. This can also be used to load additional extensions.
On Android and JVM platforms, the setup for that can look like this:
```Kotlin theme={null}
internal class MyOpenFactory: DriverBasedInMemoryFactory(createBundledDriver()), PersistentConnectionFactory {
override fun openConnection(
path: String,
openFlags: Int,
): SQLiteConnection = driver.open(path, openFlags)
override fun resolveDefaultDatabasePath(dbFilename: String): String {
TODO("On Android, use context.getDatabasePath(dbFilename).path")
}
private companion object {
fun createBundledDriver(): BundledSQLiteDriver {
return BundledSQLiteDriver().also {
it.addPowerSyncExtension()
// TODO: Bundle and resolve extension file
it.addExtension("my_custom_extension_file")
}
}
}
}
```
Bundling and resolving extensions requires platform-specific setup. For the JVM, the PowerSync SDK bundles the extension
as a pre-compiled library for all target platforms as a resource. This allows using [ClassLoader](https://github.com/powersync-ja/powersync-kotlin/blob/cd8c6aede9f59f5c19fa2473798c1b2ccb035c3f/common/src/jvmMain/kotlin/com/powersync/ExtractLib.kt#L7-L46)
APIs to extract the library to a temporary file before opening it.
On Android, use [Android NDK](https://developer.android.com/ndk) to build extension sources as a library to link with your
app. The name of the library can be passed to `addExtension` directly, as `System.loadLibrary` will resolve to it.
The safest way to load extensions on Kotlin/Native targets is to use [cinterops](https://kotlinlang.org/docs/native-c-interop.html)
building the extension as a library and making its entrypoint available from Kotlin.
This entrypoint can then be called via [`sqlite3_auto_extension`](https://github.com/powersync-ja/powersync-kotlin/blob/cd8c6aede9f59f5c19fa2473798c1b2ccb035c3f/common/src/nativeMain/kotlin/com/powersync/ConnectionFactory.native.kt#L8-L14)
before opening PowerSync databases.
## .NET
The PowerSync .NET SDK has builtin support for loading custom extensions by setting the [`MDSQLiteOptions.Extensions`](https://powersync-ja.github.io/powersync-dotnet/api/PowerSync.Common.MDSQLite.MDSQLiteOptions.Extensions.html) field to an array of [`SqliteExtension`](https://powersync-ja.github.io/powersync-dotnet/api/PowerSync.Common.MDSQLite.SqliteExtension.html) objects.
You can also disable loading the [core PowerSync SQLite extension](https://github.com/powersync-ja/powersync-sqlite-core) by setting `MDSQLiteOptions.LoadPowerSyncExtension` to false. When doing so, you must ensure that one of the extensions passed in `MDSQLiteOptions.Extensions` contains a build of the PowerSync core extension.
Running the PowerSync .NET SDK without the core extension or with an outdated version of the core extension is not supported.
It is your responsibility to build and bundle extensions you want to use with your app so that they can be loaded.
## Rust and Tauri
For the Rust and Tauri SDKs, your app is expected to link SQLite directly.
This means that regular SQLite APIs to load extensions can be used.
For `sqlite-vec` for example, depending on [this crate](https://crates.io/crates/sqlite-vec) and calling
`sqlite3_vec_init()` before opening a PowerSync database ensures the extension is loaded.
Most other extensions have a similar entrypoint to invoke. For full control, you can also invoke
[`register_auto_extension`](https://docs.rs/rusqlite/latest/rusqlite/auto_extension/fn.register_auto_extension.html)
with the entrypoint function of the extension (useful for statically linked extensions) or use
[`load_extension`](https://docs.rs/rusqlite/0.39.0/rusqlite/struct.Connection.html#method.load_extension) on a
`Connection` before passing it to a `ConnectionPool` for the PowerSync Rust SDK.
## Swift
The PowerSync Swift SDK depends on [this package](https://github.com/powersync-ja/CSQLite) to link SQLite with your app.
To add a custom extension:
1. Write Swift Package Manager definitions to build and link that extension with your app too.
2. Depend on the [PowerSync CSQLite](https://github.com/powersync-ja/CSQLite) target.
3. Similar to the setup for PowerSync, add a [module shim](https://github.com/powersync-ja/powersync-swift/tree/main/Sources/PowerSyncCoreShim)
target allowing you to use the extension from Swift.
4. Import that target and CSQLite to [load the extension statically](https://github.com/powersync-ja/powersync-swift/blob/ad9cf3d8c65dcbf059c4e5430dd35f3706ad5303/Sources/PowerSync/Implementation/sqlite3/registerPowerSyncCoreExtension.swift#L1-L17)
before opening a PowerSync database.
# State Management Libraries
Source: https://docs.powersync.com/client-sdks/advanced/state-management
Use PowerSync with state management libraries like Riverpod in Dart/Flutter apps.
This guide is currently specific to the Dart/Flutter SDK. We may expand it to cover other SDKs in the future.
Our [demo apps](/intro/examples) for Flutter are intentionally kept simple to focus on demonstrating PowerSync APIs. Instead of using heavy state management solutions, they use simple global fields to make the PowerSync database accessible to widgets.
When adopting PowerSync in your own app, you might want a more sophisticated approach for state management. This guide explains how PowerSync's Dart/Flutter SDK integrates with popular state management packages.
Adopting PowerSync can actually simplify your app architecture by using a local SQLite database as the single source of truth for all data. For a general discussion on how PowerSync fits into modern app architecture, see [this blog post](https://dinkomarinac.dev/building-local-first-flutter-apps-with-riverpod-drift-and-powersync).
PowerSync exposes database queries with the standard `Future` and `Stream` classes from `dart:async`. Given how widely used these are
in the Dart ecosystem, PowerSync works well with all popular approaches for state management, such as:
1. Providers with `package:provider`: Create your database as a `Provider` and expose watched queries to child widgets with `StreamProvider`!
The provider for databases should `close()` the database in `dispose`.
2. Providers with `package:riverpod`: We mention relevant snippets [below](#riverpod).
3. Dependency injection with `package:get_it`: PowerSync databases can be registered with `registerSingletonAsync`. Again, make sure
to `close()` the database in the `dispose` callback.
4. The BLoC pattern with the `bloc` package: You can easily listen to watched queries in Cubits (although, if you find your
Blocs and Cubits becoming trivial wrappers around database streams, consider just `watch()`ing database queries in widgets directly.
That doesn't make your app [less testable](/client-sdks/advanced/unit-testing)!).
To simplify state management, avoid the use of hydrated blocs and cubits for state that depends on database queries. With PowerSync,
regular data is already available locally and doesn't need a second local cache.
## Riverpod
We have a [complete example](https://github.com/powersync-ja/powersync.dart/tree/main/demos/supabase-todolist-drift) using PowerSync with modern Flutter libraries like Riverpod, Drift, and `auto_route`.
A good way to open PowerSync databases with Riverpod is to use an async provider. You can manage your `connect` and `disconnect` calls there, for instance by listening to authentication state:
```dart theme={null}
@Riverpod(keepAlive: true)
Future powerSyncInstance(Ref ref) async {
final db = PowerSyncDatabase(
schema: schema,
path: await _getDatabasePath(),
logger: attachedLogger,
);
await db.initialize();
// TODO: Listen for auth changes and connect() the database here.
ref.listen(yourAuthProvider, (prev, next) {
if (next.isAuthenticated && !prev.isAuthenticated) {
db.connect(connector: MyConnector());
}
// ...
});
ref.onDispose(db.close);
return db;
}
```
### Querying Data
To expose auto-updating query results, use a `StreamProvider` that reads from the database:
```dart theme={null}
final _lists = StreamProvider((ref) async* {
final database = await ref.read(powerSyncInstanceProvider.future);
yield* database.watch('SELECT * FROM lists');
});
```
### Waiting for Sync
If you were awaiting `waitForFirstSync` before, you can keep doing that:
```dart theme={null}
final db = await ref.read(powerSyncInstanceProvider.future);
await db.waitForFirstSync();
```
Alternatively, you can expose the sync status as a provider and use that to determine
whether the sync has completed:
```dart theme={null}
final syncStatus = statefulProvider((ref, change) {
final status = Stream.fromFuture(ref.read(powerSyncInstanceProvider.future))
.asyncExpand((db) => db.statusStream);
final sub = status.listen(change);
ref.onDispose(sub.cancel);
return const SyncStatus();
});
@riverpod
bool didCompleteSync(Ref ref, [BucketPriority? priority]) {
final status = ref.watch(syncStatus);
if (priority != null) {
return status.statusForPriority(priority).hasSynced ?? false;
} else {
return status.hasSynced ?? false;
}
}
final class MyWidget extends ConsumerWidget {
const MyWidget({super.key});
@override
Widget build(BuildContext context, WidgetRef ref) {
final didSync = ref.watch(didCompleteSyncProvider());
if (!didSync) {
return const Text('Busy with sync...');
}
// ... content after first sync
}
}
```
### Attachment Queue
If you're using the attachment queue helper to sync media assets, you can also wrap that in a provider:
```dart theme={null}
@Riverpod(keepAlive: true)
Future attachmentQueue(Ref ref) async {
final db = await ref.read(powerSyncInstanceProvider.future);
final queue = YourAttachmentQueue(db, remoteStorage);
await queue.init();
return queue;
}
```
Reading and awaiting this provider can then be used to show attachments:
```dart theme={null}
final class PhotoWidget extends ConsumerWidget {
final TodoItem todo;
const PhotoWidget({super.key, required this.todo});
@override
Widget build(BuildContext context, WidgetRef ref) {
final photoState = ref.watch(_getPhotoStateProvider(todo.photoId));
if (!photoState.hasValue) {
return Container();
}
final data = photoState.value;
if (data == null) {
return Container();
}
String? filePath = data.photoPath;
bool fileIsDownloading = !data.fileExists;
bool fileArchived =
data.attachment?.state == AttachmentState.archived.index;
if (fileArchived) {
return Column(
crossAxisAlignment: CrossAxisAlignment.center,
mainAxisAlignment: MainAxisAlignment.center,
children: [
const Text("Unavailable"),
const SizedBox(height: 8),
],
);
}
if (fileIsDownloading) {
return const Text("Downloading...");
}
File imageFile = File(filePath!);
int lastModified = imageFile.existsSync()
? imageFile.lastModifiedSync().millisecondsSinceEpoch
: 0;
Key key = ObjectKey('$filePath:$lastModified');
return Image.file(
key: key,
imageFile,
width: 50,
height: 50,
);
}
}
class _ResolvedPhotoState {
String? photoPath;
bool fileExists;
Attachment? attachment;
_ResolvedPhotoState(
{required this.photoPath, required this.fileExists, this.attachment});
}
@riverpod
Future<_ResolvedPhotoState> _getPhotoState(Ref ref, String? photoId) async {
if (photoId == null) {
return _ResolvedPhotoState(photoPath: null, fileExists: false);
}
final queue = await ref.read(attachmentQueueProvider.future);
final photoPath = await queue.getLocalUri('$photoId.jpg');
bool fileExists = await File(photoPath).exists();
final row = await queue.db
.getOptional('SELECT * FROM attachments_queue WHERE id = ?', [photoId]);
if (row != null) {
Attachment attachment = Attachment.fromRow(row);
return _ResolvedPhotoState(
photoPath: photoPath, fileExists: fileExists, attachment: attachment);
}
return _ResolvedPhotoState(
photoPath: photoPath, fileExists: fileExists, attachment: null);
}
```
# Unit Testing
Source: https://docs.powersync.com/client-sdks/advanced/unit-testing
Set up unit tests for PowerSync in Dart and Flutter using mock databases and isolated sync configurations.
This guide is currently specific to the Dart/Flutter SDK. We may expand it to cover other SDKs in the future.
For unit testing your projects using PowerSync (for example, testing whether your queries run as expected), you'll need the `powersync-sqlite-core` binary in your project's root directory.
## Setup
1. Download the PowerSync SQLite binary
* Go to [powersync-sqlite-core Releases](https://github.com/powersync-ja/powersync-sqlite-core/releases)
* Download the binary for your OS
2. Rename the binary
* Remove the architecture suffix from the filename
* Examples:
* `powersync_x64.dll` → `powersync.dll` (Windows)
* `libpowersync_aarch64.dylib` → `libpowersync.dylib` (macOS)
* `libpowersync_x64.so` → `libpowersync.so` (Linux)
3. Place the binary
* Move the renamed binary to your project's root directory
## Example Test
This example shows basic unit testing with PowerSync in Flutter. For more information, see the [Flutter unit testing documentation](https://docs.flutter.dev/cookbook/testing/unit/introduction).
```dart theme={null}
import 'dart:io';
import 'package:powersync/powersync.dart';
import 'package:path/path.dart';
const schema = Schema([
Table('customers', [Column.text('name'), Column.text('email')])
]);
late PowerSyncDatabase testDB;
String getTestDatabasePath() async {
const dbFilename = 'powersync-test.db';
final dir = Directory.current.absolute.path;
return join(dir, dbFilename);
}
Future openTestDatabase() async {
testDB = PowerSyncDatabase(
schema: schema,
path: await getTestDatabasePath(),
logger: testLogger,
);
await testDB.initialize();
}
test('INSERT', () async {
await testDB.execute(
'INSERT INTO customers(name, email) VALUES(?, ?)', ['John Doe', 'john@hotmail.com']);
final results = await testDB.getAll('SELECT * FROM customers');
expect(results.length, 1);
expect(results, ['John Doe', 'john@hotmail.com']);
});
```
### Troubleshooting Extension Loading
If you have trouble loading the extension, confirm the following:
Ensure that your SQLite3 binary install on your system has extension loading enabled. You can confirm this by doing the following
* Run `sqlite3` in your command-line interface.
* In the sqlite3 prompt run `PRAGMA compile_options;`
* Check the output for the option `ENABLE_LOAD_EXTENSION`.
* If you see `ENABLE_LOAD_EXTENSION`, it means extension loading is enabled.
If the above steps don't work, you can also confirm if extension loading is enabled by trying to load the extension in your command-line interface.
* Run `sqlite3` in your command-line interface.
* Run `.load /path/to/file/libpowersync.dylib` (macOS) or `.load /path/to/file/libpowersync.so` (Linux) or `.load /path/to/file/powersync.dll` (Windows).
* If this runs without error, then extension loading is enabled. If it fails with an error message about extension loading being disabled, then it’s not enabled in your SQLite installation.
If it is not enabled, you will have to download a compiled SQLite binary with extension loading enabled (e.g. using Homebrew) or [compile SQLite](https://www.sqlite.org/howtocompile.html) with extension loading enabled and
include it in your project's folder alongside the extension.
# Cascading Delete
Source: https://docs.powersync.com/client-sdks/cascading-delete
Implement cascading deletes in client-side SQLite using triggers and upload queue handlers.
PowerSync [uses SQLite views](/architecture/client-architecture#client-side-schema-and-sqlite-database-structure) instead of standard tables, so SQLite features like foreign key constraints and cascading deletes are not available.
There's no built-in support for cascading deletes on the client, but you can achieve this in two ways:
1. Manual deletion in a transaction - Delete all related records in a single transaction (recommended for most cases)
Every local mutation performed against SQLite via the PowerSync SDK will be returned in `uploadData`. As long as you're using `.execute()` for the mutation, the operation will be present in the upload queue.
2. Triggers - Create triggers on the [internal tables](/architecture/client-architecture#client-side-schema-and-sqlite-database-structure) (more complex, but more automatic)
You create triggers on the internal tables (not the views defined by the client schema), similar to what is done [here](https://github.com/powersync-ja/powersync-js/blob/e77b1abfbed91988de1f4c707c24855cd66b2219/demos/react-supabase-todolist/src/app/utils/fts_setup.ts#L50).
## Example: Manual Transaction
This example from the [React Native To-Do List demo app](https://github.com/powersync-ja/powersync-js/tree/main/demos/react-native-supabase-todolist) shows how to delete a `list` and all its associated `todos` in a single transaction:
```typescript theme={null}
const deleteList = async (id: string) => {
await system.powersync.writeTransaction(async (tx) => {
// Delete associated todos
await tx.execute(`DELETE FROM ${TODO_TABLE} WHERE list_id = ?`, [id]);
// Delete list record
await tx.execute(`DELETE FROM ${LIST_TABLE} WHERE id = ?`, [id]);
});
};
```
Every mutation performed via `.execute()` is added to the upload queue and returned in `uploadData`. PowerSync will also delete local records when your backend performs cascade deletes on the source database (as long as those tables are in the publication).
For example, if you delete a record from the local `lists` table and Supabase cascade deletes a record from the `todo` table, PowerSync will also delete the local `todo` record when online.
# Expo Go Support
Source: https://docs.powersync.com/client-sdks/frameworks/expo-go-support
Run PowerSync in Expo Go without native modules using @powersync/adapter-sql-js.
Expo Go is a sandbox environment that allows you to quickly test your application without building a development build. To enable PowerSync in Expo Go, we provide a JavaScript-based database adapter: [`@powersync/adapter-sql-js`](https://www.npmjs.com/package/@powersync/adapter-sql-js).
# @powersync/adapter-sql-js
`@powersync/adapter-sql-js` is a development package for PowerSync which uses SQL.js to provide a pure JavaScript SQLite implementation. This eliminates the need for native dependencies and enables development with Expo Go and other JavaScript-only environments. Under the hood, it uses our custom fork [powersync-sql-js](https://github.com/powersync-ja/powersync-sql-js) - a fork of SQL.js (SQLite compiled to JavaScript via Emscripten) that loads the [PowerSync SQLite core extension](https://github.com/powersync-ja/powersync-sqlite-core).
This package is in an **alpha** release.
**Expo Go Sandbox Environment Only** This adapter is specifically designed for Expo Go and similar JavaScript-only environments. It will be much slower than native database adapters and has limitations. Every write operation triggers a complete rewrite of the entire database file to persistent storage, not just the changed data. In addition to the performance overheads, this adapter doesn't provide any of the SQLite consistency guarantees - you may end up with missing data or a corrupted database file if the app is killed while writing to the database file.
## Usage
### Quickstart
1. Create a new Expo app:
```bash theme={null}
npx create-expo-app@latest my-app
```
2. Navigate to your app directory and start the development server:
```bash theme={null}
cd my-app && npm run ios
```
3. In a new terminal tab, install PowerSync dependencies:
```bash theme={null}
npm install @powersync/react-native @powersync/adapter-sql-js
```
4. Replace the code in `app/(tabs)/index.tsx` with:
```tsx app/(tabs)/index.tsx theme={null}
import { SQLJSOpenFactory } from "@powersync/adapter-sql-js";
import { PowerSyncDatabase, Schema } from "@powersync/react-native";
import { useEffect, useState } from "react";
import { Text } from "react-native";
export const powerSync = new PowerSyncDatabase({
schema: new Schema({}), // todo: define the schema - see Next Steps below
factory: new SQLJSOpenFactory({
dbFilename: "example.db",
}),
});
export default function HomeScreen() {
const [version, setVersion] = useState(null);
useEffect(() => {
powerSync.get("select powersync_rs_version();").then((r) => {setVersion(JSON.stringify(r))});
}, []);
return (
<>{version && PowerSync Initialized - {version} }
);
}
```
1. Install the SQL.js adapter:
```bash theme={null}
npm install @powersync/adapter-sql-js
```
2. Set up PowerSync by using the Sql.js factory:
```tsx SystemProvider.tsx theme={null}
import { SQLJSOpenFactory } from "@powersync/adapter-sql-js";
import { PowerSyncDatabase, Schema } from "@powersync/react-native";
import { useEffect, useState } from "react";
import { Text } from "react-native";
export const powerSync = new PowerSyncDatabase({
schema: new Schema({}), // todo: define the schema - see Next Steps below
factory: new SQLJSOpenFactory({
dbFilename: "example.db",
}),
});
export default function HomeScreen() {
const [version, setVersion] = useState(null);
useEffect(() => {
powerSync.get("select powersync_rs_version();").then((r) => {setVersion(JSON.stringify(r))});
}, []);
return (
<>{version && PowerSync Initialized - {version} }
);
}
```
## Next Steps
After adding PowerSync to your app:
1. [**Define what data to sync by setting up Sync Rules**](/sync/rules/overview)
2. [**Implement your SQLite client schema**](/client-sdks/reference/react-native-and-expo#1-define-the-client-side-schema)
3. [**Connect to PowerSync and your backend**](/client-sdks/reference/react-native-and-expo#3-integrate-with-your-backend)
## Data Persistence
The default version of this adapter uses in-memory persistence, but you can specify your own `persister` option to the open factory.
See an example in the package [README](https://www.npmjs.com/package/@powersync/adapter-sql-js).
## Moving Beyond Expo Go
When you're ready to move beyond the Expo Go sandbox environment - whether for native development builds or production deployment - we recommend switching to our recommended native database adapter,
based on [OP-SQLite](https://www.npmjs.com/package/@op-engineering/op-sqlite).
OP-SQLite cannot run in Expo Go because it requires native code compilation. Specifically, PowerSync needs a SQLite implementation that can load the PowerSync SQLite core extension, which isn't possible in Expo Go's prebuilt app container.
This adapter provides better performance, full SQLite consistency guarantees, and is suitable for both development builds and production deployment. See the SDKs [Installation](/client-sdks/reference/react-native-and-expo#installation) details for setup instructions.
### Switching Between Adapters - Example
If you want to keep using Expo Go alongside development and production builds, you can switch between different adapters based on the Expo `executionEnvironment`:
```js SystemProvider.tsx theme={null}
import { SQLJSOpenFactory } from "@powersync/adapter-sql-js";
import { DatabaseSource, PowerSyncDatabase } from "@powersync/react-native";
import Constants from "expo-constants";
const isExpoGo = Constants.executionEnvironment === "storeClient";
const source: DatabaseSource = isExpoGo ?
{ factory: new SQLJSOpenFactory({ dbFilename: "app.db" }) }
: { database: { dbFilename: "sqlite.db" } };
export const powerSync = new PowerSyncDatabase({
schema: AppSchema,
...source,
});
```
# Dart/Flutter Web Support (Beta)
Source: https://docs.powersync.com/client-sdks/frameworks/flutter-web-support
Set up the PowerSync Flutter SDK for web with Chrome and WASM support.
Since version 1.9.0, web support for Flutter is in a **beta** release. It is functionally ready for production use, provided that you've tested your use cases.
Please see the [Limitations](#limitations) detailed below.
## Demo App
The easiest way to test Flutter Web support is to run the [Supabase Todo-List](https://github.com/powersync-ja/powersync.dart/tree/main/demos/supabase-todolist) demo app:
1. Clone the [powersync.dart](https://github.com/powersync-ja/powersync.dart/tree/main) repo.
2. Run `dart pub get` and `melos run prepare` in the repo's root
3. cd into the `demos/supabase-todolist` folder
4. If you haven’t yet: `cp lib/app_config_template.dart lib/app_config.dart` (optionally update this config with your own Supabase and PowerSync project details).
5. Run `flutter run -d chrome`
## Installing PowerSync in Your Own Project
Install the [latest version](https://pub.dev/packages/powersync/versions) of the package, for example:
```bash theme={null}
dart pub add powersync:'^2.0.0'
```
### Additional Config
#### Assets
Web support requires `sqlite3.wasm` and worker (`powersync_db.worker.js` and `powersync_sync.worker.js`) assets to be served from the web application. They can be downloaded to the web directory by running the following command in your application's root folder.
```bash theme={null}
dart run powersync:setup_web
```
The same code is used for initializing native and web `PowerSyncDatabase` clients.
#### OPFS for Improved Performance
This SDK supports different storage modes of the SQLite database with varying levels of performance and compatibility:
* **OPFS** (Origin-Private File System): A fast and modern file system implementation, but not available on older browsers or Safari private browsing.
* **IndexedDB**: Highly compatible with different browsers, but performance is slow.
PowerSync will detect what features the current browser supports to pick the best option.
Starting with version 2.2.0 of the PowerSync Dart/Flutter SDK, OPFS is used on all major browsers (Chrome, Firefox and Safari).
In older versions of the SDK, apps had to be served with special headers (`Cross-Origin-Opener-Policy` and `Cross-Origin-Embedder-Policy`) to support OPFS on Safari.
These headers can be removed in version 2.2.0 of the SDK.
After upgrading the SDK and removing these headers, existing OPFS databases in Safari continue to work.
If you didn't use those headers before, old Safari users with IndexedDB databases will keep using them (but new users opening your app for the first time will get a faster OPFS database).
There is no data loss in either scenario.
## Limitations
The API for Web is essentially the same as for native platforms, however, some features within `PowerSyncDatabase` clients are not available.
### Imports
Flutter Web does not support importing directly from `sqlite3.dart` as it uses `dart:ffi`.
Change imports from:
```dart theme={null}
import 'package:powersync/sqlite3.dart`;
// or
import 'package:sqlite3/sqlite3.dart`;
```
to:
```dart theme={null}
import 'package:sqlite3/sqlite3_common.dart';
```
in code which needs to run on the Web platform. Isolated native-specific code can still import from `sqlite3.dart`.
### Database Connections
Web database connections do not support concurrency. A single database connection is used. `readLock` and `writeLock` contexts do not implement checks for preventing writable queries in read connections and vice-versa.
Direct access to the synchronous `CommonDatabase` (`sqlite.Database` equivalent for web) connection is not available. `computeWithDatabase` is not available on web.
# Next.js + PowerSync
Source: https://docs.powersync.com/client-sdks/frameworks/next-js
Learn how to build highly responsive Next.js applications with PowerSync.
## Introduction
This guide shows you how to add PowerSync to a Next.js app for real-time updates across users and devices, instant interactions, and continued functionality when offline. You install the Web SDK, define a client-side schema, and connect sync so components query the local database instead of blocking on the network.
For a runnable reference that includes API routes for JWTs, JWKS, and writes to Postgres, clone the [Next.js demo app](https://github.com/powersync-ja/powersync-js/tree/main/demos/example-nextjs) in the [powersync-js](https://github.com/powersync-ja/powersync-js) repository and follow its README (Docker, local Postgres, and the PowerSync Service).
PowerSync is tailored for client-side applications — there isn't much benefit to using SSR with PowerSync. Some frameworks like Next.js push towards enabling SSR by default, which means code is evaluated in a Node.js runtime. The PowerSync Web SDK requires browser APIs which are not available in Node.js. For ergonomics, the SDK performs no-ops if used in Node.js (rather than throwing errors), but you should not expect any data from PowerSync during server-side rendering. If you are using SSR in your application, we recommend explicitly isolating PowerSync to client-side code.
## Setup
### Next.js Project Setup
Let's start by bootstrapping a new Next.js application using [`create-next-app`](https://nextjs.org/docs/app/api-reference/cli/create-next-app).
```shell npm theme={null}
npx create-next-app@latest my-powersync-app
```
```shell yarn theme={null}
yarn create next-app my-powersync-app
```
```shell pnpm theme={null}
pnpm create next-app my-powersync-app
```
When running this command you'll be presented with a few options. The PowerSync suggested selection for the setup options Next.js offers are:
```shell theme={null}
Would you like to use TypeScript? Yes
Would you like to use ESLint? Yes
Would you like to use Tailwind CSS? Yes
Would you like your code inside a `src/` directory? Yes
Would you like to use App Router? (recommended) Yes
Would you like to use Turbopack for `next dev`? Yes
Would you like to customize the import alias (`@/*` by default)? Yes
```
Turbopack is supported in Next.js 16+. If you're using an older version of Next.js, see the [Webpack configuration (legacy)](#webpack-configuration-legacy) section below.
### Install PowerSync Dependencies
Using PowerSync in a Next.js application will require the use of the [PowerSync Web SDK](https://www.npmjs.com/package/@powersync/web) and it's peer dependencies.
In addition to this we'll also install [`@powersync/react`](https://www.npmjs.com/package/@powersync/react), which provides several hooks and providers for easier integration.
```shell npm theme={null}
npm install @powersync/web @powersync/react
```
```shell yarn theme={null}
yarn add @powersync/web @powersync/react
```
```shell pnpm theme={null}
pnpm install @powersync/web @powersync/react
```
### Copy Worker Assets
When using Turbopack, you need to copy the PowerSync worker files to your public directory. Add a `postinstall` script to your `package.json`:
```json package.json theme={null}
{
"scripts": {
"postinstall": "powersync-web copy-assets -o public"
}
}
```
Then run the script to copy the assets:
```shell npm theme={null}
npm run postinstall
```
```shell yarn theme={null}
yarn postinstall
```
```shell pnpm theme={null}
pnpm postinstall
```
This copies the pre-bundled worker files to `public/@powersync/`, which are required since Turbopack doesn't support dynamic imports of workers yet.
Add `public/@powersync/*` to your `.gitignore` file since these are generated assets.
## Next.js Config Setup
For Next.js 16+ with Turbopack, the configuration is minimal:
```typescript next.config.ts theme={null}
module.exports = {
images: {
disableStaticImages: true
},
turbopack: {}
};
```
Run `pnpm dev` to start the development server and check that everything compiles correctly, before moving onto the next section.
### Webpack Configuration (Legacy)
If you're using an older version of Next.js (before 16) or prefer to use Webpack, use this configuration instead:
```typescript next.config.ts theme={null}
module.exports = {
webpack: (config: any, { isServer }: any) => {
config.experiments = {
...config.experiments,
asyncWebAssembly: true,
topLevelAwait: true,
};
if (!isServer) {
config.module.rules.push({
test: /\.wasm$/,
type: "asset/resource",
});
}
return config;
}
}
```
## Configure a PowerSync Instance
Now that we've got our project setup, let's create a new PowerSync Cloud instance and connect our client to it.
For the purposes of this demo, we'll be using Supabase as the backend source database that PowerSync will connect to.
To set up a new PowerSync instance, follow the steps covered in the [Installation - Database Connection](/configuration/source-db/connection) docs page.
## Configure PowerSync in Your Project
### Add Core PowerSync Files
Start by adding a new directory in `./src/lib` named `powersync`.
#### `AppSchema`
Create a new file called `AppSchema.ts` in the newly created `powersync` directory and add your App Schema to the file. Here is an example of this.
```typescript lib/powersync/AppSchema.ts theme={null}
import { column, Schema, Table } from '@powersync/web';
const lists = new Table({
created_at: column.text,
name: column.text,
owner_id: column.text
});
const todos = new Table(
{
list_id: column.text,
created_at: column.text,
completed_at: column.text,
description: column.text,
created_by: column.text,
completed_by: column.text,
completed: column.integer
},
{ indexes: { list: ['list_id'] } }
);
export const AppSchema = new Schema({
todos,
lists
});
// For types
export type Database = (typeof AppSchema)['types'];
export type TodoRecord = Database['todos'];
// OR:
// export type Todo = RowType;
export type ListRecord = Database['lists'];
```
This defines the local SQLite database schema and PowerSync will hydrate the tables once the SDK connects to the PowerSync instance.
#### `BackendConnector`
Create a new file called `BackendConnector.ts` in the `powersync` directory and add the following to the file.
```typescript lib/powersync/BackendConnector.ts theme={null}
import { type CommonPowerSyncDatabase, type PowerSyncBackendConnector, UpdateType } from '@powersync/web';
export class BackendConnector implements PowerSyncBackendConnector {
private powersyncUrl: string | undefined;
private powersyncToken: string | undefined;
constructor() {
this.powersyncUrl = process.env.NEXT_PUBLIC_POWERSYNC_URL;
// This token is for development only.
// For production applications, integrate with an auth provider or custom auth.
this.powersyncToken = process.env.NEXT_PUBLIC_POWERSYNC_TOKEN;
}
async fetchCredentials() {
// TODO: Use an authentication service or custom implementation here.
if (this.powersyncToken == null || this.powersyncUrl == null) {
return null;
}
return {
endpoint: this.powersyncUrl,
token: this.powersyncToken
};
}
async uploadData(database: CommonPowerSyncDatabase): Promise {
const transaction = await database.getNextCrudTransaction();
if (!transaction) {
return;
}
try {
for (const op of transaction.crud) {
// The data that needs to be changed in the remote db
const record = { ...op.opData, id: op.id };
switch (op.op) {
case UpdateType.PUT:
// TODO: Instruct your backend API to CREATE a record
break;
case UpdateType.PATCH:
// TODO: Instruct your backend API to PATCH a record
break;
case UpdateType.DELETE:
//TODO: Instruct your backend API to DELETE a record
break;
}
}
await transaction.complete();
} catch (error: any) {
console.error(`Data upload error - discarding`, error);
await transaction.complete();
}
}
}
```
There are two core functions to this file:
* `fetchCredentials()` - Used to return a JWT token to the PowerSync Service for authentication.
* `uploadData()` - Used to upload changes captured in the local SQLite database that need to be sent to the backend source database, in this case Supabase. We'll get back to this further down.
You'll notice that we need to add a `.env` file to our project which will contain two variables:
* `NEXT_PUBLIC_POWERSYNC_URL` - This is the PowerSync instance url. You can grab this from the PowerSync Cloud dashboard.
* `NEXT_PUBLIC_POWERSYNC_TOKEN` - For development purposes we'll be using a development token. To generate one, please follow the steps outlined in [Development Token](/configuration/auth/development-tokens) from our installation docs.
### Create the PowerSync Provider
Add a new file at `./src/lib/powersync/PowerSyncProvider.tsx`. This is the client-side boundary that initializes the `PowerSyncDatabase` and exposes it via React Context.
```typescript lib/powersync/PowerSyncProvider.tsx theme={null}
'use client';
import { AppSchema } from '@/lib/powersync/AppSchema';
import { BackendConnector } from '@/lib/powersync/BackendConnector';
import { PowerSyncContext } from '@powersync/react';
import { PowerSyncDatabase, WASQLiteOpenFactory, createConsoleLogger, LogLevels } from '@powersync/web';
import React, { Suspense } from 'react';
const logger = createConsoleLogger({ minLevel: LogLevels.debug });
let dbInstance: PowerSyncDatabase | null = null;
function getDB(): PowerSyncDatabase {
if (dbInstance) return dbInstance;
dbInstance = new PowerSyncDatabase({
database: {
dbFilename: 'powersync.db',
// Use the pre-bundled worker from public/@powersync/
// This is required since Turbopack doesn't support dynamic imports of workers yet
worker: '/@powersync/worker.js'
},
schema: AppSchema,
// Use the pre-bundled sync worker from public/@powersync/
sync: { worker: '/@powersync/worker.js' },
logger
});
dbInstance.connect(new BackendConnector());
return dbInstance;
}
export function PowerSyncProvider({ children }: { children: React.ReactNode }) {
return (
{children}
);
}
```
The `'use client'` directive marks this as a Client Component — PowerSync requires browser APIs, so initialization must happen on the client. The lazy `getDB()` singleton ensures the database is only instantiated once (and only in the browser), which avoids duplicate connections when modules are re-evaluated during development.
The worker paths point to the pre-bundled workers copied to the public directory by the `powersync-web copy-assets` command. `db.connect()` is called with an instance of `BackendConnector`, which will validate the token returned from `fetchCredentials` and start syncing with the PowerSync Service.
#### Update `layout.tsx`
Update `RootLayout` to wrap `children` with the `PowerSyncProvider`. Importantly, **the root layout should remain a Server Component** — do not add `'use client'` here. This is the standard Next.js App Router convention and is required if you want to export `metadata` (or use `generateMetadata`) from the layout. The client boundary is drawn at `PowerSyncProvider`, which is all that's needed to get PowerSync running in the browser.
```typescript app/layout.tsx theme={null}
import type { Metadata } from 'next';
import { PowerSyncProvider } from '@/lib/powersync/PowerSyncProvider';
import { Geist, Geist_Mono } from "next/font/google";
import "./globals.css";
const geistSans = Geist({
variable: "--font-geist-sans",
subsets: ["latin"],
});
const geistMono = Geist_Mono({
variable: "--font-geist-mono",
subsets: ["latin"],
});
export const metadata: Metadata = {
title: 'PowerSync Next.js Example'
};
export default function RootLayout({
children,
}: Readonly<{
children: React.ReactNode;
}>) {
return (
{children}
);
}
```
#### Use PowerSync
##### Reading Data
In our `page.tsx` we can now use the `useQuery` hook or other PowerSync functions to read data from the SQLite database and render the results in our application.
```typescript app/page.tsx theme={null}
'use client';
import { useState, useEffect } from 'react';
import { useQuery, useStatus, usePowerSync } from '@powersync/react';
export default function Page() {
// Hook
const powersync = usePowerSync();
// Get database status information e.g. downloading, uploading and lastSycned dates
const status = useStatus();
// Example 1: Reactive Query
const { data: lists } = useQuery("SELECT * FROM lists");
// Example 2: Standard query
const [lists, setLists] = useState([]);
useEffect(() => {
powersync.getAll('SELECT * from lists').then(setLists)
}, []);
return (
{lists.map(list => {list.name} )}
)
}
```
##### Writing Data
Using the `execute` function we can also write data into our local SQLite database.
```typescript theme={null}
await powersync.execute("INSERT INTO lists (id, created_at, name, owner_id) VALUES (?, ?, ?, ?)", [uuid(), new Date(), "Test", user_id]);
```
Changes made against the local data will be stored in the upload queue and will be processed by the `uploadData` in the BackendConnector class.
# Nuxt Integration
Source: https://docs.powersync.com/client-sdks/frameworks/nuxt
Build Nuxt apps that use client-side SQLite for real-time updates across users and devices, instant interactions and continued functionality when offline.
## Introduction
`@powersync/nuxt` is a Nuxt module that wraps [`@powersync/vue`](https://www.npmjs.com/package/@powersync/vue) to integrate PowerSync with Nuxt applications. It re-exports all `@powersync/vue` composables so this is the only PowerSync dependency you need. It also adds a PowerSync diagnostics panel to Nuxt Devtools for inspecting sync status, local data, config, and logs.
**Alpha:** The Nuxt PowerSync integration is currently in Alpha. APIs and behavior may change. We welcome feedback in [Discord](https://discord.com/invite/powersync) or on [GitHub](https://github.com/powersync-ja/powersync-js).
PowerSync is tailored for client-side applications — there isn't much benefit to using SSR with PowerSync. Nuxt evaluates plugins server-side unless you use the `.client.ts` suffix. The PowerSync Web SDK requires browser APIs that are not available in Node.js; it performs no-ops in a Node.js runtime rather than throwing errors, but you should not expect any data from PowerSync during server-side rendering. Always create your PowerSync plugin as `plugins/powersync.client.ts` to ensure it runs only in the browser.
For a complete working example, see the [Nuxt + Supabase Todo List demo](https://github.com/powersync-ja/powersync-js/tree/main/demos/nuxt-supabase-todolist).
## Setup
### Install PowerSync Dependencies
```shell npm theme={null}
npm install @powersync/nuxt
```
```shell pnpm theme={null}
pnpm add @powersync/nuxt @powersync/vue @powersync/web
```
With **npm** (v7+), peer dependencies are installed automatically. With **pnpm**, you must install peer dependencies explicitly, as shown above.
### Add the Module
Add `@powersync/nuxt` to the `modules` array in `nuxt.config.ts` and include the required Vite configuration:
```typescript nuxt.config.ts theme={null}
export default defineNuxtConfig({
modules: ['@powersync/nuxt'],
vite: {
optimizeDeps: {
exclude: ['@powersync/web']
},
worker: {
format: 'es'
}
}
});
```
If you are using Tailwind CSS in your project, see [Known Issues](/tools/nuxt-inspector#known-issues) on the Nuxt Inspector page.
## Configure PowerSync in Your Project
### Define your Schema
Create a file at `powersync/AppSchema.ts` and define your local SQLite schema. PowerSync will hydrate these tables once the SDK connects to your PowerSync instance.
```typescript powersync/AppSchema.ts theme={null}
import { column, Schema, Table } from '@powersync/web';
const lists = new Table({
created_at: column.text,
name: column.text,
owner_id: column.text
});
const todos = new Table(
{
list_id: column.text,
created_at: column.text,
completed_at: column.text,
description: column.text,
created_by: column.text,
completed_by: column.text,
completed: column.integer
},
{ indexes: { list: ['list_id'] } }
);
export const AppSchema = new Schema({
todos,
lists
});
// For types
export type Database = (typeof AppSchema)['types'];
export type TodoRecord = Database['todos'];
export type ListRecord = Database['lists'];
```
Learn more about defining your schema in the [JavaScript Web SDK reference](/client-sdks/reference/javascript-web#1-define-the-client-side-schema).
### Create your Connector
Create a file at `powersync/PowerSyncConnector.ts`. The connector handles authentication and uploading local changes to your backend.
```typescript powersync/PowerSyncConnector.ts theme={null}
import { UpdateType, type PowerSyncBackendConnector } from '@powersync/web';
export class PowerSyncConnector implements PowerSyncBackendConnector {
async fetchCredentials() {
// Return a JWT for the PowerSync Service to authenticate the client.
// See https://docs.powersync.com/installation/authentication-setup
// For quick local testing, use a development token:
// https://docs.powersync.com/installation/authentication-setup/development-tokens
return {
endpoint: '[Your PowerSync instance URL]',
token: '[Your auth token]'
};
}
async uploadData(db: any) {
// Send local changes to your backend.
// See https://docs.powersync.com/client-sdk-references/javascript-web#3-integrate-with-your-backend
const transaction = await db.getNextCrudTransaction();
if (!transaction) return;
try {
for (const op of transaction.crud) {
const record = { ...op.opData, id: op.id };
switch (op.op) {
case UpdateType.PUT:
// TODO: send CREATE to your backend API
break;
case UpdateType.PATCH:
// TODO: send PATCH to your backend API
break;
case UpdateType.DELETE:
// TODO: send DELETE to your backend API
break;
}
}
await transaction.complete();
} catch (error: any) {
console.error('Data upload error - discarding', error);
await transaction.complete();
}
}
}
```
### Create the Plugin
Create a [Nuxt plugin](https://nuxt.com/docs/guide/directory-structure/plugins) at `plugins/powersync.client.ts`. The `.client.ts` suffix ensures this only runs in the browser.
```typescript plugins/powersync.client.ts theme={null}
import { NuxtPowerSyncDatabase, createPowerSyncPlugin } from '@powersync/nuxt';
import { AppSchema } from '~/powersync/AppSchema';
import { PowerSyncConnector } from '~/powersync/PowerSyncConnector';
export default defineNuxtPlugin({
async setup(nuxtApp) {
const db = new NuxtPowerSyncDatabase({
database: {
dbFilename: 'my-app.sqlite'
},
schema: AppSchema
});
const connector = new PowerSyncConnector();
await db.init();
await db.connect(connector);
const plugin = createPowerSyncPlugin({ database: db });
nuxtApp.vueApp.use(plugin);
}
});
```
## Using PowerSync
The module automatically exposes all `@powersync/vue` composables. You can import and use them directly in any component or composable.
### Reading Data
```vue components/TodoList.vue theme={null}
Status: {{ status.connected ? 'Connected' : 'Offline' }}
Loading...
```
### Writing Data
Use `execute` to write to the local SQLite database. Changes are queued and uploaded to your backend via `uploadData` in the connector.
```typescript theme={null}
import { usePowerSync } from '@powersync/nuxt';
import { v4 as uuid } from 'uuid';
const powersync = usePowerSync();
await powersync.execute(
'INSERT INTO lists (id, created_at, name, owner_id) VALUES (?, ?, ?, ?)',
[uuid(), new Date().toISOString(), 'My List', currentUserId]
);
```
## Kysely ORM (Optional)
The module optionally exposes a `usePowerSyncKysely()` composable for type-safe query building. You must install the driver and opt in via config.
Install the driver:
```shell npm theme={null}
npm install @powersync/kysely-driver
```
```shell pnpm theme={null}
pnpm add @powersync/kysely-driver
```
Enable it in `nuxt.config.ts`:
```typescript nuxt.config.ts theme={null}
export default defineNuxtConfig({
modules: ['@powersync/nuxt'],
powersync: {
kysely: true
},
vite: {
optimizeDeps: {
exclude: ['@powersync/web']
},
worker: {
format: 'es'
}
}
});
```
Then use `usePowerSyncKysely` with your schema's `Database` type for full type safety:
```typescript theme={null}
import { usePowerSyncKysely } from '@powersync/nuxt';
import { type Database } from '~/powersync/AppSchema';
const db = usePowerSyncKysely();
const lists = await db.selectFrom('lists').selectAll().execute();
```
## DevTools
The `@powersync/nuxt` module includes a PowerSync diagnostics panel that integrates with Nuxt DevTools. It shows sync status, local data, config, and logs. See [Nuxt DevTools Integration](/tools/nuxt-inspector) for setup instructions and available views.
# React Hooks
Source: https://docs.powersync.com/client-sdks/frameworks/react
Use PowerSync React hooks including useQuery, useStatus, and useSuspenseQuery to build reactive components with live query results.
The `@powersync/react` package provides React hooks for use with the [JavaScript Web SDK](/client-sdks/reference/javascript-web) or [React Native SDK](/client-sdks/reference/react-native-and-expo). These hooks are designed to support reactivity, and can be used to automatically re-render React components when query results update or to access PowerSync connectivity status changes.
For advanced watch query features like incremental updates and differential results for React Hooks, see [Live Queries / Watch Queries](/client-sdks/watch-queries).
The full API Reference and example code can be found here:
**Availability**
Support for React Native Web is available since versions 1.12.1 of the PowerSync [React Native SDK](/client-sdks/reference/react-native-and-expo) and 1.8.0 if the [JavaScript Web SDK](/client-sdks/reference/javascript-web), and is currently in a **beta** release.
A demo app showcasing this functionality is available here:
=> {},
options: { placeholder: string | undefined } = { placeholder: undefined }
) {
const isWeb = Platform.OS === 'web';
let name: string | null = null;
if (isWeb) {
name = window.prompt(`${title}\n${description}`, options.placeholder);
} else {
name = await new Promise((resolve) => {
rnPrompt(
title,
description,
(input) => {
resolve(input);
},
{ placeholder: options.placeholder, style: 'shimo' }
);
});
}
await onInput(name);
}
```
Which can then be used agnostically in a component.
```js theme={null}
import { Button } from 'react-native';
import { prompt } from 'util/prompt';
{
prompt(
'Add a new Todo',
'',
(text) => {
if (!text) {
return;
}
return createNewTodo(text);
},
{ placeholder: 'Todo description' }
);
}}
/>;
```
# TanStack Query & TanStack DB
Source: https://docs.powersync.com/client-sdks/frameworks/tanstack
Use PowerSync with TanStack Query for server state caching and TanStack DB for optimistic local-first mutations in JavaScript apps.
## TanStack Query
PowerSync integrates with [TanStack Query](https://tanstack.com/query/latest/docs/framework/react/overview) (formerly React Query) through the `@powersync/tanstack-react-query` package.
#### Additional hooks
We plan to support more TanStack Query hooks over time. If there are specific hooks you're interested in, please let us know on [Discord](https://discord.gg/powersync).
### Example Use Case
When navigating to or refreshing a page, you may notice a brief UI "flicker" (10-50ms). Here are a few ways to manage this with TanStack Query:
* **First load**: When a page loads for the first time, use a loading indicator or a Suspense fallback to handle queries. See the [examples](https://www.npmjs.com/package/@powersync/tanstack-react-query#usage).
* **Subsequent loads**: With TanStack's query caching, subsequent loads of the same page won't refire queries, which reduces the flicker effect.
* **Block navigation until components are ready**: Using `useSuspenseQuery`, you can ensure that navigation from page A to page B only happens after the queries for page B have loaded. You can do this by combining `useSuspenseQuery` with the `The PowerSync TanStack DB collection is currently in an [Alpha](/resources/feature-status) release.
### Quick Start
Install the TanStackDB-PowerSync collection package with a PowerSync SDK. Then define your schema, initialize the PowerSync database, and create a collection. Optionally [connect a backend connector](/configuration/app-backend/client-side-integration) for sync.
```bash theme={null}
npm install @tanstack/powersync-db-collection @powersync/web
```
```bash theme={null}
yarn add @tanstack/powersync-db-collection @powersync/web
```
```bash theme={null}
pnpm install @tanstack/powersync-db-collection @powersync/web
```
```ts theme={null}
// Other SDKs are also supported
import { Schema, Table, column } from '@powersync/web';
import { createCollection } from '@tanstack/react-db';
import { powerSyncCollectionOptions } from '@tanstack/powersync-db-collection';
// Define schema and init PowerSync database
const APP_SCHEMA = new Schema({
documents: new Table({
name: column.text,
author: column.text,
created_at: column.text,
archived: column.integer
})
});
const db = new PowerSyncDatabase({
database: { dbFilename: 'app.sqlite' },
schema: APP_SCHEMA
});
// Optional: db.connect(connector) for backend sync
// Create a TanStack DB collection (types inferred from table)
const documentsCollection = createCollection(
powerSyncCollectionOptions({
database: db,
table: APP_SCHEMA.props.documents
})
);
```
### Features
* **Blazing fast in-memory queries** — Built on differential data flow, live queries update incrementally instead of re-running entire queries, so they stay fast even for complex queries across multiple collections.
* **Reactive data flow** — Live queries update automatically when underlying data changes, so components re-render only when necessary.
* **Optimistic updates** — Mutations apply to local state immediately for instant feedback; TanStack DB keeps optimistic state on top of synced data and rolls back automatically if the server request fails.
* **Cross-collection queries** — Live queries can join across collections, seamlessly querying PowerSync and other TanStack DB collections simultaneously.
* **Schema validation and rich types** — Use a custom schema (e.g. Zod) to validate mutations and transform SQLite types into rich JavaScript types such as `Date`, boolean, and JSON. You can keep SQLite-compatible input for writes and expose transformed types on read, or accept rich input with a separate deserialization schema for synced data. See [Create a TanStack DB collection](https://tanstack.com/db/latest/docs/collections/powersync-collection#option-3-transform-sqlite-input-types-to-rich-output-types).
* **Metadata tracking** — Attach custom metadata to insert, update, and delete operations. PowerSync persists it and exposes it in `CrudEntry` when processing uploads in your connector. See [Accessing metadata during upload](https://tanstack.com/db/latest/docs/collections/powersync-collection#accessing-metadata-during-upload).
* **Configuration options** — `powerSyncCollectionOptions` supports schema and deserialization schemas, optional serializers, `onDeserializationError`, and `syncBatchSize`. See [PowerSync Collection](https://tanstack.com/db/latest/docs/collections/powersync-collection#4-create-a-tanstack-db-collection) (Configuration Options).
* **TanStackDB transactions** — Batch multiple operations with `PowerSyncTransactor` and `createTransaction`, control commit timing, and wait for persistence. See [Advanced transactions](https://tanstack.com/db/latest/docs/collections/powersync-collection#advanced-transactions).
### Framework Support
PowerSync works with all TanStack DB framework adapters:
* React ([`@tanstack/react-db`](https://tanstack.com/db/latest/docs/framework/react/overview))
* Vue ([`@tanstack/vue-db`](https://tanstack.com/db/latest/docs/framework/vue/overview))
* Solid ([`@tanstack/solid-db`](https://tanstack.com/db/latest/docs/framework/solid/overview))
* Svelte ([`@tanstack/svelte-db`](https://tanstack.com/db/latest/docs/framework/svelte/overview))
* Angular ([`@tanstack/angular-db`](https://tanstack.com/db/latest/docs/framework/angular/overview))
### Documentation
For advanced watch query features like incremental updates and differential results for Vue Hooks, see [Live Queries / Watch Queries](/client-sdks/watch-queries).
The full API Reference and example code can be found here:
openDatabase() async {
...
await configureFts(db);
}
```
To simplify implementation these examples make use of SQLite migrations. The migrations are run in [migrations/fts\_setup.dart](https://github.com/powersync-ja/powersync.dart/blob/master/demos/supabase-todolist/lib/migrations/fts_setup.dart) in the Flutter implementation. Here we use the [sqlite\_async](https://pub.dev/documentation/sqlite_async/latest/) Dart package to generate the migrations.
```dart theme={null}
// migrations/fts_setup.dart
/// This is where you can add more migrations to generate FTS tables
/// that correspond to the tables in your schema and populate them
/// with the data you would like to search on
Future configureFts(PowerSyncDatabase db) async {
migrations
..add(createFtsMigration(
migrationVersion: 1,
tableName: 'lists',
columns: ['name'],
tokenizationMethod: 'porter unicode61'))
..add(createFtsMigration(
migrationVersion: 2,
tableName: 'todos',
columns: ['description', 'list_id'],
));
await migrations.migrate(db);
}
```
The `createFtsMigration` function is key and corresponds to the below (Dart example):
```dart theme={null}
// migrations/fts_setup.dart
/// Create a Full Text Search table for the given table and columns
/// with an option to use a different tokenizer otherwise it defaults
/// to unicode61. It also creates the triggers that keep the FTS table
/// and the PowerSync table in sync.
SqliteMigration createFtsMigration(
{required int migrationVersion,
required String tableName,
required List columns,
String tokenizationMethod = 'unicode61'}) {
String internalName =
schema.tables.firstWhere((table) => table.name == tableName).internalName;
String stringColumns = columns.join(', ');
return SqliteMigration(migrationVersion, (tx) async {
// Add FTS table
await tx.execute('''
CREATE VIRTUAL TABLE IF NOT EXISTS fts_$tableName
USING fts5(id UNINDEXED, $stringColumns, tokenize='$tokenizationMethod');
''');
// Copy over records already in table
await tx.execute('''
INSERT INTO fts_$tableName(rowid, id, $stringColumns)
SELECT rowid, id, ${generateJsonExtracts(ExtractType.columnOnly, 'data', columns)}
FROM $internalName;
''');
// Add INSERT, UPDATE and DELETE and triggers to keep fts table in sync with table
await tx.execute('''
CREATE TRIGGER IF NOT EXISTS fts_insert_trigger_$tableName AFTER INSERT
ON $internalName
BEGIN
INSERT INTO fts_$tableName(rowid, id, $stringColumns)
VALUES (
NEW.rowid,
NEW.id,
${generateJsonExtracts(ExtractType.columnOnly, 'NEW.data', columns)}
);
END;
''');
await tx.execute('''
CREATE TRIGGER IF NOT EXISTS fts_update_trigger_$tableName AFTER UPDATE
ON $internalName BEGIN
UPDATE fts_$tableName
SET ${generateJsonExtracts(ExtractType.columnInOperation, 'NEW.data', columns)}
WHERE rowid = NEW.rowid;
END;
''');
await tx.execute('''
CREATE TRIGGER IF NOT EXISTS fts_delete_trigger_$tableName AFTER DELETE
ON $internalName BEGIN
DELETE FROM fts_$tableName WHERE rowid = OLD.rowid;
END;
''');
});
}
```
After this is run, you should have the following tables and triggers in your SQLite DB:
search(String searchTerm, String tableName) async {
String searchTermWithOptions = _createSearchTermWithOptions(searchTerm);
return await db.execute(
'SELECT * FROM fts_$tableName WHERE fts_$tableName MATCH ? ORDER BY rank',
[searchTermWithOptions]);
}
```
## Implementations in Other SDKs
* The React, React Native and Swift implementations do not use migrations to create the FTS tables. They create the FTS tables separately, see for example:
* [utils/fts\_setup.ts](https://github.com/powersync-ja/powersync-js/blob/main/demos/react-supabase-todolist/src/app/utils/fts_setup.ts) (React)
* [library/fts/fts\_setup.ts](https://github.com/powersync-ja/powersync-js/blob/main/demos/react-native-supabase-todolist/library/fts/fts_setup.ts) (React Native)
* [PowerSync/FtsSetup](https://github.com/powersync-ja/powersync-swift/blob/11def989bfbdc4f6ffe192192cd076abe17743c0/Demo/PowerSyncExample/PowerSync/FtsSetup.swift#L121) (Swift)
* See below for relevant snippets in the demo implementations.
```ts theme={null}
// See https://github.com/powersync-ja/powersync-js/blob/main/demos/react-supabase-todolist/src/components/providers/SystemProvider.tsx#L41
SystemProvider = ({ children }: { children: React.ReactNode }) => {
...
React.useEffect(() => {
...
configureFts();
})
}
```
```ts theme={null}
// See https://github.com/powersync-ja/powersync-js/blob/main/demos/react-native-supabase-todolist/library/powersync/system.ts#L75
export class System {
...
powersync: PowerSyncDatabase;
...
async init() {
...
await configureFts(this.powersync);
}
}
```
```swift theme={null}
// See https://github.com/powersync-ja/powersync-swift/blob/main/Demo/PowerSyncExample/PowerSync/SystemManager.swift#L89
```
# Experimental: High Performance Diffs
Source: https://docs.powersync.com/client-sdks/high-performance-diffs
Get row-level change notifications using trigger-based table diffs in JavaScript SDKs.
## Overview
While [basic/incremental watch queries](/client-sdks/watch-queries) enable reactive UIs by automatically re‑running queries when underlying data changes and returning updated results, they don't specify which individual rows were modified. To get these details, you can use [**differential watch queries**](/client-sdks/watch-queries#differential-watch-queries), which return a structured diff between successive query results. However, on large result sets they can be slow because they re‑run the query and compare full results (e.g., scanning \~1,000 rows to detect 1 new item). That’s why we introduced **trigger‑based table diffs**: a more performant approach that uses SQLite triggers to record changes on a table as they happen. This means that the overhead associated with tracking these changes overhead is more proportional to the number of rows inserted, updated, or deleted.
**JavaScript Only**: Trigger-based table diffs are currently only supported in our JavaScript SDKs, starting from:
* Web v1.26.0
* React Native v1.24.0
* Node.js v0.10.0
The `db.triggers` APIs are experimental. We're actively seeking feedback on:
* API design and developer experience
* Additional features or optimizations needed
Join our [Discord community](https://discord.gg/powersync) to share your experience and get help.
### Comparison: Trigger-Based Diffs vs Differential Watch Queries
* **Scope**: Trigger-based diffs track row-level changes on a single table. Differential watches work with arbitrary query results (including joins).
* **Overhead**: Trigger-based diffs do per-row work at write time (overhead grows with number of affected rows). Differential watches re-query and compare result sets on each change (overhead grows with result set size).
* **Processing path**: Trigger-based diffs record changes at write time and require a `writeLock` during processing (only a single `writeLock` is allowed). Differential watches run on read connections and re-query/compare results on each change (often concurrent on some platforms).
* **Storage/shape**: Trigger-based diffs store changes as rows in a temporary SQLite table that you can query with SQL. Differential watch diffs are exposed to app code as JS objects/arrays.
* **Filtering**: Trigger-based diffs can filter/skip storing diff records inside the SQLite trigger, which prevents emissions on a lower level. Differential watches query the SQLite DB on any change to the query's dependent tables, and the changes are filtered after querying SQLite.
**In summary**: Differential watch queries are the most flexible since they work with arbitrary multi-table queries, but they can be slow on large result sets. For those cases, trigger-based diffs are more efficient, though they only track a single table and add some write overhead.
### Trigger-Based Diffs
Trigger-based diffs create temporary SQLite triggers and a temporary table to record row‑level inserts, updates, and deletes as they happen. You can then query the diff table with SQL to process the changes.
**SQLite triggers and PowerSync views**
In PowerSync, the tables you define in the client schema are exposed as SQLite views. The actual data is stored in underlying SQLite tables, with each row's values encoded as JSON (commonly in a single `data` column).
SQLite cannot attach triggers to INSERT/UPDATE/DELETE operations on views — triggers must target the underlying base tables. The `db.triggers` API handles these details for you:
* You can reference the view name in `source`; PowerSync resolves and targets the corresponding underlying table internally.
* Column filters are applied by inspecting JSON changes in the underlying row and determining whether the configured columns changed.
* Diff rows can be queried as if they were real columns (not raw JSON) using the `withExtractedDiff(...)` helper.
You can also create your own triggers manually (for example, as shown in the [Full‑Text Search example](/client-sdks/full-text-search)), but be mindful of the view/trigger limitation and target the underlying table rather than the view.
### Tracking and Reacting to Changes (Recommended)
The primary API is `trackTableDiff`. It wraps the lower-level trigger setup, automatically manages a `writeLock` during processing, exposes a `DIFF` table alias to join against, and cleans up when you call the returned `stop()` function. Think of it as an automatic "watch" that processes diffs as they occur.
```javascript theme={null}
const stop = await db.triggers.trackTableDiff({
// PowerSync source table/view to trigger and track changes from.
// This should be present in the PowerSync database's schema.
source: 'todos',
// Specifies which columns from the source table to track in the diff records.
// Defaults to all columns in the source table.
// Use an empty array to track only the ID and operation.
columns: ['list_id'],
// Required WHEN clause per operation to filter inside the trigger. Use 'TRUE' to track all.
when: { INSERT: sanitizeSQL`json_extract(NEW.data, '$.list_id') = ${firstList.id}` },
onChange: async (context) => {
// // Fetches the todo records that were inserted during this diff
const newTodos = await context.withDiff(/* sql */ `
SELECT todos.*
FROM DIFF
JOIN todos ON DIFF.id = todos.id
`);
// Handle new todos here
}
});
// Later, dispose triggers and internal resources
await stop();
```
#### Filtering with `when`
The required `when` parameter lets you add conditions that determine when the triggers should fire. This corresponds to a SQLite [WHEN](https://sqlite.org/lang_createtrigger.html) clause in the trigger body.
* Use `NEW` for `INSERT`/`UPDATE` and `OLD` for `DELETE`.
* Row data is stored as JSON in the `data` column; the row identifier is `id`.
* Use `json_extract(NEW.data, '$.column')` or `json_extract(OLD.data, '$.column')` to reference logical columns.
* Set the clause to `'TRUE'` to track all changes for a given operation.
Example:
```javascript theme={null}
const stop = await db.triggers.trackTableDiff({
source: 'todos',
when: {
// Track all INSERTs
INSERT: 'TRUE',
// Only UPDATEs where status becomes 'active' for a specific record
UPDATE: sanitizeSQL`NEW.id = ${sanitizeUUID('abcd')} AND json_extract(NEW.data, '$.status') = 'active'`,
// Only DELETEs for a specific list
DELETE: sanitizeSQL`json_extract(OLD.data, '$.list_id') = 'abcd'`
}
});
```
The strings in `when` are embedded directly into the SQLite trigger creation SQL. Sanitize any user‑derived values. The `sanitizeSQL` helper performs some basic sanitization; additional sanitization is recommended.
### Lower-Level: createDiffTrigger (Advanced)
Set up temporary triggers that write change operations into a temporary table you control. Prefer `trackTableDiff` unless you need to manage lifecycle and locking manually (e.g., buffer diffs to process them later). Note that since the table is created as a temporary table on the SQLite write connection, it can only be accessed within operations performed inside a `writeLock`.
```javascript theme={null}
// Define the temporary table to store the diff
const tempTable = 'listsDiff';
// Configure triggers to record INSERT and UPDATE operations on `lists`
const dispose = await db.triggers.createDiffTrigger({
// PowerSync source table/view to trigger and track changes from.
// This should be present in the PowerSync database's schema.
source: 'lists',
// Destination table to send changes to.
// This table is created internally as a SQLite temporary table.
// This table will be dropped once the trigger is removed.
destination: tempTable,
// Required WHEN clause per operation to filter inside the trigger. Use 'TRUE' to track all.
when: {
INSERT: 'TRUE',
UPDATE: sanitizeSQL`json_extract(NEW.data, '$.name') IS NOT NULL`
},
// Specifies which columns from the source table to track in the diff records.
// Defaults to all columns in the source table.
// Use an empty array to track only the ID and operation.
columns: ['name']
});
// ... perform writes on `lists` ...
// Consume and clear changes within a writeLock
await db.writeLock(async (tx) => {
const changes = await tx.getAll(/* sql */ `
SELECT * FROM ${tempTable}
`);
// Process changes here
// Clear after processing
await tx.execute(/* sql */ `DELETE FROM ${tempTable};`);
});
// Later, clean up triggers and temp table
await dispose();
```
# Infinite Scrolling
Source: https://docs.powersync.com/client-sdks/infinite-scrolling
Implement infinite scrolling with PowerSync using paginated SQL queries against the local SQLite database.
There are a few ways to accomplish infinite scrolling with PowerSync, either by querying data from the local SQLite database, or by [lazy-loading](https://en.wikipedia.org/wiki/Lazy_loading) or lazy-syncing data from your backend.
Here is an overview of the different options with pros and cons:
### 1) Pre-sync all data and query the local database
PowerSync currently [performs well](/resources/performance-and-limits) with syncing up to 1,000,000 rows per client.
This means that in many cases, you can sync a sufficient amount of data to let a user keep scrolling a list or feed that basically feels "infinite" to them.
| Pros | Cons |
| ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| It works offline and is low-latency (data loads quickly from the local database). We don't need to load data from the backend via the network when the user reaches the bottom of the page/feed/list. | There will be cases where this approach won't work because the total volume of data might become too large for the local database - for example, when there's a wide range of tables that the user needs to be able to infinite scroll. Your app allows the user to apply filters to the displayed data, which results in fewer pages displayed from a large dataset, and therefore limited scrolling. |
### 2) Control data sync using subscription or client parameters
**Sync Streams** (recommended): Use [subscription parameters](/sync/streams/parameters#subscription-parameters) to subscribe to specific data on demand. For example, a client can subscribe to a specific "page" of data when the user scrolls to it. This is more flexible than client parameters — each subscription is independent and multiple tabs/views can subscribe with different parameters simultaneously.
**Sync Rules** (legacy): PowerSync supports the use of [client parameters](/sync/rules/client-parameters) which are specified directly by the client. The app can dynamically change these parameters on the client-side and they can be accessed in Sync Rules on the server-side. The developer can use these parameters to limit/control which data is synced, but since they are not trusted (because they are not passed via the JWT authentication token) they should not be used for access control. You should still filter data by e.g. user ID for access control purposes (using [token parameters](/sync/rules/parameter-queries) from the JWT).
Usage example: To lazy-load/lazy-sync data for infinite scrolling, you could split your data into 'pages' and use a subscription parameter (Sync Streams) or client parameter (Sync Rules) to specify which pages to sync to a user.
| Pros | Cons |
| ---------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------- |
| Does not require updating flags in your backend source database. Enables client-side control over what data is synced. | We can only sync additional data when the user is online. There will be latency while the user waits for the additional data to sync. |
### 3) Sync limited data and then load more data from an API
In this scenario we can sync a smaller number of rows to the user initially. If the user reaches the end of the page/feed/list, we make an online API call to load additional data from the backend to display to the user.
| Pros | Cons |
| ---------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| This requires syncing less data to each user, which will result in a faster initial sync time. | We can only load additional data when the user is online. There will be some latency to load the additional data (similar to a cloud-first app making API calls). In your app code, records loaded from the API will have to be treated differently from the records loaded from the local SQLite database. |
### 4) Client-side triggers a server-side function to flag data to sync
You could add a flag to certain records in your backend source database which are used by your [Sync Streams](/sync/streams/overview) or [Sync Rules](/sync/rules/overview) to determine which records to sync to specific users. Then your app could make an API call which triggers a function that updates the flags on certain records, causing more records to be synced to the user.
## Questions?
Ask on [Discord](https://discord.gg/powersync) if you need help implementing infinite scrolling.
# Dart/Flutter ORM Support
Source: https://docs.powersync.com/client-sdks/orms/flutter-orm-support
Use the Drift ORM with the PowerSync Flutter SDK via drift_sqlite_async.
ORM support is available via the following package (currently in a beta release):
```js Drizzle theme={null}
const result = await db.select().from(users);
// [{ id: '1', name: 'user1' }, { id: '2', name: 'user2' }]
```
```js PowerSync theme={null}
const result = await powerSyncDb.getAll('SELECT * from users');
// [{ id: '1', name: 'user1' }, { id: '2', name: 'user2' }]
```
### Insert Operations
```js Drizzle theme={null}
await db.insert(users).values({ id: '1', name: 'John' });
const result = await db.select().from(users);
// [{ id: '1', name: 'John' }]
```
```js PowerSync theme={null}
await powerSyncDb.execute('INSERT INTO users (id, name) VALUES(1, ?)', ['John']);
const result = await powerSyncDb.getAll('SELECT * from users');
// [{ id: '1', name: 'John' }]
```
### Delete Operations
```js Drizzle theme={null}
await db.insert(users).values({ id: '2', name: 'Ben' });
await db.delete(users).where(eq(users.name, 'Ben'));
const result = await db.select().from(users);
// []
```
```js PowerSync theme={null}
await powerSyncDb.execute('INSERT INTO users (id, name) VALUES(2, ?)', ['Ben']);
await powerSyncDb.execute(`DELETE FROM users WHERE name = ?`, ['Ben']);
const result = await powerSyncDb.getAll('SELECT * from users');
// []
```
### Update Operations
```js Drizzle theme={null}
await db.insert(users).values({ id: '3', name: 'Lucy' });
await db.update(users).set({ name: 'Lucy Smith' }).where(eq(users.name, 'Lucy'));
const result = await db.select({ name: users.name }).from(users).get();
// 'Lucy Smith'
```
```js PowerSync theme={null}
await powerSyncDb.execute('INSERT INTO users (id, name) VALUES(3, ?)', ['Lucy']);
await powerSyncDb.execute('UPDATE users SET name = ? WHERE name = ?', ['Lucy Smith', 'Lucy']);
const result = await powerSyncDb.get('SELECT name FROM users WHERE name = ?', ['Lucy Smith'])
// 'Lucy Smith'
```
### Watched Queries
For watched queries with Drizzle it's recommended to use the `watch()` function from the Drizzle integration which takes in a Drizzle query.
```js Drizzle theme={null}
const query = db.select().from(users);
db.watch(query, {
onResult(results) {
console.log(results);
},
});
// [{ id: '1', name: 'John' }]
```
```js PowerSync theme={null}
powerSyncDb.watch("select * from users", [], {
onResult(results) {
console.log(results.rows?._array);
},
});
// [{ id: '1', name: 'John' }]
```
### Transactions
```js Drizzle theme={null}
await db.transaction(async (transaction) => {
await transaction.insert(users).values({ id: "4", name: "James" });
await transaction
.update(users)
.set({ name: "Lucy James Smith" })
.where(eq(users.name, "James"));
});
const result = await db.select({ name: users.name }).from(users).get();
// 'James Smith'
```
```js PowerSync theme={null}
await powerSyncDb.writeTransaction(async (transaction) => {
await transaction.execute('INSERT INTO users (id, name) VALUES(4, ?)', ['James']);
await transaction.execute("UPDATE users SET name = ? WHERE name = ?", ['James Smith', 'James']);
})
const result = await powerSyncDb.get('SELECT name FROM users WHERE name = ?', ['James Smith'])
// 'James Smith'
```
## Developer Notes
### Table Constraint Restrictions
The Drizzle ORM relies on the underlying PowerSync table definitions which are subject to certain limitations.
This means that most Drizzle [constraint features](https://orm.drizzle.team/docs/indexes-constraints) (such as cascading deletes, foreign checks, unique) are currently not supported.
# Kysely
Source: https://docs.powersync.com/client-sdks/orms/js/kysely
Use the Kysely query builder with PowerSync's JavaScript and React Native SDKs.
(powerSyncDb)
```
For more information on Kysely typing, see [their documentation](https://kysely.dev/docs/getting-started#types).
## Usage Examples
Below are examples comparing Kysely and PowerSync syntax for common database operations.
### Select Operations
```js Kysely theme={null}
const result = await db.selectFrom('users').selectAll().execute();
// [{ id: '1', name: 'user1' }, { id: '2', name: 'user2' }]
```
```js PowerSync theme={null}
const result = await powerSyncDb.getAll('SELECT * from users');
// [{ id: '1', name: 'user1' }, { id: '2', name: 'user2' }]
```
### Insert Operations
```js Kysely theme={null}
await db.insertInto('users').values({ id: '1', name: 'John' }).execute();
const result = await db.selectFrom('users').selectAll().execute();
// [{ id: '1', name: 'John' }]
```
```js PowerSync theme={null}
await powerSyncDb.execute('INSERT INTO users (id, name) VALUES(1, ?)', ['John']);
const result = await powerSyncDb.getAll('SELECT * from users');
// [{ id: '1', name: 'John' }]
```
### Delete Operations
```js Kysely theme={null}
await db.insertInto('users').values({ id: '2', name: 'Ben' }).execute();
await db.deleteFrom('users').where('name', '=', 'Ben').execute();
const result = await db.selectFrom('users').selectAll().execute();
// []
```
```js PowerSync theme={null}
await powerSyncDb.execute('INSERT INTO users (id, name) VALUES(2, ?)', ['Ben']);
await powerSyncDb.execute(`DELETE FROM users WHERE name = ?`, ['Ben']);
const result = await powerSyncDb.getAll('SELECT * from users');
// []
```
### Update Operations
```js Kysely theme={null}
await db.insertInto('users').values({ id: '3', name: 'Lucy' }).execute();
await db.updateTable('users').where('name', '=', 'Lucy').set('name', 'Lucy Smith').execute();
const result = await db.selectFrom('users').select('name').executeTakeFirstOrThrow();
// 'Lucy Smith'
```
```js PowerSync theme={null}
await powerSyncDb.execute('INSERT INTO users (id, name) VALUES(3, ?)', ['Lucy']);
await powerSyncDb.execute('UPDATE users SET name = ? WHERE name = ?', ['Lucy Smith', 'Lucy']);
const result = await powerSyncDb.get('SELECT name FROM users WHERE name = ?', ['Lucy Smith'])
// 'Lucy Smith'
```
### Watched Queries
For watched queries with Kysely it's recommended to use the `watch()` function from the wrapper package which takes in a Kysely query.
```js Kysely theme={null}
const query = db.selectFrom('users').selectAll();
db.watch(query, {
onResult(results) {
console.log(results);
},
});
// [{ id: '1', name: 'John' }]
```
```js PowerSync theme={null}
powerSyncDb.watch("select * from users", [], {
onResult(results) {
console.log(results.rows?._array);
},
});
// [{ id: '1', name: 'John' }]
```
### Transactions
```js Kysely theme={null}
await db.transaction().execute(async (transaction) => {
await transaction.insertInto('users').values({ id: '4', name: 'James' }).execute();
await transaction.updateTable('users').where('name', '=', 'James').set('name', 'James Smith').execute();
});
const result = await db.selectFrom('users').select('name').executeTakeFirstOrThrow();
// 'James Smith'
```
```js Kysely with Raw SQL theme={null}
await db.transaction().execute(async (transaction) => {
await sql`INSERT INTO users (id, name) VALUES ('4', 'James');`.execute(transaction)
await transaction.updateTable('users').where('name', '=', 'James').set('name', 'James Smith').execute();
});
const result = await db.selectFrom('users').select('name').executeTakeFirstOrThrow();
// 'James Smith'
```
```js PowerSync theme={null}
await powerSyncDb.writeTransaction((transaction) => {
await transaction.execute('INSERT INTO users (id, name) VALUES(4, ?)', ['James']);
await transaction.execute("UPDATE users SET name = ? WHERE name = ?", ['James Smith', 'James']);
})
const result = await powerSyncDb.get('SELECT name FROM users WHERE name = ?', ['James Smith'])
// 'James Smith'
```
# JavaScript ORMs Overview
Source: https://docs.powersync.com/client-sdks/orms/js/overview
Overview of ORM and query builder integrations for PowerSync JavaScript SDKs, including Drizzle, Kysely, and TanStack DB.
The following ORMs and query libraries are officially supported:
Kysely query builder for PowerSync.
Drizzle ORM for PowerSync.
TanStack DB collection for PowerSync.
# TanStack DB
Source: https://docs.powersync.com/client-sdks/orms/js/tanstack-db
Use PowerSync with TanStack DB for optimistic mutations and live queries in JavaScript apps.
# Kotlin SQL Libraries
Source: https://docs.powersync.com/client-sdks/orms/kotlin/overview
Overview of SQL mapping library options for the PowerSync Kotlin SDK, including Room and SQLDelight.
The PowerSync Kotlin SDK allows syncing SQLite databases with your backend source database, and gives you full control over which queries you run on your client.
Manually writing SQL queries and parsing results can be prone to errors though.
Libraries like [SQLDelight](https://sqldelight.github.io/sqldelight) and [Room](https://developer.android.com/jetpack/androidx/releases/room) make this process safer by validating your schema and queries at compile-time, as well as generating code to map from raw SQLite rows into statically typed structures.
Starting with version `1.6.0` of the PowerSync Kotlin SDK, both SQLDelight and Room are officially supported on all platforms!
We've recently added support for these libraries. We're still seeking feedback on developer experience
and don't have complete examples for them yet.
Contributions and feedback are welcome!
Join our [Discord community](https://discord.gg/powersync) to share your experience and get help.
Use SQLDelight on PowerSync databases.
Use PowerSync with Room databases.
If you're not sure which library to use, consider that Room requires [raw tables](/client-sdks/advanced/raw-tables) and is more complex to set up, so:
* SQLDelight is easier to use if you're starting with an existing PowerSync database.
* We mainly recommend the Room integration if you have an existing Room database you want to add sync to.
# Room (Beta)
Source: https://docs.powersync.com/client-sdks/orms/kotlin/room
Use Android Room with PowerSync's Kotlin SDK.
Room support is currently in a beta release.
PowerSync supports the Room database library for Kotlin (Multiplatform).
Version 1.14.1 and later of `com.powersync:integration-room` requires [Room 3.0](https://developer.android.com/jetpack/androidx/releases/room3). If your app still uses Room 2.x, upgrade your Room dependency to 3.0 before updating the integration.
PowerSync acts as an addon to your existing Room database, which means that (unlike with most other PowerSync SDKs)
you are still responsible for schema management.
Room requires [raw tables](/client-sdks/advanced/raw-tables), as the views managed by PowerSync are incompatible with
the schema verification when Room opens the database.
To add PowerSync to your Room database,
1. Add a dependency on `com.powersync:core` and `com.powersync:integration-room`.
2. Add a dependency on `androidx.sqlite:sqlite-bundled`: Since PowerSync uses the [PowerSync SQLite core extension](https://github.com/powersync-ja/powersync-sqlite-core) (external SQLite extensions are unsupported on
the platform SQLite libraries on both Android and iOS), you need to bundle a SQLite with your app.
On the `RoomDatabase.Builder`, call `setDriver()` with a PowerSync-enabled driver:
```Kotlin theme={null}
val driver = BundledSQLiteDriver().also {
it.loadPowerSyncExtension() // Extension method by PowerSync
}
Room.databaseBuilder(...).setDriver(driver).build()
```
## Encrypted Databases
When using PowerSync with Room, PowerSync wraps the Room database instead of opening its own connection. To use an encrypted database with Room, implement `SQLiteDriver` by delegating to `PersistentConnectionFactory` and pass that to `RoomDatabase.Builder.setDriver`:
```kotlin theme={null}
import androidx.sqlite.SQLiteConnection
import androidx.sqlite.SQLiteDriver
import com.powersync.PersistentConnectionFactory
import kotlinx.io.files.Path
class PowerSyncConnectionFactoryAsSqliteDriver(
private val powersyncFactory: PersistentConnectionFactory
) : SQLiteDriver {
override fun open(fileName: String): SQLiteConnection {
if (fileName == ":memory:") {
return powersyncFactory.openInMemoryConnection()
}
val path = Path(fileName)
return powersyncFactory.openConnection(
dbFilename = path.name,
dbDirectory = path.parent?.toString(),
readOnly = false
)
}
}
```
Depending on your target platform, pass an `AndroidEncryptedDatabaseFactory`, `JavaEncryptedDatabaseFactory`, or `NativeEncryptedDatabaseFactory` to the wrapper.
The factory automatically installs the PowerSync SQLite extension, so you can skip the `loadPowerSyncExtension()` step.
Using the custom driver, follow the [platform-specific instructions](https://developer.android.com/kotlin/multiplatform/room#creating-database)
for opening databases:
```kotlin theme={null}
val driver = PowerSyncConnectionFactoryAsSqliteDriver(
AndroidEncryptedDatabaseFactory(context, Key.Passphrase("your encryption key"))
)
// Example for Android, all other multiplatform targets are supported too.
val appContext = context.applicationContext
val dbFile = appContext.getDatabasePath("my_encrypted_room.db")
val roomDb = Room.databaseBuilder(appContext, dbFile.absolutePath)
.setDriver(driver)
.build()
```
## Setup
Because PowerSync syncs into tables that you've created with Room, it needs to know which SQL statements to run for
inserts, updates and deletes.
Let's say you had a table like the following:
```Kotlin theme={null}
@Entity(tableName = "todos")
data class TodoItem(
// Note that PowerSync uses textual ids (usually randomly-generated UUIDs)
@PrimaryKey val id: String
val description: String
@ColumnInfo(name="created_by") val authorId: String
)
```
To inform PowerSync about that table, include it as a `RawTable` in the schema:
```Kotlin theme={null}
val schema = Schema(
// For more information on raw tables, see https://docs.powersync.com/client-sdks/advanced/raw-tables
RawTable(
name = "todos",
schema = RawTableSchema(),
),
)
```
After these steps, you can open your Room database like you normally would. Then, you can use the
following method to obtain a `PowerSyncDatabase` instance which is backed by Room:
```Kotlin theme={null}
val schema = Schema(...)
val pool = RoomConnectionPool(yourRoomDatabase, schema)
val powersync = PowerSyncDatabase.opened(
pool = pool,
scope = this,
schema = schema,
identifier = "databaseName", // Prefer to use the same path/name as your Room database
logger = Logger,
)
```
The returned `PowerSyncDatabase` behaves just like a regular PowerSync database, meaning that you can call
`connect` to establish a sync connection:
```Kotlin theme={null}
powersync.connect(YourBackendConnector())
```
## Usage
To run queries, you can keep defining Room DAOs in the usual way:
```Kotlin theme={null}
@Dao
interface TodoItemsDao {
@Insert
suspend fun create(item: TodoItem)
@Query("SELECT * FROM todo_item")
fun watchAll(): Flow>
}
// ...
todoItemsDao.create(TodoItem(
id = Uuid.random()toHexDashString(),
title = "My first todo item",
authorId = currentUserId
))
todoItemsDao.watchAll().collect { items ->
println("This flow emits events for writes from Room and synced data from PowerSync")
}
```
## Local Writes
To transfer local writes from Room to PowerSync:
1. Create triggers on your Room tables to insert rows into `ps_crud`. See [raw tables](/client-sdks/advanced/raw-tables#capture-local-writes-with-triggers) for details.
2. Ensure the `RoomConnectionPool` is constructed with your `schema` (as shown above). When the schema is provided, the pool will notify PowerSync about writes to every raw table referenced in the schema.
3. Alternatively, after performing writes through Room, invoke:
```Kotlin theme={null}
pool.transferPendingRoomUpdatesToPowerSync()
```
This explicitly transfers any pending Room updates to PowerSync if you prefer to control the timing.
# SQLDelight (Beta)
Source: https://docs.powersync.com/client-sdks/orms/kotlin/sqldelight
Use SQLDelight with PowerSync's Kotlin SDK.
PowerSync supports the SQLDelight library to safely build and run SQL statements on all platforms.
SQLDelight support is currently in beta.
There are some limitations to be aware of:
1. PowerSync migrates all databases to `user_version` 1 when created (it will never downgrade a database). If you want to use SQLDelight's schema versioning, start from version `2`.
2. `CREATE TABLE` statements in `.sq` files are only used at build time to verify queries. At runtime, PowerSync creates tables as views from your schema and ignores those statements. If you want SQLDelight to manage the schema, configure PowerSync to use [raw tables](/client-sdks/advanced/raw-tables).
3. Functions and tables provided by the [PowerSync SQLite core extension](https://github.com/powersync-ja/powersync-sqlite-core) are not visible to `.sq` files currently. We may revisit this with a custom dialect in the future.
## Learn More
See our blog post: [Using ORMs With PowerSync](https://www.powersync.com/blog/using-orms-with-powersync)
# GRDB (Alpha)
Source: https://docs.powersync.com/client-sdks/orms/swift/grdb
Use the GRDB SQLite library with PowerSync's Swift SDK.
PowerSync integrates with the [GRDB library](https://github.com/groue/GRDB.swift), a powerful SQLite tool for Swift development. GRDB is a full-fledged SQLite ecosystem that offers SQLite connection creation and pooling, SQL generation (ORM functionality), database observation (reactive queries), robust concurrency, migrations, and SwiftUI integration with [GRDBQuery](https://github.com/groue/GRDBQuery).
This integration allows you to combine PowerSync's sync capabilities with GRDB's mature tooling and Swift-friendly patterns. It provides an easier adoption path for existing GRDB users while also enabling access to GRDB's ecosystem of libraries.
GRDB support was added in v1.9.0 of the PowerSync Swift SDK and is currently in an **alpha** release.
There are some limitations to be aware of:
* Updating the PowerSync schema using `updateSchema` is not yet supported.
* Xcode previews may not yet work correctly.
* You may see thread priority inversion warnings in Xcode. We're working to ensure consistent quality-of-service classes across threads.
* The schema definition process requires manually defining both the PowerSync `AppSchema` and GRDB record types separately. Future versions may allow these to be declared together or derived from each other.
## Features
When using GRDB with PowerSync:
* **Easier adoption for existing GRDB users**: The familiar GRDB API lowers the barrier to entry for teams already using GRDB.
* **Access to GRDB ecosystem**: Use libraries built on GRDB like GRDBQuery (SwiftUI data layer with automatic UI updates) and SQLiteData.
* **Type-safe query generation**: GRDB's ORM provides compile-time error checking and Swift-idiomatic patterns that make SQLite development more productive. You get features like database observation (similar to PowerSync's watch functionality), migration support, and record protocols that reduce boilerplate while maintaining flexibility to drop down to raw SQL when needed.
* **Direct SQLite access**: GRDB provides more direct access to the actual SQLite connections being used. This enables advanced SQLite operations like registering custom SQLite functions.
## Setup
This guide assumes that you have completed the [Getting Started](/client-sdks/reference/swift#getting-started) steps in the SDK documentation, or are at least familiar with them. The GRDB-specific configuration described below applies to the "Instantiate the PowerSync Database" step (step 2) in the Getting Started guide.
To set up PowerSync with GRDB, create a `DatabasePool` with PowerSync configuration:
```swift theme={null}
var config = Configuration()
try config.configurePowerSync(
schema: schema
)
let documentsDir = FileManager.default.urls(for: .documentDirectory, in: .userDomainMask).first!
let dbURL = documentsDir.appendingPathComponent("test.sqlite")
let pool = try DatabasePool(
path: dbURL.path,
configuration: config
)
```
You can then pass this pool when creating the `PowerSyncDatabase`:
```swift theme={null}
let powerSync = openPowerSyncWithGRDB(
pool: pool,
schema: schema,
identifier: "mydatabase.sqlite"
)
```
The returned `PowerSyncDatabase` behaves just like a regular PowerSync database, meaning that you can call `connect` to establish a sync connection.
## Usage
Using the `DatabasePool` in the PowerSync SDK shares the same locking mechanisms between instances of the `PowerSyncDatabase` and `DatabasePool`, so you can safely alternate between both clients.
You can use PowerSync queries:
```swift theme={null}
try await powerSync.execute(
"INSERT INTO users(id, name, count) VALUES(uuid(), 'steven', 1)"
)
let initialUsers = try await powerSync.getAll(
"SELECT * FROM users"
) { cursor in
try cursor.getString(name: "name")
}
print("initial users \(initialUsers)")
```
And also use GRDB queries:
```swift theme={null}
// Define a GRDB record type
struct Users: Codable, Identifiable, FetchableRecord, PersistableRecord {
var id: String
var name: String
var count: Int
enum Columns {
static let name = Column(CodingKeys.name)
static let count = Column(CodingKeys.count)
}
}
let grdbUsers = try await pool.read { db in
try Users.fetchAll(db)
}
```
## Demo App
The [PowerSync Swift GRDB Demo App](https://github.com/powersync-ja/powersync-swift/tree/main/Demos/GRDBDemo) showcases how to use GRDB with PowerSync.
## Architecture
The GRDB integration works by sharing the same underlying SQLite database between PowerSync and GRDB. Instead of PowerSync creating its own SQLite database instance (as in the standard implementation), the integration uses a GRDB `DatabasePool` that has been configured with the [PowerSync SQLite core extension](https://github.com/powersync-ja/powersync-sqlite-core) (required for PowerSync features).
When you create a `DatabasePool` with PowerSync configuration and pass it to `openPowerSyncWithGRDB`, PowerSync uses that same `DatabasePool` interface for all database operations.
This shared architecture means that you can use both the GRDB `DatabasePool` and PowerSync `PowerSyncDatabase` interfaces interchangeably.
# Client SDKs Overview
Source: https://docs.powersync.com/client-sdks/overview
Choose a PowerSync SDK for web, mobile, desktop, headless, or embedded apps.
PowerSync provides client SDKs for multiple frameworks. Each SDK manages a local SQLite database that syncs with your backend.
For getting started instructions, see our [Setup Guide](/intro/setup-guide).
## Choose Your SDK
Select your client framework to get started:
## Common Tasks
Once you've installed an SDK, these guides cover the core functionality:
Query your local SQLite database
Insert, update, and delete records
Build reactive UIs with live queries
Common patterns and code examples
## Additional Resources
Use type-safe ORMs with PowerSync
Platform compatibility for each SDK
Working demo apps and starter templates
# Reading Data
Source: https://docs.powersync.com/client-sdks/reading-data
Use SQL, watch queries, and supported ORMs to read data from local SQLite with the PowerSync SDK.
Your app reads synced data directly from local SQLite, so screens do not require backend read APIs with cache invalidation, and don't wait for network round trips. The same data remains readable when connectivity is interrupted.
## Basic Queries
Read data using standard SQL queries:
```typescript TypeScript theme={null}
// Get all todos
const todos = await db.getAll('SELECT * FROM todos');
// Get a single todo
const todo = await db.get('SELECT * FROM todos WHERE id = ?', [todoId]);
// Watch for changes (reactive query)
const stream = db.watch('SELECT * FROM todos WHERE list_id = ?', [listId]);
for await (const todos of stream) {
// Update UI when data changes
console.log(todos);
}
```
```kotlin Kotlin theme={null}
// Get all todos
val todos = database.getAll("SELECT * FROM todos") { cursor ->
Todo.fromCursor(cursor)
}
// Get a single todo
val todo = database.get("SELECT * FROM todos WHERE id = ?", listOf(todoId)) { cursor ->
Todo.fromCursor(cursor)
}
// Watch for changes
database.watch("SELECT * FROM todos WHERE list_id = ?", listOf(listId))
.collect { todos ->
// Update UI when data changes
}
```
```swift Swift theme={null}
// Get all todos
let todos = try await db.getAll(
sql: "SELECT * FROM todos",
mapper: { cursor in
TodoContent(
description: try cursor.getString(name: "description")!,
completed: try cursor.getBooleanOptional(name: "completed")
)
}
)
// Watch for changes
for try await todos in db.watch(
sql: "SELECT * FROM todos WHERE list_id = ?",
parameters: [listId]
) {
// Update UI when data changes
}
```
```dart Dart/Flutter theme={null}
// Get all todos
final todos = await db.getAll('SELECT * FROM todos');
// Get a single todo
final todo = await db.get('SELECT * FROM todos WHERE id = ?', [todoId]);
// Watch for changes
db.watch('SELECT * FROM todos WHERE list_id = ?', [listId])
.listen((todos) {
// Update UI when data changes
});
```
```csharp .NET theme={null}
// Define a result type with properties matching the schema columns (some columns omitted here for brevity)
// public class TodoResult { public string id; public string description; public int completed; ... }
// Get all todos
var todos = await db.GetAll("SELECT * FROM todos");
// Get a single todo
var todo = await db.Get("SELECT * FROM todos WHERE id = ?", [todoId]);
// You can also query without specifying a type to get dynamic results:
dynamic asset = await db.Get("SELECT id, description, make FROM assets");
Console.WriteLine($"Asset ID: {asset.id}");
```
## Live Queries / Watch Queries
For reactive UI updates that automatically refresh when data changes, use watch queries. These queries execute whenever dependent tables are modified.
See [Live Queries / Watch Queries](/client-sdks/watch-queries) for more details.
## ORM Support
PowerSync integrates with popular ORM libraries, which provide type safety and additional tooling. Using an ORM is often preferable to writing raw SQL queries, especially for common operations.
See [ORM Support](/client-sdks/orms/overview) to learn which ORMs PowerSync supports and how to get started.
## Advanced Topics
* [Usage Examples](/client-sdks/usage-examples) - Code examples for common use cases
* [Full-Text Search](/client-sdks/full-text-search) - Full-text search using the [SQLite FTS5 extension](https://www.sqlite.org/fts5.html)
* [Query JSON in SQLite](/client-sdks/advanced/query-json-in-sqlite) - Learn how to work with JSON data in SQLite
* [Infinite Scrolling](/client-sdks/infinite-scrolling) - Efficiently load large datasets
* [High Performance Diffs](/client-sdks/high-performance-diffs) - Efficiently get row changes for large datasets
# Capacitor SDK (beta)
Source: https://docs.powersync.com/client-sdks/reference/capacitor
Use PowerSync in Capacitor apps.
```text Build with AI icon="sparkles" wrap theme={null}
Install the PowerSync Agent Skills with: npx skills add powersync-ja/agent-skills. Then follow the skills to onboard this project to PowerSync using the Capacitor SDK.
```
This SDK is distributed via NPM
Refer to `packages/capacitor` in the `powersync-js` repo on GitHub
Full API reference for the SDK
Gallery of example projects/demo apps built with Capacitor and PowerSync
Changelog for the SDK
This SDK is currently in a [**beta** release](/resources/feature-status).
Because the SDK is largely built on our [Web SDK](/client-sdks/reference/javascript-web), much of its functionality is stable. The portion that relies on the [Capacitor Community SQLite](https://github.com/capacitor-community/sqlite) driver is considered production-ready for tested use cases. Review the [known limitations](#limitations) below.
**Built on the Web SDK**
The PowerSync Capacitor SDK is built on top of the [PowerSync Web SDK](/client-sdks/reference/javascript-web). It shares the same API and usage patterns as the Web SDK. The main differences are:
* It uses the Capacitor-specific SQLite implementation (`@capacitor-community/sqlite`) on native Android and iOS.
* Some features are not supported on native Android and iOS. See [Limitations](#limitations).
All code examples from the Web SDK apply to Capacitor. Import general components such as `Schema` and `Table` from `@powersync/web`, and `PowerSyncDatabase` from `@powersync/capacitor`. See the [JavaScript Web SDK reference](/client-sdks/reference/javascript-web) for ORM support, SPA framework integration, and developer notes.
## SDK Features
* **Real-time streaming of database changes**: Changes made by one user are instantly streamed to all other users with access to that data. This keeps clients automatically in sync without manual polling or refresh logic.
* **Direct access to a local SQLite database**: Data is stored locally, so apps can read and write instantly without network calls. This enables offline support and faster user interactions.
* **Asynchronous background execution**: The SDK performs database operations in the background to avoid blocking the application’s main thread. This means that apps stay responsive, even during heavy data activity.
* **Query subscriptions for live updates**: The SDK supports query subscriptions that automatically push real-time updates to client applications as data changes, keeping your UI reactive and up to date.
* **Automatic schema management**: PowerSync syncs schemaless data and applies a client-defined schema using SQLite views. This architecture means that PowerSync SDKs handle schema changes without explicit migrations on the client side.
## Installation
**Requirements**: The Capacitor SDK requires Capacitor 8 or later. Minimum platform versions are iOS 15.0 and Android API level 24. On iOS, both Swift Package Manager (SPM) and CocoaPods are supported.
If you are on Capacitor 6 or 7, follow the [Capacitor v8 upgrade guide](https://capacitorjs.com/docs/updating/8-0) before installing or upgrading `@powersync/capacitor`.
Add the [PowerSync Capacitor NPM package](https://www.npmjs.com/package/@powersync/capacitor) to your project:
```bash theme={null}
npm install @powersync/capacitor
```
```bash theme={null}
yarn add @powersync/capacitor
```
```bash theme={null}
pnpm install @powersync/capacitor
```
**Install Peer Dependencies**
You must also install the following peer dependencies:
```bash theme={null}
npm install @capacitor-community/sqlite @powersync/web
```
```bash theme={null}
yarn add @capacitor-community/sqlite @powersync/web
```
```bash theme={null}
pnpm install @capacitor-community/sqlite @powersync/web
```
After installing, sync your Capacitor project:
```bash theme={null}
npx cap sync
```
## Getting Started
**Prerequisites:** Before you start, connect your source database to the PowerSync Service and deploy Sync Streams. These are steps 1-4 in the [Setup Guide](/intro/setup-guide).
### 1. Define the Client-Side Schema
The client-side schema defines the tables and columns of the SQLite database that the PowerSync client SDK manages and that your app reads from and writes to. It is usually derived from your backend database schema and your [Sync Streams](/sync/streams/overview), and it can also include [local-only tables](/client-sdks/advanced/local-only-usage). You apply the schema when you instantiate the database in the next step.
Schema migrations are not required. The SDK syncs schemaless data and applies the schema to that data with SQLite views. The exception is [raw tables](/client-sdks/advanced/raw-tables), which you create and migrate yourself.
**Generate schema automatically**
In the [PowerSync Dashboard](https://dashboard.powersync.com/), select your project and instance and click the **Connect** button in the top bar to generate the client-side schema in your preferred language. The schema is generated from your Sync Streams. The [CLI](/tools/cli) offers the same function.
The generated schema does not include an `id` column. The client SDK creates an `id` column of type `text` automatically, so you do not need to declare it. See [Client ID](/sync/advanced/client-id) for details.
The available column types are `text`, `integer`, and `real`. These should match the values produced by your Sync Streams. If a value does not match, it is cast automatically. For details on how source database types map to SQLite types, see [Types](/sync/types).
**Example:**
**Imports:** You install `@powersync/capacitor`, but the Capacitor SDK extends the Web SDK, so you import general components from `@powersync/web` (installed as a peer dependency). See the [JavaScript Web SDK schema definition section](/client-sdks/reference/javascript-web#1-define-the-client-side-schema) for more advanced examples.
```js theme={null}
// AppSchema.ts
import { column, Schema, Table } from '@powersync/web';
const lists = new Table({
created_at: column.text,
name: column.text,
owner_id: column.text
});
const todos = new Table(
{
list_id: column.text,
created_at: column.text,
completed_at: column.text,
description: column.text,
created_by: column.text,
completed_by: column.text,
completed: column.integer
},
{ indexes: { list: ['list_id'] } }
);
export const AppSchema = new Schema({
todos,
lists
});
// For types
export type Database = (typeof AppSchema)['types'];
export type TodoRecord = Database['todos'];
// OR:
// export type Todo = RowType;
export type ListRecord = Database['lists'];
```
You do not need to declare an `id` column. PowerSync creates it automatically.
### 2. Instantiate the PowerSync Database
Next, instantiate the PowerSync database. PowerSync streams changes from your backend source database into the client-side SQLite database, based on your Sync Streams. Your app reads from and writes to this local database whether the user is online or offline.
**Example:**
The Capacitor `PowerSyncDatabase` detects the platform and uses the matching database driver:
* **Android and iOS**: Uses [Capacitor Community SQLite](https://github.com/capacitor-community/sqlite) for native database access
* **Web**: Falls back to the PowerSync Web SDK
```js theme={null}
import { PowerSyncDatabase } from '@powersync/capacitor';
// Import general components from the Web SDK package
import { Schema } from '@powersync/web';
import { Connector } from './Connector';
import { AppSchema } from './AppSchema';
/**
* The Capacitor PowerSyncDatabase will automatically detect the platform
* and use the appropriate database drivers.
*/
export const db = new PowerSyncDatabase({
// The schema you defined in the previous step
schema: AppSchema,
database: {
// Filename for the SQLite database — it's important to only instantiate one instance per file.
dbFilename: 'powersync.db'
}
});
```
When you use custom database factories, specify `CapacitorSQLiteOpenFactory` for Capacitor platforms:
```js theme={null}
import { PowerSyncDatabase } from '@powersync/capacitor';
import { WASQLiteOpenFactory, CapacitorSQLiteOpenFactory } from '@powersync/capacitor';
import { Schema } from '@powersync/web';
const db = new PowerSyncDatabase({
schema: AppSchema,
factory: isWeb
? new WASQLiteOpenFactory({ dbFilename: "mydb.sqlite" })
: new CapacitorSQLiteOpenFactory({ dbFilename: "mydb.sqlite" })
});
```
After you instantiate the PowerSync database, call the [connect()](https://powersync-ja.github.io/powersync-js/common/interfaces/CommonPowerSyncDatabase#connect) method to sync data with your backend.
This section assumes that you use PowerSync to sync your backend source database with SQLite in your app. To manage a local SQLite database without sync, instantiate the PowerSync database without calling `connect()` and see the [Local-Only](/client-sdks/advanced/local-only-usage) guide.
```js theme={null}
export const setupPowerSync = async () => {
// Uses the backend connector that you create in the next step
const connector = new Connector();
db.connect(connector);
};
```
### 3. Integrate with Your Backend
The backend connector connects the PowerSync client SDK to your application backend. The SDK uses it to:
1. Get an auth token to connect to the PowerSync instance.
2. Upload client-side writes to your backend API. The SDK places every write to the SQLite database in an upload queue and uploads the queue to your backend when the user is connected. Your backend then applies the changes to the source database.
The connector must implement two methods:
1. [PowerSyncBackendConnector.fetchCredentials](https://github.com/powersync-ja/powersync-js/blob/ed5bb49b5a1dc579050304fab847feb8d09b45c7/packages/common/src/client/connection/PowerSyncBackendConnector.ts#L16) - The SDK calls this method to get authentication credentials. It caches the credentials and calls the method again only when needed, for example on the first connection or when the token is near expiry. See [When `fetchCredentials()` is Called](/configuration/app-backend/client-side-integration#when-fetchcredentials-is-called) for details and [Authentication Setup](/configuration/auth/overview) for how to generate credentials.
2. [PowerSyncBackendConnector.uploadData](https://github.com/powersync-ja/powersync-js/blob/ed5bb49b5a1dc579050304fab847feb8d09b45c7/packages/common/src/client/connection/PowerSyncBackendConnector.ts#L24) - The SDK calls this method whenever it has client-side writes to upload to your backend API. Implement how those writes are processed and uploaded. See [When `uploadData()` is Called](/configuration/app-backend/client-side-integration#when-uploaddata-is-called) for triggers, throttling, and retry behavior, and [Writing Client Changes](/handling-writes/writing-client-changes) for the app backend implementation.
**Example:**
See the [JavaScript Web SDK backend integration section](/client-sdks/reference/javascript-web#3-integrate-with-your-backend) for connector examples with Supabase and Firebase authentication, and handling `uploadData` with batch operations.
```js theme={null}
import { UpdateType } from '@powersync/web';
export class Connector {
async fetchCredentials() {
// Implement fetchCredentials to obtain a JWT from your authentication service.
// See https://docs.powersync.com/configuration/auth/overview
return {
endpoint: '[Your PowerSync instance URL or self-hosted endpoint]',
// Use a development token (see Authentication Setup https://docs.powersync.com/configuration/auth/development-tokens) to get up and running quickly
token: 'An authentication token'
};
}
async uploadData(database) {
// Implement uploadData to send local changes to your backend service.
// You can omit this method if you only want to sync data from the database to the client
// See example implementation here: https://docs.powersync.com/client-sdks/usage-examples#send-changes-in-local-data-to-your-backend-service
}
}
```
### 4. Subscribe to Sync Streams
Streams defined with `auto_subscribe: true` start syncing as soon as the client connects. For all other streams, your app must subscribe before their data downloads. The basic pattern is: subscribe to a stream, wait for its data to sync, then unsubscribe when the data is no longer needed.
```js theme={null}
// Subscribe to a stream with parameters
const sub = await db.syncStream('list_todos', { list_id: 'abc123' }).subscribe();
// Wait for the initial data to sync
await sub.waitForFirstSync();
// The stream's rows are now in the local SQLite database.
// TODO: Read the todos for this list with a local query.
// When the data is no longer needed
sub.unsubscribe();
```
After you unsubscribe, the synced data stays in the local database for the stream's time-to-live (TTL), which is 24 hours by default. If the app subscribes again within that time, the data is already available. See [Client-Side Usage](/sync/streams/client-usage) for framework hooks, per-subscription sync status, custom TTLs, priority overrides, and connection parameters.
## Using PowerSync: CRUD Functions
Once the PowerSync database is connected and your streams have synced, the data is in the local SQLite database.
**All CRUD examples from the JavaScript Web SDK apply**: The Capacitor SDK uses the same API as the Web SDK. See the [JavaScript Web SDK CRUD functions section](/client-sdks/reference/javascript-web#using-powersync-crud-functions) for examples of `get`, `getAll`, `watch`, `execute`, `writeTransaction`, incremental watch updates, and differential results.
The most commonly used CRUD functions to interact with your SQLite data are:
* [PowerSyncDatabase.get](/client-sdks/reference/javascript-web#fetching-a-single-item) - get (`SELECT`) a single row from a table.
* [PowerSyncDatabase.getAll](/client-sdks/reference/javascript-web#querying-items-powersync-getall) - get (`SELECT`) a set of rows from a table.
* [PowerSyncDatabase.watch](/client-sdks/reference/javascript-web#watching-queries-powersync-watch) - execute a read query every time a dependent table changes.
* [PowerSyncDatabase.execute](/client-sdks/reference/javascript-web#mutations-powersync-execute-powersync-writetransaction) - execute a write (`INSERT`/`UPDATE`/`DELETE`) query.
### Fetching a Single Item
The [get](https://powersync-ja.github.io/powersync-js/common/interfaces/CommonPowerSyncDatabase#get) method executes a read-only (SELECT) query and returns a single result. It throws an exception if no result is found. Use [getOptional](https://powersync-ja.github.io/powersync-js/common/interfaces/CommonPowerSyncDatabase#getoptional) to return a single optional result (returns `null` if no result is found).
```js theme={null}
// Find a list item by ID
export const findList = async (id) => {
const result = await db.get('SELECT * FROM lists WHERE id = ?', [id]);
return result;
}
```
### Querying Items (PowerSync.getAll)
The [getAll](https://powersync-ja.github.io/powersync-js/common/interfaces/CommonPowerSyncDatabase#getall) method returns a set of rows from a table.
```js theme={null}
// Get all lists
export const getLists = async () => {
const results = await db.getAll('SELECT * FROM lists');
return results;
}
```
### Watching Queries (PowerSync.watch)
The [watch](https://powersync-ja.github.io/powersync-js/common/interfaces/CommonPowerSyncDatabase#watch) method executes a read query whenever a change to a dependent table is made.
```javascript theme={null}
async function* pendingLists(): AsyncIterable {
for await (const result of db.watch(
`SELECT * FROM lists WHERE state = ?`,
['pending']
)) {
yield result.rows?._array ?? [];
}
}
```
```javascript theme={null}
const pendingLists = (onResult: (lists: any[]) => void): void => {
db.watch(
'SELECT * FROM lists WHERE state = ?',
['pending'],
{
onResult: (result: any) => {
onResult(result.rows?._array ?? []);
}
}
);
}
```
For advanced watch query features such as incremental updates and differential results, see [Live Queries / Watch Queries](/client-sdks/watch-queries).
### Mutations (PowerSync.execute, PowerSync.writeTransaction)
The [execute](https://powersync-ja.github.io/powersync-js/common/interfaces/CommonPowerSyncDatabase#execute) method can be used for executing single SQLite write statements.
```js theme={null}
// Delete a list by ID
export const deleteList = async (id) => {
await db.execute('DELETE FROM lists WHERE id = ?', [id]);
};
// OR: delete the list and its todos in one transaction
export const deleteListWithTodos = async (id) => {
await db.writeTransaction(async (tx) => {
await tx.execute('DELETE FROM todos WHERE list_id = ?', [id]);
await tx.execute('DELETE FROM lists WHERE id = ?', [id]);
});
};
```
When using the default client-side [JSON-based view system](/architecture/client-architecture#client-side-schema-and-sqlite-database-structure), writes are applied to a view, with triggers writing to the underlying table. Because of this, `result.rowsAffected` from `db.execute()` can be `0` even when an `UPDATE` or `DELETE` succeeds.
When you need to confirm whether a mutation changed any rows, add a `RETURNING` clause and check the returned rows:
```js theme={null}
const result = await db.execute(
'UPDATE tasks SET deleted_at = ? WHERE id = ? AND deleted_at IS NULL RETURNING id',
[now, id]
);
const wasUpdated = (result.rows?.length ?? 0) > 0;
```
If you need direct table writes, use [raw tables](/client-sdks/advanced/raw-tables).
## Configure Logging
```js theme={null}
import { createConsoleLogger, LogLevels } from '@powersync/web';
// Create a logger with trace minimum level to see all log messages
// Available levels: trace, debug, info, warn, error
const logger = createConsoleLogger({ minLevel: LogLevels.trace });
```
Enable verbose output in the developer tools for detailed logs.
## Limitations
* Encryption for native mobile platforms is not yet supported.
* Multiple tab support is not available for native Android and iOS targets.
* `PowerSyncDatabase.executeRaw` does not support results where multiple columns have the same name in SQLite.
* `PowerSyncDatabase.execute` has limited support on Android. The SQLCipher Android driver exposes queries and executions as separate APIs, so there is no single method that handles both. On Android, the SDK treats a statement as a query only when the SQL starts with `select` (case-insensitive).
## Additional Usage Examples
For more usage examples including accessing connection status, monitoring sync progress, and waiting for initial sync, see the [Usage Examples](/client-sdks/usage-examples) page.
## ORM Support
See [JavaScript ORM Support](/client-sdks/orms/js/overview) for details.
## Troubleshooting
See [Troubleshooting](/debugging/troubleshooting) for pointers to debug common issues.
## Supported Platforms
See [Supported Platforms -> Capacitor SDK](/resources/supported-platforms#capacitor).
## Upgrading the SDK
Run the following command in your project folder:
```bash theme={null}
npm upgrade @powersync/capacitor @powersync/web
```
```bash theme={null}
yarn upgrade @powersync/capacitor @powersync/web
```
```bash theme={null}
pnpm upgrade @powersync/capacitor @powersync/web
```
# Capacitor SDK API Reference
Source: https://docs.powersync.com/client-sdks/reference/capacitor-api
Full API reference for the PowerSync Capacitor SDK.
# .NET SDK (beta)
Source: https://docs.powersync.com/client-sdks/reference/dotnet
Use PowerSync in .NET apps (beta).
```text Build with AI icon="sparkles" wrap theme={null}
Install the PowerSync Agent Skills with: npx skills add powersync-ja/agent-skills. Then follow the skills to onboard this project to PowerSync using the .NET SDK.
```
This SDK is distributed via NuGet
Refer to the `powersync-dotnet` repo on GitHub
Full API reference for the SDK
Gallery of example projects/demo apps built with .NET PowerSync
Changelog for the SDK
This SDK is currently in a [**beta** release](/resources/feature-status). It is production-ready for tested use cases. APIs are stable, and we communicate breaking changes clearly.
## Supported Frameworks and Targets
The PowerSync .NET SDK supports:
* **.NET Versions**: 6, 8, and 9
* **.NET Standard**: 2.0 (for compatibility with older libraries and frameworks)
* **.NET Framework**: Version 4.8, which requires additional configuration (see the package [README](https://github.com/powersync-ja/powersync-dotnet/tree/main?tab=readme-ov-file))
* **MAUI**: Cross-platform support for Android, iOS, Mac Catalyst, and Windows (targeting `net8.0` and `net9.0` mobile frameworks)
* **WPF**: Windows desktop applications
* **Console/CLI**: Windows (x64, ARM), macOS (x64, ARM), and Linux (x64, ARM)
**Current limitations:**
* Blazor (web) platforms are not yet supported.
For more details, see the package [README](https://github.com/powersync-ja/powersync-dotnet/tree/main?tab=readme-ov-file).
## SDK Features
* **Real-time streaming of database changes**: Changes made by one user are instantly streamed to all other users with access to that data. This keeps clients automatically in sync without manual polling or refresh logic.
* **Direct access to a local SQLite database**: Data is stored locally, so apps can read and write instantly without network calls. This enables offline support and faster user interactions.
* **Asynchronous background execution**: The SDK performs database operations in the background to avoid blocking the application’s main thread. This means that apps stay responsive, even during heavy data activity.
* **Query subscriptions for live updates**: The SDK supports query subscriptions that automatically push real-time updates to client applications as data changes, keeping your UI reactive and up to date.
* **Automatic schema management**: PowerSync syncs schemaless data and applies a client-defined schema using SQLite views. This architecture means that PowerSync SDKs handle schema changes without explicit migrations on the client side.
## Quickstart
For desktop/server/binary use-cases and WPF, add the [`PowerSync.Common`](https://www.nuget.org/packages/PowerSync.Common/) NuGet package to your project:
```bash theme={null}
dotnet add package PowerSync.Common
```
For MAUI apps, add both [`PowerSync.Common`](https://www.nuget.org/packages/PowerSync.Common/) and [`PowerSync.Maui`](https://www.nuget.org/packages/PowerSync.Maui/) NuGet packages to your project:
```bash theme={null}
dotnet add package PowerSync.Common
dotnet add package PowerSync.Maui
```
To install a specific version, use `--version` instead: `dotnet add package PowerSync.Common --version `
**Prerequisites:** Before you start, connect your source database to the PowerSync Service and deploy Sync Streams. These are steps 1-4 in the [Setup Guide](/intro/setup-guide).
### 1. Define the Client-Side Schema
The client-side schema defines the tables and columns of the SQLite database that the PowerSync client SDK manages and that your app reads from and writes to. It is usually derived from your backend database schema and your [Sync Streams](/sync/streams/overview), and it can also include [local-only tables](/client-sdks/advanced/local-only-usage). You apply the schema when you instantiate the database in the next step.
Schema migrations are not required. The SDK syncs schemaless data and applies the schema to that data with SQLite views. The exception is [raw tables](/client-sdks/advanced/raw-tables), which you create and migrate yourself.
**Generate schema automatically**
In the [PowerSync Dashboard](https://dashboard.powersync.com/), select your project and instance and click the **Connect** button in the top bar to generate the client-side schema in your preferred language. The schema is generated from your Sync Streams. The [CLI](/tools/cli) offers the same function.
The generated schema does not include an `id` column. The client SDK creates an `id` column of type `text` automatically, so you do not need to declare it. See [Client ID](/sync/advanced/client-id) for details.
You can use [this example](https://github.com/powersync-ja/powersync-dotnet/blob/main/demos/CommandLine/AppSchema.cs) as a reference when defining your schema.
The available column types are `text`, `integer`, and `real`. These should match the values produced by your Sync Streams. If a value does not match, it is cast automatically. For details on how source database types map to SQLite types, see [Types](/sync/types).
#### Schema Definition Syntax
There are two supported syntaxes for defining the schema:
**Attribute-based (recommended):** Annotate a C# class with [`[Table]`](https://powersync-ja.github.io/powersync-dotnet/api/PowerSync.Common.DB.Schema.Attributes.TableAttribute.html), [`[Column]`](https://powersync-ja.github.io/powersync-dotnet/api/PowerSync.Common.DB.Schema.Attributes.ColumnAttribute.html), and [`[Index]`](https://powersync-ja.github.io/powersync-dotnet/api/PowerSync.Common.DB.Schema.Attributes.IndexAttribute.html) attributes. The same class can then be used directly as the result type in queries, so you define your data structure once:
```cs theme={null}
using PowerSync.Common.DB.Schema;
using PowerSync.Common.DB.Schema.Attributes;
[Table("todos"), Index("list", ["list_id"])]
public class Todo
{
[Column("id")]
public string TodoId { get; set; }
[Column("list_id")]
public string ListId { get; set; }
[Column("created_at")]
public string CreatedAt { get; set; }
[Column("completed")]
public bool Completed { get; set; }
// ... other columns
}
public static Schema PowerSyncSchema = new Schema(typeof(Todo));
// The same Todo class is used for queries:
var todos = await db.GetAll("SELECT * FROM todos");
```
Unlike the other syntaxes where PowerSync automatically creates an `id` column, the attribute-based syntax requires you to explicitly declare it. The SDK identifies the `id` property by looking for either a property named `id`, or any property with a `[Column("id")]` attribute (case-insensitive). Having none or more than one is an error.
If you prefer to keep your schema definition separate from your data classes, you can use the object initializer syntax instead:
```cs theme={null}
using PowerSync.Common.DB.Schema;
class AppSchema
{
public static Table Todos = new Table
{
Name = "todos",
Columns =
{
["list_id"] = ColumnType.Text,
["created_at"] = ColumnType.Text,
["completed"] = ColumnType.Integer,
// ... other columns
},
Indexes =
{
["list"] = ["list_id"]
}
};
public static Schema PowerSyncSchema = new Schema(Todos);
}
```
### 2. Instantiate the PowerSync Database
Next, instantiate the PowerSync database. PowerSync streams changes from your backend source database into the client-side SQLite database, based on your Sync Streams. Your app reads from and writes to this local database whether the user is online or offline.
**Example:**
The initialization syntax differs slightly between the Common and MAUI SDKs:
```cs theme={null}
using PowerSync.Common.Client;
class Demo
{
static async Task Main()
{
var db = new PowerSyncDatabase(new PowerSyncDatabaseOptions
{
Database = new SQLOpenOptions { DbFilename = "tododemo.db" },
Schema = AppSchema.PowerSyncSchema,
});
await db.Init();
}
}
```
```cs theme={null}
using PowerSync.Common.Client;
using PowerSync.Common.MDSQLite;
using PowerSync.Maui.SQLite;
class Demo
{
static async Task Main()
{
// Ensures the DB file is stored in a platform appropriate location
var dbPath = Path.Combine(FileSystem.AppDataDirectory, "maui-example.db");
var factory = new MAUISQLiteDBOpenFactory(new MDSQLiteOpenFactoryOptions()
{
DbFilename = dbPath
});
var db = new PowerSyncDatabase(new PowerSyncDatabaseOptions()
{
Database = factory, // Supply a factory
Schema = AppSchema.PowerSyncSchema,
});
await db.Init();
}
}
```
### 3. Integrate with Your Backend
The backend connector connects the PowerSync client SDK to your application backend. The SDK uses it to:
1. Get an auth token to connect to the PowerSync instance.
2. Upload client-side writes to your backend API. The SDK places every write to the SQLite database in an upload queue and uploads the queue to your backend when the user is connected. Your backend then applies the changes to the source database.
The connector must implement two methods:
1. [IPowerSyncBackendConnector.FetchCredentials](https://powersync-ja.github.io/powersync-dotnet/api/PowerSync.Common.Client.Connection.IPowerSyncBackendConnector.FetchCredentials.html) - The SDK calls this method to get authentication credentials. It caches the credentials and calls the method again only when needed, for example on the first connection or when the token is near expiry. See [When `fetchCredentials()` is Called](/configuration/app-backend/client-side-integration#when-fetchcredentials-is-called) for details and [Authentication Setup](/configuration/auth/overview) for how to generate credentials.
2. [IPowerSyncBackendConnector.UploadData](https://powersync-ja.github.io/powersync-dotnet/api/PowerSync.Common.Client.Connection.IPowerSyncBackendConnector.UploadData.html) - The SDK calls this method whenever it has client-side writes to upload to your backend API. Implement how those writes are processed and uploaded. See [When `uploadData()` is Called](/configuration/app-backend/client-side-integration#when-uploaddata-is-called) for triggers, throttling, and retry behavior, and [Writing Client Changes](/handling-writes/writing-client-changes) for the app backend implementation.
**Example:**
```cs theme={null}
using System;
using System.Collections.Generic;
using System.Net.Http;
using System.Text;
using System.Text.Json;
using System.Threading.Tasks;
using PowerSync.Common.Client;
using PowerSync.Common.Client.Connection;
using PowerSync.Common.DB.Crud;
public class MyConnector : IPowerSyncBackendConnector
{
private readonly HttpClient _httpClient;
// User credentials for the current session
public string UserId { get; private set; }
// Service endpoints
private readonly string _backendUrl;
private readonly string _powerSyncUrl;
private string? _clientId;
public MyConnector()
{
_httpClient = new HttpClient();
// In a real app, this would come from your authentication system
UserId = "user-123";
// Configure your service endpoints
_backendUrl = "[Your backend API URL]";
_powerSyncUrl = "https://your-powersync-instance.powersync.journeyapps.com";
}
public async Task FetchCredentials()
{
try {
// Implement fetchCredentials to obtain a JWT from your authentication service.
// See https://docs.powersync.com/configuration/auth/overview
var authToken = "your-auth-token"; // Use a development token (see Authentication Setup https://docs.powersync.com/configuration/auth/development-tokens) to get up and running quickly
// Return credentials with PowerSync endpoint and JWT token
return new PowerSyncCredentials(_powerSyncUrl, authToken);
}
catch (Exception ex)
{
Console.WriteLine($"Error fetching credentials: {ex.Message}");
throw;
}
}
public async Task UploadData(IPowerSyncDatabase database)
{
// Get the next transaction to upload
CrudTransaction? transaction;
try
{
transaction = await database.GetNextCrudTransaction();
}
catch (Exception ex)
{
Console.WriteLine($"UploadData Error: {ex.Message}");
return;
}
// If there's no transaction, there's nothing to upload
if (transaction == null)
{
return;
}
// Get client ID if not already retrieved
_clientId ??= await database.GetClientId();
try
{
// Convert PowerSync operations to your backend format
var batch = new List();
foreach (var operation in transaction.Crud)
{
batch.Add(new
{
op = operation.Op.ToString(), // INSERT, UPDATE, DELETE
table = operation.Table,
id = operation.Id,
data = operation.OpData
});
}
// Send the operations to your backend
var payload = JsonSerializer.Serialize(new { batch });
var content = new StringContent(payload, Encoding.UTF8, "application/json");
HttpResponseMessage response = await _httpClient.PostAsync($"{_backendUrl}/api/data", content);
response.EnsureSuccessStatusCode();
// Mark the transaction as completed
await transaction.Complete();
}
catch (Exception ex)
{
Console.WriteLine($"UploadData Error: {ex.Message}");
throw;
}
}
}
```
With your database instantiated and your connector ready, call [`Connect`](https://powersync-ja.github.io/powersync-dotnet/api/PowerSync.Common.Client.PowerSyncDatabase.Connect.html) to start syncing data with your backend:
```cs theme={null}
await db.Connect(new MyConnector());
await db.WaitForFirstSync(); // Optional, to wait for a complete snapshot of data to be available
```
This section assumes that you use PowerSync to sync your backend source database with SQLite in your app. To manage a local SQLite database without sync, instantiate the PowerSync database without calling `connect()` and see the [Local-Only](/client-sdks/advanced/local-only-usage) guide.
### 4. Subscribe to Sync Streams
Streams defined with `auto_subscribe: true` start syncing as soon as the client connects. For all other streams, your app must subscribe before their data downloads. The basic pattern is: subscribe to a stream, wait for its data to sync, then unsubscribe when the data is no longer needed.
```csharp theme={null}
// Subscribe to a stream with parameters
var sub = await db.SyncStream("list_todos", new() { ["list_id"] = "abc123" }).Subscribe();
// Wait for the initial data to sync
await sub.WaitForFirstSync();
// The stream's rows are now in the local SQLite database.
// TODO: Read the todos for this list with a local query.
// When the data is no longer needed
sub.Unsubscribe();
```
After you unsubscribe, the synced data stays in the local database for the stream's time-to-live (TTL), which is 24 hours by default. If the app subscribes again within that time, the data is already available. See [Client-Side Usage](/sync/streams/client-usage) for framework hooks, per-subscription sync status, custom TTLs, priority overrides, and connection parameters.
## Using PowerSync: CRUD Functions
Once the PowerSync database is connected and your streams have synced, the data is in the local SQLite database.
The most commonly used CRUD functions to interact with your SQLite data are:
* [`PowerSyncDatabase.Get`](https://powersync-ja.github.io/powersync-dotnet/api/PowerSync.Common.Client.PowerSyncDatabase.Get.html) - get (`SELECT`) a single row from a table.
* [`PowerSyncDatabase.GetAll`](https://powersync-ja.github.io/powersync-dotnet/api/PowerSync.Common.Client.PowerSyncDatabase.GetAll.html) - get (`SELECT`) a set of rows from a table.
* [`PowerSyncDatabase.Watch`](https://powersync-ja.github.io/powersync-dotnet/api/PowerSync.Common.Client.PowerSyncDatabase.Watch.html) - execute a read query every time a dependent table changes.
* [`PowerSyncDatabase.Execute`](https://powersync-ja.github.io/powersync-dotnet/api/PowerSync.Common.Client.PowerSyncDatabase.Execute.html) - execute a write (`INSERT`/`UPDATE`/`DELETE`) query.
### Fetching a Single Item
The `Get` method executes a read-only (SELECT) query and returns a single result. It throws an exception if no result is found. Use `GetOptional` to return a single optional result (returns `null` if no result is found).
```cs theme={null}
// Define a result type with properties matching the schema columns (some columns omitted here for brevity)
// public class ListResult { public string id; public string name; public string owner_id; ... }
var list = await db.Get("SELECT * FROM lists WHERE id = ?", [listId]);
```
### Querying Items (PowerSync.GetAll)
The `GetAll` method returns a set of rows from a table.
```cs theme={null}
// Define a result type with properties matching the schema columns (some columns omitted here for brevity)
// public class ListResult { public string id; public string name; public string owner_id; ... }
var lists = await db.GetAll("SELECT * FROM lists");
```
### Watching Queries (PowerSync.Watch)
The `Watch` method executes a read query whenever a change to a dependent table is made. It returns an `IAsyncEnumerable` so you can use `await foreach` to consume results.
```csharp theme={null}
// Define a result type with properties matching the schema columns (some columns omitted here for brevity)
// public class ListResult { public string id; public string name; public string owner_id; ... }
// Optional cancellation token to stop watching
var cts = new CancellationTokenSource();
// Register listener synchronously on the calling thread...
var listener = db.Watch(
"SELECT * FROM lists WHERE owner_id = ?",
[ownerId],
new SQLWatchOptions { Signal = cts.Token }
);
// ...then listen to changes on another thread (or await foreach directly if already in an async context)
_ = Task.Run(async () =>
{
await foreach (var results in listener)
{
Console.WriteLine("Lists: ");
foreach (var result in results)
{
Console.WriteLine($"{result.id}: {result.name}");
}
}
}, cts.Token);
// To stop watching, cancel the token: cts.Cancel();
```
### Mutations (PowerSync.Execute)
The `Execute` method can be used for executing single SQLite write statements.
```cs theme={null}
// And db.Execute for inserts, updates and deletes:
await db.Execute(
"INSERT INTO lists (id, name, owner_id, created_at) VALUES (uuid(), 'New list', ?, datetime())",
[connector.UserId]
);
```
## Configure Logging
By default, the SDK uses a no-op logger that outputs nothing. To enable logging, configure a logger with .NET's `ILogger` interface:
```cs theme={null}
using Microsoft.Extensions.Logging;
using PowerSync.Common.Client;
// Create a logger factory
ILoggerFactory loggerFactory = LoggerFactory.Create(builder =>
{
builder.AddConsole(); // Enable console logging
builder.SetMinimumLevel(LogLevel.Information); // Set minimum log level
});
var logger = loggerFactory.CreateLogger("PowerSyncLogger");
var db = new PowerSyncDatabase(new PowerSyncDatabaseOptions
{
Database = new SQLOpenOptions { DbFilename = "powersync.db" },
Schema = AppSchema.PowerSyncSchema,
Logger = logger
});
```
## Additional Usage Examples
For more usage examples including accessing connection status, monitoring sync progress, and waiting for initial sync, see the [Usage Examples](/client-sdks/usage-examples) page.
## Troubleshooting
See [Troubleshooting](/debugging/troubleshooting) for pointers to debug common issues.
## Supported Platforms
See [Supported Platforms -> .NET SDK](/resources/supported-platforms#net).
## Upgrading the SDK
To upgrade the PowerSync package, run the following command in your project folder:
```bash theme={null}
dotnet add package PowerSync.Common
```
```bash theme={null}
dotnet add package PowerSync.Common
dotnet add package PowerSync.Maui
```
# .NET SDK API Reference
Source: https://docs.powersync.com/client-sdks/reference/dotnet-api
Full API reference for the PowerSync .NET SDK.
# Dart/Flutter SDK
Source: https://docs.powersync.com/client-sdks/reference/flutter
Use PowerSync in Dart and Flutter apps.
```text Build with AI icon="sparkles" wrap theme={null}
Install the PowerSync Agent Skills with: npx skills add powersync-ja/agent-skills. Then follow the skills to onboard this project to PowerSync using the Dart/Flutter SDK.
```
The SDK is distributed via pub.dev
Refer to the `powersync.dart` repo on GitHub
Full API reference for the SDK
Gallery of example projects/demo apps built with Flutter and PowerSync
Changelog for the SDK
## Quickstart
To start from a template, use the self-hosted Flutter and Supabase template: [flutter-powersync-supabase](https://github.com/powersync-community/flutter-powersync-supabase).
## SDK Features
* **Real-time streaming of database changes**: Changes made by one user are instantly streamed to all other users with access to that data. This keeps clients automatically in sync without manual polling or refresh logic.
* **Direct access to a local SQLite database**: Data is stored locally, so apps can read and write instantly without network calls. This enables offline support and faster user interactions.
* **Asynchronous background execution**: The SDK performs database operations in the background to avoid blocking the application’s main thread. This means that apps stay responsive, even during heavy data activity.
* **Query subscriptions for live updates**: The SDK supports query subscriptions that automatically push real-time updates to client applications as data changes, keeping your UI reactive and up to date.
* **Automatic schema management**: PowerSync syncs schemaless data and applies a client-defined schema using SQLite views. This architecture means that PowerSync SDKs handle schema changes without explicit migrations on the client side.
Web support is currently in a beta release. Refer to [Flutter Web Support](/client-sdks/frameworks/flutter-web-support) for more details.
## Installation
Add the [PowerSync pub.dev package](https://pub.dev/packages/powersync) to your project:
```bash theme={null}
dart pub add powersync
```
## Getting Started
**Prerequisites:** Before you start, connect your source database to the PowerSync Service and deploy Sync Streams. These are steps 1-4 in the [Setup Guide](/intro/setup-guide).
This reference assumes a Flutter project with the following directory structure:
```plaintext theme={null}
lib/
├── models/
├── schema.dart
└── todolist.dart
├── powersync/
├── my_backend_connector.dart
└── powersync.dart
├── widgets/
├── lists_widget.dart
├── todos_widget.dart
├── main.dart
```
### 1. Define the Client-Side Schema
The client-side schema defines the tables and columns of the SQLite database that the PowerSync client SDK manages and that your app reads from and writes to. It is usually derived from your backend database schema and your [Sync Streams](/sync/streams/overview), and it can also include [local-only tables](/client-sdks/advanced/local-only-usage). You apply the schema when you instantiate the database in the next step.
Schema migrations are not required. The SDK syncs schemaless data and applies the schema to that data with SQLite views. The exception is [raw tables](/client-sdks/advanced/raw-tables), which you create and migrate yourself.
**Generate schema automatically**
In the [PowerSync Dashboard](https://dashboard.powersync.com/), select your project and instance and click the **Connect** button in the top bar to generate the client-side schema in your preferred language. The schema is generated from your Sync Streams. The [CLI](/tools/cli) offers the same function.
The generated schema does not include an `id` column. The client SDK creates an `id` column of type `text` automatically, so you do not need to declare it. See [Client ID](/sync/advanced/client-id) for details.
The available column types are `text`, `integer`, and `real`. These should match the values produced by your Sync Streams. If a value does not match, it is cast automatically. For details on how source database types map to SQLite types, see [Types](/sync/types).
**Example:**
```dart lib/models/schema.dart theme={null}
import 'package:powersync/powersync.dart';
const schema = Schema(([
Table('todos', [
Column.text('list_id'),
Column.text('created_at'),
Column.text('completed_at'),
Column.text('description'),
Column.integer('completed'),
Column.text('created_by'),
Column.text('completed_by'),
], indexes: [
// Index to allow efficient lookup within a list
Index('list', [IndexedColumn('list_id')])
]),
Table('lists', [
Column.text('created_at'),
Column.text('name'),
Column.text('owner_id')
])
]));
```
You do not need to declare an `id` column. PowerSync creates it automatically.
### 2. Instantiate the PowerSync Database
Next, instantiate the PowerSync database. PowerSync streams changes from your backend source database into the client-side SQLite database, based on your Sync Streams. Your app reads from and writes to this local database whether the user is online or offline.
To instantiate `PowerSyncDatabase`, pass the schema you defined in the previous step and a file path. Create only one `PowerSyncDatabase` instance per file.
**Example:**
```dart lib/powersync/powersync.dart theme={null}
import 'package:path/path.dart';
import 'package:path_provider/path_provider.dart';
import 'package:powersync/powersync.dart';
import '../models/schema.dart';
// TODO: Use riverpod, providers or another state management
// approach instead of a global variable to store the database.
late PowerSyncDatabase db;
Future openDatabase() async {
final dir = await getApplicationSupportDirectory();
final path = join(dir.path, 'powersync-dart.db');
// Set up the database
// Inject the Schema you defined in the previous step and a file path
db = PowerSyncDatabase(schema: schema, path: path);
await db.initialize();
}
```
After you instantiate the PowerSync database, call the [connect()](https://pub.dev/documentation/powersync/latest/powersync/PowerSyncDatabase/connect.html) method to sync data with your backend. This method requires the backend connector that you create in the next step.
This section assumes that you use PowerSync to sync your backend source database with SQLite in your app. To manage a local SQLite database without sync, instantiate the PowerSync database without calling `connect()` and see the [Local-Only](/client-sdks/advanced/local-only-usage) guide.
```dart lib/main.dart {26} theme={null}
import 'package:flutter/material.dart';
import 'package:powersync/powersync.dart';
import 'powersync/powersync.dart';
Future main() async {
WidgetsFlutterBinding.ensureInitialized();
await openDatabase();
runApp(const DemoApp());
}
class DemoApp extends StatefulWidget {
const DemoApp({super.key});
@override
State createState() => _DemoAppState();
}
class _DemoAppState extends State {
@override
void initState() {
super.initState();
// TODO: Observe a condition to connect / disconnect.
db.connect(connector: MyBackendConnector());
}
@override
Widget build(BuildContext context) {
return MaterialApp(
title: 'Demo',
// TODO: Implement your own UI here.
);
}
}
```
### 3. Integrate with Your Backend
The backend connector connects the PowerSync client SDK to your application backend. The SDK uses it to:
1. Get an auth token to connect to the PowerSync instance.
2. Upload client-side writes to your backend API. The SDK places every write to the SQLite database in an upload queue and uploads the queue to your backend when the user is connected. Your backend then applies the changes to the source database.
The connector must implement two methods:
1. [PowerSyncBackendConnector.fetchCredentials](https://pub.dev/documentation/powersync/latest/powersync/PowerSyncBackendConnector/fetchCredentials.html) - The SDK calls this method to get authentication credentials. It caches the credentials and calls the method again only when needed, for example on the first connection or when the token is near expiry. See [When `fetchCredentials()` is Called](/configuration/app-backend/client-side-integration#when-fetchcredentials-is-called) for details and [Authentication Setup](/configuration/auth/overview) for how to generate credentials.
2. [PowerSyncBackendConnector.uploadData](https://pub.dev/documentation/powersync/latest/powersync/PowerSyncBackendConnector/uploadData.html) - The SDK calls this method whenever it has client-side writes to upload to your backend API. Implement how those writes are processed and uploaded. See [When `uploadData()` is Called](/configuration/app-backend/client-side-integration#when-uploaddata-is-called) for triggers, throttling, and retry behavior, and [Writing Client Changes](/handling-writes/writing-client-changes) for the app backend implementation.
**Example:**
```dart lib/powersync/my_backend_connector.dart theme={null}
import 'package:powersync/powersync.dart';
class MyBackendConnector extends PowerSyncBackendConnector {
PowerSyncDatabase db;
MyBackendConnector(this.db);
@override
Future fetchCredentials() async {
// Implement fetchCredentials to obtain a JWT from your authentication service.
// See https://docs.powersync.com/configuration/auth/overview
// See example implementation here: https://pub.dev/documentation/powersync/latest/powersync/DevConnector/fetchCredentials.html
return PowerSyncCredentials(
endpoint: 'https://xxxxxx.powersync.journeyapps.com',
// Use a development token (see Authentication Setup https://docs.powersync.com/configuration/auth/development-tokens) to get up and running quickly
token: 'An authentication token'
);
}
// Implement uploadData to send local changes to your backend service
// You can omit this method if you only want to sync data from the server to the client
// See example implementation here: https://docs.powersync.com/client-sdks/usage-examples#send-changes-in-local-data-to-your-backend-service
@override
Future uploadData(PowerSyncDatabase database) async {
// This function is called whenever there is data to upload, whether the
// device is online or offline.
// If this call throws an error, it is retried periodically.
final transaction = await database.getNextCrudTransaction();
if (transaction == null) {
return;
}
// The data that needs to be changed in the remote db
for (var op in transaction.crud) {
switch (op.op) {
case UpdateType.put:
// TODO: Instruct your backend API to CREATE a record
case UpdateType.patch:
// TODO: Instruct your backend API to PATCH a record
case UpdateType.delete:
//TODO: Instruct your backend API to DELETE a record
}
}
// Completes the transaction and moves onto the next one
await transaction.complete();
}
}
```
### 4. Subscribe to Sync Streams
Streams defined with `auto_subscribe: true` start syncing as soon as the client connects. For all other streams, your app must subscribe before their data downloads. The basic pattern is: subscribe to a stream, wait for its data to sync, then unsubscribe when the data is no longer needed.
```dart theme={null}
// Subscribe to a stream with parameters
final sub = await db.syncStream('list_todos', {'list_id': 'abc123'}).subscribe();
// Wait for the initial data to sync
await sub.waitForFirstSync();
// The stream's rows are now in the local SQLite database.
// TODO: Read the todos for this list with a local query.
// When the data is no longer needed
sub.unsubscribe();
```
After you unsubscribe, the synced data stays in the local database for the stream's time-to-live (TTL), which is 24 hours by default. If the app subscribes again within that time, the data is already available. See [Client-Side Usage](/sync/streams/client-usage) for framework hooks, per-subscription sync status, custom TTLs, priority overrides, and connection parameters.
## Using PowerSync: CRUD Functions
Once the PowerSync database is connected and your streams have synced, the data is in the local SQLite database.
The most commonly used CRUD functions to interact with your SQLite data are:
* [PowerSyncDatabase.get](/client-sdks/reference/flutter#fetching-a-single-item) - get (`SELECT`) a single row from a table.
* [PowerSyncDatabase.getAll](/client-sdks/reference/flutter#querying-items-powersync-getall) - get (`SELECT`) a set of rows from a table.
* [PowerSyncDatabase.watch](/client-sdks/reference/flutter#watching-queries-powersync-watch) - execute a read query every time a dependent table changes.
* [PowerSyncDatabase.execute](/client-sdks/reference/flutter#mutations-powersync-execute) - execute a write (`INSERT`/`UPDATE`/`DELETE`) query.
The following examples use this `TodoList` model class:
```dart lib/models/todolist.dart theme={null}
/// A model class representing a row in the lists table
class TodoList {
final String id;
final String name;
final DateTime createdAt;
final String ownerId;
TodoList({
required this.id,
required this.name,
required this.createdAt,
required this.ownerId,
});
factory TodoList.fromRow(Map row) {
return TodoList(
id: row['id'],
name: row['name'],
createdAt: DateTime.parse(row['created_at']),
ownerId: row['owner_id'],
);
}
}
```
### Fetching a Single Item
The [get](https://pub.dev/documentation/powersync/latest/sqlite_async/SqliteConnection/get.html) method executes a read-only (SELECT) query and returns a single result. It throws an exception if no result is found. Use [getOptional](https://pub.dev/documentation/powersync/latest/sqlite_async/SqliteConnection/getOptional.html) to return a single optional result (returns `null` if no result is found).
The following example selects a list by ID:
```dart lib/widgets/lists_widget.dart theme={null}
import '../powersync/powersync.dart';
import '../models/todolist.dart';
Future find(id) async {
final result = await db.get('SELECT * FROM lists WHERE id = ?', [id]);
return TodoList.fromRow(result);
}
```
### Querying Items (PowerSync.getAll)
The [getAll](https://pub.dev/documentation/powersync/latest/sqlite_async/SqliteConnection/getAll.html) method returns a set of rows from a table.
```dart lib/widgets/lists_widget.dart theme={null}
import 'package:powersync/sqlite3.dart';
import '../powersync/powersync.dart';
Future> getLists() async {
ResultSet results = await db.getAll('SELECT id FROM lists WHERE id IS NOT NULL');
List ids = results.map((row) => row['id'] as String).toList();
return ids;
}
```
### Watching Queries (PowerSync.watch)
The [watch](https://pub.dev/documentation/powersync/latest/powersync/PowerSyncDatabase/watch.html) method executes a read query whenever a change to a dependent table is made.
```dart theme={null}
StreamBuilder(
stream: db.watch('SELECT * FROM lists WHERE state = ?', ['pending']),
builder: (context, snapshot) {
if (snapshot.hasData) {
// TODO: implement your own UI here based on the result set
return ...;
} else {
return const Center(child: CircularProgressIndicator());
}
},
)
```
### Mutations (PowerSync.execute)
The [execute](https://pub.dev/documentation/powersync/latest/sqlite_async/SqliteConnection/execute.html) method can be used for executing single SQLite write statements.
```dart lib/widgets/todos_widget.dart {12-15} theme={null}
import 'package:flutter/material.dart';
import '../powersync/powersync.dart';
// Example Todos widget
class TodosWidget extends StatelessWidget {
const TodosWidget({super.key});
@override
Widget build(BuildContext context) {
return FloatingActionButton(
onPressed: () async {
await db.execute(
'INSERT INTO lists(id, created_at, name, owner_id) VALUES(uuid(), datetime(), ?, ?)',
['name', '123'],
);
},
tooltip: '+',
child: const Icon(Icons.add),
);
}
}
```
## Configure Logging
Logging is enabled by default and outputs logs from PowerSync to the console in debug mode.
To disable this, or to configure logging in release-mode configurations, use the `logger` parameter on the `PowerSyncDatabase` constructor.
PowerSync uses [`package:logging`](https://pub.dev/packages/logging) to emit logs. See that package for additional information.
## Custom HTTP Clients and Headers
PowerSync uses a streaming HTTP response to connect to the PowerSync Service. The SDK uses the default `Client()` from the
[http package](https://pub.dev/packages/http) for that, which relies on `HttpClient` from `dart:io` on native platforms and `fetch()`
on the web.
You can also supply a custom HTTP client. This can be used to enable a faster HTTP implementation like [`cronet_http`](https://pub.dev/packages/cronet_http),
or to add custom request headers that may be required if you run the PowerSync Service behind a reverse-proxy.
```dart theme={null}
import 'package:http/http.dart';
import 'package:powersync/powersync.dart';
final class _AddHeaderClient extends BaseClient {
final Client inner;
_AddHeaderClient([Client? inner]) : inner = inner ?? Client();
@override
Future send(BaseRequest request) {
request.headers['x-my-custom-header'] = 'set for all requests';
return inner.send(request);
}
@override
void close() => inner.close();
}
Future connect(PowerSyncDatabase db) async {
await db.connect(
connector: MyBackendConnector(db),
options: SyncOptions(
httpClient: _AddHeaderClient.new
),
);
}
```
On the web, PowerSync uses a shared worker for the sync process. As Dart objects cannot be shared between tabs and workers, the worker uses a
random tab as a proxy to send requests. This can slow down the sync process slightly.
## Additional Usage Examples
For more usage examples including accessing connection status, monitoring sync progress, and waiting for initial sync, see the [Usage Examples](/client-sdks/usage-examples) page.
## ORM Support
See [ORM Support](/client-sdks/orms/flutter-orm-support) for details.
## Troubleshooting
Use the [Dart & Flutter DevTools extension](/tools/dart-devtools-extension) to inspect live databases, run SQL queries, and view sync status without leaving your IDE. Available from `powersync` v2.1.0 in debug builds.
See [Troubleshooting](/debugging/troubleshooting) for pointers to debug common issues.
## Supported Platforms
See [Supported Platforms -> Dart SDK](/resources/supported-platforms#dart/flutter).
## Upgrading the SDK
To upgrade the PowerSync package, run the following command in your project folder:
```bash theme={null}
dart pub upgrade powersync
```
# Flutter SDK API Reference
Source: https://docs.powersync.com/client-sdks/reference/flutter-api
Full API reference for the PowerSync Dart/Flutter SDK.
# JavaScript Web SDK
Source: https://docs.powersync.com/client-sdks/reference/javascript-web
Build JavaScript web apps that use client-side SQLite for real-time updates across users and devices, instant interactions and continued functionality when offline.
The PowerSync JavaScript Web SDK runs SQLite in the browser and keeps it in sync with your backend database. Your UI reads and writes local data directly, so interactions feel instant because they do not wait on network round trips. Watch queries emit new results as local or synced data changes, keeping the UI current without polling.
With Sync Streams, you define the subset of backend data each client receives rather than replicating the entire database into every browser. This partial sync reduces data transfer and local storage use by keeping unrelated data out of the client database. Reading from a reactive local database also reduces the need for per-screen read APIs, cache invalidation, and client-side state plumbing. Synced data remains available even when connectivity is interrupted, enabling offline-first PWAs.
```text Build with AI icon="sparkles" wrap theme={null}
Install the PowerSync Agent Skills with: npx skills add powersync-ja/agent-skills. Then follow the skills to onboard this project to PowerSync using the JavaScript Web SDK.
```
This SDK is distributed via NPM
Refer to packages/web in the `powersync-js` repo on GitHub
Full API reference for the SDK
Gallery of example projects/demo apps built with JavaScript Web stacks and PowerSync
Changelog for the SDK
## SDK Features
* **Real-time streaming of database changes**: Changes made by one user are instantly streamed to all other users with access to that data. This keeps clients automatically in sync without manual polling or refresh logic.
* **Direct access to a local SQLite database**: Data is stored locally, so apps can read and write instantly without network calls. This enables offline support and faster user interactions.
* **Asynchronous background execution**: The SDK performs database operations in the background to avoid blocking the application’s main thread. This means that apps stay responsive, even during heavy data activity.
* **Query subscriptions for live updates**: The SDK supports query subscriptions that automatically push real-time updates to client applications as data changes, keeping your UI reactive and up to date.
* **Automatic schema management**: PowerSync syncs schemaless data and applies a client-defined schema using SQLite views. This architecture means that PowerSync SDKs handle schema changes without explicit migrations on the client side.
## Single-Page Application (SPA) Frameworks
The PowerSync JavaScript Web SDK is compatible with popular Single-Page Application (SPA) frameworks like React, Vue, Angular, and Svelte. Integration packages are provided specifically for the following:
Wrapper package to support reactivity and live queries.
Wrapper package to support reactivity and live queries.
PowerSync integrates with TanStack Query and TanStack DB for reactive data management.
For React or React Native apps:
* The [`@powersync/react`](/client-sdks/frameworks/react) package is best for most basic use cases, especially when you only need reactive queries with loading and error states.
* For more advanced scenarios, such as query caching and pagination, use [TanStack Query](/client-sdks/frameworks/tanstack#tanstack-query). The [`@powersync/tanstack-react-query`](/client-sdks/frameworks/tanstack#tanstack-query) package extends the `useQuery` hook from `@powersync/react` with functionality from [TanStack Query](https://tanstack.com/query/latest/docs/framework/react/overview).
* For reactive data management and live query support across multiple frameworks, consider [TanStack DB](/client-sdks/frameworks/tanstack#tanstack-db). PowerSync works with all TanStack DB framework adapters (React, Vue, Solid, Svelte, Angular).
If you have a Vue app, use the Vue-specific package: [`@powersync/vue`](/client-sdks/frameworks/vue).
## Installation
Add the [PowerSync Web NPM package](https://www.npmjs.com/package/@powersync/web) to your project:
```bash theme={null}
npm install @powersync/web
```
```bash theme={null}
yarn add @powersync/web
```
```bash theme={null}
pnpm install @powersync/web
```
## Getting Started
**Prerequisites:** Before you start, connect your source database to the PowerSync Service and deploy Sync Streams. These are steps 1-4 in the [Setup Guide](/intro/setup-guide).
### 1. Define the Client-Side Schema
The client-side schema defines the tables and columns of the SQLite database that the PowerSync client SDK manages and that your app reads from and writes to. It is usually derived from your backend database schema and your [Sync Streams](/sync/streams/overview), and it can also include [local-only tables](/client-sdks/advanced/local-only-usage). You apply the schema when you instantiate the database in the next step.
Schema migrations are not required. The SDK syncs schemaless data and applies the schema to that data with SQLite views. The exception is [raw tables](/client-sdks/advanced/raw-tables), which you create and migrate yourself.
**Generate schema automatically**
In the [PowerSync Dashboard](https://dashboard.powersync.com/), select your project and instance and click the **Connect** button in the top bar to generate the client-side schema in your preferred language. The schema is generated from your Sync Streams. The [CLI](/tools/cli) offers the same function.
The generated schema does not include an `id` column. The client SDK creates an `id` column of type `text` automatically, so you do not need to declare it. See [Client ID](/sync/advanced/client-id) for details.
The available column types are `text`, `integer`, and `real`. These should match the values produced by your Sync Streams. If a value does not match, it is cast automatically. For details on how source database types map to SQLite types, see [Types](/sync/types).
**Example:**
```js theme={null}
// AppSchema.ts
import { column, Schema, Table } from '@powersync/web';
const lists = new Table({
created_at: column.text,
name: column.text,
owner_id: column.text
});
const todos = new Table(
{
list_id: column.text,
created_at: column.text,
completed_at: column.text,
description: column.text,
created_by: column.text,
completed_by: column.text,
completed: column.integer
},
{ indexes: { list: ['list_id'] } }
);
export const AppSchema = new Schema({
todos,
lists
});
// For types
export type Database = (typeof AppSchema)['types'];
export type TodoRecord = Database['todos'];
// OR:
// export type Todo = RowType;
export type ListRecord = Database['lists'];
```
You do not need to declare an `id` column. PowerSync creates it automatically.
### 2. Instantiate the PowerSync Database
Next, instantiate the PowerSync database. PowerSync streams changes from your backend source database into the client-side SQLite database, based on your Sync Streams. Your app reads from and writes to this local database whether the user is online or offline.
**Example:**
```js theme={null}
import { PowerSyncDatabase } from '@powersync/web';
import { Connector } from './Connector';
import { AppSchema } from './AppSchema';
export const db = new PowerSyncDatabase({
// The schema you defined in the previous step
schema: AppSchema,
database: {
// Filename for the SQLite database — it's important to only instantiate one instance per file.
dbFilename: 'powersync.db'
// Optional. Directory where the database file is located.
// dbLocation: 'path/to/directory'
}
});
```
After you instantiate the PowerSync database, call the [connect()](https://powersync-ja.github.io/powersync-js/common/interfaces/CommonPowerSyncDatabase#connect) method to sync data with your backend.
This section assumes that you use PowerSync to sync your backend source database with SQLite in your app. To manage a local SQLite database without sync, instantiate the PowerSync database without calling `connect()` and see the [Local-Only](/client-sdks/advanced/local-only-usage) guide.
```js theme={null}
export const setupPowerSync = async () => {
// Uses the backend connector that you create in the next step
const connector = new Connector();
db.connect(connector);
};
```
### 3. Integrate with Your Backend
The backend connector connects the PowerSync client SDK to your application backend. The SDK uses it to:
1. Get an auth token to connect to the PowerSync instance.
2. Upload client-side writes to your backend API. The SDK places every write to the SQLite database in an upload queue and uploads the queue to your backend when the user is connected. Your backend then applies the changes to the source database.
The connector must implement two methods:
1. [PowerSyncBackendConnector.fetchCredentials](https://github.com/powersync-ja/powersync-js/blob/ed5bb49b5a1dc579050304fab847feb8d09b45c7/packages/common/src/client/connection/PowerSyncBackendConnector.ts#L16) - The SDK calls this method to get authentication credentials. It caches the credentials and calls the method again only when needed, for example on the first connection or when the token is near expiry. See [When `fetchCredentials()` is Called](/configuration/app-backend/client-side-integration#when-fetchcredentials-is-called) for details and [Authentication Setup](/configuration/auth/overview) for how to generate credentials.
2. [PowerSyncBackendConnector.uploadData](https://github.com/powersync-ja/powersync-js/blob/ed5bb49b5a1dc579050304fab847feb8d09b45c7/packages/common/src/client/connection/PowerSyncBackendConnector.ts#L24) - The SDK calls this method whenever it has client-side writes to upload to your backend API. Implement how those writes are processed and uploaded. See [When `uploadData()` is Called](/configuration/app-backend/client-side-integration#when-uploaddata-is-called) for triggers, throttling, and retry behavior, and [Writing Client Changes](/handling-writes/writing-client-changes) for the app backend implementation.
**Example:**
```js theme={null}
import { UpdateType } from '@powersync/web';
export class Connector {
async fetchCredentials() {
// Implement fetchCredentials to obtain a JWT from your authentication service.
// See https://docs.powersync.com/configuration/auth/overview
return {
endpoint: '[Your PowerSync instance URL or self-hosted endpoint]',
// Use a development token (see Authentication Setup https://docs.powersync.com/configuration/auth/development-tokens) to get up and running quickly
token: 'An authentication token'
};
}
async uploadData(database) {
// Implement uploadData to send local changes to your backend service.
// You can omit this method if you only want to sync data from the database to the client
// See example implementation here: https://docs.powersync.com/client-sdks/usage-examples#send-changes-in-local-data-to-your-backend-service
}
}
```
### 4. Subscribe to Sync Streams
Streams defined with `auto_subscribe: true` start syncing as soon as the client connects. For all other streams, your app must subscribe before their data downloads. The basic pattern is: subscribe to a stream, wait for its data to sync, then unsubscribe when the data is no longer needed.
```js theme={null}
// Subscribe to a stream with parameters
const sub = await db.syncStream('list_todos', { list_id: 'abc123' }).subscribe();
// Wait for the initial data to sync
await sub.waitForFirstSync();
// The stream's rows are now in the local SQLite database.
// TODO: Read the todos for this list with a local query.
// When the data is no longer needed
sub.unsubscribe();
```
If you use React, the `useQuery` hook accepts a `streams` option and the `useSyncStream` hook manages a subscription for you. See [Framework Integrations](/sync/streams/client-usage#framework-integrations).
After you unsubscribe, the synced data stays in the local database for the stream's time-to-live (TTL), which is 24 hours by default. If the app subscribes again within that time, the data is already available. See [Client-Side Usage](/sync/streams/client-usage) for framework hooks, per-subscription sync status, custom TTLs, priority overrides, and connection parameters.
## Using PowerSync: CRUD Functions
Once the PowerSync database is connected and your streams have synced, the data is in the local SQLite database.
The most commonly used CRUD functions to interact with your SQLite data are:
* [PowerSyncDatabase.get](/client-sdks/reference/javascript-web#fetching-a-single-item) - get (`SELECT`) a single row from a table.
* [PowerSyncDatabase.getAll](/client-sdks/reference/javascript-web#querying-items-powersync-getall) - get (`SELECT`) a set of rows from a table.
* [PowerSyncDatabase.watch](/client-sdks/reference/javascript-web#watching-queries-powersync-watch) - execute a read query every time a dependent table changes.
* [PowerSyncDatabase.execute](/client-sdks/reference/javascript-web#mutations-powersync-execute-powersync-writetransaction) - execute a write (`INSERT`/`UPDATE`/`DELETE`) query.
### Fetching a Single Item
The [get](https://powersync-ja.github.io/powersync-js/common/interfaces/CommonPowerSyncDatabase#get) method executes a read-only (SELECT) query and returns a single result. It throws an exception if no result is found. Use [getOptional](https://powersync-ja.github.io/powersync-js/common/interfaces/CommonPowerSyncDatabase#getoptional) to return a single optional result (returns `null` if no result is found).
```js theme={null}
// Find a list item by ID
export const findList = async (id) => {
const result = await db.get('SELECT * FROM lists WHERE id = ?', [id]);
return result;
}
```
### Querying Items (PowerSync.getAll)
The [getAll](https://powersync-ja.github.io/powersync-js/common/interfaces/CommonPowerSyncDatabase#getall) method returns a set of rows from a table.
```js theme={null}
// Get all lists
export const getLists = async () => {
const results = await db.getAll('SELECT * FROM lists');
return results;
}
```
### Watching Queries (PowerSync.watch)
The [watch](https://powersync-ja.github.io/powersync-js/common/interfaces/CommonPowerSyncDatabase#watch) method executes a read query whenever a change to a dependent table is made.
```javascript theme={null}
async function* pendingLists(): AsyncIterable {
for await (const result of db.watch(
`SELECT * FROM lists WHERE state = ?`,
['pending']
)) {
yield result.rows?._array ?? [];
}
}
```
```javascript theme={null}
const pendingLists = (onResult: (lists: any[]) => void): void => {
db.watch(
'SELECT * FROM lists WHERE state = ?',
['pending'],
{
onResult: (result: any) => {
onResult(result.rows?._array ?? []);
}
}
);
}
```
For advanced watch query features such as incremental updates and differential results, see [Live Queries / Watch Queries](/client-sdks/watch-queries).
### Mutations (PowerSync.execute, PowerSync.writeTransaction)
The [execute](https://powersync-ja.github.io/powersync-js/common/interfaces/CommonPowerSyncDatabase#execute) method can be used for executing single SQLite write statements.
```js theme={null}
// Delete a list by ID
export const deleteList = async (id) => {
await db.execute('DELETE FROM lists WHERE id = ?', [id]);
};
// OR: delete the list and its todos in one transaction
export const deleteListWithTodos = async (id) => {
await db.writeTransaction(async (tx) => {
await tx.execute('DELETE FROM todos WHERE list_id = ?', [id]);
await tx.execute('DELETE FROM lists WHERE id = ?', [id]);
});
};
```
When using the default client-side [JSON-based view system](/architecture/client-architecture#client-side-schema-and-sqlite-database-structure), writes are applied to a view, with triggers writing to the underlying table. Because of this, `result.rowsAffected` from `db.execute()` can be `0` even when an `UPDATE` or `DELETE` succeeds.
When you need to confirm whether a mutation changed any rows, add a `RETURNING` clause and check the returned rows:
```js theme={null}
const result = await db.execute(
'UPDATE tasks SET deleted_at = ? WHERE id = ? AND deleted_at IS NULL RETURNING id',
[now, id]
);
const wasUpdated = (result.rows?.length ?? 0) > 0;
```
If you need direct table writes, use [raw tables](/client-sdks/advanced/raw-tables).
## Configure Logging
```js theme={null}
import { createConsoleLogger, LogLevels } from '@powersync/web';
// Create a logger with trace minimum level to see all log messages
// Available levels: trace, debug, info, warn, error
const logger = createConsoleLogger({ minLevel: LogLevels.trace });
```
Enable verbose output in the developer tools for detailed logs.
Note that the PowerSync SDK relies on web workers, which inherit the logger from the main database while using a separate log level.
To pass a consistent log level to workers, also pass it to the sync and database workers:
```TypeScript theme={null}
const db = new PowerSyncDatabase({
database: {
dbFilename: '...',
databaseWorkerLogLevel: LogLevels.trace,
},
sync: {
logLevel: LogLevels.trace,
}
});
```
Additionally, the [WASQLiteOpenFactory](https://powersync-ja.github.io/powersync-js/web-sdk/classes/WASQLiteOpenFactory) opens SQLite connections inside a shared web worker. This worker can be inspected in Chrome by accessing:
```
chrome://inspect/#workers
```
## Additional Usage Examples
For more usage examples including accessing connection status, monitoring sync progress, and waiting for initial sync, see the [Usage Examples](/client-sdks/usage-examples) page.
## ORM Support
See [JavaScript ORM Support](/client-sdks/orms/js/overview) for details.
## Vite Quickstart Tutorial
Template repo: [vite-react-ts-powersync-supabase](https://github.com/powersync-community/vite-react-ts-powersync-supabase/)
## Troubleshooting
See [Troubleshooting](/debugging/troubleshooting) for pointers to debug common issues.
## Supported Platforms
See [Supported Platforms -> JS/Web SDK](/resources/supported-platforms#javascript).
## Upgrading the SDK
Run the following command in your project folder:
```bash theme={null}
npm upgrade @powersync/web
```
```bash theme={null}
yarn upgrade @powersync/web
```
```bash theme={null}
pnpm upgrade @powersync/web
```
## Developer Notes
### Connection Methods
This SDK supports two methods for streaming sync commands:
1. **HTTP Streaming (Default)**
* This is the default and recommended connection method.
2. **WebSocket**
* This implementation uses RSocket over WebSocket connections.
* To customize window sizes for flow control and back-pressure, set `fetchStrategy` to `Buffered` (default) or `Sequential`.
* On the web, there is no compelling reason to use WebSockets over HTTP response streams.
By default, the `PowerSyncDatabase.connect()` method uses HTTP streaming. You can optionally specify the `connectionMethod` to override this:
```js theme={null}
// HTTP Streaming (default)
powerSync.connect(connector);
// WebSocket
powerSync.connect(connector, { connectionMethod: SyncStreamConnectionMethod.WEB_SOCKET });
```
### SQLite Virtual File Systems
This SDK supports multiple Virtual File Systems (VFS), each responsible for storing the local SQLite database. The VFS you choose determines where data persists, how multiple browser tabs interact with the database, and whether concurrent reads are possible.
#### 1. IDBBatchAtomicVFS (Default)
The default VFS for applications that need the broadest browser compatibility. It uses IndexedDB for storage. Multiple tabs are fully supported across most modern browsers, and no additional configuration is needed.
#### 2. OPFS-Based Alternatives
PowerSync supports three OPFS (Origin Private File System) implementations that are generally faster than IndexedDB:
**OPFSCoopSyncVFS**
Recommended for applications requiring multi-tab support, especially on Safari/iOS. This implementation provides multi-tab support across all major browsers and offers the most reliable compatibility with Safari and Safari iOS.
Example configuration:
```js theme={null}
import { PowerSyncDatabase, WASQLiteVFS } from '@powersync/web';
export const db = new PowerSyncDatabase({
schema: AppSchema,
database: {
dbFilename: 'exampleVFS.db',
vfs: WASQLiteVFS.OPFSCoopSyncVFS
}
});
```
**AccessHandlePoolVFS**
Use this for single-tab applications that want a straightforward OPFS setup with one worker and direct use of the OPFS Access Handle API. It is not designed for multiple tab use cases.
Example configuration:
```js theme={null}
import { PowerSyncDatabase, WASQLiteVFS } from '@powersync/web';
export const db = new PowerSyncDatabase({
schema: AppSchema,
database: {
dbFilename: 'exampleVFS.db',
vfs: WASQLiteVFS.AccessHandlePoolVFS
}
});
```
**OPFSWriteAheadVFS**
Use this for read-heavy applications that benefit from parallel reads. This VFS uses SQLite's write-ahead log (WAL) mode and opens multiple connections so reads can run concurrently.
This VFS is only supported on Chromium-based browsers (for example Chrome or Edge). Safari, Firefox, and related environments are not supported.
The `additionalReaders` option (defaults to `1`) controls how many extra read-only database connections PowerSync opens alongside the primary connection. Increase it when you routinely run many read queries at once; each extra reader uses more memory. The `useWebWorker` flag must not be set to `false`, because this setup relies on web workers.
Example configuration with 2 additional readers (3 total concurrent reads):
```js theme={null}
import { PowerSyncDatabase, WASQLiteVFS } from '@powersync/web';
export const db = new PowerSyncDatabase({
schema: AppSchema,
database: {
dbFilename: 'exampleVFS.db',
vfs: WASQLiteVFS.OPFSWriteAheadVFS,
additionalReaders: 2
}
});
```
#### 3. In-Memory VFS
Since version 1.39.0 of the `@powersync/web` package, you can use an in-memory database with `WASQLiteVFS.InMemoryVfs`. It runs queries faster than any other single-threaded VFS (both IndexedDB and OPFS, except the write-ahead VFS).
No data is persisted: local writes are lost if they aren't uploaded before the tab is closed, and all data is resynced whenever
a tab is opened. This makes it unsuitable for apps that need to work offline, but a good fit for:
* Development, where starting from a fresh database on every load makes it easy to reproduce issues from a clean state.
* Online-only apps with very frequent queries and small datasets.
When using in-memory databases in production scenarios, consider watching `SELECT * FROM ps_crud LIMIT 1` to detect whether outstanding
local mutations exist and indicate that state to the user. A `beforeunload` event listener can also be useful in this state to
call `preventDefault()` on tab close events, causing browsers to ask for confirmation before closing the tab.
With Chrome and Firefox on desktop, this VFS uses a shared worker to enable multi-tab support by default, meaning that all
tabs have access to the same data and will share a sync worker.
This behavior can be enabled or disabled on all browsers by passing the [`enableMultiTabs` flag](#available-flags).
If support for multi-tabs is not desired, consider giving each tab a uniquely-named PowerSync instance. The in-memory database
would not be shared across tabs in either case, but only one PowerSync database with the same name can sync at a time.
Unique names ensure databases across tabs are fully independent:
```js theme={null}
export const db = new PowerSyncDatabase({
schema: AppSchema,
database: {
dbFilename: `memory-${crypto.randomUUID()}.db`,
vfs: WASQLiteVFS.InMemoryVfs,
enableMultiTabs: false
}
});
```
#### VFS / Option Compatibility Matrix
| VFS Type / Option | Multi-Tab (Standard) | Multi-Tab (Safari/iOS) | Concurrent Reads | Best For |
| ------------------------- | -------------------- | ---------------------- | ---------------- | ---------------------------------------------------------------------------- |
| IDBBatchAtomicVFS | ✅ | ❌ | ❌ | Broadest compatibility, minimal setup |
| OPFSCoopSyncVFS | ✅ | ✅ | ❌ | Multi-tab + Safari/iOS support |
| AccessHandlePoolVFS | ❌ | ❌ | ❌ | Single-tab, single worker, access-handle OPFS |
| OPFSWriteAheadVFS | ✅ | ❌ | ✅ | Chromium only; parallel reads via WAL + readers |
| InMemoryVfs | ✅ | ❌ | ❌ | Development, small data sizes |
| InMemoryWriteAheadLogPool | ❌ (isolated per tab) | ❌ (isolated per tab) | ✅ | Highly concurrent, non-persistent workloads; requires cross-origin isolation |
There are known issues with OPFS (all variants) in Safari's incognito mode.
### Multi-Threaded In-Memory SQLite Connection Pool
Since version 2.2.0, the `@powersync/web` package provides an experimental, per-tab in-memory SQLite connection pool for highly concurrent query workloads. It uses a design inspired from `OPFSWriteAheadVFS`, but relies on `SharedArrayBuffer` to coordinate an in-memory database instead of persisting to OPFS. It is available under a separate import and is thus configured via the `opened` option on `PowerSyncDatabase` instead of `database.vfs`.
This setup is experimental and might change in the future.
The pool creates `numWorkers` SQLite connections in dedicated web workers. One worker is the designated writer, while the remaining workers are used as additional readers. The writer can also serve reads while it is idle.
Read-only transactions can execute in parallel, including while the writer appends changes to a custom in-memory write-ahead overlay. The database and write-ahead-log buffers are backed by growable `SharedArrayBuffer` objects shared across the workers. This parallelism benefits workloads with overlapping queries. It does not inherently make an individual sequential query faster, and additional workers increase startup time and memory usage.
Each pool instance is independent and belongs to one tab. Its data is not persisted or shared across tabs, and the application cannot assign it a database filename. Opening or refreshing a tab creates a fresh database that must be resynced. Any local writes that have not been uploaded are lost when the tab closes.
This option requires cross-origin isolation and browser support for growable `SharedArrayBuffer`.
This connection pool is primarily relevant when all of the following apply:
1. You need highly concurrent, high-performance queries in your app.
2. At the same time, the overall database size (or at least the actively synced part of the database) is relatively small, as it gets
synced every time a tab is opened.
3. You don't need persistence.
4. You can enable [cross-origin isolation](https://web.dev/articles/cross-origin-isolation-guide) by using the appropriate headers.
Without cross-origin isolation and shared array buffers, constructing the pool will throw.
5. You don't need multiple tabs to share offline state.
The pool is exposed through a separate package entry point so applications that do not use it do not include it in their main bundle:
```js theme={null}
import { InMemoryWriteAheadLogPool } from '@powersync/web/extra/shared-memory-pool';
import { PowerSyncDatabase } from '@powersync/web';
export const db = new PowerSyncDatabase({
schema: AppSchema,
opened: new InMemoryWriteAheadLogPool({
numWorkers: 3 // Uses one writer, two additional workers for reads.
}),
});
```
### Managing OPFS Storage
Unlike IndexedDB, OPFS storage cannot be managed through browser developer tools. The following utility functions can help you manage OPFS storage programmatically:
```js theme={null}
// Clear all OPFS storage
async function purgeVFS() {
await powerSync.disconnect();
await powerSync.close();
const root = await navigator.storage.getDirectory();
await new Promise(resolve => setTimeout(resolve, 1)); // Allow .db-wal to become deletable
for await (const [name, entry] of root.entries!()) {
try {
if (entry.kind === 'file') {
await root.removeEntry(name);
} else if (entry.kind === 'directory') {
await root.removeEntry(name, { recursive: true });
}
} catch (err) {
console.error(`Failed to delete ${entry.kind}: ${name}`, err);
}
}
}
// List OPFS entries
async function listVfsEntries() {
const root = await navigator.storage.getDirectory();
for await (const [name, entry] of root.entries()) {
console.log(`${entry.kind}: ${name}`);
}
}
```
### Multiple Tab Support
* Full multi-tab support relies on shared web workers, which are disabled by default on Android, iOS, and Safari. On these platforms, the SDK falls back to a less reliable broadcast-based mechanism, as described below.
* For Safari, use the [`OPFSCoopSyncVFS`](/client-sdks/reference/javascript-web#sqlite-virtual-file-systems) virtual file system to ensure stable multi-tab functionality.
Using PowerSync between multiple tabs is supported on most desktop browsers. Multiple tab support relies on shared web workers for database and sync operations. When enabled, the SDK creates a shared web worker named `shared-powersync-[dbFileName]`.
The shared sync worker connects to the PowerSync Service and applies changes to the database on behalf of all tabs. It calls the `fetchCredentials` and `uploadData` methods of the most recently opened tab. When that tab closes, the worker uses the previously opened tab instead. When using an IndexedDB-based VFS, the SDK can also open database connections in a shared worker so that writes made in one tab are instantly available in the others.
Multi-tab support is enabled by default where available. You can disable it with the [`enableMultiTabs` flag](#available-flags):
```js theme={null}
export const db = new PowerSyncDatabase({
schema: AppSchema,
database: {
dbFilename: 'my_app_db.sqlite',
enableMultiTabs: false
},
});
```
#### Behavior Without Shared Workers
When multi-tab support is disabled, whether explicitly or because the platform does not support it, each tab spawns a standard web worker for database operations. These workers can safely operate on the database concurrently. Only one tab connects and syncs at a time, and only that tab's `fetchCredentials` and `uploadData` methods are called.
The SDK still tries to share state across tabs using broadcast channels (since version 2.1.0 of the SDK): update notifications for watched queries, the sync status (fields like `hasSynced` and download progress), and sync stream subscriptions made in any tab. This is less reliable than shared workers, so updates may not reach every tab.
### Using PowerSyncDatabase Flags
The `PowerSyncDatabase` constructor accepts the following flags. Use them to enable or disable specific features.
#### Configuring Options
You can configure these options during the initialization of `PowerSyncDatabase` as top-level constructor properties.
```javascript theme={null}
import { PowerSyncDatabase } from '@powersync/web';
import { AppSchema } from '@/library/powersync/AppSchema';
export const db = new PowerSyncDatabase({
schema: AppSchema,
database: {
dbFilename: 'example.db',
enableMultiTabs: true,
},
broadcastLogs: true,
});
```
#### Available Flags
default: `true` (`false` on Android, iOS, and Safari)
Enables support for multiple tabs using shared web workers. When enabled, multiple tabs share the same database and sync connection.
default: `true`
Enables the broadcasting of logs for debugging purposes. This flag helps monitor shared worker logs in a multi-tab environment.
default: `false`
Disables warnings when running in SSR (Server-Side Rendering) mode.
default: `false`
Enables SSR mode. In this mode, only empty query results will be returned, and syncing with the backend is disabled.
default: `true`
Enables the use of web workers for database operations. Disabling this flag also disables multi-tab support.
#### Flag Behavior
**Example 1: Multi-Tab Support**
By default, multi-tab support is enabled if supported by the browser. To explicitly disable this feature:
```javascript theme={null}
export const db = new PowerSyncDatabase({
schema: AppSchema,
database: {
dbFilename: 'my_app_db.sqlite',
enableMultiTabs: false,
},
});
```
When disabled, each tab uses independent workers. The SDK tries to share sync status and update notifications for watched queries across tabs, but this is less reliable than shared workers. See [Behavior Without Shared Workers](#behavior-without-shared-workers).
**Example 2: Verbose Debugging with Broadcast Logs**
To enable detailed logging for debugging:
```javascript theme={null}
export const db = new PowerSyncDatabase({
schema: AppSchema,
database: {
dbFilename: 'my_app_db.sqlite',
databaseWorkerLogLevel: LogLevels.debug,
},
sync: {
logLevel: LogLevels.debug,
},
logger: createConsoleLogger({ minLevel: LogLevels.debug }),
});
```
Logs include details of database and sync operations.
#### Recommendations
1. **Set `enableMultiTabs`** to `true` if your application shares data across multiple tabs.
2. **Set `broadcastLogs`** to `true` during development to troubleshoot and monitor database and sync operations.
# JavaScript Web SDK API Reference
Source: https://docs.powersync.com/client-sdks/reference/javascript-web-api
Full API reference for the PowerSync JavaScript Web SDK.
# Kotlin SDK
Source: https://docs.powersync.com/client-sdks/reference/kotlin
Use PowerSync in Kotlin Multiplatform apps.
```text Build with AI icon="sparkles" wrap theme={null}
Install the PowerSync Agent Skills with: npx skills add powersync-ja/agent-skills. Then follow the skills to onboard this project to PowerSync using the Kotlin Multiplatform SDK.
```
The PowerSync Kotlin SDK is distributed via Maven Central
Refer to the `powersync-kotlin` repo on GitHub
Full API reference for the SDK
Gallery of example projects/demo apps built with Kotlin and PowerSync.
Changelog for the SDK
## SDK Features
* **Real-time streaming of database changes**: Changes made by one user are instantly streamed to all other users with access to that data. This keeps clients automatically in sync without manual polling or refresh logic.
* **Direct access to a local SQLite database**: Data is stored locally, so apps can read and write instantly without network calls. This enables offline support and faster user interactions.
* **Asynchronous background execution**: The SDK performs database operations in the background to avoid blocking the application’s main thread. This means that apps stay responsive, even during heavy data activity.
* **Query subscriptions for live updates**: The SDK supports query subscriptions that automatically push real-time updates to client applications as data changes, keeping your UI reactive and up to date.
* **Automatic schema management**: PowerSync syncs schemaless data and applies a client-defined schema using SQLite views. This architecture means that PowerSync SDKs handle schema changes without explicit migrations on the client side.
## Installation
Add the [PowerSync SDK](https://central.sonatype.com/artifact/com.powersync/core) to your project by adding the following to your `build.gradle.kts` file:
```toml gradle/libs.versions.toml theme={null}
[versions]
# Please check the latest version at https://github.com/powersync-ja/powersync-kotlin/releases/
powersync = "1.12.0"
[libraries]
powersync-core = { module = "com.powersync:core", version.ref = "powersync" }
powersync-integration-supabase = { module = "com.powersync:connector-supabase", version.ref = "powersync" }
```
```Kotlin build.gradle.kts icon="https://mintcdn.com/powersync/GTJdSKFSfUc2Sxtc/logo/gradle.svg?fit=max&auto=format&n=GTJdSKFSfUc2Sxtc&q=85&s=bb14bd89bac7520f103a2ad2abc17053" theme={null}
kotlin {
//...
sourceSets {
commonMain.dependencies {
implementation(libs.powersync.core)
// If you want to use the Supabase Connector, also add the following:
implementation(libs.powersync.integration.supabase)
}
//...
}
}
```
```Kotlin build.gradle.kts icon="https://mintcdn.com/powersync/GTJdSKFSfUc2Sxtc/logo/gradle.svg?fit=max&auto=format&n=GTJdSKFSfUc2Sxtc&q=85&s=bb14bd89bac7520f103a2ad2abc17053" theme={null}
kotlin {
//...
sourceSets {
commonMain.dependencies {
implementation("com.powersync:core:$powersyncVersion")
// If you want to use the Supabase Connector, also add the following:
implementation("com.powersync:connector-supabase:$powersyncVersion")
}
//...
}
}
```
On Kotlin SDK v1.12.0 and later, the [PowerSync SQLite core extension](https://github.com/powersync-ja/powersync-sqlite-core) is statically linked into `com.powersync:core` for Apple targets (iOS, macOS, tvOS, and watchOS), consistent with Android and JVM. Use the Gradle dependencies above only. When you upgrade from an older SDK, remove any Swift package dependency on [`powersync-sqlite-core-swift`](https://github.com/powersync-ja/powersync-sqlite-core-swift) and any `powersync-sqlite-core` CocoaPod from your Xcode or CocoaPods setup.
**Supported platforms**
* PowerSync supports Android, JVM and Apple (iOS, macOS, tvOS, watchOS) targets through Kotlin Multiplatform.
* On the JVM, the following platforms are supported: Linux AArch64, Linux X64, macOS AArch64, macOS X64, Windows X64.
* Web (JS and WebAssembly) targets have experimental support. See [Experimental Web Support](#experimental-web-support).
## Getting Started
**Prerequisites:** Before you start, connect your source database to the PowerSync Service and deploy Sync Streams. These are steps 1-4 in the [Setup Guide](/intro/setup-guide).
### 1. Define the Client-Side Schema
The client-side schema defines the tables and columns of the SQLite database that the PowerSync client SDK manages and that your app reads from and writes to. It is usually derived from your backend database schema and your [Sync Streams](/sync/streams/overview), and it can also include [local-only tables](/client-sdks/advanced/local-only-usage). You apply the schema when you instantiate the database in the next step.
Schema migrations are not required. The SDK syncs schemaless data and applies the schema to that data with SQLite views. The exception is [raw tables](/client-sdks/advanced/raw-tables), which you create and migrate yourself.
**Generate schema automatically**
In the [PowerSync Dashboard](https://dashboard.powersync.com/), select your project and instance and click the **Connect** button in the top bar to generate the client-side schema in your preferred language. The schema is generated from your Sync Streams. The [CLI](/tools/cli) offers the same function.
The generated schema does not include an `id` column. The client SDK creates an `id` column of type `text` automatically, so you do not need to declare it. See [Client ID](/sync/advanced/client-id) for details.
The available column types are `text`, `integer`, and `real`. These should match the values produced by your Sync Streams. If a value does not match, it is cast automatically. For details on how source database types map to SQLite types, see [Types](/sync/types).
**Example:**
```kotlin theme={null}
// AppSchema.kt
import com.powersync.db.schema.Column
import com.powersync.db.schema.Index
import com.powersync.db.schema.IndexedColumn
import com.powersync.db.schema.Schema
import com.powersync.db.schema.Table
val AppSchema: Schema = Schema(
listOf(
Table(
name = "todos",
columns = listOf(
Column.text("list_id"),
Column.text("created_at"),
Column.text("completed_at"),
Column.text("description"),
Column.integer("completed"),
Column.text("created_by"),
Column.text("completed_by")
),
// Index to allow efficient lookup within a list
indexes = listOf(
Index("list", listOf(IndexedColumn.descending("list_id")))
)
),
Table(
name = "lists",
columns = listOf(
Column.text("created_at"),
Column.text("name"),
Column.text("owner_id")
)
)
)
)
```
You do not need to declare an `id` column. PowerSync creates it automatically.
### 2. Instantiate the PowerSync Database
Next, instantiate the PowerSync database. PowerSync streams changes from your backend source database into the client-side SQLite database, based on your Sync Streams. Your app reads from and writes to this local database whether the user is online or offline.
**Example:**
a. Create a platform-specific `DatabaseDriverFactory` to be used by the `PowerSyncBuilder` to create the SQLite database driver.
```kotlin theme={null}
// commonMain
import com.powersync.DatabaseDriverFactory
import com.powersync.PowerSyncDatabase
// Android
val driverFactory = DatabaseDriverFactory(this)
// iOS & Desktop
val driverFactory = DatabaseDriverFactory()
```
b. Build a `PowerSyncDatabase` instance using the `PowerSyncBuilder` and the `DatabaseDriverFactory`. The schema you created in a previous step is provided as a parameter:
```kotlin theme={null}
// commonMain
val database = PowerSyncDatabase(
factory = driverFactory, // The factory you defined above
schema = AppSchema, // The schema you defined in the previous step
dbFilename = "powersync.db",
// logger = YourLogger, // Optional. Your own Kermit Logger.
// dbDirectory = "path/to/directory", // Optional. Directory for the database file. Ignored on iOS.
)
```
c. Connect the `PowerSyncDatabase` to sync data with your backend:
This section assumes that you use PowerSync to sync your backend source database with SQLite in your app. To manage a local SQLite database without sync, instantiate the PowerSync database without calling `connect()` and see the [Local-Only](/client-sdks/advanced/local-only-usage) guide.
```kotlin theme={null}
// commonMain
// Uses the backend connector that you create in the next step
database.connect(MyConnector())
```
**Special case: Compose Multiplatform**
The artifact `com.powersync:powersync-compose` provides a simpler API:
```kotlin theme={null}
// commonMain
val database = rememberPowerSyncDatabase(schema)
remember {
database.connect(MyConnector())
}
```
### 3. Integrate with Your Backend
The backend connector connects the PowerSync client SDK to your application backend. The SDK uses it to:
1. Get an auth token to connect to the PowerSync instance.
2. Upload client-side writes to your backend API. The SDK places every write to the SQLite database in an upload queue and uploads the queue to your backend when the user is connected. Your backend then applies the changes to the source database.
The connector must implement two methods:
1. `PowerSyncBackendConnector.fetchCredentials` - The SDK calls this method to get authentication credentials. It caches the credentials and calls the method again only when needed, for example on the first connection or when the token is near expiry. See [When `fetchCredentials()` is Called](/configuration/app-backend/client-side-integration#when-fetchcredentials-is-called) for details and [Authentication Setup](/configuration/auth/overview) for how to generate credentials.
2. `PowerSyncBackendConnector.uploadData` - The SDK calls this method whenever it has client-side writes to upload to your backend API. Implement how those writes are processed and uploaded. See [When `uploadData()` is Called](/configuration/app-backend/client-side-integration#when-uploaddata-is-called) for triggers, throttling, and retry behavior, and [Writing Client Changes](/handling-writes/writing-client-changes) for the app backend implementation.
**Example:**
```kotlin theme={null}
// PowerSync.kt
import com.powersync.DatabaseDriverFactory
import com.powersync.PowerSyncDatabase
class MyConnector : PowerSyncBackendConnector() {
override suspend fun fetchCredentials(): PowerSyncCredentials {
// implement fetchCredentials to obtain the necessary credentials to connect to your backend
// See an example implementation in https://github.com/powersync-ja/powersync-kotlin/blob/main/integrations/supabase/src/commonMain/kotlin/com/powersync/connector/supabase/SupabaseConnector.kt
return PowerSyncCredentials(
endpoint = "[Your PowerSync instance URL or self-hosted endpoint]",
// Use a development token (see Authentication Setup https://docs.powersync.com/configuration/auth/development-tokens) to get up and running quickly
token = "An authentication token"
)
}
override suspend fun uploadData(database: PowerSyncDatabase) {
// Implement uploadData to send local changes to your backend service
// You can omit this method if you only want to sync data from the server to the client
// See an example implementation under Usage Examples (sub-page)
// See https://docs.powersync.com/handling-writes/writing-client-changes for considerations.
}
}
```
If you use Supabase, you can use [SupabaseConnector.kt](https://github.com/powersync-ja/powersync-kotlin/blob/main/integrations/supabase/src/commonMain/kotlin/com/powersync/connector/supabase/SupabaseConnector.kt) as a starting point.
### 4. Subscribe to Sync Streams
Streams defined with `auto_subscribe: true` start syncing as soon as the client connects. For all other streams, your app must subscribe before their data downloads. The basic pattern is: subscribe to a stream, wait for its data to sync, then unsubscribe when the data is no longer needed.
```kotlin theme={null}
// Subscribe to a stream with parameters
val sub = database.syncStream("list_todos", mapOf("list_id" to JsonParam.String("abc123")))
.subscribe()
// Wait for the initial data to sync
sub.waitForFirstSync()
// The stream's rows are now in the local SQLite database.
// TODO: Read the todos for this list with a local query.
// When the data is no longer needed
sub.unsubscribe()
```
After you unsubscribe, the synced data stays in the local database for the stream's time-to-live (TTL), which is 24 hours by default. If the app subscribes again within that time, the data is already available. See [Client-Side Usage](/sync/streams/client-usage) for framework hooks, per-subscription sync status, custom TTLs, priority overrides, and connection parameters.
## Using PowerSync: CRUD Functions
Once the PowerSync database is connected and your streams have synced, the data is in the local SQLite database.
The most commonly used CRUD functions to interact with your SQLite data are:
* [PowerSyncDatabase.get](/client-sdks/reference/kotlin#fetching-a-single-item) - get (`SELECT`) a single row from a table.
* [PowerSyncDatabase.getAll](/client-sdks/reference/kotlin#querying-items-powersync-getall) - get (`SELECT`) a set of rows from a table.
* [PowerSyncDatabase.watch](/client-sdks/reference/kotlin#watching-queries-powersync-watch) - execute a read query every time a dependent table changes.
* [PowerSyncDatabase.execute](/client-sdks/reference/kotlin#mutations-powersync-execute) - execute a write (`INSERT`/`UPDATE`/`DELETE`) query.
### Fetching a Single Item
The `get` method executes a read-only (SELECT) query and returns a single result. It throws an exception if no result is found. Use `getOptional` to return a single optional result (returns `null` if no result is found).
```kotlin theme={null}
// Find a list item by ID
suspend fun find(id: Any): TodoList {
return database.get(
"SELECT * FROM lists WHERE id = ?",
listOf(id)
) { cursor ->
TodoList.fromCursor(cursor)
}
}
```
### Querying Items (PowerSync.getAll)
The `getAll` method executes a read-only (SELECT) query and returns a set of rows.
```kotlin theme={null}
// Get all list IDs
suspend fun getLists(): List {
return database.getAll(
"SELECT id FROM lists WHERE id IS NOT NULL"
) { cursor ->
cursor.getString("id")
}
}
```
### Watching Queries (PowerSync.watch)
The `watch` method executes a read query whenever a change to a dependent table is made.
```kotlin theme={null}
fun watchPendingLists(): Flow> =
db.watch(
"SELECT * FROM lists WHERE state = ?",
listOf("pending"),
) { cursor ->
ListItem(
id = cursor.getString("id"),
name = cursor.getString("name"),
)
}
```
### Mutations (PowerSync.execute)
The `execute` method executes a write query (INSERT, UPDATE, DELETE) and returns the results (if any).
```kotlin theme={null}
suspend fun insertCustomer(name: String, email: String) {
database.writeTransaction { tx ->
tx.execute(
sql = "INSERT INTO customers (id, name, email) VALUES (uuid(), ?, ?)",
parameters = listOf(name, email)
)
}
}
suspend fun updateCustomer(id: String, name: String, email: String) {
database.execute(
sql = "UPDATE customers SET name = ? WHERE email = ?",
parameters = listOf(name, email)
)
}
suspend fun deleteCustomer(id: String? = null) {
// If no id is provided, delete the first customer in the database
val targetId =
id ?: database.getOptional(
sql = "SELECT id FROM customers LIMIT 1",
mapper = { cursor ->
cursor.getString(0)!!
}
) ?: return
database.writeTransaction { tx ->
tx.execute(
sql = "DELETE FROM customers WHERE id = ?",
parameters = listOf(targetId)
)
}
}
```
## Configure Logging
You can supply your own logger. It must conform to the [Kermit Logger](https://kermit.touchlab.co/docs/):
```kotlin theme={null}
PowerSyncDatabase(
...
logger: Logger? = YourLogger
)
```
If you do not supply a logger, the SDK creates a default Kermit Logger. It shows `Warn` and above in release builds and `Verbose` in debug builds:
```kotlin theme={null}
val defaultLogger: Logger = Logger
// Severity is set to Verbose in Debug and Warn in Release
if(BuildConfig.isDebug) {
Logger.setMinSeverity(Severity.Verbose)
} else {
Logger.setMinSeverity(Severity.Warn)
}
return defaultLogger
```
You can use the logger anywhere in your code:
```kotlin theme={null}
import co.touchlab.kermit.Logger
Logger.i("Some information");
Logger.e("Some error");
...
```
## Custom HTTP Clients and Headers
PowerSync uses streaming HTTP responses or WebSockets to connect to the PowerSync Service. The SDK uses
[ktor](https://ktor.io/) as a cross-platform HTTP client. `com.powersync:core` depends on an OkHttp-based
implementation on Android and JVM targets, and on the Darwin engine for native Apple targets.
Configure how the PowerSync SDK sets up client engines by passing an instance of `SyncClientConfiguration.ExtendedConfig`
in `SyncOptions`:
```kotlin theme={null}
import com.powersync.sync.SyncClientConfiguration
import com.powersync.sync.SyncOptions
import io.ktor.client.plugins.defaultRequest
import io.ktor.client.request.header
db.connect(
connector,
options = SyncOptions(
clientConfiguration = SyncClientConfiguration.ExtendedConfig {
// For more options, see https://ktor.io/docs/client-create-and-configure.html#plugins
defaultRequest {
header("X-Custom-Header", "Used by PowerSync")
}
}
)
)
```
If you want to use a custom HTTP client engine instead, use `SyncClientConfiguration.ExistingClient`:
```kotlin theme={null}
import com.powersync.sync.configureSyncHttpClient
db.connect(
connector,
options = SyncOptions(
clientConfiguration = SyncClientConfiguration.ExistingClient(HttpClient(YourPreferredClientEngine) {
// Important: This configures a minimal set of plugins required by the PowerSync sync client.
configureSyncHttpClient()
})
)
)
```
## Additional Usage Examples
For more usage examples including accessing connection status, monitoring sync progress, and waiting for initial sync, see the [Usage Examples](/client-sdks/usage-examples) page.
## ORM / SQL Library Support
You can use the Kotlin SDK with the SQLDelight and Room libraries to define and run SQL queries.
For details, see the [SQL Library Support](/client-sdks/orms/kotlin/overview) page.
## Experimental Web Support
Version 1.14.1 of the SDK adds initial support for web targets (JS and WebAssembly). Use `WebConnectionFactory` to open a PowerSync database on the web.
Web support is experimental and incomplete. We are sharing it for early testing and prototyping purposes, and it is not ready for production use. The main limitation is that multi-tab support is not functional in this version: tabs don't coordinate sync connections or share update notifications, so use the database from a single tab only.
If you try web support, test it thoroughly and report any issues on [GitHub](https://github.com/powersync-ja/powersync-kotlin/issues). Follow the [tracking issue](https://github.com/powersync-ja/powersync-kotlin/issues/362) for progress on stabilizing web support.
For a working example, see the web target of the [Supabase To-Do List demo](https://github.com/powersync-ja/powersync-kotlin/tree/main/demos/supabase-todolist).
## Troubleshooting
See [Troubleshooting](/debugging/troubleshooting) for pointers to debug common issues.
## Supported Platforms
See [Supported Platforms -> Kotlin SDK](/resources/supported-platforms#kotlin).
## Upgrading the SDK
Update your project's Gradle file (`build.gradle.kts`) with the latest version of the [SDK](https://central.sonatype.com/artifact/com.powersync/core).
# Kotlin SDK API Reference
Source: https://docs.powersync.com/client-sdks/reference/kotlin-api
Full API reference for the PowerSync Kotlin Multiplatform SDK.
# Node.js Client SDK
Source: https://docs.powersync.com/client-sdks/reference/node
Use PowerSync in Node.js apps.
This page describes the PowerSync *client* SDK for Node.js.
If you're interested in using PowerSync for your Node.js backend, no special package is required.
Instead, follow our guides on [app backend setup](/configuration/app-backend/setup).
```text Build with AI icon="sparkles" wrap theme={null}
Install the PowerSync Agent Skills with: npx skills add powersync-ja/agent-skills. Then follow the skills to onboard this project to PowerSync using the Node.js client SDK.
```
This SDK is distributed via NPM
Refer to `packages/node` in the `powersync-js` repo on GitHub
Full API reference for the SDK
Gallery of example projects/demo apps built with Node.js and PowerSync
Changelog for the SDK
## SDK Features
* **Real-time streaming of database changes**: Changes made by one user are instantly streamed to all other users with access to that data. This keeps clients automatically in sync without manual polling or refresh logic.
* **Direct access to a local SQLite database**: Data is stored locally, so apps can read and write instantly without network calls. This enables offline support and faster user interactions.
* **Asynchronous background execution**: The SDK performs database operations in the background to avoid blocking the application’s main thread. This means that apps stay responsive, even during heavy data activity.
* **Query subscriptions for live updates**: The SDK supports query subscriptions that automatically push real-time updates to client applications as data changes, keeping your UI reactive and up to date.
* **Automatic schema management**: PowerSync syncs schemaless data and applies a client-defined schema using SQLite views. This architecture means that PowerSync SDKs handle schema changes without explicit migrations on the client side.
## Quickstart
Add the [PowerSync Node NPM package](https://www.npmjs.com/package/@powersync/node) to your project:
```bash theme={null}
npm install @powersync/node
```
```bash theme={null}
yarn add @powersync/node
```
```bash theme={null}
pnpm install @powersync/node
```
**Install Peer Dependencies**
The PowerSync SDK for Node.js supports multiple drivers. More details are available under [Encryption and Custom SQLite Drivers](/client-sdks/reference/node#encryption-and-custom-sqlite-drivers). We currently recommend the `better-sqlite3` package for most users:
```bash theme={null}
npm install better-sqlite3
```
```bash theme={null}
yarn add better-sqlite3
```
```bash theme={null}
pnpm install better-sqlite3
```
Previous versions of the PowerSync SDK for Node.js used the `@powersync/better-sqlite3` fork as a
required peer dependency.
This is no longer recommended. After upgrading to `@powersync/node` version `0.12.0` or later, ensure
the old package is no longer installed by running `npm uninstall @powersync/better-sqlite3`
**Common Installation Issues**
The `better-sqlite3` package requires native compilation, which depends on certain system tools.
Prebuilt assets are available and used by default, but a custom compilation may be started depending on the Node.js
or Electron version used.
This compilation process is handled by `node-gyp` and may fail if required dependencies are missing or misconfigured.
Refer to the [PowerSync Node package README](https://www.npmjs.com/package/@powersync/node) for more details.
**Prerequisites:** Before you start, connect your source database to the PowerSync Service and deploy Sync Streams. These are steps 1-4 in the [Setup Guide](/intro/setup-guide).
### 1. Define the Client-Side Schema
The client-side schema defines the tables and columns of the SQLite database that the PowerSync client SDK manages and that your app reads from and writes to. It is usually derived from your backend database schema and your [Sync Streams](/sync/streams/overview), and it can also include [local-only tables](/client-sdks/advanced/local-only-usage). You apply the schema when you instantiate the database in the next step.
Schema migrations are not required. The SDK syncs schemaless data and applies the schema to that data with SQLite views. The exception is [raw tables](/client-sdks/advanced/raw-tables), which you create and migrate yourself.
You can use [this example](https://github.com/powersync-ja/powersync-js/blob/e5a57a539150f4bc174e109d3898b6e533de272f/demos/example-node/src/powersync.ts#L47-L77) as a reference when defining your schema.
**Generate schema automatically**
In the [PowerSync Dashboard](https://dashboard.powersync.com/), select your project and instance and click the **Connect** button in the top bar to generate the client-side schema in your preferred language. The schema is generated from your Sync Streams. The [CLI](/tools/cli) offers the same function.
The generated schema does not include an `id` column. The client SDK creates an `id` column of type `text` automatically, so you do not need to declare it. See [Client ID](/sync/advanced/client-id) for details.
In the Dashboard, select JavaScript and replace the suggested import with `@powersync/node`.
The available column types are `text`, `integer`, and `real`. These should match the values produced by your Sync Streams. If a value does not match, it is cast automatically. For details on how source database types map to SQLite types, see [Types](/sync/types).
### 2. Instantiate the PowerSync Database
Next, instantiate the PowerSync database. PowerSync streams changes from your backend source database into the client-side SQLite database, based on your Sync Streams. Your app reads from and writes to this local database whether the user is online or offline.
**Example:**
```js theme={null}
import { PowerSyncDatabase } from '@powersync/node';
import { Connector } from './Connector';
import { AppSchema } from './Schema';
export const db = new PowerSyncDatabase({
// The schema you defined in the previous step
schema: AppSchema,
database: {
// Filename for the SQLite database — it's important to only instantiate one instance per file.
dbFilename: 'powersync.db',
// Optional. Directory where the database file is located.
// dbLocation: 'path/to/directory'
},
});
```
### 3. Integrate with Your Backend
The backend connector connects the PowerSync client SDK to your application backend. The SDK uses it to:
1. Get an auth token to connect to the PowerSync instance.
2. Upload client-side writes to your backend API. The SDK places every write to the SQLite database in an upload queue and uploads the queue to your backend when the user is connected. Your backend then applies the changes to the source database.
The connector must implement two methods:
1. [PowerSyncBackendConnector.fetchCredentials](https://github.com/powersync-ja/powersync-js/blob/ed5bb49b5a1dc579050304fab847feb8d09b45c7/packages/common/src/client/connection/PowerSyncBackendConnector.ts#L16) - The SDK calls this method to get authentication credentials. It caches the credentials and calls the method again only when needed, for example on the first connection or when the token is near expiry. See [When `fetchCredentials()` is Called](/configuration/app-backend/client-side-integration#when-fetchcredentials-is-called) for details and [Authentication Setup](/configuration/auth/overview) for how to generate credentials.
2. [PowerSyncBackendConnector.uploadData](https://github.com/powersync-ja/powersync-js/blob/ed5bb49b5a1dc579050304fab847feb8d09b45c7/packages/common/src/client/connection/PowerSyncBackendConnector.ts#L24) - The SDK calls this method whenever it has client-side writes to upload to your backend API. Implement how those writes are processed and uploaded. See [When `uploadData()` is Called](/configuration/app-backend/client-side-integration#when-uploaddata-is-called) for triggers, throttling, and retry behavior, and [Writing Client Changes](/handling-writes/writing-client-changes) for the app backend implementation.
**Example:**
```js theme={null}
import { UpdateType } from '@powersync/node';
export class Connector {
async fetchCredentials() {
// Implement fetchCredentials to obtain a JWT from your authentication service.
// See https://docs.powersync.com/configuration/auth/overview
return {
endpoint: '[Your PowerSync instance URL or self-hosted endpoint]',
// Use a development token (see Authentication Setup https://docs.powersync.com/configuration/auth/development-tokens) to get up and running quickly
token: 'An authentication token'
};
}
async uploadData(database) {
// Implement uploadData to send local changes to your backend service.
// You can omit this method if you only want to sync data from the database to the client
// See example implementation here: https://docs.powersync.com/client-sdks/usage-examples#send-changes-in-local-data-to-your-backend-service
}
}
```
With your database instantiated and your connector ready, call `connect()` to start syncing data with your backend:
```js theme={null}
await db.connect(new Connector());
await db.waitForFirstSync(); // Optional, to wait for a complete snapshot of data to be available
```
This section assumes that you use PowerSync to sync your backend source database with SQLite in your app. To manage a local SQLite database without sync, instantiate the PowerSync database without calling `connect()` and see the [Local-Only](/client-sdks/advanced/local-only-usage) guide.
### 4. Subscribe to Sync Streams
Streams defined with `auto_subscribe: true` start syncing as soon as the client connects. For all other streams, your app must subscribe before their data downloads. The basic pattern is: subscribe to a stream, wait for its data to sync, then unsubscribe when the data is no longer needed.
```js theme={null}
// Subscribe to a stream with parameters
const sub = await db.syncStream('list_todos', { list_id: 'abc123' }).subscribe();
// Wait for the initial data to sync
await sub.waitForFirstSync();
// The stream's rows are now in the local SQLite database.
// TODO: Read the todos for this list with a local query.
// When the data is no longer needed
sub.unsubscribe();
```
After you unsubscribe, the synced data stays in the local database for the stream's time-to-live (TTL), which is 24 hours by default. If the app subscribes again within that time, the data is already available. See [Client-Side Usage](/sync/streams/client-usage) for framework hooks, per-subscription sync status, custom TTLs, priority overrides, and connection parameters.
## Usage
Once the PowerSync database is connected and your streams have synced, the data is in the local SQLite database. The API to run queries and updates is identical to the [JavaScript Web SDK](/client-sdks/reference/javascript-web#using-powersync-crud-functions):
```js theme={null}
// Use db.get() to fetch a single row:
console.log(await db.get('SELECT powersync_rs_version();'));
// Or db.getAll() to fetch all:
console.log(await db.getAll('SELECT * FROM lists;'));
// And db.execute for inserts, updates and deletes:
await db.execute(
"INSERT INTO lists (id, created_at, name, owner_id) VALUES (uuid(), datetime('now'), ?, uuid());",
['My new list']
);
```
### Confirming Mutation Success with `RETURNING`
When using the default client-side [JSON-based view system](/architecture/client-architecture#client-side-schema-and-sqlite-database-structure), writes are applied to a view, with triggers writing to the underlying table. Because of this, `result.rowsAffected` from `db.execute()` can be `0` even when an `UPDATE` or `DELETE` succeeds.
When you need to confirm whether a mutation changed any rows, add a `RETURNING` clause and check the returned rows:
```js theme={null}
const result = await db.execute(
'UPDATE tasks SET deleted_at = ? WHERE id = ? AND deleted_at IS NULL RETURNING id',
[now, id]
);
const wasUpdated = (result.rows?.length ?? 0) > 0;
```
If you need direct table writes, use [raw tables](/client-sdks/advanced/raw-tables).
### Watch Queries
The `db.watch()` method executes a read query whenever a change to a dependent table is made.
```javascript theme={null}
async function* pendingLists(): AsyncIterable {
for await (const result of db.watch(
`SELECT * FROM lists WHERE state = ?`,
['pending']
)) {
yield result.rows?._array ?? [];
}
}
```
```javascript theme={null}
const pendingLists = (onResult: (lists: any[]) => void): void => {
db.watch(
'SELECT * FROM lists WHERE state = ?',
['pending'],
{
onResult: (result: any) => {
onResult(result.rows?._array ?? []);
}
}
);
}
```
For advanced watch query features such as incremental updates and differential results, see [Live Queries / Watch Queries](/client-sdks/watch-queries).
PowerSync runs queries asynchronously on a background pool of workers. It configures SQLite write-ahead logging (WAL) so that one writer and multiple readers can operate in parallel.
## Configure Logging
```js theme={null}
import { createConsoleLogger, LogLevels } from '@powersync/node';
// Create a logger with trace minimum level to see all log messages
// Available levels: trace, debug, info, warn, error
const logger = createConsoleLogger({ minLevel: LogLevels.trace });
```
Enable verbose output in the developer tools for detailed logs.
## Additional Usage Examples
For more usage examples including accessing connection status, monitoring sync progress, and waiting for initial sync, see the [Usage Examples](/client-sdks/usage-examples) page.
## ORM Support
See [JavaScript ORM Support](/client-sdks/orms/js/overview) for details.
## Troubleshooting
See [Troubleshooting](/debugging/troubleshooting) for pointers to debug common issues.
## Supported Platforms
See [Supported Platforms -> Node.js SDK](/resources/supported-platforms#node-js).
## Upgrading the SDK
Run the following command in your project folder:
```bash theme={null}
npm upgrade @powersync/node
```
```bash theme={null}
yarn upgrade @powersync/node
```
```bash theme={null}
pnpm upgrade @powersync/node
```
## Encryption and Custom SQLite Drivers
The SDK has an optional dependency on `better-sqlite3` which is used as the default SQLite
driver for that package.
Because that dependency is optional, it can be replaced or removed to customize how SQLite
gets loaded. This section lists common options.
### Encryption
To encrypt databases managed by the PowerSync SDK for Node.js, replace the `better-sqlite3`
dependency with the [`better-sqlite3-multiple-ciphers`](https://www.npmjs.com/package/better-sqlite3-multiple-ciphers) fork.
That package has the same API as `better-sqlite3` while bundling [SQLite3MultipleCiphers](https://github.com/utelle/SQLite3MultipleCiphers)
instead of upstream SQLite.
The [node example](https://github.com/powersync-ja/powersync-js/tree/main/demos/example-node) in the PowerSync
repository can use both `better-sqlite3` and `better-sqlite3-multiple-ciphers` and may be a useful example here.
Because PowerSync attempts to dynamically load `better-sqlite3` at runtime, using a different package
requires patching the database worker. To do that, create a file (say `database.worker.js`) with the following
contents:
```Typescript theme={null}
// This worker uses bindings to sqlite3 multiple ciphers instead of the original better-sqlite3 worker.
import Database from 'better-sqlite3-multiple-ciphers';
import { startPowerSyncWorker } from '@powersync/node/worker.js';
async function resolveBetterSqlite3() {
return Database;
}
startPowerSyncWorker({ loadBetterSqlite3: resolveBetterSqlite3 });
```
When opening the database, instruct PowerSync to use the custom worker.
Also use the `initializeConnection` option to install an encryption key:
```Typescript theme={null}
const encryptionKey = 'todo: generate encryption key and store it safely';
const db = new PowerSyncDatabase({
schema: AppSchema,
database: {
dbFilename: 'app.db',
openWorker: (_, options) => {
return new Worker(new URL('./database.worker.js', import.meta.url), options);
},
initializeConnection: async (db) => {
if (encryptionKey.length) {
const escapedKey = encryptionKey.replaceAll("'", "''");
await db.execute(`pragma key = '${escapedKey}'`);
}
// Make sure the database is readable, this fails early if the key is wrong.
await db.execute('pragma user_version');
}
},
logger
});
```
If you're using a custom compilation toolchain, for instance because you're compiling from TypeScript
or are applying a bundler to your project, loading workers may require additional configuration on that
toolchain.
### `node:sqlite`
Recent versions of Node.js contain an [experimental SQLite API](https://nodejs.org/api/sqlite.html).
Using the built-in SQLite API can reduce code size and external native dependencies. To enable it,
remove your dependency on `better-sqlite3` and configure PowerSync to use the built-in API:
```JavaScript theme={null}
const database = new PowerSyncDatabase({
schema: AppSchema,
database: {
dbFilename: 'app.db',
dbLocation: directory,
// Use node:sqlite instead of better-sqlite3
implementation: { type: 'node:sqlite' }
}
});
```
PowerSync has stability issues with this API. Use it for testing only.
# Node.js SDK API Reference
Source: https://docs.powersync.com/client-sdks/reference/node-api
Full API reference for the PowerSync Node.js SDK.
# React Native & Expo SDK
Source: https://docs.powersync.com/client-sdks/reference/react-native-and-expo
Use PowerSync in React Native (& Expo) apps.
```text Build with AI icon="sparkles" wrap theme={null}
Install the PowerSync Agent Skills with: npx skills add powersync-ja/agent-skills. Then follow the skills to onboard this project to PowerSync using the React Native & Expo SDK.
```
This SDK is distributed via NPM
Refer to packages/react-native in the powersync-js repo on GitHub
Full API reference for the SDK
Gallery of example projects/demo apps built with React Native and PowerSync.
Changelog for the SDK
## SDK Features
* **Real-time streaming of database changes**: Changes made by one user are instantly streamed to all other users with access to that data. This keeps clients automatically in sync without manual polling or refresh logic.
* **Direct access to a local SQLite database**: Data is stored locally, so apps can read and write instantly without network calls. This enables offline support and faster user interactions.
* **Asynchronous background execution**: The SDK performs database operations in the background to avoid blocking the application’s main thread. This means that apps stay responsive, even during heavy data activity.
* **Query subscriptions for live updates**: The SDK supports query subscriptions that automatically push real-time updates to client applications as data changes, keeping your UI reactive and up to date.
* **Automatic schema management**: PowerSync syncs schemaless data and applies a client-defined schema using SQLite views. This architecture means that PowerSync SDKs handle schema changes without explicit migrations on the client side.
## Using Hooks
The `@powersync/react` package provides React hooks for PowerSync. See [React Hooks](/client-sdks/frameworks/react) for examples.
```bash theme={null}
npx expo install @powersync/react-native @op-engineering/op-sqlite
```
```bash theme={null}
yarn expo add @powersync/react-native @op-engineering/op-sqlite
```
```
pnpm expo install @powersync/react-native @op-engineering/op-sqlite
```
`@op-engineering/op-sqlite` is the SQLite library used by the PowerSync SDK on React Native.
It needs to be installed separately because React Native only considers direct dependencies for autolinking.
The PowerSync React Native SDK currently supports version 1.17.0 or later of `@op-engineering/op-sqlite`.
**Using Expo Go?** Our native database adapter is not compatible with Expo Go's sandbox environment. To run PowerSync with Expo Go install our JavaScript-based adapter `@powersync/adapter-sql-js` instead. See details [here](/client-sdks/frameworks/expo-go-support).
**Polyfills and additional notes:**
* For async iterator support with watched queries, additional polyfills are required. See the [Babel plugins section](https://www.npmjs.com/package/@powersync/react-native#babel-plugins-watched-queries) in the README.
* We recommend adding this [metro config](https://github.com/powersync-ja/powersync-js/tree/main/packages/react-native#metro-config-optional) to avoid build issues.
## Getting Started
**Prerequisites:** Before you start, connect your source database to the PowerSync Service and deploy Sync Streams. These are steps 1-4 in the [Setup Guide](/intro/setup-guide).
### 1. Define the Client-Side Schema
The client-side schema defines the tables and columns of the SQLite database that the PowerSync client SDK manages and that your app reads from and writes to. It is usually derived from your backend database schema and your [Sync Streams](/sync/streams/overview), and it can also include [local-only tables](/client-sdks/advanced/local-only-usage). You apply the schema when you instantiate the database in the next step.
Schema migrations are not required. The SDK syncs schemaless data and applies the schema to that data with SQLite views. The exception is [raw tables](/client-sdks/advanced/raw-tables), which you create and migrate yourself.
**Generate schema automatically**
In the [PowerSync Dashboard](https://dashboard.powersync.com/), select your project and instance and click the **Connect** button in the top bar to generate the client-side schema in your preferred language. The schema is generated from your Sync Streams. The [CLI](/tools/cli) offers the same function.
The generated schema does not include an `id` column. The client SDK creates an `id` column of type `text` automatically, so you do not need to declare it. See [Client ID](/sync/advanced/client-id) for details.
The available column types are `text`, `integer`, and `real`. These should match the values produced by your Sync Streams. If a value does not match, it is cast automatically. For details on how source database types map to SQLite types, see [Types](/sync/types).
**Example:**
```typescript powersync/AppSchema.ts theme={null}
import { column, Schema, Table } from '@powersync/react-native';
const lists = new Table({
created_at: column.text,
name: column.text,
owner_id: column.text
});
const todos = new Table(
{
list_id: column.text,
created_at: column.text,
completed_at: column.text,
description: column.text,
created_by: column.text,
completed_by: column.text,
completed: column.integer
},
{ indexes: { list: ['list_id'] } }
);
export const AppSchema = new Schema({
todos,
lists
});
// For types
export type Database = (typeof AppSchema)['types'];
export type TodoRecord = Database['todos'];
// OR:
// export type Todo = RowType;
export type ListRecord = Database['lists'];
```
You do not need to declare an `id` column. PowerSync creates it automatically.
### 2. Instantiate the PowerSync Database
Next, instantiate the PowerSync database. PowerSync streams changes from your backend source database into the client-side SQLite database, based on your Sync Streams. Your app reads from and writes to this local database whether the user is online or offline.
**Example:**
```typescript powersync/system.ts theme={null}
import { PowerSyncDatabase } from '@powersync/react-native';
import { AppSchema } from './Schema';
export const powersync = new PowerSyncDatabase({
// The schema you defined in the previous step
schema: AppSchema,
database: {
// Filename for the SQLite database — it's important to only instantiate one instance per file.
// For other database options see,
// https://powersync-ja.github.io/powersync-js/web-sdk/globals#sqlopenoptions
dbFilename: 'powersync.db'
}
});
```
After you instantiate the PowerSync database, call the [connect()](https://powersync-ja.github.io/powersync-js/common/interfaces/CommonPowerSyncDatabase#connect) method to sync data with your backend.
This section assumes that you use PowerSync to sync your backend source database with SQLite in your app. To manage a local SQLite database without sync, instantiate the PowerSync database without calling `connect()` and see the [Local-Only](/client-sdks/advanced/local-only-usage) guide.
```typescript powersync/system.ts theme={null}
import { Connector } from './Connector';
export const setupPowerSync = async () => {
// Uses the backend connector that you create in the next step
const connector = new Connector();
powersync.connect(connector);
};
```
### 3. Integrate with Your Backend
The backend connector connects the PowerSync client SDK to your application backend. The SDK uses it to:
1. Get an auth token to connect to the PowerSync instance.
2. Upload client-side writes to your backend API. The SDK places every write to the SQLite database in an upload queue and uploads the queue to your backend when the user is connected. Your backend then applies the changes to the source database.
The connector must implement two methods:
1. [PowerSyncBackendConnector.fetchCredentials](https://github.com/powersync-ja/powersync-js/blob/ed5bb49b5a1dc579050304fab847feb8d09b45c7/packages/common/src/client/connection/PowerSyncBackendConnector.ts#L16) - The SDK calls this method to get authentication credentials. It caches the credentials and calls the method again only when needed, for example on the first connection or when the token is near expiry. See [When `fetchCredentials()` is Called](/configuration/app-backend/client-side-integration#when-fetchcredentials-is-called) for details and [Authentication Setup](/configuration/auth/overview) for how to generate credentials.
2. [PowerSyncBackendConnector.uploadData](https://github.com/powersync-ja/powersync-js/blob/ed5bb49b5a1dc579050304fab847feb8d09b45c7/packages/common/src/client/connection/PowerSyncBackendConnector.ts#L24) - The SDK calls this method whenever it has client-side writes to upload to your backend API. Implement how those writes are processed and uploaded. See [When `uploadData()` is Called](/configuration/app-backend/client-side-integration#when-uploaddata-is-called) for triggers, throttling, and retry behavior, and [Writing Client Changes](/handling-writes/writing-client-changes) for the app backend implementation.
**Example:**
```typescript powersync/Connector.ts theme={null}
import { type PowerSyncBackendConnector, type CommonPowerSyncDatabase, UpdateType } from "@powersync/react-native"
export class Connector implements PowerSyncBackendConnector {
/**
* Implement fetchCredentials to obtain a JWT from your authentication service.
* See https://docs.powersync.com/configuration/auth/custom
*/
async fetchCredentials() {
return {
// The PowerSync instance URL or self-hosted endpoint
endpoint: 'https://xxxxxx.powersync.journeyapps.com',
/**
* To get started quickly, use a development token, see:
* Authentication Setup https://docs.powersync.com/configuration/auth/development-tokens) to get up and running quickly
*/
token: 'An authentication token'
};
}
/**
* Implement uploadData to send local changes to your backend service.
* You can omit this method if you only want to sync data from the database to the client
* See example implementation here: https://docs.powersync.com/client-sdks/usage-examples#send-changes-in-local-data-to-your-backend-service
*/
async uploadData(database: CommonPowerSyncDatabase) {
/**
* For batched crud transactions, use data.getCrudBatch(n);
* https://powersync-ja.github.io/powersync-js/common/interfaces/CommonPowerSyncDatabase#getcrudbatch
*/
const transaction = await database.getNextCrudTransaction();
if (!transaction) {
return;
}
for (const op of transaction.crud) {
// The data that needs to be changed in the remote db
const record = { ...op.opData, id: op.id };
switch (op.op) {
case UpdateType.PUT:
// TODO: Instruct your backend API to CREATE a record
break;
case UpdateType.PATCH:
// TODO: Instruct your backend API to PATCH a record
break;
case UpdateType.DELETE:
//TODO: Instruct your backend API to DELETE a record
break;
}
}
// Completes the transaction and moves onto the next one
await transaction.complete();
}
}
```
### 4. Subscribe to Sync Streams
Streams defined with `auto_subscribe: true` start syncing as soon as the client connects. For all other streams, your app must subscribe before their data downloads. The basic pattern is: subscribe to a stream, wait for its data to sync, then unsubscribe when the data is no longer needed.
```js theme={null}
// Subscribe to a stream with parameters
const sub = await db.syncStream('list_todos', { list_id: 'abc123' }).subscribe();
// Wait for the initial data to sync
await sub.waitForFirstSync();
// The stream's rows are now in the local SQLite database.
// TODO: Read the todos for this list with a local query.
// When the data is no longer needed
sub.unsubscribe();
```
If you use React, the `useQuery` hook accepts a `streams` option and the `useSyncStream` hook manages a subscription for you. See [Framework Integrations](/sync/streams/client-usage#framework-integrations).
After you unsubscribe, the synced data stays in the local database for the stream's time-to-live (TTL), which is 24 hours by default. If the app subscribes again within that time, the data is already available. See [Client-Side Usage](/sync/streams/client-usage) for framework hooks, per-subscription sync status, custom TTLs, priority overrides, and connection parameters.
## Using PowerSync: CRUD Functions
Once the PowerSync database is connected and your streams have synced, the data is in the local SQLite database.
The most commonly used CRUD functions to interact with your SQLite data are:
* [PowerSyncDatabase.get](#fetching-a-single-item) - get (`SELECT`) a single row from a table.
* [PowerSyncDatabase.getAll](#querying-items-powersync-getall) - get (`SELECT`) a set of rows from a table.
* [PowerSyncDatabase.watch](#watching-queries-powersync-watch) - execute a read query every time a dependent table changes.
* [PowerSyncDatabase.execute](#mutations-powersync-execute) - execute a write (`INSERT`/`UPDATE`/`DELETE`) query.
### Fetching a Single Item
The [get](https://powersync-ja.github.io/powersync-js/common/interfaces/CommonPowerSyncDatabase#get) method executes a read-only (`SELECT`) query and returns a single result. It throws an exception if no result is found. Use [getOptional](https://powersync-ja.github.io/powersync-js/common/interfaces/CommonPowerSyncDatabase#getoptional) to return a single optional result (returns `null` if no result is found).
```js TodoItemWidget.jsx theme={null}
import { Text } from 'react-native';
import { powersync } from "../powersync/system";
export const TodoItemWidget = ({id}) => {
const [todoItem, setTodoItem] = React.useState([]);
const [error, setError] = React.useState([]);
React.useEffect(() => {
// .get returns the first item of the result. Throws an exception if no result is found.
powersync.get('SELECT * from todos WHERE id = ?', [id])
.then(setTodoItem)
.catch(ex => setError(ex.message))
}, []);
return {error || todoItem.description}
}
```
### Querying Items (`PowerSync.getAll`)
The [getAll](https://powersync-ja.github.io/powersync-js/common/interfaces/CommonPowerSyncDatabase#getall) method returns a set of rows from a table.
```js ListsWidget.jsx theme={null}
import { FlatList, Text} from 'react-native';
import { powersync } from "../powersync/system";
export const ListsWidget = () => {
const [lists, setLists] = React.useState([]);
React.useEffect(() => {
powersync.getAll('SELECT * from lists').then(setLists)
}, []);
return ( ({key: list.id, ...list}))}
renderItem={({item}) => {item.name} }
/>)
}
```
### Watching Queries (`PowerSync.watch`)
The [watch](https://powersync-ja.github.io/powersync-js/common/interfaces/CommonPowerSyncDatabase#watch) method executes a read query whenever a change to a dependent table is made. It can be used with an `AsyncGenerator`, or with a callback.
```javascript theme={null}
async function* pendingLists(): AsyncIterable {
for await (const result of db.watch(
`SELECT * FROM lists WHERE state = ?`,
['pending']
)) {
yield result.rows?._array ?? [];
}
}
```
```javascript theme={null}
const pendingLists = (onResult: (lists: any[]) => void): void => {
db.watch(
'SELECT * FROM lists WHERE state = ?',
['pending'],
{
onResult: (result: any) => {
onResult(result.rows?._array ?? []);
}
}
);
}
```
For advanced watch query features such as incremental updates and differential results, see [Live Queries / Watch Queries](/client-sdks/watch-queries).
### Mutations (`PowerSync.execute`)
The [execute](https://powersync-ja.github.io/powersync-js/common/interfaces/CommonPowerSyncDatabase#execute) method can be used for executing single SQLite write statements.
```js ListsWidget.jsx theme={null}
import { Alert, Button, FlatList, Text, View } from 'react-native';
import { powersync } from "../powersync/system";
export const ListsWidget = () => {
// Populate lists with one of methods listed above
const [lists, setLists] = React.useState([]);
return (
({key: list.id, ...list}))}
renderItem={({item}) => (
{item.name}
{
try {
await powersync.execute(`DELETE FROM lists WHERE id = ?`, [item.id])
// Watched queries should automatically reload after mutation
} catch (ex) {
Alert.alert('Error', ex.message)
}
}}
/>
)}
/>
{
try {
await powersync.execute('INSERT INTO lists (id, created_at, name, owner_id) VALUES (uuid(), datetime(), ?, ?) RETURNING *', [
'A list name',
"[The user's uuid]"
])
// Watched queries should automatically reload after mutation
} catch (ex) {
Alert.alert('Error', ex.message)
}
}}
/>
)
}
```
When using the default client-side [JSON-based view system](/architecture/client-architecture#client-side-schema-and-sqlite-database-structure), writes are applied to a view, with triggers writing to the underlying table. Because of this, `result.rowsAffected` from `db.execute()` can be `0` even when an `UPDATE` or `DELETE` succeeds.
When you need to confirm whether a mutation changed any rows, add a `RETURNING` clause and check the returned rows:
```js theme={null}
const result = await db.execute(
'UPDATE tasks SET deleted_at = ? WHERE id = ? AND deleted_at IS NULL RETURNING id',
[now, id]
);
const wasUpdated = (result.rows?.length ?? 0) > 0;
```
If you need direct table writes, use [raw tables](/client-sdks/advanced/raw-tables).
## Configure Logging
```js theme={null}
import { createConsoleLogger, LogLevels } from '@powersync/react-native';
// Create a logger with trace minimum level to see all log messages
// Available levels: trace, debug, info, warn, error
const logger = createConsoleLogger({ minLevel: LogLevels.trace });
```
Enable verbose output in the developer tools for detailed logs.
## Additional Usage Examples
For more usage examples including accessing connection status, monitoring sync progress, and waiting for initial sync, see the [Usage Examples](/client-sdks/usage-examples) page.
## ORM Support
See [JavaScript ORM Support](/client-sdks/orms/js/overview) for details.
## Troubleshooting
See [Troubleshooting](/debugging/troubleshooting) for pointers to debug common issues.
## Supported Platforms
See [Supported Platforms -> React Native SDK](/resources/supported-platforms#react-native).
## Upgrading the SDK
Run the following command in your project folder:
```bash theme={null}
npm upgrade @powersync/react-native
```
```bash theme={null}
yarn upgrade @powersync/react-native
```
```bash theme={null}
pnpm upgrade @powersync/react-native
```
## Developer Notes
### Connection Methods
This SDK supports two methods for streaming sync commands:
1. **HTTP Streaming (Default)**
* This is the default streaming method for Expo apps.
2. **WebSocket (Fallback)**
* This implementation uses RSocket for reactive socket streams.
* This is necessary on React Native without Expo, as `fetch()` in React Native does not support streaming responses.
* To customize window sizes for flow control and back-pressure, set `fetchStrategy` to `Buffered` (default) or `Sequential`.
By default, the `PowerSyncDatabase.connect()` method uses HTTP on Expo apps (with `expo/fetch` as an HTTP client) and WebSockets
for plain React Native apps. You can optionally specify the `connectionMethod` to override this:
```js theme={null}
// Default (HTTP on Expo apps, WebSocket on plain React Native).
powerSync.connect(connector);
// HTTP Streaming, throws if no HTTP client with support for streaming responses is available.
powerSync.connect(connector, { connectionMethod: SyncStreamConnectionMethod.HTTP });
// Explicitly use Web Sockets
powerSync.connect(connector, { connectionMethod: SyncStreamConnectionMethod.WEB_SOCKET });
```
### Android: Flipper Network Plugin for HTTP Streams
If you connect to PowerSync with HTTP streams (the default), Android requires additional configuration. React Native does not support streams natively, so the SDK uses the [polyfills](/client-sdks/reference/react-native-and-expo#installation) from the installation steps. An open Flipper [issue](https://github.com/facebook/flipper/issues/2495) prevents the Flipper network plugin from firing stream events. [Disable the plugin](https://stackoverflow.com/questions/69235694/react-native-cant-connect-to-sse-in-android/69235695#69235695) so that HTTP streams work.
**If you are using Java (Expo \< 50):**
Uncomment the following in `android/app/src/debug/java/com//ReactNativeFlipper.java`:
```java theme={null}
// NetworkFlipperPlugin networkFlipperPlugin = new NetworkFlipperPlugin();
// NetworkingModule.setCustomClientBuilder(
// new NetworkingModule.CustomClientBuilder() {
// @Override
// public void apply(OkHttpClient.Builder builder) {
// builder.addNetworkInterceptor(new FlipperOkhttpInterceptor(networkFlipperPlugin));
// }
// });
// client.addPlugin(networkFlipperPlugin);
```
Disable the dev client network inspector in `android/gradle.properties`:
```properties theme={null}
# Disable the network inspector
EX_DEV_CLIENT_NETWORK_INSPECTOR=false
```
**If you are using Kotlin (Expo > 50):**
Comment out the following in `onCreate` in `android/app/src/main/java/com//example/MainApplication.kt`:
```kotlin theme={null}
// if (BuildConfig.DEBUG) {
// ReactNativeFlipper.initializeFlipper(this, reactNativeHost.reactInstanceManager)
// }
```
### Development on iOS Simulator
If you test offline mode on an iOS simulator by disabling the host machine's internet connection, the simulator stays offline even after you restore the connection. This affects all network requests in the app.
# React Native SDK API Reference
Source: https://docs.powersync.com/client-sdks/reference/react-native-api
Full API reference for the PowerSync React Native SDK.
# Rust SDK (alpha)
Source: https://docs.powersync.com/client-sdks/reference/rust
Use PowerSync in Rust apps.
This SDK is currently in [**alpha**](/resources/feature-status), intended for external testing and public feedback. Expect breaking changes and instability as development continues.
```text Build with AI icon="sparkles" wrap theme={null}
Install the PowerSync Agent Skills with: npx skills add powersync-ja/agent-skills. Then follow the skills to onboard this project to PowerSync using the Rust SDK.
```
The SDK is distributed via crates.io
Refer to the `powersync-native` repo on GitHub
Full API reference for the SDK
Gallery of example projects/demo apps built with Rust and PowerSync
Changelog for the SDK
## SDK Features
* **Real-time streaming of database changes**: Changes made by one user are instantly streamed to all other users with access to that data. This keeps clients automatically in sync without manual polling or refresh logic.
* **Direct access to a local SQLite database**: Data is stored locally, so apps can read and write instantly without network calls. This enables offline support and faster user interactions.
* **Asynchronous background execution**: The SDK performs database operations in the background to avoid blocking the application’s main thread. This means that apps stay responsive, even during heavy data activity.
* **Query subscriptions for live updates**: The SDK supports query subscriptions that automatically push real-time updates to client applications as data changes, keeping your UI reactive and up to date.
* **Automatic schema management**: PowerSync syncs schemaless data and applies a client-defined schema using SQLite views. This architecture means that PowerSync SDKs handle schema changes without explicit migrations on the client side.
## Installation
Add the [PowerSync SDK](https://central.sonatype.com/artifact/com.powersync/core) to your project by adding the following to your `Cargo.toml` file:
```shell theme={null}
cargo add powersync
```
## Getting Started
**Prerequisites:** Before you start, connect your source database to the PowerSync Service and deploy Sync Streams. These are steps 1-4 in the [Setup Guide](/intro/setup-guide).
### 1. Define the Client-Side Schema
The client-side schema defines the tables and columns of the SQLite database that the PowerSync client SDK manages and that your app reads from and writes to. It is usually derived from your backend database schema and your [Sync Streams](/sync/streams/overview), and it can also include [local-only tables](/client-sdks/advanced/local-only-usage). You apply the schema when you instantiate the database in the next step.
Schema migrations are not required. The SDK syncs schemaless data and applies the schema to that data with SQLite views. The exception is [raw tables](/client-sdks/advanced/raw-tables), which you create and migrate yourself.
**Generate schema automatically**
In the [PowerSync Dashboard](https://dashboard.powersync.com/), select your project and instance and click the **Connect** button in the top bar to generate the client-side schema in your preferred language. The schema is generated from your Sync Streams. The [CLI](/tools/cli) offers the same function.
The generated schema does not include an `id` column. The client SDK creates an `id` column of type `text` automatically, so you do not need to declare it. See [Client ID](/sync/advanced/client-id) for details.
The available column types are `text`, `integer`, and `real`. These should match the values produced by your Sync Streams. If a value does not match, it is cast automatically. For details on how source database types map to SQLite types, see [Types](/sync/types).
**Example:**
```Rust src/schema.rs theme={null}
use powersync::schema::{Column, Schema, Table};
pub fn app_schema() -> Schema {
let mut schema = Schema::default();
let todos = Table::create(
"todos",
vec![
Column::text("list_id"),
Column::text("created_at"),
Column::text("completed_at"),
Column::text("description"),
Column::integer("completed"),
Column::text("created_by"),
Column::text("completed_by"),
],
|_| {},
);
let lists = Table::create(
"lists",
vec![
Column::text("created_at"),
Column::text("name"),
Column::text("owner_id"),
],
|_| {},
);
schema.tables.push(todos);
schema.tables.push(lists);
schema
}
```
You do not need to declare an `id` column. PowerSync creates it automatically.
### 2. Instantiate the PowerSync Database
Next, instantiate the PowerSync database. PowerSync streams changes from your backend source database into the client-side SQLite database, based on your Sync Streams. Your app reads from and writes to this local database whether the user is online or offline.
#### Process Setup
The SDK statically links the [PowerSync SQLite core extension](https://github.com/powersync-ja/powersync-sqlite-core). You must register the extension for the process before you use the SDK. We recommend calling the registration utility early in `main()`:
```Rust lib/main.rs theme={null}
use powersync::env::PowerSyncEnvironment;
mod schema;
fn main() {
PowerSyncEnvironment::powersync_auto_extension()
.expect("could not load PowerSync core extension");
// TODO: Start database and your app
}
```
#### Database Setup
You can configure the Rust SDK with different asynchronous runtimes and HTTP clients to connect to the PowerSync Service.
These dependencies can be configured through the [`PowerSyncEnvironment`](https://docs.rs/powersync/latest/powersync/env/struct.PowerSyncEnvironment.html)
struct, which wraps:
1. An HTTP client (implement the `powersync::http::HttpClient` trait). When enabling the `reqwest` feature on
the `powersync` crate, that trait is implemented for `reqwest::Client`.
2. An asynchronous pool giving out leases to SQLite connections.
3. A timer implementation allowing the sync client to implement delayed retries on connection errors.
This is typically provided by async runtimes like Tokio.
To configure PowerSync, begin by configuring a connection pool:
Use `ConnectionPool::open` to open a database file with multiple connections configured with WAL mode:
```Rust theme={null}
use powersync::{ConnectionPool, error::PowerSyncError};
use powersync::env::PowerSyncEnvironment;
fn open_pool() -> Result{
ConnectionPool::open("database.db")
}
```
```Rust theme={null}
use powersync::ConnectionPool;
use powersync::env::PowerSyncEnvironment;
use powersync::error::PowerSyncError;
use rusqlite::Connection;
fn open_pool() -> Result {
let connection = Connection::open_in_memory()?;
Ok(ConnectionPool::single_connection(connection))
}
```
Next, create a database and start asynchronous tasks used by the sync client when connecting.
To be compatible with different executors, the SDK uses a model based on long-lived actors instead of
spawning tasks dynamically. All asynchronous processes are exposed through `PowerSyncDatabase::async_tasks()`.
Spawn these tasks before connecting.
Ensure you depend on `powersync` with the `tokio` feature enabled.
```Rust theme={null}
#[tokio::main]
async fn main() {
PowerSyncEnvironment::powersync_auto_extension()
.expect("could not load PowerSync core extension");
let pool = open_pool().expect("open pool");
let env = PowerSyncEnvironment::custom(
reqwest::Client::new(),
pool,
PowerSyncEnvironment::tokio_timer(),
);
let db = PowerSyncDatabase::new(env, schema::app_schema());
db.async_tasks().spawn_with_tokio();
}
```
Ensure you depend on `powersync` with the `smol` feature enabled.
```Rust theme={null}
async fn start_app() {
let pool = open_pool().expect("open pool");
let env = PowerSyncEnvironment::custom(
reqwest::Client::new(),
pool,
// Use the async_io crate to implement timers in PowerSync
PowerSyncEnvironment::async_io_timer(),
);
let db = PowerSyncDatabase::new(env, schema::app_schema());
// TODO: Use a custom multi-threaded executor instead of the default
let tasks = db.async_tasks().spawn_with(smol::spawn);
for task in tasks {
// The task will automatically stop once the database is dropped, but we
// want to keep it running until then.
task.detach();
}
}
fn main() {
PowerSyncEnvironment::powersync_auto_extension()
.expect("could not load PowerSync core extension");
smol::block_on(start_app());
}
```
PowerSync is executor-agnostic and supports all async Rust runtimes. You need to provide:
1. A future that delays execution by scheduling its waker through a timer.
2. A way to spawn futures as a task that is polled independently.
PowerSync uses the [`Timer`](https://docs.rs/powersync/latest/powersync/env/trait.Timer.html)
trait for timers. Install your timer by creating a `PowerSyncEnvironment` with `PowerSyncEnvironment::custom`
and passing your implementation.
Spawning tasks is only necessary once after opening the database. All tasks necessary for the sync
client are exposed through `PowerSyncDatabase::async_tasks`. You can spawn these by providing
a function turning these futures into independent tasks via `AsyncDatabaseTasks::spawn_with`.
Finally, instruct PowerSync to sync data from your backend:
```Rust theme={null}
// MyBackendConnector is defined in the next step...
db.connect(SyncOptions::new(MyBackendConnector {
db: db.clone(),
})).await;
```
This section assumes that you use PowerSync to sync your backend source database with SQLite in your app. To manage a local SQLite database without sync, instantiate the PowerSync database without calling `connect()` and see the [Local-Only](/client-sdks/advanced/local-only-usage) guide.
### 3. Integrate with Your Backend
The backend connector connects the PowerSync client SDK to your application backend. The SDK uses it to:
1. Get an auth token to connect to the PowerSync instance.
2. Upload client-side writes to your backend API. The SDK places every write to the SQLite database in an upload queue and uploads the queue to your backend when the user is connected. Your backend then applies the changes to the source database.
The connector must implement two methods:
1. `fetch_credentials` - The SDK calls this method to get authentication credentials. It caches the credentials and calls the method again only when needed, for example on the first connection or when the token is near expiry. See [When `fetchCredentials()` is Called](/configuration/app-backend/client-side-integration#when-fetchcredentials-is-called) for details and [Authentication Setup](/configuration/auth/overview) for how to generate credentials.
2. `upload_data` - The SDK calls this method whenever it has client-side writes to upload to your backend API. Implement how those writes are processed and uploaded. See [When `uploadData()` is Called](/configuration/app-backend/client-side-integration#when-uploaddata-is-called) for triggers, throttling, and retry behavior, and [Writing Client Changes](/handling-writes/writing-client-changes) for the app backend implementation.
**Example:**
```Rust theme={null}
struct MyBackendConnector {
db: PowerSyncDatabase,
}
#[async_trait]
impl BackendConnector for MyBackendConnector {
async fn fetch_credentials(&self) -> Result {
// implement fetchCredentials to obtain the necessary credentials to connect to your backend
// See an example implementation in https://github.com/powersync-ja/powersync-native/blob/508193b0822b8dad1a534a16462e2fcd36a9ac68/examples/egui_todolist/src/database.rs#L119-L133
Ok(PowerSyncCredentials {
endpoint: "[Your PowerSync instance URL or self-hosted endpoint]".to_string(),
// Use a development token (see Authentication Setup https://docs.powersync.com/configuration/auth/development-tokens) to get up and running quickly
token: "An authentication token".to_string(),
})
}
async fn upload_data(&self) -> Result<(), PowerSyncError> {
// Implement uploadData to send local changes to your backend service
// You can omit this method if you only want to sync data from the server to the client
// See an example implementation under Usage Examples (sub-page)
// See https://docs.powersync.com/handling-writes/writing-client-changes for considerations.
let mut local_writes = self.db.crud_transactions();
while let Some(tx) = local_writes.try_next().await? {
todo!("Inspect tx.crud for local writes that need to be uploaded to your backend");
tx.complete().await?;
}
Ok(())
}
}
```
### 4. Subscribe to Sync Streams
Streams defined with `auto_subscribe: true` start syncing as soon as the client connects. For all other streams, your app must subscribe before their data downloads. The basic pattern is: subscribe to a stream, wait for its data to sync, then unsubscribe when the data is no longer needed.
```Rust theme={null}
use serde_json::json;
// Subscribe to a stream with parameters
let sub = db
.sync_stream("list_todos", Some(&json!({ "list_id": "abc123" })))
.subscribe()
.await?;
// Wait for the initial data to sync
sub.wait_for_first_sync().await;
// The stream's rows are now in the local SQLite database.
// TODO: Read the todos for this list with a local query.
// When the data is no longer needed.
// Dropping the subscription handle has the same effect.
sub.unsubscribe();
```
After you unsubscribe, the synced data stays in the local database for the stream's time-to-live (TTL), which is 24 hours by default. If the app subscribes again within that time, the data is already available. See [Client-Side Usage](/sync/streams/client-usage) for framework hooks, per-subscription sync status, custom TTLs, priority overrides, and connection parameters.
## Using PowerSync: CRUD Functions
Once the PowerSync database is connected and your streams have synced, the data is in the local SQLite database.
The most commonly used CRUD functions to interact with your SQLite data are:
* [`reader`](#reads) - run statements that read from the database.
* [`watch_statement`](#watching-queries) - execute a read query every time a dependent table changes.
* [`writer`](#mutations) - write to the database.
### Reads
To obtain a connection suitable for reads, call and await `PowerSyncDatabase::reader()`.
The returned lease can be used as a `rusqlite::Connection` to run queries.
```Rust theme={null}
async fn find(db: &PowerSyncDatabase, id: &str) -> Result<(), PowerSyncError> {
let reader = db.reader().await?;
let mut stmt = reader.prepare("SELECT * FROM lists WHERE id = ?")?;
let mut rows = stmt.query(params![id])?;
while let Some(row) = rows.next()? {
let id: String = row.get("id")?;
let name: String = row.get("name")?;
println!("Found todo list: {id}, {name}");
}
Ok(())
}
```
### Watching Queries
The `watch_statement` method executes a read query whenever a change to a dependent table is made.
```Rust theme={null}
async fn watch_pending_lists(db: &PowerSyncDatabase) -> Result<(), PowerSyncError> {
let stream = db.watch_statement(
"SELECT * FROM lists WHERE state = ?".to_string(),
params!["pending"],
|stmt, params| {
let mut rows = stmt.query(params)?;
let mut mapped = vec![];
while let Some(row) = rows.next()? {
mapped.push(() /* TODO: Read row into list struct */)
}
Ok(mapped)
},
);
let mut stream = pin!(stream);
// Note: The stream is never-ending, so you probably want to call this in an independent async
// task.
while let Some(event) = stream.try_next().await? {
// Update UI to display rows
}
Ok(())
}
```
### Mutations
Triggers capture local writes automatically. To obtain a connection suitable for writes, call and await
`PowerSyncDatabase::writer()`. Then call `execute` on the writer to run `INSERT`, `UPDATE`, or `DELETE` statements:
```Rust theme={null}
async fn insert_customer(
db: &PowerSyncDatabase,
name: &str,
email: &str,
) -> Result<(), PowerSyncError> {
let writer = db.writer().await?;
writer.execute(
"INSERT INTO customers (id, name, email) VALUES (uuid(), ?, ?)",
params![name, email],
)?;
Ok(())
}
```
For transactions, use the [`transaction`](https://docs.rs/rusqlite/latest/rusqlite/struct.Connection.html#method.transaction)
method from `rusqlite` on the writer.
## Configure Logging
The Rust SDK uses the `log` crate internally, so you can configure it with any backend, for example
`env_logger`:
```Rust theme={null}
fn main() {
env_logger::init();
// ...
}
```
## Additional Usage Examples
For more usage examples including accessing connection status, monitoring sync progress, and waiting for initial sync, see the [Usage Examples](/client-sdks/usage-examples) page.
## ORM / SQL Library Support
The Rust SDK does not currently support any higher-level SQL libraries, but we're investigating
support for Diesel and sqlx.
[Contact us](/resources/contact-us) if you are interested in these or other integrations.
## Troubleshooting
See [Troubleshooting](/debugging/troubleshooting) for pointers to debug common issues.
## Supported Platforms
See [Supported Platforms -> Rust SDK](/resources/supported-platforms#rust).
## Upgrading the SDK
To update the PowerSync SDK, run `cargo update powersync` or manually update to the
[latest version](https://crates.io/crates/powersync/versions).
# Swift SDK
Source: https://docs.powersync.com/client-sdks/reference/swift
Use PowerSync in Swift apps.
```text Build with AI icon="sparkles" wrap theme={null}
Install the PowerSync Agent Skills with: npx skills add powersync-ja/agent-skills. Then follow the skills to onboard this project to PowerSync using the Swift SDK.
```
Refer to the `powersync-swift` repo on GitHub
Full API reference for the SDK
Gallery of example projects/demo apps built with PowerSync and Swift
Changelog for the SDK
Up to v1.13, the Swift SDK wrapped a PowerSync Kotlin XCFramework behind Swift protocols.
From v1.14, the SDK has no Kotlin dependency. It is implemented natively in Swift and uses our [Rust core](https://github.com/powersync-ja/powersync-sqlite-core) for the sync protocol and SQLite extension.
## SDK Features
* **Real-time streaming of database changes**: Changes made by one user are instantly streamed to all other users with access to that data. This keeps clients automatically in sync without manual polling or refresh logic.
* **Direct access to a local SQLite database**: Data is stored locally, so apps can read and write instantly without network calls. This enables offline support and faster user interactions.
* **Asynchronous background execution**: The SDK performs database operations in the background to avoid blocking the application’s main thread. This means that apps stay responsive, even during heavy data activity.
* **Query subscriptions for live updates**: The SDK supports query subscriptions that automatically push real-time updates to client applications as data changes, keeping your UI reactive and up to date.
* **Automatic schema management**: PowerSync syncs schemaless data and applies a client-defined schema using SQLite views. This architecture means that PowerSync SDKs handle schema changes without explicit migrations on the client side.
## Installation
You can add the PowerSync Swift package to your project using either `Package.swift` or Xcode:
```swift theme={null}
let package = Package(
//...
dependencies: [
//...
.package(
url: "https://github.com/powersync-ja/powersync-swift",
exact: ""
),
],
targets: [
.target(
name: "YourTargetName",
dependencies: [
.product(
name: "PowerSync",
package: "powersync-swift"
)
]
)
]
)
```
1. Follow [this guide](https://developer.apple.com/documentation/xcode/adding-package-dependencies-to-your-app#Add-a-package-dependency) to add a package to your project.
2. Use `https://github.com/powersync-ja/powersync-swift.git` as the URL
3. Include the exact version (e.g., `1.0.x`)
## Getting Started
**Prerequisites:** Before you start, connect your source database to the PowerSync Service and deploy Sync Streams. These are steps 1-4 in the [Setup Guide](/intro/setup-guide).
### 1. Define the Client-Side Schema
The client-side schema defines the tables and columns of the SQLite database that the PowerSync client SDK manages and that your app reads from and writes to. It is usually derived from your backend database schema and your [Sync Streams](/sync/streams/overview), and it can also include [local-only tables](/client-sdks/advanced/local-only-usage). You apply the schema when you instantiate the database in the next step.
Schema migrations are not required. The SDK syncs schemaless data and applies the schema to that data with SQLite views. The exception is [raw tables](/client-sdks/advanced/raw-tables), which you create and migrate yourself.
**Generate schema automatically**
In the [PowerSync Dashboard](https://dashboard.powersync.com/), select your project and instance and click the **Connect** button in the top bar to generate the client-side schema in your preferred language. The schema is generated from your Sync Streams. The [CLI](/tools/cli) offers the same function.
The generated schema does not include an `id` column. The client SDK creates an `id` column of type `text` automatically, so you do not need to declare it. See [Client ID](/sync/advanced/client-id) for details.
The available column types are `text`, `integer`, and `real`. These should match the values produced by your Sync Streams. If a value does not match, it is cast automatically. For details on how source database types map to SQLite types, see [Types](/sync/types).
**Example:**
```swift theme={null}
import Foundation
import PowerSync
let LISTS_TABLE = "lists"
let TODOS_TABLE = "todos"
let lists = Table(
name: LISTS_TABLE,
columns: [
// ID column is automatically included
.text("name"),
.text("created_at"),
.text("owner_id")
]
)
let todos = Table(
name: TODOS_TABLE,
// ID column is automatically included
columns: [
.text("list_id"),
.text("photo_id"),
.text("description"),
// 0 or 1 to represent false or true
.integer("completed"),
.text("created_at"),
.text("completed_at"),
.text("created_by"),
.text("completed_by")
],
indexes: [
Index(
name: "list_id",
columns: [
IndexedColumn.ascending("list_id")
]
)
]
)
let AppSchema = Schema(lists, todos)
```
You do not need to declare an `id` column. PowerSync creates it automatically.
### 2. Instantiate the PowerSync Database
Next, instantiate the PowerSync database. PowerSync streams changes from your backend source database into the client-side SQLite database, based on your Sync Streams. Your app reads from and writes to this local database whether the user is online or offline.
**Example:**
```swift theme={null}
let schema = AppSchema // Comes from the AppSchema defined above
let db = PowerSyncDatabase(
schema: schema,
dbFilename: "powersync-swift.sqlite"
)
```
### 3. Integrate with Your Backend
The backend connector connects the PowerSync client SDK to your application backend. The SDK uses it to:
1. Get an auth token to connect to the PowerSync instance.
2. Upload client-side writes to your backend API. The SDK places every write to the SQLite database in an upload queue and uploads the queue to your backend when the user is connected. Your backend then applies the changes to the source database.
The connector must implement two methods:
1. `PowerSyncBackendConnectorProtocol.fetchCredentials` - The SDK calls this method to get authentication credentials. It caches the credentials and calls the method again only when needed, for example on the first connection or when the token is near expiry. See [When `fetchCredentials()` is Called](/configuration/app-backend/client-side-integration#when-fetchcredentials-is-called) for details and [Authentication Setup](/configuration/auth/overview) for how to generate credentials.
2. `PowerSyncBackendConnectorProtocol.uploadData` - The SDK calls this method whenever it has client-side writes to upload to your backend API. Implement how those writes are processed and uploaded. See [When `uploadData()` is Called](/configuration/app-backend/client-side-integration#when-uploaddata-is-called) for triggers, throttling, and retry behavior, and [Writing Client Changes](/handling-writes/writing-client-changes) for the app backend implementation.
**Example:**
```swift theme={null}
import PowerSync
final class MyConnector: PowerSyncBackendConnectorProtocol {
func fetchCredentials() async throws -> PowerSyncCredentials? {
// implement fetchCredentials to obtain the necessary credentials to connect to your backend
// See an example implementation in https://github.com/powersync-ja/powersync-swift/blob/main/Demo/PowerSyncExample/PowerSync/SupabaseConnector.swift
return PowerSyncCredentials(
endpoint: "Your PowerSync instance URL or self-hosted endpoint",
// Use a development token (see Authentication Setup https://docs.powersync.com/configuration/auth/development-tokens)
// to get up and running quickly
token: "An authentication token"
)
}
func uploadData(database: PowerSyncDatabaseProtocol) async throws {
// Implement uploadData to send local changes to your backend service
// You can omit this method if you only want to sync data from the server to the client
// See an example implementation under Usage Examples (sub-page)
// See https://docs.powersync.com/handling-writes/writing-client-changes for considerations.
}
}
```
Connect the PowerSync database to sync data with your backend:
```swift theme={null}
let connector = MyConnector()
try await db.connect(connector: connector)
```
This section assumes that you use PowerSync to sync your backend source database with SQLite in your app. To manage a local SQLite database without sync, instantiate the PowerSync database without calling `connect()` and see the [Local-Only](/client-sdks/advanced/local-only-usage) guide.
### 4. Subscribe to Sync Streams
Streams defined with `auto_subscribe: true` start syncing as soon as the client connects. For all other streams, your app must subscribe before their data downloads. The basic pattern is: subscribe to a stream, wait for its data to sync, then unsubscribe when the data is no longer needed.
```swift theme={null}
// Subscribe to a stream with parameters
let sub = try await db.syncStream(name: "list_todos", params: ["list_id": .string("abc123")]).subscribe()
// Wait for the initial data to sync
try await sub.waitForFirstSync()
// The stream's rows are now in the local SQLite database.
// TODO: Read the todos for this list with a local query.
// When the data is no longer needed
try await sub.unsubscribe()
```
After you unsubscribe, the synced data stays in the local database for the stream's time-to-live (TTL), which is 24 hours by default. If the app subscribes again within that time, the data is already available. See [Client-Side Usage](/sync/streams/client-usage) for framework hooks, per-subscription sync status, custom TTLs, priority overrides, and connection parameters.
## Using PowerSync: CRUD Functions
Once the PowerSync database is connected and your streams have synced, the data is in the local SQLite database.
The most commonly used CRUD functions to interact with your SQLite data are:
* [PowerSyncDatabase.get](/client-sdks/reference/swift#fetching-a-single-item-powersync-get-/-powersync-getoptional) - get (`SELECT`) a single row from a table.
* [PowerSyncDatabase.getOptional](/client-sdks/reference/swift#fetching-a-single-item-powersync-get-/-powersync-getoptional) - get (`SELECT`) a single row from a table and return `nil` if not found.
* [PowerSyncDatabase.getAll](/client-sdks/reference/swift#querying-items-powersync-getall) - get (`SELECT`) a set of rows from a table.
* [PowerSyncDatabase.watch](/client-sdks/reference/swift#watching-queries-powersync-watch) - execute a read query every time a dependent table changes.
* [PowerSyncDatabase.execute](/client-sdks/reference/swift#mutations-powersync-execute) - execute a write (`INSERT`/`UPDATE`/`DELETE`) query.
### Fetching a Single Item (PowerSync.get / PowerSync.getOptional)
The `get` method executes a read-only (SELECT) query and returns a single result. It throws an exception if no result is found. Use `getOptional` to return a single optional result (returns `nil` if no result is found).
```swift theme={null}
// Find a list by ID
func getList(_ id: String) async throws -> ListContent {
try await db.get(
sql: "SELECT * FROM \(LISTS_TABLE) WHERE id = ?",
parameters: [id],
mapper: { cursor in
ListContent(
id: try cursor.getString(name: "id")!,
name: try cursor.getString(name: "name")!,
createdAt: try cursor.getString(name: "created_at")!,
ownerId: try cursor.getString(name: "owner_id")!
)
}
)
}
```
### Querying Items (PowerSync.getAll)
The `getAll` method executes a read-only (SELECT) query and returns a set of rows.
```swift theme={null}
// Get all lists
func getLists() async throws -> [ListContent] {
try await db.getAll(
sql: "SELECT * FROM \(LISTS_TABLE)",
parameters: [],
mapper: { cursor in
ListContent(
id: try cursor.getString(name: "id")!,
name: try cursor.getString(name: "name")!,
createdAt: try cursor.getString(name: "created_at")!,
ownerId: try cursor.getString(name: "owner_id")!
)
}
)
}
```
### Watching Queries (PowerSync.watch)
The `watch` method executes a read query whenever a change to a dependent table is made.
```swift theme={null}
func watchPendingLists() throws -> AsyncThrowingStream<[ListContent], Error> {
try db.watch(
sql: "SELECT * FROM lists WHERE state = ?",
parameters: ["pending"],
) { cursor in
try ListContent(
id: cursor.getString(name: "id"),
name: cursor.getString(name: "name"),
)
}
}
```
### Mutations (PowerSync.execute)
The `execute` method executes a write query (INSERT, UPDATE, DELETE) and returns the results (if any).
```swift theme={null}
func insertTodo(_ todo: NewTodo, _ listId: String) async throws {
try await db.execute(
sql: "INSERT INTO \(TODOS_TABLE) (id, created_at, created_by, description, list_id, completed) VALUES (uuid(), datetime(), ?, ?, ?, ?)",
parameters: [connector.currentUserID, todo.description, listId, todo.isComplete]
)
}
func updateTodo(_ todo: Todo) async throws {
try await db.execute(
sql: "UPDATE \(TODOS_TABLE) SET description = ?, completed = ?, completed_at = datetime(), completed_by = ? WHERE id = ?",
parameters: [todo.description, todo.isComplete, connector.currentUserID, todo.id]
)
}
func deleteTodo(id: String) async throws {
try await db.writeTransaction(callback: { transaction in
_ = try transaction.execute(
sql: "DELETE FROM \(TODOS_TABLE) WHERE id = ?",
parameters: [id]
)
})
}
```
## Configure Logging
You can supply your own logger. It must conform to the [LoggerProtocol](https://powersync-ja.github.io/powersync-swift/documentation/powersync/loggerprotocol):
```swift theme={null}
let logger = DefaultLogger(minSeverity: .debug)
let db = PowerSyncDatabase(
schema: schema,
dbFilename: "powersync-swift.sqlite",
logger: logger
)
```
The `DefaultLogger` supports the following severity levels: `.debug`, `.info`, `.warn`, `.error`.
## App Groups and App Extensions
By default, the Swift SDK stores databases in `applicationSupportDirectory`. If you want to share local databases
between multiple apps in an app group, provide an absolute path (starting with `/`) as `dbFilename` when calling
`PowerSyncDatabase`.
The path should point towards a file in the app group's [shared container](https://developer.apple.com/documentation/xcode/configuring-app-groups#Access-an-app-groups-shared-container).
Support for app groups and extensions was added in version 1.15, and is experimental.
With app groups and extensions, multiple processes might access the database at the same time. SQLite uses shared memory
and file locking to make this safe, but there are still concerns that aren't relevant when the database is only accessed
from a single context:
1. Concurrent writes to the database are not allowed. Within a process, the PowerSync SDK uses Swift actors to schedule
writes. Outside of SQLite using file locks, there is no additional coordination for multi-process access.
* Consider using [`PRAGMA busy_timeout`](https://sqlite.org/pragma.html#pragma_busy_timeout) to make SQLite wait longer.
* Be ready to handle `SQLITE_BUSY` errors if multiple processes attempt to write to the database.
2. Do not call `connect()` in more than one process: If two processes attempt to connect to the PowerSync Service for
the same database, this wastes resources and can also lead to concurrency issues. If you share databases with extensions
or app groups, ensure there is only one context responsible for connecting.
3. Avoid multiple versions of the Swift SDK: While the SQLite file format is stable and databases can safely be accessed
from multiple processes using their own SQLite version, the PowerSync SDK is not designed with this in mind.
In particular, opening the same database file from multiple processes can cause PowerSync-internal migrations to get
reverted.
This is not a concern for app extensions released as part of the same bundle, but something to be aware of for
app groups.
## Custom HTTP Clients and Headers
PowerSync uses a streaming HTTP response to connect to the PowerSync Service. By default, the Swift SDK uses `URLSession.shared`
to run HTTP requests.
You can provide a custom `URLSession` as a sync option. This can be used to add custom HTTP headers, for example:
```swift theme={null}
let config = URLSessionConfiguration.ephemeral
config.httpAdditionalHeaders = ["x-my-custom-header": "example"]
let session = URLSession(configuration: config)
try await db.connect(
connector: connector,
options: ConnectOptions(
clientConfiguration: SyncClientConfiguration(
urlSession: session
)
)
)
```
For more information, see Apple's documentation on [`URLSessionConfiguration`](https://developer.apple.com/documentation/foundation/urlsessionconfiguration).
## Additional Usage Examples
For more usage examples including accessing connection status, monitoring sync progress, and waiting for initial sync, see the [Usage Examples](/client-sdks/usage-examples) page.
## ORM Support
PowerSync officially supports the [GRDB](/client-sdks/orms/swift/grdb) library for Swift.
Additionally, [Asier G. Morato](https://github.com/asiergmorato) contributed a [Swift Data integration](https://github.com/powersync-community/swift-data)
for PowerSync, allowing Swift Data models to be persisted and synced through a PowerSync database.
The integration is community-owned and not officially supported by PowerSync.
## Troubleshooting
See [Troubleshooting](/debugging/troubleshooting) for pointers to debug common issues.
## Supported Platforms
See [Supported Platforms -> Swift SDK](/resources/supported-platforms#swift).
## Upgrading the SDK
Update the version number in `Package.swift` or via Xcode Package Dependencies as documented in the installation instructions: [Installation](/client-sdks/reference/swift#installation).
**Upgrading to v1.14**: This release removes the internal Kotlin dependency and re-implements the SQLite connection pool logic natively in Swift. The public API is unchanged, but the internals are a full rewrite. We recommend testing queries in your app after upgrading to this version.
# Swift SDK API Reference
Source: https://docs.powersync.com/client-sdks/reference/swift-api
Full API reference for the PowerSync Swift SDK.
# Tauri SDK (alpha)
Source: https://docs.powersync.com/client-sdks/reference/tauri
Use PowerSync in Tauri apps.
```text Build with AI icon="sparkles" wrap theme={null}
Install the PowerSync Agent Skills with: npx skills add powersync-ja/agent-skills. Then follow the skills to onboard this project to PowerSync using the Tauri SDK.
```
The JavaScript part of the SDK is distributed via NPM
The Rust part of the SDK is available via crates.io
Refer to `packages/tauri` in the `powersync-js` repo on GitHub
Full API reference for the SDK
Gallery of example projects/demo apps built with Tauri and PowerSync
Changelog for the SDK
This SDK is currently in [**alpha**](/resources/feature-status), intended for external testing and public feedback. While it exposes the same stable JavaScript APIs as other PowerSync JavaScript SDKs, the underlying [Rust SDK](/client-sdks/reference/rust) and the IPC protocol between the JavaScript and Rust packages are subject to change. Expect breaking changes and instability as development continues.
There are [known limitations](#limitations) currently.
## SDK Features
* **Real-time streaming of database changes**: Changes made by one user are instantly streamed to all other users with access to that data. This keeps clients automatically in sync without manual polling or refresh logic.
* **Direct access to a local SQLite database**: Data is stored locally, so apps can read and write instantly without network calls. This enables offline support and faster user interactions.
* **Asynchronous background execution**: The SDK performs database operations in the background to avoid blocking the application’s main thread. This means that apps stay responsive, even during heavy data activity.
* **Query subscriptions for live updates**: The SDK supports query subscriptions that automatically push real-time updates to client applications as data changes, keeping your UI reactive and up to date.
* **Automatic schema management**: PowerSync syncs schemaless data and applies a client-defined schema using SQLite views. This architecture means that PowerSync SDKs handle schema changes without explicit migrations on the client side.
## Installation
Add the [PowerSync Tauri NPM package](https://www.npmjs.com/package/@powersync/tauri-plugin) to your project:
```bash theme={null}
npm install @powersync/tauri-plugin
```
```bash theme={null}
yarn add @powersync/tauri-plugin
```
```bash theme={null}
pnpm install @powersync/tauri-plugin
```
Like all trusted PowerSync packages, the Tauri plugin is only available through the `@powersync/` scope on npm.
The PowerSync Tauri plugin **cannot** be installed with the `tauri add` command.
Additionally, add the Tauri plugin crate to your Rust app (in the `src-tauri` directory):
```bash theme={null}
cargo add tauri-plugin-powersync
```
In your `lib.rs`, ensure the plugin is loaded:
```diff theme={null}
pub fn run() {
tauri::Builder::default()
.invoke_handler(tauri::generate_handler![connect])
+ .plugin(tauri_plugin_powersync::init())
.run(tauri::generate_context!())
.expect("error while running tauri application");
}
```
In `src-tauri/capabilities/default.json`, ensure `powersync:default` is listed under `permissions` to make PowerSync APIs available to JavaScript.
## Getting Started
**Prerequisites:** Before you start, connect your source database to the PowerSync Service and deploy Sync Streams. These are steps 1-4 in the [Setup Guide](/intro/setup-guide).
### 1. Define the Client-Side Schema
The client-side schema defines the tables and columns of the SQLite database that the PowerSync client SDK manages and that your app reads from and writes to. It is usually derived from your backend database schema and your [Sync Streams](/sync/streams/overview), and it can also include [local-only tables](/client-sdks/advanced/local-only-usage). You apply the schema when you instantiate the database in the next step.
Schema migrations are not required. The SDK syncs schemaless data and applies the schema to that data with SQLite views. The exception is [raw tables](/client-sdks/advanced/raw-tables), which you create and migrate yourself.
**Generate schema automatically**
In the [PowerSync Dashboard](https://dashboard.powersync.com/), select your project and instance and click the **Connect** button in the top bar to generate the client-side schema in your preferred language. The schema is generated from your Sync Streams. The [CLI](/tools/cli) offers the same function.
The generated schema does not include an `id` column. The client SDK creates an `id` column of type `text` automatically, so you do not need to declare it. See [Client ID](/sync/advanced/client-id) for details.
The available column types are `text`, `integer`, and `real`. These should match the values produced by your Sync Streams. If a value does not match, it is cast automatically. For details on how source database types map to SQLite types, see [Types](/sync/types).
The Tauri SDK inherits the same APIs from the JavaScript Web SDK via `@powersync/common`, with a few exceptions detailed below. See the [Web SDK schema definition section](/client-sdks/reference/javascript-web#1-define-the-client-side-schema) for more advanced examples.
**Example:**
```js theme={null}
// AppSchema.ts
import { column, Schema, Table } from '@powersync/common';
const lists = new Table({
created_at: column.text,
name: column.text,
owner_id: column.text
});
const todos = new Table(
{
list_id: column.text,
created_at: column.text,
completed_at: column.text,
description: column.text,
created_by: column.text,
completed_by: column.text,
completed: column.integer
},
{ indexes: { list: ['list_id'] } }
);
export const AppSchema = new Schema({
todos,
lists
});
// For types
export type Database = (typeof AppSchema)['types'];
export type TodoRecord = Database['todos'];
// OR:
// export type Todo = RowType;
export type ListRecord = Database['lists'];
```
You do not need to declare an `id` column. PowerSync creates it automatically.
### 2. Instantiate the PowerSync Database
Next, instantiate the PowerSync database. PowerSync streams changes from your backend source database into the client-side SQLite database, based on your Sync Streams. Your app reads from and writes to this local database whether the user is online or offline.
**Example:**
```js theme={null}
import { PowerSyncTauriDatabase } from '@powersync/tauri-plugin';
import { appDataDir } from '@tauri-apps/api/path';
import { AppSchema } from './AppSchema';
export const db = new PowerSyncTauriDatabase({
// The schema you defined in the previous step
schema: AppSchema,
database: {
// Filename for the SQLite database.
dbFilename: 'powersync.db',
// An optional directory to store the database file in.
dbLocationAsync: appDataDir,
}
});
```
Finally, instruct PowerSync to sync data from your backend. For Tauri apps, the sync client
must be driven from Rust code.
To do that, add a new Tauri command to your Rust application:
```Rust theme={null}
use tauri_plugin_powersync::PowerSyncExt;
#[tauri::command]
async fn connect(
app: AppHandle,
handle: usize,
) -> tauri_plugin_powersync::Result<()> {
let ps = app.powersync();
let database = ps.database_from_javascript_handle(handle)?;
// MyBackendConnector is defined in the next step...
let options = SyncOptions::new(MyRustConnector {
db: database.clone(),
});
database.connect(options).await;
Ok(())
}
```
Register the command by including it in the invoke handler with
`tauri::generate_handler![connect]`.
You can then invoke that command from your JavaScript code to connect:
```TypeScript theme={null}
import { PowerSyncTauriDatabase } from '@powersync/tauri-plugin';
import { invoke } from '@tauri-apps/api/core';
async function connect(db: PowerSyncTauriDatabase) {
await db.init();
const handle = db.rustHandle;
await invoke('connect', { handle });
}
```
### 3. Integrate with Your Backend
The backend connector connects the PowerSync client SDK to your application backend. The SDK uses it to:
1. Get an auth token to connect to the PowerSync instance.
2. Upload client-side writes to your backend API. The SDK places every write to the SQLite database in an upload queue and uploads the queue to your backend when the user is connected. Your backend then applies the changes to the source database.
At the moment, backend connectors for the Tauri SDK must be implemented in Rust.
This allows the connector to be used across multiple windows and even when no JavaScript is running.
[Let us know](/resources/contact-us) if you want to implement a backend connector in JavaScript.
The connector must implement two methods:
1. `fetch_credentials` - The SDK calls this method to get authentication credentials. It caches the credentials and calls the method again only when needed, for example on the first connection or when the token is near expiry. See [When `fetchCredentials()` is Called](/configuration/app-backend/client-side-integration#when-fetchcredentials-is-called) for details and [Authentication Setup](/configuration/auth/overview) for how to generate credentials.
2. `upload_data` - The SDK calls this method whenever it has client-side writes to upload to your backend API. Implement how those writes are processed and uploaded. See [When `uploadData()` is Called](/configuration/app-backend/client-side-integration#when-uploaddata-is-called) for triggers, throttling, and retry behavior, and [Writing Client Changes](/handling-writes/writing-client-changes) for the app backend implementation.
**Example:**
```Rust theme={null}
struct MyBackendConnector {
db: PowerSyncDatabase,
}
#[async_trait]
impl BackendConnector for MyBackendConnector {
async fn fetch_credentials(&self) -> Result {
// implement fetchCredentials to obtain the necessary credentials to connect to your backend
// See an example implementation in https://github.com/powersync-ja/powersync-native/blob/508193b0822b8dad1a534a16462e2fcd36a9ac68/examples/egui_todolist/src/database.rs#L119-L133
Ok(PowerSyncCredentials {
endpoint: "[Your PowerSync instance URL or self-hosted endpoint]".to_string(),
// Use a development token (see Authentication Setup https://docs.powersync.com/configuration/auth/development-tokens) to get up and running quickly
token: "An authentication token".to_string(),
})
}
async fn upload_data(&self) -> Result<(), PowerSyncError> {
// Implement uploadData to send local changes to your backend service
// You can omit this method if you only want to sync data from the server to the client
// See an example implementation under Usage Examples (sub-page)
// See https://docs.powersync.com/handling-writes/writing-client-changes for considerations.
let mut local_writes = self.db.crud_transactions();
while let Some(tx) = local_writes.try_next().await? {
todo!("Inspect tx.crud for local writes that need to be uploaded to your backend");
tx.complete().await?;
}
Ok(())
}
}
```
### 4. Subscribe to Sync Streams
Streams defined with `auto_subscribe: true` start syncing as soon as the client connects. For all other streams, your app must subscribe before their data downloads. The basic pattern is: subscribe to a stream, wait for its data to sync, then unsubscribe when the data is no longer needed.
```js theme={null}
// Subscribe to a stream with parameters
const sub = await db.syncStream('list_todos', { list_id: 'abc123' }).subscribe();
// Wait for the initial data to sync
await sub.waitForFirstSync();
// The stream's rows are now in the local SQLite database.
// TODO: Read the todos for this list with a local query.
// When the data is no longer needed
sub.unsubscribe();
```
After you unsubscribe, the synced data stays in the local database for the stream's time-to-live (TTL), which is 24 hours by default. If the app subscribes again within that time, the data is already available. See [Client-Side Usage](/sync/streams/client-usage) for framework hooks, per-subscription sync status, custom TTLs, priority overrides, and connection parameters.
## Using PowerSync: CRUD Functions
Once the PowerSync database is connected and your streams have synced, the data is in the local SQLite database.
**All CRUD examples from the JavaScript Web SDK apply**: The Tauri SDK exposes the same JavaScript database API as the Web SDK. See the [JavaScript Web SDK CRUD functions section](/client-sdks/reference/javascript-web#using-powersync-crud-functions) for examples of `get`, `getAll`, `watch`, `execute`, `writeTransaction`, incremental watch updates, and differential results.
The most commonly used CRUD functions to interact with your SQLite data are:
* [PowerSyncDatabase.get](/client-sdks/reference/javascript-web#fetching-a-single-item) - get (`SELECT`) a single row from a table.
* [PowerSyncDatabase.getAll](/client-sdks/reference/javascript-web#querying-items-powersync-getall) - get (`SELECT`) a set of rows from a table.
* [PowerSyncDatabase.watch](/client-sdks/reference/javascript-web#watching-queries-powersync-watch) - execute a read query every time a dependent table changes.
* [PowerSyncDatabase.execute](/client-sdks/reference/javascript-web#mutations-powersync-execute-powersync-writetransaction) - execute a write (`INSERT`/`UPDATE`/`DELETE`) query.
### Fetching a Single Item
The [get](https://powersync-ja.github.io/powersync-js/common/interfaces/CommonPowerSyncDatabase#get) method executes a read-only (SELECT) query and returns a single result. It throws an exception if no result is found. Use [getOptional](https://powersync-ja.github.io/powersync-js/common/interfaces/CommonPowerSyncDatabase#getoptional) to return a single optional result (returns `null` if no result is found).
```js theme={null}
// Find a list item by ID
export const findList = async (id) => {
const result = await db.get('SELECT * FROM lists WHERE id = ?', [id]);
return result;
}
```
### Querying Items (PowerSync.getAll)
The [getAll](https://powersync-ja.github.io/powersync-js/common/interfaces/CommonPowerSyncDatabase#getall) method returns a set of rows from a table.
```js theme={null}
// Get all lists
export const getLists = async () => {
const results = await db.getAll('SELECT * FROM lists');
return results;
}
```
### Watching Queries (PowerSync.watch)
The [watch](https://powersync-ja.github.io/powersync-js/common/interfaces/CommonPowerSyncDatabase#watch) method executes a read query whenever a change to a dependent table is made.
```javascript theme={null}
async function* pendingLists(): AsyncIterable {
for await (const result of db.watch(
`SELECT * FROM lists WHERE state = ?`,
['pending']
)) {
yield result.rows?._array ?? [];
}
}
```
```javascript theme={null}
const pendingLists = (onResult: (lists: any[]) => void): void => {
db.watch(
'SELECT * FROM lists WHERE state = ?',
['pending'],
{
onResult: (result: any) => {
onResult(result.rows?._array ?? []);
}
}
);
}
```
For advanced watch query features such as incremental updates and differential results, see [Live Queries / Watch Queries](/client-sdks/watch-queries).
### Mutations (PowerSync.execute, PowerSync.writeTransaction)
The [execute](https://powersync-ja.github.io/powersync-js/common/interfaces/CommonPowerSyncDatabase#execute) method can be used for executing single SQLite write statements.
```js theme={null}
// Delete a list by ID
export const deleteList = async (id) => {
await db.execute('DELETE FROM lists WHERE id = ?', [id]);
};
// OR: delete the list and its todos in one transaction
export const deleteListWithTodos = async (id) => {
await db.writeTransaction(async (tx) => {
await tx.execute('DELETE FROM todos WHERE list_id = ?', [id]);
await tx.execute('DELETE FROM lists WHERE id = ?', [id]);
});
};
```
When using the default client-side [JSON-based view system](/architecture/client-architecture#client-side-schema-and-sqlite-database-structure), writes are applied to a view, with triggers writing to the underlying table. Because of this, `result.rowsAffected` from `db.execute()` can be `0` even when an `UPDATE` or `DELETE` succeeds.
When you need to confirm whether a mutation changed any rows, add a `RETURNING` clause and check the returned rows:
```js theme={null}
const result = await db.execute(
'UPDATE tasks SET deleted_at = ? WHERE id = ? AND deleted_at IS NULL RETURNING id',
[now, id]
);
const wasUpdated = (result.rows?.length ?? 0) > 0;
```
If you need direct table writes, use [raw tables](/client-sdks/advanced/raw-tables).
## Configure Logging
The Rust SDK uses the `log` crate which can be enabled using the [Tauri logging plugin](https://v2.tauri.app/plugin/logging/).
## Sharing a Database with Rust Code
A `PowerSyncTauriDatabase` instance can be shared with Rust code:
1. Call and await `.init()` on the database to ensure the database has been opened.
2. Use the `.rustHandle` getter, which returns a number representing the database instance.
3. This handle can now be passed as an argument to one of your Tauri commands.
In Rust SDK, you can use `AppHandle::powersync()` and
`PowerSync::database_from_javascript_handle` to inspect the database from Rust.
See the [Rust SDK guide](/client-sdks/reference/rust) for more information.
Connection locks and table updates are automatically shared between Rust and JavaScript code.
## Limitations
* The Rust SDK, which is used to implement the Tauri plugin, does not provide `lastSyncedAt`, `hasSynced` and `priorityStatusEntries` on `SyncStatus`.
Use the status on individual Sync Streams through `SyncStatus.forStream` instead.
* Connecting to the PowerSync Service is only possible from Rust. Calling `connect()` from JavaScript
will throw.
## Additional Usage Examples
For more usage examples including accessing connection status, monitoring sync progress, and waiting for initial sync, see the [Usage Examples](/client-sdks/usage-examples) page.
## ORM Support
See [JavaScript ORM Support](/client-sdks/orms/js/overview) for details.
## Troubleshooting
See [Troubleshooting](/debugging/troubleshooting) for pointers to debug common issues.
# Usage Examples
Source: https://docs.powersync.com/client-sdks/usage-examples
Code examples and common patterns for the PowerSync Client SDKs.
## Using Transactions to Group Changes
Read and write transactions present a context where multiple changes can be made then finally committed to the DB or rolled back. This ensures that either all the changes get persisted, or no change is made to the DB (in the case of a rollback or exception).
The [writeTransaction(callback)](https://pub.dev/documentation/powersync/latest/sqlite_async/SqliteConnection/writeTransaction.html) method combines all writes into a single transaction, only committing to persistent storage once.
```dart theme={null}
deleteList(SqliteDatabase db, String id) async {
await db.writeTransaction((tx) async {
// Delete the main list
await tx.execute('DELETE FROM lists WHERE id = ?', [id]);
// Delete any children of the list
await tx.execute('DELETE FROM todos WHERE list_id = ?', [id]);
});
}
```
Also see [readTransaction(callback)](https://pub.dev/documentation/powersync/latest/sqlite_async/SqliteConnection/readTransaction.html)
Read and write transactions present a context where multiple changes can be made then finally committed to the DB or rolled back. This ensures that either all the changes get persisted, or no change is made to the DB (in the case of a rollback or exception).
[PowerSyncDatabase.writeTransaction(callback)](https://powersync-ja.github.io/powersync-js/common/interfaces/CommonPowerSyncDatabase#writetransaction) automatically commits changes after the transaction callback is completed if [`tx.rollback()`](https://github.com/powersync-ja/powersync-js/blob/ed5bb49b5a1dc579050304fab847feb8d09b45c7/packages/common/src/db/DBAdapter.ts#L53) has not explicitly been called. If an exception is thrown in the callback then changes are automatically rolled back.
```js theme={null}
// ListsWidget.jsx
import {Alert, Button, FlatList, Text, View} from 'react-native';
export const ListsWidget = () => {
// Populate lists with one of methods listed above
const [lists, setLists] = React.useState([]);
return (
({key: list.id, ...list}))}
renderItem={({item}) => (
{item.name}
{
try {
await PowerSync.writeTransaction(async (tx) => {
// Delete the main list
await tx.execute(`DELETE FROM lists WHERE id = ?`, [item.id]);
// Delete any children of the list
await tx.execute(`DELETE FROM todos WHERE list_id = ?`, [item.id]);
// Transactions are automatically committed at the end of execution
// Transactions are automatically rolled back if an exception occurred
})
// Watched queries should automatically reload after mutation
} catch (ex) {
Alert.alert('Error', ex.message)
}
}}
/>
)}
/>
{
try {
await PowerSync.execute('INSERT INTO lists (id, created_at, name, owner_id) VALUES (uuid(), datetime(), ?, ?) RETURNING *', [
'A list name',
"[The user's uuid]"
])
// Watched queries should automatically reload after mutation
} catch (ex) {
Alert.alert('Error', ex.message)
}
}}
/>
)
}
```
Also see [PowerSyncDatabase.readTransaction(callback)](https://powersync-ja.github.io/powersync-js/common/interfaces/CommonPowerSyncDatabase#readtransaction).
Read and write transactions present a context where multiple changes can be made then finally committed to the DB or rolled back. This ensures that either all the changes get persisted, or no change is made to the DB (in the case of a rollback or exception).
[PowerSyncDatabase.writeTransaction(callback)](https://powersync-ja.github.io/powersync-js/common/interfaces/CommonPowerSyncDatabase#writetransaction) automatically commits changes after the transaction callback is completed if `tx.rollback()` has not explicitly been called. If an exception is thrown in the callback then changes are automatically rolled back.
```js theme={null}
// ListsWidget.jsx
import React, { useState } from 'react';
export const ListsWidget = () => {
const [lists, setLists] = useState([]);
return (
{lists.map((list) => (
{list.name}
{
try {
await PowerSync.writeTransaction(async (tx) => {
// Delete the main list
await tx.execute(`DELETE FROM lists WHERE id = ?`, [item.id]);
// Delete any children of the list
await tx.execute(`DELETE FROM todos WHERE list_id = ?`, [item.id]);
// Transactions are automatically committed at the end of execution
// Transactions are automatically rolled back if an exception occurred
})
// Watched queries should automatically reload after mutation
} catch (ex) {
Alert.alert('Error', ex.message)
}
}}
>
Delete
))}
{
try {
await PowerSync.execute('INSERT INTO lists (id, created_at, name, owner_id) VALUES (uuid(), datetime(), ?, ?) RETURNING *', [
'A list name',
"[The user's uuid]"
])
// Watched queries should automatically reload after mutation
} catch (ex) {
Alert.alert('Error', ex.message)
}
}}
>
Create List
);
};
```
Also see [PowerSyncDatabase.readTransaction(callback)](https://powersync-ja.github.io/powersync-js/common/interfaces/CommonPowerSyncDatabase#readtransaction).
Example not yet available.
Example not yet available.
Use `writeTransaction` to group statements that can write to the database.
```kotlin theme={null}
database.writeTransaction {
database.execute(
sql = "DELETE FROM list WHERE id = ?",
parameters = listOf(listId)
)
database.execute(
sql = "DELETE FROM todos WHERE list_id = ?",
parameters = listOf(listId)
)
}
```
Read and write transactions present a context where multiple changes can be made then finally committed to the DB or rolled back. This ensures that either all the changes get persisted, or no change is made to the DB (in the case of a rollback or exception).
```swift theme={null}
// Delete a list and its todos in a transaction
func deleteList(db: PowerSyncDatabase, listId: String) async throws {
try await db.writeTransaction { tx in
try await tx.execute(sql: "DELETE FROM lists WHERE id = ?", parameters: [listId])
try await tx.execute(sql: "DELETE FROM todos WHERE list_id = ?", parameters: [listId])
}
}
```
Also see [`readTransaction`](https://powersync-ja.github.io/powersync-swift/documentation/powersync/queries/readtransaction\(callback:\)).
```cs theme={null}
using PowerSync.Common.Client;
class Demo
{
static async Task DeleteList(PowerSyncDatabase db, string listId)
{
await db.WriteTransaction(async (tx) =>
{
// Delete the main list
await tx.Execute("DELETE FROM lists WHERE id = ?", new object[] { listId });
// Delete any children of the list
await tx.Execute("DELETE FROM todos WHERE list_id = ?", new object[] { listId });
// Transactions are automatically committed at the end of execution
// Transactions are automatically rolled back if an exception occurred
});
}
}
```
Also see [`ReadTransaction`](https://powersync-ja.github.io/powersync-dotnet/api/PowerSync.Common.Client.PowerSyncDatabase.ReadTransaction.html).
Example not yet available.
## Listen for Changes in Data
Use [watch](https://pub.dev/documentation/powersync/latest/powersync/PowerSyncDatabase/watch.html) to watch for changes to the dependent tables of any SQL query.
```dart theme={null}
StreamBuilder(
stream: db.watch('SELECT * FROM lists WHERE state = ?', ['pending']),
builder: (context, snapshot) {
if (snapshot.hasData) {
// TODO: implement your own UI here based on the result set
return ...;
} else {
return const Center(child: CircularProgressIndicator());
}
},
)
```
Use [PowerSyncDatabase.watch](https://powersync-ja.github.io/powersync-js/common/interfaces/CommonPowerSyncDatabase#watch) to watch for changes in source tables.
```javascript theme={null}
async function* pendingLists(): AsyncIterable {
for await (const result of db.watch(
`SELECT * FROM lists WHERE state = ?`,
['pending']
)) {
yield result.rows?._array ?? [];
}
}
```
```javascript theme={null}
const pendingLists = (onResult: (lists: any[]) => void): void => {
db.watch(
'SELECT * FROM lists WHERE state = ?',
['pending'],
{
onResult: (result: any) => {
onResult(result.rows?._array ?? []);
}
}
);
}
```
For advanced watch query features like incremental updates and differential results, see [Live Queries / Watch Queries](/client-sdks/watch-queries).
Use [PowerSyncDatabase.watch](https://powersync-ja.github.io/powersync-js/common/interfaces/CommonPowerSyncDatabase#watch) to watch for changes in source tables.
```javascript theme={null}
async function* pendingLists(): AsyncIterable {
for await (const result of db.watch(
`SELECT * FROM lists WHERE state = ?`,
['pending']
)) {
yield result.rows?._array ?? [];
}
}
```
```javascript theme={null}
const pendingLists = (onResult: (lists: any[]) => void): void => {
db.watch(
'SELECT * FROM lists WHERE state = ?',
['pending'],
{
onResult: (result: any) => {
onResult(result.rows?._array ?? []);
}
}
);
}
```
For advanced watch query features like incremental updates and differential results, see [Live Queries / Watch Queries](/client-sdks/watch-queries).
The Capacitor SDK uses the same API as the [JavaScript Web SDK](/client-sdks/reference/javascript-web#watching-queries-powersync-watch). Use `db.watch()` to watch for changes in source tables.
```javascript theme={null}
async function* pendingLists(): AsyncIterable {
for await (const result of db.watch(
`SELECT * FROM lists WHERE state = ?`,
['pending']
)) {
yield result.rows?._array ?? [];
}
}
```
```javascript theme={null}
const pendingLists = (onResult: (lists: any[]) => void): void => {
db.watch(
'SELECT * FROM lists WHERE state = ?',
['pending'],
{
onResult: (result: any) => {
onResult(result.rows?._array ?? []);
}
}
);
}
```
For advanced watch query features like incremental updates and differential results, see [Live Queries / Watch Queries](/client-sdks/watch-queries).
Use [PowerSyncDatabase.watch](https://powersync-ja.github.io/powersync-js/common/interfaces/CommonPowerSyncDatabase#watch) to watch for changes in source tables.
```javascript theme={null}
async function* pendingLists(): AsyncIterable {
for await (const result of db.watch(
`SELECT * FROM lists WHERE state = ?`,
['pending']
)) {
yield result.rows?._array ?? [];
}
}
```
```javascript theme={null}
const pendingLists = (onResult: (lists: any[]) => void): void => {
db.watch(
'SELECT * FROM lists WHERE state = ?',
['pending'],
{
onResult: (result: any) => {
onResult(result.rows?._array ?? []);
}
}
);
}
```
For advanced watch query features like incremental updates and differential results, see [Live Queries / Watch Queries](/client-sdks/watch-queries).
Use the `watch` method to watch for changes to the dependent tables of any SQL query.
```kotlin theme={null}
fun watchPendingLists(): Flow> =
db.watch(
"SELECT * FROM lists WHERE state = ?",
listOf("pending"),
) { cursor ->
ListItem(
id = cursor.getString("id"),
name = cursor.getString("name"),
)
}
```
Use `watch` to watch for changes to the dependent tables of any SQL query.
```swift theme={null}
func watchPendingLists() throws -> AsyncThrowingStream<[ListContent], Error> {
try db.watch(
sql: "SELECT * FROM lists WHERE state = ?",
parameters: ["pending"],
) { cursor in
try ListContent(
id: cursor.getString(name: "id"),
name: cursor.getString(name: "name"),
)
}
}
```
Use `db.Watch()` to watch queries for changes. `Watch` returns an `IAsyncEnumerable` (since v0.0.11-alpha.1).
```csharp theme={null}
// Define a result type with properties matching the schema columns (some columns omitted here for brevity)
// public class ListResult { public string id; public string name; public string owner_id; ... }
// Optional cancellation token to stop watching
var cts = new CancellationTokenSource();
// Register listener synchronously on the calling thread...
var listener = db.Watch(
"SELECT * FROM lists WHERE owner_id = ?",
[ownerId],
new SQLWatchOptions { Signal = cts.Token }
);
// ...then listen to changes on another thread (or await foreach directly if already in an async context)
_ = Task.Run(async () =>
{
await foreach (var results in listener)
{
Console.WriteLine("Lists: ");
foreach (var result in results)
{
Console.WriteLine($"{result.id}: {result.name}");
}
}
}, cts.Token);
// To stop watching, cancel the token: cts.Cancel();
```
Use `watch_statement` to watch for changes to the dependent tables of any SQL query.
```Rust theme={null}
async fn watch_pending_lists(db: &PowerSyncDatabase) -> Result<(), PowerSyncError> {
let stream = db.watch_statement(
"SELECT * FROM lists WHERE state = ?".to_string(),
params!["pending"],
|stmt, params| {
let mut rows = stmt.query(params)?;
let mut mapped = vec![];
while let Some(row) = rows.next()? {
mapped.push(() /* TODO: Read row into list struct */)
}
Ok(mapped)
},
);
let mut stream = pin!(stream);
// Note: The stream is never-ending, so you probably want to call this in an independent async
// task.
while let Some(event) = stream.try_next().await? {
// Update UI to display rows
}
Ok(())
}
```
## Insert, Update, and Delete Data in the Local Database
Use [execute](https://pub.dev/documentation/powersync/latest/sqlite_async/SqliteConnection/execute.html) to run INSERT, UPDATE or DELETE queries.
```dart theme={null}
FloatingActionButton(
onPressed: () async {
await db.execute(
'INSERT INTO customers(id, name, email) VALUES(uuid(), ?, ?)',
['Fred', 'fred@example.org'],
);
},
tooltip: '+',
child: const Icon(Icons.add),
);
```
Use [PowerSyncDatabase.execute](https://powersync-ja.github.io/powersync-js/common/interfaces/CommonPowerSyncDatabase#execute) to run INSERT, UPDATE or DELETE queries.
```js theme={null}
const handleButtonClick = async () => {
await db.execute(
'INSERT INTO customers(id, name, email) VALUES(uuid(), ?, ?)',
['Fred', 'fred@example.org']
);
};
return (
+
add
);
```
Use [PowerSyncDatabase.execute](https://powersync-ja.github.io/powersync-js/common/interfaces/CommonPowerSyncDatabase#execute) to run INSERT, UPDATE or DELETE queries.
```js theme={null}
const handleButtonClick = async () => {
await db.execute(
'INSERT INTO customers(id, name, email) VALUES(uuid(), ?, ?)',
['Fred', 'fred@example.org']
);
};
return (
+
add
);
```
Example not yet available.
Example not yet available.
Use `execute` to run `INSERT`, `UPDATE` or `DELETE` queries.
```kotlin theme={null}
suspend fun updateCustomer(id: String, name: String, email: String) {
database.execute(
"UPDATE customers SET name = ? WHERE email = ?",
listOf(name, email)
)
}
```
Use `execute` to run `INSERT`, `UPDATE` or `DELETE` queries.
```swift theme={null}
// Insert a new TODO
func insertTodo(_ todo: NewTodo, _ listId: String) async throws {
try await db.execute(
sql: "INSERT INTO \(TODOS_TABLE) (id, created_at, created_by, description, list_id, completed) VALUES (uuid(), datetime(), ?, ?, ?, ?)",
parameters: [connector.currentUserID, todo.description, listId, todo.isComplete]
)
}
```
Use `Execute` to run `INSERT`, `UPDATE` or `DELETE` queries.
```cs theme={null}
// Insert a new customer
await db.Execute(
"INSERT INTO customers(id, name, email) VALUES(uuid(), ?, ?)",
new object[] { "Fred", "fred@example.org" }
);
```
Use `PowerSyncDatabase::writer` and `execute` to run INSERT, UPDATE or DELETE queries. Obtain a write connection with `db.writer().await?`, then call `writer.execute(sql, params![...])?`:
```rust theme={null}
use rusqlite::params;
async fn insert_customer(
db: &PowerSyncDatabase,
name: &str,
email: &str,
) -> Result<(), PowerSyncError> {
let writer = db.writer().await?;
writer.execute(
"INSERT INTO customers (id, name, email) VALUES (uuid(), ?, ?)",
params![name, email],
)?;
Ok(())
}
```
## Send Changes in Local Data to Your Backend Service
Override [uploadData](https://pub.dev/documentation/powersync/latest/powersync/PowerSyncBackendConnector/uploadData.html) to send local updates to your backend service.
```dart theme={null}
@override
Future uploadData(PowerSyncDatabase database) async {
final batch = await database.getCrudBatch();
if (batch == null) return;
for (var op in batch.crud) {
switch (op.op) {
case UpdateType.put:
// Send the data to your backend service
// Replace `_myApi` with your own API client or service
await _myApi.put(op.table, op.opData!);
break;
default:
// TODO: implement the other operations (patch, delete)
break;
}
}
await batch.complete();
}
```
Override [uploadData](https://github.com/powersync-ja/powersync-js/blob/ed5bb49b5a1dc579050304fab847feb8d09b45c7/packages/common/src/client/connection/PowerSyncBackendConnector.ts#L24) to send local updates to your backend service.
```js theme={null}
// Implement the uploadData method in your backend connector
async function uploadData(database) {
const batch = await database.getCrudBatch();
if (batch === null) return;
for (const op of batch.crud) {
switch (op.op) {
case 'put':
// Send the data to your backend service
// replace `_myApi` with your own API client or service
await _myApi.put(op.table, op.opData);
break;
default:
// TODO: implement the other operations (patch, delete)
break;
}
}
await batch.complete();
}
```
Override [uploadData](https://github.com/powersync-ja/powersync-js/blob/ed5bb49b5a1dc579050304fab847feb8d09b45c7/packages/common/src/client/connection/PowerSyncBackendConnector.ts#L24) to send local updates to your backend service.
```js theme={null}
// Implement the uploadData method in your backend connector
async function uploadData(database) {
const batch = await database.getCrudBatch();
if (batch === null) return;
for (const op of batch.crud) {
switch (op.op) {
case 'put':
// Send the data to your backend service
// replace `_myApi` with your own API client or service
await _myApi.put(op.table, op.opData);
break;
default:
// TODO: implement the other operations (patch, delete)
break;
}
}
await batch.complete();
}
```
Override [uploadData](https://github.com/powersync-ja/powersync-js/blob/ed5bb49b5a1dc579050304fab847feb8d09b45c7/packages/common/src/client/connection/PowerSyncBackendConnector.ts#L24) to send local updates to your backend service.
```js theme={null}
// Implement the uploadData method in your backend connector
async function uploadData(database) {
const batch = await database.getCrudBatch();
if (batch === null) return;
for (const op of batch.crud) {
switch (op.op) {
case 'put':
// Send the data to your backend service
// replace `_myApi` with your own API client or service
await _myApi.put(op.table, op.opData);
break;
default:
// TODO: implement the other operations (patch, delete)
break;
}
}
await batch.complete();
}
```
Override [uploadData](https://github.com/powersync-ja/powersync-js/blob/ed5bb49b5a1dc579050304fab847feb8d09b45c7/packages/common/src/client/connection/PowerSyncBackendConnector.ts#L24) to send local updates to your backend service.
```js theme={null}
// Implement the uploadData method in your backend connector
async function uploadData(database) {
const batch = await database.getCrudBatch();
if (batch === null) return;
for (const op of batch.crud) {
switch (op.op) {
case 'put':
// Send the data to your backend service
// replace `_myApi` with your own API client or service
await _myApi.put(op.table, op.opData);
break;
default:
// TODO: implement the other operations (patch, delete)
break;
}
}
await batch.complete();
}
```
Override `uploadData` to send local updates to your backend service. If you are using Supabase, see [SupabaseConnector.kt](https://github.com/powersync-ja/powersync-kotlin/blob/main/integrations/supabase/src/commonMain/kotlin/com/powersync/connector/supabase/SupabaseConnector.kt) for a complete implementation.
```kotlin theme={null}
/**
* This function is called whenever there is data to upload, whether the device is online or offline.
* If this call throws an error, it is retried periodically.
*/
override suspend fun uploadData(database: PowerSyncDatabase) {
val transaction = database.getNextCrudTransaction() ?: return;
var lastEntry: CrudEntry? = null;
try {
for (entry in transaction.crud) {
lastEntry = entry;
val table = supabaseClient.from(entry.table)
when (entry.op) {
UpdateType.PUT -> {
val data = entry.opData?.toMutableMap() ?: mutableMapOf()
data["id"] = entry.id
table.upsert(data)
}
UpdateType.PATCH -> {
table.update(entry.opData!!) {
filter {
eq("id", entry.id)
}
}
}
UpdateType.DELETE -> {
table.delete {
filter {
eq("id", entry.id)
}
}
}
}
}
transaction.complete(null);
} catch (e: Exception) {
println("Data upload error - retrying last entry: ${lastEntry!!}, $e")
throw e
}
}
```
Override `uploadData` to send local updates to your backend service.
```swift theme={null}
class MyConnector: PowerSyncBackendConnector {
override func uploadData(database: PowerSyncDatabaseProtocol) async throws {
guard let batch = try await database.getCrudBatch() else { return }
for entry in batch.crud {
switch entry.op {
case .put:
// Send the data to your backend service
// Replace `_myApi` with your own API client or service
// `opDataTyped` preserves SQLite types as a `[String: JsonValue]` dictionary.
// Use `opData` for the legacy `[String: String?]` representation.
try await _myApi.put(table: entry.table, data: entry.opDataTyped ?? [:])
default:
// TODO: implement the other operations (patch, delete)
break
}
}
try await batch.complete()
}
}
```
As of v1.14, `CrudBatch`, `CrudEntry`, and `CrudTransaction` are concrete `struct`s instead of protocols. Existing code that reads their properties continues to work unchanged, but they can no longer be constructed in user code (for example, in tests or mocks).
Override `UploadData` to send local updates to your backend service.
```cs theme={null}
public class MyConnector : IPowerSyncBackendConnector
{
public async Task UploadData(IPowerSyncDatabase database)
{
var transaction = await database.GetNextCrudTransaction();
if (transaction == null) return;
try
{
foreach (var operation in transaction.Crud)
{
switch (operation.Op)
{
case UpdateType.PUT:
// Send the data to your backend service
// Replace _myApi with your own API client or service
await _myApi.Put(operation.Table, operation.OpData);
break;
default:
// TODO: implement the other operations (PATCH, DELETE)
break;
}
}
await transaction.Complete();
}
catch (Exception ex)
{
Console.WriteLine($"Upload error: {ex.Message}");
throw;
}
}
}
```
Implement `upload_data` on your `BackendConnector` to send local changes to your backend service. Use `db.crud_transactions()` and iterate with `try_next()`; for each transaction, inspect `tx.crud` and call your API, then `tx.complete().await?`:
```rust theme={null}
use async_trait::async_trait;
#[async_trait]
impl BackendConnector for MyBackendConnector {
async fn upload_data(&self) -> Result<(), PowerSyncError> {
let mut local_writes = self.db.crud_transactions();
while let Some(tx) = local_writes.try_next().await? {
for op in &tx.crud {
// Send the data to your backend service
// Replace with your own API client or service
// match on op.op (e.g. Put, Patch, Delete) and op.table, op.id, op.op_data
}
tx.complete().await?;
}
Ok(())
}
}
```
## Accessing PowerSync Connection Status Information
Use [SyncStatus](https://pub.dev/documentation/powersync/latest/powersync/SyncStatus-class.html) and register an event listener with [statusStream](https://pub.dev/documentation/powersync/latest/powersync/PowerSyncDatabase/statusStream.html) to listen for status changes to your PowerSync instance.
```dart theme={null}
class _StatusAppBarState extends State {
late SyncStatus _connectionState;
StreamSubscription? _syncStatusSubscription;
@override
void initState() {
super.initState();
_connectionState = db.currentStatus;
_syncStatusSubscription = db.statusStream.listen((event) {
setState(() {
_connectionState = db.currentStatus;
});
});
}
@override
void dispose() {
super.dispose();
_syncStatusSubscription?.cancel();
}
@override
Widget build(BuildContext context) {
final statusIcon = _getStatusIcon(_connectionState);
return AppBar(
title: Text(widget.title),
actions: [
...
statusIcon
],
);
}
}
Widget _getStatusIcon(SyncStatus status) {
if (status.anyError != null) {
// The error message is verbose, could be replaced with something
// more user-friendly
if (!status.connected) {
return _makeIcon(status.anyError!.toString(), Icons.cloud_off);
} else {
return _makeIcon(status.anyError!.toString(), Icons.sync_problem);
}
} else if (status.connecting) {
return _makeIcon('Connecting', Icons.cloud_sync_outlined);
} else if (!status.connected) {
return _makeIcon('Not connected', Icons.cloud_off);
} else if (status.uploading && status.downloading) {
// The status changes often between downloading, uploading and both,
// so we use the same icon for all three
return _makeIcon('Uploading and downloading', Icons.cloud_sync_outlined);
} else if (status.uploading) {
return _makeIcon('Uploading', Icons.cloud_sync_outlined);
} else if (status.downloading) {
return _makeIcon('Downloading', Icons.cloud_sync_outlined);
} else {
return _makeIcon('Connected', Icons.cloud_queue);
}
}
```
Use [PowerSyncDatabase.connected](https://powersync-ja.github.io/powersync-js/common/interfaces/CommonPowerSyncDatabase#connected) and register an event listener with [PowerSyncDatabase.registerListener](https://powersync-ja.github.io/powersync-js/common/interfaces/CommonPowerSyncDatabase#registerlistener) to listen for status changes to your PowerSync instance.
```js theme={null}
// Example of using connected status to show online or offline
// Tap into connected
const [connected, setConnected] = React.useState(powersync.connected);
React.useEffect(() => {
// Register listener for changes made to the powersync status
return powersync.registerListener({
statusChanged: (status) => {
setConnected(status.connected);
}
});
}, [powersync]);
// Icon to show connected or not connected to powersync
// as well as the last synced time
{
Alert.alert(
'Status',
`${connected ? 'Connected' : 'Disconnected'}. \nLast Synced at ${powersync.currentStatus?.lastSyncedAt.toISOString() ?? '-'
}\nVersion: ${powersync.sdkVersion}`
);
}}
/>;
```
Use [PowerSyncDatabase.connected](https://powersync-ja.github.io/powersync-js/common/interfaces/CommonPowerSyncDatabase#connected) and register an event listener with [PowerSyncDatabase.registerListener](https://powersync-ja.github.io/powersync-js/common/interfaces/CommonPowerSyncDatabase#registerlistener) to listen for status changes to your PowerSync instance.
```js theme={null}
// Example of using connected status to show online or offline
// Tap into connected
const [connected, setConnected] = React.useState(powersync.connected);
React.useEffect(() => {
// Register listener for changes made to the powersync status
return powersync.registerListener({
statusChanged: (status) => {
setConnected(status.connected);
}
});
}, [powersync]);
// Icon to show connected or not connected to powersync
// as well as the last synced time
{
Alert.alert(
'Status',
`${connected ? 'Connected' : 'Disconnected'}. \nLast Synced at ${powersync.currentStatus?.lastSyncedAt.toISOString() ?? '-'
}\nVersion: ${powersync.sdkVersion}`
);
}}
/>;
```
Example not yet available.
Example not yet available.
```kotlin theme={null}
// Intialize the DB
val db = remember { PowerSyncDatabase(factory, schema) }
// Get the status as a flow
val status = db.currentStatus.asFlow().collectAsState(initial = null)
// Use the emitted values from the flow e.g. to check if connected
val isConnected = status.value?.connected
```
Use [`currentStatus`](https://powersync-ja.github.io/powersync-swift/documentation/powersync/powersyncdatabaseprotocol/currentstatus) and observe changes to listen for status changes to your PowerSync instance.
```swift theme={null}
import Foundation
import SwiftUI
import PowerSync
struct PowerSyncConnectionIndicator: View {
private let status: ObservableSyncStatus
init(powersync: any PowerSyncDatabaseProtocol) {
self.status = powersync.currentStatus.observable
}
var body: some View {
let connected = status.connected
let iconName = connected ? "wifi" : "wifi.slash"
let description = connected ? "Online" : "Offline"
Image(systemName: iconName)
.accessibility(label: Text(description))
}
}
```
Use [SyncStatus](https://powersync-ja.github.io/powersync-dotnet/api/PowerSync.Common.DB.Crud.SyncStatus.html) and `db.Events.OnStatusChanged.ListenAsync` (since v0.0.11-alpha.1) to listen for status changes to your PowerSync instance.
```cs theme={null}
using PowerSync.Common.Client;
using PowerSync.Common.DB.Crud;
class StatusIndicator
{
private SyncStatus? _currentStatus;
public async Task StartListeningAsync(PowerSyncDatabase db, CancellationToken ct = default)
{
var listener = db.Events.OnStatusChanged.ListenAsync(ct);
await foreach (var update in listener)
{
_currentStatus = update.Status;
UpdateStatusIcon(_currentStatus);
}
}
private void UpdateStatusIcon(SyncStatus status)
{
var dataFlow = status.DataFlowStatus;
var hasError = dataFlow.DownloadError != null || dataFlow.UploadError != null;
if (hasError)
{
var errorMessage = dataFlow.DownloadError?.Message ?? dataFlow.UploadError?.Message ?? "Unknown error";
Console.WriteLine(status.Connected ? $"Error: {errorMessage} - Sync problem" : $"Error: {errorMessage} - Not connected");
}
else if (status.Connecting)
{
Console.WriteLine("Connecting...");
}
else if (!status.Connected)
{
Console.WriteLine("Not connected");
}
else if (dataFlow.Uploading && dataFlow.Downloading)
{
Console.WriteLine("Uploading and downloading");
}
else if (dataFlow.Uploading)
{
Console.WriteLine("Uploading");
}
else if (dataFlow.Downloading)
{
Console.WriteLine("Downloading");
}
else
{
Console.WriteLine("Connected");
}
}
}
```
Example not yet available.
## Wait for the Initial Sync to Complete
Use the [hasSynced](https://pub.dev/documentation/powersync/latest/powersync/SyncStatus/hasSynced.html) property and register a listener to indicate to the user whether the initial sync is in progress.
```dart theme={null}
// Example of using hasSynced to show whether the first sync has completed
/// Global reference to the database
final PowerSyncDatabase db;
bool hasSynced = false;
StreamSubscription? _syncStatusSubscription;
// Use the exposed statusStream
Stream watchSyncStatus() {
return db.statusStream;
}
@override
void initState() {
super.initState();
_syncStatusSubscription = watchSyncStatus.listen((status) {
setState(() {
hasSynced = status.hasSynced ?? false;
});
});
}
@override
Widget build(BuildContext context) {
return Text(hasSynced ? 'Initial sync completed!' : 'Busy with initial sync...');
}
// Don't forget to dispose of stream subscriptions when the view is disposed
void dispose() {
super.dispose();
_syncStatusSubscription?.cancel();
}
```
For async use cases, see the [waitForFirstSync](https://pub.dev/documentation/powersync/latest/powersync/PowerSyncDatabase/waitForFirstSync.html) method which returns a promise that resolves once the first full sync has completed.
Use the [hasSynced](https://powersync-ja.github.io/powersync-js/common/interfaces/SyncStatus#hassynced) property and register an event listener with [PowerSyncDatabase.registerListener](https://powersync-ja.github.io/powersync-js/common/interfaces/CommonPowerSyncDatabase#registerlistener) to indicate to the user whether the initial sync is in progress.
```js theme={null}
// Example of using hasSynced to show whether the first sync has completed
// Tap into hasSynced
const [hasSynced, setHasSynced] = React.useState(powerSync.currentStatus?.hasSynced || false);
React.useEffect(() => {
// Register listener for changes made to the powersync status
return powerSync.registerListener({
statusChanged: (status) => {
setHasSynced(!!status.hasSynced);
}
});
}, [powerSync]);
return {hasSynced ? 'Initial sync completed!' : 'Busy with initial sync...'} ;
```
For async use cases, see [PowerSyncDatabase.waitForFirstSync](https://powersync-ja.github.io/powersync-js/common/interfaces/CommonPowerSyncDatabase#waitforfirstsync), which returns a promise that resolves once the first full sync has completed (it queries the internal SQL [ps\_buckets](/architecture/client-architecture) table to determine if data has been synced).
Use the [hasSynced](https://powersync-ja.github.io/powersync-js/common/interfaces/SyncStatus#hassynced) property and register an event listener with [PowerSyncDatabase.registerListener](https://powersync-ja.github.io/powersync-js/common/interfaces/CommonPowerSyncDatabase#registerlistener) to indicate to the user whether the initial sync is in progress.
```js theme={null}
// Example of using hasSynced to show whether the first sync has completed
// Tap into hasSynced
const [hasSynced, setHasSynced] = React.useState(powerSync.currentStatus?.hasSynced || false);
React.useEffect(() => {
// Register listener for changes made to the powersync status
return powerSync.registerListener({
statusChanged: (status) => {
setHasSynced(!!status.hasSynced);
}
});
}, [powerSync]);
return {hasSynced ? 'Initial sync completed!' : 'Busy with initial sync...'}
;
```
For async use cases, see [PowerSyncDatabase.waitForFirstSync()](https://powersync-ja.github.io/powersync-js/common/interfaces/CommonPowerSyncDatabase#waitforfirstsync), which returns a promise that resolves once the first full sync has completed (it queries the internal SQL [ps\_buckets](/architecture/client-architecture) table to determine if data has been synced).
Example not yet available.
Example not yet available.
Use the `hasSynced` property and register a listener to indicate to the user whether the initial sync is in progress.
```kotlin theme={null}
val db = remember { PowerSyncDatabase(factory, schema) }
val status = db.currentStatus.asFlow().collectAsState(initial = null)
val hasSynced by remember { derivedStateOf { status.value?.hasSynced } }
when {
hasSynced == null || hasSynced == false -> {
Box(
modifier = Modifier.fillMaxSize().background(MaterialTheme.colors.background),
contentAlignment = Alignment.Center
) {
Text(
text = "Busy with initial sync...",
style = MaterialTheme.typography.h6
)
}
}
else -> {
... show rest of UI
```
For async use cases, use `waitForFirstSync` method which is a suspense function that resolves once the first full sync has completed.
Use the `hasSynced` property and observe status changes to indicate to the user whether the initial sync is in progress.
```swift theme={null}
struct WaitForFirstSync: View {
private let powersync: any PowerSyncDatabaseProtocol
@State var didSync: Bool = false
init(powersync: any PowerSyncDatabaseProtocol) {
self.powersync = powersync
}
var body: some View {
if !didSync {
ProgressView().task {
do {
try await powersync.waitForFirstSync()
} catch {
// TODO: Handle errors
}
}
}
}
}
```
For async use cases, use [`waitForFirstSync`](https://powersync-ja.github.io/powersync-swift/documentation/powersync/powersyncdatabaseprotocol/waitforfirstsync\(\)).
Use the [HasSynced](https://powersync-ja.github.io/powersync-dotnet/api/PowerSync.Common.DB.Crud.SyncStatus.HasSynced.html) property to indicate to the user whether the initial sync is in progress.
```cs theme={null}
using PowerSync.Common.Client;
// Example of using HasSynced to show whether the first sync has completed
if (status?.HasSynced == true)
{
Console.WriteLine("Initial sync completed!");
}
else
{
Console.WriteLine("Busy with initial sync...");
}
// For async use cases, use WaitForFirstSync which returns a task that completes once the first full sync has completed
await db.WaitForFirstSync();
// Wait for a specific priority level to complete syncing
var prioritySyncRequest = new PowerSyncDatabase.PrioritySyncRequest{ Priority = 1 };
await db.WaitForFirstSync(request: prioritySyncRequest);
```
Example not yet available.
## Report Sync Download Progress
You can show users a progress bar when data downloads using the `downloadProgress` property from the
[SyncStatus](https://pub.dev/documentation/powersync/latest/powersync/SyncStatus/downloadProgress.html) class.
`downloadProgress.downloadedFraction` gives you a value from 0.0 to 1.0 representing the total sync progress. This is especially useful for long-running initial syncs.
As an example, this widget renders a progress bar when a download is active:
```dart theme={null}
import 'package:flutter/material.dart';
import 'package:powersync/powersync.dart' hide Column;
class SyncProgressBar extends StatelessWidget {
final PowerSyncDatabase db;
/// When set, show progress towards the [BucketPriority] instead of towards
/// the full sync.
final BucketPriority? priority;
const SyncProgressBar({
super.key,
required this.db,
this.priority,
});
@override
Widget build(BuildContext context) {
return StreamBuilder(
stream: db.statusStream,
initialData: db.currentStatus,
builder: (context, snapshot) {
final status = snapshot.requireData;
final progress = switch (priority) {
null => status.downloadProgress,
var priority? => status.downloadProgress?.untilPriority(priority),
};
if (progress != null) {
return Center(
child: Column(
children: [
const Text('Busy with sync...'),
LinearProgressIndicator(value: progress?.downloadedFraction),
Text(
'${progress.downloadedOperations} out of ${progress.totalOperations}')
],
),
);
} else {
return const SizedBox.shrink();
}
},
);
}
}
```
Also see:
* [SyncDownloadProgress API](https://pub.dev/documentation/powersync/latest/powersync/SyncDownloadProgress-extension-type.html)
* [Demo component](https://github.com/powersync-ja/powersync.dart/blob/main/demos/supabase-todolist/lib/widgets/guard_by_sync.dart)
You can show users a progress bar when data downloads using the `downloadProgress` property from the [SyncStatus](https://powersync-ja.github.io/powersync-js/common/interfaces/SyncStatus) class. This is especially useful for long-running initial syncs. `downloadProgress.downloadedFraction` gives you a value from 0.0 to 1.0 representing the total sync progress.
Example:
```jsx theme={null}
import { useStatus } from '@powersync/react';
import { FC, ReactNode } from 'react';
import { View } from 'react-native';
import { Text, LinearProgress } from '@rneui/themed';
export const SyncProgressBar: FC<{ priority?: number }> = ({ priority }) => {
const status = useStatus();
const progressUntilNextSync = status.downloadProgress;
const progress = priority == null ? progressUntilNextSync : progressUntilNextSync?.untilPriority(priority);
if (progress == null) {
return <>;
}
return (
Applying server-side changes
) : (
Downloaded {progress.downloadedOperations} out of {progress.totalOperations}.
)}
);
};
```
Also see:
* [SyncStatus API](https://powersync-ja.github.io/powersync-js/common/interfaces/SyncStatus)
* [Demo component](https://github.com/powersync-ja/powersync-js/blob/main/demos/react-native-supabase-todolist/library/widgets/GuardBySync.tsx)
You can show users a progress bar when data downloads using the `downloadProgress` property from the
[SyncStatus](https://powersync-ja.github.io/powersync-js/common/interfaces/SyncStatus) class. This is especially useful for long-running initial syncs. `downloadProgress.downloadedFraction` gives you a value from 0.0 to 1.0 representing the total sync progress.
Example (React, using [MUI](https://mui.com) components):
```jsx theme={null}
import { Box, LinearProgress, Stack, Typography } from '@mui/material';
import { useStatus } from '@powersync/react';
import { FC, ReactNode } from 'react';
export const SyncProgressBar: FC<{ priority?: number }> = ({ priority }) => {
const status = useStatus();
const progressUntilNextSync = status.downloadProgress;
const progress = priority == null ? progressUntilNextSync : progressUntilNextSync?.untilPriority(priority);
if (progress == null) {
return <>;
}
return (
{progress.downloadedOperations == progress.totalOperations ? (
Applying server-side changes
) : (
Downloaded {progress.downloadedOperations} out of {progress.totalOperations}.
)}
);
};
```
Also see:
* [SyncStatus API](https://powersync-ja.github.io/powersync-js/common/interfaces/SyncStatus)
* [Demo component](https://github.com/powersync-ja/powersync-js/blob/main/demos/react-supabase-todolist/src/components/widgets/GuardBySync.tsx)
Example not yet available.
Example not yet available.
You can show users a progress bar when data downloads using the `syncStatus.downloadProgress` property. This is especially useful for long-running initial syncs. `downloadProgress.downloadedFraction` gives a value from 0.0 to 1.0 representing the total sync progress.
Example (Compose):
```kotlin theme={null}
import androidx.compose.foundation.background
import androidx.compose.foundation.layout.Arrangement
import androidx.compose.foundation.layout.Column
import androidx.compose.foundation.layout.fillMaxSize
import androidx.compose.foundation.layout.fillMaxWidth
import androidx.compose.foundation.layout.padding
import androidx.compose.material.LinearProgressIndicator
import androidx.compose.material.MaterialTheme
import androidx.compose.material.Text
import androidx.compose.runtime.Composable
import androidx.compose.runtime.getValue
import androidx.compose.ui.Alignment
import androidx.compose.ui.Modifier
import androidx.compose.ui.unit.dp
import com.powersync.PowerSyncDatabase
import com.powersync.bucket.BucketPriority
import com.powersync.compose.composeState
/**
* Shows a progress bar while a sync is active.
*
* The [priority] parameter can be set to, instead of showing progress until the end of the entire
* sync, only show progress until data in the [BucketPriority] is synced.
*/
@Composable
fun SyncProgressBar(
db: PowerSyncDatabase,
priority: BucketPriority? = null,
) {
val state by db.currentStatus.composeState()
val progress = state.downloadProgress?.let {
if (priority == null) {
it
} else {
it.untilPriority(priority)
}
}
if (progress == null) {
return
}
Column(
modifier = Modifier.fillMaxSize().background(MaterialTheme.colors.background),
horizontalAlignment = Alignment.CenterHorizontally,
verticalArrangement = Arrangement.Center,
) {
LinearProgressIndicator(
modifier = Modifier.fillMaxWidth().padding(8.dp),
progress = progress.fraction,
)
if (progress.downloadedOperations == progress.totalOperations) {
Text("Applying server-side changes...")
} else {
Text("Downloaded ${progress.downloadedOperations} out of ${progress.totalOperations}.")
}
}
}
```
Also see:
* [SyncDownloadProgress API](https://powersync-ja.github.io/powersync-kotlin/common/com.powersync.sync/-sync-download-progress/index.html)
* [Demo component](https://github.com/powersync-ja/powersync-kotlin/blob/main/demos/supabase-todolist/shared/src/commonMain/kotlin/com/powersync/demos/components/GuardBySync.kt)
You can show users a progress bar when data downloads using the `downloadProgress` property from the [`SyncStatusData`](https://powersync-ja.github.io/powersync-swift/documentation/powersync/syncstatusdata/) object. `downloadProgress.downloadedFraction` gives you a value from 0.0 to 1.0 representing the total sync progress. This is especially useful for long-running initial syncs.
Example:
```swift theme={null}
struct SyncProgressIndicator: View {
private let status: ObservableSyncStatus
private let priority: BucketPriority?
init(powersync: any PowerSyncDatabaseProtocol, priority: BucketPriority? = nil) {
self.status = powersync.currentStatus.observable
self.priority = priority
}
var body: some View {
VStack {
if let totalProgress = status.downloadProgress {
let progress = if let priority = self.priority {
totalProgress.untilPriority(priority: priority)
} else {
totalProgress
}
ProgressView(value: progress.fraction)
if progress.downloadedOperations == progress.totalOperations {
Text("Applying server-side changes...")
} else {
Text("Downloaded \(progress.downloadedOperations) out of \(progress.totalOperations)")
}
}
}
}
}
```
Also see:
* [SyncStatusData API](https://powersync-ja.github.io/powersync-swift/documentation/powersync/syncstatusdata/)
* [SyncDownloadProgress API](https://powersync-ja.github.io/powersync-swift/documentation/powersync/syncdownloadprogress/)
* [Demo component](https://github.com/powersync-ja/powersync-swift/blob/main/Demos/PowerSyncExample/PowerSyncExample/Components/ListView.swift)
You can show users a progress bar when data downloads using the `DownloadProgress()` method from the [SyncStatus](https://powersync-ja.github.io/powersync-dotnet/api/PowerSync.Common.DB.Crud.SyncStatus.html) class. `DownloadProgress().DownloadedFraction` gives you a value from 0.0 to 1.0 representing the total sync progress. This is especially useful for long-running initial syncs.
Example:
```cs theme={null}
using PowerSync.Common.Client;
using PowerSync.Common.DB.Crud;
class SyncProgressBar
{
private PowerSyncDatabase _db;
private int? _priority; // Optional: when set, show progress towards this priority instead of the full sync
private SyncStatus? _currentStatus;
public async Task StartListeningAsync(CancellationToken ct = default)
{
var listener = _db.Events.OnStatusChanged.ListenAsync(ct);
await foreach (var update in listener)
{
_currentStatus = update.Status;
DisplayProgress();
}
}
public SyncProgressBar(PowerSyncDatabase db, int? priority = null)
{
_db = db;
_priority = priority;
}
public void DisplayProgress()
{
var status = _currentStatus;
var totalProgress = status?.DownloadProgress();
var progress = _priority == null
? totalProgress
: totalProgress?.UntilPriority(_priority.Value);
if (progress != null)
{
var fraction = progress.DownloadedFraction; // 0.0 to 1.0
var downloadedOps = progress.DownloadedOperations;
var totalOps = progress.TotalOperations;
Console.WriteLine($"Sync progress: {fraction * 100:F1}%");
if (downloadedOps == totalOps)
{
Console.WriteLine("Applying server-side changes...");
}
else
{
Console.WriteLine($"Downloaded {downloadedOps} out of {totalOps} operations");
}
}
}
}
```
Also see:
* [SyncStatus API](https://powersync-ja.github.io/powersync-dotnet/api/PowerSync.Common.DB.Crud.SyncStatus.html)
* [SyncProgress API](https://powersync-ja.github.io/powersync-dotnet/api/PowerSync.Common.DB.Crud.SyncProgress.html)
* [Demo component](https://github.com/powersync-ja/powersync-dotnet/blob/main/demos/MAUITodo/Views/ListsPage.xaml.cs)
Example not yet available.
# Live Queries / Watch Queries
Source: https://docs.powersync.com/client-sdks/watch-queries
Keep your UI current with basic, incremental, and differential watch queries over client-side SQLite.
Watch queries turn a SQL query into a subscription. When a local write or incoming sync changes a dependent table, PowerSync reruns the query against SQLite and returns the latest results to your app. Your UI can react to changes without polling, manual refetching, or separate cache invalidation logic, and these changes can arrive from the local device or from other users and devices.
Prefer not to write SQL directly? See [ORM support](/client-sdks/orms/js/overview) for type-safe query builders and ORM integrations. Reactive-query features vary by SDK and ORM.
## Overview
PowerSync provides multiple approaches to watching queries, each designed for different use cases and performance requirements:
1. Basic Watch Queries - These queries work across all SDKs, providing real-time updates when dependent tables change.
2. Incremental Watch Queries - Only emit updates when data actually changes, preventing unnecessary re-renders. Available in JavaScript SDKs only.
3. Differential Watch Queries - Provide detailed information about what specifically changed between result sets. Available in JavaScript SDKs only.
Choose the approach that best fits your platform and performance needs.
## Basic Watch Queries
PowerSync supports the following basic watch queries based on your platform. These APIs return query results whenever the underlying tables change and are available across all SDKs.
```javascript AsyncIterator theme={null}
// The original watch method using the AsyncIterator pattern. This is the
// foundational watch API that works across all JavaScript environments.
async function* pendingLists(): AsyncIterable {
for await (const result of db.watch(
`SELECT * FROM lists WHERE state = ?`,
['pending']
)) {
yield result.rows?._array ?? [];
}
}
```
```javascript Callback theme={null}
// Callback-based watch method that doesn't require AsyncIterator polyfills.
// Use this approach when you need smoother React Native compatibility or
// prefer synchronous method signatures.
const pendingLists = (onResult: (lists: any[]) => void): void => {
db.watch(
'SELECT * FROM lists WHERE state = ?',
['pending'],
{
onResult: (result: any) => {
onResult(result.rows?._array ?? []);
}
}
);
}
```
```javascript React Hook theme={null}
// React hook that combines watch functionality with built-in loading,
// fetching, and error states. Use this when you need convenient state
// management without React Suspense.
const {
data: pendingLists,
isLoading,
isFetching,
error
} = useQuery('SELECT * FROM lists WHERE state = ?', ['pending']);
```
```javascript React Suspense theme={null}
// React Suspense-based hook that automatically handles loading and error
// states through Suspense boundaries. Use this when you want to leverage
// React's concurrent features and avoid manual state handling.
const { data: pendingLists } = useSuspenseQuery('SELECT * FROM lists WHERE state = ?', ['pending']);
```
```javascript Vue Hook theme={null}
// Vue composition API hook with built-in loading, fetching, and error
// states. Use this for reactive watch queries in Vue applications.
const {
data: pendingLists,
isLoading,
isFetching,
error
} = useQuery('SELECT * FROM lists WHERE state = ?', ['pending']);
```
The `db.watch()` AsyncIterator and Callback methods above are only maintained for backwards compatibility. Use the improved `db.query().watch()` API instead (see [Incremental Watch Queries](/client-sdks/watch-queries#incremental-watch-queries) below).
Use this method to watch for changes to the dependent tables of any SQL query:
```dart theme={null}
StreamBuilder(
stream: db.watch('SELECT * FROM lists WHERE state = ?', ['pending']),
builder: (context, snapshot) {
if (snapshot.hasData) {
// TODO: implement your own UI here based on the result set
return ...;
} else {
return const Center(child: CircularProgressIndicator());
}
},
)
```
Use this method to watch for changes to the dependent tables of any SQL query:
```kotlin theme={null}
fun watchPendingLists(): Flow> =
db.watch(
"SELECT * FROM lists WHERE state = ?",
listOf("pending"),
) { cursor ->
ListItem(
id = cursor.getString("id"),
name = cursor.getString("name"),
)
}
```
Use this method to watch for changes to the dependent tables of any SQL query:
```swift theme={null}
func watchPendingLists() throws -> AsyncThrowingStream<[ListContent], Error> {
try db.watch(
sql: "SELECT * FROM lists WHERE state = ?",
parameters: ["pending"],
) { cursor in
try ListContent(
id: cursor.getString(name: "id"),
name: cursor.getString(name: "name"),
)
}
}
```
Use this method to watch for changes to the dependent tables of any SQL query:
```csharp theme={null}
// Define a result type with properties matching the schema columns (some columns omitted here for brevity)
// public class ListResult { public string id; public string name; public string owner_id; ... }
// Optional cancellation token to stop watching
var cts = new CancellationTokenSource();
// Register listener synchronously on the calling thread...
var listener = db.Watch(
"SELECT * FROM lists WHERE owner_id = ?",
[ownerId],
new SQLWatchOptions { Signal = cts.Token }
);
// ...then listen to changes on another thread (or await foreach directly if already in an async context)
_ = Task.Run(async () =>
{
await foreach (var results in listener)
{
Console.WriteLine("Lists: ");
foreach (var result in results)
{
Console.WriteLine($"{result.id}: {result.name}");
}
}
}, cts.Token);
// To stop watching, cancel the token: cts.Cancel();
```
Use this method to watch for changes to the dependent tables of any SQL query:
```Rust theme={null}
async fn watch_pending_lists(db: &PowerSyncDatabase) -> Result<(), PowerSyncError> {
let stream = db.watch_statement(
"SELECT * FROM lists WHERE state = ?".to_string(),
params!["pending"],
|stmt, params| {
let mut rows = stmt.query(params)?;
let mut mapped = vec![];
while let Some(row) = rows.next()? {
mapped.push(() /* TODO: Read row into list struct */)
}
Ok(mapped)
},
);
let mut stream = pin!(stream);
// Note: The stream is never-ending, so you probably want to call this in an independent async
// task.
while let Some(event) = stream.try_next().await? {
// Update UI to display rows
}
Ok(())
}
```
## Incremental Watch Queries
Basic watch queries can cause performance issues in UI frameworks like React because they return new data on every dependent table change, even when the actual data in the query hasn't changed. This can lead to excessive re-renders as components receive updates unnecessarily.
Incremental watch queries solve this by comparing result sets using configurable comparators and only emitting updates when the comparison detects actual data changes. These queries still query the SQLite database under the hood on each dependent table change, but compare the result sets and only yield results if a change has been made.
**JavaScript Only**: Incremental and differential watch queries are currently only available in the JavaScript SDKs starting from:
* Web v1.25.0
* React Native v1.23.1
* Node.js v0.8.1
Basic Syntax:
```javascript theme={null}
db.query({ sql: 'SELECT * FROM lists WHERE state = ?', parameters: ['pending'] }).watch();
```
### WatchedQuery
`WatchedQuery` class that comes with a better API in that it includes loading, fetching and error states, supports multiple listeners, automatic cleanup on PowerSync close, and the new `updateSettings()` API for dynamic parameter changes. This is the preferred approach for JavaScript SDKs:
```javascript WatchedQuery (Default) theme={null}
// Create an instance of a WatchedQuery
const pendingLists = db
.query({
sql: 'SELECT * FROM lists WHERE state = ?',
parameters: ['pending']
})
.watch();
// The registerListener method can be used multiple times to listen for updates
const dispose = pendingLists.registerListener({
onData: (data) => {
// This callback will be called whenever the data changes
console.log('Data updated:', data);
},
onStateChange: (state) => {
// This callback will be called whenever the state changes
// The state contains metadata about the query, such as isFetching, isLoading, etc.
console.log('State changed:', state.error, state.isFetching, state.isLoading, state.data);
},
onError: (error) => {
// This callback will be called if the query fails
console.error('Query error:', error);
}
});
```
```javascript WatchedQuery (With Comparator) theme={null}
// WatchedQuery with configurable comparator that compares result sets before
// emitting to listeners, preventing unnecessary listener invocations when
// data hasn't changed. Use this when you want shared query instances plus
// result set comparison for incremental updates.
const pendingLists = db
.query({
sql: 'SELECT * FROM lists WHERE state = ?',
parameters: ['pending']
})
.watch({
comparator: {
checkEquality: (current, previous) => {
return JSON.stringify(current) === JSON.stringify(previous);
}
}
});
// Register listeners as before...
```
### React Hooks
React hooks that preserve object references for unchanged items and use row-level comparators to minimize re-renders:
```javascript useQuery theme={null}
// Use this when you want built-in state management plus
// incremental updates for React components.
const {
data: pendingLists,
isLoading,
isFetching,
error
} = useQuery('SELECT * FROM lists WHERE state = ?', ['pending'], {
rowComparator: {
keyBy: (item) => item.id,
compareBy: (item) => JSON.stringify(item)
}
});
```
```javascript useSuspenseQuery theme={null}
// Use this when you want concurrent React features, automatic
// state handling, and memoization-friendly object stability.
const { data: lists } = useSuspenseQuery('SELECT * FROM lists WHERE state = ?', ['pending'], {
rowComparator: {
keyBy: (item) => item.id,
compareBy: (item) => JSON.stringify(item)
}
});
```
```jsx React.memo (Row-Level) theme={null}
const TodoListsWidget = () => {
const { data: lists } = useQuery('[SQL]', [...parameters], { rowComparator: DEFAULT_ROW_COMPARATOR });
return (
{
// The individual row widgets will only re-render if the corresponding row has changed
lists.map((listRecord) => (
);
};
const TodoWidget = React.memo(({ record }) => {
return {record.name} ;
});
```
Providing a `rowComparator` to the React hooks ensures that components only re-render when the query result actually changes. When combined with React memoization (e.g., `React.memo`) on row components that receive query row objects as props, this approach prevents unnecessary updates at the individual row component level, resulting in more efficient UI rendering.
### Existing Watch APIs (with Comparator)
The existing `AsyncIterator` and Callback `db.watch()` APIs also support incremental updates via a `comparator` option. Use these if you want to maintain the familiar patterns from the basic watch query API:
```javascript AsyncIterator theme={null}
async function* pendingLists(): AsyncIterable {
for await (const result of db.watch('SELECT * FROM lists WHERE state = ?', ['pending'], {
comparator: {
checkEquality: (current, previous) => JSON.stringify(current) === JSON.stringify(previous)
}
})) {
yield result.rows?._array ?? [];
}
}
```
```javascript Callback theme={null}
const pendingLists = (onResult: (lists: any[]) => void): void => {
db.watch(
'SELECT * FROM lists WHERE state = ?',
['pending'],
{
onResult: (result: any) => {
onResult(result.rows?._array ?? []);
}
},
{
comparator: {
checkEquality: (current, previous) => {
// This comparator will only report updates if the data changes.
return JSON.stringify(current) === JSON.stringify(previous);
}
}
}
);
};
```
## Differential Watch Queries
Differential watch queries go a step further than incremental watched queries by computing and reporting diffs between result sets (added/removed/updated items) while preserving object references for unchanged items. This enables more precise UI updates.
**JavaScript Only**: Incremental and differential watch queries are currently only available in the JavaScript SDKs starting from:
* Web v1.25.0
* React Native v1.23.1
* Node.js v0.8.1
For large result sets where re-running and comparing full query results becomes expensive, consider using trigger-based table diffs. See [High Performance Diffs](/client-sdks/high-performance-diffs).
Basic syntax:
```javascript theme={null}
db.query({ sql: 'SELECT * FROM lists WHERE state = ?', parameters: ['pending'] }).differentialWatch();
```
Use differential watch when you need to know exactly which items were added, removed, or updated rather than re-processing entire result sets:
```javascript theme={null}
// Create an instance of a WatchedQuery
const pendingLists = db
.query({
sql: 'SELECT * FROM lists WHERE state = ?',
parameters: ['pending']
})
.differentialWatch();
// The registerListener method can be used multiple times to listen for updates
const dispose = pendingLists.registerListener({
onData: (data) => {
// This callback will be called whenever the data changes
console.log('Data updated:', data);
},
onStateChange: (state) => {
// This callback will be called whenever the state changes
// The state contains metadata about the query, such as isFetching, isLoading, etc.
console.log('State changed:', state.error, state.isFetching, state.isLoading, state.data);
},
onError: (error) => {
// This callback will be called if the query fails
console.error('Query error:', error);
},
onDiff: (diff) => {
// This callback will be called whenever the data changes.
console.log('Data updated:', diff.added, diff.updated);
}
});
```
By default, the `differentialWatch()` method uses a `DEFAULT_ROW_COMPARATOR`. This comparator identifies (keys) each row by its `id` column if present, or otherwise by the JSON string of the entire row. For row comparison, it uses the JSON string representation of the full row. This approach is generally safe and effective for most queries.
For some queries, performance could be improved by supplying a custom `rowComparator`. Such as comparing by a `hash` column generated or stored in SQLite. These hashes currently require manual implementation.
```javascript theme={null}
const pendingLists = db
.query({
sql: 'SELECT * FROM lists WHERE state = ?',
parameters: ['pending']
})
.differentialWatch({
rowComparator: {
keyBy: (item) => item.id,
compareBy: (item) => item._hash
}
});
```
The [Yjs Document Collaboration Demo
app](https://github.com/powersync-ja/powersync-js/tree/main/demos/yjs-react-supabase-text-collab) showcases the use of
differential watch queries. New document updates are passed to Yjs for consolidation as they are synced. See the
implementation
[here](https://github.com/powersync-ja/powersync-js/blob/main/demos/yjs-react-supabase-text-collab/src/library/powersync/PowerSyncYjsProvider.ts)
for more details.
## The `WatchedQuery` Class
Both incremental and differential queries use the new `WatchedQuery` class. This class, along with a new `query` method allows building instances of `WatchedQuery`s via the `watch` and `differentialWatch` methods:
```javascript theme={null}
const watchedQuery = db.query({ sql: 'SELECT * FROM lists', parameters: [] }).watch();
```
This class provides advanced features:
* Automatically reprocesses itself if the PowerSync schema has been updated with `updateSchema`.
* Automatically closes itself when the PowerSync client has been closed.
* Allows for the query parameters to be updated after instantiation.
* Allows shared listening to state changes.
* New `updateSettings` API for dynamic parameter updates (see below).
### Query Sharing
`WatchedQuery` instances can be shared across components:
```javascript theme={null}
// Create a shared query instance
const sharedListsQuery = db.query({ sql: 'SELECT * FROM lists WHERE state = ?', parameters: ['pending'] }).watch();
// Multiple components can listen to the same query
const dispose1 = sharedListsQuery.registerListener({
onData: (data) => updatePendingListsDisplay(data)
});
const dispose2 = sharedListsQuery.registerListener({
onData: (data) => updatePendingListsCount(data.length)
});
```
### Dynamic Parameter Updates
Update query parameters to affect all listeners of the query:
```javascript theme={null}
// Updates to query parameters can be performed in a single place, affecting all listeners
sharedListsQuery.updateSettings({
query: new GetAllQuery({ sql: 'SELECT * FROM lists WHERE state = ?', parameters: ['canceled'] })
});
```
### React Hook for External WatchedQuery Instances
When you need to share query instances across components or manage their lifecycle independently from component mounting, use the `useWatchedQuerySubscription` hook. This is ideal for global state management, query caching, or when multiple components need to listen to the same data:
```javascript theme={null}
// Managing the WatchedQuery externally can extend its lifecycle and allow in-memory caching between components.
const pendingLists = db
.query({
sql: 'SELECT * FROM lists WHERE state = ?',
parameters: ['pending']
})
.watch();
// In the component
export const MyComponent = () => {
// In React one could import the `pendingLists` query or create a context provider for various queries
const { data } = useWatchedQuerySubscription(pendingLists);
return (
{data.map((item) => (
{item.name}
))}
```typescript TypeScript theme={null}
// Insert a new todo
await db.execute(
'INSERT INTO todos (id, created_at, list_id, description) VALUES (uuid(), date(), ?, ?)',
[listId, 'Buy groceries']
);
// Update a todo
await db.execute(
'UPDATE todos SET completed = 1, completed_at = date() WHERE id = ?',
[todoId]
);
// Delete a todo
await db.execute('DELETE FROM todos WHERE id = ?', [todoId]);
```
```kotlin Kotlin theme={null}
// Insert a new todo
database.writeTransaction {
database.execute(
sql = "INSERT INTO todos (id, created_at, list_id, description) VALUES (uuid(), date(), ?, ?)",
parameters = listOf(listId, "Buy groceries")
)
}
// Update a todo
database.execute(
sql = "UPDATE todos SET completed = 1, completed_at = date() WHERE id = ?",
parameters = listOf(todoId)
)
// Delete a todo
database.execute(
sql = "DELETE FROM todos WHERE id = ?",
parameters = listOf(todoId)
)
```
```swift Swift theme={null}
// Insert a new todo
try await db.execute(
sql: "INSERT INTO todos (id, created_at, list_id, description) VALUES (uuid(), date(), ?, ?)",
parameters: [listId, "Buy groceries"]
)
// Update a todo
try await db.execute(
sql: "UPDATE todos SET completed = 1, completed_at = date() WHERE id = ?",
parameters: [todoId]
)
// Delete a todo
try await db.execute(
sql: "DELETE FROM todos WHERE id = ?",
parameters: [todoId]
)
```
```dart Dart/Flutter theme={null}
// Insert a new todo
await db.execute(
'INSERT INTO todos (id, created_at, list_id, description) VALUES (uuid(), date(), ?, ?)',
[listId, 'Buy groceries']
);
// Update a todo
await db.execute(
'UPDATE todos SET completed = 1, completed_at = date() WHERE id = ?',
[todoId]
);
// Delete a todo
await db.execute('DELETE FROM todos WHERE id = ?', [todoId]);
```
```csharp .NET theme={null}
// Insert a new todo
await db.Execute(
"INSERT INTO todos (id, created_at, list_id, description) VALUES (uuid(), datetime(), ?, ?)",
new[] { listId, "Buy groceries" }
);
// Update a todo
await db.Execute(
"UPDATE todos SET completed = 1, completed_at = datetime() WHERE id = ?",
new[] { todoId }
);
// Delete a todo
await db.Execute(
"DELETE FROM todos WHERE id = ?",
new[] { todoId }
);
```
**Best practice**: Use UUIDs when inserting new rows on the client side. UUIDs can be generated offline/locally, allowing for unique identification of records created in the client database before they are synced to the server. See [Client ID](/sync/advanced/client-id) for more details.
## ORM Support
PowerSync integrates with popular ORM libraries, which provide type safety and additional tooling. Using an ORM is often preferable to writing raw SQL queries, especially for common operations.
See [ORM Support](/client-sdks/orms/overview) to learn which ORMs PowerSync supports and how to get started.
## Write Operations and Upload Queue
PowerSync automatically queues writes and uploads them to your backend. The upload queue stores three types of operations:
| Operation | Purpose | Contents | SQLite Statement |
| --------- | ------------------- | -------------------------------------------------------- | --------------------------------- |
| `PUT` | Create new row | Contains the value for each non-null column | Generated by `INSERT` statements. |
| `PATCH` | Update existing row | Contains the row `id`, and value of each changed column. | Generated by `UPDATE` statements. |
| `DELETE` | Delete existing row | Contains the row `id` | Generated by `DELETE` statements. |
The SDK automatically processes this queue by calling your `uploadData()` function. For a detailed breakdown of when and how `uploadData()` is called, including triggers, throttling, retry behavior, and error handling, see [When `uploadData()` is Called](/configuration/app-backend/client-side-integration#when-uploaddata-is-called).
For details on how to structure your backend to accept these writes, see [Writing Client Changes](/handling-writes/writing-client-changes).
## Advanced Topics
* [Usage Examples](/client-sdks/usage-examples) - Code examples for common use cases
# Client-Side Integration With Your Backend
Source: https://docs.powersync.com/configuration/app-backend/client-side-integration
Connect your client app to a PowerSync-compatible backend using the connector interface and upload queue.
## How PowerSync Uses Your Backend
After you've [instantiated](/intro/setup-guide#instantiate-the-powersync-database) the client-side PowerSync database, you will call `connect()` on it, which causes the PowerSync Client SDK to connect to the [PowerSync Service](/architecture/powersync-service) for the purpose of syncing data to the client-side SQLite database, *and* to connect to your backend application as needed, for two potential purposes:
| Purpose | Description |
| ------------------------------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Uploading mutations to your backend:** | Mutations that are made to the client-side SQLite database are uploaded to your backend application, where you control how they're applied to your backend source database (Postgres, MongoDB, MySQL, SQL Server, or Convex). This is how PowerSync achieves bi-directional syncing of data: The [PowerSync Service](/architecture/powersync-service) provides the *server-to-client read path* based on your [Sync Streams or Sync Rules (legacy)](/sync/overview), and the *client-to-server write path* goes via your backend. |
| **Authentication integration:** (optional) | PowerSync uses JWTs for authentication between the Client SDK and PowerSync Service. Some [authentication providers](/configuration/auth/overview#common-authentication-providers) generate JWTs for users which PowerSync can verify directly. For others, some code must be [added to your application backend](/configuration/auth/custom) to generate the JWTs. |
## Backend Connector
Accordingly, you must pass a *backend connector* as an argument when you call `connect()` on the client-side PowerSync database. You must define that backend connector, and it must implement two functions/methods:
| Purpose | Function | Description |
| ---------------------------------------- | -------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Uploading mutations to your backend:** | `uploadData()` | The PowerSync Client SDK automatically calls this function to upload client-side mutations to your backend. Whenever you write to the client-side SQLite database, those writes are also automatically placed into an *upload queue* by the Client SDK, and the Client SDK processes the entries in the upload queue by calling `uploadData()`. You should define your `uploadData()` function to call your backend application API to upload and apply the write operations to your backend source database. The Client SDK automatically handles retries in the case of failures. See the [detailed behavior below](#when-uploaddata-is-called) and [Writing Data](/client-sdks/writing-data) in the *Client SDKs* section for more details on the implementation of `uploadData()`. |
| **Authentication integration:** | `fetchCredentials()` | Called by the PowerSync Client SDK to obtain a JWT and the endpoint URL for your PowerSync Service instance. The SDK uses the JWT to authenticate against the PowerSync Service. `fetchCredentials()` typically returns an object with `token` (JWT) and `endpoint` fields. See [Authentication Setup](/configuration/auth/overview) for more details on JWT authentication. |
Some authentication providers generate JWTs for users which PowerSync can verify directly, and in that case, your `fetchCredentials()` function implementation can simply return that JWT from client-side state. Your `fetchCredentials()` implementation only needs to retrieve a JWT from your backend if you are using [Custom Authentication](/configuration/auth/custom) integration. See the [Authentication Overview](/configuration/auth/overview) for more background.
### When `uploadData()` is Called
The PowerSync Client SDK calls `uploadData()` automatically - you never call it directly. It is invoked in the following scenarios:
1. **After a local write** - Any `INSERT`, `UPDATE`, or `DELETE` on a PowerSync table adds an entry to the internal upload queue (the `ps_crud` table). The SDK monitors this table for changes and triggers `uploadData()` shortly after. See [How Local Writes Are Detected](#how-local-writes-are-detected) below for details on the underlying mechanism.
2. **On initial connection / reconnection** - When `connect()` establishes (or re-establishes) a sync stream and the first message is received from the PowerSync Service, the SDK triggers `uploadData()` to flush any writes that were made while offline or disconnected.
3. **On keepalive messages** - The PowerSync Service sends periodic keepalive messages (every 20 seconds, with slight jitter). Each keepalive triggers an upload attempt, ensuring pending writes are retried even if no new local writes have occurred.
4. **After an error, with retry** - If `uploadData()` throws an error, the SDK waits for a configurable delay (default: 5 seconds, controlled by `retryDelayMs`) and then retries. This continues until the upload succeeds or the sync stream is disconnected.
#### How Local Writes Are Detected
By default, each PowerSync-managed table is exposed in SQLite as a **view** with `INSTEAD OF INSERT/UPDATE/DELETE` triggers (generated by the [PowerSync SQLite core extension](https://github.com/powersync-ja/powersync-sqlite-core) when your [client-side schema](/intro/setup-guide#define-your-client-side-schema) is applied). When you write to a PowerSync table, those triggers do two things atomically:
1. Apply the change to the [underlying `ps_data__` table](/architecture/client-architecture#client-side-schema-and-sqlite-database-structure).
2. Append an entry to `ps_crud` describing the operation (see [Write Operations and Upload Queue](/client-sdks/writing-data#write-operations-and-upload-queue) for the entry format).
Because both happen in the same transaction, the upload queue can never get out of sync with local data. ([Raw Tables](/client-sdks/advanced/raw-tables#capture-local-writes-with-triggers) preserve this guarantee, except you create the triggers yourself, typically via the `powersync_create_raw_table_crud_trigger` helper.)
The SDK then detects new `ps_crud` entries by subscribing to SQLite's table-update notifications, filtered for changes to `ps_crud`. This is the same mechanism that powers reactive [watch queries](/client-sdks/watch-queries). When a change is observed, the SDK schedules an `uploadData()` call subject to the configured [throttle interval](#throttling).
The mechanism is consistent across all PowerSync SDKs (JavaScript, Dart, Kotlin, Swift, .NET, Rust), since the triggers and `ps_crud` table are defined by the shared PowerSync SQLite core extension.
#### Upload Loop Behavior
The SDK calls `uploadData()` in a **loop** - not just once per trigger. After each successful `uploadData()` call, the SDK checks whether there are more items in the upload queue. If there are, it calls `uploadData()` again immediately. The loop continues until the queue is empty. This means your `uploadData()` implementation only needs to process one batch (or one transaction) per call.
Once the queue is empty, the SDK updates an internal write checkpoint used for [consistency](/architecture/consistency) tracking.
#### Throttling
To avoid excessive calls, upload triggers are **throttled**. Rapid local writes (e.g., multiple `INSERT` statements in quick succession) are coalesced so that `uploadData()` is invoked at most once per throttle interval. If an upload is already in progress when a new write occurs, the SDK will trigger another upload after the current one completes.
The default throttle interval varies by SDK:
| SDK | Default | Option Name |
| ---------------------------------------------------- | -------- | ------------------------ |
| JavaScript / React Native / Node / Capacitor / Tauri | 1,000 ms | `crudUploadThrottleMs` |
| Kotlin | 1,000 ms | `crudThrottleMs` |
| Swift | 1 second | `crudThrottle` (seconds) |
| .NET | 1,000 ms | `CrudUploadThrottleMs` |
| Dart / Flutter | 10 ms | `crudThrottleTime` |
The throttle interval is configurable via options when calling `connect()`:
```typescript TypeScript theme={null}
await db.connect(connector, { crudUploadThrottleMs: 500 });
```
```dart Dart/Flutter theme={null}
db.connect(
connector: connector,
options: SyncOptions(
crudThrottleTime: Duration(milliseconds: 500),
),
);
```
```kotlin Kotlin theme={null}
database.connect(connector, crudThrottleMs = 500L)
```
```swift Swift theme={null}
try await database.connect(
connector: connector,
options: ConnectOptions(crudThrottle: 0.5) // seconds
)
```
```csharp .NET theme={null}
await database.Connect(connector, new PowerSyncConnectionOptions(
crudUploadThrottleMs: 500
));
```
#### Error Handling
If your `uploadData()` throws an error (e.g. due to a `4xx` or `5xx` response from your backend), the SDK will **retry** the same upload indefinitely, effectively blocking the upload queue. Your backend should return `2xx` for validation errors or write conflicts, and reserve error responses for transient failures. See [Writing Client Changes](/handling-writes/writing-client-changes#recommendations) and [Handling Write / Validation Errors](/handling-writes/handling-write-validation-errors) for recommended patterns.
When `uploadData()` throws an error:
1. The SDK logs the error and updates the sync status with an `uploadError`.
2. It waits for the retry delay (default: 5 seconds).
3. It retries the upload from the beginning of the queue.
4. If the sync stream disconnects during the retry wait, the upload loop exits and will resume when the connection is re-established.
#### Stalled Upload Queue Detection
If the SDK detects that the same CRUD entry is at the front of the queue across consecutive upload iterations (i.e., `uploadData()` returned without error but didn't call `.complete()` on the batch or transaction), it logs a warning:
> *"Potentially previously uploaded CRUD entries are still present in the upload queue. Make sure to handle uploads and complete CRUD transactions or batches by calling and awaiting their \[.complete()] method. The next upload iteration will be delayed."*
After logging this warning, the SDK **throws an internal error**, which causes it to enter the retry delay path (default: 5 seconds) before attempting the upload again. This prevents a tight loop when `uploadData()` consistently fails to process entries.
This typically means your `uploadData()` implementation is not calling `.complete()` after successfully processing entries. Always call `.complete()` on the `CrudBatch` or `CrudTransaction` to remove processed entries from the queue.
`uploadData()` is only called while the sync stream is connected. If the device is offline, writes accumulate in the upload queue and are uploaded automatically when connectivity is restored and the sync stream reconnects.
### When `fetchCredentials()` is Called
The PowerSync Client SDK **caches credentials internally** and `fetchCredentials` is called in the following scenarios:
1. **On initial connection** - When `connect()` is called and no cached credentials are available.
2. **Before the token expires** - The PowerSync Service sends periodic keepalive messages that include the remaining lifetime of the current JWT. When the token has **30 seconds or less** remaining, the SDK pre-fetches new credentials in the background to ensure a seamless transition.
3. **When the token has expired** - If the token expires (e.g. due to the device being offline), the SDK invalidates cached credentials and calls `fetchCredentials()` again when reconnecting.
4. **On authentication errors** - If the PowerSync Service responds with a `401` status, credentials are invalidated, prompting a fresh `fetchCredentials()` call on the next connection attempt.
For a JWT with a 1-hour expiry, `fetchCredentials()` is typically called **approximately once per hour**.
Your `fetchCredentials()` implementation should always return a fresh token when called, even if the currently cached token is not yet expired.
### Recommended JWT Expiry Duration
A JWT expiry (TTL) of **5 to 60 minutes** works well with PowerSync:
* **Shorter JWTs (5–15 minutes):** More secure with minimal overhead, since `fetchCredentials()` is only called when the token is near expiry.
* **Longer JWTs (30–60 minutes):** Fewer credential fetches and simpler backend implementation.
The SDK handles token rotation seamlessly regardless of the TTL, as long as `fetchCredentials()` can return a valid token when called.
Avoid JWTs with a TTL shorter than 30 seconds, since the SDK's pre-fetch threshold is 30 seconds - shorter TTLs would bypass pre-fetching and always hit the token-expired reconnection path.
### Token Expiry While Offline
If the JWT expires while the device is offline, the SDK handles reconnection automatically - it will call `fetchCredentials()` to obtain a new token when connectivity is restored.
The PowerSync sync token is used exclusively for the connection between the Client SDK and the PowerSync Service. It is separate from your application's own authentication session (e.g. Supabase Auth, Firebase Auth), which has its own lifecycle and refresh mechanism.
## Example Implementation
For an example implementation of a PowerSync 'backend connector', see the SDK guide for your platform:
## More Examples
For additional implementation examples, see the [Examples](/intro/examples) section.
# CloudCode for MongoDB Backends
Source: https://docs.powersync.com/configuration/app-backend/cloudcode
Use JourneyApps CloudCode as a managed backend for PowerSync with MongoDB.
We've made optional functionality available to MongoDB customers that handles the [backend integration](/configuration/app-backend/setup) required by PowerSync.
This makes PowerSync easier to implement for developers migrating from [MongoDB Atlas Device Sync](/migration-guides/atlas-device-sync) who prefer not having to maintain their own backend code and infrastructure (PowerSync's [typical architecture](/configuration/app-backend/setup) is to use your own backend to process mutations uploaded from clients, and to generate JWTs for authentication if needed).
Specifically, you can use the CloudCode feature of JourneyApps Platform, a [sibling product](https://journeyapps.com) of PowerSync. [CloudCode](https://docs.journeyapps.com/reference/cloudcode/cloudcode-overview) is a serverless cloud functions engine based on Node.js and AWS Lambda. It's provided as a fully-managed service running on the same cloud infrastructure as the rest of PowerSync Cloud. PowerSync and JourneyApps Platform share the same accounts backend, so you don’t need to create a separate account to use CloudCode. For further background, see [this post on our blog](https://www.powersync.com/blog/turnkey-backend-functionality-conflict-resolution-for-powersync).
We are currently making JourneyApps Platform CloudCode available for free to all our customers who use PowerSync with MongoDB. It does require a bit of "white glove" onboarding from our team. [Contact us](/resources/contact-us) if you want to use this functionality.
## Using CloudCode for MongoDB Backend Functionality
There is a MongoDB template available in CloudCode that provides the backend functionality needed for a PowerSync MongoDB implementation. Here is how to use it:
### Create a New JourneyApps Platform Project
To create a new JourneyApps Platform project in order to use CloudCode:
Navigate to the [JourneyApps Admin Portal](https://accounts.journeyapps.com/portal/admin). The JourneyApps Platform (of which CloudCode is a component) shares an accounts backend with PowerSync, so you can log in using the same username and password that you use for PowerSync. You should see a list of your projects if you've created any.
Select **Create Project** at the top right of the screen.
Select **JourneyApps Platform Project** and click **Next**.
Enter a project name and click **Next**.
There are options available for managing version control for the project. For simplicity we recommend selecting **Basic (Revisions)** and **JourneyApps** as the Git provider.
Select **TypeScript** as your template language, and **MongoDB CRUD & Auth Backend** as your template. Then click **Create App**.
### Overview of the CloudCode Tasks Created From the Template
To view the CloudCode tasks that were created in the new project using this template, select **CloudCode** at the top of the IDE:
Task
Used For
Description
`generate_keys`
[Authentication Integration](/configuration/app-backend/setup)
This is a task that can be used to generate a private/public key pair which the `jwks` and `token` tasks (see below) require. This task does **not** expose an HTTP endpoint and should only be used for development and getting started.
`jwks`
[Authentication Integration](/configuration/app-backend/setup)
This task [exposes an HTTP endpoint](https://docs.journeyapps.com/reference/cloudcode/triggering-a-cloudcode-task/trigger-cc-via-http) which has a `GET` function which returns the public [JWKS](https://stytch.com/blog/understanding-jwks/) details.
`token`
[Authentication Integration](/configuration/app-backend/setup)
This task exposes an HTTP endpoint which has a `GET` function. The HTTP endpoint can be called by your [`fetchCredentials()` function](/configuration/app-backend/client-side-integration) when implementing the PowerSync Client SDK, to generate a JWT that the Client SDK can use to authenticate against the PowerSync Service.
`upload`
[Client Mutations](/configuration/app-backend/setup)
This task exposes an HTTP endpoint which has a `POST` function which is used to process uploaded mutations from a PowerSync client and persist it to the source MongoDB database. The HTTP endpoint can be called by your [`uploadData()` function](/configuration/app-backend/client-side-integration) when implementing the PowerSync Client SDK.
If you will not be using [Custom Authentication](/configuration/auth/custom), you do not need the authentication-related tasks. Some [authentication providers](/configuration/auth/overview#common-authentication-providers) (e.g. Auth0, Clerk, Stytch, Keycloak, Azure AD, Google Identity, WorkOS, etc.) already generate JWTs for users which PowerSync can work with directly. If you are *not* using one of those authentication providers, you will need to implement [Custom Authentication](/configuration/auth/custom)
### Setup: Deployment Configuration
Before using the tasks, we need to configure a "deployment".
1. At the top of the IDE, select **Deployments**.
2. Create a new deployment by using the **+** button at the top right, *or* use the default `Testing` deployment. You can configure different deployments for different environments (e.g. staging, production)
3. Now select the **Deployment settings** button for the deployment.
4. In the **Deployment settings** - **General** tab, capture a **Domain** value in the text field. This domain name determines where the HTTP endpoints exposed by these CloudCode tasks can be accessed. The application will validate the domain name to make sure it's available.
5. Select **Save**.
6. Deploy the deployment: you can do so by selecting the **Deploy app** button, which can be found on the far right for each of the deployments you have configured. After the deployment is completed, it will take a few minutes for the domain to be available.
7. Your new domain will be available at `.poweredbyjourney.com`. Open the browser and navigate to the new domain. You should be presented with `Cannot GET /`, because there is no index route.
### Setup: Authentication Integration (Optional)
If you will not be using [Custom Authentication](/configuration/auth/custom), you can skip this part. See the explanatory note about authentication above.
#### 1. Generate Key Pair
First, you need to generate a public/private key pair. Do the following to generate the key pair:
1. Open the `generate_keys` CloudCode task.
2. Select the **Test CloudCode Task** button at the top right. This will print the public and private key in the task logs window.
This step is only meant for testing and development because the keys are printed in the log files.
For production, [generate a key pair locally](https://github.com/powersync-ja/powersync-jwks-example?tab=readme-ov-file#1-generate-a-key-pair) and move onto step 2 and 3.
#### 2. Configure Environment Variables
The following variables need to be set on the deployment for authentication integration purposes:
* `POWERSYNC_PUBLIC_KEY` - This is the `POWERSYNC_PUBLIC_KEY` from the values generated in step 1.
* `POWERSYNC_PRIVATE_KEY` - This is the `POWERSYNC_PRIVATE_KEY` from the values generated in step 1.
* `POWERSYNC_URL` - This is your PowerSync instance URL that can be found in the [PowerSync Dashboard](https://dashboard.powersync.com/).
See the [How to Set Environment Variables](#how-to-set-environment-variables) section below for instructions.
#### 3. Test
Open your browser and navigate to `.poweredbyjourney.com/jwks` (using the domain name you picked in [Setup: Deployment Configuration](#setup-deployment-configuration))
If the setup was successful, the `jwks` task will render the keys in JSON format. Make sure the format of your JWKS keys matches the format [in this example](https://hlstmcktecziostiaplz.supabase.co/functions/v1/powersync-jwks) JWKS endpoint.
### Setup: Handling Client Mutations
The following variables need to be set on the deployment for the purpose of handling uploaded client mutations:
* `POWERSYNC_URL` - This is your PowerSync instance URL that can be found in the [PowerSync Dashboard](https://dashboard.powersync.com/).
* `MONGO_URI` - This is the URI of your MongoDB source database e.g. `mongodb+srv://:@/`
See the next section for instructions.
### How to Set Environment Variables
To set environment variables, do the following:
1. At the top of the IDE, select **Deployments**.
2. Click on **Deployment settings** for the relevant deployment.
3. Select the **Environment Variables** tab.
Make sure you've configured a deployment and set up environment variables as described in the **Setup** steps above before using the HTTP API endpoints exposed by the CloudCode tasks
#### Token
You would call the `token` HTTP API endpoint when you [implement](/configuration/app-backend/client-side-integration) the `fetchCredentials()` function in your client application.
Send an HTTP GET request to `.poweredbyjourney.com/token?user_id=` to fetch a JWT for a user. You must provide a `user_id` in the query string of the request, as this is included in the JWT that is generated.
The response of the request will be structured like this:
```json theme={null}
{"token":"..."}
```
#### JWKS
The `jwks` HTTP API endpoint is used by PowerSync to validate the token returned from the `.poweredbyjourney.com/token` endpoint. This URL must be set in the configuration of your PowerSync instance.
Send an HTTP GET request to `.poweredbyjourney.com/jwks`.
An example of the response format can be found using [this link](https://hlstmcktecziostiaplz.supabase.co/functions/v1/powersync-jwks).
### Usage: Handling Client Mutations
#### Upload
You would call the `upload` HTTP API endpoint when you [implement](/configuration/app-backend/client-side-integration) the `uploadData()` function in your client application.
Send an HTTP POST request to `.poweredbyjourney.com/upload`
The body of the request payload should look like this:
```json theme={null}
{
"batch": [{
"op": "PUT",
"table": "lists",
"id": "61d19021-0565-4686-acc4-3ea4f8c48839",
"data": {
"created_at": "2024-10-31 10:33:24",
"name": "Name",
"owner_id": "8ea4310a-b7c0-4dd7-ae54-51d6e1596b83"
}
}]
}
```
* `batch` should be an array of mutations from the PowerSync Client SDK.
* `op` refers to the type of each mutation recorded by the PowerSync Client SDK (`PUT`, `PATCH` or `DELETE`). Refer to [Writing Data](/client-sdks/writing-data) and [Writing Client Changes](/handling-writes/writing-client-changes) for details.
* `table` refers to the table in SQLite where the mutation originates from, and should match the name of a collection in MongoDB.
The API will respond with HTTP status `200` if the write was successful.
### Customization: Handling Client Mutations
You can make changes to the way the `upload` task writes data to the source MongoDB database.
Here is how:
1. Go to **CloudCode** at the top of the IDE in your JourneyApps Platform project
2. Select and expand the `upload` task in the panel on the left.
3. The `index.ts` contains the entry point function that accepts the HTTP request and has a `MongoDBStorage` class which interacts with the MongoDB database to perform inserts, updates and deletes. To adjust how mutations are performed, take a look at the `updateBatch` function.
### Production Considerations
Before going into production with this solution, you will need to set up authentication on the HTTP endpoints exposed by the CloudCode tasks.
If you need more data validations and/or authorization than what is provided by the template, that will need to be customized too. Consider introducing schema validation of the data being written to the source MongoDB database. You should use a [purpose-built](https://json-schema.org/tools?query=\&sortBy=name\&sortOrder=ascending\&groupBy=toolingTypes\&licenses=\&languages=\&drafts=\&toolingTypes=\&environments=\&showObsolete=false) library for this, and use [MongoDB Schema Validation](https://www.mongodb.com/docs/manual/core/schema-validation/) to enforce the types in the database.
Please [contact us](/resources/contact-us) for assistance on any of the above.
# App Backend Setup
Source: https://docs.powersync.com/configuration/app-backend/setup
Configure your app backend to process client-side mutations from the PowerSync upload queue.
PowerSync generally assumes that you have some kind of "backend application" as part of your overall application architecture — whether it's a backend-as-a-service (e.g. Supabase), a custom backend (e.g. Node.js, Rails, Laravel, Django, ASP.NET), some kind of serverless cloud functions (e.g. Azure Functions, AWS Lambda, Google Cloud Functions, Cloudflare Workers, etc.), or any other equivalent system that allows you to run privileged logic securely.
When you integrate PowerSync into your app project, PowerSync relies on that "backend application" for a few potential purposes:
1. **Allowing client-side mutations to be uploaded** and [applied](/handling-writes/writing-client-changes) to the backend source database (Postgres, MongoDB, MySQL, SQL Server, or Convex). When you write to the client-side SQLite database provided by PowerSync, those mutations are also placed into an [upload queue](/architecture/client-architecture#writing-data-via-sqlite-database-and-upload-queue). The PowerSync Client SDK manages uploading of those mutations to your backend using the `uploadData()` function that you define in your [client-side](/configuration/app-backend/client-side-integration) *backend connector* implementation. Your `uploadData()` implementation should call your backend application API to apply the mutations to your source database. The reason why we designed PowerSync this way is to give you full control over things like server-side data validation and authorization of mutations, while PowerSync itself requires minimal permissions.
2. **Authentication integration (optional):** *If* you are implementing custom authentication (see below), your backend is responsible for securely generating the [JWTs](/configuration/auth/overview) used by the PowerSync Client SDK to authenticate with the [PowerSync Service](/architecture/powersync-service).
If you will only use the backend for applying mutations and not for authentication, you can also use some kind of data API service or API platform (e.g. Hasura).
A quick way to get started during development before implementing custom auth is to use [Development Tokens](/configuration/auth/development-tokens)
When you set up custom authentication, you define the [`fetchCredentials()` function](/configuration/app-backend/client-side-integration) in your *backend connector* to retrieve a JWT from your backend application API, making use of your [existing app-to-backend](/configuration/app-backend/setup) authentication:
This is the recommended approach for production environments. Asymmetric keys provide better security by separating signing (private key) from verification (public key), making key rotation easier and more secure.
A key pair (private + public key) is required to sign and verify JWTs. The private key is used to sign the JWT, and the public key is used to verify it.
PowerSync requires the public key(s) to be specified in [JSON Web Key Set (JWKS)](https://auth0.com/docs/secure/tokens/json-web-tokens/json-web-key-sets) format.
The JWKS can be configured in one of two ways:
* Expose the JWKS on a public URL. PowerSync fetches the keys from this endpoint. We have an example endpoint available [here](https://hlstmcktecziostiaplz.supabase.co/functions/v1/powersync-jwks) — ensure that your response looks similar. This option is useful if you're using an external authentication service with an existing JWKS endpoint and you want to automate key rotation without manual deploys.
* Configure the JWKS directly. Provide the keys directly in your PowerSync instance configuration. This option is useful if you generate tokens yourself and want simpler setup.
Requirements for the public key in the JWKS:
1. Supported signature schemes: RSA, EdDSA and ECDSA.
2. Key type (`kty`): `RSA`, `OKP` (EdDSA) or `EC` (ECDSA).
3. Algorithm (`alg`):
1. `RS256`, `RS384` or `RS512` for RSA
2. `EdDSA` for EdDSA
3. `ES256`, `ES384` or `ES512` for ECDSA
4. Curve (`crv`) - only relevant for EdDSA and ECDSA:
1. `Ed25519` or `Ed448` for EdDSA
2. `P-256`, `P-384` or `P-512` for ECDSA
5. A `kid` must be specified and must match the `kid` in the JWT.
Refer to [this example](https://github.com/powersync-ja/powersync-jwks-example) for creating and verifying JWTs for PowerSync authentication.
Since there is no way to revoke a JWT once issued without rotating the key, we recommend using short expiration periods (e.g. 5 minutes). JWTs older than 60 minutes are not accepted by PowerSync.
### Rotating Keys
If a private key is compromised, rotate the key in the JWKS.
The rotation process differs depending on your JWKS configuration method:
#### JWKS on a Public URL
When using a JWKS exposed on a public URL, PowerSync refreshes the keys from the endpoint every few minutes and will detect new keys immediately.
There is a possibility of false authentication errors until PowerSync refreshes the keys. These errors are typically retried by the client and will have little impact. However, to periodically rotate keys without any authentication failures, follow this process:
1. Add a new key to the JWKS at your endpoint.
2. Wait 5 minutes to ensure PowerSync has fetched the new key.
3. Start signing new JWT tokens using the new key.
4. Wait until all existing tokens have expired.
5. Remove the old key from your JWKS endpoint.
#### Direct JWKS Configuration
When the JWKS is configured directly in PowerSync (not via a public URL), you must deploy configuration changes for PowerSync to use the new key:
1. Add the new key to your JWKS configuration.
2. Deploy the configuration changes (via the **Save and Deploy** button in the PowerSync Dashboard, or restart the PowerSync Service for self-hosted).
3. Start signing new JWT tokens using the new key.
4. Wait until all existing tokens have expired.
5. Remove the old key from the JWKS configuration and deploy again.
### PowerSync Cloud Configuration
1. In the [PowerSync Dashboard](https://dashboard.powersync.com/), select your project and instance and go to the **Client Auth** view.
2. Configure your JWKS and audience settings. You can either configure the JWKS directly in JSON format (use the **JWKS** section), or configure a **JWKS URI**.
3. Click **Save and Deploy** to apply the changes.
### Self-Hosted Configuration
You can configure authentication using either:
* A JWKS URI endpoint
* Static public keys in the configuration file
This can be configured via your [`service.yaml`](/configuration/powersync-service/self-hosted-instances):
```yaml service.yaml theme={null}
client_auth:
# Option 1: JWKS URI endpoint
jwks_uri: http://demo-backend:6060/api/auth/keys
# Option 2: Static collection of public keys for JWT verification
# jwks:
# keys:
# - kty: 'RSA'
# n: '[rsa-modulus]'
# e: '[rsa-exponent]'
# alg: 'RS256'
# kid: '[key-id]'
audience: ['powersync-dev', 'powersync']
```
## Option 2: Symmetric JWTs — Using HS256
Using shared secrets (HS256) for JWT signing is generally not recommended for production environments due to security risks. We recommend using asymmetric keys (Option 1) instead, which provide better security through public/private key separation.
PowerSync supports HS256 symmetric JWTs for development and testing purposes.
### Generating a Shared Secret
You can generate a shared secret in the terminal using the following command:
```bash theme={null}
openssl rand -base64 32
```
### Base64 URL Encode the Shared Secret
Once you've generated the shared secret, you will need to Base64 URL encode it before setting it in the PowerSync instance Client Auth configuration.
You can use the following command to Base64 URL encode the shared secret:
```bash theme={null}
echo -n "your-value-here" | base64 -w 0 | tr '+/' '-_' | tr -d '='
```
### Set the Shared Secret in the PowerSync Instance
1. Go to the [PowerSync Cloud Dashboard](https://dashboard.powersync.com/) and select your project and instance.
2. Go to the **Client Auth** view.
3. Find the section labeled **HS256 Authentication Tokens (ADVANCED)** and click **+** button to add a new token.
4. Set the **KID** to a unique identifier for the token (you'll use the same KID to sign the token). Set the **Shared Secret** to the Base64 URL encoded shared secret.
5. Click **Save and Deploy**.
1. Add the shared secret to your PowerSync Service configuration file, e.g.:
```yaml service.yaml theme={null}
client_auth:
jwks:
keys:
- kty: oct
alg: 'HS256'
kid: '[key-id]'
k: '[base64url-encoded-shared-secret]'
```
2. Restart the PowerSync Service.
### Generate New JWTs Using the KID and Shared Secret
Using your newly-created shared secret, you can generate JWT tokens [in your backend](/configuration/app-backend/setup) using the same KID you set in the PowerSync Service configuration. Here's a example TypeScript function using the [`jose`](https://github.com/panva/jose) library:
```typescript theme={null}
import * as jose from 'jose';
export const generateToken = async (payload: Record, userId: string) => {
return await new jose.SignJWT(payload)
.setProtectedHeader({ alg: 'HS256', kid: 'your-kid' })
.setSubject(userId)
.setIssuer('https://your-domain.com')
.setAudience('https://your-powersync-instance.com')
.setExpirationTime('60m')
// Note: The shared secret should be read from a secure source or environment variable and not hardcoded.
.sign(Buffer.from('your-base64url-encoded-shared-secret', 'base64url'));
};
```
This JWT can then be used to authenticate with the PowerSync Service. In your [`fetchCredentials()` function](/configuration/app-backend/client-side-integration), you will need to retrieve the token from your backend API.
# Development Tokens
Source: https://docs.powersync.com/configuration/auth/development-tokens
Generate temporary development tokens for testing PowerSync authentication without a full auth provider.
PowerSync allows generating temporary development tokens for authentication.
This is useful for developers who want to get up and running quickly, without full auth integration.
This can also be used to generate a token for a specific user to debug issues.
## Generating a Development Token
1. In the [PowerSync Dashboard](https://dashboard.powersync.com/), select your project and instance
2. Go to the **Client Auth** view
3. Check the **Development tokens** setting and save your changes
4. Click the **Connect** button in the top bar
5. Enter a user ID:
* If your Sync Streams/Rules data isn't filtered by user (same data syncs to all users), you can use any value (e.g., `test-user`).
* If your data is filtered by parameters , use a user ID that matches a user in your database. PowerSync uses this (e.g. `auth.user_id()` in Sync Streams or `request.user_id()` in Sync Rules) to determine what to sync.
6. Click **Generate Token** and copy the token
Development tokens expire after 12 hours.
Follow the steps below. Steps 1 and 2 configure signing keys and your PowerSync Service config; in Step 3 you can use the PowerSync CLI (recommended) or the [test-client](https://github.com/powersync-ja/powersync-service/tree/main/test-client) to generate the token.
Generate a temporary private/public key-pair (RS256) or shared key (HS256) for JWT signing and verification.
Use an online JWK generator like [mkjwk.org](https://mkjwk.org/) (select RSA, 2048 bits, Signature use, RS256 algorithm).
Or generate locally with Node.js:
```bash theme={null}
# Install pem-jwk if needed
npm install -g pem-jwk
# Generate private key
openssl genrsa -out private-key.pem 2048
# Convert public key to JWK format
openssl rsa -in private-key.pem -pubout | pem-jwk
```
Use an online JWK generator like [mkjwk.org](https://mkjwk.org/) (select oct, 256 bits, Signature use, HS256 algorithm) - this outputs base64url directly.
Or generate and convert using OpenSSL:
```bash theme={null}
# Generate and convert to base64url
openssl rand -base64 32 | tr '+/' '-_' | tr -d '='
```
For production environments, shared secrets (HS256) are not recommended.
Add the `client_auth` parameter to your PowerSync config (e.g. `service.yaml`):
Copy the JWK values from [mkjwk.org](https://mkjwk.org/) or the `pem-jwk` output, then add to your config:
```yaml service.yaml theme={null}
# Client (application end user) authentication settings
client_auth:
# static collection of public keys for JWT verification
jwks:
keys:
- kty: 'RSA'
n: '[rsa-modulus]'
e: '[rsa-exponent]'
alg: 'RS256'
kid: 'dev-key-1'
```
Copy the `k` value from mkjwk.org or the OpenSSL output, then add to your config:
```yaml service.yaml theme={null}
# Client (application end user) authentication settings
client_auth:
audience: ['http://localhost:8080', 'http://127.0.0.1:8080']
# static collection of public keys for JWT verification
jwks:
keys:
- kty: oct
alg: 'HS256'
k: '[base64url-encoded-shared-secret]'
kid: 'dev-key-1'
```
These examples use static `jwks: keys:` for simplicity. For production, we recommend using `jwks_uri` to point to a JWKS endpoint instead. See [Custom Authentication](/configuration/auth/custom) for more details.
Choose either the [PowerSync CLI](/tools/cli) (recommended) or the test-client:
Apply your config changes (e.g. restart your PowerSync Service or run `powersync docker reset` if running locally with Docker), then run:
```bash theme={null}
powersync generate token --subject=test-user
```
Replace `test-user` with the user ID you want to authenticate:
* If your Sync Streams aren't filtered by user (same data syncs to all users), you can use any value (e.g., `test-user`).
* If your data is filtered by parameters , use a user ID that matches a user in your database. PowerSync uses this value (e.g. via `auth.user_id()`) to determine what to sync.
1. If you have not done so already, clone the [`powersync-service` repo](https://github.com/powersync-ja/powersync-service/tree/main)
2. Install and build:
* In the project root: `pnpm install` and `pnpm build`
* In the `test-client` directory: `pnpm build`
3. Generate a token from the `test-client` directory, pointing at your config file:
```bash theme={null}
node dist/bin.js generate-token --config path/to/service.yaml --sub test-user
```
If your config uses `!env PS_*` tags to reference environment variables, the test-client reads them from your shell. To load them from a file instead, pass `--env`:
```bash theme={null}
node dist/bin.js generate-token --config path/to/service.yaml --env path/to/.env --sub test-user
```
Replace `test-user` with the user ID you want to authenticate:
* If your Sync Streams aren't filtered by user (same data syncs to all users), you can use any value (e.g., `test-user`).
* If your data is filtered by parameters , use a user ID that matches a user in your database. PowerSync uses this value (e.g. via `auth.user_id()`) to determine what to sync.
Development tokens expire after 12 hours.
## Usage
Development tokens can be used for testing purposes either with the [Sync Diagnostics Client](https://diagnostics-app.powersync.com), the [test-client](https://github.com/powersync-ja/powersync-service/tree/main/test-client), or your app itself (for development purposes).
### Using with Sync Diagnostics Client
The [Sync Diagnostics Client](https://diagnostics-app.powersync.com) allows you to quickly test syncing and inspect a user's SQLite database, to verify that your PowerSync Service configuration and Sync Streams / Sync Rules behave as expected.
1. Open the [Sync Diagnostics Client](https://diagnostics-app.powersync.com)
2. Enter the generated development token at **PowerSync Token**.
3. Enter your PowerSync Service endpoint URL at **PowerSync Endpoint** unless already prepopulated.
4. Click **Proceed**.
5. Wait for the syncing to complete and inspect the synced data in SQLite.
### Using with `test-client`
The [test-client](https://github.com/powersync-ja/powersync-service/tree/main/test-client) is useful for testing of syncing without persisting anything to a client-side SQLite database. Amongst other things, it can be used for load testing, simulating many client syncing concurrently. Consult the [README](https://github.com/powersync-ja/powersync-service/tree/main/test-client#readme) for details on how to provide the development token as argument to `test-client` supported commands.
### Using with Your Application
To use the temporary development token in your application, update the [`fetchCredentials()` function](/configuration/app-backend/client-side-integration) in your *backend connector* to return the generated token.
```typescript React Native, Web & Capacitor (TS) theme={null}
async fetchCredentials(): Promise {
// for development: use development token
return {
endpoint: 'https://your-instance.powersync.com',
token: 'your-development-token-here'
};
}
```
```typescript Node.js (TS) theme={null}
async fetchCredentials() {
// for development: use development token
return {
endpoint: 'https://your-instance.powersync.com',
token: 'your-development-token-here'
};
}
```
```kotlin Kotlin theme={null}
override suspend fun fetchCredentials(): PowerSyncCredentials {
// for development: use development token
return PowerSyncCredentials(
endpoint = "https://your-instance.powersync.com",
token = "your-development-token-here"
)
}
```
```swift Swift theme={null}
func fetchCredentials() async throws -> PowerSyncCredentials {
// for development: use development token
return PowerSyncCredentials(
endpoint: "https://your-instance.powersync.com",
token: "your-development-token-here"
)
}
```
```dart Dart/Flutter theme={null}
@override
Future fetchCredentials() async {
return PowerSyncCredentials(
endpoint: 'https://your-instance.powersync.com',
token: 'your-development-token-here'
);
}
```
```csharp .NET theme={null}
public async Task FetchCredentials()
{
var powerSyncUrl = "https://your-instance.powersync.com";
var authToken = "your-development-token-here";
// Return credentials with PowerSync endpoint and JWT token
return new PowerSyncCredentials(powerSyncUrl, authToken);
}
```
```rust Tauri (Rust) theme={null}
// For Tauri, fetchCredentials is implemented inside your Rust BackendConnector
async fn fetch_credentials(&self) -> Result {
Ok(PowerSyncCredentials {
endpoint: "https://your-instance.powersync.com".to_string(),
token: "your-development-token-here".to_string(),
})
}
```
# Firebase Auth
Source: https://docs.powersync.com/configuration/auth/firebase-auth
Configure Firebase Authentication for PowerSync.
Configure authentication on the PowerSync instance with the following settings:
* **JWKS URI**: `https://www.googleapis.com/service_accounts/v1/jwk/securetoken@system.gserviceaccount.com`
* **JWT Audience**: Firebase project ID
Firebase signs these tokens using RS256.
PowerSync will periodically refresh the keys using the above JWKS URI, and validate tokens against the configured audience (token `aud` value).
The Firebase user UID will be available as:
* `auth.user_id()` in [Sync Streams](/sync/streams/overview) (recommended)
* `request.user_id()` in [Sync Rules](/sync/rules/overview) (previously `token_parameters.user_id`)
To use a different identifier as the user ID in Sync Streams / Sync Rules (for example, user email), use [Custom Authentication](/configuration/auth/custom).
### PowerSync Cloud Configuration
1. In the [PowerSync Dashboard](https://dashboard.powersync.com/), select your project and instance and go to the **Client Auth** view.
2. Configure your Firebase **JWKS URI** and **JWT Audience** settings.
3. Click **Save and Deploy** to apply the changes.
### Self-Hosted Configuration
This can be configured via your [`service.yaml`](/configuration/powersync-service/self-hosted-instances):
```yaml service.yaml theme={null}
client_auth:
# JWKS URIs can be specified here.
jwks_uri: 'https://www.googleapis.com/service_accounts/v1/jwk/securetoken@system.gserviceaccount.com'
audience: ['']
```
# Authentication Setup
Source: https://docs.powersync.com/configuration/auth/overview
Set up JWT-based client authentication for PowerSync using development tokens, supported auth providers like Auth0 and Firebase, or custom JWTs.
## Client Authentication
PowerSync clients (i.e. apps used by your users that embed the PowerSync Client SDK) authenticate against the server-side [PowerSync Service](/architecture/powersync-service) using [JWTs](https://jwt.io/) (signed tokens).
Scroll the table horizontally.
| Provider | Configuration Notes | Documentation | JWKS URL |
| ----------------------------------------- | ---------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------- |
| **Supabase** | Uses Supabase's **JWT Secret** | [Supabase Auth Setup](/configuration/auth/supabase-auth) | Direct integration available |
| **Firebase Auth / GCP Identity Platform** | JWT Audience: Firebase project ID | [Firebase Auth Setup](/configuration/auth/firebase-auth) | `https://www.googleapis.com/service_accounts/v1/jwk/securetoken@system.gserviceaccount.com` |
| **Auth0** | JWT Audience: PowerSync instance URL | [Auth0 Setup](/configuration/auth/auth0) | `https://{auth0-domain}/.well-known/jwks.json` |
| **Clerk** | Additional configuration may be required | [Clerk Documentation](https://clerk.com/docs/backend-requests/making/jwt-templates#create-a-jwt-template) | `https://{yourClerkDomain}/.well-known/jwks.json` |
| **Stytch** | Additional configuration may be required | [Stytch Documentation](https://stytch.com/docs/api/jwks-get) | `https://{live_or_test}.stytch.com/v1/sessions/jwks/{project-id}` |
| **Keycloak** | Additional configuration may be required | [Keycloak Documentation](https://documentation.cloud-iam.com/how-to-guides/configure-remote-jkws.html) | `https://{your-keycloak-domain}/auth/realms/{realm-name}/protocol/openid-connect/certs` |
| **Amazon Cognito** | Additional configuration may be required | [Cognito Documentation](https://docs.aws.amazon.com/cognito/latest/developerguide/amazon-cognito-user-pools-using-tokens-verifying-a-jwt.html) | `https://cognito-idp.{region}.amazonaws.com/{userPoolId}/.well-known/jwks.json` |
| **Azure AD** | Additional configuration may be required | [Azure AD Documentation](https://learn.microsoft.com/en-us/entra/identity-platform/access-tokens) | `https://login.microsoftonline.com/{tenantId}/discovery/v2.0/keys` |
| **Google Identity** | Additional configuration may be required | [Google Identity Documentation](https://developers.google.com/identity/openid-connect/openid-connect#discovery) | `https://www.googleapis.com/oauth2/v3/certs` |
| **SuperTokens** | Additional configuration may be required | [SuperTokens Documentation](https://supertokens.com/docs/quickstart/integrations/aws-lambda/session-verification/using-jwt-authorizer) | `https://{YOUR_SUPER_TOKENS_CORE_CONNECTION_URI}/.well-known/jwks.json` |
| **WorkOS** | Additional configuration may be required | [WorkOS Documentation](https://workos.com/docs/reference/user-management/session-tokens/jwks) | `https://api.workos.com/sso/jwks/{YOUR_CLIENT_ID}` |
| **Custom JWT** | See custom auth requirements | [Custom Auth Setup](/configuration/auth/custom) | Your own JWKS endpoint |
# Supabase Auth
Source: https://docs.powersync.com/configuration/auth/supabase-auth
Use Supabase Auth with PowerSync by verifying Supabase JWTs directly.
You can implement various types of authentication when using PowerSync with Supabase:
#### Standard [Supabase Auth](https://supabase.com/docs/guides/auth)
These examples show how to implement [`fetchCredentials()` in your client-side *backend connector*](/configuration/app-backend/client-side-integration#backend-connector) to get the Supabase JWT from the Supabase Client Library:
* [JavaScript example](https://github.com/powersync-ja/powersync-js/blob/58fd05937ec9ac993622666742f53200ee694585/demos/react-supabase-todolist/src/library/powersync/SupabaseConnector.ts#L87)
* [Dart/Flutter example](https://github.com/powersync-ja/powersync.dart/blob/9ef224175c8969f5602c140bcec6dd8296c31260/demos/supabase-todolist/lib/powersync.dart#L38)
* [Kotlin example](https://github.com/powersync-ja/powersync-kotlin/blob/4f60e2089745dda21b0d486c70f47adbbe24d289/connectors/supabase/src/commonMain/kotlin/com/powersync/connector/supabase/SupabaseConnector.kt#L75)
#### Anonymous Sign-Ins
This examples shows use of Supabase's `signInAnonymously()` and then the implementation of [`fetchCredentials()`](/configuration/app-backend/client-side-integration#backend-connector) to get the JWT from the Supabase Client Library:
* [JavaScript example](https://github.com/powersync-ja/powersync-js/blob/58fd05937ec9ac993622666742f53200ee694585/demos/react-multi-client/src/library/SupabaseConnector.ts#L47)
#### Fully Custom Auth
This example shows how to implement Supabase Edge Functions to generate custom JWTs for PowerSync (either for signed-in users or anonymous users) as well as expose a JWKS endpoint:
* [Example](https://github.com/powersync-ja/powersync-jwks-example/)
#### External Auth Providers
We've heard from the community that Supabase's [support for third-party auth providers](https://supabase.com/blog/third-party-auth-mfa-phone-send-hooks) works with PowerSync, but we don't have any examples for this yet.
## Supabase JWT Signing Keys
Supabase supports two types of JWT signing keys:
| Type | Algorithm | Description |
| --------------------------- | ------------------ | ----------------------------------------------------------------------------------------------------------------------------------------- |
| **Legacy JWT signing keys** | HS256 (symmetric) | Uses a shared secret to sign and verify tokens. This is the original method. |
| **New JWT signing keys** | RS256 (asymmetric) | Uses public/private key pairs. Supabase signs tokens with a private key, and PowerSync verifies them using a public key fetched via JWKS. |
PowerSync supports both methods. Which configuration you need depends on your Supabase project's JWT settings and your PowerSync deployment type.
To check which signing keys your Supabase project uses, go to [Project Settings > JWT](https://supabase.com/dashboard/project/_/settings/jwt) in your Supabase Dashboard.
## PowerSync Cloud
When using PowerSync Cloud with a Supabase-hosted database, PowerSync can auto-detect your Supabase project from the database connection string and configure authentication automatically.
### Using New JWT Signing Keys
This is the recommended approach for Supabase projects using asymmetric JWT signing keys.
1. In the [PowerSync Dashboard](https://dashboard.powersync.com/), select your project and instance and go to the **Client Auth** view.
2. Enable the **Use Supabase Auth** checkbox.
3. Leave the **Supabase JWT Secret** field empty (it's not needed for new keys).
4. Click **Save and Deploy**.
PowerSync auto-detects your Supabase project from the database connection string and configures the JWKS URI (`https://.supabase.co/auth/v1/.well-known/jwks.json`) and JWT audience (`authenticated`) automatically.
### Using Legacy JWT Signing Keys
Legacy JWT signing keys use HS256 (symmetric encryption with shared secrets), which is less secure than asymmetric keys. We recommend migrating to [new JWT signing keys](#migrating-from-legacy-to-new-jwt-signing-keys) for better security.
Use this approach if your Supabase project still uses the legacy HS256 symmetric signing keys.
1. In the [PowerSync Dashboard](https://dashboard.powersync.com/), select your project and instance and go to the **Client Auth** view.
2. Enable the **Use Supabase Auth** checkbox.
3. Copy your **JWT Secret** from your Supabase project's [JWT settings](https://supabase.com/dashboard/project/_/settings/jwt).
4. Paste the secret into the **Supabase JWT Secret (optional) Legacy** field.
5. Click **Save and Deploy**.
### Manual JWKS Configuration
Use manual configuration when PowerSync cannot auto-detect your Supabase project. This happens when:
* You're using a non-standard database connection string
* You're connecting to a self-hosted Supabase instance
* You're using Supabase local development (Docker)
Steps:
1. In the [PowerSync Dashboard](https://dashboard.powersync.com/), go to the **Client Auth** view.
2. Leave the **Use Supabase Auth** checkbox **unchecked**.
3. Add a **JWKS URI**, e.g.:
```
http://localhost:54321/auth/v1/.well-known/jwks.json
```
4. Add `authenticated` as an accepted **JWT Audience**.
5. Click **Save and Deploy**.
If you skip adding the `authenticated` audience, you will see `PSYNC_S2105` errors ("JWT payload is missing a required claim 'aud'").
## Self-Hosted PowerSync
For self-hosted PowerSync instances, configure authentication in your [`service.yaml`](/configuration/powersync-service/self-hosted-instances).
### Using New JWT Signing Keys
When using a standard Supabase connection string, PowerSync auto-detects your Supabase project:
```yaml theme={null}
client_auth:
supabase: true
```
PowerSync will automatically configure:
* **JWKS URI**: `https://.supabase.co/auth/v1/.well-known/jwks.json`
* **Audience**: `authenticated`
You'll see a log message confirming the configuration:
```
Configured Supabase Auth with https://.supabase.co/auth/v1/.well-known/jwks.json
```
### Using Legacy JWT Signing Keys
Legacy JWT signing keys use HS256 (symmetric encryption with shared secrets), which is less secure than asymmetric keys. We recommend migrating to [new JWT signing keys](#migrating-from-legacy-to-new-jwt-signing-keys) for better security.
For projects using legacy HS256 symmetric signing keys, provide your JWT secret:
```yaml theme={null}
client_auth:
supabase: true
supabase_jwt_secret: your-jwt-secret-here
```
Get your JWT secret from your Supabase project's [JWT settings](https://supabase.com/dashboard/project/_/settings/jwt).
### Manual JWKS Configuration
Use manual configuration in any of these scenarios:
* PowerSync cannot detect your Supabase project from the connection string
* You're using self-hosted Supabase
* You're using Supabase local development (Docker)
* You need explicit control over the authentication settings
```yaml theme={null}
client_auth:
jwks_uri: http://localhost:54321/auth/v1/.well-known/jwks.json
audience:
- authenticated
```
When using manual configuration, do not set `supabase: true`. Use `jwks_uri` and `audience` directly.
## Migrating from Legacy to New JWT Signing Keys
If you're migrating your Supabase project from legacy JWT signing keys to the new asymmetric keys:
### Step 1: Complete the Supabase Migration
Follow **all steps** in [Supabase's JWT signing keys migration guide](https://supabase.com/blog/jwt-signing-keys#start-using-asymmetric-jwts-today), including the **"Rotate to asymmetric JWTs"** step.
The migration is not complete until you complete the "Rotate to asymmetric JWTs" step in Supabase. Skipping this step will cause authentication failures.
### Step 2: Update PowerSync Configuration
**For PowerSync Cloud and self-hosted with standard Supabase connections:**
* No changes required. PowerSync auto-detects and uses the new JWKS endpoint.
* If you previously provided a legacy JWT secret, you can remove it (it's no longer needed).
**For manual JWKS configurations:**
* Ensure your **JWKS URI** (`jwks_uri`) points to your Supabase JWKS endpoint.
* Verify the `authenticated` **JWT Audience** (`audience`) is configured.
### Step 3: Clear Cached Tokens
Have all users sign out and sign back in. This ensures they receive new tokens signed with the asymmetric keys.
## Troubleshooting
Debugging [error codes](/debugging/error-codes):
### `PSYNC_S2101`: Could not find an appropriate key in the keystore
This error indicates PowerSync cannot verify the JWT signature. Common causes:
| Cause | Solution |
| ---------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| **Incomplete Supabase migration** | Complete the "Rotate to asymmetric JWTs" step in Supabase's [migration guide](https://supabase.com/blog/jwt-signing-keys#start-using-asymmetric-jwts-today). |
| **Cached tokens** | Have users sign out and sign back in to get fresh tokens. |
| **Non-standard connection string** | PowerSync couldn't auto-detect your Supabase project. Use [manual JWKS configuration](#manual-jwks-configuration). |
| **Wrong JWT secret** | For legacy keys, verify the JWT secret matches your Supabase project settings. |
### `PSYNC_S2105`: JWT payload is missing a required claim "aud"
This error occurs when using manual JWKS configuration without specifying an audience. Add `authenticated` to your audience configuration.
### Auto-Detection Not Working
If PowerSync logs this warning:
```
Supabase Auth is enabled, but no Supabase connection string found. Skipping Supabase JWKS URL configuration.
```
This means PowerSync couldn't detect your Supabase project from the database connection string. Use [manual JWKS configuration](#manual-jwks-configuration) instead.
## Sync Streams
The Supabase user UUID will be available as:
* `auth.user_id()` in [Sync Streams](/sync/streams/overview).
* `request.user_id()` in [Sync Rules](/sync/rules/overview)
To use a different identifier as the user ID in Sync Streams / Sync Rules (for example, user email), use [Custom Authentication](/configuration/auth/custom).
# Stytch + Supabase
Source: https://docs.powersync.com/configuration/auth/supabase-auth/stytch
Set up Stytch authentication with Supabase for PowerSync.
## Consumer Authentication
See this community project for detailed setup instructions:
When creating a project in the [PowerSync Dashboard](https://dashboard.powersync.com/), *Development* and *Production* instances of the PowerSync Service will be created by default. Select the instance you want to configure.
If you need to create a new instance, follow the steps below.
1. In the dashboard, select your project and open the instance selection dropdown. Click **Add Instance**.
Install the [PowerSync CLI](/tools/cli) (requires Node.js/npm), then log in and scaffold the config directory:
```bash theme={null}
npm install -g powersync
powersync login
powersync init cloud
```
This creates a `powersync/` directory with `service.yaml` (instance name, region, connection, auth) and `sync-config.yaml` (Sync Config). Edit `powersync/service.yaml` to set your instance name and region.
Then create the instance and link the directory to it:
```bash theme={null}
powersync link cloud --create --project-id=
```
Find your project ID in the [PowerSync Dashboard](https://dashboard.powersync.com) URL, or run `powersync fetch instances` after logging in.
## Configure Your Instance
A new instance needs three things before clients can sync:
1. A connection to your source database. See [Source Database Connection](/configuration/source-db/connection).
2. Client authentication. See [Authentication Setup](/configuration/auth/overview). When getting started, you can use temporary [development tokens](/configuration/auth/development-tokens) instead of setting up a full auth provider.
3. A sync configuration defining what data syncs to which clients, written as [Sync Streams](/sync/streams/overview).
In the Dashboard, each of these has its own view in the instance's sidebar. With the CLI, the database connection and client auth settings live in `service.yaml` and the sync configuration in `sync-config.yaml`; run `powersync deploy` to apply changes.
Other lower-level instance options, such as compacting buckets and deprovisioning, are available in the Dashboard under the instance's Settings view. The CLI has corresponding commands for most of these operations.
For more information, see:
* [PowerSync Dashboard](/tools/powersync-dashboard) documentation
* [PowerSync CLI](/tools/cli) documentation
# Self-Hosted Instance Configuration
Source: https://docs.powersync.com/configuration/powersync-service/self-hosted-instances
How to configure a self-hosted PowerSync Service instance with a config file, and a reference of all available options.
A PowerSync instance is a running deployment of the [PowerSync Service](/architecture/powersync-service): it replicates data from your source database and streams it to clients based on your [Sync Streams](/sync/streams/overview). When self-hosting, you run the Service in your own infrastructure and configure each instance with a config file, as described below.
## Configure Your Instance
A self-hosted instance needs four things before clients can sync:
1. A connection to your [source database](/configuration/source-db/setup), which PowerSync replicates data from.
2. A [bucket storage](/architecture/powersync-service#bucket-storage) database, where the PowerSync Service stores the data it prepares for syncing to clients. MongoDB and Postgres are supported.
3. [Client authentication](/configuration/auth/overview). When getting started, you can use temporary [development tokens](/configuration/auth/development-tokens) instead of setting up a full auth provider.
4. A sync configuration defining what data syncs to which clients, written as [Sync Streams](/sync/streams/overview).
You define all of these in the main config file, along with some operational settings. We recommend keeping the sync configuration in a separate file that the main config references, though it can also be defined inline. The skeleton below shows the most common options; the [Configuration Reference](#configuration-reference) documents all of them.
```yaml service.yaml theme={null}
# Settings for source database replication
replication:
# Specify database connection details
# Note only 1 connection is currently supported
# Multiple connection support is on the roadmap
connections:
- type: postgresql
# The PowerSync server container can access the Postgres DB via the DB's service name.
# In this case the hostname is pg-db
# The connection URI or individual parameters can be specified.
uri: postgresql://postgres:mypassword@pg-db:5432/postgres
# SSL settings
sslmode: disable # 'verify-full' (default) or 'verify-ca' or 'disable'
# Note: 'disable' is only suitable for local/private networks, not for public networks
# Connection settings for bucket storage (MongoDB and Postgres are supported)
storage:
# Option 1: MongoDB Storage
type: mongodb
uri: mongodb://mongo:27017/powersync_demo
# Use these if authentication is required. The user should have `readWrite` and `dbAdmin` roles
# username: myuser
# password: mypassword
# Option 2: Postgres Storage
# type: postgresql
# This accepts the same parameters as a Postgres replication source connection
# uri: postgresql://powersync_storage_user:secure_password@storage-db:5432/postgres
# sslmode: disable
# The port which the PowerSync API server will listen on (defaults to 8080)
port: 8080
# Sync configuration (see the sync_config section below)
sync_config:
path: sync-config.yaml
# Settings for client authentication
client_auth:
# Enable this if using Supabase Auth
# supabase: true
# supabase_jwt_secret: your-secret
# JWKS URIs can be specified here.
jwks_uri: http://demo-backend:6060/api/auth/keys
# JWKS audience
audience: ['powersync-dev', 'powersync']
# Settings for telemetry reporting
# See https://docs.powersync.com/maintenance-ops/self-hosting/usage-reporting
telemetry:
# Opt out of reporting anonymized usage metrics to PowerSync telemetry service
disable_telemetry_sharing: false
# System-level configuration options
system:
# Service logging configuration
logging:
# Log level for the Service logs
level: info # 'silly', 'debug', 'verbose', 'http', 'info', 'warn', 'error'
format: text # 'json' or 'text'
```
The config used by our `self-host-demo` app. Use the demo as a working reference for your own setup.
## Supplying the Config File
Both YAML and JSON config files are supported. The PowerSync Service can read the config in three ways:
1. From a config file mounted on a volume
2. From an environment variable containing the Base64 encoding of the config file
3. From a command line parameter (also Base64 encoded)
You can see examples of these methods in the [docker-compose](https://github.com/powersync-ja/self-host-demo/blob/d61cea4f1e0cc860599e897909f11fb54420c3e6/docker-compose.yaml#L46) file of our `self-host-demo` app.
### Environment Variable Substitution
The config file uses custom tags for environment variable substitution.
`!env [variable name]` substitutes the value of the environment variable named `[variable name]`. For example, with the environment variable `PS_MONGO_URI=mongodb://mongo:27017/powersync`, the YAML
```yaml service.yaml theme={null}
storage:
type: mongodb
uri: !env PS_MONGO_URI
```
resolves to `uri: mongodb://mongo:27017/powersync`.
Only environment variables with names starting with `PS_` can be substituted.
Substituted values are strings. For options that expect a number or boolean, append `::number` or `::boolean` to the variable name to cast the value. A `::boolean` cast accepts `true` or `false`.
```yaml service.yaml theme={null}
storage:
max_pool_size: !env PS_STORAGE_POOL_SIZE::number
telemetry:
disable_telemetry_sharing: !env PS_DISABLE_TELEMETRY::boolean
```
## Configuration Reference
The config file supports the following top-level keys, documented in the sections below:
| Key | Purpose |
| ----------------------------- | ----------------------------------------------- |
| [`replication`](#replication) | Source database connection(s) to replicate from |
| [`storage`](#storage) | Bucket storage database connection |
| [`port`](#port) | Port for the PowerSync API server |
| [`sync_config`](#sync_config) | Sync Streams (or legacy Sync Rules) definition |
| [`client_auth`](#client_auth) | JWT authentication for client connections |
| [`api`](#api) | Admin API tokens and performance/safety limits |
| [`telemetry`](#telemetry) | Telemetry sharing and Prometheus metrics |
| [`healthcheck`](#healthcheck) | Health check probe mechanisms |
| [`migrations`](#migrations) | Storage database schema migration behavior |
| [`system`](#system) | Service logging |
A machine-readable [JSON schema](https://unpkg.com/@powersync/service-schema@latest/json-schema/powersync-config.json) of the config file is available, published as `@powersync/service-schema`.
Add this comment to the top of your YAML config file to get validation and autocomplete in editors that support the [YAML language server](https://github.com/redhat-developer/yaml-language-server) (for example VS Code with the YAML extension):
```yaml theme={null}
# yaml-language-server: $schema=https://unpkg.com/@powersync/service-schema@latest/json-schema/powersync-config.json
```
### `replication`
The `replication` section defines the source database that PowerSync replicates data from. Specify the connection details in `replication.connections`. Only one connection is currently supported; multiple connection support is on our roadmap.
For instructions on preparing your source database, see [Source Database Setup](/configuration/source-db/setup).
Hosted Supabase only allows direct database connections over IPv6, and PowerSync cannot connect using the connection pooler. You can:
* Enable IPv6 for Docker as per [the Docker docs](https://docs.docker.com/config/daemon/ipv6/).
* Enable IPv4 on your Supabase project.
* Run Supabase locally.
All connection types support these common options:
The connection type. One of `postgresql`, `mongodb`, `mysql`, `mssql`, or `convex`.
Unique identifier for the connection. Optional when only a single connection is present.
Additional meta tag for the connection, used for categorization or grouping.
The remaining options depend on the connection type:
```yaml service.yaml theme={null}
replication:
connections:
- type: postgresql
uri: postgresql://postgres:mypassword@pg-db:5432/postgres
sslmode: verify-full
```
Connection URI in the format `postgresql://user:password@hostname:5432/database`. Individual connection parameters take precedence over values in the URI.
Database hostname. Required if not specified in `uri`.
Database port.
Database username. Required if not specified in `uri`.
Database password. Required if not specified in `uri`.
Database name. Required if not specified in `uri`.
SSL mode: `verify-full`, `verify-ca`, or `disable`. `disable` is only suitable for local/private networks, not for public networks.
CA certificate content in PEM format. Required for `verify-ca`, optional for `verify-full`.
Client certificate content in PEM format, for TLS client authentication.
Client private key content in PEM format, for TLS client authentication.
Use a servername for TLS that is different from `hostname`.
Prefix for Postgres logical replication slot names and replication stream names.
Maximum number of connections to the source database, per Service process.
Connection timeout in seconds. Takes precedence over a `connect_timeout` query parameter in the URI.
Idle timeout in seconds for snapshot connection sockets. If the storage database cannot keep up during the initial snapshot, a storage flush can stall the snapshot for longer than this timeout, which closes the source connection mid-snapshot. This appears in the Service logs as `Socket timed out` errors during the initial snapshot. Increase the timeout if you see these errors. Available since Service version 1.26.0.
Interval in seconds between source connection heartbeats. Must be between 5 and 60. Available since Service version 1.24.0.
```yaml service.yaml theme={null}
replication:
connections:
- type: mongodb
uri: mongodb+srv://myuser:mypassword@cluster0.abcde.mongodb.net/mydatabase
post_images: auto_configure
```
Connection URI in the format `mongodb://` or `mongodb+srv://`. Standard connection options such as `connectTimeoutMS`, `socketTimeoutMS`, `serverSelectionTimeoutMS`, `maxPoolSize` and `maxIdleTimeMS` can be set as query parameters in the URI.
Database name. Defaults to the database in the URI path.
Database username. Defaults to the username in the URI.
Database password. Defaults to the password in the URI.
Controls how change stream post-images are used: `off`, `auto_configure`, or `read_only`. `auto_configure` is recommended for new instances. See [Post Images](/configuration/source-db/setup#post-images) for details on each option.
Interval in seconds between source connection heartbeats. Must be between 5 and 60. Available since Service version 1.24.0.
MySQL support is currently in a [Beta release](/resources/feature-status).
```yaml service.yaml theme={null}
replication:
connections:
- type: mysql
uri: mysql://repl_user:mypassword@mysql-db:3306/inventory
```
Connection URI in the format `mysql://user:password@hostname:3306/database`. Individual connection parameters take precedence over values in the URI.
Database hostname. Required if not specified in `uri`.
Database port.
Database username. Required if not specified in `uri`.
Database password. Required if not specified in `uri`.
Database name. Required if not specified in `uri`.
CA certificate content in PEM format.
Client certificate content in PEM format, for TLS client authentication.
Client private key content in PEM format, for TLS client authentication.
The combined size in MB of binlog events that can be queued in memory before throttling is applied.
SQL Server support is currently in a [Beta release](/resources/feature-status). Also see [SQL Server Additional Configuration](/configuration/source-db/sql-server-additional-configuration).
```yaml service.yaml theme={null}
replication:
connections:
- type: mssql
uri: mssql://powersync_user:mypassword@mssql-db:1433/inventory
```
Connection URI in the format `mssql://user:password@hostname:1433/database`. Individual connection parameters take precedence over values in the URI.
Database hostname. Required if not specified in `uri`.
Database port.
Database username. Required if not specified in `uri` or `authentication`.
Database password. Required if not specified in `uri` or `authentication`.
Database name. Required if not specified in `uri`.
The database schema to replicate from.
CA certificate content in PEM format. Available since Service version 1.25.0.
Use a servername for TLS that is different from `hostname`. Available since Service version 1.25.0.
Alternative authentication configuration, instead of `username` and `password`.
Authentication method: `default` (SQL Server login) or `azure-active-directory-service-principal-secret`.
For `default`: `userName` and `password`. For `azure-active-directory-service-principal-secret`: `clientId`, `clientSecret` and `tenantId` from your registered Azure application.
Additional replication settings.
Interval in milliseconds to wait between CDC polling cycles.
Maximum number of transactions to poll per polling cycle.
Whether to trust the server certificate. Set to `true` for local development and self-signed certificates.
Interval in seconds between source connection heartbeats. Must be between 5 and 60. Available since Service version 1.24.0.
The Convex replicator is currently released as an [experimental feature](/resources/feature-status) and is available since Service version v1.22.0. See [Convex source database setup](/configuration/source-db/setup#convex).
```yaml service.yaml theme={null}
replication:
connections:
- type: convex
deployment_url: https://happy-animal-123.convex.cloud
deploy_key: !env PS_CONVEX_DEPLOY_KEY
```
The URL of your Convex deployment.
A deploy key for the Convex deployment, used to authenticate against the Convex Streaming Export API.
Interval in milliseconds between polling for new changes.
Timeout in milliseconds for requests to the Convex API.
### `storage`
The PowerSync Service requires a storage database to store the data and metadata for [buckets](/architecture/powersync-service#bucket-system). You can use either MongoDB or Postgres for this purpose.
The *bucket storage database* is separate from your *source database*.
The storage backend type: `mongodb` or `postgresql`.
Maximum number of connections to the storage database, per Service process.
Storage version to use when deploying a Sync Config that does not specify a storage version. To opt in to [storage version 4](/sync/advanced/storage-version-4) (Beta), set `config.storage_version: 4` in your Sync Config. An explicit `config.storage_version` takes precedence over this default. Available since Service version 1.26.0.
Postgres bucket storage does not currently support version 4.
#### MongoDB Storage
```yaml service.yaml theme={null}
storage:
type: mongodb
uri: mongodb://mongo:27017/powersync_demo
```
Connection URI in the format `mongodb://` or `mongodb+srv://`.
Database name. Defaults to the database in the URI path.
Database username. Defaults to the username in the URI. See **Required Permissions** below for the roles the user needs.
Database password. Defaults to the password in the URI.
Throttles the clearing of old bucket data after deploying a new sync configuration, by pausing between batches. The pause is proportional to the previous batch duration. Increase this to reduce the impact of clear operations on the storage cluster, or use `0` to clear as fast as possible. Must be between 0 and 20. Available since Service version 1.24.0.
Read preference for bulk checksum and bucket data reads: `primary`, `primaryPreferred`, `secondary`, `secondaryPreferred`, or `nearest`. If unset, MongoDB driver defaults are used. This is an experimental option and may be removed in a future release. Available since Service version 1.23.2.
Stores large blocks of bucket data in Amazon S3 or an S3-compatible object store instead of MongoDB. Requires Service version 1.26.0 or later with [storage version 4](/sync/advanced/storage-version-4) (Beta), which is not currently the default. Sync Configs on version 2 keep all data in MongoDB even when this is configured.
Follow [Self-Hosted S3 Setup](/sync/advanced/storage-version-4#self-hosted-s3-setup) to create the bucket, configure permissions and lifecycle settings, and enable object storage in the Service. The fields below describe the available configuration options.
Must be `s3`.
Name of the S3 bucket.
Region of the S3 bucket.
Key prefix for stored objects.
Custom endpoint, for S3-compatible object storage providers.
Use path-style addressing, required by some S3-compatible providers.
Access key ID for authentication.
Secret access key for authentication.
Maximum number of concurrent object storage requests.
Sets the timeouts for all object storage requests at once, based on the expected latency between the Service and the object storage endpoint, mirroring the AWS SDK's [smart configuration defaults](https://docs.aws.amazon.com/sdkref/latest/guide/feature-smart-config-defaults.html). `standard` is safe for most deployments, so you can usually leave this unset. Use `in-region` when the Service and the bucket run in the same region and you want stalled requests to fail faster, or `mobile` when the endpoint is slow enough that the standard timeouts cause spurious failures. `cross-region` is accepted for compatibility with the AWS SDK.
When this option is unset, the Service reads the `AWS_DEFAULTS_MODE` environment variable, then falls back to `standard`. Setting `defaults_mode` in an AWS shared config file has no effect. Available since Service version 1.26.0.
Chunks smaller than this byte threshold stay inline in MongoDB instead of being offloaded to object storage.
##### Required Permissions
The Service creates and manages all collections and indexes in the storage database itself, so no manual schema setup is needed. When authentication is enabled, the user needs the built-in `readWrite` and `dbAdmin` roles on the storage database:
```
readWrite@
dbAdmin@
```
No access beyond the storage database is required. The `readWrite` role covers regular operation, including creating and dropping collections and indexes. The `dbAdmin` role is additionally needed to drop the database when an instance is torn down.
Create the user with:
```javascript theme={null}
use powersync_demo
db.createUser({
user: "powersync_storage_user",
pwd: "secure_password",
roles: ["readWrite", "dbAdmin"]
})
```
MongoDB authenticates against the database where the user was created, so creating the user in the storage database itself works with the connection URI shown above. If you create the user in a different database, such as `admin`, add `authSource=admin` to the connection URI. On MongoDB Atlas, assign the same two roles restricted to the storage database.
##### Replica Set Requirement
MongoDB requires at least one replica set node. A single node is fine for development/staging environments, but a 3-node replica set is recommended [for production](/maintenance-ops/self-hosting/deployment-architecture) deployments.
[MongoDB Atlas](https://www.mongodb.com/products/platform/atlas-database) enables replica sets by default for new clusters.
However, if you're using your own environment you can enable this manually by running:
```bash theme={null}
mongosh "mongodb+srv://powersync.abcdef.mongodb.net/" --apiVersion 1 --username myuser --eval 'try{rs.status().ok && quit(0)} catch {} rs.initiate({_id: "rs0", version: 1, members: [{ _id: 0, host : "mongo:27017" }]})'
```
If you are rolling your own Docker environment, you can include this init script in your `docker-compose` file to configure a replica set as once-off operation:
```yaml docker-compose.yaml theme={null}
# Initializes the MongoDB replica set. This service will not usually be actively running
mongo-rs-init:
image: mongo:7.0
depends_on:
- mongo
restart: "no"
entrypoint:
- bash
- -c
- 'sleep 10 && mongosh --host mongo:27017 --eval ''try{rs.status().ok && quit(0)} catch {} rs.initiate({_id: "rs0", version: 1, members: [{ _id: 0, host : "mongo:27017" }]})'''
```
#### Postgres Storage
You can use Postgres as an alternative bucket storage database.
```yaml service.yaml theme={null}
storage:
type: postgresql
uri: postgresql://powersync_storage_user:secure_password@storage-db:5432/postgres
```
Postgres storage accepts the same connection options as a [Postgres replication connection](#replication): `uri`, `hostname`, `port`, `username`, `password`, `database`, `sslmode`, `cacert`, `client_certificate`, `client_private_key`, `tls_servername` and `connect_timeout`. In addition, batch limits can be tuned:
Limits for batch operations during replication. Increasing these limits can improve replication performance, at the cost of higher memory usage.
Maximum estimated byte size of operations written in a single transaction.
Maximum number of records written in a single transaction.
Maximum byte size of `current_data` documents looked up at a time.
##### Database Setup
You'll need to create a dedicated user and schema for PowerSync bucket storage. You can either:
1. Let PowerSync create the schema (recommended):
```sql theme={null}
CREATE USER powersync_storage_user WITH PASSWORD 'secure_password';
-- The user should only have access to the schema it created
GRANT CREATE ON DATABASE postgres TO powersync_storage_user;
```
2. Or manually create the schema:
```sql theme={null}
CREATE USER powersync_storage_user WITH PASSWORD 'secure_password';
CREATE SCHEMA IF NOT EXISTS powersync AUTHORIZATION powersync_storage_user;
GRANT CONNECT ON DATABASE postgres TO powersync_storage_user;
GRANT USAGE ON SCHEMA powersync TO powersync_storage_user;
GRANT ALL PRIVILEGES ON ALL TABLES IN SCHEMA powersync TO powersync_storage_user;
```
A demo app with Postgres bucket storage is available [here](https://github.com/powersync-ja/self-host-demo/tree/main/demos/nodejs-postgres-bucket-storage).
##### Postgres Version Requirements
Separate Postgres servers are required for replication connections (i.e. source database) and bucket storage **if using Postgres versions below 14**.
| Postgres Version | Server configuration |
| ---------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Below 14 | Separate servers are required for the source and bucket storage. Replication will be blocked if the same server is detected. |
| 14 and above | The source database and bucket storage database can be on the same server. Using the same database (with separate schemas) is supported but may lead to higher CPU usage. Using separate servers remains an option. |
### `port`
The `port` setting determines where clients and tools connect to your instance. Change it if the default conflicts with another service in your deployment.
The port on which the PowerSync API server will listen for connections. Can be specified as a number or string.
### `sync_config`
The `sync_config` section points the Service at your [Sync Streams](/sync/streams/overview) (or legacy [Sync Rules](/sync/rules/overview)) definition, usually a separate file referenced with `path`:
```yaml service.yaml theme={null}
sync_config:
path: sync-config.yaml
```
```yaml sync-config.yaml theme={null}
config:
edition: 3
streams:
todos:
auto_subscribe: true
query: SELECT * FROM todos WHERE owner_id = auth.user_id()
```
Path to the sync configuration YAML file. Ensure the file is available at that path, e.g. in the same directory as your main config or on a mounted volume.
The sync configuration inline as a string, as an alternative to `path`.
Whether to exit the process if there is an error parsing the sync configuration.
The top-level `sync_rules` key is a deprecated alias for `sync_config`. Use `sync_config` in new configurations.
### `client_auth`
The `client_auth` section defines how the Service verifies your app's users when they connect to sync. Clients authenticate with JWTs, which the Service validates using the settings here. For more details, see [Client Authentication](/configuration/auth/overview).
```yaml service.yaml theme={null}
client_auth:
# Enable this if using Supabase Auth
# supabase: true
# supabase_jwt_secret: your-secret
# Option 1: JWKS URI endpoint
jwks_uri: http://demo-backend:6060/api/auth/keys
# Option 2: Static collection of public keys for JWT verification
# jwks:
# keys:
# - kty: 'RSA'
# n: '[rsa-modulus]'
# e: '[rsa-exponent]'
# alg: 'RS256'
# kid: '[key-id]'
# JWKS audience
audience: ['powersync-dev', 'powersync']
```
URI or array of URIs pointing to JWKS endpoints, used to fetch public keys for JWT verification.
Inline JWKS configuration, as an alternative or in addition to `jwks_uri`.
An array of JSON Web Keys (JWKs). Supported key types are RSA (`RS256`, `RS384`, `RS512`), HMAC (`HS256`, `HS384`, `HS512`), OKP (`EdDSA` with `Ed25519` or `Ed448`) and EC (`ES256`, `ES384`, `ES512` with curves `P-256`, `P-384` or `P-521`). See [Custom Authentication](/configuration/auth/custom) for details.
Enables Supabase authentication integration. JWKS details are derived from the Supabase connection. See [Supabase Auth](/configuration/auth/supabase-auth).
Legacy JWT secret for Supabase authentication (HS256 shared secret).
Valid audiences for JWT validation.
For production environments, we recommend using JWKS with asymmetric keys (RS256, EdDSA, or ECDSA) rather than shared secrets (HS256). Asymmetric keys provide better security through public/private key separation and easier key rotation. See [Custom Authentication](/configuration/auth/custom) for more details.
### `api`
The `api` section protects the Service's admin API routes with access tokens, and sets limits that protect the Service from excessive load. Configure `tokens` if you use the [PowerSync CLI](/tools/cli) against this instance; the limits rarely need changing.
Access tokens for the Service's admin API routes, provided by clients as a Bearer token. If no tokens are configured, the admin API routes reject all requests.
Performance and safety parameters for the API.
Maximum number of connections (HTTP streams or WebSockets) per API process.
Maximum concurrency when fetching data from storage. This should not be significantly more than `storage.max_pool_size`, otherwise it would block on the pool. Increasing this can significantly increase memory usage in some cases.
Maximum number of buckets for each connection. More buckets increase latency and memory usage. While the actual number is controlled by your sync configuration, this hard limit ensures that the Service errors instead of crashing when the sync configuration is misconfigured.
Related to `max_buckets_per_connection`, but this limit applies directly to parameter query results, before they are converted into a unique set of buckets.
Number of minutes to keep client-requested write checkpoint records. Expired records are removed by the compact job. Must be a positive integer. Available since Service version 1.24.0.
How long to keep cached bucket counts before refreshing them, in minutes. Bucket counts may be affected by compacting. Available since Service version 1.23.2.
### `telemetry`
The `telemetry` section controls the operational metrics the Service shares with PowerSync and exposes for your own monitoring. See [Usage Reporting](/maintenance-ops/self-hosting/usage-reporting) and [Monitoring](/maintenance-ops/self-hosting/monitoring) for details.
When `true`, disables sharing of anonymized usage metrics with the PowerSync telemetry service.
Port on which Prometheus metrics will be exposed. When set, metrics will be available on this port for scraping.
### `healthcheck`
Configures how health check status is exposed. See [Health Checks](/maintenance-ops/self-hosting/healthchecks) for details on the available probes and endpoints.
Mechanisms for exposing health check data. If this is not configured, the Service defaults to legacy behavior for backwards compatibility (filesystem probes always enabled, plus HTTP probes depending on the Service mode). When `probes` is configured, each mechanism requires explicit opt-in.
Enables exposing health check status via filesystem files.
Enables exposing health check status via HTTP endpoints.
### `migrations`
The `migrations` section controls whether the Service updates its bucket storage database schema automatically when a new version starts up. Most deployments can keep the default automatic behavior.
When `true`, disables automatic storage database schema migrations on startup. Migrations can then be triggered externally by altering the container `command`.
### `system`
The `system` section configures how the Service itself runs. Currently this covers logging.
Service logging configuration.
Log level for the Service logs: `silly`, `debug`, `verbose`, `http`, `info`, `warn`, or `error`. The `PS_LOG_LEVEL` environment variable takes precedence over this option.
Log output format: `json` or `text`. Defaults to `text`, or to `json` when the `NODE_ENV` environment variable is set to `production`. The `PS_LOG_FORMAT` environment variable takes precedence over this option.
# Source Database Connection
Source: https://docs.powersync.com/configuration/source-db/connection
Connect a PowerSync Cloud instance to your Postgres, MongoDB, MySQL, SQL Server or Convex source database.
Each database provider has their quirks when it comes to specifying connection details, so we have documented database-specific and provider-specific instructions below:
Jump to: [Postgres](#postgres-provider-specifics) | [MongoDB](#mongodb-specifics) | [Azure DocumentDB](#azure-documentdb-specifics) | [MySQL](#mysql-specifics) | [SQL Server](#sql-server-specifics) | [Convex](#convex-specifics)
The below instructions are currently written for PowerSync Cloud. For self-hosted PowerSync instances, specify database connection details in the config file as documented [here](/configuration/powersync-service/self-hosted-instances#replication).
##
1. From your Supabase Dashboard, select **Connect** in the top navigation bar (or follow this [link](https://supabase.com/dashboard/project/_?showConnect=true)):
1. In the [PowerSync Dashboard](https://dashboard.powersync.com/), select your project and instance and go to the **Database Connection** view.
2. Select the **Postgres** tab.
3. [Locate the connection details from AWS RDS](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_ConnectToPostgreSQLInstance.html):
* Copy the **"Endpoint"** value from the AWS Management Console.
* Paste the endpoint into the "**Host**" field in the PowerSync Dashboard.
* Complete the remaining fields: "**Name**", "**Host**", "**Port**", "**Database name**", "**Username**", "**Password**" and "**SSL Mode"** are required. You can paste a connection string into the "**URI**" field to simplify this.
* "**Name**" can be any name for the connection.
* "**Port**" is 5432 for Postgres databases.
* "**Username**" and "**Password**" maps to the `powersync_role` created in [Source Database Setup](/configuration/source-db/setup).
* PowerSync has the AWS RDS CA certificate pre-configured — `verify-full` SSL mode can be used directly, without any additional configuration required.
4. Click **Test Connection** and fix any errors.
5. Click **Save Connection**.
PowerSync deploys and configures an isolated cloud environment for you, which can take a few minutes to complete.
### Troubleshooting
If you get an error such as "IPs in this range are not supported", the instance is likely not configured to be publicly accessible. A DNS lookup on the host should give a public IP, and not for example `10.x.x.x` or `172.31.x.x`.
1. In the [PowerSync Dashboard](https://dashboard.powersync.com/), select your project and instance and go to the **Database Connection** view.
2. Select the **Postgres** tab.
3. Fill in your connection details from Azure.
1. "**Name**", "**Host**", "**Port**", "**Database name**", "**Username**", "**Password**" and "**SSL Mode"** are required. You can also paste a connection string into the "**URI**" field to simplify data entry.
2. "**Name**" can be any name for the connection.
3. "**Port**" is 5432 for Postgres databases.
4. "**Username**" and "**Password**" maps to the `powersync_role` created in [Source Database Setup](/configuration/source-db/setup).
5. PowerSync has the Azure CA certificate pre-configured — `verify-full` SSL mode can be used directly, without any additional configuration required.
4. Click **Test Connection** and fix any errors.
* If you encounter the error `"must be superuser or replication role to start walsender"`, ensure that you've followed all the steps for enabling logical replication documented [here](https://learn.microsoft.com/en-us/azure/postgresql/flexible-server/concepts-logical#prerequisites-for-logical-replication-and-logical-decoding).
5. Click **Save Connection**.
PowerSync deploys and configures an isolated cloud environment for you, which can take a few minutes to complete.
1. In the [PowerSync Dashboard](https://dashboard.powersync.com/), select your project and instance and go to the **Database Connection** view.
2. Select the **Postgres** tab.
3. Fill in your connection details from Google Cloud SQL.
* "**Name**", "**Host**", "**Port**", "**Database name**", "**Username**", "**Password**" and "**SSL Mode"** are required. You can paste a connection string into the "**URI**" field to simplify data entry.
* "**Name**" can be any name for the connection.
* "**Port**" is 5432 for Postgres databases.
* "**Username**" and "**Password**" maps to the `powersync_role` created in [Source Database Setup](/configuration/source-db/setup).
* The server certificate can be downloaded from Google Cloud SQL.
* If SSL is enforced, a client certificate and key must also be created on Google Cloud SQL, and configured on the PowerSync instance.
4. Click **Test Connection** and fix any errors.
5. Click **Save Connection**.
PowerSync deploys and configures an isolated cloud environment for you, which can take a few minutes to complete.
1. In the [PowerSync Dashboard](https://dashboard.powersync.com/), select your project and instance and go to the **Database Connection** view.
2. Select the **Postgres** tab.
3. Fill in your connection details from [Neon](https://neon.tech/).
1. "**Name**", "**Host**", "**Port**", "**Database name**", "**Username**", "**Password**" and "**SSL Mode"** are required. You can paste a connection string into the "**URI**" field to simplify data entry.
2. "**Name**" can be any name for the connection.
3. "**Port**" is 5432 for Postgres databases.
4. "**Username**" and "**Password**" maps to the `powersync_role` created in [Source Database Setup](/configuration/source-db/setup).
5. Note that if you're using a self-signed SSL certificate for your database server, click the **Download Certificate** button to dynamically fetch the recommended certificate directly from your server.
6. Also note if you get any error such as `server certificate not trusted: SELF_SIGNED_CERT_IN_CHAIN`, click **Download Certificate** to attempt automatic resolution.
4. Click **Test Connection** and fix any errors.
5. Click **Save Connection**.
PowerSync deploys and configures an isolated cloud environment for you, which can take a few minutes to complete.
1. In the [PowerSync Dashboard](https://dashboard.powersync.com/), select your project and instance and go to the **Database Connection** view.
2. Select the **Postgres** tab.
3. Fill in your connection details from [Fly Postgres](https://fly.io/docs/postgres/).
1. "**Name**", "**Host**", "**Port**", "**Database name**", "**Username**", "**Password**" and "**SSL Mode"** are required. You can paste a connection string into the "**URI**" field to simplify data entry.
2. "**Name**" can be any name for the connection.
3. "**Port**" is 5432 for Postgres databases.
4. "**Username**" and "**Password**" maps to the `powersync_role` created in [Source Database Setup](/configuration/source-db/setup).
5. Note that if you're using a self-signed SSL certificate for your database server, click the **Download Certificate** button to dynamically fetch the recommended certificate directly from your server.
6. Also note if you get any error such as `server certificate not trusted: SELF_SIGNED_CERT_IN_CHAIN`, click **Download Certificate** to attempt automatic resolution.
4. Click **Test Connection** and fix any errors.
5. Click **Save Connection**.
PowerSync deploys and configures an isolated cloud environment for you, which can take a few minutes to complete.
1. In the [PowerSync Dashboard](https://dashboard.powersync.com/), select your project and instance and go to the **Database Connection** view.
2. Select the **Postgres** tab.
3. Head to your PlanetScale database dashboard page at `https://app.planetscale.com//` and click on the "Connect" button to get your database connection parameters.
1. In the PowerSync Dashboard, "**Name**", "**Host**", "**Port**", "**Database name**", "**Username**" and "**Password**" are required.
2. "**Name**" can be any name for the connection.
3. "**Host**" is the `host` connection parameter for your database.
4. "**Port**" is 5432 for Postgres databases.
5. "**Username**" and "**Password**" maps to the `powersync_role` created in [Source Database Setup](/configuration/source-db/setup).
1. Important: PlanetScale requires your branch ID to be appended to your username. The username should be `powersync_role.`. Your PlanetScale branch ID can be found on the same connection details page.
6. **SSL Mode** can remain the default `verify-full`.
4. Click **Test Connection** and fix any errors.
5. Click **Save Connection**.
PowerSync deploys and configures an isolated cloud environment for you, which can take a few minutes to complete.
1. In the [PowerSync Dashboard](https://dashboard.powersync.com/), select your project and instance and go to the **Database Connection** view.
2. Select the **Postgres** tab.
3. In your Xata dashboard, navigate to Overview -> Connect and copy the connection URI. Alternatively, use the [Xata CLI](https://xata.io/docs/cli) and run `xata branch url` to obtain the URI and copy that.
4. Paste the connection details into the "**URI**" field in the PowerSync Dashboard.
5. Click **Test Connection** and fix any errors.
6. Click **Save Connection**.
PowerSync deploys and configures an isolated cloud environment for you, which can take a few minutes to complete.
For other providers and self-hosted databases:
1. In the [PowerSync Dashboard](https://dashboard.powersync.com/), select your project and instance and go to the **Database Connection** view.
2. Select the **Postgres** tab.
3. Fill in your connection details.
4. "**Name**", "**Host**", "**Port**", "**Database name**", "**Username**", "**Password**" and "**SSL Mode"** are required. You can paste a connection string into the "**URI**" field to simplify data entry.
5. "**Name**" can be any name for the connection.
6. "**Port**" is 5432 for Postgres databases.
7. "**Username**" and "**Password**" maps to the `powersync_role` created in [Source Database Setup](/configuration/source-db/setup).
8. Note that if you're using a self-signed SSL certificate for your database server, click the **Download Certificate** button to dynamically fetch the recommended certificate directly from your server.
9. Also note if you get any error such as `server certificate not trusted: SELF_SIGNED_CERT_IN_CHAIN`, click **Download Certificate** to attempt automatic resolution.
10. Click **Test Connection** and fix any errors.
11. Click **Save Connection**.
PowerSync deploys and configures an isolated cloud environment for you, which can take a few minutes to complete.
Make sure that your Postgres database allows access to PowerSync's IPs — see [Security and IP Filtering](/configuration/source-db/security-and-ip-filtering)
Also see:
* [Postgres Source Database Setup](/configuration/source-db/setup#postgres)
* Security & IP Filtering: [TLS with Postgres](/configuration/source-db/security-and-ip-filtering#powersync-cloud-tls-with-postgres)
## @cluster0.abcde1.mongodb.net/powersync`
2. Enter your database user's password into the **Password** field. See the necessary permissions in [Source Database Setup](/configuration/source-db/setup#mongodb).
3. "**Database name**" is the database in your cluster to replicate.
4. Click **Test Connection** and fix any errors. If have any issues connecting, reach out to our support engineers on our [Discord server](https://discord.gg/powersync) or otherwise [contact us](/resources/contact-us).
5. Click **Save Connection**.
PowerSync deploys and configures an isolated cloud environment for you, which can take a few minutes to complete.
Make sure that your MongoDB database allows access to PowerSync's IPs — see [Security and IP Filtering](/configuration/source-db/security-and-ip-filtering)
Also see:
* [MongoDB Source Database Setup](/configuration/source-db/setup#mongodb)
* [MongoDB Atlas Device Sync Migration Guide](/migration-guides/atlas-device-sync)
## MySQL support is currently in a [Beta release](/resources/feature-status).
Select your MySQL hosting provider for steps to connect your newly-created PowerSync instance to your MySQL database:
To enable binary logging and GTID replication in AWS Aurora, you need to create a [DB Parameter Group](https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/AuroraMySQL.Reference.ParameterGroups.html)
and configure it with the necessary parameters. Follow these steps:
1. Navigate to [Amazon RDS console](https://console.aws.amazon.com/rds/)
In the navigation pane, select **Parameter groups** and click **Create Parameter Group**:
For other providers and self-hosted databases:
1. In the [PowerSync Dashboard](https://dashboard.powersync.com/), select your project and instance and go to the **Database Connection** view.
2. Select the **MySQL** tab.
3. Fill in your MySQL connection details:
1. "**Name**", "**Host**", "**Port**", "**Database name**", "**Username**", "**Password**" are required.
2. "**Name**" can be any name for the connection.
3. "**Host**" the endpoint for your database.
4. "**Database name**" is the default database to replicate. Additional databases are derived by qualifying the tables in your Sync Streams/Sync Rules.
5. "**Username**" and "**Password**" maps to your database user.
4. Click **Test Connection** and fix any errors.
5. Click **Save Connection**.
PowerSync deploys and configures an isolated cloud environment for you, which can take a few minutes to complete.
Make sure that your MySQL database allows access to PowerSync's IPs — see [Security and IP Filtering](/configuration/source-db/security-and-ip-filtering)
Also see:
* [MySQL Source Database Setup](/configuration/source-db/setup#mysql)
## SQL Server support is currently in a [Beta release](/resources/feature-status).
SQL Server support was [introduced](https://releases.powersync.com/announcements/powersync-service) in version 1.18.1 of the PowerSync Service.
1. In the [PowerSync Dashboard](https://dashboard.powersync.com/), select your project and instance and go to the **Database Connection** view.
2. Select the **SQL Server** tab.
3. Fill in your SQL Server connection details:
1. "**Name**", "**Host**", "**Port**", "**Database name**", "**Username**", "**Password**" are required.
2. "**Name**" can be any name for the connection.
3. "**Host**" is the endpoint for your SQL Server instance.
4. "**Port**" is typically 1433 for SQL Server (default port).
5. "**Database name**" is the database you want to replicate. CDC should be enabled on this database.
6. "**Username**" and "**Password**" maps to the database user created in [Source Database Setup](/configuration/source-db/setup#sql-server) (e.g., `powersync_user`).
4. Click **Test Connection** and fix any errors.
5. Click **Save Connection**.
PowerSync deploys and configures an isolated cloud environment for you, which can take a few minutes to complete.
Make sure that your SQL Server database allows access to PowerSync's IPs — see [Security and IP Filtering](/configuration/source-db/security-and-ip-filtering)
Also see:
* [SQL Server Source Database Setup](/configuration/source-db/setup#sql-server)
##
The Convex replicator is currently released as an [experimental feature](/resources/feature-status). APIs and
behavior may change, and we can't yet guarantee continued support or long-term stability.
Before connecting PowerSync, add the required `powersync_checkpoints` table and mutation to your Convex deployment. See [Convex Source Database Setup](/configuration/source-db/setup#convex).
1. In the [PowerSync Dashboard](https://dashboard.powersync.com/), select your project and instance and go to **Database Connection**.
2. Select the **Convex** tab.
3. Fill in your Convex connection details:
1. **Deployment URL** is your Convex deployment URL, for example `https://.convex.cloud`.
2. **Deploy key** is the deploy key for the Convex deployment. In the Convex Dashboard, go to **Settings** → **General** and generate a deploy key with **Custom permissions** that include `deployment:data:view`. The **Deploy only** option does not work for PowerSync replication.
4. Click **Test Connection** and fix any errors.
5. Click **Save Connection**.
PowerSync deploys and configures an isolated cloud environment for you, which can take a few minutes to complete.
Use a deploy key for the correct environment, grant only the permissions PowerSync needs, store it as a secret, and rotate it if it is exposed.
Also see:
* [Convex Source Database Setup](/configuration/source-db/setup#convex)
* [Type Mapping](/sync/types#convex-type-mapping)
# Postgres Maintenance
Source: https://docs.powersync.com/configuration/source-db/postgres-maintenance
Manage Postgres replication slots and WAL lag for reliable PowerSync replication.
## Logical Replication Slots
Postgres logical replication slots are used to keep track of [replication](/architecture/powersync-service#replication-from-the-source-database) progress (recorded as a [LSN](https://www.postgresql.org/docs/current/datatype-pg-lsn.html)).
Every time a new version of [Sync Streams or Sync Rules](/sync/overview) is deployed, PowerSync creates a new replication slot. Once the new version is fully processed, PowerSync switches to use the new slot and deletes the old one. The Service logs these steps and, during a snapshot, how much WAL budget remains. See [Postgres Replication Slots and WAL Budget](/debugging/log-reference#postgres-replication-slots-and-wal-budget) in the Log Reference.
The replication slots can be viewed using this query:
```sql theme={null}
select slot_name, confirmed_flush_lsn, active, pg_size_pretty(pg_wal_lsn_diff(pg_current_wal_lsn(), restart_lsn)) as lag from pg_replication_slots;
```
Example output:
| slot\_name | confirmed\_flush\_lsn | active | lag |
| ---------------------- | --------------------- | ------ | -------- |
| powersync\_1\_c3c8cf21 | 0/70D8240 | 1 | 56 bytes |
| powersync\_2\_e62d7e0f | 0/70D8240 | 1 | 56 bytes |
In some cases, a replication slot may remain without being used. In this case, the slot prevents Postgres from deleting older WAL entries. For example, this happens when a PowerSync instance has been deprovisioned.
Keeping unused slots alive prevents WAL cleanup, which can lead to excessive disk usage. If a slot is no longer needed, it should be dropped.
Inactive slots can be dropped using:
```bash theme={null}
select slot_name, pg_drop_replication_slot(slot_name) from pg_replication_slots where active = false;
```
Postgres prevents active slots from being dropped. If an active slot is somehow dropped while a PowerSync instance is disconnected, PowerSync will automatically recreate the slot when it reconnects and restart replication.
### Recovering from an Invalidated Slot
A replication slot becomes invalidated when its `wal_status` is `lost`. This happens when the WAL data needed by the slot has been removed, typically because the replication lag exceeded `max_slot_wal_keep_size`.
When this occurs, you will see an error such as:
> Replication slot powersync\_1\_xxxx was invalidated (reason: wal\_removed). Increase max\_slot\_wal\_keep\_size on the source database and delete the existing slot to recover.
To recover:
1. Increase `max_slot_wal_keep_size` on the source Postgres database to prevent re-occurrence. See [Managing and Monitoring Replication Lag](/maintenance-ops/production-readiness-guide#managing-and-monitoring-replication-lag) for sizing guidance.
2. Drop the invalidated slot:
```sql theme={null}
SELECT pg_drop_replication_slot('powersync_1_xxxx');
```
Replace `powersync_1_xxxx` with the actual slot name from the error message.
3. Restart the PowerSync Service. It will create a new replication slot and begin replication from scratch.
If the slot was invalidated during the initial snapshot (before it completed), the PowerSync Service will not automatically retry. You must drop the invalidated slot manually before the service can recover.
If the invalidation reason is `idle_timeout` (Postgres 18+), the slot was invalidated due to inactivity. In this case, increase `idle_replication_slot_timeout` on the source database instead.
### Maximum Replication Slots
Postgres is configured with a maximum number of replication slots per server. Each PowerSync instance uses one replication slot for replication and an additional one while deploying a new Sync Streams or Sync Rules version. The maximum number of PowerSync instances you can connect to one Postgres server is equal to the maximum number of replication slots, minus one.
If other clients are also using replication slots, this number is reduced further.
To configure the maximum number of slots, set `max_replication_slots` (though not all hosting providers expose this setting). Check the current value using:
```sql theme={null}
select current_setting('max_replication_slots')
```
If this number is exceeded, you'll see an error such as "all replication slots are in use".
# Private Endpoints
Source: https://docs.powersync.com/configuration/source-db/private-endpoints
Connect PowerSync Cloud to your database over AWS PrivateLink without public internet exposure.
Private Endpoints use [AWS PrivateLink](https://aws.amazon.com/privatelink/) to provide private networking between your source database and the PowerSync Service, without exposing the database to the public internet. They are available on our [Team and Enterprise plans](https://www.powersync.com/pricing) and can be managed in the [PowerSync Dashboard](https://dashboard.powersync.com/) under **Organization settings**.
Do not rely on Private Endpoints as the only form of security. Always use strong database passwords, and use client certificates if additional security is required.
## Current Limitations
1. Private Endpoints are currently only supported for Postgres and MongoDB instances. [Contact us](/resources/contact-us) if you need this for MySQL or SQL Server.
2. Only AWS is supported currently. Other cloud providers are not supported yet.
3. Private Endpoints can currently only be managed in the Dashboard. CLI support is not yet available.
## Concepts
* [AWS PrivateLink](https://aws.amazon.com/privatelink/) is the overarching AWS feature.
* **VPC/Private Endpoint Service** is the service that exposes the database. It lives in the same VPC as the source database and provides a one-way connection without exposing other resources in the VPC.
* **Service Name** is the unique identifier for the Endpoint Service, in the form `com.amazonaws.vpce..vpce-svc-`.
* Each Endpoint Service may have multiple Private Endpoints in different VPCs.
* **VPC/Private Endpoint** is the endpoint in the PowerSync VPC that the PowerSync instance connects to.
* **VPC Endpoint ID** is the identifier of this endpoint, in the form `vpce-`. You use this ID to accept the connection on the Endpoint Service side.
For custom Endpoint Services for Postgres:
* **Network Load Balancer (NLB)** forwards traffic from the Endpoint Service to the source database.
* **Target Group** specifies the IPs and ports the NLB exposes.
* **Listener** describes the incoming port on the NLB (the port the PowerSync instance connects to).
## Private Endpoint Setup
The flow is the same for both supported source databases: configure an Endpoint Service in front of your database, create a matching Private Endpoint in the PowerSync Dashboard, accept the connection, then point your database connection at the endpoint.
Set up an Endpoint Service in front of your source database and copy its **Service Name**.
MongoDB Atlas supports creating an Endpoint Service per project for AWS.
**Limitations:**
1. Only Atlas clusters in AWS are supported.
2. PowerSync does not support PrivateLink for MongoDB clusters self-hosted in AWS, only Atlas clusters.
3. If your cluster is in a newer AWS region, you may not be able to create Private Endpoints until we activate that region. See [AWS Regions](https://docs.aws.amazon.com/global-infrastructure/latest/regions/aws-regions.html#available-regions) for the list of regions enabled by default.
Create an Endpoint Service:
1. In the Atlas project dashboard, go to **Security** → **Database & Network Access** → **Network Access** -> **Private Endpoint**.
2. On the **Dedicated Cluster** tab click **Create endpoint service**.
3. Select **AWS** as the Cloud Provider and the Atlas Region matching your PowerSync cluster.
4. Under **Accepted Endpoint Regions**, select the AWS region where your PowerSync instances are hosted. See [AWS Regions](#aws-regions) for the mapping of PowerSync regions to AWS regions.
5. Click **Create endpoint service**.
6. Wait for the **Endpoint Service** to be created.
7. Copy the **Endpoint Service Name/ID**. You will use this when creating a Private Endpoint in the PowerSync Dashboard.
To expose a Postgres database via PrivateLink, you need a Network Load Balancer that forwards traffic to the database. This works for Postgres running on EC2 or RDS.
For AWS RDS, the steps below do not handle dynamic IPs if the RDS instance's IP changes. This is specifically relevant when using an RDS cluster with failover support. See this [AWS blog post](https://aws.amazon.com/blogs/database/access-amazon-rds-across-vpcs-using-aws-privatelink-and-network-load-balancer/) for handling IP changes automatically.
1. **Create a Target Group**:
1. Obtain the RDS instance's private IP address. Make sure this points to a writable instance.
2. Create a **Target Group** with IP addresses as the target type, using the IP from above. Use TCP protocol and the database port (typically `5432` for Postgres).
2. **Create a Network Load Balancer (NLB)**:
1. Select the same VPC as your RDS instance.
2. Select at least two subnets in different availability zones.
3. Configure a TCP listener on a port of your choice (for example, `5432`).
4. Associate the listener with the target group from the previous step.
3. **Modify the security group** associated with your RDS instance to permit traffic from the load balancer IP range.
4. **Create a VPC Endpoint Service**:
1. In the AWS Management Console, go to the VPC service and select **Endpoint Services**.
2. Click **Create Endpoint Service**.
3. Select the Network Load Balancer from the previous step.
4. If the load balancer is in one of the PowerSync [AWS regions](#aws-regions), you do not need to select any "Supported Region". If the load balancer is in a different region, select the region corresponding to your PowerSync instance. Cross-region support incurs additional AWS charges.
5. Decide whether to require acceptance for endpoint connections. Disabling acceptance simplifies the process but reduces control over connections.
6. Under **Supported IP address types**, select both IPv4 and IPv6.
7. Note the **Service Name**. You will use this when creating the Private Endpoint in the PowerSync Dashboard.
8. Configure the Endpoint Service to accept connections from the principal `arn:aws:iam::131569880293:root`. See the [AWS documentation](https://docs.aws.amazon.com/vpc/latest/privatelink/configure-endpoint-service.html#add-remove-permissions) for details.
In the [PowerSync Dashboard](https://dashboard.powersync.com/), open your organization's **Settings** and locate the **Private Endpoints** section.
1. Click **Create**.
2. Provide the following details:
* **Name**: a recognizable name for the endpoint (for example, `my-private-endpoint`).
* **Service Name**: the Endpoint Service Name from the previous step (for example, `com.amazonaws.vpce.us-east-1.vpce-svc-0123456`).
* **Region**: the PowerSync region to create the Private Endpoint in. This must match the region of the PowerSync instance you plan to use the endpoint with. Note that the region of your underlying AWS PrivateLink Endpoint Service does not need to match.
3. Click **Create**.
4. The endpoint is created in a `Pending Acceptance` state and will only be available once accepted on the Endpoint Service side.
Copy the **VPC Endpoint ID** from the endpoint's card in the Dashboard and use it to accept the connection on the Endpoint Service side.
In the Atlas project dashboard, go to **Security** → **Database & Network Access** → **Network Access** -> **Private Endpoint**.
1. Under the relevant Endpoint Service, click **Add Endpoint**
2. Select **Connect Existing Endpoint**.
3. Enter the **VPC Endpoint ID** you copied from the PowerSync Dashboard.
4. Click **Add endpoint**.
5. Wait for the endpoint to be added.
If you required acceptance when creating the Endpoint Service, go to **VPC** → **Endpoint Services** in the AWS Management Console, select your service, and accept the connection request matching the **VPC Endpoint ID**.
If you disabled acceptance, the connection is accepted automatically and no action is needed here.
The endpoint's status in the Dashboard reflects the AWS connection state:
| Status | Meaning |
| -------------------- | ---------------------------------------------------------------------------------- |
| `Pending acceptance` | Waiting for you to accept the Private Endpoint connection on the Endpoint Service. |
| `Pending` | The Private Endpoint is being provisioned on the Endpoint Service. |
| `Available` | Ready to use. |
| `Rejected` | The Private Endpoint connection was rejected by the Endpoint Service. |
| `Failed` | Private Endpoint creation failed. |
Once the status changes to `Available`, the endpoint can be selected when configuring a database connection.
1. In the Dashboard, open the PowerSync instance and go to the **Database Connection** view.
2. Select the **Postgres** or **MongoDB** tab.
3. In the **Private Endpoint** dropdown, select your endpoint. Only endpoints in the same region as the instance with status `Available` are selectable.
4. Fill in the rest of the connection details:
* **For Postgres**: enter your database connection details as usual. PowerSync routes traffic through the Private Endpoint to your load balancer.
* **For MongoDB**: on the Atlas cluster, click **Connect**, choose **Private Endpoint** as the connection type, select the provisioned endpoint, choose **Drivers** as the connection method, and copy the resulting connection string. It should look something like `mongodb+srv://:@your-cluster-pl-0.abcde.mongodb.net/`. Paste it into the **URI** field in the Dashboard.
5. Click **Test Connection** and resolve any errors.
6. Click **Save Connection**.
PowerSync deploys and configures an isolated cloud environment for you, which can take a few minutes. Monitor the logs to confirm the instance connects.
## AWS Regions
PowerSync Cloud currently runs in the AWS regions below. Your Endpoint Service can run in any AWS region, as long as it accepts connections from the AWS region of your PowerSync instance. The Private Endpoint itself must be created in the same region as the PowerSync instance.
1. US: `us-east-1`
2. EU: `eu-west-1`
3. BR: `sa-east-1`
4. JP: `ap-northeast-1`
5. AU: `ap-southeast-2`
# Security & IP Filtering
Source: https://docs.powersync.com/configuration/source-db/security-and-ip-filtering
Restrict database access to PowerSync Cloud using static IP allowlisting by deployment region.
## PowerSync Cloud: IP Filtering
For enhanced security, you can restrict database access to PowerSync Cloud's IP addresses. Below are the IP ranges for each region:
```
50.19.5.255
34.193.39.149
18.234.18.91
18.233.128.219
34.202.251.156
```
```
79.125.70.43
18.200.209.88
18.234.18.91
18.233.128.219
34.202.251.156
```
```
54.248.194.85
57.180.73.135
18.234.18.91
18.233.128.219
34.202.251.156
```
```
52.63.101.65
13.211.184.238
18.234.18.91
18.233.128.219
34.202.251.156
```
```
54.207.21.139
54.232.53.97
18.234.18.91
18.233.128.219
34.202.251.156
```
```
2602:817::/44
```
Do not rely on IP filtering as a primary form of security. Always use strong database passwords, and use client certificates if additional security is required. Support for private endpoints is also available in certain scenarios (see below).
## PowerSync Cloud: AWS Private Endpoints
See [Private Endpoints](./private-endpoints) for using a private network to your database using AWS PrivateLink (AWS only).
##
**Version compatibility**: PowerSync requires Postgres version 11 or greater.
Configuring your Postgres database for PowerSync generally involves three tasks:
1. Ensure logical replication is enabled
2. Create a PowerSync database user
3. Create `powersync` logical replication publication
We have documented steps for some specific hosting providers:
### 1. Ensure logical replication is enabled
No action required: Supabase has logical replication enabled by default.
### 2. Create a PowerSync database user
```sql theme={null}
-- Create a role/user with replication privileges for PowerSync
CREATE ROLE powersync_role WITH REPLICATION BYPASSRLS LOGIN PASSWORD 'myhighlyrandompassword';
-- Set up permissions for the newly created role
-- Read-only (SELECT) access is required
GRANT SELECT ON ALL TABLES IN SCHEMA public TO powersync_role;
-- Optionally, grant SELECT on all future tables (to cater for schema additions)
ALTER DEFAULT PRIVILEGES IN SCHEMA public GRANT SELECT ON TABLES TO powersync_role;
```
To restrict read access to specific tables, explicitly list allowed tables for both the `SELECT` privilege, and for the publication mentioned in the next step (as well as for any other publications that may exist).
### 3. Create `powersync` publication
```sql theme={null}
-- Create a publication to replicate tables. The publication must be named "powersync"
CREATE PUBLICATION powersync FOR ALL TABLES;
```
Note that the PowerSync Service has to read all updates present in the publication, regardless of whether the table is referenced in your Sync Streams / Sync Rules definitions. This can cause large spikes in memory usage or introduce replication delays, so if you're dealing with large data volumes, you'll want to specify a comma-separated subset of tables to replicate instead of `FOR ALL TABLES`.
The snippet above replicates all tables and is the simplest way to get started in a dev environment.
Also see our [Supabase integration guide](/integrations/supabase).
### Prerequisites
The instance must be publicly accessible using an IPv4 address.
Note that the PowerSync Service has to read all updates present in the publication, regardless of whether the table is referenced in your Sync Streams / Sync Rules definitions. This can cause large spikes in memory usage or introduce replication delays, so if you're dealing with large data volumes, you'll want to specify a comma-separated subset of tables to replicate instead of `FOR ALL TABLES`.
The snippet above replicates all tables and is the simplest way to get started in a dev environment.
PowerSync supports both **Azure Database for PostgreSQL** and **Azure Database for PostgreSQL Flexible Server**.
### Prerequisites
The database must be accessible on the public internet. Once you have created your database, navigate to **Settings** → **Networking** and enable **Public access.**
### 1. Ensure logical replication is enabled
Follow the steps as noted in [this Microsoft article](https://learn.microsoft.com/en-us/azure/postgresql/flexible-server/concepts-logical#prerequisites-for-logical-replication-and-logical-decoding) to allow logical replication.
### 2. Create a PowerSync database user
```sql theme={null}
-- Create a role/user with replication privileges for PowerSync
CREATE ROLE powersync_role WITH REPLICATION BYPASSRLS LOGIN PASSWORD 'myhighlyrandompassword';
-- Set up permissions for the newly created role
-- Read-only (SELECT) access is required
GRANT SELECT ON ALL TABLES IN SCHEMA public TO powersync_role;
-- Optionally, grant SELECT on all future tables (to cater for schema additions)
ALTER DEFAULT PRIVILEGES IN SCHEMA public GRANT SELECT ON TABLES TO powersync_role;
```
To restrict read access to specific tables, explicitly list allowed tables for both the `SELECT` privilege, and for the publication mentioned in the next step (as well as for any other publications that may exist).
### 3. Create `powersync` publication
```sql theme={null}
-- Create a publication to replicate tables. The publication must be named "powersync"
CREATE PUBLICATION powersync FOR ALL TABLES;
```
Note that the PowerSync Service has to read all updates present in the publication, regardless of whether the table is referenced in your Sync Streams / Sync Rules definitions. This can cause large spikes in memory usage or introduce replication delays, so if you're dealing with large data volumes, you'll want to specify a comma-separated subset of tables to replicate instead of `FOR ALL TABLES`.
The snippet above replicates all tables and is the simplest way to get started in a dev environment.
### 1. Ensure logical replication is enabled
In Google Cloud SQL Postgres, enabling the logical replication is done using flags:
Note that the PowerSync Service has to read all updates present in the publication, regardless of whether the table is referenced in your Sync Streams / Sync Rules definitions. This can cause large spikes in memory usage or introduce replication delays, so if you're dealing with large data volumes, you'll want to specify a comma-separated subset of tables to replicate instead of `FOR ALL TABLES`.
The snippet above replicates all tables and is the simplest way to get started in a dev environment.
### 1. Ensure logical replication is enabled
To [ensure logical replication is enabled](https://neon.tech/docs/guides/logical-replication-postgres#prepare-your-source-neon-database):
1. Select your project in the Neon Console.
2. On the Neon Dashboard, select **Settings**.
3. Select **Logical Replication**.
4. Click **Enable** to ensure logical replication is enabled.
### 2. Create a PowerSync database user
```sql theme={null}
-- Create a role/user with replication privileges for PowerSync
CREATE ROLE powersync_role WITH REPLICATION BYPASSRLS LOGIN PASSWORD 'myhighlyrandompassword';
-- Set up permissions for the newly created role
-- Read-only (SELECT) access is required
GRANT SELECT ON ALL TABLES IN SCHEMA public TO powersync_role;
-- Optionally, grant SELECT on all future tables (to cater for schema additions)
ALTER DEFAULT PRIVILEGES IN SCHEMA public GRANT SELECT ON TABLES TO powersync_role;
```
To restrict read access to specific tables, explicitly list allowed tables for both the `SELECT` privilege, and for the publication mentioned in the next step (as well as for any other publications that may exist).
### 3. Create `powersync` publication
```sql theme={null}
-- Create a publication to replicate tables. The publication must be named "powersync"
CREATE PUBLICATION powersync FOR ALL TABLES;
```
Note that the PowerSync Service has to read all updates present in the publication, regardless of whether the table is referenced in your Sync Streams / Sync Rules definitions. This can cause large spikes in memory usage or introduce replication delays, so if you're dealing with large data volumes, you'll want to specify a comma-separated subset of tables to replicate instead of `FOR ALL TABLES`.
The snippet above replicates all tables and is the simplest way to get started in a dev environment.
Also see our [Neon integration guide](/integrations/neon).
Fly Postgres is a [Fly](https://fly.io/) app with [flyctl](https://fly.io/docs/flyctl/) sugar on top to help you bootstrap and manage a database cluster for your apps.
### 1. Ensure logical replication is enabled
Once you've deployed your Fly Postgres cluster, you can use the following command to ensure logical replication is enabled:
```bash theme={null}
fly pg config update --wal-level=logical
```
Note that the PowerSync Service has to read all updates present in the publication, regardless of whether the table is referenced in your Sync Streams / Sync Rules definitions. This can cause large spikes in memory usage or introduce replication delays, so if you're dealing with large data volumes, you'll want to specify a comma-separated subset of tables to replicate instead of `FOR ALL TABLES`.
The snippet above replicates all tables and is the simplest way to get started in a dev environment.
### 1. Ensure logical replication is enabled
No action required: PlanetScale has logical replication (`wal_level = logical`) enabled by default.
### 2. Create a PowerSync database user
```sql theme={null}
-- Create a role/user with replication privileges for PowerSync
CREATE ROLE powersync_role WITH REPLICATION BYPASSRLS LOGIN PASSWORD 'myhighlyrandompassword';
-- Set up permissions for the newly created role
-- Read-only (SELECT) access is required
GRANT SELECT ON ALL TABLES IN SCHEMA public TO powersync_role;
-- Optionally, grant SELECT on all future tables (to cater for schema additions)
ALTER DEFAULT PRIVILEGES IN SCHEMA public GRANT SELECT ON TABLES TO powersync_role;
```
To restrict read access to specific tables, explicitly list allowed tables for both the `SELECT` privilege, and for the publication mentioned in the next step (as well as for any other publications that may exist).
### 3. Create `powersync` publication
```sql theme={null}
-- Create a publication to replicate tables.
-- PlanetScale does not support ON ALL TABLES so
-- Specify each table you want to sync
-- The publication must be named "powersync"
CREATE PUBLICATION powersync
FOR TABLE public.lists, public.todos;
```
Logical replication can be enabled for [Render Postgres](https://render.com/docs/postgresql) but you need to contact their support team. Here are some prerequisites before contacting them:
* The disk size must be at least 10 GB.
* You must be on a Professional workspace or higher.
The Render support team will ask you the following:
* Database user for replication (you can use the default or create a new user yourself)
* Schema(s)
* Publication name (only if you want them to set `FOR ALL TABLES`; otherwise, you'll be able to create publications per table yourself later)
If you want to create the publication `FOR ALL TABLES`, you must let their support team know that you want the publication name to be `powersync`.
Additional notes they'll share with you:
> We will reserve approximately 1/8 of your storage for `wal_keep_size`. This will not be available for your normal operations and will always be reserved no matter what.
> We will also schedule maintenance for the database to pick up the changes. It will be initially scheduled for 14 days out with a deadline of 30 days out. Once the maintenance is added, you can reschedule to any time between immediately and the deadline. If you do nothing, it will run automatically at the initially scheduled time of 14 days out.
### 1. Ensure logical replication is enabled
```sql theme={null}
ALTER SYSTEM SET wal_level = logical;
ALTER SYSTEM SET max_replication_slots = 10;
ALTER SYSTEM SET max_wal_senders = 10;
```
### 2. Create a PowerSync database user
```sql theme={null}
-- Create a role/user with replication privileges for PowerSync
CREATE ROLE powersync_role WITH REPLICATION BYPASSRLS LOGIN PASSWORD 'myhighlyrandompassword';
-- Set up permissions for the newly created role
-- Read-only (SELECT) access is required
GRANT SELECT ON ALL TABLES IN SCHEMA public TO powersync_role;
-- Optionally, grant SELECT on all future tables (to cater for schema additions)
ALTER DEFAULT PRIVILEGES IN SCHEMA public GRANT SELECT ON TABLES TO powersync_role;
```
To restrict read access to specific tables, explicitly list allowed tables for both the `SELECT` privilege, and for the publication mentioned in the next step (as well as for any other publications that may exist).
### 3. Create `powersync` publication
```sql theme={null}
-- Create a publication to replicate tables. The publication must be named "powersync"
CREATE PUBLICATION powersync FOR ALL TABLES;
```
Note that the PowerSync Service has to read all updates present in the publication, regardless of whether the table is referenced in your Sync Streams / Sync Rules definitions. This can cause large spikes in memory usage or introduce replication delays, so if you're dealing with large data volumes, you'll want to specify a comma-separated subset of tables to replicate instead of `FOR ALL TABLES`.
The snippet above replicates all tables and is the simplest way to get started in a dev environment.
See [Xata's documentation](https://xata.io/docs/tutorials/streaming-replication) for more information on setting up logical replication with Xata.
For other providers and self-hosted databases:
Need help? Simply contact us on [Discord](https://discord.gg/powersync) and we'll help you get set up.
### 1. Ensure logical replication is enabled
PowerSync reads the Postgres WAL using logical replication in order to create [buckets](/architecture/powersync-service#bucket-system) in accordance with your [Sync Streams](/sync/streams/overview) (or legacy [Sync Rules](/sync/rules/overview)).
If you are managing Postgres yourself, set `wal_level = logical` in your config file:
Note that the PowerSync Service has to read all updates present in the publication, regardless of whether the table is referenced in your Sync Streams / Sync Rules definitions. This can cause large spikes in memory usage or introduce replication delays, so if you're dealing with large data volumes, you'll want to specify a comma-separated subset of tables to replicate instead of `FOR ALL TABLES`.
The snippet above replicates all tables and is the simplest way to get started in a dev environment.
### Unsupported Hosted Postgres Providers
Due to the logical replication requirement, not all Postgres hosting providers are supported. Notably, some "serverless Postgres" providers do not support logical replication, and are therefore not supported by PowerSync yet.
### See Also
* [Postgres Maintenance: Logical Replication Slots](/configuration/source-db/postgres-maintenance)
##
**Version compatibility**: PowerSync requires MongoDB version 6.0 or greater.
For more information on migrating from MongoDB Atlas Device Sync to PowerSync, see our [migration guide](/migration-guides/atlas-device-sync).
PowerSync also supports **Azure DocumentDB** (formerly Azure Cosmos DB for MongoDB vCore) through this same MongoDB connector, as an [alpha](/resources/feature-status) feature. See [Azure DocumentDB](#azure-documentdb) below for supported variants and limitations.
### MongoDB Permissions
The permissions PowerSync requires depend on whether you use MongoDB Atlas built-in roles, or a custom role on either self-hosted MongoDB or Atlas.
#### Permissions Required: MongoDB Atlas
For MongoDB Atlas databases, the minimum permissions when using built-in roles are:
```
readWrite@._powersync_checkpoints
read@
```
To allow PowerSync to automatically enable [`changeStreamPreAndPostImages`](#post-images) on replicated collections (i.e. the [**Post Images**](#post-images) setting for the MongoDB connection on your PowerSync instance is set to **Auto-Configure**, which is the default for new PowerSync instances), additionally add the `dbAdmin` permission:
```
readWrite@._powersync_checkpoints
read@
dbAdmin@
```
If you are replicating from multiple databases in the cluster, you need read permissions on the entire cluster, in addition to the above:
```
readAnyDatabase@admin
```
#### Privileges Required: Self-Hosted or Custom Roles
For self-hosted MongoDB, or for creating custom roles on MongoDB Atlas, PowerSync requires the following privileges/granted actions:
* `listCollections`: This privilege must be granted on the database being replicated.
* `find`: This privilege must be granted either at the database level or on specific collections.
* `changeStream`: This privilege must be granted at the database level (not on individual collections). In MongoDB Atlas, set `collection: ""` or check `Apply to any collection` in MongoDB Atlas if you want to apply this privilege on any collection.
* If replicating from multiple databases, this must apply to the entire cluster. Specify `db: ""` or check `Apply to any database` in MongoDB Atlas.
* For the `_powersync_checkpoints` collection add the following privileges: `createCollection`, `dropCollection`, `find`, `changeStream`, `insert`, `update`, and `remove`
* To allow PowerSync to automatically enable [`changeStreamPreAndPostImages`](#post-images) on replicated collections (i.e. the [**Post Images**](#post-images) setting for the MongoDB connection on your PowerSync instance is set to **Auto-Configure**, which is the default for new PowerSync instances), additionally add the `collMod` permission on the database and all collections being replicated.
### Post Images
To replicate data from MongoDB to PowerSync in a consistent manner, PowerSync uses Change Streams with [post-images](https://www.mongodb.com/docs/v6.0/reference/command/collMod/#change-streams-with-document-pre--and-post-images) to get the complete document after each change.
This requires the `changeStreamPreAndPostImages` option to be enabled on replicated collections.
PowerSync supports three configuration options for post-images:
1. **Off**: (`post_images: off`): Uses `fullDocument: 'updateLookup'` for backwards compatibility. This was the default for older instances. However, this may lead to consistency issues, so we strongly recommend enabling post-images instead.
2. **Auto-Configure**: (`post_images: auto_configure`) The **default** for new instances: Automatically enables the `changeStreamPreAndPostImages` option on collections as needed. Requires the permissions/privileges mentioned above. If a collection is removed from [Sync Streams](/sync/streams/overview) (or legacy [Sync Rules](/sync/rules/overview)), you need to manually disable `changeStreamPreAndPostImages` on that collection.
3. **Read-only**: (`post_images: read_only`): Uses `fullDocument: 'required'` and requires `changeStreamPreAndPostImages: { enabled: true }` to be set on every collection referenced in your Sync Streams/Sync Rules. Replication will error if this is not configured. This option is ideal when permissions are restricted.
To manually configure collections for `read_only` mode, run this command on each collection:
```js theme={null}
db.runCommand( {
collMod: ,
changeStreamPreAndPostImages: { enabled: }
} )
```
You can view which collections have the option enabled using:
```js theme={null}
db.getCollectionInfos().filter(
(c) => c.options?.changeStreamPreAndPostImages?.enabled
);
```
Post-images can be configured for PowerSync instances as follows:
Configure the **Post Images** setting in the database connection configuration in the
[PowerSync Dashboard](https://dashboard.powersync.com/). Select your project
and instance and go to **Database Connection** to edit the connection settings.
Configure `post_images` in the `service.yaml` file.
### MongoDB Atlas Private Endpoints Using AWS PrivateLink
If you need to use private endpoints with MongoDB Atlas, see [Private Endpoints](/configuration/source-db/private-endpoints) (AWS only).
### Azure DocumentDB
Azure DocumentDB support is in **alpha**. It is subject to breaking changes and not yet covered by SLAs. See [Feature Status](/resources/feature-status) for what this means for production readiness and support.
PowerSync replicates from Azure DocumentDB (formerly Azure Cosmos DB for MongoDB vCore) through this same MongoDB connector. There is no separate connector or configuration: keep `type: mongodb`, point it at your connection string, and PowerSync detects DocumentDB automatically. Setup, permissions, and connection steps are the same as MongoDB above, except for the Post Images setting noted below.
#### Supported Variants
There are several related Microsoft products that are not interchangeable for replication. PowerSync supports only Azure DocumentDB, formerly known as Azure Cosmos DB for MongoDB vCore:
| Variant | Supported |
| ---------------------------------------------------------------------------------------------------------------------------- | --------- |
| [Azure DocumentDB](https://learn.microsoft.com/en-us/azure/documentdb/) / Azure Cosmos DB for MongoDB vCore | Yes |
| [Azure Cosmos DB for MongoDB (RU-based)](https://learn.microsoft.com/en-us/azure/cosmos-db/mongodb/find-request-unit-charge) | No |
| [Azure Cosmos DB for NoSQL](https://learn.microsoft.com/en-us/azure/cosmos-db/nosql/) | No |
| Self-hosted open-source DocumentDB engine ([`documentdb-local`](https://documentdb.io/docs/documentdb-local)) | No |
The RU-based model is a different engine with a different change-stream implementation. To use it with PowerSync, [migrate it to Azure DocumentDB](https://learn.microsoft.com/en-us/azure/cosmos-db/mongodb/how-to-migrate-documentdb) first. A source that does not report as DocumentDB is treated as standard MongoDB (Atlas, self-hosted, or replica set) and is unaffected.
#### Limitations to Check Before You Connect
DocumentDB is not fully MongoDB-compatible. The following behaviors differ from a standard MongoDB source. For full details, see the [DocumentDB limitations notes](https://github.com/powersync-ja/powersync-service/blob/main/docs/documentdb/documentdb-limitations.md) in the `powersync-service` repository.
* **Source database only.** DocumentDB is supported as a source database, not as PowerSync's [storage database](/architecture/powersync-service#bucket-storage) (especially relevant if you are self-hosting PowerSync). We don't currently plan to support it for storage.
* **Post-images are not supported.** Set [Post Images](#post-images) to **Off** (`post_images: off`, the default); the `auto_configure` and `read_only` modes fail. This does not reduce correctness: DocumentDB always includes the full current document on update events, so updates and deletes still replicate correctly.
* **Collection drop and rename are not replicated.** Already-synced rows remain in PowerSync storage under the old name. To recover, redeploy your Sync Streams to trigger a resync. Regular inserts, updates, and deletes are unaffected.
* **Large initial snapshots may not complete on legacy storage.** DocumentDB keeps source changes for a limited time. On storage versions 1 and 2, PowerSync waits for the initial scan to finish before reading those changes. On a large or busy source, the earliest required changes can expire before the scan finishes, forcing PowerSync to start over. Use [storage version 4](/sync/advanced/storage-version-4), which processes new source changes while the initial snapshot runs.
* **Rows of 15 MiB or larger are not synced.** Such rows are dropped with a logged error. This is a general [PowerSync limit](/resources/performance-and-limits), not DocumentDB-specific, but it is more reachable on DocumentDB because the source permits documents up to 16 MiB and every change event carries the full document.
* **Large documents replicate very slowly.** DocumentDB delivers large change events much more slowly than standard MongoDB. The document still replicates correctly, but latency spikes around large documents (including updates that change only a small field), and a single large event delays every change behind it. If your workload includes large documents, validate replication latency against your cluster.
* **Streaming updates can be delayed by up to roughly 1 second after an idle period.** DocumentDB does not currently support reliable long-polling on its change stream, so PowerSync adds a short, capped delay between polls when the stream is idle. This is a temporary workaround and is expected to be resolved in an upcoming Azure DocumentDB release.
* **Do not modify the internal checkpoints collection.** PowerSync maintains a `_powersync_checkpoints` collection in the source database for replication bookkeeping. Dropping it or deleting its documents disrupts replication.
For implementation and design details, including how PowerSync handles checkpoints and log sequence numbers (LSNs) on DocumentDB, see the [DocumentDB LSN and sentinel checkpoints](https://github.com/powersync-ja/powersync-service/blob/main/docs/documentdb/documentdb-lsn-sentinel-checkpoints.md) notes.
## MySQL support is currently in a [Beta release](/resources/feature-status).
**Version compatibility**: PowerSync requires MySQL version 5.7 or greater.
PowerSync reads from the MySQL [binary log](https://dev.mysql.com/doc/refman/8.4/en/binary-log.html) (binlog) to replicate changes. We use a modified version of the [Zongji MySQL](https://github.com/powersync-ja/powersync-mysql-zongji) binlog listener to achieve this.
### Binlog Configuration
To ensure that PowerSync can read the binary log, you need to configure your MySQL server to enable binary logging and configure it with the following server command options:
* [`server_id`](https://dev.mysql.com/doc/refman/8.4/en/replication-options.html#sysvar_server_id): Uniquely identifies the MySQL server instance in the replication topology. Default value is **`1`**.
* [`log_bin`](https://dev.mysql.com/doc/refman/8.4/en/replication-options-binary-log.html#sysvar_log_bin): **`ON`**. Enables binary logging. Default is **`ON`** for MySQL 8.0 and later, but **`OFF`** for MySQL 5.7.
* [`enforce_gtid_consistency`](https://dev.mysql.com/doc/refman/8.4/en/replication-options-gtids.html#sysvar_enforce_gtid_consistency): **`ON`**. Enforces GTID consistency. Default is **`OFF`**.
* [`gtid_mode`](https://dev.mysql.com/doc/refman/8.4/en/replication-options-gtids.html#sysvar_gtid_mode): **`ON`**. Enables GTID based logging. Default is **`OFF`**.
* [`binlog_format`](https://dev.mysql.com/doc/refman/8.4/en/replication-options-binary-log.html#sysvar_binlog_format): **`ROW`**. Sets the binary log format to row-based replication. This is required for PowerSync to correctly replicate changes. Default is **`ROW`**.
* [`binlog_row_image`](https://dev.mysql.com/doc/refman/8.4/en/replication-options-binary-log.html#sysvar_binlog_row_image): **`FULL`**. Captures the complete row data for each change. This is required for PowerSync to correctly replicate changes. Default is **`FULL`**. The `MINIMAL`/`NOBLOB` options will be supported in a future release.
These can be specified in a MySQL [option file](https://dev.mysql.com/doc/refman/8.4/en/option-files.html):
```
server_id=
log_bin=ON
enforce_gtid_consistency=ON
gtid_mode=ON
binlog_format=ROW
binlog_row_image=FULL
```
### Database User Configuration
PowerSync also requires a MySQL user with **`REPLICATION`** and **`SELECT`** privileges on the source databases. These can be added by running the following SQL commands:
```sql theme={null}
-- Create a user with necessary privileges
CREATE USER 'repl_user'@'%' IDENTIFIED BY '';
-- Grant replication client privilege
GRANT REPLICATION SLAVE, REPLICATION CLIENT ON *.* TO 'repl_user'@'%';
-- Grant select access to the specific database
GRANT SELECT ON .* TO 'repl_user'@'%';
-- Apply changes
FLUSH PRIVILEGES;
```
It is possible to constrain the MySQL user further and limit access to specific tables. Care should be taken to ensure that all the tables in your Sync Streams/Sync Rules are included in the grants.
```sql theme={null}
-- Grant select to the users and the invoices tables in the source database
GRANT SELECT ON .users TO 'repl_user'@'%';
GRANT SELECT ON .invoices TO 'repl_user'@'%';
-- Apply changes
FLUSH PRIVILEGES;
```
### Additional Configuration (Optional)
#### Binlog
The binlog can be configured to limit logging to specific databases. By default, events for tables in all the databases on the MySQL server will be logged.
* [`binlog-do-db`](https://dev.mysql.com/doc/refman/8.4/en/replication-options-binary-log.html#option_mysqld_binlog-do-db): Only updates for tables in the specified database will be logged.
* [`binlog-ignore-db`](https://dev.mysql.com/doc/refman/8.4/en/replication-options-binary-log.html#option_mysqld_binlog-ignore-db): No updates for tables in the specified database will be logged.
Examples:
```
# Only row events for tables in the user_db and invoices_db databases will appear in the binlog.
binlog-do-db=user_db
binlog-do-db=invoices_db
```
```
# Row events for tables in the user_db will be ignored. Events for any other database will be logged.
binlog-ignore-db=user_db
```
## SQL Server support is currently in a [Beta release](/resources/feature-status).
**Version compatibility**:
* PowerSync requires SQL Server 2019+ or Azure SQL
Database.
* SQL Server support was introduced in version 1.18.1 of the PowerSync Service.
PowerSync can replicate data from a change data capture (CDC) enabled SQL Server. The CDC process builds up change tables based on changes to tracked tables, by scanning the SQL Server transaction log on a fixed interval.
PowerSync then polls these change tables using built-in stored procedures and applies the changes to the PowerSync [bucket storage](/architecture/powersync-service#bucket-system).
For more information about CDC, see:
* [Change Data Capture (SQL Server)](https://learn.microsoft.com/en-us/sql/relational-databases/track-changes/about-change-data-capture-sql-server)
* [Change Data Capture (Azure SQL Database)](https://learn.microsoft.com/en-us/azure/azure-sql/database/change-data-capture-overview?view=azuresql)
### Supported Editions/Versions
| Database | Edition | Version | Min Service Tier |
| ---------------- | ------------------------------------------- | ------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| SQL Server 2019+ | Standard, Enterprise, Developer, Evaluation | 15.0+ | N/A |
| Azure SQL\* | Database, Managed instance | N/A | Any service tier on vCore purchasing model. S3 tier and up on DTU purchasing model. See: [Azure SQL Database compute requirements](https://learn.microsoft.com/en-us/azure/azure-sql/database/change-data-capture-overview?view=azuresql#azure-sql-database-compute-requirements) |
\* Azure SQL Database is always running on the latest version of the SQL Server DB Engine
### Limitations / Known Issues
* Schema changes are not adopted automatically. Adopting a schema change requires deploying an updated Sync Config. See [SQL Server Specifics](/maintenance-ops/implementing-schema-changes#sql-server-specifics) for the recommended workflows.
* There is an inherent latency in replicating data from SQL Server to PowerSync. See [Latency](#latency) for more details.
* Spatial data types are returned as JSON objects as supplied by the Tedious `node-mssql` client. See the notes [here](https://github.com/tediousjs/node-mssql?tab=readme-ov-file#geography-and-geometry).
* Wildcard table names (`%`) are not supported. Every replicated table must be listed by name in your Sync Streams.
### Database Setup Requirements
#### 1. Enable CDC on the Database
Change Data Capture (CDC) needs to be enabled on the database:
```sql theme={null}
-- Enable CDC on the database if not already enabled
USE ; -- Only for SQL Server. To switch databases on Azure SQL, you have to connect to the specific database.
IF (SELECT is_cdc_enabled FROM sys.databases WHERE name = '') = 0
BEGIN
EXEC sys.sp_cdc_enable_db;
END
```
#### 2. Create the PowerSync Database User
Create a database user for PowerSync with the following permissions:
**Required permissions:**
* Read/Write permissions on the `_powersync_checkpoints` table
* Read permissions on the replicated tables
* `cdc_reader` role (grants access to CDC changetables and functions)
* `SELECT` permission on the CDC schema (grants access to CDC metadata tables)
* `VIEW DATABASE PERFORMANCE STATE` (SQL Server and Azure SQL)
* `VIEW SERVER PERFORMANCE STATE` (SQL Server only)
Create the login for the user first. This is done on the server / master database level:
```sql theme={null}
-- Create a SQL login for the powersync_user if missing. Note SQL Logins are created at the server level.
USE [master]; -- Use only works on SQL Server. For Azure SQL you have to connect to the master database to run these commands.
IF NOT EXISTS (SELECT 1 FROM sys.server_principals WHERE name = 'powersync_user')
BEGIN
CREATE LOGIN [powersync_user] WITH PASSWORD = 'YOUR_DB_USER_PASSWORD', CHECK_POLICY = ON;
END
```
Create the database user next. This is done on the specific database level:
```sql theme={null}
-- Create the powersync_user database user if missing. Note DB users are created at the database level.
USE []; -- Use only works on SQL Server. For Azure SQL you have to connect to the specific database to run these commands.
IF NOT EXISTS (SELECT 1 FROM sys.database_principals WHERE name = 'powersync_user')
BEGIN
CREATE USER [powersync_user] FOR LOGIN [powersync_user];
END
```
Grant the necessary permissions for the user:
```sql theme={null}
-- Grant SELECT on the specific replicated tables
GRANT SELECT ON dbo. TO [powersync_user];
-- Grant access to CDC tables and functions using the cdc_reader role
IF IS_ROLEMEMBER('cdc_reader', 'powersync_user') = 0
BEGIN
ALTER ROLE cdc_reader ADD MEMBER powersync_user;
END
-- Grant select on the CDC schema
GRANT SELECT ON SCHEMA::cdc TO [powersync_user];
-- Grant the necessary permissions to the user to access the performance state views
-- Note: For Azure SQL, only VIEW DATABASE PERFORMANCE STATE is required. Granted at the database level.
-- PowerSync uses this to access the sys.dm_db_log_stats DMV and the sys.dm_db_partition_stats DMV
GRANT VIEW DATABASE PERFORMANCE STATE TO [powersync_user];
-- VIEW SERVER PERFORMANCE STATE is only necessary on SQL Server (not Azure SQL). Granted at the server/master database level.
-- PowerSync requires this permission to access the sys.dm_db_log_stats DMV on SQL Server.
USE [master];
BEGIN
GRANT VIEW SERVER PERFORMANCE STATE TO [powersync_user];
END
```
For Azure SQL Database, the `VIEW SERVER PERFORMANCE STATE` permission is not
available and not required. Only `VIEW DATABASE PERFORMANCE STATE` is needed.
#### 3. Create the PowerSync Checkpoints Table
PowerSync requires a `_powersync_checkpoints` table to generate regular checkpoints. CDC must be enabled for this table:
```sql theme={null}
-- Create the PowerSync checkpoints table in your schema
IF OBJECT_ID('dbo._powersync_checkpoints', 'U') IS NULL
BEGIN
CREATE TABLE dbo._powersync_checkpoints (
id INT IDENTITY PRIMARY KEY,
last_updated DATETIME NOT NULL DEFAULT GETUTCDATE()
);
END
-- Enable CDC for the powersync checkpoints table if not already enabled
-- Note: the cdc_reader role created the first time CDC is enabled on a table
IF NOT EXISTS (SELECT 1 FROM cdc.change_tables WHERE source_object_id = OBJECT_ID(N'dbo._powersync_checkpoints'))
BEGIN
EXEC sys.sp_cdc_enable_table
@source_schema = N'dbo',
@source_name = N'_powersync_checkpoints',
@role_name = N'cdc_reader',
@supports_net_changes = 0;
END
```
Grant read/write access to the table for the `powersync_user`:
```sql theme={null}
GRANT SELECT, INSERT, UPDATE ON dbo._powersync_checkpoints TO [powersync_user];
```
#### 4. Enable CDC on Tables
CDC must be enabled for all tables that need to be replicated:
```sql theme={null}
-- Enable CDC for specific tables in your schema if not already enabled
IF NOT EXISTS (SELECT 1 FROM cdc.change_tables WHERE source_object_id = OBJECT_ID(N'dbo.'))
BEGIN
EXEC sys.sp_cdc_enable_table
@source_schema = N'dbo',
@source_name = N'',
@role_name = N'cdc_reader',
@supports_net_changes = 0;
END
```
Repeat this for each table you want to replicate. Note that PowerSync does not currently use the net changes functionality so `@supports_net_changes` can be set to `0`.
Every table referenced in your Sync Streams must exist and have CDC enabled before replication can start.
### CDC Management
Management and performance tuning of CDC is left to the developer and is primarily done by modifying the change capture jobs. See [Change Data Capture Jobs (SQL Server)](https://learn.microsoft.com/en-us/sql/relational-databases/track-changes/administer-and-monitor-change-data-capture-sql-server?view=sql-server-ver17) for more details.
Capture Job settings of interest to PowerSync:
* **Polling Interval:** The frequency at which the capture job reads changes from the transaction log. Default is every 5 seconds. Can be set to 0 so that there is zero downtime between scans, but this will impact database performance.
* **Max Trans:** The maximum number of transactions that are processed per scan. Default is 500.
* **Max Scans:** The maximum number of scans that are performed per capture job scan cycle. Default is 10.
Cleanup Job settings of interest to PowerSync:
* **Retention:** The retention period before data is expired from the CDC tables. Default is 3 days. If your PowerSync instance is offline for longer than this period, data will need to be fully re-synced. Specified in minutes.
Recommended Capture Job settings:
| Parameter | Recommended Value |
| ----------------- | ----------------- |
| `maxtrans` | 5000 |
| `maxscans` | 10 |
| `pollinginterval` | 1 second |
For Azure SQL Database, the CDC capture and cleanup jobs are managed automatically. Manual configuration is greatly limited.
See [Azure CDC Customization Limitations](https://learn.microsoft.com/en-us/azure/azure-sql/database/change-data-capture-overview?view=azuresql#cdc-customization).
The main limitation is that the capture job polling interval cannot be modified and is fixed at 20 seconds. It is, however, still possible to [manually trigger](https://learn.microsoft.com/en-us/azure/azure-sql/database/change-data-capture-overview?view=azuresql#manual-cdc-control) the capture job on demand.
#### Latency
Due to the fundamental differences in how CDC works compared to logical replication (Postgres) or binlog reading (MySQL), there is an inherent latency in replicating data from SQL Server to PowerSync. The latency is determined by two factors:
1. **Transaction Log Scan Interval**: The frequency at which the CDC capture job scans the transaction log for changes. The default value of 5 seconds can be changed by modifying the capture job settings on SQL Server. The recommended value is 1 second, but this can also be set to 0 based on the database load. For Azure SQL Database, the default value is 20 seconds and cannot be changed. See [Azure CDC Customization Limitations](https://learn.microsoft.com/en-us/azure/azure-sql/database/change-data-capture-overview?view=azuresql#cdc-customization) for more details.
2. **Polling Interval**: The frequency at which PowerSync polls the CDC change tables for changes. The default value is once every 1000ms. This can be changed by setting the `pollingIntervalMs` parameter in the PowerSync configuration.
### Memory Management
During each polling cycle, PowerSync will read a limited number of transactions from the CDC change tables. The default value of 10 transactions can be changed by setting the `pollingBatchSize` parameter in the PowerSync configuration.
Increasing this will increase throughput at the cost of increased memory usage. If the volume of transactions being replicated is high, and memory is available, it is recommended to increase this value.
Connection configuration parameters for the PowerSync SQL Server Adapter like `pollingIntervalMs` and `pollingBatchSize` can currently only be set when [self-hosting](/intro/setup-guide#self-hosted-2) PowerSync.
See [SQL Server Additional Configuration](/configuration/source-db/sql-server-additional-configuration#additional-configuration-options)
for more details.
We are planning to expose these settings for SQL Server source database connections in the PowerSync Dashboard for PowerSync Cloud instances.
##
The Convex replicator is currently released as an [experimental feature](/resources/feature-status). APIs and
behavior may change, and the developer experience isn't yet on par with our other backend database connectors.
We also can't yet guarantee continued support or long-term stability. This release is intended for early testing
and to invite feedback. Your feedback will directly influence whether, and how, this integration evolves.
PowerSync reads Convex data using the Convex Streaming Export API. Initial replication pins a single Convex snapshot cursor and snapshots each selected table at that cursor. Streaming replication then reads the global `document_deltas` stream and filters rows according to your Sync Streams.
Convex does not support user-defined database schemas or namespaces in the same way as SQL databases. You do not need to qualify Convex table names in Sync Streams. If you do choose to qualify them, use the default `convex` schema.
### Connection Requirements
PowerSync can replicate from Convex Cloud or self-hosted Convex deployments. It requires:
* A Convex deployment URL.
* A Convex deploy key. In the Convex Dashboard, go to **Settings** → **General** and generate a deploy key for the deployment PowerSync should replicate. Create the key with **Custom permissions** and include `deployment:data:view`; the **Deploy only** option does not work for PowerSync replication.
* The `powersync_checkpoints` table and `createCheckpoint` mutation described below.
Use a deploy key for the correct environment, grant only the permissions PowerSync needs, store it as a secret, and
rotate it if it is exposed.
### Checkpoint Table
PowerSync uses a small Convex table to generate write checkpoint markers. Convex table names cannot start with `_`, so the table is named `powersync_checkpoints`.
Add the table to your Convex schema:
```typescript convex/schema.ts theme={null}
import { defineSchema, defineTable } from 'convex/server';
import { v } from 'convex/values';
export default defineSchema({
// ... your other tables
powersync_checkpoints: defineTable({
last_updated: v.float64()
})
});
```
### Checkpoint Mutation
Deploy a Convex mutation named `powersync_checkpoints:createCheckpoint`. PowerSync calls this mutation after recording a write checkpoint so the Convex delta stream advances even when the app is otherwise idle.
```typescript convex/powersync_checkpoints.ts theme={null}
import { mutation } from './_generated/server';
export const createCheckpoint = mutation({
args: {},
handler: async (ctx) => {
const existing = await ctx.db.query('powersync_checkpoints').first();
if (existing) {
await ctx.db.patch(existing._id, { last_updated: Date.now() });
} else {
await ctx.db.insert('powersync_checkpoints', { last_updated: Date.now() });
}
}
});
```
PowerSync excludes `powersync_checkpoints` from replicated source tables. The table exists only to advance the replication cursor for write checkpoint acknowledgements.
### Latency
PowerSync polls the Convex `document_deltas` endpoint for changes. This means there is an inherent latency between a write being committed in Convex and that change being replicated into PowerSync.
The polling interval defaults to once every 1000ms. When self-hosting PowerSync, you can configure this with the `polling_interval_ms` connection parameter.
Convex API requests time out after 60000ms by default. When self-hosting PowerSync, you can configure this with the optional `request_timeout_ms` connection parameter:
```yaml service.yaml theme={null}
replication:
connections:
- type: convex
deployment_url: https://.convex.cloud
deploy_key:
polling_interval_ms: 1000
request_timeout_ms: 60000
```
Lowering `polling_interval_ms` can reduce replication latency, but it increases the number of requests made to Convex and the work performed by the PowerSync Service. For more on monitoring and reducing replication lag, see [Replication Lag](/maintenance-ops/replication-lag).
### Client ID Mapping
Convex generates its own document IDs and exposes them as `_id`. PowerSync clients need a stable local ID before a write is uploaded, so plan your Convex schema around a client-generated UUID rather than the Convex `_id`.
Clients cannot create Convex IDs before inserting documents into Convex. Use a client-generated UUID column as the
synced `id` and keep `_id` as the Convex server-generated document ID. This is similar to the pattern described in
[Sequential ID Mapping](/client-sdks/advanced/sequential-id-mapping).
The client creates a UUID in its local `id` column before the write is uploaded. Your Convex mutation should store that value in a separate `uuid` field on the Convex document. PowerSync then syncs `uuid AS id` back to the client, so the client keeps the same stable local ID while Convex keeps its own server-generated `_id`.
For a complete [Sync Streams](/sync/streams/overview) configuration that applies this mapping, see [Convex with ID Mapping](/sync/streams/examples#convex-with-id-mapping). For how Convex data types map to SQLite columns, see [Convex Type Mapping](/sync/types#convex-type-mapping).
### Client Writes
Apply queued client-side writes to Convex by calling Convex mutations from your backend connector. See [Writing Client Changes](/handling-writes/writing-client-changes#convex) for the upload path and authentication details.
### Demo App
For a working JavaScript example, see the [PowerSync Convex To-Do List Demo](https://github.com/powersync-community/powersync-convex-todolist-demo).
## Next Step
Next, connect PowerSync to your database:
# Advanced SQL Server Configuration
Source: https://docs.powersync.com/configuration/source-db/sql-server-additional-configuration
Configure advanced SQL Server adapter options for self-hosted PowerSync.
The options below apply when self-hosting the PowerSync Service with a SQL Server source database. Cloud-hosted PowerSync instances do currently not expose these settings.
## Connect to Azure SQL Database Using a Service Principal
As an alternative to using standard SQL Login via username and password, PowerSync can connect to your Azure SQL Database using a Service Principal.
Update your `service.yaml` file to use the Service Principal for authentication:
```yaml service.yaml theme={null}
# Settings for source database replication
replication:
connections:
- type: mssql
database:
schema: # Optional, defaults to dbo
hostname:
port: 1433 # Default port for SQL Server
authentication:
type: azure-active-directory-service-principal-secret
options:
clientId:
tenantId:
clientSecret:
```
### Create a New App Registration
1. Go to the [Azure portal](https://portal.azure.com/) and navigate to **App registrations** >.
2. Click **New registration**.
3. Fill in the required fields:
* **Name**: Give your app user a name, e.g. "powersync\_user".
* **Redirect URI**: Leave this blank.
* **Supported account types**: Select **Accounts in this organizational directory only**.
4. Click **Register**.
5. Copy the **Application (Client) ID** and **Directory (Tenant) ID**.
### Create a Client Secret
1. Navigate to the app registration you created in the previous step.
2. In the left navigation menu, under **Manage**, click **Certificates & secrets**.
3. Click **New client secret**.
4. Fill in the required fields:
* **Description**: Enter a description for the secret
* **Expires**: Select an expiration date
5. Click **Add**.
6. **Important**: Copy the **Value** - this is the client secret and will not be shown again.
### Configure the Database User
1. Connect to the SQL Server database using an account with administrative privileges.
2. Create a new user linked to the app registration:
```sql theme={null}
CREATE USER [powersync_user] FROM EXTERNAL PROVIDER;
```
3. Complete the rest of the user setup by following the instructions in [Source Database Setup](/configuration/source-db/setup#2-create-the-powersync-database-user).
## Additional Configuration Options
The below configuration options can be set in the `additionalConfig` section under the `connections` section of the `service.yaml` file.
All of these configuration options are optional and can be omitted if the
default values are acceptable.
**pollingIntervalMs**:
The polling interval is the frequency at which the PowerSync SQL Server Adapter polls the SQL Server Change tables for changes.
Lowering this value helps reduce latency in the replication process, but also increases the load on the SQL Server database and the PowerSync Service.
**pollingBatchSize**
The polling batch size is the number of transactions that the PowerSync SQL Server Adapter will process in each polling cycle.
Increasing this value can increase throughput at the cost of increased memory usage. If the volume of transactions being replicated is high, and memory is available, it is recommended to increase this value.
**trustServerCertificate**:
Whether to trust the server certificate. Set to true for local development and self-signed certificates. Set to false when connecting to Azure SQL Database.
```yaml service.yaml theme={null}
connections:
- type: mssql
additionalConfig:
pollingIntervalMs: 1000 # Default is 1000ms
pollingBatchSize: 10 # Default is 10
trustServerCertificate: false # Default is false
```
# Error Codes Reference
Source: https://docs.powersync.com/debugging/error-codes
Reference of PowerSync error codes with explanations and troubleshooting guidance.
This reference documents PowerSync error codes organized by component, with troubleshooting suggestions for developers. Use the search bar to look up specific error codes (e.g., `PSYNC_R0001`).
## PSYNC\_Rxxxx: Sync Config issues
* **PSYNC\_R0001**:
Catch-all [Sync Config](/sync/overview) parsing error, if no more specific error is available
* **PSYNC\_R2201**:
A table or schema wildcard (`%`) is not supported by the configured source connector.
Wildcard support is connector-specific. For example, SQL Server requires exact table and schema names, while some other connectors support table wildcards but not schema wildcards.
## PSYNC\_Sxxxx: Service issues
* **PSYNC\_S0001**:
Internal assertion.
If you see this error, it might indicate a bug in the service code.
* **PSYNC\_S0102**:
TEARDOWN was not acknowledged.
This happens when the TEARDOWN argument was not supplied when running
the service teardown command. The TEARDOWN argument is required since
this is a destructive command.
Run the command with `teardown TEARDOWN` to confirm.
### PSYNC\_S1xxx: Replication issues
* **PSYNC\_S1002**:
Row too large.
There is a 15MB size limit on every replicated row - rows larger than
this cannot be replicated.
* **PSYNC\_S1003**:
Replication stream has been locked by another process for replication.
This error is normal in some circumstances:
1. In some cases, if a process was forcefully terminated, this error may occur for up to a minute.
2. During rolling deploys, this error may occur until the old process stops replication.
If the error persists for longer, this may indicate that multiple replication processes are running.
Make sure there is only one replication process apart from rolling deploys.
* **PSYNC\_S1004**:
JSON nested object depth exceeds the limit of 20.
This may occur if there is very deep nesting in JSON or embedded documents.
* **PSYNC\_S1005**:
Storage version not supported.
This could be caused by a downgrade to a version that does not support the current storage version.
### PSYNC\_S11xx: Postgres replication issues
* **PSYNC\_S1101**:
Replication assertion error.
If you see this error, it might indicate a bug in the service code.
* **PSYNC\_S1103**:
Aborted initial replication.
This is not an actual error - it is expected when the replication process
is stopped, or if replication is stopped for any other reason.
* **PSYNC\_S1104**:
Explicit cacert is required for `sslmode: verify-ca`.
Use either verify-full, or specify a certificate with verify-ca.
* **PSYNC\_S1105**:
`database` is required in connection config.
Specify the database explicitly, or in the `uri` field.
* **PSYNC\_S1106**:
`hostname` is required in connection config.
Specify the hostname explicitly, or in the `uri` field.
* **PSYNC\_S1107**:
`username` is required in connection config.
Specify the username explicitly, or in the `uri` field.
* **PSYNC\_S1108**:
`password` is required in connection config.
Specify the password explicitly, or in the `uri` field.
* **PSYNC\_S1109**:
Invalid database URI.
Check the URI scheme and format.
* **PSYNC\_S1110**:
Invalid port number.
Only ports in the range 1024 - 65535 are supported.
* **PSYNC\_S1120**:
Error querying the source database.
See the error details for more information about the underlying query failure.
* **PSYNC\_S1121**:
Socket timeout while querying the source database.
This can happen if Postgres takes too long to respond to a query, or if there is
a network-level issue.
* **PSYNC\_S1141**:
Publication does not exist.
Run: `CREATE PUBLICATION powersync FOR ALL TABLES` on the source database.
* **PSYNC\_S1142**:
Publication does not publish all changes.
Create a publication using `WITH (publish = "insert, update, delete, truncate")` (the default).
* **PSYNC\_S1143**:
Publication uses `publish_via_partition_root`.
* **PSYNC\_S1144**:
Invalid Postgres server configuration for replication and sync bucket storage.
The same Postgres server, running an unsupported version of Postgres, has been configured for both replication and sync bucket storage.
Using the same Postgres server is only supported on Postgres 14 and above.
This error typically indicates that the Postgres version is below 14.
Either upgrade the Postgres server to version 14 or above, or use a different Postgres server for sync bucket storage.
* **PSYNC\_S1145**:
Table has RLS enabled, but the replication role does not have the BYPASSRLS attribute.
We recommend using a dedicated replication role with the BYPASSRLS attribute for replication:
```sql theme={null}
ALTER ROLE powersync_role BYPASSRLS
```
An alternative is to create explicit policies for the replication role. If you have done that,
you may ignore this warning.
* **PSYNC\_S1146**:
Replication slot invalidated.
Postgres invalidated the replication slot, typically because WAL retention exceeded `max_slot_wal_keep_size` during a long-running snapshot. To recover, increase `max_slot_wal_keep_size` on the source database, then either redeploy your Sync Config or delete the existing replication slot. PowerSync creates a new slot and restarts replication automatically.
Other causes: `rows_removed` (catalog rows needed by the slot were removed), `wal_level_insufficient`, or `idle_timeout`.
`idle_timeout` is a Postgres 18+ slot invalidation. In this case, increase `idle_replication_slot_timeout` instead of `max_slot_wal_keep_size`.
See [Managing and Monitoring Replication Lag](/maintenance-ops/production-readiness-guide#managing-and-monitoring-replication-lag) for guidance on sizing `max_slot_wal_keep_size`.
### PSYNC\_S12xx: MySQL replication issues
### PSYNC\_S13xx: MongoDB replication issues
* **PSYNC\_S1301**:
Generic MongoServerError.
* **PSYNC\_S1302**:
Generic MongoNetworkError.
* **PSYNC\_S1303**:
MongoDB internal TLS error.
If connection to a shared cluster on MongoDB Atlas, this could be an IP Access List issue.
Check that the service IP is allowed to connect to the cluster.
* **PSYNC\_S1304**:
MongoDB connection DNS error.
Check that the hostname is correct.
* **PSYNC\_S1305**:
MongoDB connection timeout.
Check that the hostname is correct, and that the service IP is allowed to connect to the cluster.
* **PSYNC\_S1306**:
MongoDB authentication error.
Check the username and password.
* **PSYNC\_S1307**:
MongoDB authorization error.
Check that the user has the required privileges. See the required permissions for the [MongoDB source database](/configuration/source-db/setup#mongodb-permissions) or the [MongoDB storage database](/configuration/powersync-service/self-hosted-instances#mongodb-storage), depending on which connection reported the error.
* **PSYNC\_S1341**:
Sharded MongoDB Clusters are not supported yet.
* **PSYNC\_S1342**:
Standalone MongoDB instances are not supported - use a replica-set.
* **PSYNC\_S1343**:
PostImages not enabled on a source collection.
Use `post_images: auto_configure` to configure post images automatically, or enable manually:
```
db.runCommand({
collMod: 'collection-name',
changeStreamPreAndPostImages: { enabled: true }
});
```
* **PSYNC\_S1344**:
The MongoDB Change Stream has been invalidated.
Possible causes:
* Some change stream documents do not have postImages.
* startAfter/resumeToken is not valid anymore.
* The replication connection has changed.
* The database has been dropped.
Replication will be stopped for this Change Stream. Replication will restart with a new Change Stream.
* **PSYNC\_S1345**:
Failed to read MongoDB Change Stream due to a timeout.
This may happen if there is a significant delay on the source database in reading the change stream.
If this is not resolved after retries, replication may need to be restarted from scratch.
To help avoid this error, the Service writes periodic heartbeats to the `_powersync_checkpoints` collection when the change stream has not advanced recently. For self-hosted instances, the heartbeat frequency can be tuned with the `heartbeat_interval_seconds` [connection option](/configuration/powersync-service/self-hosted-instances#replication).
* **PSYNC\_S1346**:
Failed to read MongoDB Change Stream.
See the error cause for more details.
* **PSYNC\_S1347**:
Timeout while getting a resume token for an initial snapshot.
This may happen if there is very high load on the source database.
### PSYNC\_S14xx: MongoDB storage replication issues
* **PSYNC\_S1402**:
Max transaction tries exceeded.
* **PSYNC\_S1500**:
Required updates in the Change Data Capture (CDC) are no longer available.
Possible causes:
* Older data has been cleaned up due to exceeding the retention period.
### PSYNC\_S16xx: MSSQL replication issues
* **PSYNC\_S1601**:
The CDC capture instance a replicated source table was bound to has been dropped.
Possible causes:
* CDC has been disabled for the table.
* The table has been dropped, which also drops the capture instance.
Replication stops and requires a Sync Config deployment. Re-enabling CDC creates a new capture instance, which the existing replication process will not adopt.
* **PSYNC\_S1602**:
A table in the Sync Config is not ready to be replicated.
Possible causes:
* CDC has not been enabled for the table.
* The table does not exist in the source database.
For a table that has not been replicated by this deployment before, replication starts once the table exists and has CDC enabled, without a new Sync Config deployment.
For a table this deployment was already replicating, re-enabling CDC is not enough: the new capture instance has a different object ID, so the next attempt fails with `PSYNC_S1601` until the Sync Config is redeployed.
* **PSYNC\_S1603**:
A replicated source table was dropped, renamed, recreated, or no longer matches the source identity selected by the current replication process.
Replication stops because the table may still have unread changes. A Sync Config deployment is required to adopt the replacement. Existing replicated data is retained until then.
* **PSYNC\_S1604**:
MSSQL connection configuration error.
### PSYNC\_S2xxx: Service API
* **PSYNC\_S2001**:
Generic internal server error (HTTP 500).
See the error details for more info.
* **PSYNC\_S2002**:
Route not found (HTTP 404).
* **PSYNC\_S2003**:
503 service unavailable due to restart.
Wait a while then retry the request.
* **PSYNC\_S2004**:
415 unsupported media type.
This code always indicates an issue with the client.
### PSYNC\_S21xx: Auth errors originating on the client
This does not include auth configuration errors on the service.
* **PSYNC\_S2101**:
Generic authentication error.
Common causes:
1. **JWT signing key mismatch** (Supabase): The client is using tokens signed with a different key type (legacy vs. new JWT signing keys) than PowerSync expects. If you've migrated to new JWT signing keys, ensure users sign out and back in to get fresh tokens. See [Migrating from Legacy to New JWT Signing Keys](/installation/authentication-setup/supabase-auth#migrating-from-legacy-to-new-jwt-signing-keys).
2. **Missing or invalid key ID (kid)**: The token's kid header doesn't match any keys in PowerSync's keystore.
3. **Incorrect JWT secret or JWKS endpoint**: Verify your authentication configuration matches your auth provider's settings.
* **PSYNC\_S2102**:
Could not verify the auth token signature.
Typical causes include:
1. Token kid is not found in the keystore.
2. Signature does not match the kid in the keystore.
* **PSYNC\_S2103**:
Token has expired. Check the expiry date on the token.
* **PSYNC\_S2104**:
Token expiration period is too long. Issue shorter-lived tokens.
* **PSYNC\_S2105**:
Token audience does not match expected values.
Check the aud value on the token, compared to the audience values allowed in the service config.
* **PSYNC\_S2106**:
No token provided. An auth token is required for every request.
The Authorization header must start with "Token" or "Bearer", followed by the JWT.
### PSYNC\_S22xx: Auth integration errors
* **PSYNC\_S2201**:
Generic auth configuration error. See the message for details.
* **PSYNC\_S2202**:
IPv6 support is not enabled for the JWKS URI.
Use an endpoint that supports IPv4.
* **PSYNC\_S2203**:
IPs in this range are not supported.
Make sure to use a publicly-accessible JWKS URI.
* **PSYNC\_S2204**:
JWKS request failed.
### PSYNC\_S23xx: Sync API errors
* **PSYNC\_S2302**:
No Sync Config available.
This error may happen if:
1. Sync Streams/Rules have not been deployed.
2. Sync Streams/Rules have been deployed, but are still busy processing.
View the replicator logs to see if the Sync Streams/Rules are being processed.
* **PSYNC\_S2304**:
Maximum active concurrent clients limit has been reached.
* **PSYNC\_S2305**:
Too many buckets, or too many parameter query results.
This code covers two limits per active connection, both with a default of 1,000: the number of unique buckets (`Too many buckets`) and the number of parameter query results returned before deduplication (`Too many parameter query results`). Read the message to see which limit you reached, then see [Reducing Bucket Count](/sync/advanced/reducing-bucket-count) to diagnose and resolve this error.
### PSYNC\_S24xx: Sync API errors - MongoDB Storage
* **PSYNC\_S2401**:
Could not get clusterTime.
* **PSYNC\_S2402**:
Failed to connect to the MongoDB storage database.
* **PSYNC\_S2403**:
Query timed out. Could be due to a large query or a temporary load issue on the storage database.
Retry the request.
* **PSYNC\_S2404**:
Query failure on the storage database. See error details for more information.
### PSYNC\_S3xxx: Service configuration issues
### PSYNC\_S31xx: Auth configuration issues
* **PSYNC\_S3102**:
Invalid jwks\_uri.
* **PSYNC\_S3103**:
Only http(s) is supported for jwks\_uri.
### PSYNC\_S32xx: Replication configuration issue
* **PSYNC\_S3201**:
Failed to validate module configuration.
### PSYNC\_S4000: Management / Dev APIs
* **PSYNC\_S4001**:
Internal assertion error.
This error may indicate a bug in the service code.
* **PSYNC\_S4104**:
No active Sync Config.
* **PSYNC\_S4105**:
Sync Streams API disabled.
When a Sync Streams/Rules file is configured, the dynamic Sync Streams API is disabled.
* **PSYNC\_S4106**:
Sync Config reprocess is blocked because another Sync Config is deploying.
# Log Reference
Source: https://docs.powersync.com/debugging/log-reference
Understand common PowerSync Service log messages and recognize replication progress, sync activity, and errors.
Use this page to look up messages in your [Instance Logs](/maintenance-ops/monitoring-and-alerting#instance-logs). Sync & API logs cover client connections and downloads. Replicator logs cover data moving from your source database into the PowerSync Service.
Message wording and available fields vary by Service version, source database, and storage backend.
## Reading Error Entries
Service errors use `[PSYNC_] `, sometimes followed by details. Look up the code in [Error Codes](/debugging/error-codes).
`[PSYNC_S2001] Something went wrong` reports an unexpected error. The details line contains the original error message.
## Sync & API Logs
### Finding a Session
Use `user_id` to identify a user's entries and `rid` to follow one session. For Dashboard search queries, see [Correlating Sync Sessions](/maintenance-ops/monitoring-and-alerting#correlating-sync-sessions).
### Session Lifecycle
A checkpoint represents a consistent server state. See [Consistency](/architecture/consistency). Checkpoint messages can repeat throughout a session. A connection can end before the Service sends a checkpoint.
| Message | Meaning | What to Check |
| ------------------------------------------------- | ---------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------- |
| `Sync stream started` | An authenticated client started a sync session. | Data transfer follows this entry. Look for `checkpoint_complete` to see when the Service finishes sending a checkpoint's data. |
| `New checkpoint: ...` / `Updated checkpoint: ...` | The Service sends a full checkpoint or changes since the previous one. | `buckets` counts the user's buckets. `param_results` counts parameter query results when applicable. |
| `checkpoint_complete: ...` | The Service finished sending this checkpoint's data. | This entry does not confirm that the client has written the data to its local database. The session stays open for further changes. |
| `partial_checkpoint_complete: ...` | The Service finished sending a priority group during prioritized sync. | `priority` identifies the completed group. |
| `checkpoint_invalidated: ...` | The Service needs a newer checkpoint to continue. | Check `reason` for why the Service could not finish this checkpoint. |
| `Sync stream error` | An error interrupted a WebSocket session. | Read the error logged for the same `rid`. |
| `Streaming sync request failed` | An error interrupted an HTTP session. | Read the error logged for the same `rid`. |
| `Sync stream complete` | The session ended. | Check `close_reason` and the session totals. `stream_ms` measures the whole session, not the initial download. |
| `POST /sync/stream` / `STREAM /sync/stream` | The HTTP or WebSocket request ended. | `duration_ms` covers the whole session. The HTTP request can report status `200` even if an error occurred during streaming. |
### Session Fields
`Sync stream started` and `Sync stream complete` share the same `rid`. Both entries can include these fields:
| Field | Meaning |
| -------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `rid` | Request ID that identifies the session. |
| `user_id` | Authenticated user ID. |
| `client_id` | Identifier that distinguishes clients belonging to the same user. |
| `user_agent` | Client-reported SDK or user-agent information. |
| `app_metadata` | Optional application context, such as app version. See [Custom Metadata in Sync Logs](/maintenance-ops/monitoring-and-alerting#custom-metadata-in-sync-logs) to set it. |
The started entry also includes `client_params` when the client supplies query parameters. The complete entry adds session results:
| Field | Meaning |
| ------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `operations_synced` | Total operations synced during the session. |
| `operation_counts` | Operation totals by type: `put`, `remove`, `move`, and `clear`. |
| `data_synced_bytes` | Total sync data before compression, in bytes. |
| `data_sent_bytes` | Bytes sent, accounting for compression when enabled. |
| `stream_ms` | Session duration in milliseconds, including time waiting for changes. For download timing, see [Diagnosing Sync Latency](/debugging/troubleshooting#diagnosing-sync-latency). |
| `close_reason` | Why the session ended. See [Close Reasons](#close-reasons). |
On checkpoint entries, `operations_synced` and `data_synced_bytes` count only the operations and bytes since the previous reported counts, or since the session started.
### Close Reasons
The `close_reason` records the first relevant close event the Service observed:
| Value | Meaning |
| ------------------------ | ------------------------------------------------------------------------------------------------------------------------ |
| `client closing stream` | The client side closed the connection. Check [client-side logs](/debugging/troubleshooting#client-side-logging) for why. |
| `service closing stream` | The Service ended the stream, for example because the token expired or the Service switched to a new Sync Config. |
| `stream error` | An error interrupted the stream. Read the error for the same `rid`. |
| `process shutdown` | The process is shutting down, for example during a deploy. |
| `unknown` | The Service did not identify a close reason. Check nearby messages. |
### Other Requests
HTTP endpoint requests log a ` ` line with `status`, `duration_ms`, and `rid`. Failures can also log `Request failed`. Requests rejected before reaching an endpoint may have fewer fields.
| Message | Meaning |
| -------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `Write checkpoint for /: \| ` | A client requested a [write checkpoint](/architecture/powersync-protocol#write-checkpoints), typically after uploading changes. The entry gives the checkpoint number and its source replication position. |
| `Requested checkpoint for ...` | The Service handled a [Checkpoint Request](/client-sdks/advanced/checkpoint-requests). The result can be a new or existing checkpoint. |
| `Currently have active WebSocket connection(s)` | This process's connection count, including the new connection before authentication and limit checks. |
| ` ` with `status: 429` and `queue_overflow: true` | The request queue is full. For HTTP sync connections, the process reached its concurrent connection limit. |
| `[PSYNC_S2304] Maximum active concurrent connections limit has been reached` | The process rejected a WebSocket connection because it reached its connection limit. |
For self-hosted instances, set the limit with `api.parameters.max_concurrent_connections`. The Service applies this limit separately to HTTP sync connections and WebSocket connections. See [Self-Hosted Instances](/configuration/powersync-service/self-hosted-instances). For Cloud limits, see the [Usage and Billing FAQ](/resources/usage-and-billing/usage-and-billing-faq).
### Common Sync & API Errors
Authentication errors have status `401`. Repeated entries can indicate a client retrying. When present, `tokenDetails` describes the rejected token without logging it in full. See [Auth Errors](/debugging/error-codes#psync_s21xx-auth-errors-originating-on-the-client).
| Entry | Meaning |
| ----------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `[PSYNC_S2101] ...` | The Service could not verify the client's token. |
| `[PSYNC_S2103] JWT has expired` | The client's token has expired. |
| `[PSYNC_S2106] Authentication required` | The token is missing or supplied in an invalid authorization format. WebSocket wording can differ. |
| `[PSYNC_S2204] JWKS request failed` | The Service could not fetch signing keys from the JSON Web Key Set (JWKS) endpoint. Check your [JWKS configuration](/debugging/error-codes#psync_s22xx-auth-integration-errors). |
| `[PSYNC_S2302] No sync config available` | No Sync Config is active. If a deployment is still processing, check Replicator logs. Otherwise, deploy a Sync Config. |
| `[PSYNC_S2305] Too many buckets: ...` / `Too many parameter query results ...` | The user exceeded a limit. Look for counts by stream or query in nearby entries with the same `rid`. See [Reducing Bucket Count](/sync/advanced/reducing-bucket-count). |
| `[PSYNC_S2403] Query timed out while reading checksums` / `... while reading bucket data` | A MongoDB bucket storage query timed out, possibly due to a large query or database load. See [MongoDB Storage Errors](/debugging/error-codes#psync_s24xx-sync-api-errors-mongodb-storage). |
## Replicator Logs
Replication copies source data into the Service for clients to sync. A snapshot copies existing data. Ongoing replication copies new changes. Depending on the source database, both operations can run at the same time.
### Message Prefixes
Many Replicator messages identify the replication stream and Sync Config version:
```text theme={null}
[powersync__1_246a][3] Replicating "public"."large_table" 5040000/~10393720
```
Search for the first bracket's stream name to follow related messages. Here, `[3]` is the Sync Config version label. With MongoDB bucket storage, a stream can list several versions, such as `[3,4]`. MongoDB source messages can instead use `[snapshot]` or `[stream]` to identify the work.
If the version changes but the stream name stays the same, the Service reused the replication stream. A full reprocess changes the stream name. See [Incremental Reprocessing](/sync/advanced/storage-version-4#incremental-reprocessing).
### Snapshot Progress
These messages cover Postgres, MongoDB, and SQL Server snapshots. SQL Server uses `Snapshotting` where the others use `Replicating`.
| Message | Meaning |
| ------------------------------------------------------ | ---------------------------------------------------------------------------------------------------------------------------- |
| `To replicate: ...` | The Service lists a table or collection to snapshot from Postgres or MongoDB. |
| `Replicating /~ ...` | Rows or documents processed so far, out of an estimated total. `?` means no estimate is available. |
| `Skipping - snapshot already done` | The table's snapshot is complete, so the Service skips it. |
| `Skipping - not part of powersync publication` | The Postgres table matches your Sync Config but is excluded from the `powersync` publication. |
| `Initial replication already done` | The Service found a completed snapshot. For Postgres sources, the Service still checks the replication slot before resuming. |
For example, `5040000/~10393720` means 5,040,000 rows processed out of an estimated 10,393,720. A `resuming from` or `resuming at` suffix means the snapshot is continuing from saved progress.
### Flushed Batches
`Flushed` reports a batch of writes to bucket storage. With MongoDB bucket storage, the message has this form:
```text theme={null}
Flushed: ops, index entries, records. kb in ms. Last op_id: . Replication lag: s
```
The same entry includes structured properties under `flushed`:
| Message Value | Property | Meaning |
| ----------------- | ------------------------- | --------------------------------------------------------------------------------------------------------------------------- |
| `ops` | `bucket_ops_count` | Operations written to [buckets](/architecture/powersync-service#bucket-system), the groups of data clients sync. |
| `index entries` | `parameter_indexes_count` | Parameter index entries used to determine which buckets a client receives. |
| `records` | `source_records_count` | Writes to PowerSync's stored source records. |
| `kb` | `size` | Batch size in bytes. The message displays this value in rounded kilobytes. |
| `ms` | `duration` | Time taken to write the batch, in milliseconds. |
| `Replication lag` | `replication_lag_seconds` | Age of the oldest uncommitted source change in the batch, in seconds. Present only when the Service can determine that age. |
The counts can differ. For example, `0 ops, 0 index entries, 2000 records` means the Service wrote source records without bucket operations or parameter index entries. A flush alone does not confirm a committed checkpoint or delivery to a client.
See [Replication Batches](/architecture/powersync-service#replication-batches) for how the Service groups these writes. For sustained or rising lag, see [Replication Lag](/maintenance-ops/replication-lag).
Postgres bucket storage uses a shorter message format without lag or these structured `flushed` properties.
### Postgres Replication Slots and WAL Budget
Postgres records changes in its write-ahead log (WAL). PowerSync reads these changes through a replication slot with the same name as its replication stream. See [Postgres Maintenance](/configuration/source-db/postgres-maintenance).
| Message | Meaning |
| ------------------------------------------------------------------------------------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `Created replication slot ` | The Service created a slot and starts a new snapshot. |
| `Cleaning up Postgres replication slot: ...` | The Service is removing a slot it no longer needs. |
| `WAL budget: remaining of limit (% remaining). ...` | Remaining WAL retention budget during a snapshot. Low budget triggers a warning that the slot may be invalidated before the snapshot completes. See [WAL Sizing Guidance](/maintenance-ops/production-readiness-guide#managing-and-monitoring-replication-lag). |
| `WAL budget: no limit configured` | WAL retention is unlimited, or the Service could not determine the limit. If you expect a limit, check nearby warnings. |
| `[PSYNC_S1146] Replication slot was invalidated ...` / `... disappeared during snapshot.` | The slot is unusable or missing. Recovery can require action, especially during a snapshot. See [Recovering from an Invalidated Slot](/configuration/source-db/postgres-maintenance#recovering-from-an-invalidated-slot). |
### Replication Errors and Warnings
`Replication error` or `Replication failed.` means an attempt failed. The Service retries after a delay. If failures continue, read the error and any `cause` entry to identify the problem. Look up `PSYNC_` codes in [Error Codes](/debugging/error-codes).
`Fatal replication error` means the replication loop failed and the process will exit. Check the accompanying error, such as a failure to load or validate the Sync Config.
| Message | Source Database | Meaning |
| ----------------------------------------------------------------------------- | --------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `[PSYNC_S1003] Replication stream is locked by another process, standing by.` | All | Another process holds the lock. This is expected briefly after restarts or during rolling deploys. If it persists, check for multiple replication processes. See [Replication Issues](/debugging/error-codes#psync_s1xxx-replication-issues). |
| `Query error, retrying` | Postgres, MySQL, SQL Server | A database query failed. The Service retries it once. |
| `KeepAlive failed, ...` | Postgres, SQL Server | A heartbeat write failed. Heartbeats help replication advance when there are no other changes. The Service tries again at the next heartbeat. |
| `[PSYNC_S1145] Row Level Security is enabled on table ...` | Postgres | The replication role may not be able to read all rows. Check its [row-level security permissions](/debugging/error-codes#psync_s11xx-postgres-replication-issues). |
### Checking What a Deployment Reprocessed
For an [incremental deployment](/sync/advanced/storage-version-4#incremental-reprocessing), search for `Incremental reprocessing sync config update:` to see what PowerSync reuses or processes:
| List | Meaning |
| ------------------------------------- | ---------------------------------------------------------------------------------------------------- |
| `Reused definitions` | The Service keeps the existing data for these definitions. |
| `New definitions` | The Service needs to process these definitions. `tables` lists their source table names or patterns. |
| `Definitions to drop after switching` | The Service schedules these definitions for removal after the new Sync Config becomes active. |
Definitions are internal processing units. One Sync Stream can have several, with names such as `user_tasks|0`. `none` means the list is empty.
Follow [snapshot progress messages](#snapshot-progress) for the tables under `New definitions`.
# Troubleshooting
Source: https://docs.powersync.com/debugging/troubleshooting
Diagnose and fix common PowerSync issues using logs, diagnostics, and troubleshooting tools.
## Common Issues
**Tip**: Asking the AI bot on this page, or on the [#gpt-help](https://discord.com/channels/1138230179878154300/1304118313093173329) channel on our [Discord server](https://discord.com/invite/powersync), is a good way to troubleshoot common issues.
### `SqliteException: Could not load extension` or similar
This client-side error or similar typically occurs when PowerSync is used in conjunction with either another SQLite library or the standard system SQLite library. PowerSync is generally not compatible with multiple SQLite sources. If another SQLite library exists in your project dependencies, remove it if it is not required. In some cases, there might be other workarounds. For example, in Flutter projects, we've seen this issue with `sqflite 2.2.6`, but `sqflite 2.3.3+1` does not throw the same exception.
### `PSYNC_S2305`: Too many buckets / Parameter query results
PowerSync groups the data each user syncs into internal partitions called [buckets](/architecture/powersync-service#bucket-system). Each user has a limit on how many they can sync. When a user exceeds it, their sync fails with a `PSYNC_S2305` error in your Sync & API logs.
This error is caused by one of two limits, both with a [default of 1,000](/resources/performance-and-limits):
* **Buckets per connection**: the unique buckets a user syncs. The message reads `Too many buckets: N (limit of M)`.
* **Parameter query results**: the rows your parameter lookups return, counted before duplicates are removed. The message reads `Too many parameter query results (limit of N)`.
Read the message to see which limit you reached, since the fix is different for each. Your logs also show where the count comes from:
* The same `PSYNC_S2305` log entry includes a breakdown of the streams contributing the most. For a parameter-limit error, the last stream listed is the one that ran when the limit was reached. That stream is not always the cause, so check every stream in the breakdown.
* Checkpoint logs record `buckets` and `param_results` for each connection, so you can see how close a user is to either limit. Find them in your [instance logs](/maintenance-ops/monitoring-and-alerting).
To further diagnose and resolve this error, see [Reducing Bucket Count](/sync/advanced/reducing-bucket-count). For how buckets are generated and counted in the first place, see [Bucket Count](/sync/streams/bucket-count).
## Tools
Troubleshooting techniques depend on the type of issue:
1. **Connection issues between client and server:** See the tools below.
2. **Expected data not appearing on device:** See the tools below.
3. **Data lagging behind on PowerSync Service:** Data on the PowerSync Service instance cannot currently directly be inspected. This is something we are investigating.
4. **Writes to the backend source database are failing:** PowerSync is not actively involved: use normal debugging techniques (server-side logging; client and server-side error tracking).
5. **Updates are slow to sync, or queries run slow**: See [Performance](#performance)
### Sync Diagnostics Client
Access the Sync Diagnostics Client here: [https://diagnostics-app.powersync.com](https://diagnostics-app.powersync.com)
This is a standalone web app that presents data from the perspective of a specific user. It can be used to:
* See stats about the user's local database.
* Inspect tables, rows and buckets on the device.
* Query the local SQL database.
* Identify common issues, e.g. too many buckets.
See the [Readme](https://github.com/powersync-ja/powersync-js/tree/main/tools/diagnostics-app#readme) for further details.
### Instance Logs
See [Monitoring and Alerting](/maintenance-ops/monitoring-and-alerting).
### SyncStatus API
We also provide diagnostics via the `SyncStatus` APIs in the client SDKs. Examples include the connection status, last completed sync time, and local upload queue size.
If for example, a change appears to be missing on the client, you can check if the last completed sync time is greater than the time the change occurred.
For usage details, refer to the respective [client SDK docs](/client-sdks/overview).
The JavaScript SDKs ([React Native](/client-sdks/reference/react-native-and-expo), [web](/client-sdks/reference/javascript-web)) also log the contents of bucket changes to `console.debug` if verbose logging is enabled. This should log which `PUT`/`PATCH`/`DELETE` operations have been applied from the server.
### Inspect Local SQLite Database
Opening the SQLite file directly is useful for verifying sync state, inspecting raw table contents, and diagnosing unexpected data. See [Understanding the SQLite Database](/maintenance-ops/client-database-diagnostics) for platform-specific instructions (Android, iOS, Web), how to merge the WAL file, and how to analyze storage usage.
Our [Sync Diagnostics Client](/tools/diagnostics-client) and several of our [demo apps](/intro/examples) also contain a SQL console view to inspect the local database contents without pulling the file. Consider implementing similar functionality in your app. See a React example [here](https://github.com/powersync-ja/powersync-js/blob/main/tools/diagnostics-app/src/app/views/sql-console.tsx).
### Client-Side Logging
Our client SDKs support logging to troubleshoot issues. Here's how to enable logging in each SDK:
* **JavaScript-based SDKs** (Web, React Native, and Node.js) - Implement the `PowerSyncLogger` interface, or use `createConsoleLogger()`. For example: `const logger = createConsoleLogger({ minLevel: LogLevels.debug })`. Pass the logger to `PowerSyncDatabase` via the `logger` option. For the Web SDK, you can also enable the `debugMode` flag to log SQL queries on Chrome's Performance timeline.
* **Dart/Flutter SDK** - Logging is enabled by default and outputs logs to the console in debug mode.
* **Kotlin SDK** - Uses [Kermit Logger](https://kermit.touchlab.co/docs/). By default shows `Warnings` in release and `Verbose` in debug mode.
* **Swift SDK** - Supports configurable logging with `DefaultLogger` and custom loggers implementing `LoggerProtocol`. Supports severity levels: `.debug`, `.info`, `.warn`, and `.error`.
* **.NET SDK** - Uses .NET's `ILogger` interface. Configure with `LoggerFactory` to enable console logging and set minimum log level.
## Performance
When running into issues with data sync performance, first review our expected [Performance and Limits](/resources/performance-and-limits).
These are some common pointers when it comes to diagnosing and understanding performance issues:
1. You will notice differences in performance based on the **row size** (think 100 byte rows vs 8KB rows)
2. The **initial sync** on a client can take a while in cases where the operations history is large. See [Compacting Buckets](/maintenance-ops/compacting-buckets) to optimize sync performance.
3. You can get big performance gains by using **transactions & batching** as explained in this [blog post](https://www.powersync.com/blog/flutter-database-comparison-sqlite-async-sqflite-objectbox-isar).
Struggling with client-side performance issues? This page describes tools to diagnose performance of the PowerSync Service. Optimizing client-side query performance is described in [SQLite Performance Optimization](/client-sdks/advanced/performance-optimization).
### Diagnosing Sync Latency
If writes are slow to reach the client, there is no single trace that covers the full path. Isolate each stage of the pipeline to find the bottleneck.
The downstream pipeline (source database to client) has two stages:
1. **Source database to PowerSync Service** (replication).
2. **PowerSync Service to client** (sync session).
The upstream path (the client sending a write to your backend, your backend processing it, and your backend committing to the source database) sits outside PowerSync and is not covered by the diagnostics below. To measure that, instrument your backend API directly.
#### Measuring Downstream Latency (Source to Device)
To measure the downstream pipeline, put a timestamp in the data itself. When a row is written or updated in your source database, set a column to the current server time (e.g. `updated_at = NOW()`). On the client, compare that timestamp to the time the row arrives in the local database. The difference is the time from the source write being committed to the row being visible on the device.
This gives you a single number for the downstream pipeline but does not tell you which stage is slow. Use the per-stage diagnostics below to break it down.
#### Stage 1: Source Database to PowerSync Service
Check the **Replication Lag** chart in the **Metrics** view of the [PowerSync Dashboard](https://dashboard.powersync.com/). This shows whether replication from your source database is keeping up. Replicator logs in the **Logs** view surface any replication errors that would cause delays at this stage.
For a deeper walkthrough of what drives replication lag, how to interpret it for your specific source (Postgres, MongoDB, MySQL, SQL Server), and how to reduce it, see [Replication Lag](/maintenance-ops/replication-lag).
#### Stage 2: PowerSync Service to Client
Sync & API logs in the [PowerSync Dashboard](https://dashboard.powersync.com/) record a **Sync stream started** event when a client connects and a **Sync stream complete** event when the session ends. The complete event includes how many operations were synced, how much data was transferred, and how long the connection stayed open. See [Session Fields](/debugging/log-reference#session-fields) for field meanings.
[Custom metadata](/maintenance-ops/monitoring-and-alerting#custom-metadata-in-sync-logs) attached at `connect()` time is included in both events, so you can also filter by app version, environment, or other context you set.
#### Common Causes of Latency
* **Large initial sync**: if your Sync Streams/Rules result in a large dataset, the first sync after connecting will be slow. Inspect bucket sizes and sync state with the [Sync Diagnostics Client](/tools/diagnostics-client).
* **Upload queue blocking downloads**: by default, uploads are processed before downloads, so a backlogged upload queue delays receiving new data. Buckets and streams at [priority 0](/sync/advanced/prioritized-sync) are not blocked by uploads, but come with the trade-off of potential sync inconsistencies.
* **Replication lag on the source database**: high write volume, long-running transactions, bulk updates, or backfills can cause replication to fall behind faster than the service can drain it. See [Replication Lag](/maintenance-ops/replication-lag) for source-specific causes and fixes.
* **Too many buckets per user**: incremental sync overhead scales roughly linearly with the number of buckets per user. See [Too Many Buckets](#psync_s2305-too-many-buckets-/-parameter-query-results) above.
# Custom Conflict Resolution
Source: https://docs.powersync.com/handling-writes/custom-conflict-resolution
Implement custom conflict resolution in PowerSync for concurrent updates from multiple offline clients.
See [Handling Update Conflicts](/handling-writes/handling-update-conflicts) for basic guidance on how to handle conflicts. For most apps, the approaches documented there are fine. But some scenarios demand more complex conflict resolution strategies.
## When You Might Need Custom Conflict Resolution
**Retail inventory**: Two clerks ring up the same item while offline. You need to subtract both quantities, not replace one count with the other.
**Healthcare records**: A doctor updates diagnosis while a nurse updates vitals on the same patient record. Both changes matter, you can't lose either.
**Order workflows**: Once an order ships, it should lock. Status must progress logically (pending → processing → shipped), not jump around randomly.
**Collaborative documents**: Multiple people edit different paragraphs simultaneously. Automatic merging prevents losing anyone's work.
## How Data Flows Through PowerSync
Understanding the data flow helps you decide where to implement conflict resolution.
### Client to Backend
When a user updates data in your app:
1. **Client writes to local SQLite** - Changes happen instantly, even offline
2. **PowerSync queues the operation** - Stored in the upload queue
3. **Client sends operation(s) to your backend** - Your `uploadData` function processes it
4. **Backend writes to source database** - Postgres, MySQL, MongoDB etc.
### Backend to Client
When data changes on the server:
1. **Source database updates** - Direct writes or changes from other clients
2. **PowerSync Service detects changes** - Through replication stream
3. **Clients download updates** - Based on their Sync Streams (or legacy Sync Rules)
4. **Local SQLite updates** - Changes merge into the client's database
**Conflicts arise when**: Multiple clients modify the same row (or fields) before syncing, or when a client's changes conflict with server-side rules.
***
## Understanding Operations & `CrudEntry`
PowerSync tracks three operation types:
* **PUT** - Creates new row or replaces entire row (includes all non-null columns)
* **PATCH** - Updates specific fields only (includes ID + changed columns)
* **DELETE** - Removes row (includes only ID)
### `CrudEntry` Structure
When your `uploadData` receives transactions, each one has this structure:
```typescript theme={null}
interface CrudEntry {
clientId: number; // Auto-incrementing client ID
id: string; // ID of the changed row
op: UpdateType; // 'PUT' | 'PATCH' | 'DELETE'
table: string; // Table name
opData?: Record; // Changed column values (optional)
transactionId?: number; // Groups ops from same transaction
metadata?: string; // Custom metadata (trackMetadata)
trackPrevious?: Record; // Previous values (trackPrevious)
}
```
### What Your Backend Receives
**Client-side connector sends:**
```javascript theme={null}
// uploadData in your client connector
async uploadData(database) {
const transaction = await database.getNextCrudTransaction();
if (!transaction) return;
// Send to your backend API
await fetch('https://yourapi.com/data', {
method: 'POST',
body: JSON.stringify({
batch: transaction.crud // Array of CrudEntry objects
})
});
await transaction.complete();
}
```
The following structure is only received by the backend if the transactions are not mutated in your client's `uploadData` function
**Backend API receives:**
```json theme={null}
{
"batch": [
{
"op": "PATCH",
"table": "todos",
"id": "44f21466-d031-11f0-94bd-62f5a66ac26c",
"opData": {
"completed": 1,
"completed_at": "2025-12-03T10:20:04.658Z",
"completed_by": "c7b8cc68-41dd-4643-b559-66664ab6c7c5"
}
}
]
}
```
Operations are **idempotent** - your backend may receive the same operation multiple times. Use `clientId` and the operation's ID to detect and skip duplicates.
***
## Implementation Examples
The following examples demonstrate the core logic and patterns for implementing conflict resolution strategies. All client-side code is written for React/Web applications, backend examples use Node.js, and database queries target Postgres. While these examples should work as-is, they're intended as reference implementations, focus on understanding the underlying patterns and adapt them to your specific stack and requirements.
***
## Strategy 1: Timestamp-Based Detection
The idea is simple: add a `modified_at` timestamp to each row. When a client updates a row, compare their timestamp to the one in the database. If theirs is older, someone else changed the row while they were offline, so you treat it as a conflict.
This is great for quick staleness checks. You are not merging changes, just stopping outdated writes, similar to noticing a Google Doc changed while you were editing a local copy.
The only real catch is **clock drift**. If server and client clocks are out of sync, you can get false conflicts. And if clients generate timestamps themselves, make sure they all use the same timezone.
### Database Schema
**Source database (Postgres):**
```sql theme={null}
CREATE TABLE tasks (
id UUID PRIMARY KEY,
title TEXT,
status TEXT,
modified_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
);
-- Auto-update timestamp on every change
CREATE OR REPLACE FUNCTION update_modified_at()
RETURNS TRIGGER AS $$
BEGIN
NEW.modified_at = NOW();
RETURN NEW;
END;
$$ LANGUAGE plpgsql;
CREATE TRIGGER tasks_modified_at
BEFORE UPDATE ON tasks
FOR EACH ROW
EXECUTE FUNCTION update_modified_at();
```
### Backend Conflict Detection
**Backend API (Node.js):**
```javascript theme={null}
async function handleUpdate(operation, userId) {
const { id, opData } = operation;
const clientModifiedAt = opData.modified_at;
// Get current server state
const result = await db.query(
'SELECT * FROM tasks WHERE id = $1',
[id]
);
if (!result.rows[0]) {
// Row was deleted by another client
console.log(`Conflict: Row ${id} deleted`);
return { conflict: 'row_deleted' };
}
const serverModifiedAt = result.rows[0].modified_at;
// Client's version is older than server's
if (new Date(clientModifiedAt) < new Date(serverModifiedAt)) {
console.log(`Conflict: Stale update for ${id}`);
return {
conflict: 'stale_update',
serverVersion: result.rows[0],
clientVersion: opData
};
}
// No conflict - apply update
await db.query(
'UPDATE tasks SET title = $1, status = $2 WHERE id = $3',
[opData.title, opData.status, id]
);
return { success: true };
}
```
Timestamps can be unreliable if servers have **clock skew**. Additionally, if clients are writing timestamps (rather than letting the database generate them), ensure all clients use the same timezone/localization as the server. For critical data, use sequence numbers instead.
***
## Strategy 2: Sequence Number Versioning
Instead of timestamps, you can use a `version` number that increments on every change. It works like a counter on the row. Each time someone updates it, the version increases by one. When a client sends an update, they include the version they last saw. If it doesn’t match the current version in the database, another update happened and you reject the write.
This avoids **clock drift** entirely because the database manages the counter, so clients can’t get out of sync.
The tradeoff is that it’s all or nothing. You can’t merge simultaneous edits to different fields. You only know that the row changed, so the update is rejected. Use this when you want strong conflict detection and are fine asking users to refresh and redo their edits rather than risking corrupted data.
### Database Schema
**Source database (Postgres):**
```sql theme={null}
CREATE TABLE documents (
id UUID PRIMARY KEY,
content TEXT,
version BIGSERIAL NOT NULL
);
```
### Backend Conflict Detection
**Backend API (Node.js):**
```javascript theme={null}
async function handleUpdateWithVersion(operation) {
const { id, opData } = operation;
const clientVersion = opData.version;
const result = await db.query(
'SELECT version FROM documents WHERE id = $1',
[id]
);
if (!result.rows[0]) {
return { conflict: 'row_deleted' };
}
const serverVersion = result.rows[0].version;
// Client's version doesn't match server
if (clientVersion !== serverVersion) {
return {
conflict: 'version_mismatch',
expected: serverVersion,
received: clientVersion
};
}
// Update and increment version atomically
await db.query(
'UPDATE documents SET content = $1, version = version + 1 WHERE id = $2',
[opData.content, id]
);
return { success: true };
}
```
***
## Strategy 3: Field-Level Last Write Wins
Here things get more fine-grained. Instead of tracking changes for the whole row, you track them per field. If one user updates the title and another updates the status, both changes can succeed because they touched different fields.
You store a timestamp for each field you care about. When an update comes in, you compare the client’s timestamp for each field to what’s in the database and only apply the fields that are newer. This allows concurrent edits to coexist as long as they are not modifying the same field.
The downside is extra complexity. You end up with more timestamp columns, and your backend has to compare fields one by one. But for apps like task managers or form builders, where different parts of a record are often edited independently, this avoids a lot of unnecessary conflicts.
### Database Schema
**Source database (Postgres):**
```sql theme={null}
CREATE TABLE tasks (
id UUID PRIMARY KEY,
title TEXT,
title_modified_at TIMESTAMPTZ,
status TEXT,
status_modified_at TIMESTAMPTZ,
assignee TEXT,
assignee_modified_at TIMESTAMPTZ
);
```
### Client Schema with Metadata
**Client schema:**
```typescript theme={null}
const tasks = new Table(
{
title: column.text,
status: column.text,
assignee: column.text,
// Store per-field timestamps in metadata
},
{
trackMetadata: true // Enables _metadata column
}
);
```
### Client Updates with Timestamps
**Client code:**
```typescript theme={null}
await powerSync.execute(
'UPDATE tasks SET title = ?, _metadata = ? WHERE id = ?',
[
'Updated title',
JSON.stringify({
title_modified_at: new Date().toISOString(),
status_modified_at: existingTask.status_modified_at // Keep existing
}),
taskId
]
);
```
### Backend Field-Level Resolution
**Backend API (Node.js):**
```javascript theme={null}
async function fieldLevelLWW(operation) {
const { id, opData, metadata } = operation;
const timestamps = metadata ? JSON.parse(metadata) : {};
// Get current field timestamps from database
const result = await db.query(
'SELECT title_modified_at, status_modified_at, assignee_modified_at FROM tasks WHERE id = $1',
[id]
);
if (!result.rows[0]) {
return { conflict: 'row_deleted' };
}
const currentTimestamps = result.rows[0];
const updates = [];
const values = [];
let paramCount = 1;
// Check each field that was updated
for (const [field, value] of Object.entries(opData)) {
if (field === 'id') continue;
const clientTimestamp = timestamps[`${field}_modified_at`];
const serverTimestamp = currentTimestamps[`${field}_modified_at`];
// Only update if client's version is newer (or server has no timestamp)
if (!serverTimestamp ||
(clientTimestamp && new Date(clientTimestamp) > new Date(serverTimestamp))) {
updates.push(`${field} = $${paramCount}`);
updates.push(`${field}_modified_at = $${paramCount + 1}`);
values.push(value, clientTimestamp);
paramCount += 2;
}
}
if (updates.length > 0) {
values.push(id);
await db.query(
`UPDATE tasks SET ${updates.join(', ')} WHERE id = $${paramCount}`,
values
);
}
return { success: true };
}
```
***
## Strategy 4: Business Rule Validation
Sometimes conflicts aren’t about timing at all, they’re about your business rules. Maybe an order that has shipped can’t be edited, or a status can’t jump from `pending` to `completed` without hitting `processing` or **prices** can only change with manager approval.
This approach isn’t about catching concurrent edits. It’s about enforcing valid state transitions. You look at the current state in the database, compare it to what the client wants, and decide whether that move is allowed.
This is where your domain rules live. The logic becomes the gatekeeper that blocks changes that don’t make sense. You can also layer it with other methods: check timestamps first, then validate your business rules, and only then apply the update.
### Backend with Business Rules
**Backend API (Node.js):**
```javascript theme={null}
async function validateOrderUpdate(operation) {
const { id, opData } = operation;
const result = await db.query(
'SELECT * FROM orders WHERE id = $1',
[id]
);
if (!result.rows[0]) {
return { conflict: 'row_deleted' };
}
const serverOrder = result.rows[0];
// Rule 1: Shipped orders are immutable
if (serverOrder.status === 'shipped' || serverOrder.status === 'completed') {
return {
conflict: 'order_locked',
message: 'Cannot modify shipped or completed orders'
};
}
// Rule 2: Validate status transitions
const validTransitions = {
'pending': ['processing', 'cancelled'],
'processing': ['shipped', 'cancelled'],
'shipped': ['completed'],
'completed': [],
'cancelled': []
};
if (opData.status &&
!validTransitions[serverOrder.status]?.includes(opData.status)) {
return {
conflict: 'invalid_transition',
message: `Cannot change status from ${serverOrder.status} to ${opData.status}`
};
}
// Rule 3: Price changes need approval flag
if (opData.price !== undefined &&
opData.price !== serverOrder.price &&
!opData.manager_approved) {
return {
conflict: 'approval_required',
message: 'Price changes require manager approval'
};
}
// Rule 4: Stock level must be positive
if (opData.quantity !== undefined && opData.quantity < 0) {
return {
conflict: 'invalid_quantity',
message: 'Quantity cannot be negative'
};
}
// All validations passed
const updateFields = [];
const updateValues = [];
let paramCount = 1;
for (const [field, value] of Object.entries(opData)) {
if (field === 'id') continue;
updateFields.push(`${field} = $${paramCount}`);
updateValues.push(value);
paramCount++;
}
updateValues.push(id);
await db.query(
`UPDATE orders SET ${updateFields.join(', ')} WHERE id = $${paramCount}`,
updateValues
);
return { success: true };
}
```
***
## Strategy 5: Server-Side Conflict Recording
Sometimes you can’t automatically fix a conflict. Both versions might be valid, and you need a human to choose. In those cases you record the conflict instead of picking a winner. You save both versions in a write\_conflicts table and sync that back to the client so the user can decide.
The flow is simple: detect the conflict, store the client and server versions, surface it in the UI, and let the user choose or merge. After they resolve it, you mark the conflict as handled.
This is the safest option for high-stakes data where losing either version isn’t acceptable, like medical records, legal documents, or financial entries. The tradeoff is extra UI work and shifting the final decision to the user.
### Step 1: Create Conflicts Table
**Source database (Postgres):**
```sql theme={null}
CREATE TABLE write_conflicts (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
table_name TEXT NOT NULL,
row_id UUID NOT NULL,
conflict_type TEXT NOT NULL,
client_data JSONB NOT NULL,
server_data JSONB NOT NULL,
resolved BOOLEAN DEFAULT FALSE,
user_id UUID NOT NULL,
created_at TIMESTAMPTZ DEFAULT NOW()
);
```
### Step 2: Sync Conflicts to Clients
**Sync Streams / Sync Rules:**
```yaml theme={null}
config:
edition: 3
streams:
user_data:
queries:
- SELECT * FROM tasks WHERE user_id = auth.user_id()
- SELECT * FROM write_conflicts WHERE user_id = auth.user_id() AND NOT resolved
```
```yaml theme={null}
bucket_definitions:
user_data:
parameters:
- SELECT request.user_id() as user_id
data:
- SELECT * FROM tasks WHERE user_id = bucket.user_id
- SELECT * FROM write_conflicts WHERE user_id = bucket.user_id AND resolved = FALSE
```
### Step 3: Record Conflicts in Backend
**Backend API (Node.js):**
```javascript theme={null}
async function handleUpdateWithConflictRecording(operation, userId) {
const { id, opData } = operation;
const result = await db.query(
'SELECT * FROM tasks WHERE id = $1',
[id]
);
if (!result.rows[0]) {
return { conflict: 'row_deleted' };
}
const serverData = result.rows[0];
const clientModifiedAt = opData.modified_at;
const serverModifiedAt = serverData.modified_at;
// Detect conflict
if (new Date(clientModifiedAt) < new Date(serverModifiedAt)) {
// Record for manual resolution
await db.query(
`INSERT INTO write_conflicts
(table_name, row_id, conflict_type, client_data, server_data, user_id)
VALUES ($1, $2, $3, $4, $5, $6)`,
[
'tasks',
id,
'update_conflict',
JSON.stringify(opData),
JSON.stringify(serverData),
userId
]
);
// Don't apply the update - let user resolve it
return { conflict: 'recorded' };
}
// No conflict - apply update
await db.query(
'UPDATE tasks SET title = $1, status = $2 WHERE id = $3',
[opData.title, opData.status, id]
);
return { success: true };
}
```
### Step 4: Build Resolution UI
**Client UI (React):**
```typescript theme={null}
import { useQuery } from '@powersync/react';
import { powerSync } from './db'
function ConflictResolver() {
const { data: conflicts } = useQuery(
'SELECT * FROM write_conflicts WHERE resolved = FALSE'
);
const resolveConflict = async (
conflictId: string,
useClientVersion: boolean
) => {
const conflict = conflicts.find(c => c.id === conflictId);
const clientData = JSON.parse(conflict.client_data);
if (useClientVersion) {
// Reapply client's changes
const fields = Object.keys(clientData).filter(k => k !== 'id');
const placeholders = fields.map(() => '?').join(', ');
const updates = fields.map(f => `${f} = ?`).join(', ');
await powerSync.execute(
`UPDATE ${conflict.table_name} SET ${updates} WHERE id = ?`,
[...fields.map(f => clientData[f]), conflict.row_id]
);
}
// If using server version, it's already applied
// Mark as resolved
await powerSync.execute(
'UPDATE write_conflicts SET resolved = TRUE WHERE id = ?',
[conflictId]
);
};
if (!conflicts || conflicts.length === 0) {
return null;
}
return (
⚠️ {conflicts.length} Conflict(s) Need Your Attention
{conflicts.map(conflict => {
const clientData = JSON.parse(conflict.client_data);
const serverData = JSON.parse(conflict.server_data);
return (
Conflict in {conflict.table_name}
from {new Date(conflict.created_at).toLocaleString()}
Your Changes:
{Object.entries(clientData).map(([key, value]) => (
{key}: {JSON.stringify(value)}
))}
resolveConflict(conflict.id, true)}>
Keep My Version
Server Version:
{Object.entries(serverData).map(([key, value]) => (
{key}: {JSON.stringify(value)}
))}
resolveConflict(conflict.id, false)}>
Keep Server Version
);
})}
);
}
```
***
## Strategy 6: Change-Level Status Tracking
This approach works differently. Instead of merging everything in one atomic update, you log each field change as its own row in a separate table. If a user edits the title of a task, you still apply an optimistic update to the main table, but you also write a row to a `field_changes` table that records who changed what and to which value.
Your backend then processes these changes asynchronously. Each one gets a status like `pending`, `applied`, or `failed`. If a change fails validation, you mark it as `failed` and surface the error in the UI. The user can see exactly which fields succeeded and which didn’t, and retry the failed ones without resubmitting everything.
This gives you excellent visibility. You get a clear history of every change, who made it, and when it happened. The cost is extra writes, since every field update creates an additional log entry. But for compliance-heavy systems or any app that needs detailed auditing, the tradeoff could be worth it.
The implementation below shows the full version with complete status tracking. If you don't need all that complexity, see the simpler variations at the end of this section.
### Step 1: Create Change Log Table
**Source database (Postgres):**
```sql theme={null}
CREATE TABLE field_changes (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
table_name TEXT NOT NULL,
row_id UUID NOT NULL,
field_name TEXT NOT NULL,
new_value TEXT,
status TEXT DEFAULT 'pending', -- 'pending', 'applied', 'failed'
error_message TEXT,
user_id UUID NOT NULL,
created_at TIMESTAMPTZ DEFAULT NOW()
);
```
### Step 2: Client Writes to Both Tables
**Client code:**
```typescript theme={null}
async function updateTaskField(
taskId: string,
field: string,
newValue: any,
powerSync: PowerSyncDatabase
) {
await powerSync.writeTransaction(async (tx) => {
// Optimistic update to main table
await tx.execute(
`UPDATE tasks SET ${field} = ? WHERE id = ?`,
[newValue, taskId]
);
// Log the change for server tracking
await tx.execute(
`INSERT INTO field_changes
(table_name, row_id, field_name, new_value, user_id)
VALUES (?, ?, ?, ?, ?)`,
['tasks', taskId, field, String(newValue), getCurrentUserId()]
);
});
}
```
### Step 3: Backend Processes Changes
**Backend API (Node.js):**
```javascript theme={null}
async function processFieldChanges() {
const result = await db.query(
`SELECT * FROM field_changes
WHERE status = 'pending'
ORDER BY created_at ASC
LIMIT 100`
);
for (const change of result.rows) {
try {
// Validate the change
const isValid = await validateFieldChange(change);
if (!isValid.valid) {
await db.query(
`UPDATE field_changes
SET status = 'failed', error_message = $1
WHERE id = $2`,
[isValid.reason, change.id]
);
continue;
}
// Apply to main table
await db.query(
`UPDATE ${change.table_name}
SET ${change.field_name} = $1
WHERE id = $2`,
[change.new_value, change.row_id]
);
// Mark as applied
await db.query(
`UPDATE field_changes SET status = 'applied' WHERE id = $1`,
[change.id]
);
} catch (error) {
await db.query(
`UPDATE field_changes
SET status = 'failed', error_message = $1
WHERE id = $2`,
[error.message, change.id]
);
}
}
}
async function validateFieldChange(change) {
// Example validation
if (change.field_name === 'price' && parseFloat(change.new_value) < 0) {
return { valid: false, reason: 'Price cannot be negative' };
}
return { valid: true };
}
```
### Step 4: Display Change Status
**Client UI (React):**
```typescript theme={null}
function TaskEditor({ taskId }: { taskId: string }) {
const { data: pendingChanges } = useQuery(
`SELECT * FROM field_changes
WHERE row_id = ?
AND table_name = 'tasks'
AND status IN ('pending', 'failed')
ORDER BY created_at DESC`,
[taskId]
);
const retryChange = async (changeId: string) => {
await powerSync.execute(
'UPDATE field_changes SET status = ? WHERE id = ?',
['pending', changeId]
);
};
return (
{/* Your task editing form */}
{pendingChanges && pendingChanges.length > 0 && (
{pendingChanges.map(change => (
{change.status === 'pending' && (
⏳ Syncing {change.field_name}...
)}
{change.status === 'failed' && (
❌ Failed to update {change.field_name}: {change.error_message}
retryChange(change.id)}>
Retry
)}
))}
)}
⏳ {pending[0].count} change{pending[0].count > 1 ? 's' : ''} syncing...
);
}
```
**When to use:** Showing "syncing..." indicators, temporary status tracking without long-term storage overhead, cases where you want automatic cleanup after sync.
**Tradeoff:** Can't show detailed server-side error messages (unless the server writes to a separate errors table that *is* in Sync Rules). No long-term history on the client.
## Strategy 7: Cumulative Operations (Inventory)
For scenarios like inventory management, simply replacing values causes data loss. When two clerks simultaneously sell the same item while offline, both sales must be honored. The solution is to treat certain fields as **deltas** rather than absolute values, you subtract incoming quantities from the current stock rather than replacing the count.
This requires your backend to recognize which operations should be cumulative. For inventory quantity changes, you apply the delta (e.g., `-3 units`) to the current value rather than setting it directly. This ensures all concurrent sales are properly recorded without overwriting each other.
### Database Schema
**Source database (Postgres):**
```sql theme={null}
CREATE TABLE inventory (
id UUID PRIMARY KEY,
product_id UUID NOT NULL,
quantity INTEGER NOT NULL DEFAULT 0,
last_updated TIMESTAMPTZ DEFAULT NOW()
);
-- Prevent negative inventory
ALTER TABLE inventory ADD CONSTRAINT positive_quantity CHECK (quantity >= 0);
```
### Backend: Delta Detection and Application
The key is detecting when an operation should be treated as a delta versus an absolute value. You can identify this through table/field combinations, metadata flags, or operation patterns.
**Backend API (Node.js):**
```javascript theme={null}
async function handleInventoryOperation(db, operation) {
const { table, id, op, opData } = operation;
// Identify cumulative fields for specific tables
if (table === 'inventory' && 'quantity' in opData) {
return await applyInventoryDelta(db, operation);
}
// Default handling for other fields/tables
return await handleGenericOperation(db, operation);
}
async function applyInventoryDelta(db, operation) {
const { id, opData } = operation;
const quantityChange = opData.quantity; // This is the delta, not absolute value
// Get current inventory
const result = await db.query(
'SELECT quantity FROM inventory WHERE id = $1',
[id]
);
if (!result.rows[0]) {
return {
conflict: 'inventory_not_found',
message: `Inventory ${id} does not exist`
};
}
const currentQuantity = result.rows[0].quantity;
const newQuantity = currentQuantity + quantityChange;
// Validate: prevent negative inventory
if (newQuantity < 0) {
console.warn(`Insufficient stock: ${id} has ${currentQuantity}, attempted change: ${quantityChange}`);
return {
conflict: 'insufficient_stock',
message: `Cannot reduce inventory by ${Math.abs(quantityChange)}. Only ${currentQuantity} available.`,
currentQuantity
};
}
// Apply the delta atomically
await db.query(
`UPDATE inventory
SET quantity = quantity + $1,
last_updated = NOW()
WHERE id = $2`,
[quantityChange, id]
);
return {
success: true,
newQuantity,
previousQuantity: currentQuantity
};
}
```
### Client Implementation
On the client side, you need to ensure updates are sent as deltas, not absolute values. When a sale occurs, send the change amount:
**Client code:**
```typescript theme={null}
// When selling 3 units
await powerSync.execute(
'UPDATE inventory SET quantity = quantity - ? WHERE id = ?',
[3, inventoryId] // Send -3 as the delta
);
```
The backend receives this as a PATCH operation where `opData.quantity = -3`, which it then adds to the current quantity rather than replacing it.
### Alternative Approaches
**1. Metadata Flags**: Include operation type in metadata to signal delta operations:
```typescript theme={null}
await powerSync.execute(
'UPDATE inventory SET quantity = ?, _metadata = ? WHERE id = ?',
[
-3,
JSON.stringify({ operation_type: 'delta' }),
inventoryId
]
);
```
Backend checks metadata and applies accordingly.
**2. Separate Transactions Table**: Track each quantity change as its own row, then aggregate them. This provides full audit history but requires syncing an additional table.
**3. Operation-Based Detection**: Infer cumulative operations from the pattern. Negative values likely indicate sales (deltas), while large positive values might be absolute restocks requiring different handling.
***
## Using Custom Metadata
Track additional context about operations using the `_metadata` column.
### Enable in Schema
**Client schema:**
```typescript theme={null}
const tasks = new Table(
{
title: column.text,
status: column.text,
},
{
trackMetadata: true // Enables _metadata column
}
);
```
### Write Metadata
**Client code:**
```typescript theme={null}
await powerSync.execute(
'UPDATE tasks SET title = ?, _metadata = ? WHERE id = ?',
[
'New title',
JSON.stringify({
source: 'mobile_app',
device: 'iPhone 12',
priority: 'high',
reason: 'customer_request'
}),
taskId
]
);
```
### Access in Backend
**Backend API (Node.js):**
```javascript theme={null}
async function processOperation(operation) {
const metadata = operation.metadata ? JSON.parse(operation.metadata) : {};
// Route high-priority operations differently
if (metadata.priority === 'high') {
await processHighPriority(operation);
return;
}
// Track which device made the change
console.log(`Change from: ${metadata.device || 'unknown'}`);
// Custom conflict resolution based on metadata
if (metadata.reason === 'customer_request') {
// Customer requests might override other updates
await forceApplyOperation(operation);
} else {
await standardProcessing(operation);
}
}
```
**Common use cases:**
* Track which device/app version made the change
* Flag operations requiring special handling
* Store user context (role, department)
* Implement source-based conflict resolution (mobile trumps web)
* Pass approval flags or business context
***
## Complete Backend Example
Here's how to tie it all together in a Node.js backend with Postgres.
**Backend API (Node.js + Express):**
```javascript theme={null}
import express from 'express';
import { Pool } from 'pg';
const app = express();
const pool = new Pool({
connectionString: process.env.DATABASE_URL
});
app.post('/api/data', async (req, res) => {
const { batch } = req.body;
const userId = req.user.id; // From auth middleware
const db = await pool.connect();
try {
await db.query('BEGIN');
for (const operation of batch) {
// Choose strategy based on table
if (operation.table === 'orders') {
await handleOrderOperation(db, operation, userId);
} else if (operation.table === 'tasks') {
await handleTaskOperation(db, operation, userId);
} else {
// Default handling
await handleGenericOperation(db, operation);
}
}
await db.query('COMMIT');
res.json({ success: true });
} catch (error) {
await db.query('ROLLBACK');
console.error('Operation failed:', error);
res.status(500).json({ error: error.message });
} finally {
db.release();
}
});
async function handleOrderOperation(db, op, userId) {
if (op.op === 'PUT') {
// Use business rule validation (Strategy 4)
const result = await validateOrderUpdate(db, op);
if (result.conflict) {
throw new Error(result.message);
}
} else if (op.op === 'PATCH') {
await handleOrderPatch(db, op, userId);
} else if (op.op === 'DELETE') {
await handleOrderDelete(db, op);
}
}
async function handleTaskOperation(db, op, userId) {
if (op.op === 'PUT' || op.op === 'PATCH') {
// Use timestamp detection with conflict recording (Strategy 1 + 5)
const result = await handleUpdateWithConflictRecording(db, op, userId);
if (result.conflict && result.conflict !== 'recorded') {
console.warn('Conflict detected:', result);
}
} else if (op.op === 'DELETE') {
await db.query('DELETE FROM tasks WHERE id = $1', [op.id]);
}
}
async function handleGenericOperation(db, op) {
// Default last-write-wins
if (op.op === 'PUT') {
const fields = Object.keys(op.opData);
const values = Object.values(op.opData);
const placeholders = fields.map((_, i) => `$${i + 1}`).join(', ');
const updates = fields.map((f, i) => `${f} = $${i + 1}`).join(', ');
await db.query(
`INSERT INTO ${op.table} (id, ${fields.join(', ')})
VALUES ($${fields.length + 1}, ${placeholders})
ON CONFLICT (id) DO UPDATE SET ${updates}`,
[...values, op.id]
);
} else if (op.op === 'PATCH') {
const fields = Object.keys(op.opData);
const values = Object.values(op.opData);
const updates = fields.map((f, i) => `${f} = $${i + 1}`).join(', ');
await db.query(
`UPDATE ${op.table} SET ${updates} WHERE id = $${fields.length + 1}`,
[...values, op.id]
);
} else if (op.op === 'DELETE') {
await db.query(`DELETE FROM ${op.table} WHERE id = $1`, [op.id]);
}
}
app.listen(3000, () => {
console.log('Backend listening on port 3000');
});
```
***
## Best Practices
**Design for idempotency:**
Operations arrive multiple times. Check for existing records before inserting, use upserts, or track operation IDs to skip duplicates.
**Test offline scenarios:**
Simulate two clients going offline, making conflicting changes, then syncing. Does your resolution strategy behave as expected?
**Provide clear UI feedback:**
Show sync status prominently. Users should know when their changes are pending, synced, or conflicted.
**Consider partial failures:**
If batch processing fails midway, how do you recover? Use database transactions and mark progress carefully.
**Log conflicts for analysis:**
Track how often conflicts occur and why. This data helps you improve UX or adjust resolution strategies.
**Leverage CRDTs for collaborative docs:**
For scenarios with real-time collaboration, consider CRDTs to automatically handle concurrent edits. For information on CRDTs, see [our separate guide](/client-sdks/advanced/crdts).
**Collaborative editing without using CRDTs:**
You can use PowerSync for collaborative text editing without the complexity of CRDTs. See Matthew Weidner's blog post on [collaborative text editing using PowerSync](https://www.powersync.com/blog/collaborative-text-editing-over-powersync).
# Data Pipelines
Source: https://docs.powersync.com/handling-writes/custom-write-checkpoints
Use custom checkpoints to keep client data stable when your backend applies uploads to the source database asynchronously.
**Availability**: Custom checkpoints are available for customers on our [Team and Enterprise](https://www.powersync.com/pricing) plans.
PowerSync uses write checkpoints to keep client data [consistent](/architecture/consistency). After the client uploads its local writes, it obtains a checkpoint that marks the source database position after those writes. The client applies downloaded data only once the sync checkpoint includes that write checkpoint. This is why your [write endpoint must be synchronous](/handling-writes/writing-client-changes): the default checkpoint marks the source database position when `uploadData()` returns, so the uploaded changes must already be in the source database at that moment.
Some backends cannot process uploads synchronously. In a chained data pipeline, uploads first go to a queue or an intermediate database and reach the source database later. With the default checkpoints, this makes client data flicker:
1. The client uploads a change. Your backend accepts it and queues it, and `uploadData()` returns.
2. The client obtains a write checkpoint. The PowerSync Service marks the current source database position, which does not include the queued change.
3. The client receives that checkpoint and applies the server state. The change is missing, so the client reverts it locally.
4. The pipeline writes the change to the source database. The Service syncs it, and the client applies it again.
Custom checkpoints solve this. Instead of the Service marking the source database position when the client asks, your backend writes a checkpoint record into a table in the source database at the end of the pipeline. The record replicates to the Service through the same replication stream as your data, so the checkpoint always follows the uploaded changes. An event definition in your Sync Config tells the Service how to read the record.
## Choosing a Flow
PowerSync supports two custom checkpoint flows. Use custom checkpoint requests for new implementations.
| | Custom checkpoint requests | Legacy Custom Write Checkpoints |
| ---------------------- | ------------------------------------------------------------------------------------------ | ------------------------------------------------------------------------- |
| Checkpoint ID | Generated by the PowerSync Client SDK | Generated by your backend |
| Client integration | Implement `postCheckpointRequest()` on your connector. The SDK calls it after each upload. | Pass the checkpoint number to `transaction.complete()` in `uploadData()`. |
| Event definition | `checkpoint_requests` | `write_checkpoints` |
| Records in the Service | Expire after a retention period | Retained |
| Status | Alpha. Requires PowerSync Service 1.24.0 or later. | Stable |
Custom checkpoint requests are part of the [Checkpoint Requests](/client-sdks/advanced/checkpoint-requests) API. See that page for the supported client SDKs. The legacy flow continues to work and is documented in [Legacy Custom Write Checkpoints](#legacy-custom-write-checkpoints).
From PowerSync Service 1.26.0, a Sync Config can define only one of the two events. If you have app versions in production that use the legacy flow, see [Migrating to Custom Checkpoint Requests](#migrating-to-custom-checkpoint-requests).
## Setting Up Custom Checkpoint Requests
With custom checkpoint requests, the PowerSync Client SDK generates an increasing checkpoint request ID and sends it to your backend after each upload. Your backend writes the ID into a checkpoints table in the source database. When the Service replicates the record, the client knows that its uploads are in the source database.
Create a table in your source database that stores the latest checkpoint request ID for each PowerSync client:
```sql theme={null}
CREATE TABLE checkpoints (
user_id TEXT NOT NULL,
client_id TEXT NOT NULL,
checkpoint BIGINT NOT NULL,
PRIMARY KEY (user_id, client_id)
);
```
* `user_id` is the authenticated user.
* `client_id` is the PowerSync client ID. Each local database has its own client ID, so one user can have many clients.
* `checkpoint` is the checkpoint request ID. IDs are 64-bit integers, so use a `BIGINT` or equivalent column.
Column names can differ. The event definition in the next steps maps your columns to these fields.
For Postgres, add the table to the PowerSync [publication](/configuration/source-db/setup):
```sql theme={null}
CREATE PUBLICATION powersync FOR TABLE lists, todos, checkpoints;
```
For other source databases, the Service replicates every table that your Sync Config references, including tables in event definitions. Complete the same [table setup](/configuration/source-db/setup) as for your other tables, such as enabling CDC for a SQL Server table.
Add a `checkpoint_requests` event definition to your Sync Config. Its payload query must return the fields `user_id`, `client_id`, and `checkpoint`:
```yaml theme={null}
config:
edition: 3
event_definitions:
checkpoint_requests:
payloads:
- SELECT user_id, client_id, checkpoint FROM checkpoints
streams:
todos:
query: SELECT * FROM todos WHERE owner_id = auth.user_id()
```
Use aliases if your column names differ, for example `SELECT owner AS user_id, device AS client_id, request_id AS checkpoint FROM checkpoints`.
Add an endpoint that receives the client ID and checkpoint request ID from the client. Take the user ID from your session or token. The endpoint must:
1. Store the greater of the submitted ID and the stored ID for that user and client.
2. Return that value. If the submitted ID was stale, the client uses the returned ID to continue counting from there.
Write the record through the same pipeline as the uploads, so that it reaches the source database after the changes it confirms. If your backend writes the checkpoint record directly while the uploads are still queued, the client sees the checkpoint before its changes and reverts them.
For Postgres, one statement handles both new and existing rows:
```sql theme={null}
INSERT INTO checkpoints (user_id, client_id, checkpoint)
VALUES ($1, $2, $3)
ON CONFLICT (user_id, client_id) DO UPDATE
SET checkpoint = GREATEST(checkpoints.checkpoint, EXCLUDED.checkpoint)
RETURNING checkpoint;
```
Return the value as a string in JSON to avoid precision loss in JavaScript clients. See [Checkpoint Request IDs](/client-sdks/advanced/checkpoint-requests#checkpoint-request-ids) for the full reconciliation rules.
Connect with checkpoint requests enabled and add `postCheckpointRequest()` to your backend connector to call your endpoint. The SDK calls this method after each upload, so `uploadData()` needs no checkpoint handling of its own. See [Prerequisites](/client-sdks/advanced/checkpoint-requests#prerequisites) and [Connector Changes](/client-sdks/advanced/checkpoint-requests#connector-changes) for how to declare the method in each SDK.
### Record Retention
The Service keeps a replicated checkpoint request for `checkpoint_request_retention_minutes` after it stores the record. The default is 60 minutes. The next compact job then removes it. Clients send their current request ID again when they reconnect, so an expired record is recreated when a client still needs it. On self-hosted instances, you can change the period with [`api.parameters.checkpoint_request_retention_minutes`](/configuration/powersync-service/self-hosted-instances#param-checkpoint-request-retention-minutes).
Retention applies only to the Service's copy. Rows in your checkpoints table are yours to keep or delete. While a row exists, return its value from your endpoint so that a reconnecting client can resume from it.
## Legacy Custom Write Checkpoints
In the legacy flow, your backend generates an increasing checkpoint number for each client, and the client passes that number to `transaction.complete()` after each upload. The Service retains these records because legacy clients wait for a specific number and do not request it again.
Use the same table and replication setup as for [custom checkpoint requests](#setting-up-custom-checkpoint-requests).
Add a `write_checkpoints` event definition to your Sync Config:
```yaml theme={null}
config:
edition: 3
event_definitions:
write_checkpoints:
payloads:
- SELECT user_id, client_id, checkpoint FROM checkpoints
streams:
todos:
query: SELECT * FROM todos WHERE owner_id = auth.user_id()
```
Add an endpoint that increments and returns the checkpoint number for the user and client. Write the record through the same pipeline as the uploads. For Postgres:
```sql theme={null}
INSERT INTO checkpoints (user_id, client_id, checkpoint)
VALUES ($1, $2, 1)
ON CONFLICT (user_id, client_id) DO UPDATE
SET checkpoint = checkpoints.checkpoint + 1
RETURNING checkpoint;
```
In `uploadData()`, request a checkpoint from your backend after uploading the transaction and pass it to `complete()`:
```typescript theme={null}
async function uploadData(database: CommonPowerSyncDatabase): Promise {
const transaction = await database.getNextCrudTransaction();
if (!transaction) {
return;
}
for (const operation of transaction.crud) {
// Upload the operation to your backend
}
const clientId = await database.getClientId();
const checkpoint = await requestWriteCheckpoint(clientId);
await transaction.complete(checkpoint);
}
async function requestWriteCheckpoint(clientId: string): Promise {
// Call your backend endpoint. It creates the checkpoint record
// and returns the new checkpoint number as a string.
}
```
## Migrating to Custom Checkpoint Requests
While you roll out an updated app version, older versions that still use the legacy flow write legacy checkpoint numbers while updated versions write checkpoint request IDs. Because a Sync Config cannot define both `write_checkpoints` and `checkpoint_requests`, support both kinds of records by defining only `checkpoint_requests` and adding an `is_legacy` field to the payload. The field is available since Service version 1.26.0.
* Set `is_legacy` to `true` for legacy records. The Service retains them.
* Omit `is_legacy`, or set it to `false`, for checkpoint request records. The Service can expire them.
Keep both record types in separate tables where possible. Separate tables make the difference in retention visible in the Sync Config:
```yaml theme={null}
event_definitions:
checkpoint_requests:
payloads:
# Legacy checkpoints must be retained
- SELECT user_id, client_id, checkpoint, true AS is_legacy FROM legacy_checkpoints
# Checkpoint requests can expire
- SELECT user_id, client_id, checkpoint FROM checkpoint_requests
```
If both flows write to one table, distinguish the rows with a column. For example, your checkpoint request endpoint can set a `checkpoint_requested_at` timestamp that the legacy endpoint leaves `NULL`:
```yaml theme={null}
event_definitions:
checkpoint_requests:
payloads:
- SELECT user_id, client_id, checkpoint, checkpoint_requested_at IS NULL AS is_legacy FROM checkpoints
```
Once no clients use the legacy flow, remove the legacy payload or the `is_legacy` field. With [storage version 4](/sync/advanced/storage-version-4#incremental-reprocessing), changing an event definition reprocesses only that event's data.
## Example Implementations
* [Swift custom checkpoint demo](https://github.com/powersync-ja/powersync-swift/tree/main/Demos/CustomCheckpointDemo): a client that uses custom checkpoint requests with the Node.js backend demo.
* [Node.js backend demo](https://github.com/powersync-ja/powersync-nodejs-backend-todolist-demo): implements both a checkpoint request endpoint and a legacy checkpoint endpoint, with Postgres, MongoDB, and MySQL persistence.
* [Self-hosted custom checkpoints demo](https://github.com/powersync-ja/self-host-demo/tree/main/demos/nodejs-custom-checkpoints): a Docker Compose setup with Postgres that uses the legacy flow.
# Handling Update Conflicts
Source: https://docs.powersync.com/handling-writes/handling-update-conflicts
Understand what happens when multiple users update the same records while offline.
By design PowerSync is unopinionated when it comes to handling conflicts on the backend. On the client, mutations are stored in an "upload queue" - an ordered log of mutations. This upload queue must be sequentially drained against backend API endpoints configured by the developer. The developer's app backend therefore dictates how mutations from clients are processed and applied against the source database. PowerSync then replicates and syncs those mutations back to the client. This is known as server-authoritative reconcilliation.
In the simplest backend implementation, the behavior of the overall system will be per-field Last-Write-Wins (LWW). However, this can be [customized](/handling-writes/custom-conflict-resolution).
The upload queue on the client stores three types of operations:
1. PUT / Create new row — contains the value for each non-null column
2. PATCH / Update existing row — contains the ID, and value of each changed column
3. DELETE / Delete existing row — contains the ID
It is [up to your app backend](/handling-writes/writing-client-changes) to implement these operations and associated conflict handling.
The operations must be idempotent — i.e. the backend may receive the same operation multiple times in some scenarios, and must handle that appropriately.
* A per-client incrementing operation ID is included with each operation that can be used to deduplicate operations, and/or the backend can implement the operations in an idempotent way (e.g. ignore DELETE on a row that is already deleted).
A conflict may arise when two clients update the same field on the same record before seeing the other client’s update, or one client deletes the record while the other updates it.
Typically, the backend should be implemented to handle writes as follows:
1. Deletes always win: If one client deletes a row, any future updates to that row are ignored. The row may be created again with the same ID.
2. For multiple concurrent updates, the last update (as received by the server) to each individual field wins.
1. If you require different behavior to "last write wins", implement [custom conflict resolution](/handling-writes/custom-conflict-resolution).
The server could implement some validations. For example, the server could have a record of orders, and once an order is marked as "completed", reject any further updates to the order.
Future versions may include support for custom operations, e.g. "increment column by 1".
### Using CRDTs to Merge Updates Automatically
CRDT data structures such as [Yjs](https://github.com/yjs/yjs) can be stored and synced using PowerSync, allowing you to build collaborative apps that merge users' updates automatically.
See the [CRDTs](/client-sdks/advanced/crdts) section for more detail.
Built-in support for CRDT operations in PowerSync may also be added in the future.
# Handling Write / Validation Errors
Source: https://docs.powersync.com/handling-writes/handling-write-validation-errors
Handle transient and permanent write errors in the PowerSync upload queue.
For validation errors or write conflicts (see the definition of this below in [Technical Details](/handling-writes/handling-write-validation-errors#additional-technical-details)), changes are automatically rolled back on the client.
Custom logic can be implemented to propagate validation failures back to clients asynchronously. For additional details on how to do that, see the section on [Custom Conflict Resolution.](/handling-writes/custom-conflict-resolution)
## Additional Technical Details
For each change (or batch of changes), some possible scenarios are:
1. Change failed, for example due to network or temporary server error. The change is kept in the queue.
2. Change acknowledged and applied on the server. The client syncs back the change, which would match what the client already had.
3. Change acknowledged but rejected (e.g. validation error). The client rolls back the change.
4. Change acknowledged and partially applied or otherwise altered. The client syncs back the state as applied on the server.
In all cases, PowerSync ensures that the client state is fully consistent with the server state, once the queue is empty.
### Backend Implementation Recommendations
The backend should respond with "success" (HTTP 2xx) even in the case of write conflicts or validation failures, unless developer intervention is desired.
Error responses should be reserved for:
1. Network errors.
2. Temporary server errors (e.g. high load, or database unavailable).
3. Unexpected bugs or schema mismatches, where the change should stay in the client-side queue.
If a bug triggers an error, it has to be fixed before the changes from the client can be processed. It is recommended to use an error reporting service on both the server and the client to be alerted of cases like this.
To propagate validation failures or write conflicts back to the client, either:
1. Include error details in the body of a success response (HTTP 2xx).
2. Write the details to a different table, asynchronously synced back to the client.
For more details on strategies, see:
Your backend application receives the write operations based on how you defined your `uploadData()` function in the `PowerSyncBackendConnector` in your client-side app. See [Client-Side Integration](/configuration/app-backend/client-side-integration) for details, including [when `uploadData()` is called](/configuration/app-backend/client-side-integration#when-uploaddata-is-called) and its retry behavior.
Since you get to define the client-side `uploadData()` function as you wish, you have full control over how to structure your backend application API to accept write operations from the client. For example, you can have:
1. A single API endpoint that accepts a batch of write operations from the client, with minimal client-side processing.
2. Separate API endpoints based on the types of write operations. In your `uploadData()`, you can call the respective endpoints as needed.
3. A combination of the above.
You can also use any API style you want — e.g. REST, GraphQL, gRPC, etc.
It's important that your API endpoint be blocking/synchronous with underlying writes to the backend source database (Postgres, MongoDB, MySQL, SQL Server, or Convex).
In other words, don't place writes into something like a queue for processing later — process them immediately. For more details, see the explainer below.
PowerSync uses a server-authoritative architecture with a checkpoint system for conflict resolution and [consistency](/architecture/consistency). The client advances to a new write checkpoint after uploads have been processed, so if the client believes that the server has written changes into your backend source database (Postgres, MongoDB, MySQL, SQL Server, or Convex), but the next checkpoint does not contain your uploaded changes, those changes will be removed from the client. This could manifest as UI glitches for your end-users, where the changes disappear from the device for a few seconds and then re-appear.
### Write Operations Recorded on the Client
The upload queue on the client stores three types of operations:
| Operation | Purpose | Contents | SQLite Statement |
| --------- | ------------------- | -------------------------------------------------------- | --------------------------------- |
| `PUT` | Create new row | Contains the value for each non-null column | Generated by `INSERT` statements. |
| `PATCH` | Update existing row | Contains the row `id`, and value of each changed column. | Generated by `UPDATE` statements. |
| `DELETE` | Delete existing row | Contains the row `id` | Generated by `DELETE` statements. |
### Recommendations
The PowerSync Client SDK does not prescribe any specific request/response format for your backend application API that accepts the write operations. You can implement it as you wish.
We do however recommend the following:
1. Use a batch endpoint to handle high volumes of write operations.
2. Use an error response (`5xx`) only when the write operations cannot be applied due to a temporary error (e.g. backend source database not available). In this scenario, the PowerSync Client SDK can retry uploading the write operation and it should succeed at a later time.
3. For validation errors or write conflicts, you should avoid returning an error response (`4xx`), since it will block the PowerSync client's upload queue. Instead, it is best to return a `2xx` response, and if needed, propagate the validation or other error message(s) back to the client, for example by:
1. Including the error details in the `2xx` response.
2. Writing the error(s) into a separate table/collection that is synced to the client, so that the client/user can handle the error(s).
For details on approaches, see:
For details on handling write conflicts, see:
### Example Backend Implementations
See our [Example Projects](/intro/examples#backend-examples) page for examples of custom backend implementations (e.g. Django, Node.js, Rails, etc.) that you can use as a guide for your implementation.
For Postgres developers, using [Supabase](/integrations/supabase/guide) is an easy alternative to a custom backend. Several of our example/demo apps demonstrate how to use [Supabase](https://supabase.com/) as the backend. These examples use the [PostgREST API](https://supabase.com/docs/guides/api) exposed by Supabase to upload write operations. Alternatively, Supabase's [Edge Functions](https://supabase.com/docs/guides/functions) can also be used.
### Convex
For Convex, your backend connector applies writes by calling Convex mutations from `uploadData()`. Most Convex apps already define mutation functions for each writable table, so you can call those same mutations rather than building a separate API.
If you use Convex Auth tokens directly for PowerSync client authentication, configure PowerSync to accept the `convex` JWT audience. For self-hosted development this is set in `client_auth.audience`. For PowerSync Cloud you can configure a custom audience in the instance settings. See [Custom Authentication](/configuration/auth/custom).
# Neon + PowerSync
Source: https://docs.powersync.com/integrations/neon
Build apps on Neon with PowerSync that use client-side SQLite for real-time updates across users and devices, instant interactions and continued functionality when offline.
This guide provides instructions for how to configure PowerSync for use with your [Neon](https://neon.tech/) project using a TypeScript demo app.
Before you proceed, this guide assumes that you have already signed up for free accounts with both Neon and PowerSync Cloud (our cloud-hosted offering). If you haven't signed up for a **PowerSync** (Cloud) account yet, [click here](https://accounts.powersync.com/portal/powersync-signup?s=docs) (and if you haven't signed up for Neon yet, [do so now](https://console.neon.tech/signup)).
For web apps, this guide assumes that you have [pnpm](https://pnpm.io/installation#using-npm) installed.
This guide takes 10-15 minutes to complete.
## Architecture
Upon successful integration of Neon + PowerSync, your system architecture will look like this: (click to enlarge image)
For more details on PowerSync's general architecture, [see here](/architecture/architecture-overview).
## Integration Guide/Tutorial Overview
We will follow these steps to get an offline-first 'Notes' demo app up and running:
* Create a Neon project with Auth and Data API
* Set up the database schema
* Configure logical replication
* Create connection to Neon
* Configure authentication
* Configure Sync Streams
Test the configuration using our provided PowerSync-Neon 'Notes' demo app.
## Configure Neon
### Create a Neon Project with Auth and Data API
1. Go to [pg.new](https://pg.new) to create a new Neon project.
2. In the Neon Console, navigate to your project and enable:
* **Neon Auth** — Go to the **Auth** page in the left sidebar and enable it
* **Data API** — Go to the **Data API** page in the left sidebar and enable it
### Set Up the Database
The demo app uses Drizzle ORM for schema management. The schema includes `notes` and `paragraphs` tables with Row Level Security (RLS) policies.
Clone the demo repository and run the migration:
```bash theme={null}
git clone https://github.com/powersync-ja/powersync-js.git
cd powersync-js
pnpm install
pnpm build:packages
cd demos/react-neon-tanstack-query-notes
```
Create a `.env` file in the project root with your database connection string:
```env theme={null}
DATABASE_URL=postgresql://user:password@your-project-id.pooler.region.neon.tech/neondb?sslmode=require
```
Find your connection string in the Neon Console → Dashboard → Connect → Connection string (select "Pooled connection") → Copy snippet.
Run the migration to create the tables and RLS policies:
```bash theme={null}
pnpm db:migrate
```
This creates the following schema:
```sql theme={null}
CREATE TABLE "notes" (
"id" uuid PRIMARY KEY DEFAULT gen_random_uuid() NOT NULL,
"owner_id" text DEFAULT auth.user_id() NOT NULL,
"title" text DEFAULT 'untitled note' NOT NULL,
"created_at" timestamp with time zone DEFAULT now(),
"updated_at" timestamp with time zone DEFAULT now(),
"shared" boolean DEFAULT false
);
CREATE TABLE "paragraphs" (
"id" uuid PRIMARY KEY DEFAULT gen_random_uuid() NOT NULL,
"note_id" uuid REFERENCES notes(id),
"content" text NOT NULL,
"created_at" timestamp with time zone DEFAULT now()
);
```
The migration also sets up RLS policies so users can only access their own notes (and shared notes).
### Configure Logical Replication, User and Publication
PowerSync uses logical replication to sync data from your Neon database.
### 1. Ensure logical replication is enabled
To [ensure logical replication is enabled](https://neon.tech/docs/guides/logical-replication-postgres#prepare-your-source-neon-database):
1. Select your project in the Neon Console.
2. On the Neon Dashboard, select **Settings**.
3. Select **Logical Replication**.
4. Click **Enable** to ensure logical replication is enabled.
### 2. Create a PowerSync database user
```sql theme={null}
-- Create a role/user with replication privileges for PowerSync
CREATE ROLE powersync_role WITH REPLICATION BYPASSRLS LOGIN PASSWORD 'myhighlyrandompassword';
-- Set up permissions for the newly created role
-- Read-only (SELECT) access is required
GRANT SELECT ON ALL TABLES IN SCHEMA public TO powersync_role;
-- Optionally, grant SELECT on all future tables (to cater for schema additions)
ALTER DEFAULT PRIVILEGES IN SCHEMA public GRANT SELECT ON TABLES TO powersync_role;
```
To restrict read access to specific tables, explicitly list allowed tables for both the `SELECT` privilege, and for the publication mentioned in the next step (as well as for any other publications that may exist).
### 3. Create `powersync` publication
```sql theme={null}
-- Create a publication to replicate tables. The publication must be named "powersync"
CREATE PUBLICATION powersync FOR ALL TABLES;
```
Note that the PowerSync Service has to read all updates present in the publication, regardless of whether the table is referenced in your Sync Streams / Sync Rules definitions. This can cause large spikes in memory usage or introduce replication delays, so if you're dealing with large data volumes, you'll want to specify a comma-separated subset of tables to replicate instead of `FOR ALL TABLES`.
The snippet above replicates all tables and is the simplest way to get started in a dev environment.
## Configuring PowerSync
### Create a PowerSync Cloud Instance
When creating a project in the [PowerSync Dashboard](https://dashboard.powersync.com/), *Development* and *Production* instances of the PowerSync Service will be created by default. Select the instance you want to configure.
If you need to create a new instance, follow the steps below.
1. In the dashboard, select your project and open the instance selection dropdown. Click **Add Instance**.
The `aud` field is very sensitive, be sure to enter it exactly as shown above, especially removing the trailing `/` character.
5. Click **Save and Deploy** to apply the changes.
### Configure Sync Streams
[Sync Streams](/sync/streams/overview) (or legacy [Sync Rules](/sync/rules/overview)) allow developers to control which data gets synced to which user devices using a SQL-like syntax in a YAML file. For the demo app, we're going to specify that each user can only see their own notes (plus any shared notes).
1. In the PowerSync Dashboard, select your project and instance and go to the **Sync Streams** view (shown as **Sync Rules** if using legacy Sync Rules).
2. Edit the Sync Config in the editor and replace the contents with the below:
```yaml theme={null}
config:
edition: 3
streams:
user_notes:
auto_subscribe: true
# Sync notes and paragraphs belonging to the authenticated user
queries:
- SELECT * FROM notes WHERE owner_id = auth.user_id()
- SELECT paragraphs.* FROM paragraphs
INNER JOIN notes ON notes.id = paragraphs.note_id
WHERE notes.owner_id = auth.user_id()
shared_notes:
auto_subscribe: true
# Sync all shared notes to all users (not recommended for production)
queries:
- SELECT * FROM notes WHERE shared = TRUE
- SELECT paragraphs.* FROM paragraphs
INNER JOIN notes ON notes.id = paragraphs.note_id
WHERE notes.shared = TRUE
```
```yaml theme={null}
config:
edition: 2
bucket_definitions:
by_user:
# Only sync rows belonging to the user
parameters: SELECT id as note_id FROM notes WHERE owner_id = request.user_id()
data:
- SELECT * FROM notes WHERE id = bucket.note_id
- SELECT * FROM paragraphs WHERE note_id = bucket.note_id
# Sync all shared notes to all users (not recommended for production)
shared_notes:
parameters: SELECT id as note_id from notes where shared = TRUE
data:
- SELECT * FROM notes WHERE id = bucket.note_id
- SELECT * FROM paragraphs WHERE note_id = bucket.note_id
```
3. Click **"Validate"** and ensure there are no errors. This validates your Sync Config against your Postgres database.
4. Click **"Deploy"** to deploy your Sync Config.
* For additional information on PowerSync's Sync Streams, refer to the [Sync Streams](/sync/streams/overview) documentation.
* For legacy Sync Rules, refer to the [Sync Rules](/sync/rules/overview) documentation.
## Test Everything (Using Our Demo App)
In this step you'll test your setup using the 'Notes' demo app. This is a modified version of Neon's demo app.
### Configure Environment Variables
In the demo project directory (`powersync-js/demos/react-neon-notes-tanstack`), update your `.env` file with the following:
```env theme={null}
# Neon Data API URL
# Find this in Neon Console → Data API page → "API URL"
VITE_NEON_DATA_API_URL=https://your-project-id.data-api.neon.tech
# Neon Auth Base URL
# Find this in Neon Console → Auth page → "Auth URL"
VITE_NEON_AUTH_URL=https://your-project-id.auth.neon.tech
# PowerSync instance URL
# Find this in PowerSync Dashboard → Connect button
VITE_POWERSYNC_URL=https://your-instance.powersync.journeyapps.com
```
### Run the App
Start the development server:
```bash theme={null}
pnpm dev
```
Open [http://localhost:5173](http://localhost:5173) in your browser.
Once signed in to the demo app, you should see a blank list of notes, so go ahead and create a new note. Try disabling wifi on your device to test out the offline capabilities. Once back online, you should see the data automatically sync.
After deployment, update your Neon Auth settings to allow your Vercel domain. Go to Neon Console → Auth page and add your Vercel URL (e.g., `https://your-project.vercel.app`) to the allowed origins.
# Integrations Overview
Source: https://docs.powersync.com/integrations/overview
Browse PowerSync integrations with backend services including Supabase, Neon, and Serverpod.
Currently, the following integration guides are available:
If you'd like to see an integration that is not currently available, [let us know on Discord](https://discord.gg/powersync).
# Serverpod + PowerSync
Source: https://docs.powersync.com/integrations/serverpod
Add offline-capable data sync to Serverpod projects using PowerSync.
Used in conjunction with [Serverpod](https://serverpod.dev/), PowerSync enables developers to build local-first apps that are robust in poor network conditions
and that have highly responsive frontends while relying on Serverpod for shared models in a full-stack Dart project.
This guide walks you through configuring PowerSync within your Serverpod project.
## Overview
PowerSync works by:
1. Automatically streaming changes from your Postgres backend source database into a SQLite database on the client.
2. Collecting local writes that users have performed on the SQLite database, and allowing you to upload those writes to your backend.
See [Architecture Overview](/architecture/architecture-overview) for a full overview.
To integrate PowerSync into a Serverpod project, a few aspects need to be considered:
Your Serverpod models need to be persisted into a Postgres database.
PowerSync needs access to your Postgres database to stream changes to users.
To ensure each user only has access to the data they're supposed to see, Serverpod
authenticates users against PowerSync.
After configuring your clients, your Serverpod projects are offline-ready!
This guide shows all steps in detail. Here, we assume you're working with a fresh Serverpod project.
You can follow along by creating a `notes` project using the Serverpod CLI:
```
# If you haven't already, dart pub global activate serverpod_cli
serverpod create notes
```
Of course, all steps and migrations also apply to established projects.
## Database Setup
Begin by configuring your Postgres database for PowerSync. PowerSync requires logical replication
to be enabled. With the `docker-compose.yaml` file generated by Serverpod, add a `command` to the `postgres`
service to enable this option.
This is also a good opportunity to add a health check, which helps PowerSync connect at the right time later:
```yaml theme={null}
services:
# Development services
postgres:
image: pgvector/pgvector:pg16
ports:
- "8090:5432"
command: ["postgres", "-c", "wal_level=logical"] # Added for PowerSync
environment:
POSTGRES_USER: postgres
POSTGRES_DB: notes
# ...
healthcheck: # Added for PowerSync
test: ["CMD-SHELL", "pg_isready -U postgres -d postgres"]
interval: 5s
timeout: 5s
retries: 5
volumes:
- notes_data:/var/lib/postgresql/data
```
You can also find sources for the completed demo [in this repository](https://github.com/powersync-community/powersync-serverpod-demo).
More information about setting up Postgres for PowerSync is available [here](/configuration/source-db/setup).
Next, configure existing models to be persisted in the database. In the template created by
Serverpod, edit `notes_server/lib/src/greeting.spy.yaml`:
```yaml theme={null}
### A greeting message which can be sent to or from the server.
class: Greeting
table: greeting # Added table key
fields:
### Important! Each model used with PowerSync needs to have a UUID id column.
id: UuidValue,defaultModel=random,defaultPersist=random
### The user id owning this greeting, used for access control in PowerSync
owner: String
### The greeting message.
message: String
### The author of the greeting message.
author: String
### The time when the message was created.
timestamp: DateTime
```
PowerSync works best when ids are stable. And since clients can also create rows locally, using
randomized ids reduces the chance of collisions. This is why we prefer UUIDs over the default
incrementing key.
After making the changes, run `serverpod generate` and ignore the issues in `greeting_endpoint.dart` for now.
Instead, run `serverpod create-migration` and note the generated path:
```
$ serverpod create-migration
✓ Creating migration (87ms)
• Migration created: migrations/
✅ Done.
```
We will use the migration adding the `greeting` table to also configure a replication that PowerSync will hook into.
For that, edit `notes_server/migrations//migration.sql`
At the end of that file, after `COMMIT;`, add this:
```sql theme={null}
-- Create a role/user with replication privileges for PowerSync
CREATE ROLE powersync_role WITH REPLICATION BYPASSRLS LOGIN PASSWORD 'myhighlyrandompassword';
-- Set up permissions for the newly created role
-- Read-only (SELECT) access is required
GRANT SELECT ON ALL TABLES IN SCHEMA public TO powersync_role;
-- Optionally, grant SELECT on all future tables (to cater for schema additions)
ALTER DEFAULT PRIVILEGES IN SCHEMA public GRANT SELECT ON TABLES TO powersync_role;
```
To restrict read access to specific tables, explicitly list allowed tables for both the `SELECT` privilege, and for the publication mentioned in the next step (as well as for any other publications that may exist).
This is also a good place to set up a Postgres publication that a PowerSync Service will subscribe to:
```sql theme={null}
-- Create a publication to replicate tables. The publication must be named "powersync"
CREATE PUBLICATION powersync FOR ALL TABLES;
```
Note that the PowerSync Service has to read all updates present in the publication, regardless of whether the table is referenced in your Sync Streams / Sync Rules definitions. This can cause large spikes in memory usage or introduce replication delays, so if you're dealing with large data volumes, you'll want to specify a comma-separated subset of tables to replicate instead of `FOR ALL TABLES`.
The snippet above replicates all tables and is the simplest way to get started in a dev environment.
After adding these statements to `migration.sql`, also add them to `definition.sql`. The reason is that Serverpod
runs that file when instantiating the database from scratch, `migration.sql` would be ignored in that case.
## PowerSync Configuration
PowerSync requires a service to process Postgres writes into a form that can be synced to clients.
Additionally, your Serverpod backend will be responsible for generating JWTs to authenticate clients as
they connect to this service.
To set that up, begin by generating an RSA key to sign these JWTs. In the server project, run
`dart pub add jose` to add a package supporting JWTs in Dart.
Then, create a `tool/generate_keys.dart` that prints a new key pair when run:
```dart theme={null}
import 'dart:convert';
import 'dart:math';
import 'package:jose/jose.dart';
void main() {
var generatedKey = JsonWebKey.generate('RS256').toJson();
final kid = 'powersync-${generateRandomString(8)}';
generatedKey = {...generatedKey, 'kid': kid};
print('''
JS_JWK_N: ${generatedKey['n']}
PS_JWK_E: ${generatedKey['e']}
PS_JWK_KID: $kid
''');
final encodedKeys = base64Encode(utf8.encode(json.encode(generatedKey)));
print('JWT signing keys for backend: $encodedKeys');
}
String generateRandomString(int length) {
final random = Random.secure();
final buffer = StringBuffer();
for (var i = 0; i < length; i++) {
const alphabet = 'abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ';
buffer.writeCharCode(alphabet.codeUnitAt(random.nextInt(alphabet.length)));
}
return buffer.toString();
}
```
Run `dart run tool/generate_jwt.dart` and save its output, it's needed for the next step as well.
For development, you can add the PowerSync Service to the compose file.
It needs access to the source database, a Postgres database to store intermediate data,
and the public half of the generated signing key.
```yaml theme={null}
services:
powersync:
restart: unless-stopped
image: journeyapps/powersync-service:latest
depends_on:
postgres:
condition: service_healthy
command: ["start", "-r", "unified"]
volumes:
- ./service.yaml:/config/service.yaml
environment:
POWERSYNC_CONFIG_PATH: /config/service.yaml
# Use the credentials created in the previous step, the /notes is the datase name for Postgres
PS_SOURCE_URI: "postgresql://powersync_role:myhighlyrandompassword@postgres:5432/notes"
PS_STORAGE_URI: "postgresql://powersync_role:myhighlyrandompassword@postgres:5432/powersync_storage"
JS_JWK_N: # output from generate_keys.dart
PS_JWK_E: AQAB # output from generate_keys.dart
PS_JWK_KID: # output from generate_keys.dart
ports:
- 8095:8080
```
To configure PowerSync, create a file called `service.yaml` next to the compose file.
This file configures how PowerSync connects to the source database, how to authenticate users,
and which data to sync:
```yaml theme={null}
replication:
connections:
- type: postgresql
uri: !env PS_SOURCE_URI
# SSL settings
sslmode: disable # 'verify-full' (default) or 'verify-ca' or 'disable'
# Connection settings for bucket storage
storage:
type: postgresql
uri: !env PS_STORAGE_URI
sslmode: disable # 'verify-full' (default) or 'verify-ca' or 'disable'
# The port which the PowerSync API server will listen on
port: 8080
sync_config:
content: |
config:
edition: 3
streams:
todos:
# For each user, sync all greeting they own.
auto_subscribe: true # Sync by default
query: SELECT * FROM greeting WHERE owner = auth.user_id()
client_auth:
audience: [powersync]
jwks:
keys:
- kty: RSA
n: !env PS_JWK_N
e: !env PS_JWK_E
alg: RS256
kid: !env PS_JWK_KID
```
More information on available options is available under [Service Configuration](/configuration/powersync-service/self-hosted-instances)
## Authentication
PowerSync processes the entire source database into [buckets](/architecture/powersync-service#bucket-system), an efficient representation
for sync. With the configuration shown here, there is one such bucket per user storing all `greeting`s owned by that user.
For security, it is crucial each user only has access to their own bucket. This is why PowerSync gives you full access control:
1. When a client connects to PowerSync, it fetches an authentication token from your Serverpod instance.
2. Your Dart backend logic returns a JWT describing what data the user should have access to.
3. In the `sync_rules` section, you reference properties of the created JWTs to control data visible to the connecting clients.
In this guide, we will use a single virtual user for everything. For real projects, follow
[Serverpod documentation on authentication](https://docs.serverpod.dev/concepts/authentication/setup).
PowerSync needs two endpoints, one to request a JWT and one to upload local writes from clients to the backend source database.
In `notes_server/lib/src/powersync_endpoint.dart`, create those endpoints:
```dart theme={null}
import 'dart:convert';
import 'dart:isolate';
import 'generated/protocol.dart';
import 'package:serverpod/serverpod.dart';
import 'package:jose/jose.dart';
class PowerSyncEndpoint extends Endpoint {
Future createJwt(Session session) async {
// TODO: Throw if the session is unauthenticated.
// TODO: Extract user-id from session outside
final userId = 'global_user';
final token = await Isolate.run(() => _createPowerSyncToken(userId));
// Also create default greeting if none exist for this user.
if (await Greeting.db.count(session) == 0) {
await Greeting.db.insertRow(
session,
Greeting(
owner: userId,
message: 'Hello from Serverpod and PowerSync',
author: 'admin',
timestamp: DateTime.now(),
),
);
}
return token;
}
Future createGreeting(Session session, Greeting greeting) async {
// TODO: Throw if the session is unauthenticated.
await Greeting.db.insertRow(session, greeting);
}
Future updateGreeting(Session session, UuidValue id,
{String? message}) async {
// TODO: Throw if the session is unauthenticated, or if the user should not
// be able to update this greeting.
await session.db.transaction((tx) async {
final row = await Greeting.db.findById(session, id);
await Greeting.db.updateRow(session, row!.copyWith(message: message));
});
}
Future deleteGreeting(Session session, UuidValue id) async {
// TODO: Throw if the session is unauthenticated, or if the user should not
// be able to delete this greeting.
await Greeting.db.deleteWhere(session, where: (tbl) => tbl.id.equals(id));
}
}
Future _createPowerSyncToken(String userId) async {
final decoded = _jsonUtf8.decode(base64.decode(_signingKey));
final signingKey = JsonWebKey.fromJson(decoded as Map);
final now = DateTime.now();
final builder = JsonWebSignatureBuilder()
..jsonContent = {
'sub': userId,
'iat': now.millisecondsSinceEpoch ~/ 1000,
'exp': now.add(Duration(minutes: 10)).millisecondsSinceEpoch ~/ 1000,
'aud': ['powersync'],
'kid': _keyId,
}
..addRecipient(signingKey, algorithm: 'RS256');
final jwt = builder.build();
return jwt.toCompactSerialization();
}
final _jsonUtf8 = JsonCodec().fuse(Utf8Codec());
const _signingKey = 'TODO'; // The "JWT signing keys for backend" bit from tool/generate_keys.dart
const _keyId = 'TODO'; // PS_JWK_KID from tool/generate_keys.dart
```
You can delete the existing `greeting_endpoint.dart` file, it's not necessary since PowerSync is used to fetch data from your server.
Also remove invocations related to future calls in `lib/server.dart`.
Don't forget to run `serverpod generate` afterwards.
## Data Sync
With all services, configured, it's time to spin up development services:
```
docker compose down
docker compose up --detach --scale powersync=0
# This creates the PowerSync role
dart run bin/main.dart --role maintenance --apply-migrations
# Create the PowerSync bucket storage database, use password from docker-compose.yaml
psql -h 127.0.0.1 -p 8090 -U postgres
Password from user postgres:
postgres=# CREATE DATABASE powersync_storage WITH OWNER = powersync_role;
postgres=# \q
# Start PowerSync Service
docker compose up --detach
# Start backend
dart run bin/main.dart
```
With your Serverpod backend and PowerSync running, you can start connecting your clients.
Go to the `_flutter` project generated by Serverpod and run `dart pub add powersync path path_provider`.
Next, replace `main.dart` with this demo:
```dart theme={null}
import 'package:flutter/foundation.dart';
import 'package:notes_client/notes_client.dart';
import 'package:flutter/material.dart';
import 'package:path/path.dart';
import 'package:path_provider/path_provider.dart';
import 'package:powersync/powersync.dart' hide Column;
import 'package:powersync/powersync.dart' as ps;
import 'package:serverpod_flutter/serverpod_flutter.dart';
/// Sets up a global client object that can be used to talk to the server from
/// anywhere in our app. The client is generated from your server code
/// and is set up to connect to a Serverpod running on a local server on
/// the default port. You will need to modify this to connect to staging or
/// production servers.
/// In a larger app, you may want to use the dependency injection of your choice
/// instead of using a global client object. This is just a simple example.
late final Client client;
late final PowerSyncDatabase db;
late String serverUrl;
void main() async {
// When you are running the app on a physical device, you need to set the
// server URL to the IP address of your computer. You can find the IP
// address by running `ipconfig` on Windows or `ifconfig` on Mac/Linux.
// You can set the variable when running or building your app like this:
// E.g. `flutter run --dart-define=SERVER_URL=https://api.example.com/`
const serverUrlFromEnv = String.fromEnvironment('SERVER_URL');
final serverUrl =
serverUrlFromEnv.isEmpty ? 'http://$localhost:8080/' : serverUrlFromEnv;
client = Client(serverUrl)
..connectivityMonitor = FlutterConnectivityMonitor();
db = PowerSyncDatabase(
// For more options on defining the schema, see https://docs.powersync.com/client-sdks/reference/flutter#1-define-the-client-side-schema
schema: Schema([
ps.Table('greeting', [
ps.Column.text('owner'),
ps.Column.text('message'),
ps.Column.text('author'),
ps.Column.text('timestamp'),
])
]),
path: await getDatabasePath(),
logger: attachedLogger,
);
await db.initialize();
await db.connect(connector: ServerpodConnector(client.powerSync));
Object? lastError;
db.statusStream.listen((status) {
final error = status.anyError;
if (error != null && error != lastError) {
debugPrint('PowerSync error: $error');
}
lastError = error;
});
runApp(const MyApp());
}
Future getDatabasePath() async {
const dbFilename = 'powersync-demo.db';
// getApplicationSupportDirectory is not supported on Web
if (kIsWeb) {
return dbFilename;
}
final dir = await getApplicationSupportDirectory();
return join(dir.path, dbFilename);
}
final class ServerpodConnector extends PowerSyncBackendConnector {
final EndpointPowerSync _service;
ServerpodConnector(this._service);
@override
Future fetchCredentials() async {
final token = await _service.createJwt();
return PowerSyncCredentials(
endpoint: 'http://localhost:8095',
token: token,
);
}
@override
Future uploadData(PowerSyncDatabase database) async {
if (await database.getCrudBatch() case final pendingWrites?) {
for (final write in pendingWrites.crud) {
if (write.table != 'greeting') {
throw 'TODO: handle other tables';
}
switch (write.op) {
case UpdateType.put:
await _service.createGreeting(Greeting.fromJson(write.opData!));
case UpdateType.patch:
await _service.updateGreeting(
UuidValue.fromString(write.id),
message: write.opData!['message'] as String?,
);
case UpdateType.delete:
await _service.deleteGreeting(UuidValue.fromString(write.id));
}
}
await pendingWrites.complete();
}
}
}
class MyApp extends StatelessWidget {
const MyApp({super.key});
@override
Widget build(BuildContext context) {
return MaterialApp(
title: 'Serverpod Demo',
theme: ThemeData(primarySwatch: Colors.blue),
home: const GreetingListPage(),
);
}
}
final class GreetingListPage extends StatelessWidget {
const GreetingListPage({super.key});
@override
Widget build(BuildContext context) {
return Scaffold(
appBar: AppBar(
title: Text('PowerSync + Serverpod'),
actions: const [_ConnectionState()],
),
body: StreamBuilder(
stream:
db.watch('SELECT id, message, author FROM greeting ORDER BY id'),
builder: (context, snapshot) {
if (snapshot.hasData) {
return ListView(
children: [
for (final row in snapshot.requireData)
_GreetingRow(
key: ValueKey(row['id']),
id: row['id'],
message: row['message'],
author: row['author'],
),
],
);
} else if (snapshot.hasError) {
return Text(snapshot.error.toString());
} else {
return const CircularProgressIndicator();
}
},
),
);
}
}
final class _GreetingRow extends StatelessWidget {
final String id;
final String message;
final String author;
const _GreetingRow(
{super.key,
required this.id,
required this.message,
required this.author});
@override
Widget build(BuildContext context) {
return ListTile(
title: Row(
children: [
Expanded(child: Text(message)),
IconButton(
onPressed: () async {
await db.execute('DELETE FROM greeting WHERE id = ?', [id]);
},
icon: Icon(Icons.delete),
color: Colors.red,
),
],
),
subtitle: Text('Greeting from $author'),
);
}
}
final class _ConnectionState extends StatelessWidget {
const _ConnectionState({super.key});
@override
Widget build(BuildContext context) {
return StreamBuilder(
stream: db.statusStream,
initialData: db.currentStatus,
builder: (context, snapshot) {
final data = snapshot.requireData;
return Icon(data.connected ? Icons.wifi : Icons.cloud_off);
},
);
}
}
```
Ensure containers are running (`docker compose up`), start your backend `dart run bin/main.dart` in `notes_server`
and finally launch your app.
When the app is loaded, you should see a greeting synced from the server. To verify PowerSync is working,
here are some things to try:
1. Update in the source database: Connect to the Postgres database again (`psql -h 127.0.0.1 -p 8090 -U postgres`) and
run a query like `update greeting set message = upper(message);`. Note how the app's UI reflects these changes without
you having to write any code for these updates.
2. Click on a delete icon to see local writes automatically being uploaded to the backend.
3. Add new items to the database and stop your backend to simulate being offline. Deleting items still updates the client
immediately, changes will be written to Postgres as your backend comes back online.
## Next Steps
This guide demonstrated a minimal setup with PowerSync and Serverpod. To expand on this, you could explore:
* Web support: PowerSync supports Flutter web, but needs [additional assets](/client-sdks/frameworks/flutter-web-support).
* Authentication: If you already have an existing backend that is publicly-reachable, serving a [JWKS URL](https://docs.powersync.com/configuration/auth/custom)
would be safer than using pre-shared keys.
* Deploying: The easiest way to run PowerSync is to [let us host it for you](https://accounts.powersync.com/portal/powersync-signup)
(you still have full control over your source database and backend).
You can also explore [self-hosting](https://docs.powersync.com/intro/powersync-overview) the PowerSync Service.
# Improve Supabase Connector
Source: https://docs.powersync.com/integrations/supabase/connector-performance
Optimize Supabase connector performance in PowerSync apps.
## Background
The demos in the [powersync-js](https://github.com/powersync-ja/powersync-js/tree/main/demos) monorepo provide a minimal working example that illustrate the use of PowerSync with different frameworks.
The demos are therefore not necessarily optimized for performance and can therefore be improved.
This tutorial demonstrates how to improve the Supabase Connector's performance by implementing two batching strategies that reduce the number of database operations.
## Batching Strategies
The two batching strategies that will be implemented are:
1. Sequential Merge Strategy, and
2. Pre-sorted Batch Strategy
Overview:
* Merge adjacent `PUT` and `DELETE` operations for the same table
* Limit the number of operations that are merged into a single API request to Supabase
Shoutout to @christoffer\_configura for the original implementation of this optimization.
```typescript {6-12, 15, 17-19, 21, 23-24, 28-40, 43, 47-60, 63-64, 79} theme={null}
async uploadData(database: CommonPowerSyncDatabase): Promise {
const transaction = await database.getNextCrudTransaction();
if (!transaction) {
return;
}
/**
* Maximum number of PUT or DELETE operations that are merged into a single API request to Supabase.
* Larger numbers can speed up the sync process considerably, but watch out for possible payload size limitations.
* A value of 1 or below disables merging.
*/
const MERGE_BATCH_LIMIT = 100;
let batchedOps: CrudEntry[] = [];
try {
console.log(`Processing transaction with ${transaction.crud.length} operations`);
for (let i = 0; i < transaction.crud.length; i++) {
const cruds = transaction.crud;
const op = cruds[i];
const table = this.client.from(op.table);
batchedOps.push(op);
let result: any;
let batched = 1;
switch (op.op) {
case UpdateType.PUT:
const records = [{ ...cruds[i].opData, id: cruds[i].id }];
while (
i + 1 < cruds.length &&
cruds[i + 1].op === op.op &&
cruds[i + 1].table === op.table &&
batched < MERGE_BATCH_LIMIT
) {
i++;
records.push({ ...cruds[i].opData, id: cruds[i].id });
batchedOps.push(cruds[i]);
batched++;
}
result = await table.upsert(records);
break;
case UpdateType.PATCH:
batchedOps = [op];
result = await table.update(op.opData).eq('id', op.id);
break;
case UpdateType.DELETE:
batchedOps = [op];
const ids = [op.id];
while (
i + 1 < cruds.length &&
cruds[i + 1].op === op.op &&
cruds[i + 1].table === op.table &&
batched < MERGE_BATCH_LIMIT
) {
i++;
ids.push(cruds[i].id);
batchedOps.push(cruds[i]);
batched++;
}
result = await table.delete().in('id', ids);
break;
}
if (batched > 1) {
console.log(`Merged ${batched} ${op.op} operations for table ${op.table}`);
}
}
await transaction.complete();
} catch (ex: any) {
console.debug(ex);
if (typeof ex.code == 'string' && FATAL_RESPONSE_CODES.some((regex) => regex.test(ex.code))) {
/**
* Instead of blocking the queue with these errors,
* discard the (rest of the) transaction.
*
* Note that these errors typically indicate a bug in the application.
* If protecting against data loss is important, save the failing records
* elsewhere instead of discarding, and/or notify the user.
*/
console.error('Data upload error - discarding:', ex);
await transaction.complete();
} else {
// Error may be retryable - e.g. network error or temporary server error.
// Throwing an error here causes this call to be retried after a delay.
throw ex;
}
}
}
```
Overview:
* Create three collections to group operations by type:
* `putOps`: For `PUT` operations, organized by table name
* `deleteOps`: For `DELETE` operations, organized by table name
* `patchOps`: For `PATCH` operations (partial updates)
* Loop through all operations, sort them into the three collections, and then process all operations in batches.
```typescript {8-11, 17-20, 23, 26-29, 32-53, 56, 72} theme={null}
async uploadData(database: CommonPowerSyncDatabase): Promise {
const transaction = await database.getNextCrudTransaction();
if (!transaction) {
return;
}
try {
// Group operations by type and table
const putOps: { [table: string]: any[] } = {};
const deleteOps: { [table: string]: string[] } = {};
let patchOps: CrudEntry[] = [];
// Organize operations
for (const op of transaction.crud) {
switch (op.op) {
case UpdateType.PUT:
if (!putOps[op.table]) {
putOps[op.table] = [];
}
putOps[op.table].push({ ...op.opData, id: op.id });
break;
case UpdateType.PATCH:
patchOps.push(op);
break;
case UpdateType.DELETE:
if (!deleteOps[op.table]) {
deleteOps[op.table] = [];
}
deleteOps[op.table].push(op.id);
break;
}
}
// Execute bulk operations
for (const table of Object.keys(putOps)) {
const result = await this.client.from(table).upsert(putOps[table]);
if (result.error) {
console.error(result.error);
throw new Error(`Could not bulk PUT data to Supabase table ${table}: ${JSON.stringify(result)}`);
}
}
for (const table of Object.keys(deleteOps)) {
const result = await this.client.from(table).delete().in('id', deleteOps[table]);
if (result.error) {
console.error(result.error);
throw new Error(`Could not bulk DELETE data from Supabase table ${table}: ${JSON.stringify(result)}`);
}
}
// Execute PATCH operations individually since they can't be easily batched
for (const op of patchOps) {
const result = await this.client.from(op.table).update(op.opData).eq('id', op.id);
if (result.error) {
console.error(result.error);
throw new Error(`Could not PATCH data in Supabase: ${JSON.stringify(result)}`);
}
}
await transaction.complete();
} catch (ex: any) {
console.debug(ex);
if (typeof ex.code == 'string' && FATAL_RESPONSE_CODES.some((regex) => regex.test(ex.code))) {
/**
* Instead of blocking the queue with these errors,
* discard the (rest of the) transaction.
*
* Note that these errors typically indicate a bug in the application.
* If protecting against data loss is important, save the failing records
* elsewhere instead of discarding, and/or notify the user.
*/
console.error('Data upload error - discarding transaction:', ex);
await transaction.complete();
} else {
// Error may be retryable - e.g. network error or temporary server error.
// Throwing an error here causes this call to be retried after a delay.
throw ex;
}
}
}
```
## Differences
### Sequential merge strategy
```typescript theme={null}
const MERGE_BATCH_LIMIT = 100;
let batchedOps: CrudEntry[] = [];
```
* Processes operations sequentially
* Merges consecutive operations of the same type up to a batch limit
* More dynamic/streaming approach
### Pre-sorted batch strategy
```typescript theme={null}
const putOps: { [table: string]: any[] } = {};
const deleteOps: { [table: string]: string[] } = {};
let patchOps: CrudEntry[] = [];
```
* Pre-sorts all operations by type and table
* Processes each type in bulk after grouping
### Sequential merge strategy
* Uses a sliding window approach with `MERGE_BATCH_LIMIT`
* Merges consecutive operations up to the limit
* More granular control over batch sizes
* Better for mixed operation types
### Pre-sorted batch strategy
* Groups ALL operations of the same type together
* Executes one bulk operation per type per table
* Better for large numbers of similar operations
### Key Similarities and Differences
Handling of CRUD operations (PUT, PATCH, DELETE) to sync local changes to Supabase
Operation grouping strategy
## Use cases
You need more granular control over batch sizes
You want more detailed operation logging
You need to handle mixed operation types more efficiently
You have a large number of similar operations.
You want to minimize the number of network requests.
# Supabase + PowerSync
Source: https://docs.powersync.com/integrations/supabase/guide
Build Supabase apps with PowerSync that use client-side SQLite for real-time updates across users and devices, instant interactions and continued functionality when offline.
This guide shows you how to configure PowerSync with [Supabase](https://supabase.com/). PowerSync syncs selected Supabase Postgres data into client-side SQLite. Client writes (mutations) are commited locally, queued by the SDK, and uploaded via Supabase's client libraries. Your app reads and writes from this local SQLite, so it stays highly responsive even in poor network conditions.
This guide takes 10-15 minutes to complete.
Note: We are currently investigating an issue with Supabase logical replication where unused/idle instances have excessive WAL growth, resulting in maxed out disk usage. Supabase seem to have updated the default archive\_timeout setting to 2 min, which seems to be part of the issue. The current recommendation for pet/hobby projects is to set a smaller `max_wal_size` and `max_slot_wal_keep_size`:
```bash theme={null}
supabase --experimental \
--project-ref \
postgres-config update --config max_wal_size=1GB
```
and also:
```bash theme={null}
supabase --experimental \
--project-ref \
postgres-config update --config max_slot_wal_keep_size=1GB
```
**Before you start, make sure you have:**
* A [Supabase](https://supabase.com/dashboard/sign-up) account
* A [PowerSync Cloud](https://accounts.powersync.com/portal/powersync-signup?s=docs) account
* For mobile/desktop apps: Flutter, React Native, Kotlin, or Xcode set up for your platform of choice
* For web apps: [pnpm](https://pnpm.io/installation#using-npm) installed
## Architecture
Upon successful integration of Supabase + PowerSync, your system architecture will look like this: (click to enlarge image)
For more details on PowerSync's general architecture, [see here](/architecture/architecture-overview).
## Integration Guide/Tutorial Overview
We will follow these steps to get an offline-first 'To-Do List' demo app up and running:
* Create the demo database schema
* Create the Postgres user and publication
* Create connection to Supabase
* Configure Sync Streams
Test the configuration using our provided PowerSync-Supabase 'To-Do List' demo app with your framework of choice.
## Configure Supabase
Create a new Supabase project (or use an existing project if you prefer) and follow the below steps.
### Before You Start: Project Creation Settings
When you create a new Supabase project, two settings on the project creation screen affect how the demo app behaves. Reviewing them now saves debugging later.
**Automatically expose new tables and functions**
This controls whether tables you create in the `public` schema are reachable via the Supabase Data API. The Data API is the auto-generated REST and GraphQL layer that the Supabase client libraries (such as `supabase-js`) call. In this integration, PowerSync handles the download path by replicating data from Postgres to client SQLite databases, but client writes still go back to Postgres through the Data API using `supabase-js`. If the tables are not exposed, the demo app cannot write changes back to your database. Supabase is rolling out a new default:
* **From April 28, 2026:** the setting is available as an opt-in. New projects can choose to require explicit `grant` statements.
* **From May 30, 2026:** the new default for all new projects is that tables in `public` are not exposed to the Data API on creation.
* **From October 30, 2026:** the new default is enforced on all existing projects.
The schema SQL in this guide includes explicit `grant` statements, so the demo app works under either default. See [this](https://github.com/orgs/supabase/discussions/45329) for more details.
**Enable automatic RLS**
This adds an event trigger that automatically enables Row Level Security on every new table created in the `public` schema. Enabling it is recommended because RLS is the authoritative security layer for any writes that reach Postgres through the Data API. If you leave it disabled, you can still enable RLS per table as the schema SQL in this guide does.
Without RLS, any user with the `anon` or `authenticated` key can read and write every row in the table, regardless of who created it.
### Create the Demo Database Schema
To set up the Postgres database for our *To-Do List* demo app, we will create two new tables: `lists` and `todos`. The demo app will have access to these tables even while offline.
Run the below SQL statements in your **Supabase SQL Editor**:
```sql theme={null}
create table
public.lists (
id uuid not null default gen_random_uuid (),
created_at timestamp with time zone not null default now(),
name text not null,
owner_id uuid not null,
constraint lists_pkey primary key (id),
constraint lists_owner_id_fkey foreign key (owner_id) references auth.users (id) on delete cascade
) tablespace pg_default;
create table
public.todos (
id uuid not null default gen_random_uuid (),
created_at timestamp with time zone not null default now(),
completed_at timestamp with time zone null,
description text not null,
completed boolean not null default false,
created_by uuid null,
completed_by uuid null,
list_id uuid not null,
constraint todos_pkey primary key (id),
constraint todos_created_by_fkey foreign key (created_by) references auth.users (id) on delete set null,
constraint todos_completed_by_fkey foreign key (completed_by) references auth.users (id) on delete set null,
constraint todos_list_id_fkey foreign key (list_id) references lists (id) on delete cascade
) tablespace pg_default;
-- Grant Data API access so supabase-js can read and write these tables.
grant select, insert, update, delete on public.lists to authenticated;
grant select, insert, update, delete on public.todos to authenticated;
grant select, insert, update, delete on public.lists to service_role;
grant select, insert, update, delete on public.todos to service_role;
-- Enable Row Level Security and add policies so each user can only access their own data.
alter table public.lists enable row level security;
alter table public.todos enable row level security;
create policy "owned lists" on public.lists
for all to authenticated
using (auth.uid() = owner_id);
create policy "todos in owned lists" on public.todos
for all to authenticated
using (
auth.uid() in (
select lists.owner_id from public.lists where lists.id = todos.list_id
)
);
```
**Grants and RLS do different jobs, and you need both.** Grants control which Postgres roles can touch a table through the Data API. Without a grant, `supabase-js` returns a `42501: permission denied for table` error before RLS is ever evaluated. RLS runs after the grant succeeds and filters which rows that role can read or modify. Skipping RLS means any user with the `anon` or `authenticated` key can read and write every row.
This guide grants `authenticated` because the demo app requires sign-in. The `anon` role is intentionally left ungranted. If you wanted to expose a table for public read-only access (for example, a shared catalog visible to signed-out users), you would add `grant select on public.your_table to anon` along with an RLS policy scoping which rows `anon` can read.
PowerSync's Sync Streams (configured later in this guide) control which rows get downloaded to each client and generally mirror your RLS setup. For more on how these work together, see [RLS and Sync Streams](/integrations/supabase/rls-and-sync-streams).
### Create a PowerSync Database User
PowerSync uses the Postgres [Write Ahead Log (WAL)](https://www.postgresql.org/docs/current/wal-intro.html) to replicate data changes in order to keep PowerSync SDK clients up to date.
Run the below SQL statement in your **Supabase SQL Editor** to create a Postgres role/user with replication privileges:
```sql theme={null}
-- Create a role/user with replication privileges for PowerSync
CREATE ROLE powersync_role WITH REPLICATION BYPASSRLS LOGIN PASSWORD 'myhighlyrandompassword';
-- Set up permissions for the newly created role
-- Read-only (SELECT) access is required
GRANT SELECT ON ALL TABLES IN SCHEMA public TO powersync_role;
-- Optionally, grant SELECT on all future tables (to cater for schema additions)
ALTER DEFAULT PRIVILEGES IN SCHEMA public GRANT SELECT ON TABLES TO powersync_role;
```
To restrict read access to specific tables, explicitly list allowed tables for both the `SELECT` privilege, and for the publication mentioned in the next step (as well as for any other publications that may exist).
### Create the Postgres Publication
Run the below SQL statement in your **Supabase SQL Editor** to create a Postgres publication:
```sql theme={null}
-- Create a publication to replicate tables. The publication must be named "powersync"
CREATE PUBLICATION powersync FOR ALL TABLES;
```
Note that the PowerSync Service has to read all updates present in the publication, regardless of whether the table is referenced in your Sync Streams / Sync Rules definitions. This can cause large spikes in memory usage or introduce replication delays, so if you're dealing with large data volumes, you'll want to specify a comma-separated subset of tables to replicate instead of `FOR ALL TABLES`.
The snippet above replicates all tables and is the simplest way to get started in a dev environment.
## Configure PowerSync
### Create a PowerSync Cloud Instance
When creating a project in the [PowerSync Dashboard](https://dashboard.powersync.com/), *Development* and *Production* instances of the PowerSync Service will be created by default. Select the instance you want to configure.
If you need to create a new instance, follow the steps below.
1. In the dashboard, select your project and open the instance selection dropdown. Click **Add Instance**.
```yaml theme={null}
config:
edition: 3
streams:
user_data:
auto_subscribe: true
queries:
- SELECT * FROM lists WHERE owner_id = auth.user_id()
- SELECT todos.* FROM todos INNER JOIN lists ON todos.list_id = lists.id WHERE lists.owner_id = auth.user_id()
```
```yaml theme={null}
bucket_definitions:
user_lists:
# Separate bucket per To-Do list
parameters: select id as list_id from lists where owner_id = request.user_id()
data:
- select * from lists where id = bucket.list_id
- select * from todos where list_id = bucket.list_id
```
3. Click **"Validate"** and ensure there are no errors. This validates your Sync Config against your Postgres database.
4. Click **"Deploy"** to deploy your Sync Config.
* For additional information on PowerSync's Sync Streams, refer to the [Sync Streams](/sync/streams/overview) documentation.
* For legacy Sync Rules, refer to the [Sync Rules](/sync/rules/overview) documentation.
* If you're wondering how Sync Streams relate to Supabase Postgres [RLS](https://supabase.com/docs/guides/auth/row-level-security), see [this subsection](/integrations/supabase/rls-and-sync-streams).
## Test Everything (Using Our Demo App)
In this step you'll test your setup using a 'To-Do List' demo app provided by PowerSync.
#### Clone the Demo App
Clone the demo app based on your framework:
```bash Flutter theme={null}
git clone https://github.com/powersync-ja/powersync.dart.git
cd powersync.dart/demos/supabase-todolist/
```
```bash React Native theme={null}
git clone https://github.com/powersync-ja/powersync-js.git
cd powersync-js/demos/react-native-supabase-todolist
```
```bash JavaScript Web theme={null}
git clone https://github.com/powersync-ja/powersync-js.git
cd powersync-js/demos/react-supabase-todolist
```
```bash Kotlin theme={null}
git clone https://github.com/powersync-ja/powersync-kotlin.git
# Open `demos/supabase-todolist` in Android Studio
```
```bash Swift theme={null}
git clone https://github.com/powersync-ja/powersync-swift.git
# Open the Demo directory in XCode and follow the README instructions.
```
#### Configure the Demo App to Use Your PowerSync Instance
Locate the relevant config file for your framework:
```bash Flutter theme={null}
cp lib/app_config_template.dart lib/app_config.dart
# Edit `lib/app_config.dart` and insert the necessary credentials as detailed below.
```
```bash React Native theme={null}
# Edit the `.env` file and insert the necessary credentials as detailed below.
```
```bash JavaScript Web theme={null}
cp .env.local.template .env.local
# Edit `.env.local` and insert the necessary credentials as detailed below.
```
```bash Kotlin theme={null}
# Make a `local.properties` file in the root and fill in the relevant variables (see points below for further details):
# local.properties
sdk.dir=/path/to/android/sdk
# Enter your PowerSync instance URL
POWERSYNC_URL=https://foo.powersync.journeyapps.com
# Enter your Supabase project's URL and public anon key
SUPABASE_URL=https://foo.supabase.co # from https://supabase.com/dashboard/project/_/settings/api
SUPABASE_ANON_KEY=foo # from https://supabase.com/dashboard/project/_/settings/api-keys
```
```bash Swift theme={null}
# Edit the `_Secrets` file and insert the necessary credentials as detailed below.
```
Supabase has replaced the legacy anon key with **publishable keys** (prefixed `sb_publishable_…`) by default on new projects. The demos still read environment variables named `SUPABASE_ANON_KEY` / `supabaseAnonKey`, but the value is interchangeable: paste your publishable key into the same variable. Existing projects that still use legacy JWT keys can continue to use the anon key directly.
1. In the relevant config file, replace the values for `supabaseUrl` (from the [Project URL](https://supabase.com/dashboard/project/_/settings/api) section in the Supabase dashboard) and `supabaseAnonKey` (from the [API Keys](https://supabase.com/dashboard/project/_/settings/api-keys) section in the Supabase dashboard).
2. For the value of `powersyncUrl`, click **Connect** in the top bar of the [PowerSync Dashboard](https://dashboard.powersync.com/) and copy the instance URL from the dialog.
#### Run the App
```bash Flutter theme={null}
# Ensure you have [melos](https://melos.invertase.dev/~melos-latest/getting-started) installed.
melos bootstrap
flutter run
```
```bash React Native theme={null}
# In `demos/react-native-supabase-todolist`:
# Run on iOS
pnpm ios
# Run on Android
pnpm android
```
```bash JavaScript Web theme={null}
# In `demos/react-supabase-todolist`:
pnpm dev
```
```bash Kotlin theme={null}
# Run the app on Android or iOS in Android Studio using the Run widget.
```
```bash Swift theme={null}
# Run the app using XCode.
```
For ease of use of the demo app, you can disable email confirmation in your Supabase Auth settings. In your Supabase project, go to "Authentication" -> "Providers" -> "Email" and then disable "Confirm email". If you keep email confirmation enabled, the Supabase user confirmation email will reference the default Supabase Site URL of `http://localhost:3000` — you can ignore this.
This guide describes an example local dev workflow that uses ngrok and the PowerSync CLI.
This guide assumes that you have both ngrok and the Supabase CLI installed
This guide only covers using ngrok. Other configurations such as an NGINX reverse proxy are also possible.
### Configure Supabase for SSL
```bash theme={null}
# start supabase
supabase start
# get the name of the supabase-db container
docker ps -f name=supabase-db --format '{{.Names}}'
# The rest of the script assumes it's "supabase-db_supabase-test"
# bash in the container
docker exec -it supabase-db_supabase-test /bin/bash
# Now run in the container:
cd /etc/postgresql-custom
# Create a cert
openssl req -days 3650 -new -text -nodes -subj '/C=US/O=Dev/CN=supabase_dev' -keyout server.key -out server.csr
openssl req -days 3650 -x509 -text -in server.csr -key server.key -out server.cert
chown postgres:postgres server.*
# Enable ssl
echo -e '\n\nssl = on\nssl_ciphers = '\''HIGH:MEDIUM:+3DES:!aNULL'\''\nssl_prefer_server_ciphers = on\nssl_cert_file = '\''/etc/postgresql-custom/server.cert'\''\nssl_key_file = '\''/etc/postgresql-custom/server.key'\''' >> supautils.conf
# Now Ctrl+D to exit bash, and restart the container:
docker restart supabase-db_supabase-test
# Check logs for any issues:
docker logs supabase-db_supabase-test
# (optional, for debugging) validate SSL is enabled
psql -d postgres postgres
postgres=> show ssl; # should return "on"
```
### Start ngrok
Here we obtain the local port that supabase is listening on and initialize ngrok using it.
```bash theme={null}
# look for the PORTS value of the supabase-db_supabase-test container
docker ps -f name=supabase-db --format '{{.Ports}}'
# should see something like 0.0.0.0:54322->5432/tcp
# use the first port
ngrok tcp 54322
# should then see something like this:
Forwarding tcp://4.tcp.us-cal-1.ngrok.io:19263 -> localhost:54322
```
Make a note of the hostname (`4.tcp.us-cal-1.ngrok.io` and port number `19263`), your values will differ.
### Connect PowerSync (GUI)
1. Configure your PowerSync instance using the hostname and port number you noted previously. The default postgres password is "postgres", you may want to change this. NOTE: make sure that the `Host` field does not contain the `tcp://` URI Scheme outputted by ngrok
Supabase tables are often created with auto-increment IDs. For easiest use of PowerSync, make sure to convert them to text IDs as detailed [**here**](/sync/advanced/client-id)**.**
### Example
The [Supabase + PowerSync guide](/integrations/supabase/guide#create-the-demo-database-schema) sets up RLS policies for the to-do list demo app:
```sql theme={null}
create policy "owned lists" on public.lists
for all to authenticated
using (auth.uid() = owner_id);
create policy "todos in owned lists" on public.todos
for all to authenticated
using (
auth.uid() in (
select lists.owner_id from public.lists where lists.id = todos.list_id
)
);
```
`auth.uid()` in a Supabase RLS policy maps to:
* `auth.user_id()` in [Sync Streams](/sync/streams/overview)
* `request.user_id()` (previously `token_parameters.user_id`) in legacy [Sync Rules](/sync/rules/overview)
If you compare these policies to the Sync Config in the guide, you'll see the access patterns mirror each other.
If you have any questions, join us on [our community Discord](https://discord.gg/powersync) where our team is always available to help.
# Demo Apps & Example Projects
Source: https://docs.powersync.com/intro/examples
Demo apps and example projects showcasing PowerSync across all supported platforms and backends.
The best way to understand how PowerSync works is to explore it hands-on. Browse our collection of demo apps and example projects to see PowerSync in action, experiment with different features, or use as a reference for your own app.
All examples are organized by platform and backend technology. You can adapt any example to work with your preferred backend (see [App Backend](/configuration/app-backend)).
We continuously expand our collection of example projects. If you need an example that isn't available yet, [let us know on Discord](https://discord.gg/powersync).
## Official Demos/Example Projects
#### Supabase Backend
* [To-Do List App](https://github.com/powersync-ja/powersync-js/tree/main/demos/react-native-supabase-todolist#readme)
* Demonstrates [File/Attachment Handling](/client-sdks/advanced/attachments)
* [PowerChat - Group Chat App](https://github.com/powersync-ja/powersync-js/tree/main/demos/react-native-supabase-group-chat#readme)
* [To-Do List App: React Native Web & Electron](https://github.com/powersync-community/powersync-react-native-web-expo-electron#readme)
* [Background Sync Demo](https://github.com/powersync-ja/powersync-js/tree/main/demos/react-native-supabase-background-sync)
* Demonstrates [Background Syncing](/client-sdks/advanced/background-syncing) using Expo's BackgroundTask API
#### Django Custom Backend
* [To-Do List App](https://github.com/powersync-ja/powersync-js/tree/main/demos/django-react-native-todolist#readme)
* Corresponding backend: [Django To-Do List Backend](https://github.com/powersync-ja/powersync-django-backend-todolist-demo)
#### Node.js Backend
* ['MBnB' Listing Search App with Node.js MongoDB Backend](https://github.com/powersync-ja/powersync-react-native-mongodb-mbnb#readme)
* This repo contains both the React Native app and Node.js backend
#### Other
* [Point of Sale (POS) App](https://github.com/powersync-community/powersync-pos-demo#readme)
* [React Native Barebones Demo](https://github.com/powersync-ja/powersync-js/tree/main/demos/react-native-barebones)
* This is a barebones example of using PowerSync in a plain React Native app without Expo or any other framework. See [here](/client-sdks/reference/react-native-and-expo#installation) for more background.
#### Supabase Backend
* [React To-Do List App](https://github.com/powersync-ja/powersync-js/tree/main/demos/react-supabase-todolist#readme) (PWA compatible)
* Includes [Full-Text Search](/client-sdks/full-text-search) capabilities
* [React To-Do List App with Local-Only Tables](https://github.com/powersync-ja/powersync-js/tree/main/demos/react-supabase-todolist-optional-sync#readme) - Shows data persistence without syncing
* See [Local-Only Usage](/client-sdks/advanced/local-only-usage) for more background.
* [React Multi-Client Widget](https://github.com/powersync-ja/powersync-js/tree/main/demos/react-multi-client#readme)
* Featured on the [PowerSync website](https://www.powersync.com/demo) demonstrating real-time data flow between clients
* [Vue To-Do List App](https://github.com/powersync-ja/powersync-js/tree/main/demos/vue-supabase-todolist#readme)
* [Nuxt To-Do List App](https://github.com/powersync-ja/powersync-js/tree/main/demos/nuxt-supabase-todolist#readme)
* [Angular To-Do List App](https://github.com/powersync-ja/powersync-js/tree/main/demos/angular-supabase-todolist#readme)
* [Yjs CRDT Text Collaboration Demo](https://github.com/powersync-ja/powersync-js/tree/main/demos/yjs-react-supabase-text-collab#readme)
* [Vite + React + TS + PowerSync + Supabase](https://github.com/powersync-community/vite-react-ts-powersync-supabase#readme)
* [E2EE Chat App](https://github.com/powersync-community/react-supabase-chat-e2ee#readme) - End-to-end encrypted group chat demo
* [Time-Based Sync Demo](https://github.com/powersync-ja/powersync-js/tree/main/demos/react-supabase-time-based-sync) - Dynamically control which data is synced to the client based on a date
#### Convex Backend
* [PowerSync Convex To-Do List Demo](https://github.com/powersync-community/powersync-convex-todolist-demo) - JavaScript demo app using Convex as the source backend.
#### Framework Integration Examples
* [Electron](https://github.com/powersync-ja/powersync-js/tree/main/demos/example-electron) - PowerSync in an Electron web app (renderer process)
* Also see [Node.js + Electron](https://github.com/powersync-ja/powersync-js/tree/main/demos/example-electron-node) for PowerSync in the main process, and see [this blog post](https://www.powersync.com/blog/speeding-up-electron-apps-with-powersync) for more background.
* [Next.js](https://github.com/powersync-ja/powersync-js/blob/main/demos/example-nextjs#readme) - Minimal setup with Next.js
* [Webpack](https://github.com/powersync-ja/powersync-js/blob/main/demos/example-webpack#readme) - Bundling with Webpack
* [Vite](https://github.com/powersync-ja/powersync-js/blob/main/demos/example-vite#readme) - Bundling with Vite
* [Vite with Encryption](https://github.com/powersync-ja/powersync-js/blob/main/demos/example-vite-encryption#readme) - Web database encryption demo
#### Examples
* [CLI Example](https://github.com/powersync-ja/powersync-js/tree/main/demos/example-node) - Node.js CLI client connecting to PowerSync and running live queries
* [Electron Main Process](https://github.com/powersync-ja/powersync-js/tree/main/demos/example-electron-node#readme) - PowerSync in Electron's main process using the Node.js SDK. See [this blog post](https://www.powersync.com/blog/speeding-up-electron-apps-with-powersync) for more background
* [Node.js + Drizzle Minimal Demo](https://github.com/powersync-community/nodejs-drizzle-example)
#### Examples
* [Capacitor Example](https://github.com/powersync-ja/powersync-js/tree/main/demos/example-capacitor#readme) - PowerSync in a Capacitor app with web, Android, and iOS support
#### Examples
* [Tauri Example](https://github.com/powersync-ja/powersync-js/tree/main/demos/tauri-app#readme) - PowerSync in a Tauri app with Desktop support.
#### Supabase Backend
* [To-Do List App](https://github.com/powersync-ja/powersync.dart/tree/main/demos/supabase-todolist#readme)
* Includes [Full-Text Search](/client-sdks/full-text-search) capabilities
* Demonstrates [File/Attachment Handling](/client-sdks/advanced/attachments)
* [To-Do List App + Drift](https://github.com/powersync-ja/powersync.dart/tree/main/demos/supabase-todolist-drift#readme)
* See [Dart/Flutter ORM Support](/client-sdks/orms/flutter-orm-support) for more details on our Drift integration.
* [To-Do List App with Local-Only Tables](https://github.com/powersync-ja/powersync.dart/tree/main/demos/supabase-todolist-optional-sync#readme) - Shows data persistence without syncing
* See [Local-Only Usage](/client-sdks/advanced/local-only-usage) for more background.
* [Simple Chat App](https://github.com/powersync-ja/powersync.dart/tree/main/demos/supabase-simple-chat#readme)
* [Trello Clone App](https://github.com/powersync-ja/powersync.dart/tree/main/demos/supabase-trello#readme)
#### Dart Custom Backend
* [Built with Flutter and Serverpod](https://github.com/powersync-community/powersync-serverpod-demo)
* [Built with Jaspr, shelf, Riverpod and Drift](https://github.com/powersync-community/self-host-dart-fullstack)
#### Node.js Custom Backend
* [To-Do List App with Firebase Auth](https://github.com/powersync-ja/powersync.dart/tree/main/demos/firebase-nodejs-todolist#readme)
* Corresponding backend: [Node.js To-Do List Backend with Firebase Auth](https://github.com/powersync-ja/powersync-nodejs-firebase-backend-todolist-demo)
#### Rails Custom Backend
* [GoToFun App](https://github.com/powersync-ja/powersync-rails-flutter-demo)
* This repo contains both the Flutter app and Rails backend
#### Django Custom Backend
* [To-Do List App](https://github.com/powersync-ja/powersync.dart/tree/main/demos/django-todolist#readme)
* Corresponding backend: [Django To-Do List Backend](https://github.com/powersync-ja/powersync-django-backend-todolist-demo)
#### Self-Hosted Node.js Postgres Backend
* [Flutter Home Screen Widget Demo](https://github.com/powersync-community/flutter-home-widget)
#### Supabase Backend
* [To-Do List App](https://github.com/powersync-ja/powersync-kotlin/tree/main/demos/supabase-todolist#readme)
* Supports Android, iOS, and Desktop (JVM) targets
* Includes a guide for [implementing background sync on Android](https://github.com/powersync-ja/powersync-kotlin/blob/main/demos/supabase-todolist/docs/BackgroundSync.md)
* [Native Android To-Do List App](https://github.com/powersync-ja/powersync-kotlin/tree/main/demos/android-supabase-todolist#readme)
* Demonstrates [File/Attachment Handling](/client-sdks/advanced/attachments)
#### Other
* [Java Example](https://github.com/powersync-community/java-kmp-sdk-example#readme) - shows how the Kotlin SDK can be used in a Java 8+ application.
#### Supabase Backend
* [To-Do List App](https://github.com/powersync-ja/powersync-swift/tree/main/Demos/PowerSyncExample)
* Includes [Full-Text Search](/client-sdks/full-text-search) capabilities
* Demonstrates [File/Attachment Handling](/client-sdks/advanced/attachments)
* [To-Do List App + GRDB](https://github.com/powersync-ja/powersync-swift/tree/main/Demos/GRDBDemo)
* Demonstrates [GRDB integration](/client-sdks/orms/swift/grdb) with PowerSync
* [Encryption Demo](https://github.com/powersync-ja/powersync-swift/tree/main/Demos/SwiftEncryptionDemo)
* Demonstrates [Data Encryption](/usage/use-case-examples/data-encryption) using SQLite3MultipleCiphers
* [Counter App](https://github.com/powersync-community/swift-supabase-counter#readme)
#### Examples
* [CLI Application](https://github.com/powersync-ja/powersync-dotnet/tree/main/demos/CommandLine#readme)
* Includes an optional [Supabase connector](https://github.com/powersync-ja/powersync-dotnet/blob/main/demos/CommandLine/SupabaseConnector.cs)
* [WPF To-Do List App](https://github.com/powersync-ja/powersync-dotnet/tree/main/demos/WPF#readme)
* A Windows desktop to-do list app built with WPF.
* [MAUI To-Do List App](https://github.com/powersync-ja/powersync-dotnet/tree/main/demos/MAUITodo#readme)
* A cross-platform to-do list app for Android, iOS, and Windows.
* Demonstrates [File/Attachment Handling](/client-sdks/advanced/attachments)
#### Examples
* [egui To-Do List](https://github.com/powersync-ja/powersync-native/blob/main/README.md) - Desktop to-do list example using the egui framework and a self-hosted Node.js + Postgres backend.
#### Django
* [Django Backend for To-Do List App](https://github.com/powersync-ja/powersync-django-backend-todolist-demo)
* For use with:
* React Native [To-Do List App](https://github.com/powersync-ja/powersync-js/tree/main/demos/django-react-native-todolist)
* Flutter [To-Do List App](https://github.com/powersync-ja/powersync.dart/tree/main/demos/django-todolist)
#### Node.js
* [Node.js Backend for To-Do List App](https://github.com/powersync-ja/powersync-nodejs-backend-todolist-demo)
* [Node.js Backend with Firebase Auth](https://github.com/powersync-ja/powersync-nodejs-firebase-backend-todolist-demo)
* For use with: Flutter [To-Do List App with Firebase Auth](https://github.com/powersync-ja/powersync.dart/tree/main/demos/firebase-nodejs-todolist)
#### Rails
* [Rails Backend for GoToFun App](https://github.com/powersync-ja/powersync-rails-flutter-demo/tree/main/gotofun-backend)
* For use with: Flutter [GoToFun App](https://github.com/powersync-ja/powersync-rails-flutter-demo/tree/main/gotofun-app)
#### .NET
* [.NET Backend for To-Do List App](https://github.com/powersync-ja/powersync-dotnet-backend-demo)
#### Complete Stacks with Docker Compose
* [To-Do List App with Docker Compose](https://github.com/powersync-ja/self-host-demo) - React web app with various backend configurations:
* [Node.js + Postgres](https://github.com/powersync-ja/self-host-demo/tree/main/demos/nodejs)
* [Node.js + Postgres + Postgres Bucket Storage](https://github.com/powersync-ja/self-host-demo/tree/main/demos/nodejs-postgres-bucket-storage)
* [Node.js + MongoDB](https://github.com/powersync-ja/self-host-demo/tree/main/demos/nodejs-mongodb)
* [Node.js + MySQL](https://github.com/powersync-ja/self-host-demo/tree/main/demos/nodejs-mysql)
* [Node.js + SQL Server](https://github.com/powersync-ja/self-host-demo/tree/main/demos/nodejs-mssql)
* [Supabase (Postgres) + Local Development](https://github.com/powersync-ja/self-host-demo/tree/main/demos/supabase)
* [Django + Postgres](https://github.com/powersync-ja/self-host-demo/tree/main/demos/django)
## Community
### Community GitHub Org
Browse the Community GitHub Org for a collection of community-based starter projects, templates, demos and other projects to help you succeed with PowerSync:
* [Laravel Backend by @IsmailAshour](https://github.com/IsmailAshour/powersync-laravel-backend)
* [SolidJS Hooks for PowerSync Queries by @aboviq](https://github.com/aboviq/powersync-solid)
* [Effect + Kysely + Stytch Integration by @guillempuche](https://github.com/guillempuche/localfirst_react_server)
* [Tauri + Shadcn UI by @MrLightful](https://github.com/MrLightful/powersync-tauri)
* [Expo Web Integration by @ImSingee](https://github.com/ImSingee/powersync-web-workers)
* Note: Our [React Native Web support](/client-sdks/frameworks/react-native-web-support) now eliminates the need to patch the `@powersync/web` module
* [Attachments Library for Node.js by @muhammedv](https://www.npmjs.com/package/@muhammedv/powersync-attachments-for-node)
* [Flutter Instagram Clone with Supabase + Firebase by @Gambley1](https://github.com/Gambley1/flutter-instagram-offline-first-clone)
* [Jepsen PowerSync Testing - Formal consistency validation framework by @nurturenature](https://github.com/nurturenature/jepsen-powersync)
* [Bavard - An Eloquent-inspired ORM for Dart/Flutter by @ILDaviz](https://ildaviz.github.io/bavard/)
### Notable Community Tutorials
* [Account Optional Apps with PowerSync](https://www.maxmntl.com/blog/optional-account-powersync/)
* Tutorial for starting your new user app experience fully local (without sync,) and then to switch them to a synced experience
* [Building an Offline-First Mobile App with PowerSync](https://blog.stackademic.com/building-an-offline-first-mobile-app-with-powersync-40674d8b7ea1)
* Postgres + Flutter + Nest.js + Prisma ORM + Firebase Auth
* [Implementing Local-First Architecture: A Guide to MongoDB Cluster and PowerSync Integration](https://blog.stackademic.com/implementing-local-first-architecture-a-guide-to-mongodb-cluster-and-powersync-integration-6b21fa8059a1)
* MongoDB Atlas + Next.js
## Additional Resources
Haven't found what you're looking for?
Additional tutorial-style technical posts can be found on the [PowerSync Blog](https://www.powersync.com/blog). Popular pages include:
* [Migrating a MongoDB Atlas Device Sync App to PowerSync](https://www.powersync.com/blog/migrating-a-mongodb-atlas-device-sync-app-to-powersync)
* [PowerSync and Supabase: Just the Basics](https://www.powersync.com/blog/powersync-and-supabase-just-the-basics)
* [Flutter Tutorial: Building An Offline-First Chat App With Supabase And PowerSync](https://www.powersync.com/blog/flutter-tutorial-building-an-offline-first-chat-app-with-supabase-and-powersync)
* [Speeding Up Electron Apps With PowerSync](https://www.powersync.com/blog/speeding-up-electron-apps-with-powersync)
* [Building an E2EE Chat App with PowerSync + Supabase](https://www.powersync.com/blog/building-an-e2ee-chat-app-with-powersync-supabase)
* [Collaborative Text Editing Over PowerSync](https://www.powersync.com/blog/collaborative-text-editing-over-powersync) (Without CRDTs)
# PowerSync Docs
Source: https://docs.powersync.com/intro/powersync-overview
PowerSync keeps in-app SQLite synced with your backend so apps update across users and devices in real time, respond instantly, and continue working offline.
PowerSync is made up of the PowerSync Service and a set of client SDKs. The PowerSync Service replicates data from your backend database, partitions it based on what data each user should receive, and streams real-time updates to clients through the PowerSync client SDK.
```text Build with AI icon="sparkles" wrap theme={null}
Install the PowerSync Agent Skills with: npx skills add powersync-ja/agent-skills. Then follow the skills to onboard this project to PowerSync.
```
## Get Started
Go from zero to a working PowerSync implementation in your project.
A quick way to get a feel for PowerSync is to run a demo app.
The architecture and the philosophy behind PowerSync.
Official PowerSync skills for Claude Code, Codex, and other AI-powered coding tools.
## Supported Client SDKs
PowerSync is also designed to be client-side stack agnostic, and currently has client SDKs available for:
Looking for an SDK that's not listed above? Upvote it or submit it on [our roadmap](https://roadmap.powersync.com/).
## Supported Backend Source Databases
PowerSync is designed to be backend database agnostic, and supports these source databases:
## Need Help?
Can't find what you are looking for in these docs? Try **Ask AI** on this site which is trained on all our documentation, repositories and Discord discussions. Also join us on [our community Discord server](https://discord.gg/powersync) where you can browse topics from the PowerSync community and chat with our team.
# PowerSync Philosophy
Source: https://docs.powersync.com/intro/powersync-philosophy
PowerSync's vision is that local-first architecture should be easier to implement than cloud-first and deliver a better end-user experience.
## What PowerSync Means for End-Users
The app is always [fast, responsive and reactive](https://www.powersync.com/blog/local-first-is-a-big-deal-especially-for-the-web#to-the-user-everything-feels-instant-no-loading-spinners). There is no need to wait for network requests.
The app just works, whether fully online, fully offline, or with spotty connectivity.
## What PowerSync Means for the Developer
PowerSync lets you avoid the complexities of using APIs to move app state [over the network](https://www.powersync.com/blog/escaping-the-network-tarpit). Its goal is to solve the hard problems of keeping data in sync, without getting in your way.
You use a supported backend source such as Postgres, MongoDB, MySQL, SQL Server, Azure DocumentDB, or Convex on the server, a standard SQLite database on the client, and your [own backend](/configuration/app-backend/setup) to process mutations. PowerSync keeps the SQLite database in sync with your backend source.
### State Management
Once you have a local SQLite database that is always in sync, [state management](https://www.powersync.com/blog/local-first-state-management-with-sqlite) becomes much easier:
* No need for custom caching logic, whether in-memory or persisted.
* No need for maintaining in-memory state across the application.
[All state is in the local database](https://www.powersync.com/blog/local-first-state-management-with-sqlite). Queries are [reactive](/client-sdks/watch-queries): they update whenever the underlying data changes.
### Flexibility
PowerSync allows you to fully customize what data is synced to the client. You can sync the entire database, or use [Sync Streams](/sync/streams/overview) to transform and filter data for each client (partial sync ).
Writing back to the backend source database [is in full control of the developer](/handling-writes/writing-client-changes). You can use your own authentication, validation, and constraints.
Our goal is also to be stack-agnostic: whether you are switching from MySQL to Postgres, from Flutter to React Native, or using multiple different stacks - our aim is to maintain maximum engineering optionality for developers.
### Performance
[SQLite is *fast*](https://www.powersync.com/blog/sqlite-optimizations-for-ultra-high-performance). It can perform tens of thousands of updates per second, even faster reads, with support for concurrent reads. Once you get to filtering through thousands of rows in queries, indexes keep the queries fast.
### Simplicity
You use a supported backend source on the server, with no significant change in your schema required for most databases \[[1](#footnotes)]. PowerSync [uses](/configuration/source-db/setup) Postgres logical replication, MongoDB change streams, the MySQL binlog, SQL Server Change Data Capture (CDC), or Convex document deltas to replicate changes to the [PowerSync Service](/architecture/powersync-service), where data is transformed and partitioned according to [Sync Streams](/sync/streams/overview), and persisted in a way that allows efficiently streaming incremental changes to each client.
PowerSync has been used in apps with hundreds of tables. There are no complex migrations to run: You define your [Sync Streams](/sync/streams/overview) and [client-side schema](/intro/setup-guide#define-your-client-side-schema), and the data is automatically kept in sync. If you [change Sync Streams/Rules](/maintenance-ops/implementing-schema-changes), the relevant new set of data is applied atomically on the client. When you do need to make schema changes on the server while still supporting older clients, we [have the processes in place](/maintenance-ops/implementing-schema-changes) to do that without hassle.
No need for CRDTs \[[2](#footnotes)]. PowerSync is a server-client sync platform: since no peer-to-peer syncing is involved, CRDTs can be overkill. Instead, PowerSync uses a server reconciliation architecture, putting the developer in [full control](/handling-writes/writing-client-changes) of how client mutations are applied and managed on the source database. PowerSync imposes no conflict policy: the app backend decides how mutations are applied, while straightforward CRUD implementations commonly produce per-field LWW. Our [causal+ consistency guarantees](/architecture/consistency) give you peace of mind about the integrity of data on the client.
## See Also
* [Local-First Software is a Big Deal, Especially for the Web](https://www.powersync.com/blog/local-first-is-a-big-deal-especially-for-the-web)
* [PowerSync Architecture](/architecture/architecture-overview)
## Footnotes
* \[1] In some cases denormalization is required to effectively partition the data to sync to different users.
* \[2] If you want to use CRDTs for fine-grained collaboration like text editing, we have [examples](/client-sdks/advanced/crdts) of how to do that in conjunction with PowerSync, storing CRDT data in Postgres.
# Self-Hosting
Source: https://docs.powersync.com/intro/self-hosting
Get started self-hosting the PowerSync Service with Docker, including local dev setup via the PowerSync CLI.
The [PowerSync Service](https://github.com/powersync-ja/powersync-service) can be self-hosted using Docker. It is published to Docker Hub as [journeyapps/powersync-service](https://hub.docker.com/r/journeyapps/powersync-service).
* The [PowerSync Dashboard](https://dashboard.powersync.com/) is currently not available when self-hosting PowerSync.
* Please reach out on our [Discord](https://discord.gg/powersync) if you have any questions not yet covered in these docs.
Coolify is an open-source & self-hostable alternative to Heroku / Netlify / Vercel / etc.
Railway is a managed cloud platform (PaaS) for deploying and scaling applications, services, and databases via containers.
Amazon Elastic Container Service (ECS) is a fully managed container orchestration service.
Amazon Elastic Kubernetes Service (EKS) is a managed Kubernetes service for running containerized workloads.
## Enterprise Self-Hosted Edition
Self-hosting of PowerSync is also available in an Enterprise Self-Hosted Edition with dedicated support plans, advanced functionality and custom pricing. See the *Self-Host PowerSync* section on our [Pricing](https://www.powersync.com/pricing) page for more details.
To get started on the Enterprise Self-Hosted Edition please [contact us](mailto:support@powersync.com).
# PowerSync Setup Guide
Source: https://docs.powersync.com/intro/setup-guide
Step-by-step guide to adding PowerSync to your app, from connecting your source database to integrating the client SDK.
PowerSync keeps a client-side SQLite database in sync with your backend database. Changes appear across users and devices in real time, user interactions feel instant, and your app continues to work even when offline.
Your app reads and writes SQLite directly. With [Sync Streams](/sync/streams/overview), you define the subset of backend data each client receives. This enables partial sync instead of replicating the entire backend database to every device. Changes in that subset flow through the PowerSync Service into SQLite, where watched queries update the UI. Client writes are committed locally first, then enter an upload queue and are sent to your application backend for validation and persistence. For synced data, this reduces per-screen fetching, cache invalidation, and client-state plumbing.
By the end of this guide, you will have connected your source database, defined what data each client receives, and integrated a client SDK that reads, writes, and watches local data.
## 1. Configure Your Source Database
PowerSync needs to connect to your source database (Postgres, MongoDB, Azure DocumentDB, MySQL, SQL Server or Convex) to replicate data. Before setting up PowerSync, you need to configure your database with the appropriate permissions and replication settings.
Using the [PowerSync CLI](/tools/cli) and want an automatically integrated Postgres instance for local development? You can skip to [Step 2](#2-set-up-powersync-service-instance) and set one up with the **CLI (Self-Hosted)** tab.
Configuring Postgres for PowerSync involves three main tasks:
1. Enable logical replication: PowerSync reads the Postgres write-ahead log (WAL) using logical replication. Set `wal_level = logical` in your Postgres configuration.
2. Create a PowerSync database user: Create a role with replication privileges and read-only access to your tables.
3. Create a `powersync` publication: Create a logical replication publication named `powersync` to specify which tables to replicate.
```sql General theme={null}
-- 1. Enable logical replication (requires restart)
ALTER SYSTEM SET wal_level = logical;
-- 2. Create PowerSync database user/role with replication privileges and read-only access to your tables
CREATE ROLE powersync_role WITH REPLICATION BYPASSRLS LOGIN PASSWORD 'myhighlyrandompassword';
-- Set up permissions for the newly created role
-- Read-only (SELECT) access is required
GRANT SELECT ON ALL TABLES IN SCHEMA public TO powersync_role;
-- Optionally, grant SELECT on all future tables (to cater for schema additions)
ALTER DEFAULT PRIVILEGES IN SCHEMA public GRANT SELECT ON TABLES TO powersync_role;
-- 3. Create a publication to replicate tables. The publication must be named "powersync"
CREATE PUBLICATION powersync FOR ALL TABLES;
```
```sql Supabase theme={null}
-- Supabase has logical replication enabled by default
-- Just create the user and publication:
-- Create PowerSync database user/role with replication privileges and read-only access to your tables
CREATE ROLE powersync_role WITH REPLICATION BYPASSRLS LOGIN PASSWORD 'myhighlyrandompassword';
-- Set up permissions for the newly created role
-- Read-only (SELECT) access is required
GRANT SELECT ON ALL TABLES IN SCHEMA public TO powersync_role;
-- Optionally, grant SELECT on all future tables (to cater for schema additions)
ALTER DEFAULT PRIVILEGES IN SCHEMA public GRANT SELECT ON TABLES TO powersync_role;
-- Create a publication to replicate tables. The publication must be named "powersync"
CREATE PUBLICATION powersync FOR ALL TABLES;
```
```bash Docker (Self-hosting) theme={null}
# 1. Create a Docker network (if not already created)
# This allows various PowerSync containers to communicate with each other
docker network create powersync-network
# 2. Run Postgres source database with logical replication enabled (required for PowerSync)
docker run -d \
--name powersync-postgres \
--network powersync-network \
-e POSTGRES_PASSWORD="my_secure_password" \
-p 5432:5432 \
postgres:18 \
postgres -c wal_level=logical
# 3. Configure PowerSync user and publication
# This creates a PowerSync database user/role with replication privileges and read-only access to your tables
# Read-only (SELECT) access is also granted to all future tables (to cater for schema additions)
# It also creates a publication to replicate tables. The publication must be named "powersync"
docker exec -it powersync-postgres psql -U postgres -c "
CREATE ROLE powersync_role WITH REPLICATION BYPASSRLS LOGIN PASSWORD 'myhighlyrandompassword';
GRANT SELECT ON ALL TABLES IN SCHEMA public TO powersync_role;
ALTER DEFAULT PRIVILEGES IN SCHEMA public GRANT SELECT ON TABLES TO powersync_role;
CREATE PUBLICATION powersync FOR ALL TABLES;"
```
PowerSync requires Postgres version 11 or greater.
**Learn More**
* For more details on Postgres setup, including provider-specific guides (Supabase, AWS RDS, etc.), see [Source Database Setup](/configuration/source-db/setup#postgres).
* **Self-hosting PowerSync?** See the [Self-Host-Demo App](https://github.com/powersync-ja/self-host-demo/tree/main/demos/nodejs) for a complete working example of connecting a Postgres source database to PowerSync.
For MongoDB Atlas databases, the minimum permissions when using built-in roles are:
```
read@
readWrite@._powersync_checkpoints
```
To allow PowerSync to automatically enable `changeStreamPreAndPostImages` on replicated collections (optional, but recommended), additionally add:
```
dbAdmin@
```
PowerSync requires MongoDB version 6.0 or greater.
**Learn More**
* For more details including instructions for self-hosted MongoDB, or for custom roles on MongoDB Atlas, see [Source Database Setup](/configuration/source-db/setup#mongodb).
* **Self-hosting PowerSync?** See the [Self-Host-Demo App](https://github.com/powersync-ja/self-host-demo/tree/main/demos/nodejs-mongodb) for a complete working example of connecting a MongoDB source database to PowerSync.
Setup, permissions, and connection steps for Azure DocumentDB are the same as MongoDB, except for a Post Images setting. Follow the setup instructions for MongoDB, then see [Azure DocumentDB Source Database Setup](/configuration/source-db/setup#azure-documentdb) for further details.
Azure DocumentDB support is currently an [alpha release](/resources/feature-status).
For MySQL, you need to configure binary logging and create a user with replication privileges:
```sql theme={null}
-- Configure binary logging
-- Add to MySQL option file (my.cnf or my.ini):
server_id=
log_bin=ON
enforce_gtid_consistency=ON
gtid_mode=ON
binlog_format=ROW
-- Create a user with necessary privileges
CREATE USER 'repl_user'@'%' IDENTIFIED BY '';
-- Grant replication client privilege
GRANT REPLICATION SLAVE, REPLICATION CLIENT ON *.* TO 'repl_user'@'%';
-- Grant select access to the specific database
GRANT SELECT ON .* TO 'repl_user'@'%';
-- Apply changes
FLUSH PRIVILEGES;
```
PowerSync requires MySQL version 5.7 or greater.
MySQL support is currently in a [Beta release](/resources/feature-status).
**Learn More**
* For more details on MySQL setup, see [Source Database Setup](/configuration/source-db/setup#mysql).
* **Self-hosting PowerSync?** See the [Self-Host-Demo App](https://github.com/powersync-ja/self-host-demo/tree/main/demos/nodejs-mysql) for a complete working example of connecting a MySQL source database to PowerSync.
Refer to [these instructions](/configuration/source-db/setup#sql-server).
**Self-hosting PowerSync?** See the [Self-Host-Demo App](https://github.com/powersync-ja/self-host-demo/tree/main/demos/nodejs-mssql) for a complete working example of connecting a SQL Server source database to PowerSync.
SQL Server support is currently in a [Beta release](/resources/feature-status).
Configuring Convex for PowerSync involves adding a `powersync_checkpoints` table and a `createCheckpoint` mutation to your Convex deployment, then generating a deploy key. See [Convex Source Database Setup](/configuration/source-db/setup#convex) for the full steps.
For a complete working example, see the [PowerSync Convex To-Do List Demo](https://github.com/powersync-community/powersync-convex-todolist-demo).
Convex support is currently an [Experimental release](/resources/feature-status).
## 2. Set Up PowerSync Service Instance
PowerSync is available as a cloud-hosted service (PowerSync Cloud) or can be self-hosted (PowerSync Open Edition or PowerSync Enterprise Self-Hosted Edition).
If you haven't yet, sign up for a free PowerSync Cloud account [here](https://accounts.powersync.com/portal/powersync-signup?s=docs).
After signing up, you will be taken to the [PowerSync Dashboard](https://dashboard.powersync.com/).
Here, create a new project. PowerSync creates *Development* and *Production* instances of the PowerSync Service by default in the project.
If you haven't yet, sign up for a free PowerSync Cloud account [here](https://accounts.powersync.com/portal/powersync-signup?s=docs).
Install the [PowerSync CLI](/tools/cli) (requires Node.js/npm), then log in and scaffold the config directory:
```bash theme={null}
npm install -g powersync
powersync login
powersync init cloud
```
This creates a `powersync/` directory with `service.yaml` (instance name, region, connection, auth) and `sync-config.yaml` (Sync Config). Edit `powersync/service.yaml` to set your instance name and region.
Then create the instance and link the directory to it:
```bash theme={null}
powersync link cloud --create --project-id=
```
Find your project ID in the [PowerSync Dashboard](https://dashboard.powersync.com) URL, or run `powersync fetch instances` after logging in.
You'll configure the database connection in the next step.
Recommended for getting started: the CLI scaffolds your config directory and generates the Docker Compose stack (including a Postgres instance for the source database and storage) so you can run PowerSync locally with minimal setup. For custom setups use the **Manual (Self-Hosted)** tab. Install the [PowerSync CLI](/tools/cli) (requires Node.js/npm); alternative installation options (e.g. installers via GitHub releases) will be available in the near future. Then run:
```bash theme={null}
npm install -g powersync
powersync init self-hosted
powersync docker configure --database postgres --storage postgres
```
Docker sets up Postgres for both the source database and bucket storage and creates `powersync/docker/docker-compose.yaml`. Other databases are supported as well, you will learn more about this in the next step. Before starting, replace `powersync/sync-config.yaml` with this minimal Sync Config:
```yaml theme={null}
config:
edition: 3
streams:
todos:
# Streams without parameters sync the same data to all users
auto_subscribe: true
query: "SELECT * FROM todos"
```
You'll update this with your actual tables/collections in a later step.
The Docker Postgres instance runs init scripts only on first start. Create your specific tables before running `powersync docker start` for the first time. See the [Docker usage docs](https://github.com/powersync-ja/powersync-cli/blob/main/docs/usage-docker.md) in the PowerSync CLI repository for more details.
Then start the PowerSync Service:
```bash theme={null}
powersync docker start
```
Run `powersync status` to verify it's running.
**Learn More**
* [Self-Hosting Introduction](/intro/self-hosting)
* [Self-Host Demo App](https://github.com/powersync-ja/self-host-demo) for complete working examples.
* [Self-Hosted Service Configuration](/configuration/powersync-service/self-hosted-instances) for more details on the config file structure.
* [CLI documentation](/tools/cli)
Self-hosted PowerSync runs via Docker. The commands below illustrate the basic PowerSync Service requirements.
Below is a minimal example using Postgres for bucket storage. MongoDB is also supported as bucket storage. The source database connection is configured in the next step. You can use the Docker-managed Postgres from Step 1 or point to an external database instead.
```bash theme={null}
# 1. Create a directory for your config
mkdir powersync-service && cd powersync-service
# 2. Set up bucket storage (Postgres and MongoDB are supported)
docker run -d \
--name powersync-postgres-storage \
--network powersync-network \
-p 5433:5432 \
-e POSTGRES_PASSWORD="my_secure_storage_password" \
-e POSTGRES_DB=powersync_storage \
postgres:18
## Set up Postgres storage user
docker exec -it powersync-postgres-storage psql -U postgres -d powersync_storage -c "
CREATE USER powersync_storage_user WITH PASSWORD 'my_secure_user_password';
GRANT CREATE ON DATABASE powersync_storage TO powersync_storage_user;"
# 3. Create service.yaml (see below)
# 4. Run PowerSync Service
# The Service config can be specified as an environment variable (shown below), as a filepath, or as a command line parameter
# See these docs for more details: https://docs.powersync.com/configuration/powersync-service/self-hosted-instances
docker run -d \
--name powersync \
--network powersync-network \
-p 8080:8080 \
-e POWERSYNC_CONFIG_B64="$(base64 -i ./service.yaml)" \
journeyapps/powersync-service:latest
```
**Basic `service.yaml` structure:**
```yaml theme={null}
# Source database connection (see the next step for more details)
replication:
connections:
- type: postgresql # or mongodb, mysql, mssql. for azure documentdb, use mongodb
uri: postgresql://powersync_role:myhighlyrandompassword@powersync-postgres:5432/postgres
sslmode: disable # Only for local/private networks
# Connection settings for bucket storage (Postgres and MongoDB are supported)
storage:
type: postgresql
uri: postgresql://powersync_storage_user:my_secure_user_password@powersync-postgres-storage:5432/powersync_storage
sslmode: disable # Use 'disable' only for local/private networks
# Sync Streams config (defined in a later step)
sync_config:
content: |
config:
edition: 3
streams:
shared_data:
auto_subscribe: true
queries:
- SELECT * FROM lists
- SELECT * FROM todos
```
This example assumes you've configured your source database with the required user and publication (see the previous step)
and are running it via Docker in the 'powersync-network' network.
If you are not using Docker, you will need to specify the connection details in the `service.yaml` file manually (see next step for more details).
**Learn More**
* [Self-Hosting Introduction](/intro/self-hosting)
* [Self-Host Demo App](https://github.com/powersync-ja/self-host-demo) for complete working examples.
* [Self-Hosted Service Configuration](/configuration/powersync-service/self-hosted-instances) for more details on the config file structure.
* [CLI documentation](/tools/cli)
## 3. Connect PowerSync to Your Source Database
The next step is to connect your PowerSync Service instance to your source database.
In the [PowerSync Dashboard](https://dashboard.powersync.com/), select your project and instance, then go to **Database Connection**:
1. Select the appropriate database type tab (Postgres, MongoDB, MySQL, SQL Server or Convex). For Azure DocumentDB, select MongoDB.
2. Fill in your connection details:
Use the username (e.g., `powersync_role`) and password you created in Step 1: Configure your Source Database.
* **Postgres**: Host, Port (5432), Database name, Username, Password, SSL Mode
* **MongoDB**: Connection URI (e.g., `mongodb+srv://user:pass@cluster.mongodb.net/database`)
* **Azure DocumentDB**: Connection URI (e.g., `mongodb+srv://user:pass@cluster.windows.net/database`)
* **MySQL**: Host, Port (3306), Database name, Username, Password
* **Convex**: Deployment URL and deploy key. See [Convex Source Database Setup](/configuration/source-db/setup#convex).
* **SQL Server**: Name, Host, Port (1433), Database name, Username, Password
3. Click **Test Connection** to verify
4. Click **Save Connection**
PowerSync will now deploy and configure an isolated cloud environment, which can take a few minutes.
**Learn More**
For more details on database connections, including provider-specific connection details (Supabase, AWS RDS, MongoDB Atlas, etc.), see [Source Database Connection](/configuration/source-db/connection).
Edit `powersync/service.yaml` (created in the previous step) with your connection details. Use `!env` for secrets:
Use the username (e.g., `powersync_role`) and password you created in Step 1: Configure your Source Database.
```yaml Postgres theme={null}
replication:
connections:
- type: postgresql
uri: postgresql://powersync_role:myhighlyrandompassword@host:5432/postgres
sslmode: disable # 'verify-full' (default) or 'verify-ca' or 'disable'
# Note: 'disable' is only suitable for local/private networks
```
```yaml MongoDB theme={null}
replication:
connections:
- type: mongodb
uri: mongodb+srv://user:password@cluster.mongodb.net/database
post_images: auto_configure
```
```yaml Azure DocumentDB theme={null}
replication:
connections:
- type: mongodb
uri: mongodb+srv://user:password@cluster.windows.net/database
post_images: off
```
```yaml MySQL theme={null}
replication:
connections:
- type: mysql
uri: mysql://repl_user:password@host:3306/database
```
```yaml SQL Server theme={null}
replication:
connections:
- type: mssql
uri: mssql://user:password@host:1433/database
schema: dbo
```
```yaml Convex theme={null}
replication:
connections:
- type: convex
deployment_url: https://.convex.cloud
deploy_key:
```
You will run `powersync deploy` in a later step to deploy your config to the PowerSync Cloud instance.
**Learn More**
For more details on database connections, including provider-specific connection details (Supabase, AWS RDS, MongoDB Atlas, etc.), see [Source Database Connection](/configuration/source-db/connection).
If you used Docker in the previous step, the source database connection is already configured. `service.yaml` reads the connection URI from `!env PS_DATA_SOURCE_URI`. The Docker-managed Postgres (`pg-db`) was also pre-configured with `wal_level=logical` and a `powersync` publication by the init scripts.
If you want to use an **external database** instead, update `PS_DATA_SOURCE_URI` in `powersync/docker/.env` with your connection details, then restart:
```bash theme={null}
powersync docker reset
```
You'll also need to complete the source database setup from Step 1 (replication user, publication) on your external database before this will work.
Configure the source database connection in your `service.yaml` file (as you did in the previous step). Examples for the different database types are below.
Use the username (e.g., `powersync_role`) and password you created in Step 1: Configure your Source Database.
```yaml Postgres theme={null}
replication:
connections:
- type: postgresql
uri: postgresql://powersync_role:myhighlyrandompassword@powersync-postgres:5432/postgres
sslmode: disable # 'verify-full' (default) or 'verify-ca' or 'disable'
# Note: 'disable' is only suitable for local/private networks, not for public networks
```
```yaml MySQL theme={null}
replication:
connections:
- type: mysql
uri: mysql://repl_user:password@host:3306/database
```
```yaml SQL Server theme={null}
replication:
connections:
- type: mssql
uri: mssql://user:password@$host:1433/database
schema: dbo
additionalConfig:
trustServerCertificate: true
pollingIntervalMs: 1000
pollingBatchSize: 20
```
**Learn More**
See the [self-host-demo app](https://github.com/powersync-ja/self-host-demo) for complete working examples of the different database types.
## 4. Define Sync Streams
PowerSync uses **Sync Streams** to control which data gets synced to which users/devices, using SQL-like queries defined in YAML format.
Start with simple **auto-subscribed streams** that sync data to all users by default:
```yaml Postgres Example theme={null}
config:
edition: 3
streams:
shared_data:
auto_subscribe: true
queries:
- SELECT * FROM todos
- SELECT * FROM lists WHERE NOT archived
```
```yaml MongoDB/Azure DocumentDB Example theme={null}
config:
edition: 3
streams:
shared_data:
auto_subscribe: true
# MongoDB and Azure DocumentDB use "_id" but PowerSync uses "id" on the client
queries:
- SELECT *, _id as id FROM lists
- SELECT *, _id as id FROM todos WHERE archived = false
```
```yaml MySQL Example theme={null}
config:
edition: 3
streams:
shared_data:
auto_subscribe: true
queries:
- SELECT * FROM todos
- SELECT * FROM lists WHERE NOT archived
```
```yaml SQL Server Example theme={null}
config:
edition: 3
streams:
shared_data:
auto_subscribe: true
queries:
- SELECT * FROM todos
- SELECT * FROM lists WHERE NOT archived
```
**Learn more:** [Sync Streams documentation](/sync/streams/overview)
### Deploy Your Configuration
In the [PowerSync Dashboard](https://dashboard.powersync.com/):
1. Select your project and instance
2. Go to the **Sync Streams** view
3. Edit the YAML directly in the dashboard
4. Click **Deploy** to validate and deploy your Sync Streams
Edit `powersync/sync-config.yaml` with your Sync Config, then validate and deploy to the linked Cloud instance:
```bash theme={null}
powersync validate
powersync deploy
```
This deploys your full config (connection, auth, and Sync Config). For subsequent sync-only changes, use `powersync deploy sync-config` instead.
Edit `powersync/sync-config.yaml` with your Sync Config. The default file has a placeholder (`SELECT * FROM todos`). Replace it with your actual table/collection names, then apply the changes:
```bash theme={null}
powersync validate
powersync docker reset
```
Add a `sync_config` section to your `service.yaml`. Using a separate file (recommended) keeps the main config tidy:
**Recommended — reference a separate file:**
```yaml service.yaml theme={null}
sync_config:
path: sync-config.yaml
```
Put your streams in `sync-config.yaml` (see [Self-Hosted Instance Configuration](/configuration/powersync-service/self-hosted-instances#sync_config) for full examples). Alternatively, you can use inline `content: |` with the YAML nested under `sync_config`.
Table/collection names in your configuration must match the table names defined in your client-side schema (defined in a later step below).
## 5. Generate a Development Token
For quick development and testing, you can generate a temporary development token instead of implementing full authentication.
You'll use this token for two purposes:
* **Testing with the *Sync Diagnostics Client*** (in the next step) to verify your setup and Sync Streams
* **Connecting your app** (in a later step) to test the client SDK integration
1. In the [PowerSync Dashboard](https://dashboard.powersync.com/), select your project and instance
2. Go to the **Client Auth** view
3. Check the **Development tokens** setting and save your changes
4. Click the **Connect** button in the top bar
5. Enter a token subject: Since you're starting with simple streams or buckets that sync all data to all users (as recommended in the previous step), you can put something like `test-user` as the token subject (which would normally be the user ID you want to test with).
6. Click **Generate token** and copy the token
Development tokens expire after 12 hours.
Generate a development token with:
```bash theme={null}
powersync generate token --subject=test-user
```
Replace `test-user` with a user ID of your choice (this would normally be the user ID you want to test with).
Requires `allow_temporary_tokens` to be enabled on the instance. Add it to `powersync/service.yaml` if you haven't already, then redeploy:
```yaml theme={null}
client_auth:
allow_temporary_tokens: true
```
```bash theme={null}
powersync deploy
```
Development tokens expire after 12 hours.
Follow the steps below. Steps 1 and 2 configure signing keys and your PowerSync config; in Step 3 you can use the **CLI (recommended)** or the test-client to generate the token.
Generate a temporary private/public key-pair (RS256) or shared key (HS256) for JWT signing and verification.
Use an online JWK generator like [mkjwk.org](https://mkjwk.org/) (select RSA, 2048 bits, Signature use, RS256 algorithm).
Or generate locally with Node.js:
```bash theme={null}
# Install pem-jwk if needed
npm install -g pem-jwk
# Generate private key
openssl genrsa -out private-key.pem 2048
# Convert public key to JWK format
openssl rsa -in private-key.pem -pubout | pem-jwk
```
Use an online JWK generator like [mkjwk.org](https://mkjwk.org/) (select oct, 256 bits, Signature use, HS256 algorithm) - this outputs base64url directly.
Or generate and convert using OpenSSL:
```bash theme={null}
# Generate and convert to base64url
openssl rand -base64 32 | tr '+/' '-_' | tr -d '='
```
For production environments, shared secrets (HS256) are not recommended.
Add the `client_auth` parameter to your PowerSync config (e.g. `service.yaml`):
Copy the JWK values from [mkjwk.org](https://mkjwk.org/) or the `pem-jwk` output, then add to your config:
```yaml service.yaml theme={null}
# Client (application end user) authentication settings
client_auth:
# static collection of public keys for JWT verification
jwks:
keys:
- kty: 'RSA'
n: '[rsa-modulus]'
e: '[rsa-exponent]'
alg: 'RS256'
kid: 'dev-key-1'
```
Copy the `k` value from mkjwk.org or the OpenSSL output, then add to your config:
```yaml service.yaml theme={null}
# Client (application end user) authentication settings
client_auth:
audience: ['http://localhost:8080', 'http://127.0.0.1:8080']
# static collection of public keys for JWT verification
jwks:
keys:
- kty: oct
alg: 'HS256'
k: '[base64url-encoded-shared-secret]'
kid: 'dev-key-1'
```
These examples use static `jwks: keys:` for simplicity. For production, we recommend using `jwks_uri` to point to a JWKS endpoint instead. See [Custom Authentication](/configuration/auth/custom) for more details.
Choose either the [PowerSync CLI](/tools/cli) (recommended) or the test-client:
Apply your config changes (e.g. restart your PowerSync Service or run `powersync docker reset` if running locally with Docker), then run:
```bash theme={null}
powersync generate token --subject=test-user
```
Replace `test-user` with the user ID you want to authenticate:
* If your Sync Streams aren't filtered by user (same data syncs to all users), you can use any value (e.g., `test-user`).
* If your data is filtered by parameters , use a user ID that matches a user in your database. PowerSync uses this value (e.g. via `auth.user_id()`) to determine what to sync.
1. If you have not done so already, clone the [`powersync-service` repo](https://github.com/powersync-ja/powersync-service/tree/main)
2. Install and build:
* In the project root: `pnpm install` and `pnpm build`
* In the `test-client` directory: `pnpm build`
3. Generate a token from the `test-client` directory, pointing at your config file:
```bash theme={null}
node dist/bin.js generate-token --config path/to/service.yaml --sub test-user
```
If your config uses `!env PS_*` tags to reference environment variables, the test-client reads them from your shell. To load them from a file instead, pass `--env`:
```bash theme={null}
node dist/bin.js generate-token --config path/to/service.yaml --env path/to/.env --sub test-user
```
Replace `test-user` with the user ID you want to authenticate:
* If your Sync Streams aren't filtered by user (same data syncs to all users), you can use any value (e.g., `test-user`).
* If your data is filtered by parameters , use a user ID that matches a user in your database. PowerSync uses this value (e.g. via `auth.user_id()`) to determine what to sync.
Development tokens expire after 12 hours.
## 6. \[Optional] Test Sync with the Sync Diagnostics Client
Before implementing the PowerSync Client SDK in your app, you can validate that syncing is working correctly using our [Sync Diagnostics Client](https://diagnostics-app.powersync.com) (this hosted version works with both PowerSync Cloud and self-hosted setups).
Use the development token you generated in the [previous step](#5-generate-a-development-token) to connect and verify your setup:
1. Go to [https://diagnostics-app.powersync.com](https://diagnostics-app.powersync.com)
2. Enter your development token at **PowerSync Token** (from the [Generate a Development Token](#5-generate-a-development-token) step above)
3. Enter your PowerSync instance URL at **PowerSync Endpoint** (found in the [PowerSync Dashboard](https://dashboard.powersync.com/) - click **Connect** in the top bar)
4. Click **Proceed**
1. Go to [https://diagnostics-app.powersync.com](https://diagnostics-app.powersync.com)
2. Enter your development token at **PowerSync Token** (from the [Generate a Development Token](#5-generate-a-development-token) step above)
3. Enter your PowerSync Service endpoint at **PowerSync Endpoint** (the URL where your self-hosted service is running, e.g. `http://localhost:8080` if running locally)
4. Click **Proceed**
The Sync Diagnostics Client can also be run as a local standalone web app. See the [README](https://github.com/powersync-ja/powersync-js/tree/main/tools/diagnostics-app#readme) for instructions.
The Sync Diagnostics Client will connect to your PowerSync Service instance and display [information](https://github.com/powersync-ja/powersync-js/tree/main/tools/diagnostics-app#functionality) about the synced data, and allow you to [query](https://github.com/powersync-ja/powersync-js/tree/main/tools/diagnostics-app#sql-console) the client-side SQLite database.
**Checkpoint:**
Inspect your synced tables in the Sync Diagnostics Client. These should match the Sync Streams you [defined previously](#4-define-sync-streams). This confirms your setup is working correctly before integrating the client SDK into your app.
## 7. Use the Client SDK
Now it's time to integrate PowerSync into your app. This involves installing the Client SDK, defining your client-side schema, instantiating the database, connecting to your PowerSync Service instance, and reading/writing data.
### Install the Client SDK
Add the PowerSync Client SDK to your app project. PowerSync provides SDKs for various platforms and frameworks.
Add the [PowerSync React Native NPM package](https://www.npmjs.com/package/@powersync/react-native) to your project:
```bash theme={null}
npx expo install @powersync/react-native @op-engineering/op-sqlite
```
```bash theme={null}
yarn expo add @powersync/react-native @op-engineering/op-sqlite
```
```
pnpm expo install @powersync/react-native @op-engineering/op-sqlite
```
`@op-engineering/op-sqlite` is the SQLite library used by the PowerSync SDK on React Native.
It needs to be installed separately because React Native only considers direct dependencies for autolinking.
The PowerSync React Native SDK currently supports version 1.17.0 or later of `@op-engineering/op-sqlite`.
**Using Expo Go?** Our native database adapter is not compatible with Expo Go's sandbox environment. To run PowerSync with Expo Go install our JavaScript-based adapter `@powersync/adapter-sql-js` instead. See details [here](/client-sdks/frameworks/expo-go-support).
**Polyfills and additional notes:**
* For async iterator support with watched queries, additional polyfills are required. See the [Babel plugins section](https://www.npmjs.com/package/@powersync/react-native#babel-plugins-watched-queries) in the README.
* We recommend adding this [metro config](https://github.com/powersync-ja/powersync-js/tree/main/packages/react-native#metro-config-optional) to avoid build issues.
Add the [PowerSync Web NPM package](https://www.npmjs.com/package/@powersync/web) to your project:
```bash theme={null}
npm install @powersync/web
```
```bash theme={null}
yarn add @powersync/web
```
```bash theme={null}
pnpm install @powersync/web
```
Add the [PowerSync Node NPM package](https://www.npmjs.com/package/@powersync/node) to your project:
```bash theme={null}
npm install @powersync/node
```
```bash theme={null}
yarn add @powersync/node
```
```bash theme={null}
pnpm install @powersync/node
```
**Install Peer Dependencies**
The PowerSync SDK for Node.js supports multiple drivers. More details are available under [Encryption and Custom SQLite Drivers](/client-sdks/reference/node#encryption-and-custom-sqlite-drivers). We currently recommend the `better-sqlite3` package for most users:
```bash theme={null}
npm install better-sqlite3
```
```bash theme={null}
yarn add better-sqlite3
```
```bash theme={null}
pnpm install better-sqlite3
```
Previous versions of the PowerSync SDK for Node.js used the `@powersync/better-sqlite3` fork as a
required peer dependency.
This is no longer recommended. After upgrading to `@powersync/node` version `0.12.0` or later, ensure
the old package is no longer installed by running `npm uninstall @powersync/better-sqlite3`
**Common Installation Issues**
The `better-sqlite3` package requires native compilation, which depends on certain system tools.
Prebuilt assets are available and used by default, but a custom compilation may be started depending on the Node.js
or Electron version used.
This compilation process is handled by `node-gyp` and may fail if required dependencies are missing or misconfigured.
Refer to the [PowerSync Node package README](https://www.npmjs.com/package/@powersync/node) for more details.
**Requirements**: The Capacitor SDK requires Capacitor 8 or later. Minimum platform versions are iOS 15.0 and Android API level 24. On iOS, both Swift Package Manager (SPM) and CocoaPods are supported.
If you are on Capacitor 6 or 7, follow the [Capacitor v8 upgrade guide](https://capacitorjs.com/docs/updating/8-0) before installing or upgrading `@powersync/capacitor`.
Add the [PowerSync Capacitor NPM package](https://www.npmjs.com/package/@powersync/capacitor) to your project:
```bash theme={null}
npm install @powersync/capacitor
```
```bash theme={null}
yarn add @powersync/capacitor
```
```bash theme={null}
pnpm install @powersync/capacitor
```
**Install Peer Dependencies**
You must also install the following peer dependencies:
```bash theme={null}
npm install @capacitor-community/sqlite @powersync/web
```
```bash theme={null}
yarn add @capacitor-community/sqlite @powersync/web
```
```bash theme={null}
pnpm install @capacitor-community/sqlite @powersync/web
```
After installing, sync your Capacitor project:
```bash theme={null}
npx cap sync
```
Add the [PowerSync Tauri NPM package](https://www.npmjs.com/package/@powersync/tauri-plugin) to your project:
```bash theme={null}
npm install @powersync/tauri-plugin
```
```bash theme={null}
yarn add @powersync/tauri-plugin
```
```bash theme={null}
pnpm install @powersync/tauri-plugin
```
Like all trusted PowerSync packages, the Tauri plugin is only available through the `@powersync/` scope on npm.
The PowerSync Tauri plugin **cannot** be installed with the `tauri add` command.
Additionally, add the Tauri plugin crate to your Rust app (in the `src-tauri` directory):
```bash theme={null}
cargo add tauri-plugin-powersync
```
In your `lib.rs`, ensure the plugin is loaded:
```diff theme={null}
pub fn run() {
tauri::Builder::default()
.invoke_handler(tauri::generate_handler![connect])
+ .plugin(tauri_plugin_powersync::init())
.run(tauri::generate_context!())
.expect("error while running tauri application");
}
```
In `src-tauri/capabilities/default.json`, ensure `powersync:default` is listed under `permissions` to make PowerSync APIs available to JavaScript.
Add the [PowerSync pub.dev package](https://pub.dev/packages/powersync) to your project:
```bash theme={null}
dart pub add powersync
```
Add the [PowerSync SDK](https://central.sonatype.com/artifact/com.powersync/core) to your project by adding the following to your `build.gradle.kts` file:
```toml gradle/libs.versions.toml theme={null}
[versions]
# Please check the latest version at https://github.com/powersync-ja/powersync-kotlin/releases/
powersync = "1.12.0"
[libraries]
powersync-core = { module = "com.powersync:core", version.ref = "powersync" }
powersync-integration-supabase = { module = "com.powersync:connector-supabase", version.ref = "powersync" }
```
```Kotlin build.gradle.kts icon="https://mintcdn.com/powersync/GTJdSKFSfUc2Sxtc/logo/gradle.svg?fit=max&auto=format&n=GTJdSKFSfUc2Sxtc&q=85&s=bb14bd89bac7520f103a2ad2abc17053" theme={null}
kotlin {
//...
sourceSets {
commonMain.dependencies {
implementation(libs.powersync.core)
// If you want to use the Supabase Connector, also add the following:
implementation(libs.powersync.integration.supabase)
}
//...
}
}
```
```Kotlin build.gradle.kts icon="https://mintcdn.com/powersync/GTJdSKFSfUc2Sxtc/logo/gradle.svg?fit=max&auto=format&n=GTJdSKFSfUc2Sxtc&q=85&s=bb14bd89bac7520f103a2ad2abc17053" theme={null}
kotlin {
//...
sourceSets {
commonMain.dependencies {
implementation("com.powersync:core:$powersyncVersion")
// If you want to use the Supabase Connector, also add the following:
implementation("com.powersync:connector-supabase:$powersyncVersion")
}
//...
}
}
```
On Kotlin SDK v1.12.0 and later, the [PowerSync SQLite core extension](https://github.com/powersync-ja/powersync-sqlite-core) is statically linked into `com.powersync:core` for Apple targets (iOS, macOS, tvOS, and watchOS), consistent with Android and JVM. Use the Gradle dependencies above only. When you upgrade from an older SDK, remove any Swift package dependency on [`powersync-sqlite-core-swift`](https://github.com/powersync-ja/powersync-sqlite-core-swift) and any `powersync-sqlite-core` CocoaPod from your Xcode or CocoaPods setup.
You can add the PowerSync Swift package to your project using either `Package.swift` or Xcode:
```swift theme={null}
let package = Package(
//...
dependencies: [
//...
.package(
url: "https://github.com/powersync-ja/powersync-swift",
exact: ""
),
],
targets: [
.target(
name: "YourTargetName",
dependencies: [
.product(
name: "PowerSync",
package: "powersync-swift"
)
]
)
]
)
```
1. Follow [this guide](https://developer.apple.com/documentation/xcode/adding-package-dependencies-to-your-app#Add-a-package-dependency) to add a package to your project.
2. Use `https://github.com/powersync-ja/powersync-swift.git` as the URL
3. Include the exact version (e.g., `1.0.x`)
For desktop/server/binary use-cases and WPF, add the [`PowerSync.Common`](https://www.nuget.org/packages/PowerSync.Common/) NuGet package to your project:
```bash theme={null}
dotnet add package PowerSync.Common
```
For MAUI apps, add both [`PowerSync.Common`](https://www.nuget.org/packages/PowerSync.Common/) and [`PowerSync.Maui`](https://www.nuget.org/packages/PowerSync.Maui/) NuGet packages to your project:
```bash theme={null}
dotnet add package PowerSync.Common
dotnet add package PowerSync.Maui
```
To install a specific version, use `--version` instead: `dotnet add package PowerSync.Common --version `
Add the [PowerSync SDK](https://central.sonatype.com/artifact/com.powersync/core) to your project by adding the following to your `Cargo.toml` file:
```shell theme={null}
cargo add powersync
```
### Define Your Client-Side Schema
The client-side schema defines the tables and columns of the SQLite database that the PowerSync client SDK manages and that your app reads from and writes to. It is usually derived from your backend database schema and your [Sync Streams](/sync/streams/overview), and it can also include [local-only tables](/client-sdks/advanced/local-only-usage). You apply the schema when you instantiate the database in the next step.
Schema migrations are not required. The SDK syncs schemaless data and applies the schema to that data with SQLite views. The exception is [raw tables](/client-sdks/advanced/raw-tables), which you create and migrate yourself.
*PowerSync Cloud:* The easiest way to generate your schema is using the [PowerSync Dashboard](https://dashboard.powersync.com/). Click the **Connect** button in the top bar to generate the client-side schema based on your Sync Streams in your preferred language.
Here's an example schema for a simple `todos` table:
```typescript React Native (TS) theme={null}
import { column, Schema, Table } from '@powersync/react-native';
const todos = new Table(
{
list_id: column.text,
created_at: column.text,
completed_at: column.text,
description: column.text,
created_by: column.text,
completed_by: column.text,
completed: column.integer
},
{ indexes: { list: ['list_id'] } }
);
export const AppSchema = new Schema({
todos
});
```
```typescript Web & Capacitor (TS) theme={null}
import { column, Schema, Table } from '@powersync/web';
const todos = new Table(
{
list_id: column.text,
created_at: column.text,
completed_at: column.text,
description: column.text,
created_by: column.text,
completed_by: column.text,
completed: column.integer
},
{ indexes: { list: ['list_id'] } }
);
export const AppSchema = new Schema({
todos
});
```
```typescript Tauri (TS) theme={null}
import { column, Schema, Table } from '@powersync/common';
const todos = new Table(
{
list_id: column.text,
created_at: column.text,
completed_at: column.text,
description: column.text,
created_by: column.text,
completed_by: column.text,
completed: column.integer
},
{ indexes: { list: ['list_id'] } }
);
export const AppSchema = new Schema({
todos
});
```
```typescript Node.js (TS) theme={null}
import { column, Schema, Table } from '@powersync/node';
const todos = new Table(
{
list_id: column.text,
created_at: column.text,
completed_at: column.text,
description: column.text,
created_by: column.text,
completed_by: column.text,
completed: column.integer
},
{ indexes: { list: ['list_id'] } }
);
export const AppSchema = new Schema({
todos
});
```
```kotlin Kotlin theme={null}
import com.powersync.db.schema.Column
import com.powersync.db.schema.Schema
import com.powersync.db.schema.Table
import com.powersync.db.schema.Index
import com.powersync.db.schema.IndexedColumn
val AppSchema: Schema = Schema(
listOf(
Table(
name = "todos",
columns = listOf(
Column.text("list_id"),
Column.text("created_at"),
Column.text("completed_at"),
Column.text("description"),
Column.integer("completed"),
Column.text("created_by"),
Column.text("completed_by")
),
indexes = listOf(
Index("list", listOf(IndexedColumn.descending("list_id")))
)
)
)
)
```
```swift Swift theme={null}
import PowerSync
let todos = Table(
name: "todos",
columns: [
Column.text("list_id"),
Column.text("description"),
Column.integer("completed"),
Column.text("created_at"),
Column.text("completed_at"),
Column.text("created_by"),
Column.text("completed_by")
],
indexes: [
Index(
name: "list_id",
columns: [IndexedColumn.ascending("list_id")]
)
]
)
let AppSchema = Schema(todos)
```
```dart Dart/Flutter theme={null}
import 'package:powersync/powersync.dart';
const schema = Schema(([
Table('todos', [
Column.text('list_id'),
Column.text('created_at'),
Column.text('completed_at'),
Column.text('description'),
Column.integer('completed'),
Column.text('created_by'),
Column.text('completed_by'),
], indexes: [
Index('list', [IndexedColumn('list_id')])
])
]));
```
```csharp .NET theme={null}
using PowerSync.Common.DB.Schema;
using PowerSync.Common.DB.Schema.Attributes;
[Table("todos"), Index("list", ["list_id"])]
public class Todo
{
// Attribute-based schema requires an explicit id; other syntaxes define an implicit id key. Learn more in the .NET SDK reference.
[Column("id")]
public string TodoId { get; set; }
[Column("list_id")]
public string ListId { get; set; }
[Column("created_at")]
public string CreatedAt { get; set; }
[Column("completed_at")]
public string CompletedAt { get; set; }
[Column("description")]
public string Description { get; set; }
[Column("created_by")]
public string CreatedBy { get; set; }
[Column("completed_by")]
public string CompletedBy { get; set; }
[Column("completed")]
public bool Completed { get; set; }
}
public static Schema PowerSyncSchema = new Schema(typeof(Todo));
```
This uses the recommended attribute-based syntax, where your C# class doubles as both the schema definition and the result type for queries — so you only define your data structure once. If you prefer to keep your schema definition separate from your data classes, an object initializer syntax is also available. See the [.NET SDK reference](/client-sdks/reference/dotnet#schema-definition-syntax) for details.
```rust Rust theme={null}
use powersync::schema::{Column, Schema, Table};
pub fn app_schema() -> Schema {
let mut schema = Schema::default();
let todos = Table::create(
"todos",
vec![
Column::text("list_id"),
Column::text("created_at"),
Column::text("completed_at"),
Column::text("description"),
Column::integer("completed"),
Column::text("created_by"),
Column::text("completed_by"),
],
|_| {},
);
schema.tables.push(todos);
schema
}
```
**Note**: The schema does not explicitly specify an `id` column, since PowerSync automatically creates an `id` column of type `text`. PowerSync [recommends](/sync/advanced/client-id) using UUIDs.
**Learn More**
The client-side schema uses three column types: `text`, `integer`, and `real`. These map directly to values from your Sync Streams and are automatically cast if needed. For details on how backend database types map to SQLite types, see [Types](/sync/types).
### Instantiate the PowerSync Database
Now that you have your client-side schema defined, instantiate the PowerSync database in your app. This creates the client-side SQLite database that PowerSync keeps in sync with your source database based on your Sync Streams.
```typescript React Native (TS) theme={null}
import { PowerSyncDatabase } from '@powersync/react-native';
import { AppSchema } from './Schema';
export const db = new PowerSyncDatabase({
schema: AppSchema,
database: {
dbFilename: 'powersync.db'
}
});
```
```typescript Web (TS) theme={null}
import { PowerSyncDatabase } from '@powersync/web';
import { AppSchema } from './Schema';
export const db = new PowerSyncDatabase({
schema: AppSchema,
database: {
dbFilename: 'powersync.db'
}
});
```
```typescript Node.js (TS) theme={null}
import { PowerSyncDatabase } from '@powersync/node';
import { AppSchema } from './Schema';
export const db = new PowerSyncDatabase({
schema: AppSchema,
database: {
dbFilename: 'powersync.db'
}
});
```
```typescript Tauri (TS) theme={null}
import { PowerSyncTauriDatabase } from '@powersync/tauri-plugin';
import { appDataDir } from '@tauri-apps/api/path';
import { AppSchema } from './AppSchema';
export const db = new PowerSyncTauriDatabase({
schema: AppSchema,
database: {
dbFilename: 'powersync.db',
// Store the database in the app data directory
dbLocationAsync: appDataDir,
}
});
```
```typescript Capacitor (TS) theme={null}
import { PowerSyncDatabase } from '@powersync/capacitor';
// Import general components from the Web SDK package
import { Schema } from '@powersync/web';
import { Connector } from './Connector';
import { AppSchema } from './AppSchema';
/**
* The Capacitor PowerSyncDatabase will automatically detect the platform
* and use the appropriate database drivers.
*/
export const db = new PowerSyncDatabase({
// The schema you defined in the previous step
schema: AppSchema,
database: {
// Filename for the SQLite database — it's important to only instantiate one instance per file.
dbFilename: 'powersync.db'
}
});
```
```kotlin Kotlin theme={null}
import com.powersync.DatabaseDriverFactory
import com.powersync.PowerSyncDatabase
// Android
val driverFactory = DatabaseDriverFactory(this)
// iOS & Desktop
// val driverFactory = DatabaseDriverFactory()
val database = PowerSyncDatabase({
factory: driverFactory,
schema: AppSchema,
dbFilename: "powersync.db"
})
```
```swift Swift theme={null}
import PowerSync
let db = PowerSyncDatabase(
schema: AppSchema,
dbFilename: "powersync.sqlite"
)
```
```dart Dart/Flutter theme={null}
import 'package:powersync/powersync.dart';
import 'package:path_provider/path_provider.dart';
import 'package:path/path.dart';
openDatabase() async {
final dir = await getApplicationSupportDirectory();
final path = join(dir.path, 'powersync-dart.db');
db = PowerSyncDatabase(schema: schema, path: path);
await db.initialize();
}
```
```csharp .NET - Common theme={null}
using PowerSync.Common.Client;
class Demo
{
static async Task Main()
{
var db = new PowerSyncDatabase(new PowerSyncDatabaseOptions
{
Database = new SQLOpenOptions { DbFilename = "tododemo.db" },
Schema = AppSchema.PowerSyncSchema,
});
await db.Init();
}
}
```
```csharp .NET - MAUI theme={null}
using PowerSync.Common.Client;
using PowerSync.Common.MDSQLite;
using PowerSync.Maui.SQLite;
class Demo
{
static async Task Main()
{
// Ensures the DB file is stored in a platform appropriate location
var dbPath = Path.Combine(FileSystem.AppDataDirectory, "maui-example.db");
var factory = new MAUISQLiteDBOpenFactory(new MDSQLiteOpenFactoryOptions()
{
DbFilename = dbPath
});
var Db = new PowerSyncDatabase(new PowerSyncDatabaseOptions()
{
Database = factory, // Supply a factory
Schema = AppSchema.PowerSyncSchema,
});
await db.Init();
}
}
```
```rust Rust theme={null}
// 1. Process setup: register PowerSync extension early (e.g. in main()).
// 2. Open a connection pool, create env, then database. Spawn async tasks
// before connecting (see Connect step). Requires powersync with tokio feature.
use powersync::{ConnectionPool, PowerSyncDatabase, error::PowerSyncError};
use powersync::env::PowerSyncEnvironment;
use std::sync::Arc;
use http_client::IsahcClient;
fn open_pool() -> Result {
ConnectionPool::open("powersync.db")
}
// This example shows the Tokio runtime. You must call
// `PowerSyncEnvironment::powersync_auto_extension()` before using the SDK and spawn async
// tasks with `db.async_tasks().spawn_with_tokio()` (or `spawn_with` for other runtimes)
// before connecting. See the Rust SDK reference for in-memory pools, smol, or custom runtimes.
#[tokio::main]
async fn main() {
PowerSyncEnvironment::powersync_auto_extension()
.expect("could not load PowerSync core extension");
let pool = open_pool().expect("open pool");
let client = Arc::new(IsahcClient::new());
let env = PowerSyncEnvironment::custom(
client.clone(),
pool,
Box::new(PowerSyncEnvironment::tokio_timer()),
);
let db = PowerSyncDatabase::new(env, app_schema());
db.async_tasks().spawn_with_tokio();
// Connect with a backend connector in the next step.
}
```
### Connect to PowerSync Service Instance
Connect your client-side PowerSync database to the PowerSync Service instance you created in [step 2](#2-set-up-powersync-service-instance) by defining a *backend connector* and calling `connect()`. The backend connector handles authentication and uploading mutations to your backend.
This section assumes that you use PowerSync to sync your backend source database with SQLite in your app. To manage a local SQLite database without sync, instantiate the PowerSync database without calling `connect()` and see the [Local-Only](/client-sdks/advanced/local-only-usage) guide.
You don't have to worry about the *backend connector* implementation details right now. You can leave the boilerplate as-is and come back to it later.
For development, you can use the development token you generated in the [Generate a Development Token](#5-generate-a-development-token) step above. For production, you'll implement proper JWT authentication as explained further below.
```typescript React Native (TS) theme={null}
import type { CommonPowerSyncDatabase, PowerSyncBackendConnector, PowerSyncCredentials } from '@powersync/react-native';
import { db } from './Database';
class Connector implements PowerSyncBackendConnector {
async fetchCredentials(): Promise {
// for development: use development token
return {
endpoint: 'https://your-instance.powersync.com',
token: 'your-development-token-here'
};
}
async uploadData(database: CommonPowerSyncDatabase) {
const transaction = await database.getNextCrudTransaction();
if (!transaction) return;
for (const op of transaction.crud) {
const record = { ...op.opData, id: op.id };
// upload to your backend API
}
await transaction.complete();
}
}
// connect the database to PowerSync Service
const connector = new Connector();
await db.connect(connector);
```
```typescript Web & Capacitor (TS) theme={null}
import type { CommonPowerSyncDatabase, PowerSyncBackendConnector, PowerSyncCredentials } from '@powersync/web';
import { db } from './Database';
class Connector implements PowerSyncBackendConnector {
async fetchCredentials(): Promise {
// for development: use development token
return {
endpoint: 'https://your-instance.powersync.com',
token: 'your-development-token-here'
};
}
async uploadData(database: CommonPowerSyncDatabase) {
const transaction = await database.getNextCrudTransaction();
if (!transaction) return;
for (const op of transaction.crud) {
const record = { ...op.opData, id: op.id };
// upload to your backend API
}
await transaction.complete();
}
}
// connect the database to PowerSync Service
const connector = new Connector();
await db.connect(connector);
```
```typescript Node.js (TS) theme={null}
import { PowerSyncBackendConnector } from '@powersync/node';
export class Connector implements PowerSyncBackendConnector {
async fetchCredentials() {
// for development: use development token
return {
endpoint: 'https://your-instance.powersync.com',
token: 'your-development-token-here'
};
}
async uploadData(database) {
// upload to your backend API
}
}
// connect the database to PowerSync Service
const connector = new Connector();
await db.connect(connector);
```
```kotlin Kotlin theme={null}
import com.powersync.PowerSyncCredentials
import com.powersync.PowerSyncDatabase
class MyConnector : PowerSyncBackendConnector {
override suspend fun fetchCredentials(): PowerSyncCredentials {
// for development: use development token
return PowerSyncCredentials(
endpoint = "https://your-instance.powersync.com",
token = "your-development-token-here"
)
}
override suspend fun uploadData(database: PowerSyncDatabase) {
val transaction = database.getNextCrudTransaction() ?: return
for (op in transaction.crud) {
val record = op.opData + ("id" to op.id)
// upload to your backend API
}
transaction.complete()
}
}
// connect the database to PowerSync Service
database.connect(MyConnector())
```
```swift Swift theme={null}
import PowerSync
class Connector: PowerSyncBackendConnector {
func fetchCredentials() async throws -> PowerSyncCredentials {
// for development: use development token
return PowerSyncCredentials(
endpoint: "https://your-instance.powersync.com",
token: "your-development-token-here"
)
}
func uploadData(database: PowerSyncDatabase) async throws {
guard let transaction = try await database.getNextCrudTransaction() else {
return
}
for op in transaction.crud {
var record = op.opData
record["id"] = op.id
// upload to your backend API
}
try await transaction.complete()
}
}
// connect the database to PowerSync Service
let connector = Connector()
await db.connect(connector: connector)
```
```dart Dart/Flutter theme={null}
import 'package:powersync/powersync.dart';
class Connector extends PowerSyncBackendConnector {
@override
Future fetchCredentials() async {
return PowerSyncCredentials(
endpoint: 'https://your-instance.powersync.com',
token: 'your-development-token-here'
);
}
@override
Future uploadData(PowerSyncDatabase database) async {
final transaction = await database.getNextCrudTransaction();
if (transaction == null) return;
for (final op in transaction.crud) {
final record = {...op.opData, 'id': op.id};
// upload to your backend API
}
await transaction.complete();
}
}
// connect the database to PowerSync Service
final connector = Connector();
await db.connect(connector);
```
```csharp .NET theme={null}
using System;
using System.Collections.Generic;
using System.Net.Http;
using System.Text;
using System.Text.Json;
using System.Threading.Tasks;
using PowerSync.Common.Client;
using PowerSync.Common.Client.Connection;
using PowerSync.Common.DB.Crud;
public class MyConnector : IPowerSyncBackendConnector
{
public MyConnector()
{
}
public async Task FetchCredentials()
{
var powerSyncUrl = "https://your-instance.powersync.com";
var authToken = "your-development-token-here";
// Return credentials with PowerSync endpoint and JWT token
return new PowerSyncCredentials(powerSyncUrl, authToken);
}
public async Task UploadData(IPowerSyncDatabase database)
{
// upload to your backend API
}
}
// connect the database to PowerSync Service
await db.Connect(new MyConnector());
```
```rust Rust theme={null}
use async_trait::async_trait;
use powersync::{BackendConnector, PowerSyncCredentials, PowerSyncDatabase, SyncOptions};
use powersync::error::PowerSyncError;
use std::sync::Arc;
struct MyBackendConnector {
client: Arc,
db: PowerSyncDatabase,
}
#[async_trait]
impl BackendConnector for MyBackendConnector {
async fn fetch_credentials(&self) -> Result {
// for development: use development token
Ok(PowerSyncCredentials {
endpoint: "https://your-instance.powersync.com".to_string(),
token: "your-development-token-here".to_string(),
})
}
async fn upload_data(&self) -> Result<(), PowerSyncError> {
let mut local_writes = self.db.crud_transactions();
while let Some(tx) = local_writes.try_next().await? {
// upload to your backend API
tx.complete().await?;
}
Ok(())
}
}
// connect the database to PowerSync Service
db.connect(SyncOptions::new(MyBackendConnector {
client,
db: db.clone(),
}))
.await;
```
```rust Tauri (Rust) theme={null}
// For Tauri, connecting to PowerSync must be done in Rust so that sync state
// is shared across all windows. Calling connect() from JavaScript will throw.
//
// 1. Define your backend connector in Rust:
use async_trait::async_trait;
use tauri_plugin_powersync::PowerSyncExt;
use powersync::{BackendConnector, PowerSyncCredentials, PowerSyncDatabase, SyncOptions};
use powersync::error::PowerSyncError;
struct MyBackendConnector {
db: PowerSyncDatabase,
}
#[async_trait]
impl BackendConnector for MyBackendConnector {
async fn fetch_credentials(&self) -> Result {
// for development: use development token
Ok(PowerSyncCredentials {
endpoint: "https://your-instance.powersync.com".to_string(),
token: "your-development-token-here".to_string(),
})
}
async fn upload_data(&self) -> Result<(), PowerSyncError> {
let mut local_writes = self.db.crud_transactions();
while let Some(tx) = local_writes.try_next().await? {
// upload to your backend API
tx.complete().await?;
}
Ok(())
}
}
// 2. Add a Tauri command that receives the database handle from JavaScript:
#[tauri::command]
async fn connect(
app: tauri::AppHandle,
handle: usize,
) -> tauri_plugin_powersync::Result<()> {
let database = app.powersync().database_from_javascript_handle(handle)?;
let options = SyncOptions::new(MyBackendConnector { db: database.clone() });
database.connect(options).await;
Ok(())
}
// 3. Register the command in your Tauri builder:
// .invoke_handler(tauri::generate_handler![connect])
// 4. Then call it from JavaScript after initializing the database:
// await db.init();
// await invoke('connect', { handle: db.rustHandle });
```
Once connected, you can read from and write to the client-side SQLite database. PowerSync automatically syncs changes from your source database down into the SQLite database. For client-side mutations to be uploaded back to your source database, you need to complete the backend integration as explained below.
### Read Data
Read data using SQL queries. The data comes from your client-side SQLite database:
```typescript TypeScript theme={null}
// Get all todos
const todos = await db.getAll('SELECT * FROM todos');
// Get a single todo
const todo = await db.get('SELECT * FROM todos WHERE id = ?', [todoId]);
// Watch for changes (reactive query)
const stream = db.watch('SELECT * FROM todos WHERE list_id = ?', [listId]);
for await (const todos of stream) {
// Update UI when data changes
console.log(todos);
}
// Note: The above example requires async iterator support in React Native.
// If you encounter issues, use one of these callback-based APIs instead:
// Option 1: Using onResult callback
// const abortController = new AbortController();
// db.watch(
// 'SELECT * FROM todos WHERE list_id = ?',
// [listId],
// {
// onResult: (todos) => {
// // Update UI when data changes
// console.log(todos);
// }
// },
// { signal: abortController.signal }
// );
// Option 2: Using the query builder API
// const query = db
// .query({
// sql: 'SELECT * FROM todos WHERE list_id = ?',
// parameters: [listId]
// })
// .watch();
// query.registerListener({
// onData: (todos) => {
// // Update UI when data changes
// console.log(todos);
// }
// });
```
```kotlin Kotlin theme={null}
// Get all todos
val todos = database.getAll("SELECT * FROM todos") { cursor ->
Todo.fromCursor(cursor)
}
// Get a single todo
val todo = database.get("SELECT * FROM todos WHERE id = ?", listOf(todoId)) { cursor ->
Todo.fromCursor(cursor)
}
// Watch for changes
database.watch("SELECT * FROM todos WHERE list_id = ?", listOf(listId))
.collect { todos ->
// Update UI when data changes
}
```
```swift Swift theme={null}
// Get all todos
let todos = try await db.getAll(
sql: "SELECT * FROM todos",
mapper: { cursor in
TodoContent(
description: try cursor.getString(name: "description")!,
completed: try cursor.getBooleanOptional(name: "completed")
)
}
)
// Watch for changes
for try await todos in db.watch(
sql: "SELECT * FROM todos WHERE list_id = ?",
parameters: [listId]
) {
// Update UI when data changes
}
```
```dart Dart/Flutter theme={null}
// Get all todos
final todos = await db.getAll('SELECT * FROM todos');
// Get a single todo
final todo = await db.get('SELECT * FROM todos WHERE id = ?', [todoId]);
// Watch for changes
db.watch('SELECT * FROM todos WHERE list_id = ?', [listId])
.listen((todos) {
// Update UI when data changes
});
```
```csharp .NET theme={null}
// Define a result type with properties matching schema columns (some columns omitted for brevity)
// public class ListResult { public string id; public string name; public string owner_id; public string created_at; ... }
// Use db.Get() to fetch a single row:
var list = await db.Get("SELECT * FROM lists WHERE id = ?", [listId]);
// Use db.GetAll() to fetch all rows:
var lists = await db.GetAll("SELECT * FROM lists");
// Watch for changes to query results:
// Register listener synchronously on the calling thread...
var cts = new CancellationTokenSource();
var listener = db.Watch("SELECT * FROM lists", null, new SQLWatchOptions { Signal = cts.Token });
// ...then listen to changes on another thread (or await foreach directly if already in an async context)
_ = Task.Run(async () =>
{
try
{
await foreach (var results in listener)
{
Console.WriteLine($"Lists updated: {results.Length} items");
}
}
catch (Exception e)
{
Console.WriteLine($"Error: {e.Message}");
}
}, cts.Token);
// To stop watching, cancel the token:
cts.Cancel();
```
```rust Rust theme={null}
use rusqlite::params;
use futures::StreamExt; // for try_next() on the watch stream
// Get all todos
async fn get_all_todos(db: &PowerSyncDatabase) -> Result<(), PowerSyncError> {
let reader = db.reader().await?;
let mut stmt = reader.prepare("SELECT * FROM todos")?;
let mut rows = stmt.query(params![])?;
while let Some(row) = rows.next()? {
let id: String = row.get("id")?;
let description: String = row.get("description")?;
// use row data
}
Ok(())
}
// Get a single todo
async fn find_todo(db: &PowerSyncDatabase, todo_id: &str) -> Result<(), PowerSyncError> {
let reader = db.reader().await?;
let mut stmt = reader.prepare("SELECT * FROM todos WHERE id = ?")?;
let mut rows = stmt.query(params![todo_id])?;
while let Some(row) = rows.next()? {
let id: String = row.get("id")?;
let description: String = row.get("description")?;
println!("Found todo: {id}, {description}");
}
Ok(())
}
// Watch for changes
async fn watch_todos(db: &PowerSyncDatabase, list_id: &str) -> Result<(), PowerSyncError> {
let stream = db.watch_statement(
"SELECT * FROM todos WHERE list_id = ?".to_string(),
params![list_id],
|stmt, params| {
let mut rows = stmt.query(params)?;
let mut mapped = vec![];
while let Some(row) = rows.next()? {
mapped.push(() /* TODO: Read row into struct */);
}
Ok(mapped)
},
);
let mut stream = std::pin::pin!(stream);
while let Some(_event) = stream.try_next().await? {
// Update UI when data changes
}
Ok(())
}
```
**Learn More**
* [Reading Data](/client-sdks/reading-data) - Details on querying synced data
* [ORMs Overview](/client-sdks/orms/overview) - Using type-safe ORMs with PowerSync
* [Live Queries / Watch Queries](/client-sdks/watch-queries) - Building reactive UIs with automatic updates
### Write Data
Write data using SQL `INSERT`, `UPDATE`, or `DELETE` statements. PowerSync automatically queues these mutations and uploads them to your backend via the `uploadData()` function, once you've fully implemented your *backend connector* (as explained below).
```typescript TypeScript theme={null}
// Insert a new todo
await db.execute(
'INSERT INTO todos (id, created_at, list_id, description) VALUES (uuid(), date(), ?, ?)',
[listId, 'Buy groceries']
);
// Update a todo
await db.execute(
'UPDATE todos SET completed = 1, completed_at = date() WHERE id = ?',
[todoId]
);
// Delete a todo
await db.execute('DELETE FROM todos WHERE id = ?', [todoId]);
```
```kotlin Kotlin theme={null}
// Insert a new todo
database.writeTransaction { ctx ->
ctx.execute(
sql = "INSERT INTO todos (id, created_at, list_id, description) VALUES (uuid(), date(), ?, ?)",
parameters = listOf(listId, "Buy groceries")
)
}
// Update a todo
database.execute(
sql = "UPDATE todos SET completed = 1, completed_at = date() WHERE id = ?",
parameters = listOf(todoId)
)
// Delete a todo
database.execute(
sql = "DELETE FROM todos WHERE id = ?",
parameters = listOf(todoId)
)
```
```swift Swift theme={null}
// Insert a new todo
try await db.execute(
sql: "INSERT INTO todos (id, created_at, list_id, description) VALUES (uuid(), date(), ?, ?)",
parameters: [listId, "Buy groceries"]
)
// Update a todo
try await db.execute(
sql: "UPDATE todos SET completed = 1, completed_at = date() WHERE id = ?",
parameters: [todoId]
)
// Delete a todo
try await db.execute(
sql: "DELETE FROM todos WHERE id = ?",
parameters: [todoId]
)
```
```dart Dart/Flutter theme={null}
// Insert a new todo
await db.execute(
'INSERT INTO todos (id, created_at, list_id, description) VALUES (uuid(), date(), ?, ?)',
[listId, 'Buy groceries']
);
// Update a todo
await db.execute(
'UPDATE todos SET completed = 1, completed_at = date() WHERE id = ?',
[todoId]
);
// Delete a todo
await db.execute('DELETE FROM todos WHERE id = ?', [todoId]);
```
```csharp .NET theme={null}
// Insert a new todo
await db.Execute(
"INSERT INTO todos (id, created_at, list_id, description) VALUES (uuid(), datetime(), ?, ?)",
[listId, "Buy groceries"]
);
// Update a todo
await db.Execute(
"UPDATE todos SET completed = 1, completed_at = datetime() WHERE id = ?",
[todoId]
);
// Delete a todo
await db.Execute("DELETE FROM todos WHERE id = ?", [todoId]);
```
```rust Rust theme={null}
use rusqlite::params;
// Insert a new todo
async fn insert_todo(
db: &PowerSyncDatabase,
list_id: &str,
description: &str,
) -> Result<(), PowerSyncError> {
let writer = db.writer().await?;
writer.execute(
"INSERT INTO todos (id, created_at, list_id, description) VALUES (uuid(), date(), ?, ?)",
params![list_id, description],
)?;
Ok(())
}
// Update a todo
async fn complete_todo(db: &PowerSyncDatabase, todo_id: &str) -> Result<(), PowerSyncError> {
let writer = db.writer().await?;
writer.execute(
"UPDATE todos SET completed = 1, completed_at = date() WHERE id = ?",
params![todo_id],
)?;
Ok(())
}
// Delete a todo
async fn delete_todo(db: &PowerSyncDatabase, todo_id: &str) -> Result<(), PowerSyncError> {
let writer = db.writer().await?;
writer.execute("DELETE FROM todos WHERE id = ?", params![todo_id])?;
Ok(())
}
```
**Best practice**: Use UUIDs when inserting new rows on the client side. UUIDs can be generated offline/locally, allowing for unique identification of records created in the client database before they are synced to the server. See [Client ID](/sync/advanced/client-id) for more details.
**Learn More**
For more details, see the [Writing Data](/client-sdks/writing-data) page.
## Next Steps
For production deployments, you'll need to:
1. **[Implement Authentication](/configuration/auth/overview)**: Replace development tokens with proper JWT-based authentication. PowerSync supports various authentication providers including Supabase, Firebase Auth, Auth0, Clerk, and custom JWT implementations.
2. **Configure & Integrate Your Backend Application**: Set up your backend to handle mutations uploaded from clients.
* [Server-Side Setup](/configuration/app-backend/setup)
* [Client-Side Integration](/configuration/app-backend/client-side-integration)
### Additional Resources
* Learn more about [Sync Streams](/sync/streams/overview) for controlling partial syncing.
* Explore [Live Queries / Watch Queries](/client-sdks/watch-queries) for reactive UI updates.
* Check out [Example Projects](/intro/examples) for complete implementations.
* Review the [Client SDK References](/client-sdks/overview) for client-side platform-specific details.
## Questions?
Try "Ask AI" on this site which is trained on all our documentation, repositories and Discord discussions. Also join us on [our community Discord server](https://discord.gg/powersync) where you can browse topics from the PowerSync community and chat with our team.
# Understanding the SQLite Database
Source: https://docs.powersync.com/maintenance-ops/client-database-diagnostics
Analyze and debug the local PowerSync SQLite database on the client.
## Get the SQLite File
A SQLite database file can use any extension - .db, .sqlite, .sqlite3, etc. The extension doesn’t affect functionality; all contain the same SQLite format. To ensure no recent changes are lost, you can either: Pull the associated [Write-Ahead Log (WAL)](https://www.sqlite.org/wal.html) file (with the same name as the database plus the suffix `-wal`). Or alternatively, run the command `PRAGMA wal_checkpoint(TRUNCATE);` on the database before pulling it, which will merge the WAL file's changes into the main database file.
Ensure your emulator is running, then replace `com.package-name` with your application's package name and `your-db-name.sqlite` with your database file name.
This method may not work on Windows. Alternatively, you can copy the database to `/sdcard/` then use `adb pull`, though this may encounter permission issues on some devices.
```shell theme={null}
adb exec-out run-as com.package-name cat databases/your-db-name.sqlite > "your/local/path/your-db-name.sqlite"
adb exec-out run-as com.package-name cat databases/your-db-name.sqlite-wal > "your/local/path/your-db-name.sqlite-wal"
```
**Common database locations:**
* React Native (old directory used when the app was originally installed on a 1.x SDK version): `/data/data/com.package-name/files/`.
* React Native (new directory): `/data/data/com.package-name/databases/`.
**Note:** If the database is in a different location, first find it with:
```shell theme={null}
adb shell run-as com.package-name find /data/data/com.package-name -name "your-db-name.sqlite"
```
Replace `your-db-name.sqlite` with your database file name and extension.
```shell theme={null}
find ~/Library/Developer/CoreSimulator/Devices -type f -name 'your-db-name.sqlite'
find ~/Library/Developer/CoreSimulator/Devices -type f -name 'your-db-name.sqlite-wal'
```
**Common database location:**
* App sandbox: `Library/Application Support/`
Write-Ahead Log (WAL) file is not used in web environments. Browser-based SQLite implementations handle transactions differently.
Web applications use browser-based storage APIs. Database files are managed by the browser and not directly accessible via filesystem paths.
**Storage options:**
* **OPFS (Origin Private File System)**: Native filesystem API with better performance (Chrome 102+, Firefox 111+, Safari 17.2+)
* **IndexedDB**: A key-value storage API. Unlike OPFS, IndexedDB doesn't store complete database files - it stores data in a structured format that cannot be directly downloaded as a SQLite file.
Run the JavaScript code in your browser's console (F12 → Console) while on your application's page.
**Export database to your computer (OPFS only):**
```javascript theme={null}
// For OPFS
async function downloadDatabase() {
const root = await navigator.storage.getDirectory();
const fileHandle = await root.getFileHandle('your-db-name.sqlite');
const file = await fileHandle.getFile();
// Download the file
const url = URL.createObjectURL(file);
const a = document.createElement('a');
a.href = url;
a.download = 'your-db-name.sqlite';
a.click();
URL.revokeObjectURL(url);
}
downloadDatabase();
```
**Browser DevTools (inspect only):**
* Chrome/Edge: `F12` → Application → Storage → IndexedDB or OPFS
* Firefox: `F12` → Storage → IndexedDB
* Safari: Develop → Show Web Inspector → Storage
## Inspecting the SQLite File
### 1. Open Your SQLite File
Use the `sqlite3` command-line tool or a GUI tool like [DB Browser for SQLite](https://sqlitebrowser.org/) to open your database file:
```shell theme={null}
sqlite3 your-db-name.sqlite
```
### 2. Merge the WAL File
Temporary changes are stored in a separate [Write-Ahead Log (WAL)](https://www.sqlite.org/wal.html) `.wal` file. To measure the database size accurately, merge these changes into the main database:
```sql theme={null}
PRAGMA wal_checkpoint(TRUNCATE);
```
### 3. Get Storage Statistics
Query the built-in `dbstat` virtual table to see how much space each table uses on disk:
```sql theme={null}
SELECT name, pgsize AS storage_size, payload AS data_size
FROM dbstat
WHERE aggregate = true;
```
This returns:
* `name`: Table name
* `storage_size`: Total storage used on disk (in bytes, including SQLite overhead)
* `payload`: Actual data size (in bytes)
The `dbstat` table is automatically available in SQLite and provides low-level information about physical storage. Values represent on-disk usage including SQLite's internal structures (page headers, B-trees, indexes, free space), which is why they're larger than your logical data size.
## Understanding the Size Breakdown
PowerSync databases contain more data than just your application tables to support the sync functionality:
1. **Application data**: Your synced data in `ps_data__` tables
2. **Operation log (`ps_oplog`)**: A complete copy of all synced data required for offline conflict resolution and sync
3. **Indexes**: For efficient queries and lookups
4. **PowerSync metadata**: System tables and views for managing sync state (see [Client Architecture](/architecture/client-architecture#client-side-schema-and-sqlite-database-structure))
5. **SQLite overhead**: Page structure, alignment, fragmentation, and internal bookkeeping
The difference between `storage_size` and `payload` in the `dbstat` results shows SQLite's storage overhead. The `ps_oplog` table will typically be one of the largest tables since it maintains a full copy of your synced data.
To see just the JSON data size in `ps_oplog` (excluding SQLite overhead), run:
```sql theme={null}
SELECT sum(length(data)) / 1024.0 / 1024.0 AS size_mb FROM ps_oplog;
```
This measures only the raw JSON payloads, which will be smaller than the on-disk storage reported by `dbstat`.
### Finding Duplicate Rows
Rows can appear in multiple buckets. To find duplicated rows and their total size:
```sql theme={null}
SELECT
row_type,
count(*) AS duplicated_rows,
json_group_array(distinct ps_buckets.name) as buckets,
sum(data_size) as size_of_duplicates
FROM (
SELECT row_type, row_id, bucket, length(data) as data_size
FROM ps_oplog
GROUP BY row_type, row_id
HAVING count(*) > 1
)
JOIN ps_buckets ON bucket = ps_buckets.id
GROUP BY row_type
ORDER BY row_type;
```
This shows which tables have rows appearing in more than one bucket, which buckets contain them, and how much space the duplicates consume.
## Reducing SQLite File Size
Both methods of reducing the size of the SQLite file can be executed within the client using `powerSync.execute()`.
Consider these optimizations if your app's database is growing larger than expected or you're working with high data volumes in production.
### VACUUM Command
The `VACUUM` command reclaims unused space in the database:
```sql theme={null}
VACUUM;
```
The `VACUUM` command has important constraints:
* **Disk space**: Requires enough free disk space to create a temporary copy of the entire database
* **Database locking**: Locks the database during execution, which may affect app responsiveness
* Ensure sufficient space is available and run during low-activity periods
### Increase Page Size
Increasing the page size from the default **4KB** (4096 bytes) to **16KB** (16384 bytes) can reduce storage overhead significantly.
**IndexedDB Compatibility Issue**: Changing the page size is *not* supported when using `IndexedDB` on web platforms and could corrupt the database. Only use this optimization for native SQLite implementations.
**Additional caveats:**
* May increase overhead for many small writes.
* Best suited for apps with larger data records
The page size must be set before any tables are created and before running `VACUUM`. It should be one of the first **PRAGMA** statements after opening a new database:
```sql theme={null}
PRAGMA page_size = 16384;
```
If you're changing the page size on an existing database, you must run `VACUUM` immediately after setting it to apply the change. For optimal results, set the page size when first creating the database.
# Compacting Buckets
Source: https://docs.powersync.com/maintenance-ops/compacting-buckets
Reduce storage and sync overhead by compacting and defragmenting data.
This allows clients to download incremental changes efficiently — only changed rows have to be downloaded. However, over time this history can grow large, causing new clients to potentially take a long time to download the initial set of data. To handle this, we compact the history of each bucket.
## Compacting
### PowerSync Cloud
The cloud-hosted version of PowerSync will automatically compact all buckets once per day.
You can manually trigger compacting from the [PowerSync Dashboard](https://dashboard.powersync.com/) or the [CLI](/tools/cli):
* **Dashboard**: Select your project and instance, go to the **Settings** view, and click the **Compact** button in the "Compact operation history" section.
* **CLI**: Run `powersync compact` against a linked Cloud instance. The CLI polls until the operation completes, with a default timeout of 30 minutes. Pass `--timeout=` to override, or `--timeout=0` to wait indefinitely.
When using PowerSync Cloud, compact logs are available in the Dashboard's **Logs** view. See [Instance Logs](/maintenance-ops/monitoring-and-alerting#instance-logs).
[Defragmenting](/maintenance-ops/compacting-buckets#defragmenting) may still be required.
### Self-Hosted PowerSync
For self-hosted setups (PowerSync Open Edition & PowerSync Enterprise Self-Hosted Edition), the `compact` command in the Docker image can be used to compact all buckets. This can be run manually, or on a regular schedule using Kubernetes [CronJob](https://kubernetes.io/docs/concepts/workloads/controllers/cron-jobs/) or similar scheduling functionality.
On [storage version 4](/sync/advanced/storage-version-4), each run decides per bucket whether to merge newly added chunks, which is fast, or to run a full compaction that reads the whole bucket. Most runs are fast, so you can schedule `compact` to run every hour.
[Defragmenting](/maintenance-ops/compacting-buckets#defragmenting) may still be required.
## Background
### Bucket Operations
Each bucket is an ordered list of `PUT`, `REMOVE`, `MOVE` and `CLEAR` operations. In normal operation, only `PUT` and `REMOVE` operations are created.
A simplified view of a bucket may look like this:
```bash theme={null}
(1, PUT, row1, )
(2, PUT, row2, )
(3, PUT, row1, )
(4, REMOVE, row2)
```
### Compacting Step 1 - MOVE Operations
The first step of compacting involves `MOVE` operations. This just indicates that an operation is not needed anymore, since a later `PUT` or `REMOVE` operation replaces the row.
After this compact step, the bucket may look like this:
```bash theme={null}
(1, MOVE)
(2, MOVE)
(3, PUT, row1, )
(4, REMOVE, row2)
```
This does not reduce the number of operations to download, but can reduce the amount of data to download.
### Compacting Step 2 - CLEAR Operations
The second step of compacting takes a sequence of `CLEAR`, `MOVE` and/or `REMOVE` operations at the start of the bucket, and replaces them all with a single `CLEAR` operation. The `CLEAR` operation indicates to the client that "this is the start of the bucket, delete any prior operations that you may have".
After this compacting step, the bucket may look like this:
```bash theme={null}
(2, CLEAR)
(3, PUT, row1, )
(4, REMOVE, row2)
```
This reduces the number of operations for new clients to download in some cases.
The `CLEAR` operation can only remove operations at the start of the bucket, not in the middle of the bucket, which leads us to the next step.
### Defragmenting
There are cases that the above compacting steps cannot optimize efficiently. The key factor is that the oldest PUT operation in a bucket determines how much of the history can be compacted. This means:
1. If a row has never been updated since its initial creation, its original PUT operation remains at the start of the bucket
2. All operations that come after this oldest PUT cannot be fully compacted
3. This is particularly problematic when you have:
* A small number of rarely-changed rows in the same bucket as frequently-updated rows
* The rarely-changed rows' original PUT operations "block" compacting of the entire bucket
* The frequently-updated rows continue to accumulate operations that can't be fully compacted
For example, imagine this sequence of statements:
```sql theme={null}
-- Insert a single row that rarely changes
INSERT INTO lists(name) VALUES('a');
-- Insert 50k rows that change frequently
INSERT INTO lists (name) SELECT 'b' FROM generate_series(1, 50000);
-- Delete those 50k rows, but keep 'a'
DELETE FROM lists WHERE name = 'b';
```
After compacting, the bucket looks like this:
```bash theme={null}
(1, PUT, row_1, ) -- This original PUT blocks further compacting
(2, MOVE)
(3, MOVE)
...
(50001, MOVE)
(50002, REMOVE, row2)
(50003, REMOVE, row3)
...
(100001, REMOVE, row50000)
```
This is inefficient because:
1. The original PUT operation for row 'a' remains at the start
2. All subsequent operations can't be fully compacted
3. We end up with over 100k operations for what should be a simple bucket
To handle this case, we "defragment" the bucket by updating existing rows in the source database. This creates new PUT operations at the end of the bucket, allowing the compact steps to efficiently compact the entire history:
```sql theme={null}
-- Touch all rows to create new PUT operations
UPDATE lists SET name = name;
-- OR touch specific rows at the start of the bucket
UPDATE lists SET name = name WHERE name = 'a';
```
After defragmenting and compacting, the bucket looks like this:
```bash theme={null}
(100001, CLEAR)
(100002, PUT, row_1, )
```
The bucket is now back to two operations, allowing new clients to sync efficiently.
Note: All rows in the bucket must be updated for this to be effective. If some rows are never updated, they will continue to block compacting of the entire bucket.
**Bucket Design Tip**: If you have a mix of frequently-updated and rarely-changed rows, consider splitting them into separate buckets. This prevents the rarely-changed rows from blocking compacting of the frequently-updated ones.
### When to Defragment
You should consider defragmenting your buckets when:
1. **High Operations-to-Rows Ratio**: If you notice that the number of operations significantly exceeds the number of rows in a bucket. You can inspect this using the [Sync Diagnostics Client](https://github.com/powersync-ja/powersync-js/tree/main/tools/diagnostics-app).
2. **Frequent Updates**: Tables that are frequently updated (e.g., status fields, counters, or audit logs)
3. **Large Data Churn**: Tables where you frequently insert and delete many rows
### Defragmenting Strategies
There are two mechanisms with different scopes:
1. **Update source rows, then compact.** Use this when you can identify the rows whose old `PUT` operations block compacting. You can target one table or subset of rows, but each update creates a new operation that clients must download. For recurring maintenance, schedule the updates with a tool such as `pg_cron`:
```sql theme={null}
-- Daily defragmentation for high-churn tables
UPDATE audit_logs SET last_updated = now()
WHERE last_updated < now() - interval '1 day';
-- Weekly defragmentation for other tables
UPDATE users SET last_updated = now()
WHERE last_updated < now() - interval '1 week';
```
2. **Force a full reprocess.** In the [PowerSync Dashboard](https://dashboard.powersync.com/), select your project and instance, open **Settings**, and use **Defragment** in the "Compact operation history" section. This rebuilds all sync data, even when [incremental reprocessing](/sync/advanced/storage-version-4#incremental-reprocessing) would otherwise apply, and clients download all their data again. Use it for one-time cleanup of the entire Sync Config.
### Defragmenting Trade-offs
Both methods can significantly reduce the operation history, at the cost of existing clients downloading data again. Updating source rows and then compacting lets you control which rows are affected. The Dashboard action rebuilds and replaces the complete replication stream.
Key considerations:
1. **Frequency**: More frequent defragmentation means fewer operations per sync but more frequent re-syncs
2. **Scope**: Defragmenting all rows at once is more efficient but causes a larger sync cycle
3. **Monitoring**: Use the [Sync Diagnostics Client](https://github.com/powersync-ja/powersync-js/tree/main/tools/diagnostics-app) to track operations-to-rows ratio
## Sync Streams Deployments
Without incremental reprocessing, deploying [Sync Streams](/sync/streams/overview) recreates all buckets from scratch. This has the same effect as defragmenting and compacting every bucket, and clients download all their data again. With [incremental reprocessing](/sync/advanced/storage-version-4#incremental-reprocessing) (Beta), unchanged streams keep their buckets and operation history, so a deployment no longer clears their history.
Do not redeploy a Sync Config for routine history cleanup. Use compacting, targeted source-row updates, or the Dashboard's **Defragment** action instead.
## Technical Details
See the `powersync-service` repo documentation:
* [Compacting operations](https://github.com/powersync-ja/powersync-service/blob/main/docs/storage/compacting-operations.md) - How compacting rewrites operation history into `MOVE` and `CLEAR` operations. This applies to all storage versions.
* [Compaction design](https://github.com/powersync-ja/powersync-service/blob/main/docs/storage/v3-compaction-design.md) - How [storage version 4](/sync/advanced/storage-version-4) schedules chunk merges and full compactions per bucket.
# Deploying Schema Changes
Source: https://docs.powersync.com/maintenance-ops/deploying-schema-changes
Procedures for deploying schema changes in PowerSync.
The deploy process for schema or [Sync Streams](/sync/streams/overview) / [Sync Rules](/sync/rules/overview) updates depends on the type of change.
See the appropriate subsections below for details on the various scenarios.
Example: Add a new table that a new version of the app depends on, or add a new column to an existing table.
1. Apply source schema changes (i.e. in Postgres database) (often as a pre-deploy step as part of 2)
2. Deploy backend application changes
3. Deploy [Sync Streams](/sync/streams/overview) or [Sync Rules](/sync/rules/overview) changes
4. Wait for reprocessing to complete
5. Publish the app (may be deployed with delayed publishing at any prior point)
The approach here is to have the Sync Rules handle both the old and the new table name during the migration period.
Using maintenance mode on the backend here for simplicity. Other processes may be used to avoid maintenance mode, but that doesn't affect PowerSync system.
1. Deploy Sync Rules containing both the old and the new table name, with a mapping (alias) from the new name to the old one (so that both end up with the old name on the client). This will cause validation errors because of a missing table, but PowerSync will still allow the deploy.
2. Wait for Sync Rule reprocessing to complete.
3. Put the backend in maintenance mode.
1. i.e. Backend needs to be made unavailable to avoid breaking things during migrations.
4. Apply the source schema changes (i.e. in Postgres database)
5. Deploy backend changes and re-activate backend.
6. Remove the old table from Sync Rules, then deploy and activate the Sync Rules.
Define a versioned stream alongside the old one, or pass in a `schema_version` or similar parameter from the client and use it to choose either the old or new table name in the queries.
See this section for details on both approaches:
[Multiple Client Versions](/sync/advanced/multiple-client-versions)
Treat this as two separate steps and follow the process for both **Renaming a Table on the Server** and **Renaming a Table on the Client**.
Use the `ifnull` function in Sync Rules to output whichever column is available. This would handle both the old and new schema versions:
```sql theme={null}
SELECT IFNULL(description_new, description_old) AS description FROM assets
```
This may produce a validation error because of a missing column, but PowerSync will still allow the deploy.
Once the changes have been deployed and replicated, the old reference can be removed from the Sync Rules:
```sql theme={null}
SELECT description_new AS description FROM assets
```
Use the same approach as for renaming tables.
If the column types have the same representation in Sync Rules, the type can be changed freely without issues (for example changing between `VARCHAR` and `TEXT`).
Other type changes, for example changing between `INT` and `TEXT`, require more care.
To change the type, it is usually best to create a new column with the new type, then remove the old column once nothing uses it anymore.
When changing the type of a column without renaming, use a column type mapping to still use the old type for existing client applications.
# Implementing Schema Changes
Source: https://docs.powersync.com/maintenance-ops/implementing-schema-changes
How PowerSync handles schema changes across Postgres, MongoDB, MySQL, SQL Server, and Convex during replication.
## Introduction
The [PowerSync protocol](/architecture/powersync-protocol) is schemaless, and not directly affected by schema changes.
Replicating data from the source database to [buckets](/architecture/powersync-service#bucket-system) may be affected by server-side changes to the schema (in the case of Postgres), and may need [reprocessing](/maintenance-ops/compacting-buckets) in some cases.
The [client-side schema](/intro/setup-guide#define-your-client-side-schema) is just a view on top of the schemaless data. Updating this client-side schema is immediate when the new version of the app runs, with no client-side migrations required.
The developer is responsible for keeping client-side schema changes backwards-compatible with older versions of client apps. PowerSync has some functionality to assist with this:
1. [Versioned streams](/sync/advanced/multiple-client-versions) can serve different data to different client versions, either by defining a separate stream per version or by filtering on [connection parameters](/sync/streams/parameters#connection-parameters) such as client version. (In Sync Rules, this uses [client parameters](/sync/rules/client-parameters).)
2. Stream queries can apply simple data transformations to keep data in a format compatible with older clients, for example by aliasing or casting columns. (In Sync Rules, this is done via [data query expressions](/sync/rules/data-queries).)
## Client-Side Impact of Schema and Sync Config Changes
As mentioned above, the PowerSync system itself is schemaless — the client syncs any data as received, in JSON format, regardless of the data model on the client.
The schema as supplied on the client is only a view on top of the schemaless data.
1. If tables/collections not described by the client-side schema are synced, it is stored internally, but not accessible.
2. Same applies for columns/fields not described by the client-side schema.
3. When there is a type mismatch, SQLite's `CAST` functionality is used to cast to the type described by the schema.
1. Data is internally stored as JSON.
2. SQLite's `CAST` is used to cast values to `TEXT`, `INTEGER` or `REAL`.
3. Casting between types should never error, but it may not fully represent the original data. For example, casting an arbitrary string to `INTEGER` will likely result in a "0" value.
4. Full rules for casting between types are described [in the SQLite documentation here](https://www.sqlite.org/lang_expr.html#castexpr).
4. Removing a table/collection is handled on the client as if the table exists with no data.
5. Removing a column/field is handled on the client as if the values are `undefined`.
Nothing in PowerSync will fail hard if there are incompatible schema changes. But depending on how the app uses the data, app logic may break. For example, removing a table/collection that the app actively uses may break workflows in the app.
To avoid certain types of breaking changes on older clients, data transformations may be used — via column aliasing/casting in [Sync Streams](/sync/streams/queries#selecting-columns), or [data query expressions](/sync/rules/data-queries) in Sync Rules.
##
The Convex replicator is currently released as an [experimental feature](/resources/feature-status). APIs and
behavior may change, and we can't yet guarantee continued support or long-term stability.
For Convex, most schema changes are document-shape changes rather than DDL events. PowerSync reads the JSON documents returned by Convex snapshots and document deltas, not Convex schema metadata, so added fields and type changes are replicated through normal Convex writes.
Convex tables use `_id` as the replication identity for PowerSync. There is no source-specific primary key or replica identity definition to track.
Convex validates schema changes against existing data. If you change a field type, use [Convex's migration pattern](https://stack.convex.dev/intro-to-migrations): add or allow the new shape, update existing documents with mutations, then tighten the schema once the data has moved. Convex often recommends writing migrated values to a new field for this flow. Those mutation updates should appear in `document_deltas` and replicate like other writes.
You do not need to redeploy your Sync Config or re-snapshot merely because a field was added to Convex. New fields selected by your existing Sync Streams are replicated as they appear in Convex documents.
If you remove a field, first make it optional in your Convex schema, then run a migration that removes the field from existing documents, and only then remove it from the schema. PowerSync stops replicating new values for that field after the data stops containing it. Previously synced values can remain on clients until the affected data is reprocessed.
A re-snapshot is still required for cases that change the selected data set or invalidate the source cursor. This includes initial replication, a Sync Streams deployment that selects new existing data, restarting an incomplete initial snapshot, or recovering from a lost or expired Convex cursor.
### Dropping Tables
Dropping Convex tables has a known limitation. Deleting a table from the Convex dashboard does not emit per-document delete rows in `document_deltas`, so PowerSync does not automatically remove previously synced rows for that table.
To decommission a table while preserving replication correctness, clear the table before deleting it. In the Convex dashboard, use **Clear Table** first, then delete the table after those document removals have replicated. Deleting documents through Convex mutations is also valid when that path emits document delete deltas. Otherwise, treat dashboard table deletion or schema-only table removal as a Sync Config deployment change and clear or re-replicate affected PowerSync state.
## MySQL support is currently in a [Beta release](/resources/feature-status).
PowerSync keeps the [buckets](/architecture/powersync-service#bucket-system) up to date with any incremental data changes as recorded in the MySQL [binary log](https://dev.mysql.com/doc/refman/8.4/en/binary-log.html).
The binary log also provides DDL (Data Definition Language) query updates, which include:
1. Creating, dropping or renaming tables.
2. Truncating tables. (Not technically a schema change, but they appear in the query updates regardless.)
3. Changing replica identity of a table. (Creation, deletion or modification of primary keys, unique indexes, etc.)
4. Adding, dropping, renaming or changing the types of columns.
For MySQL, PowerSync detects schema changes by parsing the DDL queries in the binary log. It may not always be possible to parse the DDL queries correctly, especially if they are complex or use non-standard syntax.
In such cases, PowerSync will ignore the schema change, but will log a warning with the schema change query. If required, the schema change would then need to be manually
handled by redeploying your [Sync Streams](/sync/streams/overview) (or legacy [Sync Rules](/sync/rules/overview)). This triggers a re-replication.
### MySQL Schema Changes Affecting Sync Streams
#### DROP table
PowerSync will detect when a table is dropped, and automatically remove the data from the buckets.
#### CREATE table
Table creation is detected and handled the first time row events for the new table appear on the binary log.
#### TRUNCATE table
PowerSync will detect truncate statements in the binary log, and consequently remove all data from the buckets for that table.
#### RENAME table
A renamed table is handled similarly to dropping the old table, and then creating a new table with existing data under the new name.
This may be a slow operation if the table is large, since the "new" table has to be re-replicated. Replication will be blocked until the new table is replicated.
#### Change REPLICA IDENTITY
The replica identity of a table is considered to be changed if either:
1. The type of replica identity changes (`DEFAULT`, `INDEX`, `FULL`, `NOTHING`).
2. The name or type of columns which form part of the replica identity changes.
The latter can happen if:
1. Using `REPLICA IDENTITY FULL`, and any column is added, removed, renamed, or the type changed.
2. Using `REPLICA IDENTITY DEFAULT`, and the type of any column in the primary key is changed.
3. Using `REPLICA IDENTITY INDEX`, and the type of any column in the replica index is changed.
4. The primary key or replica index is removed or changed.
When the replication identity changes, the entire table is replicated again. This may be a slow operation if the table is large, and all other replication will be blocked until the table is replicated again.
Sync Streams/Sync Rules affected by schema changes will fail "soft" — an alert would be generated, but the system will continue processing changes.
#### Column Changes
Column changes such as adding, dropping, renaming columns, or changing column types, are detected by PowerSync but will generally not result in re-replication. (Unless the replica identity was affected as described above).
Adding a column with a `NULL` default value will generally not cause issues. Existing records will have a missing value instead of `NULL` value, but those are generally treated the same on the client.
Adding a column with a different default value, whether it's a static or computed value, will not have this default automatically replicated for existing rows. To propagate this value, make an update to every existing row.
Removing a column will not have the values automatically removed for existing rows on PowerSync. To propagate the change, make an update to every existing row.
Changing a column type, and/or changing the default value of a column using an `ALTER TABLE` statement, will not be automatically replicated to PowerSync.
In some cases, the change will have no effect on PowerSync (for example, changing between `VARCHAR` and `TEXT` types). When the values are expected to change, make an update to every existing row to propagate the changes.
##
SQL Server support is currently in a [Beta release](/resources/feature-status). The workflows below apply to PowerSync Service v1.25.0 or later. Earlier versions handled schema changes differently.
SQL Server Change Data Capture (CDC) does not include schema changes in the change stream. PowerSync can only discover them by polling database metadata, which gives no exact position in the change stream at which the schema changed.
PowerSync therefore does not adopt SQL Server schema changes automatically. Each [Sync Streams](/sync/streams/overview) deployment is processed as a replication stream that keeps the exact table set and CDC capture instances it selected when it started. To adopt a schema change, deploy your updated Sync Config: the new deployment reprocesses the data from the current schema while the previous deployment continues serving clients, and becomes active once reprocessing completes.
This model has two implications:
1. Each replicated table is pinned to a specific [capture instance](https://learn.microsoft.com/en-us/sql/relational-databases/track-changes/about-change-data-capture-sql-server?view=sql-server-ver17#capture-instance). When a newer capture instance appears, PowerSync logs a warning but continues reading from the pinned instance. If the pinned instance is removed, replication stops with [`PSYNC_S1601`](/debugging/error-codes#psync_s16xx-mssql-replication-issues).
2. Wildcard table names (`%`) are not supported for SQL Server. Every replicated table must be listed by name in your Sync Streams, must exist, and must have CDC enabled. A configured table that is unavailable or not CDC-enabled stops replication with [`PSYNC_S1602`](/debugging/error-codes#psync_s16xx-mssql-replication-issues) rather than being silently skipped.
### Capture Instances
SQL Server CDC is designed to protect downstream consumers from schema changes. Some schema changes, like changing the data type of a primary key column, are blocked at the database level while CDC is enabled on a table.
Other schema changes are allowed, but are not propagated to the capture instance for the table: the capture instance keeps the column set it was created with. To capture new or changed columns, create a new capture instance, or drop and recreate the existing one. Note that SQL Server allows a maximum of 2 capture instances per table.
#### Dropping and Recreating a Capture Instance
```sql theme={null}
-- Disable CDC for the source table
EXEC sys.sp_cdc_disable_table
@source_schema = N'dbo',
@source_name = N'',
@capture_instance = 'all'; -- This drops all capture instances for the source table. If you only want to drop a specific capture instance, use the capture instance name instead of 'all'.
-- Re-enable CDC for the source table
EXEC sys.sp_cdc_enable_table
@source_schema = N'dbo',
@source_name = N'',
@role_name = N'cdc_reader',
@supports_net_changes = 0;
```
#### Creating a New Capture Instance
```sql theme={null}
-- Create a new capture instance for the source table by specifying a new capture instance name
EXEC sys.sp_cdc_enable_table
@source_schema = N'dbo',
@source_name = N'',
@role_name = N'cdc_reader',
@supports_net_changes = 0,
@capture_instance = N''; -- If a capture instance for the table already exists, you have to specify a different name for the new capture instance.
```
### Making SQL Server Schema Changes
The workflows below keep the old table and capture instance available until the new Sync Config deployment has completed reprocessing and becomes active, so clients keep receiving updates during the transition whenever a rolling change is possible. If a pinned capture instance or replicated table is removed before then, replication stops with an error and stays stopped until an updated Sync Config is deployed.
#### Adding a Table
1. Create the table.
2. Enable CDC for the table.
3. Add the table to your Sync Streams and deploy.
The new deployment replicates the table's data before becoming active. If the Sync Config references a table before it exists with CDC enabled, replication stops with `PSYNC_S1602` and starts once the table is available.
#### Adding, Dropping, or Changing Columns
Column changes do not update an existing capture instance, and the replication stream keeps reading the column set captured by its pinned instance. PowerSync warns when it detects that the source table's schema differs from that capture instance, but continues replicating the pinned columns. No PowerSync action is needed if the affected columns do not need to be replicated.
To replicate the new column set:
1. Apply the column change in the source database.
2. Create a second capture instance for the table with the desired columns (see [Creating a New Capture Instance](#creating-a-new-capture-instance)).
3. Update your stream queries if needed and deploy.
4. Wait for reprocessing to complete and the new deployment to become active.
5. Remove the old capture instance.
#### Changing a Primary Key or Other Identity-Breaking Changes
Some changes are blocked at the database level while CDC is enabled on a table. These include column renames, changing the primary key, and changing the data type of a primary key column. They require disabling and re-enabling CDC for the table:
1. Disable CDC for the table.
2. Apply the primary key or identity change in the source database.
3. Re-enable CDC.
4. Update your stream queries if needed and deploy.
This cannot be done as a rolling change, so expect replication downtime until the new deployment becomes active. The active deployment stops with `PSYNC_S1601` when its capture instance is removed, or with `PSYNC_S1603` if the replica identity has already changed when it next checks the table. It cannot adopt the replacement capture instance. Note that disabling and re-enabling CDC stops replication with `PSYNC_S1601` even if the replica identity is unchanged.
#### Dropping a Table
1. Remove the table from your Sync Streams and deploy.
2. Wait for the new deployment to become active.
3. Drop the source table.
If the table is dropped first, replication stops with `PSYNC_S1603` because the table may still have unread changes. Already-replicated data is retained until the new deployment becomes active.
#### Renaming or Recreating a Table
Renaming a table and dropping and recreating it are handled the same way: update your stream queries to reference the intended table name, ensure the resulting table has CDC enabled with the expected schema and replica identity, and deploy. The active deployment stops with `PSYNC_S1603` when its replicated table is removed or recreated.
#### Changes That Require No Deployment
Changes that do not affect the captured columns or the replica identity of a table require no PowerSync action. This includes many index, constraint, and default-value changes, as well as changes to columns that are not replicated.
## See Also
* [JSON, Arrays and Custom Types](/client-sdks/advanced/custom-types-arrays-and-json)
* [Deploying Schema Changes](/maintenance-ops/deploying-schema-changes)
# Monitoring and Alerting
Source: https://docs.powersync.com/maintenance-ops/monitoring-and-alerting
Set up monitoring and alerting for PowerSync Cloud instances to track replication and sync health, connection status, and common errors.
You can monitor activity and alert on issues and usage for your PowerSync Cloud instance(s):
* **Monitor Usage**: View time-series and aggregated usage data with [Usage Metrics](#usage-metrics)
* **Monitor Service and Replication Activity**: Track your PowerSync Service and replication logs with [Instance Logs](#instance-logs)
* **Configure Alerts**: Set up alerts for connection or replication issues or usage activity \*
* Includes [Issue Alerts](#issue-alerts) and/or [Usage Alerts](#usage-alerts)
* **Alert Notifications**: Set up [Email notifications](#email-rules) or [Webhooks](#webhooks) to report events (like issue or usage alerts) to external systems \*
These features can assist with troubleshooting common issues (e.g. replication errors due to a logical replication slot problem), investigating usage spikes, or being notified when usage exceeds a specific threshold.
Investigating replication lag specifically? See [Replication Lag](/maintenance-ops/replication-lag) for what it is, how to monitor it, and common causes.
\* The availability of these features depends on your PowerSync Cloud plan. See the table below for a summary. More details are provided further below.
### Summary of Feature Availability (by PowerSync Cloud Plan)
Monitoring and alerting functionality varies by [PowerSync Cloud plan](https://www.powersync.com/pricing). This table provides a summary of availability:
| Feature | Free | Pro | Team & Enterprise |
| ------------------------ | ------------- | ------------------------ | ------------------------ |
| **Usage Metrics** | Available | Available | Available |
| **Instance Logs** | Available | Available | Available |
| **Log retention period** | 24 hours | 7 days | 30 days |
| **Issue Alerts** | Available | Available | Available |
| **Usage Alerts** | Not available | Not available | Available |
| **Alert Notifications** | - Email | - Email
**Self-hosting PowerSync**
Similar monitoring and alerting functionality is planned for PowerSync Open Edition users and Enterprise Self-Hosted customers.
For Open Edition users, alerting APIs are currently available in an early access release. For Enterprise Self-Hosted customers we are planning a full alerting service that includes customizable alerts and webhook integrations.
Until this is available, please chat to us on our [Discord](https://discord.gg/powersync) to discuss your use case or any questions.
## Usage Metrics
View time-series and aggregated usage data for your PowerSync instance(s), including storage size, concurrent clients, and synced data and operations. This data lets you monitor activity, spot patterns or spikes, and budget while tracking your position within our [Cloud pricing plans](https://www.powersync.com/pricing).
### View Usage Metrics
Access usage metrics in the [PowerSync Dashboard](https://dashboard.powersync.com/). Select your project and instance and go to the **Metrics** view:
This usage data is also available programmatically via APIs in an early access release. Chat to us on our [Discord](https://discord.gg/powersync) if you require details.
## Instance Logs
Use instance logs to investigate replication, client sync, and deployment issues. Choose a log type based on the activity you need to inspect:
* **Sync & API logs**: Client sync sessions and other API requests.
* **Replicator logs**: Data moving from your source database into the PowerSync Service.
* **Compact logs**: Automatic and manually triggered [compacting jobs](/maintenance-ops/compacting-buckets).
* **Migration logs**: Storage preparation during a deploy. Check these if a deploy fails at that step.
See the [Log Reference](/debugging/log-reference) for message meanings and structured fields. Log retention depends on your plan; see [Feature Availability](#summary-of-feature-availability-by-powersync-cloud-plan).
### View Instance Logs
Access instance logs through the [PowerSync Dashboard](https://dashboard.powersync.com/). Select your project and instance and go to the **Logs** view:
Log search is intended for ad-hoc investigation, not persistent monitoring or alerting. We may apply rate limits where sustained heavy usage occurs.
user_id: error:PSYNC_S2001
```
Quote values that contain spaces:
```
close_reason:"client closing stream"
```
The following aliases are supported for Sync & API logs. For most properties the alias matches the canonical property name; `error` and `lag` keep a short form because their underlying paths are awkward to type. You can also type the short aliases shown in parentheses; they are normalised to the canonical name on submit.
| Alias | Property | Description |
| ------------------------- | -------------- | ------------------------------------------------------------------------ |
| `user_id` (`user`) | `user_id` | The authenticated user ID for the sync session. |
| `client_id` (`client`) | `client_id` | The client device ID. |
| `rid` (`request`) | `rid` | The request ID. Matches a started or complete entry. |
| `error` | `error_code` | A specific PSYNC error code (see [Error Codes](/debugging/error-codes)). |
| `close_reason` (`reason`) | `close_reason` | Why the sync session closed. |
For Replication logs:
| Alias | Property | Description |
| ----- | ----------------- | ----------------------------------------------------------------------- |
| `lag` | `replication_lag` | Replication lag in seconds. Used with a numeric comparison (see below). |
#### Numeric Comparisons
The `lag` filter accepts `>`, `>=`, `<`, `<=`, and `=`:
```
lag:>=5
```
Only one numeric comparison is allowed per property. Combining `lag:>5 lag:<10` is rejected.
#### Excluding Results
Prefix a term or filter with `-` to exclude matching entries. This is useful for hiding known-noisy errors while triaging:
```
-error:PSYNC_S2106
```
Exclusion works with both syntaxes:
* `-alias:value` hides entries where the property matches the value. Repeating a negated alias hides every listed value: `-user_id:123 -user_id:456` hides both users.
* `-word` hides entries containing the text anywhere in the entry. Quote phrases: `-"token expired"`.
Only an unquoted `-` at the start of a term excludes. A hyphen inside a term matches literally (`powersync-js`), and quoting keeps a leading hyphen literal: `"-foo"` finds entries containing `-foo` rather than excluding `foo`.
Numeric comparisons **cannot** be negated. You should use the inverse comparison instead: `lag:<5` rather than `-lag:>=5`.
#### Filtering on Other Properties
The **Property filters** button in the toolbar lists the aliases for the active log type. Log entries can contain many additional properties, and you can filter on any of them using `alias:value` syntax directly in the search bar. For example:
```
buckets:1
```
Filters are evaluated against the matching property on each log entry.
You can also add a metadata property as a filter directly from any log entry. Enable the **Metadata** display option to render each entry's structured properties as pills. Click a pill to open its menu and choose **Add filter**:
This is the quickest way to filter on properties you spot in the logs without remembering their names.
#### Combined Example
You can mix free-text terms, known aliases, unknown properties, and exclusions in a single query:
```
checkpoint sync user_id:123 checkpoint:>=1224 -error:PSYNC_S2106
```
This matches entries containing both `checkpoint` and `sync` in the message, with `user_id` equal to `123` and a `checkpoint` property greater than or equal to `1224`, excluding entries with error code `PSYNC_S2106`.
### Export Logs
Click the **Download** button on the right side of the toolbar to export the currently visible log entries to a CSV file. The export reflects the active log type, time range, log levels, and search query, so you can narrow the result set before downloading.
The CSV uses fixed columns for the most useful fields and collapses anything else into a trailing `metadata` JSON column:
| Column | Description |
| -------------- | ---------------------------------------------------------------- |
| `timestamp` | ISO 8601 timestamp of the entry. |
| `level` | Log level (`info`, `error`, `warn`, `debug`). |
| `message` | Formatted log message. |
| `user_id` | Authenticated user ID (Sync & API logs). |
| `client_id` | Client device ID (Sync & API logs). |
| `rid` | Request ID (Sync & API logs). |
| `error_code` | PSYNC error code if the entry has one. |
| `close_reason` | Close reason for sync session entries. |
| `metadata` | JSON object containing every other top-level field on the entry. |
The file is named `powersync-logs--.csv` where `` is the export time in UTC.
The export only includes entries currently loaded in the view. If you have not scrolled to load more entries with **Load More**, those entries are not included in the CSV.
### Correlating Sync Sessions
To find a specific user's session in the Sync & API logs, search for their `user_id`:
```
user_id:
```
Find a `Sync stream started` or `Sync stream complete` entry and copy its `rid` (request ID). Search for that ID to follow the session:
```
rid:
```
Use the [Session Lifecycle](/debugging/log-reference#session-lifecycle) and [Session Fields](/debugging/log-reference#session-fields) reference to interpret the results.
### Replication Flush Entries
To investigate replication throughput, select Replicator logs and search for `Flushed`. Each entry reports a batch of writes to bucket storage. See [Flushed Batches](/debugging/log-reference#flushed-batches) for message and field meanings.
## Custom Metadata in Sync Logs
Custom metadata in sync logs allows clients to attach additional context to their PowerSync connection for improved observability and analytics. This metadata appears in the Sync & API logs, making it easier to track, debug, and analyze sync behavior across your app. For example, you can tag connections with app version, feature flags, or business context.
### How to Use Custom Metadata
You can specify application metadata when calling `PowerSyncDatabase.connect()`. To update the metadata, reconnect with new metadata values.
**Version compatibility**: This feature requires JavaScript/Web SDK v1.30.0+, React Native SDK v1.28.0+, Node.js SDK v0.15.0+, or Capacitor SDK v0.2.0+, and PowerSync Service v1.17.0+.
```javascript theme={null}
import { PowerSyncDatabase } from '@powersync/web'; // Update this to the appropriate SDK package
const powerSync = new PowerSyncDatabase({
schema: AppSchema,
database: {
dbFilename: 'powersync.db'
}
});
// Set custom metadata when connecting
powerSync.connect(connector, {
appMetadata: {
app_version: '1.2.3',
feature_flag: 'new_sync_flow'
}
});
```
**Version compatibility**: This feature requires Dart/Flutter SDK v1.17.0+ and PowerSync Service v1.17.0+.
```dart theme={null}
import 'package:powersync/powersync.dart';
final powerSync = PowerSyncDatabase(
schema: AppSchema,
path: 'powersync.db'
);
await powerSync.initialize();
// Set custom metadata when connecting
const options = SyncOptions(
appMetadata: {
'app_version': '1.2.3',
'feature_flag': 'new_sync_flow'
}
);
powerSync.connect(
connector: MyConnector(),
options: options
);
```
**Version compatibility**: This feature requires Kotlin SDK v1.10.0+ and PowerSync Service v1.17.0+.
```kotlin theme={null}
import com.powersync.DatabaseDriverFactory
import com.powersync.PowerSyncDatabase
// Android
val driverFactory = DatabaseDriverFactory(this)
// iOS & Desktop
// val driverFactory = DatabaseDriverFactory()
val powerSync = PowerSyncDatabase({
factory: driverFactory,
schema: AppSchema,
dbFilename: "powersync.db"
})
// Set custom metadata when connecting
powerSync.connect(
connector = MyConnector(),
appMetadata = mapOf(
"app_version" to "1.2.3",
"feature_flag" to "new_sync_flow"
)
)
```
**Version compatibility**: This feature requires Swift SDK v1.9.0+ and PowerSync Service v1.17.0+.
```swift theme={null}
import PowerSync
let schema = AppSchema
let connector = Connector() // This connector must conform to PowerSyncBackendConnector
let powerSync = PowerSyncDatabase(
schema: schema,
dbFilename: "powersync.db"
)
// Set custom metadata when connecting
try await powerSync.connect(
connector: connector,
options: ConnectOptions(
appMetadata: [
"app_version": "1.2.3",
"feature_flag": "new_sync_flow"
]
)
)
```
**Version compatibility**: This feature requires .NET SDK v0.0.6-alpha.1+ and PowerSync Service v1.17.0+.
```csharp theme={null}
using PowerSync.Common.Client;
using PowerSync.Common.Client.Sync.Stream;
var powerSync = new PowerSyncDatabase(new PowerSyncDatabaseOptions
{
Database = new SQLOpenOptions { DbFilename = "powersync.db" },
Schema = AppSchema.PowerSyncSchema,
});
await powerSync.Init();
// Set custom metadata when connecting
await powerSync.Connect(
connector: new MyConnector(),
options: new PowerSyncConnectionOptions
{
AppMetadata = new Dictionary
{
{ "app_version", "1.2.3" },
{ "feature_flag", "new_sync_flow" }
}
}
);
```
Example not yet available.
### View Custom Metadata in Logs
Custom metadata appears in the **Sync & API logs** section of the [PowerSync Dashboard](https://dashboard.powersync.com/). Navigate to your project and instance, then go to the **Logs** view. The metadata is included in **Sync stream started** and **Sync stream complete** log entries.
Make sure the **Metadata** checkbox is enabled in the logs view to see custom metadata in log entries.
To filter on a custom metadata key in the search bar, prefix it with `app_metadata_`. For example, for `{ region: 'us-east' }`:
```
app_metadata_region:us-east
```
The same prefix applies when clicking **Add filter** on a custom metadata pill.
Note the following when using custom metadata:
* Keep metadata values concise. `app_metadata` is limited to 20 keys, with each string value capped at 100 characters.
* Avoid including sensitive information in metadata as it will appear in logs.
* Metadata is set per connection. Reconnect with new metadata when user context or app state changes (e.g., feature flags).
## Issue Alerts
Issue alerts capture potential problems with your instance, such as connection or replication issues.
**Availability**
* Issue alerts are available on all Cloud plans.
### Configure Issue Alerts
Issue alerts are set up per instance. To set up a new alert, navigate to the **Alerts** section in the [PowerSync Dashboard](https://dashboard.powersync.com/) and click **Create Issue Alert**.
When creating or editing an issue alert, you can configure:
* **Alert Name**: Give your alert a descriptive name to help identify it
* **Issue Type**: Select the type of issue to monitor from the dropdown:
* **Database Connection Issue**: Trigger when there is a connection problem
* **Replication Issue**: Trigger when there is an issue with the replication process
* **Severity Levels**: Select which severity levels should trigger this alert:
* **Warning**: For non-critical issues
* **Fatal**: For critical issues that require immediate attention
**Important: Set Up Notification Rules**
Creating an issue alert only defines *what* to monitor. To actually receive notifications when alerts trigger, you must also set up [Email Rules](#email-rules) or [Webhooks](#webhooks) and configure them to notify for "Issue alert state change" events. See the [Alert Notifications](#alert-notifications) section below.
## Usage Alerts
Usage alerts trigger when specific usage metrics exceed a defined threshold. This helps with troubleshooting usage spikes, or unexpected usage activity.
**Availability**
Usage alerts are available on **Team** and **Enterprise** plans.
### Configure Usage Alerts
Usage alerts are set up per instance. Navigate to the **Alerts** section in the [PowerSync Dashboard](https://dashboard.powersync.com/) and click **Create Usage Alert**.
When creating or editing a usage alert, you can configure:
* **Alert Name**: Give your alert a descriptive name to help identify it
* **Metric**: Select from the following usage metrics to monitor:
* Data Synced
* Data Replicated
* Operations Synced
* Operations Replicated
* Peak Concurrent Clients
* Storage Size
These metrics correspond to the data shown in the [Usage Metrics](#view-usage-metrics) workspace and align with the PowerSync Service parameters outlined in our [pricing](https://www.powersync.com/pricing).
* **Window**: The number of minutes to look back when evaluating usage. All usage data points within this time window are included when determining if the configured threshold has been crossed
* **Aggregation**: Select how to aggregate all data points within the window before comparing to the threshold:
* **Avg**: Calculate the average of all values
* **Max**: Use the highest value
* **Min**: Use the lowest value
* **Condition**: Set whether the alert triggers when usage goes **Above** or **Below** the specified threshold
* **Threshold Value**: The numeric limit for the selected metric (in bytes for size-based metrics; count for all other metrics)
**Important: Set Up Notification Rules**
Creating a usage alert only defines *what* to monitor. To actually receive notifications when alerts trigger, you must also set up [Email Rules](#email-rules) or [Webhooks](#webhooks) and configure them to notify for "Usage alert state change" events. See the [Alert Notifications](#alert-notifications) section below.
## Alert Notifications
Set up notification rules to be informed of issue or usage alerts, as well as deploy state changes. PowerSync provides multiple notification methods that trigger both when an alert becomes active and when it returns to normal (indicating the monitored conditions are back within acceptable thresholds).
* **Email Rules**: Send alerts directly to your email address
* **Webhooks**: Notify external systems and services
**Availability**
* **Email Rules**: Available on all plans (**Free**, **Pro**, **Team** and **Enterprise**)
* **Webhooks**: Available on **Pro**, **Team** and **Enterprise** plans
### Email Rules
Email rules allow you to receive alerts directly to your email address when specific events occur in PowerSync.
#### Set Up Email Rules
Navigate to the **Alerts** section in the [PowerSync Dashboard](https://dashboard.powersync.com/) and scroll down to the **Notification Rules** section. Click **Create Email Rule** to set up email notifications.
Accounts on the Free plan are restricted to a single email rule; customers on paid plans can create an unlimited number of email rules.
When creating or editing an email rule, you can configure:
* **Recipient Email**: Specify the email address that will receive the notifications (required)
* **Event Triggers**: Select one or more of the following events to trigger the email notification:
* **Usage alert state change**: Fired when a usage alert changes between 'monitoring' and 'alerting' (a threshold has been crossed)
* **Issue alert state change**: Fired when an issue alert changes between 'monitoring' and 'alerting' (the instance has active issues)
* **Deploy state change**: Fired when an instance deploy starts, completes or fails. This includes deprovisioning an instance
* **Enabled**: Toggle to control whether the email rule is active
### Webhooks
Webhooks enable you to notify external systems when specific events occur in PowerSync.
#### Set Up Webhooks
Navigate to the **Alerts** section in the [PowerSync Dashboard](https://dashboard.powersync.com/) and scroll down to the **Notification Rules** section. Click **Create Webhook Rule** to set up webhook notifications.
When creating or editing a webhook rule, you can configure:
* **Webhook Endpoint (URL)**: Define the endpoint that will receive the webhook request (starting with `https://`) (required)
* **Event Triggers**: Select one or more of the following events to trigger the webhook:
* **Usage alert state change**: Fired when a usage alert changes between 'monitoring' and 'alerting' (a threshold has been crossed)
* **Issue alert state change**: Fired when an issue alert changes between 'monitoring' and 'alerting' (the instance has active issues)
* **Deploy state change**: Fired when an instance deploy starts, completes or fails. This includes deprovisioning an instance
* **Enabled**: Toggle to control whether the webhook rule is active
* **Retries**: Configure the number of retry attempts for failed webhook deliveries
After creating a webhook, a secret is automatically generated and copied to your clipboard. Store this secret since you'll need it to verify the webhook request signature.
### Webhook Signature Verification
Every webhook request contains an `x-journey-signature` header, which is a base64-encoded HMAC (Hash-based Message Authentication Code). To verify the request, you need to compute the HMAC using the shared secret that was generated when you created the webhook, and compare it to the value in the `x-journey-signature` header.
**JavaScript Example:**
```javascript theme={null}
import { createHmac } from 'crypto';
// Extract the signature from the request headers
const signature = request.header('x-journey-signature');
// Create an HMAC using your webhook secret and the request body
let verify = createHmac('sha256', '') // The secret provided during webhook setup
.update(Buffer.from(request.body, 'utf-8'))
.digest('base64');
// Compare the computed HMAC with the signature from the request
if (signature === verify) {
console.log("success");
} else {
console.log("verification failed");
}
```
# Production Readiness Best Practices Guide
Source: https://docs.powersync.com/maintenance-ops/production-readiness-guide
Best practices for running PowerSync reliably in production.
Here are the recommended items you should implement as part of supporting PowerSync in a production environment.
1. Client SDK Diagnostics - Implement a sync diagnostics screen/view in your client application that provides critical sync information.
2. Client logging - Implement logging in your client application to capture sync events and errors.
3. Issue Alerts - Trigger notifications when the PowerSync replicator runs into errors.
4. Database - Making sure your database is ready for production when integrated with PowerSync.
## Client specific
### SDK Diagnostics
It’s important to know what’s going on with a PowerSync enabled client application, this becomes useful during debugging issues with end users.
We recommend adding a view/screen in your application that offers diagnostic information about a client. Here you would want to add the following client specific information:
1. `connected` - Boolean; True if the client is connected to the PowerSync Service instance. False if not.
2. `connecting` - Boolean; True if the client is attempting to connect to the PowerSync Service instance. False if not.
3. `uploading` - Boolean; If the client has a network connection and changes in the upload queue are present this will be set to true when the client attempts to upload changes to the backend API in the `uploadData` function.
4. `downloading` - Boolean; If the client is connected to the PowerSync Service and new data is available, this will be set to true, else it will be false.
5. `hasSynced` - Boolean; True if the client completed a full sync at least once. False if the client never completed a full sync.
6. `lastSyncedAt` - DateTime; Timestamp of when the client last completed a full sync.
Each of the PowerSync Client SDKs have the `SyncStatus` class that can be used to access the fields mentioned above.
* [Flutter](https://pub.dev/documentation/powersync/latest/powersync/SyncStatus-class.html)
* [Kotlin](https://powersync-ja.github.io/powersync-kotlin/common/com.powersync.sync/-sync-status/index.html)
* [Swift](https://powersync-ja.github.io/powersync-swift/documentation/powersync/syncstatusdata)
* [JavaScript](https://powersync-ja.github.io/powersync-js/common/interfaces/SyncStatus) (re-exported from all SDK packages)
* [.NET (Beta)](https://github.com/powersync-ja/powersync-dotnet/blob/2728eab0d13849686ff3f9a603040940744599e1/PowerSync/PowerSync.Common/DB/Crud/SyncStatus.cs)
In addition to the `SyncStatus` options above, it's also a good idea to see what the current length of the upload queue looks like.
The upload queue contains all local mutations that need to be processed by the client specific `uploadData` implementation.
To get this information you can simply count the number of rows present in the internal `ps_crud` SQLite table e.g.
```sqlite theme={null}
SELECT COUNT(*) AS row_count FROM ps_crud;
```
If you're interested in learning more about the internal PowerSync SQLite schema, see the [Client Architecture](/architecture/client-architecture#client-side-schema-and-sqlite-database-structure) section of the docs.
### Client Logging
#### Using `Sentry logging` for Log Aggregation
This is just an example of how to implement Sentry logging. The actual implementation is up to you as the developer. You don't have to use `Sentry logging`, but we recommend using some sort of log aggregation service in production.
```typescript App Entry Point theme={null}
createRoot(document.getElementById("root")!,
{
onUncaughtError: Sentry.reactErrorHandler((error, errorInfo) => {
console.warn('Uncaught error', error, errorInfo.componentStack);
}),
// Callback called when React catches an error in an ErrorBoundary.
onCaughtError: Sentry.reactErrorHandler(),
// Callback called when React automatically recovers from errors.
onRecoverableError: Sentry.reactErrorHandler(),
}).render(
);
```
```typescript System.ts theme={null}
import * as Sentry from '@sentry/react';
import { createConsoleLogger, LogLevels } from '@powersync/react-native';
// Initialize Sentry
Sentry.init({
dsn: 'YOUR_SENTRY_DSN_HERE',
transport: Sentry.makeBrowserOfflineTransport(Sentry.makeFetchTransport), // Handle offline scenarios
enableLogs: true // Enable Sentry logging
});
const logger = createConsoleLogger({ minLevel: LogLevels.warn });
logger.setHandler((messages, context) => {
if (!context?.level) return;
// Get the main message and combine any additional data
const messageArray = Array.from(messages);
const mainMessage = String(messageArray[0] || 'Empty log message');
const extraData = messageArray.slice(1).reduce((acc, curr) => ({ ...acc, ...curr }), {});
const level = context.level.name.toLowerCase();
// Add breadcrumb: creates a trail of events leading up to errors
// This helps debug by showing PowerSync state/operations before crashes
// Breadcrumbs appear in Sentry error reports for context
// We capture all levels (including info/debug) since we might want to know
// what operations happened before an error occurred
Sentry.addBreadcrumb({
message: mainMessage,
level: level as Sentry.SeverityLevel,
data: extraData,
timestamp: Date.now()
});
// Only send warnings and errors to Sentry
if (level == 'warn' || level == 'error') {
console[level](`PowerSync ${level.toUpperCase()}:`, mainMessage, extraData);
Sentry.logger[level](mainMessage, extraData);
}
});
// Create PowerSync instance
export const powerSync = new PowerSyncDatabase({
schema: AppSchema,
database: {
dbFilename: 'example.db'
},
logger: logger // Pass the logger to PowerSync
});
// Register a listener to monitor PowerSync status changes and log upload/download errors that are not handled directly by the SDK
powerSync.registerListener({
statusChanged: (status) => {
// Check for download errors and log them with context
if(status.downloadError) {
logger.error("PowerSync sync download failed", {
userSession: connector.currentSession, // Current user session for tracking
lastSyncAt: status?.lastSyncedAt, // When the last successful sync occurred
connected: status?.connected, // Network connection status
sdkVersion: powerSync.sdkVersion || 'unknown', // PowerSync SDK version for debugging
downloadError: status.downloadError // The actual download error details
});
}
// Check for upload errors and log them with context
if(status.uploadError) {
logger.error("PowerSync sync upload failed", {
userSession: connector.currentSession, // Current user session for tracking
lastSyncAt: status?.lastSyncedAt, // When the last successful sync occurred
connected: status?.connected, // Network connection status
sdkVersion: powerSync.sdkVersion || 'unknown', // PowerSync SDK version for debugging
uploadError: status.uploadError // The actual upload error details
});
}
}
});
// Usage with additional context
logger.error('PowerSync sync failed', {
userId: userID,
lastSyncAt: status?.lastSyncedAt,
connected: status?.connected,
sdkVersion: powerSync.sdkVersion || 'unknown',
});
```
#### Best Practices
* **Log Level Management:** Use appropriate log levels `(WARN/ERROR)` in production
* **Structured Logging:** Include relevant context like user IDs, operation types, timestamps
* **Offline Resilience:** Always have a local fallback for critical logs
* **Performance:** Be mindful of log volume to avoid performance impacts
* **Privacy:** Ensure sensitive data is not logged or is properly sanitized
* **Retention:** Implement log rotation/cleanup for local storage to manage device storage (if applicable)
## Issue Alerts
### PowerSync Cloud
The PowerSync Cloud dashboard offers features and functionality that makes it easy to monitor the replication process from your source DB to your PowerSync Service instance and raise alerts when issues occur.
We highly recommend you read the sections below and configure alerts as suggested.
#### Replication Issue Alerts
At a minimum we recommend creating an issue alert for `Replication issues`. For details instructions on how to configure Issue Alerts, see the [Issue Alerts](/maintenance-ops/monitoring-and-alerting#issue-alerts) section of the [Monitoring and Alerting](/maintenance-ops/monitoring-and-alerting) docs.
Here's quick example of what the Issue alert should look like to catch replication issues:
For a broader overview of replication lag across source databases, how to monitor it, and common causes, see [Replication Lag](/maintenance-ops/replication-lag).
Because PowerSync relies on Postgres logical replication, it's important to consider the size of the `max_slot_wal_keep_size` and monitoring lag of replication slots used by PowerSync in a production environment to ensure lag of replication slots do not exceed the `max_slot_wal_keep_size`.
The `max_slot_wal_keep_size` Postgres [configuration parameter](https://www.postgresql.org/docs/current/runtime-config-replication.html#GUC-MAX-SLOT-WAL-KEEP-SIZE) limits the size of the Write-Ahead Log (WAL) files that replication slots can hold.
The WAL growth rate is expected to increase substantially during the initial replication of large datasets with high update frequency, particularly for tables included in the PowerSync publication.
During normal operation (after Sync Streams/Sync Rules are deployed) the WAL growth rate is much smaller than the initial replication period, since the PowerSync Service can replicate \~5k operations per second, meaning the WAL lag is typically in the MB range as opposed to the GB range.
When deciding what to set the `max_slot_wal_keep_size` configuration parameter the following should be taken in account:
1. Database size - This impacts the time it takes to complete the initial replication from the source Postgres database.
2. Sync Streams/Sync Rules complexity - This also impacts the time it takes to complete the initial replication.
3. Postgres update frequency - The frequency of updates (of tables included in the publication you create for PowerSync) during initial replication. The WAL growth rate is directly proportional to this.
To view the current replication slots that are being used by PowerSync you can run the following query:
```sql theme={null}
SELECT slot_name,
plugin,
slot_type,
active,
pg_size_pretty(pg_wal_lsn_diff(pg_current_wal_lsn(), restart_lsn)) AS replication_lag
FROM pg_replication_slots;
```
To view the current configured value of the `max_slot_wal_keep_size` you can run the following query:
```sql theme={null}
SHOW max_slot_wal_keep_size
```
If the slot is invalidated mid-snapshot, PowerSync detects the problem and stops replication with error [`PSYNC_S1146`](/debugging/error-codes#psync_s11xx-postgres-replication-issues). On the source database, increase `max_slot_wal_keep_size` and delete the existing replication slot. PowerSync creates a new slot and restarts the snapshot.
During a snapshot, PowerSync warns when less than 50% of the WAL budget remains. You may see this warning in the PowerSync dashboard, in the [Diagnostics API](/maintenance-ops/self-hosting/diagnostics) if you self-host, and in PowerSync Service logs. The [Log Reference](/debugging/log-reference#postgres-replication-slots-and-wal-budget) explains the slot and WAL budget messages. Increase `max_slot_wal_keep_size` or reduce snapshot work before the slot is invalidated. Use the considerations above to set a high enough cap.
#### Managing Replication Slots
Under normal operating conditions when new Sync Streams/Sync Rules are deployed to a PowerSync Service instance, a new replication slot will also be created and used for replication. The old replication slot from the previous version of the Sync Config will still remain, until reprocessing is completed, at which point the old replication slot will be removed by the PowerSync Service.
However, in some cases, a replication slot may remain without being used. Usually this happens when a PowerSync Service instance is de-provisioned, stopped intentionally or due to unexpected errors. This results in excessive disk usage due to the continued growth of the WAL.
To check which replication slots used by a PowerSync Service are no longer active, the following query can be executed against the source Postgres database:
```
SELECT slot_name,
pg_size_pretty(pg_wal_lsn_diff(pg_current_wal_lsn(), restart_lsn)) AS replication_lag
FROM pg_replication_slots WHERE active = false;
```
If you have inactive replication slots that need to be cleaned up, you can drop them using the following query:
```
SELECT slot_name,
pg_drop_replication_slot(slot_name)
FROM pg_replication_slots where active = false;
```
The alternative to manually checking for inactive replication slots would be to configure the `idle_replication_slot_timeout` configuration parameter on the source Postgres database.
The `idle_replication_slot_timeout` [configuration parameter](https://www.postgresql.org/docs/current/runtime-config-replication.html#GUC-IDLE-REPLICATION-SLOT-TIMEOUT) is only available from Postgres 18 onward.
The `idle_replication_slot_timeout` will invalidate replication slots that have remained inactive for longer than the value set for the `idle_replication_slot_timeout` parameter.
It's recommended to configure this parameter for source Postgres databases as this will prevent runaway WAL growth for replication slots that are no longer active or used by the PowerSync Service.
For slot mechanics, inactive slot cleanup queries, and the recovery procedure for an invalidated slot, see [Postgres Maintenance](/configuration/source-db/postgres-maintenance).
# Replication Lag
Source: https://docs.powersync.com/maintenance-ops/replication-lag
Understand, monitor, and reduce replication lag between your source database and the PowerSync Service.
Replication lag is the delay between a change being committed in your source database (Postgres, MongoDB, MySQL, SQL Server or Convex) and that change being available in the PowerSync Service for clients to sync. A small amount of lag is normal. Sustained or growing lag usually points to a specific cause that you can investigate and act on.
This page covers what replication lag is, how to monitor it, what commonly causes it, and how to reduce it.
## Overview
A change committed in the source database goes through roughly three stages before a client sees it:
1. The source database writes the change to its replication stream. The exact mechanism differs per source:
* **Postgres**: logical replication via the Write-Ahead Log (WAL), read through a replication slot.
* **MongoDB**: change streams backed by the oplog.
* **MySQL**: the binary log (binlog), read using GTIDs.
* **SQL Server**: Change Data Capture (CDC) change tables, populated by a capture job that scans the transaction log.
* **Convex**: document deltas from the Convex Streaming Export API, read on a polling interval.
2. The PowerSync Service reads the change from that stream and processes it into its internal bucket storage.
3. Connected clients receive the change on their next checkpoint.
Replication lag refers specifically to stage 2: the time or volume of changes that have been committed to the source but not yet processed by the PowerSync Service.
SQL Server has an additional source of latency inside stage 1: the CDC capture job itself runs on an interval (default 5 seconds on SQL Server, fixed at 20 seconds on Azure SQL), so changes do not appear in the CDC change tables instantly. See [SQL Server](#sql-server) below.
Convex replication also has polling latency. PowerSync polls the Convex `document_deltas` endpoint at a configurable interval, which defaults to once every 1000ms. See [Convex Source Database Setup](/configuration/source-db/setup#latency).
## How to Monitor Replication Lag
### PowerSync Dashboard
The [PowerSync Dashboard](https://dashboard.powersync.com/) exposes a **Replication Lag** chart in the **Metrics** view of each instance. Use it to spot spikes and trends over time.
See [Monitoring and Alerting](/maintenance-ops/monitoring-and-alerting) for alert and notification options available on your plan.
### Instance Logs
[Instance Logs](/maintenance-ops/monitoring-and-alerting#instance-logs) include **Replicator** entries that reflect replication activity from your source database to the PowerSync Service. Replication errors and restarts appear here and are often the first signal when lag starts climbing. See the [Log Reference](/debugging/log-reference#replicator-logs) for what common Replicator messages mean.
## What "*Normal*" Looks Like
Replication lag is not expected to be exactly zero at all times. Short fluctuations are routine and generally not a concern. As a rough guide:
* **Steady state**: lag stays low (typically in the single-digit seconds, or a few MB of WAL on Postgres) and returns to near-zero between bursts.
* **Write bursts**: a batch of writes in the source database causes a short spike while the service catches up. Lag should recover within seconds to a minute once the burst ends.
* **PowerSync infrastructure events**: brief replication lag can also occur during internal PowerSync scaling events. These are expected to recover on their own within a few minutes without any action from you.
* **Sustained or growing lag**: lag that keeps climbing, or does not recover after a burst or infrastructure event, indicates a problem worth investigating.
## Common Causes
The causes below are grouped into ones that apply to any source, and ones that are specific to a given source database.
Replication lag is separate from client sync lag. A client can be behind the PowerSync Service because of its own connection or app state, even when replication lag is zero.
### All Sources
#### Initial Replication of a Large Dataset
When you first connect a source database, or when you deploy Sync Config changes that trigger reprocessing, the PowerSync Service replicates the full set of matching rows. During this period:
* Replication lag will be elevated until the initial snapshot completes.
* The source-side replication buffer (WAL on Postgres, oplog on MongoDB, binlog on MySQL, CDC change tables on SQL Server) grows because the service has not yet acknowledged those changes.
This is expected. Plan for it by sizing the relevant retention setting appropriately (see the source-specific sections below) and by coordinating large Sync Config changes during lower-traffic windows.
#### Source Database Load
Replication lag is sensitive to activity on the source database:
* Long-running transactions on the source hold back the replication position until they commit.
* CPU, IO, or connection saturation on the source slows how fast changes are written to the replication stream in the first place.
If lag correlates with specific workloads, profile those workloads on the source database before looking at the PowerSync Service.
#### Bursty Write Workloads Exceeding Replication Throughput
Replication lag is a function of how fast changes arrive vs. how fast PowerSync can consume them. If a workload produces changes faster than the service can replicate, lag will accumulate until the burst ends and then drain as the service catches up. The service's published throughput (see [Performance and Limits](/resources/performance-and-limits#performance-expectations)) is roughly:
* **2,000-4,000 operations per second** for small rows
* **Up to 5 MB per second** for large rows
* **\~60 transactions per second** for smaller transactions
Workloads that commonly push past these rates, and therefore commonly cause visible lag spikes, include:
* **Scheduled jobs**: cron jobs, nightly batches, or queue workers that flush on a timer. These tend to produce very sharp lag spikes at predictable times.
* **Bulk `UPDATE`s across indexed columns**: a single statement can generate millions of row-change events in the replication stream, even if the SQL itself runs quickly on the source.
* **Backfills and data migrations**: schema changes, column backfills, or re-keying jobs. On Postgres these can also rewrite large portions of a table, multiplying WAL volume.
* **Bulk imports** (`COPY`, `LOAD DATA`, `BULK INSERT`, `insertMany`): import throughput on the source is often far higher than replication throughput.
If a burst is unavoidable, prefer to run it during lower-traffic windows, batch it into smaller chunks rather than one large transaction, and make sure your source-side retention setting is large enough to cover the time it takes PowerSync to catch up afterwards. See the source-specific sections below: [Postgres](#postgres), [MongoDB](#mongodb), [MySQL](#mysql), [SQL Server](#sql-server).
#### Sync Config Complexity
Slower replication performance is correlated with the number of buckets a replicated row ends up in, i.e. a row written once to the source database can be replicated to many buckets if many queries in your Sync Config reference it. If lag climbs after a Sync Config deploy and stays elevated, review the new configuration for rows that end up in many buckets. See [Performance and Limits](/resources/performance-and-limits) for limits that are worth staying well inside of.
### Postgres
#### WAL Retention (`max_slot_wal_keep_size`)
If the WAL grows faster than the PowerSync Service can consume it, and the total unconsumed WAL exceeds `max_slot_wal_keep_size`, Postgres will invalidate the replication slot. PowerSync then has to restart replication from scratch, which extends the period of elevated lag.
The [`max_slot_wal_keep_size`](https://www.postgresql.org/docs/current/runtime-config-replication.html#GUC-MAX-SLOT-WAL-KEEP-SIZE) Postgres parameter limits how much WAL a replication slot can retain. Setting it too low on a write-heavy database risks slot invalidation during bursts or during initial replication.
**Supabase defaults**: Supabase projects ship with `max_slot_wal_keep_size = 4GB` and a limit of 5 replication slots. The 4GB cap is easy to exceed during initial replication of a large dataset or a sustained write burst, after which the slot will be invalidated and PowerSync has to restart replication from scratch. Raise this value before connecting a large Supabase database to PowerSync.
See [Managing and Monitoring Replication Lag](/maintenance-ops/production-readiness-guide#managing-and-monitoring-replication-lag) for queries to check the current setting and the current slot lag, and for guidance on sizing it. When a slot is invalidated, PowerSync surfaces error [`PSYNC_S1146`](/debugging/error-codes#psync_s11xx-postgres-replication-issues) — see [Recovering from an invalidated slot](/configuration/source-db/postgres-maintenance#recovering-from-an-invalidated-slot) for the recovery procedure.
#### `TRUNCATE` on Replicated Tables
A `TRUNCATE` on a table in your Sync Config is treated as a change event for every row in that table, which can force the service to re-process large amounts of bucket data. If `TRUNCATE` runs on a regular schedule (for example, a cron that truncates-and-reloads a table), each run will produce a visible lag spike. Prefer `DELETE` with a filter, or redesign the job so it does not truncate a replicated table.
#### Inactive Replication Slots Holding WAL
When Sync Streams/Sync Rules are redeployed, PowerSync creates a new replication slot and retires the old one once reprocessing completes. If an instance is stopped, deprovisioned, or hits an error before that handover finishes, an inactive slot can remain on the source database and continue to hold WAL, which can contribute to disk pressure and can mask what "real" lag looks like.
See [Managing Replication Slots](/maintenance-ops/production-readiness-guide#managing-replication-slots) for queries to find and drop inactive slots, and for notes on the Postgres 18+ `idle_replication_slot_timeout` parameter.
### MongoDB
* **Change stream timeouts**: a significant delay on the source database in reading the change stream can cause timeouts (see [`PSYNC_S1345`](/debugging/error-codes#psync_s13xx-mongodb-replication-issues)). If this is not resolved after retries, replication may need to be restarted from scratch.
* **Change stream invalidation**: replication restarts with a new change stream if the existing one is invalidated, for example if the `startAfter`/`resumeToken` is no longer valid, if the replication connection changes, or if the database is dropped (see [`PSYNC_S1344`](/debugging/error-codes#psync_s13xx-mongodb-replication-issues)).
* **Deeply nested documents**: JSON or embedded-document nesting deeper than 20 levels will fail replication with [`PSYNC_S1004`](/debugging/error-codes#psync_s1xxx-replication-issues).
* **Post-image configuration**: if post-images are set to `read_only`, every replicated collection must have `changeStreamPreAndPostImages: { enabled: true }` set or replication will error. See [Post Images](/configuration/source-db/setup#post-images).
### MySQL
* **Binlog retention**: PowerSync reads from the MySQL binary log. If required binlog files are purged before PowerSync has read them (for example, after extended downtime or sustained lag), replication has to restart from scratch. Configure MySQL binlog retention to be long enough to cover expected downtime and lag bursts.
* **`binlog-do-db` / `binlog-ignore-db` filters**: these filters are optional, but if set, every database referenced by your Sync Config must be included. Tables in excluded databases will not produce binlog events for PowerSync to replicate. See [Additional Configuration (Optional) → Binlog](/configuration/source-db/setup#additional-configuration-optional) in the MySQL setup docs.
See [MySQL setup](/configuration/source-db/setup#mysql) for required binlog settings.
### SQL Server
* **CDC retention**: the CDC cleanup job expires data from CDC change tables after a retention window (default 3 days). If the PowerSync Service is offline longer than this period, data will need to be fully re-synced.
* **Latency from CDC polling**: end-to-end latency has two components. First, the SQL Server capture job's transaction log scan interval (default 5 seconds, recommended 1 second; fixed at 20 seconds on Azure SQL Database). Second, PowerSync's own polling interval (`pollingIntervalMs`, default 1000ms, self-hosted only). Both contribute to the minimum achievable lag.
* **`_powersync_checkpoints` table**: CDC must be enabled on `dbo._powersync_checkpoints` for PowerSync to generate regular checkpoints.
See [SQL Server setup](/configuration/source-db/setup#sql-server) for CDC configuration, recommended capture job tuning, and the [Latency](/configuration/source-db/setup#latency) section.
## Reducing Replication Lag
Start with the "All Sources" checks, then go to the section for your source database.
### All Sources
* **Confirm the source database is healthy**: check CPU, IO, connection count, and long-running transactions on the source. A saturated source will cause replication lag that no amount of tuning on the PowerSync side can fix.
* **Pause or reduce large writes while the service catches up**: if lag is already elevated, holding off on scheduled jobs, bulk updates, migrations, and backfills is usually the fastest way to let it drain. If a large write is unavoidable, batch it into smaller transactions and pace them so the service has time to drain between batches, rather than running it as one large transaction.
* **Review Sync Config**: look for Sync Config changes that could be producing significantly more buckets or heavier parameter queries than before. Simplify where possible and deploy large changes during lower-traffic windows.
* **Check for source schema changes**: `ALTER TABLE` and similar changes on replicated tables can stall or invalidate replication until reconfigured. See [Implementing Schema Changes](/maintenance-ops/implementing-schema-changes) for the recommended flow.
* **Check instance logs for errors**: [Replicator logs](/maintenance-ops/monitoring-and-alerting#instance-logs) often contain the specific error (slot invalidation, change stream failure, binlog purge, CDC retention expiry, source connectivity) behind a lag incident.
### Postgres
* Run the queries in [Managing and Monitoring Replication Lag](/maintenance-ops/production-readiness-guide#managing-and-monitoring-replication-lag) to see current slot lag and `max_slot_wal_keep_size`. Increase `max_slot_wal_keep_size` if lag routinely approaches it, especially before deploying Sync Config changes against large datasets. On Supabase, raise the default 4GB cap before connecting a large database.
* If WAL is growing on the source but lag reported by the PowerSync Service is low, look for inactive slots. See [Managing Replication Slots](/maintenance-ops/production-readiness-guide#managing-replication-slots) to identify and drop them.
* Avoid `TRUNCATE` on tables in your Sync Config. See [`TRUNCATE` on Replicated Tables](#truncate-on-replicated-tables) above.
### MongoDB
* Check [Replicator logs](/maintenance-ops/monitoring-and-alerting#instance-logs) for change-stream errors ([`PSYNC_S1344`](/debugging/error-codes#psync_s13xx-mongodb-replication-issues), [`PSYNC_S1345`](/debugging/error-codes#psync_s13xx-mongodb-replication-issues)). Persistent timeouts or invalidation generally require the change stream to be re-established, which may restart replication.
* If you are using `read_only` post-images, confirm every replicated collection has `changeStreamPreAndPostImages` enabled. See [Post Images](/configuration/source-db/setup#post-images).
### MySQL
* Confirm MySQL binlog retention is long enough to tolerate expected downtime or lag bursts, and that any `binlog-do-db` / `binlog-ignore-db` filters include every database referenced by your Sync Config. See [MySQL](#mysql) above.
### SQL Server
* Confirm the CDC capture job is running and has not exceeded its retention window (default 3 days), and that CDC is still enabled on `dbo._powersync_checkpoints`. See [SQL Server](#sql-server) above and [SQL Server setup](/configuration/source-db/setup#sql-server) for capture job tuning.
If lag persists after these checks, reach out on the PowerSync [Discord](https://discord.gg/powersync) or contact support with your instance ID, the time range of the incident, and a screenshot of the Replication Lag chart.
## Related
Configure usage metrics, logs, issue alerts, and notifications.
Database best practices, including replication slot management.
Common issues and pointers for debugging sync and replication.
Service limits that are worth staying well inside of.
# Deploy PowerSync on AWS ECS
Source: https://docs.powersync.com/maintenance-ops/self-hosting/aws-ecs
Deploy the self-hosted PowerSync Service on AWS ECS with Fargate.
[AWS ECS](https://aws.amazon.com/ecs/) with Fargate provides a serverless container orchestration platform for running PowerSync without managing servers.
A [Terraform example](https://github.com/powersync-community/terraform-self-host-aws) is available to provision a comparable ECS Fargate deployment as infrastructure-as-code. Its default architecture uses Amazon RDS for Postgres for both the source and bucket storage databases, rather than the MongoDB bucket storage used in this guide.
## Prerequisites
Before deploying PowerSync on AWS ECS, ensure you have:
* AWS account with permissions for EC2, ECS, ALB, IAM and Secrets Manager
* AWS CLI installed and configured
* Understanding of the [deployment architecture](/maintenance-ops/self-hosting/deployment-architecture) for production vs development setup
## 1. PowerSync Configuration
Create your `service.yaml` configuration file following the [Self-Hosted Configuration Guide](/configuration/powersync-service/self-hosted-instances).
Your configuration must include:
* [Sync Streams](/sync/streams/overview) (or legacy [Sync Rules](/sync/rules/overview)): Define which data to sync to clients
* [Client Auth](/configuration/auth/overview): Your authentication provider's JWKS
* [Source Database](/configuration/source-db/setup): Connection details for your source database
* [Monitoring](/maintenance-ops/self-hosting/monitoring): Enable the Prometheus metrics endpoint for connection-based auto-scaling (used in the [Auto Scaling](#auto-scaling-high-availability-setup) section):
```yaml theme={null}
telemetry:
prometheus_port: 9090
```
* [Bucket Storage](/configuration/powersync-service/self-hosted-instances#storage): Connection details for your bucket storage database. PowerSync supports MongoDB or Postgres as bucket storage databases. In this guide, we focus on MongoDB.
For bucket storage, we recommend configuring an **AWS PrivateLink** to establish a secure, private connection between your ECS tasks and MongoDB Atlas that doesn't traverse the public internet.
Follow the [AWS PrivateLink guide for MongoDB Atlas](https://aws.amazon.com/blogs/apn/connecting-applications-securely-to-a-mongodb-atlas-data-plane-with-aws-privatelink/) to configure the VPC endpoint and update your MongoDB connection string to use the private endpoint. As seen in the [Secrets Manager](#5-secrets-manager) setup, use the updated connection string in your `PS_MONGO_URI` secret.
For self-hosting MongoDB bucket storage on an EC2 instance, refer to AWS's guides (which refer to Amazon DocumentDB, but the installation steps are applicable):
1. [Launch an EC2 Instance](https://docs.aws.amazon.com/dms/latest/sbs/chap-mongodb2documentdb.01.html)
2. [Install and Configure MongoDB](https://docs.aws.amazon.com/dms/latest/sbs/chap-mongodb2documentdb.02.html)
3. **Network Configuration**
* Place MongoDB EC2 instance in the same VPC as your ECS tasks
* Configure security groups to allow ECS tasks to connect to MongoDB on port 27017:
```bash theme={null}
# Create MongoDB security group
MONGO_SG=$(aws ec2 create-security-group \
--group-name mongodb-sg \
--description "MongoDB for PowerSync" \
--vpc-id $VPC_ID \
--query 'GroupId' --output text)
# Allow ECS tasks to connect to MongoDB ($ECS_SG is the ECS tasks security group created later in the Security Groups section)
aws ec2 authorize-security-group-ingress \
--group-id $MONGO_SG \
--protocol tcp --port 27017 --source-group $ECS_SG
```
## 2. VPC and Networking Setup
This guide uses bash variables throughout for easy copy-paste execution.
```bash theme={null}
# Set your AWS region and account ID
AWS_REGION="us-east-1" # Change to your region
AWS_ACCOUNT_ID=$(aws sts get-caller-identity --query Account --output text)
# Set your VPC ID (or create a new VPC)
VPC_ID="vpc-xxxxx"
# Set PowerSync version (check Docker Hub for latest: https://hub.docker.com/r/journeyapps/powersync-service/tags)
PS_VERSION="1.20.1"
```
### VPC Architecture Overview
PowerSync on ECS requires a VPC with both **public** and **private** subnets:
* **Public subnets**: Host the Application Load Balancer (ALB) and NAT Gateway with direct internet access
* **Private subnets**: Host ECS tasks for security, with outbound-only internet access via NAT Gateway
**Network Flow:**
```
Internet → Internet Gateway → Public Subnets (ALB, NAT) → Private Subnets (ECS Tasks)
```
**Default VPC users**: The AWS default VPC only contains public subnets. You must create private subnets following the steps below.
### Check Existing Subnets
```bash theme={null}
# List all subnets in your VPC
aws ec2 describe-subnets \
--filters "Name=vpc-id,Values=$VPC_ID" \
--query 'Subnets[*].[SubnetId,CidrBlock,MapPublicIpOnLaunch,AvailabilityZone]' \
--output table
```
If `MapPublicIpOnLaunch` is `True`, those are public subnets. Save the public subnet IDs:
```bash theme={null}
# Get public subnets (for ALB and NAT Gateway)
PUBLIC_SUBNET_1=$(aws ec2 describe-subnets \
--filters "Name=vpc-id,Values=$VPC_ID" "Name=map-public-ip-on-launch,Values=true" \
--query 'Subnets[0].SubnetId' --output text)
PUBLIC_SUBNET_2=$(aws ec2 describe-subnets \
--filters "Name=vpc-id,Values=$VPC_ID" "Name=map-public-ip-on-launch,Values=true" \
--query 'Subnets[1].SubnetId' --output text)
echo "Public Subnet 1: $PUBLIC_SUBNET_1"
echo "Public Subnet 2: $PUBLIC_SUBNET_2"
```
### Create Private Subnets
Create two private subnets in different availability zones for high availability:
```bash theme={null}
# Get available zones in your region
AZ1=$(aws ec2 describe-availability-zones --region $AWS_REGION --query 'AvailabilityZones[0].ZoneName' --output text)
AZ2=$(aws ec2 describe-availability-zones --region $AWS_REGION --query 'AvailabilityZones[1].ZoneName' --output text)
echo "Availability Zone 1: $AZ1"
echo "Availability Zone 2: $AZ2"
# Get VPC CIDR to determine available address space
VPC_CIDR=$(aws ec2 describe-vpcs --vpc-ids $VPC_ID --query 'Vpcs[0].CidrBlock' --output text)
echo "VPC CIDR: $VPC_CIDR"
# Create first private subnet (adjust CIDR if conflicts exist)
PRIVATE_SUBNET_1=$(aws ec2 create-subnet \
--vpc-id $VPC_ID \
--cidr-block 172.31.96.0/20 \
--availability-zone $AZ1 \
--tag-specifications 'ResourceType=subnet,Tags=[{Key=Name,Value=powersync-private-1}]' \
--query 'Subnet.SubnetId' \
--output text)
echo "Private Subnet 1: $PRIVATE_SUBNET_1"
# Create second private subnet (adjust CIDR if conflicts exist)
PRIVATE_SUBNET_2=$(aws ec2 create-subnet \
--vpc-id $VPC_ID \
--cidr-block 172.31.112.0/20 \
--availability-zone $AZ2 \
--tag-specifications 'ResourceType=subnet,Tags=[{Key=Name,Value=powersync-private-2}]' \
--query 'Subnet.SubnetId' \
--output text)
echo "Private Subnet 2: $PRIVATE_SUBNET_2"
```
**CIDR Block Configuration**: The example uses `172.31.96.0/20` and `172.31.112.0/20`, which work for the default VPC (`172.31.0.0/16`). If you get a CIDR conflict error, adjust these blocks to match unused address space in your VPC. Each /20 block provides 4,096 IP addresses.
**Create Route Table for Private Subnets:**
```bash theme={null}
# Create private route table
PRIVATE_RTB=$(aws ec2 create-route-table \
--vpc-id $VPC_ID \
--tag-specifications 'ResourceType=route-table,Tags=[{Key=Name,Value=powersync-private-rtb}]' \
--query 'RouteTable.RouteTableId' \
--output text)
echo "Private Route Table: $PRIVATE_RTB"
# Associate private subnets with route table
aws ec2 associate-route-table \
--route-table-id $PRIVATE_RTB \
--subnet-id $PRIVATE_SUBNET_1
aws ec2 associate-route-table \
--route-table-id $PRIVATE_RTB \
--subnet-id $PRIVATE_SUBNET_2
echo "Private subnets created and associated with route table"
```
### NAT Gateway Setup
ECS tasks in private subnets need outbound internet access for:
* Pulling container images from Amazon ECR
* Fetching JWKS for authentication (if applicable in your client authentication setup)
* Connecting to external services
**Create NAT Gateway:**
```bash theme={null}
# Allocate Elastic IP
EIP_ALLOC=$(aws ec2 allocate-address \
--domain vpc \
--query 'AllocationId' \
--output text)
echo "Elastic IP Allocation: $EIP_ALLOC"
# Create NAT Gateway in a PUBLIC subnet
NAT_GW=$(aws ec2 create-nat-gateway \
--subnet-id $PUBLIC_SUBNET_1 \
--allocation-id $EIP_ALLOC \
--query 'NatGateway.NatGatewayId' \
--output text)
echo "NAT Gateway: $NAT_GW"
# Wait for NAT Gateway to become available (takes ~2 minutes)
echo "Waiting for NAT Gateway to become available (this takes ~2 minutes)..."
aws ec2 wait nat-gateway-available --nat-gateway-ids $NAT_GW
echo "NAT Gateway is now available"
```
**Add Route to Private Route Table:**
```bash theme={null}
# Add default route to NAT Gateway in private route table
aws ec2 create-route \
--route-table-id $PRIVATE_RTB \
--destination-cidr-block 0.0.0.0/0 \
--nat-gateway-id $NAT_GW
```
**Verify Setup:**
```bash theme={null}
# Verify private route table
aws ec2 describe-route-tables \
--route-table-ids $PRIVATE_RTB \
--query 'RouteTables[0].Routes' \
--output table
# Should show:
# - 172.31.0.0/16 -> local (VPC internal routing)
# - 0.0.0.0/0 -> nat-xxxxx (Internet via NAT)
```
### Create Security Groups
```bash theme={null}
# ALB security group (allows HTTPS from internet)
ALB_SG=$(aws ec2 create-security-group \
--group-name powersync-alb-sg \
--description "PowerSync ALB" \
--vpc-id $VPC_ID \
--query 'GroupId' --output text)
echo "ALB Security Group: $ALB_SG"
aws ec2 authorize-security-group-ingress \
--group-id $ALB_SG \
--protocol tcp --port 443 --cidr 0.0.0.0/0
# ECS security group (allows traffic from ALB only)
ECS_SG=$(aws ec2 create-security-group \
--group-name powersync-ecs-sg \
--description "PowerSync ECS tasks" \
--vpc-id $VPC_ID \
--query 'GroupId' --output text)
echo "ECS Security Group: $ECS_SG"
aws ec2 authorize-security-group-ingress \
--group-id $ECS_SG \
--protocol tcp --port 8080 --source-group $ALB_SG
echo "Security groups created successfully"
```
## 3. Application Load Balancer
### Domain Setup
PowerSync requires a domain name for SSL certificate provisioning. You can either:
* Use an existing domain by creating a Route 53 hosted zone and updating your registrar's nameservers
* Register a new domain directly through Route 53
For detailed instructions, follow the official AWS guides:
* [Configuring DNS routing for a new domain](https://docs.aws.amazon.com/Route53/latest/DeveloperGuide/dns-configuring-new-domain.html) - For existing domains
* [Registering a new domain](https://docs.aws.amazon.com/Route53/latest/DeveloperGuide/domain-register.html) - To register through Route 53
Once your hosted zone is created, export the zone ID:
```bash theme={null}
export HOSTED_ZONE_ID=$(aws route53 list-hosted-zones-by-name \
--dns-name yourdomain.com \
--query 'HostedZones[0].Id' \
--output text)
echo "Hosted Zone ID: $HOSTED_ZONE_ID"
```
### Request SSL Certificate
For secure HTTPS connections, request an SSL certificate using AWS Certificate Manager (ACM):
```bash theme={null}
# Set your domain name
POWERSYNC_DOMAIN="powersync.yourdomain.com" # Change to your domain
# Request certificate
CERT_ARN=$(aws acm request-certificate \
--domain-name $POWERSYNC_DOMAIN \
--validation-method DNS \
--region $AWS_REGION \
--query 'CertificateArn' \
--output text)
echo "Certificate ARN: $CERT_ARN"
# Get validation record details
VALIDATION_NAME=$(aws acm describe-certificate \
--certificate-arn $CERT_ARN \
--region $AWS_REGION \
--query 'Certificate.DomainValidationOptions[0].ResourceRecord.Name' \
--output text)
VALIDATION_VALUE=$(aws acm describe-certificate \
--certificate-arn $CERT_ARN \
--region $AWS_REGION \
--query 'Certificate.DomainValidationOptions[0].ResourceRecord.Value' \
--output text)
echo "Validation Name: $VALIDATION_NAME"
echo "Validation Value: $VALIDATION_VALUE"
```
**Add DNS Validation Record:**
Add the `CNAME` record using your DNS provider's management console:
| Type | Name | Value | TTL |
| ------- | ------------------- | -------------------- | ----- |
| `CNAME` | `[VALIDATION_NAME]` | `[VALIDATION_VALUE]` | `300` |
Add the `CNAME` record using AWS CLI:
```bash theme={null}
aws route53 change-resource-record-sets \
--hosted-zone-id $HOSTED_ZONE_ID \
--change-batch '{
"Changes": [{
"Action": "CREATE",
"ResourceRecordSet": {
"Name": "'$VALIDATION_NAME'",
"Type": "CNAME",
"TTL": 300,
"ResourceRecords": [{"Value": "'$VALIDATION_VALUE'"}]
}
}]
}'
```
**Wait for Certificate Validation:**
```bash theme={null}
aws acm wait certificate-validated --certificate-arn $CERT_ARN --region $AWS_REGION
```
### Create ALB
```bash theme={null}
# Create load balancer
ALB_ARN=$(aws elbv2 create-load-balancer \
--name powersync-alb \
--subnets $PUBLIC_SUBNET_1 $PUBLIC_SUBNET_2 \
--security-groups $ALB_SG \
--scheme internet-facing \
--query 'LoadBalancers[0].LoadBalancerArn' \
--output text)
echo "ALB ARN: $ALB_ARN"
# Create target group
TG_ARN=$(aws elbv2 create-target-group \
--name powersync-tg \
--protocol HTTP \
--port 8080 \
--vpc-id $VPC_ID \
--target-type ip \
--health-check-path /probes/liveness \
--health-check-interval-seconds 30 \
--query 'TargetGroups[0].TargetGroupArn' \
--output text)
echo "Target Group ARN: $TG_ARN"
# Create HTTPS listener
LISTENER_ARN=$(aws elbv2 create-listener \
--load-balancer-arn $ALB_ARN \
--protocol HTTPS \
--port 443 \
--certificates CertificateArn=$CERT_ARN \
--default-actions Type=forward,TargetGroupArn=$TG_ARN \
--query 'Listeners[0].ListenerArn' \
--output text)
echo "Listener ARN: $LISTENER_ARN"
# Configure WebSocket support
# PowerSync uses long-lived WebSocket connections for real-time sync
# Default ALB timeout is 60s, which would disconnect clients prematurely
# Setting to 3600s (1 hour) prevents unnecessary disconnections
aws elbv2 modify-load-balancer-attributes \
--load-balancer-arn $ALB_ARN \
--attributes Key=idle_timeout.timeout_seconds,Value=3600
```
## 4. DNS Configuration
Point your domain to the load balancer:
```bash theme={null}
# Get ALB DNS name
ALB_DNS=$(aws elbv2 describe-load-balancers \
--names powersync-alb \
--query 'LoadBalancers[0].DNSName' \
--output text)
ALB_ZONE=$(aws elbv2 describe-load-balancers \
--names powersync-alb \
--query 'LoadBalancers[0].CanonicalHostedZoneId' \
--output text)
echo "ALB DNS: $ALB_DNS"
echo "ALB Zone: $ALB_ZONE"
```
Create a `CNAME` record pointing to the ALB DNS name.
| Type | Name | Value | TTL |
| ------- | -------------------------- | ----------- | ----- |
| `CNAME` | `powersync.yourdomain.com` | `[ALB_DNS]` | `300` |
Create an alias `A` record pointing to the ALB:
```bash theme={null}
aws route53 change-resource-record-sets \
--hosted-zone-id $HOSTED_ZONE_ID \
--change-batch '{
"Changes": [{
"Action": "CREATE",
"ResourceRecordSet": {
"Name": "'$POWERSYNC_DOMAIN'",
"Type": "A",
"AliasTarget": {
"HostedZoneId": "'$ALB_ZONE'",
"DNSName": "'$ALB_DNS'",
"EvaluateTargetHealth": true
}
}
}]
}'
```
## 5. Secrets Manager
Store your PowerSync configuration and connection strings securely in AWS Secrets Manager. This allows you to reference them in your ECS task definition without hardcoding sensitive information.
```bash theme={null}
# Store config (base64-encoded, as required by the POWERSYNC_CONFIG_B64 env variable)
aws secretsmanager create-secret \
--name powersync/config \
--secret-string "$(base64 -i service.yaml)"
# Store connection strings
# Set your source database connection string (e.g., Postgres, MongoDB, MySQL, or SQL Server)
aws secretsmanager create-secret \
--name powersync/data-source-uri \
--secret-string "postgresql://user:pass@host:5432/db"
# Set your replication bucket storage connection string (e.g., MongoDB or Postgres)
aws secretsmanager create-secret \
--name powersync/storage-uri \
--secret-string "mongodb://user:pass@host:27017/?replicaSet=rs0"
aws secretsmanager create-secret \
--name powersync/jwks-url \
--secret-string "https://your-auth-provider.com/.well-known/jwks.json"
```
AWS Secrets Manager automatically appends a 6-character suffix to secret ARNs (e.g., `powersync/config-AbCdEf`).
ECS task definitions support **prefix matching**, allowing you to reference secrets using just the base name:
* Created as: `powersync/config-AbCdEf` (with suffix)
* Referenced as: `arn:aws:secretsmanager:region:account:secret:powersync/config` (without suffix)
This means you don't need to update task definitions when secrets are rotated.
## 6. ECS Task Definition
The ECS task definition specifies how to run the PowerSync container, including environment variables, secrets, resource limits, and health checks.
### Create IAM Role
```bash theme={null}
# Create execution role
aws iam create-role \
--role-name PowerSyncTaskExecutionRole \
--assume-role-policy-document '{
"Version": "2012-10-17",
"Statement": [{
"Effect": "Allow",
"Principal": {"Service": "ecs-tasks.amazonaws.com"},
"Action": "sts:AssumeRole"
}]
}'
# Wait for role to propagate
sleep 10
aws iam attach-role-policy \
--role-name PowerSyncTaskExecutionRole \
--policy-arn arn:aws:iam::aws:policy/service-role/AmazonECSTaskExecutionRolePolicy
# Add Secrets Manager access
aws iam put-role-policy \
--role-name PowerSyncTaskExecutionRole \
--policy-name SecretsAccess \
--policy-document '{
"Version": "2012-10-17",
"Statement": [{
"Effect": "Allow",
"Action": ["secretsmanager:GetSecretValue"],
"Resource": "arn:aws:secretsmanager:'$AWS_REGION':'$AWS_ACCOUNT_ID':secret:powersync/*"
}]
}'
# Create task role (used by running containers for CloudWatch metrics publishing)
aws iam create-role \
--role-name PowerSyncTaskRole \
--assume-role-policy-document '{
"Version": "2012-10-17",
"Statement": [{
"Effect": "Allow",
"Principal": {"Service": "ecs-tasks.amazonaws.com"},
"Action": "sts:AssumeRole"
}]
}'
# Wait for role to propagate
sleep 10
# Add CloudWatch permissions for the CW Agent sidecar to publish metrics
aws iam put-role-policy \
--role-name PowerSyncTaskRole \
--policy-name CloudWatchMetrics \
--policy-document '{
"Version": "2012-10-17",
"Statement": [{
"Effect": "Allow",
"Action": [
"cloudwatch:PutMetricData",
"logs:CreateLogGroup",
"logs:CreateLogStream",
"logs:PutLogEvents"
],
"Resource": "*"
}]
}'
TASK_ROLE_ARN="arn:aws:iam::$AWS_ACCOUNT_ID:role/PowerSyncTaskRole"
echo "Task Role ARN: $TASK_ROLE_ARN"
# Save role ARNs
TASK_EXECUTION_ROLE_ARN="arn:aws:iam::$AWS_ACCOUNT_ID:role/PowerSyncTaskExecutionRole"
echo "Task Execution Role ARN: $TASK_EXECUTION_ROLE_ARN"
echo "Task Role ARN: $TASK_ROLE_ARN"
```
### Create Cluster
```bash theme={null}
aws ecs create-cluster \
--cluster-name powersync-cluster \
--capacity-providers FARGATE
```
### Register Task Definition
The task definitions below allocate **2 vCPU and 4GB memory** per container. You can adjust resources based on your workload — see [Deployment Architecture](/maintenance-ops/self-hosting/deployment-architecture) for scaling guidance (recommended baseline: 1 vCPU, 2GB memory). Note that [AWS Fargate enforces specific CPU/memory combinations](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/task_definition_parameters.html#task_size) — for example, 2 vCPU (2048 CPU units) requires at least 4GB (4096 MiB) memory.
For production deployments, run separate replication and API processes to enable zero-downtime rolling updates. This allows independent scaling of API containers.
**Create Replication Task Definition**
```bash theme={null}
cat > replication-task-definition.json <
The CloudWatch Agent sidecar adds \~256MB memory overhead. The task definition below allocates 4096MB total (shared between both containers). If you need more headroom, increase the task memory to 5120MB or 6144MB.
First, create the CloudWatch Agent configuration. This tells the agent to scrape the PowerSync Prometheus endpoint on `localhost:9090` and publish the `powersync_concurrent_connections` metric to CloudWatch:
```bash theme={null}
cat > cw-agent-config.json <<'CWEOF'
{
"logs": {
"metrics_collected": {
"prometheus": {
"log_group_name": "/ecs/powersync-api/prometheus",
"prometheus_config_path": "env:PROMETHEUS_CONFIG_CONTENT",
"emf_processor": {
"metric_namespace": "PowerSync",
"metric_declaration": [
{
"source_labels": ["job"],
"label_matcher": "powersync",
"dimensions": [["ClusterName"]],
"metric_selectors": [
"^powersync_concurrent_connections$"
]
}
]
}
}
}
}
}
CWEOF
```
Store the CloudWatch Agent config in SSM Parameter Store:
```bash theme={null}
aws ssm put-parameter \
--name "/ecs/powersync/cwagent-config" \
--type "String" \
--value file://cw-agent-config.json \
--overwrite
# Grant the execution role access to read the SSM parameter
aws iam put-role-policy \
--role-name PowerSyncTaskExecutionRole \
--policy-name SSMParameterAccess \
--policy-document '{
"Version": "2012-10-17",
"Statement": [{
"Effect": "Allow",
"Action": ["ssm:GetParameters"],
"Resource": "arn:aws:ssm:'$AWS_REGION':'$AWS_ACCOUNT_ID':parameter/ecs/powersync/*"
}]
}'
```
Now create the API task definition with both the PowerSync container and the CloudWatch Agent sidecar:
```bash theme={null}
cat > api-task-definition.json <
The Prometheus port (9090) is **not** exposed through the ALB — it is only accessible within the task via `localhost` (ECS `awsvpc` networking). The CloudWatch Agent sidecar scrapes metrics locally every 30 seconds and publishes them to CloudWatch.
This basic setup runs both replication and API processes in the same container. This is not recommended for production.
Generate the task definition using your environment variables:
```bash theme={null}
cat > task-definition.json <
## 7. Deploy ECS Service
Create the ECS service to run PowerSync tasks
For production deployments, run separate replication and API processes to enable zero-downtime rolling updates. This allows independent scaling of API containers.
**Deploy Replication Service (1 Instance)**
```bash theme={null}
aws ecs create-service \
--cluster powersync-cluster \
--service-name powersync-replication \
--task-definition powersync-replication \
--desired-count 1 \
--launch-type FARGATE \
--network-configuration "awsvpcConfiguration={
subnets=[$PRIVATE_SUBNET_1,$PRIVATE_SUBNET_2],
securityGroups=[$ECS_SG],
assignPublicIp=DISABLED
}" \
--deployment-configuration "minimumHealthyPercent=0,maximumPercent=100"
```
**Deploy API Service (2+ Instances)**
```bash theme={null}
aws ecs create-service \
--cluster powersync-cluster \
--service-name powersync-api \
--task-definition powersync-api \
--desired-count 2 \
--launch-type FARGATE \
--network-configuration "awsvpcConfiguration={
subnets=[$PRIVATE_SUBNET_1,$PRIVATE_SUBNET_2],
securityGroups=[$ECS_SG],
assignPublicIp=DISABLED
}" \
--load-balancers "targetGroupArn=$TG_ARN,containerName=powersync-api,containerPort=8080" \
--health-check-grace-period-seconds 120 \
--deployment-configuration "minimumHealthyPercent=100,maximumPercent=200"
```
**Verify HA Deployment:**
```bash theme={null}
# Check replication service status
aws ecs describe-services \
--cluster powersync-cluster \
--services powersync-replication \
--query 'services[0].[serviceName,status,runningCount,desiredCount]' \
--output table
# Check API service status
aws ecs describe-services \
--cluster powersync-cluster \
--services powersync-api \
--query 'services[0].[serviceName,status,runningCount,desiredCount]' \
--output table
# Wait for tasks to be running (takes 2-3 minutes)
echo "Waiting for tasks to start..."
sleep 60
# Test endpoint (replace with your domain)
curl https://$POWERSYNC_DOMAIN/probes/liveness
# View API logs
aws logs tail /ecs/powersync-api --follow
# View replication logs
aws logs tail /ecs/powersync-replication --follow
```
This basic setup runs both replication and API processes in the same container. Running multiple instances (`desired-count > 1`) will cause **Sync Rule lock errors during rolling updates** when deploying new task definitions. A single-instance setup is not recommended for production.
```bash theme={null}
aws ecs create-service \
--cluster powersync-cluster \
--service-name powersync-service \
--task-definition powersync-service \
--desired-count 1 \
--launch-type FARGATE \
--network-configuration "awsvpcConfiguration={
subnets=[$PRIVATE_SUBNET_1,$PRIVATE_SUBNET_2],
securityGroups=[$ECS_SG],
assignPublicIp=DISABLED
}" \
--load-balancers "targetGroupArn=$TG_ARN,containerName=powersync,containerPort=8080" \
--health-check-grace-period-seconds 120 \
--deployment-configuration "minimumHealthyPercent=0,maximumPercent=100"
```
**Verify Basic Deployment:**
```bash theme={null}
# Check service status
aws ecs describe-services \
--cluster powersync-cluster \
--services powersync-service \
--query 'services[0].[serviceName,status,runningCount,desiredCount]' \
--output table
# Wait for task to be running (takes 2-3 minutes)
echo "Waiting for tasks to start..."
sleep 60
# Test endpoint (replace with your domain)
curl https://$POWERSYNC_DOMAIN/probes/liveness
# View logs
aws logs tail /ecs/powersync --follow
```
## Production Enhancements
For production deployments, consider adding the following enhancements:
### Daily Compact Job (Recommended)
PowerSync requires [daily compaction](/maintenance-ops/compacting-buckets) to optimize bucket storage. Schedule it as an ECS task with EventBridge:
Generate the compact task definition:
```bash theme={null}
cat > compact-task-definition.json <
### Auto Scaling (High-Availability Setup)
PowerSync API containers are limited to 200 concurrent connections each, with a recommended target of **100 connections or less per container** (see [Deployment Architecture](/maintenance-ops/self-hosting/deployment-architecture)). Because PowerSync sync connections are long-lived (hours or days), CPU utilization alone may not reflect the actual connection load — a container can be near its connection limit while CPU remains relatively low. For this reason, we recommend scaling on **both CPU utilization and concurrent connections**.
**ALB metrics are not suitable for PowerSync scaling.** Metrics like `ALBRequestCountPerTarget` track request rate (requests per second), but PowerSync sync connections are long-lived HTTP streams or WebSockets — a single request stays open for hours or days. Similarly, `ActiveConnectionCount` tracks total connections across the entire ALB, not per target. Use the `powersync_concurrent_connections` Prometheus metric instead.
#### Prerequisites
Connection-based auto-scaling requires:
1. **Prometheus metrics enabled** in your `service.yaml` (see [Step 1](#1-powersync-configuration)):
```yaml theme={null}
telemetry:
prometheus_port: 9090
```
2. **CloudWatch Agent sidecar** deployed in the API task definition (configured in [Step 6](#6-ecs-task-definition)). The sidecar scrapes the `powersync_concurrent_connections` metric from the PowerSync Prometheus endpoint and publishes it to CloudWatch under the `PowerSync` namespace.
3. **IAM permissions** for the task role to publish CloudWatch metrics (configured in [Step 6](#6-ecs-task-definition)).
#### Register Scalable Target
Set the minimum and maximum number of API tasks:
* **`min-capacity`**: We recommend at least **2** for high availability, ensuring your service stays available if one task fails. Auto-scaling handles load increases from there.
* **`max-capacity`**: Set this to the upper bound of tasks you want auto-scaling to provision.
```bash theme={null}
aws application-autoscaling register-scalable-target \
--service-namespace ecs \
--resource-id service/powersync-cluster/powersync-api \
--scalable-dimension ecs:service:DesiredCount \
--min-capacity 2 \
--max-capacity 10
```
**Choosing your minimum capacity:** A minimum of 2 works well for most workloads, letting auto-scaling adjust capacity as needed. However, if your traffic is very spiky (e.g., many users connecting simultaneously at a predictable time), you may want a higher `min-capacity` to avoid waiting for new tasks to start. New Fargate tasks take 1-3 minutes to launch and pass health checks, so a larger baseline reduces the risk of connection overload during sudden spikes. As a guideline, each API task handles up to 200 concurrent connections (target \~100 for headroom).
#### Scaling Policy 1: CPU Utilization
This policy scales based on average CPU utilization across API tasks:
```bash theme={null}
aws application-autoscaling put-scaling-policy \
--service-namespace ecs \
--resource-id service/powersync-cluster/powersync-api \
--scalable-dimension ecs:service:DesiredCount \
--policy-name cpu-scaling \
--policy-type TargetTrackingScaling \
--target-tracking-scaling-policy-configuration '{
"TargetValue": 70.0,
"PredefinedMetricSpecification": {"PredefinedMetricType": "ECSServiceAverageCPUUtilization"},
"ScaleInCooldown": 300,
"ScaleOutCooldown": 120
}'
```
#### Scaling Policy 2: Concurrent Connections
This policy scales based on the average number of concurrent sync connections per task, using the custom metric published by the CloudWatch Agent sidecar:
```bash theme={null}
aws application-autoscaling put-scaling-policy \
--service-namespace ecs \
--resource-id service/powersync-cluster/powersync-api \
--scalable-dimension ecs:service:DesiredCount \
--policy-name connection-scaling \
--policy-type TargetTrackingScaling \
--target-tracking-scaling-policy-configuration '{
"TargetValue": 80.0,
"CustomizedMetricSpecification": {
"MetricName": "powersync_concurrent_connections",
"Namespace": "PowerSync",
"Statistic": "Average"
},
"ScaleInCooldown": 300,
"ScaleOutCooldown": 120
}'
```
**How dual policies work:** Both policies operate independently — ECS scales to whichever policy demands the *higher* number of tasks. For example, if CPU-based scaling wants 3 tasks but connection-based scaling wants 5, ECS runs 5 tasks.
**Key configuration values:**
| Parameter | Value | Rationale |
| --------------------------- | ----- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `TargetValue` (connections) | 80 | 40% of the 200 max connection limit per container. This matches PowerSync Cloud's scaling strategy and provides headroom before the hard limit. |
| `TargetValue` (CPU) | 70.0 | Scale before CPU saturation impacts sync stream performance. |
| `ScaleOutCooldown` | 120s | New Fargate tasks take 1–3 minutes to start, pass health checks, and begin accepting connections. A shorter cooldown risks triggering multiple scale-out events before the first new task is ready. |
| `ScaleInCooldown` | 300s | Prevents rapid scale-in oscillations. When a task is removed, its clients reconnect to remaining tasks, causing a temporary connection spike. The cooldown allows this spike to settle. |
#### Scale-In Behavior
Scaling in (removing tasks) terminates active sync connections on the affected tasks. PowerSync client SDKs handle reconnection automatically, but there will be a brief interruption for affected clients.
**What happens during scale-in:**
1. ECS deregisters the task from the ALB target group — new connections are routed to other tasks
2. The ALB [deregistration delay](https://docs.aws.amazon.com/elasticloadbalancing/latest/application/load-balancer-target-groups.html#deregistration-delay) allows existing connections to drain (default: 300s). Since syncs never complete naturally, connections are forcefully closed after this timeout.
3. ECS sends `SIGTERM` to the container — PowerSync closes all active sync connections gracefully
4. After the `stopTimeout` period (configured to 120s in the task definition), ECS sends `SIGKILL`
5. Disconnected clients automatically reconnect to remaining healthy tasks
To adjust the deregistration delay:
```bash theme={null}
aws elbv2 modify-target-group-attributes \
--target-group-arn $TG_ARN \
--attributes Key=deregistration_delay.timeout_seconds,Value=300
```
#### Verify Auto-Scaling
After configuring both policies, verify they are active:
```bash theme={null}
# List scaling policies
aws application-autoscaling describe-scaling-policies \
--service-namespace ecs \
--resource-id service/powersync-cluster/powersync-api \
--query 'ScalingPolicies[*].[PolicyName,PolicyType,TargetTrackingScalingPolicyConfiguration.TargetValue]' \
--output table
# Check the custom metric is being published (may take a few minutes after deployment)
aws cloudwatch list-metrics \
--namespace "PowerSync" \
--metric-name "powersync_concurrent_connections"
# View scaling activity
aws application-autoscaling describe-scaling-activities \
--service-namespace ecs \
--resource-id service/powersync-cluster/powersync-api \
--query 'ScalingActivities[*].[StatusCode,Description,StartTime]' \
--output table
```
If you prefer not to set up the CloudWatch Agent sidecar and custom Prometheus metrics, you can scale based on CPU utilization alone:
```bash theme={null}
aws application-autoscaling register-scalable-target \
--service-namespace ecs \
--resource-id service/powersync-cluster/powersync-api \
--scalable-dimension ecs:service:DesiredCount \
--min-capacity 2 \
--max-capacity 10
# Use CPU-only scaling policy
aws application-autoscaling put-scaling-policy \
--service-namespace ecs \
--resource-id service/powersync-cluster/powersync-api \
--scalable-dimension ecs:service:DesiredCount \
--policy-name cpu-scaling \
--policy-type TargetTrackingScaling \
--target-tracking-scaling-policy-configuration '{
"TargetValue": 70.0,
"PredefinedMetricSpecification": {"PredefinedMetricType": "ECSServiceAverageCPUUtilization"},
"ScaleInCooldown": 300,
"ScaleOutCooldown": 120
}'
```
This approach is simpler but less responsive to connection spikes — CPU may not increase proportionally with new sync connections. Without connection-aware scaling, consider increasing `min-capacity` if your traffic is spiky, to provide a larger baseline while auto-scaling reacts.
## Troubleshooting
| Symptom | Solution |
| --------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Tasks fail health checks | Check logs: `aws logs tail /ecs/powersync --follow`
Helm charts for deploying PowerSync on Kubernetes. The repository is the source of truth for chart values, upgrade notes, and configuration reference.
## What the Charts Cover
The defaults in `values.yaml` follow the recommendations below. Override them only when your workload diverges.
### Workload Sizing
Start with the chart defaults and tune from there.
| Component | Baseline | When to Adjust |
| ---------------- | ----------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------ |
| API pods | 2+ replicas, 1 vCPU, 1Gi request / 2Gi limit | Client connections drive scaling. Let the HPA handle replica counts. Increase per-pod limits only if rows are unusually large. |
| Replication pods | 2 replicas (1 warm standby), 1 vCPU, 1Gi / 2Gi | Scale vertically as source database write throughput grows. Do not add more replicas, since only one ever replicates. |
| Compact | Daily `CronJob`, 100m / 1 vCPU, 512Mi / 1Gi | Schedule for off-peak hours. If runs start overlapping, increase memory before CPU. |
| Migrate | Helm pre-install/upgrade hook, 100m / 1 vCPU, 256Mi / 512Mi | Rarely needs tuning. |
Leave `NODE_OPTIONS=--max-old-space-size-percentage=80` as-is. V8 tracks the container limit automatically, so no recalculation is needed when you change `resources.limits.memory`.
### Observability
Wire monitoring up before you take traffic. Scrape Prometheus metrics from port `9464` and watch these signals:
| Metric | Why It Matters |
| ------------------------------------------ | ------------------------------------------------------------------ |
| `powersync_concurrent_connections` | Drives the HPA. Page when a pod nears the 200 connection hard cap. |
| `powersync_replication_lag_seconds` | Alert on sustained spikes. |
| `powersync_replication_storage_size_bytes` | Track bucket storage growth. Capacity-plan from the slope. |
| `powersync_operation_storage_size_bytes` | Track operation storage growth. Capacity-plan from the slope. |
| `powersync_data_sent_bytes_total` | Egress cost driver. |
See [Usage Reporting](/maintenance-ops/self-hosting/usage-reporting#what-is-collected) for the full metric catalog.
Use the file-based probes (`MICRO_PROBE_TYPE=fs`) shipped in the chart unless your platform requires HTTP probes. The chart bundles a `NetworkPolicy` (disabled by default via `networkPolicy.enabled: false`) that allows Prometheus scrapes on port `9464`. Enable it in production.
### Cluster Topology
* Run PowerSync in a dedicated namespace so RBAC, `NetworkPolicy`, and resource quotas stay scoped.
* Spread replication pods across nodes. The chart sets `podAntiAffinity` on the replication deployment as a soft preference so single-node clusters still schedule, but on multi-node clusters this matters. Putting the active and standby on the same node defeats the warm standby pattern.
* Keep `PodDisruptionBudgets` at `minAvailable: 1` to protect against node drains and upgrades. The chart skips the replication PDB automatically when `replicas: 1` so it does not block drains.
* Use `RollingUpdate` for both deployments at the default replica counts. If you drop replication to `replicas: 1`, override its strategy to `Recreate`.
### Ingress
* Use a dedicated subdomain such as `powersync.example.com`. PowerSync cannot share a host with other services.
* Use an NGINX-compatible ingress controller or any L7 with HTTP/2 and WebSocket support. Without HTTP/2, streaming sync multiplexing degrades.
* Keep the default annotations. `proxy-buffering: "off"` is required for streaming sync, and `proxy-read-timeout` and `proxy-send-timeout` of `3600` keep long-lived sync connections open.
* Terminate TLS at the ingress and reference a real certificate secret. The placeholder in `values.yaml` will not work in production.
### Scaling API Pods (Horizontal)
The API is stateless. Scale it out, not up.
* Target roughly 100 connections per pod with a hard cap of 200. Past 200 connections you will see `PSYNC_S2304` errors.
* The HPA template is bundled but disabled by default. Set `api.autoscaling.enabled: true` once you have the connections metric flowing.
* Bridge the Prometheus metric to the HPA using `prometheus-adapter` (the rule is in the chart README) or KEDA's Prometheus scaler.
* Keep the 70% CPU fallback. It catches load patterns the connection count alone misses, such as heavy queries or slow clients.
* Default scale-up stabilization of 60s reacts to traffic spikes. The 300s scale-down prevents flapping. Lengthen scale-down further if you see thrashing.
### Scaling Replication
A single PowerSync instance handles a maximum of roughly 50,000 to 100,000 concurrent clients depending on row size. Use this as a rough capacity-planning anchor.
Past that, run [multiple instances](/maintenance-ops/self-hosting/multiple-instances) by installing the chart again under a separate release name with its own bucket storage database. The same source database is fine across installs.
Pin clients to instances. Each instance maintains its own copy of bucket data, so a client switching instances forces a full resync. Either have your backend hand the client an endpoint for the PowerSync instance to connect to, or compute it deterministically (for example, `hash(user_id) % n`). Do not load-balance multiple instances behind one host.
### Networking
Open these flows before deploying:
| From | To | Protocol |
| ----------------- | ---------------------------------------- | ------------------ |
| Client | Ingress and API | HTTPS (long-lived) |
| API pods | Bucket storage database, JWKS endpoint | TCP / HTTPS egress |
| Replication pod | Source database, bucket storage database | TCP |
| Compact `CronJob` | Bucket storage database | TCP |
## Next Steps
* Review the [chart repository](https://github.com/powersync-community/powersync-helm-chart) for `values.yaml`, install instructions, and version compatibility.
* Read the [deployment architecture](/maintenance-ops/self-hosting/deployment-architecture) reference for the broader context behind these recommendations.
* Configure [monitoring](/maintenance-ops/self-hosting/monitoring) so the HPA has a connections metric to scale on.
# Deploy PowerSync Service on Coolify
Source: https://docs.powersync.com/maintenance-ops/self-hosting/coolify
Deploy the self-hosted PowerSync Service on Coolify with Docker Compose.
[Coolify](https://coolify.io/) is an open-source, self-hosted platform that simplifies the deployment and management of applications, databases, and services on your own infrastructure.
Think of it as a self-hosted alternative to platforms like Heroku or Netlify.
Before following this guide, you should:
* Read through the [Service Configuration](/configuration/powersync-service/self-hosted-instances)
guide to understand the requirements and configuration options. This guide assumes you have already done so, and will only cover the Coolify specific setup.
* Have Coolify installed and running.
## Background
For the PowerSync Service to function correctly, you will need:
* A database,
* Authentication service, and
* Data upload service.
The easiest way to get started is to use **Supabase** as it provides all three. However, you can also use a different database, and custom authentication and data upload services.
## Steps
Add the [`Compose file`](#base-compose-file) as a Docker Compose Empty resource to your project.
Update the environment variables and config files.
Click on the `Deploy` button to deploy the PowerSync Service.
## Configuration Options
The following configuration options should be updated:
* Environment variables
* `sync-config.yaml` file (according to your data requirements)
* `service.yaml` file
Environment Variable
Value
PS\_DATABASE\_TYPEpostgresql
PS\_DATABASE\_URI**Connection string obtained from Supabase**
PS\_PORT**Keep default value (8080)**
PS\_MONGO\_URImongodb://mongo:27017
PS\_JWKS\_URL**Keep default value**
```yaml {5} theme={null}
...
# Client (application end user) authentication settings
client_auth:
# Enable this if using Supabase Auth
supabase: true
...
```
Environment Variable
Value
PS\_DATABASE\_TYPEpostgresql OR mongodb OR mysql OR mssql
PS\_DATABASE\_URIThe database connection URI (according to your database type) where your data is stored.
PS\_PORT**Default value (8080)** You can change this if you want the PowerSync Service to be available on a different port.
PS\_MONGO\_URImongodb://mongo:27017
PS\_JWKS\_URLThe URL of the JWKS endpoint of your authentication service.
```yaml {5, 11-15,18, 23} theme={null}
...
# Client (application end user) authentication settings
client_auth:
# Enable this if using Supabase Auth
supabase: false
# JWKS URIs can be specified here
jwks_uri: !env PS_JWKS_URL
# Optional static collection of public keys for JWT verification
jwks:
keys:
- kty: 'oct'
k: 'use_a_better_token_in_production'
alg: 'HS256'
# JWKS audience
audience: ["powersync-dev", "powersync", "http://localhost:8080"]
api:
tokens:
# These tokens are used for local admin API route authentication
- use_a_better_token_in_production
```
## Base `Compose` File
The following Compose file serves as a universal starting point for deploying the PowerSync Service on Coolify.
```yaml theme={null}
services:
mongo:
image: mongo:7.0
command: --replSet rs0 --bind_ip_all --quiet
restart: unless-stopped
ports:
- 27017:27017
volumes:
- mongo_storage:/data/db
# Initializes the MongoDB replica set. This service will not usually be actively running
mongo-rs-init:
image: mongo:7.0
depends_on:
- mongo
restart: on-failure
entrypoint:
- bash
- -c
- 'mongosh --host mongo:27017 --eval ''try{rs.status().ok && quit(0)} catch {} rs.initiate({_id: "rs0", version: 1, members: [{ _id: 0, host : "mongo:27017" }]})'''
# PowerSync Service
powersync:
image: journeyapps/powersync-service:latest
container_name: powersync
depends_on:
- mongo-rs-init
command: [ "start", "-r", "unified"]
restart: unless-stopped
environment:
- NODE_OPTIONS="--max-old-space-size=1000"
- POWERSYNC_CONFIG_PATH=/home/config/service.yaml
- PS_DATABASE_TYPE=${PS_DEMO_BACKEND_DATABASE_TYPE:-postgresql}
- PS_DATABASE_URI=${PS_DATABASE_URI:-postgresql://postgres:postgres@localhost:5432/postgres}
- PS_PORT=${PS_PORT:-8080}
- PS_MONGO_URI=${PS_MONGO_URI:-mongodb://mongo:27017}
- PS_SUPABASE_AUTH=${USE_SUPABASE_AUTH:-false}
- PS_JWKS_URL=${PS_JWKS_URL:-http://localhost:6060/api/auth/keys}
ports:
- ${PS_PORT}:${PS_PORT}
volumes:
- ./volumes/config:/home/config
- type: bind
source: ./volumes/config/sync-config.yaml
target: /home/config/sync-config.yaml
content: |
config:
edition: 3
streams:
user_list_data:
# Sync all lists and todos for the authenticated user
auto_subscribe: true
queries:
- SELECT * FROM lists WHERE owner_id = auth.user_id()
- SELECT * FROM todos WHERE list_id IN (SELECT id FROM lists WHERE owner_id = auth.user_id())
- type: bind
source: ./volumes/config/service.yaml
target: /home/config/service.yaml
content: |
# yaml-language-server: $schema=../schema/schema.json
# Note that this example uses YAML custom tags for environment variable substitution.
# Using `!env [variable name]` will substitute the value of the environment variable named
# [variable name].
# migrations:
# # Migrations run automatically by default.
# # Setting this to true will skip automatic migrations.
# # Migrations can be triggered externally by altering the container `command`.
# disable_auto_migration: true
# Settings for telemetry reporting
# See https://docs.powersync.com/maintenance-ops/self-hosting/usage-reporting
telemetry:
# Opt out of reporting anonymized usage metrics to PowerSync telemetry service
disable_telemetry_sharing: false
# Settings for source database replication
replication:
# Specify database connection details
# Note only 1 connection is currently supported
# Multiple connection support is on the roadmap
connections:
- type: !env PS_DATABASE_TYPE
# The PowerSync server container can access the Postgres DB via the DB's service name.
# In this case the hostname is pg-db
# The connection URI or individual parameters can be specified.
# Individual params take precedence over URI params
uri: !env PS_BACKEND_DATABASE_URI
# Or use individual params
# hostname: pg-db # From the Docker Compose service name
# port: 5432
# database: postgres
# username: postgres
# password: mypassword
# SSL settings
sslmode: disable # 'verify-full' (default) or 'verify-ca' or 'disable'
# 'disable' is OK for local/private networks, not for public networks
# Required for verify-ca, optional for verify-full
# This should be the certificate(s) content in PEM format
# cacert: !env PS_PG_CA_CERT
# Include a certificate here for HTTPs
# This should be the certificate content in PEM format
# client_certificate: !env PS_PG_CLIENT_CERT
# This should be the key content in PEM format
# client_private_key: !env PS_PG_CLIENT_PRIVATE_KEY
# This is valid if using the `mongo` service defined in `ps-mongo.yaml`
# Connection settings for bucket storage
storage:
type: mongodb
uri: !env PS_MONGO_URI
# Use these if authentication is required. The user should have `readWrite` and `dbAdmin` roles
# username: my-mongo-user
# password: my-password
# The port which the PowerSync API server will listen on
port: !env PS_PORT
# Specify Sync Streams (or legacy Sync Rules)
sync_config:
path: /home/config/sync-config.yaml
# Client (application end user) authentication settings
client_auth:
# Enable this if using Supabase Auth
supabase: true
# JWKS URIs can be specified here
jwks_uri: !env PS_JWKS_URL
# Optional static collection of public keys for JWT verification
# jwks:
# keys:
# - kty: 'RSA'
# n: !env PS_JWK_N
# e: !env PS_JWK_E
# alg: 'RS256'
# kid: !env PS_JWK_KID
# JWKS audience
audience: ["powersync-dev", "powersync"]
api:
tokens:
# These tokens are used for local admin API route authentication
- use_a_better_token_in_production
```
# Deployment Architecture
Source: https://docs.powersync.com/maintenance-ops/self-hosting/deployment-architecture
Infrastructure and sizing recommendations for self-hosted PowerSync in staging and production.
## Minimal Setup
A minimal "development" setup (e.g. for a staging or a QA environment) is:
1. A single PowerSync "compute" container (API + replication) with 512MB memory, 1 vCPU.
2. A single MongoDB node in replica set mode, 2GB memory, 1 vCPU. M10+ when using Atlas.
3. Load balancer for TLS.
This setup has no redundancy. If the replica set fails, you may need to recreate it from scratch which will re-sync all clients.
## Production
For production, we recommend running a high-availability setup with the following baseline requirements:
1. 1x PowerSync replication container, 1GB memory, 1 vCPU
2. 2+ PowerSync API containers, 1GB memory each, 1vCPU each.
3. A 3-node MongoDB replica set, 2+GB memory each. Refer to the MongoDB documentation for deployment requirements. M10+ when using Atlas.
4. A load balancer with redundancy.
5. Run a daily compact job.
For further robustness, in cases where you have larger rows and higher load, we recommend increasing the memory and CPU requirements to 2GB memory and 2 vCPU. This applies to both the replication and API containers.
When scaling up, add 1x PowerSync API container per 100 connections. The MongoDB replica set should be scaled based on CPU and memory usage.
### Replication Container
The replication container handles replicating from the source database to PowerSync's bucket storage.
The replication process is run using the docker command `start -r sync`, for example `docker run powersync start -r sync`.
Only one process can replicate at a time. If multiple are running concurrently, you may see an error `[PSYNC_S1003] Sync rules have been locked by another process for replication`.
If you use rolling deploys, it is normal to see this error for a short duration while multiple processes are running.
Memory and CPU usage of the replication container is primarily driven by write load on the source database. A good starting point is 1GB memory and 1 vCPU for the container, but this may be scaled down depending on the load patterns.
Set the environment variable `NODE_OPTIONS=--max-old-space-size-percentage=80` to allocate 80% of the container's assigned memory to the Node.js heap. This avoids manually retuning when scaling resources up or down.
### API Containers
The API container handles streaming sync connections, as well as any other API calls.
The replication process is run using the docker command `start -r api`, for example `docker run powersync start -r api`.
Each API container is limited to 200 concurrent connections, but we recommend targeting 100 concurrent connections or less per container. This may change as we implement additional performance optimizations.
Memory and CPU usage of API containers are driven by:
1. Number of concurrent connections.
2. Number of buckets per connection.
3. Amount of data synced to each connection.
A good starting point is 1GB memory and 1 vCPU per container, but this may be scaled up or down depending on the specific load patterns.
Set the environment variable `NODE_OPTIONS=--max-old-space-size-percentage=80` to allocate 80% of the container's assigned memory to the Node.js heap. This avoids manually retuning when scaling resources up or down.
### Compact Job
We recommend running a compact job daily as a cron job, or after any large maintenance jobs. For details, see the documentation on [Compacting Buckets](/maintenance-ops/compacting-buckets).
Run the compact job using the docker command `compact`, for example `docker run powersync compact`.
The compact job uses up to 1GB memory for compacting, if available. Set the environment variable `NODE_OPTIONS=--max-old-space-size-percentage=80` to allocate 80% of the container's assigned memory to the Node.js heap. This avoids manually retuning when scaling resources up or down.
### Load Balancer
A load balancer is required in front of the API containers to provide TLS support and load balancing. Most cloud providers have built-in options for load balancing, such as ALB on AWS.
It is currently required to host the API container on a dedicated subdomain - we do not support running it on the same subdomain as another service.
For self-hosting, [nginx](https://nginx.org/en/) is always a good option. A basic nginx configuration could look like this:
```yaml theme={null}
server {
listen 443 ssl;
server_name powersync.example.org;
# SSL configuration here
# Reverse proxy settings
location / {
proxy_pass http://powersync_server_ip:powersync_port; # Replace with your powersync details
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
# Disable proxy response buffering.
# This is not relevant for websocket connections, but is important when using
# HTTP streaming connections (configured in the PowerSync Client SDK).
proxy_buffering off;
}
}
```
When using nginx as a Kubernetes ingress, set the proxy buffering option as an annotation on the ingress:
```yaml theme={null}
nginx.ingress.kubernetes.io/proxy-buffering: "off"
```
### Health Checks
If the load balancer supports health checks, it may be configured to poll the API container at `/probes/liveness`. This endpoint is expected to have a 200 response when the container is healthy. See [Health Checks](/maintenance-ops/self-hosting/healthchecks) for details.
### Migrations
Occasionally, new versions of the PowerSync Service image may require migrations on the underlying storage database. This is also specifically required the first time the service starts up on a new storage database.
By default, migrations are run as part of the replication and API containers. In some cases, a migration may add significant delay to the container startup.
To avoid this startup delay, the migrations may be run as a separate job on each update, before replacing the rest of the containers. To run the migrations, run the docker command `migrate up`, for example `docker run powersync migrate up`.
In this case, disable automatic migrations in the config:
```yaml theme={null}
# service.yaml
migrations:
# Setting this to false (default) enables automatic migrations on startup.
# When set to true, migrations must be triggered manually by modifying the container `command`.
disable_auto_migration: true
```
Note that if you disable automatic migrations, and do not run the migration job manually,
the service may run with an outdated storage schema version. This may lead to unexpected and potentially difficult-to-debug errors in the service.
## Backups
We recommend using Git to back up your configuration files.
None of the containers use any local storage, so no backups are required there.
The bucket storage database may be backed up using the recommendations for the storage database system. This is not a strong requirement, since this data can be recovered by re-replicating from the source database.
## Self-Hosted Architecture Diagram
Use a secure, randomly generated API token.
2. Restart the PowerSync Service.
3. Send a POST request to the diagnostics endpoint, passing the token as a Bearer token:
```shell theme={null}
curl -X POST http://localhost:8080/api/admin/v1/diagnostics \
-H "Authorization: Bearer YOUR_API_TOKEN"
```
### Response Structure
The response `data` object contains:
**`connections`** — whether PowerSync can reach the configured source database and any connection-level errors.
**`active_sync_rules`** — the currently serving Sync Config (Sync Streams or Sync Rules). Contains a `connections[]` array with details about each replication connection including slot name, WAL status, and tables being replicated. Also includes an `errors[]` array for warnings or errors.
**`deploying_sync_rules`** — only present while a new Sync Config is being deployed and the initial replication is in progress. PowerSync runs this process in parallel so clients continue to be served by the existing active config. Once initial replication completes, this section disappears and `active_sync_rules` updates.
Each connection in `active_sync_rules.connections[]` includes:
| Field | Description |
| -------------------------- | ----------------------------------------------------------------------------------------- |
| `slot_name` | The name of the Postgres replication slot used by this Sync Rules version. |
| `initial_replication_done` | Whether the initial snapshot has completed. |
| `replication_lag_bytes` | Replication lag in bytes. |
| `wal_status` | The WAL status of the replication slot (`reserved`, `extended`, `unreserved`, or `lost`). |
| `safe_wal_size` | Remaining WAL budget in bytes before the slot risks invalidation. |
| `max_slot_wal_keep_size` | The configured `max_slot_wal_keep_size` value on the source Postgres database. |
### Errors and Warnings
Warnings and errors appear in the `errors[]` array at the Sync Rules level (`active_sync_rules.errors[]` or `deploying_sync_rules.errors[]`). This includes:
* **Replication lag warnings** are raised if no replicated commit has been received in more than 5 minutes (warning level) or 15 minutes (fatal level).
* **WAL budget warnings** appear when the remaining WAL budget drops below 50%.
* **Replication errors** such as `PSYNC_S1146` appear when a replication slot is invalidated (when `wal_status` is `lost`).
For guidance on configuring `max_slot_wal_keep_size` and managing replication slots, see [Postgres maintenance](/configuration/source-db/postgres-maintenance).
# Health Checks
Source: https://docs.powersync.com/maintenance-ops/self-hosting/healthchecks
Configure health check endpoints for self-hosted PowerSync Service deployments.
## Overview
PowerSync Service provides health check endpoints and configuration options to help you monitor the health and readiness of your deployment. These checks allow you to catch issues before they impact your users.
## Health Check Endpoints
The following HTTP endpoints are available:
* **Startup Probe:**\
`GET /probes/startup`
* `200` – Service has started up correctly
* `400` – Service has **not** yet started
* **Liveness Probe:**\
`GET /probes/liveness`
* `200` – Service is alive
* `400` – Service is **not** alive
## Example: Docker Health Checks
A configuration with Docker Compose might look like:
```yaml theme={null}
healthcheck:
test: ["CMD", "node", "-e", "fetch('http://localhost:${PS_PORT}/probes/liveness').then(r => r.ok ? process.exit(0) : process.exit(1)).catch(() => process.exit(1))"]
interval: 5s
timeout: 1s
retries: 15
```
You can find a complete example in the [self-host-demo app](https://github.com/powersync-ja/self-host-demo/blob/main/services/powersync.yaml).
## Advanced: Configurable Health Check Probes (v1.12.0+)
Starting with version **1.12.0**, PowerSync Service supports configurable health check probes.\
You can now choose between filesystem-based and HTTP-based probes, or use both, via the config file. This is especially useful for environments with restricted I/O.
**Configuration options:**
```yaml theme={null}
healthcheck:
probes:
use_filesystem: true # Enables filesystem-based health probes
use_http: true # Enables HTTP-based health probes
```
If no `healthcheck` configuration is provided, the service defaults to the previous behavior for backwards compatibility.
# Migrating Between Instances
Source: https://docs.powersync.com/maintenance-ops/self-hosting/migrating-instances
Migrate users between self-hosted PowerSync instances with minimal downtime.
## Overview
In some cases, you may want to migrate users between PowerSync instances. This may be between cloud and self-hosted instances, or even just to change the endpoint.
If the PowerSync instances use the same source database and have the same basic configuration and Sync Streams (or legacy Sync Rules), you can migrate users by just changing the endpoint to the new instance.
To make this process easier, we recommend using an API to retrieve the PowerSync endpoint, instead of hardcoding the endpoint in the client application. If you're using custom authentication, this can be done in the same API call as getting the authentication token.
There should be no downtime for users when switching between endpoints. The client will have to re-sync all data, but this will all happen automatically, and the client will atomically switch between the two. The main effect visible to users will be a delay in syncing new data while the client is re-syncing. All data will remain available to read on the client for the entire process.
# Monitoring
Source: https://docs.powersync.com/maintenance-ops/self-hosting/monitoring
Monitor a self-hosted PowerSync Service by scraping metrics from a Prometheus-compatible endpoint.
## Metrics Endpoint
PowerSync exposes instance metrics via a Prometheus-compatible endpoint. This allows you to integrate with Prometheus or other monitoring systems that scrape Prometheus endpoints.
It's not recommended to scrape the Prometheus endpoint manually, we suggest using Prometheus or other compatible tools. PowerSync does not currently support pushing to OpenTelemetry collectors.
#### Configuration
1. To enable metrics, update your PowerSync YAML file to include the `prometheus_port` and set a port number.
```yaml service.yaml theme={null}
telemetry:
# Set the port at which the Prometheus metrics will be exposed
prometheus_port: 9090
```
2. Update your Docker compose file to forward the `prometheus_port`.
```yaml docker-compose.yaml theme={null}
ports:
# Forward port 8080 for the PowerSync Service
- 8080:8080
# Forward port 9090 for Prometheus metrics
- 9090:9090
```
Once enabled, restart the service and the metrics endpoint will return Prometheus-formatted metrics, as described in the [What is Collected](/maintenance-ops/self-hosting/usage-reporting#what-is-collected) section of the [Usage Reporting](/maintenance-ops/self-hosting/usage-reporting) docs.
If you're running multiple containers (e.g. splitting up replication containers and API containers) you need to scrape the metrics separately for each container.
# Multiple PowerSync Instances
Source: https://docs.powersync.com/maintenance-ops/self-hosting/multiple-instances
Run multiple self-hosted PowerSync instances for load distribution and high availability.
## Overview
Multiple instances are not required in most cases. See the [Deployment Architecture](/maintenance-ops/self-hosting/deployment-architecture) for details on standard horizontal scaling setups.
When exceeding a couple thousand concurrent clients, the standard PowerSync setup may not scale sufficiently to handle the load. In this case, we recommend you [contact us](/resources/contact-us) to discuss the options. However, we give a basic overview of using multiple PowerSync instances to scale here.
Each PowerSync "instance" is a single endpoint (URL), that is backed by:
1. One replication container.
2. Multiple API containers, scaling horizontally.
3. One bucket storage database.
This setup is described in the [Deployment Architecture](/maintenance-ops/self-hosting/deployment-architecture).
To scale further, multiple copies of this setup can be run, using the same source database.
## Mapping Users to PowerSync Endpoints
Since each PowerSync instance maintains its own copy of the bucket data, the exact list of operations and associated checksum will be different between them. This means the same client must connect to the same endpoint every time, otherwise they will have to re-sync all their data every time they switch. Multiple PowerSync instances cannot be load-balanced behind the same subdomain.
To ensure the same user always connects to the same endpoint, we recommend:
1. Do an API lookup from the client application to get the PowerSync endpoint, don't hardcode it in the application.
2. Either store the endpoint associated with each user, or compute it automatically using a hash function on the user id e.g. `hash(user_id) % n` where `n` is your number of instances.
# Self-Hosting Maintenance & Ops
Source: https://docs.powersync.com/maintenance-ops/self-hosting/overview
Production and operations guides for self-hosted PowerSync deployments.
The [PowerSync CLI](/tools/cli) provides commands that work alongside any running self-hosted instance: `powersync status`, `powersync validate`, `powersync generate schema`, `powersync generate token`. You don't need to have set up the instance with the CLI to use these.
## Production Topics
Details for production self-hosted PowerSync deployments, including architecture/setup recommendations, security, health checks, maintenance, and monitoring.
## Deployment Platforms
Guides for deploying self-hosted PowerSync on common platforms:
Coolify is an open-source & self-hostable alternative to Heroku / Netlify / Vercel / etc.
Railway is a managed cloud platform (PaaS) for deploying and scaling applications, services, and databases via containers.
Amazon Elastic Container Service (ECS) is a fully managed container orchestration service.
Amazon Elastic Kubernetes Service (EKS) is a managed Kubernetes service for running containerized workloads.
# Railway + PowerSync
Source: https://docs.powersync.com/maintenance-ops/self-hosting/railway
Deploy the PowerSync Service with a custom backend on Railway.
[Railway](https://railway.com/) is a managed cloud platform (PaaS) for deploying and scaling applications, services, and databases via containers.
## Step 1: Deploy on Railway
Find the "PowerSync Starter (Postgres)" template on the Railway Marketplace, or click the button below to get started:
The starter template will deploy and boot with a default configuration for all of the services. You can always add more services to your project as you need them and update the configuration as you go.
Once you've opened the deployed project in Railway, you'll see the following services:
* **PowerSync Service**: The PowerSync Service is responsible for syncing data between the Postgres database and your client applications.
* **Postgres Source Data**: The Postgres source data is the database that contains your application data.
* **Postgres (PowerSync Bucket Storage)**: The Postgres (PowerSync Bucket Storage) is the database that contains your PowerSync bucket data.
* **Demo Node.js Backend**: The Node.js backend is the backend for your application and is responsible for generating JWT tokens and API requests that handle upload events from a connected client. Note that this backend is not secured at all, and is intended to be used for demo purposes only.
* **Sync Diagnostics Client**: The Sync Diagnostics Client is a web app that implements the [PowerSync Web SDK](/client-sdks/reference/javascript-web) and allows you to test your PowerSync connection and see the data that is being synced.
* **Execute Scripts**: The Execute Scripts service is used to apply schema changes to your PowerSync Postgres instance.
This template automatically creates `lists` and `todos` tables in your Postgres database. The default Sync Rules are configured to sync these tables to your clients.
The `Execute Scripts` service creates the **powersync** publication for these tables. We recommend limiting the publication to only the tables you want clients to download.
Once you're up and running with the default `lists` and `todos` tables, you can add more tables at any time using either of these approaches:
**Option 1: Use your existing Postgres tools**
Manage your database schema as you normally would. For example, using `psql`:
```shell theme={null}
psql $POSTGRES_URL <<'SQL'
CREATE TABLE notes (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
list_id UUID REFERENCES lists(id) ON DELETE CASCADE,
content TEXT,
created_at TIMESTAMP DEFAULT now()
);
CREATE PUBLICATION powersync FOR TABLE notes;
SQL
```
**Option 2: Use the Execute Scripts service**
The Execute Scripts service can also be used as a general-purpose tool to apply schema changes to your PowerSync Postgres instance:
1. Add your new table creation statements and publication updates to the Execute Scripts code
2. Redeploy the Execute Scripts service
**After adding tables with either option:**
1. Update the Sync Rules to include the new tables in the `sync_rules` section of your YAML config.
2. Re-encode the YAML config to base64 and update the `POWERSYNC_CONFIG_B64` environment variable. See [Understanding the PowerSync Service Configuration](#understanding-the-powersync-service-configuration) for more details.
## Step 2: Test with the Sync Diagnostics Client
1. Generate a development token
* The `Demo Node.js Backend` service has a `/api/auth/token` endpoint you can hit to get a development JWT token. You can use this endpoint to generate a development token.
* You can also generate a development token by following the [Generate development token](/configuration/auth/development-tokens#self-hosted) tutorial.
2. Open the `Sync Diagnostics Client` service in the browser.
3. Paste your token to test your connection and Sync Rules
## Step 3: Connect Your Client
Follow our guide to connect your app to your backend and PowerSync instance. See our guide to learn more about how to implement your client-side application.
## Step 4: Implement Your Backend
PowerSync is designed to integrate with your existing backend infrastructure for persisting client mutations to your backend source database. See our guide to learn more about how to implement your backend application.
## Understanding the PowerSync Service Configuration
The PowerSync Service configuration is written in YAML and converted to base64 to be used as an environment variable.
If you need to make changes to the configuration, you can copy and edit the example YAML file below, [base64 encode](https://www.base64encode.org/) it, and update the `POWERSYNC_CONFIG_B64` environment variable in the `PowerSync Service` service. This will be required if you need to update the Sync Rules of your project.
```yaml service.yaml theme={null}
replication:
connections:
- type: postgresql
uri: !env PS_POSTGRES_SOURCE_URL
sslmode: disable
storage:
type: postgresql
uri: !env PS_POSTGRES_BUCKET_URL
sslmode: disable
port: 80
sync_config:
content: |
bucket_definitions:
global:
data:
- SELECT * FROM lists
- SELECT * FROM todos
client_auth:
jwks_uri: !env PS_AUTH_JWKS
audience:
- !env PS_AUTH_AUD
```
# Securing Your Deployment
Source: https://docs.powersync.com/maintenance-ops/self-hosting/securing-your-deployment
Network architecture and security guidance for self-hosted PowerSync deployments.
This section is a work in progress. Please reach out on [our Discord](https://discord.gg/powersync) if you have specific questions.
Below is an architecture diagram of a successful deployment:
During deployment, existing Sync Streams/Sync Rules continue serving clients while new Sync Config processes. Clients seamlessly transition once [initial replication](/architecture/powersync-service#initial-replication-vs-incremental-replication) completes.
Run `powersync validate` in the CLI before deploying to catch errors in your Sync Config without applying changes.
## Option 1: CLI
If you set up PowerSync using the CLI (`powersync docker`), update your Sync Config and apply it without a full service restart:
Update `powersync/sync-config.yaml` in your project directory.
```bash theme={null}
powersync validate
```
```bash theme={null}
powersync docker reset
```
This restarts the PowerSync Service and applies your updated Sync Config.
## Option 2: Config File
Define your Sync Config in `service.yaml` either inline or via a separate file. See [Self-Hosted Instance Configuration](/configuration/powersync-service/self-hosted-instances) for the full config reference.
Update the `sync_config:` section in your `service.yaml`. The `sync_config:` key is used for both Sync Streams and Sync Rules:
```yaml Sync Streams — Separate File (Recommended) theme={null}
sync_config:
path: sync-config.yaml
```
```yaml Sync Streams — Inline theme={null}
sync_config:
content: |
config:
edition: 3
streams:
users:
auto_subscribe: true
query: SELECT * FROM public.users
```
```yaml Sync Rules — Separate File (Legacy) theme={null}
sync_config:
path: sync-config.yaml
```
```yaml Sync Rules — Inline (Legacy) theme={null}
sync_config:
content: |
bucket_definitions:
global:
data:
- SELECT * FROM public.users
```
Restart your service to apply changes:
```shell theme={null}
docker compose restart powersync
```
Once the service starts up, it will load the updated Sync Config and begin processing it while continuing to serve the existing config until initial replication completes.
## Option 3: Deploy via API
Deploy Sync Config at runtime without restarting. Useful for quick iterations during development.
The API is disabled when Sync Streams (or legacy Sync Rules) are defined in `service.yaml`.
Sync Streams/Sync Rules defined in `service.yaml` always take precedence.
Add an API token to your `service.yaml` and restart:
```yaml service.yaml theme={null}
api:
tokens:
- !env PS_API_TOKEN
```
```shell theme={null}
curl -X POST http://:/api/sync-rules/v1/deploy \
-H "Content-Type: application/yaml" \
-H "Authorization: Bearer ${PS_API_TOKEN}" \
-d @sync-rules.yaml
```
Use `/api/sync-rules/v1/validate` first to check for errors without deploying.
### Additional Endpoints
| Endpoint | Method | Description |
| ------------------------------ | ------ | ------------------------------------------------ |
| `/api/sync-rules/v1/current` | GET | Get active and pending Sync Streams / Sync Rules |
| `/api/sync-rules/v1/reprocess` | POST | Restart replication from scratch |
## Troubleshooting
Common errors when using the API:
| Error Code | Meaning |
| ------------- | ------------------------------------------------------------------ |
| `PSYNC_S4105` | Sync Streams / Sync Rules defined in config file - API is disabled |
| `PSYNC_S4104` | No Sync Streams / Sync Rules deployed yet |
| `PSYNC_R0001` | Invalid Sync Streams / Sync Rules YAML - check `details` field |
See [Error Codes Reference](/debugging/error-codes) for the complete list.
# Usage Reporting
Source: https://docs.powersync.com/maintenance-ops/self-hosting/usage-reporting
How self-hosted PowerSync collects anonymous usage metrics, and how to opt out.
## Overview
PowerSync uses OpenTelemetry to gather certain usage metrics. These metrics are shared with PowerSync unless you opt-out. This allows us to gauge adoption and usage patterns across deployments so that we can better allocate R\&D capacity and ultimately better serve our customers (including you!). The metrics are linked to a random UUID and are therefore completely anonymous.
## What is Collected
Below are the data points collected every few minutes and associated with a random UUID representing your instance:
Type definitions for each metric dimension are available in the [powersync-service](https://github.com/powersync-ja/powersync-service/blob/main/packages/types/src/metrics.ts) repository.
| Dimension | Type |
| ------------------------------------- | ------- |
| data\_replicated\_bytes | counter |
| data\_synced\_bytes | counter |
| rows\_replicated\_total | counter |
| transactions\_replicated\_total | counter |
| chunks\_replicated\_total | counter |
| operations\_synced\_total | counter |
| replication\_storage\_size\_bytes | gauge |
| attributed\_source\_records\_bytes | gauge |
| operation\_storage\_size\_bytes | gauge |
| attributed\_bucket\_data\_bytes | gauge |
| parameter\_storage\_size\_bytes | gauge |
| attributed\_parameter\_indexes\_bytes | gauge |
| object\_storage\_size\_bytes | gauge |
| attributed\_object\_storage\_bytes | gauge |
| concurrent\_connections | gauge |
The four `attributed_*` gauges include `sync_config_id` and `sync_config_state` attributes.
To scrape your self-hosted PowerSync Service metrics, please see the [Monitoring](/maintenance-ops/self-hosting/monitoring) docs page for more details.
### Opting Out
To opt out of sharing metrics with PowerSync, set the `disable_telemetry_sharing` key in your [`service.yaml`](/configuration/powersync-service/self-hosted-instances) to `true`:
```yaml service.yaml theme={null}
telemetry:
# Opt out of reporting anonymized usage metrics to PowerSync telemetry service
disable_telemetry_sharing: true
```
# MongoDB Atlas Device Sync Migration Guide
Source: https://docs.powersync.com/migration-guides/atlas-device-sync
Migrate from MongoDB Atlas Device Sync to PowerSync with step-by-step instructions.
## Introduction
Migrating from the discontinued MongoDB Atlas Device Sync to PowerSync allows you to benefit from efficient data sync using open and proven technologies. Users get always-available, instantly-responsive offline-first apps that also stream data updates in real-time when online.
## Why PowerSync?
PowerSync’s [history](https://www.powersync.com/company) goes as far back as 2009, when the original version of the sync engine was developed as part of an app development platform used by some of the world’s largest industrial companies to power offline-capable apps deployed in harsh environments.
PowerSync was spun off as a standalone product in 2023, and gives engineering teams a proven, open and robust sync engine with a familiar server-client [architecture](/architecture/architecture-overview).
PowerSync’s MongoDB connector has been developed in collaboration with MongoDB to provide an easy setup process. It reached General Availability (GA) status with its [V1 release](https://www.powersync.com/blog/powersyncs-mongodb-connector-hits-ga-with-version-1-0) and is fully supported for production use. Multiple MongoDB customers currently use PowerSync in production environments.
The server-side [PowerSync Service](/architecture/powersync-service) connects to MongoDB and pre-processes and pre-indexes data to be efficiently synced to users based on defined *Sync Streams* (or legacy *Sync Rules*). Client applications embedding the *PowerSync Client SDK* connect to the PowerSync Service to sync only a relevant subset of data to each user, based on the Sync Streams/Rules. Incremental updates in MongoDB are synced to clients in real-time.
Client applications get a SQLite database that they can read from and write to. PowerSync provides for bi-directional syncing so that mutations in the client-side SQLite database are automatically synced back to the source MongoDB database. If users are offline or have patchy connectivity, PowerSync automatically manages network failures and retries.
By introducing PowerSync as a sync engine, you get:
* **Predictable sync behavior** that syncs relevant data to each user.
* **Instantly responsive user experience** as the app works with a zero-latency SQLite database.
* **Consistency guarantees** ensuring consistent state of the client-side SQLite database.
* **Real-time multi-user applications** as data updates are streamed to connected clients in real-time.
* **Offline-first capabilities** enabling apps to continue to work regardless of network conditions.
If you need further assistance at any point, you can:
* **Ask AI** (see lower right corner of this site), which is trained on all our documentation, repositories and Discord discussions.
* [Set up a call](https://calendly.com/powersync/powersync-chat) with us.
* Ask us anything on our [Discord server](https://discord.gg/powersync).
* [Contact us](mailto:hello@powersync.com) through email.
## Architecture: Before and After
If you have MongoDB Atlas Device Sync deployed today, at a high level your architecture will look something like this:
It is not necessary to remove Realm in order to install PowerSync. It is possible to initially run Realm and PowerSync in parallel, and remove Realm once PowerSync has been set up.
### 1. Follow the PowerSync Setup Guide
Follow the steps for MongoDB and your client platform/framework in our standard [Setup Guide](/intro/setup-guide):
* [Configure Your Source Database](/intro/setup-guide#1-configure-your-source-database)
* [Set Up PowerSync Service Instance](/intro/setup-guide#2-set-up-powersync-service-instance)
* [Connect PowerSync To Your Source Database](/intro/setup-guide#3-connect-powersync-to-your-source-database) (MongoDB)
* [Define Sync Streams](/intro/setup-guide#4-define-sync-streams)
* [Generate a Development Token](/intro/setup-guide#5-generate-a-development-token)
* [Test Sync with the Sync Diagnostics Client](/intro/setup-guide#6-optional-test-sync-with-the-sync-diagnostics-client)
* [Use the Client SDK](/intro/setup-guide#7-use-the-client-sdk)
* [Install the Client SDK](/intro/setup-guide#install-the-client-sdk)
* [Define Your Client-Side Schema](/intro/setup-guide#define-your-client-side-schema)
* [Instantiate the PowerSync Database](/intro/setup-guide#instantiate-the-powersync-database)
* [Connect to PowerSync Service Instance](/intro/setup-guide#connect-to-powersync-service-instance)
* [Read and Write Data (Using SQLite)](/intro/setup-guide#read-data)
For specific details on working with embedded documents and arrays from MongoDB, see our guide on [Querying JSON Data in SQLite](/client-sdks/advanced/query-json-in-sqlite)
Once you have completed the *Setup Guide*, the only two remaining steps are to configure & integrate a backend application to handle mutations uploaded from clients, and to implement authentication.
### 2. Accept Uploads on the Backend
MongoDB Atlas Device Sync provides built-in writes/uploads to the MongoDB database.
PowerSync offers [full customizability](/architecture/client-architecture#writing-data-via-sqlite-database-and-upload-queue) regarding how writes (mutations) are applied to the source MongoDB database, via your own application backend. This gives you control to apply your own business logic, data validations, authorization and conflict resolution logic.
There are two options:
* **"Bring your own backend"**: If you already have a backend application as part of your stack, you should use your existing backend. This can be any kind of backend environment including a custom backend (e.g. Node.js, Rails, Laravel, Django, ASP.NET), an API platform (e.g. Hasura), some kind of serverless cloud functions (e.g. Azure Functions, AWS Lambda, Google Cloud Functions, Cloudflare Workers, etc.), or any other equivalent system that allows you to run privileged logic securely or apply mutations to your MongoDB database securely. If you don’t yet have a backend application, but would like to run your own backend environment, we have example implementations available (see below).
* **Serverless cloud functions (hosted/managed)**: PowerSync offers serverless cloud functions hosted on the same infrastructure as PowerSync Cloud which can be used for the needed backend functionality. We provide a MongoDB-specific template for this which can be used as a turnkey solution.
#### Using Your Own Custom Backend API
This option gives you complete control over the backend. The simplest backend implementation is to apply mutations to MongoDB as they are received, which results in a last-write-wins conflict resolution strategy. See [App Backend Setup](/configuration/app-backend/setup) and [Writing Client Changes](/handling-writes/writing-client-changes) for more details.
We have [example backend implementations](/intro/examples#backend-examples) available (e.g. Node.js, Django, Rails, .NET).
The [Migrating A MongoDB Atlas Device Sync App To PowerSync](https://www.powersync.com/blog/migrating-a-mongodb-atlas-device-sync-app-to-powersync) practical example on our blog also provides an example of a custom Node.js backend implementation.
On the client-side, you need to wire up the `uploadData()` function in the "backend connector" to use your own backend API. The [Client-Side Integration With Your Backend](/configuration/app-backend/client-side-integration) section of our docs provides more details on this.
#### Using PowerSync’s Serverless Cloud Functions
PowerSync provides serverless cloud functions for backend functionality, with a template available for MongoDB. See the [step-by-step instructions](/configuration/app-backend/cloudcode) on how to use the template. The template can be customized, or it can be used as-is.
The template provides [turnkey conflict resolution](https://www.powersync.com/blog/turnkey-backend-functionality-conflict-resolution-for-powersync#turnkey-conflict-resolution) which roughly matches the built-in conflict resolution behavior provided by MongoDB Atlas Device Sync.
PowerSync's serverless cloud functions require a bit of "white glove" assistance from our team. If you want to use this option, please [get in touch with us](https://www.powersync.com/contact) so we can get you set up.
For more information, see our blog post: [Turnkey Backend Functionality & Conflict Resolution for PowerSync](https://www.powersync.com/blog/turnkey-backend-functionality-conflict-resolution-for-powersync).
### 3. Set Up Authentication Integration
For quick development and testing purposes, the *Setup Guide* from step 1 instructs you to generate a temporary development token to use for authentication.
At some point you will need to replace the development tokens with proper JWT-based authentication integration. PowerSync supports various authentication providers including Supabase, Firebase Auth, Auth0, Clerk, and custom JWT implementations.
The [Authentication Setup](/configuration/auth/overview) section of our docs provides full details on this.
## Questions? Need help?
* **Ask AI** (see lower right corner of this site), which is trained on all our documentation, repositories and Discord discussions.
* [Get in touch](https://www.powersync.com/contact) with us.
# Electric Cloud Migration Guide
Source: https://docs.powersync.com/migration-guides/electric
Migrate from Electric Cloud to PowerSync Cloud with step-by-step instructions.
Electric [announced](https://electric.ax/blog/2026/08/11/electric-joining-databricks) that it is joining Databricks and that Electric Cloud is shutting down. Their official guidance is to move to either self-hosted Electric or another provider. This guide is for users looking to migrate to an alternative cloud provider of a Postgres-backed sync engine, namely PowerSync.
This is an implementation guide for teams already using Electric Cloud's current Postgres Sync service. It assumes Postgres remains the system of record and covers high-level concepts to assist with a migration.
## Concepts
PowerSync Sync Streams are analogous to Electric Shapes, not Electric Streams. Electric Streams is a separate product and is out of scope for this guide.
While Electric Shapes and PowerSync Sync Streams both control partial sync, there are significant differences meaning that Shape to Sync Stream migration will most likely not be 1-for-1, and re-designing partial sync will most likely be required.
A key architectural difference between Electric Shapes and Sync Streams is that Shapes are created on demand from the client, per request, while Sync Streams are declared as named queries ahead of time that clients then dynamically subscribe to when needed, each subscription passing its own parameters. This architecture drives how authorization, offline behavior, and writes work. The table below summarizes the specific differences between Electric Shapes and Sync Streams across these and other key dimensions.
| Dimension | Electric (Shapes) | PowerSync (Sync Streams) |
| ------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Definition | Client-side, per request. Using `table` and optional `where`, `columns`, and `queryable_columns` parameters. Each parameter set identifies a distinct Shape. | Server-side, names SQL queries in a deployed YAML config. Clients subscribe by name and supply parameters; `auto_subscribe` starts a stream automatically. |
| Relational scope | Single table only. Use subqueries to filter, or multiple Shapes to sync related data. | JOIN supported. One stream can carry multiple queries. |
| Filter language | Supported Postgres WHERE expressions, positional parameters, arrays, and subqueries. No non-deterministic functions. \[[ref](https://electric.ax/docs/sync/guides/shapes#supported-operators)] | SQL subset with inner joins, subqueries, CTEs, and transformations; no GROUP BY, ORDER BY, LIMIT, or UNION. No non-deterministic functions. \[[ref](/sync/supported-sql)] |
| Authorization model | "Production apps should request shapes through your backend API for authorization and security". \[[ref](https://electric.ax/docs/sync/guides/shapes#defining-shapes)] | Sync Stream queries define download access using signed JWT claims. Treat client-provided subscription parameters as untrusted. |
| Client store | In-memory rows by default. Persistence and offline behavior require a separate client store or integration, such as PGlite or TanStack DB collections. | SQLite with a declared AppSchema. The Web SDK uses persistent IndexedDB by default; additional VFSes are available, and an in-memory VFS is optional. |
| Local queries | `shape.rows`, `useShape`, TanStack DB live queries. | SQL against SQLite (`useQuery`, `watch()`). Also supports the TanStack DB interface and SQLite ORMs like Drizzle. |
| Writes | Electric provides read-path sync only. The app defines its write path. | Local INSERT, UPDATE and DELETE operations enter a FIFO upload queue. Developer-defined `uploadData()` applies mutations via the developers' existing backend, which authorizes and applies them. |
| Primary keys and types | Must always include Postgres primary key columns. | Each synced table requires a single unique text `id` column. Columns can be aliased, concatenated or cast \[[ref](/sync/advanced/client-id)]. The PowerSync TanStack DB integration provides automatic support for rich types. \[[ref](https://tanstack.com/db/latest/docs/collections/powersync-collection)] |
| Performance consideration | Shape filter design affects throughput. \[[ref](https://electric.ax/docs/sync/guides/shapes#throughput)] | Bucket cardinality affects sync performance. \[[ref](/sync/rules/organize-data-into-buckets#limit-on-number-of-buckets-per-client)] |
## Migration
Both Electric Sync and PowerSync codebases are public, so agents should generally not struggle with implementing migrations. Also see the PowerSync [agent resources](/tools/ai-tools). If you need assistance, join the [PowerSync Discord](https://discord.gg/powersync).
### Overview of Migration Steps
1. [Configure Postgres](#configure-postgres)
2. [Connect PowerSync to Postgres](#connect-powersync-to-postgres)
3. [Define and Test Sync Streams](#define-and-test-sync-streams)
4. [Set Up Authentication](#set-up-authentication)
5. [Migrate the Frontend Code](#migrate-the-frontend-code)
### Configure Postgres
Create a Postgres role and publication as described in the [source database setup documentation](/configuration/source-db/setup).
* It is not possible to reuse the Electric publication, since PowerSync requires a publication named `powersync`.
* `BYPASSRLS` is commonly used for the PowerSync role. This is because Sync Stream definitions enforce authorization.
### Connect PowerSync to Postgres
Connect your PowerSync instance to your Postgres environment using the [Dashboard](/configuration/source-db/connection) or the [CLI](/tools/cli#cloud-workflows).
Since the PowerSync Service connects directly to Postgres, various network-level security mechanisms are [supported](/configuration/source-db/security-and-ip-filtering).
### Define and Test Sync Streams
In this step you will write your Sync Streams YAML. This will:
* Define how subsets of your Postgres data are synced to SQLite (on the client).
* Move download authorization into Sync Stream queries. You keep write authorization in your backend. PowerSync uses signed JWT claims for access checks. Sync Streams accept client parameters, but these should not be relied on for authorization checks.
Refer to the [Sync Streams documentation](/sync/streams/overview). If the [supported Sync Streams SQL](/sync/supported-sql) doesn't support the specific query you are trying to write, sync the required streams and then run ORDER BY, LIMIT, aggregates, joins, etc. in local SQLite.
#### Example
A Shape that syncs a user's projects as follows:
```tsx theme={null}
const { data } = useShape({
url: `http://localhost:3000/v1/shape`,
params: {
table: `projects`,
where: `owner_id = ${currentSession().userId}`,
},
})
```
Can be defined in Sync Streams as this `my_projects` stream:
```yaml theme={null}
streams:
my_projects:
query: SELECT * FROM projects WHERE owner_id = auth.user_id()
```
You can then define a separate Sync Stream to sync each project's tasks (`project_tasks` stream), where the client provides the `project_id`:
```yaml theme={null}
streams:
my_projects:
...
project_tasks:
query: |
SELECT * FROM tasks
WHERE project_id = subscription.parameter('project_id')
AND project_id IN (SELECT id FROM projects WHERE owner_id = auth.user_id())
```
#### Run a Sync Test from the PowerSync Dashboard
Once you've defined your Sync Streams, you can run a Sync Test in the PowerSync Dashboard to validate that data is syncing to the client as expected.
### Set Up Authentication
Electric relies on a backend to validate requests to Shapes. With PowerSync that logic is contained in your Sync Stream queries, and clients are then able to connect to the PowerSync Service with a JWT minted from your backend, instead of using the backend as a step in the middle.
`auth.user_id` illustrates this in the Sync Streams example from above:
```yaml theme={null}
query: SELECT * FROM projects WHERE owner_id = auth.user_id()
```
Our [authentication documentation](/configuration/auth/overview) covers how to set this up.
### Migrate the Frontend Code
The PowerSync client owns a local SQLite database that intelligently merges rows from all active Sync Streams. It exposes local SQL and live query (watch) APIs. It optionally records local writes into a FIFO upload queue. Note that migrations from PGlite to SQLite are possible but out of scope for this guide.
PowerSync provides SDKs for many platforms: JS/TS (Web, React Native, Node, Capacitor and Tauri), Dart, Kotlin, Swift, .NET and Rust. This section only covers Web JS/TS.
[Install](/client-sdks/reference/javascript-web#installation) the PowerSync Web SDK with `pnpm install @powersync/web`.
Use the PowerSync Dashboard or CLI to generate the client-side SQLite schema ([docs](/client-sdks/reference/javascript-web#1-define-the-client-side-schema)). Note that an `id` column is added automatically. It will look similar to this:
```js theme={null}
import { column, Schema, Table } from '@powersync/web'
const projects = new Table({
name: column.text,
owner_id: column.text,
})
const tasks = new Table({
project_id: column.text,
title: column.text,
status: column.text,
}, { indexes: { by_project: ['project_id'] } })
export const AppSchema = new Schema({ projects, tasks })
```
[Instantiate](/client-sdks/reference/javascript-web#2-instantiate-the-powersync-database) the PowerSyncDatabase. Note that you must only create one PowerSyncDatabase instance for each database file.
```js theme={null}
import { PowerSyncDatabase } from '@powersync/web'
export const db = new PowerSyncDatabase({
schema: AppSchema,
database: { dbFilename: 'powersync-v1.db' },
})
```
The Web SDK uses a persistent IndexedDB VFS by default. [Select another VFS](/client-sdks/reference/javascript-web#sqlite-virtual-file-systems) if you have specific browser, performance, or multi-tab requirements.
[Integrate](/client-sdks/reference/javascript-web#3-integrate-with-your-backend) with your backend. This requires implementing two methods for `PowerSyncBackendConnector`:
* `fetchCredentials()` returns the JWT used to authenticate against PowerSync and download synced data, as well as the PowerSync Cloud endpoint.
* `uploadData()` defines how local mutations are sent to your backend API. You should be able to re-use the existing backend you have in place today.
This is where the bulk of the work will take place, but agents should be pretty good at it. Below are some more tips to get you going:
* Replace shape [handle/offset usage](https://electric.ax/docs/sync/guides/shapes#subscribing-to-shapes) with [waitForFirstSync()](/client-sdks/usage-examples#wait-for-the-initial-sync-to-complete) and Sync Stream [status checks](/sync/streams/client-usage#checking-sync-status).
* Tie on-demand Sync Stream subscriptions to component or route lifetime. The PowerSync React Hooks can [automatically subscribe/unsubscribe](/sync/streams/client-usage#framework-integrations).
* When subscribing to a Sync Stream on the client, the TTL can be overridden. The default is 24 hours: a shorter TTL reduces disk usage, a longer TTL improves page reload performance.
* When logging the user out or switching accounts, only use `disconnectAndClear()` once the upload queue has been emptied, otherwise local mutations will get discarded.
Electric is read-path only, so this guide doesn't cover migrating a write path. However, it's highly recommended to use the PowerSync upload queue to ensure consistency instead of sending mutations directly to your backend. You might notice data flicker on the client if you write directly to your backend APIs and bypass the PowerSync upload queue.
Follow [this guide](/configuration/app-backend/client-side-integration) for integrating the PowerSync upload queue with your backend APIs.
# Contact Us
Source: https://docs.powersync.com/resources/contact-us
Contact the PowerSync team via Discord, GitHub, email, or paid support.
## Need Help or Have Questions?
### Discord Community
Join our [Discord](https://discord.gg/powersync) server where you can browse topics from our community, ask questions, share feedback, or just say hello :)
### Support for Pro, Team & Enterprise Customers
If you are a customer on our Pro, Team or Enterprise (Cloud or Self-Hosted) [plans](https://www.powersync.com/pricing), you can contact us using the support details provided to you during onboarding.
You are also welcome to use our [Discord](https://discord.gg/powersync) community for questions, but please note that [support SLAs](https://www.powersync.com/legal/commercial-license-and-services-agreement#appendix-c) (Team and Enterprise plans) are not available for Discord support.
## Found a Bug?
Bugs can be logged as [GitHub issues](https://github.com/powersync-ja) on the respective repo.
## Feedback or Ideas?
* [Submit an idea](https://roadmap.powersync.com/tabs/5-roadmap/submit-idea) via our public roadmap
* Or [schedule a chat](https://calendly.com/powersync/powersync-chat) with someone from our product team.
## Pricing or Commercial Questions?
Please [shoot us an email](mailto:hello@powersync.com) to get in touch.
# FAQ
Source: https://docs.powersync.com/resources/faq
Frequently asked questions about PowerSync.
A major product principle that has guided us is to provide an open-source real database on the client-side, with a specific focus on [SQLite](https://sqlite.org/). This is as opposed to some kind of cache, key-value store or non-standards-based relational datastore.
This approach leverages the power of SQLite and its ecosystem:
* **SQL functionality & concepts**: Millions of developers are already well-versed in SQL constructs and syntax, which means that there’s an instant familiarity with using SQLite. It also means having access to its rich functionality such as aggregations, joins, advanced indexing and JSON support.
* **Ecosystem & extensibility:** SQLite brings a lot with it: You can use popular ORMs that you’re already familiar with, such as Drizzle, Kysely and Drift. You can use SQLite extensions such as SQLCipher for encryption and FTS5 for full-text search. You can use standard tools for inspecting the database and doing more in-depth debugging. You get all the benefits of the SQLite community and the innovation around it: SQLite just keeps becoming more popular, and people keep doing more new interesting things with it.
* **Performance & maturity:** SQLite is also really fast, and extremely battle-tested: the SQLite team estimates that there are more than a trillion SQLite databases deployed, and every line in the codebase has 600 lines of test code.
The ubiquity of SQLite also creates opportunities for adopting PowerSync in the SQLite “installed base”: wherever you find SQLite, you can likely use PowerSync too. Usage of SQLite also means low lock-in. PowerSync is designed to be a “pluggable middleware” layer rather than a high lock-in monolithic system. It sits between popular backend databases on the server-side, and SQLite on the client-side. Replacing it with a different sync engine is fairly straightforward. Since PowerSync is built to work with open technologies and is itself open too, you can have an end-to-end stack optimized for low risk.
**PowerSync uses near real-time streaming of changes to the client (\< 1s delay).**
A persistent connection is used to continuously stream changes to the client.
This is implemented using a standard HTTP/2 request with a streaming response, or WebSockets.
A polling API will also be available for cases where the client only needs to update data periodically, and prefers to not keep a connection open.
The real-time streaming is not designed for "update as you type" — it still depends on explicitly saving changes. Real-time collaboration is supported as long as users do not edit the same data (same columns of the same rows) at the same time.
Concurrently working on text documents is not supported out of the box. This is solved better by CRDTs — see the [CRDTs](/client-sdks/advanced/crdts) section.
See the section on [Performance and Limits](/resources/performance-and-limits).
If no Sync Stream/Rule changes were deployed in this period, the user will only need to download the incremental changes that happened since the user was last connected.
*For example, a new record should not be displayed until the server received it, or it should be displayed as pending, or the entire screen must block with a spinner.*
**While PowerSync does not have out-of-the-box support for this due to the great variety of requirements, this is easy to build on top of the sync system.** A simple approach is to store a "status" or "pending changes" column on the table, and set that whenever the client makes a change. When the server receives the change, it then sets it to "processed" / "no pending changes". So when the server has processed the change, the client automatically syncs that status back.For more granular information, record individual changes in a separate table, as explained in [Custom Conflict Resolution](/handling-writes/custom-conflict-resolution).Note: Blocking the entire screen with a spinner is not recommended, since the change may take a very long time to be processed if the user is offline.
**Right now, we don’t have support for replicating data via APIs.** A workaround would be to have custom code to replicate the data from the API to a Postgres instance, then sync that with PowerSync. We may add a way in the future to replicate the data directly from an API to the PowerSync Service, without a database in between.
**Yes.** The PowerSync Client SDKs support real-time streaming of changes, and can automatically rerun a query if the underlying data changed. It does not support incrementally updating the result set yet, but it should be fast if the query is indexed appropriately, and the result set is small enough.
See [Troubleshooting](/debugging/troubleshooting)
**Client-side transactions are supported**, and use standard SQLite locking to avoid conflicts. **Client-server transactions are not supported.** This would require online connectivity to detect conflicts and retry the transaction, which is not possible for changes made offline. Instead, it is recommended to model the data to allow atomic changes (see previous sections on conflict detection).
**This is generally not recommended, but it can be used in some cases, with caveats.**
See the section on [client ID](/sync/advanced/client-id) for details.
**An attachment sync or caching system can be built on top of PowerSync.**
See the section on [Attachments](/client-sdks/advanced/attachments) for details.
Currently, PowerSync can only read from Postgres databases directly. GraphQL or REST APIs can be used for the write path by the PowerSync SDK.
By default PowerSync is not susceptible to SQL injection. The PowerSync execute API is parameterized, and as long as developers use that, SQL injection is not possible. It is however the developer's responsibility to ensure that they use the parameterized API and don't directly insert user-provided data into underlying SQLite tables.
[getCrudBatch()](https://pub.dev/documentation/powersync/latest/powersync/PowerSyncDatabase/getCrudBatch.html) [getNextCrudTransaction()](https://pub.dev/documentation/powersync/latest/powersync/PowerSyncDatabase/getNextCrudTransaction.html)
Use getCrudBatch() when you don't care about atomic transactions, and want to do bulk updates for performance reasons.
PowerSync will only sync the difference (buckets added or removed).
# Feature Status
Source: https://docs.powersync.com/resources/feature-status
Current status of PowerSync features, including alpha and beta labels with API stability and SLA implications.
Features in PowerSync are introduced through a phased release cycle to ensure quality and stability. Below is an overview of the release stages namely Experimental/Pre-Alpha, Closed Alpha, Open Alpha, Beta and GA (General Availability):
| **Stage** | **Production Readiness** | **API Stability** | **Support** | **Documentation** |
| -------------------------- | --------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------- | ----------------------- | ----------------------------------------------- |
| **Experimental/Pre-Alpha** | Not production-ready; purpose is early exploration and proof of concept. | API may change significantly or be removed. | Not covered under SLAs. | Limited or placeholder documentation. |
| **Closed Alpha** | Not production-ready; purpose is early feedback and testing of new ideas. | Subject to breaking changes. | Not covered under SLAs. | Limited or placeholder documentation. |
| **Open Alpha** | Not production-ready; purpose is broader testing and wider public feedback. | Subject to changes based on feedback. | Not covered under SLAs. | Basic documentation provided. |
| **Beta** | Production-ready for tested use cases. | Fully stable; breaking changes clearly communicated. | Covered under SLAs. | Documentation provided; may contain known gaps. |
| **GA** | Production-ready for all main use cases. | Fully stable; backwards compatibility maintained as far as possible; breaking changes clearly communicated. | Covered under SLAs. | Comprehensive and finalized documentation. |
## Service Release Channels
PowerSync Service releases are published to two channels:
* **Next**: New releases are published here first. They run in internal and public testing for anything from a day to a couple of weeks.
* **Stable**: Promoted from Next after that testing period. This is the most reliable release and is recommended for production.
A Stable release can include features that are not yet GA (Experimental, Alpha, or Beta stages). These are included when are sufficiently self-contained and low-risk, though they may still have known issues. The Next channel adds new features, fixes, or changes to stable functionality that need further validation.
### PowerSync Cloud
Configure the channel per instance in the PowerSync Dashboard, under the instance's Settings view. Select Stable or Next.
### Self-Hosted (Open Edition)
Select a channel through the Docker image tag you pull:
| Channel | Example tags | Use for |
| ------- | ------------------------------------ | ------------------------- |
| Stable | `1.23.0`, `1.23`, `1` | Production |
| Next | `1.23.0-next`, `1.23-next`, `1-next` | Testing upcoming releases |
```bash theme={null}
docker pull journeyapps/powersync-service:1.23.0
```
Each release is published under several tags. The full version (`1.23.0`) is pinned to a single release. The shorter major and minor tags (`1.23`, `1`) are rolling aliases that move to the latest matching release when a new one is published.
The `latest` tag tracks the most recent stable build but is recommended for development only, since it can move across major versions. Pin a version tag for production.
Development images may also be published for bleeding-edge features or hotfix testing. These are versioned as `0.0.0-dev-XXXXXXXXXXXXXX` pre-releases.
## Feature Status Summary
Below is a summary of the current main PowerSync features and their release states:
| **Category / Item** | **Status** |
| --------------------------------------------------------------- | ------------ |
| **Database Connectors** | |
| Convex | Experimental |
| Azure DocumentDB (Cosmos DB for MongoDB vCore) | Alpha |
| SQL Server | Beta |
| MySQL | Beta |
| MongoDB | GA |
| Postgres | GA |
| | |
| **PowerSync Service** | |
| Storage Version 4 (incremental reprocessing, S3 object storage) | Beta |
| Open Edition | GA |
| Enterprise Self-Hosted | GA |
| Sync Streams | GA |
| Postgres Bucket Storage | GA |
| | |
| **Client SDKs** | |
| High Performance Diffs | Experimental |
| Checkpoint Requests | Alpha |
| Tauri SDK | Alpha |
| Rust SDK | Alpha |
| TanStack Query | Alpha |
| Capacitor SDK | Beta |
| .NET SDK | Beta |
| OP-SQLite Support | Beta |
| Flutter Web Support | Beta |
| React Native Web Support | Beta |
| Flutter SQLCipher | Beta |
| Kotlin SQLite3MultipleCiphers | Beta |
| Vue Composables | Beta |
| Node.js SDK | GA |
| Swift SDK | GA |
| Kotlin SDK | GA |
| JavaScript/Web SDK | GA |
| Dart/Flutter SDK | GA |
| React Native SDK | GA |
| React Hooks | GA |
| | |
| **ORMs/SQL Libraries** | |
| TanStack DB (JS) | Alpha |
| GRDB (Swift) | Alpha |
| Drift (Flutter) | Beta |
| Drizzle (JS) | Beta |
| Kysely (JS) | Beta |
| Room (Kotlin) | Beta |
| SQLDelight (Kotlin) | Beta |
| | |
| **Attachment Helpers** | |
| Kotlin | Alpha |
| Swift | Alpha |
| JavaScript (new built-in library) | Alpha |
| Flutter (new built-in library) | Alpha |
| .NET | Alpha |
| | |
| **Other** | |
| CLI | Beta |
Also see:
* [PowerSync Roadmap](https://roadmap.powersync.com)
# HIPAA Compliance
Source: https://docs.powersync.com/resources/hipaa
HIPAA compliance for PowerSync Cloud, including BAA availability and deployment requirements.
Note: HIPAA compliance is only available on the Team and Enterprise plans of PowerSync Cloud.
The Health Insurance Portability and Accountability Act (HIPAA) is a comprehensive U.S. federal law that protects the privacy and security of individuals' health information, known as Protected Health Information (**PHI**) or electronic PHI (**ePHI**).
Entities that handle ePHI must comply with the HIPAA Privacy Rule, Security Rule, and Breach Notification Rule.
PowerSync serves as a **Business Associate (BA)** for customers (the **Covered Entity** or their BA) who utilize our service to synchronize healthcare-related data. As a BA, PowerSync has specific legal obligations to safeguard ePHI that passes through our sync service.
To achieve HIPAA compliance when using PowerSync, two primary conditions must be met:
1. The customer must execute a **Business Associate Agreement (BAA)** with PowerSync.
2. The customer must use the PowerSync Service within a HIPAA-compliant configuration, e.g., using required encryption, proper access controls (MFA), a custom deployment setup, and network restrictions.
We also ensure that all our upstream vendors and sub-processors who may handle ePHI (such as cloud infrastructure providers) are covered by their own BAAs and comply with their obligations.
**Mandatory Bucket Storage Requirement**\
With a standard setup, PowerSync Cloud provides “bucket storage” (persistent database storage where bucket data such as operation history and metadata are stored by the PowerSync Service) as part of the cloud service. For HIPAA compliant setups, however, the customer must provide a dedicated MongoDB Atlas cluster in their own Atlas account to serve as the bucket storage database for the PowerSync Service instance(s).
## Customer Responsibilities
The customer remains the owner of their application, databases, and client devices, and therefore holds critical responsibilities in the shared compliance model:
* **Business Associate Agreement (BAA)**\
Customers **must sign a BAA** with PowerSync *before* storing or synchronizing any ePHI using the service. The BAA can be requested by emailing [hello@powersync.com](mailto:hello@powersync.com)
* **Source Database**\
Customers must ensure their **source database** (which PowerSync connects to) is hosted in a HIPAA-compliant environment and is protected by the appropriate vendor BAAs (e.g., with AWS, Azure, or GCP).
* **Bucket Storage - MongoDB Database**\
Customers must ensure their **bucket storage MongoDB Atlas cluster** (which PowerSync connects to) is hosted in a HIPAA-compliant environment.
* **Client Device Security**\
Customers must implement all necessary **administrative, physical, and technical safeguards** on the **client-side devices** (mobile, web app). This includes device access controls, encryption of the client-side PowerSync SQLite database, and secure disposal of data when a user or device is de-provisioned.
* **Data Filtering and Access Control**\
Customers must configure Sync Streams / Sync Rules (legacy) to ensure only the minimum necessary ePHI is synchronized to specific client devices, and must ensure the authentication setup is correctly implemented to restrict data to the correct client devices.
* **Network Restrictions (IP Filtering, AWS Private Endpoints)**\
Customers must use [AWS PrivateLink](/configuration/source-db/private-endpoints) where possible, or configure and restrict source database and bucket storage database access to PowerSync Cloud’s [IP addresses](/configuration/source-db/security-and-ip-filtering).
* **Breach Notification**\
Customers must follow their internal policies for notifying individuals and/or [HHS](https://www.hhs.gov/), and reporting breaches discovered by the customer to PowerSync as required by the BAA.
* **PowerSync Dashboard Account**\
Customers are in full control of their PowerSync Cloud account and are responsible for managing the users who have access to the PowerSync Dashboard. Multi-factor authentication (MFA) must be enabled for the PowerSync Dashboard.
## PowerSync’s Responsibilities (as BA)
PowerSync’s core responsibility is to protect ePHI while it is in transit and temporarily processed by our sync service.
As a Business Associate, PowerSync is directly liable for compliance with certain provisions of the HIPAA Rules and adheres to the terms of the BAA by:
* **Technical Safeguards**\
**Encrypting ePHI in transit** (using TLS/SSL) between the customer's databases, the PowerSync Service, and the client devices.
* **Vendor Management**\
Ensuring all underlying cloud infrastructure providers (sub-BAs) that handle ePHI have executed a **BAA** with PowerSync.
* **Breach Reporting**\
Notifying the customer immediately upon the discovery of a **security incident** or **breach** involving unsecured ePHI processed or stored by the PowerSync Service, as outlined in the BAA.
* **Infrastructure and Auditing**\
Maintaining appropriate **administrative and physical controls** over our infrastructure, including access management, logging, monitoring, and regular third-party audits (e.g. SOC 2) to validate our security posture.
## Shared Model of Responsibility
HIPAA compliance is a continuous, shared process between the customer and PowerSync (BA).
| Area of Responsibility | Customer | PowerSync (Business Associate) |
| :--------------------------------------------- | :------------------------------------------------------------------------------------------------------------------------------------------------------- | :------------------------------------------------------------------------------------------------------------ |
| **Source Database** | Responsible for the security and HIPAA status of the source database hosting. | Responsible for the secure, encrypted connection to the database. |
| **Bucket Storage Database** | Responsible for the security and HIPAA status of the bucket storage database hosting. | Responsible for the secure, encrypted connection to the database. |
| **Synchronization Service** | Responsible for proper configuration of Sync Streams / Sync Rules data filtering to prevent unnecessary data exposure. | Responsible for securing the PowerSync Service infrastructure and ensuring data is encrypted while processed. |
| **Client Devices (e.g., Mobile App, Web App)** | **Wholly Responsible** for securing the client-side SQLite database, applying user authentication, authorization, and data purge policies on the device. | Responsible securing the client-side SDKs |
## Frequently Asked Questions
### What Is the Difference Between SOC 2 and HIPAA?
**SOC 2 (Service Organization Control 2)** is an auditing procedure that validates a company’s controls relevant to security, availability, processing integrity, confidentiality, and privacy. It is not industry-specific.
**HIPAA** is a federal regulation specific to the U.S. healthcare industry that dictates the protection of PHI.
A strong **SOC 2 Type 2 report** provides independent assurance that PowerSync maintains the necessary security posture to meet the administrative and technical safeguards required for a HIPAA Business Associate. [Learn more](/resources/security).
### How Often Is PowerSync Audited?
PowerSync undergoes **annual third-party audits** of our security controls (e.g., SOC 2 Type 2). These audits review the controls that are foundational to our ability to fulfill HIPAA BAA requirements.
### Where Can I Find PowerSync’s BAA?
The BAA is available upon request to customers seeking to process ePHI. Please contact [hello@powersync.com](mailto:hello@powersync.com) to initiate the BAA execution process. **Only the Team and Enterprise plans on PowerSync Cloud are supported.**
### Is a HIPAA Compliance Report Available?
Yes. To provide independent assurance of our security controls, PowerSync can provide a HIPAA Compliance Report to customers on the Team or Enterprise plans of PowerSync Cloud. To request a copy, please contact [hello@powersync.com](mailto:hello@powersync.com).
# Local-First Software
Source: https://docs.powersync.com/resources/local-first-software
Understand the local-first software architecture pattern and how PowerSync enables it.
## What Is Local-First Software?
### The Vision of Local-First
Local-first software is a term coined by the research lab [Ink & Switch](https://www.inkandswitch.com/) in its [2019 manifesto essay](https://www.inkandswitch.com/local-first/).
Ink & Switch's rationale for local-first is to get the best of both worlds of stand-alone desktop apps (so-called "old-fashioned" software) and cloud software:
> *"We would like both the convenient cross-device access and real-time collaboration provided by cloud apps, and also the personal ownership of your own data embodied by ‘old-fashioned’ software".*
The manifesto proceeds to define local-first as software that:
> *"prioritizes the use of local storage (the disk built into your computer) and local networks (such as your home WiFi) over servers in remote data centers".*
It also puts emphasis on the primacy of the local copy of data:
> "In local-first applications \[...] we treat the copy of the data on your local device \[...] as the primary copy. Servers still exist, but they hold secondary copies of your data in order to assist with access from multiple devices."
Expanding on this, the manifesto identifies [7 ideals](https://www.inkandswitch.com/local-first/#seven-ideals-for-local-first-software) "to strive for in local-first software", which we will explore further below.
**Much more theoretical research is still needed** to practically build software that conforms to all of the ideals of local-first software as envisioned by Ink & Switch, since it will need a fully decentralized architecture and needs many complex requirements to be addressed (see [here](https://www.powersync.com/blog/local-first-software-origins-and-evolution#why-are-the-ideals-of-local-first-difficult-to-achieve) for more details). In the meantime, the manifesto essay does provide [practical guidance](https://www.inkandswitch.com/local-first/#for-practitioners) on things you can do to bring your software closer to the ideals.
### Local-First in Practice Today
Most implementations that are referred to as "local-first" today conform to only a subset of the local-first ideals envisioned by Ink & Switch. We argue that a practical definition of most local-first implementations today is the following:
> Local-first implementation today generally refers to apps that work with a local client database which syncs automatically with a backend database in the background. All reads and writes go to the local database first.
This kind of architecture already enables large benefits for both end-users (speed, network resilience, real-time collaboration, offline usage) as well for developers (reduced backend complexity, simplified state management, etc.). Refer to [References](/resources/local-first-software#references) for more on this.
## Does PowerSync Allow Building Local-First Software?
### High-Level Concepts
Here's how building software with [PowerSync](https://www.powersync.com/) as its sync engine stacks up in terms of the high-level definitions of local-first software mentioned above:
| Local-First Concept / Definition | Does PowerSync Enable This? |
| -------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Software that prioritizes the use of local storage. All reads and writes go to the local database first. | Yes. PowerSync allows developers to build software that uses a local database for reads and writes. |
| Software that treats the data on the user's local device as the primary copy of the data. | Yes, generally. PowerSync allows the developer to treat the data in the local end-user's database as the primary copy of the data. PowerSync does use a server-authoritative architecture where the server can [resolve conflicts](/handling-writes/handling-update-conflicts) and all clients then update to match the server state. But the client [will not update](/architecture/consistency) its local state to the server state until all pending client changes have been processed by the server. |
| Software with a decentralized architecture, which allows the software "to outlive any backend services managed by their vendors" | No. PowerSync does not use a decentralized architecture. PowerSync uses a server-authoritative architecture. However, there are ways to ensure a degree of longevity of software built using PowerSync (see below). |
### The 7 Ideals of Local-First
Here's how applications built using PowerSync can be brought closer to the [7 ideals of local-first](https://www.inkandswitch.com/local-first/#seven-ideals-for-local-first-software) in the Ink & Switch manifesto essay:
| 7 Ideals of Local-First | PowerSync Perspective |
| ------------------------------------------------------------------------------------------------------------------------------------------------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Fast**: By accessing data locally, the software should be able to respond near-instantaneously to user input | PowerSync inherently provides this: All reads and writes use a local SQLite database, resulting in near-zero latency for accessing data. |
| **Multi-Device**: Data should be synchronized across all of the devices on which a user does their work. | PowerSync automatically syncs data to different user devices. |
| **Offline**: The user should be able to read and write their data anytime, even while offline. | PowerSync allows for offline usage of applications for arbitrarily long periods of time. Developers can also optionally create apps as [offline-only](/client-sdks/advanced/local-only-usage) and turn on syncing of data when it suits them, including on a per-user basis.When syncing is configured, data is synced to users based on the [Sync Streams](/sync/streams/overview) (or [Sync Rules](/sync/rules/overview)) for offline access. Mutations to data while the user is offline are placed in an upload queue and periodically attempted to be [uploaded](/configuration/app-backend/client-side-integration) when connectivity is available (this is automatically managed by the PowerSync Client SDK). |
| **Collaboration**: The ideal is to support real-time collaboration that is on par with the best cloud apps today. | PowerSync allows building collaborative applications either with [custom conflict resolution](/handling-writes/custom-conflict-resolution), or [using CRDT](/client-sdks/advanced/crdts) data structures stored as blob data for fine-grained collaboration. |
| **Longevity**: Work the user did with the software should continue to be accessible indefinitely, even after the company that produced the software is gone. | PowerSync relies on open-source and source-available software, meaning that the end-user can self-host Postgres (open-source) and the [PowerSync Service](/architecture/powersync-service) (source-available) should they wish to continue using PowerSync to sync data after the software producer shuts down backend services. There is also an onus on the software developer to ensure longevity, such as allowing exporting of data and avoiding reliance on other proprietary backend services. |
| **Privacy**: The software should use end-to-end encryption so that servers that store a copy of users’ files only hold encrypted data that they cannot read. | For details on end-to-end encryption with PowerSync, refer to our [Encryption](/client-sdks/advanced/data-encryption) section. |
| **User Control:** No company should be able to restrict what a user is allowed to do with the software. | In theory, the server-authoritative architecture of PowerSync allows the vendor's backend to override the user's local data (once all pending changes by the user have been [processed by the server](/architecture/consistency)). However, this is ultimately in the control of the developer. |
## References
* [Local-First Software: Origins and Evolution](https://www.powersync.com/blog/local-first-software-origins-and-evolution)
* [Local-First Software is a Big Deal, Especially for the Web](https://www.powersync.com/blog/local-first-is-a-big-deal-especially-for-the-web)
# Performance and Limits
Source: https://docs.powersync.com/resources/performance-and-limits
Performance benchmarks and Service limits for PowerSync Cloud.
[PowerSync Cloud plans](https://www.powersync.com/pricing) have the limits and performance expectations outlined below.
The PowerSync Cloud **Team** and **Enterprise** plans allow several of these limits to be customized based on your specific needs.
## Limits
| **Component** | **Limit** | **Details** |
| ----------------------------- | --------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Synced buckets per user** | Maximum: 1,000 by default; Configurable up to 10,000 by request | Two limits apply per connection, both defaulting to 1,000: unique buckets and parameter query results before deduplication. Exceeding either causes sync to fail with `PSYNC_S2305`. Both limits can be increased for Team and Enterprise customers on request, but higher limits can reduce performance. See [Bucket Count and Limits](/sync/streams/bucket-count). |
| **Maximum row/document size** | 15MB | Applies to both source database rows and transformed rows synced to clients. |
| **Concurrent clients** | Maximum: configurable (50k+ per instance) | PowerSync Service instances have default limits configured based on the [Pricing plan](https://www.powersync.com/pricing). These limits can be increased upon request for Team and Enterprise customers, and currently scale to over 50,000 per instance. |
| **Data hosted** | Maximum: configurable | PowerSync Service instances have default limits configured based on the [Pricing plan](https://www.powersync.com/pricing). These limits can be increased upon request for Enterprise customers. |
| **Columns per table** | 1,999 | Hard limit of the client schema, excluding the `id` column. |
| **Number of users** | No limit | No hard limit on unique users. |
| **Number of tables** | No limit | Hundreds of tables may impact startup and sync performance. |
## Performance Expectations
### Database Replication (Source DB → PowerSync Service)
* **Small rows**: 2,000-4,000 operations per second
* **Large rows**: Up to 5MB per second
* **Reprocessing**: Same rates apply when reprocessing Sync Streams/Sync Rules or adding new tables
### Sync (PowerSync Service → Client)
* **Rows per client**:
* Good performance expected up to 1 million rows per client
* Up to 10 million rows per client may still work, but this requires testing with the specific SDK
* Increasing number of rows increases initial sync time, memory usage and database size
* **Sync speed**: Expect a rate of 2,000-20,000 operations per second per client, depending on the client
# Security & HIPAA
Source: https://docs.powersync.com/resources/security
PowerSync Cloud's security posture, data protection practices, and encryption standards.
PowerSync is trusted by tens of thousands of developers for building and deploying secure applications.
## PowerSync is SOC 2 Type 2 Audited
SOC 2 Type 2 audit reports are available to customers on the [Team and Enterprise plans](https://www.powersync.com/pricing) of PowerSync Cloud, as well as customers using the Enterprise Self-Hosted Edition.
## PowerSync Cloud Security
### General
* Customer data is encrypted at rest, access to that data by support staff is strictly controlled by access control mechanisms, and robust write-only logging is present across the entire stack.
* All HTTP connections are encrypted using TLS.
* Additionally, customers on our [Enterprise plan](https://www.powersync.com/pricing) can request their data to be housed in managed, isolated tenants.
* Independent third-party cybersecurity penetration testing reports are available to customers on our [Enterprise plan](https://www.powersync.com/pricing).
### AWS Private Endpoints
See [Private Endpoints](/configuration/source-db/private-endpoints) for connecting PowerSync to your database using AWS PrivateLink.
### HIPAA Compliance
PowerSync Cloud is HIPAA compliant. You can sync Protected Health Information (PHI) or electronic PHI (ePHI) using PowerSync Cloud provided that you fulfill your obligations under our shared responsibility model. Refer to our [HIPAA Compliance](/resources/hipaa) page for details.
## Client-Side Security
Refer to: [Data Encryption](/client-sdks/advanced/data-encryption)
## Security Reporting
### Our Commitment
Security of our users’ data is of utmost importance at PowerSync. We welcome the disclosure of any vulnerability you may find in our product.
We will treat each security report with the utmost seriousness. We commit to communicating promptly while we investigate the impact on our customers and will remediate the issue if deemed necessary. Having said that, we generally see a deluge of very low quality reports, many of them AI generated, and a response from our team is not guaranteed if your submission falls into this category.
We uphold the principles of Responsible Disclosure, including but not limited to:
* Make every effort to avoid accessing data of other users, and avoid disruption of our services.
* Keep within our [Terms of Service](https://www.powersync.com/legal/licensing-terms).
* Avoid publicly disclosing any vulnerability until PowerSync has had reasonable time to resolve or mitigate the issue.
Additionally, avoid any social engineering or phishing on our customers or employees, and do not physically access any of our properties.
If you follow the responsible disclosure guidelines, we commit to:
* Treat each report with the utmost seriousness.
* Communicate promptly, and work with you to understand and resolve the issue.
PowerSync does not operate a bug bounty program at this time, but may choose to offer a reward for security reports at our discretion.
### How to Report an Issue
Contact [security@powersync.com](mailto:security@powersync.com) with details on the issue.
Include at least the following information:
* A description and severity of the issue.
* Steps to reproduce the issue.
* Any sensitive details that you may have accidentally accessed during the research.
If you plan to provide sensitive credentials or data in the report, please let us know, and we will provide you with a public GPG key for encryption.
### What Reports We Are Interested In
We are interested in any reports affecting the security of our product.
We are not interested in reports of:
* Common non-vulnerabilities, such as those listed [here](https://bughunters.google.com/about/rules/google-friends/google-and-alphabet-vulnerability-reward-program-vrp-rules#non-qualifying-vulnerabilities).
* Issues that are not exploitable.
* Security best practice concerns. For example, issues pertaining to password policies such as password complexity, password reuse, etc.
* Results from automated scans.
* Social engineering or phishing attacks.
* Extracting data using a compromised device or credentials.
Please reach out to us if anything is unclear.
### See Also
* [Security & IP Filtering](/configuration/source-db/security-and-ip-filtering)
* [Data Encryption](/client-sdks/advanced/data-encryption)
# Supported Platforms
Source: https://docs.powersync.com/resources/supported-platforms
Supported platforms and feature availability for each PowerSync Client SDK across mobile, web, desktop, headless/server and embedded.
## Widespread SDK Platform Support
The versions below are the minimum supported versions across all PowerSync Client SDKs, so you can expect them to work regardless of which SDK you use.
| Platform | Version |
| ------------------------------- | -------------------------------------------- |
|
Read more: [Hierarchy: Organization, project, instance](/tools/powersync-dashboard#hierarchy-organization-project-instance)
Your organization only has a single subscription with a single plan (Free, Pro, Team or Enterprise).
Usage quotas (e.g. data processing, storage, sync operations) apply to your entire organization, regardless of the number of projects.
Upgrading to a paid plan unlocks all benefits for every project in your organization. For example, no instances in a "Pro" organization will be paused. See our [pricing page](https://www.powersync.com/pricing) for plan details.
### Invoicing
Usage for all projects in your organization is aggregated in a monthly billing cycle. These totals are reflected in your monthly invoice.
On our paid plans, the base fee (plus applicable tax) is charged at the start of every billing cycle.
If your month's usage exceeds your plan's limits, the overage will be charged at the end of the billing cycle.
Your current billing cycle's usage and upcoming invoice total can be tracked in the Dashboard - learn more in [View and Manage Your Subscription](/resources/usage-and-billing#view-and-manage-your-subscription).
Invoices will be automatically charged to your provided payment card. Learn more in [Spending caps](/resources/usage-and-billing#spending-caps).
## View and Manage Your Subscription
Your PowerSync usage and billing can be tracked and managed in the [PowerSync Dashboard](https://dashboard.powersync.com/) at the organization level.
### Subscriptions
In the "**Subscriptions**" tab you can:
1. View your active subscription
2. View your usage for the current billing cycle
3. View the amount of your upcoming invoice
4. Upgrade or cancel your [PowerSync subscription](https://www.powersync.com/pricing)
### Billing Settings
In the "**Billing"** tab you can:
1. Update billing details, such as your billing organization name, address and email address which should receive invoices and receipts.
2. Manage your credit card(s) used for payments.
* Credit card details are never stored on our servers; all billing is securely processed by our payment provider, [Stripe](https://stripe.com/).
### Spending Caps
Spending caps are not yet available, but are planned for a future release.
In the meantime, Pro plan invoices over `$100` and Team plan invoices over `$1,000` will not immediately be charged. In these cases, we will reach out to the organization owner for review. This threshold amount can be customized per organization — [let us know](/resources/contact-us) if you need a higher or lower amount configured.
## Limits
Usage limits for PowerSync Cloud are specified on our [Pricing page](https://www.powersync.com/pricing). If your usage exceeds the included amounts, see [Reducing Usage](/resources/usage-and-billing/reducing-usage) for ways to bring it down.
### Inactive Instances
Instances on the Free plan that have had no deploys or client connections for over 7 days will be deprovisioned. This helps us optimize our cloud resources and ensure a better experience for all users.
If your instance is deprovisioned, you can easily restart it from the [PowerSync Dashboard](https://dashboard.powersync.com/) or [CLI](/tools/cli) by deploying your [Sync Streams](/sync/streams/overview) or [Sync Rules](/sync/rules/overview) to it. Note that this will reprocess it from scratch, causing data to re-sync to existing users.
For projects in production we recommend subscribing to a [paid plan](https://www.powersync.com/pricing) to avoid any interruptions. To upgrade to a paid plan, navigate to your organization in the [PowerSync Dashboard](https://dashboard.powersync.com/) and visit the **Plans & Billing** section.
# Pricing Example
Source: https://docs.powersync.com/resources/usage-and-billing/pricing-example
Example showing how PowerSync Cloud plan costs are calculated based on synced operations and active users.
## Chat App Example
Use this real-world example of a basic chat app to gauge your PowerSync usage and costs, on the [Pro plan](https://www.powersync.com/pricing) of PowerSync Cloud. This is not an exact estimate, but it can help you better understand how your PowerSync usage would be billed. Usage costs for the Team plan can be calculated in the same way since only the base plan fee differs.
This use case has the peculiarity that all data is user-generated and necessarily shared with other users (in the form of messages). More typical use cases might sync the same server-side data with many different users and have less user-generated data to sync.
### Overview: Costs by usage
To illustrate typical costs, consider an example chat app, where users can initiate chats with other users. Users can see their active chats in a list, read messages, and send messages.
For this app, all messages are stored on a backend database like Postgres. PowerSync is used to make sure users see new messages in real-time, and can access or create messages even when their devices are offline.
#### Assumptions
User base assumptions:
* **Daily Active Users (DAUs) are 10% of total app installations.** These are the users that actively open and use your app on a given day, which is typically a small subset of your total app installations. For the calculations below, we estimated DAUs as 10% of the total number of app installations. We use this assumption as an input to calculate the total number of messages sent and received every day.
* **Peak concurrent clients are 10% of DAUs.** This is the maximum number of users actively using your app at exactly the same time as other users, which is typically a small subset of your Daily Active Users. For the calculations below, we estimated peak concurrent clients as 10% of the total number of app installations.
For 50,000 app installs (5,000 Daily Active Users): **\$51/month** on the Pro plan.
## Data Synced
| | |
| --------------------- | ------------------------------------------------------------------- |
| Data synced per month | 100 messages / day \* 5,000 DAUs \* 0.25 KB \* 30 = 3.75 GB / month |
| Total data synced costs / month | |
| ------------------------------- | --------------- |
| Usage: | 3.75 GB |
| Less included usage: | (30 GB) |
| Cost for additional usage: | \$0 |
| **Total usage costs** | **\$0 / month** |
## Data Hosted
| | |
| ------------------------------------------ | ----------------------------------------------------------------- |
| Total size of replicated data to be hosted | 100 messages / day \* 5,000 DAUs \* 0.25 KB \* 90 days = 11.25 GB |
| Total data hosted costs / month | |
| ------------------------------- | ---------------- |
| Usage: | 11.25 GB |
| Less included usage: | (10 GB) |
| Cost for additional usage: | 2 GB \* \$1 / GB |
| **Total usage costs** | **\$2 / month** |
## Peak Concurrent Clients
| | |
| --------------------------------------- | ----------------------------------------------- |
| Total number of peak concurrent clients | 5,000 DAUs \* 10% = 500 peak concurrent clients |
| Total peak concurrent clients costs / month | |
| ------------------------------------------- | --------------- |
| Usage: | 500 |
| Less included usage: | (1,000) |
| Cost for additional usage: | \$0 |
| **Total usage costs** | **\$0 / month** |
| Total monthly costs | |
| ----------------------- | ---------------- |
| Pro Plan | \$49 / month |
| Data synced | \$ 0 / month |
| Data hosted | \$ 2 / month |
| Peak concurrent clients | \$ 0 / month |
| **Total monthly costs** | **\$51 / month** |
For 1,000,000 app installs (100,000 Daily Active Users): **\$399/month** on the Pro plan.
## Data Synced
| | |
| --------------------- | ------------------------------------------------------------------- |
| Data synced per month | 100 messages / day \* 100,000 DAUs \* 0.25 KB \* 30 = 75 GB / month |
| Total data synced costs / month | |
| ------------------------------- | -------------------- |
| Usage: | 75 GB |
| Less included usage: | (30 GB) |
| Cost for additional usage: | 45 GB \* \$1.00 / GB |
| **Total usage costs** | **\$45 / month** |
## Data Hosted
| | |
| ------------------------------------------ | ----------------------------------------------------------------- |
| Total size of replicated data to be hosted | 100 messages / day \* 100,000 DAUs \* 0.25 KB \* 90 days = 225 GB |
| Total data hosted costs / month | |
| ------------------------------- | ------------------ |
| Usage: | 225 GB |
| Less included usage: | (10 GB) |
| Cost for additional usage: | 215 GB \* \$1 / GB |
| **Total usage costs** | **\$215 / month** |
## Peak Concurrent Clients
| | |
| --------------------------------------- | ---------------------------------------------------- |
| Total number of peak concurrent clients | 100,000 DAUs \* 10% = 10,000 peak concurrent clients |
| Total peak concurrent clients costs / month | |
| ------------------------------------------- | --------------------- |
| Usage: | 10,000 |
| Less included usage: | (1,000) |
| Cost for additional usage: | 9,000 \* \$30 / 1,000 |
| **Total usage costs** | **\$270 / month** |
| Total monthly costs | |
| ----------------------- | ----------------- |
| Pro Plan | \$ 49 / month |
| Data synced | \$45 / month |
| Data hosted | \$215 / month |
| Peak concurrent clients | \$270 / month |
| **Total monthly costs** | **\$579 / month** |
For 10,000,000 app installs (1,000,000 Daily Active Users): **\$6,064/month** on the Pro plan.
At this scale, our [Enterprise plan](https://www.powersync.com/pricing) is typically more cost effective and provides more predictable billing.
## Data Synced
| | |
| --------------------- | ---------------------------------------------------------------------- |
| Data synced per month | 100 messages / day \* 1,000,000 DAUs \* 0.25 KB \* 30 = 750 GB / month |
| Total data synced costs / month | |
| ------------------------------- | --------------------- |
| Usage: | 750 GB |
| Less included usage: | (30 GB) |
| Cost for additional usage: | 720 GB \* \$1.00 / GB |
| **Total usage costs** | **\$720 / month** |
## Data Hosted
| | |
| ------------------------------------------ | --------------------------------------------------------------------- |
| Total size of replicated data to be hosted | 100 messages / day \* 1,000,000 DAUs \* 0.25 KB \* 90 days = 2,250 GB |
| Total data hosted costs / month | |
| ------------------------------- | -------------------- |
| Usage: | 2,250 GB |
| Less included usage: | (10 GB) |
| Cost for additional usage: | 2,240 GB \* \$1 / GB |
| **Total usage costs** | **\$2,240 / month** |
## Peak Concurrent Clients
| | |
| --------------------------------------- | ------------------------------------------------------- |
| Total number of peak concurrent clients | 1,000,000 DAUs \* 10% = 100,000 peak concurrent clients |
| Total peak concurrent clients costs / month | |
| ------------------------------------------- | ---------------------- |
| Usage: | 100,000 |
| Less included usage: | (1,000) |
| Cost for additional usage: | 99,000 \* \$30 / 1,000 |
| **Total usage costs** | **\$2,970 / month** |
| Total monthly costs | |
| ----------------------- | ---------------------- |
| Pro Plan | \$ 49.00 / month |
| Data synced | \$720.00 / month |
| Data hosted | \$2,240.00 / month |
| Peak concurrent clients | \$2,970.00 / month |
| **Total monthly costs** | **\$6,064.00 / month** |
# Reducing Usage
Source: https://docs.powersync.com/resources/usage-and-billing/reducing-usage
Practical ways to lower data synced and data hosted usage on PowerSync Cloud.
Two billed PowerSync Cloud metrics grow with how your app syncs, not only with how much source data you have. **Data synced** measures the uncompressed data streamed from the PowerSync Service to clients. **Data hosted** measures the data stored for your instances. Start with changes that reduce both metrics. If you first need to find where your usage comes from, see [Usage Troubleshooting](/resources/usage-and-billing/usage-and-billing-faq#usage-troubleshooting).
## Where to Start
### Syncing Only What the App Needs
Every table, row, and column selected by your [Sync Streams](/sync/streams/overview) is stored in buckets and downloaded by each subscribed client it applies to. Global streams return the same data to every client that subscribes. If a global stream has `auto_subscribe: true`, every connecting client downloads its data. Remove tables and columns your app does not use, and filter out rows it no longer needs, such as archived or historical records. Selecting fewer columns makes each row smaller, which reduces both stored and downloaded data. It does not reduce the number of sync operations by itself.
### Keeping Large Files Out of Synced Rows
Large columns, such as base64-encoded files, long JSON documents, or binary data, make every download of those rows larger. If the row changes often or syncs to many clients, that cost repeats. Store files in object storage, such as Amazon S3 or Supabase Storage, and sync only small metadata records that reference them. The SDKs include [attachment helpers](/client-sdks/advanced/attachments) that manage uploads, downloads, and local caching for this pattern.
### Compacting and Defragmenting
Buckets hold an operations history so that existing clients can download only the changes since their last sync. Over time, that history can grow much larger than the current set of rows. This increases hosted data and makes new clients download more data during their initial sync. Two maintenance operations reduce it:
* Compacting removes history that later operations have made redundant. For example, when a later operation replaces an earlier version of the same row, clients no longer need the earlier row data. PowerSync Cloud compacts all buckets automatically once per day. You can also trigger it from the instance's **Settings** view in the [Dashboard](https://dashboard.powersync.com/) or with the [CLI](/tools/cli). For self-hosted deployments, run the Docker image's `compact` command manually or on a schedule.
* Defragmenting handles what compacting cannot. The oldest remaining `PUT` operation in a bucket prevents PowerSync from fully compacting the operations that follow it. Buckets that mix rarely changed rows with frequently updated rows can therefore keep accumulating history. Defragmenting updates, or "touches," rows so compacting can remove the earlier history. Use the **Defragment** button in the same **Settings** view, or update the affected source rows on a schedule. Existing clients must re-sync each touched row, so choose a frequency that fits your update patterns.
If the [Sync Diagnostics Client](https://github.com/powersync-ja/powersync-js/tree/main/tools/diagnostics-app) shows far more operations than rows, either a large operations history or bucket fan-out is likely responsible. If the history is large, defragmenting can help. If a bucket mixes frequently updated rows with rows that rarely change, consider separating them so the stable rows stop blocking compacting for the rest. [Compacting Buckets](/maintenance-ops/compacting-buckets) explains the mechanics, strategies, and trade-offs.
### Reducing Bucket Fan-Out
A row is stored and synced once for every bucket it belongs to. If the same row belongs to ten buckets, PowerSync stores and syncs it ten times. Streams that place the same rows in many buckets therefore multiply both stored and downloaded data. See [Reducing Bucket Count](/sync/advanced/reducing-bucket-count).
## Reducing Data Synced
Large data synced totals often come from devices downloading the same data repeatedly, not from new data being created. Before optimizing your Sync Streams, check whether clients are repeating their initial sync. A normal reconnect resumes from the client's saved sync position and downloads only newer operations.
### Avoiding Unnecessary Full Re-Syncs
A client that loses its local database must download all its subscribed data from scratch. This is expected after a fresh installation. If the same device repeatedly starts from scratch, check for these causes:
* The app calls `disconnectAndClear()` when a user signs out. This clears the local database, so the next session starts from nothing. Clearing is appropriate when another user can access the device or your security requirements prohibit keeping the data. If the same user will return and keeping their data is safe, `disconnect()` closes the sync connection without clearing the database.
* The browser clears or cannot retain the web client's storage. Private browsing modes and browser storage policies can cause the app to lose its database between sessions.
* App code deletes or recreates the database file as part of its own lifecycle.
Do not retain one user's local data for another user to avoid an initial sync. Investigate why the database is being cleared, but preserve the security boundary your app requires.
### Avoiding Reconnect Loops
A reconnect does not normally trigger a full download because the client resumes from its saved sync position. However, an app that reconnects every few minutes can create hundreds of sync sessions per day. If each session downloads data, those transfers add up. Open one `PowerSyncDatabase` per session and call `.connect()` once instead of reconnecting when the window gains focus or whenever a token refreshes. To confirm this pattern in your instance logs, see [Repetitive Syncing by the Same User](/resources/usage-and-billing/usage-and-billing-faq#repetitive-syncing-by-the-same-user).
### On-Demand Data with Sync Streams
Data that does not need to be available while offline does not have to live on every device. With Sync Streams, your app can [subscribe](/sync/streams/client-usage) only while it needs the data, such as while a specific screen is open. After the app unsubscribes and the subscription's cache time expires, PowerSync removes data that is no longer covered by another stream from the local database. Streams with `auto_subscribe: true` behave like always-on data, so use them only for data that every connecting client needs.
### Writes That Do Not Change Data
PowerSync processes source changes at the row level. Rewriting a row selected by your Sync Streams can create a new operation even when its selected values do not change. Even a no-op update such as `UPDATE tasks SET id = id` can create an operation for every affected row. A nightly job that rewrites an entire table can therefore make active clients download those rows again the next day while also growing the operations history. Restrict batch jobs to rows whose values need to change, and confirm their effect in your [instance logs](/maintenance-ops/monitoring-and-alerting#instance-logs).
### Deployments
Deploying Sync Streams recreates the buckets, and existing clients download all their data again even if the underlying rows did not change. With [incremental reprocessing](/sync/advanced/storage-version-4#incremental-reprocessing) (Beta), clients download only the data for streams you added or changed. In both cases, group related changes when practical instead of deploying many small updates.
## Reducing Data Hosted
Data hosted covers the bucket data stored for your instances, including the operations history. It can be larger than the selected data in your source database for two main reasons: the history grows as rows change, and PowerSync stores a row once for every bucket it belongs to. The largest reductions usually come from [syncing less data](#syncing-only-what-the-app-needs), [reducing the operations history](#compacting-and-defragmenting), and [reducing bucket fan-out](#reducing-bucket-fan-out).
### Non-Production Instances
Your plan's included usage applies to your organization as a whole, so staging and development instances count toward the same included usage as production. A staging instance that replicates the full production dataset can double your hosted data on its own. Point non-production instances at a smaller dataset where possible.
## Still Seeing High Usage?
If usage stays high after these changes, work through [Usage Troubleshooting](/resources/usage-and-billing/usage-and-billing-faq#usage-troubleshooting) to identify the source, or [contact us](/resources/contact-us) with what you found.
# FAQ & Troubleshooting
Source: https://docs.powersync.com/resources/usage-and-billing/usage-and-billing-faq
Find answers to common questions about PowerSync Cloud usage and diagnose unexpected usage.
We are continuously improving the tools to help you troubleshoot usage. Please [contact us](/resources/contact-us) if you have any feedback or need help understanding or managing your usage.
## Usage and Billing Metrics FAQs
You can track usage in two ways:
* **Individual instances**: Visit the [Usage metrics](/maintenance-ops/monitoring-and-alerting#usage-metrics) workspace in the PowerSync Dashboard to see metrics for a specific instance.
* **Organization-wide**: Go to your organization in the [PowerSync Dashboard](https://dashboard.powersync.com/) and check the **Plan Usage** section for aggregated metrics across all instances in your current billing cycle.
A concurrent client is an active SDK sync connection to the PowerSync Service. Think of this as the number of open sync connections, not necessarily your total number of users or app installations. Calling `.connect()` opens one long-lived connection that streams updates.
Key points about concurrent clients:
* Billing is based on peak concurrent clients, which is the highest number of simultaneously connected clients during the billing cycle.
* **Billing (Pro/Team)**: 1,000 clients are included, then \$30 per 1,000 over the included amount.
* The PowerSync Cloud Pro plan is limited to 3,000 concurrent clients.
* The PowerSync Cloud Team plan is limited to 10,000 concurrent clients by default.
* The PowerSync Cloud Free plan is limited to 50 peak concurrent clients.
* When limits are reached, new connection attempts receive a 429 HTTP response while existing connections continue syncing. Clients retry after a delay and should connect once capacity is available.
Data synced measures the total uncompressed size of data synced from PowerSync Service instances to client devices. Each transfer counts. If the same 1 MB of data syncs to 100 devices, it contributes 100 MB to data synced.
**Billing (Pro/Team)**: 30 GB included, then \$1.00 per GB over the included amount.
For ways to lower this metric, see [Reducing Usage](/resources/usage-and-billing/reducing-usage).
Data hosted is not a direct copy of your source database. The PowerSync Service hosts three types of data:
1. A current copy of the data, which should be roughly equal to the subset of your source data covered by your Sync Streams (or legacy Sync Rules).
2. An operations history for data in buckets, which can be larger than the source data because one row can be in multiple buckets.
3. Data for parameter lookups, which is typically small.
Because of this structure, your hosted data size may be larger than your source database size.
**Billing (Pro/Team)**: 10 GB included, then \$1.00 per GB over the included amount.
For ways to lower this metric, see [Reducing Usage](/resources/usage-and-billing/reducing-usage).
A sync operation occurs when a single row is synced from the PowerSync Service to a user device. You can think of the operations history as a log of row changes. If one row changes 20 times, it can add 20 operations until compacting reduces that history.
The PowerSync Service keeps an operations history so clients can download incremental changes. This means:
* Changes to selected rows create operations, and the history can grow over time.
* During its initial sync, a client downloads the current operations for its subscribed buckets.
* Existing clients only download new operations since their last sync.
As a result, sync operation counts often exceed the number of actual changes to your data, especially for frequently updated rows. This is normal.
You can manage operations history through:
* Daily automatic compacting (built into PowerSync Cloud)
* Targeted [defragmentation](/maintenance-ops/compacting-buckets#defragmenting) when old operations prevent effective compacting
See [Compacting and Defragmenting](/resources/usage-and-billing/reducing-usage#compacting-and-defragmenting) for practical guidance, and [Usage Troubleshooting](#usage-troubleshooting) below for diagnosing high operation counts.
**Billing note:** Billing for data throughput is based on *data synced*, not sync operations. You can still use sync operation counts for diagnostics.
Data replicated travels from your backend source database (Postgres, MongoDB, MySQL, SQL Server, or Convex) to the PowerSync Service. It is not billed.
Data synced travels from the PowerSync Service to client devices. It is used for billing.
## Billing FAQs
Go to your organization in the [PowerSync Dashboard](https://dashboard.powersync.com/) and open the **Plan Usage** section. This shows your total usage (aggregated across all projects) for your current billing cycle. Data updates once a day.
Update your billing details in the **Plans & Billing** section of the [PowerSync Dashboard](https://dashboard.powersync.com/) at the organization level.
Review your historic invoices in the Stripe Customer Portal by signing in with your billing email [here](https://billing.stripe.com/p/login/7sI6pU48L42cguc7ss).
The Pro and Team plans bill the following metrics. Check the [pricing page](https://www.powersync.com/pricing) for current rates and limits.
* **Data synced**: 30 GB included, then \$1.00 per GB over the included amount.
* **Peak concurrent clients**: 1,000 included, then \$30 per 1,000 over the included amount.
* **Data hosted**: 10 GB included, then \$1.00 per GB over the included amount.
The following metrics are not billed:
* Replication operations (count)
* Data replicated (per GB)
* Sync operations (count)
## Usage Troubleshooting
If you're seeing unexpected spikes in your usage metrics, work through this section to find where they come from. Once you know the cause, [Reducing Usage](/resources/usage-and-billing/reducing-usage) covers the changes that bring usage down. If your usage is consistently high rather than spiking, start there directly.
### Troubleshooting Strategies
Use these strategies to narrow down the source of a spike. Sync operations are not billed, but they show how much row-level activity sits behind data synced. The shape of the spike usually points to the cause: many operations suggest row churn or bucket fan-out, large transfers with fewer operations suggest large rows, and repeated sessions from one client suggest a connection or local storage issue.
#### Identify Timing
Use [Usage metrics](/maintenance-ops/monitoring-and-alerting#usage-metrics) to pinpoint the exact hour or day of a spike, then check what happened around that time: a Sync Streams deployment, a batch job on your source database, defragmentation, or a group of new users onboarding.
#### Review Instance Logs
Use [Instance logs](/maintenance-ops/monitoring-and-alerting#instance-logs) to review sync service activity during the spike(s). In the **Search** input, query `"sync stream complete"` to find entries showing how many operations synced, the size of data transferred, and which clients and users were involved.
Narrow the results with structured filters:
* Type a filter in the **Search** input, such as `user_id:`, `rid:`, or `error:PSYNC_S2001`.
* Click **Property filters** next to the search input and choose a property, such as User ID, Client ID, Request ID, or Error code. Error code and Close reason offer a list of known values.
* Click any metadata pill on a log entry and choose **Add filter** to pin that value as a filter. This is useful for following `user_id`, `client_id`, or custom `app_metadata_*` log properties without typing.
MongoDB uses `_id` as the name of the ID field in collections. You must use `SELECT _id as id` (and include any other columns you need) in [Sync Streams](/sync/streams/overview) queries and [Sync Rules](/sync/rules/overview) data queries when using MongoDB as the backend source database. When inserting new documents from the client, prefer `ObjectId` values for `_id` (stored in the client's `id` column).
Custom transformations can also be used for the ID column. This is useful in certain scenarios for example when dealing with join tables, because PowerSync doesn't currently support composite primary keys. For example:
```sql theme={null}
-- Concatenate multiple columns into a single id column
SELECT *, item_id || '.' || category_id as id FROM item_categories
-- the source database schema for the above example is CREATE TABLE item_categories(item_id uuid, category_id uuid, PRIMARY KEY(item_id, category_id));
```
For multiple columns with the same name (e.g. if there was an `id` column in `*`), the last column wins. Prefer writing the `*` before other columns for this reason.
If you want to upload data to a table with a custom record ID, ensure that `uploadData()` isn't blindly using a field named `id` when handling CRUD operations. See the [Sequential ID mapping tutorial](/client-sdks/advanced/sequential-id-mapping#update-client-to-use-uuids) for an example where the record ID is aliased to `uuid` on the backend.
PowerSync does not perform any validation that IDs are unique. Duplicate IDs on a client could occur in any of these scenarios:
1. A non-unique column is used for the ID.
2. Multiple table partitions are used (Postgres), with the same ID present in different partitions.
3. Multiple data queries returning the same record. This is typically not an issue if the queries return the same values (same transformations used in each query).
We recommend using a unique index on the fields in the source database to ensure uniqueness — this will prevent (1) at least.
If the client does sync multiple records with the same ID, only one will be present in the final database. This would typically be the one modified last, but this is subject to change — do not depend on any specific record being picked.
### `:
```yaml theme={null}
config:
edition: 3 # Recommended to set to the latest available edition (see 'Supported fixes' table below)
streams:
# ...
```
Or, specify options individually:
```yaml theme={null}
config:
timestamps_iso8601: true
versioned_bucket_ids: true
fixed_json_extract: true
custom_postgres_types: true
```
## Sync Streams Requirement
**New Sync Streams configurations should use `edition: 3`**, which enables the new compiler with an expanded SQL feature set (including `JOIN`, CTEs, multiple queries per stream, `BETWEEN`, `CASE`, and more):
```yaml theme={null}
config:
edition: 3
streams:
my_stream:
query: SELECT * FROM my_table WHERE user_id = auth.user_id()
```
**Upgrading from alpha**: If you have existing Sync Streams using `edition: 2`, upgrade to `edition: 3` to enable the new compiler with an expanded SQL feature set (including `JOIN`, CTEs, multiple queries per stream, `BETWEEN`, `CASE`, and more). See [Supported SQL](/sync/supported-sql) for the full list of supported features.
## Storage Version
A storage version tells the PowerSync Service how to organize prepared sync data in the [bucket storage database](/architecture/powersync-service#bucket-storage).
Changing the version does not rewrite the current data in place. When you next deploy the Sync Config, PowerSync prepares a new copy using the selected version. Clients continue using the current copy until the new one is ready. This avoids taking the instance offline for a bucket storage migration.
### Optional `config.storage_version`
You can choose the bucket storage version in the `config` block:
```yaml theme={null}
config:
edition: 3
storage_version: 4
streams:
todos:
query: SELECT * FROM todos WHERE owner_id = auth.user_id()
```
When you omit `storage_version`, the PowerSync Service uses its default, which is version 2 in v1.26.0. On PowerSync Cloud, PowerSync manages the default. For self-hosted deployments, set `config.storage_version` explicitly to select a different version.
Set `storage_version` when you need to:
* Use [storage version 4](/sync/advanced/storage-version-4), which is in Beta and enables incremental reprocessing and S3 object storage.
* Delay a storage upgrade. When the default moves to a newer version, pin `storage_version` to the version your data already uses. This keeps later Sync Config deployments on that format. Remove the pin when you are ready for the new format.
* Prepare for a Service downgrade. Select a version supported by the older Service, deploy the Sync Config, and wait for the new copy to finish before downgrading.
### Available Versions
All PowerSync Cloud instances use MongoDB bucket storage, so they are compatible with all available storage versions. Self-hosted instances with Postgres bucket storage can use versions 1 and 2 only.
| Version | Bucket storage | Status |
| ------- | ------------------- | -------------------------------------------------------------------------------------------------------------------------------------- |
| `1` | MongoDB or Postgres | Legacy format, retained for existing deployments. |
| `2` | MongoDB or Postgres | Stable. The default in v1.26.0. |
| `3` | MongoDB | Experimental. The unstable predecessor of version 4, with the same format. Do not use it in production. Deploy with version 4 instead. |
| `4` | MongoDB | Stable. Enables [incremental reprocessing and S3 object storage](/sync/advanced/storage-version-4) (Beta). |
Version numbers follow a pattern. Even numbers are stable formats: they stay backwards compatible and later Service versions continue to support them. Stable makes no guarantee that a format is bug-free. Odd numbers are experimental formats: their layout can change without notice and support can be removed in a later release, so use them only for testing, never in production.
## Supported Fixes
This table lists all fixes currently supported:
| Name | Explanation | Added in Service version | Fixed in edition |
| ----------------------------------- | ------------------------------------------- | ------------------------ | ---------------- |
| `timestamps_iso8601` | [Link](#timestamps_iso8601) | 1.15.0 | 2 |
| `versioned_bucket_ids` | [Link](#versioned_bucket_ids) | 1.15.0 | 2 |
| `fixed_json_extract` | [Link](#fixed_json_extract) | 1.15.0 | 2 |
| `custom_postgres_types` | [Link](#custom_postgres_types) | 1.15.3 | 2 |
| `unstable_sqlite_expression_engine` | [Link](#unstable_sqlite_expression_engine). | 1.22.0 | None (unstable) |
### `timestamps_iso8601`
PowerSync is supposed to encode timestamps according to the ISO-8601 standard.
Without this fix, the service encoded timestamps from MongoDB and Postgres source databases incorrectly.
To ensure time values from Postgres compare lexicographically, they're also padded to six digits of accuracy when encoded.
Since MongoDB only stores values with an accuracy of milliseconds, only three digits of accuracy are used.
For instance, the value `2025-09-22T14:29:30` would be encoded as follows:
* For Postgres: `2025-09-22 14:29:30` without the fix, `2025-09-22T14:29:30.000000` with the fix applied.
* For MongoDB: `2025-09-22 14:29:30.000` without the fix, `2025-09-22T14:29:30.000` with the fix applied.
Note that MySQL has never been affected by this issue, and thus behaves the same regardless of the option used.
#### Configurable Sub-Second Datetime Precision
When the `timestamps_iso8601` option is enabled, PowerSync will sync date and time values with a higher
precision depending on the source database.
You can use the `timestamp_max_precision` option to configure the actual precision to use.
For instance, a Postgres timestamp value would sync as `2025-09-22T14:29:30.000000` by default.
If you don't want that level of precision, you can use the following options to make it sync as `2025-09-22T14:29:30.000`:
```yaml sync-config.yaml theme={null}
config:
edition: 3
timestamp_max_precision: milliseconds
```
Valid options for `timestamp_max_precision` are `seconds`, `milliseconds`, `microseconds` and `nanoseconds`. When an explicit
value is given, all synced time values will use that precision.
If a source value has a higher precision, it will be truncated (it is not rounded).
If a source value has a lower precision, it will be padded (so setting the option to `microseconds` with a MongoDB source database
will sync values as `2025-09-22T14:29:30.123000`, with the last three sub-second digits always being set to zero).
If no option is given, the default precision depends on the source database:
| Source database | Default precision | Max precision | Notes |
| --------------- | ----------------- | ------------- | ------------------------------------------------------------------------------------------------------- |
| MongoDB | Milliseconds | Milliseconds | |
| Postgres | Microseconds | Microseconds | |
| MySQL | Milliseconds | Microseconds | Defaults to milliseconds, but can be expanded with the option. |
| SQL Server | Nanoseconds | Nanoseconds | SQL Server supports 7 digits of accuracy, the sync service pads values to always use 9 for nanoseconds. |
### `versioned_bucket_ids`
Sync Rules define buckets, which rows to sync are then assigned to. When you run a full defragmentation or
redeploy Sync Rules, the same bucket identifiers are re-used when processing data again.
Because the second iteration uses different checksums for the same bucket ids, clients may sync data
twice before realizing that something is off and starting from scratch.
Applying this fix improves client-side progress estimation and is more efficient, since data would not get
downloaded twice.
For how bucket identifiers are represented in bucket storage at the persistence layer (including automatic use of versioned bucket names with newer storage formats), see [Storage version](#storage-version).
### `fixed_json_extract`
This fixes the `json_extract` functions as well as the `->` and `->>` operators in Sync Rules to behave similar
to recent SQLite versions: We only split on `.` if the path starts with `$.`.
For instance, `'json_extract({"foo.bar": "baz"}', 'foo.bar')` would evaluate to:
1. `baz` with the option enabled.
2. `null` with the option disabled.
### `custom_postgres_types`
If you have custom Postgres types in your backend source database schema, older versions of the PowerSync Service
would not recognize these values and sync them with the textual wire representation used by Postgres.
This is especially noticeable when defining `DOMAIN` types with e.g. a `REAL` inner type: The wrapped
`DOMAIN` type should get synced as a real value as well, but it would actually get synced as a string.
With this fix applied:
* `DOMAIN TYPE`s are synced as their inner type.
* Array types of custom types get parsed correctly, and sync as a JSON array.
* Custom types get parsed and synced as a JSON object containing their members.
* Ranges sync as a JSON object corresponding to the following TypeScript definition:
```TypeScript theme={null}
export type Range =
| {
lower: T | null;
upper: T | null;
lower_exclusive: boolean;
upper_exclusive: boolean;
}
| 'empty';
```
* Multi-ranges sync as an array of ranges.
### `unstable_sqlite_expression_engine`
This option is experimental: When enabled, updates to the PowerSync Service might change how rows are processed
and this option may be removed in a future version of the Service.
Sync Streams support scalar SQL operators (like `+`, `-` and `||`) and [functions](/sync/supported-sql#functions).
SQL in Sync Streams should behave exactly as it would in SQLite, but the Service uses a custom implementation which differs
from SQLite for some edge cases.
To perfectly align the behavior of the Service and SQLite, enabling this option makes the Service use an actual
SQLite database to evaluate Sync Streams.
Some known issues with the JavaScript evaluator that are fixed by this option are:
* Exact null handling: `NOT NULL` evaluates to `TRUE` without this option, enabling it yields `NULL`.
* Without this option, `substr()` and `length()` operate on UTF-16 code units. Enabling it makes them operate on
Unicode code points.
# Multiple Client Versions
Source: https://docs.powersync.com/sync/advanced/multiple-client-versions
Handle multiple client app versions that require different output schemas from Sync Streams.
When schema changes are additive, old clients ignore the new tables and columns, and no special handling is required. More drastic changes, such as renaming tables or changing a table's structure, can break older app versions that are still in use. In these cases, define separate versions of the affected [Sync Streams](/sync/streams/overview) so that each client version receives the tables and columns it expects.
## Versioning by Stream Name
With Sync Streams, the most convenient approach is usually to define a new stream alongside the old one. New app versions subscribe to the new stream, while older app versions continue subscribing to the old one.
For example, suppose a new app version changes the structure of the `assets` table in its [client-side schema](/intro/setup-guide#define-your-client-side-schema), defining it as `assets_v2`, while older app versions still define `assets`. Define a new stream alongside the existing one, using an alias to map the source `assets` table to the new client-side name:
```yaml theme={null}
streams:
# Old stream, kept for backward compatibility with older app versions.
# Remove it once those versions are no longer in use.
user_assets:
query: SELECT * FROM assets WHERE user_id = auth.user_id()
# New app versions subscribe to this stream.
# The alias maps the table to the new client-side name.
user_assets_v2:
query: SELECT * FROM assets AS assets_v2 WHERE user_id = auth.user_id()
```
```js theme={null}
// New app versions subscribe to the new stream
const subscription = await db.syncStream('user_assets_v2').subscribe();
```
Once the older app versions are no longer in use, remove the old stream from your configuration and deploy the change.
## Versioning with Connection Parameters
Alternatively, clients can pass their version to the PowerSync Service as a [connection parameter](/sync/streams/parameters#connection-parameters), and stream queries filter on it so each client only receives data for its version. This approach is useful when your streams are auto-subscribed: auto-subscribed streams sync to every client on connect, so clients cannot select a stream version by name. In legacy [Sync Rules](/sync/rules/overview), connection parameters are called [client parameters](/sync/rules/client-parameters).
The example below implements the same `assets` use case, with both stream versions auto-subscribed and filtered by a `schema_version` connection parameter:
```yaml theme={null}
# Client passes connection params: {"schema_version": }
streams:
user_assets:
auto_subscribe: true
query: SELECT * FROM assets
WHERE user_id = auth.user_id()
AND connection.parameter('schema_version') = '1'
user_assets_v2:
auto_subscribe: true
query: SELECT * FROM assets AS assets_v2
WHERE user_id = auth.user_id()
AND connection.parameter('schema_version') = '2'
```
```yaml theme={null}
# Client passes in: "params": {"schema_version": }
user_assets:
parameters: SELECT request.user_id() AS user_id
WHERE request.parameters() ->> 'schema_version' = '1'
data:
- SELECT * FROM assets WHERE user_id = bucket.user_id
user_assets_v2:
parameters: SELECT request.user_id() AS user_id
WHERE request.parameters() ->> 'schema_version' = '2'
data:
- SELECT * FROM assets AS assets_v2 WHERE user_id = bucket.user_id
```
Handle queries based on parameters set by the client with care. The client can send any value for these parameters, so it's not a good place to do authorization. If the parameter must be authenticated, use parameters from the JWT instead.
# Advanced Topics
Source: https://docs.powersync.com/sync/advanced/overview
Advanced Sync Streams and Sync Rules topics.
# Partitioned Tables (Postgres)
Source: https://docs.powersync.com/sync/advanced/partitioned-tables
Sync data from Postgres partitioned tables using wildcard table name matching.
For partitioned tables in Postgres, each individual partition is replicated and processed using [Sync Streams](/sync/streams/overview) (or legacy [Sync Rules](/sync/rules/overview)).
To use the same queries and same output table name for each partition, use `%` for wildcard suffix matching of the table name:
```yaml theme={null}
streams:
user_todos:
queries:
# Wildcard matches all user partition tables (e.g. users_2024, users_2025)
- SELECT * FROM "users_%" WHERE id = auth.user_id()
# Wildcard matches all todo partition tables (e.g. todos_2024, todos_2025)
- SELECT * FROM "todos_%" AS todos WHERE user_id = auth.user_id()
```
```yaml theme={null}
by_user:
# Use wildcard in a parameter query
parameters: SELECT id AS user_id FROM "users_%"
data:
# Use wildcard in a data query
- SELECT * FROM "todos_%" AS todos WHERE user_id = bucket.user_id
```
The wildcard character can only be used as the last character in the table name. To match tables across multiple schemas instead, see [Wildcard Schemas](/sync/advanced/schemas-and-connections#wildcard-schemas-postgres).
When using wildcard table names, you can filter on the original table suffix. In Sync Streams, use the `table_suffix()` function, prefixed with the table name or alias from the `FROM` clause (requires PowerSync Service v1.24.0 or later). In legacy Sync Rules, the suffix is available as the special `_table_suffix` column instead:
```yaml theme={null}
config:
edition: 3
streams:
active_todos:
query: SELECT * FROM "todos_%" AS todos WHERE todos.table_suffix() != 'archived'
```
```sql theme={null}
SELECT * FROM "todos_%" AS todos WHERE _table_suffix != 'archived'
```
When no table alias is provided, the original table name is preserved.
`publish_via_partition_root` on the publication is not supported — the individual partitions must be published.
# Prioritized Sync
Source: https://docs.powersync.com/sync/advanced/prioritized-sync
Prioritize which tables sync first so users can start working immediately while remaining data continues loading in the background.
## Overview
PowerSync supports defining sync priorities, which allows you to control the sync order for different data. This is particularly useful when certain data should be available sooner than others.
In Sync Streams, priorities are assigned to streams and PowerSync manages the underlying buckets internally. (In legacy Sync Rules, priorities were assigned to buckets explicitly.)
**Availability**
This feature was introduced in version **1.7.1** of the PowerSync Service, and in the following SDK versions:
* [Flutter v1.12.0](/client-sdks/reference/flutter)
* [React Native v1.18.1](/client-sdks/reference/react-native-and-expo)
* [JavaScript Web v1.14.2](/client-sdks/reference/javascript-web)
* [Kotlin v1.0.0-BETA26](/client-sdks/reference/kotlin)
* [Swift v1.0.0-Beta.8](/client-sdks/reference/swift)
* [.NET v0.0.6-alpha.1](/client-sdks/reference/dotnet)
## Why Use Sync Priorities?
PowerSync's standard sync protocol ensures that:
* The local data view is only updated when a fully consistent checkpoint is available.
* All pending local changes must be uploaded, acknowledged, and synced back before new data is applied.
While this guarantees consistency, it can lead to delays, especially for large datasets or continuous client-side updates. Sync priorities provide a way to speed up syncing of high-priority data while still maintaining overall integrity.
## How It Works
Each bucket is assigned a priority value between 0 and 3, where:
* 0 is the highest priority and has special behavior (detailed below).
* 3 is the default and lowest priority.
* Lower numbers indicate higher priority.
Higher-priority data syncs first, and lower-priority data syncs later. If you only use a single priority, there is no difference between priorities 1-3. The difference only comes in when you use multiple different priorities.
In Sync Streams, you assign priorities directly to streams. PowerSync manages buckets internally, so you don't need to think about bucket structure. Each stream with a given priority will have its data synced at that priority level.
```yaml theme={null}
streams:
lists:
auto_subscribe: true
query: SELECT * FROM lists WHERE owner_id = auth.user_id()
priority: 1 # Syncs first
todos:
auto_subscribe: true
query: SELECT * FROM todos WHERE list_id IN (SELECT id FROM lists WHERE owner_id = auth.user_id())
priority: 2 # Syncs after lists
```
Clients can also override the priority when subscribing:
```js theme={null}
// Override the stream's default priority for this subscription
const sub = await db.syncStream('todos', { list_id: 'abc' }).subscribe({ priority: 1 });
```
When different components subscribe to the same stream with the same parameters but different priorities, PowerSync uses the highest priority for syncing. That higher priority is kept until the subscription ends (or its TTL expires). Subscriptions with different parameters are independent and do not conflict.
In Sync Rules, you assign priorities to bucket definitions. The priority determines when data in that bucket syncs relative to other buckets.
```yaml theme={null}
bucket_definitions:
user_lists:
priority: 1 # Syncs first
parameters: SELECT id AS list_id FROM lists WHERE user_id = request.user_id()
data:
- SELECT * FROM lists WHERE id = bucket.list_id
user_todos:
priority: 2 # Syncs after lists
parameters: SELECT id AS list_id FROM lists WHERE user_id = request.user_id()
data:
- SELECT * FROM todos WHERE list_id = bucket.list_id
```
## Syntax and Configuration
In Sync Streams, set the `priority` option on the stream definition:
```yaml theme={null}
streams:
high_priority_data:
auto_subscribe: true
query: SELECT * FROM important_table WHERE user_id = auth.user_id()
priority: 1
low_priority_data:
auto_subscribe: true
query: SELECT * FROM background_table WHERE user_id = auth.user_id()
priority: 2
```
In Sync Rules, priorities can be defined using the `priority` YAML key on bucket definitions, or with the `_priority` attribute inside parameter queries:
```yaml theme={null}
bucket_definitions:
# Using the `priority` YAML key
user_data:
priority: 1
parameters: SELECT request.user_id() AS id WHERE ...
data:
# ...
# Using the `_priority` attribute (useful for multiple parameter queries with different priorities)
project_data:
parameters: SELECT id AS project_id, 2 AS _priority FROM projects WHERE ...
data:
# ...
```
Priorities must be static and cannot depend on row values within a parameter query.
## Example: Syncing Lists Before Todos
Consider a scenario where you want to display lists immediately while loading todos in the background. This approach allows users to view and interact with lists right away without waiting for todos to sync.
```yaml theme={null}
config:
edition: 3
streams:
lists:
auto_subscribe: true
query: SELECT * FROM lists WHERE owner_id = auth.user_id()
priority: 1 # Syncs first
todos:
auto_subscribe: true
query: |
SELECT * FROM todos
WHERE list_id IN (SELECT id FROM lists WHERE owner_id = auth.user_id())
priority: 2 # Syncs after lists
```
The `lists` stream syncs first (priority 1), allowing users to see and interact with their lists immediately. The `todos` stream syncs afterward (priority 2), loading in the background.
```yaml theme={null}
bucket_definitions:
user_lists:
priority: 1 # Syncs first
parameters: SELECT id AS list_id FROM lists WHERE user_id = request.user_id()
data:
- SELECT * FROM lists WHERE id = bucket.list_id
user_todos:
priority: 2 # Syncs after lists
parameters: SELECT id AS list_id FROM lists WHERE user_id = request.user_id()
data:
- SELECT * FROM todos WHERE list_id = bucket.list_id
```
The `user_lists` bucket syncs first (priority 1), allowing users to see and interact with their lists immediately. The `user_todos` bucket syncs afterward (priority 2), loading in the background.
## Behavioral Considerations
* **Interruption for Higher Priority Data**: Syncing lower-priority data *may* be interrupted if new data for higher-priority streams/buckets arrives.
* **Local Changes & Consistency**: If local writes fail due to validation or permission issues, they are only reverted after *all* data has synced.
* **Deleted Data**: Deleted data may only be removed after *all* priorities have completed syncing. Future updates may improve this behavior.
* **Data Ordering**: Lower-priority data will never appear before higher-priority data.
## Special Case: Priority 0
Priority 0 buckets sync regardless of pending uploads.
For example, in a collaborative document editing app (e.g., using Yjs), each change is stored as a separate row. Since out-of-order updates don’t affect document integrity, Priority 0 can ensure immediate availability of updates.
Caution: If misused, Priority 0 may cause flickering or inconsistencies, as updates could arrive out of order.
## Consistency Considerations
PowerSync's full consistency guarantees only apply once all priorities have completed syncing.
When higher-priority data is synced, all inserts and updates at that priority level will be consistent. However, deletes are only applied when the full sync completes, so you may still have some stale data at those priority levels.
Consider the following example:
Imagine a task management app where users create lists and todos. Some users have millions of todos. To improve first-load speed:
* Lists are assigned Priority 1, syncing first to allow UI rendering.
* Todos are assigned Priority 2, loading in the background.
Now, if another user adds new todos, it’s possible for the list count (synced at Priority 1) to temporarily not match the actual todos (synced at Priority 2). If real-time accuracy is required, both lists and todos should use the same priority.
## Client-Side Considerations
PowerSync's client SDKs provide APIs to allow applications to track sync status at different priority levels. Developers can leverage these to ensure critical data is available before proceeding with UI updates or background processing. This includes:
1. `waitForFirstSync(priority: int)`. When passing the optional `priority` parameter to this method, it will wait for specific priority level to complete syncing.
2. `SyncStatus.priorityStatusEntries()` A list containing sync information for each priority that was seen by the PowerSync Service.
3. `SyncStatus.statusForPriority(priority: int)` This method takes a fixed priority and returns the sync state for that priority by looking it up in `priorityStatusEntries`.
## Example
Using the above we can render a lists component only once the user's lists (with priority 1) have completed syncing, else display a message indicating that the sync is still in progress:
```dart theme={null}
// Define the priority level for lists
static final _listsPriority = BucketPriority(1);
@override
Widget build(BuildContext context) {
// Use FutureBuilder to wait for the first sync of the specified priority to complete
return FutureBuilder(
future: db.waitForFirstSync(priority: _listsPriority),
builder: (context, snapshot) {
if (snapshot.connectionState == ConnectionState.done) {
// Use StreamBuilder to render the lists once the sync completes
return StreamBuilder(
stream: TodoList.watchListsWithStats(),
builder: (context, snapshot) {
if (snapshot.data case final todoLists?) {
return ListView(
padding: const EdgeInsets.symmetric(vertical: 8.0),
children: todoLists.map((list) {
return ListItemWidget(list: list);
}).toList(),
);
} else {
return const CircularProgressIndicator();
}
},
);
} else {
return const Text('Busy with sync...');
}
},
);
}
```
Example implementations of prioritized sync are also available in the following apps:
* Flutter: [Supabase To-Do List](https://github.com/powersync-ja/powersync.dart/tree/main/demos/supabase-todolist)
* Kotlin:
* [Supabase To-Do List (KMP)](https://github.com/powersync-ja/powersync-kotlin/blob/main/demos/supabase-todolist/shared/src/commonMain/kotlin/com/powersync/demos/App.kt#L46)
* [Supabase To-Do List (Android)](https://github.com/powersync-ja/powersync-kotlin/blob/main/demos/android-supabase-todolist/src/main/java/com/powersync/androidexample/screens/HomeScreen.kt#L69)
* Swift: [Supabase To-Do List](https://github.com/powersync-ja/powersync-swift/tree/main/Demos/PowerSyncExample)
# Reducing Bucket Count
Source: https://docs.powersync.com/sync/advanced/reducing-bucket-count
Diagnose a high bucket count, reduce the number of buckets a user syncs, and raise the per-user limits when needed.
If a user syncs too many buckets, or you hit a `PSYNC_S2305` error, this page shows how to find the cause and bring the count down. For how buckets are counted in the first place, see [Bucket Count](/sync/streams/bucket-count).
PowerSync enforces two limits per user, both with a default of 1,000. One is the number of unique buckets. The other is the number of parameter query results, counted before duplicates are removed. Exceeding either fails the sync with a `PSYNC_S2305` error. The fix is different for each, so start by finding out which one you hit from the error message. See [Limits](/sync/streams/bucket-count#limits) for the full difference.
## Diagnosing High Bucket Count
### Reading the Error Message First
The `PSYNC_S2305` message tells you which limit you reached. The fix is different for each, so read it first.
* `Too many buckets` means you reached the bucket limit. Reduce the number of unique buckets. Any strategy below helps.
* `Too many parameter query results` means you reached the parameter limit. Reduce the rows your parameter lookups return. Only some strategies help here: [Denormalizing the Scope Key](#denormalizing-the-scope-key) and [Querying the Membership Table Directly](#querying-the-membership-table-directly) cut the lookups themselves, so they lower both counts.
```mermaid theme={null}
flowchart TD
E["PSYNC_S2305 error"] --> M{"Which message?"}
M -->|"Too many buckets"| Bk["Reduce unique buckets"]
M -->|"Too many parameter query results"| Pr["Reduce parameter rows"]
Bk --> D["Denormalize the scope key,
For a parameter-limit error, the last stream in the breakdown is the one that ran when the limit was reached. This stream is not always the cause. PowerSync adds up parameter results across streams in order. The last stream is only the one that exceeded the limit. Check every stream in the breakdown, not just the last one.
### Checkpoint Logs
Checkpoint logs record the counts for each connection. Find them in your [instance logs](/maintenance-ops/monitoring-and-alerting). For example:
```text theme={null}
New checkpoint: 800178 | write: null | buckets: 7 | param_results: 6 ["5#org_data|0[\"ef718ff3...\"]","5#org_data|1[\"1ddeddba...\"]", ...]
```
* `buckets` is the number of unique buckets for this connection.
* `param_results` is the total number of parameter rows for this connection.
* The array lists the bucket names. Each name already includes its parameter value. The list stops after 20 names.
### Sync Diagnostics Client
The [Sync Diagnostics Client](/tools/diagnostics-client) shows the buckets for one user. It does not load for a user who is over the limit, because that user's sync fails before the data loads. Use the instance logs and the error breakdown for those users. The client shows the bucket count, which may not be the limit you reached. Confirm the limit from the error message.
## The Example App
Every example on this page uses the same app. It has organizations, projects, tasks, assets, and documents.
```mermaid theme={null}
erDiagram
users ||--o{ org_membership : has
orgs ||--o{ org_membership : has
orgs ||--o{ projects : has
projects ||--o{ tasks : has
projects ||--o{ project_assets : links
assets ||--o{ project_assets : links
orgs ||--o{ documents : has
```
Here is how the tables relate:
* A user belongs to one or more organizations. The `org_membership` table links a `user_id` to an `org_id`.
* Each project belongs to one organization through `org_id`. Each project also has a plain text `region` column such as `us-east`.
* Each task belongs to one project through `project_id`.
* Projects and assets link through the `project_assets` table.
* Each document has an `owner_id`, a `shared_with` user, and an `org_id`.
* The `categories` table is shared reference data. It has no owner and no links to the other tables, so it does not appear in the diagram above.
## Reducing Bucket Count
Start with the strategy that matches your query pattern. Most high counts come from hierarchical or many-to-many data, where denormalizing the scope key gives the biggest reduction.
### Multiple Queries per Stream
**Reduces:** bucket count.
Use `queries` instead of separate streams to group related tables. All queries in a stream that filter the same way share one bucket per value. See [multiple queries per stream](/sync/streams/queries#multiple-queries-per-stream).
**Before**: 5 separate streams, each with a direct `auth.user_id()` filter, create 5 buckets per user.
**After**: 1 stream with 5 queries creates 1 bucket per user.
```yaml theme={null}
streams:
user_settings: # [!code --]
query: SELECT * FROM settings WHERE user_id = auth.user_id() # [!code --]
user_prefs: # [!code --]
query: SELECT * FROM preferences WHERE user_id = auth.user_id() # [!code --]
user_org_list: # [!code --]
query: SELECT * FROM org_membership WHERE user_id = auth.user_id() # [!code --]
user_region: # [!code --]
query: SELECT * FROM region_members WHERE user_id = auth.user_id() # [!code --]
user_profile: # [!code --]
query: SELECT * FROM profiles WHERE user_id = auth.user_id() # [!code --]
user_data: # [!code ++]
queries: # [!code ++]
- SELECT * FROM settings WHERE user_id = auth.user_id() # [!code ++]
- SELECT * FROM preferences WHERE user_id = auth.user_id() # [!code ++]
- SELECT * FROM org_membership WHERE user_id = auth.user_id() # [!code ++]
- SELECT * FROM region_members WHERE user_id = auth.user_id() # [!code ++]
- SELECT * FROM profiles WHERE user_id = auth.user_id() # [!code ++]
```
### Denormalizing the Scope Key
**Reduces:** bucket count and parameter query results.
This is the most effective fix for parent-child data. When chained queries through org → project → task create too many buckets, filter every table with the same top-level parameter, such as `org_id`. A bucket's key must be a column on the table you sync (see [The Partition Key Must Exist on the Row](#the-partition-key-must-exist-on-the-row) below). So this works only if the child tables have that column. If tasks only have `project_id`, add `org_id` to the tasks table.
**Before**: chained queries create 10 + 500 = 510 buckets for 10 orgs with 50 projects each. Projects and tasks share buckets because they use the same filter. Orgs use a different filter, so they add their own buckets.
**After**: add `org_id` to the tasks table, drop the `user_projects` CTE, and filter every table by org. This creates 10 buckets.
```yaml theme={null}
streams:
org_projects_tasks:
with:
user_orgs: SELECT org_id FROM org_membership WHERE user_id = auth.user_id()
user_projects: SELECT id FROM projects WHERE org_id IN (SELECT org_id FROM org_membership WHERE user_id = auth.user_id()) # [!code --]
queries:
- SELECT * FROM orgs WHERE id IN user_orgs
- SELECT * FROM projects WHERE id IN user_projects # [!code --]
- SELECT * FROM projects WHERE org_id IN user_orgs # [!code ++]
- SELECT * FROM tasks WHERE project_id IN user_projects # [!code --]
- SELECT * FROM tasks WHERE org_id IN user_orgs # [!code ++]
```
### Querying the Membership Table Directly
**Reduces:** bucket count and parameter query results.
When a subquery or JOIN through a membership table creates N buckets, query the membership table directly with a direct auth filter. Use no subquery and no JOIN. You often need fields from the related table, such as the org name, alongside each membership row. Denormalize those fields onto the membership table so they are available without a JOIN.
**Before**: N org memberships create N buckets.
**After**: 1 bucket per user, with org fields denormalized onto `org_membership`.
```yaml theme={null}
streams:
org_data: # [!code --]
query: SELECT * FROM orgs WHERE id IN (SELECT org_id FROM org_membership WHERE user_id = auth.user_id()) # [!code --]
my_org_memberships: # [!code ++]
query: SELECT * FROM org_membership WHERE user_id = auth.user_id() # [!code ++]
```
### Many-to-Many via a JSON Array Column
**Reduces:** bucket count.
A join through a link table creates one bucket per row of the table you select from. For assets linked to projects through `project_assets`, you get one bucket per asset.
Add a denormalized `project_ids` JSON array column to `assets`, maintained with database triggers. Then use `json_each()` to traverse it. This lets PowerSync key the bucket by project ID instead of asset ID.
**Before**: one bucket per asset. 2,000 assets create 2,000 buckets.
**After**: key by project. 50 projects create 50 buckets.
```yaml theme={null}
streams:
assets_in_projects:
with:
user_projects: SELECT id FROM projects WHERE org_id IN (SELECT org_id FROM org_membership WHERE user_id = auth.user_id())
query: SELECT assets.* FROM assets JOIN project_assets ON project_assets.asset_id = assets.id WHERE project_assets.project_id IN user_projects # [!code --]
query: SELECT assets.* FROM assets INNER JOIN json_each(assets.project_ids) AS p INNER JOIN user_projects ON p.value = user_projects.id # [!code ++]
```
The `INNER JOIN user_projects` syncs only assets that belong to at least one of the user's projects. The bucket key is the project ID, so the count matches the number of projects, not assets.
### Subscription Parameters for On-Demand Sync
**Reduces:** bucket count.
Buckets are created per active subscription, not from every possible value. Use `subscription.parameter('project_id')` so the count is bounded by how many subscriptions the client has active.
**Before**: a subquery returns all of the user's projects. 50 projects create 50 buckets.
**After**: the client subscribes per project on demand. 3 open projects create 3 buckets.
```yaml theme={null}
streams:
project_tasks:
with:
user_projects: SELECT id FROM projects WHERE org_id IN (SELECT org_id FROM org_membership WHERE user_id = auth.user_id())
query: SELECT * FROM tasks WHERE project_id IN user_projects # [!code --]
query: SELECT * FROM tasks WHERE project_id = subscription.parameter('project_id') AND project_id IN user_projects # [!code ++]
```
The client subscribes when the user opens a project and unsubscribes when they leave. This works only when the user does not need every record available offline at the same time.
## Edge Cases and Gotchas
### The Partition Key Must Exist on the Row
A bucket's key must be a value that physically exists on a row of the table you sync. You cannot split a table into buckets by a column it does not have. This is why denormalizing the scope key onto child tables is the standard fix. If tasks only have `project_id`, you cannot key their buckets by `org_id` until you add `org_id` to the tasks table.
### Subscription Parameters Choose Buckets, Not Re-Partition Them
A subscription parameter lets the client choose which existing buckets to sync. It does not change how those buckets are defined.
For a parameter to select a bucket, its value must match a value on the row being synced. For example, each task has a `project_id`, so you can use that column to group tasks into project buckets:
```yaml theme={null}
streams:
project_tasks:
query: SELECT * FROM tasks WHERE project_id = subscription.parameter('project_id')
```
Assets are different. An asset can belong to multiple projects, so the asset row does not have a single `project_id`. Passing a `project_id` as a subscription parameter therefore cannot make PowerSync group those assets by project. The asset row has no project ID to match against.
If you want to sync assets by project, the asset row needs to contain a project reference first. For example, you could add a `project_ids` array column as described in [Reducing Bucket Count](#reducing-bucket-count).
### Correlated Joins Behave Like Subqueries
A correlated JOIN and an `IN (subquery)` compile to the same internal form. They create the same number of buckets. Rewriting one as the other does not reduce the count.
### CTEs Cannot Reference Each Other
Each CTE must be self-contained. A CTE cannot reference another CTE by name. If it does, the deploy fails. Inline the nested subquery instead. See [CTE limitations](/sync/streams/ctes#limitations).
### Global Buckets Multiply Storage and Cost
A stream with no filter creates one global bucket that every user syncs. Under `auto_subscribe: true`, every write to that table fans out to every user. This drives up synced data volume and cost. Scope global buckets carefully, and only mark truly shared reference data as global.
### Bucket Storage Does Not Shrink When You Archive
Buckets are append-only. Marking a row as archived does not remove it from bucket storage on its own. A row leaves storage only when it stops matching the data query, through a hard delete or a filter on the table's own column. Storage reclaims space during [compaction](/maintenance-ops/compacting-buckets). Filtering through a parent table does not shrink a child table's stored data.
## Increasing the Limit
Raise the limit only after you exhaust the reduction strategies above.
Before you raise it, weigh the cost. Sync overhead scales roughly linearly with the number of buckets per user. Doubling the bucket count roughly doubles sync latency for a single operation. It also roughly doubles CPU and memory use on the server and the client. Many operations inside a single bucket scale much more efficiently than many buckets. The 1,000 default exists to encourage fewer, larger buckets and to protect the service from excessive counts.
On PowerSync Cloud, you can request a higher limit on [Team and Enterprise](https://www.powersync.com/pricing) plans, up to 10,000. The limit applies per user, so your instance can still track far more buckets in total.
For self-hosted deployments, set the limits under `api.parameters`:
```yaml service.yaml theme={null}
api:
parameters:
max_buckets_per_connection: 5000
max_parameter_query_results: 5000
```
Set both. Raising one without the other still leaves you capped by the limit you did not change.
## Related Pages
* [Bucket Count](/sync/streams/bucket-count) explains how buckets are counted and the two limits.
* [Writing Queries](/sync/streams/queries) covers the query syntax that determines your bucket count.
* [Common Table Expressions (CTEs)](/sync/streams/ctes) covers shared filtering logic.
* [Troubleshooting](/debugging/troubleshooting#psync_s2305-too-many-buckets-/-parameter-query-results) covers the `PSYNC_S2305` error.
* [Performance and Limits](/resources/performance-and-limits) lists the Service limits.
# Schemas and Connections
Source: https://docs.powersync.com/sync/advanced/schemas-and-connections
Configure Postgres schema usage in Sync Streams/Rules queries, including wildcard schemas for schema-per-tenant setups, and connect to high-availability replicas.
## Schemas (Postgres)
When no schema is specified, the Postgres `public` schema is used for every query. A different schema can be specified as a prefix:
```sql theme={null}
-- Note: the schema must be in double quotes
SELECT * FROM "other"."assets"
```
## Wildcard Schemas (Postgres)
Wildcard schemas require [Sync Streams](/sync/streams/overview) and PowerSync Service v1.24.0 or later. They are currently only supported for Postgres connections.
Use `%` as a wildcard in the schema name to match tables with the same name across multiple schemas. `"%"` matches every schema, and a prefix such as `"tenant_%"` matches every schema whose name starts with `tenant_`. The wildcard can only be the last character of the schema name. Postgres system schemas (`pg_*` and `information_schema`) are never matched.
Combine a wildcard schema with the [`schema()` function](/sync/supported-sql#functions), which returns the schema each row was replicated from, to filter rows by schema. This supports schema-per-tenant databases (a single database with one identical schema per tenant): one stream covers every tenant schema, and each client syncs only its own tenant's data, resolved from a JWT claim.
```yaml theme={null}
config:
edition: 3
streams:
work_orders:
query: SELECT * FROM "%".work_orders WHERE work_orders.schema() = auth.parameter('tenant_schema')
```
In this example, rows are grouped into a bucket per schema, and each client syncs only the bucket matching the `tenant_schema` claim in its JWT. Rows from all matched schemas sync into a single client-side table, named after the table in the query (`work_orders` here).
Each matched table must be part of the [PowerSync publication](/configuration/source-db/setup#3-create-powersync-publication). Tables that are not in the publication are skipped.
## High Availability / Replicated Databases (Postgres)
When the source Postgres database is replicated, for example with Amazon RDS Multi-AZ deployments, specify a single connection with multiple host endpoints. Each host endpoint will be tried in sequence, with the first available primary connection being used.
For this, each endpoint must point to the same physical database, with the same replication slots. This is the case when block-level replication is used between the databases, but not when streaming physical or logical replication is used. In those cases, replication slots are unique on each host, and all data would be re-synced in a fail-over event.
## Multiple Separate Database Connections (Planned)
This feature will be available in a future release. See this [item on our roadmap](https://roadmap.powersync.com/c/84-support-for-sharding-multiple-database-connections).
In the future, it will be possible to configure PowerSync with multiple separate source database connections, where each connection is concurrently replicated.
You should not add multiple connections to multiple replicas of the same database — this would cause data duplication. Only use this when the data on each connection does not overlap.
It will be possible for each connection to be configured with a "tag", to distinguish these connections in Sync Rules. The same tag may be used for multiple connections (if the schema is the same in each).
By default, queries will reference the "default" tag. To use a different connection or connections, assign a different tag, and specify it in the query as a schema prefix. In this case, the schema itself must also be specified.
```sql theme={null}
-- Note the usage of quotes here
SELECT * FROM "secondconnection.public"."assets"
```
# Sharded Databases
Source: https://docs.powersync.com/sync/advanced/sharded-databases
Sync data from sharded Postgres databases with per-shard PowerSync connection configuration.
In the case of Postgres, PowerSync cannot replicate Postgres [foreign tables](https://www.postgresql.org/docs/current/ddl-foreign-data.html).
However, PowerSync does have options available to support sharded databases in general.
When using MongoDB, MySQL, or SQL Server as the backend source database, PowerSync does not currently support connecting to sharded clusters.
The primary options are:
1. Use a separate PowerSync Service instance per database.
2. Add a connection for each database in the same PowerSync Service instance ([planned](https://roadmap.powersync.com/c/84-support-for-sharding-multiple-database-connections); this capability will be available in a future release).
Where feasible, using separate PowerSync Service instances would give better performance and give more control over how changes are rolled out, especially around Sync Rule reprocessing.
Some specific scenarios:
#### 1. Different Tables on Different Databases
This is common when separate "services" use separate databases, but multiple tables across those databases need to be synced to the same users.
Use a single PowerSync Service instance, with a separate connection for each source database ([planned](https://roadmap.powersync.com/c/84-support-for-sharding-multiple-database-connections); this capability will be available in a future release). Use a unique [connection tag](/sync/advanced/schemas-and-connections) for each source database, allowing them to be distinguished in your [Sync Streams](/sync/streams/overview) or [Sync Rules](/sync/rules/overview).
#### 2a. All Data for a Single Customer Is Contained in a Single Shard
This is common when sharding per customer account / organization.
In this case, use a separate PowerSync Service instance for each database.
#### 2b. Most Customer Data Is in a Single Shard, but Some Data Is in a Shared Database
If the amount of shared data is small, still use a separate PowerSync Service instance for each database, but also add the shared database connection to each PowerSync Service instance using a separate connection tag ([planned](https://roadmap.powersync.com/c/84-support-for-sharding-multiple-database-connections); this capability will be available in a future release).
#### 2c. Data for a Single Customer Is Split Across Multiple Shards
This is more complicated than the other cases listed above. Please [reach out to us](/resources/contact-us) if this is your architecture.
#### 3. Only Some Tables Are Sharded
In some cases, most tables would be on a shared server, with only a few large tables being sharded.
For this case, use a single PowerSync Service instance. Add each shard as a new connection on this instance ([planned](https://roadmap.powersync.com/c/84-support-for-sharding-multiple-database-connections); this capability will be available in a future release) — all with the same connection tag, so that the same [Sync Streams](/sync/streams/overview) or [Sync Rules](/sync/rules/overview) applies to each.
# Storage Version 4
Source: https://docs.powersync.com/sync/advanced/storage-version-4
Opt in to storage version 4 for faster sync, incremental reprocessing of Sync Streams changes, and S3 object storage.
Storage version 4 is a new version of the format the PowerSync Service uses to store the data it syncs to clients. It is in [Beta](/resources/feature-status) as of PowerSync Service v1.26.0.
Compared to version 2, it provides:
* Faster sync and faster reprocessing after a deployment.
* [Incremental reprocessing](#incremental-reprocessing): a Sync Streams deployment reprocesses only the streams you added or changed. Clients no longer download all their data again after every deployment.
* [S3 object storage](#s3-object-storage): larger blocks of synced data move from the storage database to S3. This reduces load on the storage database when many clients sync at once or sync large amounts of data.
## Availability
Storage version 4 is compatible with all PowerSync Cloud instances, which already use MongoDB [bucket storage](/architecture/powersync-service#bucket-storage). Self-hosted instances must also use MongoDB bucket storage. Postgres bucket storage is not currently supported.
The PowerSync Cloud and self-hosted columns below apply during the Beta only. Once storage version 4 is generally available, it will become the default for all supported instances. S3 object storage is then also enabled on all PowerSync Cloud instances. For self-hosted deployments, follow the [S3 setup instructions](#self-hosted-s3-setup).
| | Source database | Sync Config | PowerSync Cloud (Beta) | Self-hosted (Beta) |
| ------------------------ | --------------- | -------------------------- | ----------------------------------------------------- | ------------------------------------------------- |
| Storage version 4 | Any | Sync Streams or Sync Rules | Free plan: automatic. Other plans: [opt in](#opt-in). | [Opt in](#opt-in) |
| Incremental reprocessing | MongoDB | Sync Streams | Included with version 4 | Included with version 4 |
| S3 object storage | Any | Sync Streams or Sync Rules | Enabled per instance by PowerSync on request | [Set up S3 object storage](#self-hosted-s3-setup) |
Incremental reprocessing for Postgres and other source databases is planned. See the [proposal](https://github.com/orgs/powersync-ja/discussions/349) for background. It is not supported for legacy [Sync Rules](/sync/rules/overview). If you still use Sync Rules, [migrate to Sync Streams](/sync/streams/migration).
## Opt In
Version 4 is not the default in PowerSync Service v1.26.0. Moving a Sync Config to version 4 runs like any other deployment:
1. PowerSync reprocesses all data selected by your Sync Config in the background. The current version keeps serving clients, so there is no downtime.
2. When the new copy is ready, PowerSync switches to it. On PowerSync Cloud, this appears as a new deploy event in the PowerSync Dashboard.
3. Clients download their data again once, as after any deployment before version 4. On self-hosted deployments with many clients, scale out the API before the switch to absorb the re-sync.
After this first deployment, later Sync Streams deployments use incremental reprocessing automatically when your instance meets its requirements. There is no separate setting.
### PowerSync Cloud
Free plan instances are upgraded automatically during the Beta. No action is needed.
On other plans, add `storage_version: 4` to the `config` block of your Sync Config and deploy it:
```yaml theme={null}
config:
edition: 3
storage_version: 4
streams:
todos:
query: SELECT * FROM todos WHERE owner_id = auth.user_id()
```
### Self-Hosted
Postgres bucket storage is not supported with version 4.
Add `storage_version: 4` to the `config` block of each Sync Config as shown above, then deploy or redeploy it to use version 4.
To move a Sync Config back to version 2, set `storage_version: 2` and deploy again. This is another full reprocess.
To also enable S3 object storage, follow the [self-hosted S3 setup instructions](#self-hosted-s3-setup) to prepare a bucket and configure the Service.
## Incremental Reprocessing
Incremental reprocessing is active when you use a MongoDB source database, Sync Streams, and storage version 4.
Self-hosted instances with Postgres bucket storage are not supported.
Without it, every deployment reads all data selected by the Sync Config from your source database and prepares a complete new copy. Clients then download all their data again, even if only one stream changed.
With incremental reprocessing, PowerSync compares the new Sync Config with the current one and reprocesses only the streams you added or changed. Unchanged streams keep their data on the PowerSync Service and on clients. Deployments finish faster, your source database does less work, and clients download only the data for affected streams they subscribe to.
* Adding a stream reads only the data that stream selects.
* Removing a stream requires no new source reads. PowerSync cleans up stored definitions when no active Sync Config still uses them.
* Renaming a stream counts as removing it and adding a new one, so its data is rebuilt.
* Changing a stream's queries may reprocess affected definitions. Changes that only affect how request parameters select existing buckets do not require reprocessing.
For example, changing `SELECT * FROM projects WHERE user_id = auth.user_id()` to `SELECT * FROM projects WHERE user_id = auth.jwt() ->> 'owner'` reuses the existing bucket data. The data is still grouped by `user_id`; only the JWT field used to select buckets changes.
The time saved depends on how your data is split across streams. If one stream selects most of your data, changing that stream still takes about as long as a full reprocess.
PowerSync favors correctness over reuse. When it cannot confirm that a change leaves a stream's data unchanged, it rebuilds that stream. A deployment that reprocesses more than you expect is not an error.
Event definitions for [Custom Write Checkpoints](/handling-writes/custom-write-checkpoints) follow the same rules. Unchanged events keep their data, and new or changed events are read again.
### When PowerSync Reprocesses Everything
Some changes start a full reprocess, after which clients download all their data again:
* The first deployment on storage version 4.
* Changes to the `config` block of the Sync Config, such as `edition`, compatibility fixes, or `storage_version`.
* The **Defragment** action in the PowerSync Dashboard, which exists to rebuild all data. See [Defragmenting](/maintenance-ops/compacting-buckets#defragmenting).
* Replication failures, for example when PowerSync loses its position in the MongoDB change stream and has to start over.
### Sync Config Versions and Replication Streams
Each deployment has a Sync Config version. With incremental reprocessing, multiple versions can share a replication stream, the replication process and stored state. A full reprocess creates a new replication stream.
See the [Log Reference](/debugging/log-reference#message-prefixes) for how to identify these versions and streams in your logs.
For implementation details, see the [storage design](https://github.com/powersync-ja/powersync-service/blob/main/docs/storage/storage-v3.md). The document describes the design introduced in version 3 and carried into version 4.
### Checking What a Deployment Reprocessed
If a deployment takes longer or reprocesses more than you expect, see [Checking What a Deployment Reprocessed](/debugging/log-reference#checking-what-a-deployment-reprocessed) in the Log Reference for what to look for in your logs.
## S3 Object Storage
Your instance keeps the data it syncs to clients in its bucket storage database, alongside everything else it needs to run. With S3 object storage, larger blocks of that data move to Amazon S3 or an S3-compatible object store, and the PowerSync Service syncs them to clients directly from there. Smaller blocks, and the metadata that locates each block, stay in MongoDB.
Reading larger blocks from S3 reduces the data MongoDB must read and transfer during sync. When those reads limit performance, offloading them can speed up initial sync and let an instance serve more concurrent clients. The benefit is most noticeable when clients sync large amounts of data or many clients connect at once. The PowerSync Service still handles every client connection, so its CPU and memory capacity also limit concurrency.
For self-hosted instances, offloading bucket data to S3 can reduce storage and data transfer costs. Compare the reduction in database costs with the object store's storage, request, and data transfer charges for your workload.
Clients connect only to the PowerSync Service and never to the object store, so no client changes are needed. If the object store becomes unreachable, sync is interrupted until it recovers. Clients reconnect and resume automatically.
S3 object storage requires storage version 4 and works with Sync Streams and legacy Sync Rules. It is compatible with all PowerSync Cloud instances.
S3 object storage holds PowerSync's internal sync data. To store files uploaded by your app, use [Attachments](/client-sdks/advanced/attachments).
### PowerSync Cloud
During the Beta, PowerSync enables S3 object storage per instance. [Contact us](/resources/contact-us) if you want it on your instance before we enable it for all instances.
### Self-Hosted S3 Setup
Self-hosted instances with Postgres bucket storage are not supported.
Create a bucket. Use the same region as the PowerSync Service where possible, to keep latency low and avoid cross-region data transfer charges. Use a dedicated bucket, or a unique `prefix` per PowerSync instance, so that instances never read or delete each other's files. Give the PowerSync Service permission to list the bucket and to read, write, and delete objects under the prefix.
Leave object versioning off, or suspend it if the bucket already has it, and leave Object Lock off. PowerSync deletes files itself once they are no longer needed, so versioning keeps charging for old versions and locked objects cannot be cleaned up. Do not add an expiration lifecycle rule: an expired object may still be referenced by MongoDB, which breaks sync for that data.
Add `object_storage` to the `storage` section of `service.yaml`:
```yaml service.yaml theme={null}
storage:
type: mongodb
uri: !env PS_MONGO_STORAGE_URI
object_storage:
type: s3
bucket: powersync-bucket-data
region: us-east-1
prefix: production
```
Without `access_key_id` and `secret_access_key`, PowerSync uses the AWS credentials available to the process, such as an IAM role. For S3-compatible providers such as MinIO or Cloudflare R2, also set `endpoint`, and set `force_path_style: true` if the provider requires path-style requests.
Restart or redeploy the PowerSync Service to load the updated `service.yaml`. If you run replication, API, and compacting in separate containers or jobs, apply the same object storage configuration to each.
Deploy your Sync Configs on storage version 4 as described in [Opt In](#opt-in). Sync Configs on version 2 keep all data in MongoDB, even when `object_storage` is configured.
Once replication reaches a healthy checkpoint, confirm that objects appear under the prefix, run a test initial sync, and run `compact` once to surface permission errors early.
After enabling S3 object storage, you can raise [`max_concurrent_connections`](/configuration/powersync-service/self-hosted-instances#param-max-concurrent-connections) from its default of 200 per API process. With storage version 4 and S3 object storage, each API process can handle up to 1,000 concurrent client connections. Performance degrades if a large share of those clients run an initial sync at the same time, so scale out the API before a deployment that makes all clients download their data again. More concurrent connections also increase CPU and memory usage.
The [S3 object storage configuration reference](/configuration/powersync-service/self-hosted-instances#param-object-storage) lists all supported settings, including timeouts, request concurrency, and the size threshold below which blocks stay in MongoDB.
Keep the scheduled [compact](/maintenance-ops/compacting-buckets) job running. It removes files that are no longer needed. The `teardown` command deletes PowerSync's files under the prefix before it drops the storage database. The `powersync_object_storage_size_bytes` [metric](/maintenance-ops/self-hosting/monitoring) reports how much object storage PowerSync uses.
# Sync Data by Time with Sync Streams
Source: https://docs.powersync.com/sync/advanced/sync-data-by-time
Filter and sync data based on time ranges using Sync Streams/Sync Rules, with patterns for recent-only and sliding-window queries.
A common need in offline-first apps is syncing data based on time, for example, only syncing issues updated in the last 7 days instead of the entire dataset.
You might expect to write something like:
```yaml theme={null}
# Sync Streams
streams:
issues_after_start_date:
query: SELECT * FROM issues WHERE updated_at > subscription.parameter('start_at')
# Sync Rules
bucket_definitions:
issues_after_start_date:
parameters: SELECT request.parameters() ->> 'start_at' as start_at
data: SELECT * FROM issues WHERE updated_at > bucket.start_date
```
However, this won't work. Here's why.
## The Problem
PowerSync pre-computes and caches which rows belong to which parameters to enable efficient streaming. This means parameter-based filtering is limited to equality checks (`=`, `IN`, `IS NULL`) — range operators like `>`, `<`, `>=`, or `<=` are not supported on parameters.
Additionally, time-based functions like `now()` aren't allowed in parameter expressions because the result changes depending on when the query runs, making pre-computation impossible.
These constraints apply to both Sync Streams and legacy Sync Rules.
This guide covers a few practical workarounds.
We are working on a more elegant solution for this problem. When ready, this guide will be updated accordingly.
## Workarounds
### 1: Pre-Defined Time Ranges
Add a boolean column to your table that indicates whether a row falls within a specific time range. Keep this column updated in your source database using a scheduled job.
For example, add an `updated_this_week` column:
```sql theme={null}
ALTER TABLE issues ADD COLUMN updated_this_week BOOLEAN DEFAULT false;
```
Update it periodically using a cron job (e.g., with `pg_cron`):
```sql theme={null}
UPDATE issues SET updated_this_week = (updated_at > now() - interval '7 days');
```
```yaml theme={null}
config:
edition: 3
streams:
recent_issues:
auto_subscribe: true
query: SELECT * FROM issues WHERE updated_this_week = true
```
For multiple time ranges, define a stream per range and let the client subscribe to the one it needs:
```yaml theme={null}
config:
edition: 3
streams:
issues_1week:
query: SELECT * FROM issues WHERE updated_this_week = true
issues_1month:
query: SELECT * FROM issues WHERE updated_this_month = true
```
The client subscribes to the desired range:
```javascript theme={null}
// Subscribe to one-week range
await db.syncStream('issues_1week').subscribe();
// Or subscribe to one-month range
await db.syncStream('issues_1month').subscribe();
```
```yaml theme={null}
bucket_definitions:
recent_issues:
data:
- SELECT * FROM issues WHERE updated_this_week = true
```
For multiple time ranges, add multiple bucket definitions and let the client choose which bucket to sync:
```yaml theme={null}
bucket_definitions:
issues_1week:
parameters: SELECT WHERE request.parameters() ->> 'range' = '1week'
data:
- SELECT * FROM issues WHERE updated_this_week = true
issues_1month:
parameters: SELECT WHERE request.parameters() ->> 'range' = '1month'
data:
- SELECT * FROM issues WHERE updated_this_month = true
```
The client passes the desired range as a client parameter:
```javascript theme={null}
await db.connect(connector, {
params: {
range: '1week',
},
})
```
This approach works well when you have a small, fixed set of time ranges. However, it requires schema changes and a scheduled job to keep the columns updated.
This approach requires schema changes and scheduled jobs (e.g., `pg_cron`). Limited to pre-defined time ranges.
If you need more flexibility like letting users pick arbitrary date ranges, see Workaround 2 below.
### 2: Buckets Per Date
Instead of pre-defined ranges, create a bucket for each date and let the client specify which dates to sync.
Use `substring` to extract the date portion from a timestamp and match it with `=`:
For a complete working example, see the [PowerSync + Supabase: Time-Based Sync demo](https://github.com/powersync-ja/powersync-js/tree/main/demos/react-supabase-time-based-sync).
```yaml theme={null}
config:
edition: 3
streams:
issues_by_date:
query: SELECT * FROM issues WHERE substring(updated_at, 1, 10) = subscription.parameter('date')
```
The client subscribes once per date it wants to sync:
```javascript theme={null}
await db.syncStream('issues_by_date', { date: '2026-01-07' }).subscribe();
await db.syncStream('issues_by_date', { date: '2026-01-08' }).subscribe();
await db.syncStream('issues_by_date', { date: '2026-01-09' }).subscribe();
```
Each subscription can be managed independently — you can subscribe and unsubscribe to individual dates without affecting others.
```yaml theme={null}
bucket_definitions:
issues_by_update_at:
parameters: SELECT value as date FROM json_each(request.parameters() ->> 'dates')
data:
- SELECT * FROM issues WHERE substring(updated_at, 1, 10) = bucket.date
```
The client passes the dates it wants as client parameters:
```javascript theme={null}
await db.connect(connector, {
params: {
dates: ["2026-01-07", "2026-01-08", "2026-01-09"],
},
})
```
This gives users full control over which dates to sync, with no schema changes or scheduled jobs required.
The trade-off is granularity. In this example we're using daily buckets. If you need finer precision (hourly), syncing a large range means many buckets, which can degrade sync performance and approach [PowerSync's limit of 1,000 buckets per user](/resources/performance-and-limits#limits). If you use larger buckets (monthly), you lose the ability to filter accurately.
You must commit to a single granularity. Daily = too many buckets for long ranges. Monthly = lose precision for recent data.
You have to pick a granularity and stick with it. If that's a problem—say, you want hourly precision for recent data but don't want hundreds of buckets when syncing a full month, see Workaround 3 below.
### 3: Multiple Granularities
Combine multiple granularities in a single definition. This lets you use larger buckets (days) for older data and smaller buckets (hours, minutes) for recent data.
```yaml theme={null}
config:
edition: 3
streams:
issues_by_partition:
queries:
# By day (e.g., "2026-01-07")
- SELECT * FROM issues WHERE substring(updated_at, 1, 10) = subscription.parameter('partition')
# By hour (e.g., "2026-01-07T14")
- SELECT * FROM issues WHERE substring(updated_at, 1, 13) = subscription.parameter('partition')
# By 10 minutes (e.g., "2026-01-07T14:3")
- SELECT * FROM issues WHERE substring(updated_at, 1, 15) = subscription.parameter('partition')
```
The client subscribes once per partition, mixing granularities as needed:
```javascript theme={null}
await db.syncStream('issues_by_partition', { partition: '2026-01-05' }).subscribe();
await db.syncStream('issues_by_partition', { partition: '2026-01-06' }).subscribe();
await db.syncStream('issues_by_partition', { partition: '2026-01-07T10' }).subscribe();
await db.syncStream('issues_by_partition', { partition: '2026-01-07T11' }).subscribe();
await db.syncStream('issues_by_partition', { partition: '2026-01-07T12:0' }).subscribe();
await db.syncStream('issues_by_partition', { partition: '2026-01-07T12:1' }).subscribe();
await db.syncStream('issues_by_partition', { partition: '2026-01-07T12:2' }).subscribe();
```
Each query naturally acts as a filter based on the length of the partition value — a day-format partition only matches the day query, an hour-format partition only matches the hour query, and so on.
```yaml theme={null}
bucket_definitions:
issues_by_time:
parameters: SELECT value as partition FROM json_each(request.parameters() ->> 'partitions')
data:
# By day (e.g., "2026-01-07")
- SELECT * FROM issues WHERE substring(updated_at, 1, 10) = bucket.partition
# By hour (e.g., "2026-01-07T14")
- SELECT * FROM issues WHERE substring(updated_at, 1, 13) = bucket.partition
# By 10 minutes (e.g., "2026-01-07T14:3")
- SELECT * FROM issues WHERE substring(updated_at, 1, 15) = bucket.partition
```
The client then mixes granularities as needed:
```javascript theme={null}
await db.connect(connector, {
params: {
partitions: [
"2026-01-05",
"2026-01-06",
"2026-01-07T10",
"2026-01-07T11",
"2026-01-07T12:0",
"2026-01-07T12:1",
"2026-01-07T12:2"
]
},
})
```
This syncs January 5–6 by day, the morning of January 7 by hour, and the last 30 minutes in 10-minute chunks, without creating hundreds of buckets.
The trade-off is complexity. The client must decide which granularity to use for each time segment, and each row belongs to multiple buckets, which increases replication overhead.
When using multiple time granularities (e.g., monthly, daily, hourly), rows move between buckets as time passes. Since each granularity creates a different bucket ID, the client must re-download the row from the new bucket even if it already has the data. This re-download overhead can nullify the benefits of granular filtering. For this reason, in some cases it may be better to sync entire months avoiding the re-sync overhead, even if you sync more data initially.
Each row belongs to multiple buckets (replication overhead). Re-sync overhead when rows move between bucket granularities. Added complexity may not justify the gains over Workaround 2.
## Conclusion
Time-based sync is a common need, but PowerSync doesn't support range operators or time-based functions on parameters directly.
To recap the workarounds:
* **Pre-defined time ranges** — Simplest option. Use when you have a fixed set of time ranges and don't mind schema changes.
* **Buckets Per Date** — More flexible. Use when you need arbitrary date ranges but can live with a single granularity.
* **Multiple Granularities** — Most flexible. Use when you need precision for recent data without syncing hundreds of buckets. Be mindful of the re-sync overhead.
We're working on a more elegant solution. This guide will be updated when it's ready.
# Grammar Reference (Sync Rules)
Source: https://docs.powersync.com/sync/grammar/sync-rules/index
Railroad diagram reference for the SQL grammar supported in legacy Sync Rules queries.
This page is a formal grammar reference for Sync Rules: it shows the syntax accepted for parameter queries and data queries using railroad diagrams. This page complements the [Supported SQL](/sync/supported-sql) guide, which explains in prose what you can write, with examples and restrictions.
**When to use this page:** If you need to check whether a construct is valid, see how parameter vs data query syntax differs, or you're used to grammar specs, use the diagrams and the "Used by" / "References" links to navigate. For most users just getting started, see [Supported SQL](/sync/supported-sql) and the [Sync Rules](/sync/rules/overview) docs.
## ParameterQuery
```sql theme={null}
SELECT id FROM users WHERE id = token_parameters.user_id
```
[ParameterQuery](#parameterquery)
***
## TableParameterQuery
```sql theme={null}
SELECT id, name FROM users WHERE org_id = token_parameters.org_id
```
[ParameterQuery](#parameterquery)
***
## StaticParameterQuery
```sql theme={null}
SELECT token_parameters.user_id AS id
```
[ParameterQuery](#parameterquery)
***
## DataQuery
```sql theme={null}
SELECT id, name FROM users WHERE users.id = bucket.user_id
```
[TableValuedParameterQuery](#tablevaluedparameterquery), [TableParameterQuery](#tableparameterquery), [StaticParameterQuery](#staticparameterquery), [DataQuery](#dataquery)
***
## JsonEachCall
```sql theme={null}
JSON_EACH(token_parameters.tags)
```
[TableValuedParameterQuery](#tablevaluedparameterquery)
***
## WhereClause
```sql theme={null}
users.id = bucket.user_id AND active = true
```
[TableValuedParameterQuery](#tablevaluedparameterquery), [TableParameterQuery](#tableparameterquery), [StaticParameterQuery](#staticparameterquery), [DataQuery](#dataquery), [WhereClause](#whereclause)
***
## Predicate
```sql theme={null}
status IS NOT NULL
```
[WhereClause](#whereclause)
***
## Expression
```sql theme={null}
price * quantity + tax
```
PowerSync evaluates all binary operators with equal precedence (left to right). Use parentheses to control evaluation order.
| Precedence | Operators | Description |
| ---------- | ------------------ | --------------------------------- |
| 1 | `\|\|` | String concatenation |
| 2 | `*` `/` `%` | Multiplication, division, modulo |
| 3 | `+` `-` | Addition, subtraction |
| 4 | `&` `\|` `<<` `>>` | Bitwise AND, OR, left/right shift |
| 5 | `<` `>` `<=` `>=` | Less than, greater than, etc. |
| 6 | `=` `!=` | Equal, not equal |
**References:** [PrimaryExpression](#primaryexpression), [PropertyAccess](#propertyaccess)
[SelectItem](#selectitem), [JsonEachCall](#jsoneachcall), [Predicate](#predicate), [CastExpression](#castexpression), [FunctionCall](#functioncall), [PrimaryExpression](#primaryexpression)
***
## PropertyAccess
```sql theme={null}
->'address'->>'city'
```
[Expression](#expression)
***
## Reference
```sql theme={null}
users.id
```
[TableParameterQuery](#tableparameterquery), [DataQuery](#dataquery), [FunctionCall](#functioncall), [PrimaryExpression](#primaryexpression)
***
## CastExpression
```sql theme={null}
CAST(age AS TEXT)
```
[PrimaryExpression](#primaryexpression)
***
## FunctionCall
```sql theme={null}
upper(name)
```
[PrimaryExpression](#primaryexpression)
***
## PrimaryExpression
```sql theme={null}
(price + tax)
```
[Expression](#expression)
***
## Lexical Rules
| Token | Examples | Rule |
| --------------------------------- | ------------------------------------------------------------------ | --------------------------------------- |
| [Identifier](#identifier) | `user_id`, `MY_TABLE`, `"Column Name"`, `"with ""quotes"" inside"` | `[A-Z_] ([A-Z_0-9])*` |
| [StringLiteral](#stringliteral) | `'hello'`, `'it''s'`, `''` | `"'" ([\x20-\x26] \| [\x28-\x7E])* "'"` |
| [IntegerLiteral](#integerliteral) | `0`, `42`, `12345` | `([0-9])+` |
| [NumericLiteral](#numericliteral) | `3.14`, `42`, `0.5` | `([0-9])+ ("." ([0-9])+)?` |
### Identifier
Bare identifiers are normalized to lower case and may contain letters, digits, and underscores. Double-quoted identifiers ("name") allow any printable character and support escaped quotes ("").
[TableValuedParameterQuery](#tablevaluedparameterquery), [SelectItem](#selectitem), [Reference](#reference)
### StringLiteral
Single-quoted string literal. Embedded single quotes are escaped by doubling them ('').
[PropertyAccess](#propertyaccess), [PrimaryExpression](#primaryexpression)
### IntegerLiteral
One or more decimal digits (0-9).
[PropertyAccess](#propertyaccess)
### NumericLiteral
Decimal number: one or more digits with an optional fractional part (.digits).
[PrimaryExpression](#primaryexpression)
# Grammar Reference (Sync Streams)
Source: https://docs.powersync.com/sync/grammar/sync-streams/index
Railroad diagram reference for the SQL grammar supported in Sync Streams queries.
This page is a formal grammar reference for Sync Streams: it shows the syntax the compiler accepts using railroad diagrams. This page complements the [Supported SQL](/sync/supported-sql) guide, which explains in prose what you can write, with examples and restrictions.
**When to use this page:** If you need to check whether a construct is valid, see how rules nest (e.g. subqueries, CTEs, JOINs), or you're used to grammar specs, use the diagrams and the "Used by" / "References" links to navigate. For most users just getting started, see [Supported SQL](/sync/supported-sql) and [Writing Queries](/sync/streams/queries).
## SelectStatement
```sql theme={null}
SELECT id, name FROM users WHERE active = true
```
[SelectStatement](#selectstatement), [Subquery](#subquery)
***
## Reference
```sql theme={null}
users.id
```
[SelectItem](#selectitem), [TableSource](#tablesource), [PrimaryExpression](#primaryexpression), [FunctionCall](#functioncall)
***
## FromSource
```sql theme={null}
users
```
[SelectStatement](#selectstatement), [JoinClause](#joinclause), [Subquery](#subquery), [CteDefinition](#ctedefinition)
***
## TableSource
```sql theme={null}
users AS u
```
[FromSource](#fromsource)
***
## TableValuedCall
```sql theme={null}
json_each(data)
```
[TableValuedSource](#tablevaluedsource)
***
## TableValuedSource
```sql theme={null}
json_each(data) AS items
```
[FromSource](#fromsource)
***
## SubquerySource
```sql theme={null}
(SELECT id FROM users) AS u
```
[FromSource](#fromsource)
***
## JoinClause
```sql theme={null}
JOIN orders ON users.id = orders.user_id
```
[SelectStatement](#selectstatement), [Subquery](#subquery), [CteDefinition](#ctedefinition)
***
## WhereClause
```sql theme={null}
active = true AND age > 18
```
[SelectStatement](#selectstatement), [JoinClause](#joinclause), [Condition](#condition), [Subquery](#subquery), [CteDefinition](#ctedefinition)
***
## Condition
```sql theme={null}
age BETWEEN 10 AND 100
```
[WhereClause](#whereclause), [CaseCondition](#casecondition)
***
## PredicateTail
```sql theme={null}
IN (SELECT id FROM users)
```
[Condition](#condition)
***
## InSource
```sql theme={null}
(SELECT id FROM users)
```
[PredicateTail](#predicatetail)
***
## Expression
```sql theme={null}
price * quantity + tax
```
PowerSync evaluates all binary operators with equal precedence (left to right). Use parentheses to control evaluation order.
| Precedence | Operators | Description |
| ---------- | ------------------ | --------------------------------- |
| 1 | `\|\|` | String concatenation |
| 2 | `*` `/` `%` | Multiplication, division, modulo |
| 3 | `+` `-` | Addition, subtraction |
| 4 | `&` `\|` `<<` `>>` | Bitwise AND, OR, left/right shift |
| 5 | `<` `>` `<=` `>=` | Less than, greater than, etc. |
| 6 | `=` `!=` | Equal, not equal |
**References:** [PrimaryExpression](#primaryexpression), [PropertyAccess](#propertyaccess)
[SelectItem](#selectitem), [TableValuedCall](#tablevaluedcall), [Condition](#condition), [PredicateTail](#predicatetail), [InSource](#insource), [PrimaryExpression](#primaryexpression), [SearchedCase](#searchedcase), [WhenClause](#whenclause), [SimpleCase](#simplecase), [WhenValueClause](#whenvalueclause), [CastExpression](#castexpression), [FunctionCall](#functioncall), [CteDefinition](#ctedefinition)
***
## PropertyAccess
```sql theme={null}
->'address'->>'city'
```
[Expression](#expression)
***
## PrimaryExpression
```sql theme={null}
(price + tax)
```
[Expression](#expression)
***
## CaseExpression
```sql theme={null}
CASE WHEN age >= 18 THEN 'adult' ELSE 'minor' END
```
[PrimaryExpression](#primaryexpression)
***
## SearchedCase
```sql theme={null}
CASE WHEN x > 0 THEN 'positive' ELSE 'negative' END
```
[CaseExpression](#caseexpression)
***
## WhenClause
```sql theme={null}
WHEN age >= 18 THEN 'adult'
```
[SearchedCase](#searchedcase), [WhenClause](#whenclause)
***
## CaseCondition
```sql theme={null}
x > 0 AND y IS NOT NULL
```
[WhenClause](#whenclause)
***
## SimpleCase
```sql theme={null}
CASE status WHEN 1 THEN 'active' WHEN 0 THEN 'inactive' END
```
[CaseExpression](#caseexpression)
***
## WhenValueClause
```sql theme={null}
WHEN 1 THEN 'active'
```
[SimpleCase](#simplecase), [WhenValueClause](#whenvalueclause)
***
## CastExpression
```sql theme={null}
CAST(age AS TEXT)
```
[PrimaryExpression](#primaryexpression)
***
## FunctionCall
```sql theme={null}
upper(name)
```
[PrimaryExpression](#primaryexpression)
***
## Subquery
```sql theme={null}
SELECT id FROM orders WHERE orders.user_id = users.id
```
[SubquerySource](#subquerysource), [InSource](#insource)
***
## CteDefinition
```sql theme={null}
SELECT id, upper(name) AS display_name FROM users WHERE active = true
```
[SelectItem](#selectitem), [Reference](#reference), [TableSource](#tablesource), [TableValuedCall](#tablevaluedcall), [TableValuedSource](#tablevaluedsource), [SubquerySource](#subquerysource), [InSource](#insource), [CteDefinition](#ctedefinition)
### StringLiteral
Single-quoted string literal. Embedded single quotes are escaped by doubling them ('').
[PropertyAccess](#propertyaccess), [PrimaryExpression](#primaryexpression)
### IntegerLiteral
One or more decimal digits (0-9).
[PropertyAccess](#propertyaccess)
### NumericLiteral
Decimal number: one or more digits with an optional fractional part (.digits).
[PrimaryExpression](#primaryexpression)
# Sync Streams and Sync Rules
Source: https://docs.powersync.com/sync/overview
PowerSync Sync Streams and the legacy Sync Rules allow developers to control which data syncs to which clients/devices (i.e. they enable partial sync).
## Sync Streams — Recommended
With [Sync Streams](/sync/streams/overview), you write SQL-like queries to define streams of data. Clients subscribe to the streams they need, either on-demand or automatically on connect. Sync Streams are the recommended path to achieve partial sync for both new and existing projects.
Key improvements in Sync Streams over legacy Sync Rules include:
* **On-demand syncing**: You define Sync Streams on the PowerSync Service, and a client can then subscribe to them one or more times with different parameters, on-demand. You still have the option of auto-subscribing streams when a client connects, for "sync data upfront" behavior.
* **Temporary caching-like behavior**: Each subscription includes a configurable TTL that keeps data active after the client unsubscribes, acting as a warm cache for re-subscribing.
* **Simpler developer experience**: Simplified syntax and mental model, and capabilities such as your UI components automatically managing subscriptions (for example, React hooks).
If you're on Sync Rules, you can migrate in a few clicks. Click **Migrate to Sync Streams** in the PowerSync Dashboard, or run `powersync migrate sync-rules` in the CLI to generate a draft from your current config. See the [migration guide](/sync/streams/migration) for details.
You may also find it useful to look at the [PowerSync Service architecture](/architecture/powersync-service) for background.
Each [PowerSync Service](/architecture/powersync-service) instance has a deployed *Sync Streams* (or legacy *Sync Rules*) configuration. This takes the form of a YAML file which contains:
* **In the case of Sync Streams:** Definitions of the streams that exist, with a SQL-like query (which can also contain limited subqueries), which defines the data in the stream, and references the necessary parameters.
* **In the case of Sync Rules:** Definitions of the different [buckets](/architecture/powersync-service#bucket-system) that exist, with SQL-like queries to specify the parameters used by each bucket (if any), as well as the data contained in each bucket.
A *parameter* is a value that can be used in Sync Streams (or legacy Sync Rules) to create dynamic sync behavior for each user/client. Each client syncs only the relevant [*buckets*](/architecture/powersync-service#bucket-system) based on the parameters for that client.
* Sync Streams can make use of *authentication parameters* from the JWT token (such as the user ID or other JWT claims), *connection parameters* (specified at connection), and *subscription parameters* (specified by the client when it subscribes to a stream at any time). See [Using Parameters](/sync/streams/parameters).
* Sync Rules can make use of *authentication parameters* from the JWT token, as well as [*client parameters*](/sync/rules/client-parameters) (passed directly from the client when it connects to the PowerSync Service).
It is also possible to have buckets/streams with no parameters. In the case of Sync Rules, these buckets sync to all users/clients automatically.
The concept of *buckets* is core to PowerSync and key to its performance and scalability. The [PowerSync Service architecture overview](/architecture/powersync-service) provides more background on this.
* In *Sync Streams*, buckets and parameters are implicit — they are automatically created based on the streams, their queries and subqueries. You don't need to explicitly define the buckets that exist.
* In legacy *Sync Rules*, buckets and their parameters are [explicitly defined](/sync/rules/overview#bucket-definition).
There are limitations on the SQL syntax and functionality that is supported in Sync Streams and Sync Rules. See [Supported SQL](/sync/supported-sql) for details and limitations.
In addition to filtering data based on parameters, Sync Streams and Sync Rules also enable:
* Selecting only specific tables/collections and columns/fields to sync.
* Filtering data based on static conditions.
* Transforming column/field names and values.
### Sync Streams/Rules Determine Replication From the Source Database
A PowerSync Service instance [replicates and transforms](/architecture/powersync-service#replication-from-the-source-database) relevant data from your backend source database according to your Sync Streams/Sync Rules. During replication, data and metadata are persisted in [buckets](/architecture/powersync-service#bucket-system) on the PowerSync Service. Buckets are incrementally updated so that they contain the latest state as well as a history of changes (operations). This is key to how PowerSync achieves efficient delta syncing — having the operation history for each bucket allows clients to sync only the deltas that they need to get up to date (see [Protocol](/architecture/powersync-protocol#protocol) for more details).
As a practical example, let's say you have a bucket named `user_todo_lists` that contains the to-do lists for a user, and that bucket utilizes a `user_id` parameter (which will be embedded in the JWT). Now let's say users with IDs `A` and `B` exist in the source database. PowerSync will then replicate data from the source database and create individual buckets with IDs `user_todo_lists["A"]` and `user_todo_lists["B"]`. When the user with ID `A` connects, they can efficiently sync just the bucket with ID `user_todo_lists["A"]`.
For more information on the client-side SQLite database structure, see [Client Architecture](/architecture/client-architecture#client-side-schema-and-sqlite-database-structure).
Use client parameters with caution. Please make sure to read the [Security consideration](#security-consideration) section below.
Client parameters are parameters that are passed to the PowerSync Service instance from the client SDK, and can be used in Sync Rules' [parameter queries](/sync/rules/parameter-queries) to further filter data.
PowerSync already supports using **token parameters** in parameter queries. An example of a token parameter is a user ID, and this is commonly used to filter synced data by the user. These parameters are embedded in the JWT [authentication token](/configuration/auth/custom), and therefore can be considered trusted and can be used for access control purposes.
**Client parameters** are specified directly by the client (i.e. not through the JWT authentication token). The advantage of client parameters is that they give client-side control over what data to sync, and can therefore be used to further filter or limit synced data. A common use case is [lazy-loading](/client-sdks/infinite-scrolling#2-control-data-sync-using-subscription-or-client-parameters), where data is split into pages and a client parameter can be used to specify which page(s) to sync to a user, and this can update dynamically as the user paginates (or reaches the end of an infinite-scrolling feed).
[Sync Streams](/sync/streams/overview) make it easier to manage dynamic parameters, especially for apps where parameters are managed across different UI components and tabs. Sync Streams offer *subscription parameters* (specified when subscribing to a stream) and *connection parameters* (the equivalent of client parameters).
We recommend Sync Streams for new projects, and [migrating](/sync/streams/migration) existing projects.
### Usage
Client parameters are defined when [instantiating the PowerSync database](/intro/setup-guide#instantiate-the-powersync-database), within the options of PowerSync's `connect()` method:
```js theme={null}
const connector = new DemoConnector();
const powerSync = db;
function connectPowerSync() {
powerSync.connect(connector, {
params: { "current_page": } // Specify client parameters here
});
}
```
The parameter is then available in [Sync Rules](/sync/rules/overview) under `request.parameters` (alongside the already supported `request.user_id`).
In this example, only 'posts' from the user's current page are synced:
```yaml theme={null}
# sync-rules.yaml
bucket_definitions:
shared_posts:
parameters: SELECT (request.parameters() ->> 'current_page') as page_number
data:
- SELECT * FROM posts WHERE page_number = bucket.page_number
```
### Security Consideration
An important consideration with client parameters is that a client can pass any value, and sync data accordingly. Hence, client parameters should always be treated with care, and should not be used for access control purposes. Where permissions are required, use token parameters (`request.jwt()`) instead, or use token parameters in combination with client parameters.
The following examples show **secure** vs. **insecure** ways of using client and token parameters:
#### Secure (using a token parameter only):
```yaml theme={null}
# sync-rules.yaml
bucket_definitions:
selected_projects:
# Sync projects based on org_id from the JWT
# Since these parameters are embedded in the JWT (authentication token)
# they can be considered trusted
parameters: SELECT id as project_id FROM projects WHERE org_id IN request.jwt() ->> 'app_metadata.org_id'
data:
- ...
```
#### Insecure (using a client parameter only):
```yaml theme={null}
# sync-rules.yaml
bucket_definitions:
selected_projects:
# Do NOT do this: Sync projects based on a client parameter
# request.parameters() are specified by the client directly
# Because the client can send any value for these parameters
# it's not a good place to do authorization
parameters: SELECT id as project_id FROM projects WHERE id in request.parameters() ->> 'selected_projects'
data:
- ...
```
#### Secure (using a token parameter combined with a client parameter):
```yaml theme={null}
# sync-rules.yaml
bucket_definitions:
selected_projects:
# Sync projects based on org_id from the JWT, and additionally sync archived projects
# only when specifically requested by the client
# The JWT is a Supabase specific example with a
# custom field set in app_metadata
parameters: SELECT id as project_id FROM projects WHERE org_id IN request.jwt() ->> 'app_metadata.org_id' AND archived = true AND request.parameters() ->> 'include_archived'
data:
- ...
```
### Warning on Potentially Dangerous Queries
Based on the above security consideration, the [PowerSync Dashboard](https://dashboard.powersync.com/) will warn developers when client parameters are being used in Sync Rules in an insecure way (i.e. where the query does not also include a parameter from `request.jwt()`).
The below Sync Rules will display the warning:
> Potentially dangerous query based on parameters set by the client. The client can send any value for these parameters so it's not a good place to do authorization.
```yaml theme={null}
# sync-rules.yaml
bucket_definitions:
selected_projects:
parameters: SELECT request.parameters() ->> 'project_id' as project_id
data:
- ...
```
This warning can be disabled by specifying `accept_potentially_dangerous_queries: true` in the bucket definition:
```yaml theme={null}
# sync-rules.yaml
bucket_definitions:
selected_projects:
accept_potentially_dangerous_queries: true
parameters: SELECT request.parameters() ->> 'project_id' as project_id
data:
- ...
```
# Data Queries
Source: https://docs.powersync.com/sync/rules/data-queries
Write Data Queries in Sync Rules to select and filter data for buckets using bucket parameters.
Data Queries select the data that form part of a [bucket](/architecture/powersync-service#bucket-system), using the bucket [parameters](/sync/rules/overview#parameters).
Multiple Data Queries can be specified for a single [bucket definition](/sync/rules/overview#bucket-definition).
## Every Data Query Must Use Every Bucket Parameter
Data Queries are used to group data into buckets, so each Data Query must use every bucket [parameter](/sync/rules/overview#parameters).
When PowerSync does [incremental replication](/architecture/powersync-service#initial-replication-vs-incremental-replication) of data from your source database, it evaluates every row/document received on the CDC stream , and computes a list of [buckets](/architecture/powersync-service#bucket-system) that row/document belongs to. This allows PowerSync to efficiently update only the specific buckets that are affected by each change event received. PowerSync uses the Data Queries in the Sync Rules bucket definitions to determine which rows/documents belong to which buckets. Therefore, if it was possible for a certain bucket parameter to *not* be used in the `WHERE` clause of a Data Query, the bucket IDs to which the row/document belongs would be ambiguous — we would have to assume "all possible values" for an ambiguous parameter value in the bucket ID – and the row/document would have to be exploded into many buckets. To avoid this, PowerSync imposes the constraint that every Data Query needs to use every parameter defined on the bucket.
## Supported SQL
The supported SQL in Data Queries is based on a small subset of the SQL standard syntax. Not all SQL constructs are supported. See [Supported SQL](/sync/supported-sql) for full details.
## Examples
#### Grouping by Parameter Query Values
```yaml theme={null}
bucket_definitions:
owned_lists:
parameters: |
SELECT id as list_id FROM lists WHERE
owner_id = request.user_id()
data:
- SELECT * FROM lists WHERE lists.id = bucket.list_id
- SELECT * FROM todos WHERE todos.list_id = bucket.list_id
```
#### Selecting Output Columns/Fields
When specific columns/fields are selected, only those columns/fields are synced to the client.
This is good practice, to ensure the synced data does not unintentionally change when new columns are added to the schema (in the case of Postgres) or to the data structure (in the case of MongoDB).
Note: An `id` column must always be present, and must have a `text` type. If the primary key is different, use a column alias and/or transformations to output a `text` id column.
```yaml theme={null}
bucket_definitions:
global:
data:
- SELECT id, name, owner_id FROM lists
```
MongoDB uses `_id` as the name of the ID field in collections. Therefore, PowerSync requires using `SELECT _id as id` in the data queries when [using MongoDB](/configuration/source-db/setup) as the backend source database.
#### Renaming Columns/Fields
Different names (aliases) may be specified for columns/fields:
```yaml theme={null}
bucket_definitions:
global:
data:
- SELECT id, name, created_timestamp AS created_at FROM lists
```
#### Transforming Columns/Fields
A limited set of operators and functions are available to transform the output value of columns/fields.
```yaml theme={null}
bucket_definitions:
global:
data:
# Cast number to text
- SELECT id, item_number :: text AS item_number FROM todos
# Alternative syntax for the same cast
- SELECT id, CAST(item_number as TEXT) AS item_number FROM todos
# Convert binary data (bytea) to base64
- SELECT id, base64(thumbnail) AS thumbnail_base64 FROM todos
# Extract field from JSON or JSONB column
- SELECT id, metadata_json ->> 'description' AS description FROM todos
# Convert time to epoch number
- SELECT id, unixepoch(created_at) AS created_at FROM todos
```
# Global Buckets
Source: https://docs.powersync.com/sync/rules/global-buckets
Set up global buckets in Sync Rules to sync shared reference data to all connected users without per-user filtering or parameter queries.
Any bucket with no *Parameter Query* in the bucket definition is automatically a *Global Bucket*. These buckets will be synced to all clients/users.
For example, the following Sync Rules sync all `todos` and only unarchived `lists` to all clients/users:
```yaml theme={null}
bucket_definitions:
global_bucket:
data:
# Sync all todos
- SELECT * FROM todos
# Sync all lists except archived ones
- SELECT * FROM lists WHERE archived = false
```
`global_bucket` is *not* a reserved keyword. You can give the bucket any name. If no **Parameter Query** is specified in the bucket definition, the bucket is automatically a global bucket.
**Note**: The table/collection names that your Data Queries select from in your Sync Rules must match the table names defined in your [client-side schema](/intro/setup-guide#define-your-client-side-schema).
As explained in the [Overview & Key Concepts](/sync/rules/overview#potential-parameter-values-determine-created-buckets), PowerSync uses the possible values for parameters (found in your source database) to generate individual buckets that can be efficiently synced by clients/users depending on the specific parameters that apply to them. By contrast, with a Global Bucket, only a single bucket will be generated on the PowerSync Service that is shared between all your users/clients.
# Many-to-Many Relationships and Join Tables in Sync Rules
Source: https://docs.powersync.com/sync/rules/many-to-many-join-tables
Handle many-to-many relationships in Sync Rules using join table strategies.
Join tables are often used to implement many-to-many relationships between tables. Join queries are not directly supported in PowerSync Sync Rules, and require some workarounds depending on the use case. This guide contains some recommended strategies.
**Using Sync Streams?** Sync Streams support [JOINs](/sync/streams/queries#using-joins) and [nested subqueries](/sync/streams/queries#using-subqueries), which handle most many-to-many relationships directly without the workarounds described here. See [Many-to-Many with Sync Streams](/sync/streams/examples#many-to-many-relationships) for examples.
**Postgres users:** For Postgres source databases, you can use the [`pg_ivm` extension](https://www.powersync.com/blog/using-pg-ivm-to-enable-joins-in-powersync) to create incrementally maintained materialized views with JOINs that can be referenced directly in Sync Rules. This approach avoids the need to denormalize your schema.
## Example
As an example, consider a social media application. The app has message boards. Each user can subscribe to boards, make posts, and comment on posts. Posts may also have one or more topics.
```sql theme={null}
create table users (
id uuid not null default gen_random_uuid (),
name text not null,
last_activity timestamp with time zone,
constraint users_pkey primary key (id)
);
create table boards (
id uuid not null default gen_random_uuid (),
name text not null,
constraint boards_pkey primary key (id)
);
create table posts (
id uuid not null default gen_random_uuid (),
board_id uuid not null,
created_at timestamp with time zone not null default now(),
author_id uuid not null,
title text not null,
body text not null,
constraint posts_pkey primary key (id),
constraint posts_author_id_fkey foreign key (author_id) references users (id),
constraint posts_board_id_fkey foreign key (board_id) references boards (id)
);
create table comments (
id uuid not null default gen_random_uuid (),
post_id uuid not null,
created_at timestamp with time zone not null default now(),
author_id uuid not null,
body text not null,
constraint comments_pkey primary key (id),
constraint comments_author_id_fkey foreign key (author_id) references users (id),
constraint comments_post_id_fkey foreign key (post_id) references posts (id)
);
create table board_subscriptions (
id uuid not null default gen_random_uuid (),
user_id uuid not null,
board_id uuid not null,
constraint board_subscriptions_pkey primary key (id),
constraint board_subscriptions_board_id_fkey foreign key (board_id) references boards (id),
constraint board_subscriptions_user_id_fkey foreign key (user_id) references users (id)
);
create table topics (
id uuid not null default gen_random_uuid (),
label text not null,
constraint topics_pkey primary key (id)
);
create table post_topics (
id uuid not null default gen_random_uuid (),
board_id uuid not null,
post_id uuid not null,
topic_id uuid not null,
constraint post_topics_pkey primary key (id),
constraint post_topics_board_id_fkey foreign key (board_id) references boards (id),
constraint post_topics_post_id_fkey foreign key (post_id) references posts (id),
constraint post_topics_topic_id_fkey foreign key (topic_id) references topics (id)
);
```
### Many-to-many: Bucket parameters
For this app, we generally want to sync all posts in boards that users have subscribed to. To simplify these examples, we assume a user has to be subscribed to a board to post.
Boards make a nice grouping of data for Sync Rules: We sync the boards that a user has subscribed to, and the same board data is synced to all users subscribed to that board.
The relationship between users and boards is a many-to-many, specified via the `board_subscriptions` table.
To start with, in our PowerSync Sync Rules, we define a [bucket](/sync/rules/organize-data-into-buckets) and sync the posts. The [parameter query](/sync/rules/parameter-queries) is defined using the `board_subscriptions` table:
```yaml theme={null}
bucket_definitions:
board_data:
parameters: SELECT board_id FROM board_subscriptions WHERE user_id = request.user_id()
data:
- SELECT * FROM posts WHERE board_id = bucket.board_id
```
### Avoiding Joins in Data Queries: Denormalize Relationships (Comments)
Next, we also want to sync comments for those boards. There is a one-to-many relationship between boards and comments, via the `posts` table. This means conceptually we can add comments to the same board bucket. With general SQL, the query could be:
```sql theme={null}
SELECT comments.* FROM comments
JOIN posts ON posts.id = comments.post_id
WHERE board_id = bucket.board_id
```
Unfortunately, joins are not supported in PowerSync's Sync Rules. Instead, we denormalize the data to add a direct foreign key relationship between comments and boards: (Postgres example)
```sql theme={null}
ALTER TABLE comments ADD COLUMN board_id uuid;
ALTER TABLE comments ADD CONSTRAINT comments_board_id_fkey FOREIGN KEY (board_id) REFERENCES boards (id);
```
Now we can add it to the bucket definition in our Sync Rules:
```yaml theme={null}
bucket_definitions:
board_data:
parameters: SELECT board_id FROM board_subscriptions WHERE user_id = request.user_id()
data:
- SELECT * FROM posts WHERE board_id = bucket.board_id
# Add comments:
- SELECT * FROM comments WHERE board_id = bucket.board_id
```
Now we want to sync topics of posts. In this case we added `board_id` from the start, so `post_topics` is simple in our Sync Rules:
```yaml theme={null}
bucket_definitions:
board_data:
parameters: SELECT board_id FROM board_subscriptions WHERE user_id = request.user_id()
data:
- SELECT * FROM posts WHERE board_id = bucket.board_id
- SELECT * FROM comments WHERE board_id = bucket.board_id
# Add post_topics:
- SELECT * FROM post_topics WHERE board_id = bucket.board_id
```
### Many-to-many strategy: Sync everything (topics)
Now we need access to sync the topics for all posts synced to the device. There is a many-to-many relationship between posts and topics, and by extension boards to topics. This means there is no simple direct way to partition topics into buckets — the same topics be used on any number of boards.
If the topics table is limited in size (say 1,000 or less), the simplest solution is to just sync all topics in our Sync Rules:
```yaml theme={null}
bucket_definitions:
global_topics:
data:
- SELECT * FROM topics
```
### Many-to-many strategy: Denormalize data (topics, user names)
If there are many thousands of topics, we may want to avoid syncing everything. One option is to denormalize the data by copying the topic label over to `post_topics`: (Postgres example)
```sql theme={null}
ALTER TABLE post_topics ADD COLUMN topic_label text not null;
```
Now we don't need to sync the `topics` table itself, as everything is included in `post_topics`. Assuming the topic label never or rarely changes, this could be a good solution.
Next up, we want to sync the relevant user profiles, so we can show it together with comments and posts. For simplicity, we sync profiles for all users subscribed to a board.
One option is to add the author name to each board subscription, similar to what we've done for `topics`: (Postgres example)
```sql theme={null}
ALTER TABLE board_subscriptions ADD COLUMN user_name text;
```
Sync Rules:
```yaml theme={null}
bucket_definitions:
board_data:
parameters: SELECT board_id FROM board_subscriptions WHERE user_id = request.user_id()
data:
- SELECT * FROM posts WHERE board_id = bucket.board_id
- SELECT * FROM comments WHERE board_id = bucket.board_id
- SELECT * FROM post_topics WHERE board_id = bucket.board_id
# Add subscriptions which include the names:
- SELECT * FROM board_subscriptions WHERE board_id = bucket.board_id
```
### Many-to-many strategy: Array of IDs (user profiles)
If we need to sync more than just the name (let's say we need a last activity date, profile picture and bio text as well), the above approach doesn't scale as well. Instead, we want to sync the `users` table directly. To sync user profiles directly in the bucket for the board, we need a new array.
Adding an array to the schema in Postgres:
```sql theme={null}
ALTER TABLE users ADD COLUMN subscribed_board_ids uuid[];
```
By using an array instead of or in addition to a join table, we can use it directly in Sync Rules:
```yaml theme={null}
bucket_definitions:
board_data:
parameters: SELECT board_id FROM board_subscriptions WHERE user_id = request.user_id()
data:
- SELECT * FROM posts WHERE board_id = bucket.board_id
- SELECT * FROM comments WHERE board_id = bucket.board_id
- SELECT * FROM post_topics WHERE board_id = bucket.board_id
# Add participating users:
- SELECT name, last_activity, profile_picture, bio FROM users WHERE bucket.board_id IN subscribed_board_ids
```
This approach does require some extra effort to keep the array up to date. One option is to use a trigger in the case of Postgres:
```sql theme={null}
CREATE OR REPLACE FUNCTION recalculate_subscribed_boards()
RETURNS TRIGGER AS $$
BEGIN
-- Recalculate subscribed_board_ids for the affected user
UPDATE users
SET subscribed_board_ids = (
SELECT array_agg(board_id)
FROM board_subscriptions
WHERE user_id = COALESCE(NEW.user_id, OLD.user_id)
)
WHERE id = COALESCE(NEW.user_id, OLD.user_id);
RETURN NULL;
END;
$$ LANGUAGE plpgsql;
CREATE TRIGGER trg_board_subscriptions_change
AFTER INSERT OR UPDATE OR DELETE ON board_subscriptions
FOR EACH ROW
EXECUTE FUNCTION recalculate_subscribed_boards();
```
Note that this approach does have scaling limitations. When the number of board subscriptions per user becomes large (say over 100 rows per user), then:
1. Updating the `subscribed_board_ids` array in Postgres becomes slower.
2. The overhead is even more pronounced on PowerSync, since PowerSync maintains a separate copy of the data in each bucket.
In those cases, another approach may be more suitable.
# Organize Data Into Buckets
Source: https://docs.powersync.com/sync/rules/organize-data-into-buckets
Design Sync Rules to organize data into global and user-filtered buckets.
Designing your Sync Rules is about *organizing data into buckets*, and creating the bucket definitions accordingly. Each [bucket definition](/sync/rules/overview#bucket-definition) defines a set of tables/collections and rows/documents to sync.
* If there's some data you want to sync to *all* your users/clients, you can add bucket definitions for one or more [Global Buckets](/sync/rules/global-buckets). This is the simplest way to get started with PowerSync.
* If there's data that you want to filter by user, so that different users/clients get different subsets of data, you can add bucket definitions to your Sync Rules specifically for that purpose, using the `user_id` *Authentication Parameter* that comes from the JWT.
* You can also filter data based on other parameters, such as project, organization, etc. — using additional bucket definitions with those parameters.
When designing your buckets, it is recommended, but not required, to group all data in a bucket where the same parameters apply.
## Defining Buckets
Sync Rules take the form of a YAML file containing all your [bucket definitions](/sync/rules/overview#bucket-definition).
A *bucket definition* contains two sets of queries:
1. [Parameter Queries](/sync/rules/parameter-queries): Select bucket parameters
2. [Data Queries](/sync/rules/data-queries): Select data in the bucket using the bucket parameters
Here is an example of Sync Rules containing a single bucket definition, which will sync only the `lists` that belong to the user:
```yaml theme={null}
bucket_definitions:
user_lists:
# select parameters for the bucket - in this case we are just selecting the user_id
parameters: SELECT request.user_id() as user_id # (request.user_id() comes from the JWT token)
data:
# select data rows/documents using the parameters above
- SELECT * FROM lists WHERE owner_id = bucket.user_id
```
You can choose any name for a bucket. The above Sync Rules contains only one bucket definition, with a bucket name of `user_lists`.
**Note**: The table/collection names that your Data Queries select from in your Sync Rules must match the table names defined in your [client-side schema](/intro/setup-guide#define-your-client-side-schema).
The supported SQL in *Parameter Queries* and *Data Queries* is based on a small subset of the SQL standard syntax. Not all SQL constructs are supported. See [Supported SQL](/sync/supported-sql).
## Limit on Number of Buckets Per Client
There is a maximum cap on the number of buckets that each user/client can sync — [the default limit](/resources/performance-and-limits) is 1,000.
What this practically means is that the total number of results returned from all the **Parameter Queries** across all your bucket definitions *for each specific user/client* cannot exceed that limit. If the Parameter Queries for a specific user/client exceed that limit, you will get a [`PSYNC_S2305` error](/debugging/error-codes#psync_s23xx-sync-api-errors) from the PowerSync Service.
Note that this limit only applies to *each individual user/client*. You can have many more buckets in total in your project, as long as each user syncs no more total buckets than the limit. For example, your PowerSync Service instance could track let's say 1,000,000 buckets in total, but each user syncs a small fraction of those buckets.
This limit can be increased upon request for [Team and Enterprise](https://www.powersync.com/pricing) customers, however: as the number of buckets exceeds 1,000, performance degrades. See [Performance and Limits](/resources/performance-and-limits)
# Sync Rules (Legacy)
Source: https://docs.powersync.com/sync/rules/overview
Understand legacy Sync Rules for controlling which data syncs to each client.
Sync Rules are PowerSync's original system for partial sync , using YAML bucket definitions. They remain supported for existing projects but are considered legacy.
**Sync Streams Recommended**
[Sync Streams](/sync/streams/overview) are the recommended approach to partial sync for both new and existing projects. They support everything Sync Rules do, plus more expressive queries (including JOIN support), on-demand syncing, and a simpler developer experience (e.g. React hooks that manage subscriptions automatically).
You can migrate in a few clicks. Click **Migrate to Sync Streams** in the PowerSync Dashboard, or run `powersync migrate sync-rules` in the CLI to generate a draft from your current config.
Sync Rules are defined in a YAML file. For PowerSync Cloud, they are edited and deployed to a specific PowerSync instance in the [PowerSync Dashboard](/tools/powersync-dashboard#project-&-instance-level). For self-hosting setups, they are defined as part of your [instance configuration](/configuration/powersync-service/self-hosted-instances).
## Key Concepts
### Bucket Definition
*Sync Rules* are a set of *bucket definitions*. A *bucket definition* defines the actual individual [buckets](/architecture/powersync-service#bucket-system) that will be created by the PowerSync Service when it replicates data from your source database. Clients then sync the individual buckets that are relevant to them based on their parameters (see below).
Each *bucket definition* consists of:
* A custom **name** for the bucket, e.g `user_lists`
* Zero or more [**Parameter Queries**](#parameter-queries) which are used to explicitly select the **parameters** for the bucket. If no Parameter Query is specified in the bucket definition, it's automatically a [global bucket](#global-buckets).
* One or more [**Data Queries**](#data-queries) which selects the data for the bucket. Data Queries can make use of the **parameters** that are selected in the Parameter Query.
```yaml Sync Rules with a single bucket definition theme={null}
bucket_definitions:
user_lists: # name for the bucket
# Parameter Query, selecting a user_id parameter:
parameters: SELECT request.user_id() as user_id
data: # Data Query, selecting data, filtering using the user_id parameter:
- SELECT * FROM lists WHERE owner_id = bucket.user_id
```
### Parameters
A **Parameter** is a value that can be used in the Sync Rules to create dynamic sync behavior for each user/client. Each client syncs [only the relevant buckets](/architecture/powersync-service#bucket-system) based on the parameters for that client (i.e. the results that would be returned from the **Parameter Query** for that client).
### Parameter Queries
**Parameter Queries** are SQL-like queries that explicitly define the parameters for a bucket.
The following values can be selected in Parameter Queries:
* **Authentication Parameters** (see below)
* **Client Parameters** (see below)
* **Values From a Table/Collection** (see below)
See [Parameter Queries](/sync/rules/parameter-queries) for more details and examples. Also see [Supported SQL](/sync/supported-sql) for limitations.
### Authentication Parameters
This is a set of parameters specified in the user's JWT authentication token, and they are signed as part of the JWT (see [Authentication](/configuration/auth/overview)). This always includes the JWT subject (`sub`) which is the user ID, but may include additional and custom parameters (other claims in the JWT). *Authentication Parameters* are used to identify the user, and specify permissions for the user. They need to be explicitly selected in the Parameter Query to be used in Sync Rules:
```yaml Example of selecting Authentication Parameter in a Parameter Query theme={null}
parameters: SELECT request.user_id() as user_id
```
See [Parameter Queries](/sync/rules/parameter-queries) for more details and examples.
### Client Parameters
Clients can specify **Client Parameters** when connecting to PowerSync (i.e. when [`connect()` is called](/intro/setup-guide#connect-to-powersync-service-instance)). *Client Parameters* need to be explicitly selected in the Parameter Query to be used in Sync Rules:
```yaml Example of selecting a Client Parameter in a Parameter Query theme={null}
parameters: SELECT (request.parameters() ->> 'current_project') as current_project
```
The `->>` operator in the above example extracts a value from a string containing JSON (which is the format provided by `request.parameters()`). See [Operators and Functions](/sync/supported-sql#operators)
A client can pass any value for a Client Parameter. Hence, Client Parameters should always be treated with care, and should [not be used](/sync/rules/client-parameters#security-consideration) for access control purposes.
That being said, Client Parameters can be useful for use cases such as syncing different buckets based on state in the client app, for example only syncing data for the project currently selected, or syncing different buckets based on the client version ([see here](/sync/advanced/multiple-client-versions)).
See [Client Parameters](/sync/rules/client-parameters) and [Parameter Queries](/sync/rules/parameter-queries) for more details and examples.
### Values From a Table/Collection
A **Parameter Query** can select a parameter from a table/collection in your source database, e.g.:
```yaml Selecting a parameter from a table in the source database theme={null}
SELECT primary_list_id FROM users WHERE users.id = request.user_id()
```
See [Parameter Queries](/sync/rules/parameter-queries) for more details and examples.
### Data Queries
**Data Queries** select the data for the bucket, and can make use of any of the bucket **parameters** (i.e. values returned by the **Parameter Queries**).
When referencing a parameter, the syntax `bucket.` should be used in the Data Query:
```yaml theme={null}
data:
- SELECT * FROM lists WHERE owner_id = bucket.user_id
```
See [Data Queries](/sync/rules/data-queries) for more details and examples. Also see [Supported SQL](/sync/supported-sql) for limitations.
### Global Buckets
If no **Parameter Query** is specified in the bucket definition, the bucket is automatically a global bucket. These buckets will be synced to all clients/users.
See [Global Buckets](/sync/rules/global-buckets) for more details and examples.
## Potential Parameter Values Determine Created Buckets
When your PowerSync Service instance [replicates data from your source database](/architecture/powersync-service#replication-from-the-source-database) based on your Sync Rules (i.e. your bucket definitions), it finds all possible values for your defined parameters in the relevant tables/collections in your source database, and creates individual buckets based on those values.
For example, let's say you have a bucket named `user_todo_lists` that contains the to-do lists for a user, and that bucket utilizes a `user_id` parameter (which will be obtained from the JWT) to scope those to-do lists. Now let's say users with IDs `1`, `2` and `3` exist in the source database. PowerSync will then replicate data from the source database and preemptively create individual buckets with bucket IDs of `user_todo_lists["1"]`, `user_todo_lists["2"]` and `user_todo_lists["3"]`.
This architecture is key to the scalability and performance of PowerSync. See the [PowerSync Service architecture overview](/architecture/powersync-service) for more background.
## Designing Sync Rules
Designing your Sync Rules is basically about *organizing data into buckets*, and creating the bucket definitions accordingly.
See [Organize Data Into Buckets](/sync/rules/organize-data-into-buckets).
# Parameter Queries
Source: https://docs.powersync.com/sync/rules/parameter-queries
Define bucket parameters in Sync Rules using Parameter Queries with JWT claims and client parameters.
*Parameter Queries* allow [parameters](/sync/rules/overview#parameters) to be defined on a [bucket](/sync/rules/overview#bucket-definition) to group data.
Each [bucket](/sync/rules/overview#bucket-definition) can have zero or more Parameter Queries.
Parameter Queries can return multiple rows/documents. The values selected in each row/document become parameters for the bucket.
The following values can be selected in Parameter Queries:
* **Authentication Parameters**, which come from the JWT token.
* **Client Parameters**, which are passed directly from clients (specified [at connection](/sync/rules/client-parameters#usage))
* **Values From a Table/Collection** (in your source database)
Parameter Queries are not run directly on your source database. Instead, the Parameter Queries in your Sync Rules are used to pre-process rows/documents as they are [replicated from your source database](/architecture/powersync-service#replication-from-the-source-database). During replication, parameter values are indexed for [efficient use](/architecture/powersync-service#bucket-system) in the sync process.
## Using Authentication Parameters
The following functions allow you to select Authentication Parameters in your Parameter Queries:
| Function | Description |
| ------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `request.user_id()` | Returns the JWT subject (`sub`). Same as `request.jwt() ->> 'sub'` (see below) |
| `request.jwt()` | Returns the entire (signed) JWT payload as a JSON string. If there are other *claims* in your JWT (in addition to the user ID), you can select them from this JSON string. |
Since `request.jwt()` is a string containing JSON, use the `->>` [operator](/sync/supported-sql#operators) to select values from it:
```sql theme={null}
request.jwt() ->> 'sub' -- the 'subject' of the JWT - same as `request.user_id()
```
As an example, Supabase Auth includes [various claims](https://supabase.com/docs/guides/auth/jwt-fields) in their JWTs:
```sql theme={null}
request.jwt() ->> 'role' -- 'authenticated' or 'anonymous'
request.jwt() ->> 'email' -- automatic email field
request.jwt() ->> 'app_metadata.custom_field' -- custom field added by a service account (authenticated)
```
This is a simple example of Sync Rules with a single bucket definition with a Parameter Query that selects the user ID from the JWT:
```yaml theme={null}
bucket_definitions:
# Bucket Name
user_lists:
# Parameter Query
parameters: SELECT request.user_id() as user_id
# Data Query
data:
- SELECT * FROM lists WHERE lists.owner_id = bucket.user_id
```
A legacy syntax for Parameter Queries used `token_parameters.user_id` to return the JWT subject. Example:
```yaml theme={null}
bucket_definitions:
by_user_parameter:
parameters: SELECT token_parameters.user_id as user_id
data:
- SELECT * FROM lists WHERE lists.owner_id = bucket.user_id
```
That legacy syntax also allowed custom claims from the JWT, but only if they were nested under a `parameters` claim in the JWT.
If you are still using this legacy syntax, you can migrate to the current syntax as follows:
1. `token_parameters.user_id` references can simply be updated to `request.user_id()`
2. For custom parameters, if you keep your custom JWT in the format required by the legacy syntax, you can update `token_parameters.my_custom_field` references to `request.jwt() ->> 'parameters.my_custom_field'`
3. Alternatively, you can get custom parameters directly from the JWT payload/claims, e.g. `request.jwt() ->> 'my_custom_field'`
Example:
```yaml theme={null}
bucket_definitions:
by_user_parameter:
# request.user_id() is the same as the previous token_parameter.user_id
parameters: SELECT request.user_id() as user_id
data:
- SELECT * FROM lists WHERE lists.owner_id = bucket.user_id
```
## Using Client Parameters
| Function | Description |
| ---------------------- | ---------------------------------------------------------------------------- |
| `request.parameters()` | Returns [Client Parameters](/sync/rules/client-parameters) as a JSON string. |
Example usage:
```sql theme={null}
request.parameters() ->> 'param' -- select Client Parameter named 'param'
```
For full details, see the dedicated page on [Client Parameters](/sync/rules/client-parameters).
## Using Values From a Table/Collection
A Parameter Query can select a parameter from a table/collection in your source database, for example:
```yaml theme={null}
bucket_definitions:
user_lists_table:
# This is similar to the 'user_lists' example above, but with the advantage that access
# can instantly be revoked by deleting the user row/document from the source database:
parameters: SELECT id as user_id FROM users WHERE users.id = request.user_id()
data:
- SELECT * FROM lists WHERE lists.user_id = bucket.user_id
```
## Supported SQL
The supported SQL in Parameter Queries is based on a small subset of the SQL standard syntax. Not all SQL constructs are supported. See [Supported SQL](/sync/supported-sql) for full details.
## Usage Examples
### Filter on Additional Columns/Fields
```yaml theme={null}
bucket_definitions:
admin_users:
parameters: |
SELECT id as user_id FROM users WHERE
users.id = request.user_id() AND
users.is_admin = true
data:
- SELECT * FROM lists WHERE lists.owner_id = bucket.user_id
```
### Group According to Different Columns/Fields
```yaml theme={null}
bucket_definitions:
primary_list:
parameters: |
SELECT primary_list_id FROM users WHERE users.id = request.user_id()
data:
- SELECT * FROM todos WHERE todos.list_id = bucket.primary_list_id
```
### Using Different Tables/Collections for Parameters
```yaml theme={null}
bucket_definitions:
owned_lists:
parameters: |
SELECT id as list_id FROM lists WHERE owner_id = request.user_id()
data:
- SELECT * FROM lists WHERE lists.id = bucket.list_id
- SELECT * FROM todos WHERE todos.list_id = bucket.list_id
```
### Multiple Columns/Fields
Parameter Queries may select multiple columns/fields parameters.
Note that [every bucket parameter must be used in every Data Query](/sync/rules/data-queries#every-data-query-must-use-every-bucket-parameter).
```yaml theme={null}
bucket_definitions:
owned_org_lists:
parameters: |
SELECT id as list_id, org_id FROM lists WHERE
owner_id = request.user_id()
data:
- SELECT * FROM lists WHERE lists.id = bucket.list_id and lists.org_id = bucket.org_id
- SELECT * FROM todos WHERE todos.list_id = bucket.list_id and todos.org_id = bucket.org_id
```
### Using a Join Table/Collection
In this example, the Parameter Query can return multiple rows/documents, resulting in multiple sets of bucket parameters for a single user.
```yaml theme={null}
bucket_definitions:
user_lists:
parameters: |
SELECT list_id FROM user_lists WHERE user_lists.user_id = request.user_id()
data:
- SELECT * FROM lists WHERE lists.id = bucket.list_id
- SELECT * FROM todos WHERE todos.list_id = bucket.list_id
```
Keep in mind that the total number of buckets per user should [remain limited](/sync/rules/organize-data-into-buckets#limit-on-number-of-buckets-per-client) (\<= 1,000 [by default](/resources/performance-and-limits)), so buckets should not be too granular.
For more advanced details on many-to-many relationships and join tables, see [this guide](/sync/rules/many-to-many-join-tables).
### Expanding JSON Array Into Multiple Parameters
Using the `json_each()` [function](/sync/supported-sql#functions) and `->` [operator](/sync/supported-sql#operators), we can expand a parameter that is a JSON array into multiple rows, thereby filtering by multiple parameter values:
```yaml theme={null}
bucket_definitions:
user_projects:
parameters: SELECT project_id FROM json_each(request.jwt() -> 'project_ids')
data:
- SELECT * FROM projects WHERE id = bucket.project_id
```
### Multiple Parameter Queries
Multiple Parameter Queries can be used in the same bucket definition, however, the output columns must be exactly the same for each of these Parameter Queries:
```yaml theme={null}
bucket_definitions:
user_lists:
parameters:
- SELECT id as list_id FROM lists WHERE owner_id = request.user_id()
- SELECT list_id FROM user_lists WHERE user_lists.user_id = request.user_id()
data:
- SELECT * FROM lists WHERE lists.id = bucket.list_id
- SELECT * FROM todos WHERE todos.list_id = bucket.list_id
```
Keep in mind that the total number of buckets per user should [remain limited](/sync/rules/organize-data-into-buckets#limit-on-number-of-buckets-per-client) (\<= 1,000 [by default](/resources/performance-and-limits)), so buckets should not be too granular.
### No Output Columns/Fields
A Parameter Query with no output columns may be specified to only sync the bucket to a subset of users.
```yaml theme={null}
bucket_definitions:
global_admins:
parameters: |
SELECT FROM users WHERE
users.id = request.user_id() AND
users.is_admin = true
data:
- SELECT * FROM admin_settings
```
### No Parameter Query
Any bucket with no *Parameter Query* in the bucket definition is automatically a *Global Bucket*. These buckets will be synced to all clients/users.
See [Global Buckets](/sync/rules/global-buckets)
# Bucket Count and Limits
Source: https://docs.powersync.com/sync/streams/bucket-count
How your Sync Stream queries determine each user's bucket count, and the two per-user limits that cap it.
PowerSync groups the rows a client syncs into internal partitions called buckets. Every user has a limit on how many buckets they can sync. This limit is generous, but some query patterns can reach it quickly. Common examples include designs where users belong to many organizations, sync deep hierarchies at once, use many-to-many relationships, or give each user a very large dataset.
This page explains how buckets are counted so you can keep your Sync Streams within the limits.
Already syncing too many buckets, or hit a `PSYNC_S2305` error? See [Reducing Bucket Count](/sync/advanced/reducing-bucket-count) to diagnose and fix it.
## What Is a Bucket?
A bucket is a group of rows that PowerSync syncs together as one unit. When you define a Sync Stream, PowerSync splits the matching rows into buckets so that each client can sync the exact subset of data it needs. You do not create or name buckets yourself. PowerSync creates them for you based on your stream queries.
For background on why buckets exist and how they make sync efficient, see [Bucket System](/architecture/powersync-service#bucket-system). This page focuses on understanding how many buckets your configuration creates and how to keep that number under control.
## The Example App
Every example on this page uses the same app. It has organizations, projects, tasks, assets, and documents.
```mermaid theme={null}
erDiagram
users ||--o{ org_membership : has
orgs ||--o{ org_membership : has
orgs ||--o{ projects : has
projects ||--o{ tasks : has
projects ||--o{ project_assets : links
assets ||--o{ project_assets : links
orgs ||--o{ documents : has
```
Here is how the tables relate:
* A user belongs to one or more organizations. The `org_membership` table links a `user_id` to an `org_id`.
* Each project belongs to one organization through `org_id`. Each project also has a plain text `region` column such as `us-east`.
* Each task belongs to one project through `project_id`.
* Projects and assets link through the `project_assets` table.
* Each document has an `owner_id`, a `shared_with` user, and an `org_id`.
* The `categories` table is shared reference data. It has no owner and no links to the other tables, so it does not appear in the diagram above.
## How Sync Streams Create Buckets
### The Bucket Rule
The rule is simple. A stream creates **one bucket for each unique value of its filter**.
Take this stream:
```yaml theme={null}
streams:
my_documents:
query: SELECT * FROM documents WHERE owner_id = auth.user_id()
```
The filter is `owner_id = auth.user_id()`. Each user has one user ID, so each user gets one bucket. The value that separates one bucket from the next is called the bucket's *parameter*.
The filter value can come from several places. Each unique value still becomes its own bucket:
```mermaid theme={null}
flowchart LR
Q["Stream filter:
The exact bucket name format can change between releases. Use it to read logs. Do not depend on it in your application code.
## Counting Buckets by Query Pattern
Each stream below creates a set number of buckets per user. Work through them one at a time. The pattern you use decides the count.
### No Parameters: One Global Bucket
```yaml theme={null}
streams:
categories:
query: SELECT * FROM categories
```
This stream has no filter. It creates one bucket. Every user syncs that same bucket. Global data uses one bucket, however many users you have.
**Count: 1 bucket, shared by all users.**
### A Direct Auth Filter: One Bucket Per User
```yaml theme={null}
streams:
my_memberships:
query: SELECT * FROM org_membership WHERE user_id = auth.user_id()
```
The filter is one value: the user's ID. Each user has one ID, so each user gets one bucket.
**Count: 1 bucket per user.**
### A JWT Array: One Bucket Per Value
```yaml theme={null}
streams:
my_orgs:
query: SELECT * FROM orgs WHERE id IN auth.parameter('org_ids')
```
The user's JWT holds a list of org IDs, such as `["org-a", "org-b", "org-c"]`. PowerSync makes one bucket for each value in the list.
**Count: one bucket per org ID in the token.**
### A Subscription Parameter: One Bucket Per Subscription
```yaml theme={null}
streams:
project_tasks:
query: SELECT * FROM tasks WHERE project_id = subscription.parameter('project_id')
```
The client picks a project and subscribes with its ID. PowerSync makes one bucket for each project the client subscribes to. If the client opens three projects, it uses three buckets.
**Count: one bucket per project the client subscribes to.**
### A Subquery: One Bucket Per Result Row
```yaml theme={null}
streams:
org_projects:
with:
user_orgs: SELECT org_id FROM org_membership WHERE user_id = auth.user_id()
query: SELECT * FROM projects WHERE org_id IN user_orgs
```
The `user_orgs` filter returns the user's org IDs. The query keys on `org_id`, which is a column on `projects`. PowerSync makes one bucket for each org the user belongs to. All projects in the same org share one bucket.
**Count: one bucket per org the user belongs to.**
### A JOIN Through Another Table: One Bucket Per Joined Row
```yaml theme={null}
streams:
org_tasks:
with:
user_orgs: SELECT org_id FROM org_membership WHERE user_id = auth.user_id()
query: |
SELECT tasks.* FROM tasks
JOIN projects ON tasks.project_id = projects.id
WHERE projects.org_id IN user_orgs
```
This stream syncs tasks, but tasks have no `org_id`. They only have `project_id`. A bucket's key must be a column on the table you sync (see [The Partition Key Must Exist on the Row](/sync/advanced/reducing-bucket-count#the-partition-key-must-exist-on-the-row)). So PowerSync keys these buckets on `project_id`, not `org_id`. It makes one bucket for each project.
Compare this to the subquery above. That stream syncs `projects`, which has `org_id`, so it keys on org. This stream syncs `tasks`, which only has `project_id`, so it keys on project. The table you sync decides the key.
**Count: one bucket per project.**
### A Many-to-Many JOIN: One Bucket Per Row You Select
```yaml theme={null}
streams:
my_assets:
with:
user_projects: SELECT id FROM projects WHERE org_id IN (SELECT org_id FROM org_membership WHERE user_id = auth.user_id())
query: |
SELECT assets.* FROM assets
JOIN project_assets ON project_assets.asset_id = assets.id
WHERE project_assets.project_id IN user_projects
```
This is the pattern that surprises people. The join links assets to projects through `project_assets`. But you select from `assets`, and the join keys on `assets.id`. So PowerSync makes one bucket per asset, not one per project. A user with 2,000 assets gets 2,000 buckets.
**Count: one bucket per asset.**
The subquery, JOIN, and many-to-many examples above each run a *parameter lookup* to find the values to key on. The rows a lookup returns count toward a second limit, the parameter query results limit, which is separate from the bucket limit. For example, if a user belongs to 1,200 orgs, the `org_projects` lookup returns 1,200 rows, which is over the default limit of 1,000, and the sync fails. See [Limits](#limits) to learn how the two limits differ.
## How Bucket Counts Combine
Real streams often combine more than one filter. These examples show how the counts stack up, so you can spot the patterns that grow fastest.
### Two Filters Multiply
Write a stream with two filters. This one syncs the user's projects, but only in a region the client picks:
```yaml theme={null}
streams:
projects_by_region:
with:
user_orgs: SELECT org_id FROM org_membership WHERE user_id = auth.user_id()
query: |
SELECT * FROM projects
WHERE org_id IN user_orgs
AND region = subscription.parameter('region')
```
This stream filters on two things. The first is the user's orgs. The second is a region the client subscribes with. PowerSync makes one bucket for each pair of (org, region).
Say the user has 3 orgs and subscribes to 2 regions. That is 3 × 2 = 6 buckets. Two filters multiply. They do not add.
### Chained Queries Add Each Level
Now chain two levels. This stream syncs the user's orgs, their projects, and their tasks:
```yaml theme={null}
streams:
org_data:
auto_subscribe: true
with:
user_orgs: SELECT org_id FROM org_membership WHERE user_id = auth.user_id()
user_projects: SELECT id FROM projects WHERE org_id IN (SELECT org_id FROM org_membership WHERE user_id = auth.user_id())
queries:
- SELECT * FROM orgs WHERE id IN user_orgs
- SELECT * FROM projects WHERE id IN user_projects
- SELECT * FROM tasks WHERE project_id IN user_projects
```
Say the user has 2 orgs and 6 projects. Follow the buckets one query at a time:
* The `orgs` query keys on org. The user has 2 orgs, so it makes 2 buckets.
* The `projects` query keys on project. The user has 6 projects, so it makes 6 buckets.
* The `tasks` query also keys on project. It uses the same 6 projects, so it shares those buckets and adds 0.
That is 2 + 6 = 8 buckets.
Queries in one stream share buckets only when they key on the same value. The `orgs` query keys on org, so its buckets do not merge with the project buckets. Each new level adds more buckets. With 10 orgs and 50 projects each, this stream makes 10 + 500 = 510 buckets.
### Multiple Queries in One Stream Share Buckets
Queries in the same stream that filter the same way share one bucket per value. Put related tables in one stream to keep the count low:
```yaml theme={null}
streams:
org_overview:
with:
user_orgs: SELECT org_id FROM org_membership WHERE user_id = auth.user_id()
queries:
- SELECT * FROM orgs WHERE id IN user_orgs
- SELECT * FROM projects WHERE org_id IN user_orgs
```
Both queries key on the user's orgs. They share one bucket per org. Say the user has 3 orgs. This stream makes 3 buckets, not 6.
### Global CTEs Count Per Stream
A global CTE lets you write filtering logic once and use it in many streams. It does not share buckets between those streams. Each stream runs the CTE on its own and makes its own buckets.
```yaml theme={null}
config:
edition: 3
with:
user_orgs: SELECT org_id FROM org_membership WHERE user_id = auth.user_id()
streams:
my_orgs:
query: SELECT * FROM orgs WHERE id IN user_orgs
org_projects:
query: SELECT * FROM projects WHERE org_id IN user_orgs
```
Both streams use the same `user_orgs` CTE. But they are separate streams, so they do not share buckets. Say the user has 3 orgs. The `my_orgs` stream makes 3 buckets. The `org_projects` stream makes 3 more. That is 6 buckets, not 3.
Use a global CTE to keep your config readable. To share buckets, put the queries in one stream, as shown above.
### OR Conditions Expand
An `OR` in a filter splits into parts. Take this stream:
```yaml theme={null}
streams:
my_documents:
query: SELECT * FROM documents WHERE owner_id = auth.user_id() OR shared_with = auth.user_id()
```
The filter has an `OR`. PowerSync splits it into two parts: documents you own, and documents shared with you. Each part makes its own bucket. So this stream makes 2 buckets per user, not 1.
PowerSync rewrites `A AND (B OR C)` into `(A AND B) OR (A AND C)`. A shared part like `A` runs in each part. If `A` is a subquery or a parameter lookup, its rows count again in each part. Keep `OR` out of filters that use subqueries or parameters where you can.
Two compile-time limits protect you from runaway expansion. A single stream can define at most 100 buckets. A single filter can expand to at most 100 `OR` terms. If you exceed either, the deploy fails with an error. Move nested `OR` conditions into separate queries to stay under both.
## Limits
PowerSync limits how much a single user can sync. There are **two** limits. Both default to 1,000. Exceeding either fails the sync with a `PSYNC_S2305` error, though each has its own message (shown below). They measure different things, so you need to understand both.
| Limit | What it counts | Global buckets counted? |
| ----------------------- | -------------------------------------------------------------------------------------------------------------- | ----------------------- |
| Buckets per connection | The unique buckets for one connection, after PowerSync removes duplicates | Yes |
| Parameter query results | The rows your parameter lookups return, before duplicates are removed, added up across all lookups in one sync | No |
### Buckets Per Connection
This limit counts the unique buckets a user syncs. PowerSync removes duplicate buckets before it counts. Global buckets count toward this limit. When a user exceeds it, the log shows:
```text theme={null}
Too many buckets: 1200 (limit of 1000)
```
### Parameter Query Results
Some filters need a lookup to find the values to key on. A subquery, a CTE, or a JWT array each produces a list of values, such as the org IDs a user belongs to. This is a *parameter lookup*. The values it returns are *parameter query results*.
This limit counts those result rows. It counts them for one user, across every stream that user subscribes to, added together. It counts them *before* PowerSync removes duplicates. Global buckets use no lookup, so they do not count toward this limit.
The limit applies to the values that define buckets, not to the number of data rows synced. A separate, much higher limit controls how many data rows a client can sync (see [Performance and Limits](/resources/performance-and-limits)). So a limit of 1,000 means up to 1,000 bucket-defining values per user.
When a user exceeds it, the log shows:
```text theme={null}
Too many parameter query results (limit of 1000)
```
### Why the Two Counts Differ
The two limits measure different things, so they are usually different numbers. A stream can produce a small set of unique buckets from a large number of parameter rows. In that case you reach the parameter limit first. A config with many global buckets can reach the bucket limit first.
This is why a checkpoint log can read `buckets: 7 | param_results: 6`. One global bucket adds to the bucket count but not to the parameter count.
The parameter limit can stop a sync while the bucket count still looks safe. A user can fail with far fewer than 1,000 buckets, because their parameter lookups returned more than 1,000 rows. Always check both numbers.
In legacy [Sync Rules](/sync/rules/overview), these two limits were effectively one number, because each parameter-query result became one bucket. In Sync Streams they can diverge.
### Total Buckets vs Buckets Per User
The 1,000 limit applies to each individual user, not to your whole instance. Your PowerSync Service can track millions of buckets in total, as long as each user syncs fewer than the limit. A large total bucket count is not a problem on its own.
## Related Pages
* [Reducing Bucket Count](/sync/advanced/reducing-bucket-count) diagnoses and fixes a high bucket count.
* [Writing Queries](/sync/streams/queries) covers the query syntax that determines your bucket count.
* [Common Table Expressions (CTEs)](/sync/streams/ctes) covers shared filtering logic.
* [Using Parameters](/sync/streams/parameters) covers auth, subscription, and connection parameters.
* [Examples, Patterns & Demos](/sync/streams/examples) covers common stream patterns.
* [Bucket System](/architecture/powersync-service#bucket-system) covers how buckets work internally.
* [Troubleshooting](/debugging/troubleshooting) covers common PowerSync issues and diagnostics.
* [Performance and Limits](/resources/performance-and-limits) lists the Service limits.
# Client-Side Usage
Source: https://docs.powersync.com/sync/streams/client-usage
Subscribe to Sync Streams from your client app and manage subscriptions dynamically.
After [defining your streams](/sync/streams/overview#defining-streams) on the server-side, your client app subscribes to them to start syncing data (this is an explicit operation unless streams are configured to [auto-subscribe](/sync/streams/overview#using-auto-subscribe)). This page covers everything you need to use Sync Streams from your client code.
## Quick Start
Streams that are configured to [auto-subscribe](/sync/streams/overview#using-auto-subscribe) will automatically start syncing as soon as you connect to your PowerSync instance in your client-side application.
For any other streams, the basic pattern is: **subscribe** to a stream, **wait** for data to sync, then **unsubscribe** when done.
**Tauri SDK**: The JavaScript API shown in the TypeScript/JavaScript tabs throughout this page applies to Tauri as well. Import from `@powersync/tauri-plugin` (for `PowerSyncTauriDatabase`) or `@powersync/common` (for shared types). See the [Connection Parameters](#connection-parameters) section below for how Tauri handles connect-time parameters differently.
```js theme={null}
// Subscribe to a stream with parameters
const sub = await db.syncStream('list_todos', { list_id: 'abc123' }).subscribe();
// Wait for the initial data to sync
await sub.waitForFirstSync();
// The stream's rows are now in the local SQLite database.
// TODO: Read the todos for this list with a local query.
// When the data is no longer needed
sub.unsubscribe();
```
```dart theme={null}
// Subscribe to a stream with parameters
final sub = await db.syncStream('list_todos', {'list_id': 'abc123'}).subscribe();
// Wait for the initial data to sync
await sub.waitForFirstSync();
// The stream's rows are now in the local SQLite database.
// TODO: Read the todos for this list with a local query.
// When the data is no longer needed
sub.unsubscribe();
```
```kotlin theme={null}
// Subscribe to a stream with parameters
val sub = database.syncStream("list_todos", mapOf("list_id" to JsonParam.String("abc123")))
.subscribe()
// Wait for the initial data to sync
sub.waitForFirstSync()
// The stream's rows are now in the local SQLite database.
// TODO: Read the todos for this list with a local query.
// When the data is no longer needed
sub.unsubscribe()
```
```swift theme={null}
// Subscribe to a stream with parameters
let sub = try await db.syncStream(name: "list_todos", params: ["list_id": .string("abc123")]).subscribe()
// Wait for the initial data to sync
try await sub.waitForFirstSync()
// The stream's rows are now in the local SQLite database.
// TODO: Read the todos for this list with a local query.
// When the data is no longer needed
try await sub.unsubscribe()
```
```csharp theme={null}
// Subscribe to a stream with parameters
var sub = await db.SyncStream("list_todos", new() { ["list_id"] = "abc123" }).Subscribe();
// Wait for the initial data to sync
await sub.WaitForFirstSync();
// The stream's rows are now in the local SQLite database.
// TODO: Read the todos for this list with a local query.
// When the data is no longer needed
sub.Unsubscribe();
```
```Rust theme={null}
use serde_json::json;
// Subscribe to a stream with parameters
let sub = db
.sync_stream("list_todos", Some(&json!({ "list_id": "abc123" })))
.subscribe()
.await?;
// Wait for the initial data to sync
sub.wait_for_first_sync().await;
// The stream's rows are now in the local SQLite database.
// TODO: Read the todos for this list with a local query.
// When the data is no longer needed.
// Dropping the subscription handle has the same effect.
sub.unsubscribe();
```
## Framework Integrations
Most developers use framework-specific hooks that handle subscription lifecycle automatically.
The PowerSync React package provides three hooks relevant to Sync Stream subscriptions. Each serves a distinct role:
* **`useQuery`**: queries the local SQLite database and returns live-updating results. It also accepts a `streams` option to subscribe to streams alongside the query — use this when the stream is only needed for that specific query.
* **`useSyncStream`**: subscribes to a single named Sync Stream. Use this when the same stream's data is needed across multiple queries or components; the subscription lives independently of any particular `useQuery` call and is canceled when the component unmounts.
* **`useSyncStreams`**: the same as `useSyncStream` but for a variable number of streams. Accepts an array of stream options; all subscriptions are canceled when the component unmounts or the array changes.
### `useQuery` with `streams`: Tethering a Subscription to a Query
`useQuery` accepts an optional `streams` option that subscribes to one or more streams and ties their lifecycle to that specific query. This is the simplest option when the stream is only ever needed alongside this one query:
```jsx theme={null}
import { useQuery } from '@powersync/react';
function TodoList({ listId }) {
// The stream subscription is created alongside this query and canceled with it.
const { data: todos } = useQuery(
'SELECT * FROM todos WHERE list_id = ?',
[listId],
{ streams: [{ name: 'list_todos', parameters: { list_id: listId }, waitForStream: true }] }
);
return
Both the `useQuery` and `useQueries` hooks automatically subscribe when the component mounts and will unsubscribe when it unmounts:
```jsx theme={null}
function TodoList({ listId }) {
// Automatically subscribes/unsubscribes based on component lifecycle
const stream = useSyncStream({ name: 'list_todos', parameters: { list_id: listId } });
const { data: todos, isLoading } = useQuery({
queryKey: ['test'],
query: 'SELECT 1',
streams: [{ name: 'list_todos', parameters: { list_id: listId }, waitForStream: true }]
});
// Check if data has synced
if (isLoading) {
return
The `useSyncStream` composable automatically subscribes when the component mounts and unsubscribes when it unmounts:
```vue theme={null}
```
You can also have `useQuery` wait for a stream before running:
```Javascript theme={null}
// This query waits for the stream to sync before executing
const { data: todos } = useQuery(
'SELECT * FROM todos WHERE list_id = ?',
[listId],
{ streams: [
{ name: 'list_todos',
parameters: { list_id: listId },
waitForStream: true
}
]
}
);
```
The `composeSyncStream` extension subscribes to a stream as long as a composable is part of the composition. It returns `SyncStreamStatus?` for the subscription so you can check sync state.
The `composeSyncStream` helper was added in Kotlin SDK v1.11.0.
```kotlin theme={null}
@Composable
fun TodoList(database: PowerSyncDatabase, listId: String) {
val status = database.composeSyncStream(
name = "list_todos",
parameters = mapOf("list_id" to JsonParam.String(listId))
)
if (status?.subscription?.hasSynced != true) {
LoadingSpinner()
return
}
val todos = database.getAll("SELECT * FROM todos WHERE list_id = ?", listOf(listId))
TodoItems(todos = todos)
}
```
You can pass `ttl` and `priority` for cache duration and [sync priority](/sync/advanced/prioritized-sync):
```kotlin theme={null}
database.composeSyncStream(
name = "list_todos",
parameters = mapOf("list_id" to JsonParam.String(listId)),
ttl = 1.hours,
priority = StreamPriority(1)
)
```
## Type-Safe Stream Wrappers
When you generate your client-side schema from the [PowerSync Dashboard](https://dashboard.powersync.com) or CLI, typed stream wrappers are generated alongside the schema for all SDKs. These catch typos in stream names and parameter names at compile time — mistakes that would otherwise cause silent data-missing bugs only detectable by inspecting sync status.
For example, without typed wrappers this fails silently — no error, but data won't sync:
```js theme={null}
// Wrong stream name, wrong parameter key — no compile error, data just won't sync
await db.syncStream('note', { project_id: 'abc' }).subscribe();
```
### The Generated Code
The schema generator produces typed wrappers at the bottom of your generated schema file. Given streams like:
```yaml theme={null}
streams:
lists:
query: SELECT * FROM lists WHERE owner_id = auth.user_id()
todos:
query: SELECT * FROM todos WHERE list_id = subscription.parameter('list')
```
The generated output (JavaScript/TypeScript) looks like:
```typescript theme={null}
import { column, Schema, Table, PowerSyncDatabase, SyncStream } from '@powersync/web';
// OR: import { ... } from '@powersync/react-native';
// OR (Tauri): import { ... } from '@powersync/common';
// ... table definitions ...
export const AppSchema = new Schema({ lists, todos });
export function typedStreams(db: PowerSyncDatabase) {
return {
lists(): SyncStream {
return db.syncStream('lists', {});
},
todos(params: { list: string }): SyncStream {
return db.syncStream('todos', params);
}
};
}
```
### Usage
Use the generated wrappers instead of calling `db.syncStream()` directly. Each method returns a `SyncStream`, so you can chain `.subscribe()`, `.subscribe({ ttl, priority })`, and all other methods covered on this page.
```typescript theme={null}
import { typedStreams } from './powersync/schema';
// Stream without subscription parameters
const sub = await typedStreams(db).lists().subscribe();
// Stream with subscription parameters — names and types are enforced
const sub = await typedStreams(db).todos({ list: 'list-id-abc' }).subscribe();
// Works with framework hooks
const stream = useSyncStream(typedStreams(db).todos({ list: listId }));
```
```dart theme={null}
// Stream without subscription parameters
final sub = await TypedSyncStreams(db).lists().subscribe();
// Stream with subscription parameters
final sub = await TypedSyncStreams(db).todos(list: 'list-id-abc').subscribe();
```
```kotlin theme={null}
// Stream without subscription parameters
val sub = TypedSyncStreams(db).lists().subscribe()
// Stream with subscription parameters
val sub = TypedSyncStreams(db).todos(list = "list-id-abc").subscribe()
```
```swift theme={null}
// Stream without subscription parameters
let sub = try await TypedSyncStreams(db).lists().subscribe()
// Stream with subscription parameters
let sub = try await TypedSyncStreams(db).todos(list: "list-id-abc").subscribe()
```
```csharp theme={null}
// Stream without subscription parameters
var sub = await new TypedSyncStreams(db).Lists().Subscribe();
// Stream with subscription parameters
var sub = await new TypedSyncStreams(db).Todos(list: "list-id-abc").Subscribe();
```
Type-safe wrappers are only generated for streams that do **not** have `auto_subscribe: true`. Auto-subscribe streams start syncing automatically on connect and don't require explicit client subscriptions, so no wrapper is generated for them.
## Checking Sync Status
You can check whether a subscription has synced and monitor download progress:
```js theme={null}
const sub = await db.syncStream('list_todos', { list_id: 'abc123' }).subscribe();
// Check if this subscription has completed initial sync
const status = db.currentStatus.forStream(sub);
console.log(status?.subscription.hasSynced); // true/false
console.log(status?.progress); // download progress
```
```dart theme={null}
final sub = await db.syncStream('list_todos', {'list_id': 'abc123'}).subscribe();
// Check if this subscription has completed initial sync
final status = db.currentStatus.forStream(sub);
print(status?.subscription.hasSynced); // true/false
print(status?.progress); // download progress
```
```kotlin theme={null}
val sub = database.syncStream("list_todos", mapOf("list_id" to JsonParam.String("abc123")))
.subscribe()
// Check if this subscription has completed initial sync
val status = database.currentStatus.forStream(sub)
println(status?.subscription?.hasSynced) // true/false
println(status?.progress) // download progress
```
```swift theme={null}
let sub = try await db.syncStream(name: "list_todos", params: ["list_id": JsonValue.string("abc123")]).subscribe()
// Check if this subscription has completed initial sync
let status = db.currentStatus.forStream(stream: sub)
print(status?.subscription.hasSynced ?? false) // true/false
print(status?.progress) // download progress
```
```csharp theme={null}
var sub = await db.SyncStream("list_todos", new() { ["list_id"] = "abc123" }).Subscribe();
// Check if this subscription has completed initial sync
var status = db.CurrentStatus.ForStream(sub);
Console.WriteLine(status?.Subscription.HasSynced); // true/false
Console.WriteLine(status?.Progress); // download progress
```
## TTL (Time-To-Live)
TTL controls how long data remains cached after you unsubscribe. This enables "warm cache" behavior — when users navigate back to a screen, data may already be available without waiting for a sync.
**Default behavior:** Data is cached for 24 hours after unsubscribing. For most apps, this default works well.
### Setting a Custom TTL
```js theme={null}
// Cache for 1 hour after unsubscribe (TTL in seconds)
const sub = await db.syncStream('todos', { list_id: 'abc' })
.subscribe({ ttl: 3600 });
// Cache indefinitely (data never expires)
const sub = await db.syncStream('todos', { list_id: 'abc' })
.subscribe({ ttl: Infinity });
// No caching (remove data immediately on unsubscribe)
const sub = await db.syncStream('todos', { list_id: 'abc' })
.subscribe({ ttl: 0 });
```
```dart theme={null}
// Cache for 1 hour after unsubscribe
final sub = await db.syncStream('todos', {'list_id': 'abc'})
.subscribe(ttl: const Duration(hours: 1));
// Cache for 7 days
final sub = await db.syncStream('todos', {'list_id': 'abc'})
.subscribe(ttl: const Duration(days: 7));
```
```kotlin theme={null}
// Cache for 1 hour after unsubscribe
val sub = database.syncStream("todos", mapOf("list_id" to JsonParam.String("abc")))
.subscribe(ttl = 1.hours)
// Cache for 7 days
val sub = database.syncStream("todos", mapOf("list_id" to JsonParam.String("abc")))
.subscribe(ttl = 7.days)
```
```swift theme={null}
// Cache for 1 hour after unsubscribe (TTL in seconds)
let sub = try await db.syncStream(name: "todos", params: ["list_id": JsonValue.string("abc")])
.subscribe(ttl: 60 * 60, priority: nil)
// Cache for 7 days
let sub = try await db.syncStream(name: "todos", params: ["list_id": JsonValue.string("abc")])
.subscribe(ttl: 60 * 60 * 24 * 7, priority: nil)
```
```csharp theme={null}
// Cache for 1 hour after unsubscribe
var sub = await db.SyncStream("todos", new() { ["list_id"] = "abc" })
.Subscribe(new SyncStreamSubscribeOptions { Ttl = TimeSpan.FromHours(1) });
// Cache for 7 days
var sub = await db.SyncStream("todos", new() { ["list_id"] = "abc" })
.Subscribe(new SyncStreamSubscribeOptions { Ttl = TimeSpan.FromDays(7) });
```
### How TTL Works
* **Per-subscription**: Each `(stream name, parameters)` pair has its own TTL.
* **First subscription wins**: If you subscribe to the same stream with the same parameters multiple times, the TTL from the first subscription is used.
* **After unsubscribe**: Data continues syncing for the TTL duration, then is removed from the client-side SQLite database.
```js theme={null}
// Example: User opens two lists with different TTLs
const subA = await db.syncStream('todos', { list_id: 'A' }).subscribe({ ttl: 43200 }); // 12h
const subB = await db.syncStream('todos', { list_id: 'B' }).subscribe({ ttl: 86400 }); // 24h
// Each subscription is independent
// List A data cached for 12h after unsubscribe
// List B data cached for 24h after unsubscribe
```
## Priority Override
Streams can have a default priority set in the YAML sync configuration (see [Prioritized Sync](/sync/advanced/prioritized-sync)). When subscribing, you can override this priority for a specific subscription:
```js theme={null}
// Override the stream's default priority
const sub = await db.syncStream('todos', { list_id: 'abc' }).subscribe({ priority: 1 });
```
When different components subscribe to the same stream with the same parameters but different priorities, PowerSync uses the highest priority for syncing. That higher priority is kept until the subscription ends (or its TTL expires). Subscriptions with different parameters are independent and do not conflict.
## Connection Parameters
Connection parameters are a more advanced feature for values that apply to all streams in a session. They're the Sync Streams equivalent of [Client Parameters](/sync/rules/client-parameters) in legacy Sync Rules.
For most use cases, **subscription parameters** (passed when subscribing) are more flexible and recommended. Use connection parameters only when you need a single global value across all streams, like an environment flag.
Define streams that use connection parameters:
```yaml theme={null}
streams:
config:
auto_subscribe: true
query: SELECT * FROM config WHERE env = connection.parameter('environment')
```
Set connection parameters when connecting:
**Tauri SDK**: Since `connect()` must be called from Rust for Tauri, connection parameters are passed via `SyncOptions` in your Rust connector. See the [Tauri SDK reference](/client-sdks/reference/tauri#2-instantiate-the-powersync-database) for details on setting up the Rust connector.
```js theme={null}
await db.connect(connector, {
params: { environment: 'production' }
});
```
```dart theme={null}
await db.connect(
connector: connector,
params: {'environment': 'production'},
);
```
```kotlin theme={null}
database.connect(
connector,
params = mapOf("environment" to JsonParam.String("production"))
)
```
```swift theme={null}
try await db.connect(
connector: connector,
options: ConnectOptions(params: ["environment": JsonValue.string("production")])
)
```
```csharp theme={null}
await db.Connect(connector, new PowerSyncConnectionOptions {
Params = new() { ["environment"] = "production" }
});
```
## Opting Out of Auto-Subscribed Streams
By default, every stream defined with [`auto_subscribe: true`](/sync/streams/overview#using-auto-subscribe) starts syncing as soon as the client connects. For advanced use cases where the client needs more control over what syncs at connect time, you can set `includeDefaultStreams: false` on the connect options. The `auto_subscribe: true` flag is then ignored for that connection, and those streams only sync if the client explicitly calls `db.syncStream(name, params).subscribe()`.
Most apps will not need this option. Defining streams without `auto_subscribe: true` and subscribing on-demand from the client generally achieves a similar result with less indirection. One scenario where it can help is during a migration from auto-subscribed streams to on-demand subscriptions. You keep the auto-subscribed streams in place so older client versions continue to sync correctly, while newer versions that explicitly subscribe to the streams they need set `includeDefaultStreams: false` to skip those defaults.
```js theme={null}
// Connect without syncing auto_subscribe streams
await db.connect(connector, {
includeDefaultStreams: false
});
// Later, opt in to a specific stream
const sub = await db.syncStream('my_stream', params).subscribe();
```
```dart theme={null}
await db.connect(
connector: connector,
options: const SyncOptions(includeDefaultStreams: false),
);
```
```kotlin theme={null}
database.connect(
connector,
options = SyncOptions(includeDefaultStreams = false)
)
```
```swift theme={null}
try await db.connect(
connector: connector,
options: ConnectOptions(includeDefaultStreams: false)
)
```
```csharp theme={null}
await db.Connect(connector, new PowerSyncConnectionOptions {
IncludeDefaultStreams = false
});
```
## API Reference
For quick reference, here are the key methods available in each SDK:
| Method | Description |
| --------------------------------- | -------------------------------------------------------------------------------------------------------------- |
| `db.syncStream(name, params)` | Get a `SyncStream` instance for a stream with optional parameters |
| `stream.subscribe(options)` | Subscribe to the stream. Returns a `SyncStreamSubscription` |
| `subscription.waitForFirstSync()` | Wait until the subscription has completed its initial sync |
| `subscription.unsubscribe()` | Unsubscribe from the stream (data [remains cached](/sync/streams/client-usage#how-ttl-works) for TTL duration) |
| `db.currentStatus.forStream(sub)` | Get sync status and progress for a subscription |
# Common Table Expressions (CTEs)
Source: https://docs.powersync.com/sync/streams/ctes
Reuse common query patterns across Sync Streams using Common Table Expressions (CTEs) to simplify configurations and reduce duplication.
When streams need reusable filtering logic, you can define it once in a Common Table Expression (CTE) and reference it in queries. CTEs can be defined at the stream level (scoped to one stream) or at the top level of your Sync Config (shared across all streams). This keeps stream definitions DRY and makes it easier to maintain. For the supported syntax of the `with` block and CTE rules, see [Supported SQL — CTE and WITH syntax](/sync/supported-sql#cte-and-with-syntax).
## Why Use CTEs
Consider an app where users belong to organizations. Several tables need to filter by the user's organizations:
```yaml theme={null}
# Without CTEs - repetitive and hard to maintain
streams:
org_projects:
query: |
SELECT * FROM projects
WHERE org_id IN (SELECT org_id FROM org_members WHERE user_id = auth.user_id())
org_repositories:
query: |
SELECT * FROM repositories
WHERE org_id IN (SELECT org_id FROM org_members WHERE user_id = auth.user_id())
org_settings:
query: |
SELECT * FROM settings
WHERE org_id IN (SELECT org_id FROM org_members WHERE user_id = auth.user_id())
```
The same subquery appears three times. You can merge these into one stream and define the logic once as a stream-level CTE:
```yaml theme={null}
# With a CTE and multiple queries
streams:
org_data:
with:
user_orgs: SELECT org_id FROM org_members WHERE user_id = auth.user_id()
queries:
- SELECT * FROM projects WHERE org_id IN user_orgs
- SELECT * FROM repositories WHERE org_id IN user_orgs
- SELECT * FROM settings WHERE org_id IN user_orgs
```
If the membership logic changes, you update it in one place. When the streams cannot be merged (for example, because they have different `auto_subscribe` settings or separate subscription parameters), use a [global CTE](#global-ctes) to share the logic across streams instead.
## Stream-Level CTEs
Define CTEs in a `with` block inside a stream. Each CTE has a name and a `SELECT` query, and is available only within that stream:
```yaml theme={null}
streams:
my_stream:
with:
cte_name: SELECT columns FROM table WHERE conditions
query: SELECT * FROM some_table WHERE col IN cte_name
```
The CTE query can include any filtering logic, including parameters:
```yaml theme={null}
streams:
user_data:
with:
user_orgs: SELECT org_id FROM org_members WHERE user_id = auth.user_id()
active_projects: SELECT id FROM projects WHERE archived = false
query: SELECT * FROM projects WHERE org_id IN user_orgs AND id IN active_projects
```
## Global CTEs
When the same filtering logic is needed across streams that cannot be merged — for example because they have different `auto_subscribe` settings or different subscription parameters — define the CTE once in a top-level `with` block. All streams can then reference it without repeating the definition:
```yaml theme={null}
config:
edition: 3
with:
user_orgs: SELECT org_id FROM org_members WHERE user_id = auth.user_id()
streams:
# Auto-synced overview data
org_overview:
auto_subscribe: true
queries:
- SELECT * FROM organizations WHERE id IN user_orgs
- SELECT * FROM team_members WHERE org_id IN user_orgs
# On-demand project detail, subscribed per project
project_detail:
queries:
- SELECT * FROM projects WHERE id = subscription.parameter('project_id') AND org_id IN user_orgs
- SELECT * FROM tasks WHERE project_id = subscription.parameter('project_id')
```
If the membership logic changes, you update it in one place and all streams pick up the change.
A stream can still define its own local `with` block. Local CTEs take precedence over global CTEs when names conflict, so you can override a global CTE for a specific stream:
```yaml theme={null}
config:
edition: 3
with:
user_orgs: SELECT org_id FROM org_members WHERE user_id = auth.user_id()
streams:
admin_projects:
# Local CTE overrides the global one for this stream
with:
user_orgs: SELECT org_id FROM org_members WHERE user_id = auth.user_id() AND role = 'admin'
query: SELECT * FROM projects WHERE org_id IN user_orgs
```
## Using CTEs in Queries
Once defined — either in a stream's local `with` block or in the top-level global `with` block — use the CTE name in the stream's query or queries. You can use it like a subquery or join it as if it were a table.
**Short-hand syntax** (when the CTE has exactly one column):
```yaml theme={null}
streams:
projects:
with:
user_orgs: SELECT org_id FROM org_members WHERE user_id = auth.user_id()
query: SELECT * FROM projects WHERE org_id IN user_orgs
```
The short-hand `IN cte_name` is equivalent to `IN (SELECT * FROM cte_name)`. If the CTE has more than one column, this form is an error; use explicit subquery or join syntax instead.
**Explicit subquery syntax** (when you need to select specific columns):
```yaml theme={null}
streams:
projects:
with:
user_orgs: SELECT org_id FROM org_members WHERE user_id = auth.user_id()
query: SELECT * FROM projects WHERE org_id IN (SELECT org_id FROM user_orgs)
```
**Join syntax** (you can join a CTE as if it were a table). Only `INNER JOIN` is supported:
```yaml theme={null}
streams:
projects:
with:
user_orgs: SELECT org_id FROM org_members WHERE user_id = auth.user_id()
query: SELECT projects.* FROM projects INNER JOIN user_orgs ON user_orgs.org_id = projects.org_id
```
## Combining with Multiple Queries
CTEs work well with the `queries` feature (multiple queries per stream). This lets you share the CTE and keep all query results in one stream: the client only needs to manage one subscription instead of multiple.
```yaml theme={null}
streams:
user_data:
with:
my_org: SELECT org_id FROM org_members WHERE user_id = auth.user_id()
queries:
- SELECT * FROM projects WHERE org_id IN my_org
- SELECT * FROM repositories WHERE org_id IN my_org
- SELECT * FROM team_members WHERE org_id IN my_org
```
## Complete Example
A full configuration using both global and stream-level CTEs. The `accessible_projects` logic is shared across streams via a global CTE; `user_orgs` is only needed by `org_and_projects`, so it stays stream-level:
```yaml theme={null}
config:
edition: 3
# Global CTE - shared across all streams
with:
accessible_projects: |
SELECT id FROM projects
WHERE org_id IN (SELECT org_id FROM org_memberships WHERE user_id = auth.user_id())
OR id IN (SELECT project_id FROM project_shares WHERE shared_with = auth.user_id())
streams:
# Organization-level data (auto-sync) - stream-level CTE + global CTE
org_and_projects:
auto_subscribe: true
with:
user_orgs: |
SELECT org_id FROM org_memberships WHERE user_id = auth.user_id()
queries:
- SELECT * FROM organizations WHERE id IN user_orgs
- SELECT * FROM projects WHERE id IN accessible_projects
# Project details (on-demand) - uses only the global CTE
project_details:
queries:
- |
SELECT * FROM tasks
WHERE project_id = subscription.parameter('project_id')
AND project_id IN accessible_projects
- |
SELECT * FROM files
WHERE project_id = subscription.parameter('project_id')
AND project_id IN accessible_projects
```
## Limitations
The following rules apply to CTEs. For the full syntax reference, see [Supported SQL — CTE and WITH syntax](/sync/supported-sql#cte-and-with-syntax).
**CTE names must not shadow source table names.** This prevents hard-to-debug situations where a global CTE silently changes the behavior of a stream defined elsewhere in the config.
**Local CTEs take precedence over global CTEs.** If a stream defines a CTE with the same name as a global CTE, the stream-level definition is used within that stream.
**CTEs cannot reference other CTEs.** Each CTE must be self-contained:
```yaml theme={null}
# This won't work - cte2 cannot reference cte1
streams:
my_stream:
with:
cte1: SELECT org_id FROM org_members WHERE user_id = auth.user_id()
cte2: SELECT id FROM projects WHERE org_id IN cte1 # Error!
```
If you need to chain filters, use nested subqueries in your stream query instead:
```yaml theme={null}
streams:
tasks:
with:
user_orgs: SELECT org_id FROM org_members WHERE user_id = auth.user_id()
query: |
SELECT * FROM tasks
WHERE project_id IN (
SELECT id FROM projects WHERE org_id IN user_orgs
)
```
**The short-hand `IN cte_name` works only when the CTE has exactly one column.** If the CTE has multiple columns, use explicit subquery syntax or join the CTE as a table.
**Stream-level CTE names take precedence over table/collection names.** If you define a stream-level CTE with the same name as a database table/collection, the CTE is used within that stream. Choose distinct names to avoid confusion.
# Examples, Patterns & Demos
Source: https://docs.powersync.com/sync/streams/examples
Common Sync Streams patterns and real-world examples for different sync configurations.
## Common Patterns
These patterns show how to combine Sync Streams features to solve common real-world scenarios.
### Organization-Scoped Data
For apps where users belong to an organization (or company, team, workspace, etc.), use JWT claims to scope data. The `org_id` in the JWT ensures users only see data from their organization, without needing to pass it from the client.
```yaml theme={null}
streams:
# All projects in the user's organization (auto-sync on connect)
org_projects:
auto_subscribe: true
query: SELECT * FROM projects WHERE org_id = auth.parameter('org_id')
# Tasks for a specific project (sync on-demand)
project_tasks:
query: |
SELECT * FROM tasks
WHERE project_id = subscription.parameter('project_id')
AND project_id IN (SELECT id FROM projects WHERE org_id = auth.parameter('org_id'))
```
Your backend should include the `org_id` in the JWT payload when issuing tokens — e.g. `{ "sub": "user-123", "org_id": "org-456" }`. Clients auto-subscribe to `org_projects` when they connect, so the project list is available offline immediately. Subscribe to `project_tasks` when the user opens a project:
```js theme={null}
// When the user opens a project view
const sub = await db.syncStream('project_tasks', { project_id: projectId }).subscribe();
await sub.waitForFirstSync();
// Unsubscribe when the user navigates away
sub.unsubscribe();
```
For more complex organization structures where users can belong to multiple organizations, see [Expanding JSON Arrays](/sync/streams/parameters#expanding-json-arrays).
### Schema-per-Tenant Data (Postgres)
For multi-tenant Postgres databases with one identical schema per tenant, use a wildcard schema with the `schema()` function. A single stream covers every tenant schema, and each client syncs only the schema named by the `tenant_schema` claim in its JWT:
```yaml theme={null}
config:
edition: 3
streams:
work_orders:
query: SELECT * FROM "%".work_orders WHERE work_orders.schema() = auth.parameter('tenant_schema')
```
This requires PowerSync Service v1.24.0 or later. See [Wildcard Schemas](/sync/advanced/schemas-and-connections#wildcard-schemas-postgres) for requirements and details.
### Role-Based Access
When different users should see different data based on their role, use JWT claims to apply visibility rules. This keeps authorization logic on the server side where it's secure.
```yaml theme={null}
streams:
# Admins see all articles, others see only published or their own
articles:
auto_subscribe: true
query: |
SELECT * FROM articles
WHERE org_id = auth.parameter('org_id')
AND (
status = 'published'
OR author_id = auth.user_id()
OR auth.parameter('role') = 'admin'
)
```
Your backend should include both `org_id` and `role` in the JWT — e.g. `{ "sub": "user-123", "org_id": "org-456", "role": "admin" }`. The `role` claim is set by your backend so users can't escalate their own privileges. In this example, Clients auto-subscribe to `articles` when they connect — no client-side subscription call needed.
### Shared Resources
For apps where users can share items with each other (like documents or folders), combine ownership checks with a "shares table" lookup. This syncs both items the user owns and items others have shared with them.
```yaml theme={null}
streams:
my_documents:
auto_subscribe: true
query: |
SELECT * FROM documents
WHERE owner_id = auth.user_id()
OR id IN (SELECT document_id FROM document_shares WHERE shared_with = auth.user_id())
```
Clients auto-subscribe to `my_documents` when they connect, so the user's documents (owned and shared) are available immediately.
### Syncing Related Data
When a detail view needs data from multiple tables (like an issue and its comments), use a [CTE](/sync/streams/ctes) and [multiple queries per stream](/sync/streams/queries#multiple-queries-per-stream) to define the authorization check once and sync both tables in one subscription.
```yaml theme={null}
streams:
issue_with_comments:
with:
my_projects: SELECT project_id FROM project_members WHERE user_id = auth.user_id()
queries:
- |
SELECT * FROM issues
WHERE id = subscription.parameter('issue_id')
AND project_id IN my_projects
- |
SELECT comments.* FROM comments
INNER JOIN issues ON comments.issue_id = issues.id
WHERE comments.issue_id = subscription.parameter('issue_id')
AND issues.project_id IN my_projects
```
Subscribe once when the user opens an issue:
```js theme={null}
// When the user opens an issue view
const issueSub = await db.syncStream('issue_with_comments', { issue_id: issueId }).subscribe();
await issueSub.waitForFirstSync();
// Unsubscribe when the user navigates away
issueSub.unsubscribe();
```
If multiple streams share the same filtering logic, consider using [CTEs](/sync/streams/ctes) to avoid repetition and [multiple queries per stream](/sync/streams/queries#multiple-queries-per-stream) so the client only needs to manage one subscription instead of multiple. This is more efficient and results in fewer sync buckets .
### User's Default or Primary Item
When users have a "default" or "primary" item stored in their profile, you can sync related data automatically without the client needing to know the ID upfront.
```yaml theme={null}
streams:
# Sync todos from the user's primary list
primary_list_todos:
auto_subscribe: true
query: |
SELECT * FROM todos
WHERE list_id IN (
SELECT primary_list_id FROM users WHERE id = auth.user_id()
)
```
The subquery looks up the user's `primary_list_id` from the `users` table, then syncs all `todos` from that list. When the user changes their primary list in the database, the synced data updates automatically. Clients auto-subscribe to `primary_list_todos` when they connect — no client-side subscription call needed.
### Hierarchical Data
When your data has parent-child relationships across multiple levels, you can traverse the hierarchy using nested subqueries or joins. This is common in apps where access to child records is determined by membership at a higher level.
For example, consider an app with organizations, projects, and tasks. Users belong to organizations, and should see all tasks in projects that belong to their organizations:
```
Organization → Projects → Tasks
↑
User membership
```
**Using nested subqueries:**
```yaml theme={null}
streams:
org_tasks:
auto_subscribe: true
query: |
SELECT * FROM tasks
WHERE project_id IN (
SELECT id FROM projects WHERE org_id IN (
SELECT org_id FROM org_members WHERE user_id = auth.user_id()
)
)
```
The query reads from inside out: find the user's organizations, then find projects in those organizations, then find tasks in those projects.
**Using joins** (often easier to read for deeply nested hierarchies):
```yaml theme={null}
streams:
org_tasks:
auto_subscribe: true
query: |
SELECT tasks.* FROM tasks
INNER JOIN projects ON tasks.project_id = projects.id
INNER JOIN org_members ON projects.org_id = org_members.org_id
WHERE org_members.user_id = auth.user_id()
```
Both queries produce the same result. You can express these nested relationships directly, without denormalizing your database or adding redundant foreign keys. Keep in mind that the query pattern determines how many buckets each user syncs. Joining or filtering through a parent table creates one bucket per row of that parent table, not per row of the table you sync. Here that is one bucket per project, and all of a project's tasks share it. This can grow large at scale. See [Bucket Count and Limits](/sync/streams/bucket-count) to plan for this. Clients auto-subscribe to `org_tasks` when they connect, with no client-side subscription call needed.
### Many-to-Many Relationships
Many-to-many relationships (like users subscribing to boards) typically use a join table. Sync Streams support `INNER JOIN`s, so you can traverse these relationships directly without denormalizing your schema.
Consider a social app where users subscribe to message boards:
```
Users ←→ board_subscriptions ←→ Boards → Posts → Comments
```
```yaml theme={null}
streams:
# Posts from boards the user subscribes to
board_posts:
auto_subscribe: true
query: |
SELECT posts.* FROM posts
INNER JOIN board_subscriptions ON posts.board_id = board_subscriptions.board_id
WHERE board_subscriptions.user_id = auth.user_id()
# Comments on those posts (no denormalization needed)
board_comments:
auto_subscribe: true
query: |
SELECT comments.* FROM comments
INNER JOIN posts ON comments.post_id = posts.id
INNER JOIN board_subscriptions ON posts.board_id = board_subscriptions.board_id
WHERE board_subscriptions.user_id = auth.user_id()
# User profiles for co-subscribers (people who share a board with me)
board_users:
auto_subscribe: true
query: |
SELECT users.* FROM users
INNER JOIN board_subscriptions ON users.id = board_subscriptions.user_id
WHERE board_subscriptions.board_id IN (
SELECT board_id FROM board_subscriptions WHERE user_id = auth.user_id()
)
```
Clients auto-subscribe to all three streams when they connect. Each query joins through `board_subscriptions` to find relevant data: posts in the user's boards, comments on those posts, and other users sharing those boards.
Unlike with legacy [Sync Rules](/sync/rules/many-to-many-join-tables), you don't need to denormalize your schema or maintain array columns to express these relationships. At scale, denormalizing the scope key onto child tables can still be the best way to control how many buckets each user syncs. See [Bucket Count and Limits](/sync/streams/bucket-count).
## Use Case Examples
Complete configurations for common application types.
### To-do List App
Sync the list of `lists` upfront, but only sync `todos` when the user opens a specific list:
```yaml theme={null}
config:
edition: 3
streams:
# Always available - user can see their lists offline
lists:
auto_subscribe: true
query: SELECT * FROM lists WHERE owner_id = auth.user_id()
# Loaded on demand - only sync todos for the list being viewed
list_todos:
query: |
SELECT * FROM todos
WHERE list_id = subscription.parameter('list_id')
AND list_id IN (SELECT id FROM lists WHERE owner_id = auth.user_id())
```
Clients auto-subscribe to `lists` when they connect. Subscribe to `list_todos` when the user opens a list:
```js theme={null}
// Lists are already synced (auto_subscribe: true)
const lists = await db.getAll('SELECT * FROM lists');
// When user opens a list
const sub = await db.syncStream('list_todos', { list_id: selectedListId }).subscribe();
await sub.waitForFirstSync();
// Todos are now available locally
const todos = await db.getAll('SELECT * FROM todos WHERE list_id = ?', [selectedListId]);
// Unsubscribe when user navigates back to the list overview
sub.unsubscribe();
```
### Chat Application
Chat apps typically have many conversations but users only view one at a time. Sync the conversation list upfront so users can see all their chats immediately, but load messages on-demand to avoid syncing potentially thousands of messages across all conversations.
```yaml theme={null}
config:
edition: 3
streams:
# User's conversations - always show the conversation list
my_conversations:
auto_subscribe: true
query: |
SELECT * FROM conversations
WHERE id IN (SELECT conversation_id FROM participants WHERE user_id = auth.user_id())
# Messages - only load for the active conversation
conversation_messages:
query: |
SELECT * FROM messages
WHERE conversation_id = subscription.parameter('conversation_id')
AND conversation_id IN (
SELECT conversation_id FROM participants WHERE user_id = auth.user_id()
)
```
Clients auto-subscribe to `my_conversations` when they connect. Subscribe to `conversation_messages` when the user opens a conversation:
```js theme={null}
// Conversations are already synced (auto_subscribe: true)
const conversations = await db.getAll('SELECT * FROM conversations');
// When user opens a conversation
const sub = await db.syncStream('conversation_messages', {
conversation_id: conversationId
}).subscribe();
await sub.waitForFirstSync();
// Unsubscribe when user closes the conversation
sub.unsubscribe();
```
### Project Management App
This example shows a multi-tenant project management app where users can access public projects or projects they're members of. Each stream that needs "accessible projects" defines a CTE in that stream (alternatively, a [global CTE](/sync/streams/ctes#global-ctes) could be used).
```yaml theme={null}
config:
edition: 3
streams:
# Organization data - always available
org_info:
auto_subscribe: true
query: SELECT * FROM organizations WHERE id = auth.parameter('org_id')
# All accessible projects - always available for navigation
projects:
auto_subscribe: true
with:
user_projects: |
SELECT id FROM projects
WHERE org_id = auth.parameter('org_id')
AND (is_public OR id IN (
SELECT project_id FROM project_members WHERE user_id = auth.user_id()
))
query: SELECT * FROM projects WHERE id IN user_projects
# Project details - on demand when user opens a project (one CTE, multiple queries)
project_details:
with:
user_projects: |
SELECT id FROM projects
WHERE org_id = auth.parameter('org_id')
AND (is_public OR id IN (
SELECT project_id FROM project_members WHERE user_id = auth.user_id()
))
queries:
- |
SELECT * FROM tasks
WHERE project_id = subscription.parameter('project_id')
AND project_id IN user_projects
- |
SELECT * FROM files
WHERE project_id = subscription.parameter('project_id')
AND project_id IN user_projects
```
Your backend should include `org_id` in the JWT — e.g. `{ "sub": "user-123", "org_id": "org-456" }`. Clients auto-subscribe to `org_info` and `projects` when they connect. Subscribe to project details when the user opens a project:
```js theme={null}
// org_info and projects are already synced (auto_subscribe: true)
const projects = await db.getAll('SELECT * FROM projects');
// When user opens a project
const sub = await db.syncStream('project_details', { project_id: projectId }).subscribe();
await sub.waitForFirstSync();
// Unsubscribe when user navigates away
sub.unsubscribe();
```
### Organization Workspace (Using Multiple Queries)
When several tables share the same access pattern, you can group them into a single stream using multiple queries and a CTE. Sync is more efficient and the client only needs to manage one subscription instead of multiple.
```yaml theme={null}
config:
edition: 3
streams:
# All org-level data syncs together in one stream
org_data:
auto_subscribe: true
with:
user_orgs: SELECT org_id FROM org_members WHERE user_id = auth.user_id()
queries:
- SELECT * FROM organizations WHERE id IN user_orgs
- SELECT * FROM projects WHERE org_id IN user_orgs
- SELECT * FROM team_members WHERE org_id IN user_orgs
# Project details - on demand. CTE includes subscription.parameter so queries stay simple.
project_details:
with:
selected_project: |
SELECT projects.id FROM projects
INNER JOIN org_members ON org_members.org_id = projects.org_id AND org_members.user_id = auth.user_id()
WHERE projects.id = subscription.parameter('project_id')
queries:
- SELECT * FROM tasks WHERE project_id IN selected_project
- SELECT * FROM files WHERE project_id IN selected_project
- SELECT * FROM comments WHERE project_id IN selected_project
```
The `user_orgs` CTE in `org_data` looks up org membership using `auth.user_id()`. In `project_details`, the CTE can include `subscription.parameter('project_id')` so it both authorizes (user must be in the project's org) and applies the selected project — the queries then just filter by `project_id IN selected_project`. Clients auto-subscribe to `org_data` when they connect. Subscribe to `project_details` when the user opens a project:
```js theme={null}
// org_data is already synced (auto_subscribe: true)
const projects = await db.getAll('SELECT * FROM projects');
// When user opens a project
const sub = await db.syncStream('project_details', { project_id: projectId }).subscribe();
await sub.waitForFirstSync();
// Unsubscribe when user navigates away
sub.unsubscribe();
```
The `project_details` stream uses a [CTE](/sync/streams/ctes) and groups tasks, files, and comments for a specific project into a single subscription.
### Convex with ID Mapping
Convex source databases need a few source-specific patterns in Sync Streams. Convex generates its own document IDs and exposes them as `_id`, but PowerSync clients need a stable local ID before a write is uploaded. The client creates a UUID in its local `id` column, your Convex mutation stores that value in a separate `uuid` field, and PowerSync syncs `uuid AS id` back to the client. See [Convex Source Database Setup](/configuration/source-db/setup#convex) for the schema-side requirements and [Type Mapping](/sync/types#convex-type-mapping) for how Convex types map to SQLite columns.
This example uses one stream for a user's lists and todos:
* `uuid` as the synced client-side `id` instead of the Convex `_id`.
* `list_uuid` as the synced relationship column instead of the Convex `list_id`.
* `CAST(an_int64_column AS INTEGER)` to sync a Convex `Int64` value as a SQLite integer.
* `substring(auth.user_id(), 1, 32)` to extract the Convex user ID from a Convex Auth JWT subject. Convex Auth subjects include the 32-character user ID followed by `|` and the user session ID.
```yaml theme={null}
config:
edition: 3
streams:
user_data:
with:
# Extract the Convex user ID from the JWT subject.
# Convex Auth subjects include `[32 character user ID]|[user session ID]`.
user_lists: |
SELECT uuid
FROM lists
WHERE archived != true
AND owner_id = substring(auth.user_id(), 1, 32)
auto_subscribe: true
queries:
- SELECT uuid AS id, name, owner_id FROM lists WHERE uuid IN user_lists
- |
SELECT
-- Use the client-generated todo UUID as the synced `id`.
uuid AS id,
description,
-- Map relationships that use Convex IDs, such as `list_id`,
-- to the related table's local UUID column.
list_uuid,
-- Cast Convex Int64 values to INTEGER when you want a SQLite integer.
CAST(an_int64_column AS INTEGER) AS an_int64_column
FROM todos
WHERE list_uuid IN user_lists
```
## Demo Apps
Working demo apps that demonstrate Sync Streams in action. These show how to combine auto-subscribe streams (for data that should always be available) with on-demand streams (for data loaded when needed).
Try the [`react-supabase-todolist-sync-streams`](https://github.com/powersync-ja/powersync-js/tree/main/demos/react-supabase-todolist-sync-streams) demo app by following the instructions in the README.
In this demo:
* The app syncs `lists` by default, so they're available immediately and offline (demonstrating auto-subscribe behavior).
* The app syncs `todos` on demand when a user opens a list (demonstrating subscription parameters).
* When the user navigates back to the same list, they won't see a loading state, because the data is cached locally (demonstrating TTL caching behavior).
Try the [`supabase-todolist`](https://github.com/powersync-ja/powersync.dart/tree/main/demos/supabase-todolist) demo app, which supports Sync Streams.
Deploy the following Sync Streams:
```yaml theme={null}
config:
edition: 3
streams:
lists:
auto_subscribe: true
query: SELECT * FROM lists WHERE owner_id = auth.user_id()
todos:
query: |
SELECT * FROM todos
WHERE list_id = subscription.parameter('list')
AND list_id IN (SELECT id FROM lists WHERE owner_id = auth.user_id())
```
In this demo:
* The app syncs `lists` by default, so they're available immediately and offline (demonstrating auto-subscribe behavior).
* The app syncs `todos` on demand when a user opens a list (demonstrating subscription parameters).
* When the user navigates back to the same list, they won't see a loading state, because the data is cached locally (demonstrating TTL caching behavior).
Sync Streams support is available. Demo app coming soon.
Sync Streams support is available. Demo app coming soon.
Sync Streams support is available. Demo app coming soon.
# Migrating from Sync Rules
Source: https://docs.powersync.com/sync/streams/migration
Migrate existing projects from legacy Sync Rules to Sync Streams.
## Why Migrate?
PowerSync's original Sync Rules system was optimized for offline-first use cases where you want to "sync everything upfront" when the client connects, so data is available locally if the user goes offline.
However, many developers are building apps where users are mostly online, and you don't want to make users wait to sync a lot of data upfront. This is especially true for **web apps**: users are mostly online, you often want to sync only the data needed for the current page, and users frequently have multiple browser tabs open — each needing different subsets of data.
### The Problem with Client Parameters
[Client Parameters](/sync/rules/client-parameters) in Sync Rules partially support on-demand syncing — for example, using a `project_ids` array to sync only specific projects. However, manually managing these arrays across different browser tabs becomes painful:
* You need to aggregate IDs across all open tabs
* You need additional logic for different data types (tables)
* If you want to keep data around after a tab closes (caching), you need even more management
### How Sync Streams Solve This
Sync Streams address these limitations:
1. **On-demand syncing**: Define streams once, then subscribe from your app one or more times with different parameters. No need to manage arrays of IDs — each subscription is independent.
2. **Multi-tab support**: Each subscription manages its own lifecycle. Open the same list in two tabs? Each tab subscribes independently. Close one? The other keeps working.
3. **Built-in caching**: Each subscription has a configurable `ttl` that keeps data cached after unsubscribing. When users return to a screen, data may already be available — no loading state needed.
4. **Simpler, more powerful syntax**: Stream queries support JOINs, CTEs, subqueries, and multiple queries per stream, and the syntax is closer to plain SQL. For example, [parameter queries become inline subqueries](#data-with-subqueries-replaces-parameter-queries), so you write a single query instead of separate `parameters:` and `data:` blocks.
5. **Framework integration**: [React hooks, Vue composables, TanStack Query, and Kotlin Compose extensions](/sync/streams/client-usage#framework-integrations) let your UI components automatically manage subscriptions based on what's rendered.
### Still Need Offline-First?
If you want "sync everything upfront" behavior (like Sync Rules), set [`auto_subscribe: true`](/sync/streams/overview#using-auto-subscribe) on your Sync Streams and clients will subscribe automatically when they connect.
## Requirements
* PowerSync Service v1.20.0+ (Cloud instances already meet this)
* Latest SDK versions with [Rust-based sync client](https://releases.powersync.com/announcements/improved-sync-performance-in-our-client-sdks) (enabled by default on latest SDKs)
* `config: edition: 3` in your Sync Config
| SDK | Minimum Version | Rust Client Default |
| ------------ | --------------- | ------------------------- |
| JS Web | v1.27.0 | v1.32.0 |
| React Native | v1.25.0 | v1.29.0 |
| React hooks | v1.8.0 | — |
| Node.js | v0.11.0 | v0.16.0 |
| Capacitor | v0.0.1 | v0.3.0 |
| Tauri | v0.0.1 | Always (Rust client only) |
| Dart/Flutter | v1.16.0 | v1.17.0 |
| Kotlin | v1.7.0 | v1.9.0 |
| Swift | v1.11.0 | v1.8.0 |
| .NET | v0.0.8-alpha.1 | v0.0.5-alpha.1 |
If you're on an SDK version below the "Rust Client Default" version, enable the Rust client manually:
**JavaScript:**
```js theme={null}
await db.connect(new MyConnector(), {
clientImplementation: SyncClientImplementation.RUST
});
```
**Dart:**
```dart theme={null}
database.connect(
connector: YourConnector(),
options: const SyncOptions(
syncImplementation: SyncClientImplementation.rust,
),
);
```
**Kotlin:**
```kotlin theme={null}
database.connect(MyConnector(), options = SyncOptions(
newClientImplementation = true,
))
```
**Swift:**
```swift theme={null}
import PowerSync
try await db.connect(connector: connector, options: ConnectOptions(
newClientImplementation: true,
))
```
## Migration Tool
You can generate a Sync Streams draft from your existing Sync Rules in two ways:
1. **Dashboard:** In the [PowerSync Dashboard](https://dashboard.powersync.com/), use the **Migrate to Sync Streams** button. It converts your Sync Rules into a Sync Streams draft that you can review before deploying.
2. **CLI:** Run `powersync migrate sync-rules` to produce a Sync Streams draft from your current Sync Config.
The output uses `auto_subscribe: true` by default, preserving your existing sync-everything-upfront behavior so no client-side changes are required when you first deploy.
**Next steps:** Review the draft, then deploy it (via the Dashboard or `powersync deploy sync-config`). After that, you can optionally migrate individual streams to on-demand subscriptions over time — remove `auto_subscribe: true` from specific streams and update client code to use the `syncStream()` API where it makes sense for your app.
## Stream Definition Reference
```yaml theme={null}
config:
edition: 3
streams:
:
# CTEs (optional) - define with block inside each stream
with:
: SELECT ... FROM ...
# Behavior options (place above query/queries)
auto_subscribe: true # Auto-subscribe clients on connect (default: false)
priority: 1 # Sync priority (optional). Lower number -> higher priority
accept_potentially_dangerous_queries: true # Silence security warnings (default: false)
# Query options (use one)
query: SELECT * FROM WHERE ... # Single query
queries: # Multiple queries
- SELECT * FROM WHERE ...
- SELECT * FROM WHERE ...
```
| Option | Default | Description |
| -------------------------------------- | ------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `query` | — | SQL-like query defining which data to sync. Use either `query` or `queries`, not both. See [Writing Queries](/sync/streams/queries). |
| `queries` | — | Array of queries defining which data to sync. More efficient than defining separate streams: the client manages one subscription and PowerSync merges the data from all queries (see [Multiple Queries per Stream](/sync/streams/queries#multiple-queries-per-stream)). |
| `with` | — | [CTEs](/sync/streams/ctes) available to this stream's queries. Define the `with` block inside each stream. |
| `auto_subscribe` | `false` | When `true`, clients automatically subscribe on connect. |
| `priority` | — | Sync priority (lower value = higher priority). See [Prioritized Sync](/sync/advanced/prioritized-sync). |
| `accept_potentially_dangerous_queries` | `false` | Silences security warnings when queries use client-controlled parameters (i.e. *connection parameters* and *subscription parameters*), as opposed to *authentication parameters* that are signed as part of the JWT. Set to `true` only if you've verified the query is safe. See [Using Parameters](/sync/streams/parameters). |
## Migration Examples
### Global Data (No Parameters)
In Sync Rules, a ["global" bucket](/sync/rules/global-buckets) syncs the same data to all users. In Sync Streams, you achieve this with queries that have no parameters. Add [`auto_subscribe: true`](/sync/streams/overview#using-auto-subscribe) to maintain the Sync Rules behavior where data syncs automatically on connect.
**Sync Rules:**
```yaml theme={null}
bucket_definitions:
global:
data:
- SELECT * FROM todos
- SELECT * FROM lists WHERE archived = false
```
**Sync Streams:**
```yaml theme={null}
config:
edition: 3
streams:
shared_data:
auto_subscribe: true # Sync automatically like Sync Rules
queries:
- SELECT * FROM todos
- SELECT * FROM lists WHERE archived = false
```
Without `auto_subscribe: true`, clients would need to explicitly subscribe to these streams. This gives you flexibility to migrate incrementally or switch to on-demand syncing later.
### User-Scoped Data
**Sync Rules:**
```yaml theme={null}
bucket_definitions:
user_lists:
priority: 1
parameters: SELECT request.user_id() as user_id
data:
- SELECT * FROM lists WHERE owner_id = bucket.user_id
```
**Sync Streams:**
```yaml theme={null}
config:
edition: 3
streams:
user_lists:
auto_subscribe: true
priority: 1
query: SELECT * FROM lists WHERE owner_id = auth.user_id()
```
### Data with Subqueries (Replaces Parameter Queries)
**Sync Rules:**
```yaml theme={null}
bucket_definitions:
owned_lists:
parameters: |
SELECT id as list_id FROM lists WHERE owner_id = request.user_id()
data:
- SELECT * FROM lists WHERE lists.id = bucket.list_id
- SELECT * FROM todos WHERE todos.list_id = bucket.list_id
```
**Sync Streams:**
```yaml theme={null}
config:
edition: 3
streams:
owned_lists:
auto_subscribe: true
query: SELECT * FROM lists WHERE owner_id = auth.user_id()
list_todos:
query: |
SELECT * FROM todos
WHERE list_id = subscription.parameter('list_id')
AND list_id IN (SELECT id FROM lists WHERE owner_id = auth.user_id())
```
### Client Parameters → Subscription Parameters
**Sync Rules** used global [Client Parameters](/sync/rules/client-parameters):
```yaml theme={null}
bucket_definitions:
posts:
parameters: SELECT (request.parameters() ->> 'current_page') as page_number
data:
- SELECT * FROM posts WHERE page_number = bucket.page_number
```
**Sync Streams** use Subscription Parameters, which are more flexible — you can subscribe multiple times with different values:
```yaml theme={null}
config:
edition: 3
streams:
posts:
query: SELECT * FROM posts WHERE page_number = subscription.parameter('page_number')
```
```js theme={null}
// Subscribe to multiple pages simultaneously
const page1 = await db.syncStream('posts', { page_number: 1 }).subscribe();
const page2 = await db.syncStream('posts', { page_number: 2 }).subscribe();
```
## Parameter Syntax Changes
| Sync Rules | Sync Streams |
| -------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `request.user_id()` | `auth.user_id()` |
| `request.jwt() ->> 'claim'` | `auth.parameter('claim')` |
| `request.parameters() ->> 'key'` | `subscription.parameter('key')` ([subscription parameter](/sync/streams/parameters#subscription-parameters)) or `connection.parameter('key')` ([connection parameter](/sync/streams/parameters#connection-parameters)) |
| `bucket.param_name` | Use the parameter directly in the query e.g. `subscription.parameter('key')` |
## Client-Side Changes
After updating your Sync Config, update your client code to use subscriptions:
```js theme={null}
// Before (Sync Rules with Client Parameters)
await db.connect(connector, {
params: { current_project: projectId }
});
// After (Sync Streams with Subscriptions)
await db.connect(connector);
const sub = await db.syncStream('project_data', { project_id: projectId }).subscribe();
```
See [Client-Side Usage](/sync/streams/client-usage) for detailed examples.
# Sync Streams
Source: https://docs.powersync.com/sync/streams/overview
Introduction to Sync Streams, the recommended way to define which data syncs to each client with SQL-based stream definitions.
With Sync Streams, you write simple SQL-like queries to define streams of data, and your client app subscribes to the streams it needs. This enables *partial sync*: each client syncs only the relevant subset of data, instead of the entire database. PowerSync handles the rest, keeping subscribed data synced in real-time to a client-side SQLite database, where it stays available even when the device is offline.
For example, you might define a stream that syncs only the current user's to-do items, another for shared projects they have access to, and another for reference data that everyone needs. Your app subscribes to these streams on demand, and only that data syncs to the device. Offline-first apps that need all relevant data available upfront can use `auto_subscribe: true` so streams sync automatically when clients connect.
**Are you still using Sync Rules?** Sync Streams support everything Sync Rules do, plus more expressive queries (including JOIN support), on-demand syncing, and a simpler developer experience (e.g. React hooks that manage subscriptions automatically).
You can migrate in a few clicks. Click **Migrate to Sync Streams** in the PowerSync Dashboard, or run `powersync migrate sync-rules` in the CLI to generate a draft from your current config. See the [migration guide](/sync/streams/migration) for details.
## Defining Streams
Streams are defined in a YAML configuration file. Each stream has a **name** and a **query** that specifies which rows to sync using SQL-like syntax. The query can reference [parameters](/sync/overview#how-it-works) like the authenticated user's ID to personalize what each user receives.
In the [PowerSync Dashboard](https://dashboard.powersync.com/):
1. Select your project and instance
2. Go to **Sync Streams**
3. Edit the YAML directly in the dashboard
4. Click **Deploy** to validate and deploy
```yaml theme={null}
config:
edition: 3
streams:
todos:
query: SELECT * FROM todos WHERE owner_id = auth.user_id()
```
Add a `sync_config` section to your `service.yaml`. Using a **separate file** is recommended (e.g. `sync_config: path: sync-config.yaml`). Put the stream definition in that file:
```yaml sync-config.yaml theme={null}
config:
edition: 3
streams:
todos:
query: SELECT * FROM todos WHERE owner_id = auth.user_id()
```
You can also use inline `sync_config: content: |` with the YAML nested in your main config. See [Self-Hosted Instance Configuration](/configuration/powersync-service/self-hosted-instances#sync_config) for both options.
Available stream options:
```yaml theme={null}
config:
edition: 3
streams:
:
# CTEs (optional) - define with block inside each stream
with:
: SELECT ... FROM ...
# Behavior options (place above query/queries)
auto_subscribe: true # Auto-subscribe clients on connect (default: false)
priority: 1 # Sync priority (optional). Lower number -> higher priority
accept_potentially_dangerous_queries: true # Silence security warnings (default: false)
# Query options (use one)
query: SELECT * FROM WHERE ... # Single query
queries: # Multiple queries
- SELECT * FROM WHERE ...
- SELECT * FROM WHERE ...
```
| Option | Default | Description |
| -------------------------------------- | ------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `query` | — | SQL-like query defining which data to sync. Use either `query` or `queries`, not both. See [Writing Queries](/sync/streams/queries). |
| `queries` | — | Array of queries defining which data to sync. More efficient than defining separate streams: the client manages one subscription and PowerSync merges the data from all queries (see [Multiple Queries per Stream](/sync/streams/queries#multiple-queries-per-stream)). |
| `with` | — | [CTEs](/sync/streams/ctes) available to this stream's queries. Define the `with` block inside each stream. |
| `auto_subscribe` | `false` | When `true`, clients automatically subscribe on connect. |
| `priority` | — | Sync priority (lower value = higher priority). See [Prioritized Sync](/sync/advanced/prioritized-sync). |
| `accept_potentially_dangerous_queries` | `false` | Silences security warnings when queries use client-controlled parameters (i.e. *connection parameters* and *subscription parameters*), as opposed to *authentication parameters* that are signed as part of the JWT. Set to `true` only if you've verified the query is safe. See [Using Parameters](/sync/streams/parameters). |
## Basic Examples
There are two independent concepts to understand:
* *What* data the stream returns. For example:
* *Global data*: No parameters. Same data for all users (e.g. reference tables like categories).
* *Filtered data*: Filters the data by a parameter value. This can make use of *auth parameters* from the JWT token (such as the user ID or other JWT claims), *subscription parameters* (specified by the client when it subscribes to a stream at any time), or *connection parameters* (specified at connection). Different users will get different sets of data based on the parameters. See [Using Parameters](/sync/streams/parameters) for the full reference.
* *When* the client syncs the data
* *Auto-subscribe*: Client automatically subscribes on connect (`auto_subscribe: true`)
* *On-demand*: Client explicitly subscribes when needed (default behavior)
### Global Data
Data without parameters is "global" data, meaning the same data goes to all users/clients. This is useful for reference tables:
```yaml theme={null}
config:
edition: 3
streams:
# Same categories for everyone
categories:
query: SELECT * FROM categories
# Same active products for everyone
products:
query: SELECT * FROM products WHERE active = true
```
Global data streams still require clients to subscribe explicitly unless you set `auto_subscribe: true`
### Filtering Data by User
Use `auth.user_id()` or other [JWT claims](/sync/streams/parameters#auth-parameters) to return different data per user:
```yaml theme={null}
config:
edition: 3
streams:
# Each user gets their own lists
my_lists:
query: SELECT * FROM lists WHERE owner_id = auth.user_id()
# Each user gets their own orders
my_orders:
query: SELECT * FROM orders WHERE user_id = auth.user_id()
```
### Filtering Data Based on Subscription Parameters
Use `subscription.parameter()` for data that clients subscribe to explicitly:
```yaml theme={null}
config:
edition: 3
streams:
# Sync todos for a specific list when the client subscribes with a list_id
list_todos:
query: |
SELECT * FROM todos
WHERE list_id = subscription.parameter('list_id')
AND list_id IN (SELECT id FROM lists WHERE owner_id = auth.user_id())
```
```js theme={null}
// Client subscribes with the list they want to view
const sub = await db.syncStream('list_todos', { list_id: 'abc123' }).subscribe();
```
### Using Auto-Subscribe
Set `auto_subscribe: true` to sync data automatically when clients connect. This is useful for:
* Reference data that all users need, or that are needed in many screens in the app.
* User data that should always be available offline
* Maintaining [Sync Rules](/sync/rules/overview) default behavior ("sync everything upfront") when migrating to Sync Streams
```yaml theme={null}
config:
edition: 3
streams:
# Global data, synced automatically
categories:
auto_subscribe: true
query: SELECT * FROM categories
# User-scoped data, synced automatically
my_orders:
auto_subscribe: true
query: SELECT * FROM orders WHERE user_id = auth.user_id()
# Parameterized data, subscribed on-demand (no auto_subscribe)
order_items:
query: |
SELECT * FROM order_items
WHERE order_id = subscription.parameter('order_id')
AND order_id IN (SELECT id FROM orders WHERE user_id = auth.user_id())
```
## Client-Side Usage
Subscribe to streams from your client app:
```js theme={null}
const sub = await db.syncStream('list_todos', { list_id: 'abc123' })
.subscribe({ ttl: 3600 });
// Wait for this subscription to have synced
await sub.waitForFirstSync();
// When the component needing the subscription is no longer active...
sub.unsubscribe();
```
**React hooks:**
```jsx theme={null}
const stream = useSyncStream({ name: 'list_todos', parameters: { list_id: 'abc123' } });
// Check download progress or subscription information
stream?.progress;
stream?.subscription.hasSynced;
```
The `useQuery` hook can wait for Sync Streams before running queries:
```jsx theme={null}
const { data } = useQuery(
'SELECT * FROM todos WHERE list_id = ?',
[listId],
{ streams: [{ name: 'list_todos', parameters: { list_id: listId }, waitForStream: true }] }
);
```
```dart theme={null}
final sub = await db
.syncStream('list_todos', {'list_id': 'abc123'})
.subscribe(ttl: const Duration(hours: 1));
// Wait for this subscription to have synced
await sub.waitForFirstSync();
// When the component needing the subscription is no longer active...
sub.unsubscribe();
```
```kotlin theme={null}
val sub = database.syncStream("list_todos", mapOf("list_id" to JsonParam.String("abc123")))
.subscribe(ttl = 1.0.hours)
// Wait for this subscription to have synced
sub.waitForFirstSync()
// When the component needing the subscription is no longer active...
sub.unsubscribe()
```
```swift theme={null}
let sub = try await db.syncStream(name: "list_todos", params: ["list_id": JsonValue.string("abc123")])
.subscribe(ttl: 60 * 60, priority: nil) // 1 hour
// Wait for this subscription to have synced
try await sub.waitForFirstSync()
// When the component needing the subscription is no longer active...
try await sub.unsubscribe()
```
```csharp theme={null}
var sub = await db.SyncStream("list_todos", new() { ["list_id"] = "abc123" })
.Subscribe(new SyncStreamSubscribeOptions { Ttl = TimeSpan.FromHours(1) });
// Wait for this subscription to have synced
await sub.WaitForFirstSync();
// When the component needing the subscription is no longer active...
sub.Unsubscribe();
```
### TTL (Time-To-Live)
Each subscription has a `ttl` that keeps data cached after unsubscribing. This enables warm cache behavior — when users return to a screen and you re-subscribe to relevant streams, data is already available on the client. Default TTL is 24 hours. See [Client-Side Usage](/sync/streams/client-usage) for details.
```js theme={null}
// Set TTL in seconds when subscribing
const sub = await db.syncStream('todos', { list_id: 'abc' })
.subscribe({ ttl: 3600 }); // Cache for 1 hour after unsubscribe
```
## Developer Notes
* **SQL Syntax**: Stream queries use a SQL-like syntax with `SELECT` statements. You can use subqueries, `INNER JOIN`, and [CTEs](/sync/streams/ctes) for filtering. `GROUP BY`, `ORDER BY`, and `LIMIT` are not supported. See [Writing Queries](/sync/streams/queries) for details on joins, multiple queries per stream, and other features.
* **Type Conversion**: Data types from your source database (Postgres, MongoDB, MySQL, SQL Server or Convex) are converted when synced to the client's SQLite database. SQLite has a limited type system, so most types become `text` and you may need to parse or cast values in your app code. See [Type Mapping](/sync/types) for details on how each type is handled.
* **Primary Key**: PowerSync requires every synced table to have a primary key column named `id` of type `text`. If your backend uses a different column name or type, you'll need to map it. For MongoDB, collections use `_id` as the ID field; you must alias it in your stream queries (e.g. `SELECT *, _id as id FROM your_collection`).
* **Case Sensitivity**: To avoid issues across different databases and platforms, use **lowercase identifiers** for all table and column names in your Sync Streams. If your backend uses mixed case, see [Case Sensitivity](/sync/advanced/case-sensitivity) for how to handle it.
* **Bucket Limits**: PowerSync uses internal partitions called [buckets](/architecture/powersync-service#bucket-system) to efficiently sync data. Each user has a limit on how many buckets they can sync ([1,000 by default](/resources/performance-and-limits)), and the query pattern determines how many each stream creates. See [Bucket Count](/sync/streams/bucket-count) for how buckets are counted, and [Reducing Bucket Count](/sync/advanced/reducing-bucket-count) to resolve `PSYNC_S2305` errors.
* **Troubleshooting**: If data isn't syncing as expected, the [Sync Diagnostics Client](/tools/diagnostics-client) helps you inspect what's happening for a specific user — you can see which buckets the user has and what data is being synced.
## Examples & Demos
See [Examples & Demos](/sync/streams/examples) for working demo apps and complete application patterns.
## Migrating from Legacy Sync Rules
If you have an existing project using legacy Sync Rules, see the [Migration Guide](/sync/streams/migration) for step-by-step instructions, syntax changes, and examples.
# Using Parameters
Source: https://docs.powersync.com/sync/streams/parameters
Filter synced data dynamically using subscription, auth, and connection parameters in Sync Streams queries.
Parameters let you filter data dynamically based on who the user is and what they need to see. Sync Streams support three types of parameters, each serving a different purpose.
## Subscription Parameters
Passed from the client when it subscribes to a stream. This is the most common way to request specific data on demand.
For example, if a user opens two different to-do lists, the client subscribes to the same `list_todos` stream twice, once for each list:
```yaml theme={null}
streams:
list_todos:
query: SELECT * FROM todos WHERE list_id = subscription.parameter('list_id')
```
```js theme={null}
// User opens List A - subscribe with list_id = 'list-a'
const subA = await db.syncStream('list_todos', { list_id: 'list-a' }).subscribe();
// User also opens List B - subscribe again with list_id = 'list-b'
const subB = await db.syncStream('list_todos', { list_id: 'list-b' }).subscribe();
// Both lists' todos are now syncing independently
```
| Function | Description |
| ------------------------------- | ------------------------------------------- |
| `subscription.parameter('key')` | Get a single parameter by name |
| `subscription.parameters()` | All parameters as JSON (for dynamic access) |
## Auth Parameters
Claims from the user's JWT token. Use these to filter data based on who the user is. These values are secure and tamper-proof since they are signed as part of the JWT by your authentication system.
```yaml theme={null}
streams:
my_lists:
query: SELECT * FROM lists WHERE owner_id = auth.user_id()
# Access custom JWT claims
org_data:
query: SELECT * FROM projects WHERE org_id = auth.parameter('org_id')
```
| Function | Description |
| ----------------------- | ----------------------------------------------- |
| `auth.user_id()` | The user's ID (same as `auth.parameter('sub')`) |
| `auth.parameter('key')` | Get a specific JWT claim |
| `auth.parameters()` | Full JWT payload as JSON |
## Connection Parameters
Specified "globally" at the connection level, before any streams are subscribed. These are the equivalent of [Client Parameters](/sync/rules/client-parameters) in Sync Rules. Use them when you need a value that applies across all streams for the session.
```yaml theme={null}
streams:
app_config:
query: SELECT * FROM config WHERE environment = connection.parameter('environment')
```
| Function | Description |
| ----------------------------- | --------------------------------- |
| `connection.parameter('key')` | Get a single connection parameter |
| `connection.parameters()` | All connection parameters as JSON |
Changing connection parameters requires reconnecting. For values that change during a session, use subscription parameters instead.
See [Client Usage](/sync/streams/client-usage#connection-parameters) for details on specifying connection parameters in your client-side code.
## When to Use Each
**Subscription parameters** are the most flexible option. Use them when the client needs to choose what data to sync at runtime. Each subscription operates independently, so a user can have multiple subscriptions to the same stream with different parameters.
**Auth parameters** are the most secure option. Use them when you need to filter data based on who the user is. Since these values come from the signed JWT, they can't be tampered with by the client.
**Connection parameters** apply globally across all streams for the session. Use them for values that rarely change, like environment flags or feature toggles. Keep in mind that changing them requires reconnecting.
For most use cases, subscription parameters are the best choice. They're more flexible and work well with modern app patterns like multiple tabs.
## Expanding JSON Arrays
If a user's JWT contains an array of IDs (e.g., `{ "project_ids": ["proj-1", "proj-2", "proj-3"] }`), you can expand it to sync all matching records. The example below syncs all three projects to the user/client:
**Shorthand syntax** (recommended):
```yaml theme={null}
streams:
# User's JWT contains: { "project_ids": ["proj-1", "proj-2", "proj-3"] }
my_projects:
auto_subscribe: true
query: SELECT * FROM projects WHERE id IN auth.parameter('project_ids')
```
**JOIN syntax** with table-valued function:
```yaml theme={null}
streams:
my_projects:
auto_subscribe: true
query: |
SELECT * FROM projects
JOIN json_each(auth.parameter('project_ids')) AS allowed ON projects.id = allowed.value
```
**Subquery syntax**:
```yaml theme={null}
streams:
my_projects:
auto_subscribe: true
query: |
SELECT * FROM projects
WHERE id IN (SELECT value FROM json_each(auth.parameter('project_ids')))
```
All three sync the same data: projects whose IDs are in the user's JWT `project_ids` claim.
`json_each()` works with auth and connection parameters. It can also be used with columns from joined tables in some cases (e.g. `SELECT * FROM lists WHERE id IN (SELECT lists.value FROM access_control a, json_each(a.allowed_lists) as lists WHERE a.user = auth.user_id())`).
## Combining Parameters
You can combine different parameter types in a single query. A common pattern is using subscription parameters for on-demand data while using auth parameters for authorization:
```yaml theme={null}
streams:
# User subscribes with a list_id, but can only see lists they have access to
list_items:
query: |
SELECT * FROM items
WHERE list_id = subscription.parameter('list_id')
AND list_id IN (
SELECT id FROM lists
WHERE owner_id = auth.user_id()
OR id IN (SELECT list_id FROM list_shares WHERE shared_with = auth.user_id())
)
```
See [Writing Queries](/sync/streams/queries) for more filtering techniques using subqueries and joins.
# Writing Queries
Source: https://docs.powersync.com/sync/streams/queries
Learn Sync Streams query syntax for filtering, column selection, and type transformations.
This page covers query syntax for Sync Streams: filtering, selecting columns, and transforming data.
For parameter usage, see [Using Parameters](/sync/streams/parameters). For real-world patterns, see [Examples, Patterns & Demos](/sync/streams/examples).
## Basic Queries
The simplest stream query syncs all rows from a table:
```yaml theme={null}
streams:
auto_subscribe: true
categories:
query: SELECT * FROM categories
```
Add a `WHERE` clause to filter:
```yaml theme={null}
streams:
auto_subscribe: true
active_products:
query: SELECT * FROM products WHERE active = true
```
## Filtering by User
Most apps need to sync different data to different users. Use `auth.user_id()` to filter by the authenticated user:
```yaml theme={null}
streams:
auto_subscribe: true
my_lists:
query: SELECT * FROM lists WHERE owner_id = auth.user_id()
```
This syncs only the lists owned by the current user. The user ID comes from the `sub` claim in their JWT token. See [Auth Parameters](/sync/streams/parameters#auth-parameters).
## On-Demand Data with Subscription Parameters
For data that should only sync when the user navigates to a specific screen, use subscription parameters. The client passes these when subscribing to a stream:
```yaml theme={null}
streams:
list_todos:
query: SELECT * FROM todos WHERE list_id = subscription.parameter('list_id')
```
**Authorization:** This example filters only by `subscription.parameter('list_id')`. Any client can pass any `list_id`, so a user could access another user's todos. For production, add an authorization check so the user can only see lists they own or have access to — for example, add `AND list_id IN (SELECT id FROM lists WHERE owner_id = auth.user_id() OR id IN (SELECT list_id FROM list_shares WHERE shared_with = auth.user_id()))`. See [Combining Parameters with Subqueries](#combining-parameters-with-subqueries) below.
```js theme={null}
// When user opens a specific list, subscribe with that list's ID
const sub = await db.syncStream('list_todos', { list_id: 'abc123' }).subscribe();
```
See [Using Parameters](/sync/streams/parameters) for the full reference on parameters.
## Selecting Columns
Select specific columns instead of `*` to reduce data transfer:
```yaml theme={null}
streams:
users:
query: SELECT id, name, email, avatar_url FROM users WHERE org_id = auth.parameter('org_id')
```
### Renaming Columns
Use `AS` to rename columns in the synced data:
```yaml theme={null}
streams:
todos:
query: SELECT id, name, created_timestamp AS created_at FROM todos
```
### Type Transformations
PowerSync syncs data to SQLite on the client. You may need to transform types for compatibility:
```yaml theme={null}
streams:
items:
query: |
SELECT
id,
CAST(item_number AS TEXT) AS item_number, -- Cast to text
metadata_json ->> 'description' AS description, -- Extract field from JSON
base64(thumbnail) AS thumbnail_base64, -- Binary to base64
unixepoch(created_at) AS created_at -- DateTime to epoch
FROM items
```
See [Type Mapping](/sync/types) for details on how each database type is handled.
## Using Subqueries
Subqueries let you filter based on related tables. Use `IN (SELECT ...)` to sync data where a foreign key matches rows in another table:
```yaml theme={null}
streams:
# Sync comments for issues owned by the current user
my_issue_comments:
query: |
SELECT * FROM comments
WHERE issue_id IN (SELECT id FROM issues WHERE owner_id = auth.user_id())
```
### Nested Subqueries
Subqueries can be nested to traverse multiple levels of relationships. This is useful for normalized database schemas:
```yaml theme={null}
streams:
# Sync tasks for projects in organizations the user belongs to
org_tasks:
query: |
SELECT * FROM tasks
WHERE project_id IN (
SELECT id FROM projects WHERE org_id IN (
SELECT org_id FROM org_members WHERE user_id = auth.user_id()
)
)
```
### Combining Parameters with Subqueries
A common pattern is using subscription parameters to select what data to sync, while using subqueries for authorization:
```yaml theme={null}
streams:
# User subscribes with a list_id, but can only see lists they own or that are shared with them
list_items:
query: |
SELECT * FROM items
WHERE list_id = subscription.parameter('list_id')
AND list_id IN (
SELECT id FROM lists
WHERE owner_id = auth.user_id()
OR id IN (SELECT list_id FROM list_shares WHERE shared_with = auth.user_id())
)
```
## Using Joins
For complex queries that traverse multiple tables, join syntax is often easier to read than nested subqueries. You can use `JOIN` or `INNER JOIN` (they're equivalent). For the exact supported JOIN syntax and restrictions, see [Supported SQL — JOIN syntax](/sync/supported-sql#join-syntax).
Consider this query:
```yaml theme={null}
streams:
# Nested subquery version
user_comments:
query: |
SELECT * FROM comments WHERE issue_id IN (
SELECT id FROM issues WHERE project_id IN (
SELECT project_id FROM project_members WHERE user_id = auth.user_id()
)
)
```
The same query using joins:
```yaml theme={null}
streams:
# Join version - same result, easier to read
user_comments:
query: |
SELECT comments.* FROM comments
INNER JOIN issues ON comments.issue_id = issues.id
INNER JOIN project_members ON issues.project_id = project_members.project_id
WHERE project_members.user_id = auth.user_id()
```
Both queries sync the same data. Choose whichever style is clearer for your use case.
### Multiple Joins
You can chain multiple joins to traverse complex relationships. This example joins four tables to sync checkpoints for assignments the user has access to.
```yaml theme={null}
streams:
my_checkpoints:
query: |
SELECT checkpoint.* FROM user_assignment_scope uas
JOIN assignment a ON a.id = uas.assignment_id
JOIN assignment_checkpoint ac ON ac.assignment_id = a.id
JOIN checkpoint ON checkpoint.id = ac.checkpoint_id
WHERE uas.user_id = auth.user_id()
AND a.active = true
```
### Self-Joins
You can join the same table multiple times; aliases are required to distinguish each occurrence (e.g. `gm1` and `gm2` for the two `group_memberships` joins). This is useful for finding related records through a shared relationship — for example, finding all users who share a group with the current user:
```yaml theme={null}
streams:
users_in_my_groups:
query: |
SELECT users.* FROM users
JOIN group_memberships gm1 ON users.id = gm1.user_id
JOIN group_memberships gm2 ON gm1.group_id = gm2.group_id
WHERE gm2.user_id = auth.user_id()
```
### Join Limitations
When writing stream queries with JOINs, keep in mind: use only `JOIN` or `INNER JOIN`; select columns from a single table (e.g. `comments.*`); and use simple equality conditions (`table1.column = table2.column`). For the full list of supported JOIN syntax and invalid examples, see [Supported SQL — JOIN syntax](/sync/supported-sql#join-syntax).
## Multiple Queries per Stream
You can group multiple queries into a single stream using `queries` instead of `query`. This is useful when several tables share the same access pattern:
```yaml theme={null}
streams:
user_data:
auto_subscribe: true
queries:
- SELECT * FROM notes WHERE owner_id = auth.user_id()
- SELECT * FROM settings WHERE user_id = auth.user_id()
- SELECT * FROM preferences WHERE user_id = auth.user_id()
```
You subscribe once to the stream; PowerSync merges the data from all queries efficiently. This is more efficient than defining separate streams, each requiring its own subscription.
### When to Use Multiple Queries
Use `queries` when:
* Multiple tables have the same filtering logic (e.g., all filtered by `user_id`)
* You want to optimize sync by using one stream so the client subscribes once and PowerSync merges the data from all queries, and to reduce bucket count (see Bucket Count and Limits )
* Related data should sync together
```yaml theme={null}
streams:
# All project-related data syncs together
project_details:
queries:
- SELECT * FROM tasks WHERE project_id = subscription.parameter('project_id')
- SELECT * FROM files WHERE project_id = subscription.parameter('project_id')
- SELECT * FROM comments WHERE project_id = subscription.parameter('project_id')
```
### Compatibility Requirements
For multiple queries in one stream to be valid, they must use compatible parameter inputs. In practice, this means they should filter on the same parameters in the same way:
```yaml theme={null}
# Valid - all queries use the same parameter pattern
streams:
user_content:
queries:
- SELECT * FROM notes WHERE user_id = auth.user_id()
- SELECT * FROM bookmarks WHERE user_id = auth.user_id()
# Valid - all queries use the same subscription parameter
streams:
project_data:
queries:
- SELECT * FROM tasks WHERE project_id = subscription.parameter('project_id')
- SELECT * FROM files WHERE project_id = subscription.parameter('project_id')
```
### Combining with CTEs
Multiple queries work well with [Common Table Expressions (CTEs)](/sync/streams/ctes) to share the filtering logic and keep all results in one stream, requiring clients to manage one subscription instead of many:
```yaml theme={null}
streams:
org_data:
auto_subscribe: true
with:
user_orgs: SELECT org_id FROM org_members WHERE user_id = auth.user_id()
queries:
- SELECT * FROM projects WHERE org_id IN user_orgs
- SELECT * FROM repositories WHERE org_id IN user_orgs
- SELECT * FROM team_members WHERE org_id IN user_orgs
```
## Complete Example
A full configuration combining multiple techniques:
```yaml theme={null}
config:
edition: 3
streams:
# Global reference data (no parameters, auto-subscribed)
categories:
auto_subscribe: true
query: SELECT id, name, CAST(sort_order AS TEXT) AS sort_order FROM categories
# User's own items with transformed fields (auth parameter, auto-subscribed)
my_items:
auto_subscribe: true
query: |
SELECT
id,
name,
metadata ->> 'status' AS status,
unixepoch(created_at) AS created_at,
base64(thumbnail) AS thumbnail
FROM items
WHERE owner_id = auth.user_id()
# On-demand item details (subscription parameter with auth check)
item_comments:
query: |
SELECT * FROM comments
WHERE item_id = subscription.parameter('item_id')
AND item_id IN (SELECT id FROM items WHERE owner_id = auth.user_id())
```
See [Examples & Patterns](/sync/streams/examples) for real-world examples like multi-tenant apps and role-based access, and [Supported SQL](/sync/supported-sql) for all available operators and functions.
# Supported SQL
Source: https://docs.powersync.com/sync/supported-sql
Reference for SQL syntax, operators, built-in functions, and type casting supported in Sync Streams/Sync Rules queries.
This guide explains the SQL supported in [Sync Streams](/sync/streams/overview) and [Sync Rules (legacy)](/sync/rules/overview): what you can write, with examples and restrictions.
For the exact syntax the compiler accepts — railroad diagrams and grammar-rule references — see the [Sync Streams](/sync/grammar/sync-streams/index) or [Sync Rules](/sync/grammar/sync-rules/index) grammar reference.
Some fundamental restrictions on the usage of SQL expressions are:
1. They must be deterministic — no random or time-based functions.
2. No external state can be used.
3. They must operate on data available within a single row/document. For example, no aggregation functions are allowed.
For parameter-specific WHERE restrictions, see [Filtering: WHERE Clause](#filtering-where-clause).
## Query Syntax
The supported SQL is based on a subset of the standard SQL syntax. Sync Streams support more SQL features than the legacy Sync Rules.
* `SELECT` with column selection and [`WHERE` filtering](#filtering-where-clause)
* [Subqueries](/sync/streams/queries#using-subqueries) with `IN (SELECT ...)` and nested subqueries
* [`INNER JOIN`](#join-syntax) (selected columns must come from a single table)
* [Common Table Expressions (CTEs)](#cte-and-with-syntax) via the `with:` block
* Multiple queries per stream via `queries:`
* Table-valued functions such as `json_each()` for [expanding arrays](/sync/streams/parameters#expanding-json-arrays)
* `BETWEEN` and `CASE` expressions
* A limited set of [operators](#operators) and [functions](#functions)
**Not supported**: aggregation, sorting, or set operations (`GROUP BY`, `ORDER BY`, `LIMIT`, `UNION`, etc.). See [Writing Queries](/sync/streams/queries) for details.
* Simple `SELECT` with column selection
* `WHERE` filtering on parameters (see [Filtering: WHERE Clause](#filtering-where-clause))
* A limited set of [operators](#operators) and [functions](#functions)
**Not supported**: subqueries, JOINs, CTEs, aggregation, sorting, or set operations (`GROUP BY`, `ORDER BY`, `LIMIT`, `UNION`, etc.).
## Filtering: WHERE Clause
Sync queries support a subset of SQL `WHERE` syntax. Allowed operators and combinations differ between Sync Streams and Sync Rules, and are more restrictive than standard SQL.
**`=` and `IS NULL`** — Compare a row column to a static value, a parameter, or another column:
```sql theme={null}
-- Static value
WHERE status = 'active'
WHERE deleted_at IS NULL
-- Parameter (auth, connection, or subscription)
WHERE owner_id = auth.user_id()
WHERE region = connection.parameter('region')
```
**`AND`** — Fully supported. You can mix parameter comparisons, subqueries, and row-value conditions in the same clause.
```sql theme={null}
-- Two parameter conditions
WHERE owner_id = auth.user_id()
AND org_id = auth.parameter('org_id')
-- Parameter condition + row-value condition
WHERE owner_id = auth.user_id()
AND status = 'active'
-- Parameter condition + subquery
WHERE list_id = subscription.parameter('list_id')
AND list_id IN (SELECT id FROM lists WHERE owner_id = auth.user_id())
```
**`OR`** — Supported, including `OR` nested inside `AND`. PowerSync rewrites combinations like `A AND (B OR C)` into separate branches before evaluating. Each `OR` branch must be a valid filter on its own; you cannot have a branch that only makes sense when combined with the other.
```sql theme={null}
-- Top-level OR
WHERE owner_id = auth.user_id()
OR shared_with = auth.user_id()
-- OR nested inside AND
WHERE status = 'active'
AND (owner_id = auth.user_id() OR shared_with = auth.user_id())
```
**`NOT`** — Supported for simple conditions on row values. `NOT IN` with a literal set of values is supported: use a JSON array string (e.g. `'["draft", "hidden"]'`), or the `ARRAY['draft', 'hidden']` and `ROW('draft', 'hidden')` forms. You cannot negate a subquery or a parameter array expansion.
```sql theme={null}
-- Simple row-value conditions
WHERE status != 'archived'
WHERE deleted_at IS NOT NULL
-- NOT IN with JSON array string (any of these forms)
WHERE category NOT IN '["draft", "hidden"]'
WHERE category NOT IN ARRAY['draft', 'hidden']
WHERE category NOT IN ROW('draft', 'hidden')
-- Not supported: negating a subquery
-- WHERE issue_id NOT IN (SELECT id FROM issues WHERE owner_id = auth.user_id())
-- Not supported: negating a parameter array
-- WHERE id NOT IN subscription.parameter('excluded_ids')
```
**`=` and `IS NULL`** — Compare a row column to a static value or a bucket parameter:
```sql theme={null}
-- Static value
WHERE status = 'active'
WHERE deleted_at IS NULL
-- Bucket parameter
WHERE owner_id = bucket.user_id
```
**`AND`** — Supported in both Parameter Queries and Data Queries. In Parameter Queries, each condition may match a different parameter. However, you cannot combine two `IN` expressions on parameters in the same `AND`; split them into separate Parameter Queries instead.
```sql theme={null}
-- Supported: parameter condition + row-value condition
WHERE users.id = request.user_id()
AND users.is_admin = true
-- Not supported: two IN expressions on parameters in the same AND
-- WHERE bucket.list_id IN lists.allowed_ids
-- AND bucket.org_id IN lists.allowed_org_ids
```
**`OR`** — Supported when both sides of the `OR` reference the exact same set of parameters. If the two sides use different parameters, use separate parameter queries instead.
```sql theme={null}
-- Supported: both sides reference the same parameter
WHERE lists.owner_id = request.user_id()
OR lists.shared_with = request.user_id()
-- Not supported: sides reference different parameters
-- WHERE lists.owner_id = request.user_id()
-- OR lists.org_id = bucket.org_id
```
**`NOT`** — Supported for simple row-value conditions. Not supported on parameter-matching expressions.
```sql theme={null}
-- Supported
WHERE status != 'archived'
WHERE deleted_at IS NOT NULL
WHERE NOT users.is_admin = true
-- Not supported in parameter queries
-- WHERE NOT users.id = request.user_id()
```
## Operators
Operators can be used in `WHERE` clauses and in `SELECT` expressions. When filtering on parameters (e.g. `auth.user_id()`, `subscription.parameter('id')`), some combinations are restricted — see [Filtering: WHERE Clause](#filtering-where-clause).
* **Comparison:** `=`, `!=`, `<`, `>`, `<=`, `>=` — If either side is `null`, the result is `null`.
* **Null:** `IS NULL`, `IS NOT NULL`
* **Logical:** `AND`, `OR`, `NOT` — See [Filtering: WHERE Clause](#filtering-where-clause) for restrictions when filtering on parameters.
* **Mathematical:** `+`, `-`, `*`, `/`
* `||` — Joins two text values together.
* `json -> 'path'` - Returns the value as a JSON string.
* `json ->> 'path'` — Returns the extracted value.
* **Sync Streams:** `left IN right` — `left` can be a row column and `right` a parameter array (e.g. `id IN subscription.parameter('ids')`), or `left` a parameter and `right` a row JSON array column. Also supports subqueries: `id IN (SELECT ...)`.
* **Sync Rules:** Returns true if `left` is in the `right` JSON array. In Data Queries, `left` must be a row column and `right` cannot be a bucket parameter. In Parameter Queries, either side may be a parameter.
* `x BETWEEN a AND b`, `x NOT BETWEEN a AND b` — True if `x` is in the inclusive range `[a, b]`. Usable in `WHERE` or as a `SELECT` expression. If any operand is `null`, the result is `null`.
Example: `WHERE price BETWEEN 10 AND 100`
Supported in Sync Streams only. Not available in Sync Rules.
* ` && ` — True if the JSON array in `left` and the set `right` share at least one value. Use when the row stores an array (e.g. a `tagged_users` column). `left` must be a row column (JSON array); `right` must be a subquery or parameter array.
Example: `WHERE tagged_users && (SELECT id FROM org_members WHERE org_id = auth.parameter('org_id'))`
Use `IN` when the row has a single value to check against a set; use `&&` when the row has an array and you want to match any element.
Supported in Sync Streams only. Not available in Sync Rules.
## Functions
Functions can be used to transform columns/fields before being synced to a client. They operate on row data or parameters. Type names below (`text`, `integer`, `real`, `blob`, `null`) refer to [SQLite storage classes](https://www.sqlite.org/datatype3.html).
Most functions are from [SQLite built-in functions](https://www.sqlite.org/lang_corefunc.html) and [SQLite JSON functions](https://www.sqlite.org/json1.html).
* **[upper(text)](https://www.sqlite.org/lang_corefunc.html#upper)** — Convert text to upper case.
* **[lower(text)](https://www.sqlite.org/lang_corefunc.html#lower)** — Convert text to lower case.
* **[substring(text, start, length)](https://www.sqlite.org/lang_corefunc.html#substr)** — Extracts a portion of a string based on specified start index and length. Start index is 1-based. Example: `substring(created_at, 1, 10)` returns the date portion of the timestamp.
* **[instr(string, substring)](https://www.sqlite.org/lang_corefunc.html#instr)** — Finds the first occurrence of the substring within the string and returns the number of prior characters plus 1, or 0 if the substring is not found. Useful for locating a delimiter in compound strings. For example, `substring(value, 1, instr(value, '|') - 1)` extracts the portion before a `|` character.
* **[hex(data)](https://www.sqlite.org/lang_corefunc.html#hex)** — Convert blob or text data to hexadecimal text.
* **base64(data)** — Convert blob or text data to base64 text.
* **[length(data)](https://www.sqlite.org/lang_corefunc.html#length)** — For text, return the number of characters. For blob, return the number of bytes. For null, return null. For integer and real, convert to text and return the number of characters.
* `CAST(x AS type)` or `x :: type` — Cast to `text`, `numeric`, `integer`, `real`, or `blob`. See [Type mapping](/sync/types) and [SQLite types](https://www.sqlite.org/datatype3.html).
* **[typeof(data)](https://www.sqlite.org/lang_corefunc.html#typeof)** — Returns `text`, `integer`, `real`, `blob`, or `null`.
* **[json\_each(data)](https://www.sqlite.org/json1.html#jeach)** — Expands a JSON array into rows.
* **Sync Streams:** Works with auth and connection parameters (e.g. `JOIN json_each(auth.parameter('ids')) AS t` or `WHERE id IN (SELECT value FROM json_each(auth.parameter('ids')))`). Can also be used with columns from joined tables in some cases (e.g. `SELECT * FROM lists WHERE id IN (SELECT lists.value FROM access_control a, json_each(a.allowed_lists) as lists WHERE a.user = auth.user_id())`). See [Expanding JSON arrays](/sync/streams/parameters#expanding-json-arrays).
* **Sync Rules:** Expands a JSON array or object from a request or token parameter into a set of parameter rows. Example: `SELECT value AS project_id FROM json_each(request.jwt() -> 'project_ids')`.
* **[json\_extract(data, path)](https://www.sqlite.org/json1.html#jex)** — Same as `->>` operator, but the path must start with `$.`
* **[json\_array\_length(data)](https://www.sqlite.org/json1.html#jarraylen)** — Given a JSON array (as text), returns the length of the array. If data is null, returns null. If the value is not a JSON array, returns 0.
* **[json\_valid(data)](https://www.sqlite.org/json1.html#jvalid)** — Returns 1 if the data can be parsed as JSON, 0 otherwise.
* **json\_keys(data)** — Returns the set of keys of a JSON object as a JSON array. Example: `SELECT * FROM items WHERE bucket.user_id IN json_keys(permissions_json)`.
* **[ifnull(x, y)](https://www.sqlite.org/lang_corefunc.html#ifnull)** — Returns x if non-null, otherwise returns y.
* **[iif(x, y, z)](https://www.sqlite.org/lang_corefunc.html#iif)** — Returns y if x is true, otherwise returns z.
* **[unixepoch(time-value, \[modifier\])](https://www.sqlite.org/lang_datefunc.html)** — Returns a time-value as Unix timestamp. If modifier is "subsec", the result is a floating point number, with milliseconds included in the fraction. The time-value argument is required — this function cannot be used to get the current time.
* **[datetime(time-value, \[modifier\])](https://www.sqlite.org/lang_datefunc.html)** — Returns a time-value as a date and time string, in the format YYYY-MM-DD HH:MM:SS. If the specifier is "subsec", milliseconds are also included. If the modifier is "unixepoch", the argument is interpreted as a Unix timestamp. Both modifiers can be included: `datetime(timestamp, 'unixepoch', 'subsec')`. The time-value argument is required — this function cannot be used to get the current time.
* **[uuid\_blob(id)](https://sqlite.org/src/file/ext/misc/uuid.c)** — Convert a UUID string to bytes.
These functions return metadata about the table a row comes from, rather than operating on a value. Prefix each call with the name or alias of a table in the query's `FROM` clause, the same way you would qualify a column reference: with `FROM "todos_%" AS todos`, write `todos.table_suffix()`. The functions take no arguments, and can appear in `WHERE` clauses, in selected columns, and inside subqueries.
* **schema()** — Returns the schema the row was replicated from. Combined with a [wildcard schema](/sync/advanced/schemas-and-connections#wildcard-schemas-postgres), this allows filtering rows by their source schema, e.g. `SELECT * FROM "%".work_orders WHERE work_orders.schema() = auth.parameter('tenant_schema')`.
* **table\_name()** — Returns the name of the table the row was replicated from. This is the source table's name, not the alias or the output table name.
* **table\_suffix()** — Returns the part of the table name matched by the trailing `%` of a [wildcard table name](/sync/advanced/partitioned-tables). For example, with `FROM "todos_%" AS todos`, `todos.table_suffix()` returns `2024` for rows from the `todos_2024` table. On tables without a wildcard name, the result is always empty and the compiler reports a warning.
Supported in Sync Streams only, with PowerSync Service v1.24.0 or later.
* **[ST\_AsGeoJSON(geometry)](/client-sdks/advanced/gis-data-postgis)** — Convert [PostGIS](/client-sdks/advanced/gis-data-postgis) (in Postgres) geometry from WKB to GeoJSON. Combine with JSON operators to extract specific fields.
* **[ST\_AsText(geometry)](/client-sdks/advanced/gis-data-postgis)** — Convert [PostGIS](/client-sdks/advanced/gis-data-postgis) (in Postgres) geometry from WKB to Well-Known Text (WKT).
* **[ST\_X(point)](/client-sdks/advanced/gis-data-postgis)** — Get the X coordinate of a [PostGIS](/client-sdks/advanced/gis-data-postgis) point (in Postgres).
* **[ST\_Y(point)](/client-sdks/advanced/gis-data-postgis)** — Get the Y coordinate of a [PostGIS](/client-sdks/advanced/gis-data-postgis) point (in Postgres).
If you need an operator or function not listed, [contact us](/resources/contact-us) so we can consider adding it.
## JOIN Syntax
Supported in Sync Streams only. Not available in Sync Rules.
Sync Streams support a subset of join syntax. The following rules define what is valid:
* **Only inner joins:** Use `JOIN` or `INNER JOIN`. `LEFT`, `RIGHT`, and `OUTER` joins are not supported.
* **Single output table:** All selected columns must come from one table. Use `table.*` or list columns from that table (e.g. `comments.*`, `comments.id`). Selecting columns from multiple tables is invalid.
* **Simple join conditions:** Join conditions must be equality comparisons of the form `table1.column = table2.column`. Other comparisons (e.g. `a.x > b.y`) are not supported.
* **Table-valued functions in JOINs:** `json_each()` and similar functions work with auth or connection parameters (e.g. `json_each(auth.parameter('ids'))`). They can also be used with columns from joined tables in some cases.
```sql theme={null}
-- Valid: columns from one table
SELECT comments.* FROM comments INNER JOIN issues ON comments.issue_id = issues.id
-- Valid: multiple equality conditions (composite keys)
SELECT comments.* FROM comments, issues
WHERE comments.issue_id = issues.id
AND comments.region = issues.region
AND issues.user_id = auth.user_id()
-- Invalid: columns from multiple tables
SELECT comments.*, issues.title FROM comments JOIN issues ON comments.issue_id = issues.id
-- Invalid: non-equality join condition
SELECT * FROM a JOIN b ON a.x > b.y
```
For how to use JOINs in your stream queries (when to use them, patterns, and examples), see [Using Joins](/sync/streams/queries#using-joins).
## CTE and WITH Syntax
Supported in Sync Streams only. Not available in Sync Rules.
Common Table Expressions (CTEs) can be defined in a `with:` block **inside a stream** (stream-level, scoped to that stream) or at the **top level** of the Sync Config (global, shared across all streams). Each CTE is a name and a single `SELECT` query. The following rules apply:
* **Stream-level CTEs take precedence over global CTEs.** If a stream defines a CTE with the same name as a global CTE, the stream-level definition is used within that stream.
* **Global CTE names must not shadow source table names.** If a global CTE has the same name as a database table or collection, PowerSync reports a validation error. Stream-level CTE names are not subject to this restriction.
* **Global CTEs require `config: edition: 3`.**
* **CTEs cannot reference other CTEs.** Each CTE must be self-contained. To chain logic (e.g. orgs → projects), use nested subqueries in your stream query and reference only the CTE at the leaf level.
* **Stream-level CTE names take precedence over table names** within that stream. Use distinct names to avoid confusion.
* **Short-hand `IN cte_name`** works only when the CTE has exactly one column.
```yaml theme={null}
config:
edition: 3
# Global CTE — available to all streams
with:
user_orgs: SELECT org_id FROM org_members WHERE user_id = auth.user_id()
streams:
# Stream-level CTE — scoped to this stream only; global CTE available here too
org_data:
with:
active_projects: SELECT id FROM projects WHERE archived = false
queries:
- SELECT * FROM projects WHERE org_id IN user_orgs AND id IN active_projects
- SELECT * FROM repositories WHERE org_id IN user_orgs
- SELECT * FROM team_members WHERE org_id IN user_orgs
# Different stream - still uses the same global CTE
audit_logs:
auto_subscribe: true
query: SELECT * FROM audit_log WHERE org_id IN user_orgs
# Invalid: CTE referencing another CTE
# streams:
# my_stream:
# with:
# user_orgs: SELECT org_id FROM org_members WHERE user_id = auth.user_id()
# project_ids: SELECT id FROM projects WHERE org_id IN user_orgs # Error
```
For how to use CTEs, see [Common Table Expressions (CTEs)](/sync/streams/ctes).
## CASE Expressions
Supported in Sync Streams only. Not available in Sync Rules.
`CASE` is allowed anywhere an expression is allowed — in `SELECT` columns or `WHERE` clauses.
**Searched CASE** — Each `WHEN` is an independent boolean condition:
```sql theme={null}
CASE
WHEN THEN
WHEN THEN
ELSE
END
```
```sql theme={null}
-- Compute a label based on a column value
SELECT id,
CASE
WHEN score >= 90 THEN 'A'
WHEN score >= 70 THEN 'B'
ELSE 'C'
END AS grade
FROM results
```
**Simple CASE** — Compares one expression against a list of values:
```sql theme={null}
CASE
WHEN THEN
WHEN THEN
ELSE
END
```
```sql theme={null}
-- Map numeric status codes to readable labels
SELECT id,
CASE status
WHEN 1 THEN 'pending'
WHEN 2 THEN 'active'
WHEN 3 THEN 'closed'
ELSE 'unknown'
END AS status_label
FROM tasks
```
`ELSE` is optional. If omitted and no `WHEN` matches, the result is `null`.
# Types
Source: https://docs.powersync.com/sync/types
How Postgres, MongoDB, MySQL, SQL Server and Convex types map to PowerSync's SQLite-based sync column definitions.
The supported client-side SQLite types are:
1. `null`
2. `integer`: a 64-bit signed integer
3. `real`: a 64-bit floating point number
4. `text`: A UTF-8 text string
5. `blob`: Binary data
##
Binary data can be accessed in the Sync Streams / Sync Rules, but cannot be used as [parameters](/sync/overview#how-it-works). To sync binary columns/fields to clients, those columns need to be converted to hex or base64 representation using the relevant [functions](/sync/supported-sql#functions).
##
The Convex replicator is currently released as an [experimental feature](/resources/feature-status). APIs and
behavior may change, and we can't yet guarantee continued support or long-term stability.
Convex values are mapped to SQLite types as follows:
| Convex Type | TS/JS Type | PowerSync / SQLite Column Type | Notes |
| ----------- | ---------------- | ------------------------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `Id` | `string` | `text` | Convex document IDs are exposed as `_id` and can be synced as `text`. For synced client tables, use client-side ID mapping with a stable UUID column as `id` instead of relying on Convex-generated `_id` values. |
| `Null` | `null` | `null` | |
| `Int64` | `base-10 string` | `text` | Cast to `INTEGER` in Sync Streams when you want to sync the value as a SQLite integer. |
| `Float64` | `number` | `real` | |
| `Boolean` | `boolean` | `integer` | `1` for true, `0` for false. There is no dedicated boolean data type in SQLite. |
| `String` | `string` | `text` | |
| `Bytes` | `base64 string` | `text` | Decode from base64 in your app if you need binary data on the client. |
| `Array` | `Array` | `text` | Converted to a JSON string. |
| `Object` | `Object` | `text` | Converted to a JSON string. |
| `Record` | `Record` | `text` | Converted to a JSON string. |
* Convex documents are converted to a flat list of columns, one column per top-level field.
* Nested objects and arrays are converted to JSON, and [JSON functions and operators](/sync/supported-sql#operators) can be used to query them in Sync Streams or on the client-side SQLite database.
* Cast Convex `Int64` fields to `INTEGER` in Sync Streams when you want SQLite integer values on the client, for example `CAST(an_int64_column AS INTEGER) AS an_int64_column`.
##
Binary data can be accessed in the Sync Streams / Sync Rules, but cannot be used as [parameters](/sync/overview#how-it-works). To sync binary columns/fields to clients, those columns need to be converted to hex or base64 representation using the relevant [functions](/sync/supported-sql#functions).
## MySQL support is currently in a [Beta release](/resources/feature-status).
MySQL types are mapped to SQLite types as follows:
| MySQL Data Type | PowerSync / SQLite Column Type | Notes |
| -------------------------------------------------------------- | ------------------------------ | ------------------------------------------------------------------------------------------- |
| `tinyint`, `smallint`, `mediumint`, `bigint`, `integer`, `int` | `integer` | |
| `numeric`, `decimal` | `text` | |
| `bool`, `boolean` | `integer` | `1` for true, `0` for false. There is no dedicated boolean data type in SQLite. |
| `float`, `double`, `real` | `real` | |
| `enum` | `text` | |
| `set` | `text` | Converted to JSON array |
| `char`, `varchar` | `text` | |
| `tinytext`, `text`, `mediumtext`, `longtext` | `text` | |
| `timestamp` | `text` | ISO 8601 format: `YYYY-MM-DDTHH:mm:ss.sssZ` |
| `date` | `text` | Format: `YYYY-MM-DD` |
| `time`, `datetime` | `text` | ISO 8601 format: `YYYY-MM-DDTHH:mm:ss.sssZ` |
| `year` | `text` | |
| `json` | `text` | There is no dedicated JSON type in SQLite — JSON functions operate directly on text values. |
| `bit` | `blob` | \* See note below regarding syncing binary types |
| `binary`, `varbinary` | `blob` | |
| `image` | `blob` | |
| `geometry`, `geometrycollection` | `blob` | |
| `point`, `multipoint` | `blob` | |
| `linestring`, `multilinestring` | `blob` | |
| `polygon`, `multipolygon` | `blob` | |
Binary data can be accessed in the Sync Streams / Sync Rules, but cannot be used as [parameters](/sync/overview#how-it-works). To sync binary columns/fields to clients, those columns need to be converted to hex or base64 representation using the relevant [functions](/sync/supported-sql#functions).
## SQL Server support is currently in a [Beta release](/resources/feature-status).
SQL Server types are mapped to SQLite types as follows:
| SQL Server Data Type | PowerSync / SQLite Column Type | Notes |
| ---------------------------------------------------------- | ------------------------------ | ------------------------------------------------------ |
| `tinyint`, `smallint`, `int`, `bigint` | `integer` | |
| `numeric`, `decimal` | `text` | Numeric string |
| `float`, `real` | `real` | |
| `bit` | `integer` | |
| `money`, `smallmoney` | `text` | Numeric string |
| `xml` | `text` | |
| `char`, `nchar`, `ntext` | `text` | |
| `varchar`, `nvarchar`, `text` | `text` | |
| `uniqueidentifier` | `text` | |
| `timestamp` | `text` | ISO 8601 format: `YYYY-MM-DDTHH:mm:ss.sssZ` |
| `date` | `text` | Format: `YYYY-MM-DD` |
| `time` | `text` | Format: `HH:mm:ss.sss` |
| `datetime`, `datetime2`, `smalldatetime`, `datetimeoffset` | `text` | ISO 8601 format: `YYYY-MM-DDTHH:mm:ss.sssZ` |
| `json` | `text` | Only exists for Azure SQL Database and SQL Server 2025 |
| `geometry`, `geography` | `text` | `text` of JSON object describing the spatial data type |
| `binary`, `varbinary`, `image` | `blob` | \* See note below regarding binary types |
| `rowversion`, `timestamp` | `blob` | \* See note below regarding binary types |
| User Defined Types: `hiearchyid` | `blob` | \* See note below regarding binary types |
Binary data can be accessed in the Sync Streams / Sync Rules, but cannot be used as [parameters](/sync/overview#how-it-works). To sync binary columns/fields to clients, those columns need to be converted to hex or base64 representation using the relevant [functions](/sync/supported-sql#functions).
# AI Tools
Source: https://docs.powersync.com/tools/ai-tools
Set up AI coding tools for PowerSync, including installing agent skills and accessing context files.
A growing collection of resources to help you work with PowerSync using AI-powered tools like Claude Code, Codex, Cursor, etc. These resources give AI assistants the context they need to help you implement PowerSync features faster.
## PowerSync Agent Skills
PowerSync Agent Skills provides AI agents with the PowerSync-specific context and capabilities needed to act on tasks in your codebase. This includes knowledge of PowerSync APIs, patterns, and best practices.
To get started, run the following command in your terminal and follow the prompts:
```bash theme={null}
npx skills add powersync-ja/agent-skills
```
Learn more:
PowerSync Agent Skills repository
## Copyable Prompts
The [documentation home page](/intro/powersync-overview) includes a "Build with AI" prompt you can copy straight into your coding agent. Each SDK reference page, listed in the [client SDKs overview](/client-sdks/overview), includes a version of the prompt for that SDK. The prompt installs the Agent Skills in your project, and the skills then guide the agent through onboarding your app to PowerSync.
## AI-Accessible Documentation
### Markdown Documentation
Every page in our documentation is available in Markdown — a format that's easier for LLMs to process than HTML. There are a few ways to access it:
* Append `.md` to any page URL:
```
https://docs.powersync.com/intro/setup-guide.md
```
* Press **Ctrl/Cmd+C** on any page to copy it as Markdown
* Use the contextual menu on a page to copy or view it as Markdown, or to open it directly in ChatGPT or Claude
Install globally or run via npx
PowerSync CLI repo on GitHub
Changelog for the CLI
The PowerSync CLI lets you manage PowerSync Service instances, deploy Sync Config (your Sync Streams or Sync Rules), generate client schemas, run diagnostics, and more. It is distributed as the [powersync](https://www.npmjs.com/package/powersync) npm package.
The CLI is currently in [beta](/resources/feature-status). We recommend it for
new and existing projects.
For a full step-by-step flow using the CLI, use the [Setup
Guide](/intro/setup-guide): choose the **CLI (Cloud)** or **CLI
(Self-Hosted)** tab in steps 2–5 to configure your instance, connect the
source database, deploy Sync Config, and generate development tokens.
The CLI was overhauled in version 0.9.0. The redesign is based on this [design proposal](https://docs.google.com/document/d/1iqpJF2gog2jB-ZWeN8TBEjcad8aBKNKbue2yJ21q_-s/edit).
**Main improvements:**
* **Project-based config** — A `powersync/` directory in your repo holds `service.yaml` and `sync-config.yaml`, so you version and review config with your app code.
* **Self-hosted support** — Most commands work against any linked instance, PowerSyncCloud and self-hosted. You can also use `powersync docker configure` to scaffold a local PowerSync stack with no manual setup.
* **Better validation** — `powersync validate` checks your config before deploy and reports errors with line and column numbers.
* **Config Studio** — `powersync edit config` opens a built-in editor with schema validation, autocomplete, inline errors, and more. See the [Config Studio README](https://github.com/powersync-ja/powersync-cli/tree/main/packages/editor).
* **Browser login** — `powersync login` opens a browser to create or paste a PAT and stores it; in CI use `PS_ADMIN_TOKEN`.
* **Plugins** — npm-based plugin system ([OCLIF](https://oclif.io)); install with `powersync plugins install ` or build with `@powersync/cli-core`.
* **Open source** — Source and advanced docs are in the [PowerSync CLI repo](https://github.com/powersync-ja/powersync-cli).
See [Migrating From the Previous CLI](#migrating-from-the-previous-cli) if you used the older flow.
## Installation
Install globally or run via `npx`:
```bash theme={null}
npm install -g powersync
```
```bash theme={null}
npx powersync@0.9.0 # 0.9.0 is the first version with the new CLI
```
## Authentication (Cloud)
Cloud commands require a PowerSync **personal access token (PAT)**.
**Interactive login (recommended for local use):**
```bash theme={null}
powersync login
```
You can open a browser to [create a token in the PowerSync Dashboard](https://dashboard.powersync.com/account/access-tokens) or paste an existing token. The CLI stores the token in secure storage when available (e.g. macOS Keychain), or in a config file after confirmation.
**CI and scripts:** Set the `PS_ADMIN_TOKEN` environment variable. The CLI uses `PS_ADMIN_TOKEN` when set; otherwise it uses the token from `powersync login`.
```bash theme={null}
export PS_ADMIN_TOKEN=your-personal-access-token
powersync fetch instances --project-id=
```
To clear stored credentials: `powersync logout`.
## Config Files
The CLI uses a config directory (default `powersync/`) with YAML files:
| File | Purpose |
| ------------------ | --------------------------------------------------------------------------- |
| `service.yaml` | Instance configuration: name, region, replication connection, client auth |
| `sync-config.yaml` | Sync Streams (or Sync Rules) |
| `cli.yaml` | Link file (written by `powersync link`); ties this directory to an instance |
### Developer Notes
* Use the **`!env`** tag for secrets, e.g. `uri: !env PS_DATABASE_URI` (or `!env VAR::number` / `!env VAR::boolean` for types).
* Edit files in your IDE, then run `powersync validate` and `powersync deploy`. For schema validation and `!env` support in your editor, run **`powersync configure ide`**; or run **`powersync edit config`** to open Config Studio (built-in web-based editor).
* To use one config directory across multiple instances (e.g. dev, staging, prod), see the CLI usage docs on [configuring multiple instances](https://github.com/powersync-ja/powersync-cli/blob/main/docs/usage.md#configuring-multiple-instances-eg-dev-staging-production).
* For Cloud secrets in `service.yaml`, use `password: secret: !env VAR` to supply the value from an environment variable at deploy time; after the first deploy you can switch to `secret_ref: default_password` to reuse the stored secret. [Details](https://github.com/powersync-ja/powersync-cli#cloud-secrets-format-serviceyaml)
* `cli.yaml` records the instance ID along with the organization and project IDs. The organization and project IDs act as a cache: when both are present, the CLI uses them directly instead of looking them up from the instance, saving an API request. If they are missing, the CLI resolves them automatically from the instance ID. If they are present but incorrect for the instance, the CLI reports an error rather than overriding them.
## Cloud Workflows
You can create instances, deploy and pull config, run all Cloud commands.
### Create a New Instance
```bash theme={null}
powersync login
powersync init cloud
```
Edit `powersync/service.yaml` (name, region, replication, auth) and Sync Config; use `!env` for secrets.
```bash theme={null}
powersync link cloud --create --project-id=
```
Add `--org-id=` if your token has multiple orgs.
```bash theme={null}
powersync validate
powersync deploy
```
Use `--directory=` for a different config folder.
### Use an Existing Instance (Pull)
Pull config from an instance that already exists (e.g. created in the Dashboard):
```bash theme={null}
powersync login
powersync pull instance --instance-id=
```
Then edit `service.yaml` and `sync-config.yaml` as needed, run `powersync validate`, and `powersync deploy`. Run `powersync pull instance` again (no IDs if already linked) to refresh from the cloud.
### Run Commands Without Local Config
To run commands (e.g. `powersync generate schema`, `powersync status`) against an instance managed elsewhere (e.g. Dashboard):
* **Link once:** `powersync link cloud --instance-id=` (writes `cli.yaml`); later commands use that instance.
* **Or pass each time:** `--instance-id`, or set `INSTANCE_ID` in the environment.
The CLI resolves instance and linking context in a fixed order: flags take
precedence, then environment variables, then values in `cli.yaml`. For the
full resolution order and how to set up multiple instances (e.g. dev, staging,
prod), see [supplying linking information for Cloud and self-hosted
commands](https://github.com/powersync-ja/powersync-cli/blob/main/docs/usage.md#supplying-linking-information-for-cloud-and-self-hosted-commands)
in the CLI usage docs.
## Self-Hosted Workflows
Support is limited: you **link** to an existing PowerSync API and run a **subset of commands**. The CLI does not create, deploy to, or pull config from your server; you manage the server and its config yourself. For local development, use **Docker** to run a PowerSync Service (and optional DB/storage) in containers.
### Authenticate
In your PowerSync instance config, define API tokens in `service.yaml`:
```yaml theme={null}
api:
tokens:
- dev-token-do-not-use-in-production # or !env MY_API_TOKEN
```
```bash theme={null}
powersync link self-hosted --api-url
```
Writes `cli.yaml` with the API URL.
In `cli.yaml` set `api_key: !env PS_ADMIN_TOKEN` (or a literal value matching a server token), or set the **`PS_ADMIN_TOKEN`** environment variable. If both are set, the environment variable takes precedence.
```yaml theme={null}
# powersync/cli.yaml (self-hosted)
type: self-hosted
api_url: https://powersync.example.com
api_key: !env PS_ADMIN_TOKEN # or a literal value matching one of the tokens in service.yaml
```
### Scaffold and Link (No Docker)
When you already have a running PowerSync API:
```bash theme={null}
powersync init self-hosted
# Edit powersync/service.yaml with instance details and api.tokens
powersync link self-hosted --api-url https://powersync.example.com
powersync status
```
Use `--directory=` for a different config folder.
### Supported Commands (Self-Hosted)
Only these commands apply to self-hosted instances: **`powersync status`**, **`powersync generate schema`**, **`powersync generate token`**, **`powersync validate`**, **`powersync fetch instances`** (scans current directory for folders with `cli.yaml`).
Cloud-only commands (**`powersync deploy`**, **`powersync pull instance`**, **`powersync fetch config`**, **`powersync destroy`**, **`powersync stop`**, **`powersync compact`**) do not apply.
### Docker (Local Development)
Run a PowerSync Service (and optional DB/storage) in containers on your machine—no remote server.
```bash theme={null}
powersync init self-hosted
powersync docker configure # links to the local API automatically
powersync docker start
```
Then use the same commands as any self-hosted instance (`powersync status`, `powersync generate schema`, etc.). To stop: **`powersync docker stop`** (add `--remove` to remove containers, `--remove-volumes` to reset so init scripts run again). For a clean setup: **`powersync docker reset`** (stop and remove, then start).
For the full Docker workflow, all flags (`--database`, `--storage`,
`--remove`, `--remove-volumes`), and how the template layout and init scripts
work, see [Docker
usage](https://github.com/powersync-ja/powersync-cli/blob/main/docs/usage-docker.md)
in the CLI repo. Run `powersync docker --help` for command options.
## Common Commands
| Command | Description |
| --------------------------------------------------------------- | ----------------------------------------------------------------------------------- |
| `powersync login` | Store PAT for Cloud (interactive or paste token); use `PS_ADMIN_TOKEN` in CI |
| `powersync logout` | Remove stored token |
| `powersync init cloud` | Scaffold Cloud config directory |
| `powersync init self-hosted` | Scaffold self-hosted config directory |
| `powersync configure ide` | IDE: YAML schema validation and `!env` support |
| `powersync link cloud --instance-id=` | Link to existing Cloud instance |
| `powersync link cloud --create --project-id=` | Create new Cloud instance and link |
| `powersync link self-hosted --api-url=` | Link to self-hosted instance |
| `powersync pull instance --instance-id=` | Download Cloud config to local files |
| `powersync deploy` | Deploy full config to linked Cloud instance |
| `powersync deploy service-config` | \[Cloud] Deploy only service config |
| `powersync deploy sync-config` | \[Cloud] Deploy only Sync Config |
| `powersync validate` | Validate config and Sync Streams/Rules |
| `powersync edit config` | Open Config Studio (Monaco editor) |
| `powersync status` | Instance diagnostics (Cloud and self-hosted) |
| `powersync generate schema --output=ts --output-path=schema.ts` | Generate client schema |
| `powersync generate token --subject=user-123` | Generate development JWT |
| `powersync fetch instances` | List Cloud and linked instances |
| `powersync fetch config` | \[Cloud] Print instance config (YAML/JSON) |
| `powersync migrate sync-rules` | Migrate Sync Rules to Sync Streams |
| `powersync destroy --confirm=yes` | \[Cloud] Permanently destroy instance |
| `powersync stop --confirm=yes` | \[Cloud] Stop instance (restart with deploy) |
| `powersync compact` | \[Cloud] Trigger [bucket compacting](/maintenance-ops/compacting-buckets) on demand |
Run `powersync --help` or `powersync --help` for flags. Full [command reference](https://github.com/powersync-ja/powersync-cli/blob/main/cli/README.md#commands) in the CLI repo.
## Deploying From CI (e.g. GitHub Actions)
You can automate Sync Config (and full config) deployments using the CLI in CI. Use the config directory as the source of truth: keep `service.yaml` and `sync-config.yaml` in the repo (with secrets via `!env` and CI secrets), then run `powersync deploy` (or `powersync deploy sync-config`).
**Secrets:** Set `PS_ADMIN_TOKEN` to your PowerSync personal access token. If the workflow does not use a linked directory, also set `INSTANCE_ID`. For self-hosted, `API_URL` can specify the PowerSync API base URL.
Example: deploy Sync Config on push to main
## Migrating From the Previous CLI
If you used the older PowerSync CLI (e.g. `npx powersync init` to set token and org/project, then `powersync instance set`, `powersync instance deploy`, etc.), the new CLI uses a different flow. Version 0.9.0 and above are **not backwards compatible** with 0.8.0. If you are not ready to migrate, you can stay on the old CLI:
```bash theme={null}
npm install -g @powersync/cli@0.8.0
```
Otherwise, upgrade to the latest **powersync** npm package and follow the mapping below.
| Previous CLI | New CLI |
| ----------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `npx powersync init` (enter token, org, project) | **`powersync login`** (token only). Then **`powersync init cloud`** to scaffold config, or **`powersync pull instance --instance-id=...`** to pull an existing instance. |
| `powersync instance set --instanceId=` | **`powersync link cloud --instance-id=`** (writes `cli.yaml` in config directory). Or use `--directory` for a specific folder. |
| `powersync instance deploy` (interactive or long flag list) | Edit **`powersync/service.yaml`** and **`powersync/sync-config.yaml`**, then **`powersync deploy`**. Config is in files, not command args. |
| `powersync instance config` | **`powersync fetch config`** (output as YAML or JSON with `--output`). |
| Deploy only Sync Streams/Rules | **`powersync deploy sync-config`**. |
| `powersync instance schema` | **`powersync generate schema --output=... --output-path=...`** (and/or **`powersync status`** for diagnostics). |
| Org/project stored by init | Not required separately. The CLI determines the organization and project from the instance ID and caches their IDs in `cli.yaml`. For CI, set **`PS_ADMIN_TOKEN`** and **`INSTANCE_ID`**. |
**Summary:** Authenticate with **`powersync login`** (or `PS_ADMIN_TOKEN` in CI). Use a **config directory** with `service.yaml` and `sync-config.yaml` as the source of truth. **Link** with **`powersync link cloud`** or **`powersync pull instance`**, then run **`powersync deploy`** or **`powersync deploy sync-config`**. No more setting “current instance” separately from config—the directory and `cli.yaml` define the target.
## Additional Documentation (CLI Repository)
More information is available in the [PowerSync CLI repository](https://github.com/powersync-ja/powersync-cli).
| Resource | Description |
| ---------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| [CLI README](https://github.com/powersync-ja/powersync-cli/blob/main/cli/README.md) | Getting started, Cloud and self-hosted overview, and full **command reference** with all flags. |
| [General usage](https://github.com/powersync-ja/powersync-cli/blob/main/docs/usage.md) | **How the CLI works**: local config vs linking, resolution order (flags → env vars → `cli.yaml`), and **configuring multiple instances** (e.g. dev/staging/prod with separate directories or `!env` in `cli.yaml`). |
| [Docker (local development)](https://github.com/powersync-ja/powersync-cli/blob/main/docs/usage-docker.md) | Self-hosted Docker workflow, configure/start/stop/reset, database and storage modules, and template layout. |
| [Config Studio (editor)](https://github.com/powersync-ja/powersync-cli/tree/main/packages/editor) | Built-in Monaco-powered editor for `service.yaml` and `sync-config.yaml` (`powersync edit config`), schema validation, and local development. |
| [Examples](https://github.com/powersync-ja/powersync-cli/blob/main/examples/README.md) | Sample projects initialized with the CLI (e.g. Cloud pull, self-hosted Postgres, self-hosted Supabase). |
## Known Issues and Limitations
* When secure storage is unavailable, `powersync login` may store the token in a plaintext config file after explicit confirmation.
* Self-hosted: the CLI does not create or manage instances on your server, or deploy config to it; it only links to an existing API and runs a subset of commands (status, generate schema/token, validate). The sole exception is **Docker**: it starts a local PowerSync Service (and optional DB/storage) in containers on your machine for development — not a remote or production instance.
* Some validation checks require a connected instance to complete successfully; validation of an unprovisioned instance may show errors that resolve after the first deployment.
## Reference
Package and version history
Create or revoke tokens in the PowerSync Dashboard
Source code, usage docs, Docker usage, and examples
# Dart & Flutter DevTools Extension
Source: https://docs.powersync.com/tools/dart-devtools-extension
A Dart & Flutter DevTools extension for inspecting PowerSync databases, running SQL queries, and viewing sync status during development.
The PowerSync DevTools extension adds a Database Inspector panel to [Dart & Flutter DevTools](https://docs.flutter.dev/tools/devtools/overview). It gives you a view into an open `PowerSyncDatabase` instance while your app is running during development. The Overview helps you see how the client is connected, whether there are any sync issues, and details about each Sync Stream. You can open the same session in the [Sync Diagnostics Client](/tools/diagnostics-client) for further sync activity details. The SQL Console is for running SQL and sampling rows from the on-device SQLite database.
The extension is included in the `powersync` package from v2.1.0 and activates automatically when a `PowerSyncDatabase` is opened. It is disabled in release builds.
## Opening the Extension
First, open DevTools:
* In **VS Code**, launch your app in debug mode, then run the **Dart: Open DevTools in Browser** command from the command palette.
* In **Android Studio / Intellij**, click the **Flutter DevTools** button in the run toolbar.
* For apps launched from the command line, see [these instructions](https://docs.flutter.dev/tools/devtools/cli).
In DevTools, A **powersync** tab appears in the DevTools tab bar alongside the standard panels.
If you have multiple databases open, use the dropdown in the top-right corner to switch between them.
## Overview
The Overview tab shows the current state of your database and sync connection.
The SQL Console runs queries against the local SQLite database only. It does not send queries to your backend source database.
# DevTools Integrations
Source: https://docs.powersync.com/tools/devtools-overview
IDE and framework DevTools integrations for inspecting PowerSync client-side databases and sync state during development.
PowerSync currently provides DevTools integrations for Dart/Flutter and Nuxt. These embed a diagnostics panel into your existing IDE or framework tooling, so you can inspect your app's local database, Sync Stream state, and JWT credentials without launching a separate tool. They are available in development builds only.
Support for additional frameworks is under consideration. If you'd like to see a specific integration, submit an idea or vote for it on the [PowerSync Roadmap](https://roadmap.powersync.com).
Inspect databases, run SQL queries, and view Sync Stream state inside Dart & Flutter DevTools.
Inspect sync status, local data, config, and logs inside Nuxt DevTools.
## DevTools vs. Sync Diagnostics Client
The DevTools integrations and the [Sync Diagnostics Client](/tools/diagnostics-client) serve different purposes and complement each other.
The DevTools integrations connect to your app's running process and show you what is happening on the client — local SQLite data, sync state, and any issues within the app itself.
The Sync Diagnostics Client connects via a user JWT and PowerSync endpoint. It shows sync-level data — table, row, operation and bucket counts, and stream subscriptions — as seen through the sync connection.
| | DevTools integrations | Sync Diagnostics Client |
| -------------------- | ----------------------------------------------- | ---------------------------------------------------------- |
| **How it connects** | Directly to your app's running process | Via a user JWT + PowerSync endpoint |
| **What you inspect** | Local SQLite data and sync state within the app | Sync-level data: buckets, operations, stream subscriptions |
| **Environment** | Development only | Any environment |
| **SDK-specific** | Yes (Dart/Flutter, Nuxt) | No — works with all SDKs |
# Sync Diagnostics Client
Source: https://docs.powersync.com/tools/diagnostics-client
Inspect and debug PowerSync sync behavior with the Sync Diagnostics Client app, a standalone tool for testing Sync Streams.
# Run PowerSync Locally with Docker and the CLI
Source: https://docs.powersync.com/tools/local-development
Set up PowerSync locally for development using Docker and the CLI.
The easiest way to run PowerSync locally is with the [PowerSync CLI](/tools/cli), which scaffolds and manages a Docker Compose stack for you.
See the [Setup Guide](/intro/setup-guide) for step-by-step CLI instructions: `powersync init self-hosted` + `powersync docker configure` + `powersync docker start`.
If you'd prefer to write your own Docker Compose setup, here's a minimal example.
## Docker Compose Example
Create a working directory with three files:
```
powersync/
docker-compose.yaml
service.yaml
sync-config.yaml
```
### `docker-compose.yaml`
This example uses Postgres as the source database and MongoDB as bucket storage. Postgres is also supported as bucket storage — see [Self-Hosted Instance Configuration](/configuration/powersync-service/self-hosted-instances) for details.
```yaml theme={null}
services:
powersync:
restart: unless-stopped
depends_on:
mongo-rs-init:
condition: service_completed_successfully
postgres:
condition: service_healthy
image: journeyapps/powersync-service:latest
command: ["start", "-r", "unified"]
volumes:
- ./service.yaml:/config/service.yaml
- ./sync-config.yaml:/config/sync-config.yaml
environment:
POWERSYNC_CONFIG_PATH: /config/service.yaml
ports:
- 8080:8080
# Source database (Postgres with logical replication enabled)
postgres:
image: postgres:latest
restart: always
environment:
- POSTGRES_USER=postgres
- POSTGRES_DB=postgres
- POSTGRES_PASSWORD=postgres
- PGPORT=5432
volumes:
- pg_data:/var/lib/postgresql/data
ports:
- "5432:5432"
command: ["postgres", "-c", "wal_level=logical"]
healthcheck:
test: ["CMD-SHELL", "pg_isready -U postgres -d postgres"]
interval: 5s
timeout: 5s
retries: 5
# MongoDB used internally for bucket storage
mongo:
image: mongo:7.0
command: --replSet rs0 --bind_ip_all --quiet
restart: unless-stopped
ports:
- 27017:27017
volumes:
- mongo_storage:/data/db
# Initializes the MongoDB replica set. This service will not usually be actively running
mongo-rs-init:
image: mongo:7.0
depends_on:
- mongo
restart: on-failure
entrypoint:
- bash
- -c
- 'mongosh --host mongo:27017 --eval ''try{rs.status().ok && quit(0)} catch {} rs.initiate({_id: "rs0", version: 1, members: [{ _id: 0, host : "mongo:27017" }]})'''
volumes:
mongo_storage:
pg_data:
```
### `service.yaml`
The main PowerSync Service configuration. See [Self-Hosted Instance Configuration](/configuration/powersync-service/self-hosted-instances) for the full reference.
```yaml theme={null}
# Source database connection
replication:
connections:
- type: postgresql
uri: postgresql://postgres:postgres@postgres:5432/postgres
sslmode: disable # verify-full, verify-ca, or disable
# Bucket storage (MongoDB shown; Postgres is also supported)
storage:
type: mongodb
uri: mongodb://mongo:27017/powersync_storage
# The port which the PowerSync API server will listen on
port: 8080
# Points to the Sync Config file
sync_config:
path: sync-config.yaml
# Settings for client authentication
client_auth:
# Enable this if using Supabase Auth
supabase: false
# Or enter a static collection of public keys for JWT verification, and generate a development token using the CLI (`powersync generate token`)
# jwks:
# keys:
# - kty: 'RSA'
# n: '[rsa-modulus]'
# e: '[rsa-exponent]'
# alg: 'RS256'
# kid: '[key-id]'
# JWKS audience
# audience: ['powersync-dev', 'powersync', 'http://localhost:8080']
```
### `sync-config.yaml`
Defines what data syncs to clients. See [Sync Streams](/sync/streams/overview) for full syntax.
```yaml theme={null}
config:
edition: 3
streams:
global:
# Streams without parameters sync the same data to all users
auto_subscribe: true
queries:
- SELECT * FROM todos
- SELECT * FROM lists
```
### Start the Stack
```bash theme={null}
docker compose up
```
## Resources
* [PowerSync CLI](https://github.com/powersync-ja/powersync-cli) — open source CLI; use it to scaffold and run a Docker-based local stack
* [self-host-demo](https://github.com/powersync-ja/self-host-demo) — complete working examples with Docker Compose
* [Self-Hosted Instance Configuration](/configuration/powersync-service/self-hosted-instances) — full `service.yaml` reference
* [Sync Streams](/sync/streams/overview) — Sync Config syntax
* [Generate a Development Token](/intro/setup-guide#5-generate-a-development-token) — for testing without a full auth setup
# Nuxt DevTools Integration
Source: https://docs.powersync.com/tools/nuxt-inspector
A Nuxt DevTools integration for inspecting PowerSync sync status, local data, config, and logs during development.
The `@powersync/nuxt` module includes a PowerSync diagnostics panel that integrates with [Nuxt DevTools](https://devtools.nuxt.com/). It lets you inspect your app's sync state, local SQLite data, and configuration while running in development.
## Enabling Diagnostics
Add `powersync: { useDiagnostics: true }` to your `nuxt.config.ts`:
```typescript nuxt.config.ts theme={null}
export default defineNuxtConfig({
modules: ['@powersync/nuxt'],
powersync: {
useDiagnostics: true
},
vite: {
optimizeDeps: {
exclude: ['@powersync/web']
},
worker: {
format: 'es'
}
}
});
```
No changes to your plugin code are needed. When `useDiagnostics: true` is set, `NuxtPowerSyncDatabase` automatically extends your schema with the diagnostics schema, sets up diagnostics recording and logging, and stores the connector internally so the inspector can access it.
## Accessing the Inspector
Dashboard for PowerSync Cloud. Allows managing your PowerSync organization, projects and instances.
Manage PowerSync Cloud and self-hosted instances from the command line.
Manage PowerSync Cloud projects and instances as infrastructure-as-code.
Web app to inspect and debug syncing. Works with both cloud and self-hosted PowerSync Service instances.
Using Docker Compose to self-host PowerSync for development purposes.
Resources for working with PowerSync with AI-powered coding tools.
## DevTools Integrations
IDE and framework integrations for inspecting PowerSync databases and sync state during development.
Inspect databases, run SQL queries, and view sync state inside Dart & Flutter DevTools.
Inspect sync status, local data, config, and logs inside Nuxt DevTools.
# PowerSync Dashboard
Source: https://docs.powersync.com/tools/powersync-dashboard
Use the PowerSync Dashboard to manage instances, configure Sync Streams, and monitor usage.
The PowerSync Dashboard is available in [PowerSync Cloud](https://www.powersync.com/pricing) (our cloud-hosted offering) and provides an interface for managing your PowerSync organization, projects, instances, and account settings.
The dashboard is available here: [https://dashboard.powersync.com/](https://dashboard.powersync.com/)
## Hierarchy: Organization, Project, Instance
* After successfully [signing up](https://accounts.powersync.com/portal/powersync-signup?s=docs) for PowerSync Cloud, your **PowerSync account** is created.
* Your account is assigned an **organization** on the [Free pricing plan](https://www.powersync.com/pricing).
* To get started, you'll need to create a **project** in your organization.
Terraform Registry
Use it to:
* Provision PowerSync in the same `terraform apply` that creates your database and other cloud resources.
* Define dev, staging, and production environments once in version-controlled config rather than configuring each manually in the dashboard.
* Review changes to sync configuration, replication connections, and authentication settings as Terraform plan diffs in pull requests.
This guide covers PowerSync Cloud only. For self-hosted deployments, see the [self-hosting overview](/maintenance-ops/self-hosting/overview).
**Prerequisites**
* A PowerSync Cloud account
* A source database. The examples in this guide use Postgres via Supabase. If your database is also managed by Terraform, you can reference it in the same configuration.
## Get a Personal Access Token
The provider authenticates using a personal access token. In the dashboard, go to [Account → Access Tokens](https://dashboard.powersync.com/account/access-tokens) and create a new token. Treat it like a password. It grants account access.
Set it as an environment variable before running any Terraform commands:
```sh theme={null}
export PS_PAT_TOKEN="jpt_..."
```
The provider reads this environment variable automatically.
## Find Your Organization ID
Your organization ID appears in the dashboard URL. When you are on the organization home page, the URL contains a hex segment like `/org/64b3f8e1a2c4d5e6f7080912/`. That segment is your organization ID.
## Configure the Provider
Create a new directory and a `main.tf` file. Start with the provider declaration:
```hcl theme={null}
terraform {
required_providers {
powersync = {
source = "powersync-ja/powersync"
version = "~> 0.1"
}
}
}
provider "powersync" {
# admin_token is picked up from the PS_PAT_TOKEN environment variable (recommended).
# It can also be inlined — less secure, but if inlined via a variable the value
# itself can still be passed securely through Terraform's own env vars
# (e.g. TF_VAR_ps_admin_token).
# admin_token = var.admin_token
}
```
Then run:
```sh theme={null}
terraform init
```
This downloads the provider binary from the Terraform Registry.
## Create a Project
Add a data source to look up your organization, then a resource to create a project:
```hcl theme={null}
data "powersync_organization" "main" {
id = "64b3f8e1a2c4d5e6f7080912"
}
resource "powersync_project" "main" {
org_id = data.powersync_organization.main.id
name = "my-project"
region = "us"
}
```
The [`data "powersync_organization"`](https://registry.terraform.io/providers/powersync-ja/powersync/latest/docs/data-sources/organization) block reads your existing organization. Organizations are created when you sign up and are not managed as Terraform resources. The [`powersync_project`](https://registry.terraform.io/providers/powersync-ja/powersync/latest/docs/resources/project) resource creates a new project under it. Supported regions are `eu`, `us`, `jp`, `au`, and `br`.
## Create an Instance
Add the [`powersync_instance`](https://registry.terraform.io/providers/powersync-ja/powersync/latest/docs/resources/instance) resource, which wires together a replication connection, client authentication, and your sync configuration:
```hcl theme={null}
variable "replication_password" {
type = string
sensitive = true
}
resource "powersync_instance" "main" {
org_id = data.powersync_organization.main.id
project_id = powersync_project.main.id
name = "production"
replication_connection {
type = "postgresql"
name = "main"
hostname = "db..supabase.co"
port = 5432
username = "powersync_role"
password = var.replication_password
database = "postgres"
sslmode = "verify-full"
}
client_auth {
supabase = true
allow_temporary_tokens = true
}
sync_config_content = <<-YAML
config:
edition: 3
streams:
todos:
auto_subscribe: true
query: SELECT * FROM todos
YAML
}
```
You can manage your Supabase project with Terraform too, you can provision the database and PowerSync instance together in the same configuration. See the [Supabase Terraform provider](https://supabase.com/docs/guides/deployment/terraform).
`replication_password` is a Terraform variable. Pass its value via the environment so it never appears in plain text in your config or state files.
The `sync_config_content` field takes a Sync Streams configuration as a YAML string. For larger configs, use `file("${path.module}/sync-config.yaml")` to load the configuration from a separate file. See the [Sync Streams documentation](/sync/streams/overview) for the full configuration reference.
## Apply
Set the replication password, then run:
```sh theme={null}
export TF_VAR_replication_password="..."
terraform plan
terraform apply
```
`terraform plan` shows you exactly what Terraform will create before you commit to it. `terraform apply` creates the project and instance, then deploys it. When it completes, the instance is live. Run `terraform show` to see the full resource state including the instance URL.
## Managing Changes
To update your PowerSync instance, edit any value in your config and re-run `terraform plan` then `terraform apply`. Every change triggers a full redeploy. The instance ID and URL remain the same.
## Tear Down
```sh theme={null}
terraform destroy
```
This removes the instance and the project. If the project contains instances created outside of Terraform, the destroy fails. Set `force_destroy = true` on `powersync_project` to override this check.
## Resources
* [PowerSync Provider](https://registry.terraform.io/providers/powersync-ja/powersync/latest/docs) on the Terraform Registry
* [Supported source database setup](/configuration/source-db/setup) for configuring connection with the PowerSync Service
* [Sync Streams overview](/sync/streams/overview) for writing the `sync_config_content`
* [Connecting your client](/client-sdks/overview) to a PowerSync Service instance