MongoDB Atlas Device Sync Migration Guide

​

Introduction

Moving to PowerSync allows you to benefit from efficient data synchronization using open and proven technologies. Users get always-available, instantly-responsive offline-first apps that also stream data updates in real-time when online.

​

Why PowerSync?

PowerSync’s history goes as far back as 2009, when the original version of the sync engine was developed as part of an app development platform used by some of the world’s largest industrial companies to provide employees with offline-capable business apps deployed in harsh environments (learn more about PowerSync’s history).

PowerSync was spun off as a standalone product in 2023, and gives engineering teams a proven, open and robust sync engine with a familiar server-client architecture.

PowerSync’s MongoDB connector has been developed in collaboration with MongoDB to provide an easy setup process. It reached General Availability (GA) status with its V1 release and is fully supported for production use. Multiple MongoDB customers currently use PowerSync in production environments.

The server-side PowerSync Service connects to MongoDB and pre-processes and pre-indexes data to be efficiently synced to users based on defined “Sync Rules”. Client applications connect to the PowerSync Service to sync data relevant to each user. Incremental updates in MongoDB are synced to clients in real-time.

Client applications can read and write data to the local client-side database. PowerSync provides for bi-directional syncing so that writes in the local client-side databases are automatically synced back to the source MongoDB database. If users are offline or have patchy connectivity, PowerSync automatically manages network failures and retries.

By introducing PowerSync as a sync engine, you get:

• Predictable sync behavior that syncs relevant data to each user.
• Consistency guarantees ensuring consistent state of the client-side database.
• Real-time multi-user applications as data updates are streamed to connected clients in real-time.
• Instantly responsive user experience as the user interaction with the app is unaffected by the network.
• Offline-first capabilities enabling apps to continue to work regardless of network conditions.

Please review this guide to understand the required changes and prerequisites. Following the provided steps will help your team transition smoothly.

If you need further assistance at any point, you can:

• Set up a call with PowerSync engineers.
• Ask us anything on our Discord server.
• Contact us through email.

​

Architecture: Before and After

If you have MongoDB Atlas Device Sync deployed today, at a high level your architecture will look something like this:

Migrating to PowerSync results in this architecture: (new components in green)

Here is a quick overview of the resulting PowerSync architecture:

• PowerSync Service is available as a cloud-hosted service (PowerSync Cloud), or you can self-host using our Open Edition.
• Authentication: PowerSync piggybacks off your app’s existing authentication, and JWTs are used to authenticate between clients and the PowerSync Service. If you are using Atlas Device SDKs for authentication, you will need to implement an authentication provider.
• PowerSync Client SDKs use SQLite under the hood. Even though MongoDB is a “NoSQL” document database, PowerSync’s use of SQLite works well with MongoDB, since the PowerSync protocol is schemaless (it syncs schemaless JSON data) and we dynamically apply a client-side schema to the data in SQLite using SQLite views. Client-side queries can be written in SQL or you can make use of an ORM (we provide a few ORM integrations)
• Reads vs Writes: PowerSync handles syncing of reads differently from writes.
  • Reads: The PowerSync Service connects to your MongoDB database and replicates data in real-time to PowerSync clients. Reads are configured using PowerSync’s “Sync Rules”. Sync Rules are more flexible than MongoDB Realm Flexible Sync, but are defined on the server-side, not on the client-side.
  • Writes: The client-side application can perform writes directly on the local SQLite database. The writes are also automatically placed into an upload queue by the PowerSync Client SDK. The SDK then uses a developer-defined uploadData() function to manage the uploading of those writes sequentially to the backend.
• Authorization: Authorization is controlled separately for reads vs. writes.
  • Reads: The Sync Rules control which users can access which data.
  • Writes: The backend controls authorization for how users can modify data.
• Backend: PowerSync requires a backend API interface to upload writes to MongoDB. There are currently two options:
  • Custom self-hosted backend: If you already have a backend application as part of your stack, you should use your existing backend. If you don’t yet have one: We have example implementations available (e.g. Node.js, Django, Rails).
  • Serverless cloud functions (hosted/managed): An alternative option is to use CloudCode, a serverless cloud functions environment provided by PowerSync. We have a template available that you can use as a turnkey starting point.

​

Migration Steps

Follow the steps below to migrate a MongoDB Atlas Device Sync app to PowerSync.

​

1. Create PowerSync account and instance

To get started quickly with PowerSync, sign up for a free PowerSync Cloud account here.

It is also possible to self-host PowerSync. An end-to-end demo app using Docker Compose is available here.

​

2. Connect PowerSync to MongoDB

Once your account is set up, create a new PowerSync instance and configure the instance to connect to your source MongoDB database.

​

3. Define Sync Rules

Sync Rules allow you to control which data gets synced to which users/devices. Each PowerSync Service instance has a Sync Rules definition that consists of SQL-like queries in a YAML file.

To get a good understanding of how Sync Rules operate, have a look at our blog post: Sync Rules from First Principles: Partial Replication to SQLite.

If you have a PowerSync Service instance set up and connected, open the sync-rules.yaml file associated with your PowerSync project and edit the SQL-like queries based on your database schema. Below is a simple Sync Rules example using a simple database schema. Sync Rules involve organizing data into “buckets” (a bucket is a grouping of data). The example below uses a “global bucket” as a simple starting point — data in a “global bucket” will be synced to all users.

bucket_definitions:
   # This is the name of the bucket, in this case the global bucket synced to all users.
   global:
      # This is the query used to determine the data in each bucket
      data:
      # Note that we select the MongoDB _id field as id
      - SELECT _id as id, * FROM lists
      - SELECT _id as id, * FROM todos

To filter data based on the user and other more advanced use cases, refer to the Sync Rules documentation.

​

4. Add PowerSync to your app

Add PowerSync to your app project by following the instructions for the relevant PowerSync Client SDK.

• Visit our Client SDK directory for instructions specific to your platform.

​

5. Define your client-side schema

The PowerSync client-side schema represents a “view” of the data synced from the PowerSync Service to the client app. No migrations are required — the schema is applied directly when the local PowerSync SQLite database is constructed.

To make this step easy for you, the PowerSync Dashboard allows automatically generating the client-side schema based on the Sync Rules defined for a PowerSync instance. To generate the schema, go to the dashboard, right-click on the instance, and select “Generate Client Schema”. Alternatively you can use the PowerSync CLI to generate the schema.

Here is an example of a client-side schema for PowerSync using a simple todos table:

import { column, Schema, Table } from '@powersync/react-native';

const todos = new Table(
   {
      list_id: column.text,
      created_at: column.text,
      completed_at: column.text,
      description: column.text,
      created_by: column.text,
      completed_by: column.text,
      completed: column.integer
   },
   { indexes: { list: ['list_id'] } }
);

export const AppSchema = new Schema({
   todos
});

A few things to note regarding the PowerSync client-side schema:

• The schema does not explicitly specify an id column, since PowerSync automatically creates an id column of type text.
• SQLite has very simple data types which are used by PowerSync.
• For MongoDB specific data types, refer to MongoDB Type Mapping.
• PowerSync also supports syncing attachments or files using helper packages.

​

6. Instantiate PowerSync client database

Now that we have our Sync Rules and client-side schema defined, we can instantiate the PowerSync database on the client-side. This will allow the app to start syncing data. For more details, see Instantiate PowerSync Database.

import { PowerSyncDatabase } from '@powersync/react-native';
import { Connector } from './Connector';
import { AppSchema } from './Schema';

export const db = new PowerSyncDatabase({
   schema: AppSchema,
   database: {
      dbFilename: 'powersync.db'
   }
});

export const setupPowerSync = async () => {
   const connector = new Connector();
   db.connect(connector);
};

​

7. Reading and writing data

Reading data in the application which uses PowerSync is very simple: we use SQLite syntax to query data in our local database.

// Reading Data
export const getTodos = async () => {
   const results = await db.getAll('SELECT * FROM todos');
   return results;
}

The same applies to writing data: INSERT, UPDATE and DELETE statements are used to create, update and delete rows.

// Writing Data
export const insertTodo = async (listId: string, description: string) => {
   await db.execute('INSERT INTO todos (id, created_at, list_id, description) VALUES (uuid(), date(), ?, ?)', [listId, description]);
}

​

Live queries

PowerSync supports “live queries” or “watch queries” which automatically refresh when data in the SQLite database is updated (e.g. as a result of syncing from the server). This allows for real-time reactivity of your app UI. See the Client SDK documentation for your specific platform for more details.

​

8. Accept uploads on the backend

MongoDB Atlas Device Sync provides built-in writes/uploads to the backend MongoDB database.

PowerSync offers full customizability regarding how writes are applied. This gives you control to apply your own business logic, data validations, authorization and conflict resolution logic.

There are two options:

• Serverless cloud functions (hosted/managed): PowerSync offers serverless cloud functions hosted on the same infrastructure as PowerSync Cloud which can used for the needed backend functionality needed. We provide a MongoDB-specific template for this which can be used as a turnkey solution.
• Custom self-hosted backend: Alternatively, writes can be processed through your own backend.

​

Using PowerSync’s serverless cloud functions

PowerSync provides serverless cloud functions for backend functionality, with a template available for MongoDB. See the step-by-step instructions on how to use the template. The template can be customized, or it can be used as-is.

The template provides turnkey conflict resolution which roughly matches the built-in conflict resolution behavior provided by MongoDB Atlas Device Sync.

For more information, see our blog post: Turnkey Backend Functionality & Conflict Resolution for PowerSync.

​

Using your own custom backend API

This option gives you complete control over the backend. The simplest backend implementation is to simply apply writes to MongoDB as they are received, which results in a last-write-wins conflict resolution strategy (same as the “turnkey backend functionality” option above). See Writing Client Changes for more details.

On the client-side, you need to wire up the uploadData() function in the “backend connector” to use your own backend API. The App Backend Setup section of our docs provides step-by-step instructions for this.

Also see the section on how to set up a simple backend API in our practical MongoDB migration example on our blog.

We also have example backend implementations available (e.g. Node.js, Django, Rails)

​

Questions? Need help?

Get in touch with us.