Migrate from Electric Cloud to PowerSync Cloud

Electric announced that it is joining Databricks and Electric Cloud is shutting down. Their official guidance is to move to either self-hosted Electric or another provider. This guide is for users looking to migrate to an alternative cloud provider of a Postgres-backed sync engine, namely PowerSync.

This is an implementation guide for teams already using Electric Cloud's current Postgres Sync service. It assumes Postgres remains the system of record and covers high-level concepts to assist with a migration.

Concepts

PowerSync Sync Streams are analogous to Electric Shapes, not Electric Streams. Electric Streams is a separate product and is out of scope for this guide.

While Electric Shapes and PowerSync Sync Streams both control partial sync, there are significant differences meaning that Shape to Sync Stream migration will most likely not be 1-for-1, and re-designing partial sync will most likely be required.

A key architectural difference between Electric Shapes and Sync Streams is that Shapes are created on demand from the client, per request, while Sync Streams are declared as named queries ahead of time that clients then dynamically subscribe to when needed, each subscription passing its own parameters. This architecture drives how authorization, offline behavior, and writes work. The table below summarizes the specific differences between Electric Shapes and Sync Streams across these and other key dimensions:

Migration

Note that both Electric Sync and PowerSync codebases are public, so agents should generally not struggle with implementing migrations. Also see the PowerSync agent resources. If you need assistance, join the PowerSync Discord.

Overview of Migration Steps

1. Configure Postgres
2. Connect PowerSync
3. Define and test Sync Streams
4. Set up Authentication
5. Migrate the frontend code

Configure Postgres

1. Create a Postgres role and publication as per documentation.
   1. Note: it is not possible to reuse the Electric publication, since PowerSync requires a publication named powersync.
2. Note that BYPASSRLS is commonly used for the PowerSync role. This is because Sync Stream definitions enforce authorization.

Connect PowerSync to Postgres

1. Connect your PowerSync instance to your Postgres environment
   1. Dashboard instructions
   2. CLI instructions
2. Since the PowerSync Service connects directly to Postgres, various network-level security mechanisms are supported.

Define and Test Sync Streams

In this step you will write your Sync Streams YAML. This will:

• Define how subsets of your Postgres data are synced to SQLite (on the client)
• Move download authorization into Sync Stream queries. You keep write authorization in your backend. PowerSync uses signed JWT claims for access checks. Sync Streams accept client parameters, but these should not be relied on for authorization checks.
• Refer to the Sync Streams documentation
• Note that if the supported Sync Streams SQL doesn't support the specific query you are trying to write, sync the required streams and then run ORDER BY, LIMIT, aggregates, joins, etc. in local SQLite.

Example

A Shape that syncs a user's projects as follows:

const { data } = useShape({
  url: `http://localhost:3000/v1/shape`,
  params: {
    table: `projects`,
    where: `owner_id = ${currentSession().userId}`,
  },
})

Can be defined in Sync Streams as this my_projects stream:

streams:
  my_projects:
    query: SELECT * FROM projects WHERE owner_id = auth.user_id()

You can then define a separate Sync Stream to sync each project's tasks (project_tasks stream), where the client provides the project_id:

streams:
  my_projects:
    ...

  project_tasks:
    query: |
      SELECT * FROM tasks
      WHERE project_id = subscription.parameter('project_id')
        AND project_id IN (SELECT id FROM projects WHERE owner_id = auth.user_id())

Run a Sync Test from the PowerSync Dashboard

Once you've defined your Sync Streams, you can run a Sync Test in the PowerSync Dashboard to validate that data is syncing to the client as expected.

Set Up Authentication

Electric relies on a backend to validate requests to Shapes. With PowerSync that logic is contained in your Sync Stream queries, and clients are then able to connect to the PowerSync Service with a JWT minted from your backend, instead of using the backend as a step in the middle.

auth.user_id illustrates this in the Sync Streams example from above:

    query: SELECT * FROM projects WHERE owner_id = auth.user_id()

Our authentication documentation covers how to set this up.

Migrate the Frontend Code

The PowerSync client owns a local SQLite database that intelligently merges rows from all active Sync Streams. It exposes local SQL and live query (watch) APIs. It optionally records local writes into a FIFO upload queue. Note that migrations from PGlite to SQLite are possible but out of scope for this guide.

PowerSync provides SDKs for many platforms: Swift, Kotlin, Dart, .NET, Rust and JS/TS (Web, React Native, Node, Capacitor and Tauri). This section only covers Web JS/TS.

1. Install the PowerSync Web SDK with pnpm install @powersync/web

2. Use the PowerSync Dashboard or CLI to generate the client-side SQLite schema (docs). Note that an id column is added automatically. It will look similar to this:

   import { column, Schema, Table } from '@powersync/web'
   
   const projects = new Table({
     name: column.text,
     owner_id: column.text,
   })
   
   const tasks = new Table({
     project_id: column.text,
     title: column.text,
     status: column.text,
   }, { indexes: { by_project: ['project_id'] } })
   
   export const AppSchema = new Schema({ projects, tasks })

3. Instantiate the PowerSyncDatabase. Note that you must only create one PowerSyncDatabase instance for each database file

   import { PowerSyncDatabase } from '@powersync/web'
   
   export const db = new PowerSyncDatabase({
     schema: AppSchema,
     database: { dbFilename: 'powersync-v1.db' },
   })

4. The Web SDK uses a persistent IndexedDB VFS by default. Select another VFS if you have specific browser, performance, or multi-tab requirements.

5. Integrate with your backend. This requires implementing two methods for PowerSyncBackendConnector: fetchCredentials and uploadData

   1. fetchCredentials() returns the JWT used to authenticate against PowerSync with and download synced data, as well as the PowerSync Cloud endpoint
   2. uploadData() defines how local mutations are sent to your backend API - you should be able to re-use the existing backend you have in place today

6. Replace client-side queries. This is where the bulk of the work will take place, but agents should be pretty good at it. Below are some more tips to get you going:

   1. Replace shape handle/offset usage with waitForFirstSync() and Sync Stream status checks.
   2. Tie on-demand Sync Stream subscriptions to component or route lifetime. The PowerSync React Hooks can automatically subscribe/unsubscribe.
   3. Note that when subscribing to a Sync Stream on the client, the TTL can be overridden. The default is 24 hours: a shorter TTL reduces disk usage, a longer TTL improves page reload performance.
   4. When logging the user out or switching accounts, only use disconnectAndClear() once the upload queue has been emptied, otherwise local mutations will get discarded.

7. Implement writes

   1. Electric is read-path only, so this guide doesn't cover migrating a write path. However, it's highly recommended to use the PowerSync upload queue to ensure consistency instead of sending mutations directly to your backend. You might notice data flicker on the client if you write directly to your backend APIs / bypass the PowerSync upload queue.
   2. Follow this guide for integrating the PowerSync upload queue with your backend APIs