Updated 2024-09-11
On July 17th 2024, ElectricSQL announced a “clean rebuild” of their product, which was originally called ‘electric-next’. They also announced that they’ve stopped development on the legacy ElectricSQL [1], as it’s being superseded by electric-next.
The ElectricSQL team included a detailed rationale for this decision: The initial ElectricSQL system was too large and complex in scope, and the team had a hard time getting it to a point where it was stable and reliable. With their rebuild, they are aiming to deliver a stable system that is significantly reduced in scope, and more modular and loosely coupled.
What is interesting is that ElectricSQL’s simplified architecture has a number of similarities to PowerSync’s architecture, which is perhaps not surprising since in designing PowerSync, we aimed for a “simple, robust and proven architecture” for a Postgres-SQLite sync engine.
While both products are still sync engines that work with Postgres databases, electric-next is more limited in scope than PowerSync at the time of publishing this blog post. Below, we look at notable similarities and differences between the two.
Similarities between ElectricSQL’s electric-next and PowerSync
ElectricSQL's electric-next | PowerSync |
---|---|
Uses Postgres logical replication and stores sync data relating to ‘shapes’ and a log of their operation history in either an in-memory cache, or persistent storage. |
Uses Postgres logical replication and stores sync data relating to ‘buckets’ and their operation history in pluggable persistent storage used by the PowerSync Service (currently MongoDB is supported). |
Dynamic partial replication via Shapes: With the current version of shapes in electric-next, the client syncs one or more shapes, and each shape contains a linear log of changes to sync, sorted by “offset”. A shape is defined by a single table and a WHERE clause. No automatic syncing of related data is presented yet, and it’s currently not possible to accomplish this manually. We expect the shape functionality to expand over time, but it’s not yet clear what will be supported or how it will be implemented. |
Dynamic partial replication via Buckets & Sync Rules: PowerSync has the concept of buckets and clients sync one or more buckets, with each bucket containing operation history for the data in that bucket, sorted by an ‘operation ID’ (op_id). The data to be synced in each bucket is defined by Sync Rules which consist of parameter queries (to retrieve all relevant parameters), and one or more data queries that can make use of those parameters. |
Decoupling of writes and no finality of writes: No provision is made for writes to be applied to data via electric-next. Processing of writes has to be implemented separately from electric-next. This also means that ElectricSQL’s “finality of writes” principle is inherently not part of the current system design. However, it will be reconsidered later: “[W]e are committed in the longer term to building support for finality of local writes. However, it is no longer a key tenet of the system design.” |
The write path is decoupled from the read path of data: PowerSync allows the developer to write directly to the local SQLite database. Syncing with Postgres is optional, and if syncing is enabled, the writes to SQLite are also placed into an upload queue. The PowerSync client SDK then sequentially processes writes in the upload queue using an uploadData() function defined by the developer. Therefore, the developer is in control of how writes are applied to Postgres, typically via their backend API. PowerSync does not provide for finality of local writes. Writes can be processed on the backend server with any logic of the developer’s choosing, and all clients will update their local state to the server-authoritative state after writes have been processed on the server and syncing subsequently takes place (see here for details). |
Server-authoritative event-sourcing vs. CRDT-based architecture: There is no mention of CRDTs in the new system. In the future, once writes are included in the scope of electric-next, CRDTs might be reintroduced, similar to the legacy ElectricSQL architecture, but the details remain to be seen. |
Server-authoritative event-sourcing vs. CRDT-based architecture: Although local offline writes are supported, PowerSync does not use CRDTs as part of its protocol. Pending local writes are applied on top of the server-authoritative state, and once all local writes have been uploaded/processed by the backend, the local state is updated to match the server-authoritative state (see here for details). |
Loose coupling between client and server: The legacy ElectricSQL had tight coupling between the client and server (sync service), and several pieces of functionality were reliant on this tight coupling. By contrast, electric-next has much looser coupling, with a few notable impacts such as:
|
Somewhat loose coupling between client and server: PowerSync provides a set of client SDKs which work in conjunction with the server-side PowerSync Service to keep data in sync between the backend database and a client-side SQLite database, based on the Sync Rules configuration. However, some aspects of the system are similarly loosely-coupled compared to electric-next, for example:
|
Reduced state required on the server: The legacy ElectricSQL system relied on a significant degree of per-client state stored on the server (in-memory and/or persisted), which made the system very memory-intensive as it would scale to larger numbers of connected clients and rows synced. The new electric-next system requires the client to keep track of sync progress (by keeping track of its “offset” per shape). Since the new electric-next architecture does not rely on keeping an entire graph of data to sync for each connection, the inherent legacy scalability constraints have been addressed. |
Minimal state required on the server: With its system of buckets, PowerSync allows changes to data to be grouped together in order to deduplicate work. As a result, PowerSync does not have to keep track of individual client-side state on the server. The system can scale to high volumes of data and users with an efficient memory footprint. The PowerSync Service stores bucket state and operation history (indexed for efficient querying) in persistent storage and only stores limited information in memory. The “offset” per shape in electric-next is similar to PowerSync’s op_id per bucket which the client keeps track of. |
Differences between ElectricSQL’s electric-next and PowerSync
ElectricSQL's electric-next | PowerSync |
---|---|
No built-in SQLite support: Unlike the legacy ElectricSQL, the current version of electric-next does not sync data into a local SQLite database. The electric-next system only provides an HTTP API which exposes shape data. However, ElectricSQL’s adjacent PGlite project has a sync package available, which acts as an electric-next client (and uses the electric-next HTTP API) that can sync data into a local WASM-based Postgres database (PGlite) |
PowerSync automatically syncs data into a local SQLite database, where it can be utilized by the client application. Local offline writes to the SQLite database are also supported, as detailed below. |
Currently read-only; no handling of writes: Writes to data are not in scope for electric-next — it is the responsibility of the developer to build tooling for writes. This also means that if the developer needs support for offline writes, they need to implement that themselves. |
Tooling for offline writes as well as asynchronously uploading writes to the backend are built into the PowerSync client SDKs. |
Shapes are dynamically requested from the client, rather than pre-defined on the server. An HTTP API is provided whereby shapes can be requested and polled from the client-side. |
Sync Rules / buckets are defined on the server-side. This means that a separate permission / access control layer is not needed on the server-side, since Sync Rules serve as both authorization for access to data as well as defining which data to sync to which users. Future support is planned for allowing data to be dynamically requested from the client side, based on permissions / authorization rules defined on the server-side. |
Protocol uses HTTP instead of WebSockets. The current implementation relies on HTTP long-polling, but it appears that HTTP stream support is planned. The reason for using HTTP is to minimize state and to integrate with standard HTTP tooling such as proxies and CDNs. |
WebSockets and HTTP streams are supported. |
No provision for consistency: At the time of publishing this blog post, there is no provision in the electric-next protocol to get a consistent set of data across multiple shapes. |
Consistency guarantees across all synced data is a core part of the PowerSync protocol and client SDKs. See here for more details. |
Authentication and permissions require a proxy to be built in front of electric-next: see the documentation for details. |
Authentication makes use of JWTs, and permissions for access to data are controlled through Sync Rules. |
Flutter community client deprecated: The legacy ElectricSQL had an unofficial Flutter client that was maintained by members of the community. When the electric-next rebuild was announced, the maintainers of the Flutter client announced that the Flutter client can be considered deprecated. At the time of publishing this blog post, clients are available for TypeScript and Elixir, and a React integration is also available. |
PowerSync provides officially-supported client SDKs for Flutter/Dart, React Native, web/JavaScript (with React and Vue integrations), Kotlin Multiplatform (currently in beta) and Swift (currently in beta). |
Expanded use cases
With electric-next, ElectricSQL now aims to support a wider range of use cases, such as syncing to edge workers, pointing to a more general-purpose sync engine rather than one aimed exclusively at local-first application use cases. PowerSync does not currently have specific plans for such an expanded set of use cases.
Feedback or questions?
Join us on Discord to discuss.