Last week, I visited San Francisco to meet up with Johannes Schickling, co-founder of Prisma and an advocate for local-first software who is behind projects such as the localfirst.fm podcast and Local-First Conf, both of which we’ve sponsored.
Johannes and I share a belief that local-first (or at the least, the use of sync engines) will eventually become the dominant way of building apps, because of its benefits to end-users and developers alike.
This led to an important question: What will make the use of sync engines spread like wildfire and achieve mass adoption? Are there key factors that will cause it to accelerate, and what can we do to catalyze it?
Johannes remarked that when React came around in the mid-2010s, it was obvious to almost everyone that it was superior to the old way of building apps using jQuery (which was arguably the dominant front-end library at that time). As Johannes said, “It became clear that 99% of web use cases are well-served by the React model”.
What we are looking out for is sync engines achieving a similar breakthrough. At the moment, most of the available sync engines are still fairly immature, and there are difficulties with using them. But as the tools improve and achieve breakthroughs that strike the right chord with developers (in terms of mental model, capabilities, developer experience, and so on), it will be easier for developers to recommend sync engines more generally due to the obvious cost vs. benefit, and we believe adoption will take off.
From Imperative to Declarative: jQuery to React
The transition from jQuery to React as the dominant front-end JavaScript library is a very interesting analogy to think about:
Imperatively manipulating the web page DOM was central to using jQuery (step-by-step finding of elements, adding/removing classes, changing text content, showing/hiding elements, etc.).
Dynamically updating the page [1] was amazing in the early days of interactive web apps, but as web apps became more sophisticated and complex, managing the state of the DOM in response to changes in app state (e.g. events) became unwieldy very quickly, and large apps became quite painful to develop and maintain. It was not uncommon to have lots of spaghetti code for updating the UI. In addition, the complexity of front-end performance optimization was left in the hands of the developer.
As a lot of developers experienced that pain, the benefit of React was obvious when it arrived on the scene in the mid-2010s and introduced a declarative approach to building web UIs. With React, we describe what we want the UI to look like at any given point, and React takes care of updating the actual DOM to match that desired state. We write components that represent the UI at various states and let React manage how and when to update the DOM based on changes to component data (state and props).
This solved several aspects of the pain of jQuery: The code is more modular (through the use of components), more predictable and easier to reason about (since React abstracts away direct manipulation of the DOM), and more performant (because of React’s approach of computing a virtual DOM and updating the actual DOM with only the delta between current and previous states).
From Imperative to Declarative: API Calling to Sync Engines
Web/cloud apps are fundamentally a distributed computing environment, where we have to constantly move data between servers and clients (state transfer). The vast majority of web and cloud-backed apps use some form of RPC-style APIs to transfer data. Even with all the advances over the last two decades, such as REST, IDLs, GraphQL and RPC frameworks, moving data around using these various forms of API calling is still fundamentally imperative, requiring developers to:
- Control how and when APIs are called.
- Handle error states and retries, especially due to network unreliability.
- Implement caching and keep track of data freshness or staleness. (This is a slippery slope that quickly devolves to essentially building a database in the front-end, especially if you layer in optimistic updates.)
- Optimize various aspects of performance
Most application developers burn many hours on this sort of undifferentiated busywork, and there’s plenty of pain involved.
Moving from API calling to sync engines is a very similar shift as moving from jQuery to React was: by going from an imperative to a declarative system where much complexity is abstracted away [2], we can take away a world of pain.
With the right sync engine, the developer can declaratively define what data should be present on the client. The sync engine then automatically takes care of the complexities of partial replication of the data, including all the complexities of dealing with the network (streaming real-time updates [3], error handling, retries, online vs. offline state, etc.), reconciling local updates with remote changes, conflict resolution (in many cases), optimizing syncing performance, and so on.
A good sync engine will expose a local database like SQLite which enables the developer to deal with a real local database in the front-end — significantly reducing the complexity of working with data. The sync engine ensures the local database stays in sync with the backend database as needed, just as React ensures the UI/DOM stays in sync with the application state.
This is the future that we are trying to help usher in. Many developers who are already using sync engines today can attest to how much of a better experience this is.
