Developer interest in local-first software is continuing to grow rapidly in 2024. At the same time, it’s also apparent that developers currently have different views on what they consider to be ‘local-first software’, ranging from narrower to broader definitions. This prompted us to take a closer look at the origins and evolution of local-first.
Local-first software: The original manifesto
The research lab Ink & Switch published the original manifesto essay which coined the term ‘local-first’ in 2019, authored by Martin Kleppmann, Adam Wiggins, Peter van Hardenberg and Mark McGranaghan.
Ink & Switch’s rationale for local-first is to get the best of both worlds of stand-alone desktop apps (so-called “old-fashioned” software) and cloud software: “We would like both the convenient cross-device access and real-time collaboration provided by cloud apps, and also the personal ownership of your own data embodied by ‘old-fashioned’ software”.
The manifesto proceeds to define local-first as software that “prioritizes the use of local storage (the disk built into your computer) and local networks (such as your home WiFi) over servers in remote data centers”.
It also puts emphasis on the primacy of the local copy of data: “In local-first applications [...] we treat the copy of the data on your local device [...] as the primary copy. Servers still exist, but they hold secondary copies of your data in order to assist with access from multiple devices.”
Expanding on this, the manifesto identifies 7 ideals “to strive for in local-first software”:
- Fast: By accessing data locally, the software should respond near-instantaneously to user input.
- Multi-Device: Data should be synchronized across all of a user’s devices.
- Offline: The user should be able to read and write their data anytime, even while offline.
- Collaboration: Support real-time collaboration that is on par with the best cloud apps today.
- Longevity: A user’s data should continue to be accessible indefinitely, even after the company that produced the software is gone.
- Privacy: The software should use end-to-end encryption.
- User Control: No company should be able to restrict what a user is allowed to do with the software.
Decentralized vs centralized architecture
One of the sometimes contentious aspects of local-first is whether it necessarily requires a decentralized architecture to be considered truly local-first. The Ink & Switch manifesto states: “[T]o fully realize the longevity goal of local-first software, we want applications to outlive any backend services managed by their vendors, so a decentralized solution is the logical end goal.”
Yet if you watch talks from Local-First Conf, listen to interviews on localfirst.fm, or attend LoFi meetups, there are several systems that describe themselves as local-first despite using a server-authoritative instead of decentralized architecture (including our own product, PowerSync). Or at the very least, many have a critical dependency on servers for parts of their lifecycle. What gives?
I would argue that a big reason is the ultimate complexity of local-first as defined by the 7 ideals: there are many thorny challenges that the developer community has yet to solve in order to enable usable and practical decentralized architecture. CRDTs address one aspect of decentralization on a data structure level (deterministic merging of updates without central server authority), but there’s much more to it. (More on this later.)
Local-first is an ongoing movement to evolve software
That being said, prominent voices in the local-first community (including co-authors of the original manifesto) have made it clear that they believe that local-first software is a movement in pursuit of a worthy idea, and that they expect it to take years to solve the big challenges. They also encourage developers to innovate freely and bring different approaches to market in gradual pursuit of the “end-goal” of local-first.
Adam Wiggins’ commentary
Wiggins gave a talk at the LoFi/14 meetup about local-first economics, wherein he described the Ink & Switch paper as “full of lofty principles” that asks for “a seismic shift in data persistence”. He noted that vendors offering end-user local-first software today and monetizing using a SaaS model mostly have an “authoritative server in the middle”, and that “everything to do with peer-to-peer, self-hosting and federation is still research”.
He called for “avoiding purity calls and No True Scotsman”, and said that he believes that it is fair to describe systems with a server-authoritative architecture as ‘local-first’ as long as they focus on achieving ideals such as user ownership of data and longevity. “That is a judgment call for each individual business”.
In closing, Wiggins shared:
“For me it’s a win if the parts of local-first that are production-ready make it into software that people are using in the real-world, especially at scale. And in the meantime, there's a huge role for the academic community and wider community [...] to take the parts that aren't ready for real-world usage yet [...] and figure out how to push that further.”
Martin Kleppmann’s commentary
Kleppmann gave a talk at Local-First Conf 2024 (similar to his talk at the LoFi/12 meetup) about the “past, present and future of local-first” wherein he said local-first can currently be considered “somewhat of a gradient where there’s a gray area of being somewhat local-first but not satisfying all of the ideals”. He added: “We’ve since come to realize that these 7 ideals aren’t really a good definition of what local-first is. They’re actually more about the [end-user] benefits that local-first can bring”.
He promised a follow-up to the original local-first manifesto essay and proposed a new interim definition: “In local-first software, the availability of another computer should never prevent you from working”. Kleppmann explains:
“If [the software] doesn’t work if the app developer goes out of business and shuts down their servers, it’s not local-first. What we’re trying to get at here, is robustness not only against availability in the network sense, but availability of another computer in the sense that it got shut down and deleted all of its data. [...] On some level, the software should be resilient against the creators of that software just disappearing”.
Importantly, Kleppmann acknowledges that this vision is not yet practically achievable: The existing ways of accomplishing it (e.g. self-hosting a server, peer-to-peer architecture, file syncing using mainstream tools) are not really easily workable in practice today.
However, Kleppmann goes on to propose a very interesting future solution to this problem, which he called “The Local-First Endgame”: What if an interoperable standard sync protocol based on an open standard existed that any vendors could implement, so that end-users could sync their data with interchangeable services? In this way, if a vendor goes out of business and shuts down their servers, end-users can seamlessly switch to an equivalent service.
This is a very appealing hypothetical solution. It would go a long way towards Ink & Switch’s local-first ideals, although there would still be many complexities left to solve (as we will explore a bit later).
Kleppmann is of the opinion that it’s too early to start building this kind of standardization layer. “Right now, we don’t understand the trade-offs well enough yet. But in a few years’ time [...] we can start our standardization committee.”
In the meantime, he is encouraging experimentation:
“I’m very open-minded to lots of people building different technologies. I think that’s a great thing that there’s such lively activity in this ecosystem. The simple answer is that we don’t know yet what will work and what won’t work. We need those thousand flowers blooming so that we can gain experience of different approaches and understand the trade-offs better.”
He also stressed that “local-first is a set of principles and values, it’s not any one technology”.
The implication of Kleppmann’s proposition is quite profound: It means that the practicable way of achieving the promise of local-first does not yet exist and won’t exist any time soon.
So what is ‘local-first’ in reality today?
While the end-game vision of local-first is still a work-in-progress, what would be a good description of actual “local-first” implementations today? In my view, I would say the following: (and other developers hold similar views)
Local-first implementation today generally refers to apps that work with a local client database which syncs automatically with a backend database in the background. All reads and writes go to the local database first.
We could also state it more generically: Apps with implicit automatic state transfer in the background between backend and end-user device.
This architecture holds the promise of moving towards Martin Kleppmann’s envisioned architecture of “interoperable sync services” in the future, which would bring the software closer to achieving the local-first end-goals.
Isn’t that about the same as offline-first?
‘Local-first’ implementations that exist in practice today are indeed typically very similar to many offline-first implementations, a term coined in 2013 (“[F]or a proper offline-first experience, you [...] want to store the data in the front end, and you want it to sync to the server’s data store”).
This is not surprising, since the authors of the local-first manifesto spoke of local-first as a superset of offline-first. As Kleppmann says: "Offline-first is a part of local-first. You can't be local-first without being offline-first".
Looking at Ink & Switch’s 7 ideals for local-first, the first 4 correspond directly to cloud-backed offline-first software, whereas the last 3 ideals are what makes local-first unique from offline-first. (Also see our short explainer video on local-first vs. offline-first.)
Even ‘just offline-first’ is already a big win
I don’t think it’s a bad thing that local-first implementations today are largely similar to offline-first implementations.
As someone who has been at the coalface of building offline-first systems since 2009, my impression is that offline-first didn’t gain as much traction in the 2010s because it was too difficult or engineering-intensive for most teams to implement, despite organizations and individual developers being well-aware of its benefits (e.g. speed and network resilience for end-users).
Therefore, in a sense the local-first movement is reinvigorating the adoption of offline-first and this is spurring a new wave of innovation in tools such as sync engines, which I believe will cause “local-first” architecture to become mainstream in a way that offline-first did not. And this would already be a win for software quality.
In fact, I’d even venture to say that the average software end-user will probably care more about the end-user benefits stemming from the first 4 ideals of local-first than the latter 3: All end-users hate loading spinners and errors, and they expect to be able to access their data from anywhere and collaborate with other people. Hence, cloud-backed offline-first software gives them benefits that really matter to them:
On the other hand, I feel that power users are more likely to care about considerations such as whether their software will continue working years in the future when its creators disappear. I think everyday end-user concerns will certainly evolve over time, e.g. end-to-end-encryption (related to #6 above: privacy) is arguably gradually on the rise as a concern of the average end-user in light of increasing high-profile data breaches.
In my experience, apps from the top software companies in the world who have virtually unlimited engineering resources (Google, Meta, etc.) still suffer from loading spinners and network-related error states in various spots in urban areas, despite aggressive caching and use of content delivery networks in edge locations. Therefore, apps with local-first/offline-first architecture which use a local data store and sync with servers in the background will have an immediate positive end-user impact.
It is however true that the need for network resilience may slowly lessen over time as mobile network connectivity keeps improving, and technologies such as Starlink become more ubiquitous. Yet I don’t think that will slow down the adoption of local-first, because of one big and very important reason:
The developer benefits are by far the biggest win of local-first
I believe that the developer benefits of local-first are a bigger deal than the end-user benefits, and will be the biggest driver of local-first’s growth as a software movement and architecture.
I wrote about this in 2023, highlighting benefits such as:
- Reduced backend complexity
- Reduced backend compute load, dependency and cost (which also leads to increased robustness)
- Simplified state management (local-first essentially provides “magic state transfer” from the perspective of the developer)
- Real-time collaboration being built-in to the architecture.
Subsequently, I’ve noticed it increasingly coming up in discussions around local-first. Here’s just a small sample of recent comments:
- Kleppmann at Local-First Conf 2024: “This is another piece that we missed in our original local-first essay 5 years ago: [It] focused very much on the [end-user] benefits, which is important, but since then, we realized that actually the benefits to the app developer are perhaps at least as big as those to the end-user”. Kleppmann goes on to mention benefits such as: No backend engineering team needed, no 24/7 on-call rotations to keep the backend running, no more writing network error handling code, no more “paying through the nose” for cloud computing, and overall, “much simpler app development”.
- Tuomas Artman of Linear mentioned similar benefits at Local-First Conf 2024, and went on to say: "I'm sure I won’t ever go back to working any other way. The benefits of going local-first as an engineer are so humongous that I wouldn't want to waste my time handling network errors and making network calls.”
- Søren Bramer Schmidt: “[With local-first], we can have a better, faster and easier way to build applications that also delivers a much better user experience.”
- André Terron at LoFi/16: “I get to interact with fewer layers, [I get] features for free such as collaboration, data versioning, offline-support, multi-device [...]. And it lets me build higher quality apps for less effort.”
The impact of the developer benefits is mainly why I’m very bullish on the growth in local-first adoption that lies ahead.
Why are the ideals of local-first difficult to achieve?
As I alluded to above, there are significant technological challenges that need to be solved for the local-first 7 ideals as envisioned by Ink & Switch to be broadly achieved. Here is just a small taste of some of the topics of ongoing exploration and research: (many of which are mentioned in the Ink & Switch essay)
- Authentication / user identity in a distributed environment.
- Distributed authorization and permissions, especially as it intersects with collaboration and versioning: users should be able to share subsets of their data with other peers with varying permissions, and have the ability to revoke access, without servers mediating the interactions. In addition, users can go offline periodically and keep working. The essay does a good job of highlighting several of the complexities involved with this (see for example the section discussing the “kaleidoscopic complexity to our data”).
- Peer-to-peer communication and syncing without mediation by servers, or syncing with interchangeable backend services using a standard protocol (as envisioned by Martin Kleppmann).
- Versioning of apps and schemas (and schema migrations) in a decentralized context. “Different collaborators may be using different versions of an application, potentially with different features. As there is no central database server, there is no authoritative ‘current’ schema for the data.”
- Change history accumulation.
At PowerSync, we are keeping a close eye on the developments in these domains, and will contribute to research and experimentation where it makes sense for us to do so. In the meantime, we have documented how developers building with PowerSync can bring their software closer to the ideals of local-first.
Looking ahead
Local-first is a very promising software movement which will make a big difference in the quality of software, as well as the developer experience and productivity of building software. We’re excited to play a part in it, and look forward to seeing where it will go in the next few years.
