Johannes Schickling explores how he moved beyond the traditional three-tier web stack to a local-first approach. He shares his experience transitioning from Prisma to developing Overtone—a music curation app leveraging client-side event sourcing and SQLite. The discussion highlights the trade-offs between event sourcing and CRDTs, providing actionable insights for architects weighing the complexities of local-first development.

Key Takeaways

• Local-First is more than offline support; it's about shifting the data bottleneck from the server to the client to enable high-performance, responsive experiences.
• Choosing between event sourcing and CRDTs depends on your specific application needs; event sourcing offers robust history, while CRDTs excel in collaborative conflict resolution.
• The transition to Local-First requires rigorous upfront planning, particularly in breaking down monolithic data structures into smaller, manageable units.
• Not every application is a candidate for Local-First; assess carefully if the user experience benefits — like offline capability and data ownership — justify the architectural complexity.
• Data interoperability and ownership are critical; modern Local-First architectures like LiveStore provide frameworks to keep data portable and secure while maintaining local speed.

Subscribe on:

Transcript

Olimpiu Pop: Hello, everybody. Olimpiu Pop, an info curator, and I have the pleasure of discussing today with Johannes Schickling, and he's one of the people involved with Local-First Movement Conference, and comparing to the other discussions I had with Martin Kleppmann and Adam Wiggins, today, we'll go all in and understand what's the difference of the mindset that you have to shift from before having Local-First and Automerge and what changed with Johannes after he had that. But without any further ado, Johannes, please introduce yourself.

Johannes Schickling: Hi, I'm Johannes Schickling. I live here in wonderful Berlin. I'm originally also from Germany, just moved a little bit further north. I, in the past, am the founder of Prisma, which was all about server-side database tooling. And since 2020, I've taken a step away from Prisma and been more exploring software and the future of software by myself. And in 2020, I went all in on Local-First, which are ideas by the friends of mine at Ink and Switch, and been happily exploring since and now combine those ideas of Local-First with all the tailwinds we got from AI.

Rethinking Architecture for Superior User Experience [01:19]

Olimpiu Pop: Great. As we discussed before, Prisma got quite important or I don't know how to put it. People use it in the JavaScript/TIScript ecosystem. So, how did you get started with it? What were the challenges? If you look back, what would be the aha moments that you had while getting started?

Johannes Schickling: Yes. I mean, looking back at me now spending probably somewhat around 20 years with computers productively, there's always a common theme of me trying to build something, and then, me getting frustrated with the tools I have to build it. Even before computers, I always wanted to build something, and then, I kind of got sidetracked and rather build my own tools that I can use to build whatever I want to do. And this is kind of like the recurring pattern. Prisma was just like one instance of that where a decade before, I was just very frustrated with the database and API tooling, et cetera, always being the bottleneck to build anything. In the past, I've used a lot of PHP, et cetera, and there's a popular framework called Laravel, which is still pretty big today. And this is where I saw how good database tooling could be, also with Ruby on Rails, but then, using JavaScript mostly, and then, TypeScript also emerging.

There was nothing really comparable that was meeting my goals and my hopes, but how good this could be. And particularly, the emergence with TypeScript felt like the right moment to take another step, go all in on database tooling that embraces type safety with the goal of just having better tooling to build better applications. And I think that's still the true part today, that you need great tools to build great applications. But compared to Prisma, these days, I'm also rethinking the architecture quite a lot where Prisma was all about that traditional architecture, typically like a three-tier web stack where you have a centralized database somewhere where you have an API tier, maybe you have a client tier. But I wanted to think through a bit more, how can we actually unlock the best possible user experience? What sets the highest bar? I think it's mostly like desktop software, maybe very well-built mobile software where whenever you click on something, something immediately happens.

Ideally, the software works no matter where you are, whether you're on a plane. Think for example, an app like Apple Notes. That works just super reliably under any circumstances, but you can't quite say the same thing for your average web app. And I wanted to flip that and understand like, hey, what is the problem? Why can't we have the same excellent experience with the most dominant platform we have today, which is the web? And I got the insights into the direction to pursue through Ink and Switch, my friends there, which then have written that excellent Local-First paper or essay. And this was as important to me that I shifted my main focus to pursue this topic.

Olimpiu Pop: Okay. So, you adhere to the ideals because that's how the other members of the Local-First saw it as a couple of ideas and tried to do things properly. I'm thinking now about the discussion that I had with Adam Wiggins, and what he was saying is that, well, he's known for putting together Heroku, which has cloud written all over it. And now, it seems that is the polar opposite, but what we agreed upon is that it was this movement. People were used to having things on their local machines. They got to the cloud, and then, they didn't have the same experience, so something was missing. And Local-First is that sweet spot for each and every one of us when we're building something that allows you to have the same ideal experience, utopian, if you want, but with the benefits of having the ability of having it distributed and not losing data if something happens to the copy on your machine.

Local-First as a Radical Shift in Data Philosophy [05:32]

Johannes Schickling: It's hard to think about this in terms of a spectrum. I think it's rather kind of very radical exploration. What if we flipped a lot of the things? Local-First is all about data. And like any sort of applications, there's things you can switch out more easily. Maybe you switch out like an Apache HTTP server for an Engine X HTP server, et cetera. That's relatively easy to do. But when it comes to radically changing the data architecture of your application, it might still be easy enough to switch from MySQL to Postgres. But when you really change where does data live, where does it create it, how does it flow, et cetera, that suddenly becomes one of the hardest things you can do when you build an application. Even today, with AI and agentic engineering, et cetera, that's still one of the hardest things to change and, I think, one of the highest leverage ways as a software architect, you can influence the foundations for an application whether it feels good or whether it feels terrible in many ways.

Local-First is just like a radical departure from at least the status quo of typical web apps. In a way, it's also like back to the future to where things started in a way since most computers back then were un-networked. So, by definition, everything had to be on that computer. And then, step by step with the internet, we've shifted that into the cloud. And this is where all apps were thin shells that only communicated with a fat server. But I think Local-First wanted to see like, hey, can we have the best of both worlds? Can we have the connectivity of cloud software where it's always available where I can maybe buy a new computer, log into everything and everything is still there? That's not a big deal. But also, where we can have really high expectations for the applications that they work in any circumstances. It's also that I don't lose my data.

I'm sure like Martin and Adam, et cetera, they have done a great job of motivating the ideals of Local-First. But for me, it's all about really the quality of software, pursuing the highest quality of software, but that's never done in a vacuum. That's always relative to what the certain application is. If you're building Google Maps, you have very different trade-offs compared to if you're building Airbnb. But there's like for Google Maps, being offline in a place where you've maybe pre-cached is a very important thing. Whereas for Airbnb, other trade-offs might be very important. So, I think it's always important to think through which sort of trade-offs can you apply for the kind of application that you want to build.

Olimpiu Pop: Okay. So, you did a great job of hiding the it depends between other parts because that's what I heard you say. It all depends on what you want to build. And then, based on that, decide on your trade-offs and just make sure that you have everything that it takes. And I'm very happy that you didn't say it because as a guild of engineers, we have a tendency of always hiding behind, it depends. So, you didn't build neither a clone of Airbnb, you didn't build a clone of Google Maps, but you built an audio application or a music application, if I got it right.

Event Sourcing as a Foundational Technology [09:03]

Johannes Schickling: Yes, that's right. I mean, this is another big passion of mine for a long time is music. I used to be a DJ in the past as well. And we can see that progression of software playing out there. I started building a music library with vinyl records, but also, when I'm back in the days, et cetera... And that was really an important part of who I was, like a big hobby. And so, I was proud to have a music collection. Then, over time, we got things like Spotify and YouTube, et cetera. I still use it on a daily basis, gives us a lot of convenience, but we've kind of lost the aspect of like, hey, this is my music collection. I felt curious to see can a better client layer on top of wherever that music lives, can that fundamentally improve the experience?

And I was sitting on this idea for a very long time. And when I then, through Peter and Adam, heard about the ideas of Local-First, this is where it clicked for me and I felt, okay, I can actually now try to build this. So, Local-First was sort of like the catalyst for me to be bold enough to try to build this. So, it's sort of like a third-party client for wherever music lives in the same way as how you can use Superhuman, for example, with your Gmail account. But then, in order for me to build this, I started on that in 2020. Automerge was very, very early on. And as it all depends how you architect that data, there was one thing I felt in particular strong about with sort of a technological trauma from Prisma, if you want, which is database migrations. It's not just like how does data move around, like where is it stored? But also, how does your database schema and your data model, how does it evolve over time?

And I felt very compelled by one particular idea or technical architecture, which is event sourcing. I think event sourcing has always been explored more on the server side, maybe in the financial space, but I also saw a ton of potential to apply it directly on the client side. There's technologies, for example, Redux, JS, et cetera. We're pretty big with React, but I feel like this idea was still not pushed far enough. And this is where I wanted to see, hey, can we actually unlock the ideals of Local-First instead of with CRDTs? Can we apply the ideas of event sourcing on this? And this is where hand in hand, as yin and yang, I started building the music app, but I also teamed up with Nicholas Schiefer and Geoffrey Litt on a research project back then called Riffle.

Nicholas and Geoffrey both respectively did a PhD at MIT at that point. And this was basically the idea of like, hey, can we build another take on Local-First framework, which was SQLite-based and event sourcing-based? And that research project, I then carried forward and turned it into a project that's now called LiveStore. And that is the data foundation for my music app, which is called Overtone. By now, LiveStore is an open-source project anyone can use. It's still not fully production-ready, but it's getting there, and it's nice to see a community being built around it. And it still pursues the same goals as something like Automerge to build Local-First software just with a different technological trade-off. That's the way how I'm exploring this for myself, but it's important for someone who's new to Local-First to really explore the circumstances of the app. What are the trade-offs you can afford to make? And then, choose the best technology for you, whether that's being Automerge, whether that's being one of those other amazing technologies out there.

In fact, I'm also the host of the Local-First Event podcast. And for the Local-First Event podcast, we also did an initiative a while back to create a Local-First landscape, which we can maybe link to. This gives you a nice overview of at least back then, all the different Local-First technologies out there, trying to compare them across many different dimensions, et cetera.

Olimpiu Pop: Yes. That was definitely a technology soup. It was very healthy.

Local-First: Bridging the Gap Between Desktop and Web [13:37]

Johannes Schickling: Yes. I think the domain here of a music app, et cetera, me being a DJ as a hobby-

Olimpiu Pop: Pure coincidence.

Johannes Schickling: ... is like that just my personal motivation. But I think the more important part is this idea of an app was just really unlocked. It just became feasible with this architecture. I think in the same way, is if you think about a few technological applications that feel just distinctly a step forward. For example, if you think about Google Docs, or Figma, or Linear, all of those have in common that it feels like a distinct step forward in terms of the client-side experience. Whether for Google Docs, that you saw someone else's cursor and you could collaborate. Whether it's for Figma, kind of the same thing, but now, for design on a canvas where you could collaborate. And you couldn't just collaborate, you could do all of that in the web.

And this requires a fundamentally different data architecture compared to what your typical JavaScript web app was traditionally capable of. And something like Linear also shows how much better an application can feel compared to Jira or GitHub Issues. This is what it's all about for me. And I wanted to see, can we build the equivalent of like a Linear or Figma, but for music apps? That was just my passion and what I wanted to explore. And I know that data is always the bottleneck for a great user experience, particularly given that we have pretty solid client-side view frameworks these days, whether it's like React, or Solid, or Vue. All of those things are pretty good these days. But in terms of how you manage data, that is still typically the bottleneck. And this is what leads to using loading spinners or maybe you fill out a form, and you need to reload the page, and everything is gone. It's typically data that is still the last remaining frontier that we need to figure out. And this is what Local-First gives you a really compelling vision for.

Olimpiu Pop: That's great perspective. But what I am curious now, what does it mean? And that's why I'm insisting on Overtone because it's a bit different than everything else. Because all the other examples that you mentioned, Linear and also Figma, they are all translated into probably some kind of models. I don't know the particulars, but in case of Linear, it's probably basic stuff in terms of project administration, so on, so forth. So, mainly textual. In case of Figma, I would expect that there are, again, positions and stuff like that. So, a bit more complicated, but still something that it's easy. But music, it's a bit different because we are very careful. I have friends that were just making me listen with different type of headsets. And through all this kind of walls of wires, and they were just insisting that, "Okay, these are silver cabling", and, "Okay, I don't feel the difference". And they obviously, they send me-

Johannes Schickling: Audiophile friends.

Olimpiu Pop: Exactly. They just sent me out of their listening rooms because I just spoke nasty about their altar. So, that's why I mentioned it because what's data in case of Overtone? You did mention that LiveStore, it's based on SQLite. So, probably that's meta information that you're just gathering, and list, and stuff like that. But does Overtone take care of music as well?

Johannes Schickling: Yes, that's going to be a disappointing answer as this is not a part of Overtone that I'm trying to innovate on. For me, it was really about the metadata. I want to integrate with wherever your music lives. So, whether it's on Spotify, and then, you stream however Spotify allows that or whether you've bought music somewhere, and then... That was not necessarily the part that I spent a lot of innovation tokens on. For me, it was actually still more the metadata layer. But for music, that metadata layer is so rich that I really didn't want to make any compromises there. This is also for me as a DJ, I wanted to introduce my own tagging system, et cetera, and just annotate the music a lot. And for that, I just wanted to create my own layer inside of the app, sort of like a meta schema layer that's expressed in the app by the end user and not by the developer.

LiveStore - An SQLite With Sync Engine Capabilities [18:06]

And in order to facilitate that, this is where you need a very powerful data foundation. This is where I felt event sourcing was a great foundation for how data changes. And then, combined with SQLite as a Vue model that we've added reactivity to so the app can in real time, query all the data. Those were like the two key parts of innovation. One is making SQLite reactive and fast enough that it can actually run in the same thread where your front-end web app lives. And then, also building a sync engine by embracing the ideas of event sourcing and bring that together. This was the foundation, what led to LiveStore.

Olimpiu Pop: Okay. So, that translates that you do have buckets of music that you stack all over a given period, being it's Spotify or anything like that. And you want to just start tagging and add information that you would like and stuff like that, right?

Johannes Schickling: Exactly. And another key part of this is this data can get quite messy, and it integrates with data from a lot of different sources. This is where it maybe is different compared to Figma or Linear where each Linear issue, you have probably created ones. Whereas for all the music, I didn't create that. I just have it in my library. So, it basically becomes additionally, a huge external data integration problem or you have an incredible set of external data sources that can also change in very unexpected ways. For example, Spotify might remove something from their own catalog. What should happen locally? What I've decided is I still want to have sort of like a tombstone of a track where all the tags that I've added is still there that maybe if I have the same track available from another source where it's still there, that it's like de-duplicated, et cetera.

And this makes it a very important aspect of how can you keep that data sane over time? This is where event sourcing was a great foundation compared to CRDTs where you can also make changes to your application logic, and then, just replay throughout the entire history instead of mutating an object. But this gets kind of now rabbit-holey into the ideas of LiveStore and event sourcing. But if we're talking about Local-First, it is more important that I just wanted to provide the best possible user experience to users who what is important to them is they use Overtone to curate their own music library. And the worst thing that could happen there is that their data is gone, or that data is wrong in some way, or data is lost.

Olimpiu Pop: It's corrupted.

Johannes Schickling: This was for me, the highest-order bid basically, additionally to unlocking the best possible user experience where the app feels as fast as like a triple A game.

Olimpiu Pop: That means that the app itself is a web app, which is very, very fast on the local machine, but it has a centralized place where the data is actually safe. So, if something happens, you do have a place where the data will come back. It's not stored on your machine and which your machine is dying, right?

Johannes Schickling: It's both. So, in this particular case, it has a couple of different origin points. Obviously, whether my app exists or not, some data lives in Spotify or Spotify drops it at some point. That's something I need to integrate with. But in the app locally, all of the enrichment happens locally, and it's just synced into a sync backend that in case I use this on a different machine or on the phone, et cetera, it gets synced. But all of the data that actually has the source of truth in LiveStore, the source of truth has been the client. It's just that now a sync server helps facilitate to propagate and back up that data. In that regard, you basically get the best of both worlds where you have a client that never has to reload some data if it already has seen something. But if you, for example, need to set up a new computer, you can re-sync everything you want and you get real-time collaboration as well.

Olimpiu Pop: That's nice. Okay. So, just to make sure that I got it, whenever you make a change on your local machine, an event is generated, and then, it's stored locally, and then, probably propagated through a sync server somewhere on the web. And also, the other way around, if somebody else is using another instance or whatever, you have it on your phone. You do something on your phone. It's going the same route, and then, everything is synchronized.

Johannes Schickling: That's right. The way how it works is very similar to how Git works. In that regard, that you have sort of like a push-pull model. And with Git, typically, if you try to push something and the server's already had, you need to pull first rebates or you need to force push. LiveStore works in a very similar way. But fundamentally, this is where I think Martin is probably the data godfather in that regard about distributed systems and so on. Fundamentally, we're dealing with distributed systems here, and we need to figure out some trade-off somehow, and CRDTs figure them out in some way. Event sourcing takes a different path. Fundamentally, it's very similar ideas, just differently exposed. Martin has given a great talk at some point about turning the database inside out, which is what is event sourcing. And this is the path I've been taking.

But besides CRDTs and event sourcing, there's also different approaches that folks explore in the Local-First world and syncing world where it's more about having an authoritative centralized database that's a bit more traditional to your typical Ruby on Rails stack, et cetera, where the idea is rather that you have a sync engine based on queries that your client defines, just stream down the data that is the result for that query. And based on that, have sort of like a always up-to-date smart caching layer. That's like another approach that's been taken by a couple of really interesting technologies where it's like Zero, or ElectricSQL, PowerSync, et cetera.

Olimpiu Pop: Okay, great. Now, let's say that you share your experience. You have the experience with building Prisma, and what you took from that, you used while putting together Overtone plus building the LiveStore together with your friends. But now, you have the two options. What would you advise somebody that wants to embrace Local-First for their apps? And what would be maybe the limitations that are still available?

When to Adopt Local-First Architectures [25:12]

Johannes Schickling: I think the first question that someone needs to figure out is should they really build their app in a Local-First way or not? Local-First is not just better. And in the past, you've built this in sort of a more cloud-centric way, and now, Local-First is here, and now, clearly, you should build Local-First. That's not the case. Building apps in a Local-First way comes at a cost in multiple ways. You need to think harder about your data architecture. You need to think more about where does data need to be available? In which way? How will it grow? For example, when Uber started out, maybe in some cities, the database was much smaller than when they rolled out to all the cities across the world where the database is much larger. This is where they needed to rethink their data architecture all the time, but it was easier because the data architecture was just a backend concern and not really like a concern for the client.

With Local-First, you're making the client a much more fundamental aspect of the data architecture from the start. And if something grows in a much more unbounded way, that really needs to be taken into consideration. So, with Local-First, you need to think much more about how can you slice, and dice, and break down different parts of your data model, of your data layer? You just need to think a lot more upfront about that, and then, see, is that feasible? Is it worth it? And only if you think that the user experience that you want to unlock with that, how important are those ideals of Local-First software for your particular app? If they're very important, then go for it. And then, step number two is figure out the right data architecture, and from the right data architecture, figure out the right technology, whether you want to use something, when you want to use CRDTs, like exploring Automerge, exploring Yjs, Loro, et cetera. If you want to use a different architecture, maybe explore LiveStore, other technologies.

But I would really not skip the first step. It's like, is Local-First the right thing for you to begin with or not? You should find the reason why you shouldn't do it basically. And only if you've said, "No, I still want it. It's still worth it for us", then continue. Since it's not like a decision of like, oh, which web router am I using, that you can much more easily change later. This is going to be probably the most fundamental decision you make about your application.

Olimpiu Pop: Do you have a quiz or decision tree? Like, okay, do you want to use Local-First? Yes. And then, other questions that people should respond? What questions would you ask yourself?

Johannes Schickling: So, how important is it that your application works offline is one. And you can also think about it synonymously, how important is it that your app works on the non-ideal network situations? Maybe you don't even need to be technically fully offline in terms of going into the desert where there's no connectivity, but maybe you're just in a train. I was in Japan recently in the wonderful Shinkansen trains, and they're amazing, and you have internet most of the time, but then, sometimes you don't. In that case, when you have poor connectivity, it's actually worse for an application than being completely offline. This is an extension of how important is it that your app works offline or under poor circumstances? I think that's more and more common as we build apps that maybe work in a IoT scenario or that work for workers somewhere, et cetera.

So, another one would be how important is it for end users to own their data? For example, for me with the music app, that was actually quite important to me that I can own my music library. Basically, check out which of these ideals you feel like you need to have and you cannot live without. And chances are that Local-First is the best way to get there. So, when you have a strong enough reason that you cannot live without it, then we can basically see, okay, let's figure out how we can best build and model your app and architecture in a Local-First way. And I think this is where you need to be very rigorous on how does my data model look like?

So, instead of by default, having one big Postgres database, you rather want to think how can we make it many, many tiny pieces? So, for example, for a Linear app, how can we break that down? We can break it down by a Linear issue. Each Linear issue might be a self-contained boundary that has certain metadata, and now, you have instead of syncing the entire big Postgres database for all Linear issues across all workspaces for all customers, et cetera, you just have a much smaller unit that you need to think about, and basically, break down your entire data model into smaller units. Or maybe have smaller containers that contain references to other units, et cetera.

If you figure out a nice way to break it down this way for Figma, for example, is like the canvas for one particular design, et cetera, or a workspace might be a container for something. If you manage to break it down this way, you have good chances to make this work. If you built rather something that feels like a social network where you can't really draw that boundary where everything is connected with everything, this is a sign that you don't have a good use case for a Local-First at your hand.

And so, yes, basically, go through this exercise, and through that exercise, you will also realize, oh, okay, where does data change? How often? How high is the variance of how much that data grows? Then, maybe also find a stopping point where you don't want to overanalyze everything and just build a first version. For example, take Automerge, build a first version with it, see how it feels, and try to time box that and see like, okay, maybe there's one thing that just makes it impossible. And then, think through like, hey, can you address that in a reasonable amount of time? And if not, maybe try out a different technology or worst case say like, "Okay, I thought my app worked for Local-First, but this is not worth it". And then, you get out.

Olimpiu Pop: So, we figured that out. So, what you're mentioning is how do we split the data? Because in the classical example, as you mentioned initially, you do have a huge bucket, that cylindrical block on all the diagrams where you have everything is monolithic, and then, you have other approaches in terms of keeping the data safe. So, probably, you have to think of a way how you can break down the data in the first instance, smaller buckets and so on, so forth. And once you have that, you can try to do it. In order to just answer your question is like, okay, go to that step, and then, ask yourself, do you still want to be Local-First? And then, "Yes, well, I want to be Local-First". So, now, as you mentioned, it's about the model that you want to have. On one side is the perspective that LiveStore provides, and that's event sourcing. And that's, let's say, an array of events that are just being able to just move from one side to another, and then, having those will allow you to replicate the state of your application at any given moment of time.

Or it's the part where you want to base yourself on CRDTs. And just to make sure that the acronym is clear, conflict-free replicated data type. From my understanding, correct me if I'm wrong, it means that I have the same pockets of data, but I have the ability of merging them together without any conflict, right?

Event Sourcing vs CRDTs [33:19]

Johannes Schickling: Basically, it's a matter of which data type you have. Let's say we are looking at objects record data type, like a smart JSON thing basically where you can locally build your app where essentially you have the abstraction of a mutable object, for example, or a mutable array, or a mutable set, and you can just use it the way how you normally use it. And the CRDT makes sure that if the same app runs on two different machines with different users, they always see the same thing when they're connected. That's kind of the magic of CRDTs. The reason why I wanted to go a step further is because mutable data is sort of typically the source of all evil and where it gets really hard to track down where does change come from, how to semantically reason about what should happen, which situation?

But as a starting point, Automerge and CRDTs are a fantastic foundation and often good enough for most scenarios. If you have a data type of a number where you just want to do plus and minus or a set where let's imagine like GitHub labels where you add a label or remove a label, and it matters much more about what happened before, Automerge is all you need or CRDTs. I don't want to pick any favorites here. It's more about which data architecture, and then, you pick whatever technology speaks most to you in that data architecture.

Olimpiu Pop: Well, I don't think it's about favorites at this point. There are different ways of skinning a cat, and we know that there are multiple ways of fixing a problem. And that's what it means for me. Because in the end, event sourcing, you can build a system. I don't know, just thinking about IoT, you can use it based on events, but on the other point, you can use it also with smart data, with non-conflict data types, because in the end, you have the ability of putting those things together. It's a matter of taste. It's a matter of what tools you have or maybe your experience. So, it's not these are the two options that currently are the most popular ones. What would you advise against? I mean, what would be some limitations that people should know about? Where do you think it's not feasible to use Local-First at this point?

Johannes Schickling: I would say if your end user experience is really important, this is a good indicator for where Local-First is a real good foundation. If the end user experience is only the second or third priority, if at all, so let's say you're building a global financial system, it's much more important that there's never a mistake happening where someone losing a million dollars because of some race condition. And it's much more acceptable that when you wire your money, that you see a loading spinner for one or two seconds. Much more acceptable. But if you write your local notes that only meant for you, seeing a two-second loading spinner while you want to create the note in the first place, that's much less acceptable here. That sort of local versus global aspect, I think, is a good indicator. If you want something like a global social network, et cetera, this is where it gets just so much harder that it is no longer worth it to try to shoehorn it into a Local-First architecture. Whereas if the application use case is fundamentally all about the user, I think it's a great starting point.

Data Ownership and Interoperability [36:59]

Olimpiu Pop: Thank you.

Johannes Schickling: I mean, if those ideas or this general topic speaks to you, I'm not sure when this episode is coming out, but we're also hosting the Local-First conference where Martin will be speaking, and Adam will be speaking. I'm one of the co-organizers. It's going to happen in the mid of July, and it's happening in Berlin. So, if someone is very curious, I'm not sure whether we still have tickets left, but at the very least, worthwhile putting on your radar for hopefully next year. Local-First is just so delightful and refreshing to... Whenever there's a new big technology that is a paradigm shift, I always get very curious and excited to give it a try. And Local-First certainly has those aspects where you just get some magical experiences where you try the first time, where users try it the first time. So, I think that alone is worthwhile to just take a rainy day and play around with it, and have some fun, and expand your horizon.

Olimpiu Pop: That sounds fun. And what they remember is that the conference has one track to make sure that everybody gets all the information. And the other thing is that you have two conference days, and one of them is hacking kind of day where you can just try stuff. A lab day, if I remember correctly.

Johannes Schickling: It's two full conference days and a third hacking day. So, it's three days in total. We have a wide range of different topics, some of them more academic, some of them from very much industrial real-world scenarios where Local-First is absolutely mission-critical. There's lots of domains where Local-First is applied where you would not expect.

Olimpiu Pop: Okay. Looking forward to that. How did you manage to put together your founder cap and your user cap in terms of Overtone? Because you mentioned something that is quite important, and that's you own your data. But in many points, I'm just thinking now just to take a basic example, it's how Amazon gives you your data or how Facebook gives you your data. They just threw something at you. They just finished with the legal part, but that wasn't something that was usable pre-LLM part. Now, with LLMs, we have more chances of making some sense of what's there. But you underline that as something very important when thinking about Local-First. How did you manage to put all these things together so that if somebody wants to take the data and walk away from Overtone, they still can use it?

Johannes Schickling: I think there's two parts. One is how do I integrate with all sorts of external data, which can be very messy? And with AI, certainly, it's still a lot of work, but at least now, AIs do that huge amount of work. But my answer is always great schemas. You need to be able to model and anticipate the incoming data in the most faithful way, and then, refine your schema that you have a good structural understanding of it that from that, you can then map it into a format that is more helpful for your application. Hence, also the reason why I've used event sourcing where I can basically use any sort of external incoming data in a much more wild format where Spotify's format looks this way, Amazon Music's format looks this way, et cetera, and I can map it into a common data format. So, schemas, I think, have always been super important and are even more important now for AI as well. That's one aspect.

The other one is how can someone who is using a Local-First app, for example, Overtone, and they choose like, "Hey, this is no longer for me", and they want to use something else, how can they do that? Well, the core of it is that you have all of your library available to you. It's very flexible which sort of data format you can choose for that. Currently, you have access to the source of truth event log, which is just like a long blog of JSON blobs, but you also have a materialized SQLite database. So, you have a lot of data you can work with, and then, there could be an ecosystem of tools that makes it importable to whatever someone builds next year. Or you can give it to Claude and say like, "Hey, please import all of that data back into SoundCloud", or something. But the most important part is that you have access to something that's a credible source of truth to begin with.

To point out one thing, with the advent of GDPR, we actually got a really huge step forward in terms of data interoperability. Not that GDPR mandated one particular standard, but they said like, "Hey, at least give you some export that contains everything". And now, that is a mechanism, how you can get data from pretty much any service, maybe not real time, not in sort of a streamable way, but at least something. And that, you can then also use as a foundation to bootstrap your own Local-First app, for example.

Olimpiu Pop: I think that's the first almost good thing about GDPR that I ever heard ever since it was implemented. So, that's a bit refreshing. And my last question is, given that you're a DJ and it's your passion, any song, any tune that you would like to recommend?

Johannes Schickling: Oh, I'm into a lot of different music. I don't want to pick any favorites here. I'm based in Berlin, listen to a lot of electronic music. So, yes.

Olimpiu Pop: So, we should look at your SoundCloud profile and see what's there?

Johannes Schickling: When you're coming to the Local-First conference, we're already in the right part of town where there are lots of clubs. So, let's just see where we end up.

Olimpiu Pop: Okay. So, Kreuzberg is the place to be during the summer.

Johannes Schickling: That's right.

Olimpiu Pop: Okay. Vielen Dank, Johannes. Have a nice evening.

Johannes Schickling: Yes. Vielen Dank!

Olimpiu Pop: Thank you.

Johannes Schickling: Thank you.

Mentioned:

• Local-First paper
• Prisma
• Overtone
• LiveStore
• Riffle (Research project)
• Adam Wiggins on Local First Philosophy
• Martin Kleppmann - increasing users’ data agency
• SQLite
• Automerge - local first applications framework
• Yjs - JavaScript Collaborative Applications Framework
• Loro (CRDT technologies)
• ElectricSQL
• PowerSync (Sync engine technologies)
• Redux