Lark is now open source
Today we're releasing Lark as an open-source project under the AGPL v3. The code is up at github.com/lark-sh/lark, and it's the same code that runs Lark Cloud right now. (Note that using Lark to store your application's data doesn't put your application under the AGPL. Your app talks to Lark over the network the same way it would talk to Postgres or Redis. Build whatever you like on top of it.)
I've wanted to write this post for a long time, so I'd like to talk a bit about why we built Lark in the first place, and why it being open source is so important to us.
Who owns the data
I was one of the earliest adopters of Firebase Realtime Database (back then, just called "Firebase").
I've built businesses on it, including one with millions of users, and I want to be clear that the technology itself was genuinely wonderful to work with. The core idea, that you can take a JSON tree and simply sync it between everyone who's looking at it, is one of those ideas that seems obvious in hindsight, and yet nobody had made it feel that effortless before. I've shipped a lot of software on top of that idea and I'd make the same call again.
What always bothered me was everything else.
I couldn't read the server code, run it myself, or fix it. When the roadmap moved in a direction that didn't suit me, that was simply the direction things were going. And the whole time there was an awareness that the most important part of my application, the part that made it all work, belonged to a company that could change its pricing, its priorities, or its mind (and we all know that Google is famous for changing its mind).
That's an uncomfortable place to build from, and it's especially uncomfortable with a database. You can swap out a UI framework if you have to. Your data layer is different, because everything else in your application is shaped around it. Whatever you chose on day one is still making decisions for you years later, which is exactly why it's the piece you least want to be a black box.
Introducing Lark
If you haven't heard of Lark before, I'll take just a quick moment to introduce you to it. Lark is a realtime database that works just like Firebase RTDB: it's designed to sync a JSON tree (a 'database') between hundreds or thousands of concurrent users. And in addition to that being a very neat thing to do, its added bonus is that it knows how to talk to existing Firebase client SDKs, which means you can drop it into an existing project using Firebase RTDB and just change a few config variables and be up and running. It does a lot more than that, like adding support for volatile data and high-frequency updates over WebTransport, but for the purposes of this article, that's the gist. (If you'd like to read more about the details, check out the main site).
How Lark compares to the alternatives
There are some excellent open-source projects in this space, and Supabase is the obvious one to talk about.
I like Supabase quite a lot. If you're starting something new and you want auth, functions, storage and a database that you can host yourself, it's a great ecosystem and it's honestly open source in a way I respect. But it isn't an open-source Firebase RTDB, and I don't think it's really trying to be. It's an open-source alternative to Firebase, and there are many differences which that distinction entails.
Underneath, you're working with Postgres. You're writing queries against rows and columns, thinking about schemas and migrations, and the realtime layer sitting on top doesn't hide any of that from you (nor should it, really). It's a different data model and mental model from the simplicity of a JSON tree.
Which means that for an existing Firebase app, moving is a rewrite. Your data model gets rewritten, your security rules get changed into row-level security, and depending on how you've done things, your auth gets migrated too. For a new project that's a perfectly reasonable trade. For an app you've been running for years, it's a project with a lot of risk and a significant price tag.
Lark takes a different approach. It speaks the Firebase Realtime Database wire protocol, so the Firebase SDK connects to it without knowing anything has changed. You point your databaseURL at Lark, move your data across, and your application code stays as it is. Your existing security rules work, because Lark implements the same rules language. You can keep using Firebase Auth and leave your users exactly where they are, since Lark validates the same tokens. Firebase Hosting, Storage and everything else can stay too, because Lark only replaces the one piece.
As far as I know, that makes Lark the first project where you can drop something open source into an existing Firebase RTDB powered app rather than starting over.
About Lark Cloud
There's also Lark Cloud, which is the hosted version, and it's available to those who want someone else to handle the devops for them. The open source version is the same version that runs in Lark Cloud. The same dashboard that you use on Lark Cloud that lets you live-view and edit your data and manage your project settings is the same dashboard that we include in the open source repo. The Cloud version has some billing infrastructure on top, but we've even included all of the same metrics gathering in the open source version so in theory you could build your own if you wanted.
We want the choice between running your own Lark and using Lark Cloud to boil down simply to the question: who has to answer the phone in the middle of the night if something goes sideways?
That's the part of open sourcing this that I actually care about. Visible source code is nice to have, but being able to walk away is the thing that truly protects you. We think Lark Cloud is a good deal, and we'd love the opportunity to bring you on board, but more than that, we want the opportunity to give you a choice you didn't have before.
Getting started
If you've read this far, the fastest way to see what I'm talking about is to just run it. All you need is Docker and make:
git clone https://github.com/lark-sh/lark.git
cd lark
make up-release
That gives you a Lark instance with the admin dashboard at http://localhost:8080/admin/, and it prints a one-time password on first start.
From there, the examples/ folder has two things worth looking at. Sticky notes is a multiplayer whiteboard with live cursors and draggable notes, built on our own client SDK. The Firebase quickstart is Google's own (very old) quickstart-js social blogging app, running against Lark with a handful of lines changed, which is the one to look at if you're wondering how much work a migration actually is.
Moving forward from here
The start of any open source project is a time of celebration, but also a time of reckoning. Will this find a place in the toolbox of other developers? Will this serve a need that others have for an open source alternative to Firebase? We're excited to release Lark and find out alongside you.
If you're interested in using Lark in your existing project, we'd love to hear about it, whether you're interested in Lark Cloud or planning to spin up your own instance of the open source repo (we've included detailed deployment docs which should help!) And if you're just sitting on the sidelines, waiting to see where this goes, a star on the repo is a nice way of signaling your support.