THE REGEX KING
jeff sisson's blog (email me)
Origins of big boy chat
After leaving a job where it felt like I’d orphaned a rich history of chats with friends that were completely unrelated to that job, marooned somewhere in Slack’s data centers, I wanted to recalibrate how & where I spent time chatting on the internet. My primary goal was to continue chatting with my friends Tom and Thomas, coworkers at this old job, but my loftier goal was to find or make a chat app that satisfied my increasingly obscure list of user demands. This is the origin story of big boy chat.
self-hosted: Let’s Chat
There are lots of lists on the internet of “self-hosted” equivalents of various popular cloud-based apps (like Slack). The principle of these lists, which seems admirable, is that by running your own version of a networked app like Slack or Facebook or whatever, you exert greater control on how these apps work. In practice, though, many of these self-hosted equivalents feel like uninspired or lesser versions of the thing they seek to replace.
The self-hosted chat app I initially landed on is called Let’s Chat. It satisfied some of my requirements (drag-and-drop image upload, persistence), and was amenable to modifications in order to support some things that weren’t available out of the box (private by default, some basic tweaks to the UI). We used this to chat for quite some time, and it was a nice “third space” that wasn’t Twitter, email, or group messages.
Over a longer period, there were a few aspects of running this chat app began to grind on me. It required a MongoDB database, which has non-trivial space and CPU requirements. But more importantly, running a database meant that I was on the hook for things like security AKA ensuring that our throwaway chat messages weren’t going to be hacked into the open.
While running a database and server could be justified as a necessary chore, I think it points to a weirder flaw of all self-hosted chat apps: it feels wrong to need to “self-host” a chat app in the first place! I’m happy to maintain a space like this for friends, but I had an abstract notion that something like chat between two or more people shouldn’t require a dedicated server and complex infrastructure. It should be as simple as software running on different peoples' computers talking to each other directly, without any intermediaries.
peer-to-peer: hyperlog, hypercore and cabal
I started looking into peer-to-peer chat apps and protocols. I found substack’s chatwizard, which stored its data on hyperlog, a peer-to-peer friendly database. At a high level, writing a chat room app with hyperlog as its database would allow different chatters to create their own chat log, reference any previously “seen” chat messages when new chat messages are added to the log (to establish an ordering of the chat messages), and is able to live sync a chat history between two or more peers. The only thing it’s missing was a built-in way to verify that chat messages came from their sender and not some imposter (crypto!); while there are API hooks for doing this in hyperlog, the actual implementation is left to the user.
Around the time I started tinkering with hyperlog, the community of people building and using the peer-to-peer Dat protocol and Beaker Browser were gaining momentum. Dat is built on top of hyperdrive, which in turn is built on top of hypercore, which is a lot like the hyperlog protocol I’d been looking into. Lots of hyper! Hypercore automatically signs and verifies individiual messages appended to its log (crypto!), but doesn’t yet allow multiple people to write to a single log, which makes it harder to use for something like chat.
Enter cabal: some enterprising people in the Dat community decided to create a chat protocol called Cabal, which uses multifeed under the hood to allow chatting peers to copy any hypercore databases that a given peer has seen into their own hypercore, which is writeable. The distinction is subtle, but multifeed is what makes using hypercore as its peer-to-peer database possible for a chat use-case.
With cabal/multifeed/hypercore as a foundation, I was able to get a version of an app I started calling “p2p-party-line” working. This chat app uses cabal to create a single chat room, where people can choose handles and add some light markdown and HTML to chat messages. The chat log itself is synced using the WebRTC protocol, which requires a signalling server to introduce chatting peers to each other. While this breaks the purity of this being an exclusively peer-to-peer app, the server is only used at the beginining to find out who else is hanging out in the chat; once your computer finds those other chatters, all subsequent chat room syncing happens directly between your computers and no chat messages are stored or read or intercepted by a server at all. In fact, because of the way that chat room identifiers are hashed (this is the long string of characters in the URL), the server doesn’t even know about individual chats, making them effectively private until or if someone in the chat shares the URL publicly.
last one out please turn off the lights
One unintentional consequence of this chat room architecture is something I’ve been calling “last one out please turn off the lights” mode. p2p-party-line is a browser app, and while it could save any data for any chat you’ve participated in for offline or later use, it doesn’t. But as long as someone is still in the chat room, the history is preserved, even if you leave and come back to the chat room. This creates an interesting dynamic: as long as people are still chatting, it operates like a regular chat room. But when everyone’s ready to start over or call it quits, none of the chat history sticks around or is saved somewhere. The network acts as the chat room’s hard drive: when no one’s online, the chat room disappears.
This distributes maintanenance of the chat room to everyone participating, and away from any single participant. You could/can write a bot which “seeds” the chat history permanently, and I’ve done this on occasion to support situations where I want to chat with people who aren’t consistently available around a similar time frame. But the ephemeral aspect of the default mode encourages using big boy chat as something like a burner chat room, used for specific times and places but mauybe not intended for permanent or perpetual use. The “grain” of p2p-party-line encourages you to create a new chat as needed (they’re cheap/free!) rather than maintaining a single, monolithic chat room that lives forever. Vive big boy chat!
Please post ideas and/or bugs you encounter here!