Proposal: Status X - a minimal chat app on ChatSDK

TL;DR

  • Proposing Status X, a slimmed down Status app tracking (only) ChatSDK features as they’re developed
  • Status X de-risks chat protocol migration
  • Status X harden latest Waku research without impacting revenue
  • Status X enables definition and specification of what the Status app is

Background

Over the past few years, Waku has been taking on more and more of the Status messaging protocol stack. This has led to the status quo where Waku is developing the “ChatSDK” as a general, app-layer chat stack on top of Waku routing. Within this effort, we’re aiming to systematically solve the entire vertical of reliability, encryption, segmentation, DoS protection and routing that’s required by a decentralised messaging app. In parallel, Status developers are refactoring status-go to better isolate those same functional areas.

ChatSDK is meant to be a generally useful app-layer chat stack over Waku. However, our first client is Status. Although the new alignment is promising, we’ve seen in the past that without coordination (and a clearly defined, shared goal) there’s a risk that either team may drift from the expectations of the other and into scope creep. Furthermore, ChatSDK can benefit from fast feedback and continuous integration within an app with real world users.

Why not just integrate into Status as is?

ChatSDK is focusing on a narrow scope for now: systematic development of a chat protocol stack that works for “private conversation” use cases. That is, it has as priority developing a chat stack that is roughly compatible (only) with the existing Status 1:1 and private group chat features.

Status app currently consists of many more features, each with its own developmental state and roadmap. This includes the wallet, user features like backups, sync, push notifications, etc. In particular, there are more product decisions to be made on Status Communities, that will follow user validation of the current implementation. This variability of the product roadmap cascades down to technical requirements, which we currently accept as unspecified. This does not block ChatSDK development, because, as previously stated, our first focus is on private chats.

All of these features in the app affect each other, even as we progress with modularisation. Specifically, it makes it hard for ChatSDK to integrate and test, without having designed a solution for all those unrelated features as well.

One can expect that “integrating Chat SDK in Status” involves maintaining a status-go branch that uses the latest Chat SDK release, and uses it within status-go thanks to the refactoring. We also expect that such a Frankenstein would need to be maintained until feature parity is reached with the current Status app, as well as maturity and hardening to tackle the risky chat protocol migration.

Enter Status X

We suggest to properly release the integration of the Chat SDK in Status as a dedicated app, so that user validation and technology dogfooding can be done without impacting the product goals of Status.

Status X can be seen as a slimmed down version of Status, that focuses on the features provided by the Chat SDK so far, and put then in the hand of users.

A first phase of Status X will be a functionally complete (set of) specifications for a private chat app over Waku. Secondly it would be a working implementation of that spec stack. The expression as a set of specifications is important as it will allow us to clearly define the API and interfaces between each layer, most importantly the dividing line between Status as an app and the ChatSDK.

Moreover, this will enable de-risking the migration from status chat protocols to the chat SDK, by first proving the chat SDK robustness and reliability. Allowing the migration to be fully focused on that: migration, and not adding risk of rolling out the new chat sdk at the same time.

Finally, latest Waku development are expected to create user friction, such as the usage of ERC-20 deposits to acquire RLN memberships. Having a dedicated app to test such UX and iterate on it, dogfooding it, and evolving it, will enable enhancement without impacting Status’s user acquisition and potential revenue streams.

We will leave the technical choices on how to extract such an app from the current codebase to the Status team, but our expectation is to re-use as much code as possible, and use sensible strategy to cut out specific features that we know do not work with the chat sdk (yet).

This will also enable a focus on the Status side in terms of what composes the Status app and protocol specification.

What we do afterwards

Of course, this just describes an initial vision/first phase for Status X. As priorities get added to the ChatSDK roadmap, Status X will grow with that feature set.

Once the Status App is migrated to using the Chat SDK, we can do a review of the Status X application, that can depend on technical needs, actual overhead, state of Status specifications, and community reception.

Two possible paths forward:

  • Continue using the Status X app for experimentation such as de-MLS, RLN entry points, Codex integration, etc.
  • OR, agree on experimentation strategies in the Status app and decommission Status X

Feedback welcome!

4 Likes

I’ve been toying with a similar idea for other “experimental” changes to bring to the status desktop stack, including seaqt and an updated nimble-based build system as seen in nora which also demos desktop+mobile builds from a single codebase (thanks Alex!) - these little miniapps are indeed quite useful to play around with practical applications of a library and stress-test architectural ideas before applying them to a “live” use case.

One option for “status x” here is to simply take the original nim-stratus proof-of-concept and update it to modern standards, replacing nimqml with seaqt and expanding from there - this app already comes with a minimal gui which is easy to work with, so unless “status X” is intended to be text-based, it’s probably not a bad place to start.

Nora is also being used for prototype plugin support (thanks Michal!), which down the line will serve as a model for extensibilty in logos core, so putting it all together in a credible chat-sdk-based UI would make it “serious enough” to act as a reasonable predictor of future technical difficulties.

The important thing to remember in this case is that they are indeed prototypes, made to be discarded at any moment - the hardest part of working with a prototype is letting go when it has reached 80/20 - but also to adjust development strategies accordingly (ie no “2 reviewers, a pointy-haired boss and a design team to approve each change” but rather focus on rapid development).

In particular, I don’t think there’s appetite or bandwidth for maintaining two full-blown supported chat applications at this point, so while I appreciate the idea of creating a “status X”, the decision of whether to do it or not should also be approached with the same mentality: “just do it” and the moment the utility is gone, “just stop it”.

2 Likes

We will leave the technical choices on how to extract such an app from the current codebase

One thing of note is that it’s probably not a good idea to start from the current status-desktop codebase at all, and likewise it would be a mistake to work from status-go - an endeavor like this would largely be wasted if it were to work from the current state of affairs.

A key objective to fulfill is that “anyone” should be able to write a “status chat application” on their own starting from the specs - developing a prototype like this is a good opportunity to stress test that point by doing just that without relying on the existing monolith.

If there’s anything to be extracted from status-go/status-desktop it would be, in priority order:

  • specs
  • entirely stand-alone components (ie “double ratchet” or “key derivation scheme library”) that can be expressed as a simple C api

May a 1000 clients bloom!

In terms of the experimentation, Nora, seaqt, etc. We agree and intend to make chat SDK available in Logos Core and upgrade the chat2 prototype already done by @iurimatias to use Chat SDK, and experiment in a text based app.

This is not what the Status X proposal is about.

We are not advocating to build a new full-fledge Status application from scratch either.

As I am not sure which part is not correctly conveyed in @haelius 's post, I’ll revert the question: What is the expectation in terms of integrating the Chat SDK in the Status application? Knowing that will be juggling with:

  1. a large functionality scope (private chats + communities)
  2. a complex migration (different encryption stack, RLN protected network)
  3. new protocol and software

a. It seems unreasonable to wait until the new chat SDK is feature parity complete with the current Status application to finally deliver to user.
b. It was already stated it is not on the table to reduce the feature scope (eg communities)
c. Your response says we should not experiment within the status-go codebase to have a feature reduced app in the hand of avid users (reject Status X)

Or, am I understanding correctly that the expectation is to build the Chat SDK, experiment as much as possible in the context of Logos Core, seaqt, etc. Then proceed with big bang integration and migration once parity feature is reached for Status?

Your response says we should not experiment within the status-go codebase to have a feature reduced app in the hand of avid users

My response in particular says that an effort like this could be used to drive the spec writing and modularisation of the existing status go codebase by already extracting from it the parts that make sense as stand-alone independent parts and not wait for anything else to happen, so that writing a basic chat client becomes a low-effort endeavor for anyone that would want to attempt it, and to break the cycle of fear, uncertainty and doubt around how hard it would be to move the needle on modularising status-go.

It also says that taking the current application and trying to turn it into a feature-reduced version is unlikely to yield positive results - the ingrained complexity of having been adding features and bugs for many years and supporting two wildly different codebases with the tug-of-war between mobile and desktop has not resulted in a codebase that is amenable to trivial “removal” of things without making this the main effort - and then the proposal is to add yet another direction to it, in the form of sdk!

I think for an endeavor like this to be successful it’s important to focus on a smaller set of priorities than the initial post outlines but give these fewer priorities more attention individually. In particular, developing chat sdk itself is also just a small subset of what it would take to integrate chat sdk into desktop.

Of more interest is a clear understanding of how to keep the chat experience intact for existing and new users alike - this (likely) means running both the old and the new protocols side-by-side for an extended period, as well as having a migration path between the two available, be that a bridge, an in-protocol compatibility layer, a UI shim or whatever else - no matter what, it’s important that users are not left behind in this migration because ultimately, they couldn’t care less about whether it’s chat sdk being used or not.

Something else to keep in mind is that while the current chat protocol may seem like an impossible thing to understand and extract, I believe that at the end of the day it is actually quite small - both frankenwaku and stratus were weekend experiments that very quickly achieved compatibility, and that a clean-room implementation of it would take far less than one would think provided that as part of that effort, you extract the relevant bits of into well-documented, stand-alone specs and components. The key is to keep the effort tight and focused and not allow it to venture outside of its intended territory.

Why? Because status desktop indeed has a lot of features, bells and whistles that are not relevant for a pure chat application - in fact, the “other stuff” dominates entirely. If you were to start from “just” the chat protocol and a simple irc-like feature set with a user list, you’d end up with an application not much larger than stratus, but now with a clean understanding of the existing chat protocol which would allow you to work on the chat sdk introduction and migration path in isolation which in turn would inform how to do it in desktop when the time comes.

If you provide this spec and clean modules for the existing and new chats, the existing status desktop will have something to migrate to, and that’s a head start already.

Trying to strip down the existing application is like trying to keep the cookie while eating it - you will spend more time and resources on debating which parts to remove and which to keep, and then debugging those (“should we keep the way the current user profile picture is transmitted, in the middle of the database update to send a message? Let’s have a planning meeting next month about this feature so that we can design an good UX flow for it” and “oh, what should we do about the markdown code block support? it looks terrible, let’s spend an engineer-month on making sure code blocks have color and are properly formatted”), instead of making progress on the core experience of being a user that was chatting with their buddy and suddenly that buddy is gone because the developers were eager to push out their shiny new toy.

As I am not sure which part is not correctly conveyed

What is not conveyed are clear boundaries on what features and parts of the effort you would focus on and which you would say no to so as to understand where the effort ends - ie a picture and understanding of what “I’m done now, let’s move on” looks like - the latter being almost more important than the former - without this narrow scope and a plan for a beginning, middle and most importantly an end, the proposal effectively boils down to “let’s build two production-grade end-user applications and maintain them forever” . When would it be “ready”? “Status X will grow with that feature set.” does not indicate clear answers to this question.