Context
Store performance has a direct impact on user UX (time to load messages, time to get sent confirmation). DB size has an impact on store performance as well as infrastructure costs.
Moreover, reasonable bandwidth usage is seen as a strong requirements from the Status app team. Both historically and in more recent events where users had their data plan drained because relay mode was being enabled on mobile [1].
Hence, the volume of data transferred across Waku for Status chat feature is a key metric to the usability and performance of the Status app.
Which is why as part of the current roadmap, the Waku team committed to review the Status chat protocols [2] under a scalability and UX (bandwidth usage) lens and propose recommendations.
Note that the date has not committed as this is part of “future proofing” milestones yet to be scheduled.
Preliminary findings
While investigating store performance and data usage [3], the following preliminary analysis was issued [4]:
[…] from random sample it appears that ~60% (or more) of DB space is not related to history at all and can probably be sharded to a different DB with a much shorter retention period.
When attempting to map out the type of data for Status chat protocol, the following rough categories were identified:
| Type | Functionality | Examples | Scope | Desired retention |
|---|---|---|---|---|
| Control | Basic functioning of the application | status updates, community invites, contact requests, group chat invitations | Global or community, depending on product functionality | Likely short, few days. |
| Self-addressed | For a single user to exchange information between app installations | settings backup, device pairing | Mostly local | Few hours, or long term but only latest version matters |
| Content | Actual user generated content such as chat messages | Chat messages, images, and other content | Global or community, depending on product functionality | Forever, 30 days on Waku then BitTorrent |
Work is also defined [5] to review the usage of Waku content topic by the chat protocol to reduce resource usage, as well as optimising loading time and UX.
Finally, there may be a product drive to improve the encryption scheme of Communities to better enable inclusion and exclusion (kick ban) of members.
To provide clean protocol layering and APIs (also for reliability) we’ll need to abstract away encryption from Status Chat onto a separate layer on top of Waku too.
This will also lead to significant work. As it depends on research (de-MLS) and product appetite, I would not expect this to happen until end of 2025/early 2026.
Expected work
The chat protocol recommendation [3] is likely to be several fold and address the following problem:
- Move data that does not need to be broadcast off Waku (e.g. self-addressed data)
- Review retention needs for the different type of data to either enable several retention policy on Waku, or moving data off Waku (e.g control data vs content data)
- Looking at privileging local LAN communication over Waku (e.g. device pairing)
- Review data pattern to reduce frequency or size of data being transmitted (e.g control data)
I expect the work to be required rather sooner than later due to the potential UX impact. I currently assume early 2025.
This is likely to lead to:
- Major breaking changes in Status chat protocol. Which will have to be properly managed to match Status’ product expectations of backward compatibility. This can likely be done by deploying new changes in an isolated, experimental mode for dogfooding, then enable new code for new communities, and finally design migration strategies.
- Integration of Codex in Status chat protocol. And potentially other protocols such as BitTorrent. The recommendation [2] will provide more details on the requirements.
Waku self-sustainability
The transfer of large files and data streams over Waku has been discussed with several BD leads. There is a potential demand for solutions that enables the usage of Waku as a signal network, with some data (videos, images), being offloaded to an alternative system.
Projects want a simple technology stack, and have asked for an out-of-the-box solution.
This means there is potential value (in terms of Waku + Codex being selected) to have an SDK ready to use Waku and Codex in tandem. Ideally, such library would be used in Status, so that a single codebase can be developed, maintained, dogfed and hardened.
Proposed organisation
If my understanding of the problem and future Status app requirements is accurate, then integration of Codex in Status application should happen in 2025.
There are several ways to go about it. Considering:
- the need of codex integration for chat protocols
- the market potential for a all-in-one solution
I’d like to suggest for the Waku chat team to own the integration of Codex in Status chat protocol and do it in a manner that can be re-used as part of a Waku/Codex SDK (or similar library).
In terms of technology (Codex client is written in nim), the learnings of nwaku integration in status-go can be applied. With the codex team providing a Golang package based on ffi-bindings of the current Codex client.
This would further strengthen the expertise of the chat team in terms of protocol integration, with experience in integration Logos technology stack protocols (Waku, Codex), in real world application (Status), and able to deliver the API used in status-go to other projects.
This ownership would also assist in terms of potential other protocol integration, avoid potential misalignment in terms of ownership and team boundary.
[1] https://github.com/status-im/status-mobile/pull/21041
[2] https://roadmap.logos.co/waku/2024-milestones#milestone-bandwidth-optimization-and-protocol-review
[3] as part of PostgreSQL maintenance deliverable
[4] Discord message on Waku Discord