Waku project and progress

Introduction

Waku is a proposed fork of Whisper. It was created to incrementally improve in areas that Whisper is lacking in, with special attention to resource restricted devices.

It is a collaboration between Vac, the Status app and nim-eth / status/whisper client teams. In this thread we report on progress of this project.

Background reading

Fixing Whisper: https://discuss.status.im/t/fixing-whisper-for-great-profit/1419/17
Waku project proposal: https://notes.status.im/Ek_UZ_7nTn6y7TWU9pGOcg?view#
Waku kickoff notes: https://notes.status.im/dge-rNtCSwyUaUGPeh_bbQ#
Waku project board: https://github.com/orgs/vacp2p/projects/2

Project goals

  1. The primary goal is to fix the immediate bandwidth scalability issues with Whisper. See Fixing Whisper for great profit for background and rationale.

Secondary goals are:

  1. Ensure our already forked version of Whisper is documented enough to allow others to use the protocol, most immediately status-nim. Aside from new waku mode node modifications, this includes existing light node modifications, mailserver protocol, and additional packet codes.

  2. Increased ownership of our own existing protocol stack for Core, with iterative improvements and allowing for protocol upgrades. This includes future iterative improvements such as moving to libp2p, accounting for resources, settlement, alternative spam resistance solutions (zkSNARKs?), alternative routing (pss?), remote log usage, etc. If we do this right, culturally moving from waku/0 to waku/1/2/3 will be more efficient than trying to get whisper/v7 EIP accepted.

Teritiary goals are:

  1. Improve our way of working together and increase accountability, with a more focused group of stakeholders, closer-to-implementation research and research sprints.

Stakeholders

  • Oskar (swarm lead, research and specs)
  • Dean (research and specs)
  • Kim (Nim implementation)
  • Adam (Go implementation)
  • Andrea MP (Everywhere; Core integration)

Proposed milestones/sprints

Very rough. The goal should be to get waku in a state that it is ready for the [Status] app on the order of a few/some weeks (4-6?).

waku/0 - Nov 11 - Nov 25

  • Separate protocol that is backwards compatibility with Whisper/v6 and works through console-client/status-nim-tests
  • Fork of EIP627 and our modifications (packet codes, light client, mailserver) and new waku mode in own spec as draft

waku/1 (+2w) - Nov 25 - Dec 02

  • Specs moved over to Vac and good enough to enable e.g. Nim/Go client interop; rate limiting and resource accounting draft
  • Nim and Go clients working in isolation with waku mode as well as with its own kind (nim whisper6<>nim waku; geth whisper6<>nim waku)
  • Preparation for enabling it in app as an option

waku/1 (+2w) - Dec 02 - 16

  • Cross-client working
  • Specs in Vac in solid state and publicly available/reported on; clear problem isolation on next medium term steps (libp2p, pss, zksnarks, settlement etc)
  • Integration into app
  • Handover to Core; reflection point to see if it makes sense to continue as a separate team and where additional enhancements should lie / fit into other research work

In future posts we’ll post updates as the project progresses.

Nov 11 - Nov 25

First two weeks of the project. Planning, specs, and basic PoC.

1. Planning: Kick-off call and cutting of issues.

image
image
Project board: https://github.com/orgs/vacp2p/projects/2

2. Specs: Waku v0.1 merged

image

See Waku spec. A lot of work behind the scenes to get this done (see issue and PR. Still some work to be done to waku 0 specs, but it’s better than EIP627 and captures things such as light mode and mailserver, both of which had differences between Go/Nim client and EIP.

3. Client: New subprotocol waku/0 and bridging PoC through nim-eth

See https://github.com/status-im/nim-eth/issues/125 and https://github.com/status-im/nim-eth/pull/120

Nov 25 - Dec 9

1. Waku spec 0.2 release:

  • General style improvements.
  • Fix ABNF grammar.
  • Mailserver requesting/receiving.
  • New packet codes: topic-interest (experimental), rate limits (experimental).
  • More details on handshake modifications.
  • Accounting for resources mode (experimental)
  • Appendix with security considerations: scalablity and UX, privacy, and spam resistance.
  • Appendix with implementation notes and implementation matrix across various clients with breakdown per capability.
  • More details on handshake and parameters.
  • Describe rate limits in more detail.
  • More details on mailserver and mail client API.
  • Accounting for resources mode (very experimental).
  • Clarify differences with Whisper.

2. Client updates:

3. Write-ups and general discussion:

See Fixing Whisper with Waku for a more detailed write-up with rationale and a theoretical scaling model for bandwidth usage.

Discussions: Transport-agnostic Waku, Forward compatiblity, and Stake Priority Based Queuing.


Bonus: Panini - live ABNF editor as a side project to validate ABNF grammars.

See Waku - recap, evaluation and derisking for latest. Less backwards-facing and more focused on cutting scope and identifying/mitigating risks.