The current state of Waku v2 peer discovery

The current state of Waku v2 peer discovery

2021-10-15

NOTE: I’m not yet able to post a discussion with more than two links. Hyperlinked version of this document here. That’s one allowed link down, one to go…

The question of peer discovery for Waku v2 has been a focus area since the protocol was first conceptualized. Since then several different approaches to discovery had been proposed, investigated, implemented or placed on development roadmaps.

This post aims to:

  • give an accurate picture of the current state of research and implementation of peer discovery mechanisms for Waku v2,
  • collate the diverse requirements Waku v2 applications may have for peer discovery, and
  • revisit the benefits and tradeoffs of different discovery schemes.

It is meant to be an interactive discussion, with feedback from as many Waku v2 stakeholders as possible, to get a clear picture of where we are, what the specific peer discovery requirements are for different use cases, and how to proceed from here.

Implementation status

method nim-waku (Nim) go-waku (Go) js-waku
DNS-based discovery :heavy_check_mark: :heavy_check_mark: :construction:
GossipSub peer exchange :heavy_check_mark: ?
Node Discovery v5 :construction:
Rendezvous :heavy_check_mark:

Background

Peer discovery is the method by which nodes become aware of each other’s existence. This definition should be general enough to cover the use case of discovering peers with selected capabilities, although “peer ID discovery” and “capability discovery” could be treated as separate problem domains in implementation.

Waku v2 design goals

Since the problem of discovery is (mostly) orthogonal to the messenger-oriented problems addressed by Waku v2 protocols, Waku v2 can in theory be used in conjunction with any discovery mechanism. We are, however, researching more integrated approaches that are specifically geared towards Waku v2 design goals.

Specifically, we want to maximize the following properties in selecting discovery schemes:

  • scalability
  • resource-efficiency
  • decentralization

In practice every discovery mechanism implies a tradeoff between these three properties.

Capability discovery

Waku v2 nodes may also be interested in peers with specific capabilities, for example:

  1. peers within a specific pubsub topic mesh,
  2. peers with store capability,
  3. store peers with x days of history for a specific content topic, etc.

There may be ways for some of these capabilities to be advertised and discoverable within any of the mechanisms described below, although this could increase the complexity of finding a node with specific capabilities within the larger network (a needle in a haystack).

Peer discovery candidates

1. DNS-based discovery

Definition

Using this method a new node can retrieve an authenticated, updateable list of peers via DNS to bootstrap connection to a Waku v2 network. Our implementation is based on EIP-1459.

Benefits

  • highly scalable
  • resource-efficient
  • existing implementations in go, nim and js

Tradeoffs

  • centralized, and therefore vulnerable to censorship/attacks
  • requires multi-step, centralized updating to add new nodes to the discoverable list (i.e. retrieving the current tree, updating and signing the tree, and redeploying to a DNS provider)
  • although useful for bootstrapping connection to a Waku v2 network, it must be used in conjunction with other ambient peer discovery methods for a peer to become aware of more nodes in the network.

Implementation notes

Based on EIP-1459 across all three Waku v2 implementations. Since EIP-1459 operates on EIP-778 Ethereum Node Records there is an ongoing conversation about how to encode multiaddr information for which there exists no key definition in EIP-778. These include websocket ports, DNS addresses, etc. which are required for running Waku from a browser.

IMPORTANT NOTE: RFC 25/LIBP2P-DNS-DISCOVERY proposes an alternative approach to DNS discovery not based on EIP-778 ENR. This specification is not currently implemented or on a roadmap to be implemented.

go-waku

EIP-1459 implementation done and integrated in go-waku

nim-waku

EIP-1459 implementation done and integrated in nim-waku

js-waku

WIP. Awaiting consensus on multiaddr encoding in EIP-778 ENR.

2. GossipSub peer exchange

Definition

GossipSub Peer Exchange (PX) is a GossipSub v1.1 mechanism whereby a pruning peer may provide a pruned peer with a set of alternative peers where it can connect to reform its mesh. This is a very suitable mechanism to gradually discover more peers from an initial connection to a small set of bootstrap peers.

Benefits

  • scales with the network
  • fairly resource efficient, as it piggybacks on existing control signalling
  • specified by libp2p and already implemented in many languages

Tradeoffs

  • not useful as a general out-of-band discovery mechanism, as it piggybacks on in-band control signalling
  • requires trusting the information provided by pruning nodes
  • since this is a libp2p mechanism, not Waku v2, it relies on the respective libp2p libraries for an implementation
  • requires a bootstrapping mechanism to initiate connection to the network

Implementation notes

go-waku

Implemented in go-libp2p. Enabled and integrated in go-waku.

nim-waku

Not yet implemented in nim-libp2p. Implementation is likely to form part of the medium term nim-libp2p goal of peer and connection management.

A shorter term minimal implementation is possible by adding support in nim-libp2p for:

  • signed peer records
  • a callback function whenever a PRUNE is received and contains peer exchange information. nim-waku could then implement this callback function, handling and connecting to the discovered peers.
js-waku

???

3. Node Discovery v5

Definition

Node Discovery Protocol v5 is a DHT-based discovery mechanism adapted to store and relay node records.

Benefits

  • decentralized
  • existing implementations in nim, go, etc.

Tradeoffs

  • high resource-usage, specifically i.t.o. CPU and memory usage.
  • “needle in a haystack”: finding a peer with specific capabilities may not be feasible within the resource-restricted world of mobile clients and mobile data.
  • requires a bootstrapping mechanism to initiate connection to the network

Implementation notes

go-waku

Not yet implemented.

nim-waku

POC implementation in progress. The POC proposes that discv5 can be started and stopped via API calls to limit resource usage.

js-waku

Not yet implemented.

4. Rendezvous

Definition

Rendezvous protocol is a discovery method whereby peers register and discover other registered peers at rendezvous points. Status rendezvous introduces some differences with the original specification, including using ENR for peer information.

Benefits

  • suitable for resource-restricted devices
  • less centralized than DNS discovery

Tradeoffs

  • susceptible to adversary attacks
  • requires a bootstrapping mechanism to configure/discover rendezvous points
  • no existing implementation in nim. Unknown how useful the js implementation will be for js-waku, due to the differences with Status rendezvous.

Implementation notes

go-waku

Implemented and integrated in go-waku.

nim-waku

Not yet implemented.

js-waku

Not yet implemented.

2 Likes

go-waku also has a different implementation from both the original specification and the one used by status:

It is similar to status-im/rendezvous in using a smaller liveness TTL for records (20s), and not using an Unregister request, due to assuming that the TTL is very low (thus making it incompatible with libp2p original rendezvous spec).

But it’s also different compared to status-im/rendezvous in the usage of routing records and signed envelopes vs using ENR, thus making it closer to libp2p spec in this aspect.

1 Like

Great post! Thanks for compiling the current state of things.

There’s also this RFC: Capabilities advertising · Issue #429 · vacp2p/rfc · GitHub, which was alluded to in post but not explicitly mentioned I believe, which is slightly different from peer discovery. It’d be about finding nodes that provide a specific capability, such as storing message for some pubsub topic, for some set of time, etc.

It’d be useful to understand more precise requirements from e.g. Desktop to prioritize work here. Specifically: in order to get Waku v2 integrated (including nim-waku as a service node), what is the most pressing need here? Ditto re DappConnect side of things.

1 Like

I created this post to discuss Desktop requirements for WakuV2

1 Like

Regarding peer-exchange in js-waku. Tracking with Support peer exchange · Issue #309 · status-im/js-waku · GitHub
I haven’t checked if it’s just support via js-libp2p.

Best to wait for nim-waku/nim-libp2p implementation to be done first before doing it in js-waku.