I think this API is a middle ground between what was done for MVDS and original Waku core protocols (very leaky abstractions that expose too much of the underlying protocol to the apps) and what we are aiming to do with the Messaging API
Placing this work in bigger context, the steps are:
(1) Initial integration
We integrate SDS with the API as above in status-go. Meaning that as specified, status-go code has to handle some retries, interfacing with local db and awareness of “message not acknowledged” and deciding what to do with that.
Nwaku integration is orthogonal to this integration.
(2) Nwaku integration
We replace go-waku core protocols with nwaku Golang bindings. This means no more usage of go-lipbp2.
go-waku still lives because tunable p2p reliabilty is implemented in go-waku’s API and we want to keep it for now.
(3) Messaging API
We define a Message API across all Waku SDKs (nwaku bindings, js-waku, Rust and Golang SDKs). This API removes all leaky abstractions, and includes the tunable p2p reliability (RIP go-waku).
All this works above is planned to start in 2025H1 and likely to finish within the year.
(4) Reliability API
The next step, not planned and to be done after Messaging API or once it is well matured. It is to provide an all-integrated SDS API.
The Reliability API would replace (in most scenarios) a couple of the functions of the Messaging API. Such as send message and subscribe, by introducing the concept of channel id.
The aim would be to remove the wrap/unwrap and provide an API that automatically handle retries, syncing, etc.
Those could be parameterized by the application, but the Application would not need to “resend using Waku”.
The API should handle the various retries, and in case of unrecoverable failure, give to the applicaiton to tell the user the message sent to fail.
At this point in time, a resend would be brand new message from a SDS PoV
Lots of unknown of course, but I encouraged @arseniy to provide such an API in JS, so that when integrating in Qaku, we can already learn and have a feel for what a good API would look like.
Once such an API is defined, then we will want to make it available in Golang, and Rust etc. The way I see it right now, is that the “C SDK” would provide this new API and would use libwaku and nim-sds together to deliver it.
Let me know if I need to expand more on that.
With this end goal in mind, I now wonder if the current proposed API for SDS in status-go makes sense. Or if it is too low level, and will leak too much in status-go.
Which may be fine. The team had great success in implementing p2p reliability directly in status-go first, and has the code was proven to work, it was slowly migration to the go-waku repo.