Problem statement
The problem statement mainly specify the issue of too many content topics and impact on Waku client performance.
Another aspect is the fact that for highly permissioned Communities, the user may download messages they do not need to read (addressed in the proposal).
Also, we should consider prioritisation of retrieval/subscription.
Looking at the types of messages a user downloads, can we apply some heuristics to limit the data a user accesses, for example:
- Prioritize the channels where user is mentioned
- deprioritize channels that are muted
- deprioritize channels user cannot read (already discussed)
Finally, we should mention the e2e reliability protocol implementation to ensure we are creating incompatibilities.
Migration
The first question is to understand what is Status app policy in terms of backward compatibility. How many past versions are considered “supported”.
Then, from this we can draft an exact plan.
In general the strategy would be
| Step | App Version | Read | Write |
|---|---|---|---|
| Initial | N |
Old format | Old format |
| Preparation | N+1 |
Old and new format | Old format |
| Switch | N+2 |
Old and new format | New format |
| Clean-up | N+3 |
New format | New format |
@iurimatias can please clarify the deprecation policy for Status apps? If not existing, can this please defined?
Users probably don’t want to route the messages that they are not be able to encrypt/decrypt though.
Routing should be kept out of the scope of this discussion.
I agree with the initial idea of trying to approach the problem from a permission angle, and attempting to spread the channels on content topics based on the users’ permissions. However, from the discussion above this seems to not be a viable option because:
- permissions can change, and dynamically changing content topic for a channel is a complex problem. A public channel may become a private channel.
- permissions can be very flexible, it not a user per channel setup, but one or several token per channel, and one or several token per user
Proposal: bucketed content topic per channel
Perhaps if we want such feature in the future, a possible approach could be to partition the content topics for a community, maybe with a smaller set of 10 content topics or so, and you would only subscribe to the content topics partitions that match the channels you are interested in receiving messages from.
Indeed, it seems that a safer approach would be to:
- Define a reasonable/max number of content topics per user (Waku)
- Define reasonable/max number of communities per user (Status)
- From (1) and (2), define a content topic bucket size per community (Waku)
- Generate content topics per channel, but in the bucket. Meaning that number of content topic a community use is capped.
capped number of content topics- no impact of permissions on content topic assignment. Only impacts the user: if they get permission to read a new channel, they may subscribe to a new content topic.
a user that does not have permissions to read in all channels, may not download all messages they cannot read, but may still download some messages they cannot read
code complexity remains as it is, maybe higher