Continuing further on the role of store.
The store protocol started very generalized, especially time range queries, that provides many features for the user (requester)
- customizable start and end time for the range of messages to query
- cutomizable pagination limit
- customizable direction (forward or backward)
- pagination cursor
With some limits (now) imposed on server side:
- max message per pages (100, default 20)
- max time range of 24 hours
Generalization is a problem for both performance, bug surface, etc.
I believe that we have the opportunity to restrict store capabilities to make it work for SDS (or SDS-like) application protocol, and not more. Such capability restriction can help with performance and maintainance.
I recently heard an opinion that “code becomes become technical debt as you gather knowledge on the domain, removing technical debt is actually applying the knowledge you acquired by building, and using, the system in the first place”.
In our case, I would propose to:
- remove customizable direction (backward only)
- remove customizable start time: always applied at time of query
Hence, a time range query would either:
no pagination cursor:
- always be backward
- always be from now to some time in the past as specified by user
with pagination cursor:
- always backward
- from cursor to some time in the past as specified by user
In the case of reliable channel, it means paginating until a message with a valid causal history is found, store queries can then be switched to hash queries.
@Ivansete @Zoltan do you think such reduction of store’s functional scope would help in general with maintainability and performance?
@haelius @kaichao @pablo I believe that p2p reliability strategies as defined in specs and implement in status-go would still work, would you agree?