Core API for observing a message graph
The core API exposes methods for getting a snapshot of the message graph in a group, such as
DatabaseComponent#getDependencies(), and events for learning about changes to the message graph, such as
MessageStateChangedEvent at the sync layer, or
PrivateMessageReceivedEvent at the client layer.
The UI uses a combination of asynchronous DB queries and events to maintain a view of the message graph. Using the data attached to events to update the UI is more efficient than triggering a DB query for every event, but it causes a lot of complexity in the UI code because updates may happen concurrently with asynchronous loads (see #705 (closed)).
All interactions between the UI and the DB are funnelled through the single-threaded DatabaseExecutor, mainly to ensure that queries return their results to the UI in the same order as the queries were started. The database allows read-only transactions to run concurrently, but queries running on the DatabaseExecutor can't take advantage of this because the executor is single-threaded.
If we could provide a better API for observing a message graph then we might be able to simplify the UI code and perhaps even improve DB performance by allowing more queries to run concurrently.
The new API should allow an observer to register an interest in a group's message graph and receive an initial snapshot of the graph, followed by an ordered series of updates when the graph changes. An observer registered from the UI will need to process the observations on the UI thread. As long as the core delivers the observations in order it should be easy to move them onto the UI thread in the same order.
We also need a way to register multiple observers that are ordered with respect to each other, so that we can create UI components that represent multiple groups, such as a conversation containing private messages, invitations, etc.
To ensure ordering, the DB will need to register any changes made by a transaction before releasing the DB lock. Events attached to transactions aren't currently suitable for communicating ordered changes because they're broadcast after releasing the DB lock, so two transactions that commit their changes in the order A, B may broadcast their events in the order B, A. But communicating changes to other components while holding the DB lock would have the potential to cause deadlock.
It might be possible to modify the event broadcasting logic so that events are registered before releasing the lock and broadcast afterwards, ensuring the broadcast order is consistent with the commit order. Alternatively we could create a separate mechanism for registering and communicating ordered changes.