Incoming forum posts block the crypto executor
Steps to reproduce:
- Create a forum with a large number of posts on device A
- Share the forum with device B
- Accept the invitation on device B
- Write private messages on device B while the forum posts are syncing
Expected result:
- Locally created messages appear in the private conversation immediately
Actual result:
- Locally created messages appear in the private conversation after a long delay
What I believe is happening is that all the CryptoExecutor threads are busy validating forum posts. Creating a private message involves submitting a task to the CryptoExecutor, and those tasks get stuck behind the forum validation tasks.
We could easily fix this for private message by moving creation off the CryptoExecutor, as creating a private message doesn't involve any long-running crypto operations, but the problem would still exist for forum posts, blog posts, etc.
A proper fix would involve limiting the number of messages the validation manager queues for validation. At present there's no limit - all messages that require validation are queued on the CryptoExecutor and then passed to the DbExecutor.
(This issue existed before the ValidationManager refactoring, but I didn't get round to reproducing it until now.)
Related to #511.