Mailbox download worker for our own mailbox
The mailbox client for our own mailbox (#2290 (closed)) has a single download worker that's shared between all contacts assigned to the mailbox for download. The worker is started when the first contact is assigned for download and destroyed when the last contact is deassigned for download or the client is destroyed.
The worker keeps a reference to its current API task, if any, so the task can be cancelled when the worker is destroyed.
The worker's lifecycle is:
- Check connectivity
- Download and delete all files from all contacts' outboxes
- Wait until our hidden service has been reachable for at least the overlap duration
- Download and delete all files from all contacts' outboxes again, in case any arrived during the overlap period
Downloading and deleting files is more complex for this worker than for the download worker for a contact's mailbox (#2292 (closed)), firstly because this worker needs to download files from multiple outboxes, and secondly because it needs to ensure fairness between contacts, so that a malicious contact can't deny service to other contacts by flooding its own outbox with files.
The worker ensures fairness in two ways:
- To ensure that every contact receives service even if some contacts have flooded their own outboxes, the worker lists the outboxes that contain files, lists the files in each of those outboxes, then downloads from the outboxes in round-robin order.
- To ensure that every contact receives service even if they upload files to their outboxes after the initial check, and if some contacts had already flooded their own outboxes at the time of the initial check, the worker restarts the round-robin after downloading a certain number of files: it lists the outboxes that contain files, lists the files in each of those outboxes, then downloads from the outboxes in round-robin order.
The worker can be in the following states:
- Waiting for connectivity check
- First download
- Waiting for hidden service
- Second download
- Destroyed
When the worker is created:
- Set the current state to "waiting for connectivity check"
- Call the client's connectivity check method
When the worker is destroyed:
- Set the current state to "destroyed"
- Cancel the current API task, if any
When a connectivity check succeeds:
- If the current state is "waiting for connectivity check":
- Set the current state to "first download"
- Start a find-outboxes-with-available-files task (#2170 (closed))
When a find-outboxes-with-available-files task succeeds:
- If the current state is "first download" or "second download":
- If there are any outboxes with available files:
- Start a list-outbox task for the first outbox with available files
- Else if the current state is "first download":
- Set the current state to "waiting for hidden service"
- If there are any outboxes with available files:
When a list-outbox tasks succeeds:
- If the current state is "first download" or "second download":
- If there are files in the outbox:
- Sort the files by timestamp
- Add the file list to the round-robin queue
- If there are more outboxes with available files:
- Start a list-outbox task for the next outbox with available files
- Else if there are any files in the round-robin queue:
- Trim the round-robin queue to ensure fairness (see item 2 above)
- Start a download-file task for the first file in the round-robin queue (#2232 (closed))
- Else if the current state is "first download":
- Set the current state to "waiting for hidden service"
- (Note: It's surprising if we reach this branch, and should probably be logged. The find-outboxes-with-available-files task found some outboxes, but after listing those outboxes we didn't find any files)
- If there are files in the outbox:
When a download-file task succeeds:
- Pass the local file's path to the mailbox plugin to get a transport connection reader
- Decorate the transport connection reader to handle disposal
- Pass the decorated transport connection reader to the connection manager
- Data will be read from the local file asynchronously
- If the current state is "first download" or "second download":
- Start a delete-file task for the current file (#2233 (closed))
When the transport connection reader is disposed of:
- Delete the local file
When a delete-file task succeeds or tolerably fails:
- If the current state is "first download" or "second download":
- If there are more files in the round-robin queue:
- Start a download-file task for the next file
- Else:
- Start a find-outboxes-with-available-files task
- If there are more files in the round-robin queue:
When an event indicates that the hidden service has been reachable for at least the overlap duration:
- If the current state is "waiting for hidden service":
- Set the current state to "second download"
- Start a find-outboxes-with-available-files task