TaskQueue processing refactor
I have made changes to the way TaskQueue works:
-
m_taskPushed
member removed, originally used to indicate to TaskQueue worker that there are tasks pending -
m_ready
member added to allows us to pause queue tasks processing in the event that C is not available without losing all tasks, having to restart daemon, and to prevent transactions getting stuck - worker no longer waits for a task pushed, instead, waits on condition variable and resumes execution if queue is not empty and is ready to process
- worker now only has a single loop and wait is called for every task, instead of bulk processing tasks
-
pushToQueue
method only inserts a task and notifies worker, nom_taskPushed
- added method
updateReadyStatus
to allow us to pause queue execution, this method also notifies worker waiting on condition variable - added method
getReadyStatus
to allow for queue ready status monitoring, this can be combined with thesize
method to return both queue size and queue ready status, returned as a structure or tuple - method
stopQueue
reworked to pop everything from queue and then notify worker, this method is also called in TaskQueue destructor before joining thread
The goals of these changes are:
- allow us to pause and resume queue
- if C becomes unavailable, queue is paused, tasks are still added to the queue, but not be executed and thus not blocking communication with the network
- if C becomes available again and the queue is not full yet, queue is resumed and tasks are processed
- if C becomes available and the queue is full, queue is resumed and tasks are processed, tasks over queue capacity are lost
I have done some testing on my own, but it should be stress tested properly, I have attempted AutoNetwork a few times, and later on I also added a periodic raw scheduler task. AutoNetwork was fully executed and then all of the stacked scheduler tasks were executed back to back, in this sense, the behavior hasn't changed.
The other thing to consider is the web interface:
- what should the web interface do if a request without timeout is executed from the frontend (such as AutoNetwork) while queue is not ready
- what should the web interface do if a request without timeout and comprised of multiple DPA requests has been executed and C became unavailable in the process
- the situations above have to do with the way we currently use spinners, these situations effectively render the web interface unusable without refreshing the page, it only applies to these requests, because other requests have a timeout, so the web interface will be available, albeit not fully usable at worst within 2,5 minutes
Edited by Karel Hanák