workqueue: fix race condition in flush_workqueue()

When one flusher is cascading to the next flusher, it first sets
wq->first_flusher to the next one and sets up the next flush cycle.
If there's nothing to do for the next cycle, it clears
wq->flush_flusher and proceeds to the one after that.

If the woken up flusher checks wq->first_flusher before it gets
cleared, it will incorrectly assume the role of the first flusher,
which triggers BUG_ON() sanity check.

Fix it by checking wq->first_flusher again after grabbing the mutex.

Signed-off-by: Tejun Heo <tj@kernel.org>

diff --git a/kernel/workqueue.c b/kernel/workqueue.c
index 5587338..b59c946 100644 (file)
--- a/kernel/workqueue.c
+++ b/kernel/workqueue.c
@@ -2138,6 +2138,10 @@ void flush_workqueue(struct workqueue_struct *wq)
 
        mutex_lock(&wq->flush_mutex);
 
+       /* we might have raced, check again with mutex held */
+       if (wq->first_flusher != &this_flusher)
+               goto out_unlock;
+
        wq->first_flusher = NULL;
 
        BUG_ON(!list_empty(&this_flusher.list));