[PATCH] mm: dirty_exceeded speedup

Ravikiran reports that this variable is bouncing all around nodes on NUMA
machines, causing measurable performance problems.  Fix that up by only
writing to it when it actually changed.

And put it in a new cacheline to prevent it sharing with other things (this
happened).

Signed-off-by: Ravikiran Thirumalai <kiran@scalex86.org>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>

diff --git a/mm/page-writeback.c b/mm/page-writeback.c
index 5240e42..945559f 100644 (file)
--- a/mm/page-writeback.c
+++ b/mm/page-writeback.c
@@ -46,7 +46,7 @@
 static long ratelimit_pages = 32;
 
 static long total_pages;       /* The total number of pages in the machine. */
-static int dirty_exceeded;     /* Dirty mem may be over limit */
+static int dirty_exceeded __cacheline_aligned_in_smp;  /* Dirty mem may be over limit */
 
 /*
  * When balance_dirty_pages decides that the caller needs to perform some
@@ -212,7 +212,8 @@ static void balance_dirty_pages(struct address_space *mapping)
                if (nr_reclaimable + wbs.nr_writeback <= dirty_thresh)
                        break;
 
-               dirty_exceeded = 1;
+               if (!dirty_exceeded)
+                       dirty_exceeded = 1;
 
                /* Note: nr_reclaimable denotes nr_dirty + nr_unstable.
                 * Unstable writes are a feature of certain networked
@@ -234,7 +235,7 @@ static void balance_dirty_pages(struct address_space *mapping)
                blk_congestion_wait(WRITE, HZ/10);
        }
 
-       if (nr_reclaimable + wbs.nr_writeback <= dirty_thresh)
+       if (nr_reclaimable + wbs.nr_writeback <= dirty_thresh && dirty_exceeded)
                dirty_exceeded = 0;
 
        if (writeback_in_progress(bdi))