rcu: Make rcu_barrier() less disruptive
authorPaul E. McKenney <paul.mckenney@linaro.org>
Thu, 1 Mar 2012 21:18:08 +0000 (13:18 -0800)
committerPaul E. McKenney <paulmck@linux.vnet.ibm.com>
Wed, 9 May 2012 21:27:54 +0000 (14:27 -0700)
commitb1420f1c8bfc30ecf6380a31d0f686884834b599
tree56eb378bcd64175a7302a1031b388ecf569a30bb
parent98248a0e24327bc64eb7518145c44bff7bebebc3
rcu: Make rcu_barrier() less disruptive

The rcu_barrier() primitive interrupts each and every CPU, registering
a callback on every CPU.  Once all of these callbacks have been invoked,
rcu_barrier() knows that every callback that was registered before
the call to rcu_barrier() has also been invoked.

However, there is no point in registering a callback on a CPU that
currently has no callbacks, most especially if that CPU is in a
deep idle state.  This commit therefore makes rcu_barrier() avoid
interrupting CPUs that have no callbacks.  Doing this requires reworking
the handling of orphaned callbacks, otherwise callbacks could slip through
rcu_barrier()'s net by being orphaned from a CPU that rcu_barrier() had
not yet interrupted to a CPU that rcu_barrier() had already interrupted.
This reworking was needed anyway to take a first step towards weaning
RCU from the CPU_DYING notifier's use of stop_cpu().

Signed-off-by: Paul E. McKenney <paul.mckenney@linaro.org>
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
kernel/rcutree.c
kernel/rcutree.h
kernel/rcutree_trace.c