net dst: use a percpu_counter to track entries
authorEric Dumazet <eric.dumazet@gmail.com>
Fri, 8 Oct 2010 06:37:34 +0000 (06:37 +0000)
committerDavid S. Miller <davem@davemloft.net>
Mon, 11 Oct 2010 20:06:53 +0000 (13:06 -0700)
commitfc66f95c68b6d4535a0ea2ea15d5cf626e310956
treeac3a7f08ad741a67ff683bf93e5669ddcae95ed7
parent0ed8ddf4045fcfcac36bad753dc4046118c603ec
net dst: use a percpu_counter to track entries

struct dst_ops tracks number of allocated dst in an atomic_t field,
subject to high cache line contention in stress workload.

Switch to a percpu_counter, to reduce number of time we need to dirty a
central location. Place it on a separate cache line to avoid dirtying
read only fields.

Stress test :

(Sending 160.000.000 UDP frames,
IP route cache disabled, dual E5540 @2.53GHz,
32bit kernel, FIB_TRIE, SLUB/NUMA)

Before:

real    0m51.179s
user    0m15.329s
sys     10m15.942s

After:

real 0m45.570s
user 0m15.525s
sys 9m56.669s

With a small reordering of struct neighbour fields, subject of a
following patch, (to separate refcnt from other read mostly fields)

real 0m41.841s
user 0m15.261s
sys 8m45.949s

Signed-off-by: Eric Dumazet <eric.dumazet@gmail.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
include/net/dst_ops.h
net/bridge/br_netfilter.c
net/core/dst.c
net/decnet/dn_route.c
net/ipv4/route.c
net/ipv4/xfrm4_policy.c
net/ipv6/route.c
net/ipv6/xfrm6_policy.c