net: avoid RCU for NOCACHE dst
authorEric Dumazet <eric.dumazet@gmail.com>
Fri, 15 Oct 2010 05:44:11 +0000 (05:44 +0000)
committerDavid S. Miller <davem@davemloft.net>
Wed, 20 Oct 2010 10:02:23 +0000 (03:02 -0700)
commit27b75c95f10d249574d9c4cb9dab878107faede8
tree466656d86aaa395951e12b50903e730203c5f86f
parente6484930d7c73d324bccda7d43d131088da697b9
net: avoid RCU for NOCACHE dst

There is no point using RCU for dst we allocate for a very short time
(used once).

Change dst_release() to take DST_NOCACHE into account, but also change
skb_dst_set_noref() to force a refcount increment for such dst.

This is a _huge_ gain, because we dont waste memory to store xx thousand
of dsts. Instead of queueing them to RCU, we can free them instantly.

CPU caches can stay hot, re-using same memory blocks to hold temporary
dsts.

Note : remove unneeded smp_mb__before_atomic_dec(); in dst_release(),
since atomic_dec_return() implies a full memory barrier.

Stress test, 160.000.000 udp frames sent, IP route cache disabled
(DDOS).

Before:

real    0m38.091s
user    0m13.189s
sys     7m53.018s

After:

real 0m29.946s
user 0m12.157s
sys 7m40.605s

For reference, if IP route cache was enabled :

real 0m32.030s
user 0m10.521s
sys 8m15.243s

Signed-off-by: Eric Dumazet <eric.dumazet@gmail.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
include/linux/skbuff.h
net/core/dst.c
net/ipv4/route.c