pandora-kernel.git
17 years ago[PATCH] fix potential stack overflow in mm/slab.c
Siddha, Suresh B [Tue, 26 Sep 2006 06:31:47 +0000 (23:31 -0700)]
[PATCH] fix potential stack overflow in mm/slab.c

On High end systems (1024 or so cpus) this can potentially cause stack
overflow. Fix the stack usage.

Signed-off-by: Suresh Siddha <suresh.b.siddha@intel.com>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
17 years ago[PATCH] Define easier to handle GFP_THISNODE
Christoph Lameter [Tue, 26 Sep 2006 06:31:46 +0000 (23:31 -0700)]
[PATCH] Define easier to handle GFP_THISNODE

In many places we will need to use the same combination of flags.  Specify
a single GFP_THISNODE definition for ease of use in gfp.h.

Signed-off-by: Christoph Lameter <clameter@sgi.com>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
17 years ago[PATCH] Profiling: require buffer allocation on the correct node
Christoph Lameter [Tue, 26 Sep 2006 06:31:45 +0000 (23:31 -0700)]
[PATCH] Profiling: require buffer allocation on the correct node

Profiling really suffers with off node buffers.  Fail if no memory is
available on the nodes.  The profiling code can deal with these failures
should they occur.

Signed-off-by: Christoph Lameter <clameter@sgi.com>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
17 years ago[PATCH] Cleanup: Add zone pointer to get_page_from_freelist
Christoph Lameter [Tue, 26 Sep 2006 06:31:45 +0000 (23:31 -0700)]
[PATCH] Cleanup: Add zone pointer to get_page_from_freelist

There are frequent references to *z in get_page_from_freelist.

Add an explicit zone variable that can be used in all these places.

Signed-off-by: Christoph Lameter <clameter@sgi.com>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
17 years ago[PATCH] Guarantee that the uncached allocator gets pages on the correct node
Christoph Lameter [Tue, 26 Sep 2006 06:31:44 +0000 (23:31 -0700)]
[PATCH] Guarantee that the uncached allocator gets pages on the correct node

The uncached allocator manages per node pools.  Specify __GFP_THISNODE in
order to force allocation on the indicated node or fail.  The uncached
allocator has already logic to deal with failing allocations.

Signed-off-by: Christoph Lameter <clameter@sgi.com>
Cc: Andy Whitcroft <apw@shadowen.org>
Cc: Mel Gorman <mel@csn.ul.ie>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
17 years ago[PATCH] sys_move_pages: Do not fall back to other nodes
Christoph Lameter [Tue, 26 Sep 2006 06:31:43 +0000 (23:31 -0700)]
[PATCH] sys_move_pages: Do not fall back to other nodes

If the user specified a node where we should move the page to then we
really do not want any other node.

Signed-off-by: Christoph Lameter <clameter@sgi.com>
Cc: Andy Whitcroft <apw@shadowen.org>
Cc: Mel Gorman <mel@csn.ul.ie>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
17 years ago[PATCH] Add __GFP_THISNODE to avoid fallback to other nodes and ignore cpuset/memory...
Christoph Lameter [Tue, 26 Sep 2006 06:31:40 +0000 (23:31 -0700)]
[PATCH] Add __GFP_THISNODE to avoid fallback to other nodes and ignore cpuset/memory policy restrictions

Add a new gfp flag __GFP_THISNODE to avoid fallback to other nodes.  This
flag is essential if a kernel component requires memory to be located on a
certain node.  It will be needed for alloc_pages_node() to force allocation
on the indicated node and for alloc_pages() to force allocation on the
current node.

Signed-off-by: Christoph Lameter <clameter@sgi.com>
Cc: Andy Whitcroft <apw@shadowen.org>
Cc: Mel Gorman <mel@csn.ul.ie>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
17 years ago[PATCH] slab: fix lockdep warnings
Ravikiran G Thirumalai [Tue, 26 Sep 2006 06:31:38 +0000 (23:31 -0700)]
[PATCH] slab: fix lockdep warnings

Place the alien array cache locks of on slab malloc slab caches on a
seperate lockdep class.  This avoids false positives from lockdep

[akpm@osdl.org: build fix]
Signed-off-by: Ravikiran Thirumalai <kiran@scalex86.org>
Signed-off-by: Shai Fultheim <shai@scalex86.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Acked-by: Arjan van de Ven <arjan@linux.intel.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Pekka Enberg <penberg@cs.helsinki.fi>
Cc: Christoph Lameter <clameter@engr.sgi.com>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
17 years ago[PATCH] slab: do not panic when alloc_kmemlist fails and slab is up
Christoph Lameter [Tue, 26 Sep 2006 06:31:38 +0000 (23:31 -0700)]
[PATCH] slab: do not panic when alloc_kmemlist fails and slab is up

It is fairly easy to get a system to oops by simply sizing a cache via
/proc in such a way that one of the chaches (shared is easiest) becomes
bigger than the maximum allowed slab allocation size.  This occurs because
enable_cpucache() fails if it cannot reallocate some caches.

However, enable_cpucache() is used for multiple purposes: resizing caches,
cache creation and bootstrap.

If the slab is already up then we already have working caches.  The resize
can fail without a problem.  We just need to return the proper error code.
F.e.  after this patch:

# echo "size-64 10000 50 1000" >/proc/slabinfo
-bash: echo: write error: Cannot allocate memory

notice no OOPS.

If we are doing a kmem_cache_create() then we also should not panic but
return -ENOMEM.

If on the other hand we do not have a fully bootstrapped slab allocator yet
then we should indeed panic since we are unable to bring up the slab to its
full functionality.

Signed-off-by: Christoph Lameter <clameter@sgi.com>
Cc: Pekka Enberg <penberg@cs.helsinki.fi>
Cc: Manfred Spraul <manfred@colorfullife.com>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
17 years ago[PATCH] slab: extract __kmem_cache_destroy from kmem_cache_destroy
Christoph Lameter [Tue, 26 Sep 2006 06:31:37 +0000 (23:31 -0700)]
[PATCH] slab: extract __kmem_cache_destroy from kmem_cache_destroy

The ability to free memory allocated to a slab cache is also useful if an
error occurs during setup of a slab.  So extract the function.

Signed-off-by: Christoph Lameter <clameter@sgi.com>
Cc: Pekka Enberg <penberg@cs.helsinki.fi>
Cc: Manfred Spraul <manfred@colorfullife.com>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
17 years ago[PATCH] slab: optimize kmalloc_node the same way as kmalloc
Christoph Hellwig [Tue, 26 Sep 2006 06:31:36 +0000 (23:31 -0700)]
[PATCH] slab: optimize kmalloc_node the same way as kmalloc

[akpm@osdl.org: export fix]
Signed-off-by: Christoph Hellwig <hch@lst.de>
Acked-by: Christoph Lameter <clameter@sgi.com>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
17 years ago[PATCH] update some mm/ comments
Nick Piggin [Tue, 26 Sep 2006 06:31:35 +0000 (23:31 -0700)]
[PATCH] update some mm/ comments

Let's try to keep mm/ comments more useful and up to date. This is a start.

Signed-off-by: Nick Piggin <npiggin@suse.de>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
17 years ago[PATCH] Add some comments to slab.c
Ravikiran G Thirumalai [Tue, 26 Sep 2006 06:31:34 +0000 (23:31 -0700)]
[PATCH] Add some comments to slab.c

Also, checks if we get a valid slabp_cache for off slab slab-descriptors.
We should always get this.  If we don't, then in that case we, will have to
disable off-slab descriptors for this cache and do the calculations again.
This is a rare case, so add a BUG_ON, for now, just in case.

Signed-off-by: Alok N Kataria <alok.kataria@calsoftinc.com>
Signed-off-by: Ravikiran Thirumalai <kiran@scalex86.org>
Signed-off-by: Shai Fultheim <shai@scalex86.org>
Cc: Pekka Enberg <penberg@cs.helsinki.fi>
Cc: Manfred Spraul <manfred@colorfullife.com>
Cc: Christoph Lameter <clameter@engr.sgi.com>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
17 years ago[PATCH] bootmem: use MAX_DMA_ADDRESS instead of LOW32LIMIT
Heiko Carstens [Tue, 26 Sep 2006 06:31:33 +0000 (23:31 -0700)]
[PATCH] bootmem: use MAX_DMA_ADDRESS instead of LOW32LIMIT

Introduce ARCH_LOW_ADDRESS_LIMIT which can be set per architecture to
override the 4GB default limit used by the bootmem allocater within
__alloc_bootmem_low() and __alloc_bootmem_low_node().  E.g.  s390 needs a
2GB limit instead of 4GB.

Acked-by: Ingo Molnar <mingo@elte.hu>
Cc: Martin Schwidefsky <schwidefsky@de.ibm.com>
Signed-off-by: Heiko Carstens <heiko.carstens@de.ibm.com>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
17 years ago[PATCH] oom: more printk
Nick Piggin [Tue, 26 Sep 2006 06:31:32 +0000 (23:31 -0700)]
[PATCH] oom: more printk

Print the name of the task invoking the OOM killer.  Could make debugging
easier.

Signed-off-by: Nick Piggin <npiggin@suse.de>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
17 years ago[PATCH] oom: kthread infinite loop fix
Nick Piggin [Tue, 26 Sep 2006 06:31:32 +0000 (23:31 -0700)]
[PATCH] oom: kthread infinite loop fix

Skip kernel threads, rather than having them return 0 from badness.
Theoretically, badness might truncate all results to 0, thus a kernel thread
might be picked first, causing an infinite loop.

Signed-off-by: Nick Piggin <npiggin@suse.de>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
17 years ago[PATCH] oom: swapoff tasks tweak
Nick Piggin [Tue, 26 Sep 2006 06:31:31 +0000 (23:31 -0700)]
[PATCH] oom: swapoff tasks tweak

PF_SWAPOFF processes currently cause select_bad_process to return straight
away.  Instead, give them high priority, so we will kill them first, however
we also first ensure no parallel OOM kills are happening at the same time.

Signed-off-by: Nick Piggin <npiggin@suse.de>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
17 years ago[PATCH] oom: handle oom_disable exiting
Nick Piggin [Tue, 26 Sep 2006 06:31:30 +0000 (23:31 -0700)]
[PATCH] oom: handle oom_disable exiting

Having the oomkilladj == OOM_DISABLE check before the releasing check means
that oomkilladj == OOM_DISABLE tasks exiting will not stop the OOM killer.

Moving the test down will give the desired behaviour.  Also: it will allow
them to "OOM-kill" themselves if they are exiting.  As per the previous patch,
this is required to prevent OOM killer deadlocks (and they don't actually get
killed, because they're already exiting -- they're simply allowed access to
memory reserves).

Signed-off-by: Nick Piggin <npiggin@suse.de>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
17 years ago[PATCH] oom: handle current exiting
Nick Piggin [Tue, 26 Sep 2006 06:31:29 +0000 (23:31 -0700)]
[PATCH] oom: handle current exiting

If current *is* exiting, it should actually be allowed to access reserved
memory rather than OOM kill something else.  Can't do this via a straight
check in page_alloc.c because that would allow multiple tasks to use up
reserves.  Instead cause current to OOM-kill itself which will mark it as
TIF_MEMDIE.

The current procedure of simply aborting the OOM-kill if a task is exiting can
lead to OOM deadlocks.

In the case of killing a PF_EXITING task, don't make a lot of noise about it.
This becomes more important in future patches, where we can "kill" OOM_DISABLE
tasks.

Signed-off-by: Nick Piggin <npiggin@suse.de>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
17 years ago[PATCH] oom: cpuset hint
Nick Piggin [Tue, 26 Sep 2006 06:31:29 +0000 (23:31 -0700)]
[PATCH] oom: cpuset hint

cpuset_excl_nodes_overlap does not always indicate that killing a task will
not free any memory we for us.  For example, we may be asking for an
allocation from _anywhere_ in the machine, or the task in question may be
pinning memory that is outside its cpuset.  Fix this by just causing
cpuset_excl_nodes_overlap to reduce the badness rather than disallow it.

Signed-off-by: Nick Piggin <npiggin@suse.de>
Acked-by: Paul Jackson <pj@sgi.com>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
17 years ago[PATCH] oom: reclaim_mapped on oom
Nick Piggin [Tue, 26 Sep 2006 06:31:28 +0000 (23:31 -0700)]
[PATCH] oom: reclaim_mapped on oom

Potentially it takes several scans of the lru lists before we can even start
reclaiming pages.

mapped pages, with young ptes can take 2 passes on the active list + one on
the inactive list.  But reclaim_mapped may not always kick in instantly, so it
could take even more than that.

Raise the threshold for marking a zone as all_unreclaimable from a factor of 4
time the pages in the zone to 6.  Introduce a mechanism to force
reclaim_mapped if we've reached a factor 3 and still haven't made progress.

Previously, a customer doing stress testing was able to easily OOM the box
after using only a small fraction of its swap (~100MB).  After the patches, it
would only OOM after having used up all swap (~800MB).

Signed-off-by: Nick Piggin <npiggin@suse.de>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
17 years ago[PATCH] oom: use unreclaimable info
Nick Piggin [Tue, 26 Sep 2006 06:31:27 +0000 (23:31 -0700)]
[PATCH] oom: use unreclaimable info

__alloc_pages currently starts shooting if page reclaim has failed to free up
swap_cluster_max pages in one run through the priorities.  This is not always
a good indicator on its own, so make use of the all_unreclaimable logic as
well: don't consider going OOM until all zones we're interested in are
unreclaimable.

Signed-off-by: Nick Piggin <npiggin@suse.de>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
17 years ago[PATCH] mm: swap write failure fixup
Peter Zijlstra [Tue, 26 Sep 2006 06:31:26 +0000 (23:31 -0700)]
[PATCH] mm: swap write failure fixup

Currently we can silently drop data if the write to swap failed.  It
usually doesn't result in data-corruption because on page-in the process
will receive SIGBUS (assuming write-failure implies read-failure).

This assumption might or might not be valid.

This patch will avoid the page being discarded after a failed write.  But
will print a warning the sysadmin _should_ take to heart, if a lot of swap
space becomes un-writeable, OOM is not far off.

Tested by making the write fail 'randomly' once every 50 writes or so.

[akpm@osdl.org: printk warning fix]
Signed-off-by: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Hugh Dickins <hugh@veritas.com>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
17 years ago[PATCH] slab: respect architecture and caller mandated alignment
Pekka Enberg [Tue, 26 Sep 2006 06:31:25 +0000 (23:31 -0700)]
[PATCH] slab: respect architecture and caller mandated alignment

As explained by Heiko, on s390 (32-bit) ARCH_KMALLOC_MINALIGN is set to
eight because their common I/O layer allocates data structures that need to
have an eight byte alignment.  This does not work when CONFIG_SLAB_DEBUG is
enabled because kmem_cache_create will override alignment to BYTES_PER_WORD
which is four.

So change kmem_cache_create to ensure cache alignment is always at minimum
what the architecture or caller mandates even if slab debugging is enabled.

Cc: Heiko Carstens <heiko.carstens@de.ibm.com>
Cc: Christoph Lameter <clameter@sgi.com>
Signed-off-by: Manfred Spraul <manfred@colorfullife.com>
Signed-off-by: Pekka Enberg <penberg@cs.helsinki.fi>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
17 years ago[PATCH] mm: non syncing lock_page()
Nick Piggin [Tue, 26 Sep 2006 06:31:24 +0000 (23:31 -0700)]
[PATCH] mm: non syncing lock_page()

lock_page needs the caller to have a reference on the page->mapping inode
due to sync_page, ergo set_page_dirty_lock is obviously buggy according to
its comments.

Solve it by introducing a new lock_page_nosync which does not do a sync_page.

akpm: unpleasant solution to an unpleasant problem.  If it goes wrong it could
cause great slowdowns while the lock_page() caller waits for kblockd to
perform the unplug.  And if a filesystem has special sync_page() requirements
(none presently do), permanent hangs are possible.

otoh, set_page_dirty_lock() is usually (always?) called against userspace
pages.  They are always up-to-date, so there shouldn't be any pending read I/O
against these pages.

Signed-off-by: Nick Piggin <npiggin@suse.de>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
17 years ago[PATCH] mm: remove_mapping() safeness
Nick Piggin [Tue, 26 Sep 2006 06:31:23 +0000 (23:31 -0700)]
[PATCH] mm: remove_mapping() safeness

Some users of remove_mapping had been unsafe.

Modify the remove_mapping precondition to ensure the caller has locked the
page and obtained the correct mapping.  Modify callers to ensure the
mapping is the correct one.

[hugh@veritas.com: swapper_space fix]
Signed-off-by: Nick Piggin <npiggin@suse.de>
Signed-off-by: Hugh Dickins <hugh@veritas.com>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
17 years ago[PATCH] Add kerneldocs for some functions in mm/memory.c
Rolf Eike Beer [Tue, 26 Sep 2006 06:31:22 +0000 (23:31 -0700)]
[PATCH] Add kerneldocs for some functions in mm/memory.c

These functions are already documented quite well with long comments.  Now
add kerneldoc style header to make this turn up in everyones favorite doc
format.

Signed-off-by: Rolf Eike Beer <eike-kernel@sf-tec.de>
Cc: "Randy.Dunlap" <rdunlap@xenotime.net>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
17 years ago[PATCH] CPU hotplug compatible alloc_percpu()
Martin Peschke [Tue, 26 Sep 2006 06:31:21 +0000 (23:31 -0700)]
[PATCH] CPU hotplug compatible alloc_percpu()

This patch splits alloc_percpu() up into two phases.  Likewise for
free_percpu().  This allows clients to limit initial allocations to online
cpu's, and to populate or depopulate per-cpu data at run time as needed:

  struct my_struct *obj;

  /* initial allocation for online cpu's */
  obj = percpu_alloc(sizeof(struct my_struct), GFP_KERNEL);

  ...

  /* populate per-cpu data for cpu coming online */
  ptr = percpu_populate(obj, sizeof(struct my_struct), GFP_KERNEL, cpu);

  ...

  /* access per-cpu object */
  ptr = percpu_ptr(obj, smp_processor_id());

  ...

  /* depopulate per-cpu data for cpu going offline */
  percpu_depopulate(obj, cpu);

  ...

  /* final removal */
  percpu_free(obj);

Signed-off-by: Martin Peschke <mp3@de.ibm.com>
Cc: Paul Jackson <pj@sgi.com>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
17 years ago[PATCH] out of memory notifier
Martin Schwidefsky [Tue, 26 Sep 2006 06:31:20 +0000 (23:31 -0700)]
[PATCH] out of memory notifier

Add a notifer chain to the out of memory killer.  If one of the registered
callbacks could release some memory, do not kill the process but return and
retry the allocation that forced the oom killer to run.

The purpose of the notifier is to add a safety net in the presence of
memory ballooners.  If the resource manager inflated the balloon to a size
where memory allocations can not be satisfied anymore, it is better to
deflate the balloon a bit instead of killing processes.

The implementation for the s390 ballooner is included.

[akpm@osdl.org: cleanups]
Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
17 years ago[PATCH] linearly index zone->node_zonelists[]
Christoph Lameter [Tue, 26 Sep 2006 06:31:19 +0000 (23:31 -0700)]
[PATCH] linearly index zone->node_zonelists[]

I wonder why we need this bitmask indexing into zone->node_zonelists[]?

We always start with the highest zone and then include all lower zones
if we build zonelists.

Are there really cases where we need allocation from ZONE_DMA or
ZONE_HIGHMEM but not ZONE_NORMAL? It seems that the current implementation
of highest_zone() makes that already impossible.

If we go linear on the index then gfp_zone() == highest_zone() and a lot
of definitions fall by the wayside.

We can now revert back to the use of gfp_zone() in mempolicy.c ;-)

Signed-off-by: Christoph Lameter <clameter@sgi.com>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
17 years ago[PATCH] Apply type enum zone_type
Christoph Lameter [Tue, 26 Sep 2006 06:31:18 +0000 (23:31 -0700)]
[PATCH] Apply type enum zone_type

After we have done this we can now do some typing cleanup.

The memory policy layer keeps a policy_zone that specifies
the zone that gets memory policies applied. This variable
can now be of type enum zone_type.

The check_highest_zone function and the build_zonelists funnctionm must
then also take a enum zone_type parameter.

Plus there are a number of loops over zones that also should use
zone_type.

We run into some troubles at some points with functions that need a
zone_type variable to become -1. Fix that up.

[pj@sgi.com: fix set_mempolicy() crash]
Signed-off-by: Christoph Lameter <clameter@sgi.com>
Signed-off-by: Paul Jackson <pj@sgi.com>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
17 years ago[PATCH] mempolicies: fix policy_zone check
Christoph Lameter [Tue, 26 Sep 2006 06:31:17 +0000 (23:31 -0700)]
[PATCH] mempolicies: fix policy_zone check

There is a check in zonelist_policy that compares pieces of the bitmap
obtained from a gfp mask via GFP_ZONETYPES with a zone number in function
zonelist_policy().

The bitmap is an ORed mask of __GFP_DMA, __GFP_DMA32 and __GFP_HIGHMEM.
The policy_zone is a zone number with the possible values of ZONE_DMA,
ZONE_DMA32, ZONE_HIGHMEM and ZONE_NORMAL. These are two different domains
of values.

For some reason seemed to work before the zone reduction patchset (It
definitely works on SGI boxes since we just have one zone and the check
cannot fail).

With the zone reduction patchset this check definitely fails on systems
with two zones if the system actually has memory in both zones.

This is because ZONE_NORMAL is selected using no __GFP flag at
all and thus gfp_zone(gfpmask) == 0. ZONE_DMA is selected when __GFP_DMA
is set. __GFP_DMA is 0x01.  So gfp_zone(gfpmask) == 1.

policy_zone is set to ZONE_NORMAL (==1) if ZONE_NORMAL and ZONE_DMA are
populated.

For ZONE_NORMAL gfp_zone(<no _GFP_DMA>) yields 0 which is <
policy_zone(ZONE_NORMAL) and so policy is not applied to regular memory
allocations!

Instead gfp_zone(__GFP_DMA) == 1 which results in policy being applied
to DMA allocations!

What we realy want in that place is to establish the highest allowable
zone for a given gfp_mask. If the highest zone is higher or equal to the
policy_zone then memory policies need to be applied. We have such
a highest_zone() function in page_alloc.c.

So move the highest_zone() function from mm/page_alloc.c into
include/linux/gfp.h.  On the way we simplify the function and use the new
zone_type that was also introduced with the zone reduction patchset plus we
also specify the right type for the gfp flags parameter.

Signed-off-by: Christoph Lameter <clameter@sgi.com>
Signed-off-by: Lee Schermerhorn <Lee.Schermerhorn@hp.com>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
17 years ago[PATCH] reduce MAX_NR_ZONES: fix i386 SRAT check for MAX_NR_ZONES
Christoph Lameter [Tue, 26 Sep 2006 06:31:16 +0000 (23:31 -0700)]
[PATCH] reduce MAX_NR_ZONES: fix i386 SRAT check for MAX_NR_ZONES

We cannot check MAX_NR_ZONES since it not defined in the preprocessor
anymore.

So remove the check.

The maximum number of zones per node for i386 is 3 since i386 does not
support ZONE_DMA32.

Signed-off-by: Christoph Lameter <clameter@sgi.com>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
17 years ago[PATCH] reduce MAX_NR_ZONES: remove display of counters for unconfigured zones
Christoph Lameter [Tue, 26 Sep 2006 06:31:15 +0000 (23:31 -0700)]
[PATCH] reduce MAX_NR_ZONES: remove display of counters for unconfigured zones

eventcounters: Do not display counters for zones that are not available on an
arch

Do not define or display counters for the DMA32 and the HIGHMEM zone if such
zones were not configured.

[akpm@osdl.org: s390 fix]
[heiko.carstens@de.ibm.com: s390 fix]
Signed-off-by: Christoph Lameter <clameter@sgi.com>
Cc: Martin Schwidefsky <schwidefsky@de.ibm.com>
Signed-off-by: Heiko Carstens <heiko.carstens@de.ibm.com>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
17 years ago[PATCH] reduce MAX_NR_ZONES: make ZONE_HIGHMEM optional
Christoph Lameter [Tue, 26 Sep 2006 06:31:14 +0000 (23:31 -0700)]
[PATCH] reduce MAX_NR_ZONES: make ZONE_HIGHMEM optional

Make ZONE_HIGHMEM optional

- ifdef out code and definitions related to CONFIG_HIGHMEM

- __GFP_HIGHMEM falls back to normal allocations if there is no
  ZONE_HIGHMEM

- GFP_ZONEMASK becomes 0x01 if there is no DMA32 and no HIGHMEM
  zone.

[jdike@addtoit.com: build fix]
Signed-off-by: Jeff Dike <jdike@addtoit.com>
Signed-off-by: Christoph Lameter <clameter@engr.sgi.com>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
17 years ago[PATCH] reduce MAX_NR_ZONES: make ZONE_DMA32 optional
Christoph Lameter [Tue, 26 Sep 2006 06:31:13 +0000 (23:31 -0700)]
[PATCH] reduce MAX_NR_ZONES: make ZONE_DMA32 optional

Make ZONE_DMA32 optional

- Add #ifdefs around ZONE_DMA32 specific code and definitions.

- Add CONFIG_ZONE_DMA32 config option and use that for x86_64
  that alone needs this zone.

- Remove the use of CONFIG_DMA_IS_DMA32 and CONFIG_DMA_IS_NORMAL
  for ia64 and fix up the way per node ZVCs are calculated.

- Fall back to prior GFP_ZONEMASK of 0x03 if there is no
  DMA32 zone.

Signed-off-by: Christoph Lameter <clameter@sgi.com>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
17 years ago[PATCH] reduce MAX_NR_ZONES: use enum to define zones, reformat and comment
Christoph Lameter [Tue, 26 Sep 2006 06:31:13 +0000 (23:31 -0700)]
[PATCH] reduce MAX_NR_ZONES: use enum to define zones, reformat and comment

Use enum for zones and reformat zones dependent information

Add comments explaning the use of zones and add a zones_t type for zone
numbers.

Line up information that will be #ifdefd by the following patches.

[akpm@osdl.org: comment cleanups]
Signed-off-by: Christoph Lameter <clameter@sgi.com>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
17 years ago[PATCH] reduce MAX_NR_ZONES: page allocator ZONE_HIGHMEM cleanup
Christoph Lameter [Tue, 26 Sep 2006 06:31:12 +0000 (23:31 -0700)]
[PATCH] reduce MAX_NR_ZONES: page allocator ZONE_HIGHMEM cleanup

page allocator ZONE_HIGHMEM fixups

1. We do not need to do an #ifdef in si_meminfo since both counters
   in use are zero if !CONFIG_HIGHMEM.

2. Add #ifdef in si_meminfo_node instead to avoid referencing zone
   information for ZONE_HIGHMEM if we do not have HIGHMEM
   (may not be there after the following patches).

3. Replace the use of ZONE_HIGHMEM with MAX_NR_ZONES in build_zonelists_node

4. build_zonelists_node: Remove BUG_ON for ZONE_HIGHMEM. Zone will
   be optional soon and thus BUG_ON cannot be triggered anymore.

5. init_free_area_core: Replace a use of ZONE_HIGHMEM with NR_MAX_ZONES.

[akpm@osdl.org: cleanups]
Signed-off-by: Christoph Lameter <clameter@sgi.com>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
17 years ago[PATCH] reduce MAX_NR_ZONES: move HIGHMEM counters into highmem.c/.h
Christoph Lameter [Tue, 26 Sep 2006 06:31:11 +0000 (23:31 -0700)]
[PATCH] reduce MAX_NR_ZONES: move HIGHMEM counters into highmem.c/.h

Move totalhigh_pages and nr_free_highpages() into highmem.c/.h

Move the totalhigh_pages definition into highmem.c/.h.  Move the
nr_free_highpages function into highmem.c

[yoichi_yuasa@tripeaks.co.jp: build fix]
Signed-off-by: Christoph Lameter <clameter@sgi.com>
Signed-off-by: Yoichi Yuasa <yoichi_yuasa@tripeaks.co.jp>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
17 years ago[PATCH] reduce MAX_NR_ZONES: make display of highmem counters conditional on CONFIG_H...
Christoph Lameter [Tue, 26 Sep 2006 06:31:10 +0000 (23:31 -0700)]
[PATCH] reduce MAX_NR_ZONES: make display of highmem counters conditional on CONFIG_HIGHMEM

Do not display HIGHMEM memory sizes if CONFIG_HIGHMEM is not set.

Make HIGHMEM dependent texts and make display of highmem counters optional

Some texts are depending on CONFIG_HIGHMEM.

Remove those strings and remove the display of highmem counter values if
CONFIG_HIGHMEM is not set.

[akpm@osdl.org: remove some ifdefs]
Signed-off-by: Christoph Lameter <clameter@sgi.com>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
17 years ago[PATCH] reduce MAX_NR_ZONES: fix MAX_NR_ZONES array initializations
Christoph Lameter [Tue, 26 Sep 2006 06:31:10 +0000 (23:31 -0700)]
[PATCH] reduce MAX_NR_ZONES: fix MAX_NR_ZONES array initializations

Fix array initialization in lots of arches

The number of zones may now be reduced from 4 to 2 for many arches.  Fix the
array initialization for the zones array for all architectures so that it is
not initializing a fixed number of elements.

Signed-off-by: Christoph Lameter <clameter@sgi.com>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
17 years ago[PATCH] reduce MAX_NR_ZONES: remove two strange uses of MAX_NR_ZONES
Christoph Lameter [Tue, 26 Sep 2006 06:31:09 +0000 (23:31 -0700)]
[PATCH] reduce MAX_NR_ZONES: remove two strange uses of MAX_NR_ZONES

I keep seeing zones on various platforms that are never used and wonder why we
compile support for them into the kernel.  Counters show up for HIGHMEM and
DMA32 that are alway zero.

This patch allows the removal of ZONE_DMA32 for non x86_64 architectures and
it will get rid of ZONE_HIGHMEM for arches not using highmem (like 64 bit
architectures).  If an arch does not define CONFIG_HIGHMEM then ZONE_HIGHMEM
will not be defined.  Similarly if an arch does not define CONFIG_ZONE_DMA32
then ZONE_DMA32 will not be defined.

No current architecture uses all the 4 zones (DMA,DMA32,NORMAL,HIGH) that we
have now.  The patchset will reduce the number of zones for all platforms.

On many platforms that do not have DMA32 or HIGHMEM this will reduce the
number of zones by 50%.  F.e.  ia64 only uses DMA and NORMAL.

Large amounts of memory can be saved for larger systemss that may have a few
hundred NUMA nodes.

With ZONE_DMA32 and ZONE_HIGHMEM support optional MAX_NR_ZONES will be 2 for
many non i386 platforms and even for i386 without CONFIG_HIGHMEM set.

Tested on ia64, x86_64 and on i386 with and without highmem.

The patchset consists of 11 patches that are following this message.

One could go even further than this patchset and also make ZONE_DMA optional
because some platforms do not need a separate DMA zone and can do DMA to all
of memory.  This could reduce MAX_NR_ZONES to 1.  Such a patchset will
hopefully follow soon.

This patch:

Fix strange uses of MAX_NR_ZONES

Sometimes we use MAX_NR_ZONES - x to refer to a zone.  Make that explicit.

Signed-off-by: Christoph Lameter <clameter@sgi.com>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
17 years ago[PATCH] bootmem: miscellaneous coding style fixes
Franck Bui-Huu [Tue, 26 Sep 2006 06:31:08 +0000 (23:31 -0700)]
[PATCH] bootmem: miscellaneous coding style fixes

It fixes various coding style issues, specially when spaces are useless.  For
example '*' go next to the function name.

Signed-off-by: Franck Bui-Huu <vagabon.xyz@gmail.com>
Cc: Dave Hansen <haveblue@us.ibm.com>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
17 years ago[PATCH] bootmem: use pfn/page conversion macros
Franck Bui-Huu [Tue, 26 Sep 2006 06:31:07 +0000 (23:31 -0700)]
[PATCH] bootmem: use pfn/page conversion macros

It also creates get_mapsize() helper in order to make the code more readable
when it calculates the boot bitmap size.

Signed-off-by: Franck Bui-Huu <vagabon.xyz@gmail.com>
Cc: Dave Hansen <haveblue@us.ibm.com>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
17 years ago[PATCH] bootmem: remove useless headers inclusions
Franck Bui-Huu [Tue, 26 Sep 2006 06:31:06 +0000 (23:31 -0700)]
[PATCH] bootmem: remove useless headers inclusions

Signed-off-by: Franck Bui-Huu <vagabon.xyz@gmail.com>
Cc: Dave Hansen <haveblue@us.ibm.com>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
17 years ago[PATCH] bootmem: limit to 80 columns width
Franck Bui-Huu [Tue, 26 Sep 2006 06:31:05 +0000 (23:31 -0700)]
[PATCH] bootmem: limit to 80 columns width

Signed-off-by: Franck Bui-Huu <vagabon.xyz@gmail.com>
Cc: Dave Hansen <haveblue@us.ibm.com>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
17 years ago[PATCH] bootmem: remove useless parentheses in bootmem header file
Franck Bui-Huu [Tue, 26 Sep 2006 06:31:05 +0000 (23:31 -0700)]
[PATCH] bootmem: remove useless parentheses in bootmem header file

Signed-off-by: Franck Bui-Huu <vagabon.xyz@gmail.com>
Cc: Dave Hansen <haveblue@us.ibm.com>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
17 years ago[PATCH] bootmem: mark link_bootmem() as part of the __init section
Franck Bui-Huu [Tue, 26 Sep 2006 06:31:04 +0000 (23:31 -0700)]
[PATCH] bootmem: mark link_bootmem() as part of the __init section

Signed-off-by: Franck Bui-Huu <vagabon.xyz@gmail.com>
Cc: Dave Hansen <haveblue@us.ibm.com>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
17 years ago[PATCH] bootmem: remove useless __init in header file
Franck Bui-Huu [Tue, 26 Sep 2006 06:31:03 +0000 (23:31 -0700)]
[PATCH] bootmem: remove useless __init in header file

__init in headers is pretty useless because the compiler doesn't check it, and
they get out of sync relatively frequently.  So if you see an __init in a
header file, it's quite unreliable and you need to check the definition
anyway.

Signed-off-by: Franck Bui-Huu <vagabon.xyz@gmail.com>
Cc: Dave Hansen <haveblue@us.ibm.com>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
17 years ago[PATCH] convert i386 NUMA KVA space to bootmem
keith mannthey [Tue, 26 Sep 2006 06:31:03 +0000 (23:31 -0700)]
[PATCH] convert i386 NUMA KVA space to bootmem

Address a long standing issue of booting with an initrd on an i386 numa
system.  Currently (and always) the numa kva area is mapped into low memory
by finding the end of low memory and moving that mark down (thus creating
space for the kva).  The issue with this is that Grub loads initrds into
this similar space so when the kernel check the initrd it finds it outside
max_low_pfn and disables it (it thinks the initrd is not mapped into usable
memory) thus initrd enabled kernels can't boot i386 numa :(

My solution to the problem just converts the numa kva area to use the
bootmem allocator to save it's area (instead of moving the end of low
memory).  Using bootmem allows the kva area to be mapped into more diverse
addresses (not just the end of low memory) and enables the kva area to be
mapped below the initrd if present.

I have tested this patch on numaq(no initrd) and summit(initrd) i386 numa
based systems.

[akpm@osdl.org: cleanups]
Signed-off-by: Keith Mannthey <kmannth@us.ibm.com>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
17 years ago[PATCH] mm/: make functions static
Adrian Bunk [Tue, 26 Sep 2006 06:31:02 +0000 (23:31 -0700)]
[PATCH] mm/: make functions static

This patch makes the following needlessly global functions static:
 - slab.c: kmem_find_general_cachep()
 - swap.c: __page_cache_release()
 - vmalloc.c: __vmalloc_node()

Signed-off-by: Adrian Bunk <bunk@stusta.de>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
17 years ago[PATCH] mm: msync() cleanup
Peter Zijlstra [Tue, 26 Sep 2006 06:31:01 +0000 (23:31 -0700)]
[PATCH] mm: msync() cleanup

With the tracking of dirty pages properly done now, msync doesn't need to scan
the PTEs anymore to determine the dirty status.

From: Hugh Dickins <hugh@veritas.com>

In looking to do that, I made some other tidyups: can remove several
#includes, and sys_msync loop termination not quite right.

Most of those points are criticisms of the existing sys_msync, not of your
patch.  In particular, the loop termination errors were introduced in 2.6.17:
I did notice this shortly before it came out, but decided I was more likely to
get it wrong myself, and make matters worse if I tried to rush a last-minute
fix in.  And it's not terribly likely to go wrong, nor disastrous if it does
go wrong (may miss reporting an unmapped area; may also fsync file of a
following vma).

Signed-off-by: Peter Zijlstra <a.p.zijlstra@chello.nl>
Signed-off-by: Hugh Dickins <hugh@veritas.com>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
17 years ago[PATCH] mm: fixup do_wp_page()
Peter Zijlstra [Tue, 26 Sep 2006 06:31:00 +0000 (23:31 -0700)]
[PATCH] mm: fixup do_wp_page()

Wrt. the recent modifications in do_wp_page() Hugh Dickins pointed out:

  "I now realize it's right to the first order (normal case) and to the
   second order (ptrace poke), but not to the third order (ptrace poke
   anon page here to be COWed - perhaps can't occur without intervening
   mprotects)."

This patch restores the old COW behaviour for anonymous pages.

Signed-off-by: Peter Zijlstra <a.p.zijlstra@chello.nl>
Acked-by: Hugh Dickins <hugh@veritas.com>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
17 years ago[PATCH] mm: small cleanup of install_page()
Peter Zijlstra [Tue, 26 Sep 2006 06:30:59 +0000 (23:30 -0700)]
[PATCH] mm: small cleanup of install_page()

Smallish cleanup to install_page(), could save a memory read (haven't checked
the asm output) and sure looks nicer.

Signed-off-by: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Hugh Dickins <hugh@veritas.com>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
17 years ago[PATCH] mm: optimize the new mprotect() code a bit
Peter Zijlstra [Tue, 26 Sep 2006 06:30:59 +0000 (23:30 -0700)]
[PATCH] mm: optimize the new mprotect() code a bit

mprotect() resets the page protections, which could result in extra write
faults for those pages whose dirty state we track using write faults and are
dirty already.

Signed-off-by: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Hugh Dickins <hugh@veritas.com>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
17 years ago[PATCH] mm: balance dirty pages
Peter Zijlstra [Tue, 26 Sep 2006 06:30:58 +0000 (23:30 -0700)]
[PATCH] mm: balance dirty pages

Now that we can detect writers of shared mappings, throttle them.  Avoids OOM
by surprise.

Signed-off-by: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Hugh Dickins <hugh@veritas.com>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
17 years ago[PATCH] mm: tracking shared dirty pages
Peter Zijlstra [Tue, 26 Sep 2006 06:30:57 +0000 (23:30 -0700)]
[PATCH] mm: tracking shared dirty pages

Tracking of dirty pages in shared writeable mmap()s.

The idea is simple: write protect clean shared writeable pages, catch the
write-fault, make writeable and set dirty.  On page write-back clean all the
PTE dirty bits and write protect them once again.

The implementation is a tad harder, mainly because the default
backing_dev_info capabilities were too loosely maintained.  Hence it is not
enough to test the backing_dev_info for cap_account_dirty.

The current heuristic is as follows, a VMA is eligible when:
 - its shared writeable
    (vm_flags & (VM_WRITE|VM_SHARED)) == (VM_WRITE|VM_SHARED)
 - it is not a 'special' mapping
    (vm_flags & (VM_PFNMAP|VM_INSERTPAGE)) == 0
 - the backing_dev_info is cap_account_dirty
    mapping_cap_account_dirty(vma->vm_file->f_mapping)
 - f_op->mmap() didn't change the default page protection

Page from remap_pfn_range() are explicitly excluded because their COW
semantics are already horrid enough (see vm_normal_page() in do_wp_page()) and
because they don't have a backing store anyway.

mprotect() is taught about the new behaviour as well.  However it overrides
the last condition.

Cleaning the pages on write-back is done with page_mkclean() a new rmap call.
It can be called on any page, but is currently only implemented for mapped
pages, if the page is found the be of a VMA that accounts dirty pages it will
also wrprotect the PTE.

Finally, in fs/buffers.c:try_to_free_buffers(); remove clear_page_dirty() from
under ->private_lock.  This seems to be safe, since ->private_lock is used to
serialize access to the buffers, not the page itself.  This is needed because
clear_page_dirty() will call into page_mkclean() and would thereby violate
locking order.

[dhowells@redhat.com: Provide a page_mkclean() implementation for NOMMU]
Signed-off-by: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Hugh Dickins <hugh@veritas.com>
Signed-off-by: David Howells <dhowells@redhat.com>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
17 years ago[PATCH] mm: VM_BUG_ON
Nick Piggin [Tue, 26 Sep 2006 06:30:55 +0000 (23:30 -0700)]
[PATCH] mm: VM_BUG_ON

Introduce a VM_BUG_ON, which is turned on with CONFIG_DEBUG_VM.  Use this
in the lightweight, inline refcounting functions; PageLRU and PageActive
checks in vmscan, because they're pretty well confined to vmscan.  And in
page allocate/free fastpaths which can be the hottest parts of the kernel
for kbuilds.

Unlike BUG_ON, VM_BUG_ON must not be used to execute statements with
side-effects, and should not be used outside core mm code.

Signed-off-by: Nick Piggin <npiggin@suse.de>
Cc: Hugh Dickins <hugh@veritas.com>
Cc: Christoph Lameter <clameter@engr.sgi.com>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
17 years ago[PATCH] update to the kernel kmap/kunmap API
James Bottomley [Tue, 26 Sep 2006 06:30:55 +0000 (23:30 -0700)]
[PATCH] update to the kernel kmap/kunmap API

Give non-highmem architectures access to the kmap API for the purposes of
overriding (this is what the attached patch does).

The proposal is that we should now require all architectures with coherence
issues to manage data coherence via the kmap/kunmap API.  Thus driver
writers never have to write code like

    kmap(page)
    modify data in page
    flush_kernel_dcache_page(page)
    kunmap(page)

instead, kmap/kunmap will manage the coherence and driver (and filesystem)
writers don't need to worry about how to flush between kmap and kunmap.

For most architectures, the page only needs to be flushed if it was
actually written to *and* there are user mappings of it, so the best
implementation looks to be: clear the page dirty pte bit in the kernel page
tables on kmap and on kunmap, check page->mappings for user maps, and then
the dirty bit, and only flush if it both has user mappings and is dirty.

Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
17 years ago[PATCH] jbd: fix commit of ordered data buffers
Jan Kara [Tue, 26 Sep 2006 06:30:53 +0000 (23:30 -0700)]
[PATCH] jbd: fix commit of ordered data buffers

Original commit code assumes, that when a buffer on BJ_SyncData list is
locked, it is being written to disk.  But this is not true and hence it can
lead to a potential data loss on crash.  Also the code didn't count with
the fact that journal_dirty_data() can steal buffers from committing
transaction and hence could write buffers that no longer belong to the
committing transaction.  Finally it could possibly happen that we tried
writing out one buffer several times.

The patch below tries to solve these problems by a complete rewrite of the
data commit code.  We go through buffers on t_sync_datalist, lock buffers
needing write out and store them in an array.  Buffers are also immediately
refiled to BJ_Locked list or unfiled (if the write out is completed).  When
the array is full or we have to block on buffer lock, we submit all
accumulated buffers for IO.

[suitable for 2.6.18.x around the 2.6.19-rc2 timeframe]

Signed-off-by: Jan Kara <jack@suse.cz>
Cc: Badari Pulavarty <pbadari@us.ibm.com>
Cc: <stable@kernel.org>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
17 years ago[PATCH] trigger a syntax error if percpu macros are incorrectly used
Jan Blunck [Tue, 26 Sep 2006 06:30:53 +0000 (23:30 -0700)]
[PATCH] trigger a syntax error if percpu macros are incorrectly used

get_cpu_var()/per_cpu()/__get_cpu_var() arguments must be simple
identifiers.  Otherwise the arch dependent implementations might break.

This patch enforces the correct usage of the macros by producing a syntax
error if the variable is not a simple identifier.

Signed-off-by: Jan Blunck <jblunck@suse.de>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
17 years ago[PATCH] Fix longstanding load balancing bug in the scheduler
Christoph Lameter [Tue, 26 Sep 2006 06:30:51 +0000 (23:30 -0700)]
[PATCH] Fix longstanding load balancing bug in the scheduler

The scheduler will stop load balancing if the most busy processor contains
processes pinned via processor affinity.

The scheduler currently only does one search for busiest cpu.  If it cannot
pull any tasks away from the busiest cpu because they were pinned then the
scheduler goes into a corner and sulks leaving the idle processors idle.

F.e.  If you have processor 0 busy running four tasks pinned via taskset,
there are none on processor 1 and one just started two processes on
processor 2 then the scheduler will not move one of the two processes away
from processor 2.

This patch fixes that issue by forcing the scheduler to come out of its
corner and retrying the load balancing by considering other processors for
load balancing.

This patch was originally developed by John Hawkes and discussed at

    http://marc.theaimsgroup.com/?l=linux-kernel&m=113901368523205&w=2.

I have removed extraneous material and gone back to equipping struct rq
with the cpu the queue is associated with since this makes the patch much
easier and it is likely that others in the future will have the same
difficulty of figuring out which processor owns which runqueue.

The overhead added through these patches is a single word on the stack if
the kernel is configured to support 32 cpus or less (32 bit).  For 32 bit
environments the maximum number of cpus that can be configued is 255 which
would result in the use of 32 bytes additional on the stack.  On IA64 up to
1k cpus can be configured which will result in the use of 128 additional
bytes on the stack.  The maximum additional cache footprint is one
cacheline.  Typically memory use will be much less than a cacheline and the
additional cpumask will be placed on the stack in a cacheline that already
contains other local variable.

Signed-off-by: Christoph Lameter <clameter@sgi.com>
Cc: John Hawkes <hawkes@sgi.com>
Cc: "Siddha, Suresh B" <suresh.b.siddha@intel.com>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Nick Piggin <nickpiggin@yahoo.com.au>
Cc: Peter Williams <pwil3058@bigpond.net.au>
Cc: <stable@kernel.org>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
17 years ago[PATCH] Don't set calgary iommu as default y
Andi Kleen [Tue, 26 Sep 2006 08:52:42 +0000 (10:52 +0200)]
[PATCH] Don't set calgary iommu as default y

Most systems don't need it.

Signed-off-by: Andi Kleen <ak@suse.de>
17 years ago[PATCH] i386/x86-64: New Intel feature flags
Dave Jones [Tue, 26 Sep 2006 08:52:42 +0000 (10:52 +0200)]
[PATCH] i386/x86-64: New Intel feature flags

Add supplemental SSE3 instructions flag, and Direct Cache Access flag.
As described in "Intel Processor idenfication and the CPUID instruction
AP485 Sept 2006"

AK: also added for x86-64

Signed-off-by: Dave Jones <davej@redhat.com>
Signed-off-by: Andi Kleen <ak@suse.de>
17 years ago[PATCH] x86: Add a cumulative thermal throttle event counter.
Dmitriy Zavin [Tue, 26 Sep 2006 08:52:42 +0000 (10:52 +0200)]
[PATCH] x86: Add a cumulative thermal throttle event counter.

The counter is exported to /sys that keeps track of the
number of thermal events, such that the user knows how bad the
thermal problem might be (since the logging to syslog and mcelog
is rate limited).

AK: Fixed cpu hotplug locking

Signed-off-by: Dmitriy Zavin <dmitriyz@google.com>
Signed-off-by: Andi Kleen <ak@suse.de>
17 years ago[PATCH] i386: Make the jiffies compares use the 64bit safe macros.
Dmitriy Zavin [Tue, 26 Sep 2006 08:52:42 +0000 (10:52 +0200)]
[PATCH] i386: Make the jiffies compares use the 64bit safe macros.

Signed-off-by: Dmitriy Zavin <dmitriyz@google.com>
Signed-off-by: Andi Kleen <ak@suse.de>
17 years ago[PATCH] x86: Refactor thermal throttle processing
Dmitriy Zavin [Tue, 26 Sep 2006 08:52:42 +0000 (10:52 +0200)]
[PATCH] x86: Refactor thermal throttle processing

Refactor the event processing (syslog messaging and rate limiting)
into separate file therm_throt.c. This allows consistent reporting
of CPU thermal throttle events.

After ACK'ing the interrupt, if the event is current, the user
(p4.c/mce_intel.c) calls therm_throt_process to log (and rate limit)
the event. If that function returns 1, the user has the option to log
things further (such as to mce_log in x86_64).

AK: minor cleanup

Signed-off-by: Dmitriy Zavin <dmitriyz@google.com>
Signed-off-by: Andi Kleen <ak@suse.de>
17 years ago[PATCH] Add 64bit jiffies compares (for use with get_jiffies_64)
Dmitriy Zavin [Tue, 26 Sep 2006 08:52:42 +0000 (10:52 +0200)]
[PATCH] Add 64bit jiffies compares (for use with get_jiffies_64)

The current time_before/time_after macros will fail typechecks
when passed u64 values (as returned by get_jiffies_64()). On 64bit
systems, this will just result in a warning about mismatching types
without explicit casts, but since unsigned long and u64
(unsigned long long) are of same size, it will still work.
On 32bit systems, a long is 32bits, so the value from get_jiffies_64()
will be truncated by the cast and thus lose all the precision gained by
64bit jiffies.

Signed-off-by: Dmitriy Zavin <dmitriyz@google.com>
Signed-off-by: Andi Kleen <ak@suse.de>
17 years ago[PATCH] Fix unwinder warning in traps.c
Andi Kleen [Tue, 26 Sep 2006 08:52:42 +0000 (10:52 +0200)]
[PATCH] Fix unwinder warning in traps.c

Fix

linux/arch/x86_64/kernel/traps.c: In function 'dump_trace':
linux/arch/x86_64/kernel/traps.c:275: warning: cast to pointer from integer of different size

with allnoconfig

Cc: jbeulich@novell.com
Signed-off-by: Andi Kleen <ak@suse.de>
17 years ago[PATCH] x86: Allow disabling early pci scans with pci=noearly or disallowing conf1
Andi Kleen [Tue, 26 Sep 2006 08:52:41 +0000 (10:52 +0200)]
[PATCH] x86: Allow disabling early pci scans with pci=noearly or disallowing conf1

Some buggy systems can machine check when config space accesses
happen for some non existent devices.  i386/x86-64 do some early
device scans that might trigger this. Allow pci=noearly to disable
this. Also when type 1 is disabling also don't do any early
accesses which are always type1.

This moves the pci= configuration parsing to be a early parameter.
I don't think this can break anything because it only changes
a single global that is only used by PCI.

Cc: gregkh@suse.de
Cc: Trammell Hudson <hudson@osresearch.net>
Signed-off-by: Andi Kleen <ak@suse.de>
17 years ago[PATCH] x86: Move direct PCI scanning functions out of line
Andi Kleen [Tue, 26 Sep 2006 08:52:41 +0000 (10:52 +0200)]
[PATCH] x86: Move direct PCI scanning functions out of line

Saves about 200 bytes of code space.

Signed-off-by: Andi Kleen <ak@suse.de>
17 years ago[PATCH] i386/x86-64: Make all early PCI scans dependent on CONFIG_PCI
Andi Kleen [Tue, 26 Sep 2006 08:52:41 +0000 (10:52 +0200)]
[PATCH] i386/x86-64: Make all early PCI scans dependent on CONFIG_PCI

This is useful on systems with broken PCI bus. Affects various
scans in x86-64 and i386's early ACPI quirk scan.

Cc: gregkh@suse.de
Cc: len.brown@intel.com
Cc: Trammell Hudson <hudson@osresearch.net>
Signed-off-by: Andi Kleen <ak@suse.de>
17 years ago[PATCH] Don't leak NT bit into next task
Andi Kleen [Tue, 26 Sep 2006 08:52:41 +0000 (10:52 +0200)]
[PATCH] Don't leak NT bit into next task

SYSENTER can cause a NT to be set which might cause crashes on the IRET
in the next task.

Following similar i386 patch from Linus.

Signed-off-by: Andi Kleen <ak@suse.de>
17 years ago[PATCH] i386/x86-64: Work around gcc bug with noreturn functions in unwinder
Jan Beulich [Tue, 26 Sep 2006 08:52:41 +0000 (10:52 +0200)]
[PATCH] i386/x86-64: Work around gcc bug with noreturn functions in unwinder

Current gcc generates calls not jumps to noreturn functions. When that happens the
return address can point to the next function, which confuses the unwinder.

This patch works around it by marking asynchronous exception
frames in contrast normal call frames in the unwind information.  Then teach
the unwinder to decode this.

For normal call frames the unwinder now subtracts one from the address which avoids
this problem.  The standard libgcc unwinder uses the same trick.

It doesn't include adjustment of the printed address (i.e. for the original
example, it'd still be kernel_math_error+0 that gets displayed, but the
unwinder wouldn't get confused anymore.

This only works with binutils 2.6.17+ and some versions of H.J.Lu's 2.6.16
unfortunately because earlier binutils don't support .cfi_signal_frame

[AK: added automatic detection of the new binutils and wrote description]

Signed-off-by: Jan Beulich <jbeulich@novell.com>
Signed-off-by: Andi Kleen <ak@suse.de>
17 years ago[PATCH] Fix some broken white space in ia32_signal.c
Andi Kleen [Tue, 26 Sep 2006 08:52:41 +0000 (10:52 +0200)]
[PATCH] Fix some broken white space in ia32_signal.c

No functional changes
Signed-off-by: Andi Kleen <ak@suse.de>
17 years ago[PATCH] Initialize argument registers for 32bit signal handlers.
Andi Kleen [Tue, 26 Sep 2006 08:52:41 +0000 (10:52 +0200)]
[PATCH] Initialize argument registers for 32bit signal handlers.

In case the user space was compiled with -mregparm=3
Following i386. Pointed out by Albert Cahalan

Signed-off-by: Andi Kleen <ak@suse.de>
17 years ago[PATCH] Remove all traces of signal number conversion
Andi Kleen [Tue, 26 Sep 2006 08:52:41 +0000 (10:52 +0200)]
[PATCH] Remove all traces of signal number conversion

This was old code that was needed for iBCS and x86-64 never supported that.

Pointed out by Albert Cahalan
Signed-off-by: Andi Kleen <ak@suse.de>
17 years ago[PATCH] Don't synchronize time reading on single core AMD systems
Andi Kleen [Tue, 26 Sep 2006 08:52:41 +0000 (10:52 +0200)]
[PATCH] Don't synchronize time reading on single core AMD systems

We do some additional CPU synchronization in gettimeofday et.al. to make
sure the time stamps are always monotonic over multiple CPUs. But on
single core systems that is not needed. So don't do it.

Signed-off-by: Andi Kleen <ak@suse.de>
17 years ago[PATCH] Remove outdated comment in x86-64 mmconfig code
Andi Kleen [Tue, 26 Sep 2006 08:52:41 +0000 (10:52 +0200)]
[PATCH] Remove outdated comment in x86-64 mmconfig code

Cc: gregkh@suse.de
Signed-off-by: Andi Kleen <ak@suse.de>
17 years ago[PATCH] Use string instructions for Core2 copy/clear
Andi Kleen [Tue, 26 Sep 2006 08:52:41 +0000 (10:52 +0200)]
[PATCH] Use string instructions for Core2 copy/clear

It is faster than using a unrolled loop for the use cases the kernel
cares about (cached, sizes typically < 4K)

Signed-off-by: Andi Kleen <ak@suse.de>
17 years ago[PATCH] x86: - restore i8259A eoi status on resume
Matthew Garrett [Tue, 26 Sep 2006 08:52:41 +0000 (10:52 +0200)]
[PATCH] x86: - restore i8259A eoi status on resume

Got it. i8259A_resume calls init_8259A(0) unconditionally, even if
auto_eoi has been set. Keep track of the current status and restore that
on resume. This fixes it for AMD64 and i386.

Signed-off-by: Matthew Garrett <mjg59@srcf.ucam.org>
Signed-off-by: Andi Kleen <ak@suse.de>
17 years ago[PATCH] i386: Split multi-line printk in oops output.
Dave Jones [Tue, 26 Sep 2006 08:52:41 +0000 (10:52 +0200)]
[PATCH] i386: Split multi-line printk in oops output.

Sometimes, bug reports come in where we've had an oops, and the
only record we have is what the reporter saw on screen shortly
before the system locked up completely.  Unfortunatly, syslog
only prints lines beginning with KERN_EMERG to the console, so
some lines get lost.
An example of this can be seen at https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=203723

Some of this information isn't vital to diagnosis, but some parts
are useful, such as the tainted flag.

Signed-off-by: Dave Jones <davej@redhat.com>
Signed-off-by: Andi Kleen <ak@suse.de>
17 years ago[PATCH] i386: Fix pack_descriptor()
Jeremy Fitzhardinge [Tue, 26 Sep 2006 08:52:40 +0000 (10:52 +0200)]
[PATCH] i386: Fix pack_descriptor()

Fix pack_descriptor:
 1. flags are bits 20-23 in the high word
 2. limit's 4 msb are bits 16-19 in the high word

These haven't mattered so far, because all users have had small limits
and a flags setting of 0.

Signed-off-by: Jeremy Fitzhardinge <jeremy@xensource.com>
Signed-off-by: Andi Kleen <ak@suse.de>
===================================================================

17 years ago[PATCH] i386: Add MMCFG resources to i386 too
Andi Kleen [Tue, 26 Sep 2006 08:52:40 +0000 (10:52 +0200)]
[PATCH] i386: Add MMCFG resources to i386 too

Following earlier x86-64 patch

Cc: gregkh@suse.de
Signed-off-by: Andi Kleen <ak@suse.de>
17 years ago[PATCH] MMCONFIG and new Intel motherboards
Aaron Durbin [Tue, 26 Sep 2006 08:52:40 +0000 (10:52 +0200)]
[PATCH] MMCONFIG and new Intel motherboards

On Sat, Sep 09, 2006 at 04:14:29PM +0200, Andi Kleen wrote:
> [patch] Looks reasonable, but probably not for 2.6.18 because this stuff
> is already too fragile and it is probably too risky to do any big changes now
> since not enough testing time is left. Can you please resubmit
> it with proper description and signed-off-by line? I can queue it for .19 then
>
> -Andi

Patch inserts PCI memory mapped config region(s) into the resource map.  This
will allow for the MMCCONFIG regions to be marked as busy in the iomem
address space as well as the regions(s) showing up in /proc/iomem.

Signed-off-by: Aaron Durbin <adurbin@google.com>
Signed-off-by: Andi Kleen <ak@suse.de>
17 years ago[PATCH] Insert GART region into resource map
Aaron Durbin [Tue, 26 Sep 2006 08:52:40 +0000 (10:52 +0200)]
[PATCH] Insert GART region into resource map

Patch inserts the GART region into the iomem resource map. The GART will then
be visible within /proc/iomem. It will also allow for other users
utilizing the GART to subreserve the region (agp or IOMMU).

Signed-off-by: Aaron Durbin <adurbin@google.com>
Signed-off-by: Andi Kleen <ak@suse.de>
17 years ago[PATCH] i386/x86-64: Only do MCFG e820 check when type 1 works
Andi Kleen [Tue, 26 Sep 2006 08:52:40 +0000 (10:52 +0200)]
[PATCH] i386/x86-64: Only do MCFG e820 check when type 1 works

Needs earlier patch to split type 1 probing from use.

This patch should fix the x86 macs where type 1 PCI config space access
doesn't work, but MCFG does. They also don't have a usable e820 table
so the e820 sanity check failed.

Instead assume now that if type 1 doesn't work then MCFG must work
and don't do the e820 check.

Signed-off-by: Andi Kleen <ak@suse.de>
17 years ago[PATCH] i386/x86-64: PCI: split probing and initialization of type 1 config space...
Andi Kleen [Tue, 26 Sep 2006 08:52:40 +0000 (10:52 +0200)]
[PATCH] i386/x86-64: PCI: split probing and initialization of type 1 config space access

First probe if type1/2 accesses work, but then only initialize them at the end.

This is useful for a later patch that needs this information inbetween.

Signed-off-by: Andi Kleen <ak@suse.de>
17 years ago[PATCH] Fix idle notifiers
Andi Kleen [Tue, 26 Sep 2006 08:52:40 +0000 (10:52 +0200)]
[PATCH] Fix idle notifiers

Previously exit_idle would be called more often than enter_idle

Now instead of using complicated tests just keep track of it
using the per CPU variable as a flip flop.  I moved the idle state into the
PDA to make the access more efficient.

Original bug report and an initial patch from Stephane Eranian,
but redone by AK.

Cc: Stephane Eranian <eranian@hpl.hp.com>
Signed-off-by: Andi Kleen <ak@suse.de>
17 years ago[PATCH] Remove experimental mark of kexec
Eric W. Biederman [Tue, 26 Sep 2006 08:52:40 +0000 (10:52 +0200)]
[PATCH] Remove experimental mark of kexec

kexec has been marked experimental for a year now and all
of the serious problems have been worked through.  So it
is time (if not past time) to remove the experimental mark.

Signed-off-by: Eric W. Biederman <ebiederm@xmission.com>
Signed-off-by: Andi Kleen <ak@suse.de>
17 years ago[PATCH] i386: Remove experimental mark of kexec
Eric W. Biederman [Tue, 26 Sep 2006 08:52:40 +0000 (10:52 +0200)]
[PATCH] i386: Remove experimental mark of kexec

kexec has been marked experimental for a year now and all
of the serious problems have been worked through.  So it
is time (if not past time) to remove the experimental mark.

Signed-off-by: Eric W. Biederman <ebiederm@xmission.com>
Signed-off-by: Andi Kleen <ak@suse.de>
17 years ago[PATCH] Mark per cpu data initialization __initdata again
Andi Kleen [Tue, 26 Sep 2006 08:52:40 +0000 (10:52 +0200)]
[PATCH] Mark per cpu data initialization __initdata again

Before 2.6.16 this was changed to work around code that accessed
CPUs not in the possible map. But that code should be all fixed now,
so mark it __initdata again.
Signed-off-by: Andi Kleen <ak@suse.de>
17 years ago[PATCH] Remove unused asm-x86_64/mmx.h
Andi Kleen [Tue, 26 Sep 2006 08:52:40 +0000 (10:52 +0200)]
[PATCH] Remove unused asm-x86_64/mmx.h

Signed-off-by: Andi Kleen <ak@suse.de>
17 years ago[PATCH] Define __bad_pda_field as noreturn
Andi Kleen [Tue, 26 Sep 2006 08:52:40 +0000 (10:52 +0200)]
[PATCH] Define __bad_pda_field as noreturn

This quietens so warnings about uninitialized use of the return
value of the pda read operations.
Signed-off-by: Andi Kleen <ak@suse.de>
17 years ago[PATCH] Reindent macros in pda.h
Andi Kleen [Tue, 26 Sep 2006 08:52:40 +0000 (10:52 +0200)]
[PATCH] Reindent macros in pda.h

Reindent the macros in x86-64 pda.h, making them much more readable.
Follows Jeremy's i386 version of this.

No functional changes

Signed-off-by: Andi Kleen <ak@suse.de>
17 years ago[PATCH] Fix some stylistic issues in uaccess.h
Andi Kleen [Tue, 26 Sep 2006 08:52:40 +0000 (10:52 +0200)]
[PATCH] Fix some stylistic issues in uaccess.h

- Replace some broken white space.
- Replace __ keywords with standard names

No functional changes.

Signed-off-by: Andi Kleen <ak@suse.de>
17 years ago[PATCH] Check return values of __copy_to_user in uname emulation
Andi Kleen [Tue, 26 Sep 2006 08:52:40 +0000 (10:52 +0200)]
[PATCH] Check return values of __copy_to_user in uname emulation

Quietens some new warnings
Signed-off-by: Andi Kleen <ak@suse.de>
17 years ago[PATCH] Check return value of copy_to_user in compat_sys_pselect7
Andi Kleen [Tue, 26 Sep 2006 08:52:39 +0000 (10:52 +0200)]
[PATCH] Check return value of copy_to_user in compat_sys_pselect7

Fix

linux/fs/compat.c: In function compat_sys_pselect7
linux/fs/compat.c:1869: warning: ignoring return value of copy_to_user, declared with attribute warn_unused_result

To make it easier to handle I changed to semantics to not try to
write out a timespec if an error occurred. I hope that's ok.

Cc: dwmw2@infradead.org
Signed-off-by: Andi Kleen <ak@suse.de>
17 years ago[PATCH] Add __must_check to copy_*_user
Andi Kleen [Tue, 26 Sep 2006 08:52:39 +0000 (10:52 +0200)]
[PATCH] Add __must_check to copy_*_user

Following i386.

And also fix the two occurrences that caused warnings in arch/x86_64/*

Signed-off-by: Andi Kleen <ak@suse.de>
17 years ago[PATCH] Fix zeroing on exception in copy_*_user
Andi Kleen [Tue, 26 Sep 2006 08:52:39 +0000 (10:52 +0200)]
[PATCH] Fix zeroing on exception in copy_*_user

- Don't zero for __copy_from_user_inatomic following i386.
This will prevent spurious zeros for parallel file system writers when
one does a exception
- The string instruction version didn't zero the output on
exception. Oops.

Also I cleaned up the code a bit while I was at it and added a minor
optimization to the string instruction path.

Signed-off-by: Andi Kleen <ak@suse.de>