pandora-kernel.git
10 years agoBtrfs: use the commit_root for reading free_space_inode crcs
Chris Mason [Tue, 26 Jul 2011 19:35:09 +0000 (15:35 -0400)]
Btrfs: use the commit_root for reading free_space_inode crcs

Now that we are using regular file crcs for the free space cache,
we can deadlock if we try to read the free_space_inode while we are
updating the crc tree.

This commit fixes things by using the commit_root to read the crcs.  This is
safe because we the free space cache file would already be loaded if
that block group had been changed in the current transaction.

Signed-off-by: Chris Mason <chris.mason@oracle.com>
10 years agoBtrfs: reduce extent_state lock contention for metadata
Chris Mason [Mon, 25 Jul 2011 10:50:50 +0000 (06:50 -0400)]
Btrfs: reduce extent_state lock contention for metadata

For metadata buffers that don't straddle pages (all of them), btrfs
can safely use the page uptodate bits and extent_buffer uptodate bit
instead of needing to use the extent_state tree.

This greatly reduces contention on the state tree lock.

Signed-off-by: Chris Mason <chris.mason@oracle.com>
10 years agoBtrfs: remove lockdep magic from btrfs_next_leaf
Chris Mason [Tue, 26 Jul 2011 20:01:59 +0000 (16:01 -0400)]
Btrfs: remove lockdep magic from btrfs_next_leaf

Before the reader/writer locks, btrfs_next_leaf needed to keep
the path blocking to avoid making lockdep upset.

Now that btrfs_next_leaf only takes read locks, this isn't required.

Signed-off-by: Chris Mason <chris.mason@oracle.com>
10 years agoBtrfs: make a lockdep class for each root
Chris Mason [Tue, 26 Jul 2011 20:11:19 +0000 (16:11 -0400)]
Btrfs: make a lockdep class for each root

This patch was originally from Tejun Heo.  lockdep complains about the btrfs
locking because we sometimes take btree locks from two different trees at the
same time.  The current classes are based only on level in the btree, which
isn't enough information for lockdep to figure out if the lock is safe.

This patch makes a class for each type of tree, and lumps all the FS trees that
actually have files and directories into the same class.

Signed-off-by: Chris Mason <chris.mason@oracle.com>
10 years agoBtrfs: switch the btrfs tree locks to reader/writer
Chris Mason [Sat, 16 Jul 2011 19:23:14 +0000 (15:23 -0400)]
Btrfs: switch the btrfs tree locks to reader/writer

The btrfs metadata btree is the source of significant
lock contention, especially in the root node.   This
commit changes our locking to use a reader/writer
lock.

The lock is built on top of rw spinlocks, and it
extends the lock tracking to remember if we have a
read lock or a write lock when we go to blocking.  Atomics
count the number of blocking readers or writers at any
given time.

It removes all of the adaptive spinning from the old code
and uses only the spinning/blocking hints inside of btrfs
to decide when it should continue spinning.

In read heavy workloads this is dramatically faster.  In write
heavy workloads we're still faster because of less contention
on the root node lock.

We suffer slightly in dbench because we schedule more often
during write locks, but all other benchmarks so far are improved.

Signed-off-by: Chris Mason <chris.mason@oracle.com>
10 years agoBtrfs: fix deadlock when throttling transactions
Josef Bacik [Sun, 24 Jul 2011 19:45:34 +0000 (15:45 -0400)]
Btrfs: fix deadlock when throttling transactions

Hit this nice little deadlock.  What happens is this

__btrfs_end_transaction with throttle set, --use_count so it equals 0
  btrfs_commit_transaction
    <somebody else actually manages to start the commit>
    btrfs_end_transaction --use_count so now its -1 <== BAD
      we just return and wait on the transaction

This is bad because we just return after our use_count is -1 and don't let go
of our num_writer count on the transaction, so the guy committing the
transaction just sits there forever.  Fix this by inc'ing our use_count if we're
going to call commit_transaction so that if we call btrfs_end_transaction it's
valid.  Thanks,

Signed-off-by: Josef Bacik <josef@redhat.com>
Signed-off-by: Chris Mason <chris.mason@oracle.com>
10 years agoBtrfs: stop using highmem for extent_buffers
Chris Mason [Tue, 19 Jul 2011 16:04:14 +0000 (12:04 -0400)]
Btrfs: stop using highmem for extent_buffers

The extent_buffers have a very complex interface where
we use HIGHMEM for metadata and try to cache a kmap mapping
to access the memory.

The next commit adds reader/writer locks, and concurrent use
of this kmap cache would make it even more complex.

This commit drops the ability to use HIGHMEM with extent buffers,
and rips out all of the related code.

Signed-off-by: Chris Mason <chris.mason@oracle.com>
10 years agoBtrfs: fix BUG_ON() caused by ENOSPC when relocating space
Miao Xie [Fri, 15 Jul 2011 10:34:36 +0000 (10:34 +0000)]
Btrfs: fix BUG_ON() caused by ENOSPC when relocating space

When we balanced the chunks across the devices, BUG_ON() in
__finish_chunk_alloc() was triggered.

------------[ cut here ]------------
kernel BUG at fs/btrfs/volumes.c:2568!
[SNIP]
Call Trace:
 [<ffffffffa049525e>] btrfs_alloc_chunk+0x8e/0xa0 [btrfs]
 [<ffffffffa04546b0>] do_chunk_alloc+0x330/0x3a0 [btrfs]
 [<ffffffffa045c654>] btrfs_reserve_extent+0xb4/0x1f0 [btrfs]
 [<ffffffffa045c86b>] btrfs_alloc_free_block+0xdb/0x350 [btrfs]
 [<ffffffffa048a8d8>] ? read_extent_buffer+0xd8/0x1d0 [btrfs]
 [<ffffffffa04476fd>] __btrfs_cow_block+0x14d/0x5e0 [btrfs]
 [<ffffffffa044660d>] ? read_block_for_search+0x14d/0x4d0 [btrfs]
 [<ffffffffa0447c9b>] btrfs_cow_block+0x10b/0x240 [btrfs]
 [<ffffffffa044dd5e>] btrfs_search_slot+0x49e/0x7a0 [btrfs]
 [<ffffffffa044f07d>] btrfs_insert_empty_items+0x8d/0xf0 [btrfs]
 [<ffffffffa045e973>] insert_with_overflow+0x43/0x110 [btrfs]
 [<ffffffffa045eb0d>] btrfs_insert_dir_item+0xcd/0x1f0 [btrfs]
 [<ffffffffa0489bd0>] ? map_extent_buffer+0xb0/0xc0 [btrfs]
 [<ffffffff812276ad>] ? rb_insert_color+0x9d/0x160
 [<ffffffffa046cc40>] ? inode_tree_add+0xf0/0x150 [btrfs]
 [<ffffffffa0474801>] btrfs_add_link+0xc1/0x1c0 [btrfs]
 [<ffffffff811dacac>] ? security_inode_init_security+0x1c/0x30
 [<ffffffffa04a28aa>] ? btrfs_init_acl+0x4a/0x180 [btrfs]
 [<ffffffffa047492f>] btrfs_add_nondir+0x2f/0x70 [btrfs]
 [<ffffffffa046af16>] ? btrfs_init_inode_security+0x46/0x60 [btrfs]
 [<ffffffffa0474ac0>] btrfs_create+0x150/0x1d0 [btrfs]
 [<ffffffff81159c63>] ? generic_permission+0x23/0xb0
 [<ffffffff8115b415>] vfs_create+0xa5/0xc0
 [<ffffffff8115ce6e>] do_last+0x5fe/0x880
 [<ffffffff8115dc0d>] path_openat+0xcd/0x3d0
 [<ffffffff8115e029>] do_filp_open+0x49/0xa0
 [<ffffffff8116a965>] ? alloc_fd+0x95/0x160
 [<ffffffff8114f0c7>] do_sys_open+0x107/0x1e0
 [<ffffffff810bcc3f>] ? audit_syscall_entry+0x1bf/0x1f0
 [<ffffffff8114f1e0>] sys_open+0x20/0x30
 [<ffffffff81484ec2>] system_call_fastpath+0x16/0x1b
[SNIP]
RIP  [<ffffffffa049444a>] __finish_chunk_alloc+0x20a/0x220 [btrfs]

The reason is:
Task1 Space balance task
do_chunk_alloc()
  __finish_chunk_alloc()
    update device info
    in the chunk tree
      alloc system metadata block
relocate system metadata block group
  set system metadata block group
  readonly, This block group is the
  only one that can allocate space. So
  there is no free space that can be
  allocated now.
        find no space and don't try
        to alloc new chunk, and then
        return ENOSPC
  BUG_ON() in __finish_chunk_alloc()
  was triggered.

Fix this bug by allocating a new system metadata chunk before relocating the
old one if we find there is no free space which can be allocated after setting
the old block group to be read-only.

Reported-by: Tsutomu Itoh <t-itoh@jp.fujitsu.com>
Signed-off-by: Miao Xie <miaox@cn.fujitsu.com>
Tested-by: Tsutomu Itoh <t-itoh@jp.fujitsu.com>
Signed-off-by: Chris Mason <chris.mason@oracle.com>
10 years agoBtrfs: tag pages for writeback in sync
Josef Bacik [Fri, 15 Jul 2011 21:26:38 +0000 (21:26 +0000)]
Btrfs: tag pages for writeback in sync

Everybody else does this, we need to do it too.  If we're syncing, we need to
tag the pages we're going to write for writeback so we don't end up writing the
same stuff over and over again if somebody is constantly redirtying our file.
This will keep us from having latencies with heavy sync workloads.  Thanks,

Signed-off-by: Josef Bacik <josef@redhat.com>
Signed-off-by: Chris Mason <chris.mason@oracle.com>
10 years agoBtrfs: fix enospc problems with delalloc
Josef Bacik [Fri, 15 Jul 2011 15:16:44 +0000 (15:16 +0000)]
Btrfs: fix enospc problems with delalloc

So I had this brilliant idea to use atomic counters for outstanding and reserved
extents, but this turned out to be a bad idea.  Consider this where we have 1
outstanding extent and 1 reserved extent

Reserver Releaser
atomic_dec(outstanding) now 0
atomic_read(outstanding)+1 get 1
atomic_read(reserved) get 1
don't actually reserve anything because
they are the same
atomic_cmpxchg(reserved, 1, 0)
atomic_inc(outstanding)
atomic_add(0, reserved)
free reserved space for 1 extent

Then the reserver now has no actual space reserved for it, and when it goes to
finish the ordered IO it won't have enough space to do it's allocation and you
get those lovely warnings.

Signed-off-by: Josef Bacik <josef@redhat.com>
Signed-off-by: Chris Mason <chris.mason@oracle.com>
10 years agoBtrfs: don't flush delalloc arbitrarily
Josef Bacik [Fri, 15 Jul 2011 16:01:03 +0000 (16:01 +0000)]
Btrfs: don't flush delalloc arbitrarily

Kill the check to see if we have 512mb of reserved space in delalloc and
shrink_delalloc if we do.  This causes unexpected latencies and we have other
logic to see if we need to throttle.  Thanks,

Signed-off-by: Josef Bacik <josef@redhat.com>
Signed-off-by: Chris Mason <chris.mason@oracle.com>
10 years agoBtrfs: use find_or_create_page instead of grab_cache_page
Josef Bacik [Mon, 11 Jul 2011 14:47:06 +0000 (10:47 -0400)]
Btrfs: use find_or_create_page instead of grab_cache_page

grab_cache_page will use mapping_gfp_mask(), which for all inodes is set to
GFP_HIGHUSER_MOVABLE.  So instead use find_or_create_page in all cases where we
need GFP_NOFS so we don't deadlock.  Thanks,

Signed-off-by: Josef Bacik <josef@redhat.com>
10 years agoBtrfs: use a worker thread to do caching
Josef Bacik [Thu, 30 Jun 2011 18:42:28 +0000 (14:42 -0400)]
Btrfs: use a worker thread to do caching

A user reported a deadlock when copying a bunch of files.  This is because they
were low on memory and kthreadd got hung up trying to migrate pages for an
allocation when starting the caching kthread.  The page was locked by the person
starting the caching kthread.  To fix this we just need to use the async thread
stuff so that the threads are already created and we don't have to worry about
deadlocks.  Thanks,

Reported-by: Roman Mamedov <rm@romanrm.ru>
Signed-off-by: Josef Bacik <josef@redhat.com>
10 years agoBtrfs: fix how we merge extent states and deal with cached states
Josef Bacik [Mon, 20 Jun 2011 18:53:48 +0000 (14:53 -0400)]
Btrfs: fix how we merge extent states and deal with cached states

First, we can sometimes free the state we're merging, which means anybody who
calls merge_state() may have the state it passed in free'ed.  This is
problematic because we could end up caching the state, which makes caching
useless as the state will no longer be part of the tree.  So instead of free'ing
the state we passed into merge_state(), set it's end to the other->end and free
the other state.  This way we are sure to cache the correct state.  Also because
we can merge states together, instead of only using the cache'd state if it's
start == the start we are looking for, go ahead and use it if the start we are
looking for is within the range of the cached state.  Thanks,

Signed-off-by: Josef Bacik <josef@redhat.com>
10 years agoBtrfs: use the normal checksumming infrastructure for free space cache
Josef Bacik [Fri, 10 Jun 2011 19:31:13 +0000 (15:31 -0400)]
Btrfs: use the normal checksumming infrastructure for free space cache

We used to store the checksums of the space cache directly in the space cache,
however that doesn't work out too well if we have more space than we can fit the
checksums into the first page.  So instead use the normal checksumming
infrastructure.  There were problems with doing this originally but those
problems don't exist now so this works out fine.  Thanks,

Signed-off-by: Josef Bacik <josef@redhat.com>
10 years agoBtrfs: serialize flushers in reserve_metadata_bytes
Josef Bacik [Tue, 7 Jun 2011 20:07:44 +0000 (16:07 -0400)]
Btrfs: serialize flushers in reserve_metadata_bytes

We keep having problems with early enospc, and that's because our method of
making space is inherently racy.  The problem is we can have one guy trying to
make space for himself, and in the meantime people come in and steal his
reservation.  In order to stop this we make a waitqueue and put anybody who
comes into reserve_metadata_bytes on that waitqueue if somebody is trying to
make more space.  Thanks,

Signed-off-by: Josef Bacik <josef@redhat.com>
10 years agoBtrfs: do transaction space reservation before joining the transaction
Josef Bacik [Tue, 7 Jun 2011 19:07:51 +0000 (15:07 -0400)]
Btrfs: do transaction space reservation before joining the transaction

We have to do weird things when handling enospc in the transaction joining code.
Because we've already joined the transaction we cannot commit the transaction
within the reservation code since it will deadlock, so we have to return EAGAIN
and then make sure we don't retry too many times.  Instead of doing this, just
do the reservation the normal way before we join the transaction, that way we
can do whatever we want to try and reclaim space, and then if it fails we know
for sure we are out of space and we can return ENOSPC.  Thanks,

Signed-off-by: Josef Bacik <josef@redhat.com>
10 years agoBtrfs: try to only do one btrfs_search_slot in do_setxattr
Josef Bacik [Fri, 27 May 2011 16:06:11 +0000 (12:06 -0400)]
Btrfs: try to only do one btrfs_search_slot in do_setxattr

I've been watching how many btrfs_search_slot()'s we do and I noticed that when
we create a file with selinux enabled we were doing 2 each time we initialize
the security context.  That's because we lookup the xattr first so we can delete
it if we're setting a new value to an existing xattr.  But in the create case we
don't have any xattrs, so it is completely useless to have the extra lookup.  So
re-arrange things so that we only lookup first if we specifically have
XATTR_REPLACE.  That way in the basic case we only do 1 search, and in the more
complicated case we do the normal 2 lookups.  Thanks,

Signed-off-by: Josef Bacik <josef@redhat.com>
11 years agobtrfs: fix oops when doing space balance
Miao Xie [Wed, 6 Jul 2011 22:51:53 +0000 (18:51 -0400)]
btrfs: fix oops when doing space balance

We need to make sure the data relocation inode doesn't go through
the delayed metadata updates, otherwise we get an oops during balance:

kernel BUG at fs/btrfs/relocation.c:4303!
[SNIP]
Call Trace:
 [<ffffffffa03143fd>] ? update_ref_for_cow+0x22d/0x330 [btrfs]
 [<ffffffffa0314951>] __btrfs_cow_block+0x451/0x5e0 [btrfs]
 [<ffffffffa031355d>] ? read_block_for_search+0x14d/0x4d0 [btrfs]
 [<ffffffffa0314beb>] btrfs_cow_block+0x10b/0x240 [btrfs]
 [<ffffffffa031acae>] btrfs_search_slot+0x49e/0x7a0 [btrfs]
 [<ffffffffa032d8af>] btrfs_lookup_inode+0x2f/0xa0 [btrfs]
 [<ffffffff8147bf0e>] ? mutex_lock+0x1e/0x50
 [<ffffffffa0380cf1>] btrfs_update_delayed_inode+0x71/0x160 [btrfs]
 [<ffffffffa037ff27>] ? __btrfs_release_delayed_node+0x67/0x190 [btrfs]
 [<ffffffffa0381cf8>] btrfs_run_delayed_items+0xe8/0x120 [btrfs]
 [<ffffffffa03365e0>] btrfs_commit_transaction+0x250/0x850 [btrfs]
 [<ffffffff810f91d9>] ? find_get_pages+0x39/0x130
 [<ffffffffa0336cd5>] ? join_transaction+0x25/0x250 [btrfs]
 [<ffffffff81081de0>] ? wake_up_bit+0x40/0x40
 [<ffffffffa03785fa>] prepare_to_relocate+0xda/0xf0 [btrfs]
 [<ffffffffa037f2bb>] relocate_block_group+0x4b/0x620 [btrfs]
 [<ffffffffa0334cf5>] ? btrfs_clean_old_snapshots+0x35/0x150 [btrfs]
 [<ffffffffa037fa43>] btrfs_relocate_block_group+0x1b3/0x2e0 [btrfs]
 [<ffffffffa0368ec0>] ? btrfs_tree_unlock+0x50/0x50 [btrfs]
 [<ffffffffa035e39b>] btrfs_relocate_chunk+0x8b/0x670 [btrfs]
 [<ffffffffa031303d>] ? btrfs_set_path_blocking+0x3d/0x50 [btrfs]
 [<ffffffffa03577d8>] ? read_extent_buffer+0xd8/0x1d0 [btrfs]
 [<ffffffffa031bea1>] ? btrfs_previous_item+0xb1/0x150 [btrfs]
 [<ffffffffa03577d8>] ? read_extent_buffer+0xd8/0x1d0 [btrfs]
 [<ffffffffa035f5aa>] btrfs_balance+0x21a/0x2b0 [btrfs]
 [<ffffffffa0368898>] btrfs_ioctl+0x798/0xd20 [btrfs]
 [<ffffffff8111e358>] ? handle_mm_fault+0x148/0x270
 [<ffffffff814809e8>] ? do_page_fault+0x1d8/0x4b0
 [<ffffffff81160d6a>] do_vfs_ioctl+0x9a/0x540
 [<ffffffff811612b1>] sys_ioctl+0xa1/0xb0
 [<ffffffff81484ec2>] system_call_fastpath+0x16/0x1b
[SNIP]
RIP  [<ffffffffa037c1cc>] btrfs_reloc_cow_block+0x22c/0x270 [btrfs]

Signed-off-by: Miao Xie <miaox@cn.fujitsu.com>
Signed-off-by: Chris Mason <chris.mason@oracle.com>
11 years agoBtrfs: don't panic if we get an error while balancing V2
Josef Bacik [Sat, 2 Jul 2011 21:24:41 +0000 (21:24 +0000)]
Btrfs: don't panic if we get an error while balancing V2

A user reported an error where if we try to balance an fs after a device has
been removed it will blow up.  This is because we get an EIO back and this is
where BUG_ON(ret) bites us in the ass.  To fix we just exit.  Thanks,

Reported-by: Anand Jain <Anand.Jain@oracle.com>
Signed-off-by: Josef Bacik <josef@redhat.com>
Signed-off-by: Chris Mason <chris.mason@oracle.com>
11 years agobtrfs: add missing options displayed in mount output
David Sterba [Tue, 28 Jun 2011 15:10:37 +0000 (15:10 +0000)]
btrfs: add missing options displayed in mount output

There are three missed mount options settable by user which are not
currently displayed in mount output.

Signed-off-by: David Sterba <dsterba@suse.cz>
Signed-off-by: Chris Mason <chris.mason@oracle.com>
11 years agobtrfs: fix inconsonant inode information
Miao Xie [Thu, 23 Jun 2011 07:27:13 +0000 (07:27 +0000)]
btrfs: fix inconsonant inode information

When iputting the inode, We may leave the delayed nodes if they have some
delayed items that have not been dealt with. So when the inode is read again,
we must look up the relative delayed node, and use the information in it to
initialize the inode. Or we will get inconsonant inode information, it may
cause that the same directory index number is allocated again, and hit the
following oops:

[ 5447.554187] err add delayed dir index item(name: pglog_0.965_0) into the
insertion tree of the delayed node(root id: 262, inode id: 258, errno: -17)
[ 5447.569766] ------------[ cut here ]------------
[ 5447.575361] kernel BUG at fs/btrfs/delayed-inode.c:1301!
[SNIP]
[ 5447.790721] Call Trace:
[ 5447.793191]  [<ffffffffa0641c4e>] btrfs_insert_dir_item+0x189/0x1bb [btrfs]
[ 5447.800156]  [<ffffffffa0651a45>] btrfs_add_link+0x12b/0x191 [btrfs]
[ 5447.806517]  [<ffffffffa0651adc>] btrfs_add_nondir+0x31/0x58 [btrfs]
[ 5447.812876]  [<ffffffffa0651d6a>] btrfs_create+0xf9/0x197 [btrfs]
[ 5447.818961]  [<ffffffff8111f840>] vfs_create+0x72/0x92
[ 5447.824090]  [<ffffffff8111fa8c>] do_last+0x22c/0x40b
[ 5447.829133]  [<ffffffff8112076a>] path_openat+0xc0/0x2ef
[ 5447.834438]  [<ffffffff810c58e2>] ? __perf_event_task_sched_out+0x24/0x44
[ 5447.841216]  [<ffffffff8103ecdd>] ? perf_event_task_sched_out+0x59/0x67
[ 5447.847846]  [<ffffffff81121a79>] do_filp_open+0x3d/0x87
[ 5447.853156]  [<ffffffff811e126c>] ? strncpy_from_user+0x43/0x4d
[ 5447.859072]  [<ffffffff8111f1f5>] ? getname_flags+0x2e/0x80
[ 5447.864636]  [<ffffffff8111f179>] ? do_getname+0x14b/0x173
[ 5447.870112]  [<ffffffff8111f1b7>] ? audit_getname+0x16/0x26
[ 5447.875682]  [<ffffffff8112b1ab>] ? spin_lock+0xe/0x10
[ 5447.880882]  [<ffffffff81112d39>] do_sys_open+0x69/0xae
[ 5447.886153]  [<ffffffff81112db1>] sys_open+0x20/0x22
[ 5447.891114]  [<ffffffff813b9aab>] system_call_fastpath+0x16/0x1b

Fix it by reusing the old delayed node.

Reported-by: Jim Schutt <jaschut@sandia.gov>
Signed-off-by: Miao Xie <miaox@cn.fujitsu.com>
Tested-by: Jim Schutt <jaschut@sandia.gov>
Signed-off-by: Chris Mason <chris.mason@oracle.com>
11 years agoBtrfs: make sure to update total_bitmaps when freeing cache V3
Josef Bacik [Fri, 24 Jun 2011 16:02:51 +0000 (16:02 +0000)]
Btrfs: make sure to update total_bitmaps when freeing cache V3

A user reported this bug again where we have more bitmaps than we are supposed
to.  This is because we failed to load the free space cache, but don't update
the ctl->total_bitmaps counter when we remove entries from the tree.  This patch
fixes this problem and we should be good to go again.  Thanks,

Signed-off-by: Josef Bacik <josef@redhat.com>
Signed-off-by: Chris Mason <chris.mason@oracle.com>
11 years agoBtrfs: fix type mismatch in find_free_extent()
Ilya Dryomov [Sat, 18 Jun 2011 20:26:38 +0000 (20:26 +0000)]
Btrfs: fix type mismatch in find_free_extent()

data parameter should be u64 because a full-sized chunk flags field is
passed instead of 0/1 for distinguishing data from metadata.  All
underlying functions expect u64.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
Signed-off-by: Chris Mason <chris.mason@oracle.com>
11 years agoBtrfs: make sure to record the transid in new inodes
Chris Mason [Fri, 24 Jun 2011 17:13:29 +0000 (13:13 -0400)]
Btrfs: make sure to record the transid in new inodes

When we create a new inode, we aren't filling in the
field that records the transaction that last changed this
inode.

If we then go to fsync that inode, it will be skipped because the field
isn't filled in.

Signed-off-by: Chris Mason <chris.mason@oracle.com>
11 years agoBtrfs: avoid delayed metadata items during commits
Chris Mason [Fri, 17 Jun 2011 20:14:09 +0000 (16:14 -0400)]
Btrfs: avoid delayed metadata items during commits

Snapshot creation has two phases.  One is the initial snapshot setup,
and the second is done during commit, while nobody is allowed to modify
the root we are snapshotting.

The delayed metadata insertion code can break that rule, it does a
delayed inode update on the inode of the parent of the snapshot,
and delayed directory item insertion.

This makes sure to run the pending delayed operations before we
record the snapshot root, which avoids corruptions.

Signed-off-by: Chris Mason <chris.mason@oracle.com>
11 years agobtrfs: fix uninitialized return value
David Sterba [Mon, 13 Jun 2011 15:18:23 +0000 (15:18 +0000)]
btrfs: fix uninitialized return value

When allocation fails in btrfs_read_fs_root_no_name, ret is not set
although it is returned, holding a garbage value.

Signed-off-by: David Sterba <dsterba@suse.cz>
Reviewed-by: Li Zefan <lizf@cn.fujitsu.com>
Signed-off-by: Chris Mason <chris.mason@oracle.com>
11 years agobtrfs: fix wrong reservation when doing delayed inode operations
Miao Xie [Wed, 15 Jun 2011 10:47:30 +0000 (10:47 +0000)]
btrfs: fix wrong reservation when doing delayed inode operations

We have migrated the space for the delayed inode items from
trans_block_rsv to global_block_rsv, but we forgot to set trans->block_rsv to
global_block_rsv when we doing delayed inode operations, and the following Oops
happened:

[ 9792.654889] ------------[ cut here ]------------
[ 9792.654898] WARNING: at fs/btrfs/extent-tree.c:5681
btrfs_alloc_free_block+0xca/0x27c [btrfs]()
[ 9792.654899] Hardware name: To Be Filled By O.E.M.
[ 9792.654900] Modules linked in: btrfs zlib_deflate libcrc32c
ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 ip6table_filter ip6_tables
arc4 rt61pci rt2x00pci rt2x00lib snd_hda_codec_hdmi mac80211
snd_hda_codec_realtek cfg80211 snd_hda_intel edac_core snd_seq rfkill
pcspkr serio_raw snd_hda_codec eeprom_93cx6 edac_mce_amd sp5100_tco
i2c_piix4 k10temp snd_hwdep snd_seq_device snd_pcm floppy r8169 xhci_hcd
mii snd_timer snd soundcore snd_page_alloc ipv6 firewire_ohci pata_acpi
ata_generic firewire_core pata_via crc_itu_t radeon ttm drm_kms_helper
drm i2c_algo_bit i2c_core [last unloaded: scsi_wait_scan]
[ 9792.654919] Pid: 2762, comm: rm Tainted: G        W   2.6.39+ #1
[ 9792.654920] Call Trace:
[ 9792.654922]  [<ffffffff81053c4a>] warn_slowpath_common+0x83/0x9b
[ 9792.654925]  [<ffffffff81053c7c>] warn_slowpath_null+0x1a/0x1c
[ 9792.654933]  [<ffffffffa038e747>] btrfs_alloc_free_block+0xca/0x27c [btrfs]
[ 9792.654945]  [<ffffffffa03b8562>] ? map_extent_buffer+0x6e/0xa8 [btrfs]
[ 9792.654953]  [<ffffffffa038189b>] __btrfs_cow_block+0xfc/0x30c [btrfs]
[ 9792.654963]  [<ffffffffa0396aa6>] ? btrfs_buffer_uptodate+0x47/0x58 [btrfs]
[ 9792.654970]  [<ffffffffa0382e48>] ? read_block_for_search+0x94/0x368 [btrfs]
[ 9792.654978]  [<ffffffffa0381ba9>] btrfs_cow_block+0xfe/0x146 [btrfs]
[ 9792.654986]  [<ffffffffa03848b0>] btrfs_search_slot+0x14d/0x4b6 [btrfs]
[ 9792.654997]  [<ffffffffa03b8562>] ? map_extent_buffer+0x6e/0xa8 [btrfs]
[ 9792.655022]  [<ffffffffa03938e8>] btrfs_lookup_inode+0x2f/0x8f [btrfs]
[ 9792.655025]  [<ffffffff8147afac>] ? _cond_resched+0xe/0x22
[ 9792.655027]  [<ffffffff8147b892>] ? mutex_lock+0x29/0x50
[ 9792.655039]  [<ffffffffa03d41b1>] btrfs_update_delayed_inode+0x72/0x137 [btrfs]
[ 9792.655051]  [<ffffffffa03d4ea2>] btrfs_run_delayed_items+0x90/0xdb [btrfs]
[ 9792.655062]  [<ffffffffa039a69b>] btrfs_commit_transaction+0x228/0x654 [btrfs]
[ 9792.655064]  [<ffffffff8106e8da>] ? remove_wait_queue+0x3a/0x3a
[ 9792.655075]  [<ffffffffa03a2fa5>] btrfs_evict_inode+0x14d/0x202 [btrfs]
[ 9792.655077]  [<ffffffff81132bd6>] evict+0x71/0x111
[ 9792.655079]  [<ffffffff81132de0>] iput+0x12a/0x132
[ 9792.655081]  [<ffffffff8112aa3a>] do_unlinkat+0x106/0x155
[ 9792.655083]  [<ffffffff81127b83>] ? path_put+0x1f/0x23
[ 9792.655085]  [<ffffffff8109c53c>] ? audit_syscall_entry+0x145/0x171
[ 9792.655087]  [<ffffffff81128410>] ? putname+0x34/0x36
[ 9792.655090]  [<ffffffff8112b441>] sys_unlinkat+0x29/0x2b
[ 9792.655092]  [<ffffffff81482c42>] system_call_fastpath+0x16/0x1b
[ 9792.655093] ---[ end trace 02b696eb02b3f768 ]---

This patch fix it by setting the reservation of the transaction handle to the
correct one.

Reported-by: Josef Bacik <josef@redhat.com>
Signed-off-by: Miao Xie <miaox@cn.fujitsu.com>
Signed-off-by: Chris Mason <chris.mason@oracle.com>
11 years agobtrfs: Remove unused sysfs code
Maarten Lankhorst [Thu, 16 Jun 2011 09:04:57 +0000 (09:04 +0000)]
btrfs: Remove unused sysfs code

Removes code no longer used. The sysfs file itself is kept, because the
btrfs developers expressed interest in putting new entries to sysfs.

Signed-off-by: Maarten Lankhorst <m.b.lankhorst@gmail.com>
Signed-off-by: Chris Mason <chris.mason@oracle.com>
11 years agobtrfs: fix dereference of ERR_PTR value
David Sterba [Mon, 13 Jun 2011 17:54:22 +0000 (17:54 +0000)]
btrfs: fix dereference of ERR_PTR value

smatch reports:

btrfs_recover_log_trees error: 'wc.replay_dest' dereferencing
possible ERR_PTR()

Signed-off-by: David Sterba <dsterba@suse.cz>
Signed-off-by: Chris Mason <chris.mason@oracle.com>
11 years agoMerge branch 'for-chris' of git://git.kernel.org/pub/scm/linux/kernel/git/josef/btrfs...
Chris Mason [Fri, 17 Jun 2011 18:16:13 +0000 (14:16 -0400)]
Merge branch 'for-chris' of git://git./linux/kernel/git/josef/btrfs-work into for-linus

Conflicts:
fs/btrfs/transaction.c

Signed-off-by: Chris Mason <chris.mason@oracle.com>
11 years agoBtrfs: fix relocation races
Chris Mason [Tue, 14 Jun 2011 00:00:16 +0000 (20:00 -0400)]
Btrfs: fix relocation races

The recent commit to get rid of our trans_mutex introduced
some races with block group relocation.  The problem is that relocation
needs to do some record keeping about each root, and it was relying
on the transaction mutex to coordinate things in subtle ways.

This fix adds a mutex just for the relocation code and makes sure
it doesn't have a big impact on normal operations.  The race is
really fixed in btrfs_record_root_in_trans, which is where we
step back and wait for the relocation code to finish accounting
setup.

Signed-off-by: Chris Mason <chris.mason@oracle.com>
11 years agoBtrfs: set no_trans_join after trying to expand the transaction
Josef Bacik [Tue, 14 Jun 2011 20:22:15 +0000 (16:22 -0400)]
Btrfs: set no_trans_join after trying to expand the transaction

We can lockup if we try to allow new writers join the transaction and we have
flushoncommit set or have a pending snapshot.  This is because we set
no_trans_join and then loop around and try to wait for ordered extents again.
The problem is the ordered endio stuff needs to join the transaction, which it
can't do because no_trans_join is set.  So instead wait until after this loop to
set no_trans_join and then make sure to wait for num_writers == 1 in case
anybody got started in between us exiting the loop and setting no_trans_join.
This could easily be reproduced by mounting -o flushoncommit and running xfstest
13.  It cannot be reproduced with this patch.  Thanks,

Reported-by: Jim Schutt <jaschut@sandia.gov>
Signed-off-by: Josef Bacik <josef@redhat.com>
11 years agoBtrfs: protect the pending_snapshots list with trans_lock
Josef Bacik [Tue, 14 Jun 2011 19:16:14 +0000 (15:16 -0400)]
Btrfs: protect the pending_snapshots list with trans_lock

Currently there is nothing protecting the pending_snapshots list on the
transaction.  We only hold the directory mutex that we are snapshotting and a
read lock on the subvol_sem, so we could race with somebody else creating a
snapshot in a different directory and end up with list corruption.  So protect
this list with the trans_lock.  Thanks,

Signed-off-by: Josef Bacik <josef@redhat.com>
11 years agoBtrfs: fix path leakage on subvol deletion
Josef Bacik [Tue, 14 Jun 2011 18:24:32 +0000 (14:24 -0400)]
Btrfs: fix path leakage on subvol deletion

The delayed ref patch accidently removed the btrfs_free_path in
btrfs_unlink_subvol, this puts it back and means we don't leak a path.  Thanks,

Signed-off-by: Josef Bacik <josef@redhat.com>
11 years agoBtrfs: drop the delalloc_bytes check in shrink_delalloc
Chris Mason [Mon, 13 Jun 2011 15:30:47 +0000 (11:30 -0400)]
Btrfs: drop the delalloc_bytes check in shrink_delalloc

Even when delalloc_bytes is zero, we may need to sleep while waiting
for delalloc space.

Signed-off-by: Chris Mason <chris.mason@oracle.com>
11 years agoBtrfs: check the return value from set_anon_super
Chris Mason [Mon, 13 Jun 2011 15:28:50 +0000 (11:28 -0400)]
Btrfs: check the return value from set_anon_super

Al Viro noticed we weren't checking for set_anon_super failures.  This
adds the required checks.

Signed-off-by: Chris Mason <chris.mason@oracle.com>
11 years agoBtrfs: use join_transaction in btrfs_evict_inode()
Li Zefan [Wed, 8 Jun 2011 03:56:44 +0000 (03:56 +0000)]
Btrfs: use join_transaction in btrfs_evict_inode()

The WARN_ON() in start_transaction() was triggered while balancing.

The cause is btrfs_relocate_chunk() started a transaction and
then called iput() on the inode that stores free space cache,
and iput() called btrfs_start_transaction() again.

Reported-by: Tsutomu Itoh <t-itoh@jp.fujitsu.com>
Signed-off-by: Li Zefan <lizf@cn.fujitsu.com>
Reviewed-by: Josef Bacik <josef@redhat.com>
Signed-off-by: Chris Mason <chris.mason@oracle.com>
11 years agoBtrfs - use %pU to print fsid
Ilya Dryomov [Wed, 9 Feb 2011 14:05:31 +0000 (16:05 +0200)]
Btrfs - use %pU to print fsid

Get rid of FIXME comment.  Uuids from dmesg are now the same as uuids
given by btrfs-progs.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
Signed-off-by: Chris Mason <chris.mason@oracle.com>
11 years agoBtrfs: fix extent state leak on failed nodatasum reads
Jan Schmidt [Wed, 4 May 2011 14:18:50 +0000 (16:18 +0200)]
Btrfs: fix extent state leak on failed nodatasum reads

When encountering an EIO while reading from a nodatasum extent, we
insert an error record into the inode's failure tree.
btrfs_readpage_end_io_hook returns early for nodatasum inodes. We'd
better clear the failure tree in that case, otherwise the kernel
complains about

BUG extent_state: Objects remaining on kmem_cache_close()

on rmmod.

Signed-off-by: Jan Schmidt <list.btrfs@jan-o-sch.net>
Signed-off-by: Chris Mason <chris.mason@oracle.com>
11 years agoMerge branch 'for-chris' of git://git.kernel.org/pub/scm/linux/kernel/git/arne/btrfs...
Chris Mason [Fri, 10 Jun 2011 22:58:08 +0000 (18:58 -0400)]
Merge branch 'for-chris' of git://git./linux/kernel/git/arne/btrfs-unstable-arne into for-linus

11 years agobtrfs: fix unlocked access of delalloc_inodes
David Sterba [Thu, 9 Jun 2011 10:02:51 +0000 (10:02 +0000)]
btrfs: fix unlocked access of delalloc_inodes

list_splice_init will make delalloc_inodes empty, but without a spinlock
around, this may produce corrupted list head, accessed in many placess,
The race window is very tight and nobody seems to have hit it so far.

Signed-off-by: David Sterba <dsterba@suse.cz>
Signed-off-by: Chris Mason <chris.mason@oracle.com>
11 years agoBtrfs: avoid stack bloat in btrfs_ioctl_fs_info()
Li Zefan [Wed, 8 Jun 2011 08:27:56 +0000 (08:27 +0000)]
Btrfs: avoid stack bloat in btrfs_ioctl_fs_info()

The size of struct btrfs_ioctl_fs_info_args is as big as 1KB, so
don't declare the variable on stack.

Signed-off-by: Li Zefan <lizf@cn.fujitsu.com>
Reviewed-by: Josef Bacik <josef@redhat.com>
Signed-off-by: Chris Mason <chris.mason@oracle.com>
11 years agobtrfs: remove 64bit alignment padding to allow extent_buffer to fit into one fewer...
richard kennedy [Tue, 7 Jun 2011 10:46:32 +0000 (10:46 +0000)]
btrfs: remove 64bit alignment padding to allow extent_buffer to fit into one fewer cacheline

Reorder extent_buffer to remove 8 bytes of alignment padding on 64 bit
builds. This shrinks its size to 128 bytes allowing it to fit into one
fewer cache lines and allows more objects per slab in its kmem_cache.

slabinfo extent_buffer reports :-

 before:-
    Sizes (bytes)     Slabs
    ----------------------------------
    Object :     136  Total  :     123
    SlabObj:     136  Full   :     121
    SlabSiz:    4096  Partial:       0
    Loss   :       0  CpuSlab:       2
    Align  :       8  Objects:      30

 after :-
    Object :     128  Total  :       4
    SlabObj:     128  Full   :       2
    SlabSiz:    4096  Partial:       0
    Loss   :       0  CpuSlab:       2
    Align  :       8  Objects:      32

Signed-off-by: Richard Kennedy <richard@rsk.demon.co.uk>
Signed-off-by: Chris Mason <chris.mason@oracle.com>
11 years agoBtrfs: clear current->journal_info on async transaction commit
Sage Weil [Fri, 10 Jun 2011 18:43:13 +0000 (18:43 +0000)]
Btrfs: clear current->journal_info on async transaction commit

Normally current->jouranl_info is cleared by commit_transaction.  For an
async snap or subvol creation, though, it runs in a work queue.  Clear
it in btrfs_commit_transaction_async() to avoid leaking a non-NULL
journal_info when we return to userspace.  When the actual commit runs in
the other thread it won't care that it's current->journal_info is already
NULL.

Signed-off-by: Sage Weil <sage@newdream.net>
Tested-by: Jim Schutt <jaschut@sandia.gov>
Signed-off-by: Chris Mason <chris.mason@oracle.com>
11 years agoBtrfs: make sure to recheck for bitmaps in clusters
Chris Mason [Fri, 10 Jun 2011 20:36:57 +0000 (16:36 -0400)]
Btrfs: make sure to recheck for bitmaps in clusters

Josef recently changed the free extent cache to look in
the block group cluster for any bitmaps before trying to
add a new bitmap for the same offset.  This avoids BUG_ON()s due
covering duplicate ranges.

But it didn't go quite far enough.  A given free range might span
between one or more bitmaps or free space entries.  The code has
looping to cover this, but it doesn't check for clustered bitmaps
every time.

This shuffles our gotos to check for a bitmap in the cluster
for every new bitmap entry we try to add.

Signed-off-by: Chris Mason <chris.mason@oracle.com>
11 years agobtrfs: remove unneeded includes from scrub.c
Arne Jansen [Fri, 10 Jun 2011 11:04:58 +0000 (13:04 +0200)]
btrfs: remove unneeded includes from scrub.c

Signed-off-by: Arne Jansen <sensille@gmx.net>
11 years agobtrfs: reinitialize scrub workers
Arne Jansen [Fri, 10 Jun 2011 10:07:07 +0000 (12:07 +0200)]
btrfs: reinitialize scrub workers

Scrub starts the workers each time a scrub starts and stops them after it
finished. This patch adds an initialization for the workers before each
start, otherwise the workers behave strangely.

Signed-off-by: Arne Jansen <sensille@gmx.net>
11 years agobtrfs: scrub: errors in tree enumeration
Arne Jansen [Fri, 3 Jun 2011 08:09:26 +0000 (10:09 +0200)]
btrfs: scrub: errors in tree enumeration

due to the semantics of btrfs_search_slot the path can point to an
invalid slot when ret > 0. This condition went unnoticed, which in
turn could have led to an incomplete scrubbing.

Signed-off-by: Arne Jansen <sensille@gmx.net>
11 years agoBtrfs: don't map extent buffer if path->skip_locking is set
Josef Bacik [Wed, 8 Jun 2011 18:45:50 +0000 (14:45 -0400)]
Btrfs: don't map extent buffer if path->skip_locking is set

Arne's scrub stuff exposed a problem with mapping the extent buffer in
reada_for_search.  He searches the commit root with multiple threads and with
skip_locking set, so we can race and overwrite node->map_token since node isn't
locked.  So fix this so that we only map the extent buffer if we don't already
have a map_token and skip_locking isn't set.  Without this patch scrub would
panic almost immediately, with the patch it doesn't panic anymore.  Thanks,

Reported-by: Arne Jansen <sensille@gmx.net>
Signed-off-by: Josef Bacik <josef@redhat.com>
11 years agoBtrfs: unlock the trans lock properly
Josef Bacik [Thu, 9 Jun 2011 14:15:17 +0000 (10:15 -0400)]
Btrfs: unlock the trans lock properly

In btrfs_wait_for_commit if we came upon a transaction that had committed we
just exited, but that's bad since we are holding the trans_lock.  So break
instead so that the lock is dropped.  Thanks,

Reported-by: David Sterba <dsterba@suse.cz>
Signed-off-by: Josef Bacik <josef@redhat.com>
11 years agoBtrfs: don't map extent buffer if path->skip_locking is set
Josef Bacik [Wed, 8 Jun 2011 18:36:54 +0000 (14:36 -0400)]
Btrfs: don't map extent buffer if path->skip_locking is set

Arne's scrub stuff exposed a problem with mapping the extent buffer in
reada_for_search.  He searches the commit root with multiple threads and with
skip_locking set, so we can race and overwrite node->map_token since node isn't
locked.  So fix this so that we only map the extent buffer if we don't already
have a map_token and skip_locking isn't set.  Without this patch scrub would
panic almost immediately, with the patch it doesn't panic anymore.  Thanks,

Reported-by: Arne Jansen <sensille@gmx.net>
Signed-off-by: Josef Bacik <josef@redhat.com>
11 years agoBtrfs: fix duplicate checking logic
Josef Bacik [Mon, 6 Jun 2011 14:50:35 +0000 (10:50 -0400)]
Btrfs: fix duplicate checking logic

When merging my code into the integration test the second check for duplicate
entries got screwed up.  This patch fixes it by dropping ret2 and just using ret
for the return value, and checking if we got an error before adding the bitmap
to the local list.  Thanks,

Signed-off-by: Josef Bacik <josef@redhat.com>
11 years agoBtrfs: fix the allocator loop logic
Josef Bacik [Fri, 27 May 2011 20:11:38 +0000 (16:11 -0400)]
Btrfs: fix the allocator loop logic

I was testing with empty_cluster = 0 to try and reproduce a problem and kept
hitting early enospc panics.  This was because our loop logic was a little
confused.  So this is what I did

1) Make the loop variable the ultimate decider on wether we should loop again
isntead of checking to see if we had an uncached bg, empty size or empty
cluster.

2) Increment loop before checking to see what we are on to make the loop
definitions make more sense.

3) If we are on the chunk alloc loop don't set empty_size/empty_cluster to 0
unless we didn't actually allocate a chunk.  If we did allocate a chunk we
should be able to easily setup a new cluster so clearing
empty_size/empty_cluster makes us less efficient.

This kept me from hitting panics while trying to reproduce the other problem.
Thanks,

Signed-off-by: Josef Bacik <josef@redhat.com>
11 years agoBtrfs: fix bitmap regression
Josef Bacik [Fri, 27 May 2011 18:07:49 +0000 (14:07 -0400)]
Btrfs: fix bitmap regression

In cleaning up the clustering code I accidently introduced a regression by
adding bitmap entries to the cluster rb tree.  The problem is if we've maxed out
the number of bitmaps we can have for the block group we can only add free space
to the bitmaps, but since the bitmap is on the cluster we can't find it and we
try to create another one.  This would result in a panic because the total
bitmaps was bigger than the max bitmaps that were allowed.  This patch fixes
this by checking to see if we have a cluster, and then looking at the cluster rb
tree to see if it has a bitmap entry and if it does and that space belongs to
that bitmap, go ahead and add it to that bitmap.

I could hit this panic every time with an fs_mark test within a couple of
minutes.  With this patch I no longer hit the panic and fs_mark goes to
completion.  Thanks,

Signed-off-by: Josef Bacik <josef@redhat.com>
11 years agoBtrfs: don't commit the transaction if we dont have enough pinned bytes
Josef Bacik [Wed, 25 May 2011 17:10:16 +0000 (13:10 -0400)]
Btrfs: don't commit the transaction if we dont have enough pinned bytes

I noticed when running an enospc test that we would get stuck committing the
transaction in check_data_space even though we truly didn't have enough space.
So check to see if bytes_pinned is bigger than num_bytes, if it's not don't
commit the transaction.  Thanks,

Signed-off-by: Josef Bacik <josef@redhat.com>
11 years agoBtrfs: noinline the cluster searching functions
Josef Bacik [Wed, 25 May 2011 17:07:37 +0000 (13:07 -0400)]
Btrfs: noinline the cluster searching functions

When profiling the find cluster code it's hard to tell where we are spending our
time because the bitmap and non-bitmap functions get inlined by the compiler, so
make that not happen.  Thanks,

Signed-off-by: Josef Bacik <josef@redhat.com>
11 years agoBtrfs: cache bitmaps when searching for a cluster
Josef Bacik [Wed, 25 May 2011 17:03:16 +0000 (13:03 -0400)]
Btrfs: cache bitmaps when searching for a cluster

If we are looking for a cluster in a particularly sparse or fragmented block
group, we will do a lot of looping through the free space tree looking for
various things, and if we need to look at bitmaps we will endup doing the whole
dance twice.  So instead add the bitmap entries to a temporary list so if we
have to do the bitmap search we can just look through the list of entries we've
found quickly instead of having to loop through the entire tree again.  Thanks,

Signed-off-by: Josef Bacik <josef@redhat.com>
11 years agobtrfs: fix uninitialized variable warning
David Sterba [Fri, 3 Jun 2011 14:29:08 +0000 (16:29 +0200)]
btrfs: fix uninitialized variable warning

With Linus' tree, today's linux-next build (powercp ppc64_defconfig)
produced this warning:

fs/btrfs/delayed-inode.c: In function 'btrfs_delayed_update_inode':
fs/btrfs/delayed-inode.c:1598:6: warning: 'ret' may be used
uninitialized in this function

Introduced by commit 16cdcec736cd ("btrfs: implement delayed inode items
operation").

This fixes a bug in btrfs_update_inode(): if the returned value from
btrfs_delayed_update_inode is a nonzero garbage, inode stat data are not
updated and several call paths may hit a BUG_ON or fail with strange
code.

Reported-by: Stephen Rothwell <sfr@canb.auug.org.au>
Signed-off-by: David Sterba <dsterba@suse.cz>
11 years agobtrfs: add helper for fs_info->closing
David Sterba [Tue, 31 May 2011 16:07:27 +0000 (18:07 +0200)]
btrfs: add helper for fs_info->closing

wrap checking of filesystem 'closing' flag and fix a few missing memory
barriers.

Signed-off-by: David Sterba <dsterba@suse.cz>
11 years agoBtrfs: add mount -o inode_cache
Chris Mason [Fri, 3 Jun 2011 13:36:29 +0000 (09:36 -0400)]
Btrfs: add mount -o inode_cache

This makes the inode map cache default to off until we
fix the overflow problem when the free space crcs don't fit
inside a single page.

Signed-off-by: Chris Mason <chris.mason@oracle.com>
11 years agobtrfs: scrub: add explicit plugging
Arne Jansen [Sat, 28 May 2011 20:58:38 +0000 (20:58 +0000)]
btrfs: scrub: add explicit plugging

With the removal of the implicit plugging scrub ends up doing more and
smaller I/O than necessary. This patch adds explicit plugging per chunk.

Signed-off-by: Arne Jansen <sensille@gmx.net>
Signed-off-by: Chris Mason <chris.mason@oracle.com>
11 years agobtrfs: use btrfs_ino to access inode number
David Sterba [Tue, 31 May 2011 17:08:14 +0000 (17:08 +0000)]
btrfs: use btrfs_ino to access inode number

commit 4cb5300bc ("Btrfs: add mount -o auto_defrag") accesses inode
number directly while it should use the helper with the new inode
number allocator.

Signed-off-by: David Sterba <dsterba@suse.cz>
Signed-off-by: Chris Mason <chris.mason@oracle.com>
11 years agoBtrfs: don't save the inode cache if we are deleting this root
Josef Bacik [Tue, 31 May 2011 19:33:33 +0000 (19:33 +0000)]
Btrfs: don't save the inode cache if we are deleting this root

With xfstest 254 I can panic the box every time with the inode number caching
stuff on.  This is because we clean the inodes out when we delete the subvolume,
but then we write out the inode cache which adds an inode to the subvolume inode
tree, and then when it gets evicted again the root gets added back on the dead
roots list and is deleted again, so we have a double free.  To stop this from
happening just return 0 if refs is 0 (and we're not the tree root since tree
root always has refs of 0).  With this fix 254 no longer panics.  Thanks,

Signed-off-by: Josef Bacik <josef@redhat.com>
Tested-by: David Sterba <dsterba@suse.cz>
Signed-off-by: Chris Mason <chris.mason@oracle.com>
11 years agobtrfs: false BUG_ON when degraded
Arne Jansen [Mon, 30 May 2011 08:36:16 +0000 (08:36 +0000)]
btrfs: false BUG_ON when degraded

In degraded mode the struct btrfs_device of missing devs don't have
device->name set. A kstrdup of NULL correctly returns NULL. Don't
BUG in this case.

Signed-off-by: Arne Jansen <sensille@gmx.net>
Signed-off-by: Chris Mason <chris.mason@oracle.com>
11 years agoBtrfs: don't save the inode cache in non-FS roots
liubo [Wed, 1 Jun 2011 09:42:49 +0000 (09:42 +0000)]
Btrfs: don't save the inode cache in non-FS roots

This adds extra checks to make sure the inode map we are caching really
belongs to a FS root instead of a special relocation tree.  It
prevents crashes during balancing operations.

Signed-off-by: Liu Bo <liubo2009@cn.fujitsu.com>
Signed-off-by: Chris Mason <chris.mason@oracle.com>
11 years agoBtrfs: make sure we don't overflow the free space cache crc page
Chris Mason [Fri, 3 Jun 2011 05:26:53 +0000 (01:26 -0400)]
Btrfs: make sure we don't overflow the free space cache crc page

The free space cache uses only one page for crcs right now,
which means we can't have a cache file bigger than the
crcs we can fit in the first page.  This adds a check to
enforce that restriction.

Signed-off-by: Chris Mason <chris.mason@oracle.com>
11 years agoBtrfs: fix uninit variable in the delayed inode code
Chris Mason [Fri, 3 Jun 2011 05:13:45 +0000 (01:13 -0400)]
Btrfs: fix uninit variable in the delayed inode code

The nitems counter needs to start at zero

Signed-off-by: Chris Mason <chris.mason@oracle.com>
11 years agobtrfs: scrub: don't reuse bios and pages
Arne Jansen [Sat, 28 May 2011 19:57:55 +0000 (21:57 +0200)]
btrfs: scrub: don't reuse bios and pages

The current scrub implementation reuses bios and pages as often as possible,
allocating them only on start and releasing them when finished. This leads
to more problems with the block layer than it's worth. The elevator gets
confused when there are more pages added to the bio than bi_size suggests.
This patch completely rips out the reuse of bios and pages and allocates
them freshly for each submit.

Signed-off-by: Arne Jansen <sensille@gmx.net>
Signed-off-by: Chris Maosn <chris.mason@oracle.com>
11 years agoMerge branch 'for-chris' of
Chris Mason [Sat, 28 May 2011 11:00:39 +0000 (07:00 -0400)]
Merge branch 'for-chris' of
git://git.kernel.org/pub/scm/linux/kernel/git/josef/btrfs-work into for-linus

Conflicts:
fs/btrfs/disk-io.c
fs/btrfs/extent-tree.c
fs/btrfs/free-space-cache.c
fs/btrfs/inode.c
fs/btrfs/transaction.c

Signed-off-by: Chris Mason <chris.mason@oracle.com>
11 years agoBtrfs: use the device_list_mutex during write_dev_supers
Chris Mason [Fri, 27 May 2011 14:03:58 +0000 (10:03 -0400)]
Btrfs: use the device_list_mutex during write_dev_supers

write_dev_supers was changed to use RCU to protect the list of
devices, but it was then sleeping while it actually wrote the supers.
This fixes it to just use the mutex, since we really don't any
concurrency in write_dev_supers anyway.

Signed-off-by: Chris Mason <chris.mason@oracle.com>
11 years agoBtrfs: setup free ino caching in a more asynchronous way
Li Zefan [Thu, 26 May 2011 06:38:30 +0000 (06:38 +0000)]
Btrfs: setup free ino caching in a more asynchronous way

For a filesystem that has lots of files in it, the first time we mount
it with free ino caching support, it can take quite a long time to
setup the caching before we can create new files.

Here we fill the cache with [highest_ino, BTRFS_LAST_FREE_OBJECTID]
before we start the caching thread to search through the extent tree.

Signed-off-by: Li Zefan <lizf@cn.fujitsu.com>
Signed-off-by: Chris Mason <chris.mason@oracle.com>
11 years agobtrfs scrub: don't coalesce pages that are logically discontiguous
Arne Jansen [Wed, 25 May 2011 12:22:50 +0000 (12:22 +0000)]
btrfs scrub: don't coalesce pages that are logically discontiguous

scrub_page collects several pages into one bio as long as they are physically
contiguous. As we only save one logical address for the whole bio, don't
collect pages that are physically contiguous but logically discontiguous.

Signed-off-by: Arne Jansen <sensille@gmx.net>
Signed-off-by: Chris Mason <chris.mason@oracle.com>
11 years agoBtrfs: return -ENOMEM in clear_extent_bit
Chris Mason [Thu, 26 May 2011 21:43:59 +0000 (17:43 -0400)]
Btrfs: return -ENOMEM in clear_extent_bit

The btrfs releasepage function depends on ENOMEM coming
back when it is called atomic.

Signed-off-by: Chris Mason <chris.mason@oracle.com>
11 years agoBtrfs: add mount -o auto_defrag
Chris Mason [Tue, 24 May 2011 19:35:30 +0000 (15:35 -0400)]
Btrfs: add mount -o auto_defrag

This will detect small random writes into files and
queue the up for an auto defrag process.  It isn't well suited to
database workloads yet, but works for smaller files such as rpm, sqlite
or bdb databases.

Signed-off-by: Chris Mason <chris.mason@oracle.com>
11 years agoMerge branch 'cleanups_and_fixes' into inode_numbers
Chris Mason [Mon, 23 May 2011 18:37:47 +0000 (14:37 -0400)]
Merge branch 'cleanups_and_fixes' into inode_numbers

Conflicts:
fs/btrfs/tree-log.c
fs/btrfs/volumes.c

Signed-off-by: Chris Mason <chris.mason@oracle.com>
11 years agoBtrfs: using rcu lock in the reader side of devices list
Xiao Guangrong [Wed, 20 Apr 2011 10:09:16 +0000 (10:09 +0000)]
Btrfs: using rcu lock in the reader side of devices list

fs_devices->devices is only updated on remove and add device paths, so we can
use rcu to protect it in the reader side

Signed-off-by: Xiao Guangrong <xiaoguangrong@cn.fujitsu.com>
Signed-off-by: Chris Mason <chris.mason@oracle.com>
11 years agoBtrfs: drop unnecessary device lock
Xiao Guangrong [Wed, 20 Apr 2011 10:08:47 +0000 (10:08 +0000)]
Btrfs: drop unnecessary device lock

Drop device_list_mutex for the reader side  on clone_fs_devices and
btrfs_rm_device pathes since the fs_info->volume_mutex can ensure the device
list is not updated

btrfs_close_extra_devices is the initialized path, we can not add or remove
device at this time, so we can simply drop the mutex safely, like other
initialized function does(add_missing_dev, __find_device, __btrfs_open_devices
...).

Signed-off-by: Xiao Guangrong <xiaoguangrong@cn.fujitsu.com>
Signed-off-by: Chris Mason <chris.mason@oracle.com>
11 years agoBtrfs: fix the race between remove dev and alloc chunk
Xiao Guangrong [Wed, 20 Apr 2011 10:08:16 +0000 (10:08 +0000)]
Btrfs: fix the race between remove dev and alloc chunk

On remove device path, it updates device->dev_alloc_list but does not hold
chunk lock

Signed-off-by: Xiao Guangrong <xiaoguangrong@cn.fujitsu.com>
Signed-off-by: Chris Mason <chris.mason@oracle.com>
11 years agoBtrfs: fix the race between reading and updating devices
Xiao Guangrong [Wed, 20 Apr 2011 10:07:30 +0000 (10:07 +0000)]
Btrfs: fix the race between reading and updating devices

On btrfs_congested_fn and __unplug_io_fn paths, we should hold
device_list_mutex to avoid remove/add device path to
update fs_devices->devices

On __btrfs_close_devices and btrfs_prepare_sprout paths, the devices in
fs_devices->devices or fs_devices->devices is updated, so we should hold
the mutex to avoid the reader side to reach them

Signed-off-by: Xiao Guangrong <xiaoguangrong@cn.fujitsu.com>
Signed-off-by: Chris Mason <chris.mason@oracle.com>
11 years agoBtrfs: fix bh leak on __btrfs_open_devices path
Xiao Guangrong [Wed, 20 Apr 2011 10:06:40 +0000 (10:06 +0000)]
Btrfs: fix bh leak on __btrfs_open_devices path

'bh' is forgot to release if no error is detected

Signed-off-by: Xiao Guangrong <xiaoguangrong@cn.fujitsu.com>
Signed-off-by: Chris Mason <chris.mason@oracle.com>
11 years agoBtrfs: fix unsafe usage of merge_state
Xiao Guangrong [Wed, 20 Apr 2011 06:45:49 +0000 (06:45 +0000)]
Btrfs: fix unsafe usage of merge_state

merge_state can free the current state if it can be merged with the next node,
but in set_extent_bit(), after merge_state, we still use the current extent to
get the next node and cache it into cached_state

Signed-off-by: Xiao Guangrong <xiaoguangrong@cn.fujitsu.com>
Signed-off-by: Chris Mason <chris.mason@oracle.com>
11 years agoBtrfs: allocate extent state and check the result properly
Xiao Guangrong [Wed, 20 Apr 2011 06:44:57 +0000 (06:44 +0000)]
Btrfs: allocate extent state and check the result properly

It doesn't allocate extent_state and check the result properly:
- in set_extent_bit, it doesn't allocate extent_state if the path is not
  allowed wait

- in clear_extent_bit, it doesn't check the result after atomic-ly allocate,
  we trigger BUG_ON() if it's fail

- if allocate fail, we trigger BUG_ON instead of returning -ENOMEM since
  the return value of clear_extent_bit() is ignored by many callers

Signed-off-by: Xiao Guangrong <xiaoguangrong@cn.fujitsu.com>
Signed-off-by: Chris Mason <chris.mason@oracle.com>
11 years agofs/btrfs: Add missing btrfs_free_path
Julia Lawall [Sat, 14 May 2011 07:10:51 +0000 (07:10 +0000)]
fs/btrfs: Add missing btrfs_free_path

Btrfs_alloc_path should be matched with btrfs_free_path in error-handling code.

A simplified version of the semantic match that finds this problem is as
follows: (http://coccinelle.lip6.fr/)

// <smpl>
@r exists@
local idexpression struct btrfs_path * x;
expression ra,rb;
position p1,p2;
@@

x = btrfs_alloc_path@p1(...)
...  when != btrfs_free_path(x,...)
     when != if (...) { ... btrfs_free_path(x,...) ...}
     when != x = ra
if(...) { ... when != x = rb
     when forall
     when != btrfs_free_path(x,...)
 \(return <+...x...+>; \| return@p2...; \) }

@script:python@
p1 << r.p1;
p2 << r.p2;
@@

cocci.print_main("alloc",p1)
cocci.print_secs("return",p2)
// </smpl>

Signed-off-by: Julia Lawall <julia@diku.dk>
Signed-off-by: Chris Mason <chris.mason@oracle.com>
11 years agoBtrfs: check return value of btrfs_inc_extent_ref()
Tsutomu Itoh [Thu, 28 Apr 2011 09:18:21 +0000 (09:18 +0000)]
Btrfs: check return value of btrfs_inc_extent_ref()

If return value of btrfs_inc_extent_ref() is not 0, BUG() is called.

Signed-off-by: Tsutomu Itoh <t-itoh@jp.fujitsu.com>
Signed-off-by: Chris Mason <chris.mason@oracle.com>
11 years agoBtrfs: return error to caller if read_one_inode() fails
Tsutomu Itoh [Thu, 28 Apr 2011 09:10:23 +0000 (09:10 +0000)]
Btrfs: return error to caller if read_one_inode() fails

When read_one_inode() fails, error code is returned to caller instead
of BUG_ON().

Signed-off-by: Tsutomu Itoh <t-itoh@jp.fujitsu.com>
Signed-off-by: Chris Mason <chris.mason@oracle.com>
11 years agoBtrfs: BUG_ON is deleted from the caller of btrfs_truncate_item & btrfs_extend_item
Tsutomu Itoh [Thu, 19 May 2011 05:19:08 +0000 (05:19 +0000)]
Btrfs: BUG_ON is deleted from the caller of btrfs_truncate_item & btrfs_extend_item

Currently, btrfs_truncate_item and btrfs_extend_item returns only 0.
So, the check by BUG_ON in the caller is unnecessary.

Signed-off-by: Tsutomu Itoh <t-itoh@jp.fujitsu.com>
Signed-off-by: Chris Mason <chris.mason@oracle.com>
11 years agoBtrfs: return error code to caller when btrfs_del_item fails
Tsutomu Itoh [Thu, 19 May 2011 04:37:44 +0000 (04:37 +0000)]
Btrfs: return error code to caller when btrfs_del_item fails

The error code is returned instead of calling BUG_ON when
btrfs_del_item returns the error.

Signed-off-by: Tsutomu Itoh <t-itoh@jp.fujitsu.com>
Signed-off-by: Chris Mason <chris.mason@oracle.com>
11 years agoBtrfs: return error code to caller when btrfs_previous_item fails
Tsutomu Itoh [Thu, 19 May 2011 07:03:42 +0000 (07:03 +0000)]
Btrfs: return error code to caller when btrfs_previous_item fails

The error code is returned instead of calling BUG_ON when
btrfs_previous_item returns the error.

Signed-off-by: Tsutomu Itoh <t-itoh@jp.fujitsu.com>
Signed-off-by: Chris Mason <chris.mason@oracle.com>
11 years agobtrfs: fix typo 'testeing' -> 'testing'
Sergei Trofimovich [Fri, 20 May 2011 20:20:32 +0000 (20:20 +0000)]
btrfs: fix typo 'testeing' -> 'testing'

Signed-off-by: Sergei Trofimovich <slyfox@gentoo.org>
Signed-off-by: Chris Mason <chris.mason@oracle.com>
11 years agobtrfs: typo: 'btrfS' -> 'btrfs'
Sergei Trofimovich [Fri, 20 May 2011 20:20:31 +0000 (20:20 +0000)]
btrfs: typo: 'btrfS' -> 'btrfs'

Signed-off-by: Sergei Trofimovich <slyfox@gentoo.org>
Signed-off-by: Chris Mason <chris.mason@oracle.com>
11 years agobtrfs: don't spin in shrink_delalloc if there is nothing to free
Sergei Trofimovich [Fri, 20 May 2011 20:20:30 +0000 (20:20 +0000)]
btrfs: don't spin in shrink_delalloc if there is nothing to free

Observed as a large delay when --mixed filesystem is filled up.
Test example:
1. create tiny --mixed FS:
   $ dd if=/dev/zero of=2G.img seek=$((2048 * 1024 * 1024 - 1)) count=1 bs=1
   $ mkfs.btrfs --mixed 2G.img
   $ mount -oloop 2G.img /mnt/ut/
2. Try to fill it up:
   $ dd if=/dev/urandom of=10M.file bs=10240 count=1024
   $ seq 1 256 | while read file_no; do echo $file_no; time cp 10M.file ${file_no}.copy; done

Up to '200.copy' it goes fast, but when disk fills-up each -ENOSPC
message takes 3 seconds to pop-up _every_ ENOSPC (and in usermode linux
it's even more: 30-60 seconds!). (Maybe, time depends on kernel's timer resolution).

No IO, no CPU load, just rescheduling. Some debugging revealed busy spinning
in shrink_delalloc.

Signed-off-by: Sergei Trofimovich <slyfox@gentoo.org>
Reviewed-by: Josef Bacik <josef@redhat.com>
Signed-off-by: Chris Mason <chris.mason@oracle.com>
11 years agobtrfs: Delete unused version.sh script.
Jamey Sharp [Thu, 5 May 2011 19:03:46 +0000 (19:03 +0000)]
btrfs: Delete unused version.sh script.

In 2008, commit b4f6c45dfbf84f47c21f73f6370ad1292b0627fd dropped the use
of fs/btrfs/version.sh, but left the script behind. Kill it.

Commit by Jamey Sharp and Josh Triplett.

Signed-off-by: Jamey Sharp <jamey@minilop.net>
Signed-off-by: Josh Triplett <josh@joshtriplett.org>
Cc: Chris Mason <chris.mason@oracle.com>
Signed-off-by: Chris Mason <chris.mason@oracle.com>
11 years agobtrfs: Ensure the tree search ioctl returns the right number of records
Hugo Mills [Sat, 14 May 2011 17:43:41 +0000 (17:43 +0000)]
btrfs: Ensure the tree search ioctl returns the right number of records

Btrfs's tree search ioctl has a field to indicate that no more than a
given number of records should be returned. The ioctl doesn't honour
this, as the tested value is not incremented until the end of the
copy_to_sk function. This patch removes an unnecessary local variable,
and updates the num_found counter as each key is found in the tree.

Signed-off-by: Hugo Mills <hugo@carfax.org.uk>
Signed-off-by: Chris Mason <chris.mason@oracle.com>
11 years agoBTRFS: Remove unused node_lock
Andi Kleen [Wed, 18 May 2011 00:11:22 +0000 (00:11 +0000)]
BTRFS: Remove unused node_lock

240f62c8756 replaced the node_lock with rcu_read_lock, but forgot
to remove the actual lock in the data structure. Remove it here.

Signed-off-by: Andi Kleen <ak@linux.intel.com>
Signed-off-by: Chris Mason <chris.mason@oracle.com>
11 years agoBtrfs: leave spinning on lookup and map the leaf
Josef Bacik [Tue, 17 May 2011 13:50:54 +0000 (09:50 -0400)]
Btrfs: leave spinning on lookup and map the leaf

On lookup we only want to read the inode item, so leave the path spinning.  Also
we're just wholesale reading the leaf off, so map the leaf so we don't do a
bunch of kmap/kunmaps.  Thanks,

Signed-off-by: Josef Bacik <josef@redhat.com>
11 years agoBtrfs: check for duplicate entries in the free space cache
Josef Bacik [Fri, 13 May 2011 18:49:23 +0000 (14:49 -0400)]
Btrfs: check for duplicate entries in the free space cache

If there are duplicate entries in the free space cache, discard the entire cache
and load it the old fashioned way.  Thanks,

Signed-off-by: Josef Bacik <josef@redhat.com>
11 years agoBtrfs: don't try to allocate from a block group that doesn't have enough space
Josef Bacik [Fri, 13 May 2011 15:07:12 +0000 (11:07 -0400)]
Btrfs: don't try to allocate from a block group that doesn't have enough space

If we have a very large filesystem, we can spend a lot of time in
find_free_extent just trying to allocate from empty block groups.  So instead
check to see if the block group even has enough space for the allocation, and if
not go on to the next block group.

Signed-off-by: Josef Bacik <josef@redhat.com>
11 years agoBtrfs: don't always do readahead
Josef Bacik [Fri, 13 May 2011 14:32:11 +0000 (10:32 -0400)]
Btrfs: don't always do readahead

Our readahead is sort of sloppy, and really isn't always needed.  For example if
ls is doing a stating ls (which is the default) it's going to stat in non-disk
order, so if say you have a directory with a stupid amount of files, readahead
is going to do nothing but waste time in the case of doing the stat.  Taking the
unconditional readahead out made my test go from 57 minutes to 36 minutes.  This
means that everywhere we do loop through the tree we want to make sure we do set
path->reada properly, so I went through and found all of the places where we
loop through the path and set reada to 1.  Thanks,

Signed-off-by: Josef Bacik <josef@redhat.com>
11 years agoBtrfs: try not to sleep as much when doing slow caching
Josef Bacik [Wed, 11 May 2011 21:30:53 +0000 (17:30 -0400)]
Btrfs: try not to sleep as much when doing slow caching

When the fs is super full and we unmount the fs, we could get stuck in this
thing where unmount is waiting for the caching kthread to make progress and the
caching kthread keeps scheduling because we're in the middle of a commit.  So
instead just let the caching kthread keep going and only yeild if
need_resched().  This makes my horrible umount case go from taking up to 10
minutes to taking less than 20 seconds.  Thanks,

Signed-off-by: Josef Bacik <josef@redhat.com>