Skip to contentSkip to conversation

kernel BUG at fs/btrfs/inode.c:149!

DISCUSSION Josef Bacik 3 participants 11 messages May 5, 2011 – May 31, 2011 1 message not in the archive Btrfs file system

lore
Share & tools
lore

A message that is not in the archive (sent off-list or not yet received): [email protected]

#1 lore raw
Quoted 9 lines · On 05/05/2011 01:54 PM, [email protected] wrote:

On Thursday 05 May 2011 19:53:52 Josef Bacik wrote:
[...]

Quoted 4 lines

Hmm I didn't see what I needed, can you do it again and try to get kernel messages to go to /var/log/messages so you don't have to worry about the log buffer? The other option is to setup netconsole which would grab everything. Thanks,

Ok, this is all of it.

It doesn't look like that bit had my debugging output. Thanks,

Josef

#2 lore raw

On Thursday 05 May 2011 20:57:17 Josef Bacik wrote:
[..]

Quoted 3 lines

It doesn't look like that bit had my debugging output. Thanks,

Josef

Looks like the last message I sent didn't make to the list. Here's the link to the debug log:

http://pastebin.com/raw.php?i=fnfsnVZS

Thanks,

Signature · 1 line
Elric Milon
#3 lore raw
Quoted 9 lines · On 05/13/2011 01:19 PM, [email protected] wrote:

On Thursday 05 May 2011 20:57:17 Josef Bacik wrote:
[..]

Quoted 3 lines

It doesn't look like that bit had my debugging output. Thanks,

Josef

Looks like the last message I sent didn't make to the list. Here's the link to the debug log:

So unfortunately I don't know how we ended up with duplicate entries in the free space cache, but I can make it so we discard the cache if this happens. Please try the patch I just sent to the list

[PATCH] Btrfs: check for duplicate entries in the free space cache

this will make your fs able to be mounted at the very least. I'll try and figure out how this sort of thing happens. If you manage to make it happen on purpose let me know how you did it so I can figure out what I'm doing wrong. Thanks,

Josef

#4 lore raw
Quoted 16 lines · On Friday 13 May 2011 20:52:22 Josef Bacik wrote:
Quoted 11 lines · On 05/13/2011 01:19 PM, [email protected] wrote:

On Thursday 05 May 2011 20:57:17 Josef Bacik wrote:
[..]

Quoted 3 lines

It doesn't look like that bit had my debugging output. Thanks,

Josef

Looks like the last message I sent didn't make to the list. Here's the link to

the debug log:

So unfortunately I don't know how we ended up with duplicate entries in the free space cache, but I can make it so we discard the cache if this happens. Please try the patch I just sent to the list

Quoted 6 lines

[PATCH] Btrfs: check for duplicate entries in the free space cache

this will make your fs able to be mounted at the very least. I'll try and figure out how this sort of thing happens. If you manage to make it happen on purpose let me know how you did it so I can figure out what I'm doing wrong. Thanks,

I was able to mount it readonly and copy all the content from it without problems (with vanilla linus' tree). The only out of ordinary thing I did before the error coming up, was to defrag the filesystem (find ~/ -xdev -type f -print0 | xargs -0 sudo ./btrfs filesystem defragment -c) with btrfs-tools from the latest git. After this I think I rebooted once the same day and everything was working ok. The day after in the morning I powered up the laptop and I wasn't able to mount the home volume.
I'm on a 64 bit Debian SID with custom kernel. Using btrfs on home and root and ext4 in 3 or 4 other FS, everything is on lvm in luks.

The only thing I think of I can try is to recreate the filesystem, copy the same data on it, defrag and try to umount/remount several times... (although the data will not be fragmented so we may not hit this error) What do you say?

Thanks,

Signature · 4 lines
Elric Milon
-- 

Elric Milon
#5 lore raw
Quoted 34 lines · On 05/16/2011 05:17 AM, [email protected] wrote:
Quoted 16 lines · On Friday 13 May 2011 20:52:22 Josef Bacik wrote:
Quoted 11 lines · On 05/13/2011 01:19 PM, [email protected] wrote:

On Thursday 05 May 2011 20:57:17 Josef Bacik wrote:
[..]

Quoted 3 lines

It doesn't look like that bit had my debugging output. Thanks,

Josef

Looks like the last message I sent didn't make to the list. Here's the link to

the debug log:

So unfortunately I don't know how we ended up with duplicate entries in the free space cache, but I can make it so we discard the cache if this happens. Please try the patch I just sent to the list

Quoted 6 lines

[PATCH] Btrfs: check for duplicate entries in the free space cache

this will make your fs able to be mounted at the very least. I'll try and figure out how this sort of thing happens. If you manage to make it happen on purpose let me know how you did it so I can figure out what I'm doing wrong. Thanks,

I was able to mount it readonly and copy all the content from it without problems (with vanilla linus' tree). The only out of ordinary thing I did before the error coming up, was to defrag the filesystem (find ~/ -xdev -type f -print0 | xargs -0 sudo ./btrfs filesystem defragment -c) with btrfs-tools from the latest git. After this I think I rebooted once the same day and everything was working ok. The day after in the morning I powered up the laptop and I wasn't able to mount the home volume.
I'm on a 64 bit Debian SID with custom kernel. Using btrfs on home and root and ext4 in 3 or 4 other FS, everything is on lvm in luks.

Quoted 4 lines

The only thing I think of I can try is to recreate the filesystem, copy the same data on it, defrag and try to umount/remount several times... (although the data will not be fragmented so we may not hit this error) What do you say?

Sorry I'm having problems following what you are saying. You mean this is how you got into this current situation, or that this is what is currently happening to you. If it's the first one then cool, if you can try and make it happen again I would be grateful. If it's the second one then my patch didn't work and I need to try and figure out whats going wrong. Thanks,

Josef

#6 lore raw
Quoted 40 lines · On Monday 16 May 2011 16:11:19 Josef Bacik wrote:
Quoted 34 lines · On 05/16/2011 05:17 AM, [email protected] wrote:
Quoted 16 lines · On Friday 13 May 2011 20:52:22 Josef Bacik wrote:
Quoted 11 lines · On 05/13/2011 01:19 PM, [email protected] wrote:

On Thursday 05 May 2011 20:57:17 Josef Bacik wrote:
[..]

Quoted 3 lines

It doesn't look like that bit had my debugging output. Thanks,

Josef

Looks like the last message I sent didn't make to the list. Here's the link to

the debug log:

So unfortunately I don't know how we ended up with duplicate entries in the free space cache, but I can make it so we discard the cache if this happens. Please try the patch I just sent to the list

Quoted 6 lines

[PATCH] Btrfs: check for duplicate entries in the free space cache

this will make your fs able to be mounted at the very least. I'll try and figure out how this sort of thing happens. If you manage to make it happen on purpose let me know how you did it so I can figure out what I'm doing wrong. Thanks,

I was able to mount it readonly and copy all the content from it without problems (with vanilla linus' tree). The only out of ordinary thing I did before the error coming up, was to defrag the filesystem (find ~/ -xdev -type f -print0 | xargs -0 sudo ./btrfs filesystem defragment -c) with btrfs-tools from the latest git. After this I think I rebooted once the same day and everything was working ok. The day after in the morning I powered up the laptop and I wasn't able to mount the home volume. I'm on a 64 bit Debian SID with custom kernel. Using btrfs on home and root and ext4 in 3 or 4 other FS, everything is on lvm in luks.

Quoted 4 lines

The only thing I think of I can try is to recreate the filesystem, copy the same data on it, defrag and try to umount/remount several times... (although the data will not be fragmented so we may not hit this error) What do you say?

Quoted 8 lines

Sorry I'm having problems following what you are saying. You mean this is how you got into this current situation, or that this is what is currently happening to you. If it's the first one then cool, if you can try and make it happen again I would be grateful. If it's the second one then my patch didn't work and I need to try and figure out whats going wrong. Thanks,

Josef

Sorry yes, I meant this is how I managed to get the corrupted filesystem.

Ill try to break it again.

Thanks,

#7 lore raw
Quoted 48 lines · Whirm
Quoted 47 lines · On Monday 16 May 2011 16:11:19 Josef Bacik wrote:
Quoted 34 lines · On 05/16/2011 05:17 AM, [email protected] wrote:
Quoted 16 lines · On Friday 13 May 2011 20:52:22 Josef Bacik wrote:
Quoted 11 lines · On 05/13/2011 01:19 PM, [email protected] wrote:

On Thursday 05 May 2011 20:57:17 Josef Bacik wrote:
[..]

Quoted 3 lines

It doesn't look like that bit had my debugging output. Thanks,

Josef

Looks like the last message I sent didn't make to the list. Here's the link to

the debug log:

So unfortunately I don't know how we ended up with duplicate entries in the free space cache, but I can make it so we discard the cache if this happens. Please try the patch I just sent to the list

Quoted 6 lines

[PATCH] Btrfs: check for duplicate entries in the free space cache

this will make your fs able to be mounted at the very least. I'll try and figure out how this sort of thing happens. If you manage to make it happen on purpose let me know how you did it so I can figure out what I'm doing wrong. Thanks,

I was able to mount it readonly and copy all the content from it without problems (with vanilla linus' tree). The only out of ordinary thing I did before the error coming up, was to defrag the filesystem (find ~/ -xdev -type f -print0 | xargs -0 sudo ./btrfs filesystem defragment -c) with btrfs-tools from the latest git. After this I think I rebooted once the same day and everything was working ok. The day after in the morning I powered up the laptop and I wasn't able to mount the home volume. I'm on a 64 bit Debian SID with custom kernel. Using btrfs on home and root and ext4 in 3 or 4 other FS, everything is on lvm in luks.

Quoted 4 lines

The only thing I think of I can try is to recreate the filesystem, copy the same data on it, defrag and try to umount/remount several times... (although the data will not be fragmented so we may not hit this error) What do you say?

Sorry I'm having problems following what you are saying. You mean this is how you got into this current situation, or that this is what is currently happening to you. If it's the first one then cool, if you can try and make it happen again I would be grateful. If it's the second one then my patch didn't work and I need to try and figure out whats going wrong. Thanks,

Quoted 6 lines
Quoted 1 line

Josef

Sorry yes, I meant this is how I managed to get the corrupted filesystem.

Ill try to break it again.

Oh ok perfect, yeah I will try and do the same sort of things and see if I can get it to happen as well. Thanks,

Josef

#8 lore raw
Quoted 10 lines · On Monday 16 May 2011 18:28:49 you wrote:
Quoted 6 lines · Whirm

On Monday 16 May 2011 16:11:19 Josef Bacik wrote:

Sorry yes, I meant this is how I managed to get the corrupted filesystem.

Ill try to break it again.

Oh ok perfect, yeah I will try and do the same sort of things and see if I can get it to happen as well. Thanks,

Great, btw, I tried to see if I can mount the corrupted filesystem with the patch you told me about applied, and it fails, here's the log:

http://pastebin.com/raw.php?i=4Kfv927B

That happens using 2.6.39-rc7+ from git with the following patch applied (its the debug patch and the possible fix you told me about):

fs/btrfs/free-space-cache.c +4 -1
index 63731a1..419f91f 100644
@@ -417,10 +417,19 @@ int load_free_space_cache(struct btrfs_fs_info *fs_info,
                        }
 
                        if (entry->type == BTRFS_FREE_SPACE_EXTENT) {
+                               printk(KERN_ERR "adding extent [%llu-%llu]\n",
+                                      e->offset, e->bytes);
                                spin_lock(&block_group->tree_lock);
                                ret = link_free_space(block_group, e);
                                spin_unlock(&block_group->tree_lock);
-                               BUG_ON(ret);
+                               if (ret) {
+                                       printk(KERN_ERR "Duplicate entries in 

"
+ "free space cache,
dumping\n");

+                                       kunmap(page);
+                                       unlock_page(page);
+                                       page_cache_release(page);
+                                       goto free_cache;
+                               }
                        } else {
                                e->bitmap = kzalloc(PAGE_CACHE_SIZE, 
GFP_NOFS);
                                if (!e->bitmap) {
@@ -430,7 +439,17 @@ int load_free_space_cache(struct btrfs_fs_info *fs_info,
                                        unlock_page(page);
                                        page_cache_release(page);
                                        goto free_cache;

+ if (ret) {
+ printk(KERN_ERR "Duplicate entries in
"
+ "free space cache, dumping\n");

+                                       kunmap(page);
+                                       unlock_page(page);
+                                       page_cache_release(page);
+                                       goto free_cache;
+                               }
                                }
+                               printk(KERN_ERR "adding bitmap [%llu-%llu]\n",
+                                      e->offset, e->bytes);
                                spin_lock(&block_group->tree_lock);
                                ret = link_free_space(block_group, e);
                                block_group->total_bitmaps++;
@@ -909,10 +928,16 @@ static int tree_insert_offset(struct rb_root *root, u64 
offset,
                         * logically.
                         */
                        if (bitmap) {

- WARN_ON(info->bitmap);
+ if (info->bitmap) {
+ WARN_ON_ONCE(1);
+ return -EEXIST;
+ }

                                p = &(*p)->rb_right;
                        } else {

- WARN_ON(!info->bitmap);
+ if (!info->bitmap) {
+ WARN_ON_ONCE(1);
+ return -EEXIST;
+ }

                                p = &(*p)->rb_left;
                        }
                }

Do you want me to try some other patch before I destroy the filesystem to make the other tests?

Thanks,

Signature · 1 line
Elric Milon
#9 lore raw
Quoted 12 lines · Elric Milon
Quoted 10 lines · On Monday 16 May 2011 18:28:49 you wrote:
Quoted 6 lines · Whirm

On Monday 16 May 2011 16:11:19 Josef Bacik wrote:

Sorry yes, I meant this is how I managed to get the corrupted filesystem.

Ill try to break it again.

Oh ok perfect, yeah I will try and do the same sort of things and see if I can get it to happen as well. Thanks,

Quoted 8 lines

Great, btw, I tried to see if I can mount the corrupted filesystem with the patch you told me about applied, and it fails, here's the log:

http://pastebin.com/raw.php?i=4Kfv927B

That happens using 2.6.39-rc7+ from git with the following patch applied (its the debug patch and the possible fix you told me about):

Ok so this is a different kind of corruption, wooo! Can you apply this debug patch and get me the output so we can try and fix this bit? Thanks,

Josef

fs/btrfs/file.c +3 -0
index cd5e82e..54a8df5 100644
@@ -365,6 +365,9 @@ next_slot:
 			extent_end = key.offset +
 				btrfs_file_extent_inline_len(leaf, fi);
 		} else {
+			printk(KERN_ERR "Weird entry at slot=%d, type=%lu\n",
+			       path->slots[0], extent_type);
+			btrfs_print_leaf(root, leaf);
 			WARN_ON(1);
 			extent_end = search_start;
 		}
fs/btrfs/inode.c +0 -0
index dc8fb2b..f951053 100644
@@ -145,6 +145,9 @@ static noinline int insert_inline_extent(struct

btrfs_trans_handle *trans,
inode_add_bytes(inode, size);
ret = btrfs_insert_empty_item(trans, root, path, &key,
datasize);
+ if (ret)
+ printk(KERN_ERR "ret=%d, inode=%d, start=%llu, size=%llu\n",
+ ret, inode->i_ino, start, size);
BUG_ON(ret);
if (ret) {
err = ret;

#10 lore raw
Quoted 19 lines · On Monday 23 May 2011 21:51:57 Josef Bacik wrote:
Quoted 18 lines · Elric Milon
Quoted 11 lines · On Monday 16 May 2011 18:28:49 you wrote:
Quoted 7 lines · Whirm

On Monday 16 May 2011 16:11:19 Josef Bacik wrote:

Sorry yes, I meant this is how I managed to get the corrupted filesystem.

Ill try to break it again.

Oh ok perfect, yeah I will try and do the same sort of things and see if I can get it to happen as well. Thanks,

Great, btw, I tried to see if I can mount the corrupted filesystem with the patch you told me about applied, and it fails, here's the log:

http://pastebin.com/raw.php?i=4Kfv927B

Quoted 9 lines
Quoted 4 lines

That happens using 2.6.39-rc7+ from git with the following patch applied (its

the debug patch and the possible fix you told me about):

Ok so this is a different kind of corruption, wooo! Can you apply this debug patch and get me the output so we can try and fix this bit? Thanks,

Josef

Sorry for the delay, here's the log:

http://pastebin.com/raw.php?i=zNqJVpA1

Thanks,

Signature · 1 line
Elric Milon
#11 lore raw
Quoted 27 lines · Elric Milon
Quoted 26 lines · On Monday 23 May 2011 21:51:57 Josef Bacik wrote:
Quoted 16 lines · Elric Milon
Quoted 11 lines · On Monday 16 May 2011 18:28:49 you wrote:
Quoted 7 lines · Whirm

On Monday 16 May 2011 16:11:19 Josef Bacik wrote:

Sorry yes, I meant this is how I managed to get the corrupted filesystem.

Ill try to break it again.

Oh ok perfect, yeah I will try and do the same sort of things and see if I can get it to happen as well. Thanks,

Great, btw, I tried to see if I can mount the corrupted filesystem with the patch you told me about applied, and it fails, here's the log:

Quoted 6 lines

http://pastebin.com/raw.php?i=4Kfv927B

That happens using 2.6.39-rc7+ from git with the following patch applied (its

the debug patch and the possible fix you told me about):

Ok so this is a different kind of corruption, wooo! Can you apply this debug patch and get me the output so we can try and fix this bit? Thanks,

Quoted 7 lines
Quoted 2 lines

Josef

Sorry for the delay, here's the log:

http://pastebin.com/raw.php?i=zNqJVpA1

Ok so you have a corrupt extent entry in your tree. I will come up with something so we deal with this case better than panicing. Thanks for running these debug patches,

Josef