Skip to contentSkip to conversation

postgresql activity causes crash, 2.6.36.1

DISCUSSION Brian Neu 2 participants 6 messages Nov 7, 2010 – Nov 11, 2010 2 messages not in the archive Btrfs file system

lore
Share & tools
lore

Brian Neu

author
#0 lore raw

After a successful db startup, this crash was reproducible every time with db activity, or possibly just time. The filesystem is a raid1 and even after a reformat and restore from backup, the crash remains:

Label: slash uuid: a9730db3-a475-45bb-af94-a873e235114b

        Total devices 2 FS bytes used 39.04GB
        devid    2 size 97.66GB used 48.51GB path /dev/sdb2
        devid    1 size 97.66GB used 48.53GB path /dev/sda2

mounted as:

# mount |grep sdb2
/dev/sdb2 on / type btrfs (rw,noatime)

This is transcribed from the screen, so forgive type-o's and "<blah>" . Let
me know if I skipped something important.
(every line starts with [27163.079010] )

Call trace:
__btrfs_submit_bio_done+0x1b/0x1d [btrfs]
run_one_async_done+0x94/0x98 [btrfs]
run_ordered_completions++0x78/0xaa [btrfs]
worker_loop+0x199/0x4ec [btrfs]
kthread+0x0d/0xa5
kernel_thread_helper+0x4/0x10
? restore_args+0x0/0x30
? kthread+0x0/0xa5
? kernel_thread_helper+0x0/0x10
Code: 74 04 <blah blah>
RIP [<ffff<blah> 7>] btrfs_map_bio+0x88/0x1c9
RSP <ffb<blah>>
---[ end trace f8<blah> ]---
#1 lore raw

Excerpts from Brian Neu's message of 2010-11-06 21:08:59 -0400:

Quoted 9 lines

After a successful db startup, this crash was reproducible every time with db activity, or possibly just time. The filesystem is a raid1 and even after a reformat and restore from backup, the crash remains:

Label: slash uuid: a9730db3-a475-45bb-af94-a873e235114b

        Total devices 2 FS bytes used 39.04GB
        devid    2 size 97.66GB used 48.51GB path /dev/sdb2
        devid    1 size 97.66GB used 48.53GB path /dev/sda2
Quoted 8 lines

mounted as:

# mount |grep sdb2
/dev/sdb2 on / type btrfs (rw,noatime)

This is transcribed from the screen, so forgive type-o's and "<blah>" . Let
me know if I skipped something important.

Ok, could you please take a picture of the crash? We'll need all the lines.

Which kernel are you on?

-chris

Quoted 17 lines

(every line starts with [27163.079010] )

Call trace:
__btrfs_submit_bio_done+0x1b/0x1d [btrfs]
run_one_async_done+0x94/0x98 [btrfs]
run_ordered_completions++0x78/0xaa [btrfs]
worker_loop+0x199/0x4ec [btrfs]
kthread+0x0d/0xa5
kernel_thread_helper+0x4/0x10
? restore_args+0x0/0x30
? kthread+0x0/0xa5
? kernel_thread_helper+0x0/0x10
Code: 74 04 <blah blah>
RIP [<ffff<blah> 7>] btrfs_map_bio+0x88/0x1c9
RSP <ffb<blah>>
---[ end trace f8<blah> ]---

A message that is not in the archive (sent off-list or not yet received): [email protected]

#3 lore raw

Excerpts from Brian Neu's message of 2010-11-08 12:52:21 -0500:

Quoted 4 lines
Quoted 2 lines

Ok, could you please take a picture of the crash? We'll need all the lines.

I really hope you mean "picture" as in photo of the screen, and I'm not committing an egregious list faux pas by attaching this.

No, this is perfect ;) The only problem is that we're missing the stuff at the very top of the oops. That's the part that explains why it oopsed. The stuff in this png is great, but it only shows where it oopsed.

-chris

A message that is not in the archive (sent off-list or not yet received): [email protected]

#5 lore raw

Great, this is a known bug in the O_DIRECT code. I'll fix it up tomorrow morning.

The problem is that we're allowing bios that are too large for btrfs to deal with. It doesn't show any corruption in the FS.

-chris

Excerpts from Brian Neu's message of 2010-11-08 17:50:17 -0500:

Quoted 14 lines
Quoted 7 lines

No, this is perfect ;) The only problem is that we're missing the stuff

at the very top of the oops. That's the part that explains why it oopsed. The stuff in this png is great, but it only shows where it oopsed.

-chris

I changed the kernel line in grub.conf for 1280x1024, rebooted, and orchestrated another crash for you. This time I noted that db activity was necessary to cause the crash.

not my steadiest camera work, but still legible

author
#6 lore raw

----- Original Message ----

Quoted 4 lines

From: Chris Mason <[email protected]>

Great, this is a known bug in the O_DIRECT code. I'll fix it up tomorrow morning.

At the risk of sounding like the non-contributing, annoying moocher that I am, I thought I'd check on the status of this bug.

#7 lore raw

Excerpts from Brian Neu's message of 2010-11-11 17:05:13 -0500:

Quoted 10 lines

----- Original Message ----

Quoted 4 lines

From: Chris Mason <[email protected]>

Great, this is a known bug in the O_DIRECT code. I'll fix it up tomorrow morning.

At the risk of sounding like the non-contributing, annoying moocher that I am, I thought I'd check on the status of this bug.

Not at all, sorry for the delay I got caught up in hunting for the transparent hugepages bug (which wasn't in transparent hugepages in the end).

I'm working on the O_DIRECT stuff tonight.

-chris