kernel BUG at fs/btrfs/relocation.c:437!
Dear devs,
I tried to replace an SSD with bad S.M.A.R.T. status and since I don't have physical access to the server, I first wanted to remove it from the RAID 1 (which has 4 SSDs) and then erase it.
I ran "btrfs device delete /dev/sda2 /". After a while, the command terminated with a segfault and the system hung. I waited for 30 minutes. Fortunately, it could be resurrected with a hard reset.
dmesg, as this happened, reports that a block on a different SSD, on /dev/sdc can't be found.
See full backtrace here:
https://gist.github.com/vasyugan/340d9cd2292e3122c1d7773df718a234
Now I am afraid that if sda is just removed physically, then marked as degraded and swapped for a new SSD using the btrfs replace command, this might also go bad because of the block that can't be found.
Does any of you have advice on what to do? From the backtrace I don't even understand if the issue is a physical problem with sdc (whose S.M.A.R.T. values are just fine) or whether this is another btrfs bug and if you, if there is any way to work around it.
We are running Ubuntu 20.04, the kernel is 5.4.0-45-generic, Ubuntu's version number is: 5.4.0-45.49. It was released yesterday and was supposed to have a relocation relate bug fixed, see
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1889669
I suppose, this is a separate issue. Should I report a bug? If so, where?
Thanks a lot in advance for your support!!!
Johannes
Sorry for nagging, but since I got no replies and neither found any other reports that refer to line 437 of fs/btrfs/relocation.c I thought I should ask again.
This is about a system running Ubuntu 20.04, uname -a says:
Linux ida.rooot.de 5.4.0-45-generic #49-Ubuntu SMP Wed Aug 26 13:38:52 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
That's a kernel package that was released 31 Aug.
I can reproduce the bug by running btrfs balance start /home -musage=70
After a few seconds, the command terminates with a segfault, dmesg output is here:
https://gist.github.com/vasyugan/340d9cd2292e3122c1d7773df718a234
The core lines in dmesg seem to be:
[32387.616248] ------------[ cut here ]------------
[32387.616249] kernel BUG at fs/btrfs/relocation.c:437!
[32387.616271] invalid opcode: 0000 [#1] SMP PTI [32387.616180] BTRFS error (device sdc2): couldn't find block
(4853431877632) (level 1) in tree (19918) with key (986 96 124)
There is nothing about an IO error in dmesg, so to me as a layman this doesn't look like physical damage.
Yet when I repeat the command, it always names the same block.
I have of course run scrub, which as I understand it, does the same that an offline btrfs check would do, it finds no errors.
It would be awesome if anyone could take this up.
I have submitted the bug to bugzilla:
https://bugzilla.kernel.org/show_bug.cgi?id=209143
Cheers,
Johannes
Am 01.09.20 um 10:18 schrieb Johannes Rohr:
Quoted 38 lines
Dear devs,
I tried to replace an SSD with bad S.M.A.R.T. status and since I don't have physical access to the server, I first wanted to remove it from the RAID 1 (which has 4 SSDs) and then erase it.
I ran "btrfs device delete
/dev/sda2/". After a while, the command terminated with a segfault and the system hung. I waited for 30 minutes. Fortunately, it could be resurrected with a hard reset.dmesg, as this happened, reports that a block on a different SSD, on
/dev/sdccan't be found.See full backtrace here:
https://gist.github.com/vasyugan/340d9cd2292e3122c1d7773df718a234Now I am afraid that if sda is just removed physically, then marked as degraded and swapped for a new SSD using the btrfs replace command, this might also go bad because of the block that can't be found.
Does any of you have advice on what to do? From the backtrace I don't even understand if the issue is a physical problem with sdc (whose S.M.A.R.T. values are just fine) or whether this is another btrfs bug and if you, if there is any way to work around it.
We are running Ubuntu 20.04, the kernel is 5.4.0-45-generic, Ubuntu's version number is: 5.4.0-45.49. It was released yesterday and was supposed to have a relocation relate bug fixed, see
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1889669I suppose, this is a separate issue. Should I report a bug? If so, where?
Thanks a lot in advance for your support!!!
Johannes
Quoted 25 lines · Johannes Rohr
Dear devs,
I tried to replace an SSD with bad S.M.A.R.T. status and since I don't have physical access to the server, I first wanted to remove it from the RAID 1 (which has 4 SSDs) and then erase it.
I ran "btrfs device delete
/dev/sda2/". After a while, the command terminated with a segfault and the system hung. I waited for 30 minutes. Fortunately, it could be resurrected with a hard reset.dmesg, as this happened, reports that a block on a different SSD, on
/dev/sdccan't be found.See full backtrace here:
https://gist.github.com/vasyugan/340d9cd2292e3122c1d7773df718a234Now I am afraid that if sda is just removed physically, then marked as degraded and swapped for a new SSD using the btrfs replace command, this might also go bad because of the block that can't be found.
Does any of you have advice on what to do? From the backtrace I don't even understand if the issue is a physical problem with sdc (whose S.M.A.R.T. values are just fine) or whether this is another btrfs bug and if you, if there is any way to work around it.
Quoted 9 lines
We are running Ubuntu 20.04, the kernel is 5.4.0-45-generic, Ubuntu's version number is: 5.4.0-45.49. It was released yesterday and was supposed to have a relocation relate bug fixed, see
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1889669I suppose, this is a separate issue. Should I report a bug? If so, where?
Thanks a lot in advance for your support!!!
This error message sounds like a corrupted file system. However I fixed quite a few things in relocation recently, try a more recent kernel, 5.8 has all my recent fixes in this area. If not then I'd try btrfs check /dev/whatever to see if it complains about your fs being corrupted. Thanks,
Josef