Skip to contentSkip to conversation

parent transid verify failed on 604602368 wanted 840641 found 840639

DISCUSSION Dennis Schridde 3 participants 6 messages May 27, 2019 – May 28, 2019 Btrfs file system

lore
Share & tools
lore
#0 lore raw

Hi!

Yesterday I upgraded from Linux 5.1.1 (built with GCC 8.3.0) to Linux 5.1.4 (built with GCC 9.1.0). The next boot was extremely slow and the desktop environment (KDE Plasma) never really started, but got kind of stuck in the startup screen. So I switched to a VT and pressed ctrl+alt+del. The next boot stopped early with following message:

[T445] BTRFS: device label <...> devid 1 transid 840641 /dev/bcache0 [T599] BTRFS info (device bcache0): disk space caching is enabled [T599] BTRFS info (device bcache0): has skinny extents
[T599] BTRFS error (device bcache0): parent transid verify failed on 604602368 wanted 840641 found 840639
[T599] BTRFS error (device bcache0): open_ctree failed

How can I recover from this?

The filesystem should have several snapshots (created by snapper [1], on every boot and hourly). Will they be of any help recovering my data?

Best regards,
Dennis

[1]: http://snapper.io/

#1 lore raw
Quoted 18 lines · Dennis Schridde

Hi!

Yesterday I upgraded from Linux 5.1.1 (built with GCC 8.3.0) to Linux 5.1.4 (built with GCC 9.1.0). The next boot was extremely slow and the desktop environment (KDE Plasma) never really started, but got kind of stuck in the startup screen. So I switched to a VT and pressed ctrl+alt+del. The next boot stopped early with following message:

[T445] BTRFS: device label <...> devid 1 transid 840641 /dev/bcache0 [T599] BTRFS info (device bcache0): disk space caching is enabled [T599] BTRFS info (device bcache0): has skinny extents
[T599] BTRFS error (device bcache0): parent transid verify failed on 604602368 wanted 840641 found 840639
[T599] BTRFS error (device bcache0): open_ctree failed

How can I recover from this?

Quoted 7 lines

The filesystem should have several snapshots (created by snapper [1], on every boot and hourly). Will they be of any help recovering my data?

Best regards,
Dennis

[1]: http://snapper.io/

What happens if you revert to 5.1.1? That error suggests the super wants a newer transid than what exist on the filesystem, which suggests file system metadata was dropped. It's not certain from this information what caused that: device, or some layer in between like bcache, or Btrfs.

Signature · 1 line
Chris Murphy
#2 lore raw
Quoted 24 lines · On Dienstag, 28. Mai 2019 06:31:00 CEST Chris Murphy wrote:
Quoted 23 lines · Dennis Schridde

Yesterday I upgraded from Linux 5.1.1 (built with GCC 8.3.0) to Linux 5.1.4
(built with GCC 9.1.0). The next boot was extremely slow and the desktop environment (KDE Plasma) never really started, but got kind of stuck in the
startup screen. So I switched to a VT and pressed ctrl+alt+del. The next boot stopped early with following message:

[T445] BTRFS: device label <...> devid 1 transid 840641 /dev/bcache0 [T599] BTRFS info (device bcache0): disk space caching is enabled [T599] BTRFS info (device bcache0): has skinny extents [T599] BTRFS error (device bcache0): parent transid verify failed on 604602368 wanted 840641 found 840639
[T599] BTRFS error (device bcache0): open_ctree failed

How can I recover from this?

The filesystem should have several snapshots (created by snapper [1], on every boot and hourly). Will they be of any help recovering my data?

Best regards,
Dennis

Quoted 7 lines
Quoted 1 line

[1]: http://snapper.io/

What happens if you revert to 5.1.1? That error suggests the super wants a newer transid than what exist on the filesystem, which suggests file system metadata was dropped. It's not certain from this information what caused that: device, or some layer in between like bcache, or Btrfs.

The basic issue appears to be the same with Linux 5.1.1:

[T512] BTRFS: device label <...> devid 1 transid 840641 /dev/bcache0 [T587] BTRFS info (device bcache0): disk space caching is enabled [T587] BTRFS info (device bcache0): has skinny extents
[T587] BTRFS error (device bcache0): parent transid verify failed on 368672768 wanted 840529 found 840072
[T587] BTRFS error (device bcache0): failed to read block groups: -5 [T587] BTRFS error (device bcache0): open_ctree failed

I think on subsequent boots Linux 5.1.4 also pointed out other transids than "wanted 840641 found 840639", but I am not sure and I am hesitant to keep rebooting in case that means loosing more data.

--Dennis

#3 lore raw
In reply to Dennis Schridde

Dear Dennis,

I experienced the same, the problem is with bcache + gcc9. Immediately remove the cache device from the bcache, as it prevents more damages to your filesystem. After it downgrade gcc to 8 or avoid using bcache (with cache device) until the problem is solved by upstream.

Please see here for more information: https://bugzilla.kernel.org/show_bug.cgi?id=203573

This is a very serious issue I think, but in my case I could save all my files after reboot without bcache cache device (except 1 insignificant one), but I guess you might need backups (I hope you have).

Good luck,
László Szalma

ps: i use gentoo, not fedora by the way, so it is general issue with gcc, i read reports with gcc9 + 4.xxx kernels too.

2019. 05. 27. 23:33 keltezéssel, Dennis Schridde írta:

Quoted 24 lines

Hi!

Yesterday I upgraded from Linux 5.1.1 (built with GCC 8.3.0) to Linux 5.1.4 (built with GCC 9.1.0). The next boot was extremely slow and the desktop environment (KDE Plasma) never really started, but got kind of stuck in the startup screen. So I switched to a VT and pressed ctrl+alt+del. The next boot stopped early with following message:

[T445] BTRFS: device label <...> devid 1 transid 840641 /dev/bcache0 [T599] BTRFS info (device bcache0): disk space caching is enabled [T599] BTRFS info (device bcache0): has skinny extents
[T599] BTRFS error (device bcache0): parent transid verify failed on 604602368 wanted 840641 found 840639
[T599] BTRFS error (device bcache0): open_ctree failed

How can I recover from this?

The filesystem should have several snapshots (created by snapper [1], on every boot and hourly). Will they be of any help recovering my data?

Best regards,
Dennis

[1]: http://snapper.io/

#4 lore raw

Dear Szalma!

Thank you for the information.

Quoted 11 lines · On Dienstag, 28. Mai 2019 08:40:40 CEST Szalma László wrote:

I experienced the same, the problem is with bcache + gcc9. Immediately remove the cache device from the bcache, as it prevents more damages to your filesystem. After it downgrade gcc to 8 or avoid using bcache (with cache device) until the problem is solved by upstream.

Please see here for more information:
https://bugzilla.kernel.org/show_bug.cgi?id=203573

This is a very serious issue I think, but in my case I could save all my files after reboot without bcache cache device (except 1 insignificant one), but I guess you might need backups (I hope you have).

I did the following so far:

readlink /sys/fs/bcache/*/cache0/set | sed 's,.*/,,' > /sys/block/bcache0/ bcache/detach

How should I proceed from here on the path that you took to recover your system?

--Dennis

#5 lore raw

Dear Dennis,

Without the bcache cache device, I was able to mount the file system ( root ) for read-only. I recreated it on the same LVM volume, and rsync-ed the files. The files I lost (i/o error because checksum error) was not important. Your case might be different. But without the cache, additional corruption stopped happening, so I think it is safe to recover your files with the gcc9 + 5.1 kernel (on bcache0) - if cache device is not present. If you cannot mount the fs, you can try the standard btrfs recovery methods (mount read only) or btrfsck (please be aware you can make things worse with btrfsck, so if the data is important backup the device as byte stream for new chance)

I want to mention I had another corruption (btrfs checksum error), but it was simply not there after reboot without bcacha cache device. This means the corruption that time was not written back to the backing device, so I haven't lost data that time. I think I was lucky.

László Szalma

2019. 05. 28. 14:36 keltezéssel, Dennis Schridde írta:

Quoted 25 lines

Dear Szalma!

Thank you for the information.

Quoted 11 lines · On Dienstag, 28. Mai 2019 08:40:40 CEST Szalma László wrote:

I experienced the same, the problem is with bcache + gcc9. Immediately remove the cache device from the bcache, as it prevents more damages to your filesystem. After it downgrade gcc to 8 or avoid using bcache (with cache device) until the problem is solved by upstream.

Please see here for more information:
https://bugzilla.kernel.org/show_bug.cgi?id=203573

This is a very serious issue I think, but in my case I could save all my files after reboot without bcache cache device (except 1 insignificant one), but I guess you might need backups (I hope you have).

I did the following so far:

readlink /sys/fs/bcache/*/cache0/set | sed 's,.*/,,' > /sys/block/bcache0/ bcache/detach

How should I proceed from here on the path that you took to recover your system?

--Dennis