New: Data loss on VirtualBox VMs
https://bugzilla.kernel.org/show_bug.cgi?id=216012
Bug ID: 216012
Summary: Data loss on VirtualBox VMs
Product: File System
Version: 2.5
Kernel Version: Linux ubuntu 5.13.0-35-generic #40-Ubuntu SMP Mon Mar
7 08:03:10 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Hardware: Intel
OS: Linux
Tree: Mainline
Status: NEW
Severity: high
Priority: P1
Component: ext4
Assignee: [email protected]
Reporter: [email protected]
Regression: No
From recent kernel and/or vbox versions forward, I've started to experience data loss and/or filesystem corruption on a regular enough basis that I've enabled fsck on every reboot. Mac OSX hosted VM.
It's not clear to me what the source of the problem is, but it may not be ext4 itself vs some element of the block device drivers and/or related hypervisor facilities.
Message from syslogd@ubuntu at May 21 15:56:49 ...
kernel:[1958392.548682] EXT4-fs (dm-0): failed to convert unwritten extents to
written extents -- potential data loss! (inode 259291, error -30)
Message from syslogd@ubuntu at May 21 15:56:50 ...
kernel:[1958392.830326] EXT4-fs (dm-0): failed to convert unwritten extents to
written extents -- potential data loss! (inode 262111, error -30)
Signature · 4 lines
You may reply to this email to add a comment. You are receiving this mail because: You are watching the assignee of the bug.
[Bug 216012] Data loss on VirtualBox VMs
https://bugzilla.kernel.org/show_bug.cgi?id=216012
Artem S. Tashkinov ([email protected]) changed:
What |Removed |Added
----------------------------------------------------------------------------
Status|NEW |RESOLVED
Resolution|--- |ANSWERED--- Comment #1 from Artem S. Tashkinov ([email protected]) --- There's zero info as to how it's all happening. If you need help you need to let people understand how it can be reproduced.
Lastly, you're using the Ubuntu kernel and bugs against it need to be reported
here: https://bugs.launchpad.net/ubuntu
Signature · 4 lines
You may reply to this email to add a comment. You are receiving this mail because: You are watching the assignee of the bug.
[Bug 216012] Data loss on VirtualBox VMs
In reply to [email protected]https://bugzilla.kernel.org/show_bug.cgi?id=216012
AA ([email protected]) changed:
What |Removed |Added
----------------------------------------------------------------------------
Status|RESOLVED |REOPENED
Resolution|ANSWERED |------ Comment #2 from AA ([email protected]) ---
I provided the information I had. You're not talking to a complete noob, Artem. If this issue was deterministically reproducible, you can be assured I'd have provided the steps to reproduce it and any additional details.
There is some multi-factor problem going on here, and frankly, I don't even know where to start in order to determine the cause! This is a linux virtual machine hosted by virtualbox on mac os x.
Since the error is reported by ext4-fs, and I did not immediately find other errors reported, I reported the ext4-fs errors to ext4.
One possibility is that this occurs when the os x system is under relatively high load. Under 5x kernel versions on this VB host I have also seen some occasional CPU X stuck for 22s messages. However these messages have not been correlated with the ext4 errors.
If anyone has knowledge as to the conditions that can cause this ext4 error message, I will try to dig deeper next time it happens. The combination of factors seems to point to core issues in the VirtualBox drivers and/or virtualization interfaces but I'm just not sure. Maybe the virtual block device driver bubbles up a timeout of some kind as an unrecoverable write to ext4 via whatever kernel interface is being used to write fs blocks???
If you still feel ubuntu is the right place to chase down the issue I'll go there. I've not gotten any traction from reporting the CPU stuck issue to Oracle.
Signature · 4 lines
You may reply to this email to add a comment. You are receiving this mail because: You are watching the assignee of the bug.
[Bug 216012] Data loss on VirtualBox VMs
In reply to [email protected]https://bugzilla.kernel.org/show_bug.cgi?id=216012
Theodore Tso ([email protected]) changed:
What |Removed |Added
----------------------------------------------------------------------------
Status|REOPENED |RESOLVED
CC| |[email protected]
Resolution|--- |INSUFFICIENT_DATA--- Comment #3 from Theodore Tso ([email protected]) --- Error -30 is EROFS in this message:
EXT4-fs (dm-0): failed to convert unwritten extents to written extents -- potential data loss! (inode 259291, error -30)
This typically means that *before* this point, the ext4 file system detected an inconsistency, and the file system was set up to remount the file system read-only when an Ext4. So there would be an "EXT4-fs error" message. For example, you can trigger this behaviour like this:
root@kvm-xfstests:~# tune2fs -e remount-ro /dev/vdc tune2fs 1.46.4-orphan-file-02827d06 (4-Nov-2021) Setting error behavior to 2 root@kvm-xfstests:~# mount /dev/vdc /vdc [ 83.142333] EXT4-fs (vdc): mounted filesystem with ordered data mode. Quota mode: none. root@kvm-xfstests:~# echo test-corruption-handling > /sys/fs/ext4/vdc/trigger_fs_error [ 91.189272] EXT4-fs error (device vdc): trigger_test_error:126: comm bash: test-corruption-handling [ 91.190375] Aborting journal on device vdc-8. [ 91.193756] EXT4-fs (vdc): Remounting filesystem read-only root@kvm-xfstests:~#
Typically, when this happens, in 99.9999% of the time, it's caused by an I/O
error. In a hypervisor situation, that includes a potential hypervisor bug.
In any case, without any other evidence to the contrary, it's probably not an ext4 bug. And even if it was, unless you can replicate the bug on an upstream
kernel, the proper place to report it is with Canonical. After all, that's
why you've paid $$$ for a support contract with Canonical for Ubuntu, right?
:-) And depending on Canonical's support contract, they might or might not
be willing to track down a Virtualbox bug unless you've paid for a more
comprehensive support contract. In any case, upstream developers don't have
time to chase down something like this, especially since the probabilities are extremely high that it's not an upstream kernel issue.
Signature · 4 lines
You may reply to this email to add a comment. You are receiving this mail because: You are watching the assignee of the bug.
[Bug 216012] Data loss on VirtualBox VMs
In reply to [email protected]https://bugzilla.kernel.org/show_bug.cgi?id=216012
Artem S. Tashkinov ([email protected]) changed:
What |Removed |Added
----------------------------------------------------------------------------
Kernel Version|Linux ubuntu |Ubuntu 5.13.0-35-generic
|5.13.0-35-generic |
|#40-Ubuntu SMP Mon Mar 7 |
|08:03:10 UTC 2022 x86_64 |
|x86_64 x86_64 GNU/Linux |--- Comment #4 from Artem S. Tashkinov ([email protected]) ---
The kernel you're using has a solid ext4 filesystem driver as there are no similar reports on the net which could mean that you're having issue with either MacOS or VirtualBox or both and I'm sorry to break it to you but you're on your own.
It makes sense to try this VM on another PC or recreate the VM from scratch.
Signature · 4 lines
You may reply to this email to add a comment. You are receiving this mail because: You are watching the assignee of the bug.