Kernel Bug after corrupted filesystem
Hi,
I am experimenting with nilfs2 for a while now, because I like the Idea of this filesystem. The filesystem I have problems with now is a more complex problem. I used just for testing purpose a file on an ext3 filesystem as a cryptoloop device and I used nilfs2 on the loopback device. It worked for about half a year without a problem. But I have no Idea what was actualy going wrong (ext3 problem, cryptoloop problem, nilfs problem?) and I don't know if a filesystem under normal conditations get corrupted like this (actualy no real data loss either, because almost all files existed elswhere)
But when I now try to mount the nilfs2 filesystem it does not work. No result from the mount command, but the attached in dmsg. The system is still useable, but I am not able to shut it down (have to hold power button 3 seconds). And I am not able to unload the nilfs module either (is in use)
Hope this helps to further improve nilfs2.
robinx99
_______________________________________________
users mailing list
[email protected]
https://www.nilfs.org/mailman/listinfo/users
A message that is not in the archive (sent off-list or not yet received): [email protected]
Hi,
Quoted 6 lines · On Tue, 30 Dec 2008 14:19:53 +0100, [email protected] wrote:
Hi,
I am experimenting with nilfs2 for a while now, because I like the Idea of this filesystem. The filesystem I have problems with now is a more complex problem. I used just for testing purpose a file on an ext3 filesystem as a cryptoloop device and I used nilfs2 on the loopback device. It worked for about half a year without a
Quoted 8 lines
problem. But I have no Idea what was actualy going wrong (ext3 problem, cryptoloop problem, nilfs problem?) and I don't know if a filesystem under normal conditations get corrupted like this (actualy no real data loss either, because almost all files existed elswhere)
But when I now try to mount the nilfs2 filesystem it does not work. No result from the mount command, but the attached in dmsg.
Thank you for reporting the problem.
Again, it seems that a btree corruption happened.
Sorry, but it's hard to track the cause directly from the stack dump.
According to the log, the write barrier is disabled (it may be a restriction of cryptoloop). If you met an unexpected system down before mounting nilfs, please let me know.
Quoted 4 lines
The system is still useable, but I am not able to shut it down (have to hold power button 3 seconds). And I am not able to unload the nilfs module either (is in use) Hope this helps to further improve nilfs2. robinx99
I've modified btree lookup routines to avoid system freeze and allow clean termination; I pushed it to our git repository.
Regards,
Ryusuke
A message that is not in the archive (sent off-list or not yet received): [email protected]
Quoted 3 lines · Ryusuke Konishi
According to the log, the write barrier is disabled (it may be a restriction of cryptoloop). If you met an unexpected system down before mounting nilfs, please let me know.
Uh-oh, it's my misconception. forget it.
Ryusuke
-------- Original-Nachricht --------
Quoted 9 lines
Datum: Wed, 31 Dec 2008 11:53:54 +0900 (JST)
Von: Ryusuke Konishi <[email protected]>
An: [email protected], [email protected]
Betreff: Re: [NILFS users] Kernel Bug after corrupted filesystemThank you for reporting the problem.
Again, it seems that a btree corruption happened.
Sorry, but it's hard to track the cause directly from the stack dump.
Is there anything else I could provide? That might help to track down problems
Quoted 4 lines
I've modified btree lookup routines to avoid system freeze and allow clean termination; I pushed it to our git repository.
That looks much better the module is now unloadable after failing to mount. It now gives an error message when trying to mount
mount.nilfs2: Error while mounting /dev/loop1 on /mnt/test: Input/output error
and now there are also some other things in dmsg I will atach this to the mail in hope that it helps
_______________________________________________
users mailing list
[email protected]
https://www.nilfs.org/mailman/listinfo/users
A message that is not in the archive (sent off-list or not yet received): [email protected]
Hi,
Quoted 5 lines · Dennis Peiler
Quoted 3 lines
Again, it seems that a btree corruption happened.
Sorry, but it's hard to track the cause directly from the stack dump.Is there anything else I could provide? That might help to track down problems
Thanks.
Well, could you try to get some information with a debug build module?
First, pull the latest code from our git repository.
The debug build module can be built by commenting out the following line of the fs/Makefile.
#CONFIG_NILFS_DEBUG=y
Then, set debug levels before mounting as follows:
# modprobe nilfs2 # echo "-vvv fs -vvv recovery -vvv mdt -vvv btnode -vv dat" > /proc/fs/nilfs2/debug_option # mount -t nilfs2 /dev/xxx /test
This will log some more information in syslog including block addresses.
What I'd like to see next is content of key disk blocks, but first we need to know rough situation.
Regards,
Ryusuke
A message that is not in the archive (sent off-list or not yet received): [email protected]
Quoted 18 lines
Thanks.
Well, could you try to get some information with a debug build module?First, pull the latest code from our git repository.
The debug build module can be built by commenting out the following line of the fs/Makefile.
#CONFIG_NILFS_DEBUG=y
Then, set debug levels before mounting as follows:
# modprobe nilfs2 # echo "-vvv fs -vvv recovery -vvv mdt -vvv btnode -vv dat" > /proc/fs/nilfs2/debug_option # mount -t nilfs2 /dev/xxx /test
Quoted 5 lines
This will log some more information in syslog including block addresses.
What I'd like to see next is content of key disk blocks, but first we need to know rough situation.
Hi and happy new year,
ok have done that and atached the log hope this information is usefull and I hope this helps to further improve nilfs
Robinx99
_______________________________________________
users mailing list
[email protected]
https://www.nilfs.org/mailman/listinfo/users
Quoted 18 lines
Thanks.
Well, could you try to get some information with a debug build module?First, pull the latest code from our git repository.
The debug build module can be built by commenting out the following line of the fs/Makefile.
#CONFIG_NILFS_DEBUG=y
Then, set debug levels before mounting as follows:
# modprobe nilfs2 # echo "-vvv fs -vvv recovery -vvv mdt -vvv btnode -vv dat" > /proc/fs/nilfs2/debug_option # mount -t nilfs2 /dev/xxx /test
Quoted 5 lines
This will log some more information in syslog including block addresses.
What I'd like to see next is content of key disk blocks, but first we need to know rough situation.
Hi and happy new year,
ok have done that and atached the log hope this information is usefull and I hope this helps to further improve nilfs
Robinx99
_______________________________________________
users mailing list
[email protected]
https://www.nilfs.org/mailman/listinfo/users
A message that is not in the archive (sent off-list or not yet received): [email protected]
Happy new year,
Quoted 4 lines · On Thu, 01 Jan 2009 10:31:21 +0100, [email protected] wrote:
Hi and happy new year,
ok have done that and atached the log hope this information is usefull and I hope this helps to further improve nilfsRobinx99
Thanks for your help.
I have another request:
# dumpseg /dev/loop1 12
- This will dump the summary of the segment in question
# dd if=/dev/loop1 bs=4k skip=24576 count=2 | hd
- This will dump the (broken) root block of the file system
According to your log, the btree root of DAT file (i.e. a table file to translate disk addresses) seems inconsistent. It might not be updated properly when GC moved its blocks. I suspect that it was caused by some sort of GC problem.
And, your nilfs partition seems to have shut down uncleanly before this mount error repeatedly happened. Do you remember what the first trouble was like?
Regards,
Ryusuke
A message that is not in the archive (sent off-list or not yet received): [email protected]
Oops didn't send the mail to the mailinglist, so I will resend. Hi
Quoted 5 lines
Thanks for your help.
I have another request:# dumpseg /dev/loop1 12
root@robin-laptop:/tmp# dumpseg /dev/loop1 12
segment: segnum = 12
partial segment
creation time = 2008-12-26 11:45:06
nfinfo = 0Quoted 7 lines
- This will dump the summary of the segment in question
# dd if=/dev/loop1 bs=4k skip=24576 count=2 | hd- This will dump the (broken) root block of the file system
Ok I atached it to the email bzip2 compressed
Quoted 9 lines
According to your log, the btree root of DAT file (i.e. a table file to translate disk addresses) seems inconsistent. It might not be updated properly when GC moved its blocks. I suspect that it was caused by some sort of GC problem.
And, your nilfs partition seems to have shut down uncleanly before this mount error repeatedly happened. Do you remember what the first trouble was like?
Actualy I have no real idea. Didn't start the computer for some days over christmas, befor christmas I could use the filesystem withoul any trouble, and on december 26 I was unable to mount it.
I actualy quite often moved files of the nilfs filesystem to an USB stick. Using comands like
"mv /mnt/nilfs/directory/*.avi /mnt/usb && poweroff". So there were some shutdowns I didn't watch, and I was thinking that the normal shutdown scripts should unmount every mounted filesystem. Well the computer was definitv off, but I have no idea if it could be possible that something importend was stoped to soon when it came to the point "sending all procecess the term signal"
Quoted 3 lines
Regards,
Ryusuke
robinx99
_______________________________________________
users mailing list
[email protected]
https://www.nilfs.org/mailman/listinfo/users
A message that is not in the archive (sent off-list or not yet received): [email protected]
Hi,
Quoted 6 lines · On Sat, 03 Jan 2009 08:54:38 +0100, [email protected] wrote:
Oops didn't send the mail to the mailinglist, so I will resend. Hi
Quoted 4 lines
Thanks for your help.
I have another request:
Quoted 7 lines
Quoted 1 line
# dumpseg /dev/loop1 12root@robin-laptop:/tmp# dumpseg /dev/loop1 12 segment: segnum = 12 partial segment creation time = 2008-12-26 11:45:06 nfinfo = 0
Uh, that was not good.
Quoted 7 lines
Quoted 6 lines
- This will dump the summary of the segment in question
# dd if=/dev/loop1 bs=4k skip=24576 count=2 | hd- This will dump the (broken) root block of the file system
Ok I atached it to the email bzip2 compressed
This was informative.
The last segment was that written for GC and I actually confirmed that the inconsistency of DAT root. It must be a bug of one of GC, btree, or the DAT file.
Quoted 21 lines
Quoted 9 lines
According to your log, the btree root of DAT file (i.e. a table file to translate disk addresses) seems inconsistent. It might not be updated properly when GC moved its blocks. I suspect that it was caused by some sort of GC problem.
And, your nilfs partition seems to have shut down uncleanly before this mount error repeatedly happened. Do you remember what the first trouble was like?
Actualy I have no real idea. Didn't start the computer for some days over christmas, befor christmas I could use the filesystem withoul any trouble, and on december 26 I was unable to mount it. I actualy quite often moved files of the nilfs filesystem to an USB stick. Using comands like "mv
/mnt/nilfs/directory/*.avi/mnt/usb&& poweroff". So there were some shutdowns I didn't watch, and I was thinking that the normal shutdown scripts should unmount every mounted filesystem. Well the computer was definitv off, but I have no idea if it could be possible that something importend was stoped to soon when it came to the point "sending all procecess the term signal"
Quoted 1 line
robinx99
Thank you once again.
There is no doubt that this corruption was caused by a bug of NILFS because NILFS verifies validity of every segment with checksums.
If NILFS doesn't have any bugs, and write barrier is assured, the file system must be recovered propery even if you have reset the PC during GC (unless hardware failures occur, of course).
So we have to review code to find out the cause, and I believe your information will help this.
Thanks,
Ryusuke