Skip to contentSkip to conversation

Moving ext3 journal file

DISCUSSION RaúlNúñez de Arenas Coronado 10 participants 20 messages Nov 23, 2001 – Nov 25, 2001

lore
Share & tools
lore
#0 lore raw

Hello all :))

Is there any problem on moving the /.journal file (even renaming
it) so it doesn't lives on the root? I mean, maintaining its inode number, of course ;))

Anyway, ext3 shouldn't (just an idea) show the journal as a
normal file. It may add some load on the kernel, because the inode number should be compared with that of the journal every time a file is accessed, but it's just a suggestion ;))

And, BTW, ext3 works *really* great :))))

Ra�l

#1 lore raw
Quoted 3 lines · RaúlNúñez de Arenas Coronado

Is there any problem on moving the /.journal file (even renaming
it) so it doesn't lives on the root? I mean, maintaining its inode number, of course ;))

The name is irrelevant. The kernel only accesses the journal by inode number, and only once at boot time. Rather than "renaming" it and causing problems, you should just unmount the filesystem, and run "e2fsck -f /dev/hdX" (with e2fsck 1.25) and it will hide it for you.

Quoted 4 lines

Anyway, ext3 shouldn't (just an idea) show the journal as a
normal file. It may add some load on the kernel, because the inode number should be compared with that of the journal every time a file is accessed, but it's just a suggestion ;))

???? This doesn't make sense. Having the .journal file places no load on the system. OK, when you do "ls -a /" it has to list an extra file, but that is so little work as to be unnoticable. Even with a hidden journal, it is still in an inode with an inode number, it's just not in a directory anywhere.

As to "comparing the inode number to that of the journal every time a file is accessed" it appears you just don't understand how file access works in the kernel.

Cheers, Andreas
--
Andreas Dilger
http://sourceforge.net/projects/ext2resize/
http://www-mddsp.enel.ucalgary.ca/People/adilger/

#2 lore raw
In reply to RaúlNúñez de Arenas Coronado

Hi ;),

Quoted 8 lines

Is there any problem on moving the /.journal file (even renaming
it) so it doesn't lives on the root? I mean, maintaining its inode number, of course ;))

Anyway, ext3 shouldn't (just an idea) show the journal as a
normal file. It may add some load on the kernel, because the inode number should be compared with that of the journal every time a file is accessed, but it's just a suggestion ;))

AFAIK the .journal it's visible only when you convert an ext2 to an ext3 filesystem on a mounted partition, it was a problem with 2.4.10 kernel version, but i'm not sure if posterior releases also show this behavior. Anyway you can solve it recreating a new journal (remount it to ext2 before doing this):

'chattr -i /.journal;rm /.journal;tune2fs -j /dev/whatever'

David Gómez

"The question of whether computers can think is just like the question of
whether submarines can swim." -- Edsger W. Dijkstra

#3 lore raw
In reply to RaúlNúñez de Arenas Coronado
Quoted 7 lines · David Gómez

AFAIK the .journal it's visible only when you convert an ext2 to an ext3 filesystem on a mounted partition, it was a problem with 2.4.10 kernel version, but i'm not sure if posterior releases also show this behavior. Anyway you can solve it recreating a new journal (remount it to ext2 before doing this):

'chattr -i /.journal;rm /.journal;tune2fs -j /dev/whatever'

Don't do that. That is only good if the filesystem thinks that there is no journal, or it is using a hidden inode for the journal (i.e. if you run "tune2fs -l /dev/whatever" and it doesn't have "has_journal" listed in the filesystem features (this is what happened with 2.4.10). Otherwise, you will delete your real journal, tune2fs will complain, and then you will need to run e2fsck to clean up after yourself, before re-creating your journal again.

If you have a filesystem with a .journal file, and you want to "hide" it, just run e2fsck 1.25 while the filesystem is unmounted, and it will do it for you. If you don't want to have a .journal in the first place, run tune2fs -j while the filesystem is unmounted.

Cheers, Andreas
--
Andreas Dilger
http://sourceforge.net/projects/ext2resize/
http://www-mddsp.enel.ucalgary.ca/People/adilger/

#4 lore raw
Quoted 8 lines · Mike Fedyk
Quoted 7 lines · Mike Castle
Quoted 2 lines · Mike Fedyk

The newest e2fsck (1.25) will hide the file for you, just like it would be if the conversion was done on an unmounted FS.

Actually, to re-iterate a recent point: No, it won't.

Quoted 9 lines
Quoted 2 lines

The next version of e2fsck should. But 1.25 does NOT do that.

Ok...

Strange that aDilger has been one to say that 1.25 does hide the file...

Since I don't know myself, I'll let others say for sure.

Yep, just finished reading the thread.

You may disregard...

MF

#5 lore raw
In reply to RaúlNúñez de Arenas Coronado
Followup to:  <[email protected]>
By author:    Andreas Dilger <[email protected]>
In newsgroup: linux.dev.kernel
Quoted 9 lines

Don't do that. That is only good if the filesystem thinks that there is no journal, or it is using a hidden inode for the journal (i.e. if you run "tune2fs -l /dev/whatever" and it doesn't have "has_journal" listed in the filesystem features (this is what happened with 2.4.10). Otherwise, you will delete your real journal, tune2fs will complain, and then you will need to run e2fsck to clean up after yourself, before re-creating your journal again.

Quoted 5 lines

If you have a filesystem with a .journal file, and you want to "hide" it, just run e2fsck 1.25 while the filesystem is unmounted, and it will do it for you. If you don't want to have a .journal in the first place, run tune2fs -j while the filesystem is unmounted.

This is all fine and good except for the root partition (I'm pleased to hear that e2fsck 1.25 will move the journal to the hidden inode for non-root partitions.) It would be nice if this was done automagically by the mounting code instead of by fsck; that way migration would truly be painless.

-hpa

Signature · 3 lines
<[email protected]> at work, <[email protected]> in private!
"Unix gives you enough rope to shoot yourself in the foot."
http://www.zytor.com/~hpa/puzzle.txt	<[email protected]>
#6 lore raw
In reply to RaúlNúñez de Arenas Coronado
Quoted 14 lines · H. Peter Anvin

By author: Andreas Dilger <[email protected]>

Quoted 8 lines

Don't do that. That is only good if the filesystem thinks that there is no journal, or it is using a hidden inode for the journal (i.e. if you run "tune2fs -l /dev/whatever" and it doesn't have "has_journal" listed in the filesystem features (this is what happened with 2.4.10). Otherwise, you will delete your real journal, tune2fs will complain, and then you will need to run e2fsck to clean up after yourself, before re-creating your journal again.

Quoted 4 lines

If you have a filesystem with a .journal file, and you want to "hide" it, just run e2fsck 1.25 while the filesystem is unmounted, and it will do it for you. If you don't want to have a .journal in the first place, run tune2fs -j while the filesystem is unmounted.

Quoted 5 lines

This is all fine and good except for the root partition (I'm pleased to hear that e2fsck 1.25 will move the journal to the hidden inode for non-root partitions.) It would be nice if this was done automagically by the mounting code instead of by fsck; that way migration would truly be painless.

Hmm, it could be added into the in-kernel ext3 journal recovery code, maybe, possibly, but it would be ugly I think. I really don't see it as being a problem (other than a purely cosmetic one) to have a .journal file in your root fs. I lived with these for a couple of years OK (even some of the early ext3 tools called them "journal.dat" (i.e. not a leading dot, ugh).

In 2.5 it would be easy (and preferrable for other reasons) to have e2fsck run from the initramfs on the root fs before it is mounted. That solves the problem nicely without adding bloat into the kernel. We could even remove the in-kernel journal recovery at that point if we thought that the initramfs was a reliable environment to host _critical_ boot tools.

Cheers, Andreas
--
Andreas Dilger
http://sourceforge.net/projects/ext2resize/
http://www-mddsp.enel.ucalgary.ca/People/adilger/

#7 lore raw
In reply to RaúlNúñez de Arenas Coronado
Followup to:  <[email protected]>
By author:    Andreas Dilger <[email protected]>
In newsgroup: linux.dev.kernel
Quoted 7 lines

Hmm, it could be added into the in-kernel ext3 journal recovery code, maybe, possibly, but it would be ugly I think. I really don't see it as being a problem (other than a purely cosmetic one) to have a .journal file in your root fs. I lived with these for a couple of years OK (even some of the early ext3 tools called them "journal.dat" (i.e. not a leading dot, ugh).

It's either a cosmetic problem or a really problematic one, depending on how closely controlled that particular part of the namespace is. The .journal file could potentially try to be copied, backed up, you name it. Now, since this is now only an issue for the root partition, this is much less of a problem (the risk for namespace problems in / are much less.)

Quoted 5 lines

In 2.5 it would be easy (and preferrable for other reasons) to have e2fsck run from the initramfs on the root fs before it is mounted. That solves the problem nicely without adding bloat into the kernel. We could even remove the in-kernel journal recovery at that point if we thought that the initramfs was a reliable environment to host _critical_ boot tools.

You can do that now via initrd as well.

-hpa

Signature · 3 lines
<[email protected]> at work, <[email protected]> in private!
"Unix gives you enough rope to shoot yourself in the foot."
http://www.zytor.com/~hpa/puzzle.txt	<[email protected]>
#8 lore raw
In reply to RaúlNúñez de Arenas Coronado
Quoted 9 lines · H. Peter Anvin

By author: Andreas Dilger <[email protected]>

Quoted 3 lines

I really don't see it as being a problem (other than a purely cosmetic one) to have a .journal file in your root fs. I lived with these for a couple of years OK...

It's either a cosmetic problem or a really problematic one, depending on how closely controlled that particular part of the namespace is. The .journal file could potentially try to be copied, backed up, you name it.

Actually, unless users are actively trying to shoot themselves in the foot, none of this really matters. However, now that ext3 is in the mainline, the number of users playing with guns has increased a large amount, it seems, by the number of such reports on ext3-users.

Because .journal is created as immutable, even if it was backed up and tried to be restored, it would be impossible to write to. For the "accursed" ext2 dump, it recognizes the "nodump" flag, but also knows enough not to back up the journal file. Sadly, neither cpio or tar know about ext2 attributes.

Cheers, Andreas
--
Andreas Dilger
http://sourceforge.net/projects/ext2resize/
http://www-mddsp.enel.ucalgary.ca/People/adilger/

#9 lore raw
In reply to RaúlNúñez de Arenas Coronado
Quoted 6 lines · Andreas Dilger

Actually, unless users are actively trying to shoot themselves in the foot, none of this really matters. However, now that ext3 is in the mainline, the number of users playing with guns has increased a large amount, it seems, by the number of such reports on ext3-users.

Quoted 6 lines

Because .journal is created as immutable, even if it was backed up and tried to be restored, it would be impossible to write to. For the "accursed" ext2 dump, it recognizes the "nodump" flag, but also knows enough not to back up the journal file. Sadly, neither cpio or tar know about ext2 attributes.

Nor scp, nor rsync, nor find, nor...

-hpa

#10 lore raw

Dixitur de H. Peter Anvin respondebo ad:

Quoted 1 line

Andreas Dilger wrote:

(...)

Quoted 6 lines
Quoted 5 lines

Because .journal is created as immutable, even if it was backed up and tried to be restored, it would be impossible to write to. For the "accursed" ext2 dump, it recognizes the "nodump" flag, but also knows enough not to back up the journal file. Sadly, neither cpio or tar know about ext2 attributes.

Nor scp, nor rsync, nor find, nor...

Neither they do about UCB FFS attributes... they even have separate user/system immutable flags :)

What about pax(1)? I guess it doesn't either, because cpio is just pax.

-mirabilos

Signature · 4 lines
| This message body is covered by Germanic and International | OpenBSD30
| Copyright law. Modification of any kind and redistribution | centericq
| via AOL or the Microsoft network are strictly prohibited!! | UIN seems
| Scientific-style quotation permitted if due credits given. | 132315236
#11 lore raw
In reply to RaúlNúñez de Arenas Coronado
Quoted 18 lines · H. Peter Anvin
Followup to:  <[email protected]>
By author:    Andreas Dilger <[email protected]>
In newsgroup: linux.dev.kernel
Quoted 9 lines

Don't do that. That is only good if the filesystem thinks that there is no journal, or it is using a hidden inode for the journal (i.e. if you run "tune2fs -l /dev/whatever" and it doesn't have "has_journal" listed in the filesystem features (this is what happened with 2.4.10). Otherwise, you will delete your real journal, tune2fs will complain, and then you will need to run e2fsck to clean up after yourself, before re-creating your journal again.

Quoted 5 lines

If you have a filesystem with a .journal file, and you want to "hide" it, just run e2fsck 1.25 while the filesystem is unmounted, and it will do it for you. If you don't want to have a .journal in the first place, run tune2fs -j while the filesystem is unmounted.

Quoted 5 lines

This is all fine and good except for the root partition (I'm pleased to hear that e2fsck 1.25 will move the journal to the hidden inode for non-root partitions.) It would be nice if this was done automagically by the mounting code instead of by fsck; that way migration would truly be painless.

Hm, the e2fsck check does not work for me...
The .journal file still exists after

   # umount /dev/hda11
   # e2fsck -f /dev/hda11
   # mount /dev/hda11
   # rpm -q e2fsprogs
   e2fsprogs-1.25-1

(on redhat rawhide)

-Marcel

Signature · 6 lines
     ======--------         Marcel J.E. Mol                MESA Consulting B.V.
    =======---------        ph. +31-(0)6-54724868          P.O. Box 112
    =======---------        [email protected]                 2630 AC  Nootdorp
__==== www.mesa.nl ---____U_n_i_x______I_n_t_e_r_n_e_t____ The Netherlands ____
 They couldn't think of a number,           Linux user 1148  --  counter.li.org
    so they gave me a name!  -- Rupert Hine  --  www.ruperthine.com
#12 lore raw
In reply to RaúlNúñez de Arenas Coronado

Excuse me, if I jump in with almost zero linux-kernel-knowledge, and take my words as a third-party comment.

May be, I've not seen enough in 35 years of system programming (including designing and writing journal-systems my own),
but I've never seen a journal beeing part of the data-space to be journalled. It is simply an ugly thing in the file space. It either belongs into /proc/fs/ext3 (or the like) or is not to be shown at all. Except there was a valid neccessity to have it in the normal file space.

Greetings
Michael

#14 lore raw
In reply to RaúlNúñez de Arenas Coronado
Quoted 7 lines · Marcel J.E. Mol
Quoted 5 lines · H. Peter Anvin

This is all fine and good except for the root partition (I'm pleased to hear that e2fsck 1.25 will move the journal to the hidden inode for non-root partitions.) It would be nice if this was done automagically by the mounting code instead of by fsck; that way migration would truly be painless.

Quoted 8 lines

Hm, the e2fsck check does not work for me...
The .journal file still exists after

   # umount /dev/hda11
   # e2fsck -f /dev/hda11
   # mount /dev/hda11
   # rpm -q e2fsprogs
   e2fsprogs-1.25-1

I was just going to comment on this as well. I cannot get this to work as advertised on a homegrown system as well.

Even tune2fs -l | grep inode and ls -i .journal show it's still using the same inode.

Actually, a quick look through the code, I can't see where e2fsck would take measures to hide .journal. At least not in 1.25.

mrc

Signature · 3 lines
     Mike Castle      [email protected]      www.netcom.com/~dalgoda/
    We are all of us living in the shadow of Manhattan.  -- Watchmen
fatal ("You are in a maze of twisty compiler features, all different"); -- gcc
#15 lore raw
In reply to RaúlNúñez de Arenas Coronado
Quoted 10 lines · Mike Castle
Quoted 8 lines · Marcel J.E. Mol

Hm, the e2fsck check does not work for me...
The .journal file still exists after

   # umount /dev/hda11
   # e2fsck -f /dev/hda11
   # mount /dev/hda11
   # rpm -q e2fsprogs
   e2fsprogs-1.25-1
Quoted 8 lines

I was just going to comment on this as well. I cannot get this to work as advertised on a homegrown system as well.

Even tune2fs -l | grep inode and ls -i .journal show it's still using the same inode.

Actually, a quick look through the code, I can't see where e2fsck would take measures to hide .journal. At least not in 1.25.

OK, my bad. This is in my version of e2fsprogs (from Ted's BitKeeper repository), but is not in an official release of e2fsprogs yet. It will be in 1.26, but for now I don't even think there is a WIP release which has it. Sorry for the confusion.

Cheers, Andreas
--
Andreas Dilger
http://sourceforge.net/projects/ext2resize/
http://www-mddsp.enel.ucalgary.ca/People/adilger/

#16 lore raw
In reply to RaúlNúñez de Arenas Coronado
Quoted 4 lines · Andreas Dilger

OK, my bad. This is in my version of e2fsprogs (from Ted's BitKeeper repository), but is not in an official release of e2fsprogs yet. It will be in 1.26, but for now I don't even think there is a WIP release which has it. Sorry for the confusion.

Whew.

Better to know a feature is in the works than to think it's not working. :->

Cheers,
mrc

Signature · 3 lines
     Mike Castle      [email protected]      www.netcom.com/~dalgoda/
    We are all of us living in the shadow of Manhattan.  -- Watchmen
fatal ("You are in a maze of twisty compiler features, all different"); -- gcc
#17 lore raw
In reply to RaúlNúñez de Arenas Coronado
Quoted 10 lines · Michael Zimmermann

Excuse me, if I jump in with almost zero linux-kernel-knowledge, and take my words as a third-party comment.

May be, I've not seen enough in 35 years of system programming (including designing and writing journal-systems my own),
but I've never seen a journal beeing part of the data-space to be journalled. It is simply an ugly thing in the file space. It either belongs into /proc/fs/ext3 (or the like) or is not to be shown at all. Except there was a valid neccessity to have it in the normal file space.

You do not understand the reasons behind the thread.

.journal files are only created when an ext2 FS is converted to ext3 *while* it is mounted. In this case tune2fs just creates a file (.journal) and modifies a couple fields in the super block.

The newest e2fsck (1.25) will hide the file for you, just like it would be if the conversion was done on an unmounted FS.

MF

#18 lore raw
In reply to RaúlNúñez de Arenas Coronado
Quoted 2 lines · Mike Fedyk

The newest e2fsck (1.25) will hide the file for you, just like it would be if the conversion was done on an unmounted FS.

Actually, to re-iterate a recent point: No, it won't.

The next version of e2fsck should. But 1.25 does NOT do that.

mrc

Signature · 3 lines
     Mike Castle      [email protected]      www.netcom.com/~dalgoda/
    We are all of us living in the shadow of Manhattan.  -- Watchmen
fatal ("You are in a maze of twisty compiler features, all different"); -- gcc
#19 lore raw
In reply to RaúlNúñez de Arenas Coronado
Quoted 9 lines · Mike Castle
Quoted 2 lines · Mike Fedyk

The newest e2fsck (1.25) will hide the file for you, just like it would be if the conversion was done on an unmounted FS.

Actually, to re-iterate a recent point: No, it won't.

The next version of e2fsck should. But 1.25 does NOT do that.

Ok...

Strange that aDilger has been one to say that 1.25 does hide the file...

Since I don't know myself, I'll let others say for sure.

MF