Setting options permanently?
Hi,
just a question:
Mounting options for file systems are usually given on the command line or in /etc/fstab.
Both do not work with mobile storage devices like usb hard disks, since they are either mounted automatically through the desktop, or manually by a user who does not know or remember the mount options for this file system. At least it's error prone.
What happens if a btrfs is mounted sometimes with and sometimes without the compression option? I'd guess it generates a mixture of compressed and uncompressed files, what is not exactly smart.
Is there a way to set options like compression on a btrfs permanently to activate them even when mounted automatically by the desktop or manually by a third person?
regards
Hadmut
On Fri, Jan 27, 2012 at 9:03 AM, Hadmut Danisch <[email protected]> wro= te:
Quoted 5 lines
Hi,
just a question:
Mounting options for file systems are usually given on the command li=
ne
Quoted 3 lines
or in
/etc/fstab.Both do not work with mobile storage devices like usb hard disks, sin=
ce
Quoted 1 line
they are either mounted automatically through the desktop, or manuall=
y
Quoted 1 line
by a user who does not know or remember the mount options for this fi=
le
Quoted 5 lines
system. At least it's error prone.
What happens if a btrfs is mounted sometimes with and sometimes witho=
ut
Quoted 1 line
the compression option? I'd guess it generates a mixture of compresse=
d
Quoted 2 lines
and uncompressed files, what is not exactly smart.
It should be okay to mount with compress or without compress. Even if you mount a volume with compressed data without '-o compress' you will still be able to correctly read the data (but newly written data will not be compressed)
Quoted 1 line
Is there a way to set options like compression on a btrfs permanently=
to
Quoted 1 line
activate them even when mounted automatically by the desktop or manua=
lly
Quoted 8 lines
by a third person?
regards
Hadmut--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs=
" in
Quoted 2 lines
the body of a message to [email protected]
More majordomo info at =A0http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" = in
the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Am 28.01.2012 00:20, schrieb Chester:
Quoted 2 lines
It should be okay to mount with compress or without compress. Even if you mount a volume with compressed data without '-o compress' you wil=
l
Quoted 2 lines
still be able to correctly read the data (but newly written data will not be compressed)
But having both compressed and uncompressed files in the filesystem is exactly what I want to avoid. Not because of reading problems, but to avoid wasting disk space. I don't have a reading problem. I have a writing problem.
I want to pack some data onto USB hard disks, which exceed the plain disk capacity. With btrfs it works exactly the way as I need it, except for the fact that once the compress option was missed the file system i= s
=84tainted=94 with uncompressed files, thus wasting disk capacity and m= aybe
cause the files to not fit on the disk anymore.
I currently don't see how to repair this afterwards without removing th= e
uncompressed files and writing new ones, which on the other hand spoils hte memory saving effect of using snapshots instead of copies.
It would be much better if there was a way to set options like the compression option permanently when initially creating the file system, or later with the btrfs tools.
regards
Hadmut
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" = in
the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Hadmut Danisch posted on Sat, 28 Jan 2012 01:49:26 +0100 as excerpted:
Quoted 2 lines
Am 28.01.2012 00:20, schrieb Chester:
Quoted 1 line
It should be okay to mount with compress or without compress. Even i=
f
Quoted 1 line
Quoted 1 line
you mount a volume with compressed data without '-o compress' you wi=
ll
Quoted 1 line
Quoted 1 line
still be able to correctly read the data (but newly written data wil=
l
Quoted 3 lines
Quoted 1 line
not be compressed)
=20
But having both compressed and uncompressed files in the filesystem i=
s
Quoted 3 lines
exactly what I want to avoid. Not because of reading problems, but to avoid wasting disk space. I don't have a reading problem. I have a writing problem.
Having a compression flag in the superblock (probably more than one, th= us=20
allowing indication of compression type, etc), which is I guess what=20 you're suggesting, sounds useful to me too. Perhaps it'll get it, once= =20
development settles down a bit and they know enough about what's still=20 missing in the existing superblock to be able to intelligently allocate= =20
flags, etc, on a priority basis based on what they then know they need.
At this point I can certainly see a reluctance to make that change, tho= ,=20
since it's a rather small change to make on its own, and with continuin= g=20
tweaks to the compression types and etc, it's worth holding off=20 committing now to what might look like a bad implementation down the=20 line, when it has to continue to be supported.
Quoted 2 lines
I want to pack some data onto USB hard disks, which exceed the plain disk capacity. With btrfs it works exactly the way as I need it, exce=
pt
Quoted 1 line
for the fact that once the compress option was missed the file system=
is
Quoted 1 line
=E2=80=9Etainted=E2=80=9D with uncompressed files, thus wasting disk =
capacity and maybe
Quoted 1 line
cause the files to not fit on the disk anymore.
I'd be a bit leary of your whole idea of using btrfs on external drives= =20
attached to multiple systems, at this point. There's still enough chan= ge=20
going into btrfs that there's a chance that mounting on a system with a= =20
newer kernel will prevent mounting on a system with an older kernel.
And if you have enough control over kernel versions on all potentially=20 mounting systems to deal with that, then pretty much by definition, you= =20
also have enough control over them to hack up a solution that, for=20 instance, looks for a file in specific location on all newly mounted=20 partitions (probably using udev/udisks event hooks), and if found, uses= =20
the content of said file as a cue to remount the filesystem with the=20 options found in said file.
Quoted 1 line
I currently don't see how to repair this afterwards without removing =
the
Quoted 1 line
uncompressed files and writing new ones, which on the other hand spoi=
ls
Quoted 1 line
hte memory saving effect of using snapshots instead of copies.
A rebalance is the usual way of handling such things, since that rewrit= es=20
every chunk, taking account of current mount options, etc. If I've bee= n=20
reading correctly, a defrag should handle it now as well, altho that us= ed=20
to break snapshots/cow/reflink-copies, but I /think/ I read that with 3= =2E2=20
(or was it 3.3-rc1) it doesn't, any more.
Quoted 2 lines
It would be much better if there was a way to set options like the compression option permanently when initially creating the file syste=
m,
Quoted 1 line
or later with the btrfs tools.
As hinted above, I'd /guess/ the chances of something like that=20 eventually occurring are reasonably high, but there's just too much goi= ng=20
on still, to make superblock changes for every little new feature. =20 Better to save them up for one final change before the caution about th= e=20
on-disk format changes comes off, prioritize the changes then, and do=20 them all at once. Then test the new format for a kernel cycle or two=20 just to be sure, and remove the format change boilerplate the cycle aft= er=20
that.
--=20
Duncan - List replies preferred. No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master." Richard Stallman
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" = in
the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Quoted 1 line · Duncan
Hadmut Danisch posted on Sat, 28 Jan 2012 01:49:26 +0100 as excerpted=
:
Quoted 3 lines
Quoted 3 lines
Am 28.01.2012 00:20, schrieb Chester:
Quoted 1 line
It should be okay to mount with compress or without compress. Even =
if
Quoted 1 line
Quoted 1 line
Quoted 1 line
you mount a volume with compressed data without '-o compress' you w=
ill
Quoted 1 line
Quoted 1 line
Quoted 1 line
still be able to correctly read the data (but newly written data wi=
ll
Quoted 3 lines
Quoted 3 lines
Quoted 1 line
not be compressed)
But having both compressed and uncompressed files in the filesystem =
is
Quoted 1 line
Quoted 1 line
exactly what I want to avoid. Not because of reading problems, but t=
o
Quoted 4 lines
Quoted 2 lines
avoid wasting disk space. I don't have a reading problem. I have a writing problem.
Having a compression flag in the superblock (probably more than one, =
thus
Quoted 2 lines
allowing indication of compression type, etc), which is I guess what you're suggesting, sounds useful to me too. =C2=A0Perhaps it'll get i=
t, once
Quoted 1 line
development settles down a bit and they know enough about what's stil=
l
Quoted 1 line
missing in the existing superblock to be able to intelligently alloca=
te
Quoted 1 line
flags, etc, on a priority basis based on what they then know they nee=
d.
mkfs -O some_feature comes to mind. I know mke2fs has that, but I haven't yet actually used btrfs; I'm waiting for fsck to be able to repair a damaged filesystem.
Quoted 1 line
At this point I can certainly see a reluctance to make that change, t=
ho,
Quoted 1 line
since it's a rather small change to make on its own, and with continu=
ing
Quoted 3 lines
tweaks to the compression types and etc, it's worth holding off committing now to what might look like a bad implementation down the line, when it has to continue to be supported.
This would affect not just a compression-preference option, but existing filesystems with compression. If the compression routine changes in a later kernel/filesystem revision, then you could end up with some files using one routine and others using another. Does btrfs currently have an "algorithm" specifier for compressed files? --
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" = in
the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Quoted 1 line · On Sat, 28 Jan 2012 17:05:41 +1100 Ben Klein <[email protected]> wrote:
If the compression routine changes in a later kernel/filesystem revision,
There are already two different algorithms, zlib and lzo, and a third one - snappy - planned for inclusion.
Quoted 1 line
then you could end up with some files using one routine and others using another.
Which is absolutely not a problem even right now.
Signature · 8 lines
With respect, Roman ~~~~~~~~~~~~~~~~~~~~~~~~~~~ "Stallman had a printer, with code he could not see. So he began to tinker, and set the software free."
Quoted 11 lines · Roman Mamedov
Quoted 1 line · On Sat, 28 Jan 2012 17:05:41 +1100 Ben Klein <[email protected]> wrote:
If the compression routine changes in a later kernel/filesystem revision,
There are already two different algorithms, zlib and lzo, and a third one - snappy - planned for inclusion.
Quoted 1 line
then you could end up with some files using one routine and others using another.
Which is absolutely not a problem even right now.
My bad :)
Quoted 9 lines
--
With respect,
Roman~~~~~~~~~~~~~~~~~~~~~~~~~~~
"Stallman had a printer,
with code he could not see.
So he began to tinker,
and set the software free."
Roman Mamedov posted on Sat, 28 Jan 2012 14:23:44 +0600 as excerpted:
Quoted 6 lines
Quoted 2 lines · On Sat, 28 Jan 2012 17:05:41 +1100 Ben Klein <[email protected]> wrote:
If the compression routine changes in a later kernel/filesystem revision,
Quoted 7 lines
There are already two different algorithms, zlib and lzo, and a third one - snappy - planned for inclusion.
Quoted 2 lines
then you could end up with some files using one routine and others using another.
Which is absolutely not a problem even right now.
Reading them isn't a problem, sure, but the thread indicates a desire to set the write-compression characteristics.
But that's why I specifically mentioned more than just a compression flag. Presumably it'd be several bits (four bits, 16 values, would leave enough room for additional compression types in the future, tho three or even two would do it with the current three choices plus uncompressed), with one value indicating a no-compression default, and the others various choices as the default. The mount option could then be used to override the default, just as it is now, altho the only choice for default currently is no compression.
Actually, given that btrfs is in line to be the successor to the ext* series, I'd say it should be close to a given that many mount options will have settable defaults recorded in the superblocks at mkfs time, and updatable using some btrfs parallel to tune2fs (tunebtrfs? tunebfs?). However, as I said earlier, committing to specifics with incremental on- disk including superblock format changes doesn't really cut it. If a big format change is needed due to continuing development, so it can be tested and gotchas worked out, yeah, do it now, and hopefully there's some space in the superblock(s) reserved for more trivial stuff like mount-option defaults already, but especially for mount options since they can be tested without committing to the superblock spec changes necessary to nail it down there, just testing them using the mount options is fine at this point, and the actual recording of the default choices can be nailed down all at once in one final change, just before the "be prepared for on-disk-format-changes" label comes off.
Signature · 3 lines
Duncan - List replies preferred. No HTML msgs. "Every nonfree program has a lord, a master -- and if you use the program, he is your master." Richard Stallman
Quoted 10 lines · Hadmut Danisch
Am 28.01.2012 00:20, schrieb Chester:
Quoted 4 lines
It should be okay to mount with compress or without compress. Even if you mount a volume with compressed data without '-o compress' you will still be able to correctly read the data (but newly written data will not be compressed)
But having both compressed and uncompressed files in the filesystem is exactly what I want to avoid. Not because of reading problems, but to avoid wasting disk space. I don't have a reading problem. I have a writing problem.
If you've been using -o compress, then you should know that even then not ALL data is compressed. If btrfs predicts that a data will be unable to benefit frmo compression, it will store it uncompressed. The problem is the prediction is not always right. Which is why there's -o compress-force.
Anyway, for removable media case, there's a workaround that you can use (at least it works with gnome). Put the entry for the usb block device (e.g. /dev/sdb1) in fstab, with appropriate mount option, and the option will be used when you mount it using nautilus.
Signature · 1 line
Fajar
Quoted 10 lines
Quoted 3 lines
I currently don't see how to repair this afterwards without removing the uncompressed files and writing new ones, which on the other hand spoils hte memory saving effect of using snapshots instead of copies.
A rebalance is the usual way of handling such things, since that rewrites every chunk, taking account of current mount options, etc. If I've been reading correctly, a defrag should handle it now as well, altho that used to break snapshots/cow/reflink-copies, but I /think/ I read that with 3.2 (or was it 3.3-rc1) it doesn't, any more.
Rebalance won't do the trick, but defrag can. And defrag still breaks snapshots, which I'm working on.
Hadmut Danisch wrote (ao):
Quoted 3 lines
I currently don't see how to repair this afterwards without removing the uncompressed files and writing new ones, which on the other hand spoils hte memory saving effect of using snapshots instead of copies.
As also mentioned by Li Zefan, you can use defrag. But this will indeed not work nicely with snapshots. And you need more free space than the largest file on the filesystem.
find / -xdev -execdir btrfs filesystem defrag -czlib {} +
Sander
Signature · 2 lines
Humilis IT Services and Solutions http://www.humilis.net
Quoted 3 lines
Is there a way to set options like compression on a btrfs permanently to activate them even when mounted automatically by the desktop or manually by a third person?
Actually there is.
Btrfs supports per file compression flag, and if this flag is set to a directory, all files in that directory will inherit the flag.
So you can mount the filesystem for the first time and set the flag for the root dir, and that's it. There's a flaw that the compression method is zlib, which should be changed to lzo (or lz4, or snappy..) in the future.
chattr is used to set file flags, and there was a patchset to enable compression flag for chattr, but was never merged:
http://www.spinics.net/lists/linux-btrfs/msg09604.html
http://www.spinics.net/lists/linux-btrfs/msg09605.html