Btrfs-progs-3.16: fs metadata is both single and dup?
I updated to progs-3.16 and noticed during testing:
root>losetup
NAME SIZELIMIT OFFSET AUTOCLEAR RO BACK-FILE/dev/loop0 0 0 0 0 /tmp/img
root>mkfs.btrfs -f /dev/loop0
Btrfs v3.16
See http://btrfs.wiki.kernel.org for more information.
Performing full device TRIM (8.00GiB) ...
Turning ON incompat feature 'extref': increased hardlink limit per file to 65536
fs created label (null) on /dev/loop0
nodesize 16384 leafsize 16384 sectorsize 4096 size 8.00GiB
root>mkdir /tmp/btrfs
root>mount /dev/loop0 /tmp/btrfs
All fine until here..
root>btrfs filesystem df /tmp/btrfs
Data, single: total=8.00MiB, used=64.00KiB
System, DUP: total=8.00MiB, used=16.00KiB
System, single: total=4.00MiB, used=0.00
Metadata, DUP: total=409.56MiB, used=112.00KiB
Metadata, single: total=8.00MiB, used=0.00
..wait, what? Let's be clear..
root>btrfs balance start -mconvert=dup /tmp/btrfs
Done, had to relocate 4 out of 5 chunks
root>btrfs filesystem df /tmp/btrfs
Data, single: total=8.00MiB, used=64.00KiB
System, DUP: total=32.00MiB, used=16.00KiB
Metadata, DUP: total=128.00MiB, used=112.00KiB
Now looking as expected.
root>btrfs balance start -mconvert=single -f /tmp/btrfs Done, had to relocate 2 out of 3 chunks
root>btrfs filesystem df /tmp/btrfs
Data, single: total=8.00MiB, used=128.00KiB
System, single: total=32.00MiB, used=16.00KiB
Metadata, single: total=256.00MiB, used=112.00KiB
Looking good too (metadata doubled)
So where does the confusing initial display come from? I'm running this against a (very patched) 3.14.17, but don't remember ever seeing this with btrfs-progs-3.14.2.
Any ideas?
thanks
Holger
Quoted 19 lines · Holger Hoffstätte
I updated to progs-3.16 and noticed during testing:
root>losetup
NAME SIZELIMIT OFFSET AUTOCLEAR RO BACK-FILE/dev/loop00 0 0 0/tmp/imgroot>mkfs.btrfs -f
/dev/loop0
Btrfs v3.16
See http://btrfs.wiki.kernel.org for more information.Performing full device TRIM (8.00GiB) ...
Turning ON incompat feature 'extref': increased hardlink limit per file to 65536
fs created label (null) on/dev/loop0nodesize 16384 leafsize 16384 sectorsize 4096 size 8.00GiBroot>mkdir
/tmp/btrfs
root>mount/dev/loop0/tmp/btrfs
Quoted 8 lines
All fine until here..
root>btrfs filesystem df
/tmp/btrfs
Data, single: total=8.00MiB, used=64.00KiB
System, DUP: total=8.00MiB, used=16.00KiB
System, single: total=4.00MiB, used=0.00
Metadata, DUP: total=409.56MiB, used=112.00KiB
Metadata, single: total=8.00MiB, used=0.00
Note that the "single" chunks are empty, and will remain so.
[snip]
Quoted 3 lines
So where does the confusing initial display come from? I'm running this against a (very patched) 3.14.17, but don't remember ever seeing this with btrfs-progs-3.14.2.
Your memory is faulty, I'm afraid. It's always done that -- at
least since I started using btrfs, several years ago.
I believe it comes from mkfs creating a trivial basic filesystem
(with the single profiles), and then setting enough flags on it that the kernel can bootstrap it with the desired chunks in it -- but I may be wrong about that.
Hugo.
Signature · 3 lines
=== Hugo Mills: hugo@... carfax.org.uk | darksatanic.net | lug.org.uk ===
PGP key: 65E74AC0 from wwwkeys.eu.pgp.net or http://www.carfax.org.uk
--- Normaliser unix c'est comme pasteuriser le Camembert --- Quoted 6 lines · Hugo Mills
[snip]
Quoted 3 lines
So where does the confusing initial display come from? I'm running this against a (very patched) 3.14.17, but don't remember ever seeing this with btrfs-progs-3.14.2.
Your memory is faulty, I'm afraid. It's always done that -- at
After 40 it's all downhill.. :-)
Quoted 1 line
least since I started using btrfs, several years ago.
Alright then. I was just getting nervous I might have borked something either with my kernel or that the new progs somehow get information from the kernel that is off/wrong.
All my other filesystems display as expected, but they have been balanced previously - so nothing to see there.
Thanks!
Holger
Hugo Mills posted on Tue, 02 Sep 2014 13:13:49 +0100 as excerpted:
Quoted 20 lines · Holger Hoffstätte
Quoted 5 lines
I updated to progs-3.16 and noticed during testing:
root>mkfs.btrfs -f
/dev/loop0Quoted 8 lines
All fine until here..
root>btrfs filesystem df
/tmp/btrfs
Data, single: total=8.00MiB, used=64.00KiB
System, DUP: total=8.00MiB, used=16.00KiB
System, single: total=4.00MiB, used=0.00
Metadata, DUP: total=409.56MiB, used=112.00KiB
Metadata, single: total=8.00MiB, used=0.00Note that the "single" chunks are empty, and will remain so.
Quoted 2 lines
So where does the confusing initial display come from? [I] don't remember ever seeing this with btrfs-progs-3.14.2.
Quoted 7 lines
Your memory is faulty, I'm afraid. It's always done that -- at
least since I started using btrfs, several years ago.I believe it comes from mkfs creating a trivial basic filesystem
(with the single profiles), and then setting enough flags on it that the kernel can bootstrap it with the desired chunks in it -- but I may be wrong about that.
Agreed. It's an artifact of the mkfs.btrfs process and a btrfs fi df on a new filesystem always seems to have those extra unused single profile lines.
I got so the first thing I'd do on first mount was a balance -- before there was anything actually on the filesystem so it was real fast -- to get rid of those null entries.
Actually, I had already created a little mkfs.btrfs helper script that sets options I normally want, etc, and after doing the mkfs and balance drill a few times, I setup the script such that if at the appropriate prompt I give it a mountpoint to point balance at, it'll mount the filesystem and immediately run a balance, thus automating things and making the balance part of the same scripted process that does the mkfs.btrfs in the first place.
IOW, those null-entry lines bother me too... enough that even tho I know what they are I arranged things so they're automatically and immediately eliminated and I don't have to see 'em! =:^)
Signature · 3 lines
Duncan - List replies preferred. No HTML msgs. "Every nonfree program has a lord, a master -- and if you use the program, he is your master." Richard Stallman
Quoted 14 lines · Duncan
Hugo Mills posted on Tue, 02 Sep 2014 13:13:49 +0100 as excerpted:
Quoted 11 lines · Holger Hoffstätte
Quoted 2 lines
So where does the confusing initial display come from? [I] don't remember ever seeing this with btrfs-progs-3.14.2.
Your memory is faulty, I'm afraid. It's always done that -- at
least since I started using btrfs, several years ago.I believe it comes from mkfs creating a trivial basic filesystem
(with the single profiles), and then setting enough flags on it that the kernel can bootstrap it with the desired chunks in it -- but I may be wrong about that.
Quoted 7 lines
Agreed. It's an artifact of the mkfs.btrfs process and a btrfs fi df on a new filesystem always seems to have those extra unused single profile lines.
I got so the first thing I'd do on first mount was a balance -- before there was anything actually on the filesystem so it was real fast -- to get rid of those null entries.
Interesting. Last time I tried that (balance without any contents),
the balance removed *all* the chunks, and then the FS forgot about what configuration it should have and reverted to RAID-1/single. I usually recommend writing at least one 4k+ file to the FS first, if it's bothering someone so much that they can't let it go.
Hugo.
Quoted 12 lines
Actually, I had already created a little mkfs.btrfs helper script that sets options I normally want, etc, and after doing the mkfs and balance drill a few times, I setup the script such that if at the appropriate prompt I give it a mountpoint to point balance at, it'll mount the filesystem and immediately run a balance, thus automating things and making the balance part of the same scripted process that does the mkfs.btrfs in the first place.
IOW, those null-entry lines bother me too... enough that even tho I know what they are I arranged things so they're automatically and immediately eliminated and I don't have to see 'em! =:^)
Signature · 4 lines
=== Hugo Mills: hugo@... carfax.org.uk | darksatanic.net | lug.org.uk ===
PGP key: 65E74AC0 from wwwkeys.eu.pgp.net or http://www.carfax.org.uk
--- Never underestimate the bandwidth of a Volvo filled ---
with backup tapes. Hugo Mills posted on Wed, 03 Sep 2014 08:33:05 +0100 as excerpted:
Quoted 11 lines · Duncan
Quoted 9 lines
Hugo Mills posted on Tue, 02 Sep 2014 13:13:49 +0100 as excerpted:
[A] btrfs fi df on a new filesystem always seems to have those extra unused single profile lines.
I got so the first thing I'd do on first mount was a balance -- before there was anything actually on the filesystem so it was real fast -- to get rid of those null entries.
Quoted 5 lines
Interesting. Last time I tried that (balance without any contents),
the balance removed *all* the chunks, and then the FS forgot about what configuration it should have and reverted to RAID-1/single. I usually recommend writing at least one 4k+ file to the FS first, if it's bothering someone so much that they can't let it go.
Interesting indeed. From memory, even before I've put anything on the filesystem it always seems to have a bit of the first chunk of both data and metadata used -- not much but enough that it's obvious in the df which mode chunks are the null-chunks, and apparently obvious to the balance as well, as it has always left me with at least a first chunk of each.
I wonder what the difference might be. Perhaps it's just the versions of kernel and/or userspace I've happened to do all my mkfs.btrfs with? Or maybe it's one of the features (like thin-metadata or noholes) I enable by default, or the fact that I use labels for partition ID and tracking, so I always fill that in. Whatever it is, it seems to put a bit of something in the filesystem, possibly at first mount, so the actually used chunks, one each of data and metadata, aren't entirely empty.
Or maybe I'm remembering wrong and I've just been lucky. <shrug>
Signature · 3 lines
Duncan - List replies preferred. No HTML msgs. "Every nonfree program has a lord, a master -- and if you use the program, he is your master." Richard Stallman