multiple device usage
hello,
i have some difficulty in understanding multi-device handling in depth.
as http://btrfs.wiki.kernel.org/index.php/Using_Btrfs_with_Multiple_Devices
tells,
btrfs can span over multiple devices at the same time. (great feature, btw !)
ok then:
mkfs.btrfs -m single -d single /dev/sdb /dev/sdc - creates a btrfs spanning over /dev/sdb and sdc
mount /dev/sdb /btrfs - mounts it
btrfs-vol -b /btrfs - does a rebalancing of all data and metadata
btrfs-vol -r /dev/sdc - removes one of the volumes and redistributes any extends in use on sdc to sdb (killer feature!!!)
but what if i want to remove /dev/sdb ? (as that one is in use for the mount)
regards
roland
__________________________________________________________________________
Verschicken Sie SMS direkt vom Postfach aus - in alle deutschen und viele ausländische Netze zum gleichen Preis!
https://produkte.web.de/webde_sms/sms
Quoted 16 lines · [email protected] wrote:
hello,
i have some difficulty in understanding multi-device handling in depth.
as http://btrfs.wiki.kernel.org/index.php/Using_Btrfs_with_Multiple_Devices
tells,btrfs can span over multiple devices at the same time. (great feature, btw !)
ok then:
mkfs.btrfs -m single -d single
/dev/sdb/dev/sdc- creates a btrfs spanning over/dev/sdband sdcmount
/dev/sdb/btrfs - mounts it
Quoted 8 lines
btrfs-vol -b /btrfs - does a rebalancing of all data and metadata
btrfs-vol -r
/dev/sdc- removes one of the volumes and redistributes any extends in use on sdc to sdb (killer feature!!!)but what if i want to remove
/dev/sdb? (as that one is in use for the mount)
Devices in btrfs are equal, so you can do this. The only glitch is /proc/mounts and mount(8) get confused.
Regards
Yan Zheng
Quoted 1 line · On Sat, 27 Dec 2008 10:15:18 am [email protected] wrote:
http://btrfs.wiki.kernel.org/index.php/Using_Btrfs_with_Multiple_Devices
I'll add two questions that're not answered by the Wiki too.. :-)
1) If you add a second disk to an existing btrfs filesystem, can you get it to set it up as a RAID-1 arrangement rather than just rebalancing the metadata and then striping ?
2) With the concerns that people have about SSD reliability (hi Val :-) !) would it make sense to set up two equal sized partitions on the SSD and use RAID-1 across them, or can you tell btrfs to keep multiple copies of the data, a la ZFS ?
cheers!
Chris
Signature · 4 lines
Chris Samuel : http://www.csamuel.org/ : Melbourne, VIC This email may come with a PGP signature as a file. Do not panic. For more info see: http://en.wikipedia.org/wiki/OpenPGP
Quoted 19 lines
Quoted 19 lines
i have some difficulty in understanding multi-device handling in depth.
as http://btrfs.wiki.kernel.org/index.php/Using_Btrfs_with_Multiple_Devices
tells,btrfs can span over multiple devices at the same time. (great feature, btw !)
ok then:
mkfs.btrfs -m single -d single
/dev/sdb/dev/sdc- creates a btrfs spanning over/dev/sdband sdcmount
/dev/sdb/btrfs - mounts itbtrfs-vol -b /btrfs - does a rebalancing of all data and metadata
btrfs-vol -r
/dev/sdc- removes one of the volumes and redistributes any extends in use on sdc to sdb (killer feature!!!)
Quoted 6 lines
Quoted 3 lines
but what if i want to remove
/dev/sdb? (as that one is in use for the mount)Devices in btrfs are equal, so you can do this. The only glitch is
/proc/mountsand mount(8) get confused.
so i can do btrfs-vol -r /dev/sdb while it`s being mounted, pull the disk , replace it with a bigger one, rescan-scsi-bus, mkfs.btrfs the new disk and then re-add - all while mount telling me, that /dev/sdb is still in use !?
Quoted 2 lines
so i can do btrfs-vol -r
/dev/sdbwhile it`s being mounted, pull the disk , replace it with a bigger one, rescan-scsi-bus, mkfs.btrfs the new disk and
~~~~~~~~~~~~~~~~~~~~~~~
This step will fail, you will
get a "/dev/sdb is mounted" by mkfs.btrfs, but for other slots it's ok.
Quoted 1 line
then re-add - all while mount telling me, that
/dev/sdbis still in use !?
2008/12/27 Roland <[email protected]>:
Quoted 37 lines
Quoted 22 lines
Quoted 22 lines
i have some difficulty in understanding multi-device handling in depth.
as
http://btrfs.wiki.kernel.org/index.php/Using_Btrfs_with_Multiple_Devices
tells,btrfs can span over multiple devices at the same time. (great feature, btw !)
ok then:
mkfs.btrfs -m single -d single
/dev/sdb/dev/sdc- creates a btrfs spanning
over/dev/sdband sdcmount
/dev/sdb/btrfs - mounts itbtrfs-vol -b /btrfs - does a rebalancing of all data and metadata
btrfs-vol -r
/dev/sdc- removes one of the volumes and redistributes any extends in use on sdc to sdb (killer feature!!!)Quoted 6 lines
Quoted 3 lines
but what if i want to remove
/dev/sdb? (as that one is in use for the mount)Devices in btrfs are equal, so you can do this. The only glitch is
/proc/mountsand mount(8) get confused.so i can do btrfs-vol -r
/dev/sdbwhile it`s being mounted, pull the disk , replace it with a bigger one, rescan-scsi-bus, mkfs.btrfs the new disk and then re-add - all while mount telling me, that/dev/sdbis still in use !? --
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Signature · 2 lines
Regards, Zhu Yanhai
Quoted 5 lines · On Sun, 28 Dec 2008 21:26:11 +0800 "yanhai zhu" <[email protected]> wrote:
Quoted 2 lines
so i can do btrfs-vol -r
/dev/sdbwhile it`s being mounted, pull the disk , replace it with a bigger one, rescan-scsi-bus, mkfs.btrfs the new disk and~~~~~~~~~~~~~~~~~~~~~~~
This step will fail, you will
get a "/dev/sdbis mounted" by mkfs.btrfs, but for other slots it's ok.
Really, what average-joe-user needs btrfs for his not-always-mounted data-partition? Why shouldn't he just use any other fs for that? All users needing _uptime_ of their fs cannot simply unmount and wait hours for some cool feature to (hopefully) complete (without failures). Why does nobody talk about the real world? What's the use of features that cannot complete their job in users' lifetime due to everybody filling up TBs of data on their harddisks. How long does it really take to replace a full 300 GB hd with a 800 GB in a multi volume btrfs ?
Please play with real amounts of data and judge for yourself the use of offline-fs-features.
The times of classical PC use are gone, today people have just about everything on their hds, music collection, dvd collection, mails, documents of all kinds, sourcecode trees, name-one. 500 GB hds are very cheap. A _new_ fs not able to cope with these new usage-patterns is in fact _useless_.
Regards,
Stephan
Quoted 51 lines
Quoted 1 line
then re-add - all while mount telling me, that
/dev/sdbis still in use !?2008/12/27 Roland <[email protected]>:
Quoted 37 lines
Quoted 22 lines
Quoted 22 lines
i have some difficulty in understanding multi-device handling in depth.
as
http://btrfs.wiki.kernel.org/index.php/Using_Btrfs_with_Multiple_Devices
tells,btrfs can span over multiple devices at the same time. (great feature, btw !)
ok then:
mkfs.btrfs -m single -d single
/dev/sdb/dev/sdc- creates a btrfs spanning
over/dev/sdband sdcmount
/dev/sdb/btrfs - mounts itbtrfs-vol -b /btrfs - does a rebalancing of all data and metadata
btrfs-vol -r
/dev/sdc- removes one of the volumes and redistributes any extends in use on sdc to sdb (killer feature!!!)Quoted 6 lines
Quoted 3 lines
but what if i want to remove
/dev/sdb? (as that one is in use for the mount)Devices in btrfs are equal, so you can do this. The only glitch is
/proc/mountsand mount(8) get confused.so i can do btrfs-vol -r
/dev/sdbwhile it`s being mounted, pull the disk , replace it with a bigger one, rescan-scsi-bus, mkfs.btrfs the new disk and then re-add - all while mount telling me, that/dev/sdbis still in use !? --
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.htmlSignature · 6 lines
Regards, Zhu Yanhai -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
Quoted 1 line · Chris Samuel
I'll add two questions that're not answered by the Wiki too.. :-)
Looking at the source implies to me:
Quoted 3 lines
1) If you add a second disk to an existing btrfs filesystem, can you get it to set it up as a RAID-1 arrangement rather than just rebalancing the metadata and then striping ?
No, you can't, the data stripe numbers seem to be set at mkfs time.
The rebalancing code does appear (from a naive read of the code) to be able to rebalance over stripes, but I have no idea if the disk format currently supports changing that on the fly.
Quoted 4 lines
2) With the concerns that people have about SSD reliability (hi Val :-) !) would it make sense to set up two equal sized partitions on the SSD and use RAID-1 across them, or can you tell btrfs to keep multiple copies of the data, a la ZFS ?
Again it would appear that you need to have two partitions and that btrfs cannot (at present) keep multiple data stripes on the same partition.
Could someone who actually knows the code to comment on whether my inferences are accurate or not please ? :-)
cheers,
Chris
Signature · 4 lines
Chris Samuel : http://www.csamuel.org/ : Melbourne, VIC This email may come with a PGP signature as a file. Do not panic. For more info see: http://en.wikipedia.org/wiki/OpenPGP
Quoted 6 lines
Quoted 2 lines
so i can do btrfs-vol -r
/dev/sdbwhile it`s being mounted, pull the disk , replace it with a bigger one, rescan-scsi-bus, mkfs.btrfs the new disk and~~~~~~~~~~~~~~~~~~~~~~~
This step will fail, you
will
get a "/dev/sdbis mounted" by mkfs.btrfs, but for other slots it's ok.
isn`t this sort of a design issue then?
no way to work around this ?
i think online data migration and disk replacement is a killer feature of btrfs (even zfs doesnt have this!), but if you always have one out of X disks which cant be used for this, then it`s of no real value, imho.
regards
roland
Quoted 47 lines
Quoted 2 lines
then re-add - all while mount telling me, that
/dev/sdbis still in use !?2008/12/27 Roland <[email protected]>:
Quoted 37 lines
Quoted 22 lines
Quoted 22 lines
i have some difficulty in understanding multi-device handling in depth.
as
http://btrfs.wiki.kernel.org/index.php/Using_Btrfs_with_Multiple_Devices
tells,btrfs can span over multiple devices at the same time. (great feature, btw !)
ok then:
mkfs.btrfs -m single -d single
/dev/sdb/dev/sdc- creates a btrfs spanning
over/dev/sdband sdcmount
/dev/sdb/btrfs - mounts itbtrfs-vol -b /btrfs - does a rebalancing of all data and metadata
btrfs-vol -r
/dev/sdc- removes one of the volumes and redistributes any extends in use on sdc to sdb (killer feature!!!)Quoted 6 lines
Quoted 3 lines
but what if i want to remove
/dev/sdb? (as that one is in use for the mount)Devices in btrfs are equal, so you can do this. The only glitch is
/proc/mountsand mount(8) get confused.so i can do btrfs-vol -r
/dev/sdbwhile it`s being mounted, pull the disk , replace it with a bigger one, rescan-scsi-bus, mkfs.btrfs the new disk and then re-add - all while mount telling me, that/dev/sdbis still in use !? --
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.htmlSignature · 2 lines
Regards, Zhu Yanhai
2008/12/29 Chris Samuel <[email protected]>:
Quoted 10 lines · Chris Samuel
Quoted 2 lines
I'll add two questions that're not answered by the Wiki too.. :-)
Looking at the source implies to me:
Quoted 3 lines
1) If you add a second disk to an existing btrfs filesystem, can you get it to set it up as a RAID-1 arrangement rather than just rebalancing the metadata and then striping ?
Quoted 6 lines
No, you can't, the data stripe numbers seem to be set at mkfs time.
The rebalancing code does appear (from a naive read of the code) to be able to rebalance over stripes, but I have no idea if the disk format currently supports changing that on the fly.
The rebalancing moves data/metadata to newly created chunks. If there are two devices, the new chunks will be set up as RAID-1 by default.
Quoted 8 lines
Quoted 4 lines
2) With the concerns that people have about SSD reliability (hi Val :-) !) would it make sense to set up two equal sized partitions on the SSD and use RAID-1 across them, or can you tell btrfs to keep multiple copies of the data, a la ZFS ?
Again it would appear that you need to have two partitions and that btrfs cannot (at present) keep multiple data stripes on the same partition.
Yes. I think this is due to performance reason. Changing the code to support data duplication in single spindle configuration is easy.
Regards
Yan Zheng
2008/12/29 Roland <[email protected]>:
Quoted 5 lines
Quoted 5 lines
Quoted 4 lines
so i can do btrfs-vol -r
/dev/sdbwhile it`s being mounted, pull the disk ,
replace it with a bigger one, rescan-scsi-bus, mkfs.btrfs the new disk and
Quoted 8 lines
Quoted 4 lines
~~~~~~~~~~~~~~~~~~~~~~~
This step will fail, you
will
get a "/dev/sdbis mounted" by mkfs.btrfs, but for other slots it's ok.isn`t this sort of a design issue then?
no way to work around this ?
No, this is a (easy to fix) bug.
Regards
Yan Zheng
Hello Yan,
Quoted 8 lines · Yan Zheng
2008/12/29 Chris Samuel <[email protected]>:
Quoted 3 lines
The rebalancing code does appear (from a naive read of the code) to be able to rebalance over stripes, but I have no idea if the disk format currently supports changing that on the fly.
The rebalancing moves data/metadata to newly created chunks. If there are two devices, the new chunks will be set up as RAID-1 by default.
Very interesting! I didn't realise that - I was presuming that the BTRFS_BLOCK_GROUP_RAID1 ioctl needed to be passed through (as at mkfs time) to change it into a RAID1 array.
Could you point me at the code where this change is detected please ? Just
so I can try and wrap my mind around it!
Quoted 4 lines
Quoted 2 lines
Again it would appear that you need to have two partitions and that btrfs cannot (at present) keep multiple data stripes on the same partition.
Yes. I think this is due to performance reason.
Understood.
Quoted 2 lines
Changing the code to support data duplication in single spindle configuration is easy.
OK, that's encouraging (though for my immediate purposes creating two partitions is fine, but it might make life easier for others).
Again thanks so much for the response!
All the best,
Chris
Signature · 4 lines
Chris Samuel : http://www.csamuel.org/ : Melbourne, VIC This email may come with a PGP signature as a file. Do not panic. For more info see: http://en.wikipedia.org/wiki/OpenPGP
2008/12/29 Chris Samuel <[email protected]>:
Quoted 10 lines
Hello Yan,
Quoted 6 lines · Yan Zheng
2008/12/29 Chris Samuel <[email protected]>:
Quoted 3 lines
The rebalancing code does appear (from a naive read of the code) to be able to rebalance over stripes, but I have no idea if the disk format currently supports changing that on the fly.
Quoted 7 lines
Quoted 2 lines
The rebalancing moves data/metadata to newly created chunks. If there are two devices, the new chunks will be set up as RAID-1 by default.
Very interesting! I didn't realise that - I was presuming that the
BTRFS_BLOCK_GROUP_RAID1ioctl needed to be passed through (as at mkfs time) to change it into a RAID1 array.
See __btrfs_reserve_extent and __btrfs_alloc_chunk. There is a mistake in my previous reply. Only metadata chunks are set up as RAID-1 by default when there are two devices.
Regards
Yan Zheng
Quoted 14 lines · Yan Zheng
2008/12/29 Chris Samuel <[email protected]>:
Quoted 13 lines
Hello Yan,
Quoted 8 lines · Yan Zheng
2008/12/29 Chris Samuel <[email protected]>:
Quoted 3 lines
The rebalancing code does appear (from a naive read of the code) to be able to rebalance over stripes, but I have no idea if the disk format currently supports changing that on the fly.
The rebalancing moves data/metadata to newly created chunks. If there are two devices, the new chunks will be set up as RAID-1 by default.
Quoted 8 lines
Quoted 4 lines
Very interesting! I didn't realise that - I was presuming that the
BTRFS_BLOCK_GROUP_RAID1ioctl needed to be passed through (as at mkfs time) to change it into a RAID1 array.See
__btrfs_reserve_extentand__btrfs_alloc_chunk. There is a mistake in my previous reply. Only metadata chunks are set up as RAID-1 by default when there are two devices.
This gets confusing in a hurry, but the idea is to duplicate metadata by default. So, if you're using the default mount options on a single drive and add a second drive, it should switch metadata to raid1.
I've always planned on adding an option to btrfs-vol -b to have it change the data or metadata allocation policies (raidX to raidY, restripe etc). The kernel side code is all there, we just need something to wire it up to userland.
data duplication on a single spindle could work, I just haven't hooked it up because I couldn't think of a really strong reason to do it ;)
-chris
Quoted 11 lines · Yan Zheng
2008/12/29 Roland <[email protected]>:
Quoted 10 lines
Quoted 9 lines
Quoted 4 lines
so i can do btrfs-vol -r
/dev/sdbwhile it`s being mounted, pull the disk ,
replace it with a bigger one, rescan-scsi-bus, mkfs.btrfs the new disk and~~~~~~~~~~~~~~~~~~~~~~~
This step will fail, you
will
get a "/dev/sdbis mounted" by mkfs.btrfs, but for other slots it's ok.
Quoted 5 lines
Quoted 3 lines
isn`t this sort of a design issue then?
no way to work around this ?No, this is a (easy to fix) bug.
Right, the check for a mounted FS came after we made the device add/removal work ;) mkfs.btrfs should be changed to do the mounted FS check differently.
-chris
Quoted 3 lines · Chris Mason
This gets confusing in a hurry, but the idea is to duplicate metadata by default. So, if you're using the default mount options on a single drive and add a second drive, it should switch metadata to raid1.
Yup, that's what I inferred from your earlier posts and the code. :-)
Quoted 4 lines
I've always planned on adding an option to btrfs-vol -b to have it change the data or metadata allocation policies (raidX to raidY, restripe etc). The kernel side code is all there, we just need something to wire it up to userland.
Understood. If I get some Copious Spare Time (tm) I might take a look at that (though don't let that stop anyone else trying first!).
Quoted 2 lines
data duplication on a single spindle could work, I just haven't hooked it up because I couldn't think of a really strong reason to do it ;)
I think for SSD's it could be really handy, especially for people who have a standard distro install and decide they want to convert their ext3 partition
to btrfs using the tools. It's just a bit easier than getting them to
backup, repartition, make the filesystem and restore.
Downside of course is that those original blocks never get mirrored (unless something like btrfsck was extended to fix those things up).
cheers,
Chris
Signature · 4 lines
Chris Samuel : http://www.csamuel.org/ : Melbourne, VIC This email may come with a PGP signature as a file. Do not panic. For more info see: http://en.wikipedia.org/wiki/OpenPGP