Skip to contentSkip to conversation

btrfs raid1 balance slow

DISCUSSION kjansen387 3 participants 5 messages Jan 20, 2020 – Jan 22, 2020 Btrfs file system

lore
Share & tools
lore

kjansen387

author
#0 lore raw

Hello,

I had a btrfs raid1 with 2 4TB SATA 5400RPM disks . Regular disk I/O is about 2MB/sec per drive, ~40IOPS, mostly write. I had ~150GB free and added one 2 TB disk and started the balance:

btrfs device add -f /dev/sdb /export
btrfs filesystem balance /export

It's now running for 24 hours, 70% remaining:

# btrfs balance status -v /export
Balance on '/export' is running
1057 out of about 3561 chunks balanced (1058 considered),  70% left
Dumping filters: flags 0x7, state 0x1, force is off
  DATA (flags 0x0): balancing
  METADATA (flags 0x0): balancing
  SYSTEM (flags 0x0): balancing

I have searched for similar cases, but, I do not have quotas enabled, I do not have compression enabled, and my CPU supports sse4_2 . CPU (i7-8700K) is doing fine, 80% idle (average over all threads).

Is this normal ? I have to repeat this process 2 times (adding more 2TB disks), any way I can make it faster ?

Thanks!
Klaas

#1 lore raw
Quoted 17 lines · On Mon, Jan 20, 2020 at 11:08:40PM +0100, kjansen387 wrote:

I had a btrfs raid1 with 2 4TB SATA 5400RPM disks . Regular disk I/O is about 2MB/sec per drive, ~40IOPS, mostly write. I had ~150GB free and added one 2 TB disk and started the balance:

btrfs device add -f /dev/sdb /export
btrfs filesystem balance /export

It's now running for 24 hours, 70% remaining:

# btrfs balance status -v /export
Balance on '/export' is running
1057 out of about 3561 chunks balanced (1058 considered),  70% left
Dumping filters: flags 0x7, state 0x1, force is off
  DATA (flags 0x0): balancing
  METADATA (flags 0x0): balancing
  SYSTEM (flags 0x0): balancing
Quoted 3 lines

I have searched for similar cases, but, I do not have quotas enabled, I do not have compression enabled, and my CPU supports sse4_2 . CPU (i7-8700K) is doing fine, 80% idle (average over all threads).

Do you have lots of snapshots? It can take a lot of time on some of
the metadata chunks if there's lots of shared extents.

Quoted 2 lines

Is this normal ? I have to repeat this process 2 times (adding more 2TB disks), any way I can make it faster ?

Cancel the current balance, add the remaining disks, and then
balance only once you've added them all.

Hugo.

Signature · 4 lines
Hugo Mills             | If it's December 1941 in Casablanca, what time is it
hugo@... carfax.org.uk | in New York?
http://carfax.org.uk/  |
PGP: E2AB1DE4          |                               Rick Blaine, Casablanca
#2 lore raw
In reply to kjansen387
Quoted 19 lines · On 2020/1/21 上午6:08, kjansen387 wrote:

Hello,

I had a btrfs raid1 with 2 4TB SATA 5400RPM disks . Regular disk I/O is about 2MB/sec per drive, ~40IOPS, mostly write. I had ~150GB free and added one 2 TB disk and started the balance:

btrfs device add -f /dev/sdb /export
btrfs filesystem balance /export

It's now running for 24 hours, 70% remaining:

# btrfs balance status -v /export
Balance on '/export' is running
1057 out of about 3561 chunks balanced (1058 considered),  70% left
Dumping filters: flags 0x7, state 0x1, force is off
  DATA (flags 0x0): balancing
  METADATA (flags 0x0): balancing
  SYSTEM (flags 0x0): balancing
Quoted 3 lines

I have searched for similar cases, but, I do not have quotas enabled, I do not have compression enabled, and my CPU supports sse4_2 . CPU (i7-8700K) is doing fine, 80% idle (average over all threads).

Are you using qgroup?

If so, that would be the cause.
Please use kernel newer than v5.2, which contains the optimization.

Quoted 3 lines

Is this normal ? I have to repeat this process 2 times (adding more 2TB disks), any way I can make it faster ?

Just as Hugo said, cancel the current one, add all device, then rebalance.

Thanks,
Qu

Quoted 4 lines

Thanks!
Klaas

author
#3 lore raw
In reply to kjansen387

Forgot to mention; I'm running kernel 5.4.10-200.fc31.x86_64 .

Op 20-Jan-20 om 23:08 schreef kjansen387:

Quoted 24 lines

Hello,

I had a btrfs raid1 with 2 4TB SATA 5400RPM disks . Regular disk I/O is about 2MB/sec per drive, ~40IOPS, mostly write. I had ~150GB free and added one 2 TB disk and started the balance:

btrfs device add -f /dev/sdb /export
btrfs filesystem balance /export

It's now running for 24 hours, 70% remaining:

# btrfs balance status -v /export
Balance on '/export' is running
1057 out of about 3561 chunks balanced (1058 considered),  70% left
Dumping filters: flags 0x7, state 0x1, force is off
   DATA (flags 0x0): balancing
   METADATA (flags 0x0): balancing
   SYSTEM (flags 0x0): balancing

I have searched for similar cases, but, I do not have quotas enabled, I do not have compression enabled, and my CPU supports sse4_2 . CPU (i7-8700K) is doing fine, 80% idle (average over all threads).

Is this normal ? I have to repeat this process 2 times (adding more 2TB disks), any way I can make it faster ?

Thanks!
Klaas

author
#4 lore raw

Hmm, for some reason, I did not get the mail from Hugo and Qu (had too look it up in the archive). I think there is a delay when subscribing to the email list.

Quoted 1 line

Do you have lots of snapshots? It can take a lot of time on some of

the metadata chunks if there's lots of shared extents.

No, I do not have any snapshots. Never made one - just got started with btrfs.

# btrfs subvolume show /export | grep Snap
        Snapshot(s):
Quoted 1 line

Are you using qgroup?

No, quotas are disabled:

# btrfs qgroup show /export
ERROR: can't list qgroups: quotas not enabled
Quoted 1 line

Please use kernel newer than v5.2, which contains the optimization.

I'm running 5.4.10-200.fc31.x86_64, that should be good

Quoted 1 line

Just as Hugo said, cancel the current one, add all device, then rebalance.

Yes, that would be the best. I'm moving data however.. Expand btrfs > move the content of a 2TB disk > use that disk to expand btrfs > move the content of a 2TB disk etc. When this balance is done I'll move 2TB, find a temp drive to store the other 2TB and add the remaining 2 drives in 1 go.

I'm not in a hurry.. just wondering if this is normal. As I had 2x4TB with 150GB free and am adding 1x2TB (~1800GB, ~900 after raid1) I would expect that I will end up with 150+900=1050 GB free (usable), spread like (approx.):
4TB disk 1: 420GB (previously: 75GB)
4TB disk 2: 420GB (previously: 75GB)
2TB disk: 210GB

Thinking very simplisticly -and that's probably where I'm wrong, just wondering if it's a factor 8 wrong- this would mean 420-75 = 345GB read from each 4TB disks and written to the new 2TB disk. At a rate of 20MB/s (pretty conservative, but I do have a little bit of load) this should take like 10 hours. It will, however, at this rate, take approx. 82 hours. Even if all data needs to be read (does it? ~3500GB net) it should only take ~50 hours..

Klaas