possible bug in balance

Hi,
I have today added one device and I have converted metadata to raid1. Than I wanted to convert to raid1 also some data (with balance filter) and try if there is some speedup when reading files (starting programs)... I have issued this command:

luvar@blackdawn:~$ sudo time btrfs balance start -dconvert=raid1 -dusage=20 /home/luvar/programs/
[sudo] password for luvar: 
ERROR: error during balancing '/home/luvar/programs/' - Input/output error
There may be more info in syslog - try dmesg | tail
Command exited with non-zero status 19
0.00user 0.08system 0:08.29elapsed 1%CPU (0avgtext+0avgdata 768maxresident)k
14696inputs+6584outputs (2major+253minor)pagefaults 0swaps

Part of df command:

luvar@blackdawn:~$ df -h
Filesystem                   Size  Used Avail Use% Mounted on
/dev/sdb2                    458G  177G   59G  75% /
/dev/sdb2                    458G  177G   59G  75% /home
/dev/sdb1                    226M   96M  114M  46% /boot
/dev/sdb2                    458G  177G   59G  75% /home/luvar/eclipseWorkspaceAndroid
/dev/sdb2                    458G  177G   59G  75% /home/luvar/eclipseWorkspaceErlang
/dev/sdb2                    458G  177G   59G  75% /home/luvar/programs

root@blackdawn:/home/luvar# dmesg|tail -n 50
[ 8107.693414] attempt to access beyond end of device
[ 8107.693425] sdb2: rw=32, want=480102272, limit=473956352
[ 8107.711854] attempt to access beyond end of device
[ 8107.711863] sdb2: rw=1041, want=480102272, limit=473956352
[ 8107.771103] attempt to access beyond end of device
[ 8107.771114] sdb2: rw=32, want=482410504, limit=473956352
[ 8107.784037] attempt to access beyond end of device
[ 8107.784045] sdb2: rw=1041, want=482410504, limit=473956352
[ 8107.804923] attempt to access beyond end of device
[ 8107.804933] sdb2: rw=32, want=478657496, limit=473956352
[ 8107.817134] attempt to access beyond end of device
[ 8107.817142] sdb2: rw=1041, want=478657496, limit=473956352
[ 8107.835377] attempt to access beyond end of device
[ 8107.835384] sdb2: rw=32, want=480795752, limit=473956352
[ 8107.842977] attempt to access beyond end of device
[ 8107.842985] sdb2: rw=1041, want=480795752, limit=473956352
[ 8107.887768] attempt to access beyond end of device
[ 8107.887778] sdb2: rw=32, want=478931480, limit=473956352
[ 8107.898939] attempt to access beyond end of device
[ 8107.898946] sdb2: rw=1041, want=478931480, limit=473956352
[ 8107.958691] attempt to access beyond end of device
[ 8107.958699] sdb2: rw=32, want=479426840, limit=473956352
[ 8107.966368] attempt to access beyond end of device
[ 8107.966375] sdb2: rw=1041, want=479426840, limit=473956352
[ 8116.097908] attempt to access beyond end of device
[ 8116.097919] sdb2: rw=32, want=478334096, limit=473956352
[ 8116.097923] btrfs_dev_stat_print_on_error: 12 callbacks suppressed
[ 8116.097926] btrfs: bdev /dev/sdb2 errs: wr 638625, rd 65863, flush 0, corrupt 0, gen 0
[ 8116.133108] attempt to access beyond end of device
[ 8116.133118] sdb2: rw=1041, want=478334096, limit=473956352
[ 8116.133124] btrfs: bdev /dev/sdb2 errs: wr 638626, rd 65863, flush 0, corrupt 0, gen 0
[ 8125.065061] attempt to access beyond end of device
[ 8125.065073] sdb2: rw=32, want=481418928, limit=473956352
[ 8125.065077] btrfs: bdev /dev/sdb2 errs: wr 638626, rd 65864, flush 0, corrupt 0, gen 0
[ 8125.084522] attempt to access beyond end of device
[ 8125.084533] sdb2: rw=1041, want=481418928, limit=473956352
[ 8125.084539] btrfs: bdev /dev/sdb2 errs: wr 638627, rd 65864, flush 0, corrupt 0, gen 0
[ 8131.848768] btrfs: relocating block group 472710643712 flags 1
[ 8133.866427] attempt to access beyond end of device
[ 8133.866436] sdb2: rw=0, want=476739152, limit=473956352
[ 8133.866441] btrfs: bdev /dev/sdb2 errs: wr 638627, rd 65865, flush 0, corrupt 0, gen 0
[ 8133.866516] attempt to access beyond end of device
[ 8133.866520] sdb2: rw=0, want=476739152, limit=473956352
[ 8133.866523] btrfs: bdev /dev/sdb2 errs: wr 638627, rd 65866, flush 0, corrupt 0, gen 0
[ 8159.272179] attempt to access beyond end of device
[ 8159.272191] sdb2: rw=32, want=480110048, limit=473956352
[ 8159.272196] btrfs: bdev /dev/sdb2 errs: wr 638627, rd 65867, flush 0, corrupt 0, gen 0
[ 8159.300427] attempt to access beyond end of device
[ 8159.300434] sdb2: rw=1041, want=480110048, limit=473956352
[ 8159.300440] btrfs: bdev /dev/sdb2 errs: wr 638628, rd 65867, flush 0, corrupt 0, gen 0

root@blackdawn:/home/luvar# uname -a
Linux blackdawn 3.13.0-30-generic #55-Ubuntu SMP Fri Jul 4 21:40:53 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux

root@blackdawn:/home/luvar# btrfs v
Btrfs v0.20-rc1-189-g704a08c

Am I doing something forbidden (I have not see any structure where raid type is stored per file/subvolume item), or I just hit some problem? What should I try?

PS: After all I will convert all data to raid1, but I want to play first :-)

luvar posted on Fri, 02 Jan 2015 15:42:29 +0100 as excerpted:

Those versions are your problem. Do you know how fast btrfs is improving, how old those are (hint, there are about five kernel release series a year and we're now on the 3.19 development series with 3.18 stable, so 3.13 is... about five releases back and thus about a year old! ), and how many bugs have been fixed since then?

Try something newer than the paleolithic age, say a current 3.18.x stable series kernel or a 3.19-rc, and the recent btrfs-progs 3.18.0, and you should see rather better results. There are valid reasons to stick to old and stable, but they simply don't mix well with a new and still not completely stable filesystem that still sees regular bug fixes, and not just for code introduced in the last kernel, either! Therefore, if you want old and stable, choose a different filesystem; while if you want to run btrfs, staying current is strongly recommended.

Meanwhile, if you haven't yet, you may want to spend some time on the wiki reading up. Take a look at the stuff under section 5.1, Guides and usage information, especially. It'll answer a lot of questions and could save your data too. =:^)

https://btrfs.wiki.kernel.org

Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

[email protected] writes:

Hello,

btrfs doesn't yet support per-subvolume RAID1 levels. I'm not sure how it should behave with your command line. It probably tries to rebalance the whole filesystem.

Though I can already tell that no, there won't be a speedup, as btrfs scheduler chooses the device to access by using the process id as a seed. Therefore a single thread is never able to use 100% RAID1 input capability. Perhaps in future there will be more sophisticated schedulers. You may try to use MD raid1 for extra speed, but you would lose the automatic error recovery of btrfs (but you would still notice if data gets corrupted).

I have noticed that 'attempt to access beyond end of device' typically indicates (with other file systems, I haven't seen that with btrfs) that the partition table and the filesystem size don't match. Typically such a situation could occur when one modifies partition table after creating the file system, though I'm sure there are other ways to get into such a situation. You may find the filesystem size with btrfs filesystem show and partition sizes with cat /proc/partitions (multiply by block size = 1024 bytes).

Should the partition sizes and filesystem sizes match, I would be quite certain this would indeed be a btrfs bug. But,

should this turn out to be a bug, I'm certain trying a more recent kernel version is a terrific idea :). 3.18.x or 3.17.y where y>2 (I think those were the two versions that were bad in 3.17 series..). They won't have support for raid1'n a subvolume either, though, as far as I know.

Remember backups :).

  _____________________________________________________________________
     / __// /__ ____  __               http://www.modeemi.fi/~flux/\   \
    / /_ / // // /\ \/ /                                            \  /
   /_/  /_/ \___/ /_/\_\@modeemi.fi                                  \/