Quota Ignored On write

DISCUSSION Kevin Brandstatter 2 participants 7 messages Jul 4, 2014 Btrfs file system

Share & tools

Kevin Brandstatter

author

2014-07-04 02:13 UTC #0 lore raw

basing of the latest for-linus branch i found i can write way more than the quota

btrfs quota enable
btrfs subvolume create test
btrfs qgruop limit 1G test
dd if=/dev/zero of=test/file bs=1024 count=1500000 output:
1500000+0 records in
1500000+0 records out
1536000000 bytes (1.5 GB) copied, 5.91909 s, 259 MB/s

thats a full half gig over the quota limit. I noticed some changes to the quota
accounting in the logs, what changed that could cause this?

-Kevin Brandstatter

Satoru Takeuchi

2014-07-04 02:21 UTC #1 lore raw

Hi Kevin,

Quoted 12 lines · Kevin Brandstatter

basing of the latest for-linus branch i found i can write way more than the quota
btrfs quota enable
btrfs subvolume create test
btrfs qgruop limit 1G test
dd if=/dev/zero of=test/file bs=1024 count=1500000 output:
1500000+0 records in
1500000+0 records out
1536000000 bytes (1.5 GB) copied, 5.91909 s, 259 MB/s

Quoted 3 lines

thats a full half gig over the quota limit. I noticed some changes to the quota
accounting in the logs, what changed that could cause this?

Do you remember what kernel version quota worked correctly?

Thanks,
Satoru

Quoted 7 lines

-Kevin Brandstatter
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

Kevin Brandstatter

author

2014-07-04 02:26 UTC #2 lore raw

3.15.3 via arch/ and from linux-git

-Kevin

Quoted 33 lines · Satoru Takeuchi

Hi Kevin,
Quoted 12 lines · Kevin Brandstatter
basing of the latest for-linus branch i found i can write way more than the quota
btrfs quota enable
btrfs subvolume create test
btrfs qgruop limit 1G test
dd if=/dev/zero of=test/file bs=1024 count=1500000 output:
1500000+0 records in
1500000+0 records out
1536000000 bytes (1.5 GB) copied, 5.91909 s, 259 MB/s
Quoted 3 lines
thats a full half gig over the quota limit. I noticed some changes to the quota
accounting in the logs, what changed that could cause this?
Do you remember what kernel version quota worked correctly?
Thanks,
Satoru
Quoted 8 lines
-Kevin Brandstatter
Signature · 4 lines
To unsubscribe from this list: send the line "unsubscribe
linux-btrfs" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Satoru Takeuchi

2014-07-04 02:33 UTC #3 lore raw

Quoted 1 line · Kevin Brandstatter

3.15.3 via arch/ and from linux-git

OK, I'll bisect it.

Satoru

Quoted 43 lines

-Kevin
Quoted 15 lines · Satoru Takeuchi
Hi Kevin,
Quoted 12 lines · Kevin Brandstatter
basing of the latest for-linus branch i found i can write way more than the quota
btrfs quota enable
btrfs subvolume create test
btrfs qgruop limit 1G test
dd if=/dev/zero of=test/file bs=1024 count=1500000 output:
1500000+0 records in
1500000+0 records out
1536000000 bytes (1.5 GB) copied, 5.91909 s, 259 MB/s
Quoted 18 lines
Quoted 3 lines
thats a full half gig over the quota limit. I noticed some changes to the quota
accounting in the logs, what changed that could cause this?
Do you remember what kernel version quota worked correctly?
Thanks,
Satoru
Quoted 8 lines
-Kevin Brandstatter
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

Kevin Brandstatter

author

2014-07-04 02:35 UTC #6 lore raw

In reply to Satoru Takeuchi

hmm, is it possible that btrfs is doing some deduplication or compression? The expected behavior works fine with small quotas like 10/20MB but at 1GB i can overwrite quite a bit from /dev/zero
I also tried to dd from /dev/urandom (to get some variety other than zeros) dd if=/dev/urandom of=meow bs=1024 count=1500000
output:
dd: error writing ‘meow’: Disk quota exceeded
1163330+0 records in
1163329+0 records out
1191248896 bytes (1.2 GB) copied, 110.25 s, 10.8 MB/s

So it looks like its stopping the write, but with a 1GB quota, thats a 20% over quota

-Kevin

Quoted 33 lines · Satoru Takeuchi

Hi Kevin,
Quoted 12 lines · Kevin Brandstatter
basing of the latest for-linus branch i found i can write way more than the quota
btrfs quota enable
btrfs subvolume create test
btrfs qgruop limit 1G test
dd if=/dev/zero of=test/file bs=1024 count=1500000 output:
1500000+0 records in
1500000+0 records out
1536000000 bytes (1.5 GB) copied, 5.91909 s, 259 MB/s
Quoted 3 lines
thats a full half gig over the quota limit. I noticed some changes to the quota
accounting in the logs, what changed that could cause this?
Do you remember what kernel version quota worked correctly?
Thanks,
Satoru
Quoted 8 lines
-Kevin Brandstatter
Signature · 4 lines
To unsubscribe from this list: send the line "unsubscribe
linux-btrfs" in
the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

Satoru Takeuchi

2014-07-04 03:25 UTC #4 lore raw

In reply to Satoru Takeuchi

Hi Chris and Kevin,

Quoted 21 lines

Quoted 21 lines · Satoru Takeuchi
Quoted 20 lines
Hi Kevin,
Quoted 12 lines · Kevin Brandstatter
basing of the latest for-linus branch i found i can write way more than the quota
btrfs quota enable
btrfs subvolume create test
btrfs qgruop limit 1G test
dd if=/dev/zero of=test/file bs=1024 count=1500000 output:
1500000+0 records in
1500000+0 records out
1536000000 bytes (1.5 GB) copied, 5.91909 s, 259 MB/s
Quoted 3 lines
thats a full half gig over the quota limit. I noticed some changes to the quota
accounting in the logs, what changed that could cause this?
Do you remember what kernel version quota worked correctly?

Quoted 4 lines · Satoru Takeuchi

Quoted 1 line · Kevin Brandstatter
3.15.3 via arch/ and from linux-git
OK, I'll bisect it.

I made the following reproducer based on your operation. It succeeded with 3.15 and failed with 3.16-rc3. So, the problematic patch is not in mason/for-linux branch, but in somewhere between 3.15 and 3.16-rc3. Please wait for a while to finish my bisect...

===============================================================================
#!/bin/bash -x

TEST_DEV=/dev/vdb
TEST_MNT=/home/sat/mnt

umount $TEST_MNT
mkfs.btrfs -f $TEST_DEV
mount $TEST_DEV $TEST_MNT
btrfs quota enable $TEST_MNT

SUBVOLPATH=$TEST_MNT/quota_test LIMIT=$((1024*1000000))
btrfs subvolume create $SUBVOLPATH
btrfs qgroup limit $LIMIT $SUBVOLPATH TESTFILE=$SUBVOLPATH/test
dd if=/dev/zero of=$TESTFILE bs=1024 count=$(($LIMIT*3/2/1024)) SIZE=$(($(ls -s $TESTFILE | awk '{print $1}')*1024))

RET=0
if [ $SIZE -le $LIMIT ] ; then
echo "[PASS] quota works correctly" >&2
else
echo "[FAIL] quota doesn't work" >&2
RET=1
fi

exit $RET
===============================================================================

Thanks,
Satoru

Quoted 24 lines

Quoted 14 lines
Quoted 13 lines
Thanks,
Satoru
Quoted 8 lines
-Kevin Brandstatter
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Quoted 5 lines
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

Satoru Takeuchi

2014-07-04 05:17 UTC #5 lore raw

Hi Josef, Chris, Kevin,

Quoted 25 lines · Satoru Takeuchi

Hi Chris and Kevin,
Quoted 21 lines
Quoted 21 lines · Satoru Takeuchi
Quoted 20 lines
Hi Kevin,
Quoted 12 lines · Kevin Brandstatter
basing of the latest for-linus branch i found i can write way more than the quota
btrfs quota enable
btrfs subvolume create test
btrfs qgruop limit 1G test
dd if=/dev/zero of=test/file bs=1024 count=1500000 output:
1500000+0 records in
1500000+0 records out
1536000000 bytes (1.5 GB) copied, 5.91909 s, 259 MB/s
Quoted 3 lines
thats a full half gig over the quota limit. I noticed some changes to the quota
accounting in the logs, what changed that could cause this?
Do you remember what kernel version quota worked correctly?
Quoted 1 line · Satoru Takeuchi
(2014/07/04 11:25), Kevin Brandstatter wrote:

Quoted 8 lines

Quoted 3 lines
Quoted 1 line
3.15.3 via arch/ and from linux-git
OK, I'll bisect it.
I made the following reproducer based on your operation. It succeeded with 3.15 and failed with 3.16-rc3. So, the problematic patch is not in mason/for-linux branch, but in somewhere between 3.15 and 3.16-rc3. Please wait for a while to finish my bisect...

I bisected and found the bad commit is the following patch.

===============================================================================
commit fcebe4562dec83b3f8d3088d77584727b09130b2

Author: Josef Bacik <[email protected]>
Date:   Tue May 13 17:30:47 2014 -0700

Btrfs: rework qgroup accounting
===============================================================================

Josef, please take a look at this patch.

Thanks,
Satoru

Quoted 63 lines

===============================================================================
#!/bin/bash -x
TEST_DEV=/dev/vdb
TEST_MNT=/home/sat/mnt
umount $TEST_MNT
mkfs.btrfs -f $TEST_DEV
mount $TEST_DEV $TEST_MNT
btrfs quota enable $TEST_MNT
SUBVOLPATH=$TEST_MNT/quota_test LIMIT=$((1024*1000000))
btrfs subvolume create $SUBVOLPATH
btrfs qgroup limit $LIMIT $SUBVOLPATH TESTFILE=$SUBVOLPATH/test
dd if=/dev/zero of=$TESTFILE bs=1024 count=$(($LIMIT*3/2/1024)) SIZE=$(($(ls -s $TESTFILE | awk '{print $1}')*1024))
RET=0
if [ $SIZE -le $LIMIT ] ; then
echo "[PASS] quota works correctly" >&2
else
echo "[FAIL] quota doesn't work" >&2
RET=1
fi
exit $RET
===============================================================================
Thanks,
Satoru
Quoted 20 lines
Quoted 14 lines
Quoted 13 lines
Thanks,
Satoru
Quoted 8 lines
-Kevin Brandstatter
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Quoted 5 lines
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Quoted 4 lines
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html