Exclusive quota of snapshot exceeded despite no space used

I'm having a weird problem with snapshots and exclusive quotas. After creating a snapshot of a subvolume and setting an exclusive quota of 50MB for the snapshot, everything seems to work fine. I can write approximately 50MB before the quota kicks in.

However, if I create a snapshot, set an exclusive quota and just wait for some time, I suddenly cannot even create an empty file because I'm getting a "quota exceeded" error. The time until the bug appears seems to vary. During the waiting time, I'm changing neither the snapshot nor the original subvolume. "qgroup show -e" reports an exclusive use of only a few kilobytes for the snapshot, which is nowhere near the limit.

Steps to reproduce (/media/extern is a fresh and empty btrfs partition):

Enable quota and create an empty subvolume:

     root@t420:/media/extern# btrfs quota enable .
     root@t420:/media/extern# btrfs subvolume create sub
     Create subvolume './sub'

Snapshot the subvolume and set a limit:

     root@t420:/media/extern# btrfs subvolume snapshot sub snap
     Create a snapshot of 'sub' in './snap'
     root@t420:/media/extern# cd snap/
     root@t420:/media/extern/snap# btrfs qgroup limit -e 50M .

Sometimes it takes "longer" for the quota to kick in, so I'm touching a file every 5 minutes here:

     root@t420:/media/extern/snap# for file in {1..100}; do touch $file;
sleep 5m; done
     touch: cannot touch ‘7’: Disk quota exceeded
     ^C
     root@t420:/media/extern/snap# btrfs qgroup show -e .
     qgroupid         rfer         excl     max_excl
     --------         ----         ----     --------
     0/5          16.00KiB     16.00KiB         none
     0/257        16.00KiB     16.00KiB         none
     0/258        16.00KiB     16.00KiB     50.00MiB

Any idea why this happens?

Thanks,
Johannes

System info:

     Linux t420 4.3.0-rc5 #1 SMP Tue Oct 13 13:21:02 CEST 2015 x86_64

GNU/Linux

     Label: none  uuid: 9551e3ca-1608-469c-9d8c-77b99ce0e8ec
         Total devices 1 FS bytes used 816.00KiB
         devid    1 size 931.51GiB used 2.04GiB path /dev/sdb1

     btrfs-progs v4.1.2

     Data, single: total=8.00MiB, used=256.00KiB
     System, DUP: total=8.00MiB, used=16.00KiB
     System, single: total=4.00MiB, used=0.00B
     Metadata, DUP: total=1.00GiB, used=544.00KiB
     Metadata, single: total=8.00MiB, used=0.00B
     GlobalReserve, single: total=16.00MiB, used=0.00B

     [249174.151820]  sdb: sdb1
     [249184.387377]  sdb: sdb1
     [249184.573096]  sdb: sdb1
     [249184.656274] BTRFS: device fsid
9551e3ca-1608-469c-9d8c-77b99ce0e8ec devid 1 transid 3 /dev/sdb1
     [249186.323915]  sdb: sdb1
     [249186.534505]  sdb: sdb1
     [249186.538420]  sdb: sdb1
     [249196.781978] BTRFS info (device sdb1): disk space caching is enabled
     [249196.781986] BTRFS: has skinny extents
     [249196.781990] BTRFS: flagging fs with big metadata feature
     [249196.818164] BTRFS: creating UUID tree
     [249202.311983] BTRFS info (device sdb1): qgroup scan completed

(inconsistency flag cleared)

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

Quoted 39 lines · Johannes Henninger

I'm having a weird problem with snapshots and exclusive quotas. After creating a snapshot of a subvolume and setting an exclusive quota of 50MB for the snapshot, everything seems to work fine. I can write approximately 50MB before the quota kicks in.

However, if I create a snapshot, set an exclusive quota and just wait for some time, I suddenly cannot even create an empty file because I'm getting a "quota exceeded" error. The time until the bug appears seems to vary. During the waiting time, I'm changing neither the snapshot nor the original subvolume. "qgroup show -e" reports an exclusive use of only a few kilobytes for the snapshot, which is nowhere near the limit.

Steps to reproduce (/media/extern is a fresh and empty btrfs partition):

Enable quota and create an empty subvolume:

     root@t420:/media/extern# btrfs quota enable .
     root@t420:/media/extern# btrfs subvolume create sub
     Create subvolume './sub'

Snapshot the subvolume and set a limit:

     root@t420:/media/extern# btrfs subvolume snapshot sub snap
     Create a snapshot of 'sub' in './snap'
     root@t420:/media/extern# cd snap/
     root@t420:/media/extern/snap# btrfs qgroup limit -e 50M .

Sometimes it takes "longer" for the quota to kick in, so I'm touching a file every 5 minutes here:

     root@t420:/media/extern/snap# for file in {1..100}; do touch $file;
sleep 5m; done
     touch: cannot touch ‘7’: Disk quota exceeded
     ^C
     root@t420:/media/extern/snap# btrfs qgroup show -e .
     qgroupid         rfer         excl     max_excl
     --------         ----         ----     --------
     0/5          16.00KiB     16.00KiB         none
     0/257        16.00KiB     16.00KiB         none
     0/258        16.00KiB     16.00KiB     50.00MiB

Quoted 1 line

Any idea why this happens?

BTW, to make btrfs qgroup show work, it's better to call sync before qgroup show.

It's a known bug that even after qgroup accounting rework, qgroup reserve still has bug and can cause reserved space to underflow, making such problem happen.

For such case, btrfs qgroup show won't help as reserved space is not shown in the output.

One workaround would be, umount the filesystem and mount again. Which will reset the underflow reserved space and work for sometime.

If it's OK for you to recompile the kernel, you can try the following patchset:
[PATCH v3 00/21] Rework btrfs qgroup reserved space framework

Which should solve the problem.

Thanks,
Qu

Quoted 54 lines · Qu Wenruo

Quoted 39 lines · Johannes Henninger

I'm having a weird problem with snapshots and exclusive quotas. After creating a snapshot of a subvolume and setting an exclusive quota of 50MB for the snapshot, everything seems to work fine. I can write approximately 50MB before the quota kicks in.

However, if I create a snapshot, set an exclusive quota and just wait for some time, I suddenly cannot even create an empty file because I'm getting a "quota exceeded" error. The time until the bug appears seems to vary. During the waiting time, I'm changing neither the snapshot nor the original subvolume. "qgroup show -e" reports an exclusive use of only a few kilobytes for the snapshot, which is nowhere near the limit.

Steps to reproduce (/media/extern is a fresh and empty btrfs partition):

Enable quota and create an empty subvolume:

      root@t420:/media/extern# btrfs quota enable .
      root@t420:/media/extern# btrfs subvolume create sub
      Create subvolume './sub'

Snapshot the subvolume and set a limit:

      root@t420:/media/extern# btrfs subvolume snapshot sub snap
      Create a snapshot of 'sub' in './snap'
      root@t420:/media/extern# cd snap/
      root@t420:/media/extern/snap# btrfs qgroup limit -e 50M .

Sometimes it takes "longer" for the quota to kick in, so I'm touching a file every 5 minutes here:

      root@t420:/media/extern/snap# for file in {1..100}; do touch $file;
sleep 5m; done
      touch: cannot touch ‘7’: Disk quota exceeded
      ^C
      root@t420:/media/extern/snap# btrfs qgroup show -e .
      qgroupid         rfer         excl     max_excl
      --------         ----         ----     --------
      0/5          16.00KiB     16.00KiB         none
      0/257        16.00KiB     16.00KiB         none
      0/258        16.00KiB     16.00KiB     50.00MiB

Quoted 1 line

Any idea why this happens?

BTW, to make btrfs qgroup show work, it's better to call sync before qgroup show.

It's a known bug that even after qgroup accounting rework, qgroup reserve still has bug and can cause reserved space to underflow, making such problem happen.

For such case, btrfs qgroup show won't help as reserved space is not shown in the output.

One workaround would be, umount the filesystem and mount again. Which will reset the underflow reserved space and work for sometime.

Quoted 9 lines

If it's OK for you to recompile the kernel, you can try the following patchset:
[PATCH v3 00/21] Rework btrfs qgroup reserved space framework

Which should solve the problem.

Thanks,
Qu

Thanks a lot for your reply!

While remounting the filesystem fixes the issue temporary, it doesn't take very long for the bug to happen again so it's not really a workaround I can work with.

I did recompile the kernel using your patches, but unfortunately the problem still appears.

Thanks,
Johannes

Quoted 76 lines · Johannes Henninger

Quoted 59 lines · Qu Wenruo

Quoted 44 lines · Johannes Henninger

I'm having a weird problem with snapshots and exclusive quotas. After creating a snapshot of a subvolume and setting an exclusive quota of 50MB for the snapshot, everything seems to work fine. I can write approximately 50MB before the quota kicks in.

However, if I create a snapshot, set an exclusive quota and just wait for some time, I suddenly cannot even create an empty file because I'm getting a "quota exceeded" error. The time until the bug appears seems to vary. During the waiting time, I'm changing neither the snapshot nor
the original subvolume. "qgroup show -e" reports an exclusive use of only a few kilobytes for the snapshot, which is nowhere near the limit.

Steps to reproduce (/media/extern is a fresh and empty btrfs partition):

Enable quota and create an empty subvolume:

      root@t420:/media/extern# btrfs quota enable .
      root@t420:/media/extern# btrfs subvolume create sub
      Create subvolume './sub'

Snapshot the subvolume and set a limit:

      root@t420:/media/extern# btrfs subvolume snapshot sub snap
      Create a snapshot of 'sub' in './snap'
      root@t420:/media/extern# cd snap/
      root@t420:/media/extern/snap# btrfs qgroup limit -e 50M .

Sometimes it takes "longer" for the quota to kick in, so I'm touching a
file every 5 minutes here:

      root@t420:/media/extern/snap# for file in {1..100}; do touch
$file;
sleep 5m; done
      touch: cannot touch ‘7’: Disk quota exceeded
      ^C
      root@t420:/media/extern/snap# btrfs qgroup show -e .
      qgroupid         rfer         excl     max_excl
      --------         ----         ----     --------
      0/5          16.00KiB     16.00KiB         none
      0/257        16.00KiB     16.00KiB         none
      0/258        16.00KiB     16.00KiB     50.00MiB

Quoted 1 line

Any idea why this happens?

BTW, to make btrfs qgroup show work, it's better to call sync before qgroup show.

It's a known bug that even after qgroup accounting rework, qgroup reserve still has bug and can cause reserved space to underflow, making such problem happen.

For such case, btrfs qgroup show won't help as reserved space is not shown in the output.

One workaround would be, umount the filesystem and mount again. Which will reset the underflow reserved space and work for sometime.

Quoted 9 lines

If it's OK for you to recompile the kernel, you can try the following patchset:
[PATCH v3 00/21] Rework btrfs qgroup reserved space framework

Which should solve the problem.

Thanks,
Qu

Thanks a lot for your reply!

While remounting the filesystem fixes the issue temporary, it doesn't take very long for the bug to happen again so it's not really a workaround I can work with.

Quoted 6 lines

I did recompile the kernel using your patches, but unfortunately the problem still appears.

Thanks,
Johannes

Interesting, just touching file will cause EQUOTA is quite a big problem.

I'll try to reproduce it with my patchset and see what really caused the problem.
The problem seems to do with snapshot qgroup hacking.
But I'm not completely sure yet.

BTW, does "sync; btrfs qgroup show -prce" still show excl as 16K? 16K is the correct number with only 6 empty files, just in case.

Thanks,
Qu

Quoted 92 lines · Qu Wenruo

Quoted 76 lines · Johannes Henninger

Quoted 59 lines · Qu Wenruo

Quoted 44 lines · Johannes Henninger

I'm having a weird problem with snapshots and exclusive quotas. After creating a snapshot of a subvolume and setting an exclusive quota of 50MB for the snapshot, everything seems to work fine. I can write approximately 50MB before the quota kicks in.

However, if I create a snapshot, set an exclusive quota and just wait for some time, I suddenly cannot even create an empty file because I'm getting a "quota exceeded" error. The time until the bug appears seems to vary. During the waiting time, I'm changing neither the snapshot nor
the original subvolume. "qgroup show -e" reports an exclusive use of only a few kilobytes for the snapshot, which is nowhere near the limit.

Steps to reproduce (/media/extern is a fresh and empty btrfs partition):

Enable quota and create an empty subvolume:

       root@t420:/media/extern# btrfs quota enable .
       root@t420:/media/extern# btrfs subvolume create sub
       Create subvolume './sub'

Snapshot the subvolume and set a limit:

       root@t420:/media/extern# btrfs subvolume snapshot sub snap
       Create a snapshot of 'sub' in './snap'
       root@t420:/media/extern# cd snap/
       root@t420:/media/extern/snap# btrfs qgroup limit -e 50M .

Sometimes it takes "longer" for the quota to kick in, so I'm touching a
file every 5 minutes here:

       root@t420:/media/extern/snap# for file in {1..100}; do touch
$file;
sleep 5m; done
       touch: cannot touch ‘7’: Disk quota exceeded
       ^C
       root@t420:/media/extern/snap# btrfs qgroup show -e .
       qgroupid         rfer         excl     max_excl
       --------         ----         ----     --------
       0/5          16.00KiB     16.00KiB         none
       0/257        16.00KiB     16.00KiB         none
       0/258        16.00KiB     16.00KiB     50.00MiB

Quoted 1 line

Any idea why this happens?

BTW, to make btrfs qgroup show work, it's better to call sync before qgroup show.

It's a known bug that even after qgroup accounting rework, qgroup reserve still has bug and can cause reserved space to underflow, making such problem happen.

For such case, btrfs qgroup show won't help as reserved space is not shown in the output.

One workaround would be, umount the filesystem and mount again. Which will reset the underflow reserved space and work for sometime.

Quoted 9 lines

If it's OK for you to recompile the kernel, you can try the following patchset:
[PATCH v3 00/21] Rework btrfs qgroup reserved space framework

Which should solve the problem.

Thanks,
Qu

Thanks a lot for your reply!

While remounting the filesystem fixes the issue temporary, it doesn't take very long for the bug to happen again so it's not really a workaround I can work with.

Quoted 6 lines

I did recompile the kernel using your patches, but unfortunately the problem still appears.

Thanks,
Johannes

Interesting, just touching file will cause EQUOTA is quite a big problem.

I'll try to reproduce it with my patchset and see what really caused the problem.
The problem seems to do with snapshot qgroup hacking.
But I'm not completely sure yet.

Quoted 5 lines

BTW, does "sync; btrfs qgroup show -prce" still show excl as 16K? 16K is the correct number with only 6 empty files, just in case.

Thanks,
Qu

I ran my example from the first mail again and managed to write 7 files this time, "qgroup show" still shows 16kB after sync:

root@t420:/media/extern/snap# btrfs qg limit -e 50M .
root@t420:/media/extern/snap# for file in {1..100}; do touch $file;
sleep 5m; done
touch: cannot touch ‘8’: Disk quota exceeded
^C
root@t420:/media/extern/snap# sync
root@t420:/media/extern/snap# btrfs qgroup show -pcre .
qgroupid         rfer         excl     max_rfer     max_excl parent  child
--------         ----         ----     --------     -------- ------  -----
0/5          16.00KiB     16.00KiB         none         none ---     ---
0/257        16.00KiB     16.00KiB         none         none ---     ---
0/258        16.00KiB     16.00KiB         none     50.00MiB ---     ---

root@t420:/media/extern/snap# btrfs fi sync .
FSSync '.'
root@t420:/media/extern/snap# btrfs qgroup show -pcre .
qgroupid         rfer         excl     max_rfer     max_excl parent  child
--------         ----         ----     --------     -------- ------  -----
0/5          16.00KiB     16.00KiB         none         none ---     ---
0/257        16.00KiB     16.00KiB         none         none ---     ---
0/258        16.00KiB     16.00KiB         none     50.00MiB ---     ---

By the way, I don't if its relevant but the problem is not limited to exclusive quotas, but also happens when setting a "referenced" limit (qgroup limit without "-e").

Thanks,
Johannes

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

Quoted 20 lines

Quoted 7 lines

Thanks a lot for your reply!

While remounting the filesystem fixes the issue temporary, it doesn't take very long for the bug to happen again so it's not really a workaround I can work with.

Quoted 6 lines

I did recompile the kernel using your patches, but unfortunately the problem still appears.

Thanks,
Johannes

Interesting, just touching file will cause EQUOTA is quite a big problem.

I'll try to reproduce it with my patchset and see what really caused the problem.
The problem seems to do with snapshot qgroup hacking.
But I'm not completely sure yet.

Quoted 5 lines

BTW, does "sync; btrfs qgroup show -prce" still show excl as 16K? 16K is the correct number with only 6 empty files, just in case.

Thanks,
Qu

I ran my example from the first mail again and managed to write 7 files this time, "qgroup show" still shows 16kB after sync:

root@t420:/media/extern/snap# btrfs qg limit -e 50M .
root@t420:/media/extern/snap# for file in {1..100}; do touch $file;
sleep 5m; done
touch: cannot touch ‘8’: Disk quota exceeded
^C
root@t420:/media/extern/snap# sync
root@t420:/media/extern/snap# btrfs qgroup show -pcre .
qgroupid         rfer         excl     max_rfer     max_excl parent  child
--------         ----         ----     --------     -------- ------  -----
0/5          16.00KiB     16.00KiB         none         none ---     ---
0/257        16.00KiB     16.00KiB         none         none ---     ---
0/258        16.00KiB     16.00KiB         none     50.00MiB ---     ---
root@t420:/media/extern/snap# btrfs fi sync .
FSSync '.'
root@t420:/media/extern/snap# btrfs qgroup show -pcre .
qgroupid         rfer         excl     max_rfer     max_excl parent  child
--------         ----         ----     --------     -------- ------  -----
0/5          16.00KiB     16.00KiB         none         none ---     ---
0/257        16.00KiB     16.00KiB         none         none ---     ---
0/258        16.00KiB     16.00KiB         none     50.00MiB ---     ---

By the way, I don't if its relevant but the problem is not limited to exclusive quotas, but also happens when setting a "referenced" limit (qgroup limit without "-e").

Thanks,
Johannes

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

Quoted 55 lines

Quoted 21 lines

Quoted 7 lines

Thanks a lot for your reply!

While remounting the filesystem fixes the issue temporary, it doesn't take very long for the bug to happen again so it's not really a workaround I can work with.

Quoted 6 lines

I did recompile the kernel using your patches, but unfortunately the problem still appears.

Thanks,
Johannes

Interesting, just touching file will cause EQUOTA is quite a big problem.

I'll try to reproduce it with my patchset and see what really caused the problem.
The problem seems to do with snapshot qgroup hacking.
But I'm not completely sure yet.

Quoted 5 lines

BTW, does "sync; btrfs qgroup show -prce" still show excl as 16K? 16K is the correct number with only 6 empty files, just in case.

Thanks,
Qu

I ran my example from the first mail again and managed to write 7 files this time, "qgroup show" still shows 16kB after sync:

root@t420:/media/extern/snap# btrfs qg limit -e 50M .
root@t420:/media/extern/snap# for file in {1..100}; do touch $file;
sleep 5m; done
touch: cannot touch ‘8’: Disk quota exceeded
^C
root@t420:/media/extern/snap# sync
root@t420:/media/extern/snap# btrfs qgroup show -pcre .
qgroupid         rfer         excl     max_rfer     max_excl parent 
child
--------         ----         ----     --------     -------- ------ 
-----
0/5          16.00KiB     16.00KiB         none         none ---     ---
0/257        16.00KiB     16.00KiB         none         none ---     ---
0/258        16.00KiB     16.00KiB         none     50.00MiB ---     ---
root@t420:/media/extern/snap# btrfs fi sync .
FSSync '.'
root@t420:/media/extern/snap# btrfs qgroup show -pcre .
qgroupid         rfer         excl     max_rfer     max_excl parent 
child
--------         ----         ----     --------     -------- ------ 
-----
0/5          16.00KiB     16.00KiB         none         none ---     ---
0/257        16.00KiB     16.00KiB         none         none ---     ---
0/258        16.00KiB     16.00KiB         none     50.00MiB ---     ---

Quoted 7 lines

By the way, I don't if its relevant but the problem is not limited to exclusive quotas, but also happens when setting a "referenced" limit (qgroup limit without "-e").

Thanks,
Johannes

The bug is located, and turns out to be quite a stupid problem caused by myself.

I just forgot to include a cleanup patch during rebase!!!! AGAIN!!!

You can apply the following patch to resolve it:
[PATCH 3/3] btrfs: qgroup: Fix a rebase bug which will cause qgroup double free

Or just apply the whole patchset:
[4.4][PATCH 0/3] btrfs: Qgroup hotfix

At least, with the patchset based on Chris' integration-4.4 branch, it succeeded in touching all the 100 files in my test box.

Thanks,
Qu

Quoted 72 lines

Quoted 55 lines

Quoted 21 lines

Quoted 7 lines

Thanks a lot for your reply!

While remounting the filesystem fixes the issue temporary, it doesn't take very long for the bug to happen again so it's not really a workaround I can work with.

Quoted 6 lines

I did recompile the kernel using your patches, but unfortunately the problem still appears.

Thanks,
Johannes

Interesting, just touching file will cause EQUOTA is quite a big problem.

I'll try to reproduce it with my patchset and see what really caused the problem.
The problem seems to do with snapshot qgroup hacking.
But I'm not completely sure yet.

Quoted 5 lines

BTW, does "sync; btrfs qgroup show -prce" still show excl as 16K? 16K is the correct number with only 6 empty files, just in case.

Thanks,
Qu

I ran my example from the first mail again and managed to write 7 files this time, "qgroup show" still shows 16kB after sync:

root@t420:/media/extern/snap# btrfs qg limit -e 50M .
root@t420:/media/extern/snap# for file in {1..100}; do touch $file;
sleep 5m; done
touch: cannot touch ‘8’: Disk quota exceeded
^C
root@t420:/media/extern/snap# sync
root@t420:/media/extern/snap# btrfs qgroup show -pcre .
qgroupid         rfer         excl     max_rfer     max_excl parent
child
--------         ----         ----     --------     -------- ------
-----
0/5          16.00KiB     16.00KiB         none         none ---     ---
0/257        16.00KiB     16.00KiB         none         none ---     ---
0/258        16.00KiB     16.00KiB         none     50.00MiB ---     ---
root@t420:/media/extern/snap# btrfs fi sync .
FSSync '.'
root@t420:/media/extern/snap# btrfs qgroup show -pcre .
qgroupid         rfer         excl     max_rfer     max_excl parent
child
--------         ----         ----     --------     -------- ------
-----
0/5          16.00KiB     16.00KiB         none         none ---     ---
0/257        16.00KiB     16.00KiB         none         none ---     ---
0/258        16.00KiB     16.00KiB         none     50.00MiB ---     ---

Quoted 7 lines

By the way, I don't if its relevant but the problem is not limited to exclusive quotas, but also happens when setting a "referenced" limit (qgroup limit without "-e").

Thanks,
Johannes

The bug is located, and turns out to be quite a stupid problem caused by myself.

I just forgot to include a cleanup patch during rebase!!!! AGAIN!!!

You can apply the following patch to resolve it:
[PATCH 3/3] btrfs: qgroup: Fix a rebase bug which will cause qgroup double free

Quoted 9 lines

Or just apply the whole patchset:
[4.4][PATCH 0/3] btrfs: Qgroup hotfix

At least, with the patchset based on Chris' integration-4.4 branch, it succeeded in touching all the 100 files in my test box.

Thanks,
Qu

It's working! Thank you so much for fixing this bug, you don't even know how much this has helped me!

Thanks!
Johannes
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

Johannes Henninger wrote on 2015/10/27 01:15 +0100:

Quoted 91 lines · Qu Wenruo

Quoted 80 lines

Quoted 63 lines

Quoted 22 lines

Quoted 8 lines

Thanks a lot for your reply!

While remounting the filesystem fixes the issue temporary, it doesn't
take very long for the bug to happen again so it's not really a workaround I can work with.

Quoted 6 lines

I did recompile the kernel using your patches, but unfortunately the problem still appears.

Thanks,
Johannes

Interesting, just touching file will cause EQUOTA is quite a big problem.

I'll try to reproduce it with my patchset and see what really caused the problem.
The problem seems to do with snapshot qgroup hacking.
But I'm not completely sure yet.

Quoted 5 lines

BTW, does "sync; btrfs qgroup show -prce" still show excl as 16K? 16K is the correct number with only 6 empty files, just in case.

Thanks,
Qu

I ran my example from the first mail again and managed to write 7 files
this time, "qgroup show" still shows 16kB after sync:

root@t420:/media/extern/snap# btrfs qg limit -e 50M .
root@t420:/media/extern/snap# for file in {1..100}; do touch $file;
sleep 5m; done
touch: cannot touch ‘8’: Disk quota exceeded
^C
root@t420:/media/extern/snap# sync
root@t420:/media/extern/snap# btrfs qgroup show -pcre .
qgroupid         rfer         excl     max_rfer     max_excl parent
child
--------         ----         ----     --------     -------- ------
-----
0/5          16.00KiB     16.00KiB         none         none
---     ---
0/257        16.00KiB     16.00KiB         none         none
---     ---
0/258        16.00KiB     16.00KiB         none     50.00MiB
---     ---
root@t420:/media/extern/snap# btrfs fi sync .
FSSync '.'
root@t420:/media/extern/snap# btrfs qgroup show -pcre .
qgroupid         rfer         excl     max_rfer     max_excl parent
child
--------         ----         ----     --------     -------- ------
-----
0/5          16.00KiB     16.00KiB         none         none
---     ---
0/257        16.00KiB     16.00KiB         none         none
---     ---
0/258        16.00KiB     16.00KiB         none     50.00MiB
---     ---

Quoted 7 lines

By the way, I don't if its relevant but the problem is not limited to exclusive quotas, but also happens when setting a "referenced" limit (qgroup limit without "-e").

Thanks,
Johannes

The bug is located, and turns out to be quite a stupid problem caused by myself.

I just forgot to include a cleanup patch during rebase!!!! AGAIN!!!

You can apply the following patch to resolve it:
[PATCH 3/3] btrfs: qgroup: Fix a rebase bug which will cause qgroup double free

Quoted 9 lines

Or just apply the whole patchset:
[4.4][PATCH 0/3] btrfs: Qgroup hotfix

At least, with the patchset based on Chris' integration-4.4 branch, it succeeded in touching all the 100 files in my test box.

Thanks,
Qu

Quoted 11 lines

It's working! Thank you so much for fixing this bug, you don't even know how much this has helped me!

Thanks!
Johannes

Signature · 4 lines

To unsubscribe from this list: send the line "unsubscribe
linux-btrfs" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Glad to hear that.

If it's working for you, it would be better to add a 'Tested-by' tag for the 3rd patch.

Thanks,
Qu

Quoted 110 lines

Johannes Henninger wrote on 2015/10/27 01:15 +0100:

Quoted 94 lines · Qu Wenruo

Quoted 80 lines

Quoted 63 lines

Quoted 22 lines

Quoted 8 lines

Thanks a lot for your reply!

While remounting the filesystem fixes the issue temporary, it doesn't
take very long for the bug to happen again so it's not really a workaround I can work with.

Quoted 6 lines

I did recompile the kernel using your patches, but unfortunately the problem still appears.

Thanks,
Johannes

Interesting, just touching file will cause EQUOTA is quite a big problem.

I'll try to reproduce it with my patchset and see what really caused the problem.
The problem seems to do with snapshot qgroup hacking.
But I'm not completely sure yet.

Quoted 5 lines

BTW, does "sync; btrfs qgroup show -prce" still show excl as 16K? 16K is the correct number with only 6 empty files, just in case.

Thanks,
Qu

I ran my example from the first mail again and managed to write 7 files
this time, "qgroup show" still shows 16kB after sync:

root@t420:/media/extern/snap# btrfs qg limit -e 50M .
root@t420:/media/extern/snap# for file in {1..100}; do touch $file;
sleep 5m; done
touch: cannot touch ‘8’: Disk quota exceeded
^C
root@t420:/media/extern/snap# sync
root@t420:/media/extern/snap# btrfs qgroup show -pcre .
qgroupid         rfer         excl     max_rfer     max_excl parent
child
--------         ----         ----     --------     -------- ------
-----
0/5          16.00KiB     16.00KiB         none         none
---     ---
0/257        16.00KiB     16.00KiB         none         none
---     ---
0/258        16.00KiB     16.00KiB         none     50.00MiB
---     ---
root@t420:/media/extern/snap# btrfs fi sync .
FSSync '.'
root@t420:/media/extern/snap# btrfs qgroup show -pcre .
qgroupid         rfer         excl     max_rfer     max_excl parent
child
--------         ----         ----     --------     -------- ------
-----
0/5          16.00KiB     16.00KiB         none         none
---     ---
0/257        16.00KiB     16.00KiB         none         none
---     ---
0/258        16.00KiB     16.00KiB         none     50.00MiB
---     ---

Quoted 7 lines

By the way, I don't if its relevant but the problem is not limited to exclusive quotas, but also happens when setting a "referenced" limit (qgroup limit without "-e").

Thanks,
Johannes

The bug is located, and turns out to be quite a stupid problem caused by myself.

I just forgot to include a cleanup patch during rebase!!!! AGAIN!!!

You can apply the following patch to resolve it:
[PATCH 3/3] btrfs: qgroup: Fix a rebase bug which will cause qgroup double free

Quoted 9 lines

Or just apply the whole patchset:
[4.4][PATCH 0/3] btrfs: Qgroup hotfix

At least, with the patchset based on Chris' integration-4.4 branch, it succeeded in touching all the 100 files in my test box.

Thanks,
Qu

It's working! Thank you so much for fixing this bug, you don't even know how much this has helped me!

Quoted 8 lines

Thanks!
Johannes
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

Glad to hear that.

If it's working for you, it would be better to add a 'Tested-by' tag for the 3rd patch.

Quoted 2 lines

Thanks,
Qu

Sure! Is there anything I have to do? I'm a kernel and mailing list noob :)

Thanks,
Johannes