Skip to contentSkip to conversation

quota tests

DISCUSSION Christoph Hellwig 2 participants 6 messages Sep 6, 2011 VFS

lore
Share & tools
lore
#0 lore raw

Hi Jan,

do you have some quota test scripts? I've noticed that the quota testing in xfstests really isn't as good as it should be and I'm looking for more existing tests I can integrated before writing new ones from scratch.

Anyone else who could provide quota test scripts / tools under GPLv2 or more permissive licenses is of course welcome as well.

reviewer
#1 lore raw

Hi,

Quoted 4 lines · Christoph Hellwig

do you have some quota test scripts? I've noticed that the quota testing in xfstests really isn't as good as it should be and I'm looking for more existing tests I can integrated before writing new ones from scratch.

All I had (well, except a tool which verifies sanity of quotactl syscall
but that's out of scope for xfstests I'd say) is already integrated into xfstests.

We have test 219 which tests whether quota measures something sane. Test 230 tries to exceed various quota limits and checks that it fails
roughly at the right moment.
Tests 231 and 232 check that if we stress the filesystem (fsx, fsstress),
the resulting quota usage is indeed what we get when we sum up the usage using quotacheck (which is worthless for XFS but for ext? filesystems this is really useful).
Test 233 checks running fsstress with quotas set low so that we often hit
EDQUOT. We also verify that usage matches what quotacheck gets.
Finally test 234 stresses operations on top of quota files by adding and
removing quota structures by setquota from several processes.

What other tests would you like to have?

Honza

Signature · 2 lines
Jan Kara <[email protected]>
SUSE Labs, CR
#2 lore raw
Quoted 16 lines · Jan Kara

All I had (well, except a tool which verifies sanity of quotactl syscall
but that's out of scope for xfstests I'd say) is already integrated into xfstests.

We have test 219 which tests whether quota measures something sane. Test 230 tries to exceed various quota limits and checks that it fails
roughly at the right moment.
Tests 231 and 232 check that if we stress the filesystem (fsx, fsstress),
the resulting quota usage is indeed what we get when we sum up the usage using quotacheck (which is worthless for XFS but for ext? filesystems this is really useful).
Test 233 checks running fsstress with quotas set low so that we often hit
EDQUOT. We also verify that usage matches what quotacheck gets.
Finally test 234 stresses operations on top of quota files by adding and
removing quota structures by setquota from several processes.

Quoted 1 line

What other tests would you like to have?

In XFS we have a few counters for evens in the quota code, and I noticed that we're barely ever hitting reclaim of dquots due to memory pressure, not do we ever see racing quota lookups. So one thing I'd like to to is all kinds of heavy fs stressing with quotas enabled. One option is to simply run all of xfstests with quotas enabled, which I already did, but I'm still not hitting any of these above.

reviewer
#3 lore raw
Quoted 17 lines · Christoph Hellwig
Quoted 16 lines · Jan Kara

All I had (well, except a tool which verifies sanity of quotactl syscall
but that's out of scope for xfstests I'd say) is already integrated into xfstests.

We have test 219 which tests whether quota measures something sane. Test 230 tries to exceed various quota limits and checks that it fails
roughly at the right moment.
Tests 231 and 232 check that if we stress the filesystem (fsx, fsstress),
the resulting quota usage is indeed what we get when we sum up the usage using quotacheck (which is worthless for XFS but for ext? filesystems this is really useful).
Test 233 checks running fsstress with quotas set low so that we often hit
EDQUOT. We also verify that usage matches what quotacheck gets.
Finally test 234 stresses operations on top of quota files by adding and
removing quota structures by setquota from several processes.

Quoted 8 lines
Quoted 1 line

What other tests would you like to have?

In XFS we have a few counters for evens in the quota code, and I noticed that we're barely ever hitting reclaim of dquots due to memory pressure, not do we ever see racing quota lookups. So one thing I'd like to to is all kinds of heavy fs stressing with quotas enabled. One option is to simply run all of xfstests with quotas enabled, which I already did, but I'm still not hitting any of these above.

That's a good plan.

To verify reclaim behavior for VFS quotas, we could pull quota structures
in memory using GETQUOTA and then apply memory pressure to have them reclaimed. If we also want to combine this with updates to those structures because of allocations it gets more complicated. We'd have to do IO for several different users in sequence under memory pressure so that all inodes of one user get pushed out of memory and quota structure is thus freed for reclaim as well.

As for racing lookups / other quota operations - these are not that
much interesting for VFS quotas because
a) once quota structure is created, it's position on disk is fixed
b) any operation on quota file that does more than update accounting
information in a structure is protected by per quota-file mutex.
c) in memory operations with quota structures are protected by one global
spinlock

So there is no paralelism to worry about. But sure it's good to verify this by pushing the filesystem as hard as possible :-).

Honza

Signature · 2 lines
Jan Kara <[email protected]>
SUSE Labs, CR
#4 lore raw
Quoted 8 lines · Jan Kara

To verify reclaim behavior for VFS quotas, we could pull quota structures
in memory using GETQUOTA and then apply memory pressure to have them reclaimed. If we also want to combine this with updates to those structures because of allocations it gets more complicated. We'd have to do IO for several different users in sequence under memory pressure so that all inodes of one user get pushed out of memory and quota structure is thus freed for reclaim as well.

Quoted 7 lines

As for racing lookups / other quota operations - these are not that
much interesting for VFS quotas because
a) once quota structure is created, it's position on disk is fixed
b) any operation on quota file that does more than update accounting
information in a structure is protected by per quota-file mutex.
c) in memory operations with quota structures are protected by one global
spinlock

It's fairly similar for XFS - but we still have code to e.g. retry the lookup after we did a memory allocation in both quota implementation.

I just change the way the one in XFS worked, so I'm looking for a way to actually test that path.

reviewer
#5 lore raw
In reply to Jan Kara
Quoted 9 lines · Jan Kara
Quoted 4 lines · Christoph Hellwig

do you have some quota test scripts? I've noticed that the quota testing in xfstests really isn't as good as it should be and I'm looking for more existing tests I can integrated before writing new ones from scratch.

All I had (well, except a tool which verifies sanity of quotactl syscall
but that's out of scope for xfstests I'd say) is already integrated into xfstests.

Quoted 11 lines

We have test 219 which tests whether quota measures something sane. Test 230 tries to exceed various quota limits and checks that it fails
roughly at the right moment.
Tests 231 and 232 check that if we stress the filesystem (fsx, fsstress),
the resulting quota usage is indeed what we get when we sum up the usage using quotacheck (which is worthless for XFS but for ext? filesystems this is really useful).
Test 233 checks running fsstress with quotas set low so that we often hit
EDQUOT. We also verify that usage matches what quotacheck gets.
Finally test 234 stresses operations on top of quota files by adding and
removing quota structures by setquota from several processes.

Oh, and we have test 235 which checks whether quota gets properly
suspended during remount read-only and reenabled during remount read-write.

Honza

Signature · 2 lines
Jan Kara <[email protected]>
SUSE Labs, CR