mm, compaction: allow compaction for GFP_NOFS requests

From: Michal Hocko <[email protected]>

compaction has been disabled for GFP_NOFS and GFP_NOIO requests since the direct compaction was introduced by 56de7263fcf3 ("mm: compaction:
direct compact when a high-order allocation fails"). The main reason is that the migration of page cache pages might recurse back to fs/io layer and we could potentially deadlock. This is overly conservative because all the anonymous memory is migrateable in the GFP_NOFS context just fine. This might be a large portion of the memory in many/most workkloads.

Remove the GFP_NOFS restriction and make sure that we skip all fs pages (those with a mapping) while isolating pages to be migrated. We cannot consider clean fs pages because they might need a metadata update so only isolate pages without any mapping for nofs requests.

The effect of this patch will be probably very limited in many/most workloads because higher order GFP_NOFS requests are quite rare,

Quoted 16 lines · Michal Hocko

From: Michal Hocko <[email protected]>

compaction has been disabled for GFP_NOFS and GFP_NOIO requests since the direct compaction was introduced by 56de7263fcf3 ("mm: compaction:
direct compact when a high-order allocation fails"). The main reason is that the migration of page cache pages might recurse back to fs/io layer and we could potentially deadlock. This is overly conservative because all the anonymous memory is migrateable in the GFP_NOFS context just fine. This might be a large portion of the memory in many/most workkloads.

Remove the GFP_NOFS restriction and make sure that we skip all fs pages (those with a mapping) while isolating pages to be migrated. We cannot consider clean fs pages because they might need a metadata update so only isolate pages without any mapping for nofs requests.

Quoted 2 lines

The effect of this patch will be probably very limited in many/most workloads because higher order GFP_NOFS requests are quite rare,

You say they are rare only because you don't know how to trigger them easily. :/

Try this:

# mkfs.xfs -f -n size=64k <dev>
# mount <dev> /mnt/scratch
# time ./fs_mark  -D  10000  -S0  -n  100000  -s  0  -L  32 \
        -d  /mnt/scratch/0  -d  /mnt/scratch/1 \
        -d  /mnt/scratch/2  -d  /mnt/scratch/3 \
        -d  /mnt/scratch/4  -d  /mnt/scratch/5 \
        -d  /mnt/scratch/6  -d  /mnt/scratch/7 \
        -d  /mnt/scratch/8  -d  /mnt/scratch/9 \
        -d  /mnt/scratch/10  -d  /mnt/scratch/11 \
        -d  /mnt/scratch/12  -d  /mnt/scratch/13 \
        -d  /mnt/scratch/14  -d  /mnt/scratch/15

Quoted 17 lines · Dave Chinner

Quoted 16 lines · Michal Hocko

From: Michal Hocko <[email protected]>

compaction has been disabled for GFP_NOFS and GFP_NOIO requests since the direct compaction was introduced by 56de7263fcf3 ("mm: compaction:
direct compact when a high-order allocation fails"). The main reason is that the migration of page cache pages might recurse back to fs/io layer and we could potentially deadlock. This is overly conservative because all the anonymous memory is migrateable in the GFP_NOFS context just fine. This might be a large portion of the memory in many/most workkloads.

Remove the GFP_NOFS restriction and make sure that we skip all fs pages (those with a mapping) while isolating pages to be migrated. We cannot consider clean fs pages because they might need a metadata update so only isolate pages without any mapping for nofs requests.

Quoted 5 lines

Quoted 2 lines

The effect of this patch will be probably very limited in many/most workloads because higher order GFP_NOFS requests are quite rare,

You say they are rare only because you don't know how to trigger them easily. :/

true

Quoted 13 lines

Try this:

# mkfs.xfs -f -n size=64k <dev>
# mount <dev> /mnt/scratch
# time ./fs_mark  -D  10000  -S0  -n  100000  -s  0  -L  32 \
        -d  /mnt/scratch/0  -d  /mnt/scratch/1 \
        -d  /mnt/scratch/2  -d  /mnt/scratch/3 \
        -d  /mnt/scratch/4  -d  /mnt/scratch/5 \
        -d  /mnt/scratch/6  -d  /mnt/scratch/7 \
        -d  /mnt/scratch/8  -d  /mnt/scratch/9 \
        -d  /mnt/scratch/10  -d  /mnt/scratch/11 \
        -d  /mnt/scratch/12  -d  /mnt/scratch/13 \
        -d  /mnt/scratch/14  -d  /mnt/scratch/15

Does this simulate a standard or usual fs workload/configuration? I am

not questioning that higher order NOFS allocations are non-existent - that's why I came with the patch in the first place ;). My observation was that they are so rare that the visible effect of this patch might be quite low or even hard to notice.

Quoted 40 lines · Michal Hocko

Quoted 17 lines · Dave Chinner

Quoted 16 lines · Michal Hocko

From: Michal Hocko <[email protected]>

compaction has been disabled for GFP_NOFS and GFP_NOIO requests since the direct compaction was introduced by 56de7263fcf3 ("mm: compaction:
direct compact when a high-order allocation fails"). The main reason is that the migration of page cache pages might recurse back to fs/io layer and we could potentially deadlock. This is overly conservative because all the anonymous memory is migrateable in the GFP_NOFS context just fine. This might be a large portion of the memory in many/most workkloads.

Remove the GFP_NOFS restriction and make sure that we skip all fs pages (those with a mapping) while isolating pages to be migrated. We cannot consider clean fs pages because they might need a metadata update so only isolate pages without any mapping for nofs requests.

Quoted 5 lines

Quoted 2 lines

The effect of this patch will be probably very limited in many/most workloads because higher order GFP_NOFS requests are quite rare,

You say they are rare only because you don't know how to trigger them easily. :/

true

Quoted 13 lines

Try this:

# mkfs.xfs -f -n size=64k <dev>
# mount <dev> /mnt/scratch
# time ./fs_mark  -D  10000  -S0  -n  100000  -s  0  -L  32 \
        -d  /mnt/scratch/0  -d  /mnt/scratch/1 \
        -d  /mnt/scratch/2  -d  /mnt/scratch/3 \
        -d  /mnt/scratch/4  -d  /mnt/scratch/5 \
        -d  /mnt/scratch/6  -d  /mnt/scratch/7 \
        -d  /mnt/scratch/8  -d  /mnt/scratch/9 \
        -d  /mnt/scratch/10  -d  /mnt/scratch/11 \
        -d  /mnt/scratch/12  -d  /mnt/scratch/13 \
        -d  /mnt/scratch/14  -d  /mnt/scratch/15

Quoted 1 line

Does this simulate a standard or usual fs workload/configuration? I am

Unfortunately, there was an era of cargo cult configuration tweaks in the Ceph community that has resulted in a large number of production machines with XFS filesystems configured this way. And a lot of them store large numbers of small files and run under significant sustained memory pressure.

I slowly working towards getting rid of these high order allocations and replacing them with the equivalent number of single page allocations, but I haven't got that (complex) change working yet.

Quoted 41 lines · Dave Chinner

Quoted 40 lines · Michal Hocko

Quoted 17 lines · Dave Chinner

Quoted 16 lines · Michal Hocko

From: Michal Hocko <[email protected]>

compaction has been disabled for GFP_NOFS and GFP_NOIO requests since the direct compaction was introduced by 56de7263fcf3 ("mm: compaction:
direct compact when a high-order allocation fails"). The main reason is that the migration of page cache pages might recurse back to fs/io layer and we could potentially deadlock. This is overly conservative because all the anonymous memory is migrateable in the GFP_NOFS context just fine. This might be a large portion of the memory in many/most workkloads.

Remove the GFP_NOFS restriction and make sure that we skip all fs pages (those with a mapping) while isolating pages to be migrated. We cannot consider clean fs pages because they might need a metadata update so only isolate pages without any mapping for nofs requests.

Quoted 5 lines

Quoted 2 lines

The effect of this patch will be probably very limited in many/most workloads because higher order GFP_NOFS requests are quite rare,

You say they are rare only because you don't know how to trigger them easily. :/

true

Quoted 13 lines

Try this:

# mkfs.xfs -f -n size=64k <dev>
# mount <dev> /mnt/scratch
# time ./fs_mark  -D  10000  -S0  -n  100000  -s  0  -L  32 \
        -d  /mnt/scratch/0  -d  /mnt/scratch/1 \
        -d  /mnt/scratch/2  -d  /mnt/scratch/3 \
        -d  /mnt/scratch/4  -d  /mnt/scratch/5 \
        -d  /mnt/scratch/6  -d  /mnt/scratch/7 \
        -d  /mnt/scratch/8  -d  /mnt/scratch/9 \
        -d  /mnt/scratch/10  -d  /mnt/scratch/11 \
        -d  /mnt/scratch/12  -d  /mnt/scratch/13 \
        -d  /mnt/scratch/14  -d  /mnt/scratch/15

Quoted 7 lines

Quoted 1 line

Does this simulate a standard or usual fs workload/configuration? I am

Unfortunately, there was an era of cargo cult configuration tweaks in the Ceph community that has resulted in a large number of production machines with XFS filesystems configured this way. And a lot of them store large numbers of small files and run under significant sustained memory pressure.

I see

Quoted 3 lines

I slowly working towards getting rid of these high order allocations and replacing them with the equivalent number of single page allocations, but I haven't got that (complex) change working yet.

Definitely a good plan!

Anyway I was playing with this in my virtual machine (4CPUs, 512MB of RAM split into two NUMA nodes). Started on a freshly created fs after boot, no other load in the guest. The performance numbers should be taken with grain of salt, though, because the host has 4CPUs as well and it wasn't completely idle, but should be OK enough to give us at least some picture. This is what fs_mark told me:
Unpatched kernel:

#       Version 3.3, 16 thread(s) starting at Fri Oct  7 09:55:05 2016
#       Sync method: NO SYNC: Test does not issue sync() or fsync() calls.
#       Directories:  Time based hash between directories across 10000 subdirectories with 180 seconds per subdirectory.
#       File names: 40 bytes long, (16 initial bytes of time stamp with 24 random bytes at end of name)
#       Files info: size 0 bytes, written with an IO size of 16384 bytes per write
#       App overhead is time in microseconds spent in the test not doing file writing related system calls.
#
FSUse%        Count         Size    Files/sec     App Overhead
     1      1600000            0       4300.1         20745838
     3      3200000            0       4239.9         23849857
     5      4800000            0       4243.4         25939543
     6      6400000            0       4248.4         19514050
     8      8000000            0       4262.1         20796169
     9      9600000            0       4257.6         21288675
    11     11200000            0       4259.7         19375120
    13     12800000            0       4220.7         22734141
    14     14400000            0       4238.5         31936458
    16     16000000            0       4231.5         23409901
    18     17600000            0       4045.3         23577700
    19     19200000            0       2783.4         58299526
    21     20800000            0       2678.2         40616302
    23     22400000            0       2693.5         83973996

Ctrl+C because it just took too long.

while it doesn't seem to drop the Files/sec numbers starting with Count=19200000. I also see only a single

[ 3063.815003] XFS: fs_mark(3272) possible memory allocation deadlock size 65624 in kmem_alloc (mode:0x2408240)

Unpatched kernel
all orders
begin:44.718798 end:5774.618736 allocs:15019288
order > 0
begin:44.718798 end:5773.587195 allocs:10438610

Patched kernel
all orders
begin:64.612804 end:5794.193619 allocs:16110081 [107.2%] order > 0
begin:64.612804 end:5794.193619 allocs:11741492 [112.5%]

which would suggest that diving into the compaction rather than backing off and waiting for kcompactd to make the work for us was indeed a better strategy and helped the throughput.

On Fri, Oct 07, 2016 at 03:18:14PM +0200, Michal Hocko wrote:

Quoted 23 lines

Unpatched kernel:

#       Version 3.3, 16 thread(s) starting at Fri Oct  7 09:55:05 2016
#       Sync method: NO SYNC: Test does not issue sync() or fsync() calls.
#       Directories:  Time based hash between directories across 10000 subdirectories with 180 seconds per subdirectory.
#       File names: 40 bytes long, (16 initial bytes of time stamp with 24 random bytes at end of name)
#       Files info: size 0 bytes, written with an IO size of 16384 bytes per write
#       App overhead is time in microseconds spent in the test not doing file writing related system calls.
#
FSUse%        Count         Size    Files/sec     App Overhead
     1      1600000            0       4300.1         20745838
     3      3200000            0       4239.9         23849857
     5      4800000            0       4243.4         25939543
     6      6400000            0       4248.4         19514050
     8      8000000            0       4262.1         20796169
     9      9600000            0       4257.6         21288675
    11     11200000            0       4259.7         19375120
    13     12800000            0       4220.7         22734141
    14     14400000            0       4238.5         31936458
    16     16000000            0       4231.5         23409901
    18     17600000            0       4045.3         23577700
    19     19200000            0       2783.4         58299526
    21     20800000            0       2678.2         40616302
    23     22400000            0       2693.5         83973996

Quoted 1 line

Ctrl+C because it just took too long.

Try running it on a larger filesystem, or configure the fs with more AGs and a larger log (i.e. mkfs.xfs -f -dagcount=24 -l size=512m <dev>). That will speed up modifications and increase concurrency. This test should be able to run 5-10x faster than this (it sustains 55,000 files/s @ 300MB/s write on my test fs on a cheap SSD).

Quoted 4 lines

while it doesn't seem to drop the Files/sec numbers starting with Count=19200000. I also see only a single

[ 3063.815003] XFS: fs_mark(3272) possible memory allocation deadlock size 65624 in kmem_alloc (mode:0x2408240)

Remember that this is emitted only after /100/ consecutive allocation failures. So the fact it is still being emitted means that the situation is not drastically better....

Quoted 12 lines

Unpatched kernel
all orders
begin:44.718798 end:5774.618736 allocs:15019288
order > 0
begin:44.718798 end:5773.587195 allocs:10438610

Patched kernel
all orders
begin:64.612804 end:5794.193619 allocs:16110081 [107.2%] order > 0
begin:64.612804 end:5794.193619 allocs:11741492 [112.5%]

Quoted 3 lines

which would suggest that diving into the compaction rather than backing off and waiting for kcompactd to make the work for us was indeed a better strategy and helped the throughput.

Well, without a success/failure ratio being calculated it's hard to tell what improvement it made. Did it increase the success rate, or reduce failure latency so retries happened faster?

Quoted 1 line · Dave Chinner

On Fri, Oct 07, 2016 at 03:18:14PM +0200, Michal Hocko wrote:

[...]

Quoted 25 lines

Quoted 24 lines

Unpatched kernel:

#       Version 3.3, 16 thread(s) starting at Fri Oct  7 09:55:05 2016
#       Sync method: NO SYNC: Test does not issue sync() or fsync() calls.
#       Directories:  Time based hash between directories across 10000 subdirectories with 180 seconds per subdirectory.
#       File names: 40 bytes long, (16 initial bytes of time stamp with 24 random bytes at end of name)
#       Files info: size 0 bytes, written with an IO size of 16384 bytes per write
#       App overhead is time in microseconds spent in the test not doing file writing related system calls.
#
FSUse%        Count         Size    Files/sec     App Overhead
     1      1600000            0       4300.1         20745838
     3      3200000            0       4239.9         23849857
     5      4800000            0       4243.4         25939543
     6      6400000            0       4248.4         19514050
     8      8000000            0       4262.1         20796169
     9      9600000            0       4257.6         21288675
    11     11200000            0       4259.7         19375120
    13     12800000            0       4220.7         22734141
    14     14400000            0       4238.5         31936458
    16     16000000            0       4231.5         23409901
    18     17600000            0       4045.3         23577700
    19     19200000            0       2783.4         58299526
    21     20800000            0       2678.2         40616302
    23     22400000            0       2693.5         83973996

Ctrl+C because it just took too long.

Quoted 6 lines

Try running it on a larger filesystem, or configure the fs with more AGs and a larger log (i.e. mkfs.xfs -f -dagcount=24 -l size=512m <dev>). That will speed up modifications and increase concurrency. This test should be able to run 5-10x faster than this (it sustains 55,000 files/s @ 300MB/s write on my test fs on a cheap SSD).

Will add more memory to the machine. Will report back on that.

Quoted 28 lines · Michal Hocko

Quoted 1 line · Dave Chinner

On Fri, Oct 07, 2016 at 03:18:14PM +0200, Michal Hocko wrote:

[...]

Quoted 25 lines

Quoted 24 lines

Unpatched kernel:

#       Version 3.3, 16 thread(s) starting at Fri Oct  7 09:55:05 2016
#       Sync method: NO SYNC: Test does not issue sync() or fsync() calls.
#       Directories:  Time based hash between directories across 10000 subdirectories with 180 seconds per subdirectory.
#       File names: 40 bytes long, (16 initial bytes of time stamp with 24 random bytes at end of name)
#       Files info: size 0 bytes, written with an IO size of 16384 bytes per write
#       App overhead is time in microseconds spent in the test not doing file writing related system calls.
#
FSUse%        Count         Size    Files/sec     App Overhead
     1      1600000            0       4300.1         20745838
     3      3200000            0       4239.9         23849857
     5      4800000            0       4243.4         25939543
     6      6400000            0       4248.4         19514050
     8      8000000            0       4262.1         20796169
     9      9600000            0       4257.6         21288675
    11     11200000            0       4259.7         19375120
    13     12800000            0       4220.7         22734141
    14     14400000            0       4238.5         31936458
    16     16000000            0       4231.5         23409901
    18     17600000            0       4045.3         23577700
    19     19200000            0       2783.4         58299526
    21     20800000            0       2678.2         40616302
    23     22400000            0       2693.5         83973996

Ctrl+C because it just took too long.

Quoted 8 lines

Quoted 6 lines

Try running it on a larger filesystem, or configure the fs with more AGs and a larger log (i.e. mkfs.xfs -f -dagcount=24 -l size=512m <dev>). That will speed up modifications and increase concurrency. This test should be able to run 5-10x faster than this (it sustains 55,000 files/s @ 300MB/s write on my test fs on a cheap SSD).

Will add more memory to the machine. Will report back on that.

increasing the memory to 1G didn't help. So I've tried to add -dagcount=24 -l size=512m and that didn't help much either. I am at 5k files/s so nowhere close to your 55k. I thought this is more about CPUs count than about the amount of memory. So I've tried a larger machine with 24 CPUs (no dagcount etc...), this one doesn't have a fast storage so I've backed the fs image by ramdisk but even then I am getting very similar results. No idea what is wrong with my kvm setup.

Quoted 38 lines · Michal Hocko

Quoted 28 lines · Michal Hocko

Quoted 1 line · Dave Chinner

On Fri, Oct 07, 2016 at 03:18:14PM +0200, Michal Hocko wrote:

[...]

Quoted 25 lines

Quoted 24 lines

Unpatched kernel:

#       Version 3.3, 16 thread(s) starting at Fri Oct  7 09:55:05 2016
#       Sync method: NO SYNC: Test does not issue sync() or fsync() calls.
#       Directories:  Time based hash between directories across 10000 subdirectories with 180 seconds per subdirectory.
#       File names: 40 bytes long, (16 initial bytes of time stamp with 24 random bytes at end of name)
#       Files info: size 0 bytes, written with an IO size of 16384 bytes per write
#       App overhead is time in microseconds spent in the test not doing file writing related system calls.
#
FSUse%        Count         Size    Files/sec     App Overhead
     1      1600000            0       4300.1         20745838
     3      3200000            0       4239.9         23849857
     5      4800000            0       4243.4         25939543
     6      6400000            0       4248.4         19514050
     8      8000000            0       4262.1         20796169
     9      9600000            0       4257.6         21288675
    11     11200000            0       4259.7         19375120
    13     12800000            0       4220.7         22734141
    14     14400000            0       4238.5         31936458
    16     16000000            0       4231.5         23409901
    18     17600000            0       4045.3         23577700
    19     19200000            0       2783.4         58299526
    21     20800000            0       2678.2         40616302
    23     22400000            0       2693.5         83973996

Ctrl+C because it just took too long.

Quoted 8 lines

Quoted 6 lines

Try running it on a larger filesystem, or configure the fs with more AGs and a larger log (i.e. mkfs.xfs -f -dagcount=24 -l size=512m <dev>). That will speed up modifications and increase concurrency. This test should be able to run 5-10x faster than this (it sustains 55,000 files/s @ 300MB/s write on my test fs on a cheap SSD).

Will add more memory to the machine. Will report back on that.

Quoted 7 lines

increasing the memory to 1G didn't help. So I've tried to add -dagcount=24 -l size=512m and that didn't help much either. I am at 5k files/s so nowhere close to your 55k. I thought this is more about CPUs count than about the amount of memory. So I've tried a larger machine with 24 CPUs (no dagcount etc...), this one doesn't have a fast storage so I've backed the fs image by ramdisk but even then I am getting very similar results. No idea what is wrong with my kvm setup.

What's the backing storage? I use an image file in an XFS filesystem, configured with virtio,cache=none so it's concurrency model matches that of a real disk...

Quoted 47 lines · Dave Chinner

Quoted 45 lines · Michal Hocko

Quoted 28 lines · Michal Hocko

Quoted 1 line · Dave Chinner

On Fri, Oct 07, 2016 at 03:18:14PM +0200, Michal Hocko wrote:

[...]

Quoted 25 lines

Quoted 24 lines

Unpatched kernel:

#       Version 3.3, 16 thread(s) starting at Fri Oct  7 09:55:05 2016
#       Sync method: NO SYNC: Test does not issue sync() or fsync() calls.
#       Directories:  Time based hash between directories across 10000 subdirectories with 180 seconds per subdirectory.
#       File names: 40 bytes long, (16 initial bytes of time stamp with 24 random bytes at end of name)
#       Files info: size 0 bytes, written with an IO size of 16384 bytes per write
#       App overhead is time in microseconds spent in the test not doing file writing related system calls.
#
FSUse%        Count         Size    Files/sec     App Overhead
     1      1600000            0       4300.1         20745838
     3      3200000            0       4239.9         23849857
     5      4800000            0       4243.4         25939543
     6      6400000            0       4248.4         19514050
     8      8000000            0       4262.1         20796169
     9      9600000            0       4257.6         21288675
    11     11200000            0       4259.7         19375120
    13     12800000            0       4220.7         22734141
    14     14400000            0       4238.5         31936458
    16     16000000            0       4231.5         23409901
    18     17600000            0       4045.3         23577700
    19     19200000            0       2783.4         58299526
    21     20800000            0       2678.2         40616302
    23     22400000            0       2693.5         83973996

Ctrl+C because it just took too long.

Quoted 8 lines

Quoted 6 lines

Try running it on a larger filesystem, or configure the fs with more AGs and a larger log (i.e. mkfs.xfs -f -dagcount=24 -l size=512m <dev>). That will speed up modifications and increase concurrency. This test should be able to run 5-10x faster than this (it sustains 55,000 files/s @ 300MB/s write on my test fs on a cheap SSD).

Will add more memory to the machine. Will report back on that.

increasing the memory to 1G didn't help. So I've tried to add -dagcount=24 -l size=512m and that didn't help much either. I am at 5k files/s so nowhere close to your 55k. I thought this is more about CPUs count than about the amount of memory. So I've tried a larger machine with 24 CPUs (no dagcount etc...), this one doesn't have a fast storage so I've backed the fs image by ramdisk but even then I am getting very similar results. No idea what is wrong with my kvm setup.

Quoted 3 lines

What's the backing storage? I use an image file in an XFS filesystem, configured with virtio,cache=none so it's concurrency model matches that of a real disk...

I am using qcow qemu image exported to qemu by
-drive file=storage.img,if=ide,index=1,cache=none
parameter.

Quoted 52 lines · Michal Hocko

Quoted 50 lines · Dave Chinner

Quoted 38 lines · Michal Hocko

Quoted 28 lines · Michal Hocko

Quoted 1 line · Dave Chinner

On Fri, Oct 07, 2016 at 03:18:14PM +0200, Michal Hocko wrote:

[...]

Quoted 25 lines

Quoted 24 lines

Unpatched kernel:

#       Version 3.3, 16 thread(s) starting at Fri Oct  7 09:55:05 2016
#       Sync method: NO SYNC: Test does not issue sync() or fsync() calls.
#       Directories:  Time based hash between directories across 10000 subdirectories with 180 seconds per subdirectory.
#       File names: 40 bytes long, (16 initial bytes of time stamp with 24 random bytes at end of name)
#       Files info: size 0 bytes, written with an IO size of 16384 bytes per write
#       App overhead is time in microseconds spent in the test not doing file writing related system calls.
#
FSUse%        Count         Size    Files/sec     App Overhead
     1      1600000            0       4300.1         20745838
     3      3200000            0       4239.9         23849857
     5      4800000            0       4243.4         25939543
     6      6400000            0       4248.4         19514050
     8      8000000            0       4262.1         20796169
     9      9600000            0       4257.6         21288675
    11     11200000            0       4259.7         19375120
    13     12800000            0       4220.7         22734141
    14     14400000            0       4238.5         31936458
    16     16000000            0       4231.5         23409901
    18     17600000            0       4045.3         23577700
    19     19200000            0       2783.4         58299526
    21     20800000            0       2678.2         40616302
    23     22400000            0       2693.5         83973996

Ctrl+C because it just took too long.

Quoted 8 lines

Quoted 6 lines

Try running it on a larger filesystem, or configure the fs with more AGs and a larger log (i.e. mkfs.xfs -f -dagcount=24 -l size=512m <dev>). That will speed up modifications and increase concurrency. This test should be able to run 5-10x faster than this (it sustains 55,000 files/s @ 300MB/s write on my test fs on a cheap SSD).

Will add more memory to the machine. Will report back on that.

Quoted 7 lines

increasing the memory to 1G didn't help. So I've tried to add -dagcount=24 -l size=512m and that didn't help much either. I am at 5k files/s so nowhere close to your 55k. I thought this is more about CPUs count than about the amount of memory. So I've tried a larger machine with 24 CPUs (no dagcount etc...), this one doesn't have a fast storage so I've backed the fs image by ramdisk but even then I am getting very similar results. No idea what is wrong with my kvm setup.

What's the backing storage? I use an image file in an XFS filesystem, configured with virtio,cache=none so it's concurrency model matches that of a real disk...

Quoted 3 lines

I am using qcow qemu image exported to qemu by
-drive file=storage.img,if=ide,index=1,cache=none
parameter.

storage.img is on what type of filesystem?

Only XFs will give you
proper IO concurrency with direct IO, and you really need to use a raw image file rather than qcow2. If you're not using the special capabilities of qcow2 (e.g. snapshots), there's no reason to use it...

From: Michal Hocko <[email protected]>

compaction has been disabled for GFP_NOFS and GFP_NOIO requests since the direct compaction was introduced by 56de7263fcf3 ("mm: compaction:
direct compact when a high-order allocation fails"). The main reason is that the migration of page cache pages might recurse back to fs/io layer and we could potentially deadlock. This is overly conservative because all the anonymous memory is migrateable in the GFP_NOFS context just fine. This might be a large portion of the memory in many/most workkloads.

Remove the GFP_NOFS restriction and make sure that we skip all fs pages (those with a mapping) while isolating pages to be migrated. We cannot consider clean fs pages because they might need a metadata update so only isolate pages without any mapping for nofs requests.

The effect of this patch will be probably very limited in many/most workloads because higher order GFP_NOFS requests are quite rare, although different configurations might lead to very different results as GFP_NOFS usage is rather unleashed (e.g. I had hard time to trigger any with my setup). But still there shouldn't be any strong reason to completely back off and do nothing in that context. In the worst case we just skip parts of the block with fs pages. This might be still sufficient to make a progress for small orders.

Signed-off-by Michal Hocko <[email protected]>

---

Hi,
I am sending this as an RFC because I am not completely sure this a) is really worth it and b) it is 100% correct. I couldn't find any problems when staring into the code but as mentioned in the changelog I wasn't really able to trigger high order GFP_NOFS requests in my setup.

Thoughts?

 mm/compaction.c | 15 ++++++++++++---
 1 file changed, 12 insertions(+), 3 deletions(-)

mm/compaction.c +0 -0

index badb92bf14b4..07254a73ee32 100644

mm/compaction.c +1 -1

+
 		/* If we already hold the lock, we can skip some rechecking */
 		if (!locked) {
 			locked = compact_trylock_irqsave(zone_lru_lock(zone),

@@ -1696,14 +1703,16 @@ enum compact_result try_to_compact_pages(gfp_t gfp_mask, unsigned int order,
 		unsigned int alloc_flags, const struct alloc_context *ac,
 		enum compact_priority prio)
 {
-	int may_enter_fs = gfp_mask & __GFP_FS;
 	int may_perform_io = gfp_mask & __GFP_IO;
 	struct zoneref *z;
 	struct zone *zone;
 	enum compact_result rc = COMPACT_SKIPPED;
 

Quoted 8 lines

mm/compaction.c +0 -0

index badb92bf14b4..07254a73ee32 100644
@@ -834,6 +834,13 @@ isolate_migratepages_block(struct compact_control *cc, unsigned long low_pfn,
 		    page_count(page) > page_mapcount(page))
 			goto isolate_fail;
 

mm/compaction.c isolate_migratepages_block(struct compact_control *cc, unsigned long low_pfn, · line 842

Quoted 6 lines

mm/compaction.c +6 -0

+		/*
+		 * Only allow to migrate anonymous pages in GFP_NOFS context
+		 * because those do not depend on fs locks.
+		 */
+		if (!(cc->gfp_mask & __GFP_FS) && page_mapping(page))
+			goto isolate_fail;

Unless page can acquire a page_mapping between this check and migration, I don't see a problem with allowing this.

But make sure you don't break kcompactd and manual compaction from /proc, as they don't currently set cc->gfp_mask. Looks like until now it was only used to determine direct compactor's migratetype which is irrelevant in those contexts.

On Fri 07-10-16 07:27:37, Vlastimil Babka wrote:
[...]

Quoted 8 lines

Quoted 8 lines

mm/compaction.c +0 -0

index badb92bf14b4..07254a73ee32 100644
@@ -834,6 +834,13 @@ isolate_migratepages_block(struct compact_control *cc, unsigned long low_pfn,
 		    page_count(page) > page_mapcount(page))
 			goto isolate_fail;
 

Quoted 9 lines

mm/compaction.c isolate_migratepages_block(struct compact_control *cc, unsigned long low_pfn, · line 842

Quoted 6 lines

mm/compaction.c +6 -0

+		/*
+		 * Only allow to migrate anonymous pages in GFP_NOFS context
+		 * because those do not depend on fs locks.
+		 */
+		if (!(cc->gfp_mask & __GFP_FS) && page_mapping(page))
+			goto isolate_fail;

Unless page can acquire a page_mapping between this check and migration, I don't see a problem with allowing this.

It can be become swapcache but I guess this should be OK. We do not allow to get here with GFP_NOIO and migrating swapcache pages in NOFS mode should be OK AFAICS.

Quoted 4 lines

But make sure you don't break kcompactd and manual compaction from /proc, as they don't currently set cc->gfp_mask. Looks like until now it was only used to determine direct compactor's migratetype which is irrelevant in those contexts.

OK, I see. This is really subtle. One way to go would be to provide a fake gfp_mask for them. How does the following look to you?

---

mm/compaction.c +1 -0

index 557c165b63ad..d1d90e96ef4b 100644
@@ -1779,6 +1779,7 @@ static void compact_node(int nid)
 		.mode = MIGRATE_SYNC,
 		.ignore_skip_hint = true,
 		.whole_zone = true,
+		.gfp_mask = GFP_KERNEL,
 	};
 
 

Quoted 1 line · Michal Hocko

On Fri 07-10-16 07:27:37, Vlastimil Babka wrote:

Quoted 5 lines

Quoted 4 lines

But make sure you don't break kcompactd and manual compaction from /proc, as they don't currently set cc->gfp_mask. Looks like until now it was only used to determine direct compactor's migratetype which is irrelevant in those contexts.

Quoted 2 lines

OK, I see. This is really subtle. One way to go would be to provide a fake gfp_mask for them. How does the following look to you?

Looks OK. I'll have to think about the kcompactd case, as gfp mask implying unmovable migratetype might restrict it without good reason. But that would be separate patch anyway, yours doesn't change that (empty gfp_mask also means unmovable migratetype) and that's good.

Quoted 2 lines · Vlastimil Babka

Quoted 1 line · Michal Hocko

On Fri 07-10-16 07:27:37, Vlastimil Babka wrote:

[...]

Quoted 8 lines

Quoted 7 lines

Quoted 4 lines

But make sure you don't break kcompactd and manual compaction from /proc, as they don't currently set cc->gfp_mask. Looks like until now it was only used to determine direct compactor's migratetype which is irrelevant in those contexts.

OK, I see. This is really subtle. One way to go would be to provide a fake gfp_mask for them. How does the following look to you?

Quoted 4 lines

Looks OK. I'll have to think about the kcompactd case, as gfp mask implying unmovable migratetype might restrict it without good reason. But that would be separate patch anyway, yours doesn't change that (empty gfp_mask also means unmovable migratetype) and that's good.

OK, I see. A follow up patch would be really trivial AFAICS. Just add __GFP_MOVABLE to the mask. But I am not familiar with all these details enough to propose a patch with full description.