Skip to contentSkip to conversation

New: Very slow access on HDD when a swap partition is used on this HDD on kernel 6.16

Bug 220605 Andrew Morton 3 participants 6 messages Sep 29, 2025 – Oct 16, 2025 1 message not in the archive Memory management

lore
Share & tools
lore

A message that is not in the archive (sent off-list or not yet received): [email protected]/

Andrew Morton

author maintainer
#1 lore raw

(switched to email. Please respond via emailed reply-to-all, not via the bugzilla web interface).

Quoted 17 lines · On Sat, 27 Sep 2025 08:07:59 +0000 [email protected] wrote:

https://bugzilla.kernel.org/show_bug.cgi?id=220605

Bug ID: 220605
Summary: Very slow access on HDD when a swap partition is used
on this HDD on kernel 6.16
Product: Memory Management
Version: 2.5
Hardware: Intel
OS: Linux
Status: NEW
Severity: high
Priority: P3
Component: Page Allocator
Assignee: [email protected]
Reporter: [email protected]
Regression: No

Quoted 3 lines

Hello everybody,

I have a strange behavior on my system (Fedora 42 up to date => kernel 6.16.8).

You certainly do. I wonder what we did to cause this.

I'll cc the MM development list. Please prefer to report issues over email this way - we don't use bugzilla and it's rather annoying that MM bugzilla even exists.

Quoted 75 lines

I have many disks but but my two swap partitions are on 2 HDD, sda and sdb, which are identical (SEAGATE Barracuda 2To). Swap partition on sda has a higher priority than the one on sdb.

I don't use zram.

When a swap partion is used, the disk on wich it seats is very slow.

$ sudo hdparm -t /dev/sda
/dev/sda:
 Timing buffered disk reads:  24 MB in  3.72 seconds =   6.46 MB/sec
$ sudo hdparm -t /dev/sdb
/dev/sdb:
 Timing buffered disk reads: 540 MB in  3.16 seconds = 170.85 MB/sec
pierre@pierre:~$ 

Sometimes, speed gets down to less than 2 MB/s.

Lets turn swap off => swapoff -a

$ sudo hdparm -t /dev/sdb
/dev/sdb:
 Timing buffered disk reads: 462 MB in  3.00 seconds = 153.75 MB/sec
$ sudo hdparm -t /dev/sda

/dev/sda:
Timing buffered disk reads: 474 MB in 3.00 seconds = 157.84 MB/sec
pierre@pierre:~$

Now speed is normal.

Lets put swap on again => sudo swapon -a

Speed is still normal :

$ sudo hdparm -t /dev/sda
/dev/sda:
 Timing buffered disk reads: 612 MB in  3.01 seconds = 203.56 MB/sec
$ sudo hdparm -t /dev/sdb
/dev/sdb:
 Timing buffered disk reads: 570 MB in  3.00 seconds = 189.95 MB/sec
pierre@pierre:~$ 

But swap is not used

MiB Mem :  31952,0 total,  27091,7 free,   3057,6 used,   2343,0 buff/cache     
MiB Swap:  36000,0 total,  36000,0 free,      0,0 used.  28894,4 avail Mem 

So I suspect that, when the swap is used, there is some sort of lock that slows the whole disk that contains the swap partition that is being used.

Logically, the system becomes very slow.

There is nothing in the journal that seems related to this problem.

Since the virtual memory has been modified in 6.16, I suspect a nasty bug.

This affects 6.16.5 to 6.16.8.

This appeared "recently", probably with 6.16

Thanks,

Regards,

Pierre

Signature · 4 lines
You may reply to this email to add a comment.

You are receiving this mail because:
You are the assignee for the bug.
#2 lore raw

Hi Andrew and people reading those lists,

I still struggle to understand what is wrong with me.

I rebuilt my mdadm + bcache array to be sure it was not something linked to a (too) old configuration. Upgrading to 6.16.10 didn't change anything.

One symptom of the problem I identified is the time of the swapoff command (I have 32G, 12G are free and there is less than 1G in the swap)

MiB Mem : 31952,0 total, 12646,2 free,  2226,8 used, 17587,5 buff/cache MiB Swap: 36000,0 total, 35212,2 free,   787,8 used. 29725,2 avail Mem

*$ time sudo swapoff -a
* *
real6m41,276s
user0m0,005s
sys0m0,028s*

Waiting for more than  6 minutes elapsed for 33 ms of work is bizarre.

During that time, if I try to launch another tab with *iotop*, nothing is displayed during at least one minute.

During that time, my computer is almost idle.

So it looks like swapoff and iotop are stucked, waiting for some kind of a semaphore inside the kernel.

Any idea  to dig further ?

Thanks,

Pierre

Le 30/09/2025 à 01:07, Andrew Morton a écrit :

Quoted 107 lines

(switched to email. Please respond via emailed reply-to-all, not via the bugzilla web interface).

Quoted 17 lines · On Sat, 27 Sep 2025 08:07:59 [email protected] wrote:

https://bugzilla.kernel.org/show_bug.cgi?id=220605

Bug ID: 220605
Summary: Very slow access on HDD when a swap partition is used
on this HDD on kernel 6.16
Product: Memory Management
Version: 2.5
Hardware: Intel
OS: Linux
Status: NEW
Severity: high
Priority: P3
Component: Page Allocator
Assignee:[email protected] Reporter:[email protected]
Regression: No

Quoted 3 lines

Hello everybody,

I have a strange behavior on my system (Fedora 42 up to date => kernel 6.16.8).

You certainly do. I wonder what we did to cause this.

I'll cc the MM development list. Please prefer to report issues over email this way - we don't use bugzilla and it's rather annoying that MM bugzilla even exists.

Quoted 75 lines

I have many disks but but my two swap partitions are on 2 HDD, sda and sdb, which are identical (SEAGATE Barracuda 2To). Swap partition on sda has a higher priority than the one on sdb.

I don't use zram.

When a swap partion is used, the disk on wich it seats is very slow.

$ sudo hdparm -t /dev/sda
/dev/sda:
  Timing buffered disk reads:  24 MB in  3.72 seconds =   6.46 MB/sec
$ sudo hdparm -t /dev/sdb
/dev/sdb:
  Timing buffered disk reads: 540 MB in  3.16 seconds = 170.85 MB/sec
pierre@pierre:~$

Sometimes, speed gets down to less than 2 MB/s.

Lets turn swap off => swapoff -a

$ sudo hdparm -t /dev/sdb
/dev/sdb:
  Timing buffered disk reads: 462 MB in  3.00 seconds = 153.75 MB/sec
$ sudo hdparm -t /dev/sda

/dev/sda:
Timing buffered disk reads: 474 MB in 3.00 seconds = 157.84 MB/sec
pierre@pierre:~$

Now speed is normal.

Lets put swap on again => sudo swapon -a

Speed is still normal :

$ sudo hdparm -t /dev/sda
/dev/sda:
  Timing buffered disk reads: 612 MB in  3.01 seconds = 203.56 MB/sec
$ sudo hdparm -t /dev/sdb
/dev/sdb:
  Timing buffered disk reads: 570 MB in  3.00 seconds = 189.95 MB/sec
pierre@pierre:~$

But swap is not used

MiB Mem :  31952,0 total,  27091,7 free,   3057,6 used,   2343,0 buff/cache
MiB Swap:  36000,0 total,  36000,0 free,      0,0 used.  28894,4 avail Mem

So I suspect that, when the swap is used, there is some sort of lock that slows the whole disk that contains the swap partition that is being used.

Logically, the system becomes very slow.

There is nothing in the journal that seems related to this problem.

Since the virtual memory has been modified in 6.16, I suspect a nasty bug.

This affects 6.16.5 to 6.16.8.

This appeared "recently", probably with 6.16

Thanks,

Regards,

Pierre

Signature · 4 lines
You may reply to this email to add a comment.

You are receiving this mail because:
You are the assignee for the bug.
#3 lore raw
Quoted 2 lines · Pierre Juhen

Hi Andrew and people reading those lists,

Hi Pierre,

Quoted 111 lines

I still struggle to understand what is wrong with me.

I rebuilt my mdadm + bcache array to be sure it was not something linked to a (too) old configuration. Upgrading to 6.16.10 didn't change anything.

One symptom of the problem I identified is the time of the swapoff command (I have 32G, 12G are free and there is less than 1G in the swap)

MiB Mem :  31952,0 total,  12646,2 free,   2226,8 used,  17587,5 buff/cache
MiB Swap:  36000,0 total,  35212,2 free,    787,8 used.  29725,2 avail Mem
$ time sudo swapoff -a
real    6m41,276s
user    0m0,005s
sys     0m0,028s

Waiting for more than 6 minutes elapsed for 33 ms of work is bizarre.

During that time, if I try to launch another tab with iotop, nothing is displayed during at least one minute.

During that time, my computer is almost idle.

So it looks like swapoff and iotop are stucked, waiting for some kind of a semaphore inside the kernel.

Any idea to dig further ?

Thanks,

Pierre

Le 30/09/2025 à 01:07, Andrew Morton a écrit :

(switched to email. Please respond via emailed reply-to-all, not via the bugzilla web interface).

On Sat, 27 Sep 2025 08:07:59 +0000 [email protected] wrote:

https://bugzilla.kernel.org/show_bug.cgi?id=220605

Bug ID: 220605
Summary: Very slow access on HDD when a swap partition is used
on this HDD on kernel 6.16
Product: Memory Management
Version: 2.5
Hardware: Intel
OS: Linux
Status: NEW
Severity: high
Priority: P3
Component: Page Allocator
Assignee: [email protected]
Reporter: [email protected]
Regression: No

Hello everybody,

I have a strange behavior on my system (Fedora 42 up to date => kernel 6.16.8).

You certainly do. I wonder what we did to cause this.

I'll cc the MM development list. Please prefer to report issues over email this way - we don't use bugzilla and it's rather annoying that MM bugzilla even exists.

I have many disks but but my two swap partitions are on 2 HDD, sda and sdb, which are identical (SEAGATE Barracuda 2To). Swap partition on sda has a higher priority than the one on sdb.

I don't use zram.

When a swap partion is used, the disk on wich it seats is very slow.

$ sudo hdparm -t /dev/sda
/dev/sda:
 Timing buffered disk reads:  24 MB in  3.72 seconds =   6.46 MB/sec
$ sudo hdparm -t /dev/sdb
/dev/sdb:
 Timing buffered disk reads: 540 MB in  3.16 seconds = 170.85 MB/sec
pierre@pierre:~$

Sometimes, speed gets down to less than 2 MB/s.

Lets turn swap off => swapoff -a

$ sudo hdparm -t /dev/sdb
/dev/sdb:
 Timing buffered disk reads: 462 MB in  3.00 seconds = 153.75 MB/sec
$ sudo hdparm -t /dev/sda

/dev/sda:
Timing buffered disk reads: 474 MB in 3.00 seconds = 157.84 MB/sec
pierre@pierre:~$

Now speed is normal.

Lets put swap on again => sudo swapon -a

Speed is still normal :

$ sudo hdparm -t /dev/sda
/dev/sda:
 Timing buffered disk reads: 612 MB in  3.01 seconds = 203.56 MB/sec
$ sudo hdparm -t /dev/sdb
/dev/sdb:
 Timing buffered disk reads: 570 MB in  3.00 seconds = 189.95 MB/sec
pierre@pierre:~$
Quoted 5 lines

But swap is not used

MiB Mem :  31952,0 total,  27091,7 free,   3057,6 used,   2343,0 buff/cache
MiB Swap:  36000,0 total,  36000,0 free,      0,0 used.  28894,4 avail Mem

So the disk is already having a performance regression after swapon, while there is no memory pressure and SWAP is not used yet?

Are you using swapon on the whole disk / partition or swapon on a file on top of the disk / partition?

Quoted 5 lines

So I suspect that, when the swap is used, there is some sort of lock that slows the whole disk that contains the swap partition that is being used.

Logically, the system becomes very slow.

This is really strange if swap is not actually used. I just tested with HDD locally and didn't find anything wrong about it.

#4 lore raw
Quoted 28 lines · Kairui Song
Quoted 2 lines · Pierre Juhen

Hi Andrew and people reading those lists,

Hi Pierre,

Quoted 8 lines
$ sudo hdparm -t /dev/sda
/dev/sda:
 Timing buffered disk reads: 612 MB in  3.01 seconds = 203.56 MB/sec
$ sudo hdparm -t /dev/sdb
/dev/sdb:
 Timing buffered disk reads: 570 MB in  3.00 seconds = 189.95 MB/sec
pierre@pierre:~$
Quoted 5 lines

But swap is not used

MiB Mem :  31952,0 total,  27091,7 free,   3057,6 used,   2343,0 buff/cache
MiB Swap:  36000,0 total,  36000,0 free,      0,0 used.  28894,4 avail Mem

So the disk is already having a performance regression after swapon, while there is no memory pressure and SWAP is not used yet?

Are you using swapon on the whole disk / partition or swapon on a file on top of the disk / partition?

Quoted 1 line
Quoted 7 lines
Quoted 4 lines

So I suspect that, when the swap is used, there is some sort of lock that slows the whole disk that contains the swap partition that is being used.

Logically, the system becomes very slow.

This is really strange if swap is not actually used. I just tested with HDD locally and didn't find anything wrong about it.

I just checked the bugzilla page more carefully, sorry I had some misunderstanding of what is going on just now.

So usually when there is swap IO on an HDD, it will cause performance regression because HDDs are slow and really bad at random IO. So if you are seeing a lower performance with recent kernels, could you help check if that's because the memory pressure is higher, or the kernel prefers to do SWAP more aggressively now, or the performance is just worse when swap is working? Maybe you can check and compare the swap usage after the same workload before / after the regression?

I'll also try to test again locally with some slight pressure on swap.

#5 lore raw
In reply to Andrew Morton

Hi, Andrew, Kairui,

I finally partially found the cause of the issue.

The guilty process was baloo_file_extract (part of the KDE file indexer). The index was abnormally large, probably corrupted. I stopped it,  removed the index and restarted it, and it's now OK.

I found that by looking to the number of major page faults. In 2 hours, the process had more than 300k major page faults, whilst a few other processes (chrome, thunderbird) had only a few hundreds.

The program itself is small : 160k.

The question is now, why this faulty process was putting such a pressure on the swap disk, whilst there was a lot of free memory ?

Use of mmap for the index ?

When mmap gets a page fault, it goes through the swap instead of looking directly into the file ?

Thanks,

Regards,

Pierre

*_Iostat -x before _*

pierre@pierre:~$ iostat -x
Linux 6.16.11-200.fc42.x86_64 (pierre.juhen)    15/10/2025 _x86_64_        (12 CPU)

avg-cpu:  %user   %nice %system %iowait  %steal   %idle 0,83    0,08    0,63 *13,28*   0,00   85,17

Device            r/s     rkB/s   rrqm/s  %rrqm r_await rareq-sz     w/s     wkB/s   wrqm/s  %wrqm w_await wareq-sz  d/s     dkB/s   drqm/s  %drqm d_await dareq-sz     f/s f_await aqu-sz  %util bcache0         87,97   3264,68     0,00   0,00    4,63 37,11   17,88    212,65     0,00   0,00   74,93    11,89 0,55      0,00     0,00   0,00  155,10     0,00    0,00    0,00   1,83  30,36 dm-0             0,21      4,82     0,00   0,00   53,06 23,42    0,01      0,03     0,00   0,00   32,43     2,86 0,00      0,00     0,00   0,00    0,00     0,00    0,00    0,00   0,01   0,38 dm-1             1,35      9,15     0,00   0,00   49,42  6,80    0,01      0,03     0,00   0,00   52,43     2,86 0,00      0,00     0,00   0,00    0,00     0,00    0,00    0,00   0,07   2,35 dm-10           50,24    891,59     0,00   0,00    6,53 17,75   11,12    224,31     0,00   0,00  192,55    20,16 0,40    542,69     0,00   0,00  165,62  1357,06    0,00    0,00   2,54  23,27 dm-2             0,04      2,55     0,00   0,00   23,14 70,10    0,00      0,00     0,00   0,00    0,00     0,00 0,00      0,00     0,00   0,00    0,00     0,00    0,00    0,00   0,00   0,08 dm-3             1,46     11,72     0,00   0,00    0,42  8,01    0,01      0,03     0,00   0,00   33,00     2,86 0,00      0,00     0,00   0,00    0,00     0,00    0,00    0,00   0,00   0,05 dm-4            50,28    893,54     0,00   0,00    6,50 17,77   11,12    224,31     0,00   0,00   76,19    20,16 0,40    542,69     0,00   0,00  165,62  1357,06    0,00    0,00   1,24  23,26 dm-5            36,18   2355,75     0,00   0,00    2,22 65,11    6,74    101,36     0,00   0,00   72,94    15,03 0,15     93,81     0,00   0,00  127,33   619,31    0,00    0,00   0,59  15,11 dm-6             0,08      1,81     0,00   0,00    0,28 21,39    0,00      0,00     0,00   0,00    0,00     0,00 0,00      0,00     0,00   0,00    0,00     0,00    0,00    0,00   0,00   0,00 dm-7             0,08      1,81     0,00   0,00    0,15 21,39    0,00      0,00     0,00   0,00    0,00     0,00 0,00      0,00     0,00   0,00    0,00     0,00    0,00    0,00   0,00   0,00 dm-8             0,13      2,84     0,00   0,00    0,95 21,55    0,00      0,00     0,00   0,00   11,00     2,00 0,00      0,00     0,00   0,00    0,00     0,00    0,00    0,00   0,00   0,01 *dm-9            84,74    341,28  0,00   0,00   18,21     4,03  110,88    443,53     0,00  0,00  175,27     4,00    0,00      0,00     0,00   0,00 0,00     0,00    0,00    0,00   20,98  95,69* md126            5,36    388,18     0,00   0,00   95,94 72,40   17,66    325,71     0,00   0,00   63,39    18,44 0,00      0,00     0,00   0,00    0,00     0,00    0,00    0,00   1,63  17,62 md127            1,62     19,41     0,00   0,00   48,64 11,96    0,02      0,07     0,00   0,00   48,10     4,00 0,00      0,00     0,00   0,00    0,00     0,00    0,00    0,00   0,08   2,57 nvme0n1         66,66   2910,01    31,04  31,77    0,28 43,65   12,96    379,13     7,71  37,31    0,57    29,25 1,93    989,62     0,00   0,00    0,64   512,69    1,78    0,43   0,03   0,53 sda             60,79    557,38    28,51  31,92   21,92  9,17   33,52    615,59    87,88  72,39   40,19    18,37 0,44    318,50     0,00   0,00  116,42   731,53    0,95  233,14   2,95  92,36 sdb              2,60    204,98     0,46  14,97   11,26 78,94    4,41    153,72     4,19  48,69   42,32    34,84 0,25    318,00     0,00   0,00   22,67  1258,16    0,98   80,22   0,30   7,48 sdc              0,28      6,72     0,00   0,00    0,20 23,82    0,00      0,00     0,00   0,00    0,00     0,00 0,00      0,00     0,00   0,00    0,00     0,00    0,00    0,00   0,00   0,00 sdd              0,09      2,00     0,00   0,00    0,25 22,45    0,00      0,00     0,00   0,00    0,00     0,00 0,00      0,00     0,00   0,00    0,00     0,00    0,00    0,00   0,00   0,00

*_iostat -x now _*

iostat -x
Linux 6.16.11-200.fc42.x86_64 (pierre.juhen) 15/10/2025 _x86_64_(12 CPU)

avg-cpu:  %user   %nice %system %iowait  %steal   %idle 0,75    0,01    0,45 2,04   0,00  96,74

Device            r/s     rkB/s   rrqm/s  %rrqm r_await rareq-sz     w/s     wkB/s   wrqm/s  %wrqm w_await wareq-sz     d/s     dkB/s   drqm/s  %drqm d_await dareq-sz     f/s f_await  aqu-sz  %util bcache0    35,15   1114,60    0,00   0,00   5,57    31,71   12,76    152,85    0,00   0,00  34,75    11,98    0,41     0,00     0,00   0,00  52,19    0,00    0,00    0,00   0,66  13,69 dm-0     0,08      1,84    0,00   0,00  48,12    22,33   0,00      0,00     0,00   0,00  36,50     1,20   0,00      0,00     0,00   0,00    0,00     0,00    0,00    0,00    0,00  0,27 dm-1     0,56      3,83    0,00   0,00  48,28     6,81   0,00      0,00     0,00   0,00  36,00     1,20   0,00      0,00     0,00   0,00    0,00     0,00    0,00    0,00   0,03   0,37 dm-10    20,26    234,41    0,00   0,00   7,64    11,57    9,76    155,66    0,00   0,00  45,01    15,95    0,35    312,56    0,00   0,00  44,31   889,08   0,00    0,00   0,61  10,78 dm-2     0,02      1,08    0,00   0,00   9,55    70,10   0,00      0,00     0,00   0,00    0,00     0,00    0,00      0,00     0,00   0,00    0,00     0,00    0,00    0,00    0,00  0,01 dm-3     0,62      4,42    0,00   0,00   0,32     7,15   0,00      0,00     0,00   0,00  45,60     1,20   0,00      0,00     0,00   0,00    0,00     0,00    0,00    0,00    0,00  0,02 dm-4    20,28    235,23    0,00   0,00   7,62    11,60    9,76    155,66    0,00   0,00  24,69    15,95    0,35    312,56    0,00   0,00  44,30   889,08   0,00    0,00   0,41  10,78 dm-5    14,24    873,40    0,00   0,00   2,89    61,35    3,00     44,04    0,00   0,00  67,53    14,70    0,06     50,70    0,00   0,00  99,84   870,69   0,00    0,00   0,25   5,45 dm-6     0,04      0,77    0,00   0,00   0,24    21,39   0,00      0,00     0,00   0,00    0,00     0,00    0,00      0,00     0,00   0,00    0,00     0,00    0,00    0,00    0,00   0,00 dm-7     0,04      0,77    0,00   0,00   0,14    21,39   0,00      0,00     0,00   0,00    0,00     0,00    0,00      0,00     0,00   0,00    0,00     0,00    0,00    0,00    0,00   0,00 dm-8     0,06      1,20    0,00   0,00   1,41    21,55   0,00      0,00     0,00   0,00  14,50     2,00   0,00      0,00     0,00   0,00    0,00     0,00    0,00    0,00    0,00  0,01 *dm-9     0,06      1,20    0,00   0,00   4,55    21,55   0,00      0,00     0,00   0,00  41,00     2,00   0,00      0,00     0,00   0,00    0,00     0,00    0,00    0,00    0,00  0,03* md126     0,67      7,97    0,00   0,00  46,55    11,82   0,00     0,01    0,00   0,00  11,83     4,00   0,00      0,00     0,00   0,00    0,00     0,00    0,00    0,00   0,03   0,58 md127     1,67    104,20    0,00   0,00 156,95    62,51   12,44    199,71    0,00   0,00  26,66    16,06   0,00      0,00     0,00   0,00    0,00     0,00    0,00    0,00   0,59   5,12 nvme0n1    27,87   1025,06    11,83  29,79    0,29    36,78    8,69    168,45     5,10  36,99    0,75    19,38    0,83    424,68    0,00   0,00   0,69   512,68    1,42    0,39    0,02   0,28 sda     1,54    282,54     0,32 17,10  87,19   183,83    4,69    108,07     2,81  37,49   45,70    23,04    0,33    184,46    0,00   0,00  44,90   551,69    0,81  123,50    0,46   9,53 sdb     0,69     58,24     0,29  29,17   37,97    83,97    3,45     91,65     2,69  43,74   35,91    26,53    0,18    178,80    0,00   0,00  17,37  1006,71    0,81   89,78    0,23   6,83 sdc     0,12      2,84    0,00   0,00   0,20    23,82   0,00      0,00     0,00   0,00    0,00     0,00    0,00      0,00     0,00   0,00    0,00     0,00    0,00    0,00    0,00   0,00 sdd     0,04      0,85    0,00   0,00   0,25    22,45   0,00      0,00     0,00   0,00    0,00     0,00    0,00      0,00     0,00   0,00    0,00     0,00    0,00    0,00    0,00   0,00

Le 30/09/2025 à 01:07, Andrew Morton a écrit :

Quoted 107 lines

(switched to email. Please respond via emailed reply-to-all, not via the bugzilla web interface).

Quoted 17 lines · On Sat, 27 Sep 2025 08:07:59 [email protected] wrote:

https://bugzilla.kernel.org/show_bug.cgi?id=220605

Bug ID: 220605
Summary: Very slow access on HDD when a swap partition is used
on this HDD on kernel 6.16
Product: Memory Management
Version: 2.5
Hardware: Intel
OS: Linux
Status: NEW
Severity: high
Priority: P3
Component: Page Allocator
Assignee:[email protected] Reporter:[email protected]
Regression: No

Quoted 3 lines

Hello everybody,

I have a strange behavior on my system (Fedora 42 up to date => kernel 6.16.8).

You certainly do. I wonder what we did to cause this.

I'll cc the MM development list. Please prefer to report issues over email this way - we don't use bugzilla and it's rather annoying that MM bugzilla even exists.

Quoted 75 lines

I have many disks but but my two swap partitions are on 2 HDD, sda and sdb, which are identical (SEAGATE Barracuda 2To). Swap partition on sda has a higher priority than the one on sdb.

I don't use zram.

When a swap partion is used, the disk on wich it seats is very slow.

$ sudo hdparm -t /dev/sda
/dev/sda:
  Timing buffered disk reads:  24 MB in  3.72 seconds =   6.46 MB/sec
$ sudo hdparm -t /dev/sdb
/dev/sdb:
  Timing buffered disk reads: 540 MB in  3.16 seconds = 170.85 MB/sec
pierre@pierre:~$

Sometimes, speed gets down to less than 2 MB/s.

Lets turn swap off => swapoff -a

$ sudo hdparm -t /dev/sdb
/dev/sdb:
  Timing buffered disk reads: 462 MB in  3.00 seconds = 153.75 MB/sec
$ sudo hdparm -t /dev/sda

/dev/sda:
Timing buffered disk reads: 474 MB in 3.00 seconds = 157.84 MB/sec
pierre@pierre:~$

Now speed is normal.

Lets put swap on again => sudo swapon -a

Speed is still normal :

$ sudo hdparm -t /dev/sda
/dev/sda:
  Timing buffered disk reads: 612 MB in  3.01 seconds = 203.56 MB/sec
$ sudo hdparm -t /dev/sdb
/dev/sdb:
  Timing buffered disk reads: 570 MB in  3.00 seconds = 189.95 MB/sec
pierre@pierre:~$

But swap is not used

MiB Mem :  31952,0 total,  27091,7 free,   3057,6 used,   2343,0 buff/cache
MiB Swap:  36000,0 total,  36000,0 free,      0,0 used.  28894,4 avail Mem

So I suspect that, when the swap is used, there is some sort of lock that slows the whole disk that contains the swap partition that is being used.

Logically, the system becomes very slow.

There is nothing in the journal that seems related to this problem.

Since the virtual memory has been modified in 6.16, I suspect a nasty bug.

This affects 6.16.5 to 6.16.8.

This appeared "recently", probably with 6.16

Thanks,

Regards,

Pierre

Signature · 4 lines
You may reply to this email to add a comment.

You are receiving this mail because:
You are the assignee for the bug.
#6 lore raw
In reply to Andrew Morton

HI Andrew, Kairui,

I dug a bit around this issue.

The key-value database engine behind baloo is lmdb <https://github.com/LMDB/lmdb> (https://github.com/LMDB/lmdb).

lmdb uses mmap.

The problems of baloo are not new. If you search "baloo_file CPU", you will find hundreds of messages complaining about an heavy load on CPU and RAM. (but not on the swap).

I already had the problem, that was quite easy to find with top.

Now something seems to have changed, with 6.16.

Instead of having high CPU usage we have a strong pressure on the swap disk.

It could mean that, when there is a filemap_fault (that's normal because the index is not loaded in memory in one pass), it induces a page fault that involves the swap.

The swap is therefore overloaded, and the system slows down dramatically.

So the key issue is to understand why we shift from a filemap_fault to a major page fault.

Regards,

Pierre

Hi, Andrew, Kairui,

I finally partially found the cause of the issue.

The guilty process was baloo_file_extract (part of the KDE file indexer). The index was abnormally large, probably corrupted. I stopped it,  removed the index and restarted it, and it's now OK.

I found that by looking to the number of major page faults. In 2 hours, the process had more than 300k major page faults, whilst a few other processes (chrome, thunderbird) had only a few hundreds.

The program itself is small : 160k.

The question is now, why this faulty process was putting such a pressure on the swap disk, whilst there was a lot of free memory ?

Use of mmap for the index ?

When mmap gets a page fault, it goes through the swap instead of looking directly into the file ?

Thanks,

Regards,

Pierre

*_Iostat -x before _*

pierre@pierre:~$ iostat -x
Linux 6.16.11-200.fc42.x86_64 (pierre.juhen)    15/10/2025 _x86_64_        (12 CPU)

avg-cpu:  %user   %nice %system %iowait  %steal   %idle 0,83    0,08    0,63 *13,28*   0,00   85,17

Device            r/s     rkB/s   rrqm/s  %rrqm r_await rareq-sz     w/s     wkB/s   wrqm/s  %wrqm w_await wareq-sz  d/s     dkB/s   drqm/s  %drqm d_await dareq-sz     f/s f_await aqu-sz  %util bcache0         87,97   3264,68     0,00   0,00    4,63 37,11   17,88    212,65     0,00   0,00   74,93    11,89 0,55      0,00     0,00   0,00  155,10     0,00    0,00    0,00   1,83  30,36 dm-0             0,21      4,82     0,00   0,00   53,06 23,42    0,01      0,03     0,00   0,00   32,43     2,86 0,00      0,00     0,00   0,00    0,00     0,00    0,00    0,00   0,01   0,38 dm-1             1,35      9,15     0,00   0,00   49,42  6,80    0,01      0,03     0,00   0,00   52,43     2,86 0,00      0,00     0,00   0,00    0,00     0,00    0,00    0,00   0,07   2,35 dm-10           50,24    891,59     0,00   0,00    6,53 17,75   11,12    224,31     0,00   0,00  192,55    20,16 0,40    542,69     0,00   0,00  165,62  1357,06    0,00    0,00   2,54  23,27 dm-2             0,04      2,55     0,00   0,00   23,14 70,10    0,00      0,00     0,00   0,00    0,00     0,00 0,00      0,00     0,00   0,00    0,00     0,00    0,00    0,00   0,00   0,08 dm-3             1,46     11,72     0,00   0,00    0,42  8,01    0,01      0,03     0,00   0,00   33,00     2,86 0,00      0,00     0,00   0,00    0,00     0,00    0,00    0,00   0,00   0,05 dm-4            50,28    893,54     0,00   0,00    6,50 17,77   11,12    224,31     0,00   0,00   76,19    20,16 0,40    542,69     0,00   0,00  165,62  1357,06    0,00    0,00   1,24  23,26 dm-5            36,18   2355,75     0,00   0,00    2,22 65,11    6,74    101,36     0,00   0,00   72,94    15,03 0,15     93,81     0,00   0,00  127,33   619,31    0,00    0,00   0,59  15,11 dm-6             0,08      1,81     0,00   0,00    0,28 21,39    0,00      0,00     0,00   0,00    0,00     0,00 0,00      0,00     0,00   0,00    0,00     0,00    0,00    0,00   0,00   0,00 dm-7             0,08      1,81     0,00   0,00    0,15 21,39    0,00      0,00     0,00   0,00    0,00     0,00 0,00      0,00     0,00   0,00    0,00     0,00    0,00    0,00   0,00   0,00 dm-8             0,13      2,84     0,00   0,00    0,95 21,55    0,00      0,00     0,00   0,00   11,00     2,00 0,00      0,00     0,00   0,00    0,00     0,00    0,00    0,00   0,00   0,01 *dm-9            84,74    341,28  0,00   0,00   18,21     4,03  110,88    443,53     0,00  0,00  175,27     4,00    0,00      0,00     0,00   0,00 0,00     0,00    0,00    0,00   20,98  95,69* md126            5,36    388,18     0,00   0,00   95,94 72,40   17,66    325,71     0,00   0,00   63,39    18,44 0,00      0,00     0,00   0,00    0,00     0,00    0,00    0,00   1,63  17,62 md127            1,62     19,41     0,00   0,00   48,64 11,96    0,02      0,07     0,00   0,00   48,10     4,00 0,00      0,00     0,00   0,00    0,00     0,00    0,00    0,00   0,08   2,57 nvme0n1         66,66   2910,01    31,04  31,77    0,28 43,65   12,96    379,13     7,71  37,31    0,57    29,25 1,93    989,62     0,00   0,00    0,64   512,69    1,78    0,43   0,03   0,53 sda             60,79    557,38    28,51  31,92   21,92  9,17   33,52    615,59    87,88  72,39   40,19    18,37 0,44    318,50     0,00   0,00  116,42   731,53    0,95  233,14   2,95  92,36 sdb              2,60    204,98     0,46  14,97   11,26 78,94    4,41    153,72     4,19  48,69   42,32    34,84 0,25    318,00     0,00   0,00   22,67  1258,16    0,98   80,22   0,30   7,48 sdc              0,28      6,72     0,00   0,00    0,20 23,82    0,00      0,00     0,00   0,00    0,00     0,00 0,00      0,00     0,00   0,00    0,00     0,00    0,00    0,00   0,00   0,00 sdd              0,09      2,00     0,00   0,00    0,25 22,45    0,00      0,00     0,00   0,00    0,00     0,00 0,00      0,00     0,00   0,00    0,00     0,00    0,00    0,00   0,00   0,00

*_iostat -x now _*

iostat -x
Linux 6.16.11-200.fc42.x86_64 (pierre.juhen) 15/10/2025 _x86_64_(12 CPU)

avg-cpu:  %user   %nice %system %iowait  %steal   %idle 0,75    0,01    0,45 2,04   0,00  96,74

Device            r/s     rkB/s   rrqm/s  %rrqm r_await rareq-sz     w/s     wkB/s   wrqm/s  %wrqm w_await wareq-sz     d/s     dkB/s   drqm/s  %drqm d_await dareq-sz     f/s f_await  aqu-sz  %util bcache0    35,15   1114,60    0,00   0,00   5,57    31,71   12,76    152,85    0,00   0,00  34,75    11,98    0,41     0,00     0,00   0,00  52,19    0,00    0,00    0,00   0,66  13,69 dm-0     0,08      1,84    0,00   0,00  48,12    22,33   0,00      0,00     0,00   0,00  36,50     1,20   0,00      0,00     0,00   0,00    0,00     0,00    0,00    0,00    0,00  0,27 dm-1     0,56      3,83    0,00   0,00  48,28     6,81   0,00      0,00     0,00   0,00  36,00     1,20   0,00      0,00     0,00   0,00    0,00     0,00    0,00    0,00   0,03   0,37 dm-10    20,26    234,41    0,00   0,00   7,64    11,57    9,76    155,66    0,00   0,00  45,01    15,95    0,35    312,56    0,00   0,00  44,31   889,08   0,00    0,00   0,61  10,78 dm-2     0,02      1,08    0,00   0,00   9,55    70,10   0,00      0,00     0,00   0,00    0,00     0,00    0,00      0,00     0,00   0,00    0,00     0,00    0,00    0,00    0,00  0,01 dm-3     0,62      4,42    0,00   0,00   0,32     7,15   0,00      0,00     0,00   0,00  45,60     1,20   0,00      0,00     0,00   0,00    0,00     0,00    0,00    0,00    0,00  0,02 dm-4    20,28    235,23    0,00   0,00   7,62    11,60    9,76    155,66    0,00   0,00  24,69    15,95    0,35    312,56    0,00   0,00  44,30   889,08   0,00    0,00   0,41  10,78 dm-5    14,24    873,40    0,00   0,00   2,89    61,35    3,00     44,04    0,00   0,00  67,53    14,70    0,06     50,70    0,00   0,00  99,84   870,69   0,00    0,00   0,25   5,45 dm-6     0,04      0,77    0,00   0,00   0,24    21,39   0,00      0,00     0,00   0,00    0,00     0,00    0,00      0,00     0,00   0,00    0,00     0,00    0,00    0,00    0,00   0,00 dm-7     0,04      0,77    0,00   0,00   0,14    21,39   0,00      0,00     0,00   0,00    0,00     0,00    0,00      0,00     0,00   0,00    0,00     0,00    0,00    0,00    0,00   0,00 dm-8     0,06      1,20    0,00   0,00   1,41    21,55   0,00      0,00     0,00   0,00  14,50     2,00   0,00      0,00     0,00   0,00    0,00     0,00    0,00    0,00    0,00  0,01 *dm-9     0,06      1,20    0,00   0,00   4,55    21,55   0,00      0,00     0,00   0,00  41,00     2,00   0,00      0,00     0,00   0,00    0,00     0,00    0,00    0,00    0,00  0,03* md126     0,67      7,97    0,00   0,00  46,55    11,82   0,00     0,01    0,00   0,00  11,83     4,00   0,00      0,00     0,00   0,00    0,00     0,00    0,00    0,00   0,03   0,58 md127     1,67    104,20    0,00   0,00 156,95    62,51   12,44    199,71    0,00   0,00  26,66    16,06   0,00      0,00     0,00   0,00    0,00     0,00    0,00    0,00   0,59   5,12 nvme0n1    27,87   1025,06    11,83  29,79    0,29    36,78    8,69    168,45     5,10  36,99    0,75    19,38    0,83    424,68    0,00   0,00   0,69   512,68    1,42    0,39    0,02   0,28 sda     1,54    282,54     0,32 17,10  87,19   183,83    4,69    108,07     2,81  37,49   45,70    23,04    0,33    184,46    0,00   0,00  44,90   551,69    0,81  123,50    0,46   9,53 sdb     0,69     58,24     0,29  29,17   37,97    83,97    3,45     91,65     2,69  43,74   35,91    26,53    0,18    178,80    0,00   0,00  17,37  1006,71    0,81   89,78    0,23   6,83 sdc     0,12      2,84    0,00   0,00   0,20    23,82   0,00      0,00     0,00   0,00    0,00     0,00    0,00      0,00     0,00   0,00    0,00     0,00    0,00    0,00    0,00   0,00 sdd     0,04      0,85    0,00   0,00   0,25    22,45   0,00      0,00     0,00   0,00    0,00     0,00    0,00      0,00     0,00   0,00    0,00     0,00    0,00    0,00    0,00   0,00

Le 30/09/2025 à 01:07, Andrew Morton a écrit :

Quoted 107 lines

(switched to email. Please respond via emailed reply-to-all, not via the bugzilla web interface).

Quoted 17 lines · On Sat, 27 Sep 2025 08:07:59 [email protected] wrote:

https://bugzilla.kernel.org/show_bug.cgi?id=220605

Bug ID: 220605
Summary: Very slow access on HDD when a swap partition is used
on this HDD on kernel 6.16
Product: Memory Management
Version: 2.5
Hardware: Intel
OS: Linux
Status: NEW
Severity: high
Priority: P3
Component: Page Allocator
Assignee:[email protected] Reporter:[email protected]
Regression: No

Quoted 3 lines

Hello everybody,

I have a strange behavior on my system (Fedora 42 up to date => kernel 6.16.8).

You certainly do. I wonder what we did to cause this.

I'll cc the MM development list. Please prefer to report issues over email this way - we don't use bugzilla and it's rather annoying that MM bugzilla even exists.

Quoted 75 lines

I have many disks but but my two swap partitions are on 2 HDD, sda and sdb, which are identical (SEAGATE Barracuda 2To). Swap partition on sda has a higher priority than the one on sdb.

I don't use zram.

When a swap partion is used, the disk on wich it seats is very slow.

$ sudo hdparm -t /dev/sda
/dev/sda:
  Timing buffered disk reads:  24 MB in  3.72 seconds =   6.46 MB/sec
$ sudo hdparm -t /dev/sdb
/dev/sdb:
  Timing buffered disk reads: 540 MB in  3.16 seconds = 170.85 MB/sec
pierre@pierre:~$

Sometimes, speed gets down to less than 2 MB/s.

Lets turn swap off => swapoff -a

$ sudo hdparm -t /dev/sdb
/dev/sdb:
  Timing buffered disk reads: 462 MB in  3.00 seconds = 153.75 MB/sec
$ sudo hdparm -t /dev/sda

/dev/sda:
Timing buffered disk reads: 474 MB in 3.00 seconds = 157.84 MB/sec
pierre@pierre:~$

Now speed is normal.

Lets put swap on again => sudo swapon -a

Speed is still normal :

$ sudo hdparm -t /dev/sda
/dev/sda:
  Timing buffered disk reads: 612 MB in  3.01 seconds = 203.56 MB/sec
$ sudo hdparm -t /dev/sdb
/dev/sdb:
  Timing buffered disk reads: 570 MB in  3.00 seconds = 189.95 MB/sec
pierre@pierre:~$

But swap is not used

MiB Mem :  31952,0 total,  27091,7 free,   3057,6 used,   2343,0 buff/cache
MiB Swap:  36000,0 total,  36000,0 free,      0,0 used.  28894,4 avail Mem

So I suspect that, when the swap is used, there is some sort of lock that slows the whole disk that contains the swap partition that is being used.

Logically, the system becomes very slow.

There is nothing in the journal that seems related to this problem.

Since the virtual memory has been modified in 6.16, I suspect a nasty bug.

This affects 6.16.5 to 6.16.8.

This appeared "recently", probably with 6.16

Thanks,

Regards,

Pierre

Signature · 4 lines
You may reply to this email to add a comment.

You are receiving this mail because:
You are the assignee for the bug.