New: Single process tries to use more than 1/2 physical RAM, OS starts thrashing
A message that is not in the archive (sent off-list or not yet received): [email protected]/
Andrew Morton
maintainer(switched to email. Please respond via emailed reply-to-all, not via the bugzilla web interface).
This is ... interesting.
Quoted 34 lines · On Thu, 12 Nov 2015 18:46:35 +0000 [email protected] wrote:
https://bugzilla.kernel.org/show_bug.cgi?id=107771
Bug ID: 107771
Summary: Single process tries to use more than 1/2 physical
RAM, OS starts thrashing
Product: Memory Management
Version: 2.5
Kernel Version: 4.3.0-040300-generic (Ubuntu)
Hardware: All
OS: Linux
Tree: Mainline
Status: NEW
Severity: normal
Priority: P1
Component: Page Allocator
Assignee: [email protected]
Reporter: [email protected]
Regression: NoI have a 24-core (48 thread) system with 64GB of RAM.
When I run multiple processes, I can use all of physical RAM before swapping starts. However, if I'm running only a *single* process, the system will start swapping after I've exceeded only 1/2 of available physical RAM. Only after swap fills does it start using more of the physical RAM.
I can't find any ulimit settings or anything else that would cause this to happen intentionally.
I had originally filed this against Ubuntu, but I'm now running a more recent kernel, and the problem persists, so I think it's more appropriate to file here. There are some logs that they had me collect, so if you want to see them, the are here:
Quoted 10 lines
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1513673
I don't recall this problem happening with older kernels (whatever came with Ubuntu 15.04), although I may just not have noticed. By swapping early, I'm limited by the speed of my SSD, which is moving only about 20MB/sec in each direction, and that makes what I'm running take 10 times as long to complete.
Signature · 2 lines
You are receiving this mail because: You are the assignee for the bug.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Quoted 5 lines · Andrew Morton
(switched to email. Please respond via emailed reply-to-all, not via the
bugzilla web interface).This is ... interesting.
First things first. What is the value of/proc/sys/vm/zone_reclaim?
I am assuming this is a two socket system,
with two 12-core CPUs. Am I right?
Quoted 57 lines
Quoted 54 lines · On Thu, 12 Nov 2015 18:46:35 +0000 bugzilla- [email protected] wrote:
https://bugzilla.kernel.org/show_bug.cgi?id=107771
Bug ID: 107771
Summary: Single process tries to use more than 1/2 physical
RAM, OS starts thrashing
Product: Memory Management
Version: 2.5
Kernel Version: 4.3.0-040300-generic (Ubuntu)
Hardware: All
OS: Linux
Tree: Mainline
Status: NEW
Severity: normal
Priority: P1
Component: Page Allocator
Assignee: [email protected]
Reporter: [email protected]
Regression: NoI have a 24-core (48 thread) system with 64GB of RAM.
When I run multiple processes, I can use all of physical RAM before swapping
starts. However, if I'm running only a *single* process, the system will start
swapping after I've exceeded only 1/2 of available physical RAM. Only after
swap fills does it start using more of the physical RAM.I can't find any ulimit settings or anything else that would cause this to
happen intentionally.I had originally filed this against Ubuntu, but I'm now running a more recent
kernel, and the problem persists, so I think it's more appropriate to file
here. There are some logs that they had me collect, so if you want to see
them, the are here:https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1513673
I don't recall this problem happening with older kernels (whatever came with
Ubuntu 15.04), although I may just not have noticed. By swapping early, I'm
limited by the speed of my SSD, which is moving only about 20MB/sec in each
direction, and that makes what I'm running take 10 times as long to complete.
Signature · 1 line
All Rights Reversed.
Quoted 9 lines · Rik van Riel
Quoted 5 lines · Andrew Morton
(switched to email. Please respond via emailed reply-to-all, not via the
bugzilla web interface).This is ... interesting.
First things first. What is the value of
/proc/sys/vm/zone_reclaim?
Also snapshots of /proc/vmstat taken every 1s or so while you see the trashing would be helpful.
Signature · 8 lines
Michal Hocko SUSE Labs -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Quoted 6 lines · Michal Hocko
Quoted 5 lines · Rik van Riel
Quoted 4 lines · Andrew Morton
(switched to email. Please respond via emailed reply-to-all, not via the
bugzilla web interface).
Quoted 7 lines
Quoted 4 lines
Quoted 1 line
This is ... interesting.
First things first. What is the value of
/proc/sys/vm/zone_reclaim?Also snapshots of
/proc/vmstattaken every 1s or so while you see the trashing would be helpful.
It's been so long since I reported this bug that I don't recall exactly what I was doing. I think I was running Synopsys. I tried artificially reproducing this by just allocating a huge amount of memory and touching all the pages, but the problem didn't manifest. I wonder if Synopsys was allocating pages it didn't touch or mmaping files or some other weird thing.
Quoted 3 lines
--
Michal Hocko
SUSE Labs
Signature · 10 lines
Timothy Normand Miller, PhD Assistant Professor of Computer Science, Binghamton University http://www.cs.binghamton.edu/~millerti/ Open Graphics Project -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Quoted 9 lines · Rik van Riel
Quoted 5 lines · Andrew Morton
(switched to email. Please respond via emailed reply-to-all, not via the
bugzilla web interface).This is ... interesting.
First things first. What is the value of
/proc/sys/vm/zone_reclaim?
There is no such thing on this system. However:
$ cat /proc/sys/vm/zone_reclaim_mode 0
Quoted 3 lines
I am assuming this is a two socket system,
with two 12-core CPUs. Am I right?
Yes.
Quoted 61 lines
Quoted 49 lines
Quoted 45 lines · On Thu, 12 Nov 2015 18:46:35 +0000 bugzilla- [email protected] wrote:
https://bugzilla.kernel.org/show_bug.cgi?id=107771
Bug ID: 107771
Summary: Single process tries to use more than 1/2
physical
RAM, OS starts thrashing
Product: Memory Management
Version: 2.5
Kernel Version: 4.3.0-040300-generic (Ubuntu)
Hardware: All
OS: Linux
Tree: Mainline
Status: NEW
Severity: normal
Priority: P1
Component: Page Allocator
Assignee: [email protected]
Reporter: [email protected]
Regression: NoI have a 24-core (48 thread) system with 64GB of RAM.
When I run multiple processes, I can use all of physical RAM before swapping
starts. However, if I'm running only a *single* process, the system will start
swapping after I've exceeded only 1/2 of available physical RAM. Only after
swap fills does it start using more of the physical RAM.I can't find any ulimit settings or anything else that would cause this to
happen intentionally.I had originally filed this against Ubuntu, but I'm now running a more recent
kernel, and the problem persists, so I think it's more appropriate to file
here. There are some logs that they had me collect, so if you want to see
them, the are here:https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1513673
Quoted 9 lines
Quoted 9 lines
I don't recall this problem happening with older kernels (whatever came with
Ubuntu 15.04), although I may just not have noticed. By swapping early, I'm
limited by the speed of my SSD, which is moving only about 20MB/sec in each
direction, and that makes what I'm running take 10 times as long to complete.--
All Rights Reversed.
Signature · 10 lines
Timothy Normand Miller, PhD Assistant Professor of Computer Science, Binghamton University http://www.cs.binghamton.edu/~millerti/ Open Graphics Project -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"[email protected]"> [email protected] </a>