Skip to contentSkip to conversation

paging behavior in Linux

DISCUSSION Manfred Spraul 3 participants 3 messages Feb 28, 2001

lore
Share & tools
lore
#0 lore raw
Quoted 5 lines

When I run my program on a readhat linux machine, I dont get results as expected, work thread seems to be stuck when prefetch thread is waiting on a page fault

That's a known problem:

The paging io for a process is controlled with a per-process semaphore. The semaphore is held while waiting for the actual io. Thus the paging in multi threaded applications is single threaded.
Probably your prefetch thread is waiting for disk io, and the worker thread causes a minor pagefault --> worker thread sleeps until the disk io is completed.

--
Manfred

#1 lore raw
Quoted 6 lines · Manfred Spraul
Quoted 6 lines

When I run my program on a readhat linux machine, I dont get results as expected, work thread seems to be stuck when prefetch thread is waiting on a page fault

Quoted 8 lines

That's a known problem:

The paging io for a process is controlled with a per-process semaphore. The semaphore is held while waiting for the actual io. Thus the paging in multi threaded applications is single threaded.
Probably your prefetch thread is waiting for disk io, and the worker thread causes a minor pagefault --> worker thread sleeps until the disk io is completed.

This behavior is actually pretty annoying. There can be cases where a process wakes up from a page fault, does some work, goes back to sleep on a page fault, thereby keeping it's mmap_sem locked at all times (i.e. vmstat, top, ps unusable) on a UP system. I posted this complaint a while ago, it was discussed by Linus and Andrew Morton about how it also boiled down to semaphore wakeup unfairness (and bugs?). The current semaphore was determined to be too ugly to even look at. So it was dropped.

Is there any way that the mmap_sem could be dropped during the blocking on I/O, and reclaimed after the handle_mm_fault? Probably not, or it'd be done.

It can be a real DOS though, a 'well-written' clobbering program can make ps/vmstat useless. (it's actually /proc/pid/stat that's the killer, IIRC).

David

Signature · 3 lines
David Mansfield                                           (718) 963-2020
[email protected]
Ultramaster Group, LLC                               www.ultramaster.com
#2 lore raw
In reply to Manfred Spraul
Quoted 26 lines · David Mansfield
Quoted 6 lines · Manfred Spraul
Quoted 6 lines

When I run my program on a readhat linux machine, I dont get results as expected, work thread seems to be stuck when prefetch thread is waiting on a page fault

Quoted 8 lines

That's a known problem:

The paging io for a process is controlled with a per-process semaphore. The semaphore is held while waiting for the actual io. Thus the paging in multi threaded applications is single threaded.
Probably your prefetch thread is waiting for disk io, and the worker thread causes a minor pagefault --> worker thread sleeps until the disk io is completed.

This behavior is actually pretty annoying. There can be cases where a process wakes up from a page fault, does some work, goes back to sleep on a page fault, thereby keeping it's mmap_sem locked at all times (i.e. vmstat, top, ps unusable) on a UP system. I posted this complaint a while ago, it was discussed by Linus and Andrew Morton about how it also boiled down to semaphore wakeup unfairness (and bugs?). The current semaphore was determined to be too ugly to even look at. So it was dropped.

Quoted 7 lines

Is there any way that the mmap_sem could be dropped during the blocking on I/O, and reclaimed after the handle_mm_fault? Probably not, or it'd be done.

It can be a real DOS though, a 'well-written' clobbering program can make ps/vmstat useless. (it's actually /proc/pid/stat that's the killer, IIRC).

Did the `goto inside' trick in the semaphore code actually fix this unfairness issue?

-