Skip to contentSkip to conversation

timestamp fixes

PATCH Andrew Theurer 2 participants 3 messages Feb 28, 2005 – Mar 1, 2005 1 message not in the archive

lore
Share & tools
lore

A message that is not in the archive (sent off-list or not yet received): [email protected]

#1 lore raw

Nick, can you describe the system you run the DB tests on? Do you have any cpu idle time stats and hopefully some context switch rate stats?

I think I understand the concern [patch 6] of stealing a task from one node to an idle cpu in another node, but I wonder if we can have some sort of check for idle balance: if the domain/node to steal from has some idle cpu somewhere, we do not steal, period. To do this we have a cpu_idle bitmask, we update as cpus go idle/busy, and we reference this cpu_idle & sd->cpu_mask to see if there's at least one cpu that's idle.

Quoted 6 lines

Ingo wrote:

But i expect fork/clone balancing to be almost certainly a problem. (We didnt get it right for all workloads in 2.6.7, and i think it cannot be gotten right currently either, without userspace API help - but i'd be happy to be proven wrong.)

Perhaps initially one could balance on fork up to the domain level which has task_hot_time=0, up to a shared cache by default. Anything above that could require a numactl like preference from userspace.

-Andrew Theurer

#2 lore raw
Quoted 1 line · Andrew Theurer

Nick, can you describe the system you run the DB tests on? Do you have any cpu idle time stats and hopefully some context switch rate stats?

Yeah, it is dbt3-pgsql on OSDL's 8-way STP machines. I think they're PIII Xeons with 2MB L2 cache.

I had been having some difficulty running them recently, but I might have just worked out what the problem was, so hopefully I can benchmark the patchset I just sent to Andrew (plus fixes from Ingo, etc).

Basically what would happen is that with seemingly small changes, the "throughput" figure would go from about 260 tps to under 100. I don't know exactly why this benchmark is particularly sensitive to the problem, but it has been a good canary for too-much-idle-time regressions in the past.

Quoted 2 lines

I think I understand the concern [patch 6] of stealing a task from one node to an idle cpu in another node, but I wonder if we can have some sort of check for idle balance: if the domain/node to steal from has some idle cpu somewhere, we do not steal, period. To do this we have a cpu_idle bitmask, we update as cpus go idle/busy, and we reference this cpu_idle & sd->cpu_mask to see if there's at least one cpu that's idle.

I think this could become a scalability problem... and I'd prefer to initially try to address problems of too much idle time by tuning them out rather than use things like wake-to-idle-cpu.

One of my main problems with wake to idle is that it is explicitly introducing a bias so that wakers repel their wakees. With all else being equal, we want exactly the opposite effect (hence all the affine wakeups and wake balancing stuff).

The regular periodic balancing may be dumb, but at least it doesn't have that kind of bias.

The other thing of course is that we want to reduce internode task movement at almost all costs, and the periodic balancer can be very careful about this - something more difficult for wake_idle unless we introduce more complexity there (a very time critical path).

Anyway, I'm not saying no to anything at this stage... let's just see how we go.

Quoted 10 lines
Quoted 6 lines

Ingo wrote:

But i expect fork/clone balancing to be almost certainly a problem. (We didnt get it right for all workloads in 2.6.7, and i think it cannot be gotten right currently either, without userspace API help - but i'd be happy to be proven wrong.)

Perhaps initially one could balance on fork up to the domain level which has task_hot_time=0, up to a shared cache by default. Anything above that could require a numactl like preference from userspace.

That may end up being what we have to do. Probably doesn't make much sense to do it at all if we aren't doing it for NUMA.

Nick

#3 lore raw
Quoted 5 lines · Nick Piggin
Quoted 1 line · Andrew Theurer

Nick, can you describe the system you run the DB tests on? Do you have any cpu idle time stats and hopefully some context switch rate stats?

Quoted 7 lines

Yeah, it is dbt3-pgsql on OSDL's 8-way STP machines. I think they're PIII Xeons with 2MB L2 cache.

I had been having some difficulty running them recently, but I might have just worked out what the problem was, so hopefully I can benchmark the patchset I just sent to Andrew (plus fixes from Ingo, etc).

Nope. Still not working :P