Skip to contentSkip to conversation

UCE and NUMA

DISCUSSION Mario Smarduch 2 participants 3 messages Mar 16, 2004 – Mar 17, 2004

lore
Share & tools
lore
#0 lore raw

Hi,

we're currently evaluating a NUMA system (8-way). We're wondering for hot contention spots does the 2.4 or 2.6 kernel
make use of IA64 UCE Memory attribute (fetchadd)?
Has anyone been able to catpure relative performance differences (WB vs. UCE) for any benchmark ?

- Mario

#1 lore raw
Quoted 13 lines · David Mosberger
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line · Mario Smarduch

Mario> Hi, we're currently evaluating a NUMA system (8-way). We're Mario> wondering for hot contention spots does the 2.4 or 2.6 kernel Mario> make use of IA64 UCE Memory attribute (fetchadd)? Has anyone Mario> been able to catpure relative performance differences (WB Mario> vs. UCE) for any benchmark ?

Note that UCE is both CPU model- and platform-specific. Even then, only "fetchadd" is supported (not cmpxchg, which is what's normally used for spinlocks). Furthermore, you have to be really careful to avoid attribute-aliasing.

Quoted 7 lines

As long as those constraints are met, I suppose it would be OK to use UCE.

However, is UCE really beneficial for spinlocks? I thought it's primary use would be for atomic operations (such as atomic inc).

        --david

I suppose you could use fetchadd
to build some crude lockout mechanism, but probably more useful is shared kernel data that regularly may get updated by all CPUs (i.e. dec/inc atomically ).
That may save on WB cache-coherency waste of bandwidth for
contentious spots.

But I agree practically it would be pretty difficult to (i) first figure out what is very contentious (ii) and locate all your locks/data into those UCE pages. So in short UCE is not used in any kernel version?

- Mario

#2 lore raw
In reply to Mario Smarduch
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line · Mario Smarduch

Mario> But I agree practically it would be pretty difficult to (i) Mario> first figure out what is very contentious (ii) and locate all Mario> your locks/data into those UCE pages. So in short UCE is not Mario> used in any kernel version?

Not as far as I know. I really don't even know which platforms support fetchadd via UCE.

	--david