UCE and NUMA
Hi,
we're currently evaluating a NUMA system (8-way). We're wondering for hot contention spots does the 2.4 or 2.6 kernel
make use of IA64 UCE Memory attribute (fetchadd)?
Has anyone been able to catpure relative performance differences (WB vs. UCE) for any benchmark ?
- Mario
Quoted 13 lines · David Mosberger
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line · Mario Smarduch
Mario> Hi, we're currently evaluating a NUMA system (8-way). We're Mario> wondering for hot contention spots does the 2.4 or 2.6 kernel Mario> make use of IA64 UCE Memory attribute (fetchadd)? Has anyone Mario> been able to catpure relative performance differences (WB Mario> vs. UCE) for any benchmark ?
Note that UCE is both CPU model- and platform-specific. Even then, only "fetchadd" is supported (not cmpxchg, which is what's normally used for spinlocks). Furthermore, you have to be really careful to avoid attribute-aliasing.
Quoted 7 lines
As long as those constraints are met, I suppose it would be OK to use UCE.
However, is UCE really beneficial for spinlocks? I thought it's primary use would be for atomic operations (such as atomic inc).
--david
I suppose you could use fetchadd
to build some crude lockout mechanism, but probably more useful is shared kernel data that regularly may get updated by all CPUs (i.e. dec/inc atomically ).
That may save on WB cache-coherency waste of bandwidth for
contentious spots.
But I agree practically it would be pretty difficult to (i) first figure out what is very contentious (ii) and locate all your locks/data into those UCE pages. So in short UCE is not used in any kernel version?
- Mario
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line · Mario Smarduch
Mario> But I agree practically it would be pretty difficult to (i) Mario> first figure out what is very contentious (ii) and locate all Mario> your locks/data into those UCE pages. So in short UCE is not Mario> used in any kernel version?
Not as far as I know. I really don't even know which platforms support fetchadd via UCE.
--david