Kernel Scalability to 48 cores
A friend just sent me the following PDF about Linux Scalability to many cores and I thought it might be interesting to the list for further discussion. It's about scalability problems of the Linux kernel and applications with 48 and more cores.
Quoted 4 lines · On Lunes, 11 de Octubre de 2010 23:30:03 Cyrus Massoumi escribió:
A friend just sent me the following PDF about Linux Scalability to many cores and I thought it might be interesting to the list for further discussion. It's about scalability problems of the Linux kernel and applications with 48 and more cores.
Some of the contention points mentioned in Figure 1 seem to be the same things that Nick Piggin is trying to solve with his VFS scalability patchset.
Quoted 3 lines · Cyrus Massoumi
A friend just sent me the following PDF about Linux Scalability to many cores and I thought it might be interesting to the list for further discussion. It's about scalability problems of the Linux kernel and applications
with 48 and more cores.
Note that 48 cores is a small system these days.
Quoted 1 line
The paper is known, but it seems to be in the usual unproductive "publish a paper, but don't care about fixing anything" acedemic mode.
A lot of the issues described there are already being addressed I believe.
-Andi
Signature · 1 line
[email protected] -- Speaking for myself only.
Quoted 2 lines · Andi Kleen
The paper is known, but it seems to be in the usual unproductive "publish a paper, but don't care about fixing anything" acedemic mode.
The thing I found most disappointing about the paper is that it describe techniques that have been in use in industry to provide better scalability to commerical OS's, and which were first applied to Linux something like a decade ago.
Sure, at the time we were going to 4-16 way scalability, but the techniques used were known back then, and in fact the paper is actually pretty naive compared to what people at companies like Sequent had been doing for a very long time.
It's just that a lot of scalability work took a bit of a hiatus, oh, five years ago because at the time, Linux was good enough, and adding more scalability has costs (which is why not all of the changes SGI made for 512 and 1024-way scalability have not necessarily found their way into mainline). The fact that we've needed to worry about scalability now that 6+ cores/socket are available now, and many more are coming soon, is something we've known for at least the last year or two.
But since a lot of that work wasn't published in the appropriate hoidy-toidy "tenure-track" approved publication venues, to the academics, it didn't exist.
Oh, well....
-- Ted
Quoted 7 lines · Theodore Tso
Quoted 2 lines · Andi Kleen
The paper is known, but it seems to be in the usual unproductive "publish a paper, but don't care about fixing anything" acedemic mode.
The thing I found most disappointing about the paper is that it describe techniques that have been in use in industry to provide better scalability to commerical OS's, and which were first applied to Linux something like a decade ago.
I haven't gone fully through it yet, maybe there are still some useful nuggets in there. I would help if they published their patches though.
They used a few new workloads which were useful I guess.
Quoted 1 line
It's just that a lot of scalability work took a bit of a hiatus, oh, five years ago because at the time, Linux was good enough, and adding more scalability has costs (which is why not all of the changes SGI made for 512 and 1024-way scalability have not necessarily found their way into mainline). The fact that we've needed to worry about scalability now that 6+ cores/socket are available now, and many more are coming soon, is something we've known for at least the last year or two.
Already at 8*2 and 12 cores/socket
Fully agreed, but the other aspect is that we also need to worry now about TBs and more of memory on a single (non extreme) system
More cores come with more memory and now even with 4K pages.
As a simple exercise I configured a 1TB system for less than 50kEUR at Dell.com some time ago: that's not cheap, but definitely not an exotic super computer either and prices will go down.
I believe a lot of new challenges also come from that, although there has been a lot less work on that so far than on core scalability.
-Andi
Signature · 1 line
[email protected] -- Speaking for myself only.