'foreign' PCI cards in an rp2430
Hi all,
I'm considering buying some rp2430s (at last, PA-RISC at the
Right Price, or so it seems), and I want to run Linux on them (they're just a 'compute cluster')
I'm wondering: Would I be able to use non-HP PCI cards in these
systems, if they're supported under Linux? In particular, I'm wondering about 3ware IDE RAID cards (these work in (at least) alpha and x86 linux) and Intel gigabit ethernet NICs (based on the 82554 chip).
And (no laughter, please) what's the maximum swap partition size
parisc linux supports? Can parisc linux swap over NFS?
Sorry for the oddball questions,
Duraid
Quoted 1 line
Can parisc linux swap over NFS?
no, swapping over nfs does not work, however, using network block device should work, and is much faster than nfs.
do do that you need to enable network block device in the client's kernel, and you need two special userspace tools (debian packages nbd-server and nbd-client). the nbd-server in debian is broken you should get the latest upstream version and compile it yourself.
good luck
andi
Quoted 4 lines · Duraid Madina
I'm wondering: Would I be able to use non-HP PCI cards in these
systems, if they're supported under Linux? In particular, I'm wondering about 3ware IDE RAID cards (these work in (at least) alpha and x86 linux) and Intel gigabit ethernet NICs (based on the 82554 chip).
*puts kernel hacker's hat on*
Sure! All that stuff should work and if it doesn't it can be fixed.
*puts HP employee hat on*
Well, it might invalidate your warranty to put non-HP certified cards in your system.
Quoted 2 lines
And (no laughter, please) what's the maximum swap partition size
parisc linux supports? Can parisc linux swap over NFS?
Forget about swapping over NFS. It's prone to deadlock under stress. I think we support 2GB/file and up to 8 files, but I may be underestimating... John would know.
Signature · 1 line
Revolutions do not require corporate support.
Quoted 6 lines
Quoted 2 lines
And (no laughter, please) what's the maximum swap partition size
parisc linux supports? Can parisc linux swap over NFS?Forget about swapping over NFS. It's prone to deadlock under stress. I think we support 2GB/file and up to 8 files, but I may be underestimating... John would know.
NFS swapping probably isnt the best of ideas - if you ensure your atomic memory pool is big enough you can make it work though. Nbd will also work subject to the same kind of constraints. There are some patches around to make nfs/nbd swap reliable by putting some hacks into the network code
Quoted 2 lines · Matthew
*puts kernel hacker's hat on*
Sure! All that stuff should work and if it doesn't it can be fixed.
Sweet.
Quoted 3 lines
*puts HP employee hat on*
Well, it might invalidate your warranty to put non-HP certified cards in your system.
Grr, someone should stop HP from calling them PCI slots since they're only rated to 20W/slot anyway ;) (which is out of spec, iirc)
Quoted 7 lines
Quoted 1 line
And (no laughter, please) what's the maximum swap
partition size
Quoted 1 line
parisc linux supports? Can parisc linux swap over NFS?
Forget about swapping over NFS. It's prone to deadlock under stress. I think we support 2GB/file and up to 8 files, but I may be underestimating... John would know.
Damn! I was hoping that I could have something like 200GB of swap or so. Would it be difficult to add support for more than 8 swap files?
Thanks,
Duraid
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
"Duraid" == Duraid Madina <[email protected]> writes:
Duraid> In particular, I'm wondering about 3ware IDE RAID cards (these Duraid> work in (at least) alpha and x86 linux)
I don't think there are high hopes for the 3ware card. At least not unless you preconfigure it on a PC.
Alpha happens to work because its firmware includes a simple x86 simulator that executes the BIOS code on the card. That way you can configure your disk setup, etc.
We don't have similar functionality on PA-RISC. So you are limited what can be done from within Linux.
Unfortunately the 3ware management utility is 386 binary-only. I.e. you can't configure the card once you get Linux up and running either.
Signature · 2 lines
Martin K. Petersen Cereal Bowl Engineer, Linuxcare, Inc. http://mkp.net/ SGI XFS, Linux/PA-RISC, GNOME
Quoted 3 lines
Alpha happens to work because its firmware includes a simple x86 simulator that executes the BIOS code on the card. That way you can configure your disk setup, etc.
There is btw a passable x86 emulator for free in the XFree86 code that is used to do things like boot video cards.
Quoted 3 lines
Unfortunately the 3ware management utility is 386 binary-only. I.e. you can't configure the card once you get Linux up and running either.
Bochs 8)
Quoted 5 lines
Quoted 5 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
"Alan" == Alan Cox <[email protected]> writes:
Alpha happens to work because its firmware includes a simple x86 simulator that executes the BIOS code on the card. That way you can configure your disk setup, etc.
Alan> There is btw a passable x86 emulator for free in the XFree86 Alan> code that is used to do things like boot video cards.
Yep. It's going to take a fairly big hammer to beat libint10 into submission, though. Interactively mocking with disk configuration and whatnot is a bit more complex than kicking off some init fluff on a video card.
Quoted 3 lines
Quoted 3 lines
Unfortunately the 3ware management utility is 386 binary-only. I.e. you can't configure the card once you get Linux up and running either.
Alan> Bochs 8)
It would actually be useful to hack it so we had a generic way of running x86 card ROMs on non-{PC,Alpha,Itanium} architectures.
Signature · 2 lines
Martin K. Petersen Cereal Bowl Engineer, Linuxcare, Inc. http://mkp.net/ SGI XFS, Linux/PA-RISC, GNOME
Quoted 4 lines
Quoted 3 lines
Alpha happens to work because its firmware includes a simple x86 simulator that executes the BIOS code on the card. That way you can configure your disk setup, etc.
Quoted 9 lines
There is btw a passable x86 emulator for free in the XFree86 code that is used to do things like boot video cards.
Quoted 3 lines
Unfortunately the 3ware management utility is 386 binary-only. I.e. you can't configure the card once you get Linux up and running either.
Bochs 8)
out of sheer curiosity, i installed bochs on my 712/60
running the linux demo that comes with it, i got a bogoMIPS rating of 0.82
hehe :)
James S
Quoted 2 lines · Martin
I don't think there are high hopes for the 3ware card. At least not unless you preconfigure it on a PC.
For sure.
Quoted 3 lines
Alpha happens to work because its firmware includes a simple x86 simulator that executes the BIOS code on the card. That way you can configure your disk setup, etc.
Oops, I think I was misleading when I said 3ware 'worked' in Alpha - 3dm doesn't (of course) and yes, you need to do all array administration stuff out of the BIOS on a PC.
Quoted 2 lines
We don't have similar functionality on PA-RISC. So you are limited what can be done from within Linux.
Well, so long as it can find the array and read/write, I'm set.
Quoted 3 lines
Unfortunately the 3ware management utility is 386 binary-only. I.e. you can't configure the card once you get Linux up and running either.
I have to admit, I find hardly any need for that tool even in x86 land ;)
Duraid
Erm. Whare are you going to put the disks for the 3ware?
Nick
Quoted 26 lines · Duraid Madina
Hi all,
I'm considering buying some rp2430s (at last, PA-RISC at the
Right Price, or so it seems), and I want to run Linux on them (they're just a 'compute cluster')I'm wondering: Would I be able to use non-HP PCI cards in these
systems, if they're supported under Linux? In particular, I'm wondering about 3ware IDE RAID cards (these work in (at least) alpha and x86 linux) and Intel gigabit ethernet NICs (based on the 82554 chip).And (no laughter, please) what's the maximum swap partition size
parisc linux supports? Can parisc linux swap over NFS?Sorry for the oddball questions,
Duraid
_______________________________________________
parisc-linux mailing list
[email protected]
http://lists.parisc-linux.org/mailman/listinfo/parisc-linux
Quoted 1 line
Erm. Whare are you going to put the disks for the 3ware?
On top of the box,
unshielded disks gather dust.
I hear them rattle.
Duraid
Quoted 1 line · Duraid Madina
...and Intel gigabit ethernet NICs (based on the 82554 chip).
I've tested the e1000 (4.0.24-k1, iirc) driver on HP's "2cd generation Intel (r) Itanium (tm)" systems and it basically works. Based on this, I would expect this e1000 driver to work on A500-like systems. (ie rp24xx boxes). Someday, if i get the itch, I'll plug one into my a500 and try it.
and later:
Quoted 3 lines
A computational physics code. Ideally I would run it in 100s of GB of RAM. About two months of pain went into making it work (reasonably efficiently, in theory at least) in 100s of GB of swap.
The approach seems really odd - you are looking at 64-bit userspace so you can *swap* large amounts of virtual memory in/out. If you expected to be IO bound, tuning the app to know more about IO (instead of swapping) makes more sense.
Have you tried begging, trading, or buying some cycles on a Superdome with some 100GBs of RAM instead of waiting for weeks to see your jobs run to completion?
my $0.02,
grant
Quoted 8 lines · Grant
and later:
Quoted 2 lines
About two months of pain went into making it work (reasonably efficiently, in theory at least) in 100s of GB of swap.
The approach seems really odd - you are looking at 64-bit userspace so you can *swap* large amounts of virtual memory in/out. If you expected to be IO bound, tuning the app to know more about IO (instead of swapping) makes more sense.
Normally, yes. In my case, not really. To cut a long story short, because C++ has poor intrinsic support for reflection, even making my app *do* IO is non-trivial. E.g. my program instantiates millions of different objects (which interact in various interesting ways). The cost of serializing/deserializing these objects to store them in a light database like berkeleyDB, or simply as files is great enough that the job becomes CPU bound, and not I/O bound. Being CPU bound is not where I want to be, when I'm seeing only 2mb/sec of I/O ;)
What's even worse is that I'm using some truly unwiedly third party class libraries (Hi Richard! ;) which don't provide serialization, so again, instantiating 'plain old objects' seems to be the path of least resistance, in my case.
What I was hoping to do is take an athlon box with fast raid and gigabit ethernet, and buy one of those nice new rp2430s, having it swap over ethernet. It seems that I can do this with HP-UX, though perhaps not with G++. I would prefer to run Linux on the rp2430 though, since then I could use a cheap Intel gigabit NIC (instead of paying $$$ for an HP one) or better yet, stick my 3ware RAID card in the rp2430, and do away with the network altogether. That I *can't* do with HP-UX, so...
Quoted 3 lines
Have you tried begging, trading, or buying some cycles on a Superdome with some 100GBs of RAM instead of waiting for weeks to see your jobs run to completion?
Really, my code should be pretty efficient (it takes weeks to run anyway ;) with the setup I've described above, but the killer is: I'm a University student ("I knew it!" I hear you all say ;) and as such, my code is perpetually in the "hmm, I wonder what'd happen if.." state. So "some cycles" would become "some more cycles" faster than you could say "where's all my RAM gone?!"
Duraid
P.S. Am I really the only sicko who's wanted a 64-bit machine just so they can abuse virtual memory? Surely not... ;)
P.P.S. It's true, I do love the smell, the feel, the sound of striped, thrashing, consumer-level hard drives. C'mon guys, help this sicko make his dream come true (:
Quoted 5 lines
Quoted 4 lines
Have you tried begging, trading, or buying some cycles on a Superdome with some 100GBs of RAM instead of waiting for weeks to see your jobs run to completion?
Quoted 9 lines
Really, my code should be pretty efficient (it takes weeks to run anyway ;) with the setup I've described above, but the killer is: I'm a University student ("I knew it!" I hear you all say ;) and as such, my code is perpetually in the "hmm, I wonder what'd happen if.." state. So "some cycles" would become "some more cycles" faster than you could say "where's all my RAM gone?!"
Duraid
This is a very similar problem that we had when calculating prime alternating knots[1]. Essentially what was done was to use MPI[2] and a fast network connection, such that you could have _other_ computers holding data in RAM/SWAP.
Parallelize parallelize parallelize.
It's really easy to learn MPI and it's great for those big problems that you may not be able to do with a 64-bit box :)
a. Get a 100MBit/1GBit switch and NIC's.
b. Get some more boxes (borrow/lie/cheat/steal)
c. Tie them together.
This is what I do for a hobby :) ... and yes I'm a student aswell. It's nice to see that people like Bell and Nortel are willing to donate expensive gear for projects like yours or mine[3] :)
c.
[1] http://baldric.uwo.ca/article.php3?section=baldric&article=knots
[2] http://www-unix.mcs.anl.gov/mpi/mpich/
[2] http://www.lam-mpi.org/
[3] http://www.baldric.uwo.ca/~carlos/pdisplay.html
Quoted 8 lines · Duraid Madina
Normally, yes. In my case, not really. To cut a long story short, because C++ has poor intrinsic support for reflection, even making my app *do* IO is non-trivial. E.g. my program instantiates millions of different objects (which interact in various interesting ways). The cost of serializing/deserializing these objects to store them in a light database like berkeleyDB, or simply as files is great enough that the job becomes CPU bound, and not I/O bound. Being CPU bound is not where I want to be, when I'm seeing only 2mb/sec of I/O ;)
FWIW, you should expect to see 30-40MB/s to single disk on rp2430. More with more disks/channels.
Quoted 2 lines
I would prefer to run Linux on the rp2430 though, since then I could use a cheap Intel gigabit NIC (instead of paying $$$ for an HP
Well, 2.4 has no e1000 driver (yet) much less one that is known to work on hppa. I *think* the e1000 driver I "leveraged" from 2.5 release for HP ia64 machine should also work for hppa. But that's just an educated guess. FWIW, bcm5700 driver needed some patches to work on HP's ia64 box and I'm looking forward to testing tg3 patch soon.
And Intel's change in strategy may not work so well in pratice. So I'd stick with a tigon (acenic or bcm5700) board for the next 3 monthes.
Quoted 2 lines
one) or better yet, stick my 3ware RAID card in the rp2430, and do away with the network altogether. That I *can't* do with HP-UX, so...
I've never liked RAID controllers. too complex and failure prone. If you have spare CPU cycles, putting in simple SCSI controllers will give you at least 4 x 80MB/s and it's easy to connect 40 or so 9 (or 18GB) SCSI drives. This is not shiny, new high tech stuff (eg u160 and 15KRPM). but I expect it to be faster than any IDE solution (I could be wrong) and re-usable for other computer science projects (read database or other mutlithreaded IO) when you are done. <RANT>And you don't need any stinking x86 binaries-only software to manage the setup.</RANT>
But I'm not one to stop others from picking solutions that work for them.
grant