XFS mkfs/mount options
Quoted 7 lines · Justin Piszcz
From: Justin Piszcz <[email protected]>
Subject: Re: XFS mkfs/mount options
To: "Mark" <[email protected]>
Cc: [email protected]
Date: Tuesday, June 17, 2008, 5:27 AMHow did you tune your IRQ delivery?
Generically:
echo X > /proc/irq/[IRQ#]/smp_affinity
Where X is a 32-bit hexadecimal bitmask.
My real procedure:
First, I confirmed that both drives (sda and sdb) were triggering different interrupts on the same CPU. "cat /dev/sda > /dev/null &" for activity, followed by "cat /proc/interrupts" a few times. (NOT AS ROOT!) Interrupt 20 was triggering only on the second CPU. Killed the background task.
Repeat, using "cat /dev/sdb > /dev/null &". Interrupt 21, also routing to the second CPU. Bottleneck likely confirmed.
A short hunt in /usr/src/linux/Documentation turned up the smp_affinity files. Looking at /proc/irq/2[01]/smp_affinity shows that both contain "ffffffff", that is, use all available CPU's.
To force the matter, I typed:
echo 00000001 > /proc/irq/21/smp_affinity
echo 00000002 > /proc/irq/20/smp_affinity
I dropped privileges, then repeated the "cat /dev..." above for both drives, confirming that interrupts were indeed going to CPU0 for int21 and CPU1 for int20.
Running a homebrewed multi-threading benchmark showed a possible speed-up for writes on XFS. I have not yet run "official" tests (Bonnie++ or my own) but will do so tonight. I expect the loss from cache-bouncing to be canceled out by the win from concurrent I/O.
Signature · 5 lines
Mark
"What better place to find oneself than
on the streets of one's home village?"
--Capt. Jean-Luc Picard, "Family"Hi Mark,
Am Dienstag 17 Juni 2008 schrieb Mark:
Quoted 9 lines
Quoted 7 lines · Justin Piszcz
From: Justin Piszcz <[email protected]>
Subject: Re: XFS mkfs/mount options
To: "Mark" <[email protected]>
Cc: [email protected]
Date: Tuesday, June 17, 2008, 5:27 AMHow did you tune your IRQ delivery?
Quoted 7 lines
Generically:
echo X > /proc/irq/[IRQ#]/smp_affinity
Where X is a 32-bit hexadecimal bitmask.
My real procedure:
[... longer procedure ...]
We always use
martin@shambala> apt-cache show irqbalance
Package: irqbalance
Priority: extra
Section: utils
Installed-Size: 64
Maintainer: Kyle McMartin <[email protected]>
Architecture: i386
Version: 0.55-2.1
Depends: debconf (>= 0.5) | debconf-2.0, libc6 (>= 2.7-1), libglib2.0-0 (>= 2.12.0)
Filename: pool/main/i/irqbalance/irqbalance_0.55-2.1_i386.deb
Size: 17496
MD5sum: e4393d6ac2659e08f1bdecab2d94c1bf
SHA1: 4b49deb3b522dec88c820bde78cfb6abb20365dd
SHA256: 069104aff5561f09c8affcfe2aa9e144bed359a7c935c71c9968c78d1e8bf1b3
Description: Daemon to balance interrupts for SMP systems
Daemon to balance interrupts across multiple CPUs, which can lead to better performance and IO balance on SMP systems. This package is especially useful on systems with multi-core processors, as interrupts will typically only be serviced by the first core.
Tag: admin::hardware, admin::kernel, interface::commandline, interface::daemon, network::server, role::program
for this kind of task or well the version in Debian Etch of it, the former is Lenny/Sid:
Version: 0.12-7
Depends: libc6 (>= 2.3.6-6), debconf (>= 0.5) | debconf-2.0
Is there any need to fine-tune interrupts manually?
Ciao,
Signature · 2 lines
Martin 'Helios' Steigerwald - http://www.Lichtvoll.de GPG: 03B0 0D6C 0040 0710 4AFA B82F 991B EAAC A599 84C7