Skip to contentSkip to conversation

2.4.21 opteron 32 Gig RAM has 10k block writes/sec on running posgresql 7.4.6 disk i/o intensive app

DISCUSSION Paramveer Singh 2 participants 3 messages Dec 23, 2004

lore
Share & tools
lore
#0 lore raw

we are using a RedHat AS3 U3 box (2.4.21-4.ELsmp) to run a very intensive database app which does a _huge_ number of inserts durnig the data generation phase. However, we are noticing a unexpected performance drops with user cpu utilization numbers falling to near 0.

most of the cpu is used up in iowait.
CPU states: cpu user nice system irq softirq iowait idle

          total    2.0%    0.0%    7.6%   0.0%     0.0%  362.4%   27.2%

/proc/slabinfo showed huge numbers in buffer_head. The line was:
buffer_head 4908452 4962249 200 260854 261171 1 : 16000 4000

some lines from vmstat 1
procs memory swap io system cpu
r b swpd free buff cache si so bi bo in cs us sy wa id
4 2 10104 8585908 109820 19090508 0 0 37 1160 282 1421
8 4 67 22
1 4 10104 8585908 109824 19090684 0 0 0 4940 1359 4762
4 1 42 52

 0  3  10104 8585908 109824 19091084    0    0     0  5296 1442 17178
13  4 40 43
 8  1  10104 8585908 109824 19092068    0    0     0  2856 1437 34481
33  6 30 32
 1  3  10104 8585908 109832 19093620    0    0     0  9012 1538 28554
35  7 34 25
 0  3  10104 8585908 109832 19094004    0    0     0  6120 1552  3059 

6 2 47 45
0 3 10104 8585908 109836 19094012 0 0 0 6188 1502 332
0 0 70 30
0 3 10104 8585908 109836 19094020 0 0 0 6436 1548 865
0 1 67 32

 2  3  10104 8585908 109836 19094332    0    0     0  5980 1404  9322
10  3 52 36
 0  3  10104 8585908 109840 19094332    0    0     0  5252 1492   190 

0 0 74 25

Interestingly, the swap is not being used at all
free output:
Swap: 2040212 10104 2030108

the box is
Quad AMD Opteron 1.8Ghz
Tyan Mother board S4882UG2NR
with Rack Chassis, 12 X 1 Hot swap bays, power supply + RPS Support 32Gb DDr sdram @ 266
8 X 80Gb SATA Hot Swap Hard Drive, 7200 RPM
Adaptec SATA RAID controller

I tried to google but found some kswapd related bugs in AS3 which were fixed in U3. We upgraded even though this does not seem to be a kswapd error. I am gussing I have used some inappropriate numbers while setting up the box.

Would someone be kind to point me in a direction where I can find more information?

thanks a lot!
ps

#1 lore raw
Quoted 9 lines · Paramveer Singh

we are using a RedHat AS3 U3 box (2.4.21-4.ELsmp) to run a very intensive database app which does a _huge_ number of inserts durnig the data generation phase. However, we are noticing a unexpected performance drops with user cpu utilization numbers falling to near 0. most of the cpu is used up in iowait.
CPU states: cpu user nice system irq softirq iowait idle

          total    2.0%    0.0%    7.6%   0.0%     0.0%  362.4%   27.2%

/proc/slabinfo showed huge numbers in buffer_head. The line was:
buffer_head 4908452 4962249 200 260854 261171 1 : 16000 4000

Please try to reproduce on a more recent RHEL3, e.g. RHEL3-U4 released today.

-- wli

#2 lore raw

thanks Wli!
the update seems to have worked and the same situation has not arisen again.

regards,
ps

Quoted 17 lines · William Lee Irwin III
Quoted 9 lines · Paramveer Singh

we are using a RedHat AS3 U3 box (2.4.21-4.ELsmp) to run a very intensive database app which does a _huge_ number of inserts durnig the data generation phase. However, we are noticing a unexpected performance drops with user cpu utilization numbers falling to near 0. most of the cpu is used up in iowait.
CPU states: cpu user nice system irq softirq iowait idle

          total    2.0%    0.0%    7.6%   0.0%     0.0%  362.4%   27.2%

/proc/slabinfo showed huge numbers in buffer_head. The line was:
buffer_head 4908452 4962249 200 260854 261171 1 : 16000 4000

Please try to reproduce on a more recent RHEL3, e.g. RHEL3-U4 released today.

-- wli