Skip to contentSkip to conversation

reiserfs do_journal_end unnecessary hd wake up?

DISCUSSION Mat 3 participants 8 messages Sep 4, 2008 – Sep 5, 2008

lore
Share & tools
lore

Mat

author
#0 lore raw

ok, then try the following:

you at least should have noatime,nodiratime,commit=600

enabled for the reiserfs or >=ext3 filesystems

you can also try data=writeback (but beware this might put your data at risk !)

select the anticipatory i/o scheduler
and set following stuff

echo "16" > /proc/sys/vm/page-cluster
# default: 3
#

echo "60" > /proc/sys/vm/swappiness
# default: 60
# By default, Linux will aggressively swap processes out of physical memory onto disk in order to keep the disk cache as large as possible.
# This means that pages that haven't been used recently will be pushed into swap long before the system even comes close to running out of memory, which is an unexpected behavior compared to some operating systems.
# The /proc/sys/vm/swappiness parameter controls how aggressive Linux is in this area.

echo "3000" > /proc/sys/vm/dirty_expire_centisecs
# default: 3000 (30 seconds)
#2 how long data can be in the page cache before it is considered expired and must be written at the next opportunity. Note that this default is very long: a full 30 seconds. That means that under normal circumstances, unless you write enough to trigger the other pdflush method, Linux won't actually commit anything you write until 30 seconds later.

echo "6000" > /proc/sys/vm/dirty_writeback_centisecs
# default: 500 (5 seconds)
#1 how often pdflush wakes up to write data to disk. The default wakes up the two (or more) active threads every five seconds.
# suggestion: 6000 (every 60 seconds)

echo "15" > /proc/sys/vm/dirty_background_ratio
# default: 10
#3 Maximum percentage of active memory that can be filled with dirty pages before pdflush begins to write them

echo "50" > /proc/sys/vm/dirty_ratio #modified
# default: 40
#4 Maximum percentage of total memory that can be filled with dirty pages before processes are forced to write dirty buffers themselves during their time slice instead of being allowed to do more writes.
# modified: 50

echo "25" > /proc/sys/vm/vfs_cache_pressure

for i in /sys/block/sd*; do
/bin/echo "anticipatory" > $i/queue/scheduler
done

for i in /sys/block/sd*; do
/bin/echo "0" > $i/queue/iosched/antic_expire
done

for i in /sys/block/sd*; do

         /bin/echo "150" >  $i/queue/iosched/read_expire

done

for i in /sys/block/sd*; do

         /bin/echo "750" >  $i/queue/iosched/read_batch_expire

done

for i in /sys/block/sd*; do

         /bin/echo "1200" >  $i/queue/iosched/write_batch_expire

done

for i in /sys/block/sd*; do

         /bin/echo "1024" >  $i/queue/nr_requests

done

for i in /sys/block/sd*; do

         /bin/echo "256" >  $i/queue/read_ahead_kb

done

for i in /sys/block/sd*; do

         /bin/echo "256" >  $i/queue/max_sectors_kb

done

for i in /sys/class/scsi_host/host*; do
/bin/echo "min_power" > $i/link_power_management_policy
done

try to disable each and every unneeded daemon or applets, programs, etc.

by using:

* powertop
* iotop
* htop
* top
* lsof | grep /home
* ...

#1 lore raw

thank you very much for the great instructions. i'll evaluate then and send you the results.
bráulio

Quoted 111 lines · Mat

ok, then try the following:

you at least should have noatime,nodiratime,commit=600

enabled for the reiserfs or >=ext3 filesystems

you can also try data=writeback (but beware this might put your data at risk !)

select the anticipatory i/o scheduler
and set following stuff

echo "16" > /proc/sys/vm/page-cluster
# default: 3
#

echo "60" > /proc/sys/vm/swappiness
# default: 60
# By default, Linux will aggressively swap processes out of physical memory onto disk in order to keep the disk cache as large as possible.
# This means that pages that haven't been used recently will be pushed into swap long before the system even comes close to running out of memory, which is an unexpected behavior compared to some operating systems.
# The /proc/sys/vm/swappiness parameter controls how aggressive Linux is in this area.

echo "3000" > /proc/sys/vm/dirty_expire_centisecs
# default: 3000 (30 seconds)
#2 how long data can be in the page cache before it is considered expired and must be written at the next opportunity. Note that this default is very long: a full 30 seconds. That means that under normal circumstances, unless you write enough to trigger the other pdflush method, Linux won't actually commit anything you write until 30 seconds later.

echo "6000" > /proc/sys/vm/dirty_writeback_centisecs
# default: 500 (5 seconds)
#1 how often pdflush wakes up to write data to disk. The default wakes up the two (or more) active threads every five seconds.
# suggestion: 6000 (every 60 seconds)

echo "15" > /proc/sys/vm/dirty_background_ratio
# default: 10
#3 Maximum percentage of active memory that can be filled with dirty pages before pdflush begins to write them

echo "50" > /proc/sys/vm/dirty_ratio #modified
# default: 40
#4 Maximum percentage of total memory that can be filled with dirty pages before processes are forced to write dirty buffers themselves during their time slice instead of being allowed to do more writes.
# modified: 50

echo "25" > /proc/sys/vm/vfs_cache_pressure

for i in /sys/block/sd*; do
/bin/echo "anticipatory" > $i/queue/scheduler
done

for i in /sys/block/sd*; do
/bin/echo "0" > $i/queue/iosched/antic_expire
done

for i in /sys/block/sd*; do

        /bin/echo "150" >  $i/queue/iosched/read_expire

done

for i in /sys/block/sd*; do

        /bin/echo "750" >  $i/queue/iosched/read_batch_expire

done

for i in /sys/block/sd*; do

        /bin/echo "1200" >  $i/queue/iosched/write_batch_expire

done

for i in /sys/block/sd*; do

        /bin/echo "1024" >  $i/queue/nr_requests

done

for i in /sys/block/sd*; do

        /bin/echo "256" >  $i/queue/read_ahead_kb

done

for i in /sys/block/sd*; do

        /bin/echo "256" >  $i/queue/max_sectors_kb

done

for i in /sys/class/scsi_host/host*; do
/bin/echo "min_power" > $i/link_power_management_policy
done

try to disable each and every unneeded daemon or applets, programs, etc.

by using:

* powertop
* iotop
* htop
* top
* lsof | grep /home
* ...

--
To unsubscribe from this list: send the line "unsubscribe reiserfs-devel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

#2 lore raw

hello mat, here comes the results of each action.
in all the process i've used the iotop (what a great utility you pointed!), and always saw only the pdflush process acessing the hd. no user interface were used.
the apm is set to 128 with hdparm

ok, then try the following:

you at least should have noatime,nodiratime,commit=600

:) that did the job! also, i've tested without the commit option, which didn't changed the result.

enabled for the reiserfs or >=ext3 filesystems

you can also try data=writeback (but beware this might put your data at risk !)

* can't try

select the anticipatory i/o scheduler
and set following stuff

:( no change.

echo "16" > /proc/sys/vm/page-cluster
# default: 3
#

:( no change.

echo "60" > /proc/sys/vm/swappiness
# default: 60
# By default, Linux will aggressively swap processes out of physical memory onto disk in order to keep the disk cache as large as possible.
# This means that pages that haven't been used recently will be pushed into swap long before the system even comes close to running out of memory, which is an unexpected behavior compared to some operating systems.
# The /proc/sys/vm/swappiness parameter controls how aggressive Linux is in this area.

:( no change.

echo "3000" > /proc/sys/vm/dirty_expire_centisecs
# default: 3000 (30 seconds)
#2 how long data can be in the page cache before it is considered expired and must be written at the next opportunity. Note that this default is very long: a full 30 seconds. That means that under normal circumstances, unless you write enough to trigger the other pdflush method, Linux won't actually commit anything you write until 30 seconds later.

:( no change.

echo "6000" > /proc/sys/vm/dirty_writeback_centisecs
# default: 500 (5 seconds)
#1 how often pdflush wakes up to write data to disk. The default wakes up the two (or more) active threads every five seconds.
# suggestion: 6000 (every 60 seconds)

:( no change.

echo "15" > /proc/sys/vm/dirty_background_ratio
# default: 10
#3 Maximum percentage of active memory that can be filled with dirty pages before pdflush begins to write them

echo "50" > /proc/sys/vm/dirty_ratio #modified
# default: 40
#4 Maximum percentage of total memory that can be filled with dirty pages before processes are forced to write dirty buffers themselves during their time slice instead of being allowed to do more writes.
# modified: 50

echo "25" > /proc/sys/vm/vfs_cache_pressure

for i in /sys/block/sd*; do
/bin/echo "anticipatory" > $i/queue/scheduler
done

for i in /sys/block/sd*; do
/bin/echo "0" > $i/queue/iosched/antic_expire
done

for i in /sys/block/sd*; do

        /bin/echo "150" >  $i/queue/iosched/read_expire

done

for i in /sys/block/sd*; do

        /bin/echo "750" >  $i/queue/iosched/read_batch_expire

done

for i in /sys/block/sd*; do

        /bin/echo "1200" >  $i/queue/iosched/write_batch_expire

done

for i in /sys/block/sd*; do

        /bin/echo "1024" >  $i/queue/nr_requests

done

for i in /sys/block/sd*; do

        /bin/echo "256" >  $i/queue/read_ahead_kb

done

for i in /sys/block/sd*; do

        /bin/echo "256" >  $i/queue/max_sectors_kb

done

for i in /sys/class/scsi_host/host*; do
/bin/echo "min_power" > $i/link_power_management_policy
done

try to disable each and every unneeded daemon or applets, programs, etc.

by using:

* powertop
* iotop
* htop
* top
* lsof | grep /home
* ...

#3 lore raw

but (as always there is), the situation is not good yet when using kde4. with it, i can't find out which process is accessing the hd, cause iotop only shows a pdflush access.
i've tried to kill some apps, but it didn't help. any other trace technic?

regards,
bráulio

Quoted 124 lines · Bráulio Barros de Oliveira

hello mat, here comes the results of each action.
in all the process i've used the iotop (what a great utility you pointed!), and always saw only the pdflush process acessing the hd. no user interface were used.
the apm is set to 128 with hdparm

ok, then try the following:

you at least should have noatime,nodiratime,commit=600

:) that did the job! also, i've tested without the commit option, which didn't changed the result.

enabled for the reiserfs or >=ext3 filesystems

you can also try data=writeback (but beware this might put your data at risk !)

* can't try

select the anticipatory i/o scheduler
and set following stuff

:( no change.

echo "16" > /proc/sys/vm/page-cluster
# default: 3
#

:( no change.

echo "60" > /proc/sys/vm/swappiness
# default: 60
# By default, Linux will aggressively swap processes out of physical memory onto disk in order to keep the disk cache as large as possible.
# This means that pages that haven't been used recently will be pushed into swap long before the system even comes close to running out of memory, which is an unexpected behavior compared to some operating systems.
# The /proc/sys/vm/swappiness parameter controls how aggressive Linux is in this area.

:( no change.

echo "3000" > /proc/sys/vm/dirty_expire_centisecs
# default: 3000 (30 seconds)
#2 how long data can be in the page cache before it is considered expired and must be written at the next opportunity. Note that this default is very long: a full 30 seconds. That means that under normal circumstances, unless you write enough to trigger the other pdflush method, Linux won't actually commit anything you write until 30 seconds later.

:( no change.

echo "6000" > /proc/sys/vm/dirty_writeback_centisecs
# default: 500 (5 seconds)
#1 how often pdflush wakes up to write data to disk. The default wakes up the two (or more) active threads every five seconds.
# suggestion: 6000 (every 60 seconds)

:( no change.

echo "15" > /proc/sys/vm/dirty_background_ratio
# default: 10
#3 Maximum percentage of active memory that can be filled with dirty pages before pdflush begins to write them

echo "50" > /proc/sys/vm/dirty_ratio #modified
# default: 40
#4 Maximum percentage of total memory that can be filled with dirty pages before processes are forced to write dirty buffers themselves during their time slice instead of being allowed to do more writes.
# modified: 50

echo "25" > /proc/sys/vm/vfs_cache_pressure

for i in /sys/block/sd*; do
/bin/echo "anticipatory" > $i/queue/scheduler
done

for i in /sys/block/sd*; do
/bin/echo "0" > $i/queue/iosched/antic_expire
done

for i in /sys/block/sd*; do

       /bin/echo "150" >  $i/queue/iosched/read_expire

done

for i in /sys/block/sd*; do

       /bin/echo "750" >  $i/queue/iosched/read_batch_expire

done

for i in /sys/block/sd*; do

       /bin/echo "1200" >  $i/queue/iosched/write_batch_expire

done

for i in /sys/block/sd*; do

       /bin/echo "1024" >  $i/queue/nr_requests

done

for i in /sys/block/sd*; do

       /bin/echo "256" >  $i/queue/read_ahead_kb

done

for i in /sys/block/sd*; do

       /bin/echo "256" >  $i/queue/max_sectors_kb

done

for i in /sys/class/scsi_host/host*; do
/bin/echo "min_power" > $i/link_power_management_policy
done

try to disable each and every unneeded daemon or applets, programs, etc.

by using:

* powertop
* iotop
* htop
* top
* lsof | grep /home
* ...

#4 lore raw

just found out! it is konsole, unbelievable! going to file a bug report right now.
again, thank you very much for the support.
bráulio

Quoted 135 lines · Bráulio Barros de Oliveira

but (as always there is), the situation is not good yet when using kde4. with it, i can't find out which process is accessing the hd, cause iotop only shows a pdflush access.
i've tried to kill some apps, but it didn't help. any other trace technic?

regards,
bráulio

Quoted 124 lines · Bráulio Barros de Oliveira

hello mat, here comes the results of each action.
in all the process i've used the iotop (what a great utility you pointed!), and always saw only the pdflush process acessing the hd. no user interface were used.
the apm is set to 128 with hdparm

ok, then try the following:

you at least should have noatime,nodiratime,commit=600

:) that did the job! also, i've tested without the commit option, which didn't changed the result.

enabled for the reiserfs or >=ext3 filesystems

you can also try data=writeback (but beware this might put your data at risk !)

* can't try

select the anticipatory i/o scheduler
and set following stuff

:( no change.

echo "16" > /proc/sys/vm/page-cluster
# default: 3
#

:( no change.

echo "60" > /proc/sys/vm/swappiness
# default: 60
# By default, Linux will aggressively swap processes out of physical memory onto disk in order to keep the disk cache as large as possible.
# This means that pages that haven't been used recently will be pushed into swap long before the system even comes close to running out of memory, which is an unexpected behavior compared to some operating systems.
# The /proc/sys/vm/swappiness parameter controls how aggressive Linux is in this area.

:( no change.

echo "3000" > /proc/sys/vm/dirty_expire_centisecs
# default: 3000 (30 seconds)
#2 how long data can be in the page cache before it is considered expired and must be written at the next opportunity. Note that this default is very long: a full 30 seconds. That means that under normal circumstances, unless you write enough to trigger the other pdflush method, Linux won't actually commit anything you write until 30 seconds later.

:( no change.

echo "6000" > /proc/sys/vm/dirty_writeback_centisecs
# default: 500 (5 seconds)
#1 how often pdflush wakes up to write data to disk. The default wakes up the two (or more) active threads every five seconds.
# suggestion: 6000 (every 60 seconds)

:( no change.

echo "15" > /proc/sys/vm/dirty_background_ratio
# default: 10
#3 Maximum percentage of active memory that can be filled with dirty pages before pdflush begins to write them

echo "50" > /proc/sys/vm/dirty_ratio #modified
# default: 40
#4 Maximum percentage of total memory that can be filled with dirty pages before processes are forced to write dirty buffers themselves during their time slice instead of being allowed to do more writes.
# modified: 50

echo "25" > /proc/sys/vm/vfs_cache_pressure

for i in /sys/block/sd*; do
/bin/echo "anticipatory" > $i/queue/scheduler
done

for i in /sys/block/sd*; do
/bin/echo "0" > $i/queue/iosched/antic_expire
done

for i in /sys/block/sd*; do

       /bin/echo "150" >  $i/queue/iosched/read_expire

done

for i in /sys/block/sd*; do

       /bin/echo "750" >  $i/queue/iosched/read_batch_expire

done

for i in /sys/block/sd*; do

       /bin/echo "1200" >  $i/queue/iosched/write_batch_expire

done

for i in /sys/block/sd*; do

       /bin/echo "1024" >  $i/queue/nr_requests

done

for i in /sys/block/sd*; do

       /bin/echo "256" >  $i/queue/read_ahead_kb

done

for i in /sys/block/sd*; do

       /bin/echo "256" >  $i/queue/max_sectors_kb

done

for i in /sys/class/scsi_host/host*; do
/bin/echo "min_power" > $i/link_power_management_policy
done

try to disable each and every unneeded daemon or applets, programs, etc.

by using:

* powertop
* iotop
* htop
* top
* lsof | grep /home
* ...

#5 lore raw
Quoted 4 lines · Bráulio Barros de Oliveira

just found out! it is konsole, unbelievable! going to file a bug report right now.
again, thank you very much for the support.
bráulio

let me guess - konsole writes to a temp file in /tmp containing the backlog? That is not a bug but a feature. You can 'save' you some trouble by putting /tmp on a tempfs. 256mb should be enough.
--
To unsubscribe from this list: send the line "unsubscribe reiserfs-devel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

#6 lore raw

hello volker,

running tree command on /tmp i couldn't associate any file with konsole. that's the result
[braulio@brauliopresto tmp]$ tree
.
|-- kde-root [error opening dir]
`-- kde4-braulio

    |-- kde-braulio
    |   |-- 28165-cm81.skz
    |   |-- 86714-PicoNet.skz
    |   |-- closeditems
    |   |-- konqueror-crash-X31257.log
    |   |-- konqueror-crash-X31258.log
    |   |-- konqueror-crash-X31259.log
    |   |-- konqueror-crash-X31262.log
    |   |-- konqueror-crash-X31265.log
    |   `-- ksycoca -> /var/tmp/kdecache-braulio/ksycoca
    `-- ksocket-braulio
        |-- KSMserver__0
        |-- kdeinit-:0
        |-- kdeinit4__0
        |-- kdeinit__0
        |-- klauncherMT3820.slave-socket
        |-- klauncherT31205.slave-socket
        `-- klaunchergALyma.slave-socket

5 directories, 15 files

regards,
bráulio

Quoted 14 lines · Volker Armin Hemmann
Quoted 4 lines · Bráulio Barros de Oliveira

just found out! it is konsole, unbelievable! going to file a bug report right now.
again, thank you very much for the support.
bráulio

let me guess - konsole writes to a temp file in /tmp containing the backlog? That is not a bug but a feature. You can 'save' you some trouble by putting /tmp on a tempfs. 256mb should be enough.
--
To unsubscribe from this list: send the line "unsubscribe reiserfs-devel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

#7 lore raw
Quoted 35 lines · Bráulio Barros de Oliveira

hello volker,

running tree command on /tmp i couldn't associate any file with konsole. that's the result
[braulio@brauliopresto tmp]$ tree
.

|-- kde-root [error opening dir]

`-- kde4-braulio

    |-- kde-braulio
    |
    |   |-- 28165-cm81.skz
    |   |-- 86714-PicoNet.skz
    |   |-- closeditems
    |   |-- konqueror-crash-X31257.log
    |   |-- konqueror-crash-X31258.log
    |   |-- konqueror-crash-X31259.log
    |   |-- konqueror-crash-X31262.log
    |   |-- konqueror-crash-X31265.log
    |
    |   `-- ksycoca -> /var/tmp/kdecache-braulio/ksycoca
    `-- ksocket-braulio

        |-- KSMserver__0
        |-- kdeinit-:0
        |-- kdeinit4__0
        |-- kdeinit__0
        |-- klauncherMT3820.slave-socket
        |-- klauncherT31205.slave-socket

        `-- klaunchergALyma.slave-socket
Quoted 1 line

5 directories, 15 files

try lsof (as superuser). lsof |grep konsole and prepare to be surprised. --
To unsubscribe from this list: send the line "unsubscribe reiserfs-devel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html