Making nice niser for system hogging programs
Just a thought -----
Programs like cp -a /bigdir /backup and rsync usually bring the server to a crawl no matter how much "nice" you put on them. Is there any way to make "nice" smarter in that it limits io as well as processor usage? If cp and rsyne ran a little slower IO wise then everything else could run too.
Signature · 4 lines
Marc Perkel - [email protected] Spam Filter: http://www.junkemailfilter.com My Blog: http://marc.perkel.com
Quoted 7 lines · Marc Perkel
Just a thought -----
Programs like cp -a /bigdir /backup and rsync usually bring the server to a crawl no matter how much "nice" you put on them. Is there any way to make "nice" smarter in that it limits io as well as processor usage? If cp and rsyne ran a little slower IO wise then everything else could run too.
The latest cfq io scheduler supports io nice levels. By default it links the io nice levels to the cpu nice levels so if you use cfq and set your file commands nice 19 they will use as little io priority as possible. Note this only works on the read side but that makes a dramatic difference already.
Cheers
Con
Quoted 13 lines · Con Kolivas
Quoted 9 lines · Marc Perkel
Just a thought -----
Programs like cp -a /bigdir /backup and rsync usually bring the server to a crawl no matter how much "nice" you put on them. Is there any way to make "nice" smarter in that it limits io as well as processor usage? If cp and rsyne ran a little slower IO wise then everything else could run too.
Quoted 6 lines
The latest cfq io scheduler supports io nice levels. By default it links the io nice levels to the cpu nice levels so if you use cfq and set your file commands nice 19 they will use as little io priority as possible. Note this only works on the read side but that makes a dramatic difference already.
Cheers
Con
Kewl - so - what version is it in?
Signature · 4 lines
Marc Perkel - [email protected] Spam Filter: http://www.junkemailfilter.com My Blog: http://marc.perkel.com
Quoted 10 lines · Marc Perkel
Quoted 9 lines · Con Kolivas
Quoted 7 lines · Marc Perkel
Just a thought -----
Programs like cp -a /bigdir /backup and rsync usually bring the server to a crawl no matter how much "nice" you put on them. Is there any way to make "nice" smarter in that it limits io as well as processor usage? If cp and rsyne ran a little slower IO wise then everything else could run too.
Quoted 7 lines
Quoted 5 lines
The latest cfq io scheduler supports io nice levels. By default it links the io nice levels to the cpu nice levels so if you use cfq and set your file commands nice 19 they will use as little io priority as possible. Note this only works on the read side but that makes a dramatic difference already.
Kewl - so - what version is it in?
2.6.13 already has it. Note that the io priority is only inherited when a process first starts so doing renice to something already running will only change the cpu nice, not the ionice. You can do that on the fly using the ionice application. If you look in the kernel source in Documentation/block/ioprio.txt you'll find the source to ionice.c
Cheers,
Con
P.S. It is considered routine to reply-to-all when posting to lkml and not stripping anybody on the cc list.
Quoted 9 lines · Con Kolivas
Quoted 7 lines · Marc Perkel
Just a thought -----
Programs like cp -a /bigdir /backup and rsync usually bring the server to a crawl no matter how much "nice" you put on them. Is there any way to make "nice" smarter in that it limits io as well as processor usage? If cp and rsyne ran a little slower IO wise then everything else could run too.
Quoted 7 lines
The latest cfq io scheduler supports io nice levels. By default it links the io nice levels to the cpu nice levels so if you use cfq and set your file commands nice 19 they will use as little io priority as possible. Note this only works on the read side but that makes a dramatic difference already.
Cheers
Con
If you want a way to assign io priorities without relying on process
inheritance and (re)nice you might find CKRM, with it's cfq-based IO controller, useful.
Basically you create a set of classes that group tasks and give an
appropriate share of IO performance to tasks in that class. As processes get created CKRM will assign tasks to the IO classes based on a set of rules. You can run commands like:
mkdir /rcfs/taskclass/makin_copies
echo 'res=io,guarantee=20' > /rcfs/taskclass/makin_copies/shares echo 'path=/usr/bin/rsync,class=makin_copies' > /rcfs/ce/rsync_rule echo 'path=/bin/cp,class=makin_copies' > /rcfs/ce/cp_rule
to set this up. This should make CKRM useful for those unwilling to become full-time administrators just to run their own desktops.
Cheers,
-Matt Helsley
Quoted 27 lines · Matt Helsley
Quoted 9 lines · Con Kolivas
Quoted 7 lines · Marc Perkel
Just a thought -----
Programs like cp -a /bigdir /backup and rsync usually bring the server to a crawl no matter how much "nice" you put on them. Is there any way to make "nice" smarter in that it limits io as well as processor usage? If cp and rsyne ran a little slower IO wise then everything else could run too.
Quoted 7 lines
The latest cfq io scheduler supports io nice levels. By default it links the io nice levels to the cpu nice levels so if you use cfq and set your file commands nice 19 they will use as little io priority as possible. Note this only works on the read side but that makes a dramatic difference already.
Cheers
ConIf you want a way to assign io priorities without relying on process
inheritance and (re)nice you might find CKRM, with it's cfq-based IO controller, useful.Basically you create a set of classes that group tasks and give an
appropriate share of IO performance to tasks in that class. As processes get created CKRM will assign tasks to the IO classes based on a set of rules. You can run commands like:
Quoted 7 lines
mkdir /rcfs/taskclass/makin_copies
echo 'res=io,guarantee=20' > /rcfs/taskclass/makin_copies/shares echo 'path=/usr/bin/rsync,class=makin_copies' > /rcfs/ce/rsync_rule echo 'path=/bin/cp,class=makin_copies' > /rcfs/ce/cp_ruleto set this up. This should make CKRM useful for those unwilling to become full-time administrators just to run their own desktops.
Has the CKRM io controller been updated to 2.6.13 level CFQ, or is it still using the very basic per-request based io sharing?
Signature · 1 line
Jens Axboe