Testers wanted: dracut lazy install with cpio

Hi guys,

I made a patch for lazy installing the files with cpio.

I am interested what the difference in speed is, to evaluate, if it is worth the hazzle.

The patch:
http://git.surfsite.org/?p=dracut.git;a=commit;h=df512139fe9c23b84a2213c934e7a7feb354bdcc

can be found on git://git.surfsite.org/pub/git/dracut.git branch:lazyinstall

I am interested in:

$ time sudo make testimage
$ time sudo make hostimage

Both with and without the patch.

Thanks for testing!

I get (for Fedora 16):

with patch (lazy) without patch
testimage: 24.3 real 25.8 real
30.2 user 30.7 user
4.5 sys 4.9 sys

   hostimage:   8.4 real             9.3 real
                8.1 user             8.4 user
                1.5 sys              1.9 real

Did the cpio run in parallel? cpio reads from a pipe just fine.

cpio needs enhancement: a flag argument for "no overwrite", and another for "clone the ownership and permissions on created directories" [else a copied path "a/b/c/d/e" which must create any of a, b, c, or d will not reproduce the existing ownership and permissions on those directories.]

[wangcong@cr0]~/dracut% git branch
lazyinstall
* master
[wangcong@cr0]~/dracut% time sudo make testimage
...
sudo make testimage 38.20s user 8.04s system 95% cpu 48.509 total

[wangcong@cr0]~/dracut% time sudo make hostimage
...
sudo make hostimage 11.69s user 4.00s system 101% cpu 15.401 total

[wangcong@cr0]~/dracut% git checkout lazyinstall Switched to branch 'lazyinstall'
[wangcong@cr0]~/dracut% git branch
* lazyinstall
master
[wangcong@cr0]~/dracut% time sudo make testimage
...
sudo make testimage 38.03s user 8.11s system 133% cpu 34.474 total

[wangcong@cr0]~/dracut% time sudo make hostimage
...
sudo make hostimage 11.36s user 3.45s system 109% cpu 13.478 total

Was that run with a "cold cache"? (What are the results for the a second run on testimage, immediately after the first?) I ask because the difference (48.5 - 34.5) between master and lazy on testimage is much larger as a percentage than the (15.4 - 13.5) difference for hostimage; and the testimage case was run first.

Am 28.02.2012 13:43, schrieb Cong Wang:

34s vs 48s
13s vs 15s

not sooo bad... but still not convinced

Am 28.02.2012 16:09, schrieb John Reiser:

yes, "cold cache" should be ruled out.. better do several runs and pick the fastest one.

Is the purpose of this work to speed things up for the user, or for the developers? Your approach is correct for developers, since we're more likely to be making multiple dracut image with a warm cache. But users are more likely to hit the cold cache case.

Both users and developers. However, the cache must be controlled (or known) for any measurement, else there is no way to attribute the effects of any measured change: the cache effect could be much larger than any improvement (or degredation). The easiest way is to have a warm cache: discard the first run of a series because of the unknown cache state.

The cold cache case can be approximated by:

    # echo 1 > /proc/sys/vm/drop_caches
    # time dracut ...
    # echo 1 > /proc/sys/vm/drop_caches    # each time!
    # time dracut ...
but that path /proc/sys/vm/drop_caches is harder to remember
(and harder to get correct in a sudo!)
It is also harder to guarantee consistency on a machine that is
not otherwise idle.

Running dracut after updating kernel and/or drivers is likely to hit a mixed cache: the updated stuff will tend to be hot, the not-updated stuff probably will be cold.

Your commentary on the effects of caching and non-idle machines is true and a useful reminder, but I think you talked past my point -- Harald seemed to be wondering if the speedup was worth the complexity, and that can only be answered by benchmarking it in the appropriate environment. A fully warmed up cache is easiest and probably correct for developers, but does not cover the case for users -- though if the speed up is good there, then it is likely better for the users' case.

Your point about a mixed cache from an updated kernel is well taken; perhaps a good way to replicate the user's experience would be to put something like the following in a script -- so you don't have to remember the path and get it right in a sudo:

for i in {1..10}; do

	sudo sh -c "echo 1 > /proc/sys/vm/drop_caches"
	find /lib/modules/$(uname -r) -type f -print0 | xargs -0 cat > /dev/null
	time ...dracut...

done

and best if you do your testing from single user mode (may not even need the sudo, then). It won't be perfect, but I think it would be more meaningful than the easy case.

I tend to agree that the ~2 second improvements reported so far are not compelling, but I see those as the developers case. I'm more interested in the users' case -- is that 14 seconds Cong reported real?

I suspect that the 14 seconds difference was between a cold cache (first run) and a warm cache (third run). There wasn't anything non-real about it, but the cache effect was much larger than the lazy/non-lazy algorithm. The measurements that I reported yesterday were all warm-cache cases, and I saw much closer improvement in wall-clock time (several percent) for lazy, in both testimage and hostimage. This is consistent with the hypothesis that the improvement was due to the lazy case using one (or a small handful) cpio of many files at a time, versus the non-lazy case doing one /bin/cp for each file. The savings is in reduced usage of fork[clone]+execve+ld_linux.so+wait.