Regression ~7% write, 3.17.1 vs 3.18.0rc5
Hello,
we see the ~7% regression in the initial WRITE on xfs in in-cache mode by IOZONE test between 3.17.1-306.el7 and 3.18.0-0.rc5.git0.1.el7 (respectively 3.18.0-0.rc3.git0.2.el7)
By in-cache mode we mean that file fits into the OS cache.
For big file-sizes >4GiB (out of cache mode) there is no performance drop.
[Files: https://www.dropbox.com/sh/2sx0ejt73n4idcp/AAA0nYhZTDAMGigJ-B5yqTwga?dl=0]
1, Firstly, (xfs_SATA.pdf) we see the ~7% regression in the in-cache IOZONE test on 1 TB SATA disk with cfq io-elevator,
2, We also see it (xfs_write.pdf) using same in-cache test on the fast pci SSD disk (RealSSD P320h 350GB)
3, We have tried direct-io tests on the same pci SSD with no regression (directio.pdf)
It seems that this regression is related to cache since DIRECT IO and out-of-cache tests
do not have the regression. Also, rewrite test does not show any
regression.
Would somebody from the developers be interested to look into it and try to hunt down this bug? We can provide more details regarding our testing methodology but we have neither capacity nor expertise to debug it ourselves.
Signature · 15 lines
Matus Kocka Quality Assurance Associate Red Hat Czech s.r.o. Purkyňova 99/71, 612 45, Brno, Czech Republic tel.: +420 532 294 793 direct: 8262793 email: mkocka [at] redhat.com -- To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
Quoted 8 lines · Matus Kocka
Hello,
we see the ~7% regression in the initial WRITE on xfs in in-cache mode by IOZONE test
between 3.17.1-306.el7 and 3.18.0-0.rc5.git0.1.el7 (respectively 3.18.0-0.rc3.git0.2.el7)By in-cache mode we mean that file fits into the OS cache.
As we all know by now, this iozone test is extremely sensitive to changes in CPU cache footprint rather than anything to do with the IO subsystem. Is this "regression" reproducable on more than one machine? Chasing IOZone "regressions" measured on a single machine is a wild goose chase....
Quoted 1 line
For big file-sizes >4GiB (out of cache mode) there is no performance drop.
So it's a CPU usage issue, then?
Quoted 11 lines
[Files: https://www.dropbox.com/sh/2sx0ejt73n4idcp/AAA0nYhZTDAMGigJ-B5yqTwga?dl=0]
1, Firstly, (xfs_SATA.pdf) we see the ~7% regression in the in-cache IOZONE test on 1 TB SATA disk with cfq io-elevator,
2, We also see it (xfs_write.pdf) using same in-cache test on the fast pci SSD disk (RealSSD P320h 350GB)
3, We have tried direct-io tests on the same pci SSD with no regression (directio.pdf)
Quoted 7 lines
It seems that this regression is related to cache since DIRECT IO and out-of-cache tests
do not have the regression. Also, rewrite test does not show any
regression.Would somebody from the developers be interested to look into it and try to hunt down this bug?
Can you please bisect so we can identify the problematic commit?
Quoted 3 lines
We can provide more details regarding our testing methodology but we have neither capacity nor
expertise to debug it ourselves.
You don't need to debug the problem, just run the test repeatedly on the kernels the bisect builds until it identifies the commit that introduced the regression.
Cheers,
Dave.
Signature · 2 lines
Dave Chinner [email protected]