Skip to contentSkip to conversation

Regression ~7% write, 3.17.1 vs 3.18.0rc5

DISCUSSION Matus Kocka 2 participants 2 messages Nov 28, 2014 VFS

lore
Share & tools
lore
#0 lore raw

Hello,

we see the ~7% regression in the initial WRITE on xfs in in-cache mode by IOZONE test between 3.17.1-306.el7 and 3.18.0-0.rc5.git0.1.el7 (respectively 3.18.0-0.rc3.git0.2.el7)

By in-cache mode we mean that file fits into the OS cache.
For big file-sizes >4GiB (out of cache mode) there is no performance drop.

[Files: https://www.dropbox.com/sh/2sx0ejt73n4idcp/AAA0nYhZTDAMGigJ-B5yqTwga?dl=0]

1, Firstly, (xfs_SATA.pdf) we see the ~7% regression in the in-cache IOZONE test on 1 TB SATA disk with cfq io-elevator,

2, We also see it (xfs_write.pdf) using same in-cache test on the fast pci SSD disk (RealSSD P320h 350GB)

3, We have tried direct-io tests on the same pci SSD with no regression (directio.pdf)

It seems that this regression is related to cache since DIRECT IO and out-of-cache tests
do not have the regression. Also, rewrite test does not show any
regression.

Would somebody from the developers be interested to look into it and try to hunt down this bug? We can provide more details regarding our testing methodology but we have neither capacity nor expertise to debug it ourselves.

Signature · 15 lines
Matus Kocka

Quality Assurance Associate
Red Hat Czech s.r.o.
Purkyňova 99/71, 612 45, Brno, Czech Republic

   tel.: +420 532 294 793
direct: 8262793
  email: mkocka [at] redhat.com


--
To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
#1 lore raw
Quoted 8 lines · Matus Kocka

Hello,

we see the ~7% regression in the initial WRITE on xfs in in-cache mode by IOZONE test
between 3.17.1-306.el7 and 3.18.0-0.rc5.git0.1.el7 (respectively 3.18.0-0.rc3.git0.2.el7)

By in-cache mode we mean that file fits into the OS cache.

As we all know by now, this iozone test is extremely sensitive to changes in CPU cache footprint rather than anything to do with the IO subsystem. Is this "regression" reproducable on more than one machine? Chasing IOZone "regressions" measured on a single machine is a wild goose chase....

Quoted 1 line

For big file-sizes >4GiB (out of cache mode) there is no performance drop.

So it's a CPU usage issue, then?

Quoted 11 lines

[Files: https://www.dropbox.com/sh/2sx0ejt73n4idcp/AAA0nYhZTDAMGigJ-B5yqTwga?dl=0]

1, Firstly, (xfs_SATA.pdf) we see the ~7% regression in the in-cache IOZONE test on 1 TB SATA disk with cfq io-elevator,

2, We also see it (xfs_write.pdf) using same in-cache test on the fast pci SSD disk (RealSSD P320h 350GB)

3, We have tried direct-io tests on the same pci SSD with no regression (directio.pdf)

Quoted 7 lines

It seems that this regression is related to cache since DIRECT IO and out-of-cache tests
do not have the regression. Also, rewrite test does not show any
regression.

Would somebody from the developers be interested to look into it and try to hunt down this bug?

Can you please bisect so we can identify the problematic commit?

Quoted 3 lines

We can provide more details regarding our testing methodology but we have neither capacity nor
expertise to debug it ourselves.

You don't need to debug the problem, just run the test repeatedly on the kernels the bisect builds until it identifies the commit that introduced the regression.

Cheers,

Dave.

Signature · 2 lines
Dave Chinner
[email protected]