iov_iter: import single segments iovecs as ITER_UBUF
Subsystems (3)
RFC PATCH 1/1 iov_iter: import single segments iovecs as ITER_UBUF
Using an ITER_UBUF is more efficient than an ITER_IOV, and for the single segment case, there's no reason to use an ITER_IOV when an ITER_UBUF will do. Experimental data collected shows that ~2/3rds of iovec imports are single segments, from applications using readv/writev or recvmsg/sendmsg that are iovec based.
Explicitly check for nr_segs == 1 and import those as ubuf rather than iovec based iterators.
---
Lightly tested - boots and works just fine, and I ran this through the liburing test suite which does plenty of single segment readv/writev as well.
lib/iov_iter.c +15 -0
@@ -1813,6 +1813,21 @@ ssize_t __import_iovec(int type, const struct iovec __user *uvec,return PTR_ERR (iov); } +/* +* Fast path - single segment import. Use UBUF for these, rather +* than setup an ITER_IOV. +*/ +if (nr_segs ==1 ) { +ssize_t ret; + + total_len = iovp[0 ]->iov_len; + ret =import_ubuf (type, iovp[0 ]->iov_base, total_len, i); + *iovp =NULL ; +if (unlikely (ret <0 )) +return ret; +return total_len; + } +/* * According to the Single Unix Specification we should return EINVAL if * an element length is < 0 when cast to ssize_t or if the total length
Signature · 1 line
Jens Axboe
Al Viro
maintainerQuoted 8 lines · Jens Axboe
Using an
ITER_UBUFis more efficient than anITER_IOV, and for the single segment case, there's no reason to use anITER_IOVwhen anITER_UBUFwill do. Experimental data collected shows that ~2/3rds of iovec imports are single segments, from applications using readv/writev or recvmsg/sendmsg that are iovec based.Explicitly check for
nr_segs== 1 and import those as ubuf rather than iovec based iterators.
Hadn't we'd been through that before? There is infinibarf code that
assumes ITER_IOVEC for what its ->write_iter() gets (and yes, that's the one that has ->write() with different semantics).
And I wouldn't bet a dime on all ->sendmsg() and ->recvmsg() being flavour-agnostic either...
Al Viro
maintainerQuoted 10 lines · Al Viro
Quoted 8 lines · Jens Axboe
Using an
ITER_UBUFis more efficient than anITER_IOV, and for the single segment case, there's no reason to use anITER_IOVwhen anITER_UBUFwill do. Experimental data collected shows that ~2/3rds of iovec imports are single segments, from applications using readv/writev or recvmsg/sendmsg that are iovec based.Explicitly check for
nr_segs== 1 and import those as ubuf rather than iovec based iterators.
Quoted 6 lines
Hadn't we'd been through that before? There is infinibarf code that
assumesITER_IOVECfor what its ->write_iter() gets (and yes, that's the one that has ->write() with different semantics).And I wouldn't bet a dime on all ->sendmsg() and ->recvmsg() being flavour-agnostic either...
Incidentally, what will your patch do to one-segment readv(2) from e.g./proc/self/status? Or anything else that has no ->read_iter, for that matter...
Quoted 15 lines · Al Viro
Quoted 14 lines · Al Viro
Quoted 8 lines · Jens Axboe
Using an
ITER_UBUFis more efficient than anITER_IOV, and for the single segment case, there's no reason to use anITER_IOVwhen anITER_UBUFwill do. Experimental data collected shows that ~2/3rds of iovec imports are single segments, from applications using readv/writev or recvmsg/sendmsg that are iovec based.Explicitly check for
nr_segs== 1 and import those as ubuf rather than iovec based iterators.Hadn't we'd been through that before? There is infinibarf code that
assumesITER_IOVECfor what its ->write_iter() gets (and yes, that's the one that has ->write() with different semantics).
Quoted 6 lines
Quoted 2 lines
And I wouldn't bet a dime on all ->sendmsg() and ->recvmsg() being flavour-agnostic either...
Incidentally, what will your patch do to one-segment readv(2) from e.g.
/proc/self/status? Or anything else that has no ->read_iter, for that matter...
Yes indeed, that won't work at all... I guess this has to be up to the caller, we can't stuff it this far down.
Signature · 1 line
Jens Axboe
Quoted 7 lines · Al Viro
Quoted 6 lines · Jens Axboe
Using an
ITER_UBUFis more efficient than anITER_IOV, and for the single segment case, there's no reason to use anITER_IOVwhen anITER_UBUFwill do. Experimental data collected shows that ~2/3rds of iovec imports are single segments, from applications using readv/writev or recvmsg/sendmsg that are iovec based.
Quoted 6 lines
Quoted 2 lines
Explicitly check for
nr_segs== 1 and import those as ubuf rather than iovec based iterators.Hadn't we'd been through that before? There is infinibarf code that
assumesITER_IOVECfor what its ->write_iter() gets (and yes, that's the one that has ->write() with different semantics).
This is why it's an RFC... Would like to do something about that!
Quoted 2 lines
And I wouldn't bet a dime on all ->sendmsg() and ->recvmsg() being flavour-agnostic either...
FWIW, the liburing regressions tests do have sendmsg/recvmsg and at least basic stuff works. Haven't looked at all deeper on whether this is always the case.
Signature · 1 line
Jens Axboe
kernel test robot
build bot[If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use '--base' as documented in https://git-scm.com/docs/git-format-patch]
Hi Jens,
[FYI, it's a private test report for your RFC patch.]
[auto build test ERROR on linus/master]
[also build test ERROR on v5.19-rc2 next-20220617]
[If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch]
url: https://github.com/intel-lab-lkp/linux/commits/Jens-Axboe/iov_iter-import-single-segments-iovecs-as-ITER_UBUF/20220618-221039 base: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git 4b35035bcf80ddb47c0112c4fbd84a63a2836a18 config: i386-randconfig-a002 (https://download.01.org/0day-ci/archive/20220619/[email protected]/config) compiler: clang version 15.0.0 (https://github.com/llvm/llvm-project 91688716ba49942051dccdf7b9c4f81a7ec8feaf)
reproduce (this is a W=1 build):
wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross chmod +x ~/bin/make.cross
# https://github.com/intel-lab-lkp/linux/commit/ec74a6fdd818738cf8e0c0f10798fa001970bb81
git remote add linux-review https://github.com/intel-lab-lkp/linux
git fetch --no-tags linux-review Jens-Axboe/iov_iter-import-single-segments-iovecs-as-ITER_UBUF/20220618-221039 git checkout ec74a6fdd818738cf8e0c0f10798fa001970bb81
# save the config file
mkdir build_dir && cp config build_dir/.config
COMPILER_INSTALL_PATH=$HOME/0day COMPILER=clang make.cross W=1 O=build_dir ARCH=i386 SHELL=/bin/bash
If you fix the issue, kindly add following tag where applicable
All errors (new ones prefixed by >>):
>> lib/iov_iter.c:1950:9: error: call to undeclared function 'import_ubuf'; ISO C99 and later do not support implicit function declarations [-Wimplicit-function-declaration]
ret = import_ubuf(type, iovp[0]->iov_base, total_len, i);
^
lib/iov_iter.c:1950:9: note: did you mean 'import_uuid'?
include/linux/uuid.h:71:20: note: 'import_uuid' declared here
static inline void import_uuid(uuid_t *dst, const __u8 *src)
^1 error generated.
vim +/import_ubuf +1950 lib/iov_iter.c
1927
1928 ssize_t __import_iovec(int type, const struct iovec __user *uvec,
1929 unsigned nr_segs, unsigned fast_segs, struct iovec **iovp,
1930 struct iov_iter *i, bool compat)
1931 {
1932 ssize_t total_len = 0;
1933 unsigned long seg;
1934 struct iovec *iov;
1935
1936 iov = iovec_from_user(uvec, nr_segs, fast_segs, *iovp, compat);
1937 if (IS_ERR(iov)) {
1938 *iovp = NULL;
1939 return PTR_ERR(iov);
1940 }
1941
1942 /*
1943 * Fast path - single segment import. Use UBUF for these, rather
1944 * than setup an ITER_IOV.
1945 */
1946 if (nr_segs == 1) {
1947 ssize_t ret;
1948
1949 total_len = iovp[0]->iov_len;
> 1950 ret = import_ubuf(type, iovp[0]->iov_base, total_len, i);
1951 *iovp = NULL;
1952 if (unlikely(ret < 0))
1953 return ret;
1954 return total_len;
1955 }
1956
1957 /*
1958 * According to the Single Unix Specification we should return EINVAL if
1959 * an element length is < 0 when cast to ssize_t or if the total length
1960 * would overflow the ssize_t return value of the system call.
1961 *
1962 * Linux caps all read/write calls to MAX_RW_COUNT, and avoids the
1963 * overflow case.
1964 */
1965 for (seg = 0; seg < nr_segs; seg++) {
1966 ssize_t len = (ssize_t)iov[seg].iov_len;
1967
1968 if (!access_ok(iov[seg].iov_base, len)) {
1969 if (iov != *iovp)
1970 kfree(iov);
1971 *iovp = NULL;
1972 return -EFAULT;
1973 }
1974
1975 if (len > MAX_RW_COUNT - total_len) {
1976 len = MAX_RW_COUNT - total_len;
1977 iov[seg].iov_len = len;
1978 }
1979 total_len += len;
1980 }
1981
1982 iov_iter_init(i, type, iov, nr_segs, total_len);
1983 if (iov == *iovp)
1984 *iovp = NULL;
1985 else
1986 *iovp = iov;
1987 return total_len;
1988 }
1989
Signature · 2 lines
0-DAY CI Kernel Test Service https://01.org/lkp