Skip to contentSkip to conversation

iov_iter: import single segments iovecs as ITER_UBUF

RFC PATCH In review Jens Axboe 3 participants 6 messages Jun 18, 2022

Subsystems (3)
lore
Share & tools
lore

Jens Axboe

author
#0 lore raw

RFC PATCH 1/1 iov_iter: import single segments iovecs as ITER_UBUF

Using an ITER_UBUF is more efficient than an ITER_IOV, and for the single segment case, there's no reason to use an ITER_IOV when an ITER_UBUF will do. Experimental data collected shows that ~2/3rds of iovec imports are single segments, from applications using readv/writev or recvmsg/sendmsg that are iovec based.

Explicitly check for nr_segs == 1 and import those as ubuf rather than iovec based iterators.

---

Lightly tested - boots and works just fine, and I ran this through the liburing test suite which does plenty of single segment readv/writev as well.

lib/iov_iter.c +15 -0
index 6b2bf6f6f374..0973c622d3c0 100644
@@ -1813,6 +1813,21 @@ ssize_t __import_iovec(int type, const struct iovec __user *uvec,
 		return PTR_ERR(iov);
 	}
 
+	/*
+	 * Fast path - single segment import. Use UBUF for these, rather
+	 * than setup an ITER_IOV.
+	 */
+	if (nr_segs == 1) {
+		ssize_t ret;
+
+		total_len = iovp[0]->iov_len;
+		ret = import_ubuf(type, iovp[0]->iov_base, total_len, i);
+		*iovp = NULL;
+		if (unlikely(ret < 0))
+			return ret;
+		return total_len;
+	}
+
 	/*
 	 * According to the Single Unix Specification we should return EINVAL if
 	 * an element length is < 0 when cast to ssize_t or if the total length
Signature · 1 line
Jens Axboe
maintainer
#1 lore raw
Quoted 8 lines · Jens Axboe

Using an ITER_UBUF is more efficient than an ITER_IOV, and for the single segment case, there's no reason to use an ITER_IOV when an ITER_UBUF will do. Experimental data collected shows that ~2/3rds of iovec imports are single segments, from applications using readv/writev or recvmsg/sendmsg that are iovec based.

Explicitly check for nr_segs == 1 and import those as ubuf rather than iovec based iterators.

Hadn't we'd been through that before? There is infinibarf code that
assumes ITER_IOVEC for what its ->write_iter() gets (and yes, that's the one that has ->write() with different semantics).

And I wouldn't bet a dime on all ->sendmsg() and ->recvmsg() being flavour-agnostic either...

maintainer
#2 lore raw
Quoted 10 lines · Al Viro
Quoted 8 lines · Jens Axboe

Using an ITER_UBUF is more efficient than an ITER_IOV, and for the single segment case, there's no reason to use an ITER_IOV when an ITER_UBUF will do. Experimental data collected shows that ~2/3rds of iovec imports are single segments, from applications using readv/writev or recvmsg/sendmsg that are iovec based.

Explicitly check for nr_segs == 1 and import those as ubuf rather than iovec based iterators.

Quoted 6 lines

Hadn't we'd been through that before? There is infinibarf code that
assumes ITER_IOVEC for what its ->write_iter() gets (and yes, that's the one that has ->write() with different semantics).

And I wouldn't bet a dime on all ->sendmsg() and ->recvmsg() being flavour-agnostic either...

Incidentally, what will your patch do to one-segment readv(2) from e.g.
/proc/self/status? Or anything else that has no ->read_iter, for that matter...

author
#3 lore raw
Quoted 15 lines · Al Viro
Quoted 14 lines · Al Viro
Quoted 8 lines · Jens Axboe

Using an ITER_UBUF is more efficient than an ITER_IOV, and for the single segment case, there's no reason to use an ITER_IOV when an ITER_UBUF will do. Experimental data collected shows that ~2/3rds of iovec imports are single segments, from applications using readv/writev or recvmsg/sendmsg that are iovec based.

Explicitly check for nr_segs == 1 and import those as ubuf rather than iovec based iterators.

Hadn't we'd been through that before? There is infinibarf code that
assumes ITER_IOVEC for what its ->write_iter() gets (and yes, that's the one that has ->write() with different semantics).

Quoted 6 lines
Quoted 2 lines

And I wouldn't bet a dime on all ->sendmsg() and ->recvmsg() being flavour-agnostic either...

Incidentally, what will your patch do to one-segment readv(2) from e.g. /proc/self/status? Or anything else that has no ->read_iter, for that matter...

Yes indeed, that won't work at all... I guess this has to be up to the caller, we can't stuff it this far down.

Signature · 1 line
Jens Axboe
author
#4 lore raw
In reply to Al Viro
Quoted 7 lines · Al Viro
Quoted 6 lines · Jens Axboe

Using an ITER_UBUF is more efficient than an ITER_IOV, and for the single segment case, there's no reason to use an ITER_IOV when an ITER_UBUF will do. Experimental data collected shows that ~2/3rds of iovec imports are single segments, from applications using readv/writev or recvmsg/sendmsg that are iovec based.

Quoted 6 lines
Quoted 2 lines

Explicitly check for nr_segs == 1 and import those as ubuf rather than iovec based iterators.

Hadn't we'd been through that before? There is infinibarf code that
assumes ITER_IOVEC for what its ->write_iter() gets (and yes, that's the one that has ->write() with different semantics).

This is why it's an RFC... Would like to do something about that!

Quoted 2 lines

And I wouldn't bet a dime on all ->sendmsg() and ->recvmsg() being flavour-agnostic either...

FWIW, the liburing regressions tests do have sendmsg/recvmsg and at least basic stuff works. Haven't looked at all deeper on whether this is always the case.

Signature · 1 line
Jens Axboe
#5 lore raw
In reply to Jens Axboe
[If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use '--base' as documented in https://git-scm.com/docs/git-format-patch]

Hi Jens,

[FYI, it's a private test report for your RFC patch.]
[auto build test ERROR on linus/master]
[also build test ERROR on v5.19-rc2 next-20220617]
[If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch]

url:    https://github.com/intel-lab-lkp/linux/commits/Jens-Axboe/iov_iter-import-single-segments-iovecs-as-ITER_UBUF/20220618-221039
base:   https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git 4b35035bcf80ddb47c0112c4fbd84a63a2836a18
config: i386-randconfig-a002 (https://download.01.org/0day-ci/archive/20220619/[email protected]/config)
compiler: clang version 15.0.0 (https://github.com/llvm/llvm-project 91688716ba49942051dccdf7b9c4f81a7ec8feaf)

reproduce (this is a W=1 build):
wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross chmod +x ~/bin/make.cross
# https://github.com/intel-lab-lkp/linux/commit/ec74a6fdd818738cf8e0c0f10798fa001970bb81
git remote add linux-review https://github.com/intel-lab-lkp/linux
git fetch --no-tags linux-review Jens-Axboe/iov_iter-import-single-segments-iovecs-as-ITER_UBUF/20220618-221039 git checkout ec74a6fdd818738cf8e0c0f10798fa001970bb81
# save the config file
mkdir build_dir && cp config build_dir/.config
COMPILER_INSTALL_PATH=$HOME/0day COMPILER=clang make.cross W=1 O=build_dir ARCH=i386 SHELL=/bin/bash

If you fix the issue, kindly add following tag where applicable

All errors (new ones prefixed by >>):

>> lib/iov_iter.c:1950:9: error: call to undeclared function 'import_ubuf'; ISO C99 and later do not support implicit function declarations [-Wimplicit-function-declaration]
                   ret = import_ubuf(type, iovp[0]->iov_base, total_len, i);
                         ^
   lib/iov_iter.c:1950:9: note: did you mean 'import_uuid'?
   include/linux/uuid.h:71:20: note: 'import_uuid' declared here
   static inline void import_uuid(uuid_t *dst, const __u8 *src)
                      ^

1 error generated.

vim +/import_ubuf +1950 lib/iov_iter.c

1927
1928 ssize_t __import_iovec(int type, const struct iovec __user *uvec,
1929 unsigned nr_segs, unsigned fast_segs, struct iovec **iovp,
1930 struct iov_iter *i, bool compat)
1931 {
1932 ssize_t total_len = 0;
1933 unsigned long seg;
1934 struct iovec *iov;
1935
1936 iov = iovec_from_user(uvec, nr_segs, fast_segs, *iovp, compat);
1937 if (IS_ERR(iov)) {
1938 *iovp = NULL;
1939 return PTR_ERR(iov);
1940 }
1941
1942 /*
1943 * Fast path - single segment import. Use UBUF for these, rather
1944 * than setup an ITER_IOV.
1945 */
1946 if (nr_segs == 1) {
1947 ssize_t ret;
1948
1949 total_len = iovp[0]->iov_len;

> 1950			ret = import_ubuf(type, iovp[0]->iov_base, total_len, i);

1951 *iovp = NULL;
1952 if (unlikely(ret < 0))
1953 return ret;
1954 return total_len;
1955 }
1956
1957 /*
1958 * According to the Single Unix Specification we should return EINVAL if
1959 * an element length is < 0 when cast to ssize_t or if the total length
1960 * would overflow the ssize_t return value of the system call.
1961 *
1962 * Linux caps all read/write calls to MAX_RW_COUNT, and avoids the
1963 * overflow case.
1964 */
1965 for (seg = 0; seg < nr_segs; seg++) {
1966 ssize_t len = (ssize_t)iov[seg].iov_len;
1967
1968 if (!access_ok(iov[seg].iov_base, len)) {
1969 if (iov != *iovp)
1970 kfree(iov);
1971 *iovp = NULL;
1972 return -EFAULT;
1973 }
1974
1975 if (len > MAX_RW_COUNT - total_len) {
1976 len = MAX_RW_COUNT - total_len;
1977 iov[seg].iov_len = len;
1978 }
1979 total_len += len;
1980 }
1981
1982 iov_iter_init(i, type, iov, nr_segs, total_len);
1983 if (iov == *iovp)
1984 *iovp = NULL;
1985 else
1986 *iovp = iov;
1987 return total_len;
1988 }
1989

Signature · 2 lines
0-DAY CI Kernel Test Service
https://01.org/lkp