Skip to contentSkip to conversation

vfs fix

GIT PULL Al Viro 3 participants 7 messages Nov 27, 2016 – Nov 28, 2016 VFS

lore
Share & tools
lore

Al Viro

author maintainer
#0 lore raw

The following changes since commit 3ad0e83cf86bcaeb6ca3c37060a3ce866b25fb42:

Merge branch 'parisc-4.9-4' of git://git.kernel.org/pub/scm/linux/kernel/git/deller/parisc-linux (2016-11-25 16:47:15 -0800)

are available in the git repository at:

git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs.git for-linus

for you to fetch changes up to 8e54cadab447dae779f80f79c87cbeaea9594f60:

fix default_file_splice_read() (2016-11-26 20:05:42 -0500)

----------------------------------------------------------------
Al Viro (1):
fix default_file_splice_read()

 fs/splice.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
#1 lore raw
Quoted 3 lines · Al Viro

Al Viro (1):
fix default_file_splice_read()

Ugh. I absolutely _hate_ this:

        BUG_ON(dummy);

because it makes no sense.

I'm assuming that "dummy" here is "start_offset", and that you want to make sure that there are no initial offsets that would affect the nrpages calculation.

But dammit, if so, just *call* it "start_offset", not "dummy". A dummy value is just a place-holder, it makes no sense to have BUG_ON() on such a value.

So adding random BUG_ON() statements is evil to begin with, but when you do it on something that is mis-named and makes no sense, that's just wrong.

I'm further assuming that the reason we can do that is because "iov_iter_pipe()" has set iov_offset to zero, and as a result we end up havin g

iov_iter_get_pages_alloc() ->
pipe_get_pages_alloc() ->
data_start() will set *offp to zero.

but quite frankly, you can not tell that from the code itself, which makes no sense. You have to go digging.

I was hoping the splice code would become more readable, not filled with more crazy nonsensical code.

So I've pulled this, but _please_:

- rename "dummy" (which isn't dummy at all now that you *do* things
to it!) to something sane.

Like perhaps 'pg_offset' or 'iter_offset' or something.

- Does the "BUG_ON()" really make sense? If the issue is that you
didn't use the offset in calculations, maybe you should just do so, ie instead of

        BUG_ON(dummy);
        nr_pages = DIV_ROUND_UP(res, PAGE_SIZE);

just do

        nr_pages = DIV_ROUND_UP(res + iter_offset, PAGE_SIZE);

or something? Even if "iter_offset" ends up always being zero, why is that worthy of a BUG_ON()? The BUG_ON() is more expensive than just doing the natural math..

That's what all the other users do, and that's what should be the "right usage pattern", afaik. The number of pages really *is* calculated as

       int n = DIV_ROUND_UP(result + offs, PAGE_SIZE);

in other iov_iter_get_pages_alloc() callers, although tghe nfs code open-codes it as

        npages = (result + pgbase + PAGE_SIZE - 1) / PAGE_SIZE;

so it's not a very strong pattern.

Linus

author maintainer
#2 lore raw
Quoted 4 lines · Linus Torvalds

That's what all the other users do, and that's what should be the "right usage pattern", afaik. The number of pages really *is* calculated as

Quoted 8 lines
       int n = DIV_ROUND_UP(result + offs, PAGE_SIZE);

in other iov_iter_get_pages_alloc() callers, although tghe nfs code open-codes it as

        npages = (result + pgbase + PAGE_SIZE - 1) / PAGE_SIZE;

so it's not a very strong pattern.

Two issues here. One is that iov_iter_get_pages{,_alloc}() calling conventions are fucking ugly. I'm guilty of that atrocity; my only excuse is that this thing has congealed from many open-coded instances, quite a few of those appearing only after considerable massage of the code. I _hate_ the boilerplate we have in the functions implementing those for various iov_iter flavours and boilerplate in the callers.

I am going to try and come up with something less atrocious. As it is, renaming that variable and adding it to the return value of iov_iter_get_pages_alloc() is certainly not a problem and would be prettier, but TBH I just went "yet another place to go into that cleanup". Shouldn't have.

Another thing is that it was a leftover from "Alexei, could you see if that thing fixes your reproducer?" - just in case the things _really_ went insane and it was not a wrong rounding but somehow completely buggered pipe_get_pages_alloc(). They hadn't.

Anyway, leaving that BUG_ON() had been wrong; I can send a followup massaging that thing as you've suggested, if you are interested in that. But keep in mind that the whole iov_iter_get_pages...() calling conventions are going to be changed, hopefully soon.

author maintainer
#3 lore raw
Quoted 4 lines · Al Viro

Anyway, leaving that BUG_ON() had been wrong; I can send a followup massaging that thing as you've suggested, if you are interested in that. But keep in mind that the whole iov_iter_get_pages...() calling conventions are going to be changed, hopefully soon.

BTW, speaking of iov_iter_get_pages_alloc() callers and splice:
"ceph: combine as many iovec as possile into one OSD request" has added static size_t dio_get_pagev_size(const struct iov_iter *it)

{
    const struct iovec *iov = it->iov;
    const struct iovec *iovend = iov + it->nr_segs;
    size_t size;

    size = iov->iov_len - it->iov_offset;
    /*
     * An iov can be page vectored when both the current tail
     * and the next base are page aligned.
     */
    while (PAGE_ALIGNED((iov->iov_base + iov->iov_len)) &&
           (++iov < iovend && PAGE_ALIGNED((iov->iov_base)))) {
        size += iov->iov_len;
    }
    dout("dio_get_pagevlen len = %zu\n", size);
    return size;
}

... with 'it' possibly being bio_vec-backed iterator. Could somebody explain what that code is trying to do?

I would really like to hear details - I'm not saying that the goal mentioned in the commit message is worthless, but I don't know ceph codebase well enough to tell what would be a good way to implement that. In its current form it's obviously broken.

author maintainer
#4 lore raw

On Sun, Nov 27, 2016 at 02:51:18AM +0000, Al Viro wrote:

... and screwed up Cc due to comma in S-O-B of commit in question. My apologies.

#5 lore raw
In reply to Al Viro
Quoted 22 lines · Al Viro
Quoted 4 lines · Al Viro

Anyway, leaving that BUG_ON() had been wrong; I can send a followup massaging that thing as you've suggested, if you are interested in that. But keep in mind that the whole iov_iter_get_pages...() calling conventions are going to be changed, hopefully soon.

BTW, speaking of iov_iter_get_pages_alloc() callers and splice:
"ceph: combine as many iovec as possile into one OSD request" has added static size_t dio_get_pagev_size(const struct iov_iter *it)

{
   const struct iovec *iov = it->iov;
   const struct iovec *iovend = iov + it->nr_segs;
   size_t size;

   size = iov->iov_len - it->iov_offset;
   /*
    * An iov can be page vectored when both the current tail
    * and the next base are page aligned.
Quoted 8 lines
    */
   while (PAGE_ALIGNED((iov->iov_base + iov->iov_len)) &&
          (++iov < iovend && PAGE_ALIGNED((iov->iov_base)))) {
       size += iov->iov_len;
   }
   dout("dio_get_pagevlen len = %zu\n", size);
   return size;
}

The read/write interface for ceph cluster accepts page vector. But we can only specify offset for the first page. Except the first and last pages, all other pages must be full size. This function finds size of pages that satisfy this requirement

Regards
Yan, Zheng

Quoted 8 lines

... with 'it' possibly being bio_vec-backed iterator. Could somebody explain what that code is trying to do?

I would really like to hear details - I'm not saying that the goal mentioned in the commit message is worthless, but I don't know ceph codebase well enough to tell what would be a good way to implement that. In its current form it's obviously broken.

#6 lore raw
In reply to Al Viro
Quoted 3 lines · Al Viro

Two issues here. One is that iov_iter_get_pages{,_alloc}() calling conventions are fucking ugly.

No arguments there. I was going to suggest adding an "int *npages" argument and letting that function fill that in together with the page array, but it's not like that is going to really make the interface any prettier.

Quoted 4 lines

Anyway, leaving that BUG_ON() had been wrong; I can send a followup massaging that thing as you've suggested, if you are interested in that. But keep in mind that the whole iov_iter_get_pages...() calling conventions are going to be changed, hopefully soon.

If it indeed happens soon, I can certainly live with the existing code as a temporary "ugly but it works". But if it ends up being post-4.10 I'd prefer to clean this up independently first, ok?

Linus