topic/iomem-mmap-vs-gup

Hi Linus,

It's still the same topic branch as last merge window, but the name isn't fitting all that well anymore :-)

Anyway here's a small pull for you to ponder, now that the big ones are all through. It's been in -next almost the entire cycle, I've only done some non-code rebases due to the -rc1 fumble and to fix some commit message typos.

Christoph Hellwig also looked at these and aside from wanting to outright remove it all didn't have objections.

topic/iomem-mmap-vs-gup-2021-05-03:
unexport follow_pfn

Follow-up to my pull from last merge window: kvm and vfio lost their very unsafe use of follow_pfn, this appropriately marks up the very last user for some userptr-as-buffer use-cases in media. There was some resistance to outright removing it, maybe we can do this in a few releases.

Cheers, Daniel

The following changes since commit 0d02ec6b3136c73c09e7859f0d0e4e2c4c07b49b:

Linux 5.12-rc4 (2021-03-21 14:56:43 -0700)

are available in the Git repository at:

git://anongit.freedesktop.org/drm/drm tags/topic/iomem-mmap-vs-gup-2021-05-03

for you to fetch changes up to ac8b8400620a4b0d9ca903ee9ad440bec736f5fa:

mm: unexport follow_pfn (2021-04-08 16:54:38 +0200)

----------------------------------------------------------------
unexport follow_pfn

Follow-up to my pull from last merge window: kvm and vfio lost their very unsafe use of follow_pfn, this appropriately marks up the very last user for some userptr-as-buffer use-cases in media. There was some resistance to outright removing it, maybe we can do this in a few releases.

----------------------------------------------------------------
Daniel Vetter (3):
mm: Add unsafe_follow_pfn media/videobuf1|2: Mark follow_pfn usage as unsafe
mm: unexport follow_pfn

 drivers/media/common/videobuf2/frame_vector.c |  2 +-
 drivers/media/v4l2-core/videobuf-dma-contig.c |  2 +-
 include/linux/mm.h                            |  4 +--
 mm/memory.c                                   | 46 +++++++++++++++++----------
 mm/nommu.c                                    | 28 ++++++++++++----
 security/Kconfig                              | 13 ++++++++
 6 files changed, 68 insertions(+), 27 deletions(-)

Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch
_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel

[ You had a really odd Reply-to on this one ]

Well, _now_ I'm all caught up. Knock wood. Anyway, time to look at it:

Hmm. So this looks mostly ok to me, although I think the change to the nommu case is pretty ridiculous.

On nommu, unsafe_follow_pfn() should just be a wrapper around follow_pfn(). There's no races when you can't remap anything. No?

Do the two media cases even work on nommu?

Finally - did you intend fo this to be a real pull request? Because the email read to me like "think about this and tell me what you think" rather than "please pull"..

And I have now fulfilled that "think about and tell me" part ;)

Linus

So personally I think the entire thing should just be thrown out, it's all levels of scary and we have zero-copy buffer sharing done properly with dma-buf since years in v4l.

Iirc I've had that in some early versions of all this, but got nacked by some, supported by others from media as something that needs to go away. This here is now the next best thing as a fishing expedition to figure out whether there's actually anyone left who cares or not.

That's also why the nommu case has the same checks, even though it's all fine there.

Hopefully the answer is "no users" and then we could remove this in a year or two.

Ah yes I rushed this a bit between appreciating some local fires here at work and left out the instructions :-)

Please pull or tell me whether you want the outright removal (like Christoph Hellwig also wants).

Cheers, Daniel

Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch

[ Daniel, please fix your broken email setup. You have this insane "Reply-to" list that just duplicates all the participants. Very broken, very annoying ]

So I've been looking at this more, and the more I look at it, the less I like this series.

I think the proper fix is to just fix things.

For example, I'm looking at the v4l users of follow_pfn(), and I find get_vaddr_frames(), which is just broken.

Fine, we know users are broken, but look at what appears to be the main user of get_vaddr_frames(): vb2_dc_get_userptr().

What does that function do? Immediately after doing get_vaddr_frames(), it tries to turn those pfn's into page pointers, and then do sg_alloc_table_from_pages() on the end result.

Yes, yes, it also has that "ok, that failed, let's try to see if it's some physically contiguous mapping" and do DMA directly to those physical pages, but the point there is that that only happens when they weren't normal pages to begin with.

So thew *fix* for at least that path is to

(a) just use the regular pin_user_pages() for normal pages

(b) perhaps keep the follow_pfn() case, but then limit it to that "no
page backing" and that physical pages case.

And honestly, the "struct frame_vector" thing already *has* support for this, and the problem is simply that the v4l code has decided to have the callers ask for pfn's rather than have the callers just ask for a frame-vector that is either "pfn's with no paeg backing" _or_ "page list with proper page reference counting".

So this series of yours that just disables follow_pfn() actually seems very wrong.

I think follow_pfn() is ok for the actual "this is not a 'struct page' backed area", and disabling that case is wrong even going forward.

End result, I think the proper model is:

- keep follow_pfn(), but limit it to the "not vm_normal_page()" case,
and return error for some real page mapping

- make the get_vaddr_frames() first try "pin_user_pages()" (and
create a page array) and fall back to "follow_pfn()" if that fails (or the other way around). Set the

IOW, get_vaddr_frames() would just do

        vec->got_ref = is_pages;
        vec->is_pfns = !is_pages;

and everything would just work out - the v4l code seems to already have all the support for "it's a ofn array" vs "it's properly refcounted pages".

So the only case we should disallow is the mixed case, that the v4l code already seems to not be able to handle anyway (and honestly, it looks like "got_ref/is_pfns" should be just one flag - they always have to have the opposite values).

So I think this "unsafe_follow_pfn()" halfway step is actively wrong. It doesn't move us forward. Quite the reverse. It just makes the proper fix harder.

End result: not pulling it, unless somebody can explain to me in small words why I'm wrong and have the mental capacity of a damaged rodent.

Linus

Yup, the "rip it all out" solution amounts to replacing this all, including frame_vector helper code, with pin_user_pages.

I think this is where you miss a bit: We very much also want to stop pinned userptr to physcial addresses that aren't page backed. This might very well be some gpu pci bar, backed by vram, and vram is managed as dynamically as struct page backed stuff (and there's all the hmm dreams to make it actually use struct page, but that's another story).

So by the time the media hw access that vb2 userptr buffer there's good chances someone else's data is now there. If vb2 would have a mmu_notifier subscription or similar to follow pte updates the gpu driver does, then it would be all fine. But this vb2 model is a pinned one, hence not fixable.

The other more practical issue is that peer2peer dma on modern hw needs quite some setup. Just taking a cpu pfn and hoping that matches the bus addr your device would need is a bit optimistic.

One theoretical & proper fix I discussed with Jason Gunthrope would be to replace the pfn lookup with a lookup for a struct dma_buf. Which has proper interfaces for pinning gpu buffers, figuring out p2p dma or just figuring out the right dma mapping and all that. Idea was to make a direct vma->dma_buf lookup or something like that. But consensus is also that outside of gpus and very closely related things using dma_buf is not a great idea, because there's a few too many silly rules involved. For everyone else it's better to make the struct page managed device memory stuff work most likely.

No rodents I think, just more backstory of how this all fits. tldr; pin_user_pages is the only safe use of this vb2 userptr thing. -Daniel

Cheers, Daniel
--
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch

Yes, which is why I advocate for just ripping the follow_pfn path out entirely. It could have been used for crazy ad dangerous peer to peer transfers outside of any infrastructure making it safe, or for pre-CMA kernel memory carveouts for lage contiguous memory allocations (which are pretty broken by design as well). So IMHO the only sensible thing is to remove this cruft entirely, and if it breaks a currently working setup (which I think is unlikely) we'll have to make sure it can work the proper way.

Since I'm not getting any cozy consenus vibes here on any option I think I'll just drop this.

Stephen, can you pls drop

git://anongit.freedesktop.org/drm/drm topic/iomem-mmap-vs-gup

from linux-next? It's not going anywhere. I'll also go ahead and delete the branch, to make sure you catch this update :-)

Thanks, Daniel

Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch

Hi Daniel,

I have dropped this now. Thanks for letting me know.

Cheers,
Stephen Rothwell

Every place we've audited using follow_pfn() has been shown to have some use-after-free bugs like Daniel describes, and a failure to check permissions bug too.

All the other follow_pfn() users were moved to follow_pte() to fix the permissions check and this shifts the use-after-free bug away from being inside an MM API and into the caller mis-using the API by, say, extracting and using the PFN outside the pte lock.

eg look at how VFIO wrongly uses follow_pte():

static int follow_fault_pfn()

        ret = follow_pte(vma->vm_mm, vaddr, &ptep, &ptl);
                *pfn = pte_pfn(*ptep);
        pte_unmap_unlock(ptep, ptl);

        // no protection that pte_pfn() is still valid!
        use_pfn(*pfn)

v4l is the only user that still has the missing permissions check security bug too - so there is no outcome that should keep follow_pfn() in the tree.

At worst v4l should change to follow_pte() and use it wrongly like VFIO. At best we should delete all the v4l stuff.

Daniel I suppose we missed this relation to follow_pte(), so I agree that keeping a unsafe_follow_pfn() around is not good.

Regards,
Jason

yeah vfio is still broken for the case I care about. I think there's also some questions open still about whether kvm really uses mmu_notifier in all cases correctly, but iirc the one exception was s390, which didn't have pci mmap and that's how it gets away with that specific problem.

tbh I never really got the additional issue with the missing write checks. That users of follow_pfn (or well follow_pte + immediate lock dropping like vfio) don't subscribe to the pte updates in general is the bug I'm seeing. That v4l also glosses over the read/write access stuff is kinda just the icing on the cake :-) It's pretty well broken even if it would check that.
-Daniel

Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch

IIRC kvm doesn't either.

It is just severity. Exploiting the use after free bug is somewhat harder, exploiting the 'you can write to non-page write protected memory' bug is not so hard.

Jason

+Paolo

Yep, KVM on x86 has a non-trivial number of flows that don't properly hook into the mmu_notifier. Paolo is working on fixing the problem, but I believe the rework won't be ready until 5.14.

Yeah, I like the way it's coming, but I'm at 20-ish patches and counting.

Paolo