get_user_pages() pins in file mappings

This is a joint proposal with Dan Williams, John Hubbard, and Jérôme Glisse.

Last year we've talked with Dan about issues we have with filesystems and GUP [1]. The crux of the problem lies in the fact that there is no coordination (or even awareness) between filesystem working on a page (such as doing writeback) and GUP user modifying page contents and setting it dirty. This can (and we have user reports of this) lead to data corruption, kernel crashes, and other fun.

Since last year we have worked together on solving these problems and we have explored couple dead ends as well as hopefully found solutions to some of the partial problems. So I'd like to give some overview of where we stand and what remains to be solved and get thoughts from wider community about proposed solutions / problems to be solved.

In particular we hope to have reasonably robust mechanism of identifying pages pinned by GUP (patches will be posted soon) - I'd like to run that by MM folks (unless discussion happens on mailing lists before LSF/MM). We also have ideas how filesystems should react to pinned page in their writepages methods - there will be some changes needed in some filesystems to bounce the page if they need stable page contents. So I'd like to explain why we chose to do bouncing to fs people (i.e., why we cannot just wait, skip the page, do something else etc.) to save us from the same discussion with each fs separately and also hash out what the API for filesystems to do this should look like. Finally we plan to keep pinned page permanently dirty - again something I'd like to explain why we do this and gather input from other people.

This should be ideally shared MM + FS session.

[1] https://lwn.net/Articles/753027/

In particular we hope to have reasonably robust mechanism of identifying pages pinned by GUP (patches will be posted soon) - I'd like to run that by MM folks (unless discussion happens on mailing lists before LSF/MM). We also have ideas how filesystems should react to pinned page in their

writepages methods - there will be some changes needed in some filesystems to bounce the page if they need stable page contents. So I'd like to explain why we chose to do bouncing to fs people (i.e., why we cannot just wait, skip the page, do something else etc.) to save us from the same discussion with each fs separately and also hash out what the API for filesystems to do this should look like. Finally we plan to keep pinned page permanently dirty - again something I'd like to explain why we do this and gather input from other people.

Quoted 4 lines · Jan Kara

In particular we hope to have reasonably robust mechanism of identifying pages pinned by GUP (patches will be posted soon) - I'd like to run that by MM folks (unless discussion happens on mailing lists before LSF/MM). We also have ideas how filesystems should react to pinned page in their

Quoted 8 lines

writepages methods - there will be some changes needed in some filesystems to bounce the page if they need stable page contents. So I'd like to explain why we chose to do bouncing to fs people (i.e., why we cannot just wait, skip the page, do something else etc.) to save us from the same discussion with each fs separately and also hash out what the API for filesystems to do this should look like. Finally we plan to keep pinned page permanently dirty - again something I'd like to explain why we do this and gather input from other people.

Hi Jan,

Say, I was just talking through this point with someone on our driver team, and suddenly realized that I'm now slightly confused on one point. If we end up keeping the gup-pinned pages effectively permanently dirty while pinned, then maybe the call sites no longer need to specify "dirty" (or not) when they call put_user_page*()?

In other words, the RFC [1] has this API:

    void put_user_page(struct page *page);
    void put_user_pages_dirty(struct page **pages, unsigned long npages);
    void put_user_pages_dirty_lock(struct page **pages, unsigned long npages);
    void put_user_pages(struct page **pages, unsigned long npages);

But maybe we only really need this:

    void put_user_page(struct page *page);
    void put_user_pages(struct page **pages, unsigned long npages);

?

[1] https://lkml.kernel.org/r/[email protected]

Hi John,

Quoted 30 lines · John Hubbard

Quoted 4 lines · Jan Kara

In particular we hope to have reasonably robust mechanism of identifying pages pinned by GUP (patches will be posted soon) - I'd like to run that by MM folks (unless discussion happens on mailing lists before LSF/MM). We also have ideas how filesystems should react to pinned page in their

Quoted 8 lines

writepages methods - there will be some changes needed in some filesystems to bounce the page if they need stable page contents. So I'd like to explain why we chose to do bouncing to fs people (i.e., why we cannot just wait, skip the page, do something else etc.) to save us from the same discussion with each fs separately and also hash out what the API for filesystems to do this should look like. Finally we plan to keep pinned page permanently dirty - again something I'd like to explain why we do this and gather input from other people.

Hi Jan,

Say, I was just talking through this point with someone on our driver team, and suddenly realized that I'm now slightly confused on one point. If we end up keeping the gup-pinned pages effectively permanently dirty while pinned, then maybe the call sites no longer need to specify "dirty" (or not) when they call put_user_page*()?

In other words, the RFC [1] has this API:

    void put_user_page(struct page *page);
    void put_user_pages_dirty(struct page **pages, unsigned long npages);
    void put_user_pages_dirty_lock(struct page **pages, unsigned long npages);
    void put_user_pages(struct page **pages, unsigned long npages);

Quoted 8 lines

But maybe we only really need this:

    void put_user_page(struct page *page);
    void put_user_pages(struct page **pages, unsigned long npages);

?

[1] https://lkml.kernel.org/r/[email protected]

So you are right that if we keep gup-pinned pages dirty, drivers could get away without marking them as such. However I view "keep pages dirty" as an implementation detail, rather than a promise of the API. So I'd like to leave us the flexibility of choosing a different implementation in the future. And as such I'd just leave the put_user_pages_dirty() variants in place.

Honza