what's in nvdimm.git for v4.4?
Subsystems (3)
Here is a status summary of the topic-branches nvdimm.git is tracking for v4.4. Unless indicated these branches are not present in -next. Please ACK, NAK, or ask for a re-post of any of the below to disposition it for the merge window.
===
for-4.4/dax-fixes:
===
Core DAX and XFS fixes for DAX locking. They are carried in nvdimm.git as a dependency of the for-4.4/dax-gup branch.
Dan Williams (1):
pmem, dax: clean up clear_pmem()
Dave Chinner (5):
xfs: fix inode size update overflow in xfs_map_direct()
xfs: introduce BMAPI_ZERO for allocating zeroed extents
xfs: Don't use unwritten extents for DAX
xfs: DAX does not use IO completion callbacks
xfs: add ->pfn_mkwrite support for DAX
Ross Zwisler (2):
dax: dax_pfn_mkwrite() truncate race check
ext2: Add locking for DAX faults
===
for-4.4/numa:
===
Minor api cleanups that have been acked and published in -next for a few weeks.
Dan Williams (7):
x86, mm: quiet arch_add_memory()
pmem: kill memremap_pmem()
devm_memunmap: use devres_release()
devm_memremap: convert to return ERR_PTR
devm: make allocations numa aware by default
devm_memremap_pages: use numa_mem_id pmem, memremap: convert to numa aware allocations
===
for-4.4/dax-gup: get_user_pages() support for dax mappings
https://lists.01.org/pipermail/linux-nvdimm/2015-October/002387.html
===
Needs acks from core -mm folks particularly for:
x86, mm: introduce vmem_altmap to augment vmemmap_populate()
mm, x86: get_user_pages() for dax mappings
Dan Williams (22):
block: generic request_queue reference counting
dax: increase granularity of dax_clear_blocks() operations block, dax: fix lifetime of in-kernel dax mappings with dax_map_atomic()
mm: introduce __get_dev_pagemap()
x86, mm: introduce vmem_altmap to augment vmemmap_populate() libnvdimm, pfn, pmem: allocate memmap array in persistent memory
avr32: convert to asm-generic/memory_model.h
hugetlb: fix compile error on tile
frv: fix compiler warning from definition of __pmd()
um: kill pfn_t
kvm: rename pfn_t to kvm_pfn_t
mips: fix PAGE_MASK definition
mm, dax, pmem: introduce pfn_t
mm, dax, gpu: convert vm_insert_mixed to pfn_t, introduce _PAGE_DEVMAP mm, dax: convert vmf_insert_pfn_pmd() to pfn_t
list: introduce list_del_poison()
mm, dax, pmem: introduce {get|put}_dev_pagemap() for dax-gup
block: notify queue death confirmation
mm, pmem: devm_memunmap_pages(), truncate and unmap ZONE_DEVICE pages mm, x86: get_user_pages() for dax mappings
block: introduce file_bd_inode()
block: enable dax for raw block devices
===
for-4.4/dax-coredump
===
Recently acked by Jeff will appear in -next shortly.
Ross Zwisler (2):
coredump: add DAX filtering for ELF coredumps
coredump: add DAX filtering for FDPIC ELF coredumps
===
for-4.4/memremap
https://lwn.net/Articles/653585/
===
Some patches out of this series have started to leak into maintainer trees, but the uptake is fairly slow. I may just send a branch to Linus with the stragglers towards the end of the merge window.
Dan Williams (21):
x86: introduce arch_memremap()
arm: introduce arch_memremap()
ia64: introduce arch_memremap()
sh: introduce arch_memremap()
m68k: introduce arch_memremap()
arm: switch from ioremap_cache to memremap
x86: switch from ioremap_cache to memremap
gma500: switch from acpi_os_ioremap to memremap
i915: switch from acpi_os_ioremap to memremap drm/vmwgfx: switch from ioremap_cache to memremap
acpi: switch from ioremap_cache to memremap sound, skylake: switch from ioremap_cache to memremap
memconsole: fix __iomem mishandling, switch to memremap
intel-iommu: switch from ioremap_cache to memremap
pxa2xx-flash: switch from ioremap_cache to memremap
sfi: switch from ioremap_cache to memremap
fbdev: switch from ioremap_wt to memremap
arch: kill ioremap_cached()
arch: kill ioremap_fullcache()
arch: remove ioremap_cache, replace with arch_memremap
arch: remove ioremap_wt, optionally replace with
arch_memremap
===
for-4.4/blk-integrity:
===
Jens? I've folded my fixes with Martin's latest and block.git/for-4.4/drivers.
Dan Williams (7):
md, dm, scsi, nvme, libnvdimm: drop blk_integrity_unregister() at shutdown
md: suspend i/o during runtime blk_integrity_unregister
nvme: suspend i/o during runtime blk_integrity_unregister
block: generic request_queue reference counting
block: move blk_integrity to request_queue
block: blk_flush_integrity() for bio-based drivers
block, libnvdimm, nvme: provide a built-in blk_integrity nop profile
Martin K. Petersen (5):
block: Move integrity kobject to struct gendisk
block: Consolidate static integrity profile properties
block: Reduce the size of struct blk_integrity
block: Export integrity data interval size in sysfs
block: Inline blk_integrity in struct gendisk
===
for-4.4/hotplug
===
Changes requested for v3, but once those are addressed just need an ack from Rafael.
Vishal Verma (2):
nfit: in acpi_nfit_init, break on a 0-length table
acpi: nfit: Add support for hot-add
===
for-4.4/documentation:
===
In -next...
Konrad Rzeszutek Wilk (1):
libnvdimm: documentation clarifications
Quoted 8 lines · Dan J
Here is a status summary of the topic-branches nvdimm.git is tracking for v4.4. Unless indicated these branches are not present in -next. Please ACK, NAK, or ask for a re-post of any of the below to disposition it for the merge window.
===
for-4.4/dax-fixes:
===
...
Quoted 6 lines
Dave Chinner (5):
xfs: fix inode size update overflow inxfs_map_direct()
xfs: introduceBMAPI_ZEROfor allocating zeroed extents
xfs: Don't use unwritten extents for DAX
xfs: DAX does not use IO completion callbacks
xfs: add ->pfn_mkwrite support for DAX
Please drop these. They have not been reviewed yet, and because the changes affect more than just DAX (core XFS allocator functionality was changed) these need to go through the XFS tree.
Cheers,
Dave.
Signature · 2 lines
Dave Chinner [email protected]
Quoted 17 lines · Dave Chinner
Quoted 8 lines · Dan J
Here is a status summary of the topic-branches nvdimm.git is tracking for v4.4. Unless indicated these branches are not present in -next. Please ACK, NAK, or ask for a re-post of any of the below to disposition it for the merge window.
===
for-4.4/dax-fixes:
===...
Quoted 6 lines
Dave Chinner (5):
xfs: fix inode size update overflow inxfs_map_direct()
xfs: introduceBMAPI_ZEROfor allocating zeroed extents
xfs: Don't use unwritten extents for DAX
xfs: DAX does not use IO completion callbacks
xfs: add ->pfn_mkwrite support for DAX
Quoted 4 lines
Please drop these. They have not been reviewed yet, and because the changes affect more than just DAX (core XFS allocator functionality was changed) these need to go through the XFS tree.
Ok, thanks for the heads up. For the get_user_pages() patches that build on these fixes I'm assuming your review bandwidth is in short supply to also give an XFS sign-off on those changes for 4.4?
I'm wondering if we can take a conservative step forward with those patches for 4.4. if XFS and EXT4 interactions need more time to get worked out, which I believe they do, I can conceive just turning on get_user_pages() support for DAX-mappings of the raw block device. This would be via the new facility I posted yesterday:
https://lists.01.org/pipermail/linux-nvdimm/2015-October/002512.html.
While not very functional for applications it makes testing base DAX mechanisms straightforward.
Quoted 18 lines · Dan Williams
Quoted 17 lines · Dave Chinner
Quoted 8 lines · Dan J
Here is a status summary of the topic-branches nvdimm.git is tracking for v4.4. Unless indicated these branches are not present in -next. Please ACK, NAK, or ask for a re-post of any of the below to disposition it for the merge window.
===
for-4.4/dax-fixes:
===...
Quoted 6 lines
Dave Chinner (5):
xfs: fix inode size update overflow inxfs_map_direct()
xfs: introduceBMAPI_ZEROfor allocating zeroed extents
xfs: Don't use unwritten extents for DAX
xfs: DAX does not use IO completion callbacks
xfs: add ->pfn_mkwrite support for DAX
Quoted 8 lines
Quoted 4 lines
Please drop these. They have not been reviewed yet, and because the changes affect more than just DAX (core XFS allocator functionality was changed) these need to go through the XFS tree.
Ok, thanks for the heads up. For the
get_user_pages()patches that build on these fixes I'm assuming your review bandwidth is in short supply to also give an XFS sign-off on those changes for 4.4?
I'm not aware of any other patches that touch XFS. AFAIA, you haven't cc'd anything to [email protected], so it's not on my radar...
Quoted 4 lines
I'm wondering if we can take a conservative step forward with those patches for 4.4. if XFS and EXT4 interactions need more time to get worked out, which I believe they do, I can conceive just turning on
get_user_pages()support for DAX-mappings of the raw block device.
Regardless of the ext4/XFS status, isn't it a bit late to be proposing brand new stuff that nobody has had time to think about for the next merge window?
Cheers,
Dave.
Signature · 2 lines
Dave Chinner [email protected]
Quoted 24 lines · Dave Chinner
Quoted 23 lines · Dan Williams
Quoted 21 lines · Dave Chinner
Quoted 8 lines · Dan J
Here is a status summary of the topic-branches nvdimm.git is tracking for v4.4. Unless indicated these branches are not present in -next. Please ACK, NAK, or ask for a re-post of any of the below to disposition it for the merge window.
===
for-4.4/dax-fixes:
===...
Quoted 6 lines
Dave Chinner (5):
xfs: fix inode size update overflow inxfs_map_direct()
xfs: introduceBMAPI_ZEROfor allocating zeroed extents
xfs: Don't use unwritten extents for DAX
xfs: DAX does not use IO completion callbacks
xfs: add ->pfn_mkwrite support for DAXPlease drop these. They have not been reviewed yet, and because the changes affect more than just DAX (core XFS allocator functionality was changed) these need to go through the XFS tree.
Quoted 7 lines
Quoted 3 lines
Ok, thanks for the heads up. For the
get_user_pages()patches that build on these fixes I'm assuming your review bandwidth is in short supply to also give an XFS sign-off on those changes for 4.4?I'm not aware of any other patches that touch XFS. AFAIA, you haven't cc'd anything to [email protected], so it's not on my radar...
I can cc [email protected] on fs/dax.c changes going forward, but for these I figure it was off topic since nothing touched fs/xfs/.
Quoted 9 lines
Quoted 4 lines
I'm wondering if we can take a conservative step forward with those patches for 4.4. if XFS and EXT4 interactions need more time to get worked out, which I believe they do, I can conceive just turning on
get_user_pages()support for DAX-mappings of the raw block device.Regardless of the ext4/XFS status, isn't it a bit late to be proposing brand new stuff that nobody has had time to think about for the next merge window?
The "dax-for-raw-block support" is indeed new, but it's a fairly straightforward extension of these patches that have been out for review since 4.3-rc2, or 4.1-rc6 in the case of the pfn_t enabling. Cutting out the filesystem interactions makes it that much simpler to comprehend.
Jan Kara
reviewerSorry for replying to this email and not to patch posting directly but I didn't find the original mail in any of my mailboxes...
Quoted 27 lines · Dan Williams
Quoted 17 lines · Dave Chinner
Quoted 8 lines · Dan J
Here is a status summary of the topic-branches nvdimm.git is tracking for v4.4. Unless indicated these branches are not present in -next. Please ACK, NAK, or ask for a re-post of any of the below to disposition it for the merge window.
===
for-4.4/dax-fixes:
===...
Quoted 6 lines
Dave Chinner (5):
xfs: fix inode size update overflow inxfs_map_direct()
xfs: introduceBMAPI_ZEROfor allocating zeroed extents
xfs: Don't use unwritten extents for DAX
xfs: DAX does not use IO completion callbacks
xfs: add ->pfn_mkwrite support for DAXQuoted 4 lines
Please drop these. They have not been reviewed yet, and because the changes affect more than just DAX (core XFS allocator functionality was changed) these need to go through the XFS tree.
Ok, thanks for the heads up. For the
get_user_pages()patches that build on these fixes I'm assuming your review bandwidth is in short supply to also give an XFS sign-off on those changes for 4.4?
Quoted 8 lines
I'm wondering if we can take a conservative step forward with those patches for 4.4. if XFS and EXT4 interactions need more time to get worked out, which I believe they do, I can conceive just turning on
get_user_pages()support for DAX-mappings of the raw block device. This would be via the new facility I posted yesterday:
https://lists.01.org/pipermail/linux-nvdimm/2015-October/002512.html.
While not very functional for applications it makes testing base DAX mechanisms straightforward.
I had a look at the patch and I miss one thing: Why do we need bd_mutex to protect faults? I see a comment there:
/* check that the faulting page hasn't raced with bdev resize */
Is it really possible that bdev gets shrunk under us? Hum, looking into fs/block_dev.c, probably it is. But there are other places - like DIO path
- assuming that block device mapping cannot just disappear from under us. I wonder how that would cope with bdev size change...
Also we only call invalidate_bdev() to invalidate page cache pages of the bdev after resize which specifically skips any mmaped pages so bdev resizing in presence of mmap is unreliable to say the least.
Anyway, bd_mutex seems like a big hammer in the fast path to protect against rare size changes. Also nesting of bd_mutex under mmap_sem makes me somewhat uneasy (I'd definitely wonder whether lockdep would not complain about that)...
Honza
Signature · 2 lines
Jan Kara <[email protected]> SUSE Labs, CR
Quoted 2 lines · Jan Kara
Sorry for replying to this email and not to patch posting directly but I didn't find the original mail in any of my mailboxes...
Forgive me for not including you on the initial posting. I'll add you to that series going forward.
Quoted 37 lines · Dan Williams
Quoted 27 lines · Dave Chinner
Quoted 17 lines · Dan J
Quoted 8 lines
Here is a status summary of the topic-branches nvdimm.git is tracking for v4.4. Unless indicated these branches are not present in -next. Please ACK, NAK, or ask for a re-post of any of the below to disposition it for the merge window.
===
for-4.4/dax-fixes:
===...
Quoted 6 lines
Dave Chinner (5):
xfs: fix inode size update overflow inxfs_map_direct()
xfs: introduceBMAPI_ZEROfor allocating zeroed extents
xfs: Don't use unwritten extents for DAX
xfs: DAX does not use IO completion callbacks
xfs: add ->pfn_mkwrite support for DAXQuoted 4 lines
Please drop these. They have not been reviewed yet, and because the changes affect more than just DAX (core XFS allocator functionality was changed) these need to go through the XFS tree.
Ok, thanks for the heads up. For the
get_user_pages()patches that build on these fixes I'm assuming your review bandwidth is in short supply to also give an XFS sign-off on those changes for 4.4?Quoted 8 lines
I'm wondering if we can take a conservative step forward with those patches for 4.4. if XFS and EXT4 interactions need more time to get worked out, which I believe they do, I can conceive just turning on
get_user_pages()support for DAX-mappings of the raw block device. This would be via the new facility I posted yesterday:
https://lists.01.org/pipermail/linux-nvdimm/2015-October/002512.html.
While not very functional for applications it makes testing base DAX mechanisms straightforward.
Quoted 2 lines
I had a look at the patch and I miss one thing: Why do we need
bd_mutexto protect faults?
This is my main question for the RFC, I landed on bd_mutex as a big hammer primarily because I don't expect more than a handful of applications to be in a position to map a block device directly. Aside from needing admin privileges the application must also be satisfied with only having partitions to sub-divide the capacity.
As far as I can see, a hypervisor is the only application that is compatible with the above constraints. It establishes a few mappings that exist for the lifetime of the guest and ends up with little to no contention on an mmap lock.
Quoted 4 lines
I see a comment there:
/* check that the faulting page hasn't raced with bdev resize */
Quoted 8 lines
Is it really possible that bdev gets shrunk under us? Hum, looking into
fs/block_dev.c, probably it is. But there are other places - like DIO path
- assuming that block device mapping cannot just disappear from under us. I wonder how that would cope with bdev size change...Also we only call
invalidate_bdev()to invalidate page cache pages of the bdev after resize which specifically skips any mmaped pages so bdev resizing in presence of mmap is unreliable to say the least.
True, new locking against resize just for mmap is hard to justify when there's no protection in other paths. Perhaps it is a case of userspace getting to keep the pieces if it decides to race resize vs i/o?
At least with the pmem driver there's no risk that i/o will clobber random memory addresses past the end of the whole-disk-device since only partitions can be resized at run time. The base pmem device can only be resized while offline.
Quoted 4 lines
Anyway,
bd_mutexseems like a big hammer in the fast path to protect against rare size changes. Also nesting ofbd_mutexundermmap_semmakes me somewhat uneasy (I'd definitely wonder whether lockdep would not complain about that)...
Lockdep hasn't triggered to date, but I've only tried mapping the base block device with no filesystem present. I'll try a test that sets up both a base device mapping and one through an fs file on the same device.
Quoted 6 lines
-----Original Message-----
From: Linux-nvdimm [mailto:[email protected]] On Behalf Of Jan Kara
Sent: Wednesday, October 21, 2015 4:08 AM
...
Quoted 2 lines · Dan Williams
Quoted 1 line
On Tue, Oct 20, 2015 at 5:01 PM, Dave Chinner <[email protected]>
...
Quoted 12 lines
Quoted 8 lines
I'm wondering if we can take a conservative step forward with those patches for 4.4. if XFS and EXT4 interactions need more time to get worked out, which I believe they do, I can conceive just turning on
get_user_pages()support for DAX-mappings of the raw block device. This would be via the new facility I posted yesterday:
https://lists.01.org/pipermail/linux-nvdimm/2015-October/002512.html.
While not very functional for applications it makes testing base DAX mechanisms straightforward.I had a look at the patch and I miss one thing: Why do we need
bd_mutexto protect faults? I see a comment there:
Quoted 6 lines
/* check that the faulting page hasn't raced with bdev resize */
Is it really possible that bdev gets shrunk under us? Hum, looking into
fs/block_dev.c, probably it is. But there are other places - like DIO path
- assuming that block device mapping cannot just disappear from under us. I wonder how that would cope with bdev size change...
DIO_IGNORE_TRUNCATE was added to eliminate an awful lot of CPU time incrementing and decrementing i_dio_count for direct IO to block devices, especially at pmem type speeds. I hope that kind of accounting doesn't need to be brought back.
https://lkml.org/lkml/2015/4/3/557
https://lkml.org/lkml/2015/4/15/590
---
Robert Elliott, HPE Persistent Memory
Quoted 29 lines · Robert (Persistent Memory)
Quoted 6 lines
-----Original Message-----
From: Linux-nvdimm [mailto:[email protected]] On Behalf Of Jan Kara
Sent: Wednesday, October 21, 2015 4:08 AM...
Quoted 2 lines · Dan Williams
Quoted 1 line
On Tue, Oct 20, 2015 at 5:01 PM, Dave Chinner <[email protected]>
...
Quoted 12 lines
Quoted 8 lines
I'm wondering if we can take a conservative step forward with those patches for 4.4. if XFS and EXT4 interactions need more time to get worked out, which I believe they do, I can conceive just turning on
get_user_pages()support for DAX-mappings of the raw block device. This would be via the new facility I posted yesterday:
https://lists.01.org/pipermail/linux-nvdimm/2015-October/002512.html.
While not very functional for applications it makes testing base DAX mechanisms straightforward.I had a look at the patch and I miss one thing: Why do we need
bd_mutexto protect faults? I see a comment there:Quoted 6 lines
/* check that the faulting page hasn't raced with bdev resize */
Is it really possible that bdev gets shrunk under us? Hum, looking into
fs/block_dev.c, probably it is. But there are other places - like DIO path
- assuming that block device mapping cannot just disappear from under us. I wonder how that would cope with bdev size change...
Quoted 7 lines
DIO_IGNORE_TRUNCATEwas added to eliminate an awful lot of CPU time incrementing and decrementingi_dio_countfor direct IO to block devices, especially at pmem type speeds. I hope that kind of accounting doesn't need to be brought back.https://lkml.org/lkml/2015/4/3/557
https://lkml.org/lkml/2015/4/15/590
Thanks for that reminder Robert. I think this backs up my suspicion that the base mmap_sem protections should be enough for raw block mmap
+ DAX.
Quoted 25 lines · Dan J
===
for-4.4/dax-gup:get_user_pages()support for dax mappings
https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.01.org_pipermail_linux-2Dnvdimm_2015-2DOctober_002387.html&d=CwIGaQ&c=5VD0RTtNlTh3ycd41b3MUw&r=cK1a7KivzZRh1fKQMjSm2A&m=GDROuamLFa0boM8dkPE6fXkQ-bqUhy3M3gH-W-7UaUU&s=S8I_OUhMMB7AAcJn6TzV01M9jB6e2wB3aMIT4lTGHf8&e=
===Needs acks from core -mm folks particularly for:
x86, mm: introducevmem_altmapto augmentvmemmap_populate()
mm, x86:get_user_pages()for dax mappingsDan Williams (22):
block: genericrequest_queuereference counting
dax: increase granularity ofdax_clear_blocks()operations block, dax: fix lifetime of in-kernel dax mappings withdax_map_atomic()
mm: introduce__get_dev_pagemap()
x86, mm: introducevmem_altmapto augmentvmemmap_populate()libnvdimm, pfn, pmem: allocate memmap array in persistent memory
avr32: convert toasm-generic/memory_model.h
hugetlb: fix compile error on tile
frv: fix compiler warning from definition of__pmd()
um: killpfn_t
kvm: renamepfn_ttokvm_pfn_t
mips: fixPAGE_MASKdefinition
mm, dax, pmem: introducepfn_t
mm, dax, gpu: convertvm_insert_mixedtopfn_t, introduce_PAGE_DEVMAP
Quoted 8 lines
mm, dax: convert
vmf_insert_pfn_pmd()topfn_t
list: introducelist_del_poison()
mm, dax, pmem: introduce {get|put}_dev_pagemap() for dax-gup
block: notify queue death confirmation
mm, pmem:devm_memunmap_pages(), truncate and unmapZONE_DEVICEpages mm, x86:get_user_pages()for dax mappings
block: introducefile_bd_inode()
block: enable dax for raw block devices
We really should pull these core block changes in separately.
Quoted 16 lines
===
for-4.4/blk-integrity:
===Jens? I've folded my fixes with Martin's latest and block.git/for-4.4/drivers.
Dan Williams (7):
md, dm, scsi, nvme, libnvdimm: dropblk_integrity_unregister()at shutdown
md: suspend i/o during runtimeblk_integrity_unregister
nvme: suspend i/o during runtimeblk_integrity_unregister
block: genericrequest_queuereference counting
block: moveblk_integritytorequest_queue
block:blk_flush_integrity()for bio-based drivers
block, libnvdimm, nvme: provide a built-inblk_integritynop profile
Quoted 6 lines
Martin K. Petersen (5):
block: Move integrity kobject to struct gendisk
block: Consolidate static integrity profile properties
block: Reduce the size of structblk_integrity
block: Export integrity data interval size in sysfs
block: Inlineblk_integrityin struct gendisk
I'll pull these in.
Signature · 1 line
Jens Axboe
Quoted 4 lines · Jens Axboe
Quoted 3 lines · Dan J
===
for-4.4/dax-gup:get_user_pages()support for dax mappings
[..]
Quoted 1 line
Quoted 1 line
===
[..]
Quoted 1 line
We really should pull these core block changes in separately.
Sounds good to me. I'll refactor the block bits into a standalone series.
Quoted 3 lines
Quoted 3 lines
===
for-4.4/blk-integrity:
===
[..]
Quoted 4 lines
I'll pull these in.
Much appreciated!
Quoted 8 lines · Dan J
Here is a status summary of the topic-branches nvdimm.git is tracking for v4.4. Unless indicated these branches are not present in -next. Please ACK, NAK, or ask for a re-post of any of the below to disposition it for the merge window.
===
for-4.4/dax-fixes:
===
<snip>
Quoted 2 lines
Ross Zwisler (2):
dax:dax_pfn_mkwrite()truncate race check
We can drop this patch for dax_pfn_mkwrite() as that whole function is going away after we merge the locking fixes from XFS, ext2 and ext4.