Skip to contentSkip to conversation

infiniband: Fix alignment of mmap cookies to support VIPT caching

PATCH Applied Jason Gunthorpe 2 participants 2 messages Mar 10, 2017 – Mar 24, 2017 1 message not in the archive

Subsystems (3)
InfiniBand Soft-RoCE driver (rxe) Rdmavt - RDMA verbs software
lore
Share & tools
lore

Jason Gunthorpe

author
#0 lore raw

PATCH 1/1 infiniband: Fix alignment of mmap cookies to support VIPT caching

When vmalloc_user is used to create memory that is supposed to be mmap'd to user space, it is necessary for the mmap cookie (eg the offset) to be aligned to SHMLBA.

This creates a situation where all virtual mappings of the same physical page share the same virtual cache index and guarantees VIPT coherence. Otherwise the cache is non-coherent and the kernel will not see writes by userspace when reading the shared page (or vice-versa).

Reported-by Josh Beavers <[email protected]>
Signed-off-by Jason Gunthorpe <jgunthorpe-ePGOBjL8dl3ta4EC/59zMFaTQe2KTcn/@public.gmane.org>
---
 drivers/infiniband/sw/rdmavt/mmap.c  | 4 ++--
 drivers/infiniband/sw/rxe/rxe_mmap.c | 4 ++--
 2 files changed, 4 insertions(+), 4 deletions(-)

The psm cdevs are still broken. Josh confirmed the patch does what it is supposed to, but it doesn't fix VIVT caches.

It is depressing the exact same lines are copied between rxe and rdmavt.

drivers/infiniband/sw/rdmavt/mmap.c +2 -2
index e202b8142759f5..6b712eecbd37d9 100644
@@ -170,9 +170,9 @@ struct rvt_mmap_info *rvt_create_mmap_info(struct rvt_dev_info *rdi,
 
 	spin_lock_irq(&rdi->mmap_offset_lock);
 	if (rdi->mmap_offset == 0)
-		rdi->mmap_offset = PAGE_SIZE;
+		rdi->mmap_offset = ALIGN(PAGE_SIZE, SHMLBA);
 	ip->offset = rdi->mmap_offset;
-	rdi->mmap_offset += size;
+	rdi->mmap_offset += ALIGN(size, SHMLBA);
 	spin_unlock_irq(&rdi->mmap_offset_lock);
 
 	INIT_LIST_HEAD(&ip->pending_mmaps);
drivers/infiniband/sw/rxe/rxe_mmap.c +2 -2
index c572a4c09359c9..bd812e00988ed3 100644
@@ -156,10 +156,10 @@ struct rxe_mmap_info *rxe_create_mmap_info(struct rxe_dev *rxe,
 	spin_lock_bh(&rxe->mmap_offset_lock);
 
 	if (rxe->mmap_offset == 0)
-		rxe->mmap_offset = PAGE_SIZE;
+		rxe->mmap_offset = ALIGN(PAGE_SIZE, SHMLBA);
 
 	ip->info.offset = rxe->mmap_offset;
-	rxe->mmap_offset += size;
+	rxe->mmap_offset += ALIGN(size, SHMLBA);
 
 	spin_unlock_bh(&rxe->mmap_offset_lock);
Signature · 5 lines
2.7.4
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

A message that is not in the archive (sent off-list or not yet received): 20170310183420.GA7410-ePGOBjL8dl3ta4EC/59zMFaTQe2KTcn/@public.gmane.org

Applied
#2 lore raw
Quoted 14 lines · Jason Gunthorpe

When vmalloc_user is used to create memory that is supposed to be mmap'd
to user space, it is necessary for the mmap cookie (eg the offset) to be
aligned to SHMLBA.

This creates a situation where all virtual mappings of the same physical
page share the same virtual cache index and guarantees VIPT coherence.
Otherwise the cache is non-coherent and the kernel will not see writes
by userspace when reading the shared page (or vice-versa).

Quoted 3 lines
Reported-by Josh Beavers <[email protected]>
Signed-off-by Jason Gunthorpe <jgunthorpe-ePGOBjL8dl3ta4EC/59zMFaTQe2KTcn/@public.gmane.org>

Thanks, applied for -rc.

Signature · 9 lines
Doug Ledford <[email protected]>
    GPG KeyID: B826A3330E572FDD
   
Key fingerprint = AE6B 1BDA 122B 23B4 265B  1274 B826 A333 0E57 2FDD

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html