Skip to contentSkip to conversation

Application Data Integrity feature introduced by SPARC M7

PATCH In review Khalid Aziz 7 participants 44 messages Feb 21, 2018 – Mar 19, 2018

Subsystems (10)
lore
Share & tools
lore
#0 lore raw

v12 cover Application Data Integrity feature introduced by SPARC M7

V12 changes:
This series is same as v10 and v11 and was simply rebased on 4.16-rc2 kernel and patch 11 was added to update signal delivery code to use the new helper functions added by Eric Biederman. Can mm maintainers please review patches 2, 7, 8 and 9 which are arch independent, and include/linux/mm.h and mm/ksm.c changes in patch 10 and ack these if everything looks good?

SPARC M7 processor adds additional metadata for memory address space that can be used to secure access to regions of memory. This additional metadata is implemented as a 4-bit tag attached to each cacheline size block of memory. A task can set a tag on any number of such blocks. Access to such block is granted only if the virtual address used to access that block of memory has the tag encoded in the uppermost 4 bits of VA. Since sparc processor does not implement all 64 bits of VA, top 4 bits are available for ADI tags. Any mismatch between tag encoded in VA and tag set on the memory block results in a trap. Tags are verified in the VA presented to the MMU and tags are associated with the physical page VA maps on to. If a memory page is swapped out and page frame gets reused for another task, the tags are lost and hence must be saved when swapping or migrating the page.

A userspace task enables ADI through mprotect(). This patch series adds a page protection bit PROT_ADI and a corresponding VMA flag VM_SPARC_ADI. VM_SPARC_ADI is used to trigger setting TTE.mcd bit in the sparc pte that enables ADI checking on the corresponding page. MMU validates the tag embedded in VA for every page that has TTE.mcd bit set in its pte. After enabling ADI on a memory range, the userspace task can set ADI version tags using stxa instruction with ASI_MCD_PRIMARY or ASI_MCD_ST_BLKINIT_PRIMARY ASI.

Once userspace task calls mprotect() with PROT_ADI, kernel takes following overall steps:

1. Find the VMAs covering the address range passed in to mprotect and set VM_SPARC_ADI flag. If address range covers a subset of a VMA, the VMA will be split.

2. When a page is allocated for a VA and the VMA covering this VA has VM_SPARC_ADI flag set, set the TTE.mcd bit so MMU will check the vwersion tag.

3. Userspace can now set version tags on the memory it has enabled ADI on. Userspace accesses ADI enabled memory using a virtual address that has the version tag embedded in the high bits. MMU validates this version tag against the actual tag set on the memory. If tag matches, MMU performs the VA->PA translation and access is granted. If there is a mismatch, hypervisor sends a data access exception or precise memory corruption detected exception depending upon whether precise exceptions are enabled or not (controlled by MCDPERR register). Kernel sends SIGSEGV to the task with appropriate si_code.

4. If a page is being swapped out or migrated, kernel must save any ADI tags set on the page. Kernel maintains a page worth of tag storage descriptors. Each descriptors pointsto a tag storage space and the address range it covers. If the page being swapped out or migrated has ADI enabled on it, kernel finds a tag storage descriptor that covers the address range for the page or allocates a new descriptor if none of the existing descriptors cover the address range. Kernel saves tags from the page into the tag storage space descriptor points to.

5. When the page is swapped back in or reinstantiated after migration, kernel restores the version tags on the new physical page by retrieving the original tag from tag storage pointed to by a tag storage descriptor for the virtual address range for new page.

User task can disable ADI by calling mprotect() again on the memory range with PROT_ADI bit unset. Kernel clears the VM_SPARC_ADI flag in VMAs, merges adjacent VMAs if necessary, and clears TTE.mcd bit in the corresponding ptes.

IOMMU does not support ADI checking. Any version tags embedded in the top bits of VA meant for IOMMU, are cleared and replaced with sign extension of the first non-version tag bit (bit 59 for SPARC M7) for IOMMU addresses.

This patch series adds support for this feature in 11 patches:

Patch 1/11
Tag mismatch on access by a task results in a trap from hypervisor as data access exception or a precide memory corruption detected exception. As part of handling these exceptions, kernel sends a SIGSEGV to user process with special si_code to indicate which fault occurred. This patch adds three new si_codes to differentiate between various mismatch errors.

Patch 2/11
When a page is swapped or migrated, metadata associated with the page must be saved so it can be restored later. This patch adds a new function that saves/restores this metadata when updating pte upon a swap/migration.

Patch 3/11
SPARC M7 processor adds new fields to control registers to support ADI feature. It also adds a new exception for precise traps on tag mismatch. This patch adds definitions for the new control register fields, new ASIs for ADI and an exception handler for the precise trap on tag mismatch.

Patch 4/11
New hypervisor fault types were added by sparc M7 processor to support ADI feature. This patch adds code to handle these fault types for data access exception handler.

Patch 5/11
When ADI is in use for a page and a tag mismatch occurs, processor raises "Memory corruption Detected" trap. This patch adds a handler for this trap.

Patch 6/11
ADI usage is governed by ADI properties on a platform. These properties are provided to kernel by firmware. Thsi patch adds new auxiliary vectors that provide these values to userpsace.

Patch 7/11
arch_validate_prot() is used to validate the new protection bits asked for by the userspace app. Validating protection bits may need the context of address space the bits are being applied to. One such example is PROT_ADI bit on sparc processor that enables ADI protection on an address range. ADI protection applies only to addresses covered by physical RAM and not other PFN mapped addresses or device addresses. This patch adds "address" to the parameters being passed to arch_validate_prot() to provide that context.

Patch 8/11
When protection bits are changed on a page, kernel carries forward all protection bits except for read/write/exec. Additional code was added to allow kernel to clear PKEY bits on x86 but this requirement to clear other bits is not unique to x86. This patch extends the existing code to allow other architectures to clear any other protection bits as well on protection bit change.

Patch 9/11
When a processor supports additional metadata on memory pages, that additional metadata needs to be copied to new memory pages when those pages are moved. This patch allows architecture specific code to replace the default copy_highpage() routine with arch specific version that copies the metadata as well besides the data on the page.

Patch 10/11
This patch adds support for a user space task to enable ADI and enable tag checking for subsets of its address space. As part of enabling this feature, this patch adds to support manipulation of precise exception for memory corruption detection, adds code to save and restore tags on page swap and migration, and adds code to handle ADI tagged addresses for DMA.

Patch 11/11
Update signal delivery code in arch/sparc/kernel/traps_64.c to use the new helper function force_sig_fault() added by commit f8ec66014ffd ("signal: Add send_sig_fault and force_sig_fault").

Changelog v12:

- Rebased to 4.16-rc2
- Added patch 11 to update signal delivery functions

Changelog v11:

- Rebased to 4.15

Changelog v10:

- Patch 1/10: Updated si_codes definitions for SEGV to match 4.14
- Patch 2/10: No changes
- Patch 3/10: Updated copyright
- Patch 4/10: No changes
- Patch 5/10: No changes
- Patch 6/10: Updated copyright
- Patch 7/10: No changes
- Patch 8/10: No changes
- Patch 9/10: No changes
- Patch 10/10: Added code to return from kernel path to set PSTATE.mcde if kernel continues execution in another thread (Suggested by Anthony)

Changelog v9:

- Patch 1/10: No changes
- Patch 2/10: No changes
- Patch 3/10: No changes
- Patch 4/10: No changes
- Patch 5/10: No changes
- Patch 6/10: No changes
- Patch 7/10: No changes
- Patch 8/10: No changes
- Patch 9/10: New patch
- Patch 10/10: Patch 9 from v8. Added code to copy ADI tags when pages are migrated. Updated code to detect overflow and underflow of addresses when allocating tag storage.

Changelog v8:

- Patch 1/9: No changes
- Patch 2/9: Fixed and erroneous "}"
- Patch 3/9: Minor print formatting change
- Patch 4/9: No changes
- Patch 5/9: No changes
- Patch 6/9: Added AT_ADI_UEONADI back
- Patch 7/9: Added addr parameter to powerpc arch_validate_prot()
- Patch 8/9: No changes
- Patch 9/9:
- Documentation updates
- Added an IPI on mprotect(...PROT_ADI...) call and restore of TSTATE.MCDE on context switch
- Removed restriction on enabling ADI on read-only memory
- Changed kzalloc() for tag storage to use GFP_NOWAIT
- Added code to handle overflow and underflow when allocating tag storage
- Replaced sun_m7_patch_1insn_range() with sun4v_patch_1insn_range()
- Added membar after restoring ADI tags in copy_user_highpage()

Changelog v7:

- Patch 1/9: No changes
- Patch 2/9: Updated parameters to arch specific swap in/out handlers
- Patch 3/9: No changes
- Patch 4/9: new patch split off from patch 4/4 in v6
- Patch 5/9: new patch split off from patch 4/4 in v6
- Patch 6/9: new patch split off from patch 4/4 in v6
- Patch 7/9: new patch
- Patch 8/9: new patch
- Patch 9/9:
- Enhanced arch_validate_prot() to enable ADI only on writable addresses backed by physical RAM
- Added support for saving/restoring ADI tags for each ADI block size address range on a page on swap in/out
- copy ADI tags on COW
- Updated values for auxiliary vectors to not conflict with values on other architectures to avoid conflict in glibc
- Disable same page merging on ADI enabled pages
- Enable ADI only on writable addresses backed by physical RAM
- Split parts of patch off into separate patches

Changelog v6:
- Patch 1/4: No changes
- Patch 2/4: No changes
- Patch 3/4: Added missing nop in the delay slot in sun4v_mcd_detect_precise
- Patch 4/4: Eliminated instructions to read and write PSTATE as well as MCDPER and PMCDPER on every access to userspace addresses by setting PSTATE and PMCDPER correctly upon entry into kernel

Changelog v5:
- Patch 1/4: No changes
- Patch 2/4: Replaced set_swp_pte_at() with new architecture functions arch_do_swap_page() and arch_unmap_one() that suppoprt architecture specific actions to be taken on page swap and migration
- Patch 3/4: Fixed indentation issues in assembly code
- Patch 4/4:
- Fixed indentation issues and instrcuctions in assembly code
- Removed CONFIG_SPARC64 from mdesc.c
- Changed to maintain state of MCDPER register in thread info flags as opposed to in mm context. MCDPER is a per-thread state and belongs in thread info flag as opposed to mm context which is shared across threads. Added comments to clarify this is a lazily maintained state and must be updated on context switch and copy_process()
- Updated code to use the new arch_do_swap_page() and arch_unmap_one() functions

Testing:

- All functionality was tested with 8K normal pages as well as hugepages using malloc, mmap and shm.
- Multiple long duration stress tests were run using hugepages over 2+ months. Normal pages were tested with shorter duration stress tests.
- Tested swapping with malloc and shm by reducing max memory and allocating three times the available system memory by active processes using ADI on allocated memory. Ran through multiple hours long runs of this test.
- Tested page migration with malloc and shm by migrating data pages of active ADI test process using migratepages, back and forth between two nodes every few seconds over an hour long run. Verified page migration through /proc/<pid>/numa_maps.
- Tested COW support using test that forks children that read from ADI enabled pages shared with parent and other children and write to them as well forcing COW.

---------

Khalid Aziz (11):
signals, sparc: Add signal codes for ADI violations mm, swap: Add infrastructure for saving page metadata on swap
sparc64: Add support for ADI register fields, ASIs and traps
sparc64: Add HV fault type handlers for ADI related faults
sparc64: Add handler for "Memory Corruption Detected" trap
sparc64: Add auxiliary vectors to report platform ADI properties
mm: Add address parameter to arch_validate_prot()
mm: Clear arch specific VM flags on protection change
mm: Allow arch code to override copy_highpage()
sparc64: Add support for ADI (Application Data Integrity)
sparc64: Update signal delivery to use new helper functions

 Documentation/sparc/adi.txt             | 278 ++++++++++++++++++++++
 arch/powerpc/include/asm/mman.h         |   4 +-
 arch/powerpc/kernel/syscalls.c          |   2 +-
 arch/sparc/include/asm/adi.h            |   6 +
 arch/sparc/include/asm/adi_64.h         |  47 ++++
 arch/sparc/include/asm/elf_64.h         |   5 +
 arch/sparc/include/asm/hypervisor.h     |   2 +
 arch/sparc/include/asm/mman.h           |  84 ++++++-
 arch/sparc/include/asm/mmu_64.h         |  17 ++
 arch/sparc/include/asm/mmu_context_64.h |  50 ++++
 arch/sparc/include/asm/page_64.h        |   6 +
 arch/sparc/include/asm/pgtable_64.h     |  48 ++++
 arch/sparc/include/asm/thread_info_64.h |   2 +-
 arch/sparc/include/asm/trap_block.h     |   2 +
 arch/sparc/include/asm/ttable.h         |  10 +
 arch/sparc/include/uapi/asm/asi.h       |   5 +
 arch/sparc/include/uapi/asm/auxvec.h    |  11 +
 arch/sparc/include/uapi/asm/mman.h      |   2 +
 arch/sparc/include/uapi/asm/pstate.h    |  10 +
 arch/sparc/kernel/Makefile              |   1 +
 arch/sparc/kernel/adi_64.c              | 397 ++++++++++++++++++++++++++++++++
 arch/sparc/kernel/entry.h               |   3 +
 arch/sparc/kernel/etrap_64.S            |  27 ++-
 arch/sparc/kernel/head_64.S             |   1 +
 arch/sparc/kernel/mdesc.c               |   2 +
 arch/sparc/kernel/process_64.c          |  25 ++
 arch/sparc/kernel/rtrap_64.S            |  33 ++-
 arch/sparc/kernel/setup_64.c            |   2 +
 arch/sparc/kernel/sun4v_mcd.S           |  18 ++
 arch/sparc/kernel/traps_64.c            | 130 ++++++++++-
 arch/sparc/kernel/ttable_64.S           |   6 +-
 arch/sparc/kernel/urtt_fill.S           |   7 +-
 arch/sparc/kernel/vmlinux.lds.S         |   5 +
 arch/sparc/mm/gup.c                     |  37 +++
 arch/sparc/mm/hugetlbpage.c             |  14 +-
 arch/sparc/mm/init_64.c                 |  69 ++++++
 arch/sparc/mm/tsb.c                     |  21 ++
 arch/x86/kernel/signal_compat.c         |   2 +-
 include/asm-generic/pgtable.h           |  36 +++
 include/linux/highmem.h                 |   4 +
 include/linux/mm.h                      |   9 +
 include/linux/mman.h                    |   2 +-
 include/uapi/asm-generic/siginfo.h      |   5 +-
 mm/ksm.c                                |   4 +
 mm/memory.c                             |   1 +
 mm/mprotect.c                           |   4 +-
 mm/rmap.c                               |  14 ++
 47 files changed, 1445 insertions(+), 25 deletions(-)
 create mode 100644 Documentation/sparc/adi.txt
 create mode 100644 arch/sparc/include/asm/adi.h
 create mode 100644 arch/sparc/include/asm/adi_64.h
 create mode 100644 arch/sparc/kernel/adi_64.c
 create mode 100644 arch/sparc/kernel/sun4v_mcd.S
Signature · 1 line
2.11.0
#1 lore raw

PATCH v12 1/11 signals, sparc: Add signal codes for ADI violations

SPARC M7 processor introduces a new feature - Application Data Integrity (ADI). ADI allows MMU to catch rogue accesses to memory. When a rogue access occurs, MMU blocks the access and raises an exception. In response to the exception, kernel sends the offending task a SIGSEGV with si_code that indicates the nature of exception. This patch adds three new signal codes specific to ADI feature:

1. ADI is not enabled for the address and task attempted to access memory using ADI
2. Task attempted to access memory using wrong ADI tag and caused a deferred exception.
3. Task attempted to access memory using wrong ADI tag and caused a precise exception.

---
v10:
- Updated si_code definitions to match other definitions

 arch/x86/kernel/signal_compat.c    | 2 +-
 include/uapi/asm-generic/siginfo.h | 5 ++++-
 2 files changed, 5 insertions(+), 2 deletions(-)
arch/x86/kernel/signal_compat.c +1 -1
index ac057f9b0763..41cf5996a2a9 100644
@@ -27,7 +27,7 @@ static inline void signal_compat_build_tests(void)
 	 */
 	BUILD_BUG_ON(NSIGILL  != 11);
 	BUILD_BUG_ON(NSIGFPE  != 13);
-	BUILD_BUG_ON(NSIGSEGV != 4);
+	BUILD_BUG_ON(NSIGSEGV != 7);
 	BUILD_BUG_ON(NSIGBUS  != 5);
 	BUILD_BUG_ON(NSIGTRAP != 4);
 	BUILD_BUG_ON(NSIGCHLD != 6);
include/uapi/asm-generic/siginfo.h +4 -1
index 85dc965afd89..e94f03b3ee6f 100644
@@ -246,7 +246,10 @@ typedef struct siginfo {
 #else
 # define SEGV_PKUERR	4	/* failed protection key checks */
 #endif
-#define NSIGSEGV	4
+#define SEGV_ACCADI	5	/* ADI not enabled for mapped object */
+#define SEGV_ADIDERR	6	/* Disrupting MCD error */
+#define SEGV_ADIPERR	7	/* Precise MCD exception */
+#define NSIGSEGV	7
 
 /*
  * SIGBUS si_codes
Signature · 1 line
2.11.0
#3 lore raw

PATCH v12 2/11 mm, swap: Add infrastructure for saving page metadata on swap

In reply to Khalid Aziz

If a processor supports special metadata for a page, for example ADI version tags on SPARC M7, this metadata must be saved when the page is swapped out. The same metadata must be restored when the page is swapped back in. This patch adds two new architecture specific functions - arch_do_swap_page() to be called when a page is swapped in, and arch_unmap_one() to be called when a page is being unmapped for swap out. These architecture hooks allow page metadata to be saved if the architecture supports it.

---
v8:
- Fixed an erroneous "}"
v6:
- Updated parameter list for arch_do_swap_page() and arch_unmap_one()
v5:
- Replaced set_swp_pte() function with new architecture functions arch_do_swap_page() and arch_unmap_one()

 include/asm-generic/pgtable.h | 36 ++++++++++++++++++++++++++++++++++++
 mm/memory.c                   |  1 +
 mm/rmap.c                     | 14 ++++++++++++++
 3 files changed, 51 insertions(+)
include/asm-generic/pgtable.h +36 -0
index 2cfa3075d148..6fbbc0b6c05e 100644
@@ -400,6 +400,42 @@ static inline int pud_same(pud_t pud_a, pud_t pud_b)
 #endif /* CONFIG_TRANSPARENT_HUGEPAGE */
 #endif
 
+#ifndef __HAVE_ARCH_DO_SWAP_PAGE
+/*
+ * Some architectures support metadata associated with a page. When a
+ * page is being swapped out, this metadata must be saved so it can be
+ * restored when the page is swapped back in. SPARC M7 and newer
+ * processors support an ADI (Application Data Integrity) tag for the
+ * page as metadata for the page. arch_do_swap_page() can restore this
+ * metadata when a page is swapped back in.
+ */
+static inline void arch_do_swap_page(struct mm_struct *mm,
+				     struct vm_area_struct *vma,
+				     unsigned long addr,
+				     pte_t pte, pte_t oldpte)
+{
+
+}
+#endif
+
+#ifndef __HAVE_ARCH_UNMAP_ONE
+/*
+ * Some architectures support metadata associated with a page. When a
+ * page is being swapped out, this metadata must be saved so it can be
+ * restored when the page is swapped back in. SPARC M7 and newer
+ * processors support an ADI (Application Data Integrity) tag for the
+ * page as metadata for the page. arch_unmap_one() can save this
+ * metadata on a swap-out of a page.
+ */
+static inline int arch_unmap_one(struct mm_struct *mm,
+				  struct vm_area_struct *vma,
+				  unsigned long addr,
+				  pte_t orig_pte)
+{
+	return 0;
+}
+#endif
+
 #ifndef __HAVE_ARCH_PGD_OFFSET_GATE
 #define pgd_offset_gate(mm, addr)	pgd_offset(mm, addr)
 #endif
mm/memory.c +1 -0
index 5fcfc24904d1..aed37325d94e 100644
@@ -3053,6 +3053,7 @@ int do_swap_page(struct vm_fault *vmf)
 	if (pte_swp_soft_dirty(vmf->orig_pte))
 		pte = pte_mksoft_dirty(pte);
 	set_pte_at(vma->vm_mm, vmf->address, vmf->pte, pte);
+	arch_do_swap_page(vma->vm_mm, vma, vmf->address, pte, vmf->orig_pte);
 	vmf->orig_pte = pte;
 
 	/* ksm created a completely new copy */
mm/rmap.c +14 -0
index 47db27f8049e..144c66e688a9 100644
@@ -1497,6 +1497,14 @@ static bool try_to_unmap_one(struct page *page, struct vm_area_struct *vma,
 				(flags & (TTU_MIGRATION|TTU_SPLIT_FREEZE))) {
 			swp_entry_t entry;
 			pte_t swp_pte;
+
+			if (arch_unmap_one(mm, vma, address, pteval) < 0) {
+				set_pte_at(mm, address, pvmw.pte, pteval);
+				ret = false;
+				page_vma_mapped_walk_done(&pvmw);
+				break;
+			}
+
 			/*
 			 * Store the pfn of the page in a special migration
 			 * pte. do_swap_page() will wait until the migration
@@ -1556,6 +1564,12 @@ static bool try_to_unmap_one(struct page *page, struct vm_area_struct *vma,
 				page_vma_mapped_walk_done(&pvmw);
 				break;
 			}
+			if (arch_unmap_one(mm, vma, address, pteval) < 0) {
+				set_pte_at(mm, address, pvmw.pte, pteval);
+				ret = false;
+				page_vma_mapped_walk_done(&pvmw);
+				break;
+			}
 			if (list_empty(&mm->mmlist)) {
 				spin_lock(&mmlist_lock);
 				if (list_empty(&mm->mmlist))
Signature · 1 line
2.11.0
#11 lore raw

PATCH v12 3/11 sparc64: Add support for ADI register fields, ASIs and traps

In reply to Khalid Aziz

SPARC M7 processor adds new control register fields, ASIs and a new trap to support the ADI (Application Data Integrity) feature. This patch adds definitions for these register fields, ASIs and a handler for the new precise memory corruption detected trap.

---
v10:
- Updated copyright
v8:
- Minor print formatting change as suggested by checkpatch
v6:
- Added a missing nop in the delay slot in sun4v_mcd_detect_precise

v5:
- Fixed indentation issues in assembly code

v4:
- Broke patch up into smaller patches

v3:
- Removed CONFIG_SPARC_ADI
- Replaced prctl commands with mprotect
- Added auxiliary vectors for ADI parameters
- Enabled ADI for swappable pages

v2:
- Fixed a build error

 arch/sparc/include/asm/hypervisor.h  |  2 ++
 arch/sparc/include/asm/pgtable_64.h  |  2 ++
 arch/sparc/include/asm/ttable.h      | 10 +++++++
 arch/sparc/include/uapi/asm/asi.h    |  5 ++++
 arch/sparc/include/uapi/asm/pstate.h | 10 +++++++
 arch/sparc/kernel/entry.h            |  3 ++
 arch/sparc/kernel/head_64.S          |  1 +
 arch/sparc/kernel/sun4v_mcd.S        | 18 ++++++++++++
 arch/sparc/kernel/traps_64.c         | 54 ++++++++++++++++++++++++++++++++++++
 arch/sparc/kernel/ttable_64.S        |  6 ++--
 10 files changed, 109 insertions(+), 2 deletions(-)
 create mode 100644 arch/sparc/kernel/sun4v_mcd.S
arch/sparc/include/asm/hypervisor.h +2 -0
index ab9c6b027b75..08650d503cc2 100644
@@ -570,6 +570,8 @@ struct hv_fault_status {
 #define HV_FAULT_TYPE_RESV1	13
 #define HV_FAULT_TYPE_UNALIGNED	14
 #define HV_FAULT_TYPE_INV_PGSZ	15
+#define HV_FAULT_TYPE_MCD	17
+#define HV_FAULT_TYPE_MCD_DIS	18
 /* Values 16 --> -2 are reserved.  */
 #define HV_FAULT_TYPE_MULTIPLE	-1
 
arch/sparc/include/asm/pgtable_64.h +2 -0
index 339920fdf9ed..619332a44402 100644
@@ -164,6 +164,8 @@ bool kern_addr_valid(unsigned long addr);
 #define _PAGE_E_4V	  _AC(0x0000000000000800,UL) /* side-Effect          */
 #define _PAGE_CP_4V	  _AC(0x0000000000000400,UL) /* Cacheable in P-Cache */
 #define _PAGE_CV_4V	  _AC(0x0000000000000200,UL) /* Cacheable in V-Cache */
+/* Bit 9 is used to enable MCD corruption detection instead on M7 */
+#define _PAGE_MCD_4V      _AC(0x0000000000000200,UL) /* Memory Corruption    */
 #define _PAGE_P_4V	  _AC(0x0000000000000100,UL) /* Privileged Page      */
 #define _PAGE_EXEC_4V	  _AC(0x0000000000000080,UL) /* Executable Page      */
 #define _PAGE_W_4V	  _AC(0x0000000000000040,UL) /* Writable             */
arch/sparc/include/asm/ttable.h +10 -0
index ede2b66cf4a0..8f6469408019 100644
@@ -219,6 +219,16 @@
 	nop;						\
 	nop;
 
+#define SUN4V_MCD_PRECISE				\
+	ldxa	[%g0] ASI_SCRATCHPAD, %g2;		\
+	ldx	[%g2 + HV_FAULT_D_ADDR_OFFSET], %g4;	\
+	ldx	[%g2 + HV_FAULT_D_CTX_OFFSET], %g5;	\
+	ba,pt	%xcc, etrap;				\
+	 rd	%pc, %g7;				\
+	ba,pt	%xcc, sun4v_mcd_detect_precise;		\
+	 nop;						\
+	nop;
+
 /* Before touching these macros, you owe it to yourself to go and
  * see how arch/sparc64/kernel/winfixup.S works... -DaveM
  *
arch/sparc/include/uapi/asm/asi.h +5 -0
index d371b269571a..fbb30a5b082f 100644
@@ -145,6 +145,8 @@
  * ASIs, "(4V)" designates SUN4V specific ASIs.  "(NG4)" designates SPARC-T4
  * and later ASIs.
  */
+#define ASI_MCD_PRIV_PRIMARY	0x02 /* (NG7) Privileged MCD version VA	*/
+#define ASI_MCD_REAL		0x05 /* (NG7) Privileged MCD version PA	*/
 #define ASI_PHYS_USE_EC		0x14 /* PADDR, E-cachable		*/
 #define ASI_PHYS_BYPASS_EC_E	0x15 /* PADDR, E-bit			*/
 #define ASI_BLK_AIUP_4V		0x16 /* (4V) Prim, user, block ld/st	*/
@@ -245,6 +247,9 @@
 #define ASI_UDBL_CONTROL_R	0x7f /* External UDB control regs rd low*/
 #define ASI_INTR_R		0x7f /* IRQ vector dispatch read	*/
 #define ASI_INTR_DATAN_R	0x7f /* (III) In irq vector data reg N	*/
+#define ASI_MCD_PRIMARY		0x90 /* (NG7) MCD version load/store	*/
+#define ASI_MCD_ST_BLKINIT_PRIMARY	\
+				0x92 /* (NG7) MCD store BLKINIT primary	*/
 #define ASI_PIC			0xb0 /* (NG4) PIC registers		*/
 #define ASI_PST8_P		0xc0 /* Primary, 8 8-bit, partial	*/
 #define ASI_PST8_S		0xc1 /* Secondary, 8 8-bit, partial	*/
arch/sparc/include/uapi/asm/pstate.h +10 -0
index b6999c9e7e86..ceca96e685c2 100644
@@ -11,7 +11,12 @@
  * -----------------------------------------------------------------------
  *  63  12  11   10    9     8    7   6   5     4     3     2     1    0
  */
+/* IG on V9 conflicts with MCDE on M7. PSTATE_MCDE will only be used on
+ * processors that support ADI which do not use IG, hence there is no
+ * functional conflict
+ */
 #define PSTATE_IG   _AC(0x0000000000000800,UL) /* Interrupt Globals.	*/
+#define PSTATE_MCDE _AC(0x0000000000000800,UL) /* MCD Enable		*/
 #define PSTATE_MG   _AC(0x0000000000000400,UL) /* MMU Globals.		*/
 #define PSTATE_CLE  _AC(0x0000000000000200,UL) /* Current Little Endian.*/
 #define PSTATE_TLE  _AC(0x0000000000000100,UL) /* Trap Little Endian.	*/
@@ -48,7 +53,12 @@
 #define TSTATE_ASI	_AC(0x00000000ff000000,UL) /* AddrSpace ID.	*/
 #define TSTATE_PIL	_AC(0x0000000000f00000,UL) /* %pil (Linux traps)*/
 #define TSTATE_PSTATE	_AC(0x00000000000fff00,UL) /* PSTATE.		*/
+/* IG on V9 conflicts with MCDE on M7. TSTATE_MCDE will only be used on
+ * processors that support ADI which do not support IG, hence there is
+ * no functional conflict
+ */
 #define TSTATE_IG	_AC(0x0000000000080000,UL) /* Interrupt Globals.*/
+#define TSTATE_MCDE	_AC(0x0000000000080000,UL) /* MCD enable.       */
 #define TSTATE_MG	_AC(0x0000000000040000,UL) /* MMU Globals.	*/
 #define TSTATE_CLE	_AC(0x0000000000020000,UL) /* CurrLittleEndian.	*/
 #define TSTATE_TLE	_AC(0x0000000000010000,UL) /* TrapLittleEndian.	*/
arch/sparc/kernel/entry.h +3 -0
index 7378567b601f..c746c0fd5d6b 100644
@@ -160,6 +160,9 @@ void sun4v_resum_overflow(struct pt_regs *regs);
 void sun4v_nonresum_error(struct pt_regs *regs,
 			  unsigned long offset);
 void sun4v_nonresum_overflow(struct pt_regs *regs);
+void sun4v_mem_corrupt_detect_precise(struct pt_regs *regs,
+				      unsigned long addr,
+				      unsigned long context);
 
 extern unsigned long sun4v_err_itlb_vaddr;
 extern unsigned long sun4v_err_itlb_ctx;
arch/sparc/kernel/head_64.S +1 -0
index a41e6e16eb36..540bfc98472c 100644
@@ -897,6 +897,7 @@ sparc64_boot_end:
 #include "syscalls.S"
 #include "helpers.S"
 #include "sun4v_tlb_miss.S"
+#include "sun4v_mcd.S"
 #include "sun4v_ivec.S"
 #include "ktlb.S"
 #include "tsb.S"
arch/sparc/kernel/sun4v_mcd.S new file +18 -0
new file mode 100644
index 000000000000..d6c69ebca110
@@ -0,0 +1,18 @@
+/* sun4v_mcd.S: Sun4v memory corruption detected precise exception handler
+ *
+ * Copyright (c) 2015 Oracle and/or its affiliates. All rights reserved.
+ * Authors: Bob Picco <[email protected]>,
+ *          Khalid Aziz <[email protected]>
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2.
+ */
+	.text
+	.align 32
+
+sun4v_mcd_detect_precise:
+	mov	%l4, %o1
+	mov 	%l5, %o2
+	call	sun4v_mem_corrupt_detect_precise
+	 add	%sp, PTREGS_OFF, %o0
+	ba,a,pt	%xcc, rtrap
+	 nop
arch/sparc/kernel/traps_64.c +54 -0

60 lines not on this page · raw message

arch/sparc/kernel/ttable_64.S +4 -2
index 18685fe69b91..86e737e59c7e 100644
@@ -26,8 +26,10 @@ tl0_ill:	membar #Sync
 		TRAP_7INSNS(do_illegal_instruction)
 tl0_privop:	TRAP(do_privop)
 tl0_resv012:	BTRAP(0x12) BTRAP(0x13) BTRAP(0x14) BTRAP(0x15) BTRAP(0x16) BTRAP(0x17)
-tl0_resv018:	BTRAP(0x18) BTRAP(0x19) BTRAP(0x1a) BTRAP(0x1b) BTRAP(0x1c) BTRAP(0x1d)
-tl0_resv01e:	BTRAP(0x1e) BTRAP(0x1f)
+tl0_resv018:	BTRAP(0x18) BTRAP(0x19)
+tl0_mcd:	SUN4V_MCD_PRECISE
+tl0_resv01b:	BTRAP(0x1b)
+tl0_resv01c:	BTRAP(0x1c) BTRAP(0x1d)	BTRAP(0x1e) BTRAP(0x1f)
 tl0_fpdis:	TRAP_NOSAVE(do_fpdis)
 tl0_fpieee:	TRAP_SAVEFPU(do_fpieee)
 tl0_fpother:	TRAP_NOSAVE(do_fpother_check_fitos)
Signature · 1 line
2.11.0
#12 lore raw

PATCH v12 4/11 sparc64: Add HV fault type handlers for ADI related faults

In reply to Khalid Aziz

ADI (Application Data Integrity) feature on M7 and newer processors adds new fault types for hypervisor - Invalid ASI and MCD disabled. This patch expands data access exception handler to handle these faults.

---
v7:
- new patch split off from patch 4/4 in v6

 arch/sparc/kernel/traps_64.c | 29 ++++++++++++++++++++++++++---
 1 file changed, 26 insertions(+), 3 deletions(-)
arch/sparc/kernel/traps_64.c +26 -3
index fc73baa588f6..d273a65a0a10 100644
@@ -397,12 +397,35 @@ void sun4v_data_access_exception(struct pt_regs *regs, unsigned long addr, unsig
 	if (is_no_fault_exception(regs))
 		return;
 
-	info.si_signo = SIGSEGV;
+	/* MCD (Memory Corruption Detection) disabled trap (TT=0x19) in HV
+	 * is vectored thorugh data access exception trap with fault type
+	 * set to HV_FAULT_TYPE_MCD_DIS. Check for MCD disabled trap.
+	 * Accessing an address with invalid ASI for the address, for
+	 * example setting an ADI tag on an address with ASI_MCD_PRIMARY
+	 * when TTE.mcd is not set for the VA, is also vectored into
+	 * kerbel by HV as data access exception with fault type set to
+	 * HV_FAULT_TYPE_INV_ASI.
+	 */
 	info.si_errno = 0;
-	info.si_code = SEGV_MAPERR;
 	info.si_addr = (void __user *) addr;
 	info.si_trapno = 0;
-	force_sig_info(SIGSEGV, &info, current);
+	switch (type) {
+	case HV_FAULT_TYPE_INV_ASI:
+		info.si_signo = SIGILL;
+		info.si_code = ILL_ILLADR;
+		force_sig_info(SIGILL, &info, current);
+		break;
+	case HV_FAULT_TYPE_MCD_DIS:
+		info.si_signo = SIGSEGV;
+		info.si_code = SEGV_ACCADI;
+		force_sig_info(SIGSEGV, &info, current);
+		break;
+	default:
+		info.si_signo = SIGSEGV;
+		info.si_code = SEGV_MAPERR;
+		force_sig_info(SIGSEGV, &info, current);
+		break;
+	}
 }
 
 void sun4v_data_access_exception_tl1(struct pt_regs *regs, unsigned long addr, unsigned long type_ctx)
Signature · 1 line
2.11.0
#13 lore raw

PATCH v12 5/11 sparc64: Add handler for "Memory Corruption Detected" trap

In reply to Khalid Aziz

M7 and newer processors add a "Memory corruption Detected" trap with the addition of ADI feature. This trap is vectored into kernel by HV through resumable error trap with error attribute for the resumable error set to 0x00000800.

---
v7:
- new patch split off from patch 4/4 in v6

 arch/sparc/kernel/traps_64.c | 59 ++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 59 insertions(+)
arch/sparc/kernel/traps_64.c +59 -0
index d273a65a0a10..f36a40217f6c 100644
@@ -1870,6 +1870,7 @@ struct sun4v_error_entry {
 #define SUN4V_ERR_ATTRS_ASI		0x00000080
 #define SUN4V_ERR_ATTRS_PRIV_REG	0x00000100
 #define SUN4V_ERR_ATTRS_SPSTATE_MSK	0x00000600
+#define SUN4V_ERR_ATTRS_MCD		0x00000800
 #define SUN4V_ERR_ATTRS_SPSTATE_SHFT	9
 #define SUN4V_ERR_ATTRS_MODE_MSK	0x03000000
 #define SUN4V_ERR_ATTRS_MODE_SHFT	24
@@ -2067,6 +2068,56 @@ static void sun4v_log_error(struct pt_regs *regs, struct sun4v_error_entry *ent,
 	}
 }
 
+/* Handle memory corruption detected error which is vectored in
+ * through resumable error trap.
+ */
+void do_mcd_err(struct pt_regs *regs, struct sun4v_error_entry ent)
+{
+	siginfo_t info;
+
+	if (notify_die(DIE_TRAP, "MCD error", regs, 0, 0x34,
+		       SIGSEGV) = NOTIFY_STOP)
+		return;
+
+	if (regs->tstate & TSTATE_PRIV) {
+		/* MCD exception could happen because the task was
+		 * running a system call with MCD enabled and passed a
+		 * non-versioned pointer or pointer with bad version
+		 * tag to the system call. In such cases, hypervisor
+		 * places the address of offending instruction in the
+		 * resumable error report. This is a deferred error,
+		 * so the read/write that caused the trap was potentially
+		 * retired long time back and we may have no choice
+		 * but to send SIGSEGV to the process.
+		 */
+		const struct exception_table_entry *entry;
+
+		entry = search_exception_tables(regs->tpc);
+		if (entry) {
+			/* Looks like a bad syscall parameter */
+#ifdef DEBUG_EXCEPTIONS
+			pr_emerg("Exception: PC<%016lx> faddr<UNKNOWN>\n",
+				 regs->tpc);
+			pr_emerg("EX_TABLE: insn<%016lx> fixup<%016lx>\n",
+				 ent.err_raddr, entry->fixup);
+#endif
+			regs->tpc = entry->fixup;
+			regs->tnpc = regs->tpc + 4;
+			return;
+		}
+	}
+
+	/* Send SIGSEGV to the userspace process with the right signal
+	 * code
+	 */
+	info.si_signo = SIGSEGV;
+	info.si_errno = 0;
+	info.si_code = SEGV_ADIDERR;
+	info.si_addr = (void __user *)ent.err_raddr;
+	info.si_trapno = 0;
+	force_sig_info(SIGSEGV, &info, current);
+}
+
 /* We run with %pil set to PIL_NORMAL_MAX and PSTATE_IE enabled in %pstate.
  * Log the event and clear the first word of the entry.
  */
@@ -2104,6 +2155,14 @@ void sun4v_resum_error(struct pt_regs *regs, unsigned long offset)
 		goto out;
 	}
 
+	/* If this is a memory corruption detected error vectored in
+	 * by HV through resumable error trap, call the handler
+	 */
+	if (local_copy.err_attrs & SUN4V_ERR_ATTRS_MCD) {
+		do_mcd_err(regs, local_copy);
+		return;
+	}
+
 	sun4v_log_error(regs, &local_copy, cpu,
 			KERN_ERR "RESUMABLE ERROR",
 			&sun4v_resum_oflow_cnt);
Signature · 1 line
2.11.0
#14 lore raw

PATCH v12 6/11 sparc64: Add auxiliary vectors to report platform ADI properties

In reply to Khalid Aziz

ADI feature on M7 and newer processors has three important properties relevant to userspace apps using ADI capabilities - (1) Size of block of memory an ADI version tag applies to, (2) Number of uppermost bits in virtual address used to encode ADI tag, and (3) The value M7 processor will force the ADI tags to if it detects uncorrectable error in an ADI tagged cacheline. Kernel can retrieve these properties for a platform through machine description provided by the firmware. This patch adds code to retrieve these properties and report them to userspace through auxiliary vectors.

---
v11:
- Rebased to 4.15
v10:
- Updated copyright
v8:
- Added AT_ADI_UEONADI back
v7:
- new patch split off from patch 4/4 in v6

 arch/sparc/include/asm/adi.h         |  6 +++
 arch/sparc/include/asm/adi_64.h      | 47 ++++++++++++++++++
 arch/sparc/include/asm/elf_64.h      |  5 ++
 arch/sparc/include/uapi/asm/auxvec.h | 11 +++++
 arch/sparc/kernel/Makefile           |  1 +
 arch/sparc/kernel/adi_64.c           | 96 ++++++++++++++++++++++++++++++++++++
 arch/sparc/kernel/mdesc.c            |  2 +
 7 files changed, 168 insertions(+)
 create mode 100644 arch/sparc/include/asm/adi.h
 create mode 100644 arch/sparc/include/asm/adi_64.h
 create mode 100644 arch/sparc/kernel/adi_64.c
arch/sparc/include/asm/adi.h new file +6 -0
new file mode 100644
index 000000000000..acad0d04e4c6
@@ -0,0 +1,6 @@
+#ifndef ___ASM_SPARC_ADI_H
+#define ___ASM_SPARC_ADI_H
+#if defined(__sparc__) && defined(__arch64__)
+#include <asm/adi_64.h>
+#endif
+#endif
arch/sparc/include/asm/adi_64.h new file +47 -0

47 lines not on this page · raw message

arch/sparc/include/asm/elf_64.h +5 -0
index 25340df3570c..7e078bc73ef5 100644
@@ -10,6 +10,7 @@
 #include <asm/processor.h>
 #include <asm/extable_64.h>
 #include <asm/spitfire.h>
+#include <asm/adi.h>
 
 /*
  * Sparc section types
@@ -215,9 +216,13 @@ extern unsigned int vdso_enabled;
 
 #define	ARCH_DLINFO							\
 do {									\
+	extern struct adi_config adi_state;				\
 	if (vdso_enabled)						\
 		NEW_AUX_ENT(AT_SYSINFO_EHDR,				\
 			    (unsigned long)current->mm->context.vdso);	\
+	NEW_AUX_ENT(AT_ADI_BLKSZ, adi_state.caps.blksz);		\
+	NEW_AUX_ENT(AT_ADI_NBITS, adi_state.caps.nbits);		\
+	NEW_AUX_ENT(AT_ADI_UEONADI, adi_state.caps.ue_on_adi);		\
 } while (0)
 
 struct linux_binprm;
arch/sparc/include/uapi/asm/auxvec.h +11 -0
index 5f80a70cc901..f9937ccfcd99 100644
@@ -3,6 +3,17 @@
 
 #define AT_SYSINFO_EHDR		33
 
+#ifdef CONFIG_SPARC64
+/* Avoid overlap with other AT_* values since they are consolidated in
+ * glibc and any overlaps can cause problems
+ */
+#define AT_ADI_BLKSZ	48
+#define AT_ADI_NBITS	49
+#define AT_ADI_UEONADI	50
+
+#define AT_VECTOR_SIZE_ARCH	4
+#else
 #define AT_VECTOR_SIZE_ARCH	1
+#endif
 
 #endif /* !(__ASMSPARC_AUXVEC_H) */
arch/sparc/kernel/Makefile +1 -0
index cc97545737f0..76cb57750dda 100644
@@ -69,6 +69,7 @@ obj-$(CONFIG_SPARC64)   += visemul.o
 obj-$(CONFIG_SPARC64)   += hvapi.o
 obj-$(CONFIG_SPARC64)   += sstate.o
 obj-$(CONFIG_SPARC64)   += mdesc.o
+obj-$(CONFIG_SPARC64)   += adi_64.o
 obj-$(CONFIG_SPARC64)	+= pcr.o
 obj-$(CONFIG_SPARC64)	+= nmi.o
 obj-$(CONFIG_SPARC64_SMP) += cpumap.o
arch/sparc/kernel/adi_64.c new file +96 -0

96 lines not on this page · raw message

arch/sparc/kernel/mdesc.c +2 -0
index 418592a09b41..39a2503fa3e1 100644
@@ -22,6 +22,7 @@
 #include <linux/uaccess.h>
 #include <asm/oplib.h>
 #include <asm/smp.h>
+#include <asm/adi.h>
 
 /* Unlike the OBP device tree, the machine description is a full-on
  * DAG.  An arbitrary number of ARCs are possible from one
@@ -1345,5 +1346,6 @@ void __init sun4v_mdesc_init(void)
 
 	cur_mdesc = hp;
 
+	mdesc_adi_init();
 	report_platform_properties();
 }
Signature · 1 line
2.11.0
#15 lore raw

PATCH v12 7/11 mm: Add address parameter to arch_validate_prot()

In reply to Khalid Aziz

A protection flag may not be valid across entire address space and hence arch_validate_prot() might need the address a protection bit is being set on to ensure it is a valid protection flag. For example, sparc processors support memory corruption detection (as part of ADI feature) flag on memory addresses mapped on to physical RAM but not on PFN mapped pages or addresses mapped on to devices. This patch adds address to the parameters being passed to arch_validate_prot() so protection bits can be validated in the relevant context.

---
v8:
- Added addr parameter to powerpc arch_validate_prot() (suggested by Michael Ellerman)
v9:
- new patch

 arch/powerpc/include/asm/mman.h | 4 ++--
 arch/powerpc/kernel/syscalls.c  | 2 +-
 include/linux/mman.h            | 2 +-
 mm/mprotect.c                   | 2 +-
 4 files changed, 5 insertions(+), 5 deletions(-)
arch/powerpc/include/asm/mman.h +2 -2
index 07e3f54de9e3..e3f1b5ba5d5c 100644
@@ -43,7 +43,7 @@ static inline pgprot_t arch_vm_get_page_prot(unsigned long vm_flags)
 }
 #define arch_vm_get_page_prot(vm_flags) arch_vm_get_page_prot(vm_flags)
 
-static inline bool arch_validate_prot(unsigned long prot)
+static inline bool arch_validate_prot(unsigned long prot, unsigned long addr)
 {
 	if (prot & ~(PROT_READ | PROT_WRITE | PROT_EXEC | PROT_SEM | PROT_SAO))
 		return false;
@@ -51,7 +51,7 @@ static inline bool arch_validate_prot(unsigned long prot)
 		return false;
 	return true;
 }
-#define arch_validate_prot(prot) arch_validate_prot(prot)
+#define arch_validate_prot arch_validate_prot
 
 #endif /* CONFIG_PPC64 */
 #endif	/* _ASM_POWERPC_MMAN_H */
arch/powerpc/kernel/syscalls.c +1 -1
index a877bf8269fe..6d90ddbd2d11 100644
@@ -48,7 +48,7 @@ static inline long do_mmap2(unsigned long addr, size_t len,
 {
 	long ret = -EINVAL;
 
-	if (!arch_validate_prot(prot))
+	if (!arch_validate_prot(prot, addr))
 		goto out;
 
 	if (shift) {
include/linux/mman.h +1 -1
index 6a4d1caaff5c..4b08e9c9c538 100644
@@ -92,7 +92,7 @@ static inline void vm_unacct_memory(long pages)
  *
  * Returns true if the prot flags are valid
  */
-static inline bool arch_validate_prot(unsigned long prot)
+static inline bool arch_validate_prot(unsigned long prot, unsigned long addr)
 {
 	return (prot & ~(PROT_READ | PROT_WRITE | PROT_EXEC | PROT_SEM)) == 0;
 }
mm/mprotect.c +1 -1
index e3309fcf586b..088ea9c08678 100644
@@ -417,7 +417,7 @@ static int do_mprotect_pkey(unsigned long start, size_t len,
 	end = start + len;
 	if (end <= start)
 		return -ENOMEM;
-	if (!arch_validate_prot(prot))
+	if (!arch_validate_prot(prot, start))
 		return -EINVAL;
 
 	reqprot = prot;
Signature · 1 line
2.11.0
#18 lore raw

PATCH v12 8/11 mm: Clear arch specific VM flags on protection change

In reply to Khalid Aziz

When protection bits are changed on a VMA, some of the architecture specific flags should be cleared as well. An examples of this are the PKEY flags on x86. This patch expands the current code that clears PKEY flags for x86, to support similar functionality for other architectures as well.

---
v7:
- new patch

 include/linux/mm.h | 6 ++++++
 mm/mprotect.c      | 2 +-
 2 files changed, 7 insertions(+), 1 deletion(-)
include/linux/mm.h +6 -0
index ad06d42adb1a..ae806dbc63ee 100644
@@ -287,6 +287,12 @@ extern unsigned int kobjsize(const void *objp);
 /* This mask is used to clear all the VMA flags used by mlock */
 #define VM_LOCKED_CLEAR_MASK	(~(VM_LOCKED | VM_LOCKONFAULT))
 
+/* Arch-specific flags to clear when updating VM flags on protection change */
+#ifndef VM_ARCH_CLEAR
+# define VM_ARCH_CLEAR	VM_NONE
+#endif
+#define VM_FLAGS_CLEAR	(ARCH_VM_PKEY_FLAGS | VM_ARCH_CLEAR)
+
 /*
  * mapping from the currently active vm_flags protection bits (the
  * low four bits) to a page protection mask..
mm/mprotect.c +1 -1
index 088ea9c08678..c1d6af7455da 100644
@@ -475,7 +475,7 @@ static int do_mprotect_pkey(unsigned long start, size_t len,
 		 * cleared from the VMA.
 		 */
 		mask_off_old_flags = VM_READ | VM_WRITE | VM_EXEC |
-					ARCH_VM_PKEY_FLAGS;
+					VM_FLAGS_CLEAR;
 
 		new_vma_pkey = arch_override_mprotect_pkey(vma, prot, pkey);
 		newflags = calc_vm_prot_bits(prot, new_vma_pkey);
Signature · 1 line
2.11.0
#22 lore raw

PATCH v12 9/11 mm: Allow arch code to override copy_highpage()

In reply to Khalid Aziz

Some architectures can support metadata for memory pages and when a page is copied, its metadata must also be copied. Sparc processors from M7 onwards support metadata for memory pages. This metadata provides tag based protection for access to memory pages. To maintain this protection, the tag data must be copied to the new page when a page is migrated across NUMA nodes. This patch allows arch specific code to override default copy_highpage() and copy metadata along with page data upon migration.

---
v9:
- new patch

 include/linux/highmem.h | 4 ++++
 1 file changed, 4 insertions(+)
include/linux/highmem.h +4 -0
index 776f90f3a1cd..0690679832d4 100644
@@ -237,6 +237,8 @@ static inline void copy_user_highpage(struct page *to, struct page *from,
 
 #endif
 
+#ifndef __HAVE_ARCH_COPY_HIGHPAGE
+
 static inline void copy_highpage(struct page *to, struct page *from)
 {
 	char *vfrom, *vto;
@@ -248,4 +250,6 @@ static inline void copy_highpage(struct page *to, struct page *from)
 	kunmap_atomic(vfrom);
 }
 
+#endif
+
 #endif /* _LINUX_HIGHMEM_H */
Signature · 1 line
2.11.0
#27 lore raw

PATCH v12 10/11 sparc64: Add support for ADI (Application Data Integrity)

In reply to Khalid Aziz

ADI is a new feature supported on SPARC M7 and newer processors to allow hardware to catch rogue accesses to memory. ADI is supported for data fetches only and not instruction fetches. An app can enable ADI on its data pages, set version tags on them and use versioned addresses to access the data pages. Upper bits of the address contain the version tag. On M7 processors, upper four bits (bits 63-60) contain the version tag. If a rogue app attempts to access ADI enabled data pages, its access is blocked and processor generates an exception. Please see Documentation/sparc/adi.txt for further details.

This patch extends mprotect to enable ADI (TSTATE.mcde), enable/disable MCD (Memory Corruption Detection) on selected memory ranges, enable TTE.mcd in PTEs, return ADI parameters to userspace and save/restore ADI version tags on page swap out/in or migration. ADI is not enabled by default for any task. A task must explicitly enable ADI on a memory range and set version tag for ADI to be effective for the task.

---
v10:
- Added code to return from kernel path to set PSTATE.mcde if kernel continues execution in another thread (Suggested by Anthony Yznaga)
v9:
- Added code to migrate ADI tags to copy_highpage() to ensure tags get copied on page migration
- Improved code to detect underflow and overflow when allocating tag storage
v8:
- Added note to doc about non-faulting loads not triggering ADI tag mismatch and more details on special tag values of 0x0 and 0xf, as suggested by Anthony Yznaga)
- Added an IPI on mprotect(...PROT_ADI...) call to set TSTATE.MCDE on threads running on other processors and restore of TSTATE.MCDE on context switch (suggested by David Miller)
- Removed restriction on enabling ADI on read-only memory (suggested by Anthony Yznaga)
- Changed kzalloc() for tag storage to use GFP_NOWAIT
- Added code to handle overflow and underflow when allocating tag storage, as suggested by Anthony Yznaga
- Replaced sun_m7_patch_1insn_range() with sun4v_patch_1insn_range() which is functionally identical (suggested by Anthony Yznaga)
- Added membar after restoring ADI tags in copy_user_highpage(), as suggested by David Miller

v7:
- Enhanced arch_validate_prot() to enable ADI only on writable addresses backed by physical RAM
- Added support for saving/restoring ADI tags for each ADI block size address range on a page on swap in/out
- Added code to copy ADI tags on COW
- Updated values for auxiliary vectors to not conflict with values on other architectures to avoid conflict in glibc. glibc consolidates all auxiliary vectors into its headers and duplicate values in consolidated header are problematic
- Disable same page merging on ADI enabled pages since ADI tags may not match on pages with identical data
- Broke the patch up further into smaller patches

v6:
- Eliminated instructions to read and write PSTATE as well as MCDPER and PMCDPER on every access to userspace addresses by setting PSTATE and PMCDPER correctly upon entry into kernel. PSTATE.mcde and PMCDPER are set upon entry into kernel when running on an M7 processor. PSTATE.mcde being set only affects memory accesses that have TTE.mcd set. PMCDPER being set only affects writes to memory addresses that have TTE.mcd set. This ensures any faults caused by ADI tag mismatch on a write are exposed before kernel returns to userspace.

v5:
- Fixed indentation issues and instrcuctions in assembly code
- Removed CONFIG_SPARC64 from mdesc.c
- Changed to maintain state of MCDPER register in thread info flags as opposed to in mm context. MCDPER is a per-thread state and belongs in thread info flag as opposed to mm context which is shared across threads. Added comments to clarify this is a lazily maintained state and must be updated on context switch and copy_process()
- Updated code to use the new arch_do_swap_page() and arch_unmap_one() functions

v4:
- Broke patch up into smaller patches

v3:
- Removed CONFIG_SPARC_ADI
- Replaced prctl commands with mprotect
- Added auxiliary vectors for ADI parameters
- Enabled ADI for swappable pages

v2:
- Fixed a build error

 Documentation/sparc/adi.txt             | 278 +++++++++++++++++++++++++++++
 arch/sparc/include/asm/mman.h           |  84 ++++++++-
 arch/sparc/include/asm/mmu_64.h         |  17 ++
 arch/sparc/include/asm/mmu_context_64.h |  50 ++++++
 arch/sparc/include/asm/page_64.h        |   6 +
 arch/sparc/include/asm/pgtable_64.h     |  46 +++++
 arch/sparc/include/asm/thread_info_64.h |   2 +-
 arch/sparc/include/asm/trap_block.h     |   2 +
 arch/sparc/include/uapi/asm/mman.h      |   2 +
 arch/sparc/kernel/adi_64.c              | 301 ++++++++++++++++++++++++++++++++
 arch/sparc/kernel/etrap_64.S            |  27 ++-
 arch/sparc/kernel/process_64.c          |  25 +++
 arch/sparc/kernel/rtrap_64.S            |  33 +++-
 arch/sparc/kernel/setup_64.c            |   2 +
 arch/sparc/kernel/urtt_fill.S           |   7 +-
 arch/sparc/kernel/vmlinux.lds.S         |   5 +
 arch/sparc/mm/gup.c                     |  37 ++++
 arch/sparc/mm/hugetlbpage.c             |  14 +-
 arch/sparc/mm/init_64.c                 |  69 ++++++++
 arch/sparc/mm/tsb.c                     |  21 +++
 include/linux/mm.h                      |   3 +
 mm/ksm.c                                |   4 +
 22 files changed, 1027 insertions(+), 8 deletions(-)
 create mode 100644 Documentation/sparc/adi.txt
Documentation/sparc/adi.txt new file +278 -0
new file mode 100644
index 000000000000..e1aed155fb89
@@ -0,0 +1,278 @@
+Application Data Integrity (ADI)
+================================
+
+SPARC M7 processor adds the Application Data Integrity (ADI) feature.
+ADI allows a task to set version tags on any subset of its address
+space. Once ADI is enabled and version tags are set for ranges of
+address space of a task, the processor will compare the tag in pointers
+to memory in these ranges to the version set by the application
+previously. Access to memory is granted only if the tag in given pointer
+matches the tag set by the application. In case of mismatch, processor
+raises an exception.
+
+Following steps must be taken by a task to enable ADI fully:
+
+1. Set the user mode PSTATE.mcde bit. This acts as master switch for
+   the task's entire address space to enable/disable ADI for the task.
+
+2. Set TTE.mcd bit on any TLB entries that correspond to the range of
+   addresses ADI is being enabled on. MMU checks the version tag only
+   on the pages that have TTE.mcd bit set.
+
+3. Set the version tag for virtual addresses using stxa instruction
+   and one of the MCD specific ASIs. Each stxa instruction sets the
+   given tag for one ADI block size number of bytes. This step must
+   be repeated for entire page to set tags for entire page.
+
+ADI block size for the platform is provided by the hypervisor to kernel
+in machine description tables. Hypervisor also provides the number of
+top bits in the virtual address that specify the version tag.  Once
+version tag has been set for a memory location, the tag is stored in the
+physical memory and the same tag must be present in the ADI version tag
+bits of the virtual address being presented to the MMU. For example on
+SPARC M7 processor, MMU uses bits 63-60 for version tags and ADI block
+size is same as cacheline size which is 64 bytes. A task that sets ADI
+version to, say 10, on a range of memory, must access that memory using
+virtual addresses that contain 0xa in bits 63-60.
+
+ADI is enabled on a set of pages using mprotect() with PROT_ADI flag.
+When ADI is enabled on a set of pages by a task for the first time,
+kernel sets the PSTATE.mcde bit fot the task. Version tags for memory
+addresses are set with an stxa instruction on the addresses using
+ASI_MCD_PRIMARY or ASI_MCD_ST_BLKINIT_PRIMARY. ADI block size is
+provided by the hypervisor to the kernel.  Kernel returns the value of
+ADI block size to userspace using auxiliary vector along with other ADI
+info. Following auxiliary vectors are provided by the kernel:
+
+	AT_ADI_BLKSZ	ADI block size. This is the granularity and
+			alignment, in bytes, of ADI versioning.
+	AT_ADI_NBITS	Number of ADI version bits in the VA
+
+
+IMPORTANT NOTES:
+
+- Version tag values of 0x0 and 0xf are reserved. These values match any
+  tag in virtual address and never generate a mismatch exception.
+
+- Version tags are set on virtual addresses from userspace even though
+  tags are stored in physical memory. Tags are set on a physical page
+  after it has been allocated to a task and a pte has been created for
+  it.
+
+- When a task frees a memory page it had set version tags on, the page
+  goes back to free page pool. When this page is re-allocated to a task,
+  kernel clears the page using block initialization ASI which clears the
+  version tags as well for the page. If a page allocated to a task is
+  freed and allocated back to the same task, old version tags set by the
+  task on that page will no longer be present.
+
+- ADI tag mismatches are not detected for non-faulting loads.
+
+- Kernel does not set any tags for user pages and it is entirely a
+  task's responsibility to set any version tags. Kernel does ensure the
+  version tags are preserved if a page is swapped out to the disk and
+  swapped back in. It also preserves that version tags if a page is
+  migrated.
+
+- ADI works for any size pages. A userspace task need not be aware of
+  page size when using ADI. It can simply select a virtual address
+  range, enable ADI on the range using mprotect() and set version tags
+  for the entire range. mprotect() ensures range is aligned to page size
+  and is a multiple of page size.
+
+- ADI tags can only be set on writable memory. For example, ADI tags can
+  not be set on read-only mappings.
+
+
+
+ADI related traps
+-----------------
+
+With ADI enabled, following new traps may occur:
+
+Disrupting memory corruption
+
+	When a store accesses a memory localtion that has TTE.mcd=1,
+	the task is running with ADI enabled (PSTATE.mcde=1), and the ADI
+	tag in the address used (bits 63:60) does not match the tag set on
+	the corresponding cacheline, a memory corruption trap occurs. By
+	default, it is a disrupting trap and is sent to the hypervisor
+	first. Hypervisor creates a sun4v error report and sends a
+	resumable error (TT=0x7e) trap to the kernel. The kernel sends
+	a SIGSEGV to the task that resulted in this trap with the following
+	info:
+
+		siginfo.si_signo = SIGSEGV;
+		siginfo.errno = 0;
+		siginfo.si_code = SEGV_ADIDERR;
+		siginfo.si_addr = addr; /* PC where first mismatch occurred */
+		siginfo.si_trapno = 0;
+
+
+Precise memory corruption
+
+	When a store accesses a memory location that has TTE.mcd=1,
+	the task is running with ADI enabled (PSTATE.mcde=1), and the ADI
+	tag in the address used (bits 63:60) does not match the tag set on
+	the corresponding cacheline, a memory corruption trap occurs. If
+	MCD precise exception is enabled (MCDPERR=1), a precise
+	exception is sent to the kernel with TT=0x1a. The kernel sends
+	a SIGSEGV to the task that resulted in this trap with the following
+	info:
+
+		siginfo.si_signo = SIGSEGV;
+		siginfo.errno = 0;
+		siginfo.si_code = SEGV_ADIPERR;
+		siginfo.si_addr = addr;	/* address that caused trap */
+		siginfo.si_trapno = 0;
+
+	NOTE: ADI tag mismatch on a load always results in precise trap.
+
+
+MCD disabled
+
+	When a task has not enabled ADI and attempts to set ADI version
+	on a memory address, processor sends an MCD disabled trap. This
+	trap is handled by hypervisor first and the hypervisor vectors this
+	trap through to the kernel as Data Access Exception trap with
+	fault type set to 0xa (invalid ASI). When this occurs, the kernel
+	sends the task SIGSEGV signal with following info:
+
+		siginfo.si_signo = SIGSEGV;
+		siginfo.errno = 0;
+		siginfo.si_code = SEGV_ACCADI;
+		siginfo.si_addr = addr;	/* address that caused trap */
+		siginfo.si_trapno = 0;
+
+
+Sample program to use ADI
+-------------------------
+
+Following sample program is meant to illustrate how to use the ADI
+functionality.
+
+#include <unistd.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <elf.h>
+#include <sys/ipc.h>
+#include <sys/shm.h>
+#include <sys/mman.h>
+#include <asm/asi.h>
+
+#ifndef AT_ADI_BLKSZ
+#define AT_ADI_BLKSZ	48
+#endif
+#ifndef AT_ADI_NBITS
+#define AT_ADI_NBITS	49
+#endif
+
+#ifndef PROT_ADI
+#define PROT_ADI	0x10
+#endif
+
+#define BUFFER_SIZE     32*1024*1024UL
+
+main(int argc, char* argv[], char* envp[])
+{
+        unsigned long i, mcde, adi_blksz, adi_nbits;
+        char *shmaddr, *tmp_addr, *end, *veraddr, *clraddr;
+        int shmid, version;
+	Elf64_auxv_t *auxv;
+
+	adi_blksz = 0;
+
+	while(*envp++ != NULL);
+	for (auxv = (Elf64_auxv_t *)envp; auxv->a_type != AT_NULL; auxv++) {
+		switch (auxv->a_type) {
+		case AT_ADI_BLKSZ:
+			adi_blksz = auxv->a_un.a_val;
+			break;
+		case AT_ADI_NBITS:
+			adi_nbits = auxv->a_un.a_val;
+			break;
+		}
+	}
+	if (adi_blksz == 0) {
+		fprintf(stderr, "Oops! ADI is not supported\n");
+		exit(1);
+	}
+
+	printf("ADI capabilities:\n");
+	printf("\tBlock size = %ld\n", adi_blksz);
+	printf("\tNumber of bits = %ld\n", adi_nbits);
+
+        if ((shmid = shmget(2, BUFFER_SIZE,
+                                IPC_CREAT | SHM_R | SHM_W)) < 0) {
+                perror("shmget failed");
+                exit(1);
+        }
+
+        shmaddr = shmat(shmid, NULL, 0);
+        if (shmaddr == (char *)-1) {
+                perror("shm attach failed");
+                shmctl(shmid, IPC_RMID, NULL);
+                exit(1);
+        }
+
+	if (mprotect(shmaddr, BUFFER_SIZE, PROT_READ|PROT_WRITE|PROT_ADI)) {
+		perror("mprotect failed");
+		goto err_out;
+	}
+
+        /* Set the ADI version tag on the shm segment
+         */
+        version = 10;
+        tmp_addr = shmaddr;
+        end = shmaddr + BUFFER_SIZE;
+        while (tmp_addr < end) {
+                asm volatile(
+                        "stxa %1, [%0]0x90\n\t"
+                        :
+                        : "r" (tmp_addr), "r" (version));
+                tmp_addr += adi_blksz;
+        }
+	asm volatile("membar #Sync\n\t");
+
+        /* Create a versioned address from the normal address by placing
+	 * version tag in the upper adi_nbits bits
+         */
+        tmp_addr = (void *) ((unsigned long)shmaddr << adi_nbits);
+        tmp_addr = (void *) ((unsigned long)tmp_addr >> adi_nbits);
+        veraddr = (void *) (((unsigned long)version << (64-adi_nbits))
+                        | (unsigned long)tmp_addr);
+
+        printf("Starting the writes:\n");
+        for (i = 0; i < BUFFER_SIZE; i++) {
+                veraddr[i] = (char)(i);
+                if (!(i % (1024 * 1024)))
+                        printf(".");
+        }
+        printf("\n");
+
+        printf("Verifying data...");
+	fflush(stdout);
+        for (i = 0; i < BUFFER_SIZE; i++)
+                if (veraddr[i] != (char)i)
+                        printf("\nIndex %lu mismatched\n", i);
+        printf("Done.\n");
+
+        /* Disable ADI and clean up
+         */
+	if (mprotect(shmaddr, BUFFER_SIZE, PROT_READ|PROT_WRITE)) {
+		perror("mprotect failed");
+		goto err_out;
+	}
+
+        if (shmdt((const void *)shmaddr) != 0)
+                perror("Detach failure");
+        shmctl(shmid, IPC_RMID, NULL);
+
+        exit(0);
+
+err_out:
+        if (shmdt((const void *)shmaddr) != 0)
+                perror("Detach failure");
+        shmctl(shmid, IPC_RMID, NULL);
+        exit(1);
+}
arch/sparc/include/asm/mman.h +83 -1

88 lines not on this page · raw message

arch/sparc/include/asm/mmu_64.h +17 -0
index ad4fb93508ba..7e2704c770e9 100644
@@ -90,6 +90,20 @@ struct tsb_config {
 #define MM_NUM_TSBS	1
 #endif
 
+/* ADI tags are stored when a page is swapped out and the storage for
+ * tags is allocated dynamically. There is a tag storage descriptor
+ * associated with each set of tag storage pages. Tag storage descriptors
+ * are allocated dynamically. Since kernel will allocate a full page for
+ * each tag storage descriptor, we can store up to
+ * PAGE_SIZE/sizeof(tag storage descriptor) descriptors on that page.
+ */
+typedef struct {
+	unsigned long	start;		/* Start address for this tag storage */
+	unsigned long	end;		/* Last address for tag storage */
+	unsigned char	*tags;		/* Where the tags are */
+	unsigned long	tag_users;	/* number of references to descriptor */
+} tag_storage_desc_t;
+
 typedef struct {
 	spinlock_t		lock;
 	unsigned long		sparc64_ctx_val;
@@ -98,6 +112,9 @@ typedef struct {
 	struct tsb_config	tsb_block[MM_NUM_TSBS];
 	struct hv_tsb_descr	tsb_descr[MM_NUM_TSBS];
 	void			*vdso;
+	bool			adi;
+	tag_storage_desc_t	*tag_store;
+	spinlock_t		tag_lock;
 } mm_context_t;
 
 #endif /* !__ASSEMBLY__ */
arch/sparc/include/asm/mmu_context_64.h +50 -0

62 lines not on this page · raw message

arch/sparc/include/asm/page_64.h +6 -0
index c28379b1b0fc..e80f2d5bf62f 100644
@@ -48,6 +48,12 @@ struct page;
 void clear_user_page(void *addr, unsigned long vaddr, struct page *page);
 #define copy_page(X,Y)	memcpy((void *)(X), (void *)(Y), PAGE_SIZE)
 void copy_user_page(void *to, void *from, unsigned long vaddr, struct page *topage);
+#define __HAVE_ARCH_COPY_USER_HIGHPAGE
+struct vm_area_struct;
+void copy_user_highpage(struct page *to, struct page *from,
+			unsigned long vaddr, struct vm_area_struct *vma);
+#define __HAVE_ARCH_COPY_HIGHPAGE
+void copy_highpage(struct page *to, struct page *from);
 
 /* Unlike sparc32, sparc64's parameter passing API is more
  * sane in that structures which as small enough are passed
arch/sparc/include/asm/pgtable_64.h +46 -0

64 lines not on this page · raw message

arch/sparc/include/asm/thread_info_64.h +1 -1
index f7e7b0baec9f..7fb676360928 100644
@@ -188,7 +188,7 @@ register struct thread_info *current_thread_info_reg asm("g6");
  *       in using in assembly, else we can't use the mask as
  *       an immediate value in instructions such as andcc.
  */
-/* flag bit 12 is available */
+#define TIF_MCDPER		12	/* Precise MCD exception */
 #define TIF_MEMDIE		13	/* is terminating due to OOM killer */
 #define TIF_POLLING_NRFLAG	14
 
arch/sparc/include/asm/trap_block.h +2 -0
index 6a4c8652ad67..0f6d0c4f6683 100644
@@ -76,6 +76,8 @@ extern struct sun4v_1insn_patch_entry __sun4v_1insn_patch,
 	__sun4v_1insn_patch_end;
 extern struct sun4v_1insn_patch_entry __fast_win_ctrl_1insn_patch,
 	__fast_win_ctrl_1insn_patch_end;
+extern struct sun4v_1insn_patch_entry __sun_m7_1insn_patch,
+	__sun_m7_1insn_patch_end;
 
 struct sun4v_2insn_patch_entry {
 	unsigned int	addr;
arch/sparc/include/uapi/asm/mman.h +2 -0
index 715a2c927e79..f6f99ec65bb3 100644
@@ -6,6 +6,8 @@
 
 /* SunOS'ified... */
 
+#define PROT_ADI	0x10		/* ADI enabled */
+
 #define MAP_RENAME      MAP_ANONYMOUS   /* In SunOS terminology */
 #define MAP_NORESERVE   0x40            /* don't reserve swap pages */
 #define MAP_INHERIT     0x80            /* SunOS doesn't do this, but... */
arch/sparc/kernel/adi_64.c +301 -0

320 lines not on this page · raw message

arch/sparc/kernel/etrap_64.S +26 -1
index 5c77a2e0e991..08cc41f64725 100644
@@ -151,7 +151,32 @@ etrap_save:	save	%g2, -STACK_BIAS, %sp
 		stx	%g6, [%sp + PTREGS_OFF + PT_V9_G6]
 		stx	%g7, [%sp + PTREGS_OFF + PT_V9_G7]
 		or	%l7, %l0, %l7
-		sethi	%hi(TSTATE_TSO | TSTATE_PEF), %l0
+661:		sethi	%hi(TSTATE_TSO | TSTATE_PEF), %l0
+		/* If userspace is using ADI, it could potentially pass
+		 * a pointer with version tag embedded in it. To maintain
+		 * the ADI security, we must enable PSTATE.mcde. Userspace
+		 * would have already set TTE.mcd in an earlier call to
+		 * kernel and set the version tag for the address being
+		 * dereferenced. Setting PSTATE.mcde would ensure any
+		 * access to userspace data through a system call honors
+		 * ADI and does not allow a rogue app to bypass ADI by
+		 * using system calls. Setting PSTATE.mcde only affects
+		 * accesses to virtual addresses that have TTE.mcd set.
+		 * Set PMCDPER to ensure any exceptions caused by ADI
+		 * version tag mismatch are exposed before system call
+		 * returns to userspace. Setting PMCDPER affects only
+		 * writes to virtual addresses that have TTE.mcd set and
+		 * have a version tag set as well.
+		 */
+		.section .sun_m7_1insn_patch, "ax"
+		.word	661b
+		sethi	%hi(TSTATE_TSO | TSTATE_PEF | TSTATE_MCDE), %l0
+		.previous
+661:		nop
+		.section .sun_m7_1insn_patch, "ax"
+		.word	661b
+		.word 0xaf902001	/* wrpr %g0, 1, %pmcdper */
+		.previous
 		or	%l7, %l0, %l7
 		wrpr	%l2, %tnpc
 		wrpr	%l7, (TSTATE_PRIV | TSTATE_IE), %tstate
arch/sparc/kernel/process_64.c +25 -0
index 318efd784a0b..454a8af28f13 100644
@@ -670,6 +670,31 @@ int copy_thread(unsigned long clone_flags, unsigned long sp,
 	return 0;
 }
 
+/* TIF_MCDPER in thread info flags for current task is updated lazily upon
+ * a context switch. Update this flag in current task's thread flags
+ * before dup so the dup'd task will inherit the current TIF_MCDPER flag.
+ */
+int arch_dup_task_struct(struct task_struct *dst, struct task_struct *src)
+{
+	if (adi_capable()) {
+		register unsigned long tmp_mcdper;
+
+		__asm__ __volatile__(
+			".word 0x83438000\n\t"	/* rd  %mcdper, %g1 */
+			"mov %%g1, %0\n\t"
+			: "=r" (tmp_mcdper)
+			:
+			: "g1");
+		if (tmp_mcdper)
+			set_thread_flag(TIF_MCDPER);
+		else
+			clear_thread_flag(TIF_MCDPER);
+	}
+
+	*dst = *src;
+	return 0;
+}
+
 typedef struct {
 	union {
 		unsigned int	pr_regs[32];
arch/sparc/kernel/rtrap_64.S +30 -3
index 0b21042ab181..f6528884a2c8 100644
@@ -25,13 +25,31 @@
 		.align			32
 __handle_preemption:
 		call			SCHEDULE_USER
-		 wrpr			%g0, RTRAP_PSTATE, %pstate
+661:		 wrpr			%g0, RTRAP_PSTATE, %pstate
+		/* If userspace is using ADI, it could potentially pass
+		 * a pointer with version tag embedded in it. To maintain
+		 * the ADI security, we must re-enable PSTATE.mcde before
+		 * we continue execution in the kernel for another thread.
+		 */
+		.section .sun_m7_1insn_patch, "ax"
+		.word	661b
+		 wrpr			%g0, RTRAP_PSTATE|PSTATE_MCDE, %pstate
+		.previous
 		ba,pt			%xcc, __handle_preemption_continue
 		 wrpr			%g0, RTRAP_PSTATE_IRQOFF, %pstate
 
 __handle_user_windows:
 		call			fault_in_user_windows
-		 wrpr			%g0, RTRAP_PSTATE, %pstate
+661:		 wrpr			%g0, RTRAP_PSTATE, %pstate
+		/* If userspace is using ADI, it could potentially pass
+		 * a pointer with version tag embedded in it. To maintain
+		 * the ADI security, we must re-enable PSTATE.mcde before
+		 * we continue execution in the kernel for another thread.
+		 */
+		.section .sun_m7_1insn_patch, "ax"
+		.word	661b
+		 wrpr			%g0, RTRAP_PSTATE|PSTATE_MCDE, %pstate
+		.previous
 		ba,pt			%xcc, __handle_preemption_continue
 		 wrpr			%g0, RTRAP_PSTATE_IRQOFF, %pstate
 
@@ -48,7 +66,16 @@ __handle_signal:
 		add			%sp, PTREGS_OFF, %o0
 		mov			%l0, %o2
 		call			do_notify_resume
-		 wrpr			%g0, RTRAP_PSTATE, %pstate
+661:		 wrpr			%g0, RTRAP_PSTATE, %pstate
+		/* If userspace is using ADI, it could potentially pass
+		 * a pointer with version tag embedded in it. To maintain
+		 * the ADI security, we must re-enable PSTATE.mcde before
+		 * we continue execution in the kernel for another thread.
+		 */
+		.section .sun_m7_1insn_patch, "ax"
+		.word	661b
+		 wrpr			%g0, RTRAP_PSTATE|PSTATE_MCDE, %pstate
+		.previous
 		wrpr			%g0, RTRAP_PSTATE_IRQOFF, %pstate
 
 		/* Signal delivery can modify pt_regs tstate, so we must
arch/sparc/kernel/setup_64.c +2 -0
index 34f7a533a74f..7944b3ca216a 100644
@@ -294,6 +294,8 @@ static void __init sun4v_patch(void)
 	case SUN4V_CHIP_SPARC_M7:
 	case SUN4V_CHIP_SPARC_M8:
 	case SUN4V_CHIP_SPARC_SN:
+		sun4v_patch_1insn_range(&__sun_m7_1insn_patch,
+					&__sun_m7_1insn_patch_end);
 		sun_m7_patch_2insn_range(&__sun_m7_2insn_patch,
 					 &__sun_m7_2insn_patch_end);
 		break;
arch/sparc/kernel/urtt_fill.S +6 -1
index 44183aa59168..e4cee7be5cd0 100644
@@ -50,7 +50,12 @@ user_rtt_fill_fixup_common:
 		SET_GL(0)
 		.previous
 
-		wrpr	%g0, RTRAP_PSTATE, %pstate
+661:		wrpr	%g0, RTRAP_PSTATE, %pstate
+		.section		.sun_m7_1insn_patch, "ax"
+		.word			661b
+		/* Re-enable PSTATE.mcde to maintain ADI security */
+		wrpr	%g0, RTRAP_PSTATE|PSTATE_MCDE, %pstate
+		.previous
 
 		mov	%l1, %g6
 		ldx	[%g6 + TI_TASK], %g4
arch/sparc/kernel/vmlinux.lds.S +5 -0
index 5a2344574f39..61afd787bd0c 100644
@@ -145,6 +145,11 @@ SECTIONS
 		*(.pause_3insn_patch)
 		__pause_3insn_patch_end = .;
 	}
+	.sun_m7_1insn_patch : {
+		__sun_m7_1insn_patch = .;
+		*(.sun_m7_1insn_patch)
+		__sun_m7_1insn_patch_end = .;
+	}
 	.sun_m7_2insn_patch : {
 		__sun_m7_2insn_patch = .;
 		*(.sun_m7_2insn_patch)
arch/sparc/mm/gup.c +37 -0
index 5335ba3c850e..357b6047653a 100644
@@ -12,6 +12,7 @@
 #include <linux/pagemap.h>
 #include <linux/rwsem.h>
 #include <asm/pgtable.h>
+#include <asm/adi.h>
 
 /*
  * The performance critical leaf functions are made noinline otherwise gcc
@@ -201,6 +202,24 @@ int __get_user_pages_fast(unsigned long start, int nr_pages, int write,
 	pgd_t *pgdp;
 	int nr = 0;
 
+#ifdef CONFIG_SPARC64
+	if (adi_capable()) {
+		long addr = start;
+
+		/* If userspace has passed a versioned address, kernel
+		 * will not find it in the VMAs since it does not store
+		 * the version tags in the list of VMAs. Storing version
+		 * tags in list of VMAs is impractical since they can be
+		 * changed any time from userspace without dropping into
+		 * kernel. Any address search in VMAs will be done with
+		 * non-versioned addresses. Ensure the ADI version bits
+		 * are dropped here by sign extending the last bit before
+		 * ADI bits. IOMMU does not implement version tags.
+		 */
+		addr = (addr << (long)adi_nbits()) >> (long)adi_nbits();
+		start = addr;
+	}
+#endif
 	start &= PAGE_MASK;
 	addr = start;
 	len = (unsigned long) nr_pages << PAGE_SHIFT;
@@ -231,6 +250,24 @@ int get_user_pages_fast(unsigned long start, int nr_pages, int write,
 	pgd_t *pgdp;
 	int nr = 0;
 
+#ifdef CONFIG_SPARC64
+	if (adi_capable()) {
+		long addr = start;
+
+		/* If userspace has passed a versioned address, kernel
+		 * will not find it in the VMAs since it does not store
+		 * the version tags in the list of VMAs. Storing version
+		 * tags in list of VMAs is impractical since they can be
+		 * changed any time from userspace without dropping into
+		 * kernel. Any address search in VMAs will be done with
+		 * non-versioned addresses. Ensure the ADI version bits
+		 * are dropped here by sign extending the last bit before
+		 * ADI bits. IOMMU does not implements version tags,
+		 */
+		addr = (addr << (long)adi_nbits()) >> (long)adi_nbits();
+		start = addr;
+	}
+#endif
 	start &= PAGE_MASK;
 	addr = start;
 	len = (unsigned long) nr_pages << PAGE_SHIFT;
arch/sparc/mm/hugetlbpage.c +13 -1
index 0112d6942288..f78793a06bbd 100644
@@ -182,8 +182,20 @@ pte_t arch_make_huge_pte(pte_t entry, struct vm_area_struct *vma,
 			 struct page *page, int writeable)
 {
 	unsigned int shift = huge_page_shift(hstate_vma(vma));
+	pte_t pte;
 
-	return hugepage_shift_to_tte(entry, shift);
+	pte = hugepage_shift_to_tte(entry, shift);
+
+#ifdef CONFIG_SPARC64
+	/* If this vma has ADI enabled on it, turn on TTE.mcd
+	 */
+	if (vma->vm_flags & VM_SPARC_ADI)
+		return pte_mkmcd(pte);
+	else
+		return pte_mknotmcd(pte);
+#else
+	return pte;
+#endif
 }
 
 static unsigned int sun4v_huge_tte_to_shift(pte_t entry)
arch/sparc/mm/init_64.c +69 -0

72 lines not on this page · raw message

arch/sparc/mm/tsb.c +21 -0
index 75a04c1a2383..f5edc28aa3a5 100644
@@ -546,6 +546,9 @@ int init_new_context(struct task_struct *tsk, struct mm_struct *mm)
 
 	mm->context.sparc64_ctx_val = 0UL;
 
+	mm->context.tag_store = NULL;
+	spin_lock_init(&mm->context.tag_lock);
+
 #if defined(CONFIG_HUGETLB_PAGE) || defined(CONFIG_TRANSPARENT_HUGEPAGE)
 	/* We reset them to zero because the fork() page copying
 	 * will re-increment the counters as the parent PTEs are
@@ -611,4 +614,22 @@ void destroy_context(struct mm_struct *mm)
 	}
 
 	spin_unlock_irqrestore(&ctx_alloc_lock, flags);
+
+	/* If ADI tag storage was allocated for this task, free it */
+	if (mm->context.tag_store) {
+		tag_storage_desc_t *tag_desc;
+		unsigned long max_desc;
+		unsigned char *tags;
+
+		tag_desc = mm->context.tag_store;
+		max_desc = PAGE_SIZE/sizeof(tag_storage_desc_t);
+		for (i = 0; i < max_desc; i++) {
+			tags = tag_desc->tags;
+			tag_desc->tags = NULL;
+			kfree(tags);
+			tag_desc++;
+		}
+		kfree(mm->context.tag_store);
+		mm->context.tag_store = NULL;
+	}
 }
include/linux/mm.h +3 -0
index ae806dbc63ee..32fe6919a11b 100644
@@ -245,6 +245,9 @@ extern unsigned int kobjsize(const void *objp);
 # define VM_GROWSUP	VM_ARCH_1
 #elif defined(CONFIG_IA64)
 # define VM_GROWSUP	VM_ARCH_1
+#elif defined(CONFIG_SPARC64)
+# define VM_SPARC_ADI	VM_ARCH_1	/* Uses ADI tag for access control */
+# define VM_ARCH_CLEAR	VM_SPARC_ADI
 #elif !defined(CONFIG_MMU)
 # define VM_MAPPED_COPY	VM_ARCH_1	/* T if mapped copy of data (nommu mmap) */
 #endif
mm/ksm.c +4 -0
index 293721f5da70..adb5f991da8e 100644
@@ -2369,6 +2369,10 @@ int ksm_madvise(struct vm_area_struct *vma, unsigned long start,
 		if (*vm_flags & VM_SAO)
 			return 0;
 #endif
+#ifdef VM_SPARC_ADI
+		if (*vm_flags & VM_SPARC_ADI)
+			return 0;
+#endif
 
 		if (!test_bit(MMF_VM_MERGEABLE, &mm->flags)) {
 			err = __ksm_enter(mm);
Signature · 1 line
2.11.0
#40 lore raw

PATCH v12 11/11 sparc64: Update signal delivery to use new helper functions

In reply to Khalid Aziz

Commit f8ec66014ffd ("signal: Add send_sig_fault and force_sig_fault") added new helper functions to streamline signal delivery. This patch updates signal delivery for new/updated handlers for ADI related exceptions to use the helper function.

---
v12:
- new patch

 arch/sparc/kernel/traps_64.c | 38 +++++++++-----------------------------
 1 file changed, 9 insertions(+), 29 deletions(-)
arch/sparc/kernel/traps_64.c +9 -29
index f36a40217f6c..462a21abd105 100644
@@ -362,7 +362,6 @@ void sun4v_data_access_exception(struct pt_regs *regs, unsigned long addr, unsig
 {
 	unsigned short type = (type_ctx >> 16);
 	unsigned short ctx  = (type_ctx & 0xffff);
-	siginfo_t info;
 
 	if (notify_die(DIE_TRAP, "data access exception", regs,
 		       0, 0x8, SIGTRAP) = NOTIFY_STOP)
@@ -406,24 +405,18 @@ void sun4v_data_access_exception(struct pt_regs *regs, unsigned long addr, unsig
 	 * kerbel by HV as data access exception with fault type set to
 	 * HV_FAULT_TYPE_INV_ASI.
 	 */
-	info.si_errno = 0;
-	info.si_addr = (void __user *) addr;
-	info.si_trapno = 0;
 	switch (type) {
 	case HV_FAULT_TYPE_INV_ASI:
-		info.si_signo = SIGILL;
-		info.si_code = ILL_ILLADR;
-		force_sig_info(SIGILL, &info, current);
+		force_sig_fault(SIGILL, ILL_ILLADR, (void __user *)addr, 0,
+				current);
 		break;
 	case HV_FAULT_TYPE_MCD_DIS:
-		info.si_signo = SIGSEGV;
-		info.si_code = SEGV_ACCADI;
-		force_sig_info(SIGSEGV, &info, current);
+		force_sig_fault(SIGSEGV, SEGV_ACCADI, (void __user *)addr, 0,
+				current);
 		break;
 	default:
-		info.si_signo = SIGSEGV;
-		info.si_code = SEGV_MAPERR;
-		force_sig_info(SIGSEGV, &info, current);
+		force_sig_fault(SIGSEGV, SEGV_MAPERR, (void __user *)addr, 0,
+				current);
 		break;
 	}
 }
@@ -2073,8 +2066,6 @@ static void sun4v_log_error(struct pt_regs *regs, struct sun4v_error_entry *ent,
  */
 void do_mcd_err(struct pt_regs *regs, struct sun4v_error_entry ent)
 {
-	siginfo_t info;
-
 	if (notify_die(DIE_TRAP, "MCD error", regs, 0, 0x34,
 		       SIGSEGV) = NOTIFY_STOP)
 		return;
@@ -2110,12 +2101,8 @@ void do_mcd_err(struct pt_regs *regs, struct sun4v_error_entry ent)
 	/* Send SIGSEGV to the userspace process with the right signal
 	 * code
 	 */
-	info.si_signo = SIGSEGV;
-	info.si_errno = 0;
-	info.si_code = SEGV_ADIDERR;
-	info.si_addr = (void __user *)ent.err_raddr;
-	info.si_trapno = 0;
-	force_sig_info(SIGSEGV, &info, current);
+	force_sig_fault(SIGSEGV, SEGV_ADIDERR, (void __user *)ent.err_raddr,
+			0, current);
 }
 
 /* We run with %pil set to PIL_NORMAL_MAX and PSTATE_IE enabled in %pstate.
@@ -2748,8 +2735,6 @@ void sun4v_do_mna(struct pt_regs *regs, unsigned long addr, unsigned long type_c
 void sun4v_mem_corrupt_detect_precise(struct pt_regs *regs, unsigned long addr,
 				      unsigned long context)
 {
-	siginfo_t info;
-
 	if (notify_die(DIE_TRAP, "memory corruption precise exception", regs,
 		       0, 0x8, SIGSEGV) = NOTIFY_STOP)
 		return;
@@ -2784,12 +2769,7 @@ void sun4v_mem_corrupt_detect_precise(struct pt_regs *regs, unsigned long addr,
 		regs->tpc &= 0xffffffff;
 		regs->tnpc &= 0xffffffff;
 	}
-	info.si_signo = SIGSEGV;
-	info.si_code = SEGV_ADIPERR;
-	info.si_errno = 0;
-	info.si_addr = (void __user *) addr;
-	info.si_trapno = 0;
-	force_sig_info(SIGSEGV, &info, current);
+	force_sig_fault(SIGSEGV, SEGV_ADIPERR, (void __user *)addr, 0, current);
 }
 
 void do_privop(struct pt_regs *regs)
Signature · 1 line
2.11.0
Acked-by on v12 1/11
#2 lore raw

Re: [PATCH v12 01/11] signals, sparc: Add signal codes for ADI violations

In reply to Khalid Aziz
Quoted 14 lines · Khalid Aziz

SPARC M7 processor introduces a new feature - Application Data Integrity (ADI). ADI allows MMU to catch rogue accesses to memory. When a rogue access occurs, MMU blocks the access and raises an exception. In response to the exception, kernel sends the offending task a SIGSEGV with si_code that indicates the nature of exception. This patch adds three new signal codes specific to ADI feature:

1. ADI is not enabled for the address and task attempted to access memory using ADI
2. Task attempted to access memory using wrong ADI tag and caused a deferred exception.
3. Task attempted to access memory using wrong ADI tag and caused a precise exception.

Quoted 3 lines
Quoted 37 lines

---
v10:
- Updated si_code definitions to match other definitions

 arch/x86/kernel/signal_compat.c    | 2 +-
 include/uapi/asm-generic/siginfo.h | 5 ++++-
 2 files changed, 5 insertions(+), 2 deletions(-)
arch/x86/kernel/signal_compat.c +1 -1
index ac057f9b0763..41cf5996a2a9 100644
@@ -27,7 +27,7 @@ static inline void signal_compat_build_tests(void)
 	 */
 	BUILD_BUG_ON(NSIGILL  != 11);
 	BUILD_BUG_ON(NSIGFPE  != 13);
-	BUILD_BUG_ON(NSIGSEGV != 4);
+	BUILD_BUG_ON(NSIGSEGV != 7);
 	BUILD_BUG_ON(NSIGBUS  != 5);
 	BUILD_BUG_ON(NSIGTRAP != 4);
 	BUILD_BUG_ON(NSIGCHLD != 6);
include/uapi/asm-generic/siginfo.h +4 -1
index 85dc965afd89..e94f03b3ee6f 100644
@@ -246,7 +246,10 @@ typedef struct siginfo {
 #else
 # define SEGV_PKUERR	4	/* failed protection key checks */
 #endif
-#define NSIGSEGV	4
+#define SEGV_ACCADI	5	/* ADI not enabled for mapped object */
+#define SEGV_ADIDERR	6	/* Disrupting MCD error */
+#define SEGV_ADIPERR	7	/* Precise MCD exception */
+#define NSIGSEGV	7
 
 /*
  * SIGBUS si_codes
Acked-by on v12 11/11
#41 lore raw

Re: [PATCH v12 11/11] sparc64: Update signal delivery to use new helper functions

In reply to Khalid Aziz
Quoted 7 lines · Khalid Aziz

Commit f8ec66014ffd ("signal: Add send_sig_fault and force_sig_fault") added new helper functions to streamline signal delivery. This patch updates signal delivery for new/updated handlers for ADI related exceptions to use the helper function.

Quoted 97 lines

---
v12:
- new patch

 arch/sparc/kernel/traps_64.c | 38 +++++++++-----------------------------
 1 file changed, 9 insertions(+), 29 deletions(-)
arch/sparc/kernel/traps_64.c +9 -29
index f36a40217f6c..462a21abd105 100644
@@ -362,7 +362,6 @@ void sun4v_data_access_exception(struct pt_regs *regs, unsigned long addr, unsig
 {
 	unsigned short type = (type_ctx >> 16);
 	unsigned short ctx  = (type_ctx & 0xffff);
-	siginfo_t info;
 
 	if (notify_die(DIE_TRAP, "data access exception", regs,
 		       0, 0x8, SIGTRAP) = NOTIFY_STOP)
@@ -406,24 +405,18 @@ void sun4v_data_access_exception(struct pt_regs *regs, unsigned long addr, unsig
 	 * kerbel by HV as data access exception with fault type set to
 	 * HV_FAULT_TYPE_INV_ASI.
 	 */
-	info.si_errno = 0;
-	info.si_addr = (void __user *) addr;
-	info.si_trapno = 0;
 	switch (type) {
 	case HV_FAULT_TYPE_INV_ASI:
-		info.si_signo = SIGILL;
-		info.si_code = ILL_ILLADR;
-		force_sig_info(SIGILL, &info, current);
+		force_sig_fault(SIGILL, ILL_ILLADR, (void __user *)addr, 0,
+				current);
 		break;
 	case HV_FAULT_TYPE_MCD_DIS:
-		info.si_signo = SIGSEGV;
-		info.si_code = SEGV_ACCADI;
-		force_sig_info(SIGSEGV, &info, current);
+		force_sig_fault(SIGSEGV, SEGV_ACCADI, (void __user *)addr, 0,
+				current);
 		break;
 	default:
-		info.si_signo = SIGSEGV;
-		info.si_code = SEGV_MAPERR;
-		force_sig_info(SIGSEGV, &info, current);
+		force_sig_fault(SIGSEGV, SEGV_MAPERR, (void __user *)addr, 0,
+				current);
 		break;
 	}
 }
@@ -2073,8 +2066,6 @@ static void sun4v_log_error(struct pt_regs *regs, struct sun4v_error_entry *ent,
  */
 void do_mcd_err(struct pt_regs *regs, struct sun4v_error_entry ent)
 {
-	siginfo_t info;
-
 	if (notify_die(DIE_TRAP, "MCD error", regs, 0, 0x34,
 		       SIGSEGV) = NOTIFY_STOP)
 		return;
@@ -2110,12 +2101,8 @@ void do_mcd_err(struct pt_regs *regs, struct sun4v_error_entry ent)
 	/* Send SIGSEGV to the userspace process with the right signal
 	 * code
 	 */
-	info.si_signo = SIGSEGV;
-	info.si_errno = 0;
-	info.si_code = SEGV_ADIDERR;
-	info.si_addr = (void __user *)ent.err_raddr;
-	info.si_trapno = 0;
-	force_sig_info(SIGSEGV, &info, current);
+	force_sig_fault(SIGSEGV, SEGV_ADIDERR, (void __user *)ent.err_raddr,
+			0, current);
 }
 
 /* We run with %pil set to PIL_NORMAL_MAX and PSTATE_IE enabled in %pstate.
@@ -2748,8 +2735,6 @@ void sun4v_do_mna(struct pt_regs *regs, unsigned long addr, unsigned long type_c
 void sun4v_mem_corrupt_detect_precise(struct pt_regs *regs, unsigned long addr,
 				      unsigned long context)
 {
-	siginfo_t info;
-
 	if (notify_die(DIE_TRAP, "memory corruption precise exception", regs,
 		       0, 0x8, SIGSEGV) = NOTIFY_STOP)
 		return;
@@ -2784,12 +2769,7 @@ void sun4v_mem_corrupt_detect_precise(struct pt_regs *regs, unsigned long addr,
 		regs->tpc &= 0xffffffff;
 		regs->tnpc &= 0xffffffff;
 	}
-	info.si_signo = SIGSEGV;
-	info.si_code = SEGV_ADIPERR;
-	info.si_errno = 0;
-	info.si_addr = (void __user *) addr;
-	info.si_trapno = 0;
-	force_sig_info(SIGSEGV, &info, current);
+	force_sig_fault(SIGSEGV, SEGV_ADIPERR, (void __user *)addr, 0, current);
 }
 
 void do_privop(struct pt_regs *regs)
#28 lore raw

Re: [PATCH v12 10/11] sparc64: Add support for ADI (Application Data Integrity)

In reply to Khalid Aziz
I love your patch! Yet something to improve:

Hi Khalid,

I love your patch! Yet something to improve:

[auto build test ERROR on sparc-next/master]
[also build test ERROR on v4.16-rc2]
[cannot apply to next-20180222]
[if your patch is applied to the wrong git tree, please drop us a note to help improve the system]

url:    https://github.com/0day-ci/linux/commits/Khalid-Aziz/Application-Data-Integrity-feature-introduced-by-SPARC-M7/20180223-071725
base:   https://git.kernel.org/pub/scm/linux/kernel/git/davem/sparc-next.git master
config: sparc64-allyesconfig (attached as .config)
compiler: sparc64-linux-gnu-gcc (Debian 7.2.0-11) 7.2.0

reproduce:
wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross chmod +x ~/bin/make.cross
# save the attached .config to linux build tree
make.cross ARCH=sparc64

All error/warnings (new ones prefixed by >>):

In file included from arch/sparc/include/asm/mmu_context.h:5:0,
from include/linux/mmu_context.h:5,
from drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h:29, from drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c:23:
arch/sparc/include/asm/mmu_context_64.h: In function 'arch_start_context_switch':

>> arch/sparc/include/asm/mmu_context_64.h:157:4: error: implicit declaration of function 'set_tsk_thread_flag'; did you mean 'set_ti_thread_flag'? [-Werror=implicit-function-declaration]
       set_tsk_thread_flag(prev, TIF_MCDPER);
       ^~~~~~~~~~~~~~~~~~~
       set_ti_thread_flag
>> arch/sparc/include/asm/mmu_context_64.h:159:4: error: implicit declaration of function 'clear_tsk_thread_flag'; did you mean 'clear_ti_thread_flag'? [-Werror=implicit-function-declaration]
       clear_tsk_thread_flag(prev, TIF_MCDPER);
       ^~~~~~~~~~~~~~~~~~~~~
       clear_ti_thread_flag
   arch/sparc/include/asm/mmu_context_64.h: In function 'finish_arch_post_lock_switch':
>> arch/sparc/include/asm/mmu_context_64.h:180:25: error: dereferencing pointer to incomplete type 'struct task_struct'
      if (current && current->mm && current->mm->context.adi) {
                            ^~
   In file included from arch/sparc/include/asm/processor.h:5:0,
                    from arch/sparc/include/asm/spinlock_64.h:12,
                    from arch/sparc/include/asm/spinlock.h:5,
                    from include/linux/spinlock.h:88,
                    from arch/sparc/include/asm/mmu_context_64.h:9,
                    from arch/sparc/include/asm/mmu_context.h:5,
                    from include/linux/mmu_context.h:5,
                    from drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h:29,
                    from drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c:23:
>> arch/sparc/include/asm/processor_64.h:194:28: error: implicit declaration of function 'task_thread_info'; did you mean 'test_thread_flag'? [-Werror=implicit-function-declaration]
    #define task_pt_regs(tsk) (task_thread_info(tsk)->kregs)
                               ^
>> arch/sparc/include/asm/mmu_context_64.h:183:11: note: in expansion of macro 'task_pt_regs'
       regs = task_pt_regs(current);
              ^~~~~~~~~~~~
>> arch/sparc/include/asm/processor_64.h:194:49: error: invalid type argument of '->' (have 'int')
    #define task_pt_regs(tsk) (task_thread_info(tsk)->kregs)
                                                    ^
>> arch/sparc/include/asm/mmu_context_64.h:183:11: note: in expansion of macro 'task_pt_regs'
       regs = task_pt_regs(current);
              ^~~~~~~~~~~~
   In file included from include/linux/cred.h:21:0,
                    from include/linux/seq_file.h:12,
                    from include/linux/pinctrl/consumer.h:17,
                    from include/linux/pinctrl/devinfo.h:21,
                    from include/linux/device.h:23,
                    from include/linux/cdev.h:8,
                    from include/drm/drmP.h:36,
                    from drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c:25:
   include/linux/sched.h: At top level:
>> include/linux/sched.h:1530:20: warning: conflicting types for 'set_tsk_thread_flag'
    static inline void set_tsk_thread_flag(struct task_struct *tsk, int flag)
                       ^~~~~~~~~~~~~~~~~~~
>> include/linux/sched.h:1530:20: error: static declaration of 'set_tsk_thread_flag' follows non-static declaration
   In file included from arch/sparc/include/asm/mmu_context.h:5:0,
                    from include/linux/mmu_context.h:5,
                    from drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h:29,
                    from drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c:23:
   arch/sparc/include/asm/mmu_context_64.h:157:4: note: previous implicit declaration of 'set_tsk_thread_flag' was here
       set_tsk_thread_flag(prev, TIF_MCDPER);
       ^~~~~~~~~~~~~~~~~~~
   In file included from include/linux/cred.h:21:0,
                    from include/linux/seq_file.h:12,
                    from include/linux/pinctrl/consumer.h:17,
                    from include/linux/pinctrl/devinfo.h:21,
                    from include/linux/device.h:23,
                    from include/linux/cdev.h:8,
                    from include/drm/drmP.h:36,
                    from drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c:25:
>> include/linux/sched.h:1535:20: warning: conflicting types for 'clear_tsk_thread_flag'
    static inline void clear_tsk_thread_flag(struct task_struct *tsk, int flag)
                       ^~~~~~~~~~~~~~~~~~~~~
>> include/linux/sched.h:1535:20: error: static declaration of 'clear_tsk_thread_flag' follows non-static declaration
   In file included from arch/sparc/include/asm/mmu_context.h:5:0,
                    from include/linux/mmu_context.h:5,
                    from drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h:29,
                    from drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c:23:
   arch/sparc/include/asm/mmu_context_64.h:159:4: note: previous implicit declaration of 'clear_tsk_thread_flag' was here
       clear_tsk_thread_flag(prev, TIF_MCDPER);
       ^~~~~~~~~~~~~~~~~~~~~

cc1: some warnings being treated as errors

vim +157 arch/sparc/include/asm/mmu_context_64.h

     8	
   > 9	#include <linux/spinlock.h>
    10	#include <linux/mm_types.h>
    11	#include <linux/smp.h>
    12	
    13	#include <asm/spitfire.h>
    14	#include <asm/adi_64.h>
    15	#include <asm-generic/mm_hooks.h>
    16	#include <asm/percpu.h>
    17	
    18	static inline void enter_lazy_tlb(struct mm_struct *mm, struct task_struct *tsk)
    19	{
    20	}
    21	
    22	extern spinlock_t ctx_alloc_lock;
    23	extern unsigned long tlb_context_cache;
    24	extern unsigned long mmu_context_bmap[];
    25	
    26	DECLARE_PER_CPU(struct mm_struct *, per_cpu_secondary_mm);
    27	void get_new_mmu_context(struct mm_struct *mm);
    28	int init_new_context(struct task_struct *tsk, struct mm_struct *mm);
    29	void destroy_context(struct mm_struct *mm);
    30	
    31	void __tsb_context_switch(unsigned long pgd_pa,
    32				  struct tsb_config *tsb_base,
    33				  struct tsb_config *tsb_huge,
    34				  unsigned long tsb_descr_pa,
    35				  unsigned long secondary_ctx);
    36	
    37	static inline void tsb_context_switch_ctx(struct mm_struct *mm,
    38						  unsigned long ctx)
    39	{
    40		__tsb_context_switch(__pa(mm->pgd),
    41				     &mm->context.tsb_block[MM_TSB_BASE],
    42	#if defined(CONFIG_HUGETLB_PAGE) || defined(CONFIG_TRANSPARENT_HUGEPAGE)
    43				     (mm->context.tsb_block[MM_TSB_HUGE].tsb ?
    44				      &mm->context.tsb_block[MM_TSB_HUGE] :
    45				      NULL)
    46	#else
    47				     NULL
    48	#endif
    49				     , __pa(&mm->context.tsb_descr[MM_TSB_BASE]),
    50				     ctx);
    51	}
    52	
    53	#define tsb_context_switch(X) tsb_context_switch_ctx(X, 0)
    54	
    55	void tsb_grow(struct mm_struct *mm,
    56		      unsigned long tsb_index,
    57		      unsigned long mm_rss);
    58	#ifdef CONFIG_SMP
    59	void smp_tsb_sync(struct mm_struct *mm);
    60	#else
    61	#define smp_tsb_sync(__mm) do { } while (0)
    62	#endif
    63	
    64	/* Set MMU context in the actual hardware. */
    65	#define load_secondary_context(__mm) \
    66		__asm__ __volatile__( \
    67		"\n661:	stxa		%0, [%1] %2\n" \
    68		"	.section	.sun4v_1insn_patch, \"ax\"\n" \
    69		"	.word		661b\n" \
    70		"	stxa		%0, [%1] %3\n" \
    71		"	.previous\n" \
    72		"	flush		%%g6\n" \
    73		: /* No outputs */ \
    74		: "r" (CTX_HWBITS((__mm)->context)), \
    75		  "r" (SECONDARY_CONTEXT), "i" (ASI_DMMU), "i" (ASI_MMU))
    76	
    77	void __flush_tlb_mm(unsigned long, unsigned long);
    78	
    79	/* Switch the current MM context. */
    80	static inline void switch_mm(struct mm_struct *old_mm, struct mm_struct *mm, struct task_struct *tsk)
    81	{
    82		unsigned long ctx_valid, flags;
    83		int cpu = smp_processor_id();
    84	
    85		per_cpu(per_cpu_secondary_mm, cpu) = mm;
    86		if (unlikely(mm == &init_mm))
    87			return;
    88	
    89		spin_lock_irqsave(&mm->context.lock, flags);
    90		ctx_valid = CTX_VALID(mm->context);
    91		if (!ctx_valid)
    92			get_new_mmu_context(mm);
    93	
    94		/* We have to be extremely careful here or else we will miss
    95		 * a TSB grow if we switch back and forth between a kernel
    96		 * thread and an address space which has it's TSB size increased
    97		 * on another processor.
    98		 *
    99		 * It is possible to play some games in order to optimize the
   100		 * switch, but the safest thing to do is to unconditionally
   101		 * perform the secondary context load and the TSB context switch.
   102		 *
   103		 * For reference the bad case is, for address space "A":
   104		 *
   105		 *		CPU 0			CPU 1
   106		 *	run address space A
   107		 *	set cpu0's bits in cpu_vm_mask
   108		 *	switch to kernel thread, borrow
   109		 *	address space A via entry_lazy_tlb
   110		 *					run address space A
   111		 *					set cpu1's bit in cpu_vm_mask
   112		 *					flush_tlb_pending()
   113		 *					reset cpu_vm_mask to just cpu1
   114		 *					TSB grow
   115		 *	run address space A
   116		 *	context was valid, so skip
   117		 *	TSB context switch
   118		 *
   119		 * At that point cpu0 continues to use a stale TSB, the one from
   120		 * before the TSB grow performed on cpu1.  cpu1 did not cross-call
   121		 * cpu0 to update it's TSB because at that point the cpu_vm_mask
   122		 * only had cpu1 set in it.
   123		 */
   124		tsb_context_switch_ctx(mm, CTX_HWBITS(mm->context));
   125	
   126		/* Any time a processor runs a context on an address space
   127		 * for the first time, we must flush that context out of the
   128		 * local TLB.
   129		 */
   130		if (!ctx_valid || !cpumask_test_cpu(cpu, mm_cpumask(mm))) {
   131			cpumask_set_cpu(cpu, mm_cpumask(mm));
   132			__flush_tlb_mm(CTX_HWBITS(mm->context),
   133				       SECONDARY_CONTEXT);
   134		}
   135		spin_unlock_irqrestore(&mm->context.lock, flags);
   136	}
   137	
   138	#define deactivate_mm(tsk,mm)	do { } while (0)
   139	#define activate_mm(active_mm, mm) switch_mm(active_mm, mm, NULL)
   140	
   141	#define  __HAVE_ARCH_START_CONTEXT_SWITCH
   142	static inline void arch_start_context_switch(struct task_struct *prev)
   143	{
   144		/* Save the current state of MCDPER register for the process
   145		 * we are switching from
   146		 */
   147		if (adi_capable()) {
   148			register unsigned long tmp_mcdper;
   149	
   150			__asm__ __volatile__(
   151				".word 0x83438000\n\t"	/* rd  %mcdper, %g1 */
   152				"mov %%g1, %0\n\t"
   153				: "=r" (tmp_mcdper)
   154				:
   155				: "g1");
   156			if (tmp_mcdper)
 > 157				set_tsk_thread_flag(prev, TIF_MCDPER);
   158			else
 > 159				clear_tsk_thread_flag(prev, TIF_MCDPER);
   160		}
   161	}
   162	
   163	#define finish_arch_post_lock_switch	finish_arch_post_lock_switch
   164	static inline void finish_arch_post_lock_switch(void)
   165	{
   166		/* Restore the state of MCDPER register for the new process
   167		 * just switched to.
   168		 */
   169		if (adi_capable()) {
   170			register unsigned long tmp_mcdper;
   171	
   172			tmp_mcdper = test_thread_flag(TIF_MCDPER);
   173			__asm__ __volatile__(
   174				"mov %0, %%g1\n\t"
   175				".word 0x9d800001\n\t"	/* wr %g0, %g1, %mcdper" */
   176				".word 0xaf902001\n\t"	/* wrpr %g0, 1, %pmcdper */
   177				:
   178				: "ir" (tmp_mcdper)
   179				: "g1");
 > 180			if (current && current->mm && current->mm->context.adi) {
   181				struct pt_regs *regs;
   182	
 > 183				regs = task_pt_regs(current);

184 regs->tstate |= TSTATE_MCDE;
185 }
186 }
187 }
188

---
0-DAY kernel test infrastructure Open Source Technology Center
https://lists.01.org/pipermail/kbuild-all Intel Corporation

#29 lore raw

Re: [PATCH v12 10/11] sparc64: Add support for ADI (Application Data Integrity)

Quoted 13 lines · kbuild test robot

Hi Khalid,

I love your patch! Yet something to improve:

[auto build test ERROR on sparc-next/master]
[also build test ERROR on v4.16-rc2]
[cannot apply to next-20180222]
[if your patch is applied to the wrong git tree, please drop us a note to help improve the system]

url:    https://github.com/0day-ci/linux/commits/Khalid-Aziz/Application-Data-Integrity-feature-introduced-by-SPARC-M7/20180223-071725
base:   https://git.kernel.org/pub/scm/linux/kernel/git/davem/sparc-next.git master
config: sparc64-allyesconfig (attached as .config)
compiler: sparc64-linux-gnu-gcc (Debian 7.2.0-11) 7.2.0
Quoted 7 lines

reproduce:
wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross chmod +x ~/bin/make.cross
# save the attached .config to linux build tree
make.cross ARCH=sparc64

All error/warnings (new ones prefixed by >>):

Hi Dave,

Including linux/sched.h in arch/sparc/include/asm/mmu_context.h should eliminate these build warnings. My gcc version 6.2.1 does not report these errors. Build bot is using 7.2.0.

I can add a patch 12 to add the include, revise patch 10 or you can add the include in your tree. Let me know how you would prefer to resolve this.

Thanks,
Khalid

Quoted 268 lines

In file included from arch/sparc/include/asm/mmu_context.h:5:0,
from include/linux/mmu_context.h:5,
from drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h:29, from drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c:23:
arch/sparc/include/asm/mmu_context_64.h: In function 'arch_start_context_switch':

Quoted 1 line
Quoted 1 line

arch/sparc/include/asm/mmu_context_64.h:157:4: error: implicit declaration of function 'set_tsk_thread_flag'; did you mean 'set_ti_thread_flag'? [-Werror=implicit-function-declaration]

        set_tsk_thread_flag(prev, TIF_MCDPER);
        ^~~~~~~~~~~~~~~~~~~
        set_ti_thread_flag
Quoted 1 line
Quoted 1 line

arch/sparc/include/asm/mmu_context_64.h:159:4: error: implicit declaration of function 'clear_tsk_thread_flag'; did you mean 'clear_ti_thread_flag'? [-Werror=implicit-function-declaration]

        clear_tsk_thread_flag(prev, TIF_MCDPER);
        ^~~~~~~~~~~~~~~~~~~~~
        clear_ti_thread_flag
    arch/sparc/include/asm/mmu_context_64.h: In function 'finish_arch_post_lock_switch':
Quoted 1 line
Quoted 1 line

arch/sparc/include/asm/mmu_context_64.h:180:25: error: dereferencing pointer to incomplete type 'struct task_struct'

       if (current && current->mm && current->mm->context.adi) {
                             ^~
    In file included from arch/sparc/include/asm/processor.h:5:0,
                     from arch/sparc/include/asm/spinlock_64.h:12,
                     from arch/sparc/include/asm/spinlock.h:5,
                     from include/linux/spinlock.h:88,
                     from arch/sparc/include/asm/mmu_context_64.h:9,
                     from arch/sparc/include/asm/mmu_context.h:5,
                     from include/linux/mmu_context.h:5,
                     from drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h:29,
                     from drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c:23:
Quoted 1 line
Quoted 1 line

arch/sparc/include/asm/processor_64.h:194:28: error: implicit declaration of function 'task_thread_info'; did you mean 'test_thread_flag'? [-Werror=implicit-function-declaration]

     #define task_pt_regs(tsk) (task_thread_info(tsk)->kregs)
                                ^
Quoted 1 line
Quoted 1 line

arch/sparc/include/asm/mmu_context_64.h:183:11: note: in expansion of macro 'task_pt_regs'

        regs = task_pt_regs(current);
               ^~~~~~~~~~~~
Quoted 1 line
Quoted 1 line

arch/sparc/include/asm/processor_64.h:194:49: error: invalid type argument of '->' (have 'int')

     #define task_pt_regs(tsk) (task_thread_info(tsk)->kregs)
                                                     ^
Quoted 1 line
Quoted 1 line

arch/sparc/include/asm/mmu_context_64.h:183:11: note: in expansion of macro 'task_pt_regs'

        regs = task_pt_regs(current);
               ^~~~~~~~~~~~
    In file included from include/linux/cred.h:21:0,
                     from include/linux/seq_file.h:12,
                     from include/linux/pinctrl/consumer.h:17,
                     from include/linux/pinctrl/devinfo.h:21,
                     from include/linux/device.h:23,
                     from include/linux/cdev.h:8,
                     from include/drm/drmP.h:36,
                     from drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c:25:
    include/linux/sched.h: At top level:
Quoted 1 line
Quoted 1 line

include/linux/sched.h:1530:20: warning: conflicting types for 'set_tsk_thread_flag'

static inline void set_tsk_thread_flag(struct task_struct *tsk, int flag)
^~~~~~~~~~~~~~~~~~~

Quoted 1 line
Quoted 1 line

include/linux/sched.h:1530:20: error: static declaration of 'set_tsk_thread_flag' follows non-static declaration

In file included from arch/sparc/include/asm/mmu_context.h:5:0,
from include/linux/mmu_context.h:5,
from drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h:29,
from drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c:23:
arch/sparc/include/asm/mmu_context_64.h:157:4: note: previous implicit declaration of 'set_tsk_thread_flag' was here
set_tsk_thread_flag(prev, TIF_MCDPER);
^~~~~~~~~~~~~~~~~~~
In file included from include/linux/cred.h:21:0,
from include/linux/seq_file.h:12,
from include/linux/pinctrl/consumer.h:17,
from include/linux/pinctrl/devinfo.h:21,
from include/linux/device.h:23,
from include/linux/cdev.h:8,
from include/drm/drmP.h:36,
from drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c:25:

Quoted 1 line
Quoted 1 line

include/linux/sched.h:1535:20: warning: conflicting types for 'clear_tsk_thread_flag'

static inline void clear_tsk_thread_flag(struct task_struct *tsk, int flag)
^~~~~~~~~~~~~~~~~~~~~

Quoted 1 line
Quoted 1 line

include/linux/sched.h:1535:20: error: static declaration of 'clear_tsk_thread_flag' follows non-static declaration

In file included from arch/sparc/include/asm/mmu_context.h:5:0,
from include/linux/mmu_context.h:5,
from drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h:29,
from drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c:23:
arch/sparc/include/asm/mmu_context_64.h:159:4: note: previous implicit declaration of 'clear_tsk_thread_flag' was here
clear_tsk_thread_flag(prev, TIF_MCDPER);
^~~~~~~~~~~~~~~~~~~~~
cc1: some warnings being treated as errors

vim +157 arch/sparc/include/asm/mmu_context_64.h

      8	
    > 9	#include <linux/spinlock.h>
     10	#include <linux/mm_types.h>
     11	#include <linux/smp.h>
     12	
     13	#include <asm/spitfire.h>
     14	#include <asm/adi_64.h>
     15	#include <asm-generic/mm_hooks.h>
     16	#include <asm/percpu.h>
     17	
     18	static inline void enter_lazy_tlb(struct mm_struct *mm, struct task_struct *tsk)
     19	{
     20	}
     21	
     22	extern spinlock_t ctx_alloc_lock;
     23	extern unsigned long tlb_context_cache;
     24	extern unsigned long mmu_context_bmap[];
     25	
     26	DECLARE_PER_CPU(struct mm_struct *, per_cpu_secondary_mm);
     27	void get_new_mmu_context(struct mm_struct *mm);
     28	int init_new_context(struct task_struct *tsk, struct mm_struct *mm);
     29	void destroy_context(struct mm_struct *mm);
     30	
     31	void __tsb_context_switch(unsigned long pgd_pa,
     32				  struct tsb_config *tsb_base,
     33				  struct tsb_config *tsb_huge,
     34				  unsigned long tsb_descr_pa,
     35				  unsigned long secondary_ctx);
     36	
     37	static inline void tsb_context_switch_ctx(struct mm_struct *mm,
     38						  unsigned long ctx)
     39	{
     40		__tsb_context_switch(__pa(mm->pgd),
     41				     &mm->context.tsb_block[MM_TSB_BASE],
     42	#if defined(CONFIG_HUGETLB_PAGE) || defined(CONFIG_TRANSPARENT_HUGEPAGE)
     43				     (mm->context.tsb_block[MM_TSB_HUGE].tsb ?
     44				      &mm->context.tsb_block[MM_TSB_HUGE] :
     45				      NULL)
     46	#else
     47				     NULL
     48	#endif
     49				     , __pa(&mm->context.tsb_descr[MM_TSB_BASE]),
     50				     ctx);
     51	}
     52	
     53	#define tsb_context_switch(X) tsb_context_switch_ctx(X, 0)
     54	
     55	void tsb_grow(struct mm_struct *mm,
     56		      unsigned long tsb_index,
     57		      unsigned long mm_rss);
     58	#ifdef CONFIG_SMP
     59	void smp_tsb_sync(struct mm_struct *mm);
     60	#else
     61	#define smp_tsb_sync(__mm) do { } while (0)
     62	#endif
     63	
     64	/* Set MMU context in the actual hardware. */
     65	#define load_secondary_context(__mm) \
     66		__asm__ __volatile__( \
     67		"\n661:	stxa		%0, [%1] %2\n" \
     68		"	.section	.sun4v_1insn_patch, \"ax\"\n" \
     69		"	.word		661b\n" \
     70		"	stxa		%0, [%1] %3\n" \
     71		"	.previous\n" \
     72		"	flush		%%g6\n" \
     73		: /* No outputs */ \
     74		: "r" (CTX_HWBITS((__mm)->context)), \
     75		  "r" (SECONDARY_CONTEXT), "i" (ASI_DMMU), "i" (ASI_MMU))
     76	
     77	void __flush_tlb_mm(unsigned long, unsigned long);
     78	
     79	/* Switch the current MM context. */
     80	static inline void switch_mm(struct mm_struct *old_mm, struct mm_struct *mm, struct task_struct *tsk)
     81	{
     82		unsigned long ctx_valid, flags;
     83		int cpu = smp_processor_id();
     84	
     85		per_cpu(per_cpu_secondary_mm, cpu) = mm;
     86		if (unlikely(mm == &init_mm))
     87			return;
     88	
     89		spin_lock_irqsave(&mm->context.lock, flags);
     90		ctx_valid = CTX_VALID(mm->context);
     91		if (!ctx_valid)
     92			get_new_mmu_context(mm);
     93	
     94		/* We have to be extremely careful here or else we will miss
     95		 * a TSB grow if we switch back and forth between a kernel
     96		 * thread and an address space which has it's TSB size increased
     97		 * on another processor.
     98		 *
     99		 * It is possible to play some games in order to optimize the
    100		 * switch, but the safest thing to do is to unconditionally
    101		 * perform the secondary context load and the TSB context switch.
    102		 *
    103		 * For reference the bad case is, for address space "A":
    104		 *
    105		 *		CPU 0			CPU 1
    106		 *	run address space A
    107		 *	set cpu0's bits in cpu_vm_mask
    108		 *	switch to kernel thread, borrow
    109		 *	address space A via entry_lazy_tlb
    110		 *					run address space A
    111		 *					set cpu1's bit in cpu_vm_mask
    112		 *					flush_tlb_pending()
    113		 *					reset cpu_vm_mask to just cpu1
    114		 *					TSB grow
    115		 *	run address space A
    116		 *	context was valid, so skip
    117		 *	TSB context switch
    118		 *
    119		 * At that point cpu0 continues to use a stale TSB, the one from
    120		 * before the TSB grow performed on cpu1.  cpu1 did not cross-call
    121		 * cpu0 to update it's TSB because at that point the cpu_vm_mask
    122		 * only had cpu1 set in it.
    123		 */
    124		tsb_context_switch_ctx(mm, CTX_HWBITS(mm->context));
    125	
    126		/* Any time a processor runs a context on an address space
    127		 * for the first time, we must flush that context out of the
    128		 * local TLB.
    129		 */
    130		if (!ctx_valid || !cpumask_test_cpu(cpu, mm_cpumask(mm))) {
    131			cpumask_set_cpu(cpu, mm_cpumask(mm));
    132			__flush_tlb_mm(CTX_HWBITS(mm->context),
    133				       SECONDARY_CONTEXT);
    134		}
    135		spin_unlock_irqrestore(&mm->context.lock, flags);
    136	}
    137	
    138	#define deactivate_mm(tsk,mm)	do { } while (0)
    139	#define activate_mm(active_mm, mm) switch_mm(active_mm, mm, NULL)
    140	
    141	#define  __HAVE_ARCH_START_CONTEXT_SWITCH
    142	static inline void arch_start_context_switch(struct task_struct *prev)
    143	{
    144		/* Save the current state of MCDPER register for the process
    145		 * we are switching from
    146		 */
    147		if (adi_capable()) {
    148			register unsigned long tmp_mcdper;
    149	
    150			__asm__ __volatile__(
    151				".word 0x83438000\n\t"	/* rd  %mcdper, %g1 */
    152				"mov %%g1, %0\n\t"
    153				: "=r" (tmp_mcdper)
    154				:
    155				: "g1");
    156			if (tmp_mcdper)
Quoted 1 line

157 set_tsk_thread_flag(prev, TIF_MCDPER);

    158			else
Quoted 1 line

159 clear_tsk_thread_flag(prev, TIF_MCDPER);

    160		}
    161	}
    162	
    163	#define finish_arch_post_lock_switch	finish_arch_post_lock_switch
    164	static inline void finish_arch_post_lock_switch(void)
    165	{
    166		/* Restore the state of MCDPER register for the new process
    167		 * just switched to.
    168		 */
    169		if (adi_capable()) {
    170			register unsigned long tmp_mcdper;
    171	
    172			tmp_mcdper = test_thread_flag(TIF_MCDPER);
    173			__asm__ __volatile__(
    174				"mov %0, %%g1\n\t"
    175				".word 0x9d800001\n\t"	/* wr %g0, %g1, %mcdper" */
    176				".word 0xaf902001\n\t"	/* wrpr %g0, 1, %pmcdper */
    177				:
    178				: "ir" (tmp_mcdper)
    179				: "g1");
Quoted 1 line

180 if (current && current->mm && current->mm->context.adi) {

    181				struct pt_regs *regs;
    182	
Quoted 1 line

183 regs = task_pt_regs(current);

    184				regs->tstate |= TSTATE_MCDE;
    185			}
    186		}
    187	}
    188	

---
0-DAY kernel test infrastructure Open Source Technology Center
https://lists.01.org/pipermail/kbuild-all Intel Corporation

maintainer
#30 lore raw

Re: [PATCH v12 10/11] sparc64: Add support for ADI (Application Data Integrity)

From: Khalid Aziz <[email protected]>
Date: Fri, 23 Feb 2018 11:51:25 -0700

Quoted 26 lines · kbuild test robot
Quoted 14 lines

Hi Khalid,
I love your patch! Yet something to improve:
[auto build test ERROR on sparc-next/master]
[also build test ERROR on v4.16-rc2]
[cannot apply to next-20180222]
[if your patch is applied to the wrong git tree, please drop us a note to help improve the system]
url:
https://github.com/0day-ci/linux/commits/Khalid-Aziz/Application-Data-Integrity-feature-introduced-by-SPARC-M7/20180223-071725
base:
https://git.kernel.org/pub/scm/linux/kernel/git/davem/sparc-next.git
master
config: sparc64-allyesconfig (attached as .config)
compiler: sparc64-linux-gnu-gcc (Debian 7.2.0-11) 7.2.0

Quoted 8 lines

reproduce:
wget
https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross
-O ~/bin/make.cross
chmod +x ~/bin/make.cross
# save the attached .config to linux build tree make.cross ARCH=sparc64
All error/warnings (new ones prefixed by >>):

Hi Dave,

Quoted 7 lines

Including linux/sched.h in arch/sparc/include/asm/mmu_context.h should eliminate these build warnings. My gcc version 6.2.1 does not report these errors. Build bot is using 7.2.0.

I can add a patch 12 to add the include, revise patch 10 or you can add the include in your tree. Let me know how you would prefer to resolve this.

You need to update patch #10 so that your patch series is fully bisectable.

Thank you.

#31 lore raw

Re: [PATCH v12 10/11] sparc64: Add support for ADI (Application Data Integrity)

Quoted 33 lines · David Miller

From: Khalid Aziz <[email protected]>
Date: Fri, 23 Feb 2018 11:51:25 -0700

Quoted 30 lines · kbuild test robot
Quoted 14 lines

Hi Khalid,
I love your patch! Yet something to improve:
[auto build test ERROR on sparc-next/master]
[also build test ERROR on v4.16-rc2]
[cannot apply to next-20180222]
[if your patch is applied to the wrong git tree, please drop us a note to help improve the system]
url:
https://github.com/0day-ci/linux/commits/Khalid-Aziz/Application-Data-Integrity-feature-introduced-by-SPARC-M7/20180223-071725
base:
https://git.kernel.org/pub/scm/linux/kernel/git/davem/sparc-next.git
master
config: sparc64-allyesconfig (attached as .config)
compiler: sparc64-linux-gnu-gcc (Debian 7.2.0-11) 7.2.0

Quoted 8 lines

reproduce:
wget
https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross
-O ~/bin/make.cross
chmod +x ~/bin/make.cross
# save the attached .config to linux build tree make.cross ARCH=sparc64
All error/warnings (new ones prefixed by >>):

Hi Dave,

Including linux/sched.h in arch/sparc/include/asm/mmu_context.h should eliminate these build warnings. My gcc version 6.2.1 does not report these errors. Build bot is using 7.2.0.

Quoted 6 lines
Quoted 3 lines

I can add a patch 12 to add the include, revise patch 10 or you can add the include in your tree. Let me know how you would prefer to resolve this.

You need to update patch #10 so that your patch series is fully bisectable.

Hi Dave,

That sounds like the right thing to do. I am updating patch 10 and will send out v13 for patch 10/11. Rest of the series is unchanged but I can send the whole series if you prefer that.

Thanks,
Khalid

reviewer Acked-by on v12 7/11
#16 lore raw

Re: [PATCH v12 07/11] mm: Add address parameter to arch_validate_prot()

In reply to Khalid Aziz
Quoted 19 lines · Khalid Aziz

A protection flag may not be valid across entire address space and hence arch_validate_prot() might need the address a protection bit is being set on to ensure it is a valid protection flag. For example, sparc processors support memory corruption detection (as part of ADI feature) flag on memory addresses mapped on to physical RAM but not on PFN mapped pages or addresses mapped on to devices. This patch adds address to the parameters being passed to arch_validate_prot() so protection bits can be validated in the relevant context.

---
v8:
- Added addr parameter to powerpc arch_validate_prot() (suggested by Michael Ellerman)
v9:
- new patch

Quoted 2 lines
 arch/powerpc/include/asm/mman.h | 4 ++--
 arch/powerpc/kernel/syscalls.c  | 2 +-

These changes look fine to me:

cheers

maintainer
#4 lore raw

Re: [PATCH v12 02/11] mm, swap: Add infrastructure for saving page metadata on swap

In reply to Khalid Aziz
Quoted 8 lines · Khalid Aziz

If a processor supports special metadata for a page, for example ADI version tags on SPARC M7, this metadata must be saved when the page is swapped out. The same metadata must be restored when the page is swapped back in. This patch adds two new architecture specific functions - arch_do_swap_page() to be called when a page is swapped in, and arch_unmap_one() to be called when a page is being unmapped for swap out. These architecture hooks allow page metadata to be saved if the architecture supports it.

I still think silently squishing cacheline-level hardware data into page-level software data structures is dangerous.

But, you seem rather determined to do it this way. I don't think this will _hurt_ anyone else, though other than needlessly cluttering up the code.

maintainer
#32 lore raw

Re: [PATCH v12 10/11] sparc64: Add support for ADI (Application Data Integrity)

In reply to Khalid Aziz
Quoted 9 lines · Khalid Aziz
arch/sparc/include/asm/mman.h +9 -0
+#define arch_validate_prot(prot, addr) sparc_validate_prot(prot, addr)
+static inline int sparc_validate_prot(unsigned long prot, unsigned long addr)
+{
+	if (prot & ~(PROT_READ | PROT_WRITE | PROT_EXEC | PROT_SEM | PROT_ADI))
+		return 0;
+	if (prot & PROT_ADI) {
+		if (!adi_capable())
+			return 0;
+

arch/sparc/include/asm/mman.h line 73

Quoted 10 lines
arch/sparc/include/asm/mman.h +10 -0
+		if (addr) {
+			struct vm_area_struct *vma;
+
+			vma = find_vma(current->mm, addr);
+			if (vma) {
+				/* ADI can not be enabled on PFN
+				 * mapped pages
+				 */
+				if (vma->vm_flags & (VM_PFNMAP | VM_MIXEDMAP))
+					return 0;

You don't hold mmap_sem here. How can this work?

maintainer
#19 lore raw

Re: [PATCH v12 08/11] mm: Clear arch specific VM flags on protection change

In reply to Khalid Aziz

include/linux/mm.h extern unsigned int kobjsize(const void *objp); · line 294

Quoted 5 lines · Khalid Aziz
include/linux/mm.h +5 -0
+/* Arch-specific flags to clear when updating VM flags on protection change */
+#ifndef VM_ARCH_CLEAR
+# define VM_ARCH_CLEAR	VM_NONE
+#endif
+#define VM_FLAGS_CLEAR	(ARCH_VM_PKEY_FLAGS | VM_ARCH_CLEAR)

Shouldn't this be defining

# define VM_ARCH_CLEAR	ARCH_VM_PKEY_FLAGS

on x86?

maintainer
#23 lore raw

Re: [PATCH v12 09/11] mm: Allow arch code to override copy_highpage()

In reply to Khalid Aziz

include/linux/highmem.h static inline void copy_highpage(struct page *to, struct page *from) · line 253

Quoted 10 lines · Khalid Aziz
include/linux/highmem.h +3 -0
+#ifndef __HAVE_ARCH_COPY_HIGHPAGE
+
 static inline void copy_highpage(struct page *to, struct page *from)
 {
 	char *vfrom, *vto;
@@ -248,4 +250,6 @@ static inline void copy_highpage(struct page *to, struct page *from)
 	kunmap_atomic(vfrom);
 }
 
+#endif

I think we prefer that these are CONFIG_* options.

#5 lore raw

Re: [PATCH v12 02/11] mm, swap: Add infrastructure for saving page metadata on swap

In reply to Dave Hansen
Quoted 10 lines · Dave Hansen
Quoted 8 lines · Khalid Aziz

If a processor supports special metadata for a page, for example ADI version tags on SPARC M7, this metadata must be saved when the page is swapped out. The same metadata must be restored when the page is swapped back in. This patch adds two new architecture specific functions - arch_do_swap_page() to be called when a page is swapped in, and arch_unmap_one() to be called when a page is being unmapped for swap out. These architecture hooks allow page metadata to be saved if the architecture supports it.

Quoted 6 lines

I still think silently squishing cacheline-level hardware data into page-level software data structures is dangerous.

But, you seem rather determined to do it this way. I don't think this will _hurt_ anyone else, though other than needlessly cluttering up the code.

Hello Dave,

Thanks for taking the time to look at this patch and providing feedback.

ADI data is per page data and is held in the spare bits in the RAM. It is loaded into the cache when data is loaded from RAM and flushed out to spare bits in the RAM when data is flushed from cache. Sparc allows one tag for each ADI block size of data and ADI block size is same as cacheline size. When a page is loaded into RAM from swap space, all of the associated ADI data for the page must also be loaded into the RAM, so it looks like page level data and storing it in page level software data structure makes sense. I am open to other suggestions though.

Thanks,
Khalid

maintainer
#6 lore raw

Re: [PATCH v12 02/11] mm, swap: Add infrastructure for saving page metadata on swap

Quoted 5 lines · Khalid Aziz

ADI data is per page data and is held in the spare bits in the RAM. It is loaded into the cache when data is loaded from RAM and flushed out to spare bits in the RAM when data is flushed from cache. Sparc allows one tag for each ADI block size of data and ADI block size is same as cacheline size.

Which does not square with your earlier assertion "ADI data is per page data". It's per-cacheline data. Right?

Quoted 4 lines

When a page is loaded into RAM from swap space, all of
the associated ADI data for the page must also be loaded into the RAM, so it looks like page level data and storing it in page level software data structure makes sense. I am open to other suggestions though.

Do you have a way to tell that data is not being thrown away? Like if the ADI metadata is different for two different cachelines within a single page?

#7 lore raw

Re: [PATCH v12 02/11] mm, swap: Add infrastructure for saving page metadata on swap

Quoted 9 lines · Dave Hansen
Quoted 5 lines · Khalid Aziz

ADI data is per page data and is held in the spare bits in the RAM. It is loaded into the cache when data is loaded from RAM and flushed out to spare bits in the RAM when data is flushed from cache. Sparc allows one tag for each ADI block size of data and ADI block size is same as cacheline size.

Which does not square with your earlier assertion "ADI data is per page data". It's per-cacheline data. Right?

That is one way to look at it. Current sparc processors do implement same ADI block size as cacheline size but architecture does not require ADI block size to be same as cacheline size. If those two sizes were different, we wouldn't call it cacheline data.

Quoted 9 lines
Quoted 5 lines

When a page is loaded into RAM from swap space, all of
the associated ADI data for the page must also be loaded into the RAM, so it looks like page level data and storing it in page level software data structure makes sense. I am open to other suggestions though.

Do you have a way to tell that data is not being thrown away? Like if the ADI metadata is different for two different cachelines within a single page?

Yes, since access to tagged data is made using pointers with ADI tag embedded in the top bits, any mismatch between what app thinks the ADI tags should be and what is stored in the RAM for corresponding page will result in exception. If ADI data gets thrown away, we will get an ADI tag mismatch exception. If ADI tags for two different ADI blocks on a page are different when app expected them to be the same, we will see an exception on access to the block with wrong ADI data.

--
Khalid

#20 lore raw

Re: [PATCH v12 08/11] mm: Clear arch specific VM flags on protection change

In reply to Dave Hansen
Quoted 9 lines · Dave Hansen

include/linux/mm.h extern unsigned int kobjsize(const void *objp); · line 294

Quoted 5 lines · Khalid Aziz
include/linux/mm.h +5 -0
+/* Arch-specific flags to clear when updating VM flags on protection change */
+#ifndef VM_ARCH_CLEAR
+# define VM_ARCH_CLEAR	VM_NONE
+#endif
+#define VM_FLAGS_CLEAR	(ARCH_VM_PKEY_FLAGS | VM_ARCH_CLEAR)

Shouldn't this be defining

Quoted 3 lines
# define VM_ARCH_CLEAR	ARCH_VM_PKEY_FLAGS

on x86?

ARCH_VM_PKEY_FLAGS is used by x86 as well as powerpc. On those two architectures VM_FLAGS_CLEAR will end up being ARCH_VM_PKEY_FLAGS and thus current behavior will be retained. Defining VM_ARCH_CLEAR to be ARCH_VM_PKEY_FLAGS on x86 will just result in VM_FLAGS_CLEAR to be (ARCH_VM_PKEY_FLAGS | ARCH_VM_PKEY_FLAGS) which is superfluous.

--
Khalid

#24 lore raw

Re: [PATCH v12 09/11] mm: Allow arch code to override copy_highpage()

In reply to Dave Hansen
Quoted 10 lines · Dave Hansen
Quoted 9 lines · Khalid Aziz
include/linux/highmem.h +2 -0
+#ifndef __HAVE_ARCH_COPY_HIGHPAGE
+
  static inline void copy_highpage(struct page *to, struct page *from)
  {
  	char *vfrom, *vto;
@@ -248,4 +250,6 @@ static inline void copy_highpage(struct page *to, struct page *from)
  	kunmap_atomic(vfrom);
  }
 
Quoted 3 lines

include/linux/highmem.h static inline void copy_highpage(struct page *to, struct page *from) · line 253

Quoted 1 line
include/linux/highmem.h +1 -0
+#endif

I think we prefer that these are CONFIG_* options.

I added this mechanism to be same as what we have for copy_user_highpage():

---------------
#ifndef __HAVE_ARCH_COPY_USER_HIGHPAGE

static inline void copy_user_highpage(struct page *to, struct page *from,
unsigned long vaddr, struct vm_area_struct *vma)
{
----------------

There isn't a CONFIG_* option for copy_user_highpage() so I don't see a reason to add one for copy_highpage().

Do you see it differently? In that case, should there be a CONFIG_* option for copy_user_highpage() as well?

Thanks,
Khalid

maintainer
#25 lore raw

Re: [PATCH v12 09/11] mm: Allow arch code to override copy_highpage()

Quoted 15 lines · Khalid Aziz
Quoted 13 lines · Dave Hansen
Quoted 10 lines · Khalid Aziz
include/linux/highmem.h +2 -0
+#ifndef __HAVE_ARCH_COPY_HIGHPAGE
+

  static inline void copy_highpage(struct page *to, struct page *from) {
      char *vfrom, *vto;

include/linux/highmem.h +0 -0
@@ -248,4 +250,6 @@ static inline void copy_highpage(struct page *to,
 struct page *from)
       kunmap_atomic(vfrom);
   }

  +#endif

I think we prefer that these are CONFIG_* options.

Quoted 4 lines

I added this mechanism to be same as what we have for copy_user_highpage():

---------------
#ifndef __HAVE_ARCH_COPY_USER_HIGHPAGE

I think that's the old way that we generally don't want to add new instances of.

maintainer
#8 lore raw

Re: [PATCH v12 02/11] mm, swap: Add infrastructure for saving page metadata on swap

In reply to Khalid Aziz
Quoted 11 lines · Khalid Aziz
Quoted 3 lines

Do you have a way to tell that data is not being thrown away?  Like if the ADI metadata is different for two different cachelines within a single page?

Yes, since access to tagged data is made using pointers with ADI tag embedded in the top bits, any mismatch between what app thinks the ADI tags should be and what is stored in the RAM for corresponding page will result in exception. If ADI data gets thrown away, we will get an ADI tag mismatch exception. If ADI tags for two different ADI blocks on a page are different when app expected them to be the same, we will see an exception on access to the block with wrong ADI data.

So, when an app has two different ADI tags on two parts of a page, the page gets swapped, and the ADI block size is under PAGE_SIZE, the app will get an ADI exception after swap-in through no fault of its own?

#9 lore raw

Re: [PATCH v12 02/11] mm, swap: Add infrastructure for saving page metadata on swap

Quoted 13 lines · Dave Hansen
Quoted 11 lines · Khalid Aziz
Quoted 3 lines

Do you have a way to tell that data is not being thrown away?  Like if the ADI metadata is different for two different cachelines within a single page?

Yes, since access to tagged data is made using pointers with ADI tag embedded in the top bits, any mismatch between what app thinks the ADI tags should be and what is stored in the RAM for corresponding page will result in exception. If ADI data gets thrown away, we will get an ADI tag mismatch exception. If ADI tags for two different ADI blocks on a page are different when app expected them to be the same, we will see an exception on access to the block with wrong ADI data.

Quoted 4 lines

So, when an app has two different ADI tags on two parts of a page, the page gets swapped, and the ADI block size is under PAGE_SIZE, the app will get an ADI exception after swap-in through no fault of its own?

Only if the kernel fails to re-establish ADI tags on the swapped in page which is why I added infrastructure to save the ADI tags for a page before it is swapped out and then re-establish those tags when the page is swapped back in. Kernel needs to save as many as ADI TAGS as may exist on each page, not just one tag per page. On sparc M7 8K pages, there are 128 ADI tags for the page, so kernel will store and restore 128 ADI tags for each page on swap-out and swap-in. If kernel restores only one ADI tag for the page on swap in, app will get an exception and it will be kernel's fault.

--
Khalid

#33 lore raw

Re: [PATCH v12 10/11] sparc64: Add support for ADI (Application Data Integrity)

In reply to Dave Hansen
Quoted 14 lines · Dave Hansen
Quoted 13 lines · Khalid Aziz
arch/sparc/include/asm/mman.h +13 -0
+#define arch_validate_prot(prot, addr) sparc_validate_prot(prot, addr)
+static inline int sparc_validate_prot(unsigned long prot, unsigned long addr)
+{
+	if (prot & ~(PROT_READ | PROT_WRITE | PROT_EXEC | PROT_SEM | PROT_ADI))
+		return 0;
+	if (prot & PROT_ADI) {
+		if (!adi_capable())
+			return 0;
+
+		if (addr) {
+			struct vm_area_struct *vma;
+
+			vma = find_vma(current->mm, addr);
Quoted 9 lines

arch/sparc/include/asm/mman.h line 73

Quoted 6 lines
arch/sparc/include/asm/mman.h +6 -0
+			if (vma) {
+				/* ADI can not be enabled on PFN
+				 * mapped pages
+				 */
+				if (vma->vm_flags & (VM_PFNMAP | VM_MIXEDMAP))
+					return 0;

You don't hold mmap_sem here. How can this work?

Are you suggesting that vma returned by find_vma() could be split or merged underneath me if I do not hold mmap_sem and thus make the flag check invalid? If so, that is a good point.

Thanks,
Khalid

maintainer
#37 lore raw

Re: [PATCH v12 10/11] sparc64: Add support for ADI (Application Data Integrity)

In reply to Khalid Aziz
Quoted 3 lines · Khalid Aziz
arch/sparc/kernel/adi_64.c +3 -0
+tag_storage_desc_t *alloc_tag_store(struct mm_struct *mm,
+				    struct vm_area_struct *vma,
+				    unsigned long addr)

...

Quoted 4 lines
arch/sparc/kernel/adi_64.c +4 -0
+	tags = kzalloc(size, GFP_NOWAIT|__GFP_NOWARN);
+	if (tags == NULL) {
+		tag_desc->tag_users = 0;
+		tag_desc = NULL;

arch/sparc/kernel/adi_64.c void __init mdesc_adi_init(void) · line 298

Quoted 10 lines
arch/sparc/kernel/adi_64.c +10 -0
+		goto out;
+	}
+	tag_desc->start = addr;
+	tag_desc->tags = tags;
+	tag_desc->end = end_addr;
+
+out:
+	spin_unlock_irqrestore(&mm->context.tag_lock, flags);
+	return tag_desc;
+}

OK, sorry, I missed this. I do see that you now have per-ADI-block tag storage and it is not per-page.

How big can this storage get, btw? Superficially it seems like it might be able to be gigantic for a large, sparse VMA.

maintainer
#34 lore raw

Re: [PATCH v12 10/11] sparc64: Add support for ADI (Application Data Integrity)

In reply to Khalid Aziz
Quoted 26 lines · Khalid Aziz
Quoted 16 lines · Dave Hansen
Quoted 15 lines · Khalid Aziz
(quoted diff) +2 -0
+#define arch_validate_prot(prot, addr) sparc_validate_prot(prot, addr)
+static inline int sparc_validate_prot(unsigned long prot, unsigned

long addr)
+{
+    if (prot & ~(PROT_READ | PROT_WRITE | PROT_EXEC | PROT_SEM | PROT_ADI))
+        return 0;
+    if (prot & PROT_ADI) {
+        if (!adi_capable())
+            return 0;

arch/sparc/include/asm/mman.h +5 -0
+
+        if (addr) {
+            struct vm_area_struct *vma;
+
+            vma = find_vma(current->mm, addr);
Quoted 9 lines
Quoted 6 lines

+            if (vma) {
+                /* ADI can not be enabled on PFN
+                 * mapped pages
+                 */
+                if (vma->vm_flags & (VM_PFNMAP | VM_MIXEDMAP)) +                    return 0;

You don't hold mmap_sem here.  How can this work?

Quoted 3 lines

Are you suggesting that vma returned by find_vma() could be split or merged underneath me if I do not hold mmap_sem and thus make the flag check invalid? If so, that is a good point.

Um, yes. You can't walk the vma tree without holding mmap_sem.

maintainer
#35 lore raw

Re: [PATCH v12 10/11] sparc64: Add support for ADI (Application Data Integrity)

In reply to Khalid Aziz
Quoted 3 lines · Khalid Aziz

Are you suggesting that vma returned by find_vma() could be split or merged underneath me if I do not hold mmap_sem and thus make the flag check invalid? If so, that is a good point.

This part does make me think that this code hasn't been tested very thoroughly. Could you describe the testing that you have done? For MPX and protection keys, I added something to tools/testing/selftests/x86, for instance.

#38 lore raw

Re: [PATCH v12 10/11] sparc64: Add support for ADI (Application Data Integrity)

In reply to Dave Hansen
Quoted 23 lines · Dave Hansen
Quoted 3 lines · Khalid Aziz
arch/sparc/kernel/adi_64.c +3 -0
+tag_storage_desc_t *alloc_tag_store(struct mm_struct *mm,
+				    struct vm_area_struct *vma,
+				    unsigned long addr)

...

Quoted 4 lines
arch/sparc/kernel/adi_64.c +4 -0
+	tags = kzalloc(size, GFP_NOWAIT|__GFP_NOWARN);
+	if (tags == NULL) {
+		tag_desc->tag_users = 0;
+		tag_desc = NULL;

arch/sparc/kernel/adi_64.c void __init mdesc_adi_init(void) · line 298

Quoted 10 lines
arch/sparc/kernel/adi_64.c +10 -0
+		goto out;
+	}
+	tag_desc->start = addr;
+	tag_desc->tags = tags;
+	tag_desc->end = end_addr;
+
+out:
+	spin_unlock_irqrestore(&mm->context.tag_lock, flags);
+	return tag_desc;
+}

OK, sorry, I missed this. I do see that you now have per-ADI-block tag storage and it is not per-page.

Quoted 3 lines

How big can this storage get, btw? Superficially it seems like it might be able to be gigantic for a large, sparse VMA.

Tags are stored only for the pages being swapped out, not for the pages in entire vma. Each tag storage page can hold tags for 128 pages (each page has 128 4-bit tags, hence 64 bytes are needed to store tags for an entire page allowing each page to store tags for 128 pages). Sparse VMA does not cause any problems since holes do not have corresponding pages that will be swapped out. Tag storage pages are freed once all the pages they store tags for have been swapped back in, except for a small number of pages (maximum of 8) marked for emergency tag storage.

--
Khalid

maintainer
#39 lore raw

Re: [PATCH v12 10/11] sparc64: Add support for ADI (Application Data Integrity)

Quoted 11 lines · Khalid Aziz
Quoted 3 lines

How big can this storage get, btw?  Superficially it seems like it might be able to be gigantic for a large, sparse VMA.

Tags are stored only for the pages being swapped out, not for the pages in entire vma. Each tag storage page can hold tags for 128 pages (each page has 128 4-bit tags, hence 64 bytes are needed to store tags for an entire page allowing each page to store tags for 128 pages). Sparse VMA does not cause any problems since holes do not have corresponding pages that will be swapped out. Tag storage pages are freed once all the pages they store tags for have been swapped back in, except for a small number of pages (maximum of 8) marked for emergency tag storage.

With a linear scan holding a process-wide spinlock? If you have a fast swap device, does this become the bottleneck when swapping ADI-tagged memory?

FWIW, this tag storage is complex and subtle enough code that it deserves to be in its own well-documented patch, not buried in a thousand-line patch.

Quoted 21 lines
arch/sparc/kernel/adi_64.c +21 -0
+tag_storage_desc_t *find_tag_store(struct mm_struct *mm,
+				   struct vm_area_struct *vma,
+				   unsigned long addr)
+{
+	tag_storage_desc_t *tag_desc = NULL;
+	unsigned long i, max_desc, flags;
+
+	/* Check if this vma already has tag storage descriptor
+	 * allocated for it.
+	 */
+	max_desc = PAGE_SIZE/sizeof(tag_storage_desc_t);
+	if (mm->context.tag_store) {
+		tag_desc = mm->context.tag_store;
+		spin_lock_irqsave(&mm->context.tag_lock, flags);
+		for (i = 0; i < max_desc; i++) {
+			if ((addr >= tag_desc->start) &&
+			    ((addr + PAGE_SIZE - 1) <= tag_desc->end))
+				break;
+			tag_desc++;
+		}
+		spin_unlock_irqrestore(&mm->context.tag_lock, flags);
#36 lore raw

Re: [PATCH v12 10/11] sparc64: Add support for ADI (Application Data Integrity)

In reply to Dave Hansen
Quoted 9 lines · Dave Hansen
Quoted 3 lines · Khalid Aziz

Are you suggesting that vma returned by find_vma() could be split or merged underneath me if I do not hold mmap_sem and thus make the flag check invalid? If so, that is a good point.

This part does make me think that this code hasn't been tested very thoroughly. Could you describe the testing that you have done? For MPX and protection keys, I added something to tools/testing/selftests/x86, for instance.

This code was tested by a QA team and I ran a number of tests myself. I wrote tests to exercise all of the API, induce exceptions for invalid/illegal accesses and swapping was tested by allocating memory 2-4 times of the system RAM available across 4-8 threads and reading/writing to this memory with ADI enabled. QA team wrote unit tests to test each API with valid and invalid combinations of arguments to the API. Stress tests that allocate and free ADI tagged memory were also run. A version of database server was created that uses ADI tagged memory for in-memory copy of database to test database workload. 100's of hours of tests were run across these tests over the last 1+ year these patches have been under review for. Cover letter includes description of most of these tests. This code has held up through all of these tests. It is entirely feasible some race conditions have not been uncovered yet, just like any other piece of software. Pulling this code into mainline kernel and having lot more people exercise this code will help shake out any remaining issues.

Thanks,
Khalid

maintainer Acked-by on v12 2/11
#10 lore raw

Re: [PATCH v12 02/11] mm, swap: Add infrastructure for saving page metadata on swap

In reply to Khalid Aziz
Quoted 9 lines · On Wed, 21 Feb 2018 10:15:44 -0700 Khalid Aziz <[email protected]> wrote:

If a processor supports special metadata for a page, for example ADI version tags on SPARC M7, this metadata must be saved when the page is swapped out. The same metadata must be restored when the page is swapped back in. This patch adds two new architecture specific functions - arch_do_swap_page() to be called when a page is swapped in, and arch_unmap_one() to be called when a page is being unmapped for swap out. These architecture hooks allow page metadata to be saved if the architecture supports it.

Quoted 4 lines
maintainer Acked-by on v12 7/11
#17 lore raw

Re: [PATCH v12 07/11] mm: Add address parameter to arch_validate_prot()

In reply to Khalid Aziz
Quoted 9 lines · On Wed, 21 Feb 2018 10:15:49 -0700 Khalid Aziz <[email protected]> wrote:

A protection flag may not be valid across entire address space and hence arch_validate_prot() might need the address a protection bit is being set on to ensure it is a valid protection flag. For example, sparc processors support memory corruption detection (as part of ADI feature) flag on memory addresses mapped on to physical RAM but not on PFN mapped pages or addresses mapped on to devices. This patch adds address to the parameters being passed to arch_validate_prot() so protection bits can be validated in the relevant context.

Quoted 3 lines
maintainer Acked-by on v12 8/11
#21 lore raw

Re: [PATCH v12 08/11] mm: Clear arch specific VM flags on protection change

In reply to Khalid Aziz
Quoted 9 lines · On Wed, 21 Feb 2018 10:15:50 -0700 Khalid Aziz <[email protected]> wrote:

When protection bits are changed on a VMA, some of the architecture specific flags should be cleared as well. An examples of this are the PKEY flags on x86. This patch expands the current code that clears PKEY flags for x86, to support similar functionality for other architectures as well.

maintainer Acked-by on v12 9/11
#26 lore raw

Re: [PATCH v12 09/11] mm: Allow arch code to override copy_highpage()

In reply to Khalid Aziz
Quoted 23 lines · On Wed, 21 Feb 2018 10:15:51 -0700 Khalid Aziz <[email protected]> wrote:

Some architectures can support metadata for memory pages and when a page is copied, its metadata must also be copied. Sparc processors from M7 onwards support metadata for memory pages. This metadata provides tag based protection for access to memory pages. To maintain this protection, the tag data must be copied to the new page when a page is migrated across NUMA nodes. This patch allows arch specific code to override default copy_highpage() and copy metadata along with page data upon migration.

...

include/linux/highmem.h +2 -0
@@ -237,6 +237,8 @@ static inline void copy_user_highpage(struct page *to, struct page *from,
 
 #endif
 
+#ifndef __HAVE_ARCH_COPY_HIGHPAGE
+
 static inline void copy_highpage(struct page *to, struct page *from)
 {
 	char *vfrom, *vto;

include/linux/highmem.h static inline void copy_highpage(struct page *to, struct page *from) · line 255

Quoted 6 lines
include/linux/highmem.h +2 -0
@@ -248,4 +250,6 @@ static inline void copy_highpage(struct page *to, struct page *from)
 	kunmap_atomic(vfrom);
 }
 
+#endif
+
 #endif /* _LINUX_HIGHMEM_H */

It would be more consistent and conventional here to do

#ifndef copy_highpage
static inline void copy_highpage(struct page *to, struct page *from)

{
	...
}

#define copy_highpage copy_highpage

As is happening in [patch 07/11].

And a similar change could be made to [patch 02/11], actually.

Either way,

maintainer
#42 lore raw
In reply to Khalid Aziz

From: Khalid Aziz <[email protected]>
Date: Wed, 21 Feb 2018 10:15:42 -0700

Quoted 7 lines

V12 changes:
This series is same as v10 and v11 and was simply rebased on 4.16-rc2 kernel and patch 11 was added to update signal delivery code to use the new helper functions added by Eric Biederman. Can mm maintainers please review patches 2, 7, 8 and 9 which are arch independent, and include/linux/mm.h and mm/ksm.c changes in patch 10 and ack these if everything looks good?

Khalid I've applied this series to sparc-next, thank you!

But one thing has to be fixed up.

In uapi/asm/auxvec.h you conditionalize the ADI aux vectors on CONFIG_SPARC64.

That's not correct, you should never control user facing definitions based upon kernel configuration.

Also, both 32-bit and 64-bit applications running on ADI capable machines want to compile against and use this informaiton.

So please remove these CPP checks.

#43 lore raw
Quoted 11 lines · David Miller

In uapi/asm/auxvec.h you conditionalize the ADI aux vectors on CONFIG_SPARC64.

That's not correct, you should never control user facing definitions based upon kernel configuration.

Also, both 32-bit and 64-bit applications running on ADI capable machines want to compile against and use this informaiton.

So please remove these CPP checks.

Hi Dave,

That makes sense. I will send a patch to remove these.

Thanks,
Khalid