kmap_atomic_push
Subsystems (76)
Peter Zijlstra
maintainerRFC PATCH 1/1 kmap_atomic_push
The below patchlet changes the kmap_atomic interface to a stack based one that doesn't require the KM_types anymore.
This significantly simplifies some code (more still than are present in this patch -- ie. pte_map_nested can go now)
This obviously requires that push and pop are matched, I fixed a few cases that were not properly nested, the (x86) code checks for this and will go BUG when trying to pop a vaddr that isn't the top one so abusers should be rather visible.
build-tested on i386-defconfig
What do people think?
David, frv has a 'funny' issue in that it treats __KM_CACHE special from the other ones, something which isn't possibly anymore. Do you see another option than to add kmap_atomic_push_cache() for frv?
--- Documentation/block/biodoc.txt | 4 - Documentation/frv/mmu-layout.txt | 36 +++++------ Documentation/io-mapping.txt | 4 - arch/arm/include/asm/highmem.h | 6 - arch/arm/include/asm/pgtable.h | 12 +-- arch/arm/mm/copypage-fa.c | 12 +-- arch/arm/mm/copypage-feroceon.c | 12 +-- arch/arm/mm/copypage-v3.c | 12 +-- arch/arm/mm/copypage-v4mc.c | 8 +- arch/arm/mm/copypage-v4wb.c | 12 +-- arch/arm/mm/copypage-v4wt.c | 12 +-- arch/arm/mm/copypage-v6.c | 12 +-- arch/arm/mm/copypage-xsc3.c | 12 +-- arch/arm/mm/copypage-xscale.c | 8 +- arch/arm/mm/flush.c | 4 - arch/arm/mm/highmem.c | 20 +++--- arch/frv/include/asm/highmem.h | 63 ++++++++++---------- arch/frv/include/asm/pgtable.h | 8 +- arch/frv/kernel/head-mmu-fr451.S | 2 arch/frv/mb93090-mb00/pci-dma.c | 4 - arch/frv/mm/cache-page.c | 10 +-- arch/ia64/kernel/crash_dump.c | 2 arch/microblaze/include/asm/pgtable.h | 8 +- arch/mips/include/asm/highmem.h | 10 +-- arch/mips/mm/c-r4k.c | 6 - arch/mips/mm/highmem.c | 22 +++---- arch/mips/mm/init.c | 8 +- arch/mn10300/include/asm/highmem.h | 13 +--- arch/parisc/include/asm/cacheflush.h | 6 - arch/powerpc/include/asm/highmem.h | 9 +- arch/powerpc/include/asm/pgtable-ppc32.h | 8 +- arch/powerpc/include/asm/pgtable.h | 2 arch/powerpc/kernel/crash_dump.c | 2 arch/powerpc/mm/dma-noncoherent.c | 8 +- arch/powerpc/mm/highmem.c | 17 ++--- arch/powerpc/mm/mem.c | 4 - arch/powerpc/sysdev/ppc4xx_pci.c | 2 arch/sh/kernel/crash_dump.c | 2 arch/sh/mm/cache.c | 12 +-- arch/sparc/include/asm/highmem.h | 4 - arch/sparc/mm/highmem.c | 21 +++--- arch/sparc/mm/io-unit.c | 2 arch/sparc/mm/iommu.c | 2 arch/um/kernel/skas/uaccess.c | 4 - arch/x86/include/asm/highmem.h | 12 +-- arch/x86/include/asm/iomap.h | 15 +--- arch/x86/include/asm/paravirt.h | 4 - arch/x86/include/asm/paravirt_types.h | 2 arch/x86/include/asm/pgtable_32.h | 12 +-- arch/x86/kernel/cpu/perf_event.c | 5 - arch/x86/kernel/crash_dump_32.c | 10 +-- arch/x86/kernel/crash_dump_64.c | 2 arch/x86/kernel/paravirt.c | 2 arch/x86/kernel/vmi_32.c | 6 - arch/x86/kvm/lapic.c | 8 +- arch/x86/kvm/paging_tmpl.h | 4 - arch/x86/kvm/svm.c | 28 ++++---- arch/x86/kvm/x86.c | 8 +- arch/x86/lib/usercopy_32.c | 4 - arch/x86/mm/highmem_32.c | 45 +++++++++----- arch/x86/mm/iomap_32.c | 58 ++++-------------- arch/x86/xen/mmu.c | 6 - block/blk-settings.c | 2 crypto/ahash.c | 4 - crypto/async_tx/async_memcpy.c | 8 +- crypto/blkcipher.c | 10 +-- crypto/ccm.c | 4 - crypto/digest.c | 4 - crypto/scatterwalk.c | 8 +- crypto/shash.c | 8 +- drivers/ata/libata-sff.c | 8 +- drivers/block/brd.c | 20 +++--- drivers/block/loop.c | 16 ++--- drivers/block/pktcdvd.c | 8 +- drivers/crypto/hifn_795x.c | 10 +-- drivers/edac/edac_mc.c | 4 - drivers/gpu/drm/drm_cache.c | 8 +- drivers/gpu/drm/i915/i915_gem.c | 44 +++++++------- drivers/gpu/drm/i915/i915_gem_debug.c | 10 +-- drivers/gpu/drm/ttm/ttm_bo_util.c | 8 +- drivers/gpu/drm/ttm/ttm_tt.c | 16 ++--- drivers/ide/ide-taskfile.c | 4 - drivers/infiniband/ulp/iser/iser_memory.c | 8 +- drivers/md/bitmap.c | 36 +++++------ drivers/media/video/ivtv/ivtv-udma.c | 4 - drivers/memstick/host/jmb38x_ms.c | 4 - drivers/memstick/host/tifm_ms.c | 4 - drivers/mmc/host/at91_mci.c | 8 +- drivers/mmc/host/mmci.c | 4 - drivers/mmc/host/mmci.h | 8 +- drivers/mmc/host/msm_sdcc.c | 6 - drivers/mmc/host/sdhci.c | 16 ++--- drivers/mmc/host/tifm_sd.c | 16 ++--- drivers/mmc/host/tmio_mmc.c | 4 - drivers/mmc/host/tmio_mmc.h | 8 +- drivers/net/cassini.c | 4 - drivers/net/e1000/e1000_main.c | 6 - drivers/net/e1000e/netdev.c | 12 +-- drivers/scsi/arcmsr/arcmsr_hba.c | 8 +- drivers/scsi/cxgb3i/cxgb3i_pdu.c | 5 - drivers/scsi/dc395x.c | 12 +-- drivers/scsi/fcoe/fcoe.c | 9 +- drivers/scsi/gdth.c | 6 - drivers/scsi/ips.c | 8 +- drivers/scsi/libfc/fc_fcp.c | 12 +-- drivers/scsi/libiscsi_tcp.c | 4 - drivers/scsi/libsas/sas_host_smp.c | 8 +- drivers/scsi/megaraid.c | 4 - drivers/scsi/mvsas/mv_sas.c | 8 +- drivers/scsi/scsi_debug.c | 24 +++---- drivers/scsi/scsi_lib.c | 16 ++--- drivers/scsi/sd_dif.c | 12 +-- drivers/scsi/tmscsim.c | 4 - drivers/staging/hv/RndisFilter.c | 9 +- drivers/staging/hv/netvsc_drv.c | 9 +- drivers/staging/hv/storvsc_drv.c | 29 ++++----- drivers/staging/pohmelfs/inode.c | 8 +- fs/afs/fsclient.c | 8 +- fs/afs/mntpt.c | 4 - fs/aio.c | 36 +++++------ fs/bio-integrity.c | 10 +-- fs/btrfs/compression.c | 8 +- fs/btrfs/ctree.h | 8 +- fs/btrfs/extent_io.c | 60 +++++++++---------- fs/btrfs/file-item.c | 10 +-- fs/btrfs/inode.c | 22 +++---- fs/btrfs/zlib.c | 8 +- fs/cifs/file.c | 4 - fs/ecryptfs/mmap.c | 4 - fs/ecryptfs/read_write.c | 8 +- fs/exec.c | 4 - fs/exofs/dir.c | 4 - fs/ext2/dir.c | 4 - fs/fuse/dev.c | 12 +-- fs/fuse/file.c | 4 - fs/gfs2/aops.c | 12 +-- fs/gfs2/lops.c | 8 +- fs/gfs2/quota.c | 4 - fs/jbd/journal.c | 12 +-- fs/jbd/transaction.c | 4 - fs/jbd2/commit.c | 4 - fs/jbd2/journal.c | 12 +-- fs/jbd2/transaction.c | 4 - fs/libfs.c | 4 - fs/minix/dir.c | 4 - fs/namei.c | 4 - fs/nfs/dir.c | 4 - fs/nfs/nfs2xdr.c | 8 +- fs/nfs/nfs3xdr.c | 8 +- fs/nfs/nfs4proc.c | 4 - fs/nfs/nfs4xdr.c | 10 +-- fs/nilfs2/cpfile.c | 94 +++++++++++++++--------------- fs/nilfs2/dat.c | 38 ++++++------ fs/nilfs2/dir.c | 4 - fs/nilfs2/ifile.c | 4 - fs/nilfs2/mdt.c | 4 - fs/nilfs2/page.c | 8 +- fs/nilfs2/recovery.c | 4 - fs/nilfs2/segbuf.c | 4 - fs/nilfs2/segment.c | 4 - fs/nilfs2/sufile.c | 50 +++++++-------- fs/ntfs/ChangeLog | 4 - fs/ntfs/aops.c | 20 +++--- fs/ntfs/attrib.c | 20 +++--- fs/ntfs/file.c | 16 ++--- fs/ntfs/super.c | 8 +- fs/ocfs2/aops.c | 16 ++--- fs/pipe.c | 8 +- fs/reiserfs/stree.c | 4 - fs/reiserfs/tail_conversion.c | 4 - fs/splice.c | 4 - fs/squashfs/file.c | 8 +- fs/squashfs/symlink.c | 6 - fs/ubifs/file.c | 4 - fs/udf/file.c | 4 - include/crypto/scatterwalk.h | 35 ++--------- include/linux/bio.h | 10 +-- include/linux/highmem.h | 58 ++++++++++-------- include/linux/io-mapping.h | 5 - include/linux/scatterlist.h | 2 include/scsi/scsi_cmnd.h | 4 - kernel/power/snapshot.c | 30 ++++----- lib/scatterlist.c | 4 - lib/swiotlb.c | 5 - mm/bounce.c | 4 - mm/debug-pagealloc.c | 2 mm/filemap.c | 8 +- mm/highmem.c | 48 --------------- mm/ksm.c | 12 +-- mm/memory.c | 4 - mm/shmem.c | 18 ++--- mm/vmalloc.c | 8 +- net/core/kmap_skb.h | 4 - net/rds/ib_recv.c | 12 +-- net/rds/info.c | 6 - net/rds/iw_recv.c | 4 - net/rds/page.c | 4 - net/sunrpc/auth_gss/gss_krb5_wrap.c | 4 - net/sunrpc/socklib.c | 4 - net/sunrpc/xdr.c | 16 ++--- net/sunrpc/xprtrdma/rpc_rdma.c | 10 +-- 201 files changed, 1055 insertions(+), 1155 deletions(-)
Documentation/block/biodoc.txt +2 -2
@@ -217,7 +217,7 @@ may need to abort DMA operations and rev which case a virtual mapping of the page is required. For SCSI it is also done in some scenarios where the low level driver cannot be trusted to handle a single sg entry correctly. The driver is expected to perform the -kmaps as needed on such occasions using the __bio_kmap_atomic and bio_kmap_irq +kmaps as needed on such occasions using the __bio_kmap_atomic_push and bio_kmap_irq routines as appropriate. A driver could also use the blk_queue_bounce() routine on its own to bounce highmem i/o to low memory for specific requests if so desired.
@@ -1167,7 +1167,7 @@ use blk_rq_map_sg for scatter gather) to PIO drivers (or drivers that need to revert to PIO transfer once in a while (IDE for example)), where the CPU is doing the actual data transfer a virtual mapping is needed. If the driver supports highmem I/O, -(Sec 1.1, (ii) ) it needs to use __bio_kmap_atomic and bio_kmap_irq to +(Sec 1.1, (ii) ) it needs to use __bio_kmap_atomic_push and bio_kmap_irq to temporarily map a bio into the virtual address space.
Documentation/frv/mmu-layout.txt +18 -18
@@ -29,7 +29,7 @@ Certain control registers are used by th DAMR3 Current PGD mapping SCR0, DAMR4 Instruction TLB PGE/PTD cache SCR1, DAMR5 Data TLB PGE/PTD cache - DAMR6-10 kmap_atomic() mappings + DAMR6-10 kmap_atomic_push() mappings DAMR11 I/O mapping CXNR mm_struct context ID TTBR Page directory (PGD) pointer (physical address)
@@ -64,17 +64,17 @@ The virtual memory layout is: C0000000-CFFFFFFF 00000000 xAMPR0 -L-S--V 256MB Kernel image and data D0000000-D7FFFFFF various TLB,xAMR1 D-NS??V 128MB vmalloc area D8000000-DBFFFFFF various TLB,xAMR1 D-NS??V 64MB kmap() area - DC000000-DCFFFFFF various TLB 1MB Secondary kmap_atomic() frame - DD000000-DD27FFFF various DAMR 160KB Primary kmap_atomic() frame + DC000000-DCFFFFFF various TLB 1MB Secondary kmap_atomic_push() frame + DD000000-DD27FFFF various DAMR 160KB Primary kmap_atomic_push() frame DD040000 DAMR2/IAMR2 -L-S--V page Page cache flush attachment point DD080000 DAMR3 -L-SC-V page Page Directory (PGD) DD0C0000 DAMR4 -L-SC-V page Cached insn TLB Page Table lookup DD100000 DAMR5 -L-SC-V page Cached data TLB Page Table lookup - DD140000 DAMR6 -L-S--V page kmap_atomic(KM_BOUNCE_READ) - DD180000 DAMR7 -L-S--V page kmap_atomic(KM_SKB_SUNRPC_DATA) - DD1C0000 DAMR8 -L-S--V page kmap_atomic(KM_SKB_DATA_SOFTIRQ) - DD200000 DAMR9 -L-S--V page kmap_atomic(KM_USER0) - DD240000 DAMR10 -L-S--V page kmap_atomic(KM_USER1) + DD140000 DAMR6 -L-S--V page kmap_atomic_push(KM_BOUNCE_READ) + DD180000 DAMR7 -L-S--V page kmap_atomic_push(KM_SKB_SUNRPC_DATA) + DD1C0000 DAMR8 -L-S--V page kmap_atomic_push(KM_SKB_DATA_SOFTIRQ) + DD200000 DAMR9 -L-S--V page kmap_atomic_push(KM_USER0) + DD240000 DAMR10 -L-S--V page kmap_atomic_push(KM_USER1) E0000000-FFFFFFFF E0000000 DAMR11 -L-SC-V 512MB I/O region IAMPR1 and DAMPR1 are used as an extension to the TLB.
@@ -85,27 +85,27 @@ KMAP AND KMAP_ATOMIC ==================== To access pages in the page cache (which may not be directly accessible if highmem is available), -the kernel calls kmap(), does the access and then calls kunmap(); or it calls kmap_atomic(), does -the access and then calls kunmap_atomic(). +the kernel calls kmap(), does the access and then calls kunmap(); or it calls kmap_atomic_push(), does +the access and then calls kmap_atomic_pop(). kmap() creates an attachment between an arbitrary inaccessible page and a range of virtual addresses by installing a PTE in a special page table. The kernel can then access this page as it wills. When it's finished, the kernel calls kunmap() to clear the PTE. -kmap_atomic() does something slightly different. In the interests of speed, it chooses one of two +kmap_atomic_push() does something slightly different. In the interests of speed, it chooses one of two strategies: - (1) If possible, kmap_atomic() attaches the requested page to one of DAMPR5 through DAMPR10 - register pairs; and the matching kunmap_atomic() clears the DAMPR. This makes high memory + (1) If possible, kmap_atomic_push() attaches the requested page to one of DAMPR5 through DAMPR10 + register pairs; and the matching kmap_atomic_pop() clears the DAMPR. This makes high memory support really fast as there's no need to flush the TLB or modify the page tables. The DAMLR registers being used for this are preset during boot and don't change over the lifetime of the - process. There's a direct mapping between the first few kmap_atomic() types, DAMR number and + process. There's a direct mapping between the first few kmap_atomic_push() types, DAMR number and virtual address slot. - However, there are more kmap_atomic() types defined than there are DAMR registers available, + However, there are more kmap_atomic_push() types defined than there are DAMR registers available, so we fall back to: - (2) kmap_atomic() uses a slot in the secondary frame (determined by the type parameter), and then + (2) kmap_atomic_push() uses a slot in the secondary frame (determined by the type parameter), and then locks an entry in the TLB to translate that slot to the specified page. The number of slots is obviously limited, and their positions are controlled such that each slot is matched by a different line in the TLB. kunmap() ejects the entry from the TLB.
@@ -113,9 +113,9 @@ strategies: Note that the first three kmap atomic types are really just declared as placeholders. The DAMPR registers involved are actually modified directly. -Also note that kmap() itself may sleep, kmap_atomic() may never sleep and both always succeed; +Also note that kmap() itself may sleep, kmap_atomic_push() may never sleep and both always succeed; furthermore, a driver using kmap() may sleep before calling kunmap(), but may not sleep before -calling kunmap_atomic() if it had previously called kmap_atomic(). +calling kmap_atomic_pop() if it had previously called kmap_atomic_push(). ===============================
Documentation/io-mapping.txt +2 -2
@@ -72,8 +72,8 @@ map_atomic and map functions add the req virtual address returned by ioremap_wc. On 32-bit processors with HIGHMEM defined, io_mapping_map_atomic_wc uses -kmap_atomic_pfn to map the specified page in an atomic fashion; -kmap_atomic_pfn isn't really supposed to be used with device pages, but it +kmap_atomic_push_pfn to map the specified page in an atomic fashion; +kmap_atomic_push_pfn isn't really supposed to be used with device pages, but it provides an efficient mapping for this usage. On 32-bit processors without HIGHMEM defined, io_mapping_map_atomic_wc and
arch/arm/include/asm/highmem.h +3 -3
@@ -23,9 +23,9 @@ extern void kunmap_high(struct page *pagextern void *kmap (struct page *page);extern void kunmap (struct page *page); -extern void *kmap_atomic (struct page *page,enum km_type type); -extern void kunmap_atomic (void *kvaddr,enum km_type type); -extern void *kmap_atomic_pfn (unsigned long pfn,enum km_type type); +extern void *kmap_atomic_push (struct page *page); +extern void kmap_atomic_pop (void *kvaddr); +extern void *kmap_atomic_push_pfn (unsigned long pfn);extern struct page *kmap_atomic_to_page (const void *ptr);#endif
arch/arm/include/asm/pgtable.h +6 -6
@@ -263,17 +263,17 @@ extern struct page *empty_zero_page;#define pte_page (pte) (pfn_to_page(pte_pfn(pte)))#define pte_offset_kernel (dir,addr) (pmd_page_vaddr(*(dir)) + __pte_index(addr)) -#define pte_offset_map (dir,addr) (__pte_map(dir, KM_PTE0) + __pte_index(addr)) -#define pte_offset_map_nested (dir,addr) (__pte_map(dir, KM_PTE1) + __pte_index(addr)) -#define pte_unmap (pte) __pte_unmap(pte, KM_PTE0) -#define pte_unmap_nested (pte) __pte_unmap(pte, KM_PTE1) +#define pte_offset_map (dir,addr) (__pte_map(dir) + __pte_index(addr)) +#define pte_offset_map_nested (dir,addr) (__pte_map(dir) + __pte_index(addr)) +#define pte_unmap (pte) __pte_unmap(pte) +#define pte_unmap_nested (pte) __pte_unmap(pte)#ifndef CONFIG_HIGHPTE #define __pte_map (dir,km) pmd_page_vaddr(*(dir))#define __pte_unmap (pte,km) do { } while (0)#else -#define __pte_map (dir,km) ((pte_t *)kmap_atomic(pmd_page(*(dir)), km) + PTRS_PER_PTE) -#define __pte_unmap (pte,km) kunmap_atomic((pte - PTRS_PER_PTE), km) +#define __pte_map (dir,km) ((pte_t *)kmap_atomic_push(pmd_page(*(dir)), km) + PTRS_PER_PTE) +#define __pte_unmap (pte,km) kmap_atomic_pop((pte - PTRS_PER_PTE), km)#endif #define set_pte_ext (ptep,pte,ext) cpu_set_pte_ext(ptep,pte,ext)
arch/arm/mm/copypage-fa.c +6 -6
@@ -44,11 +44,11 @@ void fa_copy_user_highpage(struct page * {void *kto, *kfrom; - kto =kmap_atomic (to,KM_USER0 ); - kfrom =kmap_atomic (from,KM_USER1 ); + kto =kmap_atomic_push (to); + kfrom =kmap_atomic_push (from);fa_copy_user_page (kto, kfrom); -kunmap_atomic (kfrom,KM_USER1 ); -kunmap_atomic (kto,KM_USER0 ); +kmap_atomic_pop (kfrom); +kmap_atomic_pop (kto); }/*
@@ -58,7 +58,7 @@ void fa_copy_user_highpage(struct page **/ void fa_clear_user_highpage (struct page *page,unsigned long vaddr) { -void *ptr, *kaddr =kmap_atomic (page,KM_USER0 ); +void *ptr, *kaddr =kmap_atomic_push (page); asm volatile("\ mov r1, %2 @ 1\n\ mov r2, #0 @ 1\n\
@@ -77,7 +77,7 @@ void fa_clear_user_highpage(struct page : "=r" (ptr) : "0" (kaddr), "I" (PAGE_SIZE / 32) : "r1", "r2", "r3", "ip", "lr"); -kunmap_atomic (kaddr,KM_USER0 ); +kmap_atomic_pop (kaddr); }struct cpu_user_fns fa_user_fns__initdata = {
arch/arm/mm/copypage-feroceon.c +6 -6
@@ -72,16 +72,16 @@ void feroceon_copy_user_highpage(struct {void *kto, *kfrom; - kto =kmap_atomic (to,KM_USER0 ); - kfrom =kmap_atomic (from,KM_USER1 ); + kto =kmap_atomic_push (to); + kfrom =kmap_atomic_push (from);feroceon_copy_user_page (kto, kfrom); -kunmap_atomic (kfrom,KM_USER1 ); -kunmap_atomic (kto,KM_USER0 ); +kmap_atomic_pop (kfrom); +kmap_atomic_pop (kto); }void feroceon_clear_user_highpage (struct page *page,unsigned long vaddr) { -void *ptr, *kaddr =kmap_atomic (page,KM_USER0 ); +void *ptr, *kaddr =kmap_atomic_push (page); asm volatile ("\ mov r1, %2 \n\ mov r2, #0 \n\
@@ -101,7 +101,7 @@ void feroceon_clear_user_highpage(struct : "=r" (ptr) : "0" (kaddr), "I" (PAGE_SIZE / 32) : "r1", "r2", "r3", "r4", "r5", "r6", "r7", "ip", "lr"); -kunmap_atomic (kaddr,KM_USER0 ); +kmap_atomic_pop (kaddr); }struct cpu_user_fns feroceon_user_fns__initdata = {
arch/arm/mm/copypage-v3.c +6 -6
@@ -42,11 +42,11 @@ void v3_copy_user_highpage(struct page * {void *kto, *kfrom; - kto =kmap_atomic (to,KM_USER0 ); - kfrom =kmap_atomic (from,KM_USER1 ); + kto =kmap_atomic_push (to); + kfrom =kmap_atomic_push (from);v3_copy_user_page (kto, kfrom); -kunmap_atomic (kfrom,KM_USER1 ); -kunmap_atomic (kto,KM_USER0 ); +kmap_atomic_pop (kfrom); +kmap_atomic_pop (kto); }/*
@@ -56,7 +56,7 @@ void v3_copy_user_highpage(struct page **/ void v3_clear_user_highpage (struct page *page,unsigned long vaddr) { -void *ptr, *kaddr =kmap_atomic (page,KM_USER0 ); +void *ptr, *kaddr =kmap_atomic_push (page); asm volatile("\n\ mov r1, %2 @ 1\n\ mov r2, #0 @ 1\n\
@@ -72,7 +72,7 @@ void v3_clear_user_highpage(struct page : "=r" (ptr) : "0" (kaddr), "I" (PAGE_SIZE / 64) : "r1", "r2", "r3", "ip", "lr"); -kunmap_atomic (kaddr,KM_USER0 ); +kmap_atomic_pop (kaddr); }struct cpu_user_fns v3_user_fns__initdata = {
arch/arm/mm/copypage-v4mc.c +4 -4
@@ -71,7 +71,7 @@ mc_copy_user_page(void *from, void *to)void v4_mc_copy_user_highpage (struct page *to,struct page *from,unsigned long vaddr) { -void *kto =kmap_atomic (to,KM_USER1 ); +void *kto =kmap_atomic_push (to);if (test_and_clear_bit (PG_dcache_dirty, &from->flags))__flush_dcache_page (page_mapping (from), from);
@@ -85,7 +85,7 @@ void v4_mc_copy_user_highpage(struct pagspin_unlock (&minicache_lock); -kunmap_atomic (kto,KM_USER1 ); +kmap_atomic_pop (kto); }/*
@@ -93,7 +93,7 @@ void v4_mc_copy_user_highpage(struct pag*/ void v4_mc_clear_user_highpage (struct page *page,unsigned long vaddr) { -void *ptr, *kaddr =kmap_atomic (page,KM_USER0 ); +void *ptr, *kaddr =kmap_atomic_push (page); asm volatile("\ mov r1, %2 @ 1\n\ mov r2, #0 @ 1\n\
@@ -111,7 +111,7 @@ void v4_mc_clear_user_highpage(struct pa : "=r" (ptr) : "0" (kaddr), "I" (PAGE_SIZE / 64) : "r1", "r2", "r3", "ip", "lr"); -kunmap_atomic (kaddr,KM_USER0 ); +kmap_atomic_pop (kaddr); }struct cpu_user_fns v4_mc_user_fns__initdata = {
arch/arm/mm/copypage-v4wb.c +6 -6
@@ -52,11 +52,11 @@ void v4wb_copy_user_highpage(struct page {void *kto, *kfrom; - kto =kmap_atomic (to,KM_USER0 ); - kfrom =kmap_atomic (from,KM_USER1 ); + kto =kmap_atomic_push (to); + kfrom =kmap_atomic_push (from);v4wb_copy_user_page (kto, kfrom); -kunmap_atomic (kfrom,KM_USER1 ); -kunmap_atomic (kto,KM_USER0 ); +kmap_atomic_pop (kfrom); +kmap_atomic_pop (kto); }/*
@@ -66,7 +66,7 @@ void v4wb_copy_user_highpage(struct page*/ void v4wb_clear_user_highpage (struct page *page,unsigned long vaddr) { -void *ptr, *kaddr =kmap_atomic (page,KM_USER0 ); +void *ptr, *kaddr =kmap_atomic_push (page); asm volatile("\ mov r1, %2 @ 1\n\ mov r2, #0 @ 1\n\
@@ -85,7 +85,7 @@ void v4wb_clear_user_highpage(struct pag : "=r" (ptr) : "0" (kaddr), "I" (PAGE_SIZE / 64) : "r1", "r2", "r3", "ip", "lr"); -kunmap_atomic (kaddr,KM_USER0 ); +kmap_atomic_pop (kaddr); }struct cpu_user_fns v4wb_user_fns__initdata = {
arch/arm/mm/copypage-v4wt.c +6 -6
@@ -48,11 +48,11 @@ void v4wt_copy_user_highpage(struct page {void *kto, *kfrom; - kto =kmap_atomic (to,KM_USER0 ); - kfrom =kmap_atomic (from,KM_USER1 ); + kto =kmap_atomic_push (to); + kfrom =kmap_atomic_push (from);v4wt_copy_user_page (kto, kfrom); -kunmap_atomic (kfrom,KM_USER1 ); -kunmap_atomic (kto,KM_USER0 ); +kmap_atomic_pop (kfrom); +kmap_atomic_pop (kto); }/*
@@ -62,7 +62,7 @@ void v4wt_copy_user_highpage(struct page*/ void v4wt_clear_user_highpage (struct page *page,unsigned long vaddr) { -void *ptr, *kaddr =kmap_atomic (page,KM_USER0 ); +void *ptr, *kaddr =kmap_atomic_push (page); asm volatile("\ mov r1, %2 @ 1\n\ mov r2, #0 @ 1\n\
@@ -79,7 +79,7 @@ void v4wt_clear_user_highpage(struct pag : "=r" (ptr) : "0" (kaddr), "I" (PAGE_SIZE / 64) : "r1", "r2", "r3", "ip", "lr"); -kunmap_atomic (kaddr,KM_USER0 ); +kmap_atomic_pop (kaddr); }struct cpu_user_fns v4wt_user_fns__initdata = {
arch/arm/mm/copypage-v6.c +6 -6
@@ -38,11 +38,11 @@ static void v6_copy_user_highpage_nonali {void *kto, *kfrom; - kfrom =kmap_atomic (from,KM_USER0 ); - kto =kmap_atomic (to,KM_USER1 ); + kfrom =kmap_atomic_push (from); + kto =kmap_atomic_push (to);copy_page (kto, kfrom); -kunmap_atomic (kto,KM_USER1 ); -kunmap_atomic (kfrom,KM_USER0 ); +kmap_atomic_pop (kto); +kmap_atomic_pop (kfrom); }/*
@@ -51,9 +51,9 @@ static void v6_copy_user_highpage_nonali*/ static void v6_clear_user_highpage_nonaliasing (struct page *page,unsigned long vaddr) { -void *kaddr =kmap_atomic (page,KM_USER0 ); +void *kaddr =kmap_atomic_push (page);clear_page (kaddr); -kunmap_atomic (kaddr,KM_USER0 ); +kmap_atomic_pop (kaddr); }/*
arch/arm/mm/copypage-xsc3.c +6 -6
@@ -75,11 +75,11 @@ void xsc3_mc_copy_user_highpage(struct p {void *kto, *kfrom; - kto =kmap_atomic (to,KM_USER0 ); - kfrom =kmap_atomic (from,KM_USER1 ); + kto =kmap_atomic_push (to); + kfrom =kmap_atomic_push (from);xsc3_mc_copy_user_page (kto, kfrom); -kunmap_atomic (kfrom,KM_USER1 ); -kunmap_atomic (kto,KM_USER0 ); +kmap_atomic_pop (kfrom); +kmap_atomic_pop (kto); }/*
@@ -89,7 +89,7 @@ void xsc3_mc_copy_user_highpage(struct p*/ void xsc3_mc_clear_user_highpage (struct page *page,unsigned long vaddr) { -void *ptr, *kaddr =kmap_atomic (page,KM_USER0 ); +void *ptr, *kaddr =kmap_atomic_push (page); asm volatile ("\ mov r1, %2 \n\ mov r2, #0 \n\
@@ -104,7 +104,7 @@ void xsc3_mc_clear_user_highpage(struct : "=r" (ptr) : "0" (kaddr), "I" (PAGE_SIZE / 32) : "r1", "r2", "r3"); -kunmap_atomic (kaddr,KM_USER0 ); +kmap_atomic_pop (kaddr); }struct cpu_user_fns xsc3_mc_user_fns__initdata = {
arch/arm/mm/copypage-xscale.c +4 -4
@@ -93,7 +93,7 @@ mc_copy_user_page(void *from, void *to)void xscale_mc_copy_user_highpage (struct page *to,struct page *from,unsigned long vaddr) { -void *kto =kmap_atomic (to,KM_USER1 ); +void *kto =kmap_atomic_push (to);if (test_and_clear_bit (PG_dcache_dirty, &from->flags))__flush_dcache_page (page_mapping (from), from);
@@ -107,7 +107,7 @@ void xscale_mc_copy_user_highpage(structspin_unlock (&minicache_lock); -kunmap_atomic (kto,KM_USER1 ); +kmap_atomic_pop (kto); }/*
@@ -116,7 +116,7 @@ void xscale_mc_copy_user_highpage(structvoid xscale_mc_clear_user_highpage (struct page *page,unsigned long vaddr) { -void *ptr, *kaddr =kmap_atomic (page,KM_USER0 ); +void *ptr, *kaddr =kmap_atomic_push (page); asm volatile( "mov r1, %2 \n\ mov r2, #0 \n\
@@ -133,7 +133,7 @@ xscale_mc_clear_user_highpage(struct pag : "=r" (ptr) : "0" (kaddr), "I" (PAGE_SIZE / 32) : "r1", "r2", "r3", "ip"); -kunmap_atomic (kaddr,KM_USER0 ); +kmap_atomic_pop (kaddr); }struct cpu_user_fns xscale_mc_user_fns__initdata = {
arch/arm/mm/flush.c +2 -2
@@ -146,8 +146,8 @@ void __flush_dcache_page(struct address_*/ #ifdef CONFIG_HIGHMEM /* -* kmap_atomic() doesn't set the page virtual address, and -* kunmap_atomic() takes care of cache flushing already. +* kmap_atomic_push() doesn't set the page virtual address, and +* kmap_atomic_pop() takes care of cache flushing already. */ if (page_address(page))#endif
arch/arm/mm/highmem.c +10 -10
@@ -36,7 +36,7 @@ void kunmap(struct page *page) }EXPORT_SYMBOL (kunmap); -void *kmap_atomic (struct page *page,enum km_type type) +void *kmap_atomic_push (struct page *page) {unsigned int idx;unsigned long vaddr;
@@ -50,18 +50,18 @@ void *kmap_atomic(struct page *page, enuif (kmap)return kmap; - idx = type +KM_TYPE_NR *smp_processor_id (); + idx =kmap_atomic_push_idx () +KM_TYPE_NR *smp_processor_id (); vaddr =__fix_to_virt (FIX_KMAP_BEGIN + idx);#ifdef CONFIG_DEBUG_HIGHMEM /* -* With debugging enabled, kunmap_atomic forces that entry to 0. +* With debugging enabled, kmap_atomic_pop forces that entry to 0. * Make sure it was indeed properly unmapped. */ BUG_ON (!pte_none (*(TOP_PTE (vaddr))));#endif set_pte_ext (TOP_PTE (vaddr),mk_pte (page, kmap_prot),0 );/* -* When debugging is off, kunmap_atomic leaves the previous mapping +* When debugging is off, kmap_atomic_pop leaves the previous mapping * in place, so this TLB flush ensures the TLB is updated with the * new mapping. */
@@ -69,12 +69,12 @@ void *kmap_atomic(struct page *page, enureturn (void *)vaddr; } -EXPORT_SYMBOL (kmap_atomic); +EXPORT_SYMBOL (kmap_atomic_push); -void kunmap_atomic (void *kvaddr,enum km_type type) +void kmap_atomic_pop (void *kvaddr) {unsigned long vaddr = (unsigned long ) kvaddr &PAGE_MASK ; -unsigned int idx = type +KM_TYPE_NR *smp_processor_id (); +unsigned int idx =kmap_atomic_pop_idx () +KM_TYPE_NR *smp_processor_id ();if (kvaddr >= (void *)FIXADDR_START ) {__cpuc_flush_dcache_page ((void *)vaddr);
@@ -91,16 +91,16 @@ void kunmap_atomic(void *kvaddr, enum km }pagefault_enable (); } -EXPORT_SYMBOL (kunmap_atomic); +EXPORT_SYMBOL (kmap_atomic_pop); -void *kmap_atomic_pfn (unsigned long pfn,enum km_type type) +void *kmap_atomic_push_pfn (unsigned long pfn) {unsigned int idx;unsigned long vaddr;pagefault_disable (); - idx = type +KM_TYPE_NR *smp_processor_id (); + idx =kmap_atomic_push_idx () +KM_TYPE_NR *smp_processor_id (); vaddr =__fix_to_virt (FIX_KMAP_BEGIN + idx);#ifdef CONFIG_DEBUG_HIGHMEM BUG_ON (!pte_none (*(TOP_PTE (vaddr))));
arch/frv/include/asm/highmem.h +33 -30
115 lines not on this page · raw message
arch/frv/include/asm/pgtable.h +4 -4
@@ -451,11 +451,11 @@ static inline pte_t pte_modify(pte_t pte#if defined(CONFIG_HIGHPTE )#define pte_offset_map (dir, address) \ - ((pte_t *)kmap_atomic(pmd_page(*(dir)),KM_PTE0) + pte_index(address)) + ((pte_t *)kmap_atomic_push(pmd_page(*(dir))) + pte_index(address))#define pte_offset_map_nested (dir, address) \ - ((pte_t *)kmap_atomic(pmd_page(*(dir)),KM_PTE1) + pte_index(address)) -#define pte_unmap (pte) kunmap_atomic(pte, KM_PTE0) -#define pte_unmap_nested (pte) kunmap_atomic((pte), KM_PTE1) + ((pte_t *)kmap_atomic_push(pmd_page(*(dir))) + pte_index(address)) +#define pte_unmap (pte) kmap_atomic_pop(pte) +#define pte_unmap_nested (pte) kmap_atomic_pop((pte))#else #define pte_offset_map (dir, address) \ ((pte_t *)page_address(pmd_page(*(dir))) + pte_index(address))
arch/frv/kernel/head-mmu-fr451.S +1 -1
@@ -276,7 +276,7 @@ __head_fr451_set_protection: movgs gr9,damlr1 movgs gr8,dampr1 - # we use DAMR2-10 for kmap_atomic(), cache flush and TLB management + # we use DAMR2-10 for kmap_atomic_push(), cache flush and TLB management # since the DAMLR regs are not going to change, we can set them now # also set up IAMLR2 to the same as DAMLR5 sethi.p %hi(KMAP_ATOMIC_PRIMARY_FRAME),gr4
arch/frv/mb93090-mb00/pci-dma.c +2 -2
@@ -85,14 +85,14 @@ int dma_map_sg(struct device *dev, struc dampr2 =__get_DAMPR (2 );for (i =0 ; i < nents; i++) { - vaddr =kmap_atomic (sg_page (&sg[i]), __KM_CACHE); + vaddr =kmap_atomic_push (sg_page (&sg[i]), __KM_CACHE);frv_dcache_writeback ((unsigned long ) vaddr, (unsigned long ) vaddr +PAGE_SIZE ); } -kunmap_atomic (vaddr, __KM_CACHE); +kmap_atomic_pop (vaddr, __KM_CACHE);if (dampr2) {__set_DAMPR (2 , dampr2);__set_IAMPR (2 , dampr2);
arch/frv/mm/cache-page.c +5 -5
@@ -17,7 +17,7 @@/*****************************************************************************/ /* * DCF takes a virtual address and the page may not currently have one -* - temporarily hijack a kmap_atomic() slot and attach the page to it +* - temporarily hijack a kmap_atomic_push() slot and attach the page to it */ void flush_dcache_page (struct page *page) {
@@ -26,11 +26,11 @@ void flush_dcache_page(struct page *page dampr2 =__get_DAMPR (2 ); - vaddr =kmap_atomic (page, __KM_CACHE); + vaddr =kmap_atomic_push (page, __KM_CACHE);frv_dcache_writeback ((unsigned long ) vaddr, (unsigned long ) vaddr +PAGE_SIZE ); -kunmap_atomic (vaddr, __KM_CACHE); +kmap_atomic_pop (vaddr, __KM_CACHE);if (dampr2) {__set_DAMPR (2 , dampr2);
@@ -54,12 +54,12 @@ void flush_icache_user_range(struct vm_a dampr2 =__get_DAMPR (2 ); - vaddr =kmap_atomic (page, __KM_CACHE); + vaddr =kmap_atomic_push (page, __KM_CACHE); start = (start & ~PAGE_MASK ) | (unsigned long ) vaddr;frv_cache_wback_inv (start, start + len); -kunmap_atomic (vaddr, __KM_CACHE); +kmap_atomic_pop (vaddr, __KM_CACHE);if (dampr2) {__set_DAMPR (2 , dampr2);
arch/ia64/kernel/crash_dump.c +1 -1
@@ -27,7 +27,7 @@ unsigned long long elfcorehdr_addr = ELF* otherwise @buf is in kernel address space, use memcpy(). * * Copy a page from "oldmem". For this page, there is no pte mapped -* in the current kernel. We stitch up a pte, similar to kmap_atomic. +* in the current kernel. We stitch up a pte, similar to kmap_atomic_push. * * Calling copy_to_user() in atomic context is not desirable. Hence first * copying the data to a pre-allocated kernel page and then copying to user
arch/microblaze/include/asm/pgtable.h +4 -4
@@ -478,12 +478,12 @@ static inline pmd_t *pmd_offset(pgd_t *d#define pte_offset_kernel (dir, addr) \ ((pte_t *) pmd_page_kernel(*(dir)) + pte_index(addr))#define pte_offset_map (dir, addr) \ - ((pte_t *) kmap_atomic(pmd_page(*(dir)), KM_PTE0) + pte_index(addr)) + ((pte_t *) kmap_atomic_push(pmd_page(*(dir))) + pte_index(addr))#define pte_offset_map_nested (dir, addr) \ - ((pte_t *) kmap_atomic(pmd_page(*(dir)), KM_PTE1) + pte_index(addr)) + ((pte_t *) kmap_atomic_push(pmd_page(*(dir))) + pte_index(addr)) -#define pte_unmap (pte) kunmap_atomic(pte, KM_PTE0) -#define pte_unmap_nested (pte) kunmap_atomic(pte, KM_PTE1) +#define pte_unmap (pte) kmap_atomic_pop(pte) +#define pte_unmap_nested (pte) kmap_atomic_pop(pte)/* Encode and decode a nonlinear file mapping entry */ #define PTE_FILE_MAX_BITS 29
arch/mips/include/asm/highmem.h +5 -5
@@ -47,15 +47,15 @@ extern void kunmap_high(struct page *pagextern void *__kmap (struct page *page);extern void __kunmap (struct page *page); -extern void *__kmap_atomic (struct page *page,enum km_type type); -extern void __kunmap_atomic (void *kvaddr,enum km_type type); -extern void *kmap_atomic_pfn (unsigned long pfn,enum km_type type); +extern void *__kmap_atomic_push (struct page *page); +extern void __kmap_atomic_pop (void *kvaddr); +extern void *kmap_atomic_push_pfn (unsigned long pfn);extern struct page *__kmap_atomic_to_page (void *ptr);#define kmap __kmap#define kunmap __kunmap -#define kmap_atomic __kmap_atomic -#define kunmap_atomic __kunmap_atomic +#define kmap_atomic_push __kmap_atomic_push +#define kmap_atomic_pop __kmap_atomic_pop#define kmap_atomic_to_page __kmap_atomic_to_page#define flush_cache_kmaps () flush_cache_all()
arch/mips/mm/c-r4k.c +3 -3
@@ -490,7 +490,7 @@ static inline void local_r4k_flush_cache vaddr =NULL ; else {/* -* Use kmap_coherent or kmap_atomic to do flushes for +* Use kmap_coherent or kmap_atomic_push to do flushes for * another ASID than the current one. */ map_coherent = (cpu_has_dc_aliases &&
@@ -498,7 +498,7 @@ static inline void local_r4k_flush_cacheif (map_coherent) vaddr =kmap_coherent (page, addr);else - vaddr =kmap_atomic (page,KM_USER0 ); + vaddr =kmap_atomic_push (page); addr = (unsigned long )vaddr; }
@@ -521,7 +521,7 @@ static inline void local_r4k_flush_cacheif (map_coherent)kunmap_coherent ();else -kunmap_atomic (vaddr,KM_USER0 ); +kmap_atomic_pop (vaddr); } }
arch/mips/mm/highmem.c +10 -12
@@ -32,7 +32,7 @@ void __kunmap(struct page *page)EXPORT_SYMBOL (__kunmap);/* -* kmap_atomic/kunmap_atomic is significantly faster than kmap/kunmap because +* kmap_atomic_push/kmap_atomic_pop is significantly faster than kmap/kunmap because * no global lock is needed and because the kmap code must perform a global TLB * invalidation when the kmap pool wraps. *
@@ -40,7 +40,7 @@ EXPORT_SYMBOL(__kunmap);* kmaps are appropriate for short, tight code paths only. */ -void *__kmap_atomic (struct page *page,enum km_type type) +void *__kmap_atomic_push (struct page *page) {enum fixed_addresses idx;unsigned long vaddr;
@@ -50,8 +50,7 @@ void *__kmap_atomic(struct page *page, eif (!PageHighMem (page))return page_address (page); -debug_kmap_atomic (type); - idx = type +KM_TYPE_NR *smp_processor_id (); + idx =kmap_atomic_push_idx () +KM_TYPE_NR *smp_processor_id (); vaddr =__fix_to_virt (FIX_KMAP_BEGIN + idx);#ifdef CONFIG_DEBUG_HIGHMEM BUG_ON (!pte_none (*(kmap_pte - idx)));
@@ -61,13 +60,13 @@ void *__kmap_atomic(struct page *page, ereturn (void *) vaddr; } -EXPORT_SYMBOL (__kmap_atomic); +EXPORT_SYMBOL (__kmap_atomic_push); -void __kunmap_atomic (void *kvaddr,enum km_type type) +void __kmap_atomic_pop (void *kvaddr) {#ifdef CONFIG_DEBUG_HIGHMEM unsigned long vaddr = (unsigned long ) kvaddr &PAGE_MASK ; -enum fixed_addresses idx = type +KM_TYPE_NR *smp_processor_id (); +enum fixed_addresses idx =kmap_atomic_pop_idx () +KM_TYPE_NR *smp_processor_id ();if (vaddr <FIXADDR_START ) {// FIXME pagefault_enable ();
@@ -86,21 +85,20 @@ void __kunmap_atomic(void *kvaddr, enumpagefault_enable (); } -EXPORT_SYMBOL (__kunmap_atomic); +EXPORT_SYMBOL (__kmap_atomic_pop);/* -* This is the same as kmap_atomic() but can map memory that doesn't +* This is the same as kmap_atomic_push() but can map memory that doesn't * have a struct page associated with it. */ -void *kmap_atomic_pfn (unsigned long pfn,enum km_type type) +void *kmap_atomic_push_pfn (unsigned long pfn) {enum fixed_addresses idx;unsigned long vaddr;pagefault_disable (); -debug_kmap_atomic (type); - idx = type +KM_TYPE_NR *smp_processor_id (); + idx =kmap_atomic_push_idx () +KM_TYPE_NR *smp_processor_id (); vaddr =__fix_to_virt (FIX_KMAP_BEGIN + idx);set_pte (kmap_pte-idx,pfn_pte (pfn,PAGE_KERNEL ));flush_tlb_one (vaddr);
arch/mips/mm/init.c +4 -4
@@ -204,21 +204,21 @@ void copy_user_highpage(struct page *to, {void *vfrom, *vto; - vto =kmap_atomic (to,KM_USER1 ); + vto =kmap_atomic_push (to);if (cpu_has_dc_aliases &&page_mapped (from) && !Page_dcache_dirty (from)) { vfrom =kmap_coherent (from, vaddr);copy_page (vto, vfrom);kunmap_coherent (); }else { - vfrom =kmap_atomic (from,KM_USER0 ); + vfrom =kmap_atomic_push (from);copy_page (vto, vfrom); -kunmap_atomic (vfrom,KM_USER0 ); +kmap_atomic_pop (vfrom); }if ((!cpu_has_ic_fills_f_dc) ||pages_do_alias ((unsigned long )vto, vaddr &PAGE_MASK ))flush_data_cache_page ((unsigned long )vto); -kunmap_atomic (vto,KM_USER1 ); +kmap_atomic_pop (vto);/* Make sure this page is cleared on other CPU's too before using it */ smp_wmb (); }
arch/mn10300/include/asm/highmem.h +6 -7
@@ -65,12 +65,12 @@ static inline void kunmap(struct page *p }/* -* The use of kmap_atomic/kunmap_atomic is discouraged - kmap/kunmap -* gives a more generic (and caching) interface. But kmap_atomic can +* The use of kmap_atomic_push/kmap_atomic_pop is discouraged - kmap/kunmap +* gives a more generic (and caching) interface. But kmap_atomic_push can * be used in IRQ contexts, so in some (very limited) cases we need * it. */ -static inline unsigned long kmap_atomic (struct page *page,enum km_type type) +static inline unsigned long kmap_atomic_push (struct page *page) {enum fixed_addresses idx;unsigned long vaddr;
@@ -78,8 +78,7 @@ static inline unsigned long kmap_atomic(if (page < highmem_start_page)return page_address (page); -debug_kmap_atomic (type); - idx = type +KM_TYPE_NR *smp_processor_id (); + idx =kmap_atomic_push_idx () +KM_TYPE_NR *smp_processor_id (); vaddr =__fix_to_virt (FIX_KMAP_BEGIN + idx);#if HIGHMEM_DEBUG if (!pte_none (*(kmap_pte - idx)))
@@ -91,10 +90,10 @@ static inline unsigned long kmap_atomic(return vaddr; } -static inline void kunmap_atomic (unsigned long vaddr,enum km_type type) +static inline void kmap_atomic_pop (unsigned long vaddr) {#if HIGHMEM_DEBUG -enum fixed_addresses idx = type +KM_TYPE_NR *smp_processor_id (); +enum fixed_addresses idx =kmap_atomic_pop_idx () +KM_TYPE_NR *smp_processor_id ();if (vaddr <FIXADDR_START )/* FIXME */ return ;
arch/parisc/include/asm/cacheflush.h +3 -3
@@ -112,11 +112,11 @@ static inline void *kmap(struct page *pa#define kunmap (page) kunmap_parisc(page_address(page)) -#define kmap_atomic (page, idx) page_address(page) +#define kmap_atomic_push (page, idx) page_address(page) -#define kunmap_atomic (addr, idx) kunmap_parisc(addr) +#define kmap_atomic_pop (addr, idx) kunmap_parisc(addr) -#define kmap_atomic_pfn (pfn, idx) page_address(pfn_to_page(pfn)) +#define kmap_atomic_push_pfn (pfn, idx) page_address(pfn_to_page(pfn))#define kmap_atomic_to_page (ptr) virt_to_page(ptr)#endif
arch/powerpc/include/asm/highmem.h +4 -5
@@ -60,9 +60,8 @@ extern pte_t *pkmap_page_table;extern void *kmap_high (struct page *page);extern void kunmap_high (struct page *page); -extern void *kmap_atomic_prot (struct page *page,enum km_type type, -pgprot_t prot); -extern void kunmap_atomic (void *kvaddr,enum km_type type); +extern void *kmap_atomic_push_prot (struct page *page,pgprot_t prot); +extern void kmap_atomic_pop (void *kvaddr);static inline void *kmap (struct page *page) {
@@ -80,9 +79,9 @@ static inline void kunmap(struct page *pkunmap_high (page); } -static inline void *kmap_atomic (struct page *page,enum km_type type) +static inline void *kmap_atomic_push (struct page *page) { -return kmap_atomic_prot (page, type, kmap_prot); +return kmap_atomic_push_prot (page, kmap_prot); }static inline struct page *kmap_atomic_to_page (void *ptr)
arch/powerpc/include/asm/pgtable-ppc32.h +4 -4
@@ -308,12 +308,12 @@ static inline void __ptep_set_access_fla#define pte_offset_kernel (dir, addr) \ ((pte_t *) pmd_page_vaddr(*(dir)) + pte_index(addr))#define pte_offset_map (dir, addr) \ - ((pte_t *) kmap_atomic(pmd_page(*(dir)), KM_PTE0) + pte_index(addr)) + ((pte_t *) kmap_atomic_push(pmd_page(*(dir))) + pte_index(addr))#define pte_offset_map_nested (dir, addr) \ - ((pte_t *) kmap_atomic(pmd_page(*(dir)), KM_PTE1) + pte_index(addr)) + ((pte_t *) kmap_atomic_push(pmd_page(*(dir))) + pte_index(addr)) -#define pte_unmap (pte) kunmap_atomic(pte, KM_PTE0) -#define pte_unmap_nested (pte) kunmap_atomic(pte, KM_PTE1) +#define pte_unmap (pte) kmap_atomic_pop(pte) +#define pte_unmap_nested (pte) kmap_atomic_pop(pte)/* * Encode and decode a swap entry.
arch/powerpc/include/asm/pgtable.h +1 -1
@@ -95,7 +95,7 @@ static inline void __set_pte_at(struct m/* First case is 32-bit Hash MMU in SMP mode with 32-bit PTEs. We use the * helper pte_update() which does an atomic update. We need to do that * because a concurrent invalidation can clear _PAGE_HASHPTE. If it's a -* per-CPU PTE such as a kmap_atomic, we do a simple update preserving +* per-CPU PTE such as a kmap_atomic_push, we do a simple update preserving * the hash bits instead (ie, same as the non-SMP case) */ if (percpu)
arch/powerpc/kernel/crash_dump.c +1 -1
@@ -118,7 +118,7 @@ static size_t copy_oldmem_vaddr(void *va* otherwise @buf is in kernel address space, use memcpy(). * * Copy a page from "oldmem". For this page, there is no pte mapped -* in the current kernel. We stitch up a pte, similar to kmap_atomic. +* in the current kernel. We stitch up a pte, similar to kmap_atomic_push. */ ssize_t copy_oldmem_page (unsigned long pfn,char *buf,size_t csize,unsigned long offset,int userbuf)
arch/powerpc/mm/dma-noncoherent.c +4 -4
@@ -346,7 +346,7 @@ EXPORT_SYMBOL(__dma_sync);* __dma_sync_page() implementation for systems using highmem. * In this case, each page of a buffer must be kmapped/kunmapped * in order to have a virtual address for __dma_sync(). This must -* not sleep so kmap_atomic()/kunmap_atomic() are used. +* not sleep so kmap_atomic_push()/kmap_atomic_pop() are used. * * Note: yes, it is possible and correct to have a buffer extend * beyond the first page.
@@ -363,12 +363,12 @@ static inline void __dma_sync_page_highmlocal_irq_save (flags); do { - start = (unsigned long)kmap_atomic(page + seg_nr, - KM_PPC_SYNC_PAGE) + seg_offset; + start = (unsigned long)kmap_atomic_push(page + seg_nr) + + seg_offset; /* Sync this buffer segment */ __dma_sync((void *)start, seg_size, direction); - kunmap_atomic((void *)start, KM_PPC_SYNC_PAGE); + kmap_atomic_pop((void *)start); seg_nr++; /* Calculate next buffer segment size */
arch/powerpc/mm/highmem.c +8 -9
@@ -24,12 +24,12 @@#include <linux/module.h> /* -* The use of kmap_atomic/kunmap_atomic is discouraged - kmap/kunmap -* gives a more generic (and caching) interface. But kmap_atomic can +* The use of kmap_atomic_push/kmap_atomic_pop is discouraged - kmap/kunmap +* gives a more generic (and caching) interface. But kmap_atomic_push can * be used in IRQ contexts, so in some (very limited) cases we need * it. */ -void *kmap_atomic_prot (struct page *page,enum km_type type,pgprot_t prot) +void *kmap_atomic_push_prot (struct page *page,pgprot_t prot) {unsigned int idx;unsigned long vaddr;
@@ -39,8 +39,7 @@ void *kmap_atomic_prot(struct page *pageif (!PageHighMem (page))return page_address (page); -debug_kmap_atomic (type); - idx = type +KM_TYPE_NR *smp_processor_id (); + idx =kmap_atomic_push_idx () +KM_TYPE_NR *smp_processor_id (); vaddr =__fix_to_virt (FIX_KMAP_BEGIN + idx);#ifdef CONFIG_DEBUG_HIGHMEM BUG_ON (!pte_none (*(kmap_pte-idx)));
@@ -50,13 +49,13 @@ void *kmap_atomic_prot(struct page *pagereturn (void *) vaddr; } -EXPORT_SYMBOL (kmap_atomic_prot); +EXPORT_SYMBOL (kmap_atomic_push_prot); -void kunmap_atomic (void *kvaddr,enum km_type type) +void kmap_atomic_pop (void *kvaddr) {#ifdef CONFIG_DEBUG_HIGHMEM unsigned long vaddr = (unsigned long ) kvaddr &PAGE_MASK ; -enum fixed_addresses idx = type +KM_TYPE_NR *smp_processor_id (); +enum fixed_addresses idx =kmap_atomic_pop_idx () +KM_TYPE_NR *smp_processor_id ();if (vaddr <__fix_to_virt (FIX_KMAP_END )) {pagefault_enable ();
@@ -74,4 +73,4 @@ void kunmap_atomic(void *kvaddr, enum km#endif pagefault_enable (); } -EXPORT_SYMBOL (kunmap_atomic); +EXPORT_SYMBOL (kmap_atomic_pop);
arch/powerpc/mm/mem.c +2 -2
@@ -418,9 +418,9 @@ EXPORT_SYMBOL(flush_dcache_page);void flush_dcache_icache_page (struct page *page) {#ifdef CONFIG_BOOKE -void *start =kmap_atomic (page,KM_PPC_SYNC_ICACHE ); +void *start =kmap_atomic_push (page);__flush_dcache_icache (start); -kunmap_atomic (start,KM_PPC_SYNC_ICACHE ); +kmap_atomic_pop (start);#elif defined(CONFIG_8xx) || defined(CONFIG_PPC64 )/* On 8xx there is no need to kmap since highmem is not supported */ __flush_dcache_icache (page_address (page));
arch/powerpc/sysdev/ppc4xx_pci.c +1 -1
@@ -1613,7 +1613,7 @@ static void __init ppc4xx_pciex_port_set/* Because of how big mapping the config space is (1M per bus), we * limit how many busses we support. In the long run, we could replace -* that with something akin to kmap_atomic instead. We set aside 1 bus +* that with something akin to kmap_atomic_push instead. We set aside 1 bus * for the host itself too. */ busses = hose->last_busno - hose->first_busno;/* This is off by 1 */
arch/sh/kernel/crash_dump.c +1 -1
@@ -24,7 +24,7 @@ unsigned long long elfcorehdr_addr = ELF* otherwise @buf is in kernel address space, use memcpy(). * * Copy a page from "oldmem". For this page, there is no pte mapped -* in the current kernel. We stitch up a pte, similar to kmap_atomic. +* in the current kernel. We stitch up a pte, similar to kmap_atomic_push. */ ssize_t copy_oldmem_page (unsigned long pfn,char *buf,size_t csize,unsigned long offset,int userbuf)
arch/sh/mm/cache.c +6 -6
@@ -83,7 +83,7 @@ void copy_user_highpage(struct page *to, {void *vfrom, *vto; - vto =kmap_atomic (to,KM_USER1 ); + vto =kmap_atomic_push (to);if (boot_cpu_data.dcache.n_aliases &&page_mapped (from) && !test_bit (PG_dcache_dirty, &from->flags)) {
@@ -91,15 +91,15 @@ void copy_user_highpage(struct page *to,copy_page (vto, vfrom);kunmap_coherent (vfrom); }else { - vfrom =kmap_atomic (from,KM_USER0 ); + vfrom =kmap_atomic_push (from);copy_page (vto, vfrom); -kunmap_atomic (vfrom,KM_USER0 ); +kmap_atomic_pop (vfrom); }if (pages_do_alias ((unsigned long )vto, vaddr &PAGE_MASK ))__flush_purge_region (vto,PAGE_SIZE ); -kunmap_atomic (vto,KM_USER1 ); +kmap_atomic_pop (vto);/* Make sure this page is cleared on other CPU's too before using it */ smp_wmb (); }
@@ -107,14 +107,14 @@ EXPORT_SYMBOL(copy_user_highpage);void clear_user_highpage (struct page *page,unsigned long vaddr) { -void *kaddr =kmap_atomic (page,KM_USER0 ); +void *kaddr =kmap_atomic_push (page);clear_page (kaddr);if (pages_do_alias ((unsigned long )kaddr, vaddr &PAGE_MASK ))__flush_purge_region (kaddr,PAGE_SIZE ); -kunmap_atomic (kaddr,KM_USER0 ); +kmap_atomic_pop (kaddr); }EXPORT_SYMBOL (clear_user_highpage);
arch/sparc/include/asm/highmem.h +2 -2
@@ -70,8 +70,8 @@ static inline void kunmap(struct page *pkunmap_high (page); } -extern void *kmap_atomic (struct page *page,enum km_type type); -extern void kunmap_atomic (void *kvaddr,enum km_type type); +extern void *kmap_atomic_push (struct page *page); +extern void kmap_atomic_pop (void *kvaddr);extern struct page *kmap_atomic_to_page (void *vaddr);#define flush_cache_kmaps () flush_cache_all()
arch/sparc/mm/highmem.c +10 -11
@@ -3,17 +3,17 @@* * Provides kernel-static versions of atomic kmap functions originally * found as inlines in include/asm-sparc/highmem.h. These became -* needed as kmap_atomic() and kunmap_atomic() started getting +* needed as kmap_atomic_push() and kmap_atomic_pop() started getting * called from within modules. * -- Tomas Szepe <[email protected]>, September 2002 * -* But kmap_atomic() and kunmap_atomic() cannot be inlined in +* But kmap_atomic_push() and kmap_atomic_pop() cannot be inlined in * modules because they are loaded with btfixup-ped functions. */ /* -* The use of kmap_atomic/kunmap_atomic is discouraged - kmap/kunmap -* gives a more generic (and caching) interface. But kmap_atomic can +* The use of kmap_atomic_push/kmap_atomic_pop is discouraged - kmap/kunmap +* gives a more generic (and caching) interface. But kmap_atomic_push can * be used in IRQ contexts, so in some (very limited) cases we need it. * * XXX This is an old text. Actually, it's good to use atomic kmaps,
@@ -29,7 +29,7 @@#include <asm/tlbflush.h> #include <asm/fixmap.h> -void *kmap_atomic (struct page *page,enum km_type type) +void *kmap_atomic_push (struct page *page) {unsigned long idx;unsigned long vaddr;
@@ -39,8 +39,7 @@ void *kmap_atomic(struct page *page, enuif (!PageHighMem (page))return page_address (page); -debug_kmap_atomic (type); - idx = type +KM_TYPE_NR *smp_processor_id (); + idx =kmap_atomic_push_idx () +KM_TYPE_NR *smp_processor_id (); vaddr =__fix_to_virt (FIX_KMAP_BEGIN + idx);/* XXX Fix - Anton */
@@ -63,13 +62,13 @@ void *kmap_atomic(struct page *page, enureturn (void *) vaddr; } -EXPORT_SYMBOL (kmap_atomic); +EXPORT_SYMBOL (kmap_atomic_push); -void kunmap_atomic (void *kvaddr,enum km_type type) +void kmap_atomic_pop (void *kvaddr) {#ifdef CONFIG_DEBUG_HIGHMEM unsigned long vaddr = (unsigned long ) kvaddr &PAGE_MASK ; -unsigned long idx = type +KM_TYPE_NR *smp_processor_id (); +unsigned long idx =kmap_atomic_pop_idx () +KM_TYPE_NR *smp_processor_id ();if (vaddr <FIXADDR_START ) {// FIXME pagefault_enable ();
@@ -100,7 +99,7 @@ void kunmap_atomic(void *kvaddr, enum kmpagefault_enable (); } -EXPORT_SYMBOL (kunmap_atomic); +EXPORT_SYMBOL (kmap_atomic_pop);/* We may be fed a pagetable here by ptep_to_xxx and others. */ struct page *kmap_atomic_to_page (void *ptr)
arch/sparc/mm/io-unit.c +1 -1
@@ -9,7 +9,7 @@#include <linux/slab.h> #include <linux/spinlock.h> #include <linux/mm.h> -#include <linux/highmem.h> /* pte_offset_map => kmap_atomic */ +#include <linux/highmem.h> /* pte_offset_map => kmap_atomic_push */ #include <linux/bitops.h> #include <linux/scatterlist.h> #include <linux/of.h>
arch/sparc/mm/iommu.c +1 -1
@@ -11,7 +11,7 @@#include <linux/init.h> #include <linux/mm.h> #include <linux/slab.h> -#include <linux/highmem.h> /* pte_offset_map => kmap_atomic */ +#include <linux/highmem.h> /* pte_offset_map => kmap_atomic_push */ #include <linux/scatterlist.h> #include <linux/of.h> #include <linux/of_device.h>
arch/um/kernel/skas/uaccess.c +2 -2
@@ -68,7 +68,7 @@ static int do_op_one_page(unsigned longreturn -1 ; page =pte_page (*pte); - addr = (unsigned long )kmap_atomic (page,KM_UML_USERCOPY ) + + addr = (unsigned long )kmap_atomic_push (page) + (addr & ~PAGE_MASK ); current->thread.fault_catcher = &buf;
@@ -81,7 +81,7 @@ static int do_op_one_page(unsigned long current->thread.fault_catcher =NULL ; -kunmap_atomic (page,KM_UML_USERCOPY ); +kmap_atomic_pop (page);return n; }
arch/x86/include/asm/highmem.h +6 -6
@@ -59,15 +59,15 @@ extern void kunmap_high(struct page *pagvoid *kmap (struct page *page);void kunmap (struct page *page); -void *kmap_atomic_prot (struct page *page,enum km_type type,pgprot_t prot); -void *kmap_atomic (struct page *page,enum km_type type); -void kunmap_atomic (void *kvaddr,enum km_type type); -void *kmap_atomic_pfn (unsigned long pfn,enum km_type type); -void *kmap_atomic_prot_pfn (unsigned long pfn,enum km_type type,pgprot_t prot); +void *kmap_atomic_push_prot (struct page *page,pgprot_t prot); +void *kmap_atomic_push (struct page *page); +void kmap_atomic_pop (void *kvaddr); +void *kmap_atomic_push_pfn (unsigned long pfn); +void *kmap_atomic_push_prot_pfn (unsigned long pfn,pgprot_t prot);struct page *kmap_atomic_to_page (void *ptr);#ifndef CONFIG_PARAVIRT -#define kmap_atomic_pte (page, type) kmap_atomic(page, type) +#define kmap_atomic_push_pte (page) kmap_atomic_push(page)#endif #define flush_cache_kmaps () do { } while (0)
arch/x86/include/asm/iomap.h +4 -11
@@ -26,16 +26,9 @@#include <asm/pgtable.h> #include <asm/tlbflush.h> -void * -iomap_atomic_prot_pfn (unsigned long pfn,enum km_type type,pgprot_t prot); - -void -iounmap_atomic (void *kvaddr,enum km_type type); - -int -iomap_create_wc (resource_size_t base,unsigned long size,pgprot_t *prot); - -void -iomap_free (resource_size_t base,unsigned long size); +void *iomap_atomic_push_prot_pfn (unsigned long pfn,pgprot_t prot); +void iomap_atomic_pop (void *kvaddr); +int iomap_create_wc (resource_size_t base,unsigned long size,pgprot_t *prot); +void iomap_free (resource_size_t base,unsigned long size);#endif /* _ASM_X86_IOMAP_H */
arch/x86/include/asm/paravirt.h +2 -2
@@ -436,10 +436,10 @@ static inline void paravirt_release_pud( }#ifdef CONFIG_HIGHPTE -static inline void *kmap_atomic_pte (struct page *page,enum km_type type) +static inline void *kmap_atomic_push_pte (struct page *page,enum km_type type) {unsigned long ret; - ret = PVOP_CALL2(unsigned long, pv_mmu_ops.kmap_atomic_pte, page, type); + ret = PVOP_CALL2(unsigned long, pv_mmu_ops.kmap_atomic_push_pte, page, type);return (void *)ret; }#endif
arch/x86/include/asm/paravirt_types.h +1 -1
@@ -305,7 +305,7 @@ struct pv_mmu_ops {#endif /* PAGETABLE_LEVELS >= 3 */ #ifdef CONFIG_HIGHPTE -void *(*kmap_atomic_pte)(struct page *page,enum km_type type); +void *(*kmap_atomic_push_pte)(struct page *page,enum km_type type);#endif struct pv_lazy_ops lazy_mode;
arch/x86/include/asm/pgtable_32.h +4 -8
@@ -49,18 +49,14 @@ extern void set_pmd_pfn(unsigned long, u#endif #if defined(CONFIG_HIGHPTE ) -#define __KM_PTE \ - (in_nmi() ? KM_NMI_PTE : \ - in_irq() ? KM_IRQ_PTE : \ - KM_PTE0)#define pte_offset_map (dir, address) \ - ((pte_t *)kmap_atomic_pte(pmd_page(*(dir)), __KM_PTE) + \ + ((pte_t *)kmap_atomic_push_pte(pmd_page(*(dir))) + \ pte_index((address)))#define pte_offset_map_nested (dir, address) \ - ((pte_t *)kmap_atomic_pte(pmd_page(*(dir)), KM_PTE1) + \ + ((pte_t *)kmap_atomic_push_pte(pmd_page(*(dir))) + \ pte_index((address))) -#define pte_unmap (pte) kunmap_atomic((pte), __KM_PTE) -#define pte_unmap_nested (pte) kunmap_atomic((pte), KM_PTE1) +#define pte_unmap (pte) kmap_atomic_pop((pte)) +#define pte_unmap_nested (pte) kmap_atomic_pop((pte))#else #define pte_offset_map (dir, address) \ ((pte_t *)page_address(pmd_page(*(dir))) + pte_index((address)))
arch/x86/kernel/cpu/perf_event.c +2 -3
@@ -2190,7 +2190,6 @@ static unsigned long copy_from_user_nmi(void *to, const void __user *from, unsigned long n) {unsigned long offset, addr = (unsigned long )from; -int type =in_nmi () ?KM_NMI :KM_IRQ0 ;unsigned long size, len =0 ;struct page *page;void *map;
@@ -2204,9 +2203,9 @@ copy_from_user_nmi(void *to, const void offset = addr & (PAGE_SIZE -1 ); size =min (PAGE_SIZE - offset, n - len); - map =kmap_atomic (page, type); + map =kmap_atomic_push (page);memcpy (to, map+offset, size); -kunmap_atomic (map, type); +kmap_atomic_pop (map);put_page (page); len += size;
arch/x86/kernel/crash_dump_32.c +5 -5
@@ -27,7 +27,7 @@ unsigned long long elfcorehdr_addr = ELF* otherwise @buf is in kernel address space, use memcpy(). * * Copy a page from "oldmem". For this page, there is no pte mapped -* in the current kernel. We stitch up a pte, similar to kmap_atomic. +* in the current kernel. We stitch up a pte, similar to kmap_atomic_push. * * Calling copy_to_user() in atomic context is not desirable. Hence first * copying the data to a pre-allocated kernel page and then copying to user
@@ -41,20 +41,20 @@ ssize_t copy_oldmem_page(unsigned long pif (!csize)return 0 ; - vaddr =kmap_atomic_pfn (pfn,KM_PTE0 ); + vaddr =kmap_atomic_push_pfn (pfn);if (!userbuf) {memcpy (buf, (vaddr + offset), csize); -kunmap_atomic (vaddr,KM_PTE0 ); +kmap_atomic_pop (vaddr); }else {if (!kdump_buf_page) {printk (KERN_WARNING "Kdump: Kdump buffer page not" " allocated\n" ); -kunmap_atomic (vaddr,KM_PTE0 ); +kmap_atomic_pop (vaddr);return -EFAULT ; }copy_page (kdump_buf_page, vaddr); -kunmap_atomic (vaddr,KM_PTE0 ); +kmap_atomic_pop (vaddr);if (copy_to_user (buf, (kdump_buf_page + offset), csize))return -EFAULT ; }
arch/x86/kernel/crash_dump_64.c +1 -1
@@ -24,7 +24,7 @@ unsigned long long elfcorehdr_addr = ELF* otherwise @buf is in kernel address space, use memcpy(). * * Copy a page from "oldmem". For this page, there is no pte mapped -* in the current kernel. We stitch up a pte, similar to kmap_atomic. +* in the current kernel. We stitch up a pte, similar to kmap_atomic_push. */ ssize_t copy_oldmem_page (unsigned long pfn,char *buf,size_t csize,unsigned long offset,int userbuf)
arch/x86/kernel/paravirt.c +1 -1
@@ -429,7 +429,7 @@ struct pv_mmu_ops pv_mmu_ops = { .ptep_modify_prot_commit = __ptep_modify_prot_commit,#ifdef CONFIG_HIGHPTE - .kmap_atomic_pte = kmap_atomic, + .kmap_atomic_push_pte = kmap_atomic_push,#endif #if PAGETABLE_LEVELS >=3
arch/x86/kernel/vmi_32.c +3 -3
@@ -267,9 +267,9 @@ static void vmi_nop(void) }#ifdef CONFIG_HIGHPTE -static void *vmi_kmap_atomic_pte (struct page *page,enum km_type type) +static void *vmi_kmap_atomic_push_pte (struct page *page,enum km_type type) { -void *va =kmap_atomic (page, type); +void *va =kmap_atomic_push (page, type);/* * Internally, the VMI ROM must map virtual addresses to physical
@@ -780,7 +780,7 @@ static inline int __init activate_vmi(vo vmi_ops.set_linear_mapping =vmi_get_function (VMI_CALL_SetLinearMapping);#ifdef CONFIG_HIGHPTE if (vmi_ops.set_linear_mapping) - pv_mmu_ops.kmap_atomic_pte = vmi_kmap_atomic_pte; + pv_mmu_ops.kmap_atomic_push_pte = vmi_kmap_atomic_push_pte;#endif /*
arch/x86/kvm/lapic.c +4 -4
@@ -1179,9 +1179,9 @@ void kvm_lapic_sync_from_vapic(struct kvif (!irqchip_in_kernel (vcpu->kvm) || !vcpu->arch.apic->vapic_addr)return ; - vapic =kmap_atomic (vcpu->arch.apic->vapic_page,KM_USER0 ); + vapic =kmap_atomic_push (vcpu->arch.apic->vapic_page); data = *(u32 *)(vapic +offset_in_page (vcpu->arch.apic->vapic_addr)); -kunmap_atomic (vapic,KM_USER0 ); +kmap_atomic_pop (vapic);apic_set_tpr (vcpu->arch.apic, data &0xff ); }
@@ -1206,9 +1206,9 @@ void kvm_lapic_sync_to_vapic(struct kvm_ max_isr =0 ; data = (tpr &0xff ) | ((max_isr &0xf0 ) <<8 ) | (max_irr <<24 ); - vapic =kmap_atomic (vcpu->arch.apic->vapic_page,KM_USER0 ); + vapic =kmap_atomic_push (vcpu->arch.apic->vapic_page); *(u32 *)(vapic +offset_in_page (vcpu->arch.apic->vapic_addr)) = data; -kunmap_atomic (vapic,KM_USER0 ); +kmap_atomic_pop (vapic); }void kvm_lapic_set_vapic_addr (struct kvm_vcpu *vcpu,gpa_t vapic_addr)
arch/x86/kvm/paging_tmpl.h +2 -2
@@ -88,9 +88,9 @@ static bool FNAME(cmpxchg_gpte)(struct k page =gfn_to_page (kvm, table_gfn); - table =kmap_atomic (page,KM_USER0 ); + table =kmap_atomic_push (page); ret =CMPXCHG (&table[index], orig_pte, new_pte); -kunmap_atomic (table,KM_USER0 ); +kmap_atomic_pop (table);kvm_release_page_dirty (page);
arch/x86/kvm/svm.c +14 -14
@@ -1397,7 +1397,7 @@ static void *nested_svm_map(struct vcpu_if (is_error_page (page)) goto error; -return kmap_atomic (page, idx); +return kmap_atomic_push (page, idx); error:kvm_release_page_clean (page);
@@ -1415,7 +1415,7 @@ static void nested_svm_unmap(void *addr, page =kmap_atomic_to_page (addr); -kunmap_atomic (addr, idx); +kmap_atomic_pop (addr, idx);kvm_release_page_dirty (page); }
@@ -1430,7 +1430,7 @@ static bool nested_svm_exit_handled_msr(if (!(svm->nested.intercept & (1ULL <<INTERCEPT_MSR_PROT )))return false; - msrpm =nested_svm_map (svm, svm->nested.vmcb_msrpm,KM_USER0 ); + msrpm =nested_svm_map (svm, svm->nested.vmcb_msrpm);if (!msrpm) goto out;
@@ -1458,7 +1458,7 @@ static bool nested_svm_exit_handled_msr( ret = msrpm[t1] & ((1 << param) << t0); out: -nested_svm_unmap (msrpm,KM_USER0 ); +nested_svm_unmap (msrpm);return ret; }
@@ -1584,7 +1584,7 @@ static int nested_svm_vmexit(struct vcpustruct vmcb *hsave = svm->nested.hsave;struct vmcb *vmcb = svm->vmcb; - nested_vmcb =nested_svm_map (svm, svm->nested.vmcb,KM_USER0 ); + nested_vmcb =nested_svm_map (svm, svm->nested.vmcb);if (!nested_vmcb)return 1 ;
@@ -1662,7 +1662,7 @@ static int nested_svm_vmexit(struct vcpu/* Exit nested SVM mode */ svm->nested.vmcb =0 ; -nested_svm_unmap (nested_vmcb,KM_USER0 ); +nested_svm_unmap (nested_vmcb);kvm_mmu_reset_context (&svm->vcpu);kvm_mmu_load (&svm->vcpu);
@@ -1675,7 +1675,7 @@ static bool nested_svm_vmrun_msrpm(strucu32 *nested_msrpm;int i; - nested_msrpm =nested_svm_map (svm, svm->nested.vmcb_msrpm,KM_USER0 ); + nested_msrpm =nested_svm_map (svm, svm->nested.vmcb_msrpm);if (!nested_msrpm)return false;
@@ -1684,7 +1684,7 @@ static bool nested_svm_vmrun_msrpm(struc svm->vmcb->control.msrpm_base_pa =__pa (svm->nested.msrpm); -nested_svm_unmap (nested_msrpm,KM_USER0 ); +nested_svm_unmap (nested_msrpm);return true; }
@@ -1695,7 +1695,7 @@ static bool nested_svm_vmrun(struct vcpustruct vmcb *hsave = svm->nested.hsave;struct vmcb *vmcb = svm->vmcb; - nested_vmcb =nested_svm_map (svm, svm->vmcb->save.rax,KM_USER0 ); + nested_vmcb =nested_svm_map (svm, svm->vmcb->save.rax);if (!nested_vmcb)return false;
@@ -1814,7 +1814,7 @@ static bool nested_svm_vmrun(struct vcpu svm->vmcb->control.event_inj = nested_vmcb->control.event_inj; svm->vmcb->control.event_inj_err = nested_vmcb->control.event_inj_err; -nested_svm_unmap (nested_vmcb,KM_USER0 ); +nested_svm_unmap (nested_vmcb);enable_gif (svm);
@@ -1847,12 +1847,12 @@ static int vmload_interception(struct vc svm->next_rip =kvm_rip_read (&svm->vcpu) +3 ;skip_emulated_instruction (&svm->vcpu); - nested_vmcb =nested_svm_map (svm, svm->vmcb->save.rax,KM_USER0 ); + nested_vmcb =nested_svm_map (svm, svm->vmcb->save.rax);if (!nested_vmcb)return 1 ;nested_svm_vmloadsave (nested_vmcb, svm->vmcb); -nested_svm_unmap (nested_vmcb,KM_USER0 ); +nested_svm_unmap (nested_vmcb);return 1 ; }
@@ -1867,12 +1867,12 @@ static int vmsave_interception(struct vc svm->next_rip =kvm_rip_read (&svm->vcpu) +3 ;skip_emulated_instruction (&svm->vcpu); - nested_vmcb =nested_svm_map (svm, svm->vmcb->save.rax,KM_USER0 ); + nested_vmcb =nested_svm_map (svm, svm->vmcb->save.rax);if (!nested_vmcb)return 1 ;nested_svm_vmloadsave (svm->vmcb, nested_vmcb); -nested_svm_unmap (nested_vmcb,KM_USER0 ); +nested_svm_unmap (nested_vmcb);return 1 ; }
arch/x86/kvm/x86.c +4 -4
@@ -685,12 +685,12 @@ static void kvm_write_guest_time(struct*/ vcpu->hv_clock.version +=2 ; - shared_kaddr =kmap_atomic (vcpu->time_page,KM_USER0 ); + shared_kaddr =kmap_atomic_push (vcpu->time_page);memcpy (shared_kaddr + vcpu->time_offset, &vcpu->hv_clock,sizeof (vcpu->hv_clock)); -kunmap_atomic (shared_kaddr,KM_USER0 ); +kmap_atomic_pop (shared_kaddr);mark_page_dirty (v->kvm, vcpu->time >>PAGE_SHIFT ); }
@@ -2668,9 +2668,9 @@ static int emulator_cmpxchg_emulated(uns page =gfn_to_page (vcpu->kvm, gpa >>PAGE_SHIFT ); - kaddr =kmap_atomic (page,KM_USER0 ); + kaddr =kmap_atomic_push (page);set_64bit ((u64 *)(kaddr +offset_in_page (gpa)), val); -kunmap_atomic (kaddr,KM_USER0 ); +kmap_atomic_pop (kaddr);kvm_release_page_dirty (page); } emul_write:
arch/x86/lib/usercopy_32.c +2 -2
@@ -760,9 +760,9 @@ survive:break ; } - maddr =kmap_atomic (pg,KM_USER0 ); + maddr =kmap_atomic_push (pg);memcpy (maddr + offset, from, len); -kunmap_atomic (maddr,KM_USER0 ); +kmap_atomic_pop (maddr);set_page_dirty_lock (pg);put_page (pg);up_read (¤t->mm->mmap_sem);
arch/x86/mm/highmem_32.c +29 -16
91 lines not on this page · raw message
arch/x86/mm/iomap_32.c +14 -44
70 lines not on this page · raw message
arch/x86/xen/mmu.c +3 -3
@@ -1428,7 +1428,7 @@ static void xen_pgd_free(struct mm_struc }#ifdef CONFIG_HIGHPTE -static void *xen_kmap_atomic_pte (struct page *page,enum km_type type) +static void *xen_kmap_atomic_push_pte (struct page *page,enum km_type type) {pgprot_t prot =PAGE_KERNEL ;
@@ -1440,7 +1440,7 @@ static void *xen_kmap_atomic_pte(structpage_to_pfn (page), type, (unsigned long)pgprot_val(prot) & _PAGE_RW ? "WRITE" : "READ"); - return kmap_atomic_prot(page, type, prot); + return kmap_atomic_push_prot(page, type, prot); }#endif
@@ -1903,7 +1903,7 @@ static const struct pv_mmu_ops xen_mmu_o .release_pmd = xen_release_pmd_init,#ifdef CONFIG_HIGHPTE - .kmap_atomic_pte = xen_kmap_atomic_pte, + .kmap_atomic_push_pte = xen_kmap_atomic_push_pte,#endif #ifdef CONFIG_X86_64
block/blk-settings.c +1 -1
@@ -125,7 +125,7 @@ EXPORT_SYMBOL(blk_set_default_limits);* Caveat: * The driver that does this *must* be able to deal appropriately * with buffers in "highmemory". This can be accomplished by either calling -* __bio_kmap_atomic() to get a temporary kernel mapping, or by calling +* __bio_kmap_atomic_push() to get a temporary kernel mapping, or by calling * blk_queue_bounce() to create a buffer in normal memory. **/ void blk_queue_make_request (struct request_queue *q,make_request_fn *mfn)
crypto/ahash.c +2 -2
@@ -44,7 +44,7 @@ static int hash_walk_next(struct crypto_unsigned int nbytes =min (walk->entrylen, ((unsigned int )(PAGE_SIZE )) - offset); - walk->data =crypto_kmap (walk->pg,0 ); + walk->data =kmap_atomic_push (walk->pg); walk->data += offset;if (offset & alignmask)
@@ -89,7 +89,7 @@ int crypto_hash_walk_done(struct crypto_return nbytes; } -crypto_kunmap (walk->data,0 ); +kmap_atomic_pop (walk->data);crypto_yield (walk->flags);if (err)
crypto/async_tx/async_memcpy.c +4 -4
@@ -78,13 +78,13 @@ async_memcpy(struct page *dest, struct p/* wait for any prerequisite operations */ async_tx_quiesce (&submit->depend_tx); - dest_buf =kmap_atomic (dest,KM_USER0 ) + dest_offset; - src_buf =kmap_atomic (src,KM_USER1 ) + src_offset; + dest_buf =kmap_atomic_push (dest) + dest_offset; + src_buf =kmap_atomic_push (src) + src_offset;memcpy (dest_buf, src_buf, len); -kunmap_atomic (dest_buf,KM_USER0 ); -kunmap_atomic (src_buf,KM_USER1 ); +kmap_atomic_pop (src_buf); +kmap_atomic_pop (dest_buf);async_tx_sync_epilog (submit); }
crypto/blkcipher.c +5 -5
@@ -41,22 +41,22 @@ static int blkcipher_walk_first(struct bstatic inline void blkcipher_map_src (struct blkcipher_walk *walk) { - walk->src.virt.addr =scatterwalk_map (&walk->in,0 ); + walk->src.virt.addr =scatterwalk_map (&walk->in); }static inline void blkcipher_map_dst (struct blkcipher_walk *walk) { - walk->dst.virt.addr =scatterwalk_map (&walk->out,1 ); + walk->dst.virt.addr =scatterwalk_map (&walk->out); }static inline void blkcipher_unmap_src (struct blkcipher_walk *walk) { -scatterwalk_unmap (walk->src.virt.addr,0 ); +scatterwalk_unmap (walk->src.virt.addr); }static inline void blkcipher_unmap_dst (struct blkcipher_walk *walk) { -scatterwalk_unmap (walk->dst.virt.addr,1 ); +scatterwalk_unmap (walk->dst.virt.addr); }/* Get a spot of the specified length that does not straddle a page.
@@ -89,9 +89,9 @@ static inline unsigned int blkcipher_donmemcpy (walk->dst.virt.addr, walk->page, n);blkcipher_unmap_dst (walk); }else if (!(walk->flags &BLKCIPHER_WALK_PHYS )) { -blkcipher_unmap_src (walk);if (walk->flags &BLKCIPHER_WALK_DIFF )blkcipher_unmap_dst (walk); +blkcipher_unmap_src (walk); }scatterwalk_advance (&walk->in, n);
crypto/ccm.c +2 -2
@@ -216,12 +216,12 @@ static void get_data_to_compute(struct cscatterwalk_start (&walk,sg_next (walk.sg)); n =scatterwalk_clamp (&walk, len); } - data_src =scatterwalk_map (&walk,0 ); + data_src =scatterwalk_map (&walk);compute_mac (tfm, data_src, n, pctx); len -= n; -scatterwalk_unmap (data_src,0 ); +scatterwalk_unmap (data_src);scatterwalk_advance (&walk, n);scatterwalk_done (&walk,0 , len);if (len)
crypto/digest.c +2 -2
@@ -54,7 +54,7 @@ static int update2(struct hash_desc *desunsigned int bytes_from_page =min (l, ((unsigned int ) (PAGE_SIZE )) - offset); -char *src =crypto_kmap (pg,0 ); +char *src =kmap_atomic_push (pg);char *p = src + offset;if (unlikely (offset & alignmask)) {
@@ -69,7 +69,7 @@ static int update2(struct hash_desc *des } tfm->__crt_alg->cra_digest.dia_update (tfm, p, bytes_from_page); -crypto_kunmap (src,0 ); +kmap_atomic_pop (src);crypto_yield (desc->flags); offset =0 ; pg++;
crypto/scatterwalk.c +4 -4
@@ -40,9 +40,9 @@ void scatterwalk_start(struct scatter_wa }EXPORT_SYMBOL_GPL (scatterwalk_start); -void *scatterwalk_map (struct scatter_walk *walk,int out) +void *scatterwalk_map (struct scatter_walk *walk) { -return crypto_kmap (scatterwalk_page (walk), out) + +return kmap_atomic_push (scatterwalk_page (walk)) +offset_in_page (walk->offset); }EXPORT_SYMBOL_GPL (scatterwalk_map);
@@ -83,9 +83,9 @@ void scatterwalk_copychunks(void *buf, sif (len_this_page > nbytes) len_this_page = nbytes; - vaddr =scatterwalk_map (walk, out); + vaddr =scatterwalk_map (walk);memcpy_dir (buf, vaddr, len_this_page, out); -scatterwalk_unmap (vaddr, out); +scatterwalk_unmap (vaddr);scatterwalk_advance (walk, len_this_page);
crypto/shash.c +4 -4
@@ -279,10 +279,10 @@ int shash_ahash_digest(struct ahash_requif (nbytes <min (sg->length, ((unsigned int )(PAGE_SIZE )) - offset)) {void *data; - data =crypto_kmap (sg_page (sg),0 ); + data =kmap_atomic_push (sg_page (sg)); err =crypto_shash_digest (desc, data + offset, nbytes, req->result); -crypto_kunmap (data,0 ); +kmap_atomic_pop (data);crypto_yield (desc->flags); }else err = crypto_shash_init(desc) ?:
@@ -412,9 +412,9 @@ static int shash_compat_digest(struct ha desc->flags = hdesc->flags; - data =crypto_kmap (sg_page (sg),0 ); + data =kmap_atomic_push (sg_page (sg)); err =crypto_shash_digest (desc, data + offset, nbytes, out); -crypto_kunmap (data,0 ); +kmap_atomic_pop (data);crypto_yield (desc->flags); goto out; }
drivers/ata/libata-sff.c +4 -4
@@ -879,13 +879,13 @@ static void ata_pio_sector(struct ata_qu/* FIXME: use a bounce buffer */ local_irq_save (flags); - buf =kmap_atomic (page,KM_IRQ0 ); + buf =kmap_atomic_push (page);/* do the actual data transfer */ ap->ops->sff_data_xfer (qc->dev, buf + offset, qc->sect_size, do_write); -kunmap_atomic (buf,KM_IRQ0 ); +kmap_atomic_pop (buf);local_irq_restore (flags); }else { buf =page_address (page);
@@ -1017,13 +1017,13 @@ next_sg:/* FIXME: use bounce buffer */ local_irq_save (flags); - buf =kmap_atomic (page,KM_IRQ0 ); + buf =kmap_atomic_push (page);/* do the actual data transfer */ consumed = ap->ops->sff_data_xfer (dev, buf + offset, count, rw); -kunmap_atomic (buf,KM_IRQ0 ); +kmap_atomic_pop (buf);local_irq_restore (flags); }else { buf =page_address (page);
drivers/block/brd.c +10 -10
@@ -204,9 +204,9 @@ static void copy_to_brd(struct brd_devic page =brd_lookup_page (brd, sector);BUG_ON (!page); - dst =kmap_atomic (page,KM_USER1 ); + dst =kmap_atomic_push (page);memcpy (dst + offset, src, copy); -kunmap_atomic (dst,KM_USER1 ); +kmap_atomic_pop (dst);if (copy < n) { src += copy;
@@ -215,9 +215,9 @@ static void copy_to_brd(struct brd_devic page =brd_lookup_page (brd, sector);BUG_ON (!page); - dst =kmap_atomic (page,KM_USER1 ); + dst =kmap_atomic_push (page);memcpy (dst, src, copy); -kunmap_atomic (dst,KM_USER1 ); +kmap_atomic_pop (dst); } }
@@ -235,9 +235,9 @@ static void copy_from_brd(void *dst, str copy =min_t (size_t, n,PAGE_SIZE - offset); page =brd_lookup_page (brd, sector);if (page) { - src =kmap_atomic (page,KM_USER1 ); + src =kmap_atomic_push (page);memcpy (dst, src + offset, copy); -kunmap_atomic (src,KM_USER1 ); +kmap_atomic_pop (src); }else memset (dst,0 , copy);
@@ -247,9 +247,9 @@ static void copy_from_brd(void *dst, str copy = n - copy; page =brd_lookup_page (brd, sector);if (page) { - src =kmap_atomic (page,KM_USER1 ); + src =kmap_atomic_push (page);memcpy (dst, src, copy); -kunmap_atomic (src,KM_USER1 ); +kmap_atomic_pop (src); }else memset (dst,0 , copy); }
@@ -271,7 +271,7 @@ static int brd_do_bvec(struct brd_device goto out; } - mem =kmap_atomic (page,KM_USER0 ); + mem =kmap_atomic_push (page);if (rw ==READ ) {copy_from_brd (mem + off, brd, sector, len);flush_dcache_page (page);
@@ -279,7 +279,7 @@ static int brd_do_bvec(struct brd_deviceflush_dcache_page (page);copy_to_brd (brd, mem + off, sector, len); } -kunmap_atomic (mem,KM_USER0 ); +kmap_atomic_pop (mem); out:return err;
drivers/block/loop.c +8 -8
@@ -91,16 +91,16 @@ static int transfer_none(struct loop_dev struct page *loop_page, unsigned loop_off, int size, sector_t real_block) { - char *raw_buf = kmap_atomic(raw_page, KM_USER0) + raw_off; -char *loop_buf =kmap_atomic (loop_page,KM_USER1 ) + loop_off; + char *raw_buf = kmap_atomic_push(raw_page) + raw_off; +char *loop_buf =kmap_atomic_push (loop_page) + loop_off;if (cmd ==READ )memcpy (loop_buf, raw_buf, size);else memcpy (raw_buf, loop_buf, size); -kunmap_atomic (raw_buf,KM_USER0 ); -kunmap_atomic (loop_buf,KM_USER1 ); +kmap_atomic_pop (loop_buf); +kmap_atomic_pop (raw_buf);cond_resched ();return 0 ; }
@@ -110,8 +110,8 @@ static int transfer_xor(struct loop_devi struct page *loop_page, unsigned loop_off, int size, sector_t real_block) { - char *raw_buf = kmap_atomic(raw_page, KM_USER0) + raw_off; -char *loop_buf =kmap_atomic (loop_page,KM_USER1 ) + loop_off; + char *raw_buf = kmap_atomic_push(raw_page) + raw_off; +char *loop_buf =kmap_atomic_push (loop_page) + loop_off;char *in, *out, *key;int i, keysize;
@@ -128,8 +128,8 @@ static int transfer_xor(struct loop_devifor (i =0 ; i < size; i++) *out++ = *in++ ^ key[(i &511 ) % keysize]; -kunmap_atomic (raw_buf,KM_USER0 ); -kunmap_atomic (loop_buf,KM_USER1 ); +kmap_atomic_pop (loop_buf); +kmap_atomic_pop (raw_buf);cond_resched ();return 0 ; }
drivers/block/pktcdvd.c +4 -4
@@ -1021,14 +1021,14 @@ static void pkt_copy_bio_data(struct biowhile (copy_size >0 ) {struct bio_vec *src_bvl =bio_iovec_idx (src_bio, seg); -void *vfrom =kmap_atomic (src_bvl->bv_page,KM_USER0 ) + +void *vfrom =kmap_atomic_push (src_bvl->bv_page) + src_bvl->bv_offset + offs;void *vto =page_address (dst_page) + dst_offs;int len =min_t (int, copy_size, src_bvl->bv_len - offs);BUG_ON (len <0 );memcpy (vto, vfrom, len); -kunmap_atomic (vfrom,KM_USER0 ); +kmap_atomic_pop (vfrom); seg++; offs =0 ;
@@ -1053,10 +1053,10 @@ static void pkt_make_local_copy(struct p offs =0 ;for (f =0 ; f < pkt->frames; f++) {if (bvec[f].bv_page != pkt->pages[p]) { -void *vfrom =kmap_atomic (bvec[f].bv_page,KM_USER0 ) + bvec[f].bv_offset; +void *vfrom =kmap_atomic_push (bvec[f].bv_page) + bvec[f].bv_offset;void *vto =page_address (pkt->pages[p]) + offs;memcpy (vto, vfrom,CD_FRAMESIZE ); -kunmap_atomic (vfrom,KM_USER0 ); +kmap_atomic_pop (vfrom); bvec[f].bv_page = pkt->pages[p]; bvec[f].bv_offset = offs; }else {
drivers/crypto/hifn_795x.c +5 -5
@@ -1731,9 +1731,9 @@ static int ablkcipher_get(void *saddr, uwhile (size) { copy =min (srest,min (dst->length, size)); - daddr =kmap_atomic (sg_page (dst),KM_IRQ0 ); + daddr =kmap_atomic_push (sg_page (dst));memcpy (daddr + dst->offset + offset, saddr, copy); -kunmap_atomic (daddr,KM_IRQ0 ); +kmap_atomic_pop (daddr); nbytes -= copy; size -= copy;
@@ -1793,17 +1793,17 @@ static void hifn_process_ready(struct abcontinue ; } - saddr =kmap_atomic (sg_page (t),KM_SOFTIRQ0 ); + saddr =kmap_atomic_push (sg_page (t)); err =ablkcipher_get (saddr, &t->length, t->offset, dst, nbytes, &nbytes);if (err <0 ) { -kunmap_atomic (saddr,KM_SOFTIRQ0 ); +kmap_atomic_pop (saddr);break ; } idx += err; -kunmap_atomic (saddr,KM_SOFTIRQ0 ); +kmap_atomic_pop (saddr); }ablkcipher_walk_exit (&rctx->walk);
drivers/edac/edac_mc.c +2 -2
@@ -588,13 +588,13 @@ static void edac_mc_scrub_block(unsignedif (PageHighMem (pg))local_irq_save (flags); - virt_addr =kmap_atomic (pg,KM_BOUNCE_READ ); + virt_addr =kmap_atomic_push (pg);/* Perform architecture specific atomic scrub operation */ atomic_scrub (virt_addr + offset, size);/* Unmap and complete */ -kunmap_atomic (virt_addr,KM_BOUNCE_READ ); +kmap_atomic_pop (virt_addr);if (PageHighMem (pg))local_irq_restore (flags);
drivers/gpu/drm/drm_cache.c +4 -4
@@ -40,10 +40,10 @@ drm_clflush_page(struct page *page)if (unlikely (page ==NULL ))return ; - page_virtual =kmap_atomic (page,KM_USER0 ); + page_virtual =kmap_atomic_push (page);for (i =0 ; i <PAGE_SIZE ; i += boot_cpu_data.x86_clflush_size)clflush (page_virtual + i); -kunmap_atomic (page_virtual,KM_USER0 ); +kmap_atomic_pop (page_virtual); } static void drm_cache_flush_clflush(struct page *pages[],
@@ -86,10 +86,10 @@ drm_clflush_pages(struct page *pages[],if (unlikely (page ==NULL ))continue ; - page_virtual =kmap_atomic (page,KM_USER0 ); + page_virtual =kmap_atomic_push (page);flush_dcache_range ((unsigned long )page_virtual, (unsigned long )page_virtual +PAGE_SIZE ); - kunmap_atomic(page_virtual, KM_USER0); + kmap_atomic_pop(page_virtual); }#else printk (KERN_ERR "Architecture has no drm_cache.c support\n" );
drivers/gpu/drm/i915/i915_gem.c +22 -22
132 lines not on this page · raw message
drivers/gpu/drm/i915/i915_gem_debug.c +5 -5
@@ -57,13 +57,13 @@ static void i915_gem_dump_page(struct page *page, uint32_t start, uint32_t end, uint32_t bias, uint32_t mark) { - uint32_t *mem = kmap_atomic(page, KM_USER0); + uint32_t *mem = kmap_atomic_push(page);int i;for (i = start; i < end; i +=4 )DRM_INFO ("%08x: %08x%s\n" , (int ) (bias + i), mem[i /4 ], (bias + i == mark) ?" ********" :"" ); -kunmap_atomic (mem,KM_USER0 ); +kmap_atomic_pop (mem);/* give syslog time to catch up */ msleep (1 ); }
@@ -157,7 +157,7 @@ i915_gem_object_check_coherency(struct dfor (page =0 ; page < obj->size /PAGE_SIZE ; page++) {int i; - backing_map =kmap_atomic (obj_priv->pages[page],KM_USER0 ); + backing_map =kmap_atomic_push (obj_priv->pages[page]);if (backing_map ==NULL ) {DRM_ERROR ("failed to map backing page\n" );
@@ -181,13 +181,13 @@ i915_gem_object_check_coherency(struct d } } } -kunmap_atomic (backing_map,KM_USER0 ); +kmap_atomic_pop (backing_map); backing_map =NULL ; } out:if (backing_map !=NULL ) -kunmap_atomic (backing_map,KM_USER0 ); +kmap_atomic_pop (backing_map);iounmap (gtt_mapping);/* give syslog time to catch up */
drivers/gpu/drm/ttm/ttm_bo_util.c +4 -4
@@ -148,7 +148,7 @@ static int ttm_copy_io_ttm_page(struct t src = (void *)((unsigned long )src + (page <<PAGE_SHIFT ));#ifdef CONFIG_X86 - dst =kmap_atomic_prot (d,KM_USER0 , prot); + dst =kmap_atomic_push_prot (d);#else if (pgprot_val (prot) !=pgprot_val (PAGE_KERNEL )) dst =vmap (&d,1 ,0 , prot);
@@ -161,7 +161,7 @@ static int ttm_copy_io_ttm_page(struct tmemcpy_fromio (dst, src,PAGE_SIZE );#ifdef CONFIG_X86 -kunmap_atomic (dst,KM_USER0 ); +kmap_atomic_pop (dst);#else if (pgprot_val (prot) !=pgprot_val (PAGE_KERNEL ))vunmap (dst);
@@ -184,7 +184,7 @@ static int ttm_copy_ttm_io_page(struct t dst = (void *)((unsigned long )dst + (page <<PAGE_SHIFT ));#ifdef CONFIG_X86 - src =kmap_atomic_prot (s,KM_USER0 , prot); + src =kmap_atomic_push_prot (s);#else if (pgprot_val (prot) !=pgprot_val (PAGE_KERNEL )) src =vmap (&s,1 ,0 , prot);
@@ -197,7 +197,7 @@ static int ttm_copy_ttm_io_page(struct tmemcpy_toio (dst, src,PAGE_SIZE );#ifdef CONFIG_X86 -kunmap_atomic (src,KM_USER0 ); +kmap_atomic_pop (src);#else if (pgprot_val (prot) !=pgprot_val (PAGE_KERNEL ))vunmap (src);
drivers/gpu/drm/ttm/ttm_tt.c +8 -8
@@ -491,11 +491,11 @@ static int ttm_tt_swapin(struct ttm_tt * goto out_err;preempt_disable (); - from_virtual =kmap_atomic (from_page,KM_USER0 ); - to_virtual =kmap_atomic (to_page,KM_USER1 ); + from_virtual =kmap_atomic_push (from_page); + to_virtual =kmap_atomic_push (to_page);memcpy (to_virtual, from_virtual,PAGE_SIZE ); -kunmap_atomic (to_virtual,KM_USER1 ); -kunmap_atomic (from_virtual,KM_USER0 ); +kmap_atomic_pop (to_virtual); +kmap_atomic_pop (from_virtual);preempt_enable ();page_cache_release (from_page); }
@@ -558,11 +558,11 @@ int ttm_tt_swapout(struct ttm_tt *ttm, s goto out_err;preempt_disable (); - from_virtual =kmap_atomic (from_page,KM_USER0 ); - to_virtual =kmap_atomic (to_page,KM_USER1 ); + from_virtual =kmap_atomic_push (from_page); + to_virtual =kmap_atomic_push (to_page);memcpy (to_virtual, from_virtual,PAGE_SIZE ); -kunmap_atomic (to_virtual,KM_USER1 ); -kunmap_atomic (from_virtual,KM_USER0 ); +kmap_atomic_pop (to_virtual); +kmap_atomic_pop (from_virtual);preempt_enable ();set_page_dirty (to_page);mark_page_accessed (to_page);
drivers/ide/ide-taskfile.c +2 -2
@@ -252,7 +252,7 @@ void ide_pio_bytes(ide_drive_t *drive, sif (page_is_high)local_irq_save (flags); - buf =kmap_atomic (page,KM_BIO_SRC_IRQ ) + offset; + buf =kmap_atomic_push (page) + offset; cmd->nleft -= nr_bytes; cmd->cursg_ofs += nr_bytes;
@@ -268,7 +268,7 @@ void ide_pio_bytes(ide_drive_t *drive, selse hwif->tp_ops->input_data (drive, cmd, buf, nr_bytes); -kunmap_atomic (buf,KM_BIO_SRC_IRQ ); +kmap_atomic_pop (buf);if (page_is_high)local_irq_restore (flags);
drivers/infiniband/ulp/iser/iser_memory.c +4 -4
@@ -129,11 +129,11 @@ static int iser_start_rdma_unaligned_sg( p = mem;for_each_sg (sgl, sg, data->size, i) { - from =kmap_atomic (sg_page (sg),KM_USER0 ); + from =kmap_atomic_push (sg_page (sg));memcpy (p, from + sg->offset, sg->length); -kunmap_atomic (from,KM_USER0 ); +kmap_atomic_pop (from); p += sg->length; } }
@@ -189,11 +189,11 @@ void iser_finalize_rdma_unaligned_sg(str p = mem;for_each_sg (sgl, sg, sg_size, i) { - to =kmap_atomic (sg_page (sg),KM_SOFTIRQ0 ); + to =kmap_atomic_push (sg_page (sg));memcpy (to + sg->offset, p, sg->length); -kunmap_atomic (to,KM_SOFTIRQ0 ); +kmap_atomic_pop (to); p += sg->length; } }
drivers/md/bitmap.c +18 -18
@@ -494,14 +494,14 @@ void bitmap_update_sb(struct bitmap *bitreturn ; }spin_unlock_irqrestore (&bitmap->lock, flags); - sb = (bitmap_super_t *)kmap_atomic (bitmap->sb_page,KM_USER0 ); + sb = (bitmap_super_t *)kmap_atomic_push (bitmap->sb_page); sb->events =cpu_to_le64 (bitmap->mddev->events);if (bitmap->mddev->events < bitmap->events_cleared) {/* rocking back to read-only */ bitmap->events_cleared = bitmap->mddev->events; sb->events_cleared =cpu_to_le64 (bitmap->events_cleared); } -kunmap_atomic (sb,KM_USER0 ); +kmap_atomic_pop (sb);write_page (bitmap, bitmap->sb_page,1 ); }
@@ -512,7 +512,7 @@ void bitmap_print_sb(struct bitmap *bitmif (!bitmap || !bitmap->sb_page)return ; - sb = (bitmap_super_t *)kmap_atomic (bitmap->sb_page,KM_USER0 ); + sb = (bitmap_super_t *)kmap_atomic_push (bitmap->sb_page);printk (KERN_DEBUG "%s: bitmap file superblock:\n" ,bmname (bitmap));printk (KERN_DEBUG " magic: %08x\n" ,le32_to_cpu (sb->magic));printk (KERN_DEBUG " version: %d\n" ,le32_to_cpu (sb->version));
@@ -531,7 +531,7 @@ void bitmap_print_sb(struct bitmap *bitmprintk (KERN_DEBUG " sync size: %llu KB\n" , (unsigned long long )le64_to_cpu (sb->sync_size)/2 );printk (KERN_DEBUG "max write behind: %d\n" ,le32_to_cpu (sb->write_behind)); -kunmap_atomic (sb,KM_USER0 ); +kmap_atomic_pop (sb); }/* read the superblock from the bitmap file and initialize some bitmap fields */
@@ -560,7 +560,7 @@ static int bitmap_read_sb(struct bitmapreturn err; } - sb = (bitmap_super_t *)kmap_atomic (bitmap->sb_page,KM_USER0 ); + sb = (bitmap_super_t *)kmap_atomic_push (bitmap->sb_page); chunksize =le32_to_cpu (sb->chunksize); daemon_sleep =le32_to_cpu (sb->daemon_sleep);
@@ -622,7 +622,7 @@ success: bitmap->events_cleared = bitmap->mddev->events; err =0 ; out: -kunmap_atomic (sb,KM_USER0 ); +kmap_atomic_pop (sb);if (err)bitmap_print_sb (bitmap);return err;
@@ -647,7 +647,7 @@ static int bitmap_mask_state(struct bitmreturn 0 ; }spin_unlock_irqrestore (&bitmap->lock, flags); - sb = (bitmap_super_t *)kmap_atomic (bitmap->sb_page,KM_USER0 ); + sb = (bitmap_super_t *)kmap_atomic_push (bitmap->sb_page); old =le32_to_cpu (sb->state) & bits;switch (op) {case MASK_SET : sb->state |=cpu_to_le32 (bits);
@@ -656,7 +656,7 @@ static int bitmap_mask_state(struct bitmbreak ;default :BUG (); } -kunmap_atomic (sb,KM_USER0 ); +kmap_atomic_pop (sb);return old; }
@@ -824,12 +824,12 @@ static void bitmap_file_set_bit(struct b bit =file_page_offset (chunk);/* set the bit */ - kaddr =kmap_atomic (page,KM_USER0 ); + kaddr =kmap_atomic_push (page);if (bitmap->flags &BITMAP_HOSTENDIAN )set_bit (bit, kaddr);else ext2_set_bit (bit, kaddr); -kunmap_atomic (kaddr,KM_USER0 ); +kmap_atomic_pop (kaddr);PRINTK ("set file bit %lu page %lu\n" , bit, page->index);/* record page number so it gets flushed to disk when unplug occurs */
@@ -995,10 +995,10 @@ static int bitmap_init_from_disk(struct* if bitmap is out of date, dirty the * whole page and write it out */ - paddr =kmap_atomic (page,KM_USER0 ); + paddr =kmap_atomic_push (page);memset (paddr + offset,0xff ,PAGE_SIZE - offset); -kunmap_atomic (paddr,KM_USER0 ); +kmap_atomic_pop (paddr);write_page (bitmap, page,1 ); ret = -EIO ;
@@ -1006,12 +1006,12 @@ static int bitmap_init_from_disk(struct goto err; } } - paddr =kmap_atomic (page,KM_USER0 ); + paddr =kmap_atomic_push (page);if (bitmap->flags &BITMAP_HOSTENDIAN ) b =test_bit (bit, paddr);else b =ext2_test_bit (bit, paddr); -kunmap_atomic (paddr,KM_USER0 ); +kmap_atomic_pop (paddr);if (b) {/* if the disk bit is set, set the memory bit */ int needed = ((sector_t)(i+1) << (CHUNK_BLOCK_SHIFT(bitmap))
@@ -1145,10 +1145,10 @@ void bitmap_daemon_work(struct bitmap *bif (bitmap->need_sync) {bitmap_super_t *sb; bitmap->need_sync =0 ; - sb =kmap_atomic (bitmap->sb_page,KM_USER0 ); + sb =kmap_atomic_push (bitmap->sb_page); sb->events_cleared =cpu_to_le64 (bitmap->events_cleared); -kunmap_atomic (sb,KM_USER0 ); +kmap_atomic_pop (sb);write_page (bitmap, bitmap->sb_page,1 ); }spin_lock_irqsave (&bitmap->lock, flags);
@@ -1175,12 +1175,12 @@ void bitmap_daemon_work(struct bitmap *b -1);/* clear the bit */ - paddr =kmap_atomic (page,KM_USER0 ); + paddr =kmap_atomic_push (page);if (bitmap->flags &BITMAP_HOSTENDIAN )clear_bit (file_page_offset (j), paddr);else ext2_clear_bit (file_page_offset (j), paddr); -kunmap_atomic (paddr,KM_USER0 ); +kmap_atomic_pop (paddr); } }else j |=PAGE_COUNTER_MASK ;
drivers/media/video/ivtv/ivtv-udma.c +2 -2
@@ -57,9 +57,9 @@ int ivtv_udma_fill_sg_list (struct ivtv_if (dma->bouncemap[map_offset] ==NULL )return -1 ;local_irq_save (flags); - src =kmap_atomic (dma->map[map_offset],KM_BOUNCE_READ ) + offset; + src =kmap_atomic_push (dma->map[map_offset]) + offset;memcpy (page_address (dma->bouncemap[map_offset]) + offset, src, len); -kunmap_atomic (src,KM_BOUNCE_READ ); +kmap_atomic_pop (src);local_irq_restore (flags);sg_set_page (&dma->SGlist[map_offset], dma->bouncemap[map_offset], len, offset); }
drivers/memstick/host/jmb38x_ms.c +2 -2
@@ -323,7 +323,7 @@ static int jmb38x_ms_transfer_data(struc p_cnt =min (p_cnt, length);local_irq_save (flags); - buf =kmap_atomic (pg,KM_BIO_SRC_IRQ ) + p_off; + buf =kmap_atomic_push (pg) + p_off; }else { buf = host->req->data + host->block_pos; p_cnt = host->req->data_len - host->block_pos;
@@ -339,7 +339,7 @@ static int jmb38x_ms_transfer_data(struc : jmb38x_ms_read_reg_data(host, buf, p_cnt);if (host->req->long_data) { -kunmap_atomic (buf - p_off,KM_BIO_SRC_IRQ ); +kmap_atomic_pop (buf - p_off);local_irq_restore (flags); }
drivers/memstick/host/tifm_ms.c +2 -2
@@ -209,7 +209,7 @@ static unsigned int tifm_ms_transfer_dat p_cnt =min (p_cnt, length);local_irq_save (flags); - buf =kmap_atomic (pg,KM_BIO_SRC_IRQ ) + p_off; + buf =kmap_atomic_push (pg) + p_off; }else { buf = host->req->data + host->block_pos; p_cnt = host->req->data_len - host->block_pos;
@@ -220,7 +220,7 @@ static unsigned int tifm_ms_transfer_dat : tifm_ms_read_data(host, buf, p_cnt);if (host->req->long_data) { -kunmap_atomic (buf - p_off,KM_BIO_SRC_IRQ ); +kmap_atomic_pop (buf - p_off);local_irq_restore (flags); }
drivers/mmc/host/at91_mci.c +4 -4
@@ -218,7 +218,7 @@ static inline void at91_mci_sg_to_dma(st sg = &data->sg[i]; - sgbuffer =kmap_atomic (sg_page (sg),KM_BIO_SRC_IRQ ) + sg->offset; + sgbuffer =kmap_atomic_push (sg_page (sg)) + sg->offset; amount =min (size, sg->length); size -= amount;
@@ -232,7 +232,7 @@ static inline void at91_mci_sg_to_dma(st dmabuf += amount; } -kunmap_atomic (sgbuffer,KM_BIO_SRC_IRQ ); +kmap_atomic_pop (sgbuffer);if (size ==0 )break ;
@@ -351,13 +351,13 @@ static void at91_mci_post_dma_read(strucint index;/* Swap the contents of the buffer */ - buffer =kmap_atomic (sg_page (sg),KM_BIO_SRC_IRQ ) + sg->offset; + buffer =kmap_atomic_push (sg_page (sg)) + sg->offset;pr_debug ("buffer = %p, length = %d\n" , buffer, sg->length);for (index =0 ; index < (sg->length /4 ); index++) buffer[index] =swab32 (buffer[index]); -kunmap_atomic (buffer,KM_BIO_SRC_IRQ ); +kmap_atomic_pop (buffer); }flush_dcache_page (sg_page (sg));
drivers/mmc/host/mmci.c +2 -2
@@ -327,7 +327,7 @@ static irqreturn_t mmci_pio_irq(int irq,/* * Map the current scatter buffer. */ - buffer =mmci_kmap_atomic (host, &flags) + host->sg_off; + buffer =mmci_kmap_atomic_push (host, &flags) + host->sg_off; remain = host->sg_ptr->length - host->sg_off; len =0 ;
@@ -339,7 +339,7 @@ static irqreturn_t mmci_pio_irq(int irq,/* * Unmap the buffer. */ -mmci_kunmap_atomic (host, buffer, &flags); +mmci_kmap_atomic_pop (host, buffer, &flags); host->sg_off += len; host->size -= len;
drivers/mmc/host/mmci.h +4 -4
@@ -195,16 +195,16 @@ static inline int mmci_next_sg(struct mmreturn --host->sg_len; } -static inline char *mmci_kmap_atomic (struct mmci_host *host,unsigned long *flags) +static inline char *mmci_kmap_atomic_push (struct mmci_host *host,unsigned long *flags) {struct scatterlist *sg = host->sg_ptr;local_irq_save (*flags); -return kmap_atomic (sg_page (sg),KM_BIO_SRC_IRQ ) + sg->offset; +return kmap_atomic_push (sg_page (sg)) + sg->offset; } -static inline void mmci_kunmap_atomic (struct mmci_host *host,void *buffer,unsigned long *flags) +static inline void mmci_kmap_atomic_pop (struct mmci_host *host,void *buffer,unsigned long *flags) { -kunmap_atomic (buffer,KM_BIO_SRC_IRQ ); +kmap_atomic_pop (buffer);local_irq_restore (*flags); }
drivers/mmc/host/msm_sdcc.c +3 -3
@@ -505,8 +505,8 @@ msmsdcc_pio_irq(int irq, void *dev_id)/* Map the current scatter buffer */ local_irq_save (flags); - buffer =kmap_atomic (sg_page (host->pio.sg), -KM_BIO_SRC_IRQ ) + host->pio.sg->offset; + buffer =kmap_atomic_push (sg_page (host->pio.sg)) + + host->pio.sg->offset; buffer += host->pio.sg_off; remain = host->pio.sg->length - host->pio.sg_off; len =0 ;
@@ -516,7 +516,7 @@ msmsdcc_pio_irq(int irq, void *dev_id) len =msmsdcc_pio_write (host, buffer, remain, status);/* Unmap the buffer */ -kunmap_atomic (buffer,KM_BIO_SRC_IRQ ); +kmap_atomic_pop (buffer);local_irq_restore (flags); host->pio.sg_off += len;
drivers/mmc/host/sdhci.c +8 -8
@@ -364,15 +364,15 @@ static void sdhci_transfer_pio(struct sdDBG ("PIO transfer complete.\n" ); } -static char *sdhci_kmap_atomic (struct scatterlist *sg,unsigned long *flags) +static char *sdhci_kmap_atomic_push (struct scatterlist *sg,unsigned long *flags) {local_irq_save (*flags); -return kmap_atomic (sg_page (sg),KM_BIO_SRC_IRQ ) + sg->offset; +return kmap_atomic_push (sg_page (sg)) + sg->offset; } -static void sdhci_kunmap_atomic (void *buffer,unsigned long *flags) +static void sdhci_kmap_atomic_pop (void *buffer,unsigned long *flags) { -kunmap_atomic (buffer,KM_BIO_SRC_IRQ ); +kmap_atomic_pop (buffer);local_irq_restore (*flags); }
@@ -437,10 +437,10 @@ static int sdhci_adma_table_pre(struct s offset = (4 - (addr &0x3 )) &0x3 ;if (offset) {if (data->flags &MMC_DATA_WRITE ) { - buffer =sdhci_kmap_atomic (sg, &flags); + buffer =sdhci_kmap_atomic_push (sg, &flags);WARN_ON (((long )buffer &PAGE_MASK ) > (PAGE_SIZE -3 ));memcpy (align, buffer, offset); -sdhci_kunmap_atomic (buffer, &flags); +sdhci_kmap_atomic_pop (buffer, &flags); } desc[7 ] = (align_addr >>24 ) &0xff ;
@@ -559,10 +559,10 @@ static void sdhci_adma_table_post(structif (sg_dma_address (sg) &0x3 ) { size =4 - (sg_dma_address (sg) &0x3 ); - buffer =sdhci_kmap_atomic (sg, &flags); + buffer =sdhci_kmap_atomic_push (sg, &flags);WARN_ON (((long )buffer &PAGE_MASK ) > (PAGE_SIZE -3 ));memcpy (buffer, align, size); -sdhci_kunmap_atomic (buffer, &flags); +sdhci_kmap_atomic_pop (buffer, &flags); align +=4 ; }
drivers/mmc/host/tifm_sd.c +8 -8
@@ -117,7 +117,7 @@ static void tifm_sd_read_fifo(struct tifunsigned char *buf;unsigned int pos =0 , val; - buf =kmap_atomic (pg,KM_BIO_DST_IRQ ) + off; + buf =kmap_atomic_push (pg) + off;if (host->cmd_flags &DATA_CARRY ) { buf[pos++] = host->bounce_buf_data[0 ]; host->cmd_flags &= ~DATA_CARRY ;
@@ -133,7 +133,7 @@ static void tifm_sd_read_fifo(struct tif } buf[pos++] = (val >>8 ) &0xff ; } -kunmap_atomic (buf - off,KM_BIO_DST_IRQ ); +kmap_atomic_pop (buf - off); } static void tifm_sd_write_fifo(struct tifm_sd *host, struct page *pg,
@@ -143,7 +143,7 @@ static void tifm_sd_write_fifo(struct tiunsigned char *buf;unsigned int pos =0 , val; - buf =kmap_atomic (pg,KM_BIO_SRC_IRQ ) + off; + buf =kmap_atomic_push (pg) + off;if (host->cmd_flags &DATA_CARRY ) { val = host->bounce_buf_data[0 ] | ((buf[pos++] <<8 ) &0xff00 );writel (val, sock->addr +SOCK_MMCSD_DATA );
@@ -160,7 +160,7 @@ static void tifm_sd_write_fifo(struct ti val |= (buf[pos++] <<8 ) &0xff00 ;writel (val, sock->addr +SOCK_MMCSD_DATA ); } -kunmap_atomic (buf - off,KM_BIO_SRC_IRQ ); +kmap_atomic_pop (buf - off); }static void tifm_sd_transfer_data (struct tifm_sd *host)
@@ -211,13 +211,13 @@ static void tifm_sd_copy_page(struct pag struct page *src, unsigned int src_off, unsigned int count) { - unsigned char *src_buf = kmap_atomic(src, KM_BIO_SRC_IRQ) + src_off; -unsigned char *dst_buf =kmap_atomic (dst,KM_BIO_DST_IRQ ) + dst_off; + unsigned char *src_buf = kmap_atomic_push(src) + src_off; +unsigned char *dst_buf =kmap_atomic_push (dst) + dst_off;memcpy (dst_buf, src_buf, count); -kunmap_atomic (dst_buf - dst_off,KM_BIO_DST_IRQ ); -kunmap_atomic (src_buf - src_off,KM_BIO_SRC_IRQ ); +kmap_atomic_pop (dst_buf - dst_off); +kmap_atomic_pop (src_buf - src_off); }static void tifm_sd_bounce_block (struct tifm_sd *host,struct mmc_data *r_data)
drivers/mmc/host/tmio_mmc.c +2 -2
@@ -170,7 +170,7 @@ static inline void tmio_mmc_pio_irq(strureturn ; } - buf = (unsigned short *)(tmio_mmc_kmap_atomic (host, &flags) + + buf = (unsigned short *)(tmio_mmc_kmap_atomic_push (host, &flags) + host->sg_off); count = host->sg_ptr->length - host->sg_off;
@@ -188,7 +188,7 @@ static inline void tmio_mmc_pio_irq(stru host->sg_off += count; -tmio_mmc_kunmap_atomic (host, &flags); +tmio_mmc_kmap_atomic_pop (host, &flags);if (host->sg_off == host->sg_ptr->length)tmio_mmc_next_sg (host);
drivers/mmc/host/tmio_mmc.h +4 -4
@@ -200,19 +200,19 @@ static inline int tmio_mmc_next_sg(strucreturn --host->sg_len; } -static inline char *tmio_mmc_kmap_atomic (struct tmio_mmc_host *host, +static inline char *tmio_mmc_kmap_atomic_push (struct tmio_mmc_host *host,unsigned long *flags) {struct scatterlist *sg = host->sg_ptr;local_irq_save (*flags); -return kmap_atomic (sg_page (sg),KM_BIO_SRC_IRQ ) + sg->offset; +return kmap_atomic_push (sg_page (sg)) + sg->offset; } -static inline void tmio_mmc_kunmap_atomic (struct tmio_mmc_host *host, +static inline void tmio_mmc_kmap_atomic_pop (struct tmio_mmc_host *host,unsigned long *flags) { -kunmap_atomic (sg_page (host->sg_ptr),KM_BIO_SRC_IRQ ); +kmap_atomic_pop (sg_page (host->sg_ptr));local_irq_restore (*flags); }
drivers/net/cassini.c +2 -2
@@ -102,8 +102,8 @@#include <asm/byteorder.h> #include <asm/uaccess.h> -#define cas_page_map (x) kmap_atomic((x), KM_SKB_DATA_SOFTIRQ) -#define cas_page_unmap (x) kunmap_atomic((x), KM_SKB_DATA_SOFTIRQ) +#define cas_page_map (x) kmap_atomic_push((x)) +#define cas_page_unmap (x) kmap_atomic_pop((x))#define CAS_NCPUS num_online_cpus()#if defined(CONFIG_CASSINI_NAPI ) && defined(HAVE_NETDEV_POLL )
drivers/net/e1000/e1000_main.c +2 -4
@@ -3705,11 +3705,9 @@ static bool e1000_clean_jumbo_rx_irq(strif (length <= copybreak &&skb_tailroom (skb) >= length) {u8 *vaddr; - vaddr =kmap_atomic (buffer_info->page, -KM_SKB_DATA_SOFTIRQ ); + vaddr =kmap_atomic_push (buffer_info->page);memcpy (skb_tail_pointer (skb), vaddr, length); -kunmap_atomic (vaddr, -KM_SKB_DATA_SOFTIRQ ); +kmap_atomic_pop (vaddr);/* re-use the page, so don't erase * buffer_info->page */ skb_put (skb, length);
drivers/net/e1000e/netdev.c +5 -7
@@ -791,14 +791,14 @@ static bool e1000_clean_rx_irq_ps(struct/* * there is no documentation about how to call -* kmap_atomic, so we can't hold the mapping +* kmap_atomic_push, so we can't hold the mapping * very long */ pci_dma_sync_single_for_cpu (pdev, ps_page->dma,PAGE_SIZE ,PCI_DMA_FROMDEVICE ); - vaddr =kmap_atomic (ps_page->page,KM_SKB_DATA_SOFTIRQ ); + vaddr =kmap_atomic_push (ps_page->page);memcpy (skb_tail_pointer (skb), vaddr, l1); -kunmap_atomic (vaddr,KM_SKB_DATA_SOFTIRQ ); +kmap_atomic_pop (vaddr);pci_dma_sync_single_for_device (pdev, ps_page->dma,PAGE_SIZE ,PCI_DMA_FROMDEVICE );
@@ -991,12 +991,10 @@ static bool e1000_clean_jumbo_rx_irq(strif (length <= copybreak &&skb_tailroom (skb) >= length) {u8 *vaddr; - vaddr =kmap_atomic (buffer_info->page, -KM_SKB_DATA_SOFTIRQ ); + vaddr =kmap_atomic_push (buffer_info->page);memcpy (skb_tail_pointer (skb), vaddr, length); -kunmap_atomic (vaddr, -KM_SKB_DATA_SOFTIRQ ); +kmap_atomic_pop (vaddr);/* re-use the page, so don't erase * buffer_info->page */ skb_put (skb, length);
drivers/scsi/arcmsr/arcmsr_hba.c +4 -4
@@ -1370,7 +1370,7 @@ static int arcmsr_iop_message_xfer(struc/* 4 bytes: Areca io control code */ sg =scsi_sglist (cmd); - buffer =kmap_atomic (sg_page (sg),KM_IRQ0 ) + sg->offset; + buffer =kmap_atomic_push (sg_page (sg)) + sg->offset;if (scsi_sg_count (cmd) >1 ) { retvalue =ARCMSR_MESSAGE_FAIL ; goto message_out;
@@ -1573,7 +1573,7 @@ static int arcmsr_iop_message_xfer(struc } message_out: sg =scsi_sglist (cmd); -kunmap_atomic (buffer - sg->offset,KM_IRQ0 ); +kmap_atomic_pop (buffer - sg->offset);return retvalue; }
@@ -1618,11 +1618,11 @@ static void arcmsr_handle_virtual_commanstrncpy (&inqdata[32 ],"R001" ,4 );/* Product Revision */ sg =scsi_sglist (cmd); - buffer =kmap_atomic (sg_page (sg),KM_IRQ0 ) + sg->offset; + buffer =kmap_atomic_push (sg_page (sg)) + sg->offset;memcpy (buffer, inqdata,sizeof (inqdata)); sg =scsi_sglist (cmd); -kunmap_atomic (buffer - sg->offset,KM_IRQ0 ); +kmap_atomic_pop (buffer - sg->offset); cmd->scsi_done (cmd); }
drivers/scsi/cxgb3i/cxgb3i_pdu.c +2 -3
@@ -319,12 +319,11 @@ int cxgb3i_conn_init_pdu(struct iscsi_ta/* data fits in the skb's headroom */ for (i =0 ; i < tdata->nr_frags; i++, frag++) { -char *src =kmap_atomic (frag->page, -KM_SOFTIRQ0 ); +char *src =kmap_atomic_push (frag->page);memcpy (dst, src+frag->page_offset, frag->size); dst += frag->size; -kunmap_atomic (src,KM_SOFTIRQ0 ); +kmap_atomic_pop (src); }if (padlen) {memset (dst,0 , padlen);
drivers/scsi/dc395x.c +6 -6
@@ -2279,7 +2279,7 @@ static void data_in_phase0(struct Adaptelocal_irq_save (flags);/* Assumption: it's inside one page as it's at most 4 bytes and I just assume it's on a 4-byte boundary */ - base =scsi_kmap_atomic_sg (scsi_sglist (srb->cmd), + base =scsi_kmap_atomic_push_sg (scsi_sglist (srb->cmd), srb->sg_count, &offset, &len); virt = base + offset;
@@ -2322,7 +2322,7 @@ static void data_in_phase0(struct AdapteDC395x_write8 (acb,TRM_S1040_SCSI_CONFIG2 ,0 ); } -scsi_kunmap_atomic_sg (base); +scsi_kmap_atomic_pop_sg (base);local_irq_restore (flags); }/*printk(" %08x", *(u32*)(bus_to_virt (addr))); */
@@ -2496,7 +2496,7 @@ static void data_io_transfer(struct Adaplocal_irq_save (flags);/* Again, max 4 bytes */ - base =scsi_kmap_atomic_sg (scsi_sglist (srb->cmd), + base =scsi_kmap_atomic_push_sg (scsi_sglist (srb->cmd), srb->sg_count, &offset, &len); virt = base + offset;
@@ -2511,7 +2511,7 @@ static void data_io_transfer(struct Adapsg_subtract_one (srb); } -scsi_kunmap_atomic_sg (base); +scsi_kmap_atomic_pop_sg (base);local_irq_restore (flags); }if (srb->dcb->sync_period &WIDE_SYNC ) {
@@ -3465,7 +3465,7 @@ static void srb_done(struct AdapterCtlBlsize_t offset =0 , len =sizeof (struct ScsiInqData );local_irq_save (flags); - base =scsi_kmap_atomic_sg (sg,scsi_sg_count (cmd), &offset, &len); + base =scsi_kmap_atomic_push_sg (sg,scsi_sg_count (cmd), &offset, &len); ptr = (struct ScsiInqData *)(base + offset); if (!ckc_only && (cmd->result & RES_DID) == 0
@@ -3484,7 +3484,7 @@ static void srb_done(struct AdapterCtlBl } } -scsi_kunmap_atomic_sg (base); +scsi_kmap_atomic_pop_sg (base);local_irq_restore (flags); }
drivers/scsi/fcoe/fcoe.c +4 -5
@@ -1159,10 +1159,9 @@ u32 fcoe_fc_crc(struct fc_frame *fp) len = frag->size;while (len >0 ) { clen =min (len,PAGE_SIZE - (off & ~PAGE_MASK )); - data =kmap_atomic (frag->page + (off >>PAGE_SHIFT ), -KM_SKB_DATA_SOFTIRQ ); + data =kmap_atomic_push (frag->page + (off >>PAGE_SHIFT )); crc =crc32 (crc, data + (off & ~PAGE_MASK ), clen); -kunmap_atomic (data,KM_SKB_DATA_SOFTIRQ ); +kmap_atomic_pop (data); off += clen; len -= clen; }
@@ -1236,7 +1235,7 @@ int fcoe_xmit(struct fc_lport *lp, strucreturn -ENOMEM ; } frag = &skb_shinfo (skb)->frags[skb_shinfo (skb)->nr_frags -1 ]; - cp =kmap_atomic (frag->page,KM_SKB_DATA_SOFTIRQ ) + cp =kmap_atomic_push (frag->page) + frag->page_offset; }else { cp = (struct fcoe_crc_eof *)skb_put (skb, tlen);
@@ -1247,7 +1246,7 @@ int fcoe_xmit(struct fc_lport *lp, struc cp->fcoe_crc32 =cpu_to_le32 (~crc);if (skb_is_nonlinear (skb)) { -kunmap_atomic (cp,KM_SKB_DATA_SOFTIRQ ); +kmap_atomic_pop (cp); cp =NULL ; }
drivers/scsi/gdth.c +3 -3
@@ -2279,7 +2279,7 @@ static void gdth_next(gdth_ha_str *ha)/* * gdth_copy_internal_data() - copy to/from a buffer onto a scsi_cmnd's -* buffers, kmap_atomic() as needed. +* buffers, kmap_atomic_push() as needed. */ static void gdth_copy_internal_data (gdth_ha_str *ha,Scsi_Cmnd *scp,char *buffer,ushort count)
@@ -2307,10 +2307,10 @@ static void gdth_copy_internal_data(gdthreturn ; }local_irq_save (flags); - address =kmap_atomic (sg_page (sl),KM_BIO_SRC_IRQ ) + sl->offset; + address =kmap_atomic_push (sg_page (sl)) + sl->offset;memcpy (address, buffer, cpnow);flush_dcache_page (sg_page (sl)); -kunmap_atomic (address,KM_BIO_SRC_IRQ ); +kmap_atomic_pop (address);local_irq_restore (flags);if (cpsum == cpcount)break ;
drivers/scsi/ips.c +4 -4
@@ -1506,17 +1506,17 @@ static int ips_is_passthru(struct scsi_cstruct scatterlist *sg =scsi_sglist (SC );char *buffer; -/* kmap_atomic() ensures addressability of the user buffer.*/ +/* kmap_atomic_push() ensures addressability of the user buffer.*/ /* local_irq_save() protects the KM_IRQ0 address slot. */ local_irq_save (flags); - buffer =kmap_atomic (sg_page (sg),KM_IRQ0 ) + sg->offset; + buffer =kmap_atomic_push (sg_page (sg)) + sg->offset;if (buffer && buffer[0 ] =='C' && buffer[1 ] =='O' && buffer[2 ] =='P' && buffer[3 ] =='P' ) { -kunmap_atomic (buffer - sg->offset,KM_IRQ0 ); +kmap_atomic_pop (buffer - sg->offset);local_irq_restore (flags);return 1 ; } -kunmap_atomic (buffer - sg->offset,KM_IRQ0 ); +kmap_atomic_pop (buffer - sg->offset);local_irq_restore (flags); }return 0 ;
drivers/scsi/libfc/fc_fcp.c +5 -7
@@ -377,8 +377,7 @@ static void fc_fcp_recv_data(struct fc_f off = offset + sg->offset; sg_bytes =min (sg_bytes, (size_t ) (PAGE_SIZE - (off & ~PAGE_MASK ))); - page_addr =kmap_atomic (sg_page (sg) + (off >>PAGE_SHIFT ), -KM_SOFTIRQ0 ); + page_addr = kmap_atomic_push(sg_page(sg) + (off >> PAGE_SHIFT))if (!page_addr)break ;/* XXX panic? */
@@ -387,7 +386,7 @@ static void fc_fcp_recv_data(struct fc_fmemcpy ((char *)page_addr + (off & ~PAGE_MASK ), buf, sg_bytes); -kunmap_atomic (page_addr,KM_SOFTIRQ0 ); +kmap_atomic_pop (page_addr); buf += sg_bytes; offset += sg_bytes; remaining -= sg_bytes;
@@ -559,12 +558,11 @@ static int fc_fcp_send_data(struct fc_fc*/ sg_bytes =min (sg_bytes, (size_t ) (PAGE_SIZE - (off & ~PAGE_MASK ))); - page_addr =kmap_atomic (sg_page (sg) + - (off >>PAGE_SHIFT ), -KM_SOFTIRQ0 ); + page_addr =kmap_atomic_push (sg_page (sg) + + (off >>PAGE_SHIFT ));memcpy (data, (char *)page_addr + (off & ~PAGE_MASK ), sg_bytes); -kunmap_atomic (page_addr,KM_SOFTIRQ0 ); +kmap_atomic_pop (page_addr); data += sg_bytes; } offset += sg_bytes;
drivers/scsi/libiscsi_tcp.c +2 -2
@@ -131,14 +131,14 @@ static void iscsi_tcp_segment_map(structif (page_count (sg_page (sg)) >=1 && !recv)return ;
Lines of this file are not shown. View the raw message.
fs/btrfs/extent_io.c +30 -30
60 lines not on this page · raw message
fs/nilfs2/cpfile.c +47 -47
94 lines not on this page · raw message
fs/nilfs2/sufile.c +25 -25
50 lines not on this page · raw message
include/linux/highmem.h +33 -25
58 lines not on this page · raw message
mm/highmem.c +3 -45
48 lines not on this page · raw message
The rest of this diff is not shown. View the raw message.
Part of the diff is not shown here. View the raw message.
Quoted 3 lines · Peter Zijlstra
The below patchlet changes the
kmap_atomicinterface to a stack based one that doesn't require theKM_typesanymore.
I think this is how we should have done it originally.
That said, if we do this, I'd hate to have the "push" and "pop" parts to the name. They don't really add a whole lot.
Linus
Quoted 10 lines · Peter Zijlstra
The below patchlet changes the
kmap_atomicinterface to a stack based one that doesn't require theKM_typesanymore.This significantly simplifies some code (more still than are present in this patch -- ie.
pte_map_nestedcan go now)This obviously requires that push and pop are matched, I fixed a few cases that were not properly nested, the (x86) code checks for this and will go BUG when trying to pop a vaddr that isn't the top one so abusers should be rather visible.
Looks great IMO! Last i proposed this i think either Andrew or Avi had second thoughts about the hard-to-calculate worst-case mapping limit - but i dont think that's a big issue.
Lets not change the API names though - the rule is that map/unmap must be properly nested.
Ingo
Peter Zijlstra
maintainerQuoted 8 lines · Ingo Molnar
Quoted 6 lines · Peter Zijlstra
The below patchlet changes the
kmap_atomicinterface to a stack based one that doesn't require theKM_typesanymore.This significantly simplifies some code (more still than are present in this patch -- ie.
pte_map_nestedcan go now)
Quoted 8 lines
Quoted 4 lines
This obviously requires that push and pop are matched, I fixed a few cases that were not properly nested, the (x86) code checks for this and will go BUG when trying to pop a vaddr that isn't the top one so abusers should be rather visible.
Looks great IMO! Last i proposed this i think either Andrew or Avi had second thoughts about the hard-to-calculate worst-case mapping limit - but i dont think that's a big issue.
That would've been me ;-)
Quoted 2 lines
Lets not change the API names though - the rule is that map/unmap must be properly nested.
Right, so I did that full rename just so that people wouldn't get confused or something, but if both you and Linus think it should remain:kmap_atomic() and kunmap_atomic(), I can certainly undo that part.
Quoted 4 lines · Peter Zijlstra
Right, so I did that full rename just so that people wouldn't get confused or something, but if both you and Linus think it should remain:
kmap_atomic()andkunmap_atomic(), I can certainly undo that part.
I think the renaming probably helps find all the places (simple "grep -w" shows the difference, and no fear of confusion with comma-expressions and multi-line arguments etc). But once they've all been converted, you might as well then do a search-and-replace-back on the patch, and make the end result look like you just removed the (now pointless) argument.
In fact, I'd personally be inclined to split the patch into two patches:
- one that just ignores the now redundant argument (but still keeps it), and fixes the cases that didn't nest
- one that then removes the argument.
Why? The _bugs_ are going to be shown by the first patch, and it would be nice to keep that patch small. When a bug shows up, it would be either because there's something wrong in that (much smaller) patch, or because some not-properly-nested casel wasn't fixed.
In contrast, the second patch would be large, but if done right, you could then prove that it has no actual semantic changes (ie "binary is same before and after"). That just sounds _much_ nicer from a debug standpoint. Developers would look at the small and concentrated "real changes" patch, rather than be distracted by all the trivial noise.
Linus
Quoted 5 lines · Linus Torvalds
Quoted 4 lines · Peter Zijlstra
Right, so I did that full rename just so that people wouldn't get confused or something, but if both you and Linus think it should remain:
kmap_atomic()andkunmap_atomic(), I can certainly undo that part.
I love the patch, Peter: thank you. But agree with the others to keep the old names, just let the change in prototype (vanishing second arg) do the work of weeding out any stragglers.
Quoted 19 lines
I think the renaming probably helps find all the places (simple "grep -w" shows the difference, and no fear of confusion with comma-expressions and multi-line arguments etc). But once they've all been converted, you might as well then do a search-and-replace-back on the patch, and make the end result look like you just removed the (now pointless) argument.
In fact, I'd personally be inclined to split the patch into two patches:
- one that just ignores the now redundant argument (but still keeps it), and fixes the cases that didn't nest
- one that then removes the argument.
Why? The _bugs_ are going to be shown by the first patch, and it would be nice to keep that patch small. When a bug shows up, it would be either because there's something wrong in that (much smaller) patch, or because some not-properly-nested casel wasn't fixed.
Quoted 5 lines
In contrast, the second patch would be large, but if done right, you could then prove that it has no actual semantic changes (ie "binary is same before and after"). That just sounds _much_ nicer from a debug standpoint. Developers would look at the small and concentrated "real changes" patch, rather than be distracted by all the trivial noise.
And I very much agree with Linus's two patch approach: it also makes it much easier to review, separating the wheat of the interesting changes from the chaff of eliminating the unnecessary arg. It was hard to find the interesting part in the patch as you sent it.
I wasn't really checking it, but think I noticed something called swap_two_pages() somewhere, which wasn't doing the unnesting right:
you may need to swap two lines ;)
Hugh
So if I understand this correctly, the sequence:
in = kmap_atomic(inpage, KM_USER1);
out = kmap_atomic(outpage, KM_USER0);
kunmap_atomic(in, KM_USER1);
in = kmap_atomic(next_inpage, KM_USER1);
is now illegal with this patch, which breaks code I am testing now for btrfs.
My code does this because the in/out are zlib inflate and the in/out run at different rates.
OK, the code is not submitted yet and I can redesign the code using a temp buffer for out and copy every byte or use kmap(), either of them at some performance cost.
I'm just pointing out that there are cases where this stack design puts an ugly restriction on use.
So if I understand this right, I don't love the patch (:
jim
Peter Zijlstra
maintainerQuoted 11 lines · jim owens
So if I understand this correctly, the sequence:
in =
kmap_atomic(inpage, KM_USER1);out =
kmap_atomic(outpage, KM_USER0);
kunmap_atomic(in, KM_USER1);in =
kmap_atomic(next_inpage, KM_USER1);is now illegal with this patch
Yep.
Peter Zijlstra
maintainerQuoted 8 lines · jim owens
So if I understand this correctly, the sequence:
in =
kmap_atomic(inpage, KM_USER1);out =
kmap_atomic(outpage, KM_USER0);
kunmap_atomic(in, KM_USER1);
Quoted 7 lines
in =
kmap_atomic(next_inpage, KM_USER1);is now illegal with this patch, which breaks code
I am testing now for btrfs.My code does this because the in/out are zlib inflate and the in/out run at different rates.
You can do things like:
do {
in = kmap_atomic(inpage);
out = kmap_atomic(outpage);
<deflate until end of either in/out>
kunmap_atomic(outpage);
kunmap_atomic(inpage);
cond_resched();
<iterate bits>
} while (<not done>)
The double unmap gives a preemption point, which sounds like a good thing to have, because your scheme could run for a long while without enabling preemption, which is badness.
Quoted 4 lines · Peter Zijlstra
The double unmap gives a preemption point, which sounds like a good thing to have, because your scheme could run for a long while without enabling preemption, which is badness.
Thanks, optimizing my loop, I forgot all about
the old long code path not preempting problem.
Now I have to love the patch for making it harder to be stupid :)
David Howells
maintainerQuoted 7 lines · Peter Zijlstra
The below patchlet changes the
kmap_atomicinterface to a stack based one that doesn't require theKM_typesanymore.This significantly simplifies some code (more still than are present in this patch -- ie.
pte_map_nestedcan go now)
...
What do people think?
Brrr.
This makes FRV much worse. kmap_atomic() used to take a compile-time constant value - which meant that the switch-statement inside it was mostly optimised away. kmap_atomic() would be rendered down to very few instructions. Now it'll be a huge jump table plus all those few instructions repeated because the selector is now dynamic.
What I would prefer to see is something along the lines of local_irq_save() and local_irq_restore(), where the displaced value gets stored on the machine stack. In FRV, I could then represent kmap_atomic() as:
static inline void *kmap_atomic_push(struct page *page,
kmap_save_t *save)
{
unsigned long paddr, damlr, dampr;
int type;
pagefault_disable();
dampr = page_to_phys(page);
dampr |= xAMPRx_L | xAMPRx_M | xAMPRx_S | xAMPRx_SS_16Kb | xAMPRx_V;
asm volatile("movgs dampr6,%0 \n"
"movgs %0,dampr6"
: "=r"(save->dampr) : "r"(dampr) : "memory");
asm("movsg damlr6,%0" : "=r"(damlr));
return (void *) damlr;
}However, since we occasionally want a second kmap slot, we could also add:
typedef struct { unsigned long dampr; } kmap_save_t;
static inline void *kmap2_atomic_push(struct page *page,
kmap_save_t *save)
{
unsigned long paddr, damlr, dampr;
int type;
pagefault_disable();
dampr = page_to_phys(page);
dampr |= xAMPRx_L | xAMPRx_M | xAMPRx_S | xAMPRx_SS_16Kb | xAMPRx_V;
asm volatile("movgs dampr7,%0 \n"
"movgs %0,dampr7"
: "=r"(save->dampr) : "r"(dampr) : "memory");
asm("movsg damlr7,%0" : "=r"(damlr));
return (void *) damlr;
}And the reverse ops would be:
static inline void kmap_atomic_pop(kmap_save_t *save)
{
asm volatile("movgs %0,dampr6"
:: "r"(save->dampr) : "memory");
pagefault_enable();
}
static inline void kmap2_atomic_pop(kmap_save_t *save)
{
asm volatile("movgs %0,dampr7"
:: "r"(save->dampr) : "memory");
pagefault_enable();
}And I would avoid the need to lock fake TLB entries in the shallow TLB.
If it's too much trouble for an arch to extract the current kmap setting from the MMU or the page tables, *those* could be mirrored in per-CPU data.
Do we have any code that uses two slots and then calls more code that ultimately requires further slots? In other words, do we need more than two slots?
Quoted 3 lines
David, frv has a 'funny' issue in that it treats
__KM_CACHEspecial from the other ones, something which isn't possibly anymore. Do you see another option than to addkmap_atomic_push_cache()for frv?
The four __KM_xxx slots are special and must not be committed to this stack. They're used by assembly code directly for certain tasks, such as maintaining the TLB-lookup state. Your changes are guaranteed to break MMU-mode FRV.
That said, there's no reason these four slots *have* to be administered through kmap_atomic(). A kmap_frv() could be made instead for them.
David
Peter Zijlstra
maintainerQuoted 3 lines · David Howells
Do we have any code that uses two slots and then calls more code that ultimately requires further slots? In other words, do we need more than two slots?
I can think of code that does a lot more than that, suppose you have both KM_USER[01], get an interrupt that takes KM_IRQ[01], take an NMI that takes KM_NMI.
Maybe we can stack the SOFTIRQ ones in as well ;-)
David Howells
maintainerQuoted 3 lines · Peter Zijlstra
I can think of code that does a lot more than that, suppose you have both KM_USER[01], get an interrupt that takes KM_IRQ[01], take an NMI that takes
KM_NMI.
But whilst the interrupt might want to use two slots, it's probably a bug for it to want to access the mappings set up by whatever called KM_USER[01] - so it can probably reuse those slots, provided it puts them back again.
Similarly for NMI taking KM_NMI - it probably shouldn't be attempting to access the mappings set up by the normal mode or the interrupt mode - in which case, why can't it reuse those slots?
Quoted 1 line
Maybe we can stack the SOFTIRQ ones in as well ;-)
Ditto.
David
Peter Zijlstra
maintainerJust to make it clear, the stack design gets rid of crap like:
-#define __KM_PTE \ - (in_nmi() ? KM_NMI_PTE : \ - in_irq() ? KM_IRQ_PTE : \
- KM_PTE0)
and
-static inline enum km_type crypto_kmap_type(int out) -{
- enum km_type type;
-
- if (in_softirq())
- type = out * (KM_SOFTIRQ1 - KM_SOFTIRQ0) + KM_SOFTIRQ0;
- else
- type = out * (KM_USER1 - KM_USER0) + KM_USER0;
-
- return type;
-}
include/linux/highmem.h static inline void flush_kernel_dcache_p · line 28
Quoted 5 lines · Peter Zijlstra
include/linux/highmem.h +2
-2- -static inline void debug_kmap_atomic (enum km_type type) +static inline int kmap_atomic_push_idx (void ) { +int idx =__get_cpu_var (__kmap_atomic_depth)++;
The counter needs to be of local atomic type. Otherwise kmap_atomic cannot be done from interrupts/nmis, which is unfortunately occasionally needed.
-Andi
Signature · 1 line
[email protected] -- Speaking for myself only.
Quoted 9 lines · Andi Kleen
include/linux/highmem.h static inline void flush_kernel_dcache_p · line 28
Quoted 5 lines · Peter Zijlstra
include/linux/highmem.h +2
-2- -static inline void debug_kmap_atomic (enum km_type type) +static inline int kmap_atomic_push_idx (void ) { +int idx =__get_cpu_var (__kmap_atomic_depth)++;The counter needs to be of local atomic type. Otherwise
kmap_atomiccannot be done from interrupts/nmis, which is unfortunately occasionally needed.
I thought so too on lookin gat it initially, but it's not actually true.
It's both IRQ and NMI safe as-is, for a very simple reason: any interrupts that happen will always undo whatever changes they did. So even with a totally non-atomic "load + increment + store" model, it really doesn't matter if you get an interrupt or an NMI anywhere in the sequence, because by the time the interrupt returns, it will have undone any changes it did.
So as long as it's per-cpu (which it is) and non-preemptible (which it also is, thanks to kmap_atomic() doing the whole "disable_mm_fault()" thing or whatever), it's all fine.
Btw, this is not some new thing. It's exactly the same logic we rely on for other counts like the preempt-count etc.
Linus
Quoted 14 lines · Linus Torvalds
Quoted 9 lines · Andi Kleen
include/linux/highmem.h static inline void flush_kernel_dcache_p · line 28
Quoted 5 lines · Peter Zijlstra
include/linux/highmem.h +2
-2- -static inline void debug_kmap_atomic (enum km_type type) +static inline int kmap_atomic_push_idx (void ) { +int idx =__get_cpu_var (__kmap_atomic_depth)++;The counter needs to be of local atomic type. Otherwise
kmap_atomiccannot be done from interrupts/nmis, which is unfortunately occasionally needed.
Quoted 3 lines
I thought so too on lookin gat it initially, but it's not actually true.
It's both IRQ and NMI safe as-is, for a very simple reason: any interrupts
Good point, thanks.
I was thinking of CPU migration in interrupt cases, but even there it should be ok in mainline.
I suppose it's not true for the preempt-rt folks (who can migrate CPUs at any time), so it might be still more friendly to handle it for them though.
-Andi
Signature · 1 line
[email protected] -- Speaking for myself only.