Skip to contentSkip to conversation

DG1 + LMEM enabling

RFC PATCH In review Matthew Auld 10 participants 212 messages Nov 27, 2020 – Jan 12, 2021 Page 1 of 2

Subsystems (5)
lore
Share & tools
lore
#1 lore raw

RFC PATCH 1/162 drm/i915/selftest: also consider non-contiguous objects

In reply to Matthew Auld

In igt_ppgtt_sanity_check we should also exercise the non-contiguous option for LMEM, since this will give us slightly different sg layouts and alignment.

---
 drivers/gpu/drm/i915/gem/selftests/huge_pages.c | 1 +
 1 file changed, 1 insertion(+)
drivers/gpu/drm/i915/gem/selftests/huge_pages.c +1 -0
index 1f35e71429b4..0bf93947d89d 100644
@@ -1333,6 +1333,7 @@ static int igt_ppgtt_sanity_check(void *arg)
 		unsigned int flags;
 	} backends[] = {
 		{ igt_create_system, 0,                        },
+		{ igt_create_local,  0,                        },
 		{ igt_create_local,  I915_BO_ALLOC_CONTIGUOUS, },
 	};
 	struct {
Signature · 6 lines
2.26.2

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel
#0 lore raw

cover DG1 + LMEM enabling

This series includes a version of Maarten' series[1], which converts more of the driver locking over to dma-resv. On top of this we now implement things like LMEM eviction, which has a dependency on this new locking design.

In terms of new uAPI we have gem_create_ext, which offers extensions support for gem_create. For now the only extension we add is giving userspace the ability to optionally provide a priority list of potential placements for the object. The other bit of new uAPI is the query interface for memory regions, which describes the supported memory regions for the device. What this reports can then be fed into gem_create_ext to specify where an object might reside, like device local memory. Note that the class/instance complexity in the uAPI is not very relevant for DG1, but is in preparation for the Xe HP multi-tile architecture with multiple memory regions.

The series still includes relocation support, but that's purely for CI, until we have completed all the IGT rework[2] and so will not be merged. Likewise for pread/pwrite, which will also be dropped from DG1+.

[1] https://patchwork.freedesktop.org/series/82337/
[2] https://patchwork.freedesktop.org/series/82954/

Abdiel Janulgue (3):
drm/i915/query: Expose memory regions through the query uAPI drm/i915: Provide a way to disable PCIe relaxed write ordering drm/i915: Reintroduce mem->reserved

Animesh Manna (2):
drm/i915/lmem: reset the lmem buffer created by fbdev drm/i915/dsb: Enable lmem for dsb

Anshuman Gupta (1):
drm/i915/oprom: Basic sanitization

Anusha Srivatsa (1):
drm/i915/lmem: Bypass aperture when lmem is available

Bommu Krishnaiah (1):
drm/i915/gem: Update shmem available memory

CQ Tang (13):
drm/i915/dg1: Fix occasional migration error drm/i915: i915 returns -EBUSY on thread contention drm/i915: setup GPU device lmem region
drm/i915: Fix object page offset within a region drm/i915: add i915_gem_object_is_devmem() function drm/i915: finish memory region support for stolen objects. drm/i915: Create stolen memory region from local memory drm/i915/dg1: intel_memory_region_evict() changes for eviction drm/i915/dg1: i915_gem_object_memcpy(..) infrastructure drm/i915/dg1: Eviction logic
drm/i915/dg1: Add enable_eviction modparam drm/i915/dg1: Add lmem_size modparam
drm/i915: need consider system BO snoop for dgfx

Chris Wilson (2):
drm/i915/gt: Move move context layout registers and offsets to
lrc_reg.h
drm/i915/gt: Rename lrc.c to execlists_submission.c

Clint Taylor (3):
drm/i915/dg1: Read OPROM via SPI controller drm/i915/dg1: Compute MEM Bandwidth using MCHBAR drm/i915/dg1: Double memory bandwidth available

Daniele Ceraolo Spurio (5):
drm/i915: split gen8+ flush and bb_start emission functions to their
own file
drm/i915: split wa_bb code to its own file
drm/i915: Make intel_init_workaround_bb more compatible with ww
locking.
drm/i915/guc: put all guc objects in lmem when available drm/i915: WA for zero memory channel

Imre Deak (1):
drm/i915/dg1: Reserve first 1MB of local memory

Kui Wen (1):
drm/i915/dg1: Do not check r->sgt.pfn for NULL

Lucas De Marchi (2):
drm/i915: move eviction to prepare hook
drm/i915/dg1: allow pci to auto probe

Maarten Lankhorst (60):
drm/i915: Pin timeline map after first timeline pin, v5. drm/i915: Move cmd parser pinning to execbuffer
drm/i915: Add missing -EDEADLK handling to execbuf pinning, v2. drm/i915: Ensure we hold the object mutex in pin correctly v2 drm/i915: Add gem object locking to madvise.
drm/i915: Move HAS_STRUCT_PAGE to obj->flags
drm/i915: Rework struct phys attachment handling drm/i915: Convert i915_gem_object_attach_phys() to ww locking, v2. drm/i915: make lockdep slightly happier about execbuf. drm/i915: Disable userptr pread/pwrite support.
drm/i915: No longer allow exporting userptr through dma-buf drm/i915: Reject more ioctls for userptr
drm/i915: Reject UNSYNCHRONIZED for userptr, v2. drm/i915: Make compilation of userptr code depend on MMU_NOTIFIER. drm/i915: Fix userptr so we do not have to worry about obj->mm.lock,

    v5.
  drm/i915: Flatten obj->mm.lock
  drm/i915: Populate logical context during first pin.
  drm/i915: Make ring submission compatible with obj->mm.lock removal,
    v2.

drm/i915: Handle ww locking in init_status_page
drm/i915: Rework clflush to work correctly without obj->mm.lock. drm/i915: Pass ww ctx to intel_pin_to_display_plane drm/i915: Add object locking to vm_fault_cpu
drm/i915: Move pinning to inside engine_wa_list_verify() drm/i915: Take reservation lock around i915_vma_pin. drm/i915: Make __engine_unpark() compatible with ww locking v2 drm/i915: Take obj lock around set_domain ioctl
drm/i915: Defer pin calls in buffer pool until first use by caller. drm/i915: Fix pread/pwrite to work with new locking rules. drm/i915: Fix workarounds selftest, part 1
drm/i915: Add igt_spinner_pin() to allow for ww locking around
spinner.
drm/i915: Add ww locking around vm_access()
drm/i915: Increase ww locking for perf.
drm/i915: Lock ww in ucode objects correctly
drm/i915: Add ww locking to dma-buf ops.
drm/i915: Add missing ww lock in intel_dsb_prepare. drm/i915: Fix ww locking in shmem_create_from_object drm/i915: Use a single page table lock for each gtt. drm/i915/selftests: Prepare huge_pages testcases for obj->mm.lock
removal.
drm/i915/selftests: Prepare client blit for obj->mm.lock removal. drm/i915/selftests: Prepare coherency tests for obj->mm.lock removal. drm/i915/selftests: Prepare context tests for obj->mm.lock removal. drm/i915/selftests: Prepare dma-buf tests for obj->mm.lock removal. drm/i915/selftests: Prepare execbuf tests for obj->mm.lock removal. drm/i915/selftests: Prepare mman testcases for obj->mm.lock removal. drm/i915/selftests: Prepare object tests for obj->mm.lock removal. drm/i915/selftests: Prepare object blit tests for obj->mm.lock
removal.
drm/i915/selftests: Prepare igt_gem_utils for obj->mm.lock removal drm/i915/selftests: Prepare context selftest for obj->mm.lock removal drm/i915/selftests: Prepare hangcheck for obj->mm.lock removal drm/i915/selftests: Prepare execlists for obj->mm.lock removal drm/i915/selftests: Prepare mocs tests for obj->mm.lock removal drm/i915/selftests: Prepare ring submission for obj->mm.lock removal drm/i915/selftests: Prepare timeline tests for obj->mm.lock removal drm/i915/selftests: Prepare i915_request tests for obj->mm.lock
removal
drm/i915/selftests: Prepare memory region tests for obj->mm.lock
removal
drm/i915/selftests: Prepare cs engine tests for obj->mm.lock removal drm/i915/selftests: Prepare gtt tests for obj->mm.lock removal drm/i915: Finally remove obj->mm.lock.
drm/i915: Keep userpointer bindings if seqcount is unchanged, v2. drm/i915: Implement eviction locking v2

Matt Roper (1):
drm/i915/lmem: Fail driver init if LMEM training failed

Matthew Auld (19):
drm/i915/selftest: also consider non-contiguous objects drm/i915/selftest: assert we get 2M GTT pages drm/i915/selftest: handle local-memory in perf_memcpy
HAX drm/i915/lmem: support CPU relocations
HAX drm/i915/lmem: support pread and pwrite
drm/i915: introduce kernel blitter_context drm/i915/region: support basic eviction
drm/i915: support basic object migration
drm/i915/uapi: introduce drm_i915_gem_create_ext
drm/i915: setup the LMEM region
drm/i915/gtt: map the PD up front
drm/i915/gtt/dgfx: place the PD in LMEM
drm/i915/gtt: make flushing conditional
drm/i915/gtt/dg1: add PTE_LM plumbing for PPGTT drm/i915/gtt/dg1: add PTE_LM plumbing for GGTT
drm/i915: allocate context from LMEM
drm/i915: move engine scratch to LMEM
drm/i915/lmem: support optional CPU clearing for special internal use drm/i915: drop fake lmem

Michael J. Ruhl (2):
drm/i915/dmabuf: Disallow LMEM objects from dma-buf drm/i915/dg1: Introduce dmabuf mmap to LMEM

Michel Thierry (2):
drm/i915/lmem: allocate cmd ring in lmem
drm/i915/lmem: allocate HWSP in lmem

Mohammed Khajapasha (2):
drm/i915/fbdev: Use lmem physical addresses for fb_mmap() on discrete drm/i915: Return error value when bo not in LMEM for discrete

Prathap Kumar Valsan (2):
drm/i915: Store gt in memory region
drm/i915/pm: suspend and restore ppgtt mapping

Ramalingam C (6):
drm/i915: define intel_partial_pages_for_sg_table drm/i915: create and destroy dummy vma
drm/i915: blt copy between objs using pre-created vma windows drm/i915: window_blt_copy is used for swapin and swapout drm/i915: Lmem eviction statistics by category drm/i915/gem/selftest: test and measure window based blt cpy

Stuart Summers (1):
drm/i915: Allow non-uniform subslices in gen12+

Sudeep Dutt (2):
drm/i915/dg1: Track swap in/out stats via debugfs drm/i915/dg1: Measure swap in/out timing stats

Thomas Hellström (15):
HAX drm/i915: Work around the selftest timeline lock splat workaround drm/i915: Introduce drm_i915_lock_isolated
drm/i915: Lock hwsp objects isolated for pinning at create time drm/i915: Prepare for obj->mm.lock removal
drm/i915: Avoid some false positives in assert_object_held() drm/i915: Reference contending lock objects
drm/i915: Break out dma_resv ww locking utilities to separate files drm/i915: Introduce a for_i915_gem_ww(){}
drm/i915: Untangle the vma pages_mutex
drm/i915: Add blit functions that can be called from within a WW
transaction
drm/i915: Delay publishing objects on the eviction lists drm/i915: Perform execbuffer object locking as a separate step drm/i915: Support ww eviction
drm/i915: Use a ww transaction in the fault handler drm/i915: Use a ww transaction in i915_gem_object_pin_map_unlocked()

Tvrtko Ursulin (4):
drm/i915/dg1: Eliminate eviction mutex drm/i915/dg1: Keep engine awake across whole blit drm/i915/dg1: Add dedicated context for blitter eviction drm/i915: Improve accuracy of eviction stats

Venkata Ramana Nayana (8):
drm/i915: suspend/resume eviction
drm/i915: Reset blitter context when unpark engine drm/i915/gt: Allocate default ctx objects in SMEM drm/i915: suspend/resume enable blitter eviction drm/i915: suspend/resume handling of perma-pinned objects drm/i915: Support ww locks in suspend/resume drm/i915/dg1: Fix mapping type for default state object drm/i915/dg1: Fix GPU hang due to shmemfs page drop

Venkata Sandeep Dhanalakota (2):
drm/i915: Update the helper to set correct mapping drm/i915/lmem: Limit block size to 4G

Zbigniew Kempczyński (1):
drm/i915: Distinction of memory regions

 drivers/gpu/drm/i915/Kconfig.debug            |  11 +
 drivers/gpu/drm/i915/Makefile                 |   7 +-
 drivers/gpu/drm/i915/display/intel_bios.c     |  75 +-
 drivers/gpu/drm/i915/display/intel_bw.c       |  64 +-
 drivers/gpu/drm/i915/display/intel_display.c  |  80 +-
 drivers/gpu/drm/i915/display/intel_display.h  |   2 +-
 drivers/gpu/drm/i915/display/intel_dsb.c      |   9 +-
 drivers/gpu/drm/i915/display/intel_fbc.c      |  20 +-
 drivers/gpu/drm/i915/display/intel_fbdev.c    |  54 +-
 drivers/gpu/drm/i915/display/intel_opregion.c | 169 ++++
 drivers/gpu/drm/i915/display/intel_opregion.h |  31 +-
 drivers/gpu/drm/i915/display/intel_overlay.c  |  34 +-
 drivers/gpu/drm/i915/gem/i915_gem_clflush.c   |  15 +-
 drivers/gpu/drm/i915/gem/i915_gem_context.c   |   1 +
 drivers/gpu/drm/i915/gem/i915_gem_create.c    | 398 ++++++++
 drivers/gpu/drm/i915/gem/i915_gem_dmabuf.c    | 123 ++-
 drivers/gpu/drm/i915/gem/i915_gem_domain.c    |  52 +-
 .../gpu/drm/i915/gem/i915_gem_execbuffer.c    | 302 +++++-
 drivers/gpu/drm/i915/gem/i915_gem_fence.c     |  95 --
 drivers/gpu/drm/i915/gem/i915_gem_internal.c  |   6 +-
 drivers/gpu/drm/i915/gem/i915_gem_lmem.c      | 254 ++++-
 drivers/gpu/drm/i915/gem/i915_gem_lmem.h      |  22 +
 drivers/gpu/drm/i915/gem/i915_gem_mman.c      | 187 ++--
 drivers/gpu/drm/i915/gem/i915_gem_mman.h      |  11 +
 drivers/gpu/drm/i915/gem/i915_gem_object.c    | 711 +++++++++++++-
 drivers/gpu/drm/i915/gem/i915_gem_object.h    | 206 +++-
 .../gpu/drm/i915/gem/i915_gem_object_blt.c    | 101 +-
 .../gpu/drm/i915/gem/i915_gem_object_blt.h    |  10 +
 .../gpu/drm/i915/gem/i915_gem_object_types.h  |  46 +-
 drivers/gpu/drm/i915/gem/i915_gem_pages.c     | 140 ++-
 drivers/gpu/drm/i915/gem/i915_gem_phys.c      | 110 +--
 drivers/gpu/drm/i915/gem/i915_gem_region.c    | 234 ++++-
 drivers/gpu/drm/i915/gem/i915_gem_region.h    |   3 +-
 drivers/gpu/drm/i915/gem/i915_gem_shmem.c     |  46 +-
 drivers/gpu/drm/i915/gem/i915_gem_shrinker.c  |  52 +-
 drivers/gpu/drm/i915/gem/i915_gem_shrinker.h  |   6 +-
 drivers/gpu/drm/i915/gem/i915_gem_stolen.c    | 247 +++--
 drivers/gpu/drm/i915/gem/i915_gem_stolen.h    |  10 +-
 drivers/gpu/drm/i915/gem/i915_gem_tiling.c    |   2 -
 drivers/gpu/drm/i915/gem/i915_gem_userptr.c   | 870 ++++++-----------
 .../drm/i915/gem/selftests/huge_gem_object.c  |   4 +-
 .../gpu/drm/i915/gem/selftests/huge_pages.c   |  60 +-
 .../i915/gem/selftests/i915_gem_client_blt.c  |   8 +-
 .../i915/gem/selftests/i915_gem_coherency.c   |  18 +-
 .../drm/i915/gem/selftests/i915_gem_context.c |  21 +-
 .../drm/i915/gem/selftests/i915_gem_dmabuf.c  |   2 +-
 .../i915/gem/selftests/i915_gem_execbuffer.c  |   2 +-
 .../drm/i915/gem/selftests/i915_gem_mman.c    |  33 +-
 .../drm/i915/gem/selftests/i915_gem_object.c  |   2 +-
 .../i915/gem/selftests/i915_gem_object_blt.c  | 172 +++-
 .../drm/i915/gem/selftests/i915_gem_phys.c    |  10 +-
 .../drm/i915/gem/selftests/igt_gem_utils.c    |   2 +-
 drivers/gpu/drm/i915/gt/gen6_ppgtt.c          |  11 +-
 drivers/gpu/drm/i915/gt/gen8_engine_cs.c      | 393 ++++++++
 drivers/gpu/drm/i915/gt/gen8_engine_cs.h      |  26 +
 drivers/gpu/drm/i915/gt/gen8_ppgtt.c          |  91 +-
 drivers/gpu/drm/i915/gt/gen8_ppgtt.h          |   2 +
 drivers/gpu/drm/i915/gt/intel_context.c       |   3 +-
 drivers/gpu/drm/i915/gt/intel_context.h       |   2 +
 drivers/gpu/drm/i915/gt/intel_context_sseu.c  |   2 +-
 drivers/gpu/drm/i915/gt/intel_context_types.h |  13 +-
 drivers/gpu/drm/i915/gt/intel_engine.h        |   4 +
 drivers/gpu/drm/i915/gt/intel_engine_cs.c     | 125 ++-
 drivers/gpu/drm/i915/gt/intel_engine_pm.c     |  14 +-
 drivers/gpu/drm/i915/gt/intel_engine_types.h  |   2 +
 .../drm/i915/gt/intel_engine_workaround_bb.c  | 364 +++++++
 .../drm/i915/gt/intel_engine_workaround_bb.h  |  14 +
 ...tel_lrc.c => intel_execlists_submission.c} | 899 +++---------------
 .../drm/i915/gt/intel_execlists_submission.h  |  66 ++
 drivers/gpu/drm/i915/gt/intel_ggtt.c          |  93 +-
 drivers/gpu/drm/i915/gt/intel_gt.c            |  14 +-
 .../gpu/drm/i915/gt/intel_gt_buffer_pool.c    |  47 +-
 .../gpu/drm/i915/gt/intel_gt_buffer_pool.h    |   5 +
 .../drm/i915/gt/intel_gt_buffer_pool_types.h  |   1 +
 drivers/gpu/drm/i915/gt/intel_gt_irq.c        |   1 +
 drivers/gpu/drm/i915/gt/intel_gtt.c           | 105 +-
 drivers/gpu/drm/i915/gt/intel_gtt.h           |  23 +-
 drivers/gpu/drm/i915/gt/intel_lrc.h           | 128 ---
 drivers/gpu/drm/i915/gt/intel_lrc_reg.h       |  39 +
 drivers/gpu/drm/i915/gt/intel_mocs.c          |   2 +-
 drivers/gpu/drm/i915/gt/intel_ppgtt.c         |  20 +-
 drivers/gpu/drm/i915/gt/intel_renderstate.c   |   2 +-
 drivers/gpu/drm/i915/gt/intel_renderstate.h   |   1 +
 drivers/gpu/drm/i915/gt/intel_ring.c          |  24 +-
 .../gpu/drm/i915/gt/intel_ring_submission.c   | 184 ++--
 drivers/gpu/drm/i915/gt/intel_sseu.c          |   6 +-
 drivers/gpu/drm/i915/gt/intel_timeline.c      | 121 ++-
 drivers/gpu/drm/i915/gt/intel_timeline.h      |   1 +
 .../gpu/drm/i915/gt/intel_timeline_types.h    |   1 +
 drivers/gpu/drm/i915/gt/intel_workarounds.c   |  24 +-
 drivers/gpu/drm/i915/gt/mock_engine.c         |  24 +-
 drivers/gpu/drm/i915/gt/selftest_context.c    |   5 +-
 drivers/gpu/drm/i915/gt/selftest_engine_cs.c  |   4 +-
 .../{selftest_lrc.c => selftest_execlists.c}  |  37 +-
 drivers/gpu/drm/i915/gt/selftest_hangcheck.c  |   8 +-
 drivers/gpu/drm/i915/gt/selftest_mocs.c       |   2 +-
 drivers/gpu/drm/i915/gt/selftest_reset.c      |   5 +-
 .../drm/i915/gt/selftest_ring_submission.c    |   4 +-
 drivers/gpu/drm/i915/gt/selftest_timeline.c   | 100 +-
 .../gpu/drm/i915/gt/selftest_workarounds.c    | 101 +-
 drivers/gpu/drm/i915/gt/shmem_utils.c         |  11 +-
 drivers/gpu/drm/i915/gt/uc/intel_guc.c        |  13 +-
 drivers/gpu/drm/i915/gt/uc/intel_guc_ads.c    |   1 +
 drivers/gpu/drm/i915/gt/uc/intel_guc_fw.c     |  11 +-
 drivers/gpu/drm/i915/gt/uc/intel_guc_log.c    |   4 +-
 .../gpu/drm/i915/gt/uc/intel_guc_submission.c |   1 +
 drivers/gpu/drm/i915/gt/uc/intel_huc.c        |  18 +-
 drivers/gpu/drm/i915/gt/uc/intel_uc_fw.c      |  37 +-
 drivers/gpu/drm/i915/gvt/dmabuf.c             |   2 +-
 drivers/gpu/drm/i915/gvt/mmio_context.h       |   2 +
 drivers/gpu/drm/i915/gvt/scheduler.c          |   1 +
 drivers/gpu/drm/i915/i915_active.c            |  20 +-
 drivers/gpu/drm/i915/i915_cmd_parser.c        | 104 +-
 drivers/gpu/drm/i915/i915_debugfs.c           |  42 +-
 drivers/gpu/drm/i915/i915_drv.c               | 277 +++++-
 drivers/gpu/drm/i915/i915_drv.h               |  57 +-
 drivers/gpu/drm/i915/i915_gem.c               | 418 +++-----
 drivers/gpu/drm/i915/i915_gem.h               |  12 -
 drivers/gpu/drm/i915/i915_gem_gtt.c           |   2 +-
 drivers/gpu/drm/i915/i915_gem_ww.c            |  93 ++
 drivers/gpu/drm/i915/i915_gem_ww.h            |  53 ++
 drivers/gpu/drm/i915/i915_gpu_error.c         |   4 +-
 drivers/gpu/drm/i915/i915_memcpy.c            |   2 +-
 drivers/gpu/drm/i915/i915_memcpy.h            |   2 +-
 drivers/gpu/drm/i915/i915_mm.c                |   2 +-
 drivers/gpu/drm/i915/i915_params.c            |  11 +-
 drivers/gpu/drm/i915/i915_params.h            |   3 +-
 drivers/gpu/drm/i915/i915_pci.c               |   5 +-
 drivers/gpu/drm/i915/i915_perf.c              |  57 +-
 drivers/gpu/drm/i915/i915_query.c             |  62 ++
 drivers/gpu/drm/i915/i915_reg.h               |  17 +
 drivers/gpu/drm/i915/i915_selftest.h          |   2 +
 drivers/gpu/drm/i915/i915_vma.c               | 154 ++-
 drivers/gpu/drm/i915/i915_vma.h               |  28 +-
 drivers/gpu/drm/i915/intel_memory_region.c    | 229 ++++-
 drivers/gpu/drm/i915/intel_memory_region.h    |  53 +-
 drivers/gpu/drm/i915/intel_region_lmem.c      | 168 ++--
 drivers/gpu/drm/i915/intel_region_lmem.h      |   3 +-
 drivers/gpu/drm/i915/intel_uncore.c           |  12 +
 drivers/gpu/drm/i915/selftests/i915_gem_gtt.c | 100 +-
 .../drm/i915/selftests/i915_live_selftests.h  |   1 +
 drivers/gpu/drm/i915/selftests/i915_perf.c    |   3 +-
 drivers/gpu/drm/i915/selftests/i915_request.c |  10 +-
 drivers/gpu/drm/i915/selftests/igt_spinner.c  | 136 ++-
 drivers/gpu/drm/i915/selftests/igt_spinner.h  |   5 +
 .../drm/i915/selftests/intel_memory_region.c  | 442 ++++++++-
 drivers/gpu/drm/i915/selftests/mock_region.c  |   4 +-
 include/uapi/drm/i915_drm.h                   | 118 +++
 148 files changed, 7813 insertions(+), 3312 deletions(-)
 create mode 100644 drivers/gpu/drm/i915/gem/i915_gem_create.c
 delete mode 100644 drivers/gpu/drm/i915/gem/i915_gem_fence.c
 create mode 100644 drivers/gpu/drm/i915/gt/gen8_engine_cs.c
 create mode 100644 drivers/gpu/drm/i915/gt/gen8_engine_cs.h
 create mode 100644 drivers/gpu/drm/i915/gt/intel_engine_workaround_bb.c
 create mode 100644 drivers/gpu/drm/i915/gt/intel_engine_workaround_bb.h
 rename drivers/gpu/drm/i915/gt/{intel_lrc.c => intel_execlists_submission.c} (87%)
 create mode 100644 drivers/gpu/drm/i915/gt/intel_execlists_submission.h
 delete mode 100644 drivers/gpu/drm/i915/gt/intel_lrc.h
 rename drivers/gpu/drm/i915/gt/{selftest_lrc.c => selftest_execlists.c} (99%)
 create mode 100644 drivers/gpu/drm/i915/i915_gem_ww.c
 create mode 100644 drivers/gpu/drm/i915/i915_gem_ww.h
Signature · 6 lines
2.26.2

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
#3 lore raw

RFC PATCH 2/162 drm/i915/selftest: assert we get 2M GTT pages

For the LMEM case if we have suitable alignment and 2M physical pages we should always get 2M GTT pages within the constraints of the hugepages selftest. If we don't then something might be wrong in our construction of the backing pages.

---
 .../gpu/drm/i915/gem/selftests/huge_pages.c   | 21 +++++++++++++++++++
 1 file changed, 21 insertions(+)
drivers/gpu/drm/i915/gem/selftests/huge_pages.c +21 -0
index 0bf93947d89d..77a13527a7e6 100644
@@ -368,6 +368,27 @@ static int igt_check_page_sizes(struct i915_vma *vma)
 		err = -EINVAL;
 	}
 
+
+	/*
+	 * The dma-api is like a box of chocolates when it comes to the
+	 * alignment of dma addresses, however for LMEM we have total control
+	 * and so can guarantee alignment, likewise when we allocate our blocks
+	 * they should appear in descending order, and if we know that we align
+	 * to the largest page size for the GTT address, we should be able to
+	 * assert that if we see 2M physical pages then we should also get 2M
+	 * GTT pages. If we don't then something might be wrong in our
+	 * construction of the backing pages.
+	 */
+	if (i915_gem_object_is_lmem(obj) &&
+	    IS_ALIGNED(vma->node.start, SZ_2M) &&
+	    vma->page_sizes.sg & SZ_2M &&
+	    vma->page_sizes.gtt < SZ_2M) {
+		pr_err("gtt pages mismatch for LMEM, expected 2M GTT pages, sg(%u), gtt(%u)\n",
+		       vma->page_sizes.sg, vma->page_sizes.gtt);
+		err = -EINVAL;
+	}
+
+
 	if (obj->mm.page_sizes.gtt) {
 		pr_err("obj->page_sizes.gtt(%u) should never be set\n",
 		       obj->mm.page_sizes.gtt);
Signature · 6 lines
2.26.2

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
#4 lore raw

RFC PATCH 3/162 drm/i915/selftest: handle local-memory in perf_memcpy

In reply to Matthew Auld

We currently only support WC when mapping device local-memory, which is returned as a generic -ENOMEM when mapping the object with an unsupported type. Try to handle that case also, although it's starting to get pretty ugly in there.

---
 drivers/gpu/drm/i915/selftests/intel_memory_region.c | 2 ++
 1 file changed, 2 insertions(+)
drivers/gpu/drm/i915/selftests/intel_memory_region.c +2 -0
index 0aeba8e3af28..27389fb19951 100644
@@ -681,6 +681,8 @@ create_region_for_mapping(struct intel_memory_region *mr, u64 size, u32 type,
 		i915_gem_object_put(obj);
 		if (PTR_ERR(addr) == -ENXIO)
 			return ERR_PTR(-ENODEV);
+		if (PTR_ERR(addr) == -ENOMEM) /* WB local-memory */
+			return ERR_PTR(-ENODEV);
 		return addr;
 	}
 
Signature · 6 lines
2.26.2

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel
#7 lore raw

RFC PATCH 5/162 drm/i915/gt: Rename lrc.c to execlists_submission.c

In reply to Matthew Auld

From: Chris Wilson <[email protected]>

We want to separate the utility functions for controlling the logical ring context from the execlists submission mechanism (which is an overgrown scheduler).

This is similar to Daniele's work to split up the files, but being selfish I wanted to base it after my own changes to intel_lrc.c petered out.

---
 drivers/gpu/drm/i915/Makefile                 |  2 +-
 drivers/gpu/drm/i915/gem/i915_gem_context.c   |  1 +
 drivers/gpu/drm/i915/gt/intel_context_sseu.c  |  2 +-
 drivers/gpu/drm/i915/gt/intel_engine_cs.c     |  1 +
 ...tel_lrc.c => intel_execlists_submission.c} | 30 ++----------------
 ...tel_lrc.h => intel_execlists_submission.h} | 31 +++----------------
 drivers/gpu/drm/i915/gt/intel_mocs.c          |  2 +-
 .../{selftest_lrc.c => selftest_execlists.c}  |  0
 drivers/gpu/drm/i915/gt/uc/intel_guc_ads.c    |  1 +
 .../gpu/drm/i915/gt/uc/intel_guc_submission.c |  1 +
 drivers/gpu/drm/i915/gvt/scheduler.c          |  1 +
 drivers/gpu/drm/i915/i915_drv.h               |  1 -
 drivers/gpu/drm/i915/i915_perf.c              |  1 +
 13 files changed, 16 insertions(+), 58 deletions(-)
 rename drivers/gpu/drm/i915/gt/{intel_lrc.c => intel_execlists_submission.c} (99%)
 rename drivers/gpu/drm/i915/gt/{intel_lrc.h => intel_execlists_submission.h} (57%)
 rename drivers/gpu/drm/i915/gt/{selftest_lrc.c => selftest_execlists.c} (100%)
drivers/gpu/drm/i915/Makefile +1 -1
index e5574e506a5c..aedbd8f52be8 100644
@@ -91,6 +91,7 @@ gt-y += \
 	gt/intel_engine_heartbeat.o \
 	gt/intel_engine_pm.o \
 	gt/intel_engine_user.o \
+	gt/intel_execlists_submission.o \
 	gt/intel_ggtt.o \
 	gt/intel_ggtt_fencing.o \
 	gt/intel_gt.o \
@@ -102,7 +103,6 @@ gt-y += \
 	gt/intel_gt_requests.o \
 	gt/intel_gtt.o \
 	gt/intel_llc.o \
-	gt/intel_lrc.o \
 	gt/intel_mocs.o \
 	gt/intel_ppgtt.o \
 	gt/intel_rc6.o \
drivers/gpu/drm/i915/gem/i915_gem_context.c +1 -0
index a6299da64de4..ad136d009d9b 100644
@@ -72,6 +72,7 @@
 #include "gt/intel_context_param.h"
 #include "gt/intel_engine_heartbeat.h"
 #include "gt/intel_engine_user.h"
+#include "gt/intel_execlists_submission.h" /* virtual_engine */
 #include "gt/intel_ring.h"
 
 #include "i915_gem_context.h"
drivers/gpu/drm/i915/gt/intel_context_sseu.c +1 -1
index b9c8163978a3..5f94b44022dc 100644
@@ -8,7 +8,7 @@
 #include "intel_context.h"
 #include "intel_engine_pm.h"
 #include "intel_gpu_commands.h"
-#include "intel_lrc.h"
+#include "intel_execlists_submission.h"
 #include "intel_lrc_reg.h"
 #include "intel_ring.h"
 #include "intel_sseu.h"
drivers/gpu/drm/i915/gt/intel_engine_cs.c +1 -0
index 02ea16b29c9f..97ceaf7116e8 100644
@@ -33,6 +33,7 @@
 #include "intel_engine.h"
 #include "intel_engine_pm.h"
 #include "intel_engine_user.h"
+#include "intel_execlists_submission.h"
 #include "intel_gt.h"
 #include "intel_gt_requests.h"
 #include "intel_gt_pm.h"
drivers/gpu/drm/i915/gt/intel_execlists_submission.c renamed from drivers/gpu/drm/i915/gt/intel_lrc.c +3 -27
similarity index 99%
rename from drivers/gpu/drm/i915/gt/intel_lrc.c
rename to drivers/gpu/drm/i915/gt/intel_execlists_submission.c
index 43703efb36d1..fc330233ea20 100644
@@ -1,31 +1,6 @@
+// SPDX-License-Identifier: MIT
 /*
  * Copyright © 2014 Intel Corporation
- *
- * Permission is hereby granted, free of charge, to any person obtaining a
- * copy of this software and associated documentation files (the "Software"),
- * to deal in the Software without restriction, including without limitation
- * the rights to use, copy, modify, merge, publish, distribute, sublicense,
- * and/or sell copies of the Software, and to permit persons to whom the
- * Software is furnished to do so, subject to the following conditions:
- *
- * The above copyright notice and this permission notice (including the next
- * paragraph) shall be included in all copies or substantial portions of the
- * Software.
- *
- * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
- * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
- * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.  IN NO EVENT SHALL
- * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
- * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
- * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
- * IN THE SOFTWARE.
- *
- * Authors:
- *    Ben Widawsky <[email protected]>
- *    Michel Thierry <[email protected]>
- *    Thomas Daniel <[email protected]>
- *    Oscar Mateo <[email protected]>
- *
  */
 
 /**
@@ -140,6 +115,7 @@
 #include "intel_breadcrumbs.h"
 #include "intel_context.h"
 #include "intel_engine_pm.h"
+#include "intel_execlists_submission.h"
 #include "intel_gt.h"
 #include "intel_gt_pm.h"
 #include "intel_gt_requests.h"
@@ -6127,5 +6103,5 @@ intel_engine_in_execlists_submission_mode(const struct intel_engine_cs *engine)
 }
 
 #if IS_ENABLED(CONFIG_DRM_I915_SELFTEST)
-#include "selftest_lrc.c"
+#include "selftest_execlists.c"
 #endif
drivers/gpu/drm/i915/gt/intel_execlists_submission.h renamed from drivers/gpu/drm/i915/gt/intel_lrc.h +4 -27
similarity index 57%
rename from drivers/gpu/drm/i915/gt/intel_lrc.h
rename to drivers/gpu/drm/i915/gt/intel_execlists_submission.h
index 9116b46844a2..2c9d7354b42f 100644
@@ -1,35 +1,15 @@
+/* SPDX-License-Identifier: MIT */
 /*
  * Copyright © 2014 Intel Corporation
- *
- * Permission is hereby granted, free of charge, to any person obtaining a
- * copy of this software and associated documentation files (the "Software"),
- * to deal in the Software without restriction, including without limitation
- * the rights to use, copy, modify, merge, publish, distribute, sublicense,
- * and/or sell copies of the Software, and to permit persons to whom the
- * Software is furnished to do so, subject to the following conditions:
- *
- * The above copyright notice and this permission notice (including the next
- * paragraph) shall be included in all copies or substantial portions of the
- * Software.
- *
- * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
- * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
- * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.  IN NO EVENT SHALL
- * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
- * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
- * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
- * DEALINGS IN THE SOFTWARE.
  */
 
-#ifndef _INTEL_LRC_H_
-#define _INTEL_LRC_H_
+#ifndef __INTEL_EXECLISTS_SUBMISSION_H__
+#define __INTEL_EXECLISTS_SUBMISSION_H__
 
 #include <linux/types.h>
 
 struct drm_printer;
 
-struct drm_i915_private;
-struct i915_gem_context;
 struct i915_request;
 struct intel_context;
 struct intel_engine_cs;
@@ -40,9 +20,6 @@ enum {
 	INTEL_CONTEXT_SCHEDULE_PREEMPTED,
 };
 
-/* Logical Rings */
-void intel_logical_ring_cleanup(struct intel_engine_cs *engine);
-
 int intel_execlists_submission_setup(struct intel_engine_cs *engine);
 
 /* Logical Ring Contexts */
@@ -86,4 +63,4 @@ int intel_virtual_engine_attach_bond(struct intel_engine_cs *engine,
 bool
 intel_engine_in_execlists_submission_mode(const struct intel_engine_cs *engine);
 
-#endif /* _INTEL_LRC_H_ */
+#endif /* __INTEL_EXECLISTS_SUBMISSION_H__ */
drivers/gpu/drm/i915/gt/intel_mocs.c +1 -1
index b8d0c32ae9dd..516206007398 100644
@@ -24,8 +24,8 @@
 
 #include "intel_engine.h"
 #include "intel_gt.h"
+#include "intel_lrc_reg.h"
 #include "intel_mocs.h"
-#include "intel_lrc.h"
 #include "intel_ring.h"
 
 /* structures required */
drivers/gpu/drm/i915/gt/selftest_execlists.c renamed from drivers/gpu/drm/i915/gt/selftest_lrc.c +0 -0
similarity index 100%
rename from drivers/gpu/drm/i915/gt/selftest_lrc.c
rename to drivers/gpu/drm/i915/gt/selftest_execlists.c
drivers/gpu/drm/i915/gt/uc/intel_guc_ads.c +1 -0
index 5212ff844292..1a2e4f631763 100644
@@ -3,6 +3,7 @@
  * Copyright © 2014-2019 Intel Corporation
  */
 
+#include "gt/intel_execlists_submission.h" /* lrc layout */
 #include "gt/intel_gt.h"
 #include "intel_guc_ads.h"
 #include "intel_uc.h"
drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c +1 -0
index fdfeb4b9b0f5..8528ab574dbe 100644
@@ -8,6 +8,7 @@
 #include "gem/i915_gem_context.h"
 #include "gt/intel_context.h"
 #include "gt/intel_engine_pm.h"
+#include "gt/intel_execlists_submission.h" /* XXX */
 #include "gt/intel_gt.h"
 #include "gt/intel_gt_pm.h"
 #include "gt/intel_lrc_reg.h"
drivers/gpu/drm/i915/gvt/scheduler.c +1 -0
index aed2ef6466a2..ed30fdde4114 100644
@@ -37,6 +37,7 @@
 
 #include "gem/i915_gem_pm.h"
 #include "gt/intel_context.h"
+#include "gt/intel_execlists_submission.h"
 #include "gt/intel_ring.h"
 
 #include "i915_drv.h"
drivers/gpu/drm/i915/i915_drv.h +0 -1
index 15be8debae54..0f7bf6831633 100644
@@ -79,7 +79,6 @@
 #include "gem/i915_gem_shrinker.h"
 #include "gem/i915_gem_stolen.h"
 
-#include "gt/intel_lrc.h"
 #include "gt/intel_engine.h"
 #include "gt/intel_gt_types.h"
 #include "gt/intel_workarounds.h"
drivers/gpu/drm/i915/i915_perf.c +1 -0
index 3b12c8ff7182..0b300e0d9561 100644
@@ -198,6 +198,7 @@
 #include "gem/i915_gem_context.h"
 #include "gt/intel_engine_pm.h"
 #include "gt/intel_engine_user.h"
+#include "gt/intel_execlists_submission.h"
 #include "gt/intel_gt.h"
 #include "gt/intel_lrc_reg.h"
 #include "gt/intel_ring.h"
Signature · 6 lines
2.26.2

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel
#5 lore raw

RFC PATCH 4/162 drm/i915/gt: Move move context layout registers and offsets to lrc_reg.h

In reply to Matthew Auld

From: Chris Wilson <[email protected]>

Cleanup intel_lrc.h by moving some of the residual common register definitions into intel_lrc_reg.h, prior to rebranding and splitting off the submission backends.

v2: keep the SCHEDULE enum in the old file, since it is specific to the gvt usage of the execlists submission backend (John)

---
 drivers/gpu/drm/i915/gt/intel_engine_cs.c |  2 +-
 drivers/gpu/drm/i915/gt/intel_gt_irq.c    |  1 +
 drivers/gpu/drm/i915/gt/intel_lrc.h       | 39 -----------------------
 drivers/gpu/drm/i915/gt/intel_lrc_reg.h   | 39 +++++++++++++++++++++++
 drivers/gpu/drm/i915/gvt/mmio_context.h   |  2 ++
 5 files changed, 43 insertions(+), 40 deletions(-)
drivers/gpu/drm/i915/gt/intel_engine_cs.c +1 -1
index d4e988b2816a..02ea16b29c9f 100644
@@ -36,7 +36,7 @@
 #include "intel_gt.h"
 #include "intel_gt_requests.h"
 #include "intel_gt_pm.h"
-#include "intel_lrc.h"
+#include "intel_lrc_reg.h"
 #include "intel_reset.h"
 #include "intel_ring.h"
 
drivers/gpu/drm/i915/gt/intel_gt_irq.c +1 -0
index 257063a57101..9830342aa6f4 100644
@@ -11,6 +11,7 @@
 #include "intel_breadcrumbs.h"
 #include "intel_gt.h"
 #include "intel_gt_irq.h"
+#include "intel_lrc_reg.h"
 #include "intel_uncore.h"
 #include "intel_rps.h"
 
drivers/gpu/drm/i915/gt/intel_lrc.h +0 -39
index 802585a308e9..9116b46844a2 100644
@@ -34,45 +34,6 @@ struct i915_request;
 struct intel_context;
 struct intel_engine_cs;
 
-/* Execlists regs */
-#define RING_ELSP(base)				_MMIO((base) + 0x230)
-#define RING_EXECLIST_STATUS_LO(base)		_MMIO((base) + 0x234)
-#define RING_EXECLIST_STATUS_HI(base)		_MMIO((base) + 0x234 + 4)
-#define RING_CONTEXT_CONTROL(base)		_MMIO((base) + 0x244)
-#define	  CTX_CTRL_INHIBIT_SYN_CTX_SWITCH	(1 << 3)
-#define	  CTX_CTRL_ENGINE_CTX_RESTORE_INHIBIT	(1 << 0)
-#define   CTX_CTRL_RS_CTX_ENABLE		(1 << 1)
-#define	  CTX_CTRL_ENGINE_CTX_SAVE_INHIBIT	(1 << 2)
-#define	  GEN12_CTX_CTRL_OAR_CONTEXT_ENABLE	(1 << 8)
-#define RING_CONTEXT_STATUS_PTR(base)		_MMIO((base) + 0x3a0)
-#define RING_EXECLIST_SQ_CONTENTS(base)		_MMIO((base) + 0x510)
-#define RING_EXECLIST_CONTROL(base)		_MMIO((base) + 0x550)
-
-#define	  EL_CTRL_LOAD				(1 << 0)
-
-/* The docs specify that the write pointer wraps around after 5h, "After status
- * is written out to the last available status QW at offset 5h, this pointer
- * wraps to 0."
- *
- * Therefore, one must infer than even though there are 3 bits available, 6 and
- * 7 appear to be * reserved.
- */
-#define GEN8_CSB_ENTRIES 6
-#define GEN8_CSB_PTR_MASK 0x7
-#define GEN8_CSB_READ_PTR_MASK (GEN8_CSB_PTR_MASK << 8)
-#define GEN8_CSB_WRITE_PTR_MASK (GEN8_CSB_PTR_MASK << 0)
-
-#define GEN11_CSB_ENTRIES 12
-#define GEN11_CSB_PTR_MASK 0xf
-#define GEN11_CSB_READ_PTR_MASK (GEN11_CSB_PTR_MASK << 8)
-#define GEN11_CSB_WRITE_PTR_MASK (GEN11_CSB_PTR_MASK << 0)
-
-#define MAX_CONTEXT_HW_ID (1<<21) /* exclusive */
-#define MAX_GUC_CONTEXT_HW_ID (1 << 20) /* exclusive */
-#define GEN11_MAX_CONTEXT_HW_ID (1<<11) /* exclusive */
-/* in Gen12 ID 0x7FF is reserved to indicate idle */
-#define GEN12_MAX_CONTEXT_HW_ID	(GEN11_MAX_CONTEXT_HW_ID - 1)
-
 enum {
 	INTEL_CONTEXT_SCHEDULE_IN = 0,
 	INTEL_CONTEXT_SCHEDULE_OUT,
drivers/gpu/drm/i915/gt/intel_lrc_reg.h +39 -0
index 1b51f7b9a5c3..b2e03ce35599 100644
@@ -52,4 +52,43 @@
 #define GEN8_EXECLISTS_STATUS_BUF 0x370
 #define GEN11_EXECLISTS_STATUS_BUF2 0x3c0
 
+/* Execlists regs */
+#define RING_ELSP(base)				_MMIO((base) + 0x230)
+#define RING_EXECLIST_STATUS_LO(base)		_MMIO((base) + 0x234)
+#define RING_EXECLIST_STATUS_HI(base)		_MMIO((base) + 0x234 + 4)
+#define RING_CONTEXT_CONTROL(base)		_MMIO((base) + 0x244)
+#define	  CTX_CTRL_ENGINE_CTX_RESTORE_INHIBIT	REG_BIT(0)
+#define   CTX_CTRL_RS_CTX_ENABLE		REG_BIT(1)
+#define	  CTX_CTRL_ENGINE_CTX_SAVE_INHIBIT	REG_BIT(2)
+#define	  CTX_CTRL_INHIBIT_SYN_CTX_SWITCH	REG_BIT(3)
+#define	  GEN12_CTX_CTRL_OAR_CONTEXT_ENABLE	REG_BIT(8)
+#define RING_CONTEXT_STATUS_PTR(base)		_MMIO((base) + 0x3a0)
+#define RING_EXECLIST_SQ_CONTENTS(base)		_MMIO((base) + 0x510)
+#define RING_EXECLIST_CONTROL(base)		_MMIO((base) + 0x550)
+#define	  EL_CTRL_LOAD				REG_BIT(0)
+
+/*
+ * The docs specify that the write pointer wraps around after 5h, "After status
+ * is written out to the last available status QW at offset 5h, this pointer
+ * wraps to 0."
+ *
+ * Therefore, one must infer than even though there are 3 bits available, 6 and
+ * 7 appear to be * reserved.
+ */
+#define GEN8_CSB_ENTRIES 6
+#define GEN8_CSB_PTR_MASK 0x7
+#define GEN8_CSB_READ_PTR_MASK	(GEN8_CSB_PTR_MASK << 8)
+#define GEN8_CSB_WRITE_PTR_MASK	(GEN8_CSB_PTR_MASK << 0)
+
+#define GEN11_CSB_ENTRIES 12
+#define GEN11_CSB_PTR_MASK 0xf
+#define GEN11_CSB_READ_PTR_MASK		(GEN11_CSB_PTR_MASK << 8)
+#define GEN11_CSB_WRITE_PTR_MASK	(GEN11_CSB_PTR_MASK << 0)
+
+#define MAX_CONTEXT_HW_ID	(1 << 21) /* exclusive */
+#define MAX_GUC_CONTEXT_HW_ID	(1 << 20) /* exclusive */
+#define GEN11_MAX_CONTEXT_HW_ID	(1 << 11) /* exclusive */
+/* in Gen12 ID 0x7FF is reserved to indicate idle */
+#define GEN12_MAX_CONTEXT_HW_ID	(GEN11_MAX_CONTEXT_HW_ID - 1)
+
 #endif /* _INTEL_LRC_REG_H_ */
drivers/gpu/drm/i915/gvt/mmio_context.h +2 -0
index 3b25e7fe32f6..412b96ee6883 100644
@@ -36,6 +36,8 @@
 #ifndef __GVT_RENDER_H__
 #define __GVT_RENDER_H__
 
+#include "gt/intel_lrc_reg.h"
+
 struct engine_mmio {
 	enum intel_engine_id id;
 	i915_reg_t reg;
Signature · 6 lines
2.26.2

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
#9 lore raw

RFC PATCH 6/162 drm/i915: split gen8+ flush and bb_start emission functions to their own file

In reply to Matthew Auld

From: Daniele Ceraolo Spurio <[email protected]>

These functions are independent from the backend used and can therefore be split out of the exelists submission file, so they can be re-used by the upcoming GuC submission backend.

Based on a patch by Chris Wilson.

---
 drivers/gpu/drm/i915/Makefile                 |   1 +
 drivers/gpu/drm/i915/gt/gen8_engine_cs.c      | 393 ++++++++++++++++++
 drivers/gpu/drm/i915/gt/gen8_engine_cs.h      |  26 ++
 .../drm/i915/gt/intel_execlists_submission.c  | 385 +----------------
 4 files changed, 421 insertions(+), 384 deletions(-)
 create mode 100644 drivers/gpu/drm/i915/gt/gen8_engine_cs.c
 create mode 100644 drivers/gpu/drm/i915/gt/gen8_engine_cs.h
drivers/gpu/drm/i915/Makefile +1 -0
index aedbd8f52be8..f9ef5199b124 100644
@@ -82,6 +82,7 @@ gt-y += \
 	gt/gen6_engine_cs.o \
 	gt/gen6_ppgtt.o \
 	gt/gen7_renderclear.o \
+	gt/gen8_engine_cs.o \
 	gt/gen8_ppgtt.o \
 	gt/intel_breadcrumbs.o \
 	gt/intel_context.o \
drivers/gpu/drm/i915/gt/gen8_engine_cs.c new file +393 -0

393 lines not on this page · raw message

drivers/gpu/drm/i915/gt/gen8_engine_cs.h new file +26 -0
new file mode 100644
index 000000000000..c0c62284b650
@@ -0,0 +1,26 @@
+/* SPDX-License-Identifier: MIT */
+/*
+ * Copyright © 2014 Intel Corporation
+ */
+
+#ifndef __GEN8_ENGINE_CS_H__
+#define __GEN8_ENGINE_CS_H__
+
+#include <linux/types.h>
+
+struct i915_request;
+
+int gen8_emit_flush_render(struct i915_request *request, u32 mode);
+int gen8_emit_flush(struct i915_request *request, u32 mode);
+int gen11_emit_flush_render(struct i915_request *request, u32 mode);
+int gen12_emit_flush_render(struct i915_request *request, u32 mode);
+int gen12_emit_flush(struct i915_request *request, u32 mode);
+
+int gen8_emit_bb_start_noarb(struct i915_request *rq,
+			     u64 offset, u32 len,
+			     const unsigned int flags);
+int gen8_emit_bb_start(struct i915_request *rq,
+		       u64 offset, u32 len,
+		       const unsigned int flags);
+
+#endif /* __GEN8_ENGINE_CS_H__ */
drivers/gpu/drm/i915/gt/intel_execlists_submission.c +1 -384

403 lines not on this page · raw message

Signature · 6 lines
2.26.2

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
#11 lore raw

RFC PATCH 7/162 drm/i915: split wa_bb code to its own file

In reply to Matthew Auld

From: Daniele Ceraolo Spurio <[email protected]>

Continuing the split of back-end independent code from the execlist submission specific file.

Based on a patch by Chris Wilson.

---
 drivers/gpu/drm/i915/Makefile                 |   1 +
 .../drm/i915/gt/intel_engine_workaround_bb.c  | 335 ++++++++++++++++++
 .../drm/i915/gt/intel_engine_workaround_bb.h  |  14 +
 .../drm/i915/gt/intel_execlists_submission.c  | 327 +----------------
 4 files changed, 352 insertions(+), 325 deletions(-)
 create mode 100644 drivers/gpu/drm/i915/gt/intel_engine_workaround_bb.c
 create mode 100644 drivers/gpu/drm/i915/gt/intel_engine_workaround_bb.h
drivers/gpu/drm/i915/Makefile +1 -0
index f9ef5199b124..2445cc990e15 100644
@@ -92,6 +92,7 @@ gt-y += \
 	gt/intel_engine_heartbeat.o \
 	gt/intel_engine_pm.o \
 	gt/intel_engine_user.o \
+	gt/intel_engine_workaround_bb.o \
 	gt/intel_execlists_submission.o \
 	gt/intel_ggtt.o \
 	gt/intel_ggtt_fencing.o \
drivers/gpu/drm/i915/gt/intel_engine_workaround_bb.c new file +335 -0

335 lines not on this page · raw message

drivers/gpu/drm/i915/gt/intel_engine_workaround_bb.h new file +14 -0
new file mode 100644
index 000000000000..88771d77fd42
@@ -0,0 +1,14 @@
+/* SPDX-License-Identifier: MIT */
+/*
+ * Copyright © 2014 Intel Corporation
+ */
+
+#ifndef __INTEL_ENGINE_WORKAROUND_BB_H__
+#define __INTEL_ENGINE_WORKAROUND_BB_H__
+
+struct intel_engine_cs;
+
+int intel_init_workaround_bb(struct intel_engine_cs *engine);
+void intel_fini_workaround_bb(struct intel_engine_cs *engine);
+
+#endif /* __INTEL_ENGINE_WORKAROUND_BB_H__ */
drivers/gpu/drm/i915/gt/intel_execlists_submission.c +2 -325

345 lines not on this page · raw message

Signature · 6 lines
2.26.2

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
#12 lore raw

RFC PATCH 8/162 HAX drm/i915: Work around the selftest timeline lock splat workaround

In reply to Matthew Auld

From: Thomas Hellström <[email protected]>

There is a dirty hack to work around a lockdep splat because incorrect ordering of selftest timeline lock against other locks. However, some selftests recently started to use the same nesting level as the workaround and thus introduced more splats. Add a workaround to the workaround making some selftests aware of the workaround.

---
 drivers/gpu/drm/i915/gt/intel_context.c     |  3 ++-
 drivers/gpu/drm/i915/gt/intel_context.h     |  2 ++
 drivers/gpu/drm/i915/gt/selftest_timeline.c | 10 ++++++----
 3 files changed, 10 insertions(+), 5 deletions(-)
drivers/gpu/drm/i915/gt/intel_context.c +2 -1
index 349e7fa1488d..b63a8eb6c1a9 100644
@@ -495,7 +495,8 @@ struct i915_request *intel_context_create_request(struct intel_context *ce)
 	 */
 	lockdep_unpin_lock(&ce->timeline->mutex, rq->cookie);
 	mutex_release(&ce->timeline->mutex.dep_map, _RET_IP_);
-	mutex_acquire(&ce->timeline->mutex.dep_map, SINGLE_DEPTH_NESTING, 0, _RET_IP_);
+	mutex_acquire(&ce->timeline->mutex.dep_map, SELFTEST_WA_NESTING, 0,
+		      _RET_IP_);
 	rq->cookie = lockdep_pin_lock(&ce->timeline->mutex);
 
 	return rq;
drivers/gpu/drm/i915/gt/intel_context.h +2 -0
index fda2eba81e22..175d505951c7 100644
@@ -25,6 +25,8 @@
 		     ##__VA_ARGS__);					\
 } while (0)
 
+#define SELFTEST_WA_NESTING SINGLE_DEPTH_NESTING
+
 struct i915_gem_ww_ctx;
 
 void intel_context_init(struct intel_context *ce,
drivers/gpu/drm/i915/gt/selftest_timeline.c +6 -4
index e4285d5a0360..fa3fec049542 100644
@@ -688,7 +688,7 @@ static int live_hwsp_wrap(void *arg)
 
 		tl->seqno = -4u;
 
-		mutex_lock_nested(&tl->mutex, SINGLE_DEPTH_NESTING);
+		mutex_lock_nested(&tl->mutex, SELFTEST_WA_NESTING + 1);
 		err = intel_timeline_get_seqno(tl, rq, &seqno[0]);
 		mutex_unlock(&tl->mutex);
 		if (err) {
@@ -705,7 +705,7 @@ static int live_hwsp_wrap(void *arg)
 		}
 		hwsp_seqno[0] = tl->hwsp_seqno;
 
-		mutex_lock_nested(&tl->mutex, SINGLE_DEPTH_NESTING);
+		mutex_lock_nested(&tl->mutex, SELFTEST_WA_NESTING + 1);
 		err = intel_timeline_get_seqno(tl, rq, &seqno[1]);
 		mutex_unlock(&tl->mutex);
 		if (err) {
@@ -1037,7 +1037,8 @@ static int live_hwsp_read(void *arg)
 				goto out;
 			}
 
-			mutex_lock(&watcher[0].rq->context->timeline->mutex);
+			mutex_lock_nested(&watcher[0].rq->context->timeline->mutex,
+					  SELFTEST_WA_NESTING + 1);
 			err = intel_timeline_read_hwsp(rq, watcher[0].rq, &hwsp);
 			if (err == 0)
 				err = emit_read_hwsp(watcher[0].rq, /* before */
@@ -1050,7 +1051,8 @@ static int live_hwsp_read(void *arg)
 				goto out;
 			}
 
-			mutex_lock(&watcher[1].rq->context->timeline->mutex);
+			mutex_lock_nested(&watcher[1].rq->context->timeline->mutex,
+					  SELFTEST_WA_NESTING + 1);
 			err = intel_timeline_read_hwsp(rq, watcher[1].rq, &hwsp);
 			if (err == 0)
 				err = emit_read_hwsp(watcher[1].rq, /* after */
Signature · 6 lines
2.26.2

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
#13 lore raw

RFC PATCH 9/162 drm/i915: Introduce drm_i915_lock_isolated

In reply to Matthew Auld

From: Thomas Hellström <[email protected]>

When an object is just created and not yet put on any lists, there's a single owner and thus trylock will always succeed. Introduce drm_i915_lock_isolated to annotate trylock in this situation. This is similar to TTM's create_locked() functionality.

---
 drivers/gpu/drm/i915/gem/i915_gem_object.h | 16 ++++++++++++++++
 1 file changed, 16 insertions(+)
drivers/gpu/drm/i915/gem/i915_gem_object.h +16 -0
index be14486f63a7..d61194ef484e 100644
@@ -107,6 +107,13 @@ i915_gem_object_put(struct drm_i915_gem_object *obj)
 
 #define assert_object_held(obj) dma_resv_assert_held((obj)->base.resv)
 
+#define object_is_isolated(obj)					\
+	(!IS_ENABLED(CONFIG_LOCKDEP) ||				\
+	 ((kref_read(&obj->base.refcount) == 0) ||		\
+	  ((kref_read(&obj->base.refcount) == 1) &&		\
+	   list_empty_careful(&obj->mm.link) &&			\
+	   list_empty_careful(&obj->vma.list))))
+
 static inline int __i915_gem_object_lock(struct drm_i915_gem_object *obj,
 					 struct i915_gem_ww_ctx *ww,
 					 bool intr)
@@ -147,6 +154,15 @@ static inline bool i915_gem_object_trylock(struct drm_i915_gem_object *obj)
 	return dma_resv_trylock(obj->base.resv);
 }
 
+static inline void i915_gem_object_lock_isolated(struct drm_i915_gem_object *obj)
+{
+	int ret;
+
+	WARN_ON(!object_is_isolated(obj));
+	ret = dma_resv_trylock(obj->base.resv);
+	GEM_WARN_ON(!ret);
+}
+
 static inline void i915_gem_object_unlock(struct drm_i915_gem_object *obj)
 {
 	dma_resv_unlock(obj->base.resv);
Signature · 6 lines
2.26.2

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
#17 lore raw

RFC PATCH 13/162 drm/i915: Add missing -EDEADLK handling to execbuf pinning, v2.

In reply to Matthew Auld

From: Maarten Lankhorst <[email protected]>

i915_vma_pin may fail with -EDEADLK when we start locking page tables, so ensure we handle this correctly.

Cc Matthew Brost <[email protected]>
Cc Thomas Hellström <[email protected]>
---
 .../gpu/drm/i915/gem/i915_gem_execbuffer.c    | 35 +++++++++++++------
 1 file changed, 24 insertions(+), 11 deletions(-)
drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c +24 -11
index 60afa6f826d6..568c8321dc3d 100644
@@ -419,13 +419,14 @@ static u64 eb_pin_flags(const struct drm_i915_gem_exec_object2 *entry,
 	return pin_flags;
 }
 
-static inline bool
+static inline int
 eb_pin_vma(struct i915_execbuffer *eb,
 	   const struct drm_i915_gem_exec_object2 *entry,
 	   struct eb_vma *ev)
 {
 	struct i915_vma *vma = ev->vma;
 	u64 pin_flags;
+	int err;
 
 	if (vma->node.size)
 		pin_flags = vma->node.start;
@@ -437,24 +438,29 @@ eb_pin_vma(struct i915_execbuffer *eb,
 		pin_flags |= PIN_GLOBAL;
 
 	/* Attempt to reuse the current location if available */
-	/* TODO: Add -EDEADLK handling here */
-	if (unlikely(i915_vma_pin_ww(vma, &eb->ww, 0, 0, pin_flags))) {
+	err = i915_vma_pin_ww(vma, &eb->ww, 0, 0, pin_flags);
+	if (err == -EDEADLK)
+		return err;
+
+	if (unlikely(err)) {
 		if (entry->flags & EXEC_OBJECT_PINNED)
-			return false;
+			return err;
 
 		/* Failing that pick any _free_ space if suitable */
-		if (unlikely(i915_vma_pin_ww(vma, &eb->ww,
+		err = i915_vma_pin_ww(vma, &eb->ww,
 					     entry->pad_to_size,
 					     entry->alignment,
 					     eb_pin_flags(entry, ev->flags) |
-					     PIN_USER | PIN_NOEVICT)))
-			return false;
+					     PIN_USER | PIN_NOEVICT);
+		if (unlikely(err))
+			return err;
 	}
 
 	if (unlikely(ev->flags & EXEC_OBJECT_NEEDS_FENCE)) {
-		if (unlikely(i915_vma_pin_fence(vma))) {
+		err = i915_vma_pin_fence(vma);
+		if (unlikely(err)) {
 			i915_vma_unpin(vma);
-			return false;
+			return err;
 		}
 
 		if (vma->fence)
@@ -462,7 +468,10 @@ eb_pin_vma(struct i915_execbuffer *eb,
 	}
 
 	ev->flags |= __EXEC_OBJECT_HAS_PIN;
-	return !eb_vma_misplaced(entry, vma, ev->flags);
+	if (eb_vma_misplaced(entry, vma, ev->flags))
+		return -EBADSLT;
+
+	return 0;
 }
 
 static inline void
@@ -900,7 +909,11 @@ static int eb_validate_vmas(struct i915_execbuffer *eb)
 		if (err)
 			return err;
 
-		if (eb_pin_vma(eb, entry, ev)) {
+		err = eb_pin_vma(eb, entry, ev);
+		if (err == -EDEADLK)
+			return err;
+
+		if (!err) {
 			if (entry->offset != vma->node.start) {
 				entry->offset = vma->node.start | UPDATE;
 				eb->args->flags |= __EXEC_HAS_RELOC;
Signature · 6 lines
2.26.2

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
#16 lore raw

RFC PATCH 12/162 drm/i915: Move cmd parser pinning to execbuffer

In reply to Matthew Auld

From: Maarten Lankhorst <[email protected]>

We need to get rid of allocations in the cmd parser, because it needs to be called from a signaling context, first move all pinning to execbuf, where we already hold all locks.

Allocate jump_whitelist in the execbuffer, and add annotations around intel_engine_cmd_parser(), to ensure we only call the command parser without allocating any memory, or taking any locks we're not supposed to.

Because i915_gem_object_get_page() may also allocate memory, add a path to i915_gem_object_get_sg() that prevents memory allocations, and walk the sg list manually. It should be similarly fast.

This has the added benefit of being able to catch all memory allocation errors before the point of no return, and return -ENOMEM safely to the execbuf submitter.

---
 .../gpu/drm/i915/gem/i915_gem_execbuffer.c    |  74 ++++++++++++-
 drivers/gpu/drm/i915/gem/i915_gem_object.h    |  10 +-
 drivers/gpu/drm/i915/gem/i915_gem_pages.c     |  21 +++-
 drivers/gpu/drm/i915/gt/intel_ggtt.c          |   2 +-
 drivers/gpu/drm/i915/i915_cmd_parser.c        | 104 ++++++++----------
 drivers/gpu/drm/i915/i915_drv.h               |   7 +-
 drivers/gpu/drm/i915/i915_memcpy.c            |   2 +-
 drivers/gpu/drm/i915/i915_memcpy.h            |   2 +-
 8 files changed, 142 insertions(+), 80 deletions(-)
drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c +68 -6
index 1904e6e5ea64..60afa6f826d6 100644
@@ -24,6 +24,7 @@
 #include "i915_gem_clflush.h"
 #include "i915_gem_context.h"
 #include "i915_gem_ioctls.h"
+#include "i915_memcpy.h"
 #include "i915_sw_fence_work.h"
 #include "i915_trace.h"
 #include "i915_user_extensions.h"
@@ -2273,24 +2274,45 @@ struct eb_parse_work {
 	struct i915_vma *trampoline;
 	unsigned long batch_offset;
 	unsigned long batch_length;
+	unsigned long *jump_whitelist;
+	const void *batch_map;
+	void *shadow_map;
 };
 
 static int __eb_parse(struct dma_fence_work *work)
 {
 	struct eb_parse_work *pw = container_of(work, typeof(*pw), base);
+	int ret;
+	bool cookie;
 
-	return intel_engine_cmd_parser(pw->engine,
-				       pw->batch,
-				       pw->batch_offset,
-				       pw->batch_length,
-				       pw->shadow,
-				       pw->trampoline);
+	cookie = dma_fence_begin_signalling();
+	ret = intel_engine_cmd_parser(pw->engine,
+				      pw->batch,
+				      pw->batch_offset,
+				      pw->batch_length,
+				      pw->shadow,
+				      pw->jump_whitelist,
+				      pw->shadow_map,
+				      pw->batch_map);
+	dma_fence_end_signalling(cookie);
+
+	return ret;
 }
 
 static void __eb_parse_release(struct dma_fence_work *work)
 {
 	struct eb_parse_work *pw = container_of(work, typeof(*pw), base);
 
+	if (!IS_ERR_OR_NULL(pw->jump_whitelist))
+		kfree(pw->jump_whitelist);
+
+	if (pw->batch_map)
+		i915_gem_object_unpin_map(pw->batch->obj);
+	else
+		i915_gem_object_unpin_pages(pw->batch->obj);
+
+	i915_gem_object_unpin_map(pw->shadow->obj);
+
 	if (pw->trampoline)
 		i915_active_release(&pw->trampoline->active);
 	i915_active_release(&pw->shadow->active);
@@ -2340,6 +2362,8 @@ static int eb_parse_pipeline(struct i915_execbuffer *eb,
 			     struct i915_vma *trampoline)
 {
 	struct eb_parse_work *pw;
+	struct drm_i915_gem_object *batch = eb->batch->vma->obj;
+	bool needs_clflush;
 	int err;
 
 	GEM_BUG_ON(overflows_type(eb->batch_start_offset, pw->batch_offset));
@@ -2363,6 +2387,34 @@ static int eb_parse_pipeline(struct i915_execbuffer *eb,
 			goto err_shadow;
 	}
 
+	pw->shadow_map = i915_gem_object_pin_map(shadow->obj, I915_MAP_FORCE_WB);
+	if (IS_ERR(pw->shadow_map)) {
+		err = PTR_ERR(pw->shadow_map);
+		goto err_trampoline;
+	}
+
+	needs_clflush =
+		!(batch->cache_coherent & I915_BO_CACHE_COHERENT_FOR_READ);
+
+	pw->batch_map = ERR_PTR(-ENODEV);
+	if (needs_clflush && i915_has_memcpy_from_wc())
+		pw->batch_map = i915_gem_object_pin_map(batch, I915_MAP_WC);
+
+	if (IS_ERR(pw->batch_map)) {
+		err = i915_gem_object_pin_pages(batch);
+		if (err)
+			goto err_unmap_shadow;
+		pw->batch_map = NULL;
+	}
+
+	pw->jump_whitelist =
+		intel_engine_cmd_parser_alloc_jump_whitelist(eb->batch_len,
+							     trampoline);
+	if (IS_ERR(pw->jump_whitelist)) {
+		err = PTR_ERR(pw->jump_whitelist);
+		goto err_unmap_batch;
+	}
+
 	dma_fence_work_init(&pw->base, &eb_parse_ops);
 
 	pw->engine = eb->engine;
@@ -2402,6 +2454,16 @@ static int eb_parse_pipeline(struct i915_execbuffer *eb,
 	dma_fence_work_commit_imm(&pw->base);
 	return err;
 
+err_unmap_batch:
+	if (pw->batch_map)
+		i915_gem_object_unpin_map(batch);
+	else
+		i915_gem_object_unpin_pages(batch);
+err_unmap_shadow:
+	i915_gem_object_unpin_map(shadow->obj);
+err_trampoline:
+	if (trampoline)
+		i915_active_release(&trampoline->active);
 err_shadow:
 	i915_active_release(&shadow->active);
 err_batch:
drivers/gpu/drm/i915/gem/i915_gem_object.h +5 -5
index d61194ef484e..80c5b2b326f5 100644
@@ -291,22 +291,22 @@ struct scatterlist *
 __i915_gem_object_get_sg(struct drm_i915_gem_object *obj,
 			 struct i915_gem_object_page_iter *iter,
 			 unsigned int n,
-			 unsigned int *offset);
+			 unsigned int *offset, bool allow_alloc);
 
 static inline struct scatterlist *
 i915_gem_object_get_sg(struct drm_i915_gem_object *obj,
 		       unsigned int n,
-		       unsigned int *offset)
+		       unsigned int *offset, bool allow_alloc)
 {
-	return __i915_gem_object_get_sg(obj, &obj->mm.get_page, n, offset);
+	return __i915_gem_object_get_sg(obj, &obj->mm.get_page, n, offset, allow_alloc);
 }
 
 static inline struct scatterlist *
 i915_gem_object_get_sg_dma(struct drm_i915_gem_object *obj,
 			   unsigned int n,
-			   unsigned int *offset)
+			   unsigned int *offset, bool allow_alloc)
 {
-	return __i915_gem_object_get_sg(obj, &obj->mm.get_dma_page, n, offset);
+	return __i915_gem_object_get_sg(obj, &obj->mm.get_dma_page, n, offset, allow_alloc);
 }
 
 struct page *
drivers/gpu/drm/i915/gem/i915_gem_pages.c +17 -4
index e2c7b2a7895f..ca076203f5e9 100644
@@ -445,7 +445,8 @@ struct scatterlist *
 __i915_gem_object_get_sg(struct drm_i915_gem_object *obj,
 			 struct i915_gem_object_page_iter *iter,
 			 unsigned int n,
-			 unsigned int *offset)
+			 unsigned int *offset,
+			 bool allow_alloc)
 {
 	const bool dma = iter == &obj->mm.get_dma_page;
 	struct scatterlist *sg;
@@ -467,6 +468,9 @@ __i915_gem_object_get_sg(struct drm_i915_gem_object *obj,
 	if (n < READ_ONCE(iter->sg_idx))
 		goto lookup;
 
+	if (!allow_alloc)
+		goto manual_lookup;
+
 	mutex_lock(&iter->lock);
 
 	/* We prefer to reuse the last sg so that repeated lookup of this
@@ -516,7 +520,16 @@ __i915_gem_object_get_sg(struct drm_i915_gem_object *obj,
 	if (unlikely(n < idx)) /* insertion completed by another thread */
 		goto lookup;
 
-	/* In case we failed to insert the entry into the radixtree, we need
+	goto manual_walk;
+
+manual_lookup:
+	idx = 0;
+	sg = obj->mm.pages->sgl;
+	count = __sg_page_count(sg);
+
+manual_walk:
+	/*
+	 * In case we failed to insert the entry into the radixtree, we need
 	 * to look beyond the current sg.
 	 */
 	while (idx + count <= n) {
@@ -563,7 +576,7 @@ i915_gem_object_get_page(struct drm_i915_gem_object *obj, unsigned int n)
 
 	GEM_BUG_ON(!i915_gem_object_has_struct_page(obj));
 
-	sg = i915_gem_object_get_sg(obj, n, &offset);
+	sg = i915_gem_object_get_sg(obj, n, &offset, true);
 	return nth_page(sg_page(sg), offset);
 }
 
@@ -589,7 +602,7 @@ i915_gem_object_get_dma_address_len(struct drm_i915_gem_object *obj,
 	struct scatterlist *sg;
 	unsigned int offset;
 
-	sg = i915_gem_object_get_sg_dma(obj, n, &offset);
+	sg = i915_gem_object_get_sg_dma(obj, n, &offset, true);
 
 	if (len)
 		*len = sg_dma_len(sg) - (offset << PAGE_SHIFT);
drivers/gpu/drm/i915/gt/intel_ggtt.c +1 -1
index cf94525be2c1..60bd2c8ed8b0 100644
@@ -1383,7 +1383,7 @@ intel_partial_pages(const struct i915_ggtt_view *view,
 	if (ret)
 		goto err_sg_alloc;
 
-	iter = i915_gem_object_get_sg_dma(obj, view->partial.offset, &offset);
+	iter = i915_gem_object_get_sg_dma(obj, view->partial.offset, &offset, true);
 	GEM_BUG_ON(!iter);
 
 	sg = st->sgl;
drivers/gpu/drm/i915/i915_cmd_parser.c +43 -61

162 lines not on this page · raw message

drivers/gpu/drm/i915/i915_drv.h +6 -1
index 0f7bf6831633..84182a40e777 100644
@@ -1951,12 +1951,17 @@ const char *i915_cache_level_str(struct drm_i915_private *i915, int type);
 int i915_cmd_parser_get_version(struct drm_i915_private *dev_priv);
 void intel_engine_init_cmd_parser(struct intel_engine_cs *engine);
 void intel_engine_cleanup_cmd_parser(struct intel_engine_cs *engine);
+unsigned long *intel_engine_cmd_parser_alloc_jump_whitelist(u32 batch_length,
+							    bool trampoline);
+
 int intel_engine_cmd_parser(struct intel_engine_cs *engine,
 			    struct i915_vma *batch,
 			    unsigned long batch_offset,
 			    unsigned long batch_length,
 			    struct i915_vma *shadow,
-			    bool trampoline);
+			    unsigned long *jump_whitelist,
+			    void *shadow_map,
+			    const void *batch_map);
 #define I915_CMD_PARSER_TRAMPOLINE_SIZE 8
 
 /* intel_device_info.c */
drivers/gpu/drm/i915/i915_memcpy.c +1 -1
index 7b3b83bd5ab8..1b021a4902de 100644
@@ -135,7 +135,7 @@ bool i915_memcpy_from_wc(void *dst, const void *src, unsigned long len)
  * accepts that its arguments may not be aligned, but are valid for the
  * potential 16-byte read past the end.
  */
-void i915_unaligned_memcpy_from_wc(void *dst, void *src, unsigned long len)
+void i915_unaligned_memcpy_from_wc(void *dst, const void *src, unsigned long len)
 {
 	unsigned long addr;
 
drivers/gpu/drm/i915/i915_memcpy.h +1 -1
index e36d30edd987..3df063a3293b 100644
@@ -13,7 +13,7 @@ struct drm_i915_private;
 void i915_memcpy_init_early(struct drm_i915_private *i915);
 
 bool i915_memcpy_from_wc(void *dst, const void *src, unsigned long len);
-void i915_unaligned_memcpy_from_wc(void *dst, void *src, unsigned long len);
+void i915_unaligned_memcpy_from_wc(void *dst, const void *src, unsigned long len);
 
 /* The movntdqa instructions used for memcpy-from-wc require 16-byte alignment,
  * as well as SSE4.1 support. i915_memcpy_from_wc() will report if it cannot
Signature · 6 lines
2.26.2

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
#15 lore raw

RFC PATCH 11/162 drm/i915: Pin timeline map after first timeline pin, v5.

In reply to Matthew Auld

From: Maarten Lankhorst <[email protected]>

We're starting to require the reservation lock for pinning, so wait until we have that.

Update the selftests to handle this correctly, and ensure pin is called in live_hwsp_rollover_user() and mock_hwsp_freelist().

---
 drivers/gpu/drm/i915/gt/intel_timeline.c      | 49 ++++++++++----
 drivers/gpu/drm/i915/gt/intel_timeline.h      |  1 +
 .../gpu/drm/i915/gt/intel_timeline_types.h    |  1 +
 drivers/gpu/drm/i915/gt/mock_engine.c         | 24 ++++++-
 drivers/gpu/drm/i915/gt/selftest_timeline.c   | 64 ++++++++++---------
 drivers/gpu/drm/i915/i915_selftest.h          |  2 +
 6 files changed, 96 insertions(+), 45 deletions(-)
drivers/gpu/drm/i915/gt/intel_timeline.c +37 -12
index a58228d1cd3b..479eb5440bc6 100644
@@ -229,13 +229,30 @@ static void cacheline_free(struct intel_timeline_cacheline *cl)
 	i915_active_release(&cl->active);
 }
 
+I915_SELFTEST_EXPORT int
+intel_timeline_pin_map(struct intel_timeline *timeline)
+{
+	if (!timeline->hwsp_cacheline) {
+		struct drm_i915_gem_object *obj = timeline->hwsp_ggtt->obj;
+		u32 ofs = offset_in_page(timeline->hwsp_offset);
+		void *vaddr;
+
+		vaddr = i915_gem_object_pin_map(obj, I915_MAP_WB);
+		if (IS_ERR(vaddr))
+			return PTR_ERR(vaddr);
+
+		timeline->hwsp_map = vaddr;
+		timeline->hwsp_seqno = memset(vaddr + ofs, 0, CACHELINE_BYTES);
+	}
+
+	return 0;
+}
+
 static int intel_timeline_init(struct intel_timeline *timeline,
 			       struct intel_gt *gt,
 			       struct i915_vma *hwsp,
 			       unsigned int offset)
 {
-	void *vaddr;
-
 	kref_init(&timeline->kref);
 	atomic_set(&timeline->pin_count, 0);
 
@@ -260,18 +277,15 @@ static int intel_timeline_init(struct intel_timeline *timeline,
 
 		timeline->hwsp_cacheline = cl;
 		timeline->hwsp_offset = cacheline * CACHELINE_BYTES;
-
-		vaddr = page_mask_bits(cl->vaddr);
+		timeline->hwsp_map = page_mask_bits(cl->vaddr);
+		timeline->hwsp_seqno =
+			memset(timeline->hwsp_map + timeline->hwsp_offset, 0,
+			       CACHELINE_BYTES);
 	} else {
 		timeline->hwsp_offset = offset;
-		vaddr = i915_gem_object_pin_map(hwsp->obj, I915_MAP_WB);
-		if (IS_ERR(vaddr))
-			return PTR_ERR(vaddr);
+		timeline->hwsp_map = NULL;
 	}
 
-	timeline->hwsp_seqno =
-		memset(vaddr + timeline->hwsp_offset, 0, CACHELINE_BYTES);
-
 	timeline->hwsp_ggtt = i915_vma_get(hwsp);
 	GEM_BUG_ON(timeline->hwsp_offset >= hwsp->size);
 
@@ -306,7 +320,7 @@ static void intel_timeline_fini(struct intel_timeline *timeline)
 
 	if (timeline->hwsp_cacheline)
 		cacheline_free(timeline->hwsp_cacheline);
-	else
+	else if (timeline->hwsp_map)
 		i915_gem_object_unpin_map(timeline->hwsp_ggtt->obj);
 
 	i915_vma_put(timeline->hwsp_ggtt);
@@ -346,9 +360,18 @@ int intel_timeline_pin(struct intel_timeline *tl, struct i915_gem_ww_ctx *ww)
 	if (atomic_add_unless(&tl->pin_count, 1, 0))
 		return 0;
 
+	if (!tl->hwsp_cacheline) {
+		err = intel_timeline_pin_map(tl);
+		if (err)
+			return err;
+	}
+
 	err = i915_ggtt_pin(tl->hwsp_ggtt, ww, 0, PIN_HIGH);
-	if (err)
+	if (err) {
+		if (!tl->hwsp_cacheline)
+			i915_gem_object_unpin_map(tl->hwsp_ggtt->obj);
 		return err;
+	}
 
 	tl->hwsp_offset =
 		i915_ggtt_offset(tl->hwsp_ggtt) +
@@ -360,6 +383,8 @@ int intel_timeline_pin(struct intel_timeline *tl, struct i915_gem_ww_ctx *ww)
 	if (atomic_fetch_inc(&tl->pin_count)) {
 		cacheline_release(tl->hwsp_cacheline);
 		__i915_vma_unpin(tl->hwsp_ggtt);
+		if (!tl->hwsp_cacheline)
+			i915_gem_object_unpin_map(tl->hwsp_ggtt->obj);
 	}
 
 	return 0;
drivers/gpu/drm/i915/gt/intel_timeline.h +1 -0
index 634acebd0c4b..725bae16237c 100644
@@ -114,5 +114,6 @@ void intel_gt_show_timelines(struct intel_gt *gt,
 						  const struct i915_request *rq,
 						  const char *prefix,
 						  int indent));
+I915_SELFTEST_DECLARE(int intel_timeline_pin_map(struct intel_timeline *tl));
 
 #endif
drivers/gpu/drm/i915/gt/intel_timeline_types.h +1 -0
index 4474f487f589..cac7fa3dfd43 100644
@@ -45,6 +45,7 @@ struct intel_timeline {
 	atomic_t pin_count;
 	atomic_t active_count;
 
+	void *hwsp_map;
 	const u32 *hwsp_seqno;
 	struct i915_vma *hwsp_ggtt;
 	u32 hwsp_offset;
drivers/gpu/drm/i915/gt/mock_engine.c +22 -2
index 2f830017c51d..016f4f345706 100644
@@ -32,9 +32,22 @@
 #include "mock_engine.h"
 #include "selftests/mock_request.h"
 
-static void mock_timeline_pin(struct intel_timeline *tl)
+static int mock_timeline_pin(struct intel_timeline *tl)
 {
+	int err;
+
+	if (!tl->hwsp_cacheline) {
+		if (WARN_ON(!i915_gem_object_trylock(tl->hwsp_ggtt->obj)))
+			return -EBUSY;
+
+		err = intel_timeline_pin_map(tl);
+		i915_gem_object_unlock(tl->hwsp_ggtt->obj);
+		if (err)
+			return err;
+	}
+
 	atomic_inc(&tl->pin_count);
+	return 0;
 }
 
 static void mock_timeline_unpin(struct intel_timeline *tl)
@@ -152,6 +165,8 @@ static void mock_context_destroy(struct kref *ref)
 
 static int mock_context_alloc(struct intel_context *ce)
 {
+	int err;
+
 	ce->ring = mock_ring(ce->engine);
 	if (!ce->ring)
 		return -ENOMEM;
@@ -162,7 +177,12 @@ static int mock_context_alloc(struct intel_context *ce)
 		return PTR_ERR(ce->timeline);
 	}
 
-	mock_timeline_pin(ce->timeline);
+	err = mock_timeline_pin(ce->timeline);
+	if (err) {
+		intel_timeline_put(ce->timeline);
+		ce->timeline = NULL;
+		return err;
+	}
 
 	return 0;
 }
drivers/gpu/drm/i915/gt/selftest_timeline.c +33 -31

168 lines not on this page · raw message

drivers/gpu/drm/i915/i915_selftest.h +2 -0
index d53d207ab6eb..f54de0499be7 100644
@@ -107,6 +107,7 @@ int __i915_subtests(const char *caller,
 
 #define I915_SELFTEST_DECLARE(x) x
 #define I915_SELFTEST_ONLY(x) unlikely(x)
+#define I915_SELFTEST_EXPORT
 
 #else /* !IS_ENABLED(CONFIG_DRM_I915_SELFTEST) */
 
@@ -116,6 +117,7 @@ static inline int i915_perf_selftests(struct pci_dev *pdev) { return 0; }
 
 #define I915_SELFTEST_DECLARE(x)
 #define I915_SELFTEST_ONLY(x) 0
+#define I915_SELFTEST_EXPORT static
 
 #endif
 
Signature · 6 lines
2.26.2

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
#18 lore raw

RFC PATCH 14/162 drm/i915: Ensure we hold the object mutex in pin correctly v2

In reply to Matthew Auld

From: Maarten Lankhorst <[email protected]>

Currently we have a lot of places where we hold the gem object lock, but haven't yet been converted to the ww dance. Complain loudly about those places.

i915_vma_pin shouldn't have the obj lock held, so we can do a ww dance, while i915_vma_pin_ww should.

---
 drivers/gpu/drm/i915/gt/intel_renderstate.c |  2 +-
 drivers/gpu/drm/i915/gt/intel_timeline.c    |  4 +-
 drivers/gpu/drm/i915/i915_vma.c             | 46 +++++++++++++++++++--
 drivers/gpu/drm/i915/i915_vma.h             |  5 +++
 4 files changed, 50 insertions(+), 7 deletions(-)
drivers/gpu/drm/i915/gt/intel_renderstate.c +1 -1
index ea2a77c7b469..a68e5c23a67c 100644
@@ -196,7 +196,7 @@ int intel_renderstate_init(struct intel_renderstate *so,
 	if (err)
 		goto err_context;
 
-	err = i915_vma_pin(so->vma, 0, 0, PIN_GLOBAL | PIN_HIGH);
+	err = i915_vma_pin_ww(so->vma, &so->ww, 0, 0, PIN_GLOBAL | PIN_HIGH);
 	if (err)
 		goto err_context;
 
drivers/gpu/drm/i915/gt/intel_timeline.c +2 -2
index 479eb5440bc6..b2d04717db20 100644
@@ -46,7 +46,7 @@ static int __hwsp_alloc(struct intel_gt *gt, struct intel_timeline_hwsp *hwsp)
 		goto out_unlock;
 	}
 
-	/* Pin early so we can call i915_ggtt_pin unlocked. */
+	/* Pin early so we can call i915_ggtt_pin_unlocked(). */
 	hwsp->vaddr = i915_gem_object_pin_map(obj, I915_MAP_WB);
 	if (IS_ERR(hwsp->vaddr)) {
 		ret = PTR_ERR(hwsp->vaddr);
@@ -514,7 +514,7 @@ __intel_timeline_get_seqno(struct intel_timeline *tl,
 		goto err_rollback;
 	}
 
-	err = i915_ggtt_pin(vma, NULL, 0, PIN_HIGH);
+	err = i915_ggtt_pin_unlocked(vma, 0, PIN_HIGH);
 	if (err) {
 		__idle_hwsp_free(vma->private, cacheline);
 		goto err_rollback;
drivers/gpu/drm/i915/i915_vma.c +42 -4

77 lines not on this page · raw message

drivers/gpu/drm/i915/i915_vma.h +5 -0
index 5b3a3c653454..22387a361999 100644
@@ -243,12 +243,17 @@ i915_vma_pin_ww(struct i915_vma *vma, struct i915_gem_ww_ctx *ww,
 static inline int __must_check
 i915_vma_pin(struct i915_vma *vma, u64 size, u64 alignment, u64 flags)
 {
+#ifdef CONFIG_LOCKDEP
+	WARN_ON_ONCE(vma->resv && dma_resv_held(vma->resv));
+#endif
 	return i915_vma_pin_ww(vma, NULL, size, alignment, flags);
 }
 
 int i915_ggtt_pin(struct i915_vma *vma, struct i915_gem_ww_ctx *ww,
 		  u32 align, unsigned int flags);
 
+int i915_ggtt_pin_unlocked(struct i915_vma *vma, u32 align, unsigned int flags);
+
 static inline int i915_vma_pin_count(const struct i915_vma *vma)
 {
 	return atomic_read(&vma->flags) & I915_VMA_PIN_MASK;
Signature · 6 lines
2.26.2

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
#21 lore raw

RFC PATCH 17/162 drm/i915: Rework struct phys attachment handling

In reply to Matthew Auld

From: Maarten Lankhorst <[email protected]>

Instead of creating a separate object type, we make changes to the shmem type, to clear struct page backing. This will allow us to ensure we never run into a race when we exchange obj->ops with other function pointers.

---
 drivers/gpu/drm/i915/gem/i915_gem_object.h    |   8 ++
 drivers/gpu/drm/i915/gem/i915_gem_phys.c      | 102 +++++++++---------
 drivers/gpu/drm/i915/gem/i915_gem_shmem.c     |  22 +++-
 .../drm/i915/gem/selftests/i915_gem_phys.c    |   6 --
 4 files changed, 78 insertions(+), 60 deletions(-)
drivers/gpu/drm/i915/gem/i915_gem_object.h +8 -0
index 16608bf7a4e9..e549b88693a2 100644
@@ -37,7 +37,15 @@ void __i915_gem_object_release_shmem(struct drm_i915_gem_object *obj,
 				     struct sg_table *pages,
 				     bool needs_clflush);
 
+int i915_gem_object_pwrite_phys(struct drm_i915_gem_object *obj,
+				const struct drm_i915_gem_pwrite *args);
+int i915_gem_object_pread_phys(struct drm_i915_gem_object *obj,
+			       const struct drm_i915_gem_pread *args);
+
 int i915_gem_object_attach_phys(struct drm_i915_gem_object *obj, int align);
+void i915_gem_object_put_pages_phys(struct drm_i915_gem_object *obj,
+				    struct sg_table *pages);
+
 
 void i915_gem_flush_free_objects(struct drm_i915_private *i915);
 
drivers/gpu/drm/i915/gem/i915_gem_phys.c +49 -53

166 lines not on this page · raw message

drivers/gpu/drm/i915/gem/i915_gem_shmem.c +21 -1
index 31c617a1115f..d590e0c3bd00 100644
@@ -303,6 +303,11 @@ shmem_put_pages(struct drm_i915_gem_object *obj, struct sg_table *pages)
 	struct pagevec pvec;
 	struct page *page;
 
+	if (unlikely(!i915_gem_object_has_struct_page(obj))) {
+		i915_gem_object_put_pages_phys(obj, pages);
+		return;
+	}
+
 	__i915_gem_object_release_shmem(obj, pages, true);
 
 	i915_gem_gtt_finish_pages(obj, pages);
@@ -343,6 +348,9 @@ shmem_pwrite(struct drm_i915_gem_object *obj,
 	/* Caller already validated user args */
 	GEM_BUG_ON(!access_ok(user_data, arg->size));
 
+	if (!i915_gem_object_has_struct_page(obj))
+		return i915_gem_object_pwrite_phys(obj, arg);
+
 	/*
 	 * Before we instantiate/pin the backing store for our use, we
 	 * can prepopulate the shmemfs filp efficiently using a write into
@@ -421,9 +429,20 @@ shmem_pwrite(struct drm_i915_gem_object *obj,
 	return 0;
 }
 
+static int
+shmem_pread(struct drm_i915_gem_object *obj,
+	    const struct drm_i915_gem_pread *arg)
+{
+	if (!i915_gem_object_has_struct_page(obj))
+		return i915_gem_object_pread_phys(obj, arg);
+
+	return -ENODEV;
+}
+
 static void shmem_release(struct drm_i915_gem_object *obj)
 {
-	i915_gem_object_release_memory_region(obj);
+	if (obj->flags & I915_BO_ALLOC_STRUCT_PAGE)
+		i915_gem_object_release_memory_region(obj);
 
 	fput(obj->base.filp);
 }
@@ -438,6 +457,7 @@ const struct drm_i915_gem_object_ops i915_gem_shmem_ops = {
 	.writeback = shmem_writeback,
 
 	.pwrite = shmem_pwrite,
+	.pread = shmem_pread,
 
 	.release = shmem_release,
 };
drivers/gpu/drm/i915/gem/selftests/i915_gem_phys.c +0 -6
index fb6a17701310..0cfa082047fe 100644
@@ -38,12 +38,6 @@ static int mock_phys_object(void *arg)
 	}
 
 	if (i915_gem_object_has_struct_page(obj)) {
-		err = -EINVAL;
-		pr_err("shmem has a struct page\n");
-		goto out_obj;
-	}
-
-	if (obj->ops != &i915_gem_phys_ops) {
 		pr_err("i915_gem_object_attach_phys did not create a phys object\n");
 		err = -EINVAL;
 		goto out_obj;
Signature · 6 lines
2.26.2

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel
#19 lore raw

RFC PATCH 15/162 drm/i915: Add gem object locking to madvise.

In reply to Matthew Auld

From: Maarten Lankhorst <[email protected]>

Doesn't need the full ww lock, only checking if pages are bound.

---
 drivers/gpu/drm/i915/i915_gem.c | 8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)
drivers/gpu/drm/i915/i915_gem.c +7 -1
index 58276694c848..b03e245640c0 100644
@@ -1051,10 +1051,14 @@ i915_gem_madvise_ioctl(struct drm_device *dev, void *data,
 	if (!obj)
 		return -ENOENT;
 
-	err = mutex_lock_interruptible(&obj->mm.lock);
+	err = i915_gem_object_lock_interruptible(obj, NULL);
 	if (err)
 		goto out;
 
+	err = mutex_lock_interruptible(&obj->mm.lock);
+	if (err)
+		goto out_ww;
+
 	if (i915_gem_object_has_pages(obj) &&
 	    i915_gem_object_is_tiled(obj) &&
 	    i915->quirks & QUIRK_PIN_SWIZZLED_PAGES) {
@@ -1099,6 +1103,8 @@ i915_gem_madvise_ioctl(struct drm_device *dev, void *data,
 	args->retained = obj->mm.madv != __I915_MADV_PURGED;
 	mutex_unlock(&obj->mm.lock);
 
+out_ww:
+	i915_gem_object_unlock(obj);
 out:
 	i915_gem_object_put(obj);
 	return err;
Signature · 6 lines
2.26.2

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel
#23 lore raw

RFC PATCH 19/162 drm/i915: make lockdep slightly happier about execbuf.

In reply to Matthew Auld

From: Maarten Lankhorst <[email protected]>

As soon as we install fences, we should stop allocating memory in order to prevent any potential deadlocks.

This is required later on, when we start adding support for dma-fence annotations.

---
 .../gpu/drm/i915/gem/i915_gem_execbuffer.c    | 24 ++++++++++++++-----
 drivers/gpu/drm/i915/i915_active.c            | 20 ++++++++--------
 drivers/gpu/drm/i915/i915_vma.c               |  8 ++++---
 drivers/gpu/drm/i915/i915_vma.h               |  3 +++
 4 files changed, 36 insertions(+), 19 deletions(-)
drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c +18 -6
index 568c8321dc3d..31e412e5c68a 100644
@@ -49,11 +49,12 @@ enum {
 #define DBG_FORCE_RELOC 0 /* choose one of the above! */
 };
 
-#define __EXEC_OBJECT_HAS_PIN		BIT(31)
-#define __EXEC_OBJECT_HAS_FENCE		BIT(30)
-#define __EXEC_OBJECT_NEEDS_MAP		BIT(29)
-#define __EXEC_OBJECT_NEEDS_BIAS	BIT(28)
-#define __EXEC_OBJECT_INTERNAL_FLAGS	(~0u << 28) /* all of the above */
+/* __EXEC_OBJECT_NO_RESERVE is BIT(31), defined in i915_vma.h */
+#define __EXEC_OBJECT_HAS_PIN		BIT(30)
+#define __EXEC_OBJECT_HAS_FENCE		BIT(29)
+#define __EXEC_OBJECT_NEEDS_MAP		BIT(28)
+#define __EXEC_OBJECT_NEEDS_BIAS	BIT(27)
+#define __EXEC_OBJECT_INTERNAL_FLAGS	(~0u << 27) /* all of the above + */
 #define __EXEC_OBJECT_RESERVED (__EXEC_OBJECT_HAS_PIN | __EXEC_OBJECT_HAS_FENCE)
 
 #define __EXEC_HAS_RELOC	BIT(31)
@@ -929,6 +930,12 @@ static int eb_validate_vmas(struct i915_execbuffer *eb)
 			}
 		}
 
+		if (!(ev->flags & EXEC_OBJECT_WRITE)) {
+			err = dma_resv_reserve_shared(vma->resv, 1);
+			if (err)
+				return err;
+		}
+
 		GEM_BUG_ON(drm_mm_node_allocated(&vma->node) &&
 			   eb_vma_misplaced(&eb->exec[i], vma, ev->flags));
 	}
@@ -2194,7 +2201,8 @@ static int eb_move_to_gpu(struct i915_execbuffer *eb)
 		}
 
 		if (err == 0)
-			err = i915_vma_move_to_active(vma, eb->request, flags);
+			err = i915_vma_move_to_active(vma, eb->request,
+						      flags | __EXEC_OBJECT_NO_RESERVE);
 	}
 
 	if (unlikely(err))
@@ -2446,6 +2454,10 @@ static int eb_parse_pipeline(struct i915_execbuffer *eb,
 	if (err)
 		goto err_commit;
 
+	err = dma_resv_reserve_shared(shadow->resv, 1);
+	if (err)
+		goto err_commit;
+
 	/* Wait for all writes (and relocs) into the batch to complete */
 	err = i915_sw_fence_await_reservation(&pw->base.chain,
 					      pw->batch->resv, NULL, false,
drivers/gpu/drm/i915/i915_active.c +10 -10
index 10a865f3dc09..6ba4f878ab0e 100644
@@ -296,18 +296,13 @@ static struct active_node *__active_lookup(struct i915_active *ref, u64 idx)
 static struct i915_active_fence *
 active_instance(struct i915_active *ref, u64 idx)
 {
-	struct active_node *node, *prealloc;
+	struct active_node *node;
 	struct rb_node **p, *parent;
 
 	node = __active_lookup(ref, idx);
 	if (likely(node))
 		return &node->base;
 
-	/* Preallocate a replacement, just in case */
-	prealloc = kmem_cache_alloc(global.slab_cache, GFP_KERNEL);
-	if (!prealloc)
-		return NULL;
-
 	spin_lock_irq(&ref->tree_lock);
 	GEM_BUG_ON(i915_active_is_idle(ref));
 
@@ -317,10 +312,8 @@ active_instance(struct i915_active *ref, u64 idx)
 		parent = *p;
 
 		node = rb_entry(parent, struct active_node, node);
-		if (node->timeline == idx) {
-			kmem_cache_free(global.slab_cache, prealloc);
+		if (node->timeline == idx)
 			goto out;
-		}
 
 		if (node->timeline < idx)
 			p = &parent->rb_right;
@@ -328,7 +321,14 @@ active_instance(struct i915_active *ref, u64 idx)
 			p = &parent->rb_left;
 	}
 
-	node = prealloc;
+	/*
+	 * XXX: We should preallocate this before i915_active_ref() is ever
+	 *  called, but we cannot call into fs_reclaim() anyway, so use GFP_ATOMIC.
+	 */
+	node = kmem_cache_alloc(global.slab_cache, GFP_ATOMIC);
+	if (!node)
+		goto out;
+
 	__i915_active_fence_init(&node->base, NULL, node_retire);
 	node->ref = ref;
 	node->timeline = idx;
drivers/gpu/drm/i915/i915_vma.c +5 -3
index e07621825da9..5b1d78fa748e 100644
@@ -1281,9 +1281,11 @@ int i915_vma_move_to_active(struct i915_vma *vma,
 		obj->write_domain = I915_GEM_DOMAIN_RENDER;
 		obj->read_domains = 0;
 	} else {
-		err = dma_resv_reserve_shared(vma->resv, 1);
-		if (unlikely(err))
-			return err;
+		if (!(flags & __EXEC_OBJECT_NO_RESERVE)) {
+			err = dma_resv_reserve_shared(vma->resv, 1);
+			if (unlikely(err))
+				return err;
+		}
 
 		dma_resv_add_shared_fence(vma->resv, &rq->fence);
 		obj->write_domain = 0;
drivers/gpu/drm/i915/i915_vma.h +3 -0
index 22387a361999..a2e7b58b70ca 100644
@@ -52,6 +52,9 @@ static inline bool i915_vma_is_active(const struct i915_vma *vma)
 	return !i915_active_is_idle(&vma->active);
 }
 
+/* do not reserve memory to prevent deadlocks */
+#define __EXEC_OBJECT_NO_RESERVE BIT(31)
+
 int __must_check __i915_vma_move_to_active(struct i915_vma *vma,
 					   struct i915_request *rq);
 int __must_check i915_vma_move_to_active(struct i915_vma *vma,
Signature · 6 lines
2.26.2

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
#14 lore raw

RFC PATCH 10/162 drm/i915: Lock hwsp objects isolated for pinning at create time

In reply to Matthew Auld

From: Thomas Hellström <[email protected]>

We may need to create hwsp objects at request treate time in the middle of a ww transaction. Since we typically don't have easy access to the ww_acquire_context, lock the hwsp objects isolated for pinning/mapping only at create time.
For later binding to the ggtt, make sure lockdep allows binding of already pinned pages to the ggtt without the underlying object lock held.

---
 drivers/gpu/drm/i915/gt/intel_timeline.c | 58 ++++++++++++++----------
 drivers/gpu/drm/i915/i915_vma.c          | 13 ++++--
 2 files changed, 44 insertions(+), 27 deletions(-)
drivers/gpu/drm/i915/gt/intel_timeline.c +35 -23
index 512afacd2bdc..a58228d1cd3b 100644
@@ -24,25 +24,43 @@ struct intel_timeline_hwsp {
 	struct list_head free_link;
 	struct i915_vma *vma;
 	u64 free_bitmap;
+	void *vaddr;
 };
 
-static struct i915_vma *__hwsp_alloc(struct intel_gt *gt)
+static int __hwsp_alloc(struct intel_gt *gt, struct intel_timeline_hwsp *hwsp)
 {
 	struct drm_i915_private *i915 = gt->i915;
 	struct drm_i915_gem_object *obj;
-	struct i915_vma *vma;
+	int ret;
 
 	obj = i915_gem_object_create_internal(i915, PAGE_SIZE);
 	if (IS_ERR(obj))
-		return ERR_CAST(obj);
+		return PTR_ERR(obj);
 
+	i915_gem_object_lock_isolated(obj);
 	i915_gem_object_set_cache_coherency(obj, I915_CACHE_LLC);
 
-	vma = i915_vma_instance(obj, &gt->ggtt->vm, NULL);
-	if (IS_ERR(vma))
-		i915_gem_object_put(obj);
+	hwsp->vma = i915_vma_instance(obj, &gt->ggtt->vm, NULL);
+	if (IS_ERR(hwsp->vma)) {
+		ret = PTR_ERR(hwsp->vma);
+		goto out_unlock;
+	}
+
+	/* Pin early so we can call i915_ggtt_pin unlocked. */
+	hwsp->vaddr = i915_gem_object_pin_map(obj, I915_MAP_WB);
+	if (IS_ERR(hwsp->vaddr)) {
+		ret = PTR_ERR(hwsp->vaddr);
+		goto out_unlock;
+	}
+
+	i915_gem_object_unlock(obj);
+	return 0;
+
+out_unlock:
+	i915_gem_object_unlock(obj);
+	i915_gem_object_put(obj);
 
-	return vma;
+	return ret;
 }
 
 static struct i915_vma *
@@ -59,7 +77,7 @@ hwsp_alloc(struct intel_timeline *timeline, unsigned int *cacheline)
 	hwsp = list_first_entry_or_null(&gt->hwsp_free_list,
 					typeof(*hwsp), free_link);
 	if (!hwsp) {
-		struct i915_vma *vma;
+		int ret;
 
 		spin_unlock_irq(&gt->hwsp_lock);
 
@@ -67,17 +85,16 @@ hwsp_alloc(struct intel_timeline *timeline, unsigned int *cacheline)
 		if (!hwsp)
 			return ERR_PTR(-ENOMEM);
 
-		vma = __hwsp_alloc(timeline->gt);
-		if (IS_ERR(vma)) {
+		ret = __hwsp_alloc(timeline->gt, hwsp);
+		if (ret) {
 			kfree(hwsp);
-			return vma;
+			return ERR_PTR(ret);
 		}
 
 		GT_TRACE(timeline->gt, "new HWSP allocated\n");
 
-		vma->private = hwsp;
+		hwsp->vma->private = hwsp;
 		hwsp->gt = timeline->gt;
-		hwsp->vma = vma;
 		hwsp->free_bitmap = ~0ull;
 		hwsp->gt_timelines = gt;
 
@@ -113,9 +130,12 @@ static void __idle_hwsp_free(struct intel_timeline_hwsp *hwsp, int cacheline)
 
 	/* And if no one is left using it, give the page back to the system */
 	if (hwsp->free_bitmap == ~0ull) {
-		i915_vma_put(hwsp->vma);
 		list_del(&hwsp->free_link);
+		spin_unlock_irqrestore(&gt->hwsp_lock, flags);
+		i915_gem_object_unpin_map(hwsp->vma->obj);
+		i915_vma_put(hwsp->vma);
 		kfree(hwsp);
+		return;
 	}
 
 	spin_unlock_irqrestore(&gt->hwsp_lock, flags);
@@ -134,7 +154,6 @@ static void __idle_cacheline_free(struct intel_timeline_cacheline *cl)
 {
 	GEM_BUG_ON(!i915_active_is_idle(&cl->active));
 
-	i915_gem_object_unpin_map(cl->hwsp->vma->obj);
 	i915_vma_put(cl->hwsp->vma);
 	__idle_hwsp_free(cl->hwsp, ptr_unmask_bits(cl->vaddr, CACHELINE_BITS));
 
@@ -165,7 +184,6 @@ static struct intel_timeline_cacheline *
 cacheline_alloc(struct intel_timeline_hwsp *hwsp, unsigned int cacheline)
 {
 	struct intel_timeline_cacheline *cl;
-	void *vaddr;
 
 	GEM_BUG_ON(cacheline >= BIT(CACHELINE_BITS));
 
@@ -173,15 +191,9 @@ cacheline_alloc(struct intel_timeline_hwsp *hwsp, unsigned int cacheline)
 	if (!cl)
 		return ERR_PTR(-ENOMEM);
 
-	vaddr = i915_gem_object_pin_map(hwsp->vma->obj, I915_MAP_WB);
-	if (IS_ERR(vaddr)) {
-		kfree(cl);
-		return ERR_CAST(vaddr);
-	}
-
 	i915_vma_get(hwsp->vma);
 	cl->hwsp = hwsp;
-	cl->vaddr = page_pack_bits(vaddr, cacheline);
+	cl->vaddr = page_pack_bits(hwsp->vaddr, cacheline);
 
 	i915_active_init(&cl->active, __cacheline_active, __cacheline_retire);
 
drivers/gpu/drm/i915/i915_vma.c +9 -4
index caa9b041616b..8e8c80ccbe32 100644
@@ -862,10 +862,15 @@ int i915_vma_pin_ww(struct i915_vma *vma, struct i915_gem_ww_ctx *ww,
 	unsigned int bound;
 	int err;
 
-#ifdef CONFIG_PROVE_LOCKING
-	if (debug_locks && lockdep_is_held(&vma->vm->i915->drm.struct_mutex))
-		WARN_ON(!ww);
-#endif
+	if (IS_ENABLED(CONFIG_PROVE_LOCKING) && debug_locks) {
+		bool pinned_bind_wo_alloc =
+			vma->obj && i915_gem_object_has_pinned_pages(vma->obj) &&
+			!vma->vm->allocate_va_range;
+
+		if (lockdep_is_held(&vma->vm->i915->drm.struct_mutex) &&
+		    !pinned_bind_wo_alloc)
+			WARN_ON(!ww);
+	}
 
 	BUILD_BUG_ON(PIN_GLOBAL != I915_VMA_GLOBAL_BIND);
 	BUILD_BUG_ON(PIN_USER != I915_VMA_LOCAL_BIND);
Signature · 6 lines
2.26.2

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel
#20 lore raw

RFC PATCH 16/162 drm/i915: Move HAS_STRUCT_PAGE to obj->flags

In reply to Matthew Auld

From: Maarten Lankhorst <[email protected]>

We want to remove the changing of ops structure for attaching phys pages, so we need to kill off HAS_STRUCT_PAGE from ops->flags, and put it in the bo.

This will remove a potential race of dereferencing the wrong obj->ops without ww mutex held.

---
 drivers/gpu/drm/i915/gem/i915_gem_dmabuf.c           |  2 +-
 drivers/gpu/drm/i915/gem/i915_gem_internal.c         |  6 +++---
 drivers/gpu/drm/i915/gem/i915_gem_lmem.c             |  4 ++--
 drivers/gpu/drm/i915/gem/i915_gem_mman.c             |  7 +++----
 drivers/gpu/drm/i915/gem/i915_gem_object.c           |  4 +++-
 drivers/gpu/drm/i915/gem/i915_gem_object.h           |  5 +++--
 drivers/gpu/drm/i915/gem/i915_gem_object_types.h     |  8 +++++---
 drivers/gpu/drm/i915/gem/i915_gem_pages.c            |  5 ++---
 drivers/gpu/drm/i915/gem/i915_gem_phys.c             |  2 ++
 drivers/gpu/drm/i915/gem/i915_gem_region.c           |  4 +---
 drivers/gpu/drm/i915/gem/i915_gem_region.h           |  3 +--
 drivers/gpu/drm/i915/gem/i915_gem_shmem.c            |  8 ++++----
 drivers/gpu/drm/i915/gem/i915_gem_stolen.c           |  4 ++--
 drivers/gpu/drm/i915/gem/i915_gem_userptr.c          |  6 +++---
 drivers/gpu/drm/i915/gem/selftests/huge_gem_object.c |  4 ++--
 drivers/gpu/drm/i915/gem/selftests/huge_pages.c      | 10 +++++-----
 drivers/gpu/drm/i915/gem/selftests/i915_gem_mman.c   | 11 ++++-------
 drivers/gpu/drm/i915/gem/selftests/i915_gem_phys.c   | 12 ++++++++++++
 drivers/gpu/drm/i915/gvt/dmabuf.c                    |  2 +-
 drivers/gpu/drm/i915/selftests/i915_gem_gtt.c        |  2 +-
 drivers/gpu/drm/i915/selftests/mock_region.c         |  4 ++--
 21 files changed, 62 insertions(+), 51 deletions(-)
drivers/gpu/drm/i915/gem/i915_gem_dmabuf.c +1 -1
index 04e9c04545ad..36e3c2765f4c 100644
@@ -258,7 +258,7 @@ struct drm_gem_object *i915_gem_prime_import(struct drm_device *dev,
 	}
 
 	drm_gem_private_object_init(dev, &obj->base, dma_buf->size);
-	i915_gem_object_init(obj, &i915_gem_object_dmabuf_ops, &lock_class);
+	i915_gem_object_init(obj, &i915_gem_object_dmabuf_ops, &lock_class, 0);
 	obj->base.import_attach = attach;
 	obj->base.resv = dma_buf->resv;
 
drivers/gpu/drm/i915/gem/i915_gem_internal.c +3 -3
index ad22f42541bd..21cc40897ca8 100644
@@ -138,8 +138,7 @@ static void i915_gem_object_put_pages_internal(struct drm_i915_gem_object *obj,
 
 static const struct drm_i915_gem_object_ops i915_gem_object_internal_ops = {
 	.name = "i915_gem_object_internal",
-	.flags = I915_GEM_OBJECT_HAS_STRUCT_PAGE |
-		 I915_GEM_OBJECT_IS_SHRINKABLE,
+	.flags = I915_GEM_OBJECT_IS_SHRINKABLE,
 	.get_pages = i915_gem_object_get_pages_internal,
 	.put_pages = i915_gem_object_put_pages_internal,
 };
@@ -178,7 +177,8 @@ i915_gem_object_create_internal(struct drm_i915_private *i915,
 		return ERR_PTR(-ENOMEM);
 
 	drm_gem_private_object_init(&i915->drm, &obj->base, size);
-	i915_gem_object_init(obj, &i915_gem_object_internal_ops, &lock_class);
+	i915_gem_object_init(obj, &i915_gem_object_internal_ops, &lock_class,
+			     I915_BO_ALLOC_STRUCT_PAGE);
 
 	/*
 	 * Mark the object as volatile, such that the pages are marked as
drivers/gpu/drm/i915/gem/i915_gem_lmem.c +2 -2
index 932ee21e6609..e953965f8263 100644
@@ -45,13 +45,13 @@ __i915_gem_lmem_object_create(struct intel_memory_region *mem,
 		return ERR_PTR(-ENOMEM);
 
 	drm_gem_private_object_init(&i915->drm, &obj->base, size);
-	i915_gem_object_init(obj, &i915_gem_lmem_obj_ops, &lock_class);
+	i915_gem_object_init(obj, &i915_gem_lmem_obj_ops, &lock_class, flags);
 
 	obj->read_domains = I915_GEM_DOMAIN_WC | I915_GEM_DOMAIN_GTT;
 
 	i915_gem_object_set_cache_coherency(obj, I915_CACHE_NONE);
 
-	i915_gem_object_init_memory_region(obj, mem, flags);
+	i915_gem_object_init_memory_region(obj, mem);
 
 	return obj;
 }
drivers/gpu/drm/i915/gem/i915_gem_mman.c +3 -4
index ec28a6cde49b..c0034d811e50 100644
@@ -251,7 +251,7 @@ static vm_fault_t vm_fault_cpu(struct vm_fault *vmf)
 		goto out;
 
 	iomap = -1;
-	if (!i915_gem_object_type_has(obj, I915_GEM_OBJECT_HAS_STRUCT_PAGE)) {
+	if (!i915_gem_object_has_struct_page(obj)) {
 		iomap = obj->mm.region->iomap.base;
 		iomap -= obj->mm.region->region.start;
 	}
@@ -653,9 +653,8 @@ __assign_mmap_offset(struct drm_file *file,
 	}
 
 	if (mmap_type != I915_MMAP_TYPE_GTT &&
-	    !i915_gem_object_type_has(obj,
-				      I915_GEM_OBJECT_HAS_STRUCT_PAGE |
-				      I915_GEM_OBJECT_HAS_IOMEM)) {
+	    !i915_gem_object_has_struct_page(obj) &&
+	    !i915_gem_object_type_has(obj, I915_GEM_OBJECT_HAS_IOMEM)) {
 		err = -ENODEV;
 		goto out;
 	}
drivers/gpu/drm/i915/gem/i915_gem_object.c +3 -1
index 00d24000b5e8..1393988bd5af 100644
@@ -60,7 +60,7 @@ void i915_gem_object_free(struct drm_i915_gem_object *obj)
 
 void i915_gem_object_init(struct drm_i915_gem_object *obj,
 			  const struct drm_i915_gem_object_ops *ops,
-			  struct lock_class_key *key)
+			  struct lock_class_key *key, unsigned flags)
 {
 	__mutex_init(&obj->mm.lock, ops->name ?: "obj->mm.lock", key);
 
@@ -78,6 +78,8 @@ void i915_gem_object_init(struct drm_i915_gem_object *obj,
 	init_rcu_head(&obj->rcu);
 
 	obj->ops = ops;
+	GEM_BUG_ON(flags & ~I915_BO_ALLOC_FLAGS);
+	obj->flags = flags;
 
 	obj->mm.madv = I915_MADV_WILLNEED;
 	INIT_RADIX_TREE(&obj->mm.get_page.radix, GFP_KERNEL | __GFP_NOWARN);
drivers/gpu/drm/i915/gem/i915_gem_object.h +3 -2
index 80c5b2b326f5..16608bf7a4e9 100644
@@ -23,7 +23,8 @@ void i915_gem_object_free(struct drm_i915_gem_object *obj);
 
 void i915_gem_object_init(struct drm_i915_gem_object *obj,
 			  const struct drm_i915_gem_object_ops *ops,
-			  struct lock_class_key *key);
+			  struct lock_class_key *key,
+			  unsigned alloc_flags);
 struct drm_i915_gem_object *
 i915_gem_object_create_shmem(struct drm_i915_private *i915,
 			     resource_size_t size);
@@ -213,7 +214,7 @@ i915_gem_object_type_has(const struct drm_i915_gem_object *obj,
 static inline bool
 i915_gem_object_has_struct_page(const struct drm_i915_gem_object *obj)
 {
-	return i915_gem_object_type_has(obj, I915_GEM_OBJECT_HAS_STRUCT_PAGE);
+	return obj->flags & I915_BO_ALLOC_STRUCT_PAGE;
 }
 
 static inline bool
drivers/gpu/drm/i915/gem/i915_gem_object_types.h +5 -3
index e2d9b7e1e152..b53e44b06b09 100644
@@ -30,7 +30,6 @@ struct i915_lut_handle {
 
 struct drm_i915_gem_object_ops {
 	unsigned int flags;
-#define I915_GEM_OBJECT_HAS_STRUCT_PAGE	BIT(0)
 #define I915_GEM_OBJECT_HAS_IOMEM	BIT(1)
 #define I915_GEM_OBJECT_IS_SHRINKABLE	BIT(2)
 #define I915_GEM_OBJECT_IS_PROXY	BIT(3)
@@ -165,8 +164,11 @@ struct drm_i915_gem_object {
 	unsigned long flags;
 #define I915_BO_ALLOC_CONTIGUOUS BIT(0)
 #define I915_BO_ALLOC_VOLATILE   BIT(1)
-#define I915_BO_ALLOC_FLAGS (I915_BO_ALLOC_CONTIGUOUS | I915_BO_ALLOC_VOLATILE)
-#define I915_BO_READONLY         BIT(2)
+#define I915_BO_ALLOC_STRUCT_PAGE BIT(2)
+#define I915_BO_ALLOC_FLAGS (I915_BO_ALLOC_CONTIGUOUS | \
+			     I915_BO_ALLOC_VOLATILE | \
+			     I915_BO_ALLOC_STRUCT_PAGE)
+#define I915_BO_READONLY         BIT(3)
 
 	/*
 	 * Is the object to be mapped as read-only to the GPU
drivers/gpu/drm/i915/gem/i915_gem_pages.c +2 -3
index ca076203f5e9..7983423237e3 100644
@@ -328,13 +328,12 @@ void *i915_gem_object_pin_map(struct drm_i915_gem_object *obj,
 			      enum i915_map_type type)
 {
 	enum i915_map_type has_type;
-	unsigned int flags;
 	bool pinned;
 	void *ptr;
 	int err;
 
-	flags = I915_GEM_OBJECT_HAS_STRUCT_PAGE | I915_GEM_OBJECT_HAS_IOMEM;
-	if (!i915_gem_object_type_has(obj, flags))
+	if (!i915_gem_object_has_struct_page(obj) &&
+	    !i915_gem_object_type_has(obj, I915_GEM_OBJECT_HAS_IOMEM))
 		return ERR_PTR(-ENXIO);
 
 	err = mutex_lock_interruptible_nested(&obj->mm.lock, I915_MM_GET_PAGES);
drivers/gpu/drm/i915/gem/i915_gem_phys.c +2 -0
index 3a4dfe2ef1da..965590d3a570 100644
@@ -240,6 +240,7 @@ int i915_gem_object_attach_phys(struct drm_i915_gem_object *obj, int align)
 	pages = __i915_gem_object_unset_pages(obj);
 
 	obj->ops = &i915_gem_phys_ops;
+	obj->flags &= ~I915_BO_ALLOC_STRUCT_PAGE;
 
 	err = ____i915_gem_object_get_pages(obj);
 	if (err)
@@ -258,6 +259,7 @@ int i915_gem_object_attach_phys(struct drm_i915_gem_object *obj, int align)
 
 err_xfer:
 	obj->ops = &i915_gem_shmem_ops;
+	obj->flags |= I915_BO_ALLOC_STRUCT_PAGE;
 	if (!IS_ERR_OR_NULL(pages)) {
 		unsigned int sg_page_sizes = i915_sg_page_sizes(pages->sgl);
 
drivers/gpu/drm/i915/gem/i915_gem_region.c +1 -3
index 1515384d7e0e..6a96741253b3 100644
@@ -102,13 +102,11 @@ i915_gem_object_get_pages_buddy(struct drm_i915_gem_object *obj)
 }
 
 void i915_gem_object_init_memory_region(struct drm_i915_gem_object *obj,
-					struct intel_memory_region *mem,
-					unsigned long flags)
+					struct intel_memory_region *mem)
 {
 	INIT_LIST_HEAD(&obj->mm.blocks);
 	obj->mm.region = intel_memory_region_get(mem);
 
-	obj->flags |= flags;
 	if (obj->base.size <= mem->min_page_size)
 		obj->flags |= I915_BO_ALLOC_CONTIGUOUS;
 
drivers/gpu/drm/i915/gem/i915_gem_region.h +1 -2
index f2ff6f8bff74..ebddc86d78f7 100644
@@ -17,8 +17,7 @@ void i915_gem_object_put_pages_buddy(struct drm_i915_gem_object *obj,
 				     struct sg_table *pages);
 
 void i915_gem_object_init_memory_region(struct drm_i915_gem_object *obj,
-					struct intel_memory_region *mem,
-					unsigned long flags);
+					struct intel_memory_region *mem);
 void i915_gem_object_release_memory_region(struct drm_i915_gem_object *obj);
 
 struct drm_i915_gem_object *
drivers/gpu/drm/i915/gem/i915_gem_shmem.c +4 -4
index 75e8b71c18b9..31c617a1115f 100644
@@ -430,8 +430,7 @@ static void shmem_release(struct drm_i915_gem_object *obj)
 
 const struct drm_i915_gem_object_ops i915_gem_shmem_ops = {
 	.name = "i915_gem_object_shmem",
-	.flags = I915_GEM_OBJECT_HAS_STRUCT_PAGE |
-		 I915_GEM_OBJECT_IS_SHRINKABLE,
+	.flags = I915_GEM_OBJECT_IS_SHRINKABLE,
 
 	.get_pages = shmem_get_pages,
 	.put_pages = shmem_put_pages,
@@ -496,7 +495,8 @@ create_shmem(struct intel_memory_region *mem,
 	mapping_set_gfp_mask(mapping, mask);
 	GEM_BUG_ON(!(mapping_gfp_mask(mapping) & __GFP_RECLAIM));
 
-	i915_gem_object_init(obj, &i915_gem_shmem_ops, &lock_class);
+	i915_gem_object_init(obj, &i915_gem_shmem_ops, &lock_class,
+			     I915_BO_ALLOC_STRUCT_PAGE);
 
 	obj->write_domain = I915_GEM_DOMAIN_CPU;
 	obj->read_domains = I915_GEM_DOMAIN_CPU;
@@ -520,7 +520,7 @@ create_shmem(struct intel_memory_region *mem,
 
 	i915_gem_object_set_cache_coherency(obj, cache_level);
 
-	i915_gem_object_init_memory_region(obj, mem, 0);
+	i915_gem_object_init_memory_region(obj, mem);
 
 	return obj;
 
drivers/gpu/drm/i915/gem/i915_gem_stolen.c +2 -2
index 29bffc6afcc1..5372b888ba01 100644
@@ -636,7 +636,7 @@ __i915_gem_object_create_stolen(struct intel_memory_region *mem,
 		goto err;
 
 	drm_gem_private_object_init(&mem->i915->drm, &obj->base, stolen->size);
-	i915_gem_object_init(obj, &i915_gem_object_stolen_ops, &lock_class);
+	i915_gem_object_init(obj, &i915_gem_object_stolen_ops, &lock_class, 0);
 
 	obj->stolen = stolen;
 	obj->read_domains = I915_GEM_DOMAIN_CPU | I915_GEM_DOMAIN_GTT;
@@ -647,7 +647,7 @@ __i915_gem_object_create_stolen(struct intel_memory_region *mem,
 	if (err)
 		goto cleanup;
 
-	i915_gem_object_init_memory_region(obj, mem, 0);
+	i915_gem_object_init_memory_region(obj, mem);
 
 	return obj;
 
drivers/gpu/drm/i915/gem/i915_gem_userptr.c +3 -3
index f2eaed6aca3d..30edc5a0a54e 100644
@@ -702,8 +702,7 @@ i915_gem_userptr_dmabuf_export(struct drm_i915_gem_object *obj)
 
 static const struct drm_i915_gem_object_ops i915_gem_userptr_ops = {
 	.name = "i915_gem_object_userptr",
-	.flags = I915_GEM_OBJECT_HAS_STRUCT_PAGE |
-		 I915_GEM_OBJECT_IS_SHRINKABLE |
+	.flags = I915_GEM_OBJECT_IS_SHRINKABLE |
 		 I915_GEM_OBJECT_NO_MMAP |
 		 I915_GEM_OBJECT_ASYNC_CANCEL,
 	.get_pages = i915_gem_userptr_get_pages,
@@ -810,7 +809,8 @@ i915_gem_userptr_ioctl(struct drm_device *dev,
 		return -ENOMEM;
 
 	drm_gem_private_object_init(dev, &obj->base, args->user_size);
-	i915_gem_object_init(obj, &i915_gem_userptr_ops, &lock_class);
+	i915_gem_object_init(obj, &i915_gem_userptr_ops, &lock_class,
+			     I915_BO_ALLOC_STRUCT_PAGE);
 	obj->read_domains = I915_GEM_DOMAIN_CPU;
 	obj->write_domain = I915_GEM_DOMAIN_CPU;
 	i915_gem_object_set_cache_coherency(obj, I915_CACHE_LLC);
drivers/gpu/drm/i915/gem/selftests/huge_gem_object.c +2 -2
index a768ec61e966..dfad86d74dd0 100644
@@ -89,7 +89,6 @@ static void huge_put_pages(struct drm_i915_gem_object *obj,
 
 static const struct drm_i915_gem_object_ops huge_ops = {
 	.name = "huge-gem",
-	.flags = I915_GEM_OBJECT_HAS_STRUCT_PAGE,
 	.get_pages = huge_get_pages,
 	.put_pages = huge_put_pages,
 };
@@ -115,7 +114,8 @@ huge_gem_object(struct drm_i915_private *i915,
 		return ERR_PTR(-ENOMEM);
 
 	drm_gem_private_object_init(&i915->drm, &obj->base, dma_size);
-	i915_gem_object_init(obj, &huge_ops, &lock_class);
+	i915_gem_object_init(obj, &huge_ops, &lock_class,
+			     I915_BO_ALLOC_STRUCT_PAGE);
 
 	obj->read_domains = I915_GEM_DOMAIN_CPU;
 	obj->write_domain = I915_GEM_DOMAIN_CPU;
drivers/gpu/drm/i915/gem/selftests/huge_pages.c +5 -5
index 77a13527a7e6..709c63b9cfc4 100644
@@ -140,8 +140,7 @@ static void put_huge_pages(struct drm_i915_gem_object *obj,
 
 static const struct drm_i915_gem_object_ops huge_page_ops = {
 	.name = "huge-gem",
-	.flags = I915_GEM_OBJECT_HAS_STRUCT_PAGE |
-		 I915_GEM_OBJECT_IS_SHRINKABLE,
+	.flags = I915_GEM_OBJECT_IS_SHRINKABLE,
 	.get_pages = get_huge_pages,
 	.put_pages = put_huge_pages,
 };
@@ -168,7 +167,8 @@ huge_pages_object(struct drm_i915_private *i915,
 		return ERR_PTR(-ENOMEM);
 
 	drm_gem_private_object_init(&i915->drm, &obj->base, size);
-	i915_gem_object_init(obj, &huge_page_ops, &lock_class);
+	i915_gem_object_init(obj, &huge_page_ops, &lock_class,
+			     I915_BO_ALLOC_STRUCT_PAGE);
 
 	i915_gem_object_set_volatile(obj);
 
@@ -319,9 +319,9 @@ fake_huge_pages_object(struct drm_i915_private *i915, u64 size, bool single)
 	drm_gem_private_object_init(&i915->drm, &obj->base, size);
 
 	if (single)
-		i915_gem_object_init(obj, &fake_ops_single, &lock_class);
+		i915_gem_object_init(obj, &fake_ops_single, &lock_class, 0);
 	else
-		i915_gem_object_init(obj, &fake_ops, &lock_class);
+		i915_gem_object_init(obj, &fake_ops, &lock_class, 0);
 
 	i915_gem_object_set_volatile(obj);
 
drivers/gpu/drm/i915/gem/selftests/i915_gem_mman.c +4 -7
index d27d87a678c8..3ac7628f3bc4 100644
@@ -834,9 +834,8 @@ static bool can_mmap(struct drm_i915_gem_object *obj, enum i915_mmap_type type)
 		return false;
 
 	if (type != I915_MMAP_TYPE_GTT &&
-	    !i915_gem_object_type_has(obj,
-				      I915_GEM_OBJECT_HAS_STRUCT_PAGE |
-				      I915_GEM_OBJECT_HAS_IOMEM))
+	    !i915_gem_object_has_struct_page(obj) &&
+	    !i915_gem_object_type_has(obj, I915_GEM_OBJECT_HAS_IOMEM))
 		return false;
 
 	return true;
@@ -976,10 +975,8 @@ static const char *repr_mmap_type(enum i915_mmap_type type)
 
 static bool can_access(const struct drm_i915_gem_object *obj)
 {
-	unsigned int flags =
-		I915_GEM_OBJECT_HAS_STRUCT_PAGE | I915_GEM_OBJECT_HAS_IOMEM;
-
-	return i915_gem_object_type_has(obj, flags);
+	return i915_gem_object_has_struct_page(obj) ||
+	       i915_gem_object_type_has(obj, I915_GEM_OBJECT_HAS_IOMEM);
 }
 
 static int __igt_mmap_access(struct drm_i915_private *i915,
drivers/gpu/drm/i915/gem/selftests/i915_gem_phys.c +12 -0
index 8cee68c6a6dc..fb6a17701310 100644
@@ -25,12 +25,24 @@ static int mock_phys_object(void *arg)
 		goto out;
 	}
 
+	if (!i915_gem_object_has_struct_page(obj)) {
+		err = -EINVAL;
+		pr_err("shmem has no struct page\n");
+		goto out_obj;
+	}
+
 	err = i915_gem_object_attach_phys(obj, PAGE_SIZE);
 	if (err) {
 		pr_err("i915_gem_object_attach_phys failed, err=%d\n", err);
 		goto out_obj;
 	}
 
+	if (i915_gem_object_has_struct_page(obj)) {
+		err = -EINVAL;
+		pr_err("shmem has a struct page\n");
+		goto out_obj;
+	}
+
 	if (obj->ops != &i915_gem_phys_ops) {
 		pr_err("i915_gem_object_attach_phys did not create a phys object\n");
 		err = -EINVAL;
drivers/gpu/drm/i915/gvt/dmabuf.c +1 -1
index c3eb3838fe88..d4f883f35b95 100644
@@ -218,7 +218,7 @@ static struct drm_i915_gem_object *vgpu_create_gem(struct drm_device *dev,
 
 	drm_gem_private_object_init(dev, &obj->base,
 		roundup(info->size, PAGE_SIZE));
-	i915_gem_object_init(obj, &intel_vgpu_gem_ops, &lock_class);
+	i915_gem_object_init(obj, &intel_vgpu_gem_ops, &lock_class, 0);
 	i915_gem_object_set_readonly(obj);
 
 	obj->read_domains = I915_GEM_DOMAIN_GTT;
drivers/gpu/drm/i915/selftests/i915_gem_gtt.c +1 -1
index c53a222e3dec..2cfe99c79034 100644
@@ -120,7 +120,7 @@ fake_dma_object(struct drm_i915_private *i915, u64 size)
 		goto err;
 
 	drm_gem_private_object_init(&i915->drm, &obj->base, size);
-	i915_gem_object_init(obj, &fake_ops, &lock_class);
+	i915_gem_object_init(obj, &fake_ops, &lock_class, 0);
 
 	i915_gem_object_set_volatile(obj);
 
drivers/gpu/drm/i915/selftests/mock_region.c +2 -2
index 979d96f27c43..b046bd1a9ad3 100644
@@ -32,13 +32,13 @@ mock_object_create(struct intel_memory_region *mem,
 		return ERR_PTR(-ENOMEM);
 
 	drm_gem_private_object_init(&i915->drm, &obj->base, size);
-	i915_gem_object_init(obj, &mock_region_obj_ops, &lock_class);
+	i915_gem_object_init(obj, &mock_region_obj_ops, &lock_class, flags);
 
 	obj->read_domains = I915_GEM_DOMAIN_CPU | I915_GEM_DOMAIN_GTT;
 
 	i915_gem_object_set_cache_coherency(obj, I915_CACHE_NONE);
 
-	i915_gem_object_init_memory_region(obj, mem, flags);
+	i915_gem_object_init_memory_region(obj, mem);
 
 	return obj;
 }
Signature · 6 lines
2.26.2

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
#22 lore raw

RFC PATCH 18/162 drm/i915: Convert i915_gem_object_attach_phys() to ww locking, v2.

In reply to Matthew Auld

From: Maarten Lankhorst <[email protected]>

Simple adding of i915_gem_object_lock, we may start to pass ww to get_pages() in the future, but that won't be the case here; We override shmem's get_pages() handling by calling i915_gem_object_get_pages_phys(), no ww is needed.

---
 drivers/gpu/drm/i915/gem/i915_gem_object.h |  2 ++
 drivers/gpu/drm/i915/gem/i915_gem_phys.c   | 12 ++++++++++--
 drivers/gpu/drm/i915/gem/i915_gem_shmem.c  | 17 ++++++++++-------
 3 files changed, 22 insertions(+), 9 deletions(-)
drivers/gpu/drm/i915/gem/i915_gem_object.h +2 -0
index e549b88693a2..47da3aff2a79 100644
@@ -43,6 +43,8 @@ int i915_gem_object_pread_phys(struct drm_i915_gem_object *obj,
 			       const struct drm_i915_gem_pread *args);
 
 int i915_gem_object_attach_phys(struct drm_i915_gem_object *obj, int align);
+void i915_gem_object_put_pages_shmem(struct drm_i915_gem_object *obj,
+				     struct sg_table *pages);
 void i915_gem_object_put_pages_phys(struct drm_i915_gem_object *obj,
 				    struct sg_table *pages);
 
drivers/gpu/drm/i915/gem/i915_gem_phys.c +10 -2
index 4bdd0429c08b..144e4940eede 100644
@@ -201,7 +201,7 @@ static int i915_gem_object_shmem_to_phys(struct drm_i915_gem_object *obj)
 	__i915_gem_object_pin_pages(obj);
 
 	if (!IS_ERR_OR_NULL(pages))
-		i915_gem_shmem_ops.put_pages(obj, pages);
+		i915_gem_object_put_pages_shmem(obj, pages);
 
 	i915_gem_object_release_memory_region(obj);
 	return 0;
@@ -232,7 +232,13 @@ int i915_gem_object_attach_phys(struct drm_i915_gem_object *obj, int align)
 	if (err)
 		return err;
 
-	mutex_lock_nested(&obj->mm.lock, I915_MM_GET_PAGES);
+	err = i915_gem_object_lock_interruptible(obj, NULL);
+	if (err)
+		return err;
+
+	err = mutex_lock_interruptible_nested(&obj->mm.lock, I915_MM_GET_PAGES);
+	if (err)
+		goto err_unlock;
 
 	if (unlikely(!i915_gem_object_has_struct_page(obj)))
 		goto out;
@@ -263,6 +269,8 @@ int i915_gem_object_attach_phys(struct drm_i915_gem_object *obj, int align)
 
 out:
 	mutex_unlock(&obj->mm.lock);
+err_unlock:
+	i915_gem_object_unlock(obj);
 	return err;
 }
 
drivers/gpu/drm/i915/gem/i915_gem_shmem.c +10 -7
index d590e0c3bd00..7a59fd1ea4e5 100644
@@ -296,18 +296,12 @@ __i915_gem_object_release_shmem(struct drm_i915_gem_object *obj,
 	__start_cpu_write(obj);
 }
 
-static void
-shmem_put_pages(struct drm_i915_gem_object *obj, struct sg_table *pages)
+void i915_gem_object_put_pages_shmem(struct drm_i915_gem_object *obj, struct sg_table *pages)
 {
 	struct sgt_iter sgt_iter;
 	struct pagevec pvec;
 	struct page *page;
 
-	if (unlikely(!i915_gem_object_has_struct_page(obj))) {
-		i915_gem_object_put_pages_phys(obj, pages);
-		return;
-	}
-
 	__i915_gem_object_release_shmem(obj, pages, true);
 
 	i915_gem_gtt_finish_pages(obj, pages);
@@ -336,6 +330,15 @@ shmem_put_pages(struct drm_i915_gem_object *obj, struct sg_table *pages)
 	kfree(pages);
 }
 
+static void
+shmem_put_pages(struct drm_i915_gem_object *obj, struct sg_table *pages)
+{
+	if (likely(i915_gem_object_has_struct_page(obj)))
+		i915_gem_object_put_pages_shmem(obj, pages);
+	else
+		i915_gem_object_put_pages_phys(obj, pages);
+}
+
 static int
 shmem_pwrite(struct drm_i915_gem_object *obj,
 	     const struct drm_i915_gem_pwrite *arg)
Signature · 6 lines
2.26.2

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
#25 lore raw

RFC PATCH 21/162 drm/i915: No longer allow exporting userptr through dma-buf

In reply to Matthew Auld

From: Maarten Lankhorst <[email protected]>

It doesn't make sense to export a memory address, we will prevent allowing access this way to different address spaces when we rework userptr handling, so best to explicitly disable it.

---
 drivers/gpu/drm/i915/gem/i915_gem_userptr.c | 5 ++---
 1 file changed, 2 insertions(+), 3 deletions(-)
drivers/gpu/drm/i915/gem/i915_gem_userptr.c +2 -3
index 8c3d1eb2f96a..44af6265948d 100644
@@ -694,10 +694,9 @@ i915_gem_userptr_release(struct drm_i915_gem_object *obj)
 static int
 i915_gem_userptr_dmabuf_export(struct drm_i915_gem_object *obj)
 {
-	if (obj->userptr.mmu_object)
-		return 0;
+	drm_dbg(obj->base.dev, "Exporting userptr no longer allowed\n");
 
-	return i915_gem_userptr_init__mmu_notifier(obj, 0);
+	return -EINVAL;
 }
 
 static int
Signature · 6 lines
2.26.2

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
#26 lore raw

RFC PATCH 22/162 drm/i915: Reject more ioctls for userptr

In reply to Matthew Auld

From: Maarten Lankhorst <[email protected]>

There are a couple of ioctl's related to tiling and cache placement, that make no sense for userptr, reject those:
- i915_gem_set_tiling_ioctl()
Tiling should always be linear for userptr. Changing placement will fail with -ENXIO.
- i915_gem_set_caching_ioctl()
Userptr memory should always be cached. Changing will fail with -ENXIO.
- i915_gem_set_domain_ioctl()
Changed to be equivalent to gem_wait, which is correct for the cached linear userptr pointers. This is required because we cannot grab a reference to the pages in the rework, but waiting for idle will do the same.
Still needs an ack from relevant userspace that it won't break, but should be good.

---
 drivers/gpu/drm/i915/display/intel_display.c | 2 +-
 drivers/gpu/drm/i915/gem/i915_gem_domain.c   | 4 +++-
 drivers/gpu/drm/i915/gem/i915_gem_object.h   | 6 ++++++
 drivers/gpu/drm/i915/gem/i915_gem_userptr.c  | 3 ++-
 4 files changed, 12 insertions(+), 3 deletions(-)
drivers/gpu/drm/i915/display/intel_display.c +1 -1
index ba26545392bc..f36921a3c4bc 100644
@@ -17854,7 +17854,7 @@ static int intel_user_framebuffer_create_handle(struct drm_framebuffer *fb,
 	struct drm_i915_gem_object *obj = intel_fb_obj(fb);
 	struct drm_i915_private *i915 = to_i915(obj->base.dev);
 
-	if (obj->userptr.mm) {
+	if (i915_gem_object_is_userptr(obj)) {
 		drm_dbg(&i915->drm,
 			"attempting to use a userptr for a framebuffer, denied\n");
 		return -EINVAL;
drivers/gpu/drm/i915/gem/i915_gem_domain.c +3 -1
index fcce6909f201..c1d4bf62b3ea 100644
@@ -528,7 +528,9 @@ i915_gem_set_domain_ioctl(struct drm_device *dev, void *data,
 	 * considered to be outside of any cache domain.
 	 */
 	if (i915_gem_object_is_proxy(obj)) {
-		err = -ENXIO;
+		/* silently allow userptr to complete */
+		if (!i915_gem_object_is_userptr(obj))
+			err = -ENXIO;
 		goto out;
 	}
 
drivers/gpu/drm/i915/gem/i915_gem_object.h +6 -0
index 47da3aff2a79..95907b8eb4c4 100644
@@ -551,6 +551,12 @@ void __i915_gem_object_flush_frontbuffer(struct drm_i915_gem_object *obj,
 void __i915_gem_object_invalidate_frontbuffer(struct drm_i915_gem_object *obj,
 					      enum fb_op_origin origin);
 
+static inline bool
+i915_gem_object_is_userptr(struct drm_i915_gem_object *obj)
+{
+	return obj->userptr.mm;
+}
+
 static inline void
 i915_gem_object_flush_frontbuffer(struct drm_i915_gem_object *obj,
 				  enum fb_op_origin origin)
drivers/gpu/drm/i915/gem/i915_gem_userptr.c +2 -1
index 44af6265948d..64a946d5f753 100644
@@ -721,7 +721,8 @@ static const struct drm_i915_gem_object_ops i915_gem_userptr_ops = {
 	.name = "i915_gem_object_userptr",
 	.flags = I915_GEM_OBJECT_IS_SHRINKABLE |
 		 I915_GEM_OBJECT_NO_MMAP |
-		 I915_GEM_OBJECT_ASYNC_CANCEL,
+		 I915_GEM_OBJECT_ASYNC_CANCEL |
+		 I915_GEM_OBJECT_IS_PROXY,
 	.get_pages = i915_gem_userptr_get_pages,
 	.put_pages = i915_gem_userptr_put_pages,
 	.dmabuf_export = i915_gem_userptr_dmabuf_export,
Signature · 6 lines
2.26.2

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
#24 lore raw

RFC PATCH 20/162 drm/i915: Disable userptr pread/pwrite support.

In reply to Matthew Auld

From: Maarten Lankhorst <[email protected]>

Userptr should not need the kernel for a userspace memcpy, userspace needs to call memcpy directly.

Specifically, disable i915_gem_pwrite_ioctl() and i915_gem_pread_ioctl().

Still needs an ack from relevant userspace that it won't break, but should be good.

---
 drivers/gpu/drm/i915/gem/i915_gem_userptr.c | 20 ++++++++++++++++++++
 1 file changed, 20 insertions(+)
drivers/gpu/drm/i915/gem/i915_gem_userptr.c +20 -0
index 30edc5a0a54e..8c3d1eb2f96a 100644
@@ -700,6 +700,24 @@ i915_gem_userptr_dmabuf_export(struct drm_i915_gem_object *obj)
 	return i915_gem_userptr_init__mmu_notifier(obj, 0);
 }
 
+static int
+i915_gem_userptr_pwrite(struct drm_i915_gem_object *obj,
+			const struct drm_i915_gem_pwrite *args)
+{
+	drm_dbg(obj->base.dev, "pwrite to userptr no longer allowed\n");
+
+	return -EINVAL;
+}
+
+static int
+i915_gem_userptr_pread(struct drm_i915_gem_object *obj,
+		       const struct drm_i915_gem_pread *args)
+{
+	drm_dbg(obj->base.dev, "pread from userptr no longer allowed\n");
+
+	return -EINVAL;
+}
+
 static const struct drm_i915_gem_object_ops i915_gem_userptr_ops = {
 	.name = "i915_gem_object_userptr",
 	.flags = I915_GEM_OBJECT_IS_SHRINKABLE |
@@ -708,6 +726,8 @@ static const struct drm_i915_gem_object_ops i915_gem_userptr_ops = {
 	.get_pages = i915_gem_userptr_get_pages,
 	.put_pages = i915_gem_userptr_put_pages,
 	.dmabuf_export = i915_gem_userptr_dmabuf_export,
+	.pwrite = i915_gem_userptr_pwrite,
+	.pread = i915_gem_userptr_pread,
 	.release = i915_gem_userptr_release,
 };
 
Signature · 6 lines
2.26.2

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel
#29 lore raw

RFC PATCH 25/162 drm/i915: Fix userptr so we do not have to worry about obj->mm.lock, v5.

In reply to Matthew Auld

From: Maarten Lankhorst <[email protected]>

Instead of doing what we do currently, which will never work with PROVE_LOCKING, do the same as AMD does, and something similar to relocation slowpath. When all locks are dropped, we acquire the pages for pinning. When the locks are taken, we transfer those pages in .get_pages() to the bo. As a final check before installing the fences, we ensure that the mmu notifier was not called; if it is, we return -EAGAIN to userspace to signal it has to start over.

---
 .../gpu/drm/i915/gem/i915_gem_execbuffer.c    | 101 ++-
 drivers/gpu/drm/i915/gem/i915_gem_object.h    |  35 +-
 .../gpu/drm/i915/gem/i915_gem_object_types.h  |  10 +-
 drivers/gpu/drm/i915/gem/i915_gem_pages.c     |   2 +-
 drivers/gpu/drm/i915/gem/i915_gem_userptr.c   | 764 ++++++------------
 drivers/gpu/drm/i915/i915_drv.h               |   9 +-
 drivers/gpu/drm/i915/i915_gem.c               |   5 +-
 7 files changed, 344 insertions(+), 582 deletions(-)
drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c +88 -13
index 064285a5009b..f5ea49e244ca 100644
@@ -52,14 +52,16 @@ enum {
 /* __EXEC_OBJECT_NO_RESERVE is BIT(31), defined in i915_vma.h */
 #define __EXEC_OBJECT_HAS_PIN		BIT(30)
 #define __EXEC_OBJECT_HAS_FENCE		BIT(29)
-#define __EXEC_OBJECT_NEEDS_MAP		BIT(28)
-#define __EXEC_OBJECT_NEEDS_BIAS	BIT(27)
-#define __EXEC_OBJECT_INTERNAL_FLAGS	(~0u << 27) /* all of the above + */
+#define __EXEC_OBJECT_USERPTR_INIT	BIT(28)
+#define __EXEC_OBJECT_NEEDS_MAP		BIT(27)
+#define __EXEC_OBJECT_NEEDS_BIAS	BIT(26)
+#define __EXEC_OBJECT_INTERNAL_FLAGS	(~0u << 26) /* all of the above + */
 #define __EXEC_OBJECT_RESERVED (__EXEC_OBJECT_HAS_PIN | __EXEC_OBJECT_HAS_FENCE)
 
 #define __EXEC_HAS_RELOC	BIT(31)
 #define __EXEC_ENGINE_PINNED	BIT(30)
-#define __EXEC_INTERNAL_FLAGS	(~0u << 30)
+#define __EXEC_USERPTR_USED	BIT(29)
+#define __EXEC_INTERNAL_FLAGS	(~0u << 29)
 #define UPDATE			PIN_OFFSET_FIXED
 
 #define BATCH_OFFSET_BIAS (256*1024)
@@ -865,6 +867,26 @@ static int eb_lookup_vmas(struct i915_execbuffer *eb)
 		}
 
 		eb_add_vma(eb, i, batch, vma);
+
+		if (i915_gem_object_is_userptr(vma->obj)) {
+			err = i915_gem_object_userptr_submit_init(vma->obj);
+			if (err) {
+				if (i + 1 < eb->buffer_count) {
+					/*
+					 * Execbuffer code expects last vma entry to be NULL,
+					 * since we already initialized this entry,
+					 * set the next value to NULL or we mess up
+					 * cleanup handling.
+					 */
+					eb->vma[i + 1].vma = NULL;
+				}
+
+				return err;
+			}
+
+			eb->vma[i].flags |= __EXEC_OBJECT_USERPTR_INIT;
+			eb->args->flags |= __EXEC_USERPTR_USED;
+		}
 	}
 
 	if (unlikely(eb->batch->flags & EXEC_OBJECT_WRITE)) {
@@ -966,7 +988,7 @@ eb_get_vma(const struct i915_execbuffer *eb, unsigned long handle)
 	}
 }
 
-static void eb_release_vmas(struct i915_execbuffer *eb, bool final)
+static void eb_release_vmas(struct i915_execbuffer *eb, bool final, bool release_userptr)
 {
 	const unsigned int count = eb->buffer_count;
 	unsigned int i;
@@ -980,6 +1002,11 @@ static void eb_release_vmas(struct i915_execbuffer *eb, bool final)
 
 		eb_unreserve_vma(ev);
 
+		if (release_userptr && ev->flags & __EXEC_OBJECT_USERPTR_INIT) {
+			ev->flags &= ~__EXEC_OBJECT_USERPTR_INIT;
+			i915_gem_object_userptr_submit_fini(vma->obj);
+		}
+
 		if (final)
 			i915_vma_put(vma);
 	}
@@ -1915,6 +1942,31 @@ static int eb_prefault_relocations(const struct i915_execbuffer *eb)
 	return 0;
 }
 
+static int eb_reinit_userptr(struct i915_execbuffer *eb)
+{
+	const unsigned int count = eb->buffer_count;
+	unsigned int i;
+	int ret;
+
+	if (likely(!(eb->args->flags & __EXEC_USERPTR_USED)))
+		return 0;
+
+	for (i = 0; i < count; i++) {
+		struct eb_vma *ev = &eb->vma[i];
+
+		if (!i915_gem_object_is_userptr(ev->vma->obj))
+			continue;
+
+		ret = i915_gem_object_userptr_submit_init(ev->vma->obj);
+		if (ret)
+			return ret;
+
+		ev->flags |= __EXEC_OBJECT_USERPTR_INIT;
+	}
+
+	return 0;
+}
+
 static noinline int eb_relocate_parse_slow(struct i915_execbuffer *eb,
 					   struct i915_request *rq)
 {
@@ -1929,7 +1981,7 @@ static noinline int eb_relocate_parse_slow(struct i915_execbuffer *eb,
 	}
 
 	/* We may process another execbuffer during the unlock... */
-	eb_release_vmas(eb, false);
+	eb_release_vmas(eb, false, true);
 	i915_gem_ww_ctx_fini(&eb->ww);
 
 	if (rq) {
@@ -1970,10 +2022,8 @@ static noinline int eb_relocate_parse_slow(struct i915_execbuffer *eb,
 		err = 0;
 	}
 
-#ifdef CONFIG_MMU_NOTIFIER
 	if (!err)
-		flush_workqueue(eb->i915->mm.userptr_wq);
-#endif
+		err = eb_reinit_userptr(eb);
 
 err_relock:
 	i915_gem_ww_ctx_init(&eb->ww, true);
@@ -2035,7 +2085,7 @@ static noinline int eb_relocate_parse_slow(struct i915_execbuffer *eb,
 
 err:
 	if (err == -EDEADLK) {
-		eb_release_vmas(eb, false);
+		eb_release_vmas(eb, false, false);
 		err = i915_gem_ww_ctx_backoff(&eb->ww);
 		if (!err)
 			goto repeat_validate;
@@ -2132,7 +2182,7 @@ static int eb_relocate_parse(struct i915_execbuffer *eb)
 
 err:
 	if (err == -EDEADLK) {
-		eb_release_vmas(eb, false);
+		eb_release_vmas(eb, false, false);
 		err = i915_gem_ww_ctx_backoff(&eb->ww);
 		if (!err)
 			goto retry;
@@ -2207,6 +2257,30 @@ static int eb_move_to_gpu(struct i915_execbuffer *eb)
 						      flags | __EXEC_OBJECT_NO_RESERVE);
 	}
 
+#ifdef CONFIG_MMU_NOTIFIER
+	if (!err && (eb->args->flags & __EXEC_USERPTR_USED)) {
+		spin_lock(&eb->i915->mm.notifier_lock);
+
+		/*
+		 * count is always at least 1, otherwise __EXEC_USERPTR_USED
+		 * could not have been set
+		 */
+		for (i = 0; i < count; i++) {
+			struct eb_vma *ev = &eb->vma[i];
+			struct drm_i915_gem_object *obj = ev->vma->obj;
+
+			if (!i915_gem_object_is_userptr(obj))
+				continue;
+
+			err = i915_gem_object_userptr_submit_done(obj);
+			if (err)
+				break;
+		}
+
+		spin_unlock(&eb->i915->mm.notifier_lock);
+	}
+#endif
+
 	if (unlikely(err))
 		goto err_skip;
 
@@ -3347,7 +3421,7 @@ i915_gem_do_execbuffer(struct drm_device *dev,
 
 	err = eb_lookup_vmas(&eb);
 	if (err) {
-		eb_release_vmas(&eb, true);
+		eb_release_vmas(&eb, true, true);
 		goto err_engine;
 	}
 
@@ -3419,6 +3493,7 @@ i915_gem_do_execbuffer(struct drm_device *dev,
 
 	trace_i915_request_queue(eb.request, eb.batch_flags);
 	err = eb_submit(&eb, batch);
+
 err_request:
 	i915_request_get(eb.request);
 	eb_request_add(&eb);
@@ -3439,7 +3514,7 @@ i915_gem_do_execbuffer(struct drm_device *dev,
 	i915_request_put(eb.request);
 
 err_vma:
-	eb_release_vmas(&eb, true);
+	eb_release_vmas(&eb, true, true);
 	if (eb.trampoline)
 		i915_vma_unpin(eb.trampoline);
 	WARN_ON(err == -EDEADLK);
drivers/gpu/drm/i915/gem/i915_gem_object.h +19 -16
index 7b3a84f98b42..33412248f6df 100644
@@ -33,6 +33,7 @@ i915_gem_object_create_shmem_from_data(struct drm_i915_private *i915,
 				       const void *data, resource_size_t size);
 
 extern const struct drm_i915_gem_object_ops i915_gem_shmem_ops;
+
 void __i915_gem_object_release_shmem(struct drm_i915_gem_object *obj,
 				     struct sg_table *pages,
 				     bool needs_clflush);
@@ -245,12 +246,6 @@ i915_gem_object_never_mmap(const struct drm_i915_gem_object *obj)
 	return i915_gem_object_type_has(obj, I915_GEM_OBJECT_NO_MMAP);
 }
 
-static inline bool
-i915_gem_object_needs_async_cancel(const struct drm_i915_gem_object *obj)
-{
-	return i915_gem_object_type_has(obj, I915_GEM_OBJECT_ASYNC_CANCEL);
-}
-
 static inline bool
 i915_gem_object_is_framebuffer(const struct drm_i915_gem_object *obj)
 {
@@ -551,16 +546,6 @@ void __i915_gem_object_flush_frontbuffer(struct drm_i915_gem_object *obj,
 void __i915_gem_object_invalidate_frontbuffer(struct drm_i915_gem_object *obj,
 					      enum fb_op_origin origin);
 
-static inline bool
-i915_gem_object_is_userptr(struct drm_i915_gem_object *obj)
-{
-#ifdef CONFIG_MMU_NOTIFIER
-	return obj->userptr.mm;
-#else
-	return false;
-#endif
-}
-
 static inline void
 i915_gem_object_flush_frontbuffer(struct drm_i915_gem_object *obj,
 				  enum fb_op_origin origin)
@@ -577,4 +562,22 @@ i915_gem_object_invalidate_frontbuffer(struct drm_i915_gem_object *obj,
 		__i915_gem_object_invalidate_frontbuffer(obj, origin);
 }
 
+#ifdef CONFIG_MMU_NOTIFIER
+static inline bool
+i915_gem_object_is_userptr(struct drm_i915_gem_object *obj)
+{
+	return obj->userptr.notifier.mm;
+}
+
+int i915_gem_object_userptr_submit_init(struct drm_i915_gem_object *obj);
+int i915_gem_object_userptr_submit_done(struct drm_i915_gem_object *obj);
+void i915_gem_object_userptr_submit_fini(struct drm_i915_gem_object *obj);
+#else
+static inline bool i915_gem_object_is_userptr(struct drm_i915_gem_object *obj) { return false; }
+
+static inline int i915_gem_object_userptr_submit_init(struct drm_i915_gem_object *obj) { GEM_BUG_ON(1); return -ENODEV; }
+static inline int i915_gem_object_userptr_submit_done(struct drm_i915_gem_object *obj) { GEM_BUG_ON(1); return -ENODEV; }
+static inline void i915_gem_object_userptr_submit_fini(struct drm_i915_gem_object *obj) { GEM_BUG_ON(1); }
+#endif
+
 #endif
drivers/gpu/drm/i915/gem/i915_gem_object_types.h +6 -4
index 6d3f451c15c6..5234c1ed62d4 100644
@@ -7,6 +7,8 @@
 #ifndef __I915_GEM_OBJECT_TYPES_H__
 #define __I915_GEM_OBJECT_TYPES_H__
 
+#include <linux/mmu_notifier.h>
+
 #include <drm/drm_gem.h>
 #include <uapi/drm/i915_drm.h>
 
@@ -34,7 +36,6 @@ struct drm_i915_gem_object_ops {
 #define I915_GEM_OBJECT_IS_SHRINKABLE	BIT(2)
 #define I915_GEM_OBJECT_IS_PROXY	BIT(3)
 #define I915_GEM_OBJECT_NO_MMAP		BIT(4)
-#define I915_GEM_OBJECT_ASYNC_CANCEL	BIT(5)
 
 	/* Interface between the GEM object and its backing storage.
 	 * get_pages() is called once prior to the use of the associated set
@@ -292,10 +293,11 @@ struct drm_i915_gem_object {
 #ifdef CONFIG_MMU_NOTIFIER
 		struct i915_gem_userptr {
 			uintptr_t ptr;
+			unsigned long notifier_seq;
 
-			struct i915_mm_struct *mm;
-			struct i915_mmu_object *mmu_object;
-			struct work_struct *work;
+			struct mmu_interval_notifier notifier;
+			struct page **pvec;
+			int page_ref;
 		} userptr;
 #endif
 
drivers/gpu/drm/i915/gem/i915_gem_pages.c +1 -1
index 7983423237e3..60149cad6080 100644
@@ -223,7 +223,7 @@ int __i915_gem_object_put_pages(struct drm_i915_gem_object *obj)
 	 * get_pages backends we should be better able to handle the
 	 * cancellation of the async task in a more uniform manner.
 	 */
-	if (!pages && !i915_gem_object_needs_async_cancel(obj))
+	if (!pages)
 		pages = ERR_PTR(-EINVAL);
 
 	if (!IS_ERR(pages))
drivers/gpu/drm/i915/gem/i915_gem_userptr.c +226 -538

883 lines not on this page · raw message

drivers/gpu/drm/i915/i915_drv.h +3 -6
index d3c67e17cd02..ce8d5ff8b9f4 100644
@@ -591,11 +591,10 @@ struct i915_gem_mm {
 
 #ifdef CONFIG_MMU_NOTIFIER
 	/**
-	 * Workqueue to fault in userptr pages, flushed by the execbuf
-	 * when required but otherwise left to userspace to try again
-	 * on EAGAIN.
+	 * notifier_lock for mmu notifiers, memory may not be allocated
+	 * while holding this lock.
 	 */
-	struct workqueue_struct *userptr_wq;
+	spinlock_t notifier_lock;
 #endif
 
 	/* shrinker accounting, also useful for userland debugging */
@@ -978,8 +977,6 @@ struct drm_i915_private {
 	struct i915_ggtt ggtt; /* VM representing the global address space */
 
 	struct i915_gem_mm mm;
-	DECLARE_HASHTABLE(mm_structs, 7);
-	spinlock_t mm_lock;
 
 	/* Kernel Modesetting */
 
drivers/gpu/drm/i915/i915_gem.c +1 -4
index b03e245640c0..0b9eab66511c 100644
@@ -1158,10 +1158,8 @@ int i915_gem_init(struct drm_i915_private *dev_priv)
 err_unlock:
 	i915_gem_drain_workqueue(dev_priv);
 
-	if (ret != -EIO) {
+	if (ret != -EIO)
 		intel_uc_cleanup_firmwares(&dev_priv->gt.uc);
-		i915_gem_cleanup_userptr(dev_priv);
-	}
 
 	if (ret == -EIO) {
 		/*
@@ -1220,7 +1218,6 @@ void i915_gem_driver_release(struct drm_i915_private *dev_priv)
 	intel_wa_list_free(&dev_priv->gt_wa_list);
 
 	intel_uc_cleanup_firmwares(&dev_priv->gt.uc);
-	i915_gem_cleanup_userptr(dev_priv);
 
 	i915_gem_drain_freed_objects(dev_priv);
 
Signature · 6 lines
2.26.2

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel
#32 lore raw

RFC PATCH 28/162 drm/i915: Make ring submission compatible with obj->mm.lock removal, v2.

In reply to Matthew Auld

From: Maarten Lankhorst <[email protected]>

We map the initial context during first pin.

This allows us to remove pin_map from state allocation, which saves us a few retry loops. We won't need this until first pin anyway.

intel_ring_submission_setup() is also reworked slightly to do all pinning in a single ww loop.

---
 .../gpu/drm/i915/gt/intel_ring_submission.c   | 184 +++++++++++-------
 1 file changed, 118 insertions(+), 66 deletions(-)
drivers/gpu/drm/i915/gt/intel_ring_submission.c +118 -66
index a41b43f445b8..6b280904db43 100644
@@ -478,6 +478,26 @@ static void ring_context_destroy(struct kref *ref)
 	intel_context_free(ce);
 }
 
+static int ring_context_init_default_state(struct intel_context *ce,
+					   struct i915_gem_ww_ctx *ww)
+{
+	struct drm_i915_gem_object *obj = ce->state->obj;
+	void *vaddr;
+
+	vaddr = i915_gem_object_pin_map(obj, I915_MAP_WB);
+	if (IS_ERR(vaddr))
+		return PTR_ERR(vaddr);
+
+	shmem_read(ce->engine->default_state, 0,
+		   vaddr, ce->engine->context_size);
+
+	i915_gem_object_flush_map(obj);
+	__i915_gem_object_release_map(obj);
+
+	__set_bit(CONTEXT_VALID_BIT, &ce->flags);
+	return 0;
+}
+
 static int ring_context_pre_pin(struct intel_context *ce,
 				struct i915_gem_ww_ctx *ww,
 				void **unused)
@@ -485,6 +505,13 @@ static int ring_context_pre_pin(struct intel_context *ce,
 	struct i915_address_space *vm;
 	int err = 0;
 
+	if (ce->engine->default_state &&
+	    !test_bit(CONTEXT_VALID_BIT, &ce->flags)) {
+		err = ring_context_init_default_state(ce, ww);
+		if (err)
+			return err;
+	}
+
 	vm = vm_alias(ce->vm);
 	if (vm)
 		err = gen6_ppgtt_pin(i915_vm_to_ppgtt((vm)), ww);
@@ -540,22 +567,6 @@ alloc_context_vma(struct intel_engine_cs *engine)
 	if (IS_IVYBRIDGE(i915))
 		i915_gem_object_set_cache_coherency(obj, I915_CACHE_L3_LLC);
 
-	if (engine->default_state) {
-		void *vaddr;
-
-		vaddr = i915_gem_object_pin_map(obj, I915_MAP_WB);
-		if (IS_ERR(vaddr)) {
-			err = PTR_ERR(vaddr);
-			goto err_obj;
-		}
-
-		shmem_read(engine->default_state, 0,
-			   vaddr, engine->context_size);
-
-		i915_gem_object_flush_map(obj);
-		__i915_gem_object_release_map(obj);
-	}
-
 	vma = i915_vma_instance(obj, &engine->gt->ggtt->vm, NULL);
 	if (IS_ERR(vma)) {
 		err = PTR_ERR(vma);
@@ -587,8 +598,6 @@ static int ring_context_alloc(struct intel_context *ce)
 			return PTR_ERR(vma);
 
 		ce->state = vma;
-		if (engine->default_state)
-			__set_bit(CONTEXT_VALID_BIT, &ce->flags);
 	}
 
 	return 0;
@@ -1184,37 +1193,15 @@ static int gen7_ctx_switch_bb_setup(struct intel_engine_cs * const engine,
 	return gen7_setup_clear_gpr_bb(engine, vma);
 }
 
-static int gen7_ctx_switch_bb_init(struct intel_engine_cs *engine)
+static int gen7_ctx_switch_bb_init(struct intel_engine_cs *engine,
+				   struct i915_gem_ww_ctx *ww,
+				   struct i915_vma *vma)
 {
-	struct drm_i915_gem_object *obj;
-	struct i915_vma *vma;
-	int size;
 	int err;
 
-	size = gen7_ctx_switch_bb_setup(engine, NULL /* probe size */);
-	if (size <= 0)
-		return size;
-
-	size = ALIGN(size, PAGE_SIZE);
-	obj = i915_gem_object_create_internal(engine->i915, size);
-	if (IS_ERR(obj))
-		return PTR_ERR(obj);
-
-	vma = i915_vma_instance(obj, engine->gt->vm, NULL);
-	if (IS_ERR(vma)) {
-		err = PTR_ERR(vma);
-		goto err_obj;
-	}
-
-	vma->private = intel_context_create(engine); /* dummy residuals */
-	if (IS_ERR(vma->private)) {
-		err = PTR_ERR(vma->private);
-		goto err_obj;
-	}
-
-	err = i915_vma_pin(vma, 0, 0, PIN_USER | PIN_HIGH);
+	err = i915_vma_pin_ww(vma, ww, 0, 0, PIN_USER | PIN_HIGH);
 	if (err)
-		goto err_private;
+		return err;
 
 	err = i915_vma_sync(vma);
 	if (err)
@@ -1229,17 +1216,53 @@ static int gen7_ctx_switch_bb_init(struct intel_engine_cs *engine)
 
 err_unpin:
 	i915_vma_unpin(vma);
-err_private:
-	intel_context_put(vma->private);
-err_obj:
-	i915_gem_object_put(obj);
 	return err;
 }
 
+static struct i915_vma *gen7_ctx_vma(struct intel_engine_cs *engine)
+{
+	struct drm_i915_gem_object *obj;
+	struct i915_vma *vma;
+	int size, err;
+
+	if (!IS_HASWELL(engine->i915) || engine->class != RENDER_CLASS)
+		return 0;
+
+	err = gen7_ctx_switch_bb_setup(engine, NULL /* probe size */);
+	if (err < 0)
+		return ERR_PTR(err);
+	if (!err)
+		return NULL;
+
+	size = ALIGN(err, PAGE_SIZE);
+
+	obj = i915_gem_object_create_internal(engine->i915, size);
+	if (IS_ERR(obj))
+		return ERR_CAST(obj);
+
+	vma = i915_vma_instance(obj, engine->gt->vm, NULL);
+	if (IS_ERR(vma)) {
+		i915_gem_object_put(obj);
+		return ERR_CAST(vma);
+	}
+
+	vma->private = intel_context_create(engine); /* dummy residuals */
+	if (IS_ERR(vma->private)) {
+		err = PTR_ERR(vma->private);
+		vma->private = NULL;
+		i915_gem_object_put(obj);
+		return ERR_PTR(err);
+	}
+
+	return vma;
+}
+
 int intel_ring_submission_setup(struct intel_engine_cs *engine)
 {
+	struct i915_gem_ww_ctx ww;
 	struct intel_timeline *timeline;
 	struct intel_ring *ring;
+	struct i915_vma *gen7_wa_vma;
 	int err;
 
 	setup_common(engine);
@@ -1270,43 +1293,72 @@ int intel_ring_submission_setup(struct intel_engine_cs *engine)
 	}
 	GEM_BUG_ON(timeline->has_initial_breadcrumb);
 
-	err = intel_timeline_pin(timeline, NULL);
-	if (err)
-		goto err_timeline;
-
 	ring = intel_engine_create_ring(engine, SZ_16K);
 	if (IS_ERR(ring)) {
 		err = PTR_ERR(ring);
-		goto err_timeline_unpin;
+		goto err_timeline;
 	}
 
-	err = intel_ring_pin(ring, NULL);
-	if (err)
-		goto err_ring;
-
 	GEM_BUG_ON(engine->legacy.ring);
 	engine->legacy.ring = ring;
 	engine->legacy.timeline = timeline;
 
-	GEM_BUG_ON(timeline->hwsp_ggtt != engine->status_page.vma);
+	gen7_wa_vma = gen7_ctx_vma(engine);
+	if (IS_ERR(gen7_wa_vma)) {
+		err = PTR_ERR(gen7_wa_vma);
+		goto err_ring;
+	}
 
-	if (IS_HASWELL(engine->i915) && engine->class == RENDER_CLASS) {
-		err = gen7_ctx_switch_bb_init(engine);
+	i915_gem_ww_ctx_init(&ww, false);
+
+retry:
+	err = i915_gem_object_lock(timeline->hwsp_ggtt->obj, &ww);
+	if (!err && gen7_wa_vma)
+		err = i915_gem_object_lock(gen7_wa_vma->obj, &ww);
+	if (!err && engine->legacy.ring->vma->obj)
+		err = i915_gem_object_lock(engine->legacy.ring->vma->obj, &ww);
+	if (!err)
+		err = intel_timeline_pin(timeline, &ww);
+	if (!err) {
+		err = intel_ring_pin(ring, &ww);
 		if (err)
-			goto err_ring_unpin;
+			intel_timeline_unpin(timeline);
 	}
+	if (err)
+		goto out;
+
+	GEM_BUG_ON(timeline->hwsp_ggtt != engine->status_page.vma);
+
+	if (gen7_wa_vma) {
+		err = gen7_ctx_switch_bb_init(engine, &ww, gen7_wa_vma);
+		if (err) {
+			intel_ring_unpin(ring);
+			intel_timeline_unpin(timeline);
+		}
+	}
+
+out:
+	if (err == -EDEADLK) {
+		err = i915_gem_ww_ctx_backoff(&ww);
+		if (!err)
+			goto retry;
+	}
+	i915_gem_ww_ctx_fini(&ww);
+	if (err)
+		goto err_gen7_put;
 
 	/* Finally, take ownership and responsibility for cleanup! */
 	engine->release = ring_release;
 
 	return 0;
 
-err_ring_unpin:
-	intel_ring_unpin(ring);
+err_gen7_put:
+	if (gen7_wa_vma) {
+		intel_context_put(gen7_wa_vma->private);
+		i915_gem_object_put(gen7_wa_vma->obj);
+	}
 err_ring:
 	intel_ring_put(ring);
-err_timeline_unpin:
-	intel_timeline_unpin(timeline);
 err_timeline:
 	intel_timeline_put(timeline);
 err:
Signature · 6 lines
2.26.2

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel
#33 lore raw

RFC PATCH 29/162 drm/i915: Handle ww locking in init_status_page

In reply to Matthew Auld

From: Maarten Lankhorst <[email protected]>

Try to pin to ggtt first, and use a full ww loop to handle eviction correctly.

---
 drivers/gpu/drm/i915/gt/intel_engine_cs.c | 37 +++++++++++++++--------
 1 file changed, 24 insertions(+), 13 deletions(-)
drivers/gpu/drm/i915/gt/intel_engine_cs.c +24 -13
index 97ceaf7116e8..420c6a35f3ed 100644
@@ -618,6 +618,7 @@ static void cleanup_status_page(struct intel_engine_cs *engine)
 }
 
 static int pin_ggtt_status_page(struct intel_engine_cs *engine,
+				struct i915_gem_ww_ctx *ww,
 				struct i915_vma *vma)
 {
 	unsigned int flags;
@@ -638,12 +639,13 @@ static int pin_ggtt_status_page(struct intel_engine_cs *engine,
 	else
 		flags = PIN_HIGH;
 
-	return i915_ggtt_pin(vma, NULL, 0, flags);
+	return i915_ggtt_pin(vma, ww, 0, flags);
 }
 
 static int init_status_page(struct intel_engine_cs *engine)
 {
 	struct drm_i915_gem_object *obj;
+	struct i915_gem_ww_ctx ww;
 	struct i915_vma *vma;
 	void *vaddr;
 	int ret;
@@ -667,30 +669,39 @@ static int init_status_page(struct intel_engine_cs *engine)
 	vma = i915_vma_instance(obj, &engine->gt->ggtt->vm, NULL);
 	if (IS_ERR(vma)) {
 		ret = PTR_ERR(vma);
-		goto err;
+		goto err_put;
 	}
 
+	i915_gem_ww_ctx_init(&ww, true);
+retry:
+	ret = i915_gem_object_lock(obj, &ww);
+	if (!ret && !HWS_NEEDS_PHYSICAL(engine->i915))
+		ret = pin_ggtt_status_page(engine, &ww, vma);
+	if (ret)
+		goto err;
+
 	vaddr = i915_gem_object_pin_map(obj, I915_MAP_WB);
 	if (IS_ERR(vaddr)) {
 		ret = PTR_ERR(vaddr);
-		goto err;
+		goto err_unpin;
 	}
 
 	engine->status_page.addr = memset(vaddr, 0, PAGE_SIZE);
 	engine->status_page.vma = vma;
 
-	if (!HWS_NEEDS_PHYSICAL(engine->i915)) {
-		ret = pin_ggtt_status_page(engine, vma);
-		if (ret)
-			goto err_unpin;
-	}
-
-	return 0;
-
 err_unpin:
-	i915_gem_object_unpin_map(obj);
+	if (ret)
+		i915_vma_unpin(vma);
 err:
-	i915_gem_object_put(obj);
+	if (ret == -EDEADLK) {
+		ret = i915_gem_ww_ctx_backoff(&ww);
+		if (!ret)
+			goto retry;
+	}
+	i915_gem_ww_ctx_fini(&ww);
+err_put:
+	if (ret)
+		i915_gem_object_put(obj);
 	return ret;
 }
 
Signature · 6 lines
2.26.2

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel
#35 lore raw

RFC PATCH 31/162 drm/i915: Pass ww ctx to intel_pin_to_display_plane

In reply to Matthew Auld

From: Maarten Lankhorst <[email protected]>

Instead of multiple lockings, lock the object once, and perform the ww dance around attach_phys and pin_pages.

---
 drivers/gpu/drm/i915/display/intel_display.c  | 69 ++++++++++++-------
 drivers/gpu/drm/i915/display/intel_display.h  |  2 +-
 drivers/gpu/drm/i915/display/intel_fbdev.c    |  2 +-
 drivers/gpu/drm/i915/display/intel_overlay.c  | 34 +++++++--
 drivers/gpu/drm/i915/gem/i915_gem_domain.c    | 30 ++------
 drivers/gpu/drm/i915/gem/i915_gem_object.h    |  1 +
 drivers/gpu/drm/i915/gem/i915_gem_phys.c      | 10 +--
 .../drm/i915/gem/selftests/i915_gem_phys.c    |  2 +
 8 files changed, 86 insertions(+), 64 deletions(-)
drivers/gpu/drm/i915/display/intel_display.c +43 -26
index f36921a3c4bc..8a7945f55278 100644
@@ -2232,6 +2232,7 @@ static bool intel_plane_uses_fence(const struct intel_plane_state *plane_state)
 
 struct i915_vma *
 intel_pin_and_fence_fb_obj(struct drm_framebuffer *fb,
+			   bool phys_cursor,
 			   const struct i915_ggtt_view *view,
 			   bool uses_fence,
 			   unsigned long *out_flags)
@@ -2240,14 +2241,19 @@ intel_pin_and_fence_fb_obj(struct drm_framebuffer *fb,
 	struct drm_i915_private *dev_priv = to_i915(dev);
 	struct drm_i915_gem_object *obj = intel_fb_obj(fb);
 	intel_wakeref_t wakeref;
+	struct i915_gem_ww_ctx ww;
 	struct i915_vma *vma;
 	unsigned int pinctl;
 	u32 alignment;
+	int ret;
 
 	if (drm_WARN_ON(dev, !i915_gem_object_is_framebuffer(obj)))
 		return ERR_PTR(-EINVAL);
 
-	alignment = intel_surf_alignment(fb, 0);
+	if (phys_cursor)
+		alignment = intel_cursor_alignment(dev_priv);
+	else
+		alignment = intel_surf_alignment(fb, 0);
 	if (drm_WARN_ON(dev, alignment && !is_power_of_2(alignment)))
 		return ERR_PTR(-EINVAL);
 
@@ -2282,14 +2288,26 @@ intel_pin_and_fence_fb_obj(struct drm_framebuffer *fb,
 	if (HAS_GMCH(dev_priv))
 		pinctl |= PIN_MAPPABLE;
 
-	vma = i915_gem_object_pin_to_display_plane(obj,
-						   alignment, view, pinctl);
-	if (IS_ERR(vma))
+	i915_gem_ww_ctx_init(&ww, true);
+retry:
+	ret = i915_gem_object_lock(obj, &ww);
+	if (!ret && phys_cursor)
+		ret = i915_gem_object_attach_phys(obj, alignment);
+	if (!ret)
+		ret = i915_gem_object_pin_pages(obj);
+	if (ret)
 		goto err;
 
-	if (uses_fence && i915_vma_is_map_and_fenceable(vma)) {
-		int ret;
+	if (!ret) {
+		vma = i915_gem_object_pin_to_display_plane(obj, &ww, alignment,
+							   view, pinctl);
+		if (IS_ERR(vma)) {
+			ret = PTR_ERR(vma);
+			goto err_unpin;
+		}
+	}
 
+	if (uses_fence && i915_vma_is_map_and_fenceable(vma)) {
 		/*
 		 * Install a fence for tiled scan-out. Pre-i965 always needs a
 		 * fence, whereas 965+ only requires a fence if using
@@ -2310,16 +2328,28 @@ intel_pin_and_fence_fb_obj(struct drm_framebuffer *fb,
 		ret = i915_vma_pin_fence(vma);
 		if (ret != 0 && INTEL_GEN(dev_priv) < 4) {
 			i915_gem_object_unpin_from_display_plane(vma);
-			vma = ERR_PTR(ret);
-			goto err;
+			goto err_unpin;
 		}
+		ret = 0;
 
-		if (ret == 0 && vma->fence)
+		if (vma->fence)
 			*out_flags |= PLANE_HAS_FENCE;
 	}
 
 	i915_vma_get(vma);
+
+err_unpin:
+	i915_gem_object_unpin_pages(obj);
 err:
+	if (ret == -EDEADLK) {
+		ret = i915_gem_ww_ctx_backoff(&ww);
+		if (!ret)
+			goto retry;
+	}
+	i915_gem_ww_ctx_fini(&ww);
+	if (ret)
+		vma = ERR_PTR(ret);
+
 	atomic_dec(&dev_priv->gpu_error.pending_fb_pin);
 	intel_runtime_pm_put(&dev_priv->runtime_pm, wakeref);
 	return vma;
@@ -16626,19 +16656,11 @@ static int intel_plane_pin_fb(struct intel_plane_state *plane_state)
 	struct drm_i915_private *dev_priv = to_i915(plane->base.dev);
 	struct drm_framebuffer *fb = plane_state->hw.fb;
 	struct i915_vma *vma;
+	bool phys_cursor =
+		plane->id == PLANE_CURSOR &&
+		INTEL_INFO(dev_priv)->display.cursor_needs_physical;
 
-	if (plane->id == PLANE_CURSOR &&
-	    INTEL_INFO(dev_priv)->display.cursor_needs_physical) {
-		struct drm_i915_gem_object *obj = intel_fb_obj(fb);
-		const int align = intel_cursor_alignment(dev_priv);
-		int err;
-
-		err = i915_gem_object_attach_phys(obj, align);
-		if (err)
-			return err;
-	}
-
-	vma = intel_pin_and_fence_fb_obj(fb,
+	vma = intel_pin_and_fence_fb_obj(fb, phys_cursor,
 					 &plane_state->view,
 					 intel_plane_uses_fence(plane_state),
 					 &plane_state->flags);
@@ -16734,13 +16756,8 @@ intel_prepare_plane_fb(struct drm_plane *_plane,
 	if (!obj)
 		return 0;
 
-	ret = i915_gem_object_pin_pages(obj);
-	if (ret)
-		return ret;
 
 	ret = intel_plane_pin_fb(new_plane_state);
-
-	i915_gem_object_unpin_pages(obj);
 	if (ret)
 		return ret;
 
drivers/gpu/drm/i915/display/intel_display.h +1 -1
index 5e0d42d82c11..5f5e632e216b 100644
@@ -569,7 +569,7 @@ void intel_release_load_detect_pipe(struct drm_connector *connector,
 				    struct intel_load_detect_pipe *old,
 				    struct drm_modeset_acquire_ctx *ctx);
 struct i915_vma *
-intel_pin_and_fence_fb_obj(struct drm_framebuffer *fb,
+intel_pin_and_fence_fb_obj(struct drm_framebuffer *fb, bool phys_cursor,
 			   const struct i915_ggtt_view *view,
 			   bool uses_fence,
 			   unsigned long *out_flags);
drivers/gpu/drm/i915/display/intel_fbdev.c +1 -1
index 842c04e63214..bdf44e923cc0 100644
@@ -211,7 +211,7 @@ static int intelfb_create(struct drm_fb_helper *helper,
 	 * This also validates that any existing fb inherited from the
 	 * BIOS is suitable for own access.
 	 */
-	vma = intel_pin_and_fence_fb_obj(&ifbdev->fb->base,
+	vma = intel_pin_and_fence_fb_obj(&ifbdev->fb->base, false,
 					 &view, false, &flags);
 	if (IS_ERR(vma)) {
 		ret = PTR_ERR(vma);
drivers/gpu/drm/i915/display/intel_overlay.c +29 -5
index 52b4f6193b4c..9cf634cc7084 100644
@@ -755,6 +755,32 @@ static u32 overlay_cmd_reg(struct drm_intel_overlay_put_image *params)
 	return cmd;
 }
 
+static struct i915_vma *intel_overlay_pin_fb(struct drm_i915_gem_object *new_bo)
+{
+	struct i915_gem_ww_ctx ww;
+	struct i915_vma *vma;
+	int ret;
+
+	i915_gem_ww_ctx_init(&ww, true);
+retry:
+	ret = i915_gem_object_lock(new_bo, &ww);
+	if (!ret) {
+		vma = i915_gem_object_pin_to_display_plane(new_bo, &ww, 0,
+							   NULL, PIN_MAPPABLE);
+		ret = PTR_ERR_OR_ZERO(vma);
+	}
+	if (ret == -EDEADLK) {
+		ret = i915_gem_ww_ctx_backoff(&ww);
+		if (!ret)
+			goto retry;
+	}
+	i915_gem_ww_ctx_fini(&ww);
+	if (ret)
+		return ERR_PTR(ret);
+
+	return vma;
+}
+
 static int intel_overlay_do_put_image(struct intel_overlay *overlay,
 				      struct drm_i915_gem_object *new_bo,
 				      struct drm_intel_overlay_put_image *params)
@@ -776,12 +802,10 @@ static int intel_overlay_do_put_image(struct intel_overlay *overlay,
 
 	atomic_inc(&dev_priv->gpu_error.pending_fb_pin);
 
-	vma = i915_gem_object_pin_to_display_plane(new_bo,
-						   0, NULL, PIN_MAPPABLE);
-	if (IS_ERR(vma)) {
-		ret = PTR_ERR(vma);
+	vma = intel_overlay_pin_fb(new_bo);
+	if (IS_ERR(vma))
 		goto out_pin_section;
-	}
+
 	i915_gem_object_flush_frontbuffer(new_bo, ORIGIN_DIRTYFB);
 
 	if (!overlay->active) {
drivers/gpu/drm/i915/gem/i915_gem_domain.c +6 -24
index c1d4bf62b3ea..51a33c4f61d0 100644
@@ -313,12 +313,12 @@ int i915_gem_set_caching_ioctl(struct drm_device *dev, void *data,
  */
 struct i915_vma *
 i915_gem_object_pin_to_display_plane(struct drm_i915_gem_object *obj,
+				     struct i915_gem_ww_ctx *ww,
 				     u32 alignment,
 				     const struct i915_ggtt_view *view,
 				     unsigned int flags)
 {
 	struct drm_i915_private *i915 = to_i915(obj->base.dev);
-	struct i915_gem_ww_ctx ww;
 	struct i915_vma *vma;
 	int ret;
 
@@ -326,11 +326,6 @@ i915_gem_object_pin_to_display_plane(struct drm_i915_gem_object *obj,
 	if (HAS_LMEM(i915) && !i915_gem_object_is_lmem(obj))
 		return ERR_PTR(-EINVAL);
 
-	i915_gem_ww_ctx_init(&ww, true);
-retry:
-	ret = i915_gem_object_lock(obj, &ww);
-	if (ret)
-		goto err;
 	/*
 	 * The display engine is not coherent with the LLC cache on gen6.  As
 	 * a result, we make sure that the pinning that is about to occur is
@@ -345,7 +340,7 @@ i915_gem_object_pin_to_display_plane(struct drm_i915_gem_object *obj,
 					      HAS_WT(i915) ?
 					      I915_CACHE_WT : I915_CACHE_NONE);
 	if (ret)
-		goto err;
+		return ERR_PTR(ret);
 
 	/*
 	 * As the user may map the buffer once pinned in the display plane
@@ -358,32 +353,19 @@ i915_gem_object_pin_to_display_plane(struct drm_i915_gem_object *obj,
 	vma = ERR_PTR(-ENOSPC);
 	if ((flags & PIN_MAPPABLE) == 0 &&
 	    (!view || view->type == I915_GGTT_VIEW_NORMAL))
-		vma = i915_gem_object_ggtt_pin_ww(obj, &ww, view, 0, alignment,
+		vma = i915_gem_object_ggtt_pin_ww(obj, ww, view, 0, alignment,
 						  flags | PIN_MAPPABLE |
 						  PIN_NONBLOCK);
 	if (IS_ERR(vma) && vma != ERR_PTR(-EDEADLK))
-		vma = i915_gem_object_ggtt_pin_ww(obj, &ww, view, 0,
+		vma = i915_gem_object_ggtt_pin_ww(obj, ww, view, 0,
 						  alignment, flags);
-	if (IS_ERR(vma)) {
-		ret = PTR_ERR(vma);
-		goto err;
-	}
+	if (IS_ERR(vma))
+		return vma;
 
 	vma->display_alignment = max_t(u64, vma->display_alignment, alignment);
 
 	i915_gem_object_flush_if_display_locked(obj);
 
-err:
-	if (ret == -EDEADLK) {
-		ret = i915_gem_ww_ctx_backoff(&ww);
-		if (!ret)
-			goto retry;
-	}
-	i915_gem_ww_ctx_fini(&ww);
-
-	if (ret)
-		return ERR_PTR(ret);
-
 	return vma;
 }
 
drivers/gpu/drm/i915/gem/i915_gem_object.h +1 -0
index 1b85f51c6ddd..0fec91ad6f62 100644
@@ -489,6 +489,7 @@ int __must_check
 i915_gem_object_set_to_cpu_domain(struct drm_i915_gem_object *obj, bool write);
 struct i915_vma * __must_check
 i915_gem_object_pin_to_display_plane(struct drm_i915_gem_object *obj,
+				     struct i915_gem_ww_ctx *ww,
 				     u32 alignment,
 				     const struct i915_ggtt_view *view,
 				     unsigned int flags);
drivers/gpu/drm/i915/gem/i915_gem_phys.c +3 -7
index 0d176bf06405..f317be5f5e34 100644
@@ -219,6 +219,8 @@ int i915_gem_object_attach_phys(struct drm_i915_gem_object *obj, int align)
 {
 	int err;
 
+	assert_object_held(obj);
+
 	if (align > obj->base.size)
 		return -EINVAL;
 
@@ -232,13 +234,9 @@ int i915_gem_object_attach_phys(struct drm_i915_gem_object *obj, int align)
 	if (err)
 		return err;
 
-	err = i915_gem_object_lock_interruptible(obj, NULL);
-	if (err)
-		return err;
-
 	err = mutex_lock_interruptible(&obj->mm.lock);
 	if (err)
-		goto err_unlock;
+		return err;
 
 	if (unlikely(!i915_gem_object_has_struct_page(obj)))
 		goto out;
@@ -269,8 +267,6 @@ int i915_gem_object_attach_phys(struct drm_i915_gem_object *obj, int align)
 
 out:
 	mutex_unlock(&obj->mm.lock);
-err_unlock:
-	i915_gem_object_unlock(obj);
 	return err;
 }
 
drivers/gpu/drm/i915/gem/selftests/i915_gem_phys.c +2 -0
index 0cfa082047fe..3a6ce87f8b52 100644
@@ -31,7 +31,9 @@ static int mock_phys_object(void *arg)
 		goto out_obj;
 	}
 
+	i915_gem_object_lock(obj, NULL);
 	err = i915_gem_object_attach_phys(obj, PAGE_SIZE);
+	i915_gem_object_unlock(obj);
 	if (err) {
 		pr_err("i915_gem_object_attach_phys failed, err=%d\n", err);
 		goto out_obj;
Signature · 6 lines
2.26.2

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel
#28 lore raw

RFC PATCH 24/162 drm/i915: Make compilation of userptr code depend on MMU_NOTIFIER.

In reply to Matthew Auld

From: Maarten Lankhorst <[email protected]>

Now that unsynchronized mappings are removed, the only time userptr works is when the MMU notifier is enabled. Put all of the userptr code behind a mmu notifier ifdef.

---
 .../gpu/drm/i915/gem/i915_gem_execbuffer.c    |  2 +
 drivers/gpu/drm/i915/gem/i915_gem_object.h    |  4 ++
 .../gpu/drm/i915/gem/i915_gem_object_types.h  |  2 +
 drivers/gpu/drm/i915/gem/i915_gem_userptr.c   | 58 +++++++------------
 drivers/gpu/drm/i915/i915_drv.h               |  2 +
 5 files changed, 31 insertions(+), 37 deletions(-)
drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c +2 -0
index 31e412e5c68a..064285a5009b 100644
@@ -1970,8 +1970,10 @@ static noinline int eb_relocate_parse_slow(struct i915_execbuffer *eb,
 		err = 0;
 	}
 
+#ifdef CONFIG_MMU_NOTIFIER
 	if (!err)
 		flush_workqueue(eb->i915->mm.userptr_wq);
+#endif
 
 err_relock:
 	i915_gem_ww_ctx_init(&eb->ww, true);
drivers/gpu/drm/i915/gem/i915_gem_object.h +4 -0
index 95907b8eb4c4..7b3a84f98b42 100644
@@ -554,7 +554,11 @@ void __i915_gem_object_invalidate_frontbuffer(struct drm_i915_gem_object *obj,
 static inline bool
 i915_gem_object_is_userptr(struct drm_i915_gem_object *obj)
 {
+#ifdef CONFIG_MMU_NOTIFIER
 	return obj->userptr.mm;
+#else
+	return false;
+#endif
 }
 
 static inline void
drivers/gpu/drm/i915/gem/i915_gem_object_types.h +2 -0
index b53e44b06b09..6d3f451c15c6 100644
@@ -289,6 +289,7 @@ struct drm_i915_gem_object {
 	unsigned long *bit_17;
 
 	union {
+#ifdef CONFIG_MMU_NOTIFIER
 		struct i915_gem_userptr {
 			uintptr_t ptr;
 
@@ -296,6 +297,7 @@ struct drm_i915_gem_object {
 			struct i915_mmu_object *mmu_object;
 			struct work_struct *work;
 		} userptr;
+#endif
 
 		unsigned long scratch;
 		u64 encode;
drivers/gpu/drm/i915/gem/i915_gem_userptr.c +21 -37

139 lines not on this page · raw message

drivers/gpu/drm/i915/i915_drv.h +2 -0
index 84182a40e777..d3c67e17cd02 100644
@@ -589,12 +589,14 @@ struct i915_gem_mm {
 	struct notifier_block vmap_notifier;
 	struct shrinker shrinker;
 
+#ifdef CONFIG_MMU_NOTIFIER
 	/**
 	 * Workqueue to fault in userptr pages, flushed by the execbuf
 	 * when required but otherwise left to userspace to try again
 	 * on EAGAIN.
 	 */
 	struct workqueue_struct *userptr_wq;
+#endif
 
 	/* shrinker accounting, also useful for userland debugging */
 	u64 shrink_memory;
Signature · 6 lines
2.26.2

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
#27 lore raw

RFC PATCH 23/162 drm/i915: Reject UNSYNCHRONIZED for userptr, v2.

In reply to Matthew Auld

From: Maarten Lankhorst <[email protected]>

We should not allow this any more, as it will break with the new userptr implementation, it could still be made to work, but there's no point in doing so.

---
 drivers/gpu/drm/i915/gem/i915_gem_userptr.c | 10 ++--------
 1 file changed, 2 insertions(+), 8 deletions(-)
drivers/gpu/drm/i915/gem/i915_gem_userptr.c +2 -8
index 64a946d5f753..241f865077b9 100644
@@ -224,7 +224,7 @@ i915_gem_userptr_init__mmu_notifier(struct drm_i915_gem_object *obj,
 	struct i915_mmu_object *mo;
 
 	if (flags & I915_USERPTR_UNSYNCHRONIZED)
-		return capable(CAP_SYS_ADMIN) ? 0 : -EPERM;
+		return -ENODEV;
 
 	if (GEM_WARN_ON(!obj->userptr.mm))
 		return -EINVAL;
@@ -274,13 +274,7 @@ static int
 i915_gem_userptr_init__mmu_notifier(struct drm_i915_gem_object *obj,
 				    unsigned flags)
 {
-	if ((flags & I915_USERPTR_UNSYNCHRONIZED) == 0)
-		return -ENODEV;
-
-	if (!capable(CAP_SYS_ADMIN))
-		return -EPERM;
-
-	return 0;
+	return -ENODEV;
 }
 
 static void
Signature · 6 lines
2.26.2

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
#30 lore raw

RFC PATCH 26/162 drm/i915: Flatten obj->mm.lock

In reply to Matthew Auld

From: Maarten Lankhorst <[email protected]>

With userptr fixed, there is no need for all separate lockdep classes now, and we can remove all lockdep tricks used. A trylock in the shrinker is all we need now to flatten the locking hierarchy.

---
 drivers/gpu/drm/i915/gem/i915_gem_object.c   |  6 +---
 drivers/gpu/drm/i915/gem/i915_gem_object.h   | 20 ++----------
 drivers/gpu/drm/i915/gem/i915_gem_pages.c    | 34 ++++++++++----------
 drivers/gpu/drm/i915/gem/i915_gem_phys.c     |  2 +-
 drivers/gpu/drm/i915/gem/i915_gem_shrinker.c | 10 +++---
 drivers/gpu/drm/i915/gem/i915_gem_userptr.c  |  2 +-
 6 files changed, 27 insertions(+), 47 deletions(-)
drivers/gpu/drm/i915/gem/i915_gem_object.c +1 -5
index 1393988bd5af..028a556ab1a5 100644
@@ -62,7 +62,7 @@ void i915_gem_object_init(struct drm_i915_gem_object *obj,
 			  const struct drm_i915_gem_object_ops *ops,
 			  struct lock_class_key *key, unsigned flags)
 {
-	__mutex_init(&obj->mm.lock, ops->name ?: "obj->mm.lock", key);
+	mutex_init(&obj->mm.lock);
 
 	spin_lock_init(&obj->vma.lock);
 	INIT_LIST_HEAD(&obj->vma.list);
@@ -86,10 +86,6 @@ void i915_gem_object_init(struct drm_i915_gem_object *obj,
 	mutex_init(&obj->mm.get_page.lock);
 	INIT_RADIX_TREE(&obj->mm.get_dma_page.radix, GFP_KERNEL | __GFP_NOWARN);
 	mutex_init(&obj->mm.get_dma_page.lock);
-
-	if (IS_ENABLED(CONFIG_LOCKDEP) && i915_gem_object_is_shrinkable(obj))
-		i915_gem_shrinker_taints_mutex(to_i915(obj->base.dev),
-					       &obj->mm.lock);
 }
 
 /**
drivers/gpu/drm/i915/gem/i915_gem_object.h +2 -18
index 33412248f6df..1b85f51c6ddd 100644
@@ -339,27 +339,10 @@ void __i915_gem_object_set_pages(struct drm_i915_gem_object *obj,
 int ____i915_gem_object_get_pages(struct drm_i915_gem_object *obj);
 int __i915_gem_object_get_pages(struct drm_i915_gem_object *obj);
 
-enum i915_mm_subclass { /* lockdep subclass for obj->mm.lock/struct_mutex */
-	I915_MM_NORMAL = 0,
-	/*
-	 * Only used by struct_mutex, when called "recursively" from
-	 * direct-reclaim-esque. Safe because there is only every one
-	 * struct_mutex in the entire system.
-	 */
-	I915_MM_SHRINKER = 1,
-	/*
-	 * Used for obj->mm.lock when allocating pages. Safe because the object
-	 * isn't yet on any LRU, and therefore the shrinker can't deadlock on
-	 * it. As soon as the object has pages, obj->mm.lock nests within
-	 * fs_reclaim.
-	 */
-	I915_MM_GET_PAGES = 1,
-};
-
 static inline int __must_check
 i915_gem_object_pin_pages(struct drm_i915_gem_object *obj)
 {
-	might_lock_nested(&obj->mm.lock, I915_MM_GET_PAGES);
+	might_lock(&obj->mm.lock);
 
 	if (atomic_inc_not_zero(&obj->mm.pages_pin_count))
 		return 0;
@@ -403,6 +386,7 @@ i915_gem_object_unpin_pages(struct drm_i915_gem_object *obj)
 }
 
 int __i915_gem_object_put_pages(struct drm_i915_gem_object *obj);
+int __i915_gem_object_put_pages_locked(struct drm_i915_gem_object *obj);
 void i915_gem_object_truncate(struct drm_i915_gem_object *obj);
 void i915_gem_object_writeback(struct drm_i915_gem_object *obj);
 
drivers/gpu/drm/i915/gem/i915_gem_pages.c +17 -17
index 60149cad6080..5bcd21a8fc4e 100644
@@ -111,7 +111,7 @@ int __i915_gem_object_get_pages(struct drm_i915_gem_object *obj)
 {
 	int err;
 
-	err = mutex_lock_interruptible_nested(&obj->mm.lock, I915_MM_GET_PAGES);
+	err = mutex_lock_interruptible(&obj->mm.lock);
 	if (err)
 		return err;
 
@@ -193,21 +193,13 @@ __i915_gem_object_unset_pages(struct drm_i915_gem_object *obj)
 	return pages;
 }
 
-int __i915_gem_object_put_pages(struct drm_i915_gem_object *obj)
+int __i915_gem_object_put_pages_locked(struct drm_i915_gem_object *obj)
 {
 	struct sg_table *pages;
-	int err;
 
 	if (i915_gem_object_has_pinned_pages(obj))
 		return -EBUSY;
 
-	/* May be called by shrinker from within get_pages() (on another bo) */
-	mutex_lock(&obj->mm.lock);
-	if (unlikely(atomic_read(&obj->mm.pages_pin_count))) {
-		err = -EBUSY;
-		goto unlock;
-	}
-
 	i915_gem_object_release_mmap_offset(obj);
 
 	/*
@@ -223,14 +215,22 @@ int __i915_gem_object_put_pages(struct drm_i915_gem_object *obj)
 	 * get_pages backends we should be better able to handle the
 	 * cancellation of the async task in a more uniform manner.
 	 */
-	if (!pages)
-		pages = ERR_PTR(-EINVAL);
-
-	if (!IS_ERR(pages))
+	if (!IS_ERR_OR_NULL(pages))
 		obj->ops->put_pages(obj, pages);
 
-	err = 0;
-unlock:
+	return 0;
+}
+
+int __i915_gem_object_put_pages(struct drm_i915_gem_object *obj)
+{
+	int err;
+
+	if (i915_gem_object_has_pinned_pages(obj))
+		return -EBUSY;
+
+	/* May be called by shrinker from within get_pages() (on another bo) */
+	mutex_lock(&obj->mm.lock);
+	err = __i915_gem_object_put_pages_locked(obj);
 	mutex_unlock(&obj->mm.lock);
 
 	return err;
@@ -336,7 +336,7 @@ void *i915_gem_object_pin_map(struct drm_i915_gem_object *obj,
 	    !i915_gem_object_type_has(obj, I915_GEM_OBJECT_HAS_IOMEM))
 		return ERR_PTR(-ENXIO);
 
-	err = mutex_lock_interruptible_nested(&obj->mm.lock, I915_MM_GET_PAGES);
+	err = mutex_lock_interruptible(&obj->mm.lock);
 	if (err)
 		return ERR_PTR(err);
 
drivers/gpu/drm/i915/gem/i915_gem_phys.c +1 -1
index 144e4940eede..0d176bf06405 100644
@@ -236,7 +236,7 @@ int i915_gem_object_attach_phys(struct drm_i915_gem_object *obj, int align)
 	if (err)
 		return err;
 
-	err = mutex_lock_interruptible_nested(&obj->mm.lock, I915_MM_GET_PAGES);
+	err = mutex_lock_interruptible(&obj->mm.lock);
 	if (err)
 		goto err_unlock;
 
drivers/gpu/drm/i915/gem/i915_gem_shrinker.c +5 -5
index dc8f052a0ffe..afc6e5b4dcf1 100644
@@ -48,9 +48,9 @@ static bool unsafe_drop_pages(struct drm_i915_gem_object *obj,
 		flags = I915_GEM_OBJECT_UNBIND_TEST;
 
 	if (i915_gem_object_unbind(obj, flags) == 0)
-		__i915_gem_object_put_pages(obj);
+		return true;
 
-	return !i915_gem_object_has_pages(obj);
+	return false;
 }
 
 static void try_to_writeback(struct drm_i915_gem_object *obj,
@@ -199,10 +199,10 @@ i915_gem_shrink(struct drm_i915_private *i915,
 
 			spin_unlock_irqrestore(&i915->mm.obj_lock, flags);
 
-			if (unsafe_drop_pages(obj, shrink)) {
+			if (unsafe_drop_pages(obj, shrink) &&
+			    mutex_trylock(&obj->mm.lock)) {
 				/* May arrive from get_pages on another bo */
-				mutex_lock(&obj->mm.lock);
-				if (!i915_gem_object_has_pages(obj)) {
+				if (!__i915_gem_object_put_pages_locked(obj)) {
 					try_to_writeback(obj, shrink);
 					count += obj->base.size >> PAGE_SHIFT;
 				}
drivers/gpu/drm/i915/gem/i915_gem_userptr.c +1 -1
index 9ea9aa65ade1..0cab9da6669e 100644
@@ -247,7 +247,7 @@ static int i915_gem_object_userptr_unbind(struct drm_i915_gem_object *obj, bool
 	if (GEM_WARN_ON(i915_gem_object_has_pinned_pages(obj)))
 		return -EBUSY;
 
-	mutex_lock_nested(&obj->mm.lock, I915_MM_GET_PAGES);
+	mutex_lock(&obj->mm.lock);
 
 	pages = __i915_gem_object_unset_pages(obj);
 	if (!IS_ERR_OR_NULL(pages))
Signature · 6 lines
2.26.2

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel
#34 lore raw

RFC PATCH 30/162 drm/i915: Rework clflush to work correctly without obj->mm.lock.

In reply to Matthew Auld

From: Maarten Lankhorst <[email protected]>

Pin in the caller, not in the work itself. This should also work better for dma-fence annotations.

---
 drivers/gpu/drm/i915/gem/i915_gem_clflush.c | 15 +++++++--------
 1 file changed, 7 insertions(+), 8 deletions(-)
drivers/gpu/drm/i915/gem/i915_gem_clflush.c +7 -8
index bc0223716906..daf9284ef1f5 100644
@@ -27,15 +27,8 @@ static void __do_clflush(struct drm_i915_gem_object *obj)
 static int clflush_work(struct dma_fence_work *base)
 {
 	struct clflush *clflush = container_of(base, typeof(*clflush), base);
-	struct drm_i915_gem_object *obj = clflush->obj;
-	int err;
 
-	err = i915_gem_object_pin_pages(obj);
-	if (err)
-		return err;
-
-	__do_clflush(obj);
-	i915_gem_object_unpin_pages(obj);
+	__do_clflush(clflush->obj);
 
 	return 0;
 }
@@ -44,6 +37,7 @@ static void clflush_release(struct dma_fence_work *base)
 {
 	struct clflush *clflush = container_of(base, typeof(*clflush), base);
 
+	i915_gem_object_unpin_pages(clflush->obj);
 	i915_gem_object_put(clflush->obj);
 }
 
@@ -63,6 +57,11 @@ static struct clflush *clflush_work_create(struct drm_i915_gem_object *obj)
 	if (!clflush)
 		return NULL;
 
+	if (__i915_gem_object_get_pages(obj) < 0) {
+		kfree(clflush);
+		return NULL;
+	}
+
 	dma_fence_work_init(&clflush->base, &clflush_ops);
 	clflush->obj = i915_gem_object_get(obj); /* obj <-> clflush cycle */
 
Signature · 6 lines
2.26.2

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
#31 lore raw

RFC PATCH 27/162 drm/i915: Populate logical context during first pin.

In reply to Matthew Auld

From: Maarten Lankhorst <[email protected]>

This allows us to remove pin_map from state allocation, which saves us a few retry loops. We won't need this until first pin, anyway.

---
 drivers/gpu/drm/i915/gt/intel_context_types.h |  13 ++-
 .../drm/i915/gt/intel_execlists_submission.c  | 107 +++++++++---------
 2 files changed, 62 insertions(+), 58 deletions(-)
drivers/gpu/drm/i915/gt/intel_context_types.h +7 -6
index 52fa9c132746..a593c98398a7 100644
@@ -81,12 +81,13 @@ struct intel_context {
 	unsigned long flags;
 #define CONTEXT_BARRIER_BIT		0
 #define CONTEXT_ALLOC_BIT		1
-#define CONTEXT_VALID_BIT		2
-#define CONTEXT_CLOSED_BIT		3
-#define CONTEXT_USE_SEMAPHORES		4
-#define CONTEXT_BANNED			5
-#define CONTEXT_FORCE_SINGLE_SUBMISSION	6
-#define CONTEXT_NOPREEMPT		7
+#define CONTEXT_INIT_BIT		2
+#define CONTEXT_VALID_BIT		3
+#define CONTEXT_CLOSED_BIT		4
+#define CONTEXT_USE_SEMAPHORES		5
+#define CONTEXT_BANNED			6
+#define CONTEXT_FORCE_SINGLE_SUBMISSION	7
+#define CONTEXT_NOPREEMPT		8
 
 	u32 *lrc_reg_state;
 	union {
drivers/gpu/drm/i915/gt/intel_execlists_submission.c +55 -52

150 lines not on this page · raw message

Signature · 6 lines
2.26.2

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
#36 lore raw

RFC PATCH 32/162 drm/i915: Add object locking to vm_fault_cpu

In reply to Matthew Auld

From: Maarten Lankhorst <[email protected]>

Take a simple lock so we hold ww around (un)pin_pages as needed.

---
 drivers/gpu/drm/i915/gem/i915_gem_mman.c | 4 ++++
 1 file changed, 4 insertions(+)
drivers/gpu/drm/i915/gem/i915_gem_mman.c +4 -0
index c0034d811e50..163208a6260d 100644
@@ -246,6 +246,9 @@ static vm_fault_t vm_fault_cpu(struct vm_fault *vmf)
 		     area->vm_flags & VM_WRITE))
 		return VM_FAULT_SIGBUS;
 
+	if (i915_gem_object_lock_interruptible(obj, NULL))
+		return VM_FAULT_NOPAGE;
+
 	err = i915_gem_object_pin_pages(obj);
 	if (err)
 		goto out;
@@ -269,6 +272,7 @@ static vm_fault_t vm_fault_cpu(struct vm_fault *vmf)
 	i915_gem_object_unpin_pages(obj);
 
 out:
+	i915_gem_object_unlock(obj);
 	return i915_error_to_vmf_fault(err);
 }
 
Signature · 6 lines
2.26.2

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel
#37 lore raw

RFC PATCH 33/162 drm/i915: Move pinning to inside engine_wa_list_verify()

In reply to Matthew Auld

From: Maarten Lankhorst <[email protected]>

This should be done as part of the ww loop, in order to remove a i915_vma_pin that needs ww held.

Now only i915_ggtt_pin() callers remaining.

---
 drivers/gpu/drm/i915/gt/intel_workarounds.c   | 24 ++++++++----------
 .../gpu/drm/i915/gt/selftest_workarounds.c    | 25 ++++++++++++++++---
 2 files changed, 32 insertions(+), 17 deletions(-)
drivers/gpu/drm/i915/gt/intel_workarounds.c +10 -14
index a82554baa6ac..de50b7c47ea3 100644
@@ -2073,7 +2073,6 @@ create_scratch(struct i915_address_space *vm, int count)
 	struct drm_i915_gem_object *obj;
 	struct i915_vma *vma;
 	unsigned int size;
-	int err;
 
 	size = round_up(count * sizeof(u32), PAGE_SIZE);
 	obj = i915_gem_object_create_internal(vm->i915, size);
@@ -2084,20 +2083,11 @@ create_scratch(struct i915_address_space *vm, int count)
 
 	vma = i915_vma_instance(obj, vm, NULL);
 	if (IS_ERR(vma)) {
-		err = PTR_ERR(vma);
-		goto err_obj;
+		i915_gem_object_put(obj);
+		return vma;
 	}
 
-	err = i915_vma_pin(vma, 0, 0,
-			   i915_vma_is_ggtt(vma) ? PIN_GLOBAL : PIN_USER);
-	if (err)
-		goto err_obj;
-
 	return vma;
-
-err_obj:
-	i915_gem_object_put(obj);
-	return ERR_PTR(err);
 }
 
 struct mcr_range {
@@ -2215,10 +2205,15 @@ static int engine_wa_list_verify(struct intel_context *ce,
 	if (err)
 		goto err_pm;
 
+	err = i915_vma_pin_ww(vma, &ww, 0, 0,
+			   i915_vma_is_ggtt(vma) ? PIN_GLOBAL : PIN_USER);
+	if (err)
+		goto err_unpin;
+
 	rq = i915_request_create(ce);
 	if (IS_ERR(rq)) {
 		err = PTR_ERR(rq);
-		goto err_unpin;
+		goto err_vma;
 	}
 
 	err = i915_request_await_object(rq, vma->obj, true);
@@ -2259,6 +2254,8 @@ static int engine_wa_list_verify(struct intel_context *ce,
 
 err_rq:
 	i915_request_put(rq);
+err_vma:
+	i915_vma_unpin(vma);
 err_unpin:
 	intel_context_unpin(ce);
 err_pm:
@@ -2269,7 +2266,6 @@ static int engine_wa_list_verify(struct intel_context *ce,
 	}
 	i915_gem_ww_ctx_fini(&ww);
 	intel_engine_pm_put(ce->engine);
-	i915_vma_unpin(vma);
 	i915_vma_put(vma);
 	return err;
 }
drivers/gpu/drm/i915/gt/selftest_workarounds.c +22 -3
index 61a0532d0f3d..810ab026a55e 100644
@@ -386,6 +386,25 @@ static struct i915_vma *create_batch(struct i915_address_space *vm)
 	return ERR_PTR(err);
 }
 
+static struct i915_vma *
+create_scratch_pinned(struct i915_address_space *vm, int count)
+{
+	struct i915_vma *vma = create_scratch(vm, count);
+	int err;
+
+	if (IS_ERR(vma))
+		return vma;
+
+	err = i915_vma_pin(vma, 0, 0,
+			   i915_vma_is_ggtt(vma) ? PIN_GLOBAL : PIN_USER);
+	if (err) {
+		i915_vma_put(vma);
+		return ERR_PTR(err);
+	}
+
+	return vma;
+}
+
 static u32 reg_write(u32 old, u32 new, u32 rsvd)
 {
 	if (rsvd == 0x0000ffff) {
@@ -489,7 +508,7 @@ static int check_dirty_whitelist(struct intel_context *ce)
 	int err = 0, i, v;
 	u32 *cs, *results;
 
-	scratch = create_scratch(ce->vm, 2 * ARRAY_SIZE(values) + 1);
+	scratch = create_scratch_pinned(ce->vm, 2 * ARRAY_SIZE(values) + 1);
 	if (IS_ERR(scratch))
 		return PTR_ERR(scratch);
 
@@ -1043,7 +1062,7 @@ static int live_isolated_whitelist(void *arg)
 
 		vm = i915_gem_context_get_vm_rcu(c);
 
-		client[i].scratch[0] = create_scratch(vm, 1024);
+		client[i].scratch[0] = create_scratch_pinned(vm, 1024);
 		if (IS_ERR(client[i].scratch[0])) {
 			err = PTR_ERR(client[i].scratch[0]);
 			i915_vm_put(vm);
@@ -1051,7 +1070,7 @@ static int live_isolated_whitelist(void *arg)
 			goto err;
 		}
 
-		client[i].scratch[1] = create_scratch(vm, 1024);
+		client[i].scratch[1] = create_scratch_pinned(vm, 1024);
 		if (IS_ERR(client[i].scratch[1])) {
 			err = PTR_ERR(client[i].scratch[1]);
 			i915_vma_unpin_and_release(&client[i].scratch[0], 0);
Signature · 6 lines
2.26.2

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
#40 lore raw

RFC PATCH 36/162 drm/i915: Make __engine_unpark() compatible with ww locking v2

In reply to Matthew Auld

From: Maarten Lankhorst <[email protected]>

Take the ww lock around engine_unpark. Because of the many many places where rpm is used, I chose the safest option and used a trylock to opportunistically take this lock for __engine_unpark.

---
 drivers/gpu/drm/i915/gt/intel_engine_pm.c | 4 ++++
 1 file changed, 4 insertions(+)
drivers/gpu/drm/i915/gt/intel_engine_pm.c +4 -0
index 499b09cb4acf..5d51144ef074 100644
@@ -27,12 +27,16 @@ static void dbg_poison_ce(struct intel_context *ce)
 		int type = i915_coherent_map_type(ce->engine->i915);
 		void *map;
 
+		if (!i915_gem_object_trylock(ce->state->obj))
+			return;
+
 		map = i915_gem_object_pin_map(obj, type);
 		if (!IS_ERR(map)) {
 			memset(map, CONTEXT_REDZONE, obj->base.size);
 			i915_gem_object_flush_map(obj);
 			i915_gem_object_unpin_map(obj);
 		}
+		i915_gem_object_unlock(obj);
 	}
 }
 
Signature · 6 lines
2.26.2

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel
#42 lore raw

RFC PATCH 38/162 drm/i915: Defer pin calls in buffer pool until first use by caller.

In reply to Matthew Auld

From: Maarten Lankhorst <[email protected]>

We need to take the obj lock to pin pages, so wait until the callers have done so, before making the object unshrinkable.

---
 .../gpu/drm/i915/gem/i915_gem_execbuffer.c    |  2 +
 .../gpu/drm/i915/gem/i915_gem_object_blt.c    |  6 +++
 .../gpu/drm/i915/gt/intel_gt_buffer_pool.c    | 47 +++++++++----------
 .../gpu/drm/i915/gt/intel_gt_buffer_pool.h    |  5 ++
 .../drm/i915/gt/intel_gt_buffer_pool_types.h  |  1 +
 5 files changed, 35 insertions(+), 26 deletions(-)
drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c +2 -0
index f5ea49e244ca..91f0c3fd9a4b 100644
@@ -1343,6 +1343,7 @@ static int __reloc_gpu_alloc(struct i915_execbuffer *eb,
 		err = PTR_ERR(cmd);
 		goto err_pool;
 	}
+	intel_gt_buffer_pool_mark_used(pool);
 
 	batch = i915_vma_instance(pool->obj, vma->vm, NULL);
 	if (IS_ERR(batch)) {
@@ -2635,6 +2636,7 @@ static int eb_parse(struct i915_execbuffer *eb)
 		err = PTR_ERR(shadow);
 		goto err;
 	}
+	intel_gt_buffer_pool_mark_used(pool);
 	i915_gem_object_set_readonly(shadow->obj);
 	shadow->private = pool;
 
drivers/gpu/drm/i915/gem/i915_gem_object_blt.c +6 -0
index aee7ad3cc3c6..e0b873c3f46a 100644
@@ -54,6 +54,9 @@ struct i915_vma *intel_emit_vma_fill_blt(struct intel_context *ce,
 	if (unlikely(err))
 		goto out_put;
 
+	/* we pinned the pool, mark it as such */
+	intel_gt_buffer_pool_mark_used(pool);
+
 	cmd = i915_gem_object_pin_map(pool->obj, I915_MAP_WC);
 	if (IS_ERR(cmd)) {
 		err = PTR_ERR(cmd);
@@ -276,6 +279,9 @@ struct i915_vma *intel_emit_vma_copy_blt(struct intel_context *ce,
 	if (unlikely(err))
 		goto out_put;
 
+	/* we pinned the pool, mark it as such */
+	intel_gt_buffer_pool_mark_used(pool);
+
 	cmd = i915_gem_object_pin_map(pool->obj, I915_MAP_WC);
 	if (IS_ERR(cmd)) {
 		err = PTR_ERR(cmd);
drivers/gpu/drm/i915/gt/intel_gt_buffer_pool.c +21 -26

72 lines not on this page · raw message

drivers/gpu/drm/i915/gt/intel_gt_buffer_pool.h +5 -0
index 42cbac003e8a..9878ce9a07ab 100644
@@ -17,10 +17,15 @@ struct i915_request;
 struct intel_gt_buffer_pool_node *
 intel_gt_get_buffer_pool(struct intel_gt *gt, size_t size);
 
+void intel_gt_buffer_pool_mark_used(struct intel_gt_buffer_pool_node *node);
+
 static inline int
 intel_gt_buffer_pool_mark_active(struct intel_gt_buffer_pool_node *node,
 				 struct i915_request *rq)
 {
+	/* did we call mark_used? */
+	GEM_WARN_ON(!node->pinned);
+
 	return i915_active_add_request(&node->active, rq);
 }
 
drivers/gpu/drm/i915/gt/intel_gt_buffer_pool_types.h +1 -0
index bcf1658c9633..0401825e829d 100644
@@ -31,6 +31,7 @@ struct intel_gt_buffer_pool_node {
 		struct rcu_head rcu;
 	};
 	unsigned long age;
+	bool pinned;
 };
 
 #endif /* INTEL_GT_BUFFER_POOL_TYPES_H */
Signature · 6 lines
2.26.2

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel
#43 lore raw

RFC PATCH 39/162 drm/i915: Fix pread/pwrite to work with new locking rules.

In reply to Matthew Auld

From: Maarten Lankhorst <[email protected]>

We are removing obj->mm.lock, and need to take the reservation lock before we can pin pages. Move the pinning pages into the helper, and merge gtt pwrite/pread preparation and cleanup paths.

The fence lock is also removed; it will conflict with fence annotations, because of memory allocations done when pagefaulting inside copy_*_user.

---
 drivers/gpu/drm/i915/Makefile              |   1 -
 drivers/gpu/drm/i915/gem/i915_gem_fence.c  |  95 ---------
 drivers/gpu/drm/i915/gem/i915_gem_object.h |   5 -
 drivers/gpu/drm/i915/i915_gem.c            | 224 +++++++++++----------
 4 files changed, 114 insertions(+), 211 deletions(-)
 delete mode 100644 drivers/gpu/drm/i915/gem/i915_gem_fence.c
drivers/gpu/drm/i915/Makefile +0 -1
index 2445cc990e15..5112e5d79316 100644
@@ -136,7 +136,6 @@ gem-y += \
 	gem/i915_gem_dmabuf.o \
 	gem/i915_gem_domain.o \
 	gem/i915_gem_execbuffer.o \
-	gem/i915_gem_fence.o \
 	gem/i915_gem_internal.o \
 	gem/i915_gem_object.o \
 	gem/i915_gem_object_blt.o \
drivers/gpu/drm/i915/gem/i915_gem_fence.c deleted +0 -95

95 lines not on this page · raw message

drivers/gpu/drm/i915/gem/i915_gem_object.h +0 -5
index 0fec91ad6f62..9a81a80ca849 100644
@@ -180,11 +180,6 @@ static inline void i915_gem_object_unlock(struct drm_i915_gem_object *obj)
 	dma_resv_unlock(obj->base.resv);
 }
 
-struct dma_fence *
-i915_gem_object_lock_fence(struct drm_i915_gem_object *obj);
-void i915_gem_object_unlock_fence(struct drm_i915_gem_object *obj,
-				  struct dma_fence *fence);
-
 static inline void
 i915_gem_object_set_readonly(struct drm_i915_gem_object *obj)
 {
drivers/gpu/drm/i915/i915_gem.c +114 -110

341 lines not on this page · raw message

Signature · 6 lines
2.26.2

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
#38 lore raw

RFC PATCH 34/162 drm/i915: Take reservation lock around i915_vma_pin.

In reply to Matthew Auld

From: Maarten Lankhorst <[email protected]>

We previously complained when ww == NULL.

This function is now only used in selftests to pin an object, and ww locking is now fixed.

---
 .../i915/gem/selftests/i915_gem_coherency.c   | 14 +++++--------
 drivers/gpu/drm/i915/i915_gem.c               |  6 +++++-
 drivers/gpu/drm/i915/i915_vma.c               |  3 +--
 drivers/gpu/drm/i915/i915_vma.h               | 20 +++++++++++++++----
 4 files changed, 27 insertions(+), 16 deletions(-)
drivers/gpu/drm/i915/gem/selftests/i915_gem_coherency.c +5 -9
index 7049a6bbc03d..2e439bb269d6 100644
@@ -199,16 +199,14 @@ static int gpu_set(struct context *ctx, unsigned long offset, u32 v)
 	u32 *cs;
 	int err;
 
+	vma = i915_gem_object_ggtt_pin(ctx->obj, NULL, 0, 0, 0);
+	if (IS_ERR(vma))
+		return PTR_ERR(vma);
+
 	i915_gem_object_lock(ctx->obj, NULL);
 	err = i915_gem_object_set_to_gtt_domain(ctx->obj, true);
 	if (err)
-		goto out_unlock;
-
-	vma = i915_gem_object_ggtt_pin(ctx->obj, NULL, 0, 0, 0);
-	if (IS_ERR(vma)) {
-		err = PTR_ERR(vma);
-		goto out_unlock;
-	}
+		goto out_unpin;
 
 	rq = intel_engine_create_kernel_request(ctx->engine);
 	if (IS_ERR(rq)) {
@@ -248,9 +246,7 @@ static int gpu_set(struct context *ctx, unsigned long offset, u32 v)
 	i915_request_add(rq);
 out_unpin:
 	i915_vma_unpin(vma);
-out_unlock:
 	i915_gem_object_unlock(ctx->obj);
-
 	return err;
 }
 
drivers/gpu/drm/i915/i915_gem.c +5 -1
index 0b9eab66511c..b5311f7ad870 100644
@@ -1011,7 +1011,11 @@ i915_gem_object_ggtt_pin_ww(struct drm_i915_gem_object *obj,
 			return ERR_PTR(ret);
 	}
 
-	ret = i915_vma_pin_ww(vma, ww, size, alignment, flags | PIN_GLOBAL);
+	if (ww)
+		ret = i915_vma_pin_ww(vma, ww, size, alignment, flags | PIN_GLOBAL);
+	else
+		ret = i915_vma_pin(vma, size, alignment, flags | PIN_GLOBAL);
+
 	if (ret)
 		return ERR_PTR(ret);
 
drivers/gpu/drm/i915/i915_vma.c +1 -2
index 5b1d78fa748e..63bdb0cc981e 100644
@@ -868,8 +868,7 @@ int i915_vma_pin_ww(struct i915_vma *vma, struct i915_gem_ww_ctx *ww,
 			vma->obj && i915_gem_object_has_pinned_pages(vma->obj) &&
 			!vma->vm->allocate_va_range;
 
-		if (lockdep_is_held(&vma->vm->i915->drm.struct_mutex) &&
-		    !pinned_bind_wo_alloc)
+		if (!pinned_bind_wo_alloc)
 			WARN_ON(!ww);
 		if (ww && vma->resv)
 			assert_vma_held(vma);
drivers/gpu/drm/i915/i915_vma.h +16 -4
index a2e7b58b70ca..2db4f25b8d5f 100644
@@ -246,10 +246,22 @@ i915_vma_pin_ww(struct i915_vma *vma, struct i915_gem_ww_ctx *ww,
 static inline int __must_check
 i915_vma_pin(struct i915_vma *vma, u64 size, u64 alignment, u64 flags)
 {
-#ifdef CONFIG_LOCKDEP
-	WARN_ON_ONCE(vma->resv && dma_resv_held(vma->resv));
-#endif
-	return i915_vma_pin_ww(vma, NULL, size, alignment, flags);
+	struct i915_gem_ww_ctx ww;
+	int err;
+
+	i915_gem_ww_ctx_init(&ww, true);
+retry:
+	err = i915_gem_object_lock(vma->obj, &ww);
+	if (!err)
+		err = i915_vma_pin_ww(vma, &ww, size, alignment, flags);
+	if (err == -EDEADLK) {
+		err = i915_gem_ww_ctx_backoff(&ww);
+		if (!err)
+			goto retry;
+	}
+	i915_gem_ww_ctx_fini(&ww);
+
+	return err;
 }
 
 int i915_ggtt_pin(struct i915_vma *vma, struct i915_gem_ww_ctx *ww,
Signature · 6 lines
2.26.2

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel
#39 lore raw

RFC PATCH 35/162 drm/i915: Make intel_init_workaround_bb more compatible with ww locking.

In reply to Matthew Auld

From: Daniele Ceraolo Spurio <[email protected]>

Make creation separate from pinning, in order to take the lock only once, and pin the mapping with the lock held.

---
 .../drm/i915/gt/intel_engine_workaround_bb.c  | 45 +++++++++++++++----
 1 file changed, 37 insertions(+), 8 deletions(-)
drivers/gpu/drm/i915/gt/intel_engine_workaround_bb.c +37 -8
index b03bdfc92bb2..f3636b73cc10 100644
@@ -229,7 +229,7 @@ gen10_init_indirectctx_bb(struct intel_engine_cs *engine, u32 *batch)
 
 #define CTX_WA_BB_OBJ_SIZE (PAGE_SIZE)
 
-static int lrc_setup_wa_ctx(struct intel_engine_cs *engine)
+static int lrc_init_wa_ctx(struct intel_engine_cs *engine)
 {
 	struct drm_i915_gem_object *obj;
 	struct i915_vma *vma;
@@ -245,10 +245,6 @@ static int lrc_setup_wa_ctx(struct intel_engine_cs *engine)
 		goto err;
 	}
 
-	err = i915_ggtt_pin(vma, NULL, 0, PIN_HIGH);
-	if (err)
-		goto err;
-
 	engine->wa_ctx.vma = vma;
 	return 0;
 
@@ -257,6 +253,18 @@ static int lrc_setup_wa_ctx(struct intel_engine_cs *engine)
 	return err;
 }
 
+static void lrc_destroy_wa_ctx(struct intel_engine_cs *engine, bool unpin)
+{
+	if (!engine->wa_ctx.vma)
+		return;
+
+	if (unpin)
+		i915_vma_unpin(engine->wa_ctx.vma);
+
+	i915_vma_put(engine->wa_ctx.vma);
+	engine->wa_ctx.vma = NULL;
+}
+
 typedef u32 *(*wa_bb_func_t)(struct intel_engine_cs *engine, u32 *batch);
 
 int intel_init_workaround_bb(struct intel_engine_cs *engine)
@@ -266,6 +274,7 @@ int intel_init_workaround_bb(struct intel_engine_cs *engine)
 					    &wa_ctx->per_ctx };
 	wa_bb_func_t wa_bb_fn[2];
 	void *batch, *batch_ptr;
+	struct i915_gem_ww_ctx ww;
 	unsigned int i;
 	int ret;
 
@@ -293,13 +302,21 @@ int intel_init_workaround_bb(struct intel_engine_cs *engine)
 		return 0;
 	}
 
-	ret = lrc_setup_wa_ctx(engine);
+	ret = lrc_init_wa_ctx(engine);
 	if (ret) {
 		drm_dbg(&engine->i915->drm,
 			"Failed to setup context WA page: %d\n", ret);
 		return ret;
 	}
 
+	i915_gem_ww_ctx_init(&ww, true);
+retry:
+	ret = i915_gem_object_lock(wa_ctx->vma->obj, &ww);
+	if (!ret)
+		ret = i915_ggtt_pin(wa_ctx->vma, &ww, 0, PIN_HIGH);
+	if (ret)
+		goto err;
+
 	batch = i915_gem_object_pin_map(wa_ctx->vma->obj, I915_MAP_WB);
 
 	/*
@@ -323,13 +340,25 @@ int intel_init_workaround_bb(struct intel_engine_cs *engine)
 
 	__i915_gem_object_flush_map(wa_ctx->vma->obj, 0, batch_ptr - batch);
 	__i915_gem_object_release_map(wa_ctx->vma->obj);
+
+	if (ret)
+		i915_vma_unpin(wa_ctx->vma);
+
+err:
+	if (ret == -EDEADLK) {
+		ret = i915_gem_ww_ctx_backoff(&ww);
+		if (!ret)
+			goto retry;
+	}
+	i915_gem_ww_ctx_fini(&ww);
 	if (ret)
-		intel_fini_workaround_bb(engine);
+		lrc_destroy_wa_ctx(engine, false);
 
 	return ret;
 }
 
+
 void intel_fini_workaround_bb(struct intel_engine_cs *engine)
 {
-	i915_vma_unpin_and_release(&engine->wa_ctx.vma, 0);
+	lrc_destroy_wa_ctx(engine, true);
 }
Signature · 6 lines
2.26.2

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
#45 lore raw

RFC PATCH 41/162 drm/i915: Prepare for obj->mm.lock removal

In reply to Matthew Auld

From: Thomas Hellström <[email protected]>

Stolen objects need to lock, and we may call put_pages when refcount drops to 0, ensure all calls are handled correctly.

Idea-from: Thomas Hellström <[email protected]>

---
 drivers/gpu/drm/i915/gem/i915_gem_object.h | 13 +++++++++++++
 drivers/gpu/drm/i915/gem/i915_gem_pages.c  | 14 ++++++++++++--
 drivers/gpu/drm/i915/gem/i915_gem_stolen.c | 10 +++++++++-
 3 files changed, 34 insertions(+), 3 deletions(-)
drivers/gpu/drm/i915/gem/i915_gem_object.h +13 -0
index da7fd301fc8d..26ef37532f81 100644
@@ -125,6 +125,19 @@ i915_gem_object_put(struct drm_i915_gem_object *obj)
 	  ((kref_read(&obj->base.refcount) == 1) &&		\
 	   list_empty_careful(&obj->mm.link) &&			\
 	   list_empty_careful(&obj->vma.list))))
+/*
+ * If more than one potential simultaneous locker, assert held.
+ */
+static inline void assert_object_held_shared(struct drm_i915_gem_object *obj)
+{
+	/*
+	 * Note mm list lookup is protected by
+	 * kref_get_unless_zero().
+	 */
+	if (IS_ENABLED(CONFIG_LOCKDEP) &&
+	    kref_read(&obj->base.refcount) > 0)
+		lockdep_assert_held(&obj->mm.lock);
+}
 
 static inline int __i915_gem_object_lock(struct drm_i915_gem_object *obj,
 					 struct i915_gem_ww_ctx *ww,
drivers/gpu/drm/i915/gem/i915_gem_pages.c +12 -2
index b03e58106516..183aae046b68 100644
@@ -18,7 +18,7 @@ void __i915_gem_object_set_pages(struct drm_i915_gem_object *obj,
 	unsigned long supported = INTEL_INFO(i915)->page_sizes;
 	int i;
 
-	lockdep_assert_held(&obj->mm.lock);
+	assert_object_held_shared(obj);
 
 	if (i915_gem_object_is_volatile(obj))
 		obj->mm.madv = I915_MADV_DONTNEED;
@@ -67,6 +67,7 @@ void __i915_gem_object_set_pages(struct drm_i915_gem_object *obj,
 		struct list_head *list;
 		unsigned long flags;
 
+		lockdep_assert_held(&obj->mm.lock);
 		spin_lock_irqsave(&i915->mm.obj_lock, flags);
 
 		i915->mm.shrink_count++;
@@ -88,6 +89,8 @@ int ____i915_gem_object_get_pages(struct drm_i915_gem_object *obj)
 	struct drm_i915_private *i915 = to_i915(obj->base.dev);
 	int err;
 
+	assert_object_held_shared(obj);
+
 	if (unlikely(obj->mm.madv != I915_MADV_WILLNEED)) {
 		drm_dbg(&i915->drm,
 			"Attempting to obtain a purgeable object\n");
@@ -115,6 +118,8 @@ int __i915_gem_object_get_pages(struct drm_i915_gem_object *obj)
 	if (err)
 		return err;
 
+	assert_object_held_shared(obj);
+
 	if (unlikely(!i915_gem_object_has_pages(obj))) {
 		GEM_BUG_ON(i915_gem_object_has_pinned_pages(obj));
 
@@ -142,7 +147,7 @@ void i915_gem_object_truncate(struct drm_i915_gem_object *obj)
 /* Try to discard unwanted pages */
 void i915_gem_object_writeback(struct drm_i915_gem_object *obj)
 {
-	lockdep_assert_held(&obj->mm.lock);
+	assert_object_held_shared(obj);
 	GEM_BUG_ON(i915_gem_object_has_pages(obj));
 
 	if (obj->ops->writeback)
@@ -173,6 +178,8 @@ __i915_gem_object_unset_pages(struct drm_i915_gem_object *obj)
 {
 	struct sg_table *pages;
 
+	assert_object_held_shared(obj);
+
 	pages = fetch_and_zero(&obj->mm.pages);
 	if (IS_ERR_OR_NULL(pages))
 		return pages;
@@ -200,6 +207,9 @@ int __i915_gem_object_put_pages_locked(struct drm_i915_gem_object *obj)
 	if (i915_gem_object_has_pinned_pages(obj))
 		return -EBUSY;
 
+	/* May be called by shrinker from within get_pages() (on another bo) */
+	assert_object_held_shared(obj);
+
 	i915_gem_object_release_mmap_offset(obj);
 
 	/*
drivers/gpu/drm/i915/gem/i915_gem_stolen.c +9 -1
index 5372b888ba01..ce9086d3a647 100644
@@ -643,11 +643,19 @@ __i915_gem_object_create_stolen(struct intel_memory_region *mem,
 	cache_level = HAS_LLC(mem->i915) ? I915_CACHE_LLC : I915_CACHE_NONE;
 	i915_gem_object_set_cache_coherency(obj, cache_level);
 
+	if (WARN_ON(!i915_gem_object_trylock(obj))) {
+		err = -EBUSY;
+		goto cleanup;
+	}
+
 	err = i915_gem_object_pin_pages(obj);
-	if (err)
+	if (err) {
+		i915_gem_object_unlock(obj);
 		goto cleanup;
+	}
 
 	i915_gem_object_init_memory_region(obj, mem);
+	i915_gem_object_unlock(obj);
 
 	return obj;
 
Signature · 6 lines
2.26.2

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
#41 lore raw

RFC PATCH 37/162 drm/i915: Take obj lock around set_domain ioctl

In reply to Matthew Auld

From: Maarten Lankhorst <[email protected]>

We need to lock the object to move it to the correct domain, add the missing lock.

---
 drivers/gpu/drm/i915/gem/i915_gem_domain.c | 18 ++++++++++--------
 1 file changed, 10 insertions(+), 8 deletions(-)
drivers/gpu/drm/i915/gem/i915_gem_domain.c +10 -8
index 51a33c4f61d0..e62f9e8dd339 100644
@@ -516,6 +516,10 @@ i915_gem_set_domain_ioctl(struct drm_device *dev, void *data,
 		goto out;
 	}
 
+	err = i915_gem_object_lock_interruptible(obj, NULL);
+	if (err)
+		goto out;
+
 	/*
 	 * Flush and acquire obj->pages so that we are coherent through
 	 * direct access in memory with previous cached writes through
@@ -527,7 +531,7 @@ i915_gem_set_domain_ioctl(struct drm_device *dev, void *data,
 	 */
 	err = i915_gem_object_pin_pages(obj);
 	if (err)
-		goto out;
+		goto out_unlock;
 
 	/*
 	 * Already in the desired write domain? Nothing for us to do!
@@ -542,10 +546,6 @@ i915_gem_set_domain_ioctl(struct drm_device *dev, void *data,
 	if (READ_ONCE(obj->write_domain) == read_domains)
 		goto out_unpin;
 
-	err = i915_gem_object_lock_interruptible(obj, NULL);
-	if (err)
-		goto out_unpin;
-
 	if (read_domains & I915_GEM_DOMAIN_WC)
 		err = i915_gem_object_set_to_wc_domain(obj, write_domain);
 	else if (read_domains & I915_GEM_DOMAIN_GTT)
@@ -556,13 +556,15 @@ i915_gem_set_domain_ioctl(struct drm_device *dev, void *data,
 	/* And bump the LRU for this access */
 	i915_gem_object_bump_inactive_ggtt(obj);
 
+out_unpin:
+	i915_gem_object_unpin_pages(obj);
+
+out_unlock:
 	i915_gem_object_unlock(obj);
 
-	if (write_domain)
+	if (!err && write_domain)
 		i915_gem_object_invalidate_frontbuffer(obj, ORIGIN_CPU);
 
-out_unpin:
-	i915_gem_object_unpin_pages(obj);
 out:
 	i915_gem_object_put(obj);
 	return err;
Signature · 6 lines
2.26.2

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
#46 lore raw

RFC PATCH 42/162 drm/i915: Add igt_spinner_pin() to allow for ww locking around spinner.

In reply to Matthew Auld

From: Maarten Lankhorst <[email protected]>

By default, we assume that it's called inside igt_create_request to keep existing selftests working, but allow for manual pinning when passing a ww context.

---
 drivers/gpu/drm/i915/selftests/igt_spinner.c | 136 ++++++++++++-------
 drivers/gpu/drm/i915/selftests/igt_spinner.h |   5 +
 2 files changed, 95 insertions(+), 46 deletions(-)
drivers/gpu/drm/i915/selftests/igt_spinner.c +90 -46
index ec0ecb4e4ca6..9c461edb0b73 100644
@@ -11,8 +11,6 @@
 
 int igt_spinner_init(struct igt_spinner *spin, struct intel_gt *gt)
 {
-	unsigned int mode;
-	void *vaddr;
 	int err;
 
 	memset(spin, 0, sizeof(*spin));
@@ -23,6 +21,7 @@ int igt_spinner_init(struct igt_spinner *spin, struct intel_gt *gt)
 		err = PTR_ERR(spin->hws);
 		goto err;
 	}
+	i915_gem_object_set_cache_coherency(spin->hws, I915_CACHE_LLC);
 
 	spin->obj = i915_gem_object_create_internal(gt->i915, PAGE_SIZE);
 	if (IS_ERR(spin->obj)) {
@@ -30,34 +29,83 @@ int igt_spinner_init(struct igt_spinner *spin, struct intel_gt *gt)
 		goto err_hws;
 	}
 
-	i915_gem_object_set_cache_coherency(spin->hws, I915_CACHE_LLC);
-	vaddr = i915_gem_object_pin_map(spin->hws, I915_MAP_WB);
-	if (IS_ERR(vaddr)) {
-		err = PTR_ERR(vaddr);
-		goto err_obj;
-	}
-	spin->seqno = memset(vaddr, 0xff, PAGE_SIZE);
-
-	mode = i915_coherent_map_type(gt->i915);
-	vaddr = i915_gem_object_pin_map(spin->obj, mode);
-	if (IS_ERR(vaddr)) {
-		err = PTR_ERR(vaddr);
-		goto err_unpin_hws;
-	}
-	spin->batch = vaddr;
-
 	return 0;
 
-err_unpin_hws:
-	i915_gem_object_unpin_map(spin->hws);
-err_obj:
-	i915_gem_object_put(spin->obj);
 err_hws:
 	i915_gem_object_put(spin->hws);
 err:
 	return err;
 }
 
+static void *igt_spinner_pin_obj(struct intel_context *ce,
+				 struct i915_gem_ww_ctx *ww,
+				 struct drm_i915_gem_object *obj,
+				 unsigned int mode, struct i915_vma **vma)
+{
+	void *vaddr;
+	int ret;
+
+	*vma = i915_vma_instance(obj, ce->vm, NULL);
+	if (IS_ERR(*vma))
+		return ERR_CAST(*vma);
+
+	ret = i915_gem_object_lock(obj, ww);
+	if (ret)
+		return ERR_PTR(ret);
+
+	vaddr = i915_gem_object_pin_map(obj, mode);
+
+	if (!ww)
+		i915_gem_object_unlock(obj);
+
+	if (IS_ERR(vaddr))
+		return vaddr;
+
+	if (ww)
+		ret = i915_vma_pin_ww(*vma, ww, 0, 0, PIN_USER);
+	else
+		ret = i915_vma_pin(*vma, 0, 0, PIN_USER);
+
+	if (ret) {
+		i915_gem_object_unpin_map(obj);
+		return ERR_PTR(ret);
+	}
+
+	return vaddr;
+}
+
+int igt_spinner_pin(struct igt_spinner *spin,
+		    struct intel_context *ce,
+		    struct i915_gem_ww_ctx *ww)
+{
+	void *vaddr;
+
+	if (spin->ce && WARN_ON(spin->ce != ce))
+		return -ENODEV;
+	spin->ce = ce;
+
+	if (!spin->seqno) {
+		vaddr = igt_spinner_pin_obj(ce, ww, spin->hws, I915_MAP_WB, &spin->hws_vma);
+		if (IS_ERR(vaddr))
+			return PTR_ERR(vaddr);
+
+		spin->seqno = memset(vaddr, 0xff, PAGE_SIZE);
+	}
+
+	if (!spin->batch) {
+		unsigned int mode =
+			i915_coherent_map_type(spin->gt->i915);
+
+		vaddr = igt_spinner_pin_obj(ce, ww, spin->obj, mode, &spin->batch_vma);
+		if (IS_ERR(vaddr))
+			return PTR_ERR(vaddr);
+
+		spin->batch = vaddr;
+	}
+
+	return 0;
+}
+
 static unsigned int seqno_offset(u64 fence)
 {
 	return offset_in_page(sizeof(u32) * fence);
@@ -102,27 +150,18 @@ igt_spinner_create_request(struct igt_spinner *spin,
 	if (!intel_engine_can_store_dword(ce->engine))
 		return ERR_PTR(-ENODEV);
 
-	vma = i915_vma_instance(spin->obj, ce->vm, NULL);
-	if (IS_ERR(vma))
-		return ERR_CAST(vma);
-
-	hws = i915_vma_instance(spin->hws, ce->vm, NULL);
-	if (IS_ERR(hws))
-		return ERR_CAST(hws);
+	if (!spin->batch) {
+		err = igt_spinner_pin(spin, ce, NULL);
+		if (err)
+			return ERR_PTR(err);
+	}
 
-	err = i915_vma_pin(vma, 0, 0, PIN_USER);
-	if (err)
-		return ERR_PTR(err);
-
-	err = i915_vma_pin(hws, 0, 0, PIN_USER);
-	if (err)
-		goto unpin_vma;
+	hws = spin->hws_vma;
+	vma = spin->batch_vma;
 
 	rq = intel_context_create_request(ce);
-	if (IS_ERR(rq)) {
-		err = PTR_ERR(rq);
-		goto unpin_hws;
-	}
+	if (IS_ERR(rq))
+		return ERR_CAST(rq);
 
 	err = move_to_active(vma, rq, 0);
 	if (err)
@@ -185,10 +224,6 @@ igt_spinner_create_request(struct igt_spinner *spin,
 		i915_request_set_error_once(rq, err);
 		i915_request_add(rq);
 	}
-unpin_hws:
-	i915_vma_unpin(hws);
-unpin_vma:
-	i915_vma_unpin(vma);
 	return err ? ERR_PTR(err) : rq;
 }
 
@@ -202,6 +237,9 @@ hws_seqno(const struct igt_spinner *spin, const struct i915_request *rq)
 
 void igt_spinner_end(struct igt_spinner *spin)
 {
+	if (!spin->batch)
+		return;
+
 	*spin->batch = MI_BATCH_BUFFER_END;
 	intel_gt_chipset_flush(spin->gt);
 }
@@ -210,10 +248,16 @@ void igt_spinner_fini(struct igt_spinner *spin)
 {
 	igt_spinner_end(spin);
 
-	i915_gem_object_unpin_map(spin->obj);
+	if (spin->batch) {
+		i915_vma_unpin(spin->batch_vma);
+		i915_gem_object_unpin_map(spin->obj);
+	}
 	i915_gem_object_put(spin->obj);
 
-	i915_gem_object_unpin_map(spin->hws);
+	if (spin->seqno) {
+		i915_vma_unpin(spin->hws_vma);
+		i915_gem_object_unpin_map(spin->hws);
+	}
 	i915_gem_object_put(spin->hws);
 }
 
drivers/gpu/drm/i915/selftests/igt_spinner.h +5 -0
index ec62c9ef320b..fbe5b1625b05 100644
@@ -20,11 +20,16 @@ struct igt_spinner {
 	struct intel_gt *gt;
 	struct drm_i915_gem_object *hws;
 	struct drm_i915_gem_object *obj;
+	struct intel_context *ce;
+	struct i915_vma *hws_vma, *batch_vma;
 	u32 *batch;
 	void *seqno;
 };
 
 int igt_spinner_init(struct igt_spinner *spin, struct intel_gt *gt);
+int igt_spinner_pin(struct igt_spinner *spin,
+		    struct intel_context *ce,
+		    struct i915_gem_ww_ctx *ww);
 void igt_spinner_fini(struct igt_spinner *spin);
 
 struct i915_request *
Signature · 6 lines
2.26.2

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
#44 lore raw

RFC PATCH 40/162 drm/i915: Fix workarounds selftest, part 1

In reply to Matthew Auld

From: Maarten Lankhorst <[email protected]>

pin_map needs the ww lock, so ensure we pin both before submission.

---
 drivers/gpu/drm/i915/gem/i915_gem_object.h    |  3 +
 drivers/gpu/drm/i915/gem/i915_gem_pages.c     | 12 +++
 .../gpu/drm/i915/gt/selftest_workarounds.c    | 76 ++++++++++++-------
 3 files changed, 64 insertions(+), 27 deletions(-)
drivers/gpu/drm/i915/gem/i915_gem_object.h +3 -0
index 9a81a80ca849..da7fd301fc8d 100644
@@ -412,6 +412,9 @@ enum i915_map_type {
 void *__must_check i915_gem_object_pin_map(struct drm_i915_gem_object *obj,
 					   enum i915_map_type type);
 
+void *__must_check i915_gem_object_pin_map_unlocked(struct drm_i915_gem_object *obj,
+						    enum i915_map_type type);
+
 void __i915_gem_object_flush_map(struct drm_i915_gem_object *obj,
 				 unsigned long offset,
 				 unsigned long size);
drivers/gpu/drm/i915/gem/i915_gem_pages.c +12 -0
index 5bcd21a8fc4e..b03e58106516 100644
@@ -397,6 +397,18 @@ void *i915_gem_object_pin_map(struct drm_i915_gem_object *obj,
 	goto out_unlock;
 }
 
+void *i915_gem_object_pin_map_unlocked(struct drm_i915_gem_object *obj,
+				       enum i915_map_type type)
+{
+	void *ret;
+
+	i915_gem_object_lock(obj, NULL);
+	ret = i915_gem_object_pin_map(obj, type);
+	i915_gem_object_unlock(obj);
+
+	return ret;
+}
+
 void __i915_gem_object_flush_map(struct drm_i915_gem_object *obj,
 				 unsigned long offset,
 				 unsigned long size)
drivers/gpu/drm/i915/gt/selftest_workarounds.c +49 -27

177 lines not on this page · raw message

Signature · 6 lines
2.26.2

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
#47 lore raw

RFC PATCH 43/162 drm/i915: Add ww locking around vm_access()

In reply to Matthew Auld

From: Maarten Lankhorst <[email protected]>

i915_gem_object_pin_map potentially needs a ww context, so ensure we have one we can revoke.

---
 drivers/gpu/drm/i915/gem/i915_gem_mman.c | 24 ++++++++++++++++++++++--
 1 file changed, 22 insertions(+), 2 deletions(-)
drivers/gpu/drm/i915/gem/i915_gem_mman.c +22 -2
index 163208a6260d..2561a2f1e54f 100644
@@ -421,7 +421,9 @@ vm_access(struct vm_area_struct *area, unsigned long addr,
 {
 	struct i915_mmap_offset *mmo = area->vm_private_data;
 	struct drm_i915_gem_object *obj = mmo->obj;
+	struct i915_gem_ww_ctx ww;
 	void *vaddr;
+	int err = 0;
 
 	if (i915_gem_object_is_readonly(obj) && write)
 		return -EACCES;
@@ -430,10 +432,18 @@ vm_access(struct vm_area_struct *area, unsigned long addr,
 	if (addr >= obj->base.size)
 		return -EINVAL;
 
+	i915_gem_ww_ctx_init(&ww, true);
+retry:
+	err = i915_gem_object_lock(obj, &ww);
+	if (err)
+		goto out;
+
 	/* As this is primarily for debugging, let's focus on simplicity */
 	vaddr = i915_gem_object_pin_map(obj, I915_MAP_FORCE_WC);
-	if (IS_ERR(vaddr))
-		return PTR_ERR(vaddr);
+	if (IS_ERR(vaddr)) {
+		err = PTR_ERR(vaddr);
+		goto out;
+	}
 
 	if (write) {
 		memcpy(vaddr + addr, buf, len);
@@ -443,6 +453,16 @@ vm_access(struct vm_area_struct *area, unsigned long addr,
 	}
 
 	i915_gem_object_unpin_map(obj);
+out:
+	if (err == -EDEADLK) {
+		err = i915_gem_ww_ctx_backoff(&ww);
+		if (!err)
+			goto retry;
+	}
+	i915_gem_ww_ctx_fini(&ww);
+
+	if (err)
+		return err;
 
 	return len;
 }
Signature · 6 lines
2.26.2

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
#49 lore raw

RFC PATCH 45/162 drm/i915: Lock ww in ucode objects correctly

In reply to Matthew Auld

From: Maarten Lankhorst <[email protected]>

In the ucode functions, the calls are done before userspace runs, when debugging using debugfs, or when creating semi-permanent mappings; we can safely use the unlocked versions that does the ww dance for us.

Because there is no pin_pages_unlocked yet, add it as convenience function.

This removes possible lockdep splats about missing resv lock for ucode.

---
 drivers/gpu/drm/i915/gem/i915_gem_object.h |  2 ++
 drivers/gpu/drm/i915/gem/i915_gem_pages.c  | 20 ++++++++++++++++++++
 drivers/gpu/drm/i915/gt/uc/intel_guc.c     |  2 +-
 drivers/gpu/drm/i915/gt/uc/intel_guc_log.c |  4 ++--
 drivers/gpu/drm/i915/gt/uc/intel_huc.c     |  2 +-
 drivers/gpu/drm/i915/gt/uc/intel_uc_fw.c   |  2 +-
 6 files changed, 27 insertions(+), 5 deletions(-)
drivers/gpu/drm/i915/gem/i915_gem_object.h +2 -0
index 26ef37532f81..1d4b44151e0c 100644
@@ -358,6 +358,8 @@ i915_gem_object_pin_pages(struct drm_i915_gem_object *obj)
 	return __i915_gem_object_get_pages(obj);
 }
 
+int i915_gem_object_pin_pages_unlocked(struct drm_i915_gem_object *obj);
+
 static inline bool
 i915_gem_object_has_pages(struct drm_i915_gem_object *obj)
 {
drivers/gpu/drm/i915/gem/i915_gem_pages.c +20 -0
index 183aae046b68..79336735a6e4 100644
@@ -136,6 +136,26 @@ int __i915_gem_object_get_pages(struct drm_i915_gem_object *obj)
 	return err;
 }
 
+int i915_gem_object_pin_pages_unlocked(struct drm_i915_gem_object *obj)
+{
+	struct i915_gem_ww_ctx ww;
+	int err;
+
+	i915_gem_ww_ctx_init(&ww, true);
+retry:
+	err = i915_gem_object_lock(obj, &ww);
+	if (!err)
+		err = i915_gem_object_pin_pages(obj);
+
+	if (err == -EDEADLK) {
+		err = i915_gem_ww_ctx_backoff(&ww);
+		if (!err)
+			goto retry;
+	}
+	i915_gem_ww_ctx_fini(&ww);
+	return err;
+}
+
 /* Immediately discard the backing storage */
 void i915_gem_object_truncate(struct drm_i915_gem_object *obj)
 {
drivers/gpu/drm/i915/gt/uc/intel_guc.c +1 -1
index 2a343a977987..a65661eb5d5d 100644
@@ -694,7 +694,7 @@ int intel_guc_allocate_and_map_vma(struct intel_guc *guc, u32 size,
 	if (IS_ERR(vma))
 		return PTR_ERR(vma);
 
-	vaddr = i915_gem_object_pin_map(vma->obj, I915_MAP_WB);
+	vaddr = i915_gem_object_pin_map_unlocked(vma->obj, I915_MAP_WB);
 	if (IS_ERR(vaddr)) {
 		i915_vma_unpin_and_release(&vma, 0);
 		return PTR_ERR(vaddr);
drivers/gpu/drm/i915/gt/uc/intel_guc_log.c +2 -2
index 9bbe8a795cb8..8dc8678e7ab0 100644
@@ -335,7 +335,7 @@ static int guc_log_map(struct intel_guc_log *log)
 	 * buffer pages, so that we can directly get the data
 	 * (up-to-date) from memory.
 	 */
-	vaddr = i915_gem_object_pin_map(log->vma->obj, I915_MAP_WC);
+	vaddr = i915_gem_object_pin_map_unlocked(log->vma->obj, I915_MAP_WC);
 	if (IS_ERR(vaddr))
 		return PTR_ERR(vaddr);
 
@@ -744,7 +744,7 @@ int intel_guc_log_dump(struct intel_guc_log *log, struct drm_printer *p,
 	if (!obj)
 		return 0;
 
-	map = i915_gem_object_pin_map(obj, I915_MAP_WC);
+	map = i915_gem_object_pin_map_unlocked(obj, I915_MAP_WC);
 	if (IS_ERR(map)) {
 		DRM_DEBUG("Failed to pin object\n");
 		drm_puts(p, "(log data unaccessible)\n");
drivers/gpu/drm/i915/gt/uc/intel_huc.c +1 -1
index 65eeb44b397d..2126dd81ac38 100644
@@ -82,7 +82,7 @@ static int intel_huc_rsa_data_create(struct intel_huc *huc)
 	if (IS_ERR(vma))
 		return PTR_ERR(vma);
 
-	vaddr = i915_gem_object_pin_map(vma->obj, I915_MAP_WB);
+	vaddr = i915_gem_object_pin_map_unlocked(vma->obj, I915_MAP_WB);
 	if (IS_ERR(vaddr)) {
 		i915_vma_unpin_and_release(&vma, 0);
 		return PTR_ERR(vaddr);
drivers/gpu/drm/i915/gt/uc/intel_uc_fw.c +1 -1
index 180c23e2e25e..b05076d190cc 100644
@@ -541,7 +541,7 @@ int intel_uc_fw_init(struct intel_uc_fw *uc_fw)
 	if (!intel_uc_fw_is_available(uc_fw))
 		return -ENOEXEC;
 
-	err = i915_gem_object_pin_pages(uc_fw->obj);
+	err = i915_gem_object_pin_pages_unlocked(uc_fw->obj);
 	if (err) {
 		DRM_DEBUG_DRIVER("%s fw pin-pages err=%d\n",
 				 intel_uc_fw_type_repr(uc_fw->type), err);
Signature · 6 lines
2.26.2

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
#52 lore raw

RFC PATCH 48/162 drm/i915: Fix ww locking in shmem_create_from_object

In reply to Matthew Auld

From: Maarten Lankhorst <[email protected]>

Quick fix, just use the unlocked version.

---
 drivers/gpu/drm/i915/gt/shmem_utils.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/gpu/drm/i915/gt/shmem_utils.c +1 -1
index f011ea42487e..041e2a50160d 100644
@@ -39,7 +39,7 @@ struct file *shmem_create_from_object(struct drm_i915_gem_object *obj)
 		return file;
 	}
 
-	ptr = i915_gem_object_pin_map(obj, I915_MAP_WB);
+	ptr = i915_gem_object_pin_map_unlocked(obj, I915_MAP_WB);
 	if (IS_ERR(ptr))
 		return ERR_CAST(ptr);
 
Signature · 6 lines
2.26.2

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
#51 lore raw

RFC PATCH 47/162 drm/i915: Add missing ww lock in intel_dsb_prepare.

In reply to Matthew Auld

From: Maarten Lankhorst <[email protected]>

Because of the long lifetime of the mapping, we cannot wrap this in a simple limited ww lock. Just use the unlocked version of pin_map, because we'll likely release the mapping a lot later, in a different thread.

---
 drivers/gpu/drm/i915/display/intel_dsb.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/gpu/drm/i915/display/intel_dsb.c +1 -1
index 566fa72427b3..857126822a88 100644
@@ -293,7 +293,7 @@ void intel_dsb_prepare(struct intel_crtc_state *crtc_state)
 		goto out;
 	}
 
-	buf = i915_gem_object_pin_map(vma->obj, I915_MAP_WC);
+	buf = i915_gem_object_pin_map_unlocked(vma->obj, I915_MAP_WC);
 	if (IS_ERR(buf)) {
 		drm_err(&i915->drm, "Command buffer creation failed\n");
 		i915_vma_unpin_and_release(&vma, I915_VMA_RELEASE_MAP);
Signature · 6 lines
2.26.2

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel
#53 lore raw

RFC PATCH 49/162 drm/i915: Use a single page table lock for each gtt.

In reply to Matthew Auld

From: Maarten Lankhorst <[email protected]>

We may create page table objects on the fly, but we may need to wait with the ww lock held. Instead of waiting on a freed obj lock, ensure we have the same lock for each object to keep -EDEADLK working. This ensures that i915_vma_pin_ww can lock the page tables when required.

---
 drivers/gpu/drm/i915/gt/intel_ggtt.c  |  8 +++++-
 drivers/gpu/drm/i915/gt/intel_gtt.c   | 38 ++++++++++++++++++++++++++-
 drivers/gpu/drm/i915/gt/intel_gtt.h   |  5 ++++
 drivers/gpu/drm/i915/gt/intel_ppgtt.c |  3 ++-
 drivers/gpu/drm/i915/i915_vma.c       |  4 +++
 5 files changed, 55 insertions(+), 3 deletions(-)
drivers/gpu/drm/i915/gt/intel_ggtt.c +7 -1
index 60bd2c8ed8b0..17ecaef1834d 100644
@@ -615,7 +615,9 @@ static int init_aliasing_ppgtt(struct i915_ggtt *ggtt)
 	if (err)
 		goto err_ppgtt;
 
+	i915_gem_object_lock(ppgtt->vm.scratch[0], NULL);
 	err = i915_vm_pin_pt_stash(&ppgtt->vm, &stash);
+	i915_gem_object_unlock(ppgtt->vm.scratch[0]);
 	if (err)
 		goto err_stash;
 
@@ -702,6 +704,7 @@ static void ggtt_cleanup_hw(struct i915_ggtt *ggtt)
 
 	mutex_unlock(&ggtt->vm.mutex);
 	i915_address_space_fini(&ggtt->vm);
+	dma_resv_fini(&ggtt->vm.resv);
 
 	arch_phys_wc_del(ggtt->mtrr);
 
@@ -1078,6 +1081,7 @@ static int ggtt_probe_hw(struct i915_ggtt *ggtt, struct intel_gt *gt)
 	ggtt->vm.gt = gt;
 	ggtt->vm.i915 = i915;
 	ggtt->vm.dma = &i915->drm.pdev->dev;
+	dma_resv_init(&ggtt->vm.resv);
 
 	if (INTEL_GEN(i915) <= 5)
 		ret = i915_gmch_probe(ggtt);
@@ -1085,8 +1089,10 @@ static int ggtt_probe_hw(struct i915_ggtt *ggtt, struct intel_gt *gt)
 		ret = gen6_gmch_probe(ggtt);
 	else
 		ret = gen8_gmch_probe(ggtt);
-	if (ret)
+	if (ret) {
+		dma_resv_fini(&ggtt->vm.resv);
 		return ret;
+	}
 
 	if ((ggtt->vm.total - 1) >> 32) {
 		drm_err(&i915->drm,
drivers/gpu/drm/i915/gt/intel_gtt.c +37 -1
index 7bfe9072be9a..070d538cdc56 100644
@@ -13,16 +13,36 @@
 
 struct drm_i915_gem_object *alloc_pt_dma(struct i915_address_space *vm, int sz)
 {
+	struct drm_i915_gem_object *obj;
+
 	if (I915_SELFTEST_ONLY(should_fail(&vm->fault_attr, 1)))
 		i915_gem_shrink_all(vm->i915);
 
-	return i915_gem_object_create_internal(vm->i915, sz);
+	obj = i915_gem_object_create_internal(vm->i915, sz);
+	/* ensure all dma objects have the same reservation class */
+	if (!IS_ERR(obj))
+		obj->base.resv = &vm->resv;
+	return obj;
 }
 
 int pin_pt_dma(struct i915_address_space *vm, struct drm_i915_gem_object *obj)
 {
 	int err;
 
+	i915_gem_object_lock(obj, NULL);
+	err = i915_gem_object_pin_pages(obj);
+	i915_gem_object_unlock(obj);
+	if (err)
+		return err;
+
+	i915_gem_object_make_unshrinkable(obj);
+	return 0;
+}
+
+int pin_pt_dma_locked(struct i915_address_space *vm, struct drm_i915_gem_object *obj)
+{
+	int err;
+
 	err = i915_gem_object_pin_pages(obj);
 	if (err)
 		return err;
@@ -56,6 +76,20 @@ void __i915_vm_close(struct i915_address_space *vm)
 	mutex_unlock(&vm->mutex);
 }
 
+/* lock the vm into the current ww, if we lock one, we lock all */
+int i915_vm_lock_objects(struct i915_address_space *vm,
+			 struct i915_gem_ww_ctx *ww)
+{
+	if (vm->scratch[0]->base.resv == &vm->resv) {
+		return i915_gem_object_lock(vm->scratch[0], ww);
+	} else {
+		struct i915_ppgtt *ppgtt = i915_vm_to_ppgtt(vm);
+
+		/* We borrowed the scratch page from ggtt, take the top level object */
+		return i915_gem_object_lock(ppgtt->pd->pt.base, ww);
+	}
+}
+
 void i915_address_space_fini(struct i915_address_space *vm)
 {
 	drm_mm_takedown(&vm->mm);
@@ -69,6 +103,7 @@ static void __i915_vm_release(struct work_struct *work)
 
 	vm->cleanup(vm);
 	i915_address_space_fini(vm);
+	dma_resv_fini(&vm->resv);
 
 	kfree(vm);
 }
@@ -98,6 +133,7 @@ void i915_address_space_init(struct i915_address_space *vm, int subclass)
 	mutex_init(&vm->mutex);
 	lockdep_set_subclass(&vm->mutex, subclass);
 	i915_gem_shrinker_taints_mutex(vm->i915, &vm->mutex);
+	dma_resv_init(&vm->resv);
 
 	GEM_BUG_ON(!vm->total);
 	drm_mm_init(&vm->mm, 0, vm->total);
drivers/gpu/drm/i915/gt/intel_gtt.h +5 -0
index 8a33940a71f3..16063b2f0119 100644
@@ -238,6 +238,7 @@ struct i915_address_space {
 	atomic_t open;
 
 	struct mutex mutex; /* protects vma and our lists */
+	struct dma_resv resv; /* reservation lock for all pd objects, and buffer pool */
 #define VM_CLASS_GGTT 0
 #define VM_CLASS_PPGTT 1
 
@@ -346,6 +347,9 @@ struct i915_ppgtt {
 
 #define i915_is_ggtt(vm) ((vm)->is_ggtt)
 
+int __must_check
+i915_vm_lock_objects(struct i915_address_space *vm, struct i915_gem_ww_ctx *ww);
+
 static inline bool
 i915_vm_is_4lvl(const struct i915_address_space *vm)
 {
@@ -522,6 +526,7 @@ struct i915_page_directory *alloc_pd(struct i915_address_space *vm);
 struct i915_page_directory *__alloc_pd(int npde);
 
 int pin_pt_dma(struct i915_address_space *vm, struct drm_i915_gem_object *obj);
+int pin_pt_dma_locked(struct i915_address_space *vm, struct drm_i915_gem_object *obj);
 
 void free_px(struct i915_address_space *vm,
 	     struct i915_page_table *pt, int lvl);
drivers/gpu/drm/i915/gt/intel_ppgtt.c +2 -1
index 46d9aceda64c..f3ac47702aee 100644
@@ -262,7 +262,7 @@ int i915_vm_pin_pt_stash(struct i915_address_space *vm,
 
 	for (n = 0; n < ARRAY_SIZE(stash->pt); n++) {
 		for (pt = stash->pt[n]; pt; pt = pt->stash) {
-			err = pin_pt_dma(vm, pt->base);
+			err = pin_pt_dma_locked(vm, pt->base);
 			if (err)
 				return err;
 		}
@@ -304,6 +304,7 @@ void ppgtt_init(struct i915_ppgtt *ppgtt, struct intel_gt *gt)
 	ppgtt->vm.dma = &i915->drm.pdev->dev;
 	ppgtt->vm.total = BIT_ULL(INTEL_INFO(i915)->ppgtt_size);
 
+	dma_resv_init(&ppgtt->vm.resv);
 	i915_address_space_init(&ppgtt->vm, VM_CLASS_PPGTT);
 
 	ppgtt->vm.vma_ops.bind_vma    = ppgtt_bind_vma;
drivers/gpu/drm/i915/i915_vma.c +4 -0
index 63bdb0cc981e..0c7e4191811a 100644
@@ -908,6 +908,10 @@ int i915_vma_pin_ww(struct i915_vma *vma, struct i915_gem_ww_ctx *ww,
 			if (err)
 				goto err_fence;
 
+			err = i915_vm_lock_objects(vma->vm, ww);
+			if (err)
+				goto err_fence;
+
 			err = i915_vm_pin_pt_stash(vma->vm,
 						   &work->stash);
 			if (err)
Signature · 6 lines
2.26.2

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel
#50 lore raw

RFC PATCH 46/162 drm/i915: Add ww locking to dma-buf ops.

In reply to Matthew Auld

From: Maarten Lankhorst <[email protected]>

vmap is using pin_pages, but needs to use ww locking, add pin_pages_unlocked to correctly lock the mapping.

Also add ww locking to begin/end cpu access.

---
 drivers/gpu/drm/i915/gem/i915_gem_dmabuf.c | 60 ++++++++++++----------
 1 file changed, 33 insertions(+), 27 deletions(-)
drivers/gpu/drm/i915/gem/i915_gem_dmabuf.c +33 -27
index 36e3c2765f4c..c4b01e819786 100644
@@ -82,7 +82,7 @@ static int i915_gem_dmabuf_vmap(struct dma_buf *dma_buf, struct dma_buf_map *map
 	struct drm_i915_gem_object *obj = dma_buf_to_obj(dma_buf);
 	void *vaddr;
 
-	vaddr = i915_gem_object_pin_map(obj, I915_MAP_WB);
+	vaddr = i915_gem_object_pin_map_unlocked(obj, I915_MAP_WB);
 	if (IS_ERR(vaddr))
 		return PTR_ERR(vaddr);
 
@@ -123,42 +123,48 @@ static int i915_gem_begin_cpu_access(struct dma_buf *dma_buf, enum dma_data_dire
 {
 	struct drm_i915_gem_object *obj = dma_buf_to_obj(dma_buf);
 	bool write = (direction == DMA_BIDIRECTIONAL || direction == DMA_TO_DEVICE);
+	struct i915_gem_ww_ctx ww;
 	int err;
 
-	err = i915_gem_object_pin_pages(obj);
-	if (err)
-		return err;
-
-	err = i915_gem_object_lock_interruptible(obj, NULL);
-	if (err)
-		goto out;
-
-	err = i915_gem_object_set_to_cpu_domain(obj, write);
-	i915_gem_object_unlock(obj);
-
-out:
-	i915_gem_object_unpin_pages(obj);
+	i915_gem_ww_ctx_init(&ww, true);
+retry:
+	err = i915_gem_object_lock(obj, &ww);
+	if (!err)
+		err = i915_gem_object_pin_pages(obj);
+	if (!err) {
+		err = i915_gem_object_set_to_cpu_domain(obj, write);
+		i915_gem_object_unpin_pages(obj);
+	}
+	if (err == -EDEADLK) {
+		err = i915_gem_ww_ctx_backoff(&ww);
+		if (!err)
+			goto retry;
+	}
+	i915_gem_ww_ctx_fini(&ww);
 	return err;
 }
 
 static int i915_gem_end_cpu_access(struct dma_buf *dma_buf, enum dma_data_direction direction)
 {
 	struct drm_i915_gem_object *obj = dma_buf_to_obj(dma_buf);
+	struct i915_gem_ww_ctx ww;
 	int err;
 
-	err = i915_gem_object_pin_pages(obj);
-	if (err)
-		return err;
-
-	err = i915_gem_object_lock_interruptible(obj, NULL);
-	if (err)
-		goto out;
-
-	err = i915_gem_object_set_to_gtt_domain(obj, false);
-	i915_gem_object_unlock(obj);
-
-out:
-	i915_gem_object_unpin_pages(obj);
+	i915_gem_ww_ctx_init(&ww, true);
+retry:
+	err = i915_gem_object_lock(obj, &ww);
+	if (!err)
+		err = i915_gem_object_pin_pages(obj);
+	if (!err) {
+		err = i915_gem_object_set_to_gtt_domain(obj, false);
+		i915_gem_object_unpin_pages(obj);
+	}
+	if (err == -EDEADLK) {
+		err = i915_gem_ww_ctx_backoff(&ww);
+		if (!err)
+			goto retry;
+	}
+	i915_gem_ww_ctx_fini(&ww);
 	return err;
 }
 
Signature · 6 lines
2.26.2

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel
#48 lore raw

RFC PATCH 44/162 drm/i915: Increase ww locking for perf.

In reply to Matthew Auld

From: Maarten Lankhorst <[email protected]>

We need to lock a few more objects, some temporarily, add ww lock where needed.

---
 drivers/gpu/drm/i915/i915_perf.c | 56 ++++++++++++++++++++++++--------
 1 file changed, 43 insertions(+), 13 deletions(-)
drivers/gpu/drm/i915/i915_perf.c +43 -13
index 0b300e0d9561..1f574d29ece5 100644
@@ -1587,7 +1587,7 @@ static int alloc_oa_buffer(struct i915_perf_stream *stream)
 	stream->oa_buffer.vma = vma;
 
 	stream->oa_buffer.vaddr =
-		i915_gem_object_pin_map(bo, I915_MAP_WB);
+		i915_gem_object_pin_map_unlocked(bo, I915_MAP_WB);
 	if (IS_ERR(stream->oa_buffer.vaddr)) {
 		ret = PTR_ERR(stream->oa_buffer.vaddr);
 		goto err_unpin;
@@ -1640,6 +1640,7 @@ static int alloc_noa_wait(struct i915_perf_stream *stream)
 	const u32 base = stream->engine->mmio_base;
 #define CS_GPR(x) GEN8_RING_CS_GPR(base, x)
 	u32 *batch, *ts0, *cs, *jump;
+	struct i915_gem_ww_ctx ww;
 	int ret, i;
 	enum {
 		START_TS,
@@ -1657,15 +1658,21 @@ static int alloc_noa_wait(struct i915_perf_stream *stream)
 		return PTR_ERR(bo);
 	}
 
+	i915_gem_ww_ctx_init(&ww, true);
+retry:
+	ret = i915_gem_object_lock(bo, &ww);
+	if (ret)
+		goto out_ww;
+
 	/*
 	 * We pin in GGTT because we jump into this buffer now because
 	 * multiple OA config BOs will have a jump to this address and it
 	 * needs to be fixed during the lifetime of the i915/perf stream.
 	 */
-	vma = i915_gem_object_ggtt_pin(bo, NULL, 0, 0, PIN_HIGH);
+	vma = i915_gem_object_ggtt_pin_ww(bo, &ww, NULL, 0, 0, PIN_HIGH);
 	if (IS_ERR(vma)) {
 		ret = PTR_ERR(vma);
-		goto err_unref;
+		goto out_ww;
 	}
 
 	batch = cs = i915_gem_object_pin_map(bo, I915_MAP_WB);
@@ -1799,12 +1806,19 @@ static int alloc_noa_wait(struct i915_perf_stream *stream)
 	__i915_gem_object_release_map(bo);
 
 	stream->noa_wait = vma;
-	return 0;
+	goto out_ww;
 
 err_unpin:
 	i915_vma_unpin_and_release(&vma, 0);
-err_unref:
-	i915_gem_object_put(bo);
+out_ww:
+	if (ret == -EDEADLK) {
+		ret = i915_gem_ww_ctx_backoff(&ww);
+		if (!ret)
+			goto retry;
+	}
+	i915_gem_ww_ctx_fini(&ww);
+	if (ret)
+		i915_gem_object_put(bo);
 	return ret;
 }
 
@@ -1847,6 +1861,7 @@ alloc_oa_config_buffer(struct i915_perf_stream *stream,
 {
 	struct drm_i915_gem_object *obj;
 	struct i915_oa_config_bo *oa_bo;
+	struct i915_gem_ww_ctx ww;
 	size_t config_length = 0;
 	u32 *cs;
 	int err;
@@ -1867,10 +1882,16 @@ alloc_oa_config_buffer(struct i915_perf_stream *stream,
 		goto err_free;
 	}
 
+	i915_gem_ww_ctx_init(&ww, true);
+retry:
+	err = i915_gem_object_lock(obj, &ww);
+	if (err)
+		goto out_ww;
+
 	cs = i915_gem_object_pin_map(obj, I915_MAP_WB);
 	if (IS_ERR(cs)) {
 		err = PTR_ERR(cs);
-		goto err_oa_bo;
+		goto out_ww;
 	}
 
 	cs = write_cs_mi_lri(cs,
@@ -1898,19 +1919,28 @@ alloc_oa_config_buffer(struct i915_perf_stream *stream,
 				       NULL);
 	if (IS_ERR(oa_bo->vma)) {
 		err = PTR_ERR(oa_bo->vma);
-		goto err_oa_bo;
+		goto out_ww;
 	}
 
 	oa_bo->oa_config = i915_oa_config_get(oa_config);
 	llist_add(&oa_bo->node, &stream->oa_config_bos);
 
-	return oa_bo;
+out_ww:
+	if (err == -EDEADLK) {
+		err = i915_gem_ww_ctx_backoff(&ww);
+		if (!err)
+			goto retry;
+	}
+	i915_gem_ww_ctx_fini(&ww);
 
-err_oa_bo:
-	i915_gem_object_put(obj);
+	if (err)
+		i915_gem_object_put(obj);
 err_free:
-	kfree(oa_bo);
-	return ERR_PTR(err);
+	if (err) {
+		kfree(oa_bo);
+		return ERR_PTR(err);
+	}
+	return oa_bo;
 }
 
 static struct i915_vma *
Signature · 6 lines
2.26.2

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel
#54 lore raw

RFC PATCH 50/162 drm/i915/selftests: Prepare huge_pages testcases for obj->mm.lock removal.

In reply to Matthew Auld

From: Maarten Lankhorst <[email protected]>

Straightforward conversion, just convert a bunch of calls to unlocked versions.

---
 .../gpu/drm/i915/gem/selftests/huge_pages.c   | 28 ++++++++++++++-----
 1 file changed, 21 insertions(+), 7 deletions(-)
drivers/gpu/drm/i915/gem/selftests/huge_pages.c +21 -7
index 709c63b9cfc4..586d8bafd7de 100644
@@ -589,7 +589,7 @@ static int igt_mock_ppgtt_misaligned_dma(void *arg)
 			goto out_put;
 		}
 
-		err = i915_gem_object_pin_pages(obj);
+		err = i915_gem_object_pin_pages_unlocked(obj);
 		if (err)
 			goto out_put;
 
@@ -653,15 +653,19 @@ static int igt_mock_ppgtt_misaligned_dma(void *arg)
 				break;
 		}
 
+		i915_gem_object_lock(obj, NULL);
 		i915_gem_object_unpin_pages(obj);
 		__i915_gem_object_put_pages(obj);
+		i915_gem_object_unlock(obj);
 		i915_gem_object_put(obj);
 	}
 
 	return 0;
 
 out_unpin:
+	i915_gem_object_lock(obj, NULL);
 	i915_gem_object_unpin_pages(obj);
+	i915_gem_object_unlock(obj);
 out_put:
 	i915_gem_object_put(obj);
 
@@ -675,8 +679,10 @@ static void close_object_list(struct list_head *objects,
 
 	list_for_each_entry_safe(obj, on, objects, st_link) {
 		list_del(&obj->st_link);
+		i915_gem_object_lock(obj, NULL);
 		i915_gem_object_unpin_pages(obj);
 		__i915_gem_object_put_pages(obj);
+		i915_gem_object_unlock(obj);
 		i915_gem_object_put(obj);
 	}
 }
@@ -713,7 +719,7 @@ static int igt_mock_ppgtt_huge_fill(void *arg)
 			break;
 		}
 
-		err = i915_gem_object_pin_pages(obj);
+		err = i915_gem_object_pin_pages_unlocked(obj);
 		if (err) {
 			i915_gem_object_put(obj);
 			break;
@@ -889,7 +895,7 @@ static int igt_mock_ppgtt_64K(void *arg)
 			if (IS_ERR(obj))
 				return PTR_ERR(obj);
 
-			err = i915_gem_object_pin_pages(obj);
+			err = i915_gem_object_pin_pages_unlocked(obj);
 			if (err)
 				goto out_object_put;
 
@@ -943,8 +949,10 @@ static int igt_mock_ppgtt_64K(void *arg)
 			}
 
 			i915_vma_unpin(vma);
+			i915_gem_object_lock(obj, NULL);
 			i915_gem_object_unpin_pages(obj);
 			__i915_gem_object_put_pages(obj);
+			i915_gem_object_unlock(obj);
 			i915_gem_object_put(obj);
 		}
 	}
@@ -954,7 +962,9 @@ static int igt_mock_ppgtt_64K(void *arg)
 out_vma_unpin:
 	i915_vma_unpin(vma);
 out_object_unpin:
+	i915_gem_object_lock(obj, NULL);
 	i915_gem_object_unpin_pages(obj);
+	i915_gem_object_unlock(obj);
 out_object_put:
 	i915_gem_object_put(obj);
 
@@ -1024,7 +1034,7 @@ static int __cpu_check_vmap(struct drm_i915_gem_object *obj, u32 dword, u32 val)
 	if (err)
 		return err;
 
-	ptr = i915_gem_object_pin_map(obj, I915_MAP_WC);
+	ptr = i915_gem_object_pin_map_unlocked(obj, I915_MAP_WC);
 	if (IS_ERR(ptr))
 		return PTR_ERR(ptr);
 
@@ -1304,7 +1314,7 @@ static int igt_ppgtt_smoke_huge(void *arg)
 			return err;
 		}
 
-		err = i915_gem_object_pin_pages(obj);
+		err = i915_gem_object_pin_pages_unlocked(obj);
 		if (err) {
 			if (err == -ENXIO || err == -E2BIG) {
 				i915_gem_object_put(obj);
@@ -1327,8 +1337,10 @@ static int igt_ppgtt_smoke_huge(void *arg)
 			       __func__, size, i);
 		}
 out_unpin:
+		i915_gem_object_lock(obj, NULL);
 		i915_gem_object_unpin_pages(obj);
 		__i915_gem_object_put_pages(obj);
+		i915_gem_object_unlock(obj);
 out_put:
 		i915_gem_object_put(obj);
 
@@ -1402,7 +1414,7 @@ static int igt_ppgtt_sanity_check(void *arg)
 				return err;
 			}
 
-			err = i915_gem_object_pin_pages(obj);
+			err = i915_gem_object_pin_pages_unlocked(obj);
 			if (err) {
 				i915_gem_object_put(obj);
 				goto out;
@@ -1416,8 +1428,10 @@ static int igt_ppgtt_sanity_check(void *arg)
 
 			err = igt_write_huge(ctx, obj);
 
+			i915_gem_object_lock(obj, NULL);
 			i915_gem_object_unpin_pages(obj);
 			__i915_gem_object_put_pages(obj);
+			i915_gem_object_unlock(obj);
 			i915_gem_object_put(obj);
 
 			if (err) {
@@ -1462,7 +1476,7 @@ static int igt_tmpfs_fallback(void *arg)
 		goto out_restore;
 	}
 
-	vaddr = i915_gem_object_pin_map(obj, I915_MAP_WB);
+	vaddr = i915_gem_object_pin_map_unlocked(obj, I915_MAP_WB);
 	if (IS_ERR(vaddr)) {
 		err = PTR_ERR(vaddr);
 		goto out_put;
Signature · 6 lines
2.26.2

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel
#55 lore raw

RFC PATCH 51/162 drm/i915/selftests: Prepare client blit for obj->mm.lock removal.

In reply to Matthew Auld

From: Maarten Lankhorst <[email protected]>

Straightforward conversion, just convert a bunch of calls to unlocked versions.

---
 drivers/gpu/drm/i915/gem/selftests/i915_gem_client_blt.c | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)
drivers/gpu/drm/i915/gem/selftests/i915_gem_client_blt.c +4 -4
index 4e36d4897ea6..cc782569765f 100644
@@ -47,7 +47,7 @@ static int __igt_client_fill(struct intel_engine_cs *engine)
 			goto err_flush;
 		}
 
-		vaddr = i915_gem_object_pin_map(obj, I915_MAP_WB);
+		vaddr = i915_gem_object_pin_map_unlocked(obj, I915_MAP_WB);
 		if (IS_ERR(vaddr)) {
 			err = PTR_ERR(vaddr);
 			goto err_put;
@@ -159,7 +159,7 @@ static int prepare_blit(const struct tiled_blits *t,
 	u32 src_pitch, dst_pitch;
 	u32 cmd, *cs;
 
-	cs = i915_gem_object_pin_map(batch, I915_MAP_WC);
+	cs = i915_gem_object_pin_map_unlocked(batch, I915_MAP_WC);
 	if (IS_ERR(cs))
 		return PTR_ERR(cs);
 
@@ -379,7 +379,7 @@ static int verify_buffer(const struct tiled_blits *t,
 	y = i915_prandom_u32_max_state(t->height, prng);
 	p = y * t->width + x;
 
-	vaddr = i915_gem_object_pin_map(buf->vma->obj, I915_MAP_WC);
+	vaddr = i915_gem_object_pin_map_unlocked(buf->vma->obj, I915_MAP_WC);
 	if (IS_ERR(vaddr))
 		return PTR_ERR(vaddr);
 
@@ -566,7 +566,7 @@ static int tiled_blits_prepare(struct tiled_blits *t,
 	int err;
 	int i;
 
-	map = i915_gem_object_pin_map(t->scratch.vma->obj, I915_MAP_WC);
+	map = i915_gem_object_pin_map_unlocked(t->scratch.vma->obj, I915_MAP_WC);
 	if (IS_ERR(map))
 		return PTR_ERR(map);
 
Signature · 6 lines
2.26.2

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
#57 lore raw

RFC PATCH 53/162 drm/i915/selftests: Prepare context tests for obj->mm.lock removal.

In reply to Matthew Auld

From: Maarten Lankhorst <[email protected]>

Straightforward conversion, just convert a bunch of calls to unlocked versions.

---
 drivers/gpu/drm/i915/gem/selftests/i915_gem_context.c | 10 +++++-----
 1 file changed, 5 insertions(+), 5 deletions(-)
drivers/gpu/drm/i915/gem/selftests/i915_gem_context.c +5 -5
index d3f87dc4eda3..5fef592390cb 100644
@@ -1094,7 +1094,7 @@ __read_slice_count(struct intel_context *ce,
 	if (ret < 0)
 		return ret;
 
-	buf = i915_gem_object_pin_map(obj, I915_MAP_WB);
+	buf = i915_gem_object_pin_map_unlocked(obj, I915_MAP_WB);
 	if (IS_ERR(buf)) {
 		ret = PTR_ERR(buf);
 		return ret;
@@ -1511,7 +1511,7 @@ static int write_to_scratch(struct i915_gem_context *ctx,
 	if (IS_ERR(obj))
 		return PTR_ERR(obj);
 
-	cmd = i915_gem_object_pin_map(obj, I915_MAP_WB);
+	cmd = i915_gem_object_pin_map_unlocked(obj, I915_MAP_WB);
 	if (IS_ERR(cmd)) {
 		err = PTR_ERR(cmd);
 		goto out;
@@ -1622,7 +1622,7 @@ static int read_from_scratch(struct i915_gem_context *ctx,
 		if (err)
 			goto out_vm;
 
-		cmd = i915_gem_object_pin_map(obj, I915_MAP_WB);
+		cmd = i915_gem_object_pin_map_unlocked(obj, I915_MAP_WB);
 		if (IS_ERR(cmd)) {
 			err = PTR_ERR(cmd);
 			goto out;
@@ -1658,7 +1658,7 @@ static int read_from_scratch(struct i915_gem_context *ctx,
 		if (err)
 			goto out_vm;
 
-		cmd = i915_gem_object_pin_map(obj, I915_MAP_WB);
+		cmd = i915_gem_object_pin_map_unlocked(obj, I915_MAP_WB);
 		if (IS_ERR(cmd)) {
 			err = PTR_ERR(cmd);
 			goto out;
@@ -1715,7 +1715,7 @@ static int read_from_scratch(struct i915_gem_context *ctx,
 	if (err)
 		goto out_vm;
 
-	cmd = i915_gem_object_pin_map(obj, I915_MAP_WB);
+	cmd = i915_gem_object_pin_map_unlocked(obj, I915_MAP_WB);
 	if (IS_ERR(cmd)) {
 		err = PTR_ERR(cmd);
 		goto out_vm;
Signature · 6 lines
2.26.2

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
#56 lore raw

RFC PATCH 52/162 drm/i915/selftests: Prepare coherency tests for obj->mm.lock removal.

In reply to Matthew Auld

From: Maarten Lankhorst <[email protected]>

Straightforward conversion, just convert a bunch of calls to unlocked versions.

---
 drivers/gpu/drm/i915/gem/selftests/i915_gem_coherency.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
drivers/gpu/drm/i915/gem/selftests/i915_gem_coherency.c +2 -2
index 2e439bb269d6..42aa3c5e0621 100644
@@ -159,7 +159,7 @@ static int wc_set(struct context *ctx, unsigned long offset, u32 v)
 	if (err)
 		return err;
 
-	map = i915_gem_object_pin_map(ctx->obj, I915_MAP_WC);
+	map = i915_gem_object_pin_map_unlocked(ctx->obj, I915_MAP_WC);
 	if (IS_ERR(map))
 		return PTR_ERR(map);
 
@@ -182,7 +182,7 @@ static int wc_get(struct context *ctx, unsigned long offset, u32 *v)
 	if (err)
 		return err;
 
-	map = i915_gem_object_pin_map(ctx->obj, I915_MAP_WC);
+	map = i915_gem_object_pin_map_unlocked(ctx->obj, I915_MAP_WC);
 	if (IS_ERR(map))
 		return PTR_ERR(map);
 
Signature · 6 lines
2.26.2

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel
#58 lore raw

RFC PATCH 54/162 drm/i915/selftests: Prepare dma-buf tests for obj->mm.lock removal.

In reply to Matthew Auld

From: Maarten Lankhorst <[email protected]>

Use pin_pages_unlocked() where we don't have a lock.

---
 drivers/gpu/drm/i915/gem/selftests/i915_gem_dmabuf.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/gpu/drm/i915/gem/selftests/i915_gem_dmabuf.c +1 -1
index b6d43880b0c1..dd74bc09ec88 100644
@@ -194,7 +194,7 @@ static int igt_dmabuf_import_ownership(void *arg)
 
 	dma_buf_put(dmabuf);
 
-	err = i915_gem_object_pin_pages(obj);
+	err = i915_gem_object_pin_pages_unlocked(obj);
 	if (err) {
 		pr_err("i915_gem_object_pin_pages failed with err=%d\n", err);
 		goto out_obj;
Signature · 6 lines
2.26.2

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
#59 lore raw

RFC PATCH 55/162 drm/i915/selftests: Prepare execbuf tests for obj->mm.lock removal.

In reply to Matthew Auld

From: Maarten Lankhorst <[email protected]>

Also quite simple, a single call needs to use the unlocked version.

---
 drivers/gpu/drm/i915/gem/selftests/i915_gem_execbuffer.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/gpu/drm/i915/gem/selftests/i915_gem_execbuffer.c +1 -1
index e1d50a5a1477..4df505e4c53a 100644
@@ -116,7 +116,7 @@ static int igt_gpu_reloc(void *arg)
 	if (IS_ERR(scratch))
 		return PTR_ERR(scratch);
 
-	map = i915_gem_object_pin_map(scratch, I915_MAP_WC);
+	map = i915_gem_object_pin_map_unlocked(scratch, I915_MAP_WC);
 	if (IS_ERR(map)) {
 		err = PTR_ERR(map);
 		goto err_scratch;
Signature · 6 lines
2.26.2

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
#63 lore raw

RFC PATCH 59/162 drm/i915/selftests: Prepare igt_gem_utils for obj->mm.lock removal

In reply to Matthew Auld

From: Maarten Lankhorst <[email protected]>

igt_emit_store_dw needs to use the unlocked version, as it's not holding a lock. This fixes igt_gpu_fill_dw() which is used by some other selftests.

---
 drivers/gpu/drm/i915/gem/selftests/igt_gem_utils.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/gpu/drm/i915/gem/selftests/igt_gem_utils.c +1 -1
index e21b5023ca7d..f4e85b4a347d 100644
@@ -54,7 +54,7 @@ igt_emit_store_dw(struct i915_vma *vma,
 	if (IS_ERR(obj))
 		return ERR_CAST(obj);
 
-	cmd = i915_gem_object_pin_map(obj, I915_MAP_WC);
+	cmd = i915_gem_object_pin_map_unlocked(obj, I915_MAP_WC);
 	if (IS_ERR(cmd)) {
 		err = PTR_ERR(cmd);
 		goto err;
Signature · 6 lines
2.26.2

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
#62 lore raw

RFC PATCH 58/162 drm/i915/selftests: Prepare object blit tests for obj->mm.lock removal.

In reply to Matthew Auld

From: Maarten Lankhorst <[email protected]>

Use some unlocked versions where we're not holding the ww lock.

---
 drivers/gpu/drm/i915/gem/selftests/i915_gem_object_blt.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)
drivers/gpu/drm/i915/gem/selftests/i915_gem_object_blt.c +3 -3
index 23b6e11bbc3e..ee9496f3d11d 100644
@@ -262,7 +262,7 @@ static int igt_fill_blt_thread(void *arg)
 			goto err_flush;
 		}
 
-		vaddr = i915_gem_object_pin_map(obj, I915_MAP_WB);
+		vaddr = i915_gem_object_pin_map_unlocked(obj, I915_MAP_WB);
 		if (IS_ERR(vaddr)) {
 			err = PTR_ERR(vaddr);
 			goto err_put;
@@ -380,7 +380,7 @@ static int igt_copy_blt_thread(void *arg)
 			goto err_flush;
 		}
 
-		vaddr = i915_gem_object_pin_map(src, I915_MAP_WB);
+		vaddr = i915_gem_object_pin_map_unlocked(src, I915_MAP_WB);
 		if (IS_ERR(vaddr)) {
 			err = PTR_ERR(vaddr);
 			goto err_put_src;
@@ -400,7 +400,7 @@ static int igt_copy_blt_thread(void *arg)
 			goto err_put_src;
 		}
 
-		vaddr = i915_gem_object_pin_map(dst, I915_MAP_WB);
+		vaddr = i915_gem_object_pin_map_unlocked(dst, I915_MAP_WB);
 		if (IS_ERR(vaddr)) {
 			err = PTR_ERR(vaddr);
 			goto err_put_dst;
Signature · 6 lines
2.26.2

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
#61 lore raw

RFC PATCH 57/162 drm/i915/selftests: Prepare object tests for obj->mm.lock removal.

In reply to Matthew Auld

From: Maarten Lankhorst <[email protected]>

Convert a single pin_pages call to use the unlocked version.

---
 drivers/gpu/drm/i915/gem/selftests/i915_gem_object.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/gpu/drm/i915/gem/selftests/i915_gem_object.c +1 -1
index bf853c40ec65..740ee8086a27 100644
@@ -47,7 +47,7 @@ static int igt_gem_huge(void *arg)
 	if (IS_ERR(obj))
 		return PTR_ERR(obj);
 
-	err = i915_gem_object_pin_pages(obj);
+	err = i915_gem_object_pin_pages_unlocked(obj);
 	if (err) {
 		pr_err("Failed to allocate %u pages (%lu total), err=%d\n",
 		       nreal, obj->base.size / PAGE_SIZE, err);
Signature · 6 lines
2.26.2

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
#60 lore raw

RFC PATCH 56/162 drm/i915/selftests: Prepare mman testcases for obj->mm.lock removal.

In reply to Matthew Auld

From: Maarten Lankhorst <[email protected]>

Ensure we hold the lock around put_pages, and use the unlocked wrappers for pinning pages and mappings.

---
 drivers/gpu/drm/i915/gem/selftests/i915_gem_mman.c | 10 ++++++----
 1 file changed, 6 insertions(+), 4 deletions(-)
drivers/gpu/drm/i915/gem/selftests/i915_gem_mman.c +6 -4
index 3ac7628f3bc4..85fff8bed08c 100644
@@ -321,7 +321,7 @@ static int igt_partial_tiling(void *arg)
 	if (IS_ERR(obj))
 		return PTR_ERR(obj);
 
-	err = i915_gem_object_pin_pages(obj);
+	err = i915_gem_object_pin_pages_unlocked(obj);
 	if (err) {
 		pr_err("Failed to allocate %u pages (%lu total), err=%d\n",
 		       nreal, obj->base.size / PAGE_SIZE, err);
@@ -458,7 +458,7 @@ static int igt_smoke_tiling(void *arg)
 	if (IS_ERR(obj))
 		return PTR_ERR(obj);
 
-	err = i915_gem_object_pin_pages(obj);
+	err = i915_gem_object_pin_pages_unlocked(obj);
 	if (err) {
 		pr_err("Failed to allocate %u pages (%lu total), err=%d\n",
 		       nreal, obj->base.size / PAGE_SIZE, err);
@@ -797,7 +797,7 @@ static int wc_set(struct drm_i915_gem_object *obj)
 {
 	void *vaddr;
 
-	vaddr = i915_gem_object_pin_map(obj, I915_MAP_WC);
+	vaddr = i915_gem_object_pin_map_unlocked(obj, I915_MAP_WC);
 	if (IS_ERR(vaddr))
 		return PTR_ERR(vaddr);
 
@@ -813,7 +813,7 @@ static int wc_check(struct drm_i915_gem_object *obj)
 	void *vaddr;
 	int err = 0;
 
-	vaddr = i915_gem_object_pin_map(obj, I915_MAP_WC);
+	vaddr = i915_gem_object_pin_map_unlocked(obj, I915_MAP_WC);
 	if (IS_ERR(vaddr))
 		return PTR_ERR(vaddr);
 
@@ -1315,7 +1315,9 @@ static int __igt_mmap_revoke(struct drm_i915_private *i915,
 	}
 
 	if (type != I915_MMAP_TYPE_GTT) {
+		i915_gem_object_lock(obj, NULL);
 		__i915_gem_object_put_pages(obj);
+		i915_gem_object_unlock(obj);
 		if (i915_gem_object_has_pages(obj)) {
 			pr_err("Failed to put-pages object!\n");
 			err = -EINVAL;
Signature · 6 lines
2.26.2

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel
#64 lore raw

RFC PATCH 60/162 drm/i915/selftests: Prepare context selftest for obj->mm.lock removal

In reply to Matthew Auld

From: Maarten Lankhorst <[email protected]>

Only needs to convert a single call to the unlocked version.

---
 drivers/gpu/drm/i915/gt/selftest_context.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
drivers/gpu/drm/i915/gt/selftest_context.c +2 -2
index 1f4020e906a8..d9b0ebc938f1 100644
@@ -88,8 +88,8 @@ static int __live_context_size(struct intel_engine_cs *engine)
 	if (err)
 		goto err;
 
-	vaddr = i915_gem_object_pin_map(ce->state->obj,
-					i915_coherent_map_type(engine->i915));
+	vaddr = i915_gem_object_pin_map_unlocked(ce->state->obj,
+						 i915_coherent_map_type(engine->i915));
 	if (IS_ERR(vaddr)) {
 		err = PTR_ERR(vaddr);
 		intel_context_unpin(ce);
Signature · 6 lines
2.26.2

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
#65 lore raw

RFC PATCH 61/162 drm/i915/selftests: Prepare hangcheck for obj->mm.lock removal

In reply to Matthew Auld

From: Maarten Lankhorst <[email protected]>

Convert a few calls to use the unlocked versions.

---
 drivers/gpu/drm/i915/gt/selftest_hangcheck.c | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)
drivers/gpu/drm/i915/gt/selftest_hangcheck.c +4 -4
index fb5ebf930ab2..e3027cebab5b 100644
@@ -80,15 +80,15 @@ static int hang_init(struct hang *h, struct intel_gt *gt)
 	}
 
 	i915_gem_object_set_cache_coherency(h->hws, I915_CACHE_LLC);
-	vaddr = i915_gem_object_pin_map(h->hws, I915_MAP_WB);
+	vaddr = i915_gem_object_pin_map_unlocked(h->hws, I915_MAP_WB);
 	if (IS_ERR(vaddr)) {
 		err = PTR_ERR(vaddr);
 		goto err_obj;
 	}
 	h->seqno = memset(vaddr, 0xff, PAGE_SIZE);
 
-	vaddr = i915_gem_object_pin_map(h->obj,
-					i915_coherent_map_type(gt->i915));
+	vaddr = i915_gem_object_pin_map_unlocked(h->obj,
+						 i915_coherent_map_type(gt->i915));
 	if (IS_ERR(vaddr)) {
 		err = PTR_ERR(vaddr);
 		goto err_unpin_hws;
@@ -149,7 +149,7 @@ hang_create_request(struct hang *h, struct intel_engine_cs *engine)
 		return ERR_CAST(obj);
 	}
 
-	vaddr = i915_gem_object_pin_map(obj, i915_coherent_map_type(gt->i915));
+	vaddr = i915_gem_object_pin_map_unlocked(obj, i915_coherent_map_type(gt->i915));
 	if (IS_ERR(vaddr)) {
 		i915_gem_object_put(obj);
 		i915_vm_put(vm);
Signature · 6 lines
2.26.2

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
#66 lore raw

RFC PATCH 62/162 drm/i915/selftests: Prepare execlists for obj->mm.lock removal

In reply to Matthew Auld

From: Maarten Lankhorst <[email protected]>

Convert normal functions to unlocked versions where needed.

---
 drivers/gpu/drm/i915/gt/selftest_execlists.c | 34 ++++++++++----------
 1 file changed, 17 insertions(+), 17 deletions(-)
drivers/gpu/drm/i915/gt/selftest_execlists.c +17 -17
index 95d41c01d0e0..124011f6fb51 100644
@@ -1007,7 +1007,7 @@ static int live_timeslice_preempt(void *arg)
 		goto err_obj;
 	}
 
-	vaddr = i915_gem_object_pin_map(obj, I915_MAP_WC);
+	vaddr = i915_gem_object_pin_map_unlocked(obj, I915_MAP_WC);
 	if (IS_ERR(vaddr)) {
 		err = PTR_ERR(vaddr);
 		goto err_obj;
@@ -1315,7 +1315,7 @@ static int live_timeslice_queue(void *arg)
 		goto err_obj;
 	}
 
-	vaddr = i915_gem_object_pin_map(obj, I915_MAP_WC);
+	vaddr = i915_gem_object_pin_map_unlocked(obj, I915_MAP_WC);
 	if (IS_ERR(vaddr)) {
 		err = PTR_ERR(vaddr);
 		goto err_obj;
@@ -1562,7 +1562,7 @@ static int live_busywait_preempt(void *arg)
 		goto err_ctx_lo;
 	}
 
-	map = i915_gem_object_pin_map(obj, I915_MAP_WC);
+	map = i915_gem_object_pin_map_unlocked(obj, I915_MAP_WC);
 	if (IS_ERR(map)) {
 		err = PTR_ERR(map);
 		goto err_obj;
@@ -2678,7 +2678,7 @@ static int create_gang(struct intel_engine_cs *engine,
 	if (err)
 		goto err_obj;
 
-	cs = i915_gem_object_pin_map(obj, I915_MAP_WC);
+	cs = i915_gem_object_pin_map_unlocked(obj, I915_MAP_WC);
 	if (IS_ERR(cs))
 		goto err_obj;
 
@@ -2960,7 +2960,7 @@ static int live_preempt_gang(void *arg)
 		 * it will terminate the next lowest spinner until there
 		 * are no more spinners and the gang is complete.
 		 */
-		cs = i915_gem_object_pin_map(rq->batch->obj, I915_MAP_WC);
+		cs = i915_gem_object_pin_map_unlocked(rq->batch->obj, I915_MAP_WC);
 		if (!IS_ERR(cs)) {
 			*cs = 0;
 			i915_gem_object_unpin_map(rq->batch->obj);
@@ -3025,7 +3025,7 @@ create_gpr_user(struct intel_engine_cs *engine,
 		return ERR_PTR(err);
 	}
 
-	cs = i915_gem_object_pin_map(obj, I915_MAP_WC);
+	cs = i915_gem_object_pin_map_unlocked(obj, I915_MAP_WC);
 	if (IS_ERR(cs)) {
 		i915_vma_put(vma);
 		return ERR_CAST(cs);
@@ -3235,7 +3235,7 @@ static int live_preempt_user(void *arg)
 	if (IS_ERR(global))
 		return PTR_ERR(global);
 
-	result = i915_gem_object_pin_map(global->obj, I915_MAP_WC);
+	result = i915_gem_object_pin_map_unlocked(global->obj, I915_MAP_WC);
 	if (IS_ERR(result)) {
 		i915_vma_unpin_and_release(&global, 0);
 		return PTR_ERR(result);
@@ -3628,7 +3628,7 @@ static int live_preempt_smoke(void *arg)
 		goto err_free;
 	}
 
-	cs = i915_gem_object_pin_map(smoke.batch, I915_MAP_WB);
+	cs = i915_gem_object_pin_map_unlocked(smoke.batch, I915_MAP_WB);
 	if (IS_ERR(cs)) {
 		err = PTR_ERR(cs);
 		goto err_batch;
@@ -4231,7 +4231,7 @@ static int preserved_virtual_engine(struct intel_gt *gt,
 		goto out_end;
 	}
 
-	cs = i915_gem_object_pin_map(scratch->obj, I915_MAP_WB);
+	cs = i915_gem_object_pin_map_unlocked(scratch->obj, I915_MAP_WB);
 	if (IS_ERR(cs)) {
 		err = PTR_ERR(cs);
 		goto out_end;
@@ -5259,7 +5259,7 @@ static int __live_lrc_gpr(struct intel_engine_cs *engine,
 		goto err_rq;
 	}
 
-	cs = i915_gem_object_pin_map(scratch->obj, I915_MAP_WB);
+	cs = i915_gem_object_pin_map_unlocked(scratch->obj, I915_MAP_WB);
 	if (IS_ERR(cs)) {
 		err = PTR_ERR(cs);
 		goto err_rq;
@@ -5553,7 +5553,7 @@ store_context(struct intel_context *ce, struct i915_vma *scratch)
 	if (IS_ERR(batch))
 		return batch;
 
-	cs = i915_gem_object_pin_map(batch->obj, I915_MAP_WC);
+	cs = i915_gem_object_pin_map_unlocked(batch->obj, I915_MAP_WC);
 	if (IS_ERR(cs)) {
 		i915_vma_put(batch);
 		return ERR_CAST(cs);
@@ -5717,7 +5717,7 @@ static struct i915_vma *load_context(struct intel_context *ce, u32 poison)
 	if (IS_ERR(batch))
 		return batch;
 
-	cs = i915_gem_object_pin_map(batch->obj, I915_MAP_WC);
+	cs = i915_gem_object_pin_map_unlocked(batch->obj, I915_MAP_WC);
 	if (IS_ERR(cs)) {
 		i915_vma_put(batch);
 		return ERR_CAST(cs);
@@ -5831,29 +5831,29 @@ static int compare_isolation(struct intel_engine_cs *engine,
 	u32 *defaults;
 	int err = 0;
 
-	A[0] = i915_gem_object_pin_map(ref[0]->obj, I915_MAP_WC);
+	A[0] = i915_gem_object_pin_map_unlocked(ref[0]->obj, I915_MAP_WC);
 	if (IS_ERR(A[0]))
 		return PTR_ERR(A[0]);
 
-	A[1] = i915_gem_object_pin_map(ref[1]->obj, I915_MAP_WC);
+	A[1] = i915_gem_object_pin_map_unlocked(ref[1]->obj, I915_MAP_WC);
 	if (IS_ERR(A[1])) {
 		err = PTR_ERR(A[1]);
 		goto err_A0;
 	}
 
-	B[0] = i915_gem_object_pin_map(result[0]->obj, I915_MAP_WC);
+	B[0] = i915_gem_object_pin_map_unlocked(result[0]->obj, I915_MAP_WC);
 	if (IS_ERR(B[0])) {
 		err = PTR_ERR(B[0]);
 		goto err_A1;
 	}
 
-	B[1] = i915_gem_object_pin_map(result[1]->obj, I915_MAP_WC);
+	B[1] = i915_gem_object_pin_map_unlocked(result[1]->obj, I915_MAP_WC);
 	if (IS_ERR(B[1])) {
 		err = PTR_ERR(B[1]);
 		goto err_B0;
 	}
 
-	lrc = i915_gem_object_pin_map(ce->state->obj,
+	lrc = i915_gem_object_pin_map_unlocked(ce->state->obj,
 				      i915_coherent_map_type(engine->i915));
 	if (IS_ERR(lrc)) {
 		err = PTR_ERR(lrc);
Signature · 6 lines
2.26.2

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
#67 lore raw

RFC PATCH 63/162 drm/i915/selftests: Prepare mocs tests for obj->mm.lock removal

In reply to Matthew Auld

From: Maarten Lankhorst <[email protected]>

Use pin_map_unlocked when we're not holding locks.

---
 drivers/gpu/drm/i915/gt/selftest_mocs.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/gpu/drm/i915/gt/selftest_mocs.c +1 -1
index 21dcd91cbd62..eadb41b76d33 100644
@@ -105,7 +105,7 @@ static int live_mocs_init(struct live_mocs *arg, struct intel_gt *gt)
 	if (IS_ERR(arg->scratch))
 		return PTR_ERR(arg->scratch);
 
-	arg->vaddr = i915_gem_object_pin_map(arg->scratch->obj, I915_MAP_WB);
+	arg->vaddr = i915_gem_object_pin_map_unlocked(arg->scratch->obj, I915_MAP_WB);
 	if (IS_ERR(arg->vaddr)) {
 		err = PTR_ERR(arg->vaddr);
 		goto err_scratch;
Signature · 6 lines
2.26.2

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
#69 lore raw

RFC PATCH 65/162 drm/i915/selftests: Prepare timeline tests for obj->mm.lock removal

In reply to Matthew Auld

From: Maarten Lankhorst <[email protected]>

We can no longer call intel_timeline_pin with a null argument, so add a ww loop that locks the backing object.

---
 drivers/gpu/drm/i915/gt/selftest_timeline.c | 28 ++++++++++++++++++---
 1 file changed, 24 insertions(+), 4 deletions(-)
drivers/gpu/drm/i915/gt/selftest_timeline.c +24 -4
index 7435abf5a703..d468147a03de 100644
@@ -37,6 +37,26 @@ static unsigned long hwsp_cacheline(struct intel_timeline *tl)
 	return (address + offset_in_page(tl->hwsp_offset)) / CACHELINE_BYTES;
 }
 
+static int selftest_tl_pin(struct intel_timeline *tl)
+{
+	struct i915_gem_ww_ctx ww;
+	int err;
+
+	i915_gem_ww_ctx_init(&ww, false);
+retry:
+	err = i915_gem_object_lock(tl->hwsp_ggtt->obj, &ww);
+	if (!err)
+		err = intel_timeline_pin(tl, &ww);
+
+	if (err == -EDEADLK) {
+		err = i915_gem_ww_ctx_backoff(&ww);
+		if (!err)
+			goto retry;
+	}
+	i915_gem_ww_ctx_fini(&ww);
+	return err;
+}
+
 #define CACHELINES_PER_PAGE (PAGE_SIZE / CACHELINE_BYTES)
 
 struct mock_hwsp_freelist {
@@ -78,7 +98,7 @@ static int __mock_hwsp_timeline(struct mock_hwsp_freelist *state,
 		if (IS_ERR(tl))
 			return PTR_ERR(tl);
 
-		err = intel_timeline_pin(tl, NULL);
+		err = selftest_tl_pin(tl);
 		if (err) {
 			intel_timeline_put(tl);
 			return err;
@@ -464,7 +484,7 @@ checked_tl_write(struct intel_timeline *tl, struct intel_engine_cs *engine, u32
 	struct i915_request *rq;
 	int err;
 
-	err = intel_timeline_pin(tl, NULL);
+	err = selftest_tl_pin(tl);
 	if (err) {
 		rq = ERR_PTR(err);
 		goto out;
@@ -664,7 +684,7 @@ static int live_hwsp_wrap(void *arg)
 	if (!tl->has_initial_breadcrumb || !tl->hwsp_cacheline)
 		goto out_free;
 
-	err = intel_timeline_pin(tl, NULL);
+	err = selftest_tl_pin(tl);
 	if (err)
 		goto out_free;
 
@@ -811,7 +831,7 @@ static int setup_watcher(struct hwsp_watcher *w, struct intel_gt *gt)
 	if (IS_ERR(obj))
 		return PTR_ERR(obj);
 
-	w->map = i915_gem_object_pin_map(obj, I915_MAP_WB);
+	w->map = i915_gem_object_pin_map_unlocked(obj, I915_MAP_WB);
 	if (IS_ERR(w->map)) {
 		i915_gem_object_put(obj);
 		return PTR_ERR(w->map);
Signature · 6 lines
2.26.2

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
#70 lore raw

RFC PATCH 66/162 drm/i915/selftests: Prepare i915_request tests for obj->mm.lock removal

In reply to Matthew Auld

From: Maarten Lankhorst <[email protected]>

Straightforward conversion by using unlocked versions.

---
 drivers/gpu/drm/i915/selftests/i915_request.c | 10 +++++-----
 1 file changed, 5 insertions(+), 5 deletions(-)
drivers/gpu/drm/i915/selftests/i915_request.c +5 -5
index e424a6d1a68c..514fa109e40f 100644
@@ -619,7 +619,7 @@ static struct i915_vma *empty_batch(struct drm_i915_private *i915)
 	if (IS_ERR(obj))
 		return ERR_CAST(obj);
 
-	cmd = i915_gem_object_pin_map(obj, I915_MAP_WB);
+	cmd = i915_gem_object_pin_map_unlocked(obj, I915_MAP_WB);
 	if (IS_ERR(cmd)) {
 		err = PTR_ERR(cmd);
 		goto err;
@@ -781,7 +781,7 @@ static struct i915_vma *recursive_batch(struct drm_i915_private *i915)
 	if (err)
 		goto err;
 
-	cmd = i915_gem_object_pin_map(obj, I915_MAP_WC);
+	cmd = i915_gem_object_pin_map_unlocked(obj, I915_MAP_WC);
 	if (IS_ERR(cmd)) {
 		err = PTR_ERR(cmd);
 		goto err;
@@ -816,7 +816,7 @@ static int recursive_batch_resolve(struct i915_vma *batch)
 {
 	u32 *cmd;
 
-	cmd = i915_gem_object_pin_map(batch->obj, I915_MAP_WC);
+	cmd = i915_gem_object_pin_map_unlocked(batch->obj, I915_MAP_WC);
 	if (IS_ERR(cmd))
 		return PTR_ERR(cmd);
 
@@ -1069,8 +1069,8 @@ static int live_sequential_engines(void *arg)
 		if (!request[idx])
 			break;
 
-		cmd = i915_gem_object_pin_map(request[idx]->batch->obj,
-					      I915_MAP_WC);
+		cmd = i915_gem_object_pin_map_unlocked(request[idx]->batch->obj,
+						       I915_MAP_WC);
 		if (!IS_ERR(cmd)) {
 			*cmd = MI_BATCH_BUFFER_END;
 
Signature · 6 lines
2.26.2

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
#71 lore raw

RFC PATCH 67/162 drm/i915/selftests: Prepare memory region tests for obj->mm.lock removal

In reply to Matthew Auld

From: Maarten Lankhorst <[email protected]>

Use the unlocked variants for pin_map and pin_pages, and add lock around unpinning/putting pages.

---
 .../drm/i915/selftests/intel_memory_region.c   | 18 +++++++++++-------
 1 file changed, 11 insertions(+), 7 deletions(-)
drivers/gpu/drm/i915/selftests/intel_memory_region.c +11 -7
index 27389fb19951..9c20b7065fc5 100644
@@ -31,10 +31,12 @@ static void close_objects(struct intel_memory_region *mem,
 	struct drm_i915_gem_object *obj, *on;
 
 	list_for_each_entry_safe(obj, on, objects, st_link) {
+		i915_gem_object_lock(obj, NULL);
 		if (i915_gem_object_has_pinned_pages(obj))
 			i915_gem_object_unpin_pages(obj);
 		/* No polluting the memory region between tests */
 		__i915_gem_object_put_pages(obj);
+		i915_gem_object_unlock(obj);
 		list_del(&obj->st_link);
 		i915_gem_object_put(obj);
 	}
@@ -69,7 +71,7 @@ static int igt_mock_fill(void *arg)
 			break;
 		}
 
-		err = i915_gem_object_pin_pages(obj);
+		err = i915_gem_object_pin_pages_unlocked(obj);
 		if (err) {
 			i915_gem_object_put(obj);
 			break;
@@ -109,7 +111,7 @@ igt_object_create(struct intel_memory_region *mem,
 	if (IS_ERR(obj))
 		return obj;
 
-	err = i915_gem_object_pin_pages(obj);
+	err = i915_gem_object_pin_pages_unlocked(obj);
 	if (err)
 		goto put;
 
@@ -123,8 +125,10 @@ igt_object_create(struct intel_memory_region *mem,
 
 static void igt_object_release(struct drm_i915_gem_object *obj)
 {
+	i915_gem_object_lock(obj, NULL);
 	i915_gem_object_unpin_pages(obj);
 	__i915_gem_object_put_pages(obj);
+	i915_gem_object_unlock(obj);
 	list_del(&obj->st_link);
 	i915_gem_object_put(obj);
 }
@@ -356,7 +360,7 @@ static int igt_cpu_check(struct drm_i915_gem_object *obj, u32 dword, u32 val)
 	if (err)
 		return err;
 
-	ptr = i915_gem_object_pin_map(obj, I915_MAP_WC);
+	ptr = i915_gem_object_pin_map_unlocked(obj, I915_MAP_WC);
 	if (IS_ERR(ptr))
 		return PTR_ERR(ptr);
 
@@ -461,7 +465,7 @@ static int igt_lmem_create(void *arg)
 	if (IS_ERR(obj))
 		return PTR_ERR(obj);
 
-	err = i915_gem_object_pin_pages(obj);
+	err = i915_gem_object_pin_pages_unlocked(obj);
 	if (err)
 		goto out_put;
 
@@ -500,7 +504,7 @@ static int igt_lmem_write_gpu(void *arg)
 		goto out_file;
 	}
 
-	err = i915_gem_object_pin_pages(obj);
+	err = i915_gem_object_pin_pages_unlocked(obj);
 	if (err)
 		goto out_put;
 
@@ -572,7 +576,7 @@ static int igt_lmem_write_cpu(void *arg)
 	if (IS_ERR(obj))
 		return PTR_ERR(obj);
 
-	vaddr = i915_gem_object_pin_map(obj, I915_MAP_WC);
+	vaddr = i915_gem_object_pin_map_unlocked(obj, I915_MAP_WC);
 	if (IS_ERR(vaddr)) {
 		err = PTR_ERR(vaddr);
 		goto out_put;
@@ -676,7 +680,7 @@ create_region_for_mapping(struct intel_memory_region *mr, u64 size, u32 type,
 		return obj;
 	}
 
-	addr = i915_gem_object_pin_map(obj, type);
+	addr = i915_gem_object_pin_map_unlocked(obj, type);
 	if (IS_ERR(addr)) {
 		i915_gem_object_put(obj);
 		if (PTR_ERR(addr) == -ENXIO)
Signature · 6 lines
2.26.2

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel
#72 lore raw

RFC PATCH 68/162 drm/i915/selftests: Prepare cs engine tests for obj->mm.lock removal

In reply to Matthew Auld

From: Maarten Lankhorst <[email protected]>

Same as other tests, use pin_map_unlocked.

---
 drivers/gpu/drm/i915/gt/selftest_engine_cs.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
drivers/gpu/drm/i915/gt/selftest_engine_cs.c +2 -2
index 729c3c7b11e2..853d1f02131a 100644
@@ -72,7 +72,7 @@ static struct i915_vma *create_empty_batch(struct intel_context *ce)
 	if (IS_ERR(obj))
 		return ERR_CAST(obj);
 
-	cs = i915_gem_object_pin_map(obj, I915_MAP_WB);
+	cs = i915_gem_object_pin_map_unlocked(obj, I915_MAP_WB);
 	if (IS_ERR(cs)) {
 		err = PTR_ERR(cs);
 		goto err_put;
@@ -208,7 +208,7 @@ static struct i915_vma *create_nop_batch(struct intel_context *ce)
 	if (IS_ERR(obj))
 		return ERR_CAST(obj);
 
-	cs = i915_gem_object_pin_map(obj, I915_MAP_WB);
+	cs = i915_gem_object_pin_map_unlocked(obj, I915_MAP_WB);
 	if (IS_ERR(cs)) {
 		err = PTR_ERR(cs);
 		goto err_put;
Signature · 6 lines
2.26.2

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
#73 lore raw

RFC PATCH 69/162 drm/i915/selftests: Prepare gtt tests for obj->mm.lock removal

In reply to Matthew Auld

From: Maarten Lankhorst <[email protected]>

We need to lock the global gtt dma_resv, use i915_vm_lock_objects to handle this correctly. Add ww handling for this where required.

Add the object lock around unpin/put pages, and use the unlocked versions of pin_pages and pin_map where required.

---
 drivers/gpu/drm/i915/selftests/i915_gem_gtt.c | 92 ++++++++++++++-----
 1 file changed, 67 insertions(+), 25 deletions(-)
drivers/gpu/drm/i915/selftests/i915_gem_gtt.c +67 -25
index 2cfe99c79034..d07dd6780005 100644
@@ -129,7 +129,7 @@ fake_dma_object(struct drm_i915_private *i915, u64 size)
 	obj->cache_level = I915_CACHE_NONE;
 
 	/* Preallocate the "backing storage" */
-	if (i915_gem_object_pin_pages(obj))
+	if (i915_gem_object_pin_pages_unlocked(obj))
 		goto err_obj;
 
 	i915_gem_object_unpin_pages(obj);
@@ -145,6 +145,7 @@ static int igt_ppgtt_alloc(void *arg)
 {
 	struct drm_i915_private *dev_priv = arg;
 	struct i915_ppgtt *ppgtt;
+	struct i915_gem_ww_ctx ww;
 	u64 size, last, limit;
 	int err = 0;
 
@@ -170,6 +171,12 @@ static int igt_ppgtt_alloc(void *arg)
 	limit = totalram_pages() << PAGE_SHIFT;
 	limit = min(ppgtt->vm.total, limit);
 
+	i915_gem_ww_ctx_init(&ww, false);
+retry:
+	err = i915_vm_lock_objects(&ppgtt->vm, &ww);
+	if (err)
+		goto err_ppgtt_cleanup;
+
 	/* Check we can allocate the entire range */
 	for (size = 4096; size <= limit; size <<= 2) {
 		struct i915_vm_pt_stash stash = {};
@@ -214,6 +221,13 @@ static int igt_ppgtt_alloc(void *arg)
 	}
 
 err_ppgtt_cleanup:
+	if (err == -EDEADLK) {
+		err = i915_gem_ww_ctx_backoff(&ww);
+		if (!err)
+			goto retry;
+	}
+	i915_gem_ww_ctx_fini(&ww);
+
 	i915_vm_put(&ppgtt->vm);
 	return err;
 }
@@ -275,7 +289,7 @@ static int lowlevel_hole(struct i915_address_space *vm,
 
 		GEM_BUG_ON(obj->base.size != BIT_ULL(size));
 
-		if (i915_gem_object_pin_pages(obj)) {
+		if (i915_gem_object_pin_pages_unlocked(obj)) {
 			i915_gem_object_put(obj);
 			kfree(order);
 			break;
@@ -296,20 +310,36 @@ static int lowlevel_hole(struct i915_address_space *vm,
 
 			if (vm->allocate_va_range) {
 				struct i915_vm_pt_stash stash = {};
+				struct i915_gem_ww_ctx ww;
+				int err;
+
+				i915_gem_ww_ctx_init(&ww, false);
+retry:
+				err = i915_vm_lock_objects(vm, &ww);
+				if (err)
+					goto alloc_vm_end;
 
+				err = -ENOMEM;
 				if (i915_vm_alloc_pt_stash(vm, &stash,
 							   BIT_ULL(size)))
-					break;
-
-				if (i915_vm_pin_pt_stash(vm, &stash)) {
-					i915_vm_free_pt_stash(vm, &stash);
-					break;
-				}
+					goto alloc_vm_end;
 
-				vm->allocate_va_range(vm, &stash,
-						      addr, BIT_ULL(size));
+				err = i915_vm_pin_pt_stash(vm, &stash);
+				if (!err)
+					vm->allocate_va_range(vm, &stash,
+							      addr, BIT_ULL(size));
 
 				i915_vm_free_pt_stash(vm, &stash);
+alloc_vm_end:
+				if (err == -EDEADLK) {
+					err = i915_gem_ww_ctx_backoff(&ww);
+					if (!err)
+						goto retry;
+				}
+				i915_gem_ww_ctx_fini(&ww);
+
+				if (err)
+					break;
 			}
 
 			mock_vma->pages = obj->mm.pages;
@@ -1165,7 +1195,7 @@ static int igt_ggtt_page(void *arg)
 	if (IS_ERR(obj))
 		return PTR_ERR(obj);
 
-	err = i915_gem_object_pin_pages(obj);
+	err = i915_gem_object_pin_pages_unlocked(obj);
 	if (err)
 		goto out_free;
 
@@ -1332,7 +1362,7 @@ static int igt_gtt_reserve(void *arg)
 			goto out;
 		}
 
-		err = i915_gem_object_pin_pages(obj);
+		err = i915_gem_object_pin_pages_unlocked(obj);
 		if (err) {
 			i915_gem_object_put(obj);
 			goto out;
@@ -1384,7 +1414,7 @@ static int igt_gtt_reserve(void *arg)
 			goto out;
 		}
 
-		err = i915_gem_object_pin_pages(obj);
+		err = i915_gem_object_pin_pages_unlocked(obj);
 		if (err) {
 			i915_gem_object_put(obj);
 			goto out;
@@ -1548,7 +1578,7 @@ static int igt_gtt_insert(void *arg)
 			goto out;
 		}
 
-		err = i915_gem_object_pin_pages(obj);
+		err = i915_gem_object_pin_pages_unlocked(obj);
 		if (err) {
 			i915_gem_object_put(obj);
 			goto out;
@@ -1657,7 +1687,7 @@ static int igt_gtt_insert(void *arg)
 			goto out;
 		}
 
-		err = i915_gem_object_pin_pages(obj);
+		err = i915_gem_object_pin_pages_unlocked(obj);
 		if (err) {
 			i915_gem_object_put(obj);
 			goto out;
@@ -1828,7 +1858,7 @@ static int igt_cs_tlb(void *arg)
 		goto out_vm;
 	}
 
-	batch = i915_gem_object_pin_map(bbe, I915_MAP_WC);
+	batch = i915_gem_object_pin_map_unlocked(bbe, I915_MAP_WC);
 	if (IS_ERR(batch)) {
 		err = PTR_ERR(batch);
 		goto out_put_bbe;
@@ -1844,7 +1874,7 @@ static int igt_cs_tlb(void *arg)
 	}
 
 	/* Track the execution of each request by writing into different slot */
-	batch = i915_gem_object_pin_map(act, I915_MAP_WC);
+	batch = i915_gem_object_pin_map_unlocked(act, I915_MAP_WC);
 	if (IS_ERR(batch)) {
 		err = PTR_ERR(batch);
 		goto out_put_act;
@@ -1891,7 +1921,7 @@ static int igt_cs_tlb(void *arg)
 		goto out_put_out;
 	GEM_BUG_ON(vma->node.start != vm->total - PAGE_SIZE);
 
-	result = i915_gem_object_pin_map(out, I915_MAP_WB);
+	result = i915_gem_object_pin_map_unlocked(out, I915_MAP_WB);
 	if (IS_ERR(result)) {
 		err = PTR_ERR(result);
 		goto out_put_out;
@@ -1907,6 +1937,7 @@ static int igt_cs_tlb(void *arg)
 		while (!__igt_timeout(end_time, NULL)) {
 			struct i915_vm_pt_stash stash = {};
 			struct i915_request *rq;
+			struct i915_gem_ww_ctx ww;
 			u64 offset;
 
 			offset = igt_random_offset(&prng,
@@ -1925,19 +1956,30 @@ static int igt_cs_tlb(void *arg)
 			if (err)
 				goto end;
 
+			i915_gem_ww_ctx_init(&ww, false);
+retry:
+			err = i915_vm_lock_objects(vm, &ww);
+			if (err)
+				goto end_ww;
+
 			err = i915_vm_alloc_pt_stash(vm, &stash, chunk_size);
 			if (err)
-				goto end;
+				goto end_ww;
 
 			err = i915_vm_pin_pt_stash(vm, &stash);
-			if (err) {
-				i915_vm_free_pt_stash(vm, &stash);
-				goto end;
-			}
-
-			vm->allocate_va_range(vm, &stash, offset, chunk_size);
+			if (!err)
+				vm->allocate_va_range(vm, &stash, offset, chunk_size);
 
 			i915_vm_free_pt_stash(vm, &stash);
+end_ww:
+			if (err == -EDEADLK) {
+				err = i915_gem_ww_ctx_backoff(&ww);
+				if (!err)
+					goto retry;
+			}
+			i915_gem_ww_ctx_fini(&ww);
+			if (err)
+				goto end;
 
 			/* Prime the TLB with the dummy pages */
 			for (i = 0; i < count; i++) {
Signature · 6 lines
2.26.2

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
#68 lore raw

RFC PATCH 64/162 drm/i915/selftests: Prepare ring submission for obj->mm.lock removal

In reply to Matthew Auld

From: Maarten Lankhorst <[email protected]>

Use unlocked versions when the ww lock is not held.

---
 drivers/gpu/drm/i915/gt/selftest_ring_submission.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
drivers/gpu/drm/i915/gt/selftest_ring_submission.c +2 -2
index 3350e7c995bc..99609271c3a7 100644
@@ -35,7 +35,7 @@ static struct i915_vma *create_wally(struct intel_engine_cs *engine)
 		return ERR_PTR(err);
 	}
 
-	cs = i915_gem_object_pin_map(obj, I915_MAP_WC);
+	cs = i915_gem_object_pin_map_unlocked(obj, I915_MAP_WC);
 	if (IS_ERR(cs)) {
 		i915_gem_object_put(obj);
 		return ERR_CAST(cs);
@@ -212,7 +212,7 @@ static int __live_ctx_switch_wa(struct intel_engine_cs *engine)
 	if (IS_ERR(bb))
 		return PTR_ERR(bb);
 
-	result = i915_gem_object_pin_map(bb->obj, I915_MAP_WC);
+	result = i915_gem_object_pin_map_unlocked(bb->obj, I915_MAP_WC);
 	if (IS_ERR(result)) {
 		intel_context_put(bb->private);
 		i915_vma_unpin_and_release(&bb, 0);
Signature · 6 lines
2.26.2

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel
#74 lore raw

RFC PATCH 70/162 drm/i915: Finally remove obj->mm.lock.

In reply to Matthew Auld

From: Maarten Lankhorst <[email protected]>

With all callers and selftests fixed to use ww locking, we can now finally remove this lock.

---
 drivers/gpu/drm/i915/gem/i915_gem_object.c    |  2 -
 drivers/gpu/drm/i915/gem/i915_gem_object.h    |  7 ++--
 .../gpu/drm/i915/gem/i915_gem_object_types.h  |  1 -
 drivers/gpu/drm/i915/gem/i915_gem_pages.c     | 38 ++++---------------
 drivers/gpu/drm/i915/gem/i915_gem_phys.c      | 34 ++++-------------
 drivers/gpu/drm/i915/gem/i915_gem_shmem.c     |  2 +-
 drivers/gpu/drm/i915/gem/i915_gem_shrinker.c  | 37 +++++++++++++-----
 drivers/gpu/drm/i915/gem/i915_gem_shrinker.h  |  4 +-
 drivers/gpu/drm/i915/gem/i915_gem_tiling.c    |  2 -
 drivers/gpu/drm/i915/gem/i915_gem_userptr.c   |  3 +-
 drivers/gpu/drm/i915/i915_debugfs.c           |  4 +-
 drivers/gpu/drm/i915/i915_gem.c               |  8 +---
 drivers/gpu/drm/i915/i915_gem_gtt.c           |  2 +-
 13 files changed, 54 insertions(+), 90 deletions(-)
drivers/gpu/drm/i915/gem/i915_gem_object.c +0 -2
index 028a556ab1a5..08d806bbf48e 100644
@@ -62,8 +62,6 @@ void i915_gem_object_init(struct drm_i915_gem_object *obj,
 			  const struct drm_i915_gem_object_ops *ops,
 			  struct lock_class_key *key, unsigned flags)
 {
-	mutex_init(&obj->mm.lock);
-
 	spin_lock_init(&obj->vma.lock);
 	INIT_LIST_HEAD(&obj->vma.list);
 
drivers/gpu/drm/i915/gem/i915_gem_object.h +3 -4
index 1d4b44151e0c..d0cc62d1c65e 100644
@@ -136,7 +136,7 @@ static inline void assert_object_held_shared(struct drm_i915_gem_object *obj)
 	 */
 	if (IS_ENABLED(CONFIG_LOCKDEP) &&
 	    kref_read(&obj->base.refcount) > 0)
-		lockdep_assert_held(&obj->mm.lock);
+		assert_object_held(obj);
 }
 
 static inline int __i915_gem_object_lock(struct drm_i915_gem_object *obj,
@@ -350,11 +350,11 @@ int __i915_gem_object_get_pages(struct drm_i915_gem_object *obj);
 static inline int __must_check
 i915_gem_object_pin_pages(struct drm_i915_gem_object *obj)
 {
-	might_lock(&obj->mm.lock);
-
 	if (atomic_inc_not_zero(&obj->mm.pages_pin_count))
 		return 0;
 
+	assert_object_held(obj);
+
 	return __i915_gem_object_get_pages(obj);
 }
 
@@ -396,7 +396,6 @@ i915_gem_object_unpin_pages(struct drm_i915_gem_object *obj)
 }
 
 int __i915_gem_object_put_pages(struct drm_i915_gem_object *obj);
-int __i915_gem_object_put_pages_locked(struct drm_i915_gem_object *obj);
 void i915_gem_object_truncate(struct drm_i915_gem_object *obj);
 void i915_gem_object_writeback(struct drm_i915_gem_object *obj);
 
drivers/gpu/drm/i915/gem/i915_gem_object_types.h +0 -1
index 5234c1ed62d4..b172e8cc53ab 100644
@@ -209,7 +209,6 @@ struct drm_i915_gem_object {
 		 * Protects the pages and their use. Do not use directly, but
 		 * instead go through the pin/unpin interfaces.
 		 */
-		struct mutex lock;
 		atomic_t pages_pin_count;
 		atomic_t shrink_pin;
 
drivers/gpu/drm/i915/gem/i915_gem_pages.c +7 -31
index 79336735a6e4..4a8be759832b 100644
@@ -67,7 +67,7 @@ void __i915_gem_object_set_pages(struct drm_i915_gem_object *obj,
 		struct list_head *list;
 		unsigned long flags;
 
-		lockdep_assert_held(&obj->mm.lock);
+		assert_object_held(obj);
 		spin_lock_irqsave(&i915->mm.obj_lock, flags);
 
 		i915->mm.shrink_count++;
@@ -114,9 +114,7 @@ int __i915_gem_object_get_pages(struct drm_i915_gem_object *obj)
 {
 	int err;
 
-	err = mutex_lock_interruptible(&obj->mm.lock);
-	if (err)
-		return err;
+	assert_object_held(obj);
 
 	assert_object_held_shared(obj);
 
@@ -125,15 +123,13 @@ int __i915_gem_object_get_pages(struct drm_i915_gem_object *obj)
 
 		err = ____i915_gem_object_get_pages(obj);
 		if (err)
-			goto unlock;
+			return err;
 
 		smp_mb__before_atomic();
 	}
 	atomic_inc(&obj->mm.pages_pin_count);
 
-unlock:
-	mutex_unlock(&obj->mm.lock);
-	return err;
+	return 0;
 }
 
 int i915_gem_object_pin_pages_unlocked(struct drm_i915_gem_object *obj)
@@ -220,7 +216,7 @@ __i915_gem_object_unset_pages(struct drm_i915_gem_object *obj)
 	return pages;
 }
 
-int __i915_gem_object_put_pages_locked(struct drm_i915_gem_object *obj)
+int __i915_gem_object_put_pages(struct drm_i915_gem_object *obj)
 {
 	struct sg_table *pages;
 
@@ -251,21 +247,6 @@ int __i915_gem_object_put_pages_locked(struct drm_i915_gem_object *obj)
 	return 0;
 }
 
-int __i915_gem_object_put_pages(struct drm_i915_gem_object *obj)
-{
-	int err;
-
-	if (i915_gem_object_has_pinned_pages(obj))
-		return -EBUSY;
-
-	/* May be called by shrinker from within get_pages() (on another bo) */
-	mutex_lock(&obj->mm.lock);
-	err = __i915_gem_object_put_pages_locked(obj);
-	mutex_unlock(&obj->mm.lock);
-
-	return err;
-}
-
 /* The 'mapping' part of i915_gem_object_pin_map() below */
 static void *i915_gem_object_map_page(struct drm_i915_gem_object *obj,
 		enum i915_map_type type)
@@ -366,9 +347,7 @@ void *i915_gem_object_pin_map(struct drm_i915_gem_object *obj,
 	    !i915_gem_object_type_has(obj, I915_GEM_OBJECT_HAS_IOMEM))
 		return ERR_PTR(-ENXIO);
 
-	err = mutex_lock_interruptible(&obj->mm.lock);
-	if (err)
-		return ERR_PTR(err);
+	assert_object_held(obj);
 
 	pinned = !(type & I915_MAP_OVERRIDE);
 	type &= ~I915_MAP_OVERRIDE;
@@ -416,15 +395,12 @@ void *i915_gem_object_pin_map(struct drm_i915_gem_object *obj,
 		obj->mm.mapping = page_pack_bits(ptr, type);
 	}
 
-out_unlock:
-	mutex_unlock(&obj->mm.lock);
 	return ptr;
 
 err_unpin:
 	atomic_dec(&obj->mm.pages_pin_count);
 err_unlock:
-	ptr = ERR_PTR(err);
-	goto out_unlock;
+	return ERR_PTR(err);
 }
 
 void *i915_gem_object_pin_map_unlocked(struct drm_i915_gem_object *obj,
drivers/gpu/drm/i915/gem/i915_gem_phys.c +8 -26
index f317be5f5e34..435c3b54cf14 100644
@@ -234,40 +234,22 @@ int i915_gem_object_attach_phys(struct drm_i915_gem_object *obj, int align)
 	if (err)
 		return err;
 
-	err = mutex_lock_interruptible(&obj->mm.lock);
-	if (err)
-		return err;
-
-	if (unlikely(!i915_gem_object_has_struct_page(obj)))
-		goto out;
-
-	if (obj->mm.madv != I915_MADV_WILLNEED) {
-		err = -EFAULT;
-		goto out;
-	}
+	if (obj->mm.madv != I915_MADV_WILLNEED)
+		return -EFAULT;
 
-	if (obj->mm.quirked) {
-		err = -EFAULT;
-		goto out;
-	}
+	if (obj->mm.quirked)
+		return -EFAULT;
 
-	if (obj->mm.mapping || i915_gem_object_has_pinned_pages(obj)) {
-		err = -EBUSY;
-		goto out;
-	}
+	if (obj->mm.mapping || i915_gem_object_has_pinned_pages(obj))
+		return -EBUSY;
 
 	if (unlikely(obj->mm.madv != I915_MADV_WILLNEED)) {
 		drm_dbg(obj->base.dev,
 			"Attempting to obtain a purgeable object\n");
-		err = -EFAULT;
-		goto out;
+		return -EFAULT;
 	}
 
-	err = i915_gem_object_shmem_to_phys(obj);
-
-out:
-	mutex_unlock(&obj->mm.lock);
-	return err;
+	return i915_gem_object_shmem_to_phys(obj);
 }
 
 #if IS_ENABLED(CONFIG_DRM_I915_SELFTEST)
drivers/gpu/drm/i915/gem/i915_gem_shmem.c +1 -1
index 7a59fd1ea4e5..b4dd7a709800 100644
@@ -99,7 +99,7 @@ static int shmem_get_pages(struct drm_i915_gem_object *obj)
 				goto err_sg;
 			}
 
-			i915_gem_shrink(i915, 2 * page_count, NULL, *s++);
+			i915_gem_shrink(NULL, i915, 2 * page_count, NULL, *s++);
 
 			/*
 			 * We've tried hard to allocate the memory by reaping
drivers/gpu/drm/i915/gem/i915_gem_shrinker.c +27 -10
index afc6e5b4dcf1..e42192834c88 100644
@@ -93,7 +93,8 @@ static void try_to_writeback(struct drm_i915_gem_object *obj,
  * The number of pages of backing storage actually released.
  */
 unsigned long
-i915_gem_shrink(struct drm_i915_private *i915,
+i915_gem_shrink(struct i915_gem_ww_ctx *ww,
+		struct drm_i915_private *i915,
 		unsigned long target,
 		unsigned long *nr_scanned,
 		unsigned int shrink)
@@ -112,6 +113,7 @@ i915_gem_shrink(struct drm_i915_private *i915,
 	intel_wakeref_t wakeref = 0;
 	unsigned long count = 0;
 	unsigned long scanned = 0;
+	int err;
 
 	trace_i915_gem_shrink(i915, target, shrink);
 
@@ -199,23 +201,38 @@ i915_gem_shrink(struct drm_i915_private *i915,
 
 			spin_unlock_irqrestore(&i915->mm.obj_lock, flags);
 
-			if (unsafe_drop_pages(obj, shrink) &&
-			    mutex_trylock(&obj->mm.lock)) {
+			err = 0;
+			if (unsafe_drop_pages(obj, shrink)) {
 				/* May arrive from get_pages on another bo */
-				if (!__i915_gem_object_put_pages_locked(obj)) {
+				if (!ww) {
+					if (!i915_gem_object_trylock(obj))
+						goto skip;
+				} else {
+					err = i915_gem_object_lock(obj, ww);
+					if (err)
+						goto skip;
+				}
+
+				if (!__i915_gem_object_put_pages(obj)) {
 					try_to_writeback(obj, shrink);
 					count += obj->base.size >> PAGE_SHIFT;
 				}
-				mutex_unlock(&obj->mm.lock);
+				if (!ww)
+					i915_gem_object_unlock(obj);
 			}
 
 			scanned += obj->base.size >> PAGE_SHIFT;
+skip:
 			i915_gem_object_put(obj);
 
 			spin_lock_irqsave(&i915->mm.obj_lock, flags);
+			if (err)
+				break;
 		}
 		list_splice_tail(&still_in_list, phase->list);
 		spin_unlock_irqrestore(&i915->mm.obj_lock, flags);
+		if (err)
+			return err;
 	}
 
 	if (shrink & I915_SHRINK_BOUND)
@@ -246,7 +263,7 @@ unsigned long i915_gem_shrink_all(struct drm_i915_private *i915)
 	unsigned long freed = 0;
 
 	with_intel_runtime_pm(&i915->runtime_pm, wakeref) {
-		freed = i915_gem_shrink(i915, -1UL, NULL,
+		freed = i915_gem_shrink(NULL, i915, -1UL, NULL,
 					I915_SHRINK_BOUND |
 					I915_SHRINK_UNBOUND);
 	}
@@ -292,7 +309,7 @@ i915_gem_shrinker_scan(struct shrinker *shrinker, struct shrink_control *sc)
 
 	sc->nr_scanned = 0;
 
-	freed = i915_gem_shrink(i915,
+	freed = i915_gem_shrink(NULL, i915,
 				sc->nr_to_scan,
 				&sc->nr_scanned,
 				I915_SHRINK_BOUND |
@@ -301,7 +318,7 @@ i915_gem_shrinker_scan(struct shrinker *shrinker, struct shrink_control *sc)
 		intel_wakeref_t wakeref;
 
 		with_intel_runtime_pm(&i915->runtime_pm, wakeref) {
-			freed += i915_gem_shrink(i915,
+			freed += i915_gem_shrink(NULL, i915,
 						 sc->nr_to_scan - sc->nr_scanned,
 						 &sc->nr_scanned,
 						 I915_SHRINK_ACTIVE |
@@ -326,7 +343,7 @@ i915_gem_shrinker_oom(struct notifier_block *nb, unsigned long event, void *ptr)
 
 	freed_pages = 0;
 	with_intel_runtime_pm(&i915->runtime_pm, wakeref)
-		freed_pages += i915_gem_shrink(i915, -1UL, NULL,
+		freed_pages += i915_gem_shrink(NULL, i915, -1UL, NULL,
 					       I915_SHRINK_BOUND |
 					       I915_SHRINK_UNBOUND |
 					       I915_SHRINK_WRITEBACK);
@@ -364,7 +381,7 @@ i915_gem_shrinker_vmap(struct notifier_block *nb, unsigned long event, void *ptr
 	intel_wakeref_t wakeref;
 
 	with_intel_runtime_pm(&i915->runtime_pm, wakeref)
-		freed_pages += i915_gem_shrink(i915, -1UL, NULL,
+		freed_pages += i915_gem_shrink(NULL, i915, -1UL, NULL,
 					       I915_SHRINK_BOUND |
 					       I915_SHRINK_UNBOUND |
 					       I915_SHRINK_VMAPS);
drivers/gpu/drm/i915/gem/i915_gem_shrinker.h +3 -1
index b397d7785789..8512470f6fd6 100644
@@ -9,10 +9,12 @@
 #include <linux/bits.h>
 
 struct drm_i915_private;
+struct i915_gem_ww_ctx;
 struct mutex;
 
 /* i915_gem_shrinker.c */
-unsigned long i915_gem_shrink(struct drm_i915_private *i915,
+unsigned long i915_gem_shrink(struct i915_gem_ww_ctx *ww,
+			      struct drm_i915_private *i915,
 			      unsigned long target,
 			      unsigned long *nr_scanned,
 			      unsigned flags);
drivers/gpu/drm/i915/gem/i915_gem_tiling.c +0 -2
index ffcaee74a249..4523a14db86e 100644
@@ -265,7 +265,6 @@ i915_gem_object_set_tiling(struct drm_i915_gem_object *obj,
 	 * pages to prevent them being swapped out and causing corruption
 	 * due to the change in swizzling.
 	 */
-	mutex_lock(&obj->mm.lock);
 	if (i915_gem_object_has_pages(obj) &&
 	    obj->mm.madv == I915_MADV_WILLNEED &&
 	    i915->quirks & QUIRK_PIN_SWIZZLED_PAGES) {
@@ -280,7 +279,6 @@ i915_gem_object_set_tiling(struct drm_i915_gem_object *obj,
 			obj->mm.quirked = true;
 		}
 	}
-	mutex_unlock(&obj->mm.lock);
 
 	spin_lock(&obj->vma.lock);
 	for_each_ggtt_vma(vma, obj) {
drivers/gpu/drm/i915/gem/i915_gem_userptr.c +1 -2
index 0cab9da6669e..fb4bc30fbd9a 100644
@@ -247,7 +247,7 @@ static int i915_gem_object_userptr_unbind(struct drm_i915_gem_object *obj, bool
 	if (GEM_WARN_ON(i915_gem_object_has_pinned_pages(obj)))
 		return -EBUSY;
 
-	mutex_lock(&obj->mm.lock);
+	assert_object_held(obj);
 
 	pages = __i915_gem_object_unset_pages(obj);
 	if (!IS_ERR_OR_NULL(pages))
@@ -255,7 +255,6 @@ static int i915_gem_object_userptr_unbind(struct drm_i915_gem_object *obj, bool
 
 	if (get_pages)
 		err = ____i915_gem_object_get_pages(obj);
-	mutex_unlock(&obj->mm.lock);
 
 	return err;
 }
drivers/gpu/drm/i915/i915_debugfs.c +2 -2
index 263074c2c097..6d1482c82694 100644
@@ -1510,10 +1510,10 @@ i915_drop_caches_set(void *data, u64 val)
 
 	fs_reclaim_acquire(GFP_KERNEL);
 	if (val & DROP_BOUND)
-		i915_gem_shrink(i915, LONG_MAX, NULL, I915_SHRINK_BOUND);
+		i915_gem_shrink(NULL, i915, LONG_MAX, NULL, I915_SHRINK_BOUND);
 
 	if (val & DROP_UNBOUND)
-		i915_gem_shrink(i915, LONG_MAX, NULL, I915_SHRINK_UNBOUND);
+		i915_gem_shrink(NULL, i915, LONG_MAX, NULL, I915_SHRINK_UNBOUND);
 
 	if (val & DROP_SHRINK_ALL)
 		i915_gem_shrink_all(i915);
drivers/gpu/drm/i915/i915_gem.c +1 -7
index b81fbd907775..ef66c0926af6 100644
@@ -1063,10 +1063,6 @@ i915_gem_madvise_ioctl(struct drm_device *dev, void *data,
 	if (err)
 		goto out;
 
-	err = mutex_lock_interruptible(&obj->mm.lock);
-	if (err)
-		goto out_ww;
-
 	if (i915_gem_object_has_pages(obj) &&
 	    i915_gem_object_is_tiled(obj) &&
 	    i915->quirks & QUIRK_PIN_SWIZZLED_PAGES) {
@@ -1109,9 +1105,7 @@ i915_gem_madvise_ioctl(struct drm_device *dev, void *data,
 		i915_gem_object_truncate(obj);
 
 	args->retained = obj->mm.madv != __I915_MADV_PURGED;
-	mutex_unlock(&obj->mm.lock);
 
-out_ww:
 	i915_gem_object_unlock(obj);
 out:
 	i915_gem_object_put(obj);
@@ -1292,7 +1286,7 @@ int i915_gem_freeze_late(struct drm_i915_private *i915)
 
 	wakeref = intel_runtime_pm_get(&i915->runtime_pm);
 
-	i915_gem_shrink(i915, -1UL, NULL, ~0);
+	i915_gem_shrink(NULL, i915, -1UL, NULL, ~0);
 	i915_gem_drain_freed_objects(i915);
 
 	list_for_each_entry(obj, &i915->mm.shrink_list, mm.link) {
drivers/gpu/drm/i915/i915_gem_gtt.c +1 -1
index c5ee1567f3d1..729074ee33d4 100644
@@ -44,7 +44,7 @@ int i915_gem_gtt_prepare_pages(struct drm_i915_gem_object *obj,
 		 * the DMA remapper, i915_gem_shrink will return 0.
 		 */
 		GEM_BUG_ON(obj->mm.pages == pages);
-	} while (i915_gem_shrink(to_i915(obj->base.dev),
+	} while (i915_gem_shrink(NULL, to_i915(obj->base.dev),
 				 obj->base.size >> PAGE_SHIFT, NULL,
 				 I915_SHRINK_BOUND |
 				 I915_SHRINK_UNBOUND));
Signature · 6 lines
2.26.2

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel
#75 lore raw

RFC PATCH 71/162 drm/i915: Keep userpointer bindings if seqcount is unchanged, v2.

In reply to Matthew Auld

From: Maarten Lankhorst <[email protected]>

Instead of force unbinding and rebinding every time, we try to check if our notifier seqcount is still correct when pages are bound. This way we only rebind userptr when we need to, and prevent stalls.

---
 drivers/gpu/drm/i915/gem/i915_gem_userptr.c | 27 ++++++++++++++++++---
 1 file changed, 24 insertions(+), 3 deletions(-)
drivers/gpu/drm/i915/gem/i915_gem_userptr.c +24 -3
index fb4bc30fbd9a..d1ecc31b5e90 100644
@@ -275,12 +275,33 @@ int i915_gem_object_userptr_submit_init(struct drm_i915_gem_object *obj)
 	if (ret)
 		return ret;
 
-	/* Make sure userptr is unbound for next attempt, so we don't use stale pages. */
-	ret = i915_gem_object_userptr_unbind(obj, false);
+	/* optimistically try to preserve current pages while unlocked */
+	if (i915_gem_object_has_pages(obj) &&
+	    !mmu_interval_check_retry(&obj->userptr.notifier,
+				      obj->userptr.notifier_seq)) {
+		spin_lock(&i915->mm.notifier_lock);
+		if (obj->userptr.pvec &&
+		    !mmu_interval_read_retry(&obj->userptr.notifier,
+					     obj->userptr.notifier_seq)) {
+			obj->userptr.page_ref++;
+
+			/* We can keep using the current binding, this is the fastpath */
+			ret = 1;
+		}
+		spin_unlock(&i915->mm.notifier_lock);
+	}
+
+	if (!ret) {
+		/* Make sure userptr is unbound for next attempt, so we don't use stale pages. */
+		ret = i915_gem_object_userptr_unbind(obj, false);
+	}
 	i915_gem_object_unlock(obj);
-	if (ret)
+	if (ret < 0)
 		return ret;
 
+	if (ret > 0)
+		return 0;
+
 	notifier_seq = mmu_interval_read_begin(&obj->userptr.notifier);
 
 	pvec = kvmalloc_array(num_pages, sizeof(struct page *), GFP_KERNEL);
Signature · 6 lines
2.26.2

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
#76 lore raw

RFC PATCH 72/162 drm/i915: Avoid some false positives in assert_object_held()

In reply to Matthew Auld

From: Thomas Hellström <[email protected]>

In a ww transaction where we've already locked a reservation object, assert_object_held() might not throw a splat even if the object is unlocked. Improve on that situation by asserting that the reservation object's ww mutex is indeed locked.

---
 drivers/gpu/drm/i915/gem/i915_gem_object.h | 9 ++++++++-
 1 file changed, 8 insertions(+), 1 deletion(-)
drivers/gpu/drm/i915/gem/i915_gem_object.h +8 -1
index d0cc62d1c65e..d56643b3b518 100644
@@ -117,7 +117,14 @@ i915_gem_object_put(struct drm_i915_gem_object *obj)
 	__drm_gem_object_put(&obj->base);
 }
 
-#define assert_object_held(obj) dma_resv_assert_held((obj)->base.resv)
+#ifdef CONFIG_LOCKDEP
+#define assert_object_held(obj) do {					\
+		dma_resv_assert_held((obj)->base.resv);			\
+		WARN_ON(!ww_mutex_is_locked(&(obj)->base.resv->lock)); \
+	} while (0)
+#else
+#define assert_object_held(obj) do { } while (0)
+#endif
 
 #define object_is_isolated(obj)					\
 	(!IS_ENABLED(CONFIG_LOCKDEP) ||				\
Signature · 6 lines
2.26.2

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
#77 lore raw

RFC PATCH 73/162 drm/i915: Reference contending lock objects

In reply to Matthew Auld

From: Thomas Hellström <[email protected]>

When we lock objects in leaf functions, for example during eviction, they may disappear as soon as we unreference them, and the locking context contended pointer then points to a free object.
Fix this by taking a reference on that object, and also unlock the contending object as soon as we've done the ww transaction relaxation:
The restarted transaction may not even need the contending object, and keeping the lock is not needed to prevent starvation. Keeping that lock will unnecessarily requiring us to reference count all locks on the list and also creates locking confusion around -EALREADY.

---
 drivers/gpu/drm/i915/gem/i915_gem_object.h | 2 +-
 drivers/gpu/drm/i915/i915_gem.c            | 9 ++++++++-
 2 files changed, 9 insertions(+), 2 deletions(-)
drivers/gpu/drm/i915/gem/i915_gem_object.h +1 -1
index d56643b3b518..60e27738c39d 100644
@@ -163,7 +163,7 @@ static inline int __i915_gem_object_lock(struct drm_i915_gem_object *obj,
 		ret = 0;
 
 	if (ret == -EDEADLK)
-		ww->contended = obj;
+		ww->contended = i915_gem_object_get(obj);
 
 	return ret;
 }
drivers/gpu/drm/i915/i915_gem.c +8 -1
index ef66c0926af6..2248e65cf5f9 100644
@@ -1370,9 +1370,16 @@ int __must_check i915_gem_ww_ctx_backoff(struct i915_gem_ww_ctx *ww)
 	else
 		dma_resv_lock_slow(ww->contended->base.resv, &ww->ctx);
 
+	/*
+	 * Unlocking the contended lock again, as might not need it in
+	 * the retried transaction. This does not increase starvation,
+	 * but it's opening up for a wakeup flood if there are many
+	 * transactions relaxing on this object.
+	 */
 	if (!ret)
-		list_add_tail(&ww->contended->obj_link, &ww->obj_list);
+		dma_resv_unlock(ww->contended->base.resv);
 
+	i915_gem_object_put(ww->contended);
 	ww->contended = NULL;
 
 	return ret;
Signature · 6 lines
2.26.2

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
#79 lore raw

RFC PATCH 75/162 drm/i915: Introduce a for_i915_gem_ww(){}

In reply to Matthew Auld

From: Thomas Hellström <[email protected]>

Introduce a for_i915_gem_ww(){} utility to help make the code around a ww transaction more readable.

---
 drivers/gpu/drm/i915/i915_gem_ww.h | 31 +++++++++++++++++++++++++++++-
 1 file changed, 30 insertions(+), 1 deletion(-)
drivers/gpu/drm/i915/i915_gem_ww.h +30 -1
index f2d8769e4118..f6b1a796667b 100644
@@ -11,11 +11,40 @@ struct i915_gem_ww_ctx {
 	struct ww_acquire_ctx ctx;
 	struct list_head obj_list;
 	struct drm_i915_gem_object *contended;
-	bool intr;
+	unsigned short intr;
+	unsigned short loop;
 };
 
 void i915_gem_ww_ctx_init(struct i915_gem_ww_ctx *ctx, bool intr);
 void i915_gem_ww_ctx_fini(struct i915_gem_ww_ctx *ctx);
 int __must_check i915_gem_ww_ctx_backoff(struct i915_gem_ww_ctx *ctx);
 void i915_gem_ww_unlock_single(struct drm_i915_gem_object *obj);
+
+/* Internal functions used by the inlines! Don't use. */
+static inline int __i915_gem_ww_fini(struct i915_gem_ww_ctx *ww, int err)
+{
+	ww->loop = 0;
+	if (err == -EDEADLK) {
+		err = i915_gem_ww_ctx_backoff(ww);
+		if (!err)
+			ww->loop = 1;
+	}
+
+	if (!ww->loop)
+		i915_gem_ww_ctx_fini(ww);
+
+	return err;
+}
+
+static inline void
+__i915_gem_ww_init(struct i915_gem_ww_ctx *ww, bool intr)
+{
+	i915_gem_ww_ctx_init(ww, intr);
+	ww->loop = 1;
+}
+
+#define for_i915_gem_ww(_ww, _err, _intr)			\
+	for (__i915_gem_ww_init(_ww, _intr); (_ww)->loop;	\
+	     _err = __i915_gem_ww_fini(_ww, _err))
+
 #endif
Signature · 6 lines
2.26.2

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
#80 lore raw

RFC PATCH 76/162 drm/i915: Untangle the vma pages_mutex

In reply to Matthew Auld

From: Thomas Hellström <[email protected]>

Move the vma pages_mutex out of the way from the object ww locks.

---
 drivers/gpu/drm/i915/i915_vma.c | 30 ++++++++++++++++--------------
 1 file changed, 16 insertions(+), 14 deletions(-)
drivers/gpu/drm/i915/i915_vma.c +16 -14
index 0c7e4191811a..7243ab593aec 100644
@@ -792,28 +792,30 @@ static int vma_get_pages(struct i915_vma *vma)
 	if (atomic_add_unless(&vma->pages_count, 1, 0))
 		return 0;
 
+	if (vma->obj) {
+		err = i915_gem_object_pin_pages(vma->obj);
+		if (err)
+			return err;
+	}
+
 	/* Allocations ahoy! */
-	if (mutex_lock_interruptible(&vma->pages_mutex))
-		return -EINTR;
+	if (mutex_lock_interruptible(&vma->pages_mutex)) {
+		err = -EINTR;
+		goto unpin;
+	}
 
 	if (!atomic_read(&vma->pages_count)) {
-		if (vma->obj) {
-			err = i915_gem_object_pin_pages(vma->obj);
-			if (err)
-				goto unlock;
-		}
-
 		err = vma->ops->set_pages(vma);
-		if (err) {
-			if (vma->obj)
-				i915_gem_object_unpin_pages(vma->obj);
+		if (err)
 			goto unlock;
-		}
 	}
 	atomic_inc(&vma->pages_count);
 
 unlock:
 	mutex_unlock(&vma->pages_mutex);
+unpin:
+	if (err && vma->obj)
+		__i915_gem_object_unpin_pages(vma->obj);
 
 	return err;
 }
@@ -826,10 +828,10 @@ static void __vma_put_pages(struct i915_vma *vma, unsigned int count)
 	if (atomic_sub_return(count, &vma->pages_count) == 0) {
 		vma->ops->clear_pages(vma);
 		GEM_BUG_ON(vma->pages);
-		if (vma->obj)
-			i915_gem_object_unpin_pages(vma->obj);
 	}
 	mutex_unlock(&vma->pages_mutex);
+	if (vma->obj)
+		i915_gem_object_unpin_pages(vma->obj);
 }
 
 static void vma_put_pages(struct i915_vma *vma)
Signature · 6 lines
2.26.2

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
#81 lore raw

RFC PATCH 77/162 drm/i915/fbdev: Use lmem physical addresses for fb_mmap() on discrete

In reply to Matthew Auld

From: Mohammed Khajapasha <[email protected]>

use local memory io BAR address for fbdev's fb_mmap() operation on discrete, fbdev uses the physical address of our framebuffer for its fb_mmap() fn.

---
 drivers/gpu/drm/i915/display/intel_fbdev.c | 27 +++++++++++++++++-----
 1 file changed, 21 insertions(+), 6 deletions(-)
drivers/gpu/drm/i915/display/intel_fbdev.c +21 -6
index bdf44e923cc0..831e99e0785c 100644
@@ -178,6 +178,7 @@ static int intelfb_create(struct drm_fb_helper *helper,
 	unsigned long flags = 0;
 	bool prealloc = false;
 	void __iomem *vaddr;
+	struct drm_i915_gem_object *obj;
 	int ret;
 
 	if (intel_fb &&
@@ -232,13 +233,27 @@ static int intelfb_create(struct drm_fb_helper *helper,
 	info->fbops = &intelfb_ops;
 
 	/* setup aperture base/size for vesafb takeover */
-	info->apertures->ranges[0].base = ggtt->gmadr.start;
-	info->apertures->ranges[0].size = ggtt->mappable_end;
+	obj = intel_fb_obj(&intel_fb->base);
+	if (HAS_LMEM(dev_priv) && i915_gem_object_is_lmem(obj)) {
+		struct intel_memory_region *mem = obj->mm.region;
+
+		info->apertures->ranges[0].base = mem->io_start;
+		info->apertures->ranges[0].size = mem->total;
+
+		/* Use fbdev's framebuffer from lmem for discrete */
+		info->fix.smem_start =
+			(unsigned long)(mem->io_start +
+					i915_gem_object_get_dma_address(obj, 0));
+		info->fix.smem_len = obj->base.size;
+	} else {
+		info->apertures->ranges[0].base = ggtt->gmadr.start;
+		info->apertures->ranges[0].size = ggtt->mappable_end;
 
-	/* Our framebuffer is the entirety of fbdev's system memory */
-	info->fix.smem_start =
-		(unsigned long)(ggtt->gmadr.start + vma->node.start);
-	info->fix.smem_len = vma->node.size;
+		/* Our framebuffer is the entirety of fbdev's system memory */
+		info->fix.smem_start =
+			(unsigned long)(ggtt->gmadr.start + vma->node.start);
+		info->fix.smem_len = vma->node.size;
+	}
 
 	vaddr = i915_vma_pin_iomap(vma);
 	if (IS_ERR(vaddr)) {
Signature · 6 lines
2.26.2

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
#78 lore raw

RFC PATCH 74/162 drm/i915: Break out dma_resv ww locking utilities to separate files

In reply to Matthew Auld

From: Thomas Hellström <[email protected]>

As we're about to add more ww-related functionality, break out the dma_resv ww locking utilities to their own files

---
 drivers/gpu/drm/i915/Makefile               |  1 +
 drivers/gpu/drm/i915/gem/i915_gem_object.h  |  1 +
 drivers/gpu/drm/i915/gt/intel_renderstate.h |  1 +
 drivers/gpu/drm/i915/i915_gem.c             | 59 ------------------
 drivers/gpu/drm/i915/i915_gem.h             | 12 ----
 drivers/gpu/drm/i915/i915_gem_ww.c          | 66 +++++++++++++++++++++
 drivers/gpu/drm/i915/i915_gem_ww.h          | 21 +++++++
 7 files changed, 90 insertions(+), 71 deletions(-)
 create mode 100644 drivers/gpu/drm/i915/i915_gem_ww.c
 create mode 100644 drivers/gpu/drm/i915/i915_gem_ww.h
drivers/gpu/drm/i915/Makefile +1 -0
index 5112e5d79316..ec361d61230b 100644
@@ -45,6 +45,7 @@ i915-y += i915_drv.o \
 	  i915_switcheroo.o \
 	  i915_sysfs.o \
 	  i915_utils.o \
+	  i915_gem_ww.o \
 	  intel_device_info.o \
 	  intel_dram.o \
 	  intel_memory_region.o \
drivers/gpu/drm/i915/gem/i915_gem_object.h +1 -0
index 60e27738c39d..c6c7ab181a65 100644
@@ -15,6 +15,7 @@
 #include "i915_gem_object_types.h"
 #include "i915_gem_gtt.h"
 #include "i915_vma_types.h"
+#include "i915_gem_ww.h"
 
 void i915_gem_init__objects(struct drm_i915_private *i915);
 
drivers/gpu/drm/i915/gt/intel_renderstate.h +1 -0
index 713aa1e86c80..d9db833b873b 100644
@@ -26,6 +26,7 @@
 
 #include <linux/types.h>
 #include "i915_gem.h"
+#include "i915_gem_ww.h"
 
 struct i915_request;
 struct intel_context;
drivers/gpu/drm/i915/i915_gem.c +0 -59

65 lines not on this page · raw message

drivers/gpu/drm/i915/i915_gem.h +0 -12
index a4cad3f154ca..f333e88a2b6e 100644
@@ -116,16 +116,4 @@ static inline bool __tasklet_is_scheduled(struct tasklet_struct *t)
 	return test_bit(TASKLET_STATE_SCHED, &t->state);
 }
 
-struct i915_gem_ww_ctx {
-	struct ww_acquire_ctx ctx;
-	struct list_head obj_list;
-	bool intr;
-	struct drm_i915_gem_object *contended;
-};
-
-void i915_gem_ww_ctx_init(struct i915_gem_ww_ctx *ctx, bool intr);
-void i915_gem_ww_ctx_fini(struct i915_gem_ww_ctx *ctx);
-int __must_check i915_gem_ww_ctx_backoff(struct i915_gem_ww_ctx *ctx);
-void i915_gem_ww_unlock_single(struct drm_i915_gem_object *obj);
-
 #endif /* __I915_GEM_H__ */
drivers/gpu/drm/i915/i915_gem_ww.c new file +66 -0

66 lines not on this page · raw message

drivers/gpu/drm/i915/i915_gem_ww.h new file +21 -0
new file mode 100644
index 000000000000..f2d8769e4118
@@ -0,0 +1,21 @@
+/* SPDX-License-Identifier: MIT */
+/*
+ * Copyright © 2020 Intel Corporation
+ */
+#ifndef __I915_GEM_WW_H__
+#define __I915_GEM_WW_H__
+
+#include <drm/drm_drv.h>
+
+struct i915_gem_ww_ctx {
+	struct ww_acquire_ctx ctx;
+	struct list_head obj_list;
+	struct drm_i915_gem_object *contended;
+	bool intr;
+};
+
+void i915_gem_ww_ctx_init(struct i915_gem_ww_ctx *ctx, bool intr);
+void i915_gem_ww_ctx_fini(struct i915_gem_ww_ctx *ctx);
+int __must_check i915_gem_ww_ctx_backoff(struct i915_gem_ww_ctx *ctx);
+void i915_gem_ww_unlock_single(struct drm_i915_gem_object *obj);
+#endif
Signature · 6 lines
2.26.2

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
#83 lore raw

RFC PATCH 79/162 drm/i915/dmabuf: Disallow LMEM objects from dma-buf

In reply to Matthew Auld

From: "Michael J. Ruhl" <[email protected]>

The dma-buf interface for i915 does not currently support LMEM backed objects.

Check imported objects to see if they are from i915 and if they are LMEM. If they are, reject the import.

This check is needed in two places, once on import, and then a recheck in the mapping path in the off chance that an object was migrated to LMEM after import.

---
 drivers/gpu/drm/i915/gem/i915_gem_dmabuf.c | 10 ++++++++++
 1 file changed, 10 insertions(+)
drivers/gpu/drm/i915/gem/i915_gem_dmabuf.c +10 -0
index c4b01e819786..018d02cc4af5 100644
@@ -9,6 +9,7 @@
 #include <linux/dma-resv.h>
 
 #include "i915_drv.h"
+#include "i915_gem_lmem.h"
 #include "i915_gem_object.h"
 #include "i915_scatterlist.h"
 
@@ -25,6 +26,11 @@ static struct sg_table *i915_gem_map_dma_buf(struct dma_buf_attachment *attachme
 	struct scatterlist *src, *dst;
 	int ret, i;
 
+	if (i915_gem_object_is_lmem(obj)) {
+		ret = -ENOTSUPP;
+		goto err;
+	}
+
 	ret = i915_gem_object_pin_pages(obj);
 	if (ret)
 		goto err;
@@ -248,6 +254,10 @@ struct drm_gem_object *i915_gem_prime_import(struct drm_device *dev,
 			 */
 			return &i915_gem_object_get(obj)->base;
 		}
+
+		/* not our device, but still a i915 object? */
+		if (i915_gem_object_is_lmem(obj))
+			return ERR_PTR(-ENOTSUPP);
 	}
 
 	/* need to attach */
Signature · 6 lines
2.26.2

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
#84 lore raw

RFC PATCH 80/162 drm/i915/lmem: Fail driver init if LMEM training failed

In reply to Matthew Auld

From: Matt Roper <[email protected]>

Boot firmware performs memory training and health assessment during startup. If the memory training fails, the firmware will consider the GPU unusable and will instruct the punit to keep the GT powered down. If this happens, our driver will be unable to communicate with the GT (all GT registers will read back as 0, forcewake requests will timeout, etc.) so we should abort driver initialization if this happens. We can confirm that LMEM was initialized successfully via sgunit register GU_CNTL.

Bspec: 53111

---
 drivers/gpu/drm/i915/i915_reg.h     |  3 +++
 drivers/gpu/drm/i915/intel_uncore.c | 12 ++++++++++++
 2 files changed, 15 insertions(+)
drivers/gpu/drm/i915/i915_reg.h +3 -0
index 5375b219cc3b..bf9ba1e361bb 100644
@@ -487,6 +487,9 @@ static inline bool i915_mmio_reg_valid(i915_reg_t reg)
 #define GAB_CTL				_MMIO(0x24000)
 #define   GAB_CTL_CONT_AFTER_PAGEFAULT	(1 << 8)
 
+#define GU_CNTL				_MMIO(0x101010)
+#define   LMEM_INIT			REG_BIT(7)
+
 #define GEN6_STOLEN_RESERVED		_MMIO(0x1082C0)
 #define GEN6_STOLEN_RESERVED_ADDR_MASK	(0xFFF << 20)
 #define GEN7_STOLEN_RESERVED_ADDR_MASK	(0x3FFF << 18)
drivers/gpu/drm/i915/intel_uncore.c +12 -0
index 1c14a07eba7d..1630452e82b8 100644
@@ -1901,6 +1901,18 @@ int intel_uncore_init_mmio(struct intel_uncore *uncore)
 	if (ret)
 		return ret;
 
+	/*
+	 * The boot firmware initializes local memory and assesses its health.
+	 * If memory training fails, the punit will have been instructed to
+	 * keep the GT powered down; we won't be able to communicate with it
+	 * and we should not continue with driver initialization.
+	 */
+	if (IS_DGFX(i915) &&
+	    !(__raw_uncore_read32(uncore, GU_CNTL) & LMEM_INIT)) {
+		drm_err(&i915->drm, "LMEM not initialized by firmware\n");
+		return -ENODEV;
+	}
+
 	if (INTEL_GEN(i915) > 5 && !intel_vgpu_active(i915))
 		uncore->flags |= UNCORE_HAS_FORCEWAKE;
 
Signature · 6 lines
2.26.2

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
#82 lore raw

RFC PATCH 78/162 drm/i915: Return error value when bo not in LMEM for discrete

In reply to Matthew Auld

From: Mohammed Khajapasha <[email protected]>

Return EREMOTE value when frame buffer object is not backed by LMEM for discrete. If Local memory is supported by hardware the framebuffer backing gem objects should be from local memory.

Cc Michael J. Ruhl <[email protected]>
Cc Animesh Manna <[email protected]>
---
 drivers/gpu/drm/i915/display/intel_display.c | 9 +++++++++
 1 file changed, 9 insertions(+)
drivers/gpu/drm/i915/display/intel_display.c +9 -0
index 8a7945f55278..95ed1e06ea55 100644
@@ -18054,11 +18054,20 @@ intel_user_framebuffer_create(struct drm_device *dev,
 	struct drm_framebuffer *fb;
 	struct drm_i915_gem_object *obj;
 	struct drm_mode_fb_cmd2 mode_cmd = *user_mode_cmd;
+	struct drm_i915_private *i915;
 
 	obj = i915_gem_object_lookup(filp, mode_cmd.handles[0]);
 	if (!obj)
 		return ERR_PTR(-ENOENT);
 
+	/* object is backed with LMEM for discrete */
+	i915 = to_i915(obj->base.dev);
+	if (HAS_LMEM(i915) && !i915_gem_object_is_lmem(obj)) {
+		/* object is "remote", not in local memory */
+		i915_gem_object_put(obj);
+		return ERR_PTR(-EREMOTE);
+	}
+
 	fb = intel_framebuffer_create(obj, &mode_cmd);
 	i915_gem_object_put(obj);
 
Signature · 6 lines
2.26.2

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel
#85 lore raw

RFC PATCH 81/162 HAX drm/i915/lmem: support CPU relocations

In reply to Matthew Auld

** DO NOT MERGE. RELOCATION SUPPORT WILL BE DROPPED FROM DG1+ **

Add LMEM support for the CPU reloc path. When doing relocations we have both a GPU and CPU reloc path, as well as some debugging options to force a particular path. The GPU reloc path is preferred when the object is not currently idle, otherwise we use the CPU reloc path. Since we can't kmap the object, and the mappable aperture might not be available, add support for mapping it through LMEMBAR.

Cc Joonas Lahtinen <[email protected]>
Cc Abdiel Janulgue <[email protected]>
Cc Rodrigo Vivi <[email protected]>
---
 .../gpu/drm/i915/gem/i915_gem_execbuffer.c    | 53 +++++++++++++++++--
 drivers/gpu/drm/i915/gem/i915_gem_lmem.c      | 12 +++++
 drivers/gpu/drm/i915/gem/i915_gem_lmem.h      |  4 ++
 3 files changed, 65 insertions(+), 4 deletions(-)
drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c +49 -4
index 91f0c3fd9a4b..e73a761a7d1f 100644
@@ -14,6 +14,7 @@
 #include "display/intel_frontbuffer.h"
 
 #include "gem/i915_gem_ioctls.h"
+#include "gem/i915_gem_lmem.h"
 #include "gt/intel_context.h"
 #include "gt/intel_gt.h"
 #include "gt/intel_gt_buffer_pool.h"
@@ -278,6 +279,7 @@ struct i915_execbuffer {
 		bool has_llc : 1;
 		bool has_fence : 1;
 		bool needs_unfenced : 1;
+		bool is_lmem : 1;
 
 		struct i915_request *rq;
 		u32 *rq_cmd;
@@ -1049,6 +1051,7 @@ static void reloc_cache_init(struct reloc_cache *cache,
 	cache->has_fence = cache->gen < 4;
 	cache->needs_unfenced = INTEL_INFO(i915)->unfenced_needs_alignment;
 	cache->node.flags = 0;
+	cache->is_lmem = false;
 	reloc_cache_clear(cache);
 }
 
@@ -1128,10 +1131,14 @@ static void reloc_cache_reset(struct reloc_cache *cache, struct i915_execbuffer
 	} else {
 		struct i915_ggtt *ggtt = cache_to_ggtt(cache);
 
-		intel_gt_flush_ggtt_writes(ggtt->vm.gt);
+		if (!cache->is_lmem)
+			intel_gt_flush_ggtt_writes(ggtt->vm.gt);
 		io_mapping_unmap_atomic((void __iomem *)vaddr);
 
-		if (drm_mm_node_allocated(&cache->node)) {
+		if (cache->is_lmem) {
+			i915_gem_object_unpin_pages((struct drm_i915_gem_object *)cache->node.mm);
+			cache->is_lmem = false;
+		} else if (drm_mm_node_allocated(&cache->node)) {
 			ggtt->vm.clear_range(&ggtt->vm,
 					     cache->node.start,
 					     cache->node.size);
@@ -1184,6 +1191,40 @@ static void *reloc_kmap(struct drm_i915_gem_object *obj,
 	return vaddr;
 }
 
+static void *reloc_lmem(struct drm_i915_gem_object *obj,
+			struct reloc_cache *cache,
+			unsigned long page)
+{
+	void *vaddr;
+	int err;
+
+	GEM_BUG_ON(use_cpu_reloc(cache, obj));
+
+	if (cache->vaddr) {
+		io_mapping_unmap_atomic((void __force __iomem *) unmask_page(cache->vaddr));
+	} else {
+		err = i915_gem_object_pin_pages(obj);
+		if (err)
+			return ERR_PTR(err);
+
+		err = i915_gem_object_set_to_wc_domain(obj, true);
+		if (err) {
+			i915_gem_object_unpin_pages(obj);
+			return ERR_PTR(err);
+		}
+
+		cache->node.mm = (void *)obj;
+		cache->is_lmem = true;
+	}
+
+	vaddr = i915_gem_object_lmem_io_map_page_atomic(obj, page);
+
+	cache->vaddr = (unsigned long)vaddr;
+	cache->page = page;
+
+	return vaddr;
+}
+
 static void *reloc_iomap(struct drm_i915_gem_object *obj,
 			 struct i915_execbuffer *eb,
 			 unsigned long page)
@@ -1262,8 +1303,12 @@ static void *reloc_vaddr(struct drm_i915_gem_object *obj,
 		vaddr = unmask_page(cache->vaddr);
 	} else {
 		vaddr = NULL;
-		if ((cache->vaddr & KMAP) == 0)
-			vaddr = reloc_iomap(obj, eb, page);
+		if ((cache->vaddr & KMAP) == 0) {
+			if (i915_gem_object_is_lmem(obj))
+				vaddr = reloc_lmem(obj, cache, page);
+			else
+				vaddr = reloc_iomap(obj, eb, page);
+		}
 		if (!vaddr)
 			vaddr = reloc_kmap(obj, cache, page);
 	}
drivers/gpu/drm/i915/gem/i915_gem_lmem.c +12 -0
index e953965f8263..f6c4d5998ff9 100644
@@ -17,6 +17,18 @@ const struct drm_i915_gem_object_ops i915_gem_lmem_obj_ops = {
 	.release = i915_gem_object_release_memory_region,
 };
 
+void __iomem *
+i915_gem_object_lmem_io_map_page_atomic(struct drm_i915_gem_object *obj,
+					unsigned long n)
+{
+	resource_size_t offset;
+
+	offset = i915_gem_object_get_dma_address(obj, n);
+	offset -= obj->mm.region->region.start;
+
+	return io_mapping_map_atomic_wc(&obj->mm.region->iomap, offset);
+}
+
 bool i915_gem_object_is_lmem(struct drm_i915_gem_object *obj)
 {
 	return obj->ops == &i915_gem_lmem_obj_ops;
drivers/gpu/drm/i915/gem/i915_gem_lmem.h +4 -0
index fc3f15580fe3..bf7e11fad17b 100644
@@ -14,6 +14,10 @@ struct intel_memory_region;
 
 extern const struct drm_i915_gem_object_ops i915_gem_lmem_obj_ops;
 
+void __iomem *
+i915_gem_object_lmem_io_map_page_atomic(struct drm_i915_gem_object *obj,
+					unsigned long n);
+
 bool i915_gem_object_is_lmem(struct drm_i915_gem_object *obj);
 
 struct drm_i915_gem_object *
Signature · 6 lines
2.26.2

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
#86 lore raw

RFC PATCH 82/162 HAX drm/i915/lmem: support pread and pwrite

In reply to Matthew Auld

** DO NOT MERGE. PREAD/WRITE SUPPORT WILL BE DROPPED FROM DG1+ **

We need to add support for pread'ing and pwriting an LMEM object.

---
 drivers/gpu/drm/i915/gem/i915_gem_lmem.c | 186 +++++++++++++++++++++++
 drivers/gpu/drm/i915/gem/i915_gem_lmem.h |   2 +
 2 files changed, 188 insertions(+)
drivers/gpu/drm/i915/gem/i915_gem_lmem.c +186 -0
index f6c4d5998ff9..840b68eb10d3 100644
@@ -8,6 +8,177 @@
 #include "gem/i915_gem_lmem.h"
 #include "i915_drv.h"
 
+static int
+i915_ww_pin_lock_interruptible(struct drm_i915_gem_object *obj)
+{
+	struct i915_gem_ww_ctx ww;
+	int ret;
+
+	for_i915_gem_ww(&ww, ret, true) {
+		ret = i915_gem_object_lock(obj, &ww);
+		if (ret)
+			continue;
+
+		ret = i915_gem_object_pin_pages(obj);
+		if (ret)
+			continue;
+
+		ret = i915_gem_object_set_to_wc_domain(obj, false);
+		if (ret)
+			goto out_unpin;
+
+		ret = i915_gem_object_wait(obj,
+					   I915_WAIT_INTERRUPTIBLE,
+					   MAX_SCHEDULE_TIMEOUT);
+		if (!ret)
+			continue;
+
+out_unpin:
+		i915_gem_object_unpin_pages(obj);
+
+		/* Unlocking is done implicitly */
+	}
+
+	return ret;
+}
+
+int i915_gem_object_lmem_pread(struct drm_i915_gem_object *obj,
+			       const struct drm_i915_gem_pread *arg)
+{
+	struct drm_i915_private *i915 = to_i915(obj->base.dev);
+	struct intel_runtime_pm *rpm = &i915->runtime_pm;
+	intel_wakeref_t wakeref;
+	char __user *user_data;
+	unsigned int offset;
+	unsigned long idx;
+	u64 remain;
+	int ret;
+
+	ret = i915_gem_object_wait(obj,
+				   I915_WAIT_INTERRUPTIBLE,
+				   MAX_SCHEDULE_TIMEOUT);
+	if (ret)
+		return ret;
+
+	ret = i915_ww_pin_lock_interruptible(obj);
+	if (ret)
+		return ret;
+
+	wakeref = intel_runtime_pm_get(rpm);
+
+	remain = arg->size;
+	user_data = u64_to_user_ptr(arg->data_ptr);
+	offset = offset_in_page(arg->offset);
+	for (idx = arg->offset >> PAGE_SHIFT; remain; idx++) {
+		unsigned long unwritten;
+		void __iomem *vaddr;
+		int length;
+
+		length = remain;
+		if (offset + length > PAGE_SIZE)
+			length = PAGE_SIZE - offset;
+
+		vaddr = i915_gem_object_lmem_io_map_page_atomic(obj, idx);
+		if (!vaddr) {
+			ret = -ENOMEM;
+			goto out_put;
+		}
+		unwritten = __copy_to_user_inatomic(user_data,
+						    (void __force *)vaddr + offset,
+						    length);
+		io_mapping_unmap_atomic(vaddr);
+		if (unwritten) {
+			vaddr = i915_gem_object_lmem_io_map_page(obj, idx);
+			unwritten = copy_to_user(user_data,
+						 (void __force *)vaddr + offset,
+						 length);
+			io_mapping_unmap(vaddr);
+		}
+		if (unwritten) {
+			ret = -EFAULT;
+			goto out_put;
+		}
+
+		remain -= length;
+		user_data += length;
+		offset = 0;
+	}
+
+out_put:
+	intel_runtime_pm_put(rpm, wakeref);
+	i915_gem_object_unpin_pages(obj);
+
+	return ret;
+}
+
+static int i915_gem_object_lmem_pwrite(struct drm_i915_gem_object *obj,
+				       const struct drm_i915_gem_pwrite *arg)
+{
+	struct drm_i915_private *i915 = to_i915(obj->base.dev);
+	struct intel_runtime_pm *rpm = &i915->runtime_pm;
+	intel_wakeref_t wakeref;
+	char __user *user_data;
+	unsigned int offset;
+	unsigned long idx;
+	u64 remain;
+	int ret;
+
+	ret = i915_gem_object_wait(obj,
+				   I915_WAIT_INTERRUPTIBLE,
+				   MAX_SCHEDULE_TIMEOUT);
+	if (ret)
+		return ret;
+
+	ret = i915_ww_pin_lock_interruptible(obj);
+	if (ret)
+		return ret;
+
+	wakeref = intel_runtime_pm_get(rpm);
+
+	remain = arg->size;
+	user_data = u64_to_user_ptr(arg->data_ptr);
+	offset = offset_in_page(arg->offset);
+	for (idx = arg->offset >> PAGE_SHIFT; remain; idx++) {
+		unsigned long unwritten;
+		void __iomem *vaddr;
+		int length;
+
+		length = remain;
+		if (offset + length > PAGE_SIZE)
+			length = PAGE_SIZE - offset;
+
+		vaddr = i915_gem_object_lmem_io_map_page_atomic(obj, idx);
+		if (!vaddr) {
+			ret = -ENOMEM;
+			goto out_put;
+		}
+
+		unwritten = __copy_from_user_inatomic_nocache((void __force *)vaddr + offset,
+							      user_data, length);
+		io_mapping_unmap_atomic(vaddr);
+		if (unwritten) {
+			vaddr = i915_gem_object_lmem_io_map_page(obj, idx);
+			unwritten = copy_from_user((void __force *)vaddr + offset,
+						   user_data, length);
+			io_mapping_unmap(vaddr);
+		}
+		if (unwritten) {
+			ret = -EFAULT;
+			goto out_put;
+		}
+
+		remain -= length;
+		user_data += length;
+		offset = 0;
+	}
+
+out_put:
+	intel_runtime_pm_put(rpm, wakeref);
+	i915_gem_object_unpin_pages(obj);
+
+	return ret;
+}
+
 const struct drm_i915_gem_object_ops i915_gem_lmem_obj_ops = {
 	.name = "i915_gem_object_lmem",
 	.flags = I915_GEM_OBJECT_HAS_IOMEM,
@@ -15,8 +186,23 @@ const struct drm_i915_gem_object_ops i915_gem_lmem_obj_ops = {
 	.get_pages = i915_gem_object_get_pages_buddy,
 	.put_pages = i915_gem_object_put_pages_buddy,
 	.release = i915_gem_object_release_memory_region,
+
+	.pread = i915_gem_object_lmem_pread,
+	.pwrite = i915_gem_object_lmem_pwrite,
 };
 
+void __iomem *
+i915_gem_object_lmem_io_map_page(struct drm_i915_gem_object *obj,
+				 unsigned long n)
+{
+	resource_size_t offset;
+
+	offset = i915_gem_object_get_dma_address(obj, n);
+	offset -= obj->mm.region->region.start;
+
+	return io_mapping_map_wc(&obj->mm.region->iomap, offset, PAGE_SIZE);
+}
+
 void __iomem *
 i915_gem_object_lmem_io_map_page_atomic(struct drm_i915_gem_object *obj,
 					unsigned long n)
drivers/gpu/drm/i915/gem/i915_gem_lmem.h +2 -0
index bf7e11fad17b..a24d94bc380f 100644
@@ -14,6 +14,8 @@ struct intel_memory_region;
 
 extern const struct drm_i915_gem_object_ops i915_gem_lmem_obj_ops;
 
+void __iomem *i915_gem_object_lmem_io_map_page(struct drm_i915_gem_object *obj,
+					       unsigned long n);
 void __iomem *
 i915_gem_object_lmem_io_map_page_atomic(struct drm_i915_gem_object *obj,
 					unsigned long n);
Signature · 6 lines
2.26.2

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
#87 lore raw

RFC PATCH 83/162 drm/i915: Update the helper to set correct mapping

In reply to Matthew Auld

From: Venkata Sandeep Dhanalakota <[email protected]>

Determine the possible coherent map type based on object location, and if target has llc or if user requires an always coherent mapping.

---
 drivers/gpu/drm/i915/gt/intel_engine_cs.c            |  3 ++-
 drivers/gpu/drm/i915/gt/intel_engine_pm.c            |  2 +-
 drivers/gpu/drm/i915/gt/intel_execlists_submission.c |  4 ++--
 drivers/gpu/drm/i915/gt/intel_ring.c                 |  9 ++++++---
 drivers/gpu/drm/i915/gt/intel_timeline.c             |  8 ++++++--
 drivers/gpu/drm/i915/gt/selftest_context.c           |  3 ++-
 drivers/gpu/drm/i915/gt/selftest_execlists.c         |  3 ++-
 drivers/gpu/drm/i915/gt/selftest_hangcheck.c         |  4 ++--
 drivers/gpu/drm/i915/gt/uc/intel_guc.c               |  4 +++-
 drivers/gpu/drm/i915/gt/uc/intel_huc.c               |  4 +++-
 drivers/gpu/drm/i915/i915_drv.h                      | 11 +++++++++--
 drivers/gpu/drm/i915/selftests/igt_spinner.c         |  4 ++--
 12 files changed, 40 insertions(+), 19 deletions(-)
drivers/gpu/drm/i915/gt/intel_engine_cs.c +2 -1
index 420c6a35f3ed..677c97ded81d 100644
@@ -680,7 +680,8 @@ static int init_status_page(struct intel_engine_cs *engine)
 	if (ret)
 		goto err;
 
-	vaddr = i915_gem_object_pin_map(obj, I915_MAP_WB);
+	vaddr = i915_gem_object_pin_map(obj,
+					i915_coherent_map_type(engine->i915, obj, true));
 	if (IS_ERR(vaddr)) {
 		ret = PTR_ERR(vaddr);
 		goto err_unpin;
drivers/gpu/drm/i915/gt/intel_engine_pm.c +1 -1
index 5d51144ef074..1b2009b4dcb7 100644
@@ -24,7 +24,7 @@ static void dbg_poison_ce(struct intel_context *ce)
 
 	if (ce->state) {
 		struct drm_i915_gem_object *obj = ce->state->obj;
-		int type = i915_coherent_map_type(ce->engine->i915);
+		int type = i915_coherent_map_type(ce->engine->i915, obj, true);
 		void *map;
 
 		if (!i915_gem_object_trylock(ce->state->obj))
drivers/gpu/drm/i915/gt/intel_execlists_submission.c +2 -2
index 7eec42b27bc1..582a9044727e 100644
@@ -3535,8 +3535,8 @@ __execlists_context_pre_pin(struct intel_context *ce,
 	GEM_BUG_ON(!i915_vma_is_pinned(ce->state));
 
 	*vaddr = i915_gem_object_pin_map(ce->state->obj,
-					i915_coherent_map_type(ce->engine->i915) |
-					I915_MAP_OVERRIDE);
+					 i915_coherent_map_type(ce->engine->i915, ce->state->obj, false) |
+					 I915_MAP_OVERRIDE);
 	if (IS_ERR(*vaddr))
 		return PTR_ERR(*vaddr);
 
drivers/gpu/drm/i915/gt/intel_ring.c +6 -3
index 4034a4bac7f0..d636c6ed88b7 100644
@@ -51,9 +51,12 @@ int intel_ring_pin(struct intel_ring *ring, struct i915_gem_ww_ctx *ww)
 
 	if (i915_vma_is_map_and_fenceable(vma))
 		addr = (void __force *)i915_vma_pin_iomap(vma);
-	else
-		addr = i915_gem_object_pin_map(vma->obj,
-					       i915_coherent_map_type(vma->vm->i915));
+	else {
+		int type = i915_coherent_map_type(vma->vm->i915, vma->obj, false);
+
+		addr = i915_gem_object_pin_map(vma->obj, type);
+	}
+
 	if (IS_ERR(addr)) {
 		ret = PTR_ERR(addr);
 		goto err_ring;
drivers/gpu/drm/i915/gt/intel_timeline.c +6 -2
index b2d04717db20..065943781586 100644
@@ -31,6 +31,7 @@ static int __hwsp_alloc(struct intel_gt *gt, struct intel_timeline_hwsp *hwsp)
 {
 	struct drm_i915_private *i915 = gt->i915;
 	struct drm_i915_gem_object *obj;
+	int type;
 	int ret;
 
 	obj = i915_gem_object_create_internal(i915, PAGE_SIZE);
@@ -47,7 +48,8 @@ static int __hwsp_alloc(struct intel_gt *gt, struct intel_timeline_hwsp *hwsp)
 	}
 
 	/* Pin early so we can call i915_ggtt_pin_unlocked(). */
-	hwsp->vaddr = i915_gem_object_pin_map(obj, I915_MAP_WB);
+	type = i915_coherent_map_type(i915, obj, true);
+	hwsp->vaddr = i915_gem_object_pin_map(obj, type);
 	if (IS_ERR(hwsp->vaddr)) {
 		ret = PTR_ERR(hwsp->vaddr);
 		goto out_unlock;
@@ -235,9 +237,11 @@ intel_timeline_pin_map(struct intel_timeline *timeline)
 	if (!timeline->hwsp_cacheline) {
 		struct drm_i915_gem_object *obj = timeline->hwsp_ggtt->obj;
 		u32 ofs = offset_in_page(timeline->hwsp_offset);
+		int type;
 		void *vaddr;
 
-		vaddr = i915_gem_object_pin_map(obj, I915_MAP_WB);
+		type = i915_coherent_map_type(timeline->gt->i915, obj, true);
+		vaddr = i915_gem_object_pin_map(obj, type);
 		if (IS_ERR(vaddr))
 			return PTR_ERR(vaddr);
 
drivers/gpu/drm/i915/gt/selftest_context.c +2 -1
index d9b0ebc938f1..86b6795dc4f3 100644
@@ -89,7 +89,8 @@ static int __live_context_size(struct intel_engine_cs *engine)
 		goto err;
 
 	vaddr = i915_gem_object_pin_map_unlocked(ce->state->obj,
-						 i915_coherent_map_type(engine->i915));
+						 i915_coherent_map_type(engine->i915,
+									ce->state->obj, false));
 	if (IS_ERR(vaddr)) {
 		err = PTR_ERR(vaddr);
 		intel_context_unpin(ce);
drivers/gpu/drm/i915/gt/selftest_execlists.c +2 -1
index 124011f6fb51..cb17da6a616f 100644
@@ -5854,7 +5854,8 @@ static int compare_isolation(struct intel_engine_cs *engine,
 	}
 
 	lrc = i915_gem_object_pin_map_unlocked(ce->state->obj,
-				      i915_coherent_map_type(engine->i915));
+					       i915_coherent_map_type(engine->i915,
+								      ce->state->obj, true));
 	if (IS_ERR(lrc)) {
 		err = PTR_ERR(lrc);
 		goto err_B1;
drivers/gpu/drm/i915/gt/selftest_hangcheck.c +2 -2
index e3027cebab5b..bc93dba3c8df 100644
@@ -88,7 +88,7 @@ static int hang_init(struct hang *h, struct intel_gt *gt)
 	h->seqno = memset(vaddr, 0xff, PAGE_SIZE);
 
 	vaddr = i915_gem_object_pin_map_unlocked(h->obj,
-						 i915_coherent_map_type(gt->i915));
+						 i915_coherent_map_type(gt->i915, h->obj, false));
 	if (IS_ERR(vaddr)) {
 		err = PTR_ERR(vaddr);
 		goto err_unpin_hws;
@@ -149,7 +149,7 @@ hang_create_request(struct hang *h, struct intel_engine_cs *engine)
 		return ERR_CAST(obj);
 	}
 
-	vaddr = i915_gem_object_pin_map_unlocked(obj, i915_coherent_map_type(gt->i915));
+	vaddr = i915_gem_object_pin_map_unlocked(obj, i915_coherent_map_type(gt->i915, obj, false));
 	if (IS_ERR(vaddr)) {
 		i915_gem_object_put(obj);
 		i915_vm_put(vm);
drivers/gpu/drm/i915/gt/uc/intel_guc.c +3 -1
index a65661eb5d5d..b54b9de31c3e 100644
@@ -694,7 +694,9 @@ int intel_guc_allocate_and_map_vma(struct intel_guc *guc, u32 size,
 	if (IS_ERR(vma))
 		return PTR_ERR(vma);
 
-	vaddr = i915_gem_object_pin_map_unlocked(vma->obj, I915_MAP_WB);
+	vaddr = i915_gem_object_pin_map_unlocked(vma->obj,
+						 i915_coherent_map_type(guc_to_gt(guc)->i915,
+									vma->obj, true));
 	if (IS_ERR(vaddr)) {
 		i915_vma_unpin_and_release(&vma, 0);
 		return PTR_ERR(vaddr);
drivers/gpu/drm/i915/gt/uc/intel_huc.c +3 -1
index 2126dd81ac38..56d2144dc6a0 100644
@@ -82,7 +82,9 @@ static int intel_huc_rsa_data_create(struct intel_huc *huc)
 	if (IS_ERR(vma))
 		return PTR_ERR(vma);
 
-	vaddr = i915_gem_object_pin_map_unlocked(vma->obj, I915_MAP_WB);
+	vaddr = i915_gem_object_pin_map_unlocked(vma->obj,
+						 i915_coherent_map_type(gt->i915,
+									vma->obj, true));
 	if (IS_ERR(vaddr)) {
 		i915_vma_unpin_and_release(&vma, 0);
 		return PTR_ERR(vaddr);
drivers/gpu/drm/i915/i915_drv.h +9 -2
index ce8d5ff8b9f4..13cb4936f15c 100644
@@ -78,6 +78,7 @@
 #include "gem/i915_gem_context_types.h"
 #include "gem/i915_gem_shrinker.h"
 #include "gem/i915_gem_stolen.h"
+#include "gem/i915_gem_lmem.h"
 
 #include "gt/intel_engine.h"
 #include "gt/intel_gt_types.h"
@@ -2027,9 +2028,15 @@ static inline int intel_hws_csb_write_index(struct drm_i915_private *i915)
 }
 
 static inline enum i915_map_type
-i915_coherent_map_type(struct drm_i915_private *i915)
+i915_coherent_map_type(struct drm_i915_private *i915,
+		       struct drm_i915_gem_object *obj, bool always_coherent)
 {
-	return HAS_LLC(i915) ? I915_MAP_WB : I915_MAP_WC;
+	if (i915_gem_object_is_lmem(obj))
+		return I915_MAP_WC;
+	if (HAS_LLC(i915) || always_coherent)
+		return I915_MAP_WB;
+	else
+		return I915_MAP_WC;
 }
 
 static inline u64 i915_cs_timestamp_ns_to_ticks(struct drm_i915_private *i915, u64 val)
drivers/gpu/drm/i915/selftests/igt_spinner.c +2 -2
index 9c461edb0b73..b2a1f98c97f5 100644
@@ -93,9 +93,9 @@ int igt_spinner_pin(struct igt_spinner *spin,
 	}
 
 	if (!spin->batch) {
-		unsigned int mode =
-			i915_coherent_map_type(spin->gt->i915);
+		unsigned int mode;
 
+		mode = i915_coherent_map_type(spin->gt->i915, spin->obj, false);
 		vaddr = igt_spinner_pin_obj(ce, ww, spin->obj, mode, &spin->batch_vma);
 		if (IS_ERR(vaddr))
 			return PTR_ERR(vaddr);
Signature · 6 lines
2.26.2

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
#88 lore raw

RFC PATCH 84/162 drm/i915: introduce kernel blitter_context

In reply to Matthew Auld

We may be without a context to perform various internal blitter operations, for example when performing object migration. Piggybacking off the kernel_context is probably a bad idea, since it has other uses.

Cc Joonas Lahtinen <[email protected]>
Cc Abdiel Janulgue <[email protected]>
---
 drivers/gpu/drm/i915/gt/intel_engine.h       |  2 +
 drivers/gpu/drm/i915/gt/intel_engine_cs.c    | 40 +++++++++++++++++++-
 drivers/gpu/drm/i915/gt/intel_engine_types.h |  1 +
 3 files changed, 41 insertions(+), 2 deletions(-)
drivers/gpu/drm/i915/gt/intel_engine.h +2 -0
index 760fefdfe392..188c5ff6dc64 100644
@@ -186,6 +186,8 @@ intel_write_status_page(struct intel_engine_cs *engine, int reg, u32 value)
 #define I915_GEM_HWS_PREEMPT_ADDR	(I915_GEM_HWS_PREEMPT * sizeof(u32))
 #define I915_GEM_HWS_SEQNO		0x40
 #define I915_GEM_HWS_SEQNO_ADDR		(I915_GEM_HWS_SEQNO * sizeof(u32))
+#define I915_GEM_HWS_BLITTER		0x42
+#define I915_GEM_HWS_BLITTER_ADDR	(I915_GEM_HWS_BLITTER * sizeof(u32))
 #define I915_GEM_HWS_SCRATCH		0x80
 
 #define I915_HWS_CSB_BUF0_INDEX		0x10
drivers/gpu/drm/i915/gt/intel_engine_cs.c +38 -2
index 677c97ded81d..0ba020346566 100644
@@ -819,6 +819,7 @@ create_pinned_context(struct intel_engine_cs *engine,
 	int err;
 
 	ce = intel_context_create(engine);
+
 	if (IS_ERR(ce))
 		return ce;
 
@@ -851,6 +852,20 @@ create_kernel_context(struct intel_engine_cs *engine)
 				     &kernel, "kernel_context");
 }
 
+static struct intel_context *
+create_blitter_context(struct intel_engine_cs *engine)
+{
+	static struct lock_class_key blitter;
+	struct intel_context *ce;
+
+	ce = create_pinned_context(engine, I915_GEM_HWS_BLITTER_ADDR, &blitter,
+				   "blitter_context");
+	if (IS_ERR(ce))
+		return ce;
+
+	return ce;
+}
+
 /**
  * intel_engines_init_common - initialize cengine state which might require hw access
  * @engine: Engine to initialize.
@@ -881,17 +896,33 @@ static int engine_init_common(struct intel_engine_cs *engine)
 	if (IS_ERR(ce))
 		return PTR_ERR(ce);
 
+	engine->kernel_context = ce;
 	ret = measure_breadcrumb_dw(ce);
 	if (ret < 0)
 		goto err_context;
 
 	engine->emit_fini_breadcrumb_dw = ret;
-	engine->kernel_context = ce;
+
+	/*
+	 * The blitter context is used to quickly memset or migrate objects
+	 * in local memory, so it has to always be available.
+	 */
+	if (engine->class == COPY_ENGINE_CLASS) {
+		ce = create_blitter_context(engine);
+		if (IS_ERR(ce)) {
+			ret = PTR_ERR(ce);
+			goto err_unpin;
+		}
+
+		engine->blitter_context = ce;
+	}
 
 	return 0;
 
+err_unpin:
+	intel_context_unpin(engine->kernel_context);
 err_context:
-	intel_context_put(ce);
+	intel_context_put(engine->kernel_context);
 	return ret;
 }
 
@@ -947,6 +978,11 @@ void intel_engine_cleanup_common(struct intel_engine_cs *engine)
 	if (engine->default_state)
 		fput(engine->default_state);
 
+	if (engine->blitter_context) {
+		intel_context_unpin(engine->blitter_context);
+		intel_context_put(engine->blitter_context);
+	}
+
 	if (engine->kernel_context) {
 		intel_context_unpin(engine->kernel_context);
 		intel_context_put(engine->kernel_context);
drivers/gpu/drm/i915/gt/intel_engine_types.h +1 -0
index ee6312601c56..cb2de4bf86ba 100644
@@ -347,6 +347,7 @@ struct intel_engine_cs {
 	struct llist_head barrier_tasks;
 
 	struct intel_context *kernel_context; /* pinned */
+	struct intel_context *blitter_context; /* pinned; exists for BCS only */
 
 	intel_engine_mask_t saturated; /* submitting semaphores too late? */
 
Signature · 6 lines
2.26.2

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
#90 lore raw

RFC PATCH 86/162 drm/i915: Add blit functions that can be called from within a WW transaction

In reply to Matthew Auld

From: Thomas Hellström <[email protected]>

We want to be able to blit from within a ww transaction, so add blit functions that are able to do that. Also take care to unlock the blit batch-buffer after use so it isn't recycled locked.

---
 .../gpu/drm/i915/gem/i915_gem_object_blt.c    | 91 +++++++++++++------
 .../gpu/drm/i915/gem/i915_gem_object_blt.h    | 10 ++
 2 files changed, 72 insertions(+), 29 deletions(-)
drivers/gpu/drm/i915/gem/i915_gem_object_blt.c +62 -29
index e0b873c3f46a..b41b076f6864 100644
@@ -145,11 +145,11 @@ move_obj_to_gpu(struct drm_i915_gem_object *obj,
 	return i915_request_await_object(rq, obj, write);
 }
 
-int i915_gem_object_fill_blt(struct drm_i915_gem_object *obj,
-			     struct intel_context *ce,
-			     u32 value)
+int i915_gem_object_ww_fill_blt(struct drm_i915_gem_object *obj,
+				struct i915_gem_ww_ctx *ww,
+				struct intel_context *ce,
+				u32 value)
 {
-	struct i915_gem_ww_ctx ww;
 	struct i915_request *rq;
 	struct i915_vma *batch;
 	struct i915_vma *vma;
@@ -159,22 +159,16 @@ int i915_gem_object_fill_blt(struct drm_i915_gem_object *obj,
 	if (IS_ERR(vma))
 		return PTR_ERR(vma);
 
-	i915_gem_ww_ctx_init(&ww, true);
 	intel_engine_pm_get(ce->engine);
-retry:
-	err = i915_gem_object_lock(obj, &ww);
+	err = intel_context_pin_ww(ce, ww);
 	if (err)
 		goto out;
 
-	err = intel_context_pin_ww(ce, &ww);
-	if (err)
-		goto out;
-
-	err = i915_vma_pin_ww(vma, &ww, 0, 0, PIN_USER);
+	err = i915_vma_pin_ww(vma, ww, 0, 0, PIN_USER);
 	if (err)
 		goto out_ctx;
 
-	batch = intel_emit_vma_fill_blt(ce, vma, &ww, value);
+	batch = intel_emit_vma_fill_blt(ce, vma, ww, value);
 	if (IS_ERR(batch)) {
 		err = PTR_ERR(batch);
 		goto out_vma;
@@ -210,22 +204,43 @@ int i915_gem_object_fill_blt(struct drm_i915_gem_object *obj,
 
 	i915_request_add(rq);
 out_batch:
+	i915_gem_ww_unlock_single(batch->obj);
 	intel_emit_vma_release(ce, batch);
 out_vma:
 	i915_vma_unpin(vma);
 out_ctx:
 	intel_context_unpin(ce);
 out:
+	intel_engine_pm_put(ce->engine);
+	return err;
+}
+
+int i915_gem_object_fill_blt(struct drm_i915_gem_object *obj,
+				struct intel_context *ce,
+				u32 value)
+{
+	struct i915_gem_ww_ctx ww;
+	int err;
+
+	i915_gem_ww_ctx_init(&ww, true);
+retry:
+	err = i915_gem_object_lock(obj, &ww);
+	if (err)
+		goto out_err;
+
+	err = i915_gem_object_ww_fill_blt(obj, &ww, ce, value);
+out_err:
 	if (err == -EDEADLK) {
 		err = i915_gem_ww_ctx_backoff(&ww);
 		if (!err)
 			goto retry;
 	}
 	i915_gem_ww_ctx_fini(&ww);
-	intel_engine_pm_put(ce->engine);
+
 	return err;
 }
 
+
 /* Wa_1209644611:icl,ehl */
 static bool wa_1209644611_applies(struct drm_i915_private *i915, u32 size)
 {
@@ -354,13 +369,13 @@ struct i915_vma *intel_emit_vma_copy_blt(struct intel_context *ce,
 	return ERR_PTR(err);
 }
 
-int i915_gem_object_copy_blt(struct drm_i915_gem_object *src,
-			     struct drm_i915_gem_object *dst,
-			     struct intel_context *ce)
+int i915_gem_object_ww_copy_blt(struct drm_i915_gem_object *src,
+				struct drm_i915_gem_object *dst,
+				struct i915_gem_ww_ctx *ww,
+				struct intel_context *ce)
 {
 	struct i915_address_space *vm = ce->vm;
 	struct i915_vma *vma[2], *batch;
-	struct i915_gem_ww_ctx ww;
 	struct i915_request *rq;
 	int err, i;
 
@@ -372,26 +387,20 @@ int i915_gem_object_copy_blt(struct drm_i915_gem_object *src,
 	if (IS_ERR(vma[1]))
 		return PTR_ERR(vma[1]);
 
-	i915_gem_ww_ctx_init(&ww, true);
 	intel_engine_pm_get(ce->engine);
-retry:
-	err = i915_gem_object_lock(src, &ww);
-	if (!err)
-		err = i915_gem_object_lock(dst, &ww);
-	if (!err)
-		err = intel_context_pin_ww(ce, &ww);
+	err = intel_context_pin_ww(ce, ww);
 	if (err)
 		goto out;
 
-	err = i915_vma_pin_ww(vma[0], &ww, 0, 0, PIN_USER);
+	err = i915_vma_pin_ww(vma[0], ww, 0, 0, PIN_USER);
 	if (err)
 		goto out_ctx;
 
-	err = i915_vma_pin_ww(vma[1], &ww, 0, 0, PIN_USER);
+	err = i915_vma_pin_ww(vma[1], ww, 0, 0, PIN_USER);
 	if (unlikely(err))
 		goto out_unpin_src;
 
-	batch = intel_emit_vma_copy_blt(ce, &ww, vma[0], vma[1]);
+	batch = intel_emit_vma_copy_blt(ce, ww, vma[0], vma[1]);
 	if (IS_ERR(batch)) {
 		err = PTR_ERR(batch);
 		goto out_unpin_dst;
@@ -437,6 +446,7 @@ int i915_gem_object_copy_blt(struct drm_i915_gem_object *src,
 
 	i915_request_add(rq);
 out_batch:
+	i915_gem_ww_unlock_single(batch->obj);
 	intel_emit_vma_release(ce, batch);
 out_unpin_dst:
 	i915_vma_unpin(vma[1]);
@@ -445,13 +455,36 @@ int i915_gem_object_copy_blt(struct drm_i915_gem_object *src,
 out_ctx:
 	intel_context_unpin(ce);
 out:
+	intel_engine_pm_put(ce->engine);
+	return err;
+}
+
+int i915_gem_object_copy_blt(struct drm_i915_gem_object *src,
+			     struct drm_i915_gem_object *dst,
+			     struct intel_context *ce)
+{
+	struct i915_gem_ww_ctx ww;
+	int err;
+
+	i915_gem_ww_ctx_init(&ww, true);
+retry:
+	err = i915_gem_object_lock(src, &ww);
+	if (err)
+		goto out_err;
+
+	err = i915_gem_object_lock(dst, &ww);
+	if (err)
+		goto out_err;
+
+	err = i915_gem_object_ww_copy_blt(src, dst, &ww, ce);
+out_err:
 	if (err == -EDEADLK) {
 		err = i915_gem_ww_ctx_backoff(&ww);
 		if (!err)
 			goto retry;
 	}
 	i915_gem_ww_ctx_fini(&ww);
-	intel_engine_pm_put(ce->engine);
+
 	return err;
 }
 
drivers/gpu/drm/i915/gem/i915_gem_object_blt.h +10 -0
index 2409fdcccf0e..da3d66abde64 100644
@@ -36,4 +36,14 @@ int i915_gem_object_copy_blt(struct drm_i915_gem_object *src,
 			     struct drm_i915_gem_object *dst,
 			     struct intel_context *ce);
 
+int i915_gem_object_ww_fill_blt(struct drm_i915_gem_object *obj,
+				struct i915_gem_ww_ctx *ww,
+				struct intel_context *ce,
+				u32 value);
+
+int i915_gem_object_ww_copy_blt(struct drm_i915_gem_object *src,
+				struct drm_i915_gem_object *dst,
+				struct i915_gem_ww_ctx *ww,
+				struct intel_context *ce);
+
 #endif
Signature · 6 lines
2.26.2

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel
#89 lore raw

RFC PATCH 85/162 drm/i915/region: support basic eviction

In reply to Matthew Auld

Support basic eviction for regions.

Cc Joonas Lahtinen <[email protected]>
Cc Abdiel Janulgue <[email protected]>
---
 .../gpu/drm/i915/gem/i915_gem_object_types.h  |  1 +
 drivers/gpu/drm/i915/gem/i915_gem_shrinker.c  | 59 ++++++++++++++
 drivers/gpu/drm/i915/gem/i915_gem_shrinker.h  |  4 +
 drivers/gpu/drm/i915/i915_gem.c               | 17 +++++
 drivers/gpu/drm/i915/intel_memory_region.c    | 24 +++++-
 .../drm/i915/selftests/intel_memory_region.c  | 76 +++++++++++++++++++
 6 files changed, 178 insertions(+), 3 deletions(-)
drivers/gpu/drm/i915/gem/i915_gem_object_types.h +1 -0
index b172e8cc53ab..6d101275bc9d 100644
@@ -226,6 +226,7 @@ struct drm_i915_gem_object {
 		 * region->obj_lock.
 		 */
 		struct list_head region_link;
+		struct list_head tmp_link;
 
 		struct sg_table *pages;
 		void *mapping;
drivers/gpu/drm/i915/gem/i915_gem_shrinker.c +59 -0

71 lines not on this page · raw message

drivers/gpu/drm/i915/gem/i915_gem_shrinker.h +4 -0
index 8512470f6fd6..c945f3b587d6 100644
@@ -7,10 +7,12 @@
 #define __I915_GEM_SHRINKER_H__
 
 #include <linux/bits.h>
+#include <linux/types.h>
 
 struct drm_i915_private;
 struct i915_gem_ww_ctx;
 struct mutex;
+struct intel_memory_region;
 
 /* i915_gem_shrinker.c */
 unsigned long i915_gem_shrink(struct i915_gem_ww_ctx *ww,
@@ -29,5 +31,7 @@ void i915_gem_driver_register__shrinker(struct drm_i915_private *i915);
 void i915_gem_driver_unregister__shrinker(struct drm_i915_private *i915);
 void i915_gem_shrinker_taints_mutex(struct drm_i915_private *i915,
 				    struct mutex *mutex);
+int i915_gem_shrink_memory_region(struct intel_memory_region *mem,
+				  resource_size_t target);
 
 #endif /* __I915_GEM_SHRINKER_H__ */
drivers/gpu/drm/i915/i915_gem.c +17 -0
index 2662d679db6e..ef2124c17a7f 100644
@@ -1104,6 +1104,23 @@ i915_gem_madvise_ioctl(struct drm_device *dev, void *data,
 	    !i915_gem_object_has_pages(obj))
 		i915_gem_object_truncate(obj);
 
+	if (obj->mm.region && i915_gem_object_has_pages(obj)) {
+		mutex_lock(&obj->mm.region->objects.lock);
+
+		switch (obj->mm.madv) {
+		case I915_MADV_WILLNEED:
+			list_move(&obj->mm.region_link,
+				  &obj->mm.region->objects.list);
+			break;
+		default:
+			list_move(&obj->mm.region_link,
+				  &obj->mm.region->objects.purgeable);
+			break;
+		}
+
+		mutex_unlock(&obj->mm.region->objects.lock);
+	}
+
 	args->retained = obj->mm.madv != __I915_MADV_PURGED;
 
 	i915_gem_object_unlock(obj);
drivers/gpu/drm/i915/intel_memory_region.c +21 -3
index b326993a1026..308f89b87834 100644
@@ -97,7 +97,8 @@ __intel_memory_region_get_pages_buddy(struct intel_memory_region *mem,
 	do {
 		struct i915_buddy_block *block;
 		unsigned int order;
-
+		bool retry = true;
+retry:
 		order = fls(n_pages) - 1;
 		GEM_BUG_ON(order > mem->mm.max_order);
 		GEM_BUG_ON(order < min_order);
@@ -107,8 +108,25 @@ __intel_memory_region_get_pages_buddy(struct intel_memory_region *mem,
 			if (!IS_ERR(block))
 				break;
 
-			if (order-- == min_order)
-				goto err_free_blocks;
+			if (order-- == min_order) {
+				resource_size_t target;
+				int err;
+
+				if (!retry)
+					goto err_free_blocks;
+
+				target = n_pages * mem->mm.chunk_size;
+
+				mutex_unlock(&mem->mm_lock);
+				err = i915_gem_shrink_memory_region(mem,
+								    target);
+				mutex_lock(&mem->mm_lock);
+				if (err)
+					goto err_free_blocks;
+
+				retry = false;
+				goto retry;
+			}
 		} while (1);
 
 		n_pages -= BIT(order);
drivers/gpu/drm/i915/selftests/intel_memory_region.c +76 -0

88 lines not on this page · raw message

Signature · 6 lines
2.26.2

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel
#92 lore raw

RFC PATCH 88/162 drm/i915: support basic object migration

In reply to Matthew Auld

We are going want to able to move objects between different regions like system memory and local memory. In the future everything should be just another region.

Cc Abdiel Janulgue <[email protected]>
Cc Sudeep Dutt <[email protected]>
Cc Matthew Auld <[email protected]>
Cc Joonas Lahtinen <[email protected]>
Cc Abdiel Janulgue <[email protected]>
Cc Daniele Ceraolo Spurio <[email protected]>
Cc Prathap Kumar Valsan <[email protected]>
Cc CQ Tang <[email protected]>
Cc Tvrtko Ursulin <[email protected]>
---
 drivers/gpu/drm/i915/gem/i915_gem_mman.c      |  13 ++
 drivers/gpu/drm/i915/gem/i915_gem_mman.h      |   2 +
 drivers/gpu/drm/i915/gem/i915_gem_object.c    | 125 +++++++++++++
 drivers/gpu/drm/i915/gem/i915_gem_object.h    |   9 +
 drivers/gpu/drm/i915/gem/i915_gem_pages.c     |   2 +-
 .../drm/i915/selftests/intel_memory_region.c  | 174 +++++++++++++++++-
 6 files changed, 322 insertions(+), 3 deletions(-)
drivers/gpu/drm/i915/gem/i915_gem_mman.c +13 -0
index 2561a2f1e54f..4e8a05c35252 100644
@@ -546,6 +546,19 @@ void i915_gem_object_release_mmap_offset(struct drm_i915_gem_object *obj)
 	spin_unlock(&obj->mmo.lock);
 }
 
+/**
+ * i915_gem_object_release_mmap - remove physical page mappings
+ * @obj: obj in question
+ *
+ * Preserve the reservation of the mmapping with the DRM core code, but
+ * relinquish ownership of the pages back to the system.
+ */
+void i915_gem_object_release_mmap(struct drm_i915_gem_object *obj)
+{
+	i915_gem_object_release_mmap_gtt(obj);
+	i915_gem_object_release_mmap_offset(obj);
+}
+
 static struct i915_mmap_offset *
 lookup_mmo(struct drm_i915_gem_object *obj,
 	   enum i915_mmap_type mmap_type)
drivers/gpu/drm/i915/gem/i915_gem_mman.h +2 -0
index efee9e0d2508..7c5ccdf59359 100644
@@ -24,6 +24,8 @@ int i915_gem_dumb_mmap_offset(struct drm_file *file_priv,
 			      struct drm_device *dev,
 			      u32 handle, u64 *offset);
 
+void i915_gem_object_release_mmap(struct drm_i915_gem_object *obj);
+
 void __i915_gem_object_release_mmap_gtt(struct drm_i915_gem_object *obj);
 void i915_gem_object_release_mmap_gtt(struct drm_i915_gem_object *obj);
 
drivers/gpu/drm/i915/gem/i915_gem_object.c +125 -0

142 lines not on this page · raw message

drivers/gpu/drm/i915/gem/i915_gem_object.h +9 -0
index c6c7ab181a65..1a1aa71a4494 100644
@@ -51,8 +51,17 @@ void i915_gem_object_put_pages_phys(struct drm_i915_gem_object *obj,
 				    struct sg_table *pages);
 
 
+enum intel_region_id;
+int i915_gem_object_prepare_move(struct drm_i915_gem_object *obj);
+int i915_gem_object_migrate(struct drm_i915_gem_object *obj,
+			    struct i915_gem_ww_ctx *ww,
+			    struct intel_context *ce,
+			    enum intel_region_id id);
+
 void i915_gem_flush_free_objects(struct drm_i915_private *i915);
 
+void __i915_gem_object_reset_page_iter(struct drm_i915_gem_object *obj);
+
 struct sg_table *
 __i915_gem_object_unset_pages(struct drm_i915_gem_object *obj);
 void i915_gem_object_truncate(struct drm_i915_gem_object *obj);
drivers/gpu/drm/i915/gem/i915_gem_pages.c +1 -1
index eacad971b955..2cdb7cf63383 100644
@@ -183,7 +183,7 @@ void i915_gem_object_writeback(struct drm_i915_gem_object *obj)
 		obj->ops->writeback(obj);
 }
 
-static void __i915_gem_object_reset_page_iter(struct drm_i915_gem_object *obj)
+void __i915_gem_object_reset_page_iter(struct drm_i915_gem_object *obj)
 {
 	struct radix_tree_iter iter;
 	void __rcu **slot;
drivers/gpu/drm/i915/selftests/intel_memory_region.c +172 -2

210 lines not on this page · raw message

Signature · 6 lines
2.26.2

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel
#91 lore raw

RFC PATCH 87/162 drm/i915: Delay publishing objects on the eviction lists

In reply to Matthew Auld

From: Thomas Hellström <[email protected]>

When an object is published on an eviction list, it's considered for eviction and can be locked by other threads. This is strictly not necessary until the object has pages. To limit eviction lookups that need to discard the object and facilitate a longer period during which we can lock the object isolated (trylock or ww lock without chance of deadlock or interruption), delay eviction list publishing until pages are set. Also take the object off the eviction lists when pages are unset. Finally make sure that an object is either locked or isolated when eviction list manipulation happens.

---
 drivers/gpu/drm/i915/gem/i915_gem_object.c |  2 ++
 drivers/gpu/drm/i915/gem/i915_gem_pages.c  | 22 +++++++++++++++++++++-
 drivers/gpu/drm/i915/gem/i915_gem_region.c | 18 ++----------------
 3 files changed, 25 insertions(+), 17 deletions(-)
drivers/gpu/drm/i915/gem/i915_gem_object.c +2 -0
index 08d806bbf48e..5326b4b5a9f7 100644
@@ -66,6 +66,7 @@ void i915_gem_object_init(struct drm_i915_gem_object *obj,
 	INIT_LIST_HEAD(&obj->vma.list);
 
 	INIT_LIST_HEAD(&obj->mm.link);
+	INIT_LIST_HEAD(&obj->mm.region_link);
 
 	INIT_LIST_HEAD(&obj->lut_list);
 	spin_lock_init(&obj->lut_lock);
@@ -79,6 +80,7 @@ void i915_gem_object_init(struct drm_i915_gem_object *obj,
 	GEM_BUG_ON(flags & ~I915_BO_ALLOC_FLAGS);
 	obj->flags = flags;
 
+	obj->mm.region = NULL;
 	obj->mm.madv = I915_MADV_WILLNEED;
 	INIT_RADIX_TREE(&obj->mm.get_page.radix, GFP_KERNEL | __GFP_NOWARN);
 	mutex_init(&obj->mm.get_page.lock);
drivers/gpu/drm/i915/gem/i915_gem_pages.c +21 -1
index 4a8be759832b..eacad971b955 100644
@@ -16,6 +16,8 @@ void __i915_gem_object_set_pages(struct drm_i915_gem_object *obj,
 {
 	struct drm_i915_private *i915 = to_i915(obj->base.dev);
 	unsigned long supported = INTEL_INFO(i915)->page_sizes;
+	struct intel_memory_region *mem;
+	struct list_head *list;
 	int i;
 
 	assert_object_held_shared(obj);
@@ -64,7 +66,6 @@ void __i915_gem_object_set_pages(struct drm_i915_gem_object *obj,
 	GEM_BUG_ON(!HAS_PAGE_SIZES(i915, obj->mm.page_sizes.sg));
 
 	if (i915_gem_object_is_shrinkable(obj)) {
-		struct list_head *list;
 		unsigned long flags;
 
 		assert_object_held(obj);
@@ -82,6 +83,18 @@ void __i915_gem_object_set_pages(struct drm_i915_gem_object *obj,
 		atomic_set(&obj->mm.shrink_pin, 0);
 		spin_unlock_irqrestore(&i915->mm.obj_lock, flags);
 	}
+
+	mem = obj->mm.region;
+	if (mem) {
+		mutex_lock(&mem->objects.lock);
+		GEM_WARN_ON(!list_empty(&obj->mm.region_link));
+		if (obj->mm.madv != I915_MADV_WILLNEED)
+			list = &mem->objects.purgeable;
+		else
+			list = &mem->objects.list;
+		list_move_tail(&obj->mm.region_link, list);
+		mutex_unlock(&mem->objects.lock);
+	}
 }
 
 int ____i915_gem_object_get_pages(struct drm_i915_gem_object *obj)
@@ -192,6 +205,7 @@ static void unmap_object(struct drm_i915_gem_object *obj, void *ptr)
 struct sg_table *
 __i915_gem_object_unset_pages(struct drm_i915_gem_object *obj)
 {
+	struct intel_memory_region *mem = obj->mm.region;
 	struct sg_table *pages;
 
 	assert_object_held_shared(obj);
@@ -205,6 +219,12 @@ __i915_gem_object_unset_pages(struct drm_i915_gem_object *obj)
 
 	i915_gem_object_make_unshrinkable(obj);
 
+	if (mem) {
+		mutex_lock(&mem->objects.lock);
+		list_del_init(&obj->mm.region_link);
+		mutex_unlock(&mem->objects.lock);
+	}
+
 	if (obj->mm.mapping) {
 		unmap_object(obj, page_mask_bits(obj->mm.mapping));
 		obj->mm.mapping = NULL;
drivers/gpu/drm/i915/gem/i915_gem_region.c +2 -16
index 6a96741253b3..58bf5f9e3199 100644
@@ -105,30 +105,16 @@ void i915_gem_object_init_memory_region(struct drm_i915_gem_object *obj,
 					struct intel_memory_region *mem)
 {
 	INIT_LIST_HEAD(&obj->mm.blocks);
+	WARN_ON(i915_gem_object_has_pages(obj));
 	obj->mm.region = intel_memory_region_get(mem);
 
 	if (obj->base.size <= mem->min_page_size)
 		obj->flags |= I915_BO_ALLOC_CONTIGUOUS;
-
-	mutex_lock(&mem->objects.lock);
-
-	if (obj->flags & I915_BO_ALLOC_VOLATILE)
-		list_add(&obj->mm.region_link, &mem->objects.purgeable);
-	else
-		list_add(&obj->mm.region_link, &mem->objects.list);
-
-	mutex_unlock(&mem->objects.lock);
 }
 
 void i915_gem_object_release_memory_region(struct drm_i915_gem_object *obj)
 {
-	struct intel_memory_region *mem = obj->mm.region;
-
-	mutex_lock(&mem->objects.lock);
-	list_del(&obj->mm.region_link);
-	mutex_unlock(&mem->objects.lock);
-
-	intel_memory_region_put(mem);
+	intel_memory_region_put(obj->mm.region);
 }
 
 struct drm_i915_gem_object *
Signature · 6 lines
2.26.2

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel
#93 lore raw

RFC PATCH 89/162 drm/i915/dg1: Fix occasional migration error

In reply to Matthew Auld

From: CQ Tang <[email protected]>

We posted blitter copying operation. Then we call i915_gem_object_set_to_cpu_domain(), inside this function, we call i915_gem_object_wait() with interruptible flag. Sometimes this wait call gets interrupted by the blitter copying complete interrupt. This will make migration operation to fail. So before calling i915_gem_object_set_to_cpu_domain(), we call i915_gem_object_wait() with non-interruptible flag to wait for the blitter operation to finish.

Cc Matthew Auld <[email protected]>
Cc Sudeep Dutt <[email protected]>
Cc Ramalingam C <[email protected]>
---
 drivers/gpu/drm/i915/gem/i915_gem_object.c | 11 +++++++++++
 1 file changed, 11 insertions(+)
drivers/gpu/drm/i915/gem/i915_gem_object.c +11 -0
index 7ff430503497..49935245a4a8 100644
@@ -381,6 +381,17 @@ int i915_gem_object_migrate(struct drm_i915_gem_object *obj,
 		err = i915_gem_object_ww_copy_blt(obj, donor, ww, ce);
 		if (err)
 			goto unlock_donor;
+
+		/*
+		 * Occasionally i915_gem_object_wait() called inside
+		 * i915_gem_object_set_to_cpu_domain() get interrupted
+		 * and return -ERESTARTSYS, this will make migration
+		 * operation fail. So adding a non-interruptible wait
+		 * before changing the object domain.
+		 */
+		err = i915_gem_object_wait(donor, 0, MAX_SCHEDULE_TIMEOUT);
+		if (err)
+			goto unlock_donor;
 	}
 
 	err = i915_gem_object_set_to_cpu_domain(donor, false);
Signature · 6 lines
2.26.2

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel
#94 lore raw

RFC PATCH 90/162 drm/i915/query: Expose memory regions through the query uAPI

In reply to Matthew Auld

From: Abdiel Janulgue <[email protected]>

Returns the available memory region areas supported by the HW.

---
 drivers/gpu/drm/i915/gem/i915_gem_stolen.c | 12 ++++-
 drivers/gpu/drm/i915/gem/i915_gem_stolen.h |  3 ++
 drivers/gpu/drm/i915/i915_drv.c            |  2 +-
 drivers/gpu/drm/i915/i915_pci.c            |  2 +-
 drivers/gpu/drm/i915/i915_query.c          | 62 ++++++++++++++++++++++
 drivers/gpu/drm/i915/intel_memory_region.c | 32 ++++++-----
 drivers/gpu/drm/i915/intel_memory_region.h | 38 +++++++------
 include/uapi/drm/i915_drm.h                | 58 ++++++++++++++++++++
 8 files changed, 172 insertions(+), 37 deletions(-)
drivers/gpu/drm/i915/gem/i915_gem_stolen.c +10 -2
index ce9086d3a647..25e3cc53316e 100644
@@ -704,11 +704,19 @@ _i915_gem_object_create_stolen(struct intel_memory_region *mem,
 	return obj;
 }
 
+struct intel_memory_region *i915_stolen_region(struct drm_i915_private *i915)
+{
+	if (HAS_LMEM(i915))
+		return i915->mm.regions[INTEL_REGION_STOLEN_LMEM];
+
+	return i915->mm.regions[INTEL_REGION_STOLEN_SMEM];
+}
+
 struct drm_i915_gem_object *
 i915_gem_object_create_stolen(struct drm_i915_private *i915,
 			      resource_size_t size)
 {
-	return i915_gem_object_create_region(i915->mm.regions[INTEL_REGION_STOLEN],
+	return i915_gem_object_create_region(i915_stolen_region(i915),
 					     size, I915_BO_ALLOC_CONTIGUOUS);
 }
 
@@ -748,7 +756,7 @@ i915_gem_object_create_stolen_for_preallocated(struct drm_i915_private *i915,
 					       resource_size_t stolen_offset,
 					       resource_size_t size)
 {
-	struct intel_memory_region *mem = i915->mm.regions[INTEL_REGION_STOLEN];
+	struct intel_memory_region *mem = i915_stolen_region(i915);
 	struct drm_i915_gem_object *obj;
 	struct drm_mm_node *stolen;
 	int ret;
drivers/gpu/drm/i915/gem/i915_gem_stolen.h +3 -0
index 61e028063f9f..67f6264f3ff9 100644
@@ -22,6 +22,9 @@ int i915_gem_stolen_insert_node_in_range(struct drm_i915_private *dev_priv,
 void i915_gem_stolen_remove_node(struct drm_i915_private *dev_priv,
 				 struct drm_mm_node *node);
 struct intel_memory_region *i915_gem_stolen_setup(struct drm_i915_private *i915);
+
+struct intel_memory_region *i915_stolen_region(struct drm_i915_private *i915);
+
 struct drm_i915_gem_object *
 i915_gem_object_create_stolen(struct drm_i915_private *dev_priv,
 			      resource_size_t size);
drivers/gpu/drm/i915/i915_drv.c +1 -1
index 320856b665a1..07b3a89ec09e 100644
@@ -843,7 +843,7 @@ int i915_driver_probe(struct pci_dev *pdev, const struct pci_device_id *ent)
 		if (INTEL_GEN(i915) >= 9 && i915_selftest.live < 0 &&
 		    i915->params.fake_lmem_start) {
 			mkwrite_device_info(i915)->memory_regions =
-				REGION_SMEM | REGION_LMEM | REGION_STOLEN;
+				REGION_SMEM | REGION_LMEM | REGION_STOLEN_SMEM;
 			GEM_BUG_ON(!HAS_LMEM(i915));
 		}
 	}
drivers/gpu/drm/i915/i915_pci.c +1 -1
index 11fe790b1969..8243178a56f9 100644
@@ -154,7 +154,7 @@
 	.page_sizes = I915_GTT_PAGE_SIZE_4K
 
 #define GEN_DEFAULT_REGIONS \
-	.memory_regions = REGION_SMEM | REGION_STOLEN
+	.memory_regions = REGION_SMEM | REGION_STOLEN_SMEM
 
 #define I830_FEATURES \
 	GEN(2), \
drivers/gpu/drm/i915/i915_query.c +62 -0

73 lines not on this page · raw message

drivers/gpu/drm/i915/intel_memory_region.c +19 -13
index 308f89b87834..dca1e367ab98 100644
@@ -6,14 +6,19 @@
 #include "intel_memory_region.h"
 #include "i915_drv.h"
 
-/* XXX: Hysterical raisins. BIT(inst) needs to just be (inst) at some point. */
-#define REGION_MAP(type, inst) \
-	BIT((type) + INTEL_MEMORY_TYPE_SHIFT) | BIT(inst)
-
-const u32 intel_region_map[] = {
-	[INTEL_REGION_SMEM] = REGION_MAP(INTEL_MEMORY_SYSTEM, 0),
-	[INTEL_REGION_LMEM] = REGION_MAP(INTEL_MEMORY_LOCAL, 0),
-	[INTEL_REGION_STOLEN] = REGION_MAP(INTEL_MEMORY_STOLEN, 0),
+const struct intel_memory_region_info intel_region_map[] = {
+       [INTEL_REGION_SMEM] = {
+               .class = INTEL_MEMORY_SYSTEM,
+               .instance = 0,
+       },
+       [INTEL_REGION_LMEM] = {
+               .class = INTEL_MEMORY_LOCAL,
+               .instance = 0,
+       },
+       [INTEL_REGION_STOLEN_SMEM] = {
+               .class = INTEL_MEMORY_STOLEN_SYSTEM,
+               .instance = 0,
+       },
 };
 
 struct intel_memory_region *
@@ -263,17 +268,18 @@ int intel_memory_regions_hw_probe(struct drm_i915_private *i915)
 
 	for (i = 0; i < ARRAY_SIZE(i915->mm.regions); i++) {
 		struct intel_memory_region *mem = ERR_PTR(-ENODEV);
-		u32 type;
+		u16 type, instance;
 
 		if (!HAS_REGION(i915, BIT(i)))
 			continue;
 
-		type = MEMORY_TYPE_FROM_REGION(intel_region_map[i]);
+		type = intel_region_map[i].class;
+		instance = intel_region_map[i].instance;
 		switch (type) {
 		case INTEL_MEMORY_SYSTEM:
 			mem = i915_gem_shmem_setup(i915);
 			break;
-		case INTEL_MEMORY_STOLEN:
+		case INTEL_MEMORY_STOLEN_SYSTEM:
 			mem = i915_gem_stolen_setup(i915);
 			break;
 		case INTEL_MEMORY_LOCAL:
@@ -289,9 +295,9 @@ int intel_memory_regions_hw_probe(struct drm_i915_private *i915)
 			goto out_cleanup;
 		}
 
-		mem->id = intel_region_map[i];
+		mem->id = i;
 		mem->type = type;
-		mem->instance = MEMORY_INSTANCE_FROM_REGION(intel_region_map[i]);
+		mem->instance = instance;
 
 		i915->mm.regions[i] = mem;
 	}
drivers/gpu/drm/i915/intel_memory_region.h +18 -20
index 232490d89a83..c047cf7c5e7c 100644
@@ -11,6 +11,7 @@
 #include <linux/mutex.h>
 #include <linux/io-mapping.h>
 #include <drm/drm_mm.h>
+#include <drm/i915_drm.h>
 
 #include "i915_buddy.h"
 
@@ -19,30 +20,25 @@ struct drm_i915_gem_object;
 struct intel_memory_region;
 struct sg_table;
 
-/**
- *  Base memory type
- */
 enum intel_memory_type {
-	INTEL_MEMORY_SYSTEM = 0,
-	INTEL_MEMORY_LOCAL,
-	INTEL_MEMORY_STOLEN,
+	INTEL_MEMORY_SYSTEM = I915_MEMORY_CLASS_SYSTEM,
+	INTEL_MEMORY_LOCAL = I915_MEMORY_CLASS_DEVICE,
+	INTEL_MEMORY_STOLEN_SYSTEM = I915_MEMORY_CLASS_STOLEN_SYSTEM,
+	INTEL_MEMORY_STOLEN_LOCAL = I915_MEMORY_CLASS_STOLEN_DEVICE,
 };
 
 enum intel_region_id {
 	INTEL_REGION_SMEM = 0,
 	INTEL_REGION_LMEM,
-	INTEL_REGION_STOLEN,
+	INTEL_REGION_STOLEN_SMEM,
+	INTEL_REGION_STOLEN_LMEM,
 	INTEL_REGION_UNKNOWN, /* Should be last */
 };
 
 #define REGION_SMEM     BIT(INTEL_REGION_SMEM)
 #define REGION_LMEM     BIT(INTEL_REGION_LMEM)
-#define REGION_STOLEN   BIT(INTEL_REGION_STOLEN)
-
-#define INTEL_MEMORY_TYPE_SHIFT 16
-
-#define MEMORY_TYPE_FROM_REGION(r) (ilog2((r) >> INTEL_MEMORY_TYPE_SHIFT))
-#define MEMORY_INSTANCE_FROM_REGION(r) (ilog2((r) & 0xffff))
+#define REGION_STOLEN_SMEM   BIT(INTEL_REGION_STOLEN_SMEM)
+#define REGION_STOLEN_LMEM   BIT(INTEL_REGION_STOLEN_LMEM)
 
 #define I915_ALLOC_MIN_PAGE_SIZE  BIT(0)
 #define I915_ALLOC_CONTIGUOUS     BIT(1)
@@ -51,10 +47,12 @@ enum intel_region_id {
 	for (id = 0; id < ARRAY_SIZE((i915)->mm.regions); id++) \
 		for_each_if((mr) = (i915)->mm.regions[id])
 
-/**
- * Memory regions encoded as type | instance
- */
-extern const u32 intel_region_map[];
+struct intel_memory_region_info {
+       u16 class;
+       u16 instance;
+};
+
+extern const struct intel_memory_region_info intel_region_map[];
 
 struct intel_memory_region_ops {
 	unsigned int flags;
@@ -89,9 +87,9 @@ struct intel_memory_region {
 	resource_size_t total;
 	resource_size_t avail;
 
-	unsigned int type;
-	unsigned int instance;
-	unsigned int id;
+	u16 type;
+	u16 instance;
+	enum intel_region_id id;
 	char name[8];
 
 	dma_addr_t remap_addr;
include/uapi/drm/i915_drm.h +58 -0

70 lines not on this page · raw message

Signature · 6 lines
2.26.2

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
#95 lore raw

RFC PATCH 91/162 drm/i915: Store gt in memory region

In reply to Matthew Auld

From: Prathap Kumar Valsan <[email protected]>

Store pointer to gt closest to its memory region so that we can access the engines corresponding to that gt via memory region.

---
 drivers/gpu/drm/i915/intel_memory_region.c | 1 +
 drivers/gpu/drm/i915/intel_memory_region.h | 1 +
 2 files changed, 2 insertions(+)
drivers/gpu/drm/i915/intel_memory_region.c +1 -0
index dca1e367ab98..6f40748901da 100644
@@ -298,6 +298,7 @@ int intel_memory_regions_hw_probe(struct drm_i915_private *i915)
 		mem->id = i;
 		mem->type = type;
 		mem->instance = instance;
+		mem->gt = &i915->gt;
 
 		i915->mm.regions[i] = mem;
 	}
drivers/gpu/drm/i915/intel_memory_region.h +1 -0
index c047cf7c5e7c..15dcb57b4b5a 100644
@@ -91,6 +91,7 @@ struct intel_memory_region {
 	u16 instance;
 	enum intel_region_id id;
 	char name[8];
+	struct intel_gt *gt; /* GT closest to this region. */
 
 	dma_addr_t remap_addr;
 
Signature · 6 lines
2.26.2

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
#102 lore raw

RFC PATCH 93/162 drm/i915/lmem: allocate cmd ring in lmem

In reply to Matthew Auld

From: Michel Thierry <[email protected]>

Cc Joonas Lahtinen <[email protected]>
Cc Abdiel Janulgue <[email protected]>
---
 drivers/gpu/drm/i915/gt/intel_ring.c | 15 +++++++++++----
 1 file changed, 11 insertions(+), 4 deletions(-)
drivers/gpu/drm/i915/gt/intel_ring.c +11 -4
index d636c6ed88b7..aa75e644f3f2 100644
@@ -4,6 +4,7 @@
  * Copyright © 2019 Intel Corporation
  */
 
+#include "gem/i915_gem_lmem.h"
 #include "gem/i915_gem_object.h"
 #include "i915_drv.h"
 #include "i915_vma.h"
@@ -111,10 +112,16 @@ static struct i915_vma *create_ring_vma(struct i915_ggtt *ggtt, int size)
 	struct i915_vma *vma;
 
 	obj = ERR_PTR(-ENODEV);
-	if (i915_ggtt_has_aperture(ggtt))
-		obj = i915_gem_object_create_stolen(i915, size);
-	if (IS_ERR(obj))
-		obj = i915_gem_object_create_internal(i915, size);
+	if (HAS_LMEM(i915)) {
+		obj = i915_gem_object_create_lmem(i915, size,
+						  I915_BO_ALLOC_CONTIGUOUS |
+						  I915_BO_ALLOC_VOLATILE);
+	} else {
+		if (i915_ggtt_has_aperture(ggtt))
+			obj = i915_gem_object_create_stolen(i915, size);
+		if (IS_ERR(obj))
+			obj = i915_gem_object_create_internal(i915, size);
+	}
 	if (IS_ERR(obj))
 		return ERR_CAST(obj);
 
Signature · 6 lines
2.26.2

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel
#96 lore raw

RFC PATCH 92/162 drm/i915/uapi: introduce drm_i915_gem_create_ext

In reply to Matthew Auld

Same old gem_create but with now with extensions support. This is needed to support various upcoming usecases. For now we use the extensions mechanism to support setting an immutable-priority-list of potential placements, at creation time.

If we wish to set the placements/regions we can simply do:

struct drm_i915_gem_object_param region_param = { … }; /* Unchanged */ struct drm_i915_gem_create_ext_setparam setparam_region = {
.base = { .name = I915_GEM_CREATE_EXT_SETPARAM }, .param = region_param,
}

struct drm_i915_gem_create_ext create_ext = {
	.size = 16 * PAGE_SIZE,
	.extensions = (uintptr_t)&setparam_region,
};

int err = ioctl(fd, DRM_IOCTL_I915_GEM_CREATE_EXT, &create_ext); if (err) ...

If we use the normal gem_create or gem_create_ext without the extensions/placements then we still get the old behaviour with only placing the object in system memory.

One important change here is the returned size will now be rounded up to the correct size, depending on the list of placements, where we might have minimum page-size restrictions on some platforms when dealing with device local-memory.

Also, we still keep around the i915_gem_object_setparam ioctl, although that is now restricted by the placement list(i.e we are not allowed to add new placements), and longer term that will be going away wrt setting placements, since it was deemed that the kernel doesn't need to support a dynamic list of placements, which is now solidified by this uapi change.

Testcase: igt/gem_create/create-ext-placement-sanity-check
Testcase: igt/gem_create/create-ext-placement-each
Testcase: igt/gem_create/create-ext-placement-all

---
 drivers/gpu/drm/i915/Makefile                 |   1 +
 drivers/gpu/drm/i915/gem/i915_gem_create.c    | 398 ++++++++++++++++++
 drivers/gpu/drm/i915/gem/i915_gem_object.c    |   2 +
 .../gpu/drm/i915/gem/i915_gem_object_types.h  |   9 +
 drivers/gpu/drm/i915/gem/i915_gem_region.c    |   4 +
 drivers/gpu/drm/i915/i915_drv.c               |   2 +-
 drivers/gpu/drm/i915/i915_gem.c               | 103 +----
 drivers/gpu/drm/i915/intel_memory_region.c    |  20 +
 drivers/gpu/drm/i915/intel_memory_region.h    |   4 +
 include/uapi/drm/i915_drm.h                   |  60 +++
 10 files changed, 500 insertions(+), 103 deletions(-)
 create mode 100644 drivers/gpu/drm/i915/gem/i915_gem_create.c
drivers/gpu/drm/i915/Makefile +1 -0
index ec361d61230b..3955134feca7 100644
@@ -134,6 +134,7 @@ gem-y += \
 	gem/i915_gem_clflush.o \
 	gem/i915_gem_client_blt.o \
 	gem/i915_gem_context.o \
+	gem/i915_gem_create.o \
 	gem/i915_gem_dmabuf.o \
 	gem/i915_gem_domain.o \
 	gem/i915_gem_execbuffer.o \
drivers/gpu/drm/i915/gem/i915_gem_create.c new file +398 -0

398 lines not on this page · raw message

drivers/gpu/drm/i915/gem/i915_gem_object.c +2 -0
index 49935245a4a8..89b530841126 100644
@@ -254,6 +254,8 @@ static void __i915_gem_free_objects(struct drm_i915_private *i915,
 		if (obj->ops->release)
 			obj->ops->release(obj);
 
+		kfree(obj->mm.placements);
+
 		/* But keep the pointer alive for RCU-protected lookups */
 		call_rcu(&obj->rcu, __i915_gem_free_object_rcu);
 		cond_resched();
drivers/gpu/drm/i915/gem/i915_gem_object_types.h +9 -0
index 6d101275bc9d..115ad32c303f 100644
@@ -212,6 +212,15 @@ struct drm_i915_gem_object {
 		atomic_t pages_pin_count;
 		atomic_t shrink_pin;
 
+		/**
+		 * Priority list of potential placements for this object.
+		 */
+		struct intel_memory_region **placements;
+		int n_placements;
+
+		/* XXX: Nasty hack, see gem_create */
+		int gem_create_posted_err;
+
 		/**
 		 * Memory region for this object.
 		 */
drivers/gpu/drm/i915/gem/i915_gem_region.c +4 -0
index 58bf5f9e3199..8f352ba6202d 100644
@@ -33,6 +33,10 @@ i915_gem_object_get_pages_buddy(struct drm_i915_gem_object *obj)
 	unsigned int sg_page_sizes;
 	int ret;
 
+	/* XXX: Check if we have any post. This is nasty hack, see gem_create */
+	if (obj->mm.gem_create_posted_err)
+		return obj->mm.gem_create_posted_err;
+
 	st = kmalloc(sizeof(*st), GFP_KERNEL);
 	if (!st)
 		return -ENOMEM;
drivers/gpu/drm/i915/i915_drv.c +1 -1
index 07b3a89ec09e..f4540c048cd9 100644
@@ -1729,7 +1729,7 @@ static const struct drm_ioctl_desc i915_ioctls[] = {
 	DRM_IOCTL_DEF_DRV(I915_GEM_THROTTLE, i915_gem_throttle_ioctl, DRM_RENDER_ALLOW),
 	DRM_IOCTL_DEF_DRV(I915_GEM_ENTERVT, drm_noop, DRM_AUTH|DRM_MASTER|DRM_ROOT_ONLY),
 	DRM_IOCTL_DEF_DRV(I915_GEM_LEAVEVT, drm_noop, DRM_AUTH|DRM_MASTER|DRM_ROOT_ONLY),
-	DRM_IOCTL_DEF_DRV(I915_GEM_CREATE, i915_gem_create_ioctl, DRM_RENDER_ALLOW),
+	DRM_IOCTL_DEF_DRV(I915_GEM_CREATE_EXT, i915_gem_create_ioctl, DRM_RENDER_ALLOW),
 	DRM_IOCTL_DEF_DRV(I915_GEM_PREAD, i915_gem_pread_ioctl, DRM_RENDER_ALLOW),
 	DRM_IOCTL_DEF_DRV(I915_GEM_PWRITE, i915_gem_pwrite_ioctl, DRM_RENDER_ALLOW),
 	DRM_IOCTL_DEF_DRV(I915_GEM_MMAP, i915_gem_mmap_ioctl, DRM_RENDER_ALLOW),
drivers/gpu/drm/i915/i915_gem.c +1 -102

115 lines not on this page · raw message

drivers/gpu/drm/i915/intel_memory_region.c +20 -0
index 6f40748901da..67240bddf2ca 100644
@@ -21,6 +21,26 @@ const struct intel_memory_region_info intel_region_map[] = {
        },
 };
 
+struct intel_memory_region *
+intel_memory_region_lookup(struct drm_i915_private *i915,
+			   u16 class, u16 instance)
+{
+	int i;
+
+	/* XXX: consider maybe converting to an rb tree at some point */
+	for (i = 0; i < ARRAY_SIZE(i915->mm.regions); ++i) {
+		struct intel_memory_region *region = i915->mm.regions[i];
+
+		if (!region)
+			continue;
+
+		if (region->type == class && region->instance == instance)
+			return region;
+	}
+
+	return NULL;
+}
+
 struct intel_memory_region *
 intel_memory_region_by_type(struct drm_i915_private *i915,
 			    enum intel_memory_type mem_type)
drivers/gpu/drm/i915/intel_memory_region.h +4 -0
index 15dcb57b4b5a..20431d3ce490 100644
@@ -102,6 +102,10 @@ struct intel_memory_region {
 	} objects;
 };
 
+struct intel_memory_region *
+intel_memory_region_lookup(struct drm_i915_private *i915,
+			   u16 class, u16 instance);
+
 int intel_memory_region_init_buddy(struct intel_memory_region *mem);
 void intel_memory_region_release_buddy(struct intel_memory_region *mem);
 
include/uapi/drm/i915_drm.h +60 -0

78 lines not on this page · raw message

Signature · 6 lines
2.26.2

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel
#104 lore raw

RFC PATCH 94/162 drm/i915/dg1: Do not check r->sgt.pfn for NULL

In reply to Matthew Auld

From: Kui Wen <[email protected]>

When user space does mmap, kernel would map the physical page of local memory to virtual memory address. The r->sgt.pfn is page address allocated from local memory and the local memory region is from 0 to LMEM size. Hence the r->sgt.pfn is possible to be 0, but this's normal case.

---
 drivers/gpu/drm/i915/i915_mm.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/gpu/drm/i915/i915_mm.c +1 -1
index 43039dc8c607..dcf6b3e5bfdf 100644
@@ -62,7 +62,7 @@ static int remap_sg(pte_t *pte, unsigned long addr, void *data)
 {
 	struct remap_pfn *r = data;
 
-	if (GEM_WARN_ON(!r->sgt.pfn))
+	if (GEM_WARN_ON(!use_dma(r->iobase) && !r->sgt.pfn))
 		return -EINVAL;
 
 	/* Special PTE are not associated with any struct page */
Signature · 6 lines
2.26.2

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel
#105 lore raw

RFC PATCH 95/162 drm/i915/dg1: Introduce dmabuf mmap to LMEM

In reply to Matthew Auld

From: "Michael J. Ruhl" <[email protected]>

The i915 GEM dmabuf mmap interface assumes all BOs are SHMEM. When the BO is backed by LMEM, this assumption doesn't work so well.

Introduce the dmabuf mmap interface to LMEM by adding the appropriate VMA faulting mechanism and update dmabuf to allow for LMEM backed BOs by leveraging the gem_mman path.

---
 drivers/gpu/drm/i915/gem/i915_gem_dmabuf.c    |  59 +++++++---
 drivers/gpu/drm/i915/gem/i915_gem_mman.c      | 102 ++++++++++--------
 drivers/gpu/drm/i915/gem/i915_gem_mman.h      |   9 ++
 .../drm/i915/gem/selftests/i915_gem_mman.c    |  12 +--
 4 files changed, 118 insertions(+), 64 deletions(-)
drivers/gpu/drm/i915/gem/i915_gem_dmabuf.c +47 -12
index 018d02cc4af5..85528eeaacbc 100644
@@ -10,6 +10,7 @@
 
 #include "i915_drv.h"
 #include "i915_gem_lmem.h"
+#include "i915_gem_mman.h"
 #include "i915_gem_object.h"
 #include "i915_scatterlist.h"
 
@@ -105,7 +106,41 @@ static void i915_gem_dmabuf_vunmap(struct dma_buf *dma_buf, struct dma_buf_map *
 	i915_gem_object_unpin_map(obj);
 }
 
-static int i915_gem_dmabuf_mmap(struct dma_buf *dma_buf, struct vm_area_struct *vma)
+/**
+ * i915_gem_dmabuf_update_vma - Setup VMA information for exported LMEM
+ * objects
+ * @obj: valid LMEM object
+ * @vma: va;od vma
+ *
+ * NOTE: on success, the final _object_put() will be done by the VMA
+ * vm_close() callback.
+ */
+static int i915_gem_dmabuf_update_vma(struct drm_i915_gem_object *obj,
+				      struct vm_area_struct *vma)
+{
+	struct i915_mmap_offset *mmo;
+	int err;
+
+	i915_gem_object_get(obj);
+	mmo = i915_gem_mmap_offset_attach(obj, I915_MMAP_TYPE_WC, NULL);
+	if (IS_ERR(mmo)) {
+		err = PTR_ERR(mmo);
+		goto out;
+	}
+
+	err = i915_gem_update_vma_info(obj, mmo, vma);
+	if (err)
+		goto out;
+
+	return 0;
+
+out:
+	i915_gem_object_put(obj);
+	return err;
+}
+
+static int i915_gem_dmabuf_mmap(struct dma_buf *dma_buf,
+				struct vm_area_struct *vma)
 {
 	struct drm_i915_gem_object *obj = dma_buf_to_obj(dma_buf);
 	int ret;
@@ -113,16 +148,20 @@ static int i915_gem_dmabuf_mmap(struct dma_buf *dma_buf, struct vm_area_struct *
 	if (obj->base.size < vma->vm_end - vma->vm_start)
 		return -EINVAL;
 
-	if (!obj->base.filp)
-		return -ENODEV;
+	/* shmem */
+	if (obj->base.filp) {
+		ret = call_mmap(obj->base.filp, vma);
+		if (ret)
+			return ret;
 
-	ret = call_mmap(obj->base.filp, vma);
-	if (ret)
-		return ret;
+		vma_set_file(vma, obj->base.filp);
+		return 0;
+	}
 
-	vma_set_file(vma, obj->base.filp);
+	if (i915_gem_object_is_lmem(obj))
+		return i915_gem_dmabuf_update_vma(obj, vma);
 
-	return 0;
+	return -ENODEV;
 }
 
 static int i915_gem_begin_cpu_access(struct dma_buf *dma_buf, enum dma_data_direction direction)
@@ -254,10 +293,6 @@ struct drm_gem_object *i915_gem_prime_import(struct drm_device *dev,
 			 */
 			return &i915_gem_object_get(obj)->base;
 		}
-
-		/* not our device, but still a i915 object? */
-		if (i915_gem_object_is_lmem(obj))
-			return ERR_PTR(-ENOTSUPP);
 	}
 
 	/* need to attach */
drivers/gpu/drm/i915/gem/i915_gem_mman.c +56 -46

135 lines not on this page · raw message

drivers/gpu/drm/i915/gem/i915_gem_mman.h +9 -0
index 7c5ccdf59359..dfd19da0b3e7 100644
@@ -10,6 +10,8 @@
 #include <linux/mm_types.h>
 #include <linux/types.h>
 
+#include "gem/i915_gem_object_types.h"
+
 struct drm_device;
 struct drm_file;
 struct drm_i915_gem_object;
@@ -31,4 +33,11 @@ void i915_gem_object_release_mmap_gtt(struct drm_i915_gem_object *obj);
 
 void i915_gem_object_release_mmap_offset(struct drm_i915_gem_object *obj);
 
+struct i915_mmap_offset *
+i915_gem_mmap_offset_attach(struct drm_i915_gem_object *obj,
+			    enum i915_mmap_type mmap_type,
+			    struct drm_file *file);
+int i915_gem_update_vma_info(struct drm_i915_gem_object *obj,
+			     struct i915_mmap_offset *mmo,
+			     struct vm_area_struct *vma);
 #endif
drivers/gpu/drm/i915/gem/selftests/i915_gem_mman.c +6 -6
index 85fff8bed08c..5701549b5d13 100644
@@ -583,7 +583,7 @@ static bool assert_mmap_offset(struct drm_i915_private *i915,
 	if (IS_ERR(obj))
 		return false;
 
-	mmo = mmap_offset_attach(obj, I915_MMAP_OFFSET_GTT, NULL);
+	mmo = i915_gem_mmap_offset_attach(obj, I915_MMAP_OFFSET_GTT, NULL);
 	i915_gem_object_put(obj);
 
 	return PTR_ERR_OR_ZERO(mmo) == expected;
@@ -686,7 +686,7 @@ static int igt_mmap_offset_exhaustion(void *arg)
 		goto out;
 	}
 
-	mmo = mmap_offset_attach(obj, I915_MMAP_OFFSET_GTT, NULL);
+	mmo = i915_gem_mmap_offset_attach(obj, I915_MMAP_OFFSET_GTT, NULL);
 	if (IS_ERR(mmo)) {
 		pr_err("Unable to insert object into reclaimed hole\n");
 		err = PTR_ERR(mmo);
@@ -860,7 +860,7 @@ static int __igt_mmap(struct drm_i915_private *i915,
 	if (err)
 		return err;
 
-	mmo = mmap_offset_attach(obj, type, NULL);
+	mmo = i915_gem_mmap_offset_attach(obj, type, NULL);
 	if (IS_ERR(mmo))
 		return PTR_ERR(mmo);
 
@@ -996,7 +996,7 @@ static int __igt_mmap_access(struct drm_i915_private *i915,
 	if (!can_mmap(obj, type) || !can_access(obj))
 		return 0;
 
-	mmo = mmap_offset_attach(obj, type, NULL);
+	mmo = i915_gem_mmap_offset_attach(obj, type, NULL);
 	if (IS_ERR(mmo))
 		return PTR_ERR(mmo);
 
@@ -1109,7 +1109,7 @@ static int __igt_mmap_gpu(struct drm_i915_private *i915,
 	if (err)
 		return err;
 
-	mmo = mmap_offset_attach(obj, type, NULL);
+	mmo = i915_gem_mmap_offset_attach(obj, type, NULL);
 	if (IS_ERR(mmo))
 		return PTR_ERR(mmo);
 
@@ -1285,7 +1285,7 @@ static int __igt_mmap_revoke(struct drm_i915_private *i915,
 	if (!can_mmap(obj, type))
 		return 0;
 
-	mmo = mmap_offset_attach(obj, type, NULL);
+	mmo = i915_gem_mmap_offset_attach(obj, type, NULL);
 	if (IS_ERR(mmo))
 		return PTR_ERR(mmo);
 
Signature · 6 lines
2.26.2

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
#108 lore raw

RFC PATCH 97/162 drm/i915: Distinction of memory regions

In reply to Matthew Auld

From: Zbigniew Kempczyński <[email protected]>

IGTs should be able to choose testing strategy depending on memory regions and its sizes. Add region instance number to make this easier and descriptive.

Cc Matthew Auld <[email protected]>
Cc Ramalingam C <[email protected]>
Cc Tvrtko Ursulin <[email protected]>
Cc Adam Miszczak <[email protected]>
---
 drivers/gpu/drm/i915/intel_memory_region.c | 4 ++++
 1 file changed, 4 insertions(+)
drivers/gpu/drm/i915/intel_memory_region.c +4 -0
index 1f26bc06ec20..cea44ddebe46 100644
@@ -329,6 +329,10 @@ int intel_memory_regions_hw_probe(struct drm_i915_private *i915)
 		mem->instance = instance;
 		mem->gt = &i915->gt;
 
+		if (HAS_LMEM(mem->i915) && type != INTEL_MEMORY_SYSTEM)
+			intel_memory_region_set_name(mem, "%s%u",
+						     mem->name, mem->instance);
+
 		i915->mm.regions[i] = mem;
 	}
 
Signature · 6 lines
2.26.2

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
#110 lore raw

RFC PATCH 98/162 drm/i915/gtt: map the PD up front

In reply to Matthew Auld

We need to general our accessor for the page directories and tables from using the simple kmap_atomic to support local memory, and this setup must be done on acquisition of the backing storage prior to entering fence execution contexts. Here we replace the kmap with the object maping code that for simple single page shmemfs object will return a plain kmap, that is then kept for the lifetime of the page directory.

---
 .../drm/i915/gem/selftests/i915_gem_context.c | 11 +----
 drivers/gpu/drm/i915/gt/gen6_ppgtt.c          | 11 ++---
 drivers/gpu/drm/i915/gt/gen8_ppgtt.c          | 26 ++++------
 drivers/gpu/drm/i915/gt/intel_ggtt.c          |  2 +-
 drivers/gpu/drm/i915/gt/intel_gtt.c           | 48 +++++++++----------
 drivers/gpu/drm/i915/gt/intel_gtt.h           | 11 +++--
 drivers/gpu/drm/i915/gt/intel_ppgtt.c         |  7 ++-
 drivers/gpu/drm/i915/i915_vma.c               |  3 +-
 drivers/gpu/drm/i915/selftests/i915_gem_gtt.c | 10 ++--
 drivers/gpu/drm/i915/selftests/i915_perf.c    |  3 +-
 10 files changed, 54 insertions(+), 78 deletions(-)
drivers/gpu/drm/i915/gem/selftests/i915_gem_context.c +2 -9
index 5fef592390cb..ce70d0a3afb2 100644
@@ -1740,7 +1740,6 @@ static int read_from_scratch(struct i915_gem_context *ctx,
 static int check_scratch_page(struct i915_gem_context *ctx, u32 *out)
 {
 	struct i915_address_space *vm;
-	struct page *page;
 	u32 *vaddr;
 	int err = 0;
 
@@ -1748,24 +1747,18 @@ static int check_scratch_page(struct i915_gem_context *ctx, u32 *out)
 	if (!vm)
 		return -ENODEV;
 
-	page = __px_page(vm->scratch[0]);
-	if (!page) {
+	if (!vm->scratch[0]) {
 		pr_err("No scratch page!\n");
 		return -EINVAL;
 	}
 
-	vaddr = kmap(page);
-	if (!vaddr) {
-		pr_err("No (mappable) scratch page!\n");
-		return -EINVAL;
-	}
+	vaddr = __px_vaddr(vm->scratch[0]);
 
 	memcpy(out, vaddr, sizeof(*out));
 	if (memchr_inv(vaddr, *out, PAGE_SIZE)) {
 		pr_err("Inconsistent initial state of scratch page!\n");
 		err = -EINVAL;
 	}
-	kunmap(page);
 
 	return err;
 }
drivers/gpu/drm/i915/gt/gen6_ppgtt.c +4 -7
index 680bd9442eb0..78ad7d8a8bcc 100644
@@ -105,9 +105,8 @@ static void gen6_ppgtt_clear_range(struct i915_address_space *vm,
 		 * entries back to scratch.
 		 */
 
-		vaddr = kmap_atomic_px(pt);
+		vaddr = px_vaddr(pt);
 		memset32(vaddr + pte, scratch_pte, count);
-		kunmap_atomic(vaddr);
 
 		pte = 0;
 	}
@@ -129,7 +128,7 @@ static void gen6_ppgtt_insert_entries(struct i915_address_space *vm,
 
 	GEM_BUG_ON(!pd->entry[act_pt]);
 
-	vaddr = kmap_atomic_px(i915_pt_entry(pd, act_pt));
+	vaddr = px_vaddr(i915_pt_entry(pd, act_pt));
 	do {
 		GEM_BUG_ON(sg_dma_len(iter.sg) < I915_GTT_PAGE_SIZE);
 		vaddr[act_pte] = pte_encode | GEN6_PTE_ADDR_ENCODE(iter.dma);
@@ -145,12 +144,10 @@ static void gen6_ppgtt_insert_entries(struct i915_address_space *vm,
 		}
 
 		if (++act_pte == GEN6_PTES) {
-			kunmap_atomic(vaddr);
-			vaddr = kmap_atomic_px(i915_pt_entry(pd, ++act_pt));
+			vaddr = px_vaddr(i915_pt_entry(pd, ++act_pt));
 			act_pte = 0;
 		}
 	} while (1);
-	kunmap_atomic(vaddr);
 
 	vma->page_sizes.gtt = I915_GTT_PAGE_SIZE;
 }
@@ -244,7 +241,7 @@ static int gen6_ppgtt_init_scratch(struct gen6_ppgtt *ppgtt)
 		goto err_scratch0;
 	}
 
-	ret = pin_pt_dma(vm, vm->scratch[1]);
+	ret = map_pt_dma(vm, vm->scratch[1]);
 	if (ret)
 		goto err_scratch1;
 
drivers/gpu/drm/i915/gt/gen8_ppgtt.c +10 -16
index a37c968ef8f7..a3093dd4b86d 100644
@@ -237,11 +237,10 @@ static u64 __gen8_ppgtt_clear(struct i915_address_space * const vm,
 			    atomic_read(&pt->used));
 			GEM_BUG_ON(!count || count >= atomic_read(&pt->used));
 
-			vaddr = kmap_atomic_px(pt);
+			vaddr = px_vaddr(pt);
 			memset64(vaddr + gen8_pd_index(start, 0),
 				 vm->scratch[0]->encode,
 				 count);
-			kunmap_atomic(vaddr);
 
 			atomic_sub(count, &pt->used);
 			start += count;
@@ -370,7 +369,7 @@ gen8_ppgtt_insert_pte(struct i915_ppgtt *ppgtt,
 	gen8_pte_t *vaddr;
 
 	pd = i915_pd_entry(pdp, gen8_pd_index(idx, 2));
-	vaddr = kmap_atomic_px(i915_pt_entry(pd, gen8_pd_index(idx, 1)));
+	vaddr = px_vaddr(i915_pt_entry(pd, gen8_pd_index(idx, 1)));
 	do {
 		GEM_BUG_ON(sg_dma_len(iter->sg) < I915_GTT_PAGE_SIZE);
 		vaddr[gen8_pd_index(idx, 0)] = pte_encode | iter->dma;
@@ -397,12 +396,10 @@ gen8_ppgtt_insert_pte(struct i915_ppgtt *ppgtt,
 			}
 
 			clflush_cache_range(vaddr, PAGE_SIZE);
-			kunmap_atomic(vaddr);
-			vaddr = kmap_atomic_px(i915_pt_entry(pd, gen8_pd_index(idx, 1)));
+			vaddr = px_vaddr(i915_pt_entry(pd, gen8_pd_index(idx, 1)));
 		}
 	} while (1);
 	clflush_cache_range(vaddr, PAGE_SIZE);
-	kunmap_atomic(vaddr);
 
 	return idx;
 }
@@ -437,7 +434,7 @@ static void gen8_ppgtt_insert_huge(struct i915_vma *vma,
 			encode |= GEN8_PDE_PS_2M;
 			page_size = I915_GTT_PAGE_SIZE_2M;
 
-			vaddr = kmap_atomic_px(pd);
+			vaddr = px_vaddr(pd);
 		} else {
 			struct i915_page_table *pt =
 				i915_pt_entry(pd, __gen8_pte_index(start, 1));
@@ -452,7 +449,7 @@ static void gen8_ppgtt_insert_huge(struct i915_vma *vma,
 			     rem >= (I915_PDES - index) * I915_GTT_PAGE_SIZE))
 				maybe_64K = __gen8_pte_index(start, 1);
 
-			vaddr = kmap_atomic_px(pt);
+			vaddr = px_vaddr(pt);
 		}
 
 		do {
@@ -486,7 +483,6 @@ static void gen8_ppgtt_insert_huge(struct i915_vma *vma,
 		} while (rem >= page_size && index < I915_PDES);
 
 		clflush_cache_range(vaddr, PAGE_SIZE);
-		kunmap_atomic(vaddr);
 
 		/*
 		 * Is it safe to mark the 2M block as 64K? -- Either we have
@@ -500,9 +496,8 @@ static void gen8_ppgtt_insert_huge(struct i915_vma *vma,
 		      !iter->sg && IS_ALIGNED(vma->node.start +
 					      vma->node.size,
 					      I915_GTT_PAGE_SIZE_2M)))) {
-			vaddr = kmap_atomic_px(pd);
+			vaddr = px_vaddr(pd);
 			vaddr[maybe_64K] |= GEN8_PDE_IPS_64K;
-			kunmap_atomic(vaddr);
 			page_size = I915_GTT_PAGE_SIZE_64K;
 
 			/*
@@ -518,12 +513,11 @@ static void gen8_ppgtt_insert_huge(struct i915_vma *vma,
 				u16 i;
 
 				encode = vma->vm->scratch[0]->encode;
-				vaddr = kmap_atomic_px(i915_pt_entry(pd, maybe_64K));
+				vaddr = px_vaddr(i915_pt_entry(pd, maybe_64K));
 
 				for (i = 1; i < index; i += 16)
 					memset64(vaddr + i, encode, 15);
 
-				kunmap_atomic(vaddr);
 			}
 		}
 
@@ -592,7 +586,7 @@ static int gen8_init_scratch(struct i915_address_space *vm)
 		if (IS_ERR(obj))
 			goto free_scratch;
 
-		ret = pin_pt_dma(vm, obj);
+		ret = map_pt_dma(vm, obj);
 		if (ret) {
 			i915_gem_object_put(obj);
 			goto free_scratch;
@@ -629,7 +623,7 @@ static int gen8_preallocate_top_level_pdp(struct i915_ppgtt *ppgtt)
 		if (IS_ERR(pde))
 			return PTR_ERR(pde);
 
-		err = pin_pt_dma(vm, pde->pt.base);
+		err = map_pt_dma(vm, pde->pt.base);
 		if (err) {
 			i915_gem_object_put(pde->pt.base);
 			free_pd(vm, pde);
@@ -665,7 +659,7 @@ gen8_alloc_top_pd(struct i915_address_space *vm)
 		goto err_pd;
 	}
 
-	err = pin_pt_dma(vm, pd->pt.base);
+	err = map_pt_dma(vm, pd->pt.base);
 	if (err)
 		goto err_pd;
 
drivers/gpu/drm/i915/gt/intel_ggtt.c +1 -1
index 17ecaef1834d..4560e03067a7 100644
@@ -616,7 +616,7 @@ static int init_aliasing_ppgtt(struct i915_ggtt *ggtt)
 		goto err_ppgtt;
 
 	i915_gem_object_lock(ppgtt->vm.scratch[0], NULL);
-	err = i915_vm_pin_pt_stash(&ppgtt->vm, &stash);
+	err = i915_vm_map_pt_stash(&ppgtt->vm, &stash);
 	i915_gem_object_unlock(ppgtt->vm.scratch[0]);
 	if (err)
 		goto err_stash;
drivers/gpu/drm/i915/gt/intel_gtt.c +22 -26

94 lines not on this page · raw message

drivers/gpu/drm/i915/gt/intel_gtt.h +6 -5
index 16063b2f0119..5b8ea9c8c654 100644
@@ -176,6 +176,9 @@ struct page *__px_page(struct drm_i915_gem_object *p);
 dma_addr_t __px_dma(struct drm_i915_gem_object *p);
 #define px_dma(px) (__px_dma(px_base(px)))
 
+void *__px_vaddr(struct drm_i915_gem_object *p);
+#define px_vaddr(px) (__px_vaddr(px_base(px)))
+
 #define px_pt(px) \
 	__px_choose_expr(px, struct i915_page_table *, __x, \
 	__px_choose_expr(px, struct i915_page_directory *, &__x->pt, \
@@ -506,8 +509,6 @@ struct i915_ppgtt *i915_ppgtt_create(struct intel_gt *gt);
 void i915_ggtt_suspend(struct i915_ggtt *gtt);
 void i915_ggtt_resume(struct i915_ggtt *ggtt);
 
-#define kmap_atomic_px(px) kmap_atomic(__px_page(px_base(px)))
-
 void
 fill_page_dma(struct drm_i915_gem_object *p, const u64 val, unsigned int count);
 
@@ -525,8 +526,8 @@ struct i915_page_table *alloc_pt(struct i915_address_space *vm);
 struct i915_page_directory *alloc_pd(struct i915_address_space *vm);
 struct i915_page_directory *__alloc_pd(int npde);
 
-int pin_pt_dma(struct i915_address_space *vm, struct drm_i915_gem_object *obj);
-int pin_pt_dma_locked(struct i915_address_space *vm, struct drm_i915_gem_object *obj);
+int map_pt_dma(struct i915_address_space *vm, struct drm_i915_gem_object *obj);
+int map_pt_dma_locked(struct i915_address_space *vm, struct drm_i915_gem_object *obj);
 
 void free_px(struct i915_address_space *vm,
 	     struct i915_page_table *pt, int lvl);
@@ -573,7 +574,7 @@ void setup_private_pat(struct intel_uncore *uncore);
 int i915_vm_alloc_pt_stash(struct i915_address_space *vm,
 			   struct i915_vm_pt_stash *stash,
 			   u64 size);
-int i915_vm_pin_pt_stash(struct i915_address_space *vm,
+int i915_vm_map_pt_stash(struct i915_address_space *vm,
 			 struct i915_vm_pt_stash *stash);
 void i915_vm_free_pt_stash(struct i915_address_space *vm,
 			   struct i915_vm_pt_stash *stash);
drivers/gpu/drm/i915/gt/intel_ppgtt.c +3 -4
index f3ac47702aee..8e7b77cc4594 100644
@@ -85,11 +85,10 @@ write_dma_entry(struct drm_i915_gem_object * const pdma,
 		const unsigned short idx,
 		const u64 encoded_entry)
 {
-	u64 * const vaddr = kmap_atomic(__px_page(pdma));
+	u64 * const vaddr = __px_vaddr(pdma);
 
 	vaddr[idx] = encoded_entry;
 	clflush_cache_range(&vaddr[idx], sizeof(u64));
-	kunmap_atomic(vaddr);
 }
 
 void
@@ -254,7 +253,7 @@ int i915_vm_alloc_pt_stash(struct i915_address_space *vm,
 	return 0;
 }
 
-int i915_vm_pin_pt_stash(struct i915_address_space *vm,
+int i915_vm_map_pt_stash(struct i915_address_space *vm,
 			 struct i915_vm_pt_stash *stash)
 {
 	struct i915_page_table *pt;
@@ -262,7 +261,7 @@ int i915_vm_pin_pt_stash(struct i915_address_space *vm,
 
 	for (n = 0; n < ARRAY_SIZE(stash->pt); n++) {
 		for (pt = stash->pt[n]; pt; pt = pt->stash) {
-			err = pin_pt_dma_locked(vm, pt->base);
+			err = map_pt_dma_locked(vm, pt->base);
 			if (err)
 				return err;
 		}
drivers/gpu/drm/i915/i915_vma.c +1 -2
index 7243ab593aec..82f60cc43a90 100644
@@ -914,8 +914,7 @@ int i915_vma_pin_ww(struct i915_vma *vma, struct i915_gem_ww_ctx *ww,
 			if (err)
 				goto err_fence;
 
-			err = i915_vm_pin_pt_stash(vma->vm,
-						   &work->stash);
+			err = i915_vm_map_pt_stash(vma->vm, &work->stash);
 			if (err)
 				goto err_fence;
 		}
drivers/gpu/drm/i915/selftests/i915_gem_gtt.c +4 -6
index d07dd6780005..9653d7c259a5 100644
@@ -185,7 +185,7 @@ static int igt_ppgtt_alloc(void *arg)
 		if (err)
 			goto err_ppgtt_cleanup;
 
-		err = i915_vm_pin_pt_stash(&ppgtt->vm, &stash);
+		err = i915_vm_map_pt_stash(&ppgtt->vm, &stash);
 		if (err) {
 			i915_vm_free_pt_stash(&ppgtt->vm, &stash);
 			goto err_ppgtt_cleanup;
@@ -207,7 +207,7 @@ static int igt_ppgtt_alloc(void *arg)
 		if (err)
 			goto err_ppgtt_cleanup;
 
-		err = i915_vm_pin_pt_stash(&ppgtt->vm, &stash);
+		err = i915_vm_map_pt_stash(&ppgtt->vm, &stash);
 		if (err) {
 			i915_vm_free_pt_stash(&ppgtt->vm, &stash);
 			goto err_ppgtt_cleanup;
@@ -324,11 +324,10 @@ static int lowlevel_hole(struct i915_address_space *vm,
 							   BIT_ULL(size)))
 					goto alloc_vm_end;
 
-				err = i915_vm_pin_pt_stash(vm, &stash);
+				err = i915_vm_map_pt_stash(vm, &stash);
 				if (!err)
 					vm->allocate_va_range(vm, &stash,
 							      addr, BIT_ULL(size));
-
 				i915_vm_free_pt_stash(vm, &stash);
 alloc_vm_end:
 				if (err == -EDEADLK) {
@@ -1966,10 +1965,9 @@ static int igt_cs_tlb(void *arg)
 			if (err)
 				goto end_ww;
 
-			err = i915_vm_pin_pt_stash(vm, &stash);
+			err = i915_vm_map_pt_stash(vm, &stash);
 			if (!err)
 				vm->allocate_va_range(vm, &stash, offset, chunk_size);
-
 			i915_vm_free_pt_stash(vm, &stash);
 end_ww:
 			if (err == -EDEADLK) {
drivers/gpu/drm/i915/selftests/i915_perf.c +1 -2
index debbac660519..6a7abb3e2bb5 100644
@@ -307,7 +307,7 @@ static int live_noa_gpr(void *arg)
 	}
 
 	/* Poison the ce->vm so we detect writes not to the GGTT gt->scratch */
-	scratch = kmap(__px_page(ce->vm->scratch[0]));
+	scratch = __px_vaddr(ce->vm->scratch[0]);
 	memset(scratch, POISON_FREE, PAGE_SIZE);
 
 	rq = intel_context_create_request(ce);
@@ -405,7 +405,6 @@ static int live_noa_gpr(void *arg)
 out_rq:
 	i915_request_put(rq);
 out_ce:
-	kunmap(__px_page(ce->vm->scratch[0]));
 	intel_context_put(ce);
 out:
 	stream_destroy(stream);
Signature · 6 lines
2.26.2

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel
#106 lore raw

RFC PATCH 96/162 drm/i915: setup the LMEM region

In reply to Matthew Auld

Hook up the LMEM region. Addresses will start from zero, and for CPU access we get LMEM_BAR which is just a 1:1 mapping of said region.

Based on a patch from Michel Thierry.

Cc Abdiel Janulgue <[email protected]>
Cc Lucas De Marchi <[email protected]>
Cc Joonas Lahtinen <[email protected]>
Cc Rodrigo Vivi <[email protected]>
---
 drivers/gpu/drm/i915/i915_reg.h            |  3 ++
 drivers/gpu/drm/i915/intel_memory_region.c | 11 ++++++-
 drivers/gpu/drm/i915/intel_region_lmem.c   | 38 ++++++++++++++++++++++
 drivers/gpu/drm/i915/intel_region_lmem.h   |  2 ++
 4 files changed, 53 insertions(+), 1 deletion(-)
drivers/gpu/drm/i915/i915_reg.h +3 -0
index bf9ba1e361bb..1af1966ac461 100644
@@ -12063,6 +12063,9 @@ enum skl_power_gate {
 
 #define GEN12_GLOBAL_MOCS(i)	_MMIO(0x4000 + (i) * 4) /* Global MOCS regs */
 
+#define GEN12_LMEM_CFG_ADDR		_MMIO(0xcf58)
+#define   LMEM_ENABLE			(1 << 31)
+
 /* gamt regs */
 #define GEN8_L3_LRA_1_GPGPU _MMIO(0x4dd4)
 #define   GEN8_L3_LRA_1_GPGPU_DEFAULT_VALUE_BDW  0x67F1427F /* max/min for LRA1/2 */
drivers/gpu/drm/i915/intel_memory_region.c +10 -1
index 67240bddf2ca..1f26bc06ec20 100644
@@ -303,7 +303,16 @@ int intel_memory_regions_hw_probe(struct drm_i915_private *i915)
 			mem = i915_gem_stolen_setup(i915);
 			break;
 		case INTEL_MEMORY_LOCAL:
-			mem = intel_setup_fake_lmem(i915);
+#if IS_ENABLED(CONFIG_DRM_I915_SELFTEST)
+			if (IS_ENABLED(CONFIG_DRM_I915_UNSTABLE_FAKE_LMEM)) {
+				if (INTEL_GEN(i915) >= 9 && i915_selftest.live < 0 &&
+				    i915->params.fake_lmem_start)
+					mem = intel_setup_fake_lmem(i915);
+			}
+#endif
+
+			if (IS_ERR(mem))
+				mem = i915_gem_setup_lmem(i915);
 			break;
 		}
 
drivers/gpu/drm/i915/intel_region_lmem.c +38 -0
index 40d8f1a95df6..e98582c76de1 100644
@@ -136,3 +136,41 @@ intel_setup_fake_lmem(struct drm_i915_private *i915)
 
 	return mem;
 }
+
+static struct intel_memory_region *
+setup_lmem(struct drm_i915_private *dev_priv)
+{
+	struct pci_dev *pdev = dev_priv->drm.pdev;
+	struct intel_memory_region *mem;
+	resource_size_t io_start;
+	resource_size_t size;
+
+	/* Enables Local Memory functionality in GAM */
+	I915_WRITE(GEN12_LMEM_CFG_ADDR, I915_READ(GEN12_LMEM_CFG_ADDR) | LMEM_ENABLE);
+
+	io_start = pci_resource_start(pdev, 2);
+	size = pci_resource_len(pdev, 2);
+
+	mem = intel_memory_region_create(dev_priv,
+					 0,
+					 size,
+					 I915_GTT_PAGE_SIZE_4K,
+					 io_start,
+					 &intel_region_lmem_ops);
+	if (!IS_ERR(mem)) {
+		DRM_INFO("Intel graphics LMEM: %pR\n", &mem->region);
+		DRM_INFO("Intel graphics LMEM IO start: %llx\n",
+			 (u64)mem->io_start);
+		DRM_INFO("Intel graphics LMEM size: %llx\n",
+			 (u64)size);
+	}
+
+	return mem;
+}
+
+struct intel_memory_region *
+i915_gem_setup_lmem(struct drm_i915_private *i915)
+{
+	return setup_lmem(i915);
+}
+
drivers/gpu/drm/i915/intel_region_lmem.h +2 -0
index 213def7c7b8a..054e729035c1 100644
@@ -10,6 +10,8 @@ struct drm_i915_private;
 
 extern const struct intel_memory_region_ops intel_region_lmem_ops;
 
+struct intel_memory_region *i915_gem_setup_lmem(struct drm_i915_private *i915);
+
 struct intel_memory_region *
 intel_setup_fake_lmem(struct drm_i915_private *i915);
 
Signature · 6 lines
2.26.2

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
#115 lore raw

RFC PATCH 100/162 drm/i915/gtt: make flushing conditional

In reply to Matthew Auld

Now that PDs can also be mapped as WC, we can forgo all the flushing for such mappings.

---
 .../drm/i915/gem/selftests/i915_gem_context.c |  2 +-
 drivers/gpu/drm/i915/gt/gen6_ppgtt.c          |  6 ++---
 drivers/gpu/drm/i915/gt/gen8_ppgtt.c          | 26 ++++++++++++-------
 drivers/gpu/drm/i915/gt/intel_gtt.c           | 20 ++++++++++----
 drivers/gpu/drm/i915/gt/intel_gtt.h           |  4 +--
 drivers/gpu/drm/i915/gt/intel_ppgtt.c         |  6 +++--
 drivers/gpu/drm/i915/selftests/i915_perf.c    |  2 +-
 7 files changed, 42 insertions(+), 24 deletions(-)
drivers/gpu/drm/i915/gem/selftests/i915_gem_context.c +1 -1
index ce70d0a3afb2..e52cc74db2b1 100644
@@ -1752,7 +1752,7 @@ static int check_scratch_page(struct i915_gem_context *ctx, u32 *out)
 		return -EINVAL;
 	}
 
-	vaddr = __px_vaddr(vm->scratch[0]);
+	vaddr = __px_vaddr(vm->scratch[0], NULL);
 
 	memcpy(out, vaddr, sizeof(*out));
 	if (memchr_inv(vaddr, *out, PAGE_SIZE)) {
drivers/gpu/drm/i915/gt/gen6_ppgtt.c +3 -3
index 78ad7d8a8bcc..8d12e9334861 100644
@@ -105,7 +105,7 @@ static void gen6_ppgtt_clear_range(struct i915_address_space *vm,
 		 * entries back to scratch.
 		 */
 
-		vaddr = px_vaddr(pt);
+		vaddr = px_vaddr(pt, NULL);
 		memset32(vaddr + pte, scratch_pte, count);
 
 		pte = 0;
@@ -128,7 +128,7 @@ static void gen6_ppgtt_insert_entries(struct i915_address_space *vm,
 
 	GEM_BUG_ON(!pd->entry[act_pt]);
 
-	vaddr = px_vaddr(i915_pt_entry(pd, act_pt));
+	vaddr = px_vaddr(i915_pt_entry(pd, act_pt), NULL);
 	do {
 		GEM_BUG_ON(sg_dma_len(iter.sg) < I915_GTT_PAGE_SIZE);
 		vaddr[act_pte] = pte_encode | GEN6_PTE_ADDR_ENCODE(iter.dma);
@@ -144,7 +144,7 @@ static void gen6_ppgtt_insert_entries(struct i915_address_space *vm,
 		}
 
 		if (++act_pte == GEN6_PTES) {
-			vaddr = px_vaddr(i915_pt_entry(pd, ++act_pt));
+			vaddr = px_vaddr(i915_pt_entry(pd, ++act_pt), NULL);
 			act_pte = 0;
 		}
 	} while (1);
drivers/gpu/drm/i915/gt/gen8_ppgtt.c +16 -10
index f67e0332ccbc..e2f1dfc48d43 100644
@@ -237,7 +237,7 @@ static u64 __gen8_ppgtt_clear(struct i915_address_space * const vm,
 			    atomic_read(&pt->used));
 			GEM_BUG_ON(!count || count >= atomic_read(&pt->used));
 
-			vaddr = px_vaddr(pt);
+			vaddr = px_vaddr(pt, NULL);
 			memset64(vaddr + gen8_pd_index(start, 0),
 				 vm->scratch[0]->encode,
 				 count);
@@ -367,9 +367,10 @@ gen8_ppgtt_insert_pte(struct i915_ppgtt *ppgtt,
 	struct i915_page_directory *pd;
 	const gen8_pte_t pte_encode = gen8_pte_encode(0, cache_level, flags);
 	gen8_pte_t *vaddr;
+	bool needs_flush;
 
 	pd = i915_pd_entry(pdp, gen8_pd_index(idx, 2));
-	vaddr = px_vaddr(i915_pt_entry(pd, gen8_pd_index(idx, 1)));
+	vaddr = px_vaddr(i915_pt_entry(pd, gen8_pd_index(idx, 1)), &needs_flush);
 	do {
 		GEM_BUG_ON(sg_dma_len(iter->sg) < I915_GTT_PAGE_SIZE);
 		vaddr[gen8_pd_index(idx, 0)] = pte_encode | iter->dma;
@@ -395,11 +396,14 @@ gen8_ppgtt_insert_pte(struct i915_ppgtt *ppgtt,
 				pd = pdp->entry[gen8_pd_index(idx, 2)];
 			}
 
-			clflush_cache_range(vaddr, PAGE_SIZE);
-			vaddr = px_vaddr(i915_pt_entry(pd, gen8_pd_index(idx, 1)));
+			if (needs_flush)
+				clflush_cache_range(vaddr, PAGE_SIZE);
+			vaddr = px_vaddr(i915_pt_entry(pd, gen8_pd_index(idx, 1)),
+					 &needs_flush);
 		}
 	} while (1);
-	clflush_cache_range(vaddr, PAGE_SIZE);
+	if (needs_flush)
+		clflush_cache_range(vaddr, PAGE_SIZE);
 
 	return idx;
 }
@@ -412,6 +416,7 @@ static void gen8_ppgtt_insert_huge(struct i915_vma *vma,
 	const gen8_pte_t pte_encode = gen8_pte_encode(0, cache_level, flags);
 	unsigned int rem = sg_dma_len(iter->sg);
 	u64 start = vma->node.start;
+	bool needs_flush;
 
 	GEM_BUG_ON(!i915_vm_is_4lvl(vma->vm));
 
@@ -434,7 +439,7 @@ static void gen8_ppgtt_insert_huge(struct i915_vma *vma,
 			encode |= GEN8_PDE_PS_2M;
 			page_size = I915_GTT_PAGE_SIZE_2M;
 
-			vaddr = px_vaddr(pd);
+			vaddr = px_vaddr(pd, &needs_flush);
 		} else {
 			struct i915_page_table *pt =
 				i915_pt_entry(pd, __gen8_pte_index(start, 1));
@@ -449,7 +454,7 @@ static void gen8_ppgtt_insert_huge(struct i915_vma *vma,
 			     rem >= (I915_PDES - index) * I915_GTT_PAGE_SIZE))
 				maybe_64K = __gen8_pte_index(start, 1);
 
-			vaddr = px_vaddr(pt);
+			vaddr = px_vaddr(pt, &needs_flush);
 		}
 
 		do {
@@ -482,7 +487,8 @@ static void gen8_ppgtt_insert_huge(struct i915_vma *vma,
 			}
 		} while (rem >= page_size && index < I915_PDES);
 
-		clflush_cache_range(vaddr, PAGE_SIZE);
+		if (needs_flush)
+			clflush_cache_range(vaddr, PAGE_SIZE);
 
 		/*
 		 * Is it safe to mark the 2M block as 64K? -- Either we have
@@ -496,7 +502,7 @@ static void gen8_ppgtt_insert_huge(struct i915_vma *vma,
 		      !iter->sg && IS_ALIGNED(vma->node.start +
 					      vma->node.size,
 					      I915_GTT_PAGE_SIZE_2M)))) {
-			vaddr = px_vaddr(pd);
+			vaddr = px_vaddr(pd, NULL);
 			vaddr[maybe_64K] |= GEN8_PDE_IPS_64K;
 			page_size = I915_GTT_PAGE_SIZE_64K;
 
@@ -513,7 +519,7 @@ static void gen8_ppgtt_insert_huge(struct i915_vma *vma,
 				u16 i;
 
 				encode = vma->vm->scratch[0]->encode;
-				vaddr = px_vaddr(i915_pt_entry(pd, maybe_64K));
+				vaddr = px_vaddr(i915_pt_entry(pd, maybe_64K), NULL);
 
 				for (i = 1; i < index; i += 16)
 					memset64(vaddr + i, encode, 15);
drivers/gpu/drm/i915/gt/intel_gtt.c +15 -5
index 2605bfd39a15..eee8338e330b 100644
@@ -176,12 +176,19 @@ void clear_pages(struct i915_vma *vma)
 	memset(&vma->page_sizes, 0, sizeof(vma->page_sizes));
 }
 
-void *__px_vaddr(struct drm_i915_gem_object *p)
+void *__px_vaddr(struct drm_i915_gem_object *p, bool *needs_flush)
 {
 	enum i915_map_type type;
+	void *vaddr;
 
 	GEM_BUG_ON(!i915_gem_object_has_pages(p));
-	return page_unpack_bits(p->mm.mapping, &type);
+
+	vaddr = page_unpack_bits(p->mm.mapping, &type);
+
+	if (needs_flush)
+		*needs_flush = type != I915_MAP_WC;
+
+	return vaddr;
 }
 
 dma_addr_t __px_dma(struct drm_i915_gem_object *p)
@@ -199,15 +206,18 @@ struct page *__px_page(struct drm_i915_gem_object *p)
 void
 fill_page_dma(struct drm_i915_gem_object *p, const u64 val, unsigned int count)
 {
-	void *vaddr = __px_vaddr(p);
+	bool needs_flush;
+	void *vaddr;
 
+	vaddr = __px_vaddr(p, &needs_flush);
 	memset64(vaddr, val, count);
-	clflush_cache_range(vaddr, PAGE_SIZE);
+	if (needs_flush)
+		clflush_cache_range(vaddr, PAGE_SIZE);
 }
 
 static void poison_scratch_page(struct drm_i915_gem_object *scratch)
 {
-	void *vaddr = __px_vaddr(scratch);
+	void *vaddr = __px_vaddr(scratch, NULL);
 	u8 val;
 
 	val = 0;
drivers/gpu/drm/i915/gt/intel_gtt.h +2 -2
index bdbdfded60cc..d96bd19d1b47 100644
@@ -176,8 +176,8 @@ struct page *__px_page(struct drm_i915_gem_object *p);
 dma_addr_t __px_dma(struct drm_i915_gem_object *p);
 #define px_dma(px) (__px_dma(px_base(px)))
 
-void *__px_vaddr(struct drm_i915_gem_object *p);
-#define px_vaddr(px) (__px_vaddr(px_base(px)))
+void *__px_vaddr(struct drm_i915_gem_object *p, bool *needs_flush);
+#define px_vaddr(px, needs_flush) (__px_vaddr(px_base(px), needs_flush))
 
 #define px_pt(px) \
 	__px_choose_expr(px, struct i915_page_table *, __x, \
drivers/gpu/drm/i915/gt/intel_ppgtt.c +4 -2
index 8e7b77cc4594..2d74ae950e4b 100644
@@ -85,10 +85,12 @@ write_dma_entry(struct drm_i915_gem_object * const pdma,
 		const unsigned short idx,
 		const u64 encoded_entry)
 {
-	u64 * const vaddr = __px_vaddr(pdma);
+	bool needs_flush;
+	u64 * const vaddr = __px_vaddr(pdma, &needs_flush);
 
 	vaddr[idx] = encoded_entry;
-	clflush_cache_range(&vaddr[idx], sizeof(u64));
+	if (needs_flush)
+		clflush_cache_range(&vaddr[idx], sizeof(u64));
 }
 
 void
drivers/gpu/drm/i915/selftests/i915_perf.c +1 -1
index 6a7abb3e2bb5..6698750ffe8d 100644
@@ -307,7 +307,7 @@ static int live_noa_gpr(void *arg)
 	}
 
 	/* Poison the ce->vm so we detect writes not to the GGTT gt->scratch */
-	scratch = __px_vaddr(ce->vm->scratch[0]);
+	scratch = __px_vaddr(ce->vm->scratch[0], NULL);
 	memset(scratch, POISON_FREE, PAGE_SIZE);
 
 	rq = intel_context_create_request(ce);
Signature · 6 lines
2.26.2

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
#114 lore raw

RFC PATCH 99/162 drm/i915/gtt/dgfx: place the PD in LMEM

In reply to Matthew Auld

It's a requirement that for dgfx we place all the paging structures in device local-memory.

---
 drivers/gpu/drm/i915/gt/gen8_ppgtt.c |  5 ++++-
 drivers/gpu/drm/i915/gt/intel_gtt.c  | 27 +++++++++++++++++++++++++--
 drivers/gpu/drm/i915/gt/intel_gtt.h  |  1 +
 3 files changed, 30 insertions(+), 3 deletions(-)
drivers/gpu/drm/i915/gt/gen8_ppgtt.c +4 -1
index a3093dd4b86d..f67e0332ccbc 100644
@@ -702,7 +702,10 @@ struct i915_ppgtt *gen8_ppgtt_create(struct intel_gt *gt)
 	 */
 	ppgtt->vm.has_read_only = !IS_GEN_RANGE(gt->i915, 11, 12);
 
-	ppgtt->vm.alloc_pt_dma = alloc_pt_dma;
+	if (IS_DGFX(gt->i915))
+		ppgtt->vm.alloc_pt_dma = alloc_pt_lmem;
+	else
+		ppgtt->vm.alloc_pt_dma = alloc_pt_dma;
 
 	err = gen8_init_scratch(&ppgtt->vm);
 	if (err)
drivers/gpu/drm/i915/gt/intel_gtt.c +25 -2
index f3a263f09368..2605bfd39a15 100644
@@ -7,10 +7,23 @@
 
 #include <linux/fault-inject.h>
 
+#include "gem/i915_gem_lmem.h"
 #include "i915_trace.h"
 #include "intel_gt.h"
 #include "intel_gtt.h"
 
+struct drm_i915_gem_object *alloc_pt_lmem(struct i915_address_space *vm, int sz)
+{
+	struct drm_i915_gem_object *obj;
+
+	obj = i915_gem_object_create_lmem(vm->i915, sz, 0);
+
+	/* ensure all dma objects have the same reservation class */
+	if (!IS_ERR(obj))
+		obj->base.resv = &vm->resv;
+	return obj;
+}
+
 struct drm_i915_gem_object *alloc_pt_dma(struct i915_address_space *vm, int sz)
 {
 	struct drm_i915_gem_object *obj;
@@ -27,9 +40,14 @@ struct drm_i915_gem_object *alloc_pt_dma(struct i915_address_space *vm, int sz)
 
 int map_pt_dma(struct i915_address_space *vm, struct drm_i915_gem_object *obj)
 {
+	enum i915_map_type type;
 	void *vaddr;
 
-	vaddr = i915_gem_object_pin_map_unlocked(obj, I915_MAP_WB);
+	type = I915_MAP_WB;
+	if (i915_gem_object_is_lmem(obj))
+		type = I915_MAP_WC;
+
+	vaddr = i915_gem_object_pin_map_unlocked(obj, type);
 	if (IS_ERR(vaddr))
 		return PTR_ERR(vaddr);
 
@@ -39,9 +57,14 @@ int map_pt_dma(struct i915_address_space *vm, struct drm_i915_gem_object *obj)
 
 int map_pt_dma_locked(struct i915_address_space *vm, struct drm_i915_gem_object *obj)
 {
+	enum i915_map_type type;
 	void *vaddr;
 
-	vaddr = i915_gem_object_pin_map(obj, I915_MAP_WB);
+	type = I915_MAP_WB;
+	if (i915_gem_object_is_lmem(obj))
+		type = I915_MAP_WC;
+
+	vaddr = i915_gem_object_pin_map(obj, type);
 	if (IS_ERR(vaddr))
 		return PTR_ERR(vaddr);
 
drivers/gpu/drm/i915/gt/intel_gtt.h +1 -0
index 5b8ea9c8c654..bdbdfded60cc 100644
@@ -522,6 +522,7 @@ int setup_scratch_page(struct i915_address_space *vm);
 void free_scratch(struct i915_address_space *vm);
 
 struct drm_i915_gem_object *alloc_pt_dma(struct i915_address_space *vm, int sz);
+struct drm_i915_gem_object *alloc_pt_lmem(struct i915_address_space *vm, int sz);
 struct i915_page_table *alloc_pt(struct i915_address_space *vm);
 struct i915_page_directory *alloc_pd(struct i915_address_space *vm);
 struct i915_page_directory *__alloc_pd(int npde);
Signature · 6 lines
2.26.2

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel