nouveau/firmware: using dma non-coherent interfaces for fw loading.
Subsystems (3)
Dave Airlie
maintainerPATCH 1/1 nouveau/firmware: using dma non-coherent interfaces for fw loading.
From: Dave Airlie <[email protected]>
Currently, enabling SG_DEBUG in the kernel will cause nouveau to hit a BUG() on startup, when the iommu is enabled:
kernel BUG at include/linux/scatterlist.h:187!
invalid opcode: 0000 [#1] PREEMPT SMP NOPTI
CPU: 7 PID: 930 Comm: (udev-worker) Not tainted 6.9.0-rc3Lyude-Test+ #30 Hardware name: MSI MS-7A39/A320M GAMING PRO (MS-7A39), BIOS 1.I0 01/22/2019 RIP: 0010:sg_init_one+0x85/0xa0 Code: 69 88 32 01 83 e1 03 f6 c3 03 75 20 a8 01 75 1e 48 09 cb 41 89 54 24 08 49 89 1c 24 41 89 6c 24 0c 5b 5d 41 5c e9 7b b9 88 00 <0f> 0b 0f 0b 0f 0b 48 8b 05 5e 46 9a 01 eb b2 66 66 2e 0f 1f 84 00 RSP: 0018:ffffa776017bf6a0 EFLAGS: 00010246 RAX: 0000000000000000 RBX: ffffa77600d87000 RCX: 000000000000002b RDX: 0000000000000001 RSI: 0000000000000000 RDI: ffffa77680d87000 RBP: 000000000000e000 R08: 0000000000000000 R09: 0000000000000000 R10: ffff98f4c46aa508 R11: 0000000000000000 R12: ffff98f4c46aa508 R13: ffff98f4c46aa008 R14: ffffa77600d4a000 R15: ffffa77600d4a018 FS: 00007feeb5aae980(0000) GS:ffff98f5c4dc0000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 00007f22cb9a4520 CR3: 00000001043ba000 CR4: 00000000003506f0 Call Trace: <TASK> ? die+0x36/0x90 ? do_trap+0xdd/0x100 ? sg_init_one+0x85/0xa0 ? do_error_trap+0x65/0x80 ? sg_init_one+0x85/0xa0 ? exc_invalid_op+0x50/0x70 ? sg_init_one+0x85/0xa0 ? asm_exc_invalid_op+0x1a/0x20 ? sg_init_one+0x85/0xa0 nvkm_firmware_ctor+0x14a/0x250 [nouveau] nvkm_falcon_fw_ctor+0x42/0x70 [nouveau] ga102_gsp_booter_ctor+0xb4/0x1a0 [nouveau] r535_gsp_oneinit+0xb3/0x15f0 [nouveau] ? srso_return_thunk+0x5/0x5f ? srso_return_thunk+0x5/0x5f ? nvkm_udevice_new+0x95/0x140 [nouveau] ? srso_return_thunk+0x5/0x5f ? srso_return_thunk+0x5/0x5f ? ktime_get+0x47/0xb0
Fix this by using the non-coherent allocator instead, I think there might be a better answer to this, but it involve ripping up some of APIs using sg lists.
--- drivers/gpu/drm/nouveau/nvkm/core/firmware.c | 11 ++++++----- drivers/gpu/drm/nouveau/nvkm/falcon/fw.c | 6 ++++++ 2 files changed, 12 insertions(+), 5 deletions(-)
drivers/gpu/drm/nouveau/nvkm/core/firmware.c +6 -5
@@ -205,7 +205,8 @@ nvkm_firmware_dtor(struct nvkm_firmware *fw)break ;case NVKM_FIRMWARE_IMG_DMA :nvkm_memory_unref (&memory); -dma_free_coherent (fw->device->dev,sg_dma_len (&fw->mem.sgl), fw->img, fw->phys); +dma_free_noncoherent (fw->device->dev,sg_dma_len (&fw->mem.sgl), + fw->img, fw->phys,DMA_TO_DEVICE );break ;case NVKM_FIRMWARE_IMG_SGT :nvkm_memory_unref (&memory);
@@ -235,14 +236,14 @@ nvkm_firmware_ctor(const struct nvkm_firmware_func *func, const char *name, fw->img =kmemdup (src, fw->len,GFP_KERNEL );break ;case NVKM_FIRMWARE_IMG_DMA : { -dma_addr_t addr; - len =ALIGN (fw->len,PAGE_SIZE ); - fw->img =dma_alloc_coherent (fw->device->dev, len, &addr,GFP_KERNEL ); + fw->img =dma_alloc_noncoherent (fw->device->dev, + len, &fw->phys, +DMA_TO_DEVICE , +GFP_KERNEL );if (fw->img) {memcpy (fw->img, src, fw->len); - fw->phys = addr; }sg_init_one (&fw->mem.sgl, fw->img, len);
drivers/gpu/drm/nouveau/nvkm/falcon/fw.c +6 -0
@@ -89,6 +89,12 @@ nvkm_falcon_fw_boot(struct nvkm_falcon_fw *fw, struct nvkm_subdev *user,nvkm_falcon_fw_dtor_sigs (fw); } +/* after last write to the img, sync dma mappings */ +dma_sync_single_for_device (fw->fw.device->dev, + fw->fw.phys, +sg_dma_len (&fw->fw.mem.sgl), +DMA_TO_DEVICE ); +FLCNFW_DBG (fw,"resetting" ); fw->func->reset (fw);
Signature · 1 line
2.43.2
kernel test robot
build botkernel test robot noticed the following build errors:
Hi Dave,
kernel test robot noticed the following build errors:
[auto build test ERROR on drm-misc/drm-misc-next]
[also build test ERROR on linus/master v6.9 next-20240513] [cannot apply to drm-tip/drm-tip]
[If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]
url: https://github.com/intel-lab-lkp/linux/commits/Dave-Airlie/nouveau-firmware-using-dma-non-coherent-interfaces-for-fw-loading/20240513-144435 base: git://anongit.freedesktop.org/drm/drm-misc drm-misc-next patch link: https://lore.kernel.org/r/20240513064350.1050994-1-airlied%40gmail.com patch subject: [PATCH] nouveau/firmware: using dma non-coherent interfaces for fw loading. config: parisc-defconfig (https://download.01.org/0day-ci/archive/20240513/[email protected]/config) compiler: hppa-linux-gcc (GCC) 13.2.0 reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20240513/[email protected]/reproduce)
If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <[email protected]>
| Closes: https://lore.kernel.org/oe-kbuild-all/[email protected]/
All errors (new ones prefixed by >>):
drivers/gpu/drm/nouveau/nvkm/core/firmware.c: In function 'nvkm_firmware_ctor':
>> drivers/gpu/drm/nouveau/nvkm/core/firmware.c:242:54: error: passing argument 3 of 'dma_alloc_noncoherent' from incompatible pointer type [-Werror=incompatible-pointer-types]
242 | len, &fw->phys,
| ^~~~~~~~~
| |
| u64 * {aka long long unsigned int *}
In file included from include/linux/pci.h:2693,
from drivers/gpu/drm/nouveau/include/nvif/os.h:8,
from drivers/gpu/drm/nouveau/include/nvkm/core/os.h:4,
from drivers/gpu/drm/nouveau/include/nvkm/core/oclass.h:3,
from drivers/gpu/drm/nouveau/include/nvkm/core/device.h:4,
from drivers/gpu/drm/nouveau/nvkm/core/firmware.c:22:
include/linux/dma-mapping.h:321:29: note: expected 'dma_addr_t *' {aka 'unsigned int *'} but argument is of type 'u64 *' {aka 'long long unsigned int *'}
321 | dma_addr_t *dma_handle, enum dma_data_direction dir, gfp_t gfp)
| ~~~~~~~~~~~~^~~~~~~~~~cc1: some warnings being treated as errors
vim +/dma_alloc_noncoherent +242 drivers/gpu/drm/nouveau/nvkm/core/firmware.c
224
225 int
226 nvkm_firmware_ctor(const struct nvkm_firmware_func *func, const char *name,
227 struct nvkm_device *device, const void *src, int len, struct nvkm_firmware *fw)
228 {
229 fw->func = func;
230 fw->name = name;
231 fw->device = device;
232 fw->len = len;
233
234 switch (fw->func->type) {
235 case NVKM_FIRMWARE_IMG_RAM:
236 fw->img = kmemdup(src, fw->len, GFP_KERNEL);
237 break;
238 case NVKM_FIRMWARE_IMG_DMA: {
239 len = ALIGN(fw->len, PAGE_SIZE);
240
241 fw->img = dma_alloc_noncoherent(fw->device->dev,
> 242 len, &fw->phys,
Signature · 2 lines
0-DAY CI Kernel Test Service https://github.com/intel/lkp-tests/wiki
kernel test robot
build botkernel test robot noticed the following build errors:
Hi Dave,
kernel test robot noticed the following build errors:
[auto build test ERROR on drm-misc/drm-misc-next]
[also build test ERROR on drm-tip/drm-tip linus/master v6.9 next-20240513] [If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]
url: https://github.com/intel-lab-lkp/linux/commits/Dave-Airlie/nouveau-firmware-using-dma-non-coherent-interfaces-for-fw-loading/20240513-144435 base: git://anongit.freedesktop.org/drm/drm-misc drm-misc-next patch link: https://lore.kernel.org/r/20240513064350.1050994-1-airlied%40gmail.com patch subject: [PATCH] nouveau/firmware: using dma non-coherent interfaces for fw loading. config: arm-defconfig (https://download.01.org/0day-ci/archive/20240513/[email protected]/config) compiler: clang version 14.0.6 (https://github.com/llvm/llvm-project f28c006a5895fc0e329fe15fead81e37457cb1d1) reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20240513/[email protected]/reproduce)
If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <[email protected]>
| Closes: https://lore.kernel.org/oe-kbuild-all/[email protected]/
All errors (new ones prefixed by >>):
>> drivers/gpu/drm/nouveau/nvkm/core/firmware.c:242:12: error: incompatible pointer types passing 'u64 *' (aka 'unsigned long long *') to parameter of type 'dma_addr_t *' (aka 'unsigned int *') [-Werror,-Wincompatible-pointer-types]
len, &fw->phys,
^~~~~~~~~ include/linux/dma-mapping.h:321:15: note: passing argument to parameter 'dma_handle' here
dma_addr_t *dma_handle, enum dma_data_direction dir, gfp_t gfp)
^
1 error generated.
vim +242 drivers/gpu/drm/nouveau/nvkm/core/firmware.c
224
225 int
226 nvkm_firmware_ctor(const struct nvkm_firmware_func *func, const char *name,
227 struct nvkm_device *device, const void *src, int len, struct nvkm_firmware *fw)
228 {
229 fw->func = func;
230 fw->name = name;
231 fw->device = device;
232 fw->len = len;
233
234 switch (fw->func->type) {
235 case NVKM_FIRMWARE_IMG_RAM:
236 fw->img = kmemdup(src, fw->len, GFP_KERNEL);
237 break;
238 case NVKM_FIRMWARE_IMG_DMA: {
239 len = ALIGN(fw->len, PAGE_SIZE);
240
241 fw->img = dma_alloc_noncoherent(fw->device->dev,
> 242 len, &fw->phys,
Signature · 2 lines
0-DAY CI Kernel Test Service https://github.com/intel/lkp-tests/wiki