BUG: sleeping function called from invalid context - qxl_bo_kmap_atomic_page()...splat
Subsystems (3)
Greetings,
I'm meeting this splat in master, call io_mapping_map_atomic_wc(), then do something sleepy. In master-rt, I meet a variant of this doing read_lock() in find_next_iomem_res(), due to rwlocks being sleepy in RT.
[ 53.286236] BUG: sleeping function called from invalid context at mm/slab.h:432 [ 53.286809] in_atomic(): 1, irqs_disabled(): 0, pid: 2078, name: Xorg [ 53.287354] Preemption disabled at: [ 53.287360] [<ffffffffa0269c45>] qxl_bo_kmap_atomic_page+0xa5/0x100 [qxl] [ 53.288028] CPU: 0 PID: 2078 Comm: Xorg Tainted: G E 4.12.0-master #35 [ 53.288530] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.8.1-0-g4adadbd-20161202_174313-build11a 04/01/2014 [ 53.289337] Call Trace: [ 53.289513] dump_stack+0x65/0x89 [ 53.289739] ? qxl_bo_kmap_atomic_page+0xa5/0x100 [qxl] [ 53.290084] ___might_sleep+0x11a/0x190 [ 53.290336] __might_sleep+0x4a/0x80 [ 53.290575] ? reserve_memtype+0xa5/0x390 [ 53.290841] kmem_cache_alloc_trace+0x46/0x180 [ 53.291185] reserve_memtype+0xa5/0x390 [ 53.291442] ? ttm_bo_kmap+0x1f5/0x230 [ttm] [ 53.291724] __ioremap_caller+0xf2/0x330 [ 53.291987] ? preempt_count_add+0x47/0xb0 [ 53.292284] ioremap_nocache+0x17/0x20 [ 53.292536] ttm_bo_kmap+0x1f5/0x230 [ttm] [ 53.292816] qxl_bo_kmap+0x43/0x70 [qxl] [ 53.293076] qxl_cursor_atomic_update+0x8d/0x260 [qxl] [ 53.293427] ? drm_atomic_helper_update_legacy_modeset_state+0x1d6/0x210 [drm_kms_helper] [ 53.293966] drm_atomic_helper_commit_planes+0xec/0x230 [drm_kms_helper] [ 53.294403] drm_atomic_helper_commit_tail+0x2b/0x60 [drm_kms_helper] [ 53.294824] commit_tail+0x65/0x70 [drm_kms_helper] [ 53.295145] drm_atomic_helper_commit+0xa9/0x100 [drm_kms_helper] [ 53.295549] drm_atomic_commit+0x4b/0x50 [drm] [ 53.295843] drm_atomic_helper_update_plane+0xf1/0x110 [drm_kms_helper] [ 53.296407] __setplane_internal+0x19f/0x280 [drm] [ 53.296737] drm_mode_cursor_universal+0x101/0x1c0 [drm] [ 53.297091] drm_mode_cursor_common+0x15b/0x1d0 [drm] [ 53.297434] drm_mode_cursor2_ioctl+0xe/0x10 [drm] [ 53.297758] drm_ioctl+0x211/0x460 [drm] [ 53.298205] ? drm_mode_cursor_ioctl+0x50/0x50 [drm] [ 53.298638] ? handle_mm_fault+0x93/0x160 [ 53.298960] do_vfs_ioctl+0x96/0x6e0 [ 53.299200] ? __fget+0x73/0xa0 [ 53.299410] SyS_ioctl+0x41/0x70 [ 53.299630] entry_SYSCALL_64_fastpath+0x1a/0xa5 [ 53.299934] RIP: 0033:0x7fcdedfb0bc7 [ 53.300170] RSP: 002b:00007fffbde2ed48 EFLAGS: 00003246 ORIG_RAX: 0000000000000010 [ 53.300660] RAX: ffffffffffffffda RBX: 0000000000000040 RCX: 00007fcdedfb0bc7 [ 53.301130] RDX: 00007fffbde2ed80 RSI: 00000000c02464bb RDI: 000000000000000b [ 53.301609] RBP: 0000000000000040 R08: 0000000000000040 R09: 000000000000000c [ 53.302124] R10: 00007fffbde2eb80 R11: 0000000000003246 R12: 0000000000000018 [ 53.302614] R13: 000000000275b390 R14: 0000000000000040 R15: 00007fffbde2ee6c
Quoted 7 lines · Mike Galbraith
Greetings,
I'm meeting this splat in master, call
io_mapping_map_atomic_wc(), then do something sleepy. In master-rt, I meet a variant of this doingread_lock()infind_next_iomem_res(), due to rwlocks being sleepy in RT.
Hi Mike,
Thanks for reporting this. Can you confirm the following patch prevents the issue?
Quoted 1 line
8
From 407213129ef4a687378563cbb6ca78faa23f33bd Mon Sep 17 00:00:00 2001
From: Gabriel Krisman Bertazi <[email protected]>
Date: Mon, 8 May 2017 16:47:22 -0300
Subject: [PATCH] drm: qxl: Suffle allocations in atomic cursor update
--- drivers/gpu/drm/qxl/qxl_display.c | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-)
drivers/gpu/drm/qxl/qxl_display.c +9 -9
@@ -581,25 +581,25 @@ static void qxl_cursor_atomic_update(struct drm_plane *plane, obj =to_qxl_framebuffer (fb)->obj; user_bo =gem_to_qxl_bo (obj); -/* pinning is done in the prepare/cleanup framevbuffer */ - ret =qxl_bo_kmap (user_bo, &user_ptr); -if (ret) - goto out_free_release; - ret =qxl_alloc_bo_reserved (qdev, release,sizeof (struct qxl_cursor ) + size, &cursor_bo);if (ret) - goto out_kunmap; + goto out_free_release; ret =qxl_release_reserve_list (release, true);if (ret) goto out_free_bo; - ret =qxl_bo_kmap (cursor_bo, (void **)&cursor); +/* pinning is done in the prepare/cleanup framevbuffer */ + ret =qxl_bo_kmap (user_bo, &user_ptr);if (ret) goto out_backoff; + ret =qxl_bo_kmap (cursor_bo, (void **)&cursor); +if (ret) + goto out_kunmap; + cursor->header.unique =0 ; cursor->header.type =SPICE_CURSOR_TYPE_ALPHA ; cursor->header.width =64 ;
@@ -636,12 +636,12 @@ static void qxl_cursor_atomic_update(struct drm_plane *plane,return ; +out_kunmap: +qxl_bo_kunmap (user_bo); out_backoff:qxl_release_backoff_reserve_list (release); out_free_bo:qxl_bo_unref (&cursor_bo); -out_kunmap: -qxl_bo_kunmap (user_bo); out_free_release:qxl_release_free (qdev, release);return ;
Signature · 7 lines
2.11.0 _______________________________________________ dri-devel mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/dri-devel
Quoted 2 lines · Gabriel Krisman Bertazi
Thanks for reporting this. Can you confirm the following patch prevents the issue?
Nope, it still gripes.
[ 43.910362] BUG: sleeping function called from invalid context at mm/slab.h:432 [ 43.910955] in_atomic(): 1, irqs_disabled(): 0, pid: 2077, name: Xorg [ 43.911472] Preemption disabled at: [ 43.911478] [<ffffffffa02b1c45>] qxl_bo_kmap_atomic_page+0xa5/0x100 [qxl] [ 43.912103] CPU: 0 PID: 2077 Comm: Xorg Tainted: G E 4.12.0-master #38 [ 43.912550] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.8.1-0-g4adadbd-20161202_174313-build11a 04/01/2014 [ 43.913202] Call Trace: [ 43.913371] dump_stack+0x65/0x89 [ 43.913581] ? qxl_bo_kmap_atomic_page+0xa5/0x100 [qxl] [ 43.913876] ___might_sleep+0x11a/0x190 [ 43.914095] __might_sleep+0x4a/0x80 [ 43.914319] ? qxl_bo_create+0x50/0x190 [qxl] [ 43.914565] kmem_cache_alloc_trace+0x46/0x180 [ 43.914836] qxl_bo_create+0x50/0x190 [qxl] [ 43.915082] ? refcount_dec_and_test+0x11/0x20 [ 43.915332] ? ttm_mem_io_reserve+0x41/0xe0 [ttm] [ 43.915595] qxl_alloc_bo_reserved+0x37/0xb0 [qxl] [ 43.915884] qxl_cursor_atomic_update+0x8f/0x260 [qxl] [ 43.916172] ? drm_atomic_helper_update_legacy_modeset_state+0x1d6/0x210 [drm_kms_helper] [ 43.916623] drm_atomic_helper_commit_planes+0xec/0x230 [drm_kms_helper] [ 43.916995] drm_atomic_helper_commit_tail+0x2b/0x60 [drm_kms_helper] [ 43.917398] commit_tail+0x65/0x70 [drm_kms_helper] [ 43.917693] drm_atomic_helper_commit+0xa9/0x100 [drm_kms_helper] [ 43.918039] drm_atomic_commit+0x4b/0x50 [drm] [ 43.918334] drm_atomic_helper_update_plane+0xf1/0x110 [drm_kms_helper] [ 43.918902] __setplane_internal+0x19f/0x280 [drm] [ 43.919240] drm_mode_cursor_universal+0x101/0x1c0 [drm] [ 43.919541] drm_mode_cursor_common+0x15b/0x1d0 [drm] [ 43.919858] drm_mode_cursor2_ioctl+0xe/0x10 [drm] [ 43.920157] drm_ioctl+0x211/0x460 [drm] [ 43.920383] ? drm_mode_cursor_ioctl+0x50/0x50 [drm] [ 43.920664] ? handle_mm_fault+0x93/0x160 [ 43.920893] do_vfs_ioctl+0x96/0x6e0 [ 43.921117] ? __fget+0x73/0xa0 [ 43.921322] SyS_ioctl+0x41/0x70 [ 43.921545] entry_SYSCALL_64_fastpath+0x1a/0xa5 [ 43.922188] RIP: 0033:0x7f1145804bc7 [ 43.922526] RSP: 002b:00007ffcd3e50508 EFLAGS: 00003246 ORIG_RAX: 0000000000000010 [ 43.923367] RAX: ffffffffffffffda RBX: 0000000000000040 RCX: 00007f1145804bc7 [ 43.923852] RDX: 00007ffcd3e50540 RSI: 00000000c02464bb RDI: 000000000000000b [ 43.924299] RBP: 0000000000000040 R08: 0000000000000040 R09: 000000000000000c [ 43.924694] R10: 00007ffcd3e50340 R11: 0000000000003246 R12: 0000000000000018 [ 43.925128] R13: 00000000022bc390 R14: 0000000000000040 R15: 00007ffcd3e5062c
Quoted 6 lines · Mike Galbraith
Quoted 2 lines · Gabriel Krisman Bertazi
Thanks for reporting this. Can you confirm the following patch prevents the issue?
Nope, it still gripes.
The reason for this gripe is that we find that we don't have memory reserved.. a tad too late.
Xorg-2252 [000] .... 135.409756: qxl_release_map <-qxl_cursor_atomic_update
Xorg-2252 [000] .... 135.409756: qxl_bo_kmap_atomic_page <-qxl_release_map
Xorg-2252 [000] .... 135.409757: qxl_bo_kmap_atomic_page: ENTER
Xorg-2252 [000] .... 135.409757: ttm_mem_io_lock <-qxl_bo_kmap_atomic_page
Xorg-2252 [000] .... 135.409757: ttm_mem_io_reserve <-qxl_bo_kmap_atomic_page
Xorg-2252 [000] .... 135.409757: qxl_ttm_io_mem_reserve <-ttm_mem_io_reserve
Xorg-2252 [000] .... 135.409757: ttm_mem_io_unlock <-qxl_bo_kmap_atomic_page
Xorg-2252 [000] .... 135.409757: qxl_bo_kmap_atomic_page: PREEMPTION DISABLED
Xorg-2252 [000] ...1 135.409758: qxl_bo_kmap <-qxl_cursor_atomic_update
Xorg-2252 [000] ...1 135.409758: ttm_bo_kmap <-qxl_bo_kmap <== too late
Xorg-2252 [000] ...1 135.409758: ttm_mem_io_reserve <-ttm_bo_kmap
Xorg-2252 [000] ...1 135.409758: qxl_ttm_io_mem_reserve <-ttm_mem_io_reserve
Xorg-2252 [000] ...1 135.409759: ioremap_nocache <-ttm_bo_kmap <== game over
Xorg-2252 [000] ...1 135.409759: __ioremap_caller <-ioremap_nocache
Xorg-2252 [000] ...1 135.409759: walk_system_ram_range <-__ioremap_caller
Xorg-2252 [000] ...1 135.409759: find_next_iomem_res <-walk_system_ram_range
Xorg-2252 [000] ...1 135.409759: _raw_read_lock <-find_next_iomem_res
Xorg-2252 [000] ...1 135.409760: reserve_memtype <-__ioremap_caller
Xorg-2252 [000] ...1 135.409760: pat_pagerange_is_ram <-reserve_memtype
Xorg-2252 [000] ...1 135.409761: walk_system_ram_range <-pat_pagerange_is_ram
Xorg-2252 [000] ...1 135.409761: find_next_iomem_res <-walk_system_ram_range
Xorg-2252 [000] ...1 135.409761: _raw_read_lock <-find_next_iomem_res
Xorg-2252 [000] ...1 135.409761: kmem_cache_alloc_trace <-reserve_memtype
Xorg-2252 [000] ...1 135.409761: __might_sleep <-kmem_cache_alloc_trace
Xorg-2252 [000] ...1 135.409762: ___might_sleep <-__might_sleepQuoted 11 lines · Mike Galbraith
Quoted 6 lines · Mike Galbraith
Quoted 2 lines · Gabriel Krisman Bertazi
Thanks for reporting this. Can you confirm the following patch prevents the issue?
Nope, it still gripes.
The reason for this gripe is that we find that we don't have memory reserved.. a tad too late.
Thanks for the info.
Sorry I wasn't able to get back to this last week. I'll try to get another patch for -rc2.
Quoted 25 lines
Xorg-2252 [000] .... 135.409756: qxl_release_map <-qxl_cursor_atomic_update
Xorg-2252 [000] .... 135.409756: qxl_bo_kmap_atomic_page <-qxl_release_map
Xorg-2252 [000] .... 135.409757: qxl_bo_kmap_atomic_page: ENTER
Xorg-2252 [000] .... 135.409757: ttm_mem_io_lock <-qxl_bo_kmap_atomic_page
Xorg-2252 [000] .... 135.409757: ttm_mem_io_reserve <-qxl_bo_kmap_atomic_page
Xorg-2252 [000] .... 135.409757: qxl_ttm_io_mem_reserve <-ttm_mem_io_reserve
Xorg-2252 [000] .... 135.409757: ttm_mem_io_unlock <-qxl_bo_kmap_atomic_page
Xorg-2252 [000] .... 135.409757: qxl_bo_kmap_atomic_page: PREEMPTION DISABLED
Xorg-2252 [000] ...1 135.409758: qxl_bo_kmap <-qxl_cursor_atomic_update
Xorg-2252 [000] ...1 135.409758: ttm_bo_kmap <-qxl_bo_kmap <== too late
Xorg-2252 [000] ...1 135.409758: ttm_mem_io_reserve <-ttm_bo_kmap
Xorg-2252 [000] ...1 135.409758: qxl_ttm_io_mem_reserve <-ttm_mem_io_reserve
Xorg-2252 [000] ...1 135.409759: ioremap_nocache <-ttm_bo_kmap <== game over
Xorg-2252 [000] ...1 135.409759: __ioremap_caller <-ioremap_nocache
Xorg-2252 [000] ...1 135.409759: walk_system_ram_range <-__ioremap_caller
Xorg-2252 [000] ...1 135.409759: find_next_iomem_res <-walk_system_ram_range
Xorg-2252 [000] ...1 135.409759: _raw_read_lock <-find_next_iomem_res
Xorg-2252 [000] ...1 135.409760: reserve_memtype <-__ioremap_caller
Xorg-2252 [000] ...1 135.409760: pat_pagerange_is_ram <-reserve_memtype
Xorg-2252 [000] ...1 135.409761: walk_system_ram_range <-pat_pagerange_is_ram
Xorg-2252 [000] ...1 135.409761: find_next_iomem_res <-walk_system_ram_range
Xorg-2252 [000] ...1 135.409761: _raw_read_lock <-find_next_iomem_res
Xorg-2252 [000] ...1 135.409761: kmem_cache_alloc_trace <-reserve_memtype
Xorg-2252 [000] ...1 135.409761: __might_sleep <-kmem_cache_alloc_trace
Xorg-2252 [000] ...1 135.409762: ___might_sleep <-__might_sleepSignature · 5 lines
Gabriel Krisman Bertazi _______________________________________________ dri-devel mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/dri-devel
Quoted 6 lines · Mike Galbraith
Quoted 2 lines · Gabriel Krisman Bertazi
Thanks for reporting this. Can you confirm the following patch prevents the issue?
Nope, it still gripes.
Oops... Thanks for checking. The following patch fixes the issue for me. Can you please test that one?
Quoted 1 line
8
Subject: [PATCH] drm: qxl: Delay entering atomic context during cursor update
qxl_release_map will enter an atomic context, but since we still need to alloc memory for BOs, we better delay that until we have everything we need, in case we need to sleep inside the allocation. This avoids the Sleep in atomic state below, which was reported by Mike.
[ 43.910362] BUG: sleeping function called from invalid context at mm/slab.h:432 [ 43.910955] in_atomic(): 1, irqs_disabled(): 0, pid: 2077, name: Xorg [ 43.911472] Preemption disabled at: [ 43.911478] [<ffffffffa02b1c45>] qxl_bo_kmap_atomic_page+0xa5/0x100 [qxl] [ 43.912103] CPU: 0 PID: 2077 Comm: Xorg Tainted: G E 4.12.0-master #38 [ 43.912550] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.8.1-0-g4adadbd-20161202_174313-build11a 04/01/2014 [ 43.913202] Call Trace: [ 43.913371] dump_stack+0x65/0x89 [ 43.913581] ? qxl_bo_kmap_atomic_page+0xa5/0x100 [qxl] [ 43.913876] ___might_sleep+0x11a/0x190 [ 43.914095] __might_sleep+0x4a/0x80 [ 43.914319] ? qxl_bo_create+0x50/0x190 [qxl] [ 43.914565] kmem_cache_alloc_trace+0x46/0x180 [ 43.914836] qxl_bo_create+0x50/0x190 [qxl] [ 43.915082] ? refcount_dec_and_test+0x11/0x20 [ 43.915332] ? ttm_mem_io_reserve+0x41/0xe0 [ttm] [ 43.915595] qxl_alloc_bo_reserved+0x37/0xb0 [qxl] [ 43.915884] qxl_cursor_atomic_update+0x8f/0x260 [qxl] [ 43.916172] ? drm_atomic_helper_update_legacy_modeset_state+0x1d6/0x210 [drm_kms_helper] [ 43.916623] drm_atomic_helper_commit_planes+0xec/0x230 [drm_kms_helper] [ 43.916995] drm_atomic_helper_commit_tail+0x2b/0x60 [drm_kms_helper] [ 43.917398] commit_tail+0x65/0x70 [drm_kms_helper] [ 43.917693] drm_atomic_helper_commit+0xa9/0x100 [drm_kms_helper] [ 43.918039] drm_atomic_commit+0x4b/0x50 [drm] [ 43.918334] drm_atomic_helper_update_plane+0xf1/0x110 [drm_kms_helper] [ 43.918902] __setplane_internal+0x19f/0x280 [drm] [ 43.919240] drm_mode_cursor_universal+0x101/0x1c0 [drm] [ 43.919541] drm_mode_cursor_common+0x15b/0x1d0 [drm] [ 43.919858] drm_mode_cursor2_ioctl+0xe/0x10 [drm] [ 43.920157] drm_ioctl+0x211/0x460 [drm] [ 43.920383] ? drm_mode_cursor_ioctl+0x50/0x50 [drm] [ 43.920664] ? handle_mm_fault+0x93/0x160 [ 43.920893] do_vfs_ioctl+0x96/0x6e0 [ 43.921117] ? __fget+0x73/0xa0 [ 43.921322] SyS_ioctl+0x41/0x70 [ 43.921545] entry_SYSCALL_64_fastpath+0x1a/0xa5 [ 43.922188] RIP: 0033:0x7f1145804bc7 [ 43.922526] RSP: 002b:00007ffcd3e50508 EFLAGS: 00003246 ORIG_RAX: 0000000000000010 [ 43.923367] RAX: ffffffffffffffda RBX: 0000000000000040 RCX: 00007f1145804bc7 [ 43.923852] RDX: 00007ffcd3e50540 RSI: 00000000c02464bb RDI: 000000000000000b [ 43.924299] RBP: 0000000000000040 R08: 0000000000000040 R09: 000000000000000c [ 43.924694] R10: 00007ffcd3e50340 R11: 0000000000003246 R12: 0000000000000018 [ 43.925128] R13: 00000000022bc390 R14: 0000000000000040 R15: 00007ffcd3e5062c
--- drivers/gpu/drm/qxl/qxl_display.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
drivers/gpu/drm/qxl/qxl_display.c +2 -2
@@ -575,8 +575,6 @@ static void qxl_cursor_atomic_update(struct drm_plane *plane,if (ret)return ; - cmd = (struct qxl_cursor_cmd *)qxl_release_map (qdev, release); -if (fb != old_state->fb) { obj =to_qxl_framebuffer (fb)->obj; user_bo =gem_to_qxl_bo (obj);
@@ -614,6 +612,7 @@ static void qxl_cursor_atomic_update(struct drm_plane *plane,qxl_bo_kunmap (cursor_bo);qxl_bo_kunmap (user_bo); + cmd = (struct qxl_cursor_cmd *)qxl_release_map (qdev, release); cmd->u.set.visible =1 ; cmd->u.set.shape =qxl_bo_physical_address (qdev, cursor_bo,0 );
@@ -624,6 +623,7 @@ static void qxl_cursor_atomic_update(struct drm_plane *plane,if (ret) goto out_free_release; + cmd = (struct qxl_cursor_cmd *)qxl_release_map (qdev, release); cmd->type =QXL_CURSOR_MOVE ; }
Signature · 9 lines
2.11.0 _______________________________________________ dri-devel mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/dri-devel
Quoted 6 lines · Gabriel Krisman Bertazi
Quoted 6 lines
Mike Galbraith <[email protected]> writes:
Quoted 1 line · Gabriel Krisman Bertazi
Quoted 7 lines
Quoted 4 lines
Quoted 2 lines
Thanks for reporting this. Can you confirm the following patch prevents the issue?
Nope, it still gripes.
Oops... Thanks for checking. The following patch fixes the issue for me. Can you please test that one?
Yup, all better.
-Mike
Quoted 10 lines · Mike Galbraith
Quoted 9 lines · Gabriel Krisman Bertazi
Quoted 9 lines
Mike Galbraith <[email protected]> writes:
Quoted 3 lines · Gabriel Krisman Bertazi
Thanks for reporting this. Can you confirm the following patch prevents the issue?
Quoted 6 lines
Quoted 4 lines
Quoted 1 line
Nope, it still gripes.
Oops... Thanks for checking. The following patch fixes the issue for me. Can you please test that one?
Yup, all better.
Thanks for testing.
I resent it to the list with the appropriate subject line to make sure it gets properly reviewed.
Signature · 5 lines
Gabriel Krisman Bertazi _______________________________________________ dri-devel mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/dri-devel