Skip to contentSkip to conversation

nouveau. swiotlb: coherent allocation failed for device 0000:01:00.0 size=2097152

DISCUSSION Mike Galbraith 5 participants 10 messages Dec 18, 2017 – Jan 2, 2018

lore
Share & tools
lore
#0 lore raw

Greetings,

Kernel bound workloads seem to trigger the below for whatever reason. I only see this when beating up NFS.  There was a kworker wakeup latency issue, but with a bandaid applied to fix that up, I can still trigger this.

[ 1313.811031] nouveau 0000:01:00.0: swiotlb buffer is full (sz: 2097152 bytes)
[ 1313.811035] swiotlb: coherent allocation failed for device 0000:01:00.0 size=2097152
[ 1313.811038] CPU: 6 PID: 3026 Comm: Xorg Tainted: G            E    4.15.0.g1291a0d5-master #355
[ 1313.811040] Hardware name: MEDION MS-7848/MS-7848, BIOS M7848W08.20C 09/23/2013
[ 1313.811041] Call Trace:
[ 1313.811049]  dump_stack+0x7c/0xb6
[ 1313.811053]  swiotlb_alloc_coherent+0x13f/0x150
[ 1313.811060]  ttm_dma_pool_alloc_new_pages+0x106/0x3c0 [ttm]
[ 1313.811066]  ttm_dma_pool_get_pages+0x10a/0x1e0 [ttm]
[ 1313.811070]  ttm_dma_populate+0x21f/0x2f0 [ttm]
[ 1313.811075]  ttm_tt_bind+0x2f/0x60 [ttm]
[ 1313.811079]  ttm_bo_handle_move_mem+0x51f/0x580 [ttm]
[ 1313.811084]  ? ttm_bo_handle_move_mem+0x5/0x580 [ttm]
[ 1313.811088]  ttm_bo_validate+0x10c/0x120 [ttm]
[ 1313.811092]  ? ttm_bo_validate+0x5/0x120 [ttm]
[ 1313.811106]  ? drm_mode_setcrtc+0x20e/0x540 [drm]
[ 1313.811109]  ttm_bo_init_reserved+0x290/0x490 [ttm]
[ 1313.811114]  ttm_bo_init+0x52/0xb0 [ttm]
[ 1313.811141]  ? nv10_bo_put_tile_region+0x60/0x60 [nouveau]
[ 1313.811163]  nouveau_bo_new+0x465/0x5e0 [nouveau]
[ 1313.811184]  ? nv10_bo_put_tile_region+0x60/0x60 [nouveau]
[ 1313.811203]  nouveau_gem_new+0x66/0x110 [nouveau]
[ 1313.811223]  ? nouveau_gem_new+0x110/0x110 [nouveau]
[ 1313.811241]  nouveau_gem_ioctl_new+0x48/0xc0 [nouveau]
[ 1313.811249]  drm_ioctl_kernel+0x64/0xb0 [drm]
[ 1313.811257]  drm_ioctl+0x2a4/0x360 [drm]
[ 1313.811276]  ? nouveau_gem_new+0x110/0x110 [nouveau]
[ 1313.811285]  ? drm_ioctl+0x5/0x360 [drm]
[ 1313.811304]  nouveau_drm_ioctl+0x50/0xb0 [nouveau]
[ 1313.811308]  do_vfs_ioctl+0x90/0x690
[ 1313.811311]  ? do_vfs_ioctl+0x5/0x690
[ 1313.811313]  SyS_ioctl+0x3b/0x70
[ 1313.811316]  entry_SYSCALL_64_fastpath+0x1f/0x91
[ 1313.811320] RIP: 0033:0x7f3234746227
[ 1313.811321] RSP: 002b:00007ffc3ace0408 EFLAGS: 00003246 ORIG_RAX: 0000000000000010
[ 1313.811324] RAX: ffffffffffffffda RBX: 00000000025515d0 RCX: 00007f3234746227
[ 1313.811325] RDX: 00007ffc3ace0460 RSI: 00000000c0306480 RDI: 000000000000000b
[ 1313.811326] RBP: 0000000000824120 R08: 0000000002548f80 R09: 00000000025490d0
[ 1313.811328] R10: 0000000000000000 R11: 0000000000003246 R12: 000000000000093d
[ 1313.811329] R13: 0000000002aff74c R14: 0000000000824150 R15: 0000000000000000

_______________________________________________
Nouveau mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/nouveau

#1 lore raw
Quoted 6 lines · Mike Galbraith

Greetings,

Kernel bound workloads seem to trigger the below for whatever reason.
 I only see this when beating up NFS.  There was a kworker wakeup
latency issue, but with a bandaid applied to fix that up, I can still trigger this.

Hi,

i have seen this one as well with my system, but i could not find an easy way to trigger it for bisecting purpose. If you can trigger it conveniently, a bisect would be nice!

Greetings,

Tobias

Quoted 33 lines
[ 1313.811031] nouveau 0000:01:00.0: swiotlb buffer is full (sz: 2097152 bytes)
[ 1313.811035] swiotlb: coherent allocation failed for device 0000:01:00.0 size=2097152
[ 1313.811038] CPU: 6 PID: 3026 Comm: Xorg Tainted: G            E    4.15.0.g1291a0d5-master #355
[ 1313.811040] Hardware name: MEDION MS-7848/MS-7848, BIOS M7848W08.20C 09/23/2013
[ 1313.811041] Call Trace:
[ 1313.811049]  dump_stack+0x7c/0xb6
[ 1313.811053]  swiotlb_alloc_coherent+0x13f/0x150
[ 1313.811060]  ttm_dma_pool_alloc_new_pages+0x106/0x3c0 [ttm]
[ 1313.811066]  ttm_dma_pool_get_pages+0x10a/0x1e0 [ttm]
[ 1313.811070]  ttm_dma_populate+0x21f/0x2f0 [ttm]
[ 1313.811075]  ttm_tt_bind+0x2f/0x60 [ttm]
[ 1313.811079]  ttm_bo_handle_move_mem+0x51f/0x580 [ttm]
[ 1313.811084]  ? ttm_bo_handle_move_mem+0x5/0x580 [ttm]
[ 1313.811088]  ttm_bo_validate+0x10c/0x120 [ttm]
[ 1313.811092]  ? ttm_bo_validate+0x5/0x120 [ttm]
[ 1313.811106]  ? drm_mode_setcrtc+0x20e/0x540 [drm]
[ 1313.811109]  ttm_bo_init_reserved+0x290/0x490 [ttm]
[ 1313.811114]  ttm_bo_init+0x52/0xb0 [ttm]
[ 1313.811141]  ? nv10_bo_put_tile_region+0x60/0x60 [nouveau]
[ 1313.811163]  nouveau_bo_new+0x465/0x5e0 [nouveau]
[ 1313.811184]  ? nv10_bo_put_tile_region+0x60/0x60 [nouveau]
[ 1313.811203]  nouveau_gem_new+0x66/0x110 [nouveau]
[ 1313.811223]  ? nouveau_gem_new+0x110/0x110 [nouveau]
[ 1313.811241]  nouveau_gem_ioctl_new+0x48/0xc0 [nouveau]
[ 1313.811249]  drm_ioctl_kernel+0x64/0xb0 [drm]
[ 1313.811257]  drm_ioctl+0x2a4/0x360 [drm]
[ 1313.811276]  ? nouveau_gem_new+0x110/0x110 [nouveau]
[ 1313.811285]  ? drm_ioctl+0x5/0x360 [drm]
[ 1313.811304]  nouveau_drm_ioctl+0x50/0xb0 [nouveau]
[ 1313.811308]  do_vfs_ioctl+0x90/0x690
[ 1313.811311]  ? do_vfs_ioctl+0x5/0x690
[ 1313.811313]  SyS_ioctl+0x3b/0x70
Quoted 8 lines
[ 1313.811316]  entry_SYSCALL_64_fastpath+0x1f/0x91
[ 1313.811320] RIP: 0033:0x7f3234746227
[ 1313.811321] RSP: 002b:00007ffc3ace0408 EFLAGS: 00003246 ORIG_RAX: 0000000000000010
[ 1313.811324] RAX: ffffffffffffffda RBX: 00000000025515d0 RCX: 00007f3234746227
[ 1313.811325] RDX: 00007ffc3ace0460 RSI: 00000000c0306480 RDI: 000000000000000b
[ 1313.811326] RBP: 0000000000824120 R08: 0000000002548f80 R09: 00000000025490d0
[ 1313.811328] R10: 0000000000000000 R11: 0000000000003246 R12: 000000000000093d
[ 1313.811329] R13: 0000000002aff74c R14: 0000000000824150 R15: 0000000000000000

_______________________________________________
Nouveau mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/nouveau

#2 lore raw
Quoted 9 lines · Tobias Klausmann
Quoted 6 lines · Mike Galbraith

Greetings,

Kernel bound workloads seem to trigger the below for whatever reason.
 I only see this when beating up NFS.  There was a kworker wakeup
latency issue, but with a bandaid applied to fix that up, I can still trigger this.

Quoted 5 lines

Hi,

i have seen this one as well with my system, but i could not find an easy way to trigger it for bisecting purpose. If you can trigger it conveniently, a bisect would be nice!

Workload permitting.  To reproduce, mount your box NFS, cd to somewhere the NFS mount, and just do bonnie -s <memory size>.  There, maybe you'll beat me to it.  I hope so, I have multiple kernels doing the annoying "baby birds in a nest" thing at me literally endlessly :)

-Mike
_______________________________________________
Nouveau mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/nouveau

#3 lore raw
In reply to Tobias Klausmann
Quoted 9 lines · Tobias Klausmann
Quoted 6 lines · Mike Galbraith

Greetings,

Kernel bound workloads seem to trigger the below for whatever reason. I only see this when beating up NFS.  There was a kworker wakeup latency issue, but with a bandaid applied to fix that up, I can still trigger this.

Quoted 5 lines

Hi,

i have seen this one as well with my system, but i could not find an easy way to trigger it for bisecting purpose. If you can trigger it conveniently, a bisect would be nice!

I'm seeing this (with the amdgpu and radeon drivers) when restic takes a backup, creating memory pressure. I happen to have just finished bisecting, the result is:

648bc3574716400acc06f99915815f80d9563783 is the first bad commit commit 648bc3574716400acc06f99915815f80d9563783

Author: Christian König <[email protected]>
Date:   Thu Jul 6 09:59:43 2017 +0200

drm/ttm: add transparent huge page support for DMA allocations v2

Try to allocate huge pages when it makes sense.

v2: fix comment and use ifdef

Signature · 6 lines
Earthling Michel Dänzer               |               http://www.amd.com
Libre software enthusiast             |             Mesa and X developer
_______________________________________________
Nouveau mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/nouveau
#4 lore raw
Quoted 22 lines · Michel Dänzer
Quoted 9 lines · Tobias Klausmann
Quoted 6 lines · Mike Galbraith

Greetings,

Kernel bound workloads seem to trigger the below for whatever reason. I only see this when beating up NFS.  There was a kworker wakeup latency issue, but with a bandaid applied to fix that up, I can still trigger this.

Quoted 5 lines

Hi,

i have seen this one as well with my system, but i could not find an easy way to trigger it for bisecting purpose. If you can trigger it conveniently, a bisect would be nice!

I'm seeing this (with the amdgpu and radeon drivers) when restic takes a backup, creating memory pressure. I happen to have just finished bisecting, the result is:

648bc3574716400acc06f99915815f80d9563783 is the first bad commit commit 648bc3574716400acc06f99915815f80d9563783

Quoted 10 lines
Author: Christian König <[email protected]>
Date:   Thu Jul 6 09:59:43 2017 +0200

drm/ttm: add transparent huge page support for DMA allocations v2

Try to allocate huge pages when it makes sense.

v2: fix comment and use ifdef

BTW, I haven't noticed any bad effects other than the dmesg splats, so maybe it's just noise about transient failures for which there is a proper fallback in place.

Signature · 6 lines
Earthling Michel Dänzer               |               http://www.amd.com
Libre software enthusiast             |             Mesa and X developer
_______________________________________________
Nouveau mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/nouveau
#5 lore raw

Am 19.12.2017 um 11:39 schrieb Michel Dänzer:

Quoted 26 lines · Michel Dänzer
Quoted 25 lines · Tobias Klausmann
Quoted 8 lines · Mike Galbraith
Quoted 6 lines

Greetings,

Kernel bound workloads seem to trigger the below for whatever reason.
  I only see this when beating up NFS.  There was a kworker wakeup
latency issue, but with a bandaid applied to fix that up, I can still trigger this.

Quoted 5 lines

Hi,

i have seen this one as well with my system, but i could not find an easy way to trigger it for bisecting purpose. If you can trigger it conveniently, a bisect would be nice!

I'm seeing this (with the amdgpu and radeon drivers) when restic takes a backup, creating memory pressure. I happen to have just finished bisecting, the result is:

648bc3574716400acc06f99915815f80d9563783 is the first bad commit commit 648bc3574716400acc06f99915815f80d9563783

Author: Christian König <[email protected]>
Date:   Thu Jul 6 09:59:43 2017 +0200

drm/ttm: add transparent huge page support for DMA allocations v2

Quoted 8 lines
Quoted 5 lines

Try to allocate huge pages when it makes sense.

v2: fix comment and use ifdef

BTW, I haven't noticed any bad effects other than the dmesg splats, so maybe it's just noise about transient failures for which there is a proper fallback in place.

Yeah, I think that is exactly what happens here.

We try to allocate a huge page, but fail and so fall back to using multiple 4k pages instead.

Going to send out a patch to suppress the warning.

Thanks for bisecting this,
Christian.
_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel

#6 lore raw
Quoted 43 lines · Christian König

Am 19.12.2017 um 11:39 schrieb Michel Dänzer:

Quoted 32 lines · Michel Dänzer
Quoted 30 lines · Tobias Klausmann
Quoted 11 lines · Mike Galbraith
Quoted 7 lines

Greetings,

Kernel bound workloads seem to trigger the below for whatever reason.
I only see this when beating up NFS. There was a kworker wakeup
latency issue, but with a bandaid applied to fix that up, I can still trigger this.

Quoted 5 lines

Hi,

i have seen this one as well with my system, but i could not find an easy way to trigger it for bisecting purpose. If you can trigger it conveniently, a bisect would be nice!

I'm seeing this (with the amdgpu and radeon drivers) when restic takes a backup, creating memory pressure. I happen to have just finished bisecting, the result is:

648bc3574716400acc06f99915815f80d9563783 is the first bad commit commit 648bc3574716400acc06f99915815f80d9563783

Author: Christian König <[email protected]>
Date:   Thu Jul 6 09:59:43 2017 +0200

drm/ttm: add transparent huge page support for DMA allocations v2

Quoted 8 lines
Quoted 5 lines

Try to allocate huge pages when it makes sense.

v2: fix comment and use ifdef

BTW, I haven't noticed any bad effects other than the dmesg splats, so maybe it's just noise about transient failures for which there is a proper fallback in place.

Quoted 6 lines

Yeah, I think that is exactly what happens here.

We try to allocate a huge page, but fail and so fall back to using multiple 4k pages instead.

Going to send out a patch to suppress the warning.

Hi Christian,

Did you ever send out such a patch? I didn't see one on the list, but perhaps I missed it. One definitely hasn't made it upstream yet. (I just hit the issue myself with Linus's tree from last night.)

Thanks,

-ilia
_______________________________________________
Nouveau mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/nouveau

#7 lore raw
Quoted 50 lines · Ilia Mirkin
Quoted 48 lines · Christian König

Am 19.12.2017 um 11:39 schrieb Michel Dänzer:

Quoted 32 lines · Michel Dänzer
Quoted 30 lines · Tobias Klausmann
Quoted 11 lines · Mike Galbraith
Quoted 7 lines

Greetings,

Kernel bound workloads seem to trigger the below for whatever reason.
I only see this when beating up NFS. There was a kworker wakeup
latency issue, but with a bandaid applied to fix that up, I can still trigger this.

Quoted 5 lines

Hi,

i have seen this one as well with my system, but i could not find an easy way to trigger it for bisecting purpose. If you can trigger it conveniently, a bisect would be nice!

I'm seeing this (with the amdgpu and radeon drivers) when restic takes a backup, creating memory pressure. I happen to have just finished bisecting, the result is:

648bc3574716400acc06f99915815f80d9563783 is the first bad commit commit 648bc3574716400acc06f99915815f80d9563783

Author: Christian König <[email protected]>
Date:   Thu Jul 6 09:59:43 2017 +0200

drm/ttm: add transparent huge page support for DMA allocations v2

Quoted 8 lines
Quoted 5 lines

Try to allocate huge pages when it makes sense.

v2: fix comment and use ifdef

BTW, I haven't noticed any bad effects other than the dmesg splats, so maybe it's just noise about transient failures for which there is a proper fallback in place.

Yeah, I think that is exactly what happens here.

We try to allocate a huge page, but fail and so fall back to using multiple 4k pages instead.

Quoted 7 lines
Quoted 1 line

Going to send out a patch to suppress the warning.

Hi Christian,

Did you ever send out such a patch? I didn't see one on the list, but perhaps I missed it. One definitely hasn't made it upstream yet. (I just hit the issue myself with Linus's tree from last night.)

Actually, that wants a bit more methinks, because while the stack dump goes away, you still get spammed, it just comes in smaller chunks.

-Mike
_______________________________________________
Nouveau mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/nouveau

#8 lore raw
Quoted 53 lines · Mike Galbraith
Quoted 52 lines · Ilia Mirkin
Quoted 49 lines · Christian König

Am 19.12.2017 um 11:39 schrieb Michel Dänzer:

Quoted 32 lines · Michel Dänzer
Quoted 30 lines · Tobias Klausmann
Quoted 11 lines · Mike Galbraith
Quoted 7 lines

Greetings,

Kernel bound workloads seem to trigger the below for whatever reason.
I only see this when beating up NFS. There was a kworker wakeup
latency issue, but with a bandaid applied to fix that up, I can still trigger this.

Quoted 5 lines

Hi,

i have seen this one as well with my system, but i could not find an easy way to trigger it for bisecting purpose. If you can trigger it conveniently, a bisect would be nice!

I'm seeing this (with the amdgpu and radeon drivers) when restic takes a backup, creating memory pressure. I happen to have just finished bisecting, the result is:

648bc3574716400acc06f99915815f80d9563783 is the first bad commit commit 648bc3574716400acc06f99915815f80d9563783

Author: Christian König <[email protected]>
Date:   Thu Jul 6 09:59:43 2017 +0200

drm/ttm: add transparent huge page support for DMA allocations v2

Quoted 8 lines
Quoted 5 lines

Try to allocate huge pages when it makes sense.

v2: fix comment and use ifdef

BTW, I haven't noticed any bad effects other than the dmesg splats, so maybe it's just noise about transient failures for which there is a proper fallback in place.

Yeah, I think that is exactly what happens here.

We try to allocate a huge page, but fail and so fall back to using multiple 4k pages instead.

Going to send out a patch to suppress the warning.

Quoted 8 lines
Quoted 5 lines

Hi Christian,

Did you ever send out such a patch? I didn't see one on the list, but perhaps I missed it. One definitely hasn't made it upstream yet. (I just hit the issue myself with Linus's tree from last night.)

Actually, that wants a bit more methinks, because while the stack dump goes away, you still get spammed, it just comes in smaller chunks.

OK, well this has to either be fixed or reverted. Right now it's complaining all the time for me after like a day of uptime.

-ilia
_______________________________________________
Nouveau mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/nouveau

#9 lore raw

Am 01.01.2018 um 19:08 schrieb Ilia Mirkin:

Quoted 48 lines · Mike Galbraith
Quoted 47 lines · Ilia Mirkin
Quoted 46 lines
Quoted 36 lines · Christian König

Am 19.12.2017 um 11:39 schrieb Michel Dänzer:

Quoted 26 lines · Michel Dänzer
Quoted 25 lines · Tobias Klausmann
Quoted 8 lines · Mike Galbraith
Quoted 6 lines

Greetings,

Kernel bound workloads seem to trigger the below for whatever reason.
I only see this when beating up NFS. There was a kworker wakeup
latency issue, but with a bandaid applied to fix that up, I can still trigger this.

Quoted 5 lines

Hi,

i have seen this one as well with my system, but i could not find an easy way to trigger it for bisecting purpose. If you can trigger it conveniently, a bisect would be nice!

I'm seeing this (with the amdgpu and radeon drivers) when restic takes a backup, creating memory pressure. I happen to have just finished bisecting, the result is:

648bc3574716400acc06f99915815f80d9563783 is the first bad commit commit 648bc3574716400acc06f99915815f80d9563783

Author: Christian König <[email protected]>
Date:   Thu Jul 6 09:59:43 2017 +0200

drm/ttm: add transparent huge page support for DMA allocations v2

Quoted 8 lines
Quoted 5 lines

Try to allocate huge pages when it makes sense.

v2: fix comment and use ifdef

BTW, I haven't noticed any bad effects other than the dmesg splats, so maybe it's just noise about transient failures for which there is a proper fallback in place.

Quoted 6 lines

Yeah, I think that is exactly what happens here.

We try to allocate a huge page, but fail and so fall back to using multiple 4k pages instead.

Going to send out a patch to suppress the warning.

Hi Christian,

Quoted 7 lines
Quoted 5 lines
Quoted 3 lines

Did you ever send out such a patch? I didn't see one on the list, but perhaps I missed it. One definitely hasn't made it upstream yet. (I just hit the issue myself with Linus's tree from last night.)

Actually, that wants a bit more methinks, because while the stack dump goes away, you still get spammed, it just comes in smaller chunks.

OK, well this has to either be fixed or reverted. Right now it's complaining all the time for me after like a day of uptime.

I've already send out a patch to Konrad Rzeszutek Wilk and he wanted to queue that up.

But there is another warning I'm currently working on, just didn't had time to during the holidays.

Regards,
Christian.
_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel