drm/amdgpu: introduce new amdgpu_fence object to indicate the job embedded fence
Subsystems (3)
PATCH v2 1/1 drm/amdgpu: introduce new amdgpu_fence object to indicate the job embedded fence
The job embedded fence donesn't initialize the flags at dma_fence_init(). Then we will go a wrong way in amdgpu_fence_get_timeline_name callback and trigger a null pointer panic once we enabled the trace event here. So introduce new amdgpu_fence object to indicate the job embedded fence.
[ 156.131790] BUG: kernel NULL pointer dereference, address: 00000000000002a0 [ 156.131804] #PF: supervisor read access in kernel mode [ 156.131811] #PF: error_code(0x0000) - not-present page [ 156.131817] PGD 0 P4D 0 [ 156.131824] Oops: 0000 [#1] PREEMPT SMP PTI [ 156.131832] CPU: 6 PID: 1404 Comm: sdma0 Tainted: G OE 5.16.0-rc1-custom #1 [ 156.131842] Hardware name: Gigabyte Technology Co., Ltd. Z170XP-SLI/Z170XP-SLI-CF, BIOS F20 11/04/2016 [ 156.131848] RIP: 0010:strlen+0x0/0x20 [ 156.131859] Code: 89 c0 c3 0f 1f 80 00 00 00 00 48 01 fe eb 0f 0f b6 07 38 d0 74 10 48 83 c7 01 84 c0 74 05 48 39 f7 75 ec 31 c0 c3 48 89 f8 c3 <80> 3f 00 74 10 48 89 f8 48 83 c0 01 80 38 00 75 f7 48 29 f8 c3 31 [ 156.131872] RSP: 0018:ffff9bd0018dbcf8 EFLAGS: 00010206 [ 156.131880] RAX: 00000000000002a0 RBX: ffff8d0305ef01b0 RCX: 000000000000000b [ 156.131888] RDX: ffff8d03772ab924 RSI: ffff8d0305ef01b0 RDI: 00000000000002a0 [ 156.131895] RBP: ffff9bd0018dbd60 R08: ffff8d03002094d0 R09: 0000000000000000 [ 156.131901] R10: 000000000000005e R11: 0000000000000065 R12: ffff8d03002094d0 [ 156.131907] R13: 000000000000001f R14: 0000000000070018 R15: 0000000000000007 [ 156.131914] FS: 0000000000000000(0000) GS:ffff8d062ed80000(0000) knlGS:0000000000000000 [ 156.131923] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 156.131929] CR2: 00000000000002a0 CR3: 000000001120a005 CR4: 00000000003706e0 [ 156.131937] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 156.131942] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 [ 156.131949] Call Trace: [ 156.131953] <TASK> [ 156.131957] ? trace_event_raw_event_dma_fence+0xcc/0x200 [ 156.131973] ? ring_buffer_unlock_commit+0x23/0x130 [ 156.131982] dma_fence_init+0x92/0xb0 [ 156.131993] amdgpu_fence_emit+0x10d/0x2b0 [amdgpu] [ 156.132302] amdgpu_ib_schedule+0x2f9/0x580 [amdgpu] [ 156.132586] amdgpu_job_run+0xed/0x220 [amdgpu]
--- drivers/gpu/drm/amd/amdgpu/amdgpu.h | 1 + drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 3 +- drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c | 117 ++++++++++++++------- drivers/gpu/drm/amd/amdgpu/amdgpu_ring.h | 3 - 4 files changed, 80 insertions(+), 44 deletions(-)
drivers/gpu/drm/amd/amdgpu/amdgpu.h +1 -0
@@ -444,6 +444,7 @@ struct amdgpu_sa_bo {int amdgpu_fence_slab_init (void );void amdgpu_fence_slab_fini (void ); +bool is_job_embedded_fence (struct dma_fence *f);/* * IRQS.
drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +1 -2
@@ -4483,9 +4483,8 @@ int amdgpu_device_pre_asic_reset(struct amdgpu_device *adev, ptr = &ring->fence_drv.fences[j]; old =rcu_dereference_protected (*ptr,1 ); -if (old &&test_bit (AMDGPU_FENCE_FLAG_EMBED_IN_JOB_BIT , &old->flags)) { +if (old &&is_job_embedded_fence (old))RCU_INIT_POINTER (*ptr,NULL ); - } }/* after all hw jobs are reset, hw fence is meaningless, so force_completion */ amdgpu_fence_driver_force_completion (ring);
drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c +78 -39
178 lines not on this page · raw message
drivers/gpu/drm/amd/amdgpu/amdgpu_ring.h +0 -3
@@ -53,9 +53,6 @@ enum amdgpu_ring_priority_level {#define AMDGPU_FENCE_FLAG_INT (1 << 1)#define AMDGPU_FENCE_FLAG_TC_WB_ONLY (1 << 2) -/* fence flag bit to indicate the face is embedded in job*/ -#define AMDGPU_FENCE_FLAG_EMBED_IN_JOB_BIT (DMA_FENCE_FLAG_USER_BITS + 1) -#define to_amdgpu_ring (s) container_of((s), struct amdgpu_ring, sched)#define AMDGPU_IB_POOL_SIZE (1024 * 1024)
Signature · 1 line
2.25.1
Christian König
maintainerAm 14.12.21 um 12:15 schrieb Huang Rui:
Quoted 48 lines
The job embedded fence donesn't initialize the flags at
dma_fence_init(). Then we will go a wrong way inamdgpu_fence_get_timeline_namecallback and trigger a null pointer panic once we enabled the trace event here. So introduce newamdgpu_fenceobject to indicate the job embedded fence.[ 156.131790] BUG: kernel NULL pointer dereference, address: 00000000000002a0 [ 156.131804] #PF: supervisor read access in kernel mode [ 156.131811] #PF: error_code(0x0000) - not-present page [ 156.131817] PGD 0 P4D 0 [ 156.131824] Oops: 0000 [#1] PREEMPT SMP PTI [ 156.131832] CPU: 6 PID: 1404 Comm: sdma0 Tainted: G OE 5.16.0-rc1-custom #1 [ 156.131842] Hardware name: Gigabyte Technology Co., Ltd. Z170XP-SLI/Z170XP-SLI-CF, BIOS F20 11/04/2016 [ 156.131848] RIP: 0010:strlen+0x0/0x20 [ 156.131859] Code: 89 c0 c3 0f 1f 80 00 00 00 00 48 01 fe eb 0f 0f b6 07 38 d0 74 10 48 83 c7 01 84 c0 74 05 48 39 f7 75 ec 31 c0 c3 48 89 f8 c3 <80> 3f 00 74 10 48 89 f8 48 83 c0 01 80 38 00 75 f7 48 29 f8 c3 31 [ 156.131872] RSP: 0018:ffff9bd0018dbcf8 EFLAGS: 00010206 [ 156.131880] RAX: 00000000000002a0 RBX: ffff8d0305ef01b0 RCX: 000000000000000b [ 156.131888] RDX: ffff8d03772ab924 RSI: ffff8d0305ef01b0 RDI: 00000000000002a0 [ 156.131895] RBP: ffff9bd0018dbd60 R08: ffff8d03002094d0 R09: 0000000000000000 [ 156.131901] R10: 000000000000005e R11: 0000000000000065 R12: ffff8d03002094d0 [ 156.131907] R13: 000000000000001f R14: 0000000000070018 R15: 0000000000000007 [ 156.131914] FS: 0000000000000000(0000) GS:ffff8d062ed80000(0000) knlGS:0000000000000000 [ 156.131923] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 156.131929] CR2: 00000000000002a0 CR3: 000000001120a005 CR4: 00000000003706e0 [ 156.131937] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 156.131942] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 [ 156.131949] Call Trace: [ 156.131953] <TASK> [ 156.131957] ? trace_event_raw_event_dma_fence+0xcc/0x200 [ 156.131973] ? ring_buffer_unlock_commit+0x23/0x130 [ 156.131982] dma_fence_init+0x92/0xb0 [ 156.131993] amdgpu_fence_emit+0x10d/0x2b0 [amdgpu] [ 156.132302] amdgpu_ib_schedule+0x2f9/0x580 [amdgpu] [ 156.132586] amdgpu_job_run+0xed/0x220 [amdgpu]Signed-off-by Huang Rui <[email protected]>--- drivers/gpu/drm/amd/amdgpu/amdgpu.h | 1 + drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 3 +- drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c | 117 ++++++++++++++------- drivers/gpu/drm/amd/amdgpu/amdgpu_ring.h | 3 - 4 files changed, 80 insertions(+), 44 deletions(-)drivers/gpu/drm/amd/amdgpu/amdgpu.h +0
-0
drivers/gpu/drm/amd/amdgpu/amdgpu.h struct amdgpu_sa_bo { · line 447
Quoted 4 lines
drivers/gpu/drm/amd/amdgpu/amdgpu.h +1
-0@@ -444,6 +444,7 @@ struct amdgpu_sa_bo {int amdgpu_fence_slab_init (void );void amdgpu_fence_slab_fini (void ); +bool is_job_embedded_fence (struct dma_fence *f);
We need a better name for this, especially with amdgpu in it. Something like is_amdgpu_job_fence().
But maybe we can avoid that function alltogether, see below.
Quoted 8 lines
drivers/gpu/drm/amd/amdgpu/amdgpu.h +0
-0/* * IRQS. drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +0
-0
drivers/gpu/drm/amd/amdgpu/amdgpu_device.c int amdgpu_device_pre_asic_reset(struct amdgpu_device *adev, · line 4488
Quoted 7 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +1
-2@@ -4483,9 +4483,8 @@ int amdgpu_device_pre_asic_reset(struct amdgpu_device *adev, ptr = &ring->fence_drv.fences[j]; old =rcu_dereference_protected (*ptr,1 ); -if (old &&test_bit (AMDGPU_FENCE_FLAG_EMBED_IN_JOB_BIT , &old->flags)) { +if (old &&is_job_embedded_fence (old))RCU_INIT_POINTER (*ptr,NULL ); - }
This here is messing with the fence internals and so should probably be a function in amdgpu_fence.c.
This way we would have embedded the amdgpu fence in there as well. Apart from that looks rather good to me.
Christian.
Quoted 206 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +0
-0}/* after all hw jobs are reset, hw fence is meaningless, so force_completion */ amdgpu_fence_driver_force_completion (ring);drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c +78
-39@@ -77,16 +77,28 @@ void amdgpu_fence_slab_fini(void)* Cast helper */ static const struct dma_fence_ops amdgpu_fence_ops; +static const struct dma_fence_ops amdgpu_job_fence_ops;static inline struct amdgpu_fence *to_amdgpu_fence (struct dma_fence *f) { struct amdgpu_fence *__f = container_of(f, struct amdgpu_fence, base); -if (__f->base.ops == &amdgpu_fence_ops) +if (__f->base.ops == &amdgpu_fence_ops || + __f->base.ops == &amdgpu_job_fence_ops)return __f;return NULL ; } +bool is_job_embedded_fence (struct dma_fence *f) +{ + struct amdgpu_fence *__f = container_of(f, struct amdgpu_fence, base); + +if (__f->base.ops == &amdgpu_job_fence_ops) +return true; + +return false; +} +/** * amdgpu_fence_write - write a fence value * @@ -158,19 +170,18 @@ int amdgpu_fence_emit(struct amdgpu_ring *ring, struct dma_fence **f, struct amd } seq = ++ring->fence_drv.sync_seq; -if (job !=NULL && job->job_run_counter) { +if (job && job->job_run_counter) {/* reinit seq for resubmitted jobs */ fence->seqno = seq; }else { -dma_fence_init (fence, &amdgpu_fence_ops, - &ring->fence_drv.lock, - adev->fence_context + ring->idx, - seq); - } - -if (job !=NULL ) { -/* mark this fence has a parent job */ -set_bit (AMDGPU_FENCE_FLAG_EMBED_IN_JOB_BIT , &fence->flags); +if (job) +dma_fence_init (fence, &amdgpu_job_fence_ops, + &ring->fence_drv.lock, + adev->fence_context + ring->idx, seq); +else +dma_fence_init (fence, &amdgpu_fence_ops, + &ring->fence_drv.lock, + adev->fence_context + ring->idx, seq); } amdgpu_ring_emit_fence(ring, ring->fence_drv.gpu_addr,@@ -643,16 +654,14 @@ static const char *amdgpu_fence_get_driver_name(struct dma_fence *fence)static const char *amdgpu_fence_get_timeline_name (struct dma_fence *f) { -struct amdgpu_ring *ring; +return (const char *)to_amdgpu_fence (f)->ring->name; +} -if (test_bit (AMDGPU_FENCE_FLAG_EMBED_IN_JOB_BIT , &f->flags)) { - struct amdgpu_job *job = container_of(f, struct amdgpu_job, hw_fence); +static const char *amdgpu_job_fence_get_timeline_name (struct dma_fence *f) +{ + struct amdgpu_job *job = container_of(f, struct amdgpu_job, hw_fence); - ring =to_amdgpu_ring (job->base.sched); - }else { - ring =to_amdgpu_fence (f)->ring; - } -return (const char *)ring->name; +return (const char *)to_amdgpu_ring (job->base.sched)->name; }/** @@ -665,18 +674,25 @@ static const char *amdgpu_fence_get_timeline_name(struct dma_fence *f)*/ static bool amdgpu_fence_enable_signaling (struct dma_fence *f) { -struct amdgpu_ring *ring; +if (!timer_pending (&to_amdgpu_fence (f)->ring->fence_drv.fallback_timer)) +amdgpu_fence_schedule_fallback (to_amdgpu_fence (f)->ring); -if (test_bit (AMDGPU_FENCE_FLAG_EMBED_IN_JOB_BIT , &f->flags)) { - struct amdgpu_job *job = container_of(f, struct amdgpu_job, hw_fence); +return true; +} - ring =to_amdgpu_ring (job->base.sched); - }else { - ring =to_amdgpu_fence (f)->ring; - } +/** +* amdgpu_job_fence_enable_signaling - enable signalling on job fence +* @f: fence +* +* This is the simliar function with amdgpu_fence_enable_signaling above, it +* only handles the job embedded fence. +*/ +static bool amdgpu_job_fence_enable_signaling (struct dma_fence *f) +{ + struct amdgpu_job *job = container_of(f, struct amdgpu_job, hw_fence); -if (!timer_pending (&ring->fence_drv.fallback_timer)) -amdgpu_fence_schedule_fallback (ring); +if (!timer_pending (&to_amdgpu_ring (job->base.sched)->fence_drv.fallback_timer)) +amdgpu_fence_schedule_fallback (to_amdgpu_ring (job->base.sched));return true; }@@ -692,19 +708,23 @@ static void amdgpu_fence_free(struct rcu_head *rcu) { struct dma_fence *f = container_of(rcu, struct dma_fence, rcu); -if (test_bit (AMDGPU_FENCE_FLAG_EMBED_IN_JOB_BIT , &f->flags)) { -/* free job if fence has a parent job */ -struct amdgpu_job *job; - - job = container_of(f, struct amdgpu_job, hw_fence); -kfree (job); - }else {/* free fence_slab if it's separated fence*/ -struct amdgpu_fence *fence; +kmem_cache_free (amdgpu_fence_slab,to_amdgpu_fence (f)); +} - fence =to_amdgpu_fence (f); -kmem_cache_free (amdgpu_fence_slab, fence); - } +/** +* amdgpu_job_fence_free - free up the job with embedded fence +* +* @rcu: RCU callback head +* +* Free up the job with embedded fence after the RCU grace period. +*/ +static void amdgpu_job_fence_free (struct rcu_head *rcu) +{ + struct dma_fence *f = container_of(rcu, struct dma_fence, rcu); + +/* free job if fence has a parent job */ +kfree (container_of(f ,struct amdgpu_job ,hw_fence )); }/** @@ -720,6 +740,19 @@ static void amdgpu_fence_release(struct dma_fence *f)call_rcu (&f->rcu, amdgpu_fence_free); } +/** +* amdgpu_job_fence_release - callback that job embedded fence can be freed +* +* @f: fence +* +* This is the simliar function with amdgpu_fence_release above, it +* only handles the job embedded fence. +*/ +static void amdgpu_job_fence_release (struct dma_fence *f) +{ +call_rcu (&f->rcu, amdgpu_job_fence_free); +} +static const struct dma_fence_ops amdgpu_fence_ops = { .get_driver_name = amdgpu_fence_get_driver_name, .get_timeline_name = amdgpu_fence_get_timeline_name,@@ -727,6 +760,12 @@ static const struct dma_fence_ops amdgpu_fence_ops = { .release = amdgpu_fence_release, }; +static const struct dma_fence_ops amdgpu_job_fence_ops = { + .get_driver_name = amdgpu_fence_get_driver_name, + .get_timeline_name = amdgpu_job_fence_get_timeline_name, + .enable_signaling = amdgpu_job_fence_enable_signaling, + .release = amdgpu_job_fence_release, +};/* * Fence debugfs drivers/gpu/drm/amd/amdgpu/amdgpu_ring.h +0
-3@@ -53,9 +53,6 @@ enum amdgpu_ring_priority_level {#define AMDGPU_FENCE_FLAG_INT (1 << 1)#define AMDGPU_FENCE_FLAG_TC_WB_ONLY (1 << 2) -/* fence flag bit to indicate the face is embedded in job*/ -#define AMDGPU_FENCE_FLAG_EMBED_IN_JOB_BIT (DMA_FENCE_FLAG_USER_BITS + 1) -#define to_amdgpu_ring (s) container_of((s), struct amdgpu_ring, sched)#define AMDGPU_IB_POOL_SIZE (1024 * 1024)
[AMD Official Use Only]
Quoted 102 lines
-----Original Message-----
From: Koenig, Christian <[email protected]>
Sent: Tuesday, December 14, 2021 8:26 PM
To: Huang, Ray <[email protected]>; [email protected]; Daniel Vetter <[email protected]>; Sumit Semwal <[email protected]>
Cc: [email protected]; [email protected]; Deucher, Alexander <[email protected]>; Liu, Monk <[email protected]>
Subject: Re: [PATCH v2] drm/amdgpu: introduce newamdgpu_fenceobject to indicate the job embedded fenceAm 14.12.21 um 12:15 schrieb Huang Rui:
Quoted 11 lines
The job embedded fence donesn't initialize the flags at
dma_fence_init(). Then we will go a wrong way inamdgpu_fence_get_timeline_namecallback and trigger a null pointer panic once we enabled the trace event here. So introduce newamdgpu_fenceobject to indicate the job embedded fence.[ 156.131790] BUG: kernel NULL pointer dereference, address: 00000000000002a0 [ 156.131804] #PF: supervisor read access in kernel mode [ 156.131811] #PF: error_code(0x0000) - not-present page [ 156.131817] PGD 0 P4D 0 [ 156.131824] Oops: 0000 [#1] PREEMPT SMP PTI [ 156.131832] CPU: 6 PID: 1404 Comm: sdma0 Tainted: G OE 5.16.0-rc1-custom #1
Quoted 15 lines
[ 156.131842] Hardware name: Gigabyte Technology Co., Ltd.Z170XP-SLI/Z170XP-SLI-CF, BIOS F20 11/04/2016 [ 156.131848] RIP:
0010:strlen+0x0/0x20 [ 156.131859] Code: 89 c0 c3 0f 1f 80 00 00 00 00 48 01 fe eb 0f 0f b6 07 38 d0 74 10 48 83 c7 01 84 c0 74 05 48 39 f7 75 ec 31 c0 c3 48 89 f8 c3 <80> 3f 00 74 10 48 89 f8 48 83 c0 01 80 38 00 75 f7 48 29 f8 c3 31 [ 156.131872] RSP: 0018:ffff9bd0018dbcf8
EFLAGS: 00010206 [ 156.131880] RAX:00000000000002a0RBX:ffff8d0305ef01b0RCX:000000000000000b[ 156.131888] RDX:ffff8d03772ab924RSI:ffff8d0305ef01b0RDI:00000000000002a0[ 156.131895] RBP:ffff9bd0018dbd60R08:ffff8d03002094d0R09:
0000000000000000 [ 156.131901] R10:000000000000005eR11:
0000000000000065 R12:ffff8d03002094d0[ 156.131907] R13:000000000000001fR14: 0000000000070018 R15: 0000000000000007 [ 156.131914] FS: 0000000000000000(0000) GS:ffff8d062ed80000(0000) knlGS:0000000000000000 [ 156.131923] CS: 0010 DS: 0000 ES: 0000 CR0:0000000080050033 [ 156.131929] CR2:
00000000000002a0CR3:000000001120a005CR4:00000000003706e0[ 156.131937] DR0:
0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000[ 156.131942] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7:0000000000000400 [ 156.131949] Call Trace:
Quoted 8 lines
[ 156.131953] <TASK> [ 156.131957] ? trace_event_raw_event_dma_fence+0xcc/0x200 [ 156.131973] ? ring_buffer_unlock_commit+0x23/0x130 [ 156.131982] dma_fence_init+0x92/0xb0 [ 156.131993] amdgpu_fence_emit+0x10d/0x2b0 [amdgpu] [ 156.132302] amdgpu_ib_schedule+0x2f9/0x580 [amdgpu] [ 156.132586] amdgpu_job_run+0xed/0x220 [amdgpu]Quoted 5 lines
Signed-off-by Huang Rui <[email protected]>--- drivers/gpu/drm/amd/amdgpu/amdgpu.h | 1 + drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 3 +- drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c | 117++++++++++++++-------
drivers/gpu/drm/amd/amdgpu/amdgpu.h struct amdgpu_sa_bo { · line 447
Quoted 13 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_ring.h | 3 - 4 files changed, 80 insertions(+), 44 deletions(-)drivers/gpu/drm/amd/amdgpu/amdgpu.h +1
-0@@ -444,6 +444,7 @@ struct amdgpu_sa_bo {int amdgpu_fence_slab_init (void );void amdgpu_fence_slab_fini (void ); +bool is_job_embedded_fence (struct dma_fence *f);We need a better name for this, especially with amdgpu in it. Something like
is_amdgpu_job_fence().But maybe we can avoid that function alltogether, see below.
Quoted 9 lines
drivers/gpu/drm/amd/amdgpu/amdgpu.h +0
-0/* * IRQS. drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +0
-0drivers/gpu/drm/amd/amdgpu/amdgpu_device.c int amdgpu_device_pre_asic_reset(struct · line 4487
Quoted 5 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +0
-1@@ -4483,9 +4483,8 @@ int amdgpu_device_pre_asic_reset(structamdgpu_device *adev, ptr = &ring->fence_drv.fences[j]; old =rcu_dereference_protected (*ptr,1 ); - if (old &&
test_bit(AMDGPU_FENCE_FLAG_EMBED_IN_JOB_BIT, &old->flags)) {drivers/gpu/drm/amd/amdgpu/amdgpu_device.c int amdgpu_device_pre_asic_reset(struct amdgpu_device *adev, · line 4488
Quoted 3 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +1
-1+if (old &&is_job_embedded_fence (old))RCU_INIT_POINTER (*ptr,NULL ); - }This here is messing with the fence internals and so should probably be a function in
amdgpu_fence.c.
Quoted 3 lines
This way we would have embedded the amdgpu fence in there as well. Apart from that looks rather good to me.
So we can create new function in amdgpu_fence.c to implement the job fence clearing here and call it here instead. That is pure job fence operation, so we won't need to put it in amdgpu_device.c.
Thanks,
Ray
Quoted 230 lines
Christian.
drivers/gpu/drm/amd/amdgpu/amdgpu_device.c int amdgpu_device_pre_asic_reset(struct · line 4490
Quoted 2 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +0
-0}/* after all hw jobs are reset, hw fence is meaningless, so
force_completion*/drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c void amdgpu_fence_slab_fini(void) · line 81
Quoted 12 lines
amdgpu_fence_driver_force_completion(ring);drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c +1
-0@@ -77,16 +77,28 @@ void amdgpu_fence_slab_fini(void)* Cast helper */ static const struct dma_fence_ops amdgpu_fence_ops; +static const struct dma_fence_ops amdgpu_job_fence_ops; static inline struct amdgpu_fence *to_amdgpu_fence(struct dma_fence*f)
drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c int amdgpu_fence_emit(struct amdgpu_ring
Quoted 26 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c +12
-1{ struct amdgpu_fence *__f = container_of(f, struct amdgpu_fence, base); -if (__f->base.ops == &amdgpu_fence_ops) +if (__f->base.ops == &amdgpu_fence_ops || + __f->base.ops == &amdgpu_job_fence_ops)return __f;return NULL ; } +bool is_job_embedded_fence (struct dma_fence *f) { + struct amdgpu_fence *__f = container_of(f, struct amdgpu_fence, +base); + +if (__f->base.ops == &amdgpu_job_fence_ops) +return true; + +return false; +} +/** * amdgpu_fence_write - write a fence value *drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c +0
-0@@ -158,19 +170,18 @@ int amdgpu_fence_emit(struct amdgpu_ring*ring, struct
dma_fence**f, struct amdQuoted 7 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c +1
-1} seq = ++ring->fence_drv.sync_seq; -if (job !=NULL && job->job_run_counter) { +if (job && job->job_run_counter) {/* reinit seq for resubmitted jobs */ fence->seqno = seq;drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c int amdgpu_fence_emit(struct amdgpu_ring *ring, struct dma_fence **f, struct amd
Quoted 10 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c +0
-9}else { -dma_fence_init (fence, &amdgpu_fence_ops, - &ring->fence_drv.lock, - adev->fence_context + ring->idx, - seq); - } - -if (job !=NULL ) { -/* mark this fence has a parent job */ - set_bit(AMDGPU_FENCE_FLAG_EMBED_IN_JOB_BIT,
&fence->flags);drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c int amdgpu_fence_emit(struct amdgpu_ring *ring, struct dma_fence **f, struct amd
Quoted 11 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c +8
-0+if (job) +dma_fence_init (fence, &amdgpu_job_fence_ops, + &ring->fence_drv.lock, + adev->fence_context + ring->idx, seq); +else +dma_fence_init (fence, &amdgpu_fence_ops, + &ring->fence_drv.lock, + adev->fence_context + ring->idx, seq); } amdgpu_ring_emit_fence(ring, ring->fence_drv.gpu_addr, @@ -643,16
Quoted 4 lines
+654,14 @@ static const char *amdgpu_fence_get_driver_name(struct
dma_fence*fence)static const char *amdgpu_fence_get_timeline_name(struct
dma_fence*f)
drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c static const char *amdgpu_fence_get_driver_name(struct dma_fence *fence)
Quoted 5 lines
{drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c +1
-2-struct amdgpu_ring *ring; +return (const char *)to_amdgpu_fence (f)->ring->name; } -if (test_bit (AMDGPU_FENCE_FLAG_EMBED_IN_JOB_BIT , &f->flags)){
drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c
Quoted 1 line
drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c +0
-1- struct amdgpu_job *job = container_of(f, struct amdgpu_job,
hw_fence);drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c static const char
Quoted 15 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c +5
-5+static const char *amdgpu_job_fence_get_timeline_name (struct +dma_fence *f) { + struct amdgpu_job *job = container_of(f, struct amdgpu_job, +hw_fence); - ring =to_amdgpu_ring (job->base.sched); - }else { - ring =to_amdgpu_fence (f)->ring; - } -return (const char *)ring->name; +return (const char *)to_amdgpu_ring (job->base.sched)->name; }/** @@ -665,18 +674,25 @@ static const char*amdgpu_fence_get_timeline_name(struct
dma_fence*f)drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c static const char *amdgpu_fence_get_timeline_name(struct dma_fence *f)
Quoted 10 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c +2
-2*/ static bool amdgpu_fence_enable_signaling (struct dma_fence *f) { -struct amdgpu_ring *ring; +if (!timer_pending (&to_amdgpu_fence (f)->ring- fence_drv.fallback_timer)) +amdgpu_fence_schedule_fallback (to_amdgpu_fence (f)- ring); -if (test_bit (AMDGPU_FENCE_FLAG_EMBED_IN_JOB_BIT , &f->flags)){
drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c static const char · line 673
Quoted 1 line
drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c +0
-1- struct amdgpu_job *job = container_of(f, struct amdgpu_job,
hw_fence);drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c static void amdgpu_fence_free(struct rcu_head
Quoted 29 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c +15
-6+return true; +} - ring =to_amdgpu_ring (job->base.sched); - }else { - ring =to_amdgpu_fence (f)->ring; - } +/** +* amdgpu_job_fence_enable_signaling - enable signalling on job fence +* @f: fence +* +* This is the simliar function with amdgpu_fence_enable_signaling +above, it +* only handles the job embedded fence. +*/ +static bool amdgpu_job_fence_enable_signaling (struct dma_fence *f) { + struct amdgpu_job *job = container_of(f, struct amdgpu_job, +hw_fence); -if (!timer_pending (&ring->fence_drv.fallback_timer)) -amdgpu_fence_schedule_fallback (ring); +if (!timer_pending (&to_amdgpu_ring (job->base.sched)- fence_drv.fallback_timer)) + amdgpu_fence_schedule_fallback(to_amdgpu_ring(job-base.sched));
return true; }drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c +0
-0@@ -692,19 +708,23 @@ static void amdgpu_fence_free(struct rcu_head*rcu)
drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c static void amdgpu_fence_free(struct rcu_head *rcu)
Quoted 4 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c +0
-1{ struct dma_fence *f = container_of(rcu, struct dma_fence, rcu); -if (test_bit (AMDGPU_FENCE_FLAG_EMBED_IN_JOB_BIT , &f->flags)){
drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c static void amdgpu_fence_release(struct
Quoted 29 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c +13
-10-/* free job if fence has a parent job */ -struct amdgpu_job *job; - - job = container_of(f, struct amdgpu_job, hw_fence); -kfree (job); - }else {/* free fence_slab if it's separated fence*/ -struct amdgpu_fence *fence; +kmem_cache_free (amdgpu_fence_slab,to_amdgpu_fence (f)); } - fence =to_amdgpu_fence (f); -kmem_cache_free (amdgpu_fence_slab, fence); - } +/** +* amdgpu_job_fence_free - free up the job with embedded fence +* +* @rcu: RCU callback head +* +* Free up the job with embedded fence after the RCU grace period. +*/ +static void amdgpu_job_fence_free (struct rcu_head *rcu) { + struct dma_fence *f = container_of(rcu, struct dma_fence, rcu); + +/* free job if fence has a parent job */ +kfree (container_of(f ,struct amdgpu_job ,hw_fence )); }/** @@ -720,6 +740,19 @@ static void amdgpu_fence_release(struct
dma_fence*f)Quoted 8 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c +5
-0call_rcu (&f->rcu, amdgpu_fence_free); } +/** +* amdgpu_job_fence_release - callback that job embedded fence can be +freed +* +* @f: fence drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c static void amdgpu_fence_release(struct dma_fence *f)
Quoted 10 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c +7
-0+* +* This is the simliar function with amdgpu_fence_release above, it +* only handles the job embedded fence. +*/ +static void amdgpu_job_fence_release (struct dma_fence *f) { +call_rcu (&f->rcu, amdgpu_job_fence_free); } +static const struct dma_fence_ops amdgpu_fence_ops = { .get_driver_name = amdgpu_fence_get_driver_name, .get_timeline_name = amdgpu_fence_get_timeline_name, @@ -727,6
Quoted 18 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c +1
-0+760,12 @@ static const struct dma_fence_ops amdgpu_fence_ops = {.release = amdgpu_fence_release, };drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c +5
-0+static const struct dma_fence_ops amdgpu_job_fence_ops = { + .get_driver_name = amdgpu_fence_get_driver_name, + .get_timeline_name = amdgpu_job_fence_get_timeline_name, + .enable_signaling = amdgpu_job_fence_enable_signaling, + .release = amdgpu_job_fence_release, };/* * Fence debugfs drivers/gpu/drm/amd/amdgpu/amdgpu_ring.h +0
-0drivers/gpu/drm/amd/amdgpu/amdgpu_ring.h enum amdgpu_ring_priority_level { · line 57
Quoted 5 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_ring.h +0
-2@@ -53,9 +53,6 @@ enum amdgpu_ring_priority_level {#define AMDGPU_FENCE_FLAG_INT (1 << 1)#define AMDGPU_FENCE_FLAG_TC_WB_ONLY (1 << 2) -/* fence flag bit to indicate the face is embedded in job*/ -#define AMDGPU_FENCE_FLAG_EMBED_IN_JOB_BIT (DMA_FENCE_FLAG_USER_BITS + 1)Quoted 5 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_ring.h +0
-1-#define to_amdgpu_ring (s) container_of((s), struct amdgpu_ring, sched)#define AMDGPU_IB_POOL_SIZE (1024 * 1024)
Quoted 121 lines · Huang Rui
The job embedded fence donesn't initialize the flags at
dma_fence_init(). Then we will go a wrong way inamdgpu_fence_get_timeline_namecallback and trigger a null pointer panic once we enabled the trace event here. So introduce newamdgpu_fenceobject to indicate the job embedded fence.[ 156.131790] BUG: kernel NULL pointer dereference, address: 00000000000002a0 [ 156.131804] #PF: supervisor read access in kernel mode [ 156.131811] #PF: error_code(0x0000) - not-present page [ 156.131817] PGD 0 P4D 0 [ 156.131824] Oops: 0000 [#1] PREEMPT SMP PTI [ 156.131832] CPU: 6 PID: 1404 Comm: sdma0 Tainted: G OE 5.16.0-rc1-custom #1 [ 156.131842] Hardware name: Gigabyte Technology Co., Ltd. Z170XP-SLI/Z170XP-SLI-CF, BIOS F20 11/04/2016 [ 156.131848] RIP: 0010:strlen+0x0/0x20 [ 156.131859] Code: 89 c0 c3 0f 1f 80 00 00 00 00 48 01 fe eb 0f 0f b6 07 38 d0 74 10 48 83 c7 01 84 c0 74 05 48 39 f7 75 ec 31 c0 c3 48 89 f8 c3 <80> 3f 00 74 10 48 89 f8 48 83 c0 01 80 38 00 75 f7 48 29 f8 c3 31 [ 156.131872] RSP: 0018:ffff9bd0018dbcf8 EFLAGS: 00010206 [ 156.131880] RAX: 00000000000002a0 RBX: ffff8d0305ef01b0 RCX: 000000000000000b [ 156.131888] RDX: ffff8d03772ab924 RSI: ffff8d0305ef01b0 RDI: 00000000000002a0 [ 156.131895] RBP: ffff9bd0018dbd60 R08: ffff8d03002094d0 R09: 0000000000000000 [ 156.131901] R10: 000000000000005e R11: 0000000000000065 R12: ffff8d03002094d0 [ 156.131907] R13: 000000000000001f R14: 0000000000070018 R15: 0000000000000007 [ 156.131914] FS: 0000000000000000(0000) GS:ffff8d062ed80000(0000) knlGS:0000000000000000 [ 156.131923] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 156.131929] CR2: 000000amdgpu_job_alloc00000002a0 CR3: 000000001120a005 CR4: 00000000003706e0 [ 156.131937] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 156.131942] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 [ 156.131949] Call Trace: [ 156.131953] <TASK> [ 156.131957] ? trace_event_raw_event_dma_fence+0xcc/0x200 [ 156.131973] ? ring_buffer_unlock_commit+0x23/0x130 [ 156.131982] dma_fence_init+0x92/0xb0 [ 156.131993] amdgpu_fence_emit+0x10d/0x2b0 [amdgpu] [ 156.132302] amdgpu_ib_schedule+0x2f9/0x580 [amdgpu] [ 156.132586] amdgpu_job_run+0xed/0x220 [amdgpu]Signed-off-by Huang Rui <[email protected]>--- drivers/gpu/drm/amd/amdgpu/amdgpu.h | 1 + drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 3 +- drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c | 117 ++++++++++++++------- drivers/gpu/drm/amd/amdgpu/amdgpu_ring.h | 3 - 4 files changed, 80 insertions(+), 44 deletions(-)drivers/gpu/drm/amd/amdgpu/amdgpu.h +1
-0@@ -444,6 +444,7 @@ struct amdgpu_sa_bo {int amdgpu_fence_slab_init (void );void amdgpu_fence_slab_fini (void ); +bool is_job_embedded_fence (struct dma_fence *f);/* * IRQS. drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +1
-2@@ -4483,9 +4483,8 @@ int amdgpu_device_pre_asic_reset(struct amdgpu_device *adev, ptr = &ring->fence_drv.fences[j]; old =rcu_dereference_protected (*ptr,1 ); -if (old &&test_bit (AMDGPU_FENCE_FLAG_EMBED_IN_JOB_BIT , &old->flags)) { +if (old &&is_job_embedded_fence (old))RCU_INIT_POINTER (*ptr,NULL ); - } }/* after all hw jobs are reset, hw fence is meaningless, so force_completion */ amdgpu_fence_driver_force_completion (ring);drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c +14
-10@@ -77,16 +77,28 @@ void amdgpu_fence_slab_fini(void)* Cast helper */ static const struct dma_fence_ops amdgpu_fence_ops; +static const struct dma_fence_ops amdgpu_job_fence_ops;static inline struct amdgpu_fence *to_amdgpu_fence (struct dma_fence *f) { struct amdgpu_fence *__f = container_of(f, struct amdgpu_fence, base); -if (__f->base.ops == &amdgpu_fence_ops) +if (__f->base.ops == &amdgpu_fence_ops || + __f->base.ops == &amdgpu_job_fence_ops)return __f;return NULL ; } +bool is_job_embedded_fence (struct dma_fence *f) +{ + struct amdgpu_fence *__f = container_of(f, struct amdgpu_fence, base); + +if (__f->base.ops == &amdgpu_job_fence_ops) +return true; + +return false; +} +/** * amdgpu_fence_write - write a fence value * @@ -158,19 +170,18 @@ int amdgpu_fence_emit(struct amdgpu_ring *ring, struct dma_fence **f, struct amd } seq = ++ring->fence_drv.sync_seq; -if (job !=NULL && job->job_run_counter) { +if (job && job->job_run_counter) {/* reinit seq for resubmitted jobs */ fence->seqno = seq; }else { -dma_fence_init (fence, &amdgpu_fence_ops, - &ring->fence_drv.lock, - adev->fence_context + ring->idx, - seq); - } - -if (job !=NULL ) { -/* mark this fence has a parent job */
drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c int amdgpu_fence_emit(struct amdgpu_ring *ring, struct dma_fence **f, struct amd · line 185
Quoted 10 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c +8
-1-set_bit (AMDGPU_FENCE_FLAG_EMBED_IN_JOB_BIT , &fence->flags); +if (job) +dma_fence_init (fence, &amdgpu_job_fence_ops, + &ring->fence_drv.lock, + adev->fence_context + ring->idx, seq); +else +dma_fence_init (fence, &amdgpu_fence_ops, + &ring->fence_drv.lock, + adev->fence_context + ring->idx, seq); }
It's probably me missing something but why can't we just move setting of AMDGPU_FENCE_FLAG_EMBED_IN_JOB_BIT to before dma_fence_init here or even into amdgpu_job_alloc instead of all the refactoring ?
Andrey
Quoted 142 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c +56
-28amdgpu_ring_emit_fence(ring, ring->fence_drv.gpu_addr,@@ -643,16 +654,14 @@ static const char *amdgpu_fence_get_driver_name(struct dma_fence *fence)static const char *amdgpu_fence_get_timeline_name (struct dma_fence *f) { -struct amdgpu_ring *ring; +return (const char *)to_amdgpu_fence (f)->ring->name; +} -if (test_bit (AMDGPU_FENCE_FLAG_EMBED_IN_JOB_BIT , &f->flags)) { - struct amdgpu_job *job = container_of(f, struct amdgpu_job, hw_fence); +static const char *amdgpu_job_fence_get_timeline_name (struct dma_fence *f) +{ + struct amdgpu_job *job = container_of(f, struct amdgpu_job, hw_fence); - ring =to_amdgpu_ring (job->base.sched); - }else { - ring =to_amdgpu_fence (f)->ring; - } -return (const char *)ring->name; +return (const char *)to_amdgpu_ring (job->base.sched)->name; }/** @@ -665,18 +674,25 @@ static const char *amdgpu_fence_get_timeline_name(struct dma_fence *f)*/ static bool amdgpu_fence_enable_signaling (struct dma_fence *f) { -struct amdgpu_ring *ring; +if (!timer_pending (&to_amdgpu_fence (f)->ring->fence_drv.fallback_timer)) +amdgpu_fence_schedule_fallback (to_amdgpu_fence (f)->ring); -if (test_bit (AMDGPU_FENCE_FLAG_EMBED_IN_JOB_BIT , &f->flags)) { - struct amdgpu_job *job = container_of(f, struct amdgpu_job, hw_fence); +return true; +} - ring =to_amdgpu_ring (job->base.sched); - }else { - ring =to_amdgpu_fence (f)->ring; - } +/** +* amdgpu_job_fence_enable_signaling - enable signalling on job fence +* @f: fence +* +* This is the simliar function with amdgpu_fence_enable_signaling above, it +* only handles the job embedded fence. +*/ +static bool amdgpu_job_fence_enable_signaling (struct dma_fence *f) +{ + struct amdgpu_job *job = container_of(f, struct amdgpu_job, hw_fence); -if (!timer_pending (&ring->fence_drv.fallback_timer)) -amdgpu_fence_schedule_fallback (ring); +if (!timer_pending (&to_amdgpu_ring (job->base.sched)->fence_drv.fallback_timer)) +amdgpu_fence_schedule_fallback (to_amdgpu_ring (job->base.sched));return true; }@@ -692,19 +708,23 @@ static void amdgpu_fence_free(struct rcu_head *rcu) { struct dma_fence *f = container_of(rcu, struct dma_fence, rcu); -if (test_bit (AMDGPU_FENCE_FLAG_EMBED_IN_JOB_BIT , &f->flags)) { -/* free job if fence has a parent job */ -struct amdgpu_job *job; - - job = container_of(f, struct amdgpu_job, hw_fence); -kfree (job); - }else {/* free fence_slab if it's separated fence*/ -struct amdgpu_fence *fence; +kmem_cache_free (amdgpu_fence_slab,to_amdgpu_fence (f)); +} - fence =to_amdgpu_fence (f); -kmem_cache_free (amdgpu_fence_slab, fence); - } +/** +* amdgpu_job_fence_free - free up the job with embedded fence +* +* @rcu: RCU callback head +* +* Free up the job with embedded fence after the RCU grace period. +*/ +static void amdgpu_job_fence_free (struct rcu_head *rcu) +{ + struct dma_fence *f = container_of(rcu, struct dma_fence, rcu); + +/* free job if fence has a parent job */ +kfree (container_of(f ,struct amdgpu_job ,hw_fence )); }/** @@ -720,6 +740,19 @@ static void amdgpu_fence_release(struct dma_fence *f)call_rcu (&f->rcu, amdgpu_fence_free); } +/** +* amdgpu_job_fence_release - callback that job embedded fence can be freed +* +* @f: fence +* +* This is the simliar function with amdgpu_fence_release above, it +* only handles the job embedded fence. +*/ +static void amdgpu_job_fence_release (struct dma_fence *f) +{ +call_rcu (&f->rcu, amdgpu_job_fence_free); +} +static const struct dma_fence_ops amdgpu_fence_ops = { .get_driver_name = amdgpu_fence_get_driver_name, .get_timeline_name = amdgpu_fence_get_timeline_name,@@ -727,6 +760,12 @@ static const struct dma_fence_ops amdgpu_fence_ops = { .release = amdgpu_fence_release, }; +static const struct dma_fence_ops amdgpu_job_fence_ops = { + .get_driver_name = amdgpu_fence_get_driver_name, + .get_timeline_name = amdgpu_job_fence_get_timeline_name, + .enable_signaling = amdgpu_job_fence_enable_signaling, + .release = amdgpu_job_fence_release, +};/* * Fence debugfs drivers/gpu/drm/amd/amdgpu/amdgpu_ring.h +0
-3@@ -53,9 +53,6 @@ enum amdgpu_ring_priority_level {#define AMDGPU_FENCE_FLAG_INT (1 << 1)#define AMDGPU_FENCE_FLAG_TC_WB_ONLY (1 << 2) -/* fence flag bit to indicate the face is embedded in job*/ -#define AMDGPU_FENCE_FLAG_EMBED_IN_JOB_BIT (DMA_FENCE_FLAG_USER_BITS + 1) -#define to_amdgpu_ring (s) container_of((s), struct amdgpu_ring, sched)#define AMDGPU_IB_POOL_SIZE (1024 * 1024)
Quoted 15 lines · Andrey Grodzovsky
Quoted 5 lines
-
- if (job != NULL) {
- /* mark this fence has a parent job */
-set_bit(AMDGPU_FENCE_FLAG_EMBED_IN_JOB_BIT, &fence->flags); + if (job)Quoted 8 lines
+ dma_fence_init(fence,
&amdgpu_job_fence_ops, +&ring->fence_drv.lock,
+adev->fence_context+ring->idx, seq); + else
+ dma_fence_init(fence,&amdgpu_fence_ops, +&ring->fence_drv.lock,
+adev->fence_context+ring->idx, seq); }
Quoted 4 lines
It's probably me missing something but why can't we just move setting of
AMDGPU_FENCE_FLAG_EMBED_IN_JOB_BITto beforedma_fence_inithere or even intoamdgpu_job_allocinstead of all the refactoring ?Andrey
My bad, I see now that dma_fence_init just overrides flags to 0 and immediately call the trace . But why then can't we add dma_fence_init_with_flags wrapper to dma-fence ?
Andrey
[AMD Official Use Only]
Yes, exactly.
AMDGPU_FENCE_FLAG_EMBED_IN_JOB_BIT is the private flag only for amdgpu. Maybe, it's cleaner to define another amdgpu_fence_ops which is dedicated for job-embedded fence.
Thanks,
Ray
From: Grodzovsky, Andrey <[email protected]>
Sent: Wednesday, December 15, 2021 1:09 AM
To: Huang, Ray <[email protected]>; [email protected]; Koenig, Christian <[email protected]>; Daniel Vetter <[email protected]>; Sumit Semwal <[email protected]>
Cc: Deucher, Alexander <[email protected]>; [email protected]; Liu, Monk <[email protected]>; [email protected]
Subject: Re: [PATCH v2] drm/amdgpu: introduce new amdgpu_fence object to indicate the job embedded fence
On 2021-12-14 12:03 p.m., Andrey Grodzovsky wrote:
-
- if (job != NULL) {
- /* mark this fence has a parent job */
- set_bit(AMDGPU_FENCE_FLAG_EMBED_IN_JOB_BIT, &fence->flags); + if (job)
+ dma_fence_init(fence, &amdgpu_job_fence_ops, + &ring->fence_drv.lock,
+ adev->fence_context + ring->idx, seq); + else
+ dma_fence_init(fence, &amdgpu_fence_ops,
+ &ring->fence_drv.lock,
+ adev->fence_context + ring->idx, seq); }
It's probably me missing something but why can't we just move setting of AMDGPU_FENCE_FLAG_EMBED_IN_JOB_BIT to before dma_fence_init here or even into amdgpu_job_alloc instead of all the refactoring ?
Andrey
My bad, I see now that dma_fence_init just overrides flags to 0 and immediately call the trace . But why then can't we add dma_fence_init_with_flags wrapper to dma-fence ? Andrey