Implement compression support on BMG
Subsystems (2)
cover Implement compression support on BMG
According to the SAS for BMG compression, we need to decompress during eviction, and not recompress on restore. Due to this, we need to introduce encoding pat_index in case of vram too. This patch explores the solution of setting up an additional identity map for the vram, this time at the end of previous mapping offset and with compressed pat_index.
We then select the appropriate mapping during eviction/restore/clear.
Akshata Jahagirdar (6):
drm/xe/xe2: Introduce identity map for compressed pat for vram drm/xe/migrate: Handle clear ccs logic for xe2 dgfx drm/xe/migrate: Add kunit to test clear functionality drm/xe/xe_migrate: Handle migration logic for xe2+ dgfx drm/xe/migrate: Add kunit to test migration functionality for BMG drm/xe/xe2: Do not run xe_bo_test for xe2+ dgfx
drivers/gpu/drm/xe/tests/xe_bo.c | 6 + drivers/gpu/drm/xe/tests/xe_migrate.c | 388 +++++++++++++++++++++ drivers/gpu/drm/xe/tests/xe_migrate_test.c | 1 + drivers/gpu/drm/xe/tests/xe_migrate_test.h | 1 + drivers/gpu/drm/xe/xe_device.h | 5 + drivers/gpu/drm/xe/xe_migrate.c | 66 +++- 6 files changed, 449 insertions(+), 18 deletions(-)
Signature · 1 line
2.34.1
Quoted 6 lines · Akshata Jahagirdar
According to the SAS for BMG compression, we need to decompress during eviction, and not recompress on restore. Due to this, we need to introduce encoding
pat_indexin case of vram too. This patch explores the solution of setting up an additional identity map for the vram, this time at the end of previous mapping offset and with compressedpat_index.
We then select the appropriate mapping during eviction/restore/clear.
Hey, I don't see this series on the ml. Are you subscribed?
Quoted 17 lines
Akshata Jahagirdar (6):
drm/xe/xe2: Introduce identity map for compressed pat for vram drm/xe/migrate: Handle clear ccs logic for xe2 dgfx drm/xe/migrate: Add kunit to test clear functionality drm/xe/xe_migrate: Handle migration logic for xe2+ dgfx drm/xe/migrate: Add kunit to test migration functionality for BMG drm/xe/xe2: Do not runxe_bo_testfor xe2+ dgfxdrivers/gpu/drm/xe/tests/xe_bo.c | 6 + drivers/gpu/drm/xe/tests/xe_migrate.c | 388 +++++++++++++++++++++ drivers/gpu/drm/xe/tests/xe_migrate_test.c | 1 + drivers/gpu/drm/xe/tests/xe_migrate_test.h | 1 + drivers/gpu/drm/xe/xe_device.h | 5 + drivers/gpu/drm/xe/xe_migrate.c | 66 +++- 6 files changed, 449 insertions(+), 18 deletions(-)
Quoted 9 lines · Matthew Auld
Quoted 6 lines · Akshata Jahagirdar
According to the SAS for BMG compression, we need to decompress during eviction, and not recompress on restore. Due to this, we need to introduce encoding
pat_indexin case of vram too. This patch explores the solution of setting up an additional identity map for the vram, this time at the end of previous mapping offset and with compressedpat_index.
We then select the appropriate mapping during eviction/restore/clear.Hey, I don't see this series on the ml. Are you subscribed?
Hey Matt,
Yes, I am.. seems like there is some issue with my mailing list subscription, so its not getting reflected on the website. I just resubscribed , would send this out again.
Quoted 18 lines
Quoted 18 lines
Akshata Jahagirdar (6):
drm/xe/xe2: Introduce identity map for compressed pat for vram drm/xe/migrate: Handle clear ccs logic for xe2 dgfx drm/xe/migrate: Add kunit to test clear functionality drm/xe/xe_migrate: Handle migration logic for xe2+ dgfx drm/xe/migrate: Add kunit to test migration functionality for BMG drm/xe/xe2: Do not runxe_bo_testfor xe2+ dgfxdrivers/gpu/drm/xe/tests/xe_bo.c | 6 + drivers/gpu/drm/xe/tests/xe_migrate.c | 388 +++++++++++++++++++++
drivers/gpu/drm/xe/tests/xe_migrate_test.c| 1 +drivers/gpu/drm/xe/tests/xe_migrate_test.h| 1 + drivers/gpu/drm/xe/xe_device.h | 5 + drivers/gpu/drm/xe/xe_migrate.c | 66 +++-6 files changed, 449 insertions(+), 18 deletions(-)
PATCH 1/6 drm/xe/migrate: Handle clear ccs logic for xe2 dgfx
In reply to Akshata JahagirdarFor Xe2 dGPU, we clear the bo by modifying the VRAM using an uncompressed pat index which then indirectly updates the compression status as uncompressed i.e zeroed CCS. So xe_migrate_clear() should be updated for BMG to not emit CCS surf copy commands.
--- drivers/gpu/drm/xe/xe_device.h | 5 +++++ drivers/gpu/drm/xe/xe_migrate.c | 6 +++--- 2 files changed, 8 insertions(+), 3 deletions(-)
drivers/gpu/drm/xe/xe_device.h +5 -0
@@ -144,6 +144,11 @@ static inline bool xe_device_has_flat_ccs(struct xe_device *xe)return xe->info.has_flat_ccs; } +static inline bool xe_device_needs_ccs_emit (struct xe_device *xe) +{ +return xe_device_has_flat_ccs (xe) && !(GRAPHICS_VER (xe) >=20 &&IS_DGFX (xe)); +} +static inline bool xe_device_has_sriov (struct xe_device *xe) {return xe->info.has_sriov;
drivers/gpu/drm/xe/xe_migrate.c +3 -3
@@ -420,7 +420,7 @@ struct xe_migrate *xe_migrate_init(struct xe_tile *tile)return ERR_PTR (err);if (IS_DGFX (xe)) { -if (xe_device_has_flat_ccs (xe)) +if (xe_device_needs_ccs_emit (xe))/* min chunk size corresponds to 4K of CCS Metadata */ m->min_chunk_size =SZ_4K *SZ_64K /xe_device_ccs_bytes (xe,SZ_64K );
@@ -1034,7 +1034,7 @@ struct dma_fence *xe_migrate_clear(struct xe_migrate *m, clear_system_ccs ?0 :emit_clear_cmd_len (gt),0 , avail_pts); -if (xe_device_has_flat_ccs (xe)) +if (xe_device_needs_ccs_emit (xe)) batch_size +=EMIT_COPY_CCS_DW ;/* Clear commands */
@@ -1062,7 +1062,7 @@ struct dma_fence *xe_migrate_clear(struct xe_migrate *m,if (!clear_system_ccs)emit_clear (gt, bb, clear_L0_ofs, clear_L0,XE_PAGE_SIZE , clear_vram); -if (xe_device_has_flat_ccs (xe)) { +if (xe_device_needs_ccs_emit (xe)) {emit_copy_ccs (gt, bb, clear_L0_ofs, true, m->cleared_mem_ofs, false, clear_L0); flush_flags =MI_FLUSH_DW_CCS ;
Signature · 1 line
2.34.1
Re: [PATCH 1/6] drm/xe/migrate: Handle clear ccs logic for xe2 dgfx
Hi Akshata,
I have two patches(2,3) in https://patchwork.freedesktop.org/series/135743/ that deals with this which I sent to
to handle bo clear for igfx. Those two patches are independent of my series and I you can incorporate it.
https://patchwork.freedesktop.org/patch/602287/?series=135743&rev=1
https://patchwork.freedesktop.org/patch/602288/?series=135743&rev=1
Regards,
Nirmoy
Quoted 59 lines · Akshata Jahagirdar
For Xe2 dGPU, we clear the bo by modifying the VRAM using an uncompressed pat index which then indirectly updates the compression status as uncompressed i.e zeroed CCS. So
xe_migrate_clear()should be updated for BMG to not emit CCS surf copy commands.Signed-off-by Akshata Jahagirdar <[email protected]>--- drivers/gpu/drm/xe/xe_device.h | 5 +++++ drivers/gpu/drm/xe/xe_migrate.c | 6 +++--- 2 files changed, 8 insertions(+), 3 deletions(-)drivers/gpu/drm/xe/xe_device.h +5
-0@@ -144,6 +144,11 @@ static inline bool xe_device_has_flat_ccs(struct xe_device *xe)return xe->info.has_flat_ccs; } +static inline bool xe_device_needs_ccs_emit (struct xe_device *xe) +{ +return xe_device_has_flat_ccs (xe) && !(GRAPHICS_VER (xe) >=20 &&IS_DGFX (xe)); +} +static inline bool xe_device_has_sriov (struct xe_device *xe) {return xe->info.has_sriov;drivers/gpu/drm/xe/xe_migrate.c +3
-3@@ -420,7 +420,7 @@ struct xe_migrate *xe_migrate_init(struct xe_tile *tile)return ERR_PTR (err);if (IS_DGFX (xe)) { -if (xe_device_has_flat_ccs (xe)) +if (xe_device_needs_ccs_emit (xe))/* min chunk size corresponds to 4K of CCS Metadata */ m->min_chunk_size =SZ_4K *SZ_64K /xe_device_ccs_bytes (xe,SZ_64K );@@ -1034,7 +1034,7 @@ struct dma_fence *xe_migrate_clear(struct xe_migrate *m, clear_system_ccs ?0 :emit_clear_cmd_len (gt),0 , avail_pts); -if (xe_device_has_flat_ccs (xe)) +if (xe_device_needs_ccs_emit (xe)) batch_size +=EMIT_COPY_CCS_DW ;/* Clear commands */ @@ -1062,7 +1062,7 @@ struct dma_fence *xe_migrate_clear(struct xe_migrate *m,if (!clear_system_ccs)emit_clear (gt, bb, clear_L0_ofs, clear_L0,XE_PAGE_SIZE , clear_vram); -if (xe_device_has_flat_ccs (xe)) { +if (xe_device_needs_ccs_emit (xe)) {emit_copy_ccs (gt, bb, clear_L0_ofs, true, m->cleared_mem_ofs, false, clear_L0); flush_flags =MI_FLUSH_DW_CCS ;
RE: [PATCH 1/6] drm/xe/migrate: Handle clear ccs logic for xe2 dgfx
In reply to Akshata JahagirdarHello,
Ashata, please verify results from patchwork as our infrastructure is not able to process whole logs. Example - https://gfx-ci.igk.intel.com/cibuglog-ng/testresult/1840022997?query_key=5010e5e29d437861d2c89e0d98eda884b16111d8
Looks like whole memory dumped to dmesg?
Please check your series before sending at least on 1 machine to avoid such situations. Patchwork cannot be a way to test your code.
Thanks!
Kasia
-----Original Message-----
From: igt-dev <[email protected]> On Behalf Of Akshata Jahagirdar
Sent: Thursday, July 11, 2024 2:22 PM
To: [email protected]
Cc: [email protected]; Jahagirdar, Akshata <[email protected]>
Subject: [PATCH 1/6] drm/xe/migrate: Handle clear ccs logic for xe2 dgfx
For Xe2 dGPU, we clear the bo by modifying the VRAM using an uncompressed pat index which then indirectly updates the compression status as uncompressed i.e zeroed CCS. So xe_migrate_clear() should be updated for BMG to not emit CCS surf copy commands.
--- drivers/gpu/drm/xe/xe_device.h | 5 +++++ drivers/gpu/drm/xe/xe_migrate.c | 6 +++--- 2 files changed, 8 insertions(+), 3 deletions(-)
drivers/gpu/drm/xe/xe_device.h +4 -0
@@ -144,6 +144,11 @@ static inline bool xe_device_has_flat_ccs(struct xe_device *xe)return xe->info.has_flat_ccs; } +static inline bool xe_device_needs_ccs_emit (struct xe_device *xe) { +return xe_device_has_flat_ccs (xe) && !(GRAPHICS_VER (xe) >=20 && +IS_DGFX (xe)); } +static inline bool xe_device_has_sriov (struct xe_device *xe) {return xe->info.has_sriov;
drivers/gpu/drm/xe/xe_migrate.c +3 -3
@@ -420,7 +420,7 @@ struct xe_migrate *xe_migrate_init(struct xe_tile *tile)return ERR_PTR (err);if (IS_DGFX (xe)) { -if (xe_device_has_flat_ccs (xe)) +if (xe_device_needs_ccs_emit (xe))/* min chunk size corresponds to 4K of CCS Metadata */ m->min_chunk_size =SZ_4K *SZ_64K /xe_device_ccs_bytes (xe,SZ_64K );
@@ -1034,7 +1034,7 @@ struct dma_fence *xe_migrate_clear(struct xe_migrate *m, clear_system_ccs ?0 :emit_clear_cmd_len (gt),0 , avail_pts); -if (xe_device_has_flat_ccs (xe)) +if (xe_device_needs_ccs_emit (xe)) batch_size +=EMIT_COPY_CCS_DW ;/* Clear commands */
@@ -1062,7 +1062,7 @@ struct dma_fence *xe_migrate_clear(struct xe_migrate *m,if (!clear_system_ccs)emit_clear (gt, bb, clear_L0_ofs, clear_L0,XE_PAGE_SIZE , clear_vram); -if (xe_device_has_flat_ccs (xe)) { +if (xe_device_needs_ccs_emit (xe)) {emit_copy_ccs (gt, bb, clear_L0_ofs, true, m->cleared_mem_ofs, false, clear_L0); flush_flags =MI_FLUSH_DW_CCS ;
--
2.34.1
RE: [PATCH 1/6] drm/xe/migrate: Handle clear ccs logic for xe2 dgfx
Hello Kasia,
This patch is actually meant for the intel-xe and not the igt-dev patchwork. Due to some unknown reasons, my emails are not getting through to the intel-xe mailing list. I sent it to the igt-dev list as a test patch to confirm if the issue is solely with the intel-xe list or with igt-dev as well.
I am still trying to understand why this might be happening and apologize for any inconvenience caused.
Could you please let me know if these failures are solely due to my patch, or are they also being reported on the latest kernel or other patches? If it's only due to my patch, we can safely ignore it.
Let me know if further debugging is needed.
Thank you.
Best,
Akshata Jahagirdar
-----Original Message-----
From: Piecielska, Katarzyna <[email protected]>
Sent: Friday, July 12, 2024 5:33 AM
To: Jahagirdar, Akshata <[email protected]>; [email protected]
Cc: [email protected]; Jahagirdar, Akshata <[email protected]>
Subject: RE: [PATCH 1/6] drm/xe/migrate: Handle clear ccs logic for xe2 dgfx
Hello,
Ashata, please verify results from patchwork as our infrastructure is not able to process whole logs. Example - https://gfx-ci.igk.intel.com/cibuglog-ng/testresult/1840022997?query_key=5010e5e29d437861d2c89e0d98eda884b16111d8
Looks like whole memory dumped to dmesg?
Please check your series before sending at least on 1 machine to avoid such situations. Patchwork cannot be a way to test your code.
Thanks!
Kasia
-----Original Message-----
From: igt-dev <[email protected]> On Behalf Of Akshata Jahagirdar
Sent: Thursday, July 11, 2024 2:22 PM
To: [email protected]
Cc: [email protected]; Jahagirdar, Akshata <[email protected]>
Subject: [PATCH 1/6] drm/xe/migrate: Handle clear ccs logic for xe2 dgfx
For Xe2 dGPU, we clear the bo by modifying the VRAM using an uncompressed pat index which then indirectly updates the compression status as uncompressed i.e zeroed CCS. So xe_migrate_clear() should be updated for BMG to not emit CCS surf copy commands.
--- drivers/gpu/drm/xe/xe_device.h | 5 +++++ drivers/gpu/drm/xe/xe_migrate.c | 6 +++--- 2 files changed, 8 insertions(+), 3 deletions(-)
drivers/gpu/drm/xe/xe_device.h +4 -0
@@ -144,6 +144,11 @@ static inline bool xe_device_has_flat_ccs(struct xe_device *xe)return xe->info.has_flat_ccs; } +static inline bool xe_device_needs_ccs_emit (struct xe_device *xe) { +return xe_device_has_flat_ccs (xe) && !(GRAPHICS_VER (xe) >=20 && +IS_DGFX (xe)); } +static inline bool xe_device_has_sriov (struct xe_device *xe) {return xe->info.has_sriov;
drivers/gpu/drm/xe/xe_migrate.c +3 -3
@@ -420,7 +420,7 @@ struct xe_migrate *xe_migrate_init(struct xe_tile *tile)return ERR_PTR (err);if (IS_DGFX (xe)) { -if (xe_device_has_flat_ccs (xe)) +if (xe_device_needs_ccs_emit (xe))/* min chunk size corresponds to 4K of CCS Metadata */ m->min_chunk_size =SZ_4K *SZ_64K /xe_device_ccs_bytes (xe,SZ_64K );
@@ -1034,7 +1034,7 @@ struct dma_fence *xe_migrate_clear(struct xe_migrate *m, clear_system_ccs ?0 :emit_clear_cmd_len (gt),0 , avail_pts); -if (xe_device_has_flat_ccs (xe)) +if (xe_device_needs_ccs_emit (xe)) batch_size +=EMIT_COPY_CCS_DW ;/* Clear commands */
@@ -1062,7 +1062,7 @@ struct dma_fence *xe_migrate_clear(struct xe_migrate *m,if (!clear_system_ccs)emit_clear (gt, bb, clear_L0_ofs, clear_L0,XE_PAGE_SIZE , clear_vram); -if (xe_device_has_flat_ccs (xe)) { +if (xe_device_needs_ccs_emit (xe)) {emit_copy_ccs (gt, bb, clear_L0_ofs, true, m->cleared_mem_ofs, false, clear_L0); flush_flags =MI_FLUSH_DW_CCS ;
--
2.34.1
RE: [PATCH 1/6] drm/xe/migrate: Handle clear ccs logic for xe2 dgfx
Hello Akshata,
It is something that was noticed only in patchwork. It is not observed in regular CI execution. So please look closer in you series before sending for review.
Kasia
-----Original Message-----
From: Jahagirdar, Akshata <[email protected]>
Sent: Tuesday, July 16, 2024 4:16 AM
To: Piecielska, Katarzyna <[email protected]>; [email protected]
Cc: [email protected]
Subject: RE: [PATCH 1/6] drm/xe/migrate: Handle clear ccs logic for xe2 dgfx
Hello Kasia,
This patch is actually meant for the intel-xe and not the igt-dev patchwork. Due to some unknown reasons, my emails are not getting through to the intel-xe mailing list. I sent it to the igt-dev list as a test patch to confirm if the issue is solely with the intel-xe list or with igt-dev as well.
I am still trying to understand why this might be happening and apologize for any inconvenience caused.
Could you please let me know if these failures are solely due to my patch, or are they also being reported on the latest kernel or other patches? If it's only due to my patch, we can safely ignore it.
Let me know if further debugging is needed.
Thank you.
Best,
Akshata Jahagirdar
-----Original Message-----
From: Piecielska, Katarzyna <[email protected]>
Sent: Friday, July 12, 2024 5:33 AM
To: Jahagirdar, Akshata <[email protected]>; [email protected]
Cc: [email protected]; Jahagirdar, Akshata <[email protected]>
Subject: RE: [PATCH 1/6] drm/xe/migrate: Handle clear ccs logic for xe2 dgfx
Hello,
Ashata, please verify results from patchwork as our infrastructure is not able to process whole logs. Example - https://gfx-ci.igk.intel.com/cibuglog-ng/testresult/1840022997?query_key=5010e5e29d437861d2c89e0d98eda884b16111d8
Looks like whole memory dumped to dmesg?
Please check your series before sending at least on 1 machine to avoid such situations. Patchwork cannot be a way to test your code.
Thanks!
Kasia
-----Original Message-----
From: igt-dev <[email protected]> On Behalf Of Akshata Jahagirdar
Sent: Thursday, July 11, 2024 2:22 PM
To: [email protected]
Cc: [email protected]; Jahagirdar, Akshata <[email protected]>
Subject: [PATCH 1/6] drm/xe/migrate: Handle clear ccs logic for xe2 dgfx
For Xe2 dGPU, we clear the bo by modifying the VRAM using an uncompressed pat index which then indirectly updates the compression status as uncompressed i.e zeroed CCS. So xe_migrate_clear() should be updated for BMG to not emit CCS surf copy commands.
--- drivers/gpu/drm/xe/xe_device.h | 5 +++++ drivers/gpu/drm/xe/xe_migrate.c | 6 +++--- 2 files changed, 8 insertions(+), 3 deletions(-)
drivers/gpu/drm/xe/xe_device.h +4 -0
@@ -144,6 +144,11 @@ static inline bool xe_device_has_flat_ccs(struct xe_device *xe)return xe->info.has_flat_ccs; } +static inline bool xe_device_needs_ccs_emit (struct xe_device *xe) { +return xe_device_has_flat_ccs (xe) && !(GRAPHICS_VER (xe) >=20 && +IS_DGFX (xe)); } +static inline bool xe_device_has_sriov (struct xe_device *xe) {return xe->info.has_sriov;
drivers/gpu/drm/xe/xe_migrate.c +3 -3
@@ -420,7 +420,7 @@ struct xe_migrate *xe_migrate_init(struct xe_tile *tile)return ERR_PTR (err);if (IS_DGFX (xe)) { -if (xe_device_has_flat_ccs (xe)) +if (xe_device_needs_ccs_emit (xe))/* min chunk size corresponds to 4K of CCS Metadata */ m->min_chunk_size =SZ_4K *SZ_64K /xe_device_ccs_bytes (xe,SZ_64K );
@@ -1034,7 +1034,7 @@ struct dma_fence *xe_migrate_clear(struct xe_migrate *m, clear_system_ccs ?0 :emit_clear_cmd_len (gt),0 , avail_pts); -if (xe_device_has_flat_ccs (xe)) +if (xe_device_needs_ccs_emit (xe)) batch_size +=EMIT_COPY_CCS_DW ;/* Clear commands */
@@ -1062,7 +1062,7 @@ struct dma_fence *xe_migrate_clear(struct xe_migrate *m,if (!clear_system_ccs)emit_clear (gt, bb, clear_L0_ofs, clear_L0,XE_PAGE_SIZE , clear_vram); -if (xe_device_has_flat_ccs (xe)) { +if (xe_device_needs_ccs_emit (xe)) {emit_copy_ccs (gt, bb, clear_L0_ofs, true, m->cleared_mem_ofs, false, clear_L0); flush_flags =MI_FLUSH_DW_CCS ;
--
2.34.1
Quoted 24 lines · Akshata Jahagirdar
On Xe2 the compression has moved to a unified universal model (exactly one compression mode/format), where compression is now controlled via PAT on per-page basis. This now means KMD can decompress freely. This was problematic on DG2 since we had multiple compression formats, and the compression format used on a particular buffer was unknown to the KMD, so instead the raw CCS state needed to be copied around when evicting VRAM. In addition mixed VRAM and system memory buffers were not supported with compression enabled.
On Xe2 dGPU compression is still only supported with VRAM, however we can now support compression with VRAM and system memory buffers, with GPU access being seamless underneath. So long as when doing VRAM -> sysmem the KMD does the move using compressed -> uncompressed, to decompress it.
CPU access to such buffers is also possible, under the premise that userspace first decompress the corresponding pages being accessed. If the pages are already in system memory then KMD would have already decompressed them. When restoring such buffers with sysmem -> VRAM the KMD can't easily know which pages were originally compressed, so we always use uncompressed -> uncompressed here. With this it also means we can drop all the raw CCS handling on such platforms (including needing to allocate extra CCS storage).
Quoted 5 lines
In order to support this we now need to have two different identity mappings for compressed and uncompressed VRAM.
The additional identity map is the VRAM with compressedpat_index. We then select the appropriate mapping during migration/clear.
Have gone through all the patches. Please address indentation and checkpatch errors.
With above addressed all patches lgtm.
Quoted 16 lines
Akshata Jahagirdar (6):
drm/xe/xe2: Introduce identity map for compressed pat for vram drm/xe/migrate: Handle clear ccs logic for xe2 dgfx drm/xe/migrate: Add kunit to test clear functionality drm/xe/xe_migrate: Handle migration logic for xe2+ dgfx drm/xe/migrate: Add kunit to test migration functionality for BMG drm/xe/xe2: Do not runxe_bo_testfor xe2+ dgfxdrivers/gpu/drm/xe/tests/xe_bo.c | 6 + drivers/gpu/drm/xe/tests/xe_migrate.c | 388 +++++++++++++++++++++ drivers/gpu/drm/xe/tests/xe_migrate_test.c | 1 + drivers/gpu/drm/xe/tests/xe_migrate_test.h | 1 + drivers/gpu/drm/xe/xe_device.h | 5 + drivers/gpu/drm/xe/xe_migrate.c | 66 +++- 6 files changed, 449 insertions(+), 18 deletions(-)