Skip to contentSkip to conversation

AMD/Xilinx Zynq FPGA manager stopped to work after 6.16.0-g283564a43383

DISCUSSION Pavel Pisa 4 participants 7 messages Aug 4, 2025 – Aug 6, 2025

Subsystems (2)
lore
Share & tools
lore

Pavel Pisa

author
#0 lore raw

Hello Marek and others,

we are running daily tests of SocketCAN stack on mainline and RT kernel with our CTU CAN FD IP core

https://canbus.pages.fel.cvut.cz/#can-bus-channels-mutual-latency-testing

It seems that the setup is broken after the mainlie kernel version from 2025-07-29, 6.16.0-g283564a43383. There is only one commit identified in drivers/fpga after the last working version

zynq_fpga: use sgtable-based scatterlist wrappers

The last working mainlne kernel version is recorded in the last graph data

https://canbus.pages.fel.cvut.cz/can-latester/inspect.html

We use the dtbocfg out of tree module to initiate update by devicetree overlay but fpga manager is mainline one and all worked for years correctly. The log messages

[  104.934323] dtbocfg: loading out-of-tree module taints kernel.
[  104.940681] dtbocfg: 0.1.0
[  104.943543] dtbocfg: OK
[  105.022979] fpga_manager fpga0: writing system.bit.bin to Xilinx Zynq FPGA Manager
[  105.097721] fpga_manager fpga0: Unable to DMA map (TO_DEVICE)
[  105.103562] fpga_manager fpga0: Error while writing image data to FPGA
[  105.110485] fpga_region region0: failed to load FPGA image
[  105.116059] OF: overlay: overlay changeset pre-apply notifier error -12, target: /fpga-full
[  105.124499] dtbocfg_overlay_item_create: Failed to apply overlay (ret_val=-12)

The overlay source is available there

https://gitlab.fel.cvut.cz/canbus/zynq/zynq-can-sja1000-top/-/blob/can-bench-2x-xcan-4x-ctu/scripts/dts/bitstream+dt.dts

The base zynq-7000.dtsi could be a little dated, but I have not found related change except name/alias change in mainline

- fpga_full: fpga-full {
+ fpga_full: fpga-region {

but there are recorded months of correct operation after this change. I am aware that there has been change of name alias

- amba: amba {
+ amba: axi {

in the sources but again, it has not been problem.

It seems that mapping fails in

        priv->dma_nelms =
            dma_map_sgtable(mgr->dev.parent, sgt, DMA_TO_DEVICE, 0);

The last tested failing mainline kernel version is from this midnight

6.16.0-ge991acf1bce7

Do you have some idea what could be a problem?

Do you have suggestion what to test?

Best wishes

Pavel

Pavel Pisa
phone: +420 603531357
e-mail: [email protected]
Department of Control Engineering FEE CVUT Karlovo namesti 13, 121 35, Prague 2
university: http://control.fel.cvut.cz/
personal: http://cmp.felk.cvut.cz/~pisa
social: https://social.kernel.org/ppisa
projects: https://www.openhub.net/accounts/ppisa
CAN related:http://canbus.pages.fel.cvut.cz/ RISC-V education: https://comparch.edu.cvut.cz/
Open Technologies Research Education and Exchange Services
https://gitlab.fel.cvut.cz/otrees/org/-/wikis/home

#1 lore raw
Quoted 56 lines · Pavel Pisa

Hello Marek and others,

we are running daily tests of SocketCAN stack on mainline and RT kernel with our CTU CAN FD IP core

https://canbus.pages.fel.cvut.cz/#can-bus-channels-mutual-latency-testing

It seems that the setup is broken after the mainlie kernel version from 2025-07-29, 6.16.0-g283564a43383. There is only one commit identified in drivers/fpga after the last working version

zynq_fpga: use sgtable-based scatterlist wrappers

The last working mainlne kernel version is recorded in the last graph data

https://canbus.pages.fel.cvut.cz/can-latester/inspect.html

We use the dtbocfg out of tree module to initiate update by devicetree overlay but fpga manager is mainline one and all worked for years correctly. The log messages

[  104.934323] dtbocfg: loading out-of-tree module taints kernel.
[  104.940681] dtbocfg: 0.1.0
[  104.943543] dtbocfg: OK
[  105.022979] fpga_manager fpga0: writing system.bit.bin to Xilinx Zynq FPGA Manager
[  105.097721] fpga_manager fpga0: Unable to DMA map (TO_DEVICE)
[  105.103562] fpga_manager fpga0: Error while writing image data to FPGA
[  105.110485] fpga_region region0: failed to load FPGA image
[  105.116059] OF: overlay: overlay changeset pre-apply notifier error -12, target: /fpga-full
[  105.124499] dtbocfg_overlay_item_create: Failed to apply overlay (ret_val=-12)

The overlay source is available there

   https://gitlab.fel.cvut.cz/canbus/zynq/zynq-can-sja1000-top/-/blob/can-bench-2x-xcan-4x-ctu/scripts/dts/bitstream+dt.dts

The base zynq-7000.dtsi could be a little dated, but I have not found related change except name/alias change in mainline

(quoted diff) +1 -1
-       fpga_full: fpga-full {
+       fpga_full: fpga-region {

but there are recorded months of correct operation after this change. I am aware that there has been change of name alias

(quoted diff) +1 -1
-       amba: amba {
+       amba: axi {

in the sources but again, it has not been problem.

It seems that mapping fails in

         priv->dma_nelms =
             dma_map_sgtable(mgr->dev.parent, sgt, DMA_TO_DEVICE, 0);
Quoted 6 lines

The last tested failing mainline kernel version is from this midnight

    6.16.0-ge991acf1bce7

Do you have some idea what could be a problem?

Well, my fault. I forgot that dma_map_sgtable() returns only the error code or zero on success, not the number of mapped segments. It looks that the easiest way to fix this issue is to revert my commit 37e00703228a ("zynq_fpga: use sgtable-based scatterlist wrappers"). I'm sorry for this issue.

Quoted 6 lines

Do you have suggestion what to test?

Best wishes

Pavel

Quoted 14 lines

Pavel Pisa
phone: +420 603531357
e-mail: [email protected]
Department of Control Engineering FEE CVUT Karlovo namesti 13, 121 35, Prague 2
university: http://control.fel.cvut.cz/
personal: http://cmp.felk.cvut.cz/~pisa
social: https://social.kernel.org/ppisa
projects: https://protect2.fireeye.com/v1/url?k=087d50e3-6906fa6a-087cdbac-74fe48600034-5576b8f5b2b1fa2a&q=1&e=978abd42-4782-42e4-ad5b-318f390cffea&u=https%3A%2F%2Fwww.openhub.net%2Faccounts%2Fppisa
CAN related:http://canbus.pages.fel.cvut.cz/ RISC-V education: https://comparch.edu.cvut.cz/
Open Technologies Research Education and Exchange Services
https://gitlab.fel.cvut.cz/otrees/org/-/wikis/home

Best regards

Signature · 2 lines
Marek Szyprowski, PhD
Samsung R&D Institute Poland
#2 lore raw
Quoted 4 lines

Well, my fault. I forgot that dma_map_sgtable() returns only the error code or zero on success, not the number of mapped segments. It looks that the easiest way to fix this issue is to revert my commit 37e00703228a ("zynq_fpga: use sgtable-based scatterlist wrappers"). I'm

Instead of reverting, can we fix like this?

---

drivers/fpga/zynq-fpga.c +4 -4
index 0be0d569589d..b7629a0e4813 100644
@@ -405,12 +405,12 @@ static int zynq_fpga_ops_write(struct fpga_manager *mgr, struct sg_table *sgt)
                }
        }
 
-       priv->dma_nelms =
-           dma_map_sgtable(mgr->dev.parent, sgt, DMA_TO_DEVICE, 0);
-       if (priv->dma_nelms == 0) {
+       err = dma_map_sgtable(mgr->dev.parent, sgt, DMA_TO_DEVICE, 0);
+       if (err) {
                dev_err(&mgr->dev, "Unable to DMA map (TO_DEVICE)\n");
-               return -ENOMEM;
+               return err;
        }
+       priv->dma_nelms = sgt->nents;
 
        /* enable clock */
        err = clk_enable(priv->clk);

Thanks,
Yilun

#3 lore raw
Quoted 4 lines · Xu Yilun
drivers/fpga/zynq-fpga.c +0 -0
@@ -405,12 +405,12 @@ static int zynq_fpga_ops_write(struct fpga_manager *mgr, struct sg_table *sgt)
                }
        }
 

drivers/fpga/zynq-fpga.c static int zynq_fpga_ops_write(struct fpga_manager *mgr, struct sg_table *sgt) · line 413

Quoted 10 lines
drivers/fpga/zynq-fpga.c +4 -4
-       priv->dma_nelms =
-           dma_map_sgtable(mgr->dev.parent, sgt, DMA_TO_DEVICE, 0);
-       if (priv->dma_nelms == 0) {
+       err = dma_map_sgtable(mgr->dev.parent, sgt, DMA_TO_DEVICE, 0);
+       if (err) {
                dev_err(&mgr->dev, "Unable to DMA map (TO_DEVICE)\n");
-               return -ENOMEM;
+               return err;
        }
+       priv->dma_nelms = sgt->nents;

That looks pretty good, Marek is certainly right the original had a bug.

Jason

author Tested-by
#4 lore raw

Hello all

Quoted 19 lines · On Tuesday 05 of August 2025 20:52:35 Jason Gunthorpe wrote:
Quoted 4 lines · Xu Yilun
drivers/fpga/zynq-fpga.c +0 -0
@@ -405,12 +405,12 @@ static int zynq_fpga_ops_write(struct fpga_manager
 *mgr, struct sg_table *sgt) }
        }
 

drivers/fpga/zynq-fpga.c static int zynq_fpga_ops_write(struct fpga_manager · line 413

Quoted 10 lines
drivers/fpga/zynq-fpga.c +4 -4
-       priv->dma_nelms =
-           dma_map_sgtable(mgr->dev.parent, sgt, DMA_TO_DEVICE, 0);
-       if (priv->dma_nelms == 0) {
+       err = dma_map_sgtable(mgr->dev.parent, sgt, DMA_TO_DEVICE, 0);
+       if (err) {
                dev_err(&mgr->dev, "Unable to DMA map (TO_DEVICE)\n");
-               return -ENOMEM;
+               return err;
        }
+       priv->dma_nelms = sgt->nents;

That looks pretty good, Marek is certainly right the original had a bug.

Quoted 1 line

Jason

Thanks for the fast correction proposal. I have tested the change at our kernel build, actual mainline version form midnight rebuild with patch now

Linux mzapo 6.16.0+ #2 SMP Tue Aug 5 20:59:36 CEST 2025 armv7l GNU/Linux

and it works correctly, DTBOCFG and FPGA manager is able to load CTU CAN FD IP core design and then driver is loaded and its instances are setup according to the device tree overlay and driver detects CAN controllers implemented in FPGA. I have not run whole CAN test sequence but I do not expect problems there. You can add my

Do you have some idea how fast can the change propagate into mainline? I would setup automatic patching with the fix if the testing setup is broken for some longer time. If the fix can get in mainline in days or week then I would spent time on that.

Best wishes,

Pavel

Pavel Pisa
phone: +420 603531357
e-mail: [email protected]
Department of Control Engineering FEE CVUT Karlovo namesti 13, 121 35, Prague 2
university: http://control.fel.cvut.cz/
personal: http://cmp.felk.cvut.cz/~pisa
social: https://social.kernel.org/ppisa
projects: https://www.openhub.net/accounts/ppisa
CAN related:http://canbus.pages.fel.cvut.cz/ RISC-V education: https://comparch.edu.cvut.cz/
Open Technologies Research Education and Exchange Services
https://gitlab.fel.cvut.cz/otrees/org/-/wikis/home

#5 lore raw
Quoted 36 lines · Pavel Pisa

Hello all

Quoted 19 lines · On Tuesday 05 of August 2025 20:52:35 Jason Gunthorpe wrote:
Quoted 4 lines · Xu Yilun
drivers/fpga/zynq-fpga.c +0 -0
@@ -405,12 +405,12 @@ static int zynq_fpga_ops_write(struct fpga_manager
 *mgr, struct sg_table *sgt) }
        }
 

drivers/fpga/zynq-fpga.c static int zynq_fpga_ops_write(struct fpga_manager · line 413

Quoted 10 lines
drivers/fpga/zynq-fpga.c +4 -4
-       priv->dma_nelms =
-           dma_map_sgtable(mgr->dev.parent, sgt, DMA_TO_DEVICE, 0);
-       if (priv->dma_nelms == 0) {
+       err = dma_map_sgtable(mgr->dev.parent, sgt, DMA_TO_DEVICE, 0);
+       if (err) {
                dev_err(&mgr->dev, "Unable to DMA map (TO_DEVICE)\n");
-               return -ENOMEM;
+               return err;
        }
+       priv->dma_nelms = sgt->nents;

That looks pretty good, Marek is certainly right the original had a bug.

Quoted 1 line

Jason

Thanks for the fast correction proposal. I have tested the change at our kernel build, actual mainline version form midnight rebuild with patch now

Linux mzapo 6.16.0+ #2 SMP Tue Aug 5 20:59:36 CEST 2025 armv7l GNU/Linux

and it works correctly, DTBOCFG and FPGA manager is able to load CTU CAN FD IP core design and then driver is loaded and its instances are setup according to the device tree overlay and driver detects CAN controllers implemented in FPGA. I have not run whole CAN test sequence but I do not expect problems there. You can add my

Quoted 5 lines

Do you have some idea how fast can the change propagate into mainline? I would setup automatic patching with

I've sent the patch to Greg, lets see how to fix it.

https://lore.kernel.org/linux-fpga/[email protected]/

Thanks,
Yilun

Quoted 21 lines

the fix if the testing setup is broken for some longer time. If the fix can get in mainline in days or week then I would spent time on that.

Best wishes,

Pavel

Pavel Pisa
phone: +420 603531357
e-mail: [email protected]
Department of Control Engineering FEE CVUT Karlovo namesti 13, 121 35, Prague 2
university: http://control.fel.cvut.cz/
personal: http://cmp.felk.cvut.cz/~pisa
social: https://social.kernel.org/ppisa
projects: https://www.openhub.net/accounts/ppisa
CAN related:http://canbus.pages.fel.cvut.cz/ RISC-V education: https://comparch.edu.cvut.cz/
Open Technologies Research Education and Exchange Services
https://gitlab.fel.cvut.cz/otrees/org/-/wikis/home

Reviewed-by
#6 lore raw
In reply to Xu Yilun
Quoted 20 lines · Xu Yilun
Quoted 4 lines

Well, my fault. I forgot that dma_map_sgtable() returns only the error code or zero on success, not the number of mapped segments. It looks that the easiest way to fix this issue is to revert my commit 37e00703228a ("zynq_fpga: use sgtable-based scatterlist wrappers"). I'm

Instead of reverting, can we fix like this?

---

drivers/fpga/zynq-fpga.c +0 -3
index 0be0d569589d..b7629a0e4813 100644
@@ -405,12 +405,12 @@ static int zynq_fpga_ops_write(struct fpga_manager *mgr, struct sg_table *sgt)
                 }
         }
 
-       priv->dma_nelms =
-           dma_map_sgtable(mgr->dev.parent, sgt, DMA_TO_DEVICE, 0);
-       if (priv->dma_nelms == 0) {

drivers/fpga/zynq-fpga.c static int zynq_fpga_ops_write(struct fpga_manager *mgr, struct sg_table *sgt) · line 416

Quoted 10 lines
drivers/fpga/zynq-fpga.c +4 -1
+       err = dma_map_sgtable(mgr->dev.parent, sgt, DMA_TO_DEVICE, 0);
+       if (err) {
                 dev_err(&mgr->dev, "Unable to DMA map (TO_DEVICE)\n");
-               return -ENOMEM;
+               return err;
         }
+       priv->dma_nelms = sgt->nents;
 
         /* enable clock */
         err = clk_enable(priv->clk);

Yes, this is a proper fix for the current code.

Best regards

Signature · 2 lines
Marek Szyprowski, PhD
Samsung R&D Institute Poland