net/mlx5: Use unique names for software steering caches
Subsystems (4)
Tariq Toukan
maintainerPATCH RESEND 1/1 net/mlx5: Use unique names for software steering caches
From: Leon Romanovsky <[email protected]>
Each software steering domain creates its own slab caches, but all domains use the same names. When domains for two devices are alive at once, the second kmem_cache_create() triggers the following splat:
WARNING: mm/slab_common.c:111 at __kmem_cache_create_args+0xca/0x480, CPU#18: devlink/331372 Modules linked in: act_mirred act_skbedit cls_matchall act_gact cls_flower sch_ingress
vhost_vdpa veth nfnetlink_cttimeout openvswitch macvtap macvlan vfio_ap kvm nf_nat_tftp nf_conntrack_tftp nsh nf_conncount vfio_pci_core irqbypass scsi_debug vhost_net tap tunvhost_vsock vmw_vsock_virtio_transport_common vsock vhost nft_masq nft_reject_ipv4 act_csumcls_u32 sch_htb smc_diag smc ppp_deflate bsd_comp ppp_async crc_ccitt ppp_generic slhc loop algif_hash af_alg nft_fib_inet nft_fib_ipv4 nft_fib_ipv6 nft_fib nft_reject_inet nf_reject_ipv4 nf_reject_ipv6 nft_reject nft_ct nft_chain_nat nf_nat nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 nf_tables mlx5_vdpa vdpa mlx5_ib dm_service_time ib_uverbs_support vringh ib_core vhost_iotlb mlx5_core s390_trng eadm_sch vfio_ccw mdev vfio_iommu_type1 vfio sch_fq_codel drm i2c_core dm_multipath drm_panel_orientation_quirks uvdevice diag288_wdt watchdog hmac_s390 prng aes_s390 zfcp scsi_transport_fc pkey_pckmo pkey_cca pkey_ep11 zcrypt paes_s390 phmac_s390 rng_core scsi_dh_alua pkey scsi_dh_rdac scsi_dh_emc crypto_engine autofs4 ecdsa_generic ecc sha512 [last unloaded: openvswitch]
CPU: 18 UID: 0 PID: 331372 Comm: devlink Tainted: G W 7.2.0-20260712.rc2.git0.e3321fa3034d.300.fc44.s390x+debug #1 PREEMPT
Tainted: [W]=WARN
Hardware name: IBM 9175 ME1 701 (LPAR)
Krnl PSW : 0704c00180000000 0000038139a6771a (__kmem_cache_create_args+0xda/0x480)
R:0 T:1 IO:1 EX:1 Key:0 M:1 W:0 P:0 AS:3 CC:0 PM:0 RI:0 EA:3
Krnl GPRS: 0000000000000000 0000000000000000 000003813b7de578 00000380b9eb8974
00000276c417f690 00000380b9eb8974 0000030144623348 00000277234a1660
0000000000000020 00000380b9eb8974 00000277234a1600 000003813b686d30
0000000000000000 00000380b9e9a810 0000038139a6771a 0000030144623238
Krnl Code: 0000038139a6770a: c02000ebb737 larl %r2,000003813b7de578
0000038139a67710: b9040039 lgr %r3,%r9
*0000038139a67714: c0e5006e2eb2 brasl %r14,000003813a82d478
>0000038139a6771a: a7390020 lghi %r3,32
0000038139a6771e: b9040029 lgr %r2,%r9
0000038139a67722: c0e5006cf98f brasl %r14,000003813a806a40
0000038139a67728: ec26018e007c cgij %r2,0,6,0000038139a67a44
0000038139a6772e: 58d0f0a4 l %r13,164(%r15)
Call Trace:
[<0000038139a6771a>] __kmem_cache_create_args+0xda/0x480
([<0000038139a676a2>] __kmem_cache_create_args+0x62/0x480)
[<00000380b9da1c70>] dr_domain_init_mem_resources+0x80/0x240 [mlx5_core]
[<00000380b9da226e>] dr_domain_init_resources.constprop.0+0x7e/0x2c0 [mlx5_core]
[<00000380b9da28b2>] mlx5dr_domain_create+0x132/0x250 [mlx5_core]
[<00000380b9dc1e20>] mlx5_cmd_dr_create_ns+0x30/0x90 [mlx5_core]
[<00000380b9cd8dbe>] mlx5_flow_namespace_set_mode+0x6e/0x130 [mlx5_core]
[<00000380b9d846ec>] esw_create_offloads_fdb_tables+0xac/0x5a0 [mlx5_core]
[<00000380b9d865b6>] esw_offloads_steering_init+0x1c6/0x480 [mlx5_core]
[<00000380b9d86e8e>] esw_offloads_enable+0x13e/0x410 [mlx5_core]
[<00000380b9d7b04a>] mlx5_eswitch_enable_locked+0x36a/0x540 [mlx5_core]
[<00000380b9d84ff0>] esw_offloads_start+0x50/0x1d0 [mlx5_core]
[<00000380b9d8774a>] mlx5_devlink_eswitch_mode_set+0x35a/0x3f0 [mlx5_core]
[<000003813a791e68>] devlink_nl_eswitch_set_doit+0x88/0x120
[<000003813a5b93ea>] genl_family_rcv_msg_doit+0xea/0x150
[<000003813a5b95c2>] genl_family_rcv_msg+0x172/0x210
[<000003813a5b96c2>] genl_rcv_msg+0x62/0xc0
[<000003813a5b7cac>] netlink_rcv_skb+0x5c/0x120
[<000003813a5b8f0c>] genl_rcv+0x3c/0x50
[<000003813a5b74a4>] netlink_unicast+0x1f4/0x2b0
[<000003813a5b783c>] netlink_sendmsg+0x2dc/0x460
[<000003813a4c9764>] __sock_sendmsg+0x64/0xd0
[<000003813a4cc878>] __sys_sendto+0x108/0x160
[<000003813a4cdf50>] __do_sys_socketcall+0x350/0x460
[<000003813a8184d2>] __do_syscall+0x172/0x750
[<000003813a82d5d2>] system_call+0x72/0x90Prefix each cache name with the device name to make it unique.
--- .../ethernet/mellanox/mlx5/core/steering/sws/dr_domain.c | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-)
drivers/net/ethernet/mellanox/mlx5/core/steering/sws/dr_domain.c +7 -2
@@ -98,9 +98,12 @@ int mlx5dr_domain_get_recalc_cs_ft_addr(struct mlx5dr_domain *dmn,static int dr_domain_init_mem_resources (struct mlx5dr_domain *dmn) { +char name[80 ];int ret; - dmn->chunks_kmem_cache =kmem_cache_create ("mlx5_dr_chunks" , +snprintf (name,sizeof (name),"%s-mlx5_dr_chunks" , +dev_name (dmn->mdev->device)); + dmn->chunks_kmem_cache =kmem_cache_create (name,sizeof (struct mlx5dr_icm_chunk ),0 ,SLAB_HWCACHE_ALIGN ,NULL );if (!dmn->chunks_kmem_cache) {
@@ -108,7 +111,9 @@ static int dr_domain_init_mem_resources(struct mlx5dr_domain *dmn)return -ENOMEM ; } - dmn->htbls_kmem_cache =kmem_cache_create ("mlx5_dr_htbls" , +snprintf (name,sizeof (name),"%s-mlx5_dr_htbls" , +dev_name (dmn->mdev->device)); + dmn->htbls_kmem_cache =kmem_cache_create (name,sizeof (struct mlx5dr_ste_htbl ),0 ,SLAB_HWCACHE_ALIGN ,NULL );if (!dmn->htbls_kmem_cache) {
base-commit: 3bd438a58e910db5dc369aa25dfed1fc95f1b596
Signature · 1 line
2.44.0
Jakub Kicinski
maintainerQuoted 1 line · On Fri, 31 Jul 2026 00:02:00 +0300 Tariq Toukan wrote:
Prefix each cache name with the device name to make it unique.
Why do we need a per-device kmem_cache? Isn't the right fix to make this driver-level state?
Quoted 5 lines · Jakub Kicinski
Quoted 1 line · On Fri, 31 Jul 2026 00:02:00 +0300 Tariq Toukan wrote:
Prefix each cache name with the device name to make it unique.
Why do we need a per-device
kmem_cache? Isn't the right fix to make this driver-level state?
This kmem_cache is required only for software managed steering mode, which is configured per device by mlx5_fs_mode_set(). In general, resources should be allocated only when they are actually needed.
I personally run everything in FW-managed steering mode, so this kmem_cache is useless to me.
Thanks
Jakub Kicinski
maintainerQuoted 10 lines · On Tue, 4 Aug 2026 11:04:55 +0300 Leon Romanovsky wrote:
Quoted 5 lines · Jakub Kicinski
Quoted 1 line · On Fri, 31 Jul 2026 00:02:00 +0300 Tariq Toukan wrote:
Prefix each cache name with the device name to make it unique.
Why do we need a per-device
kmem_cache? Isn't the right fix to make this driver-level state?This
kmem_cacheis required only for software managed steering mode, which is configured per device bymlx5_fs_mode_set(). In general, resources should be allocated only when they are actually needed.
Of course you have to wrap the init in a lock and count users. Trivial stuff.
Quoted 8 lines · Jakub Kicinski
Quoted 7 lines · On Tue, 4 Aug 2026 11:04:55 +0300 Leon Romanovsky wrote:
Quoted 5 lines · Jakub Kicinski
Quoted 1 line · On Fri, 31 Jul 2026 00:02:00 +0300 Tariq Toukan wrote:
Prefix each cache name with the device name to make it unique.
Why do we need a per-device
kmem_cache? Isn't the right fix to make this driver-level state?
Quoted 6 lines
Quoted 3 lines
This
kmem_cacheis required only for software managed steering mode, which is configured per device bymlx5_fs_mode_set(). In general, resources should be allocated only when they are actually needed.Of course you have to wrap the init in a lock and count users. Trivial stuff.
I don't see why we should abandon the simple, architecturally correct approach of allocating and managing resources where they belong in favor of a more complex global scheme involving locking and reference counting.
Thanks
Am 06.08.26 um 13:17 schrieb Leon Romanovsky:
Quoted 13 lines · Jakub Kicinski
Quoted 12 lines · On Tue, 4 Aug 2026 11:04:55 +0300 Leon Romanovsky wrote:
Quoted 10 lines · Jakub Kicinski
Quoted 5 lines · On Fri, 31 Jul 2026 00:02:00 +0300 Tariq Toukan wrote:
Quoted 1 line
Prefix each cache name with the device name to make it unique.
Why do we need a per-device
kmem_cache? Isn't the right fix to make this driver-level state?This
kmem_cacheis required only for software managed steering mode, which is configured per device bymlx5_fs_mode_set(). In general, resources should be allocated only when they are actually needed.
Quoted 8 lines
Quoted 2 lines
Of course you have to wrap the init in a lock and count users. Trivial stuff.
I don't see why we should abandon the simple, architecturally correct approach of allocating and managing resources where they belong in favor of a more complex global scheme involving locking and reference counting.
Any updates on this?
Quoted 17 lines · Christian Borntraeger
Am 06.08.26 um 13:17 schrieb Leon Romanovsky:
Quoted 16 lines · Jakub Kicinski
Quoted 14 lines · On Tue, 4 Aug 2026 11:04:55 +0300 Leon Romanovsky wrote:
Quoted 10 lines · Jakub Kicinski
Quoted 5 lines · On Fri, 31 Jul 2026 00:02:00 +0300 Tariq Toukan wrote:
Quoted 1 line
Prefix each cache name with the device name to make it unique.
Why do we need a per-device
kmem_cache? Isn't the right fix to make this driver-level state?This
kmem_cacheis required only for software managed steering mode, which is configured per device bymlx5_fs_mode_set(). In general, resources should be allocated only when they are actually needed.Of course you have to wrap the init in a lock and count users. Trivial stuff.
Quoted 6 lines
Quoted 5 lines
I don't see why we should abandon the simple, architecturally correct approach of allocating and managing resources where they belong in favor of a more complex global scheme involving locking and reference counting.
Any updates on this?
No, we are waiting for Jakub to explain why should we complicate simple and working solution.
Thanks
Christian Borntraeger
Tested-byAm 15.09.26 um 13:13 schrieb Leon Romanovsky:
Quoted 23 lines · Christian Borntraeger
Quoted 22 lines
Am 06.08.26 um 13:17 schrieb Leon Romanovsky:
Quoted 21 lines · Jakub Kicinski
Quoted 8 lines · On Tue, 4 Aug 2026 11:04:55 +0300 Leon Romanovsky wrote:
Quoted 7 lines · Jakub Kicinski
Quoted 5 lines · On Fri, 31 Jul 2026 00:02:00 +0300 Tariq Toukan wrote:
Quoted 1 line
Prefix each cache name with the device name to make it unique.
Why do we need a per-device
kmem_cache? Isn't the right fix to make this driver-level state?Quoted 6 lines
Quoted 3 lines
This
kmem_cacheis required only for software managed steering mode, which is configured per device bymlx5_fs_mode_set(). In general, resources should be allocated only when they are actually needed.Of course you have to wrap the init in a lock and count users. Trivial stuff.
I don't see why we should abandon the simple, architecturally correct approach of allocating and managing resources where they belong in favor of a more complex global scheme involving locking and reference counting.
Quoted 4 lines
Quoted 1 line
Any updates on this?
No, we are waiting for Jakub to explain why should we complicate simple and working solution.
Ping.
So in the end this is something for you to sort out with Jakub. I can just say that we see this issue in our s390 CI without the patch so the patch as is is
I can offer to test alternative patches if any.