Skip to contentSkip to conversation

modprobe rdma_rxe failed when ipv6 disabled

DISCUSSION Yi Zhang 3 participants 7 messages Feb 26, 2021 – Feb 28, 2021 1 message not in the archive

Subsystems (2)
lore
Share & tools
lore

A message that is not in the archive (sent off-list or not yet received): [email protected]

Yi Zhang

author
#1 lore raw

Hello

I found this failure after ipv6 disabled, is that expected?

# modprobe rdma_rxe
modprobe: ERROR: could not insert 'rdma_rxe': Operation not permitted
# dmesg
[  596.783484] rdma_rxe: failed to create udp socket. err = -97
[  596.789144] rdma_rxe: Failed to create IPv6 UDP tunnel
# uname -r
5.11.0

Best Regards,
Yi Zhang

maintainer
#2 lore raw
Quoted 11 lines · Yi Zhang

Hello

I found this failure after ipv6 disabled, is that expected?

# modprobe rdma_rxe
modprobe: ERROR: could not insert 'rdma_rxe': Operation not permitted
# dmesg
[  596.783484] rdma_rxe: failed to create udp socket. err = -97
[  596.789144] rdma_rxe: Failed to create IPv6 UDP tunnel

ipv6 in RXE is based on config_ipv6. If config_ipv6 is disabled. rxe will not implement ipv6 features.
How do you disable ipv6 in your hosts?

Zhu Yanjun

Quoted 10 lines
# uname -r
5.11.0

Best Regards,
Yi Zhang

author
#3 lore raw
Quoted 8 lines · Zhu Yanjun
Quoted 7 lines · Yi Zhang

Hello

I found this failure after ipv6 disabled, is that expected?

# modprobe rdma_rxe
modprobe: ERROR: could not insert 'rdma_rxe': Operation not permitted
Quoted 6 lines
Quoted 3 lines
# dmesg
[  596.783484] rdma_rxe: failed to create udp socket. err = -97
[  596.789144] rdma_rxe: Failed to create IPv6 UDP tunnel

ipv6 in RXE is based on config_ipv6. If config_ipv6 is disabled. rxe will not implement ipv6 features.
How do you disable ipv6 in your hosts?

Sorry, I should give the detailed info, I disabled it by kernel parameter:

# cat /proc/cmdline  | grep -o ipv6.disable=1
ipv6.disable=1
Quoted 11 lines

Zhu Yanjun

Quoted 9 lines
# uname -r
5.11.0

Best Regards,
Yi Zhang

maintainer
#4 lore raw

From 9dcdd09f3ca3cf222b563866acd91d18bc4b93d4 Mon Sep 17 00:00:00 2001
From: Zhu Yanjun <[email protected]>
Date: Sat, 27 Feb 2021 23:01:15 +0000
Subject: [PATCH 1/1] RDMA/rxe: Disable ipv6 features when ipv6.disable set in
cmdline

When ipv6.disable=1 is set in cmdline, ipv6 is actually disabled in the stack. As such, the operations of ipv6 in RXE will fail. So ipv6 features in RXE should also be disabled in RXE.

---
 drivers/infiniband/sw/rxe/rxe_net.c | 9 +++++++++
 1 file changed, 9 insertions(+)
(quoted diff) +0 -0

b/drivers/infiniband/sw/rxe/rxe_net.c
index 0701bd1ffd1a..6ef092cb575e 100644

drivers/infiniband/sw/rxe/rxe_net.c +0 -0
@@ -72,6 +72,11 @@ static struct dst_entry *rxe_find_route6(struct

net_device *ndev,

        struct dst_entry *ndst;
        struct flowi6 fl6 = { { 0 } };

+ if (!ipv6_mod_enabled()) {
+ pr_info("IPv6 is disabled by ipv6.disable=1 in cmdline");
+ return NULL;
+ }
+

        memset(&fl6, 0, sizeof(fl6));
        fl6.flowi6_oif = ndev->ifindex;
        memcpy(&fl6.saddr, saddr, sizeof(*saddr));

@@ -608,6 +613,10 @@ static int rxe_net_ipv4_init(void)
static int rxe_net_ipv6_init(void)
{
#if IS_ENABLED(CONFIG_IPV6)
+ if (!ipv6_mod_enabled()) {
+ pr_info("IPv6 is disabled by ipv6.disable=1 in cmdline");
+ return 0;
+ }

recv_sockets.sk6 = rxe_setup_udp_tunnel(&init_net,
htons(ROCE_V2_UDP_DPORT), true);
--
2.25.1

Quoted 36 lines · Yi Zhang
Quoted 8 lines · Zhu Yanjun
Quoted 7 lines · Yi Zhang

Hello

I found this failure after ipv6 disabled, is that expected?

# modprobe rdma_rxe
modprobe: ERROR: could not insert 'rdma_rxe': Operation not permitted
Quoted 6 lines
Quoted 3 lines
# dmesg
[  596.783484] rdma_rxe: failed to create udp socket. err = -97
[  596.789144] rdma_rxe: Failed to create IPv6 UDP tunnel

ipv6 in RXE is based on config_ipv6. If config_ipv6 is disabled. rxe will not implement ipv6 features.
How do you disable ipv6 in your hosts?

Sorry, I should give the detailed info, I disabled it by kernel parameter:

# cat /proc/cmdline  | grep -o ipv6.disable=1
ipv6.disable=1
Quoted 11 lines

Zhu Yanjun

Quoted 9 lines
# uname -r
5.11.0

Best Regards,
Yi Zhang

maintainer
#5 lore raw

Hi, Yi, Leon and Jason

I made a patch to try to fix this problem. Please comment. Thanks a lot.

Zhu Yanjun

Quoted 86 lines · Zhu Yanjun

From 9dcdd09f3ca3cf222b563866acd91d18bc4b93d4 Mon Sep 17 00:00:00 2001
From: Zhu Yanjun <[email protected]>
Date: Sat, 27 Feb 2021 23:01:15 +0000
Subject: [PATCH 1/1] RDMA/rxe: Disable ipv6 features when ipv6.disable set in
cmdline

When ipv6.disable=1 is set in cmdline, ipv6 is actually disabled in the stack. As such, the operations of ipv6 in RXE will fail. So ipv6 features in RXE should also be disabled in RXE.

---
 drivers/infiniband/sw/rxe/rxe_net.c | 9 +++++++++
 1 file changed, 9 insertions(+)
drivers/infiniband/sw/rxe/rxe_net.c +5 -0
index 0701bd1ffd1a..6ef092cb575e 100644
@@ -72,6 +72,11 @@ static struct dst_entry *rxe_find_route6(struct
 net_device *ndev,
        struct dst_entry *ndst;
        struct flowi6 fl6 = { { 0 } };
 
+       if (!ipv6_mod_enabled()) {
+               pr_info("IPv6 is disabled by ipv6.disable=1 in cmdline");
+               return NULL;
+       }
+
        memset(&fl6, 0, sizeof(fl6));
        fl6.flowi6_oif = ndev->ifindex;
        memcpy(&fl6.saddr, saddr, sizeof(*saddr));
drivers/infiniband/sw/rxe/rxe_net.c +4 -1
@@ -608,6 +613,10 @@ static int rxe_net_ipv4_init(void)
 static int rxe_net_ipv6_init(void)
 {
 #if IS_ENABLED(CONFIG_IPV6)
+       if (!ipv6_mod_enabled()) {
+               pr_info("IPv6 is disabled by ipv6.disable=1 in cmdline");
+               return 0;
+       }
 
        recv_sockets.sk6 = rxe_setup_udp_tunnel(&init_net,
                                                htons(ROCE_V2_UDP_DPORT), true);
--

2.25.1

Quoted 36 lines · Yi Zhang
Quoted 8 lines · Zhu Yanjun
Quoted 7 lines · Yi Zhang

Hello

I found this failure after ipv6 disabled, is that expected?

# modprobe rdma_rxe
modprobe: ERROR: could not insert 'rdma_rxe': Operation not permitted
Quoted 6 lines
Quoted 3 lines
# dmesg
[  596.783484] rdma_rxe: failed to create udp socket. err = -97
[  596.789144] rdma_rxe: Failed to create IPv6 UDP tunnel

ipv6 in RXE is based on config_ipv6. If config_ipv6 is disabled. rxe will not implement ipv6 features.
How do you disable ipv6 in your hosts?

Sorry, I should give the detailed info, I disabled it by kernel parameter:

# cat /proc/cmdline  | grep -o ipv6.disable=1
ipv6.disable=1
Quoted 11 lines

Zhu Yanjun

Quoted 9 lines
# uname -r
5.11.0

Best Regards,
Yi Zhang

Reviewed-by
#6 lore raw
In reply to Zhu Yanjun
Quoted 22 lines · Zhu Yanjun

From 9dcdd09f3ca3cf222b563866acd91d18bc4b93d4 Mon Sep 17 00:00:00 2001
From: Zhu Yanjun <[email protected]>
Date: Sat, 27 Feb 2021 23:01:15 +0000
Subject: [PATCH 1/1] RDMA/rxe: Disable ipv6 features when ipv6.disable set in
cmdline

When ipv6.disable=1 is set in cmdline, ipv6 is actually disabled in the stack. As such, the operations of ipv6 in RXE will fail. So ipv6 features in RXE should also be disabled in RXE.

---
 drivers/infiniband/sw/rxe/rxe_net.c | 9 +++++++++
 1 file changed, 9 insertions(+)
drivers/infiniband/sw/rxe/rxe_net.c +0 -0
index 0701bd1ffd1a..6ef092cb575e 100644

drivers/infiniband/sw/rxe/rxe_net.c static struct dst_entry *rxe_find_route6(struct · line 80

Quoted 9 lines
drivers/infiniband/sw/rxe/rxe_net.c +5 -0
@@ -72,6 +72,11 @@ static struct dst_entry *rxe_find_route6(struct
 net_device *ndev,
        struct dst_entry *ndst;
        struct flowi6 fl6 = { { 0 } };
 
+       if (!ipv6_mod_enabled()) {
+               pr_info("IPv6 is disabled by ipv6.disable=1 in cmdline");
+               return NULL;
+       }
+

Except the info message, the change looks valid.

pr_info("IPv6 is disabled by ipv6.disable=1 in cmdline"); ->
pr_info("IPv6 is disabled");

Thanks,

author Tested-by
#7 lore raw
# modprobe rdma_rxe
# dmesg
[  598.134022] rdma_rxe: IPv6 is disabled by ipv6.disable=1 in cmdline
[  598.134026] rdma_rxe: loaded
Quoted 42 lines · Leon Romanovsky
Quoted 22 lines · Zhu Yanjun

From 9dcdd09f3ca3cf222b563866acd91d18bc4b93d4 Mon Sep 17 00:00:00 2001
From: Zhu Yanjun <[email protected]>
Date: Sat, 27 Feb 2021 23:01:15 +0000
Subject: [PATCH 1/1] RDMA/rxe: Disable ipv6 features when ipv6.disable set in
cmdline

When ipv6.disable=1 is set in cmdline, ipv6 is actually disabled in the stack. As such, the operations of ipv6 in RXE will fail. So ipv6 features in RXE should also be disabled in RXE.

---
  drivers/infiniband/sw/rxe/rxe_net.c | 9 +++++++++
  1 file changed, 9 insertions(+)
drivers/infiniband/sw/rxe/rxe_net.c +0 -0
index 0701bd1ffd1a..6ef092cb575e 100644

drivers/infiniband/sw/rxe/rxe_net.c static struct dst_entry *rxe_find_route6(struct · line 80

Quoted 9 lines
drivers/infiniband/sw/rxe/rxe_net.c +5 -0
@@ -72,6 +72,11 @@ static struct dst_entry *rxe_find_route6(struct
 net_device *ndev,
         struct dst_entry *ndst;
         struct flowi6 fl6 = { { 0 } };
 
+       if (!ipv6_mod_enabled()) {
+               pr_info("IPv6 is disabled by ipv6.disable=1 in cmdline");
+               return NULL;
+       }
+

Except the info message, the change looks valid.

pr_info("IPv6 is disabled by ipv6.disable=1 in cmdline"); ->
pr_info("IPv6 is disabled");

Thanks,