net: ipv4: remove erroneous advancement of list pointer
Subsystems (4)
cover net: ipv4: remove erroneous advancement of list pointer
Tariq reported a soft lockup on net-next that Mellanox was able to bisect to 2638eb8b50cf ("net: ipv4: provide __rcu annotation for ifa_list").
While reviewing above patch I found a regression when addresses have a lifetime specified.
Second patch extends rtnetlink.sh to trigger crash (without first patch applied).
PATCH 1/2 net: ipv4: remove erroneous advancement of list pointer
Causes crash when lifetime expires on an adress as garbage is dereferenced soon after.
This used to look like this:
for (ifap = &ifa->ifa_dev->ifa_list;
*ifap != NULL; ifap = &(*ifap)->ifa_next) {
if (*ifap == ifa) ...but this was changed to:
struct in_ifaddr *tmp;
ifap = &ifa->ifa_dev->ifa_list;
tmp = rtnl_dereference(*ifap);
while (tmp) {
tmp = rtnl_dereference(tmp->ifa_next); // Bogus
if (rtnl_dereference(*ifap) == ifa) {
... ifap = &tmp->ifa_next; // Can be NULL
tmp = rtnl_dereference(*ifap); // Dereference
}
}
Remove the bogus assigment/list entry skip.
--- net/ipv4/devinet.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-)
net/ipv4/devinet.c +1 -2
@@ -745,8 +745,7 @@ static void check_lifetime(struct work_struct *work) ifap = &ifa->ifa_dev->ifa_list; tmp =rtnl_dereference (*ifap);while (tmp) { - tmp =rtnl_dereference (tmp->ifa_next); -if (rtnl_dereference (*ifap) == ifa) { +if (tmp == ifa) {inet_del_ifa (ifa->ifa_dev, ifap,1 );break ;
Signature · 1 line
2.21.0
PATCH 2/2 selftests: rtnetlink: add addresses with fixed life time
In reply to Florian WestphalThis exercises kernel code path that deal with addresses that have a limited lifetime.
Without previous fix, this triggers following crash on net-next:
BUG: KASAN: null-ptr-deref in check_lifetime+0x403/0x670 Read of size 8 at addr 0000000000000010 by task kworker [..]
--- tools/testing/selftests/net/rtnetlink.sh | 21 +++++++++++++++++++++ 1 file changed, 21 insertions(+)
tools/testing/selftests/net/rtnetlink.sh +21 -0
@@ -249,6 +249,26 @@ kci_test_route_get() echo "PASS: route get" } +kci_test_addrlft() +{ + for i in $(seq 10 100) ;do + lft=$(((RANDOM%3) + 1)) + ip addr add 10.23.11.$i/32 dev "$devdummy" preferred_lft $lft valid_lft $((lft+1)) + check_err $? + done + + sleep 5 + + ip addr show dev "$devdummy" | grep "10.23.11." + if [ $? -eq 0 ]; then + echo "FAIL: preferred_lft addresses remaining" + check_err 1 + return + fi + + echo "PASS: preferred_lft addresses have expired" +} + kci_test_addrlabel() { ret=0
@@ -1140,6 +1160,7 @@ kci_test_rtnl() kci_test_polrouting kci_test_route_get + kci_test_addrlft kci_test_tc kci_test_gre kci_test_gretap
Signature · 1 line
2.21.0
Quoted 9 lines · Florian Westphal
Tariq reported a soft lockup on net-next that Mellanox was able to bisect to
2638eb8b50cf("net: ipv4: provide__rcuannotation forifa_list").While reviewing above patch I found a regression when addresses have a lifetime specified.
Second patch extends
rtnetlink.shto trigger crash (without first patch applied).
Thanks Florian.
Ran, can you please test?
Quoted 20 lines
-----Original Message-----
From: Tariq Toukan
Sent: Monday, June 17, 2019 19:16
To: Florian Westphal <[email protected]>; [email protected]
Cc: Ran Rozenstein <[email protected]>; Maor Gottlieb <[email protected]>; [email protected]
Subject: Re: [PATCH net-next 0/2] net: ipv4: remove erroneous advancement of list pointerQuoted 6 lines · Florian Westphal
Tariq reported a soft lockup on net-next that Mellanox was able to bisect to
2638eb8b50cf("net: ipv4: provide__rcuannotation forifa_list").While reviewing above patch I found a regression when addresses have a lifetime specified.
Quoted 7 lines
Quoted 3 lines
Second patch extends
rtnetlink.shto trigger crash (without first patch applied).Thanks Florian.
Ran, can you please test?
Tested, still reproduce.
Quoted 11 lines · Ran Rozenstein
Quoted 11 lines · Florian Westphal
Quoted 9 lines
Tariq reported a soft lockup on net-next that Mellanox was able to bisect to
2638eb8b50cf("net: ipv4: provide__rcuannotation forifa_list").While reviewing above patch I found a regression when addresses have a lifetime specified.
Second patch extends
rtnetlink.shto trigger crash (without first patch applied).
Quoted 5 lines
Quoted 3 lines
Thanks Florian.
Ran, can you please test?
Tested, still reproduce.
Can you be a little more specific? Is there any reproducer?
Quoted 25 lines
-----Original Message-----
From: Florian Westphal [mailto:[email protected]]
Sent: Tuesday, June 25, 2019 12:19
To: Ran Rozenstein <[email protected]>
Cc: Tariq Toukan <[email protected]>; Florian Westphal <[email protected]>; [email protected]; Maor Gottlieb <[email protected]>; [email protected]
Subject: Re: [PATCH net-next 0/2] net: ipv4: remove erroneous advancement of list pointerQuoted 3 lines · Ran Rozenstein
Quoted 3 lines · Florian Westphal
Quoted 2 lines
Tariq reported a soft lockup on net-next that Mellanox was able to bisect to
2638eb8b50cf("net: ipv4: provide__rcuannotation for
ifa_list").Quoted 8 lines
Quoted 8 lines
Quoted 7 lines
While reviewing above patch I found a regression when addresses have a lifetime specified.
Second patch extends
rtnetlink.shto trigger crash (without first patch applied).
Quoted 7 lines
Quoted 5 lines
Quoted 3 lines
Thanks Florian.
Ran, can you please test?
Tested, still reproduce.
Can you be a little more specific? Is there any reproducer?
The test dose stress on the interface by running this 2 commands in loop:
command is: /sbin/ip -f inet addr add $IP/16 brd + dev ens8f1 command is: ifconfig ens8f1 $IP netmask 255.255.0.0
when $IP change every iteration.
It execute every second when we see the reproduce somewhere between 40 to 200 seconds of execution.
Thanks,
Ran
Quoted 8 lines · Ran Rozenstein
The test dose stress on the interface by running this 2 commands in loop:
command is:
/sbin/ip-f inet addr add $IP/16 brd + dev ens8f1 command is: ifconfig ens8f1 $IP netmask 255.255.0.0when $IP change every iteration.
It execute every second when we see the reproduce somewhere between 40 to 200 seconds of execution.
I tried this without success:
DEV=dummy0
for j in $(seq 2 254);do
for i in $(seq 2 254);do
IP="10.$((RANDOM%254)).$((RANDOM%254)).$i"
ip -f inet addr add $IP/16 brd + dev $DEV
ifconfig $DEV $IP netmask 255.255.0.0 done
done
I'll let this loop overnight, but so far nothing turned up in dmesg (lockdep/kasan kernel).
Quoted 8 lines · Ran Rozenstein
The test dose stress on the interface by running this 2 commands in loop:
command is:
/sbin/ip-f inet addr add $IP/16 brd + dev ens8f1 command is: ifconfig ens8f1 $IP netmask 255.255.0.0when $IP change every iteration.
It execute every second when we see the reproduce somewhere between 40 to 200 seconds of execution.
I can reproduce this now, I'll submit a fix tomorrow.
David Miller
maintainerFrom: Florian Westphal <[email protected]>
Date: Mon, 17 Jun 2019 16:02:26 +0200
Quoted 8 lines
Tariq reported a soft lockup on net-next that Mellanox was able to bisect to
2638eb8b50cf("net: ipv4: provide__rcuannotation forifa_list").While reviewing above patch I found a regression when addresses have a lifetime specified.
Second patch extends
rtnetlink.shto trigger crash (without first patch applied).
Series applied, thanks Florian.