netpoll: Close race condition between poll_one_napi and napi_disable
Subsystems (2)
PATCH 1/1 netpoll: Close race condition between poll_one_napi and napi_disable
Drivers might call napi_disable while not holding the napi instance poll_lock. In those instances, its possible for a race condition to exist between poll_one_napi and napi_disable. That is to say, poll_one_napi only tests the NAPI_STATE_SCHED bit to see if there is work to do during a poll, and as such the following may happen:
CPU0 CPU1ndo_tx_timeout napi_poll_dev
napi_disable poll_one_napi
test_and_set_bit (ret 0)
test_bit (ret 1)
reset adapter napi_poll_routine
If the adapter gets a tx timeout without a napi instance scheduled, its possible for the adapter to think it has exclusive access to the hardware (as the napi instance is now scheduled via the napi_disable call), while the netpoll code thinks there is simply work to do. The result is parallel hardware access leading to corrupt data structures in the driver, and a crash.
Additionaly, there is another, more critical race between netpoll and napi_disable. The disabled napi state is actually identical to the scheduled state for a given napi instance. The implication being that, if a napi instance is disabled, a netconsole instance would see the napi state of the device as having been scheduled, and poll it, likely while the driver was dong something requiring exclusive access. In the case above, its fairly clear that not having the rings in a state ready to be polled will cause any number of crashes.
The fix should be pretty easy. netpoll uses its own bit to indicate that that the napi instance is in a state of being serviced by netpoll (NAPI_STATE_NPSVC). We can just gate disabling on that bit as well as the sched bit. That should prevent netpoll from conducting a napi poll if we convert its set bit to a test_and_set_bit operation to provide mutual exclusion
--- include/linux/netdevice.h | 2 ++ net/core/dev.c | 2 ++ net/core/netpoll.c | 12 ++++++++++-- 3 files changed, 14 insertions(+), 2 deletions(-)
include/linux/netdevice.h +2 -0
@@ -507,6 +507,8 @@ static inline void napi_enable(struct napi_struct *n)BUG_ON (!test_bit (NAPI_STATE_SCHED , &n->state));smp_mb__before_atomic ();clear_bit (NAPI_STATE_SCHED , &n->state); +clear_bit (NAPI_STATE_NPSVC , &n->state); + }#ifdef CONFIG_SMP
net/core/dev.c +2 -0
@@ -4723,6 +4723,8 @@ void napi_disable(struct napi_struct *n)while (test_and_set_bit (NAPI_STATE_SCHED , &n->state))msleep (1 ); +while (test_and_set_bit (NAPI_STATE_NPSVC , &n->state)) +msleep (1 );hrtimer_cancel (&n->timer);
net/core/netpoll.c +10 -2
@@ -142,7 +142,7 @@ static void queue_process(struct work_struct *work)*/ static int poll_one_napi (struct napi_struct *napi,int budget) { -int work; +int work =0 ;/* net_rx_action's ->poll() invocations and our's are * synchronized by this test which is only made while
@@ -151,7 +151,14 @@ static int poll_one_napi(struct napi_struct *napi, int budget)if (!test_bit (NAPI_STATE_SCHED , &napi->state))return budget; -set_bit (NAPI_STATE_NPSVC , &napi->state); +/* +* If we set this bit but see that it has already been set, +* that indicates that napi has been disabled and we need +* to abort this operation +*/ + +if (test_and_set_bit (NAPI_STATE_NPSVC , &napi->state)) + goto out; work = napi->poll (napi, budget);WARN_ONCE (work > budget,"%pF exceeded budget in poll\n" , napi->poll);
@@ -159,6 +166,7 @@ static int poll_one_napi(struct napi_struct *napi, int budget)clear_bit (NAPI_STATE_NPSVC , &napi->state); +out:return budget - work; }
Signature · 1 line
2.1.0
Quoted 114 lines · Neil Horman
Drivers might call
napi_disablewhile not holding the napi instancepoll_lock. In those instances, its possible for a race condition to exist betweenpoll_one_napiandnapi_disable. That is to say,poll_one_napionly tests theNAPI_STATE_SCHEDbit to see if there is work to do during a poll, and as such the following may happen:CPU0 CPU1
ndo_tx_timeoutnapi_poll_dev
napi_disablepoll_one_napi
test_and_set_bit(ret 0)
test_bit(ret 1)
reset adapternapi_poll_routineIf the adapter gets a tx timeout without a napi instance scheduled, its possible for the adapter to think it has exclusive access to the hardware (as the napi instance is now scheduled via the
napi_disablecall), while the netpoll code thinks there is simply work to do. The result is parallel hardware access leading to corrupt data structures in the driver, and a crash.Additionaly, there is another, more critical race between netpoll and
napi_disable. The disabled napi state is actually identical to the scheduled state for a given napi instance. The implication being that, if a napi instance is disabled, a netconsole instance would see the napi state of the device as having been scheduled, and poll it, likely while the driver was dong something requiring exclusive access. In the case above, its fairly clear that not having the rings in a state ready to be polled will cause any number of crashes.The fix should be pretty easy. netpoll uses its own bit to indicate that that the napi instance is in a state of being serviced by netpoll (
NAPI_STATE_NPSVC). We can just gate disabling on that bit as well as the sched bit. That should prevent netpoll from conducting a napi poll if we convert its set bit to atest_and_set_bitoperation to provide mutual exclusionSigned-off-by Neil Horman <[email protected]>CC "David S. Miller" <[email protected]>Tested-by [email protected]--- include/linux/netdevice.h | 2 ++ net/core/dev.c | 2 ++ net/core/netpoll.c | 12 ++++++++++-- 3 files changed, 14 insertions(+), 2 deletions(-)include/linux/netdevice.h +2
-0@@ -507,6 +507,8 @@ static inline void napi_enable(struct napi_struct *n)BUG_ON (!test_bit (NAPI_STATE_SCHED , &n->state));smp_mb__before_atomic ();clear_bit (NAPI_STATE_SCHED , &n->state); +clear_bit (NAPI_STATE_NPSVC , &n->state); + }#ifdef CONFIG_SMP net/core/dev.c +2
-0@@ -4723,6 +4723,8 @@ void napi_disable(struct napi_struct *n)while (test_and_set_bit (NAPI_STATE_SCHED , &n->state))msleep (1 ); +while (test_and_set_bit (NAPI_STATE_NPSVC , &n->state)) +msleep (1 );hrtimer_cancel (&n->timer);net/core/netpoll.c +10
-2@@ -142,7 +142,7 @@ static void queue_process(struct work_struct *work)*/ static int poll_one_napi (struct napi_struct *napi,int budget) { -int work; +int work =0 ;/* net_rx_action's ->poll() invocations and our's are * synchronized by this test which is only made while @@ -151,7 +151,14 @@ static int poll_one_napi(struct napi_struct *napi, int budget)if (!test_bit (NAPI_STATE_SCHED , &napi->state))return budget; -set_bit (NAPI_STATE_NPSVC , &napi->state); +/* +* If we set this bit but see that it has already been set, +* that indicates that napi has been disabled and we need +* to abort this operation +*/ + +if (test_and_set_bit (NAPI_STATE_NPSVC , &napi->state)) + goto out; work = napi->poll (napi, budget);WARN_ONCE (work > budget,"%pF exceeded budget in poll\n" , napi->poll);@@ -159,6 +166,7 @@ static int poll_one_napi(struct napi_struct *napi, int budget)clear_bit (NAPI_STATE_NPSVC , &napi->state); +out:return budget - work; }Signature · 6 lines
2.1.0 -- To unsubscribe from this list: send the line "unsubscribe netdev" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
Shoot, I forgot to change my subject prefix, sorry about that. Dave this applies to net-next, shall I resubmit with a proper prefix, or are you good with it as is?
Neil
Quoted 110 lines · Neil Horman
Quoted 109 lines · Neil Horman
Drivers might call
napi_disablewhile not holding the napi instancepoll_lock. In those instances, its possible for a race condition to exist betweenpoll_one_napiandnapi_disable. That is to say,poll_one_napionly tests theNAPI_STATE_SCHEDbit to see if there is work to do during a poll, and as such the following may happen:CPU0 CPU1
ndo_tx_timeoutnapi_poll_dev
napi_disablepoll_one_napi
test_and_set_bit(ret 0)
test_bit(ret 1)
reset adapternapi_poll_routineIf the adapter gets a tx timeout without a napi instance scheduled, its possible for the adapter to think it has exclusive access to the hardware (as the napi instance is now scheduled via the
napi_disablecall), while the netpoll code thinks there is simply work to do. The result is parallel hardware access leading to corrupt data structures in the driver, and a crash.Additionaly, there is another, more critical race between netpoll and
napi_disable. The disabled napi state is actually identical to the scheduled state for a given napi instance. The implication being that, if a napi instance is disabled, a netconsole instance would see the napi state of the device as having been scheduled, and poll it, likely while the driver was dong something requiring exclusive access. In the case above, its fairly clear that not having the rings in a state ready to be polled will cause any number of crashes.The fix should be pretty easy. netpoll uses its own bit to indicate that that the napi instance is in a state of being serviced by netpoll (
NAPI_STATE_NPSVC). We can just gate disabling on that bit as well as the sched bit. That should prevent netpoll from conducting a napi poll if we convert its set bit to atest_and_set_bitoperation to provide mutual exclusionSigned-off-by Neil Horman <[email protected]>CC "David S. Miller" <[email protected]>Tested-by [email protected]--- include/linux/netdevice.h | 2 ++ net/core/dev.c | 2 ++ net/core/netpoll.c | 12 ++++++++++-- 3 files changed, 14 insertions(+), 2 deletions(-)include/linux/netdevice.h +2
-0@@ -507,6 +507,8 @@ static inline void napi_enable(struct napi_struct *n)BUG_ON (!test_bit (NAPI_STATE_SCHED , &n->state));smp_mb__before_atomic ();clear_bit (NAPI_STATE_SCHED , &n->state); +clear_bit (NAPI_STATE_NPSVC , &n->state); + }#ifdef CONFIG_SMP net/core/dev.c +2
-0@@ -4723,6 +4723,8 @@ void napi_disable(struct napi_struct *n)while (test_and_set_bit (NAPI_STATE_SCHED , &n->state))msleep (1 ); +while (test_and_set_bit (NAPI_STATE_NPSVC , &n->state)) +msleep (1 );hrtimer_cancel (&n->timer);net/core/netpoll.c +10
-2@@ -142,7 +142,7 @@ static void queue_process(struct work_struct *work)*/ static int poll_one_napi (struct napi_struct *napi,int budget) { -int work; +int work =0 ;/* net_rx_action's ->poll() invocations and our's are * synchronized by this test which is only made while @@ -151,7 +151,14 @@ static int poll_one_napi(struct napi_struct *napi, int budget)if (!test_bit (NAPI_STATE_SCHED , &napi->state))return budget; -set_bit (NAPI_STATE_NPSVC , &napi->state); +/* +* If we set this bit but see that it has already been set, +* that indicates that napi has been disabled and we need +* to abort this operation +*/ + +if (test_and_set_bit (NAPI_STATE_NPSVC , &napi->state)) + goto out; work = napi->poll (napi, budget);WARN_ONCE (work > budget,"%pF exceeded budget in poll\n" , napi->poll);@@ -159,6 +166,7 @@ static int poll_one_napi(struct napi_struct *napi, int budget)clear_bit (NAPI_STATE_NPSVC , &napi->state); +out:return budget - work; }Signature · 1 line
2.1.0
Quoted 8 lines
Quoted 5 lines
--
To unsubscribe from this list: send the line "unsubscribe netdev" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.htmlShoot, I forgot to change my subject prefix, sorry about that. Dave this applies to net-next, shall I resubmit with a proper prefix, or are you good with it as is?
It looks like this patch introduces some white-space errors as well. The comment block has one trailing white space, and 4 spaces before tabs. You might want to resubmit with that fixed.
- Alex
Quoted 110 lines · Alexander Duyck
Quoted 100 lines · Neil Horman
Quoted 99 lines · Neil Horman
Drivers might call
napi_disablewhile not holding the napi instancepoll_lock. In those instances, its possible for a race condition to exist betweenpoll_one_napiandnapi_disable. That is to say,poll_one_napionly tests theNAPI_STATE_SCHEDbit to see if there is work to do during a poll, and as such the following may happen:CPU0 CPU1
ndo_tx_timeoutnapi_poll_dev
napi_disablepoll_one_napi
test_and_set_bit(ret 0)
test_bit(ret 1)
reset adapternapi_poll_routineIf the adapter gets a tx timeout without a napi instance scheduled, its possible for the adapter to think it has exclusive access to the hardware (as the napi instance is now scheduled via the
napi_disablecall), while the netpoll code thinks there is simply work to do. The result is parallel hardware access leading to corrupt data structures in the driver, and a crash.Additionaly, there is another, more critical race between netpoll and
napi_disable. The disabled napi state is actually identical to the scheduled state for a given napi instance. The implication being that, if a napi instance is disabled, a netconsole instance would see the napi state of the device as having been scheduled, and poll it, likely while the driver was dong something requiring exclusive access. In the case above, its fairly clear that not having the rings in a state ready to be polled will cause any number of crashes.The fix should be pretty easy. netpoll uses its own bit to indicate that that the napi instance is in a state of being serviced by netpoll (
NAPI_STATE_NPSVC). We can just gate disabling on that bit as well as the sched bit. That should prevent netpoll from conducting a napi poll if we convert its set bit to atest_and_set_bitoperation to provide mutual exclusionSigned-off-by Neil Horman <[email protected]>CC "David S. Miller" <[email protected]>Tested-by [email protected]--- include/linux/netdevice.h | 2 ++ net/core/dev.c | 2 ++ net/core/netpoll.c | 12 ++++++++++-- 3 files changed, 14 insertions(+), 2 deletions(-)include/linux/netdevice.h +2
-0@@ -507,6 +507,8 @@ static inline void napi_enable(struct napi_struct *n)BUG_ON (!test_bit (NAPI_STATE_SCHED , &n->state));smp_mb__before_atomic ();clear_bit (NAPI_STATE_SCHED , &n->state); +clear_bit (NAPI_STATE_NPSVC , &n->state); + }#ifdef CONFIG_SMP net/core/dev.c +2
-0@@ -4723,6 +4723,8 @@ void napi_disable(struct napi_struct *n)while (test_and_set_bit (NAPI_STATE_SCHED , &n->state))msleep (1 ); +while (test_and_set_bit (NAPI_STATE_NPSVC , &n->state)) +msleep (1 );hrtimer_cancel (&n->timer);net/core/netpoll.c +10
-2@@ -142,7 +142,7 @@ static void queue_process(struct work_struct *work)*/ static int poll_one_napi (struct napi_struct *napi,int budget) { -int work; +int work =0 ;/* net_rx_action's ->poll() invocations and our's are * synchronized by this test which is only made while @@ -151,7 +151,14 @@ static int poll_one_napi(struct napi_struct *napi, int budget)if (!test_bit (NAPI_STATE_SCHED , &napi->state))return budget; -set_bit (NAPI_STATE_NPSVC , &napi->state); +/* +* If we set this bit but see that it has already been set, +* that indicates that napi has been disabled and we need +* to abort this operation +*/ + +if (test_and_set_bit (NAPI_STATE_NPSVC , &napi->state)) + goto out; work = napi->poll (napi, budget);WARN_ONCE (work > budget,"%pF exceeded budget in poll\n" , napi->poll);@@ -159,6 +166,7 @@ static int poll_one_napi(struct napi_struct *napi, int budget)clear_bit (NAPI_STATE_NPSVC , &napi->state); +out:return budget - work; }Signature · 1 line
2.1.0Quoted 8 lines
Quoted 5 lines
--
To unsubscribe from this list: send the line "unsubscribe netdev" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.htmlShoot, I forgot to change my subject prefix, sorry about that. Dave this applies to net-next, shall I resubmit with a proper prefix, or are you good with it as is?
Quoted 4 lines
It looks like this patch introduces some white-space errors as well. The comment block has one trailing white space, and 4 spaces before tabs. You might want to resubmit with that fixed.
I don't think so. Its got one trailing space that I see, you're right about
that, but the only leading white space is directly in front of the astersks, which is necessecary for alignment, its identical to other simmilar comments in the file.
No matter though, I'll just repost it at this point Neil
Quoted 1 line
- Alex
PATCH v2 1/1 netpoll: Close race condition between poll_one_napi and napi_disable
In reply to Neil HormanDrivers might call napi_disable while not holding the napi instance poll_lock. In those instances, its possible for a race condition to exist between poll_one_napi and napi_disable. That is to say, poll_one_napi only tests the NAPI_STATE_SCHED bit to see if there is work to do during a poll, and as such the following may happen:
CPU0 CPU1ndo_tx_timeout napi_poll_dev
napi_disable poll_one_napi
test_and_set_bit (ret 0)
test_bit (ret 1)
reset adapter napi_poll_routine
If the adapter gets a tx timeout without a napi instance scheduled, its possible for the adapter to think it has exclusive access to the hardware (as the napi instance is now scheduled via the napi_disable call), while the netpoll code thinks there is simply work to do. The result is parallel hardware access leading to corrupt data structures in the driver, and a crash.
Additionaly, there is another, more critical race between netpoll and napi_disable. The disabled napi state is actually identical to the scheduled state for a given napi instance. The implication being that, if a napi instance is disabled, a netconsole instance would see the napi state of the device as having been scheduled, and poll it, likely while the driver was dong something requiring exclusive access. In the case above, its fairly clear that not having the rings in a state ready to be polled will cause any number of crashes.
The fix should be pretty easy. netpoll uses its own bit to indicate that that the napi instance is in a state of being serviced by netpoll (NAPI_STATE_NPSVC). We can just gate disabling on that bit as well as the sched bit. That should prevent netpoll from conducting a napi poll if we convert its set bit to a test_and_set_bit operation to provide mutual exclusion
Change notes:
V2)
Remove a trailing whtiespace
Resubmit with proper subject prefix
--- include/linux/netdevice.h | 2 ++ net/core/dev.c | 2 ++ net/core/netpoll.c | 12 ++++++++++-- 3 files changed, 14 insertions(+), 2 deletions(-)
include/linux/netdevice.h +2 -0
@@ -507,6 +507,8 @@ static inline void napi_enable(struct napi_struct *n)BUG_ON (!test_bit (NAPI_STATE_SCHED , &n->state));smp_mb__before_atomic ();clear_bit (NAPI_STATE_SCHED , &n->state); +clear_bit (NAPI_STATE_NPSVC , &n->state); + }#ifdef CONFIG_SMP
net/core/dev.c +2 -0
@@ -4723,6 +4723,8 @@ void napi_disable(struct napi_struct *n)while (test_and_set_bit (NAPI_STATE_SCHED , &n->state))msleep (1 ); +while (test_and_set_bit (NAPI_STATE_NPSVC , &n->state)) +msleep (1 );hrtimer_cancel (&n->timer);
net/core/netpoll.c +10 -2
@@ -142,7 +142,7 @@ static void queue_process(struct work_struct *work)*/ static int poll_one_napi (struct napi_struct *napi,int budget) { -int work; +int work =0 ;/* net_rx_action's ->poll() invocations and our's are * synchronized by this test which is only made while
@@ -151,7 +151,14 @@ static int poll_one_napi(struct napi_struct *napi, int budget)if (!test_bit (NAPI_STATE_SCHED , &napi->state))return budget; -set_bit (NAPI_STATE_NPSVC , &napi->state); +/* +* If we set this bit but see that it has already been set, +* that indicates that napi has been disabled and we need +* to abort this operation +*/ + +if (test_and_set_bit (NAPI_STATE_NPSVC , &napi->state)) + goto out; work = napi->poll (napi, budget);WARN_ONCE (work > budget,"%pF exceeded budget in poll\n" , napi->poll);
@@ -159,6 +166,7 @@ static int poll_one_napi(struct napi_struct *napi, int budget)clear_bit (NAPI_STATE_NPSVC , &napi->state); +out:return budget - work; }
Signature · 1 line
2.1.0
include/linux/netdevice.h static inline void napi_enable(struct napi_struct *n) · line 512
Quoted 3 lines · On Tue, 22 Sep 2015 16:01:36 -0400 Neil Horman <[email protected]> wrote:
include/linux/netdevice.h +2
-0+clear_bit (NAPI_STATE_NPSVC , &n->state); + }
why introduce extra line here?
net/core/netpoll.c static int poll_one_napi(struct napi_struct *napi, int budget) · line 160
Quoted 7 lines
net/core/netpoll.c +7
-0+/* +* If we set this bit but see that it has already been set, +* that indicates that napi has been disabled and we need +* to abort this operation +*/ + +if (test_and_set_bit (NAPI_STATE_NPSVC , &napi->state))
And why introduce line after comment before code.
My preference is to have comment as close to code as possible.
Quoted 8 lines · Stephen Hemminger
include/linux/netdevice.h static inline void napi_enable(struct napi_struct *n) · line 512
Quoted 3 lines · On Tue, 22 Sep 2015 16:01:36 -0400 Neil Horman <[email protected]> wrote:
include/linux/netdevice.h +2
-0+clear_bit (NAPI_STATE_NPSVC , &n->state); + }why introduce extra line here?
Quoted 10 lines
net/core/netpoll.c static int poll_one_napi(struct napi_struct *napi, int budget) · line 160
Quoted 7 lines
net/core/netpoll.c +7
-0+/* +* If we set this bit but see that it has already been set, +* that indicates that napi has been disabled and we need +* to abort this operation +*/ + +if (test_and_set_bit (NAPI_STATE_NPSVC , &napi->state))And why introduce line after comment before code.
My preference is to have comment as close to code as possible.
Mine is to add spaces, as I feel the code is more readable if its a bit more spread out.
Quoted 5 lines
--
To unsubscribe from this list: send the line "unsubscribe netdev" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
David Miller
maintainerFrom: Neil Horman <[email protected]>
Date: Tue, 22 Sep 2015 16:01:36 -0400
net/core/netpoll.c static int poll_one_napi(struct napi_struct *napi, int budget) · line 161
Quoted 10 lines
net/core/netpoll.c +8
-1-set_bit (NAPI_STATE_NPSVC , &napi->state); +/* +* If we set this bit but see that it has already been set, +* that indicates that napi has been disabled and we need +* to abort this operation +*/ + +if (test_and_set_bit (NAPI_STATE_NPSVC , &napi->state)) + goto out;
Networking comments should be styled:
/* Like * this. */
Also you need a space after the "if" and before the openning parenthesis.
And like Stephen I think the empty line between the comment and the if() statement is superfluous and eats up precious vertical space on the screen :-)
Thanks.
PATCH v3 1/1 netpoll: Close race condition between poll_one_napi and napi_disable
In reply to Neil HormanDrivers might call napi_disable while not holding the napi instance poll_lock. In those instances, its possible for a race condition to exist between poll_one_napi and napi_disable. That is to say, poll_one_napi only tests the NAPI_STATE_SCHED bit to see if there is work to do during a poll, and as such the following may happen:
CPU0 CPU1ndo_tx_timeout napi_poll_dev
napi_disable poll_one_napi
test_and_set_bit (ret 0)
test_bit (ret 1)
reset adapter napi_poll_routine
If the adapter gets a tx timeout without a napi instance scheduled, its possible for the adapter to think it has exclusive access to the hardware (as the napi instance is now scheduled via the napi_disable call), while the netpoll code thinks there is simply work to do. The result is parallel hardware access leading to corrupt data structures in the driver, and a crash.
Additionaly, there is another, more critical race between netpoll and napi_disable. The disabled napi state is actually identical to the scheduled state for a given napi instance. The implication being that, if a napi instance is disabled, a netconsole instance would see the napi state of the device as having been scheduled, and poll it, likely while the driver was dong something requiring exclusive access. In the case above, its fairly clear that not having the rings in a state ready to be polled will cause any number of crashes.
The fix should be pretty easy. netpoll uses its own bit to indicate that that the napi instance is in a state of being serviced by netpoll (NAPI_STATE_NPSVC). We can just gate disabling on that bit as well as the sched bit. That should prevent netpoll from conducting a napi poll if we convert its set bit to a test_and_set_bit operation to provide mutual exclusion
Change notes:
V2)
Remove a trailing whtiespace
Resubmit with proper subject prefix
V3)
Clean up spacing nits
--- include/linux/netdevice.h | 1 + net/core/dev.c | 2 ++ net/core/netpoll.c | 11 +++++++++-- 3 files changed, 12 insertions(+), 2 deletions(-)
include/linux/netdevice.h +1 -0
@@ -507,6 +507,7 @@ static inline void napi_enable(struct napi_struct *n)BUG_ON (!test_bit (NAPI_STATE_SCHED , &n->state));smp_mb__before_atomic ();clear_bit (NAPI_STATE_SCHED , &n->state); +clear_bit (NAPI_STATE_NPSVC , &n->state); }#ifdef CONFIG_SMP
net/core/dev.c +2 -0
@@ -4723,6 +4723,8 @@ void napi_disable(struct napi_struct *n)while (test_and_set_bit (NAPI_STATE_SCHED , &n->state))msleep (1 ); +while (test_and_set_bit (NAPI_STATE_NPSVC , &n->state)) +msleep (1 );hrtimer_cancel (&n->timer);
net/core/netpoll.c +9 -2
@@ -142,7 +142,7 @@ static void queue_process(struct work_struct *work)*/ static int poll_one_napi (struct napi_struct *napi,int budget) { -int work; +int work =0 ;/* net_rx_action's ->poll() invocations and our's are * synchronized by this test which is only made while
@@ -151,7 +151,13 @@ static int poll_one_napi(struct napi_struct *napi, int budget)if (!test_bit (NAPI_STATE_SCHED , &napi->state))return budget; -set_bit (NAPI_STATE_NPSVC , &napi->state); +/* If we set this bit but see that it has already been set, +* that indicates that napi has been disabled and we need +* to abort this operation +*/ + +if (test_and_set_bit (NAPI_STATE_NPSVC , &napi->state)) + goto out; work = napi->poll (napi, budget);WARN_ONCE (work > budget,"%pF exceeded budget in poll\n" , napi->poll);
@@ -159,6 +165,7 @@ static int poll_one_napi(struct napi_struct *napi, int budget)clear_bit (NAPI_STATE_NPSVC , &napi->state); +out:return budget - work; }
Signature · 1 line
2.1.0
From: Neil Horman <[email protected]>
Date: Wed, 23 Sep 2015 14:57:58 -0400
Quoted 38 lines
Drivers might call
napi_disablewhile not holding the napi instancepoll_lock. In those instances, its possible for a race condition to exist betweenpoll_one_napiandnapi_disable. That is to say,poll_one_napionly tests theNAPI_STATE_SCHEDbit to see if there is work to do during a poll, and as such the following may happen:CPU0 CPU1
ndo_tx_timeoutnapi_poll_dev
napi_disablepoll_one_napi
test_and_set_bit(ret 0)
test_bit(ret 1)
reset adapternapi_poll_routineIf the adapter gets a tx timeout without a napi instance scheduled, its possible for the adapter to think it has exclusive access to the hardware (as the napi instance is now scheduled via the
napi_disablecall), while the netpoll code thinks there is simply work to do. The result is parallel hardware access leading to corrupt data structures in the driver, and a crash.Additionaly, there is another, more critical race between netpoll and
napi_disable. The disabled napi state is actually identical to the scheduled state for a given napi instance. The implication being that, if a napi instance is disabled, a netconsole instance would see the napi state of the device as having been scheduled, and poll it, likely while the driver was dong something requiring exclusive access. In the case above, its fairly clear that not having the rings in a state ready to be polled will cause any number of crashes.The fix should be pretty easy. netpoll uses its own bit to indicate that that the napi instance is in a state of being serviced by netpoll (
NAPI_STATE_NPSVC). We can just gate disabling on that bit as well as the sched bit. That should prevent netpoll from conducting a napi poll if we convert its set bit to atest_and_set_bitoperation to provide mutual exclusionChange notes:
V2)
Remove a trailing whtiespace
Resubmit with proper subject prefix
Quoted 7 lines
V3)
Clean up spacing nitsSigned-off-by Neil Horman <[email protected]>CC "David S. Miller" <[email protected]>Tested-by [email protected]
Applied, thanks Neil.