net_sched: two security bug fixes and test cases
Subsystems (5)
v3 cover net_sched: two security bug fixes and test cases
From: Cong Wang <[email protected]>
This patchset contains two bug fixes reported in security mailing list, and test cases for both of them.
---
v3: added Fixes tag for patch 1/4
removed a redudant ip link add/del in patch 2/4 (thanks to Pedro Tammela)
v2: replaced dummy2 with $DUMMY in pfifo_head_drop test
reduced the number of ping's in pfifo_head_drop test improved commit messages
Cong Wang (2):
netem: Update sch->q.qlen before qdisc_tree_reduce_backlog() selftests/tc-testing: Add a test case for qdisc_tree_reduce_backlog()
Quang Le (2):
pfifo_tail_enqueue: Drop new packet when sch->limit == 0 selftests/tc-testing: Add a test case for pfifo_head_drop qdisc when
limit==0
net/sched/sch_fifo.c | 3 ++ net/sched/sch_netem.c | 2 +- .../tc-testing/tc-tests/infra/qdiscs.json | 34 ++++++++++++++++++- .../tc-testing/tc-tests/qdiscs/fifo.json | 23 +++++++++++++ 4 files changed, 60 insertions(+), 2 deletions(-)
Signature · 1 line
2.34.1
PATCH v3 1/4 pfifo_tail_enqueue: Drop new packet when sch->limit == 0
From: Quang Le <[email protected]>
Expected behaviour:
In case we reach scheduler's limit, pfifo_tail_enqueue() will drop a packet in scheduler's queue and decrease scheduler's qlen by one. Then, pfifo_tail_enqueue() enqueue new packet and increase scheduler's qlen by one. Finally, pfifo_tail_enqueue() return NET_XMIT_CN status code.
Weird behaviour:
In case we set sch->limit == 0 and trigger pfifo_tail_enqueue() on a scheduler that has no packet, the 'drop a packet' step will do nothing. This means the scheduler's qlen still has value equal 0. Then, we continue to enqueue new packet and increase scheduler's qlen by one. In summary, we can leverage pfifo_tail_enqueue() to increase qlen by one and return NET_XMIT_CN status code.
The problem is:
Let's say we have two qdiscs: Qdisc_A and Qdisc_B.
- Qdisc_A's type must have '->graft()' function to create parent/child relationship. Let's say Qdisc_A's type is hfsc. Enqueue packet to this qdisc will trigger hfsc_enqueue.
- Qdisc_B's type is pfifo_head_drop. Enqueue packet to this qdisc will trigger pfifo_tail_enqueue.
- Qdisc_B is configured to have sch->limit == 0.
- Qdisc_A is configured to route the enqueued's packet to Qdisc_B.
Enqueue packet through Qdisc_A will lead to:
- hfsc_enqueue(Qdisc_A) -> pfifo_tail_enqueue(Qdisc_B)
- Qdisc_B->q.qlen += 1
- pfifo_tail_enqueue() return NET_XMIT_CN
- hfsc_enqueue() check for NET_XMIT_SUCCESS and see NET_XMIT_CN => hfsc_enqueue() don't increase qlen of Qdisc_A.
The whole process lead to a situation where Qdisc_A->q.qlen == 0 and Qdisc_B->q.qlen == 1. Replace 'hfsc' with other type (for example: 'drr') still lead to the same problem. This violate the design where parent's qlen should equal to the sum of its childrens'qlen.
Bug impact: This issue can be used for user->kernel privilege escalation when it is reachable.
--- net/sched/sch_fifo.c | 3 +++ 1 file changed, 3 insertions(+)
net/sched/sch_fifo.c +3 -0
@@ -40,6 +40,9 @@ static int pfifo_tail_enqueue(struct sk_buff *skb, struct Qdisc *sch, {unsigned int prev_backlog; +if (unlikely (READ_ONCE (sch->limit) ==0 )) +return qdisc_drop (skb, sch, to_free); +if (likely (sch->q.qlen <READ_ONCE (sch->limit)))return qdisc_enqueue_tail (skb, sch);
Signature · 1 line
2.34.1
Simon Horman
reviewerRe: [Patch net v3 1/4] pfifo_tail_enqueue: Drop new packet when sch->limit == 0
Quoted 31 lines · Cong Wang
From: Quang Le <[email protected]>
Expected behaviour:
In case we reach scheduler's limit,pfifo_tail_enqueue()will drop a packet in scheduler's queue and decrease scheduler's qlen by one. Then,pfifo_tail_enqueue()enqueue new packet and increase scheduler's qlen by one. Finally,pfifo_tail_enqueue()returnNET_XMIT_CNstatus code.Weird behaviour:
In case we setsch->limit == 0and triggerpfifo_tail_enqueue()on a scheduler that has no packet, the 'drop a packet' step will do nothing. This means the scheduler's qlen still has value equal 0. Then, we continue to enqueue new packet and increase scheduler's qlen by one. In summary, we can leveragepfifo_tail_enqueue()to increase qlen by one and returnNET_XMIT_CNstatus code.The problem is:
Let's say we have two qdiscs:Qdisc_AandQdisc_B.
- Qdisc_A's type must have '->graft()' function to create parent/child relationship. Let's say Qdisc_A's type ishfsc. Enqueue packet to this qdisc will triggerhfsc_enqueue.
- Qdisc_B's type ispfifo_head_drop. Enqueue packet to this qdisc will triggerpfifo_tail_enqueue.
-Qdisc_Bis configured to havesch->limit == 0.
-Qdisc_Ais configured to route the enqueued's packet toQdisc_B.Enqueue packet through
Qdisc_Awill lead to:
-hfsc_enqueue(Qdisc_A)->pfifo_tail_enqueue(Qdisc_B)
-Qdisc_B->q.qlen+= 1
-pfifo_tail_enqueue()returnNET_XMIT_CN
-hfsc_enqueue()check forNET_XMIT_SUCCESSand seeNET_XMIT_CN=>hfsc_enqueue()don't increase qlen ofQdisc_A.
Quoted 7 lines
The whole process lead to a situation where
Qdisc_A->q.qlen== 0 andQdisc_B->q.qlen== 1. Replace 'hfsc' with other type (for example: 'drr') still lead to the same problem. This violate the design where parent's qlen should equal to the sum of its childrens'qlen.Bug impact: This issue can be used for
user->kernelprivilege escalation when it is reachable.Fixes f70f90672a2c ("sched: add head drop fifo queue")
Hi Cong,
Not a proper review, but I believe the hash in mainline for the cited commit is 57dbb2d83d100ea.
Quoted 3 lines
Reported-by Quang Le <[email protected]>Signed-off-by Quang Le <[email protected]>Signed-off-by Cong Wang <[email protected]>
...
Re: [Patch net v3 1/4] pfifo_tail_enqueue: Drop new packet when sch->limit == 0
Quoted 5 lines · Simon Horman
Hi Cong,
Not a proper review, but I believe the hash in mainline for the cited commit is
57dbb2d83d100ea.
Oops, my bad. I noticed (probably) Jakub fixed it before applying, I really appreciate that.
Thanks!
PATCH v3 2/4 selftests/tc-testing: Add a test case for pfifo_head_drop qdisc when limit==0
In reply to Cong WangFrom: Quang Le <[email protected]>
When limit == 0, pfifo_tail_enqueue() must drop new packet and increase dropped packets count of the qdisc.
All test results:
1..16
ok 1 a519 - Add bfifo qdisc with system default parameters on egress
ok 2 585c - Add pfifo qdisc with system default parameters on egress
ok 3 a86e - Add bfifo qdisc with system default parameters on egress with handle of maximum value ok 4 9ac8 - Add bfifo qdisc on egress with queue size of 3000 bytes
ok 5 f4e6 - Add pfifo qdisc on egress with queue size of 3000 packets
ok 6 b1b1 - Add bfifo qdisc with system default parameters on egress with invalid handle exceeding maximum value ok 7 8d5e - Add bfifo qdisc on egress with unsupported argument
ok 8 7787 - Add pfifo qdisc on egress with unsupported argument
ok 9 c4b6 - Replace bfifo qdisc on egress with new queue size
ok 10 3df6 - Replace pfifo qdisc on egress with new queue size
ok 11 7a67 - Add bfifo qdisc on egress with queue size in invalid format
ok 12 1298 - Add duplicate bfifo qdisc on egress
ok 13 45a0 - Delete nonexistent bfifo qdisc
ok 14 972b - Add prio qdisc on egress with invalid format for handles
ok 15 4d39 - Delete bfifo qdisc twice
ok 16 d774 - Check pfifo_head_drop qdisc enqueue behaviour when limit == 0
--- .../tc-testing/tc-tests/qdiscs/fifo.json | 23 +++++++++++++++++++ 1 file changed, 23 insertions(+)
tools/testing/selftests/tc-testing/tc-tests/qdiscs/fifo.json +23 -0
@@ -313,6 +313,29 @@ "matchPattern": "qdisc bfifo 1: root", "matchCount": "0", "teardown": [ + ] + }, + { + "id": "d774", + "name": "Check pfifo_head_drop qdisc enqueue behaviour when limit == 0", + "category": [ + "qdisc", + "pfifo_head_drop" + ], + "plugins": { + "requires": "nsPlugin" + }, + "setup": [ + "$IP addr add 10.10.10.10/24 dev $DUMMY || true", + "$TC qdisc add dev $DUMMY root handle 1: pfifo_head_drop limit 0", + "$IP link set dev $DUMMY up || true" + ], + "cmdUnderTest": "ping -c2 -W0.01 -I $DUMMY 10.10.10.1", + "expExitCode": "1", + "verifyCmd": "$TC -s qdisc show dev $DUMMY", + "matchPattern": "dropped 2", + "matchCount": "1", + "teardown": [ ] } ]
Signature · 1 line
2.34.1
Simon Horman
reviewerRe: [Patch net v3 2/4] selftests/tc-testing: Add a test case for pfifo_head_drop qdisc when limit==0
Quoted 25 lines · Cong Wang
From: Quang Le <[email protected]>
When limit == 0,
pfifo_tail_enqueue()must drop new packet and increase dropped packets count of the qdisc.All test results:
1..16
ok 1 a519 - Add bfifo qdisc with system default parameters on egress
ok 2 585c - Add pfifo qdisc with system default parameters on egress
ok 3 a86e - Add bfifo qdisc with system default parameters on egress with handle of maximum value ok 4 9ac8 - Add bfifo qdisc on egress with queue size of 3000 bytes
ok 5 f4e6 - Add pfifo qdisc on egress with queue size of 3000 packets
ok 6 b1b1 - Add bfifo qdisc with system default parameters on egress with invalid handle exceeding maximum value ok 7 8d5e - Add bfifo qdisc on egress with unsupported argument
ok 8 7787 - Add pfifo qdisc on egress with unsupported argument
ok 9 c4b6 - Replace bfifo qdisc on egress with new queue size
ok 10 3df6 - Replace pfifo qdisc on egress with new queue size
ok 11 7a67 - Add bfifo qdisc on egress with queue size in invalid format
ok 12 1298 - Add duplicate bfifo qdisc on egress
ok 13 45a0 - Delete nonexistent bfifo qdisc
ok 14 972b - Add prio qdisc on egress with invalid format for handles
ok 15 4d39 - Delete bfifo qdisc twice
ok 16 d774 - Checkpfifo_head_dropqdisc enqueue behaviour when limit == 0
Quoted 2 lines
Signed-off-by Quang Le <[email protected]>Signed-off-by Cong Wang <[email protected]>
Hi Cong,
Unfortunately this test still seems to be failing in the CI.
# not ok 577 d774 - Check pfifo_head_drop qdisc enqueue behaviour when limit == 0 # Could not match regex pattern. Verify command output: # qdisc pfifo_head_drop 1: root refcnt 2 limit 0p # Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) # backlog 0b 0p requeues 0
https://github.com/p4tc-dev/tc-executor/blob/storage/artifacts/977485/1-tdc-sh/stdout
Signature · 1 line
pw-bot: cr
Jakub Kicinski
maintainerRe: [Patch net v3 2/4] selftests/tc-testing: Add a test case for pfifo_head_drop qdisc when limit==0
Quoted 33 lines · On Tue, 4 Feb 2025 11:37:03 +0000 Simon Horman wrote:
Quoted 25 lines · Cong Wang
From: Quang Le <[email protected]>
When limit == 0,
pfifo_tail_enqueue()must drop new packet and increase dropped packets count of the qdisc.All test results:
1..16
ok 1 a519 - Add bfifo qdisc with system default parameters on egress
ok 2 585c - Add pfifo qdisc with system default parameters on egress
ok 3 a86e - Add bfifo qdisc with system default parameters on egress with handle of maximum value ok 4 9ac8 - Add bfifo qdisc on egress with queue size of 3000 bytes
ok 5 f4e6 - Add pfifo qdisc on egress with queue size of 3000 packets
ok 6 b1b1 - Add bfifo qdisc with system default parameters on egress with invalid handle exceeding maximum value ok 7 8d5e - Add bfifo qdisc on egress with unsupported argument
ok 8 7787 - Add pfifo qdisc on egress with unsupported argument
ok 9 c4b6 - Replace bfifo qdisc on egress with new queue size
ok 10 3df6 - Replace pfifo qdisc on egress with new queue size
ok 11 7a67 - Add bfifo qdisc on egress with queue size in invalid format
ok 12 1298 - Add duplicate bfifo qdisc on egress
ok 13 45a0 - Delete nonexistent bfifo qdisc
ok 14 972b - Add prio qdisc on egress with invalid format for handles
ok 15 4d39 - Delete bfifo qdisc twice
ok 16 d774 - Checkpfifo_head_dropqdisc enqueue behaviour when limit == 0Quoted 2 lines
Signed-off-by Quang Le <[email protected]>Signed-off-by Cong Wang <[email protected]>Hi Cong,
Unfortunately this test still seems to be failing in the CI.
Quoted 7 lines
# not ok 577 d774 - Check pfifo_head_drop qdisc enqueue behaviour when limit == 0 # Could not match regex pattern. Verify command output: # qdisc pfifo_head_drop 1: root refcnt 2 limit 0p # Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) # backlog 0b 0p requeues 0https://github.com/p4tc-dev/tc-executor/blob/storage/artifacts/977485/1-tdc-sh/stdout
This is starting to feel too much like a setup issue. Pedro, would you be able to take this series and investigate why it fails on the TDC runner?
Re: [Patch net v3 2/4] selftests/tc-testing: Add a test case for pfifo_head_drop qdisc when limit==0
Quoted 40 lines · Jakub Kicinski
Quoted 39 lines · On Tue, 4 Feb 2025 11:37:03 +0000 Simon Horman wrote:
Quoted 25 lines · Cong Wang
From: Quang Le <[email protected]>
When limit == 0,
pfifo_tail_enqueue()must drop new packet and increase dropped packets count of the qdisc.All test results:
1..16
ok 1 a519 - Add bfifo qdisc with system default parameters on egress
ok 2 585c - Add pfifo qdisc with system default parameters on egress
ok 3 a86e - Add bfifo qdisc with system default parameters on egress with handle of maximum value ok 4 9ac8 - Add bfifo qdisc on egress with queue size of 3000 bytes
ok 5 f4e6 - Add pfifo qdisc on egress with queue size of 3000 packets
ok 6 b1b1 - Add bfifo qdisc with system default parameters on egress with invalid handle exceeding maximum value ok 7 8d5e - Add bfifo qdisc on egress with unsupported argument
ok 8 7787 - Add pfifo qdisc on egress with unsupported argument
ok 9 c4b6 - Replace bfifo qdisc on egress with new queue size
ok 10 3df6 - Replace pfifo qdisc on egress with new queue size
ok 11 7a67 - Add bfifo qdisc on egress with queue size in invalid format
ok 12 1298 - Add duplicate bfifo qdisc on egress
ok 13 45a0 - Delete nonexistent bfifo qdisc
ok 14 972b - Add prio qdisc on egress with invalid format for handles
ok 15 4d39 - Delete bfifo qdisc twice
ok 16 d774 - Checkpfifo_head_dropqdisc enqueue behaviour when limit == 0Quoted 2 lines
Signed-off-by Quang Le <[email protected]>Signed-off-by Cong Wang <[email protected]>Hi Cong,
Unfortunately this test still seems to be failing in the CI.
# not ok 577 d774 - Check pfifo_head_drop qdisc enqueue behaviour when limit == 0 # Could not match regex pattern. Verify command output: # qdisc pfifo_head_drop 1: root refcnt 2 limit 0p # Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) # backlog 0b 0p requeues 0
Quoted 5 lines
This is starting to feel too much like a setup issue. Pedro, would you be able to take this series and investigate why it fails on the TDC runner?
It should be OK now
Jakub Kicinski
maintainerRe: [Patch net v3 2/4] selftests/tc-testing: Add a test case for pfifo_head_drop qdisc when limit==0
Quoted 5 lines · On Tue, 4 Feb 2025 23:21:07 -0300 Pedro Tammela wrote:
Quoted 3 lines
This is starting to feel too much like a setup issue. Pedro, would you be able to take this series and investigate why it fails on the TDC runner?
It should be OK now
Thank you!
Revived in PW, should get into the run 22 min from now.
Simon Horman
reviewerRe: [Patch net v3 2/4] selftests/tc-testing: Add a test case for pfifo_head_drop qdisc when limit==0
Quoted 10 lines · Jakub Kicinski
Quoted 5 lines · On Tue, 4 Feb 2025 23:21:07 -0300 Pedro Tammela wrote:
Quoted 3 lines
This is starting to feel too much like a setup issue. Pedro, would you be able to take this series and investigate why it fails on the TDC runner?
It should be OK now
Thank you!
Revived in PW, should get into the run 22 min from now.
TDC and all other tests appear to be passing now :)
Re: [Patch net v3 2/4] selftests/tc-testing: Add a test case for pfifo_head_drop qdisc when limit==0
Quoted 6 lines · Simon Horman
Quoted 5 lines · Jakub Kicinski
Quoted 4 lines · On Tue, 4 Feb 2025 23:21:07 -0300 Pedro Tammela wrote:
Quoted 3 lines
This is starting to feel too much like a setup issue. Pedro, would you be able to take this series and investigate why it fails on the TDC runner?
Quoted 7 lines
Quoted 5 lines
Quoted 1 line
It should be OK now
Thank you!
Revived in PW, should get into the run 22 min from now.
TDC and all other tests appear to be passing now :)
Great. Thanks to everyone here for the help!
PATCH v3 3/4 netem: Update sch->q.qlen before qdisc_tree_reduce_backlog()
In reply to Cong WangFrom: Cong Wang <[email protected]>
qdisc_tree_reduce_backlog() notifies parent qdisc only if child qdisc becomes empty, therefore we need to reduce the backlog of the child qdisc before calling it. Otherwise it would miss the opportunity to call cops->qlen_notify(), in the case of DRR, it resulted in UAF since DRR uses ->qlen_notify() to maintain its active list.
--- net/sched/sch_netem.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
net/sched/sch_netem.c +1 -1
@@ -749,9 +749,9 @@ static struct sk_buff *netem_dequeue(struct Qdisc *sch)if (err !=NET_XMIT_SUCCESS ) {if (net_xmit_drop_count (err))qdisc_qstats_drop (sch); -qdisc_tree_reduce_backlog (sch,1 , pkt_len); sch->qstats.backlog -= pkt_len; sch->q.qlen--; +qdisc_tree_reduce_backlog (sch,1 , pkt_len); } goto tfifo_dequeue; }
Signature · 1 line
2.34.1
PATCH v3 4/4 selftests/tc-testing: Add a test case for qdisc_tree_reduce_backlog()
In reply to Cong WangFrom: Cong Wang <[email protected]>
Integrate the test case provided by Mingi Cho into TDC.
All test results:
1..4
ok 1 ca5e - Check class delete notification for ffff:
ok 2 e4b7 - Check class delete notification for root ffff:
ok 3 33a9 - Check ingress is not searchable on backlog update ok 4 a4b9 - Test class qlen notification
--- .../tc-testing/tc-tests/infra/qdiscs.json | 34 ++++++++++++++++++- 1 file changed, 33 insertions(+), 1 deletion(-)
tools/testing/selftests/tc-testing/tc-tests/infra/qdiscs.json +33 -1
@@ -94,5 +94,37 @@ "$TC qdisc del dev $DUMMY ingress", "$IP addr del 10.10.10.10/24 dev $DUMMY" ] - } + }, + { + "id": "a4b9", + "name": "Test class qlen notification", + "category": [ + "qdisc" + ], + "plugins": { + "requires": "nsPlugin" + }, + "setup": [ + "$IP link set dev $DUMMY up || true", + "$IP addr add 10.10.10.10/24 dev $DUMMY || true", + "$TC qdisc add dev $DUMMY root handle 1: drr", + "$TC filter add dev $DUMMY parent 1: basic classid 1:1", + "$TC class add dev $DUMMY parent 1: classid 1:1 drr", + "$TC qdisc add dev $DUMMY parent 1:1 handle 2: netem", + "$TC qdisc add dev $DUMMY parent 2: handle 3: drr", + "$TC filter add dev $DUMMY parent 3: basic action drop", + "$TC class add dev $DUMMY parent 3: classid 3:1 drr", + "$TC class del dev $DUMMY classid 1:1", + "$TC class add dev $DUMMY parent 1: classid 1:1 drr" + ], + "cmdUnderTest": "ping -c1 -W0.01 -I $DUMMY 10.10.10.1", + "expExitCode": "1", + "verifyCmd": "$TC qdisc ls dev $DUMMY", + "matchPattern": "drr 1: root", + "matchCount": "1", + "teardown": [ + "$TC qdisc del dev $DUMMY root handle 1: drr", + "$IP addr del 10.10.10.10/24 dev $DUMMY" + ] + } ]
Signature · 1 line
2.34.1
This series was applied to netdev/net.git (main) by Jakub Kicinski <[email protected]>:
Hello:
This series was applied to netdev/net.git (main)
by Jakub Kicinski <[email protected]>:
Quoted 10 lines · On Mon, 3 Feb 2025 16:58:37 -0800 you wrote:
From: Cong Wang <[email protected]>
This patchset contains two bug fixes reported in security mailing list, and test cases for both of them.
---
v3: added Fixes tag for patch 1/4
removed a redudant ip link add/del in patch 2/4 (thanks to Pedro Tammela)[...]
Here is the summary with links:
- [net,v3,1/4] pfifo_tail_enqueue: Drop new packet when sch->limit == 0
https://git.kernel.org/netdev/net/c/647cef20e649 - [net,v3,2/4] selftests/tc-testing: Add a test case for pfifo_head_drop qdisc when limit==0 https://git.kernel.org/netdev/net/c/3fe5648d1df1 - [net,v3,3/4] netem: Update sch->q.qlen before qdisc_tree_reduce_backlog() https://git.kernel.org/netdev/net/c/638ba5089324 - [net,v3,4/4] selftests/tc-testing: Add a test case for qdisc_tree_reduce_backlog() https://git.kernel.org/netdev/net/c/91aadc16ee73
You are awesome, thank you!
Signature · 2 lines
Deet-doot-dot, I am a bot. https://korg.docs.kernel.org/patchwork/pwbot.html