net: forbid the reentrant RX
PATCH 1/2 net: forbid the reentrant RX
The memory API allows DMA into NIC's MMIO area. This means the NIC's RX routine must be reentrant. Instead of auditing all the NIC, we can simply detect the reentrancy and return early. The queue->delivering is set and cleared by qemu_net_queue_deliver() for other queue helpers to know whether the delivering in on going (NIC's receive is being called). We can check it and return early in qemu_net_queue_flush() to forbid reentrant RX.
--- net/queue.c | 3 +++ 1 file changed, 3 insertions(+)
net/queue.c +3 -0
@@ -250,6 +250,9 @@ void qemu_net_queue_purge(NetQueue *queue, NetClientState *from)bool qemu_net_queue_flush (NetQueue *queue) { +if (queue->delivering) +return false; +while (!QTAILQ_EMPTY (&queue->packets)) {NetPacket *packet;int ret;
Signature · 1 line
2.20.1
PATCH 2/2 e1000e: make TX reentrant
In loopback mode, e1000e RX can DMA into TX doorbell which requires TX to be reentrant. This patch make e1000e's TX routine reentrant by introducing a per device boolean for recording whether or not a TX rountine is being called and return early.
--- hw/net/e1000e_core.c | 8 ++++++++ hw/net/e1000e_core.h | 1 + 2 files changed, 9 insertions(+)
hw/net/e1000e_core.c +8 -0
@@ -923,6 +923,12 @@ e1000e_start_xmit(E1000ECore *core, const E1000E_TxRing *txr)return ; } +if (core->sending) { +return ; + } + + core->sending = true; +while (!e1000e_ring_empty (core, txi)) { base =e1000e_ring_head_descr (core, txi);
@@ -940,6 +946,8 @@ e1000e_start_xmit(E1000ECore *core, const E1000E_TxRing *txr)if (!ide || !e1000e_intrmgr_delay_tx_causes (core, &cause)) {e1000e_set_interrupt_cause (core, cause); } + + core->sending = false; } static bool
hw/net/e1000e_core.h +1 -0
@@ -114,6 +114,7 @@ struct E1000Core {void (*owner_start_recv)(PCIDevice *d);uint32_t msi_causes_pending; +bool sending; };void
Signature · 1 line
2.20.1
Re: [PATCH 2/2] e1000e: make TX reentrant
Quoted 6 lines · Jason Wang <[email protected]> 于2020年7月22日周三 下午4:58写道:
In loopback mode, e1000e RX can DMA into TX doorbell which requires TX to be reentrant. This patch make e1000e's TX routine reentrant by introducing a per device boolean for recording whether or not a TX rountine is being called and return early.
Could we introduce a per-queue 'sending' variable just like the RX. So we can do this in net core layer.
Thanks,
Li Qiang
Quoted 47 lines
Signed-off-by Jason Wang <[email protected]>--- hw/net/e1000e_core.c | 8 ++++++++ hw/net/e1000e_core.h | 1 + 2 files changed, 9 insertions(+)hw/net/e1000e_core.c +8
-0@@ -923,6 +923,12 @@ e1000e_start_xmit(E1000ECore *core, const E1000E_TxRing *txr)return ; } +if (core->sending) { +return ; + } + + core->sending = true; +while (!e1000e_ring_empty (core, txi)) { base =e1000e_ring_head_descr (core, txi);@@ -940,6 +946,8 @@ e1000e_start_xmit(E1000ECore *core, const E1000E_TxRing *txr)if (!ide || !e1000e_intrmgr_delay_tx_causes (core, &cause)) {e1000e_set_interrupt_cause (core, cause); } + + core->sending = false; } static boolhw/net/e1000e_core.h +1
-1@@ -114,6 +114,7 @@ struct E1000Core {void (*owner_start_recv)(PCIDevice *d);uint32_t msi_causes_pending; +bool sending; };void --2.20.1
Re: [PATCH 2/2] e1000e: make TX reentrant
Quoted 8 lines · Li Qiang
Quoted 5 lines · Jason Wang <[email protected]> 于2020年7月22日周三 下午4:58写道:
In loopback mode, e1000e RX can DMA into TX doorbell which requires TX to be reentrant. This patch make e1000e's TX routine reentrant by introducing a per device boolean for recording whether or not a TX rountine is being called and return early.
Could we introduce a per-queue 'sending' variable just like the RX. So we can do this in net core layer.
It's kind of not easy since TX routine is called before the packet can reach network queue.
Thanks
Quoted 51 lines
Thanks,
Li QiangQuoted 47 lines
Signed-off-by Jason Wang <[email protected]>--- hw/net/e1000e_core.c | 8 ++++++++ hw/net/e1000e_core.h | 1 + 2 files changed, 9 insertions(+)hw/net/e1000e_core.c +8
-0@@ -923,6 +923,12 @@ e1000e_start_xmit(E1000ECore *core, const E1000E_TxRing *txr)return ; } +if (core->sending) { +return ; + } + + core->sending = true; +while (!e1000e_ring_empty (core, txi)) { base =e1000e_ring_head_descr (core, txi);@@ -940,6 +946,8 @@ e1000e_start_xmit(E1000ECore *core, const E1000E_TxRing *txr)if (!ide || !e1000e_intrmgr_delay_tx_causes (core, &cause)) {e1000e_set_interrupt_cause (core, cause); } + + core->sending = false; } static boolhw/net/e1000e_core.h +1
-1@@ -114,6 +114,7 @@ struct E1000Core {void (*owner_start_recv)(PCIDevice *d);uint32_t msi_causes_pending; +bool sending; };void --2.20.1
Re: [PATCH 2/2] e1000e: make TX reentrant
In reply to Jason WangFWIW, this is not "making TX reentrant", it is about forbidding reentrancy instead :)
/mjt
Re: [PATCH 2/2] e1000e: make TX reentrant
Quoted 4 lines · Michael Tokarev
FWIW, this is not "making TX reentrant", it is about forbidding reentrancy instead :)
/mjt
Indeed, I will rename the title.
Thanks
Quoted 1 line
Re: [PATCH 2/2] e1000e: make TX reentrant
Quoted 9 lines · Jason Wang
Quoted 4 lines · Michael Tokarev
FWIW, this is not "making TX reentrant", it is about forbidding reentrancy instead :)
/mjt
Indeed, I will rename the title.
Please also include a comment explaining the purpose of the early return in the code.
Stefan
Re: [PATCH 2/2] e1000e: make TX reentrant
In reply to Jason WangQuoted 8 lines · Jason Wang
In loopback mode, e1000e RX can DMA into TX doorbell which requires TX to be reentrant. This patch make e1000e's TX routine reentrant by introducing a per device boolean for recording whether or not a TX rountine is being called and return early.
Signed-off-by Jason Wang <[email protected]>---
This feels like a sticking-plaster fix that's not really in the right place... It stops us from calling back into e1000e_start_xmit(), but it doesn't prevent a DMA request from touching other device registers that update state in the E100ECore struct that the transmit code is not expecting to change.
thanks
-- PMM
Re: [PATCH 2/2] e1000e: make TX reentrant
Quoted 6 lines · Peter Maydell
Quoted 5 lines · Jason Wang
In loopback mode, e1000e RX can DMA into TX doorbell which requires TX to be reentrant. This patch make e1000e's TX routine reentrant by introducing a per device boolean for recording whether or not a TX rountine is being called and return early.
Quoted 8 lines
Quoted 2 lines
Signed-off-by Jason Wang <[email protected]>---
This feels like a sticking-plaster fix that's not really in the right place... It stops us from calling back into
e1000e_start_xmit(), but it doesn't prevent a DMA request from touching other device registers that update state in the E100ECore struct that the transmit code is not expecting to change.
Right, so we can track the mr owner and fail the memory access if there's another mr transaction in memory core: memory_region_dispatch_read() and memory_region_dispatch_write().
But what's more interesting is that some device uses bh for the doorbell which may require more thought...
Thanks
Quoted 4 lines
thanks
-- PMM
Quoted 18 lines · Jason Wang
The memory API allows DMA into NIC's MMIO area. This means the NIC's RX routine must be reentrant. Instead of auditing all the NIC, we can simply detect the reentrancy and return early. The
queue->deliveringis set and cleared byqemu_net_queue_deliver()for other queue helpers to know whether the delivering in on going (NIC's receive is being called). We can check it and return early inqemu_net_queue_flush()to forbid reentrant RX.Signed-off-by Jason Wang <[email protected]>--- net/queue.c | 3 +++ 1 file changed, 3 insertions(+)net/queue.c +0
-0
net/queue.c void qemu_net_queue_purge(NetQueue *queue, NetClientState *from) · line 258
Quoted 9 lines
net/queue.c +3
-0@@ -250,6 +250,9 @@ void qemu_net_queue_purge(NetQueue *queue, NetClientState *from)bool qemu_net_queue_flush (NetQueue *queue) { +if (queue->delivering) +return false; +while (!QTAILQ_EMPTY (&queue->packets)) {NetPacket *packet;int ret;
Queued for rc2.
Thanks
Quoted 32 lines · Jason Wang
Quoted 18 lines · Jason Wang
The memory API allows DMA into NIC's MMIO area. This means the NIC's RX routine must be reentrant. Instead of auditing all the NIC, we can simply detect the reentrancy and return early. The
queue->deliveringis set and cleared byqemu_net_queue_deliver()for other queue helpers to know whether the delivering in on going (NIC's receive is being called). We can check it and return early inqemu_net_queue_flush()to forbid reentrant RX.Signed-off-by Jason Wang <[email protected]>--- net/queue.c | 3 +++ 1 file changed, 3 insertions(+)net/queue.c +0
-0net/queue.c void qemu_net_queue_purge(NetQueue *queue, NetClientState *from) · line 257
Quoted 8 lines
net/queue.c +3
-0@@ -250,6 +250,9 @@ void qemu_net_queue_purge(NetQueue *queue, NetClientState *from)bool qemu_net_queue_flush (NetQueue *queue) { +if (queue->delivering) +return false; +while (!QTAILQ_EMPTY (&queue->packets)) {NetPacket *packet;int ret;Queued for rc2.
Quoted 2 lines
Thanks
Hi Jason,
I don't think this ever made it in. Are there any remaining problems? Thanks
-Alex
Quoted 31 lines · Alexander Bulekov
Quoted 30 lines · Jason Wang
Quoted 18 lines · Jason Wang
The memory API allows DMA into NIC's MMIO area. This means the NIC's RX routine must be reentrant. Instead of auditing all the NIC, we can simply detect the reentrancy and return early. The
queue->deliveringis set and cleared byqemu_net_queue_deliver()for other queue helpers to know whether the delivering in on going (NIC's receive is being called). We can check it and return early inqemu_net_queue_flush()to forbid reentrant RX.Signed-off-by Jason Wang <[email protected]>--- net/queue.c | 3 +++ 1 file changed, 3 insertions(+)net/queue.c +0
-0net/queue.c void qemu_net_queue_purge(NetQueue *queue, NetClientState *from) · line 257
Quoted 8 lines
net/queue.c +3
-0@@ -250,6 +250,9 @@ void qemu_net_queue_purge(NetQueue *queue, NetClientState *from)bool qemu_net_queue_flush (NetQueue *queue) { +if (queue->delivering) +return false; +while (!QTAILQ_EMPTY (&queue->packets)) {NetPacket *packet;int ret;Queued for rc2.
Quoted 6 lines
Quoted 2 lines
Thanks
Hi Jason,
I don't think this ever made it in. Are there any remaining problems? Thanks
-Alex
Hi Alex:
It should have been merged:
https://git.qemu.org/?p=qemu.git;a=commit;h=22dc8663d9fc7baa22100544c600b6285a63c7a3
Thanks
Quoted 1 line
Quoted 41 lines · Jason Wang
Quoted 31 lines · Alexander Bulekov
Quoted 30 lines · Jason Wang
Quoted 18 lines · Jason Wang
The memory API allows DMA into NIC's MMIO area. This means the NIC's RX routine must be reentrant. Instead of auditing all the NIC, we can simply detect the reentrancy and return early. The
queue->deliveringis set and cleared byqemu_net_queue_deliver()for other queue helpers to know whether the delivering in on going (NIC's receive is being called). We can check it and return early inqemu_net_queue_flush()to forbid reentrant RX.Signed-off-by Jason Wang <[email protected]>--- net/queue.c | 3 +++ 1 file changed, 3 insertions(+)net/queue.c +0
-0net/queue.c void qemu_net_queue_purge(NetQueue *queue, NetClientState *from) · line 257
Quoted 8 lines
net/queue.c +3
-0@@ -250,6 +250,9 @@ void qemu_net_queue_purge(NetQueue *queue, NetClientState *from)bool qemu_net_queue_flush (NetQueue *queue) { +if (queue->delivering) +return false; +while (!QTAILQ_EMPTY (&queue->packets)) {NetPacket *packet;int ret;Queued for rc2.
Quoted 6 lines
Quoted 2 lines
Thanks
Hi Jason,
I don't think this ever made it in. Are there any remaining problems? Thanks
-Alex
Quoted 8 lines
Hi Alex:
It should have been merged:
https://git.qemu.org/?p=qemu.git;a=commit;h=22dc8663d9fc7baa22100544c600b6285a63c7a3
Thanks
Ah. I missed only 1/2 was queued. I guess the e1000 patch didn't make the cut..
Thanks
-Alex
Quoted 3 lines
Quoted 2 lines