Skip to contentSkip to conversation

Fix incorrect page size and address caclulations on systems with 64k (or other) page sizes

PATCH In review [email protected] 2 participants 6 messages Apr 18, 2025 – Apr 21, 2025 Network drivers

lore
Share & tools
lore
#0 lore raw

cover Fix incorrect page size and address caclulations on systems with 64k (or other) page sizes

From: Long Li <[email protected]>

The Hyper-V uses 4k page size regardless of system page size.

This patchset fixes issues while running on systems with page sizes other than 4k.

Long Li (4):
bus/vmbus: Align ring buffer data region to system page boundary bus/vmbus: Define Hyper-V page size
bus/vmbus: Use Hyper-V page size for mapping to UIO pages net/netvsc: Use Hyper-V page size for the driver

 drivers/bus/vmbus/bus_vmbus_driver.h |  4 ++++
 drivers/bus/vmbus/linux/vmbus_uio.c  |  2 +-
 drivers/bus/vmbus/rte_vmbus_reg.h    |  9 +++------
 drivers/bus/vmbus/vmbus_bufring.c    |  9 ++++++---
 drivers/bus/vmbus/vmbus_common_uio.c |  2 +-
 drivers/net/netvsc/hn_rndis.c        | 14 +++++++-------
 drivers/net/netvsc/hn_rxtx.c         | 16 ++++++++--------
 drivers/net/netvsc/hn_var.h          |  4 ----
 8 files changed, 30 insertions(+), 30 deletions(-)
Signature · 1 line
2.34.1
#1 lore raw

PATCH 1/4 bus/vmbus: Align ring buffer data region to system page boundary

From: Long Li <[email protected]>

The ring buffer data region always starts at the system page boundary after ring buffer head. The current code assumes the system page size is 4k. This is not always correct.

Fix this by using system page size for addressing ring buffer data.

Fixes 831dba47bd ("bus/vmbus: add Hyper-V virtual bus support")
---
 drivers/bus/vmbus/rte_vmbus_reg.h | 9 +++------
 drivers/bus/vmbus/vmbus_bufring.c | 9 ++++++---
 2 files changed, 9 insertions(+), 9 deletions(-)
drivers/bus/vmbus/rte_vmbus_reg.h +3 -6
index 54a26d12bd..fb7e3043ec 100644
@@ -100,14 +100,11 @@ struct __rte_packed_begin vmbus_bufring {
 		uint32_t value;
 	} feature_bits;
 
-	/* Pad it to rte_mem_page_size() so that data starts on page boundary */
-	uint8_t	reserved2[4028];
-
 	/*
-	 * Ring data starts here + RingDataStartOffset
-	 * !!! DO NOT place any fields below this !!!
+	 * This is the end of ring buffer head. The ring buffer data is system
+	 * page aligned and starts at rte_mem_page_size() from the beginning
+	 * of this structure
 	 */
-	uint8_t data[];
 } __rte_packed_end;
 
 /*
drivers/bus/vmbus/vmbus_bufring.c +6 -3
index c78619dc44..fcb97287dc 100644
@@ -36,7 +36,10 @@ void vmbus_br_setup(struct vmbus_br *br, void *buf, unsigned int blen)
 {
 	br->vbr = buf;
 	br->windex = br->vbr->windex;
-	br->dsize = blen - sizeof(struct vmbus_bufring);
+
+	/* The ring buffer data starts at the 2nd page of the ring buffer */
+	RTE_VERIFY(blen > rte_mem_page_size());
+	br->dsize = blen - rte_mem_page_size();
 }
 
 /*
@@ -72,7 +75,7 @@ static inline uint32_t
 vmbus_txbr_copyto(const struct vmbus_br *tbr, uint32_t windex,
 		  const void *src0, uint32_t cplen)
 {
-	uint8_t *br_data = tbr->vbr->data;
+	uint8_t *br_data = (uint8_t *)tbr->vbr + rte_mem_page_size();
 	uint32_t br_dsize = tbr->dsize;
 	const uint8_t *src = src0;
 
@@ -170,7 +173,7 @@ static inline uint32_t
 vmbus_rxbr_copyfrom(const struct vmbus_br *rbr, uint32_t rindex,
 		    void *dst0, size_t cplen)
 {
-	const uint8_t *br_data = rbr->vbr->data;
+	const uint8_t *br_data = (uint8_t *)rbr->vbr + rte_mem_page_size();
 	uint32_t br_dsize = rbr->dsize;
 	uint8_t *dst = dst0;
 
Signature · 1 line
2.34.1
#2 lore raw

PATCH 2/4 bus/vmbus: Define Hyper-V page size

In reply to [email protected]

From: Long Li <[email protected]>

Hyper-V uses 4k page size, regardless of the system page size used. Define Hyper-V page size for use in drivers.

Fixes 831dba47bd ("bus/vmbus: add Hyper-V virtual bus support")
---
 drivers/bus/vmbus/bus_vmbus_driver.h | 4 ++++
 1 file changed, 4 insertions(+)
drivers/bus/vmbus/bus_vmbus_driver.h +4 -0
index bc394208de..0a56275437 100644
@@ -14,6 +14,10 @@
 extern "C" {
 #endif
 
+#define HYPERV_PAGE_SHIFT 12
+#define HYPERV_PAGE_SIZE (1 << HYPERV_PAGE_SHIFT)
+#define HYPERV_PAGE_MASK (HYPERV_PAGE_SIZE - 1)
+
 struct vmbus_channel;
 struct vmbus_mon_page;
 
Signature · 1 line
2.34.1
#3 lore raw

PATCH 3/4 bus/vmbus: Use Hyper-V page size for mapping to UIO pages

In reply to [email protected]

From: Long Li <[email protected]>

The interrupt and monitor pages mapped from Hyper-V via kernel are always 4k in sizes. Use Hyper-V page size to map them.

Fixes 831dba47bd ("bus/vmbus: add Hyper-V virtual bus support")
---
 drivers/bus/vmbus/linux/vmbus_uio.c  | 2 +-
 drivers/bus/vmbus/vmbus_common_uio.c | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)
drivers/bus/vmbus/linux/vmbus_uio.c +1 -1
index 8edec869ac..fbafc5027d 100644
@@ -268,7 +268,7 @@ static int vmbus_uio_map_subchan(const struct rte_vmbus_device *dev,
 	}
 	file_size = sb.st_size;
 
-	if (file_size == 0 || (file_size & (rte_mem_page_size() - 1))) {
+	if (file_size == 0 || (file_size & (HYPERV_PAGE_SIZE - 1))) {
 		VMBUS_LOG(ERR, "incorrect size %s: %zu",
 			  ring_path, file_size);
 
drivers/bus/vmbus/vmbus_common_uio.c +1 -1
index d55aee6537..c6e7e07302 100644
@@ -204,7 +204,7 @@ vmbus_uio_map_resource(struct rte_vmbus_device *dev)
 	}
 
 	dev->int_page = (uint32_t *)((char *)uio_res->maps[HV_INT_PAGE_MAP].addr
-				     + (rte_mem_page_size() >> 1));
+				     + (HYPERV_PAGE_SIZE >> 1));
 	dev->monitor_page = uio_res->maps[HV_MON_PAGE_MAP].addr;
 	return 0;
 }
Signature · 1 line
2.34.1
#4 lore raw

PATCH 4/4 net/netvsc: Use Hyper-V page size for the driver

In reply to [email protected]

From: Long Li <[email protected]>

The driver should always use Hyper-V page size for implementing RNDIS and calculating PFN (Page Frame Number) for communicating with Hyper-V VSP.

It should not use system page size as it may be different to the Hyper-V page size.

Fixes 4e9c73e96e ("net/netvsc: add Hyper-V network device")
---
 drivers/net/netvsc/hn_rndis.c | 14 +++++++-------
 drivers/net/netvsc/hn_rxtx.c  | 16 ++++++++--------
 drivers/net/netvsc/hn_var.h   |  4 ----
 3 files changed, 15 insertions(+), 19 deletions(-)
drivers/net/netvsc/hn_rndis.c +7 -7
index 12bea33e37..7c54eebcef 100644
@@ -67,7 +67,7 @@ hn_rndis_rid(struct hn_data *hv)
 
 static void *hn_rndis_alloc(size_t size)
 {
-	return rte_zmalloc("RNDIS", size, rte_mem_page_size());
+	return rte_zmalloc("RNDIS", size, HYPERV_PAGE_SIZE);
 }
 
 #ifdef RTE_LIBRTE_NETVSC_DEBUG_DUMP
@@ -266,17 +266,17 @@ static int hn_nvs_send_rndis_ctrl(struct hn_data *hv,
 		return -EINVAL;
 	}
 
-	if (unlikely(reqlen > rte_mem_page_size())) {
+	if (unlikely(reqlen > HYPERV_PAGE_SIZE)) {
 		PMD_DRV_LOG(ERR, "RNDIS request %u greater than page size",
 			    reqlen);
 		return -EINVAL;
 	}
 
-	sg.page = addr / rte_mem_page_size();
-	sg.ofs  = addr & PAGE_MASK;
+	sg.page = addr / HYPERV_PAGE_SIZE;
+	sg.ofs  = addr & HYPERV_PAGE_MASK;
 	sg.len  = reqlen;
 
-	if (sg.ofs + reqlen >  rte_mem_page_size()) {
+	if (sg.ofs + reqlen >  HYPERV_PAGE_SIZE) {
 		PMD_DRV_LOG(ERR, "RNDIS request crosses page boundary");
 		return -EINVAL;
 	}
@@ -481,7 +481,7 @@ hn_rndis_query(struct hn_data *hv, uint32_t oid,
 		return -ENOMEM;
 
 	comp_len = sizeof(*comp) + odlen;
-	comp = rte_zmalloc("QUERY", comp_len, rte_mem_page_size());
+	comp = rte_zmalloc("QUERY", comp_len, HYPERV_PAGE_SIZE);
 	if (!comp) {
 		error = -ENOMEM;
 		goto done;
@@ -738,7 +738,7 @@ hn_rndis_set(struct hn_data *hv, uint32_t oid, const void *data, uint32_t dlen)
 	int error;
 
 	reqlen = sizeof(*req) + dlen;
-	req = rte_zmalloc("RNDIS_SET", reqlen, rte_mem_page_size());
+	req = rte_zmalloc("RNDIS_SET", reqlen, HYPERV_PAGE_SIZE);
 	if (!req)
 		return -ENOMEM;
 
drivers/net/netvsc/hn_rxtx.c +8 -8
index ffa6937ce2..44faf22da7 100644
@@ -1432,10 +1432,10 @@ static unsigned int hn_get_slots(const struct rte_mbuf *m)
 
 	while (m) {
 		unsigned int size = rte_pktmbuf_data_len(m);
-		unsigned int offs = rte_mbuf_data_iova(m) & PAGE_MASK;
+		unsigned int offs = rte_mbuf_data_iova(m) & HYPERV_PAGE_MASK;
 
-		slots += (offs + size + rte_mem_page_size() - 1) /
-				rte_mem_page_size();
+		slots += (offs + size + HYPERV_PAGE_SIZE - 1) /
+				HYPERV_PAGE_SIZE;
 		m = m->next;
 	}
 
@@ -1450,13 +1450,13 @@ static unsigned int hn_fill_sg(struct vmbus_gpa *sg,
 
 	while (m) {
 		rte_iova_t addr = rte_mbuf_data_iova(m);
-		unsigned int page = addr / rte_mem_page_size();
-		unsigned int offset = addr & PAGE_MASK;
+		unsigned int page = addr / HYPERV_PAGE_SIZE;
+		unsigned int offset = addr & HYPERV_PAGE_MASK;
 		unsigned int len = rte_pktmbuf_data_len(m);
 
 		while (len > 0) {
 			unsigned int bytes = RTE_MIN(len,
-					rte_mem_page_size() - offset);
+					HYPERV_PAGE_SIZE - offset);
 
 			sg[segs].page = page;
 			sg[segs].ofs = offset;
@@ -1499,8 +1499,8 @@ static int hn_xmit_sg(struct hn_tx_queue *txq,
 	addr = txq->tx_rndis_iova +
 		((char *)txd->rndis_pkt - (char *)txq->tx_rndis);
 
-	sg[0].page = addr / rte_mem_page_size();
-	sg[0].ofs = addr & PAGE_MASK;
+	sg[0].page = addr / HYPERV_PAGE_SIZE;
+	sg[0].ofs = addr & HYPERV_PAGE_MASK;
 	sg[0].len = RNDIS_PACKET_MSG_OFFSET_ABS(hn_rndis_pktlen(txd->rndis_pkt));
 	segs = 1;
 
drivers/net/netvsc/hn_var.h +0 -4
index 0f638bc5fd..f946b3d8ef 100644
@@ -31,10 +31,6 @@
 
 #define HN_RX_EXTMBUF_ENABLE	0
 
-#ifndef PAGE_MASK
-#define PAGE_MASK (rte_mem_page_size() - 1)
-#endif
-
 struct hn_data;
 struct hn_txdesc;
 
Signature · 1 line
2.34.1
#5 lore raw
In reply to [email protected]
Quoted 31 lines

From: [email protected] <[email protected]>
Sent: Saturday, April 19, 2025 3:33 AM
To: Stephen Hemminger <[email protected]>; Wei Hu <[email protected]>
Cc: [email protected]; Long Li <[email protected]>
Subject: [EXTERNAL] [PATCH 0/4] Fix incorrect page size and address caclulations on systems with 64k (or other) page sizes

From: Long Li <[email protected]>

The Hyper-V uses 4k page size regardless of system page size.

This patchset fixes issues while running on systems with page sizes other than 4k.

Long Li (4):
bus/vmbus: Align ring buffer data region to system page boundary bus/vmbus: Define Hyper-V page size
bus/vmbus: Use Hyper-V page size for mapping to UIO pages net/netvsc: Use Hyper-V page size for the driver

 drivers/bus/vmbus/bus_vmbus_driver.h |  4 ++++
drivers/bus/vmbus/linux/vmbus_uio.c  |  2 +-
 drivers/bus/vmbus/rte_vmbus_reg.h    |  9 +++------
 drivers/bus/vmbus/vmbus_bufring.c    |  9 ++++++---
 drivers/bus/vmbus/vmbus_common_uio.c |  2 +-
 drivers/net/netvsc/hn_rndis.c        | 14 +++++++-------
 drivers/net/netvsc/hn_rxtx.c         | 16 ++++++++--------
 drivers/net/netvsc/hn_var.h          |  4 ----
 8 files changed, 30 insertions(+), 30 deletions(-)
Quoted 2 lines

--
2.34.1

Reviewed by: [email protected]