Skip to contentSkip to conversation

bnx2x: minimize RAM usage in kdump

PATCH In review Michal Schmidt 3 participants 6 messages Feb 25, 2014 – Feb 26, 2014

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

cover bnx2x: minimize RAM usage in kdump

kdump kernels usually have only a small amount of memory reserved. bnx2x can be memory-hungry. Let's minimize its memory usage when running in kdump.

I detect kdump by looking at the "reset_devices" flag. A couple of storage drivers (cciss, hpsa) use it for the same purpose. I am not sure this is the best way to solve the problem, but it works.

Should it be made more generic by, say, looking at the total amount of lowmem instead? Not using TPA by default when lowmem is small and/or defaulting to fewer queues would help 32bit systems where a driver for a multi-function multi-queue NIC can consume a significant amount of available memory. Or do we want no such heuristics?

Is this something to consider doing for other network drivers too?

Michal Schmidt (3):
bnx2x: clamp num_queues to prevent passing a negative value
bnx2x: save RAM in kdump kernel by using a single queue
bnx2x: save RAM in kdump kernel by disabling TPA

 drivers/net/ethernet/broadcom/bnx2x/bnx2x_cmn.c  | 12 ++++++++----
 drivers/net/ethernet/broadcom/bnx2x/bnx2x_main.c |  2 ++
 2 files changed, 10 insertions(+), 4 deletions(-)
Signature · 1 line
1.8.5.3
#1 lore raw

PATCH 1/3 bnx2x: clamp num_queues to prevent passing a negative value

Use the clamp() macro to make the calculation of the number of queues slightly easier to understand. It also avoids a crash when someone accidentally passes a negative value in num_queues= module parameter.

---
 drivers/net/ethernet/broadcom/bnx2x/bnx2x_cmn.c | 7 +++----
 1 file changed, 3 insertions(+), 4 deletions(-)
drivers/net/ethernet/broadcom/bnx2x/bnx2x_cmn.c +3 -4
index 5ee13af..64c17b6 100644
@@ -61,10 +61,9 @@ static void bnx2x_add_all_napi(struct bnx2x *bp)
 
 static int bnx2x_calc_num_queues(struct bnx2x *bp)
 {
-	return  bnx2x_num_queues ?
-		 min_t(int, bnx2x_num_queues, BNX2X_MAX_QUEUES(bp)) :
-		 min_t(int, netif_get_num_default_rss_queues(),
-		       BNX2X_MAX_QUEUES(bp));
+	int nq = bnx2x_num_queues ? : netif_get_num_default_rss_queues();
+	nq = clamp(nq, 1, BNX2X_MAX_QUEUES(bp));
+	return nq;
 }
 
 /**
Signature · 1 line
1.8.5.3
#2 lore raw

PATCH 2/3 bnx2x: save RAM in kdump kernel by using a single queue

In reply to Michal Schmidt

When running in a kdump kernel, make sure to use only a single ethernet queue even if a num_queues option in /etc/modprobe.d/*.conf would specify otherwise. This saves memory, which tends to be scarce in kdump.

This saves about 40 MB in the kdump environment on a setup with num_queues=8 in the config file.

---
 drivers/net/ethernet/broadcom/bnx2x/bnx2x_cmn.c | 5 +++++
 1 file changed, 5 insertions(+)
drivers/net/ethernet/broadcom/bnx2x/bnx2x_cmn.c +5 -0
index 64c17b6..500d38a 100644
@@ -62,6 +62,11 @@ static void bnx2x_add_all_napi(struct bnx2x *bp)
 static int bnx2x_calc_num_queues(struct bnx2x *bp)
 {
 	int nq = bnx2x_num_queues ? : netif_get_num_default_rss_queues();
+
+	/* Reduce memory usage in kdump environment by using only one queue */
+	if (reset_devices)
+		nq = 1;
+
 	nq = clamp(nq, 1, BNX2X_MAX_QUEUES(bp));
 	return nq;
 }
Signature · 1 line
1.8.5.3
#3 lore raw

PATCH 3/3 bnx2x: save RAM in kdump kernel by disabling TPA

In reply to Michal Schmidt

When running in a kdump kernel, disable TPA. This saves memory, which tends to be scarce in kdump.

TPA, being a receive acceleration, is unlikely to be useful for kdump, whose purpose is to send the memory image out.

This saves additional 5 MB in the kdump environment.

---
 drivers/net/ethernet/broadcom/bnx2x/bnx2x_main.c | 2 ++
 1 file changed, 2 insertions(+)
drivers/net/ethernet/broadcom/bnx2x/bnx2x_main.c +2 -0
index 8443915..13de536 100644
@@ -11783,6 +11783,8 @@ static int bnx2x_init_bp(struct bnx2x *bp)
 
 	bp->disable_tpa = disable_tpa;
 	bp->disable_tpa |= IS_MF_STORAGE_SD(bp) || IS_MF_FCOE_AFEX(bp);
+	/* Reduce memory usage in kdump environment by disabling TPA */
+	bp->disable_tpa |= reset_devices;
 
 	/* Set TPA flags */
 	if (bp->disable_tpa) {
Signature · 1 line
1.8.5.3
Acked-by on the v1 series
#4 lore raw
In reply to Michal Schmidt
Quoted 28 lines

-----Original Message-----
From: Michal Schmidt [mailto:[email protected]]
Sent: Tuesday, February 25, 2014 5:04 PM
To: [email protected]
Cc: [email protected]; Ariel Elior; Dmitry Kravkov; Yuval Mintz
Subject: [PATCH net-next 0/3] bnx2x: minimize RAM usage in kdump

kdump kernels usually have only a small amount of memory reserved. bnx2x can be memory-hungry. Let's minimize its memory usage when running in kdump.

I detect kdump by looking at the "reset_devices" flag. A couple of storage drivers (cciss, hpsa) use it for the same purpose. I am not sure this is the best way to solve the problem, but it works.

Should it be made more generic by, say, looking at the total amount of lowmem instead? Not using TPA by default when lowmem is small and/or defaulting to fewer queues would help 32bit systems where a driver for a multi-function multi-queue NIC can consume a significant amount of available memory. Or do we want no such heuristics?

Is this something to consider doing for other network drivers too?

Michal Schmidt (3):
bnx2x: clamp num_queues to prevent passing a negative value
bnx2x: save RAM in kdump kernel by using a single queue
bnx2x: save RAM in kdump kernel by disabling TPA

Quoted 6 lines
 drivers/net/ethernet/broadcom/bnx2x/bnx2x_cmn.c  | 12 ++++++++----
 drivers/net/ethernet/broadcom/bnx2x/bnx2x_main.c |  2 ++
 2 files changed, 10 insertions(+), 4 deletions(-)

--
1.8.5.3

This series looks good, in the sense that it modifies the driver in a correct way to achieve the intended result (less memory on kdump). The basic question of testing lowmem vs. reset_devices or perhaps some other parameter still needs to be answered though.

Thanks,
Ariel

maintainer
#5 lore raw

From: Ariel Elior <[email protected]>
Date: Tue, 25 Feb 2014 16:41:07 +0000

Quoted 32 lines
Quoted 32 lines

-----Original Message-----
From: Michal Schmidt [mailto:[email protected]]
Sent: Tuesday, February 25, 2014 5:04 PM
To: [email protected]
Cc: [email protected]; Ariel Elior; Dmitry Kravkov; Yuval Mintz
Subject: [PATCH net-next 0/3] bnx2x: minimize RAM usage in kdump

kdump kernels usually have only a small amount of memory reserved. bnx2x can be memory-hungry. Let's minimize its memory usage when running in kdump.

I detect kdump by looking at the "reset_devices" flag. A couple of storage drivers (cciss, hpsa) use it for the same purpose. I am not sure this is the best way to solve the problem, but it works.

Should it be made more generic by, say, looking at the total amount of lowmem instead? Not using TPA by default when lowmem is small and/or defaulting to fewer queues would help 32bit systems where a driver for a multi-function multi-queue NIC can consume a significant amount of available memory. Or do we want no such heuristics?

Is this something to consider doing for other network drivers too?

Michal Schmidt (3):
bnx2x: clamp num_queues to prevent passing a negative value
bnx2x: save RAM in kdump kernel by using a single queue
bnx2x: save RAM in kdump kernel by disabling TPA

 drivers/net/ethernet/broadcom/bnx2x/bnx2x_cmn.c  | 12 ++++++++----
 drivers/net/ethernet/broadcom/bnx2x/bnx2x_main.c |  2 ++
 2 files changed, 10 insertions(+), 4 deletions(-)
Quoted 6 lines
Quoted 2 lines

--
1.8.5.3

This series looks good, in the sense that it modifies the driver in a correct way to achieve the intended result (less memory on kdump). The basic question of testing lowmem vs. reset_devices or perhaps some other parameter still needs to be answered though.

Series applied, thanks.