limit xen vnic max queues number to online cpu number

PATCH In review Joe Jin 3 participants 17 messages Oct 23, 2015

Subsystems (4)

Networking Network drivers Xen hypervisor interface Xen network backend driver

lore

Share & tools

lore

Joe Jin

author

2015-10-23 07:54 UTC #0 lore raw

cover limit xen vnic max queues number to online cpu number

Currently xen vnic allowed to create lots of queues by set module param max_queues(both netback and netfront), when queues number larger than cpu number, it does not help for performance but need more cpu time.

This patchset limit netback and netfront max queues number to online cpus number.

Joe Jin (2):
xen-netback: limit xen vif max queues number to online cpus
xen-front: limit vnic max_queues number to online cpus

 drivers/net/xen-netback/netback.c |   28 ++++++++++++++++++++++------
 drivers/net/xen-netfront.c        |   27 +++++++++++++++++++++------
 2 files changed, 43 insertions(+), 12 deletions(-)

Joe Jin

author

2015-10-23 07:58 UTC #1 lore raw

PATCH 1/2 xen-netback: limit xen vif max queues number to online cpus

Should not allocate xen vif queues number more than online cpus.

Signed-off-by Joe Jin <[email protected]>

Cc Wei Liu <[email protected]>

Cc Ian Campbell <[email protected]>

Cc Boris Ostrovsky <[email protected]>

Cc Konrad Rzeszutek Wilk <[email protected]>

---
 drivers/net/xen-netback/netback.c |   28 ++++++++++++++++++++++------
 1 files changed, 22 insertions(+), 6 deletions(-)

drivers/net/xen-netback/netback.c +22 -6

index ec98d43..387fbd1 100644
@@ -68,7 +68,9 @@ unsigned int rx_stall_timeout_msecs = 60000;
 module_param(rx_stall_timeout_msecs, uint, 0444);
 
 unsigned int xenvif_max_queues;
-module_param_named(max_queues, xenvif_max_queues, uint, 0644);
+static int xennet_set_max_queues(const char *val, struct kernel_param *kp);
+module_param_call(max_queues, xennet_set_max_queues, param_get_int,
+		  &xenvif_max_queues, 0600);
 MODULE_PARM_DESC(max_queues,
 		 "Maximum number of queues per virtual interface");

@@ -107,6 +109,20 @@ static struct xen_netif_rx_response *make_rx_response(struct xenvif_queue *queue
 					     u16      size,
 					     u16      flags);
 
+static int xennet_set_max_queues(const char *val, struct kernel_param *kp)
+{
+	unsigned int cpus = num_online_cpus();
+	unsigned int max_queues = simple_strtoul(val, NULL, 10);
+
+	if (max_queues == 0 || max_queues > cpus) {
+		pr_info("max_queues %d is out of range [0 - %d]!\n", 
+			max_queues, cpus);
+		return -EINVAL;
+	}
+
+	return param_set_int(val, kp);
+}
+
 static inline unsigned long idx_to_pfn(struct xenvif_queue *queue,
 				       u16 idx)
 {

@@ -2110,15 +2126,15 @@ int xenvif_dealloc_kthread(void *data)
 static int __init netback_init(void)
 {
 	int rc = 0;
+	unsigned int cpus = num_online_cpus();
 
 	if (!xen_domain())
 		return -ENODEV;
 
-	/* Allow as many queues as there are CPUs if user has not
-	 * specified a value.
-	 */
-	if (xenvif_max_queues == 0)
-		xenvif_max_queues = num_online_cpus();
+	/* Allow at most as many queues as CPUs. */
+	if (xenvif_max_queues == 0 || xenvif_max_queues > cpus)
+		xenvif_max_queues = cpus;
+	pr_info("vif max_queues: %d\n", xenvif_max_queues);
 
 	if (fatal_skb_slots < XEN_NETBK_LEGACY_SLOTS_MAX) {
 		pr_info("fatal_skb_slots too small (%d), bump it to XEN_NETBK_LEGACY_SLOTS_MAX (%d)\n",

Signature · 1 line

1.7.1

Jan Beulich

2015-10-23 08:20 UTC #2 lore raw

Re: [Xen-devel] [PATCH 1/2] xen-netback: limit xen vif max queues number to online cpus

drivers/net/xen-netback/netback.c unsigned int rx_stall_timeout_msecs = 60000; · line 72

Quoted 10 lines

Quoted 1 line

Quoted 1 line · On 23.10.15 at 09:58, <[email protected]> wrote:

drivers/net/xen-netback/netback.c +2 -1

@@ -68,7 +68,9 @@ unsigned int rx_stall_timeout_msecs = 60000;
 module_param(rx_stall_timeout_msecs, uint, 0444);
 
 unsigned int xenvif_max_queues;
-module_param_named(max_queues, xenvif_max_queues, uint, 0644);
+static int xennet_set_max_queues(const char *val, struct kernel_param *kp);
+module_param_call(max_queues, xennet_set_max_queues, param_get_int,

param_get_uint

drivers/net/xen-netback/netback.c unsigned int rx_stall_timeout_msecs = 60000; · line 73

Quoted 1 line

drivers/net/xen-netback/netback.c +1 -0
+		  &xenvif_max_queues, 0600);

Why the change from mode 0644 to 0600?

drivers/net/xen-netback/netback.c static struct xen_netif_rx_response *make_rx_response(struct xenvif_queue *queue · line 118

Quoted 11 lines

drivers/net/xen-netback/netback.c +7 -0

@@ -107,6 +109,20 @@ static struct xen_netif_rx_response *make_rx_response(struct xenvif_queue *queue
 					     u16      size,
 					     u16      flags);
 
+static int xennet_set_max_queues(const char *val, struct kernel_param *kp)
+{
+	unsigned int cpus = num_online_cpus();
+	unsigned int max_queues = simple_strtoul(val, NULL, 10);
+
+	if (max_queues == 0 || max_queues > cpus) {
+		pr_info("max_queues %d is out of range [0 - %d]!\n",

%u in both cases.

drivers/net/xen-netback/netback.c static struct xen_netif_rx_response *make_rx_response(struct xenvif_queue *queue · line 120

Quoted 2 lines

drivers/net/xen-netback/netback.c +2 -0
+			max_queues, cpus);
+		return -EINVAL;

Considering the message: -ERANGE?

drivers/net/xen-netback/netback.c static struct xen_netif_rx_response *make_rx_response(struct xenvif_queue *queue · line 123

Quoted 3 lines

drivers/net/xen-netback/netback.c +3 -0
+	}
+
+	return param_set_int(val, kp);

param_set_uint()

Quoted 8 lines

drivers/net/xen-netback/netback.c +1 -0

@@ -2110,15 +2126,15 @@ int xenvif_dealloc_kthread(void *data)
 static int __init netback_init(void)
 {
 	int rc = 0;
+	unsigned int cpus = num_online_cpus();
 
 	if (!xen_domain())
 		return -ENODEV;

drivers/net/xen-netback/netback.c int xenvif_dealloc_kthread(void *data) · line 2137

Quoted 10 lines

drivers/net/xen-netback/netback.c +4 -5

 
-	/* Allow as many queues as there are CPUs if user has not
-	 * specified a value.
-	 */
-	if (xenvif_max_queues == 0)
-		xenvif_max_queues = num_online_cpus();
+	/* Allow at most as many queues as CPUs. */
+	if (xenvif_max_queues == 0 || xenvif_max_queues > cpus)
+		xenvif_max_queues = cpus;
+	pr_info("vif max_queues: %d\n", xenvif_max_queues);

%u again.

Jan

Joe Jin

author

2015-10-23 08:33 UTC #3 lore raw

Re: [PATCH 1/2] xen-netback: limit xen vif max queues number to online cpus

Hi Jan,

Thanks for you review, will create new patches with your comments.

Regards,
Joe

Quoted 65 lines · Jan Beulich

drivers/net/xen-netback/netback.c unsigned int rx_stall_timeout_msecs = 60000; · line 72
Quoted 10 lines
Quoted 1 line
Quoted 1 line · On 23.10.15 at 09:58, <[email protected]> wrote:
drivers/net/xen-netback/netback.c +2 -1
@@ -68,7 +68,9 @@ unsigned int rx_stall_timeout_msecs = 60000;
 module_param(rx_stall_timeout_msecs, uint, 0444);
 
 unsigned int xenvif_max_queues;
-module_param_named(max_queues, xenvif_max_queues, uint, 0644);
+static int xennet_set_max_queues(const char *val, struct kernel_param *kp);
+module_param_call(max_queues, xennet_set_max_queues, param_get_int,
param_get_uint
drivers/net/xen-netback/netback.c unsigned int rx_stall_timeout_msecs = 60000; · line 73
Quoted 1 line
drivers/net/xen-netback/netback.c +1 -0
+		  &xenvif_max_queues, 0600);
Why the change from mode 0644 to 0600?
drivers/net/xen-netback/netback.c static struct xen_netif_rx_response *make_rx_response(struct xenvif_queue *queue · line 118
Quoted 11 lines
drivers/net/xen-netback/netback.c +7 -0
@@ -107,6 +109,20 @@ static struct xen_netif_rx_response *make_rx_response(struct xenvif_queue *queue
 					     u16      size,
 					     u16      flags);
 
+static int xennet_set_max_queues(const char *val, struct kernel_param *kp)
+{
+	unsigned int cpus = num_online_cpus();
+	unsigned int max_queues = simple_strtoul(val, NULL, 10);
+
+	if (max_queues == 0 || max_queues > cpus) {
+		pr_info("max_queues %d is out of range [0 - %d]!\n", 
%u in both cases.
drivers/net/xen-netback/netback.c static struct xen_netif_rx_response *make_rx_response(struct xenvif_queue *queue · line 120
Quoted 2 lines
drivers/net/xen-netback/netback.c +2 -0
+			max_queues, cpus);
+		return -EINVAL;
Considering the message: -ERANGE?
drivers/net/xen-netback/netback.c static struct xen_netif_rx_response *make_rx_response(struct xenvif_queue *queue · line 123
Quoted 3 lines
drivers/net/xen-netback/netback.c +3 -0
+	}
+
+	return param_set_int(val, kp);
param_set_uint()
Quoted 8 lines
drivers/net/xen-netback/netback.c +1 -0
@@ -2110,15 +2126,15 @@ int xenvif_dealloc_kthread(void *data)
 static int __init netback_init(void)
 {
 	int rc = 0;
+	unsigned int cpus = num_online_cpus();
 
 	if (!xen_domain())
 		return -ENODEV;
drivers/net/xen-netback/netback.c int xenvif_dealloc_kthread(void *data) · line 2137
Quoted 10 lines
drivers/net/xen-netback/netback.c +4 -5
 
-	/* Allow as many queues as there are CPUs if user has not
-	 * specified a value.
-	 */
-	if (xenvif_max_queues == 0)
-		xenvif_max_queues = num_online_cpus();
+	/* Allow at most as many queues as CPUs. */
+	if (xenvif_max_queues == 0 || xenvif_max_queues > cpus)
+		xenvif_max_queues = cpus;
+	pr_info("vif max_queues: %d\n", xenvif_max_queues);
%u again.
Jan

Signature · 4 lines

Oracle <http://www.oracle.com>
Joe Jin | Software Development Director | +8610.6106.5624
ORACLE | Linux and Virtualization
No. 24 Zhongguancun Software Park, Haidian District | 100193 Beijing

Joe Jin

author

2015-10-23 08:33 UTC #4 lore raw

Re: [Xen-devel] [PATCH 1/2] xen-netback: limit xen vif max queues number to online cpus

In reply to Jan Beulich

Hi Jan,

Thanks for you review, will create new patches with your comments.

Regards,
Joe

Quoted 65 lines · Jan Beulich

drivers/net/xen-netback/netback.c unsigned int rx_stall_timeout_msecs = 60000; · line 72
Quoted 10 lines
Quoted 1 line
Quoted 1 line · On 23.10.15 at 09:58, <[email protected]> wrote:
drivers/net/xen-netback/netback.c +2 -1
@@ -68,7 +68,9 @@ unsigned int rx_stall_timeout_msecs = 60000;
 module_param(rx_stall_timeout_msecs, uint, 0444);
 
 unsigned int xenvif_max_queues;
-module_param_named(max_queues, xenvif_max_queues, uint, 0644);
+static int xennet_set_max_queues(const char *val, struct kernel_param *kp);
+module_param_call(max_queues, xennet_set_max_queues, param_get_int,
param_get_uint
drivers/net/xen-netback/netback.c unsigned int rx_stall_timeout_msecs = 60000; · line 73
Quoted 1 line
drivers/net/xen-netback/netback.c +1 -0
+		  &xenvif_max_queues, 0600);
Why the change from mode 0644 to 0600?
drivers/net/xen-netback/netback.c static struct xen_netif_rx_response *make_rx_response(struct xenvif_queue *queue · line 118
Quoted 11 lines
drivers/net/xen-netback/netback.c +7 -0
@@ -107,6 +109,20 @@ static struct xen_netif_rx_response *make_rx_response(struct xenvif_queue *queue
 					     u16      size,
 					     u16      flags);
 
+static int xennet_set_max_queues(const char *val, struct kernel_param *kp)
+{
+	unsigned int cpus = num_online_cpus();
+	unsigned int max_queues = simple_strtoul(val, NULL, 10);
+
+	if (max_queues == 0 || max_queues > cpus) {
+		pr_info("max_queues %d is out of range [0 - %d]!\n", 
%u in both cases.
drivers/net/xen-netback/netback.c static struct xen_netif_rx_response *make_rx_response(struct xenvif_queue *queue · line 120
Quoted 2 lines
drivers/net/xen-netback/netback.c +2 -0
+			max_queues, cpus);
+		return -EINVAL;
Considering the message: -ERANGE?
drivers/net/xen-netback/netback.c static struct xen_netif_rx_response *make_rx_response(struct xenvif_queue *queue · line 123
Quoted 3 lines
drivers/net/xen-netback/netback.c +3 -0
+	}
+
+	return param_set_int(val, kp);
param_set_uint()
Quoted 8 lines
drivers/net/xen-netback/netback.c +1 -0
@@ -2110,15 +2126,15 @@ int xenvif_dealloc_kthread(void *data)
 static int __init netback_init(void)
 {
 	int rc = 0;
+	unsigned int cpus = num_online_cpus();
 
 	if (!xen_domain())
 		return -ENODEV;
drivers/net/xen-netback/netback.c int xenvif_dealloc_kthread(void *data) · line 2137
Quoted 10 lines
drivers/net/xen-netback/netback.c +4 -5
 
-	/* Allow as many queues as there are CPUs if user has not
-	 * specified a value.
-	 */
-	if (xenvif_max_queues == 0)
-		xenvif_max_queues = num_online_cpus();
+	/* Allow at most as many queues as CPUs. */
+	if (xenvif_max_queues == 0 || xenvif_max_queues > cpus)
+		xenvif_max_queues = cpus;
+	pr_info("vif max_queues: %d\n", xenvif_max_queues);
%u again.
Jan

Signature · 4 lines

Oracle <http://www.oracle.com>
Joe Jin | Software Development Director | +8610.6106.5624
ORACLE | Linux and Virtualization
No. 24 Zhongguancun Software Park, Haidian District | 100193 Beijing

Jan Beulich

2015-10-23 08:20 UTC #5 lore raw

Re: [PATCH 1/2] xen-netback: limit xen vif max queues number to online cpus

In reply to Joe Jin

drivers/net/xen-netback/netback.c unsigned int rx_stall_timeout_msecs = 60000; · line 72

Quoted 10 lines

Quoted 1 line

Quoted 1 line · On 23.10.15 at 09:58, <[email protected]> wrote:

drivers/net/xen-netback/netback.c +2 -1

@@ -68,7 +68,9 @@ unsigned int rx_stall_timeout_msecs = 60000;
 module_param(rx_stall_timeout_msecs, uint, 0444);
 
 unsigned int xenvif_max_queues;
-module_param_named(max_queues, xenvif_max_queues, uint, 0644);
+static int xennet_set_max_queues(const char *val, struct kernel_param *kp);
+module_param_call(max_queues, xennet_set_max_queues, param_get_int,

param_get_uint

drivers/net/xen-netback/netback.c unsigned int rx_stall_timeout_msecs = 60000; · line 73

Quoted 1 line

drivers/net/xen-netback/netback.c +1 -0
+		  &xenvif_max_queues, 0600);

Why the change from mode 0644 to 0600?

drivers/net/xen-netback/netback.c static struct xen_netif_rx_response *make_rx_response(struct xenvif_queue *queue · line 118

Quoted 11 lines

drivers/net/xen-netback/netback.c +7 -0

@@ -107,6 +109,20 @@ static struct xen_netif_rx_response *make_rx_response(struct xenvif_queue *queue
 					     u16      size,
 					     u16      flags);
 
+static int xennet_set_max_queues(const char *val, struct kernel_param *kp)
+{
+	unsigned int cpus = num_online_cpus();
+	unsigned int max_queues = simple_strtoul(val, NULL, 10);
+
+	if (max_queues == 0 || max_queues > cpus) {
+		pr_info("max_queues %d is out of range [0 - %d]!\n",

%u in both cases.

drivers/net/xen-netback/netback.c static struct xen_netif_rx_response *make_rx_response(struct xenvif_queue *queue · line 120

Quoted 2 lines

drivers/net/xen-netback/netback.c +2 -0
+			max_queues, cpus);
+		return -EINVAL;

Considering the message: -ERANGE?

drivers/net/xen-netback/netback.c static struct xen_netif_rx_response *make_rx_response(struct xenvif_queue *queue · line 123

Quoted 3 lines

drivers/net/xen-netback/netback.c +3 -0
+	}
+
+	return param_set_int(val, kp);

param_set_uint()

Quoted 8 lines

drivers/net/xen-netback/netback.c +1 -0

@@ -2110,15 +2126,15 @@ int xenvif_dealloc_kthread(void *data)
 static int __init netback_init(void)
 {
 	int rc = 0;
+	unsigned int cpus = num_online_cpus();
 
 	if (!xen_domain())
 		return -ENODEV;

drivers/net/xen-netback/netback.c int xenvif_dealloc_kthread(void *data) · line 2137

Quoted 10 lines

drivers/net/xen-netback/netback.c +4 -5

 
-	/* Allow as many queues as there are CPUs if user has not
-	 * specified a value.
-	 */
-	if (xenvif_max_queues == 0)
-		xenvif_max_queues = num_online_cpus();
+	/* Allow at most as many queues as CPUs. */
+	if (xenvif_max_queues == 0 || xenvif_max_queues > cpus)
+		xenvif_max_queues = cpus;
+	pr_info("vif max_queues: %d\n", xenvif_max_queues);

%u again.

Jan

Joe Jin

author

2015-10-23 07:58 UTC #6 lore raw

PATCH 1/2 xen-netback: limit xen vif max queues number to online cpus

In reply to Joe Jin

Should not allocate xen vif queues number more than online cpus.

Signed-off-by Joe Jin <[email protected]>

Cc Wei Liu <[email protected]>

Cc Ian Campbell <[email protected]>

Cc Boris Ostrovsky <[email protected]>

Cc Konrad Rzeszutek Wilk <[email protected]>

---
 drivers/net/xen-netback/netback.c |   28 ++++++++++++++++++++++------
 1 files changed, 22 insertions(+), 6 deletions(-)

drivers/net/xen-netback/netback.c +22 -6

index ec98d43..387fbd1 100644
@@ -68,7 +68,9 @@ unsigned int rx_stall_timeout_msecs = 60000;
 module_param(rx_stall_timeout_msecs, uint, 0444);
 
 unsigned int xenvif_max_queues;
-module_param_named(max_queues, xenvif_max_queues, uint, 0644);
+static int xennet_set_max_queues(const char *val, struct kernel_param *kp);
+module_param_call(max_queues, xennet_set_max_queues, param_get_int,
+		  &xenvif_max_queues, 0600);
 MODULE_PARM_DESC(max_queues,
 		 "Maximum number of queues per virtual interface");

@@ -107,6 +109,20 @@ static struct xen_netif_rx_response *make_rx_response(struct xenvif_queue *queue
 					     u16      size,
 					     u16      flags);
 
+static int xennet_set_max_queues(const char *val, struct kernel_param *kp)
+{
+	unsigned int cpus = num_online_cpus();
+	unsigned int max_queues = simple_strtoul(val, NULL, 10);
+
+	if (max_queues == 0 || max_queues > cpus) {
+		pr_info("max_queues %d is out of range [0 - %d]!\n", 
+			max_queues, cpus);
+		return -EINVAL;
+	}
+
+	return param_set_int(val, kp);
+}
+
 static inline unsigned long idx_to_pfn(struct xenvif_queue *queue,
 				       u16 idx)
 {

@@ -2110,15 +2126,15 @@ int xenvif_dealloc_kthread(void *data)
 static int __init netback_init(void)
 {
 	int rc = 0;
+	unsigned int cpus = num_online_cpus();
 
 	if (!xen_domain())
 		return -ENODEV;
 
-	/* Allow as many queues as there are CPUs if user has not
-	 * specified a value.
-	 */
-	if (xenvif_max_queues == 0)
-		xenvif_max_queues = num_online_cpus();
+	/* Allow at most as many queues as CPUs. */
+	if (xenvif_max_queues == 0 || xenvif_max_queues > cpus)
+		xenvif_max_queues = cpus;
+	pr_info("vif max_queues: %d\n", xenvif_max_queues);
 
 	if (fatal_skb_slots < XEN_NETBK_LEGACY_SLOTS_MAX) {
 		pr_info("fatal_skb_slots too small (%d), bump it to XEN_NETBK_LEGACY_SLOTS_MAX (%d)\n",

Signature · 1 line

1.7.1

Joe Jin

author

2015-10-23 07:59 UTC #7 lore raw

PATCH 2/2 xen-front: limit vnic max_queues number to online cpus

In reply to Joe Jin

Should not allocate vnic queues number more than online cpus.

Signed-off-by Joe Jin <[email protected]>

Cc Wei Liu <[email protected]>

Cc Ian Campbell <[email protected]>

Cc Boris Ostrovsky <[email protected]>

Cc Konrad Rzeszutek Wilk <[email protected]>

---
 drivers/net/xen-netfront.c |   27 +++++++++++++++++++++------
 1 files changed, 21 insertions(+), 6 deletions(-)

drivers/net/xen-netfront.c +21 -6

index f821a97..1c8a7cf 100644
@@ -58,7 +58,9 @@
 
 /* Module parameters */
 static unsigned int xennet_max_queues;
-module_param_named(max_queues, xennet_max_queues, uint, 0644);
+static int xennet_set_max_queues(const char *val, struct kernel_param *kp);
+module_param_call(max_queues, xennet_set_max_queues, param_get_int,
+		  &xennet_max_queues, 0600);
 MODULE_PARM_DESC(max_queues,
 		 "Maximum number of queues per virtual interface");

@@ -164,6 +166,19 @@ struct netfront_rx_info {
 	struct xen_netif_extra_info extras[XEN_NETIF_EXTRA_TYPE_MAX - 1];
 };
 
+static int xennet_set_max_queues(const char *val, struct kernel_param *kp)
+{
+	unsigned int cpus = num_online_cpus();
+	unsigned int max_queues = simple_strtoul(val, NULL, 10);
+
+	if (max_queues == 0 || max_queues > cpus) {
+		pr_err("max_queues %d is out of range [0 - %d]!\n", 
+		       max_queues, cpus);
+		return -EINVAL;
+	}
+	return param_set_int(val, kp);
+}
+
 static void skb_entry_set_link(union skb_entry *list, unsigned short id)
 {
 	list->link = id;

@@ -2126,6 +2141,8 @@ static struct xenbus_driver netfront_driver = {
 
 static int __init netif_init(void)
 {
+	unsigned int cpus = num_online_cpus();
+
 	if (!xen_domain())
 		return -ENODEV;

@@ -2134,11 +2151,9 @@ static int __init netif_init(void)
 
 	pr_info("Initialising Xen virtual ethernet driver\n");
 
-	/* Allow as many queues as there are CPUs if user has not
-	 * specified a value.
-	 */
-	if (xennet_max_queues == 0)
-		xennet_max_queues = num_online_cpus();
+	/* Allow at most as many queues as there are CPUs. */
+	if (xennet_max_queues == 0 || xennet_max_queues > cpus)
+		xennet_max_queues = cpus;
 
 	return xenbus_register_frontend(&netfront_driver);
 }

Signature · 1 line

1.7.1

Joe Jin

author

2015-10-23 07:59 UTC #8 lore raw

PATCH 2/2 xen-front: limit vnic max_queues number to online cpus

In reply to Joe Jin

Should not allocate vnic queues number more than online cpus.

Signed-off-by Joe Jin <[email protected]>

Cc Wei Liu <[email protected]>

Cc Ian Campbell <[email protected]>

Cc Boris Ostrovsky <[email protected]>

Cc Konrad Rzeszutek Wilk <[email protected]>

---
 drivers/net/xen-netfront.c |   27 +++++++++++++++++++++------
 1 files changed, 21 insertions(+), 6 deletions(-)

drivers/net/xen-netfront.c +21 -6

index f821a97..1c8a7cf 100644
@@ -58,7 +58,9 @@
 
 /* Module parameters */
 static unsigned int xennet_max_queues;
-module_param_named(max_queues, xennet_max_queues, uint, 0644);
+static int xennet_set_max_queues(const char *val, struct kernel_param *kp);
+module_param_call(max_queues, xennet_set_max_queues, param_get_int,
+		  &xennet_max_queues, 0600);
 MODULE_PARM_DESC(max_queues,
 		 "Maximum number of queues per virtual interface");

@@ -164,6 +166,19 @@ struct netfront_rx_info {
 	struct xen_netif_extra_info extras[XEN_NETIF_EXTRA_TYPE_MAX - 1];
 };
 
+static int xennet_set_max_queues(const char *val, struct kernel_param *kp)
+{
+	unsigned int cpus = num_online_cpus();
+	unsigned int max_queues = simple_strtoul(val, NULL, 10);
+
+	if (max_queues == 0 || max_queues > cpus) {
+		pr_err("max_queues %d is out of range [0 - %d]!\n", 
+		       max_queues, cpus);
+		return -EINVAL;
+	}
+	return param_set_int(val, kp);
+}
+
 static void skb_entry_set_link(union skb_entry *list, unsigned short id)
 {
 	list->link = id;

@@ -2126,6 +2141,8 @@ static struct xenbus_driver netfront_driver = {
 
 static int __init netif_init(void)
 {
+	unsigned int cpus = num_online_cpus();
+
 	if (!xen_domain())
 		return -ENODEV;

@@ -2134,11 +2151,9 @@ static int __init netif_init(void)
 
 	pr_info("Initialising Xen virtual ethernet driver\n");
 
-	/* Allow as many queues as there are CPUs if user has not
-	 * specified a value.
-	 */
-	if (xennet_max_queues == 0)
-		xennet_max_queues = num_online_cpus();
+	/* Allow at most as many queues as there are CPUs. */
+	if (xennet_max_queues == 0 || xennet_max_queues > cpus)
+		xennet_max_queues = cpus;
 
 	return xenbus_register_frontend(&netfront_driver);
 }

Signature · 1 line

1.7.1

Paul Durrant

2015-10-23 08:48 UTC #9 lore raw

In reply to Joe Jin

Quoted 10 lines

-----Original Message-----
From: [email protected] [mailto:netdev- [email protected]] On Behalf Of Joe Jin
Sent: 23 October 2015 08:54
To: Wei Liu; Ian Campbell; Boris Ostrovsky; Konrad Rzeszutek Wilk; David S. Miller
Cc: [email protected]; [email protected]
Subject: [PATCH 0/2] limit xen vnic max queues number to online cpu number

Quoted 4 lines

Currently xen vnic allowed to create lots of queues by set module param max_queues(both netback and netfront), when queues number larger than cpu number, it does not help for performance but need more cpu time.

But it's an override, so why would you want to limit it? The parameter should not be set in the common case.

Paul

Quoted 15 lines

This patchset limit netback and netfront max queues number to online cpus number.
Joe Jin (2):
xen-netback: limit xen vif max queues number to online cpus
xen-front: limit vnic max_queues number to online cpus
 drivers/net/xen-netback/netback.c |   28 ++++++++++++++++++++++------
 drivers/net/xen-netfront.c        |   27 +++++++++++++++++++++------
 2 files changed, 43 insertions(+), 12 deletions(-)
--
To unsubscribe from this list: send the line "unsubscribe netdev" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

Joe Jin

author

2015-10-23 09:05 UTC #10 lore raw

Quoted 10 lines · Paul Durrant

Quoted 10 lines
-----Original Message-----
From: [email protected] [mailto:netdev- [email protected]] On Behalf Of Joe Jin
Sent: 23 October 2015 08:54
To: Wei Liu; Ian Campbell; Boris Ostrovsky; Konrad Rzeszutek Wilk; David S. Miller
Cc: [email protected]; [email protected]
Subject: [PATCH 0/2] limit xen vnic max queues number to online cpu number

Quoted 6 lines

Quoted 4 lines
Currently xen vnic allowed to create lots of queues by set module param max_queues(both netback and netfront), when queues number larger than cpu number, it does not help for performance but need more cpu time.
But it's an override, so why would you want to limit it? The parameter should not be set in the common case.

Always we can not stop people use it because we provided it :)

If queues number is larger than cpu number, with heavy network load, cpus have to take more time for interrupt, this lead others less chance to be scheduled.
Imaging dom0 have 64 cpus, and assigned 4 vcpus to the guest, if set max_queues to 64 on guest, it will consumed more cpu times and bandwidth on backend, I think this is not we expected?

Thanks,
Joe

Quoted 18 lines

Paul
Quoted 15 lines
This patchset limit netback and netfront max queues number to online cpus number.
Joe Jin (2):
xen-netback: limit xen vif max queues number to online cpus
xen-front: limit vnic max_queues number to online cpus
 drivers/net/xen-netback/netback.c |   28 ++++++++++++++++++++++------
 drivers/net/xen-netfront.c        |   27 +++++++++++++++++++++------
 2 files changed, 43 insertions(+), 12 deletions(-)
--
To unsubscribe from this list: send the line "unsubscribe netdev" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

Jan Beulich

2015-10-23 09:28 UTC #11 lore raw

Quoted 17 lines

Quoted 1 line
Quoted 1 line · On 23.10.15 at 11:05, <[email protected]> wrote:
Quoted 15 lines · Paul Durrant
Quoted 14 lines
-----Original Message-----
From: [email protected] [mailto:netdev- [email protected]] On Behalf Of Joe Jin
Sent: 23 October 2015 08:54
To: Wei Liu; Ian Campbell; Boris Ostrovsky; Konrad Rzeszutek Wilk; David S. Miller
Cc: [email protected]; [email protected]
Subject: [PATCH 0/2] limit xen vnic max queues number to online cpu number
Currently xen vnic allowed to create lots of queues by set module param max_queues(both netback and netfront), when queues number larger than cpu number, it does not help for performance but need more cpu time.

Quoted 4 lines

Quoted 1 line
But it's an override, so why would you want to limit it? The parameter
should not be set in the common case.
Always we can not stop people use it because we provided it :)

Well, it's always a question of whether preventing the admin to shoot himself in the foot makes sense: When it leads to an unusable system, it probably does. When it leads to a sub-optimal working system, perhaps we should allow them their freedom?

Jan

Jan Beulich

2015-10-23 09:28 UTC #12 lore raw

In reply to Joe Jin

Quoted 17 lines

Quoted 1 line
Quoted 1 line · On 23.10.15 at 11:05, <[email protected]> wrote:
Quoted 15 lines · Paul Durrant
Quoted 14 lines
-----Original Message-----
From: [email protected] [mailto:netdev- [email protected]] On Behalf Of Joe Jin
Sent: 23 October 2015 08:54
To: Wei Liu; Ian Campbell; Boris Ostrovsky; Konrad Rzeszutek Wilk; David S. Miller
Cc: [email protected]; [email protected]
Subject: [PATCH 0/2] limit xen vnic max queues number to online cpu number
Currently xen vnic allowed to create lots of queues by set module param max_queues(both netback and netfront), when queues number larger than cpu number, it does not help for performance but need more cpu time.

Quoted 4 lines

Quoted 1 line
But it's an override, so why would you want to limit it? The parameter
should not be set in the common case.
Always we can not stop people use it because we provided it :)

Jan

Paul Durrant

2015-10-23 09:57 UTC #13 lore raw

In reply to Joe Jin

Quoted 23 lines

-----Original Message-----
From: Joe Jin [mailto:[email protected]]
Sent: 23 October 2015 10:05
To: Paul Durrant; Wei Liu; Ian Campbell; Boris Ostrovsky; Konrad Rzeszutek Wilk; David S. Miller
Cc: [email protected]; [email protected]
Subject: Re: [PATCH 0/2] limit xen vnic max queues number to online cpu number
Quoted 5 lines · Paul Durrant
Quoted 5 lines
-----Original Message-----
From: [email protected] [mailto:netdev- [email protected]] On Behalf Of Joe Jin
Sent: 23 October 2015 08:54
To: Wei Liu; Ian Campbell; Boris Ostrovsky; Konrad Rzeszutek Wilk; David
S.
Quoted 7 lines
Quoted 7 lines
Miller
Cc: [email protected]; [email protected]
Subject: [PATCH 0/2] limit xen vnic max queues number to online cpu number
Currently xen vnic allowed to create lots of queues by set module param max_queues(both netback and netfront), when queues number larger

Quoted 9 lines

than
Quoted 4 lines
Quoted 2 lines
cpu number, it does not help for performance but need more cpu time.
But it's an override, so why would you want to limit it? The parameter
should not be set in the common case.
Always we can not stop people use it because we provided it :)

Indeed, and I believe it was provided largely for test purposes... to force the limit to whatever the admin wants.

Quoted 7 lines

If queues number is larger than cpu number, with heavy network load, cpus have to take more time for interrupt, this lead others less chance to be scheduled.
Imaging dom0 have 64 cpus, and assigned 4 vcpus to the guest, if set max_queues to 64 on guest, it will consumed more cpu times and bandwidth on backend, I think this is not we expected?

I think that would be entirely expected.

Paul

Quoted 21 lines

Thanks,
Joe
Quoted 11 lines
Paul
Quoted 8 lines
This patchset limit netback and netfront max queues number to online cpus number.
Joe Jin (2):
xen-netback: limit xen vif max queues number to online cpus
xen-front: limit vnic max_queues number to online cpus
 drivers/net/xen-netback/netback.c |   28 ++++++++++++++++++++++---
---
Quoted 7 lines
Quoted 7 lines
 drivers/net/xen-netfront.c        |   27 +++++++++++++++++++++------
 2 files changed, 43 insertions(+), 12 deletions(-)
--
To unsubscribe from this list: send the line "unsubscribe netdev" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

Paul Durrant

2015-10-23 09:57 UTC #14 lore raw

In reply to Joe Jin

Quoted 23 lines

-----Original Message-----
From: Joe Jin [mailto:[email protected]]
Sent: 23 October 2015 10:05
To: Paul Durrant; Wei Liu; Ian Campbell; Boris Ostrovsky; Konrad Rzeszutek Wilk; David S. Miller
Cc: [email protected]; [email protected]
Subject: Re: [PATCH 0/2] limit xen vnic max queues number to online cpu number
Quoted 5 lines · Paul Durrant
Quoted 5 lines
-----Original Message-----
From: [email protected] [mailto:netdev- [email protected]] On Behalf Of Joe Jin
Sent: 23 October 2015 08:54
To: Wei Liu; Ian Campbell; Boris Ostrovsky; Konrad Rzeszutek Wilk; David
S.
Quoted 7 lines
Quoted 7 lines
Miller
Cc: [email protected]; [email protected]
Subject: [PATCH 0/2] limit xen vnic max queues number to online cpu number
Currently xen vnic allowed to create lots of queues by set module param max_queues(both netback and netfront), when queues number larger

Quoted 9 lines

than
Quoted 4 lines
Quoted 2 lines
cpu number, it does not help for performance but need more cpu time.
But it's an override, so why would you want to limit it? The parameter
should not be set in the common case.
Always we can not stop people use it because we provided it :)

Indeed, and I believe it was provided largely for test purposes... to force the limit to whatever the admin wants.

Quoted 7 lines

If queues number is larger than cpu number, with heavy network load, cpus have to take more time for interrupt, this lead others less chance to be scheduled.
Imaging dom0 have 64 cpus, and assigned 4 vcpus to the guest, if set max_queues to 64 on guest, it will consumed more cpu times and bandwidth on backend, I think this is not we expected?

I think that would be entirely expected.

Paul

Quoted 21 lines

Thanks,
Joe
Quoted 11 lines
Paul
Quoted 8 lines
This patchset limit netback and netfront max queues number to online cpus number.
Joe Jin (2):
xen-netback: limit xen vif max queues number to online cpus
xen-front: limit vnic max_queues number to online cpus
 drivers/net/xen-netback/netback.c |   28 ++++++++++++++++++++++---
---
Quoted 7 lines
Quoted 7 lines
 drivers/net/xen-netfront.c        |   27 +++++++++++++++++++++------
 2 files changed, 43 insertions(+), 12 deletions(-)
--
To unsubscribe from this list: send the line "unsubscribe netdev" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

Joe Jin

author

2015-10-23 09:05 UTC #15 lore raw

In reply to Paul Durrant

Quoted 10 lines · Paul Durrant

Quoted 10 lines
-----Original Message-----
From: [email protected] [mailto:netdev- [email protected]] On Behalf Of Joe Jin
Sent: 23 October 2015 08:54
To: Wei Liu; Ian Campbell; Boris Ostrovsky; Konrad Rzeszutek Wilk; David S. Miller
Cc: [email protected]; [email protected]
Subject: [PATCH 0/2] limit xen vnic max queues number to online cpu number

Quoted 6 lines

Quoted 4 lines
Currently xen vnic allowed to create lots of queues by set module param max_queues(both netback and netfront), when queues number larger than cpu number, it does not help for performance but need more cpu time.
But it's an override, so why would you want to limit it? The parameter should not be set in the common case.

Always we can not stop people use it because we provided it :)

Thanks,
Joe

Quoted 18 lines

Paul
Quoted 15 lines
This patchset limit netback and netfront max queues number to online cpus number.
Joe Jin (2):
xen-netback: limit xen vif max queues number to online cpus
xen-front: limit vnic max_queues number to online cpus
 drivers/net/xen-netback/netback.c |   28 ++++++++++++++++++++++------
 drivers/net/xen-netfront.c        |   27 +++++++++++++++++++++------
 2 files changed, 43 insertions(+), 12 deletions(-)
--
To unsubscribe from this list: send the line "unsubscribe netdev" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

Paul Durrant

2015-10-23 08:48 UTC #16 lore raw

In reply to Joe Jin

Quoted 10 lines

-----Original Message-----
From: [email protected] [mailto:netdev- [email protected]] On Behalf Of Joe Jin
Sent: 23 October 2015 08:54
To: Wei Liu; Ian Campbell; Boris Ostrovsky; Konrad Rzeszutek Wilk; David S. Miller
Cc: [email protected]; [email protected]
Subject: [PATCH 0/2] limit xen vnic max queues number to online cpu number

Quoted 4 lines

Currently xen vnic allowed to create lots of queues by set module param max_queues(both netback and netfront), when queues number larger than cpu number, it does not help for performance but need more cpu time.

But it's an override, so why would you want to limit it? The parameter should not be set in the common case.

Paul

Quoted 15 lines

This patchset limit netback and netfront max queues number to online cpus number.
Joe Jin (2):
xen-netback: limit xen vif max queues number to online cpus
xen-front: limit vnic max_queues number to online cpus
 drivers/net/xen-netback/netback.c |   28 ++++++++++++++++++++++------
 drivers/net/xen-netfront.c        |   27 +++++++++++++++++++++------
 2 files changed, 43 insertions(+), 12 deletions(-)
--
To unsubscribe from this list: send the line "unsubscribe netdev" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html