Skip to contentSkip to conversation

accel-config: Add command config-user-default to configure and enable all available devices and WQs

PATCH In review Fenghua Yu 2 participants 11 messages Aug 1, 2023 – Aug 2, 2023 Documentation

lore
Share & tools
lore

Fenghua Yu

author
#0 lore raw

cover accel-config: Add command config-user-default to configure and enable all available devices and WQs

Although current accel-config can configure and enable IDXD devices, user needs to know a lot of IDXD knowledges and information to do so. Sometimes user wants an easier way to configure and enable IDXD devices without sophisticated IDXD knowledges and information.

This patch set adds "config-user-default" to configure and enable all available devices and WQs with default configurations.

Fenghua Yu (7):
accel-config: Skip configuring ats_disable if the attribute is not
present
accel-config: Add config-user-default command
accel-config: Add option "-c <config_file>" to load default
configurations from the file
accel-config: Disable default configured WQs and devices
accel-config: Add "-n <wq_name>" to specify WQ name for disabling WQs
accel-config: Add user_default_profile.conf
accel-config: Add doumentation for new command "config-user-default"

 Documentation/accfg/Makefile.am               |   6 +-
 .../accel-config-config-user-default.txt      |  75 +++
 accfg/accel-config.c                          |   1 +
 accfg/config.c                                | 482 +++++++++++++++++-
 accfg/libaccel_config.h                       |   1 +
 builtin.h                                     |   1 +
 contrib/configs/profilenote.txt               |  28 +
 contrib/configs/user_default_profile.conf     |  42 ++
 8 files changed, 633 insertions(+), 3 deletions(-)
 create mode 100644 Documentation/accfg/accel-config-config-user-default.txt
 create mode 100644 contrib/configs/user_default_profile.conf
Signature · 1 line
2.31.1
author
#1 lore raw

PATCH 1/7 accel-config: Skip configuring ats_disable if the attribute is not present

Not every IDXD device supports Address Translation Service (ATS) disable feature. Add the check so the ATS disable attribute can only be written on the IDXD devices that support the feature.

---
 accfg/config.c | 15 ++++++++++++++-
 1 file changed, 14 insertions(+), 1 deletion(-)
accfg/config.c +14 -1
index 08952f8..2631fd9 100644
@@ -117,6 +117,7 @@ static const struct group_set_table group_table[] = {
 
 static bool is_wq_threshold_writable(struct accfg_wq *wq, int val);
 static bool is_wq_prs_disable_writable(struct accfg_wq *wq, int val);
+static bool is_wq_ats_disable_writable(struct accfg_wq *wq, int val);
 
 static const struct wq_set_table wq_table[] = {
 	{ "size", accfg_wq_set_size, NULL, NULL, NULL },
@@ -132,7 +133,8 @@ static const struct wq_set_table wq_table[] = {
 	{ "max_transfer_size", NULL, accfg_wq_set_max_transfer_size, NULL, NULL },
 	{ "threshold", accfg_wq_set_threshold, NULL, NULL,
 		is_wq_threshold_writable },
-	{ "ats_disable", accfg_wq_set_ats_disable, NULL, NULL, NULL },
+	{ "ats_disable", accfg_wq_set_ats_disable, NULL, NULL,
+		is_wq_ats_disable_writable },
 	{ "prs_disable", accfg_wq_set_prs_disable, NULL, NULL,
 		is_wq_prs_disable_writable },
 };
@@ -231,6 +233,17 @@ static bool is_wq_prs_disable_writable(struct accfg_wq *wq, int val)
 	return true;
 }
 
+static bool is_wq_ats_disable_writable(struct accfg_wq *wq, int val)
+{
+	if (val < 0 || val > 1)
+		return false;
+
+	if (accfg_wq_get_ats_disable(wq) < 0)
+		return false;
+
+	return true;
+}
+
 static int device_attribute_filter(char *key)
 {
 	for (int i = 0; i < (int)ARRAY_SIZE(device_table); i++) {
Signature · 1 line
2.31.1
author
#2 lore raw

PATCH 2/7 accel-config: Add config-user-default command

In reply to Fenghua Yu

Although current accel-config provides thorough ways to configure IDXD devices and WQs, sometimes user needs an easier way to configure and enable them without many knowledges of IDXD.

A new command "config-user-default" is added to configure and enable all available devices and WQs for user usage in the following default configurations:

1. Fixed configurations:

	"mode":"shared",
	"group_id":0,
	"priority":10,
	"block_on_fault":1,
	"name":"user_default_wq",
	"ats_disable":0,
	"prs_disable":1
	"type": "user"
	"driver_name":"user",

2. Calculated configurations:
"size": max WQ size / max WQs
"threshold": WQ size

3. Default configurations that have been set by IDXD driver:

	"max_batch_size"
	"max_transfer_size"
	"op_config"
---
 accfg/accel-config.c    |   1 +
 accfg/config.c          | 268 ++++++++++++++++++++++++++++++++++++++++
 accfg/libaccel_config.h |   1 +
 builtin.h               |   1 +
 4 files changed, 271 insertions(+)
accfg/accel-config.c +1 -0
index a897608..cf3882e 100644
@@ -67,6 +67,7 @@ static struct cmd_struct commands[] = {
 	{"config-group", cmd_config_group},
 	{"config-wq", cmd_config_wq},
 	{"config-engine", cmd_config_engine},
+	{"config-user-default", cmd_config_default},
 #ifdef ENABLE_TEST
 	{"test", cmd_test},
 #endif
accfg/config.c +268 -0

277 lines not on this page · raw message

accfg/libaccel_config.h +1 -0
index 3feb885..ca51b0a 100644
@@ -36,6 +36,7 @@ enum accfg_device_version {
 enum accfg_device_type {
 	ACCFG_DEVICE_DSA = 0,
 	ACCFG_DEVICE_IAX = 1,
+	ACCFG_DEVICE_MAX = 2,
 	ACCFG_DEVICE_TYPE_UNKNOWN = -1,
 };
 
builtin.h +1 -0
index e1f0b83..96a9d61 100644
@@ -30,6 +30,7 @@ int cmd_config_device(int argc, const char **argv, void *ctx);
 int cmd_config_group(int argc, const char **argv, void *ctx);
 int cmd_config_wq(int argc, const char **argv, void *ctx);
 int cmd_config_engine(int argc, const char **argv, void *ctx);
+int cmd_config_default(int argc, const char **argv, void *ctx);
 #ifdef ENABLE_TEST
 int cmd_test(int argc, const char **argv, void *ctx);
 #endif
Signature · 1 line
2.31.1
#3 lore raw

Re: [Accel-config] [PATCH 2/7] accel-config: Add config-user-default command

Quoted 124 lines · Fenghua Yu

Although current accel-config provides thorough ways to configure IDXD devices and WQs, sometimes user needs an easier way to configure and enable them without many knowledges of IDXD.

A new command "config-user-default" is added to configure and enable all available devices and WQs for user usage in the following default configurations:

1. Fixed configurations:

	"mode":"shared",
	"group_id":0,
	"priority":10,
	"block_on_fault":1,
	"name":"user_default_wq",
	"ats_disable":0,
	"prs_disable":1
	"type": "user"
	"driver_name":"user",

2. Calculated configurations:
"size": max WQ size / max WQs
"threshold": WQ size

3. Default configurations that have been set by IDXD driver:

	"max_batch_size"
	"max_transfer_size"
	"op_config"
---
  accfg/accel-config.c    |   1 +
  accfg/config.c          | 268 ++++++++++++++++++++++++++++++++++++++++
  accfg/libaccel_config.h |   1 +
  builtin.h               |   1 +
  4 files changed, 271 insertions(+)
accfg/accel-config.c +1 -0
index a897608..cf3882e 100644
@@ -67,6 +67,7 @@ static struct cmd_struct commands[] = {
  	{"config-group", cmd_config_group},
  	{"config-wq", cmd_config_wq},
  	{"config-engine", cmd_config_engine},
+	{"config-user-default", cmd_config_default},
  #ifdef ENABLE_TEST
  	{"test", cmd_test},
  #endif
accfg/config.c +67 -0
index 2631fd9..f3fa83f 100644
@@ -119,6 +119,87 @@ static bool is_wq_threshold_writable(struct accfg_wq *wq, int val);
  static bool is_wq_prs_disable_writable(struct accfg_wq *wq, int val);
  static bool is_wq_ats_disable_writable(struct accfg_wq *wq, int val);
 
+static int get_wq_size(struct accfg_device *dev)
+{
+	int max_wq_size, max_wqs;
+
+	max_wq_size = accfg_device_get_max_work_queues_size(dev);
+	max_wqs = accfg_device_get_max_work_queues(dev);
+
+	return max_wq_size / max_wqs;
+}
+
+static int config_default_wq_set_prs_disable(struct accfg_wq *wq, int val)
+{
+	if (!is_wq_prs_disable_writable(wq, val))
+		return -EPERM;
+
+	return accfg_wq_set_prs_disable(wq, val);
+}
+
+static int config_default_wq_set_ats_disable(struct accfg_wq *wq, int val)
+{
+	if (!is_wq_ats_disable_writable(wq, val))
+		return -EPERM;
+
+	return accfg_wq_set_ats_disable(wq, val);
+}
+
+static int config_default_wq_set_threshold(struct accfg_wq *wq, int val)
+{
+	if (!is_wq_threshold_writable(wq, val))
+		return -EPERM;
+
+	return accfg_wq_set_threshold(wq, val);
+}
+
+static struct conf_def_wq_param {
+	struct wq_parameters param;
+	bool configured;
+} conf_def_wq_param[ACCFG_DEVICE_MAX];
+
+/* Return WQ parameter for dev type. */
+static struct wq_parameters *get_conf_def_wq_param(enum accfg_device_type type)
+{
+	if (type == ACCFG_DEVICE_DSA)
+		return &conf_def_wq_param[ACCFG_DEVICE_DSA].param;
+	else if (type == ACCFG_DEVICE_IAX)
+		return &conf_def_wq_param[ACCFG_DEVICE_IAX].param;
+
+	return NULL;
+}
+
+/* Check if dev is configured. */
+static bool conf_def_dev_configured(struct accfg_device *dev)
+{
+	if (accfg_device_get_type(dev) == ACCFG_DEVICE_DSA)
+		return conf_def_wq_param[ACCFG_DEVICE_DSA].configured;
+	else if (accfg_device_get_type(dev) == ACCFG_DEVICE_IAX)
+		return conf_def_wq_param[ACCFG_DEVICE_IAX].configured;
+
+	return false;
+}
+
+/* Set WQ parameters based on device cap: size and threshold. */
+static int config_default_wq_set_on_dev(struct accfg_device *dev)
+{
+	enum accfg_device_type dev_type;
+	struct wq_parameters *p;
+

accfg/config.c static bool is_wq_threshold_writable(struct accfg_wq *wq, int val); · line 198

Quoted 10 lines
accfg/config.c +10 -0
+	dev_type = accfg_device_get_type(dev);
+	p = get_conf_def_wq_param(dev_type);
+	if (!p)
+		return -EINVAL;
+
+	p->wq_size = get_wq_size(dev);
+	if (p->wq_size <= 0)
+		return -ENOSPC;
+
+	p->threshold = p->wq_size;

Should check if wq is shared before setting threshold. threshold has no meaning for dwq.

Quoted 222 lines
accfg/config.c +191 -0
+
+	return 0;
+}
+
  static const struct wq_set_table wq_table[] = {
  	{ "size", accfg_wq_set_size, NULL, NULL, NULL },
  	{ "priority", accfg_wq_set_priority, NULL, NULL, NULL },
@@ -1234,3 +1315,190 @@ int cmd_config(int argc, const char **argv, void *ctx)
 
  	return rc;
  }
+
+static int config_default_wq(struct accfg_wq *wq)
+{
+	struct accfg_device *dev = accfg_wq_get_device(wq);
+	enum accfg_device_type dev_type;
+	struct wq_parameters *p;
+
+	if (!conf_def_dev_configured(dev))
+		return 0;
+
+	dev_type = accfg_device_get_type(dev);
+	p = get_conf_def_wq_param(dev_type);
+	if (!p)
+		return -EINVAL;
+
+	accfg_wq_set_priority(wq, p->priority);
+	accfg_wq_set_group_id(wq, p->group_id);
+	accfg_wq_set_block_on_fault(wq, p->block_on_fault);
+	accfg_wq_set_str_mode(wq, p->mode);
+	accfg_wq_set_str_type(wq, p->type);
+	accfg_wq_set_str_name(wq, p->name);
+	accfg_wq_set_str_driver_name(wq, p->driver_name);
+
+	accfg_wq_set_size(wq, p->wq_size);
+	config_default_wq_set_threshold(wq, p->threshold);
+	config_default_wq_set_prs_disable(wq, p->prs_disable);
+	config_default_wq_set_ats_disable(wq, p->ats_disable);
+
+	return 0;
+}
+
+static int config_default_engine(struct accfg_engine *engine,
+				 struct accfg_device *dev)
+{
+	enum accfg_device_type dev_type;
+	struct wq_parameters *p;
+
+	/* Engine's group_id is same as WQ's. */
+	dev_type = accfg_device_get_type(dev);
+	p = get_conf_def_wq_param(dev_type);
+	if (!p)
+		return -EINVAL;
+
+	return accfg_engine_set_group_id(engine, p->group_id);
+}
+
+static void config_default_activate_devices(void *ctx)
+{
+	enum accfg_device_state dev_state;
+	const char *dev_name, *wq_name;
+	struct accfg_engine *engine;
+	struct accfg_device *dev;
+	struct accfg_wq *wq;
+	int rc = 0;
+
+	accfg_device_foreach(ctx, dev) {
+		/* Skip device that is not configured. */
+		if (!conf_def_dev_configured(dev))
+			continue;
+
+		/* Don't enable WQs/engines on partially enabled devices. */
+		dev_state = accfg_device_get_state(dev);
+		if (dev_state == ACCFG_DEVICE_ENABLED)
+			continue;
+
+		/* Set WQ parameters calculated based on dev. */
+		config_default_wq_set_on_dev(dev);
+
+		/* Config WQs */
+		accfg_wq_foreach(dev, wq) {
+			if (verbose)
+				printf("config %s\n", accfg_wq_get_devname(wq));
+
+			config_default_wq(wq);
+		}
+
+		/* Config engines */
+		accfg_engine_foreach(dev, engine)
+			config_default_engine(engine, dev);
+
+		/* Enable device */
+		dev_name = accfg_device_get_devname(dev);
+		if (verbose)
+			printf("enable %s\n", dev_name);
+		rc = accfg_device_enable(dev);
+		if (rc) {
+			fprintf(stderr, "Error enabling %s\n", dev_name);
+			continue;
+		}
+
+		/* Enable WQs */
+		accfg_wq_foreach(dev, wq) {
+			wq_name = accfg_wq_get_devname(wq);
+			if (verbose)
+				printf("enable %s\n", wq_name);
+
+			rc = accfg_wq_enable(wq);
+			if (rc) {
+				fprintf(stderr, "Error enabling %s\n", wq_name);
+				continue;
+			}
+		}
+	}
+}
+
+#define CONFIG_DEFAULT_WQ_PRIORITY		10
+#define CONFIG_DEFAULT_WQ_GROUP_ID		0
+#define CONFIG_DEFAULT_WQ_BLOCK_ON_FAULT	1
+#define CONFIG_DEFAULT_WQ_PRS_DISABLE		1
+#define CONFIG_DEFAULT_WQ_ATS_DISABLE		0
+#define CONFIG_DEFAULT_WQ_NAME			"user_default_wq"
+#define CONFIG_DEFAULT_WQ_TYPE			"user"
+#define CONFIG_DEFAULT_WQ_MODE			"shared"
+#define CONFIG_DEFAULT_WQ_DRV_NAME		"user"
+
+static void config_default(void *ctx)
+{
+	struct wq_parameters *p;
+	int i;
+
+	/*
+	 * Configure WQ parameters except:
+	 * 1. size and threshold will be configured when enabling WQs.
+	 * 2. max_buffer_size, max_batch_size, op_config will be default values
+	 *    which have been initialized by driver.
+	 */
+	for (i = 0; i < ACCFG_DEVICE_MAX; i++) {
+		p = &conf_def_wq_param[i].param;
+
+		p->priority = CONFIG_DEFAULT_WQ_PRIORITY;
+		p->group_id = CONFIG_DEFAULT_WQ_GROUP_ID;
+		p->block_on_fault = CONFIG_DEFAULT_WQ_BLOCK_ON_FAULT;
+		p->mode = strdup(CONFIG_DEFAULT_WQ_MODE);
+		p->type = strdup(CONFIG_DEFAULT_WQ_TYPE);
+		p->name = strdup(CONFIG_DEFAULT_WQ_NAME);
+		p->driver_name = strdup(CONFIG_DEFAULT_WQ_DRV_NAME);
+		p->prs_disable = CONFIG_DEFAULT_WQ_PRS_DISABLE;
+		p->ats_disable = CONFIG_DEFAULT_WQ_ATS_DISABLE;
+
+		conf_def_wq_param[i].configured = true;
+	}
+}
+
+static void config_default_param_free(void)
+{
+	struct wq_parameters *p;
+	int i;
+
+	for (i = 0; i < ACCFG_DEVICE_MAX; i++) {
+		if (!conf_def_wq_param[i].configured)
+			continue;
+
+		p = &conf_def_wq_param[i].param;
+
+		free((char *)p->name);
+		free((char *)p->type);
+		free((char *)p->mode);
+		free((char *)p->driver_name);
+	}
+}
+
+int cmd_config_default(int argc, const char **argv, void *ctx)
+{
+	const struct option options[] = {
+		OPT_BOOLEAN('v', "verbose", &verbose,
+			    "emit extra debug messages to stderr"),
+		OPT_END(),
+	};
+	const char *const u[] = {
+		"accfg config-default [<options>]", NULL
+	};
+	const char *prefix = "./";
+	int i;
+
+	argc = parse_options_prefix(argc, argv, prefix, options, u, 0);
+	for (i = 0; i < argc; i++)
+		error("unknown parameter \"%s\"\n", argv[i]);
+	if (argc)
+		usage_with_options(u, options);
+
+	config_default(ctx);
+	config_default_activate_devices(ctx);
+
+	config_default_param_free();
+
+	return 0;
+}
accfg/libaccel_config.h +1 -0
index 3feb885..ca51b0a 100644
@@ -36,6 +36,7 @@ enum accfg_device_version {
  enum accfg_device_type {
  	ACCFG_DEVICE_DSA = 0,
  	ACCFG_DEVICE_IAX = 1,
+	ACCFG_DEVICE_MAX = 2,
  	ACCFG_DEVICE_TYPE_UNKNOWN = -1,
  };
 
builtin.h +1 -0
index e1f0b83..96a9d61 100644
@@ -30,6 +30,7 @@ int cmd_config_device(int argc, const char **argv, void *ctx);
  int cmd_config_group(int argc, const char **argv, void *ctx);
  int cmd_config_wq(int argc, const char **argv, void *ctx);
  int cmd_config_engine(int argc, const char **argv, void *ctx);
+int cmd_config_default(int argc, const char **argv, void *ctx);
  #ifdef ENABLE_TEST
  int cmd_test(int argc, const char **argv, void *ctx);
  #endif
author
#4 lore raw

Re: [Accel-config] [PATCH 2/7] accel-config: Add config-user-default command

Hi, Dave,

Quoted 84 lines · Dave Jiang
Quoted 81 lines · Fenghua Yu

Although current accel-config provides thorough ways to configure IDXD devices and WQs, sometimes user needs an easier way to configure and enable them without many knowledges of IDXD.

A new command "config-user-default" is added to configure and enable all available devices and WQs for user usage in the following default configurations:

1. Fixed configurations:
    "mode":"shared",
    "group_id":0,
    "priority":10,
    "block_on_fault":1,
    "name":"user_default_wq",
    "ats_disable":0,
    "prs_disable":1
    "type": "user"
    "driver_name":"user",

2. Calculated configurations:
    "size": max WQ size / max WQs
    "threshold": WQ size

3. Default configurations that have been set by IDXD driver:
    "max_batch_size"
    "max_transfer_size"
    "op_config"

---
  accfg/accel-config.c    |   1 +
  accfg/config.c          | 268 ++++++++++++++++++++++++++++++++++++++++ accfg/libaccel_config.h |   1 +
  builtin.h               |   1 +

  4 files changed, 271 insertions(+)
accfg/accel-config.c +1 -0
index a897608..cf3882e 100644
@@ -67,6 +67,7 @@ static struct cmd_struct commands[] = {
       {"config-group", cmd_config_group},
       {"config-wq", cmd_config_wq},
       {"config-engine", cmd_config_engine},
+    {"config-user-default", cmd_config_default},
   #ifdef ENABLE_TEST
       {"test", cmd_test},
   #endif
accfg/config.c +22 -0
index 2631fd9..f3fa83f 100644
@@ -119,6 +119,87 @@ static bool is_wq_threshold_writable(struct 
 accfg_wq *wq, int val);
   static bool is_wq_prs_disable_writable(struct accfg_wq *wq, int val);
   static bool is_wq_ats_disable_writable(struct accfg_wq *wq, int val);
+static int get_wq_size(struct accfg_device *dev)
+{
+    int max_wq_size, max_wqs;
+
+    max_wq_size = accfg_device_get_max_work_queues_size(dev);
+    max_wqs = accfg_device_get_max_work_queues(dev);
+
+    return max_wq_size / max_wqs;
+}
+
+static int config_default_wq_set_prs_disable(struct accfg_wq *wq, int 
 val)
+{
+    if (!is_wq_prs_disable_writable(wq, val))
+        return -EPERM;
+
+    return accfg_wq_set_prs_disable(wq, val);
+}
+
+static int config_default_wq_set_ats_disable(struct accfg_wq *wq, int 
 val)
+{
+    if (!is_wq_ats_disable_writable(wq, val))
+        return -EPERM;
Quoted 8 lines

accfg/config.c static bool is_wq_threshold_writable(struct accfg_wq *wq, int val); · line 151

Quoted 8 lines
accfg/config.c +8 -0
+
+    return accfg_wq_set_ats_disable(wq, val);
+}
+
+static int config_default_wq_set_threshold(struct accfg_wq *wq, int val)
+{
+    if (!is_wq_threshold_writable(wq, val))
+        return -EPERM;

threshold only be writable on shared group.

Quoted 42 lines
Quoted 42 lines
accfg/config.c +41 -0
+
+    return accfg_wq_set_threshold(wq, val);
+}
+
+static struct conf_def_wq_param {
+    struct wq_parameters param;
+    bool configured;
+} conf_def_wq_param[ACCFG_DEVICE_MAX];
+
+/* Return WQ parameter for dev type. */
+static struct wq_parameters *get_conf_def_wq_param(enum 
 accfg_device_type type)
+{
+    if (type == ACCFG_DEVICE_DSA)
+        return &conf_def_wq_param[ACCFG_DEVICE_DSA].param;
+    else if (type == ACCFG_DEVICE_IAX)
+        return &conf_def_wq_param[ACCFG_DEVICE_IAX].param;
+
+    return NULL;
+}
+
+/* Check if dev is configured. */
+static bool conf_def_dev_configured(struct accfg_device *dev)
+{
+    if (accfg_device_get_type(dev) == ACCFG_DEVICE_DSA)
+        return conf_def_wq_param[ACCFG_DEVICE_DSA].configured;
+    else if (accfg_device_get_type(dev) == ACCFG_DEVICE_IAX)
+        return conf_def_wq_param[ACCFG_DEVICE_IAX].configured;
+
+    return false;
+}
+
+/* Set WQ parameters based on device cap: size and threshold. */
+static int config_default_wq_set_on_dev(struct accfg_device *dev)
+{
+    enum accfg_device_type dev_type;
+    struct wq_parameters *p;
+
+    dev_type = accfg_device_get_type(dev);
+    p = get_conf_def_wq_param(dev_type);
+    if (!p)
+        return -EINVAL;
Quoted 8 lines

accfg/config.c static bool is_wq_threshold_writable(struct accfg_wq *wq, int val); · line 198

Quoted 6 lines
accfg/config.c +6 -0
+
+    p->wq_size = get_wq_size(dev);
+    if (p->wq_size <= 0)
+        return -ENOSPC;
+
+    p->threshold = p->wq_size;

Should check if wq is shared before setting threshold. threshold has no meaning for dwq.

This is just writing an internal parameter. No need to check mode here, right?

The mode check happens during writing threshold via sysfs. Please see this functions:

Quoted 4 lines

accfg/config.c static bool is_wq_threshold_writable(struct accfg_wq *wq, int val); · line 151

Quoted 4 lines
accfg/config.c +4 -0
+static int config_default_wq_set_threshold(struct accfg_wq *wq, int val)
+{
+    if (!is_wq_threshold_writable(wq, val))
+        return -EPERM;

Thanks.

-Fenghua

#5 lore raw

Re: [Accel-config] [PATCH 2/7] accel-config: Add config-user-default command

Quoted 150 lines · Fenghua Yu

Hi, Dave,

Quoted 85 lines · Dave Jiang
Quoted 82 lines · Fenghua Yu

Although current accel-config provides thorough ways to configure IDXD devices and WQs, sometimes user needs an easier way to configure and enable them without many knowledges of IDXD.

A new command "config-user-default" is added to configure and enable all available devices and WQs for user usage in the following default configurations:

1. Fixed configurations:
    "mode":"shared",
    "group_id":0,
    "priority":10,
    "block_on_fault":1,
    "name":"user_default_wq",
    "ats_disable":0,
    "prs_disable":1
    "type": "user"
    "driver_name":"user",

2. Calculated configurations:
    "size": max WQ size / max WQs
    "threshold": WQ size

3. Default configurations that have been set by IDXD driver:
    "max_batch_size"
    "max_transfer_size"
    "op_config"

---
  accfg/accel-config.c    |   1 +
  accfg/config.c          | 268 ++++++++++++++++++++++++++++++++++++++++ accfg/libaccel_config.h |   1 +
  builtin.h               |   1 +

  4 files changed, 271 insertions(+)
accfg/accel-config.c +1 -0
index a897608..cf3882e 100644
@@ -67,6 +67,7 @@ static struct cmd_struct commands[] = {
       {"config-group", cmd_config_group},
       {"config-wq", cmd_config_wq},
       {"config-engine", cmd_config_engine},
+    {"config-user-default", cmd_config_default},
   #ifdef ENABLE_TEST
       {"test", cmd_test},
   #endif
accfg/config.c +23 -0
index 2631fd9..f3fa83f 100644
@@ -119,6 +119,87 @@ static bool is_wq_threshold_writable(struct 
 accfg_wq *wq, int val);
   static bool is_wq_prs_disable_writable(struct accfg_wq *wq, int val);
   static bool is_wq_ats_disable_writable(struct accfg_wq *wq, int val);
+static int get_wq_size(struct accfg_device *dev)
+{
+    int max_wq_size, max_wqs;
+
+    max_wq_size = accfg_device_get_max_work_queues_size(dev);
+    max_wqs = accfg_device_get_max_work_queues(dev);
+
+    return max_wq_size / max_wqs;
+}
+
+static int config_default_wq_set_prs_disable(struct accfg_wq *wq, 
 int val)
+{
+    if (!is_wq_prs_disable_writable(wq, val))
+        return -EPERM;
+
+    return accfg_wq_set_prs_disable(wq, val);
+}
+
+static int config_default_wq_set_ats_disable(struct accfg_wq *wq, 
 int val)
+{
+    if (!is_wq_ats_disable_writable(wq, val))
+        return -EPERM;
+
Quoted 8 lines

accfg/config.c static bool is_wq_threshold_writable(struct accfg_wq *wq, int val); · line 151

Quoted 8 lines
accfg/config.c +7 -0
+    return accfg_wq_set_ats_disable(wq, val);
+}
+
+static int config_default_wq_set_threshold(struct accfg_wq *wq, int 
 val)
+{
+    if (!is_wq_threshold_writable(wq, val))
+        return -EPERM;

threshold only be writable on shared group.

Quoted 42 lines
Quoted 42 lines
accfg/config.c +11 -0
+
+    return accfg_wq_set_threshold(wq, val);
+}
+
+static struct conf_def_wq_param {
+    struct wq_parameters param;
+    bool configured;
+} conf_def_wq_param[ACCFG_DEVICE_MAX];
+
+/* Return WQ parameter for dev type. */
+static struct wq_parameters *get_conf_def_wq_param(enum 

accfg_device_type type)

accfg/config.c +30 -0
+{
+    if (type == ACCFG_DEVICE_DSA)
+        return &conf_def_wq_param[ACCFG_DEVICE_DSA].param;
+    else if (type == ACCFG_DEVICE_IAX)
+        return &conf_def_wq_param[ACCFG_DEVICE_IAX].param;
+
+    return NULL;
+}
+
+/* Check if dev is configured. */
+static bool conf_def_dev_configured(struct accfg_device *dev)
+{
+    if (accfg_device_get_type(dev) == ACCFG_DEVICE_DSA)
+        return conf_def_wq_param[ACCFG_DEVICE_DSA].configured;
+    else if (accfg_device_get_type(dev) == ACCFG_DEVICE_IAX)
+        return conf_def_wq_param[ACCFG_DEVICE_IAX].configured;
+
+    return false;
+}
+
+/* Set WQ parameters based on device cap: size and threshold. */
+static int config_default_wq_set_on_dev(struct accfg_device *dev)
+{
+    enum accfg_device_type dev_type;
+    struct wq_parameters *p;
+
+    dev_type = accfg_device_get_type(dev);
+    p = get_conf_def_wq_param(dev_type);
+    if (!p)
+        return -EINVAL;
Quoted 8 lines

accfg/config.c static bool is_wq_threshold_writable(struct accfg_wq *wq, int val); · line 198

Quoted 6 lines
accfg/config.c +6 -0
+
+    p->wq_size = get_wq_size(dev);
+    if (p->wq_size <= 0)
+        return -ENOSPC;
+
+    p->threshold = p->wq_size;

Should check if wq is shared before setting threshold. threshold has no meaning for dwq.

Quoted 1 line

This is just writing an internal parameter. No need to check mode here, right?

Ok

Quoted 12 lines

The mode check happens during writing threshold via sysfs. Please see this functions:

Quoted 1 line
Quoted 1 line

+static int config_default_wq_set_threshold(struct accfg_wq *wq, int

val)

Quoted 3 lines
Quoted 3 lines

+{
+    if (!is_wq_threshold_writable(wq, val))
+        return -EPERM;

Thanks.

-Fenghua

author
#6 lore raw

PATCH 3/7 accel-config: Add option "-c <config_file>" to load default configurations from the file

In reply to Fenghua Yu

The command "config-user-default" enables all available IDXD devices and WQs with default configurations. Sometime user wants to change the default configurations. A new option "-c <config_file>" allows user to load their own default configurations from the config file (.json file).

The configuration file format and how to edit it will be described shortly.

---
 accfg/config.c | 137 +++++++++++++++++++++++++++++++++++++++++++++++--
 1 file changed, 134 insertions(+), 3 deletions(-)
accfg/config.c +134 -3
index f3fa83f..9ca9f01 100644
@@ -158,6 +158,8 @@ static struct conf_def_wq_param {
 	bool configured;
 } conf_def_wq_param[ACCFG_DEVICE_MAX];
 
+static bool config_default_file;
+
 /* Return WQ parameter for dev type. */
 static struct wq_parameters *get_conf_def_wq_param(enum accfg_device_type type)
 {
@@ -637,6 +639,39 @@ static int activate_devices(void)
 	return 0;
 }
 
+static void config_default_json(struct accfg_wq *wq,
+				json_object *jobj, char *key)
+{
+	enum accfg_device_type dev_type;
+	struct accfg_device *dev;
+	struct wq_parameters *p;
+	char *dev_type_str;
+
+	dev = accfg_wq_get_device(wq);
+	dev_type = accfg_device_get_type(dev);
+	dev_type_str = accfg_device_get_type_str(dev);
+	p = get_conf_def_wq_param(dev_type);
+	if (!p) {
+		fprintf(stderr, "parsing dev type %s failed\n", dev_type_str);
+
+		return;
+	}
+
+	printf("dev type = %s, key = %s\n", dev_type_str, key);
+	if (!strcmp(key, "name"))
+		p->name = strdup(json_object_get_string(jobj));
+	else if (!strcmp(key, "priority"))
+		p->priority = json_object_get_int(jobj);
+	else if (!strcmp(key, "group_id"))
+		p->group_id = json_object_get_int(jobj);
+	else if (!strcmp(key, "block_on_fault"))
+		p->block_on_fault = json_object_get_int(jobj);
+	else if (!strcmp(key, "ats_disable"))
+		p->ats_disable = json_object_get_int(jobj);
+	else if (!strcmp(key, "prs_disable"))
+		p->prs_disable = json_object_get_int(jobj);
+}
+
 /*
  * Configuring the value corresponding to integer and strings
  */
@@ -779,6 +814,12 @@ static int configure_json_value(struct accfg_ctx *ctx,
 		warn_once = false;
 	}
 
+	if (wq && config_default_file) {
+		config_default_json(wq, jobj, key);
+
+		return 0;
+	}
+
 	if (dev && dev_state != ACCFG_DEVICE_ENABLED) {
 		rc = device_json_set_val(dev, jobj, key);
 		if (rc < 0) {
@@ -1430,6 +1471,37 @@ static void config_default_activate_devices(void *ctx)
 #define CONFIG_DEFAULT_WQ_MODE			"shared"
 #define CONFIG_DEFAULT_WQ_DRV_NAME		"user"
 
+/* Set fixed WQ parameters: mode, type, driver_name */
+static int config_default_wq_set_fixed(void)
+{
+	struct wq_parameters *p;
+	int i;
+
+	for (i = 0; i < ACCFG_DEVICE_MAX; i++) {
+		p = &conf_def_wq_param[i].param;
+
+		p->mode = strdup(CONFIG_DEFAULT_WQ_MODE);
+		if (!p->mode) {
+			fprintf(stderr, "strdup WQ mode failed\n");
+			return -ENOMEM;
+		}
+
+		p->type = strdup(CONFIG_DEFAULT_WQ_TYPE);
+		if (!p->type) {
+			fprintf(stderr, "strdup WQ type failed\n");
+			return -ENOMEM;
+		}
+
+		p->driver_name = strdup(CONFIG_DEFAULT_WQ_DRV_NAME);
+		if (!p->driver_name) {
+			fprintf(stderr, "strdup WQ driver_name failed\n");
+			return -ENOMEM;
+		}
+	}
+
+	return 0;
+}
+
 static void config_default(void *ctx)
 {
 	struct wq_parameters *p;
@@ -1476,9 +1548,37 @@ static void config_default_param_free(void)
 	}
 }
 
+static int config_default_from_file(void *ctx)
+{
+	int rc;
+
+	rc = read_config_file(ctx, &config, &util_param);
+	if (rc < 0) {
+		fprintf(stderr, "Reading config file failed: %d\n", rc);
+			return rc;
+	}
+
+	config_default_file = true;
+	rc = parse_config(ctx, &config);
+	if (rc < 0) {
+		fprintf(stderr, "Parse json and set device fail: %d\n", rc);
+		return rc;
+	}
+
+	config_default_wq_set_fixed();
+	if (conf_def_wq_param[ACCFG_DEVICE_DSA].param.name)
+		conf_def_wq_param[ACCFG_DEVICE_DSA].configured = true;
+	if (conf_def_wq_param[ACCFG_DEVICE_IAX].param.name)
+		conf_def_wq_param[ACCFG_DEVICE_IAX].configured = true;
+
+	return 0;
+}
+
 int cmd_config_default(int argc, const char **argv, void *ctx)
 {
 	const struct option options[] = {
+		OPT_FILENAME('c', "config-file", &config.config_file, "config-file",
+			     "override the default config"),
 		OPT_BOOLEAN('v', "verbose", &verbose,
 			    "emit extra debug messages to stderr"),
 		OPT_END(),
@@ -1486,8 +1586,14 @@ int cmd_config_default(int argc, const char **argv, void *ctx)
 	const char *const u[] = {
 		"accfg config-default [<options>]", NULL
 	};
+	struct util_filter_ctx fctx = {
+		0
+	};
+	struct list_filter_arg cfa = {
+		0
+	};
 	const char *prefix = "./";
-	int i;
+	int i, rc = 0;
 
 	argc = parse_options_prefix(argc, argv, prefix, options, u, 0);
 	for (i = 0; i < argc; i++)
@@ -1495,8 +1601,33 @@ int cmd_config_default(int argc, const char **argv, void *ctx)
 	if (argc)
 		usage_with_options(u, options);
 
-	config_default(ctx);
-	config_default_activate_devices(ctx);
+	cfa.jdevices = json_object_new_array();
+	if (!cfa.jdevices)
+		return -ENOMEM;
+	list_head_init(&cfa.jdev_list);
+
+	fctx.filter_device = filter_device;
+	fctx.filter_group = filter_group;
+	fctx.filter_wq = filter_wq;
+	fctx.filter_engine = filter_engine;
+	fctx.list = &cfa;
+	cfa.flags = config_opts_to_flags();
+
+	rc = util_filter_walk(ctx, &fctx, &util_param);
+	if (rc)
+		return rc;
+
+	free_containers(&cfa);
+
+	if (config.config_file) {
+		/* Parse the default config file and set configs. */
+		rc = config_default_from_file(ctx);
+	} else {
+		config_default(ctx);
+	}
+
+	if (!rc)
+		config_default_activate_devices(ctx);
 
 	config_default_param_free();
 
Signature · 1 line
2.31.1
author
#7 lore raw

PATCH 4/7 accel-config: Disable default configured WQs and devices

In reply to Fenghua Yu

To disable default configured devices and WQs, a new option "-d" is added. It disables all WQs named as "user-wq" and their devices.

---
 accfg/config.c | 60 ++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 60 insertions(+)
accfg/config.c +60 -0
index 9ca9f01..771aa6c 100644
@@ -1574,11 +1574,65 @@ static int config_default_from_file(void *ctx)
 	return 0;
 }
 
+void config_default_disable(void *ctx)
+{
+	char *user_default_wq_name;
+	struct accfg_device *dev;
+
+	user_default_wq_name = strdup(CONFIG_DEFAULT_WQ_NAME);
+	if (!user_default_wq_name) {
+		fprintf(stderr, "strdup user default wq name failed\n");
+		return;
+	}
+
+	accfg_device_foreach(ctx, dev) {
+		enum accfg_device_state dev_state;
+		enum accfg_wq_state wq_state;
+		bool non_default_wq_enabled;
+		struct accfg_wq *wq;
+		const char *wq_name;
+
+		non_default_wq_enabled = false;
+		/* Disable enabled default WQs */
+		accfg_wq_foreach(dev, wq) {
+			wq_name = accfg_wq_get_type_name(wq);
+			wq_state = accfg_wq_get_state(wq);
+			if (wq_state == ACCFG_WQ_DISABLED)
+				continue;
+
+			if (!strcmp(wq_name, user_default_wq_name)) {
+				if (verbose) {
+					printf("disable %s\n",
+					       accfg_wq_get_devname(wq));
+				}
+				accfg_wq_disable(wq, true);
+			} else {
+				non_default_wq_enabled = true;
+			}
+		}
+
+		/* Disable enabled device only when all WQs are disabled. */
+		dev_state = accfg_device_get_state(dev);
+		if (dev_state == ACCFG_DEVICE_ENABLED &&
+		    !non_default_wq_enabled) {
+			if (verbose) {
+				printf("enable %s\n",
+				       accfg_device_get_devname(dev));
+			}
+			accfg_device_disable(dev, true);
+		}
+	}
+	free(user_default_wq_name);
+}
+
 int cmd_config_default(int argc, const char **argv, void *ctx)
 {
+	bool disable = false;
 	const struct option options[] = {
 		OPT_FILENAME('c', "config-file", &config.config_file, "config-file",
 			     "override the default config"),
+		OPT_BOOLEAN('d', "disable", &disable,
+			    "disable configured default devices and wqs"),
 		OPT_BOOLEAN('v', "verbose", &verbose,
 			    "emit extra debug messages to stderr"),
 		OPT_END(),
@@ -1619,6 +1673,12 @@ int cmd_config_default(int argc, const char **argv, void *ctx)
 
 	free_containers(&cfa);
 
+	if (disable) {
+		config_default_disable(ctx);
+
+		return 0;
+	}
+
 	if (config.config_file) {
 		/* Parse the default config file and set configs. */
 		rc = config_default_from_file(ctx);
Signature · 1 line
2.31.1
author
#8 lore raw

PATCH 5/7 accel-config: Add "-n <wq_name>" to specify WQ name for disabling WQs

In reply to Fenghua Yu

Default configured WQs are named as "user-wq" by default and can be re-named in default config file. To disable the default configured WQs that are re-named, add a new option "-n <wq_name>" to specify WQ name for disabling WQs.

---
 accfg/config.c | 10 +++++++++-
 1 file changed, 9 insertions(+), 1 deletion(-)
accfg/config.c +9 -1
index 771aa6c..82b71a8 100644
@@ -39,6 +39,7 @@ static struct config {
 	bool engines;
 	bool wqs;
 	const char *config_file;
+	const char *user_default_wq_name;
 	char *buf;
 } config;
 
@@ -1579,12 +1580,17 @@ void config_default_disable(void *ctx)
 	char *user_default_wq_name;
 	struct accfg_device *dev;
 
-	user_default_wq_name = strdup(CONFIG_DEFAULT_WQ_NAME);
+	if (config.user_default_wq_name)
+		user_default_wq_name = strdup(config.user_default_wq_name);
+	else
+		user_default_wq_name = strdup(CONFIG_DEFAULT_WQ_NAME);
 	if (!user_default_wq_name) {
 		fprintf(stderr, "strdup user default wq name failed\n");
 		return;
 	}
 
+	printf("disable WQs named as %s\n", user_default_wq_name);
+
 	accfg_device_foreach(ctx, dev) {
 		enum accfg_device_state dev_state;
 		enum accfg_wq_state wq_state;
@@ -1633,6 +1639,8 @@ int cmd_config_default(int argc, const char **argv, void *ctx)
 			     "override the default config"),
 		OPT_BOOLEAN('d', "disable", &disable,
 			    "disable configured default devices and wqs"),
+		OPT_STRING('n', "name", &config.user_default_wq_name, "user default wq name",
+			   "specify user default wq name. Default \"user_default_wq\""),
 		OPT_BOOLEAN('v', "verbose", &verbose,
 			    "emit extra debug messages to stderr"),
 		OPT_END(),
Signature · 1 line
2.31.1
author
#9 lore raw

PATCH 6/7 accel-config: Add user_default_profile.conf

In reply to Fenghua Yu

Add contrib/configs/user_default_profile.conf as a user default configuration template.

It specifies one DSA and one IAX with all WQs and engines in one group in each device.

Run "accel-config config-user-default -c user_default_profile.conf" to apply the attributes to all available WQs and engines in each device.

User can modify the attributes and/or remove a device per their requirements.

---
 contrib/configs/profilenote.txt           | 28 +++++++++++++++
 contrib/configs/user_default_profile.conf | 42 +++++++++++++++++++++++
 2 files changed, 70 insertions(+)
 create mode 100644 contrib/configs/user_default_profile.conf
contrib/configs/profilenote.txt +28 -0
index 6debc9a..a48fed0 100644
@@ -34,3 +34,31 @@ profile descriptions.
               for throughput ops
             - targeting SPDK-like uses
 	    - DWQ threshold default 0
+
+5. user_default_profile.conf
+    Recommended dsa and iax accel profile template
+    - The template specifies one DSA device and one IAX device
+    - The attributes are applied to all WQs and engines on DSA and IAX
+    - Run "accel-config config-user-default -c user_default_profile.conf"
+      to load the config to all WQs and engines on all available DSA and
+      IAX devices
+    - The attributes can be edited. Two devices can be tailored to one device.
+    - 1 group with all available WQs/all engs per group
+            - Attributes that can be changed:
+              "group_id":0
+              "priority":10
+              "block_on_fault":1
+              "ats_disable":0
+              "prs_disable":1
+              "name":"user_default_wq"
+            - Calcuated attributes:
+              "size": max WQ size / max WQs
+              "threshold": WQ size
+            - Fixed attributes:
+              "type": "user"
+              "mode":"shared"
+              "driver_name":"user",
+            - Attributes with default values set by driver:
+              "max_batch_size": default value
+              "max_transfer_size": default value
+              "op_config": default value
contrib/configs/user_default_profile.conf new file +42 -0

42 lines not on this page · raw message

Signature · 1 line
2.31.1
author
#10 lore raw

PATCH 7/7 accel-config: Add doumentation for new command "config-user-default"

In reply to Fenghua Yu

The new command "config-user-default" and its options are documented.

---
 Documentation/accfg/Makefile.am               |  6 +-
 .../accel-config-config-user-default.txt      | 75 +++++++++++++++++++
 2 files changed, 79 insertions(+), 2 deletions(-)
 create mode 100644 Documentation/accfg/accel-config-config-user-default.txt
Documentation/accfg/Makefile.am +4 -2
index 2e938ed..ed2b288 100644
@@ -29,7 +29,8 @@ man1_MANS = \
 	accel-config-disable-device.1 \
 	accel-config-disable-wq.1 \
 	accel-config-enable-wq.1 \
-	accel-config-enable-device.1
+	accel-config-enable-device.1 \
+	accel-config-config-user-default.1
 
 EXTRA_DIST = \
 	$(man1_MANS) \
@@ -44,7 +45,8 @@ EXTRA_DIST = \
 	accel-config-disable-device.txt \
 	accel-config-disable-wq.txt \
 	accel-config-enable-wq.txt \
-	accel-config-enable-device.txt
+	accel-config-enable-device.txt \
+	accel-config-config-user-default.txt
 
 CLEANFILES = $(man1_MANS)
 
Documentation/accfg/accel-config-config-user-default.txt new file +75 -0

75 lines not on this page · raw message

Signature · 1 line
2.31.1