Skip to contentSkip to conversation

bidirect guest channel

PATCH Superseded Hajkowski 4 participants 21 messages Apr 5, 2019 – Oct 27, 2019

lore
Share & tools
lore

Hajkowski

author
#0 lore raw

v5 cover bidirect guest channel

From: Marcin Hajkowski <[email protected]>

Extend guest channel API to allow bidirectional communication. Modify power manager host and guest side to communicate in both directions.

---

v5:
* enhance logging

v4:
* [vm_power_manager] treat 0 as valid socket id
* [guest_manager] use user level logs
* correct code formatting

v3:
* fix global_fds[lcore_id] comparison to invalid value
* check 0 to verify if read function actually read some data
* define _NACK cmd instead of _NAK
* simplify rte_power_guest_channel_receive_msg func logic

v2:
* send ack only if power operation return positive value
* log diffent error for unexpected incoming command and error during ack/nak cmd sending

Marcin Hajkowski (4):
power: fix invalid socket indicator value
power: extend guest channel API for reading
power: process incoming confirmation cmds
power: send confirmation cmd to vm guest

 examples/vm_power_manager/channel_monitor.c   | 68 +++++++++++++++--
 examples/vm_power_manager/guest_cli/Makefile  |  1 +
 .../guest_cli/vm_power_cli_guest.c            | 73 +++++++++++++++----
 lib/librte_power/channel_commands.h           |  5 ++
 lib/librte_power/guest_channel.c              | 72 ++++++++++++++++--
 lib/librte_power/guest_channel.h              | 35 +++++++++
 lib/librte_power/rte_power_version.map        |  1 +
 7 files changed, 230 insertions(+), 25 deletions(-)
Signature · 1 line
2.17.2
author
#1 lore raw

PATCH v5 1/4 power: fix invalid socket indicator value

From: Marcin Hajkowski <[email protected]>

Currently 0 is being used for not connected slot indication. This is not consistent with linux doc which identifies 0 as valid (connected) slot, thus modification was done to change it.

Fixes cd0d5547 ("power: vm communication channels in guest")
---
 lib/librte_power/guest_channel.c | 12 ++++++------
 1 file changed, 6 insertions(+), 6 deletions(-)
lib/librte_power/guest_channel.c +6 -6
index c17ea46b4..9cf7d2cb2 100644
@@ -19,7 +19,7 @@
 
 #define RTE_LOGTYPE_GUEST_CHANNEL RTE_LOGTYPE_USER1
 
-static int global_fds[RTE_MAX_LCORE];
+static int global_fds[RTE_MAX_LCORE] = { [0 ... RTE_MAX_LCORE-1] = -1 };
 
 int
 guest_channel_host_connect(const char *path, unsigned int lcore_id)
@@ -35,7 +35,7 @@ guest_channel_host_connect(const char *path, unsigned int lcore_id)
 		return -1;
 	}
 	/* check if path is already open */
-	if (global_fds[lcore_id] != 0) {
+	if (global_fds[lcore_id] != -1) {
 		RTE_LOG(ERR, GUEST_CHANNEL, "Channel(%u) is already open with fd %d\n",
 				lcore_id, global_fds[lcore_id]);
 		return -1;
@@ -84,7 +84,7 @@ guest_channel_host_connect(const char *path, unsigned int lcore_id)
 	return 0;
 error:
 	close(fd);
-	global_fds[lcore_id] = 0;
+	global_fds[lcore_id] = -1;
 	return -1;
 }
 
@@ -100,7 +100,7 @@ guest_channel_send_msg(struct channel_packet *pkt, unsigned int lcore_id)
 		return -1;
 	}
 
-	if (global_fds[lcore_id] == 0) {
+	if (global_fds[lcore_id] < 0) {
 		RTE_LOG(ERR, GUEST_CHANNEL, "Channel is not connected\n");
 		return -1;
 	}
@@ -134,8 +134,8 @@ guest_channel_host_disconnect(unsigned int lcore_id)
 				lcore_id, RTE_MAX_LCORE-1);
 		return;
 	}
-	if (global_fds[lcore_id] == 0)
+	if (global_fds[lcore_id] < 0)
 		return;
 	close(global_fds[lcore_id]);
-	global_fds[lcore_id] = 0;
+	global_fds[lcore_id] = -1;
 }
Signature · 1 line
2.17.2
#2 lore raw

v6 cover bidirectional guest channel

Extend guest channel API to allow bidirectional communication. Modify power manager host and guest side to communicate in both directions.

---

v6:
* put map file api's in alphabetical order
* fix checkpatch issue around experimental tag

v5:
* enhance logging

v4:
* [vm_power_manager] treat 0 as valid socket id
* [guest_manager] use user level logs
* correct code formatting

v3:
* fix global_fds[lcore_id] comparison to invalid value
* check 0 to verify if read function actually read some data
* define _NACK cmd instead of _NAK
* simplify rte_power_guest_channel_receive_msg func logic

v2:
* send ack only if power operation return positive value
* log diffent error for unexpected incoming command and error during ack/nak cmd sending

Marcin Hajkowski (4):
[PATCH v6 1/4] power: fix invalid socket indicator value [PATCH v6 2/4] power: extend guest channel API for reading [PATCH v6 3/4] power: process incoming confirmation cmds [PATCH v6 4/4] power: send confirmation cmd to vm guest

#3 lore raw

PATCH v6 1/4 power: fix invalid socket indicator value

Currently 0 is being used for not connected slot indication. This is not consistent with linux doc which identifies 0 as valid (connected) slot, thus modification was done to change it.

Fixes cd0d5547 ("power: vm communication channels in guest")
---
 lib/librte_power/guest_channel.c | 12 ++++++------
 1 file changed, 6 insertions(+), 6 deletions(-)
lib/librte_power/guest_channel.c +6 -6
index c17ea46b4..9cf7d2cb2 100644
@@ -19,7 +19,7 @@
 
 #define RTE_LOGTYPE_GUEST_CHANNEL RTE_LOGTYPE_USER1
 
-static int global_fds[RTE_MAX_LCORE];
+static int global_fds[RTE_MAX_LCORE] = { [0 ... RTE_MAX_LCORE-1] = -1 };
 
 int
 guest_channel_host_connect(const char *path, unsigned int lcore_id)
@@ -35,7 +35,7 @@ guest_channel_host_connect(const char *path, unsigned int lcore_id)
 		return -1;
 	}
 	/* check if path is already open */
-	if (global_fds[lcore_id] != 0) {
+	if (global_fds[lcore_id] != -1) {
 		RTE_LOG(ERR, GUEST_CHANNEL, "Channel(%u) is already open with fd %d\n",
 				lcore_id, global_fds[lcore_id]);
 		return -1;
@@ -84,7 +84,7 @@ guest_channel_host_connect(const char *path, unsigned int lcore_id)
 	return 0;
 error:
 	close(fd);
-	global_fds[lcore_id] = 0;
+	global_fds[lcore_id] = -1;
 	return -1;
 }
 
@@ -100,7 +100,7 @@ guest_channel_send_msg(struct channel_packet *pkt, unsigned int lcore_id)
 		return -1;
 	}
 
-	if (global_fds[lcore_id] == 0) {
+	if (global_fds[lcore_id] < 0) {
 		RTE_LOG(ERR, GUEST_CHANNEL, "Channel is not connected\n");
 		return -1;
 	}
@@ -134,8 +134,8 @@ guest_channel_host_disconnect(unsigned int lcore_id)
 				lcore_id, RTE_MAX_LCORE-1);
 		return;
 	}
-	if (global_fds[lcore_id] == 0)
+	if (global_fds[lcore_id] < 0)
 		return;
 	close(global_fds[lcore_id]);
-	global_fds[lcore_id] = 0;
+	global_fds[lcore_id] = -1;
 }
Signature · 1 line
2.17.1
#4 lore raw

PATCH v6 2/4 power: extend guest channel API for reading

In reply to Marcin Hajkowski

Added new experimental API rte_power_guest_channel_receive_msg which gives possibility to receive messages send to guest.

---
 lib/librte_power/channel_commands.h    |  5 +++
 lib/librte_power/guest_channel.c       | 60 ++++++++++++++++++++++++++
 lib/librte_power/guest_channel.h       | 36 ++++++++++++++++
 lib/librte_power/rte_power_version.map |  1 +
 4 files changed, 102 insertions(+)
lib/librte_power/channel_commands.h +5 -0
index eca8ff70c..d81216b42 100644
@@ -25,6 +25,11 @@ extern "C" {
 #define CPU_POWER_SCALE_MIN     4
 #define CPU_POWER_ENABLE_TURBO  5
 #define CPU_POWER_DISABLE_TURBO 6
+
+/* Generic Power Command Response */
+#define CPU_POWER_CMD_ACK       1
+#define CPU_POWER_CMD_NACK      2
+
 #define HOURS 24
 
 #define MAX_VFS 10
lib/librte_power/guest_channel.c +60 -0

78 lines not on this page · raw message

lib/librte_power/guest_channel.h +36 -0
index 373d39898..61e142289 100644
@@ -68,6 +68,42 @@ int guest_channel_send_msg(struct channel_packet *pkt, unsigned int lcore_id);
 int rte_power_guest_channel_send_msg(struct channel_packet *pkt,
 			unsigned int lcore_id);
 
+/**
+ * Read a message contained in pkt over the Virtio-Serial
+ * from the host endpoint.
+ *
+ * @param pkt
+ *  Pointer to a populated struct channel_packet
+ *
+ * @param lcore_id
+ *  lcore_id.
+ *
+ * @return
+ *  - 0 on success.
+ *  - Negative on error.
+ */
+int power_guest_channel_read_msg(struct channel_packet *pkt,
+		unsigned int lcore_id);
+
+/**
+ * Receive a message contained in pkt over the Virtio-Serial
+ * from the host endpoint.
+ *
+ * @param pkt
+ *  Pointer to a populated struct channel_packet
+ *
+ * @param lcore_id
+ *  lcore_id.
+ *
+ * @return
+ *  - 0 on success.
+ *  - Negative on error.
+ */
+__rte_experimental
+int
+rte_power_guest_channel_receive_msg(struct channel_packet *pkt,
+			unsigned int lcore_id);
+
 #ifdef __cplusplus
 }
 #endif
lib/librte_power/rte_power_version.map +1 -0
index 042917360..772983813 100644
@@ -42,6 +42,7 @@ EXPERIMENTAL {
 	rte_power_empty_poll_stat_free;
 	rte_power_empty_poll_stat_init;
 	rte_power_empty_poll_stat_update;
+	rte_power_guest_channel_receive_msg;
 	rte_power_poll_stat_fetch;
 	rte_power_poll_stat_update;
 };
Signature · 1 line
2.17.1
#5 lore raw

PATCH v6 3/4 power: process incoming confirmation cmds

In reply to Marcin Hajkowski

Extend vm_power_guest to check incoming confirmations of messages previously sent to host.

---
 examples/vm_power_manager/guest_cli/Makefile  |  1 +
 .../guest_cli/vm_power_cli_guest.c            | 73 +++++++++++++++----
 2 files changed, 61 insertions(+), 13 deletions(-)
examples/vm_power_manager/guest_cli/Makefile +1 -0
index e35a68d0f..67cf08193 100644
@@ -18,6 +18,7 @@ SRCS-y := main.c vm_power_cli_guest.c parse.c
 
 CFLAGS += -O3 -I$(RTE_SDK)/lib/librte_power/
 CFLAGS += $(WERROR_FLAGS)
+CFLAGS += -DALLOW_EXPERIMENTAL_API
 
 # workaround for a gcc bug with noreturn attribute
 # http://gcc.gnu.org/bugzilla/show_bug.cgi?id=12603
examples/vm_power_manager/guest_cli/vm_power_cli_guest.c +60 -13

116 lines not on this page · raw message

Signature · 1 line
2.17.1
#6 lore raw

PATCH v6 4/4 power: send confirmation cmd to vm guest

In reply to Marcin Hajkowski

Use new guest channel API to send confirmation
message for received power command.

---
 examples/vm_power_manager/channel_monitor.c | 68 +++++++++++++++++++--
 1 file changed, 62 insertions(+), 6 deletions(-)
examples/vm_power_manager/channel_monitor.c +62 -6
index fe6088a18..0c73fac55 100644
@@ -671,6 +671,41 @@ apply_policy(struct policy *pol)
 		apply_workload_profile(pol);
 }
 
+static int
+write_binary_packet(struct channel_packet *pkt, struct channel_info *chan_info)
+{
+	int ret, buffer_len = sizeof(*pkt);
+	void *buffer = pkt;
+
+	if (chan_info->fd < 0) {
+		RTE_LOG(ERR, CHANNEL_MONITOR, "Channel is not connected\n");
+		return -1;
+	}
+
+	while (buffer_len > 0) {
+		ret = write(chan_info->fd, buffer, buffer_len);
+		if (ret == -1) {
+			if (errno == EINTR)
+				continue;
+			RTE_LOG(ERR, CHANNEL_MONITOR, "Write function failed due to %s.\n",
+					strerror(errno));
+			return -1;
+		}
+		buffer = (char *)buffer + ret;
+		buffer_len -= ret;
+	}
+	return 0;
+}
+
+static int
+send_ack_for_received_cmd(struct channel_packet *pkt,
+		struct channel_info *chan_info,
+		uint32_t command)
+{
+	pkt->command = command;
+	return write_binary_packet(pkt, chan_info);
+}
+
 static int
 process_request(struct channel_packet *pkt, struct channel_info *chan_info)
 {
@@ -694,33 +729,54 @@ process_request(struct channel_packet *pkt, struct channel_info *chan_info)
 		RTE_LOG(DEBUG, CHANNEL_MONITOR, "Processing requested cmd for cpu:%d\n",
 			core_num);
 
+		bool valid_unit = true;
+		int scale_res;
+
 		switch (pkt->unit) {
 		case(CPU_POWER_SCALE_MIN):
-			power_manager_scale_core_min(core_num);
+			scale_res = power_manager_scale_core_min(core_num);
 			break;
 		case(CPU_POWER_SCALE_MAX):
-			power_manager_scale_core_max(core_num);
+			scale_res = power_manager_scale_core_max(core_num);
 			break;
 		case(CPU_POWER_SCALE_DOWN):
-			power_manager_scale_core_down(core_num);
+			scale_res = power_manager_scale_core_down(core_num);
 			break;
 		case(CPU_POWER_SCALE_UP):
-			power_manager_scale_core_up(core_num);
+			scale_res = power_manager_scale_core_up(core_num);
 			break;
 		case(CPU_POWER_ENABLE_TURBO):
-			power_manager_enable_turbo_core(core_num);
+			scale_res = power_manager_enable_turbo_core(core_num);
 			break;
 		case(CPU_POWER_DISABLE_TURBO):
-			power_manager_disable_turbo_core(core_num);
+			scale_res = power_manager_disable_turbo_core(core_num);
 			break;
 		default:
+			valid_unit = false;
 			break;
 		}
+
+		if (valid_unit) {
+			ret = send_ack_for_received_cmd(pkt,
+					chan_info,
+					scale_res > 0 ?
+						CPU_POWER_CMD_ACK :
+						CPU_POWER_CMD_NACK);
+			if (ret < 0)
+				RTE_LOG(DEBUG, CHANNEL_MONITOR, "Error during sending ack command.\n");
+		} else
+			RTE_LOG(DEBUG, CHANNEL_MONITOR, "Unexpected unit type.\n");
+
 	}
 
 	if (pkt->command == PKT_POLICY) {
 		RTE_LOG(INFO, CHANNEL_MONITOR, "Processing policy request %s\n",
 				pkt->vm_name);
+		int ret = send_ack_for_received_cmd(pkt,
+				chan_info,
+				CPU_POWER_CMD_ACK);
+		if (ret < 0)
+			RTE_LOG(DEBUG, CHANNEL_MONITOR, "Error during sending ack command.\n");
 		update_policy(pkt);
 		policy_is_set = 1;
 	}
Signature · 1 line
2.17.1
#7 lore raw

Re: [dpdk-dev] [PATCH v6 0/4] bidirectional guest channel

In reply to Marcin Hajkowski

27/09/2019 10:42, Marcin Hajkowski:

Quoted 5 lines

Marcin Hajkowski (4):
[PATCH v6 1/4] power: fix invalid socket indicator value [PATCH v6 2/4] power: extend guest channel API for reading [PATCH v6 3/4] power: process incoming confirmation cmds [PATCH v6 4/4] power: send confirmation cmd to vm guest

Applied, thanks

author
#8 lore raw

PATCH v5 2/4 power: extend guest channel API for reading

In reply to Hajkowski

From: Marcin Hajkowski <[email protected]>

Added new experimental API rte_power_guest_channel_receive_msg which gives possibility to receive messages send to guest.

---
 lib/librte_power/channel_commands.h    |  5 +++
 lib/librte_power/guest_channel.c       | 60 ++++++++++++++++++++++++++
 lib/librte_power/guest_channel.h       | 35 +++++++++++++++
 lib/librte_power/rte_power_version.map |  1 +
 4 files changed, 101 insertions(+)
lib/librte_power/channel_commands.h +5 -0
index e7b93a797..33fd53a6d 100644
@@ -28,6 +28,11 @@ extern "C" {
 #define CPU_POWER_SCALE_MIN     4
 #define CPU_POWER_ENABLE_TURBO  5
 #define CPU_POWER_DISABLE_TURBO 6
+
+/* Generic Power Command Response */
+#define CPU_POWER_CMD_ACK       1
+#define CPU_POWER_CMD_NACK      2
+
 #define HOURS 24
 
 #define MAX_VFS 10
lib/librte_power/guest_channel.c +60 -0

78 lines not on this page · raw message

lib/librte_power/guest_channel.h +35 -0
index 373d39898..7c385df39 100644
@@ -68,6 +68,41 @@ int guest_channel_send_msg(struct channel_packet *pkt, unsigned int lcore_id);
 int rte_power_guest_channel_send_msg(struct channel_packet *pkt,
 			unsigned int lcore_id);
 
+/**
+ * Read a message contained in pkt over the Virtio-Serial
+ * from the host endpoint.
+ *
+ * @param pkt
+ *  Pointer to a populated struct channel_packet
+ *
+ * @param lcore_id
+ *  lcore_id.
+ *
+ * @return
+ *  - 0 on success.
+ *  - Negative on error.
+ */
+int power_guest_channel_read_msg(struct channel_packet *pkt,
+		unsigned int lcore_id);
+
+/**
+ * Receive a message contained in pkt over the Virtio-Serial
+ * from the host endpoint.
+ *
+ * @param pkt
+ *  Pointer to a populated struct channel_packet
+ *
+ * @param lcore_id
+ *  lcore_id.
+ *
+ * @return
+ *  - 0 on success.
+ *  - Negative on error.
+ */
+int __rte_experimental
+rte_power_guest_channel_receive_msg(struct channel_packet *pkt,
+			unsigned int lcore_id);
+
 #ifdef __cplusplus
 }
 #endif
lib/librte_power/rte_power_version.map +1 -0
index 042917360..69f5ea3f4 100644
@@ -44,4 +44,5 @@ EXPERIMENTAL {
 	rte_power_empty_poll_stat_update;
 	rte_power_poll_stat_fetch;
 	rte_power_poll_stat_update;
+	rte_power_guest_channel_receive_msg;
 };
Signature · 1 line
2.17.2
Tested-by on v5 2/4
#9 lore raw

Re: [dpdk-dev] [PATCH v5 2/4] power: extend guest channel API for reading

Quoted 165 lines · Hajkowski

From: Marcin Hajkowski <[email protected]>

Added new experimental API rte_power_guest_channel_receive_msg which gives possibility to receive messages send to guest.

---
  lib/librte_power/channel_commands.h    |  5 +++
  lib/librte_power/guest_channel.c       | 60 ++++++++++++++++++++++++++
  lib/librte_power/guest_channel.h       | 35 +++++++++++++++
  lib/librte_power/rte_power_version.map |  1 +
  4 files changed, 101 insertions(+)
lib/librte_power/channel_commands.h +5 -0
index e7b93a797..33fd53a6d 100644
@@ -28,6 +28,11 @@ extern "C" {
  #define CPU_POWER_SCALE_MIN     4
  #define CPU_POWER_ENABLE_TURBO  5
  #define CPU_POWER_DISABLE_TURBO 6
+
+/* Generic Power Command Response */
+#define CPU_POWER_CMD_ACK       1
+#define CPU_POWER_CMD_NACK      2
+
  #define HOURS 24
 
  #define MAX_VFS 10
lib/librte_power/guest_channel.c +60 -0
index 9cf7d2cb2..888423891 100644
@@ -10,6 +10,7 @@
  #include <fcntl.h>
  #include <string.h>
  #include <errno.h>
+#include <poll.h>
 
 
  #include <rte_log.h>
@@ -19,6 +20,9 @@
 
  #define RTE_LOGTYPE_GUEST_CHANNEL RTE_LOGTYPE_USER1
 
+/* Timeout for incoming message in milliseconds. */
+#define TIMEOUT 10
+
  static int global_fds[RTE_MAX_LCORE] = { [0 ... RTE_MAX_LCORE-1] = -1 };
 
  int
@@ -125,6 +129,62 @@ int rte_power_guest_channel_send_msg(struct channel_packet *pkt,
  	return guest_channel_send_msg(pkt, lcore_id);
  }
 
+int power_guest_channel_read_msg(struct channel_packet *pkt,
+			unsigned int lcore_id)
+{
+	int ret;
+	struct pollfd fds;
+	void *buffer = pkt;
+	int buffer_len = sizeof(*pkt);
+
+	fds.fd = global_fds[lcore_id];
+	fds.events = POLLIN;
+
+	ret = poll(&fds, 1, TIMEOUT);
+	if (ret == 0) {
+		RTE_LOG(DEBUG, GUEST_CHANNEL, "Timeout occurred during poll function.\n");
+		return -1;
+	} else if (ret < 0) {
+		RTE_LOG(ERR, GUEST_CHANNEL, "Error occurred during poll function: %s\n",
+				strerror(ret));
+		return -1;
+	}
+
+	if (lcore_id >= RTE_MAX_LCORE) {
+		RTE_LOG(ERR, GUEST_CHANNEL, "Channel(%u) is out of range 0...%d\n",
+				lcore_id, RTE_MAX_LCORE-1);
+		return -1;
+	}
+
+	if (global_fds[lcore_id] < 0) {
+		RTE_LOG(ERR, GUEST_CHANNEL, "Channel is not connected\n");
+		return -1;
+	}
+
+	while (buffer_len > 0) {
+		ret = read(global_fds[lcore_id],
+				buffer, buffer_len);
+		if (ret < 0) {
+			if (errno == EINTR)
+				continue;
+			return -1;
+		}
+		if (ret == 0) {
+			RTE_LOG(ERR, GUEST_CHANNEL, "Expected more data, but connection has been closed.\n");
+			return -1;
+		}
+		buffer = (char *)buffer + ret;
+		buffer_len -= ret;
+	}
+
+	return 0;
+}
+
+int rte_power_guest_channel_receive_msg(struct channel_packet *pkt,
+			unsigned int lcore_id)
+{
+	return power_guest_channel_read_msg(pkt, lcore_id);
+}
 
  void
  guest_channel_host_disconnect(unsigned int lcore_id)
lib/librte_power/guest_channel.h +35 -0
index 373d39898..7c385df39 100644
@@ -68,6 +68,41 @@ int guest_channel_send_msg(struct channel_packet *pkt, unsigned int lcore_id);
  int rte_power_guest_channel_send_msg(struct channel_packet *pkt,
  			unsigned int lcore_id);
 
+/**
+ * Read a message contained in pkt over the Virtio-Serial
+ * from the host endpoint.
+ *
+ * @param pkt
+ *  Pointer to a populated struct channel_packet
+ *
+ * @param lcore_id
+ *  lcore_id.
+ *
+ * @return
+ *  - 0 on success.
+ *  - Negative on error.
+ */
+int power_guest_channel_read_msg(struct channel_packet *pkt,
+		unsigned int lcore_id);
+
+/**
+ * Receive a message contained in pkt over the Virtio-Serial
+ * from the host endpoint.
+ *
+ * @param pkt
+ *  Pointer to a populated struct channel_packet
+ *
+ * @param lcore_id
+ *  lcore_id.
+ *
+ * @return
+ *  - 0 on success.
+ *  - Negative on error.
+ */
+int __rte_experimental
+rte_power_guest_channel_receive_msg(struct channel_packet *pkt,
+			unsigned int lcore_id);
+
  #ifdef __cplusplus
  }
  #endif
lib/librte_power/rte_power_version.map +0 -0
index 042917360..69f5ea3f4 100644

lib/librte_power/rte_power_version.map EXPERIMENTAL { · line 48

Quoted 5 lines
lib/librte_power/rte_power_version.map +1 -0
@@ -44,4 +44,5 @@ EXPERIMENTAL {
  	rte_power_empty_poll_stat_update;
  	rte_power_poll_stat_fetch;
  	rte_power_poll_stat_update;
+	rte_power_guest_channel_receive_msg;
  };

I tested this with the later patches in the set, and the acknowledges successfully get back to the guest from the host in the form of an Ack or Nack for various commands sent from the guest.

Acked-by
#10 lore raw

Re: [dpdk-dev] [PATCH v5 2/4] power: extend guest channel API for reading

Quoted 16 lines · Hajkowski
Quoted 15 lines

From: Marcin Hajkowski <[email protected]>

Added new experimental API rte_power_guest_channel_receive_msg which gives possibility to receive messages send to guest.

---
  lib/librte_power/channel_commands.h    |  5 +++
  lib/librte_power/guest_channel.c       | 60 ++++++++++++++++++++++++++
  lib/librte_power/guest_channel.h       | 35 +++++++++++++++
  lib/librte_power/rte_power_version.map |  1 +
  4 files changed, 101 insertions(+)
lib/librte_power/channel_commands.h +0 -0

<...>

Quoted 11 lines

lib/librte_power/rte_power_version.map EXPERIMENTAL { · line 48

Quoted 11 lines
lib/librte_power/rte_power_version.map +1 -0
index 042917360..69f5ea3f4 100644
@@ -44,4 +44,5 @@ EXPERIMENTAL {
  	rte_power_empty_poll_stat_update;
  	rte_power_poll_stat_fetch;
  	rte_power_poll_stat_update;
+	rte_power_guest_channel_receive_msg;
  };

Nit Pick, new api should be added in alphabetical order, i.e after "rte_power_empty_poll_stat_update" .

Quoted 8 lines

I tested this with the later patches in the set, and the acknowledges successfully get back to the guest from the host in the form of an Ack or Nack for various commands sent from the guest.

Feel free to add my ack after the above minor change.

author
#11 lore raw

PATCH v5 3/4 power: process incoming confirmation cmds

In reply to Hajkowski

From: Marcin Hajkowski <[email protected]>

Extend vm_power_guest to check incoming confirmations of messages previously sent to host.

---
 examples/vm_power_manager/guest_cli/Makefile  |  1 +
 .../guest_cli/vm_power_cli_guest.c            | 73 +++++++++++++++----
 2 files changed, 61 insertions(+), 13 deletions(-)
examples/vm_power_manager/guest_cli/Makefile +1 -0
index e35a68d0f..67cf08193 100644
@@ -18,6 +18,7 @@ SRCS-y := main.c vm_power_cli_guest.c parse.c
 
 CFLAGS += -O3 -I$(RTE_SDK)/lib/librte_power/
 CFLAGS += $(WERROR_FLAGS)
+CFLAGS += -DALLOW_EXPERIMENTAL_API
 
 # workaround for a gcc bug with noreturn attribute
 # http://gcc.gnu.org/bugzilla/show_bug.cgi?id=12603
examples/vm_power_manager/guest_cli/vm_power_cli_guest.c +60 -13

116 lines not on this page · raw message

Signature · 1 line
2.17.2
Tested-by on v5 3/4
#12 lore raw

Re: [dpdk-dev] [PATCH v5 3/4] power: process incoming confirmation cmds

Quoted 140 lines · Hajkowski

From: Marcin Hajkowski <[email protected]>

Extend vm_power_guest to check incoming confirmations of messages previously sent to host.

---
  examples/vm_power_manager/guest_cli/Makefile  |  1 +
  .../guest_cli/vm_power_cli_guest.c            | 73 +++++++++++++++----
  2 files changed, 61 insertions(+), 13 deletions(-)
examples/vm_power_manager/guest_cli/Makefile +1 -0
index e35a68d0f..67cf08193 100644
@@ -18,6 +18,7 @@ SRCS-y := main.c vm_power_cli_guest.c parse.c
 
  CFLAGS += -O3 -I$(RTE_SDK)/lib/librte_power/
  CFLAGS += $(WERROR_FLAGS)
+CFLAGS += -DALLOW_EXPERIMENTAL_API
 
  # workaround for a gcc bug with noreturn attribute
  # http://gcc.gnu.org/bugzilla/show_bug.cgi?id=12603
examples/vm_power_manager/guest_cli/vm_power_cli_guest.c +59 -12
index 2d9e7689a..49ed7b208 100644
@@ -27,7 +27,7 @@
  #define CHANNEL_PATH "/dev/virtio-ports/virtio.serial.port.poweragent"
 
 
-#define RTE_LOGTYPE_GUEST_CHANNEL RTE_LOGTYPE_USER1
+#define RTE_LOGTYPE_GUEST_CLI RTE_LOGTYPE_USER1
 
  struct cmd_quit_result {
  	cmdline_fixed_string_t quit;
@@ -132,6 +132,32 @@ struct cmd_set_cpu_freq_result {
  	cmdline_fixed_string_t cmd;
  };
 
+static int
+check_response_cmd(unsigned int lcore_id, int *result)
+{
+	struct channel_packet pkt;
+	int ret;
+
+	ret = rte_power_guest_channel_receive_msg(&pkt, lcore_id);
+	if (ret < 0)
+		return -1;
+
+	switch (pkt.command) {
+	case(CPU_POWER_CMD_ACK):
+		*result = 1;
+		break;
+	case(CPU_POWER_CMD_NACK):
+		*result = 0;
+		break;
+	default:
+		RTE_LOG(ERR, GUEST_CLI,
+				"Received invalid response from host, expecting ACK/NACK.\n");
+		return -1;
+	}
+
+	return 0;
+}
+
  static void
  cmd_set_cpu_freq_parsed(void *parsed_result, struct cmdline *cl,
  		       __attribute__((unused)) void *data)
@@ -139,20 +165,31 @@ cmd_set_cpu_freq_parsed(void *parsed_result, struct cmdline *cl,
  	int ret = -1;
  	struct cmd_set_cpu_freq_result *res = parsed_result;
 
-	if (!strcmp(res->cmd , "up"))
+	if (!strcmp(res->cmd, "up"))
  		ret = rte_power_freq_up(res->lcore_id);
-	else if (!strcmp(res->cmd , "down"))
+	else if (!strcmp(res->cmd, "down"))
  		ret = rte_power_freq_down(res->lcore_id);
-	else if (!strcmp(res->cmd , "min"))
+	else if (!strcmp(res->cmd, "min"))
  		ret = rte_power_freq_min(res->lcore_id);
-	else if (!strcmp(res->cmd , "max"))
+	else if (!strcmp(res->cmd, "max"))
  		ret = rte_power_freq_max(res->lcore_id);
  	else if (!strcmp(res->cmd, "enable_turbo"))
  		ret = rte_power_freq_enable_turbo(res->lcore_id);
  	else if (!strcmp(res->cmd, "disable_turbo"))
  		ret = rte_power_freq_disable_turbo(res->lcore_id);
-	if (ret != 1)
+
+	if (ret != 1) {
  		cmdline_printf(cl, "Error sending message: %s\n", strerror(ret));
+		return;
+	}
+	int result;
+	ret = check_response_cmd(res->lcore_id, &result);
+	if (ret < 0) {
+		RTE_LOG(ERR, GUEST_CLI, "No confirmation for sent message received\n");
+	} else {
+		cmdline_printf(cl, "%s received for message sent to host.\n",
+				result == 1 ? "ACK" : "NACK");
+	}
  }
 
  cmdline_parse_token_string_t cmd_set_cpu_freq =
@@ -185,16 +222,26 @@ struct cmd_send_policy_result {
  };
 
  static inline int
-send_policy(struct channel_packet *pkt)
+send_policy(struct channel_packet *pkt, struct cmdline *cl)
  {
  	int ret;
 
  	ret = rte_power_guest_channel_send_msg(pkt, 1);
-	if (ret == 0)
-		return 1;
-	RTE_LOG(DEBUG, POWER, "Error sending message: %s\n",
-			ret > 0 ? strerror(ret) : "channel not connected");
-	return -1;
+	if (ret < 0) {
+		RTE_LOG(ERR, GUEST_CLI, "Error sending message: %s\n",
+				ret > 0 ? strerror(ret) : "channel not connected");
+		return -1;
+	}
+
+	int result;
+	ret = check_response_cmd(1, &result);
+	if (ret < 0) {
+		RTE_LOG(ERR, GUEST_CLI, "No confirmation for sent policy received\n");
+	} else {
+		cmdline_printf(cl, "%s for sent policy received.\n",
+				result == 1 ? "ACK" : "NACK");
+	}
+	return 1;
  }
 
  static void

examples/vm_power_manager/guest_cli/vm_power_cli_guest.c cmd_send_policy_parsed(void *parsed_result, struct cmdline *cl, · line 259

Quoted 8 lines
examples/vm_power_manager/guest_cli/vm_power_cli_guest.c +1 -1
@@ -206,7 +253,7 @@ cmd_send_policy_parsed(void *parsed_result, struct cmdline *cl,
 
  	if (!strcmp(res->cmd, "now")) {
  		printf("Sending Policy down now!\n");
-		ret = send_policy(&policy);
+		ret = send_policy(&policy, cl);
  	}
  	if (ret != 1)
  		cmdline_printf(cl, "Error sending message: %s\n",

I tested this with the later patches in the set, and the acknowledges successfully get back to the guest from the host in the form of an Ack or Nack for various commands sent from the guest.

Acked-by
#13 lore raw

Re: [dpdk-dev] [PATCH v5 3/4] power: process incoming confirmation cmds

Quoted 18 lines · Hajkowski
Quoted 17 lines

From: Marcin Hajkowski <[email protected]>

Extend vm_power_guest to check incoming confirmations of messages previously sent to host.

---
  examples/vm_power_manager/guest_cli/Makefile  |  1 +
  .../guest_cli/vm_power_cli_guest.c            | 73 +++++++++++++++----
  2 files changed, 61 insertions(+), 13 deletions(-)
examples/vm_power_manager/guest_cli/Makefile +0 -0
index e35a68d0f..67cf08193 100644
@@ -18,6 +18,7 @@ SRCS-y := main.c vm_power_cli_guest.c parse.c

<...>

Quoted 5 lines

I tested this with the later patches in the set, and the acknowledges successfully get back to the guest from the host in the form of an Ack or Nack for various commands sent from the guest.

author
#14 lore raw

PATCH v5 4/4 power: send confirmation cmd to vm guest

In reply to Hajkowski

From: Marcin Hajkowski <[email protected]>

Use new guest channel API to send confirmation
message for received power command.

---
 examples/vm_power_manager/channel_monitor.c | 68 +++++++++++++++++++--
 1 file changed, 62 insertions(+), 6 deletions(-)
examples/vm_power_manager/channel_monitor.c +62 -6
index 7892d75de..ed580b36a 100644
@@ -627,6 +627,41 @@ apply_policy(struct policy *pol)
 		apply_workload_profile(pol);
 }
 
+static int
+write_binary_packet(struct channel_packet *pkt, struct channel_info *chan_info)
+{
+	int ret, buffer_len = sizeof(*pkt);
+	void *buffer = pkt;
+
+	if (chan_info->fd < 0) {
+		RTE_LOG(ERR, CHANNEL_MONITOR, "Channel is not connected\n");
+		return -1;
+	}
+
+	while (buffer_len > 0) {
+		ret = write(chan_info->fd, buffer, buffer_len);
+		if (ret == -1) {
+			if (errno == EINTR)
+				continue;
+			RTE_LOG(ERR, CHANNEL_MONITOR, "Write function failed due to %s.\n",
+					strerror(errno));
+			return -1;
+		}
+		buffer = (char *)buffer + ret;
+		buffer_len -= ret;
+	}
+	return 0;
+}
+
+static int
+send_ack_for_received_cmd(struct channel_packet *pkt,
+		struct channel_info *chan_info,
+		uint32_t command)
+{
+	pkt->command = command;
+	return write_binary_packet(pkt, chan_info);
+}
+
 static int
 process_request(struct channel_packet *pkt, struct channel_info *chan_info)
 {
@@ -650,33 +685,54 @@ process_request(struct channel_packet *pkt, struct channel_info *chan_info)
 		RTE_LOG(DEBUG, CHANNEL_MONITOR, "Processing requested cmd for cpu:%d\n",
 			core_num);
 
+		bool valid_unit = true;
+		int scale_res;
+
 		switch (pkt->unit) {
 		case(CPU_POWER_SCALE_MIN):
-			power_manager_scale_core_min(core_num);
+			scale_res = power_manager_scale_core_min(core_num);
 			break;
 		case(CPU_POWER_SCALE_MAX):
-			power_manager_scale_core_max(core_num);
+			scale_res = power_manager_scale_core_max(core_num);
 			break;
 		case(CPU_POWER_SCALE_DOWN):
-			power_manager_scale_core_down(core_num);
+			scale_res = power_manager_scale_core_down(core_num);
 			break;
 		case(CPU_POWER_SCALE_UP):
-			power_manager_scale_core_up(core_num);
+			scale_res = power_manager_scale_core_up(core_num);
 			break;
 		case(CPU_POWER_ENABLE_TURBO):
-			power_manager_enable_turbo_core(core_num);
+			scale_res = power_manager_enable_turbo_core(core_num);
 			break;
 		case(CPU_POWER_DISABLE_TURBO):
-			power_manager_disable_turbo_core(core_num);
+			scale_res = power_manager_disable_turbo_core(core_num);
 			break;
 		default:
+			valid_unit = false;
 			break;
 		}
+
+		if (valid_unit) {
+			ret = send_ack_for_received_cmd(pkt,
+					chan_info,
+					scale_res > 0 ?
+						CPU_POWER_CMD_ACK :
+						CPU_POWER_CMD_NACK);
+			if (ret < 0)
+				RTE_LOG(DEBUG, CHANNEL_MONITOR, "Error during sending ack command.\n");
+		} else
+			RTE_LOG(DEBUG, CHANNEL_MONITOR, "Unexpected unit type.\n");
+
 	}
 
 	if (pkt->command == PKT_POLICY) {
 		RTE_LOG(INFO, CHANNEL_MONITOR, "Processing policy request %s\n",
 				pkt->vm_name);
+		int ret = send_ack_for_received_cmd(pkt,
+				chan_info,
+				CPU_POWER_CMD_ACK);
+		if (ret < 0)
+			RTE_LOG(DEBUG, CHANNEL_MONITOR, "Error during sending ack command.\n");
 		update_policy(pkt);
 		policy_is_set = 1;
 	}
Signature · 1 line
2.17.2
Tested-by on v5 4/4
#15 lore raw

Re: [dpdk-dev] [PATCH v5 4/4] power: send confirmation cmd to vm guest

Quoted 107 lines · Hajkowski

From: Marcin Hajkowski <[email protected]>

Use new guest channel API to send confirmation
message for received power command.

---
  examples/vm_power_manager/channel_monitor.c | 68 +++++++++++++++++++--
  1 file changed, 62 insertions(+), 6 deletions(-)
examples/vm_power_manager/channel_monitor.c +57 -6
index 7892d75de..ed580b36a 100644
@@ -627,6 +627,41 @@ apply_policy(struct policy *pol)
  		apply_workload_profile(pol);
  }
 
+static int
+write_binary_packet(struct channel_packet *pkt, struct channel_info *chan_info)
+{
+	int ret, buffer_len = sizeof(*pkt);
+	void *buffer = pkt;
+
+	if (chan_info->fd < 0) {
+		RTE_LOG(ERR, CHANNEL_MONITOR, "Channel is not connected\n");
+		return -1;
+	}
+
+	while (buffer_len > 0) {
+		ret = write(chan_info->fd, buffer, buffer_len);
+		if (ret == -1) {
+			if (errno == EINTR)
+				continue;
+			RTE_LOG(ERR, CHANNEL_MONITOR, "Write function failed due to %s.\n",
+					strerror(errno));
+			return -1;
+		}
+		buffer = (char *)buffer + ret;
+		buffer_len -= ret;
+	}
+	return 0;
+}
+
+static int
+send_ack_for_received_cmd(struct channel_packet *pkt,
+		struct channel_info *chan_info,
+		uint32_t command)
+{
+	pkt->command = command;
+	return write_binary_packet(pkt, chan_info);
+}
+
  static int
  process_request(struct channel_packet *pkt, struct channel_info *chan_info)
  {
@@ -650,33 +685,54 @@ process_request(struct channel_packet *pkt, struct channel_info *chan_info)
  		RTE_LOG(DEBUG, CHANNEL_MONITOR, "Processing requested cmd for cpu:%d\n",
  			core_num);
 
+		bool valid_unit = true;
+		int scale_res;
+
  		switch (pkt->unit) {
  		case(CPU_POWER_SCALE_MIN):
-			power_manager_scale_core_min(core_num);
+			scale_res = power_manager_scale_core_min(core_num);
  			break;
  		case(CPU_POWER_SCALE_MAX):
-			power_manager_scale_core_max(core_num);
+			scale_res = power_manager_scale_core_max(core_num);
  			break;
  		case(CPU_POWER_SCALE_DOWN):
-			power_manager_scale_core_down(core_num);
+			scale_res = power_manager_scale_core_down(core_num);
  			break;
  		case(CPU_POWER_SCALE_UP):
-			power_manager_scale_core_up(core_num);
+			scale_res = power_manager_scale_core_up(core_num);
  			break;
  		case(CPU_POWER_ENABLE_TURBO):
-			power_manager_enable_turbo_core(core_num);
+			scale_res = power_manager_enable_turbo_core(core_num);
  			break;
  		case(CPU_POWER_DISABLE_TURBO):
-			power_manager_disable_turbo_core(core_num);
+			scale_res = power_manager_disable_turbo_core(core_num);
  			break;
  		default:
+			valid_unit = false;
  			break;
  		}
+
+		if (valid_unit) {
+			ret = send_ack_for_received_cmd(pkt,
+					chan_info,
+					scale_res > 0 ?
+						CPU_POWER_CMD_ACK :
+						CPU_POWER_CMD_NACK);
+			if (ret < 0)
+				RTE_LOG(DEBUG, CHANNEL_MONITOR, "Error during sending ack command.\n");
+		} else
+			RTE_LOG(DEBUG, CHANNEL_MONITOR, "Unexpected unit type.\n");
+
  	}
 
  	if (pkt->command == PKT_POLICY) {

examples/vm_power_manager/channel_monitor.c process_request(struct channel_packet *pkt, struct channel_info *chan_info) · line 738

Quoted 10 lines
examples/vm_power_manager/channel_monitor.c +5 -0
  		RTE_LOG(INFO, CHANNEL_MONITOR, "Processing policy request %s\n",
  				pkt->vm_name);
+		int ret = send_ack_for_received_cmd(pkt,
+				chan_info,
+				CPU_POWER_CMD_ACK);
+		if (ret < 0)
+			RTE_LOG(DEBUG, CHANNEL_MONITOR, "Error during sending ack command.\n");
  		update_policy(pkt);
  		policy_is_set = 1;
  	}

Using the guest_cli sample app in a VM, sending commands to the vm_power_manager app on the host, we now have Acks and Nacks coming from the host to the VM to conform the execution of the guest requests.

vmpower(guest)> set_cpu_freq 3 down ACK received for message sent to host. vmpower(guest)> set_cpu_freq 3 up ACK received for message sent to host. vmpower(guest)> set_cpu_freq 3 up NACK received for message sent to host.

(NACK because we're already at the maxumum frequency)

And in the host vm_power_manager command line, we can see when a guest request cannot be processed:
POWER: Turbo is off, frequency can't be scaled up more 31

Patchset looks good functionally.

Acked-by
#16 lore raw

Re: [dpdk-dev] [PATCH v5 4/4] power: send confirmation cmd to vm guest

Quoted 17 lines · Hajkowski
Quoted 8 lines

From: Marcin Hajkowski <[email protected]>

Use new guest channel API to send confirmation message for received power command.

---
  examples/vm_power_manager/channel_monitor.c | 68

+++++++++++++++++++--

Quoted 7 lines
  1 file changed, 62 insertions(+), 6 deletions(-)
examples/vm_power_manager/channel_monitor.c +0 -0
index 7892d75de..ed580b36a 100644

<...>

Quoted 14 lines

Using the guest_cli sample app in a VM, sending commands to the vm_power_manager app on the host, we now have Acks and Nacks coming from the host to the VM to conform the execution of the guest requests.

vmpower(guest)> set_cpu_freq 3 down
ACK received for message sent to host.
vmpower(guest)> set_cpu_freq 3 up
ACK received for message sent to host.
vmpower(guest)> set_cpu_freq 3 up
NACK received for message sent to host.

(NACK because we're already at the maxumum frequency)

Quoted 7 lines

And in the host vm_power_manager command line, we can see when a guest request cannot be processed:
POWER: Turbo is off, frequency can't be scaled up more 31

Patchset looks good functionally.

#17 lore raw
In reply to Hajkowski

05/04/2019 15:24, Hajkowski:

Quoted 5 lines

From: Marcin Hajkowski <[email protected]>

Extend guest channel API to allow bidirectional communication. Modify power manager host and guest side to communicate in both directions.

This patchset is deferred to 19.08 because of a lack of review.

Note: too many librte_power patches are not reviewed.

#18 lore raw
Quoted 10 lines · Thomas Monjalon

05/04/2019 15:24, Hajkowski:

Quoted 5 lines

From: Marcin Hajkowski <[email protected]>

Extend guest channel API to allow bidirectional communication. Modify power manager host and guest side to communicate in both directions.

This patchset is deferred to 19.08 because of a lack of review.

Note: too many librte_power patches are not reviewed.

Hi Thomas,

 The four remaining patch sets for power were initially submitted after the v1 deadline, and were not expected to make it into 19.05. We will be sure to review for 19.08.

Regards,
Dave.

#19 lore raw

24/04/2019 10:38, Hunt, David:

Quoted 10 lines · Thomas Monjalon
Quoted 9 lines

05/04/2019 15:24, Hajkowski:

Quoted 5 lines

From: Marcin Hajkowski <[email protected]>

Extend guest channel API to allow bidirectional communication. Modify power manager host and guest side to communicate in both directions.

This patchset is deferred to 19.08 because of a lack of review.

Quoted 8 lines
Quoted 1 line

Note: too many librte_power patches are not reviewed.

Hi Thomas,

The four remaining patch sets for power were initially submitted after
the v1 deadline, and were not expected to make it into 19.05. We will be sure to review for 19.08.

OK thanks for the confirmation.
In such case, it is better to add "19.08" in the subject prefix so it is explicit from the beginning. Thanks

#20 lore raw

24/04/2019 10:44, Thomas Monjalon:

Quoted 16 lines

24/04/2019 10:38, Hunt, David:

Quoted 15 lines · Thomas Monjalon
Quoted 10 lines

05/04/2019 15:24, Hajkowski:

Quoted 5 lines

From: Marcin Hajkowski <[email protected]>

Extend guest channel API to allow bidirectional communication. Modify power manager host and guest side to communicate in both directions.

This patchset is deferred to 19.08 because of a lack of review.

Note: too many librte_power patches are not reviewed.

Hi Thomas,

Quoted 7 lines
Quoted 3 lines

The four remaining patch sets for power were initially submitted after
the v1 deadline, and were not expected to make it into 19.05. We will be sure to review for 19.08.

OK thanks for the confirmation.
In such case, it is better to add "19.08" in the subject prefix so it is explicit from the beginning. Thanks

There was no more review since April.
I guess it will miss 19.08 as well.