Skip to contentSkip to conversation

s390/qeth: updates 2020-08-27

PATCH In review Julian Wiedmann 2 participants 10 messages Aug 27, 2020

Subsystems (4)
lore
Share & tools
lore
#0 lore raw

cover s390/qeth: updates 2020-08-27

Hi Dave & Jakub,

please apply the following patch series for qeth to netdev's net-next tree.

Patch 8 makes some improvements to how we handle HW address events, avoiding some uncertainty around processing stale events after we switched off the feature.
Except for that it's all straight-forward cleanups.

Thanks,
Julian

Julian Wiedmann (8):
s390/qeth: clean up qeth_l3_send_setdelmc()'s declaration s390/qeth: use to_delayed_work()
s390/qeth: make queue lock a proper spinlock s390/qeth: don't disable address events during initialization s390/qeth: don't let HW override the configured port role s390/qeth: copy less data from bridge state events s390/qeth: unify structs for bridge port state s390/qeth: strictly order bridge address events

 drivers/s390/net/qeth_core.h      | 14 ++---
 drivers/s390/net/qeth_core_main.c | 85 ++++++++-------------------
 drivers/s390/net/qeth_core_mpc.h  | 14 +----
 drivers/s390/net/qeth_l2_main.c   | 96 ++++++++++++++++++++-----------
 drivers/s390/net/qeth_l2_sys.c    |  1 +
 drivers/s390/net/qeth_l3_main.c   |  3 +-
 6 files changed, 100 insertions(+), 113 deletions(-)
Signature · 1 line
2.17.1
#1 lore raw

PATCH 1/8 s390/qeth: clean up qeth_l3_send_setdelmc()'s declaration

Clarify that the 'ipacmd' parameter is an enum, and thus compatible to what qeth_ipa_alloc_cmd() expects as input.

---
 drivers/s390/net/qeth_l3_main.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
drivers/s390/net/qeth_l3_main.c +2 -1
index fe44b0249e34..95df638de616 100644
@@ -314,7 +314,8 @@ static int qeth_l3_setdelip_cb(struct qeth_card *card, struct qeth_reply *reply,
 }
 
 static int qeth_l3_send_setdelmc(struct qeth_card *card,
-			struct qeth_ipaddr *addr, int ipacmd)
+				 struct qeth_ipaddr *addr,
+				 enum qeth_ipa_cmds ipacmd)
 {
 	struct qeth_cmd_buffer *iob;
 	struct qeth_ipa_cmd *cmd;
Signature · 1 line
2.17.1
#2 lore raw

PATCH 2/8 s390/qeth: use to_delayed_work()

In reply to Julian Wiedmann

Avoid poking around in the delayed_work struct's internals.

---
 drivers/s390/net/qeth_core_main.c | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)
drivers/s390/net/qeth_core_main.c +3 -2
index bba1b54b8aa3..5742a682a193 100644
@@ -3549,8 +3549,9 @@ static unsigned int qeth_rx_refill_queue(struct qeth_card *card,
 
 static void qeth_buffer_reclaim_work(struct work_struct *work)
 {
-	struct qeth_card *card = container_of(work, struct qeth_card,
-		buffer_reclaim_work.work);
+	struct qeth_card *card = container_of(to_delayed_work(work),
+					      struct qeth_card,
+					      buffer_reclaim_work);
 
 	local_bh_disable();
 	napi_schedule(&card->napi);
Signature · 1 line
2.17.1
#3 lore raw

PATCH 3/8 s390/qeth: make queue lock a proper spinlock

In reply to Julian Wiedmann

queue->state is a ternary spinlock in disguise, used by OSA's TX completion path to lock the Output Queue and flush any pending packets on it to the device. If the Queue is already locked by our TX code, setting the lock word to QETH_OUT_Q_LOCKED_FLUSH lets the TX completion code move on - the TX path will later take care of things when it unlocks the Queue.

This sort of DIY locking is a non-starter of course, just let the TX completion path block on the spinlock when necessary. If that ends up causing additional latency due to lock contention, then converting the OSA path to use xmit_more is the right way to go forward.

Also slightly expand the locked section and capture all of qeth_do_send_packet(), so that the update for the 'bufs_pack' statistics is done race-free.

While reworking the TX completion path's code, remove a barrier() that doesn't make any sense.

---
 drivers/s390/net/qeth_core.h      |  8 +---
 drivers/s390/net/qeth_core_main.c | 80 +++++++++----------------------
 2 files changed, 23 insertions(+), 65 deletions(-)
drivers/s390/net/qeth_core.h +1 -7
index ecfd6d152e86..00ed58428163 100644
@@ -420,12 +420,6 @@ struct qeth_qdio_out_buffer {
 
 struct qeth_card;
 
-enum qeth_out_q_states {
-       QETH_OUT_Q_UNLOCKED,
-       QETH_OUT_Q_LOCKED,
-       QETH_OUT_Q_LOCKED_FLUSH,
-};
-
 #define QETH_CARD_STAT_ADD(_c, _stat, _val)	((_c)->stats._stat += (_val))
 #define QETH_CARD_STAT_INC(_c, _stat)		QETH_CARD_STAT_ADD(_c, _stat, 1)
 
@@ -486,12 +480,12 @@ struct qeth_qdio_out_q {
 	struct qeth_qdio_out_buffer *bufs[QDIO_MAX_BUFFERS_PER_Q];
 	struct qdio_outbuf_state *bufstates; /* convenience pointer */
 	struct qeth_out_q_stats stats;
+	spinlock_t lock;
 	u8 next_buf_to_fill;
 	u8 max_elements;
 	u8 queue_no;
 	u8 do_pack;
 	struct qeth_card *card;
-	atomic_t state;
 	/*
 	 * number of buffers that are currently filled (PRIMED)
 	 * -> these buffers are hardware-owned
drivers/s390/net/qeth_core_main.c +22 -58

148 lines not on this page · raw message

Signature · 1 line
2.17.1
#4 lore raw

PATCH 4/8 s390/qeth: don't disable address events during initialization

In reply to Julian Wiedmann

A newly initialized device is disabled for address events, there's no need to explicitly disable them.

---
 drivers/s390/net/qeth_l2_main.c | 2 --
 1 file changed, 2 deletions(-)
drivers/s390/net/qeth_l2_main.c +0 -2
index 8b342a88ff5c..ea966713ce7e 100644
@@ -810,8 +810,6 @@ static void qeth_l2_setup_bridgeport_attrs(struct qeth_card *card)
 	if (card->options.sbp.hostnotification) {
 		if (qeth_bridgeport_an_set(card, 1))
 			card->options.sbp.hostnotification = 0;
-	} else {
-		qeth_bridgeport_an_set(card, 0);
 	}
 }
 
Signature · 1 line
2.17.1
#5 lore raw

PATCH 5/8 s390/qeth: don't let HW override the configured port role

In reply to Julian Wiedmann

The only time that our Bridgeport role should change is when we change the configuration ourselves. In which case we also adjust our internal state tracking, no need to do it again when we receive the corresponding event.

Removing the locked section helps a subsequent patch that needs to flush the workqueue while under sbp_lock.

It would be nice to raise a warning here in case HW does weird things after all, but this could end up generating false-positives when we change the configuration ourselves.

---
 drivers/s390/net/qeth_l2_main.c | 6 ------
 1 file changed, 6 deletions(-)
drivers/s390/net/qeth_l2_main.c +0 -6
index ea966713ce7e..5a023fc499eb 100644
@@ -1107,12 +1107,6 @@ static void qeth_bridge_state_change_worker(struct work_struct *work)
 		NULL
 	};
 
-	/* Role should not change by itself, but if it did, */
-	/* information from the hardware is authoritative.  */
-	mutex_lock(&data->card->sbp_lock);
-	data->card->options.sbp.role = entry->role;
-	mutex_unlock(&data->card->sbp_lock);
-
 	snprintf(env_locrem, sizeof(env_locrem), "BRIDGEPORT=statechange");
 	snprintf(env_role, sizeof(env_role), "ROLE=%s",
 		(entry->role == QETH_SBP_ROLE_NONE) ? "none" :
Signature · 1 line
2.17.1
#6 lore raw

PATCH 6/8 s390/qeth: copy less data from bridge state events

In reply to Julian Wiedmann

Current code copies _all_ entries from the event into a worker, when we later only need specific data from the first entry.

---
 drivers/s390/net/qeth_l2_main.c | 29 ++++++++++++++---------------
 1 file changed, 14 insertions(+), 15 deletions(-)
drivers/s390/net/qeth_l2_main.c +14 -15
index 5a023fc499eb..d96636369b4c 100644
@@ -1088,15 +1088,14 @@ static void qeth_bridge_emit_host_event(struct qeth_card *card,
 struct qeth_bridge_state_data {
 	struct work_struct worker;
 	struct qeth_card *card;
-	struct qeth_sbp_state_change qports;
+	u8 role;
+	u8 state;
 };
 
 static void qeth_bridge_state_change_worker(struct work_struct *work)
 {
 	struct qeth_bridge_state_data *data =
 		container_of(work, struct qeth_bridge_state_data, worker);
-	/* We are only interested in the first entry - local port */
-	struct qeth_sbp_port_entry *entry = &data->qports.entry[0];
 	char env_locrem[32];
 	char env_role[32];
 	char env_state[32];
@@ -1109,14 +1108,14 @@ static void qeth_bridge_state_change_worker(struct work_struct *work)
 
 	snprintf(env_locrem, sizeof(env_locrem), "BRIDGEPORT=statechange");
 	snprintf(env_role, sizeof(env_role), "ROLE=%s",
-		(entry->role == QETH_SBP_ROLE_NONE) ? "none" :
-		(entry->role == QETH_SBP_ROLE_PRIMARY) ? "primary" :
-		(entry->role == QETH_SBP_ROLE_SECONDARY) ? "secondary" :
+		(data->role == QETH_SBP_ROLE_NONE) ? "none" :
+		(data->role == QETH_SBP_ROLE_PRIMARY) ? "primary" :
+		(data->role == QETH_SBP_ROLE_SECONDARY) ? "secondary" :
 		"<INVALID>");
 	snprintf(env_state, sizeof(env_state), "STATE=%s",
-		(entry->state == QETH_SBP_STATE_INACTIVE) ? "inactive" :
-		(entry->state == QETH_SBP_STATE_STANDBY) ? "standby" :
-		(entry->state == QETH_SBP_STATE_ACTIVE) ? "active" :
+		(data->state == QETH_SBP_STATE_INACTIVE) ? "inactive" :
+		(data->state == QETH_SBP_STATE_STANDBY) ? "standby" :
+		(data->state == QETH_SBP_STATE_ACTIVE) ? "active" :
 		"<INVALID>");
 	kobject_uevent_env(&data->card->gdev->dev.kobj,
 				KOBJ_CHANGE, env);
@@ -1129,7 +1128,6 @@ static void qeth_bridge_state_change(struct qeth_card *card,
 	struct qeth_sbp_state_change *qports =
 		 &cmd->data.sbp.data.state_change;
 	struct qeth_bridge_state_data *data;
-	int extrasize;
 
 	QETH_CARD_TEXT(card, 2, "brstchng");
 	if (qports->num_entries == 0) {
@@ -1140,17 +1138,18 @@ static void qeth_bridge_state_change(struct qeth_card *card,
 		QETH_CARD_TEXT_(card, 2, "BPsz%04x", qports->entry_length);
 		return;
 	}
-	extrasize = sizeof(struct qeth_sbp_port_entry) * qports->num_entries;
-	data = kzalloc(sizeof(struct qeth_bridge_state_data) + extrasize,
-		GFP_ATOMIC);
+
+	data = kzalloc(sizeof(*data), GFP_ATOMIC);
 	if (!data) {
 		QETH_CARD_TEXT(card, 2, "BPSalloc");
 		return;
 	}
 	INIT_WORK(&data->worker, qeth_bridge_state_change_worker);
 	data->card = card;
-	memcpy(&data->qports, qports,
-			sizeof(struct qeth_sbp_state_change) + extrasize);
+	/* Information for the local port: */
+	data->role = qports->entry[0].role;
+	data->state = qports->entry[0].state;
+
 	queue_work(card->event_wq, &data->worker);
 }
 
Signature · 1 line
2.17.1
#7 lore raw

PATCH 7/8 s390/qeth: unify structs for bridge port state

In reply to Julian Wiedmann

The data returned from IPA_SBP_QUERY_BRIDGE_PORTS and IPA_SBP_BRIDGE_PORT_STATE_CHANGE has the same format. Use a single struct definition for it.

---
 drivers/s390/net/qeth_core_mpc.h | 14 +++-----------
 drivers/s390/net/qeth_l2_main.c  |  6 +++---
 2 files changed, 6 insertions(+), 14 deletions(-)
drivers/s390/net/qeth_core_mpc.h +3 -11
index b459def0fb26..6541bab96822 100644
@@ -719,15 +719,8 @@ struct qeth_sbp_port_entry {
 		struct net_if_token token;
 } __packed;
 
-struct qeth_sbp_query_ports {
-	__u8 primary_bp_supported;
-	__u8 secondary_bp_supported;
-	__u8 num_entries;
-	__u8 entry_length;
-	struct qeth_sbp_port_entry entry[];
-} __packed;
-
-struct qeth_sbp_state_change {
+/* For IPA_SBP_QUERY_BRIDGE_PORTS, IPA_SBP_BRIDGE_PORT_STATE_CHANGE */
+struct qeth_sbp_port_data {
 	__u8 primary_bp_supported;
 	__u8 secondary_bp_supported;
 	__u8 num_entries;
@@ -741,8 +734,7 @@ struct qeth_ipacmd_setbridgeport {
 	union {
 		struct qeth_sbp_query_cmds_supp query_cmds_supp;
 		struct qeth_sbp_set_primary set_primary;
-		struct qeth_sbp_query_ports query_ports;
-		struct qeth_sbp_state_change state_change;
+		struct qeth_sbp_port_data port_data;
 	} data;
 } __packed;
 
drivers/s390/net/qeth_l2_main.c +3 -3
index d96636369b4c..1e3d64ab5e46 100644
@@ -1125,8 +1125,7 @@ static void qeth_bridge_state_change_worker(struct work_struct *work)
 static void qeth_bridge_state_change(struct qeth_card *card,
 					struct qeth_ipa_cmd *cmd)
 {
-	struct qeth_sbp_state_change *qports =
-		 &cmd->data.sbp.data.state_change;
+	struct qeth_sbp_port_data *qports = &cmd->data.sbp.data.port_data;
 	struct qeth_bridge_state_data *data;
 
 	QETH_CARD_TEXT(card, 2, "brstchng");
@@ -1409,8 +1408,8 @@ static int qeth_bridgeport_query_ports_cb(struct qeth_card *card,
 	struct qeth_reply *reply, unsigned long data)
 {
 	struct qeth_ipa_cmd *cmd = (struct qeth_ipa_cmd *) data;
-	struct qeth_sbp_query_ports *qports = &cmd->data.sbp.data.query_ports;
 	struct _qeth_sbp_cbctl *cbctl = (struct _qeth_sbp_cbctl *)reply->param;
+	struct qeth_sbp_port_data *qports;
 	int rc;
 
 	QETH_CARD_TEXT(card, 2, "brqprtcb");
@@ -1418,6 +1417,7 @@ static int qeth_bridgeport_query_ports_cb(struct qeth_card *card,
 	if (rc)
 		return rc;
 
+	qports = &cmd->data.sbp.data.port_data;
 	if (qports->entry_length != sizeof(struct qeth_sbp_port_entry)) {
 		QETH_CARD_TEXT_(card, 2, "SBPs%04x", qports->entry_length);
 		return -EINVAL;
Signature · 1 line
2.17.1
#8 lore raw

PATCH 8/8 s390/qeth: strictly order bridge address events

In reply to Julian Wiedmann

The current code for bridge address events has two shortcomings in its control sequence:

1. after disabling address events via PNSO, we don't flush the remaining events from the event_wq. So if the feature is re-enabled fast enough, stale events could leak over.
2. PNSO and the events' arrival via the READ ccw device are unordered. So even if we flushed the workqueue, it's difficult to say whether the READ device might produce more events onto the workqueue afterwards.

Fix this by
1. explicitly fencing off the events when we no longer care, in the READ device's event handler. This ensures that once we flush the workqueue, it doesn't get additional address events.
2. Flush the workqueue after disabling the events & fencing them off. As the code that triggers the flush will typically hold the sbp_lock, we need to rework the worker code to avoid a deadlock here in case of a 'notifications-stopped' event. In case of lock contention, requeue such an event with a delay. We'll eventually aquire the lock, or spot that the feature has been disabled and the event can thus be discarded.

This leaves the theoretical race that a stale event could arrive _after_ we re-enabled ourselves to receive events again. Such an event would be impossible to distinguish from a 'good' event, nothing we can do about it.

---
 drivers/s390/net/qeth_core.h    |  6 ++++
 drivers/s390/net/qeth_l2_main.c | 53 ++++++++++++++++++++++++++++-----
 drivers/s390/net/qeth_l2_sys.c  |  1 +
 3 files changed, 52 insertions(+), 8 deletions(-)
drivers/s390/net/qeth_core.h +6 -0
index 00ed58428163..da46af682af8 100644
@@ -674,6 +674,11 @@ struct qeth_card_blkt {
 	int inter_packet_jumbo;
 };
 
+enum qeth_pnso_mode {
+	QETH_PNSO_NONE,
+	QETH_PNSO_BRIDGEPORT,
+};
+
 #define QETH_BROADCAST_WITH_ECHO    0x01
 #define QETH_BROADCAST_WITHOUT_ECHO 0x02
 struct qeth_card_info {
@@ -690,6 +695,7 @@ struct qeth_card_info {
 	/* no bitfield, we take a pointer on these two: */
 	u8 has_lp2lp_cso_v6;
 	u8 has_lp2lp_cso_v4;
+	enum qeth_pnso_mode pnso_mode;
 	enum qeth_card_types type;
 	enum qeth_link_types link_type;
 	int broadcast_capable;
drivers/s390/net/qeth_l2_main.c +45 -8

117 lines not on this page · raw message

drivers/s390/net/qeth_l2_sys.c +1 -0
index 86bcae992f72..4695d25e54f2 100644
@@ -157,6 +157,7 @@ static ssize_t qeth_bridgeport_hostnotification_store(struct device *dev,
 		rc = -EBUSY;
 	else if (qeth_card_hw_is_reachable(card)) {
 		rc = qeth_bridgeport_an_set(card, enable);
+		/* sbp_lock ensures ordering vs notifications-stopped events */
 		if (!rc)
 			card->options.sbp.hostnotification = enable;
 	} else
Signature · 1 line
2.17.1
maintainer
#9 lore raw
In reply to Julian Wiedmann

From: Julian Wiedmann <[email protected]>
Date: Thu, 27 Aug 2020 10:16:57 +0200

Quoted 6 lines

please apply the following patch series for qeth to netdev's net-next tree.

Patch 8 makes some improvements to how we handle HW address events, avoiding some uncertainty around processing stale events after we switched off the feature.
Except for that it's all straight-forward cleanups.

Series applied, thank you.