Skip to contentSkip to conversation

s390/qeth: fixes 2018-02-27

PATCH In review Julian Wiedmann 2 participants 8 messages Feb 27, 2018 – Feb 28, 2018

Subsystems (4)
lore
Share & tools
lore
#0 lore raw

cover s390/qeth: fixes 2018-02-27

Hi Dave,

please apply some more qeth patches for -net and stable.

One patch fixes a performance bug in the TSO path. Then there's several more fixes for IP management on L3 devices - including a revert, so that the subsequent fix cleanly applies to earlier kernels.
The final patch takes care of a race in the control IO code that causes qeth to miss the cmd response, and subsequently trigger device recovery.

Thanks,
Julian

Julian Wiedmann (6):
s390/qeth: fix overestimated count of buffer elements s390/qeth: fix IP removal on offline cards
s390/qeth: fix double-free on IP add/remove race Revert "s390/qeth: fix using of ref counter for rxip addresses" s390/qeth: fix IP address lookup for L3 devices s390/qeth: fix IPA command submission race

 drivers/s390/net/qeth_core_main.c |  29 +++++----
 drivers/s390/net/qeth_l3.h        |  34 ++++++++++-
 drivers/s390/net/qeth_l3_main.c   | 123 ++++++++++++++++----------------------
 3 files changed, 102 insertions(+), 84 deletions(-)
Signature · 1 line
2.13.5
#1 lore raw

PATCH 1/6 s390/qeth: fix overestimated count of buffer elements

qeth_get_elements_for_range() doesn't know how to handle a 0-length range (ie. start == end), and returns 1 when it should return 0. Such ranges occur on TSO skbs, where the L2/L3/L4 headers (and thus all of the skb's linear data) are skipped when mapping the skb into regular buffer elements.

This overestimation may cause several performance-related issues:
1. sub-optimal IO buffer selection, where the next buffer gets selected even though the skb would actually still fit into the current buffer.
2. forced linearization, if the element count for a non-linear skb exceeds QETH_MAX_BUFFER_ELEMENTS.

Rather than modifying qeth_get_elements_for_range() and adding overhead to every caller, fix up those callers that are in risk of passing a 0-length range.

Fixes 2863c61334aa ("qeth: refactor calculation of SBALE count")
---
 drivers/s390/net/qeth_core_main.c | 10 ++++++----
 drivers/s390/net/qeth_l3_main.c   | 11 ++++++-----
 2 files changed, 12 insertions(+), 9 deletions(-)
drivers/s390/net/qeth_core_main.c +6 -4
index ca72f3311004..30457fca30c5 100644
@@ -3898,10 +3898,12 @@ EXPORT_SYMBOL_GPL(qeth_get_elements_for_frags);
 int qeth_get_elements_no(struct qeth_card *card,
 		     struct sk_buff *skb, int extra_elems, int data_offset)
 {
-	int elements = qeth_get_elements_for_range(
-				(addr_t)skb->data + data_offset,
-				(addr_t)skb->data + skb_headlen(skb)) +
-			qeth_get_elements_for_frags(skb);
+	addr_t end = (addr_t)skb->data + skb_headlen(skb);
+	int elements = qeth_get_elements_for_frags(skb);
+	addr_t start = (addr_t)skb->data + data_offset;
+
+	if (start != end)
+		elements += qeth_get_elements_for_range(start, end);
 
 	if ((elements + extra_elems) > QETH_MAX_BUFFER_ELEMENTS(card)) {
 		QETH_DBF_MESSAGE(2, "Invalid size of IP packet "
drivers/s390/net/qeth_l3_main.c +6 -5
index b0c888e86cd4..3421893c37a4 100644
@@ -2450,11 +2450,12 @@ static void qeth_tso_fill_header(struct qeth_card *card,
 static int qeth_l3_get_elements_no_tso(struct qeth_card *card,
 			struct sk_buff *skb, int extra_elems)
 {
-	addr_t tcpdptr = (addr_t)tcp_hdr(skb) + tcp_hdrlen(skb);
-	int elements = qeth_get_elements_for_range(
-				tcpdptr,
-				(addr_t)skb->data + skb_headlen(skb)) +
-				qeth_get_elements_for_frags(skb);
+	addr_t start = (addr_t)tcp_hdr(skb) + tcp_hdrlen(skb);
+	addr_t end = (addr_t)skb->data + skb_headlen(skb);
+	int elements = qeth_get_elements_for_frags(skb);
+
+	if (start != end)
+		elements += qeth_get_elements_for_range(start, end);
 
 	if ((elements + extra_elems) > QETH_MAX_BUFFER_ELEMENTS(card)) {
 		QETH_DBF_MESSAGE(2,
Signature · 1 line
2.13.5
#2 lore raw

PATCH 2/6 s390/qeth: fix IP removal on offline cards

In reply to Julian Wiedmann

If the HW is not reachable, then none of the IPs in qeth's internal table has been registered with the HW yet. So when deleting such an IP, there's no need to stage it for deregistration - just drop it from the table.

This fixes the "add-delete-add" scenario on an offline card, where the the second "add" merely increments the IP's use count. But as the IP is still set to DISP_ADDR_DELETE from the previous "delete" step, l3_recover_ip() won't register it with the HW when the card goes online.

Fixes 5f78e29ceebf ("qeth: optimize IP handling in rx_mode callback")
---
 drivers/s390/net/qeth_l3_main.c | 14 +++-----------
 1 file changed, 3 insertions(+), 11 deletions(-)
drivers/s390/net/qeth_l3_main.c +3 -11
index 3421893c37a4..34481b51029e 100644
@@ -173,12 +173,8 @@ int qeth_l3_delete_ip(struct qeth_card *card, struct qeth_ipaddr *tmp_addr)
 	if (addr->in_progress)
 		return -EINPROGRESS;
 
-	if (!qeth_card_hw_is_reachable(card)) {
-		addr->disp_flag = QETH_DISP_ADDR_DELETE;
-		return 0;
-	}
-
-	rc = qeth_l3_deregister_addr_entry(card, addr);
+	if (qeth_card_hw_is_reachable(card))
+		rc = qeth_l3_deregister_addr_entry(card, addr);
 
 	hash_del(&addr->hnode);
 	kfree(addr);
@@ -321,11 +317,7 @@ static void qeth_l3_recover_ip(struct qeth_card *card)
 	spin_lock_bh(&card->ip_lock);
 
 	hash_for_each_safe(card->ip_htable, i, tmp, addr, hnode) {
-		if (addr->disp_flag == QETH_DISP_ADDR_DELETE) {
-			qeth_l3_deregister_addr_entry(card, addr);
-			hash_del(&addr->hnode);
-			kfree(addr);
-		} else if (addr->disp_flag == QETH_DISP_ADDR_ADD) {
+		if (addr->disp_flag == QETH_DISP_ADDR_ADD) {
 			if (addr->proto == QETH_PROT_IPV4) {
 				addr->in_progress = 1;
 				spin_unlock_bh(&card->ip_lock);
Signature · 1 line
2.13.5
#3 lore raw

PATCH 3/6 s390/qeth: fix double-free on IP add/remove race

In reply to Julian Wiedmann

Registering an IPv4 address with the HW takes quite a while, so we temporarily drop the ip_htable lock. Any concurrent add/remove of the same IP adjusts the IP's use count, and (on remove) is then blocked by addr->in_progress.
After the register call has completed, we check the use count for concurrently attempted add/remove calls - and possibly straight-away deregister the IP again. This happens via l3_delete_ip(), which
1) looks up the queried IP in the htable (getting a reference to the *same* queried object),
2) deregisters the IP from the HW, and
3) frees the IP object.

The caller in l3_add_ip() then does a second free on the same object.

For this case, skip all the extra checks and lookups in l3_delete_ip() and just deregister & free the IP object ourselves.

Fixes 5f78e29ceebf ("qeth: optimize IP handling in rx_mode callback")
---
 drivers/s390/net/qeth_l3_main.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
drivers/s390/net/qeth_l3_main.c +2 -1
index 34481b51029e..77cdb4fc7721 100644
@@ -237,7 +237,8 @@ int qeth_l3_add_ip(struct qeth_card *card, struct qeth_ipaddr *tmp_addr)
 				(rc == IPA_RC_LAN_OFFLINE)) {
 			addr->disp_flag = QETH_DISP_ADDR_DO_NOTHING;
 			if (addr->ref_counter < 1) {
-				qeth_l3_delete_ip(card, addr);
+				qeth_l3_deregister_addr_entry(card, addr);
+				hash_del(&addr->hnode);
 				kfree(addr);
 			}
 		} else {
Signature · 1 line
2.13.5
#4 lore raw

PATCH 4/6 Revert "s390/qeth: fix using of ref counter for rxip addresses"

In reply to Julian Wiedmann

This reverts commit cb816192d986f7596009dedcf2201fe2e5bc2aa7.

The issue this attempted to fix never actually occurs. l3_add_rxip() checks (via l3_ip_from_hash()) if the requested address was previously added to the card. If so, it returns -EEXIST and doesn't call l3_add_ip().
As a result, the "address exists" path in l3_add_ip() is never taken for rxip addresses, and this patch had no effect.

Fixes cb816192d986 ("s390/qeth: fix using of ref counter for rxip addresses")
---
 drivers/s390/net/qeth_l3_main.c | 8 +++-----
 1 file changed, 3 insertions(+), 5 deletions(-)
drivers/s390/net/qeth_l3_main.c +3 -5
index 77cdb4fc7721..4d8826fec6f4 100644
@@ -167,8 +167,7 @@ int qeth_l3_delete_ip(struct qeth_card *card, struct qeth_ipaddr *tmp_addr)
 		return -ENOENT;
 
 	addr->ref_counter--;
-	if (addr->ref_counter > 0 && (addr->type == QETH_IP_TYPE_NORMAL ||
-				      addr->type == QETH_IP_TYPE_RXIP))
+	if (addr->type == QETH_IP_TYPE_NORMAL && addr->ref_counter > 0)
 		return rc;
 	if (addr->in_progress)
 		return -EINPROGRESS;
@@ -246,9 +245,8 @@ int qeth_l3_add_ip(struct qeth_card *card, struct qeth_ipaddr *tmp_addr)
 			kfree(addr);
 		}
 	} else {
-		if (addr->type == QETH_IP_TYPE_NORMAL ||
-		    addr->type == QETH_IP_TYPE_RXIP)
-			addr->ref_counter++;
+			if (addr->type == QETH_IP_TYPE_NORMAL)
+				addr->ref_counter++;
 	}
 
 	return rc;
Signature · 1 line
2.13.5
#5 lore raw

PATCH 5/6 s390/qeth: fix IP address lookup for L3 devices

In reply to Julian Wiedmann

Current code ("qeth_l3_ip_from_hash()") matches a queried address object against objects in the IP table by IP address, Mask/Prefix Length and MAC address ("qeth_l3_ipaddrs_is_equal()"). But what callers actually require is either
a) "is this IP address registered" (ie. match by IP address only), before adding a new address.
b) or "is this address object registered" (ie. match all relevant attributes), before deleting an address.

Right now
1. the ADD path is too strict in its lookup, and eg. doesn't detect conflicts between an existing NORMAL address and a new VIPA address (because the NORMAL address will have mask != 0, while VIPA has a mask == 0),
2. the DELETE path is not strict enough, and eg. allows del_rxip() to delete a VIPA address as long as the IP address matches.

Fix all this by adding helpers (_addr_match_ip() and _addr_match_all()) that do the appropriate checking.

Note that the ADD path for NORMAL addresses is special, as qeth keeps track of how many times such an address is in use (and there is no immediate way of returning errors to the caller). So when a requested NORMAL address _fully_ matches an existing one, it's not considered a conflict and we merely increment the refcount.

Fixes 5f78e29ceebf ("qeth: optimize IP handling in rx_mode callback")
---
 drivers/s390/net/qeth_l3.h      | 34 ++++++++++++++-
 drivers/s390/net/qeth_l3_main.c | 91 +++++++++++++++++++----------------------
 2 files changed, 74 insertions(+), 51 deletions(-)
drivers/s390/net/qeth_l3.h +33 -1
index bdd45f4dcace..498fe9af2cdb 100644
@@ -40,8 +40,40 @@ struct qeth_ipaddr {
 			unsigned int pfxlen;
 		} a6;
 	} u;
-
 };
+
+static inline bool qeth_l3_addr_match_ip(struct qeth_ipaddr *a1,
+					 struct qeth_ipaddr *a2)
+{
+	if (a1->proto != a2->proto)
+		return false;
+	if (a1->proto == QETH_PROT_IPV6)
+		return ipv6_addr_equal(&a1->u.a6.addr, &a2->u.a6.addr);
+	return a1->u.a4.addr == a2->u.a4.addr;
+}
+
+static inline bool qeth_l3_addr_match_all(struct qeth_ipaddr *a1,
+					  struct qeth_ipaddr *a2)
+{
+	/* Assumes that the pair was obtained via qeth_l3_addr_find_by_ip(),
+	 * so 'proto' and 'addr' match for sure.
+	 *
+	 * For ucast:
+	 * -	'mac' is always 0.
+	 * -	'mask'/'pfxlen' for RXIP/VIPA is always 0. For NORMAL, matching
+	 *	values are required to avoid mixups in takeover eligibility.
+	 *
+	 * For mcast,
+	 * -	'mac' is mapped from the IP, and thus always matches.
+	 * -	'mask'/'pfxlen' is always 0.
+	 */
+	if (a1->type != a2->type)
+		return false;
+	if (a1->proto == QETH_PROT_IPV6)
+		return a1->u.a6.pfxlen == a2->u.a6.pfxlen;
+	return a1->u.a4.mask == a2->u.a4.mask;
+}
+
 static inline  u64 qeth_l3_ipaddr_hash(struct qeth_ipaddr *addr)
 {
 	u64  ret = 0;
drivers/s390/net/qeth_l3_main.c +41 -50

154 lines not on this page · raw message

Signature · 1 line
2.13.5
#6 lore raw

PATCH 6/6 s390/qeth: fix IPA command submission race

In reply to Julian Wiedmann

If multiple IPA commands are build & sent out concurrently, fill_ipacmd_header() may assign a seqno value to a command that's different from what send_control_data() later assigns to this command's reply.
This is due to other commands passing through send_control_data(), and incrementing card->seqno.ipa along the way.

So one IPA command has no reply that's waiting for its seqno, while some other IPA command has multiple reply objects waiting for it. Only one of those waiting replies wins, and the other(s) times out and triggers a recovery via send_ipa_cmd().

Fix this by making sure that the same seqno value is assigned to a command and its reply object.
Do so immediately before submitting the command & while holding the irq_pending "lock", to produce nicely ascending seqnos.

As a side effect, *all* IPA commands now use a reply object that's waiting for its actual seqno. Previously, early IPA commands that were submitted while the card was still DOWN used the "catch-all" IDX seqno.

---
 drivers/s390/net/qeth_core_main.c | 19 ++++++++++---------
 1 file changed, 10 insertions(+), 9 deletions(-)
drivers/s390/net/qeth_core_main.c +10 -9
index 30457fca30c5..c8b308cfabf1 100644
@@ -2134,24 +2134,25 @@ int qeth_send_control_data(struct qeth_card *card, int len,
 	}
 	reply->callback = reply_cb;
 	reply->param = reply_param;
-	if (card->state == CARD_STATE_DOWN)
-		reply->seqno = QETH_IDX_COMMAND_SEQNO;
-	else
-		reply->seqno = card->seqno.ipa++;
+
 	init_waitqueue_head(&reply->wait_q);
-	spin_lock_irqsave(&card->lock, flags);
-	list_add_tail(&reply->list, &card->cmd_waiter_list);
-	spin_unlock_irqrestore(&card->lock, flags);
 
 	while (atomic_cmpxchg(&card->write.irq_pending, 0, 1)) ;
-	qeth_prepare_control_data(card, len, iob);
 
 	if (IS_IPA(iob->data)) {
 		cmd = __ipa_cmd(iob);
+		cmd->hdr.seqno = card->seqno.ipa++;
+		reply->seqno = cmd->hdr.seqno;
 		event_timeout = QETH_IPA_TIMEOUT;
 	} else {
+		reply->seqno = QETH_IDX_COMMAND_SEQNO;
 		event_timeout = QETH_TIMEOUT;
 	}
+	qeth_prepare_control_data(card, len, iob);
+
+	spin_lock_irqsave(&card->lock, flags);
+	list_add_tail(&reply->list, &card->cmd_waiter_list);
+	spin_unlock_irqrestore(&card->lock, flags);
 
 	timeout = jiffies + event_timeout;
 
@@ -2933,7 +2934,7 @@ static void qeth_fill_ipacmd_header(struct qeth_card *card,
 	memset(cmd, 0, sizeof(struct qeth_ipa_cmd));
 	cmd->hdr.command = command;
 	cmd->hdr.initiator = IPA_CMD_INITIATOR_HOST;
-	cmd->hdr.seqno = card->seqno.ipa;
+	/* cmd->hdr.seqno is set by qeth_send_control_data() */
 	cmd->hdr.adapter_type = qeth_get_ipa_adp_type(card->info.link_type);
 	cmd->hdr.rel_adapter_no = (__u8) card->info.portno;
 	if (card->options.layer2)
Signature · 1 line
2.13.5
maintainer
#7 lore raw
In reply to Julian Wiedmann

From: Julian Wiedmann <[email protected]>
Date: Tue, 27 Feb 2018 18:58:11 +0100

Quoted 7 lines

please apply some more qeth patches for -net and stable.

One patch fixes a performance bug in the TSO path. Then there's several more fixes for IP management on L3 devices - including a revert, so that the subsequent fix cleanly applies to earlier kernels.
The final patch takes care of a race in the control IO code that causes qeth to miss the cmd response, and subsequently trigger device recovery.

Series applied and queued up for -stable, thanks Julian.