s390/qeth: fixes 2018-02-27
Subsystems (4)
cover s390/qeth: fixes 2018-02-27
Hi Dave,
please apply some more qeth patches for -net and stable.
One patch fixes a performance bug in the TSO path. Then there's several more fixes for IP management on L3 devices - including a revert, so that the subsequent fix cleanly applies to earlier kernels.
The final patch takes care of a race in the control IO code that causes qeth to miss the cmd response, and subsequently trigger device recovery.
Thanks,
Julian
Julian Wiedmann (6):
s390/qeth: fix overestimated count of buffer elements s390/qeth: fix IP removal on offline cards
s390/qeth: fix double-free on IP add/remove race Revert "s390/qeth: fix using of ref counter for rxip addresses" s390/qeth: fix IP address lookup for L3 devices s390/qeth: fix IPA command submission race
drivers/s390/net/qeth_core_main.c | 29 +++++---- drivers/s390/net/qeth_l3.h | 34 ++++++++++- drivers/s390/net/qeth_l3_main.c | 123 ++++++++++++++++---------------------- 3 files changed, 102 insertions(+), 84 deletions(-)
Signature · 1 line
2.13.5
PATCH 1/6 s390/qeth: fix overestimated count of buffer elements
qeth_get_elements_for_range() doesn't know how to handle a 0-length range (ie. start == end), and returns 1 when it should return 0. Such ranges occur on TSO skbs, where the L2/L3/L4 headers (and thus all of the skb's linear data) are skipped when mapping the skb into regular buffer elements.
This overestimation may cause several performance-related issues:
1. sub-optimal IO buffer selection, where the next buffer gets selected even though the skb would actually still fit into the current buffer.
2. forced linearization, if the element count for a non-linear skb exceeds QETH_MAX_BUFFER_ELEMENTS.
Rather than modifying qeth_get_elements_for_range() and adding overhead to every caller, fix up those callers that are in risk of passing a 0-length range.
--- drivers/s390/net/qeth_core_main.c | 10 ++++++---- drivers/s390/net/qeth_l3_main.c | 11 ++++++----- 2 files changed, 12 insertions(+), 9 deletions(-)
drivers/s390/net/qeth_core_main.c +6 -4
@@ -3898,10 +3898,12 @@ EXPORT_SYMBOL_GPL(qeth_get_elements_for_frags);int qeth_get_elements_no (struct qeth_card *card,struct sk_buff *skb,int extra_elems,int data_offset) { -int elements =qeth_get_elements_for_range ( - (addr_t )skb->data + data_offset, - (addr_t )skb->data +skb_headlen (skb)) + -qeth_get_elements_for_frags (skb); +addr_t end = (addr_t )skb->data +skb_headlen (skb); +int elements =qeth_get_elements_for_frags (skb); +addr_t start = (addr_t )skb->data + data_offset; + +if (start != end) + elements +=qeth_get_elements_for_range (start, end);if ((elements + extra_elems) >QETH_MAX_BUFFER_ELEMENTS (card)) { QETH_DBF_MESSAGE(2, "Invalid size of IP packet "
drivers/s390/net/qeth_l3_main.c +6 -5
@@ -2450,11 +2450,12 @@ static void qeth_tso_fill_header(struct qeth_card *card,static int qeth_l3_get_elements_no_tso (struct qeth_card *card,struct sk_buff *skb,int extra_elems) { -addr_t tcpdptr = (addr_t )tcp_hdr (skb) +tcp_hdrlen (skb); -int elements =qeth_get_elements_for_range ( - tcpdptr, - (addr_t )skb->data +skb_headlen (skb)) + -qeth_get_elements_for_frags (skb); +addr_t start = (addr_t )tcp_hdr (skb) +tcp_hdrlen (skb); +addr_t end = (addr_t )skb->data +skb_headlen (skb); +int elements =qeth_get_elements_for_frags (skb); + +if (start != end) + elements +=qeth_get_elements_for_range (start, end);if ((elements + extra_elems) >QETH_MAX_BUFFER_ELEMENTS (card)) { QETH_DBF_MESSAGE(2,
Signature · 1 line
2.13.5
PATCH 2/6 s390/qeth: fix IP removal on offline cards
In reply to Julian WiedmannIf the HW is not reachable, then none of the IPs in qeth's internal table has been registered with the HW yet. So when deleting such an IP, there's no need to stage it for deregistration - just drop it from the table.
This fixes the "add-delete-add" scenario on an offline card, where the the second "add" merely increments the IP's use count. But as the IP is still set to DISP_ADDR_DELETE from the previous "delete" step, l3_recover_ip() won't register it with the HW when the card goes online.
--- drivers/s390/net/qeth_l3_main.c | 14 +++----------- 1 file changed, 3 insertions(+), 11 deletions(-)
drivers/s390/net/qeth_l3_main.c +3 -11
@@ -173,12 +173,8 @@ int qeth_l3_delete_ip(struct qeth_card *card, struct qeth_ipaddr *tmp_addr)if (addr->in_progress)return -EINPROGRESS ; -if (!qeth_card_hw_is_reachable (card)) { - addr->disp_flag =QETH_DISP_ADDR_DELETE ; -return 0 ; - } - - rc =qeth_l3_deregister_addr_entry (card, addr); +if (qeth_card_hw_is_reachable (card)) + rc =qeth_l3_deregister_addr_entry (card, addr);hash_del (&addr->hnode);kfree (addr);
@@ -321,11 +317,7 @@ static void qeth_l3_recover_ip(struct qeth_card *card)spin_lock_bh (&card->ip_lock);hash_for_each_safe (card->ip_htable, i, tmp, addr, hnode) { -if (addr->disp_flag ==QETH_DISP_ADDR_DELETE ) { -qeth_l3_deregister_addr_entry (card, addr); -hash_del (&addr->hnode); -kfree (addr); - }else if (addr->disp_flag ==QETH_DISP_ADDR_ADD ) { +if (addr->disp_flag ==QETH_DISP_ADDR_ADD ) {if (addr->proto ==QETH_PROT_IPV4 ) { addr->in_progress =1 ;spin_unlock_bh (&card->ip_lock);
Signature · 1 line
2.13.5
PATCH 3/6 s390/qeth: fix double-free on IP add/remove race
In reply to Julian WiedmannRegistering an IPv4 address with the HW takes quite a while, so we temporarily drop the ip_htable lock. Any concurrent add/remove of the same IP adjusts the IP's use count, and (on remove) is then blocked by addr->in_progress.
After the register call has completed, we check the use count for concurrently attempted add/remove calls - and possibly straight-away deregister the IP again. This happens via l3_delete_ip(), which
1) looks up the queried IP in the htable (getting a reference to the *same* queried object),
2) deregisters the IP from the HW, and
3) frees the IP object.
The caller in l3_add_ip() then does a second free on the same object.
For this case, skip all the extra checks and lookups in l3_delete_ip() and just deregister & free the IP object ourselves.
--- drivers/s390/net/qeth_l3_main.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
drivers/s390/net/qeth_l3_main.c +2 -1
@@ -237,7 +237,8 @@ int qeth_l3_add_ip(struct qeth_card *card, struct qeth_ipaddr *tmp_addr) (rc == IPA_RC_LAN_OFFLINE)) { addr->disp_flag = QETH_DISP_ADDR_DO_NOTHING; if (addr->ref_counter < 1) { - qeth_l3_delete_ip(card, addr); + qeth_l3_deregister_addr_entry(card, addr); + hash_del(&addr->hnode); kfree(addr); } } else {
Signature · 1 line
2.13.5
PATCH 4/6 Revert "s390/qeth: fix using of ref counter for rxip addresses"
In reply to Julian WiedmannThis reverts commit cb816192d986f7596009dedcf2201fe2e5bc2aa7.
The issue this attempted to fix never actually occurs. l3_add_rxip() checks (via l3_ip_from_hash()) if the requested address was previously added to the card. If so, it returns -EEXIST and doesn't call l3_add_ip().
As a result, the "address exists" path in l3_add_ip() is never taken for rxip addresses, and this patch had no effect.
--- drivers/s390/net/qeth_l3_main.c | 8 +++----- 1 file changed, 3 insertions(+), 5 deletions(-)
drivers/s390/net/qeth_l3_main.c +3 -5
@@ -167,8 +167,7 @@ int qeth_l3_delete_ip(struct qeth_card *card, struct qeth_ipaddr *tmp_addr)return -ENOENT ; addr->ref_counter--; -if (addr->ref_counter >0 && (addr->type ==QETH_IP_TYPE_NORMAL || - addr->type ==QETH_IP_TYPE_RXIP )) +if (addr->type ==QETH_IP_TYPE_NORMAL && addr->ref_counter >0 )return rc;if (addr->in_progress)return -EINPROGRESS ;
@@ -246,9 +245,8 @@ int qeth_l3_add_ip(struct qeth_card *card, struct qeth_ipaddr *tmp_addr)kfree (addr); } }else { -if (addr->type ==QETH_IP_TYPE_NORMAL || - addr->type ==QETH_IP_TYPE_RXIP ) - addr->ref_counter++; +if (addr->type ==QETH_IP_TYPE_NORMAL ) + addr->ref_counter++; }return rc;
Signature · 1 line
2.13.5
PATCH 5/6 s390/qeth: fix IP address lookup for L3 devices
In reply to Julian WiedmannCurrent code ("qeth_l3_ip_from_hash()") matches a queried address object against objects in the IP table by IP address, Mask/Prefix Length and MAC address ("qeth_l3_ipaddrs_is_equal()"). But what callers actually require is either
a) "is this IP address registered" (ie. match by IP address only), before adding a new address.
b) or "is this address object registered" (ie. match all relevant attributes), before deleting an address.
Right now
1. the ADD path is too strict in its lookup, and eg. doesn't detect conflicts between an existing NORMAL address and a new VIPA address (because the NORMAL address will have mask != 0, while VIPA has a mask == 0),
2. the DELETE path is not strict enough, and eg. allows del_rxip() to delete a VIPA address as long as the IP address matches.
Fix all this by adding helpers (_addr_match_ip() and _addr_match_all()) that do the appropriate checking.
Note that the ADD path for NORMAL addresses is special, as qeth keeps track of how many times such an address is in use (and there is no immediate way of returning errors to the caller). So when a requested NORMAL address _fully_ matches an existing one, it's not considered a conflict and we merely increment the refcount.
--- drivers/s390/net/qeth_l3.h | 34 ++++++++++++++- drivers/s390/net/qeth_l3_main.c | 91 +++++++++++++++++++---------------------- 2 files changed, 74 insertions(+), 51 deletions(-)
drivers/s390/net/qeth_l3.h +33 -1
@@ -40,8 +40,40 @@ struct qeth_ipaddr {unsigned int pfxlen; } a6; } u; - }; + +static inline bool qeth_l3_addr_match_ip (struct qeth_ipaddr *a1, +struct qeth_ipaddr *a2) +{ +if (a1->proto != a2->proto) +return false; +if (a1->proto ==QETH_PROT_IPV6 ) +return ipv6_addr_equal (&a1->u.a6.addr, &a2->u.a6.addr); +return a1->u.a4.addr == a2->u.a4.addr; +} + +static inline bool qeth_l3_addr_match_all (struct qeth_ipaddr *a1, +struct qeth_ipaddr *a2) +{ +/* Assumes that the pair was obtained via qeth_l3_addr_find_by_ip(), +* so 'proto' and 'addr' match for sure. +* +* For ucast: +* - 'mac' is always 0. +* - 'mask'/'pfxlen' for RXIP/VIPA is always 0. For NORMAL, matching +* values are required to avoid mixups in takeover eligibility. +* +* For mcast, +* - 'mac' is mapped from the IP, and thus always matches. +* - 'mask'/'pfxlen' is always 0. +*/ +if (a1->type != a2->type) +return false; +if (a1->proto ==QETH_PROT_IPV6 ) +return a1->u.a6.pfxlen == a2->u.a6.pfxlen; +return a1->u.a4.mask == a2->u.a4.mask; +} +static inline u64 qeth_l3_ipaddr_hash (struct qeth_ipaddr *addr) {u64 ret =0 ;
drivers/s390/net/qeth_l3_main.c +41 -50
154 lines not on this page · raw message
Signature · 1 line
2.13.5
PATCH 6/6 s390/qeth: fix IPA command submission race
In reply to Julian WiedmannIf multiple IPA commands are build & sent out concurrently, fill_ipacmd_header() may assign a seqno value to a command that's different from what send_control_data() later assigns to this command's reply.
This is due to other commands passing through send_control_data(), and incrementing card->seqno.ipa along the way.
So one IPA command has no reply that's waiting for its seqno, while some other IPA command has multiple reply objects waiting for it. Only one of those waiting replies wins, and the other(s) times out and triggers a recovery via send_ipa_cmd().
Fix this by making sure that the same seqno value is assigned to a command and its reply object.
Do so immediately before submitting the command & while holding the irq_pending "lock", to produce nicely ascending seqnos.
As a side effect, *all* IPA commands now use a reply object that's waiting for its actual seqno. Previously, early IPA commands that were submitted while the card was still DOWN used the "catch-all" IDX seqno.
--- drivers/s390/net/qeth_core_main.c | 19 ++++++++++--------- 1 file changed, 10 insertions(+), 9 deletions(-)
drivers/s390/net/qeth_core_main.c +10 -9
@@ -2134,24 +2134,25 @@ int qeth_send_control_data(struct qeth_card *card, int len, } reply->callback = reply_cb; reply->param = reply_param; -if (card->state ==CARD_STATE_DOWN ) - reply->seqno =QETH_IDX_COMMAND_SEQNO ; -else - reply->seqno = card->seqno.ipa++; +init_waitqueue_head (&reply->wait_q); -spin_lock_irqsave (&card->lock, flags); -list_add_tail (&reply->list, &card->cmd_waiter_list); -spin_unlock_irqrestore (&card->lock, flags);while (atomic_cmpxchg (&card->write.irq_pending,0 ,1 )) ; -qeth_prepare_control_data (card, len, iob);if (IS_IPA (iob->data)) { cmd =__ipa_cmd (iob); + cmd->hdr.seqno = card->seqno.ipa++; + reply->seqno = cmd->hdr.seqno; event_timeout =QETH_IPA_TIMEOUT ; }else { + reply->seqno =QETH_IDX_COMMAND_SEQNO ; event_timeout =QETH_TIMEOUT ; } +qeth_prepare_control_data (card, len, iob); + +spin_lock_irqsave (&card->lock, flags); +list_add_tail (&reply->list, &card->cmd_waiter_list); +spin_unlock_irqrestore (&card->lock, flags); timeout = jiffies + event_timeout;
@@ -2933,7 +2934,7 @@ static void qeth_fill_ipacmd_header(struct qeth_card *card,memset (cmd,0 ,sizeof (struct qeth_ipa_cmd )); cmd->hdr.command = command; cmd->hdr.initiator =IPA_CMD_INITIATOR_HOST ; - cmd->hdr.seqno = card->seqno.ipa; +/* cmd->hdr.seqno is set by qeth_send_control_data() */ cmd->hdr.adapter_type =qeth_get_ipa_adp_type (card->info.link_type); cmd->hdr.rel_adapter_no = (__u8 ) card->info.portno;if (card->options.layer2)
Signature · 1 line
2.13.5
David Miller
maintainerFrom: Julian Wiedmann <[email protected]>
Date: Tue, 27 Feb 2018 18:58:11 +0100
Quoted 7 lines
please apply some more qeth patches for -net and stable.
One patch fixes a performance bug in the TSO path. Then there's several more fixes for IP management on L3 devices - including a revert, so that the subsequent fix cleanly applies to earlier kernels.
The final patch takes care of a race in the control IO code that causes qeth to miss the cmd response, and subsequently trigger device recovery.
Series applied and queued up for -stable, thanks Julian.