sfc: Pad packets to 33 bytes to prevent TX packet parser lockup
Subsystems (2)
PATCH 1/5 sfc: Pad packets to 33 bytes to prevent TX packet parser lockup
The packet parser used in the TX data path for locating checksum fields can lose synchronisation with the TX queue manager when handling packets that look like IPv4 but are too short (17-32 bytes). Work around this by padding to 33 bytes.
--- drivers/net/sfc/tx.c | 8 ++++++++ drivers/net/sfc/workarounds.h | 2 ++ 2 files changed, 10 insertions(+), 0 deletions(-)
drivers/net/sfc/tx.c +8 -0
@@ -162,6 +162,14 @@ static int efx_enqueue_skb(struct efx_tx_queue *tx_queue,/* Get size of the initial fragment */ len =skb_headlen (skb); +/* Pad if necessary */ +if (EFX_WORKAROUND_15592 (efx) && skb->len <=32 ) { +EFX_BUG_ON_PARANOID (skb->data_len); + len =32 +1 ; +if (skb_pad (skb, len - skb->len)) +return NETDEV_TX_OK ; + } + fill_level = tx_queue->insert_count - tx_queue->old_read_count; q_space = efx->type->txd_ring_mask -1 - fill_level;
drivers/net/sfc/workarounds.h +2 -0
@@ -36,6 +36,8 @@#define EFX_WORKAROUND_11482 EFX_WORKAROUND_ALWAYS/* Flush events can take a very long time to appear */ #define EFX_WORKAROUND_11557 EFX_WORKAROUND_ALWAYS +/* Truncated IPv4 packets can confuse the TX packet parser */ +#define EFX_WORKAROUND_15592 EFX_WORKAROUND_ALWAYS/* Spurious parity errors in TSORT buffers */ #define EFX_WORKAROUND_5129 EFX_WORKAROUND_FALCON_A
Signature · 3 lines
Ben Hutchings, Senior Software Engineer, Solarflare Communications Not speaking for my employer; that's the marketing department's job. They asked us to note that Solarflare product names are trademarked.
PATCH 2/5 sfc: Work around unreliable legacy interrupt status
In rare cases, reading the legacy interrupt status register can acknowledge an event queue whose attention flag has not yet been set in the register. Until we service this event queue it will not generate any more interrupts. Therefore, as a secondary check, poll the next slot in each active event queue whose flag is not set.
--- drivers/net/sfc/falcon.c | 26 ++++++++++++++------------ 1 files changed, 14 insertions(+), 12 deletions(-)
drivers/net/sfc/falcon.c +14 -12
@@ -1435,6 +1435,7 @@ static irqreturn_t falcon_legacy_interrupt_b0(int irq, void *dev_id) {struct efx_nic *efx = dev_id;efx_oword_t *int_ker = efx->irq_status.addr; +irqreturn_t result =IRQ_NONE ;struct efx_channel *channel;efx_dword_t reg;u32 queues;
@@ -1449,23 +1450,24 @@ static irqreturn_t falcon_legacy_interrupt_b0(int irq, void *dev_id)if (unlikely (syserr))return falcon_fatal_interrupt (efx); -if (queues ==0 ) -return IRQ_NONE ; - - efx->last_irq_cpu =raw_smp_processor_id (); -EFX_TRACE (efx,"IRQ %d on CPU %d status " EFX_DWORD_FMT "\n" , - irq,raw_smp_processor_id (),EFX_DWORD_VAL (reg)); -/* Schedule processing of any interrupting queues */ - channel = &efx->channel[0 ]; -while (queues) { -if (queues &0x01 ) +efx_for_each_channel (channel, efx) { +if ((queues &1 ) || +falcon_event_present ( +falcon_event (channel, channel->eventq_read_ptr))) {efx_schedule_channel (channel); - channel++; + result =IRQ_HANDLED ; + } queues >>=1 ; } -return IRQ_HANDLED ; +if (result ==IRQ_HANDLED ) { + efx->last_irq_cpu =raw_smp_processor_id (); +EFX_TRACE (efx,"IRQ %d on CPU %d status " EFX_DWORD_FMT "\n" , + irq,raw_smp_processor_id (),EFX_DWORD_VAL (reg)); + } + +return result; }
Signature · 3 lines
Ben Hutchings, Senior Software Engineer, Solarflare Communications Not speaking for my employer; that's the marketing department's job. They asked us to note that Solarflare product names are trademarked.
Re: [PATCH 2/5] sfc: Work around unreliable legacy interrupt status
From: Ben Hutchings <[email protected]>
Date: Fri, 20 Mar 2009 23:26:41 +0000
Quoted 7 lines
In rare cases, reading the legacy interrupt status register can acknowledge an event queue whose attention flag has not yet been set in the register. Until we service this event queue it will not generate any more interrupts. Therefore, as a secondary check, poll the next slot in each active event queue whose flag is not set.
Signed-off-by Ben Hutchings <[email protected]>
Applied.
PATCH 3/5 sfc: Remove unused private PCI register definitions
In reply to Ben Hutchings--- drivers/net/sfc/falcon.c | 7 ------- 1 files changed, 0 insertions(+), 7 deletions(-)
drivers/net/sfc/falcon.c +0 -7
@@ -155,13 +155,6 @@ MODULE_PARM_DESC(rx_xon_thresh_bytes, "RX fifo XON threshold");/* Dummy SRAM size code */ #define SRM_NB_BSZ_ONCHIP_ONLY (-1) -/* Be nice if these (or equiv.) were in linux/pci_regs.h, but they're not. */ -#define PCI_EXP_DEVCAP_PWR_VAL_LBN 18 -#define PCI_EXP_DEVCAP_PWR_SCL_LBN 26 -#define PCI_EXP_DEVCTL_PAYLOAD_LBN 5 -#define PCI_EXP_LNKSTA_LNK_WID 0x3f0 -#define PCI_EXP_LNKSTA_LNK_WID_LBN 4 -#define FALCON_IS_DUAL_FUNC (efx) \ (falcon_rev(efx) < FALCON_REV_B0)
Signature · 3 lines
Ben Hutchings, Senior Software Engineer, Solarflare Communications Not speaking for my employer; that's the marketing department's job. They asked us to note that Solarflare product names are trademarked.
Re: [PATCH 3/5] sfc: Remove unused private PCI register definitions
From: Ben Hutchings <[email protected]>
Date: Fri, 20 Mar 2009 23:26:55 +0000
Quoted 1 line
Signed-off-by Ben Hutchings <[email protected]>
Applied.
PATCH 4/5 sfc: Optimise falcon_writel_page_locked() for page > 0
In reply to Ben HutchingsThe bug this function works around only applies to the first set of page-mapped registers; other pages can be written without locking.
--- drivers/net/sfc/falcon_io.h | 13 ++++++++----- 1 files changed, 8 insertions(+), 5 deletions(-)
drivers/net/sfc/falcon_io.h +8 -5
@@ -238,18 +238,21 @@ static inline void falcon_writel_page(struct efx_nic *efx, efx_dword_t *value,/* Write dword to Falcon page-mapped register with an extra lock. * * As for falcon_writel_page(), but for a register that suffers from -* SFC bug 3181. Take out a lock so the BIU collector cannot be -* confused. */ +* SFC bug 3181. If writing to page 0, take out a lock so the BIU +* collector cannot be confused. +*/ static inline void falcon_writel_page_locked (struct efx_nic *efx,efx_dword_t *value,unsigned int reg,unsigned int page) { -unsigned long flags; +unsigned long flags =0 ; -spin_lock_irqsave (&efx->biu_lock, flags); +if (page ==0 ) +spin_lock_irqsave (&efx->biu_lock, flags);falcon_writel (efx, value,FALCON_PAGED_REG (page, reg)); -spin_unlock_irqrestore (&efx->biu_lock, flags); +if (page ==0 ) +spin_unlock_irqrestore (&efx->biu_lock, flags); }#endif /* EFX_FALCON_IO_H */
Signature · 3 lines
Ben Hutchings, Senior Software Engineer, Solarflare Communications Not speaking for my employer; that's the marketing department's job. They asked us to note that Solarflare product names are trademarked.
Re: [PATCH 4/5] sfc: Optimise falcon_writel_page_locked() for page > 0
From: Ben Hutchings <[email protected]>
Date: Fri, 20 Mar 2009 23:27:13 +0000
Quoted 4 lines
The bug this function works around only applies to the first set of page-mapped registers; other pages can be written without locking.
Signed-off-by Ben Hutchings <[email protected]>
Applied.
PATCH 5/5 sfc: Implement adaptive IRQ moderation
In reply to Ben HutchingsCalculate a score for each 1000 IRQs:
- TX completions are worth 1 point
- RX completions are worth 4 if merged using LRO or 2 otherwise
Reduce moderation if the score is less than 10000, down to a minimum of 5 us. Increase moderation if the score is more than 20000, up to the specified maximum.
--- drivers/net/sfc/efx.c | 46 ++++++++++++++++++++++++++++++++++++++++- drivers/net/sfc/efx.h | 2 +- drivers/net/sfc/ethtool.c | 19 +++++----------- drivers/net/sfc/falcon.c | 16 ++++++++++---- drivers/net/sfc/falcon.h | 2 + drivers/net/sfc/net_driver.h | 9 ++++++++ 6 files changed, 73 insertions(+), 21 deletions(-)
drivers/net/sfc/efx.c +44 -2
@@ -133,6 +133,16 @@ static int phy_flash_cfg;module_param (phy_flash_cfg, int,0644 );MODULE_PARM_DESC (phy_flash_cfg,"Set PHYs into reflash mode initially" ); +static unsigned irq_adapt_low_thresh =10000 ; +module_param (irq_adapt_low_thresh, uint,0644 ); +MODULE_PARM_DESC (irq_adapt_low_thresh, +"Threshold score for reducing IRQ moderation" ); + +static unsigned irq_adapt_high_thresh =20000 ; +module_param (irq_adapt_high_thresh, uint,0644 ); +MODULE_PARM_DESC (irq_adapt_high_thresh, +"Threshold score for increasing IRQ moderation" ); +/************************************************************************** * * Utility functions and prototypes
@@ -223,6 +233,35 @@ static int efx_poll(struct napi_struct *napi, int budget) rx_packets =efx_process_channel (channel, budget);if (rx_packets < budget) { +struct efx_nic *efx = channel->efx; + +if (channel->used_flags &EFX_USED_BY_RX && + efx->irq_rx_adaptive && +unlikely (++channel->irq_count ==1000 )) { +unsigned old_irq_moderation = channel->irq_moderation; + +if (unlikely (channel->irq_mod_score < + irq_adapt_low_thresh)) { + channel->irq_moderation = +max_t (int, + channel->irq_moderation - +FALCON_IRQ_MOD_RESOLUTION , +FALCON_IRQ_MOD_RESOLUTION ); + }else if (unlikely (channel->irq_mod_score > + irq_adapt_high_thresh)) { + channel->irq_moderation = +min (channel->irq_moderation + +FALCON_IRQ_MOD_RESOLUTION , + efx->irq_rx_moderation); + } + +if (channel->irq_moderation != old_irq_moderation) +falcon_set_int_moderation (channel); + + channel->irq_count =0 ; + channel->irq_mod_score =0 ; + } +/* There is no race here; although napi_disable() will * only wait for napi_complete(), this isn't a problem * since efx_channel_processed() will have no effect if
@@ -991,7 +1030,7 @@ static int efx_probe_nic(struct efx_nic *efx)efx_set_channels (efx);/* Initialise the interrupt moderation settings */ -efx_init_irq_moderation (efx, tx_irq_mod_usec, rx_irq_mod_usec); +efx_init_irq_moderation (efx, tx_irq_mod_usec, rx_irq_mod_usec, true);return 0 ; }
@@ -1188,7 +1227,8 @@ void efx_flush_queues(struct efx_nic *efx)**************************************************************************/ /* Set interrupt moderation parameters */ -void efx_init_irq_moderation (struct efx_nic *efx,int tx_usecs,int rx_usecs) +void efx_init_irq_moderation (struct efx_nic *efx,int tx_usecs,int rx_usecs, +bool rx_adaptive) {struct efx_tx_queue *tx_queue;struct efx_rx_queue *rx_queue;
@@ -1198,6 +1238,8 @@ void efx_init_irq_moderation(struct efx_nic *efx, int tx_usecs, int rx_usecs)efx_for_each_tx_queue (tx_queue, efx) tx_queue->channel->irq_moderation = tx_usecs; + efx->irq_rx_adaptive = rx_adaptive; + efx->irq_rx_moderation = rx_usecs;efx_for_each_rx_queue (rx_queue, efx) rx_queue->channel->irq_moderation = rx_usecs; }
drivers/net/sfc/efx.h +1 -1
@@ -52,7 +52,7 @@ extern void efx_schedule_reset(struct efx_nic *efx, enum reset_type type);extern void efx_suspend (struct efx_nic *efx);extern void efx_resume (struct efx_nic *efx);extern void efx_init_irq_moderation (struct efx_nic *efx,int tx_usecs, -int rx_usecs); +int rx_usecs,bool rx_adaptive);extern int efx_request_power (struct efx_nic *efx,int mw,const char *name);extern void efx_hex_dump (const u8 *,unsigned int ,const char *);
drivers/net/sfc/ethtool.c +6 -13
@@ -604,7 +604,6 @@ static int efx_ethtool_get_coalesce(struct net_device *net_dev, {struct efx_nic *efx =netdev_priv (net_dev);struct efx_tx_queue *tx_queue; -struct efx_rx_queue *rx_queue;struct efx_channel *channel;memset (coalesce,0 ,sizeof (*coalesce));
@@ -622,14 +621,8 @@ static int efx_ethtool_get_coalesce(struct net_device *net_dev, } } -/* Find lowest IRQ moderation across all used RX queues */ - coalesce->rx_coalesce_usecs_irq = ~((u32 )0 ); -efx_for_each_rx_queue (rx_queue, efx) { - channel = rx_queue->channel; -if (channel->irq_moderation < coalesce->rx_coalesce_usecs_irq) - coalesce->rx_coalesce_usecs_irq = - channel->irq_moderation; - } + coalesce->use_adaptive_rx_coalesce = efx->irq_rx_adaptive; + coalesce->rx_coalesce_usecs_irq = efx->irq_rx_moderation;return 0 ; }
@@ -643,10 +636,9 @@ static int efx_ethtool_set_coalesce(struct net_device *net_dev,struct efx_nic *efx =netdev_priv (net_dev);struct efx_channel *channel;struct efx_tx_queue *tx_queue; -unsigned tx_usecs, rx_usecs; +unsigned tx_usecs, rx_usecs, adaptive; -if (coalesce->use_adaptive_rx_coalesce || - coalesce->use_adaptive_tx_coalesce) +if (coalesce->use_adaptive_tx_coalesce)return -EOPNOTSUPP ;if (coalesce->rx_coalesce_usecs || coalesce->tx_coalesce_usecs) {
@@ -657,6 +649,7 @@ static int efx_ethtool_set_coalesce(struct net_device *net_dev, rx_usecs = coalesce->rx_coalesce_usecs_irq; tx_usecs = coalesce->tx_coalesce_usecs_irq; + adaptive = coalesce->use_adaptive_rx_coalesce;/* If the channel is shared only allow RX parameters to be set */ efx_for_each_tx_queue (tx_queue, efx) {
@@ -668,7 +661,7 @@ static int efx_ethtool_set_coalesce(struct net_device *net_dev, } } -efx_init_irq_moderation (efx, tx_usecs, rx_usecs); +efx_init_irq_moderation (efx, tx_usecs, rx_usecs, adaptive);/* Reset channel to pick up new moderation value. Note that * this may change the value of the irq_moderation field
drivers/net/sfc/falcon.c +11 -5
@@ -729,6 +729,9 @@ static void falcon_handle_tx_event(struct efx_channel *channel, tx_ev_desc_ptr =EFX_QWORD_FIELD (*event,TX_EV_DESC_PTR ); tx_ev_q_label =EFX_QWORD_FIELD (*event,TX_EV_Q_LABEL ); tx_queue = &efx->tx_queue[tx_ev_q_label]; + channel->irq_mod_score += + (tx_ev_desc_ptr - tx_queue->read_count) & + efx->type->txd_ring_mask;efx_xmit_done (tx_queue, tx_ev_desc_ptr); }else if (EFX_QWORD_FIELD (*event,TX_EV_WQ_FF_FULL )) {/* Rewrite the FIFO write pointer */
@@ -898,6 +901,8 @@ static void falcon_handle_rx_event(struct efx_channel *channel, discard = true; } + channel->irq_mod_score +=2 ; +/* Handle received packet */ efx_rx_packet (rx_queue, rx_ev_desc_ptr, rx_ev_byte_cnt, checksummed, discard);
@@ -1075,14 +1080,15 @@ void falcon_set_int_moderation(struct efx_channel *channel)* program is based at 0. So actual interrupt moderation * achieved is ((x + 1) * res). */ -unsigned int res =5 ; - channel->irq_moderation -= (channel->irq_moderation % res); -if (channel->irq_moderation < res) - channel->irq_moderation = res; + channel->irq_moderation -= (channel->irq_moderation % +FALCON_IRQ_MOD_RESOLUTION ); +if (channel->irq_moderation <FALCON_IRQ_MOD_RESOLUTION ) + channel->irq_moderation =FALCON_IRQ_MOD_RESOLUTION ;EFX_POPULATE_DWORD_2 (timer_cmd,TIMER_MODE ,TIMER_MODE_INT_HLDOFF ,TIMER_VAL , - (channel->irq_moderation / res) -1 ); + channel->irq_moderation / +FALCON_IRQ_MOD_RESOLUTION -1 ); }else { EFX_POPULATE_DWORD_2(timer_cmd, TIMER_MODE, TIMER_MODE_DIS,
drivers/net/sfc/falcon.h +2 -0
@@ -85,6 +85,8 @@ extern void falcon_set_int_moderation(struct efx_channel *channel);extern void falcon_disable_interrupts (struct efx_nic *efx);extern void falcon_fini_interrupt (struct efx_nic *efx); +#define FALCON_IRQ_MOD_RESOLUTION 5 +/* Global Resources */ extern int falcon_probe_nic (struct efx_nic *efx);extern int falcon_probe_resources (struct efx_nic *efx);
drivers/net/sfc/net_driver.h +9 -0
@@ -336,6 +336,8 @@ enum efx_rx_alloc_method {* @eventq_read_ptr: Event queue read pointer * @last_eventq_read_ptr: Last event queue read pointer value. * @eventq_magic: Event queue magic value for driver-generated test events +* @irq_count: Number of IRQs since last adaptive moderation decision +* @irq_mod_score: IRQ moderation score * @rx_alloc_level: Watermark based heuristic counter for pushing descriptors * and diagnostic counters * @rx_alloc_push_pages: RX allocation method currently in use for pushing
@@ -364,6 +366,9 @@ struct efx_channel {unsigned int last_eventq_read_ptr;unsigned int eventq_magic; +unsigned int irq_count; +unsigned int irq_mod_score; +int rx_alloc_level;int rx_alloc_push_pages;
@@ -703,6 +708,8 @@ union efx_multicast_hash {* @membase: Memory BAR value * @biu_lock: BIU (bus interface unit) lock * @interrupt_mode: Interrupt mode +* @irq_rx_adaptive: Adaptive IRQ moderation enabled for RX event queues +* @irq_rx_moderation: IRQ moderation time for RX event queues * @i2c_adap: I2C adapter * @board_info: Board-level information * @state: Device state flag. Serialised by the rtnl_lock.
@@ -784,6 +791,8 @@ struct efx_nic {void __iomem *membase;spinlock_t biu_lock;enum efx_int_mode interrupt_mode; +bool irq_rx_adaptive; +unsigned int irq_rx_moderation;struct i2c_adapter i2c_adap;struct efx_board board_info;
Signature · 3 lines
Ben Hutchings, Senior Software Engineer, Solarflare Communications Not speaking for my employer; that's the marketing department's job. They asked us to note that Solarflare product names are trademarked.
Re: [PATCH 5/5] sfc: Implement adaptive IRQ moderation
From: Ben Hutchings <[email protected]>
Date: Fri, 20 Mar 2009 23:30:37 +0000
Quoted 9 lines
Calculate a score for each 1000 IRQs:
- TX completions are worth 1 point
- RX completions are worth 4 if merged using LRO or 2 otherwiseReduce moderation if the score is less than 10000, down to a minimum of 5 us. Increase moderation if the score is more than 20000, up to the specified maximum.
Signed-off-by Ben Hutchings <[email protected]>
Applied.
From: Ben Hutchings <[email protected]>
Date: Fri, 20 Mar 2009 23:25:39 +0000
Quoted 6 lines
The packet parser used in the TX data path for locating checksum fields can lose synchronisation with the TX queue manager when handling packets that look like IPv4 but are too short (17-32 bytes). Work around this by padding to 33 bytes.
Signed-off-by Ben Hutchings <[email protected]>
Applied.