ath9k: SMP fixes
v6 cover ath9k: SMP fixes
This v6 removes the hotplug CPU stuff which although it technically would be correct is more cruft and highly unlikely. The down side is you will get serialization applied even if you only have one CPU but who cares, I doubt those nutty SGI guys are toying with ath9k in the lab.
Yesterday I started mucking with an alternative approach to serialization which worked. I added just 5 udelay()s in key places where we had a lot of consecutive IO read/writes issued and that resolved the issue at least for STA mode but it didn't do the trick for AP mode. Technically it should be possible to groom all hardware access routines to do the same but I'm tired of this issue and want to a fix merged today not a few weeks from now. Anyway if you are curious you can check out that patch replacement approach for serialization here:
http://www.kernel.org/pub/linux/kernel/people/mcgrof/patches/ath9k/2009-03-12/smp-udelay-fix.patch
Note that AP won't work though so if you have a lot of time in your hands and have an SMP box with PCI Atheros 11n and are seeing the hangs and want a neat alternative to the current serialization implementation try adding udelays like the ones in the patch in places where we do a lot of io read/writes. Send me the results :)
The compromise is to keep serialization conditional -- we do not want to do it for every read/write regardless of the type of card you have, the code overhead is not great and we maintain this anyway. Adding it for all cases would just not be optimal.
Luis R. Rodriguez (3):
ath9k: implement IO serialization
ath9k: AR9280 PCI devices must serialize IO as well
ath9k: remove dummy PCI "retry timeout" fix
drivers/net/wireless/ath9k/ath9k.h | 34 ++++++++++++++++++++++++++++++++++ drivers/net/wireless/ath9k/hw.c | 28 +++++++++++++++++++++++++++- drivers/net/wireless/ath9k/hw.h | 4 ++-- drivers/net/wireless/ath9k/main.c | 1 + drivers/net/wireless/ath9k/pci.c | 18 ------------------ 5 files changed, 64 insertions(+), 21 deletions(-)
PATCH v6 1/3 ath9k: implement IO serialization
All 802.11n PCI devices (Cardbus, PCI, mini-PCI) require serialization of IO when on non-uniprocessor systems. PCI express devices not not require this.
This should fix our only last standing open ath9k kernel.org bugzilla bug report:
http://bugzilla.kernel.org/show_bug.cgi?id=12110
A port is probably required to older kernels and I can work on that.
--- drivers/net/wireless/ath9k/ath9k.h | 34 ++++++++++++++++++++++++++++++++++ drivers/net/wireless/ath9k/hw.c | 19 +++++++++++++++++++ drivers/net/wireless/ath9k/hw.h | 4 ++-- drivers/net/wireless/ath9k/main.c | 1 + 4 files changed, 56 insertions(+), 2 deletions(-)
drivers/net/wireless/ath9k/ath9k.h +34 -0
@@ -579,6 +579,7 @@ struct ath_softc {void __iomem *mem;int irq;spinlock_t sc_resetlock; +spinlock_t sc_serial_rw;struct mutex mutex;u8 curbssid[ETH_ALEN ];
@@ -724,4 +725,37 @@ void ath9k_wiphy_pause_all_forced(struct ath_softc *sc,bool ath9k_wiphy_scanning (struct ath_softc *sc);void ath9k_wiphy_work (struct work_struct *work); +/* +* Read and write, they both share the same lock. We do this to serialize +* reads and writes on Atheros 802.11n PCI devices only. This is required +* as the FIFO on these devices can only accept sanely 2 requests. After +* that the device goes bananas. Serializing the reads/writes prevents this +* from happening. +*/ + +static inline void ath9k_iowrite32 (struct ath_hw *ah,u32 reg_offset,u32 val) +{ +if (ah->config.serialize_regmode ==SER_REG_MODE_ON ) { +unsigned long flags; +spin_lock_irqsave (&ah->ah_sc->sc_serial_rw, flags); +iowrite32 (val, ah->ah_sc->mem + reg_offset); +spin_unlock_irqrestore (&ah->ah_sc->sc_serial_rw, flags); + }else +iowrite32 (val, ah->ah_sc->mem + reg_offset); +} + +static inline unsigned int ath9k_ioread32 (struct ath_hw *ah,u32 reg_offset) +{ +u32 val; +if (ah->config.serialize_regmode ==SER_REG_MODE_ON ) { +unsigned long flags; +spin_lock_irqsave (&ah->ah_sc->sc_serial_rw, flags); + val =ioread32 (ah->ah_sc->mem + reg_offset); +spin_unlock_irqrestore (&ah->ah_sc->sc_serial_rw, flags); + }else + val =ioread32 (ah->ah_sc->mem + reg_offset); +return val; +} + +#endif /* ATH9K_H */
drivers/net/wireless/ath9k/hw.c +19 -0
@@ -391,6 +391,25 @@ static void ath9k_hw_set_defaults(struct ath_hw *ah) } ah->config.intr_mitigation =1 ; + +/* +* We need this for PCI devices only (Cardbus, PCI, miniPCI) +* _and_ if on non-uniprocessor systems (Multiprocessor/HT). +* This means we use it for all AR5416 devices, and the few +* minor PCI AR9280 devices out there. +* +* Serialization is required because these devices do not handle +* well the case of two concurrent reads/writes due to the latency +* involved. During one read/write another read/write can be issued +* on another CPU while the previous read/write may still be working +* on our hardware, if we hit this case the hardware poops in a loop. +* We prevent this by serializing reads and writes. +* +* This issue is not present on PCI-Express devices or pre-AR5416 +* devices (legacy, 802.11abg). +*/ +if (num_possible_cpus () >1 ) + ah->config.serialize_regmode =SER_REG_MODE_AUTO ; } static struct ath_hw *ath9k_hw_newstate(u16 devid, struct ath_softc *sc,
drivers/net/wireless/ath9k/hw.h +2 -2
@@ -42,8 +42,8 @@#define AR5416_MAGIC 0x19641014/* Register read/write primitives */ -#define REG_WRITE (_ah, _reg, _val) iowrite32(_val, _ah->ah_sc->mem + _reg) -#define REG_READ (_ah, _reg) ioread32(_ah->ah_sc->mem + _reg) +#define REG_WRITE (_ah, _reg, _val) ath9k_iowrite32((_ah), (_reg), (_val)) +#define REG_READ (_ah, _reg) ath9k_ioread32((_ah), (_reg))#define SM (_v, _f) (((_v) << _f##_S) & _f)#define MS (_v, _f) (((_v) & _f) >> _f##_S)
drivers/net/wireless/ath9k/main.c +1 -0
@@ -1370,6 +1370,7 @@ static int ath_init(u16 devid, struct ath_softc *sc)spin_lock_init (&sc->wiphy_lock);spin_lock_init (&sc->sc_resetlock); +spin_lock_init (&sc->sc_serial_rw);mutex_init (&sc->mutex);tasklet_init (&sc->intr_tq, ath9k_tasklet, (unsigned long )sc); tasklet_init(&sc->bcon_tasklet, ath_beacon_tasklet,
Signature · 1 line
1.6.0.6
PATCH v6 2/3 ath9k: AR9280 PCI devices must serialize IO as well
In reply to Luis R. Rodriguez--- drivers/net/wireless/ath9k/hw.c | 9 ++++++++- 1 files changed, 8 insertions(+), 1 deletions(-)
drivers/net/wireless/ath9k/hw.c +8 -1
@@ -628,8 +628,15 @@ static struct ath_hw *ath9k_hw_do_attach(u16 devid, struct ath_softc *sc, goto bad; } +/* +* All PCI devices should be put here. +* XXX: remove ah->is_pciexpress and use pdev->is_pcie, then +* we can just check for !pdev->is_pcie here, but +* consideration must be taken for handling AHB as well. +*/ if (ah->config.serialize_regmode ==SER_REG_MODE_AUTO ) { -if (ah->hw_version.macVersion ==AR_SREV_VERSION_5416_PCI ) { +if (ah->hw_version.macVersion ==AR_SREV_VERSION_5416_PCI || + (AR_SREV_9280 (ah) && !ah->is_pciexpress)) { ah->config.serialize_regmode =SER_REG_MODE_ON ; }else {
Signature · 1 line
1.6.0.6
PATCH v6 3/3 ath9k: remove dummy PCI "retry timeout" fix
In reply to Luis R. RodriguezRemove the PCI retry timeout code as that was just taken from ipw2100 due to historical reasons but in reality its a no-op, additionally its simply incorrect as each PCI devices has its own custom PCI configuration space on PCI config space >= 0x40. Not to mention we were trying to write 0 to a place that already has 0 on it.
--- drivers/net/wireless/ath9k/pci.c | 18 ------------------ 1 files changed, 0 insertions(+), 18 deletions(-)
drivers/net/wireless/ath9k/pci.c +0 -18
@@ -87,7 +87,6 @@ static int ath_pci_probe(struct pci_dev *pdev, const struct pci_device_id *id)struct ath_softc *sc;struct ieee80211_hw *hw;u8 csz; -u32 val;int ret =0 ;struct ath_hw *ah;
@@ -134,14 +133,6 @@ static int ath_pci_probe(struct pci_dev *pdev, const struct pci_device_id *id)pci_set_master (pdev); -/* -* Disable the RETRY_TIMEOUT register (0x41) to keep -* PCI Tx retries from interfering with C3 CPU state. -*/ -pci_read_config_dword (pdev,0x40 , &val); -if ((val &0x0000ff00 ) !=0 ) -pci_write_config_dword (pdev,0x40 , val &0xffff00ff ); - ret =pci_request_region (pdev,0 ,"ath9k" );if (ret) {dev_err (&pdev->dev,"PCI memory region reserve error\n" );
@@ -253,21 +244,12 @@ static int ath_pci_resume(struct pci_dev *pdev)struct ieee80211_hw *hw =pci_get_drvdata (pdev);struct ath_wiphy *aphy = hw->priv;struct ath_softc *sc = aphy->sc; -u32 val;int err; err =pci_enable_device (pdev);if (err)return err;pci_restore_state (pdev); -/* -* Suspend/Resume resets the PCI configuration space, so we have to -* re-disable the RETRY_TIMEOUT register (0x41) to keep -* PCI Tx retries from interfering with C3 CPU state -*/ -pci_read_config_dword (pdev,0x40 , &val); -if ((val &0x0000ff00 ) !=0 ) -pci_write_config_dword (pdev,0x40 , val &0xffff00ff );/* Enable LED */ ath9k_hw_cfg_output(sc->sc_ah, ATH_LED_PIN,
Signature · 1 line
1.6.0.6
v6 cover ath9k: SMP fixes
In reply to Luis R. RodriguezQuoted 34 lines · Luis R. Rodriguez
This v6 removes the hotplug CPU stuff which although it technically would be correct is more cruft and highly unlikely. The down side is you will get serialization applied even if you only have one CPU but who cares, I doubt those nutty SGI guys are toying with ath9k in the lab.
Yesterday I started mucking with an alternative approach to serialization which worked. I added just 5 udelay()s in key places where we had a lot of consecutive IO read/writes issued and that resolved the issue at least for STA mode but it didn't do the trick for AP mode. Technically it should be possible to groom all hardware access routines to do the same but I'm tired of this issue and want to a fix merged today not a few weeks from now. Anyway if you are curious you can check out that patch replacement approach for serialization here:
http://www.kernel.org/pub/linux/kernel/people/mcgrof/patches/ath9k/2009-03-12/smp-udelay-fix.patch
Note that AP won't work though so if you have a lot of time in your hands and have an SMP box with PCI Atheros 11n and are seeing the hangs and want a neat alternative to the current serialization implementation try adding udelays like the ones in the patch in places where we do a lot of io read/writes. Send me the results :)
The compromise is to keep serialization conditional -- we do not want to do it for every read/write regardless of the type of card you have, the code overhead is not great and we maintain this anyway. Adding it for all cases would just not be optimal.
Luis R. Rodriguez (3):
?ath9k: implement IO serialization
?ath9k: AR9280 PCI devices must serialize IO as well ?ath9k: remove dummy PCI "retry timeout" fix
Quoted 6 lines
?drivers/net/wireless/ath9k/ath9k.h | ? 34 ++++++++++++++++++++++++++++++++++ ?drivers/net/wireless/ath9k/hw.c ? ?| ? 28 +++++++++++++++++++++++++++- ?drivers/net/wireless/ath9k/hw.h ? ?| ? ?4 ++-- ?drivers/net/wireless/ath9k/main.c ?| ? ?1 + ?drivers/net/wireless/ath9k/pci.c ? | ? 18 ------------------?5 files changed, 64 insertions(+), 21 deletions(-)
John, Greg,
these patches require a port down to the other stable kernels, I've taken the time to port only the necessary patches (the first two) to fix the SMP hang on each stable kernel and test them accordingly:
* 2.6.27
* 2.6.28
* 2.6.29
Once merged into wireless-testing the following patches can be used to push into Linus' tree:
Once merged into stable these can be used to merge into 2.6.27 and 2.6.28:
http://www.kernel.org/pub/linux/kernel/people/mcgrof/patches/ath9k/2009-03-12/serialization-v6/for-2.6.27/
http://www.kernel.org/pub/linux/kernel/people/mcgrof/patches/ath9k/2009-03-12/serialization-v6/for-2.6.28/
Just replace:
This is a port of commit SHA1 <BLEH>
With the respective SHA1.
I can also simply resend the ports for 27 and 28 once merged into Linus' tree. Whatever makes it easier for you.
Luis
v6 cover ath9k: SMP fixes
Quoted 63 lines · Luis R. Rodriguez
Quoted 34 lines · Luis R. Rodriguez
This v6 removes the hotplug CPU stuff which although it technically would be correct is more cruft and highly unlikely. The down side is you will get serialization applied even if you only have one CPU but who cares, I doubt those nutty SGI guys are toying with ath9k in the lab.
Yesterday I started mucking with an alternative approach to serialization which worked. I added just 5 udelay()s in key places where we had a lot of consecutive IO read/writes issued and that resolved the issue at least for STA mode but it didn't do the trick for AP mode. Technically it should be possible to groom all hardware access routines to do the same but I'm tired of this issue and want to a fix merged today not a few weeks from now. Anyway if you are curious you can check out that patch replacement approach for serialization here:
http://www.kernel.org/pub/linux/kernel/people/mcgrof/patches/ath9k/2009-03-12/smp-udelay-fix.patch
Note that AP won't work though so if you have a lot of time in your hands and have an SMP box with PCI Atheros 11n and are seeing the hangs and want a neat alternative to the current serialization implementation try adding udelays like the ones in the patch in places where we do a lot of io read/writes. Send me the results :)
The compromise is to keep serialization conditional -- we do not want to do it for every read/write regardless of the type of card you have, the code overhead is not great and we maintain this anyway. Adding it for all cases would just not be optimal.
Luis R. Rodriguez (3):
?ath9k: implement IO serialization
?ath9k: AR9280 PCI devices must serialize IO as well ?ath9k: remove dummy PCI "retry timeout" fixQuoted 6 lines
?drivers/net/wireless/ath9k/ath9k.h | ? 34 ++++++++++++++++++++++++++++++++++ ?drivers/net/wireless/ath9k/hw.c ? ?| ? 28 +++++++++++++++++++++++++++- ?drivers/net/wireless/ath9k/hw.h ? ?| ? ?4 ++-- ?drivers/net/wireless/ath9k/main.c ?| ? ?1 + ?drivers/net/wireless/ath9k/pci.c ? | ? 18 ------------------?5 files changed, 64 insertions(+), 21 deletions(-)
John, Greg,
these patches require a port down to the other stable kernels, I've taken the time to port only the necessary patches (the first two) to fix the SMP hang on each stable kernel and test them accordingly:
* 2.6.27
* 2.6.28
* 2.6.29Once merged into wireless-testing the following patches can be used to push into Linus' tree:
Once merged into stable these can be used to merge into 2.6.27 and 2.6.28:
http://www.kernel.org/pub/linux/kernel/people/mcgrof/patches/ath9k/2009-03-12/serialization-v6/for-2.6.27/
http://www.kernel.org/pub/linux/kernel/people/mcgrof/patches/ath9k/2009-03-12/serialization-v6/for-2.6.28/
Quoted 8 lines
Just replace:
This is a port of commit SHA1 <BLEH>
With the respective SHA1.
I can also simply resend the ports for 27 and 28 once merged into Linus' tree. Whatever makes it easier for you.
Please, when the patches go into Linus's tree, send stable at kernel.org the backported patches, with the proper git commit id in them.
Otherwise it's just too much confusing work for me and I'll easily mess it up. The number of -stable patches is increasing and I need all the help I can get in handling this in a sane manner :)
thanks,
greg k-h
v6 cover ath9k: SMP fixes
Quoted 73 lines · Greg KH
Quoted 63 lines · Luis R. Rodriguez
Quoted 34 lines · Luis R. Rodriguez
This v6 removes the hotplug CPU stuff which although it technically would be correct is more cruft and highly unlikely. The down side is you will get serialization applied even if you only have one CPU but who cares, I doubt those nutty SGI guys are toying with ath9k in the lab.
Yesterday I started mucking with an alternative approach to serialization which worked. I added just 5 udelay()s in key places where we had a lot of consecutive IO read/writes issued and that resolved the issue at least for STA mode but it didn't do the trick for AP mode. Technically it should be possible to groom all hardware access routines to do the same but I'm tired of this issue and want to a fix merged today not a few weeks from now. Anyway if you are curious you can check out that patch replacement approach for serialization here:
http://www.kernel.org/pub/linux/kernel/people/mcgrof/patches/ath9k/2009-03-12/smp-udelay-fix.patch
Note that AP won't work though so if you have a lot of time in your hands and have an SMP box with PCI Atheros 11n and are seeing the hangs and want a neat alternative to the current serialization implementation try adding udelays like the ones in the patch in places where we do a lot of io read/writes. Send me the results :)
The compromise is to keep serialization conditional -- we do not want to do it for every read/write regardless of the type of card you have, the code overhead is not great and we maintain this anyway. Adding it for all cases would just not be optimal.
Luis R. Rodriguez (3):
ath9k: implement IO serialization
ath9k: AR9280 PCI devices must serialize IO as well
ath9k: remove dummy PCI "retry timeout" fixQuoted 6 lines
drivers/net/wireless/ath9k/ath9k.h | 34 ++++++++++++++++++++++++++++++++++ drivers/net/wireless/ath9k/hw.c | 28 +++++++++++++++++++++++++++- drivers/net/wireless/ath9k/hw.h | 4 ++-- drivers/net/wireless/ath9k/main.c | 1 + drivers/net/wireless/ath9k/pci.c | 18 ------------------ 5 files changed, 64 insertions(+), 21 deletions(-)John, Greg,
these patches require a port down to the other stable kernels, I've taken the time to port only the necessary patches (the first two) to fix the SMP hang on each stable kernel and test them accordingly:
* 2.6.27
* 2.6.28
* 2.6.29Once merged into wireless-testing the following patches can be used to push into Linus' tree:
Once merged into stable these can be used to merge into 2.6.27 and 2.6.28:
http://www.kernel.org/pub/linux/kernel/people/mcgrof/patches/ath9k/2009-03-12/serialization-v6/for-2.6.27/
http://www.kernel.org/pub/linux/kernel/people/mcgrof/patches/ath9k/2009-03-12/serialization-v6/for-2.6.28/Quoted 8 lines
Just replace:
This is a port of commit SHA1 <BLEH>
With the respective SHA1.
I can also simply resend the ports for 27 and 28 once merged into Linus' tree. Whatever makes it easier for you.
Quoted 6 lines
Please, when the patches go into Linus's tree, send stable at kernel.org the backported patches, with the proper git commit id in them.
Otherwise it's just too much confusing work for me and I'll easily mess it up. The number of -stable patches is increasing and I need all the help I can get in handling this in a sane manner :)
Sure will do thanks,
Luis