Qemu problems in -next with 's390/spinlock: add niai spinlock hints'
Hi Martin,
my s390 qemu tests in linux-next stopped working a few days ago. Bisect points to commit 's390/spinlock: add niai spinlock hints'.
Looking at the patch, this isn't really surprising; at least to me it looks like the patch is making instructions mandatory which are only available in Z14 CPUs. Does this mean that older s390 CPUs (such as the Z900 used in my qemu tests) are no longer going to be supported in Linux ?
Thanks,
Guenter
Quoted 9 lines · Guenter Roeck
Hi Martin,
my s390 qemu tests in linux-next stopped working a few days ago. Bisect points to commit 's390/spinlock: add niai spinlock hints'.
Looking at the patch, this isn't really surprising; at least to me it looks like the patch is making instructions mandatory which are only available in Z14 CPUs. Does this mean that older s390 CPUs (such as the Z900 used in my qemu tests) are no longer going to be supported in Linux ?
No, that means that the patch has a bug. The NIAI instruction is only available if the execution-hint facility is installed. That facility came with zEC12. Luckily it uses the same facility indicator bit like the miscellaneous-instruction-extensions facility, which we already use anyway if the kernel gets compiled for zEC12. In that case we have early code which verifies if all required facilities to run the kernel are installed, and if not it will print a message to the console and stop the machine.
So the easiest fix would be to generate the NIAI instruction only if the kernel gets compiled for zEC12 or newer.
Quoted 19 lines · On Wed, 26 Jul 2017 07:00:33 +0200 Heiko Carstens <[email protected]>…
Quoted 9 lines · Guenter Roeck
Hi Martin,
my s390 qemu tests in linux-next stopped working a few days ago. Bisect points to commit 's390/spinlock: add niai spinlock hints'.
Looking at the patch, this isn't really surprising; at least to me it looks like the patch is making instructions mandatory which are only available in Z14 CPUs. Does this mean that older s390 CPUs (such as the Z900 used in my qemu tests) are no longer going to be supported in Linux ?
No, that means that the patch has a bug. The NIAI instruction is only available if the execution-hint facility is installed. That facility came with zEC12. Luckily it uses the same facility indicator bit like the miscellaneous-instruction-extensions facility, which we already use anyway if the kernel gets compiled for zEC12. In that case we have early code which verifies if all required facilities to run the kernel are installed, and if not it will print a message to the console and stop the machine.
Quoted 2 lines
So the easiest fix would be to generate the NIAI instruction only if the kernel gets compiled for zEC12 or newer.
Hmm, I though that NIAI is a NOP on older machines. A runtime check for the facility bit is out of the question as the NIAI-7 gets inlined in the spin_unlock code. So yes, the only available fix is to make the NIAI hinting conditional on zEC12. Which is quite ugly as we would need an architecture level set to zEC12 for the distribution kernel to make use of NIAI.
Signature · 4 lines
blue skies, Martin. "Reality continues to ruin my life." - Calvin.
Quoted 25 lines · Martin Schwidefsky
Quoted 19 lines · On Wed, 26 Jul 2017 07:00:33 +0200 Heiko Carstens <[email protected]>…
Quoted 9 lines · Guenter Roeck
Hi Martin,
my s390 qemu tests in linux-next stopped working a few days ago. Bisect points to commit 's390/spinlock: add niai spinlock hints'.
Looking at the patch, this isn't really surprising; at least to me it looks like the patch is making instructions mandatory which are only available in Z14 CPUs. Does this mean that older s390 CPUs (such as the Z900 used in my qemu tests) are no longer going to be supported in Linux ?
No, that means that the patch has a bug. The NIAI instruction is only available if the execution-hint facility is installed. That facility came with zEC12. Luckily it uses the same facility indicator bit like the miscellaneous-instruction-extensions facility, which we already use anyway if the kernel gets compiled for zEC12. In that case we have early code which verifies if all required facilities to run the kernel are installed, and if not it will print a message to the console and stop the machine.
Quoted 2 lines
So the easiest fix would be to generate the NIAI instruction only if the kernel gets compiled for zEC12 or newer.
Quoted 6 lines
Hmm, I though that NIAI is a NOP on older machines. A runtime check for the facility bit is out of the question as the NIAI-7 gets inlined in the
spin_unlockcode. So yes, the only available fix is to make the NIAI hinting conditional on zEC12. Which is quite ugly as we would need an architecture level set to zEC12 for the distribution kernel to make use of NIAI.
Alternatively you could generate a four-byte nop, and replace that at IPL time with the needed NIAI instruction, if the facility is available. Some sort of "alternative" code patching infrastructure that x86 already has. Not sure if it is worth it, however...
Quoted 33 lines · On Wed, 26 Jul 2017 08:05:27 +0200 Heiko Carstens <[email protected]>…
Quoted 25 lines · Martin Schwidefsky
Quoted 19 lines · On Wed, 26 Jul 2017 07:00:33 +0200 Heiko Carstens <[email protected]>…
Quoted 9 lines · Guenter Roeck
Hi Martin,
my s390 qemu tests in linux-next stopped working a few days ago. Bisect points to commit 's390/spinlock: add niai spinlock hints'.
Looking at the patch, this isn't really surprising; at least to me it looks like the patch is making instructions mandatory which are only available in Z14 CPUs. Does this mean that older s390 CPUs (such as the Z900 used in my qemu tests) are no longer going to be supported in Linux ?
No, that means that the patch has a bug. The NIAI instruction is only available if the execution-hint facility is installed. That facility came with zEC12. Luckily it uses the same facility indicator bit like the miscellaneous-instruction-extensions facility, which we already use anyway if the kernel gets compiled for zEC12. In that case we have early code which verifies if all required facilities to run the kernel are installed, and if not it will print a message to the console and stop the machine.
Quoted 2 lines
So the easiest fix would be to generate the NIAI instruction only if the kernel gets compiled for zEC12 or newer.
Quoted 6 lines
Hmm, I though that NIAI is a NOP on older machines. A runtime check for the facility bit is out of the question as the NIAI-7 gets inlined in the
spin_unlockcode. So yes, the only available fix is to make the NIAI hinting conditional on zEC12. Which is quite ugly as we would need an architecture level set to zEC12 for the distribution kernel to make use of NIAI.
Quoted 4 lines
Alternatively you could generate a four-byte nop, and replace that at IPL time with the needed NIAI instruction, if the facility is available. Some sort of "alternative" code patching infrastructure that x86 already has. Not sure if it is worth it, however...
Patching all spin_unlock inlines? There are a lot of callers for this function. We could think about an out-of-line spin_unlock and patch this single function but then we'd loose the advantage of inlining. I do not think it is worthwhile.
I pushed an updated patch to the features branch of s390/linux. Should be in linux-next tomorroy. Thanks.
Signature · 4 lines
blue skies, Martin. "Reality continues to ruin my life." - Calvin.
Quoted 12 lines
Quoted 11 lines
Quoted 6 lines
Hmm, I though that NIAI is a NOP on older machines. A runtime check for the facility bit is out of the question as the NIAI-7 gets inlined in the
spin_unlockcode. So yes, the only available fix is to make the NIAI hinting conditional on zEC12. Which is quite ugly as we would need an architecture level set to zEC12 for the distribution kernel to make use of NIAI.Alternatively you could generate a four-byte nop, and replace that at IPL time with the needed NIAI instruction, if the facility is available. Some sort of "alternative" code patching infrastructure that x86 already has. Not sure if it is worth it, however...
Quoted 7 lines
Patching all
spin_unlockinlines? There are a lot of callers for this function. We could think about an out-of-linespin_unlockand patch this single function but then we'd loose the advantage of inlining. I do not think it is worthwhile.I pushed an updated patch to the features branch of s390/linux. Should be in linux-next tomorroy. Thanks.
The idea would be to generate an extra section that contains the addresses of the to be patched instructions, and how they need to be patched. That section, plus the code that will do the patching could reside in init data/code sections and would be discarded after. So the additional memory overhead after initialization would be close to zero.
However an ifdef is fine as well ;)
Quoted 22 lines · On Wed, 26 Jul 2017 07:40:44 +0200 Martin Schwidefsky <[email protected]>…
Quoted 19 lines · On Wed, 26 Jul 2017 07:00:33 +0200 Heiko Carstens <[email protected]>…
Quoted 9 lines · Guenter Roeck
Hi Martin,
my s390 qemu tests in linux-next stopped working a few days ago. Bisect points to commit 's390/spinlock: add niai spinlock hints'.
Looking at the patch, this isn't really surprising; at least to me it looks like the patch is making instructions mandatory which are only available in Z14 CPUs. Does this mean that older s390 CPUs (such as the Z900 used in my qemu tests) are no longer going to be supported in Linux ?
No, that means that the patch has a bug. The NIAI instruction is only available if the execution-hint facility is installed. That facility came with zEC12. Luckily it uses the same facility indicator bit like the miscellaneous-instruction-extensions facility, which we already use anyway if the kernel gets compiled for zEC12. In that case we have early code which verifies if all required facilities to run the kernel are installed, and if not it will print a message to the console and stop the machine.
Quoted 4 lines
Quoted 2 lines
So the easiest fix would be to generate the NIAI instruction only if the kernel gets compiled for zEC12 or newer.
Hmm, I though that NIAI is a NOP on older machines.
FWIW, the upcoming qemu 2.11 (in tcg mode) will support to activate stfle bit 49 and generate NOPs for NIAI and friends.
If I read the PoP correctly, the only correct way is to check for stfle bit 49 before trying to use NIAI. While a real zEC12 or later probably will have the facility, you can withdraw the feature bit via the cpu model when running under qemu.
Quoted 30 lines · On Wed, 26 Jul 2017 11:40:14 +0200 Cornelia Huck <[email protected]> wrote:
Quoted 22 lines · On Wed, 26 Jul 2017 07:40:44 +0200 Martin Schwidefsky <[email protected]>…
Quoted 19 lines · On Wed, 26 Jul 2017 07:00:33 +0200 Heiko Carstens <[email protected]>…
Quoted 9 lines · Guenter Roeck
Hi Martin,
my s390 qemu tests in linux-next stopped working a few days ago. Bisect points to commit 's390/spinlock: add niai spinlock hints'.
Looking at the patch, this isn't really surprising; at least to me it looks like the patch is making instructions mandatory which are only available in Z14 CPUs. Does this mean that older s390 CPUs (such as the Z900 used in my qemu tests) are no longer going to be supported in Linux ?
No, that means that the patch has a bug. The NIAI instruction is only available if the execution-hint facility is installed. That facility came with zEC12. Luckily it uses the same facility indicator bit like the miscellaneous-instruction-extensions facility, which we already use anyway if the kernel gets compiled for zEC12. In that case we have early code which verifies if all required facilities to run the kernel are installed, and if not it will print a message to the console and stop the machine.
Quoted 4 lines
Quoted 2 lines
So the easiest fix would be to generate the NIAI instruction only if the kernel gets compiled for zEC12 or newer.
Hmm, I though that NIAI is a NOP on older machines.
Quoted 7 lines
FWIW, the upcoming qemu 2.11 (in tcg mode) will support to activate stfle bit 49 and generate NOPs for NIAI and friends.
If I read the PoP correctly, the only correct way is to check for stfle bit 49 before trying to use NIAI. While a real zEC12 or later probably will have the facility, you can withdraw the feature bit via the cpu model when running under qemu.
A kernel compiled for ZEC12 or later will check STFLE bit 49 at boot time and crash if the bit is not set.
Signature · 4 lines
blue skies, Martin. "Reality continues to ruin my life." - Calvin.