lib/Kconfig: remove DEBUG_PER_CPU_MAPS dependency for CPUMASK_OFFSTACK
Subsystems (4)
cover lib/Kconfig: remove DEBUG_PER_CPU_MAPS dependency for CPUMASK_OFFSTACK
We encountered an issue related to NR_CPUMASK_BITS on ARM64. It turned out CPUMASK_OFFSTACK was not enabled and NR_CPUS is set to 4096 in our aarch64 config, so each cpumask operation is unnecessarily expensive esp. on small VMs. There are decent number of cpumask operations in the scheduler. Many of them are on the hot paths. The cumulative effect is quite siginificant. With CONFIG_CPUMASK_OFFSTACKT set, we saw ~10% gain on ApacheBench and a few percentage on redis in a 32-core ARM64 VM with 5.16 kernel.
One may argue why not just set NR_CPUS to smaller values, the thing is, for example, we support VMs with flexible shapes. The customer can specify the number of CPUs ranging from 1 to 160. You cannot just prepare 160 kernel images for each release with one for each shape. And kernel with NR_CPUS=160 doesn't perform well on a 1-CPU VM. It's important that we can set CPUMASK_OFFSTACK=y so that the kernel can dynamically allocate cpumasks based on the actual number of CPUs in the system.
The problem I am trying to address in this patch is currently CPUMASK_OFFSTACK depends on DEBUG_PER_CPU_MAPS except for x86 and it doesn't even show up in kconfig menu as well as kconfig without DEBUG_PER_CPU_MAPS=y. We should remove such outdated, unnecessary dependency.
Of course, I am open to other ideas. And people who are more familar with this matter can shed a light on why this dependency has been kept for so long. My goal here is to determine if DEBUG_PER_CPU_MAPS is absoultely necessary for CPUMASK_OFFSTACK.
RESEND:
- Correct the subject line of cover letter
- Send to more people, esp. gkh for more attention since no one seems to be responsible for this part of code
Libo Chen (1):
lib/Kconfig: remove DEBUG_PER_CPU_MAPS dependency for CPUMASK_OFFSTACK
lib/Kconfig | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
Signature · 7 lines
2.27.0 _______________________________________________ linux-arm-kernel mailing list [email protected] http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
PATCH RESEND 1/1 lib/Kconfig: remove DEBUG_PER_CPU_MAPS dependency for CPUMASK_OFFSTACK
Forcing CPUMASK_OFFSTACK to be conditoned on DEBUG_PER_CPU_MAPS doesn't make a lot of sense nowaday. Even the original patch dating back to 2008, aab46da0520a ("cpumask: Add CONFIG_CPUMASK_OFFSTACK") didn't give any rationale for such dependency.
Nowhere in the code supports the presumption that DEBUG_PER_CPU_MAPS is necessary for CONFIG_CPUMASK_OFFSTACK. Make no mistake, it's good to have DEBUG_PER_CPU_MAPS for debugging purpose or precaution, but it's simply not a hard requirement for CPUMASK_OFFSTACK. Moreover, x86 Kconfig already can set CPUMASK_OFFSTACK=y without DEBUG_PER_CPU_MAPS=y. There is no reason other architectures cannot given the fact that they have even fewer, if any, arch-specific CONFIG_DEBUG_PER_CPU_MAPS code than x86.
--- lib/Kconfig | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
lib/Kconfig +1 -1
@@ -511,7 +511,7 @@ config CHECK_SIGNATURE bool config CPUMASK_OFFSTACK - bool "Force CPU masks off stack" if DEBUG_PER_CPU_MAPS + bool "Force CPU masks off stack" help Use dynamic allocation for cpumask_var_t, instead of putting them on the stack. This is a bit more expensive, but avoids
Signature · 7 lines
2.27.0 _______________________________________________ linux-arm-kernel mailing list [email protected] http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
Hi--
Quoted 24 lines · Libo Chen
Forcing
CPUMASK_OFFSTACKto be conditoned onDEBUG_PER_CPU_MAPSdoesn't make a lot of sense nowaday. Even the original patch dating back to 2008,aab46da0520a("cpumask: AddCONFIG_CPUMASK_OFFSTACK") didn't give any rationale for such dependency.Nowhere in the code supports the presumption that
DEBUG_PER_CPU_MAPSis necessary forCONFIG_CPUMASK_OFFSTACK. Make no mistake, it's good to haveDEBUG_PER_CPU_MAPSfor debugging purpose or precaution, but it's simply not a hard requirement forCPUMASK_OFFSTACK. Moreover, x86Kconfigalready can set CPUMASK_OFFSTACK=y without DEBUG_PER_CPU_MAPS=y. There is no reason other architectures cannot given the fact that they have even fewer, if any, arch-specificCONFIG_DEBUG_PER_CPU_MAPScode than x86.Signed-off-by Libo Chen <[email protected]>--- lib/Kconfig | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)lib/Kconfig +0
-0
lib/Kconfig config CHECK_SIGNATURE · line 514
Quoted 4 lines
lib/Kconfig +0
-1@@ -511,7 +511,7 @@ config CHECK_SIGNATURE bool config CPUMASK_OFFSTACK - bool "Force CPU masks off stack" if DEBUG_PER_CPU_MAPS
This "if" dependency only controls whether the Kconfig symbol's prompt is displayed (presented) in kconfig tools. Removing it makes the prompt always be displayed.
Any architecture could select (should be able to) CPUMASK_OFFSTACK independently of DEBUG_PER_CPU_MAPS.
Is there another problem here?
Quoted 4 lines
lib/Kconfig +1
-0+ bool "Force CPU masks off stack" help Use dynamic allocation for cpumask_var_t, instead of putting them on the stack. This is a bit more expensive, but avoids
Signature · 1 line
~Randy
Hi Randy,
Quoted 29 lines · Randy Dunlap
Hi--
Quoted 26 lines · Libo Chen
Forcing
CPUMASK_OFFSTACKto be conditoned onDEBUG_PER_CPU_MAPSdoesn't make a lot of sense nowaday. Even the original patch dating back to 2008,aab46da0520a("cpumask: AddCONFIG_CPUMASK_OFFSTACK") didn't give any rationale for such dependency.Nowhere in the code supports the presumption that
DEBUG_PER_CPU_MAPSis necessary forCONFIG_CPUMASK_OFFSTACK. Make no mistake, it's good to haveDEBUG_PER_CPU_MAPSfor debugging purpose or precaution, but it's simply not a hard requirement forCPUMASK_OFFSTACK. Moreover, x86Kconfigalready can set CPUMASK_OFFSTACK=y without DEBUG_PER_CPU_MAPS=y. There is no reason other architectures cannot given the fact that they have even fewer, if any, arch-specificCONFIG_DEBUG_PER_CPU_MAPScode than x86.Signed-off-by Libo Chen <[email protected]>--- lib/Kconfig | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)lib/Kconfig +0
-0@@ -511,7 +511,7 @@ config CHECK_SIGNATURE bool
Quoted 8 lines
lib/Kconfig config CHECK_SIGNATURE · line 514
Quoted 2 lines
lib/Kconfig +0
-1config CPUMASK_OFFSTACK - bool "Force CPU masks off stack" if DEBUG_PER_CPU_MAPSThis "if" dependency only controls whether the
Kconfigsymbol's prompt is displayed (presented) in kconfig tools. Removing it makes the prompt always be displayed.Any architecture could select (should be able to)
CPUMASK_OFFSTACKindependently ofDEBUG_PER_CPU_MAPS.
Do you mean changing arch/xxxx/Kconfig to select CPUMASK_OFFSTACK under some config xxx? That will work but it requires code changes for each architecture. But if you are talking about setting CONFIG_CPUMASK_OFFSTACK=y without CONFIG_DEBUG_PER_CPU_MAPS directly in config file, I have tried, it doesn't work.
Libo
Quoted 6 lines
Is there another problem here?
Quoted 4 lines
lib/Kconfig +1
-0+ bool "Force CPU masks off stack" help Use dynamic allocation for cpumask_var_t, instead of putting them on the stack. This is a bit more expensive, but avoids
Hi,
Quoted 37 lines · Libo Chen
Hi Randy,
Quoted 34 lines · Randy Dunlap
Hi--
Quoted 24 lines · Libo Chen
Forcing
CPUMASK_OFFSTACKto be conditoned onDEBUG_PER_CPU_MAPSdoesn't make a lot of sense nowaday. Even the original patch dating back to 2008,aab46da0520a("cpumask: AddCONFIG_CPUMASK_OFFSTACK") didn't give any rationale for such dependency.Nowhere in the code supports the presumption that
DEBUG_PER_CPU_MAPSis necessary forCONFIG_CPUMASK_OFFSTACK. Make no mistake, it's good to haveDEBUG_PER_CPU_MAPSfor debugging purpose or precaution, but it's simply not a hard requirement forCPUMASK_OFFSTACK. Moreover, x86Kconfigalready can set CPUMASK_OFFSTACK=y without DEBUG_PER_CPU_MAPS=y. There is no reason other architectures cannot given the fact that they have even fewer, if any, arch-specificCONFIG_DEBUG_PER_CPU_MAPScode than x86.Signed-off-by Libo Chen <[email protected]>---
lib/Kconfig | 2 +-1 file changed, 1 insertion(+), 1 deletion(-)lib/Kconfig +0
-0@@ -511,7 +511,7 @@ config CHECK_SIGNATUREQuoted 3 lines
bool
configCPUMASK_OFFSTACK
- bool "Force CPU masks off stack" ifDEBUG_PER_CPU_MAPSThis "if" dependency only controls whether the
Kconfigsymbol's prompt is displayed (presented) in kconfig tools. Removing it makes the prompt always be displayed.
Quoted 4 lines
Quoted 2 lines
Any architecture could select (should be able to)
CPUMASK_OFFSTACKindependently ofDEBUG_PER_CPU_MAPS.Do you mean changing
arch/xxxx/Kconfigto selectCPUMASK_OFFSTACKunder some config xxx? That will work but it requires code changes for each architecture. But if you are talking about settingCONFIG_CPUMASK_OFFSTACK=ywithoutCONFIG_DEBUG_PER_CPU_MAPSdirectly in config file, I have tried, it doesn't work.
I'm just talking about the Kconfig change below. Not talking about whatever else it might require per architecture.
But you say you have tried that and it doesn't work. What part of it doesn't work? The Kconfig part or some code execution?
I'll test the Kconfig part of it later (in a few hours).
Quoted 8 lines
Libo
Quoted 6 lines
Is there another problem here?
Quoted 4 lines
+ bool "Force CPU masks off stack"
help
Use dynamic allocation forcpumask_var_t, instead of putting them on the stack. This is a bit more expensive, but avoids
Signature · 1 line
~Randy
Quoted 42 lines · Randy Dunlap
Hi,
Quoted 39 lines · Libo Chen
Hi Randy,
Quoted 36 lines · Randy Dunlap
Hi--
Quoted 24 lines · Libo Chen
Forcing
CPUMASK_OFFSTACKto be conditoned onDEBUG_PER_CPU_MAPSdoesn't make a lot of sense nowaday. Even the original patch dating back to 2008,aab46da0520a("cpumask: AddCONFIG_CPUMASK_OFFSTACK") didn't give any rationale for such dependency.Nowhere in the code supports the presumption that
DEBUG_PER_CPU_MAPSis necessary forCONFIG_CPUMASK_OFFSTACK. Make no mistake, it's good to haveDEBUG_PER_CPU_MAPSfor debugging purpose or precaution, but it's simply not a hard requirement forCPUMASK_OFFSTACK. Moreover, x86Kconfigalready can set CPUMASK_OFFSTACK=y without DEBUG_PER_CPU_MAPS=y. There is no reason other architectures cannot given the fact that they have even fewer, if any, arch-specificCONFIG_DEBUG_PER_CPU_MAPScode than x86.Signed-off-by Libo Chen <[email protected]>--- lib/Kconfig | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)lib/Kconfig +0
-0lib/Kconfig config CHECK_SIGNATURE · line 513
Quoted 3 lines
lib/Kconfig +0
-1@@ -511,7 +511,7 @@ config CHECK_SIGNATURE bool config CPUMASK_OFFSTACK - bool "Force CPU masks off stack" if DEBUG_PER_CPU_MAPSThis "if" dependency only controls whether the
Kconfigsymbol's prompt is displayed (presented) in kconfig tools. Removing it makes the prompt always be displayed.Any architecture could select (should be able to)
CPUMASK_OFFSTACKindependently ofDEBUG_PER_CPU_MAPS.
Quoted 7 lines
Quoted 2 lines
Do you mean changing
arch/xxxx/Kconfigto selectCPUMASK_OFFSTACKunder some config xxx? That will work but it requires code changes for each architecture. But if you are talking about settingCONFIG_CPUMASK_OFFSTACK=ywithoutCONFIG_DEBUG_PER_CPU_MAPSdirectly in config file, I have tried, it doesn't work.I'm just talking about the
Kconfigchange below. Not talking about whatever else it might require per architecture.But you say you have tried that and it doesn't work. What part of it doesn't work? The
Kconfigpart or some code execution?
oh the Kconfig part. For example, make olddefconfig on a config file with CPUMASK_OFFSTACK=y only turns off CPUMASK_OFFSTACK unless I explicitly set DEBUG_PER_CPU_MAPS=y
Libo
Quoted 9 lines
I'll test the
Kconfigpart of it later (in a few hours).Quoted 7 lines
Libo
Quoted 6 lines
Is there another problem here?
Quoted 4 lines
lib/Kconfig +1
-0+ bool "Force CPU masks off stack" help Use dynamic allocation for cpumask_var_t, instead of putting them on the stack. This is a bit more expensive, but avoids
Hi Libo,
Quoted 45 lines · Libo Chen
Quoted 42 lines · Randy Dunlap
Hi,
Quoted 39 lines · Libo Chen
Hi Randy,
Quoted 36 lines · Randy Dunlap
Hi--
Quoted 24 lines · Libo Chen
Forcing
CPUMASK_OFFSTACKto be conditoned onDEBUG_PER_CPU_MAPSdoesn't make a lot of sense nowaday. Even the original patch dating back to 2008,aab46da0520a("cpumask: AddCONFIG_CPUMASK_OFFSTACK") didn't give any rationale for such dependency.Nowhere in the code supports the presumption that
DEBUG_PER_CPU_MAPSis necessary forCONFIG_CPUMASK_OFFSTACK. Make no mistake, it's good to haveDEBUG_PER_CPU_MAPSfor debugging purpose or precaution, but it's simply not a hard requirement forCPUMASK_OFFSTACK. Moreover, x86Kconfigalready can set CPUMASK_OFFSTACK=y without DEBUG_PER_CPU_MAPS=y. There is no reason other architectures cannot given the fact that they have even fewer, if any, arch-specificCONFIG_DEBUG_PER_CPU_MAPScode than x86.Signed-off-by Libo Chen <[email protected]>---
lib/Kconfig | 2 +-1 file changed, 1 insertion(+), 1 deletion(-)lib/Kconfig +0
-0@@ -511,7 +511,7 @@ config CHECK_SIGNATUREQuoted 3 lines
bool
configCPUMASK_OFFSTACK
- bool "Force CPU masks off stack" ifDEBUG_PER_CPU_MAPSThis "if" dependency only controls whether the
Kconfigsymbol's prompt is displayed (presented) in kconfig tools. Removing it makes the prompt always be displayed.Any architecture could select (should be able to)
CPUMASK_OFFSTACKindependently ofDEBUG_PER_CPU_MAPS.
Quoted 8 lines
Quoted 7 lines
Quoted 2 lines
Do you mean changing
arch/xxxx/Kconfigto selectCPUMASK_OFFSTACKunder some config xxx? That will work but it requires code changes for each architecture. But if you are talking about settingCONFIG_CPUMASK_OFFSTACK=ywithoutCONFIG_DEBUG_PER_CPU_MAPSdirectly in config file, I have tried, it doesn't work.I'm just talking about the
Kconfigchange below. Not talking about whatever else it might require per architecture.But you say you have tried that and it doesn't work. What part of it doesn't work? The
Kconfigpart or some code execution?oh the
Kconfigpart. For example, make olddefconfig on a config file with CPUMASK_OFFSTACK=y only turns offCPUMASK_OFFSTACKunless I explicitly set DEBUG_PER_CPU_MAPS=y
I can enable CPUMASK_OFFSTACK for arm64 without having DEBUG_PER_CPU_MAPS enabled. (with a patch, of course.)
It builds OK. I don't know if it will run OK.
I think that you are arguing for a patch like this:
lib/Kconfig +2 -1
@@ -511,7 +511,8 @@ config CHECK_SIGNATURE bool config CPUMASK_OFFSTACK - bool "Force CPU masks off stack" if DEBUG_PER_CPU_MAPS + bool "Force CPU masks off stack" + depends on DEBUG_PER_CPU_MAPS help Use dynamic allocation for cpumask_var_t, instead of putting them on the stack. This is a bit more expensive, but avoids
As I said earlier, the "if" on the "bool" line just controls the prompt message. This patch make CPUMASK_OFFSTACK require DEBUG_PER_CPU_MAPS -- which might be overkill.
Quoted 11 lines
Libo
Quoted 9 lines
I'll test the
Kconfigpart of it later (in a few hours).Quoted 7 lines
Libo
Quoted 6 lines
Is there another problem here?
Quoted 4 lines
+ bool "Force CPU masks off stack"
help
Use dynamic allocation forcpumask_var_t, instead of putting them on the stack. This is a bit more expensive, but avoids
Signature · 1 line
~Randy
Hi Randy
Quoted 52 lines · Randy Dunlap
Hi Libo,
Quoted 49 lines · Libo Chen
Quoted 47 lines · Randy Dunlap
Hi,
Quoted 37 lines · Libo Chen
Hi Randy,
Quoted 34 lines · Randy Dunlap
Hi--
Quoted 24 lines · Libo Chen
Forcing
CPUMASK_OFFSTACKto be conditoned onDEBUG_PER_CPU_MAPSdoesn't make a lot of sense nowaday. Even the original patch dating back to 2008,aab46da0520a("cpumask: AddCONFIG_CPUMASK_OFFSTACK") didn't give any rationale for such dependency.Nowhere in the code supports the presumption that
DEBUG_PER_CPU_MAPSis necessary forCONFIG_CPUMASK_OFFSTACK. Make no mistake, it's good to haveDEBUG_PER_CPU_MAPSfor debugging purpose or precaution, but it's simply not a hard requirement forCPUMASK_OFFSTACK. Moreover, x86Kconfigalready can set CPUMASK_OFFSTACK=y without DEBUG_PER_CPU_MAPS=y. There is no reason other architectures cannot given the fact that they have even fewer, if any, arch-specificCONFIG_DEBUG_PER_CPU_MAPScode than x86.Signed-off-by Libo Chen <[email protected]>--- lib/Kconfig | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)lib/Kconfig +0
-0lib/Kconfig config CHECK_SIGNATURE · line 513
Quoted 3 lines
lib/Kconfig +0
-1@@ -511,7 +511,7 @@ config CHECK_SIGNATURE bool config CPUMASK_OFFSTACK - bool "Force CPU masks off stack" if DEBUG_PER_CPU_MAPSThis "if" dependency only controls whether the
Kconfigsymbol's prompt is displayed (presented) in kconfig tools. Removing it makes the prompt always be displayed.Quoted 4 lines
Quoted 2 lines
Any architecture could select (should be able to)
CPUMASK_OFFSTACKindependently ofDEBUG_PER_CPU_MAPS.Do you mean changing
arch/xxxx/Kconfigto selectCPUMASK_OFFSTACKunder some config xxx? That will work but it requires code changes for each architecture. But if you are talking about settingCONFIG_CPUMASK_OFFSTACK=ywithoutCONFIG_DEBUG_PER_CPU_MAPSdirectly in config file, I have tried, it doesn't work.I'm just talking about the
Kconfigchange below. Not talking about whatever else it might require per architecture.
Quoted 6 lines
Quoted 3 lines
Quoted 2 lines
But you say you have tried that and it doesn't work. What part of it doesn't work? The
Kconfigpart or some code execution?oh the
Kconfigpart. For example, make olddefconfig on a config file with CPUMASK_OFFSTACK=y only turns offCPUMASK_OFFSTACKunless I explicitly set DEBUG_PER_CPU_MAPS=yI can enable
CPUMASK_OFFSTACKfor arm64 without havingDEBUG_PER_CPU_MAPSenabled. (with a patch, of course.)
It builds OK. I don't know if it will run OK.
I am a little confused, did you succeed with your patch (replacing "if" with "depends on") or my patch (removing "if")? Because I definitely cannot enable CPUMASK_OFFSTACK for arm64 without DEBUG_PER_CPUMAPS enabled using your change.
Quoted 1 line
I think that you are arguing for a patch like this:
I am actually arguing for the opposite, I don't think CPUMASK_OFFSTACK should require DEBUG_PER_CPU_MAPS. They should be separate and independent to each other. So removing "if ..." should be enough in my opinion.
lib/Kconfig config CHECK_SIGNATURE · line 515
Quoted 9 lines
lib/Kconfig +2
-1@@ -511,7 +511,8 @@ config CHECK_SIGNATURE bool config CPUMASK_OFFSTACK - bool "Force CPU masks off stack" if DEBUG_PER_CPU_MAPS + bool "Force CPU masks off stack" + depends on DEBUG_PER_CPU_MAPS
This forces every arch to enable DEBUG_PER_CPU_MAPS if they want to enable CPUMASK_OFFSTACK, it's even stronger than "if". My whole argument is CPUMASK_OFFSTACK should be enable/disabled independent of DEBUG_PER_CPU_MASK
Quoted 8 lines
lib/Kconfig +0
-0help Use dynamic allocation for cpumask_var_t, instead of putting them on the stack. This is a bit more expensive, but avoidsAs I said earlier, the "if" on the "bool" line just controls the prompt message. This patch make
CPUMASK_OFFSTACKrequireDEBUG_PER_CPU_MAPS-- which might be overkill.
Okay I understand now "if" on the "boot" is not a dependency and it only controls the prompt message, then the question is why we cannot enable CPUMASK_OFFSTACK without DEBUG_PER_CPU_MAPS if it only controls prompt message? Is it not the behavior we expect?
Libo
Quoted 10 lines
Quoted 10 lines
Libo
Quoted 9 lines
I'll test the
Kconfigpart of it later (in a few hours).Quoted 7 lines
Libo
Quoted 6 lines
Is there another problem here?
Quoted 4 lines
lib/Kconfig +1
-0+ bool "Force CPU masks off stack" help Use dynamic allocation for cpumask_var_t, instead of putting them on the stack. This is a bit more expensive, but avoids
Quoted 92 lines · Libo Chen
Hi Randy
Quoted 52 lines · Randy Dunlap
Hi Libo,
Quoted 49 lines · Libo Chen
Quoted 47 lines · Randy Dunlap
Hi,
Quoted 37 lines · Libo Chen
Hi Randy,
Quoted 34 lines · Randy Dunlap
Hi--
Quoted 24 lines · Libo Chen
Forcing
CPUMASK_OFFSTACKto be conditoned onDEBUG_PER_CPU_MAPSdoesn't make a lot of sense nowaday. Even the original patch dating back to 2008,aab46da0520a("cpumask: AddCONFIG_CPUMASK_OFFSTACK") didn't give any rationale for such dependency.Nowhere in the code supports the presumption that
DEBUG_PER_CPU_MAPSis necessary forCONFIG_CPUMASK_OFFSTACK. Make no mistake, it's good to haveDEBUG_PER_CPU_MAPSfor debugging purpose or precaution, but it's simply not a hard requirement forCPUMASK_OFFSTACK. Moreover, x86Kconfigalready can set CPUMASK_OFFSTACK=y without DEBUG_PER_CPU_MAPS=y. There is no reason other architectures cannot given the fact that they have even fewer, if any, arch-specificCONFIG_DEBUG_PER_CPU_MAPScode than x86.Signed-off-by Libo Chen <[email protected]>--- lib/Kconfig | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)lib/Kconfig +0
-0lib/Kconfig config CHECK_SIGNATURE · line 513
Quoted 3 lines
lib/Kconfig +0
-1@@ -511,7 +511,7 @@ config CHECK_SIGNATURE bool config CPUMASK_OFFSTACK - bool "Force CPU masks off stack" if DEBUG_PER_CPU_MAPSThis "if" dependency only controls whether the
Kconfigsymbol's prompt is displayed (presented) in kconfig tools. Removing it makes the prompt always be displayed.Quoted 4 lines
Quoted 2 lines
Any architecture could select (should be able to)
CPUMASK_OFFSTACKindependently ofDEBUG_PER_CPU_MAPS.Do you mean changing
arch/xxxx/Kconfigto selectCPUMASK_OFFSTACKunder some config xxx? That will work but it requires code changes for each architecture. But if you are talking about settingCONFIG_CPUMASK_OFFSTACK=ywithoutCONFIG_DEBUG_PER_CPU_MAPSdirectly in config file, I have tried, it doesn't work.I'm just talking about the
Kconfigchange below. Not talking about whatever else it might require per architecture.Quoted 6 lines
Quoted 3 lines
Quoted 2 lines
But you say you have tried that and it doesn't work. What part of it doesn't work? The
Kconfigpart or some code execution?oh the
Kconfigpart. For example, make olddefconfig on a config file with CPUMASK_OFFSTACK=y only turns offCPUMASK_OFFSTACKunless I explicitly set DEBUG_PER_CPU_MAPS=yI can enable
CPUMASK_OFFSTACKfor arm64 without havingDEBUG_PER_CPU_MAPSenabled. (with a patch, of course.)
It builds OK. I don't know if it will run OK.I am a little confused, did you succeed with your patch (replacing "if" with "depends on") or my patch (removing "if")? Because I definitely cannot enable
CPUMASK_OFFSTACKfor arm64 withoutDEBUG_PER_CPUMAPSenabled using your change.Quoted 1 line
I think that you are arguing for a patch like this:
I am actually arguing for the opposite, I don't think
CPUMASK_OFFSTACKshould requireDEBUG_PER_CPU_MAPS. They should be separate and independent to each other. So removing "if ..." should be enough in my opinion.lib/Kconfig config CHECK_SIGNATURE · line 515
Quoted 9 lines
lib/Kconfig +2
-1@@ -511,7 +511,8 @@ config CHECK_SIGNATURE bool config CPUMASK_OFFSTACK - bool "Force CPU masks off stack" if DEBUG_PER_CPU_MAPS + bool "Force CPU masks off stack" + depends on DEBUG_PER_CPU_MAPSThis forces every arch to enable
DEBUG_PER_CPU_MAPSif they want to enableCPUMASK_OFFSTACK, it's even stronger than "if". My whole argument isCPUMASK_OFFSTACKshould be enable/disabled independent ofDEBUG_PER_CPU_MASKQuoted 5 lines
lib/Kconfig +0
-0help Use dynamic allocation for cpumask_var_t, instead of putting them on the stack. This is a bit more expensive, but avoids
Quoted 9 lines
Quoted 3 lines
As I said earlier, the "if" on the "bool" line just controls the prompt message. This patch make
CPUMASK_OFFSTACKrequireDEBUG_PER_CPU_MAPS-- which might be overkill.Okay I understand now "if" on the "boot" is not a dependency and it only controls the prompt message, then the question is why we cannot enable
CPUMASK_OFFSTACKwithoutDEBUG_PER_CPU_MAPSif it only controls prompt message? Is it not the behavior we expect?
config CPUMASK_OFFSTACK
bool "Force CPU masks off stack" if DEBUG_PER_CPU_MAPS
... is equivalent to this:
config CPUMASK_OFFSTACK
bool
prompt "Force CPU masks off stack" if DEBUG_PER_CPU_MAPS
When DEBUG_PER_CPU_MAPS is disabled, the prompt line is ignored, and CPUMASK_OFFSTACK becomes a user-unconfigurable option.
Other options still can select it,
but users cannot enable it directly from the prompt.
I see x86 and powerpc do this.
$ kgrep 'select CPUMASK_OFFSTACK' ./arch/x86/Kconfig:946: select CPUMASK_OFFSTACK ./arch/powerpc/Kconfig:164: select CPUMASK_OFFSTACK if NR_CPUS >= 8192
Quoted 13 lines
Libo
Quoted 10 lines
Quoted 10 lines
Libo
Quoted 9 lines
I'll test the
Kconfigpart of it later (in a few hours).Quoted 7 lines
Libo
Quoted 6 lines
Is there another problem here?
Quoted 4 lines
lib/Kconfig +1
-0+ bool "Force CPU masks off stack" help Use dynamic allocation for cpumask_var_t, instead of putting them on the stack. This is a bit more expensive, but avoids
Signature · 2 lines
Best Regards Masahiro Yamada
Quoted 55 lines · Libo Chen
Hi Randy
Quoted 52 lines · Randy Dunlap
Hi Libo,
Quoted 49 lines · Libo Chen
Quoted 47 lines · Randy Dunlap
Hi,
Quoted 37 lines · Libo Chen
Hi Randy,
Quoted 34 lines · Randy Dunlap
Hi--
Quoted 24 lines · Libo Chen
Forcing
CPUMASK_OFFSTACKto be conditoned onDEBUG_PER_CPU_MAPSdoesn't make a lot of sense nowaday. Even the original patch dating back to 2008,aab46da0520a("cpumask: AddCONFIG_CPUMASK_OFFSTACK") didn't give any rationale for such dependency.Nowhere in the code supports the presumption that
DEBUG_PER_CPU_MAPSis necessary forCONFIG_CPUMASK_OFFSTACK. Make no mistake, it's good to haveDEBUG_PER_CPU_MAPSfor debugging purpose or precaution, but it's simply not a hard requirement forCPUMASK_OFFSTACK. Moreover, x86Kconfigalready can set CPUMASK_OFFSTACK=y without DEBUG_PER_CPU_MAPS=y. There is no reason other architectures cannot given the fact that they have even fewer, if any, arch-specificCONFIG_DEBUG_PER_CPU_MAPScode than x86.Signed-off-by Libo Chen <[email protected]>---
lib/Kconfig | 2 +-1 file changed, 1 insertion(+), 1 deletion(-)lib/Kconfig +0
-0@@ -511,7 +511,7 @@ config CHECK_SIGNATUREQuoted 3 lines
bool
configCPUMASK_OFFSTACK
- bool "Force CPU masks off stack" ifDEBUG_PER_CPU_MAPSThis "if" dependency only controls whether the
Kconfigsymbol's prompt is displayed (presented) in kconfig tools. Removing it makes the prompt always be displayed.Quoted 4 lines
Quoted 2 lines
Any architecture could select (should be able to)
CPUMASK_OFFSTACKindependently ofDEBUG_PER_CPU_MAPS.Do you mean changing
arch/xxxx/Kconfigto selectCPUMASK_OFFSTACKunder some config xxx? That will work but it requires code changes for each architecture. But if you are talking about settingCONFIG_CPUMASK_OFFSTACK=ywithoutCONFIG_DEBUG_PER_CPU_MAPSdirectly in config file, I have tried, it doesn't work.I'm just talking about the
Kconfigchange below. Not talking about whatever else it might require per architecture.
Quoted 8 lines
Quoted 6 lines
Quoted 3 lines
Quoted 2 lines
But you say you have tried that and it doesn't work. What part of it doesn't work? The
Kconfigpart or some code execution?oh the
Kconfigpart. For example, make olddefconfig on a config file with CPUMASK_OFFSTACK=y only turns offCPUMASK_OFFSTACKunless I explicitly set DEBUG_PER_CPU_MAPS=yI can enable
CPUMASK_OFFSTACKfor arm64 without havingDEBUG_PER_CPU_MAPSenabled. (with a patch, of course.)
It builds OK. I don't know if it will run OK.I am a little confused, did you succeed with your patch (replacing "if" with "depends on") or my patch (removing "if")? Because I definitely cannot enable
CPUMASK_OFFSTACKfor arm64 withoutDEBUG_PER_CPUMAPSenabled using your change.
This patch builds cleanly for me:
--- arch/arm64/Kconfig | 1 + lib/Kconfig | 2 +- 2 files changed, 2 insertions(+), 1 deletion(-)
lib/Kconfig +1 -1
@@ -511,7 +511,7 @@ config CHECK_SIGNATURE bool config CPUMASK_OFFSTACK - bool "Force CPU masks off stack" if DEBUG_PER_CPU_MAPS + bool "Force CPU masks off stack" help Use dynamic allocation for cpumask_var_t, instead of putting them on the stack. This is a bit more expensive, but avoids
arch/arm64/Kconfig +1 -0
@@ -316,6 +316,7 @@ config ARCH_MHP_MEMMAP_ON_MEMORY_ENABLE config SMP def_bool y + select CPUMASK_OFFSTACK config KERNEL_MODE_NEON def_bool y
along with:
# CONFIG_DEBUG_PER_CPU_MAPS is not set
Quoted 3 lines
Quoted 1 line
I think that you are arguing for a patch like this:
I am actually arguing for the opposite, I don't think
CPUMASK_OFFSTACKshould requireDEBUG_PER_CPU_MAPS. They should be separate and independent to each other. So removing "if ..." should be enough in my opinion.
I agree.
Quoted 15 lines
Quoted 8 lines
lib/Kconfig +0
-0@@ -511,7 +511,8 @@ config CHECK_SIGNATUREbool
configCPUMASK_OFFSTACK
- bool "Force CPU masks off stack" ifDEBUG_PER_CPU_MAPS+ bool "Force CPU masks off stack"
+ depends onDEBUG_PER_CPU_MAPSThis forces every arch to enable
DEBUG_PER_CPU_MAPSif they want to enableCPUMASK_OFFSTACK, it's even stronger than "if". My whole argument isCPUMASK_OFFSTACKshould be enable/disabled independent ofDEBUG_PER_CPU_MASKQuoted 5 lines
help
Use dynamic allocation forcpumask_var_t, instead of putting them on the stack. This is a bit more expensive, but avoids
Quoted 5 lines
Quoted 3 lines
As I said earlier, the "if" on the "bool" line just controls the prompt message. This patch make
CPUMASK_OFFSTACKrequireDEBUG_PER_CPU_MAPS-- which might be overkill.Okay I understand now "if" on the "boot" is not a dependency and it only controls the prompt message, then the question is why we cannot enable
CPUMASK_OFFSTACKwithoutDEBUG_PER_CPU_MAPSif it only controls prompt message? Is it not the behavior we expect?
Yes, it is. I don't know that the problem is...
Signature · 1 line
~Randy
Quoted 114 lines · Randy Dunlap
Quoted 54 lines · Libo Chen
Hi Randy
Quoted 51 lines · Randy Dunlap
Hi Libo,
Quoted 48 lines · Libo Chen
Quoted 47 lines · Randy Dunlap
Hi,
Quoted 37 lines · Libo Chen
Hi Randy,
Quoted 34 lines · Randy Dunlap
Hi--
Quoted 24 lines · Libo Chen
Forcing
CPUMASK_OFFSTACKto be conditoned onDEBUG_PER_CPU_MAPSdoesn't make a lot of sense nowaday. Even the original patch dating back to 2008,aab46da0520a("cpumask: AddCONFIG_CPUMASK_OFFSTACK") didn't give any rationale for such dependency.Nowhere in the code supports the presumption that
DEBUG_PER_CPU_MAPSis necessary forCONFIG_CPUMASK_OFFSTACK. Make no mistake, it's good to haveDEBUG_PER_CPU_MAPSfor debugging purpose or precaution, but it's simply not a hard requirement forCPUMASK_OFFSTACK. Moreover, x86Kconfigalready can set CPUMASK_OFFSTACK=y without DEBUG_PER_CPU_MAPS=y. There is no reason other architectures cannot given the fact that they have even fewer, if any, arch-specificCONFIG_DEBUG_PER_CPU_MAPScode than x86.Signed-off-by Libo Chen <[email protected]>--- lib/Kconfig | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)lib/Kconfig +0
-0lib/Kconfig config CHECK_SIGNATURE · line 513
Quoted 3 lines
lib/Kconfig +0
-1@@ -511,7 +511,7 @@ config CHECK_SIGNATURE bool config CPUMASK_OFFSTACK - bool "Force CPU masks off stack" if DEBUG_PER_CPU_MAPSThis "if" dependency only controls whether the
Kconfigsymbol's prompt is displayed (presented) in kconfig tools. Removing it makes the prompt always be displayed.Quoted 4 lines
Quoted 2 lines
Any architecture could select (should be able to)
CPUMASK_OFFSTACKindependently ofDEBUG_PER_CPU_MAPS.Do you mean changing
arch/xxxx/Kconfigto selectCPUMASK_OFFSTACKunder some config xxx? That will work but it requires code changes for each architecture. But if you are talking about settingCONFIG_CPUMASK_OFFSTACK=ywithoutCONFIG_DEBUG_PER_CPU_MAPSdirectly in config file, I have tried, it doesn't work.I'm just talking about the
Kconfigchange below. Not talking about whatever else it might require per architecture.Quoted 7 lines
Quoted 6 lines
Quoted 3 lines
Quoted 2 lines
But you say you have tried that and it doesn't work. What part of it doesn't work? The
Kconfigpart or some code execution?oh the
Kconfigpart. For example, make olddefconfig on a config file with CPUMASK_OFFSTACK=y only turns offCPUMASK_OFFSTACKunless I explicitly set DEBUG_PER_CPU_MAPS=yI can enable
CPUMASK_OFFSTACKfor arm64 without havingDEBUG_PER_CPU_MAPSenabled. (with a patch, of course.)
It builds OK. I don't know if it will run OK.I am a little confused, did you succeed with your patch (replacing "if" with "depends on") or my patch (removing "if")? Because I definitely cannot enable
CPUMASK_OFFSTACKfor arm64 withoutDEBUG_PER_CPUMAPSenabled using your change.This patch builds cleanly for me:
--- arch/arm64/Kconfig | 1 + lib/Kconfig | 2 +- 2 files changed, 2 insertions(+), 1 deletion(-)lib/Kconfig +1
-1@@ -511,7 +511,7 @@ config CHECK_SIGNATURE bool config CPUMASK_OFFSTACK - bool "Force CPU masks off stack" if DEBUG_PER_CPU_MAPS + bool "Force CPU masks off stack" help Use dynamic allocation for cpumask_var_t, instead of putting them on the stack. This is a bit more expensive, but avoidsarch/arm64/Kconfig +1
-0@@ -316,6 +316,7 @@ config ARCH_MHP_MEMMAP_ON_MEMORY_ENABLE config SMP def_bool y + select CPUMASK_OFFSTACK config KERNEL_MODE_NEON def_bool yalong with:
#CONFIG_DEBUG_PER_CPU_MAPSis not setQuoted 2 lines
Quoted 1 line
I think that you are arguing for a patch like this:
I am actually arguing for the opposite, I don't think
CPUMASK_OFFSTACKshould requireDEBUG_PER_CPU_MAPS. They should be separate and independent to each other. So removing "if ..." should be enough in my opinion.I agree.
Quoted 14 lines
lib/Kconfig config CHECK_SIGNATURE · line 514
Quoted 8 lines
lib/Kconfig +2
-1@@ -511,7 +511,8 @@ config CHECK_SIGNATURE bool config CPUMASK_OFFSTACK - bool "Force CPU masks off stack" if DEBUG_PER_CPU_MAPS + bool "Force CPU masks off stack" + depends on DEBUG_PER_CPU_MAPSThis forces every arch to enable
DEBUG_PER_CPU_MAPSif they want to enableCPUMASK_OFFSTACK, it's even stronger than "if". My whole argument isCPUMASK_OFFSTACKshould be enable/disabled independent ofDEBUG_PER_CPU_MASKQuoted 5 lines
lib/Kconfig +0
-0help Use dynamic allocation for cpumask_var_t, instead of putting them on the stack. This is a bit more expensive, but avoids
Quoted 5 lines
Quoted 4 lines
Quoted 3 lines
As I said earlier, the "if" on the "bool" line just controls the prompt message. This patch make
CPUMASK_OFFSTACKrequireDEBUG_PER_CPU_MAPS-- which might be overkill.Okay I understand now "if" on the "boot" is not a dependency and it only controls the prompt message, then the question is why we cannot enable
CPUMASK_OFFSTACKwithoutDEBUG_PER_CPU_MAPSif it only controls prompt message? Is it not the behavior we expect?Yes, it is. I don't know that the problem is...
Masahiro explained that CPUMASK_OFFSTACK can only be configured by options not users if DEBUG_PER_CPU_MASK is not enabled. This doesn't seem to be what we want.
Libo
Arnd Bergmann
maintainerQuoted 16 lines · Libo Chen
Quoted 15 lines · Randy Dunlap
Quoted 14 lines · Libo Chen
lib/Kconfig config CHECK_SIGNATURE · line 514
Quoted 8 lines
lib/Kconfig +2
-1@@ -511,7 +511,8 @@ config CHECK_SIGNATURE bool config CPUMASK_OFFSTACK - bool "Force CPU masks off stack" if DEBUG_PER_CPU_MAPS + bool "Force CPU masks off stack" + depends on DEBUG_PER_CPU_MAPSThis forces every arch to enable
DEBUG_PER_CPU_MAPSif they want to enableCPUMASK_OFFSTACK, it's even stronger than "if". My whole argument isCPUMASK_OFFSTACKshould be enable/disabled independent ofDEBUG_PER_CPU_MASKQuoted 5 lines
lib/Kconfig +0
-0help Use dynamic allocation for cpumask_var_t, instead of putting them on the stack. This is a bit more expensive, but avoids
Quoted 8 lines
Quoted 5 lines
Quoted 4 lines
Quoted 3 lines
As I said earlier, the "if" on the "bool" line just controls the prompt message. This patch make
CPUMASK_OFFSTACKrequireDEBUG_PER_CPU_MAPS-- which might be overkill.Okay I understand now "if" on the "boot" is not a dependency and it only controls the prompt message, then the question is why we cannot enable
CPUMASK_OFFSTACKwithoutDEBUG_PER_CPU_MAPSif it only controls prompt message? Is it not the behavior we expect?Yes, it is. I don't know that the problem is...
Masahiro explained that
CPUMASK_OFFSTACKcan only be configured by options not users ifDEBUG_PER_CPU_MASKis not enabled. This doesn't seem to be what we want.
I think the correct way to do it is to follow x86 and powerpc, and tying CPUMASK_OFFSTACK to "large" values of CONFIG_NR_CPUS.
For smaller values of NR_CPUS, the onstack masks are obviously cheaper, we just need to decide what the cut-off point is.
In x86, the onstack masks can be selected for normal SMP builds with up to 512 CPUs, while CONFIG_MAXSMP=y raises the limit to 8192 CPUs while selecting CPUMASK_OFFSTACK.
PowerPC does it the other way round, selecting CPUMASK_OFFSTACK implicitly whenever NR_CPUS is set to 8192 or more.
I think we can easily do the same as powerpc on arm64. With the ApacheBench test you cite in the patch description, what is the value of NR_CPUS at which you start seeing a noticeable benefit for offstack masks? Can you do the same test for NR_CPUS=1024 or 2048?
Arnd
Quoted 21 lines · Arnd Bergmann
Quoted 20 lines · Libo Chen
Quoted 19 lines · Randy Dunlap
Quoted 18 lines · Libo Chen
lib/Kconfig config CHECK_SIGNATURE · line 514
Quoted 8 lines
lib/Kconfig +2
-1@@ -511,7 +511,8 @@ config CHECK_SIGNATURE bool config CPUMASK_OFFSTACK - bool "Force CPU masks off stack" if DEBUG_PER_CPU_MAPS + bool "Force CPU masks off stack" + depends on DEBUG_PER_CPU_MAPSThis forces every arch to enable
DEBUG_PER_CPU_MAPSif they want to enableCPUMASK_OFFSTACK, it's even stronger than "if". My whole argument isCPUMASK_OFFSTACKshould be enable/disabled independent ofDEBUG_PER_CPU_MASKQuoted 8 lines
lib/Kconfig +0
-0help Use dynamic allocation for cpumask_var_t, instead of putting them on the stack. This is a bit more expensive, but avoidsAs I said earlier, the "if" on the "bool" line just controls the prompt message. This patch make
CPUMASK_OFFSTACKrequireDEBUG_PER_CPU_MAPS-- which might be overkill.Okay I understand now "if" on the "boot" is not a dependency and it only controls the prompt message, then the question is why we cannot enable
CPUMASK_OFFSTACKwithoutDEBUG_PER_CPU_MAPSif it only controls prompt message? Is it not the behavior we expect?
Quoted 8 lines
Quoted 4 lines
Quoted 1 line
Yes, it is. I don't know that the problem is...
Masahiro explained that
CPUMASK_OFFSTACKcan only be configured by options not users ifDEBUG_PER_CPU_MASKis not enabled. This doesn't seem to be what we want.I think the correct way to do it is to follow x86 and powerpc, and tying
CPUMASK_OFFSTACKto "large" values ofCONFIG_NR_CPUS.
For smaller values ofNR_CPUS, the onstack masks are obviously cheaper, we just need to decide what the cut-off point is.
I agree. It appears enabling CPUMASK_OFFSTACK breaks kernel builds on some architectures such as parisc and nios2 as reported by kernel test robot. Maybe it makes sense to use DEBUG_PER_CPU_MAPS as some kind of guard on CPUMASK_OFFSTACK.
Quoted 7 lines
In x86, the onstack masks can be selected for normal SMP builds with up to 512 CPUs, while
CONFIG_MAXSMP=yraises the limit to 8192 CPUs while selectingCPUMASK_OFFSTACK.
PowerPC does it the other way round, selectingCPUMASK_OFFSTACKimplicitly wheneverNR_CPUSis set to 8192 or more.I think we can easily do the same as powerpc on arm64. With the
I am leaning more towards x86's way because even NR_CPUS=160 is too expensive for 4-core arm64 VMs according to apachebench. I highly doubt that there is a good cut-off point to make everybody happy (or not unhappy).
Quoted 4 lines
ApacheBench test you cite in the patch description, what is the value of
NR_CPUSat which you start seeing a noticeable benefit for offstack masks? Can you do the same test for NR_CPUS=1024 or 2048?
As mentioned above, a good cut-off point moves depends on the actual number of CPUs. But yeah I can do the same test for 1024 or even smaller NR_CPUs values on the same 64-core arm64 VM setup.
Libo
Quoted 2 lines
Arnd
Hi,
Quoted 23 lines · Libo Chen
Quoted 20 lines · Arnd Bergmann
Quoted 19 lines · Libo Chen
Quoted 18 lines · Randy Dunlap
Quoted 17 lines · Libo Chen
Quoted 8 lines
lib/Kconfig +0
-0@@ -511,7 +511,8 @@ config CHECK_SIGNATUREbool
configCPUMASK_OFFSTACK
- bool "Force CPU masks off stack" ifDEBUG_PER_CPU_MAPS+ bool "Force CPU masks off stack"
+ depends onDEBUG_PER_CPU_MAPSThis forces every arch to enable
DEBUG_PER_CPU_MAPSif they want to enableCPUMASK_OFFSTACK, it's even stronger than "if". My whole argument isCPUMASK_OFFSTACKshould be enable/disabled independent ofDEBUG_PER_CPU_MASKQuoted 8 lines
help
Use dynamic allocation forcpumask_var_t, instead of putting them on the stack. This is a bit more expensive, but avoidsAs I said earlier, the "if" on the "bool" line just controls the prompt message. This patch make
CPUMASK_OFFSTACKrequireDEBUG_PER_CPU_MAPS-- which might be overkill.
Quoted 7 lines
Quoted 7 lines
Quoted 5 lines
Quoted 2 lines
Quoted 1 line
Okay I understand now "if" on the "boot" is not a dependency and it only controls the prompt message, then the question is why we cannot enable
CPUMASK_OFFSTACKwithoutDEBUG_PER_CPU_MAPSif it only controls prompt message? Is it not the behavior we expect?Yes, it is. I don't know that the problem is...
Masahiro explained that
CPUMASK_OFFSTACKcan only be configured by options not users ifDEBUG_PER_CPU_MASKis not enabled. This doesn't seem to be what we want.I think the correct way to do it is to follow x86 and powerpc, and tying
CPUMASK_OFFSTACKto "large" values ofCONFIG_NR_CPUS.
Sure. Just FTR, I was just trying to see if an arch (arm64) would build OK or not when CPUMASK_OFFSTACK was enabled. and it does.
My patch wasn't meant to have a very long life.
Quoted 16 lines
Quoted 2 lines
For smaller values of
NR_CPUS, the onstack masks are obviously cheaper, we just need to decide what the cut-off point is.I agree. It appears enabling
CPUMASK_OFFSTACKbreaks kernel builds on some architectures such as parisc and nios2 as reported by kernel test robot. Maybe it makes sense to useDEBUG_PER_CPU_MAPSas some kind of guard onCPUMASK_OFFSTACK.Quoted 7 lines
In x86, the onstack masks can be selected for normal SMP builds with up to 512 CPUs, while
CONFIG_MAXSMP=yraises the limit to 8192 CPUs while selectingCPUMASK_OFFSTACK.
PowerPC does it the other way round, selectingCPUMASK_OFFSTACKimplicitly wheneverNR_CPUSis set to 8192 or more.I think we can easily do the same as powerpc on arm64. With the
I am leaning more towards x86's way because even NR_CPUS=160 is too expensive for 4-core arm64 VMs according to apachebench. I highly doubt that there is a good cut-off point to make everybody happy (or not unhappy).
Quoted 4 lines
ApacheBench test you cite in the patch description, what is the value of
NR_CPUSat which you start seeing a noticeable benefit for offstack masks? Can you do the same test for NR_CPUS=1024 or 2048?As mentioned above, a good cut-off point moves depends on the actual number of CPUs. But yeah I can do the same test for 1024 or even smaller
NR_CPUsvalues on the same 64-core arm64 VM setup.
Signature · 1 line
~Randy
Arnd Bergmann
maintainerQuoted 10 lines · Libo Chen
Quoted 8 lines · Arnd Bergmann
Quoted 4 lines
Quoted 1 line
Yes, it is. I don't know that the problem is...
Masahiro explained that
CPUMASK_OFFSTACKcan only be configured by options not users ifDEBUG_PER_CPU_MASKis not enabled. This doesn't seem to be what we want.I think the correct way to do it is to follow x86 and powerpc, and tying
CPUMASK_OFFSTACKto "large" values ofCONFIG_NR_CPUS.
For smaller values ofNR_CPUS, the onstack masks are obviously cheaper, we just need to decide what the cut-off point is.
Quoted 4 lines
I agree. It appears enabling
CPUMASK_OFFSTACKbreaks kernel builds on some architectures such as parisc and nios2 as reported by kernel test robot. Maybe it makes sense to useDEBUG_PER_CPU_MAPSas some kind of guard onCPUMASK_OFFSTACK.
NIOS2 does not support SMP builds at all, so it should never be possible to select CPUMASK_OFFSTACK there. We may want to guard DEBUG_PER_CPU_MAPS by adding a 'depends on SMP' in order to prevent it from getting selected.
For PARISC, the largest configuration is 32-way SMP, so CPUMASK_OFFSTACK is clearly pointless there as well, even though it should technically be possible
to support. What is the build error on parisc?
Quoted 10 lines
Quoted 7 lines
In x86, the onstack masks can be selected for normal SMP builds with up to 512 CPUs, while
CONFIG_MAXSMP=yraises the limit to 8192 CPUs while selectingCPUMASK_OFFSTACK.
PowerPC does it the other way round, selectingCPUMASK_OFFSTACKimplicitly wheneverNR_CPUSis set to 8192 or more.I think we can easily do the same as powerpc on arm64. With the
I am leaning more towards x86's way because even NR_CPUS=160 is too expensive for 4-core arm64 VMs according to apachebench. I highly doubt that there is a good cut-off point to make everybody happy (or not unhappy).
It seems surprising that you would see any improvement for offstack masks when using NR_CPUS=160, that is just three 64-bit words worth of data, but it requires allocating the mask dynamically, which takes way more memory to initialize.
Quoted 8 lines
Quoted 4 lines
ApacheBench test you cite in the patch description, what is the value of
NR_CPUSat which you start seeing a noticeable benefit for offstack masks? Can you do the same test for NR_CPUS=1024 or 2048?As mentioned above, a good cut-off point moves depends on the actual number of CPUs. But yeah I can do the same test for 1024 or even smaller
NR_CPUsvalues on the same 64-core arm64 VM setup.
If you see an improvement for small NR_CPUS values using offstack masks, it's possible that the actual difference is something completely different and we
can just make the on-stack case faster, possibly the cause is something about cacheline alignment or inlining decisions using your specific kernel config.
Are you able to compare the 'perf report' output between runs with either size to see where the extra time gets spent?
Arnd
Quoted 19 lines · Arnd Bergmann
Quoted 5 lines · Libo Chen
Quoted 4 lines · Arnd Bergmann
Quoted 4 lines
Quoted 1 line
Yes, it is. I don't know that the problem is...
Masahiro explained that
CPUMASK_OFFSTACKcan only be configured by options not users ifDEBUG_PER_CPU_MASKis not enabled. This doesn't seem to be what we want.Quoted 8 lines
Quoted 4 lines
I think the correct way to do it is to follow x86 and powerpc, and tying
CPUMASK_OFFSTACKto "large" values ofCONFIG_NR_CPUS.
For smaller values ofNR_CPUS, the onstack masks are obviously cheaper, we just need to decide what the cut-off point is.I agree. It appears enabling
CPUMASK_OFFSTACKbreaks kernel builds on some architectures such as parisc and nios2 as reported by kernel test robot. Maybe it makes sense to useDEBUG_PER_CPU_MAPSas some kind of guard onCPUMASK_OFFSTACK.NIOS2 does not support SMP builds at all, so it should never be possible to select
CPUMASK_OFFSTACKthere. We may want to guardDEBUG_PER_CPU_MAPSby adding a 'depends on SMP' in order to prevent it from getting selected.
Quoted 4 lines
For PARISC, the largest configuration is 32-way SMP, so
CPUMASK_OFFSTACKis clearly pointless there as well, even though it should technically be possible
to support. What is the build error on parisc?
Similar to NIOS2, A bunch of undefined references to *_cpumask_var() calls. It seems that PARISC doesn't support the cpumask offstack API at all
Quoted 22 lines
Quoted 10 lines
Quoted 7 lines
In x86, the onstack masks can be selected for normal SMP builds with up to 512 CPUs, while
CONFIG_MAXSMP=yraises the limit to 8192 CPUs while selectingCPUMASK_OFFSTACK.
PowerPC does it the other way round, selectingCPUMASK_OFFSTACKimplicitly wheneverNR_CPUSis set to 8192 or more.I think we can easily do the same as powerpc on arm64. With the
I am leaning more towards x86's way because even NR_CPUS=160 is too expensive for 4-core arm64 VMs according to apachebench. I highly doubt that there is a good cut-off point to make everybody happy (or not unhappy).
It seems surprising that you would see any improvement for offstack masks when using NR_CPUS=160, that is just three 64-bit words worth of data, but it requires allocating the mask dynamically, which takes way more memory to initialize.
Quoted 7 lines
Quoted 4 lines
ApacheBench test you cite in the patch description, what is the value of
NR_CPUSat which you start seeing a noticeable benefit for offstack masks? Can you do the same test for NR_CPUS=1024 or 2048?As mentioned above, a good cut-off point moves depends on the actual number of CPUs. But yeah I can do the same test for 1024 or even smaller
NR_CPUsvalues on the same 64-core arm64 VM setup.
Quoted 8 lines
If you see an improvement for small
NR_CPUSvalues using offstack masks, it's possible that the actual difference is something completely different and we
can just make the on-stack case faster, possibly the cause is something about cacheline alignment or inlining decisions using your specific kernel config.Are you able to compare the 'perf report' output between runs with either size to see where the extra time gets spent?
Okay yeah I will take some time to gather more data. It does appear that something else may also contribute to the performance difference.
Thanks
Libo
Quoted 1 line
Arnd
kernel test robot
build botThank you for the patch! Yet something to improve:
Hi Libo,
Thank you for the patch! Yet something to improve:
[auto build test ERROR on linus/master]
[also build test ERROR on v5.18-rc2 next-20220413]
[If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch]
url: https://github.com/intel-lab-lkp/linux/commits/Libo-Chen/lib-Kconfig-remove-DEBUG_PER_CPU_MAPS-dependency-for-CPUMASK_OFFSTACK/20220413-073657 base: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git ce522ba9ef7e2d9fb22a39eb3371c0c64e2a433e config: nios2-randconfig-r023-20220413 (https://download.01.org/0day-ci/archive/20220413/[email protected]/config) compiler: nios2-linux-gcc (GCC) 11.2.0
reproduce (this is a W=1 build):
wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross
chmod +x ~/bin/make.cross
# https://github.com/intel-lab-lkp/linux/commit/6636f7cf28d2a79cde937c0f212e8a87080da06d
git remote add linux-review https://github.com/intel-lab-lkp/linux
git fetch --no-tags linux-review Libo-Chen/lib-Kconfig-remove-DEBUG_PER_CPU_MAPS-dependency-for-CPUMASK_OFFSTACK/20220413-073657 git checkout 6636f7cf28d2a79cde937c0f212e8a87080da06d
# save the config file to linux build tree
mkdir build_dir
COMPILER_INSTALL_PATH=$HOME/0day COMPILER=gcc-11.2.0 make.cross O=build_dir ARCH=nios2 SHELL=/bin/bash
If you fix the issue, kindly add following tag as appropriate
All errors (new ones prefixed by >>):
nios2-linux-ld: nios2-linux-ld: DWARF error: could not find abbrev number 71 kernel/workqueue.o: in function `free_workqueue_attrs':
workqueue.c:(.text+0x3fb8): undefined reference to free_cpumask_var' workqueue.c:(.text+0x3fb8): relocation truncated to fit: R_NIOS2_CALL26 against free_cpumask_var'
nios2-linux-ld: kernel/workqueue.o: in function `alloc_workqueue_attrs':
>> workqueue.c:(.text+0x40d4): undefined reference to `alloc_cpumask_var' workqueue.c:(.text+0x40d4): relocation truncated to fit: R_NIOS2_CALL26 against `alloc_cpumask_var' nios2-linux-ld: kernel/workqueue.o: in function `workqueue_set_unbound_cpumask': workqueue.c:(.text+0x5584): undefined reference to `zalloc_cpumask_var' workqueue.c:(.text+0x5584): relocation truncated to fit: R_NIOS2_CALL26 against `zalloc_cpumask_var' nios2-linux-ld: workqueue.c:(.text+0x5680): undefined reference to `free_cpumask_var' workqueue.c:(.text+0x5680): relocation truncated to fit: R_NIOS2_CALL26 against `free_cpumask_var' nios2-linux-ld: kernel/workqueue.o: in function `wq_unbound_cpumask_store': workqueue.c:(.text+0x56e0): undefined reference to `zalloc_cpumask_var' workqueue.c:(.text+0x56e0): relocation truncated to fit: R_NIOS2_CALL26 against `zalloc_cpumask_var' nios2-linux-ld: workqueue.c:(.text+0x5718): undefined reference to `free_cpumask_var' workqueue.c:(.text+0x5718): relocation truncated to fit: R_NIOS2_CALL26 against `free_cpumask_var' nios2-linux-ld: kernel/workqueue.o: in function `workqueue_init_early': >> workqueue.c:(.init.text+0x1e8): undefined reference to `alloc_cpumask_var' workqueue.c:(.init.text+0x1e8): relocation truncated to fit: R_NIOS2_CALL26 against `alloc_cpumask_var' nios2-linux-ld: nios2-linux-ld: DWARF error: could not find abbrev number 15310 kernel/sched/core.o: in function `sched_setaffinity': >> core.c:(.text+0x3b40): undefined reference to `alloc_cpumask_var' core.c:(.text+0x3b40): relocation truncated to fit: R_NIOS2_CALL26 against `alloc_cpumask_var' >> nios2-linux-ld: core.c:(.text+0x3b54): undefined reference to `alloc_cpumask_var' core.c:(.text+0x3b54): relocation truncated to fit: R_NIOS2_CALL26 against `alloc_cpumask_var' >> nios2-linux-ld: core.c:(.text+0x3be0): undefined reference to `free_cpumask_var' core.c:(.text+0x3be0): relocation truncated to fit: R_NIOS2_CALL26 against `free_cpumask_var' nios2-linux-ld: core.c:(.text+0x3bf0): undefined reference to `free_cpumask_var' core.c:(.text+0x3bf0): additional relocation overflows omitted from the output nios2-linux-ld: kernel/sched/core.o: in function `__se_sys_sched_setaffinity': core.c:(.text+0x3c64): undefined reference to `alloc_cpumask_var' nios2-linux-ld: core.c:(.text+0x3cc4): undefined reference to `free_cpumask_var' nios2-linux-ld: kernel/sched/core.o: in function `__se_sys_sched_getaffinity': core.c:(.text+0x3dd4): undefined reference to `alloc_cpumask_var' nios2-linux-ld: core.c:(.text+0x3e20): undefined reference to `free_cpumask_var' nios2-linux-ld: kernel/sched/core.o: in function `sched_init': >> core.c:(.init.text+0x27c): undefined reference to `load_balance_mask' >> nios2-linux-ld: core.c:(.init.text+0x284): undefined reference to `load_balance_mask' >> nios2-linux-ld: core.c:(.init.text+0x28c): undefined reference to `select_idle_mask' nios2-linux-ld: core.c:(.init.text+0x290): undefined reference to `select_idle_mask' nios2-linux-ld: nios2-linux-ld: DWARF error: could not find abbrev number 18 kernel/sched/build_utility.o: in function `housekeeping_setup_type': build_utility.c:(.init.text+0x130): undefined reference to `alloc_bootmem_cpumask_var' nios2-linux-ld: kernel/sched/build_utility.o: in function `housekeeping_setup': build_utility.c:(.init.text+0x198): undefined reference to `alloc_bootmem_cpumask_var' nios2-linux-ld: build_utility.c:(.init.text+0x1b4): undefined reference to `alloc_bootmem_cpumask_var' nios2-linux-ld: build_utility.c:(.init.text+0x2f4): undefined reference to `free_bootmem_cpumask_var' nios2-linux-ld: build_utility.c:(.init.text+0x304): undefined reference to `free_bootmem_cpumask_var' nios2-linux-ld: nios2-linux-ld: DWARF error: could not find abbrev number 91 kernel/profile.o: in function `prof_cpu_mask_proc_write': profile.c:(.text+0x40c): undefined reference to `zalloc_cpumask_var' nios2-linux-ld: profile.c:(.text+0x450): undefined reference to `free_cpumask_var' nios2-linux-ld: kernel/profile.o: in function `profile_init': >> profile.c:(.ref.text+0xc0): undefined reference to `alloc_cpumask_var' >> nios2-linux-ld: profile.c:(.ref.text+0x134): undefined reference to `free_cpumask_var' nios2-linux-ld: nios2-linux-ld: DWARF error: could not find abbrev number 700424681 kernel/time/hrtimer.o: in function `clock_was_set': >> hrtimer.c:(.text+0xc48): undefined reference to `zalloc_cpumask_var' >> nios2-linux-ld: hrtimer.c:(.text+0xd50): undefined reference to `free_cpumask_var' nios2-linux-ld: nios2-linux-ld: DWARF error: could not find abbrev number 361628 kernel/torture.o: in function `torture_cleanup_begin': >> torture.c:(.text+0x758): undefined reference to `free_cpumask_var' nios2-linux-ld: kernel/torture.o: in function `torture_shuffle_init': >> torture.c:(.text+0xbe4): undefined reference to `alloc_cpumask_var'
nios2-linux-ld: nios2-linux-ld: DWARF error: could not find abbrev number 13757 block/blk-mq.o: in function `blk_mq_alloc_hctx':
blk-mq.c:(.text+0x1550): undefined reference to `zalloc_cpumask_var_node'
nios2-linux-ld: blk-mq.c:(.text+0x16d4): undefined reference to `free_cpumask_var'
nios2-linux-ld: nios2-linux-ld: DWARF error: could not find abbrev number 264202560 block/blk-mq-sysfs.o: in function `blk_mq_hw_sysfs_release':
blk-mq-sysfs.c:(.text+0x2c0): undefined reference to `free_cpumask_var'
nios2-linux-ld: nios2-linux-ld: DWARF error: could not find abbrev number 3666 drivers/base/cpu.o: in function `print_cpus_offline':
cpu.c:(.text+0x94): undefined reference to `alloc_cpumask_var'
nios2-linux-ld: cpu.c:(.text+0xec): undefined reference to `free_cpumask_var'
nios2-linux-ld: drivers/base/cpu.o: in function `print_cpus_isolated':
cpu.c:(.text+0x1b8): undefined reference to `alloc_cpumask_var'
nios2-linux-ld: cpu.c:(.text+0x20c): undefined reference to `free_cpumask_var'
Signature · 2 lines
0-DAY CI Kernel Test Service https://01.org/lkp
kernel test robot
build botThank you for the patch! Yet something to improve:
Hi Libo,
Thank you for the patch! Yet something to improve:
[auto build test ERROR on linus/master]
[also build test ERROR on v5.18-rc2 next-20220413]
[If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch]
url: https://github.com/intel-lab-lkp/linux/commits/Libo-Chen/lib-Kconfig-remove-DEBUG_PER_CPU_MAPS-dependency-for-CPUMASK_OFFSTACK/20220413-073657 base: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git ce522ba9ef7e2d9fb22a39eb3371c0c64e2a433e config: parisc-randconfig-r014-20220413 (https://download.01.org/0day-ci/archive/20220413/[email protected]/config) compiler: hppa-linux-gcc (GCC) 11.2.0
reproduce (this is a W=1 build):
wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross
chmod +x ~/bin/make.cross
# https://github.com/intel-lab-lkp/linux/commit/6636f7cf28d2a79cde937c0f212e8a87080da06d
git remote add linux-review https://github.com/intel-lab-lkp/linux
git fetch --no-tags linux-review Libo-Chen/lib-Kconfig-remove-DEBUG_PER_CPU_MAPS-dependency-for-CPUMASK_OFFSTACK/20220413-073657 git checkout 6636f7cf28d2a79cde937c0f212e8a87080da06d
# save the config file to linux build tree
mkdir build_dir
COMPILER_INSTALL_PATH=$HOME/0day COMPILER=gcc-11.2.0 make.cross O=build_dir ARCH=parisc SHELL=/bin/bash
If you fix the issue, kindly add following tag as appropriate
All errors (new ones prefixed by >>):
hppa-linux-ld: kernel/workqueue.o: in function `free_workqueue_attrs':
>> kernel/workqueue.c:3370: undefined reference to `free_cpumask_var' hppa-linux-ld: kernel/workqueue.o: in function `alloc_workqueue_attrs': >> kernel/workqueue.c:3390: undefined reference to `alloc_cpumask_var' hppa-linux-ld: kernel/workqueue.o: in function `workqueue_set_unbound_cpumask': >> kernel/workqueue.c:5390: undefined reference to `zalloc_cpumask_var' >> hppa-linux-ld: kernel/workqueue.c:5406: undefined reference to `free_cpumask_var' hppa-linux-ld: kernel/workqueue.o: in function `wq_unbound_cpumask_store': kernel/workqueue.c:5664: undefined reference to `zalloc_cpumask_var' hppa-linux-ld: kernel/workqueue.c:5671: undefined reference to `free_cpumask_var' hppa-linux-ld: kernel/workqueue.o: in function `workqueue_init_early': kernel/workqueue.c:5995: undefined reference to `alloc_cpumask_var' hppa-linux-ld: kernel/sched/core.o: in function `sched_setaffinity': >> kernel/sched/core.c:7948: undefined reference to `alloc_cpumask_var' >> hppa-linux-ld: kernel/sched/core.c:7951: undefined reference to `alloc_cpumask_var' >> hppa-linux-ld: kernel/sched/core.c:7978: undefined reference to `free_cpumask_var' hppa-linux-ld: kernel/sched/core.c:7980: undefined reference to `free_cpumask_var' hppa-linux-ld: kernel/sched/core.o: in function `__se_sys_sched_setaffinity': kernel/sched/core.c:8051: undefined reference to `alloc_cpumask_var' hppa-linux-ld: kernel/sched/core.c:8057: undefined reference to `free_cpumask_var' hppa-linux-ld: kernel/sched/core.o: in function `__se_sys_sched_getaffinity': kernel/sched/core.c:8108: undefined reference to `alloc_cpumask_var' hppa-linux-ld: kernel/sched/core.c:8120: undefined reference to `free_cpumask_var' hppa-linux-ld: kernel/sched/core.o: in function `sched_init': >> kernel/sched/core.c:9499: undefined reference to `load_balance_mask' >> hppa-linux-ld: kernel/sched/core.c:9499: undefined reference to `load_balance_mask' >> hppa-linux-ld: kernel/sched/core.c:9501: undefined reference to `select_idle_mask' >> hppa-linux-ld: kernel/sched/core.c:9501: undefined reference to `select_idle_mask' hppa-linux-ld: kernel/sched/build_utility.o: in function `housekeeping_setup_type': >> kernel/sched/isolation.c:104: undefined reference to `alloc_bootmem_cpumask_var' hppa-linux-ld: kernel/sched/build_utility.o: in function `housekeeping_setup': kernel/sched/isolation.c:122: undefined reference to `alloc_bootmem_cpumask_var' >> hppa-linux-ld: kernel/sched/isolation.c:128: undefined reference to `alloc_bootmem_cpumask_var' >> hppa-linux-ld: kernel/sched/isolation.c:173: undefined reference to `free_bootmem_cpumask_var' hppa-linux-ld: kernel/sched/isolation.c:175: undefined reference to `free_bootmem_cpumask_var' hppa-linux-ld: kernel/taskstats.o: in function `taskstats_user_cmd': >> kernel/taskstats.c:441: undefined reference to `alloc_cpumask_var' >> hppa-linux-ld: kernel/taskstats.c:457: undefined reference to `alloc_cpumask_var' >> hppa-linux-ld: kernel/taskstats.c:464: undefined reference to `free_cpumask_var' hppa-linux-ld: kernel/events/core.o: in function `perf_event_init': >> kernel/events/core.c:13237: undefined reference to `zalloc_cpumask_var' hppa-linux-ld: fs/io_uring.o: in function `__io_uring_register': >> fs/io_uring.c:11472: undefined reference to `alloc_cpumask_var' >> hppa-linux-ld: fs/io_uring.c:11488: undefined reference to `free_cpumask_var'
hppa-linux-ld: fs/io_uring.c:11493: undefined reference to `free_cpumask_var'
hppa-linux-ld: fs/io-wq.o: in function `io_wq_create':
fs/io-wq.c:1180: undefined reference to `alloc_cpumask_var'
hppa-linux-ld: fs/io-wq.c:1214: undefined reference to `free_cpumask_var'
hppa-linux-ld: fs/io-wq.o: in function `io_wq_put_and_exit':
fs/io-wq.c:1290: undefined reference to `free_cpumask_var'
hppa-linux-ld: block/blk-mq.o: in function `blk_mq_alloc_hctx':
block/blk-mq.c:3528: undefined reference to `zalloc_cpumask_var_node'
hppa-linux-ld: block/blk-mq.c:3575: undefined reference to `free_cpumask_var'
hppa-linux-ld: drivers/base/cpu.o: in function `print_cpus_offline':
drivers/base/cpu.c:245: undefined reference to `alloc_cpumask_var'
hppa-linux-ld: drivers/base/cpu.c:249: undefined reference to `free_cpumask_var'
hppa-linux-ld: drivers/base/cpu.o: in function `print_cpus_isolated':
drivers/base/cpu.c:274: undefined reference to `alloc_cpumask_var'
hppa-linux-ld: drivers/base/cpu.c:281: undefined reference to `free_cpumask_var'
hppa-linux-ld: drivers/net/ethernet/emulex/benet/be_main.o: in function `be_clear_queues':
drivers/net/ethernet/emulex/benet/be_main.c:2943: undefined reference to `free_cpumask_var'
hppa-linux-ld: drivers/net/ethernet/emulex/benet/be_main.o: in function `be_setup_queues':
drivers/net/ethernet/emulex/benet/be_main.c:2981: undefined reference to `zalloc_cpumask_var'
hppa-linux-ld: drivers/net/ethernet/sfc/falcon/efx.o: in function `ef4_probe_nic':
drivers/net/ethernet/sfc/falcon/efx.c:1329: undefined reference to `zalloc_cpumask_var'
hppa-linux-ld: drivers/net/ethernet/sfc/falcon/efx.c:1344: undefined reference to `free_cpumask_var'
hppa-linux-ld: net/core/dev.o: in function `netif_get_num_default_rss_queues':
net/core/dev.c:3001: undefined reference to `zalloc_cpumask_var'
hppa-linux-ld: net/core/dev.c:3009: undefined reference to `free_cpumask_var'
hppa-linux-ld: kernel/profile.o: in function `prof_cpu_mask_proc_write':
kernel/profile.c:361: undefined reference to `zalloc_cpumask_var'
hppa-linux-ld: kernel/profile.c:369: undefined reference to `free_cpumask_var'
hppa-linux-ld: kernel/profile.o: in function `profile_init':
kernel/profile.c:114: undefined reference to `alloc_cpumask_var'
hppa-linux-ld: kernel/profile.c:132: undefined reference to `free_cpumask_var'
hppa-linux-ld: kernel/torture.o: in function `torture_cleanup_begin':
kernel/torture.c:591: undefined reference to `free_cpumask_var'
hppa-linux-ld: kernel/torture.o: in function `torture_shuffle_init':
kernel/torture.c:572: undefined reference to `alloc_cpumask_var'
hppa-linux-ld: block/blk-mq-sysfs.o: in function `blk_mq_hw_sysfs_release':
block/blk-mq-sysfs.c:41: undefined reference to `free_cpumask_var'
vim +3370 kernel/workqueue.c
1fa44ecad2b864 James Bottomley 2006-02-23 3360
6ba94429c8e7b8 Frederic Weisbecker 2015-04-02 3361 /**
6ba94429c8e7b8 Frederic Weisbecker 2015-04-02 3362 * free_workqueue_attrs - free a workqueue_attrs
6ba94429c8e7b8 Frederic Weisbecker 2015-04-02 3363 * @attrs: workqueue_attrs to free
226223ab3c4118 Tejun Heo 2013-03-12 3364 *
6ba94429c8e7b8 Frederic Weisbecker 2015-04-02 3365 * Undo alloc_workqueue_attrs().
226223ab3c4118 Tejun Heo 2013-03-12 3366 */
513c98d0868295 Daniel Jordan 2019-09-05 3367 void free_workqueue_attrs(struct workqueue_attrs *attrs)
226223ab3c4118 Tejun Heo 2013-03-12 3368 {
6ba94429c8e7b8 Frederic Weisbecker 2015-04-02 3369 if (attrs) {
6ba94429c8e7b8 Frederic Weisbecker 2015-04-02 @3370 free_cpumask_var(attrs->cpumask);
6ba94429c8e7b8 Frederic Weisbecker 2015-04-02 3371 kfree(attrs);
226223ab3c4118 Tejun Heo 2013-03-12 3372 }
226223ab3c4118 Tejun Heo 2013-03-12 3373 }
226223ab3c4118 Tejun Heo 2013-03-12 3374
6ba94429c8e7b8 Frederic Weisbecker 2015-04-02 3375 /**
6ba94429c8e7b8 Frederic Weisbecker 2015-04-02 3376 * alloc_workqueue_attrs - allocate a workqueue_attrs
6ba94429c8e7b8 Frederic Weisbecker 2015-04-02 3377 *
6ba94429c8e7b8 Frederic Weisbecker 2015-04-02 3378 * Allocate a new workqueue_attrs, initialize with default settings and
6ba94429c8e7b8 Frederic Weisbecker 2015-04-02 3379 * return it.
6ba94429c8e7b8 Frederic Weisbecker 2015-04-02 3380 *
6ba94429c8e7b8 Frederic Weisbecker 2015-04-02 3381 * Return: The allocated new workqueue_attr on success. %NULL on failure.
6ba94429c8e7b8 Frederic Weisbecker 2015-04-02 3382 */
513c98d0868295 Daniel Jordan 2019-09-05 3383 struct workqueue_attrs *alloc_workqueue_attrs(void)
226223ab3c4118 Tejun Heo 2013-03-12 3384 {
6ba94429c8e7b8 Frederic Weisbecker 2015-04-02 3385 struct workqueue_attrs *attrs;
226223ab3c4118 Tejun Heo 2013-03-12 3386
be69d00d976957 Thomas Gleixner 2019-06-26 3387 attrs = kzalloc(sizeof(*attrs), GFP_KERNEL);
6ba94429c8e7b8 Frederic Weisbecker 2015-04-02 3388 if (!attrs)
6ba94429c8e7b8 Frederic Weisbecker 2015-04-02 3389 goto fail;
be69d00d976957 Thomas Gleixner 2019-06-26 @3390 if (!alloc_cpumask_var(&attrs->cpumask, GFP_KERNEL))
6ba94429c8e7b8 Frederic Weisbecker 2015-04-02 3391 goto fail;
6ba94429c8e7b8 Frederic Weisbecker 2015-04-02 3392
6ba94429c8e7b8 Frederic Weisbecker 2015-04-02 3393 cpumask_copy(attrs->cpumask, cpu_possible_mask);
6ba94429c8e7b8 Frederic Weisbecker 2015-04-02 3394 return attrs;
6ba94429c8e7b8 Frederic Weisbecker 2015-04-02 3395 fail:
6ba94429c8e7b8 Frederic Weisbecker 2015-04-02 3396 free_workqueue_attrs(attrs);
6ba94429c8e7b8 Frederic Weisbecker 2015-04-02 3397 return NULL;
226223ab3c4118 Tejun Heo 2013-03-12 3398 }
226223ab3c4118 Tejun Heo 2013-03-12 3399 Signature · 7 lines
0-DAY CI Kernel Test Service https://01.org/lkp _______________________________________________ linux-arm-kernel mailing list [email protected] http://lists.infradead.org/mailman/listinfo/linux-arm-kernel