iommu/arm-smmu-v3: Perform some simple optimizations for arm_smmu_cmdq_build_cmd()
Subsystems (2)
v2 cover iommu/arm-smmu-v3: Perform some simple optimizations for arm_smmu_cmdq_build_cmd()
v1 --> v2:
1. Add patch 1, Properly handle the return value of arm_smmu_cmdq_build_cmd()
2. Remove arm_smmu_cmdq_copy_cmd(). In addition, when build command fails, out_cmd is not filled.
Zhen Lei (2):
iommu/arm-smmu-v3: Properly handle the return value of
arm_smmu_cmdq_build_cmd()
iommu/arm-smmu-v3: Simplify useless instructions in
arm_smmu_cmdq_build_cmd()
drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 29 ++++++++++++++------- 1 file changed, 20 insertions(+), 9 deletions(-)
Signature · 1 line
2.26.0.106.g9fadedd
PATCH v2 1/2 iommu/arm-smmu-v3: Properly handle the return value of arm_smmu_cmdq_build_cmd()
1. Build command CMD_SYNC cannot fail. So the return value can be ignored.
2. The arm_smmu_cmdq_build_cmd() almost never fails, the addition of "unlikely()" can optimize the instruction pipeline.
3. Check the return value in arm_smmu_cmdq_batch_add().
--- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 18 ++++++++++++------ 1 file changed, 12 insertions(+), 6 deletions(-)
drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +12 -6
@@ -409,10 +409,7 @@ static void __arm_smmu_cmdq_skip_err(struct arm_smmu_device *smmu,dev_err (smmu->dev,"\t0x%016llx\n" , (unsigned long long )cmd[i]);/* Convert the erroneous command into a CMD_SYNC */ -if (arm_smmu_cmdq_build_cmd (cmd, &cmd_sync)) { -dev_err (smmu->dev,"failed to convert to CMD_SYNC\n" ); -return ; - } +arm_smmu_cmdq_build_cmd (cmd, &cmd_sync);queue_write (Q_ENT (q, cons), cmd, q->ent_dwords); }
@@ -860,7 +857,7 @@ static int __arm_smmu_cmdq_issue_cmd(struct arm_smmu_device *smmu, {u64 cmd[CMDQ_ENT_DWORDS ]; -if (arm_smmu_cmdq_build_cmd (cmd, ent)) { +if (unlikely (arm_smmu_cmdq_build_cmd (cmd, ent))) {dev_warn (smmu->dev,"ignoring unknown CMDQ opcode 0x%x\n" , ent->opcode);return -EINVAL ;
@@ -885,11 +882,20 @@ static void arm_smmu_cmdq_batch_add(struct arm_smmu_device *smmu,struct arm_smmu_cmdq_batch *cmds, struct arm_smmu_cmdq_ent *cmd) { + int index; +if (cmds->num ==CMDQ_BATCH_ENTRIES ) {arm_smmu_cmdq_issue_cmdlist (smmu, cmds->cmds, cmds->num, false); cmds->num =0 ; } -arm_smmu_cmdq_build_cmd (&cmds->cmds[cmds->num *CMDQ_ENT_DWORDS ], cmd); + + index = cmds->num *CMDQ_ENT_DWORDS ; +if (unlikely (arm_smmu_cmdq_build_cmd (&cmds->cmds[index], cmd))) { +dev_warn (smmu->dev,"ignoring unknown CMDQ opcode 0x%x\n" , + cmd->opcode); +return ; + } + cmds->num++; }
Signature · 1 line
2.26.0.106.g9fadedd
PATCH v2 2/2 iommu/arm-smmu-v3: Simplify useless instructions in arm_smmu_cmdq_build_cmd()
In reply to Zhen LeiAlthough the parameter 'cmd' is always passed by a local array variable, and only this function modifies it, the compiler does not know this. The compiler almost always reads the value of cmd[i] from memory rather than directly using the value cached in the register. This generates many useless instruction operations and affects the performance to some extent.
To guide the compiler for proper optimization, 'cmd' is defined as a local array variable, marked as register, and copied to the output parameter at a time when the function is returned.
The optimization effect can be viewed by running the "size arm-smmu-v3.o" command.
Before:
text data bss dec hex
26954 1348 56 28358 6ec6
After:
text data bss dec hex
26762 1348 56 28166 6e06
--- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 11 ++++++++--- 1 file changed, 8 insertions(+), 3 deletions(-)
drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +8 -3
@@ -234,10 +234,12 @@ static int queue_remove_raw(struct arm_smmu_queue *q, u64 *ent) }/* High-level queue accessors */ -static int arm_smmu_cmdq_build_cmd (u64 *cmd,struct arm_smmu_cmdq_ent *ent) +static int arm_smmu_cmdq_build_cmd (u64 *out_cmd,struct arm_smmu_cmdq_ent *ent) { -memset (cmd,0 ,1 <<CMDQ_ENT_SZ_SHIFT ); - cmd[0 ] |=FIELD_PREP (CMDQ_0_OP , ent->opcode); + registeru64 cmd[CMDQ_ENT_DWORDS ]; + + cmd[0 ] =FIELD_PREP (CMDQ_0_OP , ent->opcode); + cmd[1 ] =0 ;switch (ent->opcode) {case CMDQ_OP_TLBI_EL2_ALL :
@@ -332,6 +334,9 @@ static int arm_smmu_cmdq_build_cmd(u64 *cmd, struct arm_smmu_cmdq_ent *ent)return -ENOENT ; } + out_cmd[0 ] = cmd[0 ]; + out_cmd[1 ] = cmd[1 ]; +return 0 ; }
Signature · 1 line
2.26.0.106.g9fadedd
Will Deacon
maintainerRe: [PATCH v2 2/2] iommu/arm-smmu-v3: Simplify useless instructions in arm_smmu_cmdq_build_cmd()
Quoted 31 lines · Zhen Lei
Although the parameter 'cmd' is always passed by a local array variable, and only this function modifies it, the compiler does not know this. The compiler almost always reads the value of cmd[i] from memory rather than directly using the value cached in the register. This generates many useless instruction operations and affects the performance to some extent.
To guide the compiler for proper optimization, 'cmd' is defined as a local array variable, marked as register, and copied to the output parameter at a time when the function is returned.
The optimization effect can be viewed by running the "size arm-smmu-v3.o" command.
Before:
text data bss dec hex
26954 1348 56 28358 6ec6After:
text data bss dec hex
26762 1348 56 28166 6e06Signed-off-by Zhen Lei <[email protected]>--- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 11 ++++++++--- 1 file changed, 8 insertions(+), 3 deletions(-)drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +0
-0
drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c static int queue_remove_raw(struct arm_smmu_queue *q, u64 *ent) · line 239
Quoted 9 lines
drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +2
-3@@ -234,10 +234,12 @@ static int queue_remove_raw(struct arm_smmu_queue *q, u64 *ent) }/* High-level queue accessors */ -static int arm_smmu_cmdq_build_cmd (u64 *cmd,struct arm_smmu_cmdq_ent *ent) +static int arm_smmu_cmdq_build_cmd (u64 *out_cmd,struct arm_smmu_cmdq_ent *ent) { -memset (cmd,0 ,1 <<CMDQ_ENT_SZ_SHIFT ); - cmd[0 ] |=FIELD_PREP (CMDQ_0_OP , ent->opcode); + registeru64 cmd[CMDQ_ENT_DWORDS ];
'register' is pretty badly specified outside of a handful of limited uses in conjunction with inline asm, so I really don't think we should be using it here.
drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c static int queue_remove_raw(struct arm_smmu_queue *q, u64 *ent) · line 241
Quoted 1 line
drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +1
-0+ cmd[0 ] =FIELD_PREP (CMDQ_0_OP , ent->opcode);
This generates a compiler warning about taking the address of a 'register' variable.
Will
Re: [PATCH v2 2/2] iommu/arm-smmu-v3: Simplify useless instructions in arm_smmu_cmdq_build_cmd()
Quoted 34 lines · Will Deacon
Quoted 33 lines · Zhen Lei
Although the parameter 'cmd' is always passed by a local array variable, and only this function modifies it, the compiler does not know this. The compiler almost always reads the value of cmd[i] from memory rather than directly using the value cached in the register. This generates many useless instruction operations and affects the performance to some extent.
To guide the compiler for proper optimization, 'cmd' is defined as a local array variable, marked as register, and copied to the output parameter at a time when the function is returned.
The optimization effect can be viewed by running the "size arm-smmu-v3.o" command.
Before:
text data bss dec hex
26954 1348 56 28358 6ec6After:
text data bss dec hex
26762 1348 56 28166 6e06Signed-off-by Zhen Lei <[email protected]>--- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 11 ++++++++--- 1 file changed, 8 insertions(+), 3 deletions(-)drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +0
-0@@ -234,10 +234,12 @@ static int queue_remove_raw(struct arm_smmu_queue *q, u64 *ent) }
Quoted 11 lines
drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c static int queue_remove_raw(struct arm_smmu_queue *q, u64 *ent) · line 239
Quoted 7 lines
drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +2
-3/* High-level queue accessors */ -static int arm_smmu_cmdq_build_cmd (u64 *cmd,struct arm_smmu_cmdq_ent *ent) +static int arm_smmu_cmdq_build_cmd (u64 *out_cmd,struct arm_smmu_cmdq_ent *ent) { -memset (cmd,0 ,1 <<CMDQ_ENT_SZ_SHIFT ); - cmd[0 ] |=FIELD_PREP (CMDQ_0_OP , ent->opcode); + registeru64 cmd[CMDQ_ENT_DWORDS ];'register' is pretty badly specified outside of a handful of limited uses in conjunction with inline asm, so I really don't think we should be using it here.
OK, I think I was overly aggressive in the beginning, and I just tried to remove the register modifier and get the same optimization.
Quoted 9 lines
drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c static int queue_remove_raw(struct arm_smmu_queue *q, u64 *ent) · line 241
Quoted 2 lines
drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +1
-0+ cmd[0 ] =FIELD_PREP (CMDQ_0_OP , ent->opcode);This generates a compiler warning about taking the address of a 'register' variable.
Will
.
Will Deacon
maintainerQuoted 5 lines · Zhen Lei
v1 --> v2:
1. Add patch 1, Properly handle the return value ofarm_smmu_cmdq_build_cmd()
2. Removearm_smmu_cmdq_copy_cmd(). In addition, when build command fails,out_cmdis not filled.
Quoted 7 lines
Zhen Lei (2):
iommu/arm-smmu-v3: Properly handle the return value of
arm_smmu_cmdq_build_cmd()
iommu/arm-smmu-v3: Simplify useless instructions in
arm_smmu_cmdq_build_cmd()[...]
Applied first patch only to will (for-joerg/arm-smmu/updates), thanks!
[1/2] iommu/arm-smmu-v3: Properly handle the return value of arm_smmu_cmdq_build_cmd()
https://git.kernel.org/will/c/59d9bd727495
Cheers,
Signature · 5 lines
Will https://fixes.arm64.dev https://next.arm64.dev https://will.arm64.dev