Disable automatic load CCS load balancing
Subsystems (3)
v7 cover Disable automatic load CCS load balancing
Hi,
this series does basically two things:
1. Disables automatic load balancing as adviced by the hardware workaround.
2. Assigns all the CCS slices to one single user engine. The user will then be able to query only one CCS engine
From v5 I have created a new file, gt/intel_gt_ccs_mode.c where I added the intel_gt_apply_ccs_mode(). In the upcoming patches, this file will contain the implementation for dynamic CCS mode setting.
Thanks Tvrtko, Matt, John and Joonas for your reviews!
Andi
Changelog
=========
v6 -> v7
- find a more appropriate place where to remove the CCS engines:
remove them in init_engine_mask() instead of intel_engines_init_mmio(). (Thanks, Matt)
- Add Michal's ACK, thanks Michal!
v5 -> v6 (thanks Matt for the suggestions in v6)
- Remove the refactoring and the for_each_available_engine() macro and instead do not create the intel_engine_cs structure at all.
- In patch 1 just a trivial reordering of the bit definitions.
v4 -> v5
- Use the workaround framework to do all the CCS balancing settings in order to always apply the modes also when the engine resets. Put everything in its own specific function to be executed for the first CCS engine encountered. (Thanks Matt)
- Calculate the CCS ID for the CCS mode as the first available CCS among all the engines (Thanks Matt)
- create the intel_gt_ccs_mode.c function to host the CCS configuration. We will have it ready for the next series.
- Fix a selftest that was failing because could not set CCS2.
- Add the for_each_available_engine() macro to exclude CCS1+ and start using it in the hangcheck selftest.
v3 -> v4
- Reword correctly the comment in the workaround
- Fix a buffer overflow (Thanks Joonas)
- Handle properly the fused engines when setting the CCS mode.
v2 -> v3
- Simplified the algorithm for creating the list of the exported uabi engines. (Patch 1) (Thanks, Tvrtko)
- Consider the fused engines when creating the uabi engine list (Patch 2) (Thanks, Matt)
- Patch 4 now uses a the refactoring from patch 1, in a cleaner outcome.
v1 -> v2
- In Patch 1 use the correct workaround number (thanks Matt).
- In Patch 2 do not add the extra CCS engines to the exposed UABI engine list and adapt the engine counting accordingly (thanks Tvrtko).
- Reword the commit of Patch 2 (thanks John).
Andi Shyti (3):
drm/i915/gt: Disable HW load balancing for CCS drm/i915/gt: Do not generate the command streamer for all the CCS drm/i915/gt: Enable only one CCS for compute workload
drivers/gpu/drm/i915/Makefile | 1 + drivers/gpu/drm/i915/gt/intel_engine_cs.c | 15 ++++++++ drivers/gpu/drm/i915/gt/intel_gt_ccs_mode.c | 39 +++++++++++++++++++++ drivers/gpu/drm/i915/gt/intel_gt_ccs_mode.h | 13 +++++++ drivers/gpu/drm/i915/gt/intel_gt_regs.h | 6 ++++ drivers/gpu/drm/i915/gt/intel_workarounds.c | 30 ++++++++++++++-- 6 files changed, 102 insertions(+), 2 deletions(-) create mode 100644 drivers/gpu/drm/i915/gt/intel_gt_ccs_mode.c create mode 100644 drivers/gpu/drm/i915/gt/intel_gt_ccs_mode.h
Signature · 1 line
2.43.0
PATCH v7 1/3 drm/i915/gt: Disable HW load balancing for CCS
The hardware should not dynamically balance the load between CCS engines. Wa_14019159160 recommends disabling it across all platforms.
--- drivers/gpu/drm/i915/gt/intel_gt_regs.h | 1 + drivers/gpu/drm/i915/gt/intel_workarounds.c | 23 +++++++++++++++++++-- 2 files changed, 22 insertions(+), 2 deletions(-)
drivers/gpu/drm/i915/gt/intel_gt_regs.h +1 -0
@@ -1477,6 +1477,7 @@#define ECOBITS_PPGTT_CACHE4B (0 << 8)#define GEN12_RCU_MODE _MMIO(0x14800) +#define XEHP_RCU_MODE_FIXED_SLICE_CCS_MODE REG_BIT(1)#define GEN12_RCU_MODE_CCS_ENABLE REG_BIT(0)#define CHV_FUSE_GT _MMIO(VLV_GUNIT_BASE + 0x2168)
drivers/gpu/drm/i915/gt/intel_workarounds.c +21 -2
@@ -51,7 +51,8 @@* registers belonging to BCS, VCS or VECS should be implemented in * xcs_engine_wa_init(). Workarounds for registers not belonging to a specific * engine's MMIO range but that are part of of the common RCS/CCS reset domain -* should be implemented in general_render_compute_wa_init(). +* should be implemented in general_render_compute_wa_init(). The settings +* about the CCS load balancing should be added in ccs_engine_wa_mode(). * * - GT workarounds: the list of these WAs is applied whenever these registers * revert to their default values: on GPU reset, suspend/resume [1]_, etc.
@@ -2854,6 +2855,22 @@ add_render_compute_tuning_settings(struct intel_gt *gt,wa_write_clr (wal,GEN8_GARBCNTL ,GEN12_BUS_HASH_CTL_BIT_EXC ); } +static void ccs_engine_wa_mode (struct intel_engine_cs *engine,struct i915_wa_list *wal) +{ +struct intel_gt *gt = engine->gt; + +if (!IS_DG2 (gt->i915)) +return ; + +/* +* Wa_14019159160: This workaround, along with others, leads to +* significant challenges in utilizing load balancing among the +* CCS slices. Consequently, an architectural decision has been +* made to completely disable automatic CCS load balancing. +*/ +wa_masked_en (wal,GEN12_RCU_MODE ,XEHP_RCU_MODE_FIXED_SLICE_CCS_MODE ); +} +/* * The workarounds in this function apply to shared registers in * the general render reset domain that aren't tied to a
@@ -3000,8 +3017,10 @@ engine_init_workarounds(struct intel_engine_cs *engine, struct i915_wa_list *wal* to a single RCS/CCS engine's workaround list since * they're reset as part of the general render domain reset. */ -if (engine->flags &I915_ENGINE_FIRST_RENDER_COMPUTE ) +if (engine->flags &I915_ENGINE_FIRST_RENDER_COMPUTE ) {general_render_compute_wa_init (engine, wal); +ccs_engine_wa_mode (engine, wal); + }if (engine->class ==COMPUTE_CLASS )ccs_engine_wa_init (engine, wal);
Signature · 1 line
2.43.0
PATCH v7 2/3 drm/i915/gt: Do not generate the command streamer for all the CCS
In reply to Andi ShytiWe want a fixed load CCS balancing consisting in all slices sharing one single user engine. For this reason do not create the intel_engine_cs structure with its dedicated command streamer for CCS slices beyond the first.
--- drivers/gpu/drm/i915/gt/intel_engine_cs.c | 15 +++++++++++++++ 1 file changed, 15 insertions(+)
drivers/gpu/drm/i915/gt/intel_engine_cs.c +15 -0
@@ -908,6 +908,21 @@ static intel_engine_mask_t init_engine_mask(struct intel_gt *gt) info->engine_mask &= ~BIT (GSC0 ); } +/* +* Do not create the command streamer for CCS slices beyond the first. +* All the workload submitted to the first engine will be shared among +* all the slices. +* +* Once the user will be allowed to customize the CCS mode, then this +* check needs to be removed. +*/ +if (IS_DG2 (gt->i915)) { +intel_engine_mask_t first_ccs =BIT ((CCS0 +__ffs (CCS_MASK (gt)))); +intel_engine_mask_t all_ccs =CCS_MASK (gt) <<CCS0 ; + + info->engine_mask &= ~(all_ccs &= ~first_ccs); + } +return info->engine_mask; }
Signature · 1 line
2.43.0
Re: [PATCH v7 2/3] drm/i915/gt: Do not generate the command streamer for all the CCS
Quoted 27 lines · Andi Shyti
We want a fixed load CCS balancing consisting in all slices sharing one single user engine. For this reason do not create the
intel_engine_csstructure with its dedicated command streamer for CCS slices beyond the first.Fixes d2eae8e98d59 ("drm/i915/dg2: Drop force_probe requirement")Signed-off-by Andi Shyti <[email protected]>Cc Chris Wilson <[email protected]>Cc Joonas Lahtinen <[email protected]>Cc Matt Roper <[email protected]>Cc <[email protected]> # v6.2+Acked-by Michal Mrozek <[email protected]>--- drivers/gpu/drm/i915/gt/intel_engine_cs.c | 15 +++++++++++++++ 1 file changed, 15 insertions(+)drivers/gpu/drm/i915/gt/intel_engine_cs.c +3
-0@@ -908,6 +908,21 @@ static intel_engine_mask_t init_engine_mask(struct intel_gt *gt) info->engine_mask &= ~BIT (GSC0 ); } +/* +* Do not create the command streamer for CCS slices beyond the first. +* All the workload submitted to the first engine will be shared among
drivers/gpu/drm/i915/gt/intel_engine_cs.c static intel_engine_mask_t init_engine_mask(struct intel_gt *gt) · line 923
Quoted 10 lines
drivers/gpu/drm/i915/gt/intel_engine_cs.c +10
-0+* all the slices. +* +* Once the user will be allowed to customize the CCS mode, then this +* check needs to be removed. +*/ +if (IS_DG2 (gt->i915)) { +intel_engine_mask_t first_ccs =BIT ((CCS0 +__ffs (CCS_MASK (gt)))); +intel_engine_mask_t all_ccs =CCS_MASK (gt) <<CCS0 ; + + info->engine_mask &= ~(all_ccs &= ~first_ccs);
Shouldn't the second "&=" just be an "&" since there's no need to modify the all_ccs variable that never gets used again?
In fact since this is DG2-specific, it seems like it might be more intuitive to just write the whole thing more directly as
if (IS_DG2(gt->i915)) {
int first_ccs = __ffs(CCS_MASK(gt));
info->engine_mask &= ~GENMASK(CCS3, CCS0);
info->engine_mask |= BIT(_CCS(first_ccs));
}But up to you; if you just want to remove the unnecessary "=" that's fine too. Either way,
Matt
Quoted 8 lines
drivers/gpu/drm/i915/gt/intel_engine_cs.c +2
-0+ } +return info->engine_mask; }Signature · 1 line
2.43.0
Signature · 4 lines
Matt Roper Graphics Software Engineer Linux GPU Platform Enablement Intel Corporation
Re: [PATCH v7 2/3] drm/i915/gt: Do not generate the command streamer for all the CCS
Hi Matt,
Quoted 8 lines
Quoted 8 lines
drivers/gpu/drm/i915/gt/intel_engine_cs.c +8
-0+/* +* Do not create the command streamer for CCS slices beyond the first. +* All the workload submitted to the first engine will be shared among +* all the slices. +* +* Once the user will be allowed to customize the CCS mode, then this +* check needs to be removed. +*/
Quoted 8 lines
drivers/gpu/drm/i915/gt/intel_engine_cs.c static intel_engine_mask_t init_engine_mask(struct intel_gt *gt) · line 923
Quoted 5 lines
drivers/gpu/drm/i915/gt/intel_engine_cs.c +5
-0+if (IS_DG2 (gt->i915)) { +intel_engine_mask_t first_ccs =BIT ((CCS0 +__ffs (CCS_MASK (gt)))); +intel_engine_mask_t all_ccs =CCS_MASK (gt) <<CCS0 ; + + info->engine_mask &= ~(all_ccs &= ~first_ccs);Shouldn't the second "&=" just be an "&" since there's no need to modify the
all_ccsvariable that never gets used again?
yes, that's a leftover from me trying different ways of removing all the non first CCS engines.
Quoted 9 lines
In fact since this is DG2-specific, it seems like it might be more intuitive to just write the whole thing more directly as
if (IS_DG2(gt->i915)) { int first_ccs = __ffs(CCS_MASK(gt)); info->engine_mask &= ~GENMASK(CCS3, CCS0); info->engine_mask |= BIT(_CCS(first_ccs)); }
yes, looks a bit simpler. Will use this way.
Quoted 4 lines
But up to you; if you just want to remove the unnecessary "=" that's fine too. Either way,
Reviewed-by Matt Roper <[email protected]>
Thanks!
Andi
PATCH v7 3/3 drm/i915/gt: Enable only one CCS for compute workload
In reply to Andi ShytiEnable only one CCS engine by default with all the compute sices allocated to it.
While generating the list of UABI engines to be exposed to the user, exclude any additional CCS engines beyond the first instance.
This change can be tested with igt i915_query.
--- drivers/gpu/drm/i915/Makefile | 1 + drivers/gpu/drm/i915/gt/intel_gt_ccs_mode.c | 39 +++++++++++++++++++++ drivers/gpu/drm/i915/gt/intel_gt_ccs_mode.h | 13 +++++++ drivers/gpu/drm/i915/gt/intel_gt_regs.h | 5 +++ drivers/gpu/drm/i915/gt/intel_workarounds.c | 7 ++++ 5 files changed, 65 insertions(+) create mode 100644 drivers/gpu/drm/i915/gt/intel_gt_ccs_mode.c create mode 100644 drivers/gpu/drm/i915/gt/intel_gt_ccs_mode.h
drivers/gpu/drm/i915/Makefile +1 -0
@@ -118,6 +118,7 @@ gt-y += \ gt/intel_ggtt_fencing.o \ gt/intel_gt.o \ gt/intel_gt_buffer_pool.o \ + gt/intel_gt_ccs_mode.o \ gt/intel_gt_clock_utils.o \ gt/intel_gt_debugfs.o \ gt/intel_gt_engines_debugfs.o \
drivers/gpu/drm/i915/gt/intel_gt_ccs_mode.c new file +39 -0
@@ -0,0 +1,39 @@ +// SPDX-License-Identifier: MIT +/* +* Copyright © 2024 Intel Corporation +*/ + +#include "i915_drv.h" +#include "intel_gt.h" +#include "intel_gt_ccs_mode.h" +#include "intel_gt_regs.h" + +void intel_gt_apply_ccs_mode (struct intel_gt *gt) +{ +int cslice; +u32 mode =0 ; +int first_ccs =__ffs (CCS_MASK (gt)); + +if (!IS_DG2 (gt->i915)) +return ; + +/* Build the value for the fixed CCS load balancing */ +for (cslice =0 ; cslice <I915_MAX_CCS ; cslice++) { +if (CCS_MASK (gt) &BIT (cslice)) +/* +* If available, assign the cslice +* to the first available engine... +*/ + mode |=XEHP_CCS_MODE_CSLICE (cslice, first_ccs); + +else +/* +* ... otherwise, mark the cslice as +* unavailable if no CCS dispatches here +*/ + mode |=XEHP_CCS_MODE_CSLICE (cslice, +XEHP_CCS_MODE_CSLICE_MASK ); + } + +intel_uncore_write (gt->uncore,XEHP_CCS_MODE , mode); +}
drivers/gpu/drm/i915/gt/intel_gt_ccs_mode.h new file +13 -0
@@ -0,0 +1,13 @@ +/* SPDX-License-Identifier: MIT */ +/* +* Copyright © 2024 Intel Corporation +*/ + +#ifndef __INTEL_GT_CCS_MODE_H__ +#define __INTEL_GT_CCS_MODE_H__ + +struct intel_gt ; + +void intel_gt_apply_ccs_mode (struct intel_gt *gt); + +#endif /* __INTEL_GT_CCS_MODE_H__ */
drivers/gpu/drm/i915/gt/intel_gt_regs.h +5 -0
@@ -1480,6 +1480,11 @@#define XEHP_RCU_MODE_FIXED_SLICE_CCS_MODE REG_BIT(1)#define GEN12_RCU_MODE_CCS_ENABLE REG_BIT(0) +#define XEHP_CCS_MODE _MMIO(0x14804) +#define XEHP_CCS_MODE_CSLICE_MASK REG_GENMASK(2, 0)/* CCS0-3 + rsvd */ +#define XEHP_CCS_MODE_CSLICE_WIDTH ilog2(XEHP_CCS_MODE_CSLICE_MASK + 1) +#define XEHP_CCS_MODE_CSLICE (cslice, ccs) (ccs << (cslice * XEHP_CCS_MODE_CSLICE_WIDTH)) +#define CHV_FUSE_GT _MMIO(VLV_GUNIT_BASE + 0x2168)#define CHV_FGT_DISABLE_SS0 (1 << 10)#define CHV_FGT_DISABLE_SS1 (1 << 11)
drivers/gpu/drm/i915/gt/intel_workarounds.c +7 -0
@@ -10,6 +10,7 @@#include "intel_engine_regs.h" #include "intel_gpu_commands.h" #include "intel_gt.h" +#include "intel_gt_ccs_mode.h" #include "intel_gt_mcr.h" #include "intel_gt_print.h" #include "intel_gt_regs.h"
@@ -2869,6 +2870,12 @@ static void ccs_engine_wa_mode(struct intel_engine_cs *engine, struct i915_wa_li* made to completely disable automatic CCS load balancing. */ wa_masked_en (wal,GEN12_RCU_MODE ,XEHP_RCU_MODE_FIXED_SLICE_CCS_MODE ); + +/* +* After having disabled automatic load balancing we need to +* assign all slices to a single CCS. We will call it CCS mode 1 +*/ +intel_gt_apply_ccs_mode (gt); }/*
Signature · 1 line
2.43.0
Patchwork
CI✗ Fi.CI.CHECKPATCH: warning for Disable automatic load CCS load balancing (rev12)
In reply to Andi Shyti== Series Details ==
== Series Details ==
Series: Disable automatic load CCS load balancing (rev12)
URL : https://patchwork.freedesktop.org/series/129951/
State : warning
== Summary ==
Error: dim checkpatch failed6f6385b0a101 drm/i915/gt: Disable HW load balancing for CCS9e580948c216 drm/i915/gt: Do not generate the command streamer for all the CCS f630774aa552 drm/i915/gt: Enable only one CCS for compute workload
Traceback (most recent call last):
File "scripts/spdxcheck.py", line 6, in <module>
from ply import lex, yacc
ModuleNotFoundError: No module named 'ply'
Traceback (most recent call last):
File "scripts/spdxcheck.py", line 6, in <module>
from ply import lex, yacc
ModuleNotFoundError: No module named 'ply'
-:37: WARNING:FILE_PATH_CHANGES: added, moved or deleted file(s), does MAINTAINERS need updating? #37:
new file mode 100644
-:111: CHECK:MACRO_ARG_PRECEDENCE: Macro argument 'cslice' may be better as '(cslice)' to avoid precedence issues #111: FILE: drivers/gpu/drm/i915/gt/intel_gt_regs.h:1433:
+#define XEHP_CCS_MODE_CSLICE(cslice, ccs) (ccs << (cslice * XEHP_CCS_MODE_CSLICE_WIDTH))
-:111: CHECK:MACRO_ARG_PRECEDENCE: Macro argument 'ccs' may be better as '(ccs)' to avoid precedence issues #111: FILE: drivers/gpu/drm/i915/gt/intel_gt_regs.h:1433:
+#define XEHP_CCS_MODE_CSLICE(cslice, ccs) (ccs << (cslice * XEHP_CCS_MODE_CSLICE_WIDTH))
total: 0 errors, 1 warnings, 2 checks, 89 lines checked
Patchwork
CI✗ Fi.CI.SPARSE: warning for Disable automatic load CCS load balancing (rev12)
In reply to Andi Shyti== Series Details ==
== Series Details ==
Series: Disable automatic load CCS load balancing (rev12)
URL : https://patchwork.freedesktop.org/series/129951/
State : warning
== Summary ==
Error: dim sparse failed
Sparse version: v0.6.2
Fast mode used, each commit won't be checked separately. +./arch/x86/include/asm/bitops.h:116:1: warning: unreplaced symbol 'return' +./arch/x86/include/asm/bitops.h:147:1: warning: unreplaced symbol 'return' +./arch/x86/include/asm/bitops.h:149:9: warning: unreplaced symbol 'oldbit' +./arch/x86/include/asm/bitops.h:153:26: warning: unreplaced symbol 'oldbit' +./arch/x86/include/asm/bitops.h:155:16: warning: unreplaced symbol 'oldbit' +./arch/x86/include/asm/bitops.h:155:9: warning: unreplaced symbol 'return' +./arch/x86/include/asm/bitops.h:173:1: warning: unreplaced symbol 'return' +./arch/x86/include/asm/bitops.h:175:9: warning: unreplaced symbol 'oldbit' +./arch/x86/include/asm/bitops.h:179:35: warning: unreplaced symbol 'oldbit' +./arch/x86/include/asm/bitops.h:181:16: warning: unreplaced symbol 'oldbit' +./arch/x86/include/asm/bitops.h:181:9: warning: unreplaced symbol 'return' +./arch/x86/include/asm/bitops.h:185:1: warning: unreplaced symbol 'return' +./arch/x86/include/asm/bitops.h:187:9: warning: unreplaced symbol 'oldbit' +./arch/x86/include/asm/bitops.h:191:35: warning: unreplaced symbol 'oldbit' +./arch/x86/include/asm/bitops.h:194:16: warning: unreplaced symbol 'oldbit' +./arch/x86/include/asm/bitops.h:194:9: warning: unreplaced symbol 'return' +./arch/x86/include/asm/bitops.h:236:1: warning: unreplaced symbol 'return' +./arch/x86/include/asm/bitops.h:238:9: warning: unreplaced symbol 'return' +./arch/x86/include/asm/bitops.h:66:1: warning: unreplaced symbol 'return' +./arch/x86/include/asm/bitops.h:92:1: warning: unreplaced symbol 'return' +./include/asm-generic/bitops/generic-non-atomic.h:100:17: warning: unreplaced symbol 'old' +./include/asm-generic/bitops/generic-non-atomic.h:100:23: warning: unreplaced symbol 'mask' +./include/asm-generic/bitops/generic-non-atomic.h:100:9: warning: unreplaced symbol 'return' +./include/asm-generic/bitops/generic-non-atomic.h:105:1: warning: unreplaced symbol 'return' +./include/asm-generic/bitops/generic-non-atomic.h:107:9: warning: unreplaced symbol 'mask' +./include/asm-generic/bitops/generic-non-atomic.h:108:9: warning: unreplaced symbol 'p' +./include/asm-generic/bitops/generic-non-atomic.h:109:9: warning: unreplaced symbol 'old' +./include/asm-generic/bitops/generic-non-atomic.h:111:10: warning: unreplaced symbol 'p' +./include/asm-generic/bitops/generic-non-atomic.h:111:14: warning: unreplaced symbol 'old' +./include/asm-generic/bitops/generic-non-atomic.h:111:20: warning: unreplaced symbol 'mask' +./include/asm-generic/bitops/generic-non-atomic.h:112:17: warning: unreplaced symbol 'old' +./include/asm-generic/bitops/generic-non-atomic.h:112:23: warning: unreplaced symbol 'mask' +./include/asm-generic/bitops/generic-non-atomic.h:112:9: warning: unreplaced symbol 'return' +./include/asm-generic/bitops/generic-non-atomic.h:121:1: warning: unreplaced symbol 'return' +./include/asm-generic/bitops/generic-non-atomic.h:128:9: warning: unreplaced symbol 'return' +./include/asm-generic/bitops/generic-non-atomic.h:166:1: warning: unreplaced symbol 'return' +./include/asm-generic/bitops/generic-non-atomic.h:168:9: warning: unreplaced symbol 'p' +./include/asm-generic/bitops/generic-non-atomic.h:169:9: warning: unreplaced symbol 'mask' +./include/asm-generic/bitops/generic-non-atomic.h:170:9: warning: unreplaced symbol 'val' +./include/asm-generic/bitops/generic-non-atomic.h:172:19: warning: unreplaced symbol 'val' +./include/asm-generic/bitops/generic-non-atomic.h:172:25: warning: unreplaced symbol 'mask' +./include/asm-generic/bitops/generic-non-atomic.h:172:9: warning: unreplaced symbol 'return' +./include/asm-generic/bitops/generic-non-atomic.h:28:1: warning: unreplaced symbol 'return' +./include/asm-generic/bitops/generic-non-atomic.h:30:9: warning: unreplaced symbol 'mask' +./include/asm-generic/bitops/generic-non-atomic.h:31:9: warning: unreplaced symbol 'p' +./include/asm-generic/bitops/generic-non-atomic.h:33:10: warning: unreplaced symbol 'p' +./include/asm-generic/bitops/generic-non-atomic.h:33:16: warning: unreplaced symbol 'mask' +./include/asm-generic/bitops/generic-non-atomic.h:37:1: warning: unreplaced symbol 'return' +./include/asm-generic/bitops/generic-non-atomic.h:39:9: warning: unreplaced symbol 'mask' +./include/asm-generic/bitops/generic-non-atomic.h:40:9: warning: unreplaced symbol 'p' +./include/asm-generic/bitops/generic-non-atomic.h:42:10: warning: unreplaced symbol 'p' +./include/asm-generic/bitops/generic-non-atomic.h:42:16: warning: unreplaced symbol 'mask' +./include/asm-generic/bitops/generic-non-atomic.h:55:1: warning: unreplaced symbol 'return' +./include/asm-generic/bitops/generic-non-atomic.h:57:9: warning: unreplaced symbol 'mask' +./include/asm-generic/bitops/generic-non-atomic.h:58:9: warning: unreplaced symbol 'p' +./include/asm-generic/bitops/generic-non-atomic.h:60:10: warning: unreplaced symbol 'p' +./include/asm-generic/bitops/generic-non-atomic.h:60:15: warning: unreplaced symbol 'mask' +./include/asm-generic/bitops/generic-non-atomic.h:73:1: warning: unreplaced symbol 'return' +./include/asm-generic/bitops/generic-non-atomic.h:75:9: warning: unreplaced symbol 'mask' +./include/asm-generic/bitops/generic-non-atomic.h:76:9: warning: unreplaced symbol 'p' +./include/asm-generic/bitops/generic-non-atomic.h:77:9: warning: unreplaced symbol 'old' +./include/asm-generic/bitops/generic-non-atomic.h:79:10: warning: unreplaced symbol 'p' +./include/asm-generic/bitops/generic-non-atomic.h:79:14: warning: unreplaced symbol 'old' +./include/asm-generic/bitops/generic-non-atomic.h:79:20: warning: unreplaced symbol 'mask' +./include/asm-generic/bitops/generic-non-atomic.h:80:17: warning: unreplaced symbol 'old' +./include/asm-generic/bitops/generic-non-atomic.h:80:23: warning: unreplaced symbol 'mask' +./include/asm-generic/bitops/generic-non-atomic.h:80:9: warning: unreplaced symbol 'return' +./include/asm-generic/bitops/generic-non-atomic.h:93:1: warning: unreplaced symbol 'return' +./include/asm-generic/bitops/generic-non-atomic.h:95:9: warning: unreplaced symbol 'mask' +./include/asm-generic/bitops/generic-non-atomic.h:96:9: warning: unreplaced symbol 'p' +./include/asm-generic/bitops/generic-non-atomic.h:97:9: warning: unreplaced symbol 'old' +./include/asm-generic/bitops/generic-non-atomic.h:99:10: warning: unreplaced symbol 'p' +./include/asm-generic/bitops/generic-non-atomic.h:99:14: warning: unreplaced symbol 'old' +./include/asm-generic/bitops/generic-non-atomic.h:99:21: warning: unreplaced symbol 'mask' +./include/asm-generic/bitops/instrumented-non-atomic.h:100:9: warning: unreplaced symbol 'return' +./include/asm-generic/bitops/instrumented-non-atomic.h:112:1: warning: unreplaced symbol 'return' +./include/asm-generic/bitops/instrumented-non-atomic.h:115:9: warning: unreplaced symbol 'return' +./include/asm-generic/bitops/instrumented-non-atomic.h:127:1: warning: unreplaced symbol 'return' +./include/asm-generic/bitops/instrumented-non-atomic.h:130:9: warning: unreplaced symbol 'return' +./include/asm-generic/bitops/instrumented-non-atomic.h:139:1: warning: unreplaced symbol 'return' +./include/asm-generic/bitops/instrumented-non-atomic.h:142:9: warning: unreplaced symbol 'return' +./include/asm-generic/bitops/instrumented-non-atomic.h:26:1: warning: unreplaced symbol 'return' +./include/asm-generic/bitops/instrumented-non-atomic.h:42:1: warning: unreplaced symbol 'return' +./include/asm-generic/bitops/instrumented-non-atomic.h:58:1: warning: unreplaced symbol 'return' +./include/asm-generic/bitops/instrumented-non-atomic.h:97:1: warning: unreplaced symbol 'return'