bitmap and cpumask cleanup
Subsystems (12)
cover bitmap and cpumask cleanup
Paul Jackson [email protected] 6 May 2004 Bitmap and Cpumask Cleanup
Andrew,
Please consider the following patch set for your *-mm patches.
It removes some 27 kernel source files, simplifies cpumasks, and in the colorful language of Rusty, it gets rid of such headers as:
asm-generic/cpumask_optimized_for_large_smp_with_sparse_array_and_small_stack.h
It also forms the basis for a nodemask_t patch, that is awaiting in the wings, from Matthew Dobson.
Each patch is described below, in "Patch Details."
===
This is the fifth version of my cleanup of bitmaps and cpumasks.
This set of nine patches applies against 2.6.6-rc3-mm2 plus six optional fixup patches, also included.
The actual patches will be posted in another hour or two, as followups to this initial post.
Primay goals:
The primary goal of this patch set is to simplify the code for cpumask_t and (later) nodemask_t, make them easier to use, and reduce code duplication.
Several flavors of cpumask have been reduced to one, with some local special handling to optimize for that vast majority of systems which have less than 32 (or 64) CPUs.
The bitmap operations upon which cpumasks depend have been optimized a bit more, with a more careful mix of inline and outofline code.
By simplifying masks to a single file, it should also be easier to add other such mask types, such as the nodemask that Matthew Dobson has waiting in the wings, just by copying cpumask.h and making a few global edits.
There is no significant difference between this fifth version and the fourth version of April 21, 2004. The basis for the patch has been moved forward to 2.6.6-rc3-mm2. And a few of the more trivial patches in the set have been collapsed, reducing the patchset to 9 patches.
What's new (comparing 2.6.6-rc3-mm2 to this patch set):
1) Some 27 files matching the pattern include/*/*mask*.h are replaced with the single file include/linux/cpumask.h The variety of arch-specific redirect headers for various cpumask implementation flavors is gone.
2) The bitmap operations (bitmap.h, bitmap.c) are optimized for systems of less than 32 (or 64) CPUs.
3) The cpumask operations are now just a thin layer on top of the bitmask and a few other operations. A cpumask is a bitmask of exactly NR_CPUS bits, wrapped in a structure.
4) bitmap_complement and cpumask_complement now take two args, source and target, instead of working in place.
5) Some uses of these macros elsewhere in the kernel were fine tuned.
6) On ia64, find_first_bit and find_first_zero_bit are uninlined - saving quite a bit of kernel text space. The architectures: alpha, parisc, ppc, sh, sparc, sparc64 have this same bit find code, and might also want to uninline.
7) Comments in bitmap.h and cpumask.h list available ops for ease of browsing.
8) The MASK_ALL macro zeros out unused bits on multiword bitmaps.
9) This patch includes Bill Irwin's recent rewrite of the bitmap_shift operators to avoid assuming some fixed upper bound on bitmap sizes.
10) A few more mask macros have been added, to make it easier to code mask manipulations correctly and easily. They provide xor, andnot, intersects and subset operators.
Bug fixes:
1) *_complement macros don't leave unused high bits set
2) MASK_ALL for sizes > 1 word, but not exact word multiple, doesn't have unused high bits set
3) Explicit, documented semantics for handling these unused high bits.
4) A few missing const attributes in bitmap & bitops added.
5) The (Hamming) cpumask weight macros were using the bitops hweight*() macros, which don't mask high unused bits - fixed to use the bitmap weight macro which does this masking.
Do to the rather limited use so far of cpumask macros, I am not aware of anything that these bugs would actually break in current mm or linus kernels.
Testing so far:
Kernels have been built for i386 (SMP and not), sparc64 and ia64 (sn2_defconfig), and booted and minimally tested on SN2. Kernel text size has been compared and found to typically get a little smaller. Correct function of bit operations has been tested for a variety of NR_CPUS values in a user level framework, using gcc compiler versions 2.95.3, 3.2.3 and 3.3.2. Joe Korty has built and booted on Opteron.
Reviews and feedback:
This code has been sent to the arch maintainers, and has been reviewed in some form or other by several, including:
- William Lee Irwin III <[email protected]>
- Jesse Barnes <[email protected]>
- Joe Korty <[email protected]>
- Matthew Dobson <[email protected]>
- Rusty Russell <[email protected]>
So far as I know, no outstanding issues remain open.
================ Patch Details ================
Here's what each patch does. First are 5 fixup patches and one minor rework of the cpu_present_map fixup.
pj-fix-1-unifix pj-fix-2-ashoks-updated-cpuhotplug-6-7 pj-fix-3-ashoks-updated-cpuhotplug-7-7 pj-fix-4-include-mempolicy pj-fix-5-syscall-return-semicolon
Patches pj-fix-* should match fixes that you (Andrew) have already picked up. You should ignore these if what you already have is just as good. They are here so that I can be precise as to what source I am using for the basis of the following patches.
nonsmp-cpu-present-map
You (Andrew) worked around a non-SMP build problem in Ashok's cpuhotplog patches in init/main.c routine fixup_cpu_present_map().
==> Instead of whatever you did (an ifdef or whatever), ==> Andrew, please take this patch instead.
This patch makes cpu_present_map a real map for all configurations, instead of a constant for non-SMP. It also moves the definition of cpu_present_map out of kernel/cpu.c into kernel/sched.c, because cpu.c isn't compiled into non-SMP kernels.
==============================================================
The nine main bitmap and cpumask cleanup patches follow.
==============================================================
mask1-bitmap-cleanup-prep
Document the bitmap bit model and handling of unused bits.
Tighten up bitmap so it does not generate nonzero bits in the unused tail if it is not given any on input.
Add intersects, subset, xor and andnot operators. Change bitmap_complement to take two operands.
Add a couple of missing 'const' qualifiers on bitops test_bit and bitmap_equal args.
mask2-bitmap-extensions
This bitmap improvements make it a suitable basis for fully supporting cpumask_t and nodemask_t. Inline macros with compile-time checks enable generating tight code on both small and large systems (large meaning cpumask_t requires more than one unsigned long's worth of bits).
The existing bitmap_<op> macros in lib/bitmap.c are renamed to __bitmap_<op>, and wrappers for each bitmap_<op> are exposed in include/linux/bitmap.h
This patch _includes_ Bill Irwins recent rewrite of the bitmap_shift operators to not require a fixed length intermediate bitmap.
Improved comments list each available operator for easy browsing.
mask3-unline-find-next-bit-ia64
Move the page of code (~700 bytes of instructions) for find_next_bit and find_next_zero_bit from inline in include/asm-ia64/bitops.h to a real function in arch/ia64/lib/bitops.c, leaving a declaration and macro wrapper behind.
The other arch's with almost this same code might want to also uninline it: alpha, parisc, ppc, sh, sparc, sparc64.
These are too big to inline.
mask4-new-cpumask-h
Major rewrite of cpumask to use a single implementation, as a struct-wrapped bitmap.
This patch leaves some 26 include/asm-*/cpumask*.h header files orphaned - to be removed next patch.
Some nine cpumask macros for const variants and to coerce and promote between an unsigned long and a cpumask are obsolete. Simple emulation wrappers are provided in this patch, which can be removed once each of the 3 archs (i386, ppc64, x86_64) using them are recoded in follow-on patches to not need them.
The CPU_MASK_ALL macro now avoids leaving possible garbage one bits in any unused portion of the high word.
An inproved comment lists all available operators, for convenient browsing.
mask5-remove-old-cpumask-files
With the cpumask rewrite in the previous patch, these various include/asm-*/cpumask*.h headers are no longer used.
mask6-cpumask-i386-fixup
Remove by recoding i386 uses of the obsolete cpumask const, coerce and promote macros.
mask7-cpumask-etc-fixup
Remove by recoding other uses of the obsolete cpumask const, coerce and promote macros.
mask8-rm-old-cpumask-emul
Now that the emulation of the obsolete cpumask macros is no longer needed, remove it from cpumask.h
mask9-post-cleanup-tweaks
Make use of for_each_cpu_mask() macro to simplify and optimize a couple of sparc64 per-CPU loops.
Optimize a bit of cpumask code for asm-i386/mach-es7000
Convert physids_complement() to use both args
Remove cpumask hack from asm-x86_64/topology.h routine pcibus_to_cpumask().
Clarify and slightly optimize set_cpus_allowed() cpumask check in kernel/sched.c
Signature · 3 lines
I won't rest till it's the best ...
Programmer, Linux Scalability
Paul Jackson <[email protected]> 1.650.933.1373PATCH 1/15 pj-fix-1-unifix
From: "Randy.Dunlap" <[email protected]>
There is a general desire to reduce the quantity of noisy and/or outdated kernel boot-time messages...
Suggested by Andi Kleen.
Ulrich's (old) comments:
http://www.nsa.gov/selinux/list-archive/0107/0525.cfm
Certifying Linux (Linux Journal):
http://www.linuxjournal.com/article.php?sid=0131
init/main.c +0 -1
@@ -521,7 +521,6 @@proc_root_init ();#endif check_bugs (); -printk ("POSIX conformance testing by UNIFIX\n" );/* * We count on the initial thread going ok
Signature · 3 lines
I won't rest till it's the best ...
Programmer, Linux Scalability
Paul Jackson <[email protected]> 1.650.933.1373Re: [PATCH mask 1/15] pj-fix-1-unifix
init/main.c line 524
Quoted 1 line · Paul Jackson
init/main.c +0
-1-printk ("POSIX conformance testing by UNIFIX\n" );
While this may make sense, I really don't see how it has anything to do with the series.
Signature · 1 line
Matt Mackall : http://www.selenic.com : Linux development and consulting
Re: [PATCH mask 1/15] pj-fix-1-unifix
Quoted 1 line · Matt
I really don't see how it has anything to do with the series.
It doesn't have anything to do with the series.
The 2.6.6-rc3-mm2 patch set could not be built (and that unifix patch could not even be applied as a patch - due to being malformed). So I could not build and test the bitmap/cpumask patch against a vanilla 2.6.6-rc3-mm2. Therefore I included a set of pj-fix-* patches, to get 2.6.6-rc3-mm2 healthy, before including the real bitmap/cpumask patches.
Notice that some patches are named pj-fix-*, and others named mask*.
And notice in the opening "[PATCH mask 0/15] bitmap and cpumask cleanup" where I write:
pj-fix-1-unifix pj-fix-2-ashoks-updated-cpuhotplug-6-7 pj-fix-3-ashoks-updated-cpuhotplug-7-7 pj-fix-4-include-mempolicy pj-fix-5-syscall-return-semicolon
Patches pj-fix-* should match fixes that you (Andrew) have already picked up. You should ignore these if what you already have is just as good. They are here so that I can be precise as to what source I am using for the basis of the following patches.
Signature · 3 lines
I won't rest till it's the best ...
Programmer, Linux Scalability
Paul Jackson <[email protected]> 1.650.933.1373Re: [PATCH mask 1/15] pj-fix-1-unifix
In reply to Matt MackallOh - I was working from Andrew's broken-out patch set, not the monolithic patch. If I had been using the monolithic patch, then I probably wouldn't have noticed this broken unifix patch. Oh well.
Signature · 3 lines
I won't rest till it's the best ...
Programmer, Linux Scalability
Paul Jackson <[email protected]> 1.650.933.1373PATCH 2/15 pj-fix-2-ashoks-updated-cpuhotplug-6-7
In reply to Paul JacksonName: cpu_present_map.patch
Author: Ashok Raj (Intel Corporation)
D: This patch adds cpu_present_map, cpu_present() and for_each_cpu_present()
D: to distinguish between possible cpu's in a system and cpu's physically
D: present in a system. Before cpu hotplug was introduced cpu_possible()
D: represented cpu's physically present in the system. With hotplug capable
D: Kernel, there is a requirement to distinguish a cpu as possible verses a
D: CPU physically present in the system. This is required so thta when
D: smp_init() attempts to start all cpu's it should now only attempt
D: to start cpu's present in the system. When a hotplug cpu is
D: physically inserted cpu_present_map will have bits updated
D: dynamically.
arch/ia64/kernel/smpboot.c +13 -9
@@ -75,11 +75,11 @@task_t *task_for_booting_cpu; -/* Bitmask of currently online CPUs */ +/* Bitmasks of currently online, and possible CPUs */ cpumask_t cpu_online_map;EXPORT_SYMBOL (cpu_online_map); -cpumask_t phys_cpu_present_map; -EXPORT_SYMBOL (phys_cpu_present_map); +cpumask_t cpu_possible_map; +EXPORT_SYMBOL (cpu_possible_map);/* which logical CPU number maps to which CPU (physical APIC ID) */ volatile int ia64_cpu_to_sapicid[NR_CPUS ];
@@ -99,6 +99,7 @@ nointroute (char *str) { no_int_routing =1 ; +printk ("no_int_routing on\n" );return 1 ; }
@@ -441,14 +442,15 @@ ia64_cpu_to_sapicid[cpu] =-1 ; ia64_cpu_to_sapicid[0 ] = boot_cpu_id; -cpus_clear (phys_cpu_present_map); -cpu_set (0 , phys_cpu_present_map); - +cpus_clear (cpu_present_map); +cpu_set (0 , cpu_present_map); +cpu_set (0 , cpu_possible_map);for (cpu =1 , i =0 ; i < smp_boot_data.cpu_count; i++) { sapicid = smp_boot_data.cpu_phys_id[i];if (sapicid == boot_cpu_id)continue ; -cpu_set (cpu, phys_cpu_present_map); +cpu_set (cpu, cpu_present_map); +cpu_set (cpu, cpu_possible_map); ia64_cpu_to_sapicid[cpu] = sapicid; cpu++; }
@@ -529,9 +531,11 @@if (!max_cpus) {printk (KERN_INFO "SMP mode deactivated.\n" );cpus_clear (cpu_online_map); -cpus_clear (phys_cpu_present_map); +cpus_clear (cpu_present_map); +cpus_clear (cpu_possible_map);cpu_set (0 , cpu_online_map); -cpu_set (0 , phys_cpu_present_map); +cpu_set (0 , cpu_present_map); +cpu_set (0 , cpu_possible_map);return ; } }
include/linux/cpumask.h +11 -0
@@ -10,11 +10,15 @@extern cpumask_t cpu_online_map;extern cpumask_t cpu_possible_map; +extern cpumask_t cpu_present_map;#define num_online_cpus () cpus_weight(cpu_online_map)#define num_possible_cpus () cpus_weight(cpu_possible_map) +#define num_present_cpus () cpus_weight(cpu_present_map) +#define cpu_online (cpu) cpu_isset(cpu, cpu_online_map)#define cpu_possible (cpu) cpu_isset(cpu, cpu_possible_map) +#define cpu_present (cpu) cpu_isset(cpu, cpu_present_map)#define for_each_cpu_mask (cpu, mask) \ for (cpu = first_cpu_const(mk_cpumask_const(mask)); \
@@ -23,16 +27,23 @@#define for_each_cpu (cpu) for_each_cpu_mask(cpu, cpu_possible_map)#define for_each_online_cpu (cpu) for_each_cpu_mask(cpu, cpu_online_map) +#define for_each_present_cpu (cpu) for_each_cpu_mask(cpu, cpu_present_map)#else #define cpu_online_map cpumask_of_cpu(0)#define cpu_possible_map cpumask_of_cpu(0) +#define cpu_present_map cpumask_of_cpu(0) +#define num_online_cpus () 1#define num_possible_cpus () 1 +#define num_present_cpus () 1 +#define cpu_online (cpu) ({ BUG_ON((cpu) != 0); 1; })#define cpu_possible (cpu) ({ BUG_ON((cpu) != 0); 1; }) +#define cpu_present (cpu) ({ BUG_ON((cpu) != 0); 1; })#define for_each_cpu (cpu) for (cpu = 0; cpu < 1; cpu++)#define for_each_online_cpu (cpu) for (cpu = 0; cpu < 1; cpu++) +#define for_each_present_cpu (cpu) for (cpu = 0; cpu < 1; cpu++)#endif #define cpumask_scnprintf (buf, buflen, map) \
init/main.c +19 -2
@@ -360,10 +360,10 @@unsigned j =1 ;/* FIXME: This should be done in userspace --RR */ -for (i =0 ; i <NR_CPUS ; i++) { +for_each_present_cpu (i) {if (num_online_cpus () >= max_cpus)break ; -if (cpu_possible (i) && !cpu_online (i)) { +if (!cpu_online (i)) {cpu_up (i); j++; }
@@ -620,6 +620,22 @@execve (init_filename, argv_init, envp_init); } +static void fixup_cpu_present_map (void ) +{ +int i; + +/* +* If arch is not hotplug ready and did not populate +* cpu_present_map, just make cpu_present_map same as cpu_possible_map +* for other cpu bringup code to function as normal. e.g smp_init() etc. +*/ +if (cpus_empty (cpu_present_map)) { +for_each_cpu (i) { +cpu_set (i, cpu_present_map); + } + } +} +static int init (void * unused) {lock_kernel ();
@@ -638,6 +654,7 @@do_pre_smp_initcalls (); +fixup_cpu_present_map ();smp_init ();sched_init_smp ();do_basic_setup ();
kernel/cpu.c +9 -1
@@ -20,6 +20,14 @@DECLARE_MUTEX (cpucontrol);static struct notifier_block *cpu_chain; +/* +* Represents all cpu's present in the system +* In systems capable of hotplug, this map could dynamically grow +* as new cpu's are detected in the system via any platform specific +* method, such as ACPI for e.g. +*/ +cpumask_t cpu_present_map; +EXPORT_SYMBOL (cpu_present_map);/* Need to know about CPUs going up/down? */ int register_cpu_notifier (struct notifier_block *nb)
@@ -180,7 +188,7 @@if ((ret =down_interruptible (&cpucontrol)) !=0 )return ret; -if (cpu_online (cpu)) { +if (cpu_online (cpu) || !cpu_present (cpu)) { ret = -EINVAL ; goto out; }
fs/proc/proc_misc.c +2 -2
@@ -371,7 +371,7 @@if (wall_to_monotonic.tv_nsec) --jif; -for_each_cpu (i) { +for_each_online_cpu (i) {int j; user +=kstat_cpu (i).cpustat.user;
@@ -393,7 +393,7 @@ (unsigned long long )jiffies_64_to_clock_t (iowait), (unsigned long long )jiffies_64_to_clock_t (irq), (unsigned long long)jiffies_64_to_clock_t(softirq)); -for_each_cpu (i) { +for_each_online_cpu (i) {/* Copy values here to work around gcc-2.95.3, gcc-2.96 */ user =kstat_cpu (i).cpustat.user;
kernel/sched.c +3 -3
@@ -1219,7 +1219,7 @@ {unsigned long i, sum =0 ; -for_each_cpu (i) +for_each_online_cpu (i) sum +=cpu_rq (i)->nr_uninterruptible;return sum;
@@ -1229,7 +1229,7 @@ {unsigned long long i, sum =0 ; -for_each_cpu (i) +for_each_online_cpu (i) sum +=cpu_rq (i)->nr_switches;return sum;
@@ -1239,7 +1239,7 @@ {unsigned long i, sum =0 ; -for_each_cpu (i) +for_each_online_cpu (i) sum +=atomic_read (&cpu_rq (i)->nr_iowait);return sum;
fs/buffer.c +1 -1
@@ -2989,7 +2989,7 @@if (__get_cpu_var (bh_accounting).ratelimit++ <4096 )return ;__get_cpu_var (bh_accounting).ratelimit =0 ; -for_each_cpu (i) +for_each_online_cpu (i) tot +=per_cpu (bh_accounting, i).nr; buffer_heads_over_limit = (tot > max_buffer_heads); }
kernel/fork.c +1 -1
@@ -63,7 +63,7 @@int cpu;int total =0 ; -for_each_cpu (cpu) +for_each_online_cpu (cpu) total +=per_cpu (process_counts, cpu);return total;
kernel/timer.c +1 -1
@@ -332,7 +332,7 @@ del_again: ret +=del_timer (timer); -for_each_cpu (i) { +for_each_online_cpu (i) { base = &per_cpu (tvec_bases, i);if (base->running_timer == timer) {while (base->running_timer == timer) {
include/asm-ia64/smp.h +0 -3
@@ -38,7 +38,6 @@extern char no_int_routing__devinitdata ; -extern cpumask_t phys_cpu_present_map;extern cpumask_t cpu_online_map;extern unsigned long ipi_base_addr;extern unsigned char smp_int_redirect;
@@ -48,8 +47,6 @@extern unsigned long ap_wakeup_vector; -#define cpu_possible_map phys_cpu_present_map -/* * Function to map hard smp processor id to logical id. Slow, so don't use this in * performance-critical code.
Signature · 3 lines
I won't rest till it's the best ...
Programmer, Linux Scalability
Paul Jackson <[email protected]> 1.650.933.1373PATCH 3/15 pj-fix-3-ashoks-updated-cpuhotplug-7-7
In reply to Paul JacksonName: hotcpu_ia64.patch
Author: Ashok Raj (Intel Corporation)
D: Supports basic ability to enable hotplug functions for IA64.
D: Code is just evolving, and there are several loose ends to tie up. D:
D: What this code drop does
D: - Support logical online and offline
D: - Handles interrupt migration without loss of interrupts.
D: - Handles stress fine > 24+ hrs with make -j/ftp/rcp workloads
D: - Handles irq migration from a dying cpu without loss of interrupts.
D: What needs to be done
D: - Boot CPU removal support, with platform level authentication
D: - Putting cpu being removed in BOOT_RENDEZ mode.
---
linux-2.6.6-rc3-root/arch/ia64/Kconfig | 9 + linux-2.6.6-rc3-root/arch/ia64/kernel/iosapic.c | 7 - linux-2.6.6-rc3-root/arch/ia64/kernel/irq.c | 103 +++++++++++++++++ linux-2.6.6-rc3-root/arch/ia64/kernel/irq_ia64.c | 60 +++++++++- linux-2.6.6-rc3-root/arch/ia64/kernel/process.c | 43 +++++++ linux-2.6.6-rc3-root/arch/ia64/kernel/sal.c | 13 ++ linux-2.6.6-rc3-root/arch/ia64/kernel/smp.c | 26 ++++ linux-2.6.6-rc3-root/arch/ia64/kernel/smpboot.c | 136 +++++++++++++++++++++-- linux-2.6.6-rc3-root/arch/ia64/kernel/time.c | 5 linux-2.6.6-rc3-root/include/asm-ia64/smp.h | 2 10 files changed, 386 insertions(+), 18 deletions(-)
arch/ia64/Kconfig renamed from arch/ia64/Kconfig~hotcpu_ia64 +9 -0
@@ -78,6 +78,15 @@ config IA64_HP_SIM endchoice +config HOTPLUG_CPU + bool "Support for hot-pluggable CPUs (EXPERIMENTAL)" + depends on SMP && HOTPLUG && EXPERIMENTAL + default n + ---help--- + Say Y here to experiment with turning CPUs off and on. CPUs + can be controlled through /sys/devices/system/cpu/cpu#. + Say N if you want to disable cpu hotplug. + choice prompt "Processor type" default ITANIUM
arch/ia64/kernel/irq.c renamed from arch/ia64/kernel/irq.c~hotcpu_ia64 +103 -0
135 lines not on this page · raw message
arch/ia64/kernel/process.c renamed from arch/ia64/kernel/process.c~hotcpu_ia64 +42 -1
86 lines not on this page · raw message
arch/ia64/kernel/smpboot.c renamed from arch/ia64/kernel/smpboot.c~hotcpu_ia64 +127 -9
216 lines not on this page · raw message
arch/ia64/kernel/smp.c renamed from arch/ia64/kernel/smp.c~hotcpu_ia64 +25 -1
@@ -71,10 +71,23 @@ static volatile struct call_data_struct/* This needs to be cacheline aligned because it is written to by *other* CPUs. */ static DEFINE_PER_CPU(u64, ipi_operation) ____cacheline_aligned; +extern void cpu_halt (void ); + +void +lock_ipi_calllock (void ) +{ +spin_lock_irq (&call_lock); +} + +void +unlock_ipi_calllock (void ) +{ +spin_unlock_irq (&call_lock); +} +static void stop_this_cpu (void ) { -extern void cpu_halt (void );/* * Remove this CPU: */
@@ -84,6 +97,17 @@ stop_this_cpu (void)cpu_halt (); } +void +cpu_die (void ) +{ +max_xtp (); +local_irq_disable (); +cpu_halt (); +/* Should never be here */ +BUG (); +for (;;); +} +irqreturn_t handle_IPI (int irq,void *dev_id,struct pt_regs *regs) {
arch/ia64/kernel/time.c renamed from arch/ia64/kernel/time.c~hotcpu_ia64 +5 -0
@@ -10,6 +10,7 @@*/ #include <linux/config.h> +#include <linux/cpu.h> #include <linux/init.h> #include <linux/kernel.h> #include <linux/module.h>
@@ -244,6 +245,10 @@ timer_interrupt (int irq, void *dev_id, {unsigned long new_itm; +if (unlikely (cpu_is_offline (smp_processor_id ()))) { +return IRQ_HANDLED ; + } +platform_timer_interrupt (irq, dev_id, regs); new_itm = local_cpu_data->itm_next;
include/asm-ia64/smp.h renamed from include/asm-ia64/smp.h~hotcpu_ia64 +2 -0
@@ -120,6 +120,8 @@ extern void smp_do_timer (struct pt_regsextern int smp_call_function_single (int cpuid,void (*func) (void *info),void *info,int retry,int wait);extern void smp_send_reschedule (int cpu); +extern void lock_ipi_calllock (void ); +extern void unlock_ipi_calllock (void );#endif /* CONFIG_SMP */ #endif /* _ASM_IA64_SMP_H */
arch/ia64/kernel/iosapic.c renamed from arch/ia64/kernel/iosapic.c~hotcpu_ia64 +5 -2
@@ -32,6 +32,8 @@* 03/02/19 B. Helgaas Make pcat_compat system-wide, not per-IOSAPIC. * Remove iosapic_address & gsi_base from external interfaces. * Rationalize __init/__devinit attributes. +* 04/12/04 Ashok Raj <[email protected]> Intel Corporation 2004 +* Updated to work with irq migration necessary for CPU Hotplug */ /* * Here is what the interrupt logic between a PCI device and the kernel looks like:
@@ -189,8 +191,10 @@ set_rte (unsigned int vector, unsigned i pol = iosapic_intr_info[vector].polarity; trigger = iosapic_intr_info[vector].trigger; dmode = iosapic_intr_info[vector].dmode; + vector &= (~IA64_IRQ_REDIRECTED ); redir = (dmode ==IOSAPIC_LOWEST_PRIORITY ) ?1 :0 ; +#ifdef CONFIG_SMP {unsigned int irq;
@@ -312,9 +316,8 @@ iosapic_set_affinity (unsigned int irq,spin_lock_irqsave (&iosapic_lock, flags); { -/* get current delivery mode by reading the low32 */ -writel (IOSAPIC_RTE_LOW (rte_index), addr +IOSAPIC_REG_SELECT ); low32 = iosapic_intr_info[vec].low32 & ~(7 <<IOSAPIC_DELIVERY_SHIFT ); +if (redir)/* change delivery mode to lowest priority */ low32 |= (IOSAPIC_LOWEST_PRIORITY <<IOSAPIC_DELIVERY_SHIFT );
arch/ia64/kernel/irq_ia64.c renamed from arch/ia64/kernel/irq_ia64.c~hotcpu_ia64 +55 -5
84 lines not on this page · raw message
arch/ia64/kernel/sal.c renamed from arch/ia64/kernel/sal.c~hotcpu_ia64 +13 -0
@@ -122,10 +122,23 @@ sal_desc_entry_point (void *p)static void __init set_smp_redirect (int flag) { +#ifndef CONFIG_HOTPLUG_CPU if (no_int_routing) smp_int_redirect &= ~flag;else smp_int_redirect |= flag; +#else +/* +* For CPU Hotplug we dont want to do any chipset supported +* interrupt redirection. The reason is this would require that +* All interrupts be stopped and hard bind the irq to a cpu. +* Later when the interrupt is fired we need to set the redir hint +* on again in the vector. This is combersome for something that the +* user mode irq balancer will solve anyways. +*/ + no_int_routing=1 ; + smp_int_redirect &= ~flag; +#endif }#else #define set_smp_redirect (flag) do { } while (0)
_
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Signature · 3 lines
I won't rest till it's the best ...
Programmer, Linux Scalability
Paul Jackson <[email protected]> 1.650.933.1373PATCH 4/15 pj-fix-4-include-mempolicy
In reply to Paul JacksonFix numa build - small-numa-api-fixups.patch replaced an included with a struct forward declaration, so now some other files need to include mempolicy.h explicitly.
arch/ia64/ia32/binfmt_elf32.c +1 -0
@@ -13,6 +13,7 @@#include <linux/types.h> #include <linux/mm.h> +#include <linux/mempolicy.h> #include <linux/security.h> #include <asm/param.h>
arch/ia64/kernel/perfmon.c +1 -0
@@ -29,6 +29,7 @@#include <linux/init.h> #include <linux/vmalloc.h> #include <linux/mm.h> +#include <linux/mempolicy.h> #include <linux/sysctl.h> #include <linux/list.h> #include <linux/file.h>
arch/ia64/mm/init.c +1 -0
@@ -12,6 +12,7 @@#include <linux/efi.h> #include <linux/elf.h> #include <linux/mm.h> +#include <linux/mempolicy.h> #include <linux/mmzone.h> #include <linux/module.h> #include <linux/personality.h>
kernel/exit.c +1 -0
@@ -6,6 +6,7 @@#include <linux/config.h> #include <linux/mm.h> +#include <linux/mempolicy.h> #include <linux/slab.h> #include <linux/interrupt.h> #include <linux/smp_lock.h>
Signature · 3 lines
I won't rest till it's the best ...
Programmer, Linux Scalability
Paul Jackson <[email protected]> 1.650.933.1373PATCH 5/15 pj-fix-5-syscall-return-semicolon
In reply to Paul JacksonChanges to system call return value were missing a semi-colon.
include/asm-sparc/unistd.h +6 -6
@@ -325,7 +325,7 @@ : "=r" (__res)\ : "r" (__g1) \ : "o0", "cc"); \ - __syscall_return(type, __res) \ +__syscall_return (type, __res); \ }#define _syscall1 (type,name,type1,arg1) \
@@ -342,7 +342,7 @@ : "=r" (__res), "=&r" (__o0) \ : "1" (__o0), "r" (__g1) \ : "cc"); \ - __syscall_return(type, __res) \ +__syscall_return (type, __res); \ }#define _syscall2 (type,name,type1,arg1,type2,arg2) \
@@ -360,7 +360,7 @@ : "=r" (__res), "=&r" (__o0) \ : "1" (__o0), "r" (__o1), "r" (__g1) \ : "cc"); \ - __syscall_return(type, __res) \ +__syscall_return (type, __res); \ }#define _syscall3 (type,name,type1,arg1,type2,arg2,type3,arg3) \
@@ -379,7 +379,7 @@ : "=r" (__res), "=&r" (__o0) \ : "1" (__o0), "r" (__o1), "r" (__o2), "r" (__g1) \ : "cc"); \ - __syscall_return(type, __res) \ +__syscall_return (type, __res); \ }#define _syscall4 (type,name,type1,arg1,type2,arg2,type3,arg3,type4,arg4) \
@@ -399,7 +399,7 @@ : "=r" (__res), "=&r" (__o0) \ : "1" (__o0), "r" (__o1), "r" (__o2), "r" (__o3), "r" (__g1) \ : "cc"); \ - __syscall_return(type, __res) \ +__syscall_return (type, __res); \ } #define _syscall5(type,name,type1,arg1,type2,arg2,type3,arg3,type4,arg4, \
@@ -421,7 +421,7 @@ : "=r" (__res), "=&r" (__o0) \ : "1" (__o0), "r" (__o1), "r" (__o2), "r" (__o3), "r" (__o4), "r" (__g1) \ : "cc"); \ - __syscall_return(type, __res) \ +__syscall_return (type, __res); \ }#ifdef __KERNEL_SYSCALLS__
include/asm-sparc64/unistd.h +6 -6
@@ -324,7 +324,7 @@ : "=r" (__res)\ : "r" (__g1) \ : "o0", "cc"); \ - __syscall_return(type, __res) \ +__syscall_return (type, __res); \ }#define _syscall1 (type,name,type1,arg1) \
@@ -339,7 +339,7 @@ : "=r" (__res), "=&r" (__o0) \ : "1" (__o0), "r" (__g1) \ : "cc"); \ - __syscall_return(type, __res) \ +__syscall_return (type, __res); \ }#define _syscall2 (type,name,type1,arg1,type2,arg2) \
@@ -355,7 +355,7 @@ : "=r" (__res), "=&r" (__o0) \ : "1" (__o0), "r" (__o1), "r" (__g1) \ : "cc"); \ - __syscall_return(type, __res) \ +__syscall_return (type, __res); \ }#define _syscall3 (type,name,type1,arg1,type2,arg2,type3,arg3) \
@@ -372,7 +372,7 @@ : "=r" (__res), "=&r" (__o0) \ : "1" (__o0), "r" (__o1), "r" (__o2), "r" (__g1) \ : "cc"); \ - __syscall_return(type, __res) \ +__syscall_return (type, __res); \ }#define _syscall4 (type,name,type1,arg1,type2,arg2,type3,arg3,type4,arg4) \
@@ -390,7 +390,7 @@ : "=r" (__res), "=&r" (__o0) \ : "1" (__o0), "r" (__o1), "r" (__o2), "r" (__o3), "r" (__g1) \ : "cc"); \ - __syscall_return(type, __res) \ +__syscall_return (type, __res); \ } #define _syscall5(type,name,type1,arg1,type2,arg2,type3,arg3,type4,arg4, \
@@ -410,7 +410,7 @@ : "=r" (__res), "=&r" (__o0) \ : "1" (__o0), "r" (__o1), "r" (__o2), "r" (__o3), "r" (__o4), "r" (__g1) \ : "cc"); \ - __syscall_return(type, __res) \ +__syscall_return (type, __res); \ }#ifdef __KERNEL_SYSCALLS__
include/asm-mips/unistd.h +9 -9
@@ -847,7 +847,7 @@ : "i" (__NR_##name) \ : "$2", "$8", "$9", "$10", "$11", "$12", "$13", "$14", "$15", "$24"); \ \ - __syscall_return(type, __v0) \ + __syscall_return(type, __v0); \ }/*
@@ -871,7 +871,7 @@ : "r" (__a0), "i" (__NR_##name) \ : "$2", "$8", "$9", "$10", "$11", "$12", "$13", "$14", "$15", "$24"); \ \ - __syscall_return(type, __v0) \ + __syscall_return(type, __v0); \ }#define _syscall2 (type,name,atype,a,btype,b) \
@@ -892,7 +892,7 @@ : "r" (__a0), "r" (__a1), "i" (__NR_##name) \ : "$2", "$8", "$9", "$10", "$11", "$12", "$13", "$14", "$15", "$24"); \ \ - __syscall_return(type, __v0) \ + __syscall_return(type, __v0); \ }#define _syscall3 (type,name,atype,a,btype,b,ctype,c) \
@@ -914,7 +914,7 @@ : "r" (__a0), "r" (__a1), "r" (__a2), "i" (__NR_##name) \ : "$2", "$8", "$9", "$10", "$11", "$12", "$13", "$14", "$15", "$24"); \ \ - __syscall_return(type, __v0) \ + __syscall_return(type, __v0); \ }#define _syscall4 (type,name,atype,a,btype,b,ctype,c,dtype,d) \
@@ -936,7 +936,7 @@ : "r" (__a0), "r" (__a1), "r" (__a2), "i" (__NR_##name) \ : "$2", "$8", "$9", "$10", "$11", "$12", "$13", "$14", "$15", "$24"); \ \ - __syscall_return(type, __v0) \ + __syscall_return(type, __v0); \ }#if (_MIPS_SIM == _MIPS_SIM_ABI32)
@@ -969,7 +969,7 @@ "m" ((unsigned long)e) \ : "$2", "$8", "$9", "$10", "$11", "$12", "$13", "$14", "$15", "$24"); \ \ - __syscall_return(type, __v0) \ +__syscall_return (type, __v0); \ }#define _syscall6 (type,name,atype,a,btype,b,ctype,c,dtype,d,etype,e,ftype,f) \
@@ -998,7 +998,7 @@ "m" ((unsigned long)e), "m" ((unsigned long)f) \ : "$2", "$8", "$9", "$10", "$11", "$12", "$13", "$14", "$15", "$24"); \ \ - __syscall_return(type, __v0) \ + __syscall_return(type, __v0); \ }#endif /* (_MIPS_SIM == _MIPS_SIM_ABI32) */
@@ -1025,7 +1025,7 @@ : "r" (__a0), "r" (__a1), "r" (__a2), "i" (__NR_##name) \ : "$2","$9", "$10", "$11", "$12", "$13", "$14", "$15", "$24"); \ \ - __syscall_return(type, __v0) \ + __syscall_return(type, __v0); \ }#define _syscall6 (type,name,atype,a,btype,b,ctype,c,dtype,d,etype,e,ftype,f) \
@@ -1050,7 +1050,7 @@ "i" (__NR_##name) \ : "$2","$9", "$10", "$11", "$12", "$13", "$14", "$15", "$24"); \ \ - __syscall_return(type, __v0) \ + __syscall_return(type, __v0); \ }#endif /* (_MIPS_SIM == _MIPS_SIM_NABI32) || (_MIPS_SIM == _MIPS_SIM_ABI64) */
Signature · 3 lines
I won't rest till it's the best ...
Programmer, Linux Scalability
Paul Jackson <[email protected]> 1.650.933.1373PATCH 6/15 nonsmp-cpu-present-map
In reply to Paul JacksonFix build breakage if CONFIG_SMP disabled, in Ashok's cpuhotplog patches in init/main.c routine fixup_cpu_present_map().
Make cpu_present_map a real map for all configurations, instead of a constant for non-SMP. Also move the definition of cpu_present_map out of kernel/cpu.c into kernel/sched.c, because cpu.c isn't compiled into non-SMP kernels.
============= include/linux/cpumask.h =============
include/linux/cpumask.h +5 -8
@@ -10,15 +10,12 @@extern cpumask_t cpu_online_map;extern cpumask_t cpu_possible_map; -extern cpumask_t cpu_present_map;#define num_online_cpus () cpus_weight(cpu_online_map)#define num_possible_cpus () cpus_weight(cpu_possible_map) -#define num_present_cpus () cpus_weight(cpu_present_map)#define cpu_online (cpu) cpu_isset(cpu, cpu_online_map)#define cpu_possible (cpu) cpu_isset(cpu, cpu_possible_map) -#define cpu_present (cpu) cpu_isset(cpu, cpu_present_map)#define for_each_cpu_mask (cpu, mask) \ for (cpu = first_cpu_const(mk_cpumask_const(mask)); \
@@ -27,25 +24,25 @@ extern cpumask_t cpu_present_map;#define for_each_cpu (cpu) for_each_cpu_mask(cpu, cpu_possible_map)#define for_each_online_cpu (cpu) for_each_cpu_mask(cpu, cpu_online_map) -#define for_each_present_cpu (cpu) for_each_cpu_mask(cpu, cpu_present_map)#else #define cpu_online_map cpumask_of_cpu(0)#define cpu_possible_map cpumask_of_cpu(0) -#define cpu_present_map cpumask_of_cpu(0)#define num_online_cpus () 1#define num_possible_cpus () 1 -#define num_present_cpus () 1#define cpu_online (cpu) ({ BUG_ON((cpu) != 0); 1; })#define cpu_possible (cpu) ({ BUG_ON((cpu) != 0); 1; }) -#define cpu_present (cpu) ({ BUG_ON((cpu) != 0); 1; })#define for_each_cpu (cpu) for (cpu = 0; cpu < 1; cpu++)#define for_each_online_cpu (cpu) for (cpu = 0; cpu < 1; cpu++) -#define for_each_present_cpu (cpu) for (cpu = 0; cpu < 1; cpu++)#endif +extern cpumask_t cpu_present_map; +#define num_present_cpus () cpus_weight(cpu_present_map) +#define cpu_present (cpu) cpu_isset(cpu, cpu_present_map) +#define for_each_present_cpu (cpu) for_each_cpu_mask(cpu, cpu_present_map) +#define cpumask_scnprintf (buf, buflen, map) \ bitmap_scnprintf(buf, buflen, cpus_addr(map), NR_CPUS)
============= kernel/cpu.c =============
kernel/cpu.c +0 -8
@@ -20,14 +20,6 @@DECLARE_MUTEX (cpucontrol);static struct notifier_block *cpu_chain; -/* -* Represents all cpu's present in the system -* In systems capable of hotplug, this map could dynamically grow -* as new cpu's are detected in the system via any platform specific -* method, such as ACPI for e.g. -*/ -cpumask_t cpu_present_map; -EXPORT_SYMBOL (cpu_present_map);/* Need to know about CPUs going up/down? */ int register_cpu_notifier (struct notifier_block *nb)
============= kernel/sched.c =============
kernel/sched.c +10 -0
@@ -3052,6 +3052,16 @@ out_unlock:return retval; } +/* +* Represents all cpu's present in the system +* In systems capable of hotplug, this map could dynamically grow +* as new cpu's are detected in the system via any platform specific +* method, such as ACPI for e.g. +*/ + +cpumask_t cpu_present_map; +EXPORT_SYMBOL (cpu_present_map); +/** * sys_sched_setaffinity - set the cpu affinity of a process * @pid: pid of the process
Signature · 3 lines
I won't rest till it's the best ...
Programmer, Linux Scalability
Paul Jackson <[email protected]> 1.650.933.1373PATCH 7/15 mask1-bitmap-cleanup-prep
In reply to Paul Jacksonmask1-bitmap-cleanup-prep
Document the bitmap bit model and handling of unused bits.
Tighten up bitmap so it does not generate nonzero bits in the unused tail if it is not given any on input.
Add intersects, subset, xor and andnot operators. Change bitmap_complement to take two operands.
Add a couple of missing 'const' qualifiers on bitops test_bit and bitmap_equal args.
arch/x86_64/kernel/smpboot.c +1 -1
@@ -827,7 +827,7 @@if (apicid == boot_cpu_id || (apicid ==BAD_APICID ))continue ; -if (!cpu_isset (apicid, phys_cpu_present_map)) +if (!physid_isset (apicid, phys_cpu_present_map))continue ;if ((max_cpus >=0 ) && (max_cpus <= cpucount+1 ))continue ;
include/asm-generic/bitops.h +1 -1
@@ -42,7 +42,7 @@return retval; } -extern __inline__int test_bit (int nr,long * addr) +extern __inline__int test_bit (int nr,const unsigned long * addr) {int mask;
include/asm-generic/cpumask_array.h +1 -1
@@ -17,7 +17,7 @@#define cpus_and (dst,src1,src2) bitmap_and((dst).mask,(src1).mask, (src2).mask, NR_CPUS)#define cpus_or (dst,src1,src2) bitmap_or((dst).mask, (src1).mask, (src2).mask, NR_CPUS)#define cpus_clear (map) bitmap_zero((map).mask, NR_CPUS) -#define cpus_complement (map) bitmap_complement((map).mask, NR_CPUS) +#define cpus_complement (map) bitmap_complement((map).mask, (map).mask, NR_CPUS)#define cpus_equal (map1, map2) bitmap_equal((map1).mask, (map2).mask, NR_CPUS)#define cpus_empty (map) bitmap_empty(map.mask, NR_CPUS)#define cpus_addr (map) ((map).mask)
include/asm-i386/mpspec.h +1 -1
@@ -53,7 +53,7 @@#define physids_and (dst, src1, src2) bitmap_and((dst).mask, (src1).mask, (src2).mask, MAX_APICS)#define physids_or (dst, src1, src2) bitmap_or((dst).mask, (src1).mask, (src2).mask, MAX_APICS)#define physids_clear (map) bitmap_zero((map).mask, MAX_APICS) -#define physids_complement (map) bitmap_complement((map).mask, MAX_APICS) +#define physids_complement (map) bitmap_complement((map).mask, (map).mask, MAX_APICS)#define physids_empty (map) bitmap_empty((map).mask, MAX_APICS)#define physids_equal (map1, map2) bitmap_equal((map1).mask, (map2).mask, MAX_APICS)#define physids_weight (map) bitmap_weight((map).mask, MAX_APICS)
include/asm-x86_64/mpspec.h +1 -1
@@ -212,7 +212,7 @@#define physids_and (dst, src1, src2) bitmap_and((dst).mask, (src1).mask, (src2).mask, MAX_APICS)#define physids_or (dst, src1, src2) bitmap_or((dst).mask, (src1).mask, (src2).mask, MAX_APICS)#define physids_clear (map) bitmap_zero((map).mask, MAX_APICS) -#define physids_complement (map) bitmap_complement((map).mask, MAX_APICS) +#define physids_complement (map) bitmap_complement((map).mask, (map).mask, MAX_APICS)#define physids_empty (map) bitmap_empty((map).mask, MAX_APICS)#define physids_equal (map1, map2) bitmap_equal((map1).mask, (map2).mask, MAX_APICS)#define physids_weight (map) bitmap_weight((map).mask, MAX_APICS)
include/linux/bitmap.h +10 -2
@@ -13,8 +13,8 @@int bitmap_empty (const unsigned long *bitmap,int bits);int bitmap_full (const unsigned long *bitmap,int bits);int bitmap_equal (const unsigned long *bitmap1, -unsigned long *bitmap2,int bits); -void bitmap_complement (unsigned long *bitmap,int bits); +const unsigned long *bitmap2,int bits); +void bitmap_complement (unsigned long *dst,const unsigned long *src,int bits);static inline void bitmap_zero (unsigned long *bitmap,int bits) {
@@ -41,6 +41,14 @@ const unsigned long *bitmap2, int bits);void bitmap_or (unsigned long *dst,const unsigned long *bitmap1,const unsigned long *bitmap2,int bits); +void bitmap_xor (unsigned long *dst,const unsigned long *bitmap1, +const unsigned long *bitmap2,int bits); +void bitmap_andnot (unsigned long *dst,const unsigned long *bitmap1, +const unsigned long *bitmap2,int bits); +int bitmap_intersects (const unsigned long *bitmap1, +const unsigned long *bitmap2,int bits); +int bitmap_subset (const unsigned long *bitmap1, +const unsigned long *bitmap2,int bits);int bitmap_weight (const unsigned long *bitmap,int bits);int bitmap_scnprintf (char *buf,unsigned int buflen,const unsigned long *maskp,int bits);
lib/bitmap.c +81 -6
113 lines not on this page · raw message
mm/mempolicy.c +5 -7
@@ -92,14 +92,12 @@/* Check if all specified nodes are online */ static int nodes_online (unsigned long *nodes) { -DECLARE_BITMAP (offline,MAX_NUMNODES ); +DECLARE_BITMAP (online2,MAX_NUMNODES ); -bitmap_copy (offline, node_online_map,MAX_NUMNODES ); -if (bitmap_empty (offline,MAX_NUMNODES )) -set_bit (0 , offline); -bitmap_complement (offline,MAX_NUMNODES ); -bitmap_and (offline, offline, nodes,MAX_NUMNODES ); -if (!bitmap_empty (offline,MAX_NUMNODES )) +bitmap_copy (online2, node_online_map,MAX_NUMNODES ); +if (bitmap_empty (online2,MAX_NUMNODES )) +set_bit (0 , online2); +if (!bitmap_subset (nodes, online2,MAX_NUMNODES ))return -EINVAL ;return 0 ; }
Signature · 3 lines
I won't rest till it's the best ...
Programmer, Linux Scalability
Paul Jackson <[email protected]> 1.650.933.1373PATCH 8/15 mask2-bitmap-extensions
In reply to Paul Jacksonmask2-bitmap-extensions
This bitmap improvements make it a suitable basis for fully supporting cpumask_t and nodemask_t. Inline macros with compile-time checks enable generating tight code on both small and large systems (large meaning cpumask_t requires more than one unsigned long's worth of bits).
The existing bitmap_<op> macros in lib/bitmap.c are renamed to __bitmap_<op>, and wrappers for each bitmap_<op> are exposed in include/linux/bitmap.h
This patch _includes_ Bill Irwins rewrite of the bitmap_shift operators to not require a fixed length intermediate bitmap.
Improved comments list each available operator for easy browsing.
include/linux/bitmap.h +229 -37
@@ -3,57 +3,249 @@#ifndef __ASSEMBLY__ -#include <linux/config.h> -#include <linux/compiler.h> #include <linux/types.h> -#include <linux/kernel.h> #include <linux/bitops.h> #include <linux/string.h> -int bitmap_empty (const unsigned long *bitmap,int bits); -int bitmap_full (const unsigned long *bitmap,int bits); -int bitmap_equal (const unsigned long *bitmap1, -const unsigned long *bitmap2,int bits); -void bitmap_complement (unsigned long *dst,const unsigned long *src,int bits); +/* +* bitmaps provide bit arrays that consume one or more unsigned +* longs. The bitmap interface and available operations are listed +* here, in bitmap.h +* +* Function implementations generic to all architectures are in +* lib/bitmap.c. Functions implementations that are architecture +* specific are in various include/asm-<arch>/bitops.h headers +* and other arch/<arch> specific files. +* +* See lib/bitmap.c for more details. +*/ + +/* +* The available bitmap operations and their rough meaning in the +* case that the bitmap is a single unsigned long are thus: +* +* bitmap_zero(dst, nbits) *dst = 0UL +* bitmap_fill(dst, nbits) *dst = ~0UL +* bitmap_copy(dst, src, nbits) *dst = *src +* bitmap_and(dst, src1, src2, nbits) *dst = *src1 & *src2 +* bitmap_or(dst, src1, src2, nbits) *dst = *src1 | *src2 +* bitmap_xor(dst, src1, src2, nbits) *dst = *src1 ^ *src2 +* bitmap_andnot(dst, src1, src2, nbits) *dst = *src1 & ~(*src2) +* bitmap_complement(dst, src, nbits) *dst = ~(*src) +* bitmap_equal(src1, src2, nbits) Are *src1 and *src2 equal? +* bitmap_intersects(src1, src2, nbits) Do *src1 and *src2 overlap? +* bitmap_subset(src1, src2, nbits) Is *src1 a subset of *src2? +* bitmap_empty(src, nbits) Are all bits zero in *src? +* bitmap_full(src, nbits) Are all bits set in *src? +* bitmap_weight(src, nbits) Hamming Weight: number set bits +* bitmap_shift_right(dst, src, n, nbits) *dst = *src >> n +* bitmap_shift_left(dst, src, n, nbits) *dst = *src << n +* bitmap_scnprintf(buf, len, src, nbits) Print bitmap src to buf +* bitmap_parse(ubuf, ulen, dst, nbits) Parse bitmap dst from buf +*/ + +/* +* Also the following operations in asm/bitops.h apply to bitmaps. +* +* set_bit(bit, addr) *addr |= bit +* clear_bit(bit, addr) *addr &= ~bit +* change_bit(bit, addr) *addr ^= bit +* test_bit(bit, addr) Is bit set in *addr? +* test_and_set_bit(bit, addr) Set bit and return old value +* test_and_clear_bit(bit, addr) Clear bit and return old value +* test_and_change_bit(bit, addr) Change bit and return old value +* find_first_zero_bit(addr, nbits) Position first zero bit in *addr +* find_first_bit(addr, nbits) Position first set bit in *addr +* find_next_zero_bit(addr, nbits, bit) Position next zero bit in *addr >= bit +* find_next_bit(addr, nbits, bit) Position next set bit in *addr >= bit +*/ + +/* +* The DECLARE_BITMAP(name,bits) macro, in linux/types.h, can be used +* to declare an array named 'name' of just enough unsigned longs to +* contain all bit positions from 0 to 'bits' - 1. +*/ + +/* +* lib/bitmap.c provides these functions: +*/ + +extern int __bitmap_empty (const unsigned long *bitmap,int bits); +extern int __bitmap_full (const unsigned long *bitmap,int bits); +extern int __bitmap_equal (const unsigned long *bitmap1, +const unsigned long *bitmap2,int bits); +extern void __bitmap_complement (unsigned long *dst,const unsigned long *src, +int bits); +extern void __bitmap_shift_right (unsigned long *dst, +const unsigned long *src,int shift,int bits); +extern void __bitmap_shift_left (unsigned long *dst, +const unsigned long *src,int shift,int bits); +extern void __bitmap_and (unsigned long *dst,const unsigned long *bitmap1, +const unsigned long *bitmap2,int bits); +extern void __bitmap_or (unsigned long *dst,const unsigned long *bitmap1, +const unsigned long *bitmap2,int bits); +extern void __bitmap_xor (unsigned long *dst,const unsigned long *bitmap1, +const unsigned long *bitmap2,int bits); +extern void __bitmap_andnot (unsigned long *dst,const unsigned long *bitmap1, +const unsigned long *bitmap2,int bits); +extern int __bitmap_intersects (const unsigned long *bitmap1, +const unsigned long *bitmap2,int bits); +extern int __bitmap_subset (const unsigned long *bitmap1, +const unsigned long *bitmap2,int bits); +extern int __bitmap_weight (const unsigned long *bitmap,int bits); + +extern int bitmap_scnprintf (char *buf,unsigned int len, +const unsigned long *src,int nbits); +extern int bitmap_parse (const char __user *ubuf,unsigned int ulen, +unsigned long *dst,int nbits); + +#define BITMAP_LAST_WORD_MASK (nbits) \ +( \ + ((nbits) % BITS_PER_LONG) ? \ + (1UL<<((nbits) % BITS_PER_LONG))-1 : ~0UL \ +) -static inline void bitmap_zero (unsigned long *bitmap,int bits) +static inline void bitmap_zero (unsigned long *dst,int nbits) { -memset (bitmap,0 ,BITS_TO_LONGS (bits)*sizeof (unsigned long )); +if (nbits <=BITS_PER_LONG ) + *dst =0UL ; +else { +int len =BITS_TO_LONGS (nbits) *sizeof (unsigned long ); +memset (dst,0 , len); + } } -static inline void bitmap_fill (unsigned long *bitmap,int bits) +static inline void bitmap_fill (unsigned long *dst,int nbits) { -memset (bitmap,0xff ,BITS_TO_LONGS (bits)*sizeof (unsigned long )); +size_t nlongs =BITS_TO_LONGS (nbits); +if (nlongs >1 ) { +int len = (nlongs -1 ) *sizeof (unsigned long ); +memset (dst,0xff , len); + } + dst[nlongs -1 ] =BITMAP_LAST_WORD_MASK (nbits); } -static inline void bitmap_copy (unsigned long *dst, -const unsigned long *src,int bits) +static inline void bitmap_copy (unsigned long *dst,const unsigned long *src, +int nbits) { -int len =BITS_TO_LONGS (bits)*sizeof (unsigned long ); -memcpy (dst, src, len); +if (nbits <=BITS_PER_LONG ) + *dst = *src; +else { +int len =BITS_TO_LONGS (nbits) *sizeof (unsigned long ); +memcpy (dst, src, len); + } } -void bitmap_shift_right (unsigned long *dst, -const unsigned long *src,int shift,int bits); -void bitmap_shift_left (unsigned long *dst, -const unsigned long *src,int shift,int bits); -void bitmap_and (unsigned long *dst,const unsigned long *bitmap1, -const unsigned long *bitmap2,int bits); -void bitmap_or (unsigned long *dst,const unsigned long *bitmap1, -const unsigned long *bitmap2,int bits); -void bitmap_xor (unsigned long *dst,const unsigned long *bitmap1, -const unsigned long *bitmap2,int bits); -void bitmap_andnot (unsigned long *dst,const unsigned long *bitmap1, -const unsigned long *bitmap2,int bits); -int bitmap_intersects (const unsigned long *bitmap1, -const unsigned long *bitmap2,int bits); -int bitmap_subset (const unsigned long *bitmap1, -const unsigned long *bitmap2,int bits); -int bitmap_weight (const unsigned long *bitmap,int bits); -int bitmap_scnprintf (char *buf,unsigned int buflen, -const unsigned long *maskp,int bits); -int bitmap_parse (const char __user *ubuf,unsigned int ubuflen, -unsigned long *maskp,int bits); +static inline void bitmap_and (unsigned long *dst,const unsigned long *src1, +const unsigned long *src2,int nbits) +{ +if (nbits <=BITS_PER_LONG ) + *dst = *src1 & *src2; +else +__bitmap_and (dst, src1, src2, nbits); +} + +static inline void bitmap_or (unsigned long *dst,const unsigned long *src1, +const unsigned long *src2,int nbits) +{ +if (nbits <=BITS_PER_LONG ) + *dst = *src1 | *src2; +else +__bitmap_or (dst, src1, src2, nbits); +} + +static inline void bitmap_xor (unsigned long *dst,const unsigned long *src1, +const unsigned long *src2,int nbits) +{ +if (nbits <=BITS_PER_LONG ) + *dst = *src1 ^ *src2; +else +__bitmap_xor (dst, src1, src2, nbits); +} + +static inline void bitmap_andnot (unsigned long *dst,const unsigned long *src1, +const unsigned long *src2,int nbits) +{ +if (nbits <=BITS_PER_LONG ) + *dst = *src1 & ~(*src2); +else +__bitmap_andnot (dst, src1, src2, nbits); +} + +static inline void bitmap_complement (unsigned long *dst,const unsigned long *src, +int nbits) +{ +if (nbits <=BITS_PER_LONG ) + *dst = ~(*src) &BITMAP_LAST_WORD_MASK (nbits); +else +__bitmap_complement (dst, src, nbits); +} + +static inline int bitmap_equal (const unsigned long *src1, +const unsigned long *src2,int nbits) +{ +if (nbits <=BITS_PER_LONG ) +return ! ((*src1 ^ *src2) &BITMAP_LAST_WORD_MASK (nbits)); +else +return __bitmap_equal (src1, src2, nbits); +} + +static inline int bitmap_intersects (const unsigned long *src1, +const unsigned long *src2,int nbits) +{ +if (nbits <=BITS_PER_LONG ) +return ((*src1 & *src2) &BITMAP_LAST_WORD_MASK (nbits)) !=0 ; +else +return __bitmap_intersects (src1, src2, nbits); +} + +static inline int bitmap_subset (const unsigned long *src1, +const unsigned long *src2,int nbits) +{ +if (nbits <=BITS_PER_LONG ) +return ! ((*src1 & ~(*src2)) &BITMAP_LAST_WORD_MASK (nbits)); +else +return __bitmap_subset (src1, src2, nbits); +} + +static inline int bitmap_empty (const unsigned long *src,int nbits) +{ +if (nbits <=BITS_PER_LONG ) +return ! (*src &BITMAP_LAST_WORD_MASK (nbits)); +else +return __bitmap_empty (src, nbits); +} + +static inline int bitmap_full (const unsigned long *src,int nbits) +{ +if (nbits <=BITS_PER_LONG ) +return ! (~(*src) &BITMAP_LAST_WORD_MASK (nbits)); +else +return __bitmap_full (src, nbits); +} + +static inline int bitmap_weight (const unsigned long *src,int nbits) +{ +return __bitmap_weight (src, nbits); +} + +static inline void bitmap_shift_right (unsigned long *dst, +const unsigned long *src,int n,int nbits) +{ +if (nbits <=BITS_PER_LONG ) + *dst = *src >> n; +else +__bitmap_shift_right (dst, src, n, nbits); +} + +static inline void bitmap_shift_left (unsigned long *dst, +const unsigned long *src,int n,int nbits) +{ +if (nbits <=BITS_PER_LONG ) + *dst = (*src << n) &BITMAP_LAST_WORD_MASK (nbits); +else +__bitmap_shift_left (dst, src, n, nbits); +}#endif /* __ASSEMBLY__ */
lib/bitmap.c +47 -44
249 lines not on this page · raw message
Signature · 3 lines
I won't rest till it's the best ...
Programmer, Linux Scalability
Paul Jackson <[email protected]> 1.650.933.1373PATCH 9/15 mask3-unline-find-next-bit-ia64
In reply to Paul Jacksonmask3-unline-find-next-bit-ia64
Move the page of code (~700 bytes of instructions) for find_next_bit and find_next_zero_bit from inline in include/asm-ia64/bitops.h to a real function in arch/ia64/lib/bitops.c, leaving a declaration and macro wrapper behind.
The other arch's with almost this same code might want to also uninline it: alpha, parisc, ppc, sh, sparc, sparc64.
These are too big to inline.
include/asm-ia64/bitops.h +10 -82
@@ -11,7 +11,7 @@#include <linux/compiler.h> #include <linux/types.h> - +#include <asm/bitops.h> #include <asm/intrinsics.h> /**
@@ -359,93 +359,21 @@#endif /* __KERNEL__ */ -/* -* Find next zero bit in a bitmap reasonably efficiently.. -*/ -static inline int -find_next_zero_bit (void *addr,unsigned long size,unsigned long offset) -{ -unsigned long *p = ((unsigned long *) addr) + (offset >>6 ); -unsigned long result = offset & ~63UL ; -unsigned long tmp; - -if (offset >= size) -return size; - size -= result; - offset &=63UL ; -if (offset) { - tmp = *(p++); - tmp |= ~0UL >> (64 -offset); -if (size <64 ) - goto found_first; -if (~tmp) - goto found_middle; - size -=64 ; - result +=64 ; - } -while (size & ~63UL ) { -if (~(tmp = *(p++))) - goto found_middle; - result +=64 ; - size -=64 ; - } -if (!size) -return result; - tmp = *p; -found_first: - tmp |= ~0UL << size; -if (tmp == ~0UL )/* any bits zero? */ -return result + size;/* nope */ -found_middle: -return result +ffz (tmp); -} +extern int __find_next_zero_bit (void *addr,unsigned long size, \ +unsigned long offset); +extern int __find_next_bit (const void *addr,unsigned long size, \ +unsigned long offset); + +#define find_next_zero_bit (addr, size, offset) \ + __find_next_zero_bit((addr), (size), (offset)) +#define find_next_bit (addr, size, offset) \ + __find_next_bit((addr), (size), (offset))/* * The optimizer actually does good code for this case.. */ #define find_first_zero_bit (addr, size) find_next_zero_bit((addr), (size), 0) -/* -* Find next bit in a bitmap reasonably efficiently.. -*/ -static inline int -find_next_bit (const void *addr,unsigned long size,unsigned long offset) -{ -unsigned long *p = ((unsigned long *) addr) + (offset >>6 ); -unsigned long result = offset & ~63UL ; -unsigned long tmp; - -if (offset >= size) -return size; - size -= result; - offset &=63UL ; -if (offset) { - tmp = *(p++); - tmp &= ~0UL << offset; -if (size <64 ) - goto found_first; -if (tmp) - goto found_middle; - size -=64 ; - result +=64 ; - } -while (size & ~63UL ) { -if ((tmp = *(p++))) - goto found_middle; - result +=64 ; - size -=64 ; - } -if (!size) -return result; - tmp = *p; - found_first: - tmp &= ~0UL >> (64 -size); -if (tmp ==0UL )/* Are any bits set? */ -return result + size;/* Nope. */ - found_middle: -return result +__ffs (tmp); -} -#define find_first_bit (addr, size) find_next_bit((addr), (size), 0)#ifdef __KERNEL__
arch/ia64/lib/bitop.c +88 -0
88 lines not on this page · raw message
arch/ia64/lib/Makefile +1 -1
@@ -6,7 +6,7 @@ lib-y := __divsi3.o __udivsi3.o __modsi3.o __umodsi3.o \ __divdi3.o __udivdi3.o __moddi3.o __umoddi3.o \ - checksum.o clear_page.o csum_partial_copy.o copy_page.o \ + bitop.o checksum.o clear_page.o csum_partial_copy.o copy_page.o \ clear_user.o strncpy_from_user.o strlen_user.o strnlen_user.o \ flush.o ip_fast_csum.o do_csum.o \ memset.o strlen.o swiotlb.o
Signature · 3 lines
I won't rest till it's the best ...
Programmer, Linux Scalability
Paul Jackson <[email protected]> 1.650.933.1373PATCH 10/15 mask4-new-cpumask-h
In reply to Paul Jacksonmask4-new-cpumask-h
Major rewrite of cpumask to use a single implementation, as a struct-wrapped bitmap.
This patch leaves some 26 include/asm-*/cpumask*.h header files orphaned - to be removed next patch.
Some nine cpumask macros for const variants and to coerce and promote between an unsigned long and a cpumask are obsolete. Simple emulation wrappers are provided in this patch, which can be removed once each of the 3 archs (i386, ppc64, x86_64) using them are recoded in follow-on patches to not need them.
The CPU_MASK_ALL macro now avoids leaving possible garbage one bits in any unused portion of the high word.
An inproved comment lists all available operators, for convenient browsing.
include/linux/cpumask.h +314 -31
@@ -1,52 +1,335 @@#ifndef __LINUX_CPUMASK_H#define __LINUX_CPUMASK_H +/* +* Cpumasks provide a bitmap suitable for representing the +* set of CPU's in a system, one bit position per CPU number. +* +* See detailed comments in the file linux/bitmap.h describing the +* data type on which these cpumasks are based. +* +* For details of cpumask_scnprintf() and cpumask_parse(), +* see bitmap_scnprintf() and bitmap_parse() in lib/bitmap.c. +* +* The available cpumask operations are: +* +* void cpu_set(cpu, mask) turn on bit 'cpu' in mask +* void cpu_clear(cpu, mask) turn off bit 'cpu' in mask +* void cpus_setall(mask) set all bits +* void cpus_clear(mask) clear all bits +* int cpu_isset(cpu, mask) true iff bit 'cpu' set in mask +* int cpu_test_and_set(cpu, mask) test and set bit 'cpu' in mask +* +* void cpus_and(dst, src1, src2) dst = src1 & src2 [intersection] +* void cpus_or(dst, src1, src2) dst = src1 | src2 [union] +* void cpus_xor(dst, src1, src2) dst = src1 ^ src2 +* void cpus_andnot(dst, src1, src2) dst = src1 & ~src2 +* void cpus_complement(dst, src) dst = ~src +* +* int cpus_equal(mask1, mask2) Does mask1 == mask2? +* int cpus_intersects(mask1, mask2) Do mask1 and mask2 intersect? +* int cpus_subset(mask1, mask2) Is mask1 a subset of mask2? +* int cpus_empty(mask) Is mask empty (no bits sets)? +* int cpus_full(mask) Is mask full (all bits sets)? +* int cpus_weight(mask) Hamming weigh - number of set bits +* +* void cpus_shift_right(dst, src, n) Shift right +* void cpus_shift_left(dst, src, n) Shift left +* +* int first_cpu(mask) Number lowest set bit, or NR_CPUS +* int next_cpu(cpu, mask) Next cpu past 'cpu', or NR_CPUS +* +* cpumask_t cpumask_of_cpu(cpu) Return cpumask with bit 'cpu' set +* CPU_MASK_ALL Initializer - all bits set +* CPU_MASK_NONE Initializer - no bits set +* unsigned long *cpus_addr(mask) Array of unsigned long's in mask +* +* int cpumask_scnprintf(buf, len, mask) Format cpumask for printing +* int cpumask_parse(ubuf, ulen, mask) Parse ascii string as cpumask +* +* int num_online_cpus() Number of online CPUs +* int num_possible_cpus() Number of all possible CPUs +* int cpu_online(cpu) Is some cpu online? +* int cpu_possible(cpu) Is some cpu possible? +* void cpu_set_online(cpu) set cpu in cpu_online_map +* void cpu_set_offline(cpu) clear cpu in cpu_online_map +* int any_online_cpu(mask) First online cpu in mask +* +* for_each_cpu_mask(cpu, mask) for-loop cpu over mask +* for_each_cpu(cpu) for-loop cpu over cpu_possible_map +* for_each_online_cpu(cpu) for-loop cpu over cpu_online_map +*/ +#include <linux/threads.h> #include <linux/bitmap.h> -#include <asm/cpumask.h> #include <asm/bug.h> -#ifdef CONFIG_SMP +typedef struct { DECLARE_BITMAP(bits, NR_CPUS); } cpumask_t; +extern cpumask_t _unused_cpumask_arg_; -extern cpumask_t cpu_online_map; -extern cpumask_t cpu_possible_map; +#define cpu_set (cpu, dst) __cpu_set((cpu), &(dst)) +static inline void __cpu_set (int cpu,volatile cpumask_t *dstp) +{ +if (cpu <NR_CPUS ) +set_bit (cpu, dstp->bits); +} + +#define cpu_clear (cpu, dst) __cpu_clear((cpu), &(dst)) +static inline void __cpu_clear (int cpu,volatile cpumask_t *dstp) +{ +clear_bit (cpu, dstp->bits); +} + +#define cpus_setall (dst) __cpus_setall(&(dst), NR_CPUS) +static inline void __cpus_setall (cpumask_t *dstp,int nbits) +{ +bitmap_fill (dstp->bits, nbits); +} + +#define cpus_clear (dst) __cpus_clear(&(dst), NR_CPUS) +static inline void __cpus_clear (cpumask_t *dstp,int nbits) +{ +bitmap_zero (dstp->bits, nbits); +} + +#define cpu_isset (cpu, cpumask) __cpu_isset((cpu), &(cpumask)) +static inline int __cpu_isset (int cpu,const volatile cpumask_t *addr) +{ +return test_bit (cpu, addr->bits); +} + +#define cpu_test_and_set (cpu, cpumask) __cpu_test_and_set((cpu), &(cpumask)) +static inline int __cpu_test_and_set (int cpu,cpumask_t *addr) +{ +if (cpu <NR_CPUS ) +return test_and_set_bit (cpu, addr->bits); +else +return 0 ; +} + +#define cpus_and (dst, src1, src2) __cpus_and(&(dst), &(src1), &(src2), NR_CPUS) +static inline void __cpus_and (cpumask_t *dstp,cpumask_t *src1p, +cpumask_t *src2p,int nbits) +{ +bitmap_and (dstp->bits, src1p->bits, src2p->bits, nbits); +} + +#define cpus_or (dst, src1, src2) __cpus_or(&(dst), &(src1), &(src2), NR_CPUS) +static inline void __cpus_or (cpumask_t *dstp,cpumask_t *src1p, +cpumask_t *src2p,int nbits) +{ +bitmap_or (dstp->bits, src1p->bits, src2p->bits, nbits); +} + +#define cpus_xor (dst, src1, src2) __cpus_xor(&(dst), &(src1), &(src2), NR_CPUS) +static inline void __cpus_xor (cpumask_t *dstp,cpumask_t *src1p, +cpumask_t *src2p,int nbits) +{ +bitmap_xor (dstp->bits, src1p->bits, src2p->bits, nbits); +} + +#define cpus_andnot (dst, src1, src2) \ + __cpus_andnot(&(dst), &(src1), &(src2), NR_CPUS) +static inline void __cpus_andnot (cpumask_t *dstp,cpumask_t *src1p, +cpumask_t *src2p,int nbits) +{ +bitmap_andnot (dstp->bits, src1p->bits, src2p->bits, nbits); +} + +#define cpus_complement (dst, src) __cpus_complement(&(dst), &(src), NR_CPUS) +static inline void __cpus_complement (cpumask_t *dstp, +cpumask_t *srcp,int nbits) +{ +bitmap_complement (dstp->bits, srcp->bits, nbits); +} + +#define cpus_equal (src1, src2) __cpus_equal(&(src1), &(src2), NR_CPUS) +static inline int __cpus_equal (cpumask_t *src1p, +cpumask_t *src2p,int nbits) +{ +return bitmap_equal (src1p->bits, src2p->bits, nbits); +} + +#define cpus_intersects (src1, src2) __cpus_intersects(&(src1), &(src2), NR_CPUS) +static inline int __cpus_intersects (cpumask_t *src1p, +cpumask_t *src2p,int nbits) +{ +return bitmap_intersects (src1p->bits, src2p->bits, nbits); +} + +#define cpus_subset (src1, src2) __cpus_subset(&(src1), &(src2), NR_CPUS) +static inline int __cpus_subset (cpumask_t *src1p, +cpumask_t *src2p,int nbits) +{ +return bitmap_subset (src1p->bits, src2p->bits, nbits); +} + +#define cpus_empty (src) __cpus_empty(&(src), NR_CPUS) +static inline int __cpus_empty (cpumask_t *srcp,int nbits) +{ +return bitmap_empty (srcp->bits, nbits); +} + +#define cpus_full (cpumask) __cpus_full(&(cpumask), NR_CPUS) +static inline int __cpus_full (cpumask_t *srcp,int nbits) +{ +return bitmap_full (srcp->bits, nbits); +} + +#define cpus_weight (cpumask) __cpus_weight(&(cpumask), NR_CPUS) +static inline int __cpus_weight (cpumask_t *srcp,int nbits) +{ +return bitmap_weight (srcp->bits, nbits); +} + +#define cpus_shift_right (dst, src, n) \ + __cpus_shift_right(&(dst), &(src), (n), NR_CPUS) +static inline void __cpus_shift_right (cpumask_t *dstp, +cpumask_t *srcp,int n,int nbits) +{ +bitmap_shift_right (dstp->bits, srcp->bits, n, nbits); +} + +#define cpus_shift_left (dst, src, n) \ + __cpus_shift_left(&(dst), &(src), (n), NR_CPUS) +static inline void __cpus_shift_left (cpumask_t *dstp, +cpumask_t *srcp,int n,int nbits) +{ +bitmap_shift_left (dstp->bits, srcp->bits, n, nbits); +} + +#define first_cpu (src) __first_cpu(&(src), NR_CPUS) +static inline int __first_cpu (cpumask_t *srcp,int nbits) +{ +return find_first_bit (srcp->bits, nbits); +} + +#define next_cpu (n, src) __next_cpu((n), &(src), NR_CPUS) +static inline int __next_cpu (int n,cpumask_t *srcp,int nbits) +{ +return find_next_bit (srcp->bits, nbits, n+1 ); +} + +#define cpumask_of_cpu (cpu) \ +({ \ + typeof(_unused_cpumask_arg_) m; \ + int c = cpu; \ + if (sizeof(m) == sizeof(unsigned long)) { \ + if (c < NR_CPUS) \ + m.bits[0] = 1UL<<c; \ + } else { \ + cpus_clear(m); \ + cpu_set(c, m); \ + } \ + m; \ +}) + +#define CPU_MASK_LAST_WORD BITMAP_LAST_WORD_MASK(NR_CPUS) + +#if NR_CPUS <=BITS_PER_LONG + +#define CPU_MASK_ALL \ +((cpumask_t) { { \ + [BITS_TO_LONGS(NR_CPUS)-1] = CPU_MASK_LAST_WORD \ +} }) -#define num_online_cpus () cpus_weight(cpu_online_map) -#define num_possible_cpus () cpus_weight(cpu_possible_map) +#else -#define cpu_online (cpu) cpu_isset(cpu, cpu_online_map) -#define cpu_possible (cpu) cpu_isset(cpu, cpu_possible_map) +#define CPU_MASK_ALL \ +((cpumask_t) { { \ + [0 ... BITS_TO_LONGS(NR_CPUS)-2] = ~0UL, \ + [BITS_TO_LONGS(NR_CPUS)-1] = CPU_MASK_LAST_WORD \ +} }) -#define for_each_cpu_mask (cpu, mask) \ - for (cpu = first_cpu_const(mk_cpumask_const(mask)); \ - cpu < NR_CPUS; \ - cpu = next_cpu_const(cpu, mk_cpumask_const(mask))) +#endif -#define for_each_cpu (cpu) for_each_cpu_mask(cpu, cpu_possible_map) -#define for_each_online_cpu (cpu) for_each_cpu_mask(cpu, cpu_online_map) -#else -#define cpu_online_map cpumask_of_cpu(0) -#define cpu_possible_map cpumask_of_cpu(0) +#define CPU_MASK_NONE \ +{ { \ + [0 ... BITS_TO_LONGS(NR_CPUS)-1] = 0UL \ +} } + +#define cpus_addr (src) ((src).bits) + +#define cpumask_scnprintf (buf, len, src) \ + __cpumask_scnprintf((buf), (len), &(src), NR_CPUS) +static inline int __cpumask_scnprintf (char *buf,int len, +cpumask_t *srcp,int nbits) +{ +return bitmap_scnprintf (buf, len, srcp->bits, nbits); +} + +#define cpumask_parse (ubuf, ulen, src) \ + __cpumask_parse((ubuf), (ulen), &(src), NR_CPUS) +static inline int __cpumask_parse (const char __user *buf,int len, +cpumask_t *srcp,int nbits) +{ +return bitmap_parse (buf, len, srcp->bits, nbits); +} + +/* +* The following particular system cpumasks and operations +* on them manage all (possible) and online cpus. +*/ -#define num_online_cpus () 1 -#define num_possible_cpus () 1 +extern cpumask_t cpu_online_map; +extern cpumask_t cpu_possible_map; -#define cpu_online (cpu) ({ BUG_ON((cpu) != 0); 1; }) -#define cpu_possible (cpu) ({ BUG_ON((cpu) != 0); 1; }) +#ifdef CONFIG_SMP -#define for_each_cpu (cpu) for (cpu = 0; cpu < 1; cpu++) -#define for_each_online_cpu (cpu) for (cpu = 0; cpu < 1; cpu++) -#endif +#define num_online_cpus () cpus_weight(cpu_online_map) +#define num_possible_cpus () cpus_weight(cpu_possible_map) +#define cpu_online (cpu) cpu_isset((cpu), cpu_online_map) +#define cpu_possible (cpu) cpu_isset((cpu), cpu_possible_map) +#define cpu_set_online (cpu) cpu_set((cpu), cpu_online_map) +#define cpu_set_offline (cpu) cpu_clear((cpu), cpu_online_map) + +#define any_online_cpu (mask) \ +({ \ + cpumask_t m; \ + cpus_and(m, mask, cpu_online_map); \ + first_cpu(m); \ +}) + +#define for_each_cpu_mask (cpu, mask) \ + for (cpu = first_cpu(mask); \ + cpu < NR_CPUS; \ + cpu = next_cpu(cpu, mask)) + +#else /* !CONFIG_SMP */ + +#define num_online_cpus () 1 +#define num_possible_cpus () 1 +#define cpu_online (cpu) ({ BUG_ON((cpu) != 0); 1; }) +#define cpu_possible (cpu) ({ BUG_ON((cpu) != 0); 1; }) +#define cpu_set_online (cpu) ({ BUG_ON((cpu) != 0); }) +#define cpu_set_offline (cpu) ({ BUG(); }) + +#define any_online_cpu (mask) 0 + +#define for_each_cpu_mask (cpu, mask) for (cpu = 0; cpu < 1; cpu++) + +#endif /* CONFIG_SMP */ + +#define for_each_cpu (cpu) \ + for_each_cpu_mask(cpu, cpu_possible_map) +#define for_each_online_cpu (cpu) \ + for_each_cpu_mask(cpu, cpu_online_map)extern cpumask_t cpu_present_map; -#define num_present_cpus () cpus_weight(cpu_present_map) -#define cpu_present (cpu) cpu_isset(cpu, cpu_present_map) +#define num_present_cpus () cpus_weight(cpu_present_map) +#define cpu_present (cpu) cpu_isset(cpu, cpu_present_map)#define for_each_present_cpu (cpu) for_each_cpu_mask(cpu, cpu_present_map) -#define cpumask_scnprintf (buf, buflen, map) \ - bitmap_scnprintf(buf, buflen, cpus_addr(map), NR_CPUS) - -#define cpumask_parse (buf, buflen, map) \ - bitmap_parse(buf, buflen, cpus_addr(map), NR_CPUS) +/* Begin obsolete cpumask operator emulation */ +#define cpu_isset_const (a,b) cpu_isset(a,b) +#define cpumask_const_t cpumask_t +#define cpus_coerce (m) (cpus_addr(m)[0]) +#define cpus_coerce_const cpus_coerce +#define cpus_promote (x) ({ cpumask_t m; m.bits[0] = x; m; }) +#define cpus_weight_const cpus_weight +#define first_cpu_const first_cpu +#define mk_cpumask_const (x) x +#define next_cpu_const next_cpu +/* End of obsolete cpumask operator emulation */ #endif /* __LINUX_CPUMASK_H */
kernel/sched.c +5 -0
@@ -3112,6 +3112,11 @@return retval; } +#ifndef CONFIG_SMP +cpumask_t cpu_online_map =CPU_MASK_ALL ; +cpumask_t cpu_possible_map =CPU_MASK_ALL ; +#endif +/** * sys_sched_getaffinity - get the cpu affinity of a process * @pid: pid of the process
arch/sparc64/kernel/irq.c +4 -2
@@ -687,9 +687,10 @@* Just Do It. */ struct irqaction *ap = bp->irq_info; -cpumask_t cpu_mask =get_smpaff_in_irqaction (ap); +cpumask_t cpu_mask;unsigned int buddy, ticks; +cpus_addr (cpu_mask)[0 ] =get_smpaff_in_irqaction (ap);cpus_and (cpu_mask, cpu_mask, cpu_online_map);if (cpus_empty (cpu_mask)) cpu_mask = cpu_online_map;
@@ -1206,9 +1207,10 @@ {struct ino_bucket *bp = ivector_table + (long )data;struct irqaction *ap = bp->irq_info; -cpumask_t mask =get_smpaff_in_irqaction (ap); +cpumask_t mask;int len; +cpus_addr (mask)[0 ] =get_smpaff_in_irqaction (ap);if (cpus_empty (mask)) mask = cpu_online_map;
kernel/rcupdate.c +1 -5
@@ -103,8 +103,6 @@*/ static void rcu_start_batch (long newbatch) { -cpumask_t active; -if (rcu_batch_before (rcu_ctrlblk.maxbatch, newbatch)) { rcu_ctrlblk.maxbatch = newbatch; }
@@ -113,9 +111,7 @@return ; }/* Can't change, since spin lock held. */ - active = idle_cpu_mask; -cpus_complement (active); -cpus_and (rcu_ctrlblk.rcu_cpu_mask, cpu_online_map, active); +cpus_andnot (rcu_ctrlblk.rcu_cpu_mask, cpu_online_map, idle_cpu_mask); }/*
Signature · 3 lines
I won't rest till it's the best ...
Programmer, Linux Scalability
Paul Jackson <[email protected]> 1.650.933.1373PATCH 11/15 mask5-remove-old-cpumask-files
In reply to Paul Jacksonmask5-remove-old-cpumask-files
With the cpumask rewrite in the previous patch, these various include/asm-*/cpumask*.h headers are no longer used.
include/asm-alpha/cpumask.h +0 -6
@@ -1,6 +0,0 @@ -#ifndef _ASM_ALPHA_CPUMASK_H -#define _ASM_ALPHA_CPUMASK_H - -#include <asm-generic/cpumask.h> - -#endif /* _ASM_ALPHA_CPUMASK_H */
include/asm-arm/cpumask.h +0 -6
@@ -1,6 +0,0 @@ -#ifndef _ASM_ARM_CPUMASK_H -#define _ASM_ARM_CPUMASK_H - -#include <asm-generic/cpumask.h> - -#endif /* _ASM_ARM_CPUMASK_H */
include/asm-arm26/cpumask.h +0 -6
@@ -1,6 +0,0 @@ -#ifndef _ASM_ARM26_CPUMASK_H -#define _ASM_ARM26_CPUMASK_H - -#include <asm-generic/cpumask.h> - -#endif /* _ASM_ARM26_CPUMASK_H */
include/asm-cris/cpumask.h +0 -6
@@ -1,6 +0,0 @@ -#ifndef _ASM_CRIS_CPUMASK_H -#define _ASM_CRIS_CPUMASK_H - -#include <asm-generic/cpumask.h> - -#endif /* _ASM_CRIS_CPUMASK_H */
include/asm-generic/cpumask.h +0 -40
@@ -1,40 +0,0 @@ -#ifndef __ASM_GENERIC_CPUMASK_H -#define __ASM_GENERIC_CPUMASK_H - -#include <linux/config.h> -#include <linux/kernel.h> -#include <linux/threads.h> -#include <linux/types.h> -#include <linux/bitmap.h> - -#if NR_CPUS >BITS_PER_LONG &&NR_CPUS !=1 -#define CPU_ARRAY_SIZE BITS_TO_LONGS(NR_CPUS) - -struct cpumask -{ -unsigned long mask[CPU_ARRAY_SIZE ]; -}; - -typedef struct cpumask cpumask_t ; - -#else -typedef unsigned long cpumask_t ; -#endif - -#ifdef CONFIG_SMP -#if NR_CPUS >BITS_PER_LONG -#include <asm-generic/cpumask_array.h> -#else -#include <asm-generic/cpumask_arith.h> -#endif -#else -#include <asm-generic/cpumask_up.h> -#endif - -#if NR_CPUS <=4 *BITS_PER_LONG -#include <asm-generic/cpumask_const_value.h> -#else -#include <asm-generic/cpumask_const_reference.h> -#endif - -#endif /* __ASM_GENERIC_CPUMASK_H */
include/asm-generic/cpumask_arith.h +0 -49
49 lines not on this page · raw message
include/asm-generic/cpumask_array.h +0 -54
54 lines not on this page · raw message
include/asm-generic/cpumask_const_reference.h +0 -29
@@ -1,29 +0,0 @@ -#ifndef __ASM_GENERIC_CPUMASK_CONST_REFERENCE_H -#define __ASM_GENERIC_CPUMASK_CONST_REFERENCE_H - -struct cpumask_ref { -const cpumask_t *val; -}; - -typedef const struct cpumask_ref cpumask_const_t ; - -#define mk_cpumask_const (map) ((cpumask_const_t){ &(map) }) -#define cpu_isset_const (cpu, map) cpu_isset(cpu, *(map).val) - -#define cpus_and_const (dst,src1,src2) cpus_and(dst,*(src1).val,*(src2).val) -#define cpus_or_const (dst,src1,src2) cpus_or(dst,*(src1).val,*(src2).val) - -#define cpus_equal_const (map1, map2) cpus_equal(*(map1).val, *(map2).val) - -#define cpus_copy_const (map1, map2) bitmap_copy((map1).mask, (map2).val->mask, NR_CPUS) - -#define cpus_empty_const (map) cpus_empty(*(map).val) -#define cpus_weight_const (map) cpus_weight(*(map).val) -#define first_cpu_const (map) first_cpu(*(map).val) -#define next_cpu_const (cpu, map) next_cpu(cpu, *(map).val) - -/* only ever use this for things that are _never_ used on large boxen */ -#define cpus_coerce_const (map) cpus_coerce(*(map).val) -#define any_online_cpu_const (map) any_online_cpu(*(map).val) - -#endif /* __ASM_GENERIC_CPUMASK_CONST_REFERENCE_H */
include/asm-generic/cpumask_const_value.h +0 -21
@@ -1,21 +0,0 @@ -#ifndef __ASM_GENERIC_CPUMASK_CONST_VALUE_H -#define __ASM_GENERIC_CPUMASK_CONST_VALUE_H - -typedef const cpumask_t cpumask_const_t ; - -#define mk_cpumask_const (map) (map) -#define cpu_isset_const (cpu, map) cpu_isset(cpu, map) -#define cpus_and_const (dst,src1,src2) cpus_and(dst, src1, src2) -#define cpus_or_const (dst,src1,src2) cpus_or(dst, src1, src2) -#define cpus_equal_const (map1, map2) cpus_equal(map1, map2) -#define cpus_empty_const (map) cpus_empty(map) -#define cpus_copy_const (map1, map2) do { map1 = (cpumask_t)map2; } while (0) -#define cpus_weight_const (map) cpus_weight(map) -#define first_cpu_const (map) first_cpu(map) -#define next_cpu_const (cpu, map) next_cpu(cpu, map) - -/* only ever use this for things that are _never_ used on large boxen */ -#define cpus_coerce_const (map) cpus_coerce(map) -#define any_online_cpu_const (map) any_online_cpu(map) - -#endif /* __ASM_GENERIC_CPUMASK_CONST_VALUE_H */
include/asm-generic/cpumask_up.h +0 -59
59 lines not on this page · raw message
include/asm-h8300/cpumask.h +0 -6
@@ -1,6 +0,0 @@ -#ifndef _ASM_H8300_CPUMASK_H -#define _ASM_H8300_CPUMASK_H - -#include <asm-generic/cpumask.h> - -#endif /* _ASM_H8300_CPUMASK_H */
include/asm-i386/cpumask.h +0 -6
@@ -1,6 +0,0 @@ -#ifndef _ASM_I386_CPUMASK_H -#define _ASM_I386_CPUMASK_H - -#include <asm-generic/cpumask.h> - -#endif /* _ASM_I386_CPUMASK_H */
include/asm-m68k/cpumask.h +0 -6
@@ -1,6 +0,0 @@ -#ifndef _ASM_M68K_CPUMASK_H -#define _ASM_M68K_CPUMASK_H - -#include <asm-generic/cpumask.h> - -#endif /* _ASM_M68K_CPUMASK_H */
include/asm-m68knommu/cpumask.h +0 -6
@@ -1,6 +0,0 @@ -#ifndef _ASM_M68KNOMMU_CPUMASK_H -#define _ASM_M68KNOMMU_CPUMASK_H - -#include <asm-generic/cpumask.h> - -#endif /* _ASM_M68KNOMMU_CPUMASK_H */
include/asm-mips/cpumask.h +0 -6
@@ -1,6 +0,0 @@ -#ifndef _ASM_MIPS_CPUMASK_H -#define _ASM_MIPS_CPUMASK_H - -#include <asm-generic/cpumask.h> - -#endif /* _ASM_MIPS_CPUMASK_H */
include/asm-parisc/cpumask.h +0 -6
@@ -1,6 +0,0 @@ -#ifndef _ASM_PARISC_CPUMASK_H -#define _ASM_PARISC_CPUMASK_H - -#include <asm-generic/cpumask.h> - -#endif /* _ASM_PARISC_CPUMASK_H */
include/asm-ppc/cpumask.h +0 -6
@@ -1,6 +0,0 @@ -#ifndef _ASM_PPC_CPUMASK_H -#define _ASM_PPC_CPUMASK_H - -#include <asm-generic/cpumask.h> - -#endif /* _ASM_PPC_CPUMASK_H */
include/asm-ppc64/cpumask.h +0 -6
@@ -1,6 +0,0 @@ -#ifndef _ASM_PPC64_CPUMASK_H -#define _ASM_PPC64_CPUMASK_H - -#include <asm-generic/cpumask.h> - -#endif /* _ASM_PPC64_CPUMASK_H */
include/asm-s390/cpumask.h +0 -6
@@ -1,6 +0,0 @@ -#ifndef _ASM_S390_CPUMASK_H -#define _ASM_S390_CPUMASK_H - -#include <asm-generic/cpumask.h> - -#endif /* _ASM_S390_CPUMASK_H */
include/asm-sh/cpumask.h +0 -6
@@ -1,6 +0,0 @@ -#ifndef _ASM_SH_CPUMASK_H -#define _ASM_SH_CPUMASK_H - -#include <asm-generic/cpumask.h> - -#endif /* _ASM_SH_CPUMASK_H */
include/asm-sparc/cpumask.h +0 -6
@@ -1,6 +0,0 @@ -#ifndef _ASM_SPARC_CPUMASK_H -#define _ASM_SPARC_CPUMASK_H - -#include <asm-generic/cpumask.h> - -#endif /* _ASM_SPARC_CPUMASK_H */
include/asm-sparc64/cpumask.h +0 -6
@@ -1,6 +0,0 @@ -#ifndef _ASM_SPARC64_CPUMASK_H -#define _ASM_SPARC64_CPUMASK_H - -#include <asm-generic/cpumask.h> - -#endif /* _ASM_SPARC64_CPUMASK_H */
include/asm-um/cpumask.h +0 -6
@@ -1,6 +0,0 @@ -#ifndef _ASM_UM_CPUMASK_H -#define _ASM_UM_CPUMASK_H - -#include <asm-generic/cpumask.h> - -#endif /* _ASM_UM_CPUMASK_H */
include/asm-v850/cpumask.h +0 -6
@@ -1,6 +0,0 @@ -#ifndef _ASM_V850_CPUMASK_H -#define _ASM_V850_CPUMASK_H - -#include <asm-generic/cpumask.h> - -#endif /* _ASM_V850_CPUMASK_H */
include/asm-x86_64/cpumask.h +0 -6
@@ -1,6 +0,0 @@ -#ifndef _ASM_X86_64_CPUMASK_H -#define _ASM_X86_64_CPUMASK_H - -#include <asm-generic/cpumask.h> - -#endif /* _ASM_X86_64_CPUMASK_H */
include/asm-ia64/cpumask.h +0 -6
@@ -1,6 +0,0 @@ -#ifndef _ASM_IA64_CPUMASK_H -#define _ASM_IA64_CPUMASK_H - -#include <asm-generic/cpumask.h> - -#endif /* _ASM_IA64_CPUMASK_H */
Signature · 3 lines
I won't rest till it's the best ...
Programmer, Linux Scalability
Paul Jackson <[email protected]> 1.650.933.1373PATCH 12/15 mask6-cpumask-i386-fixup
In reply to Paul Jacksonmask6-cpumask-i386-fixup
Remove by recoding i386 uses of the obsolete cpumask const, coerce and promote macros.
arch/i386/kernel/io_apic.c +1 -1
@@ -264,7 +264,7 @@struct irq_pin_list *entry = irq_2_pin + irq;unsigned int apicid_value; - apicid_value =cpu_mask_to_apicid (mk_cpumask_const (cpumask)); + apicid_value =cpu_mask_to_apicid (cpumask);/* Prepare to do the io_apic_write */ apicid_value = apicid_value <<24 ;spin_lock_irqsave (&ioapic_lock, flags);
arch/i386/kernel/smp.c +1 -1
@@ -159,7 +159,7 @@*/ inline void send_IPI_mask_bitmask (cpumask_t cpumask,int vector) { -unsigned long mask =cpus_coerce (cpumask); +unsigned long mask =cpus_addr (cpumask)[0 ];unsigned long cfg;unsigned long flags;
arch/i386/mach-voyager/voyager_smp.c +15 -15
@@ -153,7 +153,7 @@ send_CPI_allbutself(__u8 cpi) {__u8 cpu =smp_processor_id (); -__u32 mask =cpus_coerce (cpu_online_map) & ~(1 << cpu); +__u32 mask =cpus_addr (cpu_online_map)[0 ] & ~(1 << cpu);send_CPI (mask, cpi); }
@@ -402,11 +402,11 @@/* set up everything for just this CPU, we can alter * this as we start the other CPUs later */ /* now get the CPU disposition from the extended CMOS */ - phys_cpu_present_map =cpus_promote (voyager_extended_cmos_read (VOYAGER_PROCESSOR_PRESENT_MASK )); -cpus_coerce (phys_cpu_present_map) |=voyager_extended_cmos_read (VOYAGER_PROCESSOR_PRESENT_MASK +1 ) <<8 ; -cpus_coerce (phys_cpu_present_map) |=voyager_extended_cmos_read (VOYAGER_PROCESSOR_PRESENT_MASK +2 ) <<16 ; -cpus_coerce (phys_cpu_present_map) |=voyager_extended_cmos_read (VOYAGER_PROCESSOR_PRESENT_MASK +3 ) <<24 ; -printk ("VOYAGER SMP: phys_cpu_present_map = 0x%lx\n" ,cpus_coerce (phys_cpu_present_map)); +cpus_addr (phys_cpu_present_map)[0 ] =voyager_extended_cmos_read (VOYAGER_PROCESSOR_PRESENT_MASK ); +cpus_addr (phys_cpu_present_map)[0 ] |=voyager_extended_cmos_read (VOYAGER_PROCESSOR_PRESENT_MASK +1 ) <<8 ; +cpus_addr (phys_cpu_present_map)[0 ] |=voyager_extended_cmos_read (VOYAGER_PROCESSOR_PRESENT_MASK +2 ) <<16 ; +cpus_addr (phys_cpu_present_map)[0 ] |=voyager_extended_cmos_read (VOYAGER_PROCESSOR_PRESENT_MASK +3 ) <<24 ; +printk ("VOYAGER SMP: phys_cpu_present_map = 0x%lx\n" ,cpus_addr (phys_cpu_present_map)[0 ]);/* Here we set up the VIC to enable SMP */ /* enable the CPIs by writing the base vector to their register */ outb (VIC_DEFAULT_CPI_BASE ,VIC_CPI_BASE_REGISTER );
@@ -706,12 +706,12 @@/* now that the cat has probed the Voyager System Bus, sanity * check the cpu map */ if ( ((voyager_quad_processors | voyager_extended_vic_processors) - &cpus_coerce (phys_cpu_present_map)) !=cpus_coerce (phys_cpu_present_map)) { + &cpus_addr (phys_cpu_present_map)[0 ]) !=cpus_addr (phys_cpu_present_map)[0 ]) {/* should panic */ printk ("\n\n***WARNING*** Sanity check of CPU present map FAILED\n" ); } }else if (voyager_level ==4 ) - voyager_extended_vic_processors =cpus_coerce (phys_cpu_present_map); + voyager_extended_vic_processors =cpus_addr (phys_cpu_present_map)[0 ];/* this sets up the idle task to run on the current cpu */ voyager_extended_cpus =1 ;
@@ -909,7 +909,7 @@if (!cpumask)BUG (); -if ((cpumask &cpus_coerce (cpu_online_map)) != cpumask) +if ((cpumask &cpus_addr (cpu_online_map)[0 ]) != cpumask)BUG ();if (cpumask & (1 <<smp_processor_id ()))BUG ();
@@ -952,7 +952,7 @@preempt_disable (); - cpu_mask =cpus_coerce (mm->cpu_vm_mask) & ~(1 <<smp_processor_id ()); + cpu_mask =cpus_addr (mm->cpu_vm_mask)[0 ] & ~(1 <<smp_processor_id ());local_flush_tlb ();if (cpu_mask)flush_tlb_others (cpu_mask, mm,FLUSH_ALL );
@@ -968,7 +968,7 @@preempt_disable (); - cpu_mask =cpus_coerce (mm->cpu_vm_mask) & ~(1 <<smp_processor_id ()); + cpu_mask =cpus_addr (mm->cpu_vm_mask)[0 ] & ~(1 <<smp_processor_id ());if (current->active_mm == mm) {if (current->mm)
@@ -989,7 +989,7 @@preempt_disable (); - cpu_mask =cpus_coerce (mm->cpu_vm_mask) & ~(1 <<smp_processor_id ()); + cpu_mask =cpus_addr (mm->cpu_vm_mask)[0 ] & ~(1 <<smp_processor_id ());if (current->active_mm == mm) {if (current->mm)__flush_tlb_one (va);
@@ -1098,7 +1098,7 @@ int wait) {struct call_data_struct data; -__u32 mask =cpus_coerce (cpu_online_map); +__u32 mask =cpus_addr (cpu_online_map)[0 ]; mask &= ~(1 <<smp_processor_id ());
@@ -1789,9 +1789,9 @@unsigned long irq_mask =1 << irq;int cpu; - real_mask =cpus_coerce (mask) & voyager_extended_vic_processors; + real_mask =cpus_addr (mask)[0 ] & voyager_extended_vic_processors; -if (cpus_coerce (mask) ==0 ) +if (cpus_addr (mask)[0 ] ==0 )/* can't have no cpu's to accept the interrupt -- extremely * bad things will happen */ return ;
include/asm-i386/genapic.h +1 -1
@@ -63,7 +63,7 @@unsigned (*get_apic_id)(unsigned long x);unsigned long apic_id_mask; -unsigned int (*cpu_mask_to_apicid)(cpumask_const_t cpumask); +unsigned int (*cpu_mask_to_apicid)(cpumask_t cpumask);/* ipi */ void (*send_IPI_mask)(cpumask_t mask,int vector);
include/asm-i386/mach-bigsmp/mach_apic.h +4 -4
@@ -140,14 +140,14 @@return (1 ); } -static inline unsigned int cpu_mask_to_apicid (cpumask_const_t cpumask) +static inline unsigned int cpu_mask_to_apicid (cpumask_t cpumask) {int num_bits_set;int cpus_found =0 ;int cpu;int apicid; - num_bits_set =cpus_weight_const (cpumask); + num_bits_set =cpus_weight (cpumask);/* Return id to all */ if (num_bits_set ==NR_CPUS )return (int )0xFF ;
@@ -155,10 +155,10 @@* The cpus in the mask must all be on the apic cluster. If are not * on the same apicid cluster return default value of TARGET_CPUS. */ - cpu =first_cpu_const (cpumask); + cpu =first_cpu (cpumask); apicid =cpu_to_logical_apicid (cpu);while (cpus_found < num_bits_set) { -if (cpu_isset_const (cpu, cpumask)) { +if (cpu_isset (cpu, cpumask)) {int new_apicid =cpu_to_logical_apicid (cpu);if (apicid_cluster (apicid) !=apicid_cluster (new_apicid)){
include/asm-i386/mach-default/mach_apic.h +5 -5
@@ -5,12 +5,12 @@#define APIC_DFR_VALUE (APIC_DFR_FLAT) -static inline cpumask_const_t target_cpus (void ) +static inline cpumask_t target_cpus (void ) {#ifdef CONFIG_SMP -return mk_cpumask_const (cpu_online_map); +return cpu_online_map;#else -return mk_cpumask_const (cpumask_of_cpu (0 )); +return cpumask_of_cpu (0 );#endif }#define TARGET_CPUS (target_cpus())
@@ -118,9 +118,9 @@return physid_isset (GET_APIC_ID (apic_read (APIC_ID )), phys_cpu_present_map); } -static inline unsigned int cpu_mask_to_apicid (cpumask_const_t cpumask) +static inline unsigned int cpu_mask_to_apicid (cpumask_t cpumask) { -return cpus_coerce_const (cpumask); +return cpus_addr (cpumask)[0 ]; }static inline void enable_apic_mode (void )
include/asm-i386/mach-es7000/mach_apic.h +5 -5
@@ -89,7 +89,7 @@int apic = bios_cpu_apicid[smp_processor_id ()];printk ("Enabling APIC mode: %s. Using %d I/O APICs, target cpus %lx\n" , (apic_version[apic] ==0x14 ) ? -"Physical Cluster" :"Logical Cluster" , nr_ioapics,cpus_coerce (TARGET_CPUS )); +"Physical Cluster" :"Logical Cluster" , nr_ioapics,cpus_addr (TARGET_CPUS )[0 ]); }static inline int multi_timer_check (int apic,int irq)
@@ -159,14 +159,14 @@return (1 ); } -static inline unsigned int cpu_mask_to_apicid (cpumask_const_t cpumask) +static inline unsigned int cpu_mask_to_apicid (cpumask_t cpumask) {int num_bits_set;int cpus_found =0 ;int cpu;int apicid; - num_bits_set =cpus_weight_const (cpumask); + num_bits_set =cpus_weight (cpumask);/* Return id to all */ if (num_bits_set == NR_CPUS)#if definedCONFIG_ES7000_CLUSTERED_APIC
@@ -178,10 +178,10 @@* The cpus in the mask must all be on the apic cluster. If are not * on the same apicid cluster return default value of TARGET_CPUS. */ - cpu =first_cpu_const (cpumask); + cpu =first_cpu (cpumask); apicid =cpu_to_logical_apicid (cpu);while (cpus_found < num_bits_set) { -if (cpu_isset_const (cpu, cpumask)) { +if (cpu_isset (cpu, cpumask)) {int new_apicid =cpu_to_logical_apicid (cpu);if (apicid_cluster (apicid) !=apicid_cluster (new_apicid)){
include/asm-i386/mach-numaq/mach_apic.h +1 -1
@@ -136,7 +136,7 @@* We use physical apicids here, not logical, so just return the default * physical broadcast to stop people from breaking us */ -static inline unsigned int cpu_mask_to_apicid (cpumask_const_t cpumask) +static inline unsigned int cpu_mask_to_apicid (cpumask_t cpumask) {return (int )0xF ; }
include/asm-i386/mach-summit/mach_apic.h +4 -4
@@ -140,14 +140,14 @@ { } -static inline unsigned int cpu_mask_to_apicid (cpumask_const_t cpumask) +static inline unsigned int cpu_mask_to_apicid (cpumask_t cpumask) {int num_bits_set;int cpus_found =0 ;int cpu;int apicid; - num_bits_set =cpus_weight_const (cpumask); + num_bits_set =cpus_weight (cpumask);/* Return id to all */ if (num_bits_set ==NR_CPUS )return (int )0xFF ;
@@ -155,10 +155,10 @@* The cpus in the mask must all be on the apic cluster. If are not * on the same apicid cluster return default value of TARGET_CPUS. */ - cpu =first_cpu_const (cpumask); + cpu =first_cpu (cpumask); apicid =cpu_to_logical_apicid (cpu);while (cpus_found < num_bits_set) { -if (cpu_isset_const (cpu, cpumask)) { +if (cpu_isset (cpu, cpumask)) {int new_apicid =cpu_to_logical_apicid (cpu);if (apicid_cluster (apicid) !=apicid_cluster (new_apicid)){
include/asm-i386/mach-visws/mach_apic.h +2 -2
@@ -84,9 +84,9 @@return physid_isset (boot_cpu_physical_apicid, phys_cpu_present_map); } -static inline unsigned int cpu_mask_to_apicid (cpumask_const_t cpumask) +static inline unsigned int cpu_mask_to_apicid (cpumask_t cpumask) { -return cpus_coerce_const (cpumask); +return cpus_addr (cpumask)[0 ]; }static inline u32 phys_pkg_id (u32 cpuid_apic,int index_msb)
Signature · 3 lines
I won't rest till it's the best ...
Programmer, Linux Scalability
Paul Jackson <[email protected]> 1.650.933.1373PATCH 13/15 mask7-cpumask-etc-fixup
In reply to Paul Jacksonmask7-cpumask-etc-fixup
Remove by recoding other uses of the obsolete cpumask const, coerce and promote macros.
arch/ppc64/kernel/open_pic.c +4 -4
@@ -591,7 +591,7 @@void openpic_init_processor (u_int cpumask) {openpic_write (&OpenPIC->Global.Processor_Initialization, -physmask (cpumask &cpus_coerce (cpu_online_map))); +physmask (cpumask &cpus_addr (cpu_online_map)[0 ])); }#ifdef CONFIG_SMP
@@ -625,7 +625,7 @@CHECK_THIS_CPU ;check_arg_ipi (ipi);openpic_write (&OpenPIC->THIS_CPU.IPI_Dispatch (ipi), -physmask (cpumask &cpus_coerce (cpu_online_map))); +physmask (cpumask &cpus_addr (cpu_online_map)[0 ])); }void openpic_request_IPIs (void )
@@ -711,7 +711,7 @@ {check_arg_timer (timer);openpic_write (&OpenPIC->Global.Timer[timer].Destination, -physmask (cpumask &cpus_coerce (cpu_online_map))); +physmask (cpumask &cpus_addr (cpu_online_map)[0 ])); }
@@ -844,7 +844,7 @@cpumask_t tmp;cpus_and (tmp, cpumask, cpu_online_map); -openpic_mapirq (irq_nr - open_pic_irq_offset,physmask (cpus_coerce (tmp))); +openpic_mapirq (irq_nr - open_pic_irq_offset,physmask (cpus_addr (tmp)[0 ])); }#ifdef CONFIG_SMP
arch/ppc64/kernel/rtasd.c +3 -3
@@ -415,7 +415,7 @@ }lock_cpu_hotplug (); - cpu =first_cpu_const (mk_cpumask_const (cpu_online_map)); + cpu =first_cpu (cpu_online_map);for (;;) {set_cpus_allowed (current,cpumask_of_cpu (cpu));do_event_scan (event_scan);
@@ -429,9 +429,9 @@schedule_timeout ((HZ *60 /rtas_event_scan_rate) /2 );lock_cpu_hotplug (); - cpu =next_cpu_const (cpu,mk_cpumask_const (cpu_online_map)); + cpu =next_cpu (cpu, cpu_online_map);if (cpu ==NR_CPUS ) - cpu =first_cpu_const (mk_cpumask_const (cpu_online_map)); + cpu =first_cpu (cpu_online_map); } error:
arch/x86_64/kernel/io_apic.c +1 -1
@@ -1383,7 +1383,7 @@unsigned long flags;unsigned int dest; - dest =cpu_mask_to_apicid (mk_cpumask_const (mask)); + dest =cpu_mask_to_apicid (mask);/* * Only the first 8 bits are valid.
arch/x86_64/kernel/pci-gart.c +2 -2
@@ -149,7 +149,7 @@ {unsigned long flags;int bus = dev ? dev->bus->number :-1 ; -cpumask_const_t bus_cpumask =pcibus_to_cpumask (bus); +cpumask_t bus_cpumask =pcibus_to_cpumask (bus);int flushed =0 ;int i;
@@ -159,7 +159,7 @@u32 w;if (!northbridges[i])continue ; -if (bus >=0 && !(cpu_isset_const (i, bus_cpumask))) +if (bus >=0 && !(cpu_isset (i, bus_cpumask)))continue ;pci_write_config_dword (northbridges[i],0x9c , northbridge_flush_word[i] |1 );
arch/x86_64/kernel/smp.c +1 -1
@@ -94,7 +94,7 @@static inline void send_IPI_mask (cpumask_t cpumask,int vector) { -unsigned long mask =cpus_coerce (cpumask); +unsigned long mask =cpus_addr (cpumask)[0 ];unsigned long cfg;unsigned long flags;
include/asm-x86_64/smp.h +2 -3
@@ -105,7 +105,6 @@ return BAD_APICID; } -#define cpu_online (cpu) cpu_isset(cpu, cpu_online_map)#endif /* !ASSEMBLY */ #define NO_PROC_ID 0xFF/* No processor magic marker */
@@ -115,9 +114,9 @@#define TARGET_CPUS 1#ifndef ASSEMBLY -static inline unsigned int cpu_mask_to_apicid (cpumask_const_t cpumask) +static inline unsigned int cpu_mask_to_apicid (cpumask_t cpumask) { -return cpus_coerce_const (cpumask); +return cpus_addr (cpumask)[0 ]; }#endif
Signature · 3 lines
I won't rest till it's the best ...
Programmer, Linux Scalability
Paul Jackson <[email protected]> 1.650.933.1373PATCH 14/15 mask8-rm-old-cpumask-emul
In reply to Paul Jacksonmask8-rm-old-cpumask-emul
Now that the emulation of the obsolete cpumask macros is no longer needed, remove it from cpumask.h
include/linux/cpumask.h +0 -12
@@ -320,16 +320,4 @@#define cpu_present (cpu) cpu_isset(cpu, cpu_present_map)#define for_each_present_cpu (cpu) for_each_cpu_mask(cpu, cpu_present_map) -/* Begin obsolete cpumask operator emulation */ -#define cpu_isset_const (a,b) cpu_isset(a,b) -#define cpumask_const_t cpumask_t -#define cpus_coerce (m) (cpus_addr(m)[0]) -#define cpus_coerce_const cpus_coerce -#define cpus_promote (x) ({ cpumask_t m; m.bits[0] = x; m; }) -#define cpus_weight_const cpus_weight -#define first_cpu_const first_cpu -#define mk_cpumask_const (x) x -#define next_cpu_const next_cpu -/* End of obsolete cpumask operator emulation */ -#endif /* __LINUX_CPUMASK_H */
Signature · 3 lines
I won't rest till it's the best ...
Programmer, Linux Scalability
Paul Jackson <[email protected]> 1.650.933.1373PATCH 15/15 mask9-post-cleanup-tweaks
In reply to Paul Jacksonmask9-post-cleanup-tweaks
Make use of for_each_cpu_mask() macro to simplify and optimize a couple of sparc64 per-CPU loops.
Optimize a bit of cpumask code for asm-i386/mach-es7000
Convert physids_complement() to use both args in the files include/asm-i386/mpspec.h, include/asm-x86_64/mpspec.h.
Remove cpumask hack from asm-x86_64/topology.h routine pcibus_to_cpumask().
Clarify and slightly optimize set_cpus_allowed() cpumask check in kernel/sched.c
kernel/sched.c +1 -1
@@ -3497,7 +3497,7 @@runqueue_t *rq; rq =task_rq_lock (p, &flags); -if (any_online_cpu (new_mask) ==NR_CPUS ) { +if (!cpus_intersects (new_mask, cpu_online_map)) { ret = -EINVAL ; goto out; }
arch/sparc64/kernel/smp.c +24 -42
92 lines not on this page · raw message
include/asm-i386/mach-es7000/mach_ipi.h +2 -3
@@ -10,9 +10,8 @@static inline void send_IPI_allbutself (int vector) { -cpumask_t mask =cpumask_of_cpu (smp_processor_id ()); -cpus_complement (mask); -cpus_and (mask, mask, cpu_online_map); +cpumask_t mask = cpu_online_map; +cpu_clear (smp_processor_id (), mask);if (!cpus_empty (mask))send_IPI_mask (mask, vector); }
include/asm-i386/mpspec.h +1 -1
@@ -53,7 +53,7 @@#define physids_and (dst, src1, src2) bitmap_and((dst).mask, (src1).mask, (src2).mask, MAX_APICS)#define physids_or (dst, src1, src2) bitmap_or((dst).mask, (src1).mask, (src2).mask, MAX_APICS)#define physids_clear (map) bitmap_zero((map).mask, MAX_APICS) -#define physids_complement (map) bitmap_complement((map).mask, (map).mask, MAX_APICS) +#define physids_complement (dst, src) bitmap_complement((dst).mask,(src).mask, MAX_APICS)#define physids_empty (map) bitmap_empty((map).mask, MAX_APICS)#define physids_equal (map1, map2) bitmap_equal((map1).mask, (map2).mask, MAX_APICS)#define physids_weight (map) bitmap_weight((map).mask, MAX_APICS)
include/asm-x86_64/mpspec.h +1 -1
@@ -212,7 +212,7 @@#define physids_and (dst, src1, src2) bitmap_and((dst).mask, (src1).mask, (src2).mask, MAX_APICS)#define physids_or (dst, src1, src2) bitmap_or((dst).mask, (src1).mask, (src2).mask, MAX_APICS)#define physids_clear (map) bitmap_zero((map).mask, MAX_APICS) -#define physids_complement (map) bitmap_complement((map).mask, (map).mask, MAX_APICS) +#define physids_complement (dst, src) bitmap_complement((dst).mask, (src).mask, MAX_APICS)#define physids_empty (map) bitmap_empty((map).mask, MAX_APICS)#define physids_equal (map1, map2) bitmap_equal((map1).mask, (map2).mask, MAX_APICS)#define physids_weight (map) bitmap_weight((map).mask, MAX_APICS)
include/asm-x86_64/topology.h +4 -2
@@ -20,9 +20,11 @@#define node_to_first_cpu (node) (__ffs(node_to_cpumask[node]))#define node_to_cpumask (node) (node_to_cpumask[node]) -static inline unsigned long pcibus_to_cpumask (int bus) +static inline cpumask_t pcibus_to_cpumask (int bus) { -return mp_bus_to_cpumask[bus] & cpu_online_map; +cpumask_t tmp; +cpus_and (tmp, mp_bus_to_cpumask[bus], cpu_online_map); +return tmp; }#define NODE_BALANCE_RATE 30/* CHECKME */
Signature · 3 lines
I won't rest till it's the best ...
Programmer, Linux Scalability
Paul Jackson <[email protected]> 1.650.933.1373Andrew has requested by private email that I hold off for now, and try resubmitting this patchset after he has "unloaded a few hundred patches into 2.6.7." Something about too many confusions ...
I will do so, gladly.
Signature · 3 lines
I won't rest till it's the best ...
Programmer, Linux Scalability
Paul Jackson <[email protected]> 1.650.933.1373