Skip to contentSkip to conversation

consolidate/clean up spinlock.h files

RFC PATCH In review Ingo Molnar 6 participants 11 messages Jun 2, 2005 – Jun 3, 2005

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

RFC PATCH 1/1 consolidate/clean up spinlock.h files

the attached patch (written by me and also containing many suggestions of Arjan van de Ven) does a major cleanup of the spinlock code. It does the following things:

- consolidates and enhances the spinlock/rwlock debugging code

- simplifies the asm/spinlock.h files

- encapsulates the raw spinlock types and moves generic spinlock features (such as ->break_lock) into the generic code.

- cleans up the spinlock code hierarchy to get rid of spaghetti.

most notably there's now only a single variant of the debugging code, located in lib/spinlock_debug.c. (previously we had one SMP debugging variant per architecture, plus a separate generic one for UP builds)

also, i've enhanced the rwlock debugging facility, it will now track write-owners. There is new spinlock-owner/CPU-tracking on SMP builds too.

the arch-level include files now only contain the minimally necessary subset of the spinlock code - all the rest that can be generalized now lives in the generic headers:

 include/asm-i386/spinlock.h             |  116 +----
 include/asm-i386/spinlock_types.h       |   16
 include/asm-x86_64/spinlock.h           |  146 +------
 include/asm-x86_64/spinlock_types.h     |   16

i've also split up the various spinlock variants into separate files, making it easier to see which does what. The new layout is:

***********
* here's the role of the various spinlock/rwlock related include files:
*
* on SMP builds:
*
* asm/spinlock_type.h: contains the raw_spinlock_t/raw_rwlock_t and the
* UNLOCKED initializers
*
* asm/spinlock.h: contains the __raw_spin_*()/etc. lowlevel
* implementations, mostly inline assembly code
*
* (also included on UP-debug builds:)
*
* linux/spinlock_smp.h: contains the prototypes for the _spin_*() APIs. *
* linux/spinlock.h: builds the final spin_*() APIs.
*
* on UP builds:
*
* asm-generic/spinlock_type_up.h:
* contains the generic, simplified UP spinlock type.
* (which is an empty structure on non-debug builds)
*
* asm-generic/spinlock_up.h:

 *                        contains the __raw_spin_*()/etc. version of UP
 *                        builds. (which are NOPs on non-debug, non-preempt
 *                        builds)

*
* (included on UP-non-debug builds:)
*
* linux/spinlock_up.h: builds the _spin_*() APIs. *
* linux/spinlock.h: builds the final spin_*() APIs.
***********

i've converted x86 and x64 for the time being (and have build/boot tested all spinlock variants on them), but if there's rough agreement about the details i'll convert every SMP architecture. (all UP architectures should work fine already)

there's a source-code size increase for now, mostly due to the extra debugging functionality and due to extra comments. Once every SMP architecture is converted there should be a net minus of a couple of hundred lines of code.

Ingo

kernel/sched.c renamed from kernel/sched.c.orig +4 -0
@@ -1329,6 +1329,10 @@ static inline void finish_task_switch(ta
 	 *		Manfred Spraul <[email protected]>
 	 */
 	prev_task_flags = prev->flags;
+#ifdef CONFIG_DEBUG_SPINLOCK
+	/* this is a valid case when another task releases the spinlock */
+	rq->lock.owner_pid = current->pid;
+#endif
 	finish_arch_switch(rq, prev);
 	if (mm)
 		mmdrop(mm);
kernel/Makefile renamed from kernel/Makefile.orig +1 -0
@@ -12,6 +12,7 @@ obj-y     = sched.o fork.o exec_domain.o
 obj-$(CONFIG_FUTEX) += futex.o
 obj-$(CONFIG_GENERIC_ISA_DMA) += dma.o
 obj-$(CONFIG_SMP) += cpu.o spinlock.o
+obj-$(CONFIG_DEBUG_SPINLOCK) += spinlock.o
 obj-$(CONFIG_UID16) += uid16.o
 obj-$(CONFIG_MODULES) += module.o
 obj-$(CONFIG_KALLSYMS) += kallsyms.o
kernel/spinlock.c renamed from kernel/spinlock.c.orig +6 -3
@@ -3,7 +3,10 @@
  *
  * Author: Zwane Mwaikambo <[email protected]>
  *
- * Copyright (2004) Ingo Molnar
+ * Copyright (2004, 2005) Ingo Molnar
+ *
+ * This file contains the spinlock/rwlock implementations for the
+ * SMP and the DEBUG_SPINLOCK cases. (UP-nondebug inlines them)
  */
 
 #include <linux/config.h>
@@ -57,7 +60,7 @@ int __lockfunc _write_trylock(rwlock_t *
 }
 EXPORT_SYMBOL(_write_trylock);
 
-#ifndef CONFIG_PREEMPT
+#if !defined(CONFIG_PREEMPT) || !defined(CONFIG_SMP)
 
 void __lockfunc _read_lock(rwlock_t *lock)
 {
@@ -72,7 +75,7 @@ unsigned long __lockfunc _spin_lock_irqs
 
 	local_irq_save(flags);
 	preempt_disable();
-	_raw_spin_lock_flags(lock, flags);
+	_raw_spin_lock_flags(lock, &flags);
 	return flags;
 }
 EXPORT_SYMBOL(_spin_lock_irqsave);
include/asm-generic/spinlock_types_up.h renamed from include/asm-generic/spinlock_types_up.h.orig +29 -0
@@ -0,0 +1,29 @@
+#ifndef __ASM_GENERIC_SPINLOCK_TYPES_UP_H
+#define __ASM_GENERIC_SPINLOCK_TYPES_UP_H
+
+/*
+ * include/linux/spinlock_types_up.h - spinlock type definitions for UP:
+ *
+ * portions Copyright 2005, Red Hat, Inc., Ingo Molnar
+ * Released under the General Public License (GPL).
+ */
+
+typedef struct {
+#ifdef CONFIG_DEBUG_SPINLOCK
+	volatile unsigned int slock;
+#endif
+} raw_spinlock_t;
+
+#ifdef CONFIG_DEBUG_SPINLOCK
+# define __RAW_SPIN_LOCK_UNLOCKED { 1 }
+#else
+# define __RAW_SPIN_LOCK_UNLOCKED { }
+#endif
+
+typedef struct {
+	/* no debug version on UP */
+} raw_rwlock_t;
+
+#define __RAW_RWLOCK_UNLOCKED { }
+
+#endif
include/asm-generic/spinlock_up.h renamed from include/asm-generic/spinlock_up.h.orig +68 -0

68 lines not on this page · raw message

include/linux/jbd.h renamed from include/linux/jbd.h.orig +1 -0
@@ -29,6 +29,7 @@
 #include <linux/journal-head.h>
 #include <linux/stddef.h>
 #include <asm/semaphore.h>
+#include <linux/bit_spinlock.h>
 #endif
 
 #define journal_oom_retry 1
include/linux/spinlock_smp.h renamed from include/linux/spinlock_smp.h.orig +51 -0

51 lines not on this page · raw message

include/linux/spinlock.h renamed from include/linux/spinlock.h.orig +144 -491

704 lines not on this page · raw message

include/linux/bit_spinlock.h renamed from include/linux/bit_spinlock.h.orig +77 -0

77 lines not on this page · raw message

include/linux/spinlock_up.h renamed from include/linux/spinlock_up.h.orig +220 -0

220 lines not on this page · raw message

include/asm-i386/spinlock.h renamed from include/asm-i386/spinlock.h.orig +26 -90

201 lines not on this page · raw message

include/asm-i386/spinlock_types.h renamed from include/asm-i386/spinlock_types.h.orig +16 -0
@@ -0,0 +1,16 @@
+#ifndef __ASM_SPINLOCK_TYPES_H
+#define __ASM_SPINLOCK_TYPES_H
+
+typedef struct {
+	volatile unsigned int slock;
+} raw_spinlock_t;
+
+#define __RAW_SPIN_LOCK_UNLOCKED	{ 1 }
+
+typedef struct {
+	volatile unsigned int lock;
+} raw_rwlock_t;
+
+#define __RAW_RW_LOCK_UNLOCKED		{ RW_LOCK_BIAS }
+
+#endif
include/asm-x86_64/spinlock.h renamed from include/asm-x86_64/spinlock.h.orig +32 -114

215 lines not on this page · raw message

include/asm-x86_64/spinlock_types.h renamed from include/asm-x86_64/spinlock_types.h.orig +16 -0
@@ -0,0 +1,16 @@
+#ifndef __ASM_SPINLOCK_TYPES_H
+#define __ASM_SPINLOCK_TYPES_H
+
+typedef struct {
+	volatile unsigned int slock;
+} raw_spinlock_t;
+
+#define __RAW_SPIN_LOCK_UNLOCKED	{ 1 }
+
+typedef struct {
+	volatile unsigned int lock;
+} raw_rwlock_t;
+
+#define __RAW_RW_LOCK_UNLOCKED		{ RW_LOCK_BIAS }
+
+#endif
lib/Makefile renamed from lib/Makefile.orig +1 -0
@@ -15,6 +15,7 @@ CFLAGS_kobject.o += -DDEBUG
 CFLAGS_kobject_uevent.o += -DDEBUG
 endif
 
+obj-$(CONFIG_DEBUG_SPINLOCK) += spinlock_debug.o
 lib-$(CONFIG_RWSEM_GENERIC_SPINLOCK) += rwsem-spinlock.o
 lib-$(CONFIG_RWSEM_XCHGADD_ALGORITHM) += rwsem.o
 lib-$(CONFIG_GENERIC_FIND_NEXT_BIT) += find_next_bit.o
lib/spinlock_debug.c renamed from lib/spinlock_debug.c.orig +197 -0

197 lines not on this page · raw message

#1 lore raw

Hi Ingo,

lib/spinlock_debug.c

Quoted 6 lines · you
lib/spinlock_debug.c renamed from lib/spinlock_debug.c.orig +4 -0
+#define SPIN_BUG_ON(cond, lock, msg) \
+		if (unlikely(cond)) spin_bug(lock, __FILE__, __LINE__, msg)
+#define RWLOCK_BUG_ON(cond, lock, msg) \
+		if (unlikely(cond)) rwlock_bug(lock, __FILE__, __LINE__, msg)

I would suggest propagating the __FILE__ and __LINE__ from the CALLERS of those functions in lib/spinlock_debug.c into these macros, to make this info more useful. At the moment you just know, that the bug happend on some spinlock/rwlock, but not who caused this.

Regards

Ingo Oeser

#2 lore raw
Quoted 14 lines · Ingo Oeser

Hi Ingo,

lib/spinlock_debug.c

Quoted 6 lines · you
lib/spinlock_debug.c renamed from lib/spinlock_debug.c.orig +4 -0
+#define SPIN_BUG_ON(cond, lock, msg) \
+		if (unlikely(cond)) spin_bug(lock, __FILE__, __LINE__, msg)
+#define RWLOCK_BUG_ON(cond, lock, msg) \
+		if (unlikely(cond)) rwlock_bug(lock, __FILE__, __LINE__, msg)

I would suggest propagating the __FILE__ and __LINE__ from the CALLERS of those functions in lib/spinlock_debug.c into these macros, to make this info more useful. At the moment you just know, that the bug happend on some spinlock/rwlock, but not who caused this.

the real call site info comes from dump_stack(). Maybe i should remove the __FILE__,__LINE__ info altogether. (albeit a bit redundancy wont hurt) I dont think we want to pass in __FILE__,__LINE__ all the way from the main APIs.

Ingo

#3 lore raw

Hi Ingo,

Quoted 8 lines · Ingo Molnar
Quoted 3 lines · Ingo Oeser
Quoted 1 line · you

--- linux/lib/spinlock_debug.c.orig

I would suggest propagating the __FILE__ and __LINE__ from the CALLERS

the real call site info comes from dump_stack(). Maybe i should remove the __FILE__,__LINE__ info altogether. (albeit a bit redundancy wont hurt) I dont think we want to pass in __FILE__,__LINE__ all the way from the main APIs.

Then please remove the __FILE__ and __LINE__ altogether.

It looks confusing to me and enlarges code without providing any useful additional information.

Regards

Ingo Oeser

#4 lore raw
In reply to Ingo Molnar
Quoted 1 line · Ingo Molnar

the real call site info comes from dump_stack(). Maybe i should remove the __FILE__,__LINE__ info altogether. (albeit a bit redundancy wont hurt) I dont think we want to pass in __FILE__,__LINE__ all the way from the main APIs.

Couldn't you make it a macro and hide the complexity? Or is it just a performance issue due to register starvation?

Chris

#5 lore raw
In reply to Ingo Oeser
Quoted 14 lines · Ingo Oeser

Hi Ingo,

lib/spinlock_debug.c

Quoted 6 lines · you
lib/spinlock_debug.c renamed from lib/spinlock_debug.c.orig +4 -0
+#define SPIN_BUG_ON(cond, lock, msg) \
+		if (unlikely(cond)) spin_bug(lock, __FILE__, __LINE__, msg)
+#define RWLOCK_BUG_ON(cond, lock, msg) \
+		if (unlikely(cond)) rwlock_bug(lock, __FILE__, __LINE__, msg)

I would suggest propagating the __FILE__ and __LINE__ from the CALLERS of those functions in lib/spinlock_debug.c into these macros, to make this info more useful. At the moment you just know, that the bug happend on some spinlock/rwlock, but not who caused this.

That sounds like it has the potential to really bloat things up with respect to the kernel image size.

Zwane

#6 lore raw
In reply to Ingo Molnar
Quoted 2 lines

i've also split up the various spinlock variants into separate files, making it easier to see which does what. The new layout is:

Sigh. You meant more complicated not easier, right?
It is complicated enough now that you would really wish
for an object browser (and a box of aspirin) to read it. But anyways I guess it was inevitable.

I would rename __aquire/__release/etc. they are clearly name space pollution and have even a specific meaning on IA64 which this does not follow.

-Andi

#7 lore raw
In reply to Ingo Molnar

Hi,

Quoted 8 lines · Ingo Molnar

- consolidates and enhances the spinlock/rwlock debugging code

- simplifies the asm/spinlock.h files

- encapsulates the raw spinlock types and moves generic spinlock features (such as ->break_lock) into the generic code.

- cleans up the spinlock code hierarchy to get rid of spaghetti.

That nicely splits the headers into several separate files, but the problem is that all these new header files are only of limited value outside the spinlock code.
What I'd really to see is a split of definitions and implementation. That means the definitions would be available via <linux/spinlock_types.h> and could be used in other core headers and would pull in a lot less header files. Header dependencies got worse especially since preempt got included.
The patch below does the minimum to provide spinlock_types.h. We could also include initializers.

Quoted 2 lines

* linux/spinlock_smp.h: contains the prototypes for the _spin_*() APIs.
* linux/spinlock_up.h: builds the _spin_*() APIs.

These are rather small files. Is it really worth it to separate them?

Review

Quoted 2 lines
(quoted diff) +2 -0
+#define spin_lock_init(x)	do { *(x) = SPIN_LOCK_UNLOCKED; } while (0)
+#define rwlock_init(x)		do { *(x) = RW_LOCK_UNLOCKED; } while (0)

We should fix these initializers at some point. I usually see gcc produce code that creates first the structure on the stack and then does a memcpy.

bye, Roman

include/asm-generic/spinlock_types_up.h deleted +0 -29
Index: linux-2.6-mm/include/asm-generic/spinlock_types_up.h
===================================================================
@@ -1,29 +0,0 @@
-#ifndef __ASM_GENERIC_SPINLOCK_TYPES_UP_H
-#define __ASM_GENERIC_SPINLOCK_TYPES_UP_H
-
-/*
- * include/linux/spinlock_types_up.h - spinlock type definitions for UP:
- *
- * portions Copyright 2005, Red Hat, Inc., Ingo Molnar
- * Released under the General Public License (GPL).
- */
-
-typedef struct {
-#ifdef CONFIG_DEBUG_SPINLOCK
-	volatile unsigned int slock;
-#endif
-} raw_spinlock_t;
-
-#ifdef CONFIG_DEBUG_SPINLOCK
-# define __RAW_SPIN_LOCK_UNLOCKED { 1 }
-#else
-# define __RAW_SPIN_LOCK_UNLOCKED { }
-#endif
-
-typedef struct {
-	/* no debug version on UP */
-} raw_rwlock_t;
-
-#define __RAW_RWLOCK_UNLOCKED { }
-
-#endif
include/linux/spinlock.h +3 -36
Index: linux-2.6-mm/include/linux/spinlock.h
===================================================================
@@ -67,46 +67,13 @@
 #define __lockfunc fastcall __attribute__((section(".spinlock.text")))
 
 /*
- * Pull the raw_spinlock_t and raw_rwlock_t definitions:
- */
-#if defined(CONFIG_SMP)
-# include <asm/spinlock_types.h>
-#else
-# include <asm-generic/spinlock_types_up.h>
-#endif
-
-typedef struct {
-	raw_spinlock_t raw_lock;
-#if defined(CONFIG_PREEMPT) && defined(CONFIG_SMP)
-	unsigned int break_lock;
-#endif
-#ifdef CONFIG_DEBUG_SPINLOCK
-	unsigned int magic, owner_pid, owner_cpu;
-#endif
-} spinlock_t;
-
-#define SPINLOCK_MAGIC  0xdead4ead
-
-typedef struct {
-	raw_rwlock_t raw_lock;
-#if defined(CONFIG_PREEMPT) && defined(CONFIG_SMP)
-	unsigned int break_lock;
-#endif
-#ifdef CONFIG_DEBUG_SPINLOCK
-	unsigned int magic, owner_pid, owner_cpu;
-#endif
-} rwlock_t;
-
-#define RWLOCK_MAGIC	0xdeaf1eed
-
-/*
  * Pull the __raw*() functions/declarations (UP-nondebug doesnt need them):
  */
-#if defined(CONFIG_SMP)
+#ifdef CONFIG_SMP
 # include <asm/spinlock.h>
 #else
-# include <asm-generic/spinlock_up.h>
-#endif
+
+#endif /* CONFIG_SMP */
 
 #ifdef CONFIG_DEBUG_SPINLOCK
 # define SPIN_LOCK_UNLOCKED						\
include/linux/spinlock_types.h new file +59 -0

59 lines not on this page · raw message

#8 lore raw
Quoted 13 lines · Roman Zippel

Hi,

Quoted 8 lines · Ingo Molnar

- consolidates and enhances the spinlock/rwlock debugging code

- simplifies the asm/spinlock.h files

- encapsulates the raw spinlock types and moves generic spinlock features (such as ->break_lock) into the generic code.

- cleans up the spinlock code hierarchy to get rid of spaghetti.

Quoted 10 lines

That nicely splits the headers into several separate files, but the problem is that all these new header files are only of limited value outside the spinlock code.
What I'd really to see is a split of definitions and implementation. That means the definitions would be available via <linux/spinlock_types.h> and could be used in other core headers and would pull in a lot less header files. Header dependencies got worse especially since preempt got included.
The patch below does the minimum to provide spinlock_types.h. We could also include initializers.

yes, that's what i'm working towards - separating type from implementation on the arch level was the first step needed. I already had it at such a state yesterday (complete separation of type definitions, API definitions and asm implementation - it needed the initializers in the asm/spinlock_types.h file, but otherwise it was straightforward), but undid it in the last minute because sched.c and kernel_lock.c used some intermediate/raw primitives, leading to ugly dependencies. I'll re-try this angle today and repost the patch.

Ingo

#9 lore raw

Hi,

Quoted 8 lines · Ingo Molnar

yes, that's what i'm working towards - separating type from implementation on the arch level was the first step needed. I already had it at such a state yesterday (complete separation of type definitions, API definitions and asm implementation - it needed the initializers in the asm/spinlock_types.h file, but otherwise it was straightforward), but undid it in the last minute because sched.c and kernel_lock.c used some intermediate/raw primitives, leading to ugly dependencies. I'll re-try this angle today and repost the patch.

Some time ago I posted these patches: http://www.xs4all.nl/~zippel/task_patches/
They basically only move the type definitions into separate header files, but would basically allow a much better cleanup of e.g. the spinlock header file. Right now it's an ifdef jungle with lots of duplicated code. In the end I'd like to see a single set of spinlock functions, which are either inlined or instantiated in kernel/spinlock.c. But for the macros to become inline functions, we need to cleanup the header dependencies, so that we get: spinlock implementation -> preempt/irq implementation -> task/thread definitions -> spinlock definitions.
In this context I'm a little concerned whether your up/smp separation really works out. A proper cleanup needs changes outside the spinlock code and not just splitting the existing header into smaller headers.

bye, Roman

#10 lore raw
In reply to Ingo Molnar

Hello Ingo,

Quoted 25 lines · Ingo Molnar

the attached patch (written by me and also containing many suggestions of Arjan van de Ven) does a major cleanup of the spinlock code. It does the following things:

- consolidates and enhances the spinlock/rwlock debugging code

- simplifies the asm/spinlock.h files

- encapsulates the raw spinlock types and moves generic spinlock features (such as ->break_lock) into the generic code.

- cleans up the spinlock code hierarchy to get rid of spaghetti.

most notably there's now only a single variant of the debugging code, located in lib/spinlock_debug.c. (previously we had one SMP debugging variant per architecture, plus a separate generic one for UP builds)

also, i've enhanced the rwlock debugging facility, it will now track write-owners. There is new spinlock-owner/CPU-tracking on SMP builds too.

the arch-level include files now only contain the minimally necessary subset of the spinlock code - all the rest that can be generalized now lives in the generic headers:

Quoted 7 lines
 include/asm-i386/spinlock.h             |  116 +----
 include/asm-i386/spinlock_types.h       |   16
 include/asm-x86_64/spinlock.h           |  146 +------
 include/asm-x86_64/spinlock_types.h     |   16

i've also split up the various spinlock variants into separate files, making it easier to see which does what. The new layout is:

I like the cleanup/consolidation of the debug variants,
that was pretty much overdue. My only qualm is in the increase of header files (some of them could have remained 1 file), unless of course this new structure makes work on a followup/dependent patch (is there one?) easier.

Thanks,
Zwane