The -rt git tree

The -rt branch has been static for some time. We are sorry about that, but most of those that maintain it have been working on other parts of the kernel. Mostly new mainline work.

People have also been asking about having an -rt git tree. Well, it is time to create one.

We have started working on an -rt git tree. Note, it is currently broken and not ready for use. An announcement will be made when it is ready. This email is only to let people know that we are currently actively working on the new -rt tree.

The git tree is located here:

git://git.kernel.org/pub/scm/linux/kernel/git/rostedt/linux-2.6-rt.git

The branches include:

linus - the mainline branch that the rest is based on rt-master - all the rt/* branches merged into one master - rt-master + converting of the locks (see below) rt/rt - some rt git scripts
rt/convert-scripts - the scripts to convert spinlocks (see below) rt/mainline - clean ups for mainline that we need to push forward rt/not-for-mainline - rt only patches that we never want to push
to mainline
rt/fs - filesystem changes for rt
rt/ftrace - ftrace updates for rt
rt/java - IBM patches for java
rt/kmap
rt/lockdep
rt/pagecache
rt/percpu - this may turn into rt/mm
rt/rt-locks - the new lock API (see below)
rt/sched
rt/tasklet
rt/threadirqs - full soft and hard irq threading rt/timer
rt/workqueue - priority queues

We may add more rt/* branches. Most of those are self explanatory.

The new lock API. The old -rt patch used macro magic to convert spin_locks into mutexes depending on which type they were. A spinlock that was declared as spinlock_t would convert into a mutex when PREEMPT_RT was configured, where as a spinlock that was declared as raw_spinlock_t would stay as a spinlock.

For example:

       spinlock_t      what_am_i;
       raw_spinlock_t  spinner;

       spin_lock(&what_am_i);
       spin_lock(&spinner);

The 'what_am_i' would turn into a mutex when PREEMPT_RT was configured and would be a spinlock when it was not. 'spinner' would always stay as a spinlock. The way this was done was with a builtin gcc option to compare the type used to determine what function to call:

#define spin_lock(lock) \
       if (__builtin_types_compatible(typeof(spinlock_t), lock)) \
               rt_lock(lock); \
       else if (__builtin_types_compatible(typeof(raw_spinlock_t), lock)) \
               raw_spin_lock(lock); \
       else \
               __bad_func_type();

If you are looking at this and scratching your head saying "this is not acceptable for mainline", we agree with you ;-) This is the old way of doing things, which made porting -rt to mainline much easier than converting all locks to a new API. But the time has come to do just that.

The new locks
-------------

The old way is not acceptable because a developer can not know when seeing a 'spin_lock(x)' and knowing if it will stay a spinlock or not on PREEMPT_RT. The developer would need to go search for x and see what type it is. Then they still can not know if someone might change the type of x. This is just ripe for bugs.

The new RT has added a new lock API that will be a mutex when PREEMPT_RT is configured and a spinlock when it is not. The type is called 'lock_t' and introduces a new API acquire_lock() and release_lock(). This also has all the irq variants that spin_locks have (acquire_lock_irqsave, acquire_lock_irq, etc).

       lock_t          lock;

       acquire_lock_irqsave(&lock, flags);

Although it has 'irqsave' when PREEMPT_RT is defined, it will not disable interrupts. This is fine, because with PREEMPT_RT, interrupts are threads, and we do not need to worry about being preempted by an interrupt and have that interrupt taking the same lock cause a deadlock. But now, the developer knows that this is a special lock, and that the lock is not a spinlock all the time.

Note, you can not do:

       spin_lock(&spinner);
       acquire_lock(&lock);

This would be the same as doing:

       spin_lock(&spinner);
       mutex_lock(&mutex);

Converting the new locks
------------------------

In the git tree, there is a couple of scripts in rt/convert-scripts branch that will convert all spinlocks into the lock_t. Then there are scripts to convert the necessary spinlocks back to spinlock. All rt/* branches should not need to define the type lock_t except for rt/rt-locks.

This git repo uses similar scripts that tip uses. The rt-master branch is a merge of all rt/* branches (except rt/rt). This may define PREEMPT_RT but no lock is made into 'lock_t' type yet.

The master branch is a copy of the rt-branch with the scripts to do the lock conversion run on the tree. In other words, the master branch will be the new -rt patch.

Note, the current state of this git tree is still very much broken. We are rewriting a lot of code to pull this forward to 2.6.28. Unfortunately, this means that we are skipping 2.6.27 altogether. I will be making another announcement when this is ready for prime time. Some of the branches have not been updated with the changes either.

-- Steve

Hi Steve.

That's a good news.

But after converting these spinlocks into lock_t, how will you synchronize with the mainline on each release? You will have a huge amount of conflicts at every merges...

Quoted 8 lines

Hi Steve.

That's a good news.

But after converting these spinlocks into lock_t, how will you synchronize with the mainline on each release? You will have a huge amount of conflicts at every merges...

This is why master is separate from rt-master. The rt-master will grow like tip/master does. But the master will be recreated each time. Basically, the creation of master is this:

git checkout master
git reset --hard rt-master ./scripts/convert-spinlocks
git commit -a -m 'Convert all spinlocks to lock_t' ./scripts/convert-locks
git commit -a -m 'Convert some locks back to spinlock_t'

Thus we do not need to worry about the conflicts caused by the lock conversion. That will be done automatically each time we create a new master.

Quoted 8 lines

Hi Steve.

That's a good news.

But after converting these spinlocks into lock_t, how will you synchronize with the mainline on each release? You will have a huge amount of conflicts at every merges...

This is why master is separate from rt-master. The rt-master will grow like tip/master does. But the master will be recreated each time. Basically, the creation of master is this:

git checkout master
git reset --hard rt-master ./scripts/convert-spinlocks
git commit -a -m 'Convert all spinlocks to lock_t' ./scripts/convert-locks
git commit -a -m 'Convert some locks back to spinlock_t'

Thus we do not need to worry about the conflicts caused by the lock conversion. That will be done automatically each time we create a new master.

-- Steve

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

Hello Steven,

It all sounds good!

But, I was wondering, why using the scripts to convert the spinlocks over and over, instead of pushing only the new lock API to mainline asap? That would save a lot of conversion complexity and sounds like migrating the RT patch to mainline much easier. It sounds to me like a major rename action, and as long as there is only a spinlock behind the new API, there is a very low risk in breaking things. Later on the rt-mutex can be integrated behind the new API. In the mean time everybody can get used to the new locking API. That would save complexity and improve stability of the rt-patchset, because it does not need to be figured out over and over if there are new locks in new code that need to be converted back to spinlocks... (In fact we only need to focus from there on new code that use the spinlock API, and if it is really required to use that.)

Compile-time fix.

Signed-off-by Sven-Thorsten Dietrich <[email protected]>

include/linux/mutex.h +0 -0

index 585feed..b74fdc2 100644

If you do want to help, look at any of the other branches. Some have not been updated yet, and they all should be able to compile and boot on their own. If they do not, then those need to be fixed.

Quoted 5 lines · Steven Rostedt

If you do want to help, look at any of the other branches. Some have not been updated yet, and they all should be able to compile and boot on their own. If they do not, then those need to be fixed.

A good place to start for some low-hanging fruit would be rt/workqueue. I ported the patches, but it doesn't yet boot (assuming Steve merged the work I had done...I havent looked recently). I think its a relatively trivial problem, but I have not had a chance to track it down yet.

Maybe it'll save you a half second...

Signed-off-by Sven-Thorsten Dietrich <[email protected]>

include/linux/lock.h +0 -0

index e0d10d6..785bf7d 100644