-cip-rt kernels for review/testing

Hi!

I played a bit with -rt kernels, and here are the results.

v4.19.50-rt22-cip3-rebase was reasonably easy to do, as -cip1 was empty. v4.4.179-rt181-cip34-rebase was more interesting.

Trees are at kernel.org:

https://git.kernel.org/pub/scm/linux/kernel/git/pavel/linux-cip.git/log/?h=v4.4.179-rt181-cip34-rebase
https://git.kernel.org/pub/scm/linux/kernel/git/pavel/linux-cip.git/log/?h=v4.19.50-rt22-cip3-rebase

Basically untested. Could I ask for reviews and testing?

I know that full preemption does not work on x86-32. I sent "PREEMPT_RT_FULL on x86-32 machine" mail to the lists

I don't know how to release useful non-rebasing trees. Is someone using those?

Best regards,
Pavel

DENX Software Engineering GmbH,      Managing Director: Wolfgang Denk
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 181 bytes
Desc: Digital signature
URL: <http://lists.cip-project.org/pipermail/cip-dev/attachments/20190623/bb4ed075/attachment.sig>

Thanks for picking this up.

Just like the regular -cip kernel, the -rt version's primary stable feed is merge-based. Rebasing only comes out of the fact that a second patch series is involved, but I doubt many folks use that feed as source. I don't, my colleagues neither (AFAIK).

Please sync with Daniel Wagner on the workflow options and also on 4.4-rt maintenance.

Jan

Siemens AG, Corporate Technology, CT RDA IOT SES-DE
Corporate Competence Center Embedded Linux

Hi Pavel,

Yep, the v4.4.179 brought some conflicts. Have you compared with my merge for v4.4-rt? Not sure if I understand your worklfow correctly. It looks like you used for the rebase branches the upstream -rt patchset and merged the cip branch. Is this correct?

What I did for the v4.4-cip-rt and v4.19-cip-rt is following

- initial setup (only once):

took the latest cip release and applied the current/matching -rt patcheset

- regulare update:

$ cd v4.4-rt
$ git tag -l 'v4\.4\.*' --sort=v:refname | tail
$ git merge v4.4.120      [fixup conflicts]

# do some testing (this is on Linutronix -rt test infrastrucure)
$ git push lxcvs -f --follow-tag HEAD:stable-maintenance-4.4.y-rt

$ srt commit
$ srt tag

$ cd v4.4-rt-rebase
$ git rebase -i v4.4.120  [fixup conflicts]
$ srt commit
$ srt tag

# check if the -rt-rebase branch is identical to v4.4-rt
$ git diff v4.4-rt

$ srt create v4.4.115-rt130 v4.4.120-rt135
$ srt sign v4.4.115-rt130 v4.4.120-rt135
$ srt upload v4.4.115-rt130 v4.4.120-rt135
$ srt push v4.4.115-rt130 v4.4.120-rt135

# XXX push missing tags

$ git push origin v4.4.115-rt131 v4.4.116-rt132 v4.4.118-rt133 
v4.4.119-rt134

$ srt announce v4.4.115-rt130 v4.4.120-rt135 > ../announce-rt
$ cat ../announce-rt | msmtp -t --

The srt tool is available here: https://github.com/igaw/stable-rt-tools
It a few lines around git to make the steps consist. It is based on Steven Rostedt workflow. Tom Zanussi (also -rt stable maintainer) and I am using the tool to maintain the -rt trees.

- git fetch pavel --tags
From git://git.kernel.org/pub/scm/linux/kernel/git/pavel/linux-cip
! [rejected] v4.4.176-cip31 -> v4.4.176-cip31
(would clobber existing tag)
! [rejected] v4.4.176-cip31-rebase ->
v4.4.176-cip31-rebase (would clobber existing tag)

Was v4.4.176-cip31 rebased?

- tag pattern used to be "v4.4.[0-9]+-cip[0-9]+-rt[0-9]+", it's now "v4.4.[0-9]+-rt[0-9]+-cip[0-9]+".

- in the v4.4-rt is still a problem with f01f17d3705b ("mm, vmstat: make quiet_vmstat lighter"). It is needed to boot on x86_64 but it breaks on NVIDIA's boards, e.g. Jetson. Unfortunately, I don't have a NVIDIA board in my lab to figure out what is missing. So I suspect there is a still a problem hidden in the current v4.4-rt tree.

I haven't run the v4.4 tree in 32bit so far. I can try if I get it booting in my lab.

If it helps, I am going to give a talk at Plubmers on how to maintain the -rt trees:

"""
Maintaining out of tree patches over the long term

The PREEMPT_RT patchset is the longest existing large patchset living outside the Linux kernel. Over the years, the realtime developers had to maintain several stable kernel versions of the patchset. This talk will present the lessons learned from this experience, including workflow, tooling and release management that has proven over time to scale. The workflow deals with upstream changes and changes to the patchset itself. Now that the PREEMPT_RT patchset is about the be merged upstream, we want to share our toolset and methods with others who may be able to benefit from our experience.

This talk is for people who want to maintain an external patchset with stable releases.
"""

Hope I could already answer a few question. In summary, have a look at stable-rt-tools which is the workflow implemented into a python script.

Thanks,
Daniel

Hi!

It looks like I'll need to redo the rebase at least for 4.4. I plan to rebase -cip specific patches on top of -rt.

Thanks for pointer. It does not want to run here (incompatible version of python 3?), so let me try to do one more manual attempt, and if that fails I'll try to fix/workaround python problems.

Thanks for pointer. I'm not sure I'll be at Plumbers.

Best regards,
Pavel

DENX Software Engineering GmbH,      Managing Director: Wolfgang Denk
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 181 bytes
Desc: Digital signature
URL: <http://lists.cip-project.org/pipermail/cip-dev/attachments/20190723/8e8b9e02/attachment.sig>

Hi Pavel,

stable-rt-tools might depend on a newer version of Python3. I am running Python 3.7 here. I can't remember if there are any dependencies on newer version of Python3. What's the error message?

Thanks,
Daniel

Hi Pavel,

I appreciate your effort.

How about adding these questions into our question list and ask Wagner afterward :-)

SZ

Hi!

4.19.50-cip3-rt1 should be available at kernel.org. I did limited testing, but it seems to have some bugs fixed relative to 4.19.13-cip1-rt1. (Boots on x86-32 and cyclictest on iwg20m does not hit WARN_ON).

Trees are available at

https://git.kernel.org/pub/scm/linux/kernel/git/cip/linux-cip.git/log/?h=linux-4.19.y-cip-rt-rebase

https://git.kernel.org/pub/scm/linux/kernel/git/cip/linux-cip.git/log/?h=linux-4.19.y-cip-rt

And their content should be identical.

Best regards,
Pavel

DENX Software Engineering GmbH,      Managing Director: Wolfgang Denk
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 181 bytes
Desc: Digital signature
URL: <http://lists.cip-project.org/pipermail/cip-dev/attachments/20190628/a63beb23/attachment.sig>

Shouldn't this be 4.19.50-cip3-rt2? It's the second -rt release for 4.19-cip.

Jan

Siemens AG, Corporate Technology, CT RDA IOT SES-DE
Corporate Competence Center Embedded Linux

Hi!

Its first realtime release based on 4.19.50-cip3 :-). It is also based on 4.19.50-rt22. So yes, the naming is a bit confusing.

If such version would previously be named -rt2, I'll obviously need to fix it.

Best regards,
Pavel

DENX Software Engineering GmbH,      Managing Director: Wolfgang Denk
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 181 bytes
Desc: Digital signature
URL: <http://lists.cip-project.org/pipermail/cip-dev/attachments/20190704/080465ba/attachment.sig>

Please keep the old naming scheme and increment the -rt number when releasing a new rt kernel.

Jan

Siemens AG, Corporate Technology, CT RDA IOT SES-DE
Corporate Competence Center Embedded Linux

Hi!

Oh and BTW testing would be very welcome.

I tried testing it on my thinkpad X60, but it is not suitable hardware; latency goes over 1msec quite easily. But I've seen 3msec with iwg20m board, too. I guess I should try cyclictest on 4.19.13-cip1-rt1 so I have comparison, but that is easier said than done.

Best regards,
Pavel

DENX Software Engineering GmbH,      Managing Director: Wolfgang Denk
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 181 bytes
Desc: Digital signature
URL: <http://lists.cip-project.org/pipermail/cip-dev/attachments/20190704/bbd3d4ee/attachment.sig>