Skip to contentSkip to conversation

ARM MSM updates for 2.6.35-rc1

GIT PULL Daniel Walker 8 participants 15 messages May 27, 2010 – Jun 4, 2010

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

Hi Linus,

Here's a couple of simple patches. One fixes a compile failure in certain situations, and the other is just dead code removal.

Daniel

The following changes since commit 7b52161d14fa8a22a2387f4aa2fb7b854587830d:

msm: 7x30 Kconfig and makefile changes (2010-05-13 16:08:55 -0700)

are available in the git repository at:
git://codeaurora.org/quic/kernel/dwalker/linux-msm.git msm-core

Daniel Walker (1):
msm: dma: add completion.h header

Huang Weiyi (1):
arm: msm: remove unused #include <linux/version.h>

 arch/arm/mach-msm/acpuclock-arm11.c |    1 -
 arch/arm/mach-msm/clock.c           |    1 -
 arch/arm/mach-msm/dma.c             |    1 +
 3 files changed, 1 insertions(+), 2 deletions(-)
arch/arm/mach-msm/acpuclock-arm11.c +0 -1
index af5e85b..6ca42fd 100644
@@ -17,7 +17,6 @@
  *
  */
 
-#include <linux/version.h>
 #include <linux/kernel.h>
 #include <linux/init.h>
 #include <linux/list.h>
arch/arm/mach-msm/clock.c +0 -1
index 9cb1276..c57210f 100644
@@ -14,7 +14,6 @@
  *
  */
 
-#include <linux/version.h>
 #include <linux/kernel.h>
 #include <linux/init.h>
 #include <linux/module.h>
arch/arm/mach-msm/dma.c +1 -0
index 3d725ae..3db0bfe 100644
@@ -17,6 +17,7 @@
 #include <linux/err.h>
 #include <linux/io.h>
 #include <linux/interrupt.h>
+#include <linux/completion.h>
 #include <mach/dma.h>
 
 #define MSM_DMOV_CHANNEL_COUNT 16
#1 lore raw
Quoted 8 lines · Daniel Walker

Hi Linus,

Here's a couple of simple patches. One fixes a compile failure in certain situations, and the other is just dead code removal.

Daniel

Quoted 6 lines

The following changes since commit 7b52161d14fa8a22a2387f4aa2fb7b854587830d:

msm: 7x30 Kconfig and makefile changes (2010-05-13 16:08:55 -0700)

are available in the git repository at:
git://codeaurora.org/quic/kernel/dwalker/linux-msm.git msm-core

Don't forget about this one! Or was something wrong with it?

Daniel

#2 lore raw
Quoted 2 lines · Daniel Walker

Don't forget about this one! Or was something wrong with it?

I got a bit frustrated with ten different ARM pulls per day at one point.

There's something wrong with ARM development. The amount of pure noise in the patches is incredibly annoying. Right now, ARM is already (despite me not reacting to some of the flood) 55% of all arch/ changes since 2.6.34, and it's all pointless churn in

arch/arm/configs/
arch/arm/mach-xyz
arch/arm/plat-blah

and at a certain point in the merge window I simply could not find it in me to care about it any more.

Do you guys at all talk about this problem? Have any of the ARM people bothered to look at the arch/arm diffs and see how mind-deadening they are? I try to look through these kinds of things when I pull, but after a million lines of pure noise, it gets old pretty quickly.

Somehow, I can't believe that you need thousands of lines for each random arch/arm/mach-xyz (yeah, some very few of them are smaller).

For a taste of the mind-deadening experience, just do

for i in arch/arm/mach-*
do
echo $i; wc -l $(git ls-files $i)
done | less -S

and imagine being on the receiving side of that.

Linus

#3 lore raw
Quoted 2 lines · Linus Torvalds

I got a bit frustrated with ten different ARM pulls per day at one point.

In other words - when I do a "git pull", I really want to feel like there is some point to it. When I get ten different ARM maintainers asking me to pull stuff that just looks like noise, I'm just not getting those warm and fuzzies about them.

I'm also getting the feeling that now that different arch sub-architecture maintainers all try to get their stuff in through me, things have actually gotten worse - there's even less feeling of "somebody is actually trying to keep track of all this stuff".

I understand why rmk wasn't happy. I'm also not really happy.

Linus

#4 lore raw
Quoted 10 lines · Linus Torvalds
Quoted 2 lines · Linus Torvalds

I got a bit frustrated with ten different ARM pulls per day at one point.

In other words - when I do a "git pull", I really want to feel like there is some point to it. When I get ten different ARM maintainers asking me to pull stuff that just looks like noise, I'm just not getting those warm and fuzzies about them.

Quoted 6 lines

I'm also getting the feeling that now that different arch sub-architecture maintainers all try to get their stuff in through me, things have actually gotten worse - there's even less feeling of "somebody is actually trying to keep track of all this stuff".

I understand why rmk wasn't happy. I'm also not really happy.

Should all us ARM sub-architecture maintainers get together and make a jumbo pull request with everything merged together?

Like someone, not you or RMK, would take all the sub-architecture requests and put them into a single pull request w/ lots of details as to what it all is ..

I know that doesn't solve what you said in the original email, how we develop. One thing to remember is that we have tons of ARM device to manage, and lots of chip manufactures .. So ARM is pretty diverse. I'm not saying we're all perfect however.

Daniel

#5 lore raw
In reply to Linus Torvalds
Quoted 16 lines · Linus Torvalds
Quoted 2 lines · Daniel Walker

Don't forget about this one! Or was something wrong with it?

I got a bit frustrated with ten different ARM pulls per day at one point.

There's something wrong with ARM development. The amount of pure noise in the patches is incredibly annoying. Right now, ARM is already (despite me not reacting to some of the flood) 55% of all arch/ changes since 2.6.34, and it's all pointless churn in

arch/arm/configs/
arch/arm/mach-xyz
arch/arm/plat-blah

Quoted 7 lines

and at a certain point in the merge window I simply could not find it in me to care about it any more.

Do you guys at all talk about this problem? Have any of the ARM people bothered to look at the arch/arm diffs and see how mind-deadening they are? I try to look through these kinds of things when I pull, but after a million lines of pure noise, it gets old pretty quickly.

There's room for the sub-architectures to combine stuff. I think there is work going on to do that (at some level).

Quoted 2 lines

Somehow, I can't believe that you need thousands of lines for each random arch/arm/mach-xyz (yeah, some very few of them are smaller).

I'm not an authority by any means, but every time there is a new device released (which is often) you get a new file under some mach-xyz directory, or some large modification to an already existing file. Plus a config file potentially. It's a little bit like the wild west, and seems way wilder than x86 every was.

I think from the sub-architecture maintainer perspective the pain is all hidden behind the Linus curtain , or RMK curtain or the some other maintainer curtain. So we don't really talk about it that often. I thought you we're totally happy with the situation ..

If you have some idea's for fixing things, I'm all ears along with my faithful sub-architecture maintainer brothers.

Daniel

#6 lore raw
In reply to Linus Torvalds
Quoted 18 lines · Linus Torvalds
Quoted 2 lines · Daniel Walker

Don't forget about this one! Or was something wrong with it?

I got a bit frustrated with ten different ARM pulls per day at one point.

There's something wrong with ARM development. The amount of pure noise in the patches is incredibly annoying. Right now, ARM is already (despite me not reacting to some of the flood) 55% of all arch/ changes since 2.6.34, and it's all pointless churn in

arch/arm/configs/
arch/arm/mach-xyz
arch/arm/plat-blah

and at a certain point in the merge window I simply could not find it in me to care about it any more.

Quoted 7 lines

Do you guys at all talk about this problem? Have any of the ARM people bothered to look at the arch/arm diffs and see how mind-deadening they are? I try to look through these kinds of things when I pull, but after a million lines of pure noise, it gets old pretty quickly.

Somehow, I can't believe that you need thousands of lines for each random arch/arm/mach-xyz (yeah, some very few of them are smaller).

As one of the sub-arch maintainers (arch/arm/mach-davinci), I can attempt to answer for some of the churn.

There indeed has been lots of change in mach-davinci, but I wouldn't consider it noise. In that one mach directory, I support ~10 SoCs in the same family and for each SoC there is at least one board supported. I'm also a core developer for mach-omap*, and the number of SoCs/boards supported there is equally large.

Each time we add support for a new SoC in the family some changes are needed to generalize existing code to work on the existing SoCs as well as the new ones. This generalizing is actually reducing the size of the diff compared to what it would be if it were added using copy-paste, but I understand why it might look like churn.

I certainly understand why this would be mind-numbing to anyone who doesn't care about davinci, or ARM-based devices. Indeed, even other ARM sub-arches developers don't need to care about most of what is in my mach dir and would consider it mind-numbing as well.

The fact is that ARM-based devices multiply like rabbits, and there is a huge amount of diversity between the various derivatives. Also, support most of these devices has lived out of tree for a long time. Now that we have a relatively direct path which doesn't require any single person to have to understand all the mind-numbing details of all of these ARM-based platforms, it has allowed us to significantly improve the support for these devices upstream. Anything that is common to all devices still goes through RMK.

Kevin

#7 lore raw
Quoted 6 lines · Kevin Hilman

There indeed has been lots of change in mach-davinci, but I wouldn't consider it noise. In that one mach directory, I support ~10 SoCs in the same family and for each SoC there is at least one board supported. I'm also a core developer for mach-omap*, and the number of SoCs/boards supported there is equally large.

The problem with this is that it's just not maintainable to keep on adding random stuff, especially since I doubt any of it ever gets aged away either.

arch/arm as-is is already about 800k lines. Compare that to arch/x86, which is less than a third of that. Now, what is the difference?

- x86 has its drivers elsewhere, and they are _discoverable_ and not hardcoded to some platform. They have often also been useful (to say the least) to other architecture platforms. That's not always true for all of them (we do have drivers/platform/x86, but at least that's maintained separately and is nowhere near the mess that is ARM)

- in contrast, ARM seems to be a mess. I realize it's largely because the hardware companies are so f*cked up, but guys, we need to have some handle on it too. I was willing to take the direct merges, and I still am, but I'm willing to do it only if I have a feeling that things are under control. And I'm not getting that feeling.

- TWO HUNDRED THOUSAND lines of arch/arm is just pure garbage, namely the defconfig files. Quite frankly, anybody who calls that anything but pure "noise" is just misguided and being stupid.

So yes, I do consider a lot of it "noise". When there's two hundred thousand lines of garbage, and it keeps growing without bounds, something needs to be done.

Now, I'm actually considering just getting rid of all the 'defconfig' files entirely. The x86 model is sane (there's two of them, nobody likely uses them), but ARM and POWERPC (and to a lesser config SH and MIPS) have turned the whole concept into a disgusting mess.

But while POWERPC has abused that thing too, in _other_ respects it has been much less egregious.

So I can largely fix the defconfig mess on my own (by just using a simple oneliner shell script to deletes them all) but that really only fixes one particularly annoying symptom - not the underlying issue. We do need somebody to maintain the arm platform mess, and actually tries to get some infrastructure or something in place so that it doesn't explode.

Quoted 8 lines

The fact is that ARM-based devices multiply like rabbits, and there is a huge amount of diversity between the various derivatives. Also, support most of these devices has lived out of tree for a long time. Now that we have a relatively direct path which doesn't require any single person to have to understand all the mind-numbing details of all of these ARM-based platforms, it has allowed us to significantly improve the support for these devices upstream. Anything that is common to all devices still goes through RMK.

The thing is, I bet there is way more commonality still to be taken advantage of. And even if there isn't, we need somebody who cares, and doesn't just mindlessly aggregate all the crud. Somebody with the taste to say "ok, that's just too effin ugly, you need to fix things up" occasionally.

Linus

#8 lore raw
Quoted 5 lines · Linus Torvalds

Now, I'm actually considering just getting rid of all the 'defconfig' files entirely. The x86 model is sane (there's two of them, nobody likely uses them), but ARM and POWERPC (and to a lesser config SH and MIPS) have turned the whole concept into a disgusting mess.

I agree that the defconfig mechanism is broken (ie. carrying the full text in tree), but the concept is sane IMHO.

What'd be nice is if the defconfig could just be a delta against a base config for the architecture - that would make most of them reasonably small, just turning on/off a few options.

You can sort of do that today, by just storing a delta, but oldconfig will silently turn off things you have enabled if prereqs change, so that doesn't really work I think.

cheers

#9 lore raw
Quoted 4 lines · Michael Ellerman

You can sort of do that today, by just storing a delta, but oldconfig will silently turn off things you have enabled if prereqs change, so that doesn't really work I think.

I think you can do it today with various hacks. Up to and including basically doing something that just selects the options you want.

IOW, you could likely have a human-written Kconfig.<platform> file that just does

define_bool MYPLATFORM y
select .. everything I need ..

include Kconfig.main

or a number of other tricks.

Ingo and the x86 folks (who I really think have done a very good job, and there really aren't any crazy defconfig files there) have this "make randconfig" together with scripted requirements so that you can actually _boot_ the random config, just because the requirements make sure that the things needed for booting on the test setup are selected.

I forget exactly what the build setup there is (Ingo described it to me long time ago, but since I don't even want to have a build farm in my home, I didn't care much).

But we certainly _can_ do a better job than the 'defconfig' thing that was really never meant for the kind of use it sees in ARM/POWERPC/SH/MIPS, and that really isn't appropriate for any manual editing (so people just run "make oldconfig" with tweaking or something, and then use the newly generated file).

And I suspect that it really is best to just remove the existing defconfig files. People can see them in the history to pick up what the heck they did, but no way will any sane model ever look even _remotely_ like them, so they really aren't a useful basis for going forward.

But don't worry. It didn't happen this merge window, obviously.

Linus

#10 lore raw
Quoted 14 lines · Linus Torvalds
Quoted 4 lines · Michael Ellerman

You can sort of do that today, by just storing a delta, but oldconfig will silently turn off things you have enabled if prereqs change, so that doesn't really work I think.

I think you can do it today with various hacks. Up to and including basically doing something that just selects the options you want.

IOW, you could likely have a human-written Kconfig.<platform> file that just does

Quoted 6 lines

define_bool MYPLATFORM y
select .. everything I need ..

include Kconfig.main

or a number of other tricks.

I agree all the defconfigs are a pain just for the omaps alone.

If this is of any help we could now just keep omap3_defconfig for arch/arm/mach-omap2 and get rid of 23 config files:

$ egrep "CONFIG_ARCH_OMAP[2|3|4]=y" arch/arm/configs/* | grep -v omap3_defconfig | wc -l
23

It needs some more work for omap2 though to boot to userspace as there are still some known issues with ARMv6 vs ARMv7 support and VFP2 vs 3 support. Will try to look at fixing those again when I have a chance.

Then making the multi-omap thing work on all omap1 boards would cut down another 15 defconfigs, that should be also doable.

To be able to compile in multiple arm architectures we would have to get rid of the Makefile.boot files and NR_IRQS and then have some kind of common clock framework at least.

I did some experiments compiling in both mach-omap1 and mach-omap2 a few years back using ARMv5 flags, there were probably other issues too like some conflicting defines.

Cheers,

Tony

#11 lore raw
In reply to Linus Torvalds
Quoted 31 lines · Linus Torvalds
Quoted 4 lines · Michael Ellerman

You can sort of do that today, by just storing a delta, but oldconfig will silently turn off things you have enabled if prereqs change, so that doesn't really work I think.

I think you can do it today with various hacks. Up to and including basically doing something that just selects the options you want.

IOW, you could likely have a human-written Kconfig.<platform> file that just does

define_bool MYPLATFORM y
select .. everything I need ..

include Kconfig.main

or a number of other tricks.

Ingo and the x86 folks (who I really think have done a very good job, and there really aren't any crazy defconfig files there) have this "make randconfig" together with scripted requirements so that you can actually _boot_ the random config, just because the requirements make sure that the things needed for booting on the test setup are selected.

I forget exactly what the build setup there is (Ingo described it to me long time ago, but since I don't even want to have a build farm in my home, I didn't care much).

Quoted 6 lines

But we certainly _can_ do a better job than the 'defconfig' thing that was really never meant for the kind of use it sees in ARM/POWERPC/SH/MIPS, and that really isn't appropriate for any manual editing (so people just run "make oldconfig" with tweaking or something, and then use the newly generated file).

It certainly looks a better way to handle this. However, considering the facts that there are so many platforms out there, and doing a transition without breaking any of them is a lot work, it's actually easier to just reduce the number of defconfig at this moment, provided that most ARM platforms with the same SoC are able to be built into a single kernel. There are some exceptions though, I'd suggest not to introduce any other defconfig for these platforms until their problem is solved.

Russell has setup a thread for this issue in linux-arm-kernel ML, so hopefully there will be a lot patches around to address it.

There are some specific problems with ARM, e.g. some platforms are really not maintained for a long time, and even no way to find someone or some machine to test. And even with one defconfig per SoC, there could still be about > 60 defconfigs there (compared with 178 at this moment).

Quoted 8 lines

And I suspect that it really is best to just remove the existing defconfig files. People can see them in the history to pick up what the heck they did, but no way will any sane model ever look even _remotely_ like them, so they really aren't a useful basis for going forward.

But don't worry. It didn't happen this merge window, obviously.

Linus

#12 lore raw
In reply to Linus Torvalds
Quoted 12 lines · Linus Torvalds

am, but I'm willing to do it only if I have a feeling that things are under control. And I'm not getting that feeling.

- TWO HUNDRED THOUSAND lines of arch/arm is just pure garbage, namely the defconfig files. Quite frankly, anybody who calls that anything but pure "noise" is just misguided and being stupid.

So yes, I do consider a lot of it "noise". When there's two hundred thousand lines of garbage, and it keeps growing without bounds, something needs to be done.

Quoted 7 lines

Now, I'm actually considering just getting rid of all the 'defconfig' files entirely. The x86 model is sane (there's two of them, nobody likely uses them), but ARM and POWERPC (and to a lesser config SH and MIPS) have turned the whole concept into a disgusting mess.

But while POWERPC has abused that thing too, in _other_ respects it has been much less egregious.

unfortunately there are so many different variants of the ARM architecture that these defconfigs spring up to ensure that a base compile for each one of them is performed. We run an autobuild which runs through all these defconfigs and produces a report of what happened to allow tracking of build regressions at the moment.

Quoted 5 lines

So I can largely fix the defconfig mess on my own (by just using a simple oneliner shell script to deletes them all) but that really only fixes one particularly annoying symptom - not the underlying issue. We do need somebody to maintain the arm platform mess, and actually tries to get some infrastructure or something in place so that it doesn't explode.

Someone should defiently cull the older defconfigs for sepcific machines, many of which probably don't get built for mainline. ~

Quoted 9 lines
Quoted 8 lines

The fact is that ARM-based devices multiply like rabbits, and there is a huge amount of diversity between the various derivatives. Also, support most of these devices has lived out of tree for a long time. Now that we have a relatively direct path which doesn't require any single person to have to understand all the mind-numbing details of all of these ARM-based platforms, it has allowed us to significantly improve the support for these devices upstream. Anything that is common to all devices still goes through RMK.

Quoted 5 lines

The thing is, I bet there is way more commonality still to be taken advantage of. And even if there isn't, we need somebody who cares, and doesn't just mindlessly aggregate all the crud. Somebody with the taste to say "ok, that's just too effin ugly, you need to fix things up" occasionally.

yes, there is that problem, and many of the big company players have yet to really see the necessity in this... It has taken a while now to convince the necessary people at Samsung that simply copying and modifying existing driver/support is simply not going to fly.

Signature · 3 lines
Ben ([email protected], http://www.fluff.org/)

  'a smiley only costs 4 bytes'
#13 lore raw
In reply to Linus Torvalds

[sorry if this message gets unthreaded, my mbox for linux-kernel got
corrupted so i've had to hand edit the headers]

Quoted 5 lines

- x86 has its drivers elsewhere, and they are _discoverable_ and not hardcoded to some platform. They have often also been useful (to say the least) to other architecture platforms. That's not always truefor all of them (we do have drivers/platform/x86, but at least that's maintained separately and is nowhere near the mess that is ARM)

There are some common drivers for ARM specific hardware under arch/arm/common but there's not a lot of that. I have been trying to ensure that new support re-uses common driver code. I was one of the people who worked with Piere to break the SDHCI reliance on PCI so that it could be re-used on platform device based systems such as the new Samsung SoC. I have also been trying to ensure that new support is carefully considered.

Unfortunately we often run into the following problems;

- The "Our hacked driver has been customer-tested and patched so often that it must be the one that gets submitted" argument... trying to get them to realised that firstly, it isn't of the necessary quality or has been already done elsewhere is very difficult.

- Often the hardware teams don't tell you where the blocks that have been included come from some common source. This makes the task of identifying a common driver difficult (they just make their own driver from the 'documentation' that the hardware guys hack).

I've been subject to this on at-least one occasion and currently considering renaming the s3c-hsotg driver to dw-hsotg (it turns out to be a block that was bought in from Synopsys' DesignWare)

- There is also a problem that as soon as the hardware that has even a small divergence from the common case that they need a special driver for it (against, see the SDHCI case where there's a couple of extra configuration register to control clocks, etc.)

When Harald Welte and I looked into some of these development practices we started to try and educate the developers about this. As an example we looked the clock structure we used to modle the clock trees in the SoCs, and found that in certain of their branches there where 4 copies of the same code, each with its own subtle bugs...

I have been trying very hard to make sure that these things are kept under control, but unfortunately it can be a hard task and until it hits these big companies in the pocket it very difficult to get them to actually pay for this.

As for the churn under arch/arm, then there are a few reasons for this and the size of the code involved.

- Firstly the implemetations of the ARM architecture can vary wildly depending on the implementor (and there can be several different implementing teams within one company). There is very little direct recommendation or specification from ARM on how the system around the core is layed out.

This means that there are a variety of different interupt controlers involved, and even when a standard one is used (ARM VIC) then the interrupt routing and base addresses of the control registers change between SoCs.

There is also a huge variety of memory layouts, the base of SDRAM and peripheral layout can change between individual SoCs, let alone families of them.

- Secondly there has been an rush to get as many header files in defining as many registers as possible for all the hardware, even if it turns out later to be a common block or not used.

I have been trying to remove as many #defines in headers where the result is only used once in the code to try and help the spiraling size of the project.

- Thirdly the core of these devices is also widly different, there is often far more complexity in how the SoC's bus and clock layouts are put together.

It is not like PCI where each device has a xMHZ clock and integrated PLL if needed. There are complex heirarchies of PLLs and clocks, which then need to be exported to the drivers to use, and often the clock gating is also integrated into the centralised clock control units instead of in the device.

Also, since there is often very little support for any dynamic bus discovery, a lot of the device information is dependant on knowing which SoC you are running on (unlike PCI where there's a specific configuration header).

Note, there are moves to try and get device trees supported on ARM to try and solve the previous paragraph, but we have an awfully big history to sort out, and it only really pushes that part of the problem out of the kernel onto the device manfactueres (who can barely get a bootloader to pass in a couple of registers).

So apart from that, another source for the churn is that when we do identify common parts, thigns need to get move and older systems updated to use them. These devices seem to evolve at such a rate it is difficult to predict what we'll see 6 months down the line.

The next problem is that if we don't get some support in each merge window, then the size of the task for the next one keeps growing and growing. It is already a problem just for the Samsung range of SoCs where we only really have core functionality for most of the SoCs; There are still drivers needed for some of the Security, Multimedia and connectivity subsystems to be sorted out.

I have been trying my best to keep control of these issues, but it is very difficult and it isn't my daytime job :(

Signature · 3 lines
Ben ([email protected], http://www.fluff.org/)

  'a smiley only costs 4 bytes'
#14 lore raw
In reply to Linus Torvalds
Quoted 2 lines · Linus Torvalds

snip

Quoted 5 lines

- TWO HUNDRED THOUSAND lines of arch/arm is just pure garbage, namely the defconfig files. Quite frankly, anybody who calls that anything but pure "noise" is just misguided and being stupid.

For some hard data which might be useful in arguments on this you might want to reference

http://armlinux.simtec.co.uk/reports/kautobuild-5year-report.pdf
http://armlinux.simtec.co.uk/reports/kautobuild-5year-stats.ods

The written report is not complete as I had hoped to have an announcement about the autobuilder fifth aniverasry soon. Perhaps this will be useful in keeping some default configurations which give us maximal coverage.

I provide this purely for information, I do not intend to upset you futher on this subject, pretty please do not shoot the messenger ;-)

Signature · 1 line
Regards Vincent