Skip to contentSkip to conversation

mm: Change global memory state symbols to GPL-only

PATCH Richard Yao 3 participants 4 messages Sep 1, 2015

lore
Share & tools
lore
#0 lore raw
Quoted 20 lines · Ben Hutchings
Quoted 4 lines · Michal Hocko
Quoted 3 lines · Ben Hutchings
Quoted 2 lines · Michal Hocko
Quoted 1 line · Ben Hutchings

Proprietary modules should not be able to touch vm_stat or particip=

ate

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 1 line

in shrinking.

=20
How does the external and !GPL fs does slab reclaim? Those are essent=

ial

Quoted 9 lines
Quoted 4 lines
Quoted 1 line

for the proper memory balancing.

=20
If they know how to do shrinking on Linux then they are probably derivative works of Linux.

=20
I am not sure I understand. They are shrinking their internal cached objects and that is hardly a derivative work. The shrinker API is only meant to let them know _when_ this should happen and the interface is a pretty much simple callback API.

Quoted 5 lines

It is a Linux-specific API and I don't think other kernels provide something similar to loadable modules. It enables a module to turn a large part of the system RAM into a cache and have the MM effectively tell it the correct size of that cache, thus tightly integrating with global memory management.

The idea of providing third party drivers with the hooks that they need to respond to memory pressure is by no means Linux-specific. In OpenSolaris, there are counters for drivers to keep track of memory usage and try to stay ahead of it, which works because there is no direct reclaim. There are also callbacks for the SLAB code to defragment the caches used by the drivers. Giving non-GPL drivers to respond to memory pressure seems reasonable. Given how long this has been the csae, it seems strange to change it now.

Every lawyer with whom I have spoken about ports of drivers from other platforms has informed me tht they are not derived works of Linux. I cannot imagine a simple callback to free memory on demand would change that.

Quoted 11 lines

It seemed to me that this met the test for 'should this be EXPORT_SYMBOL_GPL'.

Quoted 5 lines

I do not want to defend a proprietary code here but this sounds like an obstruction for those modules which will lead into a worse code in the end because they should somehow manage the cache and it is much better when the core (MM) tells them when it makes sense rather than external heuristics.

Yes, that's the idea, proprietary code should not be helped in this way.

The ZFSOnLinux kernel driver uses these symbols. It is fully open source and not proprietary. If this is merged, it will break the driver and it will be incredibly difficult to deal with it while maintaing the ability to respond to system memory pressure. The kernel just does not provide any other hooks to my knowledge.

#1 lore raw
Quoted 27 lines · Richard Yao
Quoted 20 lines · Ben Hutchings
Quoted 4 lines · Michal Hocko
Quoted 3 lines · Ben Hutchings
Quoted 2 lines · Michal Hocko
Quoted 1 line · Ben Hutchings

Proprietary modules should not be able to touch vm_stat or particip=

ate

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 1 line

in shrinking.

=20
How does the external and !GPL fs does slab reclaim? Those are essent=

ial

Quoted 9 lines
Quoted 4 lines
Quoted 1 line

for the proper memory balancing.

=20
If they know how to do shrinking on Linux then they are probably derivative works of Linux.

=20
I am not sure I understand. They are shrinking their internal cached objects and that is hardly a derivative work. The shrinker API is only meant to let them know _when_ this should happen and the interface is a pretty much simple callback API.

Quoted 5 lines

It is a Linux-specific API and I don't think other kernels provide something similar to loadable modules. It enables a module to turn a large part of the system RAM into a cache and have the MM effectively tell it the correct size of that cache, thus tightly integrating with global memory management.

Quoted 7 lines

The idea of providing third party drivers with the hooks that they need to respond to memory pressure is by no means Linux-specific. In OpenSolaris, there are counters for drivers to keep track of memory usage and try to stay ahead of it, which works because there is no direct reclaim. There are also callbacks for the SLAB code to defragment the caches used by the drivers. Giving non-GPL drivers to respond to memory pressure seems reasonable. Given how long this has been the csae, it seems strange to change it now.

Especially considering that certain shrinker implementations in the Linux tree can be used to directly refute the "derived work" arguments given above.

Yeah, I'm about to ramble on about XFS history again. ;)

When XFS was ported to Linux, all the "memory shakers" that XFS used on Irix were either removed or neutered because Linux didn't have any infrastructure to drive them. For example, have a look at the 1996 commit to Irix that introduced XFS quota support.

http://oss.sgi.com/cgi-bin/gitweb.cgi?p=archive/xfs-import.git;a=blob;f=fs/xfs/xfs_qm.c;hb=b5a7144e5bd96fa98db6118d040dabad896ba785

Search for xfs_qm_shake() and xfs_qm_shake_freelist(). xfs_qm_shake() is the registered shaker callback that came from the mm subsystem when memory was low. It used a priority level rather than a count of of objects to shrink, but otherwise it's function is identical to a Linux shrinker callback but was written years before an XFS linux port was even thought of.

Now fast forward to the introduction of the rudimentary shrinker infrastructure to Linux back in 2002, and we have this XFS commit:

http://oss.sgi.com/cgi-bin/gitweb.cgi?p=archive/xfs-import.git;a=commitdiff;h=27a25ec7d0cb324a0129d609edef40a7468a510f

Which connected the XFS quota shaker up to the brand spanking new Linux shrinker infrastructure. Check the code that was being run by this linux shrinker - it's clearly the same freelist/shaker code that was originally committed to Irix.

The XFS quota shrinker code has change in the 13 years since then, so it's roots are no longer quite so obvious. That, however, doesn't mean we have forgotten where the code originally came from, nor have we forgotten that kernel-based, subsystem-independent memory reclaim callbacks had existed for many, many years in other operating systems before Linux re-invented them.

So, IMO, a shrinker implementation in a kernel module does not automatically make the kernel module a derived work of the kernel. The concept is common across many operating systems, and the kernel APIs are sufficiently similar in function and specification that porting shrinkers between different operating systems is a trivial effort....

Just my 2c worth.

Cheers,

Dave.

Signature · 2 lines
Dave Chinner
[email protected]
#2 lore raw
In reply to Richard Yao
Quoted 34 lines · Richard Yao
Quoted 24 lines · Ben Hutchings
Quoted 15 lines · Michal Hocko
Quoted 6 lines · Ben Hutchings
Quoted 5 lines · Michal Hocko
Quoted 3 lines · Ben Hutchings

Proprietary modules should not be able to touch vm_stat or participate
in shrinking.

Quoted 7 lines
Quoted 3 lines

How does the external and !GPL fs does slab reclaim? Those are essential
for the proper memory balancing.

If they know how to do shrinking on Linux then they are probably
derivative works of Linux.

Quoted 7 lines

I am not sure I understand. They are shrinking their internal cached
objects and that is hardly a derivative work. The shrinker API is only
meant to let them know _when_ this should happen and the interface is
a pretty much simple callback API.

Quoted 8 lines

It is a Linux-specific API and I don't think other kernels provide something similar to loadable modules. It enables a module to turn a
large part of the system RAM into a cache and have the MM effectively
tell it the correct size of that cache, thus tightly integrating with
global memory management.

Quoted 9 lines

The idea of providing third party drivers with the hooks that they need to
respond to memory pressure is by no means Linux-specific. In OpenSolaris, there
are counters for drivers to keep track of memory usage and try to stay ahead of
it, which works because there is no direct reclaim. There are also callbacks
for the SLAB code to defragment the caches used by the drivers.

So you confirm that the Linux and Solaris APIs for this are quite different.

Quoted 2 lines

Giving non-GPL
drivers to respond to memory pressure seems reasonable.

I don't see why.

[...]

Quoted 6 lines
Quoted 2 lines

Yes, that's the idea, proprietary code should not be helped in this way.

The ZFSOnLinux kernel driver uses these symbols. It is fully open source and
not proprietary.

[...]

It's not GPL-compatible, which in terms of Linux kernel modules is no better than proprietary.

Ben.

Signature · 2 lines
Ben Hutchings
The first rule of tautology club is the first rule of tautology club.
#3 lore raw

I seem to have botched the reply headers, so I am resending this. I use mutt fairly infrequently, but I am making an effort to use it more it more.

Quoted 46 lines · Ben Hutchings
Quoted 34 lines · Richard Yao
Quoted 24 lines · Ben Hutchings
Quoted 15 lines · Michal Hocko
Quoted 6 lines · Ben Hutchings
Quoted 5 lines · Michal Hocko
Quoted 3 lines · Ben Hutchings

Proprietary modules should not be able to touch vm_stat or participate
in shrinking.

Quoted 7 lines
Quoted 3 lines

How does the external and !GPL fs does slab reclaim? Those are essential
for the proper memory balancing.

If they know how to do shrinking on Linux then they are probably
derivative works of Linux.

Quoted 7 lines

I am not sure I understand. They are shrinking their internal cached
objects and that is hardly a derivative work. The shrinker API is only
meant to let them know _when_ this should happen and the interface is
a pretty much simple callback API.

Quoted 8 lines

It is a Linux-specific API and I don't think other kernels provide something similar to loadable modules. It enables a module to turn a
large part of the system RAM into a cache and have the MM effectively
tell it the correct size of that cache, thus tightly integrating with
global memory management.

Quoted 9 lines

The idea of providing third party drivers with the hooks that they need to
respond to memory pressure is by no means Linux-specific. In OpenSolaris, there
are counters for drivers to keep track of memory usage and try to stay ahead of
it, which works because there is no direct reclaim. There are also callbacks
for the SLAB code to defragment the caches used by the drivers.

Quoted 2 lines

So you confirm that the Linux and Solaris APIs for this are quite different.

They both have VFS implementations, but the designs are different. According to attorneys, adapting code to port a filesystem driver from a foreign VFS to the Linux VFS to make a port is insufficient to form a derived work of Linux. All kernels have memory management and preferred ways of dealing with pressure. Using the preferred way of dealing with it should be similarly insufficient to form a derived work of Linux.

Quoted 4 lines
Quoted 2 lines

Giving non-GPL
drivers to respond to memory pressure seems reasonable.

I don't see why.

It is an existing function and it is needs to be avaliable to allow third party drivers to be good citizens inside the kernel.

I do not know all of the filesystems whose memory management will become worse from this, but I know that ZFS is one. Lustre is another because it hooks into ZFS. I am told that the OpenAFS client uses this too..

Quoted 10 lines
Quoted 6 lines
Quoted 2 lines

Yes, that's the idea, proprietary code should not be helped in this way.

The ZFSOnLinux kernel driver uses these symbols. It is fully open source and
not proprietary.

[...]

It's not GPL-compatible, which in terms of Linux kernel modules is no better than proprietary.

Any code under a F/OSS license follows the OSD, which means that it is under a better license than proprietary code:

http://opensource.org/definition

Rather than sabotaging working code (especially working F/OSS code), maybe it would be better to ask why it being used instead of code under your preferred license and invest energy into developing better alternatives under licenses that you prefer. If someone makes something better, people would use it and developers will flock to it. Artificially damaging competition does not help anyone.