Skip to contentSkip to conversation

sunrpc port allocation and IANA reserved list

DISCUSSION Chris Friesen 4 participants 13 messages Nov 10, 2009

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

By default sunrpc ports are allocated at random in the range 665-1023U.
However, there are many ports within this range which have been
reserved by the IANA (and others like port 921 which are not formally reserved but are "well-known").

Given that a userspace application can be stopped and restarted at any time, and a sunrpc registration can happen at any time, what is the expected mechanism to prevent the kernel from allocating a port for use by sunrpc that reserved or well-known?

Apparently Redhat and Debian have distro-specific ways of dealing with this, but is there a standard solution? Should there be?

The current setup seems suboptimal.

Chris

#1 lore raw
Quoted 10 lines · Chris Friesen

By default sunrpc ports are allocated at random in the range 665-1023U.
However, there are many ports within this range which have been
reserved by the IANA (and others like port 921 which are not formally reserved but are "well-known").

Given that a userspace application can be stopped and restarted at any time, and a sunrpc registration can happen at any time, what is the expected mechanism to prevent the kernel from allocating a port for use by sunrpc that reserved or well-known?

Quoted 4 lines

Apparently Redhat and Debian have distro-specific ways of dealing with this, but is there a standard solution? Should there be?

The current setup seems suboptimal.

I believe both RH and Debian are using the same implementation:
<http://cyberelk.net/tim/software/portreserve/>.

Ben.

Signature · 3 lines
Ben Hutchings, Senior Software Engineer, Solarflare Communications
Not speaking for my employer; that's the marketing department's job.
They asked us to note that Solarflare product names are trademarked.
#2 lore raw
Quoted 7 lines · Ben Hutchings
Quoted 5 lines · Chris Friesen

Given that a userspace application can be stopped and restarted at any time, and a sunrpc registration can happen at any time, what is the expected mechanism to prevent the kernel from allocating a port for use by sunrpc that reserved or well-known?

Quoted 7 lines
Quoted 4 lines

Apparently Redhat and Debian have distro-specific ways of dealing with this, but is there a standard solution? Should there be?

The current setup seems suboptimal.

I believe both RH and Debian are using the same implementation:
<http://cyberelk.net/tim/software/portreserve/>.

That helps with the startup case, but still leaves a possible hole if an app using a fixed port number is restarted at runtime. During the window where nobody is bound to the port, the kernel could randomly assign it to someone else.

Chris

#3 lore raw
Quoted 13 lines · Chris Friesen
Quoted 12 lines · Ben Hutchings
Quoted 9 lines · Chris Friesen

Given that a userspace application can be stopped and restarted at any time, and a sunrpc registration can happen at any time, what is the expected mechanism to prevent the kernel from allocating a port for use by sunrpc that reserved or well-known?

Apparently Redhat and Debian have distro-specific ways of dealing with this, but is there a standard solution? Should there be?

The current setup seems suboptimal.

Quoted 7 lines
Quoted 2 lines

I believe both RH and Debian are using the same implementation:
<http://cyberelk.net/tim/software/portreserve/>.

That helps with the startup case, but still leaves a possible hole if an app using a fixed port number is restarted at runtime. During the window where nobody is bound to the port, the kernel could randomly assign it to someone else.

After some reflection it seems to me that the only way to close this race condition is to store the list of reserved ports in the kernel and simply avoid handing out a reserved address unless it is specifically requested.

Maybe we could keep the config files of the existing portreserve package, but rather than maintaining the reservation list itself the portreserve app would simply feed the reservations into the kernel (via /proc or netlink or something) at startup.

This would also avoid the need to modify the startup scripts of applications wanting to use a fixed port. The config file containing the port number would still be necessary, however.

Chris

#4 lore raw
In reply to Chris Friesen
Quoted 13 lines · Chris Friesen
Quoted 12 lines · Ben Hutchings
Quoted 9 lines · Chris Friesen

Given that a userspace application can be stopped and restarted at any time, and a sunrpc registration can happen at any time, what is the expected mechanism to prevent the kernel from allocating a port for use by sunrpc that reserved or well-known?

Apparently Redhat and Debian have distro-specific ways of dealing with this, but is there a standard solution? Should there be?

The current setup seems suboptimal.

Quoted 7 lines
Quoted 2 lines

I believe both RH and Debian are using the same implementation:
<http://cyberelk.net/tim/software/portreserve/>.

That helps with the startup case, but still leaves a possible hole if an app using a fixed port number is restarted at runtime. During the window where nobody is bound to the port, the kernel could randomly assign it to someone else.

Just use /proc/sys/sunrpc/{max,min}_resvport interface to restrict the range used to a safer one. That's what it is for...

Trond

#5 lore raw
Quoted 17 lines · Trond Myklebust
Quoted 16 lines · Chris Friesen
Quoted 14 lines · Ben Hutchings
Quoted 9 lines · Chris Friesen

Given that a userspace application can be stopped and restarted at any time, and a sunrpc registration can happen at any time, what is the expected mechanism to prevent the kernel from allocating a port for use by sunrpc that reserved or well-known?

Apparently Redhat and Debian have distro-specific ways of dealing with this, but is there a standard solution? Should there be?

The current setup seems suboptimal.

I believe both RH and Debian are using the same implementation:
<http://cyberelk.net/tim/software/portreserve/>.

Quoted 7 lines
Quoted 4 lines

That helps with the startup case, but still leaves a possible hole if an app using a fixed port number is restarted at runtime. During the window where nobody is bound to the port, the kernel could randomly assign it to someone else.

Just use /proc/sys/sunrpc/{max,min}_resvport interface to restrict the range used to a safer one. That's what it is for...

What constitutes a "safer range"? IANA has ports assigned intermittently all the way through the default RPC range. The largest unassigned range is 922-988 (since 921 is used by lwresd). If someone needs more than 66 ports, how are they supposed to handle it?

Chris

#6 lore raw
Quoted 23 lines · Chris Friesen
Quoted 22 lines · Trond Myklebust
Quoted 20 lines · Chris Friesen
Quoted 7 lines · Ben Hutchings
Quoted 5 lines · Chris Friesen

Given that a userspace application can be stopped and restarted at any time, and a sunrpc registration can happen at any time, what is the expected mechanism to prevent the kernel from allocating a port for use by sunrpc that reserved or well-known?

Quoted 7 lines
Quoted 4 lines

Apparently Redhat and Debian have distro-specific ways of dealing with this, but is there a standard solution? Should there be?

The current setup seems suboptimal.

I believe both RH and Debian are using the same implementation:
<http://cyberelk.net/tim/software/portreserve/>.

That helps with the startup case, but still leaves a possible hole if an app using a fixed port number is restarted at runtime. During the window where nobody is bound to the port, the kernel could randomly assign it to someone else.

Quoted 7 lines
Quoted 2 lines

Just use /proc/sys/sunrpc/{max,min}_resvport interface to restrict the range used to a safer one. That's what it is for...

What constitutes a "safer range"? IANA has ports assigned intermittently all the way through the default RPC range. The largest unassigned range is 922-988 (since 921 is used by lwresd). If someone needs more than 66 ports, how are they supposed to handle it?

Distributions can, and should, set the default to whatever range that constitutes a broad percentile of their customers, and then leave instructions on how to change that default for the people who run the funky setups.

The people who are trying to run absolutely all IANA registered services on a single Linux machine that is also trying to run as an NFS client may have a problem, but then again, how many setups do you know who are trying to do that?

Trond

#8 lore raw
In reply to Chris Friesen
Quoted 18 lines · Chris Friesen
Quoted 17 lines · Trond Myklebust
Quoted 16 lines · Chris Friesen
Quoted 14 lines · Ben Hutchings
Quoted 9 lines · Chris Friesen

Given that a userspace application can be stopped and restarted at any time, and a sunrpc registration can happen at any time, what is the expected mechanism to prevent the kernel from allocating a port for use by sunrpc that reserved or well-known?

Apparently Redhat and Debian have distro-specific ways of dealing with this, but is there a standard solution? Should there be?

The current setup seems suboptimal.

I believe both RH and Debian are using the same implementation:
<http://cyberelk.net/tim/software/portreserve/>.

Quoted 7 lines
Quoted 7 lines
Quoted 4 lines

That helps with the startup case, but still leaves a possible hole if an app using a fixed port number is restarted at runtime. During the window where nobody is bound to the port, the kernel could randomly assign it to someone else.

Just use /proc/sys/sunrpc/{max,min}_resvport interface to restrict the range used to a safer one. That's what it is for...

Unless I'm much mistaken, that only affects in-kernel SunRPC users.

Quoted 4 lines

What constitutes a "safer range"? IANA has ports assigned intermittently all the way through the default RPC range. The largest unassigned range is 922-988 (since 921 is used by lwresd). If someone needs more than 66 ports, how are they supposed to handle it?

I'm sure we could afford 128 bytes for a blacklist of privileged ports. However, the problem is that there is no API for userland to request 'any free privileged port' - it has to just try binding to different ports until it finds one available. This means that the kernel can't tell whether a process is trying to allocate a specifically assigned port or whether the blacklist should be applied.

Ben.

Signature · 3 lines
Ben Hutchings, Senior Software Engineer, Solarflare Communications
Not speaking for my employer; that's the marketing department's job.
They asked us to note that Solarflare product names are trademarked.
#9 lore raw
Quoted 34 lines · Ben Hutchings
Quoted 18 lines · Chris Friesen
Quoted 17 lines · Trond Myklebust
Quoted 16 lines · Chris Friesen
Quoted 14 lines · Ben Hutchings
Quoted 9 lines · Chris Friesen

Given that a userspace application can be stopped and restarted at any time, and a sunrpc registration can happen at any time, what is the expected mechanism to prevent the kernel from allocating a port for use by sunrpc that reserved or well-known?

Apparently Redhat and Debian have distro-specific ways of dealing with this, but is there a standard solution? Should there be?

The current setup seems suboptimal.

I believe both RH and Debian are using the same implementation:
<http://cyberelk.net/tim/software/portreserve/>.

Quoted 7 lines
Quoted 7 lines
Quoted 4 lines

That helps with the startup case, but still leaves a possible hole if an app using a fixed port number is restarted at runtime. During the window where nobody is bound to the port, the kernel could randomly assign it to someone else.

Just use /proc/sys/sunrpc/{max,min}_resvport interface to restrict the range used to a safer one. That's what it is for...

Unless I'm much mistaken, that only affects in-kernel SunRPC users.

Quoted 4 lines

What constitutes a "safer range"? IANA has ports assigned intermittently all the way through the default RPC range. The largest unassigned range is 922-988 (since 921 is used by lwresd). If someone needs more than 66 ports, how are they supposed to handle it?

Quoted 4 lines

I'm sure we could afford 128 bytes for a blacklist of privileged ports. However, the problem is that there is no API for userland to request 'any free privileged port' - it has to just try binding to different ports until it finds one available.

bindresvport(3) and bindresvport_sa(3t) ?

Quoted 3 lines

This means that the kernel can't
tell whether a process is trying to allocate a specifically assigned port or whether the blacklist should be applied.

Such a blacklist would have to be managed by glibc or libtirpc.

--
Chuck Lever
chuck[dot]lever[at]oracle[dot]com

#10 lore raw
Quoted 4 lines · Chuck Lever
Quoted 2 lines · Ben Hutchings
Quoted 1 line · Chris Friesen

On 11/10/2009 02:26 PM, Trond Myklebust wrote:

[...]

Quoted 13 lines
Quoted 13 lines
Quoted 3 lines
Quoted 3 lines

Just use /proc/sys/sunrpc/{max,min}_resvport interface to restrict the
range used to a safer one. That's what it is for...

Unless I'm much mistaken, that only affects in-kernel SunRPC users.

Quoted 6 lines

What constitutes a "safer range"? IANA has ports assigned intermittently all the way through the default RPC range. The largest
unassigned range is 922-988 (since 921 is used by lwresd). If someone
needs more than 66 ports, how are they supposed to handle it?

Quoted 7 lines
Quoted 5 lines

I'm sure we could afford 128 bytes for a blacklist of privileged ports.
However, the problem is that there is no API for userland to request 'any free privileged port' - it has to just try binding to different ports until it finds one available.

bindresvport(3) and bindresvport_sa(3t) ?

These are library calls; they are not an API between userland the kernel.

Quoted 5 lines
Quoted 3 lines

This means that the kernel can't
tell whether a process is trying to allocate a specifically assigned port or whether the blacklist should be applied.

Such a blacklist would have to be managed by glibc or libtirpc.

Right.

Ben.

Signature · 3 lines
Ben Hutchings, Senior Software Engineer, Solarflare Communications
Not speaking for my employer; that's the marketing department's job.
They asked us to note that Solarflare product names are trademarked.
#11 lore raw
In reply to Ben Hutchings
Quoted 6 lines · Ben Hutchings

I'm sure we could afford 128 bytes for a blacklist of privileged ports. However, the problem is that there is no API for userland to request 'any free privileged port' - it has to just try binding to different ports until it finds one available. This means that the kernel can't tell whether a process is trying to allocate a specifically assigned port or whether the blacklist should be applied.

That's unfortunate. Maybe a bindresvport syscall would make sense, but that's starting to get bloated. At least for userspace apps netstat will show who the owner is and it might be possible to clean it up without rebooting.

For the ports allocated in the kernel via xs_get_random_port() and xprt_bindresvport() it would be fairly easy to check a blacklist.

Interestingly, the current kernel range starts at 665 because IPMI is known to use 664. The current glibc range to be tried in bindresvport() starts at 600, which seems to be a bug waiting to happen.

Chris

#7 lore raw
In reply to Trond Myklebust
Quoted 4 lines · Trond Myklebust

The people who are trying to run absolutely all IANA registered services on a single Linux machine that is also trying to run as an NFS client may have a problem, but then again, how many setups do you know who are trying to do that?

BTW: Even these setups can be fixed. One way to extend the available port ranges is to allocate more than one IP address; either through the use of multiple NICs, or through address aliasing. You can set up the services to listen on one interface, while setting up your routing tables to ensure that any outgoing connections to your NFS servers go through through a different interface.

You could also use virtualised machines to host the services, etc...

Trond

#12 lore raw
In reply to Ben Hutchings
Quoted 6 lines · Ben Hutchings
Quoted 3 lines · Chris Friesen
Quoted 2 lines · Trond Myklebust

Just use /proc/sys/sunrpc/{max,min}_resvport interface to restrict the range used to a safer one. That's what it is for...

Unless I'm much mistaken, that only affects in-kernel SunRPC users.

There shouldn't be that many non-kernel sunrpc users that require privileged ports.

The exceptions I can think of are

* the 'mount' program, if you are using the legacy binary mount interface. Use text mounts instead...
* rpc.statd might also use privileged ports when booting the system when it needs to notify servers of the reboot. You can set that port number using the '-o' option.

Others? Do any of the NIS services require it?

Trond