qemu: Fix race condition when binding ports
PATCH 1/1 qemu: Fix race condition when binding ports
Two Qemus can race to bind the same VNC port. It is valid for multiple listen()s on the same socket to succeed, but only the first bind() will succeed.
In the case that two Qemus are racing, the second one will fail with an EADDRINUSE, and bail with a fatal error which renders the domain functionally useless.
In the case that bind() fails with EADDRINUSE, rebind the socket again and try for the next port.
---
This fix has been in XenServer for two and a half years
--- qemu-sockets.c | 23 ++++++++++++++++------- 1 file changed, 16 insertions(+), 7 deletions(-)
qemu-sockets.c +16 -7
@@ -163,6 +163,7 @@ int inet_listen(const char *str, char *ostr, int olen,/* create socket + bind */ for (e = res; e !=NULL ; e = e->ai_next) { + rebind:getnameinfo ((struct sockaddr *)e->ai_addr,e->ai_addrlen, uaddr,INET6_ADDRSTRLEN ,uport,32 ,NI_NUMERICHOST |NI_NUMERICSERV );
@@ -186,7 +187,20 @@ int inet_listen(const char *str, char *ostr, int olen,if (sockets_debug)fprintf (stderr,"%s: bind(%s,%s,%d): OK\n" , __FUNCTION__,inet_strfamily (e->ai_family), uaddr,inet_getport (e)); - goto listen; +if (listen (slisten,1 ) ==0 ) { + goto listened; + }else { +int err = errno; + +perror ("listen" ); +closesocket (slisten); + +if (err ==EADDRINUSE ) + goto rebind; +freeaddrinfo (res); +return -1 ; + } + } try_next = to && (inet_getport (e) <= to + port_offset);if (!try_next || sockets_debug)
@@ -205,12 +219,7 @@ int inet_listen(const char *str, char *ostr, int olen,freeaddrinfo (res);return -1 ; -listen: -if (listen (slisten,1 ) !=0 ) { -perror ("listen" ); -closesocket (slisten); -return -1 ; - } +listened:if (ostr) {if (e->ai_family ==PF_INET6 ) { snprintf(ostr, olen, "[%s]:%d%s", uaddr,
Signature · 1 line
1.7.10.4
Quoted 10 lines · Andrew Cooper
Two Qemus can race to bind the same VNC port. It is valid for multiple listen()s on the same socket to succeed, but only the first
bind()will succeed.In the case that two Qemus are racing, the second one will fail with an EADDRINUSE, and bail with a fatal error which renders the domain functionally useless.
In the case that
bind()fails with EADDRINUSE, rebind the socket again and try for the next port.
Sorry if I'm being a bit dense here, but it looks like you have those mixed up: if listen() fails with EADDRINUSE, you're going back to call bind() again; which would suggest that you can have multible bind()s but only a single listen()? Or am I confused?
In any case, this is clearly a bug fix, and a very well tested one at that, so:
Quoted 62 lines
Signed-off-by Andrew Cooper <[email protected]>CC Ian Campbell <[email protected]>CC Ian Jackson <[email protected]>CC Stefano Stabellini <[email protected]>CC George Dunlap <[email protected]>---
This fix has been in XenServer for two and a half years
--- qemu-sockets.c | 23 ++++++++++++++++------- 1 file changed, 16 insertions(+), 7 deletions(-)qemu-sockets.c +16
-7@@ -163,6 +163,7 @@ int inet_listen(const char *str, char *ostr, int olen,/* create socket + bind */ for (e = res; e !=NULL ; e = e->ai_next) { + rebind:getnameinfo ((struct sockaddr *)e->ai_addr,e->ai_addrlen, uaddr,INET6_ADDRSTRLEN ,uport,32 ,NI_NUMERICHOST |NI_NUMERICSERV );@@ -186,7 +187,20 @@ int inet_listen(const char *str, char *ostr, int olen,if (sockets_debug)fprintf (stderr,"%s: bind(%s,%s,%d): OK\n" , __FUNCTION__,inet_strfamily (e->ai_family), uaddr,inet_getport (e)); - goto listen; +if (listen (slisten,1 ) ==0 ) { + goto listened; + }else { +int err = errno; + +perror ("listen" ); +closesocket (slisten); + +if (err ==EADDRINUSE ) + goto rebind; +freeaddrinfo (res); +return -1 ; + } + } try_next = to && (inet_getport (e) <= to + port_offset);if (!try_next || sockets_debug)@@ -205,12 +219,7 @@ int inet_listen(const char *str, char *ostr, int olen,freeaddrinfo (res);return -1 ; -listen: -if (listen (slisten,1 ) !=0 ) { -perror ("listen" ); -closesocket (slisten); -return -1 ; - } +listened:if (ostr) {if (e->ai_family ==PF_INET6 ) { snprintf(ostr, olen, "[%s]:%d%s", uaddr,
Quoted 10 lines · George Dunlap
Quoted 9 lines · Andrew Cooper
Two Qemus can race to bind the same VNC port. It is valid for multiple listen()s on the same socket to succeed, but only the first
bind()will succeed.In the case that two Qemus are racing, the second one will fail with an EADDRINUSE, and bail with a fatal error which renders the domain functionally
useless.
Quoted 8 lines
Quoted 3 lines
In the case that
bind()fails with EADDRINUSE, rebind the socket again and try
for the next port.Sorry if I'm being a bit dense here, but it looks like you have those mixed up: if
listen()fails with EADDRINUSE, you're going back to callbind()again; which would suggest that you can have multible bind()s but only a singlelisten()? Or am I confused?
Nope - very definitely me who messed up the message. v2 on its way.
~Andrew
PATCH v2 1/1 qemu: Fix race condition when opening ports
In reply to George DunlapTwo Qemus can race to bind the same VNC port. It is valid for multiple bind()s on the same socket to succeed, but only the first listen() will succeed.
In the case that two Qemus are starting at the same time, and both trying to grab the next free VNC port, the second one will fail with an EADDRINUSE, and bail with a fatal error which renders the domain functionally useless.
In the case that listen() fails with EADDRINUSE, rebind the socket again and try for the next port.
---
Changes in v2:
* Fix commit message.
This fix has been in XenServer for two and a half years
--- qemu-sockets.c | 23 ++++++++++++++++------- 1 file changed, 16 insertions(+), 7 deletions(-)
qemu-sockets.c +16 -7
@@ -163,6 +163,7 @@ int inet_listen(const char *str, char *ostr, int olen,/* create socket + bind */ for (e = res; e !=NULL ; e = e->ai_next) { + rebind:getnameinfo ((struct sockaddr *)e->ai_addr,e->ai_addrlen, uaddr,INET6_ADDRSTRLEN ,uport,32 ,NI_NUMERICHOST |NI_NUMERICSERV );
@@ -186,7 +187,20 @@ int inet_listen(const char *str, char *ostr, int olen,if (sockets_debug)fprintf (stderr,"%s: bind(%s,%s,%d): OK\n" , __FUNCTION__,inet_strfamily (e->ai_family), uaddr,inet_getport (e)); - goto listen; +if (listen (slisten,1 ) ==0 ) { + goto listened; + }else { +int err = errno; + +perror ("listen" ); +closesocket (slisten); + +if (err ==EADDRINUSE ) + goto rebind; +freeaddrinfo (res); +return -1 ; + } + } try_next = to && (inet_getport (e) <= to + port_offset);if (!try_next || sockets_debug)
@@ -205,12 +219,7 @@ int inet_listen(const char *str, char *ostr, int olen,freeaddrinfo (res);return -1 ; -listen: -if (listen (slisten,1 ) !=0 ) { -perror ("listen" ); -closesocket (slisten); -return -1 ; - } +listened:if (ostr) {if (e->ai_family ==PF_INET6 ) { snprintf(ostr, olen, "[%s]:%d%s", uaddr,
Signature · 1 line
1.7.10.4
Re: [PATCH v2 qemu-traditional] qemu: Fix race condition when opening ports
Quoted 10 lines · Andrew Cooper
Two Qemus can race to bind the same VNC port. It is valid for multiple bind()s on the same socket to succeed, but only the first
listen()will succeed.In the case that two Qemus are starting at the same time, and both trying to grab the next free VNC port, the second one will fail with an EADDRINUSE, and bail with a fatal error which renders the domain functionally useless.
In the case that
listen()fails with EADDRINUSE, rebind the socket again and try for the next port.
Does this do the right thing if vncunused=0? In that case the user has given a specific port which we want to bind to, so going to the next port is not desired.
Quoted 33 lines
Signed-off-by Andrew Cooper <[email protected]>CC Ian Campbell <[email protected]>CC Ian Jackson <[email protected]>CC Stefano Stabellini <[email protected]>Release-acked-by George Dunlap <[email protected]>---
Changes in v2:
* Fix commit message.This fix has been in XenServer for two and a half years
--- qemu-sockets.c | 23 ++++++++++++++++------- 1 file changed, 16 insertions(+), 7 deletions(-)qemu-sockets.c +1
-1@@ -163,6 +163,7 @@ int inet_listen(const char *str, char *ostr, int olen,/* create socket + bind */ for (e = res; e !=NULL ; e = e->ai_next) { + rebind:getnameinfo ((struct sockaddr *)e->ai_addr,e->ai_addrlen, uaddr,INET6_ADDRSTRLEN ,uport,32 ,NI_NUMERICHOST |NI_NUMERICSERV );@@ -186,7 +187,20 @@ int inet_listen(const char *str, char *ostr, int olen,if (sockets_debug)fprintf (stderr,"%s: bind(%s,%s,%d): OK\n" , __FUNCTION__,inet_strfamily (e->ai_family), uaddr,inet_getport (e)); - goto listen;
qemu-sockets.c int inet_listen(const char *str, char *ostr, int olen, · line 199
Quoted 10 lines
qemu-sockets.c +10
-0+if (listen (slisten,1 ) ==0 ) { + goto listened; + }else { +int err = errno; + +perror ("listen" ); +closesocket (slisten); + +if (err ==EADDRINUSE ) + goto rebind;
Does this code actually try for the next port? The try_next check is after the goto here (you can see it in the context below). Won't it just try to rebind the same port again and again?
Quoted 21 lines
qemu-sockets.c +5
-6+freeaddrinfo (res); +return -1 ; + } + } try_next = to && (inet_getport (e) <= to + port_offset);if (!try_next || sockets_debug)@@ -205,12 +219,7 @@ int inet_listen(const char *str, char *ostr, int olen,freeaddrinfo (res);return -1 ; -listen: -if (listen (slisten,1 ) !=0 ) { -perror ("listen" ); -closesocket (slisten); -return -1 ; - } +listened:if (ostr) {if (e->ai_family ==PF_INET6 ) { snprintf(ostr, olen, "[%s]:%d%s", uaddr,
Re: [PATCH v2 qemu-traditional] qemu: Fix race condition when opening ports
In reply to Andrew CooperQuoted 17 lines
-----Original Message-----
From: [email protected] [mailto:xen-devel- [email protected]] On Behalf Of Andrew Cooper
Sent: 09 December 2013 1:27 PM
To: Xen-devel
Cc: Andrew Cooper; Ian Jackson; Ian Campbell; Stefano Stabellini
Subject: [Xen-devel] [PATCH v2 qemu-traditional] qemu: Fix race condition when opening portsTwo Qemus can race to bind the same VNC port. It is valid for multiple bind()s on the same socket to succeed, but only the first
listen()will succeed.In the case that two Qemus are starting at the same time, and both trying to grab the next free VNC port, the second one will fail with an EADDRINUSE, and bail with a fatal error which renders the domain functionally useless.
Quoted 8 lines
In the case that
listen()fails with EADDRINUSE, rebind the socket again and try for the next port.Signed-off-by Andrew Cooper <[email protected]>CC Ian Campbell <[email protected]>CC Ian Jackson <[email protected]>CC Stefano Stabellini <[email protected]>Release-acked-by George Dunlap <[email protected]>
Andrew, if you are upstream other people stuff you should at least give credit to the original author.
Frediano
Re: [PATCH v2 qemu-traditional] qemu: Fix race condition when opening ports
Quoted 20 lines · Frediano Ziglio
Quoted 20 lines
-----Original Message-----
From: [email protected] [mailto:xen-devel- [email protected]] On Behalf Of Andrew Cooper
Sent: 09 December 2013 1:27 PM
To: Xen-devel
Cc: Andrew Cooper; Ian Jackson; Ian Campbell; Stefano Stabellini
Subject: [Xen-devel] [PATCH v2 qemu-traditional] qemu: Fix race condition when opening portsTwo Qemus can race to bind the same VNC port. It is valid for multiple bind()s on the same socket to succeed, but only the first
listen()will succeed.In the case that two Qemus are starting at the same time, and both trying to grab the next free VNC port, the second one will fail with an EADDRINUSE, and bail with a fatal error which renders the domain functionally useless.
In the case that
listen()fails with EADDRINUSE, rebind the socket again and try for the next port.
Quoted 9 lines
Quoted 5 lines
Signed-off-by Andrew Cooper <[email protected]>CC Ian Campbell <[email protected]>CC Ian Jackson <[email protected]>CC Stefano Stabellini <[email protected]>Release-acked-by George Dunlap <[email protected]>Andrew, if you are upstream other people stuff you should at least give credit to the original author.
Frediano
The original author was "xenhg" according to the history. It is not even clear from the original bug ticket who the author could have been.
In this case, my Signed-off-by line falls into category b) of the Developer’s Certificate of Origin.
This is an unfortunate problem with attempting to weed through a neglected patch queue where proper authorship was not considered at the time of commit.
~Andrew
Re: [PATCH v2 qemu-traditional] qemu: Fix race condition when opening ports
In reply to Andrew CooperAndrew Cooper writes ("[PATCH v2 qemu-traditional] qemu: Fix race condition when opening ports"):
Quoted 3 lines
Two Qemus can race to bind the same VNC port. It is valid for multiple bind()s on the same socket to succeed, but only the first
listen()will succeed.
I don't think this is correct. The SuS page for listen(2)
http://pubs.opengroup.org/onlinepubs/9699919799/functions/listen.html
doesn't list EADDRINUSE as a possible error condition.
Although wheezy's listen(2) manpage does.
Quoted 6 lines
In the case that two Qemus are starting at the same time, and both trying to grab the next free VNC port, the second one will fail with an EADDRINUSE, and bail with a fatal error which renders the domain functionally useless.
In the case that
listen()fails with EADDRINUSE, rebind the socket again and try for the next port.
My preliminary view is that this is a kernel bug.
Ian.
Re: [PATCH v2 qemu-traditional] qemu: Fix race condition when opening ports
Quoted 16 lines
Andrew Cooper writes ("[PATCH v2 qemu-traditional] qemu: Fix race condition when opening ports"):
Quoted 3 lines
Two Qemus can race to bind the same VNC port. It is valid for multiple bind()s on the same socket to succeed, but only the first
listen()will succeed.I don't think this is correct. The SuS page for listen(2)
http://pubs.opengroup.org/onlinepubs/9699919799/functions/listen.html
doesn't list EADDRINUSE as a possible error condition.Although wheezy's listen(2) manpage does.
Quoted 3 lines
In the case that two Qemus are starting at the same time, and both trying to grab the next free VNC port, the second one will fail with an EADDRINUSE, and bail with a fatal error which renders the domain
Quoted 9 lines
functionally useless.
Quoted 3 lines
In the case that
listen()fails with EADDRINUSE, rebind the socket again and try for the next port.My preliminary view is that this is a kernel bug.
Ian.
My Ubuntu "man 2 listen" reports
EADDRINUSE
Another socket is already listening on the same port.
(same from http://man7.org/linux/man-pages/man2/listen.2.html)
Frediano