Skip to contentSkip to conversation

prserv: make localhost work

PATCH In review Yu, Mingli 4 participants 6 messages Aug 31, 2021 – Sep 1, 2021 1 message not in the archive

lore
Share & tools
lore

Yu, Mingli

author
#1 lore raw

PATCH 1/1 prserv: make localhost work

In reply to a message not in the archive

From: Mingli Yu <[email protected]>

After [1] introduced, the PR server doesn't work in docker when use below setting like before.
PRSERV_HOST = "localhost:0"

And it's because the localhost is resolved to an ipv6 address ::1 as
the below bitbake-prserv shows.
bitbake$ bitbake-prserv --start --host=localhost --port=42005
bitbake$ cat prserv.log
Traceback (most recent call last):
File "/OE/nodistro/honister/bitbake/bin/bitbake-prserv", line 55, in <module>
ret = main()
File "/OE/nodistro/honister/bitbake/bin/bitbake-prserv", line 46, in main
ret=prserv.serv.start_daemon(options.dbfile, options.host, options.port,os.path.abspath(options.logfile), options.read_only)
File "/OE/nodistro/honister/bitbake/lib/prserv/serv.py", line 226, in start_daemon
run_as_daemon(daemon_main, pidfile, os.path.abspath(logfile))
File "/OE/nodistro/honister/bitbake/lib/prserv/serv.py", line 202, in run_as_daemon
func()
File "/OE/nodistro/honister/bitbake/lib/prserv/serv.py", line 224, in daemon_main
server.serve_forever()
File "/OE/nodistro/honister/bitbake/lib/bb/asyncrpc/serv.py", line 233, in serve_forever
self.start()
File "/OE/nodistro/honister/bitbake/lib/bb/asyncrpc/serv.py", line 144, in start_tcp
self.server = self.loop.run_until_complete(server_coro)
File "/usr/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
return future.result()
File "/usr/lib/python3.8/asyncio/streams.py", line 94, in start_server
return await loop.create_server(factory, host, port, **kwds)
File "/usr/lib/python3.8/asyncio/base_events.py", line 1463, in create_server
raise OSError(err.errno, 'error while attempting '
OSError: [Errno 99] error while attempting to bind on address ('::1', 42005, 0, 0): cannot assign requested address

So add the extra logic to make the localhost resolved as expected to make the PR service work especially in docker.

[1] 6a2b23e2 prserv: Replace XML RPC with modern asyncrpc implementation

---
 lib/prserv/serv.py | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
lib/prserv/serv.py +2 -2
index 68db3d3e..0a20b927 100644
@@ -126,7 +126,7 @@ class PRServSingleton(object):
 
     def start(self):
         self.prserv = PRServer(self.dbfile)
-        self.prserv.start_tcp_server(self.host, self.port)
+        self.prserv.start_tcp_server(socket.gethostbyname(self.host), self.port)
         self.process = self.prserv.serve_as_process()
 
         if not self.prserv.address:
@@ -221,7 +221,7 @@ def start_daemon(dbfile, host, port, logfile, read_only=False):
     dbfile = os.path.abspath(dbfile)
     def daemon_main():
         server = PRServer(dbfile, read_only=read_only)
-        server.start_tcp_server(host, port)
+        server.start_tcp_server(ip, port)
         server.serve_forever()
 
     run_as_daemon(daemon_main, pidfile, os.path.abspath(logfile))
Signature · 1 line
2.17.1
#2 lore raw
Quoted 58 lines · On Tue, 31 Aug 2021, [email protected] wrote:

From: Mingli Yu <[email protected]>

After [1] introduced, the PR server doesn't work in docker when use below setting like before.
PRSERV_HOST = "localhost:0"

And it's because the localhost is resolved to an ipv6 address ::1 as
the below bitbake-prserv shows.
bitbake$ bitbake-prserv --start --host=localhost --port=42005
bitbake$ cat prserv.log
Traceback (most recent call last):
File "/OE/nodistro/honister/bitbake/bin/bitbake-prserv", line 55, in <module>
ret = main()
File "/OE/nodistro/honister/bitbake/bin/bitbake-prserv", line 46, in main
ret=prserv.serv.start_daemon(options.dbfile, options.host, options.port,os.path.abspath(options.logfile), options.read_only)
File "/OE/nodistro/honister/bitbake/lib/prserv/serv.py", line 226, in start_daemon
run_as_daemon(daemon_main, pidfile, os.path.abspath(logfile))
File "/OE/nodistro/honister/bitbake/lib/prserv/serv.py", line 202, in run_as_daemon
func()
File "/OE/nodistro/honister/bitbake/lib/prserv/serv.py", line 224, in daemon_main
server.serve_forever()
File "/OE/nodistro/honister/bitbake/lib/bb/asyncrpc/serv.py", line 233, in serve_forever
self.start()
File "/OE/nodistro/honister/bitbake/lib/bb/asyncrpc/serv.py", line 144, in start_tcp
self.server = self.loop.run_until_complete(server_coro)
File "/usr/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
return future.result()
File "/usr/lib/python3.8/asyncio/streams.py", line 94, in start_server
return await loop.create_server(factory, host, port, **kwds)
File "/usr/lib/python3.8/asyncio/base_events.py", line 1463, in create_server
raise OSError(err.errno, 'error while attempting '
OSError: [Errno 99] error while attempting to bind on address ('::1', 42005, 0, 0): cannot assign requested address

So add the extra logic to make the localhost resolved as expected to make the PR service work especially in docker.

[1] 6a2b23e2 prserv: Replace XML RPC with modern asyncrpc implementation

---
 lib/prserv/serv.py | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
lib/prserv/serv.py +1 -1
index 68db3d3e..0a20b927 100644
@@ -126,7 +126,7 @@ class PRServSingleton(object):
 
     def start(self):
         self.prserv = PRServer(self.dbfile)
-        self.prserv.start_tcp_server(self.host, self.port)
+        self.prserv.start_tcp_server(socket.gethostbyname(self.host), self.port)
         self.process = self.prserv.serve_as_process()
 
         if not self.prserv.address:

lib/prserv/serv.py def start_daemon(dbfile, host, port, logfile, read_only=False): · line 227

Quoted 8 lines
lib/prserv/serv.py +1 -1
@@ -221,7 +221,7 @@ def start_daemon(dbfile, host, port, logfile, read_only=False):
     dbfile = os.path.abspath(dbfile)
     def daemon_main():
         server = PRServer(dbfile, read_only=read_only)
-        server.start_tcp_server(host, port)
+        server.start_tcp_server(ip, port)
         server.serve_forever()
 
     run_as_daemon(daemon_main, pidfile, os.path.abspath(logfile))

Richard has picked up Martin's patch in master-next, I think that would remove the need for this one?

Scott

author
#3 lore raw
Quoted 68 lines · Scott Murray

[Please note: This e-mail is from an EXTERNAL e-mail address]

Quoted 64 lines · On Tue, 31 Aug 2021, [email protected] wrote:

From: Mingli Yu <[email protected]>

After [1] introduced, the PR server doesn't work in docker when use below setting like before.
PRSERV_HOST = "localhost:0"

And it's because the localhost is resolved to an ipv6 address ::1 as
the below bitbake-prserv shows.
bitbake$ bitbake-prserv --start --host=localhost --port=42005
bitbake$ cat prserv.log
Traceback (most recent call last):
File "/OE/nodistro/honister/bitbake/bin/bitbake-prserv", line 55, in <module>
ret = main()
File "/OE/nodistro/honister/bitbake/bin/bitbake-prserv", line 46, in main
ret=prserv.serv.start_daemon(options.dbfile, options.host, options.port,os.path.abspath(options.logfile), options.read_only)
File "/OE/nodistro/honister/bitbake/lib/prserv/serv.py", line 226, in start_daemon
run_as_daemon(daemon_main, pidfile, os.path.abspath(logfile))
File "/OE/nodistro/honister/bitbake/lib/prserv/serv.py", line 202, in run_as_daemon
func()
File "/OE/nodistro/honister/bitbake/lib/prserv/serv.py", line 224, in daemon_main
server.serve_forever()
File "/OE/nodistro/honister/bitbake/lib/bb/asyncrpc/serv.py", line 233, in serve_forever
self.start()
File "/OE/nodistro/honister/bitbake/lib/bb/asyncrpc/serv.py", line 144, in start_tcp
self.server = self.loop.run_until_complete(server_coro)
File "/usr/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
return future.result()
File "/usr/lib/python3.8/asyncio/streams.py", line 94, in start_server
return await loop.create_server(factory, host, port, **kwds)
File "/usr/lib/python3.8/asyncio/base_events.py", line 1463, in create_server
raise OSError(err.errno, 'error while attempting '
OSError: [Errno 99] error while attempting to bind on address ('::1', 42005, 0, 0): cannot assign requested address

So add the extra logic to make the localhost resolved as expected to make the PR service work especially in docker.

[1] 6a2b23e2 prserv: Replace XML RPC with modern asyncrpc implementation

---
  lib/prserv/serv.py | 4 ++--
  1 file changed, 2 insertions(+), 2 deletions(-)
lib/prserv/serv.py +2 -2
index 68db3d3e..0a20b927 100644
@@ -126,7 +126,7 @@ class PRServSingleton(object):
 
      def start(self):
          self.prserv = PRServer(self.dbfile)
-        self.prserv.start_tcp_server(self.host, self.port)
+        self.prserv.start_tcp_server(socket.gethostbyname(self.host), self.port)
          self.process = self.prserv.serve_as_process()
 
          if not self.prserv.address:
@@ -221,7 +221,7 @@ def start_daemon(dbfile, host, port, logfile, read_only=False):
      dbfile = os.path.abspath(dbfile)
      def daemon_main():
          server = PRServer(dbfile, read_only=read_only)
-        server.start_tcp_server(host, port)
+        server.start_tcp_server(ip, port)
          server.serve_forever()
 
Quoted 5 lines

lib/prserv/serv.py def start_daemon(dbfile, host, port, logfile, read_only=False): · line 227

Quoted 2 lines
lib/prserv/serv.py +0 -0
      run_as_daemon(daemon_main, pidfile, os.path.abspath(logfile))

Richard has picked up Martin's patch in master-next, I think that would remove the need for this one?

The patch from Martin is just make the error message specific, but localhost still does't work in docker with Martin's patch and this patch is used to make it work in docker.

Thanks,

Quoted 3 lines

Scott

#4 lore raw
Quoted 78 lines · Mingli
Quoted 68 lines · Scott Murray

[Please note: This e-mail is from an EXTERNAL e-mail address]

Quoted 64 lines · On Tue, 31 Aug 2021, [email protected] wrote:

From: Mingli Yu <[email protected]>

After [1] introduced, the PR server doesn't work in docker when use below setting like before.
PRSERV_HOST = "localhost:0"

And it's because the localhost is resolved to an ipv6 address ::1 as
the below bitbake-prserv shows.
bitbake$ bitbake-prserv --start --host=localhost --port=42005
bitbake$ cat prserv.log
Traceback (most recent call last):
File "/OE/nodistro/honister/bitbake/bin/bitbake-prserv", line 55, in <module>
ret = main()
File "/OE/nodistro/honister/bitbake/bin/bitbake-prserv", line 46, in main
ret=prserv.serv.start_daemon(options.dbfile, options.host, options.port,os.path.abspath(options.logfile), options.read_only)
File "/OE/nodistro/honister/bitbake/lib/prserv/serv.py", line 226, in start_daemon
run_as_daemon(daemon_main, pidfile, os.path.abspath(logfile))
File "/OE/nodistro/honister/bitbake/lib/prserv/serv.py", line 202, in run_as_daemon
func()
File "/OE/nodistro/honister/bitbake/lib/prserv/serv.py", line 224, in daemon_main
server.serve_forever()
File "/OE/nodistro/honister/bitbake/lib/bb/asyncrpc/serv.py", line 233, in serve_forever
self.start()
File "/OE/nodistro/honister/bitbake/lib/bb/asyncrpc/serv.py", line 144, in start_tcp
self.server = self.loop.run_until_complete(server_coro)
File "/usr/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
return future.result()
File "/usr/lib/python3.8/asyncio/streams.py", line 94, in start_server
return await loop.create_server(factory, host, port, **kwds)
File "/usr/lib/python3.8/asyncio/base_events.py", line 1463, in create_server
raise OSError(err.errno, 'error while attempting '
OSError: [Errno 99] error while attempting to bind on address ('::1', 42005, 0, 0): cannot assign requested address

So add the extra logic to make the localhost resolved as expected to make the PR service work especially in docker.

[1] 6a2b23e2 prserv: Replace XML RPC with modern asyncrpc implementation

---
  lib/prserv/serv.py | 4 ++--
  1 file changed, 2 insertions(+), 2 deletions(-)
lib/prserv/serv.py +2 -2
index 68db3d3e..0a20b927 100644
@@ -126,7 +126,7 @@ class PRServSingleton(object):
 
      def start(self):
          self.prserv = PRServer(self.dbfile)
-        self.prserv.start_tcp_server(self.host, self.port)
+        self.prserv.start_tcp_server(socket.gethostbyname(self.host), self.port)
          self.process = self.prserv.serve_as_process()
 
          if not self.prserv.address:
@@ -221,7 +221,7 @@ def start_daemon(dbfile, host, port, logfile, read_only=False):
      dbfile = os.path.abspath(dbfile)
      def daemon_main():
          server = PRServer(dbfile, read_only=read_only)
-        server.start_tcp_server(host, port)
+        server.start_tcp_server(ip, port)
          server.serve_forever()
 
Quoted 5 lines

lib/prserv/serv.py def start_daemon(dbfile, host, port, logfile, read_only=False): · line 227

Quoted 2 lines
lib/prserv/serv.py +0 -0
      run_as_daemon(daemon_main, pidfile, os.path.abspath(logfile))

Richard has picked up Martin's patch in master-next, I think that would remove the need for this one?

Quoted 3 lines

The patch from Martin is just make the error message specific, but localhost still does't work in docker with Martin's patch and this patch is used to make it work in docker.

I think that was because IPv6 support had been incorrectly configured for Docker, perhaps that is the same thing happening in your setup?

Quoted 9 lines

Thanks,

Quoted 3 lines

Scott

author
#5 lore raw
Quoted 81 lines · Joshua Watt

[Please note: This e-mail is from an EXTERNAL e-mail address]

Quoted 78 lines · Mingli
Quoted 73 lines · Scott Murray

[Please note: This e-mail is from an EXTERNAL e-mail address]

Quoted 58 lines · On Tue, 31 Aug 2021, [email protected] wrote:

From: Mingli Yu <[email protected]>

After [1] introduced, the PR server doesn't work in docker when use below setting like before.
PRSERV_HOST = "localhost:0"

And it's because the localhost is resolved to an ipv6 address ::1 as
the below bitbake-prserv shows.
bitbake$ bitbake-prserv --start --host=localhost --port=42005
bitbake$ cat prserv.log
Traceback (most recent call last):
File "/OE/nodistro/honister/bitbake/bin/bitbake-prserv", line 55, in <module>
ret = main()
File "/OE/nodistro/honister/bitbake/bin/bitbake-prserv", line 46, in main
ret=prserv.serv.start_daemon(options.dbfile, options.host, options.port,os.path.abspath(options.logfile), options.read_only)
File "/OE/nodistro/honister/bitbake/lib/prserv/serv.py", line 226, in start_daemon
run_as_daemon(daemon_main, pidfile, os.path.abspath(logfile))
File "/OE/nodistro/honister/bitbake/lib/prserv/serv.py", line 202, in run_as_daemon
func()
File "/OE/nodistro/honister/bitbake/lib/prserv/serv.py", line 224, in daemon_main
server.serve_forever()
File "/OE/nodistro/honister/bitbake/lib/bb/asyncrpc/serv.py", line 233, in serve_forever
self.start()
File "/OE/nodistro/honister/bitbake/lib/bb/asyncrpc/serv.py", line 144, in start_tcp
self.server = self.loop.run_until_complete(server_coro)
File "/usr/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
return future.result()
File "/usr/lib/python3.8/asyncio/streams.py", line 94, in start_server
return await loop.create_server(factory, host, port, **kwds)
File "/usr/lib/python3.8/asyncio/base_events.py", line 1463, in create_server
raise OSError(err.errno, 'error while attempting '
OSError: [Errno 99] error while attempting to bind on address ('::1', 42005, 0, 0): cannot assign requested address

So add the extra logic to make the localhost resolved as expected to make the PR service work especially in docker.

[1] 6a2b23e2 prserv: Replace XML RPC with modern asyncrpc implementation

---
   lib/prserv/serv.py | 4 ++--
   1 file changed, 2 insertions(+), 2 deletions(-)
lib/prserv/serv.py +1 -1
index 68db3d3e..0a20b927 100644
@@ -126,7 +126,7 @@ class PRServSingleton(object):
 
       def start(self):
           self.prserv = PRServer(self.dbfile)
-        self.prserv.start_tcp_server(self.host, self.port)
+        self.prserv.start_tcp_server(socket.gethostbyname(self.host), self.port)
           self.process = self.prserv.serve_as_process()
 
           if not self.prserv.address:

lib/prserv/serv.py def start_daemon(dbfile, host, port, logfile, read_only=False): · line 227

Quoted 8 lines
lib/prserv/serv.py +1 -1
@@ -221,7 +221,7 @@ def start_daemon(dbfile, host, port, logfile, read_only=False):
       dbfile = os.path.abspath(dbfile)
       def daemon_main():
           server = PRServer(dbfile, read_only=read_only)
-        server.start_tcp_server(host, port)
+        server.start_tcp_server(ip, port)
           server.serve_forever()
 
       run_as_daemon(daemon_main, pidfile, os.path.abspath(logfile))

Richard has picked up Martin's patch in master-next, I think that would remove the need for this one?

Quoted 6 lines
Quoted 3 lines

The patch from Martin is just make the error message specific, but localhost still does't work in docker with Martin's patch and this patch is used to make it work in docker.

I think that was because IPv6 support had been incorrectly configured for Docker, perhaps that is the same thing happening in your setup?

In the same docker, the PR service works with PRSERV_HOST = "localhost:0" before, but it begins not work recently after some code restructure for prserv part.

BTW, the ipv6 is disabled in the docker of my env.

Thanks,

Quoted 10 lines
Quoted 10 lines

Thanks,

Quoted 3 lines

Scott

#6 lore raw
Quoted 11 lines
Quoted 4 lines

I think that was because IPv6 support had been incorrectly configured for Docker, perhaps that is the same thing happening in your setup?

In the same docker, the PR service works with PRSERV_HOST = "localhost:0" before, but it begins not work recently after some code restructure for prserv part.

BTW, the ipv6 is disabled in the docker of my env.

Check your /etc/hosts

There is ::1 localhost localhost-ip6 even when ipv6 is disabled in docker's deamon.json.

And that makes bitbake (asyncio) to try binding to ::1 instead of 127.0.0.1, if you remove "localhost" from ::1 in /etc/hosts in running container, then it works again OK.

My patch allows you to use 127.0.0.1:0 explicitly and still keep the same behavior (as autostarted prserv with random port) as when localhost:0 is being used (and also for consistency).

Your patch is IMHO good addition to that (that the localhost is resolved in python and then people don't need to use 127.0.0.1 explicitly).

Regards,