Skip to contentSkip to conversation

python/machine: Fix AF_UNIX path too long on macOS

PATCH RESEND Superseded Peter Delevoryas 3 participants 9 messages Jul 5, 2022 – Jul 12, 2022

v1 of 5 Superseded Latest is v5 →

lore
Share & tools
lore
#0 lore raw

PATCH RESEND 1/1 python/machine: Fix AF_UNIX path too long on macOS

I noticed that I can't run any avocado tests on macOS because the QMP unix socket path is too long:

$ ./configure --target-list=arm-softmmu
$ make
$ make check-avocado AVOCADO_TESTS=tests/avocado/boot_linux_console.py
changing dir to build for /Library/Developer/CommandLineTools/usr/bin/make "check-avocado"...
  GIT     ui/keycodemapdb meson tests/fp/berkeley-testfloat-3 tests/fp/berkeley-softfloat-3 dtc slirp
  AVOCADO tests/avocado
Fetching asset from tests/avocado/boot_linux_console.py:BootLinuxConsole.test_arm_virt
Fetching asset from tests/avocado/boot_linux_console.py:BootLinuxConsole.test_arm_emcraft_sf2
Fetching asset from tests/avocado/boot_linux_console.py:BootLinuxConsole.test_arm_emcraft_sf2
Fetching asset from tests/avocado/boot_linux_console.py:BootLinuxConsole.test_arm_raspi2_initrd
Fetching asset from tests/avocado/boot_linux_console.py:BootLinuxConsole.test_arm_raspi2_initrd
Fetching asset from tests/avocado/boot_linux_console.py:BootLinuxConsole.test_arm_exynos4210_initrd
Fetching asset from tests/avocado/boot_linux_console.py:BootLinuxConsole.test_arm_exynos4210_initrd
Fetching asset from tests/avocado/boot_linux_console.py:BootLinuxConsole.test_arm_cubieboard_initrd
Fetching asset from tests/avocado/boot_linux_console.py:BootLinuxConsole.test_arm_cubieboard_initrd
Fetching asset from tests/avocado/boot_linux_console.py:BootLinuxConsole.test_arm_cubieboard_sata
Fetching asset from tests/avocado/boot_linux_console.py:BootLinuxConsole.test_arm_cubieboard_sata
Fetching asset from tests/avocado/boot_linux_console.py:BootLinuxConsole.test_arm_quanta_gsj
Fetching asset from tests/avocado/boot_linux_console.py:BootLinuxConsole.test_arm_quanta_gsj_initrd
Fetching asset from tests/avocado/boot_linux_console.py:BootLinuxConsole.test_arm_quanta_gsj_initrd
Fetching asset from tests/avocado/boot_linux_console.py:BootLinuxConsole.test_arm_quanta_gsj_initrd
Fetching asset from tests/avocado/boot_linux_console.py:BootLinuxConsole.test_arm_orangepi
Fetching asset from tests/avocado/boot_linux_console.py:BootLinuxConsole.test_arm_orangepi_initrd
Fetching asset from tests/avocado/boot_linux_console.py:BootLinuxConsole.test_arm_orangepi_initrd
Fetching asset from tests/avocado/boot_linux_console.py:BootLinuxConsole.test_arm_orangepi_sd
Fetching asset from tests/avocado/boot_linux_console.py:BootLinuxConsole.test_arm_orangepi_sd
Fetching asset from tests/avocado/boot_linux_console.py:BootLinuxConsole.test_arm_orangepi_bionic_20_08
Fetching asset from tests/avocado/boot_linux_console.py:BootLinuxConsole.test_arm_orangepi_uboot_netbsd9
Fetching asset from tests/avocado/boot_linux_console.py:BootLinuxConsole.test_arm_orangepi_uboot_netbsd9
Fetching asset from tests/avocado/boot_linux_console.py:BootLinuxConsole.test_arm_ast2600_debian
JOB ID     : 84bd26125345c6f12c92d118e98acde6aacfea57
JOB LOG    : /Users/pdel/qemu/build/tests/results/job-2022-07-02T11.24-84bd261/job.log
 (01/16) tests/avocado/boot_linux_console.py:BootLinuxConsole.test_arm_virt: ERROR: ConnectError: Failed to establish connection: AF_UNIX path too lo
ng\n    Command: \n     Output: None\n (0.11 s)
Interrupting job (failfast).
RESULTS    : PASS 0 | ERROR 1 | FAIL 0 | SKIP 15 | WARN 0 | INTERRUPT 0 | CANCEL 0

The job log shows a backtrace that points to the QMP monitor's unix socket path:

2022-07-02 11:24:05,906 protocol         L0554 DEBUG| Awaiting connection on /var/folders/d7/rz20f6hd709c1ty8f6_6y_z40000gn/T/avo_qemu_sock_uh3w_dgc/qemu-37331-10bacf110-monitor.sock ...
2022-07-02 11:24:05,907 protocol         L0426 ERROR| Failed to establish connection: OSError: AF_UNIX path too long

I think the path limit for unix sockets on macOS might be 104 [1]

/*
* [XSI] Definitions for UNIX IPC domain.
*/
struct sockaddr_un {

    unsigned char   sun_len;        /* sockaddr len including null */
    sa_family_t     sun_family;     /* [XSI] AF_UNIX */
    char            sun_path[104];  /* [XSI] path name (gag) */

};

The path we're using is exactly 105 characters:

$ python
Python 2.7.10 (default, Jan 19 2016, 22:24:01)
[GCC 4.2.1 Compatible Apple LLVM 7.0.2 (clang-700.1.81)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
Quoted 1 line
Quoted 1 line
Quoted 1 line

len('/var/folders/d7/rz20f6hd709c1ty8f6_6y_z40000gn/T/avo_qemu_sock_uh3w_dgc/qemu-37331-10bacf110-monitor.sock')

105

In this change I removed some of the unnecessary elements of the path, to keep us under the limit. This seemed like the simplest thing to do, and I imagine the extra human-readable elements were just added to make it easier to identify left-over sockets or something else related to debugging.

[1]: /Library/Developer/CommandLineTools/SDKs/MacOSX12.3.sdk/usr/include/sys/un.h

---
 python/qemu/machine/machine.py | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
python/qemu/machine/machine.py +2 -2
index 37191f433b..93451774e3 100644
@@ -157,7 +157,7 @@ def __init__(self,
         self._wrapper = wrapper
         self._qmp_timer = qmp_timer
 
-        self._name = name or f"qemu-{os.getpid()}-{id(self):02x}"
+        self._name = name or f"{os.getpid()}{id(self):02x}"
         self._temp_dir: Optional[str] = None
         self._base_temp_dir = base_temp_dir
         self._sock_dir = sock_dir
@@ -167,7 +167,7 @@ def __init__(self,
             self._monitor_address = monitor_address
         else:
             self._monitor_address = os.path.join(
-                self.sock_dir, f"{self._name}-monitor.sock"
+                self.sock_dir, f"{self._name}.sock"
             )
 
         self._console_log_path = console_log
Signature · 1 line
2.37.0
#1 lore raw
Quoted 5 lines · Peter Delevoryas

I noticed that I can't run any avocado tests on macOS because the QMP unix socket path is too long:

I think the path limit for unix sockets on macOS might be 104 [1]

All platforms have a very limited path limit, so it isn't really a macOS specific problem, rather....

Quoted 8 lines

/*
* [XSI] Definitions for UNIX IPC domain.
*/
struct sockaddr_un {

    unsigned char   sun_len;        /* sockaddr len including null */
    sa_family_t     sun_family;     /* [XSI] AF_UNIX */
    char            sun_path[104];  /* [XSI] path name (gag) */
Quoted 9 lines

};

The path we're using is exactly 105 characters:

$ python
Python 2.7.10 (default, Jan 19 2016, 22:24:01)
[GCC 4.2.1 Compatible Apple LLVM 7.0.2 (clang-700.1.81)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
Quoted 1 line
Quoted 1 line
Quoted 1 line

len('/var/folders/d7/rz20f6hd709c1ty8f6_6y_z40000gn/T/avo_qemu_sock_uh3w_dgc/qemu-37331-10bacf110-monitor.sock')

It is a problem related to where the test suite is creating the paths.

/var/folders/d7/rz20f6hd709c1ty8f6_6y_z40000gn/T/avo_qemu_sock_uh3w_dgc/

is way too deep a directory location.

It seems we just create this location using 'tempfile.TemporyDirectory' to get a standard tmp dir.

Do you know why python is choosing

/var/folders/d7/rz20f6hd709c1ty8f6_6y_z40000gn/T/

as the temp dir ? Is that a standard location on macOS or is it from some env variable you have set ?

python/qemu/machine/machine.py def __init__(self, · line 160

Quoted 10 lines
python/qemu/machine/machine.py +1 -1
index 37191f433b..93451774e3 100644
@@ -157,7 +157,7 @@ def __init__(self,
         self._wrapper = wrapper
         self._qmp_timer = qmp_timer
 
-        self._name = name or f"qemu-{os.getpid()}-{id(self):02x}"
+        self._name = name or f"{os.getpid()}{id(self):02x}"

I don't think this is the right fix really, because IMHO the problem is the hugely long path, rather than the final socket name.

That said, there is redundancy in the path - avocado is passing in a dierctory created using 'tempfile.TemporyDirectory' so there is no reason why we need to add more entropy via the POD and the 'id(self)' hex string.

IMHO avocado should pass in the 'name' parameter explicitly, using a plain name and thus get a shorter string.

Quoted 16 lines
python/qemu/machine/machine.py +1 -1
         self._temp_dir: Optional[str] = None
         self._base_temp_dir = base_temp_dir
         self._sock_dir = sock_dir
@@ -167,7 +167,7 @@ def __init__(self,
             self._monitor_address = monitor_address
         else:
             self._monitor_address = os.path.join(
-                self.sock_dir, f"{self._name}-monitor.sock"
+                self.sock_dir, f"{self._name}.sock"
             )
 
         self._console_log_path = console_log
Signature · 1 line
2.37.0

With regards,
Daniel

Signature · 3 lines
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|
#2 lore raw
Quoted 26 lines · Daniel P. Berrangé
Quoted 5 lines · Peter Delevoryas

I noticed that I can't run any avocado tests on macOS because the QMP unix socket path is too long:

I think the path limit for unix sockets on macOS might be 104 [1]

All platforms have a very limited path limit, so it isn't really a macOS specific problem, rather....

Quoted 16 lines

/*
* [XSI] Definitions for UNIX IPC domain.
*/
struct sockaddr_un {

    unsigned char   sun_len;        /* sockaddr len including null */
    sa_family_t     sun_family;     /* [XSI] AF_UNIX */
    char            sun_path[104];  /* [XSI] path name (gag) */

};

The path we're using is exactly 105 characters:

$ python
Python 2.7.10 (default, Jan 19 2016, 22:24:01)
[GCC 4.2.1 Compatible Apple LLVM 7.0.2 (clang-700.1.81)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
Quoted 8 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

len('/var/folders/d7/rz20f6hd709c1ty8f6_6y_z40000gn/T/avo_qemu_sock_uh3w_dgc/qemu-37331-10bacf110-monitor.sock')

It is a problem related to where the test suite is creating the paths.

/var/folders/d7/rz20f6hd709c1ty8f6_6y_z40000gn/T/avo_qemu_sock_uh3w_dgc/

is way too deep a directory location.

That's a good point.

Quoted 10 lines

It seems we just create this location using 'tempfile.TemporyDirectory' to get a standard tmp dir.

Do you know why python is choosing

/var/folders/d7/rz20f6hd709c1ty8f6_6y_z40000gn/T/

as the temp dir ? Is that a standard location on macOS or is it from some env variable you have set ?

Hmmm yeah it is odd, I'm not really sure why it's created there or if standard glibc tmpfile creation goes there too, I'll go experiment and report back. And yeah, maybe I'll double check any environment variables or other things.

The macOS system I use happens to be a Facebook work laptop, which could also be related now that I think about it.

Quoted 9 lines
Quoted 9 lines
python/qemu/machine/machine.py +0 -0
index 37191f433b..93451774e3 100644
@@ -157,7 +157,7 @@ def __init__(self,
         self._wrapper = wrapper
         self._qmp_timer = qmp_timer
 
Quoted 5 lines

python/qemu/machine/machine.py def __init__(self, · line 160

Quoted 2 lines
python/qemu/machine/machine.py +1 -1
-        self._name = name or f"qemu-{os.getpid()}-{id(self):02x}"
+        self._name = name or f"{os.getpid()}{id(self):02x}"

I don't think this is the right fix really, because IMHO the problem is the hugely long path, rather than the final socket name.

True, yeah let me try to investigate the directory placement.

Quoted 5 lines

That said, there is redundancy in the path - avocado is passing in a dierctory created using 'tempfile.TemporyDirectory' so there is no reason why we need to add more entropy via the POD and the 'id(self)' hex string.

Oh good point, I hadn't thought about that.

Quoted 3 lines

IMHO avocado should pass in the 'name' parameter explicitly, using a plain name and thus get a shorter string.

I see, yeah that makes sense. Maybe I can include a couple patches for this, one fixing the directory location, and one refactoring the temporary file name template (If I'm understanding your suggestion correctly).

Thanks for the review! I really appreciate it.
Peter

Quoted 25 lines
Quoted 17 lines
python/qemu/machine/machine.py +1 -1
         self._temp_dir: Optional[str] = None
         self._base_temp_dir = base_temp_dir
         self._sock_dir = sock_dir
@@ -167,7 +167,7 @@ def __init__(self,
             self._monitor_address = monitor_address
         else:
             self._monitor_address = os.path.join(
-                self.sock_dir, f"{self._name}-monitor.sock"
+                self.sock_dir, f"{self._name}.sock"
             )
 
         self._console_log_path = console_log
Signature · 1 line
2.37.0

With regards,
Daniel

Signature · 3 lines
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|
#3 lore raw
Quoted 49 lines · Peter Delevoryas
Quoted 26 lines · Daniel P. Berrangé
Quoted 5 lines · Peter Delevoryas

I noticed that I can't run any avocado tests on macOS because the QMP unix socket path is too long:

I think the path limit for unix sockets on macOS might be 104 [1]

All platforms have a very limited path limit, so it isn't really a macOS specific problem, rather....

Quoted 16 lines

/*
* [XSI] Definitions for UNIX IPC domain.
*/
struct sockaddr_un {

    unsigned char   sun_len;        /* sockaddr len including null */
    sa_family_t     sun_family;     /* [XSI] AF_UNIX */
    char            sun_path[104];  /* [XSI] path name (gag) */

};

The path we're using is exactly 105 characters:

$ python
Python 2.7.10 (default, Jan 19 2016, 22:24:01)
[GCC 4.2.1 Compatible Apple LLVM 7.0.2 (clang-700.1.81)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
Quoted 8 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

len('/var/folders/d7/rz20f6hd709c1ty8f6_6y_z40000gn/T/avo_qemu_sock_uh3w_dgc/qemu-37331-10bacf110-monitor.sock')

It is a problem related to where the test suite is creating the paths.

/var/folders/d7/rz20f6hd709c1ty8f6_6y_z40000gn/T/avo_qemu_sock_uh3w_dgc/

is way too deep a directory location.

That's a good point.

Quoted 10 lines

It seems we just create this location using 'tempfile.TemporyDirectory' to get a standard tmp dir.

Do you know why python is choosing

/var/folders/d7/rz20f6hd709c1ty8f6_6y_z40000gn/T/

as the temp dir ? Is that a standard location on macOS or is it from some env variable you have set ?

Quoted 7 lines

Hmmm yeah it is odd, I'm not really sure why it's created there or if standard glibc tmpfile creation goes there too, I'll go experiment and report back. And yeah, maybe I'll double check any environment variables or other things.

The macOS system I use happens to be a Facebook work laptop, which could also be related now that I think about it.

Hmmm yeah looks like this is because my TMPDIR is weird.

$ echo $TMPDIR
/var/folders/d7/rz20f6hd709c1ty8f6_6y_z40000gn/T/

I didn't think to check this cause I wasn't familiar with TMPDIR. 🤷

Thanks for responding, I'll just use TMPDIR=/tmp for now. It's probably something wrong with the Facebook development environment.

Peter

Quoted 63 lines
Quoted 10 lines
Quoted 10 lines
python/qemu/machine/machine.py +0 -0
index 37191f433b..93451774e3 100644
@@ -157,7 +157,7 @@ def __init__(self,
         self._wrapper = wrapper
         self._qmp_timer = qmp_timer
 
Quoted 5 lines

python/qemu/machine/machine.py def __init__(self, · line 160

Quoted 2 lines
python/qemu/machine/machine.py +1 -1
-        self._name = name or f"qemu-{os.getpid()}-{id(self):02x}"
+        self._name = name or f"{os.getpid()}{id(self):02x}"

I don't think this is the right fix really, because IMHO the problem is the hugely long path, rather than the final socket name.

True, yeah let me try to investigate the directory placement.

Quoted 5 lines

That said, there is redundancy in the path - avocado is passing in a dierctory created using 'tempfile.TemporyDirectory' so there is no reason why we need to add more entropy via the POD and the 'id(self)' hex string.

Oh good point, I hadn't thought about that.

Quoted 3 lines

IMHO avocado should pass in the 'name' parameter explicitly, using a plain name and thus get a shorter string.

I see, yeah that makes sense. Maybe I can include a couple patches for this, one fixing the directory location, and one refactoring the temporary file name template (If I'm understanding your suggestion correctly).

Thanks for the review! I really appreciate it.
Peter

Quoted 25 lines
Quoted 17 lines
python/qemu/machine/machine.py +1 -1
         self._temp_dir: Optional[str] = None
         self._base_temp_dir = base_temp_dir
         self._sock_dir = sock_dir
@@ -167,7 +167,7 @@ def __init__(self,
             self._monitor_address = monitor_address
         else:
             self._monitor_address = os.path.join(
-                self.sock_dir, f"{self._name}-monitor.sock"
+                self.sock_dir, f"{self._name}.sock"
             )
 
         self._console_log_path = console_log
Signature · 1 line
2.37.0

With regards,
Daniel

Signature · 3 lines
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|
#4 lore raw
Quoted 63 lines · Peter Delevoryas
Quoted 49 lines · Peter Delevoryas
Quoted 26 lines · Daniel P. Berrangé
Quoted 5 lines · Peter Delevoryas

I noticed that I can't run any avocado tests on macOS because the QMP unix socket path is too long:

I think the path limit for unix sockets on macOS might be 104 [1]

All platforms have a very limited path limit, so it isn't really a macOS specific problem, rather....

Quoted 16 lines

/*
* [XSI] Definitions for UNIX IPC domain.
*/
struct sockaddr_un {

    unsigned char   sun_len;        /* sockaddr len including null */
    sa_family_t     sun_family;     /* [XSI] AF_UNIX */
    char            sun_path[104];  /* [XSI] path name (gag) */

};

The path we're using is exactly 105 characters:

$ python
Python 2.7.10 (default, Jan 19 2016, 22:24:01)
[GCC 4.2.1 Compatible Apple LLVM 7.0.2 (clang-700.1.81)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
Quoted 8 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

len('/var/folders/d7/rz20f6hd709c1ty8f6_6y_z40000gn/T/avo_qemu_sock_uh3w_dgc/qemu-37331-10bacf110-monitor.sock')

It is a problem related to where the test suite is creating the paths.

/var/folders/d7/rz20f6hd709c1ty8f6_6y_z40000gn/T/avo_qemu_sock_uh3w_dgc/

is way too deep a directory location.

That's a good point.

Quoted 10 lines

It seems we just create this location using 'tempfile.TemporyDirectory' to get a standard tmp dir.

Do you know why python is choosing

/var/folders/d7/rz20f6hd709c1ty8f6_6y_z40000gn/T/

as the temp dir ? Is that a standard location on macOS or is it from some env variable you have set ?

Quoted 7 lines

Hmmm yeah it is odd, I'm not really sure why it's created there or if standard glibc tmpfile creation goes there too, I'll go experiment and report back. And yeah, maybe I'll double check any environment variables or other things.

The macOS system I use happens to be a Facebook work laptop, which could also be related now that I think about it.

Hmmm yeah looks like this is because my TMPDIR is weird.

$ echo $TMPDIR
/var/folders/d7/rz20f6hd709c1ty8f6_6y_z40000gn/T/
Quoted 6 lines

I didn't think to check this cause I wasn't familiar with TMPDIR. 🤷

Thanks for responding, I'll just use TMPDIR=/tmp for now. It's probably something wrong with the Facebook development environment.

Peter

Update: Actually, this might not be a Facebook-work-laptop specific thing. I asked my non-engineer friend to print out $TMPDIR on his macbook and he got the same thing.

https://apple.stackexchange.com/questions/353832/why-is-mac-osx-temp-directory-in-weird-path

I guess this person suggests it's just to separate the permissions for each user's /tmp directory, for better isolation.

I'll resubmit this patch with the suggestions you had, because perhaps this is actually affecting other macOS users too.

Quoted 65 lines
Quoted 64 lines
Quoted 11 lines
Quoted 11 lines
python/qemu/machine/machine.py +0 -0
index 37191f433b..93451774e3 100644
@@ -157,7 +157,7 @@ def __init__(self,
         self._wrapper = wrapper
         self._qmp_timer = qmp_timer
 
Quoted 5 lines

python/qemu/machine/machine.py def __init__(self, · line 160

Quoted 2 lines
python/qemu/machine/machine.py +1 -1
-        self._name = name or f"qemu-{os.getpid()}-{id(self):02x}"
+        self._name = name or f"{os.getpid()}{id(self):02x}"

I don't think this is the right fix really, because IMHO the problem is the hugely long path, rather than the final socket name.

True, yeah let me try to investigate the directory placement.

Quoted 5 lines

That said, there is redundancy in the path - avocado is passing in a dierctory created using 'tempfile.TemporyDirectory' so there is no reason why we need to add more entropy via the POD and the 'id(self)' hex string.

Oh good point, I hadn't thought about that.

Quoted 3 lines

IMHO avocado should pass in the 'name' parameter explicitly, using a plain name and thus get a shorter string.

I see, yeah that makes sense. Maybe I can include a couple patches for this, one fixing the directory location, and one refactoring the temporary file name template (If I'm understanding your suggestion correctly).

Thanks for the review! I really appreciate it.
Peter

Quoted 25 lines
Quoted 17 lines
python/qemu/machine/machine.py +1 -1
         self._temp_dir: Optional[str] = None
         self._base_temp_dir = base_temp_dir
         self._sock_dir = sock_dir
@@ -167,7 +167,7 @@ def __init__(self,
             self._monitor_address = monitor_address
         else:
             self._monitor_address = os.path.join(
-                self.sock_dir, f"{self._name}-monitor.sock"
+                self.sock_dir, f"{self._name}.sock"
             )
 
         self._console_log_path = console_log
Signature · 1 line
2.37.0

With regards,
Daniel

Signature · 3 lines
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|
#5 lore raw
Quoted 76 lines · Peter Delevoryas
Quoted 63 lines · Peter Delevoryas
Quoted 49 lines · Peter Delevoryas
Quoted 26 lines · Daniel P. Berrangé
Quoted 5 lines · Peter Delevoryas

I noticed that I can't run any avocado tests on macOS because the QMP unix socket path is too long:

I think the path limit for unix sockets on macOS might be 104 [1]

All platforms have a very limited path limit, so it isn't really a macOS specific problem, rather....

Quoted 16 lines

/*
* [XSI] Definitions for UNIX IPC domain.
*/
struct sockaddr_un {

    unsigned char   sun_len;        /* sockaddr len including null */
    sa_family_t     sun_family;     /* [XSI] AF_UNIX */
    char            sun_path[104];  /* [XSI] path name (gag) */

};

The path we're using is exactly 105 characters:

$ python
Python 2.7.10 (default, Jan 19 2016, 22:24:01)
[GCC 4.2.1 Compatible Apple LLVM 7.0.2 (clang-700.1.81)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
Quoted 8 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

len('/var/folders/d7/rz20f6hd709c1ty8f6_6y_z40000gn/T/avo_qemu_sock_uh3w_dgc/qemu-37331-10bacf110-monitor.sock')

It is a problem related to where the test suite is creating the paths.

/var/folders/d7/rz20f6hd709c1ty8f6_6y_z40000gn/T/avo_qemu_sock_uh3w_dgc/

is way too deep a directory location.

That's a good point.

Quoted 10 lines

It seems we just create this location using 'tempfile.TemporyDirectory' to get a standard tmp dir.

Do you know why python is choosing

/var/folders/d7/rz20f6hd709c1ty8f6_6y_z40000gn/T/

as the temp dir ? Is that a standard location on macOS or is it from some env variable you have set ?

Quoted 7 lines

Hmmm yeah it is odd, I'm not really sure why it's created there or if standard glibc tmpfile creation goes there too, I'll go experiment and report back. And yeah, maybe I'll double check any environment variables or other things.

The macOS system I use happens to be a Facebook work laptop, which could also be related now that I think about it.

Hmmm yeah looks like this is because my TMPDIR is weird.

$ echo $TMPDIR
/var/folders/d7/rz20f6hd709c1ty8f6_6y_z40000gn/T/
Quoted 6 lines

I didn't think to check this cause I wasn't familiar with TMPDIR. 🤷

Thanks for responding, I'll just use TMPDIR=/tmp for now. It's probably something wrong with the Facebook development environment.

Peter

Update: Actually, this might not be a Facebook-work-laptop specific thing. I asked my non-engineer friend to print out $TMPDIR on his macbook and he got the same thing.

Quoted 45 lines

https://apple.stackexchange.com/questions/353832/why-is-mac-osx-temp-directory-in-weird-path

I guess this person suggests it's just to separate the permissions for each user's /tmp directory, for better isolation.

I'll resubmit this patch with the suggestions you had, because perhaps this is actually affecting other macOS users too.

Quoted 37 lines
Quoted 37 lines
Quoted 11 lines
Quoted 11 lines
python/qemu/machine/machine.py +0 -0
index 37191f433b..93451774e3 100644
@@ -157,7 +157,7 @@ def __init__(self,
         self._wrapper = wrapper
         self._qmp_timer = qmp_timer
 
Quoted 5 lines

python/qemu/machine/machine.py def __init__(self, · line 160

Quoted 2 lines
python/qemu/machine/machine.py +1 -1
-        self._name = name or f"qemu-{os.getpid()}-{id(self):02x}"
+        self._name = name or f"{os.getpid()}{id(self):02x}"

I don't think this is the right fix really, because IMHO the problem is the hugely long path, rather than the final socket name.

True, yeah let me try to investigate the directory placement.

Quoted 5 lines

That said, there is redundancy in the path - avocado is passing in a dierctory created using 'tempfile.TemporyDirectory' so there is no reason why we need to add more entropy via the POD and the 'id(self)' hex string.

Oh good point, I hadn't thought about that.

Quoted 3 lines

IMHO avocado should pass in the 'name' parameter explicitly, using a plain name and thus get a shorter string.

I see, yeah that makes sense. Maybe I can include a couple patches for this, one fixing the directory location, and one refactoring the temporary file name template (If I'm understanding your suggestion correctly).

Thanks for the review! I really appreciate it.
Peter

I agree with Dan: I believe the correct solution here is for Avocado to provide its own less redundant name; but the default name that machine.py provides is not *that* long and provides adequate protection against collisions with multiple instances of the VM utility within a single python process. If Avocado is creating its own directories that guard against that redundancy, Avocado should provide a shortened name for the VM.

Note that the QEMUMachine process also provides a sock_dir parameter that was introduced for precisely this reason; it should be possible to instruct the avocado tests to use a shorter path for sock_dir.

I'm not clear on what the best "just works" solution will be when certain operating environments choose a tmp dir that's quite long to begin with; maybe we need a different default sockfile naming strategy that avoids the instance collision problem in machine.py, too. Ideas?

--js

Quoted 29 lines
Quoted 28 lines
Quoted 27 lines
Quoted 26 lines
Quoted 15 lines
python/qemu/machine/machine.py +1 -2
         self._temp_dir: Optional[str] = None
         self._base_temp_dir = base_temp_dir
         self._sock_dir = sock_dir
@@ -167,7 +167,7 @@ def __init__(self,
             self._monitor_address = monitor_address
         else:
             self._monitor_address = os.path.join(
-                self.sock_dir, f"{self._name}-monitor.sock"
+                self.sock_dir, f"{self._name}.sock"
             )
 
         self._console_log_path = console_log
--
Quoted 3 lines

2.37.0

With regards,
Daniel
--

|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|
#6 lore raw
Quoted 123 lines · John Snow
Quoted 76 lines · Peter Delevoryas
Quoted 63 lines · Peter Delevoryas
Quoted 49 lines · Peter Delevoryas
Quoted 26 lines · Daniel P. Berrangé
Quoted 5 lines · Peter Delevoryas

I noticed that I can't run any avocado tests on macOS because the QMP unix socket path is too long:

I think the path limit for unix sockets on macOS might be 104 [1]

All platforms have a very limited path limit, so it isn't really a macOS specific problem, rather....

Quoted 16 lines

/*
* [XSI] Definitions for UNIX IPC domain.
*/
struct sockaddr_un {

    unsigned char   sun_len;        /* sockaddr len including null */
    sa_family_t     sun_family;     /* [XSI] AF_UNIX */
    char            sun_path[104];  /* [XSI] path name (gag) */

};

The path we're using is exactly 105 characters:

$ python
Python 2.7.10 (default, Jan 19 2016, 22:24:01)
[GCC 4.2.1 Compatible Apple LLVM 7.0.2 (clang-700.1.81)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
Quoted 8 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

len('/var/folders/d7/rz20f6hd709c1ty8f6_6y_z40000gn/T/avo_qemu_sock_uh3w_dgc/qemu-37331-10bacf110-monitor.sock')

It is a problem related to where the test suite is creating the paths.

/var/folders/d7/rz20f6hd709c1ty8f6_6y_z40000gn/T/avo_qemu_sock_uh3w_dgc/

is way too deep a directory location.

That's a good point.

Quoted 10 lines

It seems we just create this location using 'tempfile.TemporyDirectory' to get a standard tmp dir.

Do you know why python is choosing

/var/folders/d7/rz20f6hd709c1ty8f6_6y_z40000gn/T/

as the temp dir ? Is that a standard location on macOS or is it from some env variable you have set ?

Quoted 7 lines

Hmmm yeah it is odd, I'm not really sure why it's created there or if standard glibc tmpfile creation goes there too, I'll go experiment and report back. And yeah, maybe I'll double check any environment variables or other things.

The macOS system I use happens to be a Facebook work laptop, which could also be related now that I think about it.

Hmmm yeah looks like this is because my TMPDIR is weird.

$ echo $TMPDIR
/var/folders/d7/rz20f6hd709c1ty8f6_6y_z40000gn/T/
Quoted 6 lines

I didn't think to check this cause I wasn't familiar with TMPDIR. 🤷

Thanks for responding, I'll just use TMPDIR=/tmp for now. It's probably something wrong with the Facebook development environment.

Peter

Update: Actually, this might not be a Facebook-work-laptop specific thing. I asked my non-engineer friend to print out $TMPDIR on his macbook and he got the same thing.

Quoted 45 lines

https://apple.stackexchange.com/questions/353832/why-is-mac-osx-temp-directory-in-weird-path

I guess this person suggests it's just to separate the permissions for each user's /tmp directory, for better isolation.

I'll resubmit this patch with the suggestions you had, because perhaps this is actually affecting other macOS users too.

Quoted 37 lines
Quoted 37 lines
Quoted 11 lines
Quoted 11 lines
python/qemu/machine/machine.py +0 -0
index 37191f433b..93451774e3 100644
@@ -157,7 +157,7 @@ def __init__(self,
         self._wrapper = wrapper
         self._qmp_timer = qmp_timer
 
Quoted 5 lines

python/qemu/machine/machine.py def __init__(self, · line 160

Quoted 2 lines
python/qemu/machine/machine.py +1 -1
-        self._name = name or f"qemu-{os.getpid()}-{id(self):02x}"
+        self._name = name or f"{os.getpid()}{id(self):02x}"

I don't think this is the right fix really, because IMHO the problem is the hugely long path, rather than the final socket name.

True, yeah let me try to investigate the directory placement.

Quoted 5 lines

That said, there is redundancy in the path - avocado is passing in a dierctory created using 'tempfile.TemporyDirectory' so there is no reason why we need to add more entropy via the POD and the 'id(self)' hex string.

Oh good point, I hadn't thought about that.

Quoted 3 lines

IMHO avocado should pass in the 'name' parameter explicitly, using a plain name and thus get a shorter string.

I see, yeah that makes sense. Maybe I can include a couple patches for this, one fixing the directory location, and one refactoring the temporary file name template (If I'm understanding your suggestion correctly).

Thanks for the review! I really appreciate it.
Peter

Quoted 7 lines

I agree with Dan: I believe the correct solution here is for Avocado to provide its own less redundant name; but the default name that machine.py provides is not *that* long and provides adequate protection against collisions with multiple instances of the VM utility within a single python process. If Avocado is creating its own directories that guard against that redundancy, Avocado should provide a shortened name for the VM.

Hmmm, I see.

Quoted 9 lines

Note that the QEMUMachine process also provides a sock_dir parameter that was introduced for precisely this reason; it should be possible to instruct the avocado tests to use a shorter path for sock_dir.

I'm not clear on what the best "just works" solution will be when certain operating environments choose a tmp dir that's quite long to begin with; maybe we need a different default sockfile naming strategy that avoids the instance collision problem in machine.py, too. Ideas?

I guess I don't really understand why QEMU is attempting to create a unique path instead of just using mktemp(), right? This seems like something that the OS can provide. If we want some semblance of meaning to the filename, we can provide a short template, right?

-- Peter

Quoted 33 lines

--js

Quoted 29 lines
Quoted 28 lines
Quoted 27 lines
Quoted 26 lines
Quoted 15 lines
python/qemu/machine/machine.py +1 -2
         self._temp_dir: Optional[str] = None
         self._base_temp_dir = base_temp_dir
         self._sock_dir = sock_dir
@@ -167,7 +167,7 @@ def __init__(self,
             self._monitor_address = monitor_address
         else:
             self._monitor_address = os.path.join(
-                self.sock_dir, f"{self._name}-monitor.sock"
+                self.sock_dir, f"{self._name}.sock"
             )
 
         self._console_log_path = console_log
--
Quoted 3 lines

2.37.0

With regards,
Daniel
--

|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|
#7 lore raw
Quoted 167 lines · Peter Delevoryas
Quoted 7 lines · John Snow
Quoted 6 lines · Peter Delevoryas
Quoted 4 lines · Peter Delevoryas
Quoted 3 lines · Peter Delevoryas
Quoted 2 lines · Daniel P. Berrangé
Quoted 1 line · Peter Delevoryas

I noticed that I can't run any avocado tests on macOS because

the QMP

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

unix socket path is too long:

I think the path limit for unix sockets on macOS might be 104

[1]

Quoted 10 lines
Quoted 10 lines
Quoted 10 lines
Quoted 10 lines
Quoted 10 lines

All platforms have a very limited path limit, so it isn't really a macOS specific problem, rather....

Quoted 6 lines

/*
* [XSI] Definitions for UNIX IPC domain.
*/
struct sockaddr_un {
unsigned char sun_len; /* sockaddr len including

null */

Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
    sa_family_t     sun_family;     /* [XSI] AF_UNIX */
    char            sun_path[104];  /* [XSI] path name (gag) */

};

The path we're using is exactly 105 characters:

$ python
Python 2.7.10 (default, Jan 19 2016, 22:24:01)
[GCC 4.2.1 Compatible Apple LLVM 7.0.2 (clang-700.1.81)] on

darwin

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

Type "help", "copyright", "credits" or "license" for more

information.

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

len('/var/folders/d7/rz20f6hd709c1ty8f6_6y_z40000gn/T/avo_qemu_sock_uh3w_dgc/qemu-37331-10bacf110-monitor.sock')

Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines

It is a problem related to where the test suite is creating the paths.

/var/folders/d7/rz20f6hd709c1ty8f6_6y_z40000gn/T/avo_qemu_sock_uh3w_dgc/

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 2 lines

is way too deep a directory location.

That's a good point.

Quoted 2 lines

It seems we just create this location using

'tempfile.TemporyDirectory'

Quoted 10 lines
Quoted 10 lines
Quoted 10 lines
Quoted 10 lines
Quoted 8 lines

to get a standard tmp dir.

Do you know why python is choosing

/var/folders/d7/rz20f6hd709c1ty8f6_6y_z40000gn/T/

as the temp dir ? Is that a standard location on macOS or is it from some env variable you have set ?

Hmmm yeah it is odd, I'm not really sure why it's created there or

if

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

standard glibc tmpfile creation goes there too, I'll go experiment

and

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

report back. And yeah, maybe I'll double check any environment

variables or

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

other things.

The macOS system I use happens to be a Facebook work laptop, which

could

Quoted 10 lines
Quoted 10 lines
Quoted 10 lines
Quoted 1 line

also be related now that I think about it.

Hmmm yeah looks like this is because my TMPDIR is weird.

$ echo $TMPDIR
/var/folders/d7/rz20f6hd709c1ty8f6_6y_z40000gn/T/

I didn't think to check this cause I wasn't familiar with TMPDIR. 🤷

Thanks for responding, I'll just use TMPDIR=/tmp for now. It's

probably

Quoted 9 lines
Quoted 9 lines
Quoted 3 lines

something wrong with the Facebook development environment.

Peter

Update: Actually, this might not be a Facebook-work-laptop specific thing. I asked my non-engineer friend to print out $TMPDIR on his macbook and he got the same thing.

https://apple.stackexchange.com/questions/353832/why-is-mac-osx-temp-directory-in-weird-path

Quoted 11 lines
Quoted 11 lines

I guess this person suggests it's just to separate the permissions for each user's /tmp directory, for better isolation.

I'll resubmit this patch with the suggestions you had, because perhaps this is actually affecting other macOS users too.

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

Review

Quoted 4 lines
(quoted diff) +0 -0

b/python/qemu/machine/machine.py

Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines

python/qemu/machine/machine.py def __init__(self, · line 160

Quoted 8 lines

index 37191f433b..93451774e3 100644

python/qemu/machine/machine.py +0 -1
@@ -157,7 +157,7 @@ def __init__(self,
         self._wrapper = wrapper
         self._qmp_timer = qmp_timer
 
-        self._name = name or

f"qemu-{os.getpid()}-{id(self):02x}"

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

python/qemu/machine/machine.py def __init__(self, · line 160

Quoted 1 line
python/qemu/machine/machine.py +1 -0
+        self._name = name or f"{os.getpid()}{id(self):02x}"

I don't think this is the right fix really, because IMHO the

problem

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 1 line

is the hugely long path, rather than the final socket name.

True, yeah let me try to investigate the directory placement.

Quoted 2 lines

That said, there is redundancy in the path - avocado is passing

in

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

a dierctory created using 'tempfile.TemporyDirectory' so there

is no

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

reason why we need to add more entropy via the POD and the

'id(self)'

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 1 line

hex string.

Oh good point, I hadn't thought about that.

Quoted 2 lines

IMHO avocado should pass in the 'name' parameter explicitly,

using a

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 1 line

plain name and thus get a shorter string.

I see, yeah that makes sense. Maybe I can include a couple patches

for this,

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

one fixing the directory location, and one refactoring the

temporary file

Quoted 5 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

name template (If I'm understanding your suggestion correctly).

Thanks for the review! I really appreciate it.
Peter

Quoted 7 lines

I agree with Dan: I believe the correct solution here is for Avocado to provide its own less redundant name; but the default name that machine.py provides is not *that* long and provides adequate protection against collisions with multiple instances of the VM utility within a single python process. If Avocado is creating its own directories that guard against that redundancy, Avocado should provide a shortened name for the VM.

Hmmm, I see.

Quoted 9 lines

Note that the QEMUMachine process also provides a sock_dir parameter that was introduced for precisely this reason; it should be possible to instruct the avocado tests to use a shorter path for sock_dir.

I'm not clear on what the best "just works" solution will be when certain operating environments choose a tmp dir that's quite long to begin with; maybe we need a different default sockfile naming strategy that avoids the instance collision problem in machine.py, too. Ideas?

Quoted 5 lines

I guess I don't really understand why QEMU is attempting to create a unique path instead of just using mktemp(), right? This seems like something that the OS can provide. If we want some semblance of meaning to the filename, we can provide a short template, right?

Yeah, should be good. The "name" of the VM should be left unique for logging purposes, so the name should remain as-is, IMO.

However, if you want to change how we create the sockfile by using a new, temporary name (and not the vm name), that works for me as far as I can remember.

I think there was some desire to have the temp file obviously correlate to the instance, but that's not a hard requirement, so feel free to change that part.

Just make sure that come hell or high water the sockfile is cleaned up on exit.

Quoted 41 lines

-- Peter

Quoted 26 lines

--js

Quoted 23 lines
Quoted 23 lines
Quoted 23 lines
Quoted 23 lines
Quoted 15 lines
python/qemu/machine/machine.py +1 -2
         self._temp_dir: Optional[str] = None
         self._base_temp_dir = base_temp_dir
         self._sock_dir = sock_dir
@@ -167,7 +167,7 @@ def __init__(self,
             self._monitor_address = monitor_address
         else:
             self._monitor_address = os.path.join(
-                self.sock_dir, f"{self._name}-monitor.sock"
+                self.sock_dir, f"{self._name}.sock"
             )
 
         self._console_log_path = console_log
--
Quoted 3 lines

2.37.0

With regards,
Daniel
--
|: https://berrange.com -o-

https://www.flickr.com/photos/dberrange :|

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

|: https://libvirt.org -o-

https://fstop138.berrange.com :|

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

|: https://entangle-photo.org -o-

https://www.instagram.com/dberrange :|

Quoted 5 lines
Quoted 4 lines
Quoted 3 lines
Quoted 2 lines
Quoted 1 line
#8 lore raw
In reply to John Snow
Quoted 131 lines · John Snow
Quoted 76 lines · Peter Delevoryas
Quoted 63 lines · Peter Delevoryas
Quoted 49 lines · Peter Delevoryas
Quoted 26 lines · Daniel P. Berrangé
Quoted 5 lines · Peter Delevoryas

I noticed that I can't run any avocado tests on macOS because the QMP unix socket path is too long:

I think the path limit for unix sockets on macOS might be 104 [1]

All platforms have a very limited path limit, so it isn't really a macOS specific problem, rather....

Quoted 16 lines

/*
* [XSI] Definitions for UNIX IPC domain.
*/
struct sockaddr_un {

    unsigned char   sun_len;        /* sockaddr len including null */
    sa_family_t     sun_family;     /* [XSI] AF_UNIX */
    char            sun_path[104];  /* [XSI] path name (gag) */

};

The path we're using is exactly 105 characters:

$ python
Python 2.7.10 (default, Jan 19 2016, 22:24:01)
[GCC 4.2.1 Compatible Apple LLVM 7.0.2 (clang-700.1.81)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
Quoted 8 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

len('/var/folders/d7/rz20f6hd709c1ty8f6_6y_z40000gn/T/avo_qemu_sock_uh3w_dgc/qemu-37331-10bacf110-monitor.sock')

It is a problem related to where the test suite is creating the paths.

/var/folders/d7/rz20f6hd709c1ty8f6_6y_z40000gn/T/avo_qemu_sock_uh3w_dgc/

is way too deep a directory location.

That's a good point.

Quoted 10 lines

It seems we just create this location using 'tempfile.TemporyDirectory' to get a standard tmp dir.

Do you know why python is choosing

/var/folders/d7/rz20f6hd709c1ty8f6_6y_z40000gn/T/

as the temp dir ? Is that a standard location on macOS or is it from some env variable you have set ?

Quoted 7 lines

Hmmm yeah it is odd, I'm not really sure why it's created there or if standard glibc tmpfile creation goes there too, I'll go experiment and report back. And yeah, maybe I'll double check any environment variables or other things.

The macOS system I use happens to be a Facebook work laptop, which could also be related now that I think about it.

Hmmm yeah looks like this is because my TMPDIR is weird.

$ echo $TMPDIR
/var/folders/d7/rz20f6hd709c1ty8f6_6y_z40000gn/T/
Quoted 6 lines

I didn't think to check this cause I wasn't familiar with TMPDIR. 🤷

Thanks for responding, I'll just use TMPDIR=/tmp for now. It's probably something wrong with the Facebook development environment.

Peter

Update: Actually, this might not be a Facebook-work-laptop specific thing. I asked my non-engineer friend to print out $TMPDIR on his macbook and he got the same thing.

Quoted 45 lines

https://apple.stackexchange.com/questions/353832/why-is-mac-osx-temp-directory-in-weird-path

I guess this person suggests it's just to separate the permissions for each user's /tmp directory, for better isolation.

I'll resubmit this patch with the suggestions you had, because perhaps this is actually affecting other macOS users too.

Quoted 37 lines
Quoted 37 lines
Quoted 11 lines
Quoted 11 lines
python/qemu/machine/machine.py +0 -0
index 37191f433b..93451774e3 100644
@@ -157,7 +157,7 @@ def __init__(self,
         self._wrapper = wrapper
         self._qmp_timer = qmp_timer
 
Quoted 5 lines

python/qemu/machine/machine.py def __init__(self, · line 160

Quoted 2 lines
python/qemu/machine/machine.py +1 -1
-        self._name = name or f"qemu-{os.getpid()}-{id(self):02x}"
+        self._name = name or f"{os.getpid()}{id(self):02x}"

I don't think this is the right fix really, because IMHO the problem is the hugely long path, rather than the final socket name.

True, yeah let me try to investigate the directory placement.

Quoted 5 lines

That said, there is redundancy in the path - avocado is passing in a dierctory created using 'tempfile.TemporyDirectory' so there is no reason why we need to add more entropy via the POD and the 'id(self)' hex string.

Oh good point, I hadn't thought about that.

Quoted 3 lines

IMHO avocado should pass in the 'name' parameter explicitly, using a plain name and thus get a shorter string.

I see, yeah that makes sense. Maybe I can include a couple patches for this, one fixing the directory location, and one refactoring the temporary file name template (If I'm understanding your suggestion correctly).

Thanks for the review! I really appreciate it.
Peter

I agree with Dan: I believe the correct solution here is for Avocado to provide its own less redundant name; but the default name that machine.py provides is not *that* long and provides adequate protection against collisions with multiple instances of the VM utility within a single python process. If Avocado is creating its own directories that guard against that redundancy, Avocado should provide a shortened name for the VM.

Quoted 8 lines

Note that the QEMUMachine process also provides a sock_dir parameter that was introduced for precisely this reason; it should be possible to instruct the avocado tests to use a shorter path for sock_dir.

I'm not clear on what the best "just works" solution will be when certain operating environments choose a tmp dir that's quite long to begin with; maybe we need a different default sockfile naming strategy that avoids the instance collision problem in machine.py, too. Ideas?

Ultimately there's no solution that is guaranteed to work. If $HOME or $TMPDIR point to paths that are >= 108 chars long, we're doomed no matter what. To maximise the liklihood of working though, we need to be minimal in paths we append.

Right now, Peter's TMPDIR is

/var/folders/d7/rz20f6hd709c1ty8f6_6y_z40000gn/T

Onto which QEMU appends

/avo_qemu_sock_uh3w_dgc/qemu-37331-10bacf110-monitor.sock

IOW, QEMU is more than 1/2 of the problem here.

QEMU could cut down the size of the temporary directory using a more concise pattern, changing

tempfile.TemporaryDirectory(prefix="avo_qemu_sock_")

To

tempfile.TemporaryDirectory(prefix="qemu_")

And then use a fixed socket name 'qmp.sock', giving us:

/qemu_uh3w_dgc/qmp.sock

to append to TMPDIR

With regards,
Daniel

Signature · 3 lines
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|