hostmem: don't use mbind() if host-nodes is epmty
PATCH 1/1 hostmem: don't use mbind() if host-nodes is epmty
Since 5.0 QEMU uses hostmem backend for allocating main guest RAM. The backend however calls mbind() which is typically NOP in case of default policy/absent host-nodes bitmap. However when runing in container with black-listed mbind() syscall, QEMU fails to start with error
"cannot bind memory to host NUMA nodes: Operation not permitted"
even when user hasn't provided host-nodes to pin to explictly (which is the case with -m option)
To fix issue, call mbind() only in case when user has provided host-nodes explicitly (i.e. host_nodes bitmap is not empty). That should allow to run QEMU in containers with black-listed mbind() without memory pinning. If QEMU provided memory-pinning is required user still has to white-list mbind() in container configuration.
---
CC: [email protected]
CC: [email protected]
CC: [email protected]
CC: [email protected]
CC: [email protected]
--- backends/hostmem.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-)
backends/hostmem.c +4 -2
@@ -383,8 +383,10 @@ host_memory_backend_memory_complete(UserCreatable *uc, Error **errp)assert (sizeof (backend->host_nodes) >=BITS_TO_LONGS (MAX_NODES +1 ) *sizeof (unsigned long ));assert (maxnode <=MAX_NODES ); -if (mbind (ptr, sz, backend->policy, - maxnode ? backend->host_nodes :NULL , maxnode +1 , flags)) { + +if (maxnode && +mbind (ptr, sz, backend->policy, backend->host_nodes, maxnode +1 , + flags)) {if (backend->policy !=MPOL_DEFAULT || errno !=ENOSYS ) {error_setg_errno (errp, errno,"cannot bind memory to host NUMA nodes" );
Signature · 1 line
2.18.1
Typo "empty" in patch subject.
Quoted 46 lines · Igor Mammedov
Since 5.0 QEMU uses hostmem backend for allocating main guest RAM. The backend however calls
mbind()which is typically NOP in case of default policy/absent host-nodes bitmap. However when runing in container with black-listedmbind()syscall, QEMU fails to start with error
"cannot bind memory to host NUMA nodes: Operation not permitted"
even when user hasn't provided host-nodes to pin to explictly (which is the case with -m option)To fix issue, call
mbind()only in case when user has provided host-nodes explicitly (i.e.host_nodesbitmap is not empty). That should allow to run QEMU in containers with black-listedmbind()without memory pinning. If QEMU provided memory-pinning is required user still has to white-listmbind()in container configuration.Reported-by Manuel Hohmann <[email protected]>Signed-off-by Igor Mammedov <[email protected]>---
CC: [email protected]
CC: [email protected]
CC: [email protected]
CC: [email protected]
CC: [email protected]--- backends/hostmem.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-)backends/hostmem.c +4
-2@@ -383,8 +383,10 @@ host_memory_backend_memory_complete(UserCreatable *uc, Error **errp)assert (sizeof (backend->host_nodes) >=BITS_TO_LONGS (MAX_NODES +1 ) *sizeof (unsigned long ));assert (maxnode <=MAX_NODES ); -if (mbind (ptr, sz, backend->policy, - maxnode ? backend->host_nodes :NULL , maxnode +1 , flags)) { + +if (maxnode && +mbind (ptr, sz, backend->policy, backend->host_nodes, maxnode +1 , + flags)) {if (backend->policy !=MPOL_DEFAULT || errno !=ENOSYS ) {error_setg_errno (errp, errno,"cannot bind memory to host NUMA nodes" );
Thanks! I applied the patch, and now it works also inside the docker container, for all architectures (i386, x86_64, arm, aarch64) for which I have test cases at hand.
Indeed, since the container is configured by a public cloud service, there is no possibility to change any security settings. Disabling mbind unless explicitly requested seems to be the best way to go here.
Quoted 50 lines · Philippe Mathieu-Daudé
Typo "empty" in patch subject.
Quoted 46 lines · Igor Mammedov
Since 5.0 QEMU uses hostmem backend for allocating main guest RAM. The backend however calls
mbind()which is typically NOP in case of default policy/absent host-nodes bitmap. However when runing in container with black-listedmbind()syscall, QEMU fails to start with error
"cannot bind memory to host NUMA nodes: Operation not permitted" even when user hasn't provided host-nodes to pin to explictly (which is the case with -m option)To fix issue, call
mbind()only in case when user has provided host-nodes explicitly (i.e.host_nodesbitmap is not empty). That should allow to run QEMU in containers with black-listedmbind()without memory pinning. If QEMU provided memory-pinning is required user still has to white-listmbind()in container configuration.Reported-by Manuel Hohmann <[email protected]>Signed-off-by Igor Mammedov <[email protected]>---
CC: [email protected]
CC: [email protected]
CC: [email protected]
CC: [email protected]
CC: [email protected]
---
backends/hostmem.c| 6 ++++--1 file changed, 4 insertions(+), 2 deletions(-)backends/hostmem.c +0
-0@@ -383,8 +383,10 @@ host_memory_backend_memory_complete(UserCreatable *uc, Error **errp)
assert(sizeof(backend->host_nodes) >= BITS_TO_LONGS(MAX_NODES + 1) * sizeof(unsigned long));assert(maxnode <= MAX_NODES);
- if (mbind(ptr, sz,backend->policy,
- maxnode ?backend->host_nodes: NULL, maxnode + 1, flags)) { +
+ if (maxnode &&
+ mbind(ptr, sz,backend->policy,backend->host_nodes, maxnode + 1, + flags)) {
if (backend->policy!=MPOL_DEFAULT|| errno != ENOSYS) { error_setg_errno(errp, errno,
"cannot bind memory to host NUMA nodes");
Quoted 35 lines · Igor Mammedov
Since 5.0 QEMU uses hostmem backend for allocating main guest RAM. The backend however calls
mbind()which is typically NOP in case of default policy/absent host-nodes bitmap. However when runing in container with black-listedmbind()syscall, QEMU fails to start with error
"cannot bind memory to host NUMA nodes: Operation not permitted"
even when user hasn't provided host-nodes to pin to explictly (which is the case with -m option)To fix issue, call
mbind()only in case when user has provided host-nodes explicitly (i.e.host_nodesbitmap is not empty). That should allow to run QEMU in containers with black-listedmbind()without memory pinning. If QEMU provided memory-pinning is required user still has to white-listmbind()in container configuration.Reported-by Manuel Hohmann <[email protected]>Signed-off-by Igor Mammedov <[email protected]>---
CC: [email protected]
CC: [email protected]
CC: [email protected]
CC: [email protected]
CC: [email protected]--- backends/hostmem.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-)backends/hostmem.c +0
-0@@ -383,8 +383,10 @@ host_memory_backend_memory_complete(UserCreatable *uc, Error **errp)assert (sizeof (backend->host_nodes) >=BITS_TO_LONGS (MAX_NODES +1 ) *sizeof (unsigned long ));
backends/hostmem.c host_memory_backend_memory_complete(UserCreatable *uc, Error **errp) · line 392
Quoted 10 lines
backends/hostmem.c +4
-2assert (maxnode <=MAX_NODES ); -if (mbind (ptr, sz, backend->policy, - maxnode ? backend->host_nodes :NULL , maxnode +1 , flags)) { + +if (maxnode && +mbind (ptr, sz, backend->policy, backend->host_nodes, maxnode +1 , + flags)) {if (backend->policy !=MPOL_DEFAULT || errno !=ENOSYS ) {error_setg_errno (errp, errno,"cannot bind memory to host NUMA nodes" );
personally I would have found this code clearer if the check had been "if (backend->policy != MPOL_DEFAULT && ..." as I had to read quite a few lines to understand that the 'maxnode' is zero if-and-only-if policy == MPOL_DEFAULT
Regardless though, this is functionally correct so
Regards,
Daniel
Signature · 3 lines
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
Philippe Mathieu-Daudé
Tested-byQuoted 52 lines · Daniel P. Berrangé
Quoted 35 lines · Igor Mammedov
Since 5.0 QEMU uses hostmem backend for allocating main guest RAM. The backend however calls
mbind()which is typically NOP in case of default policy/absent host-nodes bitmap. However when runing in container with black-listedmbind()syscall, QEMU fails to start with error
"cannot bind memory to host NUMA nodes: Operation not permitted"
even when user hasn't provided host-nodes to pin to explictly (which is the case with -m option)To fix issue, call
mbind()only in case when user has provided host-nodes explicitly (i.e.host_nodesbitmap is not empty). That should allow to run QEMU in containers with black-listedmbind()without memory pinning. If QEMU provided memory-pinning is required user still has to white-listmbind()in container configuration.Reported-by Manuel Hohmann <[email protected]>Signed-off-by Igor Mammedov <[email protected]>---
CC: [email protected]
CC: [email protected]
CC: [email protected]
CC: [email protected]
CC: [email protected]--- backends/hostmem.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-)backends/hostmem.c +0
-0@@ -383,8 +383,10 @@ host_memory_backend_memory_complete(UserCreatable *uc, Error **errp)assert (sizeof (backend->host_nodes) >=BITS_TO_LONGS (MAX_NODES +1 ) *sizeof (unsigned long ));backends/hostmem.c host_memory_backend_memory_complete(UserCreatable *uc, Error **errp) · line 392
Quoted 10 lines
backends/hostmem.c +4
-2assert (maxnode <=MAX_NODES ); -if (mbind (ptr, sz, backend->policy, - maxnode ? backend->host_nodes :NULL , maxnode +1 , flags)) { + +if (maxnode && +mbind (ptr, sz, backend->policy, backend->host_nodes, maxnode +1 , + flags)) {if (backend->policy !=MPOL_DEFAULT || errno !=ENOSYS ) {error_setg_errno (errp, errno,"cannot bind memory to host NUMA nodes" );personally I would have found this code clearer if the check had been "if (
backend->policy!=MPOL_DEFAULT&& ..." as I had to read quite a few lines to understand that the 'maxnode' is zero if-and-only-if policy ==MPOL_DEFAULT
Quoted 3 lines
Regardless though, this is functionally correct so
Reviewed-by Daniel P. Berrangé <[email protected]>
I could reproduce running 'make check-qtest-hppa' on the qemu:fedora image:
TEST check-qtest-hppa: tests/qtest/boot-serial-test
qemu-system-hppa: cannot bind memory to host NUMA nodes: Operation not permitted
Broken pipetests/qtest/libqtest.c:166: kill_qemu() tried to terminate QEMU process but encountered exit status 1 (expected 0) ERROR - too few tests run (expected 1, got 0)
make: *** [tests/Makefile.include:637: check-qtest-hppa] Error 1
Quoted 4 lines
Regards,
Daniel
Eduardo Habkost
AppliedQuoted 16 lines · Igor Mammedov
Since 5.0 QEMU uses hostmem backend for allocating main guest RAM. The backend however calls
mbind()which is typically NOP in case of default policy/absent host-nodes bitmap. However when runing in container with black-listedmbind()syscall, QEMU fails to start with error
"cannot bind memory to host NUMA nodes: Operation not permitted"
even when user hasn't provided host-nodes to pin to explictly (which is the case with -m option)To fix issue, call
mbind()only in case when user has provided host-nodes explicitly (i.e.host_nodesbitmap is not empty). That should allow to run QEMU in containers with black-listedmbind()without memory pinning. If QEMU provided memory-pinning is required user still has to white-listmbind()in container configuration.
Quoted 2 lines
Reported-by Manuel Hohmann <[email protected]>Signed-off-by Igor Mammedov <[email protected]>
Queued on machine-next, thanks!
Signature · 1 line
Eduardo
Hi Eduardo,
Quoted 17 lines · Eduardo Habkost
Quoted 16 lines · Igor Mammedov
Since 5.0 QEMU uses hostmem backend for allocating main guest RAM. The backend however calls
mbind()which is typically NOP in case of default policy/absent host-nodes bitmap. However when runing in container with black-listedmbind()syscall, QEMU fails to start with error
"cannot bind memory to host NUMA nodes: Operation not permitted"
even when user hasn't provided host-nodes to pin to explictly (which is the case with -m option)To fix issue, call
mbind()only in case when user has provided host-nodes explicitly (i.e.host_nodesbitmap is not empty). That should allow to run QEMU in containers with black-listedmbind()without memory pinning. If QEMU provided memory-pinning is required user still has to white-listmbind()in container configuration.
Quoted 4 lines
Quoted 2 lines
Reported-by Manuel Hohmann <[email protected]>Signed-off-by Igor Mammedov <[email protected]>Queued on machine-next, thanks!
I've been debugging this issue again today and figured it was not merged, if possible can you add the "Cc: [email protected]" tag before sending your pull request?
Thanks,
Phil.
Quoted 20 lines · On Mon, 11 May 2020 18:00:01 +0200 Philippe Mathieu-Daudé <[email protected]>…
Hi Eduardo,
Quoted 17 lines · Eduardo Habkost
Quoted 16 lines · Igor Mammedov
Since 5.0 QEMU uses hostmem backend for allocating main guest RAM. The backend however calls
mbind()which is typically NOP in case of default policy/absent host-nodes bitmap. However when runing in container with black-listedmbind()syscall, QEMU fails to start with error
"cannot bind memory to host NUMA nodes: Operation not permitted"
even when user hasn't provided host-nodes to pin to explictly (which is the case with -m option)To fix issue, call
mbind()only in case when user has provided host-nodes explicitly (i.e.host_nodesbitmap is not empty). That should allow to run QEMU in containers with black-listedmbind()without memory pinning. If QEMU provided memory-pinning is required user still has to white-listmbind()in container configuration.
Quoted 8 lines
Quoted 4 lines
Quoted 2 lines
Reported-by Manuel Hohmann <[email protected]>Signed-off-by Igor Mammedov <[email protected]>Queued on machine-next, thanks!
I've been debugging this issue again today and figured it was not merged, if possible can you add the "Cc: [email protected]" tag before sending your pull request?
it's CCed already, so my impression was that will should picked up once it was reviewed.
Quoted 5 lines
Thanks,
Phil.
Quoted 26 lines · Igor Mammedov
Quoted 23 lines · On Mon, 11 May 2020 18:00:01 +0200 Philippe Mathieu-Daudé <[email protected]>…
Hi Eduardo,
Quoted 20 lines · Eduardo Habkost
Quoted 18 lines · Igor Mammedov
Since 5.0 QEMU uses hostmem backend for allocating main guest RAM. The backend however calls
mbind()which is typically NOP in case of default policy/absent host-nodes bitmap. However when runing in container with black-listedmbind()syscall, QEMU fails to start with error
"cannot bind memory to host NUMA nodes: Operation not permitted"
even when user hasn't provided host-nodes to pin to explictly (which is the case with -m option)To fix issue, call
mbind()only in case when user has provided host-nodes explicitly (i.e.host_nodesbitmap is not empty). That should allow to run QEMU in containers with black-listedmbind()without memory pinning. If QEMU provided memory-pinning is required user still has to white-listmbind()in container configuration.Reported-by Manuel Hohmann <[email protected]>Signed-off-by Igor Mammedov <[email protected]>
Quoted 6 lines
Quoted 5 lines
Quoted 1 line
Queued on machine-next, thanks!
I've been debugging this issue again today and figured it was not merged, if possible can you add the "Cc: [email protected]" tag before sending your pull request?
it's CCed already, so my impression was that will should picked up once it was reviewed.
Correct, however some distributions find easier to grep for the 'Cc: [email protected]' merged tag before qemu-stable is released.
Quoted 7 lines
Quoted 6 lines
Thanks,
Phil.