KVM: Avoid debugfs warning caused by repeated vm fd number
PATCH 1/1 KVM: Avoid debugfs warning caused by repeated vm fd number
Avoid debugfs warning like "KVM: debugfs: duplicate directory 59904-4" caused by creating VMs with the same vm fd number in a single process.
As shown in the below test case, two test() are executed sequentially in a single process, each creating a new VM.
Though the 2nd test() creates a new VM after the 1st test() closes the vm_fd, KVM prints warnings like "KVM: debugfs: duplicate directory 59904-4" on creating the 2nd VM.
This is due to the dup() of the vcpu_fd in test(). So, after closing the 1st vm_fd, kvm->users_count of the 1st VM is still > 0 when creating the 2nd VM. So, KVM has not yet invoked kvm_destroy_vm() and kvm_destroy_vm_debugfs() for the 1st VM after closing the 1st vm_fd. The 2nd test() thus will be able to create a different VM with the same vm fd number as the 1st VM.
Therefore, besides having "pid" and "fdname" in the dir_name of the debugfs, add a random number to differentiate different VMs to avoid printing warning, also allowing the 2nd VM to have a functional debugfs.
Use get_random_u32() to avoid dir_name() taking up too much memory while greatly reducing the chance of printing warning.
void test(void)
{
int kvm_fd, vm_fd, vcpu_fd;
kvm_fd = open("/dev/kvm", O_RDWR);
if (kvm_fd == -1)
return;
vm_fd = ioctl(kvm_fd, KVM_CREATE_VM, 0);
if (vm_fd == -1)
return;
vcpu_fd = ioctl(vm_fd, KVM_CREATE_VCPU, 0);
if (vcpu_fd == -1)
return;
dup(vcpu_fd);
close(vcpu_fd);
close(vm_fd);
close(kvm_fd);
}int main()
{
test();
test();
return 0;
}--- virt/kvm/kvm_main.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-)
virt/kvm/kvm_main.c +3 -2
@@ -1017,7 +1017,7 @@ static int kvm_create_vm_debugfs(struct kvm *kvm, const char *fdname) {static DEFINE_MUTEX (kvm_debugfs_lock);struct dentry *dent; -char dir_name[ITOA_MAX_LEN *2 ]; +char dir_name[ITOA_MAX_LEN *3 ];struct kvm_stat_data *stat_data;const struct _kvm_stats_desc *pdesc;int i, ret = -ENOMEM ;
@@ -1027,7 +1027,8 @@ static int kvm_create_vm_debugfs(struct kvm *kvm, const char *fdname)if (!debugfs_initialized ())return 0 ; -snprintf (dir_name,sizeof (dir_name),"%d-%s" ,task_pid_nr (current), fdname); +snprintf (dir_name,sizeof (dir_name),"%d-%s-%u" ,task_pid_nr (current), + fdname,get_random_u32 ());mutex_lock (&kvm_debugfs_lock); dent =debugfs_lookup (dir_name, kvm_debugfs_dir);if (dent) {
Signature · 1 line
2.39.2
Hi Ted,
Quoted 72 lines · Ted Chen
Avoid debugfs warning like "KVM: debugfs: duplicate directory 59904-4" caused by creating VMs with the same vm fd number in a single process.
As shown in the below test case, two
test()are executed sequentially in a single process, each creating a new VM.Though the 2nd
test()creates a new VM after the 1sttest()closes thevm_fd, KVM prints warnings like "KVM: debugfs: duplicate directory 59904-4" on creating the 2nd VM.This is due to the
dup()of thevcpu_fdintest(). So, after closing the 1stvm_fd,kvm->users_countof the 1st VM is still > 0 when creating the 2nd VM. So, KVM has not yet invokedkvm_destroy_vm()andkvm_destroy_vm_debugfs()for the 1st VM after closing the 1stvm_fd. The 2ndtest()thus will be able to create a different VM with the same vm fd number as the 1st VM.Therefore, besides having "pid" and "fdname" in the
dir_nameof the debugfs, add a random number to differentiate different VMs to avoid printing warning, also allowing the 2nd VM to have a functional debugfs.Use
get_random_u32()to avoiddir_name()taking up too much memory while greatly reducing the chance of printing warning.void test(void)
{ int kvm_fd, vm_fd, vcpu_fd; kvm_fd = open("/dev/kvm", O_RDWR); if (kvm_fd == -1) return; vm_fd = ioctl(kvm_fd, KVM_CREATE_VM, 0); if (vm_fd == -1) return; vcpu_fd = ioctl(vm_fd, KVM_CREATE_VCPU, 0); if (vcpu_fd == -1) return; dup(vcpu_fd); close(vcpu_fd); close(vm_fd); close(kvm_fd); }int
main(){ test(); test(); return 0; }Signed-off-by Ted Chen <[email protected]>--- virt/kvm/kvm_main.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-)virt/kvm/kvm_main.c +1
-1@@ -1017,7 +1017,7 @@ static int kvm_create_vm_debugfs(struct kvm *kvm, const char *fdname) {static DEFINE_MUTEX (kvm_debugfs_lock);struct dentry *dent; -char dir_name[ITOA_MAX_LEN *2 ]; +char dir_name[ITOA_MAX_LEN *3 ];struct kvm_stat_data *stat_data;const struct _kvm_stats_desc *pdesc;int i, ret = -ENOMEM ;
virt/kvm/kvm_main.c static int kvm_create_vm_debugfs(struct kvm *kvm, const char *fdname) · line 1031
Quoted 6 lines
virt/kvm/kvm_main.c +2
-1@@ -1027,7 +1027,8 @@ static int kvm_create_vm_debugfs(struct kvm *kvm, const char *fdname)if (!debugfs_initialized ())return 0 ; -snprintf (dir_name,sizeof (dir_name),"%d-%s" ,task_pid_nr (current), fdname); +snprintf (dir_name,sizeof (dir_name),"%d-%s-%u" ,task_pid_nr (current), + fdname,get_random_u32 ());
This does make the directory names (very likely) to be unique but it's not helpful in distinguishing which directory maps to which vm. I wonder if there is some better id we could use here.
Should the vm stats_id also be updated to be unique and use the same scheme?
virt/kvm/kvm_main.c static int kvm_create_vm_debugfs(struct kvm *kvm, const char *fdname) · line 1034
Quoted 3 lines
virt/kvm/kvm_main.c +0
-0mutex_lock (&kvm_debugfs_lock); dent =debugfs_lookup (dir_name, kvm_debugfs_dir);if (dent) {
Thanks,
Ben
Quoted 78 lines · Ben Horgan
Hi Ted,
Quoted 75 lines · Ted Chen
Avoid debugfs warning like "KVM: debugfs: duplicate directory 59904-4" caused by creating VMs with the same vm fd number in a single process.
As shown in the below test case, two
test()are executed sequentially in a single process, each creating a new VM.Though the 2nd
test()creates a new VM after the 1sttest()closes thevm_fd, KVM prints warnings like "KVM: debugfs: duplicate directory 59904-4" on creating the 2nd VM.This is due to the
dup()of thevcpu_fdintest(). So, after closing the 1stvm_fd,kvm->users_countof the 1st VM is still > 0 when creating the 2nd VM. So, KVM has not yet invokedkvm_destroy_vm()andkvm_destroy_vm_debugfs()for the 1st VM after closing the 1stvm_fd. The 2ndtest()thus will be able to create a different VM with the same vm fd number as the 1st VM.Therefore, besides having "pid" and "fdname" in the
dir_nameof the debugfs, add a random number to differentiate different VMs to avoid printing warning, also allowing the 2nd VM to have a functional debugfs.Use
get_random_u32()to avoiddir_name()taking up too much memory while greatly reducing the chance of printing warning.void test(void)
{ int kvm_fd, vm_fd, vcpu_fd; kvm_fd = open("/dev/kvm", O_RDWR); if (kvm_fd == -1) return; vm_fd = ioctl(kvm_fd, KVM_CREATE_VM, 0); if (vm_fd == -1) return; vcpu_fd = ioctl(vm_fd, KVM_CREATE_VCPU, 0); if (vcpu_fd == -1) return; dup(vcpu_fd); close(vcpu_fd); close(vm_fd); close(kvm_fd); }int
main(){ test(); test(); return 0; }Signed-off-by Ted Chen <[email protected]>--- virt/kvm/kvm_main.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-)virt/kvm/kvm_main.c +1
-1@@ -1017,7 +1017,7 @@ static int kvm_create_vm_debugfs(struct kvm *kvm, const char *fdname) {static DEFINE_MUTEX (kvm_debugfs_lock);struct dentry *dent; -char dir_name[ITOA_MAX_LEN *2 ]; +char dir_name[ITOA_MAX_LEN *3 ];struct kvm_stat_data *stat_data;const struct _kvm_stats_desc *pdesc;int i, ret = -ENOMEM ;@@ -1027,7 +1027,8 @@ static int kvm_create_vm_debugfs(struct kvm *kvm, const char *fdname)if (!debugfs_initialized ())return 0 ;
Quoted 7 lines
virt/kvm/kvm_main.c static int kvm_create_vm_debugfs(struct kvm *kvm, const char *fdname) · line 1031
Quoted 3 lines
virt/kvm/kvm_main.c +2
-1-snprintf (dir_name,sizeof (dir_name),"%d-%s" ,task_pid_nr (current), fdname); +snprintf (dir_name,sizeof (dir_name),"%d-%s-%u" ,task_pid_nr (current), + fdname,get_random_u32 ());This does make the directory names (very likely) to be unique but it's not helpful in distinguishing which directory maps to which vm. I wonder if there is some better id we could use here.
Good point. Maybe use timestamp instead?
So, we can know a bigger timestamp value corresponds to a VM created later. Also since VMs are created in a single thread, they can't have the same timestamp value.
virt/kvm/kvm_main.c +3 -2
@@ -1017,7 +1017,7 @@ static int kvm_create_vm_debugfs(struct kvm *kvm, const char *fdname) {static DEFINE_MUTEX (kvm_debugfs_lock);struct dentry *dent; -char dir_name[ITOA_MAX_LEN *2 ]; +char dir_name[ITOA_MAX_LEN *4 ];struct kvm_stat_data *stat_data;const struct _kvm_stats_desc *pdesc;int i, ret = -ENOMEM ;
@@ -1027,7 +1027,8 @@ static int kvm_create_vm_debugfs(struct kvm *kvm, const char *fdname)if (!debugfs_initialized ())return 0 ; -snprintf (dir_name,sizeof (dir_name),"%d-%s" ,task_pid_nr (current), fdname); +snprintf (dir_name,sizeof (dir_name),"%d-%s-%llx" ,task_pid_nr (current), + fdname,ktime_get_ns ());mutex_lock (&kvm_debugfs_lock); dent =debugfs_lookup (dir_name, kvm_debugfs_dir);if (dent) {
Quoted 1 line
Should the vm
stats_idalso be updated to be unique and use the same scheme?
I don't think so. Unlike debugfs paths that will be accessed directly by userspace, the stats_id is used by anonymous inode files openned with unique fd numbers in a single process. So, duplicated names should be ok.
Hi,
Quoted 86 lines · Ted Chen
Quoted 78 lines · Ben Horgan
Hi Ted,
Quoted 75 lines · Ted Chen
Avoid debugfs warning like "KVM: debugfs: duplicate directory 59904-4" caused by creating VMs with the same vm fd number in a single process.
As shown in the below test case, two
test()are executed sequentially in a single process, each creating a new VM.Though the 2nd
test()creates a new VM after the 1sttest()closes thevm_fd, KVM prints warnings like "KVM: debugfs: duplicate directory 59904-4" on creating the 2nd VM.This is due to the
dup()of thevcpu_fdintest(). So, after closing the 1stvm_fd,kvm->users_countof the 1st VM is still > 0 when creating the 2nd VM. So, KVM has not yet invokedkvm_destroy_vm()andkvm_destroy_vm_debugfs()for the 1st VM after closing the 1stvm_fd. The 2ndtest()thus will be able to create a different VM with the same vm fd number as the 1st VM.Therefore, besides having "pid" and "fdname" in the
dir_nameof the debugfs, add a random number to differentiate different VMs to avoid printing warning, also allowing the 2nd VM to have a functional debugfs.Use
get_random_u32()to avoiddir_name()taking up too much memory while greatly reducing the chance of printing warning.void test(void)
{ int kvm_fd, vm_fd, vcpu_fd; kvm_fd = open("/dev/kvm", O_RDWR); if (kvm_fd == -1) return; vm_fd = ioctl(kvm_fd, KVM_CREATE_VM, 0); if (vm_fd == -1) return; vcpu_fd = ioctl(vm_fd, KVM_CREATE_VCPU, 0); if (vcpu_fd == -1) return; dup(vcpu_fd); close(vcpu_fd); close(vm_fd); close(kvm_fd); }int
main(){ test(); test(); return 0; }Signed-off-by Ted Chen <[email protected]>--- virt/kvm/kvm_main.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-)virt/kvm/kvm_main.c +1
-1@@ -1017,7 +1017,7 @@ static int kvm_create_vm_debugfs(struct kvm *kvm, const char *fdname) {static DEFINE_MUTEX (kvm_debugfs_lock);struct dentry *dent; -char dir_name[ITOA_MAX_LEN *2 ]; +char dir_name[ITOA_MAX_LEN *3 ];struct kvm_stat_data *stat_data;const struct _kvm_stats_desc *pdesc;int i, ret = -ENOMEM ;@@ -1027,7 +1027,8 @@ static int kvm_create_vm_debugfs(struct kvm *kvm, const char *fdname)if (!debugfs_initialized ())return 0 ;Quoted 7 lines
virt/kvm/kvm_main.c static int kvm_create_vm_debugfs(struct kvm *kvm, const char *fdname) · line 1031
Quoted 3 lines
virt/kvm/kvm_main.c +2
-1-snprintf (dir_name,sizeof (dir_name),"%d-%s" ,task_pid_nr (current), fdname); +snprintf (dir_name,sizeof (dir_name),"%d-%s-%u" ,task_pid_nr (current), + fdname,get_random_u32 ());This does make the directory names (very likely) to be unique but it's not helpful in distinguishing which directory maps to which vm. I wonder if there is some better id we could use here.
Quoted 3 lines
Good point. Maybe use timestamp instead?
So, we can know a bigger timestamp value corresponds to a VM created later. Also since VMs are created in a single thread, they can't have the same
Why must all VMs be created in a single thread?
virt/kvm/kvm_main.c static int kvm_create_vm_debugfs(struct kvm *kvm, const char *fdname) · line 1020
Quoted 12 lines
timestamp value.
virt/kvm/kvm_main.c +1
-1@@ -1017,7 +1017,7 @@ static int kvm_create_vm_debugfs(struct kvm *kvm, const char *fdname) {static DEFINE_MUTEX (kvm_debugfs_lock);struct dentry *dent; -char dir_name[ITOA_MAX_LEN *2 ]; +char dir_name[ITOA_MAX_LEN *4 ];
Hmmm.
Quoted 15 lines
virt/kvm/kvm_main.c +2
-1struct kvm_stat_data *stat_data;const struct _kvm_stats_desc *pdesc;int i, ret = -ENOMEM ;@@ -1027,7 +1027,8 @@ static int kvm_create_vm_debugfs(struct kvm *kvm, const char *fdname)if (!debugfs_initialized ())return 0 ; -snprintf (dir_name,sizeof (dir_name),"%d-%s" ,task_pid_nr (current), fdname); +snprintf (dir_name,sizeof (dir_name),"%d-%s-%llx" ,task_pid_nr (current), + fdname,ktime_get_ns ());mutex_lock (&kvm_debugfs_lock); dent =debugfs_lookup (dir_name, kvm_debugfs_dir);if (dent) {
Quoted 5 lines
Quoted 1 line
Should the vm
stats_idalso be updated to be unique and use the same scheme?I don't think so. Unlike debugfs paths that will be accessed directly by userspace, the
stats_idis used by anonymous inode files openned with unique fd numbers in a single process. So, duplicated names should be ok.
Just sticking my oar in here, since I'm interested in how we identify VMs and vCPUs for use by other kernel subsystems:
The two proposed approaches may not guarantee uniqueness. The PID is not unique because a single process can create multiple VMs. Possibly, VMs can outlive the PID of the creating process (if a KVM fd is passed to another process, and the original process terminates). The fd is not unique because fds can be moved around with dup() and reallocated. Random numbers are not unique (only _probably_ unique.)
If VMs are created from two threads simultaneously and if ownership of kvm_debugfs_lock can move fast enough, or there is enough skid or imprecision in the timestamp, then can ktime_get_ns() be non-unique too?
_Maybe_ this is good enough for debugfs -- which is often treated in a best-effort way by kernel code.
However, if we really want a robust ID that can not only distinguish between VMs but can also tell us reliably (and usefully) which is which even when multiple VMs are created by a single process, I think KVM userspace needs to be able to set and/or retrieve the ID, or part of it.
It might make sense to define a new kind of identifier for such purposes.
I'm guessing that this is out of the scope for this patch, though.
Cheers
---Dave
Quoted 89 lines · Dave Martin
Hi,
Quoted 86 lines · Ted Chen
Quoted 85 lines · Ben Horgan
Hi Ted,
Quoted 72 lines · Ted Chen
Avoid debugfs warning like "KVM: debugfs: duplicate directory 59904-4" caused by creating VMs with the same vm fd number in a single process.
As shown in the below test case, two
test()are executed sequentially in a single process, each creating a new VM.Though the 2nd
test()creates a new VM after the 1sttest()closes thevm_fd, KVM prints warnings like "KVM: debugfs: duplicate directory 59904-4" on creating the 2nd VM.This is due to the
dup()of thevcpu_fdintest(). So, after closing the 1stvm_fd,kvm->users_countof the 1st VM is still > 0 when creating the 2nd VM. So, KVM has not yet invokedkvm_destroy_vm()andkvm_destroy_vm_debugfs()for the 1st VM after closing the 1stvm_fd. The 2ndtest()thus will be able to create a different VM with the same vm fd number as the 1st VM.Therefore, besides having "pid" and "fdname" in the
dir_nameof the debugfs, add a random number to differentiate different VMs to avoid printing warning, also allowing the 2nd VM to have a functional debugfs.Use
get_random_u32()to avoiddir_name()taking up too much memory while greatly reducing the chance of printing warning.void test(void)
{ int kvm_fd, vm_fd, vcpu_fd; kvm_fd = open("/dev/kvm", O_RDWR); if (kvm_fd == -1) return; vm_fd = ioctl(kvm_fd, KVM_CREATE_VM, 0); if (vm_fd == -1) return; vcpu_fd = ioctl(vm_fd, KVM_CREATE_VCPU, 0); if (vcpu_fd == -1) return; dup(vcpu_fd); close(vcpu_fd); close(vm_fd); close(kvm_fd); }int
main(){ test(); test(); return 0; }Signed-off-by Ted Chen <[email protected]>--- virt/kvm/kvm_main.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-)virt/kvm/kvm_main.c +1
-1@@ -1017,7 +1017,7 @@ static int kvm_create_vm_debugfs(struct kvm *kvm, const char *fdname) {static DEFINE_MUTEX (kvm_debugfs_lock);struct dentry *dent; -char dir_name[ITOA_MAX_LEN *2 ]; +char dir_name[ITOA_MAX_LEN *3 ];struct kvm_stat_data *stat_data;const struct _kvm_stats_desc *pdesc;int i, ret = -ENOMEM ;virt/kvm/kvm_main.c static int kvm_create_vm_debugfs(struct kvm *kvm, const char *fdname) · line 1031
Quoted 6 lines
virt/kvm/kvm_main.c +2
-1@@ -1027,7 +1027,8 @@ static int kvm_create_vm_debugfs(struct kvm *kvm, const char *fdname)if (!debugfs_initialized ())return 0 ; -snprintf (dir_name,sizeof (dir_name),"%d-%s" ,task_pid_nr (current), fdname); +snprintf (dir_name,sizeof (dir_name),"%d-%s-%u" ,task_pid_nr (current), + fdname,get_random_u32 ());This does make the directory names (very likely) to be unique but it's not helpful in distinguishing which directory maps to which vm. I wonder if there is some better id we could use here.
Quoted 5 lines
Quoted 3 lines
Good point. Maybe use timestamp instead?
So, we can know a bigger timestamp value corresponds to a VM created later. Also since VMs are created in a single thread, they can't have the sameWhy must all VMs be created in a single thread?
I happened to encounter this warning when I wrote a testcase which runs in a single process but needs to create several VMs sequentially for testing various cases.
Quoted 46 lines
virt/kvm/kvm_main.c static int kvm_create_vm_debugfs(struct kvm *kvm, const char *fdname) · line 1020
Quoted 12 lines
timestamp value.
virt/kvm/kvm_main.c +1
-1@@ -1017,7 +1017,7 @@ static int kvm_create_vm_debugfs(struct kvm *kvm, const char *fdname) {static DEFINE_MUTEX (kvm_debugfs_lock);struct dentry *dent; -char dir_name[ITOA_MAX_LEN *2 ]; +char dir_name[ITOA_MAX_LEN *4 ];Hmmm.
Quoted 15 lines
virt/kvm/kvm_main.c +2
-1struct kvm_stat_data *stat_data;const struct _kvm_stats_desc *pdesc;int i, ret = -ENOMEM ;@@ -1027,7 +1027,8 @@ static int kvm_create_vm_debugfs(struct kvm *kvm, const char *fdname)if (!debugfs_initialized ())return 0 ; -snprintf (dir_name,sizeof (dir_name),"%d-%s" ,task_pid_nr (current), fdname); +snprintf (dir_name,sizeof (dir_name),"%d-%s-%llx" ,task_pid_nr (current), + fdname,ktime_get_ns ());mutex_lock (&kvm_debugfs_lock); dent =debugfs_lookup (dir_name, kvm_debugfs_dir);if (dent) {Quoted 5 lines
Quoted 1 line
Should the vm
stats_idalso be updated to be unique and use the same scheme?I don't think so. Unlike debugfs paths that will be accessed directly by userspace, the
stats_idis used by anonymous inode files openned with unique fd numbers in a single process. So, duplicated names should be ok.Just sticking my oar in here, since I'm interested in how we identify VMs and vCPUs for use by other kernel subsystems:
The two proposed approaches may not guarantee uniqueness. The PID is not unique because a single process can create multiple VMs. Possibly, VMs can outlive the PID of the creating process (if a KVM fd is passed to another process, and the original process terminates). The fd is not unique because fds can be moved around with
dup()and reallocated. Random numbers are not unique (only _probably_ unique.)
Quoted 3 lines
If VMs are created from two threads simultaneously and if ownership of
kvm_debugfs_lockcan move fast enough, or there is enough skid or imprecision in the timestamp, then canktime_get_ns()be non-unique too?
I don't find a way of having 2 VMs to the same
"PID + fd number + ktime_get_ns()".
Could you elaborate if you think it's possible?
Quoted 2 lines
_Maybe_ this is good enough for debugfs -- which is often treated in a best-effort way by kernel code.
Right, so I tried to make it unique while keeping least effort to KVM.
Quoted 7 lines
However, if we really want a robust ID that can not only distinguish between VMs but can also tell us reliably (and usefully) which is which even when multiple VMs are created by a single process, I think KVM userspace needs to be able to set and/or retrieve the ID, or part of it.
It might make sense to define a new kind of identifier for such purposes.
Hmm, it may not be worthwhile in my scenario.
Quoted 4 lines
I'm guessing that this is out of the scope for this patch, though.
Cheers
---Dave