KVM: Rework how halt-polling stats and per-VM debugfs use memory
Subsystems (7)
v2 cover KVM: Rework how halt-polling stats and per-VM debugfs use memory
This series replaces the halt-polling histogram Kconfig patch, as a first step towards a general plan for KVM stats.
Take a VM with 8 vCPUs on x86. Each vCPU has 1104 bytes of stats, 768 of which are the three halt-polling histograms, i.e. 6KiB for the VM. The VM also gets ~95 debugfs entries (one file per stat, plus a directory and a few files per vCPU), each costing a dentry and an inode, i.e. ~70KiB, all duplicating the binary stats.
The plan:
1. Counters stay embedded in the stats structs; they're cheap and monitoring wants them always on.
2. Arrays and histograms move to a separately allocated block, controlled by kvm.extended_stats (default on). The binary stats ABI doesn't change; the values read as zero if the block isn't allocated. (patch 4)
3. Debug-only data goes to tracepoints and is aggregated with BPF. Adding the halt-polling time to kvm_vcpu_wakeup is enough to rebuild the three histograms. (patch 1)
In addition, kvm.per_vm_debugfs (default on) allows skipping the per-VM debugfs directories. (patch 3) Patch 2 fixes an arm64 bug that this makes easy to hit: nested stage-2 ptdump files end up in the debugfs root if the VM has no debugfs directory.
With patch 4, struct vcpu_vmx shrinks from 8384 to 7616 bytes and struct vcpu_svm from 6728 to 5968 bytes.
Tested on Intel with KVM selftests in all three configurations (default, kvm.extended_stats=0, kvm.per_vm_debugfs=0). Build-tested on powerpc and arm64.
v2:
- Add patch 2, fixing nested_mmu* ptdump files being created in the debugfs root and left dangling when the VM has no debugfs directory (reported by Sashiko AI review).
Peng Hao (4):
KVM: Add the halt-polling time to the kvm_vcpu_wakeup tracepoint
KVM: arm64: nv: Skip nested stage-2 ptdump files without a debugfs dir
KVM: Add a module param to disable the per-VM debugfs directories
KVM: Move the halt-polling histograms out of struct kvm_vcpu_stat
.../admin-guide/kernel-parameters.txt | 13 ++++ arch/arm64/kvm/ptdump.c | 4 + arch/powerpc/kvm/book3s_hv.c | 11 ++- include/linux/kvm_host.h | 24 +++++- include/linux/kvm_types.h | 10 ++- include/trace/events/kvm.h | 16 ++-- virt/kvm/binary_stats.c | 29 +++++++- virt/kvm/kvm_main.c | 73 +++++++++++++++---- 8 files changed, 148 insertions(+), 32 deletions(-)
base-commit: d21eef2084a9af87ac2a783bc7c2b175ba114e4e
Signature · 1 line
2.43.7
RFC PATCH v2 1/4 KVM: Add the halt-polling time to the kvm_vcpu_wakeup tracepoint
kvm_vcpu_wakeup only reports the total halt time, which includes both the time spent halt-polling and the time spent blocking. Report the halt-polling time as well, so that the split between polling and waiting can be derived from the tracepoint, e.g. to build histograms equivalent to the halt_poll_success_hist, halt_poll_fail_hist and halt_wait_hist stats with BPF.
--- include/trace/events/kvm.h | 16 +++++++++------- virt/kvm/kvm_main.c | 3 ++- 2 files changed, 11 insertions(+), 8 deletions(-)
include/trace/events/kvm.h +9 -7
@@ -47,24 +47,26 @@ TRACE_EVENT(kvm_userspace_exit, );TRACE_EVENT (kvm_vcpu_wakeup, - TP_PROTO(__u64 ns, bool waited, bool valid), -TP_ARGS (ns, waited, valid), + TP_PROTO(__u64 ns, __u64 poll_ns, bool waited, bool valid), +TP_ARGS (ns, poll_ns, waited, valid),TP_STRUCT__entry ( - __field( __u64, ns ) - __field( bool, waited ) -__field ( bool, valid ) + __field(__u64, ns) + __field(__u64, poll_ns) + __field(bool, waited) +__field (bool, valid) ),TP_fast_assign ( __entry->ns = ns; + __entry->poll_ns = poll_ns; __entry->waited = waited; __entry->valid = valid; ), -TP_printk ("%s time %lld ns, polling %s" , +TP_printk ("%s time %lld ns (poll %lld ns), polling %s" , __entry->waited ?"wait" :"poll" , - __entry->ns, + __entry->ns, __entry->poll_ns, __entry->valid ?"valid" :"invalid" ) );
virt/kvm/kvm_main.c +2 -1
@@ -3888,7 +3888,8 @@ void kvm_vcpu_halt(struct kvm_vcpu *vcpu) } } -trace_kvm_vcpu_wakeup (halt_ns, waited,vcpu_valid_wakeup (vcpu)); +trace_kvm_vcpu_wakeup (halt_ns,ktime_to_ns (ktime_sub (poll_end, start)), + waited,vcpu_valid_wakeup (vcpu)); }EXPORT_SYMBOL_FOR_KVM_INTERNAL (kvm_vcpu_halt);
Signature · 1 line
2.43.7
RFC PATCH v2 2/4 KVM: arm64: nv: Skip nested stage-2 ptdump files without a debugfs dir
In reply to Peng Haokvm->arch.debugfs_nv_dentry is only set if the VM's debugfs directory was created, but kvm_nested_s2_ptdump_create_debugfs() uses it unconditionally. If it is NULL, e.g. because the VM's debugfs directory name was a duplicate, the nested_mmu* files are created in the root of debugfs and are never removed, leaving dangling pointers to the nested MMUs after the VM is destroyed.
Skip creating the files if there is no "nested" directory.
--- arch/arm64/kvm/ptdump.c | 4 ++++ 1 file changed, 4 insertions(+)
arch/arm64/kvm/ptdump.c +4 -0
@@ -312,6 +312,10 @@ void kvm_nested_s2_ptdump_create_debugfs(struct kvm_s2_mmu *mmu, int idx) {char file_name[S2FNAMESZ ]; +/* The "nested" directory doesn't exist without a VM debugfs directory. */ +if (IS_ERR_OR_NULL (mmu->arch->debugfs_nv_dentry)) +return ; +snprintf (file_name,sizeof (file_name),"nested_mmu%d" , idx); debugfs_create_file(file_name, 0400,
Signature · 1 line
2.43.7
RFC PATCH v2 3/4 KVM: Add a module param to disable the per-VM debugfs directories
In reply to Peng HaoEvery VM gets a debugfs directory with one file per stat, plus a directory and a few files per vCPU, i.e. ~95 entries for an 8-vCPU VM on x86. Each entry costs a dentry and an inode, and the stats files duplicate the binary stats.
Add kvm.per_vm_debugfs (default on) so that these directories can be skipped.
--- Documentation/admin-guide/kernel-parameters.txt | 7 +++++++ virt/kvm/kvm_main.c | 12 ++++++++++-- 2 files changed, 17 insertions(+), 2 deletions(-)
Documentation/admin-guide/kernel-parameters.txt +7 -0
@@ -3211,6 +3211,13 @@ Kernel parameters If the value is 0 (the default), KVM will pick a period based on the ratio, such that a page is zapped after 1 hour on average. + kvm.per_vm_debugfs= + [KVM] Create a debugfs directory for each VM and vCPU, + containing per-VM stats and architecture specific + debug files. Global stats in debugfs and binary stats + are not affected. + Default is Y (on). + kvm-{amd,intel}.enable_mediated_pmu=[KVM,AMD,INTEL] If enabled, KVM will provide a mediated virtual PMU, instead of the default perf-based virtual PMU (if
virt/kvm/kvm_main.c +10 -2
@@ -104,6 +104,14 @@ EXPORT_SYMBOL_FOR_KVM_INTERNAL(halt_poll_ns_shrink);static bool __ro_after_init allow_unsafe_mappings;module_param (allow_unsafe_mappings, bool,0444 ); +/* +* Create a debugfs directory for every VM and vCPU. The per-VM stats files +* duplicate the binary stats, and each file costs a dentry and an inode, which +* adds up when running many VMs. +*/ +static bool __ro_after_init per_vm_debugfs = true; +module_param (per_vm_debugfs, bool,0444 ); +#ifdef CONFIG_KVM_PRIVATE_MEMATTR_VM bool __ro_after_init gmem_in_place_conversion = false;module_param (gmem_in_place_conversion, bool,0444 );
@@ -1024,7 +1032,7 @@ static int kvm_create_vm_debugfs(struct kvm *kvm, const char *fdname)int kvm_debugfs_num_entries = kvm_vm_stats_header.num_desc + kvm_vcpu_stats_header.num_desc; -if (!debugfs_initialized ()) +if (!debugfs_initialized () || !per_vm_debugfs)return 0 ;snprintf (dir_name,sizeof (dir_name),"%d-%s" ,task_pid_nr (current), fdname);
@@ -4235,7 +4243,7 @@ static void kvm_create_vcpu_debugfs(struct kvm_vcpu *vcpu)struct dentry *debugfs_dentry;char dir_name[ITOA_MAX_LEN *2 ]; -if (!debugfs_initialized ()) +if (!debugfs_initialized () || !per_vm_debugfs)return ;snprintf (dir_name,sizeof (dir_name),"vcpu%d" , vcpu->vcpu_id);
Signature · 1 line
2.43.7
RFC PATCH v2 4/4 KVM: Move the halt-polling histograms out of struct kvm_vcpu_stat
In reply to Peng HaoThe halt_poll_success_hist, halt_poll_fail_hist and halt_wait_hist histograms take 768 bytes in every vCPU, out of 1104 bytes of stats on x86. Move them into a separately allocated struct kvm_vcpu_stat_ext, and add kvm.extended_stats (default on) so that it can be skipped.
The descriptors don't change and the extended stats are laid out after struct kvm_vcpu_stat in the stats data block, so the binary stats ABI is unchanged; if the block isn't allocated the histograms read as zero.
This shrinks struct kvm_vcpu from 5776 to 5016 bytes, struct vcpu_vmx from 8384 to 7616 bytes and struct vcpu_svm from 6728 to 5968 bytes.
--- .../admin-guide/kernel-parameters.txt | 6 ++ arch/powerpc/kvm/book3s_hv.c | 11 ++-- include/linux/kvm_host.h | 24 ++++++-- include/linux/kvm_types.h | 10 +++- virt/kvm/binary_stats.c | 29 +++++++++- virt/kvm/kvm_main.c | 58 +++++++++++++++---- 6 files changed, 116 insertions(+), 22 deletions(-)
Documentation/admin-guide/kernel-parameters.txt +6 -0
@@ -3159,6 +3159,12 @@ Kernel parameters kvm.enable_vmware_backdoor=[KVM] Support VMware backdoor PV interface. Default is false (don't support). + kvm.extended_stats= + [KVM] Allocate the extended per-vCPU stats, e.g. the + halt-polling histograms. If disabled, the extended + stats are still reported but always read as zero. + Default is Y (on). + kvm.gmem_in_place_conversion= [KVM] Controls whether KVM enables in-place conversion support for guest_memfd and tracks the private/shared
arch/powerpc/kvm/book3s_hv.c +5 -6
@@ -4684,16 +4684,15 @@ static void kvmppc_vcore_blocked(struct kvmppc_vcore *vc)if (do_sleep) { vc->runner->stat.generic.halt_wait_ns +=ktime_to_ns (cur) -ktime_to_ns (start_wait); -KVM_STATS_LOG_HIST_UPDATE ( - vc->runner->stat.generic.halt_wait_hist, +KVM_VCPU_STATS_EXT_LOG_HIST_UPDATE (vc->runner, halt_wait_hist,ktime_to_ns (cur) -ktime_to_ns (start_wait));/* Attribute failed poll time */ if (vc->halt_poll_ns) { vc->runner->stat.generic.halt_poll_fail_ns +=ktime_to_ns (start_wait) -ktime_to_ns (start_poll); -KVM_STATS_LOG_HIST_UPDATE ( - vc->runner->stat.generic.halt_poll_fail_hist, +KVM_VCPU_STATS_EXT_LOG_HIST_UPDATE (vc->runner, + halt_poll_fail_hist,ktime_to_ns (start_wait) -ktime_to_ns (start_poll)); }
@@ -4703,8 +4702,8 @@ static void kvmppc_vcore_blocked(struct kvmppc_vcore *vc) vc->runner->stat.generic.halt_poll_success_ns +=ktime_to_ns (cur) -ktime_to_ns (start_poll); -KVM_STATS_LOG_HIST_UPDATE ( - vc->runner->stat.generic.halt_poll_success_hist, +KVM_VCPU_STATS_EXT_LOG_HIST_UPDATE (vc->runner, + halt_poll_success_hist,ktime_to_ns (cur) -ktime_to_ns (start_poll)); } }
include/linux/kvm_host.h +20 -4
@@ -391,6 +391,7 @@ struct kvm_vcpu {bool scheduled_out;struct kvm_vcpu_arch arch;struct kvm_vcpu_stat stat; +struct kvm_vcpu_stat_ext *stat_ext;char stats_id[KVM_STATS_NAME_SIZE ];struct kvm_dirty_ring dirty_ring;
@@ -2029,7 +2030,15 @@ struct kvm_stat_data { .offset = offsetof(struct kvm_vcpu_stat, stat), \ .name = #stat, \ } -/* SCOPE: VM, VM_GENERIC, VCPU, VCPU_GENERIC */ +/* Extended vCPU stats are laid out after struct kvm_vcpu_stat. */ +#define VCPU_EXT_STATS_DESC (stat, type, unit, base, exp, sz, bsz) \ +{ \ + STATS_DESC_COMMON(type, unit, base, exp, sz, bsz), \ + .offset = sizeof(struct kvm_vcpu_stat) + \ + offsetof(struct kvm_vcpu_stat_ext, stat), \ + .name = #stat, \ +} +/* SCOPE: VM, VM_GENERIC, VCPU, VCPU_GENERIC, VCPU_EXT */ #define STATS_DESC (SCOPE , stat, type, unit, base, exp, sz, bsz) \ SCOPE##_STATS_DESC(stat, type, unit, base, exp, sz, bsz)
@@ -2096,17 +2105,18 @@ struct kvm_stat_data {STATS_DESC_TIME_NSEC (VCPU_GENERIC , halt_poll_success_ns), \STATS_DESC_TIME_NSEC (VCPU_GENERIC , halt_poll_fail_ns), \STATS_DESC_TIME_NSEC (VCPU_GENERIC , halt_wait_ns), \ -STATS_DESC_LOGHIST_TIME_NSEC (VCPU_GENERIC , halt_poll_success_hist, \ +STATS_DESC_LOGHIST_TIME_NSEC (VCPU_EXT , halt_poll_success_hist, \HALT_POLL_HIST_COUNT ), \ -STATS_DESC_LOGHIST_TIME_NSEC (VCPU_GENERIC , halt_poll_fail_hist, \ +STATS_DESC_LOGHIST_TIME_NSEC (VCPU_EXT , halt_poll_fail_hist, \HALT_POLL_HIST_COUNT ), \ -STATS_DESC_LOGHIST_TIME_NSEC (VCPU_GENERIC , halt_wait_hist, \ +STATS_DESC_LOGHIST_TIME_NSEC (VCPU_EXT , halt_wait_hist, \HALT_POLL_HIST_COUNT ), \ STATS_DESC_IBOOLEAN(VCPU_GENERIC, blocking)ssize_t kvm_stats_read (char *id,const struct kvm_stats_header *header,const struct kvm_stats_desc *desc,void *stats,size_t size_stats, +void *stats_ext,size_t size_stats_ext,char __user *user_buffer,size_t size,loff_t *offset);/**
@@ -2148,6 +2158,12 @@ static inline void kvm_stats_log_hist_update(u64 *data, size_t size, u64 value)#define KVM_STATS_LOG_HIST_UPDATE (array, value) \ kvm_stats_log_hist_update(array, ARRAY_SIZE(array), value) +#define KVM_VCPU_STATS_EXT_LOG_HIST_UPDATE (vcpu, hist, value) \ +do { \ + if ((vcpu)->stat_ext) \ + KVM_STATS_LOG_HIST_UPDATE((vcpu)->stat_ext->hist, value); \ +} while (0) +extern const struct kvm_stats_header kvm_vm_stats_header;extern const struct kvm_stats_desc kvm_vm_stats_desc[];
include/linux/kvm_types.h +9 -1
@@ -133,10 +133,18 @@ struct kvm_vcpu_stat_generic {u64 halt_poll_success_ns;u64 halt_poll_fail_ns;u64 halt_wait_ns; +u64 blocking; +}; + +/* +* vCPU stats that are too big to embed in every vCPU, e.g. histograms. They +* are allocated only if kvm.extended_stats is enabled, and read as zero +* otherwise. +*/ +struct kvm_vcpu_stat_ext {u64 halt_poll_success_hist[HALT_POLL_HIST_COUNT ];u64 halt_poll_fail_hist[HALT_POLL_HIST_COUNT ];u64 halt_wait_hist[HALT_POLL_HIST_COUNT ]; -u64 blocking; };#define KVM_STATS_NAME_SIZE 48
virt/kvm/binary_stats.c +28 -1
@@ -19,6 +19,9 @@* @desc: start address of an array of stats descriptors for a vm or a vcpu * @stats: start address of stats data block for a vm or a vcpu * @size_stats: the size of stats data block pointed by @stats +* @stats_ext: start address of the extended stats data block, which follows +* the stats data block, or NULL if it isn't allocated +* @size_stats_ext: the size of the extended stats data block * @user_buffer: start address of userspace buffer * @size: requested read size from userspace * @offset: the start position from which the content will be read for the
@@ -34,6 +37,9 @@* +-------------+ * | Stats Data | * +-------------+ +* | Extended | +* | Stats Data | +* +-------------+ * Although this function allows userspace to read any amount of data (as long * as in the limit) from any position, the typical usage would follow below * steps:
@@ -52,6 +58,7 @@ssize_t kvm_stats_read (char *id,const struct kvm_stats_header *header,const struct kvm_stats_desc *desc,void *stats,size_t size_stats, +void *stats_ext,size_t size_stats_ext,char __user *user_buffer,size_t size,loff_t *offset) {ssize_t len;
@@ -66,7 +73,8 @@ ssize_t kvm_stats_read(char *id, const struct kvm_stats_header *header, size_header =sizeof (*header); size_desc = header->num_desc *sizeof (*desc); - len =KVM_STATS_NAME_SIZE + size_header + size_desc + size_stats - pos; + len =KVM_STATS_NAME_SIZE + size_header + size_desc + size_stats + + size_stats_ext - pos; len =min (len, remain);if (len <=0 )return 0 ;
@@ -136,6 +144,25 @@ ssize_t kvm_stats_read(char *id, const struct kvm_stats_header *header, src = stats + pos - header->data_offset;if (copy_to_user (dest, src, copylen))return -EFAULT ; + remain -= copylen; + pos += copylen; + dest += copylen; + } + +/* +* Copy extended kvm stats values, which read as zero if the extended +* stats data block isn't allocated. +*/ + copylen = header->data_offset + size_stats + size_stats_ext - pos; + copylen =min (copylen, remain); +if (copylen >0 ) { +if (stats_ext) { + src = stats_ext + pos - header->data_offset - size_stats; +if (copy_to_user (dest, src, copylen)) +return -EFAULT ; + }else if (clear_user (dest, copylen)) { +return -EFAULT ; + } pos += copylen; }
virt/kvm/kvm_main.c +48 -10
132 lines not on this page · raw message
Signature · 1 line
2.43.7