Tracefs support for pKVM
Vincent Donnefort
maintainerv2 cover Tracefs support for pKVM
The growing set of features supported by the hypervisor in protected mode necessitates debugging and profiling tools. Tracefs is the ideal candidate for this task:
* It is simple to use and to script.
* It is supported by various tools, from the trace-cmd CLI to the Android web-based perfetto.
* The ring-buffer, where are stored trace events consists of linked pages, making it an ideal structure for sharing between kernel and hypervisor.
This series introduces a method to create events and to generate them from the hypervisor (hyp_enter/hyp_exit given as an example) as well as a Tracefs user-space interface to read them.
A presentation was given on this matter during the tracing summit in 2022. [1]
1. ring-buffer
--------------
To setup the per-cpu ring-buffers, a new interface is created:
ring_buffer_remote: Describes what the kernel needs to know about the
remote writer, that is, the set of pages forming the ring-buffer and a callback for the reader/head swapping (enables consuming read)
ring_buffer_remote(): Creates a read-only ring-buffer from a
ring_buffer_remote.
To keep the internals of struct ring_buffer in sync with the writer, the meta-page is used. It was originally introduced to enable user-space mapping of the ring-buffer [1]. In this case, the kernel is not the producer anymore but the reader. The function to read that meta-page is:
ring_buffer_poll_remote():
Update struct ring_buffer based on the writer meta-page. Wake-up readers if necessary.
The kernel has to poll the meta-page to be notified of newly written events.
2. Tracefs interface
--------------------
The interface is a hypervisor/ folder at the root of the tracefs mount point. This folder is like an instance and you'll find there a subset of the regular Tracefs user-space interface:
hypervisor/
buffer_size_kb
trace_clock
trace_pipe
trace_pipe_raw
trace
per_cpu/
cpuX/
trace
trace_pipe
trace_pipe_raw
events/
hypervisor/
hyp_enter/
enable
id
Behind the scenes, kvm/hyp_trace.c must rebuild the tracing hierarchy without relying on kernel/trace/trace.c. This is due to fundamental differences:
* Hypervisor tracing doesn't support trace_array's system-specific features (snapshots, tracers, etc.).
* Logged event formats differ (e.g., no PID in hypervisor events).
* Buffer operations require specific hypervisor interactions.
3. Events
---------
In the hypervisor, "hyp events" can be generated with trace_<event_name> in a similar fashion to what the kernel does. They're also created with similar macros than the kernel (see kvm_hypevents.h)
HYP_EVENT("foboar",
HE_PROTO(void),
HE_STRUCT(),
HE_ASSIGN(),
HE_PRINTK(" "))
Despite the apparent similarities with TRACE_EVENT(), those macros internally differs: they must be used in parallel between the hypervisor (for the writing part) and the kernel (for the reading part) which makes it difficult to share anything with their kernel counterpart.
Also, events directory isn't using eventfs.
4. Limitations:
---------------
Non-consuming reading of the buffer isn't supported (i.e. cat trace -> -EPERM) due to current the lack of support in the ring-buffer meta-page.
[1] https://tracingsummit.org/ts/2022/hypervisortracing/
[2] https://lore.kernel.org/all/[email protected]/
Changes since v1: https://lore.kernel.org/all/[email protected]/
- Add 128-bits mult fallback in the unlikely event of an overflow. (John)
- Fix ELF section sort.
- __always_inline trace_* event macros.
- Fix events/<event>/enable permissions.
- Rename ring-buffer "writer" to "remote".
- Rename CONFIG_PROTECTED_NVHE_TESTING to PKVM_SELFTEST to align with Quentin's upcoming selftest
- Rebase on 6.13-rc3.
Changes since RFC: https://lore.kernel.org/all/[email protected]/
- hypervisor trace clock:
- mult/shift computed in hyp_trace.c. (John)
- Update clock when it deviates from kernel boot clock. (John)
- Add trace_clock file.
- Separate patch for better readability.
- Add a proper reset interface which does not need to teardown the tracing buffers. (Steven)
- Return -EPERM on trace access. (Steven)
- Add per-cpu trace file.
- Automatically teardown and free the tracing buffer when it is empty, without readers and not currently tracing.
- Show in buffer_size_kb if the buffer is loaded in the hypervisor or not.
- Extend tests to cover reset and unload.
- CC timekeeping folks on relevant patches (Marc)
Vincent Donnefort (12):
ring-buffer: Check for empty ring-buffer with rb_num_of_entries()
ring-buffer: Introducing ring-buffer remote
ring-buffer: Expose buffer_data_page material
KVM: arm64: Support unaligned fixmap in the nVHE hyp
KVM: arm64: Add clock support in the nVHE hyp
KVM: arm64: Add tracing support for the pKVM hyp
KVM: arm64: Add hyp tracing to tracefs
KVM: arm64: Add clock for hyp tracefs
KVM: arm64: Add raw interface for hyp tracefs
KVM: arm64: Add trace interface for hyp tracefs
KVM: arm64: Add support for hyp events
KVM: arm64: Add kselftest for tracefs hyp tracefs
arch/arm64/include/asm/kvm_asm.h | 8 + arch/arm64/include/asm/kvm_define_hypevents.h | 61 ++ arch/arm64/include/asm/kvm_hyp.h | 1 - arch/arm64/include/asm/kvm_hypevents.h | 41 + arch/arm64/include/asm/kvm_hypevents_defs.h | 41 + arch/arm64/include/asm/kvm_hyptrace.h | 37 + arch/arm64/kernel/image-vars.h | 4 + arch/arm64/kernel/vmlinux.lds.S | 18 + arch/arm64/kvm/Kconfig | 10 + arch/arm64/kvm/Makefile | 2 + arch/arm64/kvm/arm.c | 6 + arch/arm64/kvm/hyp/hyp-constants.c | 4 + arch/arm64/kvm/hyp/include/nvhe/arm-smccc.h | 13 + arch/arm64/kvm/hyp/include/nvhe/clock.h | 16 + .../kvm/hyp/include/nvhe/define_events.h | 21 + arch/arm64/kvm/hyp/include/nvhe/trace.h | 60 ++ arch/arm64/kvm/hyp/nvhe/Makefile | 1 + arch/arm64/kvm/hyp/nvhe/clock.c | 65 ++ arch/arm64/kvm/hyp/nvhe/events.c | 35 + arch/arm64/kvm/hyp/nvhe/ffa.c | 2 +- arch/arm64/kvm/hyp/nvhe/hyp-main.c | 85 ++ arch/arm64/kvm/hyp/nvhe/hyp.lds.S | 6 + arch/arm64/kvm/hyp/nvhe/mm.c | 2 +- arch/arm64/kvm/hyp/nvhe/psci-relay.c | 14 +- arch/arm64/kvm/hyp/nvhe/switch.c | 5 +- arch/arm64/kvm/hyp/nvhe/trace.c | 660 ++++++++++++ arch/arm64/kvm/hyp_events.c | 159 +++ arch/arm64/kvm/hyp_trace.c | 988 ++++++++++++++++++ arch/arm64/kvm/hyp_trace.h | 15 + include/linux/ring_buffer.h | 99 +- kernel/trace/ring_buffer.c | 308 ++++-- tools/testing/selftests/hyp-trace/Makefile | 6 + tools/testing/selftests/hyp-trace/config | 4 + .../selftests/hyp-trace/hyp-trace-test | 254 +++++ 34 files changed, 2955 insertions(+), 96 deletions(-) create mode 100644 arch/arm64/include/asm/kvm_define_hypevents.h create mode 100644 arch/arm64/include/asm/kvm_hypevents.h create mode 100644 arch/arm64/include/asm/kvm_hypevents_defs.h create mode 100644 arch/arm64/include/asm/kvm_hyptrace.h create mode 100644 arch/arm64/kvm/hyp/include/nvhe/arm-smccc.h create mode 100644 arch/arm64/kvm/hyp/include/nvhe/clock.h create mode 100644 arch/arm64/kvm/hyp/include/nvhe/define_events.h create mode 100644 arch/arm64/kvm/hyp/include/nvhe/trace.h create mode 100644 arch/arm64/kvm/hyp/nvhe/clock.c create mode 100644 arch/arm64/kvm/hyp/nvhe/events.c create mode 100644 arch/arm64/kvm/hyp/nvhe/trace.c create mode 100644 arch/arm64/kvm/hyp_events.c create mode 100644 arch/arm64/kvm/hyp_trace.c create mode 100644 arch/arm64/kvm/hyp_trace.h create mode 100644 tools/testing/selftests/hyp-trace/Makefile create mode 100644 tools/testing/selftests/hyp-trace/config create mode 100755 tools/testing/selftests/hyp-trace/hyp-trace-test
base-commit: 09a0fa92e5b45e99cf435b2fbf5ebcf889cf8780
Signature · 1 line
2.47.1.613.gc27f4b7a9f-goog
Vincent Donnefort
maintainerPATCH v2 1/12 ring-buffer: Check for empty ring-buffer with rb_num_of_entries()
Currently there are two ways of identifying an empty ring-buffer. One relying on the current status of the commit / reader page (rb_per_cpu_empty()) and the other on the write and read counters (rb_num_of_entries() used in rb_get_reader_page()).
with rb_num_of_entries(). This intends to ease later introduction of ring-buffer writers which are out of the kernel control and with whom, the only information available is through the meta-page counters.
kernel/trace/ring_buffer.c +14 -45
@@ -4682,40 +4682,22 @@ int ring_buffer_write(struct trace_buffer *buffer, }EXPORT_SYMBOL_GPL (ring_buffer_write); -static bool rb_per_cpu_empty (struct ring_buffer_per_cpu *cpu_buffer) +/* +* The total entries in the ring buffer is the running counter +* of entries entered into the ring buffer, minus the sum of +* the entries read from the ring buffer and the number of +* entries that were overwritten. +*/ +static inline unsigned long +rb_num_of_entries (struct ring_buffer_per_cpu *cpu_buffer) { -struct buffer_page *reader = cpu_buffer->reader_page; -struct buffer_page *head =rb_set_head_page (cpu_buffer); -struct buffer_page *commit = cpu_buffer->commit_page; - -/* In case of error, head will be NULL */ -if (unlikely (!head)) -return true; - -/* Reader should exhaust content in reader page */ -if (reader->read !=rb_page_size (reader)) -return false; - -/* -* If writers are committing on the reader page, knowing all -* committed content has been read, the ring buffer is empty. -*/ -if (commit == reader) -return true; - -/* -* If writers are committing on a page other than reader page -* and head page, there should always be content to read. -*/ -if (commit != head) -return false; +return local_read (&cpu_buffer->entries) - + (local_read (&cpu_buffer->overrun) + cpu_buffer->read); +} -/* -* Writers are committing on the head page, we just need -* to care about there're committed data, and the reader will -* swap reader page with head page when it is to read data. -*/ -return rb_page_commit (commit) ==0 ; +static bool rb_per_cpu_empty (struct ring_buffer_per_cpu *cpu_buffer) +{ +return !rb_num_of_entries (cpu_buffer); }/**
@@ -4861,19 +4843,6 @@ void ring_buffer_record_enable_cpu(struct trace_buffer *buffer, int cpu) }EXPORT_SYMBOL_GPL (ring_buffer_record_enable_cpu); -/* -* The total entries in the ring buffer is the running counter -* of entries entered into the ring buffer, minus the sum of -* the entries read from the ring buffer and the number of -* entries that were overwritten. -*/ -static inline unsigned long -rb_num_of_entries (struct ring_buffer_per_cpu *cpu_buffer) -{ -return local_read (&cpu_buffer->entries) - - (local_read (&cpu_buffer->overrun) + cpu_buffer->read); -} -/** * ring_buffer_oldest_event_ts - get the oldest event timestamp from the buffer * @buffer: The ring buffer
Signature · 1 line
2.47.1.613.gc27f4b7a9f-goog
Vincent Donnefort
maintainerPATCH v2 2/12 ring-buffer: Introducing ring-buffer remote
In reply to Vincent DonnefortA ring-buffer remote is an entity outside of the kernel (most likely a firmware or a hypervisor) capable of writing events in a ring-buffer following the same format as the tracefs ring-buffer.
To setup the ring-buffer on the kernel side, a description of the pages (struct trace_page_desc) is necessary. A callback (get_reader_page) must also be provided. It is called whenever it is done reading the previous reader page.
It is expected from the remote to keep the meta-page updated.
include/linux/ring_buffer.h +54 -0
@@ -248,4 +248,58 @@ int ring_buffer_map(struct trace_buffer *buffer, int cpu, struct vm_area_struct *vma);int ring_buffer_unmap (struct trace_buffer *buffer,int cpu);int ring_buffer_map_get_reader (struct trace_buffer *buffer,int cpu); + +#define meta_pages_lost (__meta) \ + ((__meta)->Reserved1) +#define meta_pages_touched (__meta) \ + ((__meta)->Reserved2) + +struct rb_page_desc { +int cpu; +int nr_page_va;/* exclude the meta page */ +unsigned long meta_va; +unsigned long page_va[]; +}; + +struct trace_page_desc { +int nr_cpus; +char __data[];/* list of rb_page_desc */ +}; + +static inline +struct rb_page_desc *__next_rb_page_desc (struct rb_page_desc *pdesc) +{ +size_t len =struct_size (pdesc, page_va, pdesc->nr_page_va); + +return (struct rb_page_desc *)((void *)pdesc + len); +} + +static inline +struct rb_page_desc *__first_rb_page_desc (struct trace_page_desc *trace_pdesc) +{ +return (struct rb_page_desc *)(&trace_pdesc->__data[0 ]); +} + +#define for_each_rb_page_desc (__pdesc, __cpu, __trace_pdesc) \ + for (__pdesc = __first_rb_page_desc(__trace_pdesc), __cpu = 0; \ + __cpu < (__trace_pdesc)->nr_cpus; \ + __cpu++, __pdesc = __next_rb_page_desc(__pdesc)) + +struct ring_buffer_remote { +struct trace_page_desc *pdesc; +int (*get_reader_page)(int cpu); +int (*reset)(int cpu); +}; + +int ring_buffer_poll_remote (struct trace_buffer *buffer,int cpu); + +struct trace_buffer * +__ring_buffer_alloc_remote (struct ring_buffer_remote *remote, +struct lock_class_key *key); + +#define ring_buffer_remote (remote) \ +({ \ + static struct lock_class_key __key; \ + __ring_buffer_alloc_remote(remote, &__key); \ +})#endif /* _LINUX_RING_BUFFER_H */
kernel/trace/ring_buffer.c +207 -7
323 lines not on this page · raw message
Signature · 1 line
2.47.1.613.gc27f4b7a9f-goog
kernel test robot
build botRe: [PATCH v2 02/12] ring-buffer: Introducing ring-buffer remote
kernel test robot noticed the following build warnings:
Hi Vincent,
kernel test robot noticed the following build warnings:
[auto build test WARNING on 09a0fa92e5b45e99cf435b2fbf5ebcf889cf8780]
url: https://github.com/intel-lab-lkp/linux/commits/Vincent-Donnefort/ring-buffer-Check-for-empty-ring-buffer-with-rb_num_of_entries/20250108-200217 base: 09a0fa92e5b45e99cf435b2fbf5ebcf889cf8780 patch link: https://lore.kernel.org/r/20250108114536.627715-3-vdonnefort%40google.com patch subject: [PATCH v2 02/12] ring-buffer: Introducing ring-buffer remote config: x86_64-defconfig (https://download.01.org/0day-ci/archive/20250110/[email protected]/config) compiler: gcc-11 (Debian 11.3.0-12) 11.3.0 reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20250110/[email protected]/reproduce)
If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <[email protected]>
| Closes: https://lore.kernel.org/oe-kbuild-all/[email protected]/
All warnings (new ones prefixed by >>):
>> kernel/trace/ring_buffer.c:2535: warning: Function parameter or struct member 'remote' not described in '__ring_buffer_alloc_remote' >> kernel/trace/ring_buffer.c:2535: warning: Function parameter or struct member 'key' not described in '__ring_buffer_alloc_remote'
vim +2535 kernel/trace/ring_buffer.c
2528
2529 /**
2530 * __ring_buffer_alloc_remote
2531 *
2532 */
2533 struct trace_buffer *__ring_buffer_alloc_remote(struct ring_buffer_remote *remote,
2534 struct lock_class_key *key)
> 2535 { 2536 return alloc_buffer(0, 0, 0, 0, 0, key, remote);
2537 }
2538
Signature · 2 lines
0-DAY CI Kernel Test Service https://github.com/intel/lkp-tests/wiki
Vincent Donnefort
maintainerPATCH v2 3/12 ring-buffer: Expose buffer_data_page material
In reply to Vincent DonnefortIn preparation for allowing the write of ring-buffer compliant pages outside of ring_buffer.c, move to the header, struct buffer_data_page and timestamp encoding functions into the publicly available ring_buffer.h.
include/linux/ring_buffer.h +44 -1
@@ -3,8 +3,10 @@#define _LINUX_RING_BUFFER_H#include <linux/mm.h> -#include <linux/seq_file.h> #include <linux/poll.h> +#include <linux/seq_file.h> + +#include <asm/local.h> #include <uapi/linux/trace_mmap.h>
@@ -20,6 +22,8 @@ struct ring_buffer_event {u32 array[]; }; +#define RB_EVNT_HDR_SIZE (offsetof(struct ring_buffer_event, array)) +/** * enum ring_buffer_type - internal ring buffer types *
@@ -61,11 +65,50 @@ enum ring_buffer_type { RINGBUF_TYPE_TIME_STAMP, }; +#define TS_SHIFT 27 +#define TS_MASK ((1ULL << TS_SHIFT) - 1) +#define TS_DELTA_TEST (~TS_MASK) + +/* +* We need to fit the time_stamp delta into 27 bits. +*/ +static inline bool test_time_stamp (u64 delta) +{ +return !!(delta &TS_DELTA_TEST ); +} +unsigned ring_buffer_event_length (struct ring_buffer_event *event);void *ring_buffer_event_data (struct ring_buffer_event *event);u64 ring_buffer_event_time_stamp (struct trace_buffer *buffer,struct ring_buffer_event *event); +#define BUF_PAGE_HDR_SIZE offsetof(struct buffer_data_page, data) + +/* Max payload is BUF_PAGE_SIZE - header (8bytes) */ +#define BUF_MAX_DATA_SIZE (BUF_PAGE_SIZE - (sizeof(u32) * 2)) + +#define BUF_PAGE_SIZE (PAGE_SIZE - BUF_PAGE_HDR_SIZE) + +#define RB_ALIGNMENT 4U +#define RB_MAX_SMALL_DATA (RB_ALIGNMENT * RINGBUF_TYPE_DATA_TYPE_LEN_MAX) +#define RB_EVNT_MIN_SIZE 8U/* two 32bit words */ + +#ifndef CONFIG_HAVE_64BIT_ALIGNED_ACCESS +# define RB_FORCE_8BYTE_ALIGNMENT 0 +# define RB_ARCH_ALIGNMENT RB_ALIGNMENT +#else +# define RB_FORCE_8BYTE_ALIGNMENT 1 +# define RB_ARCH_ALIGNMENT 8U +#endif + +#define RB_ALIGN_DATA __aligned(RB_ARCH_ALIGNMENT) + +struct buffer_data_page { +u64 time_stamp;/* page time stamp */ +local_t commit;/* write committed index */ + unsigned char data[] RB_ALIGN_DATA; /* data of buffer page */ +}; +/* * ring_buffer_discard_commit will remove an event that has not * been committed yet. If this is used, then ring_buffer_unlock_commit
kernel/trace/ring_buffer.c +0 -35
@@ -152,23 +152,6 @@ int ring_buffer_print_entry_header(struct trace_seq *s)/* Used for individual buffers (after the counter) */ #define RB_BUFFER_OFF (1 << 20) -#define BUF_PAGE_HDR_SIZE offsetof(struct buffer_data_page, data) - -#define RB_EVNT_HDR_SIZE (offsetof(struct ring_buffer_event, array)) -#define RB_ALIGNMENT 4U -#define RB_MAX_SMALL_DATA (RB_ALIGNMENT * RINGBUF_TYPE_DATA_TYPE_LEN_MAX) -#define RB_EVNT_MIN_SIZE 8U/* two 32bit words */ - -#ifndef CONFIG_HAVE_64BIT_ALIGNED_ACCESS -# define RB_FORCE_8BYTE_ALIGNMENT 0 -# define RB_ARCH_ALIGNMENT RB_ALIGNMENT -#else -# define RB_FORCE_8BYTE_ALIGNMENT 1 -# define RB_ARCH_ALIGNMENT 8U -#endif - -#define RB_ALIGN_DATA __aligned(RB_ARCH_ALIGNMENT) -/* define RINGBUF_TYPE_DATA for 'case RINGBUF_TYPE_DATA:' */ #define RINGBUF_TYPE_DATA 0 ... RINGBUF_TYPE_DATA_TYPE_LEN_MAX
@@ -311,10 +294,6 @@ EXPORT_SYMBOL_GPL(ring_buffer_event_data);#define for_each_online_buffer_cpu (buffer, cpu) \ for_each_cpu_and(cpu, buffer->cpumask, cpu_online_mask) -#define TS_SHIFT 27 -#define TS_MASK ((1ULL << TS_SHIFT) - 1) -#define TS_DELTA_TEST (~TS_MASK) -static u64 rb_event_time_stamp (struct ring_buffer_event *event) {u64 ts;
@@ -333,12 +312,6 @@ static u64 rb_event_time_stamp(struct ring_buffer_event *event)#define RB_MISSED_MASK (3 << 30) -struct buffer_data_page { -u64 time_stamp;/* page time stamp */ -local_t commit;/* write committed index */ - unsigned char data[] RB_ALIGN_DATA; /* data of buffer page */ -}; -struct buffer_data_read_page {unsigned order;/* order of the page */ struct buffer_data_page *data;/* actual data, stored in this page */
@@ -397,14 +370,6 @@ static void free_buffer_page(struct buffer_page *bpage)kfree (bpage); } -/* -* We need to fit the time_stamp delta into 27 bits. -*/ -static inline bool test_time_stamp (u64 delta) -{ -return !!(delta &TS_DELTA_TEST ); -} -struct rb_irq_work {struct irq_work work;wait_queue_head_t waiters;
Signature · 1 line
2.47.1.613.gc27f4b7a9f-goog
Vincent Donnefort
maintainerPATCH v2 4/12 KVM: arm64: Support unaligned fixmap in the nVHE hyp
In reply to Vincent DonnefortReturn the fixmap VA with the page offset, instead of the page base address.
arch/arm64/kvm/hyp/nvhe/mm.c +1 -1
@@ -240,7 +240,7 @@ void *hyp_fixmap_map(phys_addr_t phys)WRITE_ONCE (*ptep, pte);dsb (ishst); -return (void *)slot->addr; +return (void *)slot->addr +offset_in_page (phys); }static void fixmap_clear_slot (struct hyp_fixmap_slot *slot)
Signature · 1 line
2.47.1.613.gc27f4b7a9f-goog
Vincent Donnefort
maintainerPATCH v2 5/12 KVM: arm64: Add clock support in the nVHE hyp
In reply to Vincent DonnefortBy default, the arm64 host kernel is using the arch timer as a source for sched_clock. Conveniently, EL2 has access to that same counter, allowing to generate clock values that are synchronized.
The clock needs nonetheless to be setup with the same slope values as the kernel. Introducing at the same time trace_clock() which is expected to be later configured by the hypervisor tracing.
arch/arm64/include/asm/kvm_hyp.h +0 -1
@@ -144,5 +144,4 @@ extern u64 kvm_nvhe_sym(id_aa64smfr0_el1_sys_val);extern unsigned long kvm_nvhe_sym (__icache_flags );extern unsigned int kvm_nvhe_sym (kvm_arm_vmid_bits );extern unsigned int kvm_nvhe_sym (kvm_host_sve_max_vl ); -#endif /* __ARM64_KVM_HYP_H__ */
arch/arm64/kvm/hyp/include/nvhe/clock.h new file +16 -0
@@ -0,0 +1,16 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef __ARM64_KVM_HYP_NVHE_CLOCK_H +#define __ARM64_KVM_HYP_NVHE_CLOCK_H +#include <linux/types.h> + +#include <asm/kvm_hyp.h> + +#ifdef CONFIG_TRACING +void trace_clock_update (u32 mult,u32 shift,u64 epoch_ns,u64 epoch_cyc); +u64 trace_clock (void ); +#else +static inline void +trace_clock_update (u32 mult,u32 shift,u64 epoch_ns,u64 epoch_cyc) { } +static inline u64 trace_clock (void ) {return 0 ; } +#endif +#endif
arch/arm64/kvm/hyp/nvhe/Makefile +1 -0
@@ -28,6 +28,7 @@ hyp-obj-y := timer-sr.o sysreg-sr.o debug-sr.o switch.o tlb.o hyp-init.o host.o hyp-obj-y += ../vgic-v3-sr.o ../aarch32.o ../vgic-v2-cpuif-proxy.o ../entry.o \ ../fpsimd.o ../hyp-entry.o ../exception.o ../pgtable.o hyp-obj-$(CONFIG_LIST_HARDENED) += list_debug.o +hyp-obj-$(CONFIG_TRACING) += clock.o hyp-obj-y += $(lib-objs) ##
arch/arm64/kvm/hyp/nvhe/clock.c new file +65 -0
65 lines not on this page · raw message
Signature · 1 line
2.47.1.613.gc27f4b7a9f-goog
Vincent Donnefort
maintainerPATCH v2 6/12 KVM: arm64: Add tracing support for the pKVM hyp
In reply to Vincent DonnefortWhen running with protected mode, the host has very little knowledge about what is happening in the hypervisor. Of course this is an essential feature for security but nonetheless, that piece of code growing with more responsibilities, we need now a way to debug and profile it. Tracefs by its reliatility, versatility and support for user-space is the perfect tool.
There's no way the hypervisor could log events directly into the host tracefs ring-buffers. So instead let's use our own, where the hypervisor is the writer and the host the reader.
arch/arm64/include/asm/kvm_asm.h +4 -0
@@ -79,6 +79,10 @@ enum __kvm_host_smccc_func { __KVM_HOST_SMCCC_FUNC___pkvm_init_vm, __KVM_HOST_SMCCC_FUNC___pkvm_init_vcpu, __KVM_HOST_SMCCC_FUNC___pkvm_teardown_vm, + __KVM_HOST_SMCCC_FUNC___pkvm_load_tracing, + __KVM_HOST_SMCCC_FUNC___pkvm_teardown_tracing, + __KVM_HOST_SMCCC_FUNC___pkvm_enable_tracing, + __KVM_HOST_SMCCC_FUNC___pkvm_swap_reader_tracing, };#define DECLARE_KVM_VHE_SYM (sym) extern char sym[]
arch/arm64/include/asm/kvm_hyptrace.h new file +21 -0
@@ -0,0 +1,21 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +#ifndef __ARM64_KVM_HYPTRACE_H_ +#define __ARM64_KVM_HYPTRACE_H_ +#include <asm/kvm_hyp.h> + +#include <linux/ring_buffer.h> + +/* +* Host donations to the hypervisor to store the struct hyp_buffer_page. +*/ +struct hyp_buffer_pages_backing { +unsigned long start; +size_t size; +}; + +struct hyp_trace_desc { +struct hyp_buffer_pages_backing backing; +struct trace_page_desc page_desc; + +}; +#endif
arch/arm64/kvm/hyp/include/nvhe/trace.h new file +32 -0
@@ -0,0 +1,32 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +#ifndef __ARM64_KVM_HYP_NVHE_TRACE_H +#define __ARM64_KVM_HYP_NVHE_TRACE_H +#include <asm/kvm_hyptrace.h> + +/* Internal struct that needs export for hyp-constants.c */ +struct hyp_buffer_page { +struct list_head list; +struct buffer_data_page *page; +unsigned long write; +unsigned long entries; +u32 id; +}; + +#ifdef CONFIG_TRACING +void *tracing_reserve_entry (unsigned long length); +void tracing_commit_entry (void ); + +int __pkvm_load_tracing (unsigned long desc_va,size_t desc_size); +void __pkvm_teardown_tracing (void ); +int __pkvm_enable_tracing (bool enable); +int __pkvm_swap_reader_tracing (unsigned int cpu); +#else +static inline void *tracing_reserve_entry (unsigned long length) {return NULL ; } +static inline void tracing_commit_entry (void ) { } + +static inline int __pkvm_load_tracing (unsigned long desc_va,size_t desc_size) {return -ENODEV ; } +static inline void __pkvm_teardown_tracing (void ) { } +static inline int __pkvm_enable_tracing (bool enable) {return -ENODEV ; } +static inline int __pkvm_swap_reader_tracing (unsigned int cpu) {return -ENODEV ; } +#endif +#endif
arch/arm64/kvm/hyp/nvhe/Makefile +1 -1
@@ -28,7 +28,7 @@ hyp-obj-y := timer-sr.o sysreg-sr.o debug-sr.o switch.o tlb.o hyp-init.o host.o hyp-obj-y += ../vgic-v3-sr.o ../aarch32.o ../vgic-v2-cpuif-proxy.o ../entry.o \ ../fpsimd.o ../hyp-entry.o ../exception.o ../pgtable.o hyp-obj-$(CONFIG_LIST_HARDENED) += list_debug.o -hyp-obj-$(CONFIG_TRACING) += clock.o +hyp-obj-$(CONFIG_TRACING) += clock.o trace.o hyp-obj-y += $(lib-objs) ##
arch/arm64/kvm/hyp/nvhe/hyp-main.c +34 -0
@@ -17,6 +17,7 @@#include <nvhe/mem_protect.h> #include <nvhe/mm.h> #include <nvhe/pkvm.h> +#include <nvhe/trace.h> #include <nvhe/trap_handler.h> DEFINE_PER_CPU(struct kvm_nvhe_init_params, kvm_init_params);
@@ -378,6 +379,35 @@ static void handle___pkvm_teardown_vm(struct kvm_cpu_context *host_ctxt)cpu_reg (host_ctxt,1 ) =__pkvm_teardown_vm (handle); } +static void handle___pkvm_load_tracing (struct kvm_cpu_context *host_ctxt) +{ + DECLARE_REG(unsigned long, desc_hva, host_ctxt, 1); +DECLARE_REG (size_t, desc_size, host_ctxt,2 ); + +cpu_reg (host_ctxt,1 ) =__pkvm_load_tracing (desc_hva, desc_size); +} + +static void handle___pkvm_teardown_tracing (struct kvm_cpu_context *host_ctxt) +{ +__pkvm_teardown_tracing (); + +cpu_reg (host_ctxt,1 ) =0 ; +} + +static void handle___pkvm_enable_tracing (struct kvm_cpu_context *host_ctxt) +{ +DECLARE_REG (bool, enable, host_ctxt,1 ); + +cpu_reg (host_ctxt,1 ) =__pkvm_enable_tracing (enable); +} + +static void handle___pkvm_swap_reader_tracing (struct kvm_cpu_context *host_ctxt) +{ + DECLARE_REG(unsigned int, cpu, host_ctxt, 1); + +cpu_reg (host_ctxt,1 ) =__pkvm_swap_reader_tracing (cpu); +} +typedef void (*hcall_t )(struct kvm_cpu_context *);#define HANDLE_FUNC (x) [__KVM_HOST_SMCCC_FUNC_##x] = (hcall_t)handle_##x
@@ -409,6 +439,10 @@ static const hcall_t host_hcall[] = {HANDLE_FUNC (__pkvm_init_vm),HANDLE_FUNC (__pkvm_init_vcpu),HANDLE_FUNC (__pkvm_teardown_vm), +HANDLE_FUNC (__pkvm_load_tracing), +HANDLE_FUNC (__pkvm_teardown_tracing), +HANDLE_FUNC (__pkvm_enable_tracing), + HANDLE_FUNC(__pkvm_swap_reader_tracing), };static void handle_host_hcall (struct kvm_cpu_context *host_ctxt)
arch/arm64/kvm/hyp/nvhe/trace.c new file +589 -0
589 lines not on this page · raw message
Signature · 1 line
2.47.1.613.gc27f4b7a9f-goog
Vincent Donnefort
maintainerPATCH v2 7/12 KVM: arm64: Add hyp tracing to tracefs
In reply to Vincent DonnefortWhen running with KVM protected mode, the hypervisor is able to generate events into tracefs compatible ring-buffers. Plug those ring-buffers to tracefs. The interface is found in hyp/ and contains the same hierarchy as any host instances easing the support by existing user-space tools.
This currently doesn't provide any event support which will come later.
arch/arm64/kvm/Makefile +2 -0
@@ -29,6 +29,8 @@ kvm-$(CONFIG_HW_PERF_EVENTS) += pmu-emul.o pmu.o kvm-$(CONFIG_ARM64_PTR_AUTH) += pauth.o kvm-$(CONFIG_PTDUMP_STAGE2_DEBUGFS) += ptdump.o +kvm-$(CONFIG_TRACING) += hyp_trace.o + always-y := hyp_constants.h hyp-constants.s define rule_gen_hyp_constants
arch/arm64/kvm/arm.c +4 -0
@@ -25,6 +25,7 @@#define CREATE_TRACE_POINTS #include "trace_arm.h" +#include "hyp_trace.h" #include <linux/uaccess.h> #include <asm/ptrace.h>
@@ -2316,6 +2317,9 @@ static int __init init_subsystems(void)kvm_register_perf_callbacks (NULL ); + err =hyp_trace_init_tracefs (); +if (err) +kvm_err ("Failed to initialize Hyp tracing\n" ); out:if (err)hyp_cpu_pm_exit ();
arch/arm64/kvm/hyp/hyp-constants.c +4 -0
@@ -3,11 +3,15 @@#include <linux/kbuild.h> #include <nvhe/memory.h> #include <nvhe/pkvm.h> +#include <nvhe/trace.h> int main (void ) {DEFINE (STRUCT_HYP_PAGE_SIZE ,sizeof (struct hyp_page ));DEFINE (PKVM_HYP_VM_SIZE ,sizeof (struct pkvm_hyp_vm ));DEFINE (PKVM_HYP_VCPU_SIZE ,sizeof (struct pkvm_hyp_vcpu )); +#ifdef CONFIG_TRACING +DEFINE (STRUCT_HYP_BUFFER_PAGE_SIZE ,sizeof (struct hyp_buffer_page )); +#endif return 0 ; }
arch/arm64/kvm/hyp_trace.c new file +663 -0
663 lines not on this page · raw message
arch/arm64/kvm/hyp_trace.h new file +28 -0
@@ -0,0 +1,28 @@ +/* SPDX-License-Identifier: GPL-2.0 */ + +#ifndef __ARM64_KVM_HYP_TRACE_H__ +#define __ARM64_KVM_HYP_TRACE_H__ + +#include <linux/trace_seq.h> +#include <linux/workqueue.h> + +struct ht_iterator { +struct trace_buffer *trace_buffer; +int cpu; +struct hyp_entry_hdr *ent; +unsigned long lost_events; +int ent_cpu; +size_t ent_size; +u64 ts; +void *spare; +size_t copy_leftover; +struct trace_seq seq; +struct delayed_work poll_work; +}; + +#ifdef CONFIG_TRACING +int hyp_trace_init_tracefs (void ); +#else +static inline int hyp_trace_init_tracefs (void ) {return 0 ; } +#endif +#endif
Signature · 1 line
2.47.1.613.gc27f4b7a9f-goog
Vincent Donnefort
maintainerPATCH v2 8/12 KVM: arm64: Add clock for hyp tracefs
In reply to Vincent DonnefortConfigure the hypervisor tracing clock before starting tracing. For tracing purpose, the boot clock is interesting as it doesn't stop on suspend. However, it is corrected on a regular basis, which implies we need to re-evaluate it every once in a while.
arch/arm64/include/asm/kvm_asm.h +1 -0
@@ -79,6 +79,7 @@ enum __kvm_host_smccc_func { __KVM_HOST_SMCCC_FUNC___pkvm_init_vm, __KVM_HOST_SMCCC_FUNC___pkvm_init_vcpu, __KVM_HOST_SMCCC_FUNC___pkvm_teardown_vm, + __KVM_HOST_SMCCC_FUNC___pkvm_update_clock_tracing, __KVM_HOST_SMCCC_FUNC___pkvm_load_tracing, __KVM_HOST_SMCCC_FUNC___pkvm_teardown_tracing, __KVM_HOST_SMCCC_FUNC___pkvm_enable_tracing,
arch/arm64/kvm/hyp/include/nvhe/trace.h +3 -0
@@ -16,6 +16,7 @@ struct hyp_buffer_page {void *tracing_reserve_entry (unsigned long length);void tracing_commit_entry (void ); +void __pkvm_update_clock_tracing (u32 mult,u32 shift,u64 epoch_ns,u64 epoch_cyc);int __pkvm_load_tracing (unsigned long desc_va,size_t desc_size);void __pkvm_teardown_tracing (void );int __pkvm_enable_tracing (bool enable);
@@ -24,6 +25,8 @@ int __pkvm_swap_reader_tracing(unsigned int cpu);static inline void *tracing_reserve_entry (unsigned long length) {return NULL ; }static inline void tracing_commit_entry (void ) { } +static inline +void __pkvm_update_clock_tracing (u32 mult,u32 shift,u64 epoch_ns,u64 epoch_cyc) { }static inline int __pkvm_load_tracing (unsigned long desc_va,size_t desc_size) {return -ENODEV ; }static inline void __pkvm_teardown_tracing (void ) { }static inline int __pkvm_enable_tracing (bool enable) {return -ENODEV ; }
arch/arm64/kvm/hyp/nvhe/hyp-main.c +13 -0
@@ -379,6 +379,18 @@ static void handle___pkvm_teardown_vm(struct kvm_cpu_context *host_ctxt)cpu_reg (host_ctxt,1 ) =__pkvm_teardown_vm (handle); } +static void handle___pkvm_update_clock_tracing (struct kvm_cpu_context *host_ctxt) +{ +DECLARE_REG (u32, mult, host_ctxt,1 ); +DECLARE_REG (u32, shift, host_ctxt,2 ); +DECLARE_REG (u64, epoch_ns, host_ctxt,3 ); +DECLARE_REG (u64, epoch_cyc, host_ctxt,4 ); + +__pkvm_update_clock_tracing (mult, shift, epoch_ns, epoch_cyc); + +cpu_reg (host_ctxt,1 ) =0 ; +} +static void handle___pkvm_load_tracing (struct kvm_cpu_context *host_ctxt) { DECLARE_REG(unsigned long, desc_hva, host_ctxt, 1);
@@ -439,6 +451,7 @@ static const hcall_t host_hcall[] = {HANDLE_FUNC (__pkvm_init_vm),HANDLE_FUNC (__pkvm_init_vcpu),HANDLE_FUNC (__pkvm_teardown_vm), +HANDLE_FUNC (__pkvm_update_clock_tracing),HANDLE_FUNC (__pkvm_load_tracing),HANDLE_FUNC (__pkvm_teardown_tracing), HANDLE_FUNC(__pkvm_enable_tracing),
arch/arm64/kvm/hyp/nvhe/trace.c +15 -0
@@ -468,6 +468,21 @@ static void rb_teardown_bpage_backing(void) hyp_buffer_pages_backing.size =0 ; } +void __pkvm_update_clock_tracing (u32 mult,u32 shift,u64 epoch_ns,u64 epoch_cyc) +{ +int cpu; + +/* After this loop, all CPUs are observing the new bank... */ +for (cpu =0 ; cpu < hyp_nr_cpus; cpu++) { +struct hyp_rb_per_cpu *cpu_buffer =per_cpu_ptr (&trace_rb, cpu); + +while (atomic_read (&cpu_buffer->status) ==HYP_RB_WRITING ); + } + +/* ...we can now override the old one and swap. */ +trace_clock_update (mult, shift, epoch_ns, epoch_cyc); +} +int __pkvm_swap_reader_tracing (unsigned int cpu) {struct hyp_rb_per_cpu *cpu_buffer;
arch/arm64/kvm/hyp_trace.c +146 -0
202 lines not on this page · raw message
Signature · 1 line
2.47.1.613.gc27f4b7a9f-goog
Vincent Donnefort
maintainerPATCH v2 9/12 KVM: arm64: Add raw interface for hyp tracefs
In reply to Vincent DonnefortThe raw interface enables userspace tools such as trace-cmd to directly read the ring-buffer without any decoding by the kernel.
arch/arm64/kvm/hyp_trace.c +82 -0
@@ -749,6 +749,85 @@ static const struct file_operations hyp_trace_pipe_fops = { .release = hyp_trace_pipe_release, }; +static ssize_t +hyp_trace_raw_read (struct file *file,char __user *ubuf, +size_t cnt,loff_t *ppos) +{ +struct ht_iterator *iter = (struct ht_iterator *)file->private_data; +size_t size; +int ret; + +if (iter->copy_leftover) + goto read; + +again: + ret =ring_buffer_read_page (iter->trace_buffer, + (struct buffer_data_read_page *)iter->spare, + cnt, iter->cpu,0 ); +if (ret <0 ) { +if (!ring_buffer_empty_cpu (iter->trace_buffer, iter->cpu)) +return 0 ; + + ret =ring_buffer_wait (iter->trace_buffer, iter->cpu,0 ,NULL , +NULL ); +if (ret <0 ) +return ret; + + goto again; + } + + iter->copy_leftover =0 ; + +read: + size =PAGE_SIZE - iter->copy_leftover; +if (size > cnt) + size = cnt; + + ret =copy_to_user (ubuf, iter->spare +PAGE_SIZE - size, size); +if (ret == size) +return -EFAULT ; + + size -= ret; + *ppos += size; + iter->copy_leftover = ret; + +return size; +} + +static int hyp_trace_raw_open (struct inode *inode,struct file *file) +{ +int ret =hyp_trace_pipe_open (inode, file); +struct ht_iterator *iter; + +if (ret) +return ret; + + iter = file->private_data; + iter->spare =ring_buffer_alloc_read_page (iter->trace_buffer, iter->cpu); +if (IS_ERR (iter->spare)) { + ret =PTR_ERR (iter->spare); + iter->spare =NULL ; +return ret; + } + +return 0 ; +} + +static int hyp_trace_raw_release (struct inode *inode,struct file *file) +{ +struct ht_iterator *iter = file->private_data; + +ring_buffer_free_read_page (iter->trace_buffer, iter->cpu, iter->spare); + +return hyp_trace_pipe_release (inode, file); +} + +static const struct file_operations hyp_trace_raw_fops = { + .open = hyp_trace_raw_open, + .read = hyp_trace_raw_read, + .release = hyp_trace_raw_release, +}; +static int hyp_trace_clock_show (struct seq_file *m,void *v) {seq_puts (m,"[boot]\n" );
@@ -803,6 +882,9 @@ int hyp_trace_init_tracefs(void)tracefs_create_file ("trace_pipe" ,TRACEFS_MODE_READ , per_cpu_dir, (void *)cpu, &hyp_trace_pipe_fops); + +tracefs_create_file ("trace_pipe_raw" ,TRACEFS_MODE_READ , per_cpu_dir, + (void *)cpu, &hyp_trace_pipe_fops); }return 0 ;
Signature · 1 line
2.47.1.613.gc27f4b7a9f-goog
Vincent Donnefort
maintainerPATCH v2 10/12 KVM: arm64: Add trace interface for hyp tracefs
In reply to Vincent DonnefortThe trace interface is solely here to reset tracing. Non-consuming read is not yet supported due to the lack of support in the ring-buffer meta page.
arch/arm64/include/asm/kvm_asm.h +1 -0
@@ -83,6 +83,7 @@ enum __kvm_host_smccc_func { __KVM_HOST_SMCCC_FUNC___pkvm_load_tracing, __KVM_HOST_SMCCC_FUNC___pkvm_teardown_tracing, __KVM_HOST_SMCCC_FUNC___pkvm_enable_tracing, + __KVM_HOST_SMCCC_FUNC___pkvm_reset_tracing, __KVM_HOST_SMCCC_FUNC___pkvm_swap_reader_tracing, };
arch/arm64/kvm/hyp/include/nvhe/trace.h +2 -0
@@ -20,6 +20,7 @@ void __pkvm_update_clock_tracing(u32 mult, u32 shift, u64 epoch_ns, u64 epoch_cyint __pkvm_load_tracing (unsigned long desc_va,size_t desc_size);void __pkvm_teardown_tracing (void );int __pkvm_enable_tracing (bool enable); +int __pkvm_reset_tracing (unsigned int cpu);int __pkvm_swap_reader_tracing (unsigned int cpu);#else static inline void *tracing_reserve_entry (unsigned long length) {return NULL ; }
@@ -30,6 +31,7 @@ void __pkvm_update_clock_tracing(u32 mult, u32 shift, u64 epoch_ns, u64 epoch_cystatic inline int __pkvm_load_tracing (unsigned long desc_va,size_t desc_size) {return -ENODEV ; }static inline void __pkvm_teardown_tracing (void ) { }static inline int __pkvm_enable_tracing (bool enable) {return -ENODEV ; } +static inline int __pkvm_reset_tracing (unsigned int cpu) {return -ENODEV ; }static inline int __pkvm_swap_reader_tracing (unsigned int cpu) {return -ENODEV ; }#endif #endif
arch/arm64/kvm/hyp/nvhe/hyp-main.c +8 -0
@@ -413,6 +413,13 @@ static void handle___pkvm_enable_tracing(struct kvm_cpu_context *host_ctxt)cpu_reg (host_ctxt,1 ) =__pkvm_enable_tracing (enable); } +static void handle___pkvm_reset_tracing (struct kvm_cpu_context *host_ctxt) +{ + DECLARE_REG(unsigned int, cpu, host_ctxt, 1); + +cpu_reg (host_ctxt,1 ) =__pkvm_reset_tracing (cpu); +} +static void handle___pkvm_swap_reader_tracing (struct kvm_cpu_context *host_ctxt) { DECLARE_REG(unsigned int, cpu, host_ctxt, 1);
@@ -455,6 +462,7 @@ static const hcall_t host_hcall[] = {HANDLE_FUNC (__pkvm_load_tracing),HANDLE_FUNC (__pkvm_teardown_tracing),HANDLE_FUNC (__pkvm_enable_tracing), +HANDLE_FUNC (__pkvm_reset_tracing), HANDLE_FUNC(__pkvm_swap_reader_tracing), };
arch/arm64/kvm/hyp/nvhe/trace.c +57 -1
84 lines not on this page · raw message
arch/arm64/kvm/hyp_trace.c +55 -0
85 lines not on this page · raw message
Signature · 1 line
2.47.1.613.gc27f4b7a9f-goog
Vincent Donnefort
maintainerPATCH v2 11/12 KVM: arm64: Add support for hyp events
In reply to Vincent DonnefortFollowing the introduction of hyp tracing for pKVM, add the ability to describe and emit events into the hypervisor ring-buffers.
Hypervisor events are declared into kvm_hypevents.h and can be called with trace_<event_name>() in a similar fashion to the kernel tracefs events.
hyp_enter and hyp_exit events are provided as an example.
arch/arm64/include/asm/kvm_asm.h +1 -0
@@ -85,6 +85,7 @@ enum __kvm_host_smccc_func { __KVM_HOST_SMCCC_FUNC___pkvm_enable_tracing, __KVM_HOST_SMCCC_FUNC___pkvm_reset_tracing, __KVM_HOST_SMCCC_FUNC___pkvm_swap_reader_tracing, + __KVM_HOST_SMCCC_FUNC___pkvm_enable_event, };#define DECLARE_KVM_VHE_SYM (sym) extern char sym[]
arch/arm64/include/asm/kvm_define_hypevents.h new file +61 -0
61 lines not on this page · raw message
arch/arm64/include/asm/kvm_hypevents.h new file +31 -0
@@ -0,0 +1,31 @@ +/* SPDX-License-Identifier: GPL-2.0 */ + +#if !defined(__ARM64_KVM_HYPEVENTS_H_) || defined(HYP_EVENT_MULTI_READ ) +#define __ARM64_KVM_HYPEVENTS_H_ + +#ifdef __KVM_NVHE_HYPERVISOR__ +#include <nvhe/trace.h> +#endif + +/* +* Hypervisor events definitions. +*/ + +HYP_EVENT (hyp_enter, +HE_PROTO (void), +HE_STRUCT ( + ), +HE_ASSIGN ( + ), +HE_PRINTK (" " ) +); + +HYP_EVENT (hyp_exit, +HE_PROTO (void), +HE_STRUCT ( + ), +HE_ASSIGN ( + ), +HE_PRINTK (" " ) +); +#endif
arch/arm64/include/asm/kvm_hypevents_defs.h new file +41 -0
41 lines not on this page · raw message
arch/arm64/include/asm/kvm_hyptrace.h +16 -0
@@ -4,6 +4,22 @@#include <asm/kvm_hyp.h> #include <linux/ring_buffer.h> +#include <linux/trace_seq.h> +#include <linux/workqueue.h> + +struct ht_iterator { +struct trace_buffer *trace_buffer; +int cpu; +struct hyp_entry_hdr *ent; +unsigned long lost_events; +int ent_cpu; +size_t ent_size; +u64 ts; +void *spare; +size_t copy_leftover; +struct trace_seq seq; +struct delayed_work poll_work; +};/* * Host donations to the hypervisor to store the struct hyp_buffer_page.
arch/arm64/kernel/image-vars.h +4 -0
@@ -134,6 +134,10 @@ KVM_NVHE_ALIAS(__hyp_bss_start);KVM_NVHE_ALIAS (__hyp_bss_end);KVM_NVHE_ALIAS (__hyp_rodata_start);KVM_NVHE_ALIAS (__hyp_rodata_end); +#ifdef CONFIG_TRACING +KVM_NVHE_ALIAS (__hyp_event_ids_start); +KVM_NVHE_ALIAS (__hyp_event_ids_end); +#endif /* pKVM static key */ KVM_NVHE_ALIAS (kvm_protected_mode_initialized);
arch/arm64/kernel/vmlinux.lds.S +18 -0
@@ -13,12 +13,23 @@ *(__kvm_ex_table) \ __stop___kvm_ex_table = .; +#ifdef CONFIG_TRACING +#define HYPERVISOR_EVENT_IDS \ + . = ALIGN(PAGE_SIZE); \ + __hyp_event_ids_start = .; \ + *(HYP_SECTION_NAME(.event_ids)) \ + __hyp_event_ids_end = .; +#else +#define HYPERVISOR_EVENT_IDS +#endif + #define HYPERVISOR_DATA_SECTIONS \ HYP_SECTION_NAME(.rodata) : { \ . = ALIGN(PAGE_SIZE); \ __hyp_rodata_start = .; \ *(HYP_SECTION_NAME(.data..ro_after_init)) \ *(HYP_SECTION_NAME(.rodata)) \ + HYPERVISOR_EVENT_IDS \ . = ALIGN(PAGE_SIZE); \ __hyp_rodata_end = .; \ }
@@ -200,6 +211,13 @@ SECTIONS ASSERT(SIZEOF(.got.plt) == 0 || SIZEOF(.got.plt) == 0x18, "Unexpected GOT/PLT entries detected!") +#ifdef CONFIG_TRACING + .rodata.hyp_events : { + __hyp_events_start = .; + *(SORT(_hyp_events.*)) + __hyp_events_end = .; + } +#endif /* code sections that are never executed via the kernel mapping */ .rodata.text : { TRAMP_TEXT
arch/arm64/kvm/Makefile +1 -1
@@ -29,7 +29,7 @@ kvm-$(CONFIG_HW_PERF_EVENTS) += pmu-emul.o pmu.o kvm-$(CONFIG_ARM64_PTR_AUTH) += pauth.o kvm-$(CONFIG_PTDUMP_STAGE2_DEBUGFS) += ptdump.o -kvm-$(CONFIG_TRACING) += hyp_trace.o +kvm-$(CONFIG_TRACING) += hyp_events.o hyp_trace.o always-y := hyp_constants.h hyp-constants.s
arch/arm64/kvm/arm.c +2 -0
@@ -2642,6 +2642,8 @@ static int __init init_hyp_mode(void)kvm_hyp_init_symbols (); +hyp_trace_init_events (); +if (is_protected_kvm_enabled ()) {if (IS_ENABLED (CONFIG_ARM64_PTR_AUTH_KERNEL ) &&cpus_have_final_cap (ARM64_HAS_ADDRESS_AUTH ))
arch/arm64/kvm/hyp/include/nvhe/arm-smccc.h new file +13 -0
@@ -0,0 +1,13 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ + +#include <asm/kvm_hypevents.h> + +#include <linux/arm-smccc.h> + +#undef arm_smccc_1_1_smc +#define arm_smccc_1_1_smc (...) \ + do { \ + trace_hyp_exit(); \ + __arm_smccc_1_1(SMCCC_SMC_INST, __VA_ARGS__); \ + trace_hyp_enter(); \ + } while (0)
arch/arm64/kvm/hyp/include/nvhe/define_events.h new file +21 -0
@@ -0,0 +1,21 @@ +/* SPDX-License-Identifier: GPL-2.0 */ + +#ifndef HYP_EVENT_FILE +# define __HYP_EVENT_FILE <asm/kvm_hypevents.h> +#else +# define __HYP_EVENT_FILE __stringify(HYP_EVENT_FILE) +#endif + +#undef HYP_EVENT +#define HYP_EVENT (__name, __proto, __struct, __assign, __printk) \ + atomic_t __ro_after_init __name##_enabled = ATOMIC_INIT(0); \ + struct hyp_event_id hyp_event_id_##__name \ + __section(".hyp.event_ids."#__name) = { \ + .data = (void *)&__name##_enabled, \ + } + +#define HYP_EVENT_MULTI_READ +#include __HYP_EVENT_FILE +#undef HYP_EVENT_MULTI_READ + +#undef HYP_EVENT
arch/arm64/kvm/hyp/include/nvhe/trace.h +23 -0
@@ -2,6 +2,7 @@#ifndef __ARM64_KVM_HYP_NVHE_TRACE_H#define __ARM64_KVM_HYP_NVHE_TRACE_H#include <asm/kvm_hyptrace.h> +#include <asm/kvm_hypevents_defs.h> /* Internal struct that needs export for hyp-constants.c */ struct hyp_buffer_page {
@@ -15,6 +16,24 @@ struct hyp_buffer_page {#ifdef CONFIG_TRACING void *tracing_reserve_entry (unsigned long length);void tracing_commit_entry (void ); +#define HYP_EVENT (__name, __proto, __struct, __assign, __printk) \ + HYP_EVENT_FORMAT(__name, __struct); \ + extern atomic_t __name##_enabled; \ + extern struct hyp_event_id hyp_event_id_##__name; \ + static __always_inline void trace_##__name(__proto) \ + { \ + size_t length = sizeof(struct trace_hyp_format_##__name); \ + struct trace_hyp_format_##__name *__entry; \ + \ + if (!atomic_read(&__name##_enabled)) \ + return; \ + __entry = tracing_reserve_entry(length); \ + if (!__entry) \ + return; \ + __entry->hdr.id = hyp_event_id_##__name.id; \ + __assign \ + tracing_commit_entry(); \ + }void __pkvm_update_clock_tracing (u32 mult,u32 shift,u64 epoch_ns,u64 epoch_cyc);int __pkvm_load_tracing (unsigned long desc_va,size_t desc_size);
@@ -22,9 +41,12 @@ void __pkvm_teardown_tracing(void);int __pkvm_enable_tracing (bool enable);int __pkvm_reset_tracing (unsigned int cpu);int __pkvm_swap_reader_tracing (unsigned int cpu); +int __pkvm_enable_event (unsigned short id,bool enable);#else static inline void *tracing_reserve_entry (unsigned long length) {return NULL ; }static inline void tracing_commit_entry (void ) { } +#define HYP_EVENT (__name, __proto, __struct, __assign, __printk) \ + static inline void trace_##__name(__proto) {}static inline void __pkvm_update_clock_tracing (u32 mult,u32 shift,u64 epoch_ns,u64 epoch_cyc) { }
@@ -33,5 +55,6 @@ static inline void __pkvm_teardown_tracing(void) { }static inline int __pkvm_enable_tracing (bool enable) {return -ENODEV ; }static inline int __pkvm_reset_tracing (unsigned int cpu) {return -ENODEV ; }static inline int __pkvm_swap_reader_tracing (unsigned int cpu) {return -ENODEV ; } +static inline int __pkvm_enable_event (unsigned short id,bool enable) {return -ENODEV ; }#endif #endif
arch/arm64/kvm/hyp/nvhe/Makefile +1 -1
@@ -28,7 +28,7 @@ hyp-obj-y := timer-sr.o sysreg-sr.o debug-sr.o switch.o tlb.o hyp-init.o host.o hyp-obj-y += ../vgic-v3-sr.o ../aarch32.o ../vgic-v2-cpuif-proxy.o ../entry.o \ ../fpsimd.o ../hyp-entry.o ../exception.o ../pgtable.o hyp-obj-$(CONFIG_LIST_HARDENED) += list_debug.o -hyp-obj-$(CONFIG_TRACING) += clock.o trace.o +hyp-obj-$(CONFIG_TRACING) += clock.o events.o trace.o hyp-obj-y += $(lib-objs) ##
arch/arm64/kvm/hyp/nvhe/events.c new file +35 -0
@@ -0,0 +1,35 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* +* Copyright (C) 2023 Google LLC +*/ + +#include <nvhe/mm.h> +#include <nvhe/trace.h> + +#include <nvhe/define_events.h> + +extern struct hyp_event_id __hyp_event_ids_start[]; +extern struct hyp_event_id __hyp_event_ids_end[]; + +int __pkvm_enable_event (unsigned short id,bool enable) +{ +struct hyp_event_id *event_id = __hyp_event_ids_start; +atomic_t *enable_key; + +for (; (unsigned long )event_id < (unsigned long )__hyp_event_ids_end; + event_id++) { +if (event_id->id != id) +continue ; + + enable_key = (atomic_t *)event_id->data; + enable_key =hyp_fixmap_map (__hyp_pa (enable_key)); + +atomic_set (enable_key, enable); + +hyp_fixmap_unmap (); + +return 0 ; + } + +return -EINVAL ; +}
arch/arm64/kvm/hyp/nvhe/ffa.c +1 -1
@@ -26,10 +26,10 @@* the duration and are therefore serialised. */ -#include <linux/arm-smccc.h> #include <linux/arm_ffa.h> #include <asm/kvm_pkvm.h> +#include <nvhe/arm-smccc.h> #include <nvhe/ffa.h> #include <nvhe/mem_protect.h> #include <nvhe/memory.h>
arch/arm64/kvm/hyp/nvhe/hyp-main.c +16 -0
@@ -11,6 +11,7 @@#include <asm/kvm_emulate.h> #include <asm/kvm_host.h> #include <asm/kvm_hyp.h> +#include <asm/kvm_hypevents.h> #include <asm/kvm_mmu.h> #include <nvhe/ffa.h>
@@ -427,6 +428,14 @@ static void handle___pkvm_swap_reader_tracing(struct kvm_cpu_context *host_ctxt)cpu_reg (host_ctxt,1 ) =__pkvm_swap_reader_tracing (cpu); } +static void handle___pkvm_enable_event (struct kvm_cpu_context *host_ctxt) +{ + DECLARE_REG(unsigned short, id, host_ctxt, 1); +DECLARE_REG (bool, enable, host_ctxt,2 ); + +cpu_reg (host_ctxt,1 ) =__pkvm_enable_event (id, enable); +} +typedef void (*hcall_t )(struct kvm_cpu_context *);#define HANDLE_FUNC (x) [__KVM_HOST_SMCCC_FUNC_##x] = (hcall_t)handle_##x
@@ -464,6 +473,7 @@ static const hcall_t host_hcall[] = {HANDLE_FUNC (__pkvm_enable_tracing),HANDLE_FUNC (__pkvm_reset_tracing),HANDLE_FUNC (__pkvm_swap_reader_tracing), + HANDLE_FUNC(__pkvm_enable_event), };static void handle_host_hcall (struct kvm_cpu_context *host_ctxt)
@@ -504,7 +514,9 @@ static void handle_host_hcall(struct kvm_cpu_context *host_ctxt)static void default_host_smc_handler (struct kvm_cpu_context *host_ctxt) { +trace_hyp_exit ();__kvm_hyp_host_forward_smc (host_ctxt); +trace_hyp_enter (); }static void handle_host_smc (struct kvm_cpu_context *host_ctxt)
@@ -528,6 +540,8 @@ void handle_trap(struct kvm_cpu_context *host_ctxt) {u64 esr =read_sysreg_el2 (SYS_ESR ); +trace_hyp_enter (); +switch (ESR_ELx_EC (esr)) {case ESR_ELx_EC_HVC64:handle_host_hcall (host_ctxt);
@@ -548,4 +562,6 @@ void handle_trap(struct kvm_cpu_context *host_ctxt)default :BUG (); } + +trace_hyp_exit (); }
arch/arm64/kvm/hyp/nvhe/hyp.lds.S +6 -0
@@ -16,6 +16,12 @@ SECTIONS { HYP_SECTION(.text) HYP_SECTION(.data..ro_after_init) HYP_SECTION(.rodata) +#ifdef CONFIG_TRACING + . = ALIGN(PAGE_SIZE); + BEGIN_HYP_SECTION(.event_ids) + *(SORT(.hyp.event_ids.*)) + END_HYP_SECTION +#endif /* * .hyp..data..percpu needs to be page aligned to maintain the same
arch/arm64/kvm/hyp/nvhe/psci-relay.c +10 -4
@@ -6,11 +6,12 @@#include <asm/kvm_asm.h> #include <asm/kvm_hyp.h> +#include <asm/kvm_hypevents.h> #include <asm/kvm_mmu.h> -#include <linux/arm-smccc.h> #include <linux/kvm_host.h> #include <uapi/linux/psci.h> +#include <nvhe/arm-smccc.h> #include <nvhe/memory.h> #include <nvhe/trap_handler.h>
@@ -153,6 +154,7 @@ static int psci_cpu_suspend(u64 func_id, struct kvm_cpu_context *host_ctxt)DECLARE_REG (u64, power_state, host_ctxt,1 ); DECLARE_REG(unsigned long, pc, host_ctxt, 2); DECLARE_REG(unsigned long, r0, host_ctxt, 3); +int ret;struct psci_boot_args *boot_args;struct kvm_nvhe_init_params *init_params;
@@ -171,9 +173,11 @@ static int psci_cpu_suspend(u64 func_id, struct kvm_cpu_context *host_ctxt)* Will either return if shallow sleep state, or wake up into the entry * point if it is a deep sleep state. */ -return psci_call (func_id, power_state, -__hyp_pa (&kvm_hyp_cpu_resume), -__hyp_pa (init_params)); + ret =psci_call (func_id, power_state, +__hyp_pa (&kvm_hyp_cpu_resume), +__hyp_pa (init_params)); + +return ret; }static int psci_system_suspend (u64 func_id,struct kvm_cpu_context *host_ctxt)
@@ -205,6 +209,7 @@ asmlinkage void __noreturn __kvm_host_psci_cpu_entry(bool is_cpu_on)struct psci_boot_args *boot_args;struct kvm_cpu_context *host_ctxt; +trace_hyp_enter (); host_ctxt =host_data_ptr (host_ctxt);if (is_cpu_on)
@@ -218,6 +223,7 @@ asmlinkage void __noreturn __kvm_host_psci_cpu_entry(bool is_cpu_on)if (is_cpu_on)release_boot_args (boot_args); +trace_hyp_exit ();__host_enter (host_ctxt); }
arch/arm64/kvm/hyp/nvhe/switch.c +4 -1
@@ -7,7 +7,6 @@#include <hyp/switch.h> #include <hyp/sysreg-sr.h> -#include <linux/arm-smccc.h> #include <linux/kvm_host.h> #include <linux/types.h> #include <linux/jump_label.h>
@@ -21,6 +20,7 @@#include <asm/kvm_asm.h> #include <asm/kvm_emulate.h> #include <asm/kvm_hyp.h> +#include <asm/kvm_hypevents.h> #include <asm/kvm_mmu.h> #include <asm/fpsimd.h> #include <asm/debug-monitors.h>
@@ -336,10 +336,13 @@ int __kvm_vcpu_run(struct kvm_vcpu *vcpu)__debug_switch_to_guest (vcpu);do { +trace_hyp_exit (); +/* Jump in the fire! */ exit_code =__guest_enter (vcpu);/* And we're baaack! */ +trace_hyp_enter (); }while (fixup_guest_exit (vcpu, &exit_code));__sysreg_save_state_nvhe (guest_ctxt);
arch/arm64/kvm/hyp_events.c new file +159 -0
159 lines not on this page · raw message
arch/arm64/kvm/hyp_trace.c +12 -0
@@ -6,10 +6,12 @@#include <linux/arm-smccc.h> #include <linux/percpu-defs.h> +#include <linux/trace_events.h> #include <linux/tracefs.h> #include <asm/kvm_host.h> #include <asm/kvm_hyptrace.h> +#include <asm/kvm_hypevents_defs.h> #include "hyp_constants.h" #include "hyp_trace.h"
@@ -570,6 +572,8 @@ static void ht_print_trace_cpu(struct ht_iterator *iter)static int ht_print_trace_fmt (struct ht_iterator *iter) { +struct hyp_event *e; +if (iter->lost_events)trace_seq_printf (&iter->seq,"CPU:%d [LOST %lu EVENTS]\n" , iter->ent_cpu, iter->lost_events);
@@ -577,6 +581,12 @@ static int ht_print_trace_fmt(struct ht_iterator *iter)ht_print_trace_cpu (iter);ht_print_trace_time (iter); + e =hyp_trace_find_event (iter->ent->id); +if (e) + e->trace_func (iter); +else +trace_seq_printf (&iter->seq,"Unknown event id %d\n" , iter->ent->id); +return trace_seq_has_overflowed (&iter->seq) ? -EOVERFLOW :0 ; };
@@ -942,5 +952,7 @@ int hyp_trace_init_tracefs(void) (void *)cpu, &hyp_trace_fops); } +hyp_trace_init_event_tracefs (root); +return 0 ; }
arch/arm64/kvm/hyp_trace.h +4 -17
@@ -3,26 +3,13 @@#ifndef __ARM64_KVM_HYP_TRACE_H__#define __ARM64_KVM_HYP_TRACE_H__ -#include <linux/trace_seq.h> -#include <linux/workqueue.h> - -struct ht_iterator { -struct trace_buffer *trace_buffer; -int cpu; -struct hyp_entry_hdr *ent; -unsigned long lost_events; -int ent_cpu; -size_t ent_size; -u64 ts; -void *spare; -size_t copy_leftover; -struct trace_seq seq; -struct delayed_work poll_work; -}; -#ifdef CONFIG_TRACING int hyp_trace_init_tracefs (void ); +int hyp_trace_init_events (void ); +struct hyp_event *hyp_trace_find_event (int id); +void hyp_trace_init_event_tracefs (struct dentry *parent);#else static inline int hyp_trace_init_tracefs (void ) {return 0 ; } +static inline int hyp_trace_init_events (void ) {return 0 ; }#endif #endif
Signature · 1 line
2.47.1.613.gc27f4b7a9f-goog
Vincent Donnefort
maintainerPATCH v2 12/12 KVM: arm64: Add kselftest for tracefs hyp tracefs
In reply to Vincent DonnefortAdd a test to validate the newly introduced tracefs interface for the pKVM hypervisor. This test covers the usage of extended timestamp and coherence of the tracing clock.
arch/arm64/include/asm/kvm_asm.h +1 -0
@@ -86,6 +86,7 @@ enum __kvm_host_smccc_func { __KVM_HOST_SMCCC_FUNC___pkvm_reset_tracing, __KVM_HOST_SMCCC_FUNC___pkvm_swap_reader_tracing, __KVM_HOST_SMCCC_FUNC___pkvm_enable_event, + __KVM_HOST_SMCCC_FUNC___pkvm_selftest_event, };#define DECLARE_KVM_VHE_SYM (sym) extern char sym[]
arch/arm64/include/asm/kvm_hypevents.h +10 -0
@@ -28,4 +28,14 @@ HYP_EVENT(hyp_exit, ),HE_PRINTK (" " ) ); + +#ifdef CONFIG_PKVM_SELFTESTS +HYP_EVENT (selftest, +HE_PROTO (void), +HE_STRUCT (), +HE_ASSIGN (), +HE_PRINTK (" " ) +);#endif + +#endif /* __ARM64_KVM_HYPEVENTS_H_ */
arch/arm64/kvm/Kconfig +10 -0
@@ -46,6 +46,7 @@ menuconfig KVM config NVHE_EL2_DEBUG bool "Debug mode for non-VHE EL2 object" depends on KVM + select PKVM_SELFTESTS help Say Y here to enable the debug mode for the non-VHE KVM EL2 object. Failure reports will BUG() in the hypervisor. This is intended for
@@ -83,4 +84,13 @@ config PTDUMP_STAGE2_DEBUGFS If in doubt, say N. +config PKVM_SELFTESTS + bool "Protected KVM hypervisor selftests" + depends on KVM + default n + help + Say Y here to enable pKVM hypervisor testing infrastructure. + + If unsure, say N. + endif # VIRTUALIZATION
arch/arm64/kvm/hyp/nvhe/hyp-main.c +14 -0
@@ -436,6 +436,19 @@ static void handle___pkvm_enable_event(struct kvm_cpu_context *host_ctxt)cpu_reg (host_ctxt,1 ) =__pkvm_enable_event (id, enable); } +static void handle___pkvm_selftest_event (struct kvm_cpu_context *host_ctxt) +{ +int smc_ret =SMCCC_RET_NOT_SUPPORTED , ret = -EOPNOTSUPP ; + +#ifdef CONFIG_PKVM_SELFTESTS +trace_selftest (); + smc_ret =SMCCC_RET_SUCCESS ; + ret =0 ; +#endif +cpu_reg (host_ctxt,0 ) = smc_ret; +cpu_reg (host_ctxt,1 ) = ret; +} +typedef void (*hcall_t )(struct kvm_cpu_context *);#define HANDLE_FUNC (x) [__KVM_HOST_SMCCC_FUNC_##x] = (hcall_t)handle_##x
@@ -474,6 +487,7 @@ static const hcall_t host_hcall[] = {HANDLE_FUNC (__pkvm_reset_tracing),HANDLE_FUNC (__pkvm_swap_reader_tracing),HANDLE_FUNC (__pkvm_enable_event), + HANDLE_FUNC(__pkvm_selftest_event), };static void handle_host_hcall (struct kvm_cpu_context *host_ctxt)
arch/arm64/kvm/hyp_trace.c +30 -0
@@ -895,6 +895,35 @@ static int hyp_trace_clock_show(struct seq_file *m, void *v) }DEFINE_SHOW_ATTRIBUTE (hyp_trace_clock); +#ifdef CONFIG_PKVM_SELFTESTS +static int selftest_event_open (struct inode *inode,struct file *file) +{ +if (file->f_mode &FMODE_WRITE ) +return kvm_call_hyp_nvhe (__pkvm_selftest_event); + +return 0 ; +} + +static ssize_t selftest_event_write (struct file *f,const char __user *buf, +size_t cnt,loff_t *pos) +{ +return cnt; +} + +static const struct file_operations selftest_event_fops = { + .open = selftest_event_open, + .write = selftest_event_write, +}; + +static void hyp_trace_init_testing_tracefs (struct dentry *root) +{ +tracefs_create_file ("selftest_event" ,TRACEFS_MODE_WRITE , root,NULL , + &selftest_event_fops); +} +#else +static void hyp_trace_init_testing_tracefs (struct dentry *root) { } +#endif +int hyp_trace_init_tracefs (void ) {struct dentry *root, *per_cpu_root;
@@ -953,6 +982,7 @@ int hyp_trace_init_tracefs(void) }hyp_trace_init_event_tracefs (root); +hyp_trace_init_testing_tracefs (root);return 0 ; }
tools/testing/selftests/hyp-trace/Makefile new file +6 -0
@@ -0,0 +1,6 @@ +# SPDX-License-Identifier: GPL-2.0 +all: + +TEST_PROGS := hyp-trace-test + +include ../lib.mk
tools/testing/selftests/hyp-trace/config new file +4 -0
@@ -0,0 +1,4 @@ +CONFIG_FTRACE=y +CONFIG_ARM64=y +CONFIG_KVM=y +CONFIG_PROTECTED_NVHE_TESTING=y
tools/testing/selftests/hyp-trace/hyp-trace-test new file +254 -0
254 lines not on this page · raw message
Signature · 1 line
2.47.1.613.gc27f4b7a9f-goog