selftests/resctrl: Add resctrl selftest
Subsystems (2)
cover selftests/resctrl: Add resctrl selftest
With more and more resctrl features are being added by Intel, AMD and ARM, a test tool is becoming more and more useful to validate that both hardware and software functionalities work as expected.
We introduce resctrl selftest to cover resctrl features on both X86 and ARM architectures. It first implements MBM (Memory Bandwidth Monitoring) and MBA (Memory Bandwidth Allocation) tests. We can enhance the selftest tool to include more functionality tests in future.
The tests are in tools/testing/selftests/resctrl in order to have generic test code for all architectures.
Arshiya Hayatkhan Pathan (2):
selftests/resctrl: Add mbm test
selftests/resctrl: Add mba test
Fenghua Yu (1):
selftests/resctrl: Add the test in MAINTAINERS
Sai Praneeth Prakhya (4):
selftests/resctrl: Add basic resctrl file system operations and data selftests/resctrl: Read memory bandwidth from perf IMC counter and
from resctrl file system
selftests/resctrl: Add callback to start a benchmark selftests/resctrl: Add built in benchmark
MAINTAINERS | 1 + tools/testing/selftests/resctrl/Makefile | 16 ++ tools/testing/selftests/resctrl/fill_buf.c | 178 ++++++++++++ tools/testing/selftests/resctrl/mba.c | 144 ++++++++++ tools/testing/selftests/resctrl/mbm.c | 113 ++++++++ tools/testing/selftests/resctrl/resctrl.c | 367 ++++++++++++++++++++++++ tools/testing/selftests/resctrl/resctrl.h | 107 +++++++ tools/testing/selftests/resctrl/resctrl_membw.c | 328 +++++++++++++++++++++ tools/testing/selftests/resctrl/resctrl_membw.h | 49 ++++ tools/testing/selftests/resctrl/resctrl_tests.c | 104 +++++++ tools/testing/selftests/resctrl/resctrl_val.c | 192 +++++++++++++ 11 files changed, 1599 insertions(+) create mode 100644 tools/testing/selftests/resctrl/Makefile create mode 100644 tools/testing/selftests/resctrl/fill_buf.c create mode 100644 tools/testing/selftests/resctrl/mba.c create mode 100644 tools/testing/selftests/resctrl/mbm.c create mode 100644 tools/testing/selftests/resctrl/resctrl.c create mode 100644 tools/testing/selftests/resctrl/resctrl.h create mode 100644 tools/testing/selftests/resctrl/resctrl_membw.c create mode 100644 tools/testing/selftests/resctrl/resctrl_membw.h create mode 100644 tools/testing/selftests/resctrl/resctrl_tests.c create mode 100644 tools/testing/selftests/resctrl/resctrl_val.c
Signature · 1 line
2.5.0
PATCH 1/7 selftests/resctrl: Add basic resctrl file system operations and data
From: Sai Praneeth Prakhya <[email protected]>
The basic resctrl file system operations and data are added for future usage by resctrl selftest tool.
--- tools/testing/selftests/resctrl/Makefile | 10 + tools/testing/selftests/resctrl/resctrl.c | 367 ++++++++++++++++++++++++++++++ tools/testing/selftests/resctrl/resctrl.h | 74 ++++++ 3 files changed, 451 insertions(+) create mode 100644 tools/testing/selftests/resctrl/Makefile create mode 100644 tools/testing/selftests/resctrl/resctrl.c create mode 100644 tools/testing/selftests/resctrl/resctrl.h
tools/testing/selftests/resctrl/Makefile new file +10 -0
@@ -0,0 +1,10 @@ +CC = gcc +CFLAGS = -g -Wall + +*.o: *.c + $(CC) $(CFLAGS) -c *.c + +.PHONY: clean + +clean: + $(RM) *.o *~
tools/testing/selftests/resctrl/resctrl.c new file +367 -0
367 lines not on this page · raw message
tools/testing/selftests/resctrl/resctrl.h new file +74 -0
74 lines not on this page · raw message
Signature · 1 line
2.5.0
PATCH 2/7 selftests/resctrl: Read memory bandwidth from perf IMC counter and from resctrl file system
In reply to Fenghua YuFrom: Sai Praneeth Prakhya <[email protected]>
Total memory bandwidth can be monitored from perf IMC counter and from resctrl file system. Later the two will be compared to verify the total memory bandwidth read from resctrl is correct.
--- tools/testing/selftests/resctrl/resctrl_membw.c | 328 ++++++++++++++++++++++++ tools/testing/selftests/resctrl/resctrl_membw.h | 49 ++++ 2 files changed, 377 insertions(+) create mode 100644 tools/testing/selftests/resctrl/resctrl_membw.c create mode 100644 tools/testing/selftests/resctrl/resctrl_membw.h
tools/testing/selftests/resctrl/resctrl_membw.c new file +328 -0
@@ -0,0 +1,328 @@ +// SPDX-License-Identifier: GPL-2.0 +/* +* Memory bandwidth monitoring library +* +* Copyright (C) 2018 Intel Corporation +* +* Authors: +* Arshiya Hayatkhan Pathan <[email protected]> +* Sai Praneeth Prakhya <[email protected]>, +* Fenghua Yu <[email protected]> +*/ +#include "resctrl_membw.h" + +static struct imc_counter_config imc_counters_config[MAX_IMCs][2 ]; +static char mbm_total_path[1024 ]; +static int imcs; + +void membw_initialize_perf_event_attr (int i,int j) +{ +memset (&imc_counters_config[i][j].pe,0 , +sizeof (struct perf_event_attr )); + imc_counters_config[i][j].pe.type = imc_counters_config[i][j].type; + imc_counters_config[i][j].pe.size =sizeof (struct perf_event_attr ); + imc_counters_config[i][j].pe.disabled =1 ; + imc_counters_config[i][j].pe.inherit =1 ; + imc_counters_config[i][j].pe.exclude_guest =1 ; + imc_counters_config[i][j].pe.config = + imc_counters_config[i][j].umask <<8 | + imc_counters_config[i][j].event; + imc_counters_config[i][j].pe.sample_type =PERF_SAMPLE_IDENTIFIER ; + imc_counters_config[i][j].pe.read_format = +PERF_FORMAT_TOTAL_TIME_ENABLED |PERF_FORMAT_TOTAL_TIME_RUNNING ; +} + +void open_perf_event (int i,int cpu_no,int j) +{ + imc_counters_config[i][j].fd = +perf_event_open (&imc_counters_config[i][j].pe,-1 , cpu_no,-1 , +PERF_FLAG_FD_CLOEXEC ); + +if (imc_counters_config[i][j].fd ==-1 ) { +fprintf (stderr,"Error opening leader %llx\n" , + imc_counters_config[i][j].pe.config); +ctrlc_handler (0 ,NULL ,NULL ); + +exit (EXIT_FAILURE ); + } +} + +void membw_ioctl_perf_event_ioc_resest_enable (int i,int j) +{ +ioctl (imc_counters_config[i][j].fd,PERF_EVENT_IOC_RESET ,0 ); +ioctl (imc_counters_config[i][j].fd,PERF_EVENT_IOC_ENABLE ,0 ); +} + +void membw_ioctl_perf_event_ioc_disable (int i,int j) +{ +ioctl (imc_counters_config[i][j].fd,PERF_EVENT_IOC_DISABLE ,0 ); +} + +/* +* get_event_and_umask: Parse config into event and umask +* @cas_count_cfg: Config +* @count: iMC number +* @op: Operation (read/write) +*/ +void get_event_and_umask (char *cas_count_cfg,int count,bool op) +{ +char **token =malloc (MAX_TOKENS *sizeof (char *)); +int i =0 ; + +strcat (cas_count_cfg,"," ); + token[0 ] =malloc (10 *sizeof (char )); + token[0 ] =strtok (cas_count_cfg,"=," ); + +for (i =1 ; i <MAX_TOKENS ; i++) { + token[i] =malloc (10 *sizeof (char )); + token[i] =strtok (NULL ,"=," ); +if (token[i] ==NULL ) +break ; + } + +for (i =0 ; i <MAX_TOKENS ; i++) { +if (token[i] ==NULL ) +break ; +if (strcmp (token[i],"event" ) ==0 ) { +if (op ==READ ) + imc_counters_config[count][READ ].event = +strtol (token[i+1 ],NULL ,16 ); +else + imc_counters_config[count][WRITE ].event = +strtol (token[i+1 ],NULL ,16 ); + } +if (strcmp (token[i],"umask" ) ==0 ) { +if (op ==READ ) + imc_counters_config[count][READ ].umask = +strtol (token[i+1 ],NULL ,16 ); +else + imc_counters_config[count][WRITE ].umask = +strtol (token[i+1 ],NULL ,16 ); + } + } +} + +/* Get type and config (read and write) of an iMC counter */ +void read_from_imc_dir (char *imc_dir,int count) +{ +char cas_count_cfg[1024 ], imc_counter_cfg[1024 ], imc_counter_type[1024 ]; +FILE *fp; + +/* Get type of iMC counter */ +sprintf (imc_counter_type,"%s%s" , imc_dir,"type" ); + fp =fopen (imc_counter_type,"r" ); +if (!fp ||fscanf (fp,"%u" , + &imc_counters_config[count][READ ].type) <=0 ||fclose (fp) ==EOF ) +CHILD_EXIT ("Could not get imc type" ); + + imc_counters_config[count][WRITE ].type = + imc_counters_config[count][READ ].type; + +/* Get read config */ +sprintf (imc_counter_cfg,"%s%s" , imc_dir,READ_FILE_NAME ); + fp =fopen (imc_counter_cfg,"r" ); +if (!fp ||fscanf (fp,"%s" , cas_count_cfg) <=0 ||fclose (fp) ==EOF ) +CHILD_EXIT ("Could not get imc cas count read" ); + +get_event_and_umask (cas_count_cfg, count,READ ); + +/* Get write config */ +sprintf (imc_counter_cfg,"%s%s" , imc_dir,WRITE_FILE_NAME ); + fp =fopen (imc_counter_cfg,"r" ); +if (!fp ||fscanf (fp,"%s" , cas_count_cfg) <=0 ||fclose (fp) ==EOF ) +CHILD_EXIT ("Could not get imc cas count write" ); + +get_event_and_umask (cas_count_cfg, count,WRITE ); +} + +/* +* A system can have 'n' number of iMC (Integrated Memory Controller) +* counters, get that 'n'. For each iMC counter get it's type and config. +* Also, each counter has two configs, one for read and the other for write. +* A config again has two parts, event and umask. +* Enumerate all these details into an array of structures. +*/ +static int num_of_imcs (void ) +{ +char imc_dir[1024 ]; +struct dirent *ep; +unsigned int count =0 ; +DIR *dp; + + dp =opendir (DYN_PMU_PATH ); +if (dp !=NULL ) { +while ((ep =readdir (dp)) !=NULL ) { +if (strstr (ep->d_name,UNCORE_IMC ) !=NULL ) { +sprintf (imc_dir,"%s/%s/" ,DYN_PMU_PATH , + ep->d_name); +read_from_imc_dir (imc_dir, count); + count++; + } + } +closedir (dp); +if (count ==0 ) +CHILD_EXIT ("Unable find iMC counters!\n" ); + }else { +CHILD_EXIT ("Unable to open PMU directory!\n" ); + } + +return count; +} + +void initialize_mem_bw_imc (void ) +{ +int imc; + + imcs =num_of_imcs (); + +/* Initialize perf_event_attr structures for all iMC's */ +for (imc =0 ; imc < imcs; imc++) { +for (int j =0 ; j <2 ; j++) +membw_initialize_perf_event_attr (imc, j); + } +} + +/* +* get_mem_bw_imc: Memory band width as reported by iMC counters +* @cpu_no: CPU number that the benchmark PID is binded to +* @bw_report: Bandwidth report type (reads, writes) +* +* Memory B/W utilized by a process on a socket can be calculated using +* iMC counters. Perf events are used to read these counters. +*/ +float get_mem_bw_imc (int cpu_no,char *bw_report) +{ +float reads, writes, of_mul_read, of_mul_write; +int imc; + +/* Start all iMC counters to log values (both read and write) */ + reads =0 , writes =0 , of_mul_read =1 , of_mul_write =1 ; +for (imc =0 ; imc < imcs; imc++) { +for (int j =0 ; j <2 ; j++) +open_perf_event (imc, cpu_no, j); +for (int j =0 ; j <2 ; j++) +membw_ioctl_perf_event_ioc_resest_enable (imc, j); + } + +sleep (1 ); + +/* Stop counters after a second to get results (both read and write) */ +for (imc =0 ; imc < imcs; imc++) { +for (int j =0 ; j <2 ; j++) +membw_ioctl_perf_event_ioc_disable (imc, j); + } + +/* +* Get results which are stored in struct type imc_counter_config +* Take over flow into consideration before calculating total b/w +*/ +for (imc =0 ; imc < imcs; imc++) { +struct imc_counter_config *r = + &imc_counters_config[imc][READ ]; +struct imc_counter_config *w = + &imc_counters_config[imc][WRITE ]; + +if (read (r->fd, &r->return_value, +sizeof (struct membw_read_format )) ==-1 ) +CHILD_EXIT ("Couldn't get read b/w through iMC" ); + +if (read (w->fd, &w->return_value, +sizeof (struct membw_read_format )) ==-1 ) +CHILD_EXIT ("Couldn't get write bw through iMC" ); + +__u64 r_time_enabled = r->return_value.time_enabled; +__u64 r_time_running = r->return_value.time_running; + +if (r_time_enabled != r_time_running) + of_mul_read = (float )r_time_enabled / + (float )r_time_running; + +__u64 w_time_enabled = w->return_value.time_enabled; +__u64 w_time_running = w->return_value.time_running; + +if (w_time_enabled != w_time_running) + of_mul_write = (float ) w_time_enabled / + (float ) w_time_running; + reads += r->return_value.value * of_mul_read *SCALE ; + writes += w->return_value.value * of_mul_write *SCALE ; + } + +for (imc =0 ; imc < imcs; imc++) { +close (imc_counters_config[imc][READ ].fd); +close (imc_counters_config[imc][WRITE ].fd); + } + +if (strcmp (bw_report,"reads" ) ==0 ) +return reads; + +if (strcmp (bw_report,"writes" ) ==0 ) +return writes; + +return (reads + writes); +} + +void set_mbm_path (const char *ctrlgrp,const char *mongrp,char sock_num) +{ +if (ctrlgrp && mongrp) +sprintf (mbm_total_path,CON_MON_MBM_LOCAL_BYTES_PATH , +RESCTRL_PATH , ctrlgrp, mongrp, sock_num); +else if (!ctrlgrp && mongrp) +sprintf (mbm_total_path,MON_MBM_LOCAL_BYTES_PATH ,RESCTRL_PATH , + mongrp, sock_num); +else if (ctrlgrp && !mongrp) +sprintf (mbm_total_path,CON_MBM_LOCAL_BYTES_PATH ,RESCTRL_PATH , + ctrlgrp, sock_num); +else if (!ctrlgrp && !mongrp) +sprintf (mbm_total_path,MBM_LOCAL_BYTES_PATH ,RESCTRL_PATH , + sock_num); + +} + +/* +* initialize_mem_bw_resctrl: Appropriately populate "mbm_total_path" +* @ctrlgrp: Name of the control monitor group (con_mon grp) +* @mongrp: Name of the monitor group (mon grp) +* @cpu_no: CPU number that the benchmark PID is binded to +* @resctrl_val: Resctrl feature (Eg: mbm, mba.. etc) +*/ +void initialize_mem_bw_resctrl (const char *ctrlgrp,const char *mongrp, +int cpu_no,char *resctrl_val) +{ +char sock_num; + + sock_num =get_sock_num (cpu_no); + +if (strcmp (resctrl_val,"mbm" ) ==0 ) +set_mbm_path (ctrlgrp, mongrp, sock_num); + +if ((strcmp (resctrl_val,"mba" ) ==0 )) { +if (ctrlgrp) +sprintf (mbm_total_path,CON_MBM_LOCAL_BYTES_PATH , +RESCTRL_PATH , ctrlgrp, sock_num); +else +sprintf (mbm_total_path,MBM_LOCAL_BYTES_PATH , +RESCTRL_PATH , sock_num); + } +} + +/* +* Get MBM Local bytes as reported by resctrl FS +* For MBM, +* 1. If con_mon grp and mon grp are given, then read from con_mon grp's mon grp +* 2. If only con_mon grp is given, then read from con_mon grp +* 3. If both are not given, then read from root con_mon grp +* For MBA, +* 1. If con_mon grp is given, then read from it +* 2. If con_mon grp is not given, then read from root con_mon grp +*/ +unsigned long long get_mem_bw_resctrl (void ) +{ +unsigned long long mbm_total =0 ; +FILE *fp; + + fp =fopen (mbm_total_path,"r" ); +if (!fp ||fscanf (fp,"%llu" , &mbm_total) <=0 ||fclose (fp) ==EOF ) +CHILD_EXIT ("Could not get mbm local bytes" ); + +return mbm_total; +}
tools/testing/selftests/resctrl/resctrl_membw.h new file +49 -0
49 lines not on this page · raw message
Signature · 1 line
2.5.0
PATCH 3/7 selftests/resctrl: Add callback to start a benchmark
In reply to Fenghua YuFrom: Sai Praneeth Prakhya <[email protected]>
The callback starts a child and puts the child pid in created resctrl group with specified memory bandwidth in schemata. The child starts running benchmark. Later the callback will be used by tests to start the benchmark.
--- tools/testing/selftests/resctrl/resctrl.h | 27 ++++ tools/testing/selftests/resctrl/resctrl_val.c | 192 ++++++++++++++++++++++++++ 2 files changed, 219 insertions(+) create mode 100644 tools/testing/selftests/resctrl/resctrl_val.c
tools/testing/selftests/resctrl/resctrl.h +27 -0
@@ -51,6 +51,32 @@exit (EXIT_FAILURE ); \ }while (0 ) +/* +* resctrl_val: Functional validation of resctrl features +* @resctrl_val: Resctrl feature (Eg: mbm, mba.. etc) +* @ctrlgrp: Name of the control monitor group (con_mon grp) +* @mongrp: Name of the monitor group (mon grp) +* @schemata: Schemata while validating allocation type features +* @cpu_no: CPU number to which the benchmark would be binded +* @mum_resctrlfs: Should the resctrl FS be remounted? +* @num_of_runs: Number of runs before exiting +* @filename: Name of file to which the o/p should be written +* @bw_report: Bandwidth report type (reads vs writes) +*/ +struct resctrl_val_param { +char *resctrl_val; +char ctrlgrp[64 ]; +char mongrp[64 ]; +char *schemata; +int cpu_no; +int span; +int mum_resctrlfs; +int num_of_runs; +char filename[64 ]; +char *bw_report; +char *bm_type; +}; +pid_t bm_pid, ppid;int ben_count;
@@ -70,5 +96,6 @@ void write_bm_pid_to_resctrl(pid_t bm_pid, char *ctrlgrp, char *mongrp,int perf_event_open (struct perf_event_attr *hw_event,pid_t pid,int cpu,int group_fd,unsigned long flags);int run_fill_buf (int span,int malloc_and_init_memory,int memflush,int op); +void resctrl_val (char **benchmark_cmd,struct resctrl_val_param *param);#endif /* RESCTRL_H */
tools/testing/selftests/resctrl/resctrl_val.c new file +192 -0
192 lines not on this page · raw message
Signature · 1 line
2.5.0
PATCH 4/7 selftests/resctrl: Add built in benchmark
In reply to Fenghua YuFrom: Sai Praneeth Prakhya <[email protected]>
Built-in benchmark fill_buf generates stressful memory bandwidth and cache traffic.
Later it will be used as a default benchmark by various resctrl tests such as MBA (Memory Bandwidth Allocation) and MBM (Memory Bandwidth Monitoring) tests.
--- tools/testing/selftests/resctrl/fill_buf.c | 178 +++++++++++++++++++++++++++++ 1 file changed, 178 insertions(+) create mode 100644 tools/testing/selftests/resctrl/fill_buf.c
tools/testing/selftests/resctrl/fill_buf.c new file +178 -0
@@ -0,0 +1,178 @@ +// SPDX-License-Identifier: GPL-2.0 +/* +* fill_buf benchmark +* +* Copyright (C) 2018 Intel Corporation +* +* Authors: +* Arshiya Hayatkhan Pathan <[email protected]> +* Sai Praneeth Prakhya <[email protected]>, +* Fenghua Yu <[email protected]> +*/ +#include <stdio.h> +#include <unistd.h> +#include <stdlib.h> +#include <sys/types.h> +#include <sys/wait.h> +#include <inttypes.h> +#include <malloc.h> +#include <string.h> + +#include "resctrl.h" + +#define CL_SIZE (64) +#define PAGE_SIZE (4 * 1024) +#define MB (1024 * 1024) + +static unsigned char *startptr; + +static void sb (void ) +{ + asmvolatile ("sfence\n\t" + : : :"memory" ); +} + +static void ctrl_handler (int signo) +{ +free (startptr); +printf ("\nEnding\n" ); +sb (); +exit (EXIT_SUCCESS ); +} + +static void cl_flush (void *p) +{ + asmvolatile ("clflush (%0)\n\t" + : :"r" (p) :"memory" ); +} + +static void mem_flush (void *p,size_t s) +{ +char *cp = (char *)p; +size_t i =0 ; + + s = s /CL_SIZE ;/* mem size in cache llines */ + +for (i =0 ; i < s; i++) +cl_flush (&cp[i *CL_SIZE ]); + +sb (); +} + +static void *malloc_and_init_memory (size_t s) +{ +uint64_t *p64; +size_t s64; + +void *p =memalign (PAGE_SIZE , s); + + p64 = (uint64_t *)p; + s64 = s /sizeof (uint64_t ); + +while (s64 >0 ) { + *p64 = (uint64_t )rand (); + p64 += (CL_SIZE /sizeof (uint64_t )); + s64 -= (CL_SIZE /sizeof (uint64_t )); + } + +return p; +} + +static void fill_cache_read (unsigned char *start_ptr,unsigned char *end_ptr) +{ +while (1 ) { +unsigned char sum, *p; + + p = start_ptr; +/* Read two chars in each cache line to stress cache */ +while (p < (end_ptr -1024 )) { + sum += p[0 ] + p[32 ] + p[64 ] + p[96 ] + p[128 ] + + p[160 ] + p[192 ] + p[224 ] + p[256 ] + p[288 ] + + p[320 ] + p[352 ] + p[384 ] + p[416 ] + p[448 ] + + p[480 ] + p[512 ] + p[544 ] + p[576 ] + p[608 ] + + p[640 ] + p[672 ] + p[704 ] + p[736 ] + p[768 ] + + p[800 ] + p[832 ] + p[864 ] + p[896 ] + p[928 ] + + p[960 ] + p[992 ]; + p +=1024 ; + } + } +} + +static void fill_cache_write (unsigned char *start_ptr,unsigned char *end_ptr) +{ +while (1 ) { +while (start_ptr < end_ptr) { + *start_ptr ='1' ; + start_ptr += (CL_SIZE /2 ); + } + start_ptr = startptr; + } +} + +static void +fill_cache (unsigned long long buf_size,int malloc_and_init, +int memflush,int op) +{ +unsigned char *start_ptr, *end_ptr; +unsigned long long i; + +if (malloc_and_init) { + start_ptr =malloc_and_init_memory (buf_size); +printf ("Started benchmark with memalign\n" ); + }else { + start_ptr =malloc (buf_size); +printf ("Started benchmark with malloc\n" ); + } + +if (start_ptr ==NULL ) +return ; + + startptr = start_ptr; + end_ptr = start_ptr + buf_size; + +/* +* It's better to touch the memory once to avoid any compiler +* optimizations +*/ +if (!malloc_and_init) { +for (i =0 ; i < buf_size; i++) + *start_ptr++ = (unsigned char )rand (); + } + + start_ptr = startptr; + +/* Flush the memory before using to avoid "cache hot pages" effect */ +if (memflush) { +mem_flush (start_ptr, buf_size); +printf ("Started benchmark with memflush\n" ); + }else { +printf ("Started benchmark *without* memflush\n" ); + } + +if (op ==0 ) +fill_cache_read (start_ptr, end_ptr); +else +fill_cache_write (start_ptr, end_ptr); + +free (startptr); + +exit (EXIT_FAILURE ); +} + +int run_fill_buf (int span,int malloc_and_init_memory,int memflush,int op) +{ +unsigned long long cache_size = span *MB ; + +/* set up ctrl-c handler */ +if (signal (SIGINT , ctrl_handler) ==SIG_ERR ) +printf ("Failed to catch SIGINT!\n" ); +if (signal (SIGHUP , ctrl_handler) ==SIG_ERR ) +printf ("Failed to catch SIGHUP!\n" ); + +printf ("Cache size in Bytes = %llu\n" , cache_size); + +fill_cache (cache_size, malloc_and_init_memory, memflush, op); +printf ("Unable to allocate memory\n" ); + +exit (EXIT_FAILURE ); +}
Signature · 1 line
2.5.0
PATCH 5/7 selftests/resctrl: Add mbm test
In reply to Fenghua YuFrom: Arshiya Hayatkhan Pathan <[email protected]>
mbm test is the first implemented selftest. It starts a stressful memory bandwidth benchmark and assigns the bandwidth pid in a resctrl monitoring group. Read and compare perf IMC counter and mbm total bytes for the benchmark. The numbers should be close enough to pass the test.
Default benchmark is built-in fill_buf. But users can specify their own benchmark by option "-ben".
We can add memory bandwidth monitoring for multiple processes in the future.
--- tools/testing/selftests/resctrl/Makefile | 8 +- tools/testing/selftests/resctrl/mbm.c | 113 ++++++++++++++++++++++++ tools/testing/selftests/resctrl/resctrl.h | 3 + tools/testing/selftests/resctrl/resctrl_tests.c | 94 ++++++++++++++++++++ 4 files changed, 217 insertions(+), 1 deletion(-) create mode 100644 tools/testing/selftests/resctrl/mbm.c create mode 100644 tools/testing/selftests/resctrl/resctrl_tests.c
tools/testing/selftests/resctrl/Makefile +7 -1
@@ -1,10 +1,16 @@ CC = gcc CFLAGS = -g -Wall +all: resctrl_tests + *.o: *.c $(CC) $(CFLAGS) -c *.c +resctrl_tests: *.o + $(CC) $(CFLAGS) -o resctrl_tests resctrl_tests.o resctrl.o \ + resctrl_val.o resctrl_membw.o fill_buf.o mbm.o + .PHONY: clean clean: - $(RM) *.o *~ + $(RM) *.o *~ resctrl_tests
tools/testing/selftests/resctrl/mbm.c new file +113 -0
113 lines not on this page · raw message
tools/testing/selftests/resctrl/resctrl.h +3 -0
@@ -98,4 +98,7 @@ int perf_event_open(struct perf_event_attr *hw_event, pid_t pid, int cpu,int run_fill_buf (int span,int malloc_and_init_memory,int memflush,int op);void resctrl_val (char **benchmark_cmd,struct resctrl_val_param *param); +int mbm_bw_change (char *resctrl_val_type,int core_id,int span, +char *bw_report,char *bm_type,char **benchmark_cmd); +#endif /* RESCTRL_H */
tools/testing/selftests/resctrl/resctrl_tests.c new file +94 -0
94 lines not on this page · raw message
Signature · 1 line
2.5.0
PATCH 6/7 selftests/resctrl: Add mba test
In reply to Fenghua YuFrom: Arshiya Hayatkhan Pathan <[email protected]>
mba test starts a stressful memory bandwidth benchmark and allocates memory bandwidth from 100% down to 10% for the benchmark process. For each allocation, compare perf IMC counter and mbm total bytes from resctrl. The difference between the two values should be within a threshold to pass the test.
Default benchmark is built-in fill_buf. But users can specify their own benchmark by option "-ben".
We can add memory bandwidth allocation for multiple processes in the future.
--- tools/testing/selftests/resctrl/Makefile | 2 +- tools/testing/selftests/resctrl/mba.c | 144 ++++++++++++++++++++++++ tools/testing/selftests/resctrl/resctrl.h | 3 + tools/testing/selftests/resctrl/resctrl_tests.c | 10 ++ 4 files changed, 158 insertions(+), 1 deletion(-) create mode 100644 tools/testing/selftests/resctrl/mba.c
tools/testing/selftests/resctrl/Makefile +1 -1
@@ -8,7 +8,7 @@ all: resctrl_tests resctrl_tests: *.o $(CC) $(CFLAGS) -o resctrl_tests resctrl_tests.o resctrl.o \ - resctrl_val.o resctrl_membw.o fill_buf.o mbm.o + resctrl_val.o resctrl_membw.o fill_buf.o mbm.o mba.o .PHONY: clean
tools/testing/selftests/resctrl/mba.c new file +144 -0
144 lines not on this page · raw message
tools/testing/selftests/resctrl/resctrl.h +3 -0
@@ -100,5 +100,8 @@ void resctrl_val(char **benchmark_cmd, struct resctrl_val_param *param);int mbm_bw_change (char *resctrl_val_type,int core_id,int span,char *bw_report,char *bm_type,char **benchmark_cmd); +int mba_schemata_change (char *resctrl_val_type,int core_id,int span, +char *bw_report,char *bm_type, +char **benchmark_cmd);#endif /* RESCTRL_H */
tools/testing/selftests/resctrl/resctrl_tests.c +10 -0
@@ -90,5 +90,15 @@ int main(int argc, char **argv)if (res)printf ("Error in running tests for mbm bw change!\n" ); +printf ("\nTest 2: MBA Schemata Change Starting..\n" ); +if (!has_ben) { + span =250 ; +sprintf (benchmark_cmd[1 ],"%d" , span); + } + res =mba_schemata_change ("mba" , core_id, span, bw_report, + bm_type, benchmark_cmd); +if (res) +printf ("Error in running tests for mba-change-schemata!\n" ); +return 0 ; }
Signature · 1 line
2.5.0
PATCH 7/7 selftests/resctrl: Add the test in MAINTAINERS
In reply to Fenghua YuThe resctrl selftests will be maintained by RDT maintainers.
--- MAINTAINERS | 1 + 1 file changed, 1 insertion(+)
MAINTAINERS +1 -0
@@ -12271,6 +12271,7 @@ S: Supported F: arch/x86/kernel/cpu/intel_rdt* F: arch/x86/include/asm/intel_rdt_sched.h F: Documentation/x86/intel_rdt* +F: tools/testing/selftests/resctrl READ-COPY UPDATE (RCU) M: "Paul E. McKenney" <[email protected]>
Signature · 1 line
2.5.0
Moger, Babu
reviewerHi Fenghua,
Quoted 3 lines · Fenghua Yu
With more and more resctrl features are being added by Intel, AMD and ARM, a test tool is becoming more and more useful to validate that both hardware and software functionalities work as expected.
I like the initiative here. It is always good to have a single code base.
One question. I see that there is a tool at
https://github.com/intel/intel-cmt-cat to test and verify the functionality of resctrl feature. I also see some of the distros have this tool already. Is this tool going to replace intel-cmt-cat? I have not looked at the patches closely yet.
Quoted 46 lines
We introduce resctrl selftest to cover resctrl features on both X86 and ARM architectures. It first implements MBM (Memory Bandwidth Monitoring) and MBA (Memory Bandwidth Allocation) tests. We can enhance the selftest tool to include more functionality tests in future.
The tests are in
tools/testing/selftests/resctrlin order to have generic test code for all architectures.Arshiya Hayatkhan Pathan (2):
selftests/resctrl: Add mbm test
selftests/resctrl: Add mba testFenghua Yu (1):
selftests/resctrl: Add the test inMAINTAINERSSai Praneeth Prakhya (4):
selftests/resctrl: Add basic resctrl file system operations and data selftests/resctrl: Read memory bandwidth from perf IMC counter and
from resctrl file system
selftests/resctrl: Add callback to start a benchmark selftests/resctrl: Add built in benchmarkMAINTAINERS | 1 + tools/testing/selftests/resctrl/Makefile | 16 ++ tools/testing/selftests/resctrl/fill_buf.c | 178 ++++++++++++ tools/testing/selftests/resctrl/mba.c | 144 ++++++++++ tools/testing/selftests/resctrl/mbm.c | 113 ++++++++ tools/testing/selftests/resctrl/resctrl.c | 367 ++++++++++++++++++++++++ tools/testing/selftests/resctrl/resctrl.h | 107 +++++++ tools/testing/selftests/resctrl/resctrl_membw.c | 328 +++++++++++++++++++++ tools/testing/selftests/resctrl/resctrl_membw.h | 49 ++++ tools/testing/selftests/resctrl/resctrl_tests.c | 104 +++++++ tools/testing/selftests/resctrl/resctrl_val.c | 192 +++++++++++++ 11 files changed, 1599 insertions(+) create mode 100644 tools/testing/selftests/resctrl/Makefile create mode 100644 tools/testing/selftests/resctrl/fill_buf.c create mode 100644 tools/testing/selftests/resctrl/mba.c create mode 100644 tools/testing/selftests/resctrl/mbm.c create mode 100644 tools/testing/selftests/resctrl/resctrl.c create mode 100644 tools/testing/selftests/resctrl/resctrl.h create mode 100644 tools/testing/selftests/resctrl/resctrl_membw.c create mode 100644 tools/testing/selftests/resctrl/resctrl_membw.h create mode 100644 tools/testing/selftests/resctrl/resctrl_tests.c create mode 100644 tools/testing/selftests/resctrl/resctrl_val.c
A message that is not in the archive (sent off-list or not yet received): FFF73D592F13FD46B8700F0A279B802F485D47DE@ORSMSX114.amr.corp.intel.com
Quoted 11 lines
From: Moger, Babu [mailto:[email protected]]
Quoted 3 lines · Fenghua Yu
With more and more resctrl features are being added by Intel, AMD and ARM, a test tool is becoming more and more useful to validate that both hardware and software functionalities work as expected.
I like the initiative here. It is always good to have a single code base.
One question. I see that there is a tool at https://github.com/intel/intel-cmt-cat to test and verify the functionality of resctrl feature. I also see some of the distros have this tool already. Is this tool going to replace intel-cmt-cat? I have not looked at the
patches closely yet.
No, the selftest in this patch set will not replace intel-cmt-cat or vice versa.
The selftest in this patch set has a different purpose from intel-cmt-cat:
the selftest is a test tool which validates resctrl functionalities while intel-cmt-cat is mainly a utility that provides base library for higher level applications including performance analysis tools, benchmark measurement tools, and potential resctrl tests. For example, running MBA test in the selftests tells MBA working or not working (fail/pass) right way. The intel-cmt-cat doesn't have this testing capability unless we extend the tool.
And intel-cmt-cat is maintained and developed by Intel. I don't think it's easy to extend it to AMD and ARM features. The selftest will be maintained and developed by the community and will hopefully cover all architectures.
We have seen a few issues recently in resctrl and may see more issues while expending the features. A convevient selftest may be useful to help identify and fix those potential issues.
Thanks.
-Fenghua
Moger, Babu
reviewerQuoted 15 lines · Fenghua Yu
Quoted 11 lines
From: Moger, Babu [mailto:[email protected]]
Quoted 3 lines · Fenghua Yu
With more and more resctrl features are being added by Intel, AMD and ARM, a test tool is becoming more and more useful to validate that both hardware and software functionalities work as expected.
I like the initiative here. It is always good to have a single code base.
One question. I see that there is a tool at https://github.com/intel/intel-cmt-cat to test and verify the functionality of resctrl feature. I also see some of the distros have this tool already. Is this tool going to replace intel-cmt-cat? I have not looked at the
patches closely yet.No, the selftest in this patch set will not replace intel-cmt-cat or vice versa.
Quoted 6 lines
The selftest in this patch set has a different purpose from intel-cmt-cat:
the selftest is a test tool which validates resctrl functionalities while intel-cmt-cat is mainly a utility that provides base library for higher level applications including performance analysis tools, benchmark measurement tools, and potential resctrl tests. For example, running MBA test in the selftests tells MBA working or not working (fail/pass) right way. The
Ok. Sure. Let me take a look at selftest closely. Will send my feedback soon.
Quoted 5 lines
intel-cmt-cat doesn't have this testing capability unless we extend the tool.
And intel-cmt-cat is maintained and developed by Intel. I don't think it's easy to extend it to AMD and ARM features. The selftest will be maintained
We1l.. We were hoping to have a common tool across. It makes it easy for distros. Probably, we can have a separate discussion on this.
Quoted 11 lines
and developed by the community and will hopefully cover all architectures.
We have seen a few issues recently in resctrl and may see more issues while expending the features. A convevient selftest may be useful to help identify and fix those potential issues.
Thanks.
-Fenghua
Quoted 11 lines
Quoted 10 lines
No, the selftest in this patch set will not replace intel-cmt-cat or vice versa.
The selftest in this patch set has a different purpose from intel-cmt-cat:
the selftest is a test tool which validates resctrl functionalities while intel-cmt-cat is mainly a utility that provides base library for higher level applications including performance analysis tools, benchmark measurement tools, and potential resctrl tests. For example, running MBA test in the selftests tells MBA working or not working (fail/pass) right way. The
Quoted 1 line
Ok. Sure. Let me take a look at selftest closely. Will send my feedback soon.
Great!
Quoted 11 lines
Quoted 7 lines
intel-cmt-cat doesn't have this testing capability unless we extend the tool.
And intel-cmt-cat is maintained and developed by Intel. I don't think it's easy to extend it to AMD and ARM features. The selftest will be maintained
We1l.. We were hoping to have a common tool across. It makes it easy for distros. Probably, we can have a separate discussion on this.
The main goal of this test suite is to validate values reported by resctrl features i.e. we need _some_way_ to check if values reported by resctrl features are correct or not.
For (Intel) Memory Bandwidth features like MBA and MBM, perf iMC (Integrated Memory Controller) is used for validation. As a part of test, we run a benchmark (eg: fill_buf) and get memory bandwidth values from MBM and iMC and verify if the difference between both the reported values are in a reasonable range.
To make this common across x86, could you please let us know if similar things (i.e. perf iMC) exist on AMD? If so, you could add it in. The same applies for ARM.
Regards,
Sai
Moger, Babu
reviewerQuoted 31 lines · Sai Praneeth
Quoted 11 lines
Quoted 10 lines
No, the selftest in this patch set will not replace intel-cmt-cat or vice versa.
The selftest in this patch set has a different purpose from intel-cmt-cat:
the selftest is a test tool which validates resctrl functionalities while intel-cmt-cat is mainly a utility that provides base library for higher level applications including performance analysis tools, benchmark measurement tools, and potential resctrl tests. For example, running MBA test in the selftests tells MBA working or not working (fail/pass) right way. TheQuoted 1 line
Ok. Sure. Let me take a look at selftest closely. Will send my feedback soon.
Great!
Quoted 11 lines
Quoted 7 lines
intel-cmt-cat doesn't have this testing capability unless we extend the tool.
And intel-cmt-cat is maintained and developed by Intel. I don't think it's easy to extend it to AMD and ARM features. The selftest will be maintained
We1l.. We were hoping to have a common tool across. It makes it easy for distros. Probably, we can have a separate discussion on this.
The main goal of this test suite is to validate values reported by resctrl features i.e. we need _some_way_ to check if values reported by resctrl features are correct or not.
Quoted 7 lines
For (Intel) Memory Bandwidth features like MBA and MBM, perf iMC (Integrated Memory Controller) is used for validation. As a part of test, we run a benchmark (eg:
fill_buf) and get memory bandwidth values from MBM and iMC and verify if the difference between both the reported values are in a reasonable range.To make this common across x86, could you please let us know if similar things (i.e. perf iMC) exist on AMD? If so, you could add it in. The same applies for ARM.
We dont have iMC. You can go ahead with what you have right now. We can add those details later if required.
Quoted 4 lines
Regards,
Sai
Quoted 4 lines
Quoted 4 lines
For (Intel) Memory Bandwidth features like MBA and MBM, perf iMC (Integrated Memory Controller) is used for validation. As a part of test, we run a benchmark (eg:
fill_buf) and get memory bandwidth values from MBM and iMC and verify if the difference between both the
Quoted 8 lines
reported values are in a reasonable range.
Quoted 3 lines
To make this common across x86, could you please let us know if similar things (i.e. perf iMC) exist on AMD? If so, you could add it in. The same
applies for ARM.
We dont have iMC. You can go ahead with what you have right now. We can add those details later if required.
Sure! Makes sense to me.
Regards,
Sai
Moger, Babu
reviewerHi Fenghua,
My few comments.
Quoted 27 lines · Babu
Quoted 15 lines · Fenghua Yu
Quoted 11 lines
From: Moger, Babu [mailto:[email protected]]
Quoted 3 lines · Fenghua Yu
With more and more resctrl features are being added by Intel, AMD and ARM, a test tool is becoming more and more useful to validate that both hardware and software functionalities work as expected.
I like the initiative here. It is always good to have a single code base.
One question. I see that there is a tool at https://github.com/intel/intel-cmt-cat to test and verify the functionality of resctrl feature. I also see some of the distros have this tool already. Is this tool going to replace intel-cmt-cat? I have not looked at the
patches closely yet.No, the selftest in this patch set will not replace intel-cmt-cat or vice versa.
Quoted 6 lines
The selftest in this patch set has a different purpose from intel-cmt-cat:
the selftest is a test tool which validates resctrl functionalities while intel-cmt-cat is mainly a utility that provides base library for higher level applications including performance analysis tools, benchmark measurement tools, and potential resctrl tests. For example, running MBA test in the selftests tells MBA working or not working (fail/pass) right way. TheOk. Sure. Let me take a look at selftest closely. Will send my feedback soon.
Quoted 14 lines
Quoted 5 lines
intel-cmt-cat doesn't have this testing capability unless we extend the tool.
And intel-cmt-cat is maintained and developed by Intel. I don't think it's easy to extend it to AMD and ARM features. The selftest will be maintained
We1l.. We were hoping to have a common tool across. It makes it easy for distros. Probably, we can have a separate discussion on this.
Quoted 5 lines
and developed by the community and will hopefully cover all architectures.
We have seen a few issues recently in resctrl and may see more issues while expending the features. A convevient selftest may be useful to help identify and fix those potential issues.
I don't know the rules for selftest. Here are my general comments.
1. File names are not consistent.
# ls *.c fill_buf.c mba.c mbm.c resctrl.c resctrl_membw.c resctrl_tests.c Few files start with resctrl_ prefix and others don't.
2. Do we need README(or USAGE) here? I had too
3. I saw lots of these errors.
"mba.c:111:2: error: ‘for’ loop initial declarations are only allowed
in C99 mode"
for (int i = 0; i < 10; i++) {
^
I had to change it to
int i;
for (i = 0; i < 10; i++) {
Quoted 6 lines
Quoted 6 lines
Thanks.
-Fenghua
Moger, Babu
reviewerQuoted 46 lines · Babu
Hi Fenghua,
My few comments.Quoted 27 lines · Babu
Quoted 15 lines · Fenghua Yu
Quoted 11 lines
From: Moger, Babu [mailto:[email protected]]
Quoted 3 lines · Fenghua Yu
With more and more resctrl features are being added by Intel, AMD and ARM, a test tool is becoming more and more useful to validate that both hardware and software functionalities work as expected.
I like the initiative here. It is always good to have a single code base.
One question. I see that there is a tool at https://github.com/intel/intel-cmt-cat to test and verify the functionality of resctrl feature. I also see some of the distros have this tool already. Is this tool going to replace intel-cmt-cat? I have not looked at the
patches closely yet.No, the selftest in this patch set will not replace intel-cmt-cat or vice versa.
Quoted 6 lines
The selftest in this patch set has a different purpose from intel-cmt-cat:
the selftest is a test tool which validates resctrl functionalities while intel-cmt-cat is mainly a utility that provides base library for higher level applications including performance analysis tools, benchmark measurement tools, and potential resctrl tests. For example, running MBA test in the selftests tells MBA working or not working (fail/pass) right way. TheOk. Sure. Let me take a look at selftest closely. Will send my feedback soon.
Quoted 14 lines
Quoted 5 lines
intel-cmt-cat doesn't have this testing capability unless we extend the tool.
And intel-cmt-cat is maintained and developed by Intel. I don't think it's easy to extend it to AMD and ARM features. The selftest will be maintained
We1l.. We were hoping to have a common tool across. It makes it easy for distros. Probably, we can have a separate discussion on this.
Quoted 5 lines
and developed by the community and will hopefully cover all architectures.
We have seen a few issues recently in resctrl and may see more issues while expending the features. A convevient selftest may be useful to help identify and fix those potential issues.
Quoted 9 lines
I don't know the rules for selftest. Here are my general comments.
1. File names are not consistent.
# ls *.c fill_buf.c mba.c mbm.c resctrl.c resctrl_membw.c resctrl_tests.c Few files start with resctrl_ prefix and others don't.2. Do we need README(or USAGE) here? I had too
Correction..
2. Do we need README(or USAGE) here? I had too look at the code to check on "usage options" to run this utility.
Quoted 18 lines
3. I saw lots of these errors.
"mba.c:111:2: error: ‘for’ loop initial declarations are only allowed
in C99 mode"
for (int i = 0; i < 10; i++) {
^I had to change it to
int i; for (i = 0; i < 10; i++) {Quoted 6 lines
Quoted 6 lines
Thanks.
-Fenghua
A message that is not in the archive (sent off-list or not yet received): CALBSrqBXdXfNdZ1yJ_-zUSkk_2m_8R-VHNqk8DH3HmxCti4hhA@mail.gmail.com
Quoted 34 lines · Babu
Hi Fenghua,
My few comments.On 10/17/2018 09:40 AM, Moger, Babu wrote:
> On 10/16/2018 03:32 PM, Fenghua Yu wrote:
>>> From: Moger, Babu [mailto:[email protected]]
>>> On 10/16/2018 11:56 AM, Fenghua Yu wrote:
>>>> With more and more resctrl features are being added by Intel, AMD and >>>> ARM, a test tool is becoming more and more useful to validate that >>>> both hardware and software functionalities work as expected. >>>
>>> I like the initiative here. It is always good to have a single code base.
>>>
>>> One question. I see that there is a tool at https://github.com/intel/
intel-cmt-cat to test and verify the functionality of resctrl feature. I also see some of the distros have this tool already.
>>> Is this tool going to replace intel-cmt-cat? I have not looked at the >>> patches closely yet.
>>
>> No, the selftest in this patch set will not replace intel-cmt-cat or >> vice versa.
>>
>> The selftest in this patch set has a different purpose from intel-cmt-cat:
>> the selftest is a test tool which validates resctrl functionalities while
>> intel-cmt-cat is mainly a utility that provides base library for higher >> level applications including performance analysis tools, benchmark measurement
>> tools, and potential resctrl tests. For example, running MBA test in the >> selftests tells MBA working or not working (fail/pass) right way. The >
Quoted 2 lines
> Ok. Sure. Let me take a look at selftest closely. Will send my feedback soon.
Thanks.
Quoted 20 lines
Quoted 5 lines
Quoted 5 lines
intel-cmt-cat doesn't have this testing capability unless we extend the tool.
And intel-cmt-cat is maintained and developed by Intel. I don't think
it's
>> easy to extend it to AMD and ARM features. The selftest will be maintained
>
> We1l.. We were hoping to have a common tool across. It makes it easy for > distros. Probably, we can have a separate discussion on this. >
>> and developed by the community and will hopefully cover all architectures.
>>
>> We have seen a few issues recently in resctrl and may see more issues >> while expending the features. A convevient selftest may be useful to help
>> identify and fix those potential issues.
Quoted 6 lines
I don't know the rules for selftest. Here are my general comments.
1. File names are not consistent.
# ls *.c
fill_buf.cmba.cmbm.cresctrl.cresctrl_membw.cresctrl_tests.c
Few files start withresctrl_prefix and others don't.
All files are in resctrl directory. It's redundant to have resctrl prefix for each file. Probably we will remove the resctrl prefix for all files to keep uniform naming style.
Quoted 2 lines
2. Do we need README(or USAGE) here? I had too
We can add that in next version.
Quoted 10 lines
3. I saw lots of these errors.
"mba.c:111:2: error: ‘for’ loop initial declarations are only allowed
in C99 mode"
for (int i = 0; i < 10; i++) {
^I had to change it to
int i;
for (i = 0; i < 10; i++) {
Will fix the issues.
Thanks.
-Fenghua