bb-matrix performance metrics gathering and visualization scripts
cover bb-matrix performance metrics gathering and visualization scripts
Add bb-matrix.sh and bb-matrix-plot.sh. Example output of these scripts is viewable here:
https://wiki.yoctoproject.org/wiki/Build_Performance#bb-matrix
The following changes since commit 9f4eaeef33da5595748253d59d95c7ca548e28fa:
libx11: enable xcb support (2011-07-08 23:02:08 +0100)
are available in the git repository at:
git://git.pokylinux.org/poky-contrib dvhart/bb-perf
http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=dvhart/bb-perf
Darren Hart (1):
bb-matrix: initial scripts to record TIME(1) metrics for BB and PM
combinations
scripts/contrib/bb-perf/bb-matrix-plot.sh | 137 +++++++++++++++++++++++++++++ scripts/contrib/bb-perf/bb-matrix.sh | 78 ++++++++++++++++ 2 files changed, 215 insertions(+), 0 deletions(-) create mode 100755 scripts/contrib/bb-perf/bb-matrix-plot.sh create mode 100755 scripts/contrib/bb-perf/bb-matrix.sh
PATCH 1/1 bb-matrix: initial scripts to record TIME(1) metrics for BB and PM combinations
The bb-matrix.sh script will run a bitbake command, building core-image-minimal by default, for various combinations of BB_NUMBER_THREADS and PARALLEL_MAKE. It records all relevant metrics of the TIME(1) command for each combination in a data file.
The bb-matrix-plot.sh script can be used to visualize each of these metrics via a 3d surface plot, either solid surface or wireframe with a value-map projection on the XY plane.
--- scripts/contrib/bb-perf/bb-matrix-plot.sh | 137 +++++++++++++++++++++++++++++ scripts/contrib/bb-perf/bb-matrix.sh | 78 ++++++++++++++++ 2 files changed, 215 insertions(+), 0 deletions(-) create mode 100755 scripts/contrib/bb-perf/bb-matrix-plot.sh create mode 100755 scripts/contrib/bb-perf/bb-matrix.sh
scripts/contrib/bb-perf/bb-matrix-plot.sh new file +137 -0
@@ -0,0 +1,137 @@ +#!/bin/bash +# +# Copyright (c) 2011, Intel Corporation. +# All rights reserved. +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program; if not, write to the Free Software +# Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. +# +# DESCRIPTION +# This script operates on the .dat file generated by bb-matrix.sh. It tolerates +# the header by skipping the first line, but error messages and bad data records +# need to be removed first. It will generate three views of the plot, and leave +# an interactive view open for further analysis. +# +# AUTHORS +# Darren Hart <[email protected]> +# + +# Setup the defaults +DATFILE="bb-matrix.dat" +XLABEL="BB_NUMBER_THREADS" +YLABEL="PARALLEL_MAKE" +FIELD=3 +DEF_TITLE="Elapsed Time (seconds)" +PM3D_FRAGMENT="unset surface; set pm3d at s hidden3d 100" +SIZE="640,480" + +function usage { +CMD=$(basename $0) +cat <<EOM +Usage: $CMD [-d datfile] [-f field] [-h] [-t title] [-w] + -d datfile The data file generated by bb-matrix.sh (default: $DATFILE) + -f field The field index to plot as the Z axis from the data file + (default: $FIELD, "$DEF_TITLE") + -h Display this help message + -s W,H PNG and window size in pixels (default: $SIZE) + -t title The title to display, should describe the field (-f) and units + (default: "$DEF_TITLE") + -w Render the plot as wireframe with a 2D colormap projected on the + XY plane rather than as the texture for the surface +EOM +} + +# Parse and validate arguments +while getopts "d:f:hs:t:w" OPT; do + case $OPT in + d) + DATFILE="$OPTARG" + ;; + f) + FIELD="$OPTARG" + ;; + h) + usage + exit 0 + ;; + s) + SIZE="$OPTARG" + ;; + t) + TITLE="$OPTARG" + ;; + w) + PM3D_FRAGMENT="set pm3d at b" + W="-w" + ;; + *) + usage + exit 1 + ;; + esac +done + +# Ensure the data file exists +if [ ! -f "$DATFILE" ]; then + echo "ERROR: $DATFILE does not exist" + usage + exit 1 +fi +PLOT_BASENAME=${DATFILE%.*}-f$FIELD$W + +# Set a sane title +# TODO: parse the header and define titles for each format parameter for TIME(1) +if [ -z "$TITLE" ]; then + if [ ! "$FIELD" == "3" ]; then + TITLE="Field $FIELD" + else + TITLE="$DEF_TITLE" + fi +fi + +# Determine the dgrid3d mesh dimensions size +MIN=$(tail -n +2 "$DATFILE" | cut -d ' ' -f 1 | sort | uniq | head -n1) +MAX=$(tail -n +2 "$DATFILE" | cut -d ' ' -f 1 | sort | uniq | tail -n1) +BB_CNT=$[${MAX#*0} - $MIN + 1] +MIN=$(tail -n +2 "$DATFILE" | cut -d ' ' -f 2 | sort | uniq | head -n1) +MAX=$(tail -n +2 "$DATFILE" | cut -d ' ' -f 2 | sort | uniq | tail -n1) +PM_CNT=$[${MAX#*0} - $MIN + 1] + + +(cat <<EOF +set title "$TITLE" +set xlabel "$XLABEL" +set ylabel "$YLABEL" +set style line 100 lt 5 lw 1.5 +$PM3D_FRAGMENT +set dgrid3d $PM_CNT,$BB_CNT +set ticslevel 0.2 + +set term png size $SIZE +set output "$PLOT_BASENAME.png" +splot "$DATFILE" every ::1 using 1:2:$FIELD with lines ls 100 + +set view 90,0 +set output "$PLOT_BASENAME-bb.png" +replot + +set view 90,90 +set output "$PLOT_BASENAME-pm.png" +replot + +set view 60,30 +set term wxt size $SIZE +replot +EOF +) | gnuplot --persist
scripts/contrib/bb-perf/bb-matrix.sh new file +78 -0
78 lines not on this page · raw message
Signature · 1 line
1.7.1
Quoted 2 lines · Darren Hart
bb-matrix: initial scripts to record TIME(1) metrics for BB and PM combinations
please drop initial scripts; it doesn't matter et all if it is a script and if it is merged it is because it is usable and worth having it so the initial status makes no useful contribution to the short description. Long description can make clear and more detailed explanation.
Signature · 3 lines
Otavio Salvador O.S. Systems E-mail: [email protected] http://www.ossystems.com.br Mobile: +55 53 9981-7854 http://projetos.ossystems.com.br
Richard Purdie
AppliedRe: [PATCH 1/1] bb-matrix: initial scripts to record TIME(1) metrics for BB and PM combinations
In reply to Darren HartQuoted 8 lines · Darren Hart
The
bb-matrix.shscript will run a bitbake command, building core-image-minimal by default, for various combinations ofBB_NUMBER_THREADSandPARALLEL_MAKE. It records all relevant metrics of the TIME(1) command for each combination in a data file.The
bb-matrix-plot.shscript can be used to visualize each of these metrics via a 3d surface plot, either solid surface or wireframe with a value-map projection on the XY plane.
Merged to master, thanks.
Richard