Add btrecord/btreplay capability

From d5095b9054fb40e33a76bc790e6ce459c9a0ee91 Mon Sep 17 00:00:00 2001
From: Alan D. Brunelle <[email protected]>
Date: Tue, 2 Oct 2007 12:35:07 -0400
Subject: [PATCH] Add btrecord/btreplay capability

These facilities allow one to attempt to replay a stream of IOs captured with blktrace. The general workflow is:

1. Initiate blktrace to capture traces
2. Do whatever to generate initial IO stream...
3. Stop blktrace
4. Run btrecord to convert traces into IO records
5. Run btreplay to replay IOs

The IO stream characteristics during replay will try to respect the following characteristics of the original IO stream:

1. The IOs will target the same device(s) as originally seen. [One can alter this behavior by specifyin the -M option to btreplay, which allows one to remap IOs slated to one set of devices to a specified other set of devices.]

2. IO direction: the IOs will follow the same read/write (from-device/to-device) characteristics of the originating flow. [Note:
By default replay will /not/ do writes, one must specify the -W option to do this. THis is a meager attempt to stop someone from shooting themselves in the foot (with a very large-caliber weapon).]

3. IO offset & size are maintained.

4. CPU: IOs are submitted on the originating CPU whenever possible. [Note:
Since we are using asynchronous IO, IOs may be routed to another CPU prior to being processed by the block IO layer.]

In order to try and replicate inter-IO timing as much as possible, btrecord will combine IOs "close in time" into one set, or bunch, of IOs. Then btreplay will replay all the IOs in one go (via asynchronous direct IO - io_submit). The size of the bunches are configurable via the -m flag to btrecord (which specifies the a time-based bunch size) and/or the -M flag (which specifies the maximum amount of IOs to put into a bunch). At the low-end, specifying '-M 1' instructs btrecord to act like fio - replay each IO as an individual unit.

Besides the potential to remap devices (utilizing the -M option to replay, as noted above), one can also limit the number of CPUs on the replay machine - so if you have fewer CPUs on the replay machine you specify the -c option to btreplay.

Lastly, one can specify the -N option to btreplay to instruct it to ignore inter-IO (inter-bunch of IOs) timings. Thus, this instructs btreplay to replay the bunches as fast as possible, ignoring the original delays between original IOs.

The utilities include a write-up in the docs directory.

---
 Makefile                  |   11 +-
 btreplay/Makefile         |   45 ++
 btreplay/btrecord.c       |  780 ++++++++++++++++++++++
 btreplay/btrecord.h       |   95 +++
 btreplay/btreplay.c       | 1590 +++++++++++++++++++++++++++++++++++++++++++++
 btreplay/doc/Makefile     |   18 +
 btreplay/doc/abstract.tex |   34 +
 btreplay/doc/btreplay.tex |  521 +++++++++++++++
 8 files changed, 3093 insertions(+), 1 deletions(-)
 create mode 100644 btreplay/Makefile
 create mode 100644 btreplay/btrecord.c
 create mode 100644 btreplay/btrecord.h
 create mode 100644 btreplay/btreplay.c
 create mode 100644 btreplay/doc/Makefile
 create mode 100644 btreplay/doc/abstract.tex
 create mode 100644 btreplay/doc/btreplay.tex

index e36f5a2..fda76f4 100644
@@ -5,13 +5,19 @@ PROGS	= blkparse blktrace verify_blkparse blkrawverify
 LIBS	= -lpthread
 SCRIPTS	= btrace
 
-ALL = $(PROGS) $(SCRIPTS) btt/btt
+ALL = $(PROGS) $(SCRIPTS) btt/btt btreplay/btrecord btreplay/btreplay
 
 all: $(ALL)
 
 btt/btt:
 	$(MAKE) -C btt
 
+btreplay/btrecord:
+	$(MAKE) -C btreplay
+
+btreplay/btreplay:
+	$(MAKE) -C btreplay
+
 %.o: %.c
 	$(CC) -o $*.o -c $(ALL_CFLAGS) $<
 

@@ -32,10 +38,12 @@ $(PROGS): | depend
 docs:
 	$(MAKE) -C doc all
 	$(MAKE) -C btt docs
+	$(MAKE) -C btreplay docs
 
 docsclean:
 	$(MAKE) -C doc clean
 	$(MAKE) -C btt clean
+	$(MAKE) -C btreplay clean
 
 depend:
 	@$(CC) -MM $(ALL_CFLAGS) *.c 1> .depend

@@ -63,6 +71,7 @@ rpm: dist
 clean: docsclean
 	-rm -f *.o $(PROGS) .depend btrace-1.0.tar.bz2
 	$(MAKE) -C btt clean
+	$(MAKE) -C btreplay clean
 
 install: all
 	$(INSTALL) -m 755 -d $(DESTDIR)$(bindir)

new file mode 100644
index 0000000..e3b383e
@@ -0,0 +1,18 @@
+DOCTMP  = btreplay.log btreplay.aux btreplay.dvi btreplay.toc
+
+all: btreplay.dvi btreplay.pdf
+
+btreplay.tex:
+	@touch btreplay.tex
+
+btreplay.dvi: btreplay.tex abstract.tex
+	@latex btreplay.tex
+	@latex btreplay.tex
+
+btreplay.pdf: btreplay.dvi
+	@dvipdfm -p letter btreplay
+
+clean:
+	-rm -f $(DOCTMP)
+	-rm -f *.bak *.ps *.pdf
+	@rm -rf btreplay

new file mode 100644
index 0000000..314d820
@@ -0,0 +1,34 @@
+%
+% Copyright (C) 2007 Alan D. Brunelle <[email protected]>
+%
+%  This program is free software; you can redistribute it and/or modify
+%  it under the terms of the GNU General Public License as published by
+%  the Free Software Foundation; either version 2 of the License, or
+%  (at your option) any later version.
+%
+%  This program is distributed in the hope that it will be useful,
+%  but WITHOUT ANY WARRANTY; without even the implied warranty of
+%  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+%  GNU General Public License for more details.
+%
+%  You should have received a copy of the GNU General Public License
+%  along with this program; if not, write to the Free Software
+%  Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA  02111-1307  USA
+%
+%  vi :set textwidth=75
+%
+The \texttt{btrecord} and \texttt{btreplay} tools provide the ability to
+record and replay IOs captured by the \texttt{blktrace} utility. Attempts
+are made to maintain ordering, CPU mappings and time-separation of IOs. The
+general workflow is expected to be:
+
+\begin{enumerate}
+  \item Initiate \texttt{blktrace} to capture traces
+  \item Generate traces\ldots
+  \item Stop \texttt{blktrace}
+  \item Run \texttt{btrecord} to convert traces into IO records 
+  \item Utilize \texttt{btreplay} to replay IOs 
+\end{enumerate}
+
+This document will discuss the operating characteristics of
+\texttt{btreplay} and provide detailed command line option descriptions.

1.5.2.5

Part of the diff is not shown here. View the raw message.

This looks pretty nifty and useful. I've applied both your patches. They do have a few build warnings here, involving fatal():

btrecord.c: In function 'stream_open':
btrecord.c:685: warning: the address of 'ofile_name' will always evaluate as 'true'
btrecord.c:704: warning: the address of 'vfile_name' will always evaluate as 'true'

btreplay.c: In function 'tip_init':
btreplay.c:783: warning: the address of 'fn' will always evaluate as 'true'
btreplay.c: In function 'replay_sub':
btreplay.c:1314: warning: the address of 'path' will always evaluate as 'true'

Jens Axboe

Hi Jens -

Thanks for taking these in - I have no idea why I'm not seeing that here, it builds clean on ia64 (SLES10/gcc=4.1.0) and x86_64 (Ubuntu/gcc=4.1.2).

I see why the warnings come out, and will think of a way to clean that up.

Alan

Probably gcc-4.2.1 being more chatty, I see it on i386 and x86_64 here.

Jens Axboe