ibacm fixes/updates

Hi Sean,

We were asked to integrate this into our product for scalability purposes. While working on that, I ran across a number of issues that needed fixed up (typical for a new package). So, here's a number of patches and fix ups for those things.

First, the package is ibacm but the binary is ib_acm. Trust me, it's best to pick one of the other and stick with it. Especially since you used ibacm for the log and pid files too. In my build, I went ahead and changed the binary name to match the package and log and pid file names.
I patched the Makefile.am and reran the autoconf tools to make this
happen. I've attached that patch.

In the package spec file don't differentiate between %{ver} and %{version}, these should always be the same and you should never need to use anything other than %{version}. Your spec file had a srv component, that I suspect was intended to hold the init script, but no init script was ever written and the section was incomplete (so your spec file wouldn't even build). I rewrote the spec file so that the base component is the service. In truth, for this you really want it to be a service and on by default if installed. I've attached my spec file. I've also attached the init script I made.

By default ibacm expects to find its configuration files in /etc/ibacm.
This adds to the proliferation of directories in /etc/ needlessly. We
already have a number of RDMA related directories to choose from depending on your install (OFED == /etc/ofed or /etc/openib in the old days, RHEL5 == /etc/openib from the old days, RHEL6 and Fedora are /etc/rdma, don't know what SuSE uses). Would be best if these files were in the same place as the other RDMA related files. And to make that happen easily, it would be best if they picked up on %{sysconfdir} from the ./configure invocation in order to set the directory. I used a static patch for now because I'm behind on my work, so updating the configure script was more than I had time to do. I'm not bothering to include my patch as it needs done the other way so my patch is purely a stopgap that should not see the time of day ;-)

OK, gotta run. Would like to see this stuff picked up, especially the init stuff. Getting it in your release before we have one init script, SuSE another, and OFED another will help keep things uniform.

Doug Ledford <[email protected]>
              GPG KeyID: 0E572FDD
	      http://people.redhat.com/dledford

#!/bin/bash
#
# Bring up/down the ibacm daemon
#
# chkconfig: 2345 25 75
# description: Starts/Stops InfiniBand ACM service
#
### BEGIN INIT INFO
# Provides: ibacm
# Default-Start: 2 3 4 5
# Default-Stop: 0 1 6
# Required-Start: rdma $network
# Required-Stop: rdma $network
# Should-Start: opensm
# Should-Stop: opensm
# Short-Description: Starts and stops the InfiniBand ACM service # Description: The InfiniBand ACM service provides a user space implementation
# of someting resembling an ARP cache for InfiniBand SA queries and
# host route lookups.
### END INIT INFO

pidfile=/var/run/ibacm.pid
subsys=/var/lock/subsys/ibacm
prog=/usr/sbin/ibacm

. /etc/rc.d/init.d/functions

start()

{
    echo -n "Starting ibacm daemon:"

    daemon $prog
    RC=$?
    [ $RC -eq 0 ] && touch $subsys
    echo
    return $RC
}

stop()
{
echo -n "Stopping ibacm daemon:"

killproc -p $pidfile $prog
RC=$?
rm -f $subsys
echo
return $RC
}

status()
{
if [ ! -f $subsys -a ! -f $pidfile ]; then
return 3
fi
if [ -f $pidfile ]; then
checkpid cat $pidfile
return $?
fi
if [ -f $subsys ]; then
return 2
fi
}

restart ()
{
stop
start
}

condrestart ()

{
    [ -e $subsys ] && restart || return 0
}

usage ()
{
echo
echo "Usage: basename $0 {start|stop|restart|condrestart|try-restart|force-reload|status}" echo
return 2
}

case $1 in

    start|stop|restart|condrestart|try-restart|force-reload)
	[ `id -u` != "0" ] && exit 4 ;;

esac

case $1 in

    start) start; RC=$? ;;
    stop) stop; RC=$? ;;
    restart) restart; RC=$? ;;
    reload) RC=3 ;;
    condrestart) condrestart; RC=$? ;;
    try-restart) condrestart; RC=$? ;;
    force-reload) condrestart; RC=$? ;;
    status) status; RC=$? ;;
    *) usage; RC=$? ;;

esac

exit $RC

Name: ibacm
Version: 1.0.5
Release: 1%{?dist}
Summary: InfiniBand Communication Manager Assistant
Group: System Environment/Daemons
License: GPLv2 or BSD
Url: http://www.openfabrics.org/
Source: http://www.openfabrics.org/downloads/rdmacm/%{name}-%{version}.tar.gz
Source1: ibacm.init
Patch0: ibacm-1.0.5-make.patch
Patch1: ibacm-1.0.5-conf.patch
BuildRoot: %{_tmppath}/%{name}-%{version}-%{release}-root-%(%{__id_u} -n)
BuildRequires: libibverbs-devel >= 1.1-1, autoconf, libtool, libibumad-devel Requires(post): chkconfig
Requires(preun): chkconfig
ExcludeArch: s390 s390x

%description
The ib_acm daemon helps reduce the load of managing path record lookups on large InfiniBand fabrics by providing a user space implementation of what is functionally similar to an ARP cache. The use of ib_acm, when properly configured, can reduce the SA packet load of a large IB cluster from O(n^2) to O(n). The ib_acm daemon is started and normally runs in the background, user applications need not know about this daemon as long as their app uses librdmacm to handle connection bring up/tear down. The librdmacm library knows how to talk directly to the ib_acm daemon to retrieve data.

%package devel
Summary: Headers file needed when building apps to talk directly to ib_acm
Requires: %{name} = %{version}-%{release}

%description devel
Most applications do not need to know how to talk directly to the ib_acm daemon, but it does have a socket that it listens on, and it has a specific protocol for incoming/outgoing data. So if you wish to build the ability to communicate directly with ib_acm into your own application, the protocol used to communicate with it, and the data structures involved, are in this header file. Please note that this is an unsupported method of using this daemon. The only supported means of using this is via librdmacm. As such, even though this header file is provided, no further documentation is available. One must read the source if they wish to make use of this header file.

%prep
%setup -q -n %{name}-%{version}
%patch0 -p1 -b .make
%patch1 -p1 -b .conf

%build

aclocal -I config && libtoolize --force --copy && autoheader && \
	automake --foreign --add-missing --copy && autoconf

%configure --sysconfdir=/etc/rdma CFLAGS="$CXXFLAGS -fno-strict-aliasing" LDFLAGS=-lpthread mv man/ib_acm.1 man/ibacm.1
mv man/ib_acm.7 man/ibacm.7
make %{?_smp_mflags}

%install
rm -rf $RPM_BUILD_ROOT
make DESTDIR=%{buildroot} install
install -D -m 755 %{SOURCE1} %{buildroot}%{_initrddir}/ibacm

%clean
rm -rf $RPM_BUILD_ROOT

%post
if [ $1 = 1 ]; then

	chkconfig --add ibacm

fi

%preun
if [ $1 = 0 ]; then

	chkconfig --del ibacm

fi

%files
%defattr(-,root,root,-)
%doc AUTHORS COPYING README
%{_bindir}/ib_acme
%{_sbindir}/ibacm
%{_mandir}/man1/*
%{_mandir}/man7/*
%config(noreplace) %{_sysconfdir}/rdma/*
%{_initrddir}/ibacm

%files devel
%defattr(-,root,root,-)
%{_includedir}/infiniband/acm.h

%changelog
* Tue Feb 28 2012 Doug Ledford <[email protected]> - 1.0.5-1
- Ininital version for rhel6
- Related: bz700285

diff -up ibacm-1.0.5/Makefile.am.dist ibacm-1.0.5/Makefile.am
@@ -3,10 +3,10 @@ INCLUDES = -I$(srcdir)/include -I$(srcdi
 AM_CFLAGS = -g -Wall -D_GNU_SOURCE
 
 bin_PROGRAMS = util/ib_acme
-sbin_PROGRAMS = svc/ib_acm
-svc_ib_acm_SOURCES = src/acm.c
+sbin_PROGRAMS = svc/ibacm
+svc_ibacm_SOURCES = src/acm.c
 util_ib_acme_SOURCES = src/acme.c linux/acme_linux.c src/libacm.c linux/libacm_linux.c src/parse.c
-svc_ib_acm_CFLAGS = $(AM_CFLAGS)
+svc_ibacm_CFLAGS = $(AM_CFLAGS)
 util_ib_acme_CFLAGS = $(AM_CFLAGS)
 
 ibacmincludedir = $(includedir)/infiniband

@@ -15,12 +15,19 @@ ibacminclude_HEADERS = include/infiniban
 
 man_MANS = \
 	man/ib_acme.1 \
-	man/ib_acm.1 \
-	man/ib_acm.7
+	man/ibacm.1 \
+	man/ibacm.7
 
 EXTRA_DIST = src/acm_mad.h src/libacm.h \
 	     linux/osd.h linux/dlist.h ibacm.spec.in $(man_MANS) acm_opts.cfg \
 	     acm_addr.cfg
 
+install-exec-hook:
+	if ! test -d $(DESTDIR)$(sysconfdir); then \
+		mkdir -p $(DESTDIR)$(sysconfdir); \
+	fi; \
+	install -m 644 acm_opts.cfg $(DESTDIR)$(sysconfdir)/acm_opts.cfg; \
+	install -m 644 acm_addr.cfg $(DESTDIR)$(sysconfdir)/acm_addr.cfg; 
+
 dist-hook: ibacm.spec
 	cp ibacm.spec $(distdir)

Oh, and another issue, the pid and lock files are not in the LSB standard locations:

Pid file: /var/run/
Lock file: /var/lock/subsys/ (this is created by the init script, and it is doing the right thing here)

Right now the program creates the pid file in /var/lock (or /var/run/lock, don't know which and they are linked so it shows up both places), but not in /var/run which is the standard location for pid files. The port file can stay where it is.

Doug Ledford <[email protected]>
              GPG KeyID: 0E572FDD
	      http://people.redhat.com/dledford

Thanks - I'll try to add these patches to the next release and work to get things fixed up.

- Sean
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

I've moved the config files to /etc/rdma. I'm still trying to figure out how to make them use %{sysconfdir}, so I can avoid hard coding any paths.

Note that the config files are NOT needed by default. If the option config file (acm_opts.cfg) is not found, the ibacm service will simply use the default settings. The default values are the same as defined in the sample option file.

The acm_addr.cfg file included with the package is a sample file only. The address file should be generated by running the ib_acme program on the same node. Copying the acm_addr.cfg file included with the package will not work, since the addresses must be unique per node. If no address config file is found, the ibacm service will automatically call ib_acme to generate one.

So... I think you want to remove the following install lines from your makefile.am patch.

+install-exec-hook:
+ if ! test -d $(DESTDIR)$(sysconfdir); then \
+ mkdir -p $(DESTDIR)$(sysconfdir); \
+ fi; \
+ install -m 644 acm_opts.cfg $(DESTDIR)$(sysconfdir)/acm_opts.cfg; \
+ install -m 644 acm_addr.cfg $(DESTDIR)$(sysconfdir)/acm_addr.cfg;

- Sean
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html