rdma-core and endianness checking with sparse

Hello Jason,

Since commit https://github.com/linux-rdma/rdma-core/commit/dceee9494bfbadf0cf16b8fef81f3c2cb5355243
checking endianness locally with sparse is broken on my development setup.
The following error messages appear:

-- Performing Test HAVE_NO_SPARSE
-- Performing Test HAVE_NO_SPARSE - Failed
Patch from '/home/bart/software/infiniband/rdma-core/buildlib/sparse-include/19/netinet-in.h.diff' failed Patch from '/home/bart/software/infiniband/rdma-core/buildlib/sparse-include/23/netinet-in.h.diff' failed Traceback (most recent call last):
  File "/home/bart/software/infiniband/rdma-core/buildlib/gen-sparse.py", line 129, in <module> replace_header(I);
  File "/home/bart/software/infiniband/rdma-core/buildlib/gen-sparse.py", line 82, in replace_header raise ValueError("Unable to apply any patch to %r, tries %u"%(fn,tries));
ValueError: Unable to apply any patch to 'netinet/in.h', tries 2

I was surprised to see that merge request in the rdma-core repository although I had warned from beforehand that this could happen?

Bart.--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

I am surprised, I did check that tumbleweed worked for you on tumbleweed..

Ah, but I see glibc 25 is out :(

As you say, it is not unexpected with this approach.

I would like to take small steps here. This is enough to make sparse work in travis, and it finds bugs, so we are overall better off.

Let us find some way to make it work on more reliably on development systems. But, I am not really that optimistic :(

I added some revisions here which use include_next where it is easy:

https://github.com/jgunthorpe/rdma-plumbing/commits/sparse

But still patches the harder things. I also fixed things up for glibc 25, so it should make it work for you on tumbleweed

I continue to think a 'cbuild make sparse' is the way to go for desktop use..

Jason

From: Jason Gunthorpe <jgunthorpe-ePGOBjL8dl3ta4EC/59zMFaTQe2KTcn/@public.gmane.org>
Date: Tue, 21 Mar 2017 12:14:57 -0600
Subject: [PATCH] sparse: Improve the glibc header file mangling

- Use include_next wrappers for the very simple cases, hopefully these will be more robust against different glibc versions
- Check patching works on tumbleweed, C7, trusty and xenial.
- Check for patching failure from cmake and provide some guidance

---
 buildlib/RDMA_Sparse.cmake                   |   7 +-
 buildlib/gen-sparse.py                       |  11 ++-
 buildlib/sparse-include/23/endian.h.diff     |  57 -------------
 buildlib/sparse-include/23/pthread.h.diff    |  11 ---
 buildlib/sparse-include/25/netinet-in.h.diff | 121 +++++++++++++++++++++++++++
 buildlib/sparse-include/endian.h             |  43 ++++++++++
 buildlib/sparse-include/pthread.h            |  12 +++
 7 files changed, 191 insertions(+), 71 deletions(-)
 delete mode 100644 buildlib/sparse-include/23/endian.h.diff
 delete mode 100644 buildlib/sparse-include/23/pthread.h.diff
 create mode 100644 buildlib/sparse-include/25/netinet-in.h.diff
 create mode 100644 buildlib/sparse-include/endian.h
 create mode 100644 buildlib/sparse-include/pthread.h

index 0f42de6937e56e..e6bd41e22a7daa 100644
@@ -20,7 +20,12 @@ int main(int argc,const char *argv[]) {return 0;}
     # Replace various glibc headers with our own versions that have embedded sparse annotations.
     execute_process(COMMAND "${BUILDLIB}/gen-sparse.py"
       "--out" "${BUILD_INCLUDE}/"
-      "--src" "${CMAKE_SOURCE_DIR}/")
+      "--src" "${CMAKE_SOURCE_DIR}/"
+      RESULT_VARIABLE retcode)
+    if(NOT "${retcode}" STREQUAL "0")
+      message(FATAL_ERROR "glibc header file patching for sparse failed. Review include/*.rej and fix the rejects, then do "
+	"${BUILDLIB}/gen-sparse.py -out ${BUILD_INCLUDE}/ --src ${CMAKE_SOURCE_DIR}/ --save")
+    endif()
   endif()
 
   # Enable endian analysis in sparse

index 2116da0788d012..27c773ebcdd429 100755
@@ -35,7 +35,7 @@ def get_buildlib_patches(dfn):
                 bn = 0;
 
             res.append((bn,os.path.join(d,I)));
-    res.sort();
+    res.sort(reverse=True);
 
     ret = collections.defaultdict(list);
     for _,I in res:

@@ -79,7 +79,10 @@ def replace_header(fn):
                        pfn,os.path.join(args.INCLUDE,fn)):
             return;
         tries = tries + 1;
-    raise ValueError("Unable to apply any patch to %r, tries %u"%(fn,tries));
+
+    print "Unable to apply any patch to %r, tries %u"%(fn,tries);
+    global failed;
+    failed = True;
 
 def save(fn,outdir):
     """Diff the header file in our include directory against the system header and

@@ -124,6 +127,10 @@ if args.save:
     for I in headers:
         save(I,outdir);
 else:
+    failed = False;
     patches = get_buildlib_patches(os.path.join(args.SRC,"buildlib","sparse-include"));
     for I in headers:
         replace_header(I);
+
+    if failed:
+        raise ValueError("Patch applications failed");

deleted file mode 100644
index 6ff2985840cb8e..00000000000000
@@ -1,11 +0,0 @@
---- /usr/include/pthread.h	2016-11-16 15:43:59.000000000 -0700
-+++ build-sparse/include/pthread.h	2017-03-15 12:43:28.736376893 -0600
-@@ -84,7 +84,7 @@
- 
- #ifdef __PTHREAD_MUTEX_HAVE_PREV
- # define PTHREAD_MUTEX_INITIALIZER \
--  { { 0, 0, 0, 0, 0, __PTHREAD_SPINS, { 0, 0 } } }
-+  { { 0, 0, 0, 0, 0, __PTHREAD_SPINS, { } } }
- # ifdef __USE_GNU
- #  define PTHREAD_RECURSIVE_MUTEX_INITIALIZER_NP \
-   { { 0, 0, 0, 0, PTHREAD_MUTEX_RECURSIVE_NP, __PTHREAD_SPINS, { 0, 0 } } }

new file mode 100644
index 00000000000000..cf9de0c6064d1d
@@ -0,0 +1,12 @@
+/* COPYRIGHT (c) 2017 Obsidian Research Corporation. See COPYING file */
+
+#ifndef _SPARSE_PTHREAD_H_
+#define _SPARSE_PTHREAD_H_
+
+#include_next <pthread.h>
+
+/* Sparse complains that the glibc version of this has 0 instead of NULL */
+#undef PTHREAD_MUTEX_INITIALIZER
+#define PTHREAD_MUTEX_INITIALIZER {}
+
+#endif

2.7.4

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html