Skip to contentSkip to conversation

Add ndctl check-namespace

PATCH Superseded Vishal Verma 4 participants 12 messages Mar 4, 2017 – Apr 1, 2017

Subsystems (3)

v3 of 4 Superseded Latest is v4 →

lore
Share & tools
lore

Vishal Verma

author maintainer
#0 lore raw

v3 cover Add ndctl check-namespace

Changes in v3:
- Move the addition of ccan/bitmap to its own patch(es) (Dan)
- Drop the changelog update from the spec (Dan)
- Fix the [verse] section in the documentation text for check-namespace (Dan)
- Unify all namespace_disable paths to perform checking for a mounted filesystem (Dan)
- Change the logging to use util/log.h (Dan)
- Use BTT_START_OFFSET for the initial offset, and store it in bttc (Jeff, Dan)
- Fix a number of line > 80 chars (everything but strings) (Jeff)
- Fix short write error handling, add fsync (Jeff)
- Save system page size in bttc to avoid calling sysconf repeatedly (Jeff)
- In check_log_map(), loop through the entire log even in case of an error, and if there was a saved error, fail. (Jeff)
- btt-check.sh: in the post repair test, validate that the data read back is the same as what was written (Jeff)
- Stop playing games with pre-adding/subtracting the initial 4K offset (Jeff)
- btt_read_info doesn't need to use 'rc', return directly.

Changes in v2:
- Move checking functionality to a separate file (Dan, Jeff)
- Rename btt-structs.h to check.h (Dan)
- Don't provide a configure option for building the checker, always build it in. (Dan, Jeff)
- Fix the Documentation example to also include disable-namespace (Linda)
- Update the description text to note the namespace needs to be disabled before checking (Linda)
- Use util/size.h for sizes (Dan)
- Use --repair to do repairs instead of --dry-run to disable repairs (Dan)
- Fix btt_read_info short read error handling (Jeff)
- Simplify the map lookup/write routines (Jeff)
- Differentiate the use off BTT_PG_SIZE, sysconf(_SC_PAGESIZE), and SZ_4K (for the fixed start offset) in the different places they're used (Jeff)
- Add the missing msync when copying over info2 (Jeff)
- Add unit tests to test the checker (Jeff)
- Add a missing error case check in do_xaction_namespace for check
- Add a --force option that allows running on an active namespace (Jeff)
- Add a bitmap test for checking all internal blocks are referenced exactly once between the map and flog (Jeff)
- Remove unused #defines in check.h
- Add comments to explain what we do with raw_mode (Jeff)
- Add some sanity checking when parsing an arena's metadata (Jeff)
- Refactor some read-verify sequences into a helper that combines the two (Jeff)
- Additional bounds checking on the 'offset' in recover_first_sb attempt 3 (Jeff)
- Add a missing ACTION_DESTROY string in parse_namespace_options (Dan)
- Use uXX, and cpu_to_XX from ccan/endian (Dan)
- Move the fletcher64 Routing to util/ as it is shared by builtin-dimm.c (Dan)
- Open the raw block device only once with O_EXCL instead of every time on read/write/mmap (Dan)
- Add a new 'inform' routing in util/usage.c, and use it for some non-critical messages (Dan)
- Remove namespace_is_offline() from builtin-check.c. Instead, use util_namespace_active() from util/json.c
- Add a missing return value check after info block restoration in discover_arenas

Vishal Verma (7):
libndctl: add a ndctl_namespace_is_active helper
libndctl: add a ndctl_namespace_disable_safe() API
ccan: Add ccan/bitmap in preparation for the BTT checker ccan/bitmap: fix a set of gcc warnings (with -Wshadow)
ndctl: move the fletcher64 routine to util/
ndctl: add a BTT check utility
ndctl, test: Add a unit test for the BTT checker

 Documentation/Makefile.am               |   1 +
 Documentation/ndctl-check-namespace.txt |  64 +++
 Documentation/ndctl.txt                 |   1 +
 Makefile.am                             |   7 +-
 builtin.h                               |   1 +
 ccan/bitmap/LICENSE                     |   1 +
 ccan/bitmap/bitmap.c                    | 125 ++++
 ccan/bitmap/bitmap.h                    | 243 ++++++++
 contrib/ndctl                           |   3 +
 licenses/LGPL-2.1                       | 508 ++++++++++++++++
 ndctl.spec.in                           |   2 +-
 ndctl/Makefile.am                       |   1 +
 ndctl/builtin-check.c                   | 988 ++++++++++++++++++++++++++++++++
 ndctl/builtin-dimm.c                    |  18 +-
 ndctl/builtin-list.c                    |   2 +-
 ndctl/builtin-xaction-namespace.c       | 112 ++--
 ndctl/check.h                           | 127 ++++
 ndctl/lib/libndctl.c                    |  59 ++
 ndctl/lib/libndctl.sym                  |   2 +
 ndctl/libndctl.h.in                     |   3 +
 ndctl/ndctl.c                           |   1 +
 test/Makefile.am                        |   5 +-
 test/btt-check.sh                       | 170 ++++++
 util/fletcher.c                         |  23 +
 util/fletcher.h                         |   8 +
 util/json.c                             |  17 +-
 util/json.h                             |   1 -
 util/util.h                             |   8 +
 28 files changed, 2423 insertions(+), 78 deletions(-)
 create mode 100644 Documentation/ndctl-check-namespace.txt
 create mode 120000 ccan/bitmap/LICENSE
 create mode 100644 ccan/bitmap/bitmap.c
 create mode 100644 ccan/bitmap/bitmap.h
 create mode 100644 licenses/LGPL-2.1
 create mode 100644 ndctl/builtin-check.c
 create mode 100644 ndctl/check.h
 create mode 100755 test/btt-check.sh
 create mode 100644 util/fletcher.c
 create mode 100644 util/fletcher.h
Signature · 6 lines
2.9.3

_______________________________________________
Linux-nvdimm mailing list
[email protected]
https://lists.01.org/mailman/listinfo/linux-nvdimm
author maintainer
#1 lore raw

PATCH v3 1/7 libndctl: add a ndctl_namespace_is_active helper

The pattern of checking if a namespace is currently active was repeated in many places. Convert the scattered usage into a libndctl API. This gets rid of util_namespace_active from util/json.c which was an awkward place for this anyway.

---
 ndctl/builtin-list.c   |  2 +-
 ndctl/lib/libndctl.c   | 15 +++++++++++++++
 ndctl/lib/libndctl.sym |  1 +
 ndctl/libndctl.h.in    |  2 ++
 util/json.c            | 17 +----------------
 util/json.h            |  1 -
 6 files changed, 20 insertions(+), 18 deletions(-)
ndctl/builtin-list.c +1 -1
index e8d0070..536d333 100644
@@ -84,7 +84,7 @@ static struct json_object *list_namespaces(struct ndctl_region *region,
 		if (param.mode && mode_to_type(param.mode) != mode)
 			continue;
 
-		if (!list.idle && !util_namespace_active(ndns))
+		if (!list.idle && !ndctl_namespace_is_active(ndns))
 			continue;
 
 		if (!jnamespaces) {
ndctl/lib/libndctl.c +15 -0
index 565c969..8396e48 100644
@@ -3208,6 +3208,21 @@ static void region_refresh_children(struct ndctl_region *region)
 	daxs_init(region);
 }
 
+NDCTL_EXPORT bool ndctl_namespace_is_active(struct ndctl_namespace *ndns)
+{
+	struct ndctl_btt *btt = ndctl_namespace_get_btt(ndns);
+	struct ndctl_pfn *pfn = ndctl_namespace_get_pfn(ndns);
+	struct ndctl_dax *dax = ndctl_namespace_get_dax(ndns);
+
+	if ((btt && ndctl_btt_is_enabled(btt))
+			|| (pfn && ndctl_pfn_is_enabled(pfn))
+			|| (dax && ndctl_dax_is_enabled(dax))
+			|| (!btt && !pfn && !dax
+				&& ndctl_namespace_is_enabled(ndns)))
+		return true;
+	return false;
+}
+
 /*
  * Return 0 if enabled, < 0 if failed to enable, and > 0 if claimed by
  * another device and that device is enabled.  In the > 0 case a
ndctl/lib/libndctl.sym +1 -0
index be2e368..5f47a22 100644
@@ -171,6 +171,7 @@ global:
 	ndctl_namespace_enable;
 	ndctl_namespace_disable;
 	ndctl_namespace_disable_invalidate;
+	ndctl_namespace_is_active;
 	ndctl_namespace_is_valid;
 	ndctl_namespace_is_configured;
 	ndctl_namespace_delete;
ndctl/libndctl.h.in +2 -0
index c27581d..c70215c 100644
@@ -13,6 +13,7 @@
 #ifndef _LIBNDCTL_H_
 #define _LIBNDCTL_H_
 
+#include <stdbool.h>
 #include <stdarg.h>
 #include <unistd.h>
 
@@ -483,6 +484,7 @@ int ndctl_namespace_is_enabled(struct ndctl_namespace *ndns);
 int ndctl_namespace_enable(struct ndctl_namespace *ndns);
 int ndctl_namespace_disable(struct ndctl_namespace *ndns);
 int ndctl_namespace_disable_invalidate(struct ndctl_namespace *ndns);
+bool ndctl_namespace_is_active(struct ndctl_namespace *ndns);
 int ndctl_namespace_is_valid(struct ndctl_namespace *ndns);
 int ndctl_namespace_is_configured(struct ndctl_namespace *ndns);
 int ndctl_namespace_delete(struct ndctl_namespace *ndns);
util/json.c +1 -16
index d6a8d4c..82d8073 100644
@@ -86,21 +86,6 @@ struct json_object *util_dimm_to_json(struct ndctl_dimm *dimm)
 	return NULL;
 }
 
-bool util_namespace_active(struct ndctl_namespace *ndns)
-{
-	struct ndctl_btt *btt = ndctl_namespace_get_btt(ndns);
-	struct ndctl_pfn *pfn = ndctl_namespace_get_pfn(ndns);
-	struct ndctl_dax *dax = ndctl_namespace_get_dax(ndns);
-
-	if ((btt && ndctl_btt_is_enabled(btt))
-			|| (pfn && ndctl_pfn_is_enabled(pfn))
-			|| (dax && ndctl_dax_is_enabled(dax))
-			|| (!btt && !pfn && !dax
-				&& ndctl_namespace_is_enabled(ndns)))
-		return true;
-	return false;
-}
-
 struct json_object *util_daxctl_dev_to_json(struct daxctl_dev *dev)
 {
 	const char *devname = daxctl_dev_get_devname(dev);
@@ -334,7 +319,7 @@ struct json_object *util_namespace_to_json(struct ndctl_namespace *ndns,
 		json_object_object_add(jndns, "blockdev", jobj);
 	}
 
-	if (!util_namespace_active(ndns)) {
+	if (!ndctl_namespace_is_active(ndns)) {
 		jobj = json_object_new_string("disabled");
 		if (!jobj)
 			goto err;
util/json.h +0 -1
index a9afb2d..2449c2d 100644
@@ -6,7 +6,6 @@
 
 struct json_object;
 void util_display_json_array(FILE *f_out, struct json_object *jarray, int jflag);
-bool util_namespace_active(struct ndctl_namespace *ndns);
 struct json_object *util_bus_to_json(struct ndctl_bus *bus);
 struct json_object *util_dimm_to_json(struct ndctl_dimm *dimm);
 struct json_object *util_mapping_to_json(struct ndctl_mapping *mapping);
Signature · 6 lines
2.9.3

_______________________________________________
Linux-nvdimm mailing list
[email protected]
https://lists.01.org/mailman/listinfo/linux-nvdimm
author maintainer
#2 lore raw

PATCH v3 2/7 libndctl: add a ndctl_namespace_disable_safe() API

In reply to Vishal Verma

Disabling a namespace which has a filesystem mounted on it is unsafe as filesystems are not prepared for a block device to be yanked from under them. The destroy_namespace routine checked for an active mount by performing an O_EXCL open of the backing block device, but many other callers of ndctl_namespace_disable* could benefit from this checking.

Codify the mounted filesystem check in a new libndctl API - ndctl_namespace_disable_safe(), and use it for the destroy/disable namespace ndctl commands as well as the upcoming check-namespace command.

---
 ndctl/builtin-xaction-namespace.c | 46 +++++++--------------------------------
 ndctl/lib/libndctl.c              | 44 +++++++++++++++++++++++++++++++++++++
 ndctl/lib/libndctl.sym            |  1 +
 ndctl/libndctl.h.in               |  1 +
 4 files changed, 54 insertions(+), 38 deletions(-)
ndctl/builtin-xaction-namespace.c +8 -38
index 46d651e..d6b0c37 100644
@@ -731,10 +731,7 @@ static int namespace_destroy(struct ndctl_region *region,
 	struct ndctl_pfn *pfn = ndctl_namespace_get_pfn(ndns);
 	struct ndctl_dax *dax = ndctl_namespace_get_dax(ndns);
 	struct ndctl_btt *btt = ndctl_namespace_get_btt(ndns);
-	const char *bdev = NULL;
-	bool dax_active = false;
-	char path[50];
-	int fd, rc;
+	int rc;
 
 	if (ndctl_region_get_ro(region)) {
 		error("%s: read-only, re-configuration disabled\n",
@@ -742,42 +739,15 @@ static int namespace_destroy(struct ndctl_region *region,
 		return -ENXIO;
 	}
 
-	if (pfn && ndctl_pfn_is_enabled(pfn))
-		bdev = ndctl_pfn_get_block_device(pfn);
-	else if (dax && ndctl_dax_is_enabled(dax))
-		dax_active = true;
-	else if (btt && ndctl_btt_is_enabled(btt))
-		bdev = ndctl_btt_get_block_device(btt);
-	else if (ndctl_namespace_is_enabled(ndns))
-		bdev = ndctl_namespace_get_block_device(ndns);
-
-	if ((bdev || dax_active) && !force) {
+	if (ndctl_namespace_is_active(ndns) && !force) {
 		error("%s is active, specify --force for re-configuration\n",
 				devname);
 		return -EBUSY;
-	} else if (bdev) {
-		sprintf(path, "/dev/%s", bdev);
-		fd = open(path, O_RDWR|O_EXCL);
-		if (fd >= 0) {
-			/*
-			 * Got it, now block new mounts while we have it
-			 * pinned.
-			 */
-			ndctl_namespace_disable_invalidate(ndns);
-			close(fd);
-		} else {
-			/*
-			 * Yes, TOCTOU hole, but if you're racing namespace
-			 * creation you have other problems, and there's nothing
-			 * stopping the !bdev case from racing to mount an fs or
-			 * re-enabling the namepace.
-			 */
-			error("%s: %s failed exlusive open: %s\n",
-					devname, bdev, strerror(errno));
-			return -errno;
-		}
-	} else if (dax_active)
-		ndctl_namespace_disable_invalidate(ndns);
+	} else {
+		rc = ndctl_namespace_disable_safe(ndns);
+		if (rc)
+			return rc;
+	}
 
 	if (pfn || btt || dax) {
 		rc = zero_info_block(ndns);
@@ -869,7 +839,7 @@ static int do_xaction_namespace(const char *namespace,
 					continue;
 				switch (action) {
 				case ACTION_DISABLE:
-					rc = ndctl_namespace_disable_invalidate(ndns);
+					rc = ndctl_namespace_disable_safe(ndns);
 					break;
 				case ACTION_ENABLE:
 					rc = ndctl_namespace_enable(ndns);
ndctl/lib/libndctl.c +44 -0

50 lines not on this page · raw message

ndctl/lib/libndctl.sym +1 -0
index 5f47a22..2163a09 100644
@@ -171,6 +171,7 @@ global:
 	ndctl_namespace_enable;
 	ndctl_namespace_disable;
 	ndctl_namespace_disable_invalidate;
+	ndctl_namespace_disable_safe;
 	ndctl_namespace_is_active;
 	ndctl_namespace_is_valid;
 	ndctl_namespace_is_configured;
ndctl/libndctl.h.in +1 -0
index c70215c..06e6d49 100644
@@ -484,6 +484,7 @@ int ndctl_namespace_is_enabled(struct ndctl_namespace *ndns);
 int ndctl_namespace_enable(struct ndctl_namespace *ndns);
 int ndctl_namespace_disable(struct ndctl_namespace *ndns);
 int ndctl_namespace_disable_invalidate(struct ndctl_namespace *ndns);
+int ndctl_namespace_disable_safe(struct ndctl_namespace *ndns);
 bool ndctl_namespace_is_active(struct ndctl_namespace *ndns);
 int ndctl_namespace_is_valid(struct ndctl_namespace *ndns);
 int ndctl_namespace_is_configured(struct ndctl_namespace *ndns);
Signature · 6 lines
2.9.3

_______________________________________________
Linux-nvdimm mailing list
[email protected]
https://lists.01.org/mailman/listinfo/linux-nvdimm
author maintainer
#3 lore raw

PATCH v3 3/7 ccan: Add ccan/bitmap in preparation for the BTT checker

In reply to Vishal Verma

The BTT checker will include a bitmap test where we mark a bit for each post-map and free block, and check if the bitmap is full. Add ccan/bitmap to facilitate this.

---
 Makefile.am          |   4 +-
 ccan/bitmap/LICENSE  |   1 +
 ccan/bitmap/bitmap.c | 125 +++++++++++++
 ccan/bitmap/bitmap.h | 243 ++++++++++++++++++++++++
 licenses/LGPL-2.1    | 508 +++++++++++++++++++++++++++++++++++++++++++++++++++
 ndctl.spec.in        |   2 +-
 6 files changed, 881 insertions(+), 2 deletions(-)
 create mode 120000 ccan/bitmap/LICENSE
 create mode 100644 ccan/bitmap/bitmap.c
 create mode 100644 ccan/bitmap/bitmap.h
 create mode 100644 licenses/LGPL-2.1
Makefile.am +3 -1
index 06cd1b0..1d5a81c 100644
@@ -56,7 +56,9 @@ libccan_a_SOURCES = \
 	ccan/array_size/array_size.h \
 	ccan/minmax/minmax.h \
 	ccan/short_types/short_types.h \
-	ccan/endian/endian.h
+	ccan/endian/endian.h \
+	ccan/bitmap/bitmap.h \
+	ccan/bitmap/bitmap.c
 
 noinst_LIBRARIES += libutil.a
 libutil_a_SOURCES = \
ccan/bitmap/LICENSE new file +1 -0
new file mode 120000
index 0000000..dc314ec
@@ -0,0 +1 @@
+../../licenses/LGPL-2.1
\ No newline at end of file
ccan/bitmap/bitmap.c new file +125 -0

125 lines not on this page · raw message

ccan/bitmap/bitmap.h new file +243 -0

243 lines not on this page · raw message

licenses/LGPL-2.1 new file +60 -0

60 lines not on this page · raw message

+ Finally, software patents pose a constant threat to the existence of +any free program. We wish to make sure that a company cannot +effectively restrict the users of a free program by obtaining a +restrictive license from a patent holder. Therefore, we insist that +any patent license obtained for a version of the library must be +consistent with the full freedom of use specified in this license. +
+ Most GNU software, including some libraries, is covered by the +ordinary GNU General Public License. This license, the GNU Lesser +General Public License, applies to certain designated libraries, and +is quite different from the ordinary General Public License. We use +this license for certain libraries in order to permit linking those +libraries into non-free programs.
+
+ When a program is linked with a library, whether statically or using +a shared library, the combination of the two is legally speaking a +combined work, a derivative of the original library. The ordinary +General Public License therefore permits such linking only if the +entire combination fits its criteria of freedom. The Lesser General +Public License permits more lax criteria for linking other code with +the library.
+
+ We call this license the "Lesser" General Public License because it +does Less to protect the user's freedom than the ordinary General +Public License. It also provides other free software developers Less +of an advantage over competing non-free programs. These disadvantages +are the reason we use the ordinary General Public License for many +libraries. However, the Lesser license provides advantages in certain +special circumstances.
+
+ For example, on rare occasions, there may be a special need to +encourage the widest possible use of a certain library, so that it +becomes a de-facto standard. To achieve this, non-free programs must +be allowed to use the library. A more frequent case is that a free +library does the same job as widely used non-free libraries. In this +case, there is little to gain by limiting the free library to free +software only, so we use the Lesser General Public License. +
+ In other cases, permission to use a particular library in non-free +programs enables a greater number of people to use a large body of +free software. For example, permission to use the GNU C Library in +non-free programs enables many more people to use the whole GNU +operating system, as well as its variant, the GNU/Linux operating +system.
+
+ Although the Lesser General Public License is Less protective of the +users' freedom, it does ensure that the user of a program that is +linked with the Library has the freedom and the wherewithal to run +that program using a modified version of the Library.
+
+ The precise terms and conditions for copying, distribution and +modification follow. Pay close attention to the difference between a +"work based on the library" and a "work that uses the library". The +former contains code derived from the library, whereas the latter must +be combined with the library in order to run.
+

+ GNU LESSER GENERAL PUBLIC LICENSE
+ TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND MODIFICATION
+
+ 0. This License Agreement applies to any software library or other +program which contains a notice placed by the copyright holder or +other authorized party saying it may be distributed under the terms of +this Lesser General Public License (also called "this License"). +Each licensee is addressed as "you".
+
+ A "library" means a collection of software functions and/or data +prepared so as to be conveniently linked with application programs +(which use some of those functions and data) to form executables. +
+ The "Library", below, refers to any such software library or work +which has been distributed under these terms. A "work based on the +Library" means either the Library or any derivative work under +copyright law: that is to say, a work containing the Library or a +portion of it, either verbatim or with modifications and/or translated +straightforwardly into another language. (Hereinafter, translation is +included without limitation in the term "modification".)
+
+ "Source code" for a work means the preferred form of the work for +making modifications to it. For a library, complete source code means +all the source code for all modules it contains, plus any associated +interface definition files, plus the scripts used to control +compilation and installation of the library.
+
+ Activities other than copying, distribution and modification are not +covered by this License; they are outside its scope. The act of +running a program using the Library is not restricted, and output from +such a program is covered only if its contents constitute a work based +on the Library (independent of the use of the Library in a tool for +writing it). Whether that is true depends on what the Library does +and what the program that uses the Library does.
+
+ 1. You may copy and distribute verbatim copies of the Library's +complete source code as you receive it, in any medium, provided that +you conspicuously and appropriately publish on each copy an +appropriate copyright notice and disclaimer of warranty; keep intact +all the notices that refer to this License and to the absence of any +warranty; and distribute a copy of this License along with the +Library.
+
+ You may charge a fee for the physical act of transferring a copy, +and you may at your option offer warranty protection in exchange for a +fee.
+

+ 2. You may modify your copy or copies of the Library or any portion +of it, thus forming a work based on the Library, and copy and +distribute such modifications or work under the terms of Section 1 +above, provided that you also meet all of these conditions:
+
+ a) The modified work must itself be a software library.
+
+ b) You must cause the files modified to carry prominent notices
+ stating that you changed the files and the date of any change.
+
+ c) You must cause the whole of the work to be licensed at no
+ charge to all third parties under the terms of this License.
+

+    d) If a facility in the modified Library refers to a function or a
+    table of data to be supplied by an application program that uses
+    the facility, other than as an argument passed when the facility
+    is invoked, then you must make a good faith effort to ensure that,
+    in the event an application does not supply such function or
+    table, the facility still operates, and performs whatever part of
+    its purpose remains meaningful.
+
+    (For example, a function in a library to compute square roots has
+    a purpose that is entirely well-defined independent of the
+    application.  Therefore, Subsection 2d requires that any
+    application-supplied function or table used by this function must
+    be optional: if the application does not supply it, the square
+    root function must still compute square roots.)

+
+These requirements apply to the modified work as a whole. If +identifiable sections of that work are not derived from the Library, +and can be reasonably considered independent and separate works in +themselves, then this License, and its terms, do not apply to those +sections when you distribute them as separate works. But when you +distribute the same sections as part of a whole which is a work based +on the Library, the distribution of the whole must be on the terms of +this License, whose permissions for other licensees extend to the +entire whole, and thus to each and every part regardless of who wrote +it.
+
+Thus, it is not the intent of this section to claim rights or contest +your rights to work written entirely by you; rather, the intent is to +exercise the right to control the distribution of derivative or +collective works based on the Library.
+
+In addition, mere aggregation of another work not based on the Library +with the Library (or with a work based on the Library) on a volume of +a storage or distribution medium does not bring the other work under +the scope of this License.
+
+ 3. You may opt to apply the terms of the ordinary GNU General Public +License instead of this License to a given copy of the Library. To do +this, you must alter all the notices that refer to this License, so +that they refer to the ordinary GNU General Public License, version 2, +instead of to this License. (If a newer version than version 2 of the +ordinary GNU General Public License has appeared, then you can specify +that version instead if you wish.) Do not make any other change in +these notices.
+

+ Once this change is made in a given copy, it is irreversible for +that copy, so the ordinary GNU General Public License applies to all +subsequent copies and derivative works made from that copy. +
+ This option is useful when you wish to copy part of the code of +the Library into a program that is not a library.
+
+ 4. You may copy and distribute the Library (or a portion or +derivative of it, under Section 2) in object code or executable form +under the terms of Sections 1 and 2 above provided that you accompany +it with the complete corresponding machine-readable source code, which +must be distributed under the terms of Sections 1 and 2 above on a +medium customarily used for software interchange.
+
+ If distribution of object code is made by offering access to copy +from a designated place, then offering equivalent access to copy the +source code from the same place satisfies the requirement to +distribute the source code, even though third parties are not +compelled to copy the source along with the object code.
+
+ 5. A program that contains no derivative of any portion of the +Library, but is designed to work with the Library by being compiled or +linked with it, is called a "work that uses the Library". Such a +work, in isolation, is not a derivative work of the Library, and +therefore falls outside the scope of this License.
+
+ However, linking a "work that uses the Library" with the Library +creates an executable that is a derivative of the Library (because it +contains portions of the Library), rather than a "work that uses the +library". The executable is therefore covered by this License. +Section 6 states terms for distribution of such executables. +
+ When a "work that uses the Library" uses material from a header file +that is part of the Library, the object code for the work may be a +derivative work of the Library even though the source code is not. +Whether this is true is especially significant if the work can be +linked without the Library, or if the work is itself a library. The +threshold for this to be true is not precisely defined by law. +
+ If such an object file uses only numerical parameters, data +structure layouts and accessors, and small macros and small inline +functions (ten lines or less in length), then the use of the object +file is unrestricted, regardless of whether it is legally a derivative +work. (Executables containing this object code plus portions of the +Library will still fall under Section 6.)
+
+ Otherwise, if the work is a derivative of the Library, you may +distribute the object code for the work under the terms of Section 6. +Any executables containing that work also fall under Section 6, +whether or not they are linked directly with the Library itself. +

+ 6. As an exception to the Sections above, you may also combine or +link a "work that uses the Library" with the Library to produce a +work containing portions of the Library, and distribute that work +under terms of your choice, provided that the terms permit +modification of the work for the customer's own use and reverse +engineering for debugging such modifications.
+
+ You must give prominent notice with each copy of the work that the +Library is used in it and that the Library and its use are covered by +this License. You must supply a copy of this License. If the work +during execution displays copyright notices, you must include the +copyright notice for the Library among them, as well as a reference +directing the user to the copy of this License. Also, you must do one +of these things:
+

+    a) Accompany the work with the complete corresponding
+    machine-readable source code for the Library including whatever
+    changes were used in the work (which must be distributed under
+    Sections 1 and 2 above); and, if the work is an executable linked
+    with the Library, with the complete machine-readable "work that
+    uses the Library", as object code and/or source code, so that the
+    user can modify the Library and then relink to produce a modified
+    executable containing the modified Library.  (It is understood
+    that the user who changes the contents of definitions files in the
+    Library will not necessarily be able to recompile the application
+    to use the modified definitions.)
+
+    b) Use a suitable shared library mechanism for linking with the
+    Library.  A suitable mechanism is one that (1) uses at run time a
+    copy of the library already present on the user's computer system,
+    rather than copying library functions into the executable, and (2)
+    will operate properly with a modified version of the library, if
+    the user installs one, as long as the modified version is
+    interface-compatible with the version that the work was made with.
+
+    c) Accompany the work with a written offer, valid for at least
+    three years, to give the same user the materials specified in
+    Subsection 6a, above, for a charge no more than the cost of
+    performing this distribution.
+
+    d) If distribution of the work is made by offering access to copy
+    from a designated place, offer equivalent access to copy the above
+    specified materials from the same place.

+
+ e) Verify that the user has already received a copy of these
+ materials or that you have already sent this user a copy.
+
+ For an executable, the required form of the "work that uses the +Library" must include any data and utility programs needed for +reproducing the executable from it. However, as a special exception, +the materials to be distributed need not include anything that is +normally distributed (in either source or binary form) with the major +components (compiler, kernel, and so on) of the operating system on +which the executable runs, unless that component itself accompanies +the executable.
+
+ It may happen that this requirement contradicts the license +restrictions of other proprietary libraries that do not normally +accompany the operating system. Such a contradiction means you cannot +use both them and the Library together in an executable that you +distribute.
+

+ 7. You may place library facilities that are a work based on the +Library side-by-side in a single library together with other library +facilities not covered by this License, and distribute such a combined +library, provided that the separate distribution of the work based on +the Library and of the other library facilities is otherwise +permitted, and provided that you do these two things:
+

+    a) Accompany the combined library with a copy of the same work
+    based on the Library, uncombined with any other library
+    facilities.  This must be distributed under the terms of the
+    Sections above.
+
+    b) Give prominent notice with the combined library of the fact
+    that part of it is a work based on the Library, and explaining
+    where to find the accompanying uncombined form of the same work.

+
+ 8. You may not copy, modify, sublicense, link with, or distribute +the Library except as expressly provided under this License. Any +attempt otherwise to copy, modify, sublicense, link with, or +distribute the Library is void, and will automatically terminate your +rights under this License. However, parties who have received copies, +or rights, from you under this License will not have their licenses +terminated so long as such parties remain in full compliance. +
+ 9. You are not required to accept this License, since you have not +signed it. However, nothing else grants you permission to modify or +distribute the Library or its derivative works. These actions are +prohibited by law if you do not accept this License. Therefore, by +modifying or distributing the Library (or any work based on the +Library), you indicate your acceptance of this License to do so, and +all its terms and conditions for copying, distributing or modifying +the Library or works based on it.
+
+ 10. Each time you redistribute the Library (or any work based on the +Library), the recipient automatically receives a license from the +original licensor to copy, distribute, link with or modify the Library +subject to these terms and conditions. You may not impose any further +restrictions on the recipients' exercise of the rights granted herein. +You are not responsible for enforcing compliance by third parties with +this License.
+

+ 11. If, as a consequence of a court judgment or allegation of patent +infringement or for any other reason (not limited to patent issues), +conditions are imposed on you (whether by court order, agreement or +otherwise) that contradict the conditions of this License, they do not +excuse you from the conditions of this License. If you cannot +distribute so as to satisfy simultaneously your obligations under this +License and any other pertinent obligations, then as a consequence you +may not distribute the Library at all. For example, if a patent +license would not permit royalty-free redistribution of the Library by +all those who receive copies directly or indirectly through you, then +the only way you could satisfy both it and this License would be to +refrain entirely from distribution of the Library.
+
+If any portion of this section is held invalid or unenforceable under +any particular circumstance, the balance of the section is intended to +apply, and the section as a whole is intended to apply in other +circumstances.
+
+It is not the purpose of this section to induce you to infringe any +patents or other property right claims or to contest validity of any +such claims; this section has the sole purpose of protecting the +integrity of the free software distribution system which is +implemented by public license practices. Many people have made +generous contributions to the wide range of software distributed +through that system in reliance on consistent application of that +system; it is up to the author/donor to decide if he or she is willing +to distribute software through any other system and a licensee cannot +impose that choice.
+
+This section is intended to make thoroughly clear what is believed to +be a consequence of the rest of this License.
+
+ 12. If the distribution and/or use of the Library is restricted in +certain countries either by patents or by copyrighted interfaces, the +original copyright holder who places the Library under this License +may add an explicit geographical distribution limitation excluding those +countries, so that distribution is permitted only in or among +countries not thus excluded. In such case, this License incorporates +the limitation as if written in the body of this License.
+
+ 13. The Free Software Foundation may publish revised and/or new +versions of the Lesser General Public License from time to time. +Such new versions will be similar in spirit to the present version, +but may differ in detail to address new problems or concerns. +
+Each version is given a distinguishing version number. If the Library +specifies a version number of this License which applies to it and +"any later version", you have the option of following the terms and +conditions either of that version or of any later version published by +the Free Software Foundation. If the Library does not specify a +license version number, you may choose any version ever published by +the Free Software Foundation.
+

+ 14. If you wish to incorporate parts of the Library into other free +programs whose distribution conditions are incompatible with these, +write to the author to ask for permission. For software which is +copyrighted by the Free Software Foundation, write to the Free +Software Foundation; we sometimes make exceptions for this. Our +decision will be guided by the two goals of preserving the free status +of all derivatives of our free software and of promoting the sharing +and reuse of software generally.
+
+ NO WARRANTY
+
+ 15. BECAUSE THE LIBRARY IS LICENSED FREE OF CHARGE, THERE IS NO +WARRANTY FOR THE LIBRARY, TO THE EXTENT PERMITTED BY APPLICABLE LAW. +EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT HOLDERS AND/OR +OTHER PARTIES PROVIDE THE LIBRARY "AS IS" WITHOUT WARRANTY OF ANY +KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE +IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR +PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE +LIBRARY IS WITH YOU. SHOULD THE LIBRARY PROVE DEFECTIVE, YOU ASSUME +THE COST OF ALL NECESSARY SERVICING, REPAIR OR CORRECTION. +
+ 16. IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN +WRITING WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MAY MODIFY +AND/OR REDISTRIBUTE THE LIBRARY AS PERMITTED ABOVE, BE LIABLE TO YOU +FOR DAMAGES, INCLUDING ANY GENERAL, SPECIAL, INCIDENTAL OR +CONSEQUENTIAL DAMAGES ARISING OUT OF THE USE OR INABILITY TO USE THE +LIBRARY (INCLUDING BUT NOT LIMITED TO LOSS OF DATA OR DATA BEING +RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD PARTIES OR A +FAILURE OF THE LIBRARY TO OPERATE WITH ANY OTHER SOFTWARE), EVEN IF +SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH +DAMAGES.
+
+ END OF TERMS AND CONDITIONS
+

+ How to Apply These Terms to Your New Libraries
+
+ If you develop a new library, and you want it to be of the greatest +possible use to the public, we recommend making it free software that +everyone can redistribute and change. You can do so by permitting +redistribution under these terms (or, alternatively, under the terms +of the ordinary General Public License).
+
+ To apply these terms, attach the following notices to the library. +It is safest to attach them to the start of each source file to most +effectively convey the exclusion of warranty; and each file should +have at least the "copyright" line and a pointer to where the full +notice is found.
+
+
+ <one line to give the library's name and a brief idea of what it does.>
+ Copyright (C) <year> <name of author>
+

+    This library is free software; you can redistribute it and/or
+    modify it under the terms of the GNU Lesser General Public
+    License as published by the Free Software Foundation; either
+    version 2.1 of the License, or (at your option) any later version.
+
+    This library is distributed in the hope that it will be useful,
+    but WITHOUT ANY WARRANTY; without even the implied warranty of
+    MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+    Lesser General Public License for more details.
+
+    You should have received a copy of the GNU Lesser General Public
+    License along with this library; if not, write to the Free Software
+    Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA

+
+Also add information on how to contact you by electronic and paper mail. +
+You should also get your employer (if you work as a programmer) or +your school, if any, to sign a "copyright disclaimer" for the library, +if necessary. Here is a sample; alter the names:
+

+  Yoyodyne, Inc., hereby disclaims all copyright interest in the
+  library `Frob' (a library for tweaking knobs) written by James
+  Random Hacker.

+
+ <signature of Ty Coon>, 1 April 1990
+ Ty Coon, President of Vice
+
+That's all there is to it!

ndctl.spec.in +1 -1
index b481762..b06b669 100644
@@ -112,7 +112,7 @@ make check
 
 %files
 %defattr(-,root,root)
-%license util/COPYING licenses/BSD-MIT licenses/CC0
+%license util/COPYING licenses/BSD-MIT licenses/CC0 licenses/LGPL-2.1
 %{_bindir}/ndctl
 %{_mandir}/man1/ndctl*
 %{bashcompdir}/
Signature · 6 lines
2.9.3

_______________________________________________
Linux-nvdimm mailing list
[email protected]
https://lists.01.org/mailman/listinfo/linux-nvdimm
#4 lore raw

Re: [ndctl PATCH v3 3/7] ccan: Add ccan/bitmap in preparation for the BTT checker

Quoted 3 lines · Vishal Verma

The BTT checker will include a bitmap test where we mark a bit for each post-map and free block, and check if the bitmap is full. Add ccan/bitmap to facilitate this.

Our license compliance tooling flagged this as incompatible for use with the ndctl utility. We could include it in libndctl since that is LGPL, but since ndctl is GPLv2 the static linking to this LGPL code gets flagged. Lets just copy the GPLv2 bitmap implementation from the kernel.
_______________________________________________
Linux-nvdimm mailing list
[email protected]
https://lists.01.org/mailman/listinfo/linux-nvdimm

author maintainer
#5 lore raw

PATCH v3 4/7 ccan/bitmap: fix a set of gcc warnings (with -Wshadow)

In reply to Vishal Verma

Having the typedef named the same as an instance of the struct causes gcc with -Wshadow to spew warnings like:

./ccan/bitmap/bitmap.h: In function ‘bitmap_clear_bit’:
./ccan/bitmap/bitmap.h:63:45: warning: declaration of ‘bitmap’ shadows a global declaration [-Wshadow]
static inline void bitmap_clear_bit(bitmap *bitmap, unsigned long n)

                                            ^~~~~~
./ccan/bitmap/bitmap.h:23:3: note: shadowed declaration is here } bitmap;
                                                                  ^~~~~~

Stop these by converting the typedef to bitmap_t so that it doesn't conflict with the many places a 'bitmap' variable is used.

Cc Dan Williams <[email protected]>
Cc Ross Zwisler <[email protected]>
---
 ccan/bitmap/bitmap.c |  6 +++---
 ccan/bitmap/bitmap.h | 52 ++++++++++++++++++++++++++--------------------------
 2 files changed, 29 insertions(+), 29 deletions(-)
ccan/bitmap/bitmap.c +3 -3
index d812af6..554e000 100644
@@ -9,7 +9,7 @@
 #define BIT_ALIGN_DOWN(n)	((n) & ~(BITMAP_WORD_BITS - 1))
 #define BIT_ALIGN_UP(n)		BIT_ALIGN_DOWN((n) + BITMAP_WORD_BITS - 1)
 
-void bitmap_zero_range(bitmap *bitmap, unsigned long n, unsigned long m)
+void bitmap_zero_range(bitmap_t *bitmap, unsigned long n, unsigned long m)
 {
 	unsigned long an = BIT_ALIGN_UP(n);
 	unsigned long am = BIT_ALIGN_DOWN(m);
@@ -34,7 +34,7 @@ void bitmap_zero_range(bitmap *bitmap, unsigned long n, unsigned long m)
 		BITMAP_WORD(bitmap, m) &= ~bitmap_bswap(tailmask);
 }
 
-void bitmap_fill_range(bitmap *bitmap, unsigned long n, unsigned long m)
+void bitmap_fill_range(bitmap_t *bitmap, unsigned long n, unsigned long m)
 {
 	unsigned long an = BIT_ALIGN_UP(n);
 	unsigned long am = BIT_ALIGN_DOWN(m);
@@ -76,7 +76,7 @@ static int bitmap_clz(bitmap_word w)
 #endif
 }
 
-unsigned long bitmap_ffs(const bitmap *bitmap,
+unsigned long bitmap_ffs(const bitmap_t *bitmap,
 			 unsigned long n, unsigned long m)
 {
 	unsigned long an = BIT_ALIGN_UP(n);
ccan/bitmap/bitmap.h +26 -26

168 lines not on this page · raw message

Signature · 6 lines
2.9.3

_______________________________________________
Linux-nvdimm mailing list
[email protected]
https://lists.01.org/mailman/listinfo/linux-nvdimm
author maintainer
#6 lore raw

PATCH v3 5/7 ndctl: move the fletcher64 routine to util/

In reply to Vishal Verma

In preparation for check-namespace, since it will also use the fletcher64 routine, move it to util/ so that it can be shared by both builtin-check.c and builtin-dimm.c

---
 Makefile.am          |  3 ++-
 ndctl/builtin-dimm.c | 18 ++----------------
 util/fletcher.c      | 23 +++++++++++++++++++++++
 util/fletcher.h      |  8 ++++++++
 4 files changed, 35 insertions(+), 17 deletions(-)
 create mode 100644 util/fletcher.c
 create mode 100644 util/fletcher.h
Makefile.am +2 -1
index 1d5a81c..26c7309 100644
@@ -70,6 +70,7 @@ libutil_a_SOURCES = \
 	util/help.c \
 	util/strbuf.c \
 	util/wrapper.c \
-	util/filter.c
+	util/filter.c \
+	util/fletcher.c
 
 nobase_include_HEADERS = daxctl/libdaxctl.h
ndctl/builtin-dimm.c +2 -16
index 637b10b..93f9530 100644
@@ -22,6 +22,7 @@
 #include <util/json.h>
 #include <util/filter.h>
 #include <json-c/json.h>
+#include <util/fletcher.h>
 #include <ndctl/libndctl.h>
 #include <util/parse-options.h>
 #include <ccan/minmax/minmax.h>
@@ -358,7 +359,7 @@ struct nvdimm_data {
 };
 
 /*
- * Note, best_seq(), inc_seq(), fletcher64(), sizeof_namespace_index()
+ * Note, best_seq(), inc_seq(), sizeof_namespace_index()
  * nvdimm_num_label_slots(), label_validate(), and label_write_index()
  * are copied from drivers/nvdimm/label.c in the Linux kernel with the
  * following modifications:
@@ -371,21 +372,6 @@ struct nvdimm_data {
  * 7/ dropped clear_bit_le() usage in label_write_index
  */
 
-static u64 fletcher64(void *addr, size_t len, bool le)
-{
-	u32 *buf = addr;
-	u32 lo32 = 0;
-	u64 hi32 = 0;
-	size_t i;
-
-	for (i = 0; i < len / sizeof(u32); i++) {
-		lo32 += le ? le32_to_cpu((le32) buf[i]) : buf[i];
-		hi32 += lo32;
-	}
-
-	return hi32 << 32 | lo32;
-}
-
 static unsigned inc_seq(unsigned seq)
 {
 	static const unsigned next[] = { 0, 2, 3, 1 };
util/fletcher.c new file +23 -0
new file mode 100644
index 0000000..cee2fc3
@@ -0,0 +1,23 @@
+#include <stdlib.h>
+#include <stdbool.h>
+#include <util/fletcher.h>
+#include <ccan/endian/endian.h>
+#include <ccan/short_types/short_types.h>
+
+/*
+ * Note, fletcher64() is copied from drivers/nvdimm/label.c in the Linux kernel
+ */
+u64 fletcher64(void *addr, size_t len, bool le)
+{
+	u32 *buf = addr;
+	u32 lo32 = 0;
+	u64 hi32 = 0;
+	size_t i;
+
+	for (i = 0; i < len / sizeof(u32); i++) {
+		lo32 += le ? le32_to_cpu((le32) buf[i]) : buf[i];
+		hi32 += lo32;
+	}
+
+	return hi32 << 32 | lo32;
+}
util/fletcher.h new file +8 -0
new file mode 100644
index 0000000..e3bbce3
@@ -0,0 +1,8 @@
+#ifndef _NDCTL_FLETCHER_H_
+#define _NDCTL_FLETCHER_H_
+
+#include <ccan/short_types/short_types.h>
+
+u64 fletcher64(void *addr, size_t len, bool le);
+
+#endif /* _NDCTL_FLETCHER_H_ */
Signature · 6 lines
2.9.3

_______________________________________________
Linux-nvdimm mailing list
[email protected]
https://lists.01.org/mailman/listinfo/linux-nvdimm
author maintainer
#7 lore raw

PATCH v3 6/7 ndctl: add a BTT check utility

In reply to Vishal Verma

Add the check-namespace command to ndctl. This will check the BTT metadata layout for the given namespace, and if requested, correct any errors found. Not all metadata corruption is detectable or fixable.

Cc Dan Williams <[email protected]>
Cc Jeff Moyer <[email protected]>
Cc Linda Knippers <[email protected]>
---
 Documentation/Makefile.am               |   1 +
 Documentation/ndctl-check-namespace.txt |  64 +++
 Documentation/ndctl.txt                 |   1 +
 builtin.h                               |   1 +
 contrib/ndctl                           |   3 +
 ndctl/Makefile.am                       |   1 +
 ndctl/builtin-check.c                   | 988 ++++++++++++++++++++++++++++++++
 ndctl/builtin-xaction-namespace.c       |  66 ++-
 ndctl/check.h                           | 127 ++++
 ndctl/ndctl.c                           |   1 +
 util/util.h                             |   8 +
 11 files changed, 1259 insertions(+), 2 deletions(-)
 create mode 100644 Documentation/ndctl-check-namespace.txt
 create mode 100644 ndctl/builtin-check.c
 create mode 100644 ndctl/check.h
Documentation/Makefile.am +1 -0
index 6daeb56..eea11e0 100644
@@ -12,6 +12,7 @@ man1_MANS = \
 	ndctl-disable-namespace.1 \
 	ndctl-create-namespace.1 \
 	ndctl-destroy-namespace.1 \
+	ndctl-check-namespace.1 \
 	ndctl-list.1 \
 	daxctl-list.1
 
Documentation/ndctl-check-namespace.txt new file +64 -0

64 lines not on this page · raw message

Documentation/ndctl.txt +1 -0
index 883a59c..c26cc2f 100644
@@ -34,6 +34,7 @@ SEE ALSO
 --------
 linkndctl:ndctl-create-namespace[1],
 linkndctl:ndctl-destroy-namespace[1],
+linkndctl:ndctl-check-namespace[1],
 linkndctl:ndctl-enable-region[1],
 linkndctl:ndctl-disable-region[1],
 linkndctl:ndctl-enable-dimm[1],
builtin.h +1 -0
index 9b66196..200bd8e 100644
@@ -13,6 +13,7 @@ int cmd_enable_namespace(int argc, const char **argv, void *ctx);
 int cmd_create_namespace(int argc, const char **argv, void *ctx);
 int cmd_destroy_namespace(int argc, const char **argv, void *ctx);
 int cmd_disable_namespace(int argc, const char **argv, void *ctx);
+int cmd_check_namespace(int argc, const char **argv, void *ctx);
 int cmd_enable_region(int argc, const char **argv, void *ctx);
 int cmd_disable_region(int argc, const char **argv, void *ctx);
 int cmd_enable_dimm(int argc, const char **argv, void *ctx);
contrib/ndctl +3 -0
index ea7303c..c97adcc 100755
@@ -194,6 +194,9 @@ __ndctl_comp_non_option_args()
 	destroy-namespace)
 		opts="$(__ndctl_get_ns) all"
 		;;
+	check-namespace)
+		opts="$(__ndctl_get_ns -i) all"
+		;;
 	enable-region)
 		opts="$(__ndctl_get_regions -i) all"
 		;;
ndctl/Makefile.am +1 -0
index c563e94..f9158d9 100644
@@ -5,6 +5,7 @@ bin_PROGRAMS = ndctl
 ndctl_SOURCES = ndctl.c \
 		builtin-create-nfit.c \
 		builtin-xaction-namespace.c \
+		builtin-check.c \
 		builtin-xable-region.c \
 		builtin-dimm.c \
 		 ../util/log.c \
ndctl/builtin-check.c new file +988 -0

988 lines not on this page · raw message

ndctl/builtin-xaction-namespace.c +64 -2

116 lines not on this page · raw message

ndctl/check.h new file +127 -0

127 lines not on this page · raw message

ndctl/ndctl.c +1 -0
index 80a0491..0678a9a 100644
@@ -57,6 +57,7 @@ static struct cmd_struct commands[] = {
 	{ "disable-namespace", cmd_disable_namespace },
 	{ "create-namespace", cmd_create_namespace },
 	{ "destroy-namespace", cmd_destroy_namespace },
+	{ "check-namespace", cmd_check_namespace },
 	{ "enable-region", cmd_enable_region },
 	{ "disable-region", cmd_disable_region },
 	{ "enable-dimm", cmd_enable_dimm },
util/util.h +8 -0
index e0e5f26..d280d10 100644
@@ -23,6 +23,13 @@
 
 #define alloc_nr(x) (((x)+16)*3/2)
 
+#define rounddown(x, y) (				\
+{							\
+	typeof(x) __x = (x);				\
+	__x - (__x % (y));				\
+}							\
+)
+
 /*
  * Realloc the buffer pointed at by variable 'x' so that it can hold
  * at least 'nr' entries; the number of entries currently allocated
@@ -44,6 +51,7 @@
 #define zfree(ptr) ({ free(*ptr); *ptr = NULL; })
 
 #define BUILD_BUG_ON_ZERO(e) (sizeof(struct { int:-!!(e); }))
+#define BUILD_BUG_ON(condition) ((void)sizeof(char[1 - 2*!!(condition)]))
 
 static inline const char *skip_prefix(const char *str, const char *prefix)
 {
Signature · 6 lines
2.9.3

_______________________________________________
Linux-nvdimm mailing list
[email protected]
https://lists.01.org/mailman/listinfo/linux-nvdimm
#8 lore raw

Re: [ndctl PATCH v3 6/7] ndctl: add a BTT check utility

Hi Vishal,

Thanks for all your work on this. I finally got a chance to do some testing and have a few usability suggestions for you to consider.

On 03/04/2017 01:13 AM, Vishal Verma wrote:
<snip>

Quoted 40 lines
ndctl/builtin-check.c new file +40 -0
+int namespace_check(struct ndctl_namespace *ndns, struct check_opts *opts)
+{
+	const char *devname = ndctl_namespace_get_devname(ndns);
+	int raw_mode, rc, disabled_flag = 0, open_flags;
+	struct btt_sb *btt_sb;
+	struct btt_chk *bttc;
+	char path[50];
+
+	bttc = calloc(1, sizeof(*bttc));
+	if (bttc == NULL)
+		return -ENOMEM;
+
+	log_init(&bttc->ctx, devname, "NDCTL_CHECK_NAMESPACE");
+	if (opts->verbose)
+		bttc->ctx.log_priority = LOG_DEBUG;
+
+	bttc->opts = opts;
+	bttc->start_off = BTT_START_OFFSET;
+	bttc->sys_page_size = sysconf(_SC_PAGESIZE);
+	bttc->rawsize = ndctl_namespace_get_size(ndns);
+	ndctl_namespace_get_uuid(ndns, bttc->parent_uuid);
+
+	info(bttc, "checking %s\n", devname);
+	if (ndctl_namespace_is_active(ndns)) {
+		if (opts->force) {
+			rc = ndctl_namespace_disable_safe(ndns);
+			if (rc)
+				return rc;
+			disabled_flag = 1;
+		} else {
+			err(bttc, "%s: check aborted, namespace online\n",
+				devname);
+			rc = -EBUSY;
+			goto out_bttc;
+		}
+	}
+
+	/* In typical usage, the current raw_mode should be false. */
+	raw_mode = ndctl_namespace_get_raw_mode(ndns);
+

ndctl/builtin-check.c line 918

Quoted 10 lines
ndctl/builtin-check.c new file +10 -0
+	/*
+	 * Putting the namespace into raw mode will allow us to access
+	 * the btt metadata.
+	 */
+	rc = ndctl_namespace_set_raw_mode(ndns, 1);
+	if (rc < 0) {
+		err(bttc, "%s: failed to set the raw mode flag: %d\n",
+			devname, rc);
+		goto out_ns;
+	}

When I ran the utility as non-root, I got this:

$ ndctl check-namespace namespace3.0
namespace3.0: namespace_check: namespace3.0: failed to set the raw mode flag: -6
error checking namespaces: No such device or address

I think it ought to end with Permission denied, which it would do if ndctl_namespace_get_raw_mode() returned with -errno rather than a hard coded ENXIO.

If that's main reason that setting the raw mode flag can fail, then the err() could be an info(). That would simplify the output.

Quoted 29 lines
ndctl/builtin-check.c new file +29 -0
+	/*
+	 * Now enable the namespace.  This will result in a pmem device
+	 * node showing up in /dev that is in raw mode.
+	 */
+	rc = ndctl_namespace_enable(ndns);
+	if (rc != 0) {
+		err(bttc, "%s: failed to enable in raw mode: %d\n",
+			devname, rc);
+		goto out_ns;
+	}
+
+	sprintf(path, "/dev/%s", ndctl_namespace_get_block_device(ndns));
+	bttc->path = path;
+
+	btt_sb = malloc(sizeof(*btt_sb));
+	if (btt_sb == NULL) {
+		rc = -ENOMEM;
+		goto out_ns;
+	}
+
+	if (!bttc->opts->repair)
+		open_flags = O_RDONLY|O_EXCL;
+	else
+		open_flags = O_RDWR|O_EXCL;
+
+	bttc->fd = open(bttc->path, open_flags);
+	if (bttc->fd < 0) {
+		err(bttc, "unable to open %s: %s\n",
+			bttc->path, strerror(errno));

ndctl/builtin-check.c line 957

Quoted 10 lines
ndctl/builtin-check.c new file +10 -0
+		rc = -errno;
+		goto out_sb;
+	}
+
+	rc = btt_info_read_verify(bttc, btt_sb, bttc->start_off);
+	if (rc) {
+		rc = btt_recover_first_sb(bttc);
+		if (rc) {
+			info(bttc, "Unable to recover any BTT info blocks\n");
+			goto out_close;

When I ran this on a namespace that never had a BTT, I thought I'd get a error basically saying that I'm screwed, but instead I got:

$ sudo ndctl check-namespace namespace1.0
error checking namespaces: No such device or address

It seems like the info() above is rather important and should be an err(). For a checker, maybe verbose should be the default anyway.

Quoted 17 lines
ndctl/builtin-check.c new file +17 -0
+		}
+		rc = btt_info_read_verify(bttc, btt_sb, bttc->start_off);
+		if (rc)
+			goto out_close;
+	}
+	rc = btt_discover_arenas(bttc);
+	if (rc)
+		goto out_close;
+
+	rc = btt_create_mappings(bttc);
+	if (rc)
+		goto out_close;
+
+	rc = btt_check_arenas(bttc);
+
+	btt_remove_mappings(bttc);
+ out_close:

ndctl/builtin-check.c line 984

Quoted 10 lines
ndctl/builtin-check.c new file +10 -0
+	close(bttc->fd);
+ out_sb:
+	free(btt_sb);
+ out_ns:
+	ndctl_namespace_set_raw_mode(ndns, raw_mode);
+	ndctl_namespace_disable_invalidate(ndns);
+	if (disabled_flag)
+		if(ndctl_namespace_enable(ndns) < 0)
+			info(bttc, "%s: failed to re-enable namespace\n",
+				devname);

I didn't get this error but it seems rather important too.

Thanks,

- ljk

ndctl/builtin-check.c line 988

Quoted 4 lines

+ out_bttc:

ndctl/builtin-check.c new file +3 -0
+	free(bttc);
+	return rc;
+}

_______________________________________________
Linux-nvdimm mailing list
[email protected]
https://lists.01.org/mailman/listinfo/linux-nvdimm

author maintainer
#9 lore raw

Re: [ndctl PATCH v3 6/7] ndctl: add a BTT check utility

Quoted 4 lines · Linda Knippers

Hi Vishal,

Thanks for all your work on this.  I finally got a chance to do some testing and have a few usability suggestions for you to consider.

Thanks for the comments, Linda. See below, but yes I can send follow on patches for these.

Quoted 26 lines · Vishal Verma

<snip>

Quoted 13 lines

*/

ndctl/builtin-check.c new file +4 -0
+	raw_mode = ndctl_namespace_get_raw_mode(ndns);
+
+	/*
+	 * Putting the namespace into raw mode will allow us to

access

ndctl/builtin-check.c new file +5 -0
+	 * the btt metadata.
+	 */
+	rc = ndctl_namespace_set_raw_mode(ndns, 1);
+	if (rc < 0) {
+		err(bttc, "%s: failed to set the raw mode flag:

%d\n",

ndctl/builtin-check.c line 918

Quoted 3 lines
ndctl/builtin-check.c new file +3 -0
+			devname, rc);
+		goto out_ns;
+	}

When I ran the utility as non-root, I got this:

$ ndctl check-namespace namespace3.0
namespace3.0: namespace_check: namespace3.0: failed to set the raw
mode flag: -6
error checking namespaces: No such device or address
Quoted 4 lines

I think it ought to end with Permission denied, which it would do if ndctl_namespace_get_raw_mode() returned with -errno rather than a hard
coded ENXIO.

Good point - I'll fix this (in a follow on patch).

Quoted 4 lines

If that's main reason that setting the raw mode flag can fail, then the
err() could be an info().  That would simplify the output.

I think it should still be an error as it is a fatal problem and we have to abort..

<snip>

Quoted 13 lines

Review

Quoted 8 lines
ndctl/builtin-check.c new file +6 -0
+
+	rc = btt_info_read_verify(bttc, btt_sb, bttc->start_off);
+	if (rc) {
+		rc = btt_recover_first_sb(bttc);
+		if (rc) {
+			info(bttc, "Unable to recover any BTT info

blocks\n");

(quoted diff) +1 -0
+			goto out_close;

When I ran this on a namespace that never had a BTT, I thought I'd get a error
basically saying that I'm screwed, but instead I got:

Quoted 6 lines
$ sudo ndctl check-namespace namespace1.0
error checking namespaces: No such device or address

It seems like the info() above is rather important and should be an err().
For a checker, maybe verbose should be the default anyway.

Ok - let me revisit the no BTT case and see if we can print out something better.

Quoted 24 lines
Quoted 24 lines
(quoted diff) +2 -0
+		}
+		rc = btt_info_read_verify(bttc, btt_sb, bttc-
Quoted 1 line

start_off);

ndctl/builtin-check.c new file +20 -0
+		if (rc)
+			goto out_close;
+	}
+	rc = btt_discover_arenas(bttc);
+	if (rc)
+		goto out_close;
+
+	rc = btt_create_mappings(bttc);
+	if (rc)
+		goto out_close;
+
+	rc = btt_check_arenas(bttc);
+
+	btt_remove_mappings(bttc);
+ out_close:
+	close(bttc->fd);
+ out_sb:
+	free(btt_sb);
+ out_ns:
+	ndctl_namespace_set_raw_mode(ndns, raw_mode);
Quoted 8 lines

Review

Quoted 6 lines
ndctl/builtin-check.c new file +4 -0
+	ndctl_namespace_disable_invalidate(ndns);
+	if (disabled_flag)
+		if(ndctl_namespace_enable(ndns) < 0)
+			info(bttc, "%s: failed to re-enable

namespace\n",

(quoted diff) +1 -0
+				devname);

I didn't get this error but it seems rather important too.

This should normally never happen, and I had it as an info because at this point we can't really do anything about it. Any check/repair has already completed, and if we weren't able to enable the namespace, the user likely won't either for the same reason.. In any case, it seems more like an informational message since nothing can be done about it, and it doesn't affect any future flow..

Quoted 11 lines

Thanks,

- ljk

ndctl/builtin-check.c line 988

Quoted 4 lines

+ out_bttc:

ndctl/builtin-check.c new file +3 -0
+	free(bttc);
+	return rc;
+}

_______________________________________________
Linux-nvdimm mailing list
[email protected]
https://lists.01.org/mailman/listinfo/linux-nvdimm

#10 lore raw

Re: [ndctl PATCH v3 6/7] ndctl: add a BTT check utility

Quoted 8 lines · Vishal L
Quoted 4 lines · Linda Knippers

Hi Vishal,

Thanks for all your work on this. I finally got a chance to do some testing and have a few usability suggestions for you to consider.

Thanks for the comments, Linda. See below, but yes I can send follow on patches for these.

Great, later is fine. More below.

Quoted 35 lines
Quoted 27 lines · Vishal Verma

<snip>

Quoted 13 lines

*/

ndctl/builtin-check.c new file +4 -0
+	raw_mode = ndctl_namespace_get_raw_mode(ndns);
+
+	/*
+	 * Putting the namespace into raw mode will allow us to

access

ndctl/builtin-check.c new file +5 -0
+	 * the btt metadata.
+	 */
+	rc = ndctl_namespace_set_raw_mode(ndns, 1);
+	if (rc < 0) {
+		err(bttc, "%s: failed to set the raw mode flag:

%d\n",

ndctl/builtin-check.c line 918

Quoted 3 lines
ndctl/builtin-check.c new file +3 -0
+			devname, rc);
+		goto out_ns;
+	}

When I ran the utility as non-root, I got this:

$ ndctl check-namespace namespace3.0
namespace3.0: namespace_check: namespace3.0: failed to set the raw
mode flag: -6
error checking namespaces: No such device or address
Quoted 4 lines

I think it ought to end with Permission denied, which it would do if ndctl_namespace_get_raw_mode() returned with -errno rather than a hard
coded ENXIO.

Good point - I'll fix this (in a follow on patch).

Quoted 1 line
Quoted 6 lines
Quoted 3 lines

If that's main reason that setting the raw mode flag can fail, then the
err() could be an info(). That would simplify the output.

I think it should still be an error as it is a fatal problem and we have to abort..

It's just a bit confusing when the most likely cause is not running as root. Otherwise, that error probably can't really happen. In other places you don't even check the return value for the function, probably because it doesn't really happen.

Quoted 19 lines

<snip>

Quoted 16 lines

Review

Quoted 8 lines
ndctl/builtin-check.c new file +6 -0
+
+	rc = btt_info_read_verify(bttc, btt_sb, bttc->start_off);
+	if (rc) {
+		rc = btt_recover_first_sb(bttc);
+		if (rc) {
+			info(bttc, "Unable to recover any BTT info

blocks\n");

(quoted diff) +1 -0
+			goto out_close;

When I ran this on a namespace that never had a BTT, I thought I'd get a error
basically saying that I'm screwed, but instead I got:

$ sudo ndctl check-namespace namespace1.0
error checking namespaces: No such device or address
Quoted 6 lines
Quoted 3 lines

It seems like the info() above is rather important and should be an err().
For a checker, maybe verbose should be the default anyway.

Ok - let me revisit the no BTT case and see if we can print out something better.

The message that you have there is probably good enough, it just didn't come out without the verbose option. The "No such device or address" part is a bit wrong though because the device is there, there's just no btt on it.

Quoted 34 lines
Quoted 25 lines
Quoted 25 lines
(quoted diff) +2 -0
+		}
+		rc = btt_info_read_verify(bttc, btt_sb, bttc-
Quoted 1 line

start_off);

ndctl/builtin-check.c new file +20 -0
+		if (rc)
+			goto out_close;
+	}
+	rc = btt_discover_arenas(bttc);
+	if (rc)
+		goto out_close;
+
+	rc = btt_create_mappings(bttc);
+	if (rc)
+		goto out_close;
+
+	rc = btt_check_arenas(bttc);
+
+	btt_remove_mappings(bttc);
+ out_close:
+	close(bttc->fd);
+ out_sb:
+	free(btt_sb);
+ out_ns:
+	ndctl_namespace_set_raw_mode(ndns, raw_mode);
Quoted 8 lines

Review

Quoted 6 lines
ndctl/builtin-check.c new file +4 -0
+	ndctl_namespace_disable_invalidate(ndns);
+	if (disabled_flag)
+		if(ndctl_namespace_enable(ndns) < 0)
+			info(bttc, "%s: failed to re-enable

namespace\n",

(quoted diff) +1 -0
+				devname);

I didn't get this error but it seems rather important too.

Quoted 6 lines

This should normally never happen, and I had it as an info because at this point we can't really do anything about it. Any check/repair has already completed, and if we weren't able to enable the namespace, the user likely won't either for the same reason.. In any case, it seems more like an informational message since nothing can be done about it, and it doesn't affect any future flow..

The reason I think it should be an error is that something went wrong, whether you can do anything about it in the checker or not. Without an error the user will have no idea because the function returns successfully. That's actually true for the two ndctl_namespace...() functions above where the return value isn't even checked. If the namespace is left in an unclean state or unusable state, that would be good to know before the user attempts to use the namespace and gets some other error.

For something like a checker, more information is better, which contradicts my first comment. :-)

-- ljk

Quoted 11 lines
Quoted 11 lines

Thanks,

- ljk

ndctl/builtin-check.c line 988

Quoted 4 lines

+ out_bttc:

ndctl/builtin-check.c new file +3 -0
+	free(bttc);
+	return rc;
+}

_______________________________________________
Linux-nvdimm mailing list
[email protected]
https://lists.01.org/mailman/listinfo/linux-nvdimm

author maintainer
#11 lore raw

PATCH v3 7/7 ndctl, test: Add a unit test for the BTT checker

In reply to Vishal Verma

Add a new unit test that will set up BTTs, corrupt them in known ways, and test that the checker is able to detect or repair the corruption in the expected way.

Cc Jeff Moyer <[email protected]>
Cc Dan Williams <[email protected]>
---
 test/Makefile.am  |   5 +-
 test/btt-check.sh | 170 ++++++++++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 174 insertions(+), 1 deletion(-)
 create mode 100755 test/btt-check.sh
test/Makefile.am +4 -1
index 524fafa..73c4463 100644
@@ -8,7 +8,8 @@ TESTS =\
 	multi-pmem \
 	create.sh \
 	clear.sh \
-	dax-errors.sh
+	dax-errors.sh \
+	btt-check.sh
 
 check_PROGRAMS =\
 	libndctl \
@@ -78,6 +79,7 @@ device_dax_SOURCES = \
 		dax-pmd.c \
 		$(testcore) \
 		../ndctl/builtin-xaction-namespace.c \
+		../ndctl/builtin-check.c \
 		../util/json.c
 device_dax_LDADD = \
 		$(LIBNDCTL_LIB) \
@@ -89,6 +91,7 @@ multi_pmem_SOURCES = \
 		multi-pmem.c \
 		$(testcore) \
 		../ndctl/builtin-xaction-namespace.c \
+		../ndctl/builtin-check.c \
 		../util/json.c
 multi_pmem_LDADD = \
 		$(LIBNDCTL_LIB) \
test/btt-check.sh new file +170 -0

170 lines not on this page · raw message

Signature · 6 lines
2.9.3

_______________________________________________
Linux-nvdimm mailing list
[email protected]
https://lists.01.org/mailman/listinfo/linux-nvdimm