Skip to contentSkip to conversation

error: auto propagated local_err part I

PATCH Superseded Vladimir Sementsov-Ogievskiy 7 participants 43 messages Mar 6, 2020 – Mar 12, 2020

Subsystems (2)

v8 of 11 Superseded Latest is v11 →

lore
Share & tools
lore
#0 lore raw

v8 cover error: auto propagated local_err part I

File with errp-cleaning APIs dropped for two reasons:

1. I'm tired after a 3-days war with coccinelle, and don't want to add more patches here.

2. Markus noted, that we forget two more functions which needs such wrappers and corresponding conversion, so seems better to handle all these things in same manner for now.

changes in v8:

01: - update comments
- fix bug in macro [Markus]
- use do {} while(0)

02: a lot of changes
- about error propagation: try to update only patterns where we propagate local_error to errp. So, patches 09 and 10 changed (wow!)
- Now clearing functions are not defined. Still, do the conversion, so that when script applied where these functions needed, compilation will be broken.
- improve ordering of hunks, and comment everything

09,10: drop hunks, which are converted for nothing.

v8 is available at
https://src.openvz.org/scm/~vsementsov/qemu.git #tag up-auto-local-err-partI-v8
v7 is available at
https://src.openvz.org/scm/~vsementsov/qemu.git #tag up-auto-local-err-partI-v7

In these series, there is no commit-per-subsystem script, each generated commit is generated in separate.

Still, generating commands are very similar, and looks like

    sed -n '/^<Subsystem name>$/,/^$/{s/^F: //p}' MAINTAINERS | \
    xargs git ls-files | grep '\.[hc]$' | \
    xargs spatch \
        --sp-file scripts/coccinelle/auto-propagated-errp.cocci \
        --macro-file scripts/cocci-macro-file.h \
        --in-place --no-show-diff --max-width 80

Note, that in each generated commit, generation command is the only text, indented by 8 spaces in 'git log -1' output, so, to regenerate all commits (for example, after rebase, or change in coccinelle script), you may use the following command:

git rebase -x "sh -c \"git show --pretty= --name-only | xargs git checkout HEAD^ -- ; git reset; git log -1 | grep '^ ' | sh\"" HEAD~7

Which will start automated interactive rebase for generated patches, which will stop if generated patch changed
(you may do git commit --amend to apply updated generated changes).

Note:
git show --pretty= --name-only - lists files, changed in HEAD
git log -1 | grep '^ ' | sh - rerun generation command of HEAD

Check for compilation of changed .c files
git rebase -x "sh -c \"git show --pretty= --name-only | sed -n 's/\.c$/.o/p' | xargs make -j9\"" HEAD~7

Vladimir Sementsov-Ogievskiy (10):
error: auto propagated local_err
scripts: add coccinelle script to use auto propagated errp hw/sd/ssi-sd: fix error handling in ssi_sd_realize SD (Secure Card): introduce ERRP_AUTO_PROPAGATE
pflash: introduce ERRP_AUTO_PROPAGATE
fw_cfg: introduce ERRP_AUTO_PROPAGATE
virtio-9p: introduce ERRP_AUTO_PROPAGATE
TPM: introduce ERRP_AUTO_PROPAGATE
nbd: introduce ERRP_AUTO_PROPAGATE
xen: introduce ERRP_AUTO_PROPAGATE

 include/block/nbd.h                           |   1 +
 include/qapi/error.h                          | 206 +++++++++++++---
 block/nbd.c                                   |  21 +-
 hw/9pfs/9p-local.c                            |  12 +-
 hw/9pfs/9p.c                                  |   1 +
 hw/block/dataplane/xen-block.c                |  17 +-
 hw/block/pflash_cfi01.c                       |   7 +-
 hw/block/pflash_cfi02.c                       |   7 +-
 hw/block/xen-block.c                          | 125 ++++------
 hw/nvram/fw_cfg.c                             |  14 +-
 hw/pci-host/xen_igd_pt.c                      |   7 +-
 hw/sd/sdhci-pci.c                             |   7 +-
 hw/sd/sdhci.c                                 |  21 +-
 hw/sd/ssi-sd.c                                |  26 +-
 hw/tpm/tpm_util.c                             |   7 +-
 hw/xen/xen-backend.c                          |   7 +-
 hw/xen/xen-bus.c                              |  92 +++----
 hw/xen/xen-host-pci-device.c                  |  27 +-
 hw/xen/xen_pt.c                               |  25 +-
 hw/xen/xen_pt_config_init.c                   |  20 +-
 nbd/client.c                                  |   5 +
 nbd/server.c                                  |   5 +
 tpm.c                                         |   7 +-
 scripts/coccinelle/auto-propagated-errp.cocci | 231 ++++++++++++++++++
 24 files changed, 617 insertions(+), 281 deletions(-)
 create mode 100644 scripts/coccinelle/auto-propagated-errp.cocci
Cc Eric Blake <[email protected]>
Cc Kevin Wolf <[email protected]>
Cc Max Reitz <[email protected]>
Cc Greg Kurz <[email protected]>
Cc Christian Schoenebeck <[email protected]>
Cc Stefano Stabellini <[email protected]>
Cc Anthony Perard <[email protected]>
Cc Paul Durrant <[email protected]>
Cc Stefan Hajnoczi <[email protected]>
Cc "Philippe Mathieu-Daudé" <[email protected]>
Cc Laszlo Ersek <[email protected]>
Cc Gerd Hoffmann <[email protected]>
Cc Stefan Berger <[email protected]>
Cc Markus Armbruster <[email protected]>
Cc Michael Roth <[email protected]>
Signature · 7 lines
2.21.0


_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
#1 lore raw

PATCH v8 1/10 error: auto propagated local_err

Here is introduced ERRP_AUTO_PROPAGATE macro, to be used at start of functions with an errp OUT parameter.

It has three goals:

1. Fix issue with error_fatal and error_prepend/error_append_hint: user can't see this additional information, because exit() happens in error_setg earlier than information is added. [Reported by Greg Kurz]

2. Fix issue with error_abort and error_propagate: when we wrap error_abort by local_err+error_propagate, the resulting coredump will refer to error_propagate and not to the place where error happened. (the macro itself doesn't fix the issue, but it allows us to [3.] drop the local_err+error_propagate pattern, which will definitely fix the issue) [Reported by Kevin Wolf]

3. Drop local_err+error_propagate pattern, which is used to workaround void functions with errp parameter, when caller wants to know resulting status. (Note: actually these functions could be merely updated to return int error code).

To achieve these goals, later patches will add invocations of this macro at the start of functions with either use error_prepend/error_append_hint (solving 1) or which use local_err+error_propagate to check errors, switching those functions to use *errp instead (solving 2 and 3).

---

Cc Eric Blake <[email protected]>
Cc Kevin Wolf <[email protected]>
Cc Max Reitz <[email protected]>
Cc Greg Kurz <[email protected]>
Cc Christian Schoenebeck <[email protected]>
Cc Stefano Stabellini <[email protected]>
Cc Anthony Perard <[email protected]>
Cc Paul Durrant <[email protected]>
Cc Stefan Hajnoczi <[email protected]>
Cc "Philippe Mathieu-Daudé" <[email protected]>
Cc Laszlo Ersek <[email protected]>
Cc Gerd Hoffmann <[email protected]>
Cc Stefan Berger <[email protected]>
Cc Markus Armbruster <[email protected]>
Cc Michael Roth <[email protected]>
 include/qapi/error.h | 203 ++++++++++++++++++++++++++++++++++++-------
 1 file changed, 170 insertions(+), 33 deletions(-)
include/qapi/error.h +170 -33
index ad5b6e896d..bb9bcf02fb 100644
@@ -15,6 +15,8 @@
 /*
  * Error reporting system loosely patterned after Glib's GError.
  *
+ * = Deal with Error object =
+ *
  * Create an error:
  *     error_setg(&err, "situation normal, all fouled up");
  *
@@ -47,28 +49,88 @@
  * reporting it (primarily useful in testsuites):
  *     error_free_or_abort(&err);
  *
- * Pass an existing error to the caller:
- *     error_propagate(errp, err);
- * where Error **errp is a parameter, by convention the last one.
+ * = Deal with Error ** function parameter =
  *
- * Pass an existing error to the caller with the message modified:
- *     error_propagate_prepend(errp, err);
+ * Function may use error system to return errors. In this case function
+ * defines Error **errp parameter, which should be the last one (except for
+ * functions which varidic argument list), which has the following API:
  *
- * Avoid
- *     error_propagate(errp, err);
- *     error_prepend(errp, "Could not frobnicate '%s': ", name);
- * because this fails to prepend when @errp is &error_fatal.
+ * Caller may pass as errp:
+ * 1. &error_abort
+ *    This means abort on any error
+ * 2. &error_fatal
+ *    Exit with non-zero return code on error
+ * 3. NULL
+ *    Ignore errors
+ * 4. Another value
+ *    On error allocate error object and set errp
  *
- * Create a new error and pass it to the caller:
- *     error_setg(errp, "situation normal, all fouled up");
+ * Error API functions with Error ** (like error_setg) argument supports these
+ * rules, so user functions just need to use them appropriately (read below).
  *
- * Call a function and receive an error from it:
+ * Simple pass error to the caller:
+ *     error_setg(errp, "Some error");
+ *
+ * Subcall of another errp-based function, passing the error to the caller
+ *     f(..., errp);
+ *
+ * == Checking success of subcall ==
+ *
+ * If function returns error code in addition to errp (which is recommended),
+ * you don't need any additional code, just do:
+ *     int ret = f(..., errp);
+ *     if (ret < 0) {
+ *         ... handle error ...
+ *         return ret;
+ *     }
+ *
+ * If function returns nothing (which is not recommended API) and the only way
+ * to check success is checking errp, we must care about cases [1-3] above. We
+ * need to use macro ERRP_AUTO_PROPAGATE (see below for details) like this:
+ *
+ *     int our_func(..., Error **errp) {
+ *         ERRP_AUTO_PROPAGATE();
+ *         ...
+ *         subcall(..., errp);
+ *         if (*errp) {
+ *             ...
+ *             return -ERRNO;
+ *         }
+ *         ...
+ *     }
+ *
+ * ERRP_AUTO_PROPAGATE cares about Error ** API, wraps original errp if needed,
+ * so that it can be safely used (including dereferencing), and auto-propagates
+ * error to original errp on function end.
+ *
+ * In some cases, we need to check result of subcall, but do not want to
+ * propagate the Error object to our caller. In such cases we don't need
+ * ERRP_AUTO_PROPAGATE, but just a local Error object:
+ *
+ * Receive an error and not pass it:
  *     Error *err = NULL;
- *     foo(arg, &err);
+ *     subcall(arg, &err);
  *     if (err) {
  *         handle the error...
+ *         error_free(err);
  *     }
  *
+ * Note, that before ERRP_AUTO_PROPAGATE introduction the pattern above (with
+ * error_propagate() instead of error_free()) was used to check and pass error
+ * to the caller. Now this is DEPRECATED* (see below).
+ *
+ * Note also, that if you want to use error_append_hint/error_prepend or their
+ * variants, you must use ERRP_AUTO_PROPAGATE too. Otherwise, in case of
+ * error_fatal, you'll miss the chance to insert your additional information
+ * into Error object.
+ *
+ * In rare cases, we need to pass existing Error object to the caller by hand:
+ *     error_propagate(errp, err);
+ *
+ * Pass an existing error to the caller with the message modified:
+ *     error_propagate_prepend(errp, err);
+ *
+ *
  * Call a function ignoring errors:
  *     foo(arg, NULL);
  *
@@ -78,26 +140,6 @@
  * Call a function treating errors as fatal:
  *     foo(arg, &error_fatal);
  *
- * Receive an error and pass it on to the caller:
- *     Error *err = NULL;
- *     foo(arg, &err);
- *     if (err) {
- *         handle the error...
- *         error_propagate(errp, err);
- *     }
- * where Error **errp is a parameter, by convention the last one.
- *
- * Do *not* "optimize" this to
- *     foo(arg, errp);
- *     if (*errp) { // WRONG!
- *         handle the error...
- *     }
- * because errp may be NULL!
- *
- * But when all you do with the error is pass it on, please use
- *     foo(arg, errp);
- * for readability.
- *
  * Receive and accumulate multiple errors (first one wins):
  *     Error *err = NULL, *local_err = NULL;
  *     foo(arg, &err);
@@ -114,6 +156,61 @@
  *         handle the error...
  *     }
  * because this may pass a non-null err to bar().
+ *
+ * DEPRECATED*
+ *
+ * The following pattern of receiving checking and passing the caller of the
+ * error by hand is deprecated now:
+ *
+ *     Error *err = NULL;
+ *     foo(arg, &err);
+ *     if (err) {
+ *         handle the error...
+ *         error_propagate(errp, err);
+ *     }
+ *
+ * Instead, use ERRP_AUTO_PROPAGATE macro (defined below).
+ *
+ * The old pattern is deprecated because of two things:
+ *
+ * 1. Issue with error_abort and error_propagate: when we wrap error_abort by
+ * local_err+error_propagate, the resulting coredump will refer to
+ * error_propagate and not to the place where error happened.
+ *
+ * 2. A lot of extra code of the same pattern
+ *
+ * How to update old code to use ERRP_AUTO_PROPAGATE?
+ *
+ * All you need is to add ERRP_AUTO_PROPAGATE() invocation at function start,
+ * than you may safely dereference errp to check errors and do not need any
+ * additional local Error variables or calls to error_propagate().
+ *
+ * Example:
+ *
+ * old code
+ *
+ *     void fn(..., Error **errp) {
+ *         Error *err = NULL;
+ *         foo(arg, &err);
+ *         if (err) {
+ *             handle the error...
+ *             error_propagate(errp, err);
+ *             return;
+ *         }
+ *         ...
+ *     }
+ *
+ * updated code
+ *
+ *     void fn(..., Error **errp) {
+ *         ERRP_AUTO_PROPAGATE();
+ *         foo(arg, errp);
+ *         if (*errp) {
+ *             handle the error...
+ *             return;
+ *         }
+ *         ...
+ *     }
  */
 
 #ifndef ERROR_H
@@ -322,6 +419,46 @@ void error_set_internal(Error **errp,
                         ErrorClass err_class, const char *fmt, ...)
     GCC_FMT_ATTR(6, 7);
 
+typedef struct ErrorPropagator {
+    Error *local_err;
+    Error **errp;
+} ErrorPropagator;
+
+static inline void error_propagator_cleanup(ErrorPropagator *prop)
+{
+    error_propagate(prop->errp, prop->local_err);
+}
+
+G_DEFINE_AUTO_CLEANUP_CLEAR_FUNC(ErrorPropagator, error_propagator_cleanup);
+
+/*
+ * ERRP_AUTO_PROPAGATE
+ *
+ * This macro is created to be the first line of a function which use
+ * Error **errp parameter to report error. It's needed only in cases where we
+ * want to use error_prepend, error_append_hint or dereference *errp. It's
+ * still safe (but useless) in other cases.
+ *
+ * If errp is NULL or points to error_fatal, it is rewritten to point to a
+ * local Error object, which will be automatically propagated to the original
+ * errp on function exit (see error_propagator_cleanup).
+ *
+ * After invocation of this macro it is always safe to dereference errp
+ * (as it's not NULL anymore) and to add information by error_prepend or
+ * error_append_hint (as, if it was error_fatal, we swapped it with a
+ * local_error to be propagated on cleanup).
+ *
+ * Note: we don't wrap the error_abort case, as we want resulting coredump
+ * to point to the place where the error happened, not to error_propagate.
+ */
+#define ERRP_AUTO_PROPAGATE() \
+    g_auto(ErrorPropagator) _auto_errp_prop = {.errp = errp}; \
+    do { \
+        if (!errp || errp == &error_fatal) { \
+            errp = &_auto_errp_prop.local_err; \
+        } \
+    } while (0)
+
 /*
  * Special error destination to abort on error.
  * See error_setg() and error_propagate() for details.
Signature · 7 lines
2.21.0


_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
Reviewed-by on v8 1/10
#2 lore raw

Re: [PATCH v8 01/10] error: auto propagated local_err

Quoted 22 lines · Vladimir Sementsov-Ogievskiy

Here is introduced ERRP_AUTO_PROPAGATE macro, to be used at start of functions with an errp OUT parameter.

It has three goals:

1. Fix issue with error_fatal and error_prepend/error_append_hint: user can't see this additional information, because exit() happens in error_setg earlier than information is added. [Reported by Greg Kurz]

2. Fix issue with error_abort and error_propagate: when we wrap error_abort by local_err+error_propagate, the resulting coredump will refer to error_propagate and not to the place where error happened. (the macro itself doesn't fix the issue, but it allows us to [3.] drop the local_err+error_propagate pattern, which will definitely fix the issue) [Reported by Kevin Wolf]

3. Drop local_err+error_propagate pattern, which is used to workaround void functions with errp parameter, when caller wants to know resulting status. (Note: actually these functions could be merely updated to return int error code).

Quoted 7 lines

To achieve these goals, later patches will add invocations of this macro at the start of functions with either use error_prepend/error_append_hint (solving 1) or which use local_err+error_propagate to check errors, switching those functions to use *errp instead (solving 2 and 3).

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

Reviewed-by on v8 1/10
#3 lore raw

Re: [PATCH v8 01/10] error: auto propagated local_err

In reply to Vladimir Sementsov-Ogievskiy
Quoted 21 lines · On Fri, 6 Mar 2020 08:15:27 +0300 Vladimir Sementsov-Ogievskiy…

Here is introduced ERRP_AUTO_PROPAGATE macro, to be used at start of functions with an errp OUT parameter.

It has three goals:

1. Fix issue with error_fatal and error_prepend/error_append_hint: user can't see this additional information, because exit() happens in error_setg earlier than information is added. [Reported by Greg Kurz]

2. Fix issue with error_abort and error_propagate: when we wrap error_abort by local_err+error_propagate, the resulting coredump will refer to error_propagate and not to the place where error happened. (the macro itself doesn't fix the issue, but it allows us to [3.] drop the local_err+error_propagate pattern, which will definitely fix the issue) [Reported by Kevin Wolf]

3. Drop local_err+error_propagate pattern, which is used to workaround void functions with errp parameter, when caller wants to know resulting status. (Note: actually these functions could be merely updated to return int error code).

Quoted 9 lines

To achieve these goals, later patches will add invocations of this macro at the start of functions with either use error_prepend/error_append_hint (solving 1) or which use local_err+error_propagate to check errors, switching those functions to use *errp instead (solving 2 and 3).

---

Thanks for this impressive work Vladimir !

Quoted 263 lines
Cc Eric Blake <[email protected]>
Cc Kevin Wolf <[email protected]>
Cc Max Reitz <[email protected]>
Cc Greg Kurz <[email protected]>
Cc Christian Schoenebeck <[email protected]>
Cc Stefano Stabellini <[email protected]>
Cc Anthony Perard <[email protected]>
Cc Paul Durrant <[email protected]>
Cc Stefan Hajnoczi <[email protected]>
Cc "Philippe Mathieu-Daudé" <[email protected]>
Cc Laszlo Ersek <[email protected]>
Cc Gerd Hoffmann <[email protected]>
Cc Stefan Berger <[email protected]>
Cc Markus Armbruster <[email protected]>
Cc Michael Roth <[email protected]>
 include/qapi/error.h | 203 ++++++++++++++++++++++++++++++++++++-------
 1 file changed, 170 insertions(+), 33 deletions(-)
include/qapi/error.h +163 -33
index ad5b6e896d..bb9bcf02fb 100644
@@ -15,6 +15,8 @@
 /*
  * Error reporting system loosely patterned after Glib's GError.
  *
+ * = Deal with Error object =
+ *
  * Create an error:
  *     error_setg(&err, "situation normal, all fouled up");
  *
@@ -47,28 +49,88 @@
  * reporting it (primarily useful in testsuites):
  *     error_free_or_abort(&err);
  *
- * Pass an existing error to the caller:
- *     error_propagate(errp, err);
- * where Error **errp is a parameter, by convention the last one.
+ * = Deal with Error ** function parameter =
  *
- * Pass an existing error to the caller with the message modified:
- *     error_propagate_prepend(errp, err);
+ * Function may use error system to return errors. In this case function
+ * defines Error **errp parameter, which should be the last one (except for
+ * functions which varidic argument list), which has the following API:
  *
- * Avoid
- *     error_propagate(errp, err);
- *     error_prepend(errp, "Could not frobnicate '%s': ", name);
- * because this fails to prepend when @errp is &error_fatal.
+ * Caller may pass as errp:
+ * 1. &error_abort
+ *    This means abort on any error
+ * 2. &error_fatal
+ *    Exit with non-zero return code on error
+ * 3. NULL
+ *    Ignore errors
+ * 4. Another value
+ *    On error allocate error object and set errp
  *
- * Create a new error and pass it to the caller:
- *     error_setg(errp, "situation normal, all fouled up");
+ * Error API functions with Error ** (like error_setg) argument supports these
+ * rules, so user functions just need to use them appropriately (read below).
  *
- * Call a function and receive an error from it:
+ * Simple pass error to the caller:
+ *     error_setg(errp, "Some error");
+ *
+ * Subcall of another errp-based function, passing the error to the caller
+ *     f(..., errp);
+ *
+ * == Checking success of subcall ==
+ *
+ * If function returns error code in addition to errp (which is recommended),
+ * you don't need any additional code, just do:
+ *     int ret = f(..., errp);
+ *     if (ret < 0) {
+ *         ... handle error ...
+ *         return ret;
+ *     }
+ *
+ * If function returns nothing (which is not recommended API) and the only way
+ * to check success is checking errp, we must care about cases [1-3] above. We
+ * need to use macro ERRP_AUTO_PROPAGATE (see below for details) like this:
+ *
+ *     int our_func(..., Error **errp) {
+ *         ERRP_AUTO_PROPAGATE();
+ *         ...
+ *         subcall(..., errp);
+ *         if (*errp) {
+ *             ...
+ *             return -ERRNO;
+ *         }
+ *         ...
+ *     }
+ *
+ * ERRP_AUTO_PROPAGATE cares about Error ** API, wraps original errp if needed,
+ * so that it can be safely used (including dereferencing), and auto-propagates
+ * error to original errp on function end.
+ *
+ * In some cases, we need to check result of subcall, but do not want to
+ * propagate the Error object to our caller. In such cases we don't need
+ * ERRP_AUTO_PROPAGATE, but just a local Error object:
+ *
+ * Receive an error and not pass it:
  *     Error *err = NULL;
- *     foo(arg, &err);
+ *     subcall(arg, &err);
  *     if (err) {
  *         handle the error...
+ *         error_free(err);
  *     }
  *
+ * Note, that before ERRP_AUTO_PROPAGATE introduction the pattern above (with
+ * error_propagate() instead of error_free()) was used to check and pass error
+ * to the caller. Now this is DEPRECATED* (see below).
+ *
+ * Note also, that if you want to use error_append_hint/error_prepend or their
+ * variants, you must use ERRP_AUTO_PROPAGATE too. Otherwise, in case of
+ * error_fatal, you'll miss the chance to insert your additional information
+ * into Error object.
+ *
+ * In rare cases, we need to pass existing Error object to the caller by hand:
+ *     error_propagate(errp, err);
+ *
+ * Pass an existing error to the caller with the message modified:
+ *     error_propagate_prepend(errp, err);
+ *
+ *
  * Call a function ignoring errors:
  *     foo(arg, NULL);
  *
@@ -78,26 +140,6 @@
  * Call a function treating errors as fatal:
  *     foo(arg, &error_fatal);
  *
- * Receive an error and pass it on to the caller:
- *     Error *err = NULL;
- *     foo(arg, &err);
- *     if (err) {
- *         handle the error...
- *         error_propagate(errp, err);
- *     }
- * where Error **errp is a parameter, by convention the last one.
- *
- * Do *not* "optimize" this to
- *     foo(arg, errp);
- *     if (*errp) { // WRONG!
- *         handle the error...
- *     }
- * because errp may be NULL!
- *
- * But when all you do with the error is pass it on, please use
- *     foo(arg, errp);
- * for readability.
- *
  * Receive and accumulate multiple errors (first one wins):
  *     Error *err = NULL, *local_err = NULL;
  *     foo(arg, &err);
@@ -114,6 +156,61 @@
  *         handle the error...
  *     }
  * because this may pass a non-null err to bar().
+ *
+ * DEPRECATED*
+ *
+ * The following pattern of receiving checking and passing the caller of the
+ * error by hand is deprecated now:
+ *
+ *     Error *err = NULL;
+ *     foo(arg, &err);
+ *     if (err) {
+ *         handle the error...
+ *         error_propagate(errp, err);
+ *     }
+ *
+ * Instead, use ERRP_AUTO_PROPAGATE macro (defined below).
+ *
+ * The old pattern is deprecated because of two things:
+ *
+ * 1. Issue with error_abort and error_propagate: when we wrap error_abort by
+ * local_err+error_propagate, the resulting coredump will refer to
+ * error_propagate and not to the place where error happened.
+ *
+ * 2. A lot of extra code of the same pattern
+ *
+ * How to update old code to use ERRP_AUTO_PROPAGATE?
+ *
+ * All you need is to add ERRP_AUTO_PROPAGATE() invocation at function start,
+ * than you may safely dereference errp to check errors and do not need any
+ * additional local Error variables or calls to error_propagate().
+ *
+ * Example:
+ *
+ * old code
+ *
+ *     void fn(..., Error **errp) {
+ *         Error *err = NULL;
+ *         foo(arg, &err);
+ *         if (err) {
+ *             handle the error...
+ *             error_propagate(errp, err);
+ *             return;
+ *         }
+ *         ...
+ *     }
+ *
+ * updated code
+ *
+ *     void fn(..., Error **errp) {
+ *         ERRP_AUTO_PROPAGATE();
+ *         foo(arg, errp);
+ *         if (*errp) {
+ *             handle the error...
+ *             return;
+ *         }
+ *         ...
+ *     }
  */
 
 #ifndef ERROR_H
@@ -322,6 +419,46 @@ void error_set_internal(Error **errp,
                         ErrorClass err_class, const char *fmt, ...)
     GCC_FMT_ATTR(6, 7);
 
+typedef struct ErrorPropagator {
+    Error *local_err;
+    Error **errp;
+} ErrorPropagator;
+
+static inline void error_propagator_cleanup(ErrorPropagator *prop)
+{
+    error_propagate(prop->errp, prop->local_err);
+}
+
+G_DEFINE_AUTO_CLEANUP_CLEAR_FUNC(ErrorPropagator, error_propagator_cleanup);
+
+/*
+ * ERRP_AUTO_PROPAGATE
+ *
+ * This macro is created to be the first line of a function which use
+ * Error **errp parameter to report error. It's needed only in cases where we
+ * want to use error_prepend, error_append_hint or dereference *errp. It's
+ * still safe (but useless) in other cases.
+ *
+ * If errp is NULL or points to error_fatal, it is rewritten to point to a
+ * local Error object, which will be automatically propagated to the original
+ * errp on function exit (see error_propagator_cleanup).
+ *
+ * After invocation of this macro it is always safe to dereference errp
+ * (as it's not NULL anymore) and to add information by error_prepend or
+ * error_append_hint (as, if it was error_fatal, we swapped it with a
+ * local_error to be propagated on cleanup).
+ *
+ * Note: we don't wrap the error_abort case, as we want resulting coredump
+ * to point to the place where the error happened, not to error_propagate.
+ */
+#define ERRP_AUTO_PROPAGATE() \

include/qapi/error.h void error_set_internal(Error **errp, · line 464

Quoted 10 lines
include/qapi/error.h +7 -0
+    g_auto(ErrorPropagator) _auto_errp_prop = {.errp = errp}; \
+    do { \
+        if (!errp || errp == &error_fatal) { \
+            errp = &_auto_errp_prop.local_err; \
+        } \
+    } while (0)
+
 /*
  * Special error destination to abort on error.
  * See error_setg() and error_propagate() for details.

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

#4 lore raw

Re: [PATCH v8 01/10] error: auto propagated local_err

In reply to Vladimir Sementsov-Ogievskiy

On Fri 06 Mar 2020 06:15:27 AM CET, Vladimir Sementsov-Ogievskiy wrote:

Sorry I just gave a quick look at these patches and noticed this:

Quoted 3 lines

+ * Function may use error system to return errors. In this case function
+ * defines Error **errp parameter, which should be the last one (except for
+ * functions which varidic argument list), which has the following API:

I don't think that "functions which varidic argument list" is correct English. Perhaps something like "except for variadic functions" or "except for functions with a variable number of arguments".

Berto

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

Reviewed-by on v8 1/10
#5 lore raw

Re: [PATCH v8 01/10] error: auto propagated local_err

In reply to Vladimir Sementsov-Ogievskiy
Quoted 2 lines · Vladimir Sementsov-Ogievskiy

Here is introduced ERRP_AUTO_PROPAGATE macro, to be used at start of functions with an errp OUT parameter.

As an aid to writing imperative-style commit messages, I like to prepend an implicit "Apply this patch to..." before the user's text, to see if things still make sense. By that construct, this paragraph might read better as:

Introduce a new ERRP_AUTO_PROPAGATE macro, ...

Quoted 19 lines

It has three goals:

1. Fix issue with error_fatal and error_prepend/error_append_hint: user can't see this additional information, because exit() happens in error_setg earlier than information is added. [Reported by Greg Kurz]

2. Fix issue with error_abort and error_propagate: when we wrap error_abort by local_err+error_propagate, the resulting coredump will refer to error_propagate and not to the place where error happened. (the macro itself doesn't fix the issue, but it allows us to [3.] drop the local_err+error_propagate pattern, which will definitely fix the issue) [Reported by Kevin Wolf]

3. Drop local_err+error_propagate pattern, which is used to workaround void functions with errp parameter, when caller wants to know resulting status. (Note: actually these functions could be merely updated to return int error code).

Quoted 8 lines

To achieve these goals, later patches will add invocations of this macro at the start of functions with either use error_prepend/error_append_hint (solving 1) or which use local_err+error_propagate to check errors, switching those functions to use *errp instead (solving 2 and 3).

---

I have lots of grammar suggestions for the comments (and I know Markus is generally okay doing wording tweaks, so it may still end up different than my suggestions):

Quoted 14 lines

+++ b/include/qapi/error.h

include/qapi/error.h +2 -0
@@ -15,6 +15,8 @@
  /*
   * Error reporting system loosely patterned after Glib's GError.
   *
+ * = Deal with Error object =
+ *
   * Create an error:
   *     error_setg(&err, "situation normal, all fouled up");
   *
@@ -47,28 +49,88 @@
   * reporting it (primarily useful in testsuites):
   *     error_free_or_abort(&err);
   *

include/qapi/error.h line 56

Quoted 10 lines
include/qapi/error.h +4 -5
- * Pass an existing error to the caller:
- *     error_propagate(errp, err);
- * where Error **errp is a parameter, by convention the last one.
+ * = Deal with Error ** function parameter =
   *
- * Pass an existing error to the caller with the message modified:
- *     error_propagate_prepend(errp, err);
+ * Function may use error system to return errors. In this case function
+ * defines Error **errp parameter, which should be the last one (except for
+ * functions which varidic argument list), which has the following API:

A function may use the error system to return errors. In this case, the function defines an Error **errp parameter, by convention the last one (with exceptions for functions using ... or va_list).

include/qapi/error.h line 58

Quoted 6 lines
include/qapi/error.h +1 -4
   *
- * Avoid
- *     error_propagate(errp, err);
- *     error_prepend(errp, "Could not frobnicate '%s': ", name);
- * because this fails to prepend when @errp is &error_fatal.
+ * Caller may pass as errp:

The caller may then pass in the following errp values:

include/qapi/error.h line 60

Quoted 2 lines
include/qapi/error.h +2 -0
+ * 1. &error_abort
+ *    This means abort on any error

Any error will result in abort()

include/qapi/error.h line 62

Quoted 2 lines
include/qapi/error.h +2 -0
+ * 2. &error_fatal
+ *    Exit with non-zero return code on error

Any error will result in exit() with a non-zero status

include/qapi/error.h line 64

Quoted 2 lines
include/qapi/error.h +2 -0
+ * 3. NULL
+ *    Ignore errors

Any error will be ignored

include/qapi/error.h line 65

Quoted 1 line
include/qapi/error.h +1 -0
+ * 4. Another value

4. The address of a NULL-initialized Error *err

include/qapi/error.h line 66

Quoted 1 line
include/qapi/error.h +1 -0
+ *    On error allocate error object and set errp

Any error will populate errp with an error object

include/qapi/error.h line 69

Quoted 5 lines
include/qapi/error.h +2 -2
   *
- * Create a new error and pass it to the caller:
- *     error_setg(errp, "situation normal, all fouled up");
+ * Error API functions with Error ** (like error_setg) argument supports these
+ * rules, so user functions just need to use them appropriately (read below).

The following rules then implement the correct semantics desired by the caller.

include/qapi/error.h line 71

Quoted 3 lines
include/qapi/error.h +1 -1
   *
- * Call a function and receive an error from it:
+ * Simple pass error to the caller:

Create a new error to pass to the caller:

include/qapi/error.h line 72

Quoted 1 line
include/qapi/error.h +1 -0
+ *     error_setg(errp, "Some error");

You lost the fun wording in Markus' earlier example ("situation normal, all fouled up").

include/qapi/error.h line 75

Quoted 3 lines
include/qapi/error.h +3 -0
+ *
+ * Subcall of another errp-based function, passing the error to the caller
+ *     f(..., errp);

Calling another errp-based function:

include/qapi/error.h line 79

Quoted 4 lines
include/qapi/error.h +4 -0
+ *
+ * == Checking success of subcall ==
+ *
+ * If function returns error code in addition to errp (which is recommended),

If a function returns a value indicating an error in addition to setting errp (which is recommended), then

include/qapi/error.h line 89

Quoted 10 lines
include/qapi/error.h +10 -0
+ * you don't need any additional code, just do:
+ *     int ret = f(..., errp);
+ *     if (ret < 0) {
+ *         ... handle error ...
+ *         return ret;
+ *     }
+ *
+ * If function returns nothing (which is not recommended API) and the only way
+ * to check success is checking errp, we must care about cases [1-3] above. We
+ * need to use macro ERRP_AUTO_PROPAGATE (see below for details) like this:

If a function returns nothing (not recommended for new code), the only way to check success is by consulting errp; doing this safely requires the use of the ERRP_AUTO_PROPAGATE macro, like this:

include/qapi/error.h line 97

Quoted 8 lines
include/qapi/error.h +8 -0
+ *
+ *     int our_func(..., Error **errp) {
+ *         ERRP_AUTO_PROPAGATE();
+ *         ...
+ *         subcall(..., errp);
+ *         if (*errp) {
+ *             ...
+ *             return -ERRNO;

do we want ERRNO capitalized here?

include/qapi/error.h line 104

Quoted 7 lines
include/qapi/error.h +7 -0
+ *         }
+ *         ...
+ *     }
+ *
+ * ERRP_AUTO_PROPAGATE cares about Error ** API, wraps original errp if needed,
+ * so that it can be safely used (including dereferencing), and auto-propagates
+ * error to original errp on function end.

ERRP_AUTO_PROPAGATE takes care of wrapping the original errp as needed, so that the rest of the function can directly use errp (including dereferencing), where any errors will then be propagated on to the original errp when leaving the function.

Quoted 7 lines
include/qapi/error.h +6 -0
+ *
+ * In some cases, we need to check result of subcall, but do not want to
+ * propagate the Error object to our caller. In such cases we don't need
+ * ERRP_AUTO_PROPAGATE, but just a local Error object:
+ *
+ * Receive an error and not pass it:
   *     Error *err = NULL;

include/qapi/error.h line 120

Quoted 10 lines
include/qapi/error.h +5 -1
- *     foo(arg, &err);
+ *     subcall(arg, &err);
   *     if (err) {
   *         handle the error...
+ *         error_free(err);
   *     }
   *
+ * Note, that before ERRP_AUTO_PROPAGATE introduction the pattern above (with
+ * error_propagate() instead of error_free()) was used to check and pass error
+ * to the caller. Now this is DEPRECATED* (see below).

Hmm - if we bulk-convert the entire tree, then there won't be any deprecated uses to be worth documenting. But if we do keep this paragraph:

Note that older code that did not use ERRP_AUTO_PROPAGATE would instead need a local Err variable and the use of error_propagate() to properly handle all possible caller values of errp.

include/qapi/error.h line 125

Quoted 5 lines
include/qapi/error.h +5 -0
+ *
+ * Note also, that if you want to use error_append_hint/error_prepend or their
+ * variants, you must use ERRP_AUTO_PROPAGATE too. Otherwise, in case of
+ * error_fatal, you'll miss the chance to insert your additional information
+ * into Error object.

Note that any function that wants to modify an error object, such as by calling error_append_hint or error_prepend, must use ERRP_AUTO_PROPAGATE, in order for a caller's use of &error_fatal to see the additional information.

Quoted 39 lines
include/qapi/error.h +8 -20
+ *
+ * In rare cases, we need to pass existing Error object to the caller by hand:
+ *     error_propagate(errp, err);
+ *
+ * Pass an existing error to the caller with the message modified:
+ *     error_propagate_prepend(errp, err);
+ *
+ *
   * Call a function ignoring errors:
   *     foo(arg, NULL);
   *
@@ -78,26 +140,6 @@
   * Call a function treating errors as fatal:
   *     foo(arg, &error_fatal);
   *
- * Receive an error and pass it on to the caller:
- *     Error *err = NULL;
- *     foo(arg, &err);
- *     if (err) {
- *         handle the error...
- *         error_propagate(errp, err);
- *     }
- * where Error **errp is a parameter, by convention the last one.
- *
- * Do *not* "optimize" this to
- *     foo(arg, errp);
- *     if (*errp) { // WRONG!
- *         handle the error...
- *     }
- * because errp may be NULL!
- *
- * But when all you do with the error is pass it on, please use
- *     foo(arg, errp);
- * for readability.
- *
   * Receive and accumulate multiple errors (first one wins):
   *     Error *err = NULL, *local_err = NULL;
   *     foo(arg, &err);

include/qapi/error.h line 161

Quoted 6 lines
include/qapi/error.h +3 -0
@@ -114,6 +156,61 @@
   *         handle the error...
   *     }
   * because this may pass a non-null err to bar().
+ *
+ * DEPRECATED*
+ *

Again, I'm not sure we need this section in the codebase if we do a bulk conversion. But moving it to the commit message is still useful.

include/qapi/error.h line 163

Quoted 2 lines
include/qapi/error.h +2 -0
+ * The following pattern of receiving checking and passing the caller of the
+ * error by hand is deprecated now:

The following pattern of receiving, checking, and then forwarding an error to the caller by hand is now deprecated:

include/qapi/error.h line 172

Quoted 9 lines
include/qapi/error.h +9 -0
+ *
+ *     Error *err = NULL;
+ *     foo(arg, &err);
+ *     if (err) {
+ *         handle the error...
+ *         error_propagate(errp, err);
+ *     }
+ *
+ * Instead, use ERRP_AUTO_PROPAGATE macro (defined below).

Drop "(defined below)".

Quoted 31 lines
include/qapi/error.h +31 -0
+ *
+ * The old pattern is deprecated because of two things:
+ *
+ * 1. Issue with error_abort and error_propagate: when we wrap error_abort by
+ * local_err+error_propagate, the resulting coredump will refer to
+ * error_propagate and not to the place where error happened.
+ *
+ * 2. A lot of extra code of the same pattern
+ *
+ * How to update old code to use ERRP_AUTO_PROPAGATE?
+ *
+ * All you need is to add ERRP_AUTO_PROPAGATE() invocation at function start,
+ * than you may safely dereference errp to check errors and do not need any
+ * additional local Error variables or calls to error_propagate().
+ *
+ * Example:
+ *
+ * old code
+ *
+ *     void fn(..., Error **errp) {
+ *         Error *err = NULL;
+ *         foo(arg, &err);
+ *         if (err) {
+ *             handle the error...
+ *             error_propagate(errp, err);
+ *             return;
+ *         }
+ *         ...
+ *     }
+ *
+ * updated code

include/qapi/error.h line 213

Quoted 10 lines
include/qapi/error.h +10 -0
+ *
+ *     void fn(..., Error **errp) {
+ *         ERRP_AUTO_PROPAGATE();
+ *         foo(arg, errp);
+ *         if (*errp) {
+ *             handle the error...
+ *             return;
+ *         }
+ *         ...
+ *     }

Again, I'm thinking the above example is more useful in the commit message instead of permanently in the .h file.

Quoted 16 lines
include/qapi/error.h +9 -0
   */
 
  #ifndef ERROR_H
@@ -322,6 +419,46 @@ void error_set_internal(Error **errp,
                          ErrorClass err_class, const char *fmt, ...)
      GCC_FMT_ATTR(6, 7);
 
+typedef struct ErrorPropagator {
+    Error *local_err;
+    Error **errp;
+} ErrorPropagator;
+
+static inline void error_propagator_cleanup(ErrorPropagator *prop)
+{
+    error_propagate(prop->errp, prop->local_err);
+}

include/qapi/error.h void error_set_internal(Error **errp, · line 440

Quoted 10 lines
include/qapi/error.h +10 -0
+
+G_DEFINE_AUTO_CLEANUP_CLEAR_FUNC(ErrorPropagator, error_propagator_cleanup);
+
+/*
+ * ERRP_AUTO_PROPAGATE
+ *
+ * This macro is created to be the first line of a function which use
+ * Error **errp parameter to report error. It's needed only in cases where we
+ * want to use error_prepend, error_append_hint or dereference *errp. It's
+ * still safe (but useless) in other cases.

This macro exists to assist with proper error handling in a function which uses an Error **errp parameter. It must be used as the first line of a function which modifies an error (with error_prepend, error_append_hint, or similar) or which wants to dereference *errp. It is still safe (but useless) to use in other functions.

Quoted 15 lines
include/qapi/error.h +14 -0
+ *
+ * If errp is NULL or points to error_fatal, it is rewritten to point to a
+ * local Error object, which will be automatically propagated to the original
+ * errp on function exit (see error_propagator_cleanup).
+ *
+ * After invocation of this macro it is always safe to dereference errp
+ * (as it's not NULL anymore) and to add information by error_prepend or
+ * error_append_hint (as, if it was error_fatal, we swapped it with a
+ * local_error to be propagated on cleanup).
+ *
+ * Note: we don't wrap the error_abort case, as we want resulting coredump
+ * to point to the place where the error happened, not to error_propagate.
+ */
+#define ERRP_AUTO_PROPAGATE() \

include/qapi/error.h void error_set_internal(Error **errp, · line 464

Quoted 10 lines
include/qapi/error.h +7 -0
+    g_auto(ErrorPropagator) _auto_errp_prop = {.errp = errp}; \
+    do { \
+        if (!errp || errp == &error_fatal) { \
+            errp = &_auto_errp_prop.local_err; \
+        } \
+    } while (0)
+
  /*
   * Special error destination to abort on error.
   * See error_setg() and error_propagate() for details.

The macro itself looks correct. I'll leave it up to Markus how to handle the comment text, but you can add:

Signature · 9 lines
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3226
Virtualization:  qemu.org | libvirt.org


_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
#6 lore raw

Re: [PATCH v8 01/10] error: auto propagated local_err

Quoted 151 lines · Eric Blake
Quoted 2 lines · Vladimir Sementsov-Ogievskiy

Here is introduced ERRP_AUTO_PROPAGATE macro, to be used at start of functions with an errp OUT parameter.

As an aid to writing imperative-style commit messages, I like to prepend an implicit "Apply this patch to..." before the user's text, to see if things still make sense.  By that construct, this paragraph might read better as:

Introduce a new ERRP_AUTO_PROPAGATE macro, ...

Quoted 19 lines

It has three goals:

1. Fix issue with error_fatal and error_prepend/error_append_hint: user can't see this additional information, because exit() happens in error_setg earlier than information is added. [Reported by Greg Kurz]

2. Fix issue with error_abort and error_propagate: when we wrap error_abort by local_err+error_propagate, the resulting coredump will refer to error_propagate and not to the place where error happened. (the macro itself doesn't fix the issue, but it allows us to [3.] drop the local_err+error_propagate pattern, which will definitely fix the issue) [Reported by Kevin Wolf]

3. Drop local_err+error_propagate pattern, which is used to workaround void functions with errp parameter, when caller wants to know resulting status. (Note: actually these functions could be merely updated to return int error code).

Quoted 8 lines

To achieve these goals, later patches will add invocations of this macro at the start of functions with either use error_prepend/error_append_hint (solving 1) or which use local_err+error_propagate to check errors, switching those functions to use *errp instead (solving 2 and 3).

---

I have lots of grammar suggestions for the comments (and I know Markus is generally okay doing wording tweaks, so it may still end up different than my suggestions):

Quoted 14 lines

+++ b/include/qapi/error.h

include/qapi/error.h +2 -0
@@ -15,6 +15,8 @@
   /*
    * Error reporting system loosely patterned after Glib's GError.
    *
+ * = Deal with Error object =
+ *
    * Create an error:
    *     error_setg(&err, "situation normal, all fouled up");
    *
@@ -47,28 +49,88 @@
    * reporting it (primarily useful in testsuites):
    *     error_free_or_abort(&err);
    *

include/qapi/error.h line 56

Quoted 10 lines
include/qapi/error.h +4 -5
- * Pass an existing error to the caller:
- *     error_propagate(errp, err);
- * where Error **errp is a parameter, by convention the last one.
+ * = Deal with Error ** function parameter =
    *
- * Pass an existing error to the caller with the message modified:
- *     error_propagate_prepend(errp, err);
+ * Function may use error system to return errors. In this case function
+ * defines Error **errp parameter, which should be the last one (except for
+ * functions which varidic argument list), which has the following API:

A function may use the error system to return errors.  In this case, the function defines an Error **errp parameter, by convention the last one (with exceptions for functions using ... or va_list).

include/qapi/error.h line 58

Quoted 6 lines
include/qapi/error.h +1 -4
    *
- * Avoid
- *     error_propagate(errp, err);
- *     error_prepend(errp, "Could not frobnicate '%s': ", name);
- * because this fails to prepend when @errp is &error_fatal.
+ * Caller may pass as errp:

The caller may then pass in the following errp values:

include/qapi/error.h line 60

Quoted 2 lines
include/qapi/error.h +2 -0
+ * 1. &error_abort
+ *    This means abort on any error

Any error will result in abort()

include/qapi/error.h line 62

Quoted 2 lines
include/qapi/error.h +2 -0
+ * 2. &error_fatal
+ *    Exit with non-zero return code on error

Any error will result in exit() with a non-zero status

include/qapi/error.h line 64

Quoted 2 lines
include/qapi/error.h +2 -0
+ * 3. NULL
+ *    Ignore errors

Any error will be ignored

include/qapi/error.h line 65

Quoted 1 line
include/qapi/error.h +1 -0
+ * 4. Another value

4. The address of a NULL-initialized Error *err

include/qapi/error.h line 66

Quoted 1 line
include/qapi/error.h +1 -0
+ *    On error allocate error object and set errp

Any error will populate errp with an error object

include/qapi/error.h line 69

Quoted 5 lines
include/qapi/error.h +2 -2
    *
- * Create a new error and pass it to the caller:
- *     error_setg(errp, "situation normal, all fouled up");
+ * Error API functions with Error ** (like error_setg) argument supports these
+ * rules, so user functions just need to use them appropriately (read below).

The following rules then implement the correct semantics desired by the caller.

include/qapi/error.h line 71

Quoted 3 lines
include/qapi/error.h +1 -1
    *
- * Call a function and receive an error from it:
+ * Simple pass error to the caller:

Create a new error to pass to the caller:

include/qapi/error.h line 72

Quoted 1 line
include/qapi/error.h +1 -0
+ *     error_setg(errp, "Some error");

You lost the fun wording in Markus' earlier example ("situation normal, all fouled up").

include/qapi/error.h line 75

Quoted 3 lines
include/qapi/error.h +3 -0
+ *
+ * Subcall of another errp-based function, passing the error to the caller
+ *     f(..., errp);

Calling another errp-based function:

include/qapi/error.h line 79

Quoted 4 lines
include/qapi/error.h +4 -0
+ *
+ * == Checking success of subcall ==
+ *
+ * If function returns error code in addition to errp (which is recommended),

If a function returns a value indicating an error in addition to setting errp (which is recommended), then

include/qapi/error.h line 89

Quoted 10 lines
include/qapi/error.h +10 -0
+ * you don't need any additional code, just do:
+ *     int ret = f(..., errp);
+ *     if (ret < 0) {
+ *         ... handle error ...
+ *         return ret;
+ *     }
+ *
+ * If function returns nothing (which is not recommended API) and the only way
+ * to check success is checking errp, we must care about cases [1-3] above. We
+ * need to use macro ERRP_AUTO_PROPAGATE (see below for details) like this:

If a function returns nothing (not recommended for new code), the only way to check success is by consulting errp; doing this safely requires the use of the ERRP_AUTO_PROPAGATE macro, like this:

include/qapi/error.h line 97

Quoted 8 lines
include/qapi/error.h +8 -0
+ *
+ *     int our_func(..., Error **errp) {
+ *         ERRP_AUTO_PROPAGATE();
+ *         ...
+ *         subcall(..., errp);
+ *         if (*errp) {
+ *             ...
+ *             return -ERRNO;
Quoted 1 line

do we want ERRNO capitalized here?

I wrote it capitalized to not conflict with errno variable, to make it obvious that this is just a pseudo-code. But yes, it looks weird anyway.

We can write -errno here, or just -EINVAL (I hope, nobady will thing, that EINVAL should always be used :)

Quoted 29 lines

include/qapi/error.h line 104

Quoted 8 lines
include/qapi/error.h +7 -0
+ *         }
+ *         ...
+ *     }
+ *
+ * ERRP_AUTO_PROPAGATE cares about Error ** API, wraps original errp if needed,
+ * so that it can be safely used (including dereferencing), and auto-propagates
+ * error to original errp on function end.

ERRP_AUTO_PROPAGATE takes care of wrapping the original errp as needed, so that the rest of the function can directly use errp (including dereferencing), where any errors will then be propagated on to the original errp when leaving the function.

Quoted 7 lines
include/qapi/error.h +6 -0
+ *
+ * In some cases, we need to check result of subcall, but do not want to
+ * propagate the Error object to our caller. In such cases we don't need
+ * ERRP_AUTO_PROPAGATE, but just a local Error object:
+ *
+ * Receive an error and not pass it:
    *     Error *err = NULL;

include/qapi/error.h line 120

Quoted 10 lines
include/qapi/error.h +5 -1
- *     foo(arg, &err);
+ *     subcall(arg, &err);
    *     if (err) {
    *         handle the error...
+ *         error_free(err);
    *     }
    *
+ * Note, that before ERRP_AUTO_PROPAGATE introduction the pattern above (with
+ * error_propagate() instead of error_free()) was used to check and pass error
+ * to the caller. Now this is DEPRECATED* (see below).
Quoted 1 line

Hmm - if we bulk-convert the entire tree, then there won't be any deprecated uses to be worth documenting.

Yes, I think we'll drop it after everything is converted.

Quoted 51 lines

  But if we do keep this paragraph:

Note that older code that did not use ERRP_AUTO_PROPAGATE would instead need a local Err variable and the use of error_propagate() to properly handle all possible caller values of errp.

include/qapi/error.h line 125

Quoted 5 lines
include/qapi/error.h +5 -0
+ *
+ * Note also, that if you want to use error_append_hint/error_prepend or their
+ * variants, you must use ERRP_AUTO_PROPAGATE too. Otherwise, in case of
+ * error_fatal, you'll miss the chance to insert your additional information
+ * into Error object.

Note that any function that wants to modify an error object, such as by calling error_append_hint or error_prepend, must use ERRP_AUTO_PROPAGATE, in order for a caller's use of &error_fatal to see the additional information.

Quoted 39 lines
include/qapi/error.h +8 -20
+ *
+ * In rare cases, we need to pass existing Error object to the caller by hand:
+ *     error_propagate(errp, err);
+ *
+ * Pass an existing error to the caller with the message modified:
+ *     error_propagate_prepend(errp, err);
+ *
+ *
    * Call a function ignoring errors:
    *     foo(arg, NULL);
    *
@@ -78,26 +140,6 @@
    * Call a function treating errors as fatal:
    *     foo(arg, &error_fatal);
    *
- * Receive an error and pass it on to the caller:
- *     Error *err = NULL;
- *     foo(arg, &err);
- *     if (err) {
- *         handle the error...
- *         error_propagate(errp, err);
- *     }
- * where Error **errp is a parameter, by convention the last one.
- *
- * Do *not* "optimize" this to
- *     foo(arg, errp);
- *     if (*errp) { // WRONG!
- *         handle the error...
- *     }
- * because errp may be NULL!
- *
- * But when all you do with the error is pass it on, please use
- *     foo(arg, errp);
- * for readability.
- *
    * Receive and accumulate multiple errors (first one wins):
    *     Error *err = NULL, *local_err = NULL;
    *     foo(arg, &err);
@@ -114,6 +156,61 @@
Quoted 8 lines

include/qapi/error.h line 161

Quoted 6 lines
include/qapi/error.h +3 -0
    *         handle the error...
    *     }
    * because this may pass a non-null err to bar().
+ *
+ * DEPRECATED*
+ *

Again, I'm not sure we need this section in the codebase if we do a bulk conversion.  But moving it to the commit message is still useful.

Conversion can't be atomic anyway, there would be several series. But I'm not against dropping the paragraph, so, it's up to Markus.

Quoted 117 lines

include/qapi/error.h line 163

Quoted 3 lines
include/qapi/error.h +2 -0
+ * The following pattern of receiving checking and passing the caller of the
+ * error by hand is deprecated now:

The following pattern of receiving, checking, and then forwarding an error to the caller by hand is now deprecated:

include/qapi/error.h line 172

Quoted 9 lines
include/qapi/error.h +9 -0
+ *
+ *     Error *err = NULL;
+ *     foo(arg, &err);
+ *     if (err) {
+ *         handle the error...
+ *         error_propagate(errp, err);
+ *     }
+ *
+ * Instead, use ERRP_AUTO_PROPAGATE macro (defined below).

Drop "(defined below)".

Quoted 31 lines
include/qapi/error.h +31 -0
+ *
+ * The old pattern is deprecated because of two things:
+ *
+ * 1. Issue with error_abort and error_propagate: when we wrap error_abort by
+ * local_err+error_propagate, the resulting coredump will refer to
+ * error_propagate and not to the place where error happened.
+ *
+ * 2. A lot of extra code of the same pattern
+ *
+ * How to update old code to use ERRP_AUTO_PROPAGATE?
+ *
+ * All you need is to add ERRP_AUTO_PROPAGATE() invocation at function start,
+ * than you may safely dereference errp to check errors and do not need any
+ * additional local Error variables or calls to error_propagate().
+ *
+ * Example:
+ *
+ * old code
+ *
+ *     void fn(..., Error **errp) {
+ *         Error *err = NULL;
+ *         foo(arg, &err);
+ *         if (err) {
+ *             handle the error...
+ *             error_propagate(errp, err);
+ *             return;
+ *         }
+ *         ...
+ *     }
+ *
+ * updated code

include/qapi/error.h line 213

Quoted 10 lines
include/qapi/error.h +10 -0
+ *
+ *     void fn(..., Error **errp) {
+ *         ERRP_AUTO_PROPAGATE();
+ *         foo(arg, errp);
+ *         if (*errp) {
+ *             handle the error...
+ *             return;
+ *         }
+ *         ...
+ *     }

Again, I'm thinking the above example is more useful in the commit message instead of permanently in the .h file.

Quoted 14 lines
include/qapi/error.h +9 -0
    */
   #ifndef ERROR_H
@@ -322,6 +419,46 @@ void error_set_internal(Error **errp,
                           ErrorClass err_class, const char *fmt, ...)
       GCC_FMT_ATTR(6, 7);
+typedef struct ErrorPropagator {
+    Error *local_err;
+    Error **errp;
+} ErrorPropagator;
+
+static inline void error_propagator_cleanup(ErrorPropagator *prop)
+{
+    error_propagate(prop->errp, prop->local_err);
+}

include/qapi/error.h void error_set_internal(Error **errp, · line 439

Quoted 10 lines
include/qapi/error.h +10 -0
+
+G_DEFINE_AUTO_CLEANUP_CLEAR_FUNC(ErrorPropagator, error_propagator_cleanup);
+
+/*
+ * ERRP_AUTO_PROPAGATE
+ *
+ * This macro is created to be the first line of a function which use
+ * Error **errp parameter to report error. It's needed only in cases where we
+ * want to use error_prepend, error_append_hint or dereference *errp. It's
+ * still safe (but useless) in other cases.

This macro exists to assist with proper error handling in a function which uses an Error **errp parameter.  It must be used as the first line of a function which modifies an error (with error_prepend, error_append_hint, or similar) or which wants to dereference *errp.  It is still safe (but useless) to use in other functions.

Quoted 15 lines
include/qapi/error.h +15 -0
+ *
+ * If errp is NULL or points to error_fatal, it is rewritten to point to a
+ * local Error object, which will be automatically propagated to the original
+ * errp on function exit (see error_propagator_cleanup).
+ *
+ * After invocation of this macro it is always safe to dereference errp
+ * (as it's not NULL anymore) and to add information by error_prepend or
+ * error_append_hint (as, if it was error_fatal, we swapped it with a
+ * local_error to be propagated on cleanup).
+ *
+ * Note: we don't wrap the error_abort case, as we want resulting coredump
+ * to point to the place where the error happened, not to error_propagate.
+ */
+#define ERRP_AUTO_PROPAGATE() \
+    g_auto(ErrorPropagator) _auto_errp_prop = {.errp = errp}; \

include/qapi/error.h void error_set_internal(Error **errp,

Quoted 10 lines
include/qapi/error.h +6 -0
+    do { \
+        if (!errp || errp == &error_fatal) { \
+            errp = &_auto_errp_prop.local_err; \
+        } \
+    } while (0)
+
   /*
    * Special error destination to abort on error.
    * See error_setg() and error_propagate() for details.
 

The macro itself looks correct. I'll leave it up to Markus how to handle the comment text, but you can add:

Quoted 2 lines

Thanks for reviewing and wording suggestions, all looks good, as always.

Signature · 7 lines
Best regards,
Vladimir

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
#7 lore raw

PATCH v8 2/10 scripts: add coccinelle script to use auto propagated errp

In reply to Vladimir Sementsov-Ogievskiy

Script adds ERRP_AUTO_PROPAGATE macro invocation where appropriate and does corresponding changes in code (look for details in include/qapi/error.h)

Usage example:
spatch --sp-file scripts/coccinelle/auto-propagated-errp.cocci \
--macro-file scripts/cocci-macro-file.h --in-place --no-show-diff \
blockdev-nbd.c qemu-nbd.c {block/nbd*,nbd/*,include/block/nbd*}.[hc]

---

Cc Eric Blake <[email protected]>
Cc Kevin Wolf <[email protected]>
Cc Max Reitz <[email protected]>
Cc Greg Kurz <[email protected]>
Cc Christian Schoenebeck <[email protected]>
Cc Stefano Stabellini <[email protected]>
Cc Anthony Perard <[email protected]>
Cc Paul Durrant <[email protected]>
Cc Stefan Hajnoczi <[email protected]>
Cc "Philippe Mathieu-Daudé" <[email protected]>
Cc Laszlo Ersek <[email protected]>
Cc Gerd Hoffmann <[email protected]>
Cc Stefan Berger <[email protected]>
Cc Markus Armbruster <[email protected]>
Cc Michael Roth <[email protected]>
 include/qapi/error.h                          |   3 +
 scripts/coccinelle/auto-propagated-errp.cocci | 231 ++++++++++++++++++
 2 files changed, 234 insertions(+)
 create mode 100644 scripts/coccinelle/auto-propagated-errp.cocci
include/qapi/error.h +3 -0
index bb9bcf02fb..fbfc6f1c0b 100644
@@ -211,6 +211,9 @@
  *         }
  *         ...
  *     }
+ *
+ * For mass conversion use script
+ *   scripts/coccinelle/auto-propagated-errp.cocci
  */
 
 #ifndef ERROR_H
scripts/coccinelle/auto-propagated-errp.cocci new file +231 -0

231 lines not on this page · raw message

Signature · 7 lines
2.21.0


_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
#8 lore raw

Re: [PATCH v8 02/10] scripts: add coccinelle script to use auto propagated errp

Quoted 4 lines · Vladimir Sementsov-Ogievskiy

Script adds ERRP_AUTO_PROPAGATE macro invocation where appropriate and does corresponding changes in code (look for details in include/qapi/error.h)

Quoted 8 lines

Usage example:
spatch --sp-file scripts/coccinelle/auto-propagated-errp.cocci \
--macro-file scripts/cocci-macro-file.h --in-place --no-show-diff \
blockdev-nbd.c qemu-nbd.c {block/nbd*,nbd/*,include/block/nbd*}.[hc]

---

I'll let Markus do the final review of this, but my personal take is that if the subsequent patches created by using this script are reasonable, then this script was good enough.

Quoted 1 line

+// Always use the same patter for checking error

pattern

Quoted 13 lines

+@@

scripts/coccinelle/auto-propagated-errp.cocci new file +11 -0
+identifier rule1.fn;
+symbol errp;
+@@
+
+ fn(...)
+ {
+     <...
+-    *errp != NULL
++    *errp
+     ...>
+ }
Signature · 9 lines
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3226
Virtualization:  qemu.org | libvirt.org


_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
#9 lore raw

Re: [PATCH v8 02/10] scripts: add coccinelle script to use auto propagated errp

In reply to Vladimir Sementsov-Ogievskiy

scripts/coccinelle/auto-propagated-errp.cocci line 3

Quoted 9 lines · On Freitag, 6. März 2020 06:15:28 CET Vladimir Sementsov-Ogievskiy wrote:
(quoted diff) +0 -0

b/scripts/coccinelle/auto-propagated-errp.cocci new file mode 100644 index 0000000000..bff274bd6d

scripts/coccinelle/auto-propagated-errp.cocci new file +3 -0
@@ -0,0 +1,231 @@
+// Use ERRP_AUTO_PROPAGATE (see include/qapi/error.h)
+//
+// Copyright (c) 2020 Virtuozzo International GmbH.

Just in case:

WARNING: added, moved or deleted file(s), does MAINTAINERS need updating?

Best regards,
Christian Schoenebeck

#10 lore raw

Re: [PATCH v8 02/10] scripts: add coccinelle script to use auto propagated errp

Quoted 13 lines · Christian Schoenebeck

scripts/coccinelle/auto-propagated-errp.cocci line 3

Quoted 9 lines · On Freitag, 6. März 2020 06:15:28 CET Vladimir Sementsov-Ogievskiy wrote:
(quoted diff) +0 -0

b/scripts/coccinelle/auto-propagated-errp.cocci new file mode 100644 index 0000000000..bff274bd6d

scripts/coccinelle/auto-propagated-errp.cocci new file +3 -0
@@ -0,0 +1,231 @@
+// Use ERRP_AUTO_PROPAGATE (see include/qapi/error.h)
+//
+// Copyright (c) 2020 Virtuozzo International GmbH.

Just in case:

Quoted 1 line

WARNING: added, moved or deleted file(s), does MAINTAINERS need updating?

Hmm, seems this, and some other coccinelle scripts should be added to "Error reporting" section.

Quoted 6 lines

Best regards,
Christian Schoenebeck

Signature · 7 lines
Best regards,
Vladimir

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
#11 lore raw

Re: [PATCH v8 02/10] scripts: add coccinelle script to use auto propagated errp

In reply to Vladimir Sementsov-Ogievskiy

Suggest

scripts: Coccinelle script to use auto-propagated errp

or

scripts: Coccinelle script to use ERRP_AUTO_PROPAGATE()

Quoted 8 lines · Vladimir Sementsov-Ogievskiy

Script adds ERRP_AUTO_PROPAGATE macro invocation where appropriate and does corresponding changes in code (look for details in include/qapi/error.h)

Usage example:
spatch --sp-file scripts/coccinelle/auto-propagated-errp.cocci \
--macro-file scripts/cocci-macro-file.h --in-place --no-show-diff \
blockdev-nbd.c qemu-nbd.c {block/nbd*,nbd/*,include/block/nbd*}.[hc]

Suggest FILES... instead of a specific set of files.

Quoted 32 lines

---

Cc Eric Blake <[email protected]>
Cc Kevin Wolf <[email protected]>
Cc Max Reitz <[email protected]>
Cc Greg Kurz <[email protected]>
Cc Christian Schoenebeck <[email protected]>
Cc Stefano Stabellini <[email protected]>
Cc Anthony Perard <[email protected]>
Cc Paul Durrant <[email protected]>
Cc Stefan Hajnoczi <[email protected]>
Cc "Philippe Mathieu-Daudé" <[email protected]>
Cc Laszlo Ersek <[email protected]>
Cc Gerd Hoffmann <[email protected]>
Cc Stefan Berger <[email protected]>
Cc Markus Armbruster <[email protected]>
Cc Michael Roth <[email protected]>
 include/qapi/error.h                          |   3 +
 scripts/coccinelle/auto-propagated-errp.cocci | 231 ++++++++++++++++++
 2 files changed, 234 insertions(+)
 create mode 100644 scripts/coccinelle/auto-propagated-errp.cocci
include/qapi/error.h +0 -0
index bb9bcf02fb..fbfc6f1c0b 100644

include/qapi/error.h line 215

Quoted 5 lines
include/qapi/error.h +2 -0
@@ -211,6 +211,9 @@
  *         }
  *         ...
  *     }
+ *
+ * For mass conversion use script

mass-conversion (we're not converting mass, we're converting en masse)

scripts/coccinelle/auto-propagated-errp.cocci

Quoted 9 lines
include/qapi/error.h +1 -0
+ *   scripts/coccinelle/auto-propagated-errp.cocci
  */
 
 #ifndef ERROR_H
scripts/coccinelle/auto-propagated-errp.cocci new file +0 -0
new file mode 100644
index 0000000000..bff274bd6d

Preface to my review of this script: may aim isn't to make it bullet-proof. I want to (1) make it good enough (explained in a jiffie), and (2) automatically identify the spots where it still isn't obviously safe for manual review.

The latter may involve additional scripting. That's okay.

The script is good enough when the number of possibly unsafe spots is low enough for careful manual review.

When I ask for improvements that, in your opinion, go beyond "good enough", please push back. I'm sure we can work it out together.

Quoted 12 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +11 -0
@@ -0,0 +1,231 @@
+// Use ERRP_AUTO_PROPAGATE (see include/qapi/error.h)
+//
+// Copyright (c) 2020 Virtuozzo International GmbH.
+//
+// This program is free software; you can redistribute it and/or modify
+// it under the terms of the GNU General Public License as published by
+// the Free Software Foundation; either version 2 of the License, or
+// (at your option) any later version.
+//
+// This program is distributed in the hope that it will be useful,
+// but WITHOUT ANY WARRANTY; without even the implied warranty of

scripts/coccinelle/auto-propagated-errp.cocci line 21

Quoted 10 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +10 -0
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+// GNU General Public License for more details.
+//
+// You should have received a copy of the GNU General Public License
+// along with this program.  If not, see <http://www.gnu.org/licenses/>.
+//
+// Usage example:
+// spatch --sp-file scripts/coccinelle/auto-propagated-errp.cocci \
+//  --macro-file scripts/cocci-macro-file.h --in-place --no-show-diff \
+//  --max-width 80 blockdev-nbd.c qemu-nbd.c \

You have --max-width 80 here, but not in the commit message. Default seems to be 78. Any particular reason to change it to 80?

scripts/coccinelle/auto-propagated-errp.cocci line 24

Quoted 3 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +3 -0
+//  {block/nbd*,nbd/*,include/block/nbd*}.[hc]
+
+// Switch unusual (Error **) parameter names to errp

Let's drop the parenthesis around Error **

scripts/coccinelle/auto-propagated-errp.cocci line 25

Quoted 1 line
scripts/coccinelle/auto-propagated-errp.cocci new file +1 -0
+// (this is necessary to use ERRP_AUTO_PROPAGATE).

Perhaps ERRP_AUTO_PROPAGATE() should be ERRP_AUTO_PROPAGATE(errp) to make the fact we're messing with @errp more obvious. Too late; I shouldn't rock the boat that much now.

scripts/coccinelle/auto-propagated-errp.cocci line 31

Quoted 6 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +6 -0
+//
+// Disable optional_qualifier to skip functions with "Error *const *errp"
+// parameter.
+//
+// Skip functions with "assert(_errp && *_errp)" statement, as they have
+// non generic semantics and may have unusual Error ** argument name for purpose

non-generic

for a purpose

Wrap comment lines around column 70, please. It's easier to read.

Maybe

// Skip functions with "assert(_errp && *_errp)" statement, because that // signals unusual semantics, and the parameter name may well serve a // purpose.

scripts/coccinelle/auto-propagated-errp.cocci line 35

Quoted 4 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +4 -0
+// (like nbd_iter_channel_error()).
+//
+// Skip util/error.c to not touch, for example, error_propagate and
+// error_propagate_prepend().

error_propagate()

I much appreciate your meticulous explanation of what you skip and why.

Quoted 9 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +9 -0
+@ depends on !(file in "util/error.c") disable optional_qualifier@
+identifier fn;
+identifier _errp != errp;
+@@
+
+ fn(...,
+-   Error **_errp
++   Error **errp
+    ,...)

scripts/coccinelle/auto-propagated-errp.cocci line 54

Quoted 10 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +10 -0
+ {
+(
+     ... when != assert(_errp && *_errp)
+&
+     <...
+-    _errp
++    errp
+     ...>
+)
+ }

This rule is required to make the actual transformations (below) work even for parameters with names other than @errp. I believe it's not used in this series. In fact, I can't see a use for it in the entire tree right now. Okay anyway.

scripts/coccinelle/auto-propagated-errp.cocci line 58

Quoted 4 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +4 -0
+
+// Add invocation of ERRP_AUTO_PROPAGATE to errp-functions where necessary
+//
+// Note, that without "when any" final "..." may not want to mach something

s/final "..." may not mach/the final "..." does not match/

scripts/coccinelle/auto-propagated-errp.cocci line 60

Quoted 2 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +2 -0
+// matched by previous pattern, i.e. the rule will not match double
+// error_prepend in control flow like in vfio_set_irq_signaling().

Can't say I fully understand Coccinelle there. I figure you came to this knowledge the hard way.

scripts/coccinelle/auto-propagated-errp.cocci line 64

Quoted 4 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +4 -0
+//
+// Note, "exists" says that we want apply rule even if it matches not on
+// all possible control flows (otherwise, it will not match standard pattern
+// when error_propagate() call is in if branch).

Learned something new. Example: kvm_set_kvm_shadow_mem().

Spelling it "exists disable optional_qualifier" would avoid giving readers the idea we're disabling "exists", but Coccinelle doesn't let us. Oh well.

scripts/coccinelle/auto-propagated-errp.cocci line 66

Quoted 2 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +2 -0
+@ disable optional_qualifier exists@
+identifier fn, local_err, errp;

I believe this causes

warning: line 98: errp, previously declared as a metavariable, is used as an identifier
warning: line 104: errp, previously declared as a metavariable, is used as an identifier
warning: line 106: errp, previously declared as a metavariable, is used as an identifier
warning: line 131: errp, previously declared as a metavariable, is used as an identifier
warning: line 192: errp, previously declared as a metavariable, is used as an identifier
warning: line 195: errp, previously declared as a metavariable, is used as an identifier
warning: line 228: errp, previously declared as a metavariable, is used as an identifier

Making @errp symbol instead of identifier should fix this.

Quoted 31 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +31 -0
+@@
+
+ fn(..., Error **errp, ...)
+ {
++   ERRP_AUTO_PROPAGATE();
+    ...  when != ERRP_AUTO_PROPAGATE();
+(
+    error_append_hint(errp, ...);
+|
+    error_prepend(errp, ...);
+|
+    error_vprepend(errp, ...);
+|
+    Error *local_err = NULL;
+    ...
+(
+    error_propagate_prepend(errp, local_err, ...);
+|
+    error_propagate(errp, local_err);
+)
+)
+    ... when any
+ }
+
+
+// Match scenarios with propagation of local error to errp.
+@rule1 disable optional_qualifier exists@
+identifier fn, local_err;
+symbol errp;
+@@
+

scripts/coccinelle/auto-propagated-errp.cocci line 107

Quoted 10 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +10 -0
+ fn(..., Error **errp, ...)
+ {
+     ...
+     Error *local_err = NULL;
+     ...
+(
+    error_propagate_prepend(errp, local_err, ...);
+|
+    error_propagate(errp, local_err);
+)

Indentation off by one.

scripts/coccinelle/auto-propagated-errp.cocci line 111

Quoted 4 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +4 -0
+     ...
+ }
+
+// Convert special case with goto in separate.

s/in separate/separately/

scripts/coccinelle/auto-propagated-errp.cocci line 113

Quoted 2 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +2 -0
+// We can probably merge this into the following hunk with help of ( | )
+// operator, but it significantly reduce performance on block.c parsing (or it

s/reduce/reduces/

scripts/coccinelle/auto-propagated-errp.cocci line 114

Quoted 1 line
scripts/coccinelle/auto-propagated-errp.cocci new file +1 -0
+// hangs, I don't know)

Sounds like you tried to merge this into the following hunk, but then spatch took so long on block.c that you killed it. Correct?

scripts/coccinelle/auto-propagated-errp.cocci line 118

Quoted 4 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +4 -0
+//
+// Note interesting thing: if we don't do it here, and try to fixup "out: }"
+// things later after all transformations (the rule will be the same, just
+// without error_propagate() call), coccinelle fails to match this "out: }".

Weird, but not worth further investigation.

scripts/coccinelle/auto-propagated-errp.cocci line 131

Quoted 13 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +13 -0
+@@
+identifier rule1.fn, rule1.local_err, out;
+symbol errp;
+@@
+
+ fn(...)
+ {
+     <...
+-    goto out;
++    return;
+     ...>
+- out:
+-    error_propagate(errp, local_err);

You neglect to match error_propagate_prepend(). Okay, because (1) that pattern doesn't occur in the tree right now, and (2) if it gets added, gcc will complain.

scripts/coccinelle/auto-propagated-errp.cocci line 134

Quoted 3 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +3 -0
+ }
+
+// Convert most of local_err related staff.

s/staff/stuff/

scripts/coccinelle/auto-propagated-errp.cocci line 141

Quoted 7 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +7 -0
+//
+// Note, that we update everything related to matched by rule1 function name
+// and local_err name. We may match something not related to the pattern
+// matched by rule1. For example, local_err may be defined with the same name
+// in different blocks inside one function, and in one block follow the
+// propagation pattern and in other block doesn't. Or we may have several
+// functions with the same name (for different configurations).

Context: rule1 matches functions that have all three of

* an Error **errp parameter

* an Error *local_err = NULL variable declaration

* an error_propagate(errp, local_err) or error_propagate_prepend(errp, local_err, ...) expression, where @errp is the parameter and @local_err is the variable.

If I understand you correctly, you're pointing out two potential issues:

1. This rule can match functions rule1 does not match if there is another function with the same name that rule1 does match.

2. This rule matches in the entire function matched by rule1, even when parts of that function use a different @errp or @local_err.

I figure these apply to all rules with identifier rule1.fn, not just this one. Correct?

Regarding 1. There must be a better way to chain rules together, but I don't know it. Can we make Coccinelle at least warn us when it converts multiple functions with the same name? What about this:

@initialize:python@
@@
fnprev = {}

def pr(fn, p):

       print("### %s:%s: %s()" % (p[0].file, p[0].line, fn))
   @r@
   identifier rule1.fn;
   position p;
   @@
    fn(...)@p
    {
        ...
    }
   @script:python@
       fn << rule1.fn;
       p << r.p;
   @@
   if fn not in fnprev:
       fnprev[fn] = p
   else:
       if fnprev[fn]:
           pr(fn, fnprev[fn])
           fnprev[fn] = None
       pr(fn, p)

For each function @fn matched by rule1, fncnt[fn] is an upper limit of the number of functions with the same name we touch. If it's more than one, we print.

Reports about a dozen function names for the whole tree in my testing. Inspecting the changes to them manually is feasible. None of them are in files touched by this series.

The line printed for the first match is pretty useless for me: it points to a Coccinelle temporary file *shrug*.

Regarding 2. Shadowing @errp or @local_err would be in bad taste, and I sure hope we don't do that. Multiple @local_err variables... hmm. Perhaps we could again concoct some script rules to lead us to spots to check manually. See below for my attempt.

What's the worst that could happen if we blindly converted such code? The answer to that question tells us how hard to work on finding and checking these guys.

Quoted 17 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +17 -0
+//
+// Note also that errp-cleaning functions
+//   error_free_errp
+//   error_report_errp
+//   error_reportf_errp
+//   warn_report_errp
+//   warn_reportf_errp
+// are not yet implemented. They must call corresponding Error* - freeing
+// function and then set *errp to NULL, to avoid further propagation to
+// original errp (consider ERRP_AUTO_PROPAGATE in use).
+// For example, error_free_errp may look like this:
+//
+//    void error_free_errp(Error **errp)
+//    {
+//        error_free(*errp);
+//        *errp = NULL;
+//    }

scripts/coccinelle/auto-propagated-errp.cocci line 168

Quoted 10 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +10 -0
+@ exists@
+identifier rule1.fn, rule1.local_err;
+expression list args;
+symbol errp;
+@@
+
+ fn(...)
+ {
+     <...
+(

Each of the following patterns applies anywhere in the function.

First pattern: delete @local_err

scripts/coccinelle/auto-propagated-errp.cocci line 169

Quoted 1 line
scripts/coccinelle/auto-propagated-errp.cocci new file +1 -0
+-    Error *local_err = NULL;

Common case: occurs just once, not nested. Anything else is suspicious.

Both can be detected in the resulting patches with a bit of AWK wizardry:

    $ git-diff -U0 master..review-error-v8 | awk '/^@@ / { ctx = $5; for (i = 6; i <= NF; i++) ctx = ctx " " $i; if (ctx != octx) { octx = ctx; n = 0 } } /^- *Error *\* *[A-Za-z0-9_]+ *= *NULL;/ { if (index($0, "E") > 6) print "nested\n    " ctx; if (n) print "more than one\n    " ctx; n++ }'
    nested
        static void xen_block_drive_destroy(XenBlockDrive *drive, Error **errp)
    nested
        static void xen_block_device_destroy(XenBackendInstance *backend,
    nested
        static void xen_block_device_destroy(XenBackendInstance *backend,
    more than one
        static void xen_block_device_destroy(XenBackendInstance *backend,

Oh.

xen_block_drive_destroy() nests its Error *local_err in a conditional.

xen_block_device_destroy() has multiple Error *local_err.

In both cases, manual review is required to ensure the conversion is okay. I believe it is.

Note that the AWK script relies on diff showing the function name in @@ lines, which doesn't always work due to our coding style.

For the whole tree, I get some 30 spots. Feasible.

scripts/coccinelle/auto-propagated-errp.cocci line 170

Quoted 1 line
scripts/coccinelle/auto-propagated-errp.cocci new file +1 -0
+|

Second pattern: clear @errp after freeing it

scripts/coccinelle/auto-propagated-errp.cocci line 172

Quoted 2 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +2 -0
+
+// Convert error clearing functions

Suggest: Ensure @local_err is cleared on free

Quoted 6 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +6 -0
+(
+-    error_free(local_err);
++    error_free_errp(errp);
+|
+-    error_report_err(local_err);
++    error_report_errp(errp);

scripts/coccinelle/auto-propagated-errp.cocci line 188

Quoted 10 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +10 -0
+|
+-    error_reportf_err(local_err, args);
++    error_reportf_errp(errp, args);
+|
+-    warn_report_err(local_err);
++    warn_report_errp(errp);
+|
+-    warn_reportf_err(local_err, args);
++    warn_reportf_errp(errp, args);
+)

As you mention above, these guys don't exist, yet. Builds anyway, because this part of the rule is not used in this patch series. You don't want to omit it, because then the script becomes unsafe to use.

We could also open-code:

// Convert error clearing functions
(

   -    error_free(local_err);
   +    error_free(*errp);
   +    *errp = NULL;

|
... and so forth ...
)

Matter of taste. Whatever is easier to explain in the comments. Since you already wrote one...

We talked about extending this series slightly so these guys are used. I may still look into that.

scripts/coccinelle/auto-propagated-errp.cocci line 190

Quoted 2 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +2 -0
+?-    local_err = NULL;
+

The new helpers clear @local_err. Assignment now redundant, delete. Okay.

scripts/coccinelle/auto-propagated-errp.cocci line 191

Quoted 1 line
scripts/coccinelle/auto-propagated-errp.cocci new file +1 -0
+|

Third and fourth pattern: delete error_propagate()

scripts/coccinelle/auto-propagated-errp.cocci line 196

Quoted 5 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +5 -0
+-    error_propagate_prepend(errp, local_err, args);
++    error_prepend(errp, args);
+|
+-    error_propagate(errp, local_err);
+|

Fifth pattern: use @errp directly

scripts/coccinelle/auto-propagated-errp.cocci line 204

Quoted 8 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +8 -0
+-    &local_err
++    errp
+)
+     ...>
+ }
+
+// Convert remaining local_err usage. It should be different kinds of error
+// checking in if operators. We can't merge this into previous hunk, as this

In if conditionals, I suppose. It's the case for this patch. If I apply the script to the whole tree, the rule gets also applied in other contexts. The sentence might mislead as much as it helps. Keep it or delete it?

Quoted 5 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +5 -0
+// conflicts with other substitutions in it (at least with "- local_err = NULL").
+@@
+identifier rule1.fn, rule1.local_err;
+symbol errp;
+@@

scripts/coccinelle/auto-propagated-errp.cocci line 219

Quoted 10 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +10 -0
+
+ fn(...)
+ {
+     <...
+-    local_err
++    *errp
+     ...>
+ }
+
+// Always use the same patter for checking error

s/patter/pattern/

scripts/coccinelle/auto-propagated-errp.cocci line 231

Quoted 12 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +12 -0
+@@
+identifier rule1.fn;
+symbol errp;
+@@
+
+ fn(...)
+ {
+     <...
+-    *errp != NULL
++    *errp
+     ...>
+ }

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

#12 lore raw

Re: [PATCH v8 02/10] scripts: add coccinelle script to use auto propagated errp

Quoted 101 lines · Markus Armbruster

Suggest

scripts: Coccinelle script to use auto-propagated errp

or

scripts: Coccinelle script to use ERRP_AUTO_PROPAGATE()

Quoted 8 lines · Vladimir Sementsov-Ogievskiy

Script adds ERRP_AUTO_PROPAGATE macro invocation where appropriate and does corresponding changes in code (look for details in include/qapi/error.h)

Usage example:
spatch --sp-file scripts/coccinelle/auto-propagated-errp.cocci \
--macro-file scripts/cocci-macro-file.h --in-place --no-show-diff \
blockdev-nbd.c qemu-nbd.c {block/nbd*,nbd/*,include/block/nbd*}.[hc]

Suggest FILES... instead of a specific set of files.

Quoted 32 lines

---

Cc Eric Blake <[email protected]>
Cc Kevin Wolf <[email protected]>
Cc Max Reitz <[email protected]>
Cc Greg Kurz <[email protected]>
Cc Christian Schoenebeck <[email protected]>
Cc Stefano Stabellini <[email protected]>
Cc Anthony Perard <[email protected]>
Cc Paul Durrant <[email protected]>
Cc Stefan Hajnoczi <[email protected]>
Cc "Philippe Mathieu-Daudé" <[email protected]>
Cc Laszlo Ersek <[email protected]>
Cc Gerd Hoffmann <[email protected]>
Cc Stefan Berger <[email protected]>
Cc Markus Armbruster <[email protected]>
Cc Michael Roth <[email protected]>
  include/qapi/error.h                          |   3 +
  scripts/coccinelle/auto-propagated-errp.cocci | 231 ++++++++++++++++++
  2 files changed, 234 insertions(+)

create mode 100644 scripts/coccinelle/auto-propagated-errp.cocci

include/qapi/error.h +0 -0
index bb9bcf02fb..fbfc6f1c0b 100644

include/qapi/error.h line 215

Quoted 5 lines
include/qapi/error.h +2 -0
@@ -211,6 +211,9 @@
   *         }
   *         ...
   *     }
+ *
+ * For mass conversion use script

mass-conversion (we're not converting mass, we're converting en masse)

scripts/coccinelle/auto-propagated-errp.cocci

Quoted 9 lines
include/qapi/error.h +1 -0
+ *   scripts/coccinelle/auto-propagated-errp.cocci
   */
 
  #ifndef ERROR_H
scripts/coccinelle/auto-propagated-errp.cocci new file +0 -0
new file mode 100644
index 0000000000..bff274bd6d

Preface to my review of this script: may aim isn't to make it bullet-proof. I want to (1) make it good enough (explained in a jiffie), and (2) automatically identify the spots where it still isn't obviously safe for manual review.

The latter may involve additional scripting. That's okay.

The script is good enough when the number of possibly unsafe spots is low enough for careful manual review.

When I ask for improvements that, in your opinion, go beyond "good enough", please push back. I'm sure we can work it out together.

Quoted 17 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +16 -0
@@ -0,0 +1,231 @@
+// Use ERRP_AUTO_PROPAGATE (see include/qapi/error.h)
+//
+// Copyright (c) 2020 Virtuozzo International GmbH.
+//
+// This program is free software; you can redistribute it and/or modify
+// it under the terms of the GNU General Public License as published by
+// the Free Software Foundation; either version 2 of the License, or
+// (at your option) any later version.
+//
+// This program is distributed in the hope that it will be useful,
+// but WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+// GNU General Public License for more details.
+//
+// You should have received a copy of the GNU General Public License
+// along with this program.  If not, see <http://www.gnu.org/licenses/>.
Quoted 8 lines

scripts/coccinelle/auto-propagated-errp.cocci line 21

Quoted 5 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +5 -0
+//
+// Usage example:
+// spatch --sp-file scripts/coccinelle/auto-propagated-errp.cocci \
+//  --macro-file scripts/cocci-macro-file.h --in-place --no-show-diff \
+//  --max-width 80 blockdev-nbd.c qemu-nbd.c \

You have --max-width 80 here, but not in the commit message. Default seems to be 78. Any particular reason to change it to 80?

Hmm. As I remember, without this parameter, reindenting doesn't work correctly. So, I'm OK with "--max-width 78", but I doubt that it will work without a parameter. Still, may be I'm wrong, we can check it.

Quoted 24 lines

scripts/coccinelle/auto-propagated-errp.cocci line 24

Quoted 4 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +3 -0
+//  {block/nbd*,nbd/*,include/block/nbd*}.[hc]
+
+// Switch unusual (Error **) parameter names to errp

Let's drop the parenthesis around Error **

scripts/coccinelle/auto-propagated-errp.cocci line 25

Quoted 1 line
scripts/coccinelle/auto-propagated-errp.cocci new file +1 -0
+// (this is necessary to use ERRP_AUTO_PROPAGATE).

Perhaps ERRP_AUTO_PROPAGATE() should be ERRP_AUTO_PROPAGATE(errp) to make the fact we're messing with @errp more obvious. Too late; I shouldn't rock the boat that much now.

scripts/coccinelle/auto-propagated-errp.cocci line 31

Quoted 6 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +6 -0
+//
+// Disable optional_qualifier to skip functions with "Error *const *errp"
+// parameter.
+//
+// Skip functions with "assert(_errp && *_errp)" statement, as they have
+// non generic semantics and may have unusual Error ** argument name for purpose

non-generic

for a purpose

Quoted 7 lines

Wrap comment lines around column 70, please. It's easier to read.

Maybe

    // Skip functions with "assert(_errp && *_errp)" statement, because that
    // signals unusual semantics, and the parameter name may well serve a
    // purpose.

Sounds good.

Quoted 42 lines

scripts/coccinelle/auto-propagated-errp.cocci line 35

Quoted 5 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +4 -0
+// (like nbd_iter_channel_error()).
+//
+// Skip util/error.c to not touch, for example, error_propagate and
+// error_propagate_prepend().

error_propagate()

I much appreciate your meticulous explanation of what you skip and why.

Quoted 9 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +9 -0
+@ depends on !(file in "util/error.c") disable optional_qualifier@
+identifier fn;
+identifier _errp != errp;
+@@
+
+ fn(...,
+-   Error **_errp
++   Error **errp
+    ,...)

scripts/coccinelle/auto-propagated-errp.cocci line 54

Quoted 10 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +10 -0
+ {
+(
+     ... when != assert(_errp && *_errp)
+&
+     <...
+-    _errp
++    errp
+     ...>
+)
+ }

This rule is required to make the actual transformations (below) work even for parameters with names other than @errp. I believe it's not used in this series. In fact, I can't see a use for it in the entire tree right now. Okay anyway.

scripts/coccinelle/auto-propagated-errp.cocci line 58

Quoted 4 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +4 -0
+
+// Add invocation of ERRP_AUTO_PROPAGATE to errp-functions where necessary
+//
+// Note, that without "when any" final "..." may not want to mach something

s/final "..." may not mach/the final "..." does not match/

Quoted 5 lines

scripts/coccinelle/auto-propagated-errp.cocci line 60

Quoted 2 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +2 -0
+// matched by previous pattern, i.e. the rule will not match double
+// error_prepend in control flow like in vfio_set_irq_signaling().

Can't say I fully understand Coccinelle there. I figure you came to this knowledge the hard way.

It's follows from smpl grammar document:

"Implicitly, “...” matches the shortest path between something that matches the pattern before the dots (or the beginning of the function, if there is nothing before the dots) and something that matches the pattern after the dots (or the end of the function, if there is nothing after the dots)."
...
"_when any_ removes the aforementioned constraint that “...” matches the shortest path"

Quoted 25 lines

scripts/coccinelle/auto-propagated-errp.cocci line 64

Quoted 5 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +4 -0
+//
+// Note, "exists" says that we want apply rule even if it matches not on
+// all possible control flows (otherwise, it will not match standard pattern
+// when error_propagate() call is in if branch).

Learned something new. Example: kvm_set_kvm_shadow_mem().

Spelling it "exists disable optional_qualifier" would avoid giving readers the idea we're disabling "exists", but Coccinelle doesn't let us. Oh well.

scripts/coccinelle/auto-propagated-errp.cocci line 66

Quoted 2 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +2 -0
+@ disable optional_qualifier exists@
+identifier fn, local_err, errp;

I believe this causes

warning: line 98: errp, previously declared as a metavariable, is used as an identifier
warning: line 104: errp, previously declared as a metavariable, is used as an identifier
warning: line 106: errp, previously declared as a metavariable, is used as an identifier
warning: line 131: errp, previously declared as a metavariable, is used as an identifier
warning: line 192: errp, previously declared as a metavariable, is used as an identifier
warning: line 195: errp, previously declared as a metavariable, is used as an identifier
warning: line 228: errp, previously declared as a metavariable, is used as an identifier

Quoted 1 line

Making @errp symbol instead of identifier should fix this.

Hmm, I didn't see these warnings.. But yes, it should be symbol.

Quoted 57 lines
Quoted 32 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +31 -0
+@@
+
+ fn(..., Error **errp, ...)
+ {
++   ERRP_AUTO_PROPAGATE();
+    ...  when != ERRP_AUTO_PROPAGATE();
+(
+    error_append_hint(errp, ...);
+|
+    error_prepend(errp, ...);
+|
+    error_vprepend(errp, ...);
+|
+    Error *local_err = NULL;
+    ...
+(
+    error_propagate_prepend(errp, local_err, ...);
+|
+    error_propagate(errp, local_err);
+)
+)
+    ... when any
+ }
+
+
+// Match scenarios with propagation of local error to errp.
+@rule1 disable optional_qualifier exists@
+identifier fn, local_err;
+symbol errp;
+@@
+

scripts/coccinelle/auto-propagated-errp.cocci line 107

Quoted 10 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +10 -0
+ fn(..., Error **errp, ...)
+ {
+     ...
+     Error *local_err = NULL;
+     ...
+(
+    error_propagate_prepend(errp, local_err, ...);
+|
+    error_propagate(errp, local_err);
+)

Indentation off by one.

scripts/coccinelle/auto-propagated-errp.cocci line 111

Quoted 4 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +4 -0
+     ...
+ }
+
+// Convert special case with goto in separate.

s/in separate/separately/

scripts/coccinelle/auto-propagated-errp.cocci line 113

Quoted 2 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +2 -0
+// We can probably merge this into the following hunk with help of ( | )
+// operator, but it significantly reduce performance on block.c parsing (or it

s/reduce/reduces/

Quoted 4 lines

scripts/coccinelle/auto-propagated-errp.cocci line 114

Quoted 1 line
scripts/coccinelle/auto-propagated-errp.cocci new file +1 -0
+// hangs, I don't know)

Sounds like you tried to merge this into the following hunk, but then spatch took so long on block.c that you killed it. Correct?

Yes.

Quoted 7 lines

scripts/coccinelle/auto-propagated-errp.cocci line 118

Quoted 5 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +4 -0
+//
+// Note interesting thing: if we don't do it here, and try to fixup "out: }"
+// things later after all transformations (the rule will be the same, just
+// without error_propagate() call), coccinelle fails to match this "out: }".

Weird, but not worth further investigation.

It partially match to the idea which I saw somewhere in coccinelle documentation, that coccinelle converts correct C code to correct C code. "out: }" is an example of incorrect, impossible code flow, and coccinelle can't work with it... But it's just a thought.

Quoted 10 lines
Quoted 10 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +9 -0
+@@
+identifier rule1.fn, rule1.local_err, out;
+symbol errp;
+@@
+
+ fn(...)
+ {
+     <...
+-    goto out;
Quoted 8 lines

scripts/coccinelle/auto-propagated-errp.cocci line 131

Quoted 4 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +4 -0
++    return;
+     ...>
+- out:
+-    error_propagate(errp, local_err);

You neglect to match error_propagate_prepend(). Okay, because (1) that pattern doesn't occur in the tree right now, and (2) if it gets added, gcc will complain.

No, because it should not removed. error_propagate_prepend should be converted to prepend, not removed. So, corresponding gotos should not be removed as well.

Quoted 27 lines

scripts/coccinelle/auto-propagated-errp.cocci line 134

Quoted 4 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +3 -0
+ }
+
+// Convert most of local_err related staff.

s/staff/stuff/

scripts/coccinelle/auto-propagated-errp.cocci line 141

Quoted 7 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +7 -0
+//
+// Note, that we update everything related to matched by rule1 function name
+// and local_err name. We may match something not related to the pattern
+// matched by rule1. For example, local_err may be defined with the same name
+// in different blocks inside one function, and in one block follow the
+// propagation pattern and in other block doesn't. Or we may have several
+// functions with the same name (for different configurations).

Context: rule1 matches functions that have all three of

* an Error **errp parameter

* an Error *local_err = NULL variable declaration

* an error_propagate(errp, local_err) or error_propagate_prepend(errp, local_err, ...) expression, where @errp is the parameter and @local_err is the variable.

If I understand you correctly, you're pointing out two potential issues:

Quoted 8 lines

1. This rule can match functions rule1 does not match if there is another function with the same name that rule1 does match.

2. This rule matches in the entire function matched by rule1, even when parts of that function use a different @errp or @local_err.

I figure these apply to all rules with identifier rule1.fn, not just this one. Correct?

Yes.

Quoted 21 lines

Regarding 1. There must be a better way to chain rules together, but I don't know it.
Can we make Coccinelle at least warn us when it converts
multiple functions with the same name? What about this:

    @initialize:python@
    @@
    fnprev = {}

    def pr(fn, p):
        print("### %s:%s: %s()" % (p[0].file, p[0].line, fn))

    @r@
    identifier rule1.fn;
    position p;
    @@
     fn(...)@p
     {
         ...
     }
Quoted 8 lines
    @script:python@
        fn << rule1.fn;
        p << r.p;
    @@
    if fn not in fnprev:
        fnprev[fn] = p
    else:
        if fnprev[fn]:

hmm, the condition can't be false

Quoted 3 lines
            pr(fn, fnprev[fn])
            fnprev[fn] = None
        pr(fn, p)

and we'll miss next duplication..

But I like the idea.

Quoted 125 lines

For each function @fn matched by rule1, fncnt[fn] is an upper limit of the number of functions with the same name we touch. If it's more than one, we print.

Reports about a dozen function names for the whole tree in my testing. Inspecting the changes to them manually is feasible. None of them are in files touched by this series.

The line printed for the first match is pretty useless for me: it points to a Coccinelle temporary file *shrug*.

Regarding 2. Shadowing @errp or @local_err would be in bad taste, and I sure hope we don't do that. Multiple @local_err variables... hmm. Perhaps we could again concoct some script rules to lead us to spots to check manually. See below for my attempt.

What's the worst that could happen if we blindly converted such code? The answer to that question tells us how hard to work on finding and checking these guys.

Quoted 17 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +17 -0
+//
+// Note also that errp-cleaning functions
+//   error_free_errp
+//   error_report_errp
+//   error_reportf_errp
+//   warn_report_errp
+//   warn_reportf_errp
+// are not yet implemented. They must call corresponding Error* - freeing
+// function and then set *errp to NULL, to avoid further propagation to
+// original errp (consider ERRP_AUTO_PROPAGATE in use).
+// For example, error_free_errp may look like this:
+//
+//    void error_free_errp(Error **errp)
+//    {
+//        error_free(*errp);
+//        *errp = NULL;
+//    }

scripts/coccinelle/auto-propagated-errp.cocci line 168

Quoted 10 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +10 -0
+@ exists@
+identifier rule1.fn, rule1.local_err;
+expression list args;
+symbol errp;
+@@
+
+ fn(...)
+ {
+     <...
+(

Each of the following patterns applies anywhere in the function.

First pattern: delete @local_err

scripts/coccinelle/auto-propagated-errp.cocci line 169

Quoted 1 line
scripts/coccinelle/auto-propagated-errp.cocci new file +1 -0
+-    Error *local_err = NULL;

Common case: occurs just once, not nested. Anything else is suspicious.

Both can be detected in the resulting patches with a bit of AWK wizardry:

     $ git-diff -U0 master..review-error-v8 | awk '/^@@ / { ctx = $5; for (i = 6; i <= NF; i++) ctx = ctx " " $i; if (ctx != octx) { octx = ctx; n = 0 } } /^- *Error *\* *[A-Za-z0-9_]+ *= *NULL;/ { if (index($0, "E") > 6) print "nested\n    " ctx; if (n) print "more than one\n    " ctx; n++ }'
     nested
         static void xen_block_drive_destroy(XenBlockDrive *drive, Error **errp)
     nested
         static void xen_block_device_destroy(XenBackendInstance *backend,
     nested
         static void xen_block_device_destroy(XenBackendInstance *backend,
     more than one
         static void xen_block_device_destroy(XenBackendInstance *backend,

Oh.

xen_block_drive_destroy() nests its Error *local_err in a conditional.

xen_block_device_destroy() has multiple Error *local_err.

In both cases, manual review is required to ensure the conversion is okay. I believe it is.

Note that the AWK script relies on diff showing the function name in @@ lines, which doesn't always work due to our coding style.

For the whole tree, I get some 30 spots. Feasible.

scripts/coccinelle/auto-propagated-errp.cocci line 170

Quoted 1 line
scripts/coccinelle/auto-propagated-errp.cocci new file +1 -0
+|

Second pattern: clear @errp after freeing it

scripts/coccinelle/auto-propagated-errp.cocci line 172

Quoted 2 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +2 -0
+
+// Convert error clearing functions

Suggest: Ensure @local_err is cleared on free

Quoted 6 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +6 -0
+(
+-    error_free(local_err);
++    error_free_errp(errp);
+|
+-    error_report_err(local_err);
++    error_report_errp(errp);

scripts/coccinelle/auto-propagated-errp.cocci line 188

Quoted 10 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +10 -0
+|
+-    error_reportf_err(local_err, args);
++    error_reportf_errp(errp, args);
+|
+-    warn_report_err(local_err);
++    warn_report_errp(errp);
+|
+-    warn_reportf_err(local_err, args);
++    warn_reportf_errp(errp, args);
+)

As you mention above, these guys don't exist, yet. Builds anyway, because this part of the rule is not used in this patch series. You don't want to omit it, because then the script becomes unsafe to use.

We could also open-code:

    // Convert error clearing functions
    (
    -    error_free(local_err);
    +    error_free(*errp);
    +    *errp = NULL;
    |
    ... and so forth ...
    )
Quoted 2 lines

Matter of taste. Whatever is easier to explain in the comments. Since you already wrote one...

I just feel that using helper functions is safer way..

Quoted 22 lines

We talked about extending this series slightly so these guys are used. I may still look into that.

scripts/coccinelle/auto-propagated-errp.cocci line 190

Quoted 2 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +2 -0
+?-    local_err = NULL;
+

The new helpers clear @local_err. Assignment now redundant, delete. Okay.

scripts/coccinelle/auto-propagated-errp.cocci line 191

Quoted 1 line
scripts/coccinelle/auto-propagated-errp.cocci new file +1 -0
+|

Third and fourth pattern: delete error_propagate()

scripts/coccinelle/auto-propagated-errp.cocci line 196

Quoted 5 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +5 -0
+-    error_propagate_prepend(errp, local_err, args);
++    error_prepend(errp, args);
+|
+-    error_propagate(errp, local_err);
+|

Fifth pattern: use @errp directly

Quoted 13 lines

scripts/coccinelle/auto-propagated-errp.cocci line 204

Quoted 8 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +8 -0
+-    &local_err
++    errp
+)
+     ...>
+ }
+
+// Convert remaining local_err usage. It should be different kinds of error
+// checking in if operators. We can't merge this into previous hunk, as this

In if conditionals, I suppose. It's the case for this patch. If I apply the script to the whole tree, the rule gets also applied in other contexts. The sentence might mislead as much as it helps. Keep it or delete it?

Maybe, just be more honest: "It should be ..., but it may be any other pattern, be careful"

Quoted 32 lines
Quoted 6 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +5 -0
+// conflicts with other substitutions in it (at least with "- local_err = NULL").
+@@
+identifier rule1.fn, rule1.local_err;
+symbol errp;
+@@

scripts/coccinelle/auto-propagated-errp.cocci line 219

Quoted 10 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +10 -0
+
+ fn(...)
+ {
+     <...
+-    local_err
++    *errp
+     ...>
+ }
+
+// Always use the same patter for checking error

s/patter/pattern/

Quoted 12 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +12 -0
+@@
+identifier rule1.fn;
+symbol errp;
+@@
+
+ fn(...)
+ {
+     <...
+-    *errp != NULL
++    *errp
+     ...>
+ }
Signature · 7 lines
Best regards,
Vladimir

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
#13 lore raw

Re: [PATCH v8 02/10] scripts: add coccinelle script to use auto propagated errp

Quoted 110 lines · Vladimir Sementsov-Ogievskiy
Quoted 100 lines · Markus Armbruster

Suggest

scripts: Coccinelle script to use auto-propagated errp

or

scripts: Coccinelle script to use ERRP_AUTO_PROPAGATE()

Quoted 8 lines · Vladimir Sementsov-Ogievskiy

Script adds ERRP_AUTO_PROPAGATE macro invocation where appropriate and does corresponding changes in code (look for details in include/qapi/error.h)

Usage example:
spatch --sp-file scripts/coccinelle/auto-propagated-errp.cocci \
--macro-file scripts/cocci-macro-file.h --in-place --no-show-diff \
blockdev-nbd.c qemu-nbd.c {block/nbd*,nbd/*,include/block/nbd*}.[hc]

Suggest FILES... instead of a specific set of files.

Quoted 32 lines

---

Cc Eric Blake <[email protected]>
Cc Kevin Wolf <[email protected]>
Cc Max Reitz <[email protected]>
Cc Greg Kurz <[email protected]>
Cc Christian Schoenebeck <[email protected]>
Cc Stefano Stabellini <[email protected]>
Cc Anthony Perard <[email protected]>
Cc Paul Durrant <[email protected]>
Cc Stefan Hajnoczi <[email protected]>
Cc "Philippe Mathieu-Daudé" <[email protected]>
Cc Laszlo Ersek <[email protected]>
Cc Gerd Hoffmann <[email protected]>
Cc Stefan Berger <[email protected]>
Cc Markus Armbruster <[email protected]>
Cc Michael Roth <[email protected]>
  include/qapi/error.h                          |   3 +
  scripts/coccinelle/auto-propagated-errp.cocci | 231 ++++++++++++++++++
  2 files changed, 234 insertions(+)

create mode 100644 scripts/coccinelle/auto-propagated-errp.cocci

include/qapi/error.h +0 -0
index bb9bcf02fb..fbfc6f1c0b 100644

include/qapi/error.h line 215

Quoted 5 lines
include/qapi/error.h +2 -0
@@ -211,6 +211,9 @@
   *         }
   *         ...
   *     }
+ *
+ * For mass conversion use script

mass-conversion (we're not converting mass, we're converting en masse)

scripts/coccinelle/auto-propagated-errp.cocci

Quoted 8 lines
include/qapi/error.h +1 -0
+ *   scripts/coccinelle/auto-propagated-errp.cocci
   */
    #ifndef ERROR_H
scripts/coccinelle/auto-propagated-errp.cocci new file +0 -0
new file mode 100644
index 0000000000..bff274bd6d

Preface to my review of this script: may aim isn't to make it bullet-proof. I want to (1) make it good enough (explained in a jiffie), and (2) automatically identify the spots where it still isn't obviously safe for manual review.

The latter may involve additional scripting. That's okay.

The script is good enough when the number of possibly unsafe spots is low enough for careful manual review.

When I ask for improvements that, in your opinion, go beyond "good enough", please push back. I'm sure we can work it out together.

Quoted 17 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +16 -0
@@ -0,0 +1,231 @@
+// Use ERRP_AUTO_PROPAGATE (see include/qapi/error.h)
+//
+// Copyright (c) 2020 Virtuozzo International GmbH.
+//
+// This program is free software; you can redistribute it and/or modify
+// it under the terms of the GNU General Public License as published by
+// the Free Software Foundation; either version 2 of the License, or
+// (at your option) any later version.
+//
+// This program is distributed in the hope that it will be useful,
+// but WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+// GNU General Public License for more details.
+//
+// You should have received a copy of the GNU General Public License
+// along with this program.  If not, see <http://www.gnu.org/licenses/>.
Quoted 8 lines

scripts/coccinelle/auto-propagated-errp.cocci line 21

Quoted 5 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +5 -0
+//
+// Usage example:
+// spatch --sp-file scripts/coccinelle/auto-propagated-errp.cocci \
+//  --macro-file scripts/cocci-macro-file.h --in-place --no-show-diff \
+//  --max-width 80 blockdev-nbd.c qemu-nbd.c \

You have --max-width 80 here, but not in the commit message. Default seems to be 78. Any particular reason to change it to 80?

Quoted 3 lines

Hmm. As I remember, without this parameter, reindenting doesn't work correctly. So, I'm OK with "--max-width 78", but I doubt that it will work without a parameter. Still, may be I'm wrong, we can check it.

If you can point to an example where --max-width helps, keep it, and update the commit message to match. Else, drop it.

Quoted 82 lines
Quoted 24 lines

scripts/coccinelle/auto-propagated-errp.cocci line 24

Quoted 4 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +3 -0
+//  {block/nbd*,nbd/*,include/block/nbd*}.[hc]
+
+// Switch unusual (Error **) parameter names to errp

Let's drop the parenthesis around Error **

scripts/coccinelle/auto-propagated-errp.cocci line 25

Quoted 1 line
scripts/coccinelle/auto-propagated-errp.cocci new file +1 -0
+// (this is necessary to use ERRP_AUTO_PROPAGATE).

Perhaps ERRP_AUTO_PROPAGATE() should be ERRP_AUTO_PROPAGATE(errp) to make the fact we're messing with @errp more obvious. Too late; I shouldn't rock the boat that much now.

scripts/coccinelle/auto-propagated-errp.cocci line 31

Quoted 6 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +6 -0
+//
+// Disable optional_qualifier to skip functions with "Error *const *errp"
+// parameter.
+//
+// Skip functions with "assert(_errp && *_errp)" statement, as they have
+// non generic semantics and may have unusual Error ** argument name for purpose

non-generic

for a purpose

Quoted 7 lines

Wrap comment lines around column 70, please. It's easier to read.

Maybe

    // Skip functions with "assert(_errp && *_errp)" statement, because that
    // signals unusual semantics, and the parameter name may well serve a
    // purpose.

Sounds good.

Quoted 42 lines

scripts/coccinelle/auto-propagated-errp.cocci line 35

Quoted 5 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +4 -0
+// (like nbd_iter_channel_error()).
+//
+// Skip util/error.c to not touch, for example, error_propagate and
+// error_propagate_prepend().

error_propagate()

I much appreciate your meticulous explanation of what you skip and why.

Quoted 9 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +9 -0
+@ depends on !(file in "util/error.c") disable optional_qualifier@
+identifier fn;
+identifier _errp != errp;
+@@
+
+ fn(...,
+-   Error **_errp
++   Error **errp
+    ,...)

scripts/coccinelle/auto-propagated-errp.cocci line 54

Quoted 10 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +10 -0
+ {
+(
+     ... when != assert(_errp && *_errp)
+&
+     <...
+-    _errp
++    errp
+     ...>
+)
+ }

This rule is required to make the actual transformations (below) work even for parameters with names other than @errp. I believe it's not used in this series. In fact, I can't see a use for it in the entire tree right now. Okay anyway.

scripts/coccinelle/auto-propagated-errp.cocci line 58

Quoted 4 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +4 -0
+
+// Add invocation of ERRP_AUTO_PROPAGATE to errp-functions where necessary
+//
+// Note, that without "when any" final "..." may not want to mach something

s/final "..." may not mach/the final "..." does not match/

Quoted 5 lines

scripts/coccinelle/auto-propagated-errp.cocci line 60

Quoted 2 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +2 -0
+// matched by previous pattern, i.e. the rule will not match double
+// error_prepend in control flow like in vfio_set_irq_signaling().

Can't say I fully understand Coccinelle there. I figure you came to this knowledge the hard way.

Quoted 5 lines

It's follows from smpl grammar document:

"Implicitly, “...” matches the shortest path between something that matches the pattern before the dots (or the beginning of the function, if there is nothing before the dots) and something that matches the pattern after the dots (or the end of the function, if there is nothing after the dots)."
...
"_when any_ removes the aforementioned constraint that “...” matches the shortest path"

Let me think that through.

The pattern with the cases other than error_prepend() omitted:

     fn(..., Error **errp, ...)
     {
    +   ERRP_AUTO_PROPAGATE();
        ...  when != ERRP_AUTO_PROPAGATE();
        error_prepend(errp, ...);
        ... when any
     }

Tail of vfio_set_irq_signaling():

        name = index_to_str(vbasedev, index);
        if (name) {
            error_prepend(errp, "%s-%d: ", name, subindex);
        } else {
            error_prepend(errp, "index %d-%d: ", index, subindex);
        }
        error_prepend(errp,
                      "Failed to %s %s eventfd signaling for interrupt ",
                      fd < 0 ? "tear down" : "set up", action_to_str(action));
        return ret;
    }

The pattern's first ... matches a "shortest" path to an error_prepend(), where "shortest" means "does not cross an error_prepend(). Its when clause makes us ignore functions that already use ERRP_AUTO_PROPAGATE().

There are two such "shortest" paths, one to the first error_prepend() in vfio_set_irq_signaling(), and one to the second. Neither path to the third one is not "shortest": they both cross one of the other two error_prepend().

The pattern' s second ... matches a path from a matched error_prepend() to the end of the function. There are two paths. Both cross the third error_prepend(). You need "when any" to make the pattern match anyway.

Alright, I think I got it. But now I'm paranoid about ... elsewhere. For instance, here's rule1 with error_propagate_prepend() omitted:

    // Match scenarios with propagation of local error to errp.
    @rule1 disable optional_qualifier exists@
    identifier fn, local_err;
    symbol errp;
    @@

     fn(..., Error **errp, ...)
     {
         ...
         Error *local_err = NULL;
         ...
         error_propagate(errp, local_err);
         ...
     }

The second and third ... won't match anything containing error_propagate(). What if a function has multiple error_propagate() on all paths? Like this one:

    extern foo(int, Error **);
    extern bar(int, Error **);

    void frob(Error **errp)
    {
        Error *local_err = NULL;
        int arg;

        foo(arg, errp);
        bar(arg, &local_err);
        error_propagate(errp, local_err);
        bar(arg + 1, &local_err);
        error_propagate(errp, local_err);
    }

This is actually a variation of error.h's "Receive and accumulate multiple errors (first one wins)" code snippet.

The Coccinelle script transforms it like this:

     void frob(Error **errp)
     {
    +    ERRP_AUTO_PROPAGATE();
         Error *local_err = NULL;
         int arg;

The rule that adds ERRP_AUTO_PROPAGATE() matches (it has ... when any), but rule1 does not, and we therefore don't convert any of the error_propagate().

The result isn't wrong, just useless.

Is this the worst case?

Possible improvement to the ERRP_AUTO_PROPAGATE() rule: don't use "... when any" in the error_propagate() case, only in the other cases. Would that help?

I think this is the only other rule with "..." matching control flow.

Quoted 86 lines
Quoted 25 lines

scripts/coccinelle/auto-propagated-errp.cocci line 64

Quoted 5 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +4 -0
+//
+// Note, "exists" says that we want apply rule even if it matches not on
+// all possible control flows (otherwise, it will not match standard pattern
+// when error_propagate() call is in if branch).

Learned something new. Example: kvm_set_kvm_shadow_mem().

Spelling it "exists disable optional_qualifier" would avoid giving readers the idea we're disabling "exists", but Coccinelle doesn't let us. Oh well.

scripts/coccinelle/auto-propagated-errp.cocci line 66

Quoted 2 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +2 -0
+@ disable optional_qualifier exists@
+identifier fn, local_err, errp;

I believe this causes

warning: line 98: errp, previously declared as a metavariable, is used as an identifier
warning: line 104: errp, previously declared as a metavariable, is used as an identifier
warning: line 106: errp, previously declared as a metavariable, is used as an identifier
warning: line 131: errp, previously declared as a metavariable, is used as an identifier
warning: line 192: errp, previously declared as a metavariable, is used as an identifier
warning: line 195: errp, previously declared as a metavariable, is used as an identifier
warning: line 228: errp, previously declared as a metavariable, is used as an identifier

Quoted 1 line

Making @errp symbol instead of identifier should fix this.

Hmm, I didn't see these warnings.. But yes, it should be symbol.

Quoted 57 lines
Quoted 32 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +31 -0
+@@
+
+ fn(..., Error **errp, ...)
+ {
++   ERRP_AUTO_PROPAGATE();
+    ...  when != ERRP_AUTO_PROPAGATE();
+(
+    error_append_hint(errp, ...);
+|
+    error_prepend(errp, ...);
+|
+    error_vprepend(errp, ...);
+|
+    Error *local_err = NULL;
+    ...
+(
+    error_propagate_prepend(errp, local_err, ...);
+|
+    error_propagate(errp, local_err);
+)
+)
+    ... when any
+ }
+
+
+// Match scenarios with propagation of local error to errp.
+@rule1 disable optional_qualifier exists@
+identifier fn, local_err;
+symbol errp;
+@@
+

scripts/coccinelle/auto-propagated-errp.cocci line 107

Quoted 10 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +10 -0
+ fn(..., Error **errp, ...)
+ {
+     ...
+     Error *local_err = NULL;
+     ...
+(
+    error_propagate_prepend(errp, local_err, ...);
+|
+    error_propagate(errp, local_err);
+)

Indentation off by one.

scripts/coccinelle/auto-propagated-errp.cocci line 111

Quoted 4 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +4 -0
+     ...
+ }
+
+// Convert special case with goto in separate.

s/in separate/separately/

scripts/coccinelle/auto-propagated-errp.cocci line 113

Quoted 2 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +2 -0
+// We can probably merge this into the following hunk with help of ( | )
+// operator, but it significantly reduce performance on block.c parsing (or it

s/reduce/reduces/

Quoted 6 lines
Quoted 4 lines

scripts/coccinelle/auto-propagated-errp.cocci line 114

Quoted 1 line
scripts/coccinelle/auto-propagated-errp.cocci new file +1 -0
+// hangs, I don't know)

Sounds like you tried to merge this into the following hunk, but then spatch took so long on block.c that you killed it. Correct?

Yes.

I'd say something like "I tried merging this into the following rule the obvious way, but it made Coccinelle hang on block.c."

Quoted 32 lines
Quoted 7 lines

scripts/coccinelle/auto-propagated-errp.cocci line 118

Quoted 5 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +4 -0
+//
+// Note interesting thing: if we don't do it here, and try to fixup "out: }"
+// things later after all transformations (the rule will be the same, just
+// without error_propagate() call), coccinelle fails to match this "out: }".

Weird, but not worth further investigation.

It partially match to the idea which I saw somewhere in coccinelle documentation, that coccinelle converts correct C code to correct C code. "out: }" is an example of incorrect, impossible code flow, and coccinelle can't work with it... But it's just a thought.

Quoted 10 lines
Quoted 10 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +9 -0
+@@
+identifier rule1.fn, rule1.local_err, out;
+symbol errp;
+@@
+
+ fn(...)
+ {
+     <...
+-    goto out;
Quoted 8 lines

scripts/coccinelle/auto-propagated-errp.cocci line 131

Quoted 4 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +4 -0
++    return;
+     ...>
+- out:
+-    error_propagate(errp, local_err);

You neglect to match error_propagate_prepend(). Okay, because (1) that pattern doesn't occur in the tree right now, and (2) if it gets added, gcc will complain.

Quoted 2 lines

No, because it should not removed. error_propagate_prepend should be converted to prepend, not removed. So, corresponding gotos should not be removed as well.

You're right.

Quoted 30 lines
Quoted 30 lines

scripts/coccinelle/auto-propagated-errp.cocci line 134

Quoted 4 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +3 -0
+ }
+
+// Convert most of local_err related staff.

s/staff/stuff/

scripts/coccinelle/auto-propagated-errp.cocci line 141

Quoted 7 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +7 -0
+//
+// Note, that we update everything related to matched by rule1 function name
+// and local_err name. We may match something not related to the pattern
+// matched by rule1. For example, local_err may be defined with the same name
+// in different blocks inside one function, and in one block follow the
+// propagation pattern and in other block doesn't. Or we may have several
+// functions with the same name (for different configurations).

Context: rule1 matches functions that have all three of

* an Error **errp parameter

* an Error *local_err = NULL variable declaration

* an error_propagate(errp, local_err) or error_propagate_prepend(errp, local_err, ...) expression, where @errp is the parameter and @local_err is the variable.

If I understand you correctly, you're pointing out two potential issues:

1. This rule can match functions rule1 does not match if there is another function with the same name that rule1 does match.

Quoted 7 lines
Quoted 5 lines

2. This rule matches in the entire function matched by rule1, even when parts of that function use a different @errp or @local_err.

I figure these apply to all rules with identifier rule1.fn, not just this one. Correct?

Yes.

Thanks!

Quoted 30 lines
Quoted 21 lines

Regarding 1. There must be a better way to chain rules together, but I don't know it.
Can we make Coccinelle at least warn us when it converts
multiple functions with the same name? What about this:

    @initialize:python@
    @@
    fnprev = {}

    def pr(fn, p):
        print("### %s:%s: %s()" % (p[0].file, p[0].line, fn))

    @r@
    identifier rule1.fn;
    position p;
    @@
     fn(...)@p
     {
         ...
     }
Quoted 8 lines
    @script:python@
        fn << rule1.fn;
        p << r.p;
    @@
    if fn not in fnprev:
        fnprev[fn] = p
    else:
        if fnprev[fn]:
Quoted 7 lines

hmm, the condition can't be false

Quoted 3 lines
            pr(fn, fnprev[fn])
            fnprev[fn] = None
        pr(fn, p)

and we'll miss next duplication..

The idea is

first instance of fn:
fn not in fnprev
fnprev[fn] = position of instance
don't print
second instance:
fnprev[fn] is the position of the first instance print first two instances
subsequent instances: fnprev[fn] is None
print this instance

I might have screwed up the coding, of course :)

Quoted 168 lines

But I like the idea.

Quoted 125 lines

For each function @fn matched by rule1, fncnt[fn] is an upper limit of the number of functions with the same name we touch. If it's more than one, we print.

Reports about a dozen function names for the whole tree in my testing. Inspecting the changes to them manually is feasible. None of them are in files touched by this series.

The line printed for the first match is pretty useless for me: it points to a Coccinelle temporary file *shrug*.

Regarding 2. Shadowing @errp or @local_err would be in bad taste, and I sure hope we don't do that. Multiple @local_err variables... hmm. Perhaps we could again concoct some script rules to lead us to spots to check manually. See below for my attempt.

What's the worst that could happen if we blindly converted such code? The answer to that question tells us how hard to work on finding and checking these guys.

Quoted 17 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +17 -0
+//
+// Note also that errp-cleaning functions
+//   error_free_errp
+//   error_report_errp
+//   error_reportf_errp
+//   warn_report_errp
+//   warn_reportf_errp
+// are not yet implemented. They must call corresponding Error* - freeing
+// function and then set *errp to NULL, to avoid further propagation to
+// original errp (consider ERRP_AUTO_PROPAGATE in use).
+// For example, error_free_errp may look like this:
+//
+//    void error_free_errp(Error **errp)
+//    {
+//        error_free(*errp);
+//        *errp = NULL;
+//    }

scripts/coccinelle/auto-propagated-errp.cocci line 168

Quoted 10 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +10 -0
+@ exists@
+identifier rule1.fn, rule1.local_err;
+expression list args;
+symbol errp;
+@@
+
+ fn(...)
+ {
+     <...
+(

Each of the following patterns applies anywhere in the function.

First pattern: delete @local_err

scripts/coccinelle/auto-propagated-errp.cocci line 169

Quoted 1 line
scripts/coccinelle/auto-propagated-errp.cocci new file +1 -0
+-    Error *local_err = NULL;

Common case: occurs just once, not nested. Anything else is suspicious.

Both can be detected in the resulting patches with a bit of AWK wizardry:

     $ git-diff -U0 master..review-error-v8 | awk '/^@@ / { ctx = $5; for (i = 6; i <= NF; i++) ctx = ctx " " $i; if (ctx != octx) { octx = ctx; n = 0 } } /^- *Error *\* *[A-Za-z0-9_]+ *= *NULL;/ { if (index($0, "E") > 6) print "nested\n    " ctx; if (n) print "more than one\n    " ctx; n++ }'
     nested
         static void xen_block_drive_destroy(XenBlockDrive *drive, Error **errp)
     nested
         static void xen_block_device_destroy(XenBackendInstance *backend,
     nested
         static void xen_block_device_destroy(XenBackendInstance *backend,
     more than one
         static void xen_block_device_destroy(XenBackendInstance *backend,

Oh.

xen_block_drive_destroy() nests its Error *local_err in a conditional.

xen_block_device_destroy() has multiple Error *local_err.

In both cases, manual review is required to ensure the conversion is okay. I believe it is.

Note that the AWK script relies on diff showing the function name in @@ lines, which doesn't always work due to our coding style.

For the whole tree, I get some 30 spots. Feasible.

scripts/coccinelle/auto-propagated-errp.cocci line 170

Quoted 1 line
scripts/coccinelle/auto-propagated-errp.cocci new file +1 -0
+|

Second pattern: clear @errp after freeing it

scripts/coccinelle/auto-propagated-errp.cocci line 172

Quoted 2 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +2 -0
+
+// Convert error clearing functions

Suggest: Ensure @local_err is cleared on free

Quoted 6 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +6 -0
+(
+-    error_free(local_err);
++    error_free_errp(errp);
+|
+-    error_report_err(local_err);
++    error_report_errp(errp);

scripts/coccinelle/auto-propagated-errp.cocci line 188

Quoted 10 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +10 -0
+|
+-    error_reportf_err(local_err, args);
++    error_reportf_errp(errp, args);
+|
+-    warn_report_err(local_err);
++    warn_report_errp(errp);
+|
+-    warn_reportf_err(local_err, args);
++    warn_reportf_errp(errp, args);
+)

As you mention above, these guys don't exist, yet. Builds anyway, because this part of the rule is not used in this patch series. You don't want to omit it, because then the script becomes unsafe to use.

We could also open-code:

    // Convert error clearing functions
    (
    -    error_free(local_err);
    +    error_free(*errp);
    +    *errp = NULL;
    |
    ... and so forth ...
    )
Quoted 2 lines

Matter of taste. Whatever is easier to explain in the comments. Since you already wrote one...

I just feel that using helper functions is safer way..

Quoted 22 lines

We talked about extending this series slightly so these guys are used. I may still look into that.

scripts/coccinelle/auto-propagated-errp.cocci line 190

Quoted 2 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +2 -0
+?-    local_err = NULL;
+

The new helpers clear @local_err. Assignment now redundant, delete. Okay.

scripts/coccinelle/auto-propagated-errp.cocci line 191

Quoted 1 line
scripts/coccinelle/auto-propagated-errp.cocci new file +1 -0
+|

Third and fourth pattern: delete error_propagate()

scripts/coccinelle/auto-propagated-errp.cocci line 196

Quoted 5 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +5 -0
+-    error_propagate_prepend(errp, local_err, args);
++    error_prepend(errp, args);
+|
+-    error_propagate(errp, local_err);
+|

Fifth pattern: use @errp directly

Quoted 13 lines

scripts/coccinelle/auto-propagated-errp.cocci line 204

Quoted 8 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +8 -0
+-    &local_err
++    errp
+)
+     ...>
+ }
+
+// Convert remaining local_err usage. It should be different kinds of error
+// checking in if operators. We can't merge this into previous hunk, as this

In if conditionals, I suppose. It's the case for this patch. If I apply the script to the whole tree, the rule gets also applied in other contexts. The sentence might mislead as much as it helps. Keep it or delete it?

Quoted 1 line

Maybe, just be more honest: "It should be ..., but it may be any other pattern, be careful"

"Need to be careful" means "needs careful manual review", which I believe is not feasible; see "Preface to my review of this script" above.

But do we really need to be careful here?

This rule should apply only where we added ERRP_AUTO_PROPAGATE().

Except when rule chaining via function name fails us, but we plan to detect that and review manually, so let's ignore this issue here.

Thanks to ERRP_AUTO_PROPAGATE(), @errp is not null. Enabling replacement of @local_err by @errp is its whole point.

What exactly do we need to be careful about?

Quoted 20 lines
Quoted 20 lines
Quoted 7 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +5 -0
+// conflicts with other substitutions in it (at least with "- local_err = NULL").
+@@
+identifier rule1.fn, rule1.local_err;
+symbol errp;
+@@

scripts/coccinelle/auto-propagated-errp.cocci line 219

Quoted 10 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +10 -0
+
+ fn(...)
+ {
+     <...
+-    local_err
++    *errp
+     ...>
+ }
+
+// Always use the same patter for checking error

s/patter/pattern/

Quoted 13 lines
Quoted 4 lines
Quoted 4 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +4 -0
+@@
+identifier rule1.fn;
+symbol errp;
+@@
Quoted 9 lines

scripts/coccinelle/auto-propagated-errp.cocci line 231

Quoted 8 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +8 -0
+
+ fn(...)
+ {
+     <...
+-    *errp != NULL
++    *errp
+     ...>
+ }

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

#14 lore raw

Re: [PATCH v8 02/10] scripts: add coccinelle script to use auto propagated errp

Quoted 264 lines · Markus Armbruster
Quoted 110 lines · Vladimir Sementsov-Ogievskiy
Quoted 100 lines · Markus Armbruster

Suggest

scripts: Coccinelle script to use auto-propagated errp

or

scripts: Coccinelle script to use ERRP_AUTO_PROPAGATE()

Quoted 8 lines · Vladimir Sementsov-Ogievskiy

Script adds ERRP_AUTO_PROPAGATE macro invocation where appropriate and does corresponding changes in code (look for details in include/qapi/error.h)

Usage example:

spatch --sp-file scripts/coccinelle/auto-propagated-errp.cocci \
   --macro-file scripts/cocci-macro-file.h --in-place --no-show-diff \
   blockdev-nbd.c qemu-nbd.c {block/nbd*,nbd/*,include/block/nbd*}.[hc]

Suggest FILES... instead of a specific set of files.

Quoted 32 lines

---

Cc Eric Blake <[email protected]>
Cc Kevin Wolf <[email protected]>
Cc Max Reitz <[email protected]>
Cc Greg Kurz <[email protected]>
Cc Christian Schoenebeck <[email protected]>
Cc Stefano Stabellini <[email protected]>
Cc Anthony Perard <[email protected]>
Cc Paul Durrant <[email protected]>
Cc Stefan Hajnoczi <[email protected]>
Cc "Philippe Mathieu-Daudé" <[email protected]>
Cc Laszlo Ersek <[email protected]>
Cc Gerd Hoffmann <[email protected]>
Cc Stefan Berger <[email protected]>
Cc Markus Armbruster <[email protected]>
Cc Michael Roth <[email protected]>
   include/qapi/error.h                          |   3 +
   scripts/coccinelle/auto-propagated-errp.cocci | 231 ++++++++++++++++++
   2 files changed, 234 insertions(+)

create mode 100644 scripts/coccinelle/auto-propagated-errp.cocci

include/qapi/error.h +0 -0
index bb9bcf02fb..fbfc6f1c0b 100644

include/qapi/error.h line 215

Quoted 5 lines
include/qapi/error.h +2 -0
@@ -211,6 +211,9 @@
    *         }
    *         ...
    *     }
+ *
+ * For mass conversion use script

mass-conversion (we're not converting mass, we're converting en masse)

scripts/coccinelle/auto-propagated-errp.cocci

Quoted 8 lines
include/qapi/error.h +1 -0
+ *   scripts/coccinelle/auto-propagated-errp.cocci
    */
     #ifndef ERROR_H
scripts/coccinelle/auto-propagated-errp.cocci new file +0 -0
new file mode 100644
index 0000000000..bff274bd6d

Preface to my review of this script: may aim isn't to make it bullet-proof. I want to (1) make it good enough (explained in a jiffie), and (2) automatically identify the spots where it still isn't obviously safe for manual review.

The latter may involve additional scripting. That's okay.

The script is good enough when the number of possibly unsafe spots is low enough for careful manual review.

When I ask for improvements that, in your opinion, go beyond "good enough", please push back. I'm sure we can work it out together.

Quoted 17 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +16 -0
@@ -0,0 +1,231 @@
+// Use ERRP_AUTO_PROPAGATE (see include/qapi/error.h)
+//
+// Copyright (c) 2020 Virtuozzo International GmbH.
+//
+// This program is free software; you can redistribute it and/or modify
+// it under the terms of the GNU General Public License as published by
+// the Free Software Foundation; either version 2 of the License, or
+// (at your option) any later version.
+//
+// This program is distributed in the hope that it will be useful,
+// but WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+// GNU General Public License for more details.
+//
+// You should have received a copy of the GNU General Public License
+// along with this program.  If not, see <http://www.gnu.org/licenses/>.
Quoted 8 lines

scripts/coccinelle/auto-propagated-errp.cocci line 21

Quoted 5 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +5 -0
+//
+// Usage example:
+// spatch --sp-file scripts/coccinelle/auto-propagated-errp.cocci \
+//  --macro-file scripts/cocci-macro-file.h --in-place --no-show-diff \
+//  --max-width 80 blockdev-nbd.c qemu-nbd.c \

You have --max-width 80 here, but not in the commit message. Default seems to be 78. Any particular reason to change it to 80?

Quoted 3 lines

Hmm. As I remember, without this parameter, reindenting doesn't work correctly. So, I'm OK with "--max-width 78", but I doubt that it will work without a parameter. Still, may be I'm wrong, we can check it.

If you can point to an example where --max-width helps, keep it, and update the commit message to match. Else, drop it.

Quoted 82 lines
Quoted 24 lines

scripts/coccinelle/auto-propagated-errp.cocci line 24

Quoted 4 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +3 -0
+//  {block/nbd*,nbd/*,include/block/nbd*}.[hc]
+
+// Switch unusual (Error **) parameter names to errp

Let's drop the parenthesis around Error **

scripts/coccinelle/auto-propagated-errp.cocci line 25

Quoted 1 line
scripts/coccinelle/auto-propagated-errp.cocci new file +1 -0
+// (this is necessary to use ERRP_AUTO_PROPAGATE).

Perhaps ERRP_AUTO_PROPAGATE() should be ERRP_AUTO_PROPAGATE(errp) to make the fact we're messing with @errp more obvious. Too late; I shouldn't rock the boat that much now.

scripts/coccinelle/auto-propagated-errp.cocci line 31

Quoted 6 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +6 -0
+//
+// Disable optional_qualifier to skip functions with "Error *const *errp"
+// parameter.
+//
+// Skip functions with "assert(_errp && *_errp)" statement, as they have
+// non generic semantics and may have unusual Error ** argument name for purpose

non-generic

for a purpose

Quoted 7 lines

Wrap comment lines around column 70, please. It's easier to read.

Maybe

     // Skip functions with "assert(_errp && *_errp)" statement, because that
     // signals unusual semantics, and the parameter name may well serve a
     // purpose.

Sounds good.

Quoted 42 lines

scripts/coccinelle/auto-propagated-errp.cocci line 35

Quoted 5 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +4 -0
+// (like nbd_iter_channel_error()).
+//
+// Skip util/error.c to not touch, for example, error_propagate and
+// error_propagate_prepend().

error_propagate()

I much appreciate your meticulous explanation of what you skip and why.

Quoted 9 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +9 -0
+@ depends on !(file in "util/error.c") disable optional_qualifier@
+identifier fn;
+identifier _errp != errp;
+@@
+
+ fn(...,
+-   Error **_errp
++   Error **errp
+    ,...)

scripts/coccinelle/auto-propagated-errp.cocci line 54

Quoted 10 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +10 -0
+ {
+(
+     ... when != assert(_errp && *_errp)
+&
+     <...
+-    _errp
++    errp
+     ...>
+)
+ }

This rule is required to make the actual transformations (below) work even for parameters with names other than @errp. I believe it's not used in this series. In fact, I can't see a use for it in the entire tree right now. Okay anyway.

scripts/coccinelle/auto-propagated-errp.cocci line 58

Quoted 4 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +4 -0
+
+// Add invocation of ERRP_AUTO_PROPAGATE to errp-functions where necessary
+//
+// Note, that without "when any" final "..." may not want to mach something

s/final "..." may not mach/the final "..." does not match/

Quoted 5 lines

scripts/coccinelle/auto-propagated-errp.cocci line 60

Quoted 2 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +2 -0
+// matched by previous pattern, i.e. the rule will not match double
+// error_prepend in control flow like in vfio_set_irq_signaling().

Can't say I fully understand Coccinelle there. I figure you came to this knowledge the hard way.

Quoted 5 lines

It's follows from smpl grammar document:

"Implicitly, “...” matches the shortest path between something that matches the pattern before the dots (or the beginning of the function, if there is nothing before the dots) and something that matches the pattern after the dots (or the end of the function, if there is nothing after the dots)."
...
"_when any_ removes the aforementioned constraint that “...” matches the shortest path"

Let me think that through.

The pattern with the cases other than error_prepend() omitted:

      fn(..., Error **errp, ...)
      {
     +   ERRP_AUTO_PROPAGATE();
         ...  when != ERRP_AUTO_PROPAGATE();
         error_prepend(errp, ...);
         ... when any
      }

Tail of vfio_set_irq_signaling():

         name = index_to_str(vbasedev, index);
         if (name) {
             error_prepend(errp, "%s-%d: ", name, subindex);
         } else {
             error_prepend(errp, "index %d-%d: ", index, subindex);
         }
         error_prepend(errp,
                       "Failed to %s %s eventfd signaling for interrupt ",
                       fd < 0 ? "tear down" : "set up", action_to_str(action));
         return ret;
     }

The pattern's first ... matches a "shortest" path to an error_prepend(), where "shortest" means "does not cross an error_prepend(). Its when clause makes us ignore functions that already use ERRP_AUTO_PROPAGATE().

There are two such "shortest" paths, one to the first error_prepend() in vfio_set_irq_signaling(), and one to the second. Neither path to the third one is not "shortest": they both cross one of the other two error_prepend().

The pattern' s second ... matches a path from a matched error_prepend() to the end of the function. There are two paths. Both cross the third error_prepend(). You need "when any" to make the pattern match anyway.

Alright, I think I got it. But now I'm paranoid about ... elsewhere. For instance, here's rule1 with error_propagate_prepend() omitted:

     // Match scenarios with propagation of local error to errp.
     @rule1 disable optional_qualifier exists@
     identifier fn, local_err;
     symbol errp;
     @@

      fn(..., Error **errp, ...)
      {
          ...
          Error *local_err = NULL;
          ...
          error_propagate(errp, local_err);
          ...
      }
Quoted 3 lines

The second and third ... won't match anything containing error_propagate(). What if a function has multiple error_propagate() on all paths?

I thought about this, but decided that double error propagation is a strange pattern, and may be better not match it...

Quoted 17 lines

Like this one:

     extern foo(int, Error **);
     extern bar(int, Error **);

     void frob(Error **errp)
     {
         Error *local_err = NULL;
         int arg;

         foo(arg, errp);
         bar(arg, &local_err);
         error_propagate(errp, local_err);
         bar(arg + 1, &local_err);
         error_propagate(errp, local_err);
     }
Quoted 2 lines

This is actually a variation of error.h's "Receive and accumulate multiple errors (first one wins)" code snippet.

ah yes, we can propagate to already filled errp, which just clean local_err.

Quoted 13 lines

The Coccinelle script transforms it like this:

      void frob(Error **errp)
      {
     +    ERRP_AUTO_PROPAGATE();
          Error *local_err = NULL;
          int arg;

The rule that adds ERRP_AUTO_PROPAGATE() matches (it has ... when any), but rule1 does not, and we therefore don't convert any of the error_propagate().

Quoted 7 lines

The result isn't wrong, just useless.

Is this the worst case?

Possible improvement to the ERRP_AUTO_PROPAGATE() rule: don't use "... when any" in the error_propagate() case, only in the other cases. Would that help?

I think not, as it will anyway match functions with error_prepend (and any number of following error_propagate calls)...

Quoted 406 lines

I think this is the only other rule with "..." matching control flow.

Quoted 86 lines
Quoted 25 lines

scripts/coccinelle/auto-propagated-errp.cocci line 64

Quoted 5 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +4 -0
+//
+// Note, "exists" says that we want apply rule even if it matches not on
+// all possible control flows (otherwise, it will not match standard pattern
+// when error_propagate() call is in if branch).

Learned something new. Example: kvm_set_kvm_shadow_mem().

Spelling it "exists disable optional_qualifier" would avoid giving readers the idea we're disabling "exists", but Coccinelle doesn't let us. Oh well.

scripts/coccinelle/auto-propagated-errp.cocci line 66

Quoted 2 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +2 -0
+@ disable optional_qualifier exists@
+identifier fn, local_err, errp;

I believe this causes

warning: line 98: errp, previously declared as a metavariable, is used as an identifier
warning: line 104: errp, previously declared as a metavariable, is used as an identifier
warning: line 106: errp, previously declared as a metavariable, is used as an identifier
warning: line 131: errp, previously declared as a metavariable, is used as an identifier
warning: line 192: errp, previously declared as a metavariable, is used as an identifier
warning: line 195: errp, previously declared as a metavariable, is used as an identifier
warning: line 228: errp, previously declared as a metavariable, is used as an identifier

Quoted 1 line

Making @errp symbol instead of identifier should fix this.

Hmm, I didn't see these warnings.. But yes, it should be symbol.

Quoted 57 lines
Quoted 32 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +31 -0
+@@
+
+ fn(..., Error **errp, ...)
+ {
++   ERRP_AUTO_PROPAGATE();
+    ...  when != ERRP_AUTO_PROPAGATE();
+(
+    error_append_hint(errp, ...);
+|
+    error_prepend(errp, ...);
+|
+    error_vprepend(errp, ...);
+|
+    Error *local_err = NULL;
+    ...
+(
+    error_propagate_prepend(errp, local_err, ...);
+|
+    error_propagate(errp, local_err);
+)
+)
+    ... when any
+ }
+
+
+// Match scenarios with propagation of local error to errp.
+@rule1 disable optional_qualifier exists@
+identifier fn, local_err;
+symbol errp;
+@@
+

scripts/coccinelle/auto-propagated-errp.cocci line 107

Quoted 10 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +10 -0
+ fn(..., Error **errp, ...)
+ {
+     ...
+     Error *local_err = NULL;
+     ...
+(
+    error_propagate_prepend(errp, local_err, ...);
+|
+    error_propagate(errp, local_err);
+)

Indentation off by one.

scripts/coccinelle/auto-propagated-errp.cocci line 111

Quoted 4 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +4 -0
+     ...
+ }
+
+// Convert special case with goto in separate.

s/in separate/separately/

scripts/coccinelle/auto-propagated-errp.cocci line 113

Quoted 2 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +2 -0
+// We can probably merge this into the following hunk with help of ( | )
+// operator, but it significantly reduce performance on block.c parsing (or it

s/reduce/reduces/

Quoted 6 lines
Quoted 4 lines

scripts/coccinelle/auto-propagated-errp.cocci line 114

Quoted 1 line
scripts/coccinelle/auto-propagated-errp.cocci new file +1 -0
+// hangs, I don't know)

Sounds like you tried to merge this into the following hunk, but then spatch took so long on block.c that you killed it. Correct?

Yes.

I'd say something like "I tried merging this into the following rule the obvious way, but it made Coccinelle hang on block.c."

Quoted 32 lines
Quoted 7 lines

scripts/coccinelle/auto-propagated-errp.cocci line 118

Quoted 5 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +4 -0
+//
+// Note interesting thing: if we don't do it here, and try to fixup "out: }"
+// things later after all transformations (the rule will be the same, just
+// without error_propagate() call), coccinelle fails to match this "out: }".

Weird, but not worth further investigation.

It partially match to the idea which I saw somewhere in coccinelle documentation, that coccinelle converts correct C code to correct C code. "out: }" is an example of incorrect, impossible code flow, and coccinelle can't work with it... But it's just a thought.

Quoted 10 lines
Quoted 10 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +9 -0
+@@
+identifier rule1.fn, rule1.local_err, out;
+symbol errp;
+@@
+
+ fn(...)
+ {
+     <...
+-    goto out;
Quoted 8 lines

scripts/coccinelle/auto-propagated-errp.cocci line 131

Quoted 4 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +4 -0
++    return;
+     ...>
+- out:
+-    error_propagate(errp, local_err);

You neglect to match error_propagate_prepend(). Okay, because (1) that pattern doesn't occur in the tree right now, and (2) if it gets added, gcc will complain.

Quoted 2 lines

No, because it should not removed. error_propagate_prepend should be converted to prepend, not removed. So, corresponding gotos should not be removed as well.

You're right.

Quoted 30 lines
Quoted 30 lines

scripts/coccinelle/auto-propagated-errp.cocci line 134

Quoted 4 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +3 -0
+ }
+
+// Convert most of local_err related staff.

s/staff/stuff/

scripts/coccinelle/auto-propagated-errp.cocci line 141

Quoted 7 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +7 -0
+//
+// Note, that we update everything related to matched by rule1 function name
+// and local_err name. We may match something not related to the pattern
+// matched by rule1. For example, local_err may be defined with the same name
+// in different blocks inside one function, and in one block follow the
+// propagation pattern and in other block doesn't. Or we may have several
+// functions with the same name (for different configurations).

Context: rule1 matches functions that have all three of

* an Error **errp parameter

* an Error *local_err = NULL variable declaration

* an error_propagate(errp, local_err) or error_propagate_prepend(errp, local_err, ...) expression, where @errp is the parameter and @local_err is the variable.

If I understand you correctly, you're pointing out two potential issues:

1. This rule can match functions rule1 does not match if there is another function with the same name that rule1 does match.

Quoted 7 lines
Quoted 5 lines

2. This rule matches in the entire function matched by rule1, even when parts of that function use a different @errp or @local_err.

I figure these apply to all rules with identifier rule1.fn, not just this one. Correct?

Yes.

Thanks!

Quoted 30 lines
Quoted 21 lines

Regarding 1. There must be a better way to chain rules together, but I don't know it.
Can we make Coccinelle at least warn us when it converts
multiple functions with the same name? What about this:

     @initialize:python@
     @@
     fnprev = {}

     def pr(fn, p):
         print("### %s:%s: %s()" % (p[0].file, p[0].line, fn))

     @r@
     identifier rule1.fn;
     position p;
     @@
      fn(...)@p
      {
          ...
      }
Quoted 8 lines
     @script:python@
         fn << rule1.fn;
         p << r.p;
     @@
     if fn not in fnprev:
         fnprev[fn] = p
     else:
         if fnprev[fn]:
Quoted 7 lines

hmm, the condition can't be false

Quoted 3 lines
             pr(fn, fnprev[fn])
             fnprev[fn] = None
         pr(fn, p)

and we'll miss next duplication..

The idea is

first instance of fn:
fn not in fnprev
fnprev[fn] = position of instance
don't print
second instance:
fnprev[fn] is the position of the first instance print first two instances
subsequent instances: fnprev[fn] is None
print this instance

I might have screwed up the coding, of course :)

Quoted 168 lines

But I like the idea.

Quoted 125 lines

For each function @fn matched by rule1, fncnt[fn] is an upper limit of the number of functions with the same name we touch. If it's more than one, we print.

Reports about a dozen function names for the whole tree in my testing. Inspecting the changes to them manually is feasible. None of them are in files touched by this series.

The line printed for the first match is pretty useless for me: it points to a Coccinelle temporary file *shrug*.

Regarding 2. Shadowing @errp or @local_err would be in bad taste, and I sure hope we don't do that. Multiple @local_err variables... hmm. Perhaps we could again concoct some script rules to lead us to spots to check manually. See below for my attempt.

What's the worst that could happen if we blindly converted such code? The answer to that question tells us how hard to work on finding and checking these guys.

Quoted 17 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +17 -0
+//
+// Note also that errp-cleaning functions
+//   error_free_errp
+//   error_report_errp
+//   error_reportf_errp
+//   warn_report_errp
+//   warn_reportf_errp
+// are not yet implemented. They must call corresponding Error* - freeing
+// function and then set *errp to NULL, to avoid further propagation to
+// original errp (consider ERRP_AUTO_PROPAGATE in use).
+// For example, error_free_errp may look like this:
+//
+//    void error_free_errp(Error **errp)
+//    {
+//        error_free(*errp);
+//        *errp = NULL;
+//    }

scripts/coccinelle/auto-propagated-errp.cocci line 168

Quoted 10 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +10 -0
+@ exists@
+identifier rule1.fn, rule1.local_err;
+expression list args;
+symbol errp;
+@@
+
+ fn(...)
+ {
+     <...
+(

Each of the following patterns applies anywhere in the function.

First pattern: delete @local_err

scripts/coccinelle/auto-propagated-errp.cocci line 169

Quoted 1 line
scripts/coccinelle/auto-propagated-errp.cocci new file +1 -0
+-    Error *local_err = NULL;

Common case: occurs just once, not nested. Anything else is suspicious.

Both can be detected in the resulting patches with a bit of AWK wizardry:

      $ git-diff -U0 master..review-error-v8 | awk '/^@@ / { ctx = $5; for (i = 6; i <= NF; i++) ctx = ctx " " $i; if (ctx != octx) { octx = ctx; n = 0 } } /^- *Error *\* *[A-Za-z0-9_]+ *= *NULL;/ { if (index($0, "E") > 6) print "nested\n    " ctx; if (n) print "more than one\n    " ctx; n++ }'
      nested
          static void xen_block_drive_destroy(XenBlockDrive *drive, Error **errp)
      nested
          static void xen_block_device_destroy(XenBackendInstance *backend,
      nested
          static void xen_block_device_destroy(XenBackendInstance *backend,
      more than one
          static void xen_block_device_destroy(XenBackendInstance *backend,

Oh.

xen_block_drive_destroy() nests its Error *local_err in a conditional.

xen_block_device_destroy() has multiple Error *local_err.

In both cases, manual review is required to ensure the conversion is okay. I believe it is.

Note that the AWK script relies on diff showing the function name in @@ lines, which doesn't always work due to our coding style.

For the whole tree, I get some 30 spots. Feasible.

scripts/coccinelle/auto-propagated-errp.cocci line 170

Quoted 1 line
scripts/coccinelle/auto-propagated-errp.cocci new file +1 -0
+|

Second pattern: clear @errp after freeing it

scripts/coccinelle/auto-propagated-errp.cocci line 172

Quoted 2 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +2 -0
+
+// Convert error clearing functions

Suggest: Ensure @local_err is cleared on free

Quoted 6 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +6 -0
+(
+-    error_free(local_err);
++    error_free_errp(errp);
+|
+-    error_report_err(local_err);
++    error_report_errp(errp);

scripts/coccinelle/auto-propagated-errp.cocci line 188

Quoted 10 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +10 -0
+|
+-    error_reportf_err(local_err, args);
++    error_reportf_errp(errp, args);
+|
+-    warn_report_err(local_err);
++    warn_report_errp(errp);
+|
+-    warn_reportf_err(local_err, args);
++    warn_reportf_errp(errp, args);
+)

As you mention above, these guys don't exist, yet. Builds anyway, because this part of the rule is not used in this patch series. You don't want to omit it, because then the script becomes unsafe to use.

We could also open-code:

     // Convert error clearing functions
     (
     -    error_free(local_err);
     +    error_free(*errp);
     +    *errp = NULL;
     |
     ... and so forth ...
     )
Quoted 2 lines

Matter of taste. Whatever is easier to explain in the comments. Since you already wrote one...

I just feel that using helper functions is safer way..

Quoted 22 lines

We talked about extending this series slightly so these guys are used. I may still look into that.

scripts/coccinelle/auto-propagated-errp.cocci line 190

Quoted 2 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +2 -0
+?-    local_err = NULL;
+

The new helpers clear @local_err. Assignment now redundant, delete. Okay.

scripts/coccinelle/auto-propagated-errp.cocci line 191

Quoted 1 line
scripts/coccinelle/auto-propagated-errp.cocci new file +1 -0
+|

Third and fourth pattern: delete error_propagate()

scripts/coccinelle/auto-propagated-errp.cocci line 196

Quoted 5 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +5 -0
+-    error_propagate_prepend(errp, local_err, args);
++    error_prepend(errp, args);
+|
+-    error_propagate(errp, local_err);
+|

Fifth pattern: use @errp directly

Quoted 13 lines

scripts/coccinelle/auto-propagated-errp.cocci line 204

Quoted 8 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +8 -0
+-    &local_err
++    errp
+)
+     ...>
+ }
+
+// Convert remaining local_err usage. It should be different kinds of error
+// checking in if operators. We can't merge this into previous hunk, as this

In if conditionals, I suppose. It's the case for this patch. If I apply the script to the whole tree, the rule gets also applied in other contexts. The sentence might mislead as much as it helps. Keep it or delete it?

Quoted 1 line

Maybe, just be more honest: "It should be ..., but it may be any other pattern, be careful"

"Need to be careful" means "needs careful manual review", which I believe is not feasible; see "Preface to my review of this script" above.

But do we really need to be careful here?

This rule should apply only where we added ERRP_AUTO_PROPAGATE().

Quoted 7 lines

Except when rule chaining via function name fails us, but we plan to detect that and review manually, so let's ignore this issue here.

Thanks to ERRP_AUTO_PROPAGATE(), @errp is not null. Enabling replacement of @local_err by @errp is its whole point.

What exactly do we need to be careful about?

Hmm.. About some unpredicted patterns. OK, then "For example, different kinds of .."

Quoted 35 lines
Quoted 34 lines
Quoted 34 lines
Quoted 8 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +5 -0
+// conflicts with other substitutions in it (at least with "- local_err = NULL").
+@@
+identifier rule1.fn, rule1.local_err;
+symbol errp;
+@@

scripts/coccinelle/auto-propagated-errp.cocci line 219

Quoted 10 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +10 -0
+
+ fn(...)
+ {
+     <...
+-    local_err
++    *errp
+     ...>
+ }
+
+// Always use the same patter for checking error

s/patter/pattern/

Quoted 12 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +12 -0
+@@
+identifier rule1.fn;
+symbol errp;
+@@
+
+ fn(...)
+ {
+     <...
+-    *errp != NULL
++    *errp
+     ...>
+ }
Signature · 7 lines
Best regards,
Vladimir

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
#15 lore raw

Re: [PATCH v8 02/10] scripts: add coccinelle script to use auto propagated errp

Quoted 308 lines · Vladimir Sementsov-Ogievskiy
Quoted 264 lines · Markus Armbruster
Quoted 110 lines · Vladimir Sementsov-Ogievskiy
Quoted 100 lines · Markus Armbruster

Suggest

      scripts: Coccinelle script to use auto-propagated errp

or

      scripts: Coccinelle script to use ERRP_AUTO_PROPAGATE()

Quoted 8 lines · Vladimir Sementsov-Ogievskiy

Script adds ERRP_AUTO_PROPAGATE macro invocation where appropriate and does corresponding changes in code (look for details in include/qapi/error.h)

Usage example:
spatch --sp-file scripts/coccinelle/auto-propagated-errp.cocci \
   --macro-file scripts/cocci-macro-file.h --in-place --no-show-diff \
   blockdev-nbd.c qemu-nbd.c {block/nbd*,nbd/*,include/block/nbd*}.[hc]

Suggest FILES... instead of a specific set of files.

Quoted 32 lines

---

Cc Eric Blake <[email protected]>
Cc Kevin Wolf <[email protected]>
Cc Max Reitz <[email protected]>
Cc Greg Kurz <[email protected]>
Cc Christian Schoenebeck <[email protected]>
Cc Stefano Stabellini <[email protected]>
Cc Anthony Perard <[email protected]>
Cc Paul Durrant <[email protected]>
Cc Stefan Hajnoczi <[email protected]>
Cc "Philippe Mathieu-Daudé" <[email protected]>
Cc Laszlo Ersek <[email protected]>
Cc Gerd Hoffmann <[email protected]>
Cc Stefan Berger <[email protected]>
Cc Markus Armbruster <[email protected]>
Cc Michael Roth <[email protected]>

   include/qapi/error.h                          |   3 + scripts/coccinelle/auto-propagated-errp.cocci | 231 ++++++++++++++++++

   2 files changed, 234 insertions(+)

   create mode 100644 scripts/coccinelle/auto-propagated-errp.cocci

include/qapi/error.h +0 -0
index bb9bcf02fb..fbfc6f1c0b 100644

include/qapi/error.h line 215

Quoted 5 lines
include/qapi/error.h +2 -0
@@ -211,6 +211,9 @@
     *         }
     *         ...
     *     }
+ *
+ * For mass conversion use script

mass-conversion (we're not converting mass, we're converting en masse)

scripts/coccinelle/auto-propagated-errp.cocci

Quoted 8 lines
include/qapi/error.h +1 -0
+ *   scripts/coccinelle/auto-propagated-errp.cocci
     */
      #ifndef ERROR_H
scripts/coccinelle/auto-propagated-errp.cocci new file +0 -0
new file mode 100644
index 0000000000..bff274bd6d

Preface to my review of this script: may aim isn't to make it bullet-proof.  I want to (1) make it good enough (explained in a jiffie), and (2) automatically identify the spots where it still isn't obviously safe for manual review.

The latter may involve additional scripting.  That's okay.

The script is good enough when the number of possibly unsafe spots is low enough for careful manual review.

When I ask for improvements that, in your opinion, go beyond "good enough", please push back.  I'm sure we can work it out together.

Quoted 17 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +16 -0
@@ -0,0 +1,231 @@
+// Use ERRP_AUTO_PROPAGATE (see include/qapi/error.h)
+//
+// Copyright (c) 2020 Virtuozzo International GmbH.
+//
+// This program is free software; you can redistribute it and/or modify
+// it under the terms of the GNU General Public License as published by
+// the Free Software Foundation; either version 2 of the License, or
+// (at your option) any later version.
+//
+// This program is distributed in the hope that it will be useful,
+// but WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+// GNU General Public License for more details.
+//
+// You should have received a copy of the GNU General Public License
+// along with this program.  If not, see <http://www.gnu.org/licenses/>.
Quoted 8 lines

scripts/coccinelle/auto-propagated-errp.cocci line 21

Quoted 5 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +5 -0
+//
+// Usage example:
+// spatch --sp-file scripts/coccinelle/auto-propagated-errp.cocci \
+//  --macro-file scripts/cocci-macro-file.h --in-place --no-show-diff \
+//  --max-width 80 blockdev-nbd.c qemu-nbd.c \

You have --max-width 80 here, but not in the commit message.  Default seems to be 78.  Any particular reason to change it to 80?

Quoted 3 lines

Hmm. As I remember, without this parameter, reindenting doesn't work correctly. So, I'm OK with "--max-width 78", but I doubt that it will work without a parameter. Still, may be I'm wrong, we can check it.

If you can point to an example where --max-width helps, keep it, and update the commit message to match.  Else, drop it.

Quoted 82 lines
Quoted 24 lines

scripts/coccinelle/auto-propagated-errp.cocci line 24

Quoted 4 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +3 -0
+//  {block/nbd*,nbd/*,include/block/nbd*}.[hc]
+
+// Switch unusual (Error **) parameter names to errp

Let's drop the parenthesis around Error **

scripts/coccinelle/auto-propagated-errp.cocci line 25

Quoted 1 line
scripts/coccinelle/auto-propagated-errp.cocci new file +1 -0
+// (this is necessary to use ERRP_AUTO_PROPAGATE).

Perhaps ERRP_AUTO_PROPAGATE() should be ERRP_AUTO_PROPAGATE(errp) to make the fact we're messing with @errp more obvious.  Too late; I shouldn't rock the boat that much now.

scripts/coccinelle/auto-propagated-errp.cocci line 31

Quoted 6 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +6 -0
+//
+// Disable optional_qualifier to skip functions with "Error *const *errp"
+// parameter.
+//
+// Skip functions with "assert(_errp && *_errp)" statement, as they have
+// non generic semantics and may have unusual Error ** argument name for purpose

non-generic

for a purpose

Quoted 7 lines

Wrap comment lines around column 70, please.  It's easier to read.

Maybe

     // Skip functions with "assert(_errp && *_errp)" statement, because that // signals unusual semantics, and the parameter name may well serve a // purpose.

Sounds good.

Quoted 42 lines

scripts/coccinelle/auto-propagated-errp.cocci line 35

Quoted 5 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +4 -0
+// (like nbd_iter_channel_error()).
+//
+// Skip util/error.c to not touch, for example, error_propagate and
+// error_propagate_prepend().

error_propagate()

I much appreciate your meticulous explanation of what you skip and why.

Quoted 9 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +9 -0
+@ depends on !(file in "util/error.c") disable optional_qualifier@
+identifier fn;
+identifier _errp != errp;
+@@
+
+ fn(...,
+-   Error **_errp
++   Error **errp
+    ,...)

scripts/coccinelle/auto-propagated-errp.cocci line 54

Quoted 10 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +10 -0
+ {
+(
+     ... when != assert(_errp && *_errp)
+&
+     <...
+-    _errp
++    errp
+     ...>
+)
+ }

This rule is required to make the actual transformations (below) work even for parameters with names other than @errp.  I believe it's not used in this series.  In fact, I can't see a use for it in the entire tree right now.  Okay anyway.

scripts/coccinelle/auto-propagated-errp.cocci line 58

Quoted 4 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +4 -0
+
+// Add invocation of ERRP_AUTO_PROPAGATE to errp-functions where necessary
+//
+// Note, that without "when any" final "..." may not want to mach something

s/final "..." may not mach/the final "..." does not match/

Quoted 5 lines

scripts/coccinelle/auto-propagated-errp.cocci line 60

Quoted 2 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +2 -0
+// matched by previous pattern, i.e. the rule will not match double
+// error_prepend in control flow like in vfio_set_irq_signaling().

Can't say I fully understand Coccinelle there.  I figure you came to this knowledge the hard way.

Quoted 5 lines

It's follows from smpl grammar document:

"Implicitly, “...” matches the shortest path between something that matches the pattern before the dots (or the beginning of the function, if there is nothing before the dots) and something that matches the pattern after the dots (or the end of the function, if there is nothing after the dots)."
...
"_when any_ removes the aforementioned constraint that “...” matches the shortest path"

Let me think that through.

The pattern with the cases other than error_prepend() omitted:

      fn(..., Error **errp, ...)
      {
     +   ERRP_AUTO_PROPAGATE();
         ...  when != ERRP_AUTO_PROPAGATE();
         error_prepend(errp, ...);
         ... when any
      }

Tail of vfio_set_irq_signaling():

         name = index_to_str(vbasedev, index);
         if (name) {
             error_prepend(errp, "%s-%d: ", name, subindex);
         } else {
             error_prepend(errp, "index %d-%d: ", index, subindex); }
         error_prepend(errp,
                       "Failed to %s %s eventfd signaling for interrupt ", fd < 0 ? "tear down" : "set up", action_to_str(action)); return ret;
     }

The pattern's first ... matches a "shortest" path to an error_prepend(), where "shortest" means "does not cross an error_prepend().  Its when clause makes us ignore functions that already use ERRP_AUTO_PROPAGATE().

There are two such "shortest" paths, one to the first error_prepend() in vfio_set_irq_signaling(), and one to the second.  Neither path to the third one is not "shortest": they both cross one of the other two error_prepend().

The pattern' s second ... matches a path from a matched error_prepend() to the end of the function.  There are two paths.  Both cross the third error_prepend().  You need "when any" to make the pattern match anyway.

Alright, I think I got it.  But now I'm paranoid about ... elsewhere. For instance, here's rule1 with error_propagate_prepend() omitted:

     // Match scenarios with propagation of local error to errp. @rule1 disable optional_qualifier exists@ identifier fn, local_err;
     symbol errp;
     @@

      fn(..., Error **errp, ...)
      {
          ...
          Error *local_err = NULL;
          ...
          error_propagate(errp, local_err);
          ...
      }

Quoted 3 lines

The second and third ... won't match anything containing error_propagate().  What if a function has multiple error_propagate() on all paths?

I thought about this, but decided that double error propagation is a strange pattern, and may be better not match it...

Quoted 17 lines

Like this one:

     extern foo(int, Error **);
     extern bar(int, Error **);

     void frob(Error **errp)
     {
         Error *local_err = NULL;
         int arg;

         foo(arg, errp);
         bar(arg, &local_err);
         error_propagate(errp, local_err);
         bar(arg + 1, &local_err);
         error_propagate(errp, local_err);
     }

Quoted 2 lines

This is actually a variation of error.h's "Receive and accumulate multiple errors (first one wins)" code snippet.

ah yes, we can propagate to already filled errp, which just clean local_err.

Quoted 15 lines

The Coccinelle script transforms it like this:

      void frob(Error **errp)
      {
     +    ERRP_AUTO_PROPAGATE();
          Error *local_err = NULL;
          int arg;

The rule that adds ERRP_AUTO_PROPAGATE() matches (it has ... when any), but rule1 does not, and we therefore don't convert any of the error_propagate().

The result isn't wrong, just useless.

Quoted 8 lines
Quoted 5 lines

Is this the worst case?

Possible improvement to the ERRP_AUTO_PROPAGATE() rule: don't use "... when any" in the error_propagate() case, only in the other cases. Would that help?

I think not, as it will anyway match functions with error_prepend (and any number of following error_propagate calls)...

But it's correct to add ERRP_AUTO_PROPAGATE to such functions.. So, may be you are right. Let's do it, seems better.

Quoted 454 lines
Quoted 407 lines

I think this is the only other rule with "..." matching control flow.

Quoted 86 lines
Quoted 25 lines

scripts/coccinelle/auto-propagated-errp.cocci line 64

Quoted 5 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +4 -0
+//
+// Note, "exists" says that we want apply rule even if it matches not on
+// all possible control flows (otherwise, it will not match standard pattern
+// when error_propagate() call is in if branch).

Learned something new.  Example: kvm_set_kvm_shadow_mem().

Spelling it "exists disable optional_qualifier" would avoid giving readers the idea we're disabling "exists", but Coccinelle doesn't let us.  Oh well.

scripts/coccinelle/auto-propagated-errp.cocci line 66

Quoted 2 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +2 -0
+@ disable optional_qualifier exists@
+identifier fn, local_err, errp;

I believe this causes

      warning: line 98: errp, previously declared as a metavariable, is used as an identifier
      warning: line 104: errp, previously declared as a metavariable, is used as an identifier
      warning: line 106: errp, previously declared as a metavariable, is used as an identifier
      warning: line 131: errp, previously declared as a metavariable, is used as an identifier
      warning: line 192: errp, previously declared as a metavariable, is used as an identifier
      warning: line 195: errp, previously declared as a metavariable, is used as an identifier
      warning: line 228: errp, previously declared as a metavariable, is used as an identifier

Quoted 1 line

Making @errp symbol instead of identifier should fix this.

Hmm, I didn't see these warnings.. But yes, it should be symbol.

Quoted 57 lines
Quoted 32 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +31 -0
+@@
+
+ fn(..., Error **errp, ...)
+ {
++   ERRP_AUTO_PROPAGATE();
+    ...  when != ERRP_AUTO_PROPAGATE();
+(
+    error_append_hint(errp, ...);
+|
+    error_prepend(errp, ...);
+|
+    error_vprepend(errp, ...);
+|
+    Error *local_err = NULL;
+    ...
+(
+    error_propagate_prepend(errp, local_err, ...);
+|
+    error_propagate(errp, local_err);
+)
+)
+    ... when any
+ }
+
+
+// Match scenarios with propagation of local error to errp.
+@rule1 disable optional_qualifier exists@
+identifier fn, local_err;
+symbol errp;
+@@
+

scripts/coccinelle/auto-propagated-errp.cocci line 107

Quoted 10 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +10 -0
+ fn(..., Error **errp, ...)
+ {
+     ...
+     Error *local_err = NULL;
+     ...
+(
+    error_propagate_prepend(errp, local_err, ...);
+|
+    error_propagate(errp, local_err);
+)

Indentation off by one.

scripts/coccinelle/auto-propagated-errp.cocci line 111

Quoted 4 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +4 -0
+     ...
+ }
+
+// Convert special case with goto in separate.

s/in separate/separately/

scripts/coccinelle/auto-propagated-errp.cocci line 113

Quoted 2 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +2 -0
+// We can probably merge this into the following hunk with help of ( | )
+// operator, but it significantly reduce performance on block.c parsing (or it

s/reduce/reduces/

Quoted 6 lines
Quoted 4 lines

scripts/coccinelle/auto-propagated-errp.cocci line 114

Quoted 1 line
scripts/coccinelle/auto-propagated-errp.cocci new file +1 -0
+// hangs, I don't know)

Sounds like you tried to merge this into the following hunk, but then spatch took so long on block.c that you killed it.  Correct?

Yes.

I'd say something like "I tried merging this into the following rule the obvious way, but it made Coccinelle hang on block.c."

Quoted 32 lines
Quoted 7 lines

scripts/coccinelle/auto-propagated-errp.cocci line 118

Quoted 5 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +4 -0
+//
+// Note interesting thing: if we don't do it here, and try to fixup "out: }"
+// things later after all transformations (the rule will be the same, just
+// without error_propagate() call), coccinelle fails to match this "out: }".

Weird, but not worth further investigation.

It partially match to the idea which I saw somewhere in coccinelle documentation, that coccinelle converts correct C code to correct C code. "out: }" is an example of incorrect, impossible code flow, and coccinelle can't work with it... But it's just a thought.

Quoted 10 lines
Quoted 10 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +9 -0
+@@
+identifier rule1.fn, rule1.local_err, out;
+symbol errp;
+@@
+
+ fn(...)
+ {
+     <...
+-    goto out;
Quoted 8 lines

scripts/coccinelle/auto-propagated-errp.cocci line 131

Quoted 4 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +4 -0
++    return;
+     ...>
+- out:
+-    error_propagate(errp, local_err);

You neglect to match error_propagate_prepend().  Okay, because (1) that pattern doesn't occur in the tree right now, and (2) if it gets added, gcc will complain.

Quoted 2 lines

No, because it should not removed. error_propagate_prepend should be converted to prepend, not removed. So, corresponding gotos should not be removed as well.

You're right.

Quoted 30 lines
Quoted 30 lines

scripts/coccinelle/auto-propagated-errp.cocci line 134

Quoted 4 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +3 -0
+ }
+
+// Convert most of local_err related staff.

s/staff/stuff/

scripts/coccinelle/auto-propagated-errp.cocci line 141

Quoted 7 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +7 -0
+//
+// Note, that we update everything related to matched by rule1 function name
+// and local_err name. We may match something not related to the pattern
+// matched by rule1. For example, local_err may be defined with the same name
+// in different blocks inside one function, and in one block follow the
+// propagation pattern and in other block doesn't. Or we may have several
+// functions with the same name (for different configurations).

Context: rule1 matches functions that have all three of

* an Error **errp parameter

* an Error *local_err = NULL variable declaration

* an error_propagate(errp, local_err) or error_propagate_prepend(errp, local_err, ...) expression, where @errp is the parameter and @local_err is the variable.

If I understand you correctly, you're pointing out two potential issues:

1. This rule can match functions rule1 does not match if there is another function with the same name that rule1 does match.

Quoted 7 lines
Quoted 5 lines

2. This rule matches in the entire function matched by rule1, even when parts of that function use a different @errp or @local_err.

I figure these apply to all rules with identifier rule1.fn, not just this one.  Correct?

Yes.

Thanks!

Quoted 30 lines
Quoted 13 lines

Regarding 1.  There must be a better way to chain rules together, but I don't know it.
  Can we make Coccinelle at least warn us when it converts multiple functions with the same name?  What about this:

     @initialize:python@
     @@
     fnprev = {}

     def pr(fn, p):
         print("### %s:%s: %s()" % (p[0].file, p[0].line, fn))

Quoted 16 lines

     @r@
     identifier rule1.fn;
     position p;
     @@
      fn(...)@p
      {
          ...
      }
     @script:python@
         fn << rule1.fn;
         p << r.p;
     @@
     if fn not in fnprev:
         fnprev[fn] = p
     else:
         if fnprev[fn]:

Quoted 7 lines

hmm, the condition can't be false

Quoted 3 lines

             pr(fn, fnprev[fn])
             fnprev[fn] = None
         pr(fn, p)

and we'll miss next duplication..

The idea is

     first instance of fn:
         fn not in fnprev
         fnprev[fn] = position of instance
         don't print
     second instance:
         fnprev[fn] is the position of the first instance print first two instances
     subsequent instances: fnprev[fn] is None print this instance

I might have screwed up the coding, of course :)

Quoted 168 lines

But I like the idea.

Quoted 125 lines

For each function @fn matched by rule1, fncnt[fn] is an upper limit of the number of functions with the same name we touch.  If it's more than one, we print.

Reports about a dozen function names for the whole tree in my testing. Inspecting the changes to them manually is feasible.  None of them are in files touched by this series.

The line printed for the first match is pretty useless for me: it points to a Coccinelle temporary file *shrug*.

Regarding 2.  Shadowing @errp or @local_err would be in bad taste, and I sure hope we don't do that.  Multiple @local_err variables... hmm. Perhaps we could again concoct some script rules to lead us to spots to check manually.  See below for my attempt.

What's the worst that could happen if we blindly converted such code? The answer to that question tells us how hard to work on finding and checking these guys.

Quoted 17 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +17 -0
+//
+// Note also that errp-cleaning functions
+//   error_free_errp
+//   error_report_errp
+//   error_reportf_errp
+//   warn_report_errp
+//   warn_reportf_errp
+// are not yet implemented. They must call corresponding Error* - freeing
+// function and then set *errp to NULL, to avoid further propagation to
+// original errp (consider ERRP_AUTO_PROPAGATE in use).
+// For example, error_free_errp may look like this:
+//
+//    void error_free_errp(Error **errp)
+//    {
+//        error_free(*errp);
+//        *errp = NULL;
+//    }

scripts/coccinelle/auto-propagated-errp.cocci line 168

Quoted 10 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +10 -0
+@ exists@
+identifier rule1.fn, rule1.local_err;
+expression list args;
+symbol errp;
+@@
+
+ fn(...)
+ {
+     <...
+(

Each of the following patterns applies anywhere in the function.

First pattern: delete @local_err

scripts/coccinelle/auto-propagated-errp.cocci line 169

Quoted 1 line
scripts/coccinelle/auto-propagated-errp.cocci new file +1 -0
+-    Error *local_err = NULL;

Common case: occurs just once, not nested.  Anything else is suspicious.

Both can be detected in the resulting patches with a bit of AWK wizardry:

      $ git-diff -U0 master..review-error-v8 | awk '/^@@ / { ctx = $5; for (i = 6; i <= NF; i++) ctx = ctx " " $i; if (ctx != octx) { octx = ctx; n = 0 } } /^- *Error *\* *[A-Za-z0-9_]+ *= *NULL;/ { if (index($0, "E") > 6) print "nested\n    " ctx; if (n) print "more than one\n    " ctx; n++ }'
      nested
          static void xen_block_drive_destroy(XenBlockDrive *drive, Error **errp)
      nested
          static void xen_block_device_destroy(XenBackendInstance *backend,
      nested
          static void xen_block_device_destroy(XenBackendInstance *backend,
      more than one
          static void xen_block_device_destroy(XenBackendInstance *backend,

Oh.

xen_block_drive_destroy() nests its Error *local_err in a conditional.

xen_block_device_destroy() has multiple Error *local_err.

In both cases, manual review is required to ensure the conversion is okay.  I believe it is.

Note that the AWK script relies on diff showing the function name in @@ lines, which doesn't always work due to our coding style.

For the whole tree, I get some 30 spots.  Feasible.

scripts/coccinelle/auto-propagated-errp.cocci line 170

Quoted 1 line
scripts/coccinelle/auto-propagated-errp.cocci new file +1 -0
+|

Second pattern: clear @errp after freeing it

scripts/coccinelle/auto-propagated-errp.cocci line 172

Quoted 2 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +2 -0
+
+// Convert error clearing functions

Suggest: Ensure @local_err is cleared on free

Quoted 6 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +6 -0
+(
+-    error_free(local_err);
++    error_free_errp(errp);
+|
+-    error_report_err(local_err);
++    error_report_errp(errp);

scripts/coccinelle/auto-propagated-errp.cocci line 188

Quoted 10 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +10 -0
+|
+-    error_reportf_err(local_err, args);
++    error_reportf_errp(errp, args);
+|
+-    warn_report_err(local_err);
++    warn_report_errp(errp);
+|
+-    warn_reportf_err(local_err, args);
++    warn_reportf_errp(errp, args);
+)

As you mention above, these guys don't exist, yet.  Builds anyway, because this part of the rule is not used in this patch series.  You don't want to omit it, because then the script becomes unsafe to use.

We could also open-code:

     // Convert error clearing functions
     (
     -    error_free(local_err);
     +    error_free(*errp);
     +    *errp = NULL;
     |
     ... and so forth ...
     )

Quoted 2 lines

Matter of taste.  Whatever is easier to explain in the comments.  Since you already wrote one...

I just feel that using helper functions is safer way..

Quoted 22 lines

We talked about extending this series slightly so these guys are used. I may still look into that.

scripts/coccinelle/auto-propagated-errp.cocci line 190

Quoted 2 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +2 -0
+?-    local_err = NULL;
+

The new helpers clear @local_err.  Assignment now redundant, delete. Okay.

scripts/coccinelle/auto-propagated-errp.cocci line 191

Quoted 1 line
scripts/coccinelle/auto-propagated-errp.cocci new file +1 -0
+|

Third and fourth pattern: delete error_propagate()

scripts/coccinelle/auto-propagated-errp.cocci line 196

Quoted 5 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +5 -0
+-    error_propagate_prepend(errp, local_err, args);
++    error_prepend(errp, args);
+|
+-    error_propagate(errp, local_err);
+|

Fifth pattern: use @errp directly

Quoted 13 lines

scripts/coccinelle/auto-propagated-errp.cocci line 204

Quoted 8 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +8 -0
+-    &local_err
++    errp
+)
+     ...>
+ }
+
+// Convert remaining local_err usage. It should be different kinds of error
+// checking in if operators. We can't merge this into previous hunk, as this

In if conditionals, I suppose.  It's the case for this patch.  If I apply the script to the whole tree, the rule gets also applied in other contexts.  The sentence might mislead as much as it helps.  Keep it or delete it?

Quoted 1 line

Maybe, just be more honest: "It should be ..., but it may be any other pattern, be careful"

"Need to be careful" means "needs careful manual review", which I believe is not feasible; see "Preface to my review of this script" above.

But do we really need to be careful here?

This rule should apply only where we added ERRP_AUTO_PROPAGATE().

Quoted 7 lines

Except when rule chaining via function name fails us, but we plan to detect that and review manually, so let's ignore this issue here.

Thanks to ERRP_AUTO_PROPAGATE(), @errp is not null.  Enabling replacement of @local_err by @errp is its whole point.

What exactly do we need to be careful about?

Hmm.. About some unpredicted patterns. OK, then "For example, different kinds of .."

Quoted 35 lines
Quoted 34 lines
Quoted 34 lines
Quoted 8 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +5 -0
+// conflicts with other substitutions in it (at least with "- local_err = NULL").
+@@
+identifier rule1.fn, rule1.local_err;
+symbol errp;
+@@

scripts/coccinelle/auto-propagated-errp.cocci line 219

Quoted 10 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +10 -0
+
+ fn(...)
+ {
+     <...
+-    local_err
++    *errp
+     ...>
+ }
+
+// Always use the same patter for checking error

s/patter/pattern/

Quoted 12 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +12 -0
+@@
+identifier rule1.fn;
+symbol errp;
+@@
+
+ fn(...)
+ {
+     <...
+-    *errp != NULL
++    *errp
+     ...>
+ }
Signature · 7 lines
Best regards,
Vladimir

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
#16 lore raw

Re: [PATCH v8 02/10] scripts: add coccinelle script to use auto propagated errp

In reply to Vladimir Sementsov-Ogievskiy
Quoted 117 lines · Vladimir Sementsov-Ogievskiy
Quoted 116 lines · Markus Armbruster
Quoted 113 lines · Vladimir Sementsov-Ogievskiy
Quoted 100 lines · Markus Armbruster

Suggest

scripts: Coccinelle script to use auto-propagated errp

or

scripts: Coccinelle script to use ERRP_AUTO_PROPAGATE()

Quoted 8 lines · Vladimir Sementsov-Ogievskiy

Script adds ERRP_AUTO_PROPAGATE macro invocation where appropriate and does corresponding changes in code (look for details in include/qapi/error.h)

Usage example:

spatch --sp-file scripts/coccinelle/auto-propagated-errp.cocci \
   --macro-file scripts/cocci-macro-file.h --in-place --no-show-diff \
   blockdev-nbd.c qemu-nbd.c {block/nbd*,nbd/*,include/block/nbd*}.[hc]

Suggest FILES... instead of a specific set of files.

Quoted 32 lines

---

Cc Eric Blake <[email protected]>
Cc Kevin Wolf <[email protected]>
Cc Max Reitz <[email protected]>
Cc Greg Kurz <[email protected]>
Cc Christian Schoenebeck <[email protected]>
Cc Stefano Stabellini <[email protected]>
Cc Anthony Perard <[email protected]>
Cc Paul Durrant <[email protected]>
Cc Stefan Hajnoczi <[email protected]>
Cc "Philippe Mathieu-Daudé" <[email protected]>
Cc Laszlo Ersek <[email protected]>
Cc Gerd Hoffmann <[email protected]>
Cc Stefan Berger <[email protected]>
Cc Markus Armbruster <[email protected]>
Cc Michael Roth <[email protected]>
   include/qapi/error.h                          |   3 +
   scripts/coccinelle/auto-propagated-errp.cocci | 231 ++++++++++++++++++
   2 files changed, 234 insertions(+)

create mode 100644 scripts/coccinelle/auto-propagated-errp.cocci

include/qapi/error.h +0 -0
index bb9bcf02fb..fbfc6f1c0b 100644

include/qapi/error.h line 215

Quoted 5 lines
include/qapi/error.h +2 -0
@@ -211,6 +211,9 @@
    *         }
    *         ...
    *     }
+ *
+ * For mass conversion use script

mass-conversion (we're not converting mass, we're converting en masse)

scripts/coccinelle/auto-propagated-errp.cocci

Quoted 8 lines
include/qapi/error.h +1 -0
+ *   scripts/coccinelle/auto-propagated-errp.cocci
    */
     #ifndef ERROR_H
scripts/coccinelle/auto-propagated-errp.cocci new file +0 -0
new file mode 100644
index 0000000000..bff274bd6d

Preface to my review of this script: may aim isn't to make it bullet-proof. I want to (1) make it good enough (explained in a jiffie), and (2) automatically identify the spots where it still isn't obviously safe for manual review.

The latter may involve additional scripting. That's okay.

The script is good enough when the number of possibly unsafe spots is low enough for careful manual review.

When I ask for improvements that, in your opinion, go beyond "good enough", please push back. I'm sure we can work it out together.

Quoted 17 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +16 -0
@@ -0,0 +1,231 @@
+// Use ERRP_AUTO_PROPAGATE (see include/qapi/error.h)
+//
+// Copyright (c) 2020 Virtuozzo International GmbH.
+//
+// This program is free software; you can redistribute it and/or modify
+// it under the terms of the GNU General Public License as published by
+// the Free Software Foundation; either version 2 of the License, or
+// (at your option) any later version.
+//
+// This program is distributed in the hope that it will be useful,
+// but WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+// GNU General Public License for more details.
+//
+// You should have received a copy of the GNU General Public License
+// along with this program.  If not, see <http://www.gnu.org/licenses/>.
Quoted 8 lines

scripts/coccinelle/auto-propagated-errp.cocci line 21

Quoted 5 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +5 -0
+//
+// Usage example:
+// spatch --sp-file scripts/coccinelle/auto-propagated-errp.cocci \
+//  --macro-file scripts/cocci-macro-file.h --in-place --no-show-diff \
+//  --max-width 80 blockdev-nbd.c qemu-nbd.c \

You have --max-width 80 here, but not in the commit message. Default seems to be 78. Any particular reason to change it to 80?

Hmm. As I remember, without this parameter, reindenting doesn't work correctly. So, I'm OK with "--max-width 78", but I doubt that it will work without a parameter. Still, may be I'm wrong, we can check it.

Quoted 5 lines
Quoted 5 lines

If you can point to an example where --max-width helps, keep it, and update the commit message to match. Else, drop it.

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +1 -0
+//  {block/nbd*,nbd/*,include/block/nbd*}.[hc]

As our discussion shows, this script is somewhat hard to understand. That's okay, it solves a somewhat thorny problem, and I don't have better ideas. But let me state the intended transformations once more, so I don't get lost in the details.

Motivation:

1. Make error propagation less error-prone and improve stack backtraces

The "receive error in &local_error, propagate to @errp" pattern is tedious, error-prone, and produces unhelpful stack backtraces with &error_abort.

ERRP_AUTO_PROPAGATE() removes manual propagation. It additionally gives us the stack backtraces we want.

2. Improve error messages with &error_fatal

Passing @errp to error_append_hint(), error_prepend() or error_vprepend() is useless when @errp is &error_fatal.

ERRP_AUTO_PROPAGATE() fixes this by delaying &error_fatal handling until the automatic propagation.

The intended transformation has three parts:

* Replace declaration of @local_err by ERRP_AUTO_PROPAGATE() where needed for 1 or 2.

It's needed when we also drop some error propagation from the function (motivation 1), or the function calls error_append_hint(), error_prepend() or error_vprepend() (motivation 2). A function could do both. I'll refer back to this below.

* Drop error_propagate(errp, local_err)

Special case: error_propagate_prepend(errp, local_err, ...) becomes error_prepend(errp, ...).

Only correct if there is no other such error_propagate() / error_propagate_prepend() on any path from the here to return.

* Replace remaining use of @local_err by *errp

Quoted 143 lines
Quoted 143 lines
Quoted 80 lines
Quoted 22 lines

scripts/coccinelle/auto-propagated-errp.cocci line 24

Quoted 2 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +2 -0
+
+// Switch unusual (Error **) parameter names to errp

Let's drop the parenthesis around Error **

scripts/coccinelle/auto-propagated-errp.cocci line 25

Quoted 1 line
scripts/coccinelle/auto-propagated-errp.cocci new file +1 -0
+// (this is necessary to use ERRP_AUTO_PROPAGATE).

Perhaps ERRP_AUTO_PROPAGATE() should be ERRP_AUTO_PROPAGATE(errp) to make the fact we're messing with @errp more obvious. Too late; I shouldn't rock the boat that much now.

scripts/coccinelle/auto-propagated-errp.cocci line 31

Quoted 6 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +6 -0
+//
+// Disable optional_qualifier to skip functions with "Error *const *errp"
+// parameter.
+//
+// Skip functions with "assert(_errp && *_errp)" statement, as they have
+// non generic semantics and may have unusual Error ** argument name for purpose

non-generic

for a purpose

Quoted 7 lines

Wrap comment lines around column 70, please. It's easier to read.

Maybe

     // Skip functions with "assert(_errp && *_errp)" statement, because that
     // signals unusual semantics, and the parameter name may well serve a
     // purpose.

Sounds good.

Quoted 42 lines

scripts/coccinelle/auto-propagated-errp.cocci line 35

Quoted 5 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +4 -0
+// (like nbd_iter_channel_error()).
+//
+// Skip util/error.c to not touch, for example, error_propagate and
+// error_propagate_prepend().

error_propagate()

I much appreciate your meticulous explanation of what you skip and why.

Quoted 9 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +9 -0
+@ depends on !(file in "util/error.c") disable optional_qualifier@
+identifier fn;
+identifier _errp != errp;
+@@
+
+ fn(...,
+-   Error **_errp
++   Error **errp
+    ,...)

scripts/coccinelle/auto-propagated-errp.cocci line 54

Quoted 10 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +10 -0
+ {
+(
+     ... when != assert(_errp && *_errp)
+&
+     <...
+-    _errp
++    errp
+     ...>
+)
+ }

This rule is required to make the actual transformations (below) work even for parameters with names other than @errp. I believe it's not used in this series. In fact, I can't see a use for it in the entire tree right now. Okay anyway.

scripts/coccinelle/auto-propagated-errp.cocci line 58

Quoted 4 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +4 -0
+
+// Add invocation of ERRP_AUTO_PROPAGATE to errp-functions where necessary
+//
+// Note, that without "when any" final "..." may not want to mach something

s/final "..." may not mach/the final "..." does not match/

Quoted 5 lines

scripts/coccinelle/auto-propagated-errp.cocci line 60

Quoted 2 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +2 -0
+// matched by previous pattern, i.e. the rule will not match double
+// error_prepend in control flow like in vfio_set_irq_signaling().

Can't say I fully understand Coccinelle there. I figure you came to this knowledge the hard way.

Quoted 5 lines

It's follows from smpl grammar document:

"Implicitly, “...” matches the shortest path between something that matches the pattern before the dots (or the beginning of the function, if there is nothing before the dots) and something that matches the pattern after the dots (or the end of the function, if there is nothing after the dots)."
...
"_when any_ removes the aforementioned constraint that “...” matches the shortest path"

Let me think that through.

The pattern with the cases other than error_prepend() omitted:

      fn(..., Error **errp, ...)
      {
     +   ERRP_AUTO_PROPAGATE();
         ...  when != ERRP_AUTO_PROPAGATE();
         error_prepend(errp, ...);
         ... when any
      }

Tail of vfio_set_irq_signaling():

         name = index_to_str(vbasedev, index);
         if (name) {
             error_prepend(errp, "%s-%d: ", name, subindex);
         } else {
             error_prepend(errp, "index %d-%d: ", index, subindex);
         }
         error_prepend(errp,
                       "Failed to %s %s eventfd signaling for interrupt ",
                       fd < 0 ? "tear down" : "set up", action_to_str(action));
         return ret;
     }

The pattern's first ... matches a "shortest" path to an error_prepend(), where "shortest" means "does not cross an error_prepend(). Its when clause makes us ignore functions that already use ERRP_AUTO_PROPAGATE().

There are two such "shortest" paths, one to the first error_prepend() in vfio_set_irq_signaling(), and one to the second. Neither path to the third one is not "shortest": they both cross one of the other two error_prepend().

The pattern' s second ... matches a path from a matched error_prepend() to the end of the function. There are two paths. Both cross the third error_prepend(). You need "when any" to make the pattern match anyway.

Alright, I think I got it. But now I'm paranoid about ... elsewhere. For instance, here's rule1 with error_propagate_prepend() omitted:

     // Match scenarios with propagation of local error to errp.
     @rule1 disable optional_qualifier exists@
     identifier fn, local_err;
     symbol errp;
     @@

      fn(..., Error **errp, ...)
      {
          ...
          Error *local_err = NULL;
          ...
          error_propagate(errp, local_err);
          ...
      }
Quoted 5 lines
Quoted 3 lines

The second and third ... won't match anything containing error_propagate(). What if a function has multiple error_propagate() on all paths?

I thought about this, but decided that double error propagation is a strange pattern, and may be better not match it...

I'm fine with not touching "strange" patterns. But we do at least in the example I gave below: we still add ERRP_AUTO_PROPAGATE().

We either avoid that in the Coccinelle script, or we catch it afterwards. Catching it afterwards should be feasible:

* If we also drop some error propagation from this function: okay.

* If this function call error_append_hint(), error_prepend() or error_vprepend(): okay.

* Else: unwanted, back out.

Moreover, I'd like us to double-check we really don't want to touch the things we don't touch. Feels feasible to me, too: after running Coccinelle, search for unconverted error_append_hint(), error_prepend(), error_vprepend(), error_propagate_prepend(), error_propagate().

Quoted 37 lines
Quoted 17 lines

Like this one:

     extern foo(int, Error **);
     extern bar(int, Error **);

     void frob(Error **errp)
     {
         Error *local_err = NULL;
         int arg;

         foo(arg, errp);
         bar(arg, &local_err);
         error_propagate(errp, local_err);
         bar(arg + 1, &local_err);
         error_propagate(errp, local_err);
     }
Quoted 2 lines

This is actually a variation of error.h's "Receive and accumulate multiple errors (first one wins)" code snippet.

ah yes, we can propagate to already filled errp, which just clean local_err.

Quoted 15 lines

The Coccinelle script transforms it like this:

      void frob(Error **errp)
      {
     +    ERRP_AUTO_PROPAGATE();
          Error *local_err = NULL;
          int arg;

The rule that adds ERRP_AUTO_PROPAGATE() matches (it has ... when any), but rule1 does not, and we therefore don't convert any of the error_propagate().

The result isn't wrong, just useless.

Quoted 8 lines
Quoted 5 lines

Is this the worst case?

Possible improvement to the ERRP_AUTO_PROPAGATE() rule: don't use "... when any" in the error_propagate() case, only in the other cases. Would that help?

I think not, as it will anyway match functions with error_prepend (and any number of following error_propagate calls)...

I'm not sure I understand this sentence.

The aim of my "possible improvement" is to avoid unwanted ERRP_AUTO_PROPAGATE() in the Coccinelle script. We can instead search for unwanted ones after the Coccinelle run, and back them out.

Quoted 408 lines
Quoted 408 lines

I think this is the only other rule with "..." matching control flow.

Quoted 86 lines
Quoted 25 lines

scripts/coccinelle/auto-propagated-errp.cocci line 64

Quoted 5 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +4 -0
+//
+// Note, "exists" says that we want apply rule even if it matches not on
+// all possible control flows (otherwise, it will not match standard pattern
+// when error_propagate() call is in if branch).

Learned something new. Example: kvm_set_kvm_shadow_mem().

Spelling it "exists disable optional_qualifier" would avoid giving readers the idea we're disabling "exists", but Coccinelle doesn't let us. Oh well.

scripts/coccinelle/auto-propagated-errp.cocci line 66

Quoted 2 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +2 -0
+@ disable optional_qualifier exists@
+identifier fn, local_err, errp;

I believe this causes

warning: line 98: errp, previously declared as a metavariable, is used as an identifier
warning: line 104: errp, previously declared as a metavariable, is used as an identifier
warning: line 106: errp, previously declared as a metavariable, is used as an identifier
warning: line 131: errp, previously declared as a metavariable, is used as an identifier
warning: line 192: errp, previously declared as a metavariable, is used as an identifier
warning: line 195: errp, previously declared as a metavariable, is used as an identifier
warning: line 228: errp, previously declared as a metavariable, is used as an identifier

Quoted 1 line

Making @errp symbol instead of identifier should fix this.

Hmm, I didn't see these warnings.. But yes, it should be symbol.

Quoted 57 lines
Quoted 32 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +31 -0
+@@
+
+ fn(..., Error **errp, ...)
+ {
++   ERRP_AUTO_PROPAGATE();
+    ...  when != ERRP_AUTO_PROPAGATE();
+(
+    error_append_hint(errp, ...);
+|
+    error_prepend(errp, ...);
+|
+    error_vprepend(errp, ...);
+|
+    Error *local_err = NULL;
+    ...
+(
+    error_propagate_prepend(errp, local_err, ...);
+|
+    error_propagate(errp, local_err);
+)
+)
+    ... when any
+ }
+
+
+// Match scenarios with propagation of local error to errp.
+@rule1 disable optional_qualifier exists@
+identifier fn, local_err;
+symbol errp;
+@@
+

scripts/coccinelle/auto-propagated-errp.cocci line 107

Quoted 10 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +10 -0
+ fn(..., Error **errp, ...)
+ {
+     ...
+     Error *local_err = NULL;
+     ...
+(
+    error_propagate_prepend(errp, local_err, ...);
+|
+    error_propagate(errp, local_err);
+)

Indentation off by one.

scripts/coccinelle/auto-propagated-errp.cocci line 111

Quoted 4 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +4 -0
+     ...
+ }
+
+// Convert special case with goto in separate.

s/in separate/separately/

scripts/coccinelle/auto-propagated-errp.cocci line 113

Quoted 2 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +2 -0
+// We can probably merge this into the following hunk with help of ( | )
+// operator, but it significantly reduce performance on block.c parsing (or it

s/reduce/reduces/

Quoted 6 lines
Quoted 4 lines

scripts/coccinelle/auto-propagated-errp.cocci line 114

Quoted 1 line
scripts/coccinelle/auto-propagated-errp.cocci new file +1 -0
+// hangs, I don't know)

Sounds like you tried to merge this into the following hunk, but then spatch took so long on block.c that you killed it. Correct?

Yes.

I'd say something like "I tried merging this into the following rule the obvious way, but it made Coccinelle hang on block.c."

Quoted 32 lines
Quoted 7 lines

scripts/coccinelle/auto-propagated-errp.cocci line 118

Quoted 5 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +4 -0
+//
+// Note interesting thing: if we don't do it here, and try to fixup "out: }"
+// things later after all transformations (the rule will be the same, just
+// without error_propagate() call), coccinelle fails to match this "out: }".

Weird, but not worth further investigation.

It partially match to the idea which I saw somewhere in coccinelle documentation, that coccinelle converts correct C code to correct C code. "out: }" is an example of incorrect, impossible code flow, and coccinelle can't work with it... But it's just a thought.

Quoted 10 lines
Quoted 10 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +9 -0
+@@
+identifier rule1.fn, rule1.local_err, out;
+symbol errp;
+@@
+
+ fn(...)
+ {
+     <...
+-    goto out;
Quoted 8 lines

scripts/coccinelle/auto-propagated-errp.cocci line 131

Quoted 4 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +4 -0
++    return;
+     ...>
+- out:
+-    error_propagate(errp, local_err);

You neglect to match error_propagate_prepend(). Okay, because (1) that pattern doesn't occur in the tree right now, and (2) if it gets added, gcc will complain.

Quoted 2 lines

No, because it should not removed. error_propagate_prepend should be converted to prepend, not removed. So, corresponding gotos should not be removed as well.

You're right.

Quoted 30 lines
Quoted 30 lines

scripts/coccinelle/auto-propagated-errp.cocci line 134

Quoted 4 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +3 -0
+ }
+
+// Convert most of local_err related staff.

s/staff/stuff/

scripts/coccinelle/auto-propagated-errp.cocci line 141

Quoted 7 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +7 -0
+//
+// Note, that we update everything related to matched by rule1 function name
+// and local_err name. We may match something not related to the pattern
+// matched by rule1. For example, local_err may be defined with the same name
+// in different blocks inside one function, and in one block follow the
+// propagation pattern and in other block doesn't. Or we may have several
+// functions with the same name (for different configurations).

Context: rule1 matches functions that have all three of

* an Error **errp parameter

* an Error *local_err = NULL variable declaration

* an error_propagate(errp, local_err) or error_propagate_prepend(errp, local_err, ...) expression, where @errp is the parameter and @local_err is the variable.

If I understand you correctly, you're pointing out two potential issues:

1. This rule can match functions rule1 does not match if there is another function with the same name that rule1 does match.

Quoted 7 lines
Quoted 5 lines

2. This rule matches in the entire function matched by rule1, even when parts of that function use a different @errp or @local_err.

I figure these apply to all rules with identifier rule1.fn, not just this one. Correct?

Yes.

Thanks!

Quoted 30 lines
Quoted 21 lines

Regarding 1. There must be a better way to chain rules together, but I don't know it.
Can we make Coccinelle at least warn us when it converts
multiple functions with the same name? What about this:

     @initialize:python@
     @@
     fnprev = {}

     def pr(fn, p):
         print("### %s:%s: %s()" % (p[0].file, p[0].line, fn))

     @r@
     identifier rule1.fn;
     position p;
     @@
      fn(...)@p
      {
          ...
      }
Quoted 8 lines
     @script:python@
         fn << rule1.fn;
         p << r.p;
     @@
     if fn not in fnprev:
         fnprev[fn] = p
     else:
         if fnprev[fn]:
Quoted 7 lines

hmm, the condition can't be false

Quoted 3 lines
             pr(fn, fnprev[fn])
             fnprev[fn] = None
         pr(fn, p)

and we'll miss next duplication..

The idea is

first instance of fn:
fn not in fnprev
fnprev[fn] = position of instance
don't print
second instance:
fnprev[fn] is the position of the first instance print first two instances
subsequent instances: fnprev[fn] is None
print this instance

I might have screwed up the coding, of course :)

Quoted 168 lines

But I like the idea.

Quoted 125 lines

For each function @fn matched by rule1, fncnt[fn] is an upper limit of the number of functions with the same name we touch. If it's more than one, we print.

Reports about a dozen function names for the whole tree in my testing. Inspecting the changes to them manually is feasible. None of them are in files touched by this series.

The line printed for the first match is pretty useless for me: it points to a Coccinelle temporary file *shrug*.

Regarding 2. Shadowing @errp or @local_err would be in bad taste, and I sure hope we don't do that. Multiple @local_err variables... hmm. Perhaps we could again concoct some script rules to lead us to spots to check manually. See below for my attempt.

What's the worst that could happen if we blindly converted such code? The answer to that question tells us how hard to work on finding and checking these guys.

Quoted 17 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +17 -0
+//
+// Note also that errp-cleaning functions
+//   error_free_errp
+//   error_report_errp
+//   error_reportf_errp
+//   warn_report_errp
+//   warn_reportf_errp
+// are not yet implemented. They must call corresponding Error* - freeing
+// function and then set *errp to NULL, to avoid further propagation to
+// original errp (consider ERRP_AUTO_PROPAGATE in use).
+// For example, error_free_errp may look like this:
+//
+//    void error_free_errp(Error **errp)
+//    {
+//        error_free(*errp);
+//        *errp = NULL;
+//    }

scripts/coccinelle/auto-propagated-errp.cocci line 168

Quoted 10 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +10 -0
+@ exists@
+identifier rule1.fn, rule1.local_err;
+expression list args;
+symbol errp;
+@@
+
+ fn(...)
+ {
+     <...
+(

Each of the following patterns applies anywhere in the function.

First pattern: delete @local_err

scripts/coccinelle/auto-propagated-errp.cocci line 169

Quoted 1 line
scripts/coccinelle/auto-propagated-errp.cocci new file +1 -0
+-    Error *local_err = NULL;

Common case: occurs just once, not nested. Anything else is suspicious.

Both can be detected in the resulting patches with a bit of AWK wizardry:

      $ git-diff -U0 master..review-error-v8 | awk '/^@@ / { ctx = $5; for (i = 6; i <= NF; i++) ctx = ctx " " $i; if (ctx != octx) { octx = ctx; n = 0 } } /^- *Error *\* *[A-Za-z0-9_]+ *= *NULL;/ { if (index($0, "E") > 6) print "nested\n    " ctx; if (n) print "more than one\n    " ctx; n++ }'
      nested
          static void xen_block_drive_destroy(XenBlockDrive *drive, Error **errp)
      nested
          static void xen_block_device_destroy(XenBackendInstance *backend,
      nested
          static void xen_block_device_destroy(XenBackendInstance *backend,
      more than one
          static void xen_block_device_destroy(XenBackendInstance *backend,

Oh.

xen_block_drive_destroy() nests its Error *local_err in a conditional.

xen_block_device_destroy() has multiple Error *local_err.

In both cases, manual review is required to ensure the conversion is okay. I believe it is.

Note that the AWK script relies on diff showing the function name in @@ lines, which doesn't always work due to our coding style.

For the whole tree, I get some 30 spots. Feasible.

scripts/coccinelle/auto-propagated-errp.cocci line 170

Quoted 1 line
scripts/coccinelle/auto-propagated-errp.cocci new file +1 -0
+|

Second pattern: clear @errp after freeing it

scripts/coccinelle/auto-propagated-errp.cocci line 172

Quoted 2 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +2 -0
+
+// Convert error clearing functions

Suggest: Ensure @local_err is cleared on free

Quoted 6 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +6 -0
+(
+-    error_free(local_err);
++    error_free_errp(errp);
+|
+-    error_report_err(local_err);
++    error_report_errp(errp);

scripts/coccinelle/auto-propagated-errp.cocci line 188

Quoted 10 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +10 -0
+|
+-    error_reportf_err(local_err, args);
++    error_reportf_errp(errp, args);
+|
+-    warn_report_err(local_err);
++    warn_report_errp(errp);
+|
+-    warn_reportf_err(local_err, args);
++    warn_reportf_errp(errp, args);
+)

As you mention above, these guys don't exist, yet. Builds anyway, because this part of the rule is not used in this patch series. You don't want to omit it, because then the script becomes unsafe to use.

We could also open-code:

     // Convert error clearing functions
     (
     -    error_free(local_err);
     +    error_free(*errp);
     +    *errp = NULL;
     |
     ... and so forth ...
     )
Quoted 2 lines

Matter of taste. Whatever is easier to explain in the comments. Since you already wrote one...

I just feel that using helper functions is safer way..

Quoted 22 lines

We talked about extending this series slightly so these guys are used. I may still look into that.

scripts/coccinelle/auto-propagated-errp.cocci line 190

Quoted 2 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +2 -0
+?-    local_err = NULL;
+

The new helpers clear @local_err. Assignment now redundant, delete. Okay.

scripts/coccinelle/auto-propagated-errp.cocci line 191

Quoted 1 line
scripts/coccinelle/auto-propagated-errp.cocci new file +1 -0
+|

Third and fourth pattern: delete error_propagate()

scripts/coccinelle/auto-propagated-errp.cocci line 196

Quoted 5 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +5 -0
+-    error_propagate_prepend(errp, local_err, args);
++    error_prepend(errp, args);
+|
+-    error_propagate(errp, local_err);
+|

Fifth pattern: use @errp directly

Quoted 13 lines

scripts/coccinelle/auto-propagated-errp.cocci line 204

Quoted 8 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +8 -0
+-    &local_err
++    errp
+)
+     ...>
+ }
+
+// Convert remaining local_err usage. It should be different kinds of error
+// checking in if operators. We can't merge this into previous hunk, as this

In if conditionals, I suppose. It's the case for this patch. If I apply the script to the whole tree, the rule gets also applied in other contexts. The sentence might mislead as much as it helps. Keep it or delete it?

Quoted 1 line

Maybe, just be more honest: "It should be ..., but it may be any other pattern, be careful"

"Need to be careful" means "needs careful manual review", which I believe is not feasible; see "Preface to my review of this script" above.

But do we really need to be careful here?

This rule should apply only where we added ERRP_AUTO_PROPAGATE().

Except when rule chaining via function name fails us, but we plan to detect that and review manually, so let's ignore this issue here.

Quoted 6 lines
Quoted 4 lines

Thanks to ERRP_AUTO_PROPAGATE(), @errp is not null. Enabling replacement of @local_err by @errp is its whole point.

What exactly do we need to be careful about?

Hmm.. About some unpredicted patterns. OK, then "For example, different kinds of .."

Something like this, perhaps?

       // Convert remaining local_err usage, typically error checking.
       // We can't merge this into previous hunk, as this conflicts with other
       // substitutions in it (at least with "- local_err = NULL").
Quoted 18 lines
Quoted 18 lines
Quoted 18 lines
Quoted 18 lines
Quoted 5 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +5 -0
+// conflicts with other substitutions in it (at least with "- local_err = NULL").
+@@
+identifier rule1.fn, rule1.local_err;
+symbol errp;
+@@

scripts/coccinelle/auto-propagated-errp.cocci line 219

Quoted 10 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +10 -0
+
+ fn(...)
+ {
+     <...
+-    local_err
++    *errp
+     ...>
+ }
+
+// Always use the same patter for checking error

s/patter/pattern/

Quoted 14 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +5 -0
+@@
+identifier rule1.fn;
+symbol errp;
+@@
+
Quoted 9 lines
Quoted 8 lines
Quoted 8 lines

scripts/coccinelle/auto-propagated-errp.cocci line 231

Quoted 7 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +7 -0
+ fn(...)
+ {
+     <...
+-    *errp != NULL
++    *errp
+     ...>
+ }

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

#17 lore raw

Re: [PATCH v8 02/10] scripts: add coccinelle script to use auto propagated errp

Quoted 146 lines · Markus Armbruster
Quoted 117 lines · Vladimir Sementsov-Ogievskiy
Quoted 116 lines · Markus Armbruster
Quoted 113 lines · Vladimir Sementsov-Ogievskiy
Quoted 100 lines · Markus Armbruster

Suggest

scripts: Coccinelle script to use auto-propagated errp

or

scripts: Coccinelle script to use ERRP_AUTO_PROPAGATE()

Quoted 8 lines · Vladimir Sementsov-Ogievskiy

Script adds ERRP_AUTO_PROPAGATE macro invocation where appropriate and does corresponding changes in code (look for details in include/qapi/error.h)

Usage example:

spatch --sp-file scripts/coccinelle/auto-propagated-errp.cocci \
    --macro-file scripts/cocci-macro-file.h --in-place --no-show-diff \
    blockdev-nbd.c qemu-nbd.c {block/nbd*,nbd/*,include/block/nbd*}.[hc]

Suggest FILES... instead of a specific set of files.

Quoted 32 lines

---

Cc Eric Blake <[email protected]>
Cc Kevin Wolf <[email protected]>
Cc Max Reitz <[email protected]>
Cc Greg Kurz <[email protected]>
Cc Christian Schoenebeck <[email protected]>
Cc Stefano Stabellini <[email protected]>
Cc Anthony Perard <[email protected]>
Cc Paul Durrant <[email protected]>
Cc Stefan Hajnoczi <[email protected]>
Cc "Philippe Mathieu-Daudé" <[email protected]>
Cc Laszlo Ersek <[email protected]>
Cc Gerd Hoffmann <[email protected]>
Cc Stefan Berger <[email protected]>
Cc Markus Armbruster <[email protected]>
Cc Michael Roth <[email protected]>
    include/qapi/error.h                          |   3 +
    scripts/coccinelle/auto-propagated-errp.cocci | 231 ++++++++++++++++++
    2 files changed, 234 insertions(+)

create mode 100644 scripts/coccinelle/auto-propagated-errp.cocci

include/qapi/error.h +0 -0
index bb9bcf02fb..fbfc6f1c0b 100644

include/qapi/error.h line 215

Quoted 5 lines
include/qapi/error.h +2 -0
@@ -211,6 +211,9 @@
     *         }
     *         ...
     *     }
+ *
+ * For mass conversion use script

mass-conversion (we're not converting mass, we're converting en masse)

scripts/coccinelle/auto-propagated-errp.cocci

Quoted 8 lines
include/qapi/error.h +1 -0
+ *   scripts/coccinelle/auto-propagated-errp.cocci
     */
      #ifndef ERROR_H
scripts/coccinelle/auto-propagated-errp.cocci new file +0 -0
new file mode 100644
index 0000000000..bff274bd6d

Preface to my review of this script: may aim isn't to make it bullet-proof. I want to (1) make it good enough (explained in a jiffie), and (2) automatically identify the spots where it still isn't obviously safe for manual review.

The latter may involve additional scripting. That's okay.

The script is good enough when the number of possibly unsafe spots is low enough for careful manual review.

When I ask for improvements that, in your opinion, go beyond "good enough", please push back. I'm sure we can work it out together.

Quoted 17 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +16 -0
@@ -0,0 +1,231 @@
+// Use ERRP_AUTO_PROPAGATE (see include/qapi/error.h)
+//
+// Copyright (c) 2020 Virtuozzo International GmbH.
+//
+// This program is free software; you can redistribute it and/or modify
+// it under the terms of the GNU General Public License as published by
+// the Free Software Foundation; either version 2 of the License, or
+// (at your option) any later version.
+//
+// This program is distributed in the hope that it will be useful,
+// but WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+// GNU General Public License for more details.
+//
+// You should have received a copy of the GNU General Public License
+// along with this program.  If not, see <http://www.gnu.org/licenses/>.
Quoted 8 lines

scripts/coccinelle/auto-propagated-errp.cocci line 21

Quoted 5 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +5 -0
+//
+// Usage example:
+// spatch --sp-file scripts/coccinelle/auto-propagated-errp.cocci \
+//  --macro-file scripts/cocci-macro-file.h --in-place --no-show-diff \
+//  --max-width 80 blockdev-nbd.c qemu-nbd.c \

You have --max-width 80 here, but not in the commit message. Default seems to be 78. Any particular reason to change it to 80?

Hmm. As I remember, without this parameter, reindenting doesn't work correctly. So, I'm OK with "--max-width 78", but I doubt that it will work without a parameter. Still, may be I'm wrong, we can check it.

Quoted 5 lines
Quoted 5 lines

If you can point to an example where --max-width helps, keep it, and update the commit message to match. Else, drop it.

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +1 -0
+//  {block/nbd*,nbd/*,include/block/nbd*}.[hc]

As our discussion shows, this script is somewhat hard to understand. That's okay, it solves a somewhat thorny problem, and I don't have better ideas. But let me state the intended transformations once more, so I don't get lost in the details.

Motivation:

1. Make error propagation less error-prone and improve stack backtraces

The "receive error in &local_error, propagate to @errp" pattern is tedious, error-prone, and produces unhelpful stack backtraces with &error_abort.

ERRP_AUTO_PROPAGATE() removes manual propagation. It additionally gives us the stack backtraces we want.

2. Improve error messages with &error_fatal

Passing @errp to error_append_hint(), error_prepend() or error_vprepend() is useless when @errp is &error_fatal.

Quoted 7 lines

ERRP_AUTO_PROPAGATE() fixes this by delaying &error_fatal handling until the automatic propagation.

The intended transformation has three parts:

* Replace declaration of @local_err by ERRP_AUTO_PROPAGATE() where needed for 1 or 2.

actually, we just add macro invocation. local_err is removed together with other conversions.

Quoted 163 lines

It's needed when we also drop some error propagation from the function (motivation 1), or the function calls error_append_hint(), error_prepend() or error_vprepend() (motivation 2). A function could do both. I'll refer back to this below.

* Drop error_propagate(errp, local_err)

Special case: error_propagate_prepend(errp, local_err, ...) becomes error_prepend(errp, ...).

Only correct if there is no other such error_propagate() / error_propagate_prepend() on any path from the here to return.

* Replace remaining use of @local_err by *errp

Quoted 147 lines
Quoted 146 lines
Quoted 80 lines
Quoted 22 lines

scripts/coccinelle/auto-propagated-errp.cocci line 24

Quoted 2 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +2 -0
+
+// Switch unusual (Error **) parameter names to errp

Let's drop the parenthesis around Error **

scripts/coccinelle/auto-propagated-errp.cocci line 25

Quoted 1 line
scripts/coccinelle/auto-propagated-errp.cocci new file +1 -0
+// (this is necessary to use ERRP_AUTO_PROPAGATE).

Perhaps ERRP_AUTO_PROPAGATE() should be ERRP_AUTO_PROPAGATE(errp) to make the fact we're messing with @errp more obvious. Too late; I shouldn't rock the boat that much now.

scripts/coccinelle/auto-propagated-errp.cocci line 31

Quoted 6 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +6 -0
+//
+// Disable optional_qualifier to skip functions with "Error *const *errp"
+// parameter.
+//
+// Skip functions with "assert(_errp && *_errp)" statement, as they have
+// non generic semantics and may have unusual Error ** argument name for purpose

non-generic

for a purpose

Quoted 7 lines

Wrap comment lines around column 70, please. It's easier to read.

Maybe

      // Skip functions with "assert(_errp && *_errp)" statement, because that
      // signals unusual semantics, and the parameter name may well serve a
      // purpose.

Sounds good.

Quoted 42 lines

scripts/coccinelle/auto-propagated-errp.cocci line 35

Quoted 5 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +4 -0
+// (like nbd_iter_channel_error()).
+//
+// Skip util/error.c to not touch, for example, error_propagate and
+// error_propagate_prepend().

error_propagate()

I much appreciate your meticulous explanation of what you skip and why.

Quoted 9 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +9 -0
+@ depends on !(file in "util/error.c") disable optional_qualifier@
+identifier fn;
+identifier _errp != errp;
+@@
+
+ fn(...,
+-   Error **_errp
++   Error **errp
+    ,...)

scripts/coccinelle/auto-propagated-errp.cocci line 54

Quoted 10 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +10 -0
+ {
+(
+     ... when != assert(_errp && *_errp)
+&
+     <...
+-    _errp
++    errp
+     ...>
+)
+ }

This rule is required to make the actual transformations (below) work even for parameters with names other than @errp. I believe it's not used in this series. In fact, I can't see a use for it in the entire tree right now. Okay anyway.

scripts/coccinelle/auto-propagated-errp.cocci line 58

Quoted 4 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +4 -0
+
+// Add invocation of ERRP_AUTO_PROPAGATE to errp-functions where necessary
+//
+// Note, that without "when any" final "..." may not want to mach something

s/final "..." may not mach/the final "..." does not match/

Quoted 5 lines

scripts/coccinelle/auto-propagated-errp.cocci line 60

Quoted 2 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +2 -0
+// matched by previous pattern, i.e. the rule will not match double
+// error_prepend in control flow like in vfio_set_irq_signaling().

Can't say I fully understand Coccinelle there. I figure you came to this knowledge the hard way.

Quoted 5 lines

It's follows from smpl grammar document:

"Implicitly, “...” matches the shortest path between something that matches the pattern before the dots (or the beginning of the function, if there is nothing before the dots) and something that matches the pattern after the dots (or the end of the function, if there is nothing after the dots)."
...
"_when any_ removes the aforementioned constraint that “...” matches the shortest path"

Let me think that through.

The pattern with the cases other than error_prepend() omitted:

       fn(..., Error **errp, ...)
       {
      +   ERRP_AUTO_PROPAGATE();
          ...  when != ERRP_AUTO_PROPAGATE();
          error_prepend(errp, ...);
          ... when any
       }

Tail of vfio_set_irq_signaling():

          name = index_to_str(vbasedev, index);
          if (name) {
              error_prepend(errp, "%s-%d: ", name, subindex);
          } else {
              error_prepend(errp, "index %d-%d: ", index, subindex);
          }
          error_prepend(errp,
                        "Failed to %s %s eventfd signaling for interrupt ",
                        fd < 0 ? "tear down" : "set up", action_to_str(action));
          return ret;
      }

The pattern's first ... matches a "shortest" path to an error_prepend(), where "shortest" means "does not cross an error_prepend(). Its when clause makes us ignore functions that already use ERRP_AUTO_PROPAGATE().

There are two such "shortest" paths, one to the first error_prepend() in vfio_set_irq_signaling(), and one to the second. Neither path to the third one is not "shortest": they both cross one of the other two error_prepend().

The pattern' s second ... matches a path from a matched error_prepend() to the end of the function. There are two paths. Both cross the third error_prepend(). You need "when any" to make the pattern match anyway.

Alright, I think I got it. But now I'm paranoid about ... elsewhere. For instance, here's rule1 with error_propagate_prepend() omitted:

      // Match scenarios with propagation of local error to errp.
      @rule1 disable optional_qualifier exists@
      identifier fn, local_err;
      symbol errp;
      @@

       fn(..., Error **errp, ...)
       {
           ...
           Error *local_err = NULL;
           ...
           error_propagate(errp, local_err);
           ...
       }

The second and third ... won't match anything containing error_propagate(). What if a function has multiple error_propagate() on all paths?

Quoted 6 lines
Quoted 1 line

I thought about this, but decided that double error propagation is a strange pattern, and may be better not match it...

I'm fine with not touching "strange" patterns. But we do at least in the example I gave below: we still add ERRP_AUTO_PROPAGATE().

We either avoid that in the Coccinelle script,

I think avoiding is best thing, and it seems possible as you proposed.

Quoted 58 lines

or we catch it
afterwards. Catching it afterwards should be feasible:

* If we also drop some error propagation from this function: okay.

* If this function call error_append_hint(), error_prepend() or error_vprepend(): okay.

* Else: unwanted, back out.

Moreover, I'd like us to double-check we really don't want to touch the things we don't touch. Feels feasible to me, too: after running Coccinelle, search for unconverted error_append_hint(), error_prepend(), error_vprepend(), error_propagate_prepend(), error_propagate().

Quoted 43 lines
Quoted 17 lines

Like this one:

      extern foo(int, Error **);
      extern bar(int, Error **);

      void frob(Error **errp)
      {
          Error *local_err = NULL;
          int arg;

          foo(arg, errp);
          bar(arg, &local_err);
          error_propagate(errp, local_err);
          bar(arg + 1, &local_err);
          error_propagate(errp, local_err);
      }
Quoted 2 lines

This is actually a variation of error.h's "Receive and accumulate multiple errors (first one wins)" code snippet.

ah yes, we can propagate to already filled errp, which just clean local_err.

Quoted 20 lines

The Coccinelle script transforms it like this:

       void frob(Error **errp)
       {
      +    ERRP_AUTO_PROPAGATE();
           Error *local_err = NULL;
           int arg;

The rule that adds ERRP_AUTO_PROPAGATE() matches (it has ... when any), but rule1 does not, and we therefore don't convert any of the error_propagate().

The result isn't wrong, just useless.

Is this the worst case?

Possible improvement to the ERRP_AUTO_PROPAGATE() rule: don't use "... when any" in the error_propagate() case, only in the other cases. Would that help?

Quoted 8 lines
Quoted 2 lines

I think not, as it will anyway match functions with error_prepend (and any number of following error_propagate calls)...

I'm not sure I understand this sentence.

The aim of my "possible improvement" is to avoid unwanted ERRP_AUTO_PROPAGATE() in the Coccinelle script. We can instead search for unwanted ones after the Coccinelle run, and back them out.

You are right.

Quoted 414 lines
Quoted 414 lines
Quoted 413 lines

I think this is the only other rule with "..." matching control flow.

Quoted 86 lines
Quoted 25 lines

scripts/coccinelle/auto-propagated-errp.cocci line 64

Quoted 5 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +4 -0
+//
+// Note, "exists" says that we want apply rule even if it matches not on
+// all possible control flows (otherwise, it will not match standard pattern
+// when error_propagate() call is in if branch).

Learned something new. Example: kvm_set_kvm_shadow_mem().

Spelling it "exists disable optional_qualifier" would avoid giving readers the idea we're disabling "exists", but Coccinelle doesn't let us. Oh well.

scripts/coccinelle/auto-propagated-errp.cocci line 66

Quoted 2 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +2 -0
+@ disable optional_qualifier exists@
+identifier fn, local_err, errp;

I believe this causes

warning: line 98: errp, previously declared as a metavariable, is used as an identifier
warning: line 104: errp, previously declared as a metavariable, is used as an identifier
warning: line 106: errp, previously declared as a metavariable, is used as an identifier
warning: line 131: errp, previously declared as a metavariable, is used as an identifier
warning: line 192: errp, previously declared as a metavariable, is used as an identifier
warning: line 195: errp, previously declared as a metavariable, is used as an identifier
warning: line 228: errp, previously declared as a metavariable, is used as an identifier

Quoted 1 line

Making @errp symbol instead of identifier should fix this.

Hmm, I didn't see these warnings.. But yes, it should be symbol.

Quoted 57 lines
Quoted 32 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +31 -0
+@@
+
+ fn(..., Error **errp, ...)
+ {
++   ERRP_AUTO_PROPAGATE();
+    ...  when != ERRP_AUTO_PROPAGATE();
+(
+    error_append_hint(errp, ...);
+|
+    error_prepend(errp, ...);
+|
+    error_vprepend(errp, ...);
+|
+    Error *local_err = NULL;
+    ...
+(
+    error_propagate_prepend(errp, local_err, ...);
+|
+    error_propagate(errp, local_err);
+)
+)
+    ... when any
+ }
+
+
+// Match scenarios with propagation of local error to errp.
+@rule1 disable optional_qualifier exists@
+identifier fn, local_err;
+symbol errp;
+@@
+

scripts/coccinelle/auto-propagated-errp.cocci line 107

Quoted 10 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +10 -0
+ fn(..., Error **errp, ...)
+ {
+     ...
+     Error *local_err = NULL;
+     ...
+(
+    error_propagate_prepend(errp, local_err, ...);
+|
+    error_propagate(errp, local_err);
+)

Indentation off by one.

scripts/coccinelle/auto-propagated-errp.cocci line 111

Quoted 4 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +4 -0
+     ...
+ }
+
+// Convert special case with goto in separate.

s/in separate/separately/

scripts/coccinelle/auto-propagated-errp.cocci line 113

Quoted 2 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +2 -0
+// We can probably merge this into the following hunk with help of ( | )
+// operator, but it significantly reduce performance on block.c parsing (or it

s/reduce/reduces/

Quoted 6 lines
Quoted 4 lines

scripts/coccinelle/auto-propagated-errp.cocci line 114

Quoted 1 line
scripts/coccinelle/auto-propagated-errp.cocci new file +1 -0
+// hangs, I don't know)

Sounds like you tried to merge this into the following hunk, but then spatch took so long on block.c that you killed it. Correct?

Yes.

I'd say something like "I tried merging this into the following rule the obvious way, but it made Coccinelle hang on block.c."

Quoted 32 lines
Quoted 7 lines

scripts/coccinelle/auto-propagated-errp.cocci line 118

Quoted 5 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +4 -0
+//
+// Note interesting thing: if we don't do it here, and try to fixup "out: }"
+// things later after all transformations (the rule will be the same, just
+// without error_propagate() call), coccinelle fails to match this "out: }".

Weird, but not worth further investigation.

It partially match to the idea which I saw somewhere in coccinelle documentation, that coccinelle converts correct C code to correct C code. "out: }" is an example of incorrect, impossible code flow, and coccinelle can't work with it... But it's just a thought.

Quoted 10 lines
Quoted 10 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +9 -0
+@@
+identifier rule1.fn, rule1.local_err, out;
+symbol errp;
+@@
+
+ fn(...)
+ {
+     <...
+-    goto out;
Quoted 8 lines

scripts/coccinelle/auto-propagated-errp.cocci line 131

Quoted 4 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +4 -0
++    return;
+     ...>
+- out:
+-    error_propagate(errp, local_err);

You neglect to match error_propagate_prepend(). Okay, because (1) that pattern doesn't occur in the tree right now, and (2) if it gets added, gcc will complain.

Quoted 2 lines

No, because it should not removed. error_propagate_prepend should be converted to prepend, not removed. So, corresponding gotos should not be removed as well.

You're right.

Quoted 30 lines
Quoted 30 lines

scripts/coccinelle/auto-propagated-errp.cocci line 134

Quoted 4 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +3 -0
+ }
+
+// Convert most of local_err related staff.

s/staff/stuff/

scripts/coccinelle/auto-propagated-errp.cocci line 141

Quoted 7 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +7 -0
+//
+// Note, that we update everything related to matched by rule1 function name
+// and local_err name. We may match something not related to the pattern
+// matched by rule1. For example, local_err may be defined with the same name
+// in different blocks inside one function, and in one block follow the
+// propagation pattern and in other block doesn't. Or we may have several
+// functions with the same name (for different configurations).

Context: rule1 matches functions that have all three of

* an Error **errp parameter

* an Error *local_err = NULL variable declaration

* an error_propagate(errp, local_err) or error_propagate_prepend(errp, local_err, ...) expression, where @errp is the parameter and @local_err is the variable.

If I understand you correctly, you're pointing out two potential issues:

1. This rule can match functions rule1 does not match if there is another function with the same name that rule1 does match.

Quoted 7 lines
Quoted 5 lines

2. This rule matches in the entire function matched by rule1, even when parts of that function use a different @errp or @local_err.

I figure these apply to all rules with identifier rule1.fn, not just this one. Correct?

Yes.

Thanks!

Quoted 30 lines
Quoted 21 lines

Regarding 1. There must be a better way to chain rules together, but I don't know it.
Can we make Coccinelle at least warn us when it converts
multiple functions with the same name? What about this:

      @initialize:python@
      @@
      fnprev = {}

      def pr(fn, p):
          print("### %s:%s: %s()" % (p[0].file, p[0].line, fn))

      @r@
      identifier rule1.fn;
      position p;
      @@
       fn(...)@p
       {
           ...
       }
Quoted 8 lines
      @script:python@
          fn << rule1.fn;
          p << r.p;
      @@
      if fn not in fnprev:
          fnprev[fn] = p
      else:
          if fnprev[fn]:
Quoted 7 lines

hmm, the condition can't be false

Quoted 3 lines
              pr(fn, fnprev[fn])
              fnprev[fn] = None
          pr(fn, p)

and we'll miss next duplication..

The idea is

first instance of fn:
fn not in fnprev
fnprev[fn] = position of instance
don't print
second instance:
fnprev[fn] is the position of the first instance print first two instances
subsequent instances: fnprev[fn] is None
print this instance

I might have screwed up the coding, of course :)

Quoted 168 lines

But I like the idea.

Quoted 125 lines

For each function @fn matched by rule1, fncnt[fn] is an upper limit of the number of functions with the same name we touch. If it's more than one, we print.

Reports about a dozen function names for the whole tree in my testing. Inspecting the changes to them manually is feasible. None of them are in files touched by this series.

The line printed for the first match is pretty useless for me: it points to a Coccinelle temporary file *shrug*.

Regarding 2. Shadowing @errp or @local_err would be in bad taste, and I sure hope we don't do that. Multiple @local_err variables... hmm. Perhaps we could again concoct some script rules to lead us to spots to check manually. See below for my attempt.

What's the worst that could happen if we blindly converted such code? The answer to that question tells us how hard to work on finding and checking these guys.

Quoted 17 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +17 -0
+//
+// Note also that errp-cleaning functions
+//   error_free_errp
+//   error_report_errp
+//   error_reportf_errp
+//   warn_report_errp
+//   warn_reportf_errp
+// are not yet implemented. They must call corresponding Error* - freeing
+// function and then set *errp to NULL, to avoid further propagation to
+// original errp (consider ERRP_AUTO_PROPAGATE in use).
+// For example, error_free_errp may look like this:
+//
+//    void error_free_errp(Error **errp)
+//    {
+//        error_free(*errp);
+//        *errp = NULL;
+//    }

scripts/coccinelle/auto-propagated-errp.cocci line 168

Quoted 10 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +10 -0
+@ exists@
+identifier rule1.fn, rule1.local_err;
+expression list args;
+symbol errp;
+@@
+
+ fn(...)
+ {
+     <...
+(

Each of the following patterns applies anywhere in the function.

First pattern: delete @local_err

scripts/coccinelle/auto-propagated-errp.cocci line 169

Quoted 1 line
scripts/coccinelle/auto-propagated-errp.cocci new file +1 -0
+-    Error *local_err = NULL;

Common case: occurs just once, not nested. Anything else is suspicious.

Both can be detected in the resulting patches with a bit of AWK wizardry:

       $ git-diff -U0 master..review-error-v8 | awk '/^@@ / { ctx = $5; for (i = 6; i <= NF; i++) ctx = ctx " " $i; if (ctx != octx) { octx = ctx; n = 0 } } /^- *Error *\* *[A-Za-z0-9_]+ *= *NULL;/ { if (index($0, "E") > 6) print "nested\n    " ctx; if (n) print "more than one\n    " ctx; n++ }'
       nested
           static void xen_block_drive_destroy(XenBlockDrive *drive, Error **errp)
       nested
           static void xen_block_device_destroy(XenBackendInstance *backend,
       nested
           static void xen_block_device_destroy(XenBackendInstance *backend,
       more than one
           static void xen_block_device_destroy(XenBackendInstance *backend,

Oh.

xen_block_drive_destroy() nests its Error *local_err in a conditional.

xen_block_device_destroy() has multiple Error *local_err.

In both cases, manual review is required to ensure the conversion is okay. I believe it is.

Note that the AWK script relies on diff showing the function name in @@ lines, which doesn't always work due to our coding style.

For the whole tree, I get some 30 spots. Feasible.

scripts/coccinelle/auto-propagated-errp.cocci line 170

Quoted 1 line
scripts/coccinelle/auto-propagated-errp.cocci new file +1 -0
+|

Second pattern: clear @errp after freeing it

scripts/coccinelle/auto-propagated-errp.cocci line 172

Quoted 2 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +2 -0
+
+// Convert error clearing functions

Suggest: Ensure @local_err is cleared on free

Quoted 6 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +6 -0
+(
+-    error_free(local_err);
++    error_free_errp(errp);
+|
+-    error_report_err(local_err);
++    error_report_errp(errp);

scripts/coccinelle/auto-propagated-errp.cocci line 188

Quoted 10 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +10 -0
+|
+-    error_reportf_err(local_err, args);
++    error_reportf_errp(errp, args);
+|
+-    warn_report_err(local_err);
++    warn_report_errp(errp);
+|
+-    warn_reportf_err(local_err, args);
++    warn_reportf_errp(errp, args);
+)

As you mention above, these guys don't exist, yet. Builds anyway, because this part of the rule is not used in this patch series. You don't want to omit it, because then the script becomes unsafe to use.

We could also open-code:

      // Convert error clearing functions
      (
      -    error_free(local_err);
      +    error_free(*errp);
      +    *errp = NULL;
      |
      ... and so forth ...
      )
Quoted 2 lines

Matter of taste. Whatever is easier to explain in the comments. Since you already wrote one...

I just feel that using helper functions is safer way..

Quoted 22 lines

We talked about extending this series slightly so these guys are used. I may still look into that.

scripts/coccinelle/auto-propagated-errp.cocci line 190

Quoted 2 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +2 -0
+?-    local_err = NULL;
+

The new helpers clear @local_err. Assignment now redundant, delete. Okay.

scripts/coccinelle/auto-propagated-errp.cocci line 191

Quoted 1 line
scripts/coccinelle/auto-propagated-errp.cocci new file +1 -0
+|

Third and fourth pattern: delete error_propagate()

scripts/coccinelle/auto-propagated-errp.cocci line 196

Quoted 5 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +5 -0
+-    error_propagate_prepend(errp, local_err, args);
++    error_prepend(errp, args);
+|
+-    error_propagate(errp, local_err);
+|

Fifth pattern: use @errp directly

Quoted 13 lines

scripts/coccinelle/auto-propagated-errp.cocci line 204

Quoted 8 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +8 -0
+-    &local_err
++    errp
+)
+     ...>
+ }
+
+// Convert remaining local_err usage. It should be different kinds of error
+// checking in if operators. We can't merge this into previous hunk, as this

In if conditionals, I suppose. It's the case for this patch. If I apply the script to the whole tree, the rule gets also applied in other contexts. The sentence might mislead as much as it helps. Keep it or delete it?

Quoted 1 line

Maybe, just be more honest: "It should be ..., but it may be any other pattern, be careful"

"Need to be careful" means "needs careful manual review", which I believe is not feasible; see "Preface to my review of this script" above.

But do we really need to be careful here?

This rule should apply only where we added ERRP_AUTO_PROPAGATE().

Except when rule chaining via function name fails us, but we plan to detect that and review manually, so let's ignore this issue here.

Thanks to ERRP_AUTO_PROPAGATE(), @errp is not null. Enabling replacement of @local_err by @errp is its whole point.

What exactly do we need to be careful about?

Quoted 7 lines
Quoted 1 line

Hmm.. About some unpredicted patterns. OK, then "For example, different kinds of .."

Something like this, perhaps?

        // Convert remaining local_err usage, typically error checking.
        // We can't merge this into previous hunk, as this conflicts with other
        // substitutions in it (at least with "- local_err = NULL").

OK

Quoted 34 lines
Quoted 33 lines
Quoted 33 lines
Quoted 32 lines
Quoted 32 lines
Quoted 6 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +5 -0
+// conflicts with other substitutions in it (at least with "- local_err = NULL").
+@@
+identifier rule1.fn, rule1.local_err;
+symbol errp;
+@@

scripts/coccinelle/auto-propagated-errp.cocci line 219

Quoted 10 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +10 -0
+
+ fn(...)
+ {
+     <...
+-    local_err
++    *errp
+     ...>
+ }
+
+// Always use the same patter for checking error

s/patter/pattern/

Quoted 12 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +12 -0
+@@
+identifier rule1.fn;
+symbol errp;
+@@
+
+ fn(...)
+ {
+     <...
+-    *errp != NULL
++    *errp
+     ...>
+ }
Signature · 7 lines
Best regards,
Vladimir

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
#18 lore raw

Re: [PATCH v8 02/10] scripts: add coccinelle script to use auto propagated errp

In reply to Markus Armbruster
Quoted 329 lines · Markus Armbruster

Suggest

scripts: Coccinelle script to use auto-propagated errp

or

scripts: Coccinelle script to use ERRP_AUTO_PROPAGATE()

Quoted 8 lines · Vladimir Sementsov-Ogievskiy

Script adds ERRP_AUTO_PROPAGATE macro invocation where appropriate and does corresponding changes in code (look for details in include/qapi/error.h)

Usage example:
spatch --sp-file scripts/coccinelle/auto-propagated-errp.cocci \
--macro-file scripts/cocci-macro-file.h --in-place --no-show-diff \
blockdev-nbd.c qemu-nbd.c {block/nbd*,nbd/*,include/block/nbd*}.[hc]

Suggest FILES... instead of a specific set of files.

Quoted 32 lines

---

Cc Eric Blake <[email protected]>
Cc Kevin Wolf <[email protected]>
Cc Max Reitz <[email protected]>
Cc Greg Kurz <[email protected]>
Cc Christian Schoenebeck <[email protected]>
Cc Stefano Stabellini <[email protected]>
Cc Anthony Perard <[email protected]>
Cc Paul Durrant <[email protected]>
Cc Stefan Hajnoczi <[email protected]>
Cc "Philippe Mathieu-Daudé" <[email protected]>
Cc Laszlo Ersek <[email protected]>
Cc Gerd Hoffmann <[email protected]>
Cc Stefan Berger <[email protected]>
Cc Markus Armbruster <[email protected]>
Cc Michael Roth <[email protected]>
  include/qapi/error.h                          |   3 +
  scripts/coccinelle/auto-propagated-errp.cocci | 231 ++++++++++++++++++
  2 files changed, 234 insertions(+)

create mode 100644 scripts/coccinelle/auto-propagated-errp.cocci

include/qapi/error.h +0 -0
index bb9bcf02fb..fbfc6f1c0b 100644

include/qapi/error.h line 215

Quoted 5 lines
include/qapi/error.h +2 -0
@@ -211,6 +211,9 @@
   *         }
   *         ...
   *     }
+ *
+ * For mass conversion use script

mass-conversion (we're not converting mass, we're converting en masse)

scripts/coccinelle/auto-propagated-errp.cocci

Quoted 9 lines
include/qapi/error.h +1 -0
+ *   scripts/coccinelle/auto-propagated-errp.cocci
   */
 
  #ifndef ERROR_H
scripts/coccinelle/auto-propagated-errp.cocci new file +0 -0
new file mode 100644
index 0000000000..bff274bd6d

Preface to my review of this script: may aim isn't to make it bullet-proof. I want to (1) make it good enough (explained in a jiffie), and (2) automatically identify the spots where it still isn't obviously safe for manual review.

The latter may involve additional scripting. That's okay.

The script is good enough when the number of possibly unsafe spots is low enough for careful manual review.

When I ask for improvements that, in your opinion, go beyond "good enough", please push back. I'm sure we can work it out together.

Quoted 12 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +11 -0
@@ -0,0 +1,231 @@
+// Use ERRP_AUTO_PROPAGATE (see include/qapi/error.h)
+//
+// Copyright (c) 2020 Virtuozzo International GmbH.
+//
+// This program is free software; you can redistribute it and/or modify
+// it under the terms of the GNU General Public License as published by
+// the Free Software Foundation; either version 2 of the License, or
+// (at your option) any later version.
+//
+// This program is distributed in the hope that it will be useful,
+// but WITHOUT ANY WARRANTY; without even the implied warranty of

scripts/coccinelle/auto-propagated-errp.cocci line 21

Quoted 10 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +10 -0
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+// GNU General Public License for more details.
+//
+// You should have received a copy of the GNU General Public License
+// along with this program.  If not, see <http://www.gnu.org/licenses/>.
+//
+// Usage example:
+// spatch --sp-file scripts/coccinelle/auto-propagated-errp.cocci \
+//  --macro-file scripts/cocci-macro-file.h --in-place --no-show-diff \
+//  --max-width 80 blockdev-nbd.c qemu-nbd.c \

You have --max-width 80 here, but not in the commit message. Default seems to be 78. Any particular reason to change it to 80?

scripts/coccinelle/auto-propagated-errp.cocci line 24

Quoted 3 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +3 -0
+//  {block/nbd*,nbd/*,include/block/nbd*}.[hc]
+
+// Switch unusual (Error **) parameter names to errp

Let's drop the parenthesis around Error **

scripts/coccinelle/auto-propagated-errp.cocci line 25

Quoted 1 line
scripts/coccinelle/auto-propagated-errp.cocci new file +1 -0
+// (this is necessary to use ERRP_AUTO_PROPAGATE).

Perhaps ERRP_AUTO_PROPAGATE() should be ERRP_AUTO_PROPAGATE(errp) to make the fact we're messing with @errp more obvious. Too late; I shouldn't rock the boat that much now.

scripts/coccinelle/auto-propagated-errp.cocci line 31

Quoted 6 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +6 -0
+//
+// Disable optional_qualifier to skip functions with "Error *const *errp"
+// parameter.
+//
+// Skip functions with "assert(_errp && *_errp)" statement, as they have
+// non generic semantics and may have unusual Error ** argument name for purpose

non-generic

for a purpose

Wrap comment lines around column 70, please. It's easier to read.

Maybe

    // Skip functions with "assert(_errp && *_errp)" statement, because that
    // signals unusual semantics, and the parameter name may well serve a
    // purpose.

scripts/coccinelle/auto-propagated-errp.cocci line 35

Quoted 4 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +4 -0
+// (like nbd_iter_channel_error()).
+//
+// Skip util/error.c to not touch, for example, error_propagate and
+// error_propagate_prepend().

error_propagate()

I much appreciate your meticulous explanation of what you skip and why.

Quoted 9 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +9 -0
+@ depends on !(file in "util/error.c") disable optional_qualifier@
+identifier fn;
+identifier _errp != errp;
+@@
+
+ fn(...,
+-   Error **_errp
++   Error **errp
+    ,...)

scripts/coccinelle/auto-propagated-errp.cocci line 54

Quoted 10 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +10 -0
+ {
+(
+     ... when != assert(_errp && *_errp)
+&
+     <...
+-    _errp
++    errp
+     ...>
+)
+ }

This rule is required to make the actual transformations (below) work even for parameters with names other than @errp. I believe it's not used in this series. In fact, I can't see a use for it in the entire tree right now. Okay anyway.

scripts/coccinelle/auto-propagated-errp.cocci line 58

Quoted 4 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +4 -0
+
+// Add invocation of ERRP_AUTO_PROPAGATE to errp-functions where necessary
+//
+// Note, that without "when any" final "..." may not want to mach something

s/final "..." may not mach/the final "..." does not match/

scripts/coccinelle/auto-propagated-errp.cocci line 60

Quoted 2 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +2 -0
+// matched by previous pattern, i.e. the rule will not match double
+// error_prepend in control flow like in vfio_set_irq_signaling().

Can't say I fully understand Coccinelle there. I figure you came to this knowledge the hard way.

scripts/coccinelle/auto-propagated-errp.cocci line 64

Quoted 4 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +4 -0
+//
+// Note, "exists" says that we want apply rule even if it matches not on
+// all possible control flows (otherwise, it will not match standard pattern
+// when error_propagate() call is in if branch).

Learned something new. Example: kvm_set_kvm_shadow_mem().

Spelling it "exists disable optional_qualifier" would avoid giving readers the idea we're disabling "exists", but Coccinelle doesn't let us. Oh well.

scripts/coccinelle/auto-propagated-errp.cocci line 66

Quoted 2 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +2 -0
+@ disable optional_qualifier exists@
+identifier fn, local_err, errp;

I believe this causes

warning: line 98: errp, previously declared as a metavariable, is used as an identifier
warning: line 104: errp, previously declared as a metavariable, is used as an identifier
warning: line 106: errp, previously declared as a metavariable, is used as an identifier
warning: line 131: errp, previously declared as a metavariable, is used as an identifier
warning: line 192: errp, previously declared as a metavariable, is used as an identifier
warning: line 195: errp, previously declared as a metavariable, is used as an identifier
warning: line 228: errp, previously declared as a metavariable, is used as an identifier

Making @errp symbol instead of identifier should fix this.

Quoted 31 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +31 -0
+@@
+
+ fn(..., Error **errp, ...)
+ {
++   ERRP_AUTO_PROPAGATE();
+    ...  when != ERRP_AUTO_PROPAGATE();
+(
+    error_append_hint(errp, ...);
+|
+    error_prepend(errp, ...);
+|
+    error_vprepend(errp, ...);
+|
+    Error *local_err = NULL;
+    ...
+(
+    error_propagate_prepend(errp, local_err, ...);
+|
+    error_propagate(errp, local_err);
+)
+)
+    ... when any
+ }
+
+
+// Match scenarios with propagation of local error to errp.
+@rule1 disable optional_qualifier exists@
+identifier fn, local_err;
+symbol errp;
+@@
+

scripts/coccinelle/auto-propagated-errp.cocci line 107

Quoted 10 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +10 -0
+ fn(..., Error **errp, ...)
+ {
+     ...
+     Error *local_err = NULL;
+     ...
+(
+    error_propagate_prepend(errp, local_err, ...);
+|
+    error_propagate(errp, local_err);
+)

Indentation off by one.

scripts/coccinelle/auto-propagated-errp.cocci line 111

Quoted 4 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +4 -0
+     ...
+ }
+
+// Convert special case with goto in separate.

s/in separate/separately/

scripts/coccinelle/auto-propagated-errp.cocci line 113

Quoted 2 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +2 -0
+// We can probably merge this into the following hunk with help of ( | )
+// operator, but it significantly reduce performance on block.c parsing (or it

s/reduce/reduces/

scripts/coccinelle/auto-propagated-errp.cocci line 114

Quoted 1 line
scripts/coccinelle/auto-propagated-errp.cocci new file +1 -0
+// hangs, I don't know)

Sounds like you tried to merge this into the following hunk, but then spatch took so long on block.c that you killed it. Correct?

scripts/coccinelle/auto-propagated-errp.cocci line 118

Quoted 4 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +4 -0
+//
+// Note interesting thing: if we don't do it here, and try to fixup "out: }"
+// things later after all transformations (the rule will be the same, just
+// without error_propagate() call), coccinelle fails to match this "out: }".

Weird, but not worth further investigation.

scripts/coccinelle/auto-propagated-errp.cocci line 131

Quoted 13 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +13 -0
+@@
+identifier rule1.fn, rule1.local_err, out;
+symbol errp;
+@@
+
+ fn(...)
+ {
+     <...
+-    goto out;
++    return;
+     ...>
+- out:
+-    error_propagate(errp, local_err);

You neglect to match error_propagate_prepend(). Okay, because (1) that pattern doesn't occur in the tree right now, and (2) if it gets added, gcc will complain.

scripts/coccinelle/auto-propagated-errp.cocci line 134

Quoted 3 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +3 -0
+ }
+
+// Convert most of local_err related staff.

s/staff/stuff/

scripts/coccinelle/auto-propagated-errp.cocci line 141

Quoted 7 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +7 -0
+//
+// Note, that we update everything related to matched by rule1 function name
+// and local_err name. We may match something not related to the pattern
+// matched by rule1. For example, local_err may be defined with the same name
+// in different blocks inside one function, and in one block follow the
+// propagation pattern and in other block doesn't. Or we may have several
+// functions with the same name (for different configurations).

Context: rule1 matches functions that have all three of

* an Error **errp parameter

* an Error *local_err = NULL variable declaration

* an error_propagate(errp, local_err) or error_propagate_prepend(errp, local_err, ...) expression, where @errp is the parameter and @local_err is the variable.

If I understand you correctly, you're pointing out two potential issues:

1. This rule can match functions rule1 does not match if there is another function with the same name that rule1 does match.

Quoted 8 lines

2. This rule matches in the entire function matched by rule1, even when parts of that function use a different @errp or @local_err.

I figure these apply to all rules with identifier rule1.fn, not just this one. Correct?

Regarding 1. There must be a better way to chain rules together, but I don't know it.

Hmm, what about something like this:

@rule1 disable optional_qualifier exists@
identifier fn, local_err;
symbol errp;
@@

fn(..., Error **
- errp
+ ___errp_coccinelle_updating___

    , ...)
 {
     ...
     Error *local_err = NULL;
     ...
(
    error_propagate_prepend(errp, local_err, ...);
|
    error_propagate(errp, local_err);
)
     ...

}

[..]

match symbol ___errp_coccinelle_updating___ in following rules in function header

[..]

@ disable optional_qualifier@
identifier fn, local_err;
symbol errp;
@@

fn(..., Error **
- ___errp_coccinelle_updating___
+ errp

    , ...)
 {
     ...

}

- hacky, but seems not more hacky than python detection, and should work better

Quoted 223 lines

Can we make Coccinelle at least warn us when it converts
multiple functions with the same name? What about this:

    @initialize:python@
    @@
    fnprev = {}

    def pr(fn, p):
        print("### %s:%s: %s()" % (p[0].file, p[0].line, fn))

    @r@
    identifier rule1.fn;
    position p;
    @@
     fn(...)@p
     {
         ...
     }
    @script:python@
        fn << rule1.fn;
        p << r.p;
    @@
    if fn not in fnprev:
        fnprev[fn] = p
    else:
        if fnprev[fn]:
            pr(fn, fnprev[fn])
            fnprev[fn] = None
        pr(fn, p)

For each function @fn matched by rule1, fncnt[fn] is an upper limit of the number of functions with the same name we touch. If it's more than one, we print.

Reports about a dozen function names for the whole tree in my testing. Inspecting the changes to them manually is feasible. None of them are in files touched by this series.

The line printed for the first match is pretty useless for me: it points to a Coccinelle temporary file *shrug*.

Regarding 2. Shadowing @errp or @local_err would be in bad taste, and I sure hope we don't do that. Multiple @local_err variables... hmm. Perhaps we could again concoct some script rules to lead us to spots to check manually. See below for my attempt.

What's the worst that could happen if we blindly converted such code? The answer to that question tells us how hard to work on finding and checking these guys.

Quoted 17 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +17 -0
+//
+// Note also that errp-cleaning functions
+//   error_free_errp
+//   error_report_errp
+//   error_reportf_errp
+//   warn_report_errp
+//   warn_reportf_errp
+// are not yet implemented. They must call corresponding Error* - freeing
+// function and then set *errp to NULL, to avoid further propagation to
+// original errp (consider ERRP_AUTO_PROPAGATE in use).
+// For example, error_free_errp may look like this:
+//
+//    void error_free_errp(Error **errp)
+//    {
+//        error_free(*errp);
+//        *errp = NULL;
+//    }

scripts/coccinelle/auto-propagated-errp.cocci line 168

Quoted 10 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +10 -0
+@ exists@
+identifier rule1.fn, rule1.local_err;
+expression list args;
+symbol errp;
+@@
+
+ fn(...)
+ {
+     <...
+(

Each of the following patterns applies anywhere in the function.

First pattern: delete @local_err

scripts/coccinelle/auto-propagated-errp.cocci line 169

Quoted 1 line
scripts/coccinelle/auto-propagated-errp.cocci new file +1 -0
+-    Error *local_err = NULL;

Common case: occurs just once, not nested. Anything else is suspicious.

Both can be detected in the resulting patches with a bit of AWK wizardry:

     $ git-diff -U0 master..review-error-v8 | awk '/^@@ / { ctx = $5; for (i = 6; i <= NF; i++) ctx = ctx " " $i; if (ctx != octx) { octx = ctx; n = 0 } } /^- *Error *\* *[A-Za-z0-9_]+ *= *NULL;/ { if (index($0, "E") > 6) print "nested\n    " ctx; if (n) print "more than one\n    " ctx; n++ }'
     nested
         static void xen_block_drive_destroy(XenBlockDrive *drive, Error **errp)
     nested
         static void xen_block_device_destroy(XenBackendInstance *backend,
     nested
         static void xen_block_device_destroy(XenBackendInstance *backend,
     more than one
         static void xen_block_device_destroy(XenBackendInstance *backend,

Oh.

xen_block_drive_destroy() nests its Error *local_err in a conditional.

xen_block_device_destroy() has multiple Error *local_err.

In both cases, manual review is required to ensure the conversion is okay. I believe it is.

Note that the AWK script relies on diff showing the function name in @@ lines, which doesn't always work due to our coding style.

For the whole tree, I get some 30 spots. Feasible.

scripts/coccinelle/auto-propagated-errp.cocci line 170

Quoted 1 line
scripts/coccinelle/auto-propagated-errp.cocci new file +1 -0
+|

Second pattern: clear @errp after freeing it

scripts/coccinelle/auto-propagated-errp.cocci line 172

Quoted 2 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +2 -0
+
+// Convert error clearing functions

Suggest: Ensure @local_err is cleared on free

Quoted 6 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +6 -0
+(
+-    error_free(local_err);
++    error_free_errp(errp);
+|
+-    error_report_err(local_err);
++    error_report_errp(errp);

scripts/coccinelle/auto-propagated-errp.cocci line 188

Quoted 10 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +10 -0
+|
+-    error_reportf_err(local_err, args);
++    error_reportf_errp(errp, args);
+|
+-    warn_report_err(local_err);
++    warn_report_errp(errp);
+|
+-    warn_reportf_err(local_err, args);
++    warn_reportf_errp(errp, args);
+)

As you mention above, these guys don't exist, yet. Builds anyway, because this part of the rule is not used in this patch series. You don't want to omit it, because then the script becomes unsafe to use.

We could also open-code:

    // Convert error clearing functions
    (
    -    error_free(local_err);
    +    error_free(*errp);
    +    *errp = NULL;
    |
    ... and so forth ...
    )

Matter of taste. Whatever is easier to explain in the comments. Since you already wrote one...

We talked about extending this series slightly so these guys are used. I may still look into that.

scripts/coccinelle/auto-propagated-errp.cocci line 190

Quoted 2 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +2 -0
+?-    local_err = NULL;
+

The new helpers clear @local_err. Assignment now redundant, delete. Okay.

scripts/coccinelle/auto-propagated-errp.cocci line 191

Quoted 1 line
scripts/coccinelle/auto-propagated-errp.cocci new file +1 -0
+|

Third and fourth pattern: delete error_propagate()

scripts/coccinelle/auto-propagated-errp.cocci line 196

Quoted 5 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +5 -0
+-    error_propagate_prepend(errp, local_err, args);
++    error_prepend(errp, args);
+|
+-    error_propagate(errp, local_err);
+|

Fifth pattern: use @errp directly

scripts/coccinelle/auto-propagated-errp.cocci line 204

Quoted 8 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +8 -0
+-    &local_err
++    errp
+)
+     ...>
+ }
+
+// Convert remaining local_err usage. It should be different kinds of error
+// checking in if operators. We can't merge this into previous hunk, as this

In if conditionals, I suppose. It's the case for this patch. If I apply the script to the whole tree, the rule gets also applied in other contexts. The sentence might mislead as much as it helps. Keep it or delete it?

Quoted 5 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +5 -0
+// conflicts with other substitutions in it (at least with "- local_err = NULL").
+@@
+identifier rule1.fn, rule1.local_err;
+symbol errp;
+@@

scripts/coccinelle/auto-propagated-errp.cocci line 219

Quoted 10 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +10 -0
+
+ fn(...)
+ {
+     <...
+-    local_err
++    *errp
+     ...>
+ }
+
+// Always use the same patter for checking error

s/patter/pattern/

Quoted 12 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +12 -0
+@@
+identifier rule1.fn;
+symbol errp;
+@@
+
+ fn(...)
+ {
+     <...
+-    *errp != NULL
++    *errp
+     ...>
+ }
Signature · 7 lines
Best regards,
Vladimir

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
#19 lore raw

Re: [PATCH v8 02/10] scripts: add coccinelle script to use auto propagated errp

Quoted 10 lines · Vladimir Sementsov-Ogievskiy
Quoted 9 lines · Markus Armbruster

Suggest

scripts: Coccinelle script to use auto-propagated errp

or

scripts: Coccinelle script to use ERRP_AUTO_PROPAGATE()

Vladimir Sementsov-Ogievskiy <[email protected]> writes:

[...]

Quoted 70 lines
Quoted 22 lines

scripts/coccinelle/auto-propagated-errp.cocci line 141

Quoted 6 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +6 -0
+// Note, that we update everything related to matched by rule1 function name
+// and local_err name. We may match something not related to the pattern
+// matched by rule1. For example, local_err may be defined with the same name
+// in different blocks inside one function, and in one block follow the
+// propagation pattern and in other block doesn't. Or we may have several
+// functions with the same name (for different configurations).

Context: rule1 matches functions that have all three of

* an Error **errp parameter

* an Error *local_err = NULL variable declaration

* an error_propagate(errp, local_err) or error_propagate_prepend(errp, local_err, ...) expression, where @errp is the parameter and @local_err is the variable.

If I understand you correctly, you're pointing out two potential issues:

1. This rule can match functions rule1 does not match if there is another function with the same name that rule1 does match.

Quoted 8 lines

2. This rule matches in the entire function matched by rule1, even when parts of that function use a different @errp or @local_err.

I figure these apply to all rules with identifier rule1.fn, not just this one. Correct?

Regarding 1. There must be a better way to chain rules together, but I don't know it.

Hmm, what about something like this:

@rule1 disable optional_qualifier exists@
identifier fn, local_err;
symbol errp;
@@

fn(..., Error **
- errp
+ ___errp_coccinelle_updating___

    , ...)
 {
     ...
     Error *local_err = NULL;
     ...
(
    error_propagate_prepend(errp, local_err, ...);
|
    error_propagate(errp, local_err);
)
     ...

}

[..]

match symbol ___errp_coccinelle_updating___ in following rules in function header

[..]

@ disable optional_qualifier@
identifier fn, local_err;
symbol errp;
@@

fn(..., Error **
- ___errp_coccinelle_updating___
+ errp

Quoted 7 lines
    , ...)
 {
     ...

}

- hacky, but seems not more hacky than python detection, and should work better

As simple, forceful and unsubtle as a sledgehammer. I like it :)

[...]

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

#20 lore raw

Re: [PATCH v8 02/10] scripts: add coccinelle script to use auto propagated errp

Quoted 89 lines · Markus Armbruster
Quoted 10 lines · Vladimir Sementsov-Ogievskiy
Quoted 9 lines · Markus Armbruster

Suggest

scripts: Coccinelle script to use auto-propagated errp

or

scripts: Coccinelle script to use ERRP_AUTO_PROPAGATE()

Vladimir Sementsov-Ogievskiy <[email protected]> writes:

[...]

Quoted 76 lines
Quoted 22 lines

scripts/coccinelle/auto-propagated-errp.cocci line 141

Quoted 6 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +6 -0
+// Note, that we update everything related to matched by rule1 function name
+// and local_err name. We may match something not related to the pattern
+// matched by rule1. For example, local_err may be defined with the same name
+// in different blocks inside one function, and in one block follow the
+// propagation pattern and in other block doesn't. Or we may have several
+// functions with the same name (for different configurations).

Context: rule1 matches functions that have all three of

* an Error **errp parameter

* an Error *local_err = NULL variable declaration

* an error_propagate(errp, local_err) or error_propagate_prepend(errp, local_err, ...) expression, where @errp is the parameter and @local_err is the variable.

If I understand you correctly, you're pointing out two potential issues:

1. This rule can match functions rule1 does not match if there is another function with the same name that rule1 does match.

Quoted 8 lines

2. This rule matches in the entire function matched by rule1, even when parts of that function use a different @errp or @local_err.

I figure these apply to all rules with identifier rule1.fn, not just this one. Correct?

Regarding 1. There must be a better way to chain rules together, but I don't know it.

Hmm, what about something like this:

@rule1 disable optional_qualifier exists@
identifier fn, local_err;
symbol errp;
@@

fn(..., Error **
- errp
+ ___errp_coccinelle_updating___

     , ...)
  {
      ...
      Error *local_err = NULL;
      ...
(
     error_propagate_prepend(errp, local_err, ...);
|
     error_propagate(errp, local_err);
)
      ...

}

[..]

match symbol ___errp_coccinelle_updating___ in following rules in function header

[..]

@ disable optional_qualifier@
identifier fn, local_err;
symbol errp;
@@

fn(..., Error **
- ___errp_coccinelle_updating___
+ errp

     , ...)
  {
      ...

}

Quoted 4 lines
Quoted 1 line

- hacky, but seems not more hacky than python detection, and should work better

As simple, forceful and unsubtle as a sledgehammer. I like it :)

Hmm, not so simple.

It leads to reindenting of function header, which is bad.

Possible solution is instead

fn(...)
{
+ ___errp_coccinelle_updating___();

but this slow down coccinelle. For example, on block.c from ~3s to 1m16s.

.

So, I'm returning to just a warning.

I think something simple like

@@
identifier rule1.fn;
position p != rule1.p;
@@

fn@p(...) {...}

@ script:python@

<print warning>

should work.

Signature · 7 lines
Best regards,
Vladimir

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
#21 lore raw

Re: [PATCH v8 02/10] scripts: add coccinelle script to use auto propagated errp

Quoted 92 lines · Vladimir Sementsov-Ogievskiy
Quoted 91 lines · Markus Armbruster
Quoted 10 lines · Vladimir Sementsov-Ogievskiy
Quoted 9 lines · Markus Armbruster

Suggest

scripts: Coccinelle script to use auto-propagated errp

or

scripts: Coccinelle script to use ERRP_AUTO_PROPAGATE()

Vladimir Sementsov-Ogievskiy <[email protected]> writes:

[...]

Quoted 77 lines
Quoted 22 lines

scripts/coccinelle/auto-propagated-errp.cocci line 141

Quoted 6 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +6 -0
+// Note, that we update everything related to matched by rule1 function name
+// and local_err name. We may match something not related to the pattern
+// matched by rule1. For example, local_err may be defined with the same name
+// in different blocks inside one function, and in one block follow the
+// propagation pattern and in other block doesn't. Or we may have several
+// functions with the same name (for different configurations).

Context: rule1 matches functions that have all three of

* an Error **errp parameter

* an Error *local_err = NULL variable declaration

* an error_propagate(errp, local_err) or error_propagate_prepend(errp, local_err, ...) expression, where @errp is the parameter and @local_err is the variable.

If I understand you correctly, you're pointing out two potential issues:

1. This rule can match functions rule1 does not match if there is another function with the same name that rule1 does match.

Quoted 8 lines

2. This rule matches in the entire function matched by rule1, even when parts of that function use a different @errp or @local_err.

I figure these apply to all rules with identifier rule1.fn, not just this one. Correct?

Regarding 1. There must be a better way to chain rules together, but I don't know it.

Hmm, what about something like this:

@rule1 disable optional_qualifier exists@
identifier fn, local_err;
symbol errp;
@@

fn(..., Error **
- errp
+ ___errp_coccinelle_updating___

     , ...)
  {
      ...
      Error *local_err = NULL;
      ...
(
     error_propagate_prepend(errp, local_err, ...);
|
     error_propagate(errp, local_err);
)
      ...

}

[..]

match symbol ___errp_coccinelle_updating___ in following rules in function header

[..]

@ disable optional_qualifier@
identifier fn, local_err;
symbol errp;
@@

fn(..., Error **
- ___errp_coccinelle_updating___
+ errp

     , ...)
  {
      ...

}

- hacky, but seems not more hacky than python detection, and should work better

Quoted 7 lines
Quoted 2 lines

As simple, forceful and unsubtle as a sledgehammer. I like it :)

Hmm, not so simple.

It leads to reindenting of function header, which is bad.

Because ___errp_coccinelle_updating___ is longer than errp, I guess. Try ____?

Quoted 20 lines

Possible solution is instead

fn(...)
{
+ ___errp_coccinelle_updating___();

but this slow down coccinelle. For example, on block.c from ~3s to 1m16s.

.

So, I'm returning to just a warning.

I think something simple like

@@
identifier rule1.fn;
position p != rule1.p;
@@

Quoted 7 lines

fn@p(...) {...}

@ script:python@

<print warning>

should work.

Up to you.

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

#22 lore raw

Re: [PATCH v8 02/10] scripts: add coccinelle script to use auto propagated errp

Quoted 98 lines · Markus Armbruster
Quoted 96 lines · Vladimir Sementsov-Ogievskiy
Quoted 93 lines · Markus Armbruster
Quoted 10 lines · Vladimir Sementsov-Ogievskiy
Quoted 9 lines · Markus Armbruster

Suggest

scripts: Coccinelle script to use auto-propagated errp

or

scripts: Coccinelle script to use ERRP_AUTO_PROPAGATE()

Vladimir Sementsov-Ogievskiy <[email protected]> writes:

[...]

Quoted 73 lines
Quoted 22 lines

scripts/coccinelle/auto-propagated-errp.cocci line 141

Quoted 6 lines
scripts/coccinelle/auto-propagated-errp.cocci new file +6 -0
+// Note, that we update everything related to matched by rule1 function name
+// and local_err name. We may match something not related to the pattern
+// matched by rule1. For example, local_err may be defined with the same name
+// in different blocks inside one function, and in one block follow the
+// propagation pattern and in other block doesn't. Or we may have several
+// functions with the same name (for different configurations).

Context: rule1 matches functions that have all three of

* an Error **errp parameter

* an Error *local_err = NULL variable declaration

* an error_propagate(errp, local_err) or error_propagate_prepend(errp, local_err, ...) expression, where @errp is the parameter and @local_err is the variable.

If I understand you correctly, you're pointing out two potential issues:

1. This rule can match functions rule1 does not match if there is another function with the same name that rule1 does match.

Quoted 8 lines

2. This rule matches in the entire function matched by rule1, even when parts of that function use a different @errp or @local_err.

I figure these apply to all rules with identifier rule1.fn, not just this one. Correct?

Regarding 1. There must be a better way to chain rules together, but I don't know it.

Hmm, what about something like this:

@rule1 disable optional_qualifier exists@
identifier fn, local_err;
symbol errp;
@@

   fn(..., Error **
- errp
+ ___errp_coccinelle_updating___
      , ...)
   {
       ...
       Error *local_err = NULL;
       ...
(
      error_propagate_prepend(errp, local_err, ...);
|
      error_propagate(errp, local_err);
)
       ...

}

[..]

match symbol ___errp_coccinelle_updating___ in following rules in function header

[..]

@ disable optional_qualifier@
identifier fn, local_err;
symbol errp;
@@

   fn(..., Error **
- ___errp_coccinelle_updating___
+ errp
      , ...)
   {
       ...
Quoted 4 lines

}

- hacky, but seems not more hacky than python detection, and should work better

As simple, forceful and unsubtle as a sledgehammer. I like it :)

Quoted 6 lines
Quoted 3 lines

Hmm, not so simple.

It leads to reindenting of function header, which is bad.

Because ___errp_coccinelle_updating___ is longer than errp, I guess. Try ____?

I'm afraid not. It's because it just adds \n, when I do

...,

- errp
+ ___errp_coccinelle_updating___
,...

Quoted 31 lines
Quoted 21 lines

Possible solution is instead

fn(...)
{
+ ___errp_coccinelle_updating___();

but this slow down coccinelle. For example, on block.c from ~3s to 1m16s.

.

So, I'm returning to just a warning.

I think something simple like

@@
identifier rule1.fn;
position p != rule1.p;
@@

Quoted 7 lines

fn@p(...) {...}

@ script:python@

<print warning>

should work.

Up to you.

Signature · 7 lines
Best regards,
Vladimir

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
#23 lore raw

Re: [PATCH v8 02/10] scripts: add coccinelle script to use auto propagated errp

Quoted 106 lines · Vladimir Sementsov-Ogievskiy
Quoted 98 lines · Markus Armbruster
Quoted 96 lines · Vladimir Sementsov-Ogievskiy
Quoted 93 lines · Markus Armbruster
Quoted 10 lines · Vladimir Sementsov-Ogievskiy
Quoted 9 lines · Markus Armbruster

Suggest

scripts: Coccinelle script to use auto-propagated errp

or

scripts: Coccinelle script to use ERRP_AUTO_PROPAGATE()

Vladimir Sementsov-Ogievskiy <[email protected]> writes:

[...]

Quoted 73 lines
Quoted 22 lines

Review

Quoted 6 lines
(quoted diff) +6 -0
+// Note, that we update everything related to matched by rule1 function name
+// and local_err name. We may match something not related to the pattern
+// matched by rule1. For example, local_err may be defined with the same name
+// in different blocks inside one function, and in one block follow the
+// propagation pattern and in other block doesn't. Or we may have several
+// functions with the same name (for different configurations).

Context: rule1 matches functions that have all three of

* an Error **errp parameter

* an Error *local_err = NULL variable declaration

* an error_propagate(errp, local_err) or error_propagate_prepend(errp, local_err, ...) expression, where @errp is the parameter and @local_err is the variable.

If I understand you correctly, you're pointing out two potential issues:

1. This rule can match functions rule1 does not match if there is another function with the same name that rule1 does match.

Quoted 8 lines

2. This rule matches in the entire function matched by rule1, even when parts of that function use a different @errp or @local_err.

I figure these apply to all rules with identifier rule1.fn, not just this one. Correct?

Regarding 1. There must be a better way to chain rules together, but I don't know it.

Hmm, what about something like this:

@rule1 disable optional_qualifier exists@
identifier fn, local_err;
symbol errp;
@@

   fn(..., Error **
- errp
+ ___errp_coccinelle_updating___
      , ...)
   {
       ...
       Error *local_err = NULL;
       ...
(
      error_propagate_prepend(errp, local_err, ...);
|
      error_propagate(errp, local_err);
)
       ...

}

[..]

match symbol ___errp_coccinelle_updating___ in following rules in function header

[..]

@ disable optional_qualifier@
identifier fn, local_err;
symbol errp;
@@

   fn(..., Error **
- ___errp_coccinelle_updating___
+ errp
      , ...)
   {
       ...
Quoted 4 lines

}

- hacky, but seems not more hacky than python detection, and should work better

As simple, forceful and unsubtle as a sledgehammer. I like it :)

Quoted 6 lines
Quoted 3 lines

Hmm, not so simple.

It leads to reindenting of function header, which is bad.

Because ___errp_coccinelle_updating___ is longer than errp, I guess. Try ____?

Quoted 7 lines

I'm afraid not. It's because it just adds \n, when I do

...,

- errp
+ ___errp_coccinelle_updating___
,...

I was thinking of something like the appended patch, which in my (superficial!) testing leaves alone newlines unless lines are long, but hangs for block.c. Oh well.

scripts/coccinelle/auto-propagated-errp.cocci +44 -31
index bff274bd6d..492a4db826 100644
@@ -35,12 +35,12 @@
 // error_propagate_prepend().
 @ depends on !(file in "util/error.c") disable optional_qualifier@
 identifier fn;
-identifier _errp != errp;
+identifier _errp;
 @@
 
  fn(...,
 -   Error **_errp
-+   Error **errp
++   Error **____
     ,...)
  {
 (
@@ -48,7 +48,7 @@ identifier _errp != errp;
 &
      <...
 -    _errp
-+    errp
++    ____
      ...>
 )
  }
@@ -63,26 +63,26 @@ identifier _errp != errp;
 // all possible control flows (otherwise, it will not match standard pattern
 // when error_propagate() call is in if branch).
 @ disable optional_qualifier exists@
-identifier fn, local_err, errp;
+identifier fn, local_err;
 @@
 
- fn(..., Error **errp, ...)
+ fn(..., Error **____, ...)
  {
 +   ERRP_AUTO_PROPAGATE();
     ...  when != ERRP_AUTO_PROPAGATE();
 (
-    error_append_hint(errp, ...);
+    error_append_hint(____, ...);
 |
-    error_prepend(errp, ...);
+    error_prepend(____, ...);
 |
-    error_vprepend(errp, ...);
+    error_vprepend(____, ...);
 |
     Error *local_err = NULL;
     ...
 (
-    error_propagate_prepend(errp, local_err, ...);
+    error_propagate_prepend(____, local_err, ...);
 |
-    error_propagate(errp, local_err);
+    error_propagate(____, local_err);
 )
 )
     ... when any
@@ -92,18 +92,17 @@ identifier fn, local_err, errp;
 // Match scenarios with propagation of local error to errp.
 @rule1 disable optional_qualifier exists@
 identifier fn, local_err;
-symbol errp;
 @@
 
- fn(..., Error **errp, ...)
+ fn(..., Error **____, ...)
  {
      ...
      Error *local_err = NULL;
      ...
 (
-    error_propagate_prepend(errp, local_err, ...);
+    error_propagate_prepend(____, local_err, ...);
 |
-    error_propagate(errp, local_err);
+    error_propagate(____, local_err);
 )
      ...
  }
@@ -118,7 +117,6 @@ symbol errp;
 // without error_propagate() call), coccinelle fails to match this "out: }".
 @@
 identifier rule1.fn, rule1.local_err, out;
-symbol errp;
 @@
 
  fn(...)
@@ -128,7 +126,7 @@ symbol errp;
 +    return;
      ...>
 - out:
--    error_propagate(errp, local_err);
+-    error_propagate(____, local_err);
  }
 
 // Convert most of local_err related staff.
@@ -159,7 +157,6 @@ symbol errp;
 @ exists@
 identifier rule1.fn, rule1.local_err;
 expression list args;
-symbol errp;
 @@
 
  fn(...)
@@ -172,30 +169,30 @@ symbol errp;
 // Convert error clearing functions
 (
 -    error_free(local_err);
-+    error_free_errp(errp);
++    error_free_errp(____);
 |
 -    error_report_err(local_err);
-+    error_report_errp(errp);
++    error_report_errp(____);
 |
 -    error_reportf_err(local_err, args);
-+    error_reportf_errp(errp, args);
++    error_reportf_errp(____, args);
 |
 -    warn_report_err(local_err);
-+    warn_report_errp(errp);
++    warn_report_errp(____);
 |
 -    warn_reportf_err(local_err, args);
-+    warn_reportf_errp(errp, args);
++    warn_reportf_errp(____, args);
 )
 ?-    local_err = NULL;
 
 |
--    error_propagate_prepend(errp, local_err, args);
-+    error_prepend(errp, args);
+-    error_propagate_prepend(____, local_err, args);
++    error_prepend(____, args);
 |
--    error_propagate(errp, local_err);
+-    error_propagate(____, local_err);
 |
 -    &local_err
-+    errp
++    ____
 )
      ...>
  }
@@ -205,27 +202,43 @@ symbol errp;
 // conflicts with other substitutions in it (at least with "- local_err = NULL").
 @@
 identifier rule1.fn, rule1.local_err;
-symbol errp;
 @@
 
  fn(...)
  {
      <...
 -    local_err
-+    *errp
++    *____
      ...>
  }
 
 // Always use the same patter for checking error
 @@
 identifier rule1.fn;
-symbol errp;
 @@
 
  fn(...)
  {
      <...
--    *errp != NULL
-+    *errp
+-    *____ != NULL
++    *____
      ...>
  }
+
+@@
+identifier fn;
+symbol errp;
+@@
+
+ fn(...,
+-   Error **____
++   Error **errp
+    ,...)
+ {
+ ...
+ }
+
+@@
+@@
+-____
++errp

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

#24 lore raw

Re: [PATCH v8 02/10] scripts: add coccinelle script to use auto propagated errp

Quoted 112 lines · Markus Armbruster
Quoted 110 lines · Vladimir Sementsov-Ogievskiy
Quoted 98 lines · Markus Armbruster
Quoted 96 lines · Vladimir Sementsov-Ogievskiy
Quoted 93 lines · Markus Armbruster
Quoted 10 lines · Vladimir Sementsov-Ogievskiy
Quoted 9 lines · Markus Armbruster

Suggest

scripts: Coccinelle script to use auto-propagated errp

or

scripts: Coccinelle script to use ERRP_AUTO_PROPAGATE()

Vladimir Sementsov-Ogievskiy <[email protected]> writes:

[...]

Quoted 76 lines
Quoted 22 lines

Review

Quoted 6 lines
(quoted diff) +6 -0
+// Note, that we update everything related to matched by rule1 function name
+// and local_err name. We may match something not related to the pattern
+// matched by rule1. For example, local_err may be defined with the same name
+// in different blocks inside one function, and in one block follow the
+// propagation pattern and in other block doesn't. Or we may have several
+// functions with the same name (for different configurations).

Context: rule1 matches functions that have all three of

* an Error **errp parameter

* an Error *local_err = NULL variable declaration

* an error_propagate(errp, local_err) or error_propagate_prepend(errp, local_err, ...) expression, where @errp is the parameter and @local_err is the variable.

If I understand you correctly, you're pointing out two potential issues:

1. This rule can match functions rule1 does not match if there is another function with the same name that rule1 does match.

Quoted 8 lines

2. This rule matches in the entire function matched by rule1, even when parts of that function use a different @errp or @local_err.

I figure these apply to all rules with identifier rule1.fn, not just this one. Correct?

Regarding 1. There must be a better way to chain rules together, but I don't know it.

Hmm, what about something like this:

@rule1 disable optional_qualifier exists@
identifier fn, local_err;
symbol errp;
@@

    fn(..., Error **
- errp
+ ___errp_coccinelle_updating___
       , ...)
    {
        ...
        Error *local_err = NULL;
        ...
(
       error_propagate_prepend(errp, local_err, ...);
|
       error_propagate(errp, local_err);
)
        ...
    }

[..]

match symbol ___errp_coccinelle_updating___ in following rules in function header

[..]

@ disable optional_qualifier@
identifier fn, local_err;
symbol errp;
@@

    fn(..., Error **
- ___errp_coccinelle_updating___
+ errp
       , ...)
    {
        ...
    }
Quoted 1 line

- hacky, but seems not more hacky than python detection, and should work better

As simple, forceful and unsubtle as a sledgehammer. I like it :)

Quoted 6 lines
Quoted 3 lines

Hmm, not so simple.

It leads to reindenting of function header, which is bad.

Because ___errp_coccinelle_updating___ is longer than errp, I guess. Try ____?

I'm afraid not. It's because it just adds \n, when I do

...,

Quoted 7 lines
Quoted 3 lines

- errp
+ ___errp_coccinelle_updating___
,...

I was thinking of something like the appended patch, which in my (superficial!) testing leaves alone newlines unless lines are long, but hangs for block.c. Oh well.

Sorry, I didn't say, but I've checked and I was wrong: ___ works fine. But we need --max-width 80, otherwise coccinelle wraps some lines which fit into 80 characters and produces extra hunks.

So, now I'm preparing v9 with errp->____->errp concept. It differs from your patch and works on block.c

We don't need to switch all errp into ____, it's an extra complication. Changing name only in function header is enough. And don't worry about broken compilation: it's broken anyway among the rules, and all is OK after all rules applied.

Quoted 201 lines
scripts/coccinelle/auto-propagated-errp.cocci +44 -31
index bff274bd6d..492a4db826 100644
@@ -35,12 +35,12 @@
  // error_propagate_prepend().
  @ depends on !(file in "util/error.c") disable optional_qualifier@
  identifier fn;
-identifier _errp != errp;
+identifier _errp;
  @@
 
   fn(...,
  -   Error **_errp
-+   Error **errp
++   Error **____
      ,...)
   {
  (
@@ -48,7 +48,7 @@ identifier _errp != errp;
  &
       <...
  -    _errp
-+    errp
++    ____
       ...>
  )
   }
@@ -63,26 +63,26 @@ identifier _errp != errp;
  // all possible control flows (otherwise, it will not match standard pattern
  // when error_propagate() call is in if branch).
  @ disable optional_qualifier exists@
-identifier fn, local_err, errp;
+identifier fn, local_err;
  @@
 
- fn(..., Error **errp, ...)
+ fn(..., Error **____, ...)
   {
  +   ERRP_AUTO_PROPAGATE();
      ...  when != ERRP_AUTO_PROPAGATE();
  (
-    error_append_hint(errp, ...);
+    error_append_hint(____, ...);
  |
-    error_prepend(errp, ...);
+    error_prepend(____, ...);
  |
-    error_vprepend(errp, ...);
+    error_vprepend(____, ...);
  |
      Error *local_err = NULL;
...
  (
-    error_propagate_prepend(errp, local_err, ...);
+    error_propagate_prepend(____, local_err, ...);
  |
-    error_propagate(errp, local_err);
+    error_propagate(____, local_err);
  )
  )
      ... when any
@@ -92,18 +92,17 @@ identifier fn, local_err, errp;
  // Match scenarios with propagation of local error to errp.
  @rule1 disable optional_qualifier exists@
  identifier fn, local_err;
-symbol errp;
  @@
 
- fn(..., Error **errp, ...)
+ fn(..., Error **____, ...)
   {
...
       Error *local_err = NULL;
...
  (
-    error_propagate_prepend(errp, local_err, ...);
+    error_propagate_prepend(____, local_err, ...);
  |
-    error_propagate(errp, local_err);
+    error_propagate(____, local_err);
  )
...
   }
@@ -118,7 +117,6 @@ symbol errp;
  // without error_propagate() call), coccinelle fails to match this "out: }".
  @@
  identifier rule1.fn, rule1.local_err, out;
-symbol errp;
  @@
 
   fn(...)
@@ -128,7 +126,7 @@ symbol errp;
  +    return;
       ...>
  - out:
--    error_propagate(errp, local_err);
+-    error_propagate(____, local_err);
   }
 
  // Convert most of local_err related staff.
@@ -159,7 +157,6 @@ symbol errp;
  @ exists@
  identifier rule1.fn, rule1.local_err;
  expression list args;
-symbol errp;
  @@
 
   fn(...)
@@ -172,30 +169,30 @@ symbol errp;
  // Convert error clearing functions
  (
  -    error_free(local_err);
-+    error_free_errp(errp);
++    error_free_errp(____);
  |
  -    error_report_err(local_err);
-+    error_report_errp(errp);
++    error_report_errp(____);
  |
  -    error_reportf_err(local_err, args);
-+    error_reportf_errp(errp, args);
++    error_reportf_errp(____, args);
  |
  -    warn_report_err(local_err);
-+    warn_report_errp(errp);
++    warn_report_errp(____);
  |
  -    warn_reportf_err(local_err, args);
-+    warn_reportf_errp(errp, args);
++    warn_reportf_errp(____, args);
  )
  ?-    local_err = NULL;
 
  |
--    error_propagate_prepend(errp, local_err, args);
-+    error_prepend(errp, args);
+-    error_propagate_prepend(____, local_err, args);
++    error_prepend(____, args);
  |
--    error_propagate(errp, local_err);
+-    error_propagate(____, local_err);
  |
  -    &local_err
-+    errp
++    ____
  )
       ...>
   }
@@ -205,27 +202,43 @@ symbol errp;
  // conflicts with other substitutions in it (at least with "- local_err = NULL").
  @@
  identifier rule1.fn, rule1.local_err;
-symbol errp;
  @@
 
   fn(...)
   {
       <...
  -    local_err
-+    *errp
++    *____
       ...>
   }
 
  // Always use the same patter for checking error
  @@
  identifier rule1.fn;
-symbol errp;
  @@
 
   fn(...)
   {
       <...
--    *errp != NULL
-+    *errp
+-    *____ != NULL
++    *____
       ...>
   }
+
+@@
+identifier fn;
+symbol errp;
+@@
+
+ fn(...,
+-   Error **____
++   Error **errp
+    ,...)
+ {
+ ...
+ }
+
+@@
+@@
+-____
++errp
Signature · 7 lines
Best regards,
Vladimir

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
#25 lore raw

Re: [PATCH v8 02/10] scripts: add coccinelle script to use auto propagated errp

In reply to Markus Armbruster
Quoted 3 lines · Markus Armbruster

Review

Quoted 2 lines
(quoted diff) +2 -0
+
+// Convert error clearing functions

Suggest: Ensure @local_err is cleared on free

But there is no local_err after conversion

Quoted 15 lines
Quoted 15 lines

+(

+-    error_free(local_err);
++    error_free_errp(errp);
+|
+-    error_report_err(local_err);
++    error_report_errp(errp);
+|
+-    error_reportf_err(local_err, args);
++    error_reportf_errp(errp, args);
+|
+-    warn_report_err(local_err);
++    warn_report_errp(errp);
+|
+-    warn_reportf_err(local_err, args);
Signature · 7 lines
Best regards,
Vladimir

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
#26 lore raw

Re: [PATCH v8 02/10] scripts: add coccinelle script to use auto propagated errp

Quoted 6 lines · Vladimir Sementsov-Ogievskiy
Quoted 3 lines · Markus Armbruster

Review

Quoted 2 lines
(quoted diff) +2 -0
+
+// Convert error clearing functions

Suggest: Ensure @local_err is cleared on free

But there is no local_err after conversion

True. Hmm. What about this:

     // Convert calls to error_free(), possibly indirect
     // In addition to replacing @local_err by *errp, we have to clear *errp
     // to avoid use-after-free in the automatic error propagation.
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines

+(

+-    error_free(local_err);
++    error_free_errp(errp);
+|
+-    error_report_err(local_err);
++    error_report_errp(errp);
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines

+|

+-    error_reportf_err(local_err, args);
++    error_reportf_errp(errp, args);
+|
+-    warn_report_err(local_err);
++    warn_report_errp(errp);
+|
+-    warn_reportf_err(local_err, args);

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

#27 lore raw

Re: [PATCH v8 02/10] scripts: add coccinelle script to use auto propagated errp

Quoted 7 lines · Markus Armbruster
Quoted 5 lines · Vladimir Sementsov-Ogievskiy
Quoted 3 lines · Markus Armbruster

Review

Quoted 2 lines
(quoted diff) +2 -0
+
+// Convert error clearing functions

Suggest: Ensure @local_err is cleared on free

Quoted 8 lines
Quoted 1 line

But there is no local_err after conversion

True. Hmm. What about this:

      // Convert calls to error_free(), possibly indirect
      // In addition to replacing @local_err by *errp, we have to clear *errp
      // to avoid use-after-free in the automatic error propagation.

OK

Quoted 15 lines
Quoted 14 lines
Quoted 14 lines
Quoted 14 lines

+(

+-    error_free(local_err);
++    error_free_errp(errp);
+|
+-    error_report_err(local_err);
++    error_report_errp(errp);
+|
+-    error_reportf_err(local_err, args);
++    error_reportf_errp(errp, args);
+|
+-    warn_report_err(local_err);
++    warn_report_errp(errp);
+|
+-    warn_reportf_err(local_err, args);
Signature · 7 lines
Best regards,
Vladimir

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
#28 lore raw

Re: [PATCH v8 02/10] scripts: add coccinelle script to use auto propagated errp

In reply to Vladimir Sementsov-Ogievskiy

I think a v9 makes sense now.

If any of the improvement ideas should turn into time sinks for you, let's talk. We don't need perfection, we only need to get to the point where we trust the script to do what we believe it does, understand its limitations, and know how to compensate for them.

Right now I'm optimistic v9 will be ready for merging, perhaps with some minor tweaking in my tree.

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

#29 lore raw

Re: [PATCH v8 02/10] scripts: add coccinelle script to use auto propagated errp

Quoted 10 lines · Markus Armbruster

I think a v9 makes sense now.

If any of the improvement ideas should turn into time sinks for you, let's talk. We don't need perfection, we only need to get to the point where we trust the script to do what we believe it does, understand its limitations, and know how to compensate for them.

Right now I'm optimistic v9 will be ready for merging, perhaps with some minor tweaking in my tree.

Good. I hope, I'll resend today.

Signature · 7 lines
Best regards,
Vladimir

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
#30 lore raw

PATCH v8 3/10 hw/sd/ssi-sd: fix error handling in ssi_sd_realize

In reply to Vladimir Sementsov-Ogievskiy

It's wrong to use same err object as errp parameter for several function calls without intermediate checking for error: we'll crash if try to set err object twice.

Fix that, using new ERRP_AUTO_PROPAGATE macro.

---
 hw/sd/ssi-sd.c | 26 +++++++++++++++++++-------
 1 file changed, 19 insertions(+), 7 deletions(-)
hw/sd/ssi-sd.c +19 -7
index 91db069212..bc44e1a0f5 100644
@@ -241,10 +241,10 @@ static const VMStateDescription vmstate_ssi_sd = {
 
 static void ssi_sd_realize(SSISlave *d, Error **errp)
 {
+    ERRP_AUTO_PROPAGATE();
     ssi_sd_state *s = FROM_SSI_SLAVE(ssi_sd_state, d);
     DeviceState *carddev;
     DriveInfo *dinfo;
-    Error *err = NULL;
 
     qbus_create_inplace(&s->sdbus, sizeof(s->sdbus), TYPE_SD_BUS,
                         DEVICE(d), "sd-bus");
@@ -254,14 +254,26 @@ static void ssi_sd_realize(SSISlave *d, Error **errp)
     dinfo = drive_get_next(IF_SD);
     carddev = qdev_create(BUS(&s->sdbus), TYPE_SD_CARD);
     if (dinfo) {
-        qdev_prop_set_drive(carddev, "drive", blk_by_legacy_dinfo(dinfo), &err);
+        qdev_prop_set_drive(carddev, "drive", blk_by_legacy_dinfo(dinfo), errp);
+        if (*errp) {
+            goto fail;
+        }
+    }
+
+    object_property_set_bool(OBJECT(carddev), true, "spi", errp);
+    if (*errp) {
+        goto fail;
     }
-    object_property_set_bool(OBJECT(carddev), true, "spi", &err);
-    object_property_set_bool(OBJECT(carddev), true, "realized", &err);
-    if (err) {
-        error_setg(errp, "failed to init SD card: %s", error_get_pretty(err));
-        return;
+
+    object_property_set_bool(OBJECT(carddev), true, "realized", errp);
+    if (*errp) {
+        goto fail;
     }
+
+    return;
+
+fail:
+    error_prepend(errp, "failed to init SD card: ");
 }
 
 static void ssi_sd_reset(DeviceState *dev)
Signature · 1 line
2.21.0
#31 lore raw

PATCH v8 4/10 SD (Secure Card): introduce ERRP_AUTO_PROPAGATE

In reply to Vladimir Sementsov-Ogievskiy

If we want to add some info to errp (by error_prepend() or error_append_hint()), we must use the ERRP_AUTO_PROPAGATE macro. Otherwise, this info will not be added when errp == &error_fatal (the program will exit prior to the error_append_hint() or error_prepend() call). Fix such cases.

If we want to check error after errp-function call, we need to introduce local_err and then propagate it to errp. Instead, use ERRP_AUTO_PROPAGATE macro, benefits are:
1. No need of explicit error_propagate call
2. No need of explicit local_err variable: use errp directly
3. ERRP_AUTO_PROPAGATE leaves errp as is if it's not NULL or &error_fatal, this means that we don't break error_abort (we'll abort on error_set, not on error_propagate)

This commit is generated by command

    sed -n '/^SD (Secure Card)$/,/^$/{s/^F: //p}' \
        MAINTAINERS | \
    xargs git ls-files | grep '\.[hc]$' | \
    xargs spatch \
        --sp-file scripts/coccinelle/auto-propagated-errp.cocci \
        --macro-file scripts/cocci-macro-file.h \
        --in-place --no-show-diff --max-width 80
---
 hw/sd/sdhci-pci.c |  7 +++----
 hw/sd/sdhci.c     | 21 +++++++++------------
 2 files changed, 12 insertions(+), 16 deletions(-)
hw/sd/sdhci-pci.c +3 -4
index 19fa8bd8ed..ceaf3ea35b 100644
@@ -29,13 +29,12 @@ static Property sdhci_pci_properties[] = {
 
 static void sdhci_pci_realize(PCIDevice *dev, Error **errp)
 {
+    ERRP_AUTO_PROPAGATE();
     SDHCIState *s = PCI_SDHCI(dev);
-    Error *local_err = NULL;
 
     sdhci_initfn(s);
-    sdhci_common_realize(s, &local_err);
-    if (local_err) {
-        error_propagate(errp, local_err);
+    sdhci_common_realize(s, errp);
+    if (*errp) {
         return;
     }
 
hw/sd/sdhci.c +9 -12
index de63ffb037..c25ac3c099 100644
@@ -1288,7 +1288,7 @@ static const MemoryRegionOps sdhci_mmio_ops = {
 
 static void sdhci_init_readonly_registers(SDHCIState *s, Error **errp)
 {
-    Error *local_err = NULL;
+    ERRP_AUTO_PROPAGATE();
 
     switch (s->sd_spec_version) {
     case 2 ... 3:
@@ -1299,9 +1299,8 @@ static void sdhci_init_readonly_registers(SDHCIState *s, Error **errp)
     }
     s->version = (SDHC_HCVER_VENDOR << 8) | (s->sd_spec_version - 1);
 
-    sdhci_check_capareg(s, &local_err);
-    if (local_err) {
-        error_propagate(errp, local_err);
+    sdhci_check_capareg(s, errp);
+    if (*errp) {
         return;
     }
 }
@@ -1332,11 +1331,10 @@ void sdhci_uninitfn(SDHCIState *s)
 
 void sdhci_common_realize(SDHCIState *s, Error **errp)
 {
-    Error *local_err = NULL;
+    ERRP_AUTO_PROPAGATE();
 
-    sdhci_init_readonly_registers(s, &local_err);
-    if (local_err) {
-        error_propagate(errp, local_err);
+    sdhci_init_readonly_registers(s, errp);
+    if (*errp) {
         return;
     }
     s->buf_maxsz = sdhci_get_fifolen(s);
@@ -1456,13 +1454,12 @@ static void sdhci_sysbus_finalize(Object *obj)
 
 static void sdhci_sysbus_realize(DeviceState *dev, Error **errp)
 {
+    ERRP_AUTO_PROPAGATE();
     SDHCIState *s = SYSBUS_SDHCI(dev);
     SysBusDevice *sbd = SYS_BUS_DEVICE(dev);
-    Error *local_err = NULL;
 
-    sdhci_common_realize(s, &local_err);
-    if (local_err) {
-        error_propagate(errp, local_err);
+    sdhci_common_realize(s, errp);
+    if (*errp) {
         return;
     }
 
Signature · 1 line
2.21.0
#32 lore raw

PATCH v8 5/10 pflash: introduce ERRP_AUTO_PROPAGATE

In reply to Vladimir Sementsov-Ogievskiy

If we want to add some info to errp (by error_prepend() or error_append_hint()), we must use the ERRP_AUTO_PROPAGATE macro. Otherwise, this info will not be added when errp == &error_fatal (the program will exit prior to the error_append_hint() or error_prepend() call). Fix such cases.

If we want to check error after errp-function call, we need to introduce local_err and then propagate it to errp. Instead, use ERRP_AUTO_PROPAGATE macro, benefits are:
1. No need of explicit error_propagate call
2. No need of explicit local_err variable: use errp directly
3. ERRP_AUTO_PROPAGATE leaves errp as is if it's not NULL or &error_fatal, this means that we don't break error_abort (we'll abort on error_set, not on error_propagate)

This commit is generated by command

    sed -n '/^Parallel NOR Flash devices$/,/^$/{s/^F: //p}' \
        MAINTAINERS | \
    xargs git ls-files | grep '\.[hc]$' | \
    xargs spatch \
        --sp-file scripts/coccinelle/auto-propagated-errp.cocci \
        --macro-file scripts/cocci-macro-file.h \
        --in-place --no-show-diff --max-width 80
---
 hw/block/pflash_cfi01.c | 7 +++----
 hw/block/pflash_cfi02.c | 7 +++----
 2 files changed, 6 insertions(+), 8 deletions(-)
hw/block/pflash_cfi01.c +3 -4
index 24f3bce7ef..a5755e6119 100644
@@ -700,12 +700,12 @@ static const MemoryRegionOps pflash_cfi01_ops = {
 
 static void pflash_cfi01_realize(DeviceState *dev, Error **errp)
 {
+    ERRP_AUTO_PROPAGATE();
     PFlashCFI01 *pfl = PFLASH_CFI01(dev);
     uint64_t total_len;
     int ret;
     uint64_t blocks_per_device, sector_len_per_device, device_len;
     int num_devices;
-    Error *local_err = NULL;
 
     if (pfl->sector_len == 0) {
         error_setg(errp, "attribute \"sector-length\" not specified or zero.");
@@ -739,9 +739,8 @@ static void pflash_cfi01_realize(DeviceState *dev, Error **errp)
         &pfl->mem, OBJECT(dev),
         &pflash_cfi01_ops,
         pfl,
-        pfl->name, total_len, &local_err);
-    if (local_err) {
-        error_propagate(errp, local_err);
+        pfl->name, total_len, errp);
+    if (*errp) {
         return;
     }
 
hw/block/pflash_cfi02.c +3 -4
index 12f18d401a..3da9f56391 100644
@@ -718,9 +718,9 @@ static const MemoryRegionOps pflash_cfi02_ops = {
 
 static void pflash_cfi02_realize(DeviceState *dev, Error **errp)
 {
+    ERRP_AUTO_PROPAGATE();
     PFlashCFI02 *pfl = PFLASH_CFI02(dev);
     int ret;
-    Error *local_err = NULL;
 
     if (pfl->uniform_sector_len == 0 && pfl->sector_len[0] == 0) {
         error_setg(errp, "attribute \"sector-length\" not specified or zero.");
@@ -786,9 +786,8 @@ static void pflash_cfi02_realize(DeviceState *dev, Error **errp)
 
     memory_region_init_rom_device(&pfl->orig_mem, OBJECT(pfl),
                                   &pflash_cfi02_ops, pfl, pfl->name,
-                                  pfl->chip_len, &local_err);
-    if (local_err) {
-        error_propagate(errp, local_err);
+                                  pfl->chip_len, errp);
+    if (*errp) {
         return;
     }
 
Signature · 1 line
2.21.0
#33 lore raw

PATCH v8 6/10 fw_cfg: introduce ERRP_AUTO_PROPAGATE

In reply to Vladimir Sementsov-Ogievskiy

If we want to add some info to errp (by error_prepend() or error_append_hint()), we must use the ERRP_AUTO_PROPAGATE macro. Otherwise, this info will not be added when errp == &error_fatal (the program will exit prior to the error_append_hint() or error_prepend() call). Fix such cases.

If we want to check error after errp-function call, we need to introduce local_err and then propagate it to errp. Instead, use ERRP_AUTO_PROPAGATE macro, benefits are:
1. No need of explicit error_propagate call
2. No need of explicit local_err variable: use errp directly
3. ERRP_AUTO_PROPAGATE leaves errp as is if it's not NULL or &error_fatal, this means that we don't break error_abort (we'll abort on error_set, not on error_propagate)

This commit is generated by command

    sed -n '/^Firmware configuration (fw_cfg)$/,/^$/{s/^F: //p}' \
        MAINTAINERS | \
    xargs git ls-files | grep '\.[hc]$' | \
    xargs spatch \
        --sp-file scripts/coccinelle/auto-propagated-errp.cocci \
        --macro-file scripts/cocci-macro-file.h \
        --in-place --no-show-diff --max-width 80
---
 hw/nvram/fw_cfg.c | 14 ++++++--------
 1 file changed, 6 insertions(+), 8 deletions(-)
hw/nvram/fw_cfg.c +6 -8
index 179b302f01..22ff2be0c0 100644
@@ -1142,12 +1142,11 @@ static Property fw_cfg_io_properties[] = {
 
 static void fw_cfg_io_realize(DeviceState *dev, Error **errp)
 {
+    ERRP_AUTO_PROPAGATE();
     FWCfgIoState *s = FW_CFG_IO(dev);
-    Error *local_err = NULL;
 
-    fw_cfg_file_slots_allocate(FW_CFG(s), &local_err);
-    if (local_err) {
-        error_propagate(errp, local_err);
+    fw_cfg_file_slots_allocate(FW_CFG(s), errp);
+    if (*errp) {
         return;
     }
 
@@ -1193,14 +1192,13 @@ static Property fw_cfg_mem_properties[] = {
 
 static void fw_cfg_mem_realize(DeviceState *dev, Error **errp)
 {
+    ERRP_AUTO_PROPAGATE();
     FWCfgMemState *s = FW_CFG_MEM(dev);
     SysBusDevice *sbd = SYS_BUS_DEVICE(dev);
     const MemoryRegionOps *data_ops = &fw_cfg_data_mem_ops;
-    Error *local_err = NULL;
 
-    fw_cfg_file_slots_allocate(FW_CFG(s), &local_err);
-    if (local_err) {
-        error_propagate(errp, local_err);
+    fw_cfg_file_slots_allocate(FW_CFG(s), errp);
+    if (*errp) {
         return;
     }
 
Signature · 1 line
2.21.0
#34 lore raw

PATCH v8 7/10 virtio-9p: introduce ERRP_AUTO_PROPAGATE

In reply to Vladimir Sementsov-Ogievskiy

If we want to add some info to errp (by error_prepend() or error_append_hint()), we must use the ERRP_AUTO_PROPAGATE macro. Otherwise, this info will not be added when errp == &error_fatal (the program will exit prior to the error_append_hint() or error_prepend() call). Fix such cases.

If we want to check error after errp-function call, we need to introduce local_err and then propagate it to errp. Instead, use ERRP_AUTO_PROPAGATE macro, benefits are:
1. No need of explicit error_propagate call
2. No need of explicit local_err variable: use errp directly
3. ERRP_AUTO_PROPAGATE leaves errp as is if it's not NULL or &error_fatal, this means that we don't break error_abort (we'll abort on error_set, not on error_propagate)

This commit is generated by command

    sed -n '/^virtio-9p$/,/^$/{s/^F: //p}' MAINTAINERS | \
    xargs git ls-files | grep '\.[hc]$' | \
    xargs spatch \
        --sp-file scripts/coccinelle/auto-propagated-errp.cocci \
        --macro-file scripts/cocci-macro-file.h \
        --in-place --no-show-diff --max-width 80
---
 hw/9pfs/9p-local.c | 12 +++++-------
 hw/9pfs/9p.c       |  1 +
 2 files changed, 6 insertions(+), 7 deletions(-)
hw/9pfs/9p-local.c +5 -7
index 54e012e5b4..0361e0c0b4 100644
@@ -1479,10 +1479,10 @@ static void error_append_security_model_hint(Error *const *errp)
 
 static int local_parse_opts(QemuOpts *opts, FsDriverEntry *fse, Error **errp)
 {
+    ERRP_AUTO_PROPAGATE();
     const char *sec_model = qemu_opt_get(opts, "security_model");
     const char *path = qemu_opt_get(opts, "path");
     const char *multidevs = qemu_opt_get(opts, "multidevs");
-    Error *local_err = NULL;
 
     if (!sec_model) {
         error_setg(errp, "security_model property not set");
@@ -1516,11 +1516,10 @@ static int local_parse_opts(QemuOpts *opts, FsDriverEntry *fse, Error **errp)
             fse->export_flags &= ~V9FS_FORBID_MULTIDEVS;
             fse->export_flags &= ~V9FS_REMAP_INODES;
         } else {
-            error_setg(&local_err, "invalid multidevs property '%s'",
+            error_setg(errp, "invalid multidevs property '%s'",
                        multidevs);
-            error_append_hint(&local_err, "Valid options are: multidevs="
+            error_append_hint(errp, "Valid options are: multidevs="
                               "[remap|forbid|warn]\n");
-            error_propagate(errp, local_err);
             return -1;
         }
     }
@@ -1530,9 +1529,8 @@ static int local_parse_opts(QemuOpts *opts, FsDriverEntry *fse, Error **errp)
         return -1;
     }
 
-    if (fsdev_throttle_parse_opts(opts, &fse->fst, &local_err)) {
-        error_propagate_prepend(errp, local_err,
-                                "invalid throttle configuration: ");
+    if (fsdev_throttle_parse_opts(opts, &fse->fst, errp)) {
+        error_prepend(errp, "invalid throttle configuration: ");
         return -1;
     }
 
hw/9pfs/9p.c +1 -0
index 9e046f7acb..3aa6a57f3a 100644
@@ -4023,6 +4023,7 @@ void pdu_submit(V9fsPDU *pdu, P9MsgHeader *hdr)
 int v9fs_device_realize_common(V9fsState *s, const V9fsTransport *t,
                                Error **errp)
 {
+    ERRP_AUTO_PROPAGATE();
     int i, len;
     struct stat stat;
     FsDriverEntry *fse;
Signature · 1 line
2.21.0
Reviewed-by on v8 7/10
#35 lore raw

Re: [PATCH v8 07/10] virtio-9p: introduce ERRP_AUTO_PROPAGATE

Quoted 24 lines · On Freitag, 6. März 2020 06:15:33 CET Vladimir Sementsov-Ogievskiy wrote:

If we want to add some info to errp (by error_prepend() or error_append_hint()), we must use the ERRP_AUTO_PROPAGATE macro. Otherwise, this info will not be added when errp == &error_fatal (the program will exit prior to the error_append_hint() or error_prepend() call). Fix such cases.

If we want to check error after errp-function call, we need to introduce local_err and then propagate it to errp. Instead, use ERRP_AUTO_PROPAGATE macro, benefits are:
1. No need of explicit error_propagate call
2. No need of explicit local_err variable: use errp directly
3. ERRP_AUTO_PROPAGATE leaves errp as is if it's not NULL or &error_fatal, this means that we don't break error_abort (we'll abort on error_set, not on error_propagate)

This commit is generated by command

    sed -n '/^virtio-9p$/,/^$/{s/^F: //p}' MAINTAINERS | \
    xargs git ls-files | grep '\.[hc]$' | \
    xargs spatch \
        --sp-file scripts/coccinelle/auto-propagated-errp.cocci \
        --macro-file scripts/cocci-macro-file.h \
        --in-place --no-show-diff --max-width 80
Quoted 4 lines
Quoted 61 lines
---
 hw/9pfs/9p-local.c | 12 +++++-------
 hw/9pfs/9p.c       |  1 +
 2 files changed, 6 insertions(+), 7 deletions(-)
hw/9pfs/9p-local.c +1 -1
index 54e012e5b4..0361e0c0b4 100644
@@ -1479,10 +1479,10 @@ static void error_append_security_model_hint(Error
 *const *errp)
 
 static int local_parse_opts(QemuOpts *opts, FsDriverEntry *fse, Error
 **errp) {
+    ERRP_AUTO_PROPAGATE();
     const char *sec_model = qemu_opt_get(opts, "security_model");
     const char *path = qemu_opt_get(opts, "path");
     const char *multidevs = qemu_opt_get(opts, "multidevs");
-    Error *local_err = NULL;
 
     if (!sec_model) {
         error_setg(errp, "security_model property not set");
hw/9pfs/9p-local.c +0 -0
@@ -1516,11 +1516,10 @@ static int local_parse_opts(QemuOpts *opts,

FsDriverEntry *fse, Error **errp) fse->export_flags &=

~V9FS_FORBID_MULTIDEVS;
             fse->export_flags &= ~V9FS_REMAP_INODES;
         } else {
hw/9pfs/9p-local.c +4 -6
-            error_setg(&local_err, "invalid multidevs property '%s'",
+            error_setg(errp, "invalid multidevs property '%s'",
                        multidevs);
-            error_append_hint(&local_err, "Valid options are: multidevs="
+            error_append_hint(errp, "Valid options are: multidevs="
                               "[remap|forbid|warn]\n");
-            error_propagate(errp, local_err);
             return -1;
         }
     }
@@ -1530,9 +1529,8 @@ static int local_parse_opts(QemuOpts *opts,
 FsDriverEntry *fse, Error **errp) return -1;
     }
 
-    if (fsdev_throttle_parse_opts(opts, &fse->fst, &local_err)) {
-        error_propagate_prepend(errp, local_err,
-                                "invalid throttle configuration: ");
+    if (fsdev_throttle_parse_opts(opts, &fse->fst, errp)) {
+        error_prepend(errp, "invalid throttle configuration: ");
         return -1;
     }
 
hw/9pfs/9p.c +1 -0
index 9e046f7acb..3aa6a57f3a 100644
@@ -4023,6 +4023,7 @@ void pdu_submit(V9fsPDU *pdu, P9MsgHeader *hdr)
 int v9fs_device_realize_common(V9fsState *s, const V9fsTransport *t,
                                Error **errp)
 {
+    ERRP_AUTO_PROPAGATE();
     int i, len;
     struct stat stat;
     FsDriverEntry *fse;
#36 lore raw

PATCH v8 8/10 TPM: introduce ERRP_AUTO_PROPAGATE

In reply to Vladimir Sementsov-Ogievskiy

If we want to add some info to errp (by error_prepend() or error_append_hint()), we must use the ERRP_AUTO_PROPAGATE macro. Otherwise, this info will not be added when errp == &error_fatal (the program will exit prior to the error_append_hint() or error_prepend() call). Fix such cases.

If we want to check error after errp-function call, we need to introduce local_err and then propagate it to errp. Instead, use ERRP_AUTO_PROPAGATE macro, benefits are:
1. No need of explicit error_propagate call
2. No need of explicit local_err variable: use errp directly
3. ERRP_AUTO_PROPAGATE leaves errp as is if it's not NULL or &error_fatal, this means that we don't break error_abort (we'll abort on error_set, not on error_propagate)

This commit is generated by command

    sed -n '/^TPM$/,/^$/{s/^F: //p}' MAINTAINERS | \
    xargs git ls-files | grep '\.[hc]$' | \
    xargs spatch \
        --sp-file scripts/coccinelle/auto-propagated-errp.cocci \
        --macro-file scripts/cocci-macro-file.h \
        --in-place --no-show-diff --max-width 80
---
 hw/tpm/tpm_util.c | 7 +++----
 tpm.c             | 7 +++----
 2 files changed, 6 insertions(+), 8 deletions(-)
hw/tpm/tpm_util.c +3 -4
index c0a0f3d71f..66e0e0c7f3 100644
@@ -47,8 +47,8 @@ static void get_tpm(Object *obj, Visitor *v, const char *name, void *opaque,
 static void set_tpm(Object *obj, Visitor *v, const char *name, void *opaque,
                     Error **errp)
 {
+    ERRP_AUTO_PROPAGATE();
     DeviceState *dev = DEVICE(obj);
-    Error *local_err = NULL;
     Property *prop = opaque;
     TPMBackend *s, **be = qdev_get_prop_ptr(dev, prop);
     char *str;
@@ -58,9 +58,8 @@ static void set_tpm(Object *obj, Visitor *v, const char *name, void *opaque,
         return;
     }
 
-    visit_type_str(v, name, &str, &local_err);
-    if (local_err) {
-        error_propagate(errp, local_err);
+    visit_type_str(v, name, &str, errp);
+    if (*errp) {
         return;
     }
 
tpm.c +3 -4
index 9c9e20bbb7..359ebb7f68 100644
@@ -81,11 +81,11 @@ TPMBackend *qemu_find_tpm_be(const char *id)
 
 static int tpm_init_tpmdev(void *dummy, QemuOpts *opts, Error **errp)
 {
+    ERRP_AUTO_PROPAGATE();
     const char *value;
     const char *id;
     const TPMBackendClass *be;
     TPMBackend *drv;
-    Error *local_err = NULL;
     int i;
 
     if (!QLIST_EMPTY(&tpm_backends)) {
@@ -116,9 +116,8 @@ static int tpm_init_tpmdev(void *dummy, QemuOpts *opts, Error **errp)
     }
 
     /* validate backend specific opts */
-    qemu_opts_validate(opts, be->opts, &local_err);
-    if (local_err) {
-        error_propagate(errp, local_err);
+    qemu_opts_validate(opts, be->opts, errp);
+    if (*errp) {
         return 1;
     }
 
Signature · 1 line
2.21.0
#37 lore raw

PATCH v8 9/10 nbd: introduce ERRP_AUTO_PROPAGATE

In reply to Vladimir Sementsov-Ogievskiy

If we want to add some info to errp (by error_prepend() or error_append_hint()), we must use the ERRP_AUTO_PROPAGATE macro. Otherwise, this info will not be added when errp == &error_fatal (the program will exit prior to the error_append_hint() or error_prepend() call). Fix such cases.

If we want to check error after errp-function call, we need to introduce local_err and then propagate it to errp. Instead, use ERRP_AUTO_PROPAGATE macro, benefits are:
1. No need of explicit error_propagate call
2. No need of explicit local_err variable: use errp directly
3. ERRP_AUTO_PROPAGATE leaves errp as is if it's not NULL or &error_fatal, this means that we don't break error_abort (we'll abort on error_set, not on error_propagate)

This commit is generated by command

    sed -n '/^Network Block Device (NBD)$/,/^$/{s/^F: //p}' \
        MAINTAINERS | \
    xargs git ls-files | grep '\.[hc]$' | \
    xargs spatch \
        --sp-file scripts/coccinelle/auto-propagated-errp.cocci \
        --macro-file scripts/cocci-macro-file.h \
        --in-place --no-show-diff --max-width 80
---
 include/block/nbd.h |  1 +
 block/nbd.c         | 21 +++++++++------------
 nbd/client.c        |  5 +++++
 nbd/server.c        |  5 +++++
 4 files changed, 20 insertions(+), 12 deletions(-)
include/block/nbd.h +1 -0
index 7f46932d80..4ab8d917ed 100644
@@ -360,6 +360,7 @@ void nbd_server_start(SocketAddress *addr, const char *tls_creds,
 static inline int nbd_read(QIOChannel *ioc, void *buffer, size_t size,
                            const char *desc, Error **errp)
 {
+    ERRP_AUTO_PROPAGATE();
     int ret = qio_channel_read_all(ioc, buffer, size, errp) < 0 ? -EIO : 0;
 
     if (ret < 0) {
block/nbd.c +9 -12
index 976be76647..70274c31de 100644
@@ -1410,16 +1410,15 @@ static void nbd_client_close(BlockDriverState *bs)
 static QIOChannelSocket *nbd_establish_connection(SocketAddress *saddr,
                                                   Error **errp)
 {
+    ERRP_AUTO_PROPAGATE();
     QIOChannelSocket *sioc;
-    Error *local_err = NULL;
 
     sioc = qio_channel_socket_new();
     qio_channel_set_name(QIO_CHANNEL(sioc), "nbd-client");
 
-    qio_channel_socket_connect_sync(sioc, saddr, &local_err);
-    if (local_err) {
+    qio_channel_socket_connect_sync(sioc, saddr, errp);
+    if (*errp) {
         object_unref(OBJECT(sioc));
-        error_propagate(errp, local_err);
         return NULL;
     }
 
@@ -1725,10 +1724,10 @@ static bool nbd_process_legacy_socket_options(QDict *output_options,
 static SocketAddress *nbd_config(BDRVNBDState *s, QDict *options,
                                  Error **errp)
 {
+    ERRP_AUTO_PROPAGATE();
     SocketAddress *saddr = NULL;
     QDict *addr = NULL;
     Visitor *iv = NULL;
-    Error *local_err = NULL;
 
     qdict_extract_subqdict(options, &addr, "server.");
     if (!qdict_size(addr)) {
@@ -1741,9 +1740,8 @@ static SocketAddress *nbd_config(BDRVNBDState *s, QDict *options,
         goto done;
     }
 
-    visit_type_SocketAddress(iv, NULL, &saddr, &local_err);
-    if (local_err) {
-        error_propagate(errp, local_err);
+    visit_type_SocketAddress(iv, NULL, &saddr, errp);
+    if (*errp) {
         goto done;
     }
 
@@ -1836,15 +1834,14 @@ static QemuOptsList nbd_runtime_opts = {
 static int nbd_process_options(BlockDriverState *bs, QDict *options,
                                Error **errp)
 {
+    ERRP_AUTO_PROPAGATE();
     BDRVNBDState *s = bs->opaque;
     QemuOpts *opts;
-    Error *local_err = NULL;
     int ret = -EINVAL;
 
     opts = qemu_opts_create(&nbd_runtime_opts, NULL, 0, &error_abort);
-    qemu_opts_absorb_qdict(opts, options, &local_err);
-    if (local_err) {
-        error_propagate(errp, local_err);
+    qemu_opts_absorb_qdict(opts, options, errp);
+    if (*errp) {
         goto error;
     }
 
nbd/client.c +5 -0
index ba173108ba..e258ef3f7e 100644
@@ -68,6 +68,7 @@ static int nbd_send_option_request(QIOChannel *ioc, uint32_t opt,
                                    uint32_t len, const char *data,
                                    Error **errp)
 {
+    ERRP_AUTO_PROPAGATE();
     NBDOption req;
     QEMU_BUILD_BUG_ON(sizeof(req) != 16);
 
@@ -153,6 +154,7 @@ static int nbd_receive_option_reply(QIOChannel *ioc, uint32_t opt,
 static int nbd_handle_reply_err(QIOChannel *ioc, NBDOptionReply *reply,
                                 bool strict, Error **errp)
 {
+    ERRP_AUTO_PROPAGATE();
     g_autofree char *msg = NULL;
 
     if (!(reply->type & (1 << 31))) {
@@ -337,6 +339,7 @@ static int nbd_receive_list(QIOChannel *ioc, char **name, char **description,
 static int nbd_opt_info_or_go(QIOChannel *ioc, uint32_t opt,
                               NBDExportInfo *info, Error **errp)
 {
+    ERRP_AUTO_PROPAGATE();
     NBDOptionReply reply;
     uint32_t len = strlen(info->name);
     uint16_t type;
@@ -882,6 +885,7 @@ static int nbd_start_negotiate(AioContext *aio_context, QIOChannel *ioc,
                                bool structured_reply, bool *zeroes,
                                Error **errp)
 {
+    ERRP_AUTO_PROPAGATE();
     uint64_t magic;
 
     trace_nbd_start_negotiate(tlscreds, hostname ? hostname : "<null>");
@@ -1017,6 +1021,7 @@ int nbd_receive_negotiate(AioContext *aio_context, QIOChannel *ioc,
                           const char *hostname, QIOChannel **outioc,
                           NBDExportInfo *info, Error **errp)
 {
+    ERRP_AUTO_PROPAGATE();
     int result;
     bool zeroes;
     bool base_allocation = info->base_allocation;
nbd/server.c +5 -0
index 11a31094ff..ffc0cc69a7 100644
@@ -211,6 +211,7 @@ static int GCC_FMT_ATTR(4, 0)
 nbd_negotiate_send_rep_verr(NBDClient *client, uint32_t type,
                             Error **errp, const char *fmt, va_list va)
 {
+    ERRP_AUTO_PROPAGATE();
     g_autofree char *msg = NULL;
     int ret;
     size_t len;
@@ -369,6 +370,7 @@ static int nbd_opt_read_name(NBDClient *client, char **name, uint32_t *length,
 static int nbd_negotiate_send_rep_list(NBDClient *client, NBDExport *exp,
                                        Error **errp)
 {
+    ERRP_AUTO_PROPAGATE();
     size_t name_len, desc_len;
     uint32_t len;
     const char *name = exp->name ? exp->name : "";
@@ -432,6 +434,7 @@ static void nbd_check_meta_export(NBDClient *client)
 static int nbd_negotiate_handle_export_name(NBDClient *client, bool no_zeroes,
                                             Error **errp)
 {
+    ERRP_AUTO_PROPAGATE();
     g_autofree char *name = NULL;
     char buf[NBD_REPLY_EXPORT_NAME_SIZE] = "";
     size_t len;
@@ -1272,6 +1275,7 @@ static int nbd_negotiate_options(NBDClient *client, Error **errp)
  */
 static coroutine_fn int nbd_negotiate(NBDClient *client, Error **errp)
 {
+    ERRP_AUTO_PROPAGATE();
     char buf[NBD_OLDSTYLE_NEGOTIATE_SIZE] = "";
     int ret;
 
@@ -1646,6 +1650,7 @@ void nbd_export_close(NBDExport *exp)
 
 void nbd_export_remove(NBDExport *exp, NbdServerRemoveMode mode, Error **errp)
 {
+    ERRP_AUTO_PROPAGATE();
     if (mode == NBD_SERVER_REMOVE_MODE_HARD || QTAILQ_EMPTY(&exp->clients)) {
         nbd_export_close(exp);
         return;
Signature · 1 line
2.21.0
Reviewed-by on v8 9/10
#38 lore raw

Re: [PATCH v8 09/10] nbd: introduce ERRP_AUTO_PROPAGATE

Quoted 25 lines · Vladimir Sementsov-Ogievskiy

If we want to add some info to errp (by error_prepend() or error_append_hint()), we must use the ERRP_AUTO_PROPAGATE macro. Otherwise, this info will not be added when errp == &error_fatal (the program will exit prior to the error_append_hint() or error_prepend() call). Fix such cases.

If we want to check error after errp-function call, we need to introduce local_err and then propagate it to errp. Instead, use ERRP_AUTO_PROPAGATE macro, benefits are:
1. No need of explicit error_propagate call
2. No need of explicit local_err variable: use errp directly
3. ERRP_AUTO_PROPAGATE leaves errp as is if it's not NULL or &error_fatal, this means that we don't break error_abort (we'll abort on error_set, not on error_propagate)

This commit is generated by command

     sed -n '/^Network Block Device (NBD)$/,/^$/{s/^F: //p}' \
         MAINTAINERS | \
     xargs git ls-files | grep '\.[hc]$' | \
     xargs spatch \
         --sp-file scripts/coccinelle/auto-propagated-errp.cocci \
         --macro-file scripts/cocci-macro-file.h \
         --in-place --no-show-diff --max-width 80
Quoted 4 lines

---

Signature · 3 lines
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3226
Virtualization:  qemu.org | libvirt.org
#39 lore raw

PATCH v8 10/10 xen: introduce ERRP_AUTO_PROPAGATE

In reply to Vladimir Sementsov-Ogievskiy

If we want to add some info to errp (by error_prepend() or error_append_hint()), we must use the ERRP_AUTO_PROPAGATE macro. Otherwise, this info will not be added when errp == &error_fatal (the program will exit prior to the error_append_hint() or error_prepend() call). Fix such cases.

If we want to check error after errp-function call, we need to introduce local_err and then propagate it to errp. Instead, use ERRP_AUTO_PROPAGATE macro, benefits are:
1. No need of explicit error_propagate call
2. No need of explicit local_err variable: use errp directly
3. ERRP_AUTO_PROPAGATE leaves errp as is if it's not NULL or &error_fatal, this means that we don't break error_abort (we'll abort on error_set, not on error_propagate)

This commit is generated by command

    sed -n '/^X86 Xen CPUs$/,/^$/{s/^F: //p}' MAINTAINERS | \
    xargs git ls-files | grep '\.[hc]$' | \
    xargs spatch \
        --sp-file scripts/coccinelle/auto-propagated-errp.cocci \
        --macro-file scripts/cocci-macro-file.h \
        --in-place --no-show-diff --max-width 80
---
 hw/block/dataplane/xen-block.c |  17 ++---
 hw/block/xen-block.c           | 125 ++++++++++++++-------------------
 hw/pci-host/xen_igd_pt.c       |   7 +-
 hw/xen/xen-backend.c           |   7 +-
 hw/xen/xen-bus.c               |  92 ++++++++++--------------
 hw/xen/xen-host-pci-device.c   |  27 ++++---
 hw/xen/xen_pt.c                |  25 +++----
 hw/xen/xen_pt_config_init.c    |  20 +++---
 8 files changed, 138 insertions(+), 182 deletions(-)
hw/block/dataplane/xen-block.c +7 -10
index 288a87a814..3a8469fe87 100644
@@ -739,8 +739,8 @@ void xen_block_dataplane_start(XenBlockDataPlane *dataplane,
                                unsigned int protocol,
                                Error **errp)
 {
+    ERRP_AUTO_PROPAGATE();
     XenDevice *xendev = dataplane->xendev;
-    Error *local_err = NULL;
     unsigned int ring_size;
     unsigned int i;
 
@@ -776,9 +776,8 @@ void xen_block_dataplane_start(XenBlockDataPlane *dataplane,
     }
 
     xen_device_set_max_grant_refs(xendev, dataplane->nr_ring_ref,
-                                  &local_err);
-    if (local_err) {
-        error_propagate(errp, local_err);
+                                  errp);
+    if (*errp) {
         goto stop;
     }
 
@@ -786,9 +785,8 @@ void xen_block_dataplane_start(XenBlockDataPlane *dataplane,
                                               dataplane->ring_ref,
                                               dataplane->nr_ring_ref,
                                               PROT_READ | PROT_WRITE,
-                                              &local_err);
-    if (local_err) {
-        error_propagate(errp, local_err);
+                                              errp);
+    if (*errp) {
         goto stop;
     }
 
@@ -821,9 +819,8 @@ void xen_block_dataplane_start(XenBlockDataPlane *dataplane,
     dataplane->event_channel =
         xen_device_bind_event_channel(xendev, event_channel,
                                       xen_block_dataplane_event, dataplane,
-                                      &local_err);
-    if (local_err) {
-        error_propagate(errp, local_err);
+                                      errp);
+    if (*errp) {
         goto stop;
     }
 
hw/block/xen-block.c +52 -73

301 lines not on this page · raw message

hw/pci-host/xen_igd_pt.c +3 -4
index efcc9347ff..29ade9ca25 100644
@@ -79,17 +79,16 @@ static void host_pci_config_read(int pos, int len, uint32_t *val, Error **errp)
 
 static void igd_pt_i440fx_realize(PCIDevice *pci_dev, Error **errp)
 {
+    ERRP_AUTO_PROPAGATE();
     uint32_t val = 0;
     size_t i;
     int pos, len;
-    Error *local_err = NULL;
 
     for (i = 0; i < ARRAY_SIZE(igd_host_bridge_infos); i++) {
         pos = igd_host_bridge_infos[i].offset;
         len = igd_host_bridge_infos[i].len;
-        host_pci_config_read(pos, len, &val, &local_err);
-        if (local_err) {
-            error_propagate(errp, local_err);
+        host_pci_config_read(pos, len, &val, errp);
+        if (*errp) {
             return;
         }
         pci_default_write_config(pci_dev, pos, val, len);
hw/xen/xen-backend.c +3 -4
index da065f81b7..1cc0694053 100644
@@ -98,9 +98,9 @@ static void xen_backend_list_remove(XenBackendInstance *backend)
 void xen_backend_device_create(XenBus *xenbus, const char *type,
                                const char *name, QDict *opts, Error **errp)
 {
+    ERRP_AUTO_PROPAGATE();
     const XenBackendImpl *impl = xen_backend_table_lookup(type);
     XenBackendInstance *backend;
-    Error *local_error = NULL;
 
     if (!impl) {
         return;
@@ -110,9 +110,8 @@ void xen_backend_device_create(XenBus *xenbus, const char *type,
     backend->xenbus = xenbus;
     backend->name = g_strdup(name);
 
-    impl->create(backend, opts, &local_error);
-    if (local_error) {
-        error_propagate(errp, local_error);
+    impl->create(backend, opts, errp);
+    if (*errp) {
         g_free(backend->name);
         g_free(backend);
         return;
hw/xen/xen-bus.c +39 -53

219 lines not on this page · raw message

hw/xen/xen-host-pci-device.c +13 -14
index 1b44dcafaf..02379c341c 100644
@@ -333,8 +333,8 @@ void xen_host_pci_device_get(XenHostPCIDevice *d, uint16_t domain,
                              uint8_t bus, uint8_t dev, uint8_t func,
                              Error **errp)
 {
+    ERRP_AUTO_PROPAGATE();
     unsigned int v;
-    Error *err = NULL;
 
     d->config_fd = -1;
     d->domain = domain;
@@ -342,36 +342,36 @@ void xen_host_pci_device_get(XenHostPCIDevice *d, uint16_t domain,
     d->dev = dev;
     d->func = func;
 
-    xen_host_pci_config_open(d, &err);
-    if (err) {
+    xen_host_pci_config_open(d, errp);
+    if (*errp) {
         goto error;
     }
 
-    xen_host_pci_get_resource(d, &err);
-    if (err) {
+    xen_host_pci_get_resource(d, errp);
+    if (*errp) {
         goto error;
     }
 
-    xen_host_pci_get_hex_value(d, "vendor", &v, &err);
-    if (err) {
+    xen_host_pci_get_hex_value(d, "vendor", &v, errp);
+    if (*errp) {
         goto error;
     }
     d->vendor_id = v;
 
-    xen_host_pci_get_hex_value(d, "device", &v, &err);
-    if (err) {
+    xen_host_pci_get_hex_value(d, "device", &v, errp);
+    if (*errp) {
         goto error;
     }
     d->device_id = v;
 
-    xen_host_pci_get_dec_value(d, "irq", &v, &err);
-    if (err) {
+    xen_host_pci_get_dec_value(d, "irq", &v, errp);
+    if (*errp) {
         goto error;
     }
     d->irq = v;
 
-    xen_host_pci_get_hex_value(d, "class", &v, &err);
-    if (err) {
+    xen_host_pci_get_hex_value(d, "class", &v, errp);
+    if (*errp) {
         goto error;
     }
     d->class_code = v;
@@ -381,7 +381,6 @@ void xen_host_pci_device_get(XenHostPCIDevice *d, uint16_t domain,
     return;
 
 error:
-    error_propagate(errp, err);
 
     if (d->config_fd >= 0) {
         close(d->config_fd);
hw/xen/xen_pt.c +11 -14
index b91082cb8b..f57b81588e 100644
@@ -767,12 +767,12 @@ static void xen_pt_destroy(PCIDevice *d) {
 
 static void xen_pt_realize(PCIDevice *d, Error **errp)
 {
+    ERRP_AUTO_PROPAGATE();
     XenPCIPassthroughState *s = XEN_PT_DEVICE(d);
     int i, rc = 0;
     uint8_t machine_irq = 0, scratch;
     uint16_t cmd = 0;
     int pirq = XEN_PT_UNASSIGNED_PIRQ;
-    Error *err = NULL;
 
     /* register real device */
     XEN_PT_LOG(d, "Assigning real physical device %02x:%02x.%d"
@@ -783,10 +783,9 @@ static void xen_pt_realize(PCIDevice *d, Error **errp)
     xen_host_pci_device_get(&s->real_device,
                             s->hostaddr.domain, s->hostaddr.bus,
                             s->hostaddr.slot, s->hostaddr.function,
-                            &err);
-    if (err) {
-        error_append_hint(&err, "Failed to \"open\" the real pci device");
-        error_propagate(errp, err);
+                            errp);
+    if (*errp) {
+        error_append_hint(errp, "Failed to \"open\" the real pci device");
         return;
     }
 
@@ -813,11 +812,10 @@ static void xen_pt_realize(PCIDevice *d, Error **errp)
             return;
         }
 
-        xen_pt_setup_vga(s, &s->real_device, &err);
-        if (err) {
-            error_append_hint(&err, "Setup VGA BIOS of passthrough"
-                    " GFX failed");
-            error_propagate(errp, err);
+        xen_pt_setup_vga(s, &s->real_device, errp);
+        if (*errp) {
+            error_append_hint(errp, "Setup VGA BIOS of passthrough"
+                              " GFX failed");
             xen_host_pci_device_put(&s->real_device);
             return;
         }
@@ -830,10 +828,9 @@ static void xen_pt_realize(PCIDevice *d, Error **errp)
     xen_pt_register_regions(s, &cmd);
 
     /* reinitialize each config register to be emulated */
-    xen_pt_config_init(s, &err);
-    if (err) {
-        error_append_hint(&err, "PCI Config space initialisation failed");
-        error_propagate(errp, err);
+    xen_pt_config_init(s, errp);
+    if (*errp) {
+        error_append_hint(errp, "PCI Config space initialisation failed");
         rc = -1;
         goto err_out;
     }
hw/xen/xen_pt_config_init.c +10 -10
index 31ec5add1d..af3fbd1bfb 100644
@@ -2008,8 +2008,8 @@ static void xen_pt_config_reg_init(XenPCIPassthroughState *s,
 
 void xen_pt_config_init(XenPCIPassthroughState *s, Error **errp)
 {
+    ERRP_AUTO_PROPAGATE();
     int i, rc;
-    Error *err = NULL;
 
     QLIST_INIT(&s->reg_grps);
 
@@ -2052,10 +2052,9 @@ void xen_pt_config_init(XenPCIPassthroughState *s, Error **errp)
                                                   reg_grp_offset,
                                                   &reg_grp_entry->size);
             if (rc < 0) {
-                error_setg(&err, "Failed to initialize %d/%zu, type = 0x%x,"
+                error_setg(errp, "Failed to initialize %d/%zu, type = 0x%x,"
                            " rc: %d", i, ARRAY_SIZE(xen_pt_emu_reg_grps),
                            xen_pt_emu_reg_grps[i].grp_type, rc);
-                error_propagate(errp, err);
                 xen_pt_config_delete(s);
                 return;
             }
@@ -2068,13 +2067,14 @@ void xen_pt_config_init(XenPCIPassthroughState *s, Error **errp)
 
                 /* initialize capability register */
                 for (j = 0; regs->size != 0; j++, regs++) {
-                    xen_pt_config_reg_init(s, reg_grp_entry, regs, &err);
-                    if (err) {
-                        error_append_hint(&err, "Failed to init register %d"
-                                " offsets 0x%x in grp_type = 0x%x (%d/%zu)", j,
-                                regs->offset, xen_pt_emu_reg_grps[i].grp_type,
-                                i, ARRAY_SIZE(xen_pt_emu_reg_grps));
-                        error_propagate(errp, err);
+                    xen_pt_config_reg_init(s, reg_grp_entry, regs, errp);
+                    if (*errp) {
+                        error_append_hint(errp, "Failed to init register %d"
+                                          " offsets 0x%x in grp_type = 0x%x (%d/%zu)",
+                                          j,
+                                          regs->offset,
+                                          xen_pt_emu_reg_grps[i].grp_type,
+                                          i, ARRAY_SIZE(xen_pt_emu_reg_grps));
                         xen_pt_config_delete(s);
                         return;
                     }
Signature · 7 lines
2.21.0


_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
Reviewed-by on v8 10/10
#40 lore raw

Re: [PATCH v8 10/10] xen: introduce ERRP_AUTO_PROPAGATE

Quoted 25 lines · Vladimir Sementsov-Ogievskiy

If we want to add some info to errp (by error_prepend() or error_append_hint()), we must use the ERRP_AUTO_PROPAGATE macro. Otherwise, this info will not be added when errp == &error_fatal (the program will exit prior to the error_append_hint() or error_prepend() call). Fix such cases.

If we want to check error after errp-function call, we need to introduce local_err and then propagate it to errp. Instead, use ERRP_AUTO_PROPAGATE macro, benefits are:
1. No need of explicit error_propagate call
2. No need of explicit local_err variable: use errp directly
3. ERRP_AUTO_PROPAGATE leaves errp as is if it's not NULL or &error_fatal, this means that we don't break error_abort (we'll abort on error_set, not on error_propagate)

This commit is generated by command

    sed -n '/^X86 Xen CPUs$/,/^$/{s/^F: //p}' MAINTAINERS | \
    xargs git ls-files | grep '\.[hc]$' | \
    xargs spatch \
        --sp-file scripts/coccinelle/auto-propagated-errp.cocci \
        --macro-file scripts/cocci-macro-file.h \
        --in-place --no-show-diff --max-width 80
Quoted 3 lines

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

#41 lore raw

Re: [PATCH v8 10/10] xen: introduce ERRP_AUTO_PROPAGATE

Quoted 27 lines · Paul Durrant
Quoted 25 lines · Vladimir Sementsov-Ogievskiy

If we want to add some info to errp (by error_prepend() or error_append_hint()), we must use the ERRP_AUTO_PROPAGATE macro. Otherwise, this info will not be added when errp == &error_fatal (the program will exit prior to the error_append_hint() or error_prepend() call). Fix such cases.

If we want to check error after errp-function call, we need to introduce local_err and then propagate it to errp. Instead, use ERRP_AUTO_PROPAGATE macro, benefits are:
1. No need of explicit error_propagate call
2. No need of explicit local_err variable: use errp directly
3. ERRP_AUTO_PROPAGATE leaves errp as is if it's not NULL or &error_fatal, this means that we don't break error_abort (we'll abort on error_set, not on error_propagate)

This commit is generated by command

     sed -n '/^X86 Xen CPUs$/,/^$/{s/^F: //p}' MAINTAINERS | \
     xargs git ls-files | grep '\.[hc]$' | \
     xargs spatch \
         --sp-file scripts/coccinelle/auto-propagated-errp.cocci \
         --macro-file scripts/cocci-macro-file.h \
         --in-place --no-show-diff --max-width 80
Quoted 6 lines
Quoted 3 lines

Thanks for reviewing!

Signature · 7 lines
Best regards,
Vladimir

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
#42 lore raw
In reply to Vladimir Sementsov-Ogievskiy
Quoted 4 lines · Vladimir Sementsov-Ogievskiy

File with errp-cleaning APIs dropped for two reasons:

1. I'm tired after a 3-days war with coccinelle, and don't want to add more patches here.

Oww. In my experience, Coccinelle is both awesome and terrible. I hope you didn't do all that work just to address minor complaints from me. I *try* to make it clear where on the spectrum from "I want you to improve this" to "perhaps you'd like to try this idea" my review comments are, but I'm certainly not perfect there. Anyway, here we are, so let's look at the patches.

Quoted 18 lines

2. Markus noted, that we forget two more functions which needs such wrappers and corresponding conversion, so seems better to handle all these things in same manner for now.

changes in v8:

01: - update comments
- fix bug in macro [Markus]
- use do {} while(0)

02: a lot of changes
- about error propagation: try to update only patterns where we propagate local_error to errp. So, patches 09 and 10 changed (wow!)
- Now clearing functions are not defined. Still, do the conversion, so that when script applied where these functions needed, compilation will be broken.
- improve ordering of hunks, and comment everything

Quoted 1 line

09,10: drop hunks, which are converted for nothing.

Quick eye-over: you no longer wrap nbd_co_receive_cmdread_reply(), nbd_co_receive_blockstatus_reply(), and xen_bus_realize(). Clearly better.

Quoted 28 lines

v8 is available at
https://src.openvz.org/scm/~vsementsov/qemu.git #tag up-auto-local-err-partI-v8
v7 is available at
https://src.openvz.org/scm/~vsementsov/qemu.git #tag up-auto-local-err-partI-v7

In these series, there is no commit-per-subsystem script, each generated commit is generated in separate.

Still, generating commands are very similar, and looks like

    sed -n '/^<Subsystem name>$/,/^$/{s/^F: //p}' MAINTAINERS | \
    xargs git ls-files | grep '\.[hc]$' | \
    xargs spatch \
        --sp-file scripts/coccinelle/auto-propagated-errp.cocci \
        --macro-file scripts/cocci-macro-file.h \
        --in-place --no-show-diff --max-width 80

Note, that in each generated commit, generation command is the only text, indented by 8 spaces in 'git log -1' output, so, to regenerate all commits (for example, after rebase, or change in coccinelle script), you may use the following command:

git rebase -x "sh -c \"git show --pretty= --name-only | xargs git checkout HEAD^ -- ; git reset; git log -1 | grep '^ ' | sh\"" HEAD~7

Which will start automated interactive rebase for generated patches, which will stop if generated patch changed
(you may do git commit --amend to apply updated generated changes).

Quoted 7 lines

Note:
git show --pretty= --name-only - lists files, changed in HEAD
git log -1 | grep '^ ' | sh - rerun generation command of HEAD

Check for compilation of changed .c files
git rebase -x "sh -c \"git show --pretty= --name-only | sed -n 's/\.c$/.o/p' | xargs make -j9\"" HEAD~7

Thanks for the write-up!

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel