introduce QArray
v2 cover introduce QArray
Patches 1 and 2 introduce include/qemu/qarray.h which implements a deep auto free mechanism for arrays. Unlike GArray it does not require special macros, function calls or member dereferencing to access the individual array elements. So existing C-style array code can be retained with only very little changes.
In this initial version QArray only supports the concept of unique pointers, i.e. it does not support reference counting. The array (and all dynamically allocated memory of individual array elements) is auto freed once execution leaves the scope of the reference variable (unique pointer) associated with the array.
Patches 3..5 are provided (e.g. as example) for 9p being the first user of this new QArray API. These particular patches 3..5 are rebased on my current 9p queue: https://github.com/cschoenebeck/qemu/commits/9p.next
which are basically just the following two queued patches:
https://github.com/cschoenebeck/qemu/commit/7772715d43908235940f5b7dec68d0458b1ccdf4
https://github.com/cschoenebeck/qemu/commit/838b55e392ea7d52e714fdba1db777f658aee2cc
v1 -> v2:
* Minor API comment changes [patch 1].
* Perform strong type check by using __builtin_types_compatible_p() instead of a weak check using sizeof() [patch 2].
Christian Schoenebeck (5):
qemu/qarray.h: introduce QArray qemu/qarray.h: check scalar type in QARRAY_CREATE() 9pfs: make V9fsString usable via QArray API 9pfs: make V9fsPath usable via QArray API 9pfs: use QArray in v9fs_walk()
fsdev/9p-marshal.c | 2 + fsdev/9p-marshal.h | 3 + fsdev/file-op-9p.h | 2 + hw/9pfs/9p.c | 19 ++--- include/qemu/qarray.h | 156 ++++++++++++++++++++++++++++++++++++++++++ 5 files changed, 170 insertions(+), 12 deletions(-) create mode 100644 include/qemu/qarray.h
Signature · 1 line
2.20.1
PATCH v2 1/5 qemu/qarray.h: introduce QArray
Implements deep auto free of arrays while retaining common C-style squared bracket access.
--- include/qemu/qarray.h | 150 ++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 150 insertions(+) create mode 100644 include/qemu/qarray.h
include/qemu/qarray.h new file +150 -0
@@ -0,0 +1,150 @@ +/* +* QArray - deep auto free C-array +* +* Copyright (c) 2021 Crudebyte +* +* Authors: +* Christian Schoenebeck <[email protected]> +* +* Permission is hereby granted, free of charge, to any person obtaining a copy +* of this software and associated documentation files (the "Software"), to deal +* in the Software without restriction, including without limitation the rights +* to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +* copies of the Software, and to permit persons to whom the Software is +* furnished to do so, subject to the following conditions: +* +* The above copyright notice and this permission notice shall be included in +* all copies or substantial portions of the Software. +* +* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL +* THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +* LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +* OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN +* THE SOFTWARE. +*/ +#ifndef QEMU_QARRAY_H +#define QEMU_QARRAY_H + +/** +* QArray provides a mechanism to access arrays in common C-style (e.g. by +* square bracket [] operator) in conjunction with reference variables that +* perform deep auto free of the array when leaving the scope of the auto +* reference variable. That means not only is the array itself automatically +* freed, but also memory dynamically allocated by the individual array +* elements. +* +* Example: +* +* Consider the following user struct @c Foo which shall be used as scalar +* (element) type of an array: +* @code +* typedef struct Foo { +* int i; +* char *s; +* } Foo; +* @endcode +* and assume it has the following function to free memory allocated by @c Foo +* instances: +* @code +* void free_foo(Foo *foo) { +* free(foo->s); +* } +* @endcode +* Add the following to a shared header file: +* @code +* DECLARE_QARRAY_TYPE(Foo); +* @endcode +* and the following to a C unit file: +* @code +* DEFINE_QARRAY_TYPE(Foo, free_foo); +* @endcode +* Finally the array may then be used like this: +* @code +* void doSomething(int n) { +* QArrayRef(Foo) foos = NULL; +* QARRAY_CREATE(Foo, foos, n); +* for (size_t i = 0; i < n; ++i) { +* foos[i].i = i; +* foos[i].s = calloc(4096, 1); +* snprintf(foos[i].s, 4096, "foo %d", i); +* } +* } +* @endcode +*/ + +/** +* Declares an array type for the passed @a scalar_type. +* +* This is typically used from a shared header file. +* +* @param scalar_type - type of the individual array elements +*/ +#define DECLARE_QARRAY_TYPE (scalar_type) \ + typedef struct QArray##scalar_type { \ + size_t len; \ + scalar_type first[]; \ + } QArray##scalar_type; \ + \ + void qarray_create_##scalar_type(scalar_type **auto_var, size_t len); \ + void qarray_auto_free_##scalar_type(scalar_type **auto_var); \ + +/** +* Defines an array type for the passed @a scalar_type and appropriate +* @a scalar_cleanup_func. +* +* This is typically used from a C unit file. +* +* @param scalar_type - type of the individual array elements +* @param scalar_cleanup_func - appropriate function to free memory dynamically +* allocated by individual array elements before +*/ +#define DEFINE_QARRAY_TYPE (scalar_type, scalar_cleanup_func) \ + void qarray_create_##scalar_type(scalar_type **auto_var, size_t len) \ + { \ + qarray_auto_free_##scalar_type(auto_var); \ + QArray##scalar_type *arr = g_malloc0(sizeof(QArray##scalar_type) + \ + len * sizeof(scalar_type)); \ + arr->len = len; \ + *auto_var = &arr->first[0]; \ + } \ + \ + void qarray_auto_free_##scalar_type(scalar_type **auto_var) \ + { \ + scalar_type *first = (*auto_var); \ + if (!first) { \ + return; \ + } \ + QArray##scalar_type *arr = (QArray##scalar_type *) ( \ + ((char *)first) - offsetof(QArray##scalar_type, first) \ + ); \ + for (size_t i = 0; i < arr->len; ++i) { \ + scalar_cleanup_func(&arr->first[i]); \ + } \ + g_free(arr); \ + } \ + +/** +* Used to declare a reference variable (unique pointer) for an array. After +* leaving the scope of the reference variable, the associated array is +* automatically freed. +* +* @param scalar_type - type of the individual array elements +*/ +#define QArrayRef (scalar_type) \ + __attribute((__cleanup__(qarray_auto_free_##scalar_type))) scalar_type* + +/** +* Allocates a new array of passed @a scalar_type with @a len number of array +* elements and assigns the created array to the reference variable +* @a auto_var. +* +* @param scalar_type - type of the individual array elements +* @param auto_var - destination reference variable +* @param len - amount of array elements to be allocated immediately +*/ +#define QARRAY_CREATE (scalar_type, auto_var, len) \ + qarray_create_##scalar_type((&auto_var), len) + +#endif /* QEMU_QARRAY_H */
Signature · 1 line
2.20.1
PATCH v2 3/5 9pfs: make V9fsString usable via QArray API
In reply to Christian Schoenebeck--- fsdev/9p-marshal.c | 2 ++ fsdev/9p-marshal.h | 3 +++ 2 files changed, 5 insertions(+)
fsdev/9p-marshal.c +2 -0
@@ -18,6 +18,8 @@#include "9p-marshal.h" +DEFINE_QARRAY_TYPE (V9fsString, v9fs_string_free); +void v9fs_string_free (V9fsString *str) {g_free (str->data);
fsdev/9p-marshal.h +3 -0
@@ -1,10 +1,13 @@ #ifndef QEMU_9P_MARSHAL_H #define QEMU_9P_MARSHAL_H +#include "qemu/qarray.h" + typedef struct V9fsString { uint16_t size; char *data; } V9fsString; +DECLARE_QARRAY_TYPE(V9fsString); typedef struct V9fsQID { uint8_t type;
Signature · 1 line
2.20.1
PATCH v2 5/5 9pfs: use QArray in v9fs_walk()
In reply to Christian Schoenebeck--- hw/9pfs/9p.c | 17 +++++------------ 1 file changed, 5 insertions(+), 12 deletions(-)
hw/9pfs/9p.c +5 -12
@@ -1707,13 +1707,14 @@ static void coroutine_fn v9fs_walk(void *opaque)int name_idx; g_autofree V9fsQID *qids = NULL;int i, err =0 ; -V9fsPath dpath, path, *pathes =NULL ; +V9fsPath dpath, path; +QArrayRef (V9fsPath) pathes =NULL ;uint16_t nwnames;struct stat stbuf, fidst; g_autofree struct stat *stbufs = NULL;size_t offset =7 ;int32_t fid, newfid; -V9fsString *wnames =NULL ; +QArrayRef (V9fsString) wnames =NULL ;V9fsFidState *fidp;V9fsFidState *newfidp =NULL ;V9fsPDU *pdu = opaque;
@@ -1734,10 +1735,10 @@ static void coroutine_fn v9fs_walk(void *opaque) goto out_nofid; }if (nwnames) { - wnames =g_new0 (V9fsString, nwnames); +QARRAY_CREATE (V9fsString, wnames, nwnames); qids =g_new0 (V9fsQID, nwnames); stbufs = g_new0(struct stat, nwnames); - pathes =g_new0 (V9fsPath, nwnames); +QARRAY_CREATE (V9fsPath, pathes, nwnames);for (i =0 ; i < nwnames; i++) { err =pdu_unmarshal (pdu, offset,"s" , &wnames[i]);if (err <0 ) {
@@ -1869,14 +1870,6 @@ out:v9fs_path_free (&path); out_nofid:pdu_complete (pdu, err); -if (nwnames && nwnames <=P9_MAXWELEM ) { -for (name_idx =0 ; name_idx < nwnames; name_idx++) { -v9fs_string_free (&wnames[name_idx]); -v9fs_path_free (&pathes[name_idx]); - } -g_free (wnames); -g_free (pathes); - } } static int32_t coroutine_fn get_iounit(V9fsPDU *pdu, V9fsPath *path)
Signature · 1 line
2.20.1
PATCH v2 2/5 qemu/qarray.h: check scalar type in QARRAY_CREATE()
In reply to Christian SchoenebeckMake sure at compile time that the scalar type of the array requested to be created via QARRAY_CREATE() matches the scalar type of the passed auto reference variable (unique pointer).
--- include/qemu/qarray.h | 6 ++++++ 1 file changed, 6 insertions(+)
include/qemu/qarray.h +6 -0
@@ -27,6 +27,8 @@#ifndef QEMU_QARRAY_H #define QEMU_QARRAY_H +#include "qemu/compiler.h" +/** * QArray provides a mechanism to access arrays in common C-style (e.g. by * square bracket [] operator) in conjunction with reference variables that
@@ -145,6 +147,10 @@* @param len - amount of array elements to be allocated immediately */ #define QARRAY_CREATE (scalar_type, auto_var, len) \ + QEMU_BUILD_BUG_MSG( \ + !__builtin_types_compatible_p(scalar_type, typeof(*auto_var)), \ + "QArray scalar type mismatch" \ + ); \ qarray_create_##scalar_type((&auto_var), len)#endif /* QEMU_QARRAY_H */
Signature · 1 line
2.20.1
PATCH v2 4/5 9pfs: make V9fsPath usable via QArray API
In reply to Christian Schoenebeck--- fsdev/file-op-9p.h | 2 ++ hw/9pfs/9p.c | 2 ++ 2 files changed, 4 insertions(+)
fsdev/file-op-9p.h +2 -0
@@ -18,6 +18,7 @@#include <utime.h> #include <sys/vfs.h> #include "qemu-fsdev-throttle.h" +#include "qemu/qarray.h" #define SM_LOCAL_MODE_BITS 0600#define SM_LOCAL_DIR_MODE_BITS 0700
@@ -105,6 +106,7 @@ struct V9fsPath {uint16_t size;char *data; }; +DECLARE_QARRAY_TYPE (V9fsPath);typedef union V9fsFidOpenState V9fsFidOpenState ;
hw/9pfs/9p.c +2 -0
@@ -50,6 +50,8 @@ enum { Oappend = 0x80, }; +DEFINE_QARRAY_TYPE (V9fsPath, v9fs_path_free); +static ssize_t pdu_marshal (V9fsPDU *pdu,size_t offset,const char *fmt, ...) {ssize_t ret;
Signature · 1 line
2.20.1
Re: [PATCH v2 1/5] qemu/qarray.h: introduce QArray
In reply to Christian SchoenebeckQuoted 4 lines · Christian Schoenebeck
Implements deep auto free of arrays while retaining common C-style squared bracket access.
Signed-off-by Christian Schoenebeck <[email protected]>
You provide some motivation for this, but only in your cover letter:
Patches 1 and 2 introduce include/qemu/qarray.h which implements a deep auto free mechanism for arrays. Unlike GArray it does not require special macros, function calls or member dereferencing to access the individual array elements. So existing C-style array code can be retained with only very little changes.
In this initial version QArray only supports the concept of unique pointers, i.e. it does not support reference counting. The array (and all dynamically allocated memory of individual array elements) is auto freed once execution leaves the scope of the reference variable (unique pointer) associated with the array.
Please put it into the commit message, so it gets committed.
An example to illustrate how QArray is better than GArray (for some value of "better") would help make your case that QArray is worth its maintenance cost.
Quoted 27 lines
--- include/qemu/qarray.h | 150 ++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 150 insertions(+) create mode 100644 include/qemu/qarray.hinclude/qemu/qarray.h new file +16
-0@@ -0,0 +1,150 @@ +/* +* QArray - deep auto free C-array +* +* Copyright (c) 2021 Crudebyte +* +* Authors: +* Christian Schoenebeck <[email protected]> +* +* Permission is hereby granted, free of charge, to any person obtaining a copy +* of this software and associated documentation files (the "Software"), to deal +* in the Software without restriction, including without limitation the rights +* to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +* copies of the Software, and to permit persons to whom the Software is +* furnished to do so, subject to the following conditions: +* +* The above copyright notice and this permission notice shall be included in
include/qemu/qarray.h line 26
Quoted 10 lines
include/qemu/qarray.h new file +10
-0+* all copies or substantial portions of the Software. +* +* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL +* THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +* LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +* OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN +* THE SOFTWARE. +*/
Please use GPLv2+ unless you have a compelling reason not to.
[...]
Re: [PATCH v2 1/5] qemu/qarray.h: introduce QArray
Quoted 14 lines · On Dienstag, 24. August 2021 10:22:52 CEST Markus Armbruster wrote:
Quoted 4 lines · Christian Schoenebeck
Implements deep auto free of arrays while retaining common C-style squared bracket access.
Signed-off-by Christian Schoenebeck <[email protected]>You provide some motivation for this, but only in your cover letter:
Patches 1 and 2 introduce
include/qemu/qarray.hwhich implements a deep
auto free mechanism for arrays. Unlike GArray it does not require special macros, function calls or member dereferencing to access the individual array elements. So existing C-style array code can be retained with only very little changes.
Quoted 7 lines
In this initial version QArray only supports the concept of unique
pointers, i.e. it does not support reference counting. The array (and all dynamically allocated memory of individual array elements) is auto freed once execution leaves the scope of the reference variable (unique pointer) associated with the array.Please put it into the commit message, so it gets committed.
Sure, np.
Quoted 3 lines
An example to illustrate how QArray is better than GArray (for some value of "better") would help make your case that QArray is worth its maintenance cost.
Ok, I'll put that into the commit message as well, instead of into the API comments.
Quoted 43 lines
Quoted 32 lines
---
include/qemu/qarray.h | 150 ++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 150 insertions(+) create mode 100644 include/qemu/qarray.hinclude/qemu/qarray.h new file +9
-0@@ -0,0 +1,150 @@ +/* +* QArray - deep auto free C-array +* +* Copyright (c) 2021 Crudebyte +* +* Authors: +* Christian Schoenebeck <[email protected]> +* +* Permission is hereby granted, free of charge, to any person obtaining a copy + * of this software and associated documentation files (the "Software"), to deal + * in the Software without restriction, including without limitation the rights + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + * copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in + * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,Quoted 8 lines
EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL + * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN + * THE SOFTWARE.
+ */Please use GPLv2+ unless you have a compelling reason not to.
Quoted 1 line
[...]
Is that a requirement?
It is just my personal license preference. AFAICS there are numerous sources in QEMU released under MIT license as well.
Best regards,
Christian Schoenebeck
Re: [PATCH v2 1/5] qemu/qarray.h: introduce QArray
In reply to Christian SchoenebeckQuoted 87 lines · On Sonntag, 22. August 2021 15:16:46 CEST Christian Schoenebeck wrote:
Implements deep auto free of arrays while retaining common C-style squared bracket access.
Signed-off-by Christian Schoenebeck <[email protected]>--- include/qemu/qarray.h | 150 ++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 150 insertions(+) create mode 100644 include/qemu/qarray.hinclude/qemu/qarray.h new file +9
-0@@ -0,0 +1,150 @@ +/* +* QArray - deep auto free C-array +* +* Copyright (c) 2021 Crudebyte +* +* Authors: +* Christian Schoenebeck <[email protected]> +* +* Permission is hereby granted, free of charge, to any person obtaining a copy + * of this software and associated documentation files (the "Software"), to deal + * in the Software without restriction, including without limitation the rights + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + * copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in + * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL + * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN + * THE SOFTWARE.
+ */include/qemu/qarray.h new file +8
-0+#ifndef QEMU_QARRAY_H +#define QEMU_QARRAY_H + +/** +* QArray provides a mechanism to access arrays in common C-style (e.g. by +* square bracket [] operator) in conjunction with reference variables that +* perform deep auto free of the array when leaving the scope of the auto +* reference variable. That means not only is the array itself automatically + * freed, but also memory dynamically allocated by the individual array + * elements.
+ *
+ * Example:
+ *
+ * Consider the following user struct @c Foo which shall be used as scalar
+ * (element) type of an array:
+ * @code
+ * typedef struct Foo {
+ * int i;
+ * char *s;
+ * } Foo;
+ * @endcode
+ * and assume it has the following function to free memory allocated by @c Foo + * instances:
+ * @code
+ * voidfree_foo(Foo *foo){
+ *free(foo->s);
+ * }
+ * @endcode
+ * Add the following to a shared header file:
+ * @code
+ *DECLARE_QARRAY_TYPE(Foo);
+ * @endcode
+ * and the following to a C unit file:
+ * @code
+ *DEFINE_QARRAY_TYPE(Foo, free_foo);
+ * @endcode
+ * Finally the array may then be used like this:
+ * @code
+ * void doSomething(int n) {+ * QArrayRef(Foo) foos = NULL; + * QARRAY_CREATE(Foo, foos, n); + * for (size_t i = 0; i < n; ++i) { + * foos[i].i = i; + * foos[i].s = calloc(4096, 1); + * snprintf(foos[i].s, 4096, "foo %d", i);
Quoted 3 lines
+ * }
+ * }
+ * @endcode
Or should that probably be changed to upper case QArrayRef() -> QARRAY_REF(), because ...
Quoted 59 lines
+ */
include/qemu/qarray.h new file +24
-0+ +/** +* Declares an array type for the passed @a scalar_type. +* +* This is typically used from a shared header file. +* +* @param scalar_type - type of the individual array elements +*/ +#define DECLARE_QARRAY_TYPE (scalar_type) \ + typedef struct QArray##scalar_type { \ + size_t len; \ + scalar_type first[]; \ + } QArray##scalar_type; \ + \ + void qarray_create_##scalar_type(scalar_type **auto_var, size_t len); \ + void qarray_auto_free_##scalar_type(scalar_type **auto_var); \ + +/** +* Defines an array type for the passed @a scalar_type and appropriate +* @a scalar_cleanup_func. +* +* This is typically used from a C unit file. +* +* @param scalar_type - type of the individual array elements +* @param scalar_cleanup_func - appropriate function to free memory dynamically + * allocated by individual array
elements before + */include/qemu/qarray.h new file +5
-0+#define DEFINE_QARRAY_TYPE (scalar_type, scalar_cleanup_func) \ + void qarray_create_##scalar_type(scalar_type **auto_var, size_t len) \ + { \ + qarray_auto_free_##scalar_type(auto_var); \ + QArray##scalar_type *arr = g_malloc0(sizeof(QArray##scalar_type) +\ + len *
sizeof(scalar_type)); \include/qemu/qarray.h new file +21
-0+ arr->len = len; \ + *auto_var = &arr->first[0 ]; \ + } \ + \ + void qarray_auto_free_##scalar_type(scalar_type **auto_var) \ + { \ + scalar_type *first = (*auto_var); \ + if (!first) { \ + return; \ + } \ + QArray##scalar_type *arr = (QArray##scalar_type *) ( \ + ((char *)first) - offsetof(QArray##scalar_type, first) \ + ); \ + for (size_t i = 0; i < arr->len; ++i) { \ + scalar_cleanup_func(&arr->first[i]); \ + } \ + g_free(arr); \ + } \ + +/** +* Used to declare a reference variable (unique pointer) for an array. After + * leaving the scope of the reference variable, the associated array is + * automatically freed.
+ *
+ * @paramscalar_type- type of the individual array elements
+ */
include/qemu/qarray.h line 137
Quoted 3 lines
include/qemu/qarray.h new file +3
-0+#define QArrayRef (scalar_type) \ + __attribute((__cleanup__(qarray_auto_free_##scalar_type))) scalar_type* +
... it is actually a macro?
include/qemu/qarray.h line 150
Quoted 13 lines
+/**
+ * Allocates a new array of passed @ascalar_typewith @a len number of array + * elements and assigns the created array to the reference variable
+ * @aauto_var.
+ *
+ * @paramscalar_type- type of the individual array elements
+ * @paramauto_var- destination reference variable
+ * @param len - amount of array elements to be allocated immediately
+ */include/qemu/qarray.h new file +4
-0+#define QARRAY_CREATE (scalar_type, auto_var, len) \ + qarray_create_##scalar_type((&auto_var), len) + +#endif /* QEMU_QARRAY_H */
Best regards,
Christian Schoenebeck
Re: [PATCH v2 1/5] qemu/qarray.h: introduce QArray
Quoted 16 lines · Christian Schoenebeck
Quoted 15 lines · On Sonntag, 22. August 2021 15:16:46 CEST Christian Schoenebeck wrote:
Implements deep auto free of arrays while retaining common C-style squared bracket access.
Signed-off-by Christian Schoenebeck <[email protected]>--- include/qemu/qarray.h | 150 ++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 150 insertions(+) create mode 100644 include/qemu/qarray.hinclude/qemu/qarray.h new file +0
-0@@ -0,0 +1,150 @@
[...]
Quoted 40 lines
Quoted 32 lines
+ * Consider the following user struct @c Foo which shall be used as scalar
+ * (element) type of an array:
+ * @code
+ * typedef struct Foo {
+ * int i;
+ * char *s;
+ * } Foo;
+ * @endcode
+ * and assume it has the following function to free memory allocated by @c Foo
+ * instances:
+ * @code
+ * voidfree_foo(Foo *foo){
+ *free(foo->s);
+ * }
+ * @endcode
+ * Add the following to a shared header file:
+ * @code
+ *DECLARE_QARRAY_TYPE(Foo);
+ * @endcode
+ * and the following to a C unit file:
+ * @code
+ *DEFINE_QARRAY_TYPE(Foo, free_foo);
+ * @endcode
+ * Finally the array may then be used like this:
+ * @code
+ * void doSomething(int n) {+ * QArrayRef(Foo) foos = NULL; + * QARRAY_CREATE(Foo, foos, n); + * for (size_t i = 0; i < n; ++i) { + * foos[i].i = i; + * foos[i].s = calloc(4096, 1); + * snprintf(foos[i].s, 4096, "foo %d", i);Quoted 3 lines
+ * }
+ * }
+ * @endcodeOr should that probably be changed to upper case
QArrayRef()->QARRAY_REF(), because ...Quoted 1 line
+ */
[...]
Quoted 4 lines
Quoted 4 lines
+/**
+ * Used to declare a reference variable (unique pointer) for an array. After
+ * leaving the scope of the reference variable, the associated array is
+ * automatically freed.
Quoted 8 lines
include/qemu/qarray.h line 137
Quoted 6 lines
+ *
+ * @paramscalar_type- type of the individual array elements
+ */include/qemu/qarray.h new file +3
-0+#define QArrayRef (scalar_type) \ + __attribute((__cleanup__(qarray_auto_free_##scalar_type))) scalar_type* +... it is actually a macro?
I'd change it.
[...]
Re: [PATCH v2 1/5] qemu/qarray.h: introduce QArray
In reply to Christian SchoenebeckQuoted 1 line · Christian Schoenebeck
On Dienstag, 24. August 2021 10:22:52 CEST Markus Armbruster wrote:
[...]
Quoted 8 lines
Quoted 3 lines
Please use GPLv2+ unless you have a compelling reason not to.
[...]
Is that a requirement?
It is just my personal license preference. AFAICS there are numerous sources in QEMU released under MIT license as well.
The licensing situation is a mess.
The only hard requirement is "compatible with GPLv2+". We prefer GPLv2+ for new code, except as detailed in ./LICENSE. We're stuck with a sizable body of existing code that is GPLv2 (no +), but decided to put limits to that madness. Again, see ./LICENSE for details.
I'm asking you to help with limiting the madness by sticking to GPLv2+ whenever possible.
Re: [PATCH v2 1/5] qemu/qarray.h: introduce QArray
Quoted 15 lines · On Dienstag, 24. August 2021 16:45:12 CEST Markus Armbruster wrote:
Quoted 1 line · Christian Schoenebeck
On Dienstag, 24. August 2021 10:22:52 CEST Markus Armbruster wrote:
[...]
Quoted 8 lines
Quoted 3 lines
Please use GPLv2+ unless you have a compelling reason not to.
[...]
Is that a requirement?
It is just my personal license preference. AFAICS there are numerous sources in QEMU released under MIT license as well.
The licensing situation is a mess.
Quoted 7 lines
The only hard requirement is "compatible with GPLv2+". We prefer GPLv2+ for new code, except as detailed in ./LICENSE. We're stuck with a sizable body of existing code that is GPLv2 (no +), but decided to put limits to that madness. Again, see ./LICENSE for details.
I'm asking you to help with limiting the madness by sticking to GPLv2+ whenever possible.
Okay, I see that there is quite a homogenous license structure in Qemu. However the MIT license is a very permissive license, so I don't see any conflicts.
What if I release this file under public domain? That's not even copyleft at all. What that be OK for you?
My idea was that people might simply take this header file and use it in other C projects as well. Putting it under GPL might cause conflicts for other projects.
Best regards,
Christian Schoenebeck
Re: [PATCH v2 1/5] qemu/qarray.h: introduce QArray
Quoted 21 lines · On Dienstag, 24. August 2021 17:24:50 CEST Christian Schoenebeck wrote:
Quoted 20 lines · On Dienstag, 24. August 2021 16:45:12 CEST Markus Armbruster wrote:
Quoted 1 line · Christian Schoenebeck
On Dienstag, 24. August 2021 10:22:52 CEST Markus Armbruster wrote:
[...]
Quoted 8 lines
Quoted 3 lines
Please use GPLv2+ unless you have a compelling reason not to.
[...]
Is that a requirement?
It is just my personal license preference. AFAICS there are numerous sources in QEMU released under MIT license as well.
The licensing situation is a mess.
The only hard requirement is "compatible with GPLv2+". We prefer GPLv2+ for new code, except as detailed in ./LICENSE. We're stuck with a sizable body of existing code that is GPLv2 (no +), but decided to put limits to that madness. Again, see ./LICENSE for details.
Quoted 6 lines
Quoted 2 lines
I'm asking you to help with limiting the madness by sticking to GPLv2+ whenever possible.
Okay, I see that there is quite a homogenous license structure in Qemu. However the MIT license is a very permissive license, so I don't see any conflicts.
s/homogenous/heterogeneous/
Quoted 2 lines
What if I release this file under public domain? That's not even copyleft at all. What that be OK for you?
"Would" that be OK for you?
Quoted 3 lines
My idea was that people might simply take this header file and use it in other C projects as well. Putting it under GPL might cause conflicts for other projects.
Best regards,
Christian Schoenebeck
Re: [PATCH v2 1/5] qemu/qarray.h: introduce QArray
Quoted 22 lines · Christian Schoenebeck
Quoted 21 lines · On Dienstag, 24. August 2021 17:24:50 CEST Christian Schoenebeck wrote:
Quoted 20 lines · On Dienstag, 24. August 2021 16:45:12 CEST Markus Armbruster wrote:
Quoted 1 line · Christian Schoenebeck
On Dienstag, 24. August 2021 10:22:52 CEST Markus Armbruster wrote:
[...]
Quoted 8 lines
Quoted 3 lines
Please use GPLv2+ unless you have a compelling reason not to.
[...]
Is that a requirement?
It is just my personal license preference. AFAICS there are numerous sources in QEMU released under MIT license as well.
The licensing situation is a mess.
The only hard requirement is "compatible with GPLv2+". We prefer GPLv2+ for new code, except as detailed in ./LICENSE. We're stuck with a sizable body of existing code that is GPLv2 (no +), but decided to put limits to that madness. Again, see ./LICENSE for details.
Quoted 4 lines
Quoted 4 lines
Quoted 2 lines
I'm asking you to help with limiting the madness by sticking to GPLv2+ whenever possible.
Okay, I see that there is quite a homogenous license structure in Qemu.
Self-inflicted wound. We should have insisted on GPLv2+.
Quoted 9 lines
Quoted 2 lines
However the MIT license is a very permissive license, so I don't see any conflicts.
s/homogenous/heterogeneous/
Quoted 2 lines
What if I release this file under public domain? That's not even copyleft at all. What that be OK for you?
"Would" that be OK for you?
My preference: GPLv2+ > MIT > public domain.
If you go with anything but GPLv2+, please explain why in your commit message. One sentence should suffice, say "MIT license to minimize license issues when "stealing" this code for other projects."
Quoted 3 lines
Quoted 3 lines
My idea was that people might simply take this header file and use it in other C projects as well. Putting it under GPL might cause conflicts for other projects.
Re: [PATCH v2 1/5] qemu/qarray.h: introduce QArray
In reply to Christian SchoenebeckQuoted 53 lines · Christian Schoenebeck
Implements deep auto free of arrays while retaining common C-style squared bracket access.
Signed-off-by Christian Schoenebeck <[email protected]>--- include/qemu/qarray.h | 150 ++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 150 insertions(+) create mode 100644 include/qemu/qarray.hinclude/qemu/qarray.h new file +37
-0@@ -0,0 +1,150 @@ +#ifndef QEMU_QARRAY_H +#define QEMU_QARRAY_H + +/** +* QArray provides a mechanism to access arrays in common C-style (e.g. by +* square bracket [] operator) in conjunction with reference variables that +* perform deep auto free of the array when leaving the scope of the auto +* reference variable. That means not only is the array itself automatically +* freed, but also memory dynamically allocated by the individual array +* elements. +* +* Example: +* +* Consider the following user struct @c Foo which shall be used as scalar +* (element) type of an array: +* @code +* typedef struct Foo { +* int i; +* char *s; +* } Foo; +* @endcode +* and assume it has the following function to free memory allocated by @c Foo +* instances: +* @code +* void free_foo(Foo *foo) { +* free(foo->s); +* } +* @endcode +* Add the following to a shared header file: +* @code +* DECLARE_QARRAY_TYPE(Foo); +* @endcode +* and the following to a C unit file: +* @code +* DEFINE_QARRAY_TYPE(Foo, free_foo); +* @endcode +* Finally the array may then be used like this:
include/qemu/qarray.h line 73
Quoted 10 lines
include/qemu/qarray.h new file +10
-0+* @code +* void doSomething(int n) { +* QArrayRef(Foo) foos = NULL; +* QARRAY_CREATE(Foo, foos, n); +* for (size_t i = 0; i < n; ++i) { +* foos[i].i = i; +* foos[i].s = calloc(4096, 1); +* snprintf(foos[i].s, 4096, "foo %d", i); +* } +* }
So essentially here the 'foos' variable is still a plain C array from POV of the caller ie it is a 'Foo *foos'
By QARRAY_CREATE() is actually allocating a 'QArrayFoo' which is a struct containing a 'size_t len' along with the 'Foo *foos' entry.
This is a clever trick, and it works in the example above, because the code already has the length available in a convenient way via the 'int n' parameter.
IMHO this makes the code less useful than it otherwise would be in the general case, because there are places where the code needs to pass around the array and its assoicated length, and so this with this magic hidden length, we're still left to pass around the length separately.
Consider this example:
int open_conn(const char *hostname) {
SocketAddress *addrs;
size_t naddrs;
int ret = -1;
size_t i; if (resolve_hostname(hostname, &addrs, &naddrs) < 0)
return -1;
for (i = 0; i < naddrs; i++) {
...try to connect to addrs[i]...
}
ret = 0
cleanup:
for (i = 0; i < naddrs; i++) {
qapi_free_SocketAddress(addrs[i])
}
return ret;}
To simplify this it is deisrable to autofree the 'addrs' array.
If using QArray, it still has to keep passing around the 'size_t naddrs' value because QArray hides the length field from the code.
int open_conn(const char *hostname) {
QArrayRef(SocketAddress) addrs = NULL;
size_t naddrs;
int ret = -1;
size_t i; if (resolve_hostname(hostname, &addrs, &naddrs) < 0)
return -1;
for (i = 0; i < naddrs; i++) {
...try to connect to addrs[i]...
}
ret = 0
cleanup:
return ret;}
If it instead just exposed the array struct explicitly, it can use the normal g_autoptr() declarator, and can also now just return the array directly since it is a single pointer
int open_conn(const char *hostname) {
g_autoptr(SocketAddressArray) addrs = NULL;
int ret = -1;
size_t i; if (!(addrs = resolve_hostname(hostname)))
return -1;
for (i = 0; i < addrs.len; i++) {
...try to connect to addrs.data[i]...
}
ret = 0
cleanup:
return ret;}
In terms of the first example, it adds an indirection to access the array data, but on the plus side IMHO the code is clearer because it uses 'g_autoptr' which is what is more in line with what is expected for variables that are automatically freed. QArrayRef() as a name doesn't make it clear that the value will be freed.
void doSomething(int n) {
g_autoptr(FooArray) foos = NULL;
QARRAY_CREATE(Foo, foos, n);
for (size_t i = 0; i < foos.len; ++i) {
foos.data[i].i = i;
foos.data[i].s = calloc(4096, 1);
snprintf(foos.data[i].s, 4096, "foo %d", i);
}
}I would also suggest that QARRAY_CREATE doesn't need to exist as a macro - callers could just use the allocator function directly for clearer code, if it was changed to return the ptr rather than use an out parameter:
void doSomething(int n) {
g_autoptr(FooArray) foos = foo_array_new(n);
for (size_t i = 0; i < foos.len; ++i) {
foos.data[i].i = i;
foos.data[i].s = calloc(4096, 1);
snprintf(foos.data[i].s, 4096, "foo %d", i);
}
}For this it needs to pass 2 args into the DECLARE_QARRAY_TYPE macro - the struct name and desired method name - basically the method name is the struct name in lowercase with underscores.
Overall I think the goal of having an convenient sized array for types is good, but I think we should make it look a bit less magic. I think we only need the DECLARE_QARRAY_TYPE and DEFINE_QARRAY_TYPE macros.
Incidentally, I'd suggest naming to be QARRAY_DECLARE_TYPE and QARRAY_DEFINE_TYPE.
Quoted 81 lines
+ * @endcode
+ */include/qemu/qarray.h new file +75
-0+ +/** +* Declares an array type for the passed @a scalar_type. +* +* This is typically used from a shared header file. +* +* @param scalar_type - type of the individual array elements +*/ +#define DECLARE_QARRAY_TYPE (scalar_type) \ + typedef struct QArray##scalar_type { \ + size_t len; \ + scalar_type first[]; \ + } QArray##scalar_type; \ + \ + void qarray_create_##scalar_type(scalar_type **auto_var, size_t len); \ + void qarray_auto_free_##scalar_type(scalar_type **auto_var); \ + +/** +* Defines an array type for the passed @a scalar_type and appropriate +* @a scalar_cleanup_func. +* +* This is typically used from a C unit file. +* +* @param scalar_type - type of the individual array elements +* @param scalar_cleanup_func - appropriate function to free memory dynamically +* allocated by individual array elements before +*/ +#define DEFINE_QARRAY_TYPE (scalar_type, scalar_cleanup_func) \ + void qarray_create_##scalar_type(scalar_type **auto_var, size_t len) \ + { \ + qarray_auto_free_##scalar_type(auto_var); \ + QArray##scalar_type *arr = g_malloc0(sizeof(QArray##scalar_type) + \ + len * sizeof(scalar_type)); \ + arr->len = len; \ + *auto_var = &arr->first[0]; \ + } \ + \ + void qarray_auto_free_##scalar_type(scalar_type **auto_var) \ + { \ + scalar_type *first = (*auto_var); \ + if (!first) { \ + return; \ + } \ + QArray##scalar_type *arr = (QArray##scalar_type *) ( \ + ((char *)first) - offsetof(QArray##scalar_type, first) \ + ); \ + for (size_t i = 0; i < arr->len; ++i) { \ + scalar_cleanup_func(&arr->first[i]); \ + } \ + g_free(arr); \ + } \ + +/** +* Used to declare a reference variable (unique pointer) for an array. After +* leaving the scope of the reference variable, the associated array is +* automatically freed. +* +* @param scalar_type - type of the individual array elements +*/ +#define QArrayRef (scalar_type) \ + __attribute((__cleanup__(qarray_auto_free_##scalar_type))) scalar_type* + +/** +* Allocates a new array of passed @a scalar_type with @a len number of array +* elements and assigns the created array to the reference variable +* @a auto_var. +* +* @param scalar_type - type of the individual array elements +* @param auto_var - destination reference variable +* @param len - amount of array elements to be allocated immediately +*/ +#define QARRAY_CREATE (scalar_type, auto_var, len) \ + qarray_create_##scalar_type((&auto_var), len) + +#endif /* QEMU_QARRAY_H */ Signature · 1 line
2.20.1
Regards,
Daniel
Signature · 3 lines
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
Re: [PATCH v2 1/5] qemu/qarray.h: introduce QArray
Quoted 62 lines · On Dienstag, 28. September 2021 15:04:36 CEST Daniel P. Berrangé wrote:
Quoted 61 lines · Christian Schoenebeck
Implements deep auto free of arrays while retaining common C-style squared bracket access.
Signed-off-by Christian Schoenebeck <[email protected]>---
include/qemu/qarray.h | 150 ++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 150 insertions(+) create mode 100644 include/qemu/qarray.hinclude/qemu/qarray.h new file +5
-0@@ -0,0 +1,150 @@ +#ifndef QEMU_QARRAY_H +#define QEMU_QARRAY_H + +/** +* QArray provides a mechanism to access arrays in common C-style (e.g. by
+ * square bracket [] operator) in conjunction with reference variables that + * perform deep auto free of the array when leaving the scope of the auto + * reference variable. That means not only is the array itself automatically + * freed, but also memory dynamically allocated by the individual array + * elements.
+ *
+ * Example:
+ *
+ * Consider the following user struct @c Foo which shall be used as scalar
+ * (element) type of an array:
+ * @code
+ * typedef struct Foo {
+ * int i;
+ * char *s;
+ * } Foo;
+ * @endcode
+ * and assume it has the following function to free memory allocated by @c Foo + * instances:
+ * @code
+ * voidfree_foo(Foo *foo){
+ *free(foo->s);
+ * }
+ * @endcode
+ * Add the following to a shared header file:
+ * @code
+ *DECLARE_QARRAY_TYPE(Foo);
+ * @endcode
+ * and the following to a C unit file:
+ * @code
+ *DEFINE_QARRAY_TYPE(Foo, free_foo);
+ * @endcode
+ * Finally the array may then be used like this:
+ * @code
+ * void doSomething(int n) {+ * QArrayRef(Foo) foos = NULL; + * QARRAY_CREATE(Foo, foos, n); + * for (size_t i = 0; i < n; ++i) {
Quoted 8 lines
Quoted 5 lines
+ * foos[i].i = i; + * foos[i].s = calloc(4096, 1); + * snprintf(foos[i].s, 4096, "foo %d", i);+ * }
+ * }So essentially here the 'foos' variable is still a plain C array from POV of the caller ie it is a 'Foo *foos'
Yes, that's the main feature probably: getting rid of any (n times) manually written deep-free code branches while at the same time keeping the burden low, i.e. requiring to refactor as little existing code as possible, as it is still a "normal" C-array from user code perspective, just with meta info prepended in front of the C-array that the user code however does not need to care about at all:
Contiguous Memory Space ->
----------------------------------------------------------
| Array-Meta-Info | Scalar 0 | Scalar 1 | ... | Scalar n |
----------------------------------------------------------
^- QArrayFoo* ^- Foo*
So switching between the two is just +- const_offset math.
Plus this has the positive side effect that a C-style array code is better to the human eye.
For now that "Array-Meta-Info" is really just the array size, in future this might be extended to contain a reference count for instance. And as this concept is actually generic code (a.k.a template code), you could also extend this in a way where you just pull in features that you really need in user code. I.e. if you don't need an atomic reference count at one place (and avoid the sync overhead), don't enable it for your use case. If you need it somewhere else, then enable it there.
Quoted 35 lines
By
QARRAY_CREATE()is actually allocating a 'QArrayFoo' which is a struct containing a 'size_tlen' along with the 'Foo *foos' entry.This is a clever trick, and it works in the example above, because the code already has the length available in a convenient way via the 'int n' parameter.
IMHO this makes the code less useful than it otherwise would be in the general case, because there are places where the code needs to pass around the array and its assoicated length, and so this with this magic hidden length, we're still left to pass around the length separately.
Consider this example:
int open_conn(const char *hostname) { SocketAddress *addrs; size_t naddrs; int ret = -1; size_t i;if (resolve_hostname(hostname, &addrs, &naddrs) < 0) return -1; for (i = 0; i < naddrs; i++) { ...try to connect to addrs[i]... } ret = 0 cleanup: for (i = 0; i < naddrs; i++) { qapi_free_SocketAddress(addrs[i]) } return ret;
Quoted 8 lines
}
To simplify this it is deisrable to autofree the 'addrs' array.
If using QArray, it still has to keep passing around the '
size_tnaddrs' value because QArray hides the length field from the code.
Well no, you don't need to pass around anything, as the array length is always accessible; it is always just (compile time) constant -wordsize (-padding) offset away from the C-array pointer. Maybe the phrasing "private" was a bit misleading in the QArray.h comments.
It is correct that my 9p use case so far did not need the array length info by means of accessing an API, for that reason I really just ommitted (yet) to add a separate patch for that. All it would take was extending QArray.h in a way like (roughly):
typedef struct _QArrayGeneric {
size_t len;
char first[];
} _QArrayGeneric;/**
* Returns the amount of scalar elements in the passed array. *
* @param first - start of array
*/size_t qarray_len(void* first)
{
if (!first) {
return 0;
}
_QArrayGeneric *arr = (_QArrayGeneric *) (
((char *)first) - offsetof(_QArrayGeneric, first)
);
return arr->len;
}#define QARRAY_LEN(arr) qarray_len(arr)
And as this is generic code for all array scalar types, it would probably be partly placed in a separate qarray.c file.
After that change your user example would become:
for (i = 0; i < QARRAY_LEN(addrs); i++) {
...try to connect to addrs[i]...
}If you want I can post a v3 with a formal patch (or two) to handle that array length API.
Quoted 30 lines
If it instead just exposed the array struct explicitly, it can use the normal
g_autoptr()declarator, and can also now just return the array directly since it is a single pointerint open_conn(const char *hostname) { g_autoptr(SocketAddressArray) addrs = NULL; int ret = -1; size_t i;if (!(addrs = resolve_hostname(hostname))) return -1; for (i = 0; i < addrs.len; i++) { ...try to connect to addrs.data[i]... } ret = 0 cleanup: return ret;}
In terms of the first example, it adds an indirection to access the array data, but on the plus side IMHO the code is clearer because it uses '
g_autoptr' which is what is more in line with what is expected for variables that are automatically freed.QArrayRef()as a name doesn't make it clear that the value will be freed.
Quoted 9 lines
void doSomething(int n) {
g_autoptr(FooArray) foos = NULL;
QARRAY_CREATE(Foo, foos, n);
for (size_t i = 0; i < foos.len; ++i) {
foos.data[i].i = i;
foos.data[i].s = calloc(4096, 1);
snprintf(foos.data[i].s, 4096, "foo %d", i);
}
}Well, that would destroy the intended major feature "as little refactoring as possible". The amount of locations where you define a reference variable is usually much smaller than the amount of code locations where you actually access arrays.
Personally I would not mix in this case macros of foreign libraries (glib) with macros of a local framework (QArray), because if for some reason one of the two deviate in future in a certain way, you would need to refactor a whole bunch of user code. By just separating those definitions from day one, you can avoid such future refactoring work right from the start.
As far as the terminology is concerned: probably a matter of taste. For me a "reference" implies (either unique or shared) ownership, a "pointer" IMO doesn't. And the usage of QArray alone makes it clear that an array without any references gets automatically freed.
Quoted 14 lines
I would also suggest that
QARRAY_CREATEdoesn't need to exist as a macro - callers could just use the allocator function directly for clearer code, if it was changed to return the ptr rather than use an out parameter:void doSomething(int n) { g_autoptr(FooArray) foos = foo_array_new(n); for (size_t i = 0; i < foos.len; ++i) { foos.data[i].i = i; foos.data[i].s = calloc(4096, 1); snprintf(foos.data[i].s, 4096, "foo %d", i); } }
Quoted 3 lines
For this it needs to pass 2 args into the
DECLARE_QARRAY_TYPEmacro - the struct name and desired method name - basically the method name is the struct name in lowercase with underscores.
As you can see with patch 2, one of the movations of making this a macro was the intention to increase strictness of type safety, e.g to make things like:
void *p; ... QARRAY_CREATE(FooType, p, n);
to raise a compiler error immediately, but that's not all ...
Quoted 4 lines
Overall I think the goal of having an convenient sized array for types is good, but I think we should make it look a bit less magic. I think we only need the
DECLARE_QARRAY_TYPEandDEFINE_QARRAY_TYPEmacros.
... actually making it appear anything like magic was not my intention. The actual main reason for wrapping these things into macros is because that's actually the only way to write generic code in C. Especially in larger projects like this one I favour clear separation of API ("how to use it") from its actual implementation ("how does it do it exactly").
So if you use macros for all those things from the beginning, it is far less likely that you will need to refactor a huge amount of user code with future changes of this array framework.
Quoted 2 lines
Incidentally, I'd suggest naming to be
QARRAY_DECLARE_TYPEandQARRAY_DEFINE_TYPE.
Also a matter of taste I guess. The suggested naming DECLARE_QARRAY_TYPE() and DEFINE_QARRAY_TYPE() reflect more natural language IMO.
Quoted 80 lines
Quoted 73 lines
+ * @endcode
+ */include/qemu/qarray.h new file +15
-0+ +/** +* Declares an array type for the passed @a scalar_type. +* +* This is typically used from a shared header file. +* +* @param scalar_type - type of the individual array elements +*/ +#define DECLARE_QARRAY_TYPE (scalar_type) \ + typedef struct QArray##scalar_type { \ + size_t len; \ + scalar_type first[]; \ + } QArray##scalar_type; \ + \ + void qarray_create_##scalar_type(scalar_type **auto_var, size_t len);\ + void qarray_auto_free_##scalar_type(scalar_type **auto_var); \ +
+/**
+ * Defines an array type for the passed @ascalar_typeand appropriate
+ * @ascalar_cleanup_func.
+ *
+ * This is typically used from a C unit file.
+ *
+ * @paramscalar_type- type of the individual array elements
+ * @paramscalar_cleanup_func- appropriate function to free memory
dynamically + * allocated by individual
array elements before + */include/qemu/qarray.h new file +2
-0+#define DEFINE_QARRAY_TYPE (scalar_type, scalar_cleanup_func) \ + void qarray_create_##scalar_type(scalar_type **auto_var, size_t len)\
include/qemu/qarray.h new file +25
-0+ { \ + qarray_auto_free_##scalar_type(auto_var); \ + QArray##scalar_type *arr = g_malloc0(sizeof(QArray##scalar_type) + \ + len * sizeof(scalar_type)); \ + arr->len = len; \ + *auto_var = &arr->first[0 ]; \ + } \ + \ + void qarray_auto_free_##scalar_type(scalar_type **auto_var) \ + { \ + scalar_type *first = (*auto_var); \ + if (!first) { \ + return; \ + } \ + QArray##scalar_type *arr = (QArray##scalar_type *) ( \ + ((char *)first) - offsetof(QArray##scalar_type, first) \ + ); \ + for (size_t i = 0; i < arr->len; ++i) { \ + scalar_cleanup_func(&arr->first[i]); \ + } \ + g_free(arr); \ + } \ + +/** +* Used to declare a reference variable (unique pointer) for an array. After + * leaving the scope of the reference variable, the associated array is + * automatically freed.
+ *
+ * @paramscalar_type- type of the individual array elements
+ */include/qemu/qarray.h new file +2
-0+#define QArrayRef (scalar_type) \ + __attribute((__cleanup__(qarray_auto_free_##scalar_type)))scalar_type* +
+/**
+ * Allocates a new array of passed @ascalar_typewith @a len number of array + * elements and assigns the created array to the reference variable + * @aauto_var.
+ *
+ * @paramscalar_type- type of the individual array elements
+ * @paramauto_var- destination reference variable
+ * @param len - amount of array elements to be allocated immediately
+ */include/qemu/qarray.h line 150
Quoted 4 lines
include/qemu/qarray.h new file +4
-0+#define QARRAY_CREATE (scalar_type, auto_var, len) \ + qarray_create_##scalar_type((&auto_var), len) + +#endif /* QEMU_QARRAY_H */ Regards,
Daniel
Re: [PATCH v2 1/5] qemu/qarray.h: introduce QArray
Quoted 179 lines · Christian Schoenebeck
Quoted 62 lines · On Dienstag, 28. September 2021 15:04:36 CEST Daniel P. Berrangé wrote:
Quoted 61 lines · Christian Schoenebeck
Implements deep auto free of arrays while retaining common C-style squared bracket access.
Signed-off-by Christian Schoenebeck <[email protected]>---
include/qemu/qarray.h | 150 ++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 150 insertions(+) create mode 100644 include/qemu/qarray.hinclude/qemu/qarray.h new file +5
-0@@ -0,0 +1,150 @@ +#ifndef QEMU_QARRAY_H +#define QEMU_QARRAY_H + +/** +* QArray provides a mechanism to access arrays in common C-style (e.g. by
+ * square bracket [] operator) in conjunction with reference variables that + * perform deep auto free of the array when leaving the scope of the auto + * reference variable. That means not only is the array itself automatically + * freed, but also memory dynamically allocated by the individual array + * elements.
+ *
+ * Example:
+ *
+ * Consider the following user struct @c Foo which shall be used as scalar
+ * (element) type of an array:
+ * @code
+ * typedef struct Foo {
+ * int i;
+ * char *s;
+ * } Foo;
+ * @endcode
+ * and assume it has the following function to free memory allocated by @c Foo + * instances:
+ * @code
+ * voidfree_foo(Foo *foo){
+ *free(foo->s);
+ * }
+ * @endcode
+ * Add the following to a shared header file:
+ * @code
+ *DECLARE_QARRAY_TYPE(Foo);
+ * @endcode
+ * and the following to a C unit file:
+ * @code
+ *DEFINE_QARRAY_TYPE(Foo, free_foo);
+ * @endcode
+ * Finally the array may then be used like this:
+ * @code
+ * void doSomething(int n) {+ * QArrayRef(Foo) foos = NULL; + * QARRAY_CREATE(Foo, foos, n); + * for (size_t i = 0; i < n; ++i) {Quoted 8 lines
Quoted 5 lines
+ * foos[i].i = i; + * foos[i].s = calloc(4096, 1); + * snprintf(foos[i].s, 4096, "foo %d", i);+ * }
+ * }So essentially here the 'foos' variable is still a plain C array from POV of the caller ie it is a 'Foo *foos'
Yes, that's the main feature probably: getting rid of any (n times) manually written deep-free code branches while at the same time keeping the burden low, i.e. requiring to refactor as little existing code as possible, as it is still a "normal" C-array from user code perspective, just with meta info prepended in front of the C-array that the user code however does not need to care about at all:
Contiguous Memory Space ->
----------------------------------------------------------
| Array-Meta-Info | Scalar 0 | Scalar 1 | ... | Scalar n |
----------------------------------------------------------^- QArrayFoo* ^- Foo*
So switching between the two is just +-
const_offsetmath.Plus this has the positive side effect that a C-style array code is better to the human eye.
For now that "Array-Meta-Info" is really just the array size, in future this might be extended to contain a reference count for instance. And as this concept is actually generic code (a.k.a template code), you could also extend this in a way where you just pull in features that you really need in user code. I.e. if you don't need an atomic reference count at one place (and avoid the sync overhead), don't enable it for your use case. If you need it somewhere else, then enable it there.
Quoted 35 lines
By
QARRAY_CREATE()is actually allocating a 'QArrayFoo' which is a struct containing a 'size_tlen' along with the 'Foo *foos' entry.This is a clever trick, and it works in the example above, because the code already has the length available in a convenient way via the 'int n' parameter.
IMHO this makes the code less useful than it otherwise would be in the general case, because there are places where the code needs to pass around the array and its assoicated length, and so this with this magic hidden length, we're still left to pass around the length separately.
Consider this example:
int open_conn(const char *hostname) { SocketAddress *addrs; size_t naddrs; int ret = -1; size_t i;if (resolve_hostname(hostname, &addrs, &naddrs) < 0) return -1; for (i = 0; i < naddrs; i++) { ...try to connect to addrs[i]... } ret = 0 cleanup: for (i = 0; i < naddrs; i++) { qapi_free_SocketAddress(addrs[i]) } return ret;Quoted 8 lines
}
To simplify this it is deisrable to autofree the 'addrs' array.
If using QArray, it still has to keep passing around the '
size_tnaddrs' value because QArray hides the length field from the code.Well no, you don't need to pass around anything, as the array length is always accessible; it is always just (compile time) constant -wordsize (-padding) offset away from the C-array pointer. Maybe the phrasing "private" was a bit misleading in the
QArray.hcomments.It is correct that my 9p use case so far did not need the array length info by means of accessing an API, for that reason I really just ommitted (yet) to add a separate patch for that. All it would take was extending
QArray.hin a way like (roughly):typedef struct _QArrayGeneric { size_t len; char first[]; } _QArrayGeneric;/**
* Returns the amount of scalar elements in the passed array. *
* @param first - start of array
*/size_tqarray_len(void* first){ if (!first) { return 0; } _QArrayGeneric *arr = (_QArrayGeneric *) ( ((char *)first) - offsetof(_QArrayGeneric, first) ); return arr->len; }#define
QARRAY_LEN(arr)qarray_len(arr)And as this is generic code for all array scalar types, it would probably be partly placed in a separate
qarray.cfile.
Quoted 8 lines
After that change your user example would become:
for (i = 0; i < QARRAY_LEN(addrs); i++) { ...try to connect to addrs[i]... }If you want I can post a v3 with a formal patch (or two) to handle that array length API.
I still find this all overkill compared to just exposing the array struct explicitly.
Quoted 40 lines
Quoted 30 lines
If it instead just exposed the array struct explicitly, it can use the normal
g_autoptr()declarator, and can also now just return the array directly since it is a single pointerint open_conn(const char *hostname) { g_autoptr(SocketAddressArray) addrs = NULL; int ret = -1; size_t i;if (!(addrs = resolve_hostname(hostname))) return -1; for (i = 0; i < addrs.len; i++) { ...try to connect to addrs.data[i]... } ret = 0 cleanup: return ret;}
In terms of the first example, it adds an indirection to access the array data, but on the plus side IMHO the code is clearer because it uses '
g_autoptr' which is what is more in line with what is expected for variables that are automatically freed.QArrayRef()as a name doesn't make it clear that the value will be freed.Quoted 9 lines
void doSomething(int n) { g_autoptr(FooArray) foos = NULL; QARRAY_CREATE(Foo, foos, n); for (size_t i = 0; i < foos.len; ++i) { foos.data[i].i = i; foos.data[i].s = calloc(4096, 1); snprintf(foos.data[i].s, 4096, "foo %d", i); } }
Quoted 4 lines
Well, that would destroy the intended major feature "as little refactoring as possible". The amount of locations where you define a reference variable is usually much smaller than the amount of code locations where you actually access arrays.
If there's a large amount of existing code refactoring to be avoided an intermediate variable can be declared to point to the struct field to avoid the field references.
Quoted 5 lines
Personally I would not mix in this case macros of foreign libraries (glib) with macros of a local framework (QArray), because if for some reason one of the two deviate in future in a certain way, you would need to refactor a whole bunch of user code. By just separating those definitions from day one, you can avoid such future refactoring work right from the start.
The GLib automatic memory support is explicitly designed to be extendd with support for application specific types. We already do exactly that all over QEMU with many calls to G_DEFINE_AUTOPTR_CLEANUP_FUNC(..) to register functions for free'ing specific types, such that you can use 'g_autoptr' with them.
Quoted 4 lines
As far as the terminology is concerned: probably a matter of taste. For me a "reference" implies (either unique or shared) ownership, a "pointer" IMO doesn't. And the usage of QArray alone makes it clear that an array without any references gets automatically freed.
It is more important than a matter of taste - it is about having a consistent approach throughout QEMU. That means automatic free'ing of variables should involve g_autoptr, not something custom to a specific type with different terminology.
Quoted 38 lines
Quoted 14 lines
I would also suggest that
QARRAY_CREATEdoesn't need to exist as a macro - callers could just use the allocator function directly for clearer code, if it was changed to return the ptr rather than use an out parameter:void doSomething(int n) { g_autoptr(FooArray) foos = foo_array_new(n); for (size_t i = 0; i < foos.len; ++i) { foos.data[i].i = i; foos.data[i].s = calloc(4096, 1); snprintf(foos.data[i].s, 4096, "foo %d", i); } }Quoted 3 lines
For this it needs to pass 2 args into the
DECLARE_QARRAY_TYPEmacro - the struct name and desired method name - basically the method name is the struct name in lowercase with underscores.As you can see with patch 2, one of the movations of making this a macro was the intention to increase strictness of type safety, e.g to make things like:
void *p; ... QARRAY_CREATE(FooType, p, n);to raise a compiler error immediately, but that's not all ...
Quoted 4 lines
Overall I think the goal of having an convenient sized array for types is good, but I think we should make it look a bit less magic. I think we only need the
DECLARE_QARRAY_TYPEandDEFINE_QARRAY_TYPEmacros.... actually making it appear anything like magic was not my intention. The actual main reason for wrapping these things into macros is because that's actually the only way to write generic code in C. Especially in larger projects like this one I favour clear separation of API ("how to use it") from its actual implementation ("how does it do it exactly").
Quoted 3 lines
So if you use macros for all those things from the beginning, it is far less likely that you will need to refactor a huge amount of user code with future changes of this array framework.
I can't see the array framework being complex enough that it will be changed in a way that invalidates existing usage.
Quoted 6 lines
Quoted 3 lines
Incidentally, I'd suggest naming to be
QARRAY_DECLARE_TYPEandQARRAY_DEFINE_TYPE.Also a matter of taste I guess. The suggested naming
DECLARE_QARRAY_TYPE()andDEFINE_QARRAY_TYPE()reflect more natural language IMO.
I consider the QEMU normal practice for namespacing types/macros/functions is to have the typename as the first component.
Regards,
Daniel
Signature · 3 lines
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
Re: [PATCH v2 1/5] qemu/qarray.h: introduce QArray
Quoted 3 lines · On Dienstag, 28. September 2021 18:41:17 CEST Daniel P. Berrangé wrote:
Quoted 2 lines · Christian Schoenebeck
Quoted 1 line · On Dienstag, 28. September 2021 15:04:36 CEST Daniel P. Berrangé wrote:
On Sun, Aug 22, 2021 at 03:16:46PM +0200, Christian Schoenebeck wrote:
[...]
Quoted 55 lines
Quoted 55 lines
Quoted 3 lines
If using QArray, it still has to keep passing around the '
size_tnaddrs' value because QArray hides the length field from the code.Well no, you don't need to pass around anything, as the array length is always accessible; it is always just (compile time) constant -wordsize (-padding) offset away from the C-array pointer. Maybe the phrasing "private" was a bit misleading in the
QArray.hcomments.It is correct that my 9p use case so far did not need the array length info by means of accessing an API, for that reason I really just ommitted (yet) to add a separate patch for that. All it would take was extending
QArray.hin a way like (roughly):typedef struct
_QArrayGeneric{size_t len; char first[];}
_QArrayGeneric;/**
* Returns the amount of scalar elements in the passed array. *
* @param first - start of array
*/
size_tqarray_len(void* first)
{if (!first) { return 0; } _QArrayGeneric *arr = (_QArrayGeneric *) ( ((char *)first) - offsetof(_QArrayGeneric, first) ); return arr->len;}
#define
QARRAY_LEN(arr)qarray_len(arr)And as this is generic code for all array scalar types, it would probably be partly placed in a separate
qarray.cfile.After that change your user example would become:
for (i = 0; i <QARRAY_LEN(addrs); i++) {...try to connect to addrs[i]...
Quoted 7 lines
Quoted 4 lines
}If you want I can post a v3 with a formal patch (or two) to handle that array length API.
I still find this all overkill compared to just exposing the array struct explicitly.
Yes, you made that clear. :)
Quoted 48 lines
Quoted 48 lines
Quoted 47 lines
If it instead just exposed the array struct explicitly, it can use the normal
g_autoptr()declarator, and can also now just return the array directly since it is a single pointerint open_conn(const char *hostname) {g_autoptr(SocketAddressArray) addrs = NULL; int ret = -1; size_t i; if (!(addrs = resolve_hostname(hostname))) return -1; for (i = 0; i < addrs.len; i++) { ...try to connect to addrs.data[i]... } ret = 0cleanup: return ret;}In terms of the first example, it adds an indirection to access the array data, but on the plus side IMHO the code is clearer because it uses '
g_autoptr' which is what is more in line with what is expected for variables that are automatically freed.QArrayRef()as a name doesn't make it clear that the value will be freed.void doSomething(int n) {g_autoptr(FooArray) foos = NULL; QARRAY_CREATE(Foo, foos, n); for (size_t i = 0; i < foos.len; ++i) { foos.data[i].i = i; foos.data[i].s = calloc(4096, 1); snprintf(foos.data[i].s, 4096, "foo %d", i); }}
Quoted 8 lines
Quoted 4 lines
Well, that would destroy the intended major feature "as little refactoring as possible". The amount of locations where you define a reference variable is usually much smaller than the amount of code locations where you actually access arrays.
If there's a large amount of existing code refactoring to be avoided an intermediate variable can be declared to point to the struct field to avoid the field references.
That would be one additional (unguarded) raw pointer variable per array & function, that multiplied by the amount of arrays and functions ...
... the suggested shared utility code is 34 lines LOC net.
Quoted 11 lines
Quoted 5 lines
Personally I would not mix in this case macros of foreign libraries (glib) with macros of a local framework (QArray), because if for some reason one of the two deviate in future in a certain way, you would need to refactor a whole bunch of user code. By just separating those definitions from day one, you can avoid such future refactoring work right from the start.
The GLib automatic memory support is explicitly designed to be extendd with support for application specific types. We already do exactly that all over QEMU with many calls to
G_DEFINE_AUTOPTR_CLEANUP_FUNC(..)to register functions for free'ing specific types, such that you can use 'g_autoptr' with them.
Ok, just to make sure that I am not missing something here, because really if there is already something that does the job that I simply haven't seen, then I happily drop this QArray code.
But AFAICS this G_DEFINE_AUTOPTR_CLEANUP_FUNC() & g_autoptr concept does not have any notion of "size" or "amount", right?
So let's say you already have the following type and cleanup function in your existing code:
typedef struct MyScalar {
int a;
char *b;
} MyScalar;void myscalar_free(MayScalar *s) {
g_free(s->b);
}Then if you want to use G_DEFINE_AUTOPTR_CLEANUP_FUNC() for an array on that scalar type, then you still would need to *manually* write additionally a separate type and cleanup function like:
typedef struct MyArray {
MyScalar *s;
int n;
};void myarray_free(MyArray *a) {
for (int i = 0; i < a->n; ++i) {
myscalar_free(a->s[i]);
}
g_free(a);
}Plus you have to manually populate that field 'n' after allocation.
Am I wrong?
Quoted 9 lines
Quoted 4 lines
As far as the terminology is concerned: probably a matter of taste. For me a "reference" implies (either unique or shared) ownership, a "pointer" IMO doesn't. And the usage of QArray alone makes it clear that an array without any references gets automatically freed.
It is more important than a matter of taste - it is about having a consistent approach throughout QEMU. That means automatic free'ing of variables should involve
g_autoptr, not something custom to a specific type with different terminology.
The barriers to add few lines of utility code are really high. :)
Quoted 18 lines
Quoted 18 lines
Quoted 18 lines
I would also suggest that
QARRAY_CREATEdoesn't need to exist as a macro - callers could just use the allocator function directly for clearer code, if it was changed toreturn the ptr rather than use an out parameter:
void doSomething(int n) { g_autoptr(FooArray) foos = foo_array_new(n); for (size_t i = 0; i < foos.len; ++i) { foos.data[i].i = i; foos.data[i].s = calloc(4096, 1); snprintf(foos.data[i].s, 4096, "foo %d", i); } }
Quoted 7 lines
Quoted 7 lines
Quoted 3 lines
For this it needs to pass 2 args into the
DECLARE_QARRAY_TYPEmacro - the struct name and desired method name - basically the method name is the struct name in lowercase with underscores.As you can see with patch 2, one of the movations of making this a macro was>
the intention to increase strictness of type safety, e.g to make things
like:
Quoted 18 lines
Quoted 18 lines
void *p; ... QARRAY_CREATE(FooType, p, n);to raise a compiler error immediately, but that's not all ...
Quoted 4 lines
Overall I think the goal of having an convenient sized array for types is good, but I think we should make it look a bit less magic. I think we only need the
DECLARE_QARRAY_TYPEandDEFINE_QARRAY_TYPEmacros.... actually making it appear anything like magic was not my intention. The
actual main reason for wrapping these things into macros is because that's actually the only way to write generic code in C. Especially in larger projects like this one I favour clear separation of API ("how to use it") from its actual implementation ("how does it do it exactly").
Quoted 6 lines
Quoted 3 lines
So if you use macros for all those things from the beginning, it is far less likely that you will need to refactor a huge amount of user code with future changes of this array framework.
I can't see the array framework being complex enough that it will be changed in a way that invalidates existing usage.
Well, there are some things that would come to my mind (e.g. strong vs. weak refs) , but I think for now my upper question is more important ATM, i.e. whether there is already something that does the job (right).
Quoted 11 lines
Quoted 5 lines
Quoted 2 lines
Incidentally, I'd suggest naming to be
QARRAY_DECLARE_TYPEandQARRAY_DEFINE_TYPE.Also a matter of taste I guess. The suggested naming
DECLARE_QARRAY_TYPE()andDEFINE_QARRAY_TYPE()reflect more natural language IMO.I consider the QEMU normal practice for namespacing types/macros/functions is to have the typename as the first component.
Regards,
Daniel
Re: [PATCH v2 1/5] qemu/qarray.h: introduce QArray
Quoted 18 lines · Christian Schoenebeck
Quoted 3 lines · On Dienstag, 28. September 2021 18:41:17 CEST Daniel P. Berrangé wrote:
Quoted 2 lines · Christian Schoenebeck
Quoted 1 line · On Dienstag, 28. September 2021 15:04:36 CEST Daniel P. Berrangé wrote:
On Sun, Aug 22, 2021 at 03:16:46PM +0200, Christian Schoenebeck wrote:
[...]
Quoted 11 lines
Quoted 5 lines
Personally I would not mix in this case macros of foreign libraries (glib) with macros of a local framework (QArray), because if for some reason one of the two deviate in future in a certain way, you would need to refactor a whole bunch of user code. By just separating those definitions from day one, you can avoid such future refactoring work right from the start.
The GLib automatic memory support is explicitly designed to be extendd with support for application specific types. We already do exactly that all over QEMU with many calls to
G_DEFINE_AUTOPTR_CLEANUP_FUNC(..)to register functions for free'ing specific types, such that you can use 'g_autoptr' with them.
Quoted 3 lines
Ok, just to make sure that I am not missing something here, because really if there is already something that does the job that I simply haven't seen, then I happily drop this QArray code.
I don't believe there is anything that currently addresses this well.
Quoted 2 lines
But AFAICS this
G_DEFINE_AUTOPTR_CLEANUP_FUNC()&g_autoptrconcept does not have any notion of "size" or "amount", right?
Correct, all it knows is that there's a data type and an associated free function.
Quoted 28 lines
So let's say you already have the following type and cleanup function in your existing code:
typedef struct MyScalar { int a; char *b; } MyScalar;void myscalar_free(MayScalar *s) { g_free(s->b); }Then if you want to use
G_DEFINE_AUTOPTR_CLEANUP_FUNC()for an array on that scalar type, then you still would need to *manually* write additionally a separate type and cleanup function like:typedef struct MyArray { MyScalar *s; int n; };void myarray_free(MyArray *a) { for (int i = 0; i < a->n; ++i) { myscalar_free(a->s[i]); } g_free(a); }
Quoted 3 lines
Plus you have to manually populate that field 'n' after allocation.
Am I wrong?
Yes and no. You can of course manually write all these stuff as you describe, but since we expect the array wrappers to be needed for more than one type it makes more sense to have that all done via macros.
Your patch contains a DECLARE_QARRAY_TYPE and DEFINE_QARRAY_TYPE that provide all this reqiured boilerplate code. The essential difference that I'm suggesting is that the array struct type emitted by the macro is explicitly visible as a concept to calling code such that it is used directly used with g_autoptr.
Regards,
Daniel
Signature · 3 lines
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
Re: [PATCH v2 1/5] qemu/qarray.h: introduce QArray
Quoted 4 lines · On Mittwoch, 29. September 2021 19:48:38 CEST Daniel P. Berrangé wrote:
Quoted 3 lines · Christian Schoenebeck
Quoted 2 lines · On Dienstag, 28. September 2021 18:41:17 CEST Daniel P. Berrangé wrote:
Quoted 1 line · Christian Schoenebeck
On Dienstag, 28. September 2021 15:04:36 CEST Daniel P. Berrangé
Quoted 1 line · wrote:
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
On Sun, Aug 22, 2021 at 03:16:46PM +0200, Christian Schoenebeck
Quoted 11 lines · wrote:
Quoted 10 lines
[...]
Quoted 5 lines
The GLib automatic memory support is explicitly designed to be extendd with support for application specific types. We already do exactly that all over QEMU with many calls to
G_DEFINE_AUTOPTR_CLEANUP_FUNC(..)to register functions for free'ing specific types, such that you can use 'g_autoptr' with them.Ok, just to make sure that I am not missing something here, because really if there is already something that does the job that I simply haven't seen, then I happily drop this QArray code.
Quoted 7 lines
I don't believe there is anything that currently addresses this well.
Quoted 2 lines
But AFAICS this
G_DEFINE_AUTOPTR_CLEANUP_FUNC()&g_autoptrconcept does not have any notion of "size" or "amount", right?Correct, all it knows is that there's a data type and an associated free function.
Ok, thanks for the clarification.
Quoted 47 lines
Quoted 36 lines
So let's say you already have the following type and cleanup function in your existing code:
typedef struct MyScalar {
int a; char *b;} MyScalar;
void
myscalar_free(MayScalar *s){g_free(s->b);}
Then if you want to use
G_DEFINE_AUTOPTR_CLEANUP_FUNC()for an array on that scalar type, then you still would need to *manually* write additionally a separate type and cleanup function like:typedef struct MyArray {
MyScalar *s; int n;};
void
myarray_free(MyArray *a){for (int i = 0; i < a->n; ++i) { myscalar_free(a->s[i]); } g_free(a);Quoted 5 lines
}
Plus you have to manually populate that field 'n' after allocation.
Am I wrong?
Yes and no. You can of course manually write all these stuff as you describe, but since we expect the array wrappers to be needed for more than one type it makes more sense to have that all done via macros.
Quoted 5 lines
Your patch contains a
DECLARE_QARRAY_TYPEandDEFINE_QARRAY_TYPEthat provide all this reqiured boilerplate code. The essential difference that I'm suggesting is that the array struct type emitted by the macro is explicitly visible as a concept to calling code such that it is used directly used withg_autoptr.
I got that, but your preferred user pattern was this:
DECLARE_QARRAY_TYPE(Foo);
...
g_autoptr(FooArray) foos = foo_array_new(n);I don't see a portable way to do upper-case to lower-case conversion with the C preprocessor. So you would end up like this instead:
g_autoptr(FooArray) foos = Foo_array_new(n);
Which does not really fit into common QEMU naming conventions either, does it?
And I can help it, I don't see what's wrong in exposing a regular C-array to user code. I mean in the Linux kernel for instance it is absolutely normal to convert from a compound structure to its parent structure. I don't find anything magical about that and it is simply less code and better readable.
Best regards,
Christian Schoenebeck
Re: [PATCH v2 1/5] qemu/qarray.h: introduce QArray
In reply to Daniel P. BerrangéQuoted 4 lines · On Donnerstag, 30. September 2021 15:31:10 CEST Daniel P. Berrangé wrote:
Quoted 3 lines · Christian Schoenebeck
Quoted 2 lines · On Mittwoch, 29. September 2021 19:48:38 CEST Daniel P. Berrangé wrote:
Quoted 1 line · Christian Schoenebeck
On Dienstag, 28. September 2021 18:41:17 CEST Daniel P. Berrangé
Quoted 1 line · wrote:
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
On Tue, Sep 28, 2021 at 06:23:23PM +0200, Christian Schoenebeck
Quoted 114 lines · wrote:
Quoted 95 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
On Dienstag, 28. September 2021 15:04:36 CEST Daniel P. Berrangé
Quoted 1 line · wrote:
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
On Sun, Aug 22, 2021 at 03:16:46PM +0200, Christian Schoenebeck
Quoted 16 lines · wrote:
Quoted 11 lines
[...]
Quoted 8 lines
The GLib automatic memory support is explicitly designed to be extendd
with support for application specific types. We already do exactly that
all over QEMU with many calls toG_DEFINE_AUTOPTR_CLEANUP_FUNC(..)to
register functions for free'ing specific types, such that you can use 'g_autoptr' with them.Quoted 4 lines
Ok, just to make sure that I am not missing something here, because really
if there is already something that does the job that I simply haven't seen, then I happily drop this QArray code.Quoted 8 lines
I don't believe there is anything that currently addresses this well.
Quoted 3 lines
But AFAICS this
G_DEFINE_AUTOPTR_CLEANUP_FUNC()&g_autoptrconcept does
not have any notion of "size" or "amount", right?Correct, all it knows is that there's a data type and an associated free function.
Ok, thanks for the clarification.
Quoted 49 lines
Quoted 38 lines
So let's say you already have the following type and cleanup function in
your existing code:typedef struct MyScalar {
int a; char *b;} MyScalar;
void
myscalar_free(MayScalar *s){g_free(s->b);}
Then if you want to use
G_DEFINE_AUTOPTR_CLEANUP_FUNC()for an array on
that scalar type, then you still would need to *manually* write additionally a separate type and cleanup function like:typedef struct MyArray {
MyScalar *s; int n;};
void
myarray_free(MyArray *a){for (int i = 0; i < a->n; ++i) { myscalar_free(a->s[i]); } g_free(a);Quoted 5 lines
}
Plus you have to manually populate that field 'n' after allocation.
Am I wrong?
Yes and no. You can of course manually write all these stuff as you describe, but since we expect the array wrappers to be needed for more than one type it makes more sense to have that all done via macros.
Quoted 5 lines
Your patch contains a
DECLARE_QARRAY_TYPEandDEFINE_QARRAY_TYPEthat provide all this reqiured boilerplate code. The essential difference that I'm suggesting is that the array struct type emitted by the macro is explicitly visible as a concept to calling code such that it is used directly used withg_autoptr.I got that, but your preferred user pattern was this:
DECLARE_QARRAY_TYPE(Foo); ... g_autoptr(FooArray) foos = foo_array_new(n);Quoted 7 lines
I don't see a portable way to do upper-case to lower-case conversion with the>
C preprocessor. So you would end up like this instead:g_autoptr(FooArray) foos = Foo_array_new(n);Which does not really fit into common QEMU naming conventions either, does it?
Right, it would need to be a two arg macro:
DECLARE_QARRAY_TYPE(Foo, foo);similar to what we do with macros for declaring QOM types becuase of the same case conversion needs.
Quoted 5 lines
And I can help it, I don't see what's wrong in exposing a regular C-array to user code. I mean in the Linux kernel for instance it is absolutely normal to convert from a compound structure to its parent structure. I don't find anything magical about that and it is simply less code and better readable.
Quoted 4 lines
QEMU code is not Linux code. We're following the GLib practices for automatic memory deallocation, and QOM is also modelled on GLib. The proposal looks magical from the POV of QEMU's code patterns, as it is not making use of GLib's g_auto* code.
Hmm, I start to think whether I should just make it some 9p local utility code for now instead, e.g. "P9Array" or something.
Best regards,
Christian Schoenebeck
Re: [PATCH v2 1/5] qemu/qarray.h: introduce QArray
In reply to Christian SchoenebeckQuoted 88 lines · Christian Schoenebeck
Quoted 4 lines · On Mittwoch, 29. September 2021 19:48:38 CEST Daniel P. Berrangé wrote:
Quoted 3 lines · Christian Schoenebeck
Quoted 2 lines · On Dienstag, 28. September 2021 18:41:17 CEST Daniel P. Berrangé wrote:
Quoted 1 line · Christian Schoenebeck
On Dienstag, 28. September 2021 15:04:36 CEST Daniel P. Berrangé
Quoted 1 line · wrote:
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
On Sun, Aug 22, 2021 at 03:16:46PM +0200, Christian Schoenebeck
Quoted 11 lines · wrote:
Quoted 10 lines
[...]
Quoted 5 lines
The GLib automatic memory support is explicitly designed to be extendd with support for application specific types. We already do exactly that all over QEMU with many calls to
G_DEFINE_AUTOPTR_CLEANUP_FUNC(..)to register functions for free'ing specific types, such that you can use 'g_autoptr' with them.Ok, just to make sure that I am not missing something here, because really if there is already something that does the job that I simply haven't seen, then I happily drop this QArray code.
Quoted 7 lines
I don't believe there is anything that currently addresses this well.
Quoted 2 lines
But AFAICS this
G_DEFINE_AUTOPTR_CLEANUP_FUNC()&g_autoptrconcept does not have any notion of "size" or "amount", right?Correct, all it knows is that there's a data type and an associated free function.
Ok, thanks for the clarification.
Quoted 47 lines
Quoted 36 lines
So let's say you already have the following type and cleanup function in your existing code:
typedef struct MyScalar {
int a; char *b;} MyScalar;
void
myscalar_free(MayScalar *s){g_free(s->b);}
Then if you want to use
G_DEFINE_AUTOPTR_CLEANUP_FUNC()for an array on that scalar type, then you still would need to *manually* write additionally a separate type and cleanup function like:typedef struct MyArray {
MyScalar *s; int n;};
void
myarray_free(MyArray *a){for (int i = 0; i < a->n; ++i) { myscalar_free(a->s[i]); } g_free(a);Quoted 5 lines
}
Plus you have to manually populate that field 'n' after allocation.
Am I wrong?
Yes and no. You can of course manually write all these stuff as you describe, but since we expect the array wrappers to be needed for more than one type it makes more sense to have that all done via macros.
Quoted 5 lines
Your patch contains a
DECLARE_QARRAY_TYPEandDEFINE_QARRAY_TYPEthat provide all this reqiured boilerplate code. The essential difference that I'm suggesting is that the array struct type emitted by the macro is explicitly visible as a concept to calling code such that it is used directly used withg_autoptr.I got that, but your preferred user pattern was this:
DECLARE_QARRAY_TYPE(Foo); ... g_autoptr(FooArray) foos = foo_array_new(n);
Quoted 6 lines
I don't see a portable way to do upper-case to lower-case conversion with the C preprocessor. So you would end up like this instead:
g_autoptr(FooArray) foos = Foo_array_new(n);Which does not really fit into common QEMU naming conventions either, does it?
Right, it would need to be a two arg macro:
DECLARE_QARRAY_TYPE(Foo, foo);
similar to what we do with macros for declaring QOM types becuase of the same case conversion needs.
Quoted 4 lines
And I can help it, I don't see what's wrong in exposing a regular C-array to user code. I mean in the Linux kernel for instance it is absolutely normal to convert from a compound structure to its parent structure. I don't find anything magical about that and it is simply less code and better readable.
QEMU code is not Linux code. We're following the GLib practices for automatic memory deallocation, and QOM is also modelled on GLib. The proposal looks magical from the POV of QEMU's code patterns, as it is not making use of GLib's g_auto* code.
Regards,
Daniel
Signature · 3 lines
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
Re: [PATCH v2 1/5] qemu/qarray.h: introduce QArray
In reply to Christian SchoenebeckQuoted 122 lines · Christian Schoenebeck
Quoted 4 lines · On Donnerstag, 30. September 2021 15:31:10 CEST Daniel P. Berrangé wrote:
Quoted 3 lines · Christian Schoenebeck
Quoted 2 lines · On Mittwoch, 29. September 2021 19:48:38 CEST Daniel P. Berrangé wrote:
Quoted 1 line · Christian Schoenebeck
On Dienstag, 28. September 2021 18:41:17 CEST Daniel P. Berrangé
Quoted 1 line · wrote:
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
On Tue, Sep 28, 2021 at 06:23:23PM +0200, Christian Schoenebeck
Quoted 114 lines · wrote:
Quoted 95 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
On Dienstag, 28. September 2021 15:04:36 CEST Daniel P. Berrangé
Quoted 1 line · wrote:
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
On Sun, Aug 22, 2021 at 03:16:46PM +0200, Christian Schoenebeck
Quoted 16 lines · wrote:
Quoted 11 lines
[...]
Quoted 8 lines
The GLib automatic memory support is explicitly designed to be extendd
with support for application specific types. We already do exactly that
all over QEMU with many calls toG_DEFINE_AUTOPTR_CLEANUP_FUNC(..)to
register functions for free'ing specific types, such that you can use 'g_autoptr' with them.Quoted 4 lines
Ok, just to make sure that I am not missing something here, because really
if there is already something that does the job that I simply haven't seen, then I happily drop this QArray code.Quoted 8 lines
I don't believe there is anything that currently addresses this well.
Quoted 3 lines
But AFAICS this
G_DEFINE_AUTOPTR_CLEANUP_FUNC()&g_autoptrconcept does
not have any notion of "size" or "amount", right?Correct, all it knows is that there's a data type and an associated free function.
Ok, thanks for the clarification.
Quoted 49 lines
Quoted 38 lines
So let's say you already have the following type and cleanup function in
your existing code:typedef struct MyScalar {
int a; char *b;} MyScalar;
void
myscalar_free(MayScalar *s){g_free(s->b);}
Then if you want to use
G_DEFINE_AUTOPTR_CLEANUP_FUNC()for an array on
that scalar type, then you still would need to *manually* write additionally a separate type and cleanup function like:typedef struct MyArray {
MyScalar *s; int n;};
void
myarray_free(MyArray *a){for (int i = 0; i < a->n; ++i) { myscalar_free(a->s[i]); } g_free(a);Quoted 5 lines
}
Plus you have to manually populate that field 'n' after allocation.
Am I wrong?
Yes and no. You can of course manually write all these stuff as you describe, but since we expect the array wrappers to be needed for more than one type it makes more sense to have that all done via macros.
Quoted 5 lines
Your patch contains a
DECLARE_QARRAY_TYPEandDEFINE_QARRAY_TYPEthat provide all this reqiured boilerplate code. The essential difference that I'm suggesting is that the array struct type emitted by the macro is explicitly visible as a concept to calling code such that it is used directly used withg_autoptr.I got that, but your preferred user pattern was this:
DECLARE_QARRAY_TYPE(Foo); ... g_autoptr(FooArray) foos = foo_array_new(n);Quoted 7 lines
I don't see a portable way to do upper-case to lower-case conversion with the>
C preprocessor. So you would end up like this instead:g_autoptr(FooArray) foos = Foo_array_new(n);Which does not really fit into common QEMU naming conventions either, does it?
Right, it would need to be a two arg macro:
DECLARE_QARRAY_TYPE(Foo, foo);similar to what we do with macros for declaring QOM types becuase of the same case conversion needs.
Quoted 5 lines
And I can help it, I don't see what's wrong in exposing a regular C-array to user code. I mean in the Linux kernel for instance it is absolutely normal to convert from a compound structure to its parent structure. I don't find anything magical about that and it is simply less code and better readable.
Quoted 7 lines
Quoted 4 lines
QEMU code is not Linux code. We're following the GLib practices for automatic memory deallocation, and QOM is also modelled on GLib. The proposal looks magical from the POV of QEMU's code patterns, as it is not making use of GLib's g_auto* code.
Hmm, I start to think whether I should just make it some 9p local utility code for now instead, e.g. "P9Array" or something.
IMHO even if it was private to a subsystem it should still be using the standard g_auto functionality for automatically deallocating memory, because this is a QEMU wide standard.
Regards,
Daniel
Signature · 3 lines
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
Re: [PATCH v2 1/5] qemu/qarray.h: introduce QArray
Quoted 4 lines · On Donnerstag, 30. September 2021 16:01:38 CEST Daniel P. Berrangé wrote:
Quoted 3 lines · Christian Schoenebeck
Quoted 2 lines · On Donnerstag, 30. September 2021 15:31:10 CEST Daniel P. Berrangé wrote:
Quoted 1 line · Christian Schoenebeck
On Mittwoch, 29. September 2021 19:48:38 CEST Daniel P. Berrangé
Quoted 1 line · wrote:
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
On Wed, Sep 29, 2021 at 07:32:39PM +0200, Christian Schoenebeck
Quoted 142 lines · wrote:
Quoted 142 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
On Dienstag, 28. September 2021 18:41:17 CEST Daniel P. Berrangé
Quoted 1 line · wrote:
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
On Tue, Sep 28, 2021 at 06:23:23PM +0200, Christian Schoenebeck
Quoted 135 lines · wrote:
Quoted 111 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
On Dienstag, 28. September 2021 15:04:36 CEST Daniel P. Berrangé
Quoted 2 lines · wrote:
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
On Sun, Aug 22, 2021 at 03:16:46PM +0200, Christian Schoenebeck
Quoted 24 lines · wrote:
Quoted 14 lines
[...]
Quoted 11 lines
The GLib automatic memory support is explicitly designed to be extendd
with support for application specific types. We already do exactly
that
all over QEMU with many calls toG_DEFINE_AUTOPTR_CLEANUP_FUNC(..)
to
register functions for free'ing specific types, such that you can
use 'g_autoptr' with them.Quoted 6 lines
Ok, just to make sure that I am not missing something here, because
really
if there is already something that does the job that I simply haven't
seen, then I happily drop this QArray code.I don't believe there is anything that currently addresses this well.
Quoted 7 lines
Quoted 4 lines
But AFAICS this
G_DEFINE_AUTOPTR_CLEANUP_FUNC()&g_autoptrconcept
does
not have any notion of "size" or "amount", right?Correct, all it knows is that there's a data type and an associated free function.
Ok, thanks for the clarification.
Quoted 52 lines
Quoted 40 lines
So let's say you already have the following type and cleanup function
in
your existing code:typedef struct MyScalar {
int a; char *b;} MyScalar;
void
myscalar_free(MayScalar *s){g_free(s->b);}
Then if you want to use
G_DEFINE_AUTOPTR_CLEANUP_FUNC()for an array
on
that scalar type, then you still would need to *manually* write additionally a separate type and cleanup function like:typedef struct MyArray {
MyScalar *s; int n;};
void
myarray_free(MyArray *a){for (int i = 0; i < a->n; ++i) { myscalar_free(a->s[i]); } g_free(a);Quoted 6 lines
}
Plus you have to manually populate that field 'n' after allocation.
Am I wrong?
Yes and no. You can of course manually write all these stuff as you describe, but since we expect the array wrappers to be needed for more than one type it makes more sense to have that all done via macros.
Quoted 5 lines
Your patch contains a
DECLARE_QARRAY_TYPEandDEFINE_QARRAY_TYPEthat provide all this reqiured boilerplate code. The essential difference that I'm suggesting is that the array struct type emitted by the macro is explicitly visible as a concept to calling code such that it is used directly used withg_autoptr.I got that, but your preferred user pattern was this:
DECLARE_QARRAY_TYPE(Foo); ... g_autoptr(FooArray) foos = foo_array_new(n);I don't see a portable way to do upper-case to lower-case conversion with
the>Quoted 6 lines
C preprocessor. So you would end up like this instead:
g_autoptr(FooArray) foos = Foo_array_new(n);Which does not really fit into common QEMU naming conventions either, does
it?Right, it would need to be a two arg macro:
DECLARE_QARRAY_TYPE(Foo, foo);similar to what we do with macros for declaring QOM types becuase of the same case conversion needs.
Quoted 6 lines
And I can help it, I don't see what's wrong in exposing a regular C-array
to user code. I mean in the Linux kernel for instance it is absolutely normal to convert from a compound structure to its parent structure. I don't find anything magical about that and it is simply less code and better readable.QEMU code is not Linux code. We're following the GLib practices for automatic memory deallocation, and QOM is also modelled on GLib. The proposal looks magical from the POV of QEMU's code patterns, as it is not making use of GLib's g_auto* code.
Quoted 6 lines
Quoted 2 lines
Hmm, I start to think whether I should just make it some 9p local utility code for now instead, e.g. "P9Array" or something.
IMHO even if it was private to a subsystem it should still be using the standard
g_autofunctionality for automatically deallocating memory, because this is a QEMU wide standard.
There are already things like V9fsString, V9fsPath, ... not introduced by me BTW. There is a whole bunch of stuff that you could argue that it does not comply with common project patterns and nobody cared about.
I follow project standards wherever possible. But in this particular case, if it does not fit, it simply does not fit. So I will go on making at a local type for now and if there is really some need for project wide usage we can resume that issue at a later point.
Best regards,
Christian Schoenebeck