tools/xenstore: drop TDB
v3 cover tools/xenstore: drop TDB
Using TDB for storing the Xenstore nodes is adding more complexity instead of removing it. With keeping the data in memory only, the main reason for using TDB has disappeared.
This series is replacing TDB with a hashlist referencing directly individually allocated Xenstore nodes.
This in turn allows to use references of the stored nodes instead of copying them in case only read access is needed. Some simple tests using the test-xenstore utility have shown no single test being slower with this series applied, but some tests experienced up to 10% better performance.
Note that this series is based on top of [1].
Changes in V3:
- addressed comments
- add patches 14+15
- replace patch 18 of v2 with patches 20-25
Changes in V2:
- addressed comments
- split former patch 9 into 2 patches
- added patches 13-18
[1]: https://lists.xen.org/archives/html/xen-devel/2023-07/threads.html#01625
Juergen Gross (25):
tools/xenstore: explicitly specify create or modify for tdb_store() tools/xenstore: replace key in struct node with data base name tools/xenstore: let transaction_prepend() return the name for access tools/xenstore: rename do_tdb_delete() and change parameter type tools/xenstore: rename do_tdb_write() and change parameter type tools/xenstore: switch get_acc_data() to use name instead of key tools/xenstore: add wrapper for tdb_fetch() tools/xenstore: make hashtable key and value parameters const tools/xenstore: let hashtable_add() fail in case of existing entry tools/xenstore: add hashtable_replace() function tools/xenstore: drop use of tdb
tools/xenstore: remove tdb code
tools/xenstore: let db_delete() return void tools/xenstore: change talloc_free() to take a const pointer tools/xenstore: make data parameter of db_write() const tools/xenstore: move copying of node data out of db_fetch() tools/xenstore: rework struct xs_tdb_record_hdr tools/xenstore: don't use struct node_perms in struct node tools/xenstore: use struct node_hdr in struct node tools/xenstore: alloc new memory in domain_adjust_node_perms() tools/xenstore: introduce read_node_nocopy() tools/xenstore: merge get_spec_node() into get_node_canonicalized() tools/xenstore: merge is_valid_nodename() into canonicalize() tools/xenstore: rework get_node()
tools/xenstore: introduce get_node_const()
tools/xenstore/Makefile.common | 2 +- tools/xenstore/hashtable.c | 62 +- tools/xenstore/hashtable.h | 20 +- tools/xenstore/talloc.c | 8 +- tools/xenstore/talloc.h | 4 +- tools/xenstore/tdb.c | 1748 ------------------------ tools/xenstore/tdb.h | 132 -- tools/xenstore/utils.h | 9 - tools/xenstore/xenstored_core.c | 704 +++++----- tools/xenstore/xenstored_core.h | 86 +- tools/xenstore/xenstored_domain.c | 33 +- tools/xenstore/xenstored_transaction.c | 110 +- tools/xenstore/xenstored_transaction.h | 5 +- tools/xenstore/xenstored_watch.c | 42 +- tools/xenstore/xenstored_watch.h | 3 +- 15 files changed, 591 insertions(+), 2377 deletions(-) delete mode 100644 tools/xenstore/tdb.c delete mode 100644 tools/xenstore/tdb.h
Signature · 1 line
2.35.3
PATCH v3 1/25 tools/xenstore: explicitly specify create or modify for tdb_store()
Instead of using TDB_REPLACE for either creating or modifying a TDB entry, use either TDB_INSERT or TDB_MODIFY when calling tdb_store().
At higher function levels use the abstract mode values NODE_CREATE and NODE_MODIFY.
This is for preparing to get rid of TDB, even if it is beneficial while using TDB, too.
---
V2:
- use an enum for the new "flag" parameter of do_tdb_write() (Julien Grall)
- add comment to read_state_node() (Julien Grall)
V3:
- use enum in finalize_transaction() and reduce the scope of it (Julien Grall)
--- tools/xenstore/xenstored_core.c | 32 +++++++++++++++----------- tools/xenstore/xenstored_core.h | 10 ++++++-- tools/xenstore/xenstored_domain.c | 2 +- tools/xenstore/xenstored_transaction.c | 9 ++++++-- 4 files changed, 35 insertions(+), 18 deletions(-)
tools/xenstore/xenstored_core.c +19 -13
@@ -601,7 +601,8 @@ static unsigned int get_acc_domid(struct connection *conn, TDB_DATA *key, }int do_tdb_write (struct connection *conn,TDB_DATA *key,TDB_DATA *data, -struct node_account_data *acc,bool no_quota_check) +struct node_account_data *acc,enum write_node_mode mode, +bool no_quota_check) {struct xs_tdb_record_hdr *hdr = (void *)data->dptr;struct node_account_data old_acc = {};
@@ -635,7 +636,8 @@ int do_tdb_write(struct connection *conn, TDB_DATA *key, TDB_DATA *data, }/* TDB should set errno, but doesn't even set ecode AFAICT. */ -if (tdb_store (tdb_ctx, *key, *data,TDB_REPLACE ) !=0 ) { +if (tdb_store (tdb_ctx, *key, *data, + (mode ==NODE_CREATE ) ?TDB_INSERT :TDB_MODIFY ) !=0 ) {domain_memory_add_nochk (conn, new_domid, -data->dsize - key->dsize);/* Error path, so no quota check. */
@@ -774,7 +776,7 @@ static bool read_node_can_propagate_errno(void) }int write_node_raw (struct connection *conn,TDB_DATA *key,struct node *node, -bool no_quota_check) +enum write_node_mode mode,bool no_quota_check) {TDB_DATA data;void *p;
@@ -812,7 +814,7 @@ int write_node_raw(struct connection *conn, TDB_DATA *key, struct node *node, p += node->datalen;memcpy (p, node->children, node->childlen); -if (do_tdb_write (conn, key, &data, &node->acc, no_quota_check)) +if (do_tdb_write (conn, key, &data, &node->acc, mode, no_quota_check))return EIO ;return 0 ;
@@ -823,14 +825,14 @@ int write_node_raw(struct connection *conn, TDB_DATA *key, struct node *node,* node->key. This can later be used if the change needs to be reverted. */ static int write_node (struct connection *conn,struct node *node, -bool no_quota_check) +enum write_node_mode mode,bool no_quota_check) {int ret;if (access_node (conn, node,NODE_ACCESS_WRITE , &node->key))return errno; - ret =write_node_raw (conn, &node->key, node, no_quota_check); + ret =write_node_raw (conn, &node->key, node, mode, no_quota_check);if (ret && conn && conn->transaction) {/* * Reverting access_node() is hard, so just fail the
@@ -1500,7 +1502,8 @@ static struct node *create_node(struct connection *conn, const void *ctx, goto err; } - ret =write_node (conn, i, false); + ret =write_node (conn, i, i->parent ?NODE_CREATE :NODE_MODIFY , + false);if (ret) goto err;
@@ -1564,7 +1567,7 @@ static int do_write(const void *ctx, struct connection *conn, }else { node->data = in->buffer + offset; node->datalen = datalen; -if (write_node (conn, node, false)) +if (write_node (conn, node,NODE_MODIFY , false))return errno; }
@@ -1614,7 +1617,7 @@ static int remove_child_entry(struct connection *conn, struct node *node,memdel (node->children, offset, childlen +1 , node->childlen); node->childlen -= childlen +1 ; -return write_node (conn, node, true); +return write_node (conn, node,NODE_MODIFY , true); } static int delete_child(struct connection *conn,
@@ -1812,7 +1815,7 @@ static int do_set_perms(const void *ctx, struct connection *conn,if (domain_nbentry_inc (conn,get_node_owner (node)))return ENOMEM ; -if (write_node (conn, node, false)) +if (write_node (conn, node,NODE_MODIFY , false))return errno;fire_watches (conn, ctx, name, node, false, &old_perms);
@@ -2326,7 +2329,7 @@ static void manual_node(const char *name, const char *child)if (child) node->childlen =strlen (child) +1 ; -if (write_node (NULL , node, false)) +if (write_node (NULL , node,NODE_CREATE , false))barf_perror ("Could not create initial node %s" , name);talloc_free (node); }
@@ -3474,12 +3477,15 @@ void read_state_node(const void *ctx, const void *state)barf ("allocation error restoring node" );set_tdb_key (parentname, &key); -if (write_node_raw (NULL , &key, parent, true)) +if (write_node_raw (NULL , &key, parent,NODE_MODIFY , true))barf ("write parent error restoring node" ); }set_tdb_key (name, &key); -if (write_node_raw (NULL , &key, node, true)) + +/* The "/" node is already existing, so it can only be modified here. */ +if (write_node_raw (NULL , &key, node, +strcmp (name,"/" ) ?NODE_CREATE :NODE_MODIFY , true))barf ("write node error restoring node" );if (domain_nbentry_inc (&conn,get_node_owner (node)))
tools/xenstore/xenstored_core.h +8 -2
@@ -238,8 +238,13 @@ static inline unsigned int get_node_owner(const struct node *node) }/* Write a node to the tdb data base. */ +enum write_node_mode { +NODE_CREATE , +NODE_MODIFY +}; +int write_node_raw (struct connection *conn,TDB_DATA *key,struct node *node, -bool no_quota_check); +enum write_node_mode mode,bool no_quota_check);/* Get a node from the tdb data base. */ struct node *read_node(struct connection *conn, const void *ctx,
@@ -359,7 +364,8 @@ int remember_string(struct hashtable *hash, const char *str);void set_tdb_key (const char *name,TDB_DATA *key);int do_tdb_write (struct connection *conn,TDB_DATA *key,TDB_DATA *data, -struct node_account_data *acc,bool no_quota_check); +struct node_account_data *acc,enum write_node_mode mode, +bool no_quota_check);int do_tdb_delete (struct connection *conn,TDB_DATA *key,struct node_account_data *acc);
tools/xenstore/xenstored_domain.c +1 -1
@@ -523,7 +523,7 @@ static int domain_tree_remove_sub(const void *ctx, struct connection *conn, node->perms.p[0 ].id = priv_domid; node->acc.memory =0 ;domain_nbentry_inc (NULL , priv_domid); -if (write_node_raw (NULL , &key, node, true)) { +if (write_node_raw (NULL , &key, node,NODE_MODIFY , true)) {/* That's unfortunate. We only can try to continue. */ syslog(LOG_ERR, "error when moving orphaned node %s to dom0\n",
tools/xenstore/xenstored_transaction.c +7 -2
@@ -290,7 +290,8 @@ int access_node(struct connection *conn, struct node *node, i->check_gen = true;if (node->generation !=NO_GENERATION ) {set_tdb_key (i->trans_name, &local_key); - ret =write_node_raw (conn, &local_key, node, true); + ret =write_node_raw (conn, &local_key, node, +NODE_CREATE , true);if (ret) goto err; i->ta_node = true;
@@ -401,12 +402,16 @@ static int finalize_transaction(struct connection *conn,set_tdb_key (i->trans_name, &ta_key); data =tdb_fetch (tdb_ctx, ta_key);if (data.dptr) { +enum write_node_mode mode; +trace_tdb ("read %s size %zu\n" , ta_key.dptr, ta_key.dsize + data.dsize); hdr = (void *)data.dptr; hdr->generation = ++generation; + mode = (i->generation ==NO_GENERATION ) + ?NODE_CREATE :NODE_MODIFY ; *is_corrupt |=do_tdb_write (conn, &key, &data, -NULL , true); +NULL , mode, true);talloc_free (data.dptr);if (do_tdb_delete (conn, &ta_key,NULL )) *is_corrupt = true;
Signature · 1 line
2.35.3
Re: [PATCH v3 01/25] tools/xenstore: explicitly specify create or modify for tdb_store()
Hi Juergen,
Quoted 10 lines · Juergen Gross
Instead of using
TDB_REPLACEfor either creating or modifying a TDB entry, use eitherTDB_INSERTorTDB_MODIFYwhen callingtdb_store().At higher function levels use the abstract mode values
NODE_CREATEandNODE_MODIFY.This is for preparing to get rid of TDB, even if it is beneficial while using TDB, too.
Signed-off-by Juergen Gross <[email protected]>
Cheers,
Signature · 1 line
Julien Grall
PATCH v3 2/25 tools/xenstore: replace key in struct node with data base name
In reply to Juergen GrossInstead of storing the TDB key in struct node, only store the name of the node used to access it in the data base.
Associated with that change replace the key parameter of access_node() with the equivalent db_name.
This is in preparation to replace TDB with a more simple data storage.
--- tools/xenstore/xenstored_core.c | 19 +++++++++++++------ tools/xenstore/xenstored_core.h | 4 ++-- tools/xenstore/xenstored_transaction.c | 10 +++++----- tools/xenstore/xenstored_transaction.h | 2 +- 4 files changed, 21 insertions(+), 14 deletions(-)
tools/xenstore/xenstored_core.c +13 -6
@@ -821,18 +821,20 @@ int write_node_raw(struct connection *conn, TDB_DATA *key, struct node *node, }/* -* Write the node. If the node is written, caller can find the key used in -* node->key. This can later be used if the change needs to be reverted. +* Write the node. If the node is written, caller can find the DB name used in +* node->db_name. This can later be used if the change needs to be reverted. */ static int write_node (struct connection *conn,struct node *node,enum write_node_mode mode,bool no_quota_check) {int ret; +TDB_DATA key; -if (access_node (conn, node,NODE_ACCESS_WRITE , &node->key)) +if (access_node (conn, node,NODE_ACCESS_WRITE , &node->db_name))return errno; - ret =write_node_raw (conn, &node->key, node, mode, no_quota_check); +set_tdb_key (node->db_name, &key); + ret =write_node_raw (conn, &key, node, mode, no_quota_check);if (ret && conn && conn->transaction) {/* * Reverting access_node() is hard, so just fail the
@@ -1450,10 +1452,13 @@ nomem:static void destroy_node_rm (struct connection *conn,struct node *node) { +TDB_DATA key; +if (streq (node->name,"/" ))corrupt (NULL ,"Destroying root node!" ); -do_tdb_delete (conn, &node->key, &node->acc); +set_tdb_key (node->db_name, &key); +do_tdb_delete (conn, &key, &node->acc); }static int destroy_node (struct connection *conn,struct node *node)
@@ -1643,10 +1648,11 @@ static int delnode_sub(const void *ctx, struct connection *conn,const char *root = arg;bool watch_exact;int ret; +const char *db_name;TDB_DATA key;/* Any error here will probably be repeated for all following calls. */ - ret =access_node (conn, node,NODE_ACCESS_DELETE , &key); + ret =access_node (conn, node,NODE_ACCESS_DELETE , &db_name);if (ret >0 )return WALK_TREE_SUCCESS_STOP ;
@@ -1654,6 +1660,7 @@ static int delnode_sub(const void *ctx, struct connection *conn,return WALK_TREE_ERROR_STOP ;/* In case of error stop the walk. */ +set_tdb_key (db_name, &key);if (!ret &&do_tdb_delete (conn, &key, &node->acc))return WALK_TREE_ERROR_STOP ;
tools/xenstore/xenstored_core.h +2 -2
@@ -181,8 +181,8 @@ struct node_account_data {struct node {const char *name; -/* Key used to update TDB */ -TDB_DATA key; +/* Name used to access data base. */ +const char *db_name;/* Parent (optional) */ struct node *parent;
tools/xenstore/xenstored_transaction.c +5 -5
@@ -227,7 +227,7 @@ void transaction_prepend(struct connection *conn, const char *name,* to be accessed in the data base. */ int access_node (struct connection *conn,struct node *node, -enum node_access_type type,TDB_DATA *key) +enum node_access_type type,const char **db_name) {struct accessed_node *i =NULL ;struct transaction *trans;
@@ -243,8 +243,8 @@ int access_node(struct connection *conn, struct node *node,if (!conn || !conn->transaction) {/* They're changing the global database. */ -if (key) -set_tdb_key (node->name, key); +if (db_name) + *db_name = node->name;return 0 ; }
@@ -308,8 +308,8 @@ int access_node(struct connection *conn, struct node *node,/* Nothing to delete. */ return -1 ; -if (key) { -set_tdb_key (i->trans_name, key); +if (db_name) { + *db_name = i->trans_name;if (type ==NODE_ACCESS_WRITE ) i->ta_node = true;if (type ==NODE_ACCESS_DELETE )
tools/xenstore/xenstored_transaction.h +1 -1
@@ -41,7 +41,7 @@ void ta_node_created(struct transaction *trans);/* This node was accessed. */ int __must_check access_node (struct connection *conn,struct node *node, -enum node_access_type type,TDB_DATA *key); +enum node_access_type type,const char **db_name);/* Queue watches for a modified node. */ void queue_watches (struct connection *conn,const char *name,bool watch_exact);
Signature · 1 line
2.35.3
PATCH v3 3/25 tools/xenstore: let transaction_prepend() return the name for access
In reply to Juergen GrossInstead of setting the TDB key for accessing the node in the data base, let transaction_prepend() return the associated name instead.
This is in preparation to replace TDB with a more simple data storage.
--- tools/xenstore/xenstored_core.c | 4 +++- tools/xenstore/xenstored_transaction.c | 11 ++++------- tools/xenstore/xenstored_transaction.h | 3 +-- 3 files changed, 8 insertions(+), 10 deletions(-)
tools/xenstore/xenstored_core.c +3 -1
@@ -695,6 +695,7 @@ struct node *read_node(struct connection *conn, const void *ctx,TDB_DATA key, data;struct xs_tdb_record_hdr *hdr;struct node *node; +const char *db_name;int err; node = talloc(ctx, struct node);
@@ -709,7 +710,8 @@ struct node *read_node(struct connection *conn, const void *ctx,return NULL ; } -transaction_prepend (conn, name, &key); + db_name =transaction_prepend (conn, name); +set_tdb_key (db_name, &key); data =tdb_fetch (tdb_ctx, key);
tools/xenstore/xenstored_transaction.c +4 -7
@@ -196,20 +196,17 @@ static char *transaction_get_node_name(void *ctx, struct transaction *trans,* Prepend the transaction to name if node has been modified in the current * transaction. */ -void transaction_prepend (struct connection *conn,const char *name, -TDB_DATA *key) +const char *transaction_prepend (struct connection *conn,const char *name) {struct accessed_node *i;if (conn && conn->transaction) { i =find_accessed_node (conn->transaction, name); -if (i) { -set_tdb_key (i->trans_name, key); -return ; - } +if (i) +return i->trans_name; } -set_tdb_key (name, key); +return name; }/*
tools/xenstore/xenstored_transaction.h +1 -2
@@ -47,8 +47,7 @@ int __must_check access_node(struct connection *conn, struct node *node,void queue_watches (struct connection *conn,const char *name,bool watch_exact);/* Prepend the transaction to name if appropriate. */ -void transaction_prepend (struct connection *conn,const char *name, -TDB_DATA *key); +const char *transaction_prepend (struct connection *conn,const char *name);/* Mark the transaction as failed. This will prevent it to be committed. */ void fail_transaction (struct transaction *trans);
Signature · 1 line
2.35.3
PATCH v3 4/25 tools/xenstore: rename do_tdb_delete() and change parameter type
In reply to Juergen GrossRename do_tdb_delete() to db_delete() and replace the key parameter with db_name specifying the name of the node in the data base.
This is in preparation to replace TDB with a more simple data storage.
---
V2:
- remove unrelated change (Julien Grall)
--- tools/xenstore/xenstored_core.c | 31 ++++++++++++-------------- tools/xenstore/xenstored_core.h | 5 +++-- tools/xenstore/xenstored_transaction.c | 16 +++++-------- 3 files changed, 23 insertions(+), 29 deletions(-)
tools/xenstore/xenstored_core.c +14 -17
@@ -658,28 +658,31 @@ int do_tdb_write(struct connection *conn, TDB_DATA *key, TDB_DATA *data,return 0 ; } -int do_tdb_delete (struct connection *conn,TDB_DATA *key, -struct node_account_data *acc) +int db_delete (struct connection *conn,const char *name, +struct node_account_data *acc) {struct node_account_data tmp_acc;unsigned int domid; +TDB_DATA key; + +set_tdb_key (name, &key);if (!acc) { acc = &tmp_acc; acc->memory =-1 ; } -get_acc_data (key, acc); +get_acc_data (&key, acc); -if (tdb_delete (tdb_ctx, *key)) { +if (tdb_delete (tdb_ctx, key)) { errno =EIO ;return errno; } -trace_tdb ("delete %s\n" , key->dptr); +trace_tdb ("delete %s\n" , name);if (acc->memory) { - domid =get_acc_domid (conn, key, acc->domid); -domain_memory_add_nochk (conn, domid, -acc->memory - key->dsize); + domid =get_acc_domid (conn, &key, acc->domid); +domain_memory_add_nochk (conn, domid, -acc->memory - key.dsize); }return 0 ;
@@ -1454,13 +1457,10 @@ nomem:static void destroy_node_rm (struct connection *conn,struct node *node) { -TDB_DATA key; -if (streq (node->name,"/" ))corrupt (NULL ,"Destroying root node!" ); -set_tdb_key (node->db_name, &key); -do_tdb_delete (conn, &key, &node->acc); +db_delete (conn, node->db_name, &node->acc); }static int destroy_node (struct connection *conn,struct node *node)
@@ -1651,7 +1651,6 @@ static int delnode_sub(const void *ctx, struct connection *conn,bool watch_exact;int ret;const char *db_name; -TDB_DATA key;/* Any error here will probably be repeated for all following calls. */ ret =access_node (conn, node,NODE_ACCESS_DELETE , &db_name);
@@ -1662,8 +1661,7 @@ static int delnode_sub(const void *ctx, struct connection *conn,return WALK_TREE_ERROR_STOP ;/* In case of error stop the walk. */ -set_tdb_key (db_name, &key); -if (!ret &&do_tdb_delete (conn, &key, &node->acc)) +if (!ret &&db_delete (conn, db_name, &node->acc))return WALK_TREE_ERROR_STOP ;/*
@@ -2489,9 +2487,8 @@ static int clean_store_(TDB_CONTEXT *tdb, TDB_DATA key, TDB_DATA val, }if (!hashtable_search (reachable, name)) {log ("clean_store: '%s' is orphaned!" , name); -if (recovery) { -do_tdb_delete (NULL , &key,NULL ); - } +if (recovery) +db_delete (NULL , name,NULL ); }talloc_free (name);
tools/xenstore/xenstored_core.h +3 -2
@@ -362,12 +362,13 @@ extern xengnttab_handle **xgt_handle;int remember_string (struct hashtable *hash,const char *str); +/* Data base access functions. */ void set_tdb_key (const char *name,TDB_DATA *key);int do_tdb_write (struct connection *conn,TDB_DATA *key,TDB_DATA *data,struct node_account_data *acc,enum write_node_mode mode,bool no_quota_check); -int do_tdb_delete (struct connection *conn,TDB_DATA *key, -struct node_account_data *acc); +int db_delete (struct connection *conn,const char *name, +struct node_account_data *acc);void conn_free_buffered_data (struct connection *conn);
tools/xenstore/xenstored_transaction.c +6 -10
@@ -384,8 +384,7 @@ static int finalize_transaction(struct connection *conn,/* Entries for unmodified nodes can be removed early. */ if (!i->modified) {if (i->ta_node) { -set_tdb_key (i->trans_name, &ta_key); -if (do_tdb_delete (conn, &ta_key,NULL )) +if (db_delete (conn, i->trans_name,NULL ))return EIO ; }list_del (&i->list);
@@ -394,7 +393,6 @@ static int finalize_transaction(struct connection *conn, } while ((i = list_top(&trans->accessed, struct accessed_node, list))) { -set_tdb_key (i->node, &key);if (i->ta_node) {set_tdb_key (i->trans_name, &ta_key); data =tdb_fetch (tdb_ctx, ta_key);
@@ -407,10 +405,11 @@ static int finalize_transaction(struct connection *conn, hdr->generation = ++generation; mode = (i->generation ==NO_GENERATION ) ?NODE_CREATE :NODE_MODIFY ; +set_tdb_key (i->node, &key); *is_corrupt |=do_tdb_write (conn, &key, &data,NULL , mode, true);talloc_free (data.dptr); -if (do_tdb_delete (conn, &ta_key,NULL )) +if (db_delete (conn, i->trans_name,NULL )) *is_corrupt = true; }else { *is_corrupt = true;
@@ -423,7 +422,7 @@ static int finalize_transaction(struct connection *conn,*/ *is_corrupt |= (i->generation ==NO_GENERATION ) ? false - :do_tdb_delete (conn, &key,NULL ); + :db_delete (conn, i->node,NULL ); }if (i->fire_watch) fire_watches(conn, trans, i->node, NULL, i->watch_exact,
@@ -440,15 +439,12 @@ static int destroy_transaction(void *_transaction) {struct transaction *trans = _transaction;struct accessed_node *i; -TDB_DATA key; wrl_ntransactions--;trace_destroy (trans,"transaction" ); while ((i = list_top(&trans->accessed, struct accessed_node, list))) { -if (i->ta_node) { -set_tdb_key (i->trans_name, &key); -do_tdb_delete (trans->conn, &key,NULL ); - } +if (i->ta_node) +db_delete (trans->conn, i->trans_name,NULL );list_del (&i->list);talloc_free (i); }
Signature · 1 line
2.35.3
PATCH v3 5/25 tools/xenstore: rename do_tdb_write() and change parameter type
In reply to Juergen GrossRename do_tdb_write() to db_write() and replace the key parameter with db_name specifying the name of the node in the data base, and the data parameter with a data pointer and a length.
Do the same key parameter type change for write_node_raw(), too.
This is in preparation to replace TDB with a more simple data storage.
---
V2:
- keep ta_key in finalize_transaction() (Julien Grall)
--- tools/xenstore/xenstored_core.c | 70 +++++++++++++------------- tools/xenstore/xenstored_core.h | 11 ++-- tools/xenstore/xenstored_domain.c | 4 +- tools/xenstore/xenstored_transaction.c | 10 ++-- 4 files changed, 46 insertions(+), 49 deletions(-)
tools/xenstore/xenstored_core.c +35 -35
@@ -600,23 +600,27 @@ static unsigned int get_acc_domid(struct connection *conn, TDB_DATA *key, ? domid : conn->id; } -int do_tdb_write (struct connection *conn,TDB_DATA *key,TDB_DATA *data, -struct node_account_data *acc,enum write_node_mode mode, -bool no_quota_check) +int db_write (struct connection *conn,const char *db_name,void *data, +size_t size,struct node_account_data *acc, +enum write_node_mode mode,bool no_quota_check) { -struct xs_tdb_record_hdr *hdr = (void *)data->dptr; +struct xs_tdb_record_hdr *hdr = data;struct node_account_data old_acc = {};unsigned int old_domid, new_domid;int ret; +TDB_DATA key, dat; +set_tdb_key (db_name, &key); + dat.dptr = data; + dat.dsize = size;if (!acc) old_acc.memory =-1 ;else old_acc = *acc; -get_acc_data (key, &old_acc); - old_domid =get_acc_domid (conn, key, old_acc.domid); - new_domid =get_acc_domid (conn, key, hdr->perms[0 ].id); +get_acc_data (&key, &old_acc); + old_domid =get_acc_domid (conn, &key, old_acc.domid); + new_domid =get_acc_domid (conn, &key, hdr->perms[0 ].id);/* * Don't check for ENOENT, as we want to be able to switch orphaned
@@ -624,35 +628,34 @@ int do_tdb_write(struct connection *conn, TDB_DATA *key, TDB_DATA *data,*/ if (old_acc.memory)domain_memory_add_nochk (conn, old_domid, - -old_acc.memory - key->dsize); - ret =domain_memory_add (conn, new_domid, - data->dsize + key->dsize, no_quota_check); + -old_acc.memory - key.dsize); + ret =domain_memory_add (conn, new_domid, size + key.dsize, + no_quota_check);if (ret) {/* Error path, so no quota check. */ if (old_acc.memory)domain_memory_add_nochk (conn, old_domid, - old_acc.memory + key->dsize); + old_acc.memory + key.dsize);return ret; }/* TDB should set errno, but doesn't even set ecode AFAICT. */ -if (tdb_store (tdb_ctx, *key, *data, +if (tdb_store (tdb_ctx, key, dat, (mode ==NODE_CREATE ) ?TDB_INSERT :TDB_MODIFY ) !=0 ) { -domain_memory_add_nochk (conn, new_domid, - -data->dsize - key->dsize); +domain_memory_add_nochk (conn, new_domid, -size - key.dsize);/* Error path, so no quota check. */ if (old_acc.memory)domain_memory_add_nochk (conn, old_domid, - old_acc.memory + key->dsize); + old_acc.memory + key.dsize); errno =EIO ;return errno; } -trace_tdb ("store %s size %zu\n" , key->dptr, data->dsize + key->dsize); +trace_tdb ("store %s size %zu\n" , db_name, size + key.dsize);if (acc) {/* Don't use new_domid, as it might be a transaction node. */ acc->domid = hdr->perms[0 ].id; - acc->memory = data->dsize; + acc->memory = size; }return 0 ;
@@ -780,33 +783,35 @@ static bool read_node_can_propagate_errno(void)return errno ==ENOMEM || errno ==ENOSPC ; } -int write_node_raw (struct connection *conn,TDB_DATA *key,struct node *node, -enum write_node_mode mode,bool no_quota_check) +int write_node_raw (struct connection *conn,const char *db_name, +struct node *node,enum write_node_mode mode, +bool no_quota_check) { -TDB_DATA data; +void *data; +size_t size;void *p;struct xs_tdb_record_hdr *hdr;if (domain_adjust_node_perms (node))return errno; - data.dsize =sizeof (*hdr) + size =sizeof (*hdr) + node->perms.num *sizeof (node->perms.p[0 ]) + node->datalen + node->childlen;/* Call domain_max_chk() in any case in order to record max values. */ -if (domain_max_chk (conn,ACC_NODESZ , data.dsize) && !no_quota_check) { +if (domain_max_chk (conn,ACC_NODESZ , size) && !no_quota_check) { errno =ENOSPC ;return errno; } - data.dptr =talloc_size (node, data.dsize); -if (!data.dptr) { + data =talloc_size (node, size); +if (!data) { errno =ENOMEM ;return errno; } - hdr = (void *)data.dptr; + hdr = data; hdr->generation = node->generation; hdr->num_perms = node->perms.num; hdr->datalen = node->datalen;
@@ -819,7 +824,8 @@ int write_node_raw(struct connection *conn, TDB_DATA *key, struct node *node, p += node->datalen;memcpy (p, node->children, node->childlen); -if (do_tdb_write (conn, key, &data, &node->acc, mode, no_quota_check)) +if (db_write (conn, db_name, data, size, &node->acc, mode, + no_quota_check))return EIO ;return 0 ;
@@ -833,13 +839,11 @@ static int write_node(struct connection *conn, struct node *node, enum write_node_mode mode, bool no_quota_check) { int ret; -TDB_DATA key;if (access_node (conn, node,NODE_ACCESS_WRITE , &node->db_name))return errno; -set_tdb_key (node->db_name, &key); - ret =write_node_raw (conn, &key, node, mode, no_quota_check); + ret =write_node_raw (conn, node->db_name, node, mode, no_quota_check);if (ret && conn && conn->transaction) {/* * Reverting access_node() is hard, so just fail the
@@ -3429,7 +3433,6 @@ void read_state_node(const void *ctx, const void *state) {const struct xs_state_node *sn = state;struct node *node, *parent; -TDB_DATA key;char *name, *parentname;unsigned int i;struct connection conn = { .id = priv_domid };
@@ -3482,15 +3485,12 @@ void read_state_node(const void *ctx, const void *state)if (add_child (node, parent, name))barf ("allocation error restoring node" ); -set_tdb_key (parentname, &key); -if (write_node_raw (NULL , &key, parent,NODE_MODIFY , true)) +if (write_node_raw (NULL , parentname, parent,NODE_MODIFY , true))barf ("write parent error restoring node" ); } -set_tdb_key (name, &key); -/* The "/" node is already existing, so it can only be modified here. */ -if (write_node_raw (NULL , &key, node, +if (write_node_raw (NULL , name, node,strcmp (name,"/" ) ?NODE_CREATE :NODE_MODIFY , true))barf ("write node error restoring node" );
tools/xenstore/xenstored_core.h +6 -5
@@ -243,8 +243,9 @@ enum write_node_mode {NODE_MODIFY }; -int write_node_raw (struct connection *conn,TDB_DATA *key,struct node *node, -enum write_node_mode mode,bool no_quota_check); +int write_node_raw (struct connection *conn,const char *db_name, +struct node *node,enum write_node_mode mode, +bool no_quota_check);/* Get a node from the tdb data base. */ struct node *read_node(struct connection *conn, const void *ctx,
@@ -364,9 +365,9 @@ int remember_string(struct hashtable *hash, const char *str);/* Data base access functions. */ void set_tdb_key (const char *name,TDB_DATA *key); -int do_tdb_write (struct connection *conn,TDB_DATA *key,TDB_DATA *data, -struct node_account_data *acc,enum write_node_mode mode, -bool no_quota_check); +int db_write (struct connection *conn,const char *db_name,void *data, +size_t size,struct node_account_data *acc, +enum write_node_mode mode,bool no_quota_check);int db_delete (struct connection *conn,const char *name,struct node_account_data *acc);
tools/xenstore/xenstored_domain.c +1 -3
@@ -511,19 +511,17 @@ static int domain_tree_remove_sub(const void *ctx, struct connection *conn, struct node *node, void *arg) { struct domain *domain = arg; -TDB_DATA key;int ret =WALK_TREE_OK ;if (node->perms.p[0 ].id != domain->domid)return WALK_TREE_OK ;if (keep_orphans) { -set_tdb_key (node->name, &key);domain_nbentry_dec (NULL , domain->domid); node->perms.p[0 ].id = priv_domid; node->acc.memory =0 ;domain_nbentry_inc (NULL , priv_domid); -if (write_node_raw (NULL , &key, node,NODE_MODIFY , true)) { +if (write_node_raw (NULL , node->name, node,NODE_MODIFY , true)) {/* That's unfortunate. We only can try to continue. */ syslog(LOG_ERR, "error when moving orphaned node %s to dom0\n",
tools/xenstore/xenstored_transaction.c +4 -6
@@ -228,7 +228,6 @@ int access_node(struct connection *conn, struct node *node, {struct accessed_node *i =NULL ;struct transaction *trans; -TDB_DATA local_key;int ret;bool introduce = false;
@@ -286,8 +285,7 @@ int access_node(struct connection *conn, struct node *node, i->generation = node->generation; i->check_gen = true;if (node->generation !=NO_GENERATION ) { -set_tdb_key (i->trans_name, &local_key); - ret =write_node_raw (conn, &local_key, node, + ret =write_node_raw (conn, i->trans_name, node,NODE_CREATE , true);if (ret) goto err;
@@ -405,9 +403,9 @@ static int finalize_transaction(struct connection *conn, hdr->generation = ++generation; mode = (i->generation ==NO_GENERATION ) ?NODE_CREATE :NODE_MODIFY ; -set_tdb_key (i->node, &key); - *is_corrupt |=do_tdb_write (conn, &key, &data, -NULL , mode, true); + *is_corrupt |=db_write (conn, i->node, + data.dptr, data.dsize, +NULL , mode, true);talloc_free (data.dptr);if (db_delete (conn, i->trans_name,NULL )) *is_corrupt = true;
Signature · 1 line
2.35.3
PATCH v3 6/25 tools/xenstore: switch get_acc_data() to use name instead of key
In reply to Juergen GrossEliminate further TDB_DATA usage by switching get_acc_data() and get_acc_domid() from a TDB key to the name of the node in the data base as a parameter.
This is in preparation to replace TDB with a more simple data storage.
--- tools/xenstore/xenstored_core.c | 26 +++++++++++++------------- 1 file changed, 13 insertions(+), 13 deletions(-)
tools/xenstore/xenstored_core.c +13 -13
@@ -566,19 +566,20 @@ void set_tdb_key(const char *name, TDB_DATA *key) key->dsize =strlen (name); } -static void get_acc_data (TDB_DATA *key,struct node_account_data *acc) +static void get_acc_data (const char *name,struct node_account_data *acc) { -TDB_DATA old_data; +TDB_DATA key, old_data;struct xs_tdb_record_hdr *hdr;if (acc->memory <0 ) { - old_data =tdb_fetch (tdb_ctx, *key); +set_tdb_key (name, &key); + old_data =tdb_fetch (tdb_ctx, key);/* No check for error, as the node might not exist. */ if (old_data.dptr ==NULL ) { acc->memory =0 ; }else { -trace_tdb ("read %s size %zu\n" , key->dptr, - old_data.dsize + key->dsize); +trace_tdb ("read %s size %zu\n" , name, + old_data.dsize + key.dsize); hdr = (void *)old_data.dptr; acc->memory = old_data.dsize; acc->domid = hdr->perms[0 ].id;
@@ -593,11 +594,10 @@ static void get_acc_data(TDB_DATA *key, struct node_account_data *acc)* count prepended (e.g. 123/local/domain/...). So testing for the node's * key not to start with "/" or "@" is sufficient. */ -static unsigned int get_acc_domid (struct connection *conn,TDB_DATA *key, +static unsigned int get_acc_domid (struct connection *conn,const char *name,unsigned int domid) { -return (!conn || key->dptr[0 ] =='/' || key->dptr[0 ] =='@' ) - ? domid : conn->id; +return (!conn || name[0 ] =='/' || name[0 ] =='@' ) ? domid : conn->id; } int db_write(struct connection *conn, const char *db_name, void *data,
@@ -618,9 +618,9 @@ int db_write(struct connection *conn, const char *db_name, void *data,else old_acc = *acc; -get_acc_data (&key, &old_acc); - old_domid =get_acc_domid (conn, &key, old_acc.domid); - new_domid =get_acc_domid (conn, &key, hdr->perms[0 ].id); +get_acc_data (db_name, &old_acc); + old_domid =get_acc_domid (conn, db_name, old_acc.domid); + new_domid =get_acc_domid (conn, db_name, hdr->perms[0 ].id);/* * Don't check for ENOENT, as we want to be able to switch orphaned
@@ -675,7 +675,7 @@ int db_delete(struct connection *conn, const char *name, acc->memory =-1 ; } -get_acc_data (&key, acc); +get_acc_data (name, acc);if (tdb_delete (tdb_ctx, key)) { errno =EIO ;
@@ -684,7 +684,7 @@ int db_delete(struct connection *conn, const char *name,trace_tdb ("delete %s\n" , name);if (acc->memory) { - domid =get_acc_domid (conn, &key, acc->domid); + domid =get_acc_domid (conn, name, acc->domid);domain_memory_add_nochk (conn, domid, -acc->memory - key.dsize); }
Signature · 1 line
2.35.3
PATCH v3 7/25 tools/xenstore: add wrapper for tdb_fetch()
In reply to Juergen GrossAdd a wrapper function for tdb_fetch taking the name of the node in the data base as a parameter. Let it return a data pointer and the length of the data via a length pointer provided as additional parameter.
Move logging of the TDB access from the callers into the wrapper.
This enables to make set_tdb_key() and tdb_ctx static.
This is in preparation to replace TDB with a more simple data storage.
---
V2:
- always set *size in db_fetch() (Julien Grall)
- change db_fetch() return type (Julien Grall)
- move logging
--- tools/xenstore/xenstored_core.c | 55 +++++++++++++++----------- tools/xenstore/xenstored_core.h | 3 +- tools/xenstore/xenstored_transaction.c | 31 ++++++--------- 3 files changed, 44 insertions(+), 45 deletions(-)
tools/xenstore/xenstored_core.c +32 -23
@@ -85,7 +85,7 @@ bool keep_orphans = false;static int reopen_log_pipe[2 ];static int reopen_log_pipe0_pollfd_idx =-1 ;char *tracefile =NULL ; -TDB_CONTEXT *tdb_ctx =NULL ; +static TDB_CONTEXT *tdb_ctx =NULL ;unsigned int trace_flags =TRACE_OBJ |TRACE_IO ;static const char *sockmsg_string (enum xsd_sockmsg_type type);
@@ -556,7 +556,7 @@ static void initialize_fds(int *p_sock_pollfd_idx, int *ptimeout) } } -void set_tdb_key (const char *name,TDB_DATA *key) +static void set_tdb_key (const char *name,TDB_DATA *key) {/* * Dropping const is fine here, as the key will never be modified
@@ -566,25 +566,39 @@ void set_tdb_key(const char *name, TDB_DATA *key) key->dsize =strlen (name); } +struct xs_tdb_record_hdr *db_fetch (const char *db_name,size_t *size) +{ +TDB_DATA key, data; + +set_tdb_key (db_name, &key); + data =tdb_fetch (tdb_ctx, key); +if (!data.dptr) { + errno = (tdb_error (tdb_ctx) ==TDB_ERR_NOEXIST ) ?ENOENT :EIO ; + *size =0 ; + }else { + *size = data.dsize; +trace_tdb ("read %s size %zu\n" , db_name, + *size +strlen (db_name)); + } + +return (struct xs_tdb_record_hdr *)data.dptr; +} +static void get_acc_data (const char *name,struct node_account_data *acc) { -TDB_DATA key, old_data; +size_t size;struct xs_tdb_record_hdr *hdr;if (acc->memory <0 ) { -set_tdb_key (name, &key); - old_data =tdb_fetch (tdb_ctx, key); + hdr =db_fetch (name, &size);/* No check for error, as the node might not exist. */ -if (old_data.dptr ==NULL ) { +if (hdr ==NULL ) { acc->memory =0 ; }else { -trace_tdb ("read %s size %zu\n" , name, - old_data.dsize + key.dsize); - hdr = (void *)old_data.dptr; - acc->memory = old_data.dsize; + acc->memory = size; acc->domid = hdr->perms[0 ].id; } -talloc_free (old_data.dptr); +talloc_free (hdr); } }
@@ -698,7 +712,7 @@ int db_delete(struct connection *conn, const char *name,struct node *read_node (struct connection *conn,const void *ctx,const char *name) { -TDB_DATA key, data; +size_t size;struct xs_tdb_record_hdr *hdr;struct node *node;const char *db_name;
@@ -717,29 +731,24 @@ struct node *read_node(struct connection *conn, const void *ctx, } db_name =transaction_prepend (conn, name); -set_tdb_key (db_name, &key); + hdr =db_fetch (db_name, &size); - data =tdb_fetch (tdb_ctx, key); - -if (data.dptr ==NULL ) { -if (tdb_error (tdb_ctx) ==TDB_ERR_NOEXIST ) { +if (hdr ==NULL ) { +if (errno ==ENOENT ) { node->generation =NO_GENERATION ; err =access_node (conn, node,NODE_ACCESS_READ ,NULL ); errno = err ? :ENOENT ; }else { -log ("TDB error on read: %s" ,tdb_errorstr (tdb_ctx)); +log ("DB error on read: %s" ,strerror (errno)); errno =EIO ; } goto error; } -trace_tdb ("read %s size %zu\n" , key.dptr, data.dsize + key.dsize); - node->parent =NULL ; -talloc_steal (node, data.dptr); +talloc_steal (node, hdr);/* Datalen, childlen, number of permissions */ - hdr = (void *)data.dptr; node->generation = hdr->generation; node->perms.num = hdr->num_perms; node->datalen = hdr->datalen;
@@ -748,7 +757,7 @@ struct node *read_node(struct connection *conn, const void *ctx,/* Permissions are struct xs_permissions. */ node->perms.p = hdr->perms; node->acc.domid =get_node_owner (node); - node->acc.memory = data.dsize; + node->acc.memory = size;if (domain_adjust_node_perms (node)) goto error;
tools/xenstore/xenstored_core.h +1 -2
@@ -315,7 +315,6 @@ do { \trace ("tdb: " __VA_ARGS__); \ }while (0 ) -extern TDB_CONTEXT *tdb_ctx; extern int dom0_domid;extern int dom0_event;extern int priv_domid;
@@ -364,7 +363,7 @@ extern xengnttab_handle **xgt_handle;int remember_string (struct hashtable *hash,const char *str);/* Data base access functions. */ -void set_tdb_key (const char *name,TDB_DATA *key); +struct xs_tdb_record_hdr *db_fetch (const char *db_name,size_t *size);int db_write (struct connection *conn,const char *db_name,void *data,size_t size,struct node_account_data *acc,enum write_node_mode mode,bool no_quota_check);
tools/xenstore/xenstored_transaction.c +11 -20
@@ -356,25 +356,21 @@ static int finalize_transaction(struct connection *conn, struct transaction *trans, bool *is_corrupt) { struct accessed_node *i, *n; -TDB_DATA key, ta_key, data; +size_t size;struct xs_tdb_record_hdr *hdr;uint64_t gen;list_for_each_entry_safe (i, n, &trans->accessed, list) {if (i->check_gen) { -set_tdb_key (i->node, &key); - data =tdb_fetch (tdb_ctx, key); - hdr = (void *)data.dptr; -if (!data.dptr) { -if (tdb_error (tdb_ctx) !=TDB_ERR_NOEXIST ) -return EIO ; + hdr =db_fetch (i->node, &size); +if (!hdr) { +if (errno !=ENOENT ) +return errno; gen =NO_GENERATION ; }else { -trace_tdb ("read %s size %zu\n" , key.dptr, - key.dsize + data.dsize); gen = hdr->generation; } -talloc_free (data.dptr); +talloc_free (hdr);if (i->generation != gen)return EAGAIN ; }
@@ -392,21 +388,16 @@ static int finalize_transaction(struct connection *conn, while ((i = list_top(&trans->accessed, struct accessed_node, list))) {if (i->ta_node) { -set_tdb_key (i->trans_name, &ta_key); - data =tdb_fetch (tdb_ctx, ta_key); -if (data.dptr) { + hdr =db_fetch (i->trans_name, &size); +if (hdr) {enum write_node_mode mode; -trace_tdb ("read %s size %zu\n" , ta_key.dptr, - ta_key.dsize + data.dsize); - hdr = (void *)data.dptr; hdr->generation = ++generation; mode = (i->generation ==NO_GENERATION ) ?NODE_CREATE :NODE_MODIFY ; - *is_corrupt |=db_write (conn, i->node, - data.dptr, data.dsize, -NULL , mode, true); -talloc_free (data.dptr); + *is_corrupt |=db_write (conn, i->node, hdr, + size,NULL , mode, true); +talloc_free (hdr);if (db_delete (conn, i->trans_name,NULL )) *is_corrupt = true; }else {
Signature · 1 line
2.35.3
PATCH v3 8/25 tools/xenstore: make hashtable key and value parameters const
In reply to Juergen GrossThe key and value are never modified by hashtable code, so they should be marked as const.
---
V3:
- make value const, too.
--- tools/xenstore/hashtable.c | 7 ++++--- tools/xenstore/hashtable.h | 4 ++-- 2 files changed, 6 insertions(+), 5 deletions(-)
tools/xenstore/hashtable.c +4 -3
@@ -11,7 +11,8 @@struct entry { -void *k, *v; +const void *k; +void *v;unsigned int h;struct entry *next; };
@@ -140,7 +141,7 @@ static int hashtable_expand(struct hashtable *h)return 0 ; } -int hashtable_add (struct hashtable *h,void *k,void *v) +int hashtable_add (struct hashtable *h,const void *k,const void *v) {/* This method allows duplicate keys - but they shouldn't be used */ unsigned int index;
@@ -164,7 +165,7 @@ int hashtable_add(struct hashtable *h, void *k, void *v) e->k = k;if (h->flags &HASHTABLE_FREE_KEY )talloc_steal (e, k); - e->v = v; + e->v = (void *)v;if (h->flags &HASHTABLE_FREE_VALUE )talloc_steal (e, v); e->next = h->table[index];
tools/xenstore/hashtable.h +2 -2
@@ -48,8 +48,8 @@ create_hashtable(const void *ctx, const char *name,* If in doubt, remove before insert. */ -int -hashtable_add (struct hashtable *h,void *k,void *v); +int +hashtable_add (struct hashtable *h,const void *k,const void *v);/***************************************************************************** * hashtable_search
Signature · 1 line
2.35.3
Re: [PATCH v3 08/25] tools/xenstore: make hashtable key and value parameters const
Hi,
Quoted 2 lines · Juergen Gross
The key and value are never modified by hashtable code, so they should be marked as const.
You wrote this but...
Quoted 15 lines
Signed-off-by Juergen Gross <[email protected]>---
V3:
- make value const, too.--- tools/xenstore/hashtable.c | 7 ++++--- tools/xenstore/hashtable.h | 4 ++-- 2 files changed, 6 insertions(+), 5 deletions(-)tools/xenstore/hashtable.c +0
-0
tools/xenstore/hashtable.c line 15
Quoted 6 lines
tools/xenstore/hashtable.c +2
-1@@ -11,7 +11,8 @@struct entry { -void *k, *v; +const void *k; +void *v;
... this is not const and ...
Quoted 13 lines
tools/xenstore/hashtable.c +1
-1unsigned int h;struct entry *next; };@@ -140,7 +141,7 @@ static int hashtable_expand(struct hashtable *h)return 0 ; } -int hashtable_add (struct hashtable *h,void *k,void *v) +int hashtable_add (struct hashtable *h,const void *k,const void *v) {/* This method allows duplicate keys - but they shouldn't be used */ unsigned int index;
tools/xenstore/hashtable.c int hashtable_add(struct hashtable *h, void *k, void *v) · line 168
Quoted 5 lines
tools/xenstore/hashtable.c +1
-1@@ -164,7 +165,7 @@ int hashtable_add(struct hashtable *h, void *k, void *v) e->k = k;if (h->flags &HASHTABLE_FREE_KEY )talloc_steal (e, k); - e->v = v; + e->v = (void *)v;
... you cast-away the const here. I think this is a pretty bad idea.
Can you clarify why you are doing like that?
Cheers,
Signature · 1 line
Julien Grall
Re: [PATCH v3 08/25] tools/xenstore: make hashtable key and value parameters const
Quoted 52 lines · Julien Grall
Hi,
Quoted 2 lines · Juergen Gross
The key and value are never modified by hashtable code, so they should be marked as const.
You wrote this but...
Quoted 15 lines
Signed-off-by Juergen Gross <[email protected]>---
V3:
- make value const, too.
---
tools/xenstore/hashtable.c| 7 ++++---
tools/xenstore/hashtable.h| 4 ++--2 files changed, 6 insertions(+), 5 deletions(-)tools/xenstore/hashtable.c +0
-0@@ -11,7 +11,8 @@Quoted 5 lines
struct entry
{
- void *k, *v;
+ const void *k;
+ void *v;... this is not const and ...
Quoted 12 lines
unsigned int h;
struct entry *next;
};tools/xenstore/hashtable.c +1
-1@@ -140,7 +141,7 @@ static int hashtable_expand(struct hashtable *h) return 0; } -int hashtable_add (struct hashtable *h,void *k,void *v) +int hashtable_add (struct hashtable *h,const void *k,const void *v) { /* This method allows duplicate keys - but they shouldn't be used */ unsigned int index;tools/xenstore/hashtable.c int hashtable_add(struct hashtable *h, void *k, void *v) · line 168
Quoted 5 lines
tools/xenstore/hashtable.c +1
-1@@ -164,7 +165,7 @@ int hashtable_add(struct hashtable *h, void *k, void *v) e->k = k; if (h->flags & HASHTABLE_FREE_KEY) talloc_steal(e, k); - e->v = v; + e->v = (void *)v;... you cast-away the const here. I think this is a pretty bad idea.
Quoted 1 line
Can you clarify why you are doing like that?
The value is never changed by the hashtable code, but it might be changed by e.g. a caller of hashtable_search() (see e.g. callers of find_domain_struct()).
Somewhere I need to cast the const away. I could do so in hashtable_search() in case you prefer me to do so.
Juergen
Re: [PATCH v3 08/25] tools/xenstore: make hashtable key and value parameters const
Hi Juergen,
Quoted 54 lines · Juergen Gross
Quoted 53 lines · Julien Grall
Hi,
Quoted 2 lines · Juergen Gross
The key and value are never modified by hashtable code, so they should be marked as const.
You wrote this but...
Quoted 15 lines
Signed-off-by Juergen Gross <[email protected]>---
V3:
- make value const, too.
---
tools/xenstore/hashtable.c| 7 ++++---
tools/xenstore/hashtable.h| 4 ++--2 files changed, 6 insertions(+), 5 deletions(-)tools/xenstore/hashtable.c +0
-0@@ -11,7 +11,8 @@Quoted 5 lines
struct entry
{
- void *k, *v;
+ const void *k;
+ void *v;... this is not const and ...
Quoted 12 lines
unsigned int h;
struct entry *next;
};tools/xenstore/hashtable.c +1
-1@@ -140,7 +141,7 @@ static int hashtable_expand(struct hashtable *h) return 0; } -int hashtable_add (struct hashtable *h,void *k,void *v) +int hashtable_add (struct hashtable *h,const void *k,const void *v) { /* This method allows duplicate keys - but they shouldn't be used */ unsigned int index;tools/xenstore/hashtable.c int hashtable_add(struct hashtable *h, void *k, · line 169
Quoted 6 lines
tools/xenstore/hashtable.c +1
-1@@ -164,7 +165,7 @@ int hashtable_add(struct hashtable *h, void *k, void *v) e->k = k; if (h->flags & HASHTABLE_FREE_KEY) talloc_steal(e, k); - e->v = v; + e->v = (void *)v;... you cast-away the const here. I think this is a pretty bad idea.
Quoted 7 lines
Quoted 1 line
Can you clarify why you are doing like that?
The value is never changed by the hashtable code, but it might be changed by e.g. a caller of
hashtable_search()(see e.g. callers offind_domain_struct()).Somewhere I need to cast the const away. I could do so in
hashtable_search()in case you prefer me to do so.
My problem is not with the placement of the const but the fact you are removing the const.
I agree that the hashtable code is not meant to modify the content. However, as you wrote, the caller of hashtable_search() could modify the content. So, for me, the value should not be const in the hashtable code.
To give a concrete example, with the current interface we are telling the user that what they store in the hashtable can be modified at some point. By adding 'const' for the value in hashtable_add(), we can mislead a user to think it is fine to store static string, yet this is not enforced all the way through. So one could mistakenly think that values returned hashtable_search() can be modified. And the compiler will not be here to help enforcing it because you cast-away the const.
Do you have any code in this series that requires the 'const' in hashtable_add()? If so, can you point me to the patch and I will have a look?
If not, then I will strongly argue that this should be dropped because dropping a const is always a recipe for disaster.
Cheers,
Signature · 1 line
Julien Grall
Re: [PATCH v3 08/25] tools/xenstore: make hashtable key and value parameters const
Quoted 61 lines · Julien Grall
Hi Juergen,
Quoted 58 lines · Juergen Gross
Quoted 52 lines · Julien Grall
Hi,
Quoted 2 lines · Juergen Gross
The key and value are never modified by hashtable code, so they should be marked as const.
You wrote this but...
Quoted 15 lines
Signed-off-by Juergen Gross <[email protected]>---
V3:
- make value const, too.
---
tools/xenstore/hashtable.c| 7 ++++---
tools/xenstore/hashtable.h| 4 ++--2 files changed, 6 insertions(+), 5 deletions(-)tools/xenstore/hashtable.c +0
-0@@ -11,7 +11,8 @@Quoted 5 lines
struct entry
{
- void *k, *v;
+ const void *k;
+ void *v;... this is not const and ...
Quoted 12 lines
unsigned int h;
struct entry *next;
};tools/xenstore/hashtable.c +1
-1@@ -140,7 +141,7 @@ static int hashtable_expand(struct hashtable *h) return 0; } -int hashtable_add (struct hashtable *h,void *k,void *v) +int hashtable_add (struct hashtable *h,const void *k,const void *v) { /* This method allows duplicate keys - but they shouldn't be used */ unsigned int index;tools/xenstore/hashtable.c int hashtable_add(struct hashtable *h, void *k, void *v) · line 168
Quoted 5 lines
tools/xenstore/hashtable.c +1
-1@@ -164,7 +165,7 @@ int hashtable_add(struct hashtable *h, void *k, void *v) e->k = k; if (h->flags & HASHTABLE_FREE_KEY) talloc_steal(e, k); - e->v = v; + e->v = (void *)v;... you cast-away the const here. I think this is a pretty bad idea.
Quoted 1 line
Can you clarify why you are doing like that?
The value is never changed by the hashtable code, but it might be changed by e.g. a caller of
hashtable_search()(see e.g. callers offind_domain_struct()).
Quoted 6 lines
Quoted 2 lines
Somewhere I need to cast the const away. I could do so in
hashtable_search()in case you prefer me to do so.My problem is not with the placement of the const but the fact you are removing the const.
I agree that the hashtable code is not meant to modify the content. However, as you wrote, the caller of
hashtable_search()could modify the content. So, for me, the value should not be const in the hashtable code.
This is arguable IMHO.
Quoted 1 line
To give a concrete example, with the current interface we are telling the user that what they store in the hashtable can be modified at some point. By adding 'const' for the value in
hashtable_add(), we can mislead a user to think it is fine to store static string, yet this is not enforced all the way through. So one could mistakenly think that values returnedhashtable_search()can be modified. And the compiler will not be here to help enforcing it because you cast-away the const.
Yes, like in the case of strstr().
It takes two const char * parameters and it is returning char *, even with it pointing into the first parameter.
Quoted 1 line
Do you have any code in this series that requires the 'const' in
hashtable_add()? If so, can you point me to the patch and I will have a look?
I had it when writing this patch, but this requirement is gone now. But please note that this means to drop the const from db_write(), too.
Quoted 1 line
If not, then I will strongly argue that this should be dropped because dropping a const is always a recipe for disaster.
Depends IMO.
I believe it is better as I've done it, but in case you insist on it I can drop the patch.
An alternative would be make hashtable_search() return a const and only cast the const away where it is really needed (and probably with a prominent comment at the related hashtable_add() place). I think this will hit xenstored_domain.c use cases only.
Juergen
Re: [PATCH v3 08/25] tools/xenstore: make hashtable key and value parameters const
Hi,
Quoted 7 lines · Juergen Gross
Quoted 1 line · Julien Grall
To give a concrete example, with the current interface we are telling the user that what they store in the hashtable can be modified at some point. By adding 'const' for the value in
hashtable_add(), we can mislead a user to think it is fine to store static string, yet this is not enforced all the way through. So one could mistakenly think that values returnedhashtable_search()can be modified. And the compiler will not be here to help enforcing it because you cast-away the const.Yes, like in the case of
strstr().It takes two const char * parameters and it is returning char *, even with it pointing into the first parameter.
This is a pretty good example on how to not write an interface. :)
Quoted 6 lines
Quoted 2 lines
Do you have any code in this series that requires the 'const' in
hashtable_add()? If so, can you point me to the patch and I will have a look?I had it when writing this patch, but this requirement is gone now. But please note that this means to drop the const from
db_write(), too.
Quoted 7 lines
Quoted 1 line
If not, then I will strongly argue that this should be dropped because dropping a const is always a recipe for disaster.
Depends IMO.
I believe it is better as I've done it,
but in case you insist on it I can drop
the patch.
Well... I can always be swayed if there is a good argument to make it const. So far, you mention that hashtable doesn't modify the content but you don't really explain why waiving away the help from the compiler is ok. Therefore, to me it seems the downside is bigger than the benefit.
Also, I am not asking to drop the patch. The const on the key is ok. I am only requesting to remove the const on the value.
Quoted 5 lines
An alternative would be make
hashtable_search()return a const and only cast the const away where it is really needed (and probably with a prominent comment at the relatedhashtable_add()place). I think this will hitxenstored_domain.cuse cases only.
Again, this still means we are casting away the const somewhere. This is the part I am against if there is no strong justification for it (i.e. there is no other way to do it).
Cheers,
Signature · 1 line
Julien Grall
Re: [PATCH v3 08/25] tools/xenstore: make hashtable key and value parameters const
Quoted 32 lines · Julien Grall
Hi,
Quoted 7 lines · Juergen Gross
Quoted 1 line · Julien Grall
To give a concrete example, with the current interface we are telling the user that what they store in the hashtable can be modified at some point. By adding 'const' for the value in
hashtable_add(), we can mislead a user to think it is fine to store static string, yet this is not enforced all the way through. So one could mistakenly think that values returnedhashtable_search()can be modified. And the compiler will not be here to help enforcing it because you cast-away the const.Yes, like in the case of
strstr().It takes two const char * parameters and it is returning char *, even with it pointing into the first parameter.
This is a pretty good example on how to not write an interface. :)
Quoted 6 lines
Quoted 2 lines
Do you have any code in this series that requires the 'const' in
hashtable_add()? If so, can you point me to the patch and I will have a look?I had it when writing this patch, but this requirement is gone now. But please note that this means to drop the const from
db_write(), too.Quoted 7 lines
Quoted 1 line
If not, then I will strongly argue that this should be dropped because dropping a const is always a recipe for disaster.
Depends IMO.
I believe it is better as I've done it,
but in case you insist on it I can drop
the patch.Well... I can always be swayed if there is a good argument to make it const. So far, you mention that hashtable doesn't modify the content but you don't really explain why waiving away the help from the compiler is ok. Therefore, to me it seems the downside is bigger than the benefit.
Also, I am not asking to drop the patch. The const on the key is ok. I am only requesting to remove the const on the value.
Quoted 1 line
Quoted 6 lines
Quoted 4 lines
An alternative would be make
hashtable_search()return a const and only cast the const away where it is really needed (and probably with a prominent comment at the relatedhashtable_add()place). I think this will hitxenstored_domain.cuse cases only.Again, this still means we are casting away the const somewhere. This is the part I am against if there is no strong justification for it (i.e. there is no other way to do it).
Okay, I'll drop the const attribute for the value parameter.
Juergen
PATCH v3 9/25 tools/xenstore: let hashtable_add() fail in case of existing entry
In reply to Juergen GrossSilently adding another entry with the same key to a hashtable is a perfect receipt for later failure with hard to diagnose symptoms.
Let hashtable_add() fail in case another entry with the same key is already existing.
---
V2:
- split off from next patch (Julien Grall)
- fix coding style (Julien Grall)
- use for () loop (Julien Grall)
--- tools/xenstore/hashtable.c | 40 ++++++++++++++++++++++++++------------ 1 file changed, 28 insertions(+), 12 deletions(-)
tools/xenstore/hashtable.c +28 -12
@@ -141,11 +141,34 @@ static int hashtable_expand(struct hashtable *h)return 0 ; } +static struct entry *hashtable_search_entry (const struct hashtable *h, +const void *k) +{ +struct entry *e; +unsigned int hashvalue, index; + + hashvalue =hash (h, k); + index =indexFor (h->tablelength, hashvalue); + e = h->table[index]; + +for (e = h->table[index]; e; e = e->next) + { +/* Check hash value to short circuit heavier comparison */ +if ((hashvalue == e->h) && (h->eqfn (k, e->k))) +return e; + } + +return NULL ; +} +int hashtable_add (struct hashtable *h,const void *k,const void *v) { -/* This method allows duplicate keys - but they shouldn't be used */ unsigned int index;struct entry *e; + +if (hashtable_search_entry (h, k)) +return EEXIST ; +if (++(h->entrycount) > h->loadlimit) {/* Ignore the return value. If expand fails, we should
@@ -176,17 +199,10 @@ int hashtable_add(struct hashtable *h, const void *k, const void *v)void *hashtable_search (const struct hashtable *h,const void *k) {struct entry *e; -unsigned int hashvalue, index; - hashvalue =hash (h,k); - index =indexFor (h->tablelength,hashvalue); - e = h->table[index]; -while (NULL != e) - { -/* Check hash value to short circuit heavier comparison */ -if ((hashvalue == e->h) && (h->eqfn (k, e->k)))return e->v; - e = e->next; - } -return NULL ; + + e =hashtable_search_entry (h, k); + +return e ? e->v :NULL ; }void
Signature · 1 line
2.35.3
PATCH v3 10/25 tools/xenstore: add hashtable_replace() function
In reply to Juergen GrossFor an effective way to replace a hashtable entry add a new function hashtable_replace().
This is in preparation to replace TDB with a more simple data storage.
---
V3:
- fix commit message (Julien Grall)
- move unrelated change to previous patch (Julien Grall)
- make value parameter const
--- tools/xenstore/hashtable.c | 19 +++++++++++++++++++ tools/xenstore/hashtable.h | 16 ++++++++++++++++ 2 files changed, 35 insertions(+)
tools/xenstore/hashtable.c +19 -0
@@ -205,6 +205,25 @@ void *hashtable_search(const struct hashtable *h, const void *k)return e ? e->v :NULL ; } +int hashtable_replace (struct hashtable *h,const void *k,const void *v) +{ +struct entry *e; + + e =hashtable_search_entry (h, k); +if (!e) +return ENOENT ; + +if (h->flags &HASHTABLE_FREE_VALUE ) + { +talloc_free (e->v); +talloc_steal (e, v); + } + + e->v = (void *)v; + +return 0 ; +} +void hashtable_remove (struct hashtable *h,const void *k) {
tools/xenstore/hashtable.h +16 -0
@@ -51,6 +51,22 @@ create_hashtable(const void *ctx, const char *name,int hashtable_add (struct hashtable *h,const void *k,const void *v); +/***************************************************************************** +* hashtable_replace + +* @name hashtable_nsert +* @param h the hashtable to insert into +* @param k the key - hashtable claims ownership and will free on removal +* @param v the value - does not claim ownership +* @return zero for successful insertion +* +* This function does check for an entry being present before replacing it +* with a new value. +*/ + +int +hashtable_replace (struct hashtable *h,const void *k,const void *v); +/***************************************************************************** * hashtable_search
Signature · 1 line
2.35.3
Re: [PATCH v3 10/25] tools/xenstore: add hashtable_replace() function
Hi Juergen,
Quoted 21 lines · Juergen Gross
For an effective way to replace a hashtable entry add a new function
hashtable_replace().This is in preparation to replace TDB with a more simple data storage.
Signed-off-by Juergen Gross <[email protected]>---
V3:
- fix commit message (Julien Grall)
- move unrelated change to previous patch (Julien Grall)
- make value parameter const--- tools/xenstore/hashtable.c | 19 +++++++++++++++++++ tools/xenstore/hashtable.h | 16 ++++++++++++++++ 2 files changed, 35 insertions(+)tools/xenstore/hashtable.c +0
-0
tools/xenstore/hashtable.c void *hashtable_search(const struct hashtable *h, const void *k) · line 208
Quoted 4 lines
tools/xenstore/hashtable.c +1
-0@@ -205,6 +205,25 @@ void *hashtable_search(const struct hashtable *h, const void *k)return e ? e->v :NULL ; } +int hashtable_replace (struct hashtable *h,const void *k,const void *v)
With the const dropped for 'v' and ...
Quoted 4 lines
tools/xenstore/hashtable.c +4
-0+{ +struct entry *e; + + e =hashtable_search_entry (h, k);
tools/xenstore/hashtable.c void *hashtable_search(const struct hashtable *h, const void *k) · line 222
Quoted 10 lines
tools/xenstore/hashtable.c +10
-0+if (!e) +return ENOENT ; + +if (h->flags &HASHTABLE_FREE_VALUE ) + { +talloc_free (e->v); +talloc_steal (e, v); + } + + e->v = (void *)v;
... cast:
Cheers,
Signature · 1 line
Julien Grall
PATCH v3 11/25 tools/xenstore: drop use of tdb
In reply to Juergen GrossToday all Xenstore nodes are stored in a TDB data base. This data base has several disadvantages:
- It is using a fixed sized hash table, resulting in high memory overhead for small installations with only very few VMs, and a rather large performance hit for systems with lots of VMs due to many collisions.
The hash table size today is 7919 entries. This means that e.g. in case of a simple desktop use case with 2 or 3 VMs probably far less than 10% of the entries will be used (assuming roughly 100 nodes per VM). OTOH a setup on a large server with 500 VMs would result in heavy conflicts in the hash list with 5-10 nodes per hash table entry.
- TDB is using a single large memory area for storing the nodes. It only ever increases this area and will never shrink it afterwards. This will result in more memory usage than necessary after a peak of Xenstore usage.
- Xenstore is only single-threaded, while TDB is designed to be fit for multi-threaded use cases, resulting in much higher code complexity than needed.
- Special use cases of Xenstore are not possible to implement with TDB in an effective way, while an implementation of a data base tailored for Xenstore could simplify some handling (e.g. transactions) a lot.
So drop using TDB and store the nodes directly in memory making them easily accessible. Use a hash-based lookup mechanism for fast lookup of nodes by their full path.
For now only replace TDB keeping the current access functions.
---
V2:
- add const (Julien Grall)
- use specific pointer type instead of void * (Julien Grall)
- add comment to db_fetch() (Julien Grall)
V3:
- use talloc_memdup() (Julien Grall)
--- tools/xenstore/xenstored_core.c | 156 ++++++++++--------------- tools/xenstore/xenstored_core.h | 5 +- tools/xenstore/xenstored_transaction.c | 1 - 3 files changed, 62 insertions(+), 100 deletions(-)
tools/xenstore/xenstored_core.c +60 -96
@@ -53,7 +53,6 @@#include "xenstored_domain.h" #include "xenstored_control.h" #include "xenstored_lu.h" -#include "tdb.h" #ifndef NO_SOCKETS #if defined(HAVE_SYSTEMD )
@@ -85,7 +84,7 @@ bool keep_orphans = false;static int reopen_log_pipe[2 ];static int reopen_log_pipe0_pollfd_idx =-1 ;char *tracefile =NULL ; -static TDB_CONTEXT *tdb_ctx =NULL ; +static struct hashtable *nodes;unsigned int trace_flags =TRACE_OBJ |TRACE_IO ;static const char *sockmsg_string (enum xsd_sockmsg_type type);
@@ -556,32 +555,30 @@ static void initialize_fds(int *p_sock_pollfd_idx, int *ptimeout) } } -static void set_tdb_key (const char *name,TDB_DATA *key) -{ -/* -* Dropping const is fine here, as the key will never be modified -* by TDB. -*/ - key->dptr = (char *)name; - key->dsize =strlen (name); -} -struct xs_tdb_record_hdr *db_fetch (const char *db_name,size_t *size) { -TDB_DATA key, data; +const struct xs_tdb_record_hdr *hdr; +struct xs_tdb_record_hdr *p; -set_tdb_key (db_name, &key); - data =tdb_fetch (tdb_ctx, key); -if (!data.dptr) { - errno = (tdb_error (tdb_ctx) ==TDB_ERR_NOEXIST ) ?ENOENT :EIO ; - *size =0 ; - }else { - *size = data.dsize; -trace_tdb ("read %s size %zu\n" , db_name, - *size +strlen (db_name)); + hdr =hashtable_search (nodes, db_name); +if (!hdr) { + errno =ENOENT ; +return NULL ; } -return (struct xs_tdb_record_hdr *)data.dptr; + *size =sizeof (*hdr) + hdr->num_perms *sizeof (hdr->perms[0 ]) + + hdr->datalen + hdr->childlen; + +/* Return a copy, avoiding a potential modification in the DB. */ + p =talloc_memdup (NULL , hdr, *size); +if (!p) { + errno =ENOMEM ; +return NULL ; + } + +trace_tdb ("read %s size %zu\n" , db_name, *size +strlen (db_name)); + +return p; }static void get_acc_data (const char *name,struct node_account_data *acc)
@@ -621,12 +618,10 @@ int db_write(struct connection *conn, const char *db_name, void *data,struct xs_tdb_record_hdr *hdr = data;struct node_account_data old_acc = {};unsigned int old_domid, new_domid; +size_t name_len =strlen (db_name); +const char *name;int ret; -TDB_DATA key, dat; -set_tdb_key (db_name, &key); - dat.dptr = data; - dat.dsize = size;if (!acc) old_acc.memory =-1 ;else
@@ -642,29 +637,36 @@ int db_write(struct connection *conn, const char *db_name, void *data,*/ if (old_acc.memory)domain_memory_add_nochk (conn, old_domid, - -old_acc.memory - key.dsize); - ret =domain_memory_add (conn, new_domid, size + key.dsize, + -old_acc.memory - name_len); + ret =domain_memory_add (conn, new_domid, size + name_len, no_quota_check);if (ret) {/* Error path, so no quota check. */ if (old_acc.memory)domain_memory_add_nochk (conn, old_domid, - old_acc.memory + key.dsize); + old_acc.memory + name_len);return ret; } -/* TDB should set errno, but doesn't even set ecode AFAICT. */ -if (tdb_store (tdb_ctx, key, dat, - (mode ==NODE_CREATE ) ?TDB_INSERT :TDB_MODIFY ) !=0 ) { -domain_memory_add_nochk (conn, new_domid, -size - key.dsize); +if (mode ==NODE_CREATE ) { +/* db_name could be modified later, so allocate a copy. */ + name =talloc_strdup (data, db_name); + ret = name ?hashtable_add (nodes, name, data) :ENOMEM ; + }else + ret =hashtable_replace (nodes, db_name, data); + +if (ret) { +/* Free data, as it isn't owned by hashtable now. */ +talloc_free (data); +domain_memory_add_nochk (conn, new_domid, -size - name_len);/* Error path, so no quota check. */ if (old_acc.memory)domain_memory_add_nochk (conn, old_domid, - old_acc.memory + key.dsize); - errno =EIO ; + old_acc.memory + name_len); + errno = ret;return errno; } -trace_tdb ("store %s size %zu\n" , db_name, size + key.dsize); +trace_tdb ("store %s size %zu\n" , db_name, size + name_len);if (acc) {/* Don't use new_domid, as it might be a transaction node. */
@@ -680,9 +682,6 @@ int db_delete(struct connection *conn, const char *name, {struct node_account_data tmp_acc;unsigned int domid; -TDB_DATA key; - -set_tdb_key (name, &key);if (!acc) { acc = &tmp_acc;
@@ -691,15 +690,13 @@ int db_delete(struct connection *conn, const char *name,get_acc_data (name, acc); -if (tdb_delete (tdb_ctx, key)) { - errno =EIO ; -return errno; - } +hashtable_remove (nodes, name);trace_tdb ("delete %s\n" , name);if (acc->memory) { domid =get_acc_domid (conn, name, acc->domid); -domain_memory_add_nochk (conn, domid, -acc->memory - key.dsize); +domain_memory_add_nochk (conn, domid, + -acc->memory -strlen (name)); }return 0 ;
@@ -2354,43 +2351,29 @@ static void manual_node(const char *name, const char *child)talloc_free (node); } -static void tdb_logger (TDB_CONTEXT *tdb,int level,const char * fmt, ...) +static unsigned int hash_from_key_fn (const void *k) { -va_list ap; -char *s; -int saved_errno = errno; +const char *str = k; +unsigned int hash =5381 ; +char c; -va_start (ap, fmt); - s =talloc_vasprintf (NULL , fmt, ap); -va_end (ap); +while ((c = *str++)) + hash = ((hash <<5 ) + hash) + (unsigned int )c; -if (s) { -trace ("TDB: %s\n" , s); -syslog (LOG_ERR ,"TDB: %s" , s); -if (verbose) -xprintf ("TDB: %s" , s); -talloc_free (s); - }else { -trace ("talloc failure during logging\n" ); -syslog (LOG_ERR ,"talloc failure during logging\n" ); - } +return hash; +} - errno = saved_errno; +static int keys_equal_fn (const void *key1,const void *key2) +{ +return 0 ==strcmp (key1, key2); }void setup_structure (bool live_update) { -char *tdbname; - - tdbname =talloc_strdup (talloc_autofree_context (),"/dev/mem" ); -if (!tdbname) -barf_perror ("Could not create tdbname" ); - - tdb_ctx =tdb_open_ex (tdbname,7919 ,TDB_INTERNAL |TDB_NOLOCK , -O_RDWR |O_CREAT |O_EXCL |O_CLOEXEC , -0640 , &tdb_logger,NULL ); -if (!tdb_ctx) -barf_perror ("Could not create tdb file %s" , tdbname); + nodes =create_hashtable (NULL ,"nodes" , hash_from_key_fn, keys_equal_fn, +HASHTABLE_FREE_KEY |HASHTABLE_FREE_VALUE ); +if (!nodes) +barf_perror ("Could not create nodes hashtable" );if (live_update)manual_node ("/" ,NULL );
@@ -2404,24 +2387,6 @@ void setup_structure(bool live_update) } } -static unsigned int hash_from_key_fn (const void *k) -{ -const char *str = k; -unsigned int hash =5381 ; -char c; - -while ((c = *str++)) - hash = ((hash <<5 ) + hash) + (unsigned int )c; - -return hash; -} - - -static int keys_equal_fn (const void *key1,const void *key2) -{ -return 0 ==strcmp (key1, key2); -} -int remember_string (struct hashtable *hash,const char *str) {char *k =talloc_strdup (NULL , str);
@@ -2481,12 +2446,11 @@ static int check_store_enoent(const void *ctx, struct connection *conn,/** * Helper to clean_store below. */ -static int clean_store_ (TDB_CONTEXT *tdb,TDB_DATA key,TDB_DATA val, -void *private) +static int clean_store_ (const void *key,void *val,void *private) {struct hashtable *reachable = private;char *slash; -char * name =talloc_strndup (NULL , key.dptr, key.dsize); +char *name =talloc_strdup (NULL , key);if (!name) {log ("clean_store: ENOMEM" );
@@ -2516,7 +2480,7 @@ static int clean_store_(TDB_CONTEXT *tdb, TDB_DATA key, TDB_DATA val,*/ static void clean_store (struct check_store_data *data) { -tdb_traverse (tdb_ctx, &clean_store_, data->reachable); +hashtable_iterate (nodes, clean_store_, data->reachable);domain_check_acc (data->domains); }
tools/xenstore/xenstored_core.h +2 -3
@@ -33,7 +33,6 @@#include "xenstore_lib.h" #include "xenstore_state.h" #include "list.h" -#include "tdb.h" #include "hashtable.h" #ifndef O_CLOEXEC
@@ -237,7 +236,7 @@ static inline unsigned int get_node_owner(const struct node *node)return node->perms.p[0 ].id; } -/* Write a node to the tdb data base. */ +/* Write a node to the data base. */ enum write_node_mode {NODE_CREATE ,NODE_MODIFY
@@ -247,7 +246,7 @@ int write_node_raw(struct connection *conn, const char *db_name, struct node *node, enum write_node_mode mode, bool no_quota_check); -/* Get a node from the tdb data base. */ +/* Get a node from the data base. */ struct node *read_node (struct connection *conn,const void *ctx,const char *name);
tools/xenstore/xenstored_transaction.c +0 -1
@@ -397,7 +397,6 @@ static int finalize_transaction(struct connection *conn, ? NODE_CREATE : NODE_MODIFY; *is_corrupt |=db_write (conn, i->node, hdr, size,NULL , mode, true); -talloc_free (hdr);if (db_delete (conn, i->trans_name,NULL )) *is_corrupt = true; }else {
Signature · 1 line
2.35.3
Re: [PATCH v3 11/25] tools/xenstore: drop use of tdb
Hi Juergen,
Quoted 26 lines · Juergen Gross
Today all Xenstore nodes are stored in a TDB data base. This data base has several disadvantages:
- It is using a fixed sized hash table, resulting in high memory overhead for small installations with only very few VMs, and a rather large performance hit for systems with lots of VMs due to many collisions.
The hash table size today is 7919 entries. This means that e.g. in case of a simple desktop use case with 2 or 3 VMs probably far less than 10% of the entries will be used (assuming roughly 100 nodes per VM). OTOH a setup on a large server with 500 VMs would result in heavy conflicts in the hash list with 5-10 nodes per hash table entry.- TDB is using a single large memory area for storing the nodes. It only ever increases this area and will never shrink it afterwards. This will result in more memory usage than necessary after a peak of Xenstore usage.
- Xenstore is only single-threaded, while TDB is designed to be fit for multi-threaded use cases, resulting in much higher code complexity than needed.
- Special use cases of Xenstore are not possible to implement with TDB in an effective way, while an implementation of a data base tailored for Xenstore could simplify some handling (e.g. transactions) a lot.
Quoted 7 lines
So drop using TDB and store the nodes directly in memory making them easily accessible. Use a hash-based lookup mechanism for fast lookup of nodes by their full path.
For now only replace TDB keeping the current access functions.
Signed-off-by Juergen Gross <[email protected]>
Cheers,
Signature · 1 line
Julien Grall
PATCH v3 12/25 tools/xenstore: remove tdb code
In reply to Juergen GrossNow that TDB isn't used anymore, remove it.
--- tools/xenstore/Makefile.common | 2 +- tools/xenstore/tdb.c | 1748 -------------------------------- tools/xenstore/tdb.h | 132 --- 3 files changed, 1 insertion(+), 1881 deletions(-) delete mode 100644 tools/xenstore/tdb.c delete mode 100644 tools/xenstore/tdb.h
tools/xenstore/Makefile.common +1 -1
@@ -2,7 +2,7 @@ XENSTORED_OBJS-y := xenstored_core.o xenstored_watch.o xenstored_domain.o XENSTORED_OBJS-y += xenstored_transaction.o xenstored_control.o xenstored_lu.o -XENSTORED_OBJS-y += talloc.o utils.o tdb.o hashtable.o +XENSTORED_OBJS-y += talloc.o utils.o hashtable.o XENSTORED_OBJS-$(CONFIG_Linux) += xenstored_posix.o xenstored_lu_daemon.o XENSTORED_OBJS-$(CONFIG_NetBSD) += xenstored_posix.o xenstored_lu_daemon.o
tools/xenstore/tdb.c deleted +0 -1748
1,748 lines not on this page · raw message
tools/xenstore/tdb.h deleted +0 -132
132 lines not on this page · raw message
Signature · 1 line
2.35.3
PATCH v3 13/25 tools/xenstore: let db_delete() return void
In reply to Juergen Grossdb_delete() only ever is returning 0. Switch it to return void and remove all the error handling dealing wit a non-zero return value.
---
V2:
- new patch
--- tools/xenstore/xenstored_core.c | 11 ++++------- tools/xenstore/xenstored_core.h | 4 ++-- tools/xenstore/xenstored_transaction.c | 14 +++++--------- 3 files changed, 11 insertions(+), 18 deletions(-)
tools/xenstore/xenstored_core.c +4 -7
@@ -677,8 +677,8 @@ int db_write(struct connection *conn, const char *db_name, void *data,return 0 ; } -int db_delete (struct connection *conn,const char *name, -struct node_account_data *acc) +void db_delete (struct connection *conn,const char *name, +struct node_account_data *acc) {struct node_account_data tmp_acc;unsigned int domid;
@@ -698,8 +698,6 @@ int db_delete(struct connection *conn, const char *name,domain_memory_add_nochk (conn, domid, -acc->memory -strlen (name)); } - -return 0 ; }/*
@@ -1670,9 +1668,8 @@ static int delnode_sub(const void *ctx, struct connection *conn,if (domain_nbentry_dec (conn,get_node_owner (node)))return WALK_TREE_ERROR_STOP ; -/* In case of error stop the walk. */ -if (!ret &&db_delete (conn, db_name, &node->acc)) -return WALK_TREE_ERROR_STOP ; +if (!ret) +db_delete (conn, db_name, &node->acc);/* * Fire the watches now, when we can still see the node permissions.
tools/xenstore/xenstored_core.h +2 -2
@@ -366,8 +366,8 @@ struct xs_tdb_record_hdr *db_fetch(const char *db_name, size_t *size);int db_write (struct connection *conn,const char *db_name,void *data,size_t size,struct node_account_data *acc,enum write_node_mode mode,bool no_quota_check); -int db_delete (struct connection *conn,const char *name, -struct node_account_data *acc); +void db_delete (struct connection *conn,const char *name, +struct node_account_data *acc);void conn_free_buffered_data (struct connection *conn);
tools/xenstore/xenstored_transaction.c +5 -9
@@ -377,10 +377,8 @@ static int finalize_transaction(struct connection *conn,/* Entries for unmodified nodes can be removed early. */ if (!i->modified) { -if (i->ta_node) { -if (db_delete (conn, i->trans_name,NULL )) -return EIO ; - } +if (i->ta_node) +db_delete (conn, i->trans_name,NULL );list_del (&i->list);talloc_free (i); }
@@ -397,8 +395,7 @@ static int finalize_transaction(struct connection *conn, ? NODE_CREATE : NODE_MODIFY; *is_corrupt |=db_write (conn, i->node, hdr, size,NULL , mode, true); -if (db_delete (conn, i->trans_name,NULL )) - *is_corrupt = true; +db_delete (conn, i->trans_name,NULL ); }else { *is_corrupt = true; }
@@ -408,9 +405,8 @@ static int finalize_transaction(struct connection *conn,* in this transaction will have no generation * information stored. */ - *is_corrupt |= (i->generation ==NO_GENERATION ) - ? false - :db_delete (conn, i->node,NULL ); +if (i->generation !=NO_GENERATION ) +db_delete (conn, i->node,NULL ); }if (i->fire_watch) fire_watches(conn, trans, i->node, NULL, i->watch_exact,
Signature · 1 line
2.35.3
PATCH v3 14/25 tools/xenstore: change talloc_free() to take a const pointer
In reply to Juergen GrossWith talloc_free() and related functions not taking a ponter to const it is tedious to use the const attribute for talloc()-ed memory in many cases.
Change the related prototypes to use "const void *" instead of "void *".
---
V3:
- new patch
--- tools/xenstore/talloc.c | 8 ++++---- tools/xenstore/talloc.h | 4 ++-- 2 files changed, 6 insertions(+), 6 deletions(-)
tools/xenstore/talloc.c +4 -4
@@ -319,7 +319,7 @@ static int talloc_unreference(const void *context, const void *ptr)remove a specific parent context from a pointer. This is a more controlled varient of talloc_free() */ -int talloc_unlink (const void *context,void *ptr) +int talloc_unlink (const void *context,const void *ptr) {struct talloc_chunk *tc_p, *new_p;void *new_parent;
@@ -499,7 +499,7 @@ void *talloc_init(const char *fmt, ...)should probably not be used in new code. It's in here to keep the talloc code consistent across Samba 3 and 4. */ -static void talloc_free_children (void *ptr) +static void talloc_free_children (const void *ptr) {struct talloc_chunk *tc;
@@ -539,7 +539,7 @@ static void talloc_free_children(void *ptr)will not be freed if the ref_count is > 1 or the destructor (if any) returns non-zero */ -int talloc_free (void *ptr) +int talloc_free (const void *ptr) {int saved_errno = errno;struct talloc_chunk *tc;
@@ -571,7 +571,7 @@ int talloc_free(void *ptr) goto err; } tc->destructor = (talloc_destructor_t)-1 ; -if (d (ptr) ==-1 ) { +if (d ((void *)ptr) ==-1 ) { tc->destructor = d; goto err; }
tools/xenstore/talloc.h +2 -2
@@ -92,7 +92,7 @@ void *_talloc(const void *context, size_t size);void talloc_set_destructor (const void *ptr,int (*destructor)(void *));void talloc_increase_ref_count (const void *ptr);void *talloc_reference (const void *context,const void *ptr); -int talloc_unlink (const void *context,void *ptr); +int talloc_unlink (const void *context,const void *ptr); void talloc_set_name(const void *ptr, const char *fmt, ...) PRINTF_ATTRIBUTE(2,3);void talloc_set_name_const (const void *ptr,const char *name); void *talloc_named(const void *context, size_t size,
@@ -103,7 +103,7 @@ void *talloc_check_name(const void *ptr, const char *name);void talloc_report_depth (const void *ptr,FILE *f,int depth);void *talloc_parent (const void *ptr);void *talloc_init (const char *fmt, ...)PRINTF_ATTRIBUTE (1 ,2 ); -int talloc_free (void *ptr); +int talloc_free (const void *ptr);void *_talloc_realloc (const void *context,void *ptr,size_t size,const char *name);void *talloc_steal (const void *new_ctx,const void *ptr);off_t talloc_total_size (const void *ptr);
Signature · 1 line
2.35.3
Re: [PATCH v3 14/25] tools/xenstore: change talloc_free() to take a const pointer
Hi Juergen,
Quoted 1 line · Juergen Gross
With
talloc_free()and related functions not taking a ponter to const
typo: s/ponter/pointer/
Quoted 47 lines
it is tedious to use the const attribute for talloc()-ed memory in many cases.
Change the related prototypes to use "const void *" instead of "void *".
Signed-off-by Juergen Gross <[email protected]>---
V3:
- new patch--- tools/xenstore/talloc.c | 8 ++++---- tools/xenstore/talloc.h | 4 ++-- 2 files changed, 6 insertions(+), 6 deletions(-)tools/xenstore/talloc.c +3
-3@@ -319,7 +319,7 @@ static int talloc_unreference(const void *context, const void *ptr)remove a specific parent context from a pointer. This is a more controlled varient of talloc_free() */ -int talloc_unlink (const void *context,void *ptr) +int talloc_unlink (const void *context,const void *ptr) {struct talloc_chunk *tc_p, *new_p;void *new_parent;@@ -499,7 +499,7 @@ void *talloc_init(const char *fmt, ...)should probably not be used in new code. It's in here to keep the talloc code consistent across Samba 3 and 4. */ -static void talloc_free_children (void *ptr) +static void talloc_free_children (const void *ptr) {struct talloc_chunk *tc;@@ -539,7 +539,7 @@ static void talloc_free_children(void *ptr)will not be freed if the ref_count is > 1 or the destructor (if any) returns non-zero */ -int talloc_free (void *ptr) +int talloc_free (const void *ptr) {int saved_errno = errno;struct talloc_chunk *tc;
tools/xenstore/talloc.c int talloc_free(void *ptr) · line 574
Quoted 5 lines
tools/xenstore/talloc.c +1
-1@@ -571,7 +571,7 @@ int talloc_free(void *ptr) goto err; } tc->destructor = (talloc_destructor_t)-1 ; -if (d (ptr) ==-1 ) { +if (d ((void *)ptr) ==-1 ) {
AFAICT, you can't propagate the const because the destructor may need to modify the content. I guess this is a necessary evil here but it would be good to document it.
Cheers,
Signature · 1 line
Julien Grall
Re: [PATCH v3 14/25] tools/xenstore: change talloc_free() to take a const pointer
Quoted 53 lines · Julien Grall
Hi Juergen,
Quoted 1 line · Juergen Gross
With
talloc_free()and related functions not taking a ponter to consttypo: s/ponter/pointer/
Quoted 46 lines
it is tedious to use the const attribute for talloc()-ed memory in many cases.
Change the related prototypes to use "const void *" instead of "void *".
Signed-off-by Juergen Gross <[email protected]>---
V3:
- new patch
---
tools/xenstore/talloc.c| 8 ++++----
tools/xenstore/talloc.h| 4 ++--2 files changed, 6 insertions(+), 6 deletions(-)tools/xenstore/talloc.c +0
-0@@ -319,7 +319,7 @@ static int talloc_unreference(const void *context, const void *ptr)remove a specific parent context from a pointer. This is a more controlled varient of
talloc_free()
*/tools/xenstore/talloc.c +1
-1-int talloc_unlink (const void *context,void *ptr) +int talloc_unlink (const void *context,const void *ptr){
structtalloc_chunk*tc_p,*new_p;
void*new_parent;tools/xenstore/talloc.c +0
-0@@ -499,7 +499,7 @@ void *talloc_init(const char *fmt, ...)should probably not be used in new code. It's in here to keep the talloc code consistent across Samba 3 and 4.
*/tools/xenstore/talloc.c +1
-1-static void talloc_free_children (void *ptr) +static void talloc_free_children (const void *ptr){
structtalloc_chunk*tc;tools/xenstore/talloc.c +0
-0@@ -539,7 +539,7 @@ static void talloc_free_children(void *ptr)will not be freed if the
ref_countis > 1 or the destructor (if any) returns non-zero
*/tools/xenstore/talloc.c +1
-1-int talloc_free (void *ptr) +int talloc_free (const void *ptr){
intsaved_errno= errno;
structtalloc_chunk*tc;
Quoted 8 lines
tools/xenstore/talloc.c int talloc_free(void *ptr) · line 574
Quoted 6 lines
tools/xenstore/talloc.c +1
-1@@ -571,7 +571,7 @@ int talloc_free(void *ptr) goto err; } tc->destructor = (talloc_destructor_t)-1; - if (d(ptr) == -1) { + if (d((void *)ptr) == -1) {AFAICT, you can't propagate the const because the destructor may need to modify the content. I guess this is a necessary evil here but it would be good to document it.
Okay, will add a comment.
Juergen
PATCH v3 15/25 tools/xenstore: make data parameter of db_write() const
In reply to Juergen Grossdb_write() doesn't change the data written to the data base, so the data parameter can be const.
---
V3:
- new patch
--- tools/xenstore/xenstored_core.c | 4 ++-- tools/xenstore/xenstored_core.h | 2 +- 2 files changed, 3 insertions(+), 3 deletions(-)
tools/xenstore/xenstored_core.c +2 -2
@@ -611,11 +611,11 @@ static unsigned int get_acc_domid(struct connection *conn, const char *name,return (!conn || name[0 ] =='/' || name[0 ] =='@' ) ? domid : conn->id; } -int db_write (struct connection *conn,const char *db_name,void *data, +int db_write (struct connection *conn,const char *db_name,const void *data,size_t size,struct node_account_data *acc,enum write_node_mode mode,bool no_quota_check) { -struct xs_tdb_record_hdr *hdr = data; +const struct xs_tdb_record_hdr *hdr = data;struct node_account_data old_acc = {};unsigned int old_domid, new_domid;size_t name_len =strlen (db_name);
tools/xenstore/xenstored_core.h +1 -1
@@ -363,7 +363,7 @@ int remember_string(struct hashtable *hash, const char *str);/* Data base access functions. */ struct xs_tdb_record_hdr *db_fetch (const char *db_name,size_t *size); -int db_write (struct connection *conn,const char *db_name,void *data, +int db_write (struct connection *conn,const char *db_name,const void *data,size_t size,struct node_account_data *acc,enum write_node_mode mode,bool no_quota_check); void db_delete(struct connection *conn, const char *name,
Signature · 1 line
2.35.3
PATCH v3 16/25 tools/xenstore: move copying of node data out of db_fetch()
In reply to Juergen GrossToday the node data is copied in db_fetch() on each data base read in order to avoid accidental data base modifications when working on a node.
read_node() is the only caller of db_fetch() which isn't freeing the returned data area immediately after using it. The other callers don't modify the returned data, so they don't need the data to be copied.
Move copying of the data into read_node(), resulting in a speedup of the other callers due to no memory allocation and no copying being needed anymore.
This allows to let db_fetch() return a pointer to const data.
As db_fetch() can't return any error other than ENOENT now, error handling for the callers can be simplified.
---
V2:
- new patch
V3:
- modify return type of db_fetch() to return a pointer to const (Julien Grall)
- drop stale comment (Julien Grall)
- fix transaction handling
--- tools/xenstore/xenstored_core.c | 45 +++++++++++--------------- tools/xenstore/xenstored_core.h | 2 +- tools/xenstore/xenstored_transaction.c | 23 +++++++++---- 3 files changed, 35 insertions(+), 35 deletions(-)
tools/xenstore/xenstored_core.c +18 -27
@@ -555,10 +555,9 @@ static void initialize_fds(int *p_sock_pollfd_idx, int *ptimeout) } } -struct xs_tdb_record_hdr *db_fetch (const char *db_name,size_t *size) +const struct xs_tdb_record_hdr *db_fetch (const char *db_name,size_t *size) { -const struct xs_tdb_record_hdr *hdr; -struct xs_tdb_record_hdr *p; +struct xs_tdb_record_hdr *hdr; hdr =hashtable_search (nodes, db_name);if (!hdr) {
@@ -569,22 +568,15 @@ struct xs_tdb_record_hdr *db_fetch(const char *db_name, size_t *size) *size =sizeof (*hdr) + hdr->num_perms *sizeof (hdr->perms[0 ]) + hdr->datalen + hdr->childlen; -/* Return a copy, avoiding a potential modification in the DB. */ - p =talloc_memdup (NULL , hdr, *size); -if (!p) { - errno =ENOMEM ; -return NULL ; - } -trace_tdb ("read %s size %zu\n" , db_name, *size +strlen (db_name)); -return p; +return hdr; }static void get_acc_data (const char *name,struct node_account_data *acc) {size_t size; -struct xs_tdb_record_hdr *hdr; +const struct xs_tdb_record_hdr *hdr;if (acc->memory <0 ) { hdr =db_fetch (name, &size);
@@ -595,7 +587,6 @@ static void get_acc_data(const char *name, struct node_account_data *acc) acc->memory = size; acc->domid = hdr->perms[0 ].id; } -talloc_free (hdr); } }
@@ -708,7 +699,7 @@ struct node *read_node(struct connection *conn, const void *ctx, const char *name) {size_t size; -struct xs_tdb_record_hdr *hdr; +const struct xs_tdb_record_hdr *hdr;struct node *node;const char *db_name;int err;
@@ -729,30 +720,30 @@ struct node *read_node(struct connection *conn, const void *ctx, hdr =db_fetch (db_name, &size);if (hdr ==NULL ) { -if (errno ==ENOENT ) { - node->generation =NO_GENERATION ; - err =access_node (conn, node,NODE_ACCESS_READ ,NULL ); - errno = err ? :ENOENT ; - }else { -log ("DB error on read: %s" ,strerror (errno)); - errno =EIO ; - } + node->generation =NO_GENERATION ; + err =access_node (conn, node,NODE_ACCESS_READ ,NULL ); + errno = err ? :ENOENT ; goto error; } node->parent =NULL ; -talloc_steal (node, hdr);/* Datalen, childlen, number of permissions */ node->generation = hdr->generation; node->perms.num = hdr->num_perms; node->datalen = hdr->datalen; node->childlen = hdr->childlen; - -/* Permissions are struct xs_permissions. */ - node->perms.p = hdr->perms; - node->acc.domid =get_node_owner (node); + node->acc.domid = hdr->perms[0 ].id; node->acc.memory = size; + +/* Copy node data to new memory area, starting with permissions. */ + size -=sizeof (*hdr); + node->perms.p =talloc_memdup (node, hdr->perms, size); +if (node->perms.p ==NULL ) { + errno =ENOMEM ; + goto error; + } +if (domain_adjust_node_perms (node)) goto error;
tools/xenstore/xenstored_core.h +1 -1
@@ -362,7 +362,7 @@ extern xengnttab_handle **xgt_handle;int remember_string (struct hashtable *hash,const char *str);/* Data base access functions. */ -struct xs_tdb_record_hdr *db_fetch (const char *db_name,size_t *size); +const struct xs_tdb_record_hdr *db_fetch (const char *db_name,size_t *size);int db_write (struct connection *conn,const char *db_name,const void *data,size_t size,struct node_account_data *acc,enum write_node_mode mode,bool no_quota_check);
tools/xenstore/xenstored_transaction.c +16 -7
@@ -357,20 +357,17 @@ static int finalize_transaction(struct connection *conn, {struct accessed_node *i, *n;size_t size; -struct xs_tdb_record_hdr *hdr; +const struct xs_tdb_record_hdr *hdr;uint64_t gen;list_for_each_entry_safe (i, n, &trans->accessed, list) {if (i->check_gen) { hdr =db_fetch (i->node, &size);if (!hdr) { -if (errno !=ENOENT ) -return errno; gen =NO_GENERATION ; }else { gen = hdr->generation; } -talloc_free (hdr);if (i->generation != gen)return EAGAIN ; }
@@ -388,14 +385,26 @@ static int finalize_transaction(struct connection *conn,if (i->ta_node) { hdr =db_fetch (i->trans_name, &size);if (hdr) { +/* +* Delete transaction entry and write it as +* no-TA entry. As we only hold a reference +* to the data, increment its ref count, then +* delete it from the DB. Now we own it and can +* drop the const attribute for changing the +* generation count. +*/ enum write_node_mode mode; +struct xs_tdb_record_hdr *own; - hdr->generation = ++generation; +talloc_increase_ref_count (hdr); +db_delete (conn, i->trans_name,NULL ); + + own = (struct xs_tdb_record_hdr *)hdr; + own->generation = ++generation; mode = (i->generation ==NO_GENERATION ) ?NODE_CREATE :NODE_MODIFY ; - *is_corrupt |=db_write (conn, i->node, hdr, + *is_corrupt |=db_write (conn, i->node, own, size,NULL , mode, true); -db_delete (conn, i->trans_name,NULL ); }else { *is_corrupt = true; }
Signature · 1 line
2.35.3
Re: [PATCH v3 16/25] tools/xenstore: move copying of node data out of db_fetch()
Hi Juergen,
Quoted 37 lines · Juergen Gross
Today the node data is copied in
db_fetch()on each data base read in order to avoid accidental data base modifications when working on a node.
read_node()is the only caller ofdb_fetch()which isn't freeing the returned data area immediately after using it. The other callers don't modify the returned data, so they don't need the data to be copied.Move copying of the data into
read_node(), resulting in a speedup of the other callers due to no memory allocation and no copying being needed anymore.This allows to let
db_fetch()return a pointer to const data.As
db_fetch()can't return any error other than ENOENT now, error handling for the callers can be simplified.Signed-off-by Juergen Gross <[email protected]>---
V2:
- new patch
V3:
- modify return type ofdb_fetch()to return a pointer to const (Julien Grall)
- drop stale comment (Julien Grall)
- fix transaction handling--- tools/xenstore/xenstored_core.c | 45 +++++++++++--------------- tools/xenstore/xenstored_core.h | 2 +- tools/xenstore/xenstored_transaction.c | 23 +++++++++---- 3 files changed, 35 insertions(+), 35 deletions(-)tools/xenstore/xenstored_core.c +0
-0
tools/xenstore/xenstored_core.c static void initialize_fds(int *p_sock_pollfd_idx, int *ptimeout) · line 560
Quoted 9 lines
tools/xenstore/xenstored_core.c +2
-3@@ -555,10 +555,9 @@ static void initialize_fds(int *p_sock_pollfd_idx, int *ptimeout) } } -struct xs_tdb_record_hdr *db_fetch (const char *db_name,size_t *size) +const struct xs_tdb_record_hdr *db_fetch (const char *db_name,size_t *size) { -const struct xs_tdb_record_hdr *hdr; -struct xs_tdb_record_hdr *p; +struct xs_tdb_record_hdr *hdr;
Can't 'hdr' stay const?
Quoted 3 lines
tools/xenstore/xenstored_core.c +0
-0hdr =hashtable_search (nodes, db_name);if (!hdr) {
[...]
tools/xenstore/xenstored_core.c static int finalize_transaction(struct connection *conn, · line 393
Quoted 10 lines
tools/xenstore/xenstored_core.c +6
-0@@ -388,14 +385,26 @@ static int finalize_transaction(struct connection *conn,if (i->ta_node) { hdr =db_fetch (i->trans_name, &size);if (hdr) { +/* +* Delete transaction entry and write it as +* no-TA entry. As we only hold a reference +* to the data, increment its ref count, then +* delete it from the DB. Now we own it and can +* drop the const attribute for changing the
It is not great, but I understand this is another necessary evil. So I am ok with this cast-away const. It is also well documented.
Quoted 10 lines
tools/xenstore/xenstored_core.c +7
-1+* generation count. +*/ enum write_node_mode mode; +struct xs_tdb_record_hdr *own; - hdr->generation = ++generation; +talloc_increase_ref_count (hdr); +db_delete (conn, i->trans_name,NULL ); + + own = (struct xs_tdb_record_hdr *)hdr;
tools/xenstore/xenstored_core.c static int finalize_transaction(struct connection *conn, · line 410
Quoted 10 lines
tools/xenstore/xenstored_core.c +2
-2+ own->generation = ++generation; mode = (i->generation ==NO_GENERATION ) ?NODE_CREATE :NODE_MODIFY ; - *is_corrupt |=db_write (conn, i->node, hdr, + *is_corrupt |=db_write (conn, i->node, own, size,NULL , mode, true); -db_delete (conn, i->trans_name,NULL ); }else { *is_corrupt = true; }
Cheers,
Signature · 1 line
Julien Grall
Re: [PATCH v3 16/25] tools/xenstore: move copying of node data out of db_fetch()
Quoted 35 lines · Julien Grall
Hi Juergen,
Quoted 32 lines · Juergen Gross
Today the node data is copied in
db_fetch()on each data base read in order to avoid accidental data base modifications when working on a node.
read_node()is the only caller ofdb_fetch()which isn't freeing the returned data area immediately after using it. The other callers don't modify the returned data, so they don't need the data to be copied.Move copying of the data into
read_node(), resulting in a speedup of the other callers due to no memory allocation and no copying being needed anymore.This allows to let
db_fetch()return a pointer to const data.As
db_fetch()can't return any error other than ENOENT now, error handling for the callers can be simplified.Signed-off-by Juergen Gross <[email protected]>---
V2:
- new patch
V3:
- modify return type ofdb_fetch()to return a pointer to const (Julien Grall)
- drop stale comment (Julien Grall)
- fix transaction handling
---
tools/xenstore/xenstored_core.c | 45 +++++++++++--------------- tools/xenstore/xenstored_core.h | 2 +-tools/xenstore/xenstored_transaction.c| 23 +++++++++----3 files changed, 35 insertions(+), 35 deletions(-)
Quoted 16 lines
Quoted 5 lines
tools/xenstore/xenstored_core.c +0
-0tools/xenstore/xenstored_core.c static void initialize_fds(int *p_sock_pollfd_idx, int · line 560
Quoted 9 lines
tools/xenstore/xenstored_core.c +2
-3@@ -555,10 +555,9 @@ static void initialize_fds(int *p_sock_pollfd_idx, int *ptimeout) } } -struct xs_tdb_record_hdr *db_fetch (const char *db_name,size_t *size) +const struct xs_tdb_record_hdr *db_fetch (const char *db_name,size_t *size) { - const struct xs_tdb_record_hdr *hdr; - struct xs_tdb_record_hdr *p; + struct xs_tdb_record_hdr *hdr;Can't 'hdr' stay const?
Oh yes, I think so.
Quoted 6 lines
Quoted 3 lines
tools/xenstore/xenstored_core.c +0
-0hdr = hashtable_search(nodes, db_name); if (!hdr) {[...]
Quoted 12 lines
tools/xenstore/xenstored_core.c static int finalize_transaction(struct connection *conn, · line 393
Quoted 10 lines
tools/xenstore/xenstored_core.c +6
-0@@ -388,14 +385,26 @@ static int finalize_transaction(struct connection *conn, if (i->ta_node) { hdr = db_fetch(i->trans_name, &size); if (hdr) { + /* +* Delete transaction entry and write it as +* no-TA entry. As we only hold a reference +* to the data, increment its ref count, then +* delete it from the DB. Now we own it and can +* drop the const attribute for changing the It is not great, but I understand this is another necessary evil. So I am ok with this cast-away const. It is also well documented.
Thanks. In fact this is a fix resulting from letting db_fetch() return a const pointer. So the const attribute already paid off.
Juergen
PATCH v3 17/25 tools/xenstore: rework struct xs_tdb_record_hdr
In reply to Juergen GrossStruct xs_tdb_record_hdr is used for nodes stored in the data base. When working on a node, struct node is being used, which is including the same information as struct xs_tdb_record_hdr, but in a different format. Rework struct xs_tdb_record_hdr in order to prepare including it in struct node.
Do the following modifications:
- move its definition to xenstored_core.h, as the reason to put it into utils.h are no longer existing
- rename it to struct node_hdr, as the "tdb" in its name has only historical reasons
- replace the empty permission array at the end with a comment about the layout of data in the data base (concatenation of header, permissions, node contents, and children list)
- use narrower types for num_perms and datalen, as those are naturally limited to XENSTORE_PAYLOAD_MAX (childlen is different here, as it is in theory basically unlimited)
---
V2:
- new patch
--- tools/xenstore/utils.h | 9 ------- tools/xenstore/xenstored_core.c | 35 +++++++++++++++----------- tools/xenstore/xenstored_core.h | 20 ++++++++++++++- tools/xenstore/xenstored_transaction.c | 6 ++--- 4 files changed, 42 insertions(+), 28 deletions(-)
tools/xenstore/utils.h +0 -9
@@ -9,15 +9,6 @@#include "xenstore_lib.h" -/* Header of the node record in tdb. */ -struct xs_tdb_record_hdr { -uint64_t generation; -uint32_t num_perms; -uint32_t datalen; -uint32_t childlen; -struct xs_permissions perms[0 ]; -}; -/* Is A == B ? */ #define streq (a,b) (strcmp((a),(b)) == 0)
tools/xenstore/xenstored_core.c +20 -15
@@ -555,9 +555,9 @@ static void initialize_fds(int *p_sock_pollfd_idx, int *ptimeout) } } -const struct xs_tdb_record_hdr *db_fetch (const char *db_name,size_t *size) +const struct node_hdr *db_fetch (const char *db_name,size_t *size) { -struct xs_tdb_record_hdr *hdr; +struct node_hdr *hdr; hdr =hashtable_search (nodes, db_name);if (!hdr) {
@@ -565,7 +565,7 @@ const struct xs_tdb_record_hdr *db_fetch(const char *db_name, size_t *size)return NULL ; } - *size =sizeof (*hdr) + hdr->num_perms *sizeof (hdr->perms[0 ]) + + *size =sizeof (*hdr) + hdr->num_perms *sizeof (struct xs_permissions ) + hdr->datalen + hdr->childlen;trace_tdb ("read %s size %zu\n" , db_name, *size +strlen (db_name));
@@ -573,10 +573,15 @@ const struct xs_tdb_record_hdr *db_fetch(const char *db_name, size_t *size)return hdr; } +static struct xs_permissions *perms_from_node_hdr (const struct node_hdr *hdr) +{ +return (struct xs_permissions *)(hdr +1 ); +} +static void get_acc_data (const char *name,struct node_account_data *acc) {size_t size; -const struct xs_tdb_record_hdr *hdr; +const struct node_hdr *hdr;if (acc->memory <0 ) { hdr =db_fetch (name, &size);
@@ -585,7 +590,7 @@ static void get_acc_data(const char *name, struct node_account_data *acc) acc->memory =0 ; }else { acc->memory = size; - acc->domid = hdr->perms[0 ].id; + acc->domid =perms_from_node_hdr (hdr)->id; } } }
@@ -606,7 +611,7 @@ int db_write(struct connection *conn, const char *db_name, const void *data, size_t size, struct node_account_data *acc, enum write_node_mode mode, bool no_quota_check) { - const struct xs_tdb_record_hdr *hdr = data; + const struct node_hdr *hdr = data;struct node_account_data old_acc = {};unsigned int old_domid, new_domid;size_t name_len =strlen (db_name);
@@ -620,7 +625,7 @@ int db_write(struct connection *conn, const char *db_name, const void *data,get_acc_data (db_name, &old_acc); old_domid =get_acc_domid (conn, db_name, old_acc.domid); - new_domid =get_acc_domid (conn, db_name, hdr->perms[0 ].id); + new_domid =get_acc_domid (conn, db_name,perms_from_node_hdr (hdr)->id);/* * Don't check for ENOENT, as we want to be able to switch orphaned
@@ -661,7 +666,7 @@ int db_write(struct connection *conn, const char *db_name, const void *data,if (acc) {/* Don't use new_domid, as it might be a transaction node. */ - acc->domid = hdr->perms[0 ].id; + acc->domid =perms_from_node_hdr (hdr)->id; acc->memory = size; }
@@ -699,7 +704,7 @@ struct node *read_node(struct connection *conn, const void *ctx, const char *name) {size_t size; -const struct xs_tdb_record_hdr *hdr; +const struct node_hdr *hdr;struct node *node;const char *db_name;int err;
@@ -733,12 +738,12 @@ struct node *read_node(struct connection *conn, const void *ctx, node->perms.num = hdr->num_perms; node->datalen = hdr->datalen; node->childlen = hdr->childlen; - node->acc.domid = hdr->perms[0 ].id; + node->acc.domid =perms_from_node_hdr (hdr)->id; node->acc.memory = size;/* Copy node data to new memory area, starting with permissions. */ size -=sizeof (*hdr); - node->perms.p =talloc_memdup (node, hdr->perms, size); + node->perms.p =talloc_memdup (node,perms_from_node_hdr (hdr), size);if (node->perms.p ==NULL ) { errno =ENOMEM ; goto error;
@@ -785,7 +790,7 @@ int write_node_raw(struct connection *conn, const char *db_name,void *data;size_t size;void *p; -struct xs_tdb_record_hdr *hdr; +struct node_hdr *hdr;if (domain_adjust_node_perms (node))return errno;
@@ -812,9 +817,9 @@ int write_node_raw(struct connection *conn, const char *db_name, hdr->datalen = node->datalen; hdr->childlen = node->childlen; -memcpy (hdr->perms, node->perms.p, - node->perms.num *sizeof (*node->perms.p)); - p = hdr->perms + node->perms.num; + p =perms_from_node_hdr (hdr); +memcpy (p, node->perms.p, node->perms.num *sizeof (*node->perms.p)); + p += node->perms.num *sizeof (*node->perms.p);memcpy (p, node->data, node->datalen); p += node->datalen;memcpy (p, node->children, node->childlen);
tools/xenstore/xenstored_core.h +19 -1
@@ -168,6 +168,24 @@ struct connection };extern struct list_head connections; +/* +* Header of the node record in the data base. +* In the data base the memory of the node is a single memory chunk with the +* following format: +* struct { +* node_hdr hdr; +* struct xs_permissions perms[hdr.num_perms]; +* char data[hdr.datalen]; +* char children[hdr.childlen]; +* }; +*/ +struct node_hdr { +uint64_t generation; +uint16_t num_perms; +uint16_t datalen; +uint32_t childlen; +}; +struct node_perms {unsigned int num;struct xs_permissions *p;
@@ -362,7 +380,7 @@ extern xengnttab_handle **xgt_handle;int remember_string (struct hashtable *hash,const char *str);/* Data base access functions. */ -const struct xs_tdb_record_hdr *db_fetch (const char *db_name,size_t *size); +const struct node_hdr *db_fetch (const char *db_name,size_t *size);int db_write (struct connection *conn,const char *db_name,const void *data,size_t size,struct node_account_data *acc,enum write_node_mode mode,bool no_quota_check);
tools/xenstore/xenstored_transaction.c +3 -3
@@ -357,7 +357,7 @@ static int finalize_transaction(struct connection *conn, {struct accessed_node *i, *n;size_t size; -const struct xs_tdb_record_hdr *hdr; +const struct node_hdr *hdr;uint64_t gen;list_for_each_entry_safe (i, n, &trans->accessed, list) {
@@ -394,12 +394,12 @@ static int finalize_transaction(struct connection *conn,* generation count. */ enum write_node_mode mode; -struct xs_tdb_record_hdr *own; +struct node_hdr *own;talloc_increase_ref_count (hdr);db_delete (conn, i->trans_name,NULL ); - own = (struct xs_tdb_record_hdr *)hdr; + own = (struct node_hdr *)hdr; own->generation = ++generation; mode = (i->generation ==NO_GENERATION ) ?NODE_CREATE :NODE_MODIFY ;
Signature · 1 line
2.35.3
Re: [PATCH v3 17/25] tools/xenstore: rework struct xs_tdb_record_hdr
Hi Juergen,
Quoted 14 lines · Juergen Gross
Struct
xs_tdb_record_hdris used for nodes stored in the data base. When working on a node, struct node is being used, which is including the same information as structxs_tdb_record_hdr, but in a different format. Rework structxs_tdb_record_hdrin order to prepare including it in struct node.Do the following modifications:
- move its definition to
xenstored_core.h, as the reason to put it intoutils.hare no longer existing- rename it to struct
node_hdr, as the "tdb" in its name has only historical reasons
Quoted 7 lines
- replace the empty permission array at the end with a comment about the layout of data in the data base (concatenation of header, permissions, node contents, and children list)
- use narrower types for
num_permsand datalen, as those are naturally limited toXENSTORE_PAYLOAD_MAX(childlen is different here, as it is in theory basically unlimited)
The assumption is XENSTORE_PAYLOAD_MAX will never change and/or always apply for all the connection. I am aware of at least one downstream use where this is not the case.
I am happy to use narrower types, but I would at least like some checks in Xenstore to ensure the limits are not reached.
Quoted 58 lines
Signed-off-by Juergen Gross <[email protected]>---
V2:
- new patch--- tools/xenstore/utils.h | 9 ------- tools/xenstore/xenstored_core.c | 35 +++++++++++++++----------- tools/xenstore/xenstored_core.h | 20 ++++++++++++++- tools/xenstore/xenstored_transaction.c | 6 ++--- 4 files changed, 42 insertions(+), 28 deletions(-)tools/xenstore/utils.h +0
-9@@ -9,15 +9,6 @@#include "xenstore_lib.h" -/* Header of the node record in tdb. */ -struct xs_tdb_record_hdr { -uint64_t generation; -uint32_t num_perms; -uint32_t datalen; -uint32_t childlen; -struct xs_permissions perms[0 ]; -}; -/* Is A == B ? */ #define streq (a,b) (strcmp((a),(b)) == 0)tools/xenstore/xenstored_core.c +3
-3@@ -555,9 +555,9 @@ static void initialize_fds(int *p_sock_pollfd_idx, int *ptimeout) } } -const struct xs_tdb_record_hdr *db_fetch (const char *db_name,size_t *size) +const struct node_hdr *db_fetch (const char *db_name,size_t *size) { -struct xs_tdb_record_hdr *hdr; +struct node_hdr *hdr; hdr =hashtable_search (nodes, db_name);if (!hdr) {@@ -565,7 +565,7 @@ const struct xs_tdb_record_hdr *db_fetch(const char *db_name, size_t *size)return NULL ; } - *size =sizeof (*hdr) + hdr->num_perms *sizeof (hdr->perms[0 ]) + + *size =sizeof (*hdr) + hdr->num_perms *sizeof (struct xs_permissions ) + hdr->datalen + hdr->childlen;trace_tdb ("read %s size %zu\n" , db_name, *size +strlen (db_name));
tools/xenstore/xenstored_core.c const struct xs_tdb_record_hdr *db_fetch(const char *db_name, size_t *size) · line 576
Quoted 4 lines
tools/xenstore/xenstored_core.c +1
-0@@ -573,10 +573,15 @@ const struct xs_tdb_record_hdr *db_fetch(const char *db_name, size_t *size)return hdr; } +static struct xs_permissions *perms_from_node_hdr (const struct node_hdr *hdr)
I don't much like the casting-away the const here. Looking at the code, most of the users seems to not modify the value. So how about return const here and open-coding or casting-away the const in the caller (with a proper explanation)?
Quoted 106 lines
tools/xenstore/xenstored_core.c +16
-12+{ +return (struct xs_permissions *)(hdr +1 ); +} +static void get_acc_data (const char *name,struct node_account_data *acc) {size_t size; -const struct xs_tdb_record_hdr *hdr; +const struct node_hdr *hdr;if (acc->memory <0 ) { hdr =db_fetch (name, &size);@@ -585,7 +590,7 @@ static void get_acc_data(const char *name, struct node_account_data *acc) acc->memory =0 ; }else { acc->memory = size; - acc->domid = hdr->perms[0 ].id; + acc->domid =perms_from_node_hdr (hdr)->id; } } }@@ -606,7 +611,7 @@ int db_write(struct connection *conn, const char *db_name, const void *data, size_t size, struct node_account_data *acc, enum write_node_mode mode, bool no_quota_check) { - const struct xs_tdb_record_hdr *hdr = data; + const struct node_hdr *hdr = data;struct node_account_data old_acc = {};unsigned int old_domid, new_domid;size_t name_len =strlen (db_name);@@ -620,7 +625,7 @@ int db_write(struct connection *conn, const char *db_name, const void *data,get_acc_data (db_name, &old_acc); old_domid =get_acc_domid (conn, db_name, old_acc.domid); - new_domid =get_acc_domid (conn, db_name, hdr->perms[0 ].id); + new_domid =get_acc_domid (conn, db_name,perms_from_node_hdr (hdr)->id);/* * Don't check for ENOENT, as we want to be able to switch orphaned @@ -661,7 +666,7 @@ int db_write(struct connection *conn, const char *db_name, const void *data,if (acc) {/* Don't use new_domid, as it might be a transaction node. */ - acc->domid = hdr->perms[0 ].id; + acc->domid =perms_from_node_hdr (hdr)->id; acc->memory = size; }@@ -699,7 +704,7 @@ struct node *read_node(struct connection *conn, const void *ctx, const char *name) {size_t size; -const struct xs_tdb_record_hdr *hdr; +const struct node_hdr *hdr;struct node *node;const char *db_name;int err;@@ -733,12 +738,12 @@ struct node *read_node(struct connection *conn, const void *ctx, node->perms.num = hdr->num_perms; node->datalen = hdr->datalen; node->childlen = hdr->childlen; - node->acc.domid = hdr->perms[0 ].id; + node->acc.domid =perms_from_node_hdr (hdr)->id; node->acc.memory = size;/* Copy node data to new memory area, starting with permissions. */ size -=sizeof (*hdr); - node->perms.p =talloc_memdup (node, hdr->perms, size); + node->perms.p =talloc_memdup (node,perms_from_node_hdr (hdr), size);if (node->perms.p ==NULL ) { errno =ENOMEM ; goto error;@@ -785,7 +790,7 @@ int write_node_raw(struct connection *conn, const char *db_name,void *data;size_t size;void *p; -struct xs_tdb_record_hdr *hdr; +struct node_hdr *hdr;if (domain_adjust_node_perms (node))return errno;@@ -812,9 +817,9 @@ int write_node_raw(struct connection *conn, const char *db_name, hdr->datalen = node->datalen; hdr->childlen = node->childlen; -memcpy (hdr->perms, node->perms.p, - node->perms.num *sizeof (*node->perms.p)); - p = hdr->perms + node->perms.num; + p =perms_from_node_hdr (hdr); +memcpy (p, node->perms.p, node->perms.num *sizeof (*node->perms.p)); + p += node->perms.num *sizeof (*node->perms.p);memcpy (p, node->data, node->datalen); p += node->datalen;memcpy (p, node->children, node->childlen);tools/xenstore/xenstored_core.h +4
-0@@ -168,6 +168,24 @@ struct connection };extern struct list_head connections; +/* +* Header of the node record in the data base. +* In the data base the memory of the node is a single memory chunk with the +* following format:
tools/xenstore/xenstored_core.h struct connection · line 184
Quoted 10 lines
tools/xenstore/xenstored_core.h +10
-0+* struct { +* node_hdr hdr; +* struct xs_permissions perms[hdr.num_perms]; +* char data[hdr.datalen]; +* char children[hdr.childlen]; +* }; +*/ +struct node_hdr { +uint64_t generation; +uint16_t num_perms;
OOI, do you have a use case where a node would be shared with more than 255 domains?
Quoted 34 lines
tools/xenstore/xenstored_core.h +5
-1+uint16_t datalen; +uint32_t childlen; +}; +struct node_perms {unsigned int num;struct xs_permissions *p;@@ -362,7 +380,7 @@ extern xengnttab_handle **xgt_handle;int remember_string (struct hashtable *hash,const char *str);/* Data base access functions. */ -const struct xs_tdb_record_hdr *db_fetch (const char *db_name,size_t *size); +const struct node_hdr *db_fetch (const char *db_name,size_t *size);int db_write (struct connection *conn,const char *db_name,const void *data,size_t size,struct node_account_data *acc,enum write_node_mode mode,bool no_quota_check);tools/xenstore/xenstored_transaction.c +1
-2@@ -357,7 +357,7 @@ static int finalize_transaction(struct connection *conn, {struct accessed_node *i, *n;size_t size; -const struct xs_tdb_record_hdr *hdr; +const struct node_hdr *hdr;uint64_t gen;list_for_each_entry_safe (i, n, &trans->accessed, list) {@@ -394,12 +394,12 @@ static int finalize_transaction(struct connection *conn,* generation count. */ enum write_node_mode mode; -struct xs_tdb_record_hdr *own;
tools/xenstore/xenstored_transaction.c static int finalize_transaction(struct connection *conn, · line 405
Quoted 10 lines
tools/xenstore/xenstored_transaction.c +2
-1+struct node_hdr *own;talloc_increase_ref_count (hdr);db_delete (conn, i->trans_name,NULL ); - own = (struct xs_tdb_record_hdr *)hdr; + own = (struct node_hdr *)hdr; own->generation = ++generation; mode = (i->generation ==NO_GENERATION ) ?NODE_CREATE :NODE_MODIFY ;
Cheers,
Signature · 1 line
Julien Grall
Re: [PATCH v3 17/25] tools/xenstore: rework struct xs_tdb_record_hdr
Quoted 21 lines · Julien Grall
Hi Juergen,
Quoted 18 lines · Juergen Gross
Struct
xs_tdb_record_hdris used for nodes stored in the data base. When working on a node, struct node is being used, which is including the same information as structxs_tdb_record_hdr, but in a different format. Rework structxs_tdb_record_hdrin order to prepare including it in struct node.Do the following modifications:
- move its definition to
xenstored_core.h, as the reason to put it intoutils.hare no longer existing- rename it to struct
node_hdr, as the "tdb" in its name has only historical reasons- replace the empty permission array at the end with a comment about the layout of data in the data base (concatenation of header, permissions, node contents, and children list)
Quoted 7 lines
Quoted 3 lines
- use narrower types for
num_permsand datalen, as those are naturally limited toXENSTORE_PAYLOAD_MAX(childlen is different here, as it is in theory basically unlimited)The assumption is
XENSTORE_PAYLOAD_MAXwill never change and/or always apply for all the connection. I am aware of at least one downstream use where this is not the case.I am happy to use narrower types, but I would at least like some checks in Xenstore to ensure the limits are not reached.
I will add a BUILD_BUG_ON().
Quoted 52 lines
Quoted 52 lines
Signed-off-by Juergen Gross <[email protected]>---
V2:
- new patch
---
tools/xenstore/utils.h | 9 ------- tools/xenstore/xenstored_core.c | 35 +++++++++++++++----------- tools/xenstore/xenstored_core.h | 20 ++++++++++++++-tools/xenstore/xenstored_transaction.c| 6 ++---4 files changed, 42 insertions(+), 28 deletions(-)tools/xenstore/utils.h +0
-9@@ -9,15 +9,6 @@ #include "xenstore_lib.h" -/* Header of the node record in tdb. */ -struct xs_tdb_record_hdr { - uint64_t generation; - uint32_t num_perms; - uint32_t datalen; - uint32_t childlen; - struct xs_permissions perms[0]; -}; - /* Is A == B ? */ #define streq(a,b) (strcmp((a),(b)) == 0)tools/xenstore/xenstored_core.c +3
-3@@ -555,9 +555,9 @@ static void initialize_fds(int *p_sock_pollfd_idx, int *ptimeout) } } -const struct xs_tdb_record_hdr *db_fetch (const char *db_name,size_t *size) +const struct node_hdr *db_fetch (const char *db_name,size_t *size) { - struct xs_tdb_record_hdr *hdr; + struct node_hdr *hdr; hdr = hashtable_search(nodes, db_name); if (!hdr) {@@ -565,7 +565,7 @@ const struct xs_tdb_record_hdr *db_fetch(const char *db_name, size_t *size) return NULL; } - *size = sizeof(*hdr) + hdr->num_perms * sizeof(hdr->perms[0]) + + *size = sizeof(*hdr) + hdr->num_perms * sizeof(struct xs_permissions) + hdr->datalen + hdr->childlen;
Quoted 8 lines
tools/xenstore/xenstored_core.c const struct xs_tdb_record_hdr *db_fetch(const char · line 576
Quoted 6 lines
tools/xenstore/xenstored_core.c +1
-0trace_tdb("read %s size %zu\n", db_name, *size + strlen(db_name));@@ -573,10 +573,15 @@ const struct xs_tdb_record_hdr *db_fetch(const char *db_name, size_t *size) return hdr; } +static struct xs_permissions *perms_from_node_hdr (const struct node_hdr *hdr)I don't much like the casting-away the const here. Looking at the code, most of the users seems to not modify the value. So how about return const here and open-coding or casting-away the const in the caller (with a proper explanation)?
I'll look into that.
Quoted 110 lines
Quoted 110 lines
tools/xenstore/xenstored_core.c +6
-2+{ + return (struct xs_permissions *)(hdr + 1); +} + static void get_acc_data(const char *name, struct node_account_data *acc) { size_t size; - const struct xs_tdb_record_hdr *hdr; + const struct node_hdr *hdr; if (acc->memory < 0) { hdr = db_fetch(name, &size);@@ -585,7 +590,7 @@ static void get_acc_data(const char *name, struct node_account_data *acc) acc->memory = 0; } else { acc->memory = size; - acc->domid = hdr->perms[0].id; + acc->domid = perms_from_node_hdr(hdr)->id; } }}
tools/xenstore/xenstored_core.c +1
-1@@ -606,7 +611,7 @@ int db_write(struct connection *conn, const char *db_name,const void *data, size_t size, struct node_account_data *acc, enum write_node_mode mode, bool no_quota_check) { - const struct xs_tdb_record_hdr *hdr = data; + const struct node_hdr *hdr = data; struct node_account_data old_acc = {}; unsigned int old_domid, new_domid;
size_tname_len=strlen(db_name);tools/xenstore/xenstored_core.c +3
-3@@ -620,7 +625,7 @@ int db_write(struct connection *conn, const char *db_name,const void *data, get_acc_data(db_name, &old_acc); old_domid = get_acc_domid(conn, db_name, old_acc.domid); - new_domid = get_acc_domid(conn, db_name, hdr->perms[0].id); + new_domid = get_acc_domid(conn, db_name, perms_from_node_hdr(hdr)->id); /** Don't check for ENOENT, as we want to be able to switch orphaned @@ -661,7 +666,7 @@ int db_write(struct connection *conn, const char *db_name,const void *data, if (acc) { /* Don't use new_domid, as it might be a transaction node. */ - acc->domid = hdr->perms[0].id; + acc->domid = perms_from_node_hdr(hdr)->id; acc->memory = size; }@@ -699,7 +704,7 @@ struct node *read_node(struct connection *conn, const void *ctx, const char *name) { size_t size; - const struct xs_tdb_record_hdr *hdr; + const struct node_hdr *hdr; struct node *node; const char *db_name;int err;
tools/xenstore/xenstored_core.c +6
-6@@ -733,12 +738,12 @@ struct node *read_node(struct connection *conn, constvoid *ctx, node->perms.num = hdr->num_perms; node->datalen = hdr->datalen; node->childlen = hdr->childlen; - node->acc.domid = hdr->perms[0].id; + node->acc.domid = perms_from_node_hdr(hdr)->id; node->acc.memory = size; /* Copy node data to new memory area, starting with permissions. */ size -= sizeof(*hdr); - node->perms.p = talloc_memdup(node, hdr->perms, size); + node->perms.p = talloc_memdup(node, perms_from_node_hdr(hdr), size); if (node->perms.p == NULL) { errno = ENOMEM; goto error;@@ -785,7 +790,7 @@ int write_node_raw(struct connection *conn, const char *db_name, void *data; size_t size; void *p; - struct xs_tdb_record_hdr *hdr; + struct node_hdr *hdr; if (domain_adjust_node_perms(node)) return errno;@@ -812,9 +817,9 @@ int write_node_raw(struct connection *conn, const char *db_name, hdr->datalen = node->datalen; hdr->childlen = node->childlen; - memcpy(hdr->perms, node->perms.p, - node->perms.num * sizeof(*node->perms.p)); - p = hdr->perms + node->perms.num; + p = perms_from_node_hdr(hdr); + memcpy(p, node->perms.p, node->perms.num * sizeof(*node->perms.p)); + p += node->perms.num * sizeof(*node->perms.p); memcpy(p, node->data, node->datalen); p += node->datalen; memcpy(p, node->children, node->childlen);tools/xenstore/xenstored_core.h +8
-0@@ -168,6 +168,24 @@ struct connection }; extern struct list_head connections; +/* +* Header of the node record in the data base. +* In the data base the memory of the node is a single memory chunk with the +* following format: +* struct { +* node_hdr hdr; +* struct xs_permissions perms[hdr.num_perms]; +* char data[hdr.datalen];
Quoted 8 lines
tools/xenstore/xenstored_core.h struct connection · line 184
Quoted 6 lines
tools/xenstore/xenstored_core.h +6
-0+* char children[hdr.childlen]; +* }; +*/ +struct node_hdr { + uint64_t generation; + uint16_t num_perms;OOI, do you have a use case where a node would be shared with more than 255 domains?
No, but limiting it wouldn't give any real advantage.
Quoted 33 lines
Quoted 33 lines
tools/xenstore/xenstored_core.h +5
-1+ uint16_t datalen; + uint32_t childlen; +}; + struct node_perms { unsigned int num; struct xs_permissions *p;@@ -362,7 +380,7 @@ extern xengnttab_handle **xgt_handle; int remember_string(struct hashtable *hash, const char *str); /* Data base access functions. */ -const struct xs_tdb_record_hdr *db_fetch (const char *db_name,size_t *size); +const struct node_hdr *db_fetch (const char *db_name,size_t *size); int db_write(struct connection *conn, const char *db_name, const void *data, size_t size, struct node_account_data *acc, enum write_node_mode mode, bool no_quota_check);tools/xenstore/xenstored_transaction.c +1
-2@@ -357,7 +357,7 @@ static int finalize_transaction(struct connection *conn, { struct accessed_node *i, *n; size_t size; - const struct xs_tdb_record_hdr *hdr; + const struct node_hdr *hdr; uint64_t gen; list_for_each_entry_safe(i, n, &trans->accessed, list) {@@ -394,12 +394,12 @@ static int finalize_transaction(struct connection *conn,* generation count. */ enum write_node_mode mode; - struct xs_tdb_record_hdr *own;
Quoted 8 lines
tools/xenstore/xenstored_transaction.c static int finalize_transaction(struct connection *conn, · line 405
Quoted 8 lines
tools/xenstore/xenstored_transaction.c +2
-1+ struct node_hdr *own; talloc_increase_ref_count(hdr); db_delete(conn, i->trans_name, NULL); - own = (struct xs_tdb_record_hdr *)hdr; + own = (struct node_hdr *)hdr; own->generation = ++generation; mode = (i->generation == NO_GENERATION) ? NODE_CREATE : NODE_MODIFY;
Juergen
Re: [PATCH v3 17/25] tools/xenstore: rework struct xs_tdb_record_hdr
Hi Juergen,
Quoted 26 lines · Juergen Gross
Quoted 25 lines · Julien Grall
Hi Juergen,
Quoted 21 lines · Juergen Gross
Struct
xs_tdb_record_hdris used for nodes stored in the data base. When working on a node, struct node is being used, which is including the same information as structxs_tdb_record_hdr, but in a different format. Rework structxs_tdb_record_hdrin order to prepare including it in struct node.Do the following modifications:
- move its definition to
xenstored_core.h, as the reason to put it intoutils.hare no longer existing- rename it to struct
node_hdr, as the "tdb" in its name has only historical reasons- replace the empty permission array at the end with a comment about the layout of data in the data base (concatenation of header, permissions, node contents, and children list)
- use narrower types for
num_permsand datalen, as those are naturally limited toXENSTORE_PAYLOAD_MAX(childlen is different here, as it is in theory basically unlimited)
Quoted 5 lines
Quoted 3 lines
The assumption is
XENSTORE_PAYLOAD_MAXwill never change and/or always apply for all the connection. I am aware of at least one downstream use where this is not the case.I am happy to use narrower types, but I would at least like some checks in Xenstore to ensure the limits are not reached.
I will add a
BUILD_BUG_ON().
Initially I was thinking about a runtime check, but I am also fine with a BUILD_BUG_ON() if it is right next to length check in handle_input(). So if someone decided to add different payload size depending on the domain (such as privileged domain could do more), it would be easier to spot what else needs to be changed.
Quoted 3 lines
Quoted 1 line
OOI, do you have a use case where a node would be shared with more than 255 domains?
No, but limiting it wouldn't give any real advantage.
I guess by advantage you mean that the size of the structure would still be the same. I thought this was the rationale but I asked just in case you had something else in mind. For instance, Xen technically supports up to ~32 000 domains. But I think it would be crazy to decide to have more than a few tens permissions on a node :).
Cheers,
Signature · 1 line
Julien Grall
Re: [PATCH v3 17/25] tools/xenstore: rework struct xs_tdb_record_hdr
Quoted 29 lines · Julien Grall
Hi Juergen,
Quoted 26 lines · Juergen Gross
Quoted 25 lines · Julien Grall
Hi Juergen,
Quoted 21 lines · Juergen Gross
Struct
xs_tdb_record_hdris used for nodes stored in the data base. When working on a node, struct node is being used, which is including the same information as structxs_tdb_record_hdr, but in a different format. Rework structxs_tdb_record_hdrin order to prepare including it in struct node.Do the following modifications:
- move its definition to
xenstored_core.h, as the reason to put it intoutils.hare no longer existing- rename it to struct
node_hdr, as the "tdb" in its name has only historical reasons- replace the empty permission array at the end with a comment about the layout of data in the data base (concatenation of header, permissions, node contents, and children list)
- use narrower types for
num_permsand datalen, as those are naturally limited toXENSTORE_PAYLOAD_MAX(childlen is different here, as it is in theory basically unlimited)
Quoted 7 lines
Quoted 5 lines
Quoted 3 lines
The assumption is
XENSTORE_PAYLOAD_MAXwill never change and/or always apply for all the connection. I am aware of at least one downstream use where this is not the case.I am happy to use narrower types, but I would at least like some checks in Xenstore to ensure the limits are not reached.
I will add a
BUILD_BUG_ON().Initially I was thinking about a runtime check, but I am also fine with a
BUILD_BUG_ON()if it is right next to length check inhandle_input(). So if someone decided to add different payload size depending on the domain (such as privileged domain could do more), it would be easier to spot what else needs to be changed.
Is this really the correct placement?
I've put it into write_node_raw(), as this is where the related datalen member is being filled. This placement has the further advantage that I already have a related pointer at hand, so I can use:
BUILD_BUG_ON(XENSTORE_PAYLOAD_MAX >= (typeof(hdr->datalen))(-1));
which is exactly what should be tested.
Quoted 6 lines
Quoted 4 lines
Quoted 2 lines
OOI, do you have a use case where a node would be shared with more than 255 domains?
No, but limiting it wouldn't give any real advantage.
I guess by advantage you mean that the size of the structure would still be the same. I thought this was the rationale but I asked just in case you had something else in mind. For instance, Xen technically supports up to ~32 000 domains. But I think it would be crazy to decide to have more than a few tens permissions on a node :).
Yes to both. OTOH why should we deny to allow that without any urgent need to do so?
Juergen
Re: [PATCH v3 17/25] tools/xenstore: rework struct xs_tdb_record_hdr
Hi Juergen,
Quoted 33 lines · Juergen Gross
Quoted 32 lines · Julien Grall
Hi Juergen,
Quoted 29 lines · Juergen Gross
Quoted 28 lines · Julien Grall
Hi Juergen,
Quoted 15 lines · Juergen Gross
Struct
xs_tdb_record_hdris used for nodes stored in the data base. When working on a node, struct node is being used, which is including the same information as structxs_tdb_record_hdr, but in a different format. Rework structxs_tdb_record_hdrin order to prepare including it in struct node.Do the following modifications:
- move its definition to
xenstored_core.h, as the reason to put it into
utils.hare no longer existing- rename it to struct
node_hdr, as the "tdb" in its name has only historical reasonsQuoted 7 lines
- replace the empty permission array at the end with a comment about the layout of data in the data base (concatenation of header, permissions, node contents, and children list)
- use narrower types for
num_permsand datalen, as those are naturally limited toXENSTORE_PAYLOAD_MAX(childlen is different here, as it is in theory basically unlimited)The assumption is
XENSTORE_PAYLOAD_MAXwill never change and/or always apply for all the connection. I am aware of at least one downstream use where this is not the case.
Quoted 7 lines
Quoted 5 lines
Quoted 3 lines
Quoted 1 line
I am happy to use narrower types, but I would at least like some checks in Xenstore to ensure the limits are not reached.
I will add a
BUILD_BUG_ON().Initially I was thinking about a runtime check, but I am also fine with a
BUILD_BUG_ON()if it is right next to length check inhandle_input(). So if someone decided to add different payload size depending on the domain (such as privileged domain could do more), it would be easier to spot what else needs to be changed.Is this really the correct placement?
I think so. By adding the BUILD_ON_BUG() right above the length check, it would be easier for everyone to know that the datastructure may also need to change. This ...
Quoted 3 lines
I've put it into
write_node_raw(), as this is where the related datalen member is being filled.
... would be less obvious here and I think it might be miss.
Cheers,
Signature · 1 line
Julien Grall
Re: [PATCH v3 17/25] tools/xenstore: rework struct xs_tdb_record_hdr
Quoted 43 lines · Julien Grall
Hi Juergen,
Quoted 32 lines · Juergen Gross
Quoted 31 lines · Julien Grall
Hi Juergen,
Quoted 28 lines · Juergen Gross
Quoted 27 lines · Julien Grall
Hi Juergen,
Quoted 14 lines · Juergen Gross
Struct
xs_tdb_record_hdris used for nodes stored in the data base. When working on a node, struct node is being used, which is including the same information as structxs_tdb_record_hdr, but in a different format. Rework structxs_tdb_record_hdrin order to prepare including it in struct node.Do the following modifications:
- move its definition to
xenstored_core.h, as the reason to put it intoutils.hare no longer existing- rename it to struct
node_hdr, as the "tdb" in its name has only historical reasonsQuoted 7 lines
- replace the empty permission array at the end with a comment about the layout of data in the data base (concatenation of header, permissions, node contents, and children list)
- use narrower types for
num_permsand datalen, as those are naturally limited toXENSTORE_PAYLOAD_MAX(childlen is different here, as it is in theory basically unlimited)The assumption is
XENSTORE_PAYLOAD_MAXwill never change and/or always apply for all the connection. I am aware of at least one downstream use where this is not the case.Quoted 7 lines
Quoted 5 lines
Quoted 3 lines
Quoted 1 line
I am happy to use narrower types, but I would at least like some checks in Xenstore to ensure the limits are not reached.
I will add a
BUILD_BUG_ON().Initially I was thinking about a runtime check, but I am also fine with a
BUILD_BUG_ON()if it is right next to length check inhandle_input(). So if someone decided to add different payload size depending on the domain (such as privileged domain could do more), it would be easier to spot what else needs to be changed.Is this really the correct placement?
Quoted 7 lines
I think so. By adding the
BUILD_ON_BUG()right above the length check, it would be easier for everyone to know that the datastructure may also need to change. This ...Quoted 3 lines
I've put it into
write_node_raw(), as this is where the related datalen member is being filled.... would be less obvious here and I think it might be miss.
Assuming that someone changing XENSTORE_PAYLOAD_MAX would do a build afterwards, I don't see how such a failure could be missed. In case of a runtime check I agree that a more central place would be preferred.
In the end I don't mind that much, but
BUILD_BUG_ON(XENSTORE_PAYLOAD_MAX >= (typeof((struct node_hdr *)NULL->datalen))(-1));
is a little bit clumsy IMHO.
Juergen
Re: [PATCH v3 17/25] tools/xenstore: rework struct xs_tdb_record_hdr
Hi,
Quoted 49 lines · Juergen Gross
Quoted 48 lines · Julien Grall
Hi Juergen,
Quoted 34 lines · Juergen Gross
Quoted 33 lines · Julien Grall
Hi Juergen,
Quoted 30 lines · Juergen Gross
Quoted 29 lines · Julien Grall
Hi Juergen,
Quoted 15 lines · Juergen Gross
Struct
xs_tdb_record_hdris used for nodes stored in the data base. When working on a node, struct node is being used, which is including the same information as structxs_tdb_record_hdr, but in a different format. Rework structxs_tdb_record_hdrin order to prepare including it in struct node.Do the following modifications:
- move its definition to
xenstored_core.h, as the reason to put it into
utils.hare no longer existing- rename it to struct
node_hdr, as the "tdb" in its name has only historical reasonsQuoted 8 lines
- replace the empty permission array at the end with a comment about the layout of data in the data base (concatenation of header, permissions, node contents, and children list)
- use narrower types for
num_permsand datalen, as those are naturally
limited toXENSTORE_PAYLOAD_MAX(childlen is different here, as it is in theory basically unlimited)The assumption is
XENSTORE_PAYLOAD_MAXwill never change and/or always apply for all the connection. I am aware of at least one downstream use where this is not the case.Quoted 7 lines
Quoted 5 lines
Quoted 3 lines
Quoted 1 line
I am happy to use narrower types, but I would at least like some checks in Xenstore to ensure the limits are not reached.
I will add a
BUILD_BUG_ON().Initially I was thinking about a runtime check, but I am also fine with a
BUILD_BUG_ON()if it is right next to length check inhandle_input(). So if someone decided to add different payload size depending on the domain (such as privileged domain could do more), it would be easier to spot what else needs to be changed.Is this really the correct placement?
I think so. By adding the
BUILD_ON_BUG()right above the length check, it would be easier for everyone to know that the datastructure may also need to change. This ...Quoted 1 line
Quoted 7 lines
Quoted 4 lines
Quoted 2 lines
I've put it into
write_node_raw(), as this is where the related datalen member is being filled.... would be less obvious here and I think it might be miss.
Assuming that someone changing
XENSTORE_PAYLOAD_MAXwould do a build afterwards, I don't see how such a failure could be missed.
Because one may want dom0 to send payload bigger than XENSTORE_PAYLOAD_MAX. Something like:
if ( conn->id != 0 && len < XENSTORE_PAYLOAD_MAX )
Such change would not be caught during compilation. AWS has a similar check in the downstream tree because the implementation of non-cooperative migration is using Xenstore command and we want to be able to transfer the state in a single command.
Quoted 9 lines
In case of a runtime check I
agree that a more central place would be preferred.In the end I don't mind that much, but
BUILD_BUG_ON(XENSTORE_PAYLOAD_MAX >= (
typeof((struct node_hdr *)NULL->datalen))(-1));is a little bit clumsy IMHO.
Agree. We could introduce FIELD_SIZEOF() (as Linux did) to hide the complexity. The code would then look like:
Quoted 1 line
= (8 *
FIELD_SIZEOF(struct node_hdr, datalen))
Cheers,
Signature · 1 line
Julien Grall
Re: [PATCH v3 17/25] tools/xenstore: rework struct xs_tdb_record_hdr
Quoted 55 lines · Julien Grall
Hi,
Quoted 52 lines · Juergen Gross
Quoted 50 lines · Julien Grall
Hi Juergen,
Quoted 32 lines · Juergen Gross
Quoted 31 lines · Julien Grall
Hi Juergen,
Quoted 28 lines · Juergen Gross
Quoted 27 lines · Julien Grall
Hi Juergen,
Quoted 14 lines · Juergen Gross
Struct
xs_tdb_record_hdris used for nodes stored in the data base. When working on a node, struct node is being used, which is including the same information as structxs_tdb_record_hdr, but in a different format. Rework structxs_tdb_record_hdrin order to prepare including it in struct node.Do the following modifications:
- move its definition to
xenstored_core.h, as the reason to put it intoutils.hare no longer existing- rename it to struct
node_hdr, as the "tdb" in its name has only historical reasonsQuoted 7 lines
- replace the empty permission array at the end with a comment about the layout of data in the data base (concatenation of header, permissions, node contents, and children list)
- use narrower types for
num_permsand datalen, as those are naturally limited toXENSTORE_PAYLOAD_MAX(childlen is different here, as it is in theory basically unlimited)The assumption is
XENSTORE_PAYLOAD_MAXwill never change and/or always apply for all the connection. I am aware of at least one downstream use where this is not the case.Quoted 7 lines
Quoted 5 lines
Quoted 3 lines
Quoted 1 line
I am happy to use narrower types, but I would at least like some checks in Xenstore to ensure the limits are not reached.
I will add a
BUILD_BUG_ON().Initially I was thinking about a runtime check, but I am also fine with a
BUILD_BUG_ON()if it is right next to length check inhandle_input(). So if someone decided to add different payload size depending on the domain (such as privileged domain could do more), it would be easier to spot what else needs to be changed.Is this really the correct placement?
I think so. By adding the
BUILD_ON_BUG()right above the length check, it would be easier for everyone to know that the datastructure may also need to change. This ...Quoted 3 lines
I've put it into
write_node_raw(), as this is where the related datalen member is being filled.... would be less obvious here and I think it might be miss.
Quoted 8 lines
Quoted 2 lines
Assuming that someone changing
XENSTORE_PAYLOAD_MAXwould do a build afterwards, I don't see how such a failure could be missed.Because one may want dom0 to send payload bigger than
XENSTORE_PAYLOAD_MAX. Something like:if (
conn->id!= 0 && len <XENSTORE_PAYLOAD_MAX)Such change would not be caught during compilation. AWS has a similar check in the downstream tree because the implementation of non-cooperative migration is using Xenstore command and we want to be able to transfer the state in a single command.
And how directly is this related to the max data size of node contents?
As soon as you are allowing dom0 to write larger nodes, you are risking to kill client connections trying to read such a node. So the node size should still be limited to XENSTORE_PAYLOAD_MAX.
IMO another reason to use the placement I've suggested. AWS should even add a size check when writing nodes to make sure dom0 doesn't do evil things.
Quoted 4 lines
Quoted 4 lines
In case of a runtime check I
agree that a more central place would be preferred.
Quoted 10 lines
Quoted 6 lines
In the end I don't mind that much, but
BUILD_BUG_ON(XENSTORE_PAYLOAD_MAX >= (
typeof((struct node_hdr *)NULL->datalen))(-1));is a little bit clumsy IMHO.
Agree. We could introduce
FIELD_SIZEOF()(as Linux did) to hide the complexity. The code would then look like:Quoted 1 line
= (8 *
FIELD_SIZEOF(struct node_hdr, datalen))
Oh, I guess you mean sizeof_field().
And even with that it would look quite clumsy:
BUILD_BUG_ON(XENSTORE_PAYLOAD_MAX >= (1UL << (8 * sizeof_field(struct node_hdr, datalen))));
Juergen
Re: [PATCH v3 17/25] tools/xenstore: rework struct xs_tdb_record_hdr
Quoted 8 lines · Juergen Gross
Quoted 5 lines · Julien Grall
Because one may want dom0 to send payload bigger than
XENSTORE_PAYLOAD_MAX. Something like:if (
conn->id!= 0 && len <XENSTORE_PAYLOAD_MAX)Such change would not be caught during compilation. AWS has a similar check in the downstream tree because the implementation of non-cooperative migration is using Xenstore command and we want to be able to transfer the state in a single command.
And how directly is this related to the max data size of node contents?
I think you missed my point. Until this patch, the existing field would be able to accomodate very large payload. This doesn't hold anymore.
What I was trying to convey is that anyone looking at relaxing the check in handle_input() needs to be able to find "easily" that other part of Xenstored are making some assumption based on the maximum length.
Quoted 6 lines
As soon as you are allowing dom0 to write larger nodes, you are risking to kill client connections trying to read such a node. So the node size should still be limited to
XENSTORE_PAYLOAD_MAX.IMO another reason to use the placement I've suggested.
I agree that BUILD_BUG_ON() makes sense where you suggest if you think about where the runtime check would happen.
It seems like we have two different aims here. Mine is to make sure we make it more difficult to introduce a security hole if the lenght check is relaxed.
I have made a proposal below that may suit both our aim.
Quoted 2 lines
AWS should even add
a size check when writing nodes to make sure dom0 doesn't do evil things.
What make you think we don't already have such checked? ;)
Also, I noticed you mention about datalen. What about the number of permissions?
Quoted 14 lines
Quoted 14 lines
Quoted 11 lines
In case of a runtime check I
agree that a more central place would be preferred.In the end I don't mind that much, but
BUILD_BUG_ON(XENSTORE_PAYLOAD_MAX >= (
typeof((struct node_hdr *)NULL->datalen))(-1));is a little bit clumsy IMHO.
Agree. We could introduce
FIELD_SIZEOF()(as Linux did) to hide the complexity. The code would then look like:
Quoted 8 lines
Quoted 1 line
>= (8 *
FIELD_SIZEOF(struct node_hdr, datalen))Oh, I guess you mean
sizeof_field().And even with that it would look quite clumsy:
BUILD_BUG_ON(XENSTORE_PAYLOAD_MAX >=
(1UL << (8 *sizeof_field(struct node_hdr, datalen))));
How about keeping the BUILD_BUG_ON() in write_node_raw() and add the following comment on top of handle_input():
Some fields in Xenstored are sized based on the max payload (see various BUILD_BUG_ON()). This would need extra runtime check if we ever decide to have a dynamic payload size.
Cheers,
Signature · 1 line
Julien Grall
Re: [PATCH v3 17/25] tools/xenstore: rework struct xs_tdb_record_hdr
Quoted 6 lines · Julien Grall
Quoted 3 lines · Juergen Gross
Quoted 2 lines · Julien Grall
Because one may want dom0 to send payload bigger than
XENSTORE_PAYLOAD_MAX. Something like:
Quoted 7 lines
Quoted 5 lines
Quoted 3 lines
if (
conn->id!= 0 && len <XENSTORE_PAYLOAD_MAX)Such change would not be caught during compilation. AWS has a similar check in the downstream tree because the implementation of non-cooperative migration is using Xenstore command and we want to be able to transfer the state in a single command.
And how directly is this related to the max data size of node contents?
I think you missed my point. Until this patch, the existing field would be able to accomodate very large payload. This doesn't hold anymore.
Yes. And the field is related to node data only, so it should be checked to be large enough where it is written, as this is the critical operation where truncation would happen.
Quoted 1 line
What I was trying to convey is that anyone looking at relaxing the check in
handle_input()needs to be able to find "easily" that other part of Xenstored are making some assumption based on the maximum length.
As the AWS example is showing, the limitation isn't in handle_input().
And TBH: allowing sometimes a payload larger than XENSTORE_PAYLOAD_MAX isn't something we should encourage. This is rather a very bad design decision.
Quoted 9 lines
Quoted 7 lines
As soon as you are allowing dom0 to write larger nodes, you are risking to kill client connections trying to read such a node. So the node size should still be limited to
XENSTORE_PAYLOAD_MAX.IMO another reason to use the placement I've suggested.
I agree that
BUILD_BUG_ON()makes sense where you suggest if you think about where the runtime check would happen.
Correct.
Quoted 1 line
It seems like we have two different aims here. Mine is to make sure we make it more difficult to introduce a security hole if the lenght check is relaxed.
Which (as written above) would set a very bad precedence.
XENSTORE_PAYLOAD_MAX should be a hard limit.
Quoted 6 lines
I have made a proposal below that may suit both our aim.
Quoted 2 lines
AWS should even add
a size check when writing nodes to make sure dom0 doesn't do evil things.What make you think we don't already have such checked? ;)
Just a wild guess. :-)
Quoted 1 line
Also, I noticed you mention about datalen. What about the number of permissions?
More than max number of domains? Now this would really be crazy. And it would need operations using more data than XENSTORE_PAYLOAD_MAX allows. :-)
Quoted 16 lines
Quoted 16 lines
Quoted 5 lines
Quoted 5 lines
In case of a runtime check I
agree that a more central place would be preferred.In the end I don't mind that much, but
Quoted 8 lines
Quoted 4 lines
BUILD_BUG_ON(XENSTORE_PAYLOAD_MAX >= (
typeof((struct node_hdr *)NULL->datalen))(-1));is a little bit clumsy IMHO.
Agree. We could introduce
FIELD_SIZEOF()(as Linux did) to hide the complexity. The code would then look like:>= (8 *
FIELD_SIZEOF(struct node_hdr, datalen))Oh, I guess you mean
sizeof_field().
Quoted 8 lines
Quoted 4 lines
And even with that it would look quite clumsy:
BUILD_BUG_ON(XENSTORE_PAYLOAD_MAX >=
(1UL << (8 *sizeof_field(struct node_hdr, datalen))));How about keeping the
BUILD_BUG_ON()inwrite_node_raw()and add the following comment on top ofhandle_input():Some fields in Xenstored are sized based on the max payload (see various
BUILD_BUG_ON()). This would need extra runtime check if we ever decide to have a dynamic payload size.
I _could_ do that, but where to stop adding such comments?
TBH, I really don't see the point doing that.
In case a patch came up upstream trying to violate XENSTORE_PAYLOAD_MAX I would surely NACK it.
In case we need payloads larger than XENSTORE_PAYLOAD_MAX we should split the related operation in multiple parts (see e.g. XS_DIRECTORY_PART or XS_CONTROL for uploading a new kernel to Xenstore-stubdom for live update). Which is, BTW, the way AWS should have handled the migration problem (transactions come to my mind in this context).
Juergen
Re: [PATCH v3 17/25] tools/xenstore: rework struct xs_tdb_record_hdr
Hi,
Quoted 19 lines · Juergen Gross
Quoted 18 lines · Julien Grall
Quoted 18 lines
Quoted 5 lines
Quoted 5 lines
In case of a runtime check I
agree that a more central place would be preferred.In the end I don't mind that much, but
Quoted 8 lines
Quoted 4 lines
BUILD_BUG_ON(XENSTORE_PAYLOAD_MAX >= (
typeof((struct node_hdr *)NULL->datalen))(-1));is a little bit clumsy IMHO.
Agree. We could introduce
FIELD_SIZEOF()(as Linux did) to hide the complexity. The code would then look like:>= (8 *
FIELD_SIZEOF(struct node_hdr, datalen))Oh, I guess you mean
sizeof_field().And even with that it would look quite clumsy:
Quoted 8 lines
Quoted 6 lines
Quoted 2 lines
BUILD_BUG_ON(XENSTORE_PAYLOAD_MAX >=
(1UL << (8 *sizeof_field(struct node_hdr, datalen))));How about keeping the
BUILD_BUG_ON()inwrite_node_raw()and add the following comment on top ofhandle_input():Some fields in Xenstored are sized based on the max payload (see various
BUILD_BUG_ON()). This would need extra runtime check if we ever decide to have a dynamic payload size.I _could_ do that, but where to stop adding such comments?
When someone other than the author is able to understand the code without too much effort. More comments never hurts, less will in the longer run (see below).
Quoted 5 lines
TBH, I really don't see the point doing that.
In case a patch came up upstream trying to violate
XENSTORE_PAYLOAD_MAXI would surely NACK it.
That's assuming you will still be around when this happens :). I am not wishing anything bad but the code will likely outlast any of us.
So we need to make easy for a future maintainers/reviewers to know the assumptions and implications of changing some of the limits.
Quoted 5 lines
In case we need payloads larger than
XENSTORE_PAYLOAD_MAXwe should split the related operation in multiple parts (see e.g.XS_DIRECTORY_PARTorXS_CONTROLfor uploading a new kernel to Xenstore-stubdom for live update). Which is, BTW, the way AWS should have handled the migration problem (transactions come to my mind in this context).
I wasn't part of the original design, but I can see why it was done like that.
Using multiple commands has also its downside. The first that comes to my mind if that you need to keep around the data. But, with your proposal, you we wouldn't be able to store it in the database (like for transaction update) as datalen can only be 65KB.
So one command as the advantage to simply a lot the logic in Xenstored.
Anyway, this is getting a bit off topic. My only request is to write down assumption more explicitly rather than hiding them. A comment on top of the check is a nice way to help the developper to avoid making a "bad" decision.
I am happy to rewrite the comment so it doesn't lead to think that you (as the maintainer) are open to have a more relax length check.
Cheers,
Signature · 1 line
Julien Grall
Re: [PATCH v3 17/25] tools/xenstore: rework struct xs_tdb_record_hdr
Quoted 25 lines · Julien Grall
Hi,
Quoted 22 lines · Juergen Gross
Quoted 21 lines · Julien Grall
Quoted 20 lines
Quoted 5 lines
Quoted 5 lines
In case of a runtime check I
agree that a more central place would be preferred.In the end I don't mind that much, but
Quoted 8 lines
Quoted 4 lines
BUILD_BUG_ON(XENSTORE_PAYLOAD_MAX >= (
typeof((struct node_hdr *)NULL->datalen))(-1));is a little bit clumsy IMHO.
Agree. We could introduce
FIELD_SIZEOF()(as Linux did) to hide the complexity. The code would then look like:>= (8 *
FIELD_SIZEOF(struct node_hdr, datalen))Oh, I guess you mean
sizeof_field().And even with that it would look quite clumsy:
BUILD_BUG_ON(XENSTORE_PAYLOAD_MAX >=
(1UL << (8 *sizeof_field(struct node_hdr, datalen))));
Quoted 7 lines
Quoted 5 lines
Quoted 3 lines
How about keeping the
BUILD_BUG_ON()inwrite_node_raw()and add the following comment on top ofhandle_input():Some fields in Xenstored are sized based on the max payload (see various
BUILD_BUG_ON()). This would need extra runtime check if we ever decide to have a dynamic payload size.I _could_ do that, but where to stop adding such comments?
When someone other than the author is able to understand the code without too much effort. More comments never hurts, less will in the longer run (see below).
I agree with that statement in general, but requesting a comment to aid a future potential change violating the Xenstore wire protocol is a little bit weird.
Quoted 7 lines
Quoted 6 lines
TBH, I really don't see the point doing that.
In case a patch came up upstream trying to violate
XENSTORE_PAYLOAD_MAXI would surely NACK it.That's assuming you will still be around when this happens :). I am not wishing anything bad but the code will likely outlast any of us.
Maybe. But would you really Ack patches adding comments like that in other areas? E.g. cases where the domid is stored in an unsigned int and me adding a comment to be careful in case domids ever grow to 64 bits?
Quoted 1 line
So we need to make easy for a future maintainers/reviewers to know the assumptions and implications of changing some of the limits.
Yes, that's what the BUILD_BUG_ON() is meant for.
Quoted 7 lines
Quoted 5 lines
In case we need payloads larger than
XENSTORE_PAYLOAD_MAXwe should split the related operation in multiple parts (see e.g.XS_DIRECTORY_PARTorXS_CONTROLfor uploading a new kernel to Xenstore-stubdom for live update). Which is, BTW, the way AWS should have handled the migration problem (transactions come to my mind in this context).I wasn't part of the original design, but I can see why it was done like that.
I can see why it was done that way, but this doesn't mean I can understand why such a design should be supported by adding comments helping to repeat such a bad decision.
Quoted 1 line
Using multiple commands has also its downside. The first that comes to my mind if that you need to keep around the data. But, with your proposal, you we wouldn't be able to store it in the database (like for transaction update) as datalen can only be 65KB.
I wasn't aware that a complete transaction needs to be kept in a single data base record. :-)
It would work perfectly fine to allocate the needed memory via talloc() and to reference it from a special node being part of the transaction, or to not use a node at all (see again the XS_CONTROL example).
Quoted 2 lines
So one command as the advantage to simply a lot the logic in Xenstored.
Simplifying the logic while violating the basic wire protocol is a bad design decision (I'm repeating myself, I know).
Quoted 1 line
Anyway, this is getting a bit off topic. My only request is to write down assumption more explicitly rather than hiding them. A comment on top of the check is a nice way to help the developper to avoid making a "bad" decision.
I have added a brief comment why the check is existing at all. I even managed to avoid any strong language. :-)
/* Some downstreams modify XENSTORE_PAYLOAD_MAX. */
But maybe that comment was based on wrong assumptions, like the mentioned change not violating the protocol.
Quoted 1 line
I am happy to rewrite the comment so it doesn't lead to think that you (as the maintainer) are open to have a more relax length check.
Yes, please make a suggestion for a proper comment not suggesting we are fine to violate the wire protocol.
Juergen
Re: [PATCH v3 17/25] tools/xenstore: rework struct xs_tdb_record_hdr
Hi Juergen,
Quoted 30 lines · Juergen Gross
Quoted 29 lines · Julien Grall
Hi,
Quoted 26 lines · Juergen Gross
Quoted 24 lines · Julien Grall
Quoted 12 lines
Quoted 12 lines
Quoted 9 lines
In case of a runtime check I
agree that a more central place would be preferred.In the end I don't mind that much, but
BUILD_BUG_ON(XENSTORE_PAYLOAD_MAX >= (
typeof((struct node_hdr *)NULL->datalen))(-1));is a little bit clumsy IMHO.
Agree. We could introduce
FIELD_SIZEOF()(as Linux did) to hide the complexity. The code would then look like:Quoted 8 lines
Quoted 1 line
>= (8 *
FIELD_SIZEOF(struct node_hdr, datalen))Oh, I guess you mean
sizeof_field().And even with that it would look quite clumsy:
BUILD_BUG_ON(XENSTORE_PAYLOAD_MAX >=
(1UL << (8 *sizeof_field(struct node_hdr, datalen))));How about keeping the
BUILD_BUG_ON()inwrite_node_raw()and add the following comment on top ofhandle_input():Some fields in Xenstored are sized based on the max payload (see various
BUILD_BUG_ON()). This would need extra runtime check if we ever decide to have a dynamic payload size.
Quoted 7 lines
Quoted 3 lines
Quoted 1 line
I _could_ do that, but where to stop adding such comments?
When someone other than the author is able to understand the code without too much effort. More comments never hurts, less will in the longer run (see below).
I agree with that statement in general, but requesting a comment to aid a future potential change violating the Xenstore wire protocol is a little bit weird.
Well... This is violating the existing protocol, but it is not set in stone and I think this is acceptable to update it when there is no change for the VMs and for new features (e.g. Live-Update/Live-Migration).
Quoted 9 lines
Quoted 6 lines
Quoted 5 lines
TBH, I really don't see the point doing that.
In case a patch came up upstream trying to violate
XENSTORE_PAYLOAD_MAXI would surely NACK it.That's assuming you will still be around when this happens :). I am not wishing anything bad but the code will likely outlast any of us.
Maybe. But would you really Ack patches adding comments like that in other areas?
Potentially yes. We had a similar discussion on Arm when allowing paddr_t to be 32-bit.
[...]
Quoted 8 lines
Quoted 7 lines
Quoted 5 lines
In case we need payloads larger than
XENSTORE_PAYLOAD_MAXwe should split the related operation in multiple parts (see e.g.XS_DIRECTORY_PARTorXS_CONTROLfor uploading a new kernel to Xenstore-stubdom for live update). Which is, BTW, the way AWS should have handled the migration problem (transactions come to my mind in this context).I wasn't part of the original design, but I can see why it was done like that.
Quoted 8 lines
I can see why it was done that way, but this doesn't mean I can understand why such a design should be supported by adding comments helping to repeat such a bad decision.
Quoted 1 line
Using multiple commands has also its downside. The first that comes to my mind if that you need to keep around the data. But, with your proposal, you we wouldn't be able to store it in the database (like for transaction update) as datalen can only be 65KB.
I wasn't aware that a complete transaction needs to be kept in a single data base record. :-)
IIUC, you are thinking that the client will restore all the state bits by bits. But if you look at the design in docs/designs/xenstore-migration.md, this is a blob.
Quoted 4 lines
It would work perfectly fine to allocate the needed memory via
talloc()and to reference it from a special node being part of the transaction, or to not use a node at all (see again theXS_CONTROLexample).
I am not convinced the complexity is worth it here. To be honest, I think the payload limit should have been relaxed for Live-Update as well as you don't gain much to split. That said, this is less a concern because you are not time constrained.
[...]
Quoted 6 lines
But maybe that comment was based on wrong assumptions, like the mentioned change not violating the protocol. >
Quoted 1 line
I am happy to rewrite the comment so it doesn't lead to think that you (as the maintainer) are open to have a more relax length check.
Yes, please make a suggestion for a proper comment not suggesting we are fine to violate the wire protocol.
Here we go:
"The payload size is not only currently restricted by the protocol but also the internal implementation (see various BUILD_BUG_ON())."
Signature · 1 line
Julien Grall
Re: [PATCH v3 17/25] tools/xenstore: rework struct xs_tdb_record_hdr
Quoted 35 lines · Julien Grall
Hi Juergen,
Quoted 32 lines · Juergen Gross
Quoted 31 lines · Julien Grall
Hi,
Quoted 27 lines · Juergen Gross
Quoted 16 lines · Julien Grall
Quoted 16 lines
Quoted 5 lines
Quoted 5 lines
In case of a runtime check I
agree that a more central place would be preferred.In the end I don't mind that much, but
Quoted 8 lines
Quoted 4 lines
BUILD_BUG_ON(XENSTORE_PAYLOAD_MAX >= (
typeof((struct node_hdr *)NULL->datalen))(-1));is a little bit clumsy IMHO.
Agree. We could introduce
FIELD_SIZEOF()(as Linux did) to hide the complexity. The code would then look like:>= (8 *
FIELD_SIZEOF(struct node_hdr, datalen))Oh, I guess you mean
sizeof_field().Quoted 8 lines
Quoted 4 lines
And even with that it would look quite clumsy:
BUILD_BUG_ON(XENSTORE_PAYLOAD_MAX >=
(1UL << (8 *sizeof_field(struct node_hdr, datalen))));How about keeping the
BUILD_BUG_ON()inwrite_node_raw()and add the following comment on top ofhandle_input():Some fields in Xenstored are sized based on the max payload (see various
BUILD_BUG_ON()). This would need extra runtime check if we ever decide to have a dynamic payload size.I _could_ do that, but where to stop adding such comments?
Quoted 7 lines
Quoted 5 lines
Quoted 1 line
When someone other than the author is able to understand the code without too much effort. More comments never hurts, less will in the longer run (see below).
I agree with that statement in general, but requesting a comment to aid a future potential change violating the Xenstore wire protocol is a little bit weird.
Well... This is violating the existing protocol, but it is not set in stone and I think this is acceptable to update it when there is no change for the VMs and for new features (e.g. Live-Update/Live-Migration).
No, I don't think so.
Think of Xenstore living in a stubdom. This means that even dom0 requests have to go via a ring page, and you are just assuming that the kernel side driver forwarding the requests from user mode is fine with XENSTORE_PAYLOAD_MAX _not_ being the absolute maximum of data transferred via a single command. I could perfectly understand that the kernel might have a XENSTORE_PAYLOAD_MAX sized buffer for the payload, which would be not large enough for your use case.
So violating XENSTORE_PAYLOAD_MAX might be a bad idea with some implementations, at least in theory.
This means, BTW, that changing XENSTORE_PAYLOAD_MAX is a bad idea, too, as this would require to sync between all components knowing its value.
Quoted 25 lines
Quoted 8 lines
Quoted 5 lines
Quoted 4 lines
TBH, I really don't see the point doing that.
In case a patch came up upstream trying to violate
XENSTORE_PAYLOAD_MAXI would surely NACK it.That's assuming you will still be around when this happens :). I am not wishing anything bad but the code will likely outlast any of us.
Maybe. But would you really Ack patches adding comments like that in other areas?
Potentially yes. We had a similar discussion on Arm when allowing
paddr_tto be 32-bit.[...]
Quoted 12 lines
Quoted 7 lines
Quoted 5 lines
In case we need payloads larger than
XENSTORE_PAYLOAD_MAXwe should split the related operation in multiple parts (see e.g.XS_DIRECTORY_PARTorXS_CONTROLfor uploading a new kernel to Xenstore-stubdom for live update). Which is, BTW, the way AWS should have handled the migration problem (transactions come to my mind in this context).I wasn't part of the original design, but I can see why it was done like that.
I can see why it was done that way, but this doesn't mean I can understand why such a design should be supported by adding comments helping to repeat such a bad decision.
Quoted 6 lines
Quoted 4 lines
Quoted 1 line
Using multiple commands has also its downside. The first that comes to my mind if that you need to keep around the data. But, with your proposal, you we wouldn't be able to store it in the database (like for transaction update) as datalen can only be 65KB.
I wasn't aware that a complete transaction needs to be kept in a single data base record. :-)
IIUC, you are thinking that the client will restore all the state bits by bits. But if you look at the design in
docs/designs/xenstore-migration.md, this is a blob.
Of course it is.
I was never assuming that the state would be applied piecemeal, this has to happen atomically.
Quoted 10 lines
Quoted 3 lines
It would work perfectly fine to allocate the needed memory via
talloc()and to reference it from a special node being part of the transaction, or to not use a node at all (see again theXS_CONTROLexample).I am not convinced the complexity is worth it here. To be honest, I think the payload limit should have been relaxed for Live-Update as well as you don't gain much to split. That said, this is less a concern because you are not time constrained.
[...]
Quoted 2 lines
But maybe that comment was based on wrong assumptions, like the mentioned change not violating the protocol. >
Quoted 8 lines
Quoted 4 lines
Quoted 1 line
I am happy to rewrite the comment so it doesn't lead to think that you (as the maintainer) are open to have a more relax length check.
Yes, please make a suggestion for a proper comment not suggesting we are fine to violate the wire protocol.
Here we go:
"The payload size is not only currently restricted by the protocol but also the internal implementation (see various
BUILD_BUG_ON())."
Hmm, I'm still feeling uneasy to imply that the payload size might be changed. See above reasoning.
The only way I could imagine this being possible would be a per-ring-page attribute with both sides agreeing to the max allowed size (the minimum being today's value).
With that in mind I can hesitantly add the comment, maybe with the addition:
"Any potential change of the maximum payload size needs to be negotiated between the involved parties."
Juergen
Re: [PATCH v3 17/25] tools/xenstore: rework struct xs_tdb_record_hdr
Hi Juergen,
Quoted 6 lines · Juergen Gross
On 28.07.23 16:08, Julien Grall wrote: >>>> Using multiple commands has also its downside. The first that comes
Quoted 5 lines
Quoted 4 lines
Quoted 1 line
to my mind if that you need to keep around the data. But, with your proposal, you we wouldn't be able to store it in the database (like for transaction update) as datalen can only be 65KB.
I wasn't aware that a complete transaction needs to be kept in a single data base record. :-)
Quoted 6 lines
Quoted 1 line
IIUC, you are thinking that the client will restore all the state bits by bits. But if you look at the design in
docs/designs/xenstore-migration.md, this is a blob.Of course it is.
I was never assuming that the state would be applied piecemeal, this has to happen atomically.
I am confused because I don't see how this related to the discussion. Above, you mention a transaction, which I interpreted as the client would open a transaction and do a bunch of "write note", "set permissions"... And then commit the transaction.
I thought this is what you talked about and this would still be atomically. My point with the blob is that the parsing of the state is done by Xenstored, not the client.
Quoted 23 lines
Quoted 11 lines
Quoted 4 lines
It would work perfectly fine to allocate the needed memory via
talloc()and to reference it from a special node being part of the transaction, or to not use a node at all (see again theXS_CONTROLexample).I am not convinced the complexity is worth it here. To be honest, I think the payload limit should have been relaxed for Live-Update as well as you don't gain much to split. That said, this is less a concern because you are not time constrained.
[...]
Quoted 2 lines
But maybe that comment was based on wrong assumptions, like the mentioned change not violating the protocol. >
Quoted 8 lines
Quoted 4 lines
Quoted 1 line
I am happy to rewrite the comment so it doesn't lead to think that you (as the maintainer) are open to have a more relax length check.
Yes, please make a suggestion for a proper comment not suggesting we are fine to violate the wire protocol.
Here we go:
"The payload size is not only currently restricted by the protocol but also the internal implementation (see various
BUILD_BUG_ON())."Hmm, I'm still feeling uneasy to imply that the payload size might be changed. See above reasoning.
Quoted 7 lines
The only way I could imagine this being possible would be a per-ring-page attribute with both sides agreeing to the max allowed size (the minimum being today's value).
With that in mind I can hesitantly add the comment, maybe with the addition:
"Any potential change of the maximum payload size needs to be negotiated between the involved parties."
I am ok with that.
Cheers,
Signature · 1 line
Julien Grall
Re: [PATCH v3 17/25] tools/xenstore: rework struct xs_tdb_record_hdr
Quoted 11 lines · Julien Grall
Hi Juergen,
Quoted 8 lines · Juergen Gross
On 28.07.23 16:08, Julien Grall wrote: >>>> Using multiple commands has also its downside. The first that comes
Quoted 6 lines
Quoted 4 lines
Quoted 1 line
to my mind if that you need to keep around the data. But, with your proposal, you we wouldn't be able to store it in the database (like for transaction update) as datalen can only be 65KB.
I wasn't aware that a complete transaction needs to be kept in a single data base record. :-)
IIUC, you are thinking that the client will restore all the state bits by bits. But if you look at the design in
docs/designs/xenstore-migration.md, this is a blob.
Quoted 8 lines
Quoted 4 lines
Of course it is.
I was never assuming that the state would be applied piecemeal, this has to happen atomically.
I am confused because I don't see how this related to the discussion. Above, you mention a transaction, which I interpreted as the client would open a transaction and do a bunch of "write note", "set permissions"... And then commit the transaction.
I thought this is what you talked about and this would still be atomically. My point with the blob is that the parsing of the state is done by Xenstored, not the client.
My idea was that the transaction would be used to mark the related sub-commands to belong to a single migration. This would make cleaning up in case of client death much simpler.
Not using transactions would be possible, too, of course.
Quoted 28 lines
Quoted 28 lines
Quoted 12 lines
Quoted 5 lines
It would work perfectly fine to allocate the needed memory via
talloc()and to reference it from a special node being part of the transaction, or to not use a node at all (see again theXS_CONTROLexample).I am not convinced the complexity is worth it here. To be honest, I think the payload limit should have been relaxed for Live-Update as well as you don't gain much to split. That said, this is less a concern because you are not time constrained.
[...]
Quoted 2 lines
But maybe that comment was based on wrong assumptions, like the mentioned change not violating the protocol. >
Quoted 8 lines
Quoted 4 lines
Quoted 1 line
I am happy to rewrite the comment so it doesn't lead to think that you (as the maintainer) are open to have a more relax length check.
Yes, please make a suggestion for a proper comment not suggesting we are fine to violate the wire protocol.
Here we go:
"The payload size is not only currently restricted by the protocol but also the internal implementation (see various
BUILD_BUG_ON())."Hmm, I'm still feeling uneasy to imply that the payload size might be changed. See above reasoning.
The only way I could imagine this being possible would be a per-ring-page attribute with both sides agreeing to the max allowed size (the minimum being today's value).
Quoted 5 lines
Quoted 3 lines
With that in mind I can hesitantly add the comment, maybe with the addition:
"Any potential change of the maximum payload size needs to be negotiated between the involved parties."I am ok with that.
Okay.
Juergen
PATCH v3 18/25 tools/xenstore: don't use struct node_perms in struct node
In reply to Juergen GrossOpen code struct node_perms in struct node in order to prepare using struct node_hdr in struct node.
Add two helpers to transfer permissions between struct node and struct node_perms.
---
V2:
- new patch
--- tools/xenstore/xenstored_core.c | 76 ++++++++++++++------------ tools/xenstore/xenstored_core.h | 21 ++++++- tools/xenstore/xenstored_domain.c | 13 ++--- tools/xenstore/xenstored_transaction.c | 8 +-- tools/xenstore/xenstored_watch.c | 7 ++- 5 files changed, 75 insertions(+), 50 deletions(-)
tools/xenstore/xenstored_core.c +41 -35
@@ -735,7 +735,7 @@ struct node *read_node(struct connection *conn, const void *ctx,/* Datalen, childlen, number of permissions */ node->generation = hdr->generation; - node->perms.num = hdr->num_perms; + node->num_perms = hdr->num_perms; node->datalen = hdr->datalen; node->childlen = hdr->childlen; node->acc.domid =perms_from_node_hdr (hdr)->id;
@@ -743,8 +743,8 @@ struct node *read_node(struct connection *conn, const void *ctx,/* Copy node data to new memory area, starting with permissions. */ size -=sizeof (*hdr); - node->perms.p =talloc_memdup (node,perms_from_node_hdr (hdr), size); -if (node->perms.p ==NULL ) { + node->perms =talloc_memdup (node,perms_from_node_hdr (hdr), size); +if (node->perms ==NULL ) { errno =ENOMEM ; goto error; }
@@ -757,7 +757,7 @@ struct node *read_node(struct connection *conn, const void *ctx, node->acc.memory =0 ;/* Data is binary blob (usually ascii, no nul). */ - node->data = node->perms.p + hdr->num_perms; + node->data = node->perms + hdr->num_perms;/* Children is strings, nul separated. */ node->children = node->data + node->datalen;
@@ -796,7 +796,7 @@ int write_node_raw(struct connection *conn, const char *db_name,return errno; size =sizeof (*hdr) - + node->perms.num *sizeof (node->perms.p[0 ]) + + node->num_perms *sizeof (node->perms[0 ]) + node->datalen + node->childlen;/* Call domain_max_chk() in any case in order to record max values. */
@@ -813,13 +813,13 @@ int write_node_raw(struct connection *conn, const char *db_name, hdr = data; hdr->generation = node->generation; - hdr->num_perms = node->perms.num; + hdr->num_perms = node->num_perms; hdr->datalen = node->datalen; hdr->childlen = node->childlen; p =perms_from_node_hdr (hdr); -memcpy (p, node->perms.p, node->perms.num *sizeof (*node->perms.p)); - p += node->perms.num *sizeof (*node->perms.p); +memcpy (p, node->perms, node->num_perms *sizeof (*node->perms)); + p += node->num_perms *sizeof (*node->perms);memcpy (p, node->data, node->datalen); p += node->datalen;memcpy (p, node->children, node->childlen);
@@ -900,6 +900,7 @@ static int ask_parents(struct connection *conn, const void *ctx, const char *name, unsigned int *perm) { struct node *node; +struct node_perms perms; do { name = get_parent(ctx, name);
@@ -919,7 +920,8 @@ static int ask_parents(struct connection *conn, const void *ctx,return 0 ; } - *perm =perm_for_conn (conn, &node->perms); +node_to_node_perms (node, &perms); + *perm =perm_for_conn (conn, &perms);return 0 ; }
@@ -956,11 +958,13 @@ static struct node *get_node(struct connection *conn, unsigned int perm) {struct node *node; +struct node_perms perms; node =read_node (conn, ctx, name);/* If we don't have permission, we don't have node. */ if (node) { -if ((perm_for_conn (conn, &node->perms) & perm) != perm) { +node_to_node_perms (node, &perms); +if ((perm_for_conn (conn, &perms) & perm) != perm) { errno =EACCES ; node =NULL ; }
@@ -1434,14 +1438,14 @@ static struct node *construct_node(struct connection *conn, const void *ctx, node->name =talloc_steal (node, names[levels -1 ]);/* Inherit permissions, unpriv domains own what they create. */ - node->perms.num = parent->perms.num; - node->perms.p =talloc_memdup (node, parent->perms.p, - node->perms.num * -sizeof (*node->perms.p)); -if (!node->perms.p) + node->num_perms = parent->num_perms; + node->perms =talloc_memdup (node, parent->perms, + node->num_perms * +sizeof (*node->perms)); +if (!node->perms) goto nomem;if (domain_is_unprivileged (conn)) - node->perms.p[0 ].id = conn->id; + node->perms[0 ].id = conn->id;/* No children, no data */ node->children = node->data =NULL ;
@@ -1764,12 +1768,14 @@ static int do_get_perms(const void *ctx, struct connection *conn,struct node *node;char *strings;unsigned int len; +struct node_perms perms; node =get_spec_node (conn, ctx,onearg (in),NULL ,XS_PERM_READ );if (!node)return errno; - strings =perms_to_strings (node, &node->perms, &len); +node_to_node_perms (node, &perms); + strings =perms_to_strings (node, &perms, &len);if (!strings)return errno;
@@ -1818,10 +1824,10 @@ static int do_set_perms(const void *ctx, struct connection *conn, perms.p[0].id != get_node_owner(node))return EPERM ; - old_perms = node->perms; +node_to_node_perms (node, &old_perms);if (domain_nbentry_dec (conn,get_node_owner (node)))return ENOMEM ; - node->perms = perms; +node_perms_to_node (&perms, node);if (domain_nbentry_inc (conn,get_node_owner (node)))return ENOMEM ;
@@ -2333,8 +2339,8 @@ static void manual_node(const char *name, const char *child)barf_perror ("Could not allocate initial node %s" , name); node->name = name; - node->perms.p = &perms; - node->perms.num =1 ; + node->perms = &perms; + node->num_perms =1 ; node->children = (char *)child;if (child) node->childlen =strlen (child) +1 ;
@@ -3205,10 +3211,10 @@ static int dump_state_node(const void *ctx, struct connection *conn, sn.conn_id =0 ; sn.ta_id =0 ; sn.ta_access =0 ; - sn.perm_n = node->perms.num; + sn.perm_n = node->num_perms; sn.path_len = pathlen; sn.data_len = node->datalen; - head.length += node->perms.num *sizeof (*sn.perms); + head.length += node->num_perms *sizeof (*sn.perms); head.length += pathlen; head.length += node->datalen; head.length =ROUNDUP (head.length,3 );
@@ -3218,7 +3224,7 @@ static int dump_state_node(const void *ctx, struct connection *conn,if (fwrite (&sn,sizeof (sn),1 , fp) !=1 )return dump_state_node_err (data,"Dump node state error" ); - ret =dump_state_node_perms (fp, node->perms.p, node->perms.num); + ret =dump_state_node_perms (fp, node->perms, node->num_perms);if (ret)return dump_state_node_err (data, ret);
@@ -3415,29 +3421,29 @@ void read_state_node(const void *ctx, const void *state) node->data = name + sn->path_len; node->childlen =0 ; node->children =NULL ; - node->perms.num = sn->perm_n; - node->perms.p = talloc_array(node, struct xs_permissions, - node->perms.num); -if (!node->perms.p) + node->num_perms = sn->perm_n; + node->perms = talloc_array(node, struct xs_permissions, + node->num_perms); +if (!node->perms)barf ("allocation error restoring node" ); -for (i =0 ; i < node->perms.num; i++) { +for (i =0 ; i < node->num_perms; i++) {switch (sn->perms[i].access) {case 'r' : - node->perms.p[i].perms =XS_PERM_READ ; + node->perms[i].perms =XS_PERM_READ ;break ;case 'w' : - node->perms.p[i].perms =XS_PERM_WRITE ; + node->perms[i].perms =XS_PERM_WRITE ;break ;case 'b' : - node->perms.p[i].perms =XS_PERM_READ |XS_PERM_WRITE ; + node->perms[i].perms =XS_PERM_READ |XS_PERM_WRITE ;break ;default : - node->perms.p[i].perms =XS_PERM_NONE ; + node->perms[i].perms =XS_PERM_NONE ;break ; }if (sn->perms[i].flags &XS_STATE_NODE_PERM_IGNORE ) - node->perms.p[i].perms |=XS_PERM_IGNORE ; - node->perms.p[i].id = sn->perms[i].domid; + node->perms[i].perms |=XS_PERM_IGNORE ; + node->perms[i].id = sn->perms[i].domid; }if (!strstarts (name,"@" )) {
tools/xenstore/xenstored_core.h +19 -2
@@ -209,7 +209,8 @@ struct node {#define NO_GENERATION ~((uint64_t)0)/* Permissions. */ -struct node_perms perms; +unsigned int num_perms; +struct xs_permissions *perms;/* Contents. */ unsigned int datalen;
@@ -251,7 +252,23 @@ unsigned int perm_for_conn(struct connection *conn,/* Get owner of a node. */ static inline unsigned int get_node_owner (const struct node *node) { -return node->perms.p[0 ].id; +return node->perms[0 ].id; +} + +/* Transfer permissions from node to struct node_perms. */ +static inline void node_to_node_perms (const struct node *node, +struct node_perms *perms) +{ + perms->num = node->num_perms; + perms->p = node->perms; +} + +/* Transfer permissions from struct node_perms to node. */ +static inline void node_perms_to_node (const struct node_perms *perms, +struct node *node) +{ + node->num_perms = perms->num; + node->perms = perms->p; }/* Write a node to the data base. */
tools/xenstore/xenstored_domain.c +6 -7
@@ -513,12 +513,12 @@ static int domain_tree_remove_sub(const void *ctx, struct connection *conn,struct domain *domain = arg;int ret =WALK_TREE_OK ; -if (node->perms.p[0 ].id != domain->domid) +if (node->perms[0 ].id != domain->domid)return WALK_TREE_OK ;if (keep_orphans) {domain_nbentry_dec (NULL , domain->domid); - node->perms.p[0 ].id = priv_domid; + node->perms[0 ].id = priv_domid; node->acc.memory =0 ;domain_nbentry_inc (NULL , priv_domid);if (write_node_raw (NULL , node->name, node,NODE_MODIFY , true)) {
@@ -1335,12 +1335,11 @@ int domain_adjust_node_perms(struct node *node) {unsigned int i; -for (i =1 ; i < node->perms.num; i++) { -if (node->perms.p[i].perms &XS_PERM_IGNORE ) +for (i =1 ; i < node->num_perms; i++) { +if (node->perms[i].perms &XS_PERM_IGNORE )continue ; -if (!chk_domain_generation (node->perms.p[i].id, - node->generation)) - node->perms.p[i].perms |=XS_PERM_IGNORE ; +if (!chk_domain_generation (node->perms[i].id, node->generation)) + node->perms[i].perms |=XS_PERM_IGNORE ; }return 0 ;
tools/xenstore/xenstored_transaction.c +4 -4
@@ -259,13 +259,13 @@ int access_node(struct connection *conn, struct node *node,if (!i->trans_name) goto nomem; i->node =strchr (i->trans_name,'/' ) +1 ; -if (node->generation !=NO_GENERATION && node->perms.num) { +if (node->generation !=NO_GENERATION && node->num_perms) { i->perms.p = talloc_array(i, struct xs_permissions, - node->perms.num); + node->num_perms);if (!i->perms.p) goto nomem; - i->perms.num = node->perms.num; -memcpy (i->perms.p, node->perms.p, + i->perms.num = node->num_perms; +memcpy (i->perms.p, node->perms, i->perms.num *sizeof (*i->perms.p)); }
tools/xenstore/xenstored_watch.c +5 -2
@@ -79,6 +79,7 @@ static bool watch_permitted(struct connection *conn, const void *ctx,unsigned int perm;struct node *parent;char *parent_name; +struct node_perms node_perms;if (perms) { perm =perm_for_conn (conn, perms);
@@ -92,7 +93,8 @@ static bool watch_permitted(struct connection *conn, const void *ctx,return false; } - perm =perm_for_conn (conn, &node->perms); +node_to_node_perms (node, &node_perms); + perm =perm_for_conn (conn, &node_perms);if (perm &XS_PERM_READ )return true;
@@ -106,7 +108,8 @@ static bool watch_permitted(struct connection *conn, const void *ctx,return false; } - perm =perm_for_conn (conn, &parent->perms); +node_to_node_perms (parent, &node_perms); + perm =perm_for_conn (conn, &node_perms);return perm &XS_PERM_READ ; }
Signature · 1 line
2.35.3
Re: [PATCH v3 18/25] tools/xenstore: don't use struct node_perms in struct node
Hi Juergen,
Quoted 86 lines · Juergen Gross
Open code struct
node_permsin struct node in order to prepare using structnode_hdrin struct node.Add two helpers to transfer permissions between struct node and struct
node_perms.Signed-off-by Juergen Gross <[email protected]>---
V2:
- new patch--- tools/xenstore/xenstored_core.c | 76 ++++++++++++++------------ tools/xenstore/xenstored_core.h | 21 ++++++- tools/xenstore/xenstored_domain.c | 13 ++--- tools/xenstore/xenstored_transaction.c | 8 +-- tools/xenstore/xenstored_watch.c | 7 ++- 5 files changed, 75 insertions(+), 50 deletions(-)tools/xenstore/xenstored_core.c +9
-8@@ -735,7 +735,7 @@ struct node *read_node(struct connection *conn, const void *ctx,/* Datalen, childlen, number of permissions */ node->generation = hdr->generation; - node->perms.num = hdr->num_perms; + node->num_perms = hdr->num_perms; node->datalen = hdr->datalen; node->childlen = hdr->childlen; node->acc.domid =perms_from_node_hdr (hdr)->id;@@ -743,8 +743,8 @@ struct node *read_node(struct connection *conn, const void *ctx,/* Copy node data to new memory area, starting with permissions. */ size -=sizeof (*hdr); - node->perms.p =talloc_memdup (node,perms_from_node_hdr (hdr), size); -if (node->perms.p ==NULL ) { + node->perms =talloc_memdup (node,perms_from_node_hdr (hdr), size); +if (node->perms ==NULL ) { errno =ENOMEM ; goto error; }@@ -757,7 +757,7 @@ struct node *read_node(struct connection *conn, const void *ctx, node->acc.memory =0 ;/* Data is binary blob (usually ascii, no nul). */ - node->data = node->perms.p + hdr->num_perms; + node->data = node->perms + hdr->num_perms;/* Children is strings, nul separated. */ node->children = node->data + node->datalen;@@ -796,7 +796,7 @@ int write_node_raw(struct connection *conn, const char *db_name,return errno; size =sizeof (*hdr) - + node->perms.num *sizeof (node->perms.p[0 ]) + + node->num_perms *sizeof (node->perms[0 ]) + node->datalen + node->childlen;/* Call domain_max_chk() in any case in order to record max values. */ @@ -813,13 +813,13 @@ int write_node_raw(struct connection *conn, const char *db_name, hdr = data; hdr->generation = node->generation; - hdr->num_perms = node->perms.num; + hdr->num_perms = node->num_perms; hdr->datalen = node->datalen; hdr->childlen = node->childlen; p =perms_from_node_hdr (hdr); -memcpy (p, node->perms.p, node->perms.num *sizeof (*node->perms.p)); - p += node->perms.num *sizeof (*node->perms.p); +memcpy (p, node->perms, node->num_perms *sizeof (*node->perms)); + p += node->num_perms *sizeof (*node->perms);memcpy (p, node->data, node->datalen); p += node->datalen;memcpy (p, node->children, node->childlen);@@ -900,6 +900,7 @@ static int ask_parents(struct connection *conn, const void *ctx, const char *name, unsigned int *perm) { struct node *node; +struct node_perms perms; do { name = get_parent(ctx, name);
tools/xenstore/xenstored_core.c static int ask_parents(struct connection *conn, const void *ctx, · line 924
Quoted 6 lines
tools/xenstore/xenstored_core.c +2
-1@@ -919,7 +920,8 @@ static int ask_parents(struct connection *conn, const void *ctx,return 0 ; } - *perm =perm_for_conn (conn, &node->perms); +node_to_node_perms (node, &perms); + *perm =perm_for_conn (conn, &perms);
This seems to be a common pattern. Can you introduce a wrapper?
Quoted 42 lines
tools/xenstore/xenstored_core.c +9
-7return 0 ; }@@ -956,11 +958,13 @@ static struct node *get_node(struct connection *conn, unsigned int perm) {struct node *node; +struct node_perms perms; node =read_node (conn, ctx, name);/* If we don't have permission, we don't have node. */ if (node) { -if ((perm_for_conn (conn, &node->perms) & perm) != perm) { +node_to_node_perms (node, &perms); +if ((perm_for_conn (conn, &perms) & perm) != perm) { errno =EACCES ; node =NULL ; }@@ -1434,14 +1438,14 @@ static struct node *construct_node(struct connection *conn, const void *ctx, node->name =talloc_steal (node, names[levels -1 ]);/* Inherit permissions, unpriv domains own what they create. */ - node->perms.num = parent->perms.num; - node->perms.p =talloc_memdup (node, parent->perms.p, - node->perms.num * -sizeof (*node->perms.p)); -if (!node->perms.p) + node->num_perms = parent->num_perms; + node->perms =talloc_memdup (node, parent->perms, + node->num_perms * +sizeof (*node->perms)); +if (!node->perms) goto nomem;if (domain_is_unprivileged (conn)) - node->perms.p[0 ].id = conn->id; + node->perms[0 ].id = conn->id;/* No children, no data */ node->children = node->data =NULL ;@@ -1764,12 +1768,14 @@ static int do_get_perms(const void *ctx, struct connection *conn,struct node *node;char *strings;
tools/xenstore/xenstored_core.c static int do_get_perms(const void *ctx, struct connection *conn, · line 1778
Quoted 10 lines
tools/xenstore/xenstored_core.c +3
-1unsigned int len; +struct node_perms perms; node =get_spec_node (conn, ctx,onearg (in),NULL ,XS_PERM_READ );if (!node)return errno; - strings =perms_to_strings (node, &node->perms, &len); +node_to_node_perms (node, &perms); + strings =perms_to_strings (node, &perms, &len);
This is the only user of perms_to_strings(). So can we just pass the node rather than the perms? This would avoid the call to node_to_node_perms().
Quoted 5 lines
tools/xenstore/xenstored_core.c +0
-0if (!strings)return errno;@@ -1818,10 +1824,10 @@ static int do_set_perms(const void *ctx, struct connection *conn, perms.p[0].id != get_node_owner(node))
tools/xenstore/xenstored_core.c static int do_set_perms(const void *ctx, struct connection *conn, · line 1832
Quoted 10 lines
tools/xenstore/xenstored_core.c +2
-2return EPERM ; - old_perms = node->perms; +node_to_node_perms (node, &old_perms);if (domain_nbentry_dec (conn,get_node_owner (node)))return ENOMEM ; - node->perms = perms; +node_perms_to_node (&perms, node);if (domain_nbentry_inc (conn,get_node_owner (node)))return ENOMEM ;
Cheers,
Signature · 1 line
Julien Grall
Re: [PATCH v3 18/25] tools/xenstore: don't use struct node_perms in struct node
Quoted 86 lines · Julien Grall
Hi Juergen,
Quoted 83 lines · Juergen Gross
Open code struct
node_permsin struct node in order to prepare using structnode_hdrin struct node.Add two helpers to transfer permissions between struct node and struct
node_perms.Signed-off-by Juergen Gross <[email protected]>---
V2:
- new patch
---
tools/xenstore/xenstored_core.c | 76 ++++++++++++++------------ tools/xenstore/xenstored_core.h | 21 ++++++- tools/xenstore/xenstored_domain.c | 13 ++---tools/xenstore/xenstored_transaction.c| 8 +-- tools/xenstore/xenstored_watch.c | 7 ++-5 files changed, 75 insertions(+), 50 deletions(-)tools/xenstore/xenstored_core.c +9
-8@@ -735,7 +735,7 @@ struct node *read_node(struct connection *conn, const void *ctx, /* Datalen, childlen, number of permissions */ node->generation = hdr->generation; - node->perms.num = hdr->num_perms; + node->num_perms = hdr->num_perms; node->datalen = hdr->datalen; node->childlen = hdr->childlen; node->acc.domid = perms_from_node_hdr(hdr)->id;@@ -743,8 +743,8 @@ struct node *read_node(struct connection *conn, const void *ctx, /* Copy node data to new memory area, starting with permissions. */ size -= sizeof(*hdr); - node->perms.p = talloc_memdup(node, perms_from_node_hdr(hdr), size); - if (node->perms.p == NULL) { + node->perms = talloc_memdup(node, perms_from_node_hdr(hdr), size); + if (node->perms == NULL) { errno = ENOMEM; goto error; }@@ -757,7 +757,7 @@ struct node *read_node(struct connection *conn, const void *ctx, node->acc.memory = 0; /* Data is binary blob (usually ascii, no nul). */ - node->data = node->perms.p + hdr->num_perms; + node->data = node->perms + hdr->num_perms; /* Children is strings, nul separated. */ node->children = node->data + node->datalen;@@ -796,7 +796,7 @@ int write_node_raw(struct connection *conn, const char *db_name, return errno; size = sizeof(*hdr) - + node->perms.num * sizeof(node->perms.p[0]) + + node->num_perms * sizeof(node->perms[0]) + node->datalen + node->childlen; /* Call domain_max_chk() in any case in order to record max values. */@@ -813,13 +813,13 @@ int write_node_raw(struct connection *conn, const char *db_name, hdr = data; hdr->generation = node->generation; - hdr->num_perms = node->perms.num; + hdr->num_perms = node->num_perms; hdr->datalen = node->datalen; hdr->childlen = node->childlen; p = perms_from_node_hdr(hdr); - memcpy(p, node->perms.p, node->perms.num * sizeof(*node->perms.p)); - p += node->perms.num * sizeof(*node->perms.p); + memcpy(p, node->perms, node->num_perms * sizeof(*node->perms)); + p += node->num_perms * sizeof(*node->perms); memcpy(p, node->data, node->datalen); p += node->datalen; memcpy(p, node->children, node->childlen);@@ -900,6 +900,7 @@ static int ask_parents(struct connection *conn, const void *ctx, const char *name, unsigned int *perm) { struct node *node; + struct node_perms perms; do { name = get_parent(ctx, name);@@ -919,7 +920,8 @@ static int ask_parents(struct connection *conn, const void
Quoted 8 lines
tools/xenstore/xenstored_core.c static int ask_parents(struct connection *conn, const void *ctx, · line 924
Quoted 6 lines
tools/xenstore/xenstored_core.c +2
-1*ctx, return 0; } - *perm = perm_for_conn(conn, &node->perms); + node_to_node_perms(node, &perms); + *perm = perm_for_conn(conn, &perms);This seems to be a common pattern. Can you introduce a wrapper?
Okay.
Quoted 43 lines
Quoted 43 lines
tools/xenstore/xenstored_core.c +10
-7return 0; }@@ -956,11 +958,13 @@ static struct node *get_node(struct connection *conn, unsigned int perm) { struct node *node; + struct node_perms perms; node = read_node(conn, ctx, name); /* If we don't have permission, we don't have node. */ if (node) { - if ((perm_for_conn(conn, &node->perms) & perm) != perm) { + node_to_node_perms(node, &perms); + if ((perm_for_conn(conn, &perms) & perm) != perm) { errno = EACCES; node = NULL; }@@ -1434,14 +1438,14 @@ static struct node *construct_node(struct connection *conn, const void *ctx, node->name = talloc_steal(node, names[levels - 1]); /* Inherit permissions, unpriv domains own what they create. */ - node->perms.num = parent->perms.num; - node->perms.p = talloc_memdup(node, parent->perms.p, - node->perms.num * - sizeof(*node->perms.p)); - if (!node->perms.p) + node->num_perms = parent->num_perms; + node->perms = talloc_memdup(node, parent->perms, + node->num_perms * + sizeof(*node->perms)); + if (!node->perms) goto nomem; if (domain_is_unprivileged(conn)) - node->perms.p[0].id = conn->id; + node->perms[0].id = conn->id; /* No children, no data */ node->children = node->data = NULL;@@ -1764,12 +1768,14 @@ static int do_get_perms(const void *ctx, struct connection *conn, struct node *node; char *strings; unsigned int len; + struct node_perms perms;
Quoted 8 lines
tools/xenstore/xenstored_core.c static int do_get_perms(const void *ctx, struct connection *conn, · line 1778
Quoted 6 lines
tools/xenstore/xenstored_core.c +2
-1node = get_spec_node(conn, ctx, onearg(in), NULL, XS_PERM_READ); if (!node) return errno; - strings = perms_to_strings(node, &node->perms, &len); + node_to_node_perms(node, &perms); + strings = perms_to_strings(node, &perms, &len);This is the only user of
perms_to_strings(). So can we just pass the node rather than the perms? This would avoid the call tonode_to_node_perms().
Fine with me.
Juergen
PATCH v3 19/25 tools/xenstore: use struct node_hdr in struct node
In reply to Juergen GrossReplace some individual fields in struct node with struct node_hdr.
This allows to add a helper for calculating the accounted memory size of a node.
---
V2:
- new patch
--- tools/xenstore/xenstored_core.c | 108 ++++++++++++------------- tools/xenstore/xenstored_core.h | 16 ++-- tools/xenstore/xenstored_domain.c | 5 +- tools/xenstore/xenstored_transaction.c | 13 +-- 4 files changed, 71 insertions(+), 71 deletions(-)
tools/xenstore/xenstored_core.c +54 -54
@@ -555,6 +555,12 @@ static void initialize_fds(int *p_sock_pollfd_idx, int *ptimeout) } } +static size_t calc_node_acc_size (struct node_hdr *hdr) +{ +return sizeof (*hdr) + hdr->num_perms *sizeof (struct xs_permissions ) + + hdr->datalen + hdr->childlen; +} +const struct node_hdr *db_fetch (const char *db_name,size_t *size) {struct node_hdr *hdr;
@@ -565,8 +571,7 @@ const struct node_hdr *db_fetch(const char *db_name, size_t *size)return NULL ; } - *size =sizeof (*hdr) + hdr->num_perms *sizeof (struct xs_permissions ) + - hdr->datalen + hdr->childlen; + *size =calc_node_acc_size (hdr);trace_tdb ("read %s size %zu\n" , db_name, *size +strlen (db_name));
@@ -725,7 +730,7 @@ struct node *read_node(struct connection *conn, const void *ctx, hdr =db_fetch (db_name, &size);if (hdr ==NULL ) { - node->generation =NO_GENERATION ; + node->hdr.generation =NO_GENERATION ; err =access_node (conn, node,NODE_ACCESS_READ ,NULL ); errno = err ? :ENOENT ; goto error;
@@ -734,10 +739,7 @@ struct node *read_node(struct connection *conn, const void *ctx, node->parent =NULL ;/* Datalen, childlen, number of permissions */ - node->generation = hdr->generation; - node->num_perms = hdr->num_perms; - node->datalen = hdr->datalen; - node->childlen = hdr->childlen; + node->hdr = *hdr; node->acc.domid =perms_from_node_hdr (hdr)->id; node->acc.memory = size;
@@ -759,7 +761,7 @@ struct node *read_node(struct connection *conn, const void *ctx,/* Data is binary blob (usually ascii, no nul). */ node->data = node->perms + hdr->num_perms;/* Children is strings, nul separated. */ - node->children = node->data + node->datalen; + node->children = node->data + node->hdr.datalen;if (access_node (conn, node,NODE_ACCESS_READ ,NULL )) goto error;
@@ -795,9 +797,7 @@ int write_node_raw(struct connection *conn, const char *db_name,if (domain_adjust_node_perms (node))return errno; - size =sizeof (*hdr) - + node->num_perms *sizeof (node->perms[0 ]) - + node->datalen + node->childlen; + size =calc_node_acc_size (&node->hdr);/* Call domain_max_chk() in any case in order to record max values. */ if (domain_max_chk (conn,ACC_NODESZ , size) && !no_quota_check) {
@@ -812,17 +812,14 @@ int write_node_raw(struct connection *conn, const char *db_name, } hdr = data; - hdr->generation = node->generation; - hdr->num_perms = node->num_perms; - hdr->datalen = node->datalen; - hdr->childlen = node->childlen; + *hdr = node->hdr; p =perms_from_node_hdr (hdr); -memcpy (p, node->perms, node->num_perms *sizeof (*node->perms)); - p += node->num_perms *sizeof (*node->perms); -memcpy (p, node->data, node->datalen); - p += node->datalen; -memcpy (p, node->children, node->childlen); +memcpy (p, node->perms, node->hdr.num_perms *sizeof (*node->perms)); + p += node->hdr.num_perms *sizeof (*node->perms); +memcpy (p, node->data, node->hdr.datalen); + p += node->hdr.datalen; +memcpy (p, node->children, node->hdr.childlen);if (db_write (conn, db_name, data, size, &node->acc, mode, no_quota_check))
@@ -1289,7 +1286,7 @@ static int send_directory(const void *ctx, struct connection *conn,if (!node)return errno; -send_reply (conn,XS_DIRECTORY , node->children, node->childlen); +send_reply (conn,XS_DIRECTORY , node->children, node->hdr.childlen);return 0 ; }
@@ -1314,10 +1311,11 @@ static int send_directory_part(const void *ctx, struct connection *conn,/* Second arg is childlist offset. */ off =atoi (in->buffer +strlen (in->buffer) +1 ); - genlen =snprintf (gen,sizeof (gen),"%" PRIu64, node->generation) +1 ; + genlen =snprintf (gen,sizeof (gen),"%" PRIu64, node->hdr.generation) + +1 ;/* Offset behind list: just return a list with an empty string. */ -if (off >= node->childlen) { +if (off >= node->hdr.childlen) { gen[genlen] =0 ;send_reply (conn,XS_DIRECTORY_PART , gen, genlen +1 );return 0 ;
@@ -1330,7 +1328,7 @@ static int send_directory_part(const void *ctx, struct connection *conn,while (len +strlen (child) < maxlen) { len +=strlen (child) +1 ; child +=strlen (child) +1 ; -if (off + len == node->childlen) +if (off + len == node->hdr.childlen)break ; }
@@ -1340,7 +1338,7 @@ static int send_directory_part(const void *ctx, struct connection *conn,memcpy (data, gen, genlen);memcpy (data + genlen, node->children + off, len); -if (off + len == node->childlen) { +if (off + len == node->hdr.childlen) { data[genlen + len] =0 ; len++; }
@@ -1360,7 +1358,7 @@ static int do_read(const void *ctx, struct connection *conn,if (!node)return errno; -send_reply (conn,XS_READ , node->data, node->datalen); +send_reply (conn,XS_READ , node->data, node->hdr.datalen);return 0 ; }
@@ -1379,13 +1377,13 @@ static int add_child(const void *ctx, struct node *parent, const char *name) base =basename (name); baselen =strlen (base) +1 ; - children =talloc_array (ctx, char, parent->childlen + baselen); + children =talloc_array (ctx, char, parent->hdr.childlen + baselen);if (!children)return ENOMEM ; -memcpy (children, parent->children, parent->childlen); -memcpy (children + parent->childlen, base, baselen); +memcpy (children, parent->children, parent->hdr.childlen); +memcpy (children + parent->hdr.childlen, base, baselen); parent->children = children; - parent->childlen += baselen; + parent->hdr.childlen += baselen;return 0 ; }
@@ -1438,9 +1436,9 @@ static struct node *construct_node(struct connection *conn, const void *ctx, node->name =talloc_steal (node, names[levels -1 ]);/* Inherit permissions, unpriv domains own what they create. */ - node->num_perms = parent->num_perms; + node->hdr.num_perms = parent->hdr.num_perms; node->perms =talloc_memdup (node, parent->perms, - node->num_perms * + node->hdr.num_perms *sizeof (*node->perms));if (!node->perms) goto nomem;
@@ -1449,7 +1447,7 @@ static struct node *construct_node(struct connection *conn, const void *ctx,/* No children, no data */ node->children = node->data =NULL ; - node->childlen = node->datalen =0 ; + node->hdr.childlen = node->hdr.datalen =0 ; node->acc.memory =0 ; node->parent = parent;
@@ -1501,7 +1499,7 @@ static struct node *create_node(struct connection *conn, const void *ctx,ta_node_created (conn->transaction); node->data = data; - node->datalen = datalen; + node->hdr.datalen = datalen;/* * We write out the nodes bottom up.
@@ -1581,7 +1579,7 @@ static int do_write(const void *ctx, struct connection *conn,return errno; }else { node->data = in->buffer + offset; - node->datalen = datalen; + node->hdr.datalen = datalen;if (write_node (conn, node,NODE_MODIFY , false))return errno; }
@@ -1629,8 +1627,8 @@ static int remove_child_entry(struct connection *conn, struct node *node, {size_t childlen =strlen (node->children + offset); -memdel (node->children, offset, childlen +1 , node->childlen); - node->childlen -= childlen +1 ; +memdel (node->children, offset, childlen +1 , node->hdr.childlen); + node->hdr.childlen -= childlen +1 ;return write_node (conn, node,NODE_MODIFY , true); }
@@ -1640,8 +1638,9 @@ static int delete_child(struct connection *conn, {unsigned int i; -for (i =0 ; i < node->childlen; i +=strlen (node->children+i) +1 ) { -if (streq (node->children+i, childname)) { +for (i =0 ; i < node->hdr.childlen; + i +=strlen (node->children + i) +1 ) { +if (streq (node->children + i, childname)) { errno =remove_child_entry (conn, node, i) ?EIO :0 ;return errno; }
@@ -1910,7 +1909,7 @@ int walk_node_tree(const void *ctx, struct connection *conn, const char *root,/* node == NULL possible only for the initial loop iteration. */ if (node) {/* Go one step up if ret or if last child finished. */ -if (ret || node->childoff >= node->childlen) { +if (ret || node->childoff >= node->hdr.childlen) { parent = node->parent;/* Call function AFTER processing a node. */ ret = walk_call_func(ctx, conn, node, parent,
@@ -2340,10 +2339,10 @@ static void manual_node(const char *name, const char *child) node->name = name; node->perms = &perms; - node->num_perms =1 ; + node->hdr.num_perms =1 ; node->children = (char *)child;if (child) - node->childlen =strlen (child) +1 ; + node->hdr.childlen =strlen (child) +1 ;if (write_node (NULL , node,NODE_CREATE , false))barf_perror ("Could not create initial node %s" , name);
@@ -3211,12 +3210,12 @@ static int dump_state_node(const void *ctx, struct connection *conn, sn.conn_id =0 ; sn.ta_id =0 ; sn.ta_access =0 ; - sn.perm_n = node->num_perms; + sn.perm_n = node->hdr.num_perms; sn.path_len = pathlen; - sn.data_len = node->datalen; - head.length += node->num_perms *sizeof (*sn.perms); + sn.data_len = node->hdr.datalen; + head.length += node->hdr.num_perms *sizeof (*sn.perms); head.length += pathlen; - head.length += node->datalen; + head.length += node->hdr.datalen; head.length =ROUNDUP (head.length,3 );if (fwrite (&head,sizeof (head),1 , fp) !=1 )
@@ -3224,14 +3223,15 @@ static int dump_state_node(const void *ctx, struct connection *conn,if (fwrite (&sn,sizeof (sn),1 , fp) !=1 )return dump_state_node_err (data,"Dump node state error" ); - ret =dump_state_node_perms (fp, node->perms, node->num_perms); + ret =dump_state_node_perms (fp, node->perms, node->hdr.num_perms);if (ret)return dump_state_node_err (data, ret);if (fwrite (node->name, pathlen,1 , fp) !=1 )return dump_state_node_err (data,"Dump node path error" ); -if (node->datalen &&fwrite (node->data, node->datalen,1 , fp) !=1 ) +if (node->hdr.datalen && +fwrite (node->data, node->hdr.datalen,1 , fp) !=1 )return dump_state_node_err (data,"Dump node data error" ); ret =dump_state_align (fp);
@@ -3416,17 +3416,17 @@ void read_state_node(const void *ctx, const void *state) node->acc.memory =0 ; node->name = name; - node->generation = ++generation; - node->datalen = sn->data_len; + node->hdr.generation = ++generation; + node->hdr.datalen = sn->data_len; node->data = name + sn->path_len; - node->childlen =0 ; + node->hdr.childlen =0 ; node->children =NULL ; - node->num_perms = sn->perm_n; + node->hdr.num_perms = sn->perm_n; node->perms = talloc_array(node, struct xs_permissions, - node->num_perms); + node->hdr.num_perms);if (!node->perms)barf ("allocation error restoring node" ); -for (i =0 ; i < node->num_perms; i++) { +for (i =0 ; i < node->hdr.num_perms; i++) {switch (sn->perms[i].access) {case 'r' : node->perms[i].perms =XS_PERM_READ ;
tools/xenstore/xenstored_core.h +7 -9
@@ -181,6 +181,7 @@ extern struct list_head connections;*/ struct node_hdr {uint64_t generation; +#define NO_GENERATION ~((uint64_t)0)uint16_t num_perms;uint16_t datalen;uint32_t childlen;
@@ -197,6 +198,10 @@ struct node_account_data { };struct node { +/* Data direct for data base. */ +struct node_hdr hdr; + +/* Xenstore path. */ const char *name;/* Name used to access data base. */ const char *db_name;
@@ -204,20 +209,13 @@ struct node {/* Parent (optional) */ struct node *parent; -/* Generation count. */ -uint64_t generation; -#define NO_GENERATION ~((uint64_t)0) -/* Permissions. */ -unsigned int num_perms;struct xs_permissions *perms;/* Contents. */ -unsigned int datalen;void *data;/* Children, each nul-terminated. */ -unsigned int childlen;unsigned int childoff;/* Used by walk_node_tree() internally. */ char *children;
@@ -259,7 +257,7 @@ static inline unsigned int get_node_owner(const struct node *node)static inline void node_to_node_perms (const struct node *node,struct node_perms *perms) { - perms->num = node->num_perms; + perms->num = node->hdr.num_perms; perms->p = node->perms; }
@@ -267,7 +265,7 @@ static inline void node_to_node_perms(const struct node *node,static inline void node_perms_to_node (const struct node_perms *perms,struct node *node) { - node->num_perms = perms->num; + node->hdr.num_perms = perms->num; node->perms = perms->p; }
tools/xenstore/xenstored_domain.c +3 -2
@@ -1335,10 +1335,11 @@ int domain_adjust_node_perms(struct node *node) {unsigned int i; -for (i =1 ; i < node->num_perms; i++) { +for (i =1 ; i < node->hdr.num_perms; i++) {if (node->perms[i].perms &XS_PERM_IGNORE )continue ; -if (!chk_domain_generation (node->perms[i].id, node->generation)) +if (!chk_domain_generation (node->perms[i].id, + node->hdr.generation)) node->perms[i].perms |=XS_PERM_IGNORE ; }
tools/xenstore/xenstored_transaction.c +7 -6
@@ -232,7 +232,7 @@ int access_node(struct connection *conn, struct node *node,bool introduce = false;if (type !=NODE_ACCESS_READ ) { - node->generation = ++generation; + node->hdr.generation = ++generation;if (conn && !conn->transaction)wrl_apply_debit_direct (conn); }
@@ -259,12 +259,13 @@ int access_node(struct connection *conn, struct node *node,if (!i->trans_name) goto nomem; i->node =strchr (i->trans_name,'/' ) +1 ; -if (node->generation !=NO_GENERATION && node->num_perms) { +if (node->hdr.generation !=NO_GENERATION && + node->hdr.num_perms) { i->perms.p = talloc_array(i, struct xs_permissions, - node->num_perms); + node->hdr.num_perms);if (!i->perms.p) goto nomem; - i->perms.num = node->num_perms; + i->perms.num = node->hdr.num_perms;memcpy (i->perms.p, node->perms, i->perms.num *sizeof (*i->perms.p)); }
@@ -282,9 +283,9 @@ int access_node(struct connection *conn, struct node *node,* from the write types. */ if (type ==NODE_ACCESS_READ ) { - i->generation = node->generation; + i->generation = node->hdr.generation; i->check_gen = true; -if (node->generation !=NO_GENERATION ) { +if (node->hdr.generation !=NO_GENERATION ) { ret =write_node_raw (conn, i->trans_name, node,NODE_CREATE , true);if (ret)
Signature · 1 line
2.35.3
Re: [PATCH v3 19/25] tools/xenstore: use struct node_hdr in struct node
Hi Juergen,
tools/xenstore/xenstored_core.c static void initialize_fds(int *p_sock_pollfd_idx, int *ptimeout) · line 558
Quoted 9 lines · Juergen Gross
tools/xenstore/xenstored_core.c +1
-0@@ -555,6 +555,12 @@ static void initialize_fds(int *p_sock_pollfd_idx, int *ptimeout) } } +static size_t calc_node_acc_size (struct node_hdr *hdr)
The parameter can be const.
Quoted 5 lines
tools/xenstore/xenstored_core.c +5
-0+{ +return sizeof (*hdr) + hdr->num_perms *sizeof (struct xs_permissions ) + + hdr->datalen + hdr->childlen; +} +
[...]
Quoted 13 lines
tools/xenstore/xenstored_core.h +1
-0@@ -181,6 +181,7 @@ extern struct list_head connections;*/ struct node_hdr {uint64_t generation; +#define NO_GENERATION ~((uint64_t)0)uint16_t num_perms;uint16_t datalen;uint32_t childlen;
tools/xenstore/xenstored_core.h struct node_account_data { · line 201
Quoted 4 lines
tools/xenstore/xenstored_core.h +1
-0@@ -197,6 +198,10 @@ struct node_account_data { };struct node { +/* Data direct for data base. */
I can't parse it. Did you mean 'from' rather than 'for'?
tools/xenstore/xenstored_core.h struct node { · line 210
Quoted 9 lines
tools/xenstore/xenstored_core.h +3
-0+struct node_hdr hdr; + +/* Xenstore path. */ const char *name;/* Name used to access data base. */ const char *db_name;@@ -204,20 +209,13 @@ struct node {/* Parent (optional) */ struct node *parent;
Cheers,
Signature · 1 line
Julien Grall
Re: [PATCH v3 19/25] tools/xenstore: use struct node_hdr in struct node
Quoted 14 lines · Julien Grall
Hi Juergen,
tools/xenstore/xenstored_core.c static void initialize_fds(int *p_sock_pollfd_idx, int · line 558
Quoted 9 lines · Juergen Gross
tools/xenstore/xenstored_core.c +1
-0@@ -555,6 +555,12 @@ static void initialize_fds(int *p_sock_pollfd_idx, int *ptimeout) } } +static size_t calc_node_acc_size (struct node_hdr *hdr)The parameter can be const.
I've changed this already in my pending V4. :-)
Quoted 19 lines
Quoted 6 lines
tools/xenstore/xenstored_core.c +5
-0+{ + return sizeof(*hdr) + hdr->num_perms * sizeof(struct xs_permissions) + + hdr->datalen + hdr->childlen; +} +[...]
Quoted 10 lines
tools/xenstore/xenstored_core.h +1
-0@@ -181,6 +181,7 @@ extern struct list_head connections;*/ struct node_hdr { uint64_t generation; +#define NO_GENERATION ~((uint64_t)0) uint16_t num_perms;
Quoted 8 lines
tools/xenstore/xenstored_core.h struct node_account_data { · line 200
Quoted 6 lines
tools/xenstore/xenstored_core.h +1
-0uint16_t datalen; uint32_t childlen;@@ -197,6 +198,10 @@ struct node_account_data { }; struct node { + /* Data direct for data base. */I can't parse it. Did you mean 'from' rather than 'for'?
I'll change it to:
/* Copied to/from data base. */
Juergen
PATCH v3 20/25 tools/xenstore: alloc new memory in domain_adjust_node_perms()
In reply to Juergen GrossIn order to avoid modifying the node data in the data base in case a domain is gone, let domain_adjust_node_perms() allocate new memory for the permissions in case they need to be modified. As this should happen only in very rare cases, it is fine to do this even when having copied the node data already.
---
V3:
- new patch
--- tools/xenstore/xenstored_core.c | 10 +++++----- tools/xenstore/xenstored_domain.c | 19 +++++++++++++++---- 2 files changed, 20 insertions(+), 9 deletions(-)
tools/xenstore/xenstored_core.c +5 -5
@@ -751,6 +751,11 @@ struct node *read_node(struct connection *conn, const void *ctx, goto error; } +/* Data is binary blob (usually ascii, no nul). */ + node->data = node->perms + hdr->num_perms; +/* Children is strings, nul separated. */ + node->children = node->data + node->hdr.datalen; +if (domain_adjust_node_perms (node)) goto error;
@@ -758,11 +763,6 @@ struct node *read_node(struct connection *conn, const void *ctx,if (node->acc.domid !=get_node_owner (node)) node->acc.memory =0 ; -/* Data is binary blob (usually ascii, no nul). */ - node->data = node->perms + hdr->num_perms; -/* Children is strings, nul separated. */ - node->children = node->data + node->hdr.datalen; -if (access_node (conn, node,NODE_ACCESS_READ ,NULL )) goto error;
tools/xenstore/xenstored_domain.c +15 -4
@@ -1334,13 +1334,24 @@ int domain_alloc_permrefs(struct node_perms *perms)int domain_adjust_node_perms (struct node *node) {unsigned int i; +struct xs_permissions *perms = node->perms; +bool copied = false;for (i =1 ; i < node->hdr.num_perms; i++) { -if (node->perms[i].perms &XS_PERM_IGNORE ) +if ((perms[i].perms &XS_PERM_IGNORE ) || +chk_domain_generation (perms[i].id, node->hdr.generation))continue ; -if (!chk_domain_generation (node->perms[i].id, - node->hdr.generation)) - node->perms[i].perms |=XS_PERM_IGNORE ; + +if (!copied) { + perms =talloc_memdup (node, node->perms, + node->hdr.num_perms *sizeof (*perms)); +if (!perms) +return ENOMEM ; + node->perms = perms; + copied = true; + } + + perms[i].perms |=XS_PERM_IGNORE ; }return 0 ;
Signature · 1 line
2.35.3
Re: [PATCH v3 20/25] tools/xenstore: alloc new memory in domain_adjust_node_perms()
Hi Juergen,
Quoted 20 lines · Juergen Gross
In order to avoid modifying the node data in the data base in case a domain is gone, let
domain_adjust_node_perms()allocate new memory for the permissions in case they need to be modified. As this should happen only in very rare cases, it is fine to do this even when having copied the node data already.Signed-off-by Juergen Gross <[email protected]>---
V3:
- new patch--- tools/xenstore/xenstored_core.c | 10 +++++----- tools/xenstore/xenstored_domain.c | 19 +++++++++++++++---- 2 files changed, 20 insertions(+), 9 deletions(-)tools/xenstore/xenstored_core.c +0
-0
tools/xenstore/xenstored_core.c struct node *read_node(struct connection *conn, const void *ctx, · line 758
Quoted 8 lines
tools/xenstore/xenstored_core.c +5
-0@@ -751,6 +751,11 @@ struct node *read_node(struct connection *conn, const void *ctx, goto error; } +/* Data is binary blob (usually ascii, no nul). */ + node->data = node->perms + hdr->num_perms; +/* Children is strings, nul separated. */ + node->children = node->data + node->hdr.datalen; +
It took me a while to understand why you move the lines above. I tihnk it would be worth documenting in the code (possibly on top of the declaration domain_adjust_node_perms()) that domain_adjust_node_perms() may re-allocate the permissions.
Quoted 26 lines
tools/xenstore/xenstored_core.c +0
-5if (domain_adjust_node_perms (node)) goto error;@@ -758,11 +763,6 @@ struct node *read_node(struct connection *conn, const void *ctx,if (node->acc.domid !=get_node_owner (node)) node->acc.memory =0 ; -/* Data is binary blob (usually ascii, no nul). */ - node->data = node->perms + hdr->num_perms; -/* Children is strings, nul separated. */ - node->children = node->data + node->hdr.datalen; -if (access_node (conn, node,NODE_ACCESS_READ ,NULL )) goto error;tools/xenstore/xenstored_domain.c +2
-0@@ -1334,13 +1334,24 @@ int domain_alloc_permrefs(struct node_perms *perms)int domain_adjust_node_perms (struct node *node) {unsigned int i; +struct xs_permissions *perms = node->perms; +bool copied = false;
tools/xenstore/xenstored_domain.c int domain_alloc_permrefs(struct node_perms *perms) · line 1345
Quoted 10 lines
tools/xenstore/xenstored_domain.c +4
-4for (i =1 ; i < node->hdr.num_perms; i++) { -if (node->perms[i].perms &XS_PERM_IGNORE ) +if ((perms[i].perms &XS_PERM_IGNORE ) || +chk_domain_generation (perms[i].id, node->hdr.generation))continue ; -if (!chk_domain_generation (node->perms[i].id, - node->hdr.generation)) - node->perms[i].perms |=XS_PERM_IGNORE ; + +if (!copied) {
This wants a coment explain why you need to copy it.
tools/xenstore/xenstored_domain.c int domain_alloc_permrefs(struct node_perms *perms) · line 1357
Quoted 12 lines
tools/xenstore/xenstored_domain.c +9
-0+ perms =talloc_memdup (node, node->perms, + node->hdr.num_perms *sizeof (*perms)); +if (!perms) +return ENOMEM ; + node->perms = perms; + copied = true; + } + + perms[i].perms |=XS_PERM_IGNORE ; }return 0 ;
Cheers,
Signature · 1 line
Julien Grall
Re: [PATCH v3 20/25] tools/xenstore: alloc new memory in domain_adjust_node_perms()
Quoted 18 lines · Julien Grall
Hi Juergen,
Quoted 15 lines · Juergen Gross
In order to avoid modifying the node data in the data base in case a domain is gone, let
domain_adjust_node_perms()allocate new memory for the permissions in case they need to be modified. As this should happen only in very rare cases, it is fine to do this even when having copied the node data already.Signed-off-by Juergen Gross <[email protected]>---
V3:
- new patch
---
tools/xenstore/xenstored_core.c | 10 +++++-----tools/xenstore/xenstored_domain.c| 19 +++++++++++++++----2 files changed, 20 insertions(+), 9 deletions(-)
Quoted 15 lines
tools/xenstore/xenstored_core.c struct node *read_node(struct connection *conn, const · line 758
Quoted 13 lines
tools/xenstore/xenstored_core.c +5
-0@@ -751,6 +751,11 @@ struct node *read_node(struct connection *conn, constvoid *ctx, goto error; } + /* Data is binary blob (usually ascii, no nul). */ + node->data = node->perms + hdr->num_perms; + /* Children is strings, nul separated. */ + node->children = node->data + node->hdr.datalen; +It took me a while to understand why you move the lines above. I tihnk it would be worth documenting in the code (possibly on top of the declaration
domain_adjust_node_perms()) thatdomain_adjust_node_perms()may re-allocate the permissions.
Okay.
Quoted 28 lines
Quoted 28 lines
tools/xenstore/xenstored_core.c +0
-5if (domain_adjust_node_perms(node)) goto error;@@ -758,11 +763,6 @@ struct node *read_node(struct connection *conn, const void *ctx, if (node->acc.domid != get_node_owner(node)) node->acc.memory = 0; - /* Data is binary blob (usually ascii, no nul). */ - node->data = node->perms + hdr->num_perms; - /* Children is strings, nul separated. */ - node->children = node->data + node->hdr.datalen; - if (access_node(conn, node, NODE_ACCESS_READ, NULL)) goto error;tools/xenstore/xenstored_domain.c +4
-1@@ -1334,13 +1334,24 @@ int domain_alloc_permrefs(struct node_perms *perms) int domain_adjust_node_perms(struct node *node) { unsigned int i; + struct xs_permissions *perms = node->perms; + bool copied = false; for (i = 1; i < node->hdr.num_perms; i++) { - if (node->perms[i].perms & XS_PERM_IGNORE) + if ((perms[i].perms & XS_PERM_IGNORE) || + chk_domain_generation(perms[i].id, node->hdr.generation))
Quoted 8 lines
tools/xenstore/xenstored_domain.c int domain_alloc_permrefs(struct node_perms *perms) · line 1345
Quoted 6 lines
tools/xenstore/xenstored_domain.c +2
-3continue; - if (!chk_domain_generation(node->perms[i].id, - node->hdr.generation)) - node->perms[i].perms |= XS_PERM_IGNORE; + + if (!copied) {This wants a coment explain why you need to copy it.
Okay.
Juergen
PATCH v3 21/25 tools/xenstore: introduce read_node_nocopy()
In reply to Juergen GrossIntroduce a read_node() variant returning a pointer to const struct node, which doesn't do a copy of the node data after retrieval from the data base.
Call this variant where appropriate.
---
V3:
- new approach (Julien Grall)
--- tools/xenstore/xenstored_core.c | 104 ++++++++++++++++++++++-------- tools/xenstore/xenstored_core.h | 2 + tools/xenstore/xenstored_domain.c | 4 +- tools/xenstore/xenstored_watch.c | 10 +-- tools/xenstore/xenstored_watch.h | 3 +- 5 files changed, 89 insertions(+), 34 deletions(-)
tools/xenstore/xenstored_core.c +78 -26
@@ -705,11 +705,11 @@ void db_delete(struct connection *conn, const char *name,* If it fails, returns NULL and sets errno. * Temporary memory allocations will be done with ctx. */ -struct node *read_node (struct connection *conn,const void *ctx, -const char *name) +static struct node *read_node_alloc (struct connection *conn,const void *ctx, +const char *name, +const struct node_hdr **hdr) {size_t size; -const struct node_hdr *hdr;struct node *node;const char *db_name;int err;
@@ -719,17 +719,16 @@ struct node *read_node(struct connection *conn, const void *ctx, errno =ENOMEM ;return NULL ; } + node->name =talloc_strdup (node, name);if (!node->name) { -talloc_free (node); errno =ENOMEM ; -return NULL ; + goto error; } db_name =transaction_prepend (conn, name); - hdr =db_fetch (db_name, &size); - -if (hdr ==NULL ) { + *hdr =db_fetch (db_name, &size); +if (*hdr ==NULL ) { node->hdr.generation =NO_GENERATION ; err =access_node (conn, node,NODE_ACCESS_READ ,NULL ); errno = err ? :ENOENT ;
@@ -739,31 +738,79 @@ struct node *read_node(struct connection *conn, const void *ctx, node->parent =NULL ;/* Datalen, childlen, number of permissions */ - node->hdr = *hdr; - node->acc.domid =perms_from_node_hdr (hdr)->id; + node->hdr = **hdr; + node->acc.domid =perms_from_node_hdr (*hdr)->id; node->acc.memory = size; -/* Copy node data to new memory area, starting with permissions. */ - size -=sizeof (*hdr); - node->perms =talloc_memdup (node,perms_from_node_hdr (hdr), size); -if (node->perms ==NULL ) { - errno =ENOMEM ; - goto error; - } +return node; + error: +talloc_free (node); +return NULL ; +} + +static int read_node_helper (struct connection *conn,struct node *node) +{/* Data is binary blob (usually ascii, no nul). */ - node->data = node->perms + hdr->num_perms; + node->data = node->perms + node->hdr.num_perms;/* Children is strings, nul separated. */ node->children = node->data + node->hdr.datalen;if (domain_adjust_node_perms (node)) - goto error; +return -1 ;/* If owner is gone reset currently accounted memory size. */ if (node->acc.domid !=get_node_owner (node)) node->acc.memory =0 ;if (access_node (conn, node,NODE_ACCESS_READ ,NULL )) +return -1 ; + +return 0 ; +} + +struct node *read_node (struct connection *conn,const void *ctx, +const char *name) +{ +size_t size; +const struct node_hdr *hdr; +struct node *node; + + node =read_node_alloc (conn, ctx, name, &hdr); +if (!node) +return NULL ; + +/* Copy node data to new memory area, starting with permissions. */ + size = node->acc.memory -sizeof (*hdr); + node->perms =talloc_memdup (node,perms_from_node_hdr (hdr), size); +if (node->perms ==NULL ) { + errno =ENOMEM ; + goto error; + } + +if (read_node_helper (conn, node)) + goto error; + +return node; + + error: +talloc_free (node); +return NULL ; +} + +const struct node *read_node_const (struct connection *conn,const void *ctx, +const char *name) +{ +const struct node_hdr *hdr; +struct node *node; + + node =read_node_alloc (conn, ctx, name, &hdr); +if (!node) +return NULL ; + + node->perms =perms_from_node_hdr (hdr); + +if (read_node_helper (conn, node)) goto error;return node;
@@ -896,14 +943,14 @@ char *get_parent(const void *ctx, const char *node)static int ask_parents (struct connection *conn,const void *ctx,const char *name,unsigned int *perm) { -struct node *node; +const struct node *node;struct node_perms perms; do { name = get_parent(ctx, name); if (!name) return errno; - node = read_node(conn, ctx, name); + node = read_node_const(conn, ctx, name); if (node) break; if (read_node_can_propagate_errno())
@@ -3194,9 +3241,8 @@ static int dump_state_node_err(struct dump_node_data *data, const char *err) }static int dump_state_node (const void *ctx,struct connection *conn, -struct node *node,void *arg) +const struct node *node,struct dump_node_data *data) { -struct dump_node_data *data = arg;FILE *fp = data->fp;unsigned int pathlen;struct xs_state_record_header head;
@@ -3241,14 +3287,20 @@ static int dump_state_node(const void *ctx, struct connection *conn,return WALK_TREE_OK ; } +static int dump_state_node_enter (const void *ctx,struct connection *conn, +struct node *node,void *arg) +{ +return dump_state_node (ctx, conn, node, arg); +} +static int dump_state_special_node (FILE *fp,const void *ctx,struct dump_node_data *data,const char *name) { -struct node *node; +const struct node *node;int ret; - node =read_node (NULL , ctx, name); + node =read_node_const (NULL , ctx, name);if (!node)return dump_state_node_err (data,"Dump node read node error" );
@@ -3264,7 +3316,7 @@ const char *dump_state_nodes(FILE *fp, const void *ctx) .fp = fp, .err = "Dump node walk error" }; -struct walk_funcs walkfuncs = { .enter = dump_state_node }; +struct walk_funcs walkfuncs = { .enter = dump_state_node_enter };if (walk_node_tree (ctx,NULL ,"/" , &walkfuncs, &data))return data.err;
tools/xenstore/xenstored_core.h +2 -0
@@ -282,6 +282,8 @@ int write_node_raw(struct connection *conn, const char *db_name,/* Get a node from the data base. */ struct node *read_node (struct connection *conn,const void *ctx,const char *name); +const struct node *read_node_const (struct connection *conn,const void *ctx, +const char *name);/* Remove a node and its children. */ int rm_node (struct connection *conn,const void *ctx,const char *name);
tools/xenstore/xenstored_domain.c +2 -2
@@ -563,12 +563,12 @@ static void domain_tree_remove(struct domain *domain)static void fire_special_watches (const char *name) {void *ctx =talloc_new (NULL ); -struct node *node; +const struct node *node;if (!ctx)return ; - node =read_node (NULL , ctx, name); + node =read_node_const (NULL , ctx, name);if (node)fire_watches (NULL , ctx, name, node, true,NULL );
tools/xenstore/xenstored_watch.c +5 -5
@@ -73,11 +73,11 @@ static const char *get_watch_path(const struct watch *watch, const char *name)* changed permissions we need to take the old permissions into account, too. */ static bool watch_permitted (struct connection *conn,const void *ctx, -const char *name,struct node *node, +const char *name,const struct node *node,struct node_perms *perms) {unsigned int perm; -struct node *parent; +const struct node *parent;char *parent_name;struct node_perms node_perms;
@@ -88,7 +88,7 @@ static bool watch_permitted(struct connection *conn, const void *ctx, }if (!node) { - node =read_node (conn, ctx, name); + node =read_node_const (conn, ctx, name);if (!node)return false; }
@@ -103,7 +103,7 @@ static bool watch_permitted(struct connection *conn, const void *ctx, parent_name =get_parent (ctx, node->name);if (!parent_name)return false; - parent =read_node (conn, ctx, parent_name); + parent =read_node_const (conn, ctx, parent_name);if (!parent)return false; }
@@ -122,7 +122,7 @@ static bool watch_permitted(struct connection *conn, const void *ctx,* watch event, too. */ void fire_watches (struct connection *conn,const void *ctx,const char *name, -struct node *node,bool exact,struct node_perms *perms) +const struct node *node,bool exact,struct node_perms *perms) {struct connection *i;struct buffered_data *req;
tools/xenstore/xenstored_watch.h +2 -1
@@ -28,7 +28,8 @@ int do_unwatch(const void *ctx, struct connection *conn,/* Fire all watches: !exact means all the children are affected (ie. rm). */ void fire_watches (struct connection *conn,const void *tmp,const char *name, -struct node *node,bool exact,struct node_perms *perms); +const struct node *node,bool exact, +struct node_perms *perms);void conn_delete_all_watches (struct connection *conn);
Signature · 1 line
2.35.3
Re: [PATCH v3 21/25] tools/xenstore: introduce read_node_nocopy()
Hi Juergen,
Title: I can't find read_node_nocopy(). I found a read_node_const(). Which name did you intend to use?
tools/xenstore/xenstored_core.c struct node *read_node(struct connection *conn, const void *ctx, · line 760
Quoted 11 lines · Juergen Gross
tools/xenstore/xenstored_core.c +4
-2+static int read_node_helper (struct connection *conn,struct node *node) +{/* Data is binary blob (usually ascii, no nul). */ - node->data = node->perms + hdr->num_perms; + node->data = node->perms + node->hdr.num_perms;/* Children is strings, nul separated. */ node->children = node->data + node->hdr.datalen;if (domain_adjust_node_perms (node)) - goto error; +return -1 ;
You are either returning 0 or -1 which is then only used in if ( ... ). Can we simply return a boolean (true for success, false for a failure)?
The rest LGTM.
tools/xenstore/xenstored_core.c struct node *read_node(struct connection *conn, const void *ctx, · line 770
Quoted 10 lines
/* If owner is gone reset currently accounted memory size. */ if (
node->acc.domid!=get_node_owner(node))
node->acc.memory= 0;if (
access_node(conn, node, NODE_ACCESS_READ, NULL))tools/xenstore/xenstored_core.c +4
-0+return -1 ; + +return 0 ; +}
Cheers,
Signature · 1 line
Julien Grall
Re: [PATCH v3 21/25] tools/xenstore: introduce read_node_nocopy()
Quoted 3 lines · Julien Grall
Hi Juergen,
Title: I can't find
read_node_nocopy(). I found aread_node_const(). Which name did you intend to use?
Oh, sorry for that. I think read_node_const() is the better choice.
Quoted 6 lines · Juergen Gross
Quoted 4 lines
(quoted diff) +2
-0+static int read_node_helper(struct connection *conn, struct node *node) +{/* Data is binary blob (usually ascii, no nul). */ -
node->data=node->perms+hdr->num_perms;
Quoted 8 lines
Quoted 6 lines
+
node->data=node->perms+node->hdr.num_perms; /* Children is strings, nul separated. */node->children=node->data+node->hdr.datalen; if (domain_adjust_node_perms(node))
- goto error;
+ return -1;You are either returning 0 or -1 which is then only used in if ( ... ). Can we simply return a boolean (true for success, false for a failure)?
Fine with me.
Quoted 2 lines
The rest LGTM.
Thanks,
Juergen
PATCH v3 22/25 tools/xenstore: merge get_spec_node() into get_node_canonicalized()
In reply to Juergen GrossAdd a "allow_special" parameter to get_node_canonicalized() allowing to merge get_spec_node() into get_node_canonicalized().
Add the same parameter to is_valid_nodename(), as this will simplify check_watch_path().
This is done in preparation to introducing a get_node() variant returning a pointer to const struct node.
---
V3:
- new patch
--- tools/xenstore/xenstored_core.c | 45 +++++++++++++------------------- tools/xenstore/xenstored_core.h | 3 ++- tools/xenstore/xenstored_watch.c | 19 +++++--------- 3 files changed, 26 insertions(+), 41 deletions(-)
tools/xenstore/xenstored_core.c +18 -27
@@ -1219,13 +1219,14 @@ static bool valid_chars(const char *node) "0123456789-/_@") == strlen(node)); } -bool is_valid_nodename (const struct connection *conn,const char *node) +bool is_valid_nodename (const struct connection *conn,const char *node, +bool allow_special) {int local_off =0 ;unsigned int domid; -/* Must start in /. */ -if (!strstarts (node,"/" )) +/* Must start in / or - if special nodes are allowed - in @. */ +if (!strstarts (node,"/" ) && (!allow_special || !strstarts (node,"@" )))return false;/* Cannot end in / (unless it's just "/"). */
@@ -1294,7 +1295,8 @@ static struct node *get_node_canonicalized(struct connection *conn,const void *ctx, const char *name, const char **canonical_name, - unsigned int perm) + unsigned int perm, + bool allow_special) { const char *tmp_name;
@@ -1303,33 +1305,20 @@ static struct node *get_node_canonicalized(struct connection *conn, *canonical_name =canonicalize (conn, ctx, name);if (!*canonical_name)return NULL ; -if (!is_valid_nodename (conn, *canonical_name)) { +if (!is_valid_nodename (conn, *canonical_name, allow_special)) { errno =EINVAL ;return NULL ; }return get_node (conn, ctx, *canonical_name, perm); } -static struct node *get_spec_node (struct connection *conn,const void *ctx, -const char *name,const char **canonical_name, -unsigned int perm) -{ -if (name[0 ] =='@' ) { -if (canonical_name) - *canonical_name = name; -return get_node (conn, ctx, name, perm); - } - -return get_node_canonicalized (conn, ctx, name, canonical_name, perm); -} -static int send_directory (const void *ctx,struct connection *conn,struct buffered_data *in) {struct node *node; node =get_node_canonicalized (conn, ctx,onearg (in),NULL , -XS_PERM_READ ); +XS_PERM_READ , false);if (!node)return errno;
@@ -1351,7 +1340,7 @@ static int send_directory_part(const void *ctx, struct connection *conn,/* First arg is node name. */ node =get_node_canonicalized (conn, ctx, in->buffer,NULL , -XS_PERM_READ ); +XS_PERM_READ , false);if (!node)return errno;
@@ -1401,7 +1390,7 @@ static int do_read(const void *ctx, struct connection *conn,struct node *node; node =get_node_canonicalized (conn, ctx,onearg (in),NULL , -XS_PERM_READ ); +XS_PERM_READ , false);if (!node)return errno;
@@ -1615,7 +1604,8 @@ static int do_write(const void *ctx, struct connection *conn, offset =strlen (vec[0 ]) +1 ; datalen = in->used - offset; - node =get_node_canonicalized (conn, ctx, vec[0 ], &name,XS_PERM_WRITE ); + node =get_node_canonicalized (conn, ctx, vec[0 ], &name,XS_PERM_WRITE , + false);if (!node) {/* No permissions, invalid input? */ if (errno !=ENOENT )
@@ -1644,7 +1634,7 @@ static int do_mkdir(const void *ctx, struct connection *conn,const char *name; node =get_node_canonicalized (conn, ctx,onearg (in), &name, -XS_PERM_WRITE ); +XS_PERM_WRITE , false);/* If it already exists, fine. */ if (!node) {
@@ -1774,7 +1764,7 @@ static int do_rm(const void *ctx, struct connection *conn,char *parentname; node =get_node_canonicalized (conn, ctx,onearg (in), &name, -XS_PERM_WRITE ); +XS_PERM_WRITE , false);if (!node) {/* Didn't exist already? Fine, if parent exists. */ if (errno ==ENOENT ) {
@@ -1816,7 +1806,8 @@ static int do_get_perms(const void *ctx, struct connection *conn,unsigned int len;struct node_perms perms; - node =get_spec_node (conn, ctx,onearg (in),NULL ,XS_PERM_READ ); + node =get_node_canonicalized (conn, ctx,onearg (in),NULL ,XS_PERM_READ , + true);if (!node)return errno;
@@ -1860,8 +1851,8 @@ static int do_set_perms(const void *ctx, struct connection *conn,return ENOENT ;/* We must own node to do this (tools can do this too). */ - node =get_spec_node (conn, ctx, in->buffer, &name, -XS_PERM_WRITE |XS_PERM_OWNER ); + node =get_node_canonicalized (conn, ctx, in->buffer, &name, +XS_PERM_WRITE |XS_PERM_OWNER , true);if (!node)return errno;
tools/xenstore/xenstored_core.h +2 -1
@@ -295,7 +295,8 @@ void check_store(void);void corrupt (struct connection *conn,const char *fmt, ...);/* Is this a valid node name? */ -bool is_valid_nodename (const struct connection *conn,const char *node); +bool is_valid_nodename (const struct connection *conn,const char *node, +bool allow_special);/* Get name of parent node. */ char *get_parent (const void *ctx,const char *node);
tools/xenstore/xenstored_watch.c +6 -13
@@ -166,19 +166,12 @@ static int destroy_watch(void *_watch)static int check_watch_path (struct connection *conn,const void *ctx,const char **path,bool *relative) { -/* Check if valid event. */ -if (strstarts (*path,"@" )) { - *relative = false; -if (strlen (*path) >XENSTORE_REL_PATH_MAX ) - goto inval; - }else { - *relative = !strstarts (*path,"/" ); - *path =canonicalize (conn, ctx, *path); -if (!*path) -return errno; -if (!is_valid_nodename (conn, *path)) - goto inval; - } + *relative = !strstarts (*path,"/" ) && !strstarts (*path,"@" ); + *path =canonicalize (conn, ctx, *path); +if (!*path) +return errno; +if (!is_valid_nodename (conn, *path, true)) + goto inval;return 0 ;
Signature · 1 line
2.35.3
Re: [PATCH v3 22/25] tools/xenstore: merge get_spec_node() into get_node_canonicalized()
Hi,
tools/xenstore/xenstored_watch.c static int destroy_watch(void *_watch) · line 173
Quoted 13 lines · Juergen Gross
tools/xenstore/xenstored_watch.c +0
-5@@ -166,19 +166,12 @@ static int destroy_watch(void *_watch)static int check_watch_path (struct connection *conn,const void *ctx,const char **path,bool *relative) { -/* Check if valid event. */ -if (strstarts (*path,"@" )) { - *relative = false; -if (strlen (*path) >XENSTORE_REL_PATH_MAX ) - goto inval;
I can't find an exact matching check in is_valid_nodename(). The call also seems to put more restriction on '@' node. Can you clarify?
Quoted 7 lines
tools/xenstore/xenstored_watch.c +0
-7- }else { - *relative = !strstarts (*path,"/" ); - *path =canonicalize (conn, ctx, *path); -if (!*path) -return errno; -if (!is_valid_nodename (conn, *path)) - goto inval;
tools/xenstore/xenstored_watch.c static int destroy_watch(void *_watch) · line 177
Quoted 10 lines
tools/xenstore/xenstored_watch.c +6
-1- } + *relative = !strstarts (*path,"/" ) && !strstarts (*path,"@" ); + *path =canonicalize (conn, ctx, *path); +if (!*path) +return errno; +if (!is_valid_nodename (conn, *path, true)) + goto inval;return 0 ;
Cheers,
Signature · 1 line
Julien Grall
Re: [PATCH v3 22/25] tools/xenstore: merge get_spec_node() into get_node_canonicalized()
Quoted 18 lines · Julien Grall
Hi,
tools/xenstore/xenstored_watch.c static int destroy_watch(void *_watch) · line 173
Quoted 13 lines · Juergen Gross
tools/xenstore/xenstored_watch.c +0
-5@@ -166,19 +166,12 @@ static int destroy_watch(void *_watch) static int check_watch_path(struct connection *conn, const void *ctx, const char **path, bool *relative) { - /* Check if valid event. */ - if (strstarts(*path, "@")) { - *relative = false; - if (strlen(*path) > XENSTORE_REL_PATH_MAX) - goto inval;I can't find an exact matching check in
is_valid_nodename(). The call also seems to put more restriction on '@' node. Can you clarify?
The call of domain_max_chk() in is_valid_nodename() will check the length of the node name (at least for unprivileged callers, which is the important case).
The additional restrictions for special nodes are:
- they can't end with "/"
- they can't contain "//"
- they can't contain characters other than the ones allowed for normal nodes
None of those restrictions are problematic. I can add something to the commit message if you want.
Juergen
Re: [PATCH v3 22/25] tools/xenstore: merge get_spec_node() into get_node_canonicalized()
Hi,
Quoted 20 lines · Juergen Gross
Quoted 18 lines · Julien Grall
Hi,
tools/xenstore/xenstored_watch.c static int destroy_watch(void *_watch) · line 173
Quoted 13 lines · Juergen Gross
tools/xenstore/xenstored_watch.c +0
-5@@ -166,19 +166,12 @@ static int destroy_watch(void *_watch) static int check_watch_path(struct connection *conn, const void *ctx, const char **path, bool *relative) { - /* Check if valid event. */ - if (strstarts(*path, "@")) { - *relative = false; - if (strlen(*path) > XENSTORE_REL_PATH_MAX) - goto inval;I can't find an exact matching check in
is_valid_nodename(). The call also seems to put more restriction on '@' node. Can you clarify?
Quoted 3 lines
The call of
domain_max_chk()inis_valid_nodename()will check the length of the node name (at least for unprivileged callers, which is the important case).
Right, but from my understanding, this may not check against XENSTORE_REL_PATH_MAX but whatever limit the user set.
This is a change of behavior that you ought to be explained.
Quoted 10 lines
The additional restrictions for special nodes are:
- they can't end with "/"
- they can't contain "//"
- they can't contain characters other than the ones allowed for normal nodesNone of those restrictions are problematic. I can add something to the commit message if you want.
Yes please.
Cheers,
Signature · 1 line
Julien Grall
Re: [PATCH v3 22/25] tools/xenstore: merge get_spec_node() into get_node_canonicalized()
Quoted 23 lines · Julien Grall
Hi,
Quoted 20 lines · Juergen Gross
Quoted 18 lines · Julien Grall
Hi,
tools/xenstore/xenstored_watch.c static int destroy_watch(void *_watch) · line 173
Quoted 13 lines · Juergen Gross
tools/xenstore/xenstored_watch.c +0
-5@@ -166,19 +166,12 @@ static int destroy_watch(void *_watch) static int check_watch_path(struct connection *conn, const void *ctx, const char **path, bool *relative) { - /* Check if valid event. */ - if (strstarts(*path, "@")) { - *relative = false; - if (strlen(*path) > XENSTORE_REL_PATH_MAX) - goto inval;I can't find an exact matching check in
is_valid_nodename(). The call also seems to put more restriction on '@' node. Can you clarify?
Quoted 5 lines
Quoted 3 lines
The call of
domain_max_chk()inis_valid_nodename()will check the length of the node name (at least for unprivileged callers, which is the important case).Right, but from my understanding, this may not check against
XENSTORE_REL_PATH_MAXbut whatever limit the user set.
Yes, and that's what should be tested, right? I don't see why special nodes should not adhere to the same limits as other nodes. In case an unprivileged user should have access to special nodes, the limits shouldn't hinder the user to access those nodes (setting a limit below 15 would be ridiculous anyway, and that is the length of longest special node name today).
Quoted 5 lines
This is a change of behavior that you ought to be explained.
Quoted 3 lines
The additional restrictions for special nodes are:
Quoted 8 lines
Quoted 6 lines
- they can't end with "/"
- they can't contain "//"
- they can't contain characters other than the ones allowed for normal nodesNone of those restrictions are problematic. I can add something to the commit message if you want.
Yes please.
Juergen
Re: [PATCH v3 22/25] tools/xenstore: merge get_spec_node() into get_node_canonicalized()
Quoted 28 lines · Juergen Gross
Quoted 27 lines · Julien Grall
Hi,
Quoted 23 lines · Juergen Gross
Quoted 18 lines · Julien Grall
Hi,
tools/xenstore/xenstored_watch.c static int destroy_watch(void *_watch) · line 173
Quoted 13 lines · Juergen Gross
tools/xenstore/xenstored_watch.c +0
-5@@ -166,19 +166,12 @@ static int destroy_watch(void *_watch) static int check_watch_path(struct connection *conn, const void *ctx, const char **path, bool *relative) { - /* Check if valid event. */ - if (strstarts(*path, "@")) { - *relative = false; - if (strlen(*path) > XENSTORE_REL_PATH_MAX) - goto inval;I can't find an exact matching check in
is_valid_nodename(). The call also seems to put more restriction on '@' node. Can you clarify?The call of
domain_max_chk()inis_valid_nodename()will check the length of the node name (at least for unprivileged callers, which is the important case).
Quoted 7 lines
Quoted 1 line
Right, but from my understanding, this may not check against
XENSTORE_REL_PATH_MAXbut whatever limit the user set.Yes, and that's what should be tested, right? I don't see why special nodes should not adhere to the same limits as other nodes. In case an unprivileged user should have access to special nodes, the limits shouldn't hinder the user to access those nodes (setting a limit below 15 would be ridiculous anyway, and that is the length of longest special node name today).
I don't mind you want to test against a different value. My point is more that you didn't mention that the limit will be changed.
Cheers,
Signature · 1 line
Julien Grall
Re: [PATCH v3 22/25] tools/xenstore: merge get_spec_node() into get_node_canonicalized()
Quoted 31 lines · Julien Grall
Quoted 28 lines · Juergen Gross
Quoted 27 lines · Julien Grall
Hi,
Quoted 23 lines · Juergen Gross
Quoted 18 lines · Julien Grall
Hi,
tools/xenstore/xenstored_watch.c static int destroy_watch(void *_watch) · line 173
Quoted 13 lines · Juergen Gross
tools/xenstore/xenstored_watch.c +0
-5@@ -166,19 +166,12 @@ static int destroy_watch(void *_watch) static int check_watch_path(struct connection *conn, const void *ctx, const char **path, bool *relative) { - /* Check if valid event. */ - if (strstarts(*path, "@")) { - *relative = false; - if (strlen(*path) > XENSTORE_REL_PATH_MAX) - goto inval;I can't find an exact matching check in
is_valid_nodename(). The call also seems to put more restriction on '@' node. Can you clarify?The call of
domain_max_chk()inis_valid_nodename()will check the length of the node name (at least for unprivileged callers, which is the important case).
Quoted 8 lines
Quoted 7 lines
Quoted 1 line
Right, but from my understanding, this may not check against
XENSTORE_REL_PATH_MAXbut whatever limit the user set.Yes, and that's what should be tested, right? I don't see why special nodes should not adhere to the same limits as other nodes. In case an unprivileged user should have access to special nodes, the limits shouldn't hinder the user to access those nodes (setting a limit below 15 would be ridiculous anyway, and that is the length of longest special node name today).
I don't mind you want to test against a different value. My point is more that you didn't mention that the limit will be changed.
Will mention it.
Juergen
PATCH v3 23/25 tools/xenstore: merge is_valid_nodename() into canonicalize()
In reply to Juergen GrossToday is_valid_nodename() is always called directly after calling canonicalize(), with the exception of do_unwatch(), where the call is missing (which is not correct, but results just in a wrong error reason being returned).
Merge is_valid_nodename() into canonicalize(). While at it merge valid_chars() into it, too.
---
V3:
- new patch
--- tools/xenstore/xenstored_core.c | 89 ++++++++++++++------------------ tools/xenstore/xenstored_core.h | 6 +-- tools/xenstore/xenstored_watch.c | 16 ++---- 3 files changed, 45 insertions(+), 66 deletions(-)
tools/xenstore/xenstored_core.c +40 -49
@@ -1210,42 +1210,6 @@ void send_ack(struct connection *conn, enum xsd_sockmsg_type type)send_reply (conn, type,"OK" ,sizeof ("OK" )); } -static bool valid_chars (const char *node) -{ -/* Nodes can have lots of crap. */ -return (strspn (node, -"ABCDEFGHIJKLMNOPQRSTUVWXYZ" -"abcdefghijklmnopqrstuvwxyz" -"0123456789-/_@" ) ==strlen (node)); -} - -bool is_valid_nodename (const struct connection *conn,const char *node, -bool allow_special) -{ -int local_off =0 ; -unsigned int domid; - -/* Must start in / or - if special nodes are allowed - in @. */ -if (!strstarts (node,"/" ) && (!allow_special || !strstarts (node,"@" ))) -return false; - -/* Cannot end in / (unless it's just "/"). */ -if (strends (node,"/" ) && !streq (node,"/" )) -return false; - -/* No double //. */ -if (strstr (node,"//" )) -return false; - -if (sscanf (node,"/local/domain/%5u/%n" , &domid, &local_off) !=1 ) - local_off =0 ; - -if (domain_max_chk (conn,ACC_PATHLEN ,strlen (node) - local_off)) -return false; - -return valid_chars (node); -} -/* We expect one arg in the input: return NULL otherwise. * The payload must contain exactly one nul, at the end. */
@@ -1279,16 +1243,46 @@ static char *perms_to_strings(const void *ctx, const struct node_perms *perms, }const char *canonicalize (struct connection *conn,const void *ctx, -const char *node) +const char *node,bool allow_special) { -const char *prefix; +char *name; +int local_off =0 ; +unsigned int domid; -if (!node || (node[0 ] =='/' ) || (node[0 ] =='@' )) -return node; - prefix =get_implicit_path (conn); -if (prefix) -return talloc_asprintf (ctx,"%s/%s" , prefix, node); -return node; + errno =EINVAL ; +if (!node) +return NULL ; + +if (strspn (node,"ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz" +"0123456789-/_@" ) !=strlen (node)) +return NULL ; + +if (node[0 ] =='@' && !allow_special) +return NULL ; + +if (node[0 ] !='/' && node[0 ] !='@' ) { + name =talloc_asprintf (ctx,"%s/%s" ,get_implicit_path (conn), + node); +if (!name) +return NULL ; + }else + name = (char *)node; + +/* Cannot end in / (unless it's just "/"). */ +if (strends (name,"/" ) && !streq (name,"/" )) +return NULL ; + +/* No double //. */ +if (strstr (name,"//" )) +return NULL ; + +if (sscanf (name,"/local/domain/%5u/%n" , &domid, &local_off) !=1 ) + local_off =0 ; + +if (domain_max_chk (conn,ACC_PATHLEN ,strlen (name) - local_off)) +return NULL ; + +return name; } static struct node *get_node_canonicalized(struct connection *conn,
@@ -1302,13 +1296,10 @@ static struct node *get_node_canonicalized(struct connection *conn,if (!canonical_name) canonical_name = &tmp_name; - *canonical_name =canonicalize (conn, ctx, name); + *canonical_name =canonicalize (conn, ctx, name, allow_special);if (!*canonical_name)return NULL ; -if (!is_valid_nodename (conn, *canonical_name, allow_special)) { - errno =EINVAL ; -return NULL ; - } +return get_node (conn, ctx, *canonical_name, perm); }
tools/xenstore/xenstored_core.h +1 -5
@@ -241,7 +241,7 @@ void send_ack(struct connection *conn, enum xsd_sockmsg_type type);/* Canonicalize this path if possible. */ const char *canonicalize (struct connection *conn,const void *ctx, -const char *node); +const char *node,bool allow_special);/* Get access permissions. */ unsigned int perm_for_conn(struct connection *conn,
@@ -294,10 +294,6 @@ struct connection *get_connection_by_id(unsigned int conn_id);void check_store (void );void corrupt (struct connection *conn,const char *fmt, ...); -/* Is this a valid node name? */ -bool is_valid_nodename (const struct connection *conn,const char *node, -bool allow_special); -/* Get name of parent node. */ char *get_parent (const void *ctx,const char *node);
tools/xenstore/xenstored_watch.c +4 -12
@@ -167,17 +167,9 @@ static int check_watch_path(struct connection *conn, const void *ctx, const char **path, bool *relative) { *relative = !strstarts (*path,"/" ) && !strstarts (*path,"@" ); - *path =canonicalize (conn, ctx, *path); -if (!*path) -return errno; -if (!is_valid_nodename (conn, *path, true)) - goto inval; - -return 0 ; + *path =canonicalize (conn, ctx, *path, true); - inval: - errno =EINVAL ; -return errno; +return *path ?0 : errno; } static struct watch *add_watch(struct connection *conn, const char *path,
@@ -261,9 +253,9 @@ int do_unwatch(const void *ctx, struct connection *conn,if (get_strings (in, vec,ARRAY_SIZE (vec)) !=ARRAY_SIZE (vec))return EINVAL ; - node =canonicalize (conn, ctx, vec[0 ]); + node =canonicalize (conn, ctx, vec[0 ], true);if (!node) -return ENOMEM ; +return errno;list_for_each_entry (watch, &conn->watches, list) {if (streq (watch->node, node) &&streq (watch->token, vec[1 ])) {list_del (&watch->list);
Signature · 1 line
2.35.3
Re: [PATCH v3 23/25] tools/xenstore: merge is_valid_nodename() into canonicalize()
Hi,
Quoted 4 lines · Juergen Gross
Today
is_valid_nodename()is always called directly after callingcanonicalize(), with the exception ofdo_unwatch(), where the call is missing (which is not correct, but results just in a wrong error reason being returned).
While this change makes sense...
Quoted 3 lines
Merge
is_valid_nodename()intocanonicalize(). While at it mergevalid_chars()into it, too.
... I am not in favor of folding the code is_valid_nodename() and valid_chars() into canonicalize() because the code is now more difficult to read. Also, the keeping the split would allow to free the 'name' in case of an error without adding too much goto in the code.
Quoted 80 lines
Signed-off-by Juergen Gross <[email protected]>---
V3:
- new patch--- tools/xenstore/xenstored_core.c | 89 ++++++++++++++------------------ tools/xenstore/xenstored_core.h | 6 +-- tools/xenstore/xenstored_watch.c | 16 ++---- 3 files changed, 45 insertions(+), 66 deletions(-)tools/xenstore/xenstored_core.c +8
-44@@ -1210,42 +1210,6 @@ void send_ack(struct connection *conn, enum xsd_sockmsg_type type)send_reply (conn, type,"OK" ,sizeof ("OK" )); } -static bool valid_chars (const char *node) -{ -/* Nodes can have lots of crap. */ -return (strspn (node, -"ABCDEFGHIJKLMNOPQRSTUVWXYZ" -"abcdefghijklmnopqrstuvwxyz" -"0123456789-/_@" ) ==strlen (node)); -} - -bool is_valid_nodename (const struct connection *conn,const char *node, -bool allow_special) -{ -int local_off =0 ; -unsigned int domid; - -/* Must start in / or - if special nodes are allowed - in @. */ -if (!strstarts (node,"/" ) && (!allow_special || !strstarts (node,"@" ))) -return false; - -/* Cannot end in / (unless it's just "/"). */ -if (strends (node,"/" ) && !streq (node,"/" )) -return false; - -/* No double //. */ -if (strstr (node,"//" )) -return false; - -if (sscanf (node,"/local/domain/%5u/%n" , &domid, &local_off) !=1 ) - local_off =0 ; - -if (domain_max_chk (conn,ACC_PATHLEN ,strlen (node) - local_off)) -return false; - -return valid_chars (node); -} -/* We expect one arg in the input: return NULL otherwise. * The payload must contain exactly one nul, at the end. */ @@ -1279,16 +1243,46 @@ static char *perms_to_strings(const void *ctx, const struct node_perms *perms, }const char *canonicalize (struct connection *conn,const void *ctx, -const char *node) +const char *node,bool allow_special) { -const char *prefix; +char *name; +int local_off =0 ; +unsigned int domid; -if (!node || (node[0 ] =='/' ) || (node[0 ] =='@' )) -return node; - prefix =get_implicit_path (conn); -if (prefix) -return talloc_asprintf (ctx,"%s/%s" , prefix, node); -return node; + errno =EINVAL ; +if (!node) +return NULL ; +
tools/xenstore/xenstored_core.c static char *perms_to_strings(const void *ctx, const struct node_perms *perms, · line 1265
Quoted 10 lines
tools/xenstore/xenstored_core.c +10
-0+if (strspn (node,"ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz" +"0123456789-/_@" ) !=strlen (node)) +return NULL ; + +if (node[0 ] =='@' && !allow_special) +return NULL ; + +if (node[0 ] !='/' && node[0 ] !='@' ) { + name =talloc_asprintf (ctx,"%s/%s" ,get_implicit_path (conn), + node);
This is allocated but not freed on error. I understand this is part of the 'ctxt' and therefore will be free later on. But this means temporary memory will be allocated for longer. So best to clean-up when you can.
tools/xenstore/xenstored_core.c static char *perms_to_strings(const void *ctx, const struct node_perms *perms, · line 1269
Quoted 4 lines
tools/xenstore/xenstored_core.c +4
-0+if (!name) +return NULL ; + }else + name = (char *)node;
Why does name need to be const?
Quoted 7 lines
tools/xenstore/xenstored_core.c +7
-0+ +/* Cannot end in / (unless it's just "/"). */ +if (strends (name,"/" ) && !streq (name,"/" )) +return NULL ; + +/* No double //. */ +if (strstr (name,"//" ))
tools/xenstore/xenstored_core.c static char *perms_to_strings(const void *ctx, const struct node_perms *perms, · line 1286
Quoted 10 lines
tools/xenstore/xenstored_core.c +9
-0+return NULL ; + +if (sscanf (name,"/local/domain/%5u/%n" , &domid, &local_off) !=1 ) + local_off =0 ; + +if (domain_max_chk (conn,ACC_PATHLEN ,strlen (name) - local_off)) +return NULL ; + +return name; }
Cheers,
Signature · 1 line
Julien Grall
Re: [PATCH v3 23/25] tools/xenstore: merge is_valid_nodename() into canonicalize()
Quoted 8 lines · Julien Grall
Hi,
Quoted 4 lines · Juergen Gross
Today
is_valid_nodename()is always called directly after callingcanonicalize(), with the exception ofdo_unwatch(), where the call is missing (which is not correct, but results just in a wrong error reason being returned).
Quoted 7 lines
While this change makes sense...
Quoted 3 lines
Merge
is_valid_nodename()intocanonicalize(). While at it mergevalid_chars()into it, too.... I am not in favor of folding the code
is_valid_nodename()andvalid_chars()intocanonicalize()because the code is now more difficult to read. Also, the keeping the split would allow to free the 'name' in case of an error without adding too much goto in the code.
I don't think we can easily free name in an error case, at that would require to keep knowledge that name was just allocated in the non-canonical case.
Quoted 84 lines
Quoted 84 lines
Signed-off-by Juergen Gross <[email protected]>---
V3:
- new patch
---
tools/xenstore/xenstored_core.c | 89 ++++++++++++++------------------ tools/xenstore/xenstored_core.h | 6 +--tools/xenstore/xenstored_watch.c| 16 ++----3 files changed, 45 insertions(+), 66 deletions(-)tools/xenstore/xenstored_core.c +12
-44@@ -1210,42 +1210,6 @@ void send_ack(struct connection *conn, enum xsd_sockmsg_type type) send_reply(conn, type, "OK", sizeof("OK")); } -static bool valid_chars (const char *node) -{ - /* Nodes can have lots of crap. */ - return (strspn(node, - "ABCDEFGHIJKLMNOPQRSTUVWXYZ" - "abcdefghijklmnopqrstuvwxyz" - "0123456789-/_@") == strlen(node)); -} - -bool is_valid_nodename (const struct connection *conn,const char *node, - bool allow_special) -{ - int local_off = 0; - unsigned int domid; - - /* Must start in / or - if special nodes are allowed - in @. */ - if (!strstarts(node, "/") && (!allow_special || !strstarts(node, "@"))) - return false; - - /* Cannot end in / (unless it's just "/"). */ - if (strends(node, "/") && !streq(node, "/")) - return false; - - /* No double //. */ - if (strstr(node, "//")) - return false; - - if (sscanf(node, "/local/domain/%5u/%n", &domid, &local_off) != 1) - local_off = 0; - - if (domain_max_chk(conn, ACC_PATHLEN, strlen(node) - local_off)) - return false; - - return valid_chars(node); -} - /* We expect one arg in the input: return NULL otherwise.* The payload must contain exactly one nul, at the end. */ @@ -1279,16 +1243,46 @@ static char *perms_to_strings(const void *ctx, conststruct node_perms *perms, } const char *canonicalize(struct connection *conn, const void *ctx, - const char *node) + const char *node, bool allow_special) { - const char *prefix; + char *name; + int local_off = 0; + unsigned int domid; - if (!node || (node[0] == '/') || (node[0] == '@')) - return node; - prefix = get_implicit_path(conn); - if (prefix) - return talloc_asprintf(ctx, "%s/%s", prefix, node); - return node; + errno = EINVAL; + if (!node) + return NULL; + + if (strspn(node, "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz" + "0123456789-/_@") != strlen(node)) + return NULL; +
Quoted 8 lines
tools/xenstore/xenstored_core.c static char *perms_to_strings(const void *ctx, const struct node_perms *perms, · line 1265
Quoted 6 lines
tools/xenstore/xenstored_core.c +6
-0+ if (node[0] == '@' && !allow_special) + return NULL; + + if (node[0] != '/' && node[0] != '@') { + name = talloc_asprintf(ctx, "%s/%s", get_implicit_path(conn), + node);This is allocated but not freed on error. I understand this is part of the 'ctxt' and therefore will be free later on. But this means temporary memory will be allocated for longer. So best to clean-up when you can.
Really?
It is possible, of course, but it is adding more code churn. Remember that "name" is allocated only in case of a relative path, so freeing it needs to be conditional, too (yes, it would be possible via comparing name to node).
In case you want me to go this route, I can rearrange the code in order to avoid multiple error exits by having only one large if () testing for all possible violations.
Quoted 7 lines
tools/xenstore/xenstored_core.c static char *perms_to_strings(const void *ctx, const struct node_perms *perms, · line 1269
Quoted 5 lines
tools/xenstore/xenstored_core.c +4
-0+ if (!name) + return NULL; + } else + name = (char *)node;Why does name need to be const?
I think the question was posed in a wrong way. :-)
I'll change name to be const char *.
Juergen
Re: [PATCH v3 23/25] tools/xenstore: merge is_valid_nodename() into canonicalize()
Hi,
Quoted 12 lines · Juergen Gross
Quoted 11 lines · Julien Grall
Hi,
Quoted 4 lines · Juergen Gross
Today
is_valid_nodename()is always called directly after callingcanonicalize(), with the exception ofdo_unwatch(), where the call is missing (which is not correct, but results just in a wrong error reason being returned).While this change makes sense...
Quoted 1 line
Quoted 7 lines
Quoted 4 lines
Quoted 2 lines
Merge
is_valid_nodename()intocanonicalize(). While at it mergevalid_chars()into it, too.... I am not in favor of folding the code
is_valid_nodename()andvalid_chars()intocanonicalize()because the code is now more difficult to read. Also, the keeping the split would allow to free the 'name' in case of an error without adding too much goto in the code.I don't think we can easily free name in an error case, at that would require to keep knowledge that name was just allocated in the non-canonical case.
How about this:
const char *canonicalize(struct connection *conn, const void *ctx,
const char *node, bool allow_special)
{
const char *prefix;
const char *name;
if (!node)
return NULL;
if (node[0] == '@' && !allow_special)
return NULL;
if (!node || (node[0] == '/') || (node[0] == '@'))
return node;
prefix = get_implicit_path(conn);
if (prefix) {
name = talloc_asprintf(ctx, "%s/%s", prefix, node);
if (name)
return NULL;
} else
name = node;
if (!is_valid_nodename(conn, node, allow_special)) {
/* Release the memory if 'name' was allocated by us */
if ( name != node )
talloc_free(name);
return NULL;
}
return name;
}And before you ask, I don't see the benefits to partially validate the name before allocating. Hence why I suggest to keep is_valid_nodename() as this keep the function small.
Quoted 93 lines
Quoted 84 lines
Quoted 84 lines
Signed-off-by Juergen Gross <[email protected]>---
V3:
- new patch
---
tools/xenstore/xenstored_core.c | 89 ++++++++++++++------------------ tools/xenstore/xenstored_core.h | 6 +--tools/xenstore/xenstored_watch.c| 16 ++----3 files changed, 45 insertions(+), 66 deletions(-)tools/xenstore/xenstored_core.c +0
-36@@ -1210,42 +1210,6 @@ void send_ack(struct connection *conn, enum xsd_sockmsg_type type) send_reply(conn, type, "OK", sizeof("OK")); } -static bool valid_chars (const char *node) -{ - /* Nodes can have lots of crap. */ - return (strspn(node, - "ABCDEFGHIJKLMNOPQRSTUVWXYZ" - "abcdefghijklmnopqrstuvwxyz" - "0123456789-/_@") == strlen(node)); -} - -bool is_valid_nodename (const struct connection *conn,const char *node, - bool allow_special) -{ - int local_off = 0; - unsigned int domid; - - /* Must start in / or - if special nodes are allowed - in @. */ - if (!strstarts(node, "/") && (!allow_special || !strstarts(node, "@"))) - return false; - - /* Cannot end in / (unless it's just "/"). */ - if (strends(node, "/") && !streq(node, "/")) - return false; - - /* No double //. */ - if (strstr(node, "//")) - return false; - - if (sscanf(node, "/local/domain/%5u/%n", &domid, &local_off) != 1) - local_off = 0; - - if (domain_max_chk(conn, ACC_PATHLEN, strlen(node) - local_off)) - return false; - - return valid_chars(node); -} - /* We expect one arg in the input: return NULL otherwise.* The payload must contain exactly one nul, at the end. */
tools/xenstore/xenstored_core.c +9
-8@@ -1279,16 +1243,46 @@ static char *perms_to_strings(const void *ctx, const struct node_perms *perms, } const char *canonicalize(struct connection *conn, const void *ctx, - const char *node) + const char *node, bool allow_special) { - const char *prefix; + char *name; + int local_off = 0; + unsigned int domid; - if (!node || (node[0] == '/') || (node[0] == '@')) - return node; - prefix = get_implicit_path(conn); - if (prefix) - return talloc_asprintf(ctx, "%s/%s", prefix, node); - return node; + errno = EINVAL; + if (!node) + return NULL; + + if (strspn(node,"ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz" + "0123456789-/_@") != strlen(node)) + return NULL;
+Quoted 8 lines
Quoted 6 lines
+ if (node[0] == '@' && !allow_special)
+ return NULL;
+
+ if (node[0] != '/' && node[0] != '@') {
+ name =talloc_asprintf(ctx, "%s/%s", get_implicit_path(conn), + node);This is allocated but not freed on error. I understand this is part of the 'ctxt' and therefore will be free later on. But this means temporary memory will be allocated for longer. So best to clean-up when you can.
Quoted 1 line
Really?
Let me reply with a different question. Why should we keep the memory allocated longer than necessary?
Quoted 4 lines
It is possible, of course, but it is adding more code churn. Remember that "name" is allocated only in case of a relative path, so freeing it needs to be conditional, too (yes, it would be possible via comparing name to node).
See above, a proposal.
Quoted 6 lines
In case you want me to go this route, I can rearrange the code in order to avoid multiple error exits by having only one large if () testing for all possible violations.
Quoted 1 line
Quoted 8 lines
Quoted 6 lines
Quoted 4 lines
+ if (!name)
+ return NULL;
+ } else
+ name = (char *)node;Why does name need to be const?
I think the question was posed in a wrong way. :-)
Whoops yes.
Quoted 2 lines
I'll change name to be const char *.
Cheers,
Signature · 1 line
Julien Grall
Re: [PATCH v3 23/25] tools/xenstore: merge is_valid_nodename() into canonicalize()
Quoted 54 lines · Julien Grall
Hi,
Quoted 12 lines · Juergen Gross
Quoted 11 lines · Julien Grall
Hi,
Quoted 4 lines · Juergen Gross
Today
is_valid_nodename()is always called directly after callingcanonicalize(), with the exception ofdo_unwatch(), where the call is missing (which is not correct, but results just in a wrong error reason being returned).While this change makes sense...
Quoted 1 line
Quoted 7 lines
Quoted 4 lines
Quoted 2 lines
Merge
is_valid_nodename()intocanonicalize(). While at it mergevalid_chars()into it, too.... I am not in favor of folding the code
is_valid_nodename()andvalid_chars()intocanonicalize()because the code is now more difficult to read. Also, the keeping the split would allow to free the 'name' in case of an error without adding too much goto in the code.I don't think we can easily free name in an error case, at that would require to keep knowledge that name was just allocated in the non-canonical case.
How about this:
const char *canonicalize(struct connection *conn, const void *ctx,
const char *node, boolallow_special)
{
const char *prefix;
const char *name;if (!node)
return NULL;if (node[0] == '@' && !allow_special)
return NULL;if (!node || (node[0] == '/') || (node[0] == '@')) return node;
prefix =get_implicit_path(conn);
if (prefix) {
name =talloc_asprintf(ctx, "%s/%s", prefix, node); if (name)
return NULL;
} else
name = node;if (!is_valid_nodename(conn, node,
allow_special)) { /* Release the memory if 'name' was allocated by us */ if ( name != node )
talloc_free(name);
return NULL;
}
Quoted 4 lines
return name;
}And before you ask, I don't see the benefits to partially validate the name before allocating. Hence why I suggest to keep
is_valid_nodename()as this keep the function small.
Partially validating before doing the allocation is a benefit as it doesn't spend cycles on validating the known good prefix.
Additionally your example won't validate an absolute pathname at all.
What about this variant:
const char *canonicalize(struct connection *conn, const void *ctx,
const char *node, bool allow_special)
{
const char *name;
int local_off = 0;
unsigned int domid;
/*
* Invalid if any of:
* - no node at all
* - illegal character in node
* - starts with '@' but no special node allowed
*/
errno = EINVAL;
if (!node ||
strspn(node, "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz"
"0123456789-/_@") != strlen(node) ||
(node[0] == '@' && !allow_special))
return NULL;
if (node[0] != '/' && node[0] != '@') {
name = talloc_asprintf(ctx, "%s/%s", get_implicit_path(conn),
node);
if (!name)
return NULL;
} else
name = node;
if (sscanf(name, "/local/domain/%5u/%n", &domid, &local_off) != 1)
local_off = 0;
/*
* Only valid if:
* - doesn't end in / (unless it's just "/")
* - no double //
* - not violating max allowed path length
*/
if (!(strends(name, "/") && !streq(name, "/")) &&
!strstr(name, "//") &&
!domain_max_chk(conn, ACC_PATHLEN, strlen(name) - local_off))
return name;
/* Release the memory if 'name' was allocated by us. */
if (name != node)
talloc_free(name);
return NULL;
}Juergen
Re: [PATCH v3 23/25] tools/xenstore: merge is_valid_nodename() into canonicalize()
Hi Juergen,
Quoted 55 lines · Juergen Gross
Quoted 54 lines · Julien Grall
Hi,
Quoted 12 lines · Juergen Gross
Quoted 11 lines · Julien Grall
Hi,
Quoted 4 lines · Juergen Gross
Today
is_valid_nodename()is always called directly after callingcanonicalize(), with the exception ofdo_unwatch(), where the call is missing (which is not correct, but results just in a wrong error reason being returned).While this change makes sense...
Quoted 1 line
Quoted 7 lines
Quoted 4 lines
Quoted 2 lines
Merge
is_valid_nodename()intocanonicalize(). While at it mergevalid_chars()into it, too.... I am not in favor of folding the code
is_valid_nodename()andvalid_chars()intocanonicalize()because the code is now more difficult to read. Also, the keeping the split would allow to free the 'name' in case of an error without adding too much goto in the code.I don't think we can easily free name in an error case, at that would require to keep knowledge that name was just allocated in the non-canonical case.
How about this:
const char *canonicalize(struct connection *conn, const void *ctx, const char *node, bool
allow_special) {
const char *prefix;
const char *name;if (!node)
return NULL;if (node[0] == '@' && !allow_special)
return NULL;if (!node || (node[0] == '/') || (node[0] == '@')) return node;
prefix =get_implicit_path(conn);
if (prefix) {
name =talloc_asprintf(ctx, "%s/%s", prefix, node); if (name)
return NULL;
} else
name = node;if (!is_valid_nodename(conn, node,
allow_special)) { /* Release the memory if 'name' was allocated by us */ if ( name != node )
talloc_free(name);
return NULL;
}
Quoted 7 lines
Quoted 4 lines
return name;
}And before you ask, I don't see the benefits to partially validate the name before allocating. Hence why I suggest to keep
is_valid_nodename()as this keep the function small.Partially validating before doing the allocation is a benefit as it doesn't spend cycles on validating the known good prefix.
Which is pretty much a drop in the ocean in the context of Xenstored :). In reality most of the validation can be done before the allocation with a bit of work.
Quoted 2 lines
Additionally your example won't validate an absolute pathname at all.
That's an error in the logic. This can be sorted out.
Quoted 2 lines
What about this variant:
I still don't see the value of folding is_valid_node_name(). But if the other agrees with you then...
Quoted 8 lines
const char *canonicalize(struct connection *conn, const void *ctx,
const char *node, boolallow_special)
{
const char *name;
intlocal_off= 0;
unsigned int domid;
Quoted 11 lines
/*
* Invalid if any of:
* - no node at all
* - illegal character in node
* - starts with '@' but no special node allowed */
errno = EINVAL;
if (!node ||
strspn(node,
"ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz"
"0123456789-/_@") != strlen(node) ||
... I would rather keep calling valid_chars() here. The rest looks fine even though this is definitely not my preference.
Quoted 26 lines
(node[0] == '@' && !allow_special))
return NULL;if (node[0] != '/' && node[0] != '@') { name = talloc_asprintf(ctx, "%s/%s",
get_implicit_path(conn),
node); if (!name)
return NULL;
} else
name = node;if (
sscanf(name, "/local/domain/%5u/%n", &domid, &local_off)!= 1)local_off= 0;/*
* Only valid if:
* - doesn't end in / (unless it's just "/")
* - no double //
* - not violating max allowed path length
*/
if (!(strends(name, "/") && !streq(name, "/")) && !strstr(name, "//") &&
!domain_max_chk(conn,ACC_PATHLEN, strlen(name) -local_off)) return name;
Quoted 5 lines
/* Release the memory if 'name' was allocated by us. */ if (name != node)
talloc_free(name);
return NULL;
}
Cheers,
Signature · 1 line
Julien Grall
Re: [PATCH v3 23/25] tools/xenstore: merge is_valid_nodename() into canonicalize()
Quoted 61 lines · Julien Grall
Hi Juergen,
Quoted 58 lines · Juergen Gross
Quoted 57 lines · Julien Grall
Hi,
Quoted 12 lines · Juergen Gross
Quoted 11 lines · Julien Grall
Hi,
Quoted 4 lines · Juergen Gross
Today
is_valid_nodename()is always called directly after callingcanonicalize(), with the exception ofdo_unwatch(), where the call is missing (which is not correct, but results just in a wrong error reason being returned).While this change makes sense...
Quoted 1 line
Quoted 7 lines
Quoted 4 lines
Quoted 2 lines
Merge
is_valid_nodename()intocanonicalize(). While at it mergevalid_chars()into it, too.... I am not in favor of folding the code
is_valid_nodename()andvalid_chars()intocanonicalize()because the code is now more difficult to read. Also, the keeping the split would allow to free the 'name' in case of an error without adding too much goto in the code.I don't think we can easily free name in an error case, at that would require to keep knowledge that name was just allocated in the non-canonical case.
How about this:
const char *canonicalize(struct connection *conn, const void *ctx, const char *node, bool
allow_special) {
const char *prefix;
const char *name;if (!node)
return NULL;if (node[0] == '@' && !allow_special)
return NULL;if (!node || (node[0] == '/') || (node[0] == '@')) return node;
prefix =get_implicit_path(conn);
if (prefix) {
name =talloc_asprintf(ctx, "%s/%s", prefix, node); if (name)
return NULL;
} else
name = node;if (!is_valid_nodename(conn, node,
allow_special)) { /* Release the memory if 'name' was allocated by us */ if ( name != node )
talloc_free(name);
return NULL;
}return name;
}
Quoted 6 lines
Quoted 4 lines
Quoted 1 line
And before you ask, I don't see the benefits to partially validate the name before allocating. Hence why I suggest to keep
is_valid_nodename()as this keep the function small.Partially validating before doing the allocation is a benefit as it doesn't spend cycles on validating the known good prefix.
Which is pretty much a drop in the ocean in the context of Xenstored :). In reality most of the validation can be done before the allocation with a bit of work.
Yes, but this would need a more complicated logic related to the handling of local_off. I thought about that, but didn't want to make the patch more complicated.
And regarding performance - yes, it will be minimal, but it was rather low hanging fruit.
Quoted 8 lines
Quoted 1 line
Additionally your example won't validate an absolute pathname at all.
That's an error in the logic. This can be sorted out.
Quoted 2 lines
What about this variant:
I still don't see the value of folding
is_valid_node_name(). But if the other agrees with you then...
I didn't see the value of keeping them apart, as the only caller of is_valid_node_name() would be canonicalize() after this patch. :-)
Quoted 20 lines
Quoted 9 lines
const char *canonicalize(struct connection *conn, const void *ctx, const char *node, bool
allow_special) {
const char *name;
intlocal_off= 0;
unsigned int domid;Quoted 10 lines
/*
* Invalid if any of:
* - no node at all
* - illegal character in node
* - starts with '@' but no special node allowed
*/
errno = EINVAL;
if (!node ||
strspn(node, "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz" "0123456789-/_@")!= strlen(node) ||
Quoted 1 line
... I would rather keep calling
valid_chars()here. The rest looks fine even though this is definitely not my preference.
I can do that, even if I don't see the real value with the comment above the if.
Juergen
Re: [PATCH v3 23/25] tools/xenstore: merge is_valid_nodename() into canonicalize()
Hi Juergen,
Quoted 19 lines · Juergen Gross
Quoted 18 lines · Julien Grall
Quoted 17 lines
const char *canonicalize(struct connection *conn, const void *ctx, const char *node, bool
allow_special) {
const char *name;
intlocal_off= 0;
unsigned int domid;/*
* Invalid if any of:
* - no node at all
* - illegal character in node
* - starts with '@' but no special node allowed
*/
errno = EINVAL;
if (!node ||
strspn(node, "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz" "0123456789-/_@")!= strlen(node) ||
Quoted 3 lines
Quoted 1 line
... I would rather keep calling
valid_chars()here. The rest looks fine even though this is definitely not my preference.I can do that, even if I don't see the real value with the comment above the if.
How about writing Xenstored in a single function then? After all with comments it should be easy to read, right? :)
There are a few difficulty with the current approach. There are:
* a large function call that needs to be split over two lines
* multiple || which also need to split over multiple lines.
* No parentheses over strspn(....) != strlen(node)
Maybe you can parse/understand this 'if' very quickly. But I can't and this is just slowing down review and increasing the risk of introducing bugs.
Cheers,
Signature · 1 line
Julien Grall
Re: [PATCH v3 23/25] tools/xenstore: merge is_valid_nodename() into canonicalize()
Quoted 22 lines · Julien Grall
Hi Juergen,
Quoted 19 lines · Juergen Gross
Quoted 18 lines · Julien Grall
Quoted 17 lines
const char *canonicalize(struct connection *conn, const void *ctx, const char *node, bool
allow_special) {
const char *name;
intlocal_off= 0;
unsigned int domid;/*
* Invalid if any of:
* - no node at all
* - illegal character in node
* - starts with '@' but no special node allowed
*/
errno = EINVAL;
if (!node ||
strspn(node, "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz" "0123456789-/_@")!= strlen(node) ||
Quoted 5 lines
Quoted 3 lines
Quoted 1 line
... I would rather keep calling
valid_chars()here. The rest looks fine even though this is definitely not my preference.I can do that, even if I don't see the real value with the comment above the if.
How about writing Xenstored in a single function then? After all with comments it should be easy to read, right? :)
Yeah, right.
Might come with the downside of a little bit of code duplication. ;-)
Quoted 7 lines
There are a few difficulty with the current approach. There are:
* a large function call that needs to be split over two lines
* multiple || which also need to split over multiple lines.
* No parentheses over strspn(....) != strlen(node)Maybe you can parse/understand this 'if' very quickly. But I can't and this is just slowing down review and increasing the risk of introducing bugs.
Okay, as said above: I can do that.
Juergen
PATCH v3 24/25 tools/xenstore: rework get_node()
In reply to Juergen GrossToday get_node_canonicalized() is the only caller of get_node().
In order to prepare introducing a get_node() variant returning a pointer to const struct node, do the following restructuring:
- move the call of read_node() from get_node() into get_node_canonicalized()
- rename get_node() to get_node_chk_perm()
- rename get_node_canonicalized() to get_node()
---
V3:
- new patch
--- tools/xenstore/xenstored_core.c | 57 +++++++++++++++------------------ 1 file changed, 25 insertions(+), 32 deletions(-)
tools/xenstore/xenstored_core.c +25 -32
@@ -996,27 +996,26 @@ static int errno_from_parents(struct connection *conn, const void *ctx,* If it fails, returns NULL and sets errno. * Temporary memory allocations are done with ctx. */ -static struct node *get_node (struct connection *conn, -const void *ctx, -const char *name, -unsigned int perm) +static bool get_node_chk_perm (struct connection *conn,const void *ctx, +const struct node *node,const char *name, +unsigned int perm) { -struct node *node;struct node_perms perms; +bool err = false; - node =read_node (conn, ctx, name);/* If we don't have permission, we don't have node. */ if (node) {node_to_node_perms (node, &perms);if ((perm_for_conn (conn, &perms) & perm) != perm) { errno =EACCES ; - node =NULL ; + err = true; } }/* Clean up errno if they weren't supposed to know. */ -if (!node && !read_node_can_propagate_errno ()) +if (err && !read_node_can_propagate_errno ()) errno =errno_from_parents (conn, ctx, name, errno, perm); -return node; + +return err; }static struct buffered_data *new_buffer (void *ctx)
@@ -1285,14 +1284,12 @@ const char *canonicalize(struct connection *conn, const void *ctx,return name; } -static struct node *get_node_canonicalized (struct connection *conn, -const void *ctx, -const char *name, -const char **canonical_name, -unsigned int perm, -bool allow_special) +static struct node *get_node (struct connection *conn,const void *ctx, +const char *name,const char **canonical_name, +unsigned int perm,bool allow_special) {const char *tmp_name; +struct node *node;if (!canonical_name) canonical_name = &tmp_name;
@@ -1300,7 +1297,10 @@ static struct node *get_node_canonicalized(struct connection *conn,if (!*canonical_name)return NULL ; -return get_node (conn, ctx, *canonical_name, perm); + node =read_node (conn, ctx, *canonical_name); + +return get_node_chk_perm (conn, ctx, node, *canonical_name, perm) + ?NULL : node; } static int send_directory(const void *ctx, struct connection *conn,
@@ -1308,8 +1308,7 @@ static int send_directory(const void *ctx, struct connection *conn, {struct node *node; - node =get_node_canonicalized (conn, ctx,onearg (in),NULL , -XS_PERM_READ , false); + node =get_node (conn, ctx,onearg (in),NULL ,XS_PERM_READ , false);if (!node)return errno;
@@ -1330,8 +1329,7 @@ static int send_directory_part(const void *ctx, struct connection *conn,return EINVAL ;/* First arg is node name. */ - node =get_node_canonicalized (conn, ctx, in->buffer,NULL , -XS_PERM_READ , false); + node =get_node (conn, ctx, in->buffer,NULL ,XS_PERM_READ , false);if (!node)return errno;
@@ -1380,8 +1378,7 @@ static int do_read(const void *ctx, struct connection *conn, {struct node *node; - node =get_node_canonicalized (conn, ctx,onearg (in),NULL , -XS_PERM_READ , false); + node =get_node (conn, ctx,onearg (in),NULL ,XS_PERM_READ , false);if (!node)return errno;
@@ -1595,8 +1592,7 @@ static int do_write(const void *ctx, struct connection *conn, offset =strlen (vec[0 ]) +1 ; datalen = in->used - offset; - node =get_node_canonicalized (conn, ctx, vec[0 ], &name,XS_PERM_WRITE , - false); + node =get_node (conn, ctx, vec[0 ], &name,XS_PERM_WRITE , false);if (!node) {/* No permissions, invalid input? */ if (errno !=ENOENT )
@@ -1624,8 +1620,7 @@ static int do_mkdir(const void *ctx, struct connection *conn,struct node *node;const char *name; - node =get_node_canonicalized (conn, ctx,onearg (in), &name, -XS_PERM_WRITE , false); + node =get_node (conn, ctx,onearg (in), &name,XS_PERM_WRITE , false);/* If it already exists, fine. */ if (!node) {
@@ -1754,8 +1749,7 @@ static int do_rm(const void *ctx, struct connection *conn,const char *name;char *parentname; - node =get_node_canonicalized (conn, ctx,onearg (in), &name, -XS_PERM_WRITE , false); + node =get_node (conn, ctx,onearg (in), &name,XS_PERM_WRITE , false);if (!node) {/* Didn't exist already? Fine, if parent exists. */ if (errno ==ENOENT ) {
@@ -1797,8 +1791,7 @@ static int do_get_perms(const void *ctx, struct connection *conn,unsigned int len;struct node_perms perms; - node =get_node_canonicalized (conn, ctx,onearg (in),NULL ,XS_PERM_READ , - true); + node =get_node (conn, ctx,onearg (in),NULL ,XS_PERM_READ , true);if (!node)return errno;
@@ -1842,8 +1835,8 @@ static int do_set_perms(const void *ctx, struct connection *conn,return ENOENT ;/* We must own node to do this (tools can do this too). */ - node =get_node_canonicalized (conn, ctx, in->buffer, &name, -XS_PERM_WRITE |XS_PERM_OWNER , true); + node =get_node (conn, ctx, in->buffer, &name, +XS_PERM_WRITE |XS_PERM_OWNER , true);if (!node)return errno;
Signature · 1 line
2.35.3
Re: [PATCH v3 24/25] tools/xenstore: rework get_node()
Hi Juergen,
Quoted 43 lines · Juergen Gross
Today
get_node_canonicalized()is the only caller ofget_node().In order to prepare introducing a
get_node()variant returning a pointer to const struct node, do the following restructuring:- move the call of
read_node()fromget_node()intoget_node_canonicalized()- rename
get_node()toget_node_chk_perm()- rename
get_node_canonicalized()toget_node()Signed-off-by Juergen Gross <[email protected]>---
V3:
- new patch--- tools/xenstore/xenstored_core.c | 57 +++++++++++++++------------------ 1 file changed, 25 insertions(+), 32 deletions(-)tools/xenstore/xenstored_core.c +4
-6@@ -996,27 +996,26 @@ static int errno_from_parents(struct connection *conn, const void *ctx,* If it fails, returns NULL and sets errno. * Temporary memory allocations are done with ctx. */ -static struct node *get_node (struct connection *conn, -const void *ctx, -const char *name, -unsigned int perm) +static bool get_node_chk_perm (struct connection *conn,const void *ctx, +const struct node *node,const char *name, +unsigned int perm) { -struct node *node;struct node_perms perms; +bool err = false; - node =read_node (conn, ctx, name);/* If we don't have permission, we don't have node. */ if (node) {
tools/xenstore/xenstored_core.c static int errno_from_parents(struct connection *conn, const void *ctx, · line 1015
Quoted 10 lines
tools/xenstore/xenstored_core.c +2
-2node_to_node_perms (node, &perms);if ((perm_for_conn (conn, &perms) & perm) != perm) { errno =EACCES ; - node =NULL ; + err = true; } }/* Clean up errno if they weren't supposed to know. */ -if (!node && !read_node_can_propagate_errno ()) +if (err && !read_node_can_propagate_errno ())
Looking at the caller for get_node_chk_perm(), node could be NULL. In this case, err would be false. So there is a change of behavior here.
It is not entirely clear why it is fine. But it might be better to have err equals to true when node is NULL.
Cheers,
Signature · 1 line
Julien Grall
Re: [PATCH v3 24/25] tools/xenstore: rework get_node()
Quoted 59 lines · Julien Grall
Hi Juergen,
Quoted 43 lines · Juergen Gross
Today
get_node_canonicalized()is the only caller ofget_node().In order to prepare introducing a
get_node()variant returning a pointer to const struct node, do the following restructuring:- move the call of
read_node()fromget_node()intoget_node_canonicalized()- rename
get_node()toget_node_chk_perm()- rename
get_node_canonicalized()toget_node()Signed-off-by Juergen Gross <[email protected]>---
V3:
- new patch
---
tools/xenstore/xenstored_core.c| 57 +++++++++++++++------------------1 file changed, 25 insertions(+), 32 deletions(-)tools/xenstore/xenstored_core.c +4
-6@@ -996,27 +996,26 @@ static int errno_from_parents(struct connection *conn,const void *ctx, * If it fails, returns NULL and sets errno. * Temporary memory allocations are done with ctx. */ -static struct node *get_node (struct connection *conn, - const void *ctx, - const char *name, - unsigned int perm) +static bool get_node_chk_perm (struct connection *conn,const void *ctx, + const struct node *node, const char *name, + unsigned int perm) { - struct node *node; struct node_perms perms; + bool err = false; - node = read_node(conn, ctx, name); /* If we don't have permission, we don't have node. */ if (node) {tools/xenstore/xenstored_core.c static int errno_from_parents(struct connection *conn, · line 1015
Quoted 10 lines
tools/xenstore/xenstored_core.c +2
-2node_to_node_perms(node, &perms); if ((perm_for_conn(conn, &perms) & perm) != perm) { errno = EACCES; - node = NULL; + err = true; } } /* Clean up errno if they weren't supposed to know. */ - if (!node && !read_node_can_propagate_errno()) + if (err && !read_node_can_propagate_errno())Looking at the caller for
get_node_chk_perm(), node could be NULL. In this case, err would be false. So there is a change of behavior here.
Quoted 1 line
It is not entirely clear why it is fine. But it might be better to have err equals to true when node is NULL.
Yes, you are right. I'll switch to:
+ bool err = !node;
Juergen
Re: [PATCH v3 24/25] tools/xenstore: rework get_node()
In reply to Juergen GrossHi Juergen,
One more remark.
Quoted 25 lines · Juergen Gross
Today
get_node_canonicalized()is the only caller ofget_node().In order to prepare introducing a
get_node()variant returning a pointer to const struct node, do the following restructuring:- move the call of
read_node()fromget_node()intoget_node_canonicalized()- rename
get_node()toget_node_chk_perm()- rename
get_node_canonicalized()toget_node()Signed-off-by Juergen Gross <[email protected]>---
V3:
- new patch--- tools/xenstore/xenstored_core.c | 57 +++++++++++++++------------------ 1 file changed, 25 insertions(+), 32 deletions(-)tools/xenstore/xenstored_core.c +0
-0
tools/xenstore/xenstored_core.c static int errno_from_parents(struct connection *conn, const void *ctx, · line 1001
Quoted 10 lines
tools/xenstore/xenstored_core.c +3
-4@@ -996,27 +996,26 @@ static int errno_from_parents(struct connection *conn, const void *ctx,* If it fails, returns NULL and sets errno. * Temporary memory allocations are done with ctx. */ -static struct node *get_node (struct connection *conn, -const void *ctx, -const char *name, -unsigned int perm) +static bool get_node_chk_perm (struct connection *conn,const void *ctx, +const struct node *node,const char *name, +unsigned int perm)
As you use bool, I find a bit confusing that 'true' would be returned on error while 'false' indicate it is ok. I feel the inverse is more intuitive.
I can understand this is a matter of taste. So the only thing I would ask is documenting this oddity as I can foresee someone that misinterpreting the return.
Cheers,
Signature · 1 line
Julien Grall
Re: [PATCH v3 24/25] tools/xenstore: rework get_node()
Quoted 35 lines · Julien Grall
Hi Juergen,
One more remark.
Quoted 30 lines · Juergen Gross
Today
get_node_canonicalized()is the only caller ofget_node().In order to prepare introducing a
get_node()variant returning a pointer to const struct node, do the following restructuring:- move the call of
read_node()fromget_node()intoget_node_canonicalized()- rename
get_node()toget_node_chk_perm()- rename
get_node_canonicalized()toget_node()Signed-off-by Juergen Gross <[email protected]>---
V3:
- new patch
---
tools/xenstore/xenstored_core.c| 57 +++++++++++++++------------------1 file changed, 25 insertions(+), 32 deletions(-)tools/xenstore/xenstored_core.c +0
-1@@ -996,27 +996,26 @@ static int errno_from_parents(struct connection *conn,const void *ctx, * If it fails, returns NULL and sets errno. * Temporary memory allocations are done with ctx. */ -static struct node *get_node(struct connection *conn,
Quoted 8 lines
tools/xenstore/xenstored_core.c static int errno_from_parents(struct connection *conn, const void *ctx, · line 1001
Quoted 6 lines
tools/xenstore/xenstored_core.c +3
-3- const void *ctx, - const char *name, - unsigned int perm) +static bool get_node_chk_perm (struct connection *conn,const void *ctx, + const struct node *node, const char *name, + unsigned int perm)As you use bool, I find a bit confusing that 'true' would be returned on error while 'false' indicate it is ok. I feel the inverse is more intuitive.
Depends how you are looking at it. On the caller side it is IMO more intuitive the other way round (it is a common pattern for function returning an int to return 0 in the no-error case).
OTOH returning "true" for the no-error case seems to be the standard in Xenstore for return type bool.
Quoted 1 line
I can understand this is a matter of taste. So the only thing I would ask is documenting this oddity as I can foresee someone that misinterpreting the return.
I'll switch err to success.
Juergen
PATCH v3 25/25 tools/xenstore: introduce get_node_const()
In reply to Juergen GrossAdd a variant of get_node() returning a const struct node pointer.
Note that all callers of this new variant don't supply a pointer where to store the canonical node name, while all callers needing a non-const node do supply this pointer. This results in an asymmetric simplification of the two variants.
---
V3:
- new approach (Julien Grall)
--- tools/xenstore/xenstored_core.c | 35 ++++++++++++++++++++++----------- 1 file changed, 24 insertions(+), 11 deletions(-)
tools/xenstore/xenstored_core.c +24 -11
@@ -1288,11 +1288,8 @@ static struct node *get_node(struct connection *conn, const void *ctx, const char *name, const char **canonical_name, unsigned int perm, bool allow_special) { - const char *tmp_name; struct node *node; -if (!canonical_name) - canonical_name = &tmp_name; *canonical_name =canonicalize (conn, ctx, name, allow_special);if (!*canonical_name)return NULL ;
@@ -1303,12 +1300,28 @@ static struct node *get_node(struct connection *conn, const void *ctx, ?NULL : node; } +static const struct node *get_node_const (struct connection *conn, +const void *ctx,const char *name, +unsigned int perm,bool allow_special) +{ +const char *tmp_name; +const struct node *node; + + tmp_name =canonicalize (conn, ctx, name, allow_special); +if (!tmp_name) +return NULL ; + + node =read_node_const (conn, ctx, tmp_name); + +return get_node_chk_perm (conn, ctx, node, tmp_name, perm) ?NULL : node; +} +static int send_directory (const void *ctx,struct connection *conn,struct buffered_data *in) { -struct node *node; +const struct node *node; - node =get_node (conn, ctx,onearg (in),NULL ,XS_PERM_READ , false); + node =get_node_const (conn, ctx,onearg (in),XS_PERM_READ , false);if (!node)return errno;
@@ -1322,14 +1335,14 @@ static int send_directory_part(const void *ctx, struct connection *conn, {unsigned int off, len, maxlen, genlen;char *child, *data; -struct node *node; +const struct node *node;char gen[24 ];if (xenstore_count_strings (in->buffer, in->used) !=2 )return EINVAL ;/* First arg is node name. */ - node =get_node (conn, ctx, in->buffer,NULL ,XS_PERM_READ , false); + node =get_node_const (conn, ctx, in->buffer,XS_PERM_READ , false);if (!node)return errno;
@@ -1376,9 +1389,9 @@ static int send_directory_part(const void *ctx, struct connection *conn,static int do_read (const void *ctx,struct connection *conn,struct buffered_data *in) { -struct node *node; +const struct node *node; - node =get_node (conn, ctx,onearg (in),NULL ,XS_PERM_READ , false); + node =get_node_const (conn, ctx,onearg (in),XS_PERM_READ , false);if (!node)return errno;
@@ -1786,12 +1799,12 @@ static int do_rm(const void *ctx, struct connection *conn,static int do_get_perms (const void *ctx,struct connection *conn,struct buffered_data *in) { -struct node *node; +const struct node *node;char *strings;unsigned int len;struct node_perms perms; - node =get_node (conn, ctx,onearg (in),NULL ,XS_PERM_READ , true); + node =get_node_const (conn, ctx,onearg (in),XS_PERM_READ , true);if (!node)return errno;
Signature · 1 line
2.35.3
Re: [PATCH v3 25/25] tools/xenstore: introduce get_node_const()
Hi,
Quoted 6 lines · Juergen Gross
Add a variant of
get_node()returning a const struct node pointer.Note that all callers of this new variant don't supply a pointer where to store the canonical node name, while all callers needing a non-const node do supply this pointer. This results in an asymmetric simplification of the two variants.
And I am guessing there are no near term plan to use get_node() with canonical_name = NULL or the caller of get_node_const() expecting the name?
Cheers,
Signature · 1 line
Julien Grall
Re: [PATCH v3 25/25] tools/xenstore: introduce get_node_const()
Quoted 11 lines · Julien Grall
Hi,
Quoted 6 lines · Juergen Gross
Add a variant of
get_node()returning a const struct node pointer.Note that all callers of this new variant don't supply a pointer where to store the canonical node name, while all callers needing a non-const node do supply this pointer. This results in an asymmetric simplification of the two variants.
And I am guessing there are no near term plan to use
get_node()withcanonical_name= NULL or the caller ofget_node_const()expecting the name?
Correct.
The canonical name is needed by the caller only in case he wants to store the node afterwards. And this will be done only after a modification of the node, implying that using the "const" variant would be wrong.
Juergen
Hi Juergen,
Quoted 8 lines · Juergen Gross
Juergen Gross (25):
tools/xenstore: explicitly specify create or modify fortdb_store()tools/xenstore: replace key in struct node with data base nametools/xenstore: lettransaction_prepend()return the name for accesstools/xenstore: renamedo_tdb_delete()and change parameter typetools/xenstore: renamedo_tdb_write()and change parameter typetools/xenstore: switchget_acc_data()to use name instead of keytools/xenstore: add wrapper fortdb_fetch()
I have committed up to this patch.
Cheers,
Signature · 1 line
Julien Grall