Skip to contentSkip to conversation

tools/xenstore: more cleanups

PATCH Superseded Juergen Gross 3 participants 26 messages Mar 30, 2023 – May 3, 2023

v2 of 4 Superseded Latest is v4 →

lore
Share & tools
lore
#0 lore raw

v2 cover tools/xenstore: more cleanups

Some more cleanups of Xenstore.

Based on top of the previous Xenstore series "tools/xenstore: rework internal accounting".

Changes in V2:
- rebase
- one small modification of patch 10
- added patches 11-13

Juergen Gross (13):
tools/xenstore: verify command line parameters better tools/xenstore: do some cleanup of hashtable.c tools/xenstore: modify interface of create_hashtable() tools/xenstore: let hashtable_insert() return 0 on success tools/xenstore: make some write limit functions static tools/xenstore: switch write limiting to use millisecond time base tools/xenstore: remove stale TODO file
tools/xenstore: remove unused events list tools/xenstore: remove support of file backed data base tools/xenstore: remove no longer needed functions from xs_lib.c tools/xenstore: split out environment specific live update code tools/xenstore: split out rest of live update control code tools/xenstore: remove unused stuff from list.h

 tools/include/xenstore.h               |   3 +
 tools/include/xenstore_lib.h           |   3 -
 tools/libs/store/xs.c                  |  38 ++
 tools/xenstore/Makefile                |   5 +-
 tools/xenstore/Makefile.common         |  10 +-
 tools/xenstore/TODO                    |  10 -
 tools/xenstore/hashtable.c             |  98 ++--
 tools/xenstore/hashtable.h             |  16 +-
 tools/xenstore/list.h                  | 227 ---------
 tools/xenstore/xenstore_client.c       | 129 +++++
 tools/xenstore/xenstored_control.c     | 657 +------------------------
 tools/xenstore/xenstored_control.h     |   8 -
 tools/xenstore/xenstored_core.c        |  52 +-
 tools/xenstore/xenstored_core.h        |   7 +-
 tools/xenstore/xenstored_domain.c      | 458 +++++++++--------
 tools/xenstore/xenstored_domain.h      |  24 +-
 tools/xenstore/xenstored_lu.c          | 401 +++++++++++++++
 tools/xenstore/xenstored_lu.h          |  81 +++
 tools/xenstore/xenstored_lu_daemon.c   | 132 +++++
 tools/xenstore/xenstored_lu_minios.c   | 121 +++++
 tools/xenstore/xenstored_transaction.c |   4 +-
 tools/xenstore/xenstored_watch.c       |   5 -
 tools/xenstore/xs_lib.c                | 156 ------
 tools/xenstore/xs_lib.h                |  17 -
 tools/xenstore/xs_tdb_dump.c           |  86 ----
 25 files changed, 1204 insertions(+), 1544 deletions(-)
 delete mode 100644 tools/xenstore/TODO
 create mode 100644 tools/xenstore/xenstored_lu.c
 create mode 100644 tools/xenstore/xenstored_lu.h
 create mode 100644 tools/xenstore/xenstored_lu_daemon.c
 create mode 100644 tools/xenstore/xenstored_lu_minios.c
 delete mode 100644 tools/xenstore/xs_tdb_dump.c
Signature · 1 line
2.35.3
#1 lore raw

PATCH v2 1/13 tools/xenstore: verify command line parameters better

Add some more verification of command line parameters.

---
 tools/xenstore/xenstored_core.c | 19 +++++++++----------
 1 file changed, 9 insertions(+), 10 deletions(-)
tools/xenstore/xenstored_core.c +9 -10
index 6e2fc06840..7214b3df03 100644
@@ -2809,7 +2809,7 @@ int main(int argc, char *argv[])
 			no_domain_init = true;
 			break;
 		case 'E':
-			hard_quotas[ACC_NODES].val = strtoul(optarg, NULL, 10);
+			hard_quotas[ACC_NODES].val = get_optval_int(optarg);
 			break;
 		case 'F':
 			pidfile = optarg;
@@ -2827,10 +2827,10 @@ int main(int argc, char *argv[])
 			recovery = false;
 			break;
 		case 'S':
-			hard_quotas[ACC_NODESZ].val = strtoul(optarg, NULL, 10);
+			hard_quotas[ACC_NODESZ].val = get_optval_int(optarg);
 			break;
 		case 't':
-			hard_quotas[ACC_TRANS].val = strtoul(optarg, NULL, 10);
+			hard_quotas[ACC_TRANS].val = get_optval_int(optarg);
 			break;
 		case 'T':
 			tracefile = optarg;
@@ -2850,14 +2850,13 @@ int main(int argc, char *argv[])
 			verbose = true;
 			break;
 		case 'W':
-			hard_quotas[ACC_WATCH].val = strtoul(optarg, NULL, 10);
+			hard_quotas[ACC_WATCH].val = get_optval_int(optarg);
 			break;
 		case 'A':
-			hard_quotas[ACC_NPERM].val = strtoul(optarg, NULL, 10);
+			hard_quotas[ACC_NPERM].val = get_optval_int(optarg);
 			break;
 		case 'M':
-			hard_quotas[ACC_PATHLEN].val =
-				strtoul(optarg, NULL, 10);
+			hard_quotas[ACC_PATHLEN].val = get_optval_int(optarg);
 			hard_quotas[ACC_PATHLEN].val =
 				 min((unsigned int)XENSTORE_REL_PATH_MAX,
 				     hard_quotas[ACC_PATHLEN].val);
@@ -2872,13 +2871,13 @@ int main(int argc, char *argv[])
 			set_timeout(optarg);
 			break;
 		case 'e':
-			dom0_event = strtol(optarg, NULL, 10);
+			dom0_event = get_optval_int(optarg);
 			break;
 		case 'm':
-			dom0_domid = strtol(optarg, NULL, 10);
+			dom0_domid = get_optval_int(optarg);
 			break;
 		case 'p':
-			priv_domid = strtol(optarg, NULL, 10);
+			priv_domid = get_optval_int(optarg);
 			break;
 #ifndef NO_LIVE_UPDATE
 		case 'U':
Signature · 1 line
2.35.3
Acked-by on v2 1/13
#2 lore raw

Re: [PATCH v2 01/13] tools/xenstore: verify command line parameters better

Hi Juergen,

Quoted 12 lines · Juergen Gross

Add some more verification of command line parameters.

---
  tools/xenstore/xenstored_core.c | 19 +++++++++----------
  1 file changed, 9 insertions(+), 10 deletions(-)
tools/xenstore/xenstored_core.c +0 -0
index 6e2fc06840..7214b3df03 100644

tools/xenstore/xenstored_core.c int main(int argc, char *argv[]) · line 2812

Quoted 5 lines
tools/xenstore/xenstored_core.c +1 -1
@@ -2809,7 +2809,7 @@ int main(int argc, char *argv[])
  			no_domain_init = true;
  			break;
  		case 'E':
-			hard_quotas[ACC_NODES].val = strtoul(optarg, NULL, 10);
+			hard_quotas[ACC_NODES].val = get_optval_int(optarg);

Ah, so that exactly what I was exactly on the other series.

I would be OK if this is kept separate:

Cheers,

Signature · 1 line
Julien Grall
#3 lore raw

PATCH v2 2/13 tools/xenstore: do some cleanup of hashtable.c

In reply to Juergen Gross

Do the following cleanups:
- hashtable_count() isn't used at all, so remove it
- replace prime_table_length and max_load_factor with macros
- make hash() static
- add a loadlimit() helper function
- remove the /***/ lines between functions
- do some style corrections

---
 tools/xenstore/hashtable.c | 71 ++++++++++++++------------------------
 tools/xenstore/hashtable.h | 10 ------
 2 files changed, 26 insertions(+), 55 deletions(-)
tools/xenstore/hashtable.c +26 -45
index 3d4466b597..c1b11743bb 100644
@@ -40,22 +40,25 @@ static const unsigned int primes[] = {
 50331653, 100663319, 201326611, 402653189,
 805306457, 1610612741
 };
-const unsigned int prime_table_length = sizeof(primes)/sizeof(primes[0]);
-const unsigned int max_load_factor = 65; /* percentage */
 
-/*****************************************************************************/
-/* indexFor */
-static inline unsigned int
-indexFor(unsigned int tablelength, unsigned int hashvalue) {
+#define PRIME_TABLE_LEN   ARRAY_SIZE(primes)
+#define MAX_LOAD_PERCENT  65
+
+static inline unsigned int indexFor(unsigned int tablelength,
+                                    unsigned int hashvalue)
+{
     return (hashvalue % tablelength);
 }
 
-/*****************************************************************************/
-struct hashtable *
-create_hashtable(const void *ctx, unsigned int minsize,
-                 unsigned int (*hashf) (const void *),
-                 int (*eqf) (const void *, const void *),
-                 unsigned int flags)
+static unsigned int loadlimit(unsigned int pindex)
+{
+    return ((uint64_t)primes[pindex] * MAX_LOAD_PERCENT) / 100;
+}
+
+struct hashtable *create_hashtable(const void *ctx, unsigned int minsize,
+                                   unsigned int (*hashf) (const void *),
+                                   int (*eqf) (const void *, const void *),
+                                   unsigned int flags)
 {
     struct hashtable *h;
     unsigned int pindex, size = primes[0];
@@ -64,7 +67,7 @@ create_hashtable(const void *ctx, unsigned int minsize,
     if (minsize > (1u << 30)) return NULL;
 
     /* Enforce size as prime */
-    for (pindex=0; pindex < prime_table_length; pindex++) {
+    for (pindex=0; pindex < PRIME_TABLE_LEN; pindex++) {
         if (primes[pindex] > minsize) { size = primes[pindex]; break; }
     }
 
@@ -81,7 +84,7 @@ create_hashtable(const void *ctx, unsigned int minsize,
     h->entrycount   = 0;
     h->hashfn       = hashf;
     h->eqfn         = eqf;
-    h->loadlimit    = (unsigned int)(((uint64_t)size * max_load_factor) / 100);
+    h->loadlimit    = loadlimit(pindex);
     return h;
 
 err1:
@@ -90,9 +93,7 @@ err0:
    return NULL;
 }
 
-/*****************************************************************************/
-unsigned int
-hash(const struct hashtable *h, const void *k)
+static unsigned int hash(const struct hashtable *h, const void *k)
 {
     /* Aim to protect against poor hash functions by adding logic here
      * - logic taken from java 1.4 hashtable source */
@@ -104,9 +105,7 @@ hash(const struct hashtable *h, const void *k)
     return i;
 }
 
-/*****************************************************************************/
-static int
-hashtable_expand(struct hashtable *h)
+static int hashtable_expand(struct hashtable *h)
 {
     /* Double the size of the table to accomodate more entries */
     struct entry **newtable;
@@ -114,7 +113,7 @@ hashtable_expand(struct hashtable *h)
     struct entry **pE;
     unsigned int newsize, i, index;
     /* Check we're not hitting max capacity */
-    if (h->primeindex == (prime_table_length - 1)) return 0;
+    if (h->primeindex == (PRIME_TABLE_LEN - 1)) return 0;
     newsize = primes[++(h->primeindex)];
 
     newtable = talloc_realloc(h, h->table, struct entry *, newsize);
@@ -144,21 +143,11 @@ hashtable_expand(struct hashtable *h)
     }
 
     h->tablelength = newsize;
-    h->loadlimit   = (unsigned int)
-        (((uint64_t)newsize * max_load_factor) / 100);
+    h->loadlimit   = loadlimit(h->primeindex);
     return -1;
 }
 
-/*****************************************************************************/
-unsigned int
-hashtable_count(const struct hashtable *h)
-{
-    return h->entrycount;
-}
-
-/*****************************************************************************/
-int
-hashtable_insert(struct hashtable *h, void *k, void *v)
+int hashtable_insert(struct hashtable *h, void *k, void *v)
 {
     /* This method allows duplicate keys - but they shouldn't be used */
     unsigned int index;
@@ -186,9 +175,7 @@ hashtable_insert(struct hashtable *h, void *k, void *v)
     return -1;
 }
 
-/*****************************************************************************/
-void * /* returns value associated with key */
-hashtable_search(const struct hashtable *h, const void *k)
+void *hashtable_search(const struct hashtable *h, const void *k)
 {
     struct entry *e;
     unsigned int hashvalue, index;
@@ -204,7 +191,6 @@ hashtable_search(const struct hashtable *h, const void *k)
     return NULL;
 }
 
-/*****************************************************************************/
 void
 hashtable_remove(struct hashtable *h, const void *k)
 {
@@ -234,10 +220,8 @@ hashtable_remove(struct hashtable *h, const void *k)
     }
 }
 
-/*****************************************************************************/
-int
-hashtable_iterate(struct hashtable *h,
-                  int (*func)(const void *k, void *v, void *arg), void *arg)
+int hashtable_iterate(struct hashtable *h,
+                      int (*func)(const void *k, void *v, void *arg), void *arg)
 {
     int ret;
     unsigned int i;
@@ -260,10 +244,7 @@ hashtable_iterate(struct hashtable *h,
     return 0;
 }
 
-/*****************************************************************************/
-/* destroy */
-void
-hashtable_destroy(struct hashtable *h)
+void hashtable_destroy(struct hashtable *h)
 {
     talloc_free(h);
 }
tools/xenstore/hashtable.h +0 -10
index cc0090f133..04310783b6 100644
@@ -74,16 +74,6 @@ hashtable_search(const struct hashtable *h, const void *k);
 void
 hashtable_remove(struct hashtable *h, const void *k);
 
-/*****************************************************************************
- * hashtable_count
-   
- * @name        hashtable_count
- * @param   h   the hashtable
- * @return      the number of items stored in the hashtable
- */
-unsigned int
-hashtable_count(const struct hashtable *h);
-
 /*****************************************************************************
  * hashtable_iterate
 
Signature · 1 line
2.35.3
Reviewed-by on v2 2/13
#4 lore raw

Re: [PATCH v2 02/13] tools/xenstore: do some cleanup of hashtable.c

Hi Juergen,

Quoted 57 lines · Juergen Gross

Do the following cleanups:
- hashtable_count() isn't used at all, so remove it
- replace prime_table_length and max_load_factor with macros
- make hash() static
- add a loadlimit() helper function
- remove the /***/ lines between functions
- do some style corrections

---
  tools/xenstore/hashtable.c | 71 ++++++++++++++------------------------
  tools/xenstore/hashtable.h | 10 ------
  2 files changed, 26 insertions(+), 55 deletions(-)
tools/xenstore/hashtable.c +15 -12
index 3d4466b597..c1b11743bb 100644
@@ -40,22 +40,25 @@ static const unsigned int primes[] = {
  50331653, 100663319, 201326611, 402653189,
  805306457, 1610612741
  };
-const unsigned int prime_table_length = sizeof(primes)/sizeof(primes[0]);
-const unsigned int max_load_factor = 65; /* percentage */
 
-/*****************************************************************************/
-/* indexFor */
-static inline unsigned int
-indexFor(unsigned int tablelength, unsigned int hashvalue) {
+#define PRIME_TABLE_LEN   ARRAY_SIZE(primes)
+#define MAX_LOAD_PERCENT  65
+
+static inline unsigned int indexFor(unsigned int tablelength,
+                                    unsigned int hashvalue)
+{
      return (hashvalue % tablelength);
  }
 
-/*****************************************************************************/
-struct hashtable *
-create_hashtable(const void *ctx, unsigned int minsize,
-                 unsigned int (*hashf) (const void *),
-                 int (*eqf) (const void *, const void *),
-                 unsigned int flags)
+static unsigned int loadlimit(unsigned int pindex)
+{
+    return ((uint64_t)primes[pindex] * MAX_LOAD_PERCENT) / 100;
+}
+
+struct hashtable *create_hashtable(const void *ctx, unsigned int minsize,
+                                   unsigned int (*hashf) (const void *),
+                                   int (*eqf) (const void *, const void *),
+                                   unsigned int flags)
  {
      struct hashtable *h;
      unsigned int pindex, size = primes[0];

tools/xenstore/hashtable.c create_hashtable(const void *ctx, unsigned int minsize, · line 70

Quoted 5 lines
tools/xenstore/hashtable.c +1 -1
@@ -64,7 +67,7 @@ create_hashtable(const void *ctx, unsigned int minsize,
      if (minsize > (1u << 30)) return NULL;
 
      /* Enforce size as prime */
-    for (pindex=0; pindex < prime_table_length; pindex++) {
+    for (pindex=0; pindex < PRIME_TABLE_LEN; pindex++) {

As you fix the style, how about adding a space before/after '=' and...

tools/xenstore/hashtable.c create_hashtable(const void *ctx, unsigned int minsize, · line 71

Quoted 1 line
tools/xenstore/hashtable.c +0 -0
          if (primes[pindex] > minsize) { size = primes[pindex]; break; }

... break this line in multiple ones?

With or without this included here:

Cheers,

Signature · 1 line
Julien Grall
#5 lore raw

Re: [PATCH v2 02/13] tools/xenstore: do some cleanup of hashtable.c

Quoted 65 lines · Julien Grall

Hi Juergen,

Quoted 55 lines · Juergen Gross

Do the following cleanups:
- hashtable_count() isn't used at all, so remove it
- replace prime_table_length and max_load_factor with macros
- make hash() static
- add a loadlimit() helper function
- remove the /***/ lines between functions
- do some style corrections

---
  tools/xenstore/hashtable.c | 71 ++++++++++++++------------------------ tools/xenstore/hashtable.h | 10 ------

  2 files changed, 26 insertions(+), 55 deletions(-)
tools/xenstore/hashtable.c +0 -0
index 3d4466b597..c1b11743bb 100644
@@ -40,22 +40,25 @@ static const unsigned int primes[] = {
   50331653, 100663319, 201326611, 402653189,

  805306457, 1610612741
  };

tools/xenstore/hashtable.c +6 -6
-const unsigned int prime_table_length = sizeof(primes)/sizeof(primes[0]);
-const unsigned int max_load_factor = 65; /* percentage */
-/*****************************************************************************/
-/* indexFor */
-static inline unsigned int
-indexFor(unsigned int tablelength, unsigned int hashvalue) {
+#define PRIME_TABLE_LEN   ARRAY_SIZE(primes)
+#define MAX_LOAD_PERCENT  65
+
+static inline unsigned int indexFor(unsigned int tablelength,
+                                    unsigned int hashvalue)
+{

      return (hashvalue % tablelength);
  }

tools/xenstore/hashtable.c +9 -6
-/*****************************************************************************/
-struct hashtable *
-create_hashtable(const void *ctx, unsigned int minsize,
-                 unsigned int (*hashf) (const void *),
-                 int (*eqf) (const void *, const void *),
-                 unsigned int flags)
+static unsigned int loadlimit(unsigned int pindex)
+{
+    return ((uint64_t)primes[pindex] * MAX_LOAD_PERCENT) / 100;
+}
+
+struct hashtable *create_hashtable(const void *ctx, unsigned int minsize,
+                                   unsigned int (*hashf) (const void *),
+                                   int (*eqf) (const void *, const void *),
+                                   unsigned int flags)

  {
      struct hashtable *h;
      unsigned int pindex, size = primes[0];

tools/xenstore/hashtable.c create_hashtable(const void *ctx, unsigned int minsize, · line 69

Quoted 4 lines
tools/xenstore/hashtable.c +1 -1
@@ -64,7 +67,7 @@ create_hashtable(const void *ctx, unsigned int minsize,
       if (minsize > (1u << 30)) return NULL;
       /* Enforce size as prime */
-    for (pindex=0; pindex < prime_table_length; pindex++) {
+    for (pindex=0; pindex < PRIME_TABLE_LEN; pindex++) {

As you fix the style, how about adding a space before/after '=' and...

Quoted 3 lines

tools/xenstore/hashtable.c create_hashtable(const void *ctx, unsigned int minsize, · line 71

Quoted 1 line
tools/xenstore/hashtable.c +0 -0
           if (primes[pindex] > minsize) { size = primes[pindex]; break; }

... break this line in multiple ones?

Will do both.

Quoted 4 lines

With or without this included here:

Thanks,

Juergen

#6 lore raw

PATCH v2 3/13 tools/xenstore: modify interface of create_hashtable()

In reply to Juergen Gross

The minsize parameter of create_hashtable() doesn't have any real use case for Xenstore, so drop it.

For better talloc_report_full() diagnostic output add a name parameter to create_hashtable().

---
 tools/xenstore/hashtable.c        | 20 ++++++--------------
 tools/xenstore/hashtable.h        |  4 ++--
 tools/xenstore/xenstored_core.c   |  2 +-
 tools/xenstore/xenstored_domain.c |  4 ++--
 4 files changed, 11 insertions(+), 19 deletions(-)
tools/xenstore/hashtable.c +6 -14
index c1b11743bb..ab1e687d0b 100644
@@ -55,36 +55,28 @@ static unsigned int loadlimit(unsigned int pindex)
     return ((uint64_t)primes[pindex] * MAX_LOAD_PERCENT) / 100;
 }
 
-struct hashtable *create_hashtable(const void *ctx, unsigned int minsize,
+struct hashtable *create_hashtable(const void *ctx, const char *name,
                                    unsigned int (*hashf) (const void *),
                                    int (*eqf) (const void *, const void *),
                                    unsigned int flags)
 {
     struct hashtable *h;
-    unsigned int pindex, size = primes[0];
-
-    /* Check requested hashtable isn't too large */
-    if (minsize > (1u << 30)) return NULL;
-
-    /* Enforce size as prime */
-    for (pindex=0; pindex < PRIME_TABLE_LEN; pindex++) {
-        if (primes[pindex] > minsize) { size = primes[pindex]; break; }
-    }
 
     h = talloc_zero(ctx, struct hashtable);
     if (NULL == h)
         goto err0;
-    h->table = talloc_zero_array(h, struct entry *, size);
+    talloc_set_name_const(h, name);
+    h->table = talloc_zero_array(h, struct entry *, primes[0]);
     if (NULL == h->table)
         goto err1;
 
-    h->tablelength  = size;
+    h->tablelength  = primes[0];
     h->flags        = flags;
-    h->primeindex   = pindex;
+    h->primeindex   = 0;
     h->entrycount   = 0;
     h->hashfn       = hashf;
     h->eqfn         = eqf;
-    h->loadlimit    = loadlimit(pindex);
+    h->loadlimit    = loadlimit(0);
     return h;
 
 err1:
tools/xenstore/hashtable.h +2 -2
index 04310783b6..0e1a6f61c2 100644
@@ -10,7 +10,7 @@ struct hashtable;
    
  * @name                    create_hashtable
  * @param   ctx             talloc context to use for allocations
- * @param   minsize         minimum initial size of hashtable
+ * @param   name            talloc name of the hashtable
  * @param   hashfunction    function for hashing keys
  * @param   key_eq_fn       function for determining key equality
  * @param   flags           flags HASHTABLE_*
@@ -23,7 +23,7 @@ struct hashtable;
 #define HASHTABLE_FREE_KEY   (1U << 1)
 
 struct hashtable *
-create_hashtable(const void *ctx, unsigned int minsize,
+create_hashtable(const void *ctx, const char *name,
                  unsigned int (*hashfunction) (const void *),
                  int (*key_eq_fn) (const void *, const void *),
                  unsigned int flags
tools/xenstore/xenstored_core.c +1 -1
index 7214b3df03..6ce7be3161 100644
@@ -2511,7 +2511,7 @@ void check_store(void)
 	struct check_store_data data;
 
 	/* Don't free values (they are all void *1) */
-	data.reachable = create_hashtable(NULL, 16, hash_from_key_fn,
+	data.reachable = create_hashtable(NULL, "checkstore", hash_from_key_fn,
 					  keys_equal_fn, HASHTABLE_FREE_KEY);
 	if (!data.reachable) {
 		log("check_store: ENOMEM");
tools/xenstore/xenstored_domain.c +2 -2
index df64c87efc..6d40aefc63 100644
@@ -1017,7 +1017,7 @@ void domain_init(int evtfd)
 	int rc;
 
 	/* Start with a random rather low domain count for the hashtable. */
-	domhash = create_hashtable(NULL, 8, domhash_fn, domeq_fn, 0);
+	domhash = create_hashtable(NULL, "domains", domhash_fn, domeq_fn, 0);
 	if (!domhash)
 		barf_perror("Failed to allocate domain hashtable");
 
@@ -1804,7 +1804,7 @@ struct hashtable *domain_check_acc_init(void)
 {
 	struct hashtable *domains;
 
-	domains = create_hashtable(NULL, 8, domhash_fn, domeq_fn,
+	domains = create_hashtable(NULL, "domain_check", domhash_fn, domeq_fn,
 				   HASHTABLE_FREE_VALUE);
 	if (!domains)
 		return NULL;
Signature · 1 line
2.35.3
#7 lore raw

Re: [PATCH v2 03/13] tools/xenstore: modify interface of create_hashtable()

Hi Juergen,

Quoted 40 lines · Juergen Gross

The minsize parameter of create_hashtable() doesn't have any real use case for Xenstore, so drop it.

For better talloc_report_full() diagnostic output add a name parameter to create_hashtable().

---
  tools/xenstore/hashtable.c        | 20 ++++++--------------
  tools/xenstore/hashtable.h        |  4 ++--
  tools/xenstore/xenstored_core.c   |  2 +-
  tools/xenstore/xenstored_domain.c |  4 ++--
  4 files changed, 11 insertions(+), 19 deletions(-)
tools/xenstore/hashtable.c +1 -10
index c1b11743bb..ab1e687d0b 100644
@@ -55,36 +55,28 @@ static unsigned int loadlimit(unsigned int pindex)
      return ((uint64_t)primes[pindex] * MAX_LOAD_PERCENT) / 100;
  }
 
-struct hashtable *create_hashtable(const void *ctx, unsigned int minsize,
+struct hashtable *create_hashtable(const void *ctx, const char *name,
                                     unsigned int (*hashf) (const void *),
                                     int (*eqf) (const void *, const void *),
                                     unsigned int flags)
  {
      struct hashtable *h;
-    unsigned int pindex, size = primes[0];
-
-    /* Check requested hashtable isn't too large */
-    if (minsize > (1u << 30)) return NULL;
-
-    /* Enforce size as prime */
-    for (pindex=0; pindex < PRIME_TABLE_LEN; pindex++) {
-        if (primes[pindex] > minsize) { size = primes[pindex]; break; }
-    }
 
      h = talloc_zero(ctx, struct hashtable);

tools/xenstore/hashtable.c static unsigned int loadlimit(unsigned int pindex) · line 73

Quoted 10 lines
tools/xenstore/hashtable.c +3 -2
      if (NULL == h)
          goto err0;
-    h->table = talloc_zero_array(h, struct entry *, size);
+    talloc_set_name_const(h, name);
+    h->table = talloc_zero_array(h, struct entry *, primes[0]);
      if (NULL == h->table)
          goto err1;
 
-    h->tablelength  = size;
+    h->tablelength  = primes[0];

I find the connection between this line, ...

tools/xenstore/hashtable.c static unsigned int loadlimit(unsigned int pindex) · line 75

Quoted 3 lines
tools/xenstore/hashtable.c +1 -1
      h->flags        = flags;
-    h->primeindex   = pindex;
+    h->primeindex   = 0;

... this one and ...

tools/xenstore/hashtable.c static unsigned int loadlimit(unsigned int pindex) · line 79

Quoted 5 lines
tools/xenstore/hashtable.c +1 -1
      h->entrycount   = 0;
      h->hashfn       = hashf;
      h->eqfn         = eqf;
-    h->loadlimit    = loadlimit(pindex);
+    h->loadlimit    = loadlimit(0);

... now more difficult to find. How about setting h->primeindex first and then using it in place of 0?

Quoted 52 lines
tools/xenstore/hashtable.c +0 -0
      return h;
 
  err1:
tools/xenstore/hashtable.h +2 -2
index 04310783b6..0e1a6f61c2 100644
@@ -10,7 +10,7 @@ struct hashtable;
 
   * @name                    create_hashtable
   * @param   ctx             talloc context to use for allocations
- * @param   minsize         minimum initial size of hashtable
+ * @param   name            talloc name of the hashtable
   * @param   hashfunction    function for hashing keys
   * @param   key_eq_fn       function for determining key equality
   * @param   flags           flags HASHTABLE_*
@@ -23,7 +23,7 @@ struct hashtable;
  #define HASHTABLE_FREE_KEY   (1U << 1)
 
  struct hashtable *
-create_hashtable(const void *ctx, unsigned int minsize,
+create_hashtable(const void *ctx, const char *name,
                   unsigned int (*hashfunction) (const void *),
                   int (*key_eq_fn) (const void *, const void *),
                   unsigned int flags
tools/xenstore/xenstored_core.c +1 -1
index 7214b3df03..6ce7be3161 100644
@@ -2511,7 +2511,7 @@ void check_store(void)
  	struct check_store_data data;
 
  	/* Don't free values (they are all void *1) */
-	data.reachable = create_hashtable(NULL, 16, hash_from_key_fn,
+	data.reachable = create_hashtable(NULL, "checkstore", hash_from_key_fn,
  					  keys_equal_fn, HASHTABLE_FREE_KEY);
  	if (!data.reachable) {
  		log("check_store: ENOMEM");
tools/xenstore/xenstored_domain.c +1 -1
index df64c87efc..6d40aefc63 100644
@@ -1017,7 +1017,7 @@ void domain_init(int evtfd)
  	int rc;
 
  	/* Start with a random rather low domain count for the hashtable. */
-	domhash = create_hashtable(NULL, 8, domhash_fn, domeq_fn, 0);
+	domhash = create_hashtable(NULL, "domains", domhash_fn, domeq_fn, 0);
  	if (!domhash)
  		barf_perror("Failed to allocate domain hashtable");
 

tools/xenstore/xenstored_domain.c struct hashtable *domain_check_acc_init(void) · line 1810

Quoted 8 lines
tools/xenstore/xenstored_domain.c +1 -1
@@ -1804,7 +1804,7 @@ struct hashtable *domain_check_acc_init(void)
  {
  	struct hashtable *domains;
 
-	domains = create_hashtable(NULL, 8, domhash_fn, domeq_fn,
+	domains = create_hashtable(NULL, "domain_check", domhash_fn, domeq_fn,
  				   HASHTABLE_FREE_VALUE);
  	if (!domains)
  		return NULL;

Cheers,

Signature · 1 line
Julien Grall
#8 lore raw

Re: [PATCH v2 03/13] tools/xenstore: modify interface of create_hashtable()

Quoted 59 lines · Julien Grall

Hi Juergen,

Quoted 37 lines · Juergen Gross

The minsize parameter of create_hashtable() doesn't have any real use case for Xenstore, so drop it.

For better talloc_report_full() diagnostic output add a name parameter to create_hashtable().

---
  tools/xenstore/hashtable.c        | 20 ++++++-------------- tools/xenstore/hashtable.h        |  4 ++-- tools/xenstore/xenstored_core.c   |  2 +- tools/xenstore/xenstored_domain.c |  4 ++--

  4 files changed, 11 insertions(+), 19 deletions(-)
tools/xenstore/hashtable.c +1 -10
index c1b11743bb..ab1e687d0b 100644
@@ -55,36 +55,28 @@ static unsigned int loadlimit(unsigned int pindex)
       return ((uint64_t)primes[pindex] * MAX_LOAD_PERCENT) / 100;
   }
-struct hashtable *create_hashtable(const void *ctx, unsigned int minsize,
+struct hashtable *create_hashtable(const void *ctx, const char *name,
                                      unsigned int (*hashf) (const void *),
                                      int (*eqf) (const void *, const void *),
                                      unsigned int flags)
   {
       struct hashtable *h;
-    unsigned int pindex, size = primes[0];
-
-    /* Check requested hashtable isn't too large */
-    if (minsize > (1u << 30)) return NULL;
-
-    /* Enforce size as prime */
-    for (pindex=0; pindex < PRIME_TABLE_LEN; pindex++) {
-        if (primes[pindex] > minsize) { size = primes[pindex]; break; }
-    }

tools/xenstore/hashtable.c static unsigned int loadlimit(unsigned int pindex) · line 70

Quoted 10 lines
tools/xenstore/hashtable.c +3 -2
       h = talloc_zero(ctx, struct hashtable);
       if (NULL == h)
           goto err0;
-    h->table = talloc_zero_array(h, struct entry *, size);
+    talloc_set_name_const(h, name);
+    h->table = talloc_zero_array(h, struct entry *, primes[0]);
       if (NULL == h->table)
           goto err1;
-    h->tablelength  = size;
+    h->tablelength  = primes[0];

I find the connection between this line, ...

tools/xenstore/hashtable.c static unsigned int loadlimit(unsigned int pindex) · line 75

Quoted 3 lines
tools/xenstore/hashtable.c +1 -1
       h->flags        = flags;
-    h->primeindex   = pindex;
+    h->primeindex   = 0;

... this one and ...

Quoted 7 lines

tools/xenstore/hashtable.c static unsigned int loadlimit(unsigned int pindex) · line 79

Quoted 5 lines
tools/xenstore/hashtable.c +1 -1
       h->entrycount   = 0;
       h->hashfn       = hashf;
       h->eqfn         = eqf;
-    h->loadlimit    = loadlimit(pindex);
+    h->loadlimit    = loadlimit(0);

... now more difficult to find. How about setting h->primeindex first and then using it in place of 0?

Fine with me.

Juergen

#9 lore raw

PATCH v2 4/13 tools/xenstore: let hashtable_insert() return 0 on success

In reply to Juergen Gross

Today hashtable_insert() returns 0 in case of an error. Change that to let it return an errno value in the error case and 0 in case of success.

Even if not used today, do the same switch for the return value of hashtable_expand().

---
 tools/xenstore/hashtable.c             | 15 ++++++++++-----
 tools/xenstore/hashtable.h             |  2 +-
 tools/xenstore/xenstored_core.c        |  4 ++--
 tools/xenstore/xenstored_domain.c      |  4 ++--
 tools/xenstore/xenstored_transaction.c |  4 ++--
 5 files changed, 17 insertions(+), 12 deletions(-)
tools/xenstore/hashtable.c +10 -5
index ab1e687d0b..084d562b22 100644
@@ -105,14 +105,15 @@ static int hashtable_expand(struct hashtable *h)
     struct entry **pE;
     unsigned int newsize, i, index;
     /* Check we're not hitting max capacity */
-    if (h->primeindex == (PRIME_TABLE_LEN - 1)) return 0;
+    if (h->primeindex == (PRIME_TABLE_LEN - 1))
+        return ENOSPC;
     newsize = primes[++(h->primeindex)];
 
     newtable = talloc_realloc(h, h->table, struct entry *, newsize);
     if (!newtable)
     {
         h->primeindex--;
-        return 0;
+        return ENOMEM;
     }
 
     h->table = newtable;
@@ -136,7 +137,7 @@ static int hashtable_expand(struct hashtable *h)
 
     h->tablelength = newsize;
     h->loadlimit   = loadlimit(h->primeindex);
-    return -1;
+    return 0;
 }
 
 int hashtable_insert(struct hashtable *h, void *k, void *v)
@@ -153,7 +154,11 @@ int hashtable_insert(struct hashtable *h, void *k, void *v)
         hashtable_expand(h);
     }
     e = talloc_zero(h, struct entry);
-    if (NULL == e) { --(h->entrycount); return 0; } /*oom*/
+    if (NULL == e)
+    {
+        --h->entrycount;
+       return ENOMEM;
+    }
     e->h = hash(h,k);
     index = indexFor(h->tablelength,e->h);
     e->k = k;
@@ -164,7 +169,7 @@ int hashtable_insert(struct hashtable *h, void *k, void *v)
         talloc_steal(e, v);
     e->next = h->table[index];
     h->table[index] = e;
-    return -1;
+    return 0;
 }
 
 void *hashtable_search(const struct hashtable *h, const void *k)
tools/xenstore/hashtable.h +1 -1
index 0e1a6f61c2..ee4e25cbbf 100644
@@ -36,7 +36,7 @@ create_hashtable(const void *ctx, const char *name,
  * @param   h   the hashtable to insert into
  * @param   k   the key - hashtable claims ownership and will free on removal
  * @param   v   the value - does not claim ownership
- * @return      non-zero for successful insertion
+ * @return      zero for successful insertion
  *
  * This function will cause the table to expand if the insertion would take
  * the ratio of entries to table size over the maximum load factor.
tools/xenstore/xenstored_core.c +2 -2
index 6ce7be3161..c5433e5b3f 100644
@@ -2394,7 +2394,7 @@ int remember_string(struct hashtable *hash, const char *str)
 	char *k = talloc_strdup(NULL, str);
 
 	if (!k)
-		return 0;
+		return ENOMEM;
 	return hashtable_insert(hash, k, (void *)1);
 }
 
@@ -2428,7 +2428,7 @@ static int check_store_step(const void *ctx, struct connection *conn,
 				: WALK_TREE_SKIP_CHILDREN;
 	}
 
-	if (!remember_string(data->reachable, node->name))
+	if (remember_string(data->reachable, node->name))
 		return WALK_TREE_ERROR_STOP;
 
 	domain_check_acc_add(node, data->domains);
tools/xenstore/xenstored_domain.c +2 -2
index 6d40aefc63..be9175a461 100644
@@ -540,7 +540,7 @@ static struct domain *alloc_domain(const void *context, unsigned int domid)
 	domain->generation = generation;
 	domain->introduced = false;
 
-	if (!hashtable_insert(domhash, &domain->domid, domain)) {
+	if (hashtable_insert(domhash, &domain->domid, domain)) {
 		talloc_free(domain);
 		errno = ENOMEM;
 		return NULL;
@@ -1792,7 +1792,7 @@ static int domain_check_acc_init_sub(const void *k, void *v, void *arg)
 	 */
 	dom->nodes = -d->acc[ACC_NODES].val;
 
-	if (!hashtable_insert(domains, &dom->domid, dom)) {
+	if (hashtable_insert(domains, &dom->domid, dom)) {
 		talloc_free(dom);
 		return -1;
 	}
tools/xenstore/xenstored_transaction.c +2 -2
index e8968d7a1d..c615c0de13 100644
@@ -600,13 +600,13 @@ int check_transactions(struct hashtable *hash)
 		list_for_each_entry(trans, &conn->transaction_list, list) {
 			tname = talloc_asprintf(trans, "%"PRIu64,
 						trans->generation);
-			if (!tname || !remember_string(hash, tname))
+			if (!tname || remember_string(hash, tname))
 				goto nomem;
 
 			list_for_each_entry(i, &trans->accessed, list) {
 				if (!i->ta_node)
 					continue;
-				if (!remember_string(hash, i->trans_name))
+				if (remember_string(hash, i->trans_name))
 					goto nomem;
 			}
 
Signature · 1 line
2.35.3
#10 lore raw

Re: [PATCH v2 04/13] tools/xenstore: let hashtable_insert() return 0 on success

Hi,

Quoted 2 lines · Juergen Gross

Today hashtable_insert() returns 0 in case of an error. Change that to let it return an errno value in the error case and 0 in case of success.

I usually find such change risky because it makes the backport more complex if we introduce a new call to hashtable_insert() and it is also quite difficult to review (the compiler would not help to confirm all the callers have changed).

So can you provide a compelling reason for doing the change? (consistency would not be one IMO)

Cheers,

Signature · 1 line
Julien Grall
#11 lore raw

Re: [PATCH v2 04/13] tools/xenstore: let hashtable_insert() return 0 on success

Quoted 9 lines · Julien Grall

Hi,

Quoted 2 lines · Juergen Gross

Today hashtable_insert() returns 0 in case of an error. Change that to let it return an errno value in the error case and 0 in case of success.

I usually find such change risky because it makes the backport more complex if we introduce a new call to hashtable_insert() and it is also quite difficult to review (the compiler would not help to confirm all the callers have changed).

So can you provide a compelling reason for doing the change? (consistency would not be one IMO)

The motivation was consistency. :-)

The alternative would be to really set errno in the error case, which would add additional lines.

I'm not really feeling strong here, BTW.

Juergen

#12 lore raw

PATCH v2 5/13 tools/xenstore: make some write limit functions static

In reply to Juergen Gross

Some wrl_*() functions are only used in xenstored_domain.c, so make them static. In order to avoid the need of forward declarations, move the whole function block to the start of the file.

---
 tools/xenstore/xenstored_domain.c | 456 +++++++++++++++---------------
 tools/xenstore/xenstored_domain.h |   3 -
 2 files changed, 228 insertions(+), 231 deletions(-)
tools/xenstore/xenstored_domain.c +228 -228
index be9175a461..0a8353dad5 100644
@@ -159,6 +159,234 @@ struct changed_domain
 
 static struct hashtable *domhash;
 
+static wrl_creditt wrl_config_writecost      = WRL_FACTOR;
+static wrl_creditt wrl_config_rate           = WRL_RATE   * WRL_FACTOR;
+static wrl_creditt wrl_config_dburst         = WRL_DBURST * WRL_FACTOR;
+static wrl_creditt wrl_config_gburst         = WRL_GBURST * WRL_FACTOR;
+static wrl_creditt wrl_config_newdoms_dburst =
+	                         WRL_DBURST * WRL_NEWDOMS * WRL_FACTOR;
+
+long wrl_ntransactions;
+
+static long wrl_ndomains;
+static wrl_creditt wrl_reserve; /* [-wrl_config_newdoms_dburst, +_gburst ] */
+static time_t wrl_log_last_warning; /* 0: no previous warning */
+
+#define trace_wrl(...)				\
+do {						\
+	if (trace_flags & TRACE_WRL)		\
+		trace("wrl: " __VA_ARGS__);	\
+} while (0)
+
+void wrl_gettime_now(struct wrl_timestampt *now_wt)
+{
+	struct timespec now_ts;
+	int r;
+
+	r = clock_gettime(CLOCK_MONOTONIC, &now_ts);
+	if (r)
+		barf_perror("Could not find time (clock_gettime failed)");
+
+	now_wt->sec = now_ts.tv_sec;
+	now_wt->msec = now_ts.tv_nsec / 1000000;
+}
+
+static void wrl_xfer_credit(wrl_creditt *debit,  wrl_creditt debit_floor,
+			    wrl_creditt *credit, wrl_creditt credit_ceil)
+	/*
+	 * Transfers zero or more credit from "debit" to "credit".
+	 * Transfers as much as possible while maintaining
+	 * debit >= debit_floor and credit <= credit_ceil.
+	 * (If that's violated already, does nothing.)
+	 *
+	 * Sufficient conditions to avoid overflow, either of:
+	 *  |every argument| <= 0x3fffffff
+	 *  |every argument| <= 1E9
+	 *  |every argument| <= WRL_CREDIT_MAX
+	 * (And this condition is preserved.)
+	 */
+{
+	wrl_creditt xfer = MIN( *debit      - debit_floor,
+			        credit_ceil - *credit      );
+	if (xfer > 0) {
+		*debit -= xfer;
+		*credit += xfer;
+	}
+}
+
+static void wrl_domain_new(struct domain *domain)
+{
+	domain->wrl_credit = 0;
+	wrl_gettime_now(&domain->wrl_timestamp);
+	wrl_ndomains++;
+	/* Steal up to DBURST from the reserve */
+	wrl_xfer_credit(&wrl_reserve, -wrl_config_newdoms_dburst,
+			&domain->wrl_credit, wrl_config_dburst);
+}
+
+static void wrl_domain_destroy(struct domain *domain)
+{
+	wrl_ndomains--;
+	/*
+	 * Don't bother recalculating domain's credit - this just
+	 * means we don't give the reserve the ending domain's credit
+	 * for time elapsed since last update.
+	 */
+	wrl_xfer_credit(&domain->wrl_credit, 0,
+			&wrl_reserve, wrl_config_dburst);
+}
+
+static void wrl_credit_update(struct domain *domain, struct wrl_timestampt now)
+{
+	/*
+	 * We want to calculate
+	 *    credit += (now - timestamp) * RATE / ndoms;
+	 * But we want it to saturate, and to avoid floating point.
+	 * To avoid rounding errors from constantly adding small
+	 * amounts of credit, we only add credit for whole milliseconds.
+	 */
+	long seconds      = now.sec -  domain->wrl_timestamp.sec;
+	long milliseconds = now.msec - domain->wrl_timestamp.msec;
+	long msec;
+	int64_t denom, num;
+	wrl_creditt surplus;
+
+	seconds = MIN(seconds, 1000*1000); /* arbitrary, prevents overflow */
+	msec = seconds * 1000 + milliseconds;
+
+	if (msec < 0)
+                /* shouldn't happen with CLOCK_MONOTONIC */
+		msec = 0;
+
+	/* 32x32 -> 64 cannot overflow */
+	denom = (int64_t)msec * wrl_config_rate;
+	num  =  (int64_t)wrl_ndomains * 1000;
+	/* denom / num <= 1E6 * wrl_config_rate, so with
+	   reasonable wrl_config_rate, denom / num << 2^64 */
+
+	/* at last! */
+	domain->wrl_credit = MIN( (int64_t)domain->wrl_credit + denom / num,
+				  WRL_CREDIT_MAX );
+	/* (maybe briefly violating the DBURST cap on wrl_credit) */
+
+	/* maybe take from the reserve to make us nonnegative */
+	wrl_xfer_credit(&wrl_reserve,        0,
+			&domain->wrl_credit, 0);
+
+	/* return any surplus (over DBURST) to the reserve */
+	surplus = 0;
+	wrl_xfer_credit(&domain->wrl_credit, wrl_config_dburst,
+			&surplus,            WRL_CREDIT_MAX);
+	wrl_xfer_credit(&surplus,     0,
+			&wrl_reserve, wrl_config_gburst);
+	/* surplus is now implicitly discarded */
+
+	domain->wrl_timestamp = now;
+
+	trace_wrl("dom %4d %6ld msec %9ld credit  %9ld reserve %9ld discard\n",
+		  domain->domid, msec, (long)domain->wrl_credit,
+		  (long)wrl_reserve, (long)surplus);
+}
+
+void wrl_check_timeout(struct domain *domain,
+		       struct wrl_timestampt now,
+		       int *ptimeout)
+{
+	uint64_t num, denom;
+	int wakeup;
+
+	wrl_credit_update(domain, now);
+
+	if (domain->wrl_credit >= 0)
+		/* not blocked */
+		return;
+
+	if (!*ptimeout)
+		/* already decided on immediate wakeup,
+		   so no need to calculate our timeout */
+		return;
+
+	/* calculate  wakeup = now + -credit / (RATE / ndoms); */
+
+	/* credit cannot go more -ve than one transaction,
+	 * so the first multiplication cannot overflow even 32-bit */
+	num   = (uint64_t)(-domain->wrl_credit * 1000) * wrl_ndomains;
+	denom = wrl_config_rate;
+
+	wakeup = MIN( num / denom /* uint64_t */, INT_MAX );
+	if (*ptimeout==-1 || wakeup < *ptimeout)
+		*ptimeout = wakeup;
+
+	trace_wrl("domain %u credit=%ld (reserve=%ld) SLEEPING for %d\n",
+		  domain->domid, (long)domain->wrl_credit, (long)wrl_reserve,
+		  wakeup);
+}
+
+#define WRL_LOG(now, ...) \
+	(syslog(LOG_WARNING, "write rate limit: " __VA_ARGS__))
+
+void wrl_apply_debit_actual(struct domain *domain)
+{
+	struct wrl_timestampt now;
+
+	if (!domain || !domain_is_unprivileged(domain->conn))
+		/* sockets and privileged domain escape the write rate limit */
+		return;
+
+	wrl_gettime_now(&now);
+	wrl_credit_update(domain, now);
+
+	domain->wrl_credit -= wrl_config_writecost;
+	trace_wrl("domain %u credit=%ld (reserve=%ld)\n", domain->domid,
+		  (long)domain->wrl_credit, (long)wrl_reserve);
+
+	if (domain->wrl_credit < 0) {
+		if (!domain->wrl_delay_logged) {
+			domain->wrl_delay_logged = true;
+			WRL_LOG(now, "domain %ld is affected\n",
+				(long)domain->domid);
+		} else if (!wrl_log_last_warning) {
+			WRL_LOG(now, "rate limiting restarts\n");
+		}
+		wrl_log_last_warning = now.sec;
+	}
+}
+
+void wrl_log_periodic(struct wrl_timestampt now)
+{
+	if (wrl_log_last_warning &&
+	    (now.sec - wrl_log_last_warning) > WRL_LOGEVERY) {
+		WRL_LOG(now, "not in force recently\n");
+		wrl_log_last_warning = 0;
+	}
+}
+
+void wrl_apply_debit_direct(struct connection *conn)
+{
+	if (!conn)
+		/* some writes are generated internally */
+		return;
+
+	if (conn->transaction)
+		/* these are accounted for when the transaction ends */
+		return;
+
+	if (!wrl_ntransactions)
+		/* we don't conflict with anyone */
+		return;
+
+	wrl_apply_debit_actual(conn->domain);
+}
+
+void wrl_apply_debit_trans_commit(struct connection *conn)
+{
+	if (wrl_ntransactions <= 1)
+		/* our own transaction appears in the counter */
+		return;
+
+	wrl_apply_debit_actual(conn->domain);
+}
+
 static bool check_indexes(XENSTORE_RING_IDX cons, XENSTORE_RING_IDX prod)
 {
 	return ((prod - cons) <= XENSTORE_RING_SIZE);
@@ -1443,234 +1671,6 @@ unsigned int domain_transaction_get(struct connection *conn)
 		: 0;
 }
 
-static wrl_creditt wrl_config_writecost      = WRL_FACTOR;
-static wrl_creditt wrl_config_rate           = WRL_RATE   * WRL_FACTOR;
-static wrl_creditt wrl_config_dburst         = WRL_DBURST * WRL_FACTOR;
-static wrl_creditt wrl_config_gburst         = WRL_GBURST * WRL_FACTOR;
-static wrl_creditt wrl_config_newdoms_dburst =
-	                         WRL_DBURST * WRL_NEWDOMS * WRL_FACTOR;
-
-long wrl_ntransactions;
-
-static long wrl_ndomains;
-static wrl_creditt wrl_reserve; /* [-wrl_config_newdoms_dburst, +_gburst ] */
-static time_t wrl_log_last_warning; /* 0: no previous warning */
-
-#define trace_wrl(...)				\
-do {						\
-	if (trace_flags & TRACE_WRL)		\
-		trace("wrl: " __VA_ARGS__);	\
-} while (0)
-
-void wrl_gettime_now(struct wrl_timestampt *now_wt)
-{
-	struct timespec now_ts;
-	int r;
-
-	r = clock_gettime(CLOCK_MONOTONIC, &now_ts);
-	if (r)
-		barf_perror("Could not find time (clock_gettime failed)");
-
-	now_wt->sec = now_ts.tv_sec;
-	now_wt->msec = now_ts.tv_nsec / 1000000;
-}
-
-static void wrl_xfer_credit(wrl_creditt *debit,  wrl_creditt debit_floor,
-			    wrl_creditt *credit, wrl_creditt credit_ceil)
-	/*
-	 * Transfers zero or more credit from "debit" to "credit".
-	 * Transfers as much as possible while maintaining
-	 * debit >= debit_floor and credit <= credit_ceil.
-	 * (If that's violated already, does nothing.)
-	 *
-	 * Sufficient conditions to avoid overflow, either of:
-	 *  |every argument| <= 0x3fffffff
-	 *  |every argument| <= 1E9
-	 *  |every argument| <= WRL_CREDIT_MAX
-	 * (And this condition is preserved.)
-	 */
-{
-	wrl_creditt xfer = MIN( *debit      - debit_floor,
-			        credit_ceil - *credit      );
-	if (xfer > 0) {
-		*debit -= xfer;
-		*credit += xfer;
-	}
-}
-
-void wrl_domain_new(struct domain *domain)
-{
-	domain->wrl_credit = 0;
-	wrl_gettime_now(&domain->wrl_timestamp);
-	wrl_ndomains++;
-	/* Steal up to DBURST from the reserve */
-	wrl_xfer_credit(&wrl_reserve, -wrl_config_newdoms_dburst,
-			&domain->wrl_credit, wrl_config_dburst);
-}
-
-void wrl_domain_destroy(struct domain *domain)
-{
-	wrl_ndomains--;
-	/*
-	 * Don't bother recalculating domain's credit - this just
-	 * means we don't give the reserve the ending domain's credit
-	 * for time elapsed since last update.
-	 */
-	wrl_xfer_credit(&domain->wrl_credit, 0,
-			&wrl_reserve, wrl_config_dburst);
-}
-
-void wrl_credit_update(struct domain *domain, struct wrl_timestampt now)
-{
-	/*
-	 * We want to calculate
-	 *    credit += (now - timestamp) * RATE / ndoms;
-	 * But we want it to saturate, and to avoid floating point.
-	 * To avoid rounding errors from constantly adding small
-	 * amounts of credit, we only add credit for whole milliseconds.
-	 */
-	long seconds      = now.sec -  domain->wrl_timestamp.sec;
-	long milliseconds = now.msec - domain->wrl_timestamp.msec;
-	long msec;
-	int64_t denom, num;
-	wrl_creditt surplus;
-
-	seconds = MIN(seconds, 1000*1000); /* arbitrary, prevents overflow */
-	msec = seconds * 1000 + milliseconds;
-
-	if (msec < 0)
-                /* shouldn't happen with CLOCK_MONOTONIC */
-		msec = 0;
-
-	/* 32x32 -> 64 cannot overflow */
-	denom = (int64_t)msec * wrl_config_rate;
-	num  =  (int64_t)wrl_ndomains * 1000;
-	/* denom / num <= 1E6 * wrl_config_rate, so with
-	   reasonable wrl_config_rate, denom / num << 2^64 */
-
-	/* at last! */
-	domain->wrl_credit = MIN( (int64_t)domain->wrl_credit + denom / num,
-				  WRL_CREDIT_MAX );
-	/* (maybe briefly violating the DBURST cap on wrl_credit) */
-
-	/* maybe take from the reserve to make us nonnegative */
-	wrl_xfer_credit(&wrl_reserve,        0,
-			&domain->wrl_credit, 0);
-
-	/* return any surplus (over DBURST) to the reserve */
-	surplus = 0;
-	wrl_xfer_credit(&domain->wrl_credit, wrl_config_dburst,
-			&surplus,            WRL_CREDIT_MAX);
-	wrl_xfer_credit(&surplus,     0,
-			&wrl_reserve, wrl_config_gburst);
-	/* surplus is now implicitly discarded */
-
-	domain->wrl_timestamp = now;
-
-	trace_wrl("dom %4d %6ld msec %9ld credit %9ld reserve %9ld discard\n",
-		  domain->domid, msec, (long)domain->wrl_credit,
-		  (long)wrl_reserve, (long)surplus);
-}
-
-void wrl_check_timeout(struct domain *domain,
-		       struct wrl_timestampt now,
-		       int *ptimeout)
-{
-	uint64_t num, denom;
-	int wakeup;
-
-	wrl_credit_update(domain, now);
-
-	if (domain->wrl_credit >= 0)
-		/* not blocked */
-		return;
-
-	if (!*ptimeout)
-		/* already decided on immediate wakeup,
-		   so no need to calculate our timeout */
-		return;
-
-	/* calculate  wakeup = now + -credit / (RATE / ndoms); */
-
-	/* credit cannot go more -ve than one transaction,
-	 * so the first multiplication cannot overflow even 32-bit */
-	num   = (uint64_t)(-domain->wrl_credit * 1000) * wrl_ndomains;
-	denom = wrl_config_rate;
-
-	wakeup = MIN( num / denom /* uint64_t */, INT_MAX );
-	if (*ptimeout==-1 || wakeup < *ptimeout)
-		*ptimeout = wakeup;
-
-	trace_wrl("domain %u credit=%ld (reserve=%ld) SLEEPING for %d\n",
-		  domain->domid, (long)domain->wrl_credit, (long)wrl_reserve,
-		  wakeup);
-}
-
-#define WRL_LOG(now, ...) \
-	(syslog(LOG_WARNING, "write rate limit: " __VA_ARGS__))
-
-void wrl_apply_debit_actual(struct domain *domain)
-{
-	struct wrl_timestampt now;
-
-	if (!domain || !domid_is_unprivileged(domain->domid))
-		/* sockets and privileged domain escape the write rate limit */
-		return;
-
-	wrl_gettime_now(&now);
-	wrl_credit_update(domain, now);
-
-	domain->wrl_credit -= wrl_config_writecost;
-	trace_wrl("domain %u credit=%ld (reserve=%ld)\n", domain->domid,
-		  (long)domain->wrl_credit, (long)wrl_reserve);
-
-	if (domain->wrl_credit < 0) {
-		if (!domain->wrl_delay_logged) {
-			domain->wrl_delay_logged = true;
-			WRL_LOG(now, "domain %ld is affected\n",
-				(long)domain->domid);
-		} else if (!wrl_log_last_warning) {
-			WRL_LOG(now, "rate limiting restarts\n");
-		}
-		wrl_log_last_warning = now.sec;
-	}
-}
-
-void wrl_log_periodic(struct wrl_timestampt now)
-{
-	if (wrl_log_last_warning &&
-	    (now.sec - wrl_log_last_warning) > WRL_LOGEVERY) {
-		WRL_LOG(now, "not in force recently\n");
-		wrl_log_last_warning = 0;
-	}
-}
-
-void wrl_apply_debit_direct(struct connection *conn)
-{
-	if (!conn)
-		/* some writes are generated internally */
-		return;
-
-	if (conn->transaction)
-		/* these are accounted for when the transaction ends */
-		return;
-
-	if (!wrl_ntransactions)
-		/* we don't conflict with anyone */
-		return;
-
-	wrl_apply_debit_actual(conn->domain);
-}
-
-void wrl_apply_debit_trans_commit(struct connection *conn)
-{
-	if (wrl_ntransactions <= 1)
-		/* our own transaction appears in the counter */
-		return;
-
-	wrl_apply_debit_actual(conn->domain);
-}
-
 const char *dump_state_connections(FILE *fp)
 {
 	const char *ret = NULL;
tools/xenstore/xenstored_domain.h +0 -3
index b9e38abff5..8cd69d42ab 100644
@@ -162,9 +162,6 @@ struct wrl_timestampt {
 extern long wrl_ntransactions;
 
 void wrl_gettime_now(struct wrl_timestampt *now_ts);
-void wrl_domain_new(struct domain *domain);
-void wrl_domain_destroy(struct domain *domain);
-void wrl_credit_update(struct domain *domain, struct wrl_timestampt now);
 void wrl_check_timeout(struct domain *domain,
                        struct wrl_timestampt now,
                        int *ptimeout);
Signature · 1 line
2.35.3
#13 lore raw

Re: [PATCH v2 05/13] tools/xenstore: make some write limit functions static

Quoted 7 lines · Juergen Gross
tools/xenstore/xenstored_domain.c +7 -0
+static void wrl_xfer_credit(wrl_creditt *debit,  wrl_creditt debit_floor,
+			    wrl_creditt *credit, wrl_creditt credit_ceil)
+	/*
+	 * Transfers zero or more credit from "debit" to "credit".
+	 * Transfers as much as possible while maintaining
+	 * debit >= debit_floor and credit <= credit_ceil.
+	 * (If that's violated already, does nothing.)

tools/xenstore/xenstored_domain.c struct changed_domain · line 210

Quoted 10 lines
tools/xenstore/xenstored_domain.c +10 -0
+	 *
+	 * Sufficient conditions to avoid overflow, either of:
+	 *  |every argument| <= 0x3fffffff
+	 *  |every argument| <= 1E9
+	 *  |every argument| <= WRL_CREDIT_MAX
+	 * (And this condition is preserved.)
+	 */
+{
+	wrl_creditt xfer = MIN( *debit      - debit_floor,
+			        credit_ceil - *credit      );

MIN() evaluates its parameters multiple times.  I believe the only legal way for the compiler to emit this code is to interleave double reads.

As with pretty much any C code, you want to read the pointers into locals first, then operate on them, then write them out at the end.

~Andrew

#14 lore raw

Re: [PATCH v2 05/13] tools/xenstore: make some write limit functions static

Quoted 22 lines · Andrew Cooper
Quoted 7 lines · Juergen Gross
tools/xenstore/xenstored_domain.c +7 -0
+static void wrl_xfer_credit(wrl_creditt *debit,  wrl_creditt debit_floor,
+			    wrl_creditt *credit, wrl_creditt credit_ceil)
+	/*
+	 * Transfers zero or more credit from "debit" to "credit".
+	 * Transfers as much as possible while maintaining
+	 * debit >= debit_floor and credit <= credit_ceil.
+	 * (If that's violated already, does nothing.)

tools/xenstore/xenstored_domain.c struct changed_domain · line 210

Quoted 10 lines
tools/xenstore/xenstored_domain.c +10 -0
+	 *
+	 * Sufficient conditions to avoid overflow, either of:
+	 *  |every argument| <= 0x3fffffff
+	 *  |every argument| <= 1E9
+	 *  |every argument| <= WRL_CREDIT_MAX
+	 * (And this condition is preserved.)
+	 */
+{
+	wrl_creditt xfer = MIN( *debit      - debit_floor,
+			        credit_ceil - *credit      );

MIN() evaluates its parameters multiple times.  I believe the only legal way for the compiler to emit this code is to interleave double reads.

Quoted 2 lines

As with pretty much any C code, you want to read the pointers into locals first, then operate on them, then write them out at the end.

xenstored is single-threaded. So no need to worry here.

Juergen

#15 lore raw

PATCH v2 6/13 tools/xenstore: switch write limiting to use millisecond time base

In reply to Juergen Gross

There is no need to keep struct wrl_timestampt, as it serves the same purpose as the more simple time base provided by get_now().

Move some more stuff from xenstored_domain.h into xenstored_domain.c as it is being used nowhere else.

---
 tools/xenstore/xenstored_core.c   |  8 ++---
 tools/xenstore/xenstored_core.h   |  7 ++--
 tools/xenstore/xenstored_domain.c | 56 +++++++++++++------------------
 tools/xenstore/xenstored_domain.h | 21 ++----------
 4 files changed, 32 insertions(+), 60 deletions(-)
tools/xenstore/xenstored_core.c +3 -5
index c5433e5b3f..6be2a3aa42 100644
@@ -192,7 +192,7 @@ void reopen_log(void)
 	}
 }
 
-static uint64_t get_now_msec(void)
+uint64_t get_now_msec(void)
 {
 	struct timespec now_ts;
 
@@ -510,7 +510,6 @@ fail:
 static void initialize_fds(int *p_sock_pollfd_idx, int *ptimeout)
 {
 	struct connection *conn;
-	struct wrl_timestampt now;
 	uint64_t msecs;
 
 	if (fds)
@@ -530,13 +529,12 @@ static void initialize_fds(int *p_sock_pollfd_idx, int *ptimeout)
 		xce_pollfd_idx = set_fd(xenevtchn_fd(xce_handle),
 					POLLIN|POLLPRI);
 
-	wrl_gettime_now(&now);
-	wrl_log_periodic(now);
 	msecs = get_now_msec();
+	wrl_log_periodic(msecs);
 
 	list_for_each_entry(conn, &connections, list) {
 		if (conn->domain) {
-			wrl_check_timeout(conn->domain, now, ptimeout);
+			wrl_check_timeout(conn->domain, msecs, ptimeout);
 			check_event_timeout(conn, msecs, ptimeout);
 			if (conn_can_read(conn) ||
 			    (conn_can_write(conn) &&
tools/xenstore/xenstored_core.h +3 -4
index 92d5b50f3c..84a611cbb5 100644
@@ -47,10 +47,6 @@
 /* DEFAULT_BUFFER_SIZE should be large enough for each errno string. */
 #define DEFAULT_BUFFER_SIZE 16
 
-typedef int32_t wrl_creditt;
-#define WRL_CREDIT_MAX (1000*1000*1000)
-/* ^ satisfies non-overflow condition for wrl_xfer_credit */
-
 struct xs_state_connection;
 
 struct buffered_data
@@ -320,6 +316,9 @@ extern bool keep_orphans;
 
 extern unsigned int timeout_watch_event_msec;
 
+/* Get internal time in milliseconds. */
+uint64_t get_now_msec(void);
+
 /* Map the kernel's xenstore page. */
 void *xenbus_map(void);
 void unmap_xenbus(void *interface);
tools/xenstore/xenstored_domain.c +24 -32

130 lines not on this page · raw message

tools/xenstore/xenstored_domain.h +2 -19
index 8cd69d42ab..f9ee0bd141 100644
@@ -145,27 +145,10 @@ void domain_reset_global_acc(void);
 bool domain_max_chk(const struct connection *conn, unsigned int what,
 		    unsigned int val);
 
-/* Write rate limiting */
-
-#define WRL_FACTOR   1000 /* for fixed-point arithmetic */
-#define WRL_RATE      200
-#define WRL_DBURST     10
-#define WRL_GBURST   1000
-#define WRL_NEWDOMS     5
-#define WRL_LOGEVERY  120 /* seconds */
-
-struct wrl_timestampt {
-	time_t sec;
-	int msec;
-};
-
 extern long wrl_ntransactions;
 
-void wrl_gettime_now(struct wrl_timestampt *now_ts);
-void wrl_check_timeout(struct domain *domain,
-                       struct wrl_timestampt now,
-                       int *ptimeout);
-void wrl_log_periodic(struct wrl_timestampt now);
+void wrl_check_timeout(struct domain *domain, uint64_t now, int *ptimeout);
+void wrl_log_periodic(uint64_t now);
 void wrl_apply_debit_direct(struct connection *conn);
 void wrl_apply_debit_trans_commit(struct connection *conn);
 
Signature · 1 line
2.35.3
#16 lore raw

PATCH v2 7/13 tools/xenstore: remove stale TODO file

In reply to Juergen Gross

The TODO file is not really helpful any longer. It contains only entries which no longer apply or it is unknown what they are meant for ("Dynamic/supply nodes", "Remove assumption that rename doesn't fail").

---
 tools/xenstore/TODO | 10 ----------
 1 file changed, 10 deletions(-)
 delete mode 100644 tools/xenstore/TODO
tools/xenstore/TODO deleted +0 -10
deleted file mode 100644
index 71d5bbbf50..0000000000
@@ -1,10 +0,0 @@
-TODO in no particular order.  Some of these will never be done.  There
-are omissions of important but necessary things.  It is up to the
-reader to fill in the blanks.
-
-- Timeout failed watch responses
-- Dynamic/supply nodes
-- Persistant storage of introductions, watches and transactions, so daemon can restart
-- Remove assumption that rename doesn't fail
-- Multi-root transactions, for setting up front and back ends at same time.
-
Signature · 1 line
2.35.3
Acked-by on v2 7/13
#17 lore raw

Re: [PATCH v2 07/13] tools/xenstore: remove stale TODO file

Hi Juergen,

Quoted 6 lines · Juergen Gross

The TODO file is not really helpful any longer. It contains only entries which no longer apply or it is unknown what they are meant for ("Dynamic/supply nodes", "Remove assumption that rename doesn't fail").

Cheers,

Signature · 1 line
Julien Grall
#18 lore raw

PATCH v2 8/13 tools/xenstore: remove unused events list

In reply to Juergen Gross

struct watch contains an used struct list_head events. Remove it.

---
 tools/xenstore/xenstored_watch.c | 5 -----
 1 file changed, 5 deletions(-)
tools/xenstore/xenstored_watch.c +0 -5
index e8eb35de02..4195c59e17 100644
@@ -36,9 +36,6 @@ struct watch
 	/* Watches on this connection */
 	struct list_head list;
 
-	/* Current outstanding events applying to this watch. */
-	struct list_head events;
-
 	/* Offset into path for skipping prefix (used for relative paths). */
 	unsigned int prefix_len;
 
@@ -205,8 +202,6 @@ static struct watch *add_watch(struct connection *conn, char *path, char *token,
 
 	watch->prefix_len = relative ? strlen(get_implicit_path(conn)) + 1 : 0;
 
-	INIT_LIST_HEAD(&watch->events);
-
 	domain_watch_inc(conn);
 	list_add_tail(&watch->list, &conn->watches);
 	talloc_set_destructor(watch, destroy_watch);
Signature · 1 line
2.35.3
Acked-by on v2 8/13
#19 lore raw

Re: [PATCH v2 08/13] tools/xenstore: remove unused events list

Hi Juergen,

Quoted 1 line · Juergen Gross

struct watch contains an used struct list_head events. Remove it.

Typo: s/used/unused/?

Quoted 2 lines

Cheers,

Quoted 27 lines
---
  tools/xenstore/xenstored_watch.c | 5 -----
  1 file changed, 5 deletions(-)
tools/xenstore/xenstored_watch.c +0 -5
index e8eb35de02..4195c59e17 100644
@@ -36,9 +36,6 @@ struct watch
  	/* Watches on this connection */
  	struct list_head list;
 
-	/* Current outstanding events applying to this watch. */
-	struct list_head events;
-
  	/* Offset into path for skipping prefix (used for relative paths). */
  	unsigned int prefix_len;
 
@@ -205,8 +202,6 @@ static struct watch *add_watch(struct connection *conn, char *path, char *token,
 
  	watch->prefix_len = relative ? strlen(get_implicit_path(conn)) + 1 : 0;
 
-	INIT_LIST_HEAD(&watch->events);
-
  	domain_watch_inc(conn);
  	list_add_tail(&watch->list, &conn->watches);
  	talloc_set_destructor(watch, destroy_watch);
Signature · 1 line
Julien Grall
#21 lore raw

PATCH v2 9/13 tools/xenstore: remove support of file backed data base

In reply to Juergen Gross

In order to prepare the replacement of TDB with direct accessible nodes in memory, remove the support for a file backed data base.

This allows to remove xs_tdb_dump, too.

---
 tools/xenstore/Makefile         |  5 +-
 tools/xenstore/xenstored_core.c | 18 ++-----
 tools/xenstore/xs_tdb_dump.c    | 86 ---------------------------------
 3 files changed, 4 insertions(+), 105 deletions(-)
 delete mode 100644 tools/xenstore/xs_tdb_dump.c
tools/xenstore/Makefile +1 -4
index ce7a68178f..56723139a1 100644
@@ -29,7 +29,7 @@ CLIENTS += xenstore-write xenstore-ls xenstore-watch
 
 TARGETS := xenstore $(CLIENTS) xenstore-control
 ifeq ($(XENSTORE_XENSTORED),y)
-TARGETS += xs_tdb_dump xenstored
+TARGETS += xenstored
 endif
 
 .PHONY: all
@@ -50,9 +50,6 @@ xenstore: xenstore_client.o xs_lib.o
 xenstore-control: xenstore_control.o
 	$(CC) $(LDFLAGS) $^ $(LDLIBS) -o $@ $(APPEND_LDFLAGS)
 
-xs_tdb_dump: xs_tdb_dump.o utils.o tdb.o talloc.o
-	$(CC) $(LDFLAGS) $^ -o $@ $(APPEND_LDFLAGS)
-
 .PHONY: clean
 clean::
 	$(RM) $(TARGETS) $(DEPS_RM)
tools/xenstore/xenstored_core.c +3 -15
index 6be2a3aa42..e16d0a6d53 100644
@@ -2291,8 +2291,6 @@ static void accept_connection(int sock)
 }
 #endif
 
-static int tdb_flags = TDB_INTERNAL | TDB_NOLOCK;
-
 /* We create initial nodes manually. */
 static void manual_node(const char *name, const char *child)
 {
@@ -2344,14 +2342,11 @@ void setup_structure(bool live_update)
 {
 	char *tdbname;
 
-	tdbname = talloc_strdup(talloc_autofree_context(), xs_daemon_tdb());
+	tdbname = talloc_strdup(talloc_autofree_context(), "/dev/mem");
 	if (!tdbname)
 		barf_perror("Could not create tdbname");
 
-	if (!(tdb_flags & TDB_INTERNAL))
-		unlink(tdbname);
-
-	tdb_ctx = tdb_open_ex(tdbname, 7919, tdb_flags,
+	tdb_ctx = tdb_open_ex(tdbname, 7919, TDB_INTERNAL | TDB_NOLOCK,
 			      O_RDWR | O_CREAT | O_EXCL | O_CLOEXEC,
 			      0640, &tdb_logger, NULL);
 	if (!tdb_ctx)
@@ -2649,8 +2644,6 @@ static void usage(void)
 "                          watch-event: time a watch-event is kept pending\n"
 "  -R, --no-recovery       to request that no recovery should be attempted when\n"
 "                          the store is corrupted (debug only),\n"
-"  -I, --internal-db [on|off] store database in memory, not on disk, default is\n"
-"                          memory, with \"--internal-db off\" it is on disk\n"
 "  -K, --keep-orphans      don't delete nodes owned by a domain when the\n"
 "                          domain is deleted (this is a security risk!)\n"
 "  -V, --verbose           to request verbose execution.\n");
@@ -2677,7 +2670,6 @@ static struct option options[] = {
 	{ "quota-soft", 1, NULL, 'q' },
 	{ "timeout", 1, NULL, 'w' },
 	{ "no-recovery", 0, NULL, 'R' },
-	{ "internal-db", 2, NULL, 'I' },
 	{ "keep-orphans", 0, NULL, 'K' },
 	{ "verbose", 0, NULL, 'V' },
 	{ "watch-nb", 1, NULL, 'W' },
@@ -2800,7 +2792,7 @@ int main(int argc, char *argv[])
 	orig_argv = argv;
 
 	while ((opt = getopt_long(argc, argv,
-				  "DE:F:HI::KNPS:t:A:M:Q:q:T:RVW:w:U",
+				  "DE:F:H::KNPS:t:A:M:Q:q:T:RVW:w:U",
 				  options, NULL)) != -1) {
 		switch (opt) {
 		case 'D':
@@ -2837,10 +2829,6 @@ int main(int argc, char *argv[])
 			if (set_trace_switch(optarg))
 				barf("Illegal trace switch \"%s\"\n", optarg);
 			break;
-		case 'I':
-			if (optarg && !strcmp(optarg, "off"))
-				tdb_flags = 0;
-			break;
 		case 'K':
 			keep_orphans = true;
 			break;
tools/xenstore/xs_tdb_dump.c deleted +0 -86

86 lines not on this page · raw message

Signature · 1 line
2.35.3
#22 lore raw

PATCH v2 10/13 tools/xenstore: remove no longer needed functions from xs_lib.c

In reply to Juergen Gross

xs_daemon_tdb() in xs_lib.c is no longer used at all, so it can be removed. xs_domain_dev() and xs_write_all() are not used by xenstored, so they can be moved to tools/libs/store/xs.c.

Move functions used by xenstore-client only to xenstore_client.c.

---
V2:
- move xs_write_all(), too

---
 tools/include/xenstore.h         |   3 +
 tools/include/xenstore_lib.h     |   3 -
 tools/libs/store/xs.c            |  38 ++++++++
 tools/xenstore/xenstore_client.c | 129 +++++++++++++++++++++++++
 tools/xenstore/xs_lib.c          | 156 -------------------------------
 tools/xenstore/xs_lib.h          |  17 ----
 6 files changed, 170 insertions(+), 176 deletions(-)
tools/include/xenstore.h +3 -0
index 2b3f69fb61..a442252849 100644
@@ -113,6 +113,9 @@ void *xs_read(struct xs_handle *h, xs_transaction_t t,
 bool xs_write(struct xs_handle *h, xs_transaction_t t,
 	      const char *path, const void *data, unsigned int len);
 
+/* Simple write function: loops for you. */
+bool xs_write_all(int fd, const void *data, unsigned int len);
+
 /* Create a new directory.
  * Returns false on failure, or success if it already exists.
  */
tools/include/xenstore_lib.h +0 -3
index 2266009ec1..43eec87379 100644
@@ -47,9 +47,6 @@ const char *xs_daemon_rundir(void);
 const char *xs_daemon_socket(void);
 const char *xs_daemon_socket_ro(void);
 
-/* Simple write function: loops for you. */
-bool xs_write_all(int fd, const void *data, unsigned int len);
-
 /* Convert strings to permissions.  False if a problem. */
 bool xs_strings_to_perms(struct xs_permissions *perms, unsigned int num,
 			 const char *strings);
tools/libs/store/xs.c +38 -0
index 7a9a8b1656..48b0e917be 100644
@@ -311,6 +311,26 @@ struct xs_handle *xs_domain_open(void)
 	return xs_open(0);
 }
 
+static const char *xs_domain_dev(void)
+{
+	char *s = getenv("XENSTORED_PATH");
+	if (s)
+		return s;
+#if defined(__RUMPUSER_XEN__) || defined(__RUMPRUN__)
+	return "/dev/xen/xenbus";
+#elif defined(__linux__)
+	if (access("/dev/xen/xenbus", F_OK) == 0)
+		return "/dev/xen/xenbus";
+	return "/proc/xen/xenbus";
+#elif defined(__NetBSD__)
+	return "/kern/xen/xenbus";
+#elif defined(__FreeBSD__)
+	return "/dev/xen/xenstore";
+#else
+	return "/dev/xen/xenbus";
+#endif
+}
+
 struct xs_handle *xs_open(unsigned long flags)
 {
 	struct xs_handle *xsh = NULL;
@@ -431,6 +451,24 @@ out_false:
 #ifdef XSTEST
 #define read_all read_all_choice
 #define xs_write_all write_all_choice
+#else
+/* Simple routine for writing to sockets, etc. */
+bool xs_write_all(int fd, const void *data, unsigned int len)
+{
+	while (len) {
+		int done;
+
+		done = write(fd, data, len);
+		if (done < 0 && errno == EINTR)
+			continue;
+		if (done <= 0)
+			return false;
+		data += done;
+		len -= done;
+	}
+
+	return true;
+}
 #endif
 
 static int get_error(const char *errorstring)
tools/xenstore/xenstore_client.c +129 -0

147 lines not on this page · raw message

tools/xenstore/xs_lib.c +0 -156

171 lines not on this page · raw message

tools/xenstore/xs_lib.h +0 -17
index efa05997d6..61229aa435 100644
@@ -22,8 +22,6 @@
 #include "xenstore_lib.h"
 
 const char *xs_daemon_rootdir(void);
-const char *xs_domain_dev(void);
-const char *xs_daemon_tdb(void);
 
 /* Convert permissions to a string (up to len MAX_STRLEN(unsigned int)+1). */
 bool xs_perm_to_string(const struct xs_permissions *perm,
@@ -32,19 +30,4 @@ bool xs_perm_to_string(const struct xs_permissions *perm,
 /* Given a string and a length, count how many strings (nul terms). */
 unsigned int xs_count_strings(const char *strings, unsigned int len);
 
-/* Sanitising (quoting) possibly-binary strings. */
-struct expanding_buffer {
-	char *buf;
-	int avail;
-};
-
-/* Ensure that given expanding buffer has at least min_avail characters. */
-char *expanding_buffer_ensure(struct expanding_buffer *, int min_avail);
-
-/* sanitise_value() may return NULL if malloc fails. */
-char *sanitise_value(struct expanding_buffer *, const char *val, unsigned len);
-
-/* *out_len_r on entry is ignored; out must be at least strlen(in)+1 bytes. */
-void unsanitise_value(char *out, unsigned *out_len_r, const char *in);
-
 #endif /* XS_LIB_H */
Signature · 1 line
2.35.3
#23 lore raw

PATCH v2 11/13 tools/xenstore: split out environment specific live update code

In reply to Juergen Gross

Instead of using #ifdef in xenstored_control.c split out the code of environment specific functions (daemon or Mini-OS) to dedicated source files.

---
 tools/xenstore/Makefile.common       |   8 +-
 tools/xenstore/xenstored_control.c   | 252 +--------------------------
 tools/xenstore/xenstored_lu.h        |  56 ++++++
 tools/xenstore/xenstored_lu_daemon.c | 132 ++++++++++++++
 tools/xenstore/xenstored_lu_minios.c | 121 +++++++++++++
 5 files changed, 316 insertions(+), 253 deletions(-)
 create mode 100644 tools/xenstore/xenstored_lu.h
 create mode 100644 tools/xenstore/xenstored_lu_daemon.c
 create mode 100644 tools/xenstore/xenstored_lu_minios.c
tools/xenstore/Makefile.common +4 -4
index b18f95c103..dffab34603 100644
@@ -4,10 +4,10 @@ XENSTORED_OBJS-y := xenstored_core.o xenstored_watch.o xenstored_domain.o
 XENSTORED_OBJS-y += xenstored_transaction.o xenstored_control.o
 XENSTORED_OBJS-y += xs_lib.o talloc.o utils.o tdb.o hashtable.o
 
-XENSTORED_OBJS-$(CONFIG_Linux) += xenstored_posix.o
-XENSTORED_OBJS-$(CONFIG_NetBSD) += xenstored_posix.o
-XENSTORED_OBJS-$(CONFIG_FreeBSD) += xenstored_posix.o
-XENSTORED_OBJS-$(CONFIG_MiniOS) += xenstored_minios.o
+XENSTORED_OBJS-$(CONFIG_Linux) += xenstored_posix.o xenstored_lu_daemon.o
+XENSTORED_OBJS-$(CONFIG_NetBSD) += xenstored_posix.o xenstored_lu_daemon.o
+XENSTORED_OBJS-$(CONFIG_FreeBSD) += xenstored_posix.o xenstored_lu_daemon.o
+XENSTORED_OBJS-$(CONFIG_MiniOS) += xenstored_minios.o xenstored_lu_minios.o
 
 # Include configure output (config.h)
 CFLAGS += -include $(XEN_ROOT)/tools/config.h
tools/xenstore/xenstored_control.c +3 -249

268 lines not on this page · raw message

tools/xenstore/xenstored_lu.h new file +56 -0

56 lines not on this page · raw message

tools/xenstore/xenstored_lu_daemon.c new file +132 -0

132 lines not on this page · raw message

tools/xenstore/xenstored_lu_minios.c new file +121 -0

121 lines not on this page · raw message

Signature · 1 line
2.35.3
#24 lore raw

PATCH v2 12/13 tools/xenstore: split out rest of live update control code

In reply to Juergen Gross

Move the rest of live update related code from xenstored_control.c to a dedicated new source file.

---
 tools/xenstore/Makefile.common     |   2 +-
 tools/xenstore/xenstored_control.c | 409 -----------------------------
 tools/xenstore/xenstored_control.h |   8 -
 tools/xenstore/xenstored_core.c    |   1 +
 tools/xenstore/xenstored_lu.c      | 401 ++++++++++++++++++++++++++++
 tools/xenstore/xenstored_lu.h      |  25 ++
 6 files changed, 428 insertions(+), 418 deletions(-)
 create mode 100644 tools/xenstore/xenstored_lu.c
tools/xenstore/Makefile.common +1 -1
index dffab34603..2befede03d 100644
@@ -1,7 +1,7 @@
 # Makefile shared with stubdom
 
 XENSTORED_OBJS-y := xenstored_core.o xenstored_watch.o xenstored_domain.o
-XENSTORED_OBJS-y += xenstored_transaction.o xenstored_control.o
+XENSTORED_OBJS-y += xenstored_transaction.o xenstored_control.o xenstored_lu.o
 XENSTORED_OBJS-y += xs_lib.o talloc.o utils.o tdb.o hashtable.o
 
 XENSTORED_OBJS-$(CONFIG_Linux) += xenstored_posix.o xenstored_lu_daemon.o
tools/xenstore/xenstored_control.c +0 -409

434 lines not on this page · raw message

tools/xenstore/xenstored_control.h +0 -8
index a8cb76559b..faa955968d 100644
@@ -18,11 +18,3 @@
 
 int do_control(const void *ctx, struct connection *conn,
 	       struct buffered_data *in);
-void lu_read_state(void);
-
-struct connection *lu_get_connection(void);
-
-/* Write the "OK" response for the live-update command */
-unsigned int lu_write_response(FILE *fp);
-
-bool lu_is_pending(void);
tools/xenstore/xenstored_core.c +1 -0
index e16d0a6d53..3842e122ce 100644
@@ -52,6 +52,7 @@
 #include "xenstored_transaction.h"
 #include "xenstored_domain.h"
 #include "xenstored_control.h"
+#include "xenstored_lu.h"
 #include "tdb.h"
 
 #ifndef NO_SOCKETS
tools/xenstore/xenstored_lu.c new file +401 -0

401 lines not on this page · raw message

tools/xenstore/xenstored_lu.h +25 -0
index d2f8e4e57c..7aa524a07e 100644
@@ -43,6 +43,16 @@ struct lu_dump_state {
 
 extern struct live_update *lu_status;
 
+struct connection *lu_get_connection(void);
+bool lu_is_pending(void);
+void lu_read_state(void);
+
+/* Write the "OK" response for the live-update command */
+unsigned int lu_write_response(FILE *fp);
+
+int do_control_lu(const void *ctx, struct connection *conn, char **vec,
+		  int num);
+
 /* Live update private interfaces. */
 void lu_get_dump_state(struct lu_dump_state *state);
 void lu_close_dump_state(struct lu_dump_state *state);
@@ -53,4 +63,19 @@ const char *lu_arch(const void *ctx, struct connection *conn, char **vec,
 		    int num);
 const char *lu_begin(struct connection *conn);
 int lu_destroy(void *data);
+#else
+static inline struct connection *lu_get_connection(void)
+{
+	return NULL;
+}
+
+static inline unsigned int lu_write_response(FILE *fp)
+{
+	return 0;
+}
+
+static inline bool lu_is_pending(void)
+{
+	return false;
+}
 #endif
Signature · 1 line
2.35.3
#25 lore raw

PATCH v2 13/13 tools/xenstore: remove unused stuff from list.h

In reply to Juergen Gross

Remove the hlist devines/functions and the rcu related functions from tools/xenstore/list.h, as they are not used.

---
 tools/xenstore/list.h | 227 ------------------------------------------
 1 file changed, 227 deletions(-)
tools/xenstore/list.h +0 -227
index a464a38b61..d722a91220 100644
@@ -88,48 +88,6 @@ static inline void list_add_tail(struct list_head *new, struct list_head *head)
 	__list_add(new, head->prev, head);
 }
 
-/*
- * Insert a new entry between two known consecutive entries. 
- *
- * This is only for internal list manipulation where we know
- * the prev/next entries already!
- */
-static __inline__ void __list_add_rcu(struct list_head * new,
-	struct list_head * prev,
-	struct list_head * next)
-{
-	new->next = next;
-	new->prev = prev;
-	next->prev = new;
-	prev->next = new;
-}
-
-/**
- * list_add_rcu - add a new entry to rcu-protected list
- * @new: new entry to be added
- * @head: list head to add it after
- *
- * Insert a new entry after the specified head.
- * This is good for implementing stacks.
- */
-static __inline__ void list_add_rcu(struct list_head *new, struct list_head *head)
-{
-	__list_add_rcu(new, head, head->next);
-}
-
-/**
- * list_add_tail_rcu - add a new entry to rcu-protected list
- * @new: new entry to be added
- * @head: list head to add it before
- *
- * Insert a new entry before the specified head.
- * This is useful for implementing queues.
- */
-static __inline__ void list_add_tail_rcu(struct list_head *new, struct list_head *head)
-{
-	__list_add_rcu(new, head->prev, head);
-}
-
 /*
  * Delete a list entry by making the prev/next entries
  * point to each other.
@@ -156,23 +114,6 @@ static inline void list_del(struct list_head *entry)
 	entry->prev = LIST_POISON2;
 }
 
-/**
- * list_del_rcu - deletes entry from list without re-initialization
- * @entry: the element to delete from the list.
- *
- * Note: list_empty on entry does not return true after this, 
- * the entry is in an undefined state. It is useful for RCU based
- * lockfree traversal.
- *
- * In particular, it means that we can not poison the forward 
- * pointers that may still be used for walking the list.
- */
-static inline void list_del_rcu(struct list_head *entry)
-{
-	__list_del(entry->prev, entry->next);
-	entry->prev = LIST_POISON2;
-}
-
 /**
  * list_del_init - deletes entry from list and reinitialize it.
  * @entry: the element to delete from the list.
@@ -339,172 +280,4 @@ static inline void list_splice_init(struct list_head *list,
 	     &pos->member != (head); 					\
 	     pos = n, n = list_entry(n->member.next, typeof(*n), member))
 
-
-/* 
- * Double linked lists with a single pointer list head. 
- * Mostly useful for hash tables where the two pointer list head is 
- * too wasteful.
- * You lose the ability to access the tail in O(1).
- */ 
-
-struct hlist_head { 
-	struct hlist_node *first; 
-}; 
-
-struct hlist_node { 
-	struct hlist_node *next, **pprev; 
-}; 
-
-#define HLIST_HEAD_INIT { .first = NULL } 
-#define HLIST_HEAD(name) struct hlist_head name = {  .first = NULL }
-#define INIT_HLIST_HEAD(ptr) ((ptr)->first = NULL) 
-#define INIT_HLIST_NODE(ptr) ((ptr)->next = NULL, (ptr)->pprev = NULL)
-
-static __inline__ int hlist_unhashed(struct hlist_node *h) 
-{ 
-	return !h->pprev;
-} 
-
-static __inline__ int hlist_empty(struct hlist_head *h) 
-{ 
-	return !h->first;
-} 
-
-static __inline__ void __hlist_del(struct hlist_node *n) 
-{
-	struct hlist_node *next = n->next;
-	struct hlist_node **pprev = n->pprev;
-	*pprev = next;  
-	if (next) 
-		next->pprev = pprev;
-}  
-
-static __inline__ void hlist_del(struct hlist_node *n)
-{
-	__hlist_del(n);
-	n->next = LIST_POISON1;
-	n->pprev = LIST_POISON2;
-}
-
-/**
- * hlist_del_rcu - deletes entry from hash list without re-initialization
- * @entry: the element to delete from the hash list.
- *
- * Note: list_unhashed() on entry does not return true after this, 
- * the entry is in an undefined state. It is useful for RCU based
- * lockfree traversal.
- *
- * In particular, it means that we can not poison the forward
- * pointers that may still be used for walking the hash list.
- */
-static inline void hlist_del_rcu(struct hlist_node *n)
-{
-	__hlist_del(n);
-	n->pprev = LIST_POISON2;
-}
-
-static __inline__ void hlist_del_init(struct hlist_node *n) 
-{
-	if (n->pprev)  {
-		__hlist_del(n);
-		INIT_HLIST_NODE(n);
-	}
-}  
-
-#define hlist_del_rcu_init hlist_del_init
-
-static __inline__ void hlist_add_head(struct hlist_node *n, struct hlist_head *h) 
-{ 
-	struct hlist_node *first = h->first;
-	n->next = first; 
-	if (first) 
-		first->pprev = &n->next;
-	h->first = n; 
-	n->pprev = &h->first; 
-} 
-
-static __inline__ void hlist_add_head_rcu(struct hlist_node *n, struct hlist_head *h) 
-{ 
-	struct hlist_node *first = h->first;
-	n->next = first;
-	n->pprev = &h->first; 
-	if (first) 
-		first->pprev = &n->next;
-	h->first = n; 
-} 
-
-/* next must be != NULL */
-static __inline__ void hlist_add_before(struct hlist_node *n, struct hlist_node *next)
-{
-	n->pprev = next->pprev;
-	n->next = next; 
-	next->pprev = &n->next; 
-	*(n->pprev) = n;
-}
-
-static __inline__ void hlist_add_after(struct hlist_node *n,
-				       struct hlist_node *next)
-{
-	next->next	= n->next;
-	*(next->pprev)	= n;
-	n->next		= next;
-}
-
-#define hlist_entry(ptr, type, member) container_of(ptr,type,member)
-
-/* Cannot easily do prefetch unfortunately */
-#define hlist_for_each(pos, head) \
-	for (pos = (head)->first; pos; pos = pos->next) 
-
-#define hlist_for_each_safe(pos, n, head) \
-	for (pos = (head)->first; n = pos ? pos->next : 0, pos; \
-	     pos = n)
-
-/**
- * hlist_for_each_entry	- iterate over list of given type
- * @tpos:	the type * to use as a loop counter.
- * @pos:	the &struct hlist_node to use as a loop counter.
- * @head:	the head for your list.
- * @member:	the name of the hlist_node within the struct.
- */
-#define hlist_for_each_entry(tpos, pos, head, member)			 \
-	for (pos = (head)->first;					 \
-	     pos && ({ tpos = hlist_entry(pos, typeof(*tpos), member); 1;}); \
-	     pos = pos->next)
-
-/**
- * hlist_for_each_entry_continue - iterate over a hlist continuing after existing point
- * @tpos:	the type * to use as a loop counter.
- * @pos:	the &struct hlist_node to use as a loop counter.
- * @member:	the name of the hlist_node within the struct.
- */
-#define hlist_for_each_entry_continue(tpos, pos, member)		 \
-	for (pos = (pos)->next;						 \
-	     pos && ({ tpos = hlist_entry(pos, typeof(*tpos), member); 1;}); \
-	     pos = pos->next)
-
-/**
- * hlist_for_each_entry_from - iterate over a hlist continuing from existing point
- * @tpos:	the type * to use as a loop counter.
- * @pos:	the &struct hlist_node to use as a loop counter.
- * @member:	the name of the hlist_node within the struct.
- */
-#define hlist_for_each_entry_from(tpos, pos, member)			 \
-	for (; pos && ({ tpos = hlist_entry(pos, typeof(*tpos), member); 1;}); \
-	     pos = pos->next)
-
-/**
- * hlist_for_each_entry_safe - iterate over list of given type safe against removal of list entry
- * @tpos:	the type * to use as a loop counter.
- * @pos:	the &struct hlist_node to use as a loop counter.
- * @n:		another &struct hlist_node to use as temporary storage
- * @head:	the head for your list.
- * @member:	the name of the hlist_node within the struct.
- */
-#define hlist_for_each_entry_safe(tpos, pos, n, head, member) 		 \
-	for (pos = (head)->first;					 \
-	     pos && ({ n = pos->next; 1; }) && 				 \
-		({ tpos = hlist_entry(pos, typeof(*tpos), member); 1;}); \
-	     pos = n)
-
 #endif
Signature · 1 line
2.35.3