Skip to contentSkip to conversation

xen/evtchn: rework per event channel lock

PATCH In review Juergen Gross 2 participants 4 messages Nov 4, 2020 – Nov 5, 2020

lore
Share & tools
lore
#0 lore raw

PATCH 2/2 xen/evtchn: rework per event channel lock

Currently the lock for a single event channel needs to be taken with interrupts off, which causes deadlocks in some cases.

Rework the per event channel lock to be non-blocking for the case of sending an event and removing the need for disabling interrupts for taking the lock.

The lock is needed for avoiding races between event channel state changes (creation, closing, binding) against normal operations (set pending, [un]masking, priority changes).

Use a rwlock, but with some restrictions:

- normal operations use read_trylock(), in case of not obtaining the lock the operation is omitted or a default state is returned

- closing an event channel is using write_lock(), with ASSERT()ing that the lock is taken as writer only when the state of the event channel is either before or after the locked region appropriate (either free or unbound).

Fixes e045199c7c9c54 ("evtchn: address races with evtchn_reset()")

---
V4.1:
- corrected comment about lock discipline

V4:
- switch to rwlock
- add ASSERT() to verify correct write_lock() usage

V3:
- corrected a copy-and-paste error (Jan Beulich)
- corrected unlocking in two cases (Jan Beulich)
- renamed evtchn_read_trylock() (Jan Beulich)
- added some comments and an ASSERT() for evtchn_write_lock()
- set EVENT_WRITE_LOCK_INC to INT_MIN

V2:
- added needed barriers

---
 xen/arch/x86/irq.c         |   6 +-
 xen/arch/x86/pv/shim.c     |   9 +--
 xen/common/event_channel.c | 120 ++++++++++++++++++-------------------
 xen/include/xen/event.h    |  55 +++++++++++++----
 xen/include/xen/sched.h    |   3 +-
 5 files changed, 111 insertions(+), 82 deletions(-)
xen/arch/x86/irq.c +2 -4
index 93c4fb9a79..8d1f9a9fc6 100644
@@ -2495,14 +2495,12 @@ static void dump_irqs(unsigned char key)
                 pirq = domain_irq_to_pirq(d, irq);
                 info = pirq_info(d, pirq);
                 evtchn = evtchn_from_port(d, info->evtchn);
-                local_irq_disable();
-                if ( spin_trylock(&evtchn->lock) )
+                if ( evtchn_read_trylock(evtchn) )
                 {
                     pending = evtchn_is_pending(d, evtchn);
                     masked = evtchn_is_masked(d, evtchn);
-                    spin_unlock(&evtchn->lock);
+                    evtchn_read_unlock(evtchn);
                 }
-                local_irq_enable();
                 printk("d%d:%3d(%c%c%c)%c",
                        d->domain_id, pirq, "-P?"[pending],
                        "-M?"[masked], info->masked ? 'M' : '-',
xen/arch/x86/pv/shim.c +5 -4
index 9aef7a860a..b4e83e0778 100644
@@ -660,11 +660,12 @@ void pv_shim_inject_evtchn(unsigned int port)
     if ( port_is_valid(guest, port) )
     {
         struct evtchn *chn = evtchn_from_port(guest, port);
-        unsigned long flags;
 
-        spin_lock_irqsave(&chn->lock, flags);
-        evtchn_port_set_pending(guest, chn->notify_vcpu_id, chn);
-        spin_unlock_irqrestore(&chn->lock, flags);
+        if ( evtchn_read_trylock(chn) )
+        {
+            evtchn_port_set_pending(guest, chn->notify_vcpu_id, chn);
+            evtchn_read_unlock(chn);
+        }
     }
 }
 
xen/common/event_channel.c +57 -63

358 lines not on this page · raw message

xen/include/xen/event.h +45 -10

75 lines not on this page · raw message

xen/include/xen/sched.h +2 -1
index a298ff4df8..97c65d2917 100644
@@ -85,7 +85,7 @@ extern domid_t hardware_domid;
 
 struct evtchn
 {
-    spinlock_t lock;
+    rwlock_t lock;
 #define ECS_FREE         0 /* Channel is available for use.                  */
 #define ECS_RESERVED     1 /* Channel is reserved.                           */
 #define ECS_UNBOUND      2 /* Channel is waiting to bind to a remote domain. */
@@ -114,6 +114,7 @@ struct evtchn
         u16 virq;      /* state == ECS_VIRQ */
     } u;
     u8 priority;
+    u8 old_state;      /* State when taking lock in write mode. */
     u32 fifo_lastq;    /* Data for fifo events identifying last queue. */
 #ifdef CONFIG_XSM
     union {
Signature · 1 line
2.26.2
#1 lore raw

xen/common/event_channel.c int evtchn_send(struct domain *ld, unsigned int lport) · line 729

Quoted 7 lines
xen/common/event_channel.c +2 -1
@@ -738,7 +725,8 @@ int evtchn_send(struct domain *ld, unsigned int lport)
 
     lchn = evtchn_from_port(ld, lport);
 
-    spin_lock_irqsave(&lchn->lock, flags);
+    if ( !evtchn_read_trylock(lchn) )
+        return 0;

Isn't there a change in behavior here? While sends through ECS_UNBOUND ports indeed get silently ignored, ECS_FREE ones ought to be getting -EINVAL (as should ECS_UNBOUND ones if they're Xen-consumer ones). With the failed trylock you don't know which of the two the port is in the process of being transitioned to/from. The same would apply for other operations distinguishing the two states. Right now both evtchn_status() and evtchn_bind_vcpu() only use the domain-wide lock, but the latter is getting switched by "evtchn: convert domain event lock to an r/w one" (granted there's an RFC remark there whether that transformation is worthwhile). Anyway, the main point of my remark is that there's another subtlety here which I don't think becomes obvious from description or comments - where the two states are mentioned, it gets to look as if they can be treated equally.

Quoted 18 lines
xen/include/xen/event.h +12 -0
@@ -105,6 +105,39 @@ void notify_via_xen_event_channel(struct domain *ld, int lport);
 #define bucket_from_port(d, p) \
     ((group_from_port(d, p))[((p) % EVTCHNS_PER_GROUP) / EVTCHNS_PER_BUCKET])
 
+/*
+ * Lock an event channel exclusively. This is allowed only when the channel is
+ * free or unbound either when taking or when releasing the lock, as any
+ * concurrent operation on the event channel using evtchn_read_trylock() will
+ * just assume the event channel is free or unbound at the moment when the
+ * evtchn_read_trylock() returns false.
+ */
+static inline void evtchn_write_lock(struct evtchn *evtchn)
+{
+    write_lock(&evtchn->lock);
+
+    evtchn->old_state = evtchn->state;

xen/include/xen/event.h void notify_via_xen_event_channel(struct domain *ld, int lport); · line 129

Quoted 10 lines
xen/include/xen/event.h +10 -0
+}
+
+static inline void evtchn_write_unlock(struct evtchn *evtchn)
+{
+    /* Enforce lock discipline. */
+    ASSERT(evtchn->old_state == ECS_FREE || evtchn->old_state == ECS_UNBOUND ||
+           evtchn->state == ECS_FREE || evtchn->state == ECS_UNBOUND);
+
+    write_unlock(&evtchn->lock);
+}

These two aren't needed outside of event_channel.c, are they? If so, and if they ought to go in a header rather than directly into the .c file (where I'd prefer the latter, for the sake of minimal exposure), then it should be event_channel.h.

xen/include/xen/event.h struct evtchn · line 120

Quoted 8 lines
xen/include/xen/event.h +1 -0
@@ -114,6 +114,7 @@ struct evtchn
         u16 virq;      /* state == ECS_VIRQ */
     } u;
     u8 priority;
+    u8 old_state;      /* State when taking lock in write mode. */
     u32 fifo_lastq;    /* Data for fifo events identifying last queue. */
 #ifdef CONFIG_XSM
     union {

While there's a gap here after the prior patch (which I'm still not overly happy about), I'm still inclined to ask that the field be put inside #ifndef NDEBUG, as its only consumer is an ASSERT().

Jan

#2 lore raw
Quoted 14 lines · Jan Beulich

xen/common/event_channel.c int evtchn_send(struct domain *ld, unsigned int lport) · line 729

Quoted 7 lines
xen/common/event_channel.c +2 -1
@@ -738,7 +725,8 @@ int evtchn_send(struct domain *ld, unsigned int lport)
 
      lchn = evtchn_from_port(ld, lport);
 
-    spin_lock_irqsave(&lchn->lock, flags);
+    if ( !evtchn_read_trylock(lchn) )
+        return 0;

Isn't there a change in behavior here? While sends through ECS_UNBOUND ports indeed get silently ignored, ECS_FREE ones ought to be getting -EINVAL (as should ECS_UNBOUND ones if they're Xen-consumer ones). With the failed trylock you don't know which of the two the port is in the process of being transitioned to/from. The same would apply for other operations distinguishing

Quoted 8 lines

the two states. Right now both evtchn_status() and evtchn_bind_vcpu() only use the domain-wide lock, but the latter is getting switched by "evtchn: convert domain event lock to an r/w one" (granted there's an RFC remark there whether that transformation is worthwhile). Anyway, the main point of my remark is that there's another subtlety here which I don't think becomes obvious from description or comments - where the two states are mentioned, it gets to look as if they can be treated equally.

Hmm, evtchn_send() seems to be called always with interrupts enabled. We could just use a standard read_lock() here if you think the different states should be treated as today.

Quoted 26 lines
Quoted 26 lines
xen/include/xen/event.h +19 -0
@@ -105,6 +105,39 @@ void notify_via_xen_event_channel(struct domain *ld, int lport);
  #define bucket_from_port(d, p) \
      ((group_from_port(d, p))[((p) % EVTCHNS_PER_GROUP) / EVTCHNS_PER_BUCKET])
 
+/*
+ * Lock an event channel exclusively. This is allowed only when the channel is
+ * free or unbound either when taking or when releasing the lock, as any
+ * concurrent operation on the event channel using evtchn_read_trylock() will
+ * just assume the event channel is free or unbound at the moment when the
+ * evtchn_read_trylock() returns false.
+ */
+static inline void evtchn_write_lock(struct evtchn *evtchn)
+{
+    write_lock(&evtchn->lock);
+
+    evtchn->old_state = evtchn->state;
+}
+
+static inline void evtchn_write_unlock(struct evtchn *evtchn)
+{
+    /* Enforce lock discipline. */
+    ASSERT(evtchn->old_state == ECS_FREE || evtchn->old_state == ECS_UNBOUND ||
+           evtchn->state == ECS_FREE || evtchn->state == ECS_UNBOUND);
Quoted 8 lines

xen/include/xen/event.h void notify_via_xen_event_channel(struct domain *ld, int lport); · line 129

Quoted 3 lines
xen/include/xen/event.h +3 -0
+
+    write_unlock(&evtchn->lock);
+}

These two aren't needed outside of event_channel.c, are they? If so, and if they ought to go in a header rather than directly into the .c file (where I'd prefer the latter, for the sake of minimal exposure), then it should be event_channel.h.

I wanted to have the locking functions in one place.

In case you prefer it otherwise (and you seem to do so) I'd rather move the write lock functions to the .c file.

Quoted 14 lines

xen/include/xen/event.h struct evtchn · line 120

Quoted 9 lines
xen/include/xen/event.h +1 -0
@@ -114,6 +114,7 @@ struct evtchn
          u16 virq;      /* state == ECS_VIRQ */
      } u;
      u8 priority;
+    u8 old_state;      /* State when taking lock in write mode. */
      u32 fifo_lastq;    /* Data for fifo events identifying last queue. */
  #ifdef CONFIG_XSM
      union {

While there's a gap here after the prior patch (which I'm still not overly happy about), I'm still inclined to ask that the field be put inside #ifndef NDEBUG, as its only consumer is an ASSERT().

Fine with me

Juergen

#3 lore raw
Quoted 24 lines · Jürgen Groß
Quoted 14 lines · Jan Beulich

xen/common/event_channel.c int evtchn_send(struct domain *ld, unsigned int lport) · line 729

Quoted 7 lines
xen/common/event_channel.c +2 -1
@@ -738,7 +725,8 @@ int evtchn_send(struct domain *ld, unsigned int lport)
 
      lchn = evtchn_from_port(ld, lport);
 
-    spin_lock_irqsave(&lchn->lock, flags);
+    if ( !evtchn_read_trylock(lchn) )
+        return 0;

Isn't there a change in behavior here? While sends through ECS_UNBOUND ports indeed get silently ignored, ECS_FREE ones ought to be getting -EINVAL (as should ECS_UNBOUND ones if they're Xen-consumer ones). With the failed trylock you don't know which of the two the port is in the process of being transitioned to/from. The same would apply for other operations distinguishing

Quoted 8 lines

the two states. Right now both evtchn_status() and evtchn_bind_vcpu() only use the domain-wide lock, but the latter is getting switched by "evtchn: convert domain event lock to an r/w one" (granted there's an RFC remark there whether that transformation is worthwhile). Anyway, the main point of my remark is that there's another subtlety here which I don't think becomes obvious from description or comments - where the two states are mentioned, it gets to look as if they can be treated equally.

Quoted 3 lines

Hmm, evtchn_send() seems to be called always with interrupts enabled. We could just use a standard read_lock() here if you think the different states should be treated as today.

This would avoid the caveat in this specific case, but it would remain elsewhere (at least as an abstract trap to fall into). I suppose evtchn_status() could use a regular read_lock() too, for the same reason (if it was to be switched), and evtchn_bind_vcpu() may need a write lock anyway (which is forbidden in your model, i.e. I'd likely need to drop the switch to the finer grained lock there).

Quoted 34 lines
Quoted 25 lines
Quoted 25 lines
xen/include/xen/event.h +19 -0
@@ -105,6 +105,39 @@ void notify_via_xen_event_channel(struct domain *ld, int lport);
  #define bucket_from_port(d, p) \
      ((group_from_port(d, p))[((p) % EVTCHNS_PER_GROUP) / EVTCHNS_PER_BUCKET])
 
+/*
+ * Lock an event channel exclusively. This is allowed only when the channel is
+ * free or unbound either when taking or when releasing the lock, as any
+ * concurrent operation on the event channel using evtchn_read_trylock() will
+ * just assume the event channel is free or unbound at the moment when the
+ * evtchn_read_trylock() returns false.
+ */
+static inline void evtchn_write_lock(struct evtchn *evtchn)
+{
+    write_lock(&evtchn->lock);
+
+    evtchn->old_state = evtchn->state;
+}
+
+static inline void evtchn_write_unlock(struct evtchn *evtchn)
+{
+    /* Enforce lock discipline. */
+    ASSERT(evtchn->old_state == ECS_FREE || evtchn->old_state == ECS_UNBOUND ||
+           evtchn->state == ECS_FREE || evtchn->state == ECS_UNBOUND);
Quoted 8 lines

xen/include/xen/event.h void notify_via_xen_event_channel(struct domain *ld, int lport); · line 129

Quoted 3 lines
xen/include/xen/event.h +3 -0
+
+    write_unlock(&evtchn->lock);
+}

These two aren't needed outside of event_channel.c, are they? If so, and if they ought to go in a header rather than directly into the .c file (where I'd prefer the latter, for the sake of minimal exposure), then it should be event_channel.h.

Quoted 4 lines

I wanted to have the locking functions in one place.

In case you prefer it otherwise (and you seem to do so) I'd rather move the write lock functions to the .c file.

Yes please.

Jan