Skip to contentSkip to conversation

KVM: s390: Multiple-epoch facility fixes

RFC PATCH Applied David Hildenbrand 5 participants 34 messages Feb 7, 2018 – Mar 7, 2018

Subsystems (4)
lore
Share & tools
lore
#0 lore raw

cover KVM: s390: Multiple-epoch facility fixes

It think I found various BUGs in the recently added Multiple-epoch facility support in KVM. Along with it, two cleanups.

1. The clock-comparator sign control is not considered. I sent a patch with this previously. This now contains fixes and simplifications.
2. SET CLOCK from the guest does not work reliably if the facility is enabled (epoch index not set).
3. Hotplugged CPUs don't inherit the epoch index.
4. TOD clock syncs don't take care of overflows/underflows in the epoch value and miss to update the epoch index.

This is RFC as I have basically no machine to test. Hopefully somebody can jump in and verify that we now handle the epoch index in all scenarios correctly.

Most importantly, with Multiple-epoch facility, the condition
Guest TOD = Host TOD - 1
Is represented by both, epoch and epoch_idx containing 0xff. So it is treated as a 64+8bit signed number - we have to properly take care of over/underflows when modifying the epoch.

David Hildenbrand (6):
KVM: s390: take care of clock-comparator sign control
KVM: s390: provide only a single function for setting the tod
KVM: s390: consider epoch index on hotplugged CPUs
KVM: s390: consider epoch index on TOD clock syncs
KVM: s390: no need to inititalize kvm->arch members to 0
KVM: s390: generalize kvm_s390_get_tod_clock_ext()

 arch/s390/kvm/interrupt.c |  25 ++++++++---
 arch/s390/kvm/kvm-s390.c  | 109 +++++++++++++++++++++++-----------------------
 arch/s390/kvm/kvm-s390.h  |   5 +--
 arch/s390/kvm/priv.c      |   9 ++--
 4 files changed, 80 insertions(+), 68 deletions(-)
Signature · 1 line
2.14.3
#1 lore raw

RFC PATCH 1/6 KVM: s390: take care of clock-comparator sign control

Missed when enabling the Multiple-epoch facility. If the facility is installed and the control is set, a sign based comaprison has to be performed.

Right now we would inject wrong interrupts and ignore interrupt conditions. Also the sleep time is calculated in a wrong way.

---
 arch/s390/kvm/interrupt.c | 25 +++++++++++++++++++------
 1 file changed, 19 insertions(+), 6 deletions(-)
arch/s390/kvm/interrupt.c +19 -6
index 3ea9cfa31b16..a616e9b65f91 100644
@@ -169,8 +169,15 @@ static int ckc_interrupts_enabled(struct kvm_vcpu *vcpu)
 
 static int ckc_irq_pending(struct kvm_vcpu *vcpu)
 {
-	if (vcpu->arch.sie_block->ckc >= kvm_s390_get_tod_clock_fast(vcpu->kvm))
+	const u64 now = kvm_s390_get_tod_clock_fast(vcpu->kvm);
+	const u64 ckc = vcpu->arch.sie_block->ckc;
+
+	if (vcpu->arch.sie_block->gcr[0] & 0x0020000000000000ul) {
+		if ((s64)ckc >= (s64)now)
+			return 0;
+	} else if (ckc >= now) {
 		return 0;
+	}
 	return ckc_interrupts_enabled(vcpu);
 }
 
@@ -1042,13 +1049,19 @@ int kvm_cpu_has_pending_timer(struct kvm_vcpu *vcpu)
 
 static u64 __calculate_sltime(struct kvm_vcpu *vcpu)
 {
-	u64 now, cputm, sltime = 0;
+	const u64 now = kvm_s390_get_tod_clock_fast(vcpu->kvm);
+	const u64 ckc = vcpu->arch.sie_block->ckc;
+	u64 cputm, sltime = 0;
 
 	if (ckc_interrupts_enabled(vcpu)) {
-		now = kvm_s390_get_tod_clock_fast(vcpu->kvm);
-		sltime = tod_to_ns(vcpu->arch.sie_block->ckc - now);
-		/* already expired or overflow? */
-		if (!sltime || vcpu->arch.sie_block->ckc <= now)
+		if (vcpu->arch.sie_block->gcr[0] & 0x0020000000000000ul) {
+			if ((s64)now < (s64)ckc)
+				sltime = tod_to_ns((s64)ckc - (s64)now);
+		} else if (now < ckc) {
+			sltime = tod_to_ns(ckc - now);
+		}
+		/* already expired */
+		if (!sltime)
 			return 0;
 		if (cpu_timer_interrupts_enabled(vcpu)) {
 			cputm = kvm_s390_get_cpu_timer(vcpu);
Signature · 1 line
2.14.3
#2 lore raw

Re: [PATCH RFC 1/6] KVM: s390: take care of clock-comparator sign control

Quoted 48 lines · David Hildenbrand

Missed when enabling the Multiple-epoch facility. If the facility is installed and the control is set, a sign based comaprison has to be performed.

Right now we would inject wrong interrupts and ignore interrupt conditions. Also the sleep time is calculated in a wrong way.

---
  arch/s390/kvm/interrupt.c | 25 +++++++++++++++++++------
  1 file changed, 19 insertions(+), 6 deletions(-)
arch/s390/kvm/interrupt.c +12 -6
index 3ea9cfa31b16..a616e9b65f91 100644
@@ -169,8 +169,15 @@ static int ckc_interrupts_enabled(struct kvm_vcpu *vcpu)
 
  static int ckc_irq_pending(struct kvm_vcpu *vcpu)
  {
-	if (vcpu->arch.sie_block->ckc >= kvm_s390_get_tod_clock_fast(vcpu->kvm))
+	const u64 now = kvm_s390_get_tod_clock_fast(vcpu->kvm);
+	const u64 ckc = vcpu->arch.sie_block->ckc;
+
+	if (vcpu->arch.sie_block->gcr[0] & 0x0020000000000000ul) {
+		if ((s64)ckc >= (s64)now)
+			return 0;
+	} else if (ckc >= now) {
  		return 0;
+	}
  	return ckc_interrupts_enabled(vcpu);
  }
 
@@ -1042,13 +1049,19 @@ int kvm_cpu_has_pending_timer(struct kvm_vcpu *vcpu)
 
  static u64 __calculate_sltime(struct kvm_vcpu *vcpu)
  {
-	u64 now, cputm, sltime = 0;
+	const u64 now = kvm_s390_get_tod_clock_fast(vcpu->kvm);
+	const u64 ckc = vcpu->arch.sie_block->ckc;
+	u64 cputm, sltime = 0;
 
  	if (ckc_interrupts_enabled(vcpu)) {
-		now = kvm_s390_get_tod_clock_fast(vcpu->kvm);
-		sltime = tod_to_ns(vcpu->arch.sie_block->ckc - now);
-		/* already expired or overflow? */
-		if (!sltime || vcpu->arch.sie_block->ckc <= now)
+		if (vcpu->arch.sie_block->gcr[0] & 0x0020000000000000ul) {

arch/s390/kvm/interrupt.c int kvm_cpu_has_pending_timer(struct kvm_vcpu *vcpu) · line 1067

Quoted 10 lines
arch/s390/kvm/interrupt.c +7 -0
+			if ((s64)now < (s64)ckc)
+				sltime = tod_to_ns((s64)ckc - (s64)now);
+		} else if (now < ckc) {
+			sltime = tod_to_ns(ckc - now);
+		}
+		/* already expired */
+		if (!sltime)
  			return 0;
  		if (cpu_timer_interrupts_enabled(vcpu)) {
  			cputm = kvm_s390_get_cpu_timer(vcpu);

I think it would assist with readability if you defined the sign comparison bit. Seeing something that yells "SIGNED" would make sense as to what's going on here.

Other than that, I don't see anything wrong.

I'll get to reviewing the rest of these patches throughout the day. I have to revisit the docs :)

Signature · 1 line
- Collin L Walling
#3 lore raw

Re: [PATCH RFC 1/6] KVM: s390: take care of clock-comparator sign control

Quoted 54 lines · Collin L. Walling
Quoted 53 lines · David Hildenbrand

Missed when enabling the Multiple-epoch facility. If the facility is installed and the control is set, a sign based comaprison has to be performed.

Right now we would inject wrong interrupts and ignore interrupt conditions. Also the sleep time is calculated in a wrong way.

---
  arch/s390/kvm/interrupt.c | 25 +++++++++++++++++++------
  1 file changed, 19 insertions(+), 6 deletions(-)
arch/s390/kvm/interrupt.c +17 -6
index 3ea9cfa31b16..a616e9b65f91 100644
@@ -169,8 +169,15 @@ static int ckc_interrupts_enabled(struct kvm_vcpu *vcpu)
 
  static int ckc_irq_pending(struct kvm_vcpu *vcpu)
  {
-	if (vcpu->arch.sie_block->ckc >= kvm_s390_get_tod_clock_fast(vcpu->kvm))
+	const u64 now = kvm_s390_get_tod_clock_fast(vcpu->kvm);
+	const u64 ckc = vcpu->arch.sie_block->ckc;
+
+	if (vcpu->arch.sie_block->gcr[0] & 0x0020000000000000ul) {
+		if ((s64)ckc >= (s64)now)
+			return 0;
+	} else if (ckc >= now) {
  		return 0;
+	}
  	return ckc_interrupts_enabled(vcpu);
  }
 
@@ -1042,13 +1049,19 @@ int kvm_cpu_has_pending_timer(struct kvm_vcpu *vcpu)
 
  static u64 __calculate_sltime(struct kvm_vcpu *vcpu)
  {
-	u64 now, cputm, sltime = 0;
+	const u64 now = kvm_s390_get_tod_clock_fast(vcpu->kvm);
+	const u64 ckc = vcpu->arch.sie_block->ckc;
+	u64 cputm, sltime = 0;
 
  	if (ckc_interrupts_enabled(vcpu)) {
-		now = kvm_s390_get_tod_clock_fast(vcpu->kvm);
-		sltime = tod_to_ns(vcpu->arch.sie_block->ckc - now);
-		/* already expired or overflow? */
-		if (!sltime || vcpu->arch.sie_block->ckc <= now)
+		if (vcpu->arch.sie_block->gcr[0] & 0x0020000000000000ul) {
+			if ((s64)now < (s64)ckc)
+				sltime = tod_to_ns((s64)ckc - (s64)now);
+		} else if (now < ckc) {
+			sltime = tod_to_ns(ckc - now);
+		}
Quoted 8 lines

arch/s390/kvm/interrupt.c int kvm_cpu_has_pending_timer(struct kvm_vcpu *vcpu) · line 1067

Quoted 5 lines
arch/s390/kvm/interrupt.c +2 -0
+		/* already expired */
+		if (!sltime)
  			return 0;
  		if (cpu_timer_interrupts_enabled(vcpu)) {
  			cputm = kvm_s390_get_cpu_timer(vcpu);

I think it would assist with readability if you defined the sign comparison bit. Seeing
something that yells "SIGNED" would make sense as to what's going on here.

If we want that than I suggest introducing defines for all control registers we use in kvm code in a separate patch.

Quoted 3 lines

Other than that, I don't see anything wrong.

Thanks!

Quoted 4 lines

I'll get to reviewing the rest of these patches throughout the day. I have to revisit
the docs :)

Signature · 3 lines
Thanks,

David / dhildenb
#4 lore raw

Re: [PATCH RFC 1/6] KVM: s390: take care of clock-comparator sign control

Quoted 64 lines · David Hildenbrand
Quoted 54 lines · Collin L. Walling
Quoted 53 lines · David Hildenbrand

Missed when enabling the Multiple-epoch facility. If the facility is installed and the control is set, a sign based comaprison has to be performed.

Right now we would inject wrong interrupts and ignore interrupt conditions. Also the sleep time is calculated in a wrong way.

---
  arch/s390/kvm/interrupt.c | 25 +++++++++++++++++++------
  1 file changed, 19 insertions(+), 6 deletions(-)
arch/s390/kvm/interrupt.c +17 -6
index 3ea9cfa31b16..a616e9b65f91 100644
@@ -169,8 +169,15 @@ static int ckc_interrupts_enabled(struct kvm_vcpu *vcpu)
 
  static int ckc_irq_pending(struct kvm_vcpu *vcpu)
  {
-	if (vcpu->arch.sie_block->ckc >= kvm_s390_get_tod_clock_fast(vcpu->kvm))
+	const u64 now = kvm_s390_get_tod_clock_fast(vcpu->kvm);
+	const u64 ckc = vcpu->arch.sie_block->ckc;
+
+	if (vcpu->arch.sie_block->gcr[0] & 0x0020000000000000ul) {
+		if ((s64)ckc >= (s64)now)
+			return 0;
+	} else if (ckc >= now) {
  		return 0;
+	}
  	return ckc_interrupts_enabled(vcpu);
  }
 
@@ -1042,13 +1049,19 @@ int kvm_cpu_has_pending_timer(struct kvm_vcpu *vcpu)
 
  static u64 __calculate_sltime(struct kvm_vcpu *vcpu)
  {
-	u64 now, cputm, sltime = 0;
+	const u64 now = kvm_s390_get_tod_clock_fast(vcpu->kvm);
+	const u64 ckc = vcpu->arch.sie_block->ckc;
+	u64 cputm, sltime = 0;
 
  	if (ckc_interrupts_enabled(vcpu)) {
-		now = kvm_s390_get_tod_clock_fast(vcpu->kvm);
-		sltime = tod_to_ns(vcpu->arch.sie_block->ckc - now);
-		/* already expired or overflow? */
-		if (!sltime || vcpu->arch.sie_block->ckc <= now)
+		if (vcpu->arch.sie_block->gcr[0] & 0x0020000000000000ul) {
+			if ((s64)now < (s64)ckc)
+				sltime = tod_to_ns((s64)ckc - (s64)now);
+		} else if (now < ckc) {
+			sltime = tod_to_ns(ckc - now);
+		}
Quoted 8 lines

arch/s390/kvm/interrupt.c int kvm_cpu_has_pending_timer(struct kvm_vcpu *vcpu) · line 1067

Quoted 5 lines
arch/s390/kvm/interrupt.c +2 -0
+		/* already expired */
+		if (!sltime)
  			return 0;
  		if (cpu_timer_interrupts_enabled(vcpu)) {
  			cputm = kvm_s390_get_cpu_timer(vcpu);

I think it would assist with readability if you defined the sign comparison bit. Seeing
something that yells "SIGNED" would make sense as to what's going on here.

Quoted 2 lines

If we want that than I suggest introducing defines for all control registers we use in kvm code in a separate patch.

We could add those defines to
arch/s390/include/asm/ctl_reg.h

Reviewed-by on v1 1/6
#5 lore raw

Re: [PATCH RFC 1/6] KVM: s390: take care of clock-comparator sign control

In reply to David Hildenbrand
Quoted 8 lines · David Hildenbrand

Missed when enabling the Multiple-epoch facility. If the facility is installed and the control is set, a sign based comaprison has to be performed.

Right now we would inject wrong interrupts and ignore interrupt conditions. Also the sleep time is calculated in a wrong way.

applied. This will need some more testing but it probably deserves a CC stable.

Quoted 51 lines
---
 arch/s390/kvm/interrupt.c | 25 +++++++++++++++++++------
 1 file changed, 19 insertions(+), 6 deletions(-)
arch/s390/kvm/interrupt.c +19 -6
index 3ea9cfa31b16..a616e9b65f91 100644
@@ -169,8 +169,15 @@ static int ckc_interrupts_enabled(struct kvm_vcpu *vcpu)
 
 static int ckc_irq_pending(struct kvm_vcpu *vcpu)
 {
-	if (vcpu->arch.sie_block->ckc >= kvm_s390_get_tod_clock_fast(vcpu->kvm))
+	const u64 now = kvm_s390_get_tod_clock_fast(vcpu->kvm);
+	const u64 ckc = vcpu->arch.sie_block->ckc;
+
+	if (vcpu->arch.sie_block->gcr[0] & 0x0020000000000000ul) {
+		if ((s64)ckc >= (s64)now)
+			return 0;
+	} else if (ckc >= now) {
 		return 0;
+	}
 	return ckc_interrupts_enabled(vcpu);
 }
 
@@ -1042,13 +1049,19 @@ int kvm_cpu_has_pending_timer(struct kvm_vcpu *vcpu)
 
 static u64 __calculate_sltime(struct kvm_vcpu *vcpu)
 {
-	u64 now, cputm, sltime = 0;
+	const u64 now = kvm_s390_get_tod_clock_fast(vcpu->kvm);
+	const u64 ckc = vcpu->arch.sie_block->ckc;
+	u64 cputm, sltime = 0;
 
 	if (ckc_interrupts_enabled(vcpu)) {
-		now = kvm_s390_get_tod_clock_fast(vcpu->kvm);
-		sltime = tod_to_ns(vcpu->arch.sie_block->ckc - now);
-		/* already expired or overflow? */
-		if (!sltime || vcpu->arch.sie_block->ckc <= now)
+		if (vcpu->arch.sie_block->gcr[0] & 0x0020000000000000ul) {
+			if ((s64)now < (s64)ckc)
+				sltime = tod_to_ns((s64)ckc - (s64)now);
+		} else if (now < ckc) {
+			sltime = tod_to_ns(ckc - now);
+		}
+		/* already expired */
+		if (!sltime)
 			return 0;
 		if (cpu_timer_interrupts_enabled(vcpu)) {
 			cputm = kvm_s390_get_cpu_timer(vcpu);
maintainer Applied
#6 lore raw

Patch "KVM: s390: take care of clock-comparator sign control" has been added to the 4.14-stable tree

In reply to David Hildenbrand

This is a note to let you know that I've just added the patch titled

KVM: s390: take care of clock-comparator sign control

to the 4.14-stable tree which can be found at:
http://www.kernel.org/git/?p=linux/kernel/git/stable/stable-queue.git;a=summary

The filename of the patch is:
kvm-s390-take-care-of-clock-comparator-sign-control.patch
and it can be found in the queue-4.14 subdirectory.

If you, or anyone else, feels it should not be added to the stable tree, please let <[email protected]> know about it.

Quoted 1 line

From 5fe01793dd953ab947fababe8abaf5ed5258c8df Mon Sep 17 00:00:00 2001

From: David Hildenbrand <[email protected]>
Date: Wed, 7 Feb 2018 12:46:42 +0100
Subject: KVM: s390: take care of clock-comparator sign control

From: David Hildenbrand <[email protected]>

commit 5fe01793dd953ab947fababe8abaf5ed5258c8df upstream.

Missed when enabling the Multiple-epoch facility. If the facility is installed and the control is set, a sign based comaprison has to be performed.

Right now we would inject wrong interrupts and ignore interrupt conditions. Also the sleep time is calculated in a wrong way.

Message-Id: <[email protected]>

Fixes 8fa1696ea781 ("KVM: s390: Multiple Epoch Facility support")
---
 arch/s390/kvm/interrupt.c |   25 +++++++++++++++++++------
 1 file changed, 19 insertions(+), 6 deletions(-)
arch/s390/kvm/interrupt.c +19 -6
@@ -173,8 +173,15 @@ static int ckc_interrupts_enabled(struct
 
 static int ckc_irq_pending(struct kvm_vcpu *vcpu)
 {
-	if (vcpu->arch.sie_block->ckc >= kvm_s390_get_tod_clock_fast(vcpu->kvm))
+	const u64 now = kvm_s390_get_tod_clock_fast(vcpu->kvm);
+	const u64 ckc = vcpu->arch.sie_block->ckc;
+
+	if (vcpu->arch.sie_block->gcr[0] & 0x0020000000000000ul) {
+		if ((s64)ckc >= (s64)now)
+			return 0;
+	} else if (ckc >= now) {
 		return 0;
+	}
 	return ckc_interrupts_enabled(vcpu);
 }
 
@@ -1004,13 +1011,19 @@ int kvm_cpu_has_pending_timer(struct kvm
 
 static u64 __calculate_sltime(struct kvm_vcpu *vcpu)
 {
-	u64 now, cputm, sltime = 0;
+	const u64 now = kvm_s390_get_tod_clock_fast(vcpu->kvm);
+	const u64 ckc = vcpu->arch.sie_block->ckc;
+	u64 cputm, sltime = 0;
 
 	if (ckc_interrupts_enabled(vcpu)) {
-		now = kvm_s390_get_tod_clock_fast(vcpu->kvm);
-		sltime = tod_to_ns(vcpu->arch.sie_block->ckc - now);
-		/* already expired or overflow? */
-		if (!sltime || vcpu->arch.sie_block->ckc <= now)
+		if (vcpu->arch.sie_block->gcr[0] & 0x0020000000000000ul) {
+			if ((s64)now < (s64)ckc)
+				sltime = tod_to_ns((s64)ckc - (s64)now);
+		} else if (now < ckc) {
+			sltime = tod_to_ns(ckc - now);
+		}
+		/* already expired */
+		if (!sltime)
 			return 0;
 		if (cpu_timer_interrupts_enabled(vcpu)) {
 			cputm = kvm_s390_get_cpu_timer(vcpu);

Patches currently in stable-queue which might be from [email protected] are

queue-4.14/kvm-s390-take-care-of-clock-comparator-sign-control.patch queue-4.14/kvm-s390-provide-only-a-single-function-for-setting-the-tod-fix-sck.patch queue-4.14/kvm-s390-consider-epoch-index-on-tod-clock-syncs.patch queue-4.14/kvm-s390-consider-epoch-index-on-hotplugged-cpus.patch

maintainer Applied
#7 lore raw

Patch "KVM: s390: take care of clock-comparator sign control" has been added to the 4.15-stable tree

In reply to David Hildenbrand

This is a note to let you know that I've just added the patch titled

KVM: s390: take care of clock-comparator sign control

to the 4.15-stable tree which can be found at:
http://www.kernel.org/git/?p=linux/kernel/git/stable/stable-queue.git;a=summary

The filename of the patch is:
kvm-s390-take-care-of-clock-comparator-sign-control.patch
and it can be found in the queue-4.15 subdirectory.

If you, or anyone else, feels it should not be added to the stable tree, please let <[email protected]> know about it.

Quoted 1 line

From 5fe01793dd953ab947fababe8abaf5ed5258c8df Mon Sep 17 00:00:00 2001

From: David Hildenbrand <[email protected]>
Date: Wed, 7 Feb 2018 12:46:42 +0100
Subject: KVM: s390: take care of clock-comparator sign control

From: David Hildenbrand <[email protected]>

commit 5fe01793dd953ab947fababe8abaf5ed5258c8df upstream.

Missed when enabling the Multiple-epoch facility. If the facility is installed and the control is set, a sign based comaprison has to be performed.

Right now we would inject wrong interrupts and ignore interrupt conditions. Also the sleep time is calculated in a wrong way.

Message-Id: <[email protected]>

Fixes 8fa1696ea781 ("KVM: s390: Multiple Epoch Facility support")
---
 arch/s390/kvm/interrupt.c |   25 +++++++++++++++++++------
 1 file changed, 19 insertions(+), 6 deletions(-)
arch/s390/kvm/interrupt.c +19 -6
@@ -170,8 +170,15 @@ static int ckc_interrupts_enabled(struct
 
 static int ckc_irq_pending(struct kvm_vcpu *vcpu)
 {
-	if (vcpu->arch.sie_block->ckc >= kvm_s390_get_tod_clock_fast(vcpu->kvm))
+	const u64 now = kvm_s390_get_tod_clock_fast(vcpu->kvm);
+	const u64 ckc = vcpu->arch.sie_block->ckc;
+
+	if (vcpu->arch.sie_block->gcr[0] & 0x0020000000000000ul) {
+		if ((s64)ckc >= (s64)now)
+			return 0;
+	} else if (ckc >= now) {
 		return 0;
+	}
 	return ckc_interrupts_enabled(vcpu);
 }
 
@@ -1011,13 +1018,19 @@ int kvm_cpu_has_pending_timer(struct kvm
 
 static u64 __calculate_sltime(struct kvm_vcpu *vcpu)
 {
-	u64 now, cputm, sltime = 0;
+	const u64 now = kvm_s390_get_tod_clock_fast(vcpu->kvm);
+	const u64 ckc = vcpu->arch.sie_block->ckc;
+	u64 cputm, sltime = 0;
 
 	if (ckc_interrupts_enabled(vcpu)) {
-		now = kvm_s390_get_tod_clock_fast(vcpu->kvm);
-		sltime = tod_to_ns(vcpu->arch.sie_block->ckc - now);
-		/* already expired or overflow? */
-		if (!sltime || vcpu->arch.sie_block->ckc <= now)
+		if (vcpu->arch.sie_block->gcr[0] & 0x0020000000000000ul) {
+			if ((s64)now < (s64)ckc)
+				sltime = tod_to_ns((s64)ckc - (s64)now);
+		} else if (now < ckc) {
+			sltime = tod_to_ns(ckc - now);
+		}
+		/* already expired */
+		if (!sltime)
 			return 0;
 		if (cpu_timer_interrupts_enabled(vcpu)) {
 			cputm = kvm_s390_get_cpu_timer(vcpu);

Patches currently in stable-queue which might be from [email protected] are

queue-4.15/kvm-s390-take-care-of-clock-comparator-sign-control.patch queue-4.15/kvm-s390-provide-only-a-single-function-for-setting-the-tod-fix-sck.patch queue-4.15/kvm-s390-consider-epoch-index-on-tod-clock-syncs.patch queue-4.15/kvm-s390-consider-epoch-index-on-hotplugged-cpus.patch

#8 lore raw

RFC PATCH 2/6 KVM: s390: provide only a single function for setting the tod

In reply to David Hildenbrand

Right now, SET CLOCK called in the guest does not properly take care of the epoch index, as the call goes via the old kvm_s390_set_tod_clock() interface. So the epoch index is neither reset to 0, if required, nor properly set to e.g. 0xff on negative values.

Fix this by providing a single kvm_s390_set_tod_clock() function. Move Multiple-epoch facility handling into it.

---
 arch/s390/kvm/kvm-s390.c | 46 +++++++++++++++-------------------------------
 arch/s390/kvm/kvm-s390.h |  5 ++---
 arch/s390/kvm/priv.c     |  9 +++++----
 3 files changed, 22 insertions(+), 38 deletions(-)
arch/s390/kvm/kvm-s390.c +15 -31
index ba4c7092335a..b514ee427077 100644
@@ -902,12 +902,9 @@ static int kvm_s390_set_tod_ext(struct kvm *kvm, struct kvm_device_attr *attr)
 	if (copy_from_user(&gtod, (void __user *)attr->addr, sizeof(gtod)))
 		return -EFAULT;
 
-	if (test_kvm_facility(kvm, 139))
-		kvm_s390_set_tod_clock_ext(kvm, &gtod);
-	else if (gtod.epoch_idx == 0)
-		kvm_s390_set_tod_clock(kvm, gtod.tod);
-	else
+	if (!test_kvm_facility(kvm, 139) && gtod.epoch_idx)
 		return -EINVAL;
+	kvm_s390_set_tod_clock(kvm, &gtod);
 
 	VM_EVENT(kvm, 3, "SET: TOD extension: 0x%x, TOD base: 0x%llx",
 		gtod.epoch_idx, gtod.tod);
@@ -932,13 +929,14 @@ static int kvm_s390_set_tod_high(struct kvm *kvm, struct kvm_device_attr *attr)
 
 static int kvm_s390_set_tod_low(struct kvm *kvm, struct kvm_device_attr *attr)
 {
-	u64 gtod;
+	struct kvm_s390_vm_tod_clock gtod = { 0 };
 
-	if (copy_from_user(&gtod, (void __user *)attr->addr, sizeof(gtod)))
+	if (copy_from_user(&gtod.tod, (void __user *)attr->addr,
+			   sizeof(gtod.tod)))
 		return -EFAULT;
 
-	kvm_s390_set_tod_clock(kvm, gtod);
-	VM_EVENT(kvm, 3, "SET: TOD base: 0x%llx", gtod);
+	kvm_s390_set_tod_clock(kvm, &gtod);
+	VM_EVENT(kvm, 3, "SET: TOD base: 0x%llx", gtod.tod);
 	return 0;
 }
 
@@ -3021,8 +3019,8 @@ static int kvm_s390_handle_requests(struct kvm_vcpu *vcpu)
 	return 0;
 }
 
-void kvm_s390_set_tod_clock_ext(struct kvm *kvm,
-				 const struct kvm_s390_vm_tod_clock *gtod)
+void kvm_s390_set_tod_clock(struct kvm *kvm,
+			    const struct kvm_s390_vm_tod_clock *gtod)
 {
 	struct kvm_vcpu *vcpu;
 	struct kvm_s390_tod_clock_ext htod;
@@ -3034,10 +3032,12 @@ void kvm_s390_set_tod_clock_ext(struct kvm *kvm,
 	get_tod_clock_ext((char *)&htod);
 
 	kvm->arch.epoch = gtod->tod - htod.tod;
-	kvm->arch.epdx = gtod->epoch_idx - htod.epoch_idx;
-
-	if (kvm->arch.epoch > gtod->tod)
-		kvm->arch.epdx -= 1;
+	kvm->arch.epdx = 0;
+	if (test_kvm_facility(kvm, 139)) {
+		kvm->arch.epdx = gtod->epoch_idx - htod.epoch_idx;
+		if (kvm->arch.epoch > gtod->tod)
+			kvm->arch.epdx -= 1;
+	}
 
 	kvm_s390_vcpu_block_all(kvm);
 	kvm_for_each_vcpu(i, vcpu, kvm) {
@@ -3050,22 +3050,6 @@ void kvm_s390_set_tod_clock_ext(struct kvm *kvm,
 	mutex_unlock(&kvm->lock);
 }
 
-void kvm_s390_set_tod_clock(struct kvm *kvm, u64 tod)
-{
-	struct kvm_vcpu *vcpu;
-	int i;
-
-	mutex_lock(&kvm->lock);
-	preempt_disable();
-	kvm->arch.epoch = tod - get_tod_clock();
-	kvm_s390_vcpu_block_all(kvm);
-	kvm_for_each_vcpu(i, vcpu, kvm)
-		vcpu->arch.sie_block->epoch = kvm->arch.epoch;
-	kvm_s390_vcpu_unblock_all(kvm);
-	preempt_enable();
-	mutex_unlock(&kvm->lock);
-}
-
 /**
  * kvm_arch_fault_in_page - fault-in guest page if necessary
  * @vcpu: The corresponding virtual cpu
arch/s390/kvm/kvm-s390.h +2 -3
index bd31b37b0e6f..19d719262765 100644
@@ -283,9 +283,8 @@ int kvm_s390_handle_sigp(struct kvm_vcpu *vcpu);
 int kvm_s390_handle_sigp_pei(struct kvm_vcpu *vcpu);
 
 /* implemented in kvm-s390.c */
-void kvm_s390_set_tod_clock_ext(struct kvm *kvm,
-				 const struct kvm_s390_vm_tod_clock *gtod);
-void kvm_s390_set_tod_clock(struct kvm *kvm, u64 tod);
+void kvm_s390_set_tod_clock(struct kvm *kvm,
+			    const struct kvm_s390_vm_tod_clock *gtod);
 long kvm_arch_fault_in_page(struct kvm_vcpu *vcpu, gpa_t gpa, int writable);
 int kvm_s390_store_status_unloaded(struct kvm_vcpu *vcpu, unsigned long addr);
 int kvm_s390_vcpu_store_status(struct kvm_vcpu *vcpu, unsigned long addr);
arch/s390/kvm/priv.c +5 -4
index c4c4e157c036..33505c32c48a 100644
@@ -85,9 +85,10 @@ int kvm_s390_handle_e3(struct kvm_vcpu *vcpu)
 /* Handle SCK (SET CLOCK) interception */
 static int handle_set_clock(struct kvm_vcpu *vcpu)
 {
+	struct kvm_s390_vm_tod_clock gtod = { 0 };
 	int rc;
 	u8 ar;
-	u64 op2, val;
+	u64 op2;
 
 	vcpu->stat.instruction_sck++;
 
@@ -97,12 +98,12 @@ static int handle_set_clock(struct kvm_vcpu *vcpu)
 	op2 = kvm_s390_get_base_disp_s(vcpu, &ar);
 	if (op2 & 7)	/* Operand must be on a doubleword boundary */
 		return kvm_s390_inject_program_int(vcpu, PGM_SPECIFICATION);
-	rc = read_guest(vcpu, op2, ar, &val, sizeof(val));
+	rc = read_guest(vcpu, op2, ar, &gtod.tod, sizeof(gtod.tod));
 	if (rc)
 		return kvm_s390_inject_prog_cond(vcpu, rc);
 
-	VCPU_EVENT(vcpu, 3, "SCK: setting guest TOD to 0x%llx", val);
-	kvm_s390_set_tod_clock(vcpu->kvm, val);
+	VCPU_EVENT(vcpu, 3, "SCK: setting guest TOD to 0x%llx", gtod.tod);
+	kvm_s390_set_tod_clock(vcpu->kvm, &gtod);
 
 	kvm_s390_set_psw_cc(vcpu, 0);
 	return 0;
Signature · 1 line
2.14.3
#9 lore raw

Re: [PATCH RFC 2/6] KVM: s390: provide only a single function for setting the tod

Quoted 54 lines · David Hildenbrand

Right now, SET CLOCK called in the guest does not properly take care of the epoch index, as the call goes via the old kvm_s390_set_tod_clock() interface. So the epoch index is neither reset to 0, if required, nor properly set to e.g. 0xff on negative values.

Fix this by providing a single kvm_s390_set_tod_clock() function. Move Multiple-epoch facility handling into it.

---
  arch/s390/kvm/kvm-s390.c | 46 +++++++++++++++-------------------------------
  arch/s390/kvm/kvm-s390.h |  5 ++---
  arch/s390/kvm/priv.c     |  9 +++++----
  3 files changed, 22 insertions(+), 38 deletions(-)
arch/s390/kvm/kvm-s390.c +7 -9
index ba4c7092335a..b514ee427077 100644
@@ -902,12 +902,9 @@ static int kvm_s390_set_tod_ext(struct kvm *kvm, struct kvm_device_attr *attr)
  	if (copy_from_user(&gtod, (void __user *)attr->addr, sizeof(gtod)))
  		return -EFAULT;
 
-	if (test_kvm_facility(kvm, 139))
-		kvm_s390_set_tod_clock_ext(kvm, &gtod);
-	else if (gtod.epoch_idx == 0)
-		kvm_s390_set_tod_clock(kvm, gtod.tod);
-	else
+	if (!test_kvm_facility(kvm, 139) && gtod.epoch_idx)
  		return -EINVAL;
+	kvm_s390_set_tod_clock(kvm, &gtod);
 
  	VM_EVENT(kvm, 3, "SET: TOD extension: 0x%x, TOD base: 0x%llx",
  		gtod.epoch_idx, gtod.tod);
@@ -932,13 +929,14 @@ static int kvm_s390_set_tod_high(struct kvm *kvm, struct kvm_device_attr *attr)
 
  static int kvm_s390_set_tod_low(struct kvm *kvm, struct kvm_device_attr *attr)
  {
-	u64 gtod;
+	struct kvm_s390_vm_tod_clock gtod = { 0 };
 
-	if (copy_from_user(&gtod, (void __user *)attr->addr, sizeof(gtod)))
+	if (copy_from_user(&gtod.tod, (void __user *)attr->addr,
+			   sizeof(gtod.tod)))
  		return -EFAULT;
 
-	kvm_s390_set_tod_clock(kvm, gtod);
-	VM_EVENT(kvm, 3, "SET: TOD base: 0x%llx", gtod);
+	kvm_s390_set_tod_clock(kvm, &gtod);
+	VM_EVENT(kvm, 3, "SET: TOD base: 0x%llx", gtod.tod);
  	return 0;
  }
 

arch/s390/kvm/kvm-s390.c static int kvm_s390_handle_requests(struct kvm_vcpu *vcpu) · line 3023

Quoted 7 lines
arch/s390/kvm/kvm-s390.c +2 -2
@@ -3021,8 +3019,8 @@ static int kvm_s390_handle_requests(struct kvm_vcpu *vcpu)
  	return 0;
  }
 
-void kvm_s390_set_tod_clock_ext(struct kvm *kvm,
-				 const struct kvm_s390_vm_tod_clock *gtod)
+void kvm_s390_set_tod_clock(struct kvm *kvm,
+			    const struct kvm_s390_vm_tod_clock *gtod)

Nit: off by one space --------^

Quoted 48 lines
arch/s390/kvm/kvm-s390.c +6 -20
  {
  	struct kvm_vcpu *vcpu;
  	struct kvm_s390_tod_clock_ext htod;
@@ -3034,10 +3032,12 @@ void kvm_s390_set_tod_clock_ext(struct kvm *kvm,
  	get_tod_clock_ext((char *)&htod);
 
  	kvm->arch.epoch = gtod->tod - htod.tod;
-	kvm->arch.epdx = gtod->epoch_idx - htod.epoch_idx;
-
-	if (kvm->arch.epoch > gtod->tod)
-		kvm->arch.epdx -= 1;
+	kvm->arch.epdx = 0;
+	if (test_kvm_facility(kvm, 139)) {
+		kvm->arch.epdx = gtod->epoch_idx - htod.epoch_idx;
+		if (kvm->arch.epoch > gtod->tod)
+			kvm->arch.epdx -= 1;
+	}
 
  	kvm_s390_vcpu_block_all(kvm);
  	kvm_for_each_vcpu(i, vcpu, kvm) {
@@ -3050,22 +3050,6 @@ void kvm_s390_set_tod_clock_ext(struct kvm *kvm,
  	mutex_unlock(&kvm->lock);
  }
 
-void kvm_s390_set_tod_clock(struct kvm *kvm, u64 tod)
-{
-	struct kvm_vcpu *vcpu;
-	int i;
-
-	mutex_lock(&kvm->lock);
-	preempt_disable();
-	kvm->arch.epoch = tod - get_tod_clock();
-	kvm_s390_vcpu_block_all(kvm);
-	kvm_for_each_vcpu(i, vcpu, kvm)
-		vcpu->arch.sie_block->epoch = kvm->arch.epoch;
-	kvm_s390_vcpu_unblock_all(kvm);
-	preempt_enable();
-	mutex_unlock(&kvm->lock);
-}
-
  /**
   * kvm_arch_fault_in_page - fault-in guest page if necessary
   * @vcpu: The corresponding virtual cpu
arch/s390/kvm/kvm-s390.h +0 -0
index bd31b37b0e6f..19d719262765 100644

arch/s390/kvm/kvm-s390.h int kvm_s390_handle_sigp(struct kvm_vcpu *vcpu); · line 287

Quoted 8 lines
arch/s390/kvm/kvm-s390.h +2 -3
@@ -283,9 +283,8 @@ int kvm_s390_handle_sigp(struct kvm_vcpu *vcpu);
  int kvm_s390_handle_sigp_pei(struct kvm_vcpu *vcpu);
 
  /* implemented in kvm-s390.c */
-void kvm_s390_set_tod_clock_ext(struct kvm *kvm,
-				 const struct kvm_s390_vm_tod_clock *gtod);
-void kvm_s390_set_tod_clock(struct kvm *kvm, u64 tod);
+void kvm_s390_set_tod_clock(struct kvm *kvm,
+			    const struct kvm_s390_vm_tod_clock *gtod);

Same nit here.

Quoted 25 lines
arch/s390/kvm/kvm-s390.h +0 -0
  long kvm_arch_fault_in_page(struct kvm_vcpu *vcpu, gpa_t gpa, int writable);
  int kvm_s390_store_status_unloaded(struct kvm_vcpu *vcpu, unsigned long addr);
  int kvm_s390_vcpu_store_status(struct kvm_vcpu *vcpu, unsigned long addr);
arch/s390/kvm/priv.c +3 -2
index c4c4e157c036..33505c32c48a 100644
@@ -85,9 +85,10 @@ int kvm_s390_handle_e3(struct kvm_vcpu *vcpu)
  /* Handle SCK (SET CLOCK) interception */
  static int handle_set_clock(struct kvm_vcpu *vcpu)
  {
+	struct kvm_s390_vm_tod_clock gtod = { 0 };
  	int rc;
  	u8 ar;
-	u64 op2, val;
+	u64 op2;
 
  	vcpu->stat.instruction_sck++;
 
@@ -97,12 +98,12 @@ static int handle_set_clock(struct kvm_vcpu *vcpu)
  	op2 = kvm_s390_get_base_disp_s(vcpu, &ar);
  	if (op2 & 7)	/* Operand must be on a doubleword boundary */
  		return kvm_s390_inject_program_int(vcpu, PGM_SPECIFICATION);
-	rc = read_guest(vcpu, op2, ar, &val, sizeof(val));
+	rc = read_guest(vcpu, op2, ar, &gtod.tod, sizeof(gtod.tod));

arch/s390/kvm/priv.c static int handle_set_clock(struct kvm_vcpu *vcpu) · line 109

Quoted 10 lines
arch/s390/kvm/priv.c +2 -2
  	if (rc)
  		return kvm_s390_inject_prog_cond(vcpu, rc);
 
-	VCPU_EVENT(vcpu, 3, "SCK: setting guest TOD to 0x%llx", val);
-	kvm_s390_set_tod_clock(vcpu->kvm, val);
+	VCPU_EVENT(vcpu, 3, "SCK: setting guest TOD to 0x%llx", gtod.tod);
+	kvm_s390_set_tod_clock(vcpu->kvm, &gtod);
 
  	kvm_s390_set_psw_cc(vcpu, 0);
  	return 0;

Set clock only concerns the left-most 64 bits of the TOD-Clock, right? (thinking out loud) I wonder if we need to also concern ourselves about the epoch extension... but perhaps current use cases of set clock do not warrant that?

Signature · 1 line
- Collin L Walling
#10 lore raw

Re: [PATCH RFC 2/6] KVM: s390: provide only a single function for setting the tod

Quoted 8 lines · Collin L. Walling
Quoted 7 lines · David Hildenbrand

[...]

arch/s390/kvm/kvm-s390.c +0 -0
@@ -3021,8 +3019,8 @@ static int kvm_s390_handle_requests(struct 
 kvm_vcpu *vcpu)
       return 0;
   }
 
Quoted 5 lines

arch/s390/kvm/kvm-s390.c static int kvm_s390_handle_requests(struct kvm_vcpu *vcpu) · line 3023

Quoted 4 lines
arch/s390/kvm/kvm-s390.c +2 -2
-void kvm_s390_set_tod_clock_ext(struct kvm *kvm,
-                 const struct kvm_s390_vm_tod_clock *gtod)
+void kvm_s390_set_tod_clock(struct kvm *kvm,
+                const struct kvm_s390_vm_tod_clock *gtod)

Nit: off by one space --------^

or perhaps my email client is lying to me? ignore my comment if it looks fine on your end.

Signature · 1 line
- Collin L Walling
#11 lore raw

Re: [PATCH RFC 2/6] KVM: s390: provide only a single function for setting the tod

In reply to Collin L. Walling
Quoted 7 lines

arch/s390/kvm/kvm-s390.c static int kvm_s390_handle_requests(struct kvm_vcpu *vcpu) · line 3023

Quoted 6 lines
arch/s390/kvm/kvm-s390.c +2 -2
-void kvm_s390_set_tod_clock_ext(struct kvm *kvm,
-				 const struct kvm_s390_vm_tod_clock *gtod)
+void kvm_s390_set_tod_clock(struct kvm *kvm,
+			    const struct kvm_s390_vm_tod_clock *gtod)

Nit: off by one space --------^

This looks fine in code, just a quirk in the visual representation of the diff file.

[...]

Quoted 7 lines

arch/s390/kvm/kvm-s390.h int kvm_s390_handle_sigp(struct kvm_vcpu *vcpu); · line 287

Quoted 6 lines

/* implemented in kvm-s390.c */

arch/s390/kvm/kvm-s390.h +2 -3
-void kvm_s390_set_tod_clock_ext(struct kvm *kvm,
-				 const struct kvm_s390_vm_tod_clock *gtod);
-void kvm_s390_set_tod_clock(struct kvm *kvm, u64 tod);
+void kvm_s390_set_tod_clock(struct kvm *kvm,
+			    const struct kvm_s390_vm_tod_clock *gtod);

Same nit here.

Dito.

Quoted 33 lines
Quoted 33 lines
  long kvm_arch_fault_in_page(struct kvm_vcpu *vcpu, gpa_t gpa, int writable);
  int kvm_s390_store_status_unloaded(struct kvm_vcpu *vcpu, unsigned long addr);
  int kvm_s390_vcpu_store_status(struct kvm_vcpu *vcpu, unsigned long addr);
arch/s390/kvm/priv.c +5 -4
index c4c4e157c036..33505c32c48a 100644
@@ -85,9 +85,10 @@ int kvm_s390_handle_e3(struct kvm_vcpu *vcpu)
  /* Handle SCK (SET CLOCK) interception */
  static int handle_set_clock(struct kvm_vcpu *vcpu)
  {
+	struct kvm_s390_vm_tod_clock gtod = { 0 };
  	int rc;
  	u8 ar;
-	u64 op2, val;
+	u64 op2;
 
  	vcpu->stat.instruction_sck++;
 
@@ -97,12 +98,12 @@ static int handle_set_clock(struct kvm_vcpu *vcpu)
  	op2 = kvm_s390_get_base_disp_s(vcpu, &ar);
  	if (op2 & 7)	/* Operand must be on a doubleword boundary */
  		return kvm_s390_inject_program_int(vcpu, PGM_SPECIFICATION);
-	rc = read_guest(vcpu, op2, ar, &val, sizeof(val));
+	rc = read_guest(vcpu, op2, ar, &gtod.tod, sizeof(gtod.tod));
  	if (rc)
  		return kvm_s390_inject_prog_cond(vcpu, rc);
 
-	VCPU_EVENT(vcpu, 3, "SCK: setting guest TOD to 0x%llx", val);
-	kvm_s390_set_tod_clock(vcpu->kvm, val);
+	VCPU_EVENT(vcpu, 3, "SCK: setting guest TOD to 0x%llx", gtod.tod);
+	kvm_s390_set_tod_clock(vcpu->kvm, &gtod);
 
Quoted 7 lines

arch/s390/kvm/priv.c static int handle_set_clock(struct kvm_vcpu *vcpu) · line 109

Quoted 2 lines
arch/s390/kvm/priv.c +0 -0
  	kvm_s390_set_psw_cc(vcpu, 0);
  	return 0;

Set clock only concerns the left-most 64 bits of the TOD-Clock, right? (thinking out
loud) I wonder if we need to also concern ourselves about the epoch extension... but
perhaps current use cases of set clock do not warrant that?

SET CLOCK will always set the right-most 64 bits of the TOD. The left-most bits are assumed to be 0 (that's how I understand the architecture). SET CLOCK can at one point no longer be reliably used.

But until these years come, SET CLOCK has to continue to work (even if the guest has Multiple-epoch facility). And that can be guaranteed by setting the epoch index accordingly.

Especially, if we have with mepoch: "Guest TOD = HOST TOD - 1", the epoch index has to be set to 0xff and the epoch to 0xffff...fff. Right now, the epoch index would not be set.

Should be easy to reproduced by doing a

STORE CLOCK %d followed by A SET_CLOCK %d in the guest. As the Host TOD has been incremented, we have "Guest TOD = Host TOD - X", and not setting the epoch index to 0xff results in a wrong Guest TOD calculation on the next STORE CLOCK.

Thanks!

Signature · 3 lines
Thanks,

David / dhildenb
maintainer Applied
#12 lore raw

Patch "KVM: s390: provide only a single function for setting the tod (fix SCK)" has been added to the 4.14-stable tree

In reply to David Hildenbrand

This is a note to let you know that I've just added the patch titled

KVM: s390: provide only a single function for setting the tod (fix SCK)

to the 4.14-stable tree which can be found at:
http://www.kernel.org/git/?p=linux/kernel/git/stable/stable-queue.git;a=summary

The filename of the patch is:
kvm-s390-provide-only-a-single-function-for-setting-the-tod-fix-sck.patch
and it can be found in the queue-4.14 subdirectory.

If you, or anyone else, feels it should not be added to the stable tree, please let <[email protected]> know about it.

Quoted 1 line

From 0e7def5fb0dc53ddbb9f62a497d15f1e11ccdc36 Mon Sep 17 00:00:00 2001

From: David Hildenbrand <[email protected]>
Date: Wed, 7 Feb 2018 12:46:43 +0100
Subject: KVM: s390: provide only a single function for setting the tod (fix SCK)

From: David Hildenbrand <[email protected]>

commit 0e7def5fb0dc53ddbb9f62a497d15f1e11ccdc36 upstream.

Right now, SET CLOCK called in the guest does not properly take care of the epoch index, as the call goes via the old kvm_s390_set_tod_clock() interface. So the epoch index is neither reset to 0, if required, nor properly set to e.g. 0xff on negative values.

Fix this by providing a single kvm_s390_set_tod_clock() function. Move Multiple-epoch facility handling into it.

Message-Id: <[email protected]>

Fixes 8fa1696ea781 ("KVM: s390: Multiple Epoch Facility support")
---
 arch/s390/kvm/kvm-s390.c |   46 +++++++++++++++-------------------------------
 arch/s390/kvm/kvm-s390.h |    5 ++---
 arch/s390/kvm/priv.c     |    9 +++++----
 3 files changed, 22 insertions(+), 38 deletions(-)
arch/s390/kvm/kvm-s390.c +15 -31
@@ -888,12 +888,9 @@ static int kvm_s390_set_tod_ext(struct k
 	if (copy_from_user(&gtod, (void __user *)attr->addr, sizeof(gtod)))
 		return -EFAULT;
 
-	if (test_kvm_facility(kvm, 139))
-		kvm_s390_set_tod_clock_ext(kvm, &gtod);
-	else if (gtod.epoch_idx == 0)
-		kvm_s390_set_tod_clock(kvm, gtod.tod);
-	else
+	if (!test_kvm_facility(kvm, 139) && gtod.epoch_idx)
 		return -EINVAL;
+	kvm_s390_set_tod_clock(kvm, &gtod);
 
 	VM_EVENT(kvm, 3, "SET: TOD extension: 0x%x, TOD base: 0x%llx",
 		gtod.epoch_idx, gtod.tod);
@@ -918,13 +915,14 @@ static int kvm_s390_set_tod_high(struct
 
 static int kvm_s390_set_tod_low(struct kvm *kvm, struct kvm_device_attr *attr)
 {
-	u64 gtod;
+	struct kvm_s390_vm_tod_clock gtod = { 0 };
 
-	if (copy_from_user(&gtod, (void __user *)attr->addr, sizeof(gtod)))
+	if (copy_from_user(&gtod.tod, (void __user *)attr->addr,
+			   sizeof(gtod.tod)))
 		return -EFAULT;
 
-	kvm_s390_set_tod_clock(kvm, gtod);
-	VM_EVENT(kvm, 3, "SET: TOD base: 0x%llx", gtod);
+	kvm_s390_set_tod_clock(kvm, &gtod);
+	VM_EVENT(kvm, 3, "SET: TOD base: 0x%llx", gtod.tod);
 	return 0;
 }
 
@@ -2945,8 +2943,8 @@ retry:
 	return 0;
 }
 
-void kvm_s390_set_tod_clock_ext(struct kvm *kvm,
-				 const struct kvm_s390_vm_tod_clock *gtod)
+void kvm_s390_set_tod_clock(struct kvm *kvm,
+			    const struct kvm_s390_vm_tod_clock *gtod)
 {
 	struct kvm_vcpu *vcpu;
 	struct kvm_s390_tod_clock_ext htod;
@@ -2958,10 +2956,12 @@ void kvm_s390_set_tod_clock_ext(struct k
 	get_tod_clock_ext((char *)&htod);
 
 	kvm->arch.epoch = gtod->tod - htod.tod;
-	kvm->arch.epdx = gtod->epoch_idx - htod.epoch_idx;
-
-	if (kvm->arch.epoch > gtod->tod)
-		kvm->arch.epdx -= 1;
+	kvm->arch.epdx = 0;
+	if (test_kvm_facility(kvm, 139)) {
+		kvm->arch.epdx = gtod->epoch_idx - htod.epoch_idx;
+		if (kvm->arch.epoch > gtod->tod)
+			kvm->arch.epdx -= 1;
+	}
 
 	kvm_s390_vcpu_block_all(kvm);
 	kvm_for_each_vcpu(i, vcpu, kvm) {
@@ -2972,22 +2972,6 @@ void kvm_s390_set_tod_clock_ext(struct k
 	kvm_s390_vcpu_unblock_all(kvm);
 	preempt_enable();
 	mutex_unlock(&kvm->lock);
-}
-
-void kvm_s390_set_tod_clock(struct kvm *kvm, u64 tod)
-{
-	struct kvm_vcpu *vcpu;
-	int i;
-
-	mutex_lock(&kvm->lock);
-	preempt_disable();
-	kvm->arch.epoch = tod - get_tod_clock();
-	kvm_s390_vcpu_block_all(kvm);
-	kvm_for_each_vcpu(i, vcpu, kvm)
-		vcpu->arch.sie_block->epoch = kvm->arch.epoch;
-	kvm_s390_vcpu_unblock_all(kvm);
-	preempt_enable();
-	mutex_unlock(&kvm->lock);
 }
 
 /**
arch/s390/kvm/kvm-s390.h +2 -3
@@ -272,9 +272,8 @@ int kvm_s390_handle_sigp_pei(struct kvm_
 int handle_sthyi(struct kvm_vcpu *vcpu);
 
 /* implemented in kvm-s390.c */
-void kvm_s390_set_tod_clock_ext(struct kvm *kvm,
-				 const struct kvm_s390_vm_tod_clock *gtod);
-void kvm_s390_set_tod_clock(struct kvm *kvm, u64 tod);
+void kvm_s390_set_tod_clock(struct kvm *kvm,
+			    const struct kvm_s390_vm_tod_clock *gtod);
 long kvm_arch_fault_in_page(struct kvm_vcpu *vcpu, gpa_t gpa, int writable);
 int kvm_s390_store_status_unloaded(struct kvm_vcpu *vcpu, unsigned long addr);
 int kvm_s390_vcpu_store_status(struct kvm_vcpu *vcpu, unsigned long addr);
arch/s390/kvm/priv.c +5 -4
@@ -84,9 +84,10 @@ int kvm_s390_handle_e3(struct kvm_vcpu *
 /* Handle SCK (SET CLOCK) interception */
 static int handle_set_clock(struct kvm_vcpu *vcpu)
 {
+	struct kvm_s390_vm_tod_clock gtod = { 0 };
 	int rc;
 	u8 ar;
-	u64 op2, val;
+	u64 op2;
 
 	if (vcpu->arch.sie_block->gpsw.mask & PSW_MASK_PSTATE)
 		return kvm_s390_inject_program_int(vcpu, PGM_PRIVILEGED_OP);
@@ -94,12 +95,12 @@ static int handle_set_clock(struct kvm_v
 	op2 = kvm_s390_get_base_disp_s(vcpu, &ar);
 	if (op2 & 7)	/* Operand must be on a doubleword boundary */
 		return kvm_s390_inject_program_int(vcpu, PGM_SPECIFICATION);
-	rc = read_guest(vcpu, op2, ar, &val, sizeof(val));
+	rc = read_guest(vcpu, op2, ar, &gtod.tod, sizeof(gtod.tod));
 	if (rc)
 		return kvm_s390_inject_prog_cond(vcpu, rc);
 
-	VCPU_EVENT(vcpu, 3, "SCK: setting guest TOD to 0x%llx", val);
-	kvm_s390_set_tod_clock(vcpu->kvm, val);
+	VCPU_EVENT(vcpu, 3, "SCK: setting guest TOD to 0x%llx", gtod.tod);
+	kvm_s390_set_tod_clock(vcpu->kvm, &gtod);
 
 	kvm_s390_set_psw_cc(vcpu, 0);
 	return 0;

Patches currently in stable-queue which might be from [email protected] are

queue-4.14/kvm-s390-take-care-of-clock-comparator-sign-control.patch queue-4.14/kvm-s390-provide-only-a-single-function-for-setting-the-tod-fix-sck.patch queue-4.14/kvm-s390-consider-epoch-index-on-tod-clock-syncs.patch queue-4.14/kvm-s390-consider-epoch-index-on-hotplugged-cpus.patch

maintainer Applied
#13 lore raw

Patch "KVM: s390: provide only a single function for setting the tod (fix SCK)" has been added to the 4.15-stable tree

In reply to David Hildenbrand

This is a note to let you know that I've just added the patch titled

KVM: s390: provide only a single function for setting the tod (fix SCK)

to the 4.15-stable tree which can be found at:
http://www.kernel.org/git/?p=linux/kernel/git/stable/stable-queue.git;a=summary

The filename of the patch is:
kvm-s390-provide-only-a-single-function-for-setting-the-tod-fix-sck.patch
and it can be found in the queue-4.15 subdirectory.

If you, or anyone else, feels it should not be added to the stable tree, please let <[email protected]> know about it.

Quoted 1 line

From 0e7def5fb0dc53ddbb9f62a497d15f1e11ccdc36 Mon Sep 17 00:00:00 2001

From: David Hildenbrand <[email protected]>
Date: Wed, 7 Feb 2018 12:46:43 +0100
Subject: KVM: s390: provide only a single function for setting the tod (fix SCK)

From: David Hildenbrand <[email protected]>

commit 0e7def5fb0dc53ddbb9f62a497d15f1e11ccdc36 upstream.

Right now, SET CLOCK called in the guest does not properly take care of the epoch index, as the call goes via the old kvm_s390_set_tod_clock() interface. So the epoch index is neither reset to 0, if required, nor properly set to e.g. 0xff on negative values.

Fix this by providing a single kvm_s390_set_tod_clock() function. Move Multiple-epoch facility handling into it.

Message-Id: <[email protected]>

Fixes 8fa1696ea781 ("KVM: s390: Multiple Epoch Facility support")
---
 arch/s390/kvm/kvm-s390.c |   46 +++++++++++++++-------------------------------
 arch/s390/kvm/kvm-s390.h |    5 ++---
 arch/s390/kvm/priv.c     |    9 +++++----
 3 files changed, 22 insertions(+), 38 deletions(-)
arch/s390/kvm/kvm-s390.c +15 -31
@@ -889,12 +889,9 @@ static int kvm_s390_set_tod_ext(struct k
 	if (copy_from_user(&gtod, (void __user *)attr->addr, sizeof(gtod)))
 		return -EFAULT;
 
-	if (test_kvm_facility(kvm, 139))
-		kvm_s390_set_tod_clock_ext(kvm, &gtod);
-	else if (gtod.epoch_idx == 0)
-		kvm_s390_set_tod_clock(kvm, gtod.tod);
-	else
+	if (!test_kvm_facility(kvm, 139) && gtod.epoch_idx)
 		return -EINVAL;
+	kvm_s390_set_tod_clock(kvm, &gtod);
 
 	VM_EVENT(kvm, 3, "SET: TOD extension: 0x%x, TOD base: 0x%llx",
 		gtod.epoch_idx, gtod.tod);
@@ -919,13 +916,14 @@ static int kvm_s390_set_tod_high(struct
 
 static int kvm_s390_set_tod_low(struct kvm *kvm, struct kvm_device_attr *attr)
 {
-	u64 gtod;
+	struct kvm_s390_vm_tod_clock gtod = { 0 };
 
-	if (copy_from_user(&gtod, (void __user *)attr->addr, sizeof(gtod)))
+	if (copy_from_user(&gtod.tod, (void __user *)attr->addr,
+			   sizeof(gtod.tod)))
 		return -EFAULT;
 
-	kvm_s390_set_tod_clock(kvm, gtod);
-	VM_EVENT(kvm, 3, "SET: TOD base: 0x%llx", gtod);
+	kvm_s390_set_tod_clock(kvm, &gtod);
+	VM_EVENT(kvm, 3, "SET: TOD base: 0x%llx", gtod.tod);
 	return 0;
 }
 
@@ -2947,8 +2945,8 @@ retry:
 	return 0;
 }
 
-void kvm_s390_set_tod_clock_ext(struct kvm *kvm,
-				 const struct kvm_s390_vm_tod_clock *gtod)
+void kvm_s390_set_tod_clock(struct kvm *kvm,
+			    const struct kvm_s390_vm_tod_clock *gtod)
 {
 	struct kvm_vcpu *vcpu;
 	struct kvm_s390_tod_clock_ext htod;
@@ -2960,10 +2958,12 @@ void kvm_s390_set_tod_clock_ext(struct k
 	get_tod_clock_ext((char *)&htod);
 
 	kvm->arch.epoch = gtod->tod - htod.tod;
-	kvm->arch.epdx = gtod->epoch_idx - htod.epoch_idx;
-
-	if (kvm->arch.epoch > gtod->tod)
-		kvm->arch.epdx -= 1;
+	kvm->arch.epdx = 0;
+	if (test_kvm_facility(kvm, 139)) {
+		kvm->arch.epdx = gtod->epoch_idx - htod.epoch_idx;
+		if (kvm->arch.epoch > gtod->tod)
+			kvm->arch.epdx -= 1;
+	}
 
 	kvm_s390_vcpu_block_all(kvm);
 	kvm_for_each_vcpu(i, vcpu, kvm) {
@@ -2974,22 +2974,6 @@ void kvm_s390_set_tod_clock_ext(struct k
 	kvm_s390_vcpu_unblock_all(kvm);
 	preempt_enable();
 	mutex_unlock(&kvm->lock);
-}
-
-void kvm_s390_set_tod_clock(struct kvm *kvm, u64 tod)
-{
-	struct kvm_vcpu *vcpu;
-	int i;
-
-	mutex_lock(&kvm->lock);
-	preempt_disable();
-	kvm->arch.epoch = tod - get_tod_clock();
-	kvm_s390_vcpu_block_all(kvm);
-	kvm_for_each_vcpu(i, vcpu, kvm)
-		vcpu->arch.sie_block->epoch = kvm->arch.epoch;
-	kvm_s390_vcpu_unblock_all(kvm);
-	preempt_enable();
-	mutex_unlock(&kvm->lock);
 }
 
 /**
arch/s390/kvm/kvm-s390.h +2 -3
@@ -268,9 +268,8 @@ int kvm_s390_handle_sigp(struct kvm_vcpu
 int kvm_s390_handle_sigp_pei(struct kvm_vcpu *vcpu);
 
 /* implemented in kvm-s390.c */
-void kvm_s390_set_tod_clock_ext(struct kvm *kvm,
-				 const struct kvm_s390_vm_tod_clock *gtod);
-void kvm_s390_set_tod_clock(struct kvm *kvm, u64 tod);
+void kvm_s390_set_tod_clock(struct kvm *kvm,
+			    const struct kvm_s390_vm_tod_clock *gtod);
 long kvm_arch_fault_in_page(struct kvm_vcpu *vcpu, gpa_t gpa, int writable);
 int kvm_s390_store_status_unloaded(struct kvm_vcpu *vcpu, unsigned long addr);
 int kvm_s390_vcpu_store_status(struct kvm_vcpu *vcpu, unsigned long addr);
arch/s390/kvm/priv.c +5 -4
@@ -81,9 +81,10 @@ int kvm_s390_handle_e3(struct kvm_vcpu *
 /* Handle SCK (SET CLOCK) interception */
 static int handle_set_clock(struct kvm_vcpu *vcpu)
 {
+	struct kvm_s390_vm_tod_clock gtod = { 0 };
 	int rc;
 	u8 ar;
-	u64 op2, val;
+	u64 op2;
 
 	if (vcpu->arch.sie_block->gpsw.mask & PSW_MASK_PSTATE)
 		return kvm_s390_inject_program_int(vcpu, PGM_PRIVILEGED_OP);
@@ -91,12 +92,12 @@ static int handle_set_clock(struct kvm_v
 	op2 = kvm_s390_get_base_disp_s(vcpu, &ar);
 	if (op2 & 7)	/* Operand must be on a doubleword boundary */
 		return kvm_s390_inject_program_int(vcpu, PGM_SPECIFICATION);
-	rc = read_guest(vcpu, op2, ar, &val, sizeof(val));
+	rc = read_guest(vcpu, op2, ar, &gtod.tod, sizeof(gtod.tod));
 	if (rc)
 		return kvm_s390_inject_prog_cond(vcpu, rc);
 
-	VCPU_EVENT(vcpu, 3, "SCK: setting guest TOD to 0x%llx", val);
-	kvm_s390_set_tod_clock(vcpu->kvm, val);
+	VCPU_EVENT(vcpu, 3, "SCK: setting guest TOD to 0x%llx", gtod.tod);
+	kvm_s390_set_tod_clock(vcpu->kvm, &gtod);
 
 	kvm_s390_set_psw_cc(vcpu, 0);
 	return 0;

Patches currently in stable-queue which might be from [email protected] are

queue-4.15/kvm-s390-take-care-of-clock-comparator-sign-control.patch queue-4.15/kvm-s390-provide-only-a-single-function-for-setting-the-tod-fix-sck.patch queue-4.15/kvm-s390-consider-epoch-index-on-tod-clock-syncs.patch queue-4.15/kvm-s390-consider-epoch-index-on-hotplugged-cpus.patch

#14 lore raw

RFC PATCH 3/6 KVM: s390: consider epoch index on hotplugged CPUs

In reply to David Hildenbrand

We must copy both, the epoch and the epoch_idx.

---
 arch/s390/kvm/kvm-s390.c | 1 +
 1 file changed, 1 insertion(+)
arch/s390/kvm/kvm-s390.c +1 -0
index b514ee427077..d007b737cd4d 100644
@@ -2387,6 +2387,7 @@ void kvm_arch_vcpu_postcreate(struct kvm_vcpu *vcpu)
 	mutex_lock(&vcpu->kvm->lock);
 	preempt_disable();
 	vcpu->arch.sie_block->epoch = vcpu->kvm->arch.epoch;
+	vcpu->arch.sie_block->epdx = vcpu->kvm->arch.epdx;
 	preempt_enable();
 	mutex_unlock(&vcpu->kvm->lock);
 	if (!kvm_is_ucontrol(vcpu->kvm)) {
Signature · 1 line
2.14.3
Reviewed-by on v1 3/6
#15 lore raw

Re: [PATCH RFC 3/6] KVM: s390: consider epoch index on hotplugged CPUs

Quoted 12 lines · On Wed, 7 Feb 2018 12:46:44 +0100 David Hildenbrand <[email protected]> wrote:

We must copy both, the epoch and the epoch_idx.

---
 arch/s390/kvm/kvm-s390.c | 1 +
 1 file changed, 1 insertion(+)
arch/s390/kvm/kvm-s390.c +0 -0
index b514ee427077..d007b737cd4d 100644

arch/s390/kvm/kvm-s390.c void kvm_arch_vcpu_postcreate(struct kvm_vcpu *vcpu) · line 2393

Quoted 7 lines
arch/s390/kvm/kvm-s390.c +1 -0
@@ -2387,6 +2387,7 @@ void kvm_arch_vcpu_postcreate(struct kvm_vcpu *vcpu)
 	mutex_lock(&vcpu->kvm->lock);
 	preempt_disable();
 	vcpu->arch.sie_block->epoch = vcpu->kvm->arch.epoch;
+	vcpu->arch.sie_block->epdx = vcpu->kvm->arch.epdx;
 	preempt_enable();
 	mutex_unlock(&vcpu->kvm->lock);
 	if (!kvm_is_ucontrol(vcpu->kvm)) {
#16 lore raw

Re: [PATCH RFC 3/6] KVM: s390: consider epoch index on hotplugged CPUs

In reply to David Hildenbrand
Quoted 3 lines · David Hildenbrand

We must copy both, the epoch and the epoch_idx.

Thanks applied. Deserves a cc stable I think.

Quoted 17 lines
---
 arch/s390/kvm/kvm-s390.c | 1 +
 1 file changed, 1 insertion(+)
arch/s390/kvm/kvm-s390.c +1 -0
index b514ee427077..d007b737cd4d 100644
@@ -2387,6 +2387,7 @@ void kvm_arch_vcpu_postcreate(struct kvm_vcpu *vcpu)
 	mutex_lock(&vcpu->kvm->lock);
 	preempt_disable();
 	vcpu->arch.sie_block->epoch = vcpu->kvm->arch.epoch;
+	vcpu->arch.sie_block->epdx = vcpu->kvm->arch.epdx;
 	preempt_enable();
 	mutex_unlock(&vcpu->kvm->lock);
 	if (!kvm_is_ucontrol(vcpu->kvm)) {
maintainer Applied
#17 lore raw

Patch "KVM: s390: consider epoch index on hotplugged CPUs" has been added to the 4.14-stable tree

In reply to David Hildenbrand

This is a note to let you know that I've just added the patch titled

KVM: s390: consider epoch index on hotplugged CPUs

to the 4.14-stable tree which can be found at:
http://www.kernel.org/git/?p=linux/kernel/git/stable/stable-queue.git;a=summary

The filename of the patch is:
kvm-s390-consider-epoch-index-on-hotplugged-cpus.patch
and it can be found in the queue-4.14 subdirectory.

If you, or anyone else, feels it should not be added to the stable tree, please let <[email protected]> know about it.

Quoted 1 line

From d16b52cb9cdb6f06dea8ab2f0a428e7d7f0b0a81 Mon Sep 17 00:00:00 2001

From: David Hildenbrand <[email protected]>
Date: Wed, 7 Feb 2018 12:46:44 +0100
Subject: KVM: s390: consider epoch index on hotplugged CPUs

From: David Hildenbrand <[email protected]>

commit d16b52cb9cdb6f06dea8ab2f0a428e7d7f0b0a81 upstream.

We must copy both, the epoch and the epoch_idx.

Message-Id: <[email protected]>

Fixes 8fa1696ea781 ("KVM: s390: Multiple Epoch Facility support")
Fixes 8fa1696ea781 ("KVM: s390: Multiple Epoch Facility support")
---
 arch/s390/kvm/kvm-s390.c |    1 +
 1 file changed, 1 insertion(+)
arch/s390/kvm/kvm-s390.c +1 -0
@@ -2357,6 +2357,7 @@ void kvm_arch_vcpu_postcreate(struct kvm
 	mutex_lock(&vcpu->kvm->lock);
 	preempt_disable();
 	vcpu->arch.sie_block->epoch = vcpu->kvm->arch.epoch;
+	vcpu->arch.sie_block->epdx = vcpu->kvm->arch.epdx;
 	preempt_enable();
 	mutex_unlock(&vcpu->kvm->lock);
 	if (!kvm_is_ucontrol(vcpu->kvm)) {

Patches currently in stable-queue which might be from [email protected] are

queue-4.14/kvm-s390-take-care-of-clock-comparator-sign-control.patch queue-4.14/kvm-s390-provide-only-a-single-function-for-setting-the-tod-fix-sck.patch queue-4.14/kvm-s390-consider-epoch-index-on-tod-clock-syncs.patch queue-4.14/kvm-s390-consider-epoch-index-on-hotplugged-cpus.patch

maintainer Applied
#18 lore raw

Patch "KVM: s390: consider epoch index on hotplugged CPUs" has been added to the 4.15-stable tree

In reply to David Hildenbrand

This is a note to let you know that I've just added the patch titled

KVM: s390: consider epoch index on hotplugged CPUs

to the 4.15-stable tree which can be found at:
http://www.kernel.org/git/?p=linux/kernel/git/stable/stable-queue.git;a=summary

The filename of the patch is:
kvm-s390-consider-epoch-index-on-hotplugged-cpus.patch
and it can be found in the queue-4.15 subdirectory.

If you, or anyone else, feels it should not be added to the stable tree, please let <[email protected]> know about it.

Quoted 1 line

From d16b52cb9cdb6f06dea8ab2f0a428e7d7f0b0a81 Mon Sep 17 00:00:00 2001

From: David Hildenbrand <[email protected]>
Date: Wed, 7 Feb 2018 12:46:44 +0100
Subject: KVM: s390: consider epoch index on hotplugged CPUs

From: David Hildenbrand <[email protected]>

commit d16b52cb9cdb6f06dea8ab2f0a428e7d7f0b0a81 upstream.

We must copy both, the epoch and the epoch_idx.

Message-Id: <[email protected]>

Fixes 8fa1696ea781 ("KVM: s390: Multiple Epoch Facility support")
Fixes 8fa1696ea781 ("KVM: s390: Multiple Epoch Facility support")
---
 arch/s390/kvm/kvm-s390.c |    1 +
 1 file changed, 1 insertion(+)
arch/s390/kvm/kvm-s390.c +1 -0
@@ -2359,6 +2359,7 @@ void kvm_arch_vcpu_postcreate(struct kvm
 	mutex_lock(&vcpu->kvm->lock);
 	preempt_disable();
 	vcpu->arch.sie_block->epoch = vcpu->kvm->arch.epoch;
+	vcpu->arch.sie_block->epdx = vcpu->kvm->arch.epdx;
 	preempt_enable();
 	mutex_unlock(&vcpu->kvm->lock);
 	if (!kvm_is_ucontrol(vcpu->kvm)) {

Patches currently in stable-queue which might be from [email protected] are

queue-4.15/kvm-s390-take-care-of-clock-comparator-sign-control.patch queue-4.15/kvm-s390-provide-only-a-single-function-for-setting-the-tod-fix-sck.patch queue-4.15/kvm-s390-consider-epoch-index-on-tod-clock-syncs.patch queue-4.15/kvm-s390-consider-epoch-index-on-hotplugged-cpus.patch

#19 lore raw

RFC PATCH 4/6 KVM: s390: consider epoch index on TOD clock syncs

In reply to David Hildenbrand

For now, we don't take care of over/underflows. Especially underflows are critical:

Assume the epoch is currently 0 and we get a sync request for delta=1, meaning the TOD is moved forward by 1 and we have to fix it up by subtracting 1 from the epoch. Right now, this will leave the epoch index untouched, resulting in epoch=-1, epoch_idx=0, which is wrong.

We have to take care of over and underflows, also for the VSIE case. So let's factor out calculation into a separate function.

---
 arch/s390/kvm/kvm-s390.c | 32 +++++++++++++++++++++++++++++---
 1 file changed, 29 insertions(+), 3 deletions(-)
arch/s390/kvm/kvm-s390.c +29 -3
index d007b737cd4d..c2b62379049e 100644
@@ -179,6 +179,28 @@ int kvm_arch_hardware_enable(void)
 static void kvm_gmap_notifier(struct gmap *gmap, unsigned long start,
 			      unsigned long end);
 
+static void kvm_clock_sync_scb(struct kvm_s390_sie_block *scb, u64 delta)
+{
+	u64 delta_idx = 0;
+
+	/*
+	 * The TOD jumps by delta, we have to compensate this by adding
+	 * -delta to the epoch.
+	 */
+	delta = -delta;
+
+	/* sign-extension - we're adding to signed values below */
+	if ((s64)delta < 0)
+		delta_idx = 0xff;
+
+	scb->epoch += delta;
+	if (scb->ecd & ECD_MEF) {
+		scb->epdx += delta_idx;
+		if (scb->epoch < delta)
+			scb->epdx += 1;
+	}
+}
+
 /*
  * This callback is executed during stop_machine(). All CPUs are therefore
  * temporarily stopped. In order not to change guest behavior, we have to
@@ -194,13 +216,17 @@ static int kvm_clock_sync(struct notifier_block *notifier, unsigned long val,
 	unsigned long long *delta = v;
 
 	list_for_each_entry(kvm, &vm_list, vm_list) {
-		kvm->arch.epoch -= *delta;
 		kvm_for_each_vcpu(i, vcpu, kvm) {
-			vcpu->arch.sie_block->epoch -= *delta;
+			kvm_clock_sync_scb(vcpu->arch.sie_block, *delta);
+			if (i == 0) {
+				kvm->arch.epoch = vcpu->arch.sie_block->epoch;
+				kvm->arch.epdx = vcpu->arch.sie_block->epdx;
+			}
 			if (vcpu->arch.cputm_enabled)
 				vcpu->arch.cputm_start += *delta;
 			if (vcpu->arch.vsie_block)
-				vcpu->arch.vsie_block->epoch -= *delta;
+				kvm_clock_sync_scb(vcpu->arch.vsie_block,
+						   *delta);
 		}
 	}
 	return NOTIFY_OK;
Signature · 1 line
2.14.3
#20 lore raw

Re: [PATCH RFC 4/6] KVM: s390: consider epoch index on TOD clock syncs

Quoted 36 lines · David Hildenbrand

For now, we don't take care of over/underflows. Especially underflows are critical:

Assume the epoch is currently 0 and we get a sync request for delta=1, meaning the TOD is moved forward by 1 and we have to fix it up by subtracting 1 from the epoch. Right now, this will leave the epoch index untouched, resulting in epoch=-1, epoch_idx=0, which is wrong.

We have to take care of over and underflows, also for the VSIE case. So let's factor out calculation into a separate function.

---
  arch/s390/kvm/kvm-s390.c | 32 +++++++++++++++++++++++++++++---
  1 file changed, 29 insertions(+), 3 deletions(-)
arch/s390/kvm/kvm-s390.c +12 -0
index d007b737cd4d..c2b62379049e 100644
@@ -179,6 +179,28 @@ int kvm_arch_hardware_enable(void)
  static void kvm_gmap_notifier(struct gmap *gmap, unsigned long start,
  			      unsigned long end);
 
+static void kvm_clock_sync_scb(struct kvm_s390_sie_block *scb, u64 delta)
+{
+	u64 delta_idx = 0;
+
+	/*
+	 * The TOD jumps by delta, we have to compensate this by adding
+	 * -delta to the epoch.
+	 */
+	delta = -delta;
+
+	/* sign-extension - we're adding to signed values below */
+	if ((s64)delta < 0)

arch/s390/kvm/kvm-s390.c int kvm_arch_hardware_enable(void) · line 203

Quoted 10 lines
arch/s390/kvm/kvm-s390.c +10 -0
+		delta_idx = 0xff;
+
+	scb->epoch += delta;
+	if (scb->ecd & ECD_MEF) {
+		scb->epdx += delta_idx;
+		if (scb->epoch < delta)
+			scb->epdx += 1;
+	}
+}
+

Is the sync always a jump forward? Do we need to worry about a borrow from the epdx in case of underflow?

Quoted 4 lines
arch/s390/kvm/kvm-s390.c +0 -0
  /*
   * This callback is executed during stop_machine(). All CPUs are therefore
   * temporarily stopped. In order not to change guest behavior, we have to

arch/s390/kvm/kvm-s390.c static int kvm_clock_sync(struct notifier_block *notifier, unsigned long val, · line 223

Quoted 10 lines
arch/s390/kvm/kvm-s390.c +4 -2
@@ -194,13 +216,17 @@ static int kvm_clock_sync(struct notifier_block *notifier, unsigned long val,
  	unsigned long long *delta = v;
 
  	list_for_each_entry(kvm, &vm_list, vm_list) {
-		kvm->arch.epoch -= *delta;
  		kvm_for_each_vcpu(i, vcpu, kvm) {
-			vcpu->arch.sie_block->epoch -= *delta;
+			kvm_clock_sync_scb(vcpu->arch.sie_block, *delta);
+			if (i == 0) {
+				kvm->arch.epoch = vcpu->arch.sie_block->epoch;
+				kvm->arch.epdx = vcpu->arch.sie_block->epdx;

Are we safe by setting the kvm epochs to the sie epochs wrt migration?

Quoted 10 lines
arch/s390/kvm/kvm-s390.c +3 -1
+			}
  			if (vcpu->arch.cputm_enabled)
  				vcpu->arch.cputm_start += *delta;
  			if (vcpu->arch.vsie_block)
-				vcpu->arch.vsie_block->epoch -= *delta;
+				kvm_clock_sync_scb(vcpu->arch.vsie_block,
+						   *delta);
  		}
  	}
  	return NOTIFY_OK;
Signature · 1 line
- Collin L Walling
#21 lore raw

Re: [PATCH RFC 4/6] KVM: s390: consider epoch index on TOD clock syncs

Quoted 42 lines · Collin L. Walling
Quoted 41 lines · David Hildenbrand

For now, we don't take care of over/underflows. Especially underflows are critical:

Assume the epoch is currently 0 and we get a sync request for delta=1, meaning the TOD is moved forward by 1 and we have to fix it up by subtracting 1 from the epoch. Right now, this will leave the epoch index untouched, resulting in epoch=-1, epoch_idx=0, which is wrong.

We have to take care of over and underflows, also for the VSIE case. So let's factor out calculation into a separate function.

---
  arch/s390/kvm/kvm-s390.c | 32 +++++++++++++++++++++++++++++---
  1 file changed, 29 insertions(+), 3 deletions(-)
arch/s390/kvm/kvm-s390.c +17 -0
index d007b737cd4d..c2b62379049e 100644
@@ -179,6 +179,28 @@ int kvm_arch_hardware_enable(void)
  static void kvm_gmap_notifier(struct gmap *gmap, unsigned long start,
  			      unsigned long end);
 
+static void kvm_clock_sync_scb(struct kvm_s390_sie_block *scb, u64 delta)
+{
+	u64 delta_idx = 0;
+
+	/*
+	 * The TOD jumps by delta, we have to compensate this by adding
+	 * -delta to the epoch.
+	 */
+	delta = -delta;
+
+	/* sign-extension - we're adding to signed values below */
+	if ((s64)delta < 0)
+		delta_idx = 0xff;
+
+	scb->epoch += delta;
+	if (scb->ecd & ECD_MEF) {
+		scb->epdx += delta_idx;
Quoted 8 lines

arch/s390/kvm/kvm-s390.c int kvm_arch_hardware_enable(void) · line 203

Quoted 5 lines
arch/s390/kvm/kvm-s390.c +5 -0
+		if (scb->epoch < delta)
+			scb->epdx += 1;
+	}
+}
+

Is the sync always a jump forward? Do we need to worry about a borrow from the epdx in case of underflow?

I can jump forward and backwards, so delta can be positive or negative, resulting in -delta being positive or negative.

The rules of unsigned addition should make sure that all cases are covered. (I tried to find a counter example but wasn't able to find one)

(Especially, this is the same code pattern as found in arch/s390/kvm/vsie.c:register_shadow_scb(), which also adds two signed numbers.)

Quoted 7 lines
Quoted 7 lines
arch/s390/kvm/kvm-s390.c +0 -0
  /*
   * This callback is executed during stop_machine(). All CPUs are therefore
   * temporarily stopped. In order not to change guest behavior, we have to
@@ -194,13 +216,17 @@ static int kvm_clock_sync(struct notifier_block *notifier, unsigned long val,
  	unsigned long long *delta = v;
 
Quoted 10 lines

arch/s390/kvm/kvm-s390.c static int kvm_clock_sync(struct notifier_block *notifier, unsigned long val, · line 223

Quoted 8 lines
arch/s390/kvm/kvm-s390.c +4 -2
  	list_for_each_entry(kvm, &vm_list, vm_list) {
-		kvm->arch.epoch -= *delta;
  		kvm_for_each_vcpu(i, vcpu, kvm) {
-			vcpu->arch.sie_block->epoch -= *delta;
+			kvm_clock_sync_scb(vcpu->arch.sie_block, *delta);
+			if (i == 0) {
+				kvm->arch.epoch = vcpu->arch.sie_block->epoch;
+				kvm->arch.epdx = vcpu->arch.sie_block->epdx;

Are we safe by setting the kvm epochs to the sie epochs wrt migration?

Yes, in fact they should be the same for all VCPUs, otherwise we are in trouble. The TOD has to be the same over all VCPUs.

So we should always have
- kvm->arch.epoch == vcpu->arch.sie_block->epoch
- kvm->arch.epdx == vcpu->arch.sie_block->epdx
for all VCPUs, otherwise their TOD could differ.

This is also guaranteed by the way we calculate and update these numbers.

The only special case is if somebody would be using set_on_reg/get_one_reg with KVM_REG_S390_EPOCHDIFF, setting different values - very unlikely and bad.

Quoted 12 lines
Quoted 11 lines
arch/s390/kvm/kvm-s390.c +3 -1
+			}
  			if (vcpu->arch.cputm_enabled)
  				vcpu->arch.cputm_start += *delta;
  			if (vcpu->arch.vsie_block)
-				vcpu->arch.vsie_block->epoch -= *delta;
+				kvm_clock_sync_scb(vcpu->arch.vsie_block,
+						   *delta);
  		}
  	}
  	return NOTIFY_OK;
Signature · 3 lines
Thanks,

David / dhildenb
#22 lore raw

Re: [PATCH RFC 4/6] KVM: s390: consider epoch index on TOD clock syncs

Quoted 50 lines · David Hildenbrand
Quoted 41 lines · Collin L. Walling
Quoted 40 lines · David Hildenbrand

For now, we don't take care of over/underflows. Especially underflows are critical:

Assume the epoch is currently 0 and we get a sync request for delta=1, meaning the TOD is moved forward by 1 and we have to fix it up by subtracting 1 from the epoch. Right now, this will leave the epoch index untouched, resulting in epoch=-1, epoch_idx=0, which is wrong.

We have to take care of over and underflows, also for the VSIE case. So let's factor out calculation into a separate function.

---
   arch/s390/kvm/kvm-s390.c | 32 +++++++++++++++++++++++++++++---
   1 file changed, 29 insertions(+), 3 deletions(-)
arch/s390/kvm/kvm-s390.c +16 -0
index d007b737cd4d..c2b62379049e 100644
@@ -179,6 +179,28 @@ int kvm_arch_hardware_enable(void)
   static void kvm_gmap_notifier(struct gmap *gmap, unsigned long start,
   			      unsigned long end);
 
+static void kvm_clock_sync_scb(struct kvm_s390_sie_block *scb, u64 delta)
+{
+	u64 delta_idx = 0;
+
+	/*
+	 * The TOD jumps by delta, we have to compensate this by adding
+	 * -delta to the epoch.
+	 */
+	delta = -delta;
+
+	/* sign-extension - we're adding to signed values below */
+	if ((s64)delta < 0)
+		delta_idx = 0xff;
+
+	scb->epoch += delta;
+	if (scb->ecd & ECD_MEF) {
Quoted 8 lines

arch/s390/kvm/kvm-s390.c int kvm_arch_hardware_enable(void) · line 203

Quoted 6 lines
arch/s390/kvm/kvm-s390.c +6 -0
+		scb->epdx += delta_idx;
+		if (scb->epoch < delta)
+			scb->epdx += 1;
+	}
+}
+

Is the sync always a jump forward? Do we need to worry about a borrow from the epdx in case of underflow?

Quoted 5 lines

I can jump forward and backwards, so delta can be positive or negative, resulting in -delta being positive or negative.

The rules of unsigned addition should make sure that all cases are covered. (I tried to find a counter example but wasn't able to find one)

Agreed. I just wrote down a few edge cases myself... it seems to check out nicely.

Quoted 20 lines

(Especially, this is the same code pattern as found in arch/s390/kvm/vsie.c:register_shadow_scb(), which also adds two signed numbers.)

Quoted 6 lines
Quoted 6 lines
arch/s390/kvm/kvm-s390.c +0 -0
   /*
    * This callback is executed during stop_machine(). All CPUs are therefore
    * temporarily stopped. In order not to change guest behavior, we have to
@@ -194,13 +216,17 @@ static int kvm_clock_sync(struct notifier_block *notifier, unsigned long val,
   	unsigned long long *delta = v;
 
Quoted 9 lines

arch/s390/kvm/kvm-s390.c static int kvm_clock_sync(struct notifier_block *notifier, unsigned long val, · line 223

Quoted 8 lines
arch/s390/kvm/kvm-s390.c +4 -2
   	list_for_each_entry(kvm, &vm_list, vm_list) {
-		kvm->arch.epoch -= *delta;
   		kvm_for_each_vcpu(i, vcpu, kvm) {
-			vcpu->arch.sie_block->epoch -= *delta;
+			kvm_clock_sync_scb(vcpu->arch.sie_block, *delta);
+			if (i == 0) {
+				kvm->arch.epoch = vcpu->arch.sie_block->epoch;
+				kvm->arch.epdx = vcpu->arch.sie_block->epdx;

Are we safe by setting the kvm epochs to the sie epochs wrt migration?

Quoted 7 lines

Yes, in fact they should be the same for all VCPUs, otherwise we are in trouble. The TOD has to be the same over all VCPUs.

So we should always have
- kvm->arch.epoch == vcpu->arch.sie_block->epoch
- kvm->arch.epdx == vcpu->arch.sie_block->epdx
for all VCPUs, otherwise their TOD could differ.

Perhaps then this could be shortened to calculate the epochs only once, then set each vcpu to those values instead ofcalculating on each iteration?

I imagine the number of iterations would never be large enough to cause any considerable performance hits, though.

Quoted 18 lines

This is also guaranteed by the way we calculate and update these numbers.

The only special case is if somebody would be using set_on_reg/get_one_reg with KVM_REG_S390_EPOCHDIFF, setting different values - very unlikely and bad.

Quoted 10 lines
Quoted 10 lines
arch/s390/kvm/kvm-s390.c +3 -1
+			}
   			if (vcpu->arch.cputm_enabled)
   				vcpu->arch.cputm_start += *delta;
   			if (vcpu->arch.vsie_block)
-				vcpu->arch.vsie_block->epoch -= *delta;
+				kvm_clock_sync_scb(vcpu->arch.vsie_block,
+						   *delta);
   		}
   	}
   	return NOTIFY_OK;
Signature · 1 line
- Collin L Walling
#23 lore raw

Re: [PATCH RFC 4/6] KVM: s390: consider epoch index on TOD clock syncs

Quoted 29 lines
Quoted 2 lines

The rules of unsigned addition should make sure that all cases are covered. (I tried to find a counter example but wasn't able to find one)

Agreed. I just wrote down a few edge cases myself... it seems to check out nicely.

Quoted 23 lines

(Especially, this is the same code pattern as found in arch/s390/kvm/vsie.c:register_shadow_scb(), which also adds two signed numbers.)

Quoted 6 lines
Quoted 6 lines

/*
* This callback is executed during stop_machine(). All CPUs are therefore
* temporarily stopped. In order not to change guest behavior, we have to

arch/s390/kvm/kvm-s390.c +0 -0
@@ -194,13 +216,17 @@ static int kvm_clock_sync(struct notifier_block *notifier, unsigned long val,
   	unsigned long long *delta = v;
 
Quoted 9 lines

arch/s390/kvm/kvm-s390.c static int kvm_clock_sync(struct notifier_block *notifier, unsigned long val, · line 223

Quoted 8 lines
arch/s390/kvm/kvm-s390.c +4 -2
   	list_for_each_entry(kvm, &vm_list, vm_list) {
-		kvm->arch.epoch -= *delta;
   		kvm_for_each_vcpu(i, vcpu, kvm) {
-			vcpu->arch.sie_block->epoch -= *delta;
+			kvm_clock_sync_scb(vcpu->arch.sie_block, *delta);
+			if (i == 0) {
+				kvm->arch.epoch = vcpu->arch.sie_block->epoch;
+				kvm->arch.epdx = vcpu->arch.sie_block->epdx;

Are we safe by setting the kvm epochs to the sie epochs wrt migration?

Yes, in fact they should be the same for all VCPUs, otherwise we are in trouble. The TOD has to be the same over all VCPUs.

Quoted 9 lines
Quoted 4 lines

So we should always have
- kvm->arch.epoch == vcpu->arch.sie_block->epoch
- kvm->arch.epdx == vcpu->arch.sie_block->epdx
for all VCPUs, otherwise their TOD could differ.

Perhaps then this could be shortened to calculate the epochs only once, then set
each vcpu to those values instead ofcalculating on each iteration?

I had that before, but changed it to this. Especially because some weird user space could set the epochs differently on different CPUs (e.g. for testing purposes or IDK).

So something like this is not shorter and possibly performs less calculations

        list_for_each_entry(kvm, &vm_list, vm_list) {
                kvm_for_each_vcpu(i, vcpu, kvm) {
-                       kvm_clock_sync_scb(vcpu->arch.sie_block, *delta);
                        if (i == 0) {
+                               kvm_clock_sync_scb(vcpu->arch.sie_block, *delta);
                                kvm->arch.epoch = vcpu->arch.sie_block->epoch;
                                kvm->arch.epdx = vcpu->arch.sie_block->epdx;
+                       } else {
+                               vcpu->arch.sie_block->epoch = kvm->arch.epoch;
+                               vcpu->arch.sie_block->epdx = kvm->arch.epdx;
                        }
                        if (vcpu->arch.cputm_enabled)
                                vcpu->arch.cputm_start += *delta;

I'll let the Maintainers decide :)

Quoted 2 lines

I imagine the number of iterations would never be large enough to cause any considerable performance hits, though.

Thanks!

Signature · 3 lines
Thanks,

David / dhildenb
maintainer Applied
#24 lore raw

Patch "KVM: s390: consider epoch index on TOD clock syncs" has been added to the 4.14-stable tree

In reply to David Hildenbrand

This is a note to let you know that I've just added the patch titled

KVM: s390: consider epoch index on TOD clock syncs

to the 4.14-stable tree which can be found at:
http://www.kernel.org/git/?p=linux/kernel/git/stable/stable-queue.git;a=summary

The filename of the patch is:
kvm-s390-consider-epoch-index-on-tod-clock-syncs.patch
and it can be found in the queue-4.14 subdirectory.

If you, or anyone else, feels it should not be added to the stable tree, please let <[email protected]> know about it.

Quoted 1 line

From 1575767ef3cf5326701d2ae3075b7732cbc855e4 Mon Sep 17 00:00:00 2001

From: David Hildenbrand <[email protected]>
Date: Wed, 7 Feb 2018 12:46:45 +0100
Subject: KVM: s390: consider epoch index on TOD clock syncs

From: David Hildenbrand <[email protected]>

commit 1575767ef3cf5326701d2ae3075b7732cbc855e4 upstream.

For now, we don't take care of over/underflows. Especially underflows are critical:

Assume the epoch is currently 0 and we get a sync request for delta=1, meaning the TOD is moved forward by 1 and we have to fix it up by subtracting 1 from the epoch. Right now, this will leave the epoch index untouched, resulting in epoch=-1, epoch_idx=0, which is wrong.

We have to take care of over and underflows, also for the VSIE case. So let's factor out calculation into a separate function.

Message-Id: <[email protected]>

Fixes 8fa1696ea781 ("KVM: s390: Multiple Epoch Facility support")
[use u8 for idx]
---
 arch/s390/kvm/kvm-s390.c |   32 +++++++++++++++++++++++++++++---
 1 file changed, 29 insertions(+), 3 deletions(-)
arch/s390/kvm/kvm-s390.c +29 -3
@@ -169,6 +169,28 @@ int kvm_arch_hardware_enable(void)
 static void kvm_gmap_notifier(struct gmap *gmap, unsigned long start,
 			      unsigned long end);
 
+static void kvm_clock_sync_scb(struct kvm_s390_sie_block *scb, u64 delta)
+{
+	u8 delta_idx = 0;
+
+	/*
+	 * The TOD jumps by delta, we have to compensate this by adding
+	 * -delta to the epoch.
+	 */
+	delta = -delta;
+
+	/* sign-extension - we're adding to signed values below */
+	if ((s64)delta < 0)
+		delta_idx = -1;
+
+	scb->epoch += delta;
+	if (scb->ecd & ECD_MEF) {
+		scb->epdx += delta_idx;
+		if (scb->epoch < delta)
+			scb->epdx += 1;
+	}
+}
+
 /*
  * This callback is executed during stop_machine(). All CPUs are therefore
  * temporarily stopped. In order not to change guest behavior, we have to
@@ -184,13 +206,17 @@ static int kvm_clock_sync(struct notifie
 	unsigned long long *delta = v;
 
 	list_for_each_entry(kvm, &vm_list, vm_list) {
-		kvm->arch.epoch -= *delta;
 		kvm_for_each_vcpu(i, vcpu, kvm) {
-			vcpu->arch.sie_block->epoch -= *delta;
+			kvm_clock_sync_scb(vcpu->arch.sie_block, *delta);
+			if (i == 0) {
+				kvm->arch.epoch = vcpu->arch.sie_block->epoch;
+				kvm->arch.epdx = vcpu->arch.sie_block->epdx;
+			}
 			if (vcpu->arch.cputm_enabled)
 				vcpu->arch.cputm_start += *delta;
 			if (vcpu->arch.vsie_block)
-				vcpu->arch.vsie_block->epoch -= *delta;
+				kvm_clock_sync_scb(vcpu->arch.vsie_block,
+						   *delta);
 		}
 	}
 	return NOTIFY_OK;

Patches currently in stable-queue which might be from [email protected] are

queue-4.14/kvm-s390-take-care-of-clock-comparator-sign-control.patch queue-4.14/kvm-s390-provide-only-a-single-function-for-setting-the-tod-fix-sck.patch queue-4.14/kvm-s390-consider-epoch-index-on-tod-clock-syncs.patch queue-4.14/kvm-s390-consider-epoch-index-on-hotplugged-cpus.patch

maintainer Applied
#25 lore raw

Patch "KVM: s390: consider epoch index on TOD clock syncs" has been added to the 4.15-stable tree

In reply to David Hildenbrand

This is a note to let you know that I've just added the patch titled

KVM: s390: consider epoch index on TOD clock syncs

to the 4.15-stable tree which can be found at:
http://www.kernel.org/git/?p=linux/kernel/git/stable/stable-queue.git;a=summary

The filename of the patch is:
kvm-s390-consider-epoch-index-on-tod-clock-syncs.patch
and it can be found in the queue-4.15 subdirectory.

If you, or anyone else, feels it should not be added to the stable tree, please let <[email protected]> know about it.

Quoted 1 line

From 1575767ef3cf5326701d2ae3075b7732cbc855e4 Mon Sep 17 00:00:00 2001

From: David Hildenbrand <[email protected]>
Date: Wed, 7 Feb 2018 12:46:45 +0100
Subject: KVM: s390: consider epoch index on TOD clock syncs

From: David Hildenbrand <[email protected]>

commit 1575767ef3cf5326701d2ae3075b7732cbc855e4 upstream.

For now, we don't take care of over/underflows. Especially underflows are critical:

Assume the epoch is currently 0 and we get a sync request for delta=1, meaning the TOD is moved forward by 1 and we have to fix it up by subtracting 1 from the epoch. Right now, this will leave the epoch index untouched, resulting in epoch=-1, epoch_idx=0, which is wrong.

We have to take care of over and underflows, also for the VSIE case. So let's factor out calculation into a separate function.

Message-Id: <[email protected]>

Fixes 8fa1696ea781 ("KVM: s390: Multiple Epoch Facility support")
[use u8 for idx]
---
 arch/s390/kvm/kvm-s390.c |   32 +++++++++++++++++++++++++++++---
 1 file changed, 29 insertions(+), 3 deletions(-)
arch/s390/kvm/kvm-s390.c +29 -3
@@ -166,6 +166,28 @@ int kvm_arch_hardware_enable(void)
 static void kvm_gmap_notifier(struct gmap *gmap, unsigned long start,
 			      unsigned long end);
 
+static void kvm_clock_sync_scb(struct kvm_s390_sie_block *scb, u64 delta)
+{
+	u8 delta_idx = 0;
+
+	/*
+	 * The TOD jumps by delta, we have to compensate this by adding
+	 * -delta to the epoch.
+	 */
+	delta = -delta;
+
+	/* sign-extension - we're adding to signed values below */
+	if ((s64)delta < 0)
+		delta_idx = -1;
+
+	scb->epoch += delta;
+	if (scb->ecd & ECD_MEF) {
+		scb->epdx += delta_idx;
+		if (scb->epoch < delta)
+			scb->epdx += 1;
+	}
+}
+
 /*
  * This callback is executed during stop_machine(). All CPUs are therefore
  * temporarily stopped. In order not to change guest behavior, we have to
@@ -181,13 +203,17 @@ static int kvm_clock_sync(struct notifie
 	unsigned long long *delta = v;
 
 	list_for_each_entry(kvm, &vm_list, vm_list) {
-		kvm->arch.epoch -= *delta;
 		kvm_for_each_vcpu(i, vcpu, kvm) {
-			vcpu->arch.sie_block->epoch -= *delta;
+			kvm_clock_sync_scb(vcpu->arch.sie_block, *delta);
+			if (i == 0) {
+				kvm->arch.epoch = vcpu->arch.sie_block->epoch;
+				kvm->arch.epdx = vcpu->arch.sie_block->epdx;
+			}
 			if (vcpu->arch.cputm_enabled)
 				vcpu->arch.cputm_start += *delta;
 			if (vcpu->arch.vsie_block)
-				vcpu->arch.vsie_block->epoch -= *delta;
+				kvm_clock_sync_scb(vcpu->arch.vsie_block,
+						   *delta);
 		}
 	}
 	return NOTIFY_OK;

Patches currently in stable-queue which might be from [email protected] are

queue-4.15/kvm-s390-take-care-of-clock-comparator-sign-control.patch queue-4.15/kvm-s390-provide-only-a-single-function-for-setting-the-tod-fix-sck.patch queue-4.15/kvm-s390-consider-epoch-index-on-tod-clock-syncs.patch queue-4.15/kvm-s390-consider-epoch-index-on-hotplugged-cpus.patch

#26 lore raw

RFC PATCH 5/6 KVM: s390: no need to inititalize kvm->arch members to 0

In reply to David Hildenbrand

KVM is allocated with kzalloc(), so these members are already 0.

---
 arch/s390/kvm/kvm-s390.c | 8 --------
 1 file changed, 8 deletions(-)
arch/s390/kvm/kvm-s390.c +0 -8
index c2b62379049e..0c0eed7d60a8 100644
@@ -1972,8 +1972,6 @@ int kvm_arch_init_vm(struct kvm *kvm, unsigned long type)
 	for (i = 0; i < S390_ARCH_FAC_LIST_SIZE_U64; i++) {
 		if (i < kvm_s390_fac_list_mask_size())
 			kvm->arch.model.fac_mask[i] &= kvm_s390_fac_list_mask[i];
-		else
-			kvm->arch.model.fac_mask[i] = 0UL;
 	}
 
 	/* Populate the facility list initially. */
@@ -1998,8 +1996,6 @@ int kvm_arch_init_vm(struct kvm *kvm, unsigned long type)
 	kvm_s390_crypto_init(kvm);
 
 	mutex_init(&kvm->arch.float_int.ais_lock);
-	kvm->arch.float_int.simm = 0;
-	kvm->arch.float_int.nimm = 0;
 	spin_lock_init(&kvm->arch.float_int.lock);
 	for (i = 0; i < FIRQ_LIST_COUNT; i++)
 		INIT_LIST_HEAD(&kvm->arch.float_int.lists[i]);
@@ -2025,10 +2021,6 @@ int kvm_arch_init_vm(struct kvm *kvm, unsigned long type)
 		kvm->arch.gmap->pfault_enabled = 0;
 	}
 
-	kvm->arch.css_support = 0;
-	kvm->arch.use_irqchip = 0;
-	kvm->arch.epoch = 0;
-
 	spin_lock_init(&kvm->arch.start_stop_lock);
 	kvm_s390_vsie_init(kvm);
 	kvm_s390_gisa_init(kvm);
Signature · 1 line
2.14.3
Reviewed-by on v1 5/6
#27 lore raw

Re: [PATCH RFC 5/6] KVM: s390: no need to inititalize kvm->arch members to 0

Quoted 6 lines · On Wed, 7 Feb 2018 12:46:46 +0100 David Hildenbrand <[email protected]> wrote:

KVM is allocated with kzalloc(), so these members are already 0.

---
 arch/s390/kvm/kvm-s390.c | 8 --------
 1 file changed, 8 deletions(-)

use_esca is also initialized to 0, but we'd lose the comment.

#28 lore raw

RFC PATCH 6/6 KVM: s390: generalize kvm_s390_get_tod_clock_ext()

In reply to David Hildenbrand

Move the Multiple-epoch facility handling into it and rename it to kvm_s390_get_tod_clock().

---
 arch/s390/kvm/kvm-s390.c | 22 +++++++++-------------
 1 file changed, 9 insertions(+), 13 deletions(-)
arch/s390/kvm/kvm-s390.c +9 -13
index 0c0eed7d60a8..df452b8b4f26 100644
@@ -990,8 +990,8 @@ static int kvm_s390_set_tod(struct kvm *kvm, struct kvm_device_attr *attr)
 	return ret;
 }
 
-static void kvm_s390_get_tod_clock_ext(struct kvm *kvm,
-					struct kvm_s390_vm_tod_clock *gtod)
+static void kvm_s390_get_tod_clock(struct kvm *kvm,
+				   struct kvm_s390_vm_tod_clock *gtod)
 {
 	struct kvm_s390_tod_clock_ext htod;
 
@@ -1000,10 +1000,12 @@ static void kvm_s390_get_tod_clock_ext(struct kvm *kvm,
 	get_tod_clock_ext((char *)&htod);
 
 	gtod->tod = htod.tod + kvm->arch.epoch;
-	gtod->epoch_idx = htod.epoch_idx + kvm->arch.epdx;
-
-	if (gtod->tod < htod.tod)
-		gtod->epoch_idx += 1;
+	gtod->epoch_idx = 0;
+	if (test_kvm_facility(kvm, 139)) {
+		gtod->epoch_idx = htod.epoch_idx + kvm->arch.epdx;
+		if (gtod->tod < htod.tod)
+			gtod->epoch_idx += 1;
+	}
 
 	preempt_enable();
 }
@@ -1012,13 +1014,7 @@ static int kvm_s390_get_tod_ext(struct kvm *kvm, struct kvm_device_attr *attr)
 {
 	struct kvm_s390_vm_tod_clock gtod;
 
-	memset(&gtod, 0, sizeof(gtod));
-
-	if (test_kvm_facility(kvm, 139))
-		kvm_s390_get_tod_clock_ext(kvm, &gtod);
-	else
-		gtod.tod = kvm_s390_get_tod_clock_fast(kvm);
-
+	kvm_s390_get_tod_clock(kvm, &gtod);
 	if (copy_to_user((void __user *)attr->addr, &gtod, sizeof(gtod)))
 		return -EFAULT;
 
Signature · 1 line
2.14.3
#29 lore raw

Re: [PATCH RFC 6/6] KVM: s390: generalize kvm_s390_get_tod_clock_ext()

Quoted 7 lines · On Wed, 7 Feb 2018 12:46:47 +0100 David Hildenbrand <[email protected]> wrote:

Move the Multiple-epoch facility handling into it and rename it to kvm_s390_get_tod_clock().

---
 arch/s390/kvm/kvm-s390.c | 22 +++++++++-------------
 1 file changed, 9 insertions(+), 13 deletions(-)

Looks correct, but I'm not sure what this buys us?

Quoted 48 lines
arch/s390/kvm/kvm-s390.c +9 -13
index 0c0eed7d60a8..df452b8b4f26 100644
@@ -990,8 +990,8 @@ static int kvm_s390_set_tod(struct kvm *kvm, struct kvm_device_attr *attr)
 	return ret;
 }
 
-static void kvm_s390_get_tod_clock_ext(struct kvm *kvm,
-					struct kvm_s390_vm_tod_clock *gtod)
+static void kvm_s390_get_tod_clock(struct kvm *kvm,
+				   struct kvm_s390_vm_tod_clock *gtod)
 {
 	struct kvm_s390_tod_clock_ext htod;
 
@@ -1000,10 +1000,12 @@ static void kvm_s390_get_tod_clock_ext(struct kvm *kvm,
 	get_tod_clock_ext((char *)&htod);
 
 	gtod->tod = htod.tod + kvm->arch.epoch;
-	gtod->epoch_idx = htod.epoch_idx + kvm->arch.epdx;
-
-	if (gtod->tod < htod.tod)
-		gtod->epoch_idx += 1;
+	gtod->epoch_idx = 0;
+	if (test_kvm_facility(kvm, 139)) {
+		gtod->epoch_idx = htod.epoch_idx + kvm->arch.epdx;
+		if (gtod->tod < htod.tod)
+			gtod->epoch_idx += 1;
+	}
 
 	preempt_enable();
 }
@@ -1012,13 +1014,7 @@ static int kvm_s390_get_tod_ext(struct kvm *kvm, struct kvm_device_attr *attr)
 {
 	struct kvm_s390_vm_tod_clock gtod;
 
-	memset(&gtod, 0, sizeof(gtod));
-
-	if (test_kvm_facility(kvm, 139))
-		kvm_s390_get_tod_clock_ext(kvm, &gtod);
-	else
-		gtod.tod = kvm_s390_get_tod_clock_fast(kvm);
-
+	kvm_s390_get_tod_clock(kvm, &gtod);
 	if (copy_to_user((void __user *)attr->addr, &gtod, sizeof(gtod)))
 		return -EFAULT;
 
#30 lore raw

Re: [PATCH RFC 6/6] KVM: s390: generalize kvm_s390_get_tod_clock_ext()

Quoted 6 lines · Cornelia Huck
Quoted 3 lines · On Wed, 7 Feb 2018 12:46:47 +0100 David Hildenbrand <[email protected]> wrote:

Move the Multiple-epoch facility handling into it and rename it to kvm_s390_get_tod_clock().

Quoted 6 lines
Quoted 4 lines
---
 arch/s390/kvm/kvm-s390.c | 22 +++++++++-------------
 1 file changed, 9 insertions(+), 13 deletions(-)

Looks correct, but I'm not sure what this buys us?

That we have functions that can be called without having to care about multiple epoch facility

Namely

kvm_s390_set_tod_clock()
kvm_s390_get_tod_clock()
kvm_s390_get_tod_clock_fast()

Signature · 3 lines
Thanks,

David / dhildenb
Reviewed-by
#31 lore raw

Re: [PATCH RFC 6/6] KVM: s390: generalize kvm_s390_get_tod_clock_ext()

Quoted 14 lines · On Thu, 15 Feb 2018 15:14:37 +0100 David Hildenbrand <[email protected]> wrote:
Quoted 6 lines · Cornelia Huck
Quoted 3 lines · On Wed, 7 Feb 2018 12:46:47 +0100 David Hildenbrand <[email protected]> wrote:

Move the Multiple-epoch facility handling into it and rename it to kvm_s390_get_tod_clock().

Quoted 6 lines
Quoted 4 lines
---
 arch/s390/kvm/kvm-s390.c | 22 +++++++++-------------
 1 file changed, 9 insertions(+), 13 deletions(-)  

Looks correct, but I'm not sure what this buys us?

Quoted 9 lines

That we have functions that can be called without having to care about multiple epoch facility

Namely

kvm_s390_set_tod_clock()
kvm_s390_get_tod_clock()
kvm_s390_get_tod_clock_fast()

OK, that makes sense. Maybe add something like that to the patch description?

#32 lore raw

Re: [PATCH RFC 6/6] KVM: s390: generalize kvm_s390_get_tod_clock_ext()

Quoted 27 lines · Cornelia Huck
Quoted 14 lines · On Thu, 15 Feb 2018 15:14:37 +0100 David Hildenbrand <[email protected]> wrote:
Quoted 6 lines · Cornelia Huck
Quoted 3 lines · On Wed, 7 Feb 2018 12:46:47 +0100 David Hildenbrand <[email protected]> wrote:

Move the Multiple-epoch facility handling into it and rename it to kvm_s390_get_tod_clock().

Quoted 6 lines
Quoted 4 lines
---
 arch/s390/kvm/kvm-s390.c | 22 +++++++++-------------
 1 file changed, 9 insertions(+), 13 deletions(-)  

Looks correct, but I'm not sure what this buys us?

Quoted 9 lines

That we have functions that can be called without having to care about multiple epoch facility

Namely

kvm_s390_set_tod_clock()
kvm_s390_get_tod_clock()
kvm_s390_get_tod_clock_fast()

Quoted 5 lines

OK, that makes sense. Maybe add something like that to the patch description?

Sure, can do! Thanks!

Signature · 3 lines
Thanks,

David / dhildenb
#33 lore raw
In reply to David Hildenbrand
Quoted 31 lines · David Hildenbrand

It think I found various BUGs in the recently added Multiple-epoch facility support in KVM. Along with it, two cleanups.

1. The clock-comparator sign control is not considered. I sent a patch with this previously. This now contains fixes and simplifications.
2. SET CLOCK from the guest does not work reliably if the facility is enabled (epoch index not set).
3. Hotplugged CPUs don't inherit the epoch index.
4. TOD clock syncs don't take care of overflows/underflows in the epoch value and miss to update the epoch index.

This is RFC as I have basically no machine to test. Hopefully somebody can jump in and verify that we now handle the epoch index in all scenarios correctly.

Most importantly, with Multiple-epoch facility, the condition
Guest TOD = Host TOD - 1
Is represented by both, epoch and epoch_idx containing 0xff. So it is treated as a 64+8bit signed number - we have to properly take care of over/underflows when modifying the epoch.

David Hildenbrand (6):
KVM: s390: take care of clock-comparator sign control
KVM: s390: provide only a single function for setting the tod
KVM: s390: consider epoch index on hotplugged CPUs
KVM: s390: consider epoch index on TOD clock syncs
KVM: s390: no need to inititalize kvm->arch members to 0
KVM: s390: generalize kvm_s390_get_tod_clock_ext()

Quoted 6 lines
 arch/s390/kvm/interrupt.c |  25 ++++++++---
 arch/s390/kvm/kvm-s390.c  | 109 +++++++++++++++++++++++-----------------------
 arch/s390/kvm/kvm-s390.h  |   5 +--
 arch/s390/kvm/priv.c      |   9 ++--
 4 files changed, 80 insertions(+), 68 deletions(-)

CC'ing Collin (sorry missed you)

Signature · 3 lines
Thanks,

David / dhildenb