s390: Enable CONTEXT_ANALYSIS for cio and virtio
Subsystems (4)
Heiko Carstens
maintainercover s390: Enable CONTEXT_ANALYSIS for cio and virtio
Enable CONTEXT_ANALYSYS for cio and virtio.
Static code checking for acquiring and releasing locks used to be done with sparse. That was removed with [1] and replaced with a clang based approach [2]. The new approach requires that each subsystem needs to be explicitly enabled for checking.
Do that for cio and virtio. Add a __must_hold() attribute to vfio_ccw_sch_quiesce() to address the only valid warning.
Then enable CONTEXT_ANALYSIS for both cio and virtio.
Christian, Claudio, Janosch, I think it would make most sense if this goes via kvms390.
Thanks,
Heiko
[1] 5b63d0ae94cc ("compiler-context-analysis: Remove Sparse support")
[2] 3269701cb256 ("compiler-context-analysis: Add infrastructure for Context Analysis with Clang")
Heiko Carstens (3):
s390/vfio_ccw: Add __must_hold() attribute to vfio_ccw_sch_quiesce() s390/cio: Enable CONTEXT_ANALYSIS
s390/virtio: Enable CONTEXT_ANALYSIS
drivers/s390/cio/Makefile | 2 ++ drivers/s390/cio/vfio_ccw_drv.c | 1 + drivers/s390/cio/vfio_ccw_private.h | 3 ++- drivers/s390/virtio/Makefile | 2 ++ 4 files changed, 7 insertions(+), 1 deletion(-)
Signature · 1 line
2.53.0
Heiko Carstens
maintainerPATCH 1/3 s390/vfio_ccw: Add __must_hold() attribute to vfio_ccw_sch_quiesce()
Add __must_hold() attribute to vfio_ccw_sch_quiesce() in order to let clang's context analysis know that sch->lock must be held on function entry. This can also be easily verified when inspecting the function.
Without this annotation this leads to a valid warning when context analysis is enabled:
drivers/s390/cio/vfio_ccw_drv.c:55:9: warning:
expecting spinlock 'sch->lock' to be held at start of each loop [-Wthread-safety-analysis]
55 | ret = cio_cancel_halt_clear(sch, &iretry);
| ^
--- drivers/s390/cio/vfio_ccw_drv.c | 1 + drivers/s390/cio/vfio_ccw_private.h | 3 ++- 2 files changed, 3 insertions(+), 1 deletion(-)
drivers/s390/cio/vfio_ccw_drv.c +1 -0
@@ -35,6 +35,7 @@ debug_info_t *vfio_ccw_debug_trace_id;* Helpers */ int vfio_ccw_sch_quiesce (struct subchannel *sch) + __must_hold(&sch->lock); {struct vfio_ccw_parent *parent =dev_get_drvdata (&sch->dev);struct vfio_ccw_private *private =dev_get_drvdata (&parent->dev);
drivers/s390/cio/vfio_ccw_private.h +2 -1
@@ -127,7 +127,8 @@ struct vfio_ccw_private {struct work_struct crw_work; }__aligned(8) ; -int vfio_ccw_sch_quiesce (struct subchannel *sch); +int vfio_ccw_sch_quiesce(struct subchannel *sch) + __must_hold(&sch->lock);void vfio_ccw_sch_io_todo (struct work_struct *work);void vfio_ccw_crw_todo (struct work_struct *work);
Signature · 1 line
2.53.0
Heiko Carstens
maintainerPATCH 2/3 s390/cio: Enable CONTEXT_ANALYSIS
In reply to Heiko CarstensAll cio code passes clang's compile time context analysis. Therefore enable CONTEXT_ANALYSIS.
--- drivers/s390/cio/Makefile | 2 ++ 1 file changed, 2 insertions(+)
drivers/s390/cio/Makefile +2 -0
@@ -3,6 +3,8 @@ # Makefile for the S/390 common i/o drivers # +CONTEXT_ANALYSIS := y + # The following is required for define_trace.h to find ./trace.h CFLAGS_trace.o := -I$(src) CFLAGS_vfio_ccw_trace.o := -I$(src)
Signature · 1 line
2.53.0
Heiko Carstens
maintainerPATCH 3/3 s390/virtio: Enable CONTEXT_ANALYSIS
In reply to Heiko CarstensAll virtio code passes clang's compile time context analysis. Therefore enable CONTEXT_ANALYSIS.
--- drivers/s390/virtio/Makefile | 2 ++ 1 file changed, 2 insertions(+)
drivers/s390/virtio/Makefile +2 -0
@@ -3,4 +3,6 @@ # # Copyright IBM Corp. 2008 +CONTEXT_ANALYSIS := y + obj-$(CONFIG_S390_GUEST) += virtio_ccw.o
Signature · 1 line
2.53.0
[email protected]
AI reviewRe: [PATCH 3/3] s390/virtio: Enable CONTEXT_ANALYSIS
Sashiko has reviewed this patch and found no issues. It looks great!
Quoted 4 lines
All virtio code passes clang's compile time context analysis. Therefore enable
CONTEXT_ANALYSIS.Signed-off-by Heiko Carstens <[email protected]>
Sashiko has reviewed this patch and found no issues. It looks great!
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=3
[email protected]
AI reviewRe: [PATCH 2/3] s390/cio: Enable CONTEXT_ANALYSIS
In reply to Heiko CarstensSashiko has reviewed this patch and found no issues. It looks great!
Quoted 4 lines
All cio code passes clang's compile time context analysis. Therefore enable
CONTEXT_ANALYSIS.Signed-off-by Heiko Carstens <[email protected]>
Sashiko has reviewed this patch and found no issues. It looks great!
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=2
[email protected]
AI reviewRe: [PATCH 1/3] s390/vfio_ccw: Add __must_hold() attribute to vfio_ccw_sch_quiesce()
In reply to Heiko CarstensThank you for your contribution! Sashiko AI review found 4 potential issue(s) to consider:
Thank you for your contribution! Sashiko AI review found 4 potential issue(s) to consider:
Pre-existing issues:
- [High] Premature loop termination in vfio_ccw_sch_quiesce due to re-used completion without re-initialization.
- [High] Unconditional NULL pointer dereference in asynchronous subchannel callbacks.
- [High] Leaked workqueue items (io_work and crw_work) during device teardown leading to Use-After-Free.
- [High] Lockless state access in vfio_ccw_sch_io_todo causes a double-free of channel program (cp_free). --
--- Patch [1]: [PATCH 1/3] s390/vfio_ccw: Add __must_hold() attribute to vfio_ccw_sch_quiesce() --- commit c7b4889a75b6cf11527a57bff8438c3c08dac9c4
Author: Heiko Carstens <[email protected]>
Subject: s390/vfio_ccw: Add __must_hold() attribute to vfio_ccw_sch_quiesce()
This patch adds the __must_hold attribute to vfio_ccw_sch_quiesce() to inform context analysis that sch->lock must be held on function entry. This resolves a thread-safety-analysis warning during compilation.
drivers/s390/cio/vfio_ccw_drv.c debug_info_t *vfio_ccw_debug_trace_id; · line 41
Quoted 12 lines
drivers/s390/cio/vfio_ccw_drv.c +1
-0@@ -35,6 +35,7 @@ debug_info_t *vfio_ccw_debug_trace_id;* Helpers */ int vfio_ccw_sch_quiesce (struct subchannel *sch) + __must_hold(&sch->lock); {struct vfio_ccw_parent *parent =dev_get_drvdata (&sch->dev);struct vfio_ccw_private *private =dev_get_drvdata (&parent->dev);
[Severity: High]
This is a pre-existing issue, but does the completion handling in this loop allow premature termination?
If wait_for_completion_timeout() times out, but a late interrupt arrives and calls complete(), the completion's done counter becomes greater than 0.
On the next iteration of the do/while (ret == -EBUSY) loop in vfio_ccw_sch_quiesce():
do {
ret = cio_cancel_halt_clear(sch, &iretry);
...
if (ret == -EBUSY)
wait_for_completion_timeout(&completion, 3*HZ);
...
} while (ret == -EBUSY);Could wait_for_completion_timeout() see done > 0 and return immediately without waiting, causing a tight spin loop calling cio_disable_subchannel() since there is no reinit_completion() inside the loop?
[Severity: High]
This isn't a bug introduced by this patch, but is there a risk of a NULL pointer dereference when fetching the parent pointer?
During device removal, vfio_ccw_sch_remove() sets the drvdata to NULL:dev_set_drvdata(&sch->dev, NULL);
If an asynchronous event like an interrupt triggers concurrently before the CSS core fully detaches the driver, could vfio_ccw_sch_irq() or vfio_ccw_sch_event() fetch a NULL parent and immediately dereference it in this sequence?
struct vfio_ccw_parent *parent = dev_get_drvdata(&sch->dev); struct vfio_ccw_private *private = dev_get_drvdata(&parent->dev);
[ ... ]
drivers/s390/cio/vfio_ccw_private.h struct vfio_ccw_private { · line 128
Quoted 7 lines
drivers/s390/cio/vfio_ccw_private.h +0
-0@@ -127,7 +127,8 @@ struct vfio_ccw_private {struct work_struct crw_work; }__aligned(8) ;
[Severity: High]
This is a pre-existing issue, but does the device teardown path leak these pending work items?
When a mediated device is removed, vfio_ccw_mdev_release_dev() frees the private structure without calling cancel_work_sync() on io_work or crw_work:
static void vfio_ccw_mdev_release_dev(struct vfio_device *vdev)
{
...
kfree(private);
}Additionally, fsm_close() can skip the global flush_workqueue() if cio_disable_subchannel() immediately returns 0:
ret = cio_disable_subchannel(sch);
if (ret == -EBUSY)
ret = vfio_ccw_sch_quiesce(sch);
If a work item is left pending on the workqueue, could it execute after the private structure has been freed, resulting in a use-after-free?
drivers/s390/cio/vfio_ccw_private.h struct vfio_ccw_private { · line 133
Quoted 5 lines
drivers/s390/cio/vfio_ccw_private.h +2
-1-int vfio_ccw_sch_quiesce (struct subchannel *sch); +int vfio_ccw_sch_quiesce(struct subchannel *sch) + __must_hold(&sch->lock);void vfio_ccw_sch_io_todo (struct work_struct *work);void vfio_ccw_crw_todo (struct work_struct *work);
[Severity: High]
This isn't a bug introduced by this patch, but does the lockless state check in vfio_ccw_sch_io_todo() allow a double-free of the channel program?
The workqueue function vfio_ccw_sch_io_todo() reads private->state locklessly:
if (is_final && private->state == VFIO_CCW_STATE_CP_PENDING) {
cp_free(&private->cp);
cp_is_finished = true;
}Concurrently, fsm_close() holds sch->lock, changes the state to STANDBY, unlocks, and also calls cp_free():
spin_unlock_irq(&sch->lock);cp_free(&private->cp);
Since vfio_ccw_sch_io_todo() operates outside sch->lock, could both threads evaluate their conditions simultaneously and concurrently execute cp_free() on the same channel program?
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=1
Heiko Carstens
maintainerRe: [PATCH 1/3] s390/vfio_ccw: Add __must_hold() attribute to vfio_ccw_sch_quiesce()
Eric, Matthew,
not sure if you are aware of the below (nothing I would address). Please handle :)
Quoted 130 lines · On Thu, Aug 06, 2026 at 03:52:23PM +0000, [email protected] wrote:
Thank you for your contribution! Sashiko AI review found 4 potential issue(s) to consider:
Pre-existing issues:
- [High] Premature loop termination invfio_ccw_sch_quiescedue to re-used completion without re-initialization.
- [High] Unconditional NULL pointer dereference in asynchronous subchannel callbacks.
- [High] Leaked workqueue items (io_workandcrw_work) during device teardown leading to Use-After-Free.
- [High] Lockless state access invfio_ccw_sch_io_todocauses a double-free of channel program (cp_free). ----- Patch [1]: [PATCH 1/3] s390/vfio_ccw: Add
__must_hold()attribute tovfio_ccw_sch_quiesce()--- commitc7b4889a75b6cf11527a57bff8438c3c08dac9c4
Author: Heiko Carstens <[email protected]>
Subject: s390/vfio_ccw: Add__must_hold()attribute tovfio_ccw_sch_quiesce()This patch adds the
__must_holdattribute tovfio_ccw_sch_quiesce()to inform context analysis thatsch->lockmust be held on function entry. This resolves a thread-safety-analysis warning during compilation.drivers/s390/cio/vfio_ccw_drv.c debug_info_t *vfio_ccw_debug_trace_id; · line 41
Quoted 12 lines
drivers/s390/cio/vfio_ccw_drv.c +1
-0@@ -35,6 +35,7 @@ debug_info_t *vfio_ccw_debug_trace_id;* Helpers */ int vfio_ccw_sch_quiesce (struct subchannel *sch) + __must_hold(&sch->lock); {struct vfio_ccw_parent *parent =dev_get_drvdata (&sch->dev);struct vfio_ccw_private *private =dev_get_drvdata (&parent->dev);[Severity: High]
This is a pre-existing issue, but does the completion handling in this loop allow premature termination?If
wait_for_completion_timeout()times out, but a late interrupt arrives and callscomplete(), the completion's done counter becomes greater than 0.On the next iteration of the do/while (ret ==
-EBUSY) loop invfio_ccw_sch_quiesce():do { ret = cio_cancel_halt_clear(sch, &iretry); ... if (ret == -EBUSY) wait_for_completion_timeout(&completion, 3*HZ); ... } while (ret == -EBUSY);Could
wait_for_completion_timeout()see done > 0 and return immediately without waiting, causing a tight spin loop callingcio_disable_subchannel()since there is noreinit_completion()inside the loop?[Severity: High]
This isn't a bug introduced by this patch, but is there a risk of a NULL pointer dereference when fetching the parent pointer?During device removal,
vfio_ccw_sch_remove()sets the drvdata to NULL:dev_set_drvdata(&sch->dev, NULL);If an asynchronous event like an interrupt triggers concurrently before the CSS core fully detaches the driver, could
vfio_ccw_sch_irq()orvfio_ccw_sch_event()fetch a NULL parent and immediately dereference it in this sequence?struct
vfio_ccw_parent*parent =dev_get_drvdata(&sch->dev); structvfio_ccw_private*private =dev_get_drvdata(&parent->dev);[ ... ]
drivers/s390/cio/vfio_ccw_private.h struct vfio_ccw_private { · line 128
Quoted 7 lines
drivers/s390/cio/vfio_ccw_private.h +0
-0@@ -127,7 +127,8 @@ struct vfio_ccw_private {struct work_struct crw_work; }__aligned(8) ;[Severity: High]
This is a pre-existing issue, but does the device teardown path leak these pending work items?When a mediated device is removed,
vfio_ccw_mdev_release_dev()frees the private structure without callingcancel_work_sync()onio_workorcrw_work:static void
vfio_ccw_mdev_release_dev(struct vfio_device *vdev){ ... kfree(private); }Additionally,
fsm_close()can skip the globalflush_workqueue()ifcio_disable_subchannel()immediately returns 0:ret =
cio_disable_subchannel(sch);
if (ret ==-EBUSY)ret = vfio_ccw_sch_quiesce(sch);If a work item is left pending on the workqueue, could it execute after the private structure has been freed, resulting in a use-after-free?
drivers/s390/cio/vfio_ccw_private.h struct vfio_ccw_private { · line 133
Quoted 5 lines
drivers/s390/cio/vfio_ccw_private.h +2
-1-int vfio_ccw_sch_quiesce (struct subchannel *sch); +int vfio_ccw_sch_quiesce(struct subchannel *sch) + __must_hold(&sch->lock);void vfio_ccw_sch_io_todo (struct work_struct *work);void vfio_ccw_crw_todo (struct work_struct *work);[Severity: High]
This isn't a bug introduced by this patch, but does the lockless state check invfio_ccw_sch_io_todo()allow a double-free of the channel program?The workqueue function
vfio_ccw_sch_io_todo()readsprivate->statelocklessly:if (is_final && private->state == VFIO_CCW_STATE_CP_PENDING) { cp_free(&private->cp); cp_is_finished = true; }Concurrently,
fsm_close()holdssch->lock, changes the state to STANDBY, unlocks, and also callscp_free():
spin_unlock_irq(&sch->lock);cp_free(&private->cp);Since
vfio_ccw_sch_io_todo()operates outsidesch->lock, could both threads evaluate their conditions simultaneously and concurrently executecp_free()on the same channel program?Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=1
Matthew Rosato
maintainerRe: [PATCH 1/3] s390/vfio_ccw: Add __must_hold() attribute to vfio_ccw_sch_quiesce()
Quoted 4 lines · Heiko Carstens
Eric, Matthew,
not sure if you are aware of the below (nothing I would address). Please handle :)
Thanks. At least some of these are handled by fixes in the following kvm PR which hasn't landed in master yet:
https://lore.kernel.org/linux-s390/[email protected]/
But anyway: ACK - will check w/ Eric, if any of these aren't already addressed we will follow up on them independent of this series.
Thanks,
Matt
Re: [PATCH 1/3] s390/vfio_ccw: Add __must_hold() attribute to vfio_ccw_sch_quiesce()
In reply to Heiko CarstensQuoted 7 lines · Heiko Carstens
Add
__must_hold()attribute tovfio_ccw_sch_quiesce()in order to let clang's context analysis know thatsch->lockmust be held on function entry. This can also be easily verified when inspecting the function.Without this annotation this leads to a valid warning when context analysis is enabled:
Quoted 6 lines
drivers/s390/cio/vfio_ccw_drv.c:55:9: warning:
expecting spinlock 'sch->lock' to be held at start of each loop [-Wthread-safety-analysis]
55 | ret =cio_cancel_halt_clear(sch, &iretry);
| ^Signed-off-by Heiko Carstens <[email protected]>
Re: [PATCH 3/3] s390/virtio: Enable CONTEXT_ANALYSIS
In reply to Heiko CarstensQuoted 4 lines · Heiko Carstens
All virtio code passes clang's compile time context analysis. Therefore enable
CONTEXT_ANALYSIS.Signed-off-by Heiko Carstens <[email protected]>
Re: [PATCH 1/3] s390/vfio_ccw: Add __must_hold() attribute to vfio_ccw_sch_quiesce()
In reply to Heiko CarstensAm 06.08.26 um 17:38 schrieb Heiko Carstens:
Quoted 7 lines
Add
__must_hold()attribute tovfio_ccw_sch_quiesce()in order to let clang's context analysis know thatsch->lockmust be held on function entry. This can also be easily verified when inspecting the function.Without this annotation this leads to a valid warning when context analysis is enabled:
Quoted 6 lines
drivers/s390/cio/vfio_ccw_drv.c:55:9: warning:
expecting spinlock 'sch->lock' to be held at start of each loop [-Wthread-safety-analysis]55 | ret = cio_cancel_halt_clear(sch, &iretry); | ^Signed-off-by Heiko Carstens <[email protected]>
I assume this series should go via the s390 tree with the others?
Quoted 31 lines
--- drivers/s390/cio/vfio_ccw_drv.c | 1 + drivers/s390/cio/vfio_ccw_private.h | 3 ++- 2 files changed, 3 insertions(+), 1 deletion(-)drivers/s390/cio/vfio_ccw_drv.c +1
-0@@ -35,6 +35,7 @@ debug_info_t *vfio_ccw_debug_trace_id;* Helpers */ int vfio_ccw_sch_quiesce (struct subchannel *sch) + __must_hold(&sch->lock); {struct vfio_ccw_parent *parent =dev_get_drvdata (&sch->dev);struct vfio_ccw_private *private =dev_get_drvdata (&parent->dev);drivers/s390/cio/vfio_ccw_private.h +2
-1@@ -127,7 +127,8 @@ struct vfio_ccw_private {struct work_struct crw_work; }__aligned(8) ; -int vfio_ccw_sch_quiesce (struct subchannel *sch); +int vfio_ccw_sch_quiesce(struct subchannel *sch) + __must_hold(&sch->lock);void vfio_ccw_sch_io_todo (struct work_struct *work);void vfio_ccw_crw_todo (struct work_struct *work);
Heiko Carstens
maintainerRe: [PATCH 1/3] s390/vfio_ccw: Add __must_hold() attribute to vfio_ccw_sch_quiesce()
Quoted 15 lines · Christian Borntraeger
Am 06.08.26 um 17:38 schrieb Heiko Carstens:
Quoted 12 lines
Add
__must_hold()attribute tovfio_ccw_sch_quiesce()in order to let clang's context analysis know thatsch->lockmust be held on function entry. This can also be easily verified when inspecting the function.Without this annotation this leads to a valid warning when context analysis is enabled:
drivers/s390/cio/vfio_ccw_drv.c:55:9: warning:
expecting spinlock 'sch->lock' to be held at start of each loop [-Wthread-safety-analysis]55 | ret = cio_cancel_halt_clear(sch, &iretry); | ^
Quoted 5 lines
Quoted 1 line
Signed-off-by Heiko Carstens <[email protected]>I assume this series should go via the s390 tree with the others?
As written in the cover-letter:
"Christian, Claudio, Janosch, I think it would make most sense if this
goes via kvms390."
:)
Christian Borntraeger
reviewerRe: [PATCH 1/3] s390/vfio_ccw: Add __must_hold() attribute to vfio_ccw_sch_quiesce()
In reply to Heiko CarstensAm 06.08.26 um 17:38 schrieb Heiko Carstens:
Quoted 23 lines
Add
__must_hold()attribute tovfio_ccw_sch_quiesce()in order to let clang's context analysis know thatsch->lockmust be held on function entry. This can also be easily verified when inspecting the function.Without this annotation this leads to a valid warning when context analysis is enabled:
drivers/s390/cio/vfio_ccw_drv.c:55:9: warning:
expecting spinlock 'sch->lock' to be held at start of each loop [-Wthread-safety-analysis]55 | ret = cio_cancel_halt_clear(sch, &iretry); | ^Signed-off-by Heiko Carstens <[email protected]>--- drivers/s390/cio/vfio_ccw_drv.c | 1 + drivers/s390/cio/vfio_ccw_private.h | 3 ++- 2 files changed, 3 insertions(+), 1 deletion(-)drivers/s390/cio/vfio_ccw_drv.c +0
-0
drivers/s390/cio/vfio_ccw_drv.c debug_info_t *vfio_ccw_debug_trace_id; · line 38
Quoted 4 lines
drivers/s390/cio/vfio_ccw_drv.c +1
-0@@ -35,6 +35,7 @@ debug_info_t *vfio_ccw_debug_trace_id;* Helpers */ int vfio_ccw_sch_quiesce(struct subchannel *sch) + __must_hold(&sch->lock);
Looks like this ; is wrong ^
will fixup when applying.
Quoted 17 lines
drivers/s390/cio/vfio_ccw_drv.c +0
-0{struct vfio_ccw_parent *parent =dev_get_drvdata (&sch->dev);struct vfio_ccw_private *private =dev_get_drvdata (&parent->dev);drivers/s390/cio/vfio_ccw_private.h +2
-1@@ -127,7 +127,8 @@ struct vfio_ccw_private {struct work_struct crw_work; }__aligned(8) ; -int vfio_ccw_sch_quiesce (struct subchannel *sch); +int vfio_ccw_sch_quiesce(struct subchannel *sch) + __must_hold(&sch->lock);void vfio_ccw_sch_io_todo (struct work_struct *work);void vfio_ccw_crw_todo (struct work_struct *work);
Heiko Carstens
maintainerRe: [PATCH 1/3] s390/vfio_ccw: Add __must_hold() attribute to vfio_ccw_sch_quiesce()
Quoted 5 lines · Christian Borntraeger
Review
Quoted 2 lines
int
vfio_ccw_sch_quiesce(struct subchannel *sch)(quoted diff) +1
-0+ __must_hold(&sch->lock);Looks like this ; is wrong ^
will fixup when applying.
Right. Thanks!
Christian Borntraeger
reviewerAm 06.08.26 um 17:38 schrieb Heiko Carstens:
Quoted 7 lines
Enable
CONTEXT_ANALYSYSfor cio and virtio.Static code checking for acquiring and releasing locks used to be done with sparse. That was removed with [1] and replaced with a clang based approach [2]. The new approach requires that each subsystem needs to be explicitly enabled for checking.
Quoted 7 lines
Do that for cio and virtio. Add a
__must_hold()attribute tovfio_ccw_sch_quiesce()to address the only valid warning.Then enable
CONTEXT_ANALYSISfor both cio and virtio.Christian, Claudio, Janosch, I think it would make most sense if this goes via kvms390.
applied with the small ";" fixup and scheduled for next. Thanks you.