Drivers: hv: vmbus: Miscellaneous enhancements and fixes
Subsystems (2)
cover Drivers: hv: vmbus: Miscellaneous enhancements and fixes
From: "K. Y. Srinivasan" <[email protected]>
Miscellaneous enhancements and fixes.
K. Y. Srinivasan (1):
Drivers: hv: vmbus: Make pannic reporting to be more useful
Stephen Hemminger (2):
Drivers: hv: vmbus: Expose per-channel interrupts and events counters
vmbus: initialize reserved fields in messages
Documentation/ABI/stable/sysfs-bus-vmbus | 14 ++++++++++++++ arch/x86/hyperv/hv_init.c | 11 +++++++---- arch/x86/include/asm/mshyperv.h | 2 +- drivers/hv/channel.c | 2 ++ drivers/hv/connection.c | 2 ++ drivers/hv/vmbus_drv.c | 22 ++++++++++++++++++++-- include/linux/hyperv.h | 4 ++++ 7 files changed, 50 insertions(+), 7 deletions(-)
Signature · 1 line
2.14.1
PATCH 1/3 Drivers: hv: vmbus: Expose per-channel interrupts and events counters
From: Stephen Hemminger <[email protected]>
When investigating performance, it is useful to be able to look at the number of host and guest events per-channel. This is equivalent to per-device interrupt statistics.
--- Documentation/ABI/stable/sysfs-bus-vmbus | 14 ++++++++++++++ drivers/hv/connection.c | 2 ++ drivers/hv/vmbus_drv.c | 18 ++++++++++++++++++ include/linux/hyperv.h | 4 ++++ 4 files changed, 38 insertions(+)
Documentation/ABI/stable/sysfs-bus-vmbus +14 -0
@@ -97,3 +97,17 @@ KernelVersion: 4.14 Contact: Stephen Hemminger <[email protected]> Description: Bytes availabble to write Users: Debuggig tools + +What: /sys/bus/vmbus/devices/vmbus_*/channels/relid/events +Date: September. 2017 +KernelVersion: 4.14 +Contact: Stephen Hemminger <[email protected]> +Description: Number of times we have signaled the host +Users: Debuggig tools + +What: /sys/bus/vmbus/devices/vmbus_*/channels/relid/interrupts +Date: September. 2017 +KernelVersion: 4.14 +Contact: Stephen Hemminger <[email protected]> +Description: Number of times we have taken an interrupt (incoming) +Users: Debuggig tools
drivers/hv/connection.c +2 -0
@@ -409,6 +409,8 @@ void vmbus_set_event(struct vmbus_channel *channel)if (!channel->is_dedicated_interrupt)vmbus_send_interrupt (child_relid); + ++channel->sig_events; +hv_do_fast_hypercall8 (HVCALL_SIGNAL_EVENT , channel->sig_event); }EXPORT_SYMBOL_GPL (vmbus_set_event);
drivers/hv/vmbus_drv.c +18 -0
@@ -945,6 +945,8 @@ static void vmbus_chan_sched(struct hv_per_cpu_context *hv_cpu)if (channel->rescind)continue ; + ++channel->interrupts; +switch (channel->callback_mode) {case HV_CALL_ISR :vmbus_channel_isr (channel);
@@ -1238,6 +1240,20 @@ static ssize_t channel_latency_show(const struct vmbus_channel *channel, }VMBUS_CHAN_ATTR (latency,S_IRUGO , channel_latency_show,NULL ); +static ssize_t channel_interrupts_show (const struct vmbus_channel *channel,char *buf) +{ +return sprintf (buf,"%u\n" , channel->interrupts); +} +VMBUS_CHAN_ATTR (interrupts,S_IRUGO , channel_interrupts_show,NULL ); + +static ssize_t channel_events_show (const struct vmbus_channel *channel,char *buf) +{ +return sprintf (buf,"%u\n" , channel->sig_events); +} +VMBUS_CHAN_ATTR (events,S_IRUGO , channel_events_show,NULL ); + + +static struct attribute *vmbus_chan_attrs[] = { &chan_attr_out_mask.attr, &chan_attr_in_mask.attr,
@@ -1246,6 +1262,8 @@ static struct attribute *vmbus_chan_attrs[] = { &chan_attr_cpu.attr, &chan_attr_pending.attr, &chan_attr_latency.attr, + &chan_attr_interrupts.attr, + &chan_attr_events.attr, NULL };
include/linux/hyperv.h +4 -0
@@ -719,6 +719,10 @@ struct vmbus_channel {struct vmbus_close_msg close_msg; +/* Statistics */ +unsigned int interrupts;/* Host to Guest interrupts */ +unsigned int sig_events;/* Guest to Host events */ +/* Channel callback's invoked in softirq context */ struct tasklet_struct callback_event;void (*onchannel_callback)(void *context);
Signature · 1 line
2.14.1
PATCH 2/3 vmbus: initialize reserved fields in messages
From: Stephen Hemminger <[email protected]>
Make sure and initialize reserved fields in messages to host, rather than passing stack junk.
--- drivers/hv/channel.c | 2 ++ 1 file changed, 2 insertions(+)
drivers/hv/channel.c +2 -0
@@ -745,6 +745,7 @@ int vmbus_sendpacket_pagebuffer(struct vmbus_channel *channel, desc.dataoffset8 = descsize >>3 ;/* in 8-bytes granularity */ desc.length8 = (u16)(packetlen_aligned >>3 ); desc.transactionid = requestid; + desc.reserved =0 ; desc.rangecount = pagecount;for (i =0 ; i < pagecount; i++) {
@@ -788,6 +789,7 @@ int vmbus_sendpacket_mpb_desc(struct vmbus_channel *channel, desc->dataoffset8 = desc_size >>3 ;/* in 8-bytes granularity */ desc->length8 = (u16)(packetlen_aligned >>3 ); desc->transactionid = requestid; + desc->reserved =0 ; desc->rangecount =1 ; bufferlist[0 ].iov_base = desc;
Signature · 1 line
2.14.1
PATCH 3/3 Drivers: hv: vmbus: Make pannic reporting to be more useful
In reply to [email protected]From: "K. Y. Srinivasan" <[email protected]>
Hyper-V allows the guest to report panic and the guest can pass additional information. All this is logged on the host. Currently Linux is passing back information that is not particularly useful. Make the following changes:
1. Windows uses crash MSR P0 to report bugcheck code. Follow the same convention for Linux as well.
2. It will be useful to know the gust ID of the Linux guest that has paniced. Pass back this information.
These changes will help in better supporting Linux on Hyper-V
--- arch/x86/hyperv/hv_init.c | 11 +++++++---- arch/x86/include/asm/mshyperv.h | 2 +- drivers/hv/vmbus_drv.c | 4 ++-- 3 files changed, 10 insertions(+), 7 deletions(-)
arch/x86/hyperv/hv_init.c +7 -4
@@ -205,9 +205,10 @@ void hyperv_cleanup(void) }EXPORT_SYMBOL_GPL (hyperv_cleanup); -void hyperv_report_panic (struct pt_regs *regs) +void hyperv_report_panic (struct pt_regs *regs,long err) {static bool panic_reported; +u64 guest_id;/* * We prefer to report panic on 'die' chain as we have proper
@@ -218,9 +219,11 @@ void hyperv_report_panic(struct pt_regs *regs)return ; panic_reported = true; -wrmsrl (HV_X64_MSR_CRASH_P0 , regs->ip); -wrmsrl (HV_X64_MSR_CRASH_P1 , regs->ax); -wrmsrl (HV_X64_MSR_CRASH_P2 , regs->bx); +rdmsrl (HV_X64_MSR_GUEST_OS_ID , guest_id); + +wrmsrl (HV_X64_MSR_CRASH_P0 , err); +wrmsrl (HV_X64_MSR_CRASH_P1 , guest_id); +wrmsrl (HV_X64_MSR_CRASH_P2 , regs->ip);wrmsrl (HV_X64_MSR_CRASH_P3 , regs->cx);wrmsrl (HV_X64_MSR_CRASH_P4 , regs->dx);
arch/x86/include/asm/mshyperv.h +1 -1
@@ -311,7 +311,7 @@ static inline int hv_cpu_number_to_vp_number(int cpu_number)void hyperv_init (void );void hyperv_setup_mmu_ops (void );void hyper_alloc_mmu (void ); -void hyperv_report_panic (struct pt_regs *regs); +void hyperv_report_panic (struct pt_regs *regs,long err);bool hv_is_hypercall_page_setup (void );void hyperv_cleanup (void );#else /* CONFIG_HYPERV */
drivers/hv/vmbus_drv.c +2 -2
@@ -65,7 +65,7 @@ static int hyperv_panic_event(struct notifier_block *nb, unsigned long val, regs =current_pt_regs (); -hyperv_report_panic (regs); +hyperv_report_panic (regs, val);return NOTIFY_DONE ; }
@@ -75,7 +75,7 @@ static int hyperv_die_event(struct notifier_block *nb, unsigned long val,struct die_args *die = (struct die_args *)args;struct pt_regs *regs = die->regs; -hyperv_report_panic (regs); +hyperv_report_panic (regs, val);return NOTIFY_DONE ; }
Signature · 1 line
2.14.1
Re: [PATCH 3/3] Drivers: hv: vmbus: Make pannic reporting to be more useful
Quoted 37 lines · [email protected] writes:
From: "K. Y. Srinivasan" <[email protected]>
Hyper-V allows the guest to report panic and the guest can pass additional information. All this is logged on the host. Currently Linux is passing back information that is not particularly useful. Make the following changes:
1. Windows uses crash MSR P0 to report bugcheck code. Follow the same convention for Linux as well.
2. It will be useful to know the gust ID of the Linux guest that has paniced. Pass back this information.These changes will help in better supporting Linux on Hyper-V
Signed-off-by K. Y. Srinivasan <[email protected]>--- arch/x86/hyperv/hv_init.c | 11 +++++++---- arch/x86/include/asm/mshyperv.h | 2 +- drivers/hv/vmbus_drv.c | 4 ++-- 3 files changed, 10 insertions(+), 7 deletions(-)arch/x86/hyperv/hv_init.c +2
-1@@ -205,9 +205,10 @@ void hyperv_cleanup(void) }EXPORT_SYMBOL_GPL (hyperv_cleanup); -void hyperv_report_panic (struct pt_regs *regs) +void hyperv_report_panic (struct pt_regs *regs,long err) {static bool panic_reported; +u64 guest_id;/* * We prefer to report panic on 'die' chain as we have proper
arch/x86/hyperv/hv_init.c void hyperv_report_panic(struct pt_regs *regs) · line 225
Quoted 10 lines
arch/x86/hyperv/hv_init.c +4
-3@@ -218,9 +219,11 @@ void hyperv_report_panic(struct pt_regs *regs)return ; panic_reported = true; -wrmsrl (HV_X64_MSR_CRASH_P0 , regs->ip); -wrmsrl (HV_X64_MSR_CRASH_P1 , regs->ax); -wrmsrl (HV_X64_MSR_CRASH_P2 , regs->bx); +rdmsrl (HV_X64_MSR_GUEST_OS_ID , guest_id); + +wrmsrl (HV_X64_MSR_CRASH_P0 , err); +wrmsrl (HV_X64_MSR_CRASH_P1 , guest_id);
This is a constant we write in hyperv_init() (0x810000040e000000 for Linux guests). Do I get it right that we need this to basically distinguigh Windows guests crashes from Linux guest crashes in the log?
arch/x86/hyperv/hv_init.c void hyperv_report_panic(struct pt_regs *regs) · line 228
Quoted 3 lines
arch/x86/hyperv/hv_init.c +1
-0+wrmsrl (HV_X64_MSR_CRASH_P2 , regs->ip);wrmsrl (HV_X64_MSR_CRASH_P3 , regs->cx);wrmsrl (HV_X64_MSR_CRASH_P4 , regs->dx);
We don't write ax and bx any more but write cx and dx. Not that I see these regs really useful in the log but I'd change these to ax and bx for consistency. Or mayme make it sp and ax?
Quoted 35 lines
arch/x86/include/asm/mshyperv.h +1
-1@@ -311,7 +311,7 @@ static inline int hv_cpu_number_to_vp_number(int cpu_number)void hyperv_init (void );void hyperv_setup_mmu_ops (void );void hyper_alloc_mmu (void ); -void hyperv_report_panic (struct pt_regs *regs); +void hyperv_report_panic (struct pt_regs *regs,long err);bool hv_is_hypercall_page_setup (void );void hyperv_cleanup (void );#else /* CONFIG_HYPERV */ drivers/hv/vmbus_drv.c +2
-2@@ -65,7 +65,7 @@ static int hyperv_panic_event(struct notifier_block *nb, unsigned long val, regs =current_pt_regs (); -hyperv_report_panic (regs); +hyperv_report_panic (regs, val);return NOTIFY_DONE ; }@@ -75,7 +75,7 @@ static int hyperv_die_event(struct notifier_block *nb, unsigned long val,struct die_args *die = (struct die_args *)args;struct pt_regs *regs = die->regs; -hyperv_report_panic (regs); +hyperv_report_panic (regs, val);return NOTIFY_DONE ; }
Signature · 1 line
Vitaly
RE: [PATCH 3/3] Drivers: hv: vmbus: Make pannic reporting to be more useful
Quoted 56 lines
-----Original Message-----
From: Vitaly Kuznetsov [mailto:[email protected]]
Sent: Thursday, October 12, 2017 6:40 AM
To: [email protected]
Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; Stephen Hemminger <[email protected]>; KY Srinivasan <[email protected]>
Subject: Re: [PATCH 3/3] Drivers: hv: vmbus: Make pannic reporting to be more usefulQuoted 40 lines · [email protected] writes:
From: "K. Y. Srinivasan" <[email protected]>
Hyper-V allows the guest to report panic and the guest can pass additional information. All this is logged on the host. Currently Linux is passing back information that is not particularly useful. Make the following changes:
1. Windows uses crash MSR P0 to report bugcheck code. Follow the same convention for Linux as well.
2. It will be useful to know the gust ID of the Linux guest that has paniced. Pass back this information.These changes will help in better supporting Linux on Hyper-V
Signed-off-by K. Y. Srinivasan <[email protected]>--- arch/x86/hyperv/hv_init.c | 11 +++++++---- arch/x86/include/asm/mshyperv.h | 2 +- drivers/hv/vmbus_drv.c | 4 ++-- 3 files changed, 10 insertions(+), 7 deletions(-)arch/x86/hyperv/hv_init.c +2
-1@@ -205,9 +205,10 @@ void hyperv_cleanup(void) }EXPORT_SYMBOL_GPL (hyperv_cleanup); -void hyperv_report_panic (struct pt_regs *regs) +void hyperv_report_panic (struct pt_regs *regs,long err) {static bool panic_reported; +u64 guest_id;/* * We prefer to report panic on 'die' chain as we have proper @@ -218,9 +219,11 @@ void hyperv_report_panic(struct pt_regs *regs)return ; panic_reported = true;
Quoted 11 lines
arch/x86/hyperv/hv_init.c void hyperv_report_panic(struct pt_regs *regs) · line 225
Quoted 7 lines
arch/x86/hyperv/hv_init.c +4
-3-wrmsrl (HV_X64_MSR_CRASH_P0 , regs->ip); -wrmsrl (HV_X64_MSR_CRASH_P1 , regs->ax); -wrmsrl (HV_X64_MSR_CRASH_P2 , regs->bx); +rdmsrl (HV_X64_MSR_GUEST_OS_ID , guest_id); + +wrmsrl (HV_X64_MSR_CRASH_P0 , err); +wrmsrl (HV_X64_MSR_CRASH_P1 , guest_id);This is a constant we write in
hyperv_init()(0x810000040e000000for Linux guests). Do I get it right that we need this to basically distinguigh Windows guests crashes from Linux guest crashes in the log?
Yes; we have a huge infrastructure for analyzing the event logs on Azure hosts and guest panics are logged into this pipeline. Unfortunately, there is not a simple way to distinguish between Windows and Linux in this information stream.
Quoted 8 lines
arch/x86/hyperv/hv_init.c void hyperv_report_panic(struct pt_regs *regs) · line 228
Quoted 4 lines
arch/x86/hyperv/hv_init.c +1
-0+wrmsrl (HV_X64_MSR_CRASH_P2 , regs->ip);wrmsrl (HV_X64_MSR_CRASH_P3 , regs->cx);wrmsrl (HV_X64_MSR_CRASH_P4 , regs->dx);We don't write ax and bx any more but write cx and dx. Not that I see these regs really useful in the log but I'd change these to ax and bx for consistency. Or mayme make it sp and ax?
I will make it sp and ax.
Thanks,
K. Y
Quoted 43 lines
Review
Quoted 3 lines
(quoted diff) +0
-0
b/arch/x86/include/asm/mshyperv.harch/x86/include/asm/mshyperv.h static inline int hv_cpu_number_to_vp_number(int
Quoted 4 lines
index
63cc96f064dc..dd2dc54ddf20100644arch/x86/include/asm/mshyperv.h +0
-0@@ -311,7 +311,7 @@ static inline int hv_cpu_number_to_vp_number(int
cpu_number)drivers/hv/vmbus_drv.c static int hyperv_panic_event(struct notifier_block
Quoted 13 lines
arch/x86/include/asm/mshyperv.h +1
-1void hyperv_init (void );void hyperv_setup_mmu_ops (void );void hyper_alloc_mmu (void ); -void hyperv_report_panic (struct pt_regs *regs); +void hyperv_report_panic (struct pt_regs *regs,long err);bool hv_is_hypercall_page_setup (void );void hyperv_cleanup (void );#else /* CONFIG_HYPERV */ drivers/hv/vmbus_drv.c +0
-0@@ -65,7 +65,7 @@ static int hyperv_panic_event(struct notifier_block*nb, unsigned long val,
drivers/hv/vmbus_drv.c static int hyperv_die_event(struct notifier_block *nb,
Quoted 9 lines
drivers/hv/vmbus_drv.c +1
-1regs =current_pt_regs (); -hyperv_report_panic (regs); +hyperv_report_panic (regs, val);return NOTIFY_DONE ; }@@ -75,7 +75,7 @@ static int hyperv_die_event(struct notifier_block *nb,unsigned long val,
drivers/hv/vmbus_drv.c static int hyperv_die_event(struct notifier_block *nb, unsigned long val, · line 80
Quoted 7 lines
drivers/hv/vmbus_drv.c +1
-1struct die_args *die = (struct die_args *)args;struct pt_regs *regs = die->regs; -hyperv_report_panic (regs); +hyperv_report_panic (regs, val);return NOTIFY_DONE ; }--
Vitaly
Re: [PATCH 1/3] Drivers: hv: vmbus: Expose per-channel interrupts and events counters
In reply to [email protected]Quoted 20 lines · On Thu, Oct 05, 2017 at 05:35:04PM -0700, [email protected] wrote:
From: Stephen Hemminger <[email protected]>
When investigating performance, it is useful to be able to look at the number of host and guest events per-channel. This is equivalent to per-device interrupt statistics.
Signed-off-by Stephen Hemminger <[email protected]>Signed-off-by K. Y. Srinivasan <[email protected]>--- Documentation/ABI/stable/sysfs-bus-vmbus | 14 ++++++++++++++ drivers/hv/connection.c | 2 ++ drivers/hv/vmbus_drv.c | 18 ++++++++++++++++++ include/linux/hyperv.h | 4 ++++ 4 files changed, 38 insertions(+)Documentation/ABI/stable/sysfs-bus-vmbus +0
-0
Documentation/ABI/stable/sysfs-bus-vmbus KernelVersion: 4.14 · line 106
Quoted 10 lines
Documentation/ABI/stable/sysfs-bus-vmbus +7
-0@@ -97,3 +97,17 @@ KernelVersion: 4.14 Contact: Stephen Hemminger <[email protected]> Description: Bytes availabble to write Users: Debuggig tools + +What: /sys/bus/vmbus/devices/vmbus_*/channels/relid/events +Date: September. 2017 +KernelVersion: 4.14 +Contact: Stephen Hemminger <[email protected]> +Description: Number of times we have signaled the host +Users: Debuggig tools
Odd spelling :)
Documentation/ABI/stable/sysfs-bus-vmbus KernelVersion: 4.14 · line 113
Quoted 7 lines
Documentation/ABI/stable/sysfs-bus-vmbus +7
-0+ +What: /sys/bus/vmbus/devices/vmbus_*/channels/relid/interrupts +Date: September. 2017 +KernelVersion: 4.14 +Contact: Stephen Hemminger <[email protected]> +Description: Number of times we have taken an interrupt (incoming) +Users: Debuggig tools
Same odd spelling :(
Quoted 34 lines
drivers/hv/connection.c +2
-0@@ -409,6 +409,8 @@ void vmbus_set_event(struct vmbus_channel *channel)if (!channel->is_dedicated_interrupt)vmbus_send_interrupt (child_relid); + ++channel->sig_events; +hv_do_fast_hypercall8 (HVCALL_SIGNAL_EVENT , channel->sig_event); }EXPORT_SYMBOL_GPL (vmbus_set_event);drivers/hv/vmbus_drv.c +6
-0@@ -945,6 +945,8 @@ static void vmbus_chan_sched(struct hv_per_cpu_context *hv_cpu)if (channel->rescind)continue ; + ++channel->interrupts; +switch (channel->callback_mode) {case HV_CALL_ISR :vmbus_channel_isr (channel);@@ -1238,6 +1240,20 @@ static ssize_t channel_latency_show(const struct vmbus_channel *channel, }VMBUS_CHAN_ATTR (latency,S_IRUGO , channel_latency_show,NULL ); +static ssize_t channel_interrupts_show (const struct vmbus_channel *channel,char *buf) +{ +return sprintf (buf,"%u\n" , channel->interrupts); +}
drivers/hv/vmbus_drv.c static ssize_t channel_latency_show(const struct vmbus_channel *channel, · line 1256
Quoted 10 lines
drivers/hv/vmbus_drv.c +10
-0+VMBUS_CHAN_ATTR (interrupts,S_IRUGO , channel_interrupts_show,NULL ); + +static ssize_t channel_events_show (const struct vmbus_channel *channel,char *buf) +{ +return sprintf (buf,"%u\n" , channel->sig_events); +} +VMBUS_CHAN_ATTR (events,S_IRUGO , channel_events_show,NULL ); + + +
Why so many blank lines?
Quoted 17 lines
drivers/hv/vmbus_drv.c +2
-0static struct attribute *vmbus_chan_attrs[] = { &chan_attr_out_mask.attr, &chan_attr_in_mask.attr,@@ -1246,6 +1262,8 @@ static struct attribute *vmbus_chan_attrs[] = { &chan_attr_cpu.attr, &chan_attr_pending.attr, &chan_attr_latency.attr, + &chan_attr_interrupts.attr, + &chan_attr_events.attr, NULL };include/linux/hyperv.h +0
-0
include/linux/hyperv.h struct vmbus_channel { · line 724
Quoted 6 lines
include/linux/hyperv.h +3
-0@@ -719,6 +719,10 @@ struct vmbus_channel {struct vmbus_close_msg close_msg; +/* Statistics */ +unsigned int interrupts;/* Host to Guest interrupts */ +unsigned int sig_events;/* Guest to Host events */
u32 or u64 here? How is userspace going to handle these overflowing?
thanks,
greg k-h
Quoted 11 lines · On Thu, Oct 05, 2017 at 05:33:59PM -0700, [email protected] wrote:
From: "K. Y. Srinivasan" <[email protected]>
Miscellaneous enhancements and fixes.
K. Y. Srinivasan (1):
Drivers: hv: vmbus: Make pannic reporting to be more usefulStephen Hemminger (2):
Drivers: hv: vmbus: Expose per-channel interrupts and events counters
vmbus: initialize reserved fields in messages
Patch 2 of this series applied, the rest I'll expect a new version of.
thanks,
greg k-h
Quoted 23 lines
-----Original Message-----
From: Greg KH [mailto:[email protected]]
Sent: Friday, October 20, 2017 3:28 PM
To: KY Srinivasan <[email protected]>
Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; Stephen Hemminger <[email protected]>
Subject: Re: [PATCH 0/3] Drivers: hv: vmbus: Miscellaneous enhancements and fixesQuoted 7 lines · On Thu, Oct 05, 2017 at 05:33:59PM -0700, [email protected] wrote:
From: "K. Y. Srinivasan" <[email protected]>
Miscellaneous enhancements and fixes.
K. Y. Srinivasan (1):
Drivers: hv: vmbus: Make pannic reporting to be more useful
Quoted 6 lines
Quoted 4 lines
Stephen Hemminger (2):
Drivers: hv: vmbus: Expose per-channel interrupts and events counters
vmbus: initialize reserved fields in messagesPatch 2 of this series applied, the rest I'll expect a new version of.
Will send.
K. Y
Quoted 4 lines
thanks,
greg k-h