net: add a flag to disable mac/vlan filtering
RFC PATCH 1/1 net: add a flag to disable mac/vlan filtering
New bridge in linux 2.6.34 adds IGMP snooping support, after which bridge should not normally flood any packets. While we still need mac table to arm forwarding tables after migration, we can thus ignore it for rx datapath.
For vlan, it's possible to do filtering down the stack simply by using bridge per guest and binding said bridge to vlan device, which some people do.
Since qemu has no easy way to check IGMP snooping support in bridge or how it's connected, add options to disable rx filtering, so that management can set it as appropriate.
Use these options to optimise virtio-net rx path. We still ask guest for the list of vlans/macs for migration.
--- hw/virtio-net.c | 10 +++++++++- net.h | 12 +++++++++++- 2 files changed, 20 insertions(+), 2 deletions(-)
hw/virtio-net.c +9 -1
@@ -47,6 +47,7 @@ typedef struct VirtIONetuint8_t nomulti;uint8_t nouni;uint8_t nobcast; +uint32_t filtering;struct {int in_use;int first_multi;
@@ -475,12 +476,17 @@ static int receive_filter(VirtIONet *n, const uint8_t *buf, int size) ptr +=sizeof (struct virtio_net_hdr ); } -if (!memcmp (&ptr[12 ], vlan,sizeof (vlan))) { +if ((n->filtering & (0x1 <<NICCONF_F_VLAN_FILTERING )) && + !memcmp (&ptr[12 ], vlan,sizeof (vlan))) {int vid =be16_to_cpup ((uint16_t *)(ptr +14 )) &0xfff ;if (!(n->vlans[vid >>5 ] & (1U << (vid &0x1f ))))return 0 ; } +if (!(n->filtering & (0x1 <<NICCONF_F_MAC_FILTERING ))) { +return 1 ; + } +if (ptr[0 ] &1 ) {// multicast if (!memcmp (ptr, bcast,sizeof (bcast))) {return !n->nobcast;
@@ -863,6 +869,8 @@ VirtIODevice *virtio_net_init(DeviceState *dev, NICConf *conf) n->mac_table.macs =qemu_mallocz (MAC_TABLE_ENTRIES *ETH_ALEN ); + n->filtering = conf->filtering; + n->vlans =qemu_mallocz (MAX_VLAN >>3 ); register_savevm("virtio-net", virtio_net_id++, VIRTIO_NET_VM_VERSION,
net.h +11 -1
@@ -18,12 +18,22 @@ typedef struct NICConf {MACAddr macaddr;VLANState *vlan;VLANClientState *peer; +uint32_t filtering; } NICConf; +enum { +NICCONF_F_MAC_FILTERING =0 , +NICCONF_F_VLAN_FILTERING =1 +}; +#define DEFINE_NIC_PROPERTIES (_state, _conf) \ DEFINE_PROP_MACADDR("mac", _state, _conf.macaddr), \ DEFINE_PROP_VLAN("vlan", _state, _conf.vlan), \ - DEFINE_PROP_NETDEV("netdev", _state, _conf.peer) + DEFINE_PROP_NETDEV("netdev", _state, _conf.peer), \ + DEFINE_PROP_BIT("mac_filtering", _state, _conf.filtering, \ + NICCONF_F_MAC_FILTERING, true) \ + DEFINE_PROP_BIT("vlan_filtering", _state, _conf.filtering, \ + NICCONF_F_VLAN_FILTERING, true) \/* VLANs support */
Signature · 1 line
1.7.0.18.g0d53a5
Quoted 17 lines · Michael S. Tsirkin
New bridge in linux 2.6.34 adds IGMP snooping support, after which bridge should not normally flood any packets. While we still need mac table to arm forwarding tables after migration, we can thus ignore it for rx datapath.
For vlan, it's possible to do filtering down the stack simply by using bridge per guest and binding said bridge to vlan device, which some people do.
Since qemu has no easy way to check IGMP snooping support in bridge or how it's connected, add options to disable rx filtering, so that management can set it as appropriate.
Use these options to optimise virtio-net rx path. We still ask guest for the list of vlans/macs for migration.
Quoted 2 lines
Signed-off-by Michael S. Tsirkin<[email protected]>
Can't this be achieved by just disabling the feature bits? IOW,
ctrl_vq=0,ctrl_vlan=0?
Regards,
Anthony Liguori
Quoted 76 lines
Cc Alex Williamson<[email protected]>Cc Andreas Plesner Jacobsen<[email protected]>--- hw/virtio-net.c | 10 +++++++++- net.h | 12 +++++++++++- 2 files changed, 20 insertions(+), 2 deletions(-)hw/virtio-net.c +9
-1@@ -47,6 +47,7 @@ typedef struct VirtIONetuint8_t nomulti;uint8_t nouni;uint8_t nobcast; +uint32_t filtering;struct {int in_use;int first_multi;@@ -475,12 +476,17 @@ static int receive_filter(VirtIONet *n, const uint8_t *buf, int size) ptr +=sizeof (struct virtio_net_hdr ); } -if (!memcmp (&ptr[12 ], vlan,sizeof (vlan))) { +if ((n->filtering& (0x1 <<NICCONF_F_VLAN_FILTERING ))&& + !memcmp (&ptr[12 ], vlan,sizeof (vlan))) {int vid =be16_to_cpup ((uint16_t *)(ptr +14 ))&0xfff ;if (!(n->vlans[vid>>5 ]& (1U << (vid&0x1f ))))return 0 ; } +if (!(n->filtering& (0x1 <<NICCONF_F_MAC_FILTERING ))) { +return 1 ; + } +if (ptr[0 ]&1 ) {// multicast if (!memcmp (ptr, bcast,sizeof (bcast))) {return !n->nobcast;@@ -863,6 +869,8 @@ VirtIODevice *virtio_net_init(DeviceState *dev, NICConf *conf) n->mac_table.macs =qemu_mallocz (MAC_TABLE_ENTRIES *ETH_ALEN ); + n->filtering = conf->filtering; + n->vlans =qemu_mallocz (MAX_VLAN >>3 ); register_savevm("virtio-net", virtio_net_id++, VIRTIO_NET_VM_VERSION,net.h +11
-1@@ -18,12 +18,22 @@ typedef struct NICConf {MACAddr macaddr;VLANState *vlan;VLANClientState *peer; +uint32_t filtering; } NICConf; +enum { +NICCONF_F_MAC_FILTERING =0 , +NICCONF_F_VLAN_FILTERING =1 +}; +#define DEFINE_NIC_PROPERTIES (_state, _conf) \ DEFINE_PROP_MACADDR("mac", _state, _conf.macaddr), \ DEFINE_PROP_VLAN("vlan", _state, _conf.vlan), \ - DEFINE_PROP_NETDEV("netdev", _state, _conf.peer) + DEFINE_PROP_NETDEV("netdev", _state, _conf.peer), \ + DEFINE_PROP_BIT("mac_filtering", _state, _conf.filtering, \ + NICCONF_F_MAC_FILTERING, true) \ + DEFINE_PROP_BIT("vlan_filtering", _state, _conf.filtering, \ + NICCONF_F_VLAN_FILTERING, true) \/* VLANs support */
Quoted 18 lines · Anthony Liguori
Quoted 17 lines · Michael S. Tsirkin
New bridge in linux 2.6.34 adds IGMP snooping support, after which bridge should not normally flood any packets. While we still need mac table to arm forwarding tables after migration, we can thus ignore it for rx datapath.
For vlan, it's possible to do filtering down the stack simply by using bridge per guest and binding said bridge to vlan device, which some people do.
Since qemu has no easy way to check IGMP snooping support in bridge or how it's connected, add options to disable rx filtering, so that management can set it as appropriate.
Use these options to optimise virtio-net rx path. We still ask guest for the list of vlans/macs for migration.
Quoted 6 lines
Quoted 2 lines
Signed-off-by Michael S. Tsirkin<[email protected]>Can't this be achieved by just disabling the feature bits? IOW,
ctrl_vq=0,ctrl_vlan=0?
Michael still wants the guest to populate the MAC filter table so that we can use it to announce MACs on the bridge, especially after a migration.
Alex
Quoted 7 lines
Quoted 3 lines
Can't this be achieved by just disabling the feature bits? IOW,
ctrl_vq=0,ctrl_vlan=0?
Michael still wants the guest to populate the MAC filter table so that we can use it to announce MACs on the bridge, especially after a migration.
We don't do that to start with. We only set gratuitous ARPs for the MAC address of the emulated NIC. I'm not convinced anything else is useful.
Paul
Quoted 21 lines · Anthony Liguori
Quoted 17 lines · Michael S. Tsirkin
New bridge in linux 2.6.34 adds IGMP snooping support, after which bridge should not normally flood any packets. While we still need mac table to arm forwarding tables after migration, we can thus ignore it for rx datapath.
For vlan, it's possible to do filtering down the stack simply by using bridge per guest and binding said bridge to vlan device, which some people do.
Since qemu has no easy way to check IGMP snooping support in bridge or how it's connected, add options to disable rx filtering, so that management can set it as appropriate.
Use these options to optimise virtio-net rx path. We still ask guest for the list of vlans/macs for migration.Quoted 2 lines
Signed-off-by Michael S. Tsirkin<[email protected]>
Quoted 7 lines
Can't this be achieved by just disabling the feature bits? IOW,
ctrl_vq=0,ctrl_vlan=0?
Regards,
Anthony Liguori
It can, but then we won't be able to migrate to a host that does not do the filtering in host kernel.
Quoted 76 lines
Quoted 76 lines
Cc Alex Williamson<[email protected]>Cc Andreas Plesner Jacobsen<[email protected]>--- hw/virtio-net.c | 10 +++++++++- net.h | 12 +++++++++++- 2 files changed, 20 insertions(+), 2 deletions(-)hw/virtio-net.c +9
-1@@ -47,6 +47,7 @@ typedef struct VirtIONetuint8_t nomulti;uint8_t nouni;uint8_t nobcast; +uint32_t filtering;struct {int in_use;int first_multi;@@ -475,12 +476,17 @@ static int receive_filter(VirtIONet *n, const uint8_t *buf, int size) ptr +=sizeof (struct virtio_net_hdr ); } -if (!memcmp (&ptr[12 ], vlan,sizeof (vlan))) { +if ((n->filtering& (0x1 <<NICCONF_F_VLAN_FILTERING ))&& + !memcmp (&ptr[12 ], vlan,sizeof (vlan))) {int vid =be16_to_cpup ((uint16_t *)(ptr +14 ))&0xfff ;if (!(n->vlans[vid>>5 ]& (1U << (vid&0x1f ))))return 0 ; } +if (!(n->filtering& (0x1 <<NICCONF_F_MAC_FILTERING ))) { +return 1 ; + } +if (ptr[0 ]&1 ) {// multicast if (!memcmp (ptr, bcast,sizeof (bcast))) {return !n->nobcast;@@ -863,6 +869,8 @@ VirtIODevice *virtio_net_init(DeviceState *dev, NICConf *conf) n->mac_table.macs =qemu_mallocz (MAC_TABLE_ENTRIES *ETH_ALEN ); + n->filtering = conf->filtering; + n->vlans =qemu_mallocz (MAX_VLAN >>3 ); register_savevm("virtio-net", virtio_net_id++, VIRTIO_NET_VM_VERSION,net.h +11
-1@@ -18,12 +18,22 @@ typedef struct NICConf {MACAddr macaddr;VLANState *vlan;VLANClientState *peer; +uint32_t filtering; } NICConf; +enum { +NICCONF_F_MAC_FILTERING =0 , +NICCONF_F_VLAN_FILTERING =1 +}; +#define DEFINE_NIC_PROPERTIES (_state, _conf) \ DEFINE_PROP_MACADDR("mac", _state, _conf.macaddr), \ DEFINE_PROP_VLAN("vlan", _state, _conf.vlan), \ - DEFINE_PROP_NETDEV("netdev", _state, _conf.peer) + DEFINE_PROP_NETDEV("netdev", _state, _conf.peer), \ + DEFINE_PROP_BIT("mac_filtering", _state, _conf.filtering, \ + NICCONF_F_MAC_FILTERING, true) \ + DEFINE_PROP_BIT("vlan_filtering", _state, _conf.filtering, \ + NICCONF_F_VLAN_FILTERING, true) \/* VLANs support */
Quoted 18 lines · Michael S. Tsirkin
hw/virtio-net.c +2
-1@@ -47,6 +47,7 @@ typedef struct VirtIONetuint8_t nomulti;uint8_t nouni;uint8_t nobcast; +uint32_t filtering;struct {int in_use;int first_multi;@@ -475,12 +476,17 @@ static int receive_filter(VirtIONet *n, const uint8_t *buf, int size) ptr +=sizeof (struct virtio_net_hdr ); } -if (!memcmp (&ptr[12 ], vlan,sizeof (vlan))) { + if ((n->filtering & (0x1 << NICCONF_F_VLAN_FILTERING)) &&
hw/virtio-net.c static int receive_filter(VirtIONet *n, const uint8_t *buf, int size) · line 489
Quoted 10 lines
hw/virtio-net.c +5
-0+ !memcmp(&ptr[12], vlan, sizeof(vlan))) {int vid =be16_to_cpup ((uint16_t *)(ptr +14 )) &0xfff ;if (!(n->vlans[vid >>5 ] & (1U << (vid &0x1f ))))return 0 ; } +if (!(n->filtering & (0x1 <<NICCONF_F_MAC_FILTERING ))) { +return 1 ; + } +
A filtering flags bitmap is a logical choice here, but I found the overhead to be non-trivial, which is why we have separate variables for the other filtering options.
hw/virtio-net.c VirtIODevice *virtio_net_init(DeviceState *dev, NICConf *conf) · line 873
Quoted 9 lines
hw/virtio-net.c +2
-0if (ptr[0 ] &1 ) {// multicast if (!memcmp (ptr, bcast,sizeof (bcast))) {return !n->nobcast;@@ -863,6 +869,8 @@ VirtIODevice *virtio_net_init(DeviceState *dev, NICConf *conf) n->mac_table.macs =qemu_mallocz (MAC_TABLE_ENTRIES *ETH_ALEN ); + n->filtering = conf->filtering; +
Since we're not touching this in the savevm code, I assume the intention is that we can transparently migrate between filtered bridges and non-filtered bridges, right? Thanks,
Alex
Quoted 25 lines · Alex Williamson
Quoted 24 lines · Michael S. Tsirkin
hw/virtio-net.c +3
-1@@ -47,6 +47,7 @@ typedef struct VirtIONetuint8_t nomulti;uint8_t nouni;uint8_t nobcast; +uint32_t filtering;struct {int in_use;int first_multi;@@ -475,12 +476,17 @@ static int receive_filter(VirtIONet *n, const uint8_t *buf, int size) ptr +=sizeof (struct virtio_net_hdr ); } -if (!memcmp (&ptr[12 ], vlan,sizeof (vlan))) { +if ((n->filtering & (0x1 <<NICCONF_F_VLAN_FILTERING )) && + !memcmp (&ptr[12 ], vlan,sizeof (vlan))) {int vid =be16_to_cpup ((uint16_t *)(ptr +14 )) &0xfff ;if (!(n->vlans[vid >>5 ] & (1U << (vid &0x1f ))))return 0 ; }
Quoted 8 lines
hw/virtio-net.c static int receive_filter(VirtIONet *n, const uint8_t *buf, int size) · line 489
Quoted 4 lines
hw/virtio-net.c +4
-0+if (!(n->filtering & (0x1 <<NICCONF_F_MAC_FILTERING ))) { +return 1 ; + } +A filtering flags bitmap is a logical choice here, but I found the overhead to be non-trivial, which is why we have separate variables for the other filtering options.
You suggest more flags for multicast etc?
Quoted 5 lines
Quoted 5 lines
hw/virtio-net.c +0
-0if (ptr[0 ] &1 ) {// multicast if (!memcmp (ptr, bcast,sizeof (bcast))) {return !n->nobcast;@@ -863,6 +869,8 @@ VirtIODevice *virtio_net_init(DeviceState *dev, NICConf *conf)
Quoted 8 lines
hw/virtio-net.c VirtIODevice *virtio_net_init(DeviceState *dev, NICConf *conf) · line 873
Quoted 4 lines
hw/virtio-net.c +2
-0n->mac_table.macs =qemu_mallocz (MAC_TABLE_ENTRIES *ETH_ALEN ); + n->filtering = conf->filtering; +Since we're not touching this in the savevm code, I assume the intention is that we can transparently migrate between filtered bridges and non-filtered bridges, right?
Right.
Quoted 3 lines
Thanks,
Alex
Quoted 35 lines · Michael S. Tsirkin
Quoted 25 lines · Alex Williamson
Quoted 24 lines · Michael S. Tsirkin
hw/virtio-net.c +3
-1@@ -47,6 +47,7 @@ typedef struct VirtIONetuint8_t nomulti;uint8_t nouni;uint8_t nobcast; +uint32_t filtering;struct {int in_use;int first_multi;@@ -475,12 +476,17 @@ static int receive_filter(VirtIONet *n, const uint8_t *buf, int size) ptr +=sizeof (struct virtio_net_hdr ); } -if (!memcmp (&ptr[12 ], vlan,sizeof (vlan))) { +if ((n->filtering & (0x1 <<NICCONF_F_VLAN_FILTERING )) && + !memcmp (&ptr[12 ], vlan,sizeof (vlan))) {int vid =be16_to_cpup ((uint16_t *)(ptr +14 )) &0xfff ;if (!(n->vlans[vid >>5 ] & (1U << (vid &0x1f ))))return 0 ; }Quoted 8 lines
hw/virtio-net.c static int receive_filter(VirtIONet *n, const uint8_t *buf, int size) · line 489
Quoted 4 lines
hw/virtio-net.c +4
-0+if (!(n->filtering & (0x1 <<NICCONF_F_MAC_FILTERING ))) { +return 1 ; + } +A filtering flags bitmap is a logical choice here, but I found the overhead to be non-trivial, which is why we have separate variables for the other filtering options.
Quoted 1 line
You suggest more flags for multicast etc?
Might not be a bad idea: e.g. tap filtering table in kernel is limited in size, so we might win from only downloading necessary mac addresses there.
Quoted 19 lines
Quoted 5 lines
Quoted 5 lines
hw/virtio-net.c +0
-0if (ptr[0 ] &1 ) {// multicast if (!memcmp (ptr, bcast,sizeof (bcast))) {return !n->nobcast;@@ -863,6 +869,8 @@ VirtIODevice *virtio_net_init(DeviceState *dev, NICConf *conf)Quoted 8 lines
hw/virtio-net.c VirtIODevice *virtio_net_init(DeviceState *dev, NICConf *conf) · line 873
Quoted 4 lines
hw/virtio-net.c +2
-0n->mac_table.macs =qemu_mallocz (MAC_TABLE_ENTRIES *ETH_ALEN ); + n->filtering = conf->filtering; +Since we're not touching this in the savevm code, I assume the intention is that we can transparently migrate between filtered bridges and non-filtered bridges, right?
Right.
Quoted 3 lines
Thanks,
Alex
Quoted 35 lines · Michael S. Tsirkin
Quoted 25 lines · Alex Williamson
Quoted 24 lines · Michael S. Tsirkin
hw/virtio-net.c +3
-1@@ -47,6 +47,7 @@ typedef struct VirtIONetuint8_t nomulti;uint8_t nouni;uint8_t nobcast; +uint32_t filtering;struct {int in_use;int first_multi;@@ -475,12 +476,17 @@ static int receive_filter(VirtIONet *n, const uint8_t *buf, int size) ptr +=sizeof (struct virtio_net_hdr ); } -if (!memcmp (&ptr[12 ], vlan,sizeof (vlan))) { +if ((n->filtering & (0x1 <<NICCONF_F_VLAN_FILTERING )) && + !memcmp (&ptr[12 ], vlan,sizeof (vlan))) {int vid =be16_to_cpup ((uint16_t *)(ptr +14 )) &0xfff ;if (!(n->vlans[vid >>5 ] & (1U << (vid &0x1f ))))return 0 ; }Quoted 8 lines
hw/virtio-net.c static int receive_filter(VirtIONet *n, const uint8_t *buf, int size) · line 489
Quoted 4 lines
hw/virtio-net.c +4
-0+if (!(n->filtering & (0x1 <<NICCONF_F_MAC_FILTERING ))) { +return 1 ; + } +A filtering flags bitmap is a logical choice here, but I found the overhead to be non-trivial, which is why we have separate variables for the other filtering options.
Quoted 1 line
You suggest more flags for multicast etc?
I'm suggesting we may get slightly better performance if we use separate filter_mac and filter_vlan variable flags instead of a single "filtering" flags bitmap. However, a couple other ideas... Should we call receive_filter() as a function pointer so we can make filter specific versions or remove it completely? Some overhead to calling it as a pointer, but could still be a win. Alternatively we could make "effective" filter flags which are used by receive_filter(), but maintained separately from the guest requested flags. For instance:
virtio_net_handle_rx_mode(...)
{
...n->alluni_effective = (n->filtering & (0x1 << NICCONF_F_MAC_FILTERING)) ? n->alluni : 1; n->allmulti_effective = (n->filtering & (0x1 << NICCONF_F_MAC_FILTERING)) ? n->allmulti : 1;
These could be recreated on loadvm, so we still wouldn't need to save them. Question would be whether that creates a sufficiently fast path through receive_filter(). We'd still need a new flag for vlan filtering or maybe make the vlan header match bytes settable.
Alex
Quoted 36 lines · Alex Williamson
Quoted 35 lines · Michael S. Tsirkin
Quoted 33 lines · Alex Williamson
Quoted 18 lines · Michael S. Tsirkin
hw/virtio-net.c +2
-1@@ -47,6 +47,7 @@ typedef struct VirtIONetuint8_t nomulti;uint8_t nouni;uint8_t nobcast; +uint32_t filtering;struct {int in_use;int first_multi;@@ -475,12 +476,17 @@ static int receive_filter(VirtIONet *n, const uint8_t *buf, int size) ptr +=sizeof (struct virtio_net_hdr ); } -if (!memcmp (&ptr[12 ], vlan,sizeof (vlan))) { + if ((n->filtering & (0x1 << NICCONF_F_VLAN_FILTERING)) &&hw/virtio-net.c static int receive_filter(VirtIONet *n, const uint8_t *buf, int size) · line 489
Quoted 10 lines
hw/virtio-net.c +5
-0+ !memcmp(&ptr[12], vlan, sizeof(vlan))) {int vid =be16_to_cpup ((uint16_t *)(ptr +14 )) &0xfff ;if (!(n->vlans[vid >>5 ] & (1U << (vid &0x1f ))))return 0 ; } +if (!(n->filtering & (0x1 <<NICCONF_F_MAC_FILTERING ))) { +return 1 ; + } +A filtering flags bitmap is a logical choice here, but I found the overhead to be non-trivial, which is why we have separate variables for the other filtering options.
Quoted 5 lines
Quoted 1 line
You suggest more flags for multicast etc?
I'm suggesting we may get slightly better performance if we use separate
filter_macandfilter_vlanvariable flags instead of a single "filtering" flags bitmap.
Why? It's a single operation anyway, and we use less cache.
Quoted 13 lines
However, a couple other ideas... Should we
callreceive_filter()as a function pointer so we can make filter specific versions or remove it completely? Some overhead to calling it as a pointer, but could still be a win. Alternatively we could make "effective" filter flags which are used byreceive_filter(), but maintained separately from the guest requested flags. For instance:
virtio_net_handle_rx_mode(...)
{
...n->alluni_effective= (n->filtering& (0x1<<NICCONF_F_MAC_FILTERING)) ?n->alluni: 1;n->allmulti_effective= (n->filtering& (0x1<<NICCONF_F_MAC_FILTERING)) ?n->allmulti: 1;
Quoted 6 lines
These could be recreated on loadvm, so we still wouldn't need to save them. Question would be whether that creates a sufficiently fast path through
receive_filter(). We'd still need a new flag for vlan filtering or maybe make the vlan header match bytes settable.Alex
I agree, merging flags set by guest and by host makes sense, this way we don't need to test both. E.g. all filtering off is equivalent to promisc mode, we test that already. More complex ideas would I guess need to be benchmarked to be shown being worth it. So I'll do the simple thing and we can tweak it further later on.
Signature · 1 line
MST
Quoted 7 lines · Michael S. Tsirkin
Quoted 6 lines · Alex Williamson
Quoted 5 lines · Michael S. Tsirkin
Quoted 3 lines · Alex Williamson
A filtering flags bitmap is a logical choice here, but I found the overhead to be non-trivial, which is why we have separate variables for the other filtering options.
Quoted 7 lines
Quoted 5 lines
Quoted 1 line
You suggest more flags for multicast etc?
I'm suggesting we may get slightly better performance if we use separate
filter_macandfilter_vlanvariable flags instead of a single "filtering" flags bitmap.Why? It's a single operation anyway, and we use less cache.
Sorry, I must have been thinking of a runtime vs compile time shift.
Alex