IPVS: Backup Adding Ipv6 and Persistence support
v3 cover IPVS: Backup Adding Ipv6 and Persistence support
This patch series adds/(updates) the following functionality in the synchronization between master and backup daemons.
- IPv6
- Persistence Engine
- Firewall marks transferred
- Time-outs transferred.
- Flag field increased to 32 bits.
Note:
This patches depends on Patch 1-9 in staging branch:
"IPVS: Backup, Prepare for transferring firewall marks (fwmark) to the backup daemon." to ...
"IPVS: skb defrag in L7 helpers"
A new message format is also introduced, not understood by old backup daemons. For compatibility reasons receiving the old version (version 0) is still possible. Old (version 0) backups will just drop new (Version 1) messages.
It's also possible to send sync msg in version 0 format, by sysctl #sysclt -w net.ipv4.vs.sync_version=0
Update scenario:
Update the Machine with Bakup daemon first.
Message structure:
A new 32 bit word is added to the header,
where the old count_Conns is set to 0 so old Backup daemons just will drop the packet. Added is version (1) and there is also a spare 16 bit field at the end. This is a reworked version based upon Simon Hormans work.
Version 1:
Note, first byte should be Zero, so ver 0 receivers will drop the packet.
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| 0 | SyncID | Size |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Count Conns | Version | Reserved, set to Zero |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| |
| IPVS Sync Connection (1) |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| . |
~ . ~
| . |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| |
| IPVS Sync Connection (n) |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Version 0 Header
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Count Conns | SyncID | Size |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| IPVS Sync Connection (1) |
The Sync. Conn. message format is also changed to allow future enhancements and versions.
Sync Connection format (sync_conn)
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Type | Protocol | Ver. | Size |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Flags |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| State | cport |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| vport | dport |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| fwmark |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| timeout (in sec.) |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| ... |
| IP-Addresses (v4 or v6) |
| ... |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Optional Parameters.
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Param. Type | Param. Length | Param. data |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |
| ... |
| +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| | Param Type | Param. Length |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Param data |
| Last Param data should be padded for 32 bit alignment |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+New fields that might need some explanation,
Type:
Bit 7 6 . . . 1 0
+--------------------------------------------------+-------+
| Spare | IPv6 |
+--------------------------------------------------+-------+Bit 0 = 1 This sync_conn contains IPv6 addresses.
Ver. Version of specified type right now it's 0
fwmark: Firewall mark from skb.
timeout: from ip_vs_conn struct, converted to sec.
Optional Parameters will be added at the end without any kind of alignment each Param will have a header of two bytes
Param. header
Bit 7 6 . . . 0 7 0
+----------+----------------------+---------------------------+ | Optional | Parameter Type | Parameter length | +----------+----------------------+---------------------------+ | Data ... | | ... | +----------+----------------------+---------------------------+
Bit 7 (msb) Optional Param, conn. entry could be keept, if not type
is unknown.
Bit 6-0 Option Type defined below
#define IPVS_OPT_SEQ_DATA 1 #define IPVS_OPT_PE_DATA 2 #define IPVS_OPT_PE_NAME 3
Option Length, 1-255 bytes 0 is an illegal value.
PATCH STATUS:
- Persistence data is not tested.
THANKS
To Julian and Simon for the review and alternative solutions.
*v3
New: Ability to send ver. 0 sync messages.
Optional data, renamed to Optional parameter
Corrections, see comments in each patch,
*v2
Simplified fwmark handling patch 1/4
New handling of optional data patch 2/4
Seconds in timeout
Basically all changes is based on Julians and Simons comments on the RFC For details see individual patches.
--
Regards
Hans Schillstrom <[email protected]>
Julian Anastasov
maintainerHello,
Quoted 15 lines · Hans Schillstrom
This patch series adds/(updates) the following functionality in the synchronization between master and backup daemons.
- IPv6
- Persistence Engine
- Firewall marks transferred
- Time-outs transferred.
- Flag field increased to 32 bits.Note:
This patches depends on Patch 1-9 in staging branch:
"IPVS: Backup, Prepare for transferring firewall marks (fwmark) to the backup daemon." to ...
"IPVS: skb defrag in L7 helpers"
Quoted 5 lines
A new message format is also introduced, not understood by old backup daemons. For compatibility reasons receiving the old version (version 0) is still possible. Old (version 0) backups will just drop new (Version 1) messages.
It's also possible to send sync msg in version 0 format, by sysctl #sysclt -w net.ipv4.vs.sync_version=0
v3 PATCH 1/3
- ntoh_seq can use directly get_unaligned_be32, it is BE to CPU,
no need for ntohl
- In ip_vs_conn_fill_param_sync:
- should we ignore the conn entry when pe_name_len
is 0 and pe_data_len != 0 (return 1)? May be such checks:
/* Handle pe data */
if ((pe_data_len != 0) != (pe_name_len != 0)) {
IP_VS_DBG(3, "BACKUP, Invalid PE parameters\n");
return 1;
}
if (!pe_name_len)
return 0;
...
- make sure p->pe is _put when p->pe_data
allocation fails (-ENOMEM)
- ip_vs_proc_str: use 'unsigned int', not just 'unsigned'
- ip_vs_proc_sync_conn:
- pe_data_len, pe_name_len, pe_data, pe_name are
not initialized
- 'retc 10;' must be 'retc = 10;' ?
- inconsistent spaces:
'} else if (! s->v4.type) {'
'while (p< msg_end && *p && !retc ) {'
- please convert char *p, char *msg_end to '__u8 *' or
'unsigned char *' because this is risky for char * when
value is above 127:
int ptype = *(p++);
int plen = *(p++);
- add some sanity checks because 'p< msg_end' guarantees
that only ptype is present, for example:
while (p < msg_end) {
int ptype;
int plen;
if (p + 2 > msg_end)
return -30;
ptype = *(p++);
plen = *(p++);
- for IPVS_OPT_PE_NAME ip_vs_proc_str must be called
with IPVS_OPT_F_PE_NAME
- ip_vs_process_message:
- before 'size = ntohs(s->v4.ver_size) & SVER_MASK;'
there must be a check for present v4 structure:
if (msg_end + sizeof(s->v4) > buffer+buflen) {
IP_VS_ERR_RL("BACKUP, Dropping buffer, msg > buffer\n");
return;
}
size = ntohs(s->v4.ver_size) & SVER_MASK;
- we must account padding later after
ip_vs_proc_sync_conn is called, it should not be
included in size because the parameter
parsing will fail while walking the padding:
size += 3;
size &= ~3;
msg_end = p + size;
there is no need to check if new msg_end
exceeds buflen because if there are more
conn entries the above code will check for
present v4 structure.
Another option is the parameter parsing
to consider ptype=0 as 1 byte padding but
then there should be no plen octet for this
case. Then ver_size can include the padding.
For example:
while (p < msg_end) {
int ptype = *(p++);
int plen;
/* Padding ? */
if (!ptype)
continue;
if (p + 1 > msg_end)
return -30;
plen = *(p++);v3 PATCH 2/3
- hton_seq can use put_unaligned_be32
- ip_vs_sync_conn:
- move init of pe_name_len after sloop
- The 'Sanity checks' code must be after sloop
- we can use cp->pe instead of cp->dest->svc->pe
- before 'len = (len+3) & 0xffc;' may be
we have to calculate pad = (4 - len) & 3
so that we can use it later, len should not
include the padding, for example:
pad = (4 - len) & 3;
/* check if there is a space for this one */
if (curr_sb && (curr_sb->head+len+pad > curr_sb->end)) {
...
- then after all parameters:
curr_sb->head += len;
if (pad) {
memset(curr_sb->head, 0, pad);
curr_sb->head += pad;
}
or
curr_sb->head += len;
while (pad --)
*curr_sb->head ++ = 0;
then this code is not needed:
+ if (p < curr_sb->head)
+ *p = 0; /* Dont leave random bytes at end */
- use cp->pe for IPVS_OPT_PE_NAMEv3 PATCH 3/3
OK
Regards
--
Julian Anastasov <[email protected]>
Quoted 27 lines · On Saturday 13 November 2010 00:31:21 you wrote:
Hello,
Quoted 15 lines · Hans Schillstrom
This patch series adds/(updates) the following functionality in the synchronization between master and backup daemons.
- IPv6
- Persistence Engine
- Firewall marks transferred
- Time-outs transferred.
- Flag field increased to 32 bits.Note:
This patches depends on Patch 1-9 in staging branch:
"IPVS: Backup, Prepare for transferring firewall marks (fwmark) to the backup daemon." to ...
"IPVS: skb defrag in L7 helpers"Quoted 5 lines
A new message format is also introduced, not understood by old backup daemons. For compatibility reasons receiving the old version (version 0) is still possible. Old (version 0) backups will just drop new (Version 1) messages.
It's also possible to send sync msg in version 0 format, by sysctl #sysclt -w net.ipv4.vs.sync_version=0v3 PATCH 1/3
Review
Quoted 2 lines
(quoted diff) +0
-0- ntoh_seq can use directly get_unaligned_be32, it is BE to CPU, no need for ntohl
Tanks,
Quoted 4 lines
(quoted diff) +0
-0- In ip_vs_conn_fill_param_sync: - should we ignore the conn entry when pe_name_len
Review
Quoted 10 lines
(quoted diff) +0
-0is 0 and pe_data_len != 0 (return 1)? May be such checks: /* Handle pe data */ if ((pe_data_len != 0) != (pe_name_len != 0)) { IP_VS_DBG(3, "BACKUP, Invalid PE parameters\n"); return 1; } if (!pe_name_len) return 0;
It's OK, for me, what do you think Simon ?
Lets do it this way,
if (pe_data_len) {
if (pe_name_len) {
...
} else {
IP_VS_DBG(3, "BACKUP, Invalid PE parameters\n");
return 1;
}
...Review
Quoted 4 lines
(quoted diff) +0
-0- make sure p->pe is _put when p->pe_data allocation fails (-ENOMEM)
if (!p->pe_data) {
if (p->pe->module)
module_put(p->pe->module);
return -ENOMEM;
}Quoted 2 lines
-
ip_vs_proc_str: use 'unsigned int', not just 'unsigned'
OK
Review
Quoted 3 lines
(quoted diff) +0
-0- ip_vs_proc_sync_conn: - pe_data_len, pe_name_len, pe_data, pe_name are not initialized
Ooops, how did that happen :-?
Quoted 2 lines
- 'retc 10;' must be 'retc = 10;' ?
Yes it should, I wounder why the compiler didn't complain...
Review
Quoted 4 lines
(quoted diff) +0
-0- inconsistent spaces: '} else if (! s->v4.type) {' 'while (p< msg_end && *p && !retc ) {'
My fingers seems to like the space bar :-)
Review
Quoted 7 lines
(quoted diff) +0
-0- please convert char *p, char *msg_end to '__u8 *' or 'unsigned char *' because this is risky for char * when value is above 127: int ptype = *(p++); int plen = *(p++);
OK, I switch to __u8 in general;
Review
Quoted 2 lines
(quoted diff) +0
-0- add some sanity checks because 'p< msg_end' guarantees that only ptype is present, for example:
Thats true
Review
Quoted 13 lines
(quoted diff) +0
-0while (p < msg_end) { int ptype; int plen; if (p + 2 > msg_end) return -30; ptype = *(p++); plen = *(p++); - for IPVS_OPT_PE_NAME ip_vs_proc_str must be called with IPVS_OPT_F_PE_NAME
Oops
Review
Quoted 9 lines
(quoted diff) +0
-0- ip_vs_process_message: - before 'size = ntohs(s->v4.ver_size) & SVER_MASK;' there must be a check for present v4 structure: if (msg_end + sizeof(s->v4) > buffer+buflen) { IP_VS_ERR_RL("BACKUP, Dropping buffer, msg > buffer\n"); return; }
OK
Quoted 5 lines
(quoted diff) +0
-0size = ntohs(s->v4.ver_size) & SVER_MASK; - we must account padding later after ip_vs_proc_sync_conn is called, it should not be included in size because the parameter
Review
Quoted 10 lines
(quoted diff) +0
-0parsing will fail while walking the padding: size += 3; size &= ~3; msg_end = p + size; there is no need to check if new msg_end exceeds buflen because if there are more conn entries the above code will check for present v4 structure.
I think this one produce cleaner code,
Quoted 18 lines
(quoted diff) +0
-0Another option is the parameter parsing to consider ptype=0 as 1 byte padding but then there should be no plen octet for this case. Then ver_size can include the padding. For example: while (p < msg_end) { int ptype = *(p++); int plen; /* Padding ? */ if (!ptype) continue; if (p + 1 > msg_end) return -30; plen = *(p++);
Quoted 2 lines
v3 PATCH 2/3
-hton_seqcan useput_unaligned_be32
Yes
Review
Quoted 3 lines
(quoted diff) +0
-0- ip_vs_sync_conn: - move init of pe_name_len after sloop
Ooops
Quoted 2 lines
- The 'Sanity checks' code must be after sloop
Yes,
Quoted 1 line
- we can use
cp->peinstead ofcp->dest->svc->pe
OK
Quoted 41 lines
(quoted diff) +0
-0- before 'len = (len+3) & 0xffc;' may be we have to calculate pad = (4 - len) & 3 so that we can use it later, len should not include the padding, for example: pad = (4 - len) & 3; /* check if there is a space for this one */ if (curr_sb && (curr_sb->head+len+pad > curr_sb->end)) { - then after all parameters: curr_sb->head += len; if (pad) { memset(curr_sb->head, 0, pad); curr_sb->head += pad; } or curr_sb->head += len; while (pad --) *curr_sb->head ++ = 0; then this code is not needed: + if (p < curr_sb->head) + *p = 0; /* Dont leave random bytes at end */ - use cp->pe for IPVS_OPT_PE_NAMEv3 PATCH 3/3
OKRegards
--
Julian Anastasov <[email protected]>
Signature · 2 lines
Regards Hans Schillstrom <[email protected]>
Simon Horman
maintainerQuoted 59 lines · Hans Schillstrom
Quoted 27 lines · On Saturday 13 November 2010 00:31:21 you wrote:
Hello,
Quoted 15 lines · Hans Schillstrom
This patch series adds/(updates) the following functionality in the synchronization between master and backup daemons.
- IPv6
- Persistence Engine
- Firewall marks transferred
- Time-outs transferred.
- Flag field increased to 32 bits.Note:
This patches depends on Patch 1-9 in staging branch:
"IPVS: Backup, Prepare for transferring firewall marks (fwmark) to the backup daemon." to ...
"IPVS: skb defrag in L7 helpers"Quoted 5 lines
A new message format is also introduced, not understood by old backup daemons. For compatibility reasons receiving the old version (version 0) is still possible. Old (version 0) backups will just drop new (Version 1) messages.
It's also possible to send sync msg in version 0 format, by sysctl #sysclt -w net.ipv4.vs.sync_version=0v3 PATCH 1/3
Review
Quoted 2 lines
(quoted diff) +0
-0- ntoh_seq can use directly get_unaligned_be32, it is BE to CPU, no need for ntohlTanks,
Quoted 4 lines
(quoted diff) +0
-0- In ip_vs_conn_fill_param_sync: - should we ignore the conn entry when pe_name_lenReview
Quoted 10 lines
(quoted diff) +0
-0is 0 and pe_data_len != 0 (return 1)? May be such checks: /* Handle pe data */ if ((pe_data_len != 0) != (pe_name_len != 0)) { IP_VS_DBG(3, "BACKUP, Invalid PE parameters\n"); return 1; } if (!pe_name_len) return 0;It's OK, for me, what do you think Simon ?
Lets do it this way,if (pe_data_len) { if (pe_name_len) { ... } else { IP_VS_DBG(3, "BACKUP, Invalid PE parameters\n"); return 1; } ...Quoted 1 line
Quoted 8 lines
Review
Quoted 3 lines
(quoted diff) +0
-0- make sure p->pe is _put when p->pe_data allocation fails (-ENOMEM)if (!p->pe_data) { if (p->pe->module) module_put(p->pe->module); return -ENOMEM; }
I think that looks reasonable