Simplify network setup
v5 cover Simplify network setup
Hi,
another version of tst_net_vars network setup helper.v4->v5:
* Clenup unneded variables (reverse IP related).
* Don't require to have mask in IP address. Use netlink to detect prefix when address is presented on some of the NIC on the system or fallback to default ones (24 for IPv4, 64 for IPv6).
* Fix detecting wrong subnet with network not round up by 8 (IPv4 resp 16 (IPv6). E.g.: problem IP addressed like 10.2.0.1/22 10.2.1.1/22.
* Add more environment variables (e.g. prefix, whole IP address)
* Rewritten (again) unused network generation to fix overlap with used IP addresses.
* Simplify tst_ipaddr() (in testcases/lib/test_net.sh).
* Use tst_brk() to exit on error.
* Cleanup warnings and formatting.
There is still requirement network prefix to be < 8 (IPv4) resp. 16 (IPv6) as then IPV{4,6}_NETWORK variables would be empty, which is not expected by tests (they join it with address separators with {L,R}HOST_IPV{4,6}_HOST. Even we got rid of reverse related variables we still need IPV{4,6}_NETWORK variables for some tests.
BTW: I probably have source lines > 80 chars. My Vim doesn't highlight anything, but checkpatch.pl complain. I might have wrong vim configuration.
Petr Vorel (3):
gitignore: Prefixing filenames
network: Add tool for setup IP variables
network: Use tst_net_vars to set up IPv4 and IPv6 related variables
testcases/lib/.gitignore | 13 +- testcases/lib/Makefile | 2 +- testcases/lib/test_net.sh | 43 +-- testcases/lib/tst_net_vars.c | 760 +++++++++++++++++++++++++++++++++++++++++++ 4 files changed, 781 insertions(+), 37 deletions(-) create mode 100644 testcases/lib/tst_net_vars.c
Signature · 1 line
2.12.2
RFC PATCH v5 1/3 gitignore: Prefixing filenames
--- testcases/lib/.gitignore | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-)
testcases/lib/.gitignore +6 -6
@@ -1,6 +1,6 @@ -tst_sleep -tst_random -tst_checkpoint -tst_rod -tst_kvcmp -tst_device +/tst_sleep +/tst_random +/tst_checkpoint +/tst_rod +/tst_kvcmp +/tst_device
Signature · 1 line
2.12.2
RFC PATCH v5 2/3 network: Add tool for setup IP variables
In reply to Petr VorelThese environment variables are exported by setup_network_variables:IPV4_LHOST: IPv4 address of the local hostIPV4_RHOST: IPv4 address of the local hostIPV4_NETWORK: IPv4 common part of IPV4_LHOST and IPV4_RHOST LHOST_IPV4_HOST IPv4 unique part of IPV4_LHOST RHOST_IPV4_HOST IPv4 unique part of IPV4_RHOSTIPV4_NET16_UNUSED: IPv4 16 bit unused subnetIPV4_PREFIX: IPv4 prefixIPV6_LHOST: IPv6 address of local hostIPV6_RHOST: IPv6 address of the remote hostIPV6_NETWORK: IPv6 common part of IPV6_LHOST and IPV6_RHOSTLHOST_IPV6_HOST: IPv6 unique part of IPV6_LHOSTRHOST_IPV6_HOST: IPv6 unique part of IPV6_RHOSTIPV6_NET32_UNUSED: IPv6 32 bit unused subnetIPV6_PREFIX: IPv6 prefix
--- testcases/lib/.gitignore | 1 + testcases/lib/Makefile | 2 +- testcases/lib/tst_net_vars.c | 760 +++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 762 insertions(+), 1 deletion(-) create mode 100644 testcases/lib/tst_net_vars.c
testcases/lib/.gitignore +1 -0
@@ -4,3 +4,4 @@ /tst_rod /tst_kvcmp /tst_device +/tst_net_vars
testcases/lib/Makefile +1 -1
@@ -27,6 +27,6 @@ include $(top_srcdir)/include/mk/testcases.mk INSTALL_TARGETS := *.sh MAKE_TARGETS := tst_sleep tst_random tst_checkpoint tst_rod tst_kvcmp\ - tst_device + tst_device tst_net_vars include $(top_srcdir)/include/mk/generic_leaf_target.mk
testcases/lib/tst_net_vars.c new file +760 -0
760 lines not on this page · raw message
Signature · 1 line
2.12.2
RFC PATCH v5 2/3 network: Add tool for setup IP variables
Hi,
Just a couple of minor comments, I haven't properly read it.
testcases/lib/tst_net_vars.c line 127
Quoted 8 lines · On Wed, 3 May 2017 18:07:50 +0200 "Petr Vorel" <[email protected]> wrote:
testcases/lib/tst_net_vars.c new file +8
-0+/* +* Function prefix2mask is from ipcalc project, ipcalc.c. +*/ +struct in_addr prefix2mask (int prefix) +{ +struct in_addr mask; + +memset (&mask,0 ,sizeof (mask));
FYI for these memsets you could use 'struct foo bar = { 0 }' instead. Although it generates warnings on older compilers if the struct starts with another composite data type.
Quoted 16 lines
testcases/lib/tst_net_vars.c new file +16
-0+ +static int read_prefix (const char *ip_str,int is_ipv6) +{ +uint8_t family = is_ipv6 ?AF_INET6 :AF_INET ; + +char buf[16384 ]; +int len; + +struct { +struct nlmsghdr nlhdr; +struct ifaddrmsg addrmsg; + } msg; + +struct nlmsghdr *retmsg; + +int sock =socket (AF_NETLINK ,SOCK_RAW ,NETLINK_ROUTE );
testcases/lib/tst_net_vars.c line 499
Quoted 10 lines
testcases/lib/tst_net_vars.c new file +10
-0+ +memset (&msg,0 ,sizeof (msg)); + msg.nlhdr.nlmsg_len =NLMSG_LENGTH (sizeof (struct ifaddrmsg )); + msg.nlhdr.nlmsg_flags =NLM_F_REQUEST |NLM_F_ROOT ; + msg.nlhdr.nlmsg_type =RTM_GETADDR ; + msg.addrmsg.ifa_family = family; + +send (sock, &msg, msg.nlhdr.nlmsg_len,0 ); + len =recv (sock, buf,sizeof (buf),0 ); + retmsg = (struct nlmsghdr *)buf;
No error checking? I suppose it is unlikely to fail, but it can't hurt.
Quoted 16 lines
testcases/lib/tst_net_vars.c new file +16
-0+ + while NLMSG_OK(retmsg, len) { +struct ifaddrmsg *retaddr; +struct rtattr *retrta; +char pradd[128 ]; +int attlen; + + retaddr = (struct ifaddrmsg *)NLMSG_DATA (retmsg); + retrta = (struct rtattr *)IFA_RTA (retaddr); + attlen =IFA_PAYLOAD (retmsg); + + while RTA_OK(retrta, attlen) { +if (retrta->rta_type ==IFA_ADDRESS ) { +inet_ntop (family,RTA_DATA (retrta), pradd,sizeof (pradd)); +if (!strcmp (pradd, ip_str)) +return retaddr->ifa_prefixlen;
testcases/lib/tst_net_vars.c line 525
Quoted 10 lines
testcases/lib/tst_net_vars.c new file +10
-0+ } + retrta =RTA_NEXT (retrta, attlen); + + } + retmsg =NLMSG_NEXT (retmsg, len); + } + +return -1 ; +} +
Thank you,
Richard.
RFC PATCH v5 2/3 network: Add tool for setup IP variables
Hi Richard,
Quoted 7 lines
testcases/lib/tst_net_vars.c line 127
Quoted 3 lines
testcases/lib/tst_net_vars.c new file +3
-0+struct in_addr mask; + +memset (&mask,0 ,sizeof (mask));FYI for these memsets you could use 'struct foo bar = { 0 }' instead. Although it generates warnings on older compilers if the struct starts with another composite data type.
Right, I used it for struct in_addr and in6_addr. Not used it for struct msg as it contains another struct (even it does not produce a warning when compiled with --std=gnu89).
Quoted 7 lines
Quoted 4 lines
testcases/lib/tst_net_vars.c new file +4
-0+struct { +struct nlmsghdr nlhdr; +struct ifaddrmsg addrmsg; + } msg;
Quoted 10 lines
testcases/lib/tst_net_vars.c line 499
Quoted 8 lines
testcases/lib/tst_net_vars.c new file +8
-0+ msg.nlhdr.nlmsg_len =NLMSG_LENGTH (sizeof (struct ifaddrmsg )); + msg.nlhdr.nlmsg_flags =NLM_F_REQUEST |NLM_F_ROOT ; + msg.nlhdr.nlmsg_type =RTM_GETADDR ; + msg.addrmsg.ifa_family = family; + +send (sock, &msg, msg.nlhdr.nlmsg_len,0 ); + len =recv (sock, buf,sizeof (buf),0 ); + retmsg = (struct nlmsghdr *)buf;No error checking? I suppose it is unlikely to fail, but it can't hurt.
IMHO NLMSG_OK RTA_NEXT macros do some sort of checks [1], [2]. Sample code [3] checks for more, but that's IMHO irrelevant.
Quoted 21 lines
Quoted 21 lines
testcases/lib/tst_net_vars.c new file +21
-0+ + while NLMSG_OK(retmsg, len) { +struct ifaddrmsg *retaddr; +struct rtattr *retrta; +char pradd[128 ]; +int attlen; + + retaddr = (struct ifaddrmsg *)NLMSG_DATA (retmsg); + retrta = (struct rtattr *)IFA_RTA (retaddr); + attlen =IFA_PAYLOAD (retmsg); + + while RTA_OK(retrta, attlen) { +if (retrta->rta_type ==IFA_ADDRESS ) { +inet_ntop (family,RTA_DATA (retrta), pradd,sizeof (pradd)); +if (!strcmp (pradd, ip_str)) +return retaddr->ifa_prefixlen; + } + retrta =RTA_NEXT (retrta, attlen); + + } + retmsg =NLMSG_NEXT (retmsg, len);
Quoted 8 lines
testcases/lib/tst_net_vars.c line 107
Quoted 5 lines
testcases/lib/tst_net_vars.c new file +5
-0+ } + +return -1 ; +} +Thank you,
Richard.
Thanks for your comments and time!
Petr
[1] http://man7.org/linux/man-pages/man3/netlink.3.html
[2] http://man7.org/linux/man-pages/man3/rtnetlink.3.html
[3] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/samples/bpf/bpf_load.c?id=refs/tags/v4.11#n535
RFC PATCH v5 3/3 network: Use tst_net_vars to set up IPv4 and IPv6 related variables
In reply to Petr Voreltst_net_vars simplifies setup as it only needs to have passed 4 IP addresses (2 IPv4 and 2 IPv6).
--- testcases/lib/test_net.sh | 43 +++++++++++++------------------------------ 1 file changed, 13 insertions(+), 30 deletions(-)
testcases/lib/test_net.sh +13 -30
@@ -191,18 +191,11 @@ tst_ipaddr() { local type="${1:-lhost}" local ipv="${TST_IPV6:-4}" - local tst_host= - - if [ "$type" = "lhost" ]; then - eval "tst_host=\$LHOST_IPV${ipv}_HOST" - else - eval "tst_host=\$RHOST_IPV${ipv}_HOST" - fi if [ "$TST_IPV6" ]; then - echo "${IPV6_NETWORK}:${tst_host}" + [ "$type" = "lhost" ] && echo "$IPV6_LHOST" || echo "$IPV6_RHOST" else - echo "${IPV4_NETWORK}.${tst_host}" + [ "$type" = "lhost" ] && echo "$IPV4_LHOST" || echo "$IPV4_RHOST" fi }
@@ -242,8 +235,8 @@ tst_add_ipaddr() local type="${1:-lhost}" local link_num="${2:-0}" - local mask=24 - [ "$TST_IPV6" ] && mask=64 + local mask=$IPV4_PREFIX + [ "$TST_IPV6" ] && mask=$IPV6_PREFIX local iface=$(tst_iface $type $link_num)
@@ -487,25 +480,15 @@ export PASSWD="${PASSWD:-}" # Don't use it in new tests, use tst_rhost_run() from test_net.sh instead. export LTP_RSH="${LTP_RSH:-rsh -n}" -# Test Links -# Set first three octets of the network address, default is '10.0.0' -export IPV4_NETWORK="${IPV4_NETWORK:-10.0.0}" -# Set local host last octet, default is '2' -export LHOST_IPV4_HOST="${LHOST_IPV4_HOST:-2}" -# Set remote host last octet, default is '1' -export RHOST_IPV4_HOST="${RHOST_IPV4_HOST:-1}" -# Set the reverse of IPV4_NETWORK -export IPV4_NET_REV="${IPV4_NET_REV:-0.0.10}" -# Set first three octets of the network address, default is 'fd00:1:1:1' -export IPV6_NETWORK="${IPV6_NETWORK:-fd00:1:1:1}" -# Set local host last octet, default is '2' -export LHOST_IPV6_HOST="${LHOST_IPV6_HOST:-:2}" -# Set remote host last octet, default is '1' -export RHOST_IPV6_HOST="${RHOST_IPV6_HOST:-:1}" - -# Networks that aren't reachable through the test links -export IPV4_NET16_UNUSED="${IPV4_NET16_UNUSED:-10.23}" -export IPV6_NET32_UNUSED="${IPV6_NET32_UNUSED:-fd00:23}" +# tst_net_vars exports environment variables related to test links + networks +# that aren't reachable through the test links. +# See tst_net_vars -h for a list of exported variables. +IPV4_LHOST="${IPV4_LHOST:-10.0.0.2}" +IPV4_RHOST="${IPV4_RHOST:-10.0.0.1}" +eval `tst_net_vars $IPV4_LHOST $IPV4_RHOST` +IPV6_LHOST="${IPV6_LHOST:-fd00:1:1:1::2}" +IPV6_RHOST="${IPV6_RHOST:-fd00:1:1:1::1}" +eval `tst_net_vars $IPV6_LHOST $IPV6_RHOST` export HTTP_DOWNLOAD_DIR="${HTTP_DOWNLOAD_DIR:-/var/www/html}" export FTP_DOWNLOAD_DIR="${FTP_DOWNLOAD_DIR:-/var/ftp}"
Signature · 1 line
2.12.2
v5 cover Simplify network setup
In reply to Petr VorelQuoted 16 lines · Petr Vorel
Hi,
another version of
tst_net_varsnetwork setup helper.v4->v5:
* Clenup unneded variables (reverse IP related).
* Don't require to have mask in IP address. Use netlink to detect prefix when address is presented on some of the NIC on the system or fallback to default ones (24 for IPv4, 64 for IPv6).
* Fix detecting wrong subnet with network not round up by 8 (IPv4 resp 16 (IPv6). E.g.: problem IP addressed like 10.2.0.1/22 10.2.1.1/22.
* Add more environment variables (e.g. prefix, whole IP address)
* Rewritten (again) unused network generation to fix overlap with used IP addresses.
* Simplifytst_ipaddr()(intestcases/lib/test_net.sh).
* Usetst_brk()to exit on error.
* Cleanup warnings and formatting.
Quoted 5 lines
There is still requirement network prefix to be < 8 (IPv4) resp. 16 (IPv6) as then IPV{4,6}_NETWORK variables would be empty, which is not expected by tests (they join it with address separators with {L,R}HOST_IPV{4,6}_HOST. Even we got rid of reverse related variables we still need IPV{4,6}_NETWORK variables for some tests.
Hi Petr,
But you removed such dependency on the prefix from test_net.sh in the third patch... so we could just print the warning that configuration is not appropriate for the old tests, that is for stress/multicast, stress/route groups only.
Thanks,
Alexey
v5 cover Simplify network setup
Quoted 22 lines · Alexey Kodanev
Quoted 16 lines · Petr Vorel
Hi,
another version of
tst_net_varsnetwork setup helper.v4->v5:
* Clenup unneded variables (reverse IP related).
* Don't require to have mask in IP address. Use netlink to detect prefix when address is presented on some of the NIC on the system or fallback to default ones (24 for IPv4, 64 for IPv6).
* Fix detecting wrong subnet with network not round up by 8 (IPv4 resp 16 (IPv6). E.g.: problem IP addressed like 10.2.0.1/22 10.2.1.1/22.
* Add more environment variables (e.g. prefix, whole IP address)
* Rewritten (again) unused network generation to fix overlap with used IP addresses.
* Simplifytst_ipaddr()(intestcases/lib/test_net.sh).
* Usetst_brk()to exit on error.
* Cleanup warnings and formatting.Quoted 5 lines
There is still requirement network prefix to be < 8 (IPv4) resp. 16 (IPv6) as then IPV{4,6}_NETWORK variables would be empty, which is not expected by tests (they join it with address separators with {L,R}HOST_IPV{4,6}_HOST. Even we got rid of reverse related variables we still need IPV{4,6}_NETWORK variables for some tests.
Quoted 6 lines
Hi Petr,
But you removed such dependency on the prefix from
test_net.shin the third patch... so we could just print the warning that configuration is not appropriate for the old tests, that is for stress/multicast, stress/route groups only.
Regarding IPV{4,6}_NETWORK calculation, I believe it would much simpler ifwe saved the same part of address from IPV{4,6}_HOST variables to_NETWORK variable and what is differ to the other variables.There is no need to any additional checks for these environment variables or I'm missing something?
Hmm, I didn't find how the patch is getting network prefix on the remote host, is it because of the assumption that they should be the same on both hosts? It is fine for now as it's not so common for a test environment to have different ones, so the better approach in the future is to make a separate tool (installed on both hosts with LTP) and get prefixes from local and remote host similar to tst_get_hwaddrs() in test_net.sh.
Thanks,
Alexey
v5 cover Simplify network setup
Hi Alexey,
Quoted 5 lines
Regarding IPV{4,6}_NETWORK calculation, I believe it would much simpler ifwe saved the same part of address from IPV{4,6}_HOST variables
to_NETWORKvariable and what is differ to the other variables.There is no need to any additional checks for these environment variables or I'm missing something?
You're right - calculating just network part and then get host part by substring operation will be probably less error-prone. I planned to write some tests latter (sorry for not including it now), but this is really better solution.
Quoted 3 lines
Hmm, I didn't find how the patch is getting network prefix on the remote host, is it because of the assumption that they should be the same on both hosts?
Yes. So far we have just one IPV{4,6}_NETWORK variable for both, so I guess this makes us to use the same network for both.
Quoted 5 lines
It is fine for now as it's
not so common for a test environment to have different ones, so the better approach in the future is to make a separate tool (installed on both hosts with LTP) and get prefixes from local and remote host similar totst_get_hwaddrs()intest_net.sh.
I'm ok to make C binary which reads prefix, which will be run on both local and remote hosts. But to be actually used, we'd need to change all tests using IPV{4,6}_NETWORK to use different variables for local and remote hosts. I'd like to do it after having this merged. I have also in my TODO list creating tst_net_reverse_ip binary for stress/multicast and stress/route tests groups.
BTW is there a better/simpler way to get prefix than what I wrote?
Kind regards,
Petr
v5 cover Simplify network setup
In reply to Alexey KodanevHi Alexey,
Quoted 6 lines
Quoted 5 lines
There is still requirement network prefix to be < 8 (IPv4) resp. 16 (IPv6) as then IPV{4,6}_NETWORK variables would be empty, which is not expected by tests (they join it with address separators with {L,R}HOST_IPV{4,6}_HOST. Even we got rid of reverse related variables we still need IPV{4,6}_NETWORK variables for some tests.
Quoted 6 lines
Hi Petr,
But you removed such dependency on the prefix from
test_net.shin the third patch... so we could just print the warning that configuration is not appropriate for the old tests, that is for stress/multicast, stress/route groups only.
Right, I'll remove requirement network prefix to be < 8 (IPv4) resp. 16 (IPv6). There are other restrictions you probably don't like:
* Don't use 32 (IPv4) and 128 (IPv6): I'd keep this restriction. What do you think?
* Require to use the same prefix: I'd keep it for now.
Kind regards,
Petr
v5 cover Simplify network setup
Quoted 12 lines · Petr Vorel
Hi Alexey,
Quoted 10 lines
Quoted 5 lines
There is still requirement network prefix to be < 8 (IPv4) resp. 16 (IPv6) as then IPV{4,6}_NETWORK variables would be empty, which is not expected by tests (they join it with address separators with {L,R}HOST_IPV{4,6}_HOST. Even we got rid of reverse related variables we still need IPV{4,6}_NETWORK variables for some tests.
Hi Petr,
But you removed such dependency on the prefix fromtest_net.shin the third patch... so we could just print the warning that configuration is not appropriate for the old tests, that is for stress/multicast, stress/route groups only.
Quoted 4 lines
Right, I'll remove requirement network prefix to be < 8 (IPv4) resp. 16 (IPv6). There are other restrictions you probably don't like:
* Don't use 32 (IPv4) and 128 (IPv6): I'd keep this restriction. What do you think?
* Require to use the same prefix: I'd keep it for now.
I'm for a warning message that some network/stress tests not supporting such config.
Thanks,
Alexey
v5 cover Simplify network setup
Hi Alexey,
Quoted 7 lines
Quoted 4 lines
Right, I'll remove requirement network prefix to be < 8 (IPv4) resp. 16 (IPv6). There are other restrictions you probably don't like:
* Don't use 32 (IPv4) and 128 (IPv6): I'd keep this restriction. What do you think?
* Require to use the same prefix: I'd keep it for now.I'm for a warning message that some network/stress tests not supporting such config.
Although I think prefix (mask) 32 resp. 128 doesn't make much sense, I'll remove both restrictions. If I have enough time I'll implement use of different network variables for local and remote hosts.
Thanks for your time and comments!
Kind regards,
Petr
v5 cover Simplify network setup
In reply to Petr VorelHi,
Quoted 2 lines · On Wed, 3 May 2017 18:07:48 +0200 "Petr Vorel" <[email protected]> wrote:
BTW: I probably have source lines > 80 chars. My Vim doesn't highlight anything, but checkpatch.pl complain. I might have wrong vim configuration.
Are you using checkpatch.pl from kernel 4.11? It was recently changed to follow the actual style guidelines which say lines over 80 chars are preferable to strings being split over multiple lines. Maybe Vim is aware of that?
Thank you,
Richard.
v5 cover Simplify network setup
Quoted 11 lines · Richard Palethorpe
Hi,
Quoted 2 lines · On Wed, 3 May 2017 18:07:48 +0200 "Petr Vorel" <[email protected]> wrote:
BTW: I probably have source lines > 80 chars. My Vim doesn't highlight anything, but checkpatch.pl complain. I might have wrong vim configuration.
Are you using checkpatch.pl from kernel 4.11? It was recently changed to follow the actual style guidelines which say lines over 80 chars are preferable to strings being split over multiple lines. Maybe Vim is aware of that?
Or may be vim's configuration has wrong tab size, should be 8 characters for Linux coding style.
Thanks,
Alexey
v5 cover Simplify network setup
Hi Alexey, Richard,
Quoted 9 lines
Quoted 6 lines
Quoted 2 lines
BTW: I probably have source lines > 80 chars. My Vim doesn't highlight anything, but checkpatch.pl complain. I might have wrong vim configuration.
Are you using checkpatch.pl from kernel 4.11? It was recently changed to follow the actual style guidelines which say lines over 80 chars are preferable to strings being split over multiple lines. Maybe Vim is aware of that?
Or may be vim's configuration has wrong tab size, should be 8 characters for Linux coding style.
I'm using recent version of checkpatch.pl.
My fault, I've installed linuxsty plugin [1], but default is off, so I had these options:
tabstop=4 softtabstop=4 shiftwidth=4
[1]: https://github.com/vivien/vim-linux-coding-style/blob/master/plugin/linuxsty.vim
Thank you both for tips!
Petr