Sending just ssh traffic via wg
Hi, all:
I'm trying to figure out the right PostUP/PostDown incantations to send just ssh traffic (on port tcp/22) via the wg tunnel, but I'm having a bit of a hard time. I should be able to do this with --set-mark for iptables/PREROUTING and a fwmark ip route rule, but it doesn't appear to be working as described in many guides I've consulted.
Has anyone done something like this before?
-K
_______________________________________________
WireGuard mailing list
[email protected]
https://lists.zx2c4.com/mailman/listinfo/wireguard
Hey Konstantin,
When you're doing policy routing with packets that are being forwarded by the system -- a router, for example -- then the prerouting table is sufficient. But for locally generated packets, you have to use the OUTPUT table and also probably MASQUERADE. I just reproduced everything here and confirm this works:
ip route add default dev wg0 table 2468
ip rule add fwmark 1234 table 2468
wg set wg0 peer [...] allowed-ips 0.0.0.0/0
sysctl net.ipv4.conf.wg0.rp_filter=0
iptables -t nat -A POSTROUTING -p tcp --dport 22 -m addrtype --src-type LOCAL -j MASQUERADE
iptables -t mangle -A OUTPUT -p tcp --dport 22 -j MARK --set-mark 1234
That works pretty well for me. If you don't want to disable rp_filter, you can do a little dance of setting and restoring the connmark in egress and ingress so that incoming packets are matched against that routing table too.
Alternatively, if your goal is actually to just send certain processes through the tunnel, you have three more options:
- Network namespaces, and then ip netns exec chicken ssh 1.2.3.4 ...
- VRFs, and then ip vrf exec chicken ssh 1.2.3.4 ...
- Cgroups and net_cls.
All three work well and are differently convenient depending on your needs. I wrote up the netns stuff on wireguard.com/netns/ but haven't gotten around to documenting VRFs and cgroups with wireguard, but they in fact should work the same as for every other situation that uses those, so any old tutorial will do.
Jason
_______________________________________________
WireGuard mailing list
[email protected]
https://lists.zx2c4.com/mailman/listinfo/wireguard
Quoted 8 lines · Jason A. Donenfeld
Hey Konstantin,
When you're doing policy routing with packets that are being forwarded by the system -- a router, for example -- then the prerouting table is sufficient. But for locally generated packets, you have to use the OUTPUT table and also probably MASQUERADE. I just reproduced everything here and confirm this works:
Quoted 7 lines
ip route add default dev wg0 table 2468
ip rule add fwmark 1234 table 2468
wg set wg0 peer [...] allowed-ips 0.0.0.0/0
sysctl net.ipv4.conf.wg0.rp_filter=0
iptables -t nat -A POSTROUTING -p tcp --dport 22 -m addrtype --src-type LOCAL -j MASQUERADE
iptables -t mangle -A OUTPUT -p tcp --dport 22 -j MARK --set-mark 1234
Any reason why you can't just do
ip rule add dport 22 lookup 2468
?
-Toke
_______________________________________________
WireGuard mailing list
[email protected]
https://lists.zx2c4.com/mailman/listinfo/wireguard
Quoted 18 lines · On Fri, Oct 5, 2018, 12:03 Toke Høiland-Jørgensen <[email protected]> wrote:
Quoted 8 lines · Jason A. Donenfeld
Hey Konstantin,
When you're doing policy routing with packets that are being forwarded by the system -- a router, for example -- then the prerouting table is sufficient. But for locally generated packets, you have to use the OUTPUT table and also probably MASQUERADE. I just reproduced everything here and confirm this works:
Quoted 7 lines
ip route add default dev wg0 table 2468
ip rule add fwmark 1234 table 2468
wg set wg0 peer [...] allowed-ips 0.0.0.0/0
sysctl net.ipv4.conf.wg0.rp_filter=0
iptables -t nat -A POSTROUTING -p tcp --dport 22 -m addrtype --src-type LOCAL -j MASQUERADE
iptables -t mangle -A OUTPUT -p tcp --dport 22 -j MARK --set-mark 1234
Quoted 4 lines
Any reason why you can't just do
ip rule add dport 22 lookup 2468
That's indeed the best by far as long as other netfilter fanciness isn't desired. Probably should set ipproto to tcp too in the rule.
Jason
Quoted 4 lines
?
-Toke
_______________________________________________
WireGuard mailing list
[email protected]
https://lists.zx2c4.com/mailman/listinfo/wireguard
Quoted 14 lines · Toke Høiland-Jørgensen
Quoted 6 lines
When you're doing policy routing with packets that are being forwarded by the system -- a router, for example -- then the prerouting table is sufficient. But for locally generated packets, you have to use the OUTPUT table and also probably MASQUERADE. I just reproduced everything here and confirm this works:
Quoted 7 lines
ip route add default dev wg0 table 2468
ip rule add fwmark 1234 table 2468
wg set wg0 peer [...] allowed-ips 0.0.0.0/0
sysctl net.ipv4.conf.wg0.rp_filter=0
iptables -t nat -A POSTROUTING -p tcp --dport 22 -m addrtype --src-type LOCAL -j MASQUERADE
iptables -t mangle -A OUTPUT -p tcp --dport 22 -j MARK --set-mark 1234
Quoted 3 lines
Any reason why you can't just do
ip rule add dport 22 lookup 2468
Yeah, this works, too, and is quite a bit simpler. Jason, any reason why I wouldn't use this?
To help explain the use-case here, here's what I'm trying to achieve.
We currently require that all admin-level access goes through 2-factor authentication. For ssh, we require using SmartCard-capable hardware tokens (Yubikey, Nitrokey), such that the private key for establishing a connection is never exposed to the main OS -- so all our ssh connections are inherently 2-factor authenticated.
However, that is not a mechanism we can use for accessing non-ssh things such as various web-management interfaces on internal networks. Therefore, we require that the VPN connection itself goes through 2-factor authentication step -- admins put in their password and their TOTP/HOTP token value when establishing a connection. This works, but is annoying to have to do when most of the time all an admin needs is ssh. Every time there is a network blip, the admin loses their OpenVPN link and, if they don't re-establish it quickly enough (typing in their username, password, TOTP token value), then their ssh sessions reset. Quite possibly the worst thing to happen to an admin in the middle of troubleshooting something.
Similarly, if there's an alert in the middle of the night that requires checking something out, it's annoying to have to first establish an OpenVPN connection before being able to ssh in to a system.
So, we're working on a new setup where admins would have an always-on WireGuard connection to the infra, but that connection only allows ssh traffic. In this case, don't need 2-factor on the wireguard link, just packet encapsulation. But should the admin need to bring up the OpenVPN link for accessing something like an iDrac interface on a Dell, they need to be able to do this without needing to shut down their WireGuard tunnel first (since both WG and OpenVPN provide routing to the same internal ip ranges). Therefore, I was looking for a way to *only* send port 22 traffic on the wg link.
The following achieves what we need:
[Interface]
PrivateKey = [omitted]
Address = [omitted]
DNS = 127.0.0.1
Table = 2468
PostUp = ip rule add to 10.10.0.0/16 dport 22 lookup 2468 PostDown = ip rule del to 10.10.0.0/16 dport 22 lookup 2468
[Peer]
PublicKey = [omitted]
AllowedIPs = 10.10.0.0/16
Endpoint = [omitted]
This achieves what we need *quite* nicely!
Thanks for your help!
-K
_______________________________________________
WireGuard mailing list
[email protected]
https://lists.zx2c4.com/mailman/listinfo/wireguard
Quoted 1 line · Konstantin Ryabitsev
But should the admin need to bring up the OpenVPN link
This may be a stupid question, but why do you need OpenVPN any more, if you have Wireguard?
I'd set up a simple server-side login page that allows people to use their user+pass+TOTP to enable non-SSH traffic on "their" link for the next N minutes, with an easily-clickable Refresh button (and a browser-based notification that the timeout is imminent), plus a small (= easily-verified-to-be-correct) backend that enables/disables your link's iptables rules. Problem solved.
Signature · 6 lines
-- Matthias Urlichs _______________________________________________ WireGuard mailing list [email protected] https://lists.zx2c4.com/mailman/listinfo/wireguard
Quoted 5 lines · Matthias Urlichs
Quoted 1 line · Konstantin Ryabitsev
But should the admin need to bring up the OpenVPN link
This may be a stupid question, but why do you need OpenVPN any more, if you have Wireguard?
Because it's already there? :)
Furthermore, some members of our IT team use macs (gasp!) and for them it would be much easier to continue to use OpenVPN than to set up wireguard-go.
Quoted 6 lines
I'd set up a simple server-side login page that allows people to use their user+pass+TOTP to enable non-SSH traffic on "their" link for the next N minutes, with an easily-clickable Refresh button (and a browser-based notification that the timeout is imminent), plus a small (= easily-verified-to-be-correct) backend that enables/disables your link's iptables rules. Problem solved.
Well, not quite. OpenVPN requires 2-factor validation each time there is a renegotiation -- so even if an attacker steals the private key and username/password of the administrator, they won't be able to establish a new session without having the TOTP secret as well. In the situation you describe, we create a window during which an attacker *would* be able to establish a VPN connection if they have the wireguard private key. We can make this window very short, but this would probably greatly annoy sysadmins, who will not enjoy needing to have a browser window open so they can re-validate. If we make this window sufficiently long not to annoy sysadmins, then this weakens our setup.
Furthermore, doing what you suggest would require writing a webapp and having some kind of queueing system for the backend process to read and adjust firewalls based on the data in the queue. It's a pretty complicated setup that seems orthogonal to the simplicity offered by WireGuard.
I've actually considered something like this -- to elevate their access privileges, an admin would need to simply ssh into the wireguard vpn server using their wireguard connection. A backend process could react to the ssh session being established and would apply additional firewall rules to give more access to that peer address. When that ssh session terminates, the same process would then drop these firewall rules back to their "ssh only" level. However, this still seems hacky and I would rather wait for Jason to come up with a More Proper way to do 2-factor, especially since the OpenVPN tunnel is already set up and functioning, so I don't have to invent anything to get what we need.
-K
_______________________________________________
WireGuard mailing list
[email protected]
https://lists.zx2c4.com/mailman/listinfo/wireguard
Hey Konstantin,
Quoted 6 lines · Konstantin Ryabitsev
Quoted 3 lines
Any reason why you can't just do
ip rule add dport 22 lookup 2468
Yeah, this works, too, and is quite a bit simpler. Jason, any reason why I wouldn't use this?
Definitely use that. A reason for preferring netfilter for this would be if you're doing lots of crazier netfilter stuff as well and want complex rules. But for just tcp:22 matching, Toke's suggestion is by far the best. I imagine internally, the kernel can just look into struct flowi during the route lookups and doen't need to do much subsequent parsing.
The one thing I'd change is you should add "ipproto tcp" to the command so you don't match udp:22 as well.
Quoted 10 lines
Every time there is a network blip, the admin loses their OpenVPN link and, if they don't re-establish it quickly enough (typing in their username, password, TOTP token value), then their ssh sessions reset. Quite possibly the worst thing to happen to an admin in the middle of troubleshooting something.
Similarly, if there's an alert in the middle of the night that requires checking something out, it's annoying to have to first establish an OpenVPN connection before being able to ssh in to a system.
Quoted 9 lines
So, we're working on a new setup where admins would have an always-on WireGuard connection to the infra, but that connection only allows ssh traffic. In this case, don't need 2-factor on the wireguard link, just packet encapsulation. But should the admin need to bring up the OpenVPN link for accessing something like an iDrac interface on a Dell, they need to be able to do this without needing to shut down their WireGuard tunnel first (since both WG and OpenVPN provide routing to the same internal ip ranges). Therefore, I was looking for a way to *only* send port 22 traffic on the wg link.
This seems like a reasonable and simple way of doing it. You could, instead, make a little ssh wrapper that does the netns/vrf/cgroup stuff if you wanted this at the process level, but probably the heuristic of ssh==22 is a totally good and acceptable one that will be less error prone.
By the way, hopefully as core development simmers down, I'll be able to focus a bit more on infrastructure projects like adding 2FA on top of wireguard.
Quoted 10 lines
The following achieves what we need:
[Interface]
PrivateKey = [omitted]
Address = [omitted]
DNS = 127.0.0.1
Table = 2468
PostUp = ip rule add to 10.10.0.0/16 dport 22 lookup 2468 PostDown = ip rule del to 10.10.0.0/16 dport 22 lookup 2468
Quoted 6 lines
[Peer]
PublicKey = [omitted]
AllowedIPs = 10.10.0.0/16
Endpoint = [omitted]This achieves what we need *quite* nicely!
I've add this example to the wg-quick(8) man page:
https://git.zx2c4.com/WireGuard/commit/?id=3e2f5495ea684d7f06fbefc50290e7d8985fc3de
Regards,
Jason
_______________________________________________
WireGuard mailing list
[email protected]
https://lists.zx2c4.com/mailman/listinfo/wireguard