Skip to contentSkip to conversation

net: WireGuard secure network tunnel

RFC PATCH In review Jason A. Donenfeld 5 participants 8 messages Nov 20, 2019 – Nov 27, 2019

Subsystems (5)
lore
Share & tools
lore

Jason A. Donenfeld

author maintainer
#0 lore raw

RFC PATCH 1/1 net: WireGuard secure network tunnel

RFC Note:
This is a RFC for folks who want to play with this early, because Herbert's cryptodev-2.6 tree hasn't yet made it into net-next. I'll repost this as a v1 (possibly with feedback incorporated) once the various trees are in the right place. This compiles on top of the Frankenzinc patchset from Ard, though it hasn't yet received suitable testing there for me to call it v1 just yet. Preliminary testing with the usual netns.sh test suite on x86 indicates it's at least mostly functional, but I'll be giving things further scrutiny in the days to come.

WireGuard is a layer 3 secure networking tunnel made specifically for the kernel, that aims to be much simpler and easier to audit than IPsec. Extensive documentation and description of the protocol and considerations, along with formal proofs of the cryptography, are available at:

* https://www.wireguard.com/
* https://www.wireguard.com/papers/wireguard.pdf

This commit implements WireGuard as a simple network device driver, accessible in the usual RTNL way used by virtual network drivers. It makes use of the udp_tunnel APIs, GRO, GSO, NAPI, and the usual set of networking subsystem APIs. It has a somewhat novel multicore queueing system designed for maximum throughput and minimal latency of encryption operations, but it is implemented modestly using workqueues and NAPI. Configuration is done via generic Netlink, and following a review from the Netlink maintainer a year ago, several high profile userspace have already implemented the API.

This commit also comes with several different tests, both in-kernel tests and out-of-kernel tests based on network namespaces, taking profit of the fact that sockets used by WireGuard intentionally stay in the namespace the WireGuard interface was originally created, exactly like the semantics of userspace tun devices. See wireguard.com/netns/ for pictures and examples.

The source code is fairly short, but rather than combining everything into a single file, WireGuard is developed as cleanly separable files, making auditing and comprehension easier. Things are laid out as follows:

* noise.[ch], cookie.[ch], messages.h: These implement the bulk of the cryptographic aspects of the protocol, and are mostly data-only in nature, taking in buffers of bytes and spitting out buffers of bytes. They also handle reference counting for their various shared pieces of data, like keys and key lists.

* ratelimiter.[ch]: Used as an integral part of cookie.[ch] for ratelimiting certain types of cryptographic operations in accordance with particular WireGuard semantics.

* allowedips.[ch], peerlookup.[ch]: The main lookup structures of WireGuard, the former being trie-like with particular semantics, an integral part of the design of the protocol, and the latter just being nice helper functions around the various hashtables we use.

* device.[ch]: Implementation of functions for the netdevice and for rtnl, responsible for maintaining the life of a given interface and wiring it up to the rest of WireGuard.

* peer.[ch]: Each interface has a list of peers, with helper functions available here for creation, destruction, and reference counting.

* socket.[ch]: Implementation of functions related to udp_socket and the general set of kernel socket APIs, for sending and receiving ciphertext UDP packets, and taking care of WireGuard-specific sticky socket routing semantics for the automatic roaming.

* netlink.[ch]: Userspace API entry point for configuring WireGuard peers and devices. The API has been implemented by several userspace tools and network management utility, and the WireGuard project distributes the basic wg(8) tool.

* queueing.[ch]: Shared function on the rx and tx path for handling the various queues used in the multicore algorithms.

* send.c: Handles encrypting outgoing packets in parallel on multiple cores, before sending them in order on a single core, via workqueues and ring buffers. Also handles sending handshake and cookie messages as part of the protocol, in parallel.

* receive.c: Handles decrypting incoming packets in parallel on multiple cores, before passing them off in order to be ingested via the rest of the networking subsystem with GRO via the typical NAPI poll function. Also handles receiving handshake and cookie messages as part of the protocol, in parallel.

* timers.[ch]: Uses the timer wheel to implement protocol particular event timeouts, and gives a set of very simple event-driven entry point functions for callers.

* main.c, version.h: Initialization and deinitialization of the module.

* selftest/*.h: Runtime unit tests for some of the most security sensitive functions.

* tools/testing/selftests/wireguard/netns.sh: Aforementioned testing script using network namespaces.

This commit aims to be as self-contained as possible, implementing WireGuard as a standalone module not needing much special handling or coordination from the network subsystem. I expect for future optimizations to the network stack to positively improve WireGuard, and vice-versa, but for the time being, this exists as intentionally standalone.

We introduce a menu option for CONFIG_WIREGUARD, as well as providing a verbose debug log and self-tests via CONFIG_WIREGUARD_DEBUG.

---
 MAINTAINERS                                  |   8 +
 drivers/net/Kconfig                          |  45 +
 drivers/net/Makefile                         |   1 +
 drivers/net/wireguard/Makefile               |  18 +
 drivers/net/wireguard/allowedips.c           | 377 +++++++++
 drivers/net/wireguard/allowedips.h           |  59 ++
 drivers/net/wireguard/cookie.c               | 236 ++++++
 drivers/net/wireguard/cookie.h               |  59 ++
 drivers/net/wireguard/device.c               | 458 ++++++++++
 drivers/net/wireguard/device.h               |  65 ++
 drivers/net/wireguard/main.c                 |  64 ++
 drivers/net/wireguard/messages.h             | 128 +++
 drivers/net/wireguard/netlink.c              | 648 +++++++++++++++
 drivers/net/wireguard/netlink.h              |  12 +
 drivers/net/wireguard/noise.c                | 828 +++++++++++++++++++
 drivers/net/wireguard/noise.h                | 137 +++
 drivers/net/wireguard/peer.c                 | 240 ++++++
 drivers/net/wireguard/peer.h                 |  83 ++
 drivers/net/wireguard/peerlookup.c           | 221 +++++
 drivers/net/wireguard/peerlookup.h           |  64 ++
 drivers/net/wireguard/queueing.c             |  53 ++
 drivers/net/wireguard/queueing.h             | 197 +++++
 drivers/net/wireguard/ratelimiter.c          | 223 +++++
 drivers/net/wireguard/ratelimiter.h          |  19 +
 drivers/net/wireguard/receive.c              | 595 +++++++++++++
 drivers/net/wireguard/selftest/.clang-format | 424 ++++++++++
 drivers/net/wireguard/selftest/allowedips.c  | 682 +++++++++++++++
 drivers/net/wireguard/selftest/counter.c     | 104 +++
 drivers/net/wireguard/selftest/ratelimiter.c | 226 +++++
 drivers/net/wireguard/send.c                 | 424 ++++++++++
 drivers/net/wireguard/socket.c               | 436 ++++++++++
 drivers/net/wireguard/socket.h               |  44 +
 drivers/net/wireguard/timers.c               | 243 ++++++
 drivers/net/wireguard/timers.h               |  31 +
 drivers/net/wireguard/version.h              |   1 +
 include/uapi/linux/wireguard.h               | 196 +++++
 tools/testing/selftests/wireguard/netns.sh   | 537 ++++++++++++
 37 files changed, 8186 insertions(+)
 create mode 100644 drivers/net/wireguard/Makefile
 create mode 100644 drivers/net/wireguard/allowedips.c
 create mode 100644 drivers/net/wireguard/allowedips.h
 create mode 100644 drivers/net/wireguard/cookie.c
 create mode 100644 drivers/net/wireguard/cookie.h
 create mode 100644 drivers/net/wireguard/device.c
 create mode 100644 drivers/net/wireguard/device.h
 create mode 100644 drivers/net/wireguard/main.c
 create mode 100644 drivers/net/wireguard/messages.h
 create mode 100644 drivers/net/wireguard/netlink.c
 create mode 100644 drivers/net/wireguard/netlink.h
 create mode 100644 drivers/net/wireguard/noise.c
 create mode 100644 drivers/net/wireguard/noise.h
 create mode 100644 drivers/net/wireguard/peer.c
 create mode 100644 drivers/net/wireguard/peer.h
 create mode 100644 drivers/net/wireguard/peerlookup.c
 create mode 100644 drivers/net/wireguard/peerlookup.h
 create mode 100644 drivers/net/wireguard/queueing.c
 create mode 100644 drivers/net/wireguard/queueing.h
 create mode 100644 drivers/net/wireguard/ratelimiter.c
 create mode 100644 drivers/net/wireguard/ratelimiter.h
 create mode 100644 drivers/net/wireguard/receive.c
 create mode 100644 drivers/net/wireguard/selftest/.clang-format
 create mode 100644 drivers/net/wireguard/selftest/allowedips.c
 create mode 100644 drivers/net/wireguard/selftest/counter.c
 create mode 100644 drivers/net/wireguard/selftest/ratelimiter.c
 create mode 100644 drivers/net/wireguard/send.c
 create mode 100644 drivers/net/wireguard/socket.c
 create mode 100644 drivers/net/wireguard/socket.h
 create mode 100644 drivers/net/wireguard/timers.c
 create mode 100644 drivers/net/wireguard/timers.h
 create mode 100644 drivers/net/wireguard/version.h
 create mode 100644 include/uapi/linux/wireguard.h
 create mode 100755 tools/testing/selftests/wireguard/netns.sh
MAINTAINERS +8 -0
index 654a54258d8d..4db76374e8b9 100644
@@ -17596,6 +17596,14 @@ L:	[email protected]
 S:	Maintained
 F:	drivers/gpio/gpio-ws16c48.c
 
+WIREGUARD SECURE NETWORK TUNNEL
+M:	Jason A. Donenfeld <[email protected]>
+S:	Maintained
+F:	drivers/net/wireguard/
+F:	tools/testing/selftests/wireguard/
+L:	[email protected]
+L:	[email protected]
+
 WISTRON LAPTOP BUTTON DRIVER
 M:	Miloslav Trmac <[email protected]>
 S:	Maintained
drivers/net/Kconfig +45 -0

51 lines not on this page · raw message

drivers/net/Makefile +1 -0
index 0d3ba056cda3..953b7c12f0b0 100644
@@ -10,6 +10,7 @@ obj-$(CONFIG_BONDING) += bonding/
 obj-$(CONFIG_IPVLAN) += ipvlan/
 obj-$(CONFIG_IPVTAP) += ipvlan/
 obj-$(CONFIG_DUMMY) += dummy.o
+obj-$(CONFIG_WIREGUARD) += wireguard/
 obj-$(CONFIG_EQUALIZER) += eql.o
 obj-$(CONFIG_IFB) += ifb.o
 obj-$(CONFIG_MACSEC) += macsec.o
drivers/net/wireguard/Makefile new file +18 -0
new file mode 100644
index 000000000000..fc52b2cb500b
@@ -0,0 +1,18 @@
+ccflags-y := -O3
+ccflags-y += -D'pr_fmt(fmt)=KBUILD_MODNAME ": " fmt'
+ccflags-$(CONFIG_WIREGUARD_DEBUG) += -DDEBUG
+wireguard-y := main.o
+wireguard-y += noise.o
+wireguard-y += device.o
+wireguard-y += peer.o
+wireguard-y += timers.o
+wireguard-y += queueing.o
+wireguard-y += send.o
+wireguard-y += receive.o
+wireguard-y += socket.o
+wireguard-y += peerlookup.o
+wireguard-y += allowedips.o
+wireguard-y += ratelimiter.o
+wireguard-y += cookie.o
+wireguard-y += netlink.o
+obj-$(CONFIG_WIREGUARD) := wireguard.o
drivers/net/wireguard/allowedips.c new file +377 -0

377 lines not on this page · raw message

drivers/net/wireguard/allowedips.h new file +59 -0

59 lines not on this page · raw message

drivers/net/wireguard/cookie.c new file +236 -0

236 lines not on this page · raw message

drivers/net/wireguard/cookie.h new file +59 -0

59 lines not on this page · raw message

drivers/net/wireguard/device.c new file +458 -0

458 lines not on this page · raw message

drivers/net/wireguard/device.h new file +65 -0

65 lines not on this page · raw message

drivers/net/wireguard/main.c new file +64 -0

64 lines not on this page · raw message

drivers/net/wireguard/messages.h new file +128 -0

128 lines not on this page · raw message

drivers/net/wireguard/netlink.c new file +648 -0

648 lines not on this page · raw message

drivers/net/wireguard/netlink.h new file +12 -0
new file mode 100644
index 000000000000..15100d92e2e3
@@ -0,0 +1,12 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2015-2019 Jason A. Donenfeld <[email protected]>. All Rights Reserved.
+ */
+
+#ifndef _WG_NETLINK_H
+#define _WG_NETLINK_H
+
+int wg_genetlink_init(void);
+void wg_genetlink_uninit(void);
+
+#endif /* _WG_NETLINK_H */
drivers/net/wireguard/noise.c new file +828 -0

804 lines not on this page · raw message

drivers/net/wireguard/noise.h new file +137 -0

137 lines not on this page · raw message

drivers/net/wireguard/peer.c new file +240 -0

240 lines not on this page · raw message

drivers/net/wireguard/peer.h new file +83 -0

83 lines not on this page · raw message

drivers/net/wireguard/peerlookup.c new file +221 -0

221 lines not on this page · raw message

drivers/net/wireguard/peerlookup.h new file +64 -0

64 lines not on this page · raw message

drivers/net/wireguard/queueing.c new file +53 -0

53 lines not on this page · raw message

drivers/net/wireguard/queueing.h new file +197 -0

197 lines not on this page · raw message

drivers/net/wireguard/ratelimiter.c new file +223 -0

223 lines not on this page · raw message

drivers/net/wireguard/ratelimiter.h new file +19 -0
new file mode 100644
index 000000000000..83067f71ea99

Lines of this file are not shown. View the raw message.

drivers/net/wireguard/receive.c new file +595 -0

595 lines not on this page · raw message

drivers/net/wireguard/selftest/.clang-format new file +424 -0

424 lines not on this page · raw message

drivers/net/wireguard/selftest/allowedips.c new file +682 -0

682 lines not on this page · raw message

drivers/net/wireguard/selftest/counter.c new file +104 -0

104 lines not on this page · raw message

drivers/net/wireguard/selftest/ratelimiter.c new file +226 -0

226 lines not on this page · raw message

drivers/net/wireguard/send.c new file +424 -0

424 lines not on this page · raw message

drivers/net/wireguard/socket.c new file +436 -0

436 lines not on this page · raw message

drivers/net/wireguard/socket.h new file +44 -0

44 lines not on this page · raw message

drivers/net/wireguard/timers.c new file +243 -0

243 lines not on this page · raw message

drivers/net/wireguard/timers.h new file +31 -0
new file mode 100644
index 000000000000..f0653dcb1326

Lines of this file are not shown. View the raw message.

drivers/net/wireguard/version.h new file +1 -0
new file mode 100644
index 000000000000..a1a269a11634

Lines of this file are not shown. View the raw message.

include/uapi/linux/wireguard.h new file +196 -0

196 lines not on this page · raw message

tools/testing/selftests/wireguard/netns.sh new file +537 -0

537 lines not on this page · raw message

The rest of this diff is not shown. View the raw message.

Signature · 1 line
2.24.0

Part of the diff is not shown here. View the raw message.

#1 lore raw
Quoted 10 lines · Jason A. Donenfeld

RFC Note:
This is a RFC for folks who want to play with this early, because Herbert's cryptodev-2.6 tree hasn't yet made it into net-next. I'll repost this as a v1 (possibly with feedback incorporated) once the various trees are in the right place. This compiles on top of the Frankenzinc patchset from Ard, though it hasn't yet received suitable testing there for me to call it v1 just yet. Preliminary testing with the usual netns.sh test suite on x86 indicates it's at least mostly functional, but I'll be giving things further scrutiny in the days to come.

Hi Jason

Great to see this! Just a few small comments for now:

Quoted 3 lines

+/*
+ * Copyright (C) 2015-2019 Jason A. Donenfeld <[email protected]>. All Rights Reserved.
+ */

Could you please get rid of the "All Rights Reserved" (here, and everywhere else)? All rights are *not* reserved: this is licensed under the GPL. Besides, that phrase is in general dubious at best:
https://en.wikipedia.org/wiki/All_rights_reserved

drivers/net/wireguard/messages.h line 53

Quoted 3 lines
drivers/net/wireguard/messages.h new file +3 -0
+	MAX_QUEUED_INCOMING_HANDSHAKES = 4096, /* TODO: replace this with DQL */
+	MAX_STAGED_PACKETS = 128,
+	MAX_QUEUED_PACKETS = 1024 /* TODO: replace this with DQL */

Yes, please (on the TODO) :)

FWIW, since you're using pointer rings I think the way to do this is probably to just keep the limits in place as a maximum size, and then use DQL (or CoDel) to throttle enqueue to those pointer rings instead of just letting them fill.

Happy to work with you on this (as I believe I've already promised), but we might as well do that after the initial version is merged...

-Toke

author maintainer
#2 lore raw
Quoted 4 lines · Toke Høiland-Jørgensen

Could you please get rid of the "All Rights Reserved" (here, and everywhere else)? All rights are *not* reserved: this is licensed under the GPL. Besides, that phrase is in general dubious at best:
https://en.wikipedia.org/wiki/All_rights_reserved

I've seen people make arguments for this in both ways, and code from major corporation enters the kernel every day bearing that line, which means lawyers of important kernel contributors disagree with you. Neither one of us are lawyers. Lacking any wide scale precedent for such changes, or the expertise to even be properly persuaded in any direction, I follow the advice of my council to stick to the norm and not to mess with copyright headers.

I think there are some Linux Foundation mailing lists that have license lawyers on them, relating to SPDX mainly. Maybe we can ask there?

Quoted 6 lines

drivers/net/wireguard/messages.h line 53

Quoted 3 lines
drivers/net/wireguard/messages.h new file +3 -0
+     MAX_QUEUED_INCOMING_HANDSHAKES = 4096, /* TODO: replace this with DQL */
+     MAX_STAGED_PACKETS = 128,
+     MAX_QUEUED_PACKETS = 1024 /* TODO: replace this with DQL */

Yes, please (on the TODO) :)

Quoted 7 lines

FWIW, since you're using pointer rings I think the way to do this is probably to just keep the limits in place as a maximum size, and then use DQL (or CoDel) to throttle enqueue to those pointer rings instead of just letting them fill.

Happy to work with you on this (as I believe I've already promised), but we might as well do that after the initial version is merged...

I've actually implemented this a few times, but DQL always seems too slow to react properly, and I haven't yet been able to figure out what's happening. Let's indeed work on this after the initial version is merged. I think this change, and several more like it, will be the topic of some interesting discussions. But that doesn't need to happen /now/ I don't think.

#3 lore raw
Quoted 14 lines · Jason A. Donenfeld
Quoted 6 lines

drivers/net/wireguard/messages.h line 53

Quoted 3 lines
drivers/net/wireguard/messages.h new file +3 -0
+     MAX_QUEUED_INCOMING_HANDSHAKES = 4096, /* TODO: replace this with DQL */
+     MAX_STAGED_PACKETS = 128,
+     MAX_QUEUED_PACKETS = 1024 /* TODO: replace this with DQL */

Yes, please (on the TODO) :)

Quoted 7 lines

FWIW, since you're using pointer rings I think the way to do this is probably to just keep the limits in place as a maximum size, and then use DQL (or CoDel) to throttle enqueue to those pointer rings instead of just letting them fill.

Happy to work with you on this (as I believe I've already promised), but we might as well do that after the initial version is merged...

Quoted 6 lines

I've actually implemented this a few times, but DQL always seems too slow to react properly, and I haven't yet been able to figure out what's happening. Let's indeed work on this after the initial version is merged. I think this change, and several more like it, will be the topic of some interesting discussions. But that doesn't need to happen /now/ I don't think.

Agreed. Let's wait until the initial version is merged and use that as a base to benchmark against... :)

-Toke

#6 lore raw
In reply to Toke Høiland-Jørgensen
Quoted 17 lines · Toke Høiland-Jørgensen
Quoted 10 lines · Jason A. Donenfeld

RFC Note:
This is a RFC for folks who want to play with this early, because Herbert's cryptodev-2.6 tree hasn't yet made it into net-next. I'll repost this as a v1 (possibly with feedback incorporated) once the various trees are in the right place. This compiles on top of the Frankenzinc patchset from Ard, though it hasn't yet received suitable testing there for me to call it v1 just yet. Preliminary testing with the usual netns.sh test suite on x86 indicates it's at least mostly functional, but I'll be giving things further scrutiny in the days to come.

Hi Jason

Great to see this! Just a few small comments for now:

Quoted 8 lines
Quoted 3 lines

+/*
+ * Copyright (C) 2015-2019 Jason A. Donenfeld <[email protected]>. All Rights Reserved.
+ */

Could you please get rid of the "All Rights Reserved" (here, and everywhere else)? All rights are *not* reserved: this is licensed under the GPL. Besides, that phrase is in general dubious at best:
https://en.wikipedia.org/wiki/All_rights_reserved

Don't take legal questions/advice on a developer mailing list :)

This text is fine as-is, please consult your RedHat lawyers for why that is so.

And if RedHat does have issues with this, have them please clean up the instances in the kernel where they have this same text before asking anyone else to do so.

thanks,

greg k-h

author maintainer
#7 lore raw
In reply to Jason A. Donenfeld
Quoted 11 lines · Jason A. Donenfeld

RFC Note:
This is a RFC for folks who want to play with this early, because Herbert's cryptodev-2.6 tree hasn't yet made it into net-next. I'll repost this as a v1 (possibly with feedback incorporated) once the various trees are in the right place. This compiles on top of the Frankenzinc patchset from Ard, though it hasn't yet received suitable testing there for me to call it v1 just yet. Preliminary testing with the usual netns.sh test suite on x86 indicates it's at least mostly functional, but I'll be giving things further scrutiny in the days to come.

Quoted 2 lines

WireGuard is a layer 3 secure networking tunnel made specifically for [...]

FYI, as the various merges happen between crypto-2.6.git and net*.git, I'll be keeping this tag up to date with the latest WireGuard patch:

https://git.kernel.org/pub/scm/linux/kernel/git/zx2c4/linux.git/patch/?id=734bd9ed21b0b0057bd2a131c9129a50cd910f6c
https://git.kernel.org/pub/scm/linux/kernel/git/zx2c4/linux.git/commit/?h=wireguard