ktls-utils 1.0.0
A message that is not in the archive (sent off-list or not yet received): oracle/ktls-utils/push/refs/tags/ktls-utils-1.0.0/[email protected]
This is the official release of ktls-utils 1.0.0
The marquee feature of this release is initial support for the Linux kernel's QUICv1 implementation. In-kernel QUICv1 support has not yet been merged but it is reaching maturity.
I had hoped to include a few fixes that enable ktls-utils to be built on MUSL-based platforms, but the contributor has not responded to requests to sign the OCA (he said he would sign, but has since disappeared).
-------- Forwarded Message --------
Subject: [oracle/ktls-utils]
Date: Mon, 05 May 2025 11:02:34 -0700
From: Chuck Lever <[email protected]>
To: [email protected]
Branch: refs/tags/ktls-utils-1.0.0
Home: https://github.com/oracle/ktls-utils
To unsubscribe from these emails, change your notification settings at
https://github.com/oracle/ktls-utils/settings/notifications
kernel TLS configuration, was: Re: [ANNOUNCE] ktls-utils 1.0.0
Hi Chuck,
let me use this as a vehicle to rant^H^H^H^Hprovide constructive feedback about configuration of the tls upcalls now that I finally got around playing with both NVMe and TCP over TLS.
For modular systems configuations the amount of monolithic state in tlshd.conf is a bit unfortunate.
For NVMe it isn't all that bad, but having to hardcode the .nvme keyring still means that:
- we need userspace configuration past just enabling tlshd to enable any kernel subsystem using TLS upcalls.
- hard code the keyring used in the userspace configuration
Can't we ensure the upcall passes the keyring to use and avoid this issue entirely?
For NFS hardcoding the keys and certs in tlshd.conf is even more anoying, because it limits to a single client key and cert for the entire system.
I have a hacky little patch for the NFS client to pass keyids for the client key and the certificate as mount options, which seems to work nicely, but it still requires adding another keyring (see above) or giving the user read permissions for the keys while it would prefer that it would just work and we would not need to give any root login session a way to read the keys.
Re: kernel TLS configuration, was: Re: [ANNOUNCE] ktls-utils 1.0.0
Quoted 12 lines · Christoph Hellwig
Hi Chuck,
let me use this as a vehicle to rant^H^H^H^Hprovide constructive feedback about configuration of the tls upcalls now that I finally got around playing with both NVMe and TCP over TLS.
For modular systems configuations the amount of monolithic state in tlshd.conf is a bit unfortunate.
For NVMe it isn't all that bad, but having to hardcode the .nvme keyring still means that:
Quoted 7 lines
- we need userspace configuration past just enabling tlshd to enable any kernel subsystem using TLS upcalls.
- hard code the keyring used in the userspace configurationCan't we ensure the upcall passes the keyring to use and avoid this issue entirely?
Hmm. We do that already:
dev_dbg(nctrl->device, "queue %d: start TLS with key %x\n",
qid, pskid);
memset(&args, 0, sizeof(args));
args.ta_sock = queue->sock;
args.ta_done = nvme_tcp_tls_done;
args.ta_data = queue;
args.ta_my_peerids[0] = pskid;
args.ta_num_peerids = 1;
if (nctrl->opts->keyring)
keyring = key_serial(nctrl->opts->keyring);
args.ta_keyring = keyring;
args.ta_timeout_ms = tls_handshake_timeout * 1000;
queue->tls_err = -EOPNOTSUPP;
init_completion(&queue->tls_complete);
ret = tls_client_hello_psk(&args, GFP_KERNEL);... but we never evaluate the 'keyring' parameter from tlshd. Should be easy enough to fix.
Cheers,
Hannes
Signature · 4 lines
Dr. Hannes Reinecke Kernel Storage Architect [email protected] +49 911 74053 688 SUSE Software Solutions GmbH, Frankenstr. 146, 90461 Nürnberg HRB 36809 (AG Nürnberg), GF: I. Totev, A. McDonald, W. Knoblich
Re: kernel TLS configuration, was: Re: [ANNOUNCE] ktls-utils 1.0.0
Quoted 18 lines · Hannes Reinecke
Hmm. We do that already:
dev_dbg(nctrl->device, "queue %d: start TLS with key %x\n", qid, pskid); memset(&args, 0, sizeof(args)); args.ta_sock = queue->sock; args.ta_done = nvme_tcp_tls_done; args.ta_data = queue; args.ta_my_peerids[0] = pskid; args.ta_num_peerids = 1; if (nctrl->opts->keyring) keyring = key_serial(nctrl->opts->keyring); args.ta_keyring = keyring; args.ta_timeout_ms = tls_handshake_timeout * 1000; queue->tls_err = -EOPNOTSUPP; init_completion(&queue->tls_complete); ret = tls_client_hello_psk(&args, GFP_KERNEL);
Quoted 2 lines
... but we never evaluate the 'keyring' parameter from tlshd. Should be easy enough to fix.
That is only used to link the keyrind in tls_handshake_private_keyring and never passed over netlink.
Re: kernel TLS configuration, was: Re: [ANNOUNCE] ktls-utils 1.0.0
Quoted 19 lines · Christoph Hellwig
Quoted 18 lines · Hannes Reinecke
Hmm. We do that already:
dev_dbg(nctrl->device, "queue %d: start TLS with key %x\n", qid, pskid); memset(&args, 0, sizeof(args)); args.ta_sock = queue->sock; args.ta_done = nvme_tcp_tls_done; args.ta_data = queue; args.ta_my_peerids[0] = pskid; args.ta_num_peerids = 1; if (nctrl->opts->keyring) keyring = key_serial(nctrl->opts->keyring); args.ta_keyring = keyring; args.ta_timeout_ms = tls_handshake_timeout * 1000; queue->tls_err = -EOPNOTSUPP; init_completion(&queue->tls_complete); ret = tls_client_hello_psk(&args, GFP_KERNEL);
Quoted 7 lines
Quoted 2 lines
... but we never evaluate the 'keyring' parameter from tlshd. Should be easy enough to fix.
That is only used to link the keyrind in
tls_handshake_private_keyringand never passed over netlink.
ktls-utils pull request #94.
Cheers,
Hannes
Signature · 4 lines
Dr. Hannes Reinecke Kernel Storage Architect [email protected] +49 911 74053 688 SUSE Software Solutions GmbH, Frankenstr. 146, 90461 Nürnberg HRB 36809 (AG Nürnberg), GF: I. Totev, A. McDonald, W. Knoblich
Re: kernel TLS configuration, was: Re: [ANNOUNCE] ktls-utils 1.0.0
In reply to Christoph HellwigQuoted 12 lines · Christoph Hellwig
Hi Chuck,
let me use this as a vehicle to rant^H^H^H^Hprovide constructive feedback about configuration of the tls upcalls now that I finally got around playing with both NVMe and TCP over TLS.
For modular systems configuations the amount of monolithic state in tlshd.conf is a bit unfortunate.
For NVMe it isn't all that bad, but having to hardcode the .nvme keyring still means that:
Quoted 6 lines
- we need userspace configuration past just enabling tlshd to enable any kernel subsystem using TLS upcalls.
- hard code the keyring used in the userspace configurationCan't we ensure the upcall passes the keyring to use and avoid this issue entirely?
As Hannes mentioned in another email, that is the ultimate goal, and there may already be some code to do that.
Quoted 2 lines
For NFS hardcoding the keys and certs in tlshd.conf is even more anoying, because it limits to a single client key and cert for the entire system.
IIRC the keyring mechanism will work with NFS too.
Quoted 6 lines
I have a hacky little patch for the NFS client to pass keyids for the client key and the certificate as mount options, which seems to work nicely, but it still requires adding another keyring (see above) or giving the user read permissions for the keys while it would prefer that it would just work and we would not need to give any root login session a way to read the keys.
No-one has claimed that the current implementation is complete. I've just sort of petered out on significant new changes. But it's definitely still open to contribution. Patches on kernel-tls-handshake@ are very welcome. (But you do have to sign the Oracle Contributor's Agreement, unfortunately, to get the patches into ktls-utils).
Signature · 1 line
Chuck Lever
Re: kernel TLS configuration, was: Re: [ANNOUNCE] ktls-utils 1.0.0
Quoted 2 lines · Chuck Lever
are very welcome. (But you do have to sign the Oracle Contributor's Agreement, unfortunately, to get the patches into ktls-utils).
I guess we should just for it, which would also take care of the musl support?
Re: kernel TLS configuration, was: Re: [ANNOUNCE] ktls-utils 1.0.0
Quoted 6 lines · Christoph Hellwig
Quoted 2 lines · Chuck Lever
are very welcome. (But you do have to sign the Oracle Contributor's Agreement, unfortunately, to get the patches into ktls-utils).
I guess we should just for it, which would also take care of the musl support?
My employer might not take kindly to that.
The "MUSL issue" isn't due to the OCA, directly. The developer got agreement from his employer (WD) to sign, he just hasn't gotten around to it.
Also, we've discussed forking before, but "just fork it" does need a little more detail. Distributions consider oracle/ktls-utils the official source code, and tlshd is a security facility, after all.
Let me discuss this with my boss.
Signature · 1 line
Chuck Lever
Re: kernel TLS configuration, was: Re: [ANNOUNCE] ktls-utils 1.0.0
Quoted 5 lines · Chuck Lever
Quoted 4 lines · Christoph Hellwig
Quoted 2 lines · Chuck Lever
are very welcome. (But you do have to sign the Oracle Contributor's Agreement, unfortunately, to get the patches into ktls-utils).
Quoted 8 lines
Quoted 2 lines
I guess we should just for it, which would also take care of the musl support?
My employer might not take kindly to that.
The "MUSL issue" isn't due to the OCA, directly. The developer got agreement from his employer (WD) to sign, he just hasn't gotten around to it.
the point is that any kind of CLA is completely unacceptable for something needed in the core Linux stack.
Someone at your employer might have smoked some really bad stuff if they thought it as a good idea and I don't think anyone gives a rats ass if they take it kindly or not.
Re: kernel TLS configuration, was: Re: [ANNOUNCE] ktls-utils 1.0.0
Quoted 9 lines · Christoph Hellwig
Quoted 8 lines · Chuck Lever
Quoted 6 lines · Christoph Hellwig
Quoted 2 lines · Chuck Lever
are very welcome. (But you do have to sign the Oracle Contributor's Agreement, unfortunately, to get the patches into ktls-utils).
I guess we should just for it, which would also take care of the musl support?
Quoted 8 lines
Quoted 5 lines
My employer might not take kindly to that.
The "MUSL issue" isn't due to the OCA, directly. The developer got agreement from his employer (WD) to sign, he just hasn't gotten around to it.
the point is that any kind of CLA is completely unacceptable for something needed in the core Linux stack.
Agreed.
Quoted 2 lines
Someone at your employer might have smoked some really bad stuff if they thought it as a good idea
Note that the OCA is a blanket policy. I'm not sure they thought very carefully (or at all) about whether it is appropriate for this specific project.
Note also, we got this published two years ago as an experiment. It certainly was not a "core part of the Linux stack" at that time.
and I don't think anyone gives a rats
Quoted 1 line
ass if they take it kindly or not.
I do. I can't burn that bridge and stay employed. So calm yourself, sir. Let me ask around.
Signature · 1 line
Chuck Lever
Re: kernel TLS configuration, was: Re: [ANNOUNCE] ktls-utils 1.0.0
Quoted 2 lines · Chuck Lever
I do. I can't burn that bridge and stay employed. So calm yourself, sir. Let me ask around.
No one expects you to support this. But you should also not expect much support for a project with a silly CLA.
Re: kernel TLS configuration, was: Re: [ANNOUNCE] ktls-utils 1.0.0
Quoted 5 lines · Christoph Hellwig
Quoted 2 lines · Chuck Lever
I do. I can't burn that bridge and stay employed. So calm yourself, sir. Let me ask around.
No one expects you to support this.
That's just the kind of detail we need to decide before we can "just fork it" -- who will be the maintainer going forward?
Quoted 2 lines
But you should also not expect
much support for a project with a silly CLA.
I respect my employer enough that I think they deserve your feedback first before the rug is yanked. The best outcome would be that we have their permission to move this project somewhere more open.
Signature · 1 line
Chuck Lever
Re: kernel TLS configuration, was: Re: [ANNOUNCE] ktls-utils 1.0.0
Quoted 11 lines · Chuck Lever
Quoted 5 lines · Christoph Hellwig
Quoted 2 lines · Chuck Lever
I do. I can't burn that bridge and stay employed. So calm yourself, sir. Let me ask around.
No one expects you to support this.
That's just the kind of detail we need to decide before we can "just fork it" -- who will be the maintainer going forward?
Quoted 6 lines
Quoted 2 lines
But you should also not expect
much support for a project with a silly CLA.I respect my employer enough that I think they deserve your feedback first before the rug is yanked. The best outcome would be that we have their permission to move this project somewhere more open.
That would be extremely helpful. In the meantime I'm happy to host a development branch freed from the CLA pains.
Re: kernel TLS configuration, was: Re: [ANNOUNCE] ktls-utils 1.0.0
Quoted 15 lines · Christoph Hellwig
Quoted 14 lines · Chuck Lever
Quoted 5 lines · Christoph Hellwig
Quoted 2 lines · Chuck Lever
I do. I can't burn that bridge and stay employed. So calm yourself, sir. Let me ask around.
No one expects you to support this.
That's just the kind of detail we need to decide before we can "just fork it" -- who will be the maintainer going forward?
Quoted 2 lines
But you should also not expect
much support for a project with a silly CLA.
Quoted 7 lines
Quoted 3 lines
I respect my employer enough that I think they deserve your feedback first before the rug is yanked. The best outcome would be that we have their permission to move this project somewhere more open.
That would be extremely helpful. In the meantime I'm happy to host a development branch freed from the CLA pains.
For the record, even I do that:
https://github.com/chucklever/ktls-utils
Signature · 1 line
Chuck Lever
Sagi Grimberg
maintainerRe: kernel TLS configuration, was: Re: [ANNOUNCE] ktls-utils 1.0.0
In reply to Christoph HellwigQuoted 22 lines · Christoph Hellwig
Hi Chuck,
let me use this as a vehicle to rant^H^H^H^Hprovide constructive feedback about configuration of the tls upcalls now that I finally got around playing with both NVMe and TCP over TLS.
For modular systems configuations the amount of monolithic state in tlshd.conf is a bit unfortunate.
For NVMe it isn't all that bad, but having to hardcode the .nvme keyring still means that:
- we need userspace configuration past just enabling tlshd to enable any kernel subsystem using TLS upcalls.
- hard code the keyring used in the userspace configurationCan't we ensure the upcall passes the keyring to use and avoid this issue entirely?
For NFS hardcoding the keys and certs in tlshd.conf is even more anoying, because it limits to a single client key and cert for the entire system.
Quoted 7 lines
I have a hacky little patch for the NFS client to pass keyids for the client key and the certificate as mount options, which seems to work nicely, but it still requires adding another keyring (see above) or giving the user read permissions for the keys while it would prefer that it would just work and we would not need to give any root login session a way to read the keys.
Christoph,
Just so I understand, this is a separate issue from passing the keyring to tlshd correct? Is the suggesting that nfs will create a special .nfs keyring similar to what nvme does?
Note that nvme also allows the user to pass its own keyring (never tried it before - we probably need a blktest for it //hannes). So in this case, the possessor will need to set user READ perms on the key itself (assuming that it updated tlshd.conf to know this keyring)?
Re: kernel TLS configuration, was: Re: [ANNOUNCE] ktls-utils 1.0.0
Quoted 3 lines · Sagi Grimberg
Just so I understand, this is a separate issue from passing the keyring to tlshd correct? Is the suggesting that nfs will create a special .nfs keyring similar to what nvme does?
Yeah.
Quoted 5 lines
Note that nvme also allows the user to pass its own keyring (never tried it before - we probably need a blktest for it //hannes). So in this case, the
possessor will need to set user READ perms on the key itself (assuming that it updated tlshd.conf to know this keyring)?
I think so. Or give user read permissions for the keys (which from my limited undertanding renders the patches a bit useless).
Let me send out my current patches and discuss it there.
Re: kernel TLS configuration, was: Re: [ANNOUNCE] ktls-utils 1.0.0
Quoted 13 lines · Christoph Hellwig
Quoted 3 lines · Sagi Grimberg
Just so I understand, this is a separate issue from passing the keyring to tlshd correct? Is the suggesting that nfs will create a special .nfs keyring similar to what nvme does?
Yeah.
Quoted 5 lines
Note that nvme also allows the user to pass its own keyring (never tried it before - we probably need a blktest for it //hannes). So in this case, the
possessor will need to set user READ perms on the key itself (assuming that it updated tlshd.conf to know this keyring)?
Quoted 5 lines
I think so. Or give user read permissions for the keys (which from my limited undertanding renders the patches a bit useless).
Let me send out my current patches and discuss it there.
The canonical way here is to link the requested keyring into the session keyring of the calling process. That way you have access to the keys in that keyring.
Cheers,
Hannes
Signature · 4 lines
Dr. Hannes Reinecke Kernel Storage Architect [email protected] +49 911 74053 688 SUSE Software Solutions GmbH, Frankenstr. 146, 90461 Nürnberg HRB 36809 (AG Nürnberg), GF: I. Totev, A. McDonald, W. Knoblich