Skip to contentSkip to conversation

UAC2 Gadget: feedback endpoint support

PATCH New Jerome Brunet 1 participant 4 messages Jun 3, 2021

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

v3 cover UAC2 Gadget: feedback endpoint support

Current UAC2 gadget implements capture/sync paths as two USB ISO ASYNC endpoints (IN and OUT).

This violates USB spec which says that ISO ASYNC OUT endpoint should have feedback companion endpoint. See USB2.0 spec "5.12.4.1 Synchronization Type": asynchronous sink provides explicit feedback (isochronous pipe). Interesting that for ISO ASYNC *IN* endpoint respective feedback isn't required since source provides implicit feedforward (data stream).

While it's not an issue if UAC2 Gadget is connected to Linux host (Linux ignores missing feedback endpoint), with other hosts like Windows or MacOS the UAC2 Gadget isn't enumerated due to missing feedback endpoint.

This patch series adds feedback endpoint support to UAC2 function, new control to UAC2 mixer which can be used by userspace tools (like alsaloop from alsa-utils) for updating feedback frequency reported to the host. This is useful for usecases when UAC2 Gadget's audio samples are played to another codec or audio card with its own internal freerunning clock so host can be notified that more/less samples are required.

The alsaloop tool requires some (relatively small) modifications in order to start support driving feedback frequency through UAC2 mixer control. That change will be sent as a separate patch to ALSA community.

Also added ability to switch ISO ASYNC OUT endpoint into adaptive endpoint which doesn't require feedback endpoint (as per USB spec).

v2 [2] -> v3:
* Just fold 9 left shift in the feedback calculation to avoid dealing with big numbers (as suggested by Pavel Hofman)

v1 [0] -> v2:
* Instead of HW rate used by gadget (which the controlling application might not be directly dealing with), the kcontrol used to provide the feedback value is expressed in PPM. This way the correction is agnostic of the rate used by the gadget while retaining enough precision to allow a smooth adaptation of the clock on the playback side, if necessary.
* The minimum value for the correction is aligned with sound/usb/endpoint.c at -25%.
* The maximum correction must be chosen more carefully as it impact the bandwidth required by the output EP. The minimum is one extra frame per packet. This is ~500PPM at 192KHz. This seemed more than enough while testing.
Of course, there is a trade off between saving bandwidth and enabling devices with a significant (abnormal) clock deviation. The default extra reserved bandwidth is set at 500PPM but it is configurable through configfs, like the rest of the gadget.

I have kept Ruslan assigned as the author of series. He did the vast majority of the work and deserve the credit. I have merely tweaked in his original patchset.

This patchset has been tested on an arm64 host at HIGH_SPEED, with various rates and channel configuration. I have not tested with FULL_SPEED or SUPER_SPEED.

See here [1] an example of regulation with a patched version of alsaloop.

[0]: https://lore.kernel.org/r/[email protected]
[1]: https://pastebin.com/0hiK7LAe
[2]: https://lore.kernel.org/r/[email protected]

Ruslan Bilovol (3):
usb: gadget: f_uac2/u_audio: add feedback endpoint support
usb: gadget: f_uac2: add adaptive sync support for capture
usb: gadget: u_audio: add real feedback implementation

 .../ABI/testing/configfs-usb-gadget-uac2      |   2 +
 Documentation/usb/gadget-testing.rst          |   2 +
 drivers/usb/gadget/function/f_uac2.c          | 144 ++++++++++-
 drivers/usb/gadget/function/u_audio.c         | 225 +++++++++++++++++-
 drivers/usb/gadget/function/u_audio.h         |  12 +
 drivers/usb/gadget/function/u_uac2.h          |   4 +
 6 files changed, 383 insertions(+), 6 deletions(-)
Signature · 1 line
2.32.0.rc0
#1 lore raw

PATCH v3 1/3 usb: gadget: f_uac2/u_audio: add feedback endpoint support

From: Ruslan Bilovol <[email protected]>

As per USB and UAC2 specs, asynchronous audio sink endpoint requires explicit synchronization mechanism (Isochronous Feedback Endpoint)

Implement feedback companion endpoint for ISO OUT endpoint

This patch adds all required infrastructure and USB requests handling for feedback endpoint. Syncrhonization itself is still dummy (feedback ep always reports 'nomimal frequency'
e.g. no adjustement is needed). This satisfies hosts that
require feedback endpoint (like Win10) and poll it periodically

Actual synchronization mechanism should be implemented separately

---
 drivers/usb/gadget/function/f_uac2.c  |  49 ++++++++++-
 drivers/usb/gadget/function/u_audio.c | 119 +++++++++++++++++++++++++-
 drivers/usb/gadget/function/u_audio.h |   3 +
 3 files changed, 168 insertions(+), 3 deletions(-)
drivers/usb/gadget/function/f_uac2.c +48 -1
index 7aa4c8bc5a1a..5d63244ba319 100644
@@ -240,7 +240,7 @@ static struct usb_interface_descriptor std_as_out_if1_desc = {
 	.bDescriptorType = USB_DT_INTERFACE,
 
 	.bAlternateSetting = 1,
-	.bNumEndpoints = 1,
+	.bNumEndpoints = 2,
 	.bInterfaceClass = USB_CLASS_AUDIO,
 	.bInterfaceSubClass = USB_SUBCLASS_AUDIOSTREAMING,
 	.bInterfaceProtocol = UAC_VERSION_2,
@@ -317,6 +317,37 @@ static struct uac2_iso_endpoint_descriptor as_iso_out_desc = {
 	.wLockDelay = 0,
 };
 
+/* STD AS ISO IN Feedback Endpoint */
+static struct usb_endpoint_descriptor fs_epin_fback_desc = {
+	.bLength = USB_DT_ENDPOINT_SIZE,
+	.bDescriptorType = USB_DT_ENDPOINT,
+
+	.bEndpointAddress = USB_DIR_IN,
+	.bmAttributes = USB_ENDPOINT_XFER_ISOC | USB_ENDPOINT_USAGE_FEEDBACK,
+	.wMaxPacketSize = cpu_to_le16(3),
+	.bInterval = 1,
+};
+
+static struct usb_endpoint_descriptor hs_epin_fback_desc = {
+	.bLength = USB_DT_ENDPOINT_SIZE,
+	.bDescriptorType = USB_DT_ENDPOINT,
+
+	.bmAttributes = USB_ENDPOINT_XFER_ISOC | USB_ENDPOINT_USAGE_FEEDBACK,
+	.wMaxPacketSize = cpu_to_le16(4),
+	.bInterval = 4,
+};
+
+static struct usb_endpoint_descriptor ss_epin_fback_desc = {
+	.bLength = USB_DT_ENDPOINT_SIZE,
+	.bDescriptorType = USB_DT_ENDPOINT,
+
+	.bEndpointAddress = USB_DIR_IN,
+	.bmAttributes = USB_ENDPOINT_XFER_ISOC | USB_ENDPOINT_USAGE_FEEDBACK,
+	.wMaxPacketSize = cpu_to_le16(4),
+	.bInterval = 4,
+};
+
+
 /* Audio Streaming IN Interface - Alt0 */
 static struct usb_interface_descriptor std_as_in_if0_desc = {
 	.bLength = sizeof std_as_in_if0_desc,
@@ -431,6 +462,7 @@ static struct usb_descriptor_header *fs_audio_desc[] = {
 	(struct usb_descriptor_header *)&as_out_fmt1_desc,
 	(struct usb_descriptor_header *)&fs_epout_desc,
 	(struct usb_descriptor_header *)&as_iso_out_desc,
+	(struct usb_descriptor_header *)&fs_epin_fback_desc,
 
 	(struct usb_descriptor_header *)&std_as_in_if0_desc,
 	(struct usb_descriptor_header *)&std_as_in_if1_desc,
@@ -461,6 +493,7 @@ static struct usb_descriptor_header *hs_audio_desc[] = {
 	(struct usb_descriptor_header *)&as_out_fmt1_desc,
 	(struct usb_descriptor_header *)&hs_epout_desc,
 	(struct usb_descriptor_header *)&as_iso_out_desc,
+	(struct usb_descriptor_header *)&hs_epin_fback_desc,
 
 	(struct usb_descriptor_header *)&std_as_in_if0_desc,
 	(struct usb_descriptor_header *)&std_as_in_if1_desc,
@@ -492,6 +525,7 @@ static struct usb_descriptor_header *ss_audio_desc[] = {
 	(struct usb_descriptor_header *)&ss_epout_desc,
 	(struct usb_descriptor_header *)&ss_epout_desc_comp,
 	(struct usb_descriptor_header *)&as_iso_out_desc,
+	(struct usb_descriptor_header *)&ss_epin_fback_desc,
 
 	(struct usb_descriptor_header *)&std_as_in_if0_desc,
 	(struct usb_descriptor_header *)&std_as_in_if1_desc,
@@ -568,22 +602,26 @@ static void setup_headers(struct f_uac2_opts *opts,
 	struct usb_ss_ep_comp_descriptor *epin_desc_comp = NULL;
 	struct usb_endpoint_descriptor *epout_desc;
 	struct usb_endpoint_descriptor *epin_desc;
+	struct usb_endpoint_descriptor *epin_fback_desc;
 	int i;
 
 	switch (speed) {
 	case USB_SPEED_FULL:
 		epout_desc = &fs_epout_desc;
 		epin_desc = &fs_epin_desc;
+		epin_fback_desc = &fs_epin_fback_desc;
 		break;
 	case USB_SPEED_HIGH:
 		epout_desc = &hs_epout_desc;
 		epin_desc = &hs_epin_desc;
+		epin_fback_desc = &hs_epin_fback_desc;
 		break;
 	default:
 		epout_desc = &ss_epout_desc;
 		epin_desc = &ss_epin_desc;
 		epout_desc_comp = &ss_epout_desc_comp;
 		epin_desc_comp = &ss_epin_desc_comp;
+		epin_fback_desc = &ss_epin_fback_desc;
 	}
 
 	i = 0;
@@ -611,6 +649,7 @@ static void setup_headers(struct f_uac2_opts *opts,
 			headers[i++] = USBDHDR(epout_desc_comp);
 
 		headers[i++] = USBDHDR(&as_iso_out_desc);
+		headers[i++] = USBDHDR(epin_fback_desc);
 	}
 	if (EPIN_EN(opts)) {
 		headers[i++] = USBDHDR(&std_as_in_if0_desc);
@@ -844,6 +883,12 @@ afunc_bind(struct usb_configuration *cfg, struct usb_function *fn)
 			dev_err(dev, "%s:%d Error!\n", __func__, __LINE__);
 			return -ENODEV;
 		}
+		agdev->in_ep_fback = usb_ep_autoconfig(gadget,
+						       &fs_epin_fback_desc);
+		if (!agdev->in_ep_fback) {
+			dev_err(dev, "%s:%d Error!\n", __func__, __LINE__);
+			return -ENODEV;
+		}
 	}
 
 	if (EPIN_EN(uac2_opts)) {
@@ -867,8 +912,10 @@ afunc_bind(struct usb_configuration *cfg, struct usb_function *fn)
 				le16_to_cpu(ss_epout_desc.wMaxPacketSize));
 
 	hs_epout_desc.bEndpointAddress = fs_epout_desc.bEndpointAddress;
+	hs_epin_fback_desc.bEndpointAddress = fs_epin_fback_desc.bEndpointAddress;
 	hs_epin_desc.bEndpointAddress = fs_epin_desc.bEndpointAddress;
 	ss_epout_desc.bEndpointAddress = fs_epout_desc.bEndpointAddress;
+	ss_epin_fback_desc.bEndpointAddress = fs_epin_fback_desc.bEndpointAddress;
 	ss_epin_desc.bEndpointAddress = fs_epin_desc.bEndpointAddress;
 
 	setup_descriptor(uac2_opts);
drivers/usb/gadget/function/u_audio.c +117 -2

161 lines not on this page · raw message

drivers/usb/gadget/function/u_audio.h +3 -0
index 5ea6b86f1fda..53e6baf55cbf 100644
@@ -30,7 +30,10 @@ struct g_audio {
 	struct usb_gadget *gadget;
 
 	struct usb_ep *in_ep;
+
 	struct usb_ep *out_ep;
+	/* feedback IN endpoint corresponding to out_ep */
+	struct usb_ep *in_ep_fback;
 
 	/* Max packet size for all in_ep possible speeds */
 	unsigned int in_ep_maxpsize;
Signature · 1 line
2.32.0.rc0
#2 lore raw

PATCH v3 2/3 usb: gadget: f_uac2: add adaptive sync support for capture

In reply to Jerome Brunet

From: Ruslan Bilovol <[email protected]>

Current f_uac2 USB OUT (aka 'capture') synchronization implements 'ASYNC' scenario which means USB Gadget has it's own freerunning clock and can update Host about real clock frequency through feedback endpoint so Host can align number of samples sent to the USB gadget to prevent overruns/underruns

In case if Gadget can has no it's internal clock and can consume audio samples at any rate (for example, on the Gadget side someone records audio directly to a file, or audio samples are played through an external DAC as soon as they arrive), UAC2 spec suggests 'ADAPTIVE' synchronization type.

Change UAC2 driver to make it configurable through additional 'c_sync' configfs file.

Default remains 'asynchronous' with possibility to switch it to 'adaptive'

---
 .../ABI/testing/configfs-usb-gadget-uac2      |   1 +
 Documentation/usb/gadget-testing.rst          |   1 +
 drivers/usb/gadget/function/f_uac2.c          | 100 ++++++++++++++++--
 drivers/usb/gadget/function/u_uac2.h          |   2 +
 4 files changed, 95 insertions(+), 9 deletions(-)
Documentation/ABI/testing/configfs-usb-gadget-uac2 +1 -0
index d4356c8b8cd6..e7e59d7fb12f 100644
@@ -8,6 +8,7 @@ Description:
 		c_chmask   capture channel mask
 		c_srate    capture sampling rate
 		c_ssize    capture sample size (bytes)
+		c_sync     capture synchronization type (async/adaptive)
 		p_chmask   playback channel mask
 		p_srate    playback sampling rate
 		p_ssize    playback sample size (bytes)
Documentation/usb/gadget-testing.rst +1 -0
index 2085e7b24eeb..f5a12667bd41 100644
@@ -728,6 +728,7 @@ The uac2 function provides these attributes in its function directory:
 	c_chmask	capture channel mask
 	c_srate		capture sampling rate
 	c_ssize		capture sample size (bytes)
+	c_sync		capture synchronization type (async/adaptive)
 	p_chmask	playback channel mask
 	p_srate		playback sampling rate
 	p_ssize		playback sample size (bytes)
drivers/usb/gadget/function/f_uac2.c +91 -9

172 lines not on this page · raw message

drivers/usb/gadget/function/u_uac2.h +2 -0
index b5035711172d..13589c3c805c 100644
@@ -21,6 +21,7 @@
 #define UAC2_DEF_CCHMASK 0x3
 #define UAC2_DEF_CSRATE 64000
 #define UAC2_DEF_CSSIZE 2
+#define UAC2_DEF_CSYNC		USB_ENDPOINT_SYNC_ASYNC
 #define UAC2_DEF_REQ_NUM 2
 
 struct f_uac2_opts {
@@ -31,6 +32,7 @@ struct f_uac2_opts {
 	int				c_chmask;
 	int				c_srate;
 	int				c_ssize;
+	int				c_sync;
 	int				req_number;
 	bool				bound;
 
Signature · 1 line
2.32.0.rc0
#3 lore raw

PATCH v3 3/3 usb: gadget: u_audio: add real feedback implementation

In reply to Jerome Brunet

From: Ruslan Bilovol <[email protected]>

This adds interface between userspace and feedback endpoint to report real feedback frequency to the Host.

Current implementation adds new userspace interface ALSA mixer control "Capture Pitch 1000000" (similar to aloop driver's "PCM Rate Shift 100000" mixer control)

Value in PPM is chosen to have correction value agnostic of the actual HW rate, which the application is not necessarily dealing with, while still retaining a good enough precision to allow smooth clock correction on the playback side, if necessary.

Similar to sound/usb/endpoint.c, a slow down is allowed up to 25%. This has no impact on the required bandwidth. Speedup correction has an impact on the bandwidth reserved for the isochronous endpoint. The default allowed speedup is 500ppm. This seems to be more than enough but, if necessary, this is configurable through a module parameter. The reserved bandwidth is rounded up to the next packet size.

Usage of this new control is easy to implement in existing userspace tools like alsaloop from alsa-utils.

---
 .../ABI/testing/configfs-usb-gadget-uac2      |   1 +
 Documentation/usb/gadget-testing.rst          |   1 +
 drivers/usb/gadget/function/f_uac2.c          |   9 +-
 drivers/usb/gadget/function/u_audio.c         | 124 ++++++++++++++++--
 drivers/usb/gadget/function/u_audio.h         |   9 ++
 drivers/usb/gadget/function/u_uac2.h          |   2 +
 6 files changed, 136 insertions(+), 10 deletions(-)
Documentation/ABI/testing/configfs-usb-gadget-uac2 +1 -0
index e7e59d7fb12f..26fb8e9b4e61 100644
@@ -9,6 +9,7 @@ Description:
 		c_srate    capture sampling rate
 		c_ssize    capture sample size (bytes)
 		c_sync     capture synchronization type (async/adaptive)
+		fb_max     maximum extra bandwidth in async mode
 		p_chmask   playback channel mask
 		p_srate    playback sampling rate
 		p_ssize    playback sample size (bytes)
Documentation/usb/gadget-testing.rst +1 -0
index f5a12667bd41..9d6276f82774 100644
@@ -729,6 +729,7 @@ The uac2 function provides these attributes in its function directory:
 	c_srate		capture sampling rate
 	c_ssize		capture sample size (bytes)
 	c_sync		capture synchronization type (async/adaptive)
+	fb_max          maximum extra bandwidth in async mode
 	p_chmask	playback channel mask
 	p_srate		playback sampling rate
 	p_ssize		playback sample size (bytes)
drivers/usb/gadget/function/f_uac2.c +8 -1
index 321e6c05ba93..ae29ff2b2b68 100644
@@ -584,8 +584,11 @@ static int set_ep_max_packet_size(const struct f_uac2_opts *uac2_opts,
 		ssize = uac2_opts->c_ssize;
 	}
 
+	if (!is_playback && (uac2_opts->c_sync == USB_ENDPOINT_SYNC_ASYNC))
+		srate = srate * (1000 + uac2_opts->fb_max) / 1000;
+
 	max_size_bw = num_channels(chmask) * ssize *
-		((srate / (factor / (1 << (ep_desc->bInterval - 1)))) + 1);
+		DIV_ROUND_UP(srate, factor / (1 << (ep_desc->bInterval - 1)));
 	ep_desc->wMaxPacketSize = cpu_to_le16(min_t(u16, max_size_bw,
 						    max_size_ep));
 
@@ -957,6 +960,7 @@ afunc_bind(struct usb_configuration *cfg, struct usb_function *fn)
 	agdev->params.c_srate = uac2_opts->c_srate;
 	agdev->params.c_ssize = uac2_opts->c_ssize;
 	agdev->params.req_number = uac2_opts->req_number;
+	agdev->params.fb_max = uac2_opts->fb_max;
 	ret = g_audio_setup(agdev, "UAC2 PCM", "UAC2_Gadget");
 	if (ret)
 		goto err_free_descs;
@@ -1329,6 +1333,7 @@ UAC2_ATTRIBUTE(c_srate);
 UAC2_ATTRIBUTE_SYNC(c_sync);
 UAC2_ATTRIBUTE(c_ssize);
 UAC2_ATTRIBUTE(req_number);
+UAC2_ATTRIBUTE(fb_max);
 
 static struct configfs_attribute *f_uac2_attrs[] = {
 	&f_uac2_opts_attr_p_chmask,
@@ -1339,6 +1344,7 @@ static struct configfs_attribute *f_uac2_attrs[] = {
 	&f_uac2_opts_attr_c_ssize,
 	&f_uac2_opts_attr_c_sync,
 	&f_uac2_opts_attr_req_number,
+	&f_uac2_opts_attr_fb_max,
 	NULL,
 };
 
@@ -1378,6 +1384,7 @@ static struct usb_function_instance *afunc_alloc_inst(void)
 	opts->c_ssize = UAC2_DEF_CSSIZE;
 	opts->c_sync = UAC2_DEF_CSYNC;
 	opts->req_number = UAC2_DEF_REQ_NUM;
+	opts->fb_max = UAC2_DEF_FB_MAX;
 	return &opts->func_inst;
 }
 
drivers/usb/gadget/function/u_audio.c +115 -9

202 lines not on this page · raw message

drivers/usb/gadget/function/u_audio.h +9 -0
index 53e6baf55cbf..a218cdf771fe 100644
@@ -11,6 +11,14 @@
 
 #include <linux/usb/composite.h>
 
+/*
+ * Same maximum frequency deviation on the slower side as in
+ * sound/usb/endpoint.c. Value is expressed in per-mil deviation.
+ * The maximum deviation on the faster side will be provided as
+ * parameter, as it impacts the endpoint required bandwidth.
+ */
+#define FBACK_SLOW_MAX	250
+
 struct uac_params {
 	/* playback */
 	int p_chmask;	/* channel mask */
@@ -23,6 +31,7 @@ struct uac_params {
 	int c_ssize;	/* sample size */
 
 	int req_number; /* number of preallocated requests */
+	int fb_max;	/* upper frequency drift feedback limit per-mil */
 };
 
 struct g_audio {
drivers/usb/gadget/function/u_uac2.h +2 -0
index 13589c3c805c..179d3ef6a195 100644
@@ -23,6 +23,7 @@
 #define UAC2_DEF_CSSIZE 2
 #define UAC2_DEF_CSYNC		USB_ENDPOINT_SYNC_ASYNC
 #define UAC2_DEF_REQ_NUM 2
+#define UAC2_DEF_FB_MAX 5
 
 struct f_uac2_opts {
 	struct usb_function_instance	func_inst;
@@ -34,6 +35,7 @@ struct f_uac2_opts {
 	int				c_ssize;
 	int				c_sync;
 	int				req_number;
+	int				fb_max;
 	bool				bound;
 
 	struct mutex			lock;
Signature · 1 line
2.32.0.rc0