Skip to contentSkip to conversation

configfs on dwc3: msc enum failed if three functions defined

DISCUSSION Bin Liu 3 participants 7 messages Apr 22, 2019 – Jul 3, 2019 USB

lore
Share & tools
lore

Bin Liu

author
#0 lore raw

Hi Felipe,

I am having an issue with dwc3 on TI AM57x device, and would like to ask for your comments.

I use configfs to create a multi-function gadget on dwc3, mass_storage is the last function, it seems if I create 3 functions, the mass_storage enumeration will fail on the host. It works fine if only create 2 functions.

The dwc3 tracepoints log shows after all the ep0 transfers for mass_storage, the very first epXin transfer is not complete - dwc3 programmed the urb, but never generates RX completion event. This also matches the bus analyzer trace - dwc3 NAKs the very first IN token for ever.

I use the attached script to create the gadget, The macro FUNCS in the beginning of the script defines the functions to be created.

Any comments are appreciated.

-Bin.

#1 lore raw

Hi Bin,

Quoted 16 lines · Bin Liu

Hi Felipe,

I am having an issue with dwc3 on TI AM57x device, and would like to ask for your comments.

I use configfs to create a multi-function gadget on dwc3, mass_storage is the last function, it seems if I create 3 functions, the mass_storage enumeration will fail on the host. It works fine if only create 2 functions.

The dwc3 tracepoints log shows after all the ep0 transfers for mass_storage, the very first epXin transfer is not complete - dwc3 programmed the urb, but never generates RX completion event. This also matches the bus analyzer trace - dwc3 NAKs the very first IN token for ever.

Quoted 4 lines

I use the attached script to create the gadget, The macro FUNCS in the beginning of the script defines the functions to be created.

Any comments are appreciated.

A stab in the dark here but what is the value of GTXFIFOSIZ(X)[15:0] for epXin on your device? Is it at least wMaxPacketSize? Depending on the default hardware values it might be deficient as compared to the working endpoint that gets assigned in your 2-function config.

Jack

Signature · 2 lines
The Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum,
a Linux Foundation Collaborative Project
author
#2 lore raw

Hi,

Sorry for the delay getting back on this. I was offline for quite some time.

Quoted 22 lines · Jack Pham

Hi Bin,

Quoted 19 lines · Bin Liu

Hi Felipe,

I am having an issue with dwc3 on TI AM57x device, and would like to ask for your comments.

I use configfs to create a multi-function gadget on dwc3, mass_storage is the last function, it seems if I create 3 functions, the mass_storage enumeration will fail on the host. It works fine if only create 2 functions.

The dwc3 tracepoints log shows after all the ep0 transfers for mass_storage, the very first epXin transfer is not complete - dwc3 programmed the urb, but never generates RX completion event. This also matches the bus analyzer trace - dwc3 NAKs the very first IN token for ever.

I use the attached script to create the gadget, The macro FUNCS in the beginning of the script defines the functions to be created.

Quoted 6 lines
Quoted 1 line

Any comments are appreciated.

A stab in the dark here but what is the value of GTXFIFOSIZ(X)[15:0] for epXin on your device? Is it at least wMaxPacketSize? Depending on the default hardware values it might be deficient as compared to the working endpoint that gets assigned in your 2-function config.

Jack,

thanks for the pointer, it is indeed the issue on AM57x device. The reset value of GTXFIFOSIZ for ep1~4 have size of 0x184, but ep5~15 have only size of 0x13 (which is 120 bytes), which is not enough for high-speed bulk xfers. I manually adjusted the fifo memory allocation, now my test case works.

Felipe,

Is there anything the dwc3 gadget driver can do to better handle this kind of devices, which don't have enough fifo buffers for all TX eps?

-Bin.

Tested-by Kishon Vijay Abraham I
#3 lore raw

Hi Bin,

Quoted 35 lines · Bin Liu

Hi,

Sorry for the delay getting back on this. I was offline for quite some time.

Quoted 22 lines · Jack Pham

Hi Bin,

Quoted 19 lines · Bin Liu

Hi Felipe,

I am having an issue with dwc3 on TI AM57x device, and would like to ask for your comments.

I use configfs to create a multi-function gadget on dwc3, mass_storage is the last function, it seems if I create 3 functions, the mass_storage enumeration will fail on the host. It works fine if only create 2 functions.

The dwc3 tracepoints log shows after all the ep0 transfers for mass_storage, the very first epXin transfer is not complete - dwc3 programmed the urb, but never generates RX completion event. This also matches the bus analyzer trace - dwc3 NAKs the very first IN token for ever.

I use the attached script to create the gadget, The macro FUNCS in the beginning of the script defines the functions to be created.

Quoted 6 lines
Quoted 1 line

Any comments are appreciated.

A stab in the dark here but what is the value of GTXFIFOSIZ(X)[15:0] for epXin on your device? Is it at least wMaxPacketSize? Depending on the default hardware values it might be deficient as compared to the working endpoint that gets assigned in your 2-function config.

Quoted 7 lines

Jack,

thanks for the pointer, it is indeed the issue on AM57x device. The reset value of GTXFIFOSIZ for ep1~4 have size of 0x184, but ep5~15 have only size of 0x13 (which is 120 bytes), which is not enough for high-speed bulk xfers. I manually adjusted the fifo memory allocation, now my test case works.

Cool! I'm glad my suggestion was on the right track.

Quoted 4 lines

Felipe,

Is there anything the dwc3 gadget driver can do to better handle this kind of devices, which don't have enough fifo buffers for all TX eps?

A long time ago...

commit bc5081617faeb3b2f0c126dc37264b87af7da47f

Author: Felipe Balbi <[email protected]>
Date:   Thu Feb 4 14:18:01 2016 +0200

usb: dwc3: drop FIFO resizing logic

That FIFO resizing logic was added to support OMAP5 ES1.0 which had a bogus default FIFO size. I can't remember the exact size of default FIFO, but it was less than one bulk superspeed packet (<1024) which would prevent USB3 from ever working on OMAP5 ES1.0.

However, OMAP5 ES1.0 support has been dropped by commit aa2f4b16f830 ("ARM: OMAP5: id: Remove ES1.0 support") which renders FIFO resizing unnecessary.

Prior to this there was a function dwc3_gadget_resize_tx_fifo(), enabled via DT flag, which enumerated all the endpoints and recalculated the TX FIFO sizes based on the transfer type of the EPs' configuration.

Unfortunately some Qualcomm SoCs are still plagued by having insufficient defaults, so we resort to carrying this function on our downstream kernels. It seems TI AM57x still has this problem too?

Jack

Signature · 2 lines
The Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum,
a Linux Foundation Collaborative Project
author
#4 lore raw
Quoted 74 lines · Jack Pham

Hi Bin,

Quoted 35 lines · Bin Liu

Hi,

Sorry for the delay getting back on this. I was offline for quite some time.

Quoted 22 lines · Jack Pham

Hi Bin,

Quoted 19 lines · Bin Liu

Hi Felipe,

I am having an issue with dwc3 on TI AM57x device, and would like to ask for your comments.

I use configfs to create a multi-function gadget on dwc3, mass_storage is the last function, it seems if I create 3 functions, the mass_storage enumeration will fail on the host. It works fine if only create 2 functions.

The dwc3 tracepoints log shows after all the ep0 transfers for mass_storage, the very first epXin transfer is not complete - dwc3 programmed the urb, but never generates RX completion event. This also matches the bus analyzer trace - dwc3 NAKs the very first IN token for ever.

I use the attached script to create the gadget, The macro FUNCS in the beginning of the script defines the functions to be created.

Quoted 6 lines
Quoted 1 line

Any comments are appreciated.

A stab in the dark here but what is the value of GTXFIFOSIZ(X)[15:0] for epXin on your device? Is it at least wMaxPacketSize? Depending on the default hardware values it might be deficient as compared to the working endpoint that gets assigned in your 2-function config.

Quoted 7 lines

Jack,

thanks for the pointer, it is indeed the issue on AM57x device. The reset value of GTXFIFOSIZ for ep1~4 have size of 0x184, but ep5~15 have only size of 0x13 (which is 120 bytes), which is not enough for high-speed bulk xfers. I manually adjusted the fifo memory allocation, now my test case works.

Cool! I'm glad my suggestion was on the right track.

Quoted 4 lines

Felipe,

Is there anything the dwc3 gadget driver can do to better handle this kind of devices, which don't have enough fifo buffers for all TX eps?

A long time ago...

commit bc5081617faeb3b2f0c126dc37264b87af7da47f

Author: Felipe Balbi <[email protected]>
Date:   Thu Feb 4 14:18:01 2016 +0200

usb: dwc3: drop FIFO resizing logic

That FIFO resizing logic was added to support OMAP5 ES1.0 which had a bogus default FIFO size. I can't remember the exact size of default FIFO, but it was less than one bulk superspeed packet (<1024) which would prevent USB3 from ever working on OMAP5 ES1.0.

However, OMAP5 ES1.0 support has been dropped by commit aa2f4b16f830 ("ARM: OMAP5: id: Remove ES1.0 support") which renders FIFO resizing unnecessary.

Quoted 8 lines

Prior to this there was a function dwc3_gadget_resize_tx_fifo(), enabled via DT flag, which enumerated all the endpoints and recalculated the TX FIFO sizes based on the transfer type of the EPs' configuration.

Unfortunately some Qualcomm SoCs are still plagued by having insufficient defaults, so we resort to carrying this function on our downstream kernels. It seems TI AM57x still has this problem too?

Yes, it appears to be the case.

-Bin.

author
#5 lore raw
Quoted 80 lines · Bin Liu
Quoted 79 lines · Jack Pham

Hi Bin,

Quoted 35 lines · Bin Liu

Hi,

Sorry for the delay getting back on this. I was offline for quite some time.

Quoted 22 lines · Jack Pham

Hi Bin,

Quoted 19 lines · Bin Liu

Hi Felipe,

I am having an issue with dwc3 on TI AM57x device, and would like to ask for your comments.

I use configfs to create a multi-function gadget on dwc3, mass_storage is the last function, it seems if I create 3 functions, the mass_storage enumeration will fail on the host. It works fine if only create 2 functions.

The dwc3 tracepoints log shows after all the ep0 transfers for mass_storage, the very first epXin transfer is not complete - dwc3 programmed the urb, but never generates RX completion event. This also matches the bus analyzer trace - dwc3 NAKs the very first IN token for ever.

I use the attached script to create the gadget, The macro FUNCS in the beginning of the script defines the functions to be created.

Quoted 6 lines
Quoted 1 line

Any comments are appreciated.

A stab in the dark here but what is the value of GTXFIFOSIZ(X)[15:0] for epXin on your device? Is it at least wMaxPacketSize? Depending on the default hardware values it might be deficient as compared to the working endpoint that gets assigned in your 2-function config.

Quoted 7 lines

Jack,

thanks for the pointer, it is indeed the issue on AM57x device. The reset value of GTXFIFOSIZ for ep1~4 have size of 0x184, but ep5~15 have only size of 0x13 (which is 120 bytes), which is not enough for high-speed bulk xfers. I manually adjusted the fifo memory allocation, now my test case works.

Cool! I'm glad my suggestion was on the right track.

Quoted 4 lines

Felipe,

Is there anything the dwc3 gadget driver can do to better handle this kind of devices, which don't have enough fifo buffers for all TX eps?

A long time ago...

commit bc5081617faeb3b2f0c126dc37264b87af7da47f

Author: Felipe Balbi <[email protected]>
Date:   Thu Feb 4 14:18:01 2016 +0200

usb: dwc3: drop FIFO resizing logic

That FIFO resizing logic was added to support OMAP5 ES1.0 which had a bogus default FIFO size. I can't remember the exact size of default FIFO, but it was less than one bulk superspeed packet (<1024) which would prevent USB3 from ever working on OMAP5 ES1.0.

However, OMAP5 ES1.0 support has been dropped by commit aa2f4b16f830 ("ARM: OMAP5: id: Remove ES1.0 support") which renders FIFO resizing unnecessary.

Prior to this there was a function dwc3_gadget_resize_tx_fifo(), enabled via DT flag, which enumerated all the endpoints and recalculated the TX FIFO sizes based on the transfer type of the EPs' configuration.

Quoted 5 lines
Quoted 3 lines

Unfortunately some Qualcomm SoCs are still plagued by having insufficient defaults, so we resort to carrying this function on our downstream kernels. It seems TI AM57x still has this problem too?

Yes, it appears to be the case.

Wait, AM57x seems to be different. The issue mentioned in the commit above is about bogus FIFO size, but GTXFIFOSIZ on AM57x are correct. it is just that AM57x only has about total 13K+ FIFO memory, so only up to 4 TX endpoints which can have 3K FIFO, any use case which requires more than 4 TX endpoints will fail.

So maybe the driver could have a sanity check on FIFO size when allocating endpoints?

-Bin.

#6 lore raw

Hi,

Quoted 76 lines · Bin Liu
Quoted 75 lines · Bin Liu
Quoted 74 lines · Jack Pham

Hi Bin,

Quoted 35 lines · Bin Liu

Hi,

Sorry for the delay getting back on this. I was offline for quite some time.

Quoted 22 lines · Jack Pham

Hi Bin,

Quoted 19 lines · Bin Liu

Hi Felipe,

I am having an issue with dwc3 on TI AM57x device, and would like to ask for your comments.

I use configfs to create a multi-function gadget on dwc3, mass_storage is the last function, it seems if I create 3 functions, the mass_storage enumeration will fail on the host. It works fine if only create 2 functions.

The dwc3 tracepoints log shows after all the ep0 transfers for mass_storage, the very first epXin transfer is not complete - dwc3 programmed the urb, but never generates RX completion event. This also matches the bus analyzer trace - dwc3 NAKs the very first IN token for ever.

I use the attached script to create the gadget, The macro FUNCS in the beginning of the script defines the functions to be created.

Quoted 6 lines
Quoted 1 line

Any comments are appreciated.

A stab in the dark here but what is the value of GTXFIFOSIZ(X)[15:0] for epXin on your device? Is it at least wMaxPacketSize? Depending on the default hardware values it might be deficient as compared to the working endpoint that gets assigned in your 2-function config.

Quoted 7 lines

Jack,

thanks for the pointer, it is indeed the issue on AM57x device. The reset value of GTXFIFOSIZ for ep1~4 have size of 0x184, but ep5~15 have only size of 0x13 (which is 120 bytes), which is not enough for high-speed bulk xfers. I manually adjusted the fifo memory allocation, now my test case works.

Cool! I'm glad my suggestion was on the right track.

Quoted 4 lines

Felipe,

Is there anything the dwc3 gadget driver can do to better handle this kind of devices, which don't have enough fifo buffers for all TX eps?

A long time ago...

commit bc5081617faeb3b2f0c126dc37264b87af7da47f

Author: Felipe Balbi <[email protected]>
Date:   Thu Feb 4 14:18:01 2016 +0200

usb: dwc3: drop FIFO resizing logic

That FIFO resizing logic was added to support OMAP5 ES1.0 which had a bogus default FIFO size. I can't remember the exact size of default FIFO, but it was less than one bulk superspeed packet (<1024) which would prevent USB3 from ever working on OMAP5 ES1.0.

However, OMAP5 ES1.0 support has been dropped by commit aa2f4b16f830 ("ARM: OMAP5: id: Remove ES1.0 support") which renders FIFO resizing unnecessary.

Quoted 8 lines
Quoted 8 lines
Quoted 8 lines

Prior to this there was a function dwc3_gadget_resize_tx_fifo(), enabled via DT flag, which enumerated all the endpoints and recalculated the TX FIFO sizes based on the transfer type of the EPs' configuration.

Unfortunately some Qualcomm SoCs are still plagued by having insufficient defaults, so we resort to carrying this function on our downstream kernels. It seems TI AM57x still has this problem too?

Why was I never told about this? We had ONE SoC that needed this feature because of a known-bad silicon that was completely dropped from mainline. Note that this commit was doing a "best-effort" at getting something know-bad to work. There's no guarantee that we even have enough FIFO space while configuring everything to 1024 bytes FIFO size.

If you to bring that back, please propose a patch, but it'll be difficult to make it work reliably for everybody.

Quoted 7 lines
Quoted 1 line

Yes, it appears to be the case.

Wait, AM57x seems to be different. The issue mentioned in the commit above is about bogus FIFO size, but GTXFIFOSIZ on AM57x are correct. it is just that AM57x only has about total 13K+ FIFO memory, so only up to 4 TX endpoints which can have 3K FIFO, any use case which requires more than 4 TX endpoints will fail.

wait, your GTXFIFOSZ is configured for 3K? That's completely unnecessary, even for high bandwidth highspeed isoc endpoints. Otherwise we would use 16kiB FIFO size for bulk endpoints with burst enabled.

Quoted 2 lines

So maybe the driver could have a sanity check on FIFO size when allocating endpoints?

We have a "sanity check". We read FIFO configuration from HW while telling the framework what the endpoint can do. Now, if you during coreConsultant the core was badly configured, then the driver doesn't do much.

static int dwc3_gadget_init_in_endpoint(struct dwc3_ep *dep)

{
	struct dwc3 *dwc = dep->dwc;
	int mdwidth;
	int kbytes;
	int size;

	mdwidth = DWC3_MDWIDTH(dwc->hwparams.hwparams0);
	/* MDWIDTH is represented in bits, we need it in bytes */
	mdwidth /= 8;

	size = dwc3_readl(dwc->regs, DWC3_GTXFIFOSIZ(dep->number >> 1));
	if (dwc3_is_usb31(dwc))
		size = DWC31_GTXFIFOSIZ_TXFDEF(size);
	else
		size = DWC3_GTXFIFOSIZ_TXFDEF(size);

	/* FIFO Depth is in MDWDITH bytes. Multiply */
	size *= mdwidth;

	kbytes = size / 1024;
	if (kbytes == 0)
		kbytes = 1;

	/*
	 * FIFO sizes account an extra MDWIDTH * (kbytes + 1) bytes for
	 * internal overhead. We don't really know how these are used,
	 * but documentation say it exists.
	 */
	size -= mdwidth * (kbytes + 1);
	size /= kbytes;

	usb_ep_set_maxpacket_limit(&dep->endpoint, size);

	dep->endpoint.max_streams = 15;
	dep->endpoint.ops = &dwc3_gadget_ep_ops;
	list_add_tail(&dep->endpoint.ep_list,
			&dwc->gadget.ep_list);
	dep->endpoint.caps.type_iso = true;
	dep->endpoint.caps.type_bulk = true;
	dep->endpoint.caps.type_int = true;

	return dwc3_alloc_trb_pool(dep);
}
Signature · 1 line
balbi