Failing to allocate dma trb pool memory in usb gadget driver

[Aswath-Govindraju]

In the gadget driver(drivers/usb/cdns3/gadget.c) the dma trb buffers are being allocated in the cdns3_gadget_ep_enable() function using cdns3_allocate_trb_pool(). This function is called in atomic context because of which the memory allocation is done with an atomic flag. The coherent memory in TI devices is limited and because of this some modules (g_ether) are failing currently.

In other drivers the allocation was done in init() functions and the allocation was not done in atomic context (For example gadget driver from Synopsys, dwc3).

Can the allocation be shifted to a non-atomic context ??

[sparmarcdns]

In the current implementation, We allocation TRB ring for only used endpoints.
If we move the allocation part in init() function, We need to allocate the memory for all available endpoints.
This is not the ideal solution for a limited memory platform.

Another solution is to reduce the ring size.

We are looking into this and we will revert back with an optimal solution.

[Aswath-Govindraju]

May I know what is the impact of ring size as in the upstream linux kernel the size has been increased to 7200 from 480 bytes ?

Can you also give an estimate of when you will be able to revert with a solution for us to track internally ?

[sparmarcdns]

In mainline, The ring size is increased to provide some buffer and avoid "no free TRB error" in case of some use case, like ADB with MTP.

Are you testing this on the mainline kernel?

[Aswath-Govindraju]

No, I am not testing this using mainline kernel.

[sparmarcdns]

We need further details to work on this.

1. As you mentioned that you are not using the mainline kernel, the EP TRB ring size is 480 bytes in your kernel. Correct me if I am wrong.
2. Could you please provide more details of the test? Please provide logs, used modules, platform details.
3. g_ether uses 5 EPs and 480 bytes per EP is very low and it should be allocated. We are suspecting there might be
   some memory leaks by other modules.
4. I am working on a patch to move the allocation in init() call. This will be only for debug purpose.

[Aswath-Govindraju]

As you mentioned that you are not using the mainline kernel, the EP TRB ring size is 480 bytes in your kernel. Correct me if I am wrong.
->Yes the EP TRB ring size is 480 bytes in my kernel where I am performing tests.

Could you please provide more details of the test? Please provide logs, used modules, platform details.

->I am probing the module g_ether using modprobe g_ether. After doing this there were no error logs on the device side but on the host side the following is the error log

[33288.049867] usb 1-1: new high-speed USB device number 15 using xhci_hcd
[33288.199166] usb 1-1: New USB device found, idVendor=0525, idProduct=a4a2, bcdDevice= 5.04
[33288.199172] usb 1-1: New USB device strings: Mfr=1, Product=2, SerialNumber=0
[33288.199175] usb 1-1: Product: RNDIS/Ethernet Gadget
[33288.199178] usb 1-1: Manufacturer: Linux 5.4.77-00363-g70adb374f488-dirty with usb-ss-gadget
[33288.207618] cdc_subset: probe of 1-1:1.0 failed with error -22
[33288.208880] cdc_ether: probe of 1-1:1.0 failed with error -32
[33288.210275] cdc_subset: probe of 1-1:1.1 failed with error -32

And after this the link on the host side does not come up. The platform that I am using is AM642-evm

I am working on a patch to move the allocation in init() call. This will be only for debug purpose.
-> I performed this test of moving the allocation to init() (also removed the atomic flag) and allocating all the endpoint with a TRB buffer size of 480 bytes. The above error was not coming up.

[Aswath-Govindraju]

Hi,
May I know the reason behind using spin lock and not a mutex in the driver?