dwc_otg issues

[jefferee]

Hi, everyone.

I have a Pi 3, running piCore 8.1.5, connected to a USB/RS-232 converter, with FT232RL chipsets.
The converter has 16 RS-232 outs, so there are 16 FT232RL chips, with three high-speed multi-TT hubs inside the box.
I have my own software communicating with various devices through the RS-232 ports.

Every few days, the Pi will lock up (no RS-232, no USB, no ethernet, no serial tty on the GPIO pins...) and after an event over the weekend I got the following kernel dump.
Any ideas what's going on? I know the USB controller on the Pi handles a lot of IRQ through software, am I just overtaxing the bus or processor trying to manage that many serial ports?

Thanks for your input.

Code: [Select]

tc@box:~$ [102323.479317] ------------[ cut here ]------------
[102323.485262] kernel BUG at drivers/usb/host/dwc_otg/dwc_otg_hcd_intr.c:2438!
[102323.493532] Internal error: Oops - BUG: 0 [#1] SMP ARM
[102323.499941] Modules linked in: ipv6 i2c_dev brcmfmac brcmutil cfg80211 ftdi_sio usbserial squashfs rfkill i2c_bcm2708 bcm2835_gpiomem bcm2835_wdt zram zsmalloc lz4_compress spi_bcm2835 uio_pdrv_genirq uio
[102323.522003] CPU: 0 PID: 0 Comm: swapper/0 Not tainted 4.4.39-piCore_v7+ #1
[102323.530215] Hardware name: BCM2709
[102323.534919] task: 80873ce8 ti: 8086e000 task.ti: 8086e000
[102323.541666] PC is at dwc_otg_hcd_handle_hc_fsm+0x17c/0x64c
[102323.548473] LR is at 0x8
[102323.552271] pc : [<8044db68>]    lr : [<00000008>]    psr: 400001d3
[102323.552271] sp : 8086fd28  ip : b8aaf500  fp : 8086fd5c
[102323.566343] r10: 00000002  r9 : b9031fc0  r8 : b9a65800
[102323.572836] r7 : b9031fc0  r6 : b8ec2b80  r5 : 00000006  r4 : b9a65400
[102323.580617] r3 : 00000000  r2 : f39805c0  r1 : 00000002  r0 : 00000001
[102323.588375] Flags: nZcv  IRQs off  FIQs off  Mode SVC_32  ISA ARM  Segment kernel
[102323.598254] Control: 10c5383d  Table: 3007c06a  DAC: 00000055
[102323.605244] Process swapper/0 (pid: 0, stack limit = 0x8086e210)
[102323.612489] Stack: (0x8086fd28 to 0x80870000)
[102323.618074] fd20:                   8086fd5c 80448ff8 8044bd40 80448ce0 00000006 00000001
[102323.628616] fd40: b9a65400 b8ec2b80 f39805c0 b9031fc0 8086fdac 8086fd60 8044e1b0 8044d9f8
[102323.639169] fd60: 8086fdac 8086fd70 80448ff8 80448320 b9a6541c f3980000 00000000 b44a8700
[102323.649723] fd80: 7f062530 00000006 0000004f b9a65400 00000001 808bf0d0 f301080e 00000002
[102323.660329] fda0: 8086fdcc 8086fdb0 8044e740 8044e044 b9a65400 b8a6f800 00000001 02000008
[102323.671040] fdc0: 8086fdfc 8086fdd0 8044e9e4 8044e69c 8044afc4 b8eabb00 b9880f5c 00000000
[102323.681848] fde0: 00000000 0000003e 00000000 8086e000 8086fe0c 8086fe00 8044afe0 8044e78c
[102323.692777] fe00: 8086fe1c 8086fe10 8041ccb0 8044afd0 8086fe5c 8086fe20 80071680 8041cc88
[102323.703810] fe20: 80499d70 80499a30 b9880f00 00000000 80499d90 b9880f00 b9880f5c 00000000
[102323.714873] fe40: 00000000 00000000 80846a30 00000000 8086fe7c 8086fe60 8007187c 800715e0
[102323.726014] fe60: b9880f00 b9880f10 00000000 00000000 8086fe94 8086fe80 80074cb0 80071834
[102323.737208] fe80: 80871aec 00000000 8086fea4 8086fe98 80070c3c 80074c08 8086febc 8086fea8
[102323.748416] fea0: 8035347c 80070c18 808694ec 00000014 8086fecc 8086fec0 80070c3c 8035344c
[102323.759625] fec0: 8086fef4 8086fed0 80070f40 80070c18 80010a30 60000013 ffffffff 8086ff4c
[102323.770832] fee0: 805c9e04 80846a30 8086ff04 8086fef8 80010970 80070eec 8086ff14 8086ff08
[102323.782040] ff00: 80009538 80010954 8086ff74 8086ff18 805c5344 80009490 00000000 b9f623c8
[102323.793247] ff20: 79c51b58 00000000 8086e000 808705dc ffffffff 80870580 805c9e04 80846a30
[102323.804455] ff40: 00000000 8086ff74 8086ff68 8086ff68 80010a2c 80010a30 60000013 ffffffff
[102323.815664] ff60: 80010a2c 800d9940 8086ff84 8086ff78 800643dc 80010a0c 8086ffa4 8086ff88
[102323.826873] ff80: 800645fc 800643b4 808673a0 8086ff88 805c1304 80870500 8086ffb4 8086ffa8
[102323.838081] ffa0: 805bfad4 800643fc 8086fff4 8086ffb8 807f8cec 805bfa6c ffffffff ffffffff
[102323.849290] ffc0: 00000000 807f86e0 00000000 80846a30 808dd294 80870560 80846a2c 80875094
[102323.860499] ffe0: 0000406a 410fd034 00000000 8086fff8 00008078 807f8984 00000000 00000000
[102323.871720] [<8044db68>] (dwc_otg_hcd_handle_hc_fsm) from [<8044e1b0>] (dwc_otg_hcd_handle_hc_n_intr+0x178/0x658)
[102323.885042] [<8044e1b0>] (dwc_otg_hcd_handle_hc_n_intr) from [<8044e740>] (dwc_otg_hcd_handle_hc_intr+0xb0/0xf0)
[102323.898279] [<8044e740>] (dwc_otg_hcd_handle_hc_intr) from [<8044e9e4>] (dwc_otg_hcd_handle_intr+0x264/0x35c)
[102323.911255] [<8044e9e4>] (dwc_otg_hcd_handle_intr) from [<8044afe0>] (dwc_otg_hcd_irq+0x1c/0x28)
[102323.923094] [<8044afe0>] (dwc_otg_hcd_irq) from [<8041ccb0>] (usb_hcd_irq+0x34/0x48)
[102323.933884] [<8041ccb0>] (usb_hcd_irq) from [<80071680>] (handle_irq_event_percpu+0xac/0x254)
[102323.945465] [<80071680>] (handle_irq_event_percpu) from [<8007187c>] (handle_irq_event+0x54/0x78)
[102323.957399] [<8007187c>] (handle_irq_event) from [<80074cb0>] (handle_level_irq+0xb4/0x164)
[102323.968811] [<80074cb0>] (handle_level_irq) from [<80070c3c>] (generic_handle_irq+0x30/0x44)
[102323.980317] [<80070c3c>] (generic_handle_irq) from [<8035347c>] (bcm2836_chained_handle_irq+0x3c/0x50)
[102323.992700] [<8035347c>] (bcm2836_chained_handle_irq) from [<80070c3c>] (generic_handle_irq+0x30/0x44)
[102324.005082] [<80070c3c>] (generic_handle_irq) from [<80070f40>] (__handle_domain_irq+0x60/0xc8)
[102324.016851] [<80070f40>] (__handle_domain_irq) from [<80010970>] (handle_IRQ+0x28/0x2c)
[102324.027918] [<80010970>] (handle_IRQ) from [<80009538>] (bcm2836_arm_irqchip_handle_irq+0xb4/0xbc)
[102324.039952] [<80009538>] (bcm2836_arm_irqchip_handle_irq) from [<805c5344>] (__irq_svc+0x44/0x5c)
[102324.051892] Exception stack(0x8086ff18 to 0x8086ff60)
[102324.058503] ff00:                                                       00000000 b9f623c8
[102324.069607] ff20: 79c51b58 00000000 8086e000 808705dc ffffffff 80870580 805c9e04 80846a30
[102324.080664] ff40: 00000000 8086ff74 8086ff68 8086ff68 80010a2c 80010a30 60000013 ffffffff
[102324.091721] [<805c5344>] (__irq_svc) from [<80010a30>] (arch_cpu_idle+0x30/0x4c)
[102324.101989] [<80010a30>] (arch_cpu_idle) from [<800643dc>] (default_idle_call+0x34/0x48)
[102324.112959] [<800643dc>] (default_idle_call) from [<800645fc>] (cpu_startup_entry+0x20c/0x2b8)
[102324.124456] [<800645fc>] (cpu_startup_entry) from [<805bfad4>] (rest_init+0x74/0x8c)
[102324.135076] [<805bfad4>] (rest_init) from [<807f8cec>] (start_kernel+0x374/0x3e0)
[102324.145431] Code: 1a0000c9 e3110004 1affffdb f10c0040 (e7f001f2)

[Rich]

Hi jefferee
I think you need to get the source package for your kernel and take a look around line 2438 of:
drivers/usb/host/dwc_otg/dwc_otg_hcd_intr.c
to get a better idea of what the actual error is.

[jefferee]

Thanks, Rich. I'll have a look.

[curaga]

The Pi's usb is just buggy, that's been known for years. You probably knew about it before deciding to buy one.

See
https://github.com/raspberrypi/linux/issues/739
The very same thing reported in 2014.

[jefferee]

Thanks, Rich. I'll have a look.

So it looks like we get into this case of

Code: [Select]

switch (st->fsmand hit the BUG() statement on line 2438.

Code: [Select]

case FIQ_NP_SPLIT_LS_ABORTED:
/* A few cases can cause this - either an unknown state on a SSPLIT or
* STALL/data toggle error response on a CSPLIT */
if (hcint.b.stall) {
handle_hc_stall_intr(hcd, hc, hc_regs, qtd);
} else if (hcint.b.datatglerr) {
handle_hc_datatglerr_intr(hcd, hc, hc_regs, qtd);
} else if (hcint.b.bblerr) {
handle_hc_babble_intr(hcd, hc, hc_regs, qtd);
} else if (hcint.b.ahberr) {
handle_hc_ahberr_intr(hcd, hc, hc_regs, qtd);
} else {
local_fiq_disable();
BUG();
}
break;
Throwing FIQ_NP_SPLIT_LS_ABORTED: into da Google shows the following bug report, which declares it fixed and closed in the 4.9 kernel. Guess it's time to try piCore 9.0.3!
https://github.com/raspberrypi/linux/issues/1506

[jefferee]

The Pi's usb is just buggy, that's been known for years. You probably knew about it before deciding to buy one.

See
https://github.com/raspberrypi/linux/issues/739
The very same thing reported in 2014.

Yes, I'm well aware that OTG in general is a weak point for what I am trying to do here. Dumping the R-Pi for something that has better (i.e., not OTG) usb support is definitely on the table.