WelcomeWelcome | FAQFAQ | DownloadsDownloads | WikiWiki

Author Topic: how do firmware extensions work without startup script?  (Read 2135 times)

Offline GNUser

  • Wiki Author
  • Hero Member
  • *****
  • Posts: 1461
how do firmware extensions work without startup script?
« on: August 26, 2024, 06:24:31 PM »
I've noticed that the firmware extensions (e.g., firmware-atheros.tcz, firmware-mediatek.tcz, etc.) do not have a startup script. However, device drivers (e.g., ath9k) are able to find firmware files early in the boot process without any problems.

In TCL, things that need to be available very early in the boot process are usually made available by an additional, custom initramfs. It seems I'm missing the magic sauce in the TCL boot process that ensures firmware extensions are loaded before device drivers need the firmware files.
« Last Edit: August 26, 2024, 06:29:34 PM by GNUser »

Offline GNUser

  • Wiki Author
  • Hero Member
  • *****
  • Posts: 1461
Re: how do firmware extensions work without startup script?
« Reply #1 on: August 26, 2024, 06:41:40 PM »
Is the trick simply to make sure that the firmware extension (e.g., firmware-atheros.tcz) appears before the driver extension (e.g., wireless-KERNEL.tcz) in onboot.lst?
« Last Edit: August 26, 2024, 06:44:26 PM by GNUser »

Offline Rich

  • Administrator
  • Hero Member
  • *****
  • Posts: 11518
Re: how do firmware extensions work without startup script?
« Reply #2 on: August 26, 2024, 08:07:54 PM »
Hi GNUser
I don't have a precise answer for you, but I do make it a
point to place firmware extensions at the top of onboot.lst.

Two things I noticed:
1. dmesg will often show multiple attempts to load firmware.
2. /lib/udev  contains  firmware.sh.

This makes me suspect  udev  might be involved.

Offline curaga

  • Administrator
  • Hero Member
  • *****
  • Posts: 11027
Re: how do firmware extensions work without startup script?
« Reply #3 on: August 27, 2024, 03:26:17 AM »
The udev trigger for new modules on boot is delayed until after all extensions are loaded. So usually the order in onboot.lst isn't significant.
The only barriers that can stop you are the ones you create yourself.

Offline GNUser

  • Wiki Author
  • Hero Member
  • *****
  • Posts: 1461
Re: how do firmware extensions work without startup script?
« Reply #4 on: August 27, 2024, 09:59:45 AM »
So usually the order in onboot.lst isn't significant.
I can confirm that. Putting firmware-mediatek.tcz (for example) at very bottom of onboot.lst has no ill effect--after boot process is complete, firmware has been loaded and device is operable.

The udev trigger for new modules on boot is delayed until after all extensions are loaded.
curaga, can you please help me see how this is the case? This is the sequence of events as best as I can tell: /init -> /sbin/init (Busybox init) -> /etc/inittab -> /etc/init.d/rcS -> /etc/init.d/tc-config

In tc-config, udev is started around line 107, but extensions don't get loaded until around line 557. Something seems to be happening after extensions are loaded that causes modules to go looking for firmware. What am I missing?
« Last Edit: August 27, 2024, 10:03:04 AM by GNUser »

Offline Rich

  • Administrator
  • Hero Member
  • *****
  • Posts: 11518
Re: how do firmware extensions work without startup script?
« Reply #5 on: August 27, 2024, 10:27:00 AM »
Hi GNUser
Check tc-config for:
Code: [Select]
/usr/bin/tce-setup "booting"tce-setup fires off udev if it detects /etc/sysconfig/newmodules.

Offline GNUser

  • Wiki Author
  • Hero Member
  • *****
  • Posts: 1461
Re: how do firmware extensions work without startup script?
« Reply #6 on: August 27, 2024, 10:32:42 AM »
Nice find, Rich. I almost have the complete picture now. Do you know what creates /etc/sysconfig/newmodules?

Offline GNUser

  • Wiki Author
  • Hero Member
  • *****
  • Posts: 1461
Re: how do firmware extensions work without startup script?
« Reply #7 on: August 27, 2024, 10:36:24 AM »
Do you know what creates /etc/sysconfig/newmodules?
I found it: It's /usr/bin/tce-load:

Code: [Select]
...
[ -n "`find /mnt/test/ -type d -name modules`" ] && MODULES=TRUE
...

update_system() {
if [ "$BOOTING" ]; then
[ "$MODULES" ] && sudo /bin/touch /etc/sysconfig/newmodules
So if tce-load loads an extension that contains modules, it creates the newmodules flag. The new modules flag causes tce-setup to fire off udev after all extensions have been loaded. Mystery solved. Thank you both very much.
« Last Edit: August 27, 2024, 10:40:19 AM by GNUser »

Offline Rich

  • Administrator
  • Hero Member
  • *****
  • Posts: 11518
Re: how do firmware extensions work without startup script?
« Reply #8 on: August 27, 2024, 10:45:22 AM »
Hi GNUser
Nice find, Rich. I almost have the complete picture now. ...
Yeah, there's a lot going on there. It can be tricky trying to follow it all.

Offline GNUser

  • Wiki Author
  • Hero Member
  • *****
  • Posts: 1461
Re: how do firmware extensions work without startup script?
« Reply #9 on: August 30, 2024, 07:51:51 AM »
Hi Rich. I did some simple experiments and it seems I'm missing a small piece of the puzzle.

I connected a wifi usb adapter that uses the Atheros AR9271 chipset.

First experiment (firmware extension loaded during boot):
    I add firmware-atheros.tcz to my onboot.lst. Then I reboot. When machine is done booting, I do ifconfig -a I can see wlan1 listed, so the usb adapter is working.

Second experiment (firmware extension loaded manually after boot):
    I remove firmware-atheros.tcz from onboot.lst. Then I reboot. After boot is complete, I open a terminal and run these commands:
Code: [Select]
$ dmesg | grep firmware # the failed firmware load here is not a surprise
[    0.071368] Spectre V2 : Enabling Restricted Speculation for firmware calls
[    2.880738] psmouse serio2: trackpoint: IBM TrackPoint firmware: 0x0e, buttons: 3/3
[   13.148438] platform regulatory.0: Direct firmware load for regulatory.db failed with error -2
[   13.233707] usb 2-1.2: Direct firmware load for ath9k_htc/htc_9271-1.4.0.fw failed with error -2
[   13.285943] usb 2-1.2: Direct firmware load for htc_9271.fw failed with error -2
[   13.295216] usb 2-1.2: no suitable firmware found!
[   13.295221] usb 2-1.2: ath9k_htc: Failed to get firmware htc_9271.fw
$ tce-load -i firmware-atheros
$ sudo depmod -a
$ sudo udevadm trigger
$ dmesg | grep firmware # why no new attempt at loading firmware?
[    0.071368] Spectre V2 : Enabling Restricted Speculation for firmware calls
[    2.880738] psmouse serio2: trackpoint: IBM TrackPoint firmware: 0x0e, buttons: 3/3
[   13.148438] platform regulatory.0: Direct firmware load for regulatory.db failed with error -2
[   13.233707] usb 2-1.2: Direct firmware load for ath9k_htc/htc_9271-1.4.0.fw failed with error -2
[   13.285943] usb 2-1.2: Direct firmware load for htc_9271.fw failed with error -2
[   13.295216] usb 2-1.2: no suitable firmware found!
[   13.295221] usb 2-1.2: ath9k_htc: Failed to get firmware htc_9271.fw
Then I run ifconfig -a there is no wlan1 in the output (obviously, since firmware has not been loaded onto the device). I must manually unplug the adapter and plug it back in for it to work.

Any idea what's the secret sauce in the boot process that re-detects usb devices after driver and firmware extensions have been loaded? (It's not udevadm trigger.)  I've looked in /etc/init.d/ as well as in /usr/bin/tce-setup and have not found an obvious answer.
« Last Edit: August 30, 2024, 08:22:19 AM by GNUser »

Offline Paul_123

  • Administrator
  • Hero Member
  • *****
  • Posts: 1229
Re: how do firmware extensions work without startup script?
« Reply #10 on: August 30, 2024, 08:22:10 AM »
That doesn't work, udev will find the usb device and tell the kernel to load the driver.  The driver is what loads the firmware.   The firmware must be loaded before (or during boot) the kernel device driver is loaded.


Offline GNUser

  • Wiki Author
  • Hero Member
  • *****
  • Posts: 1461
Re: how do firmware extensions work without startup script?
« Reply #11 on: August 30, 2024, 08:32:36 AM »
The driver is what loads the firmware.   The firmware must be loaded before (or during boot) the kernel device driver is loaded.
I figured it's the driver that loads the firmware. But that doesn't explain everything either, because the following doesn't work :-\
Code: [Select]
$ tce-load -i firmware-atheros
firmware-atheros.tcz: OK
$ sudo modprobe -r ath9k_htc
$ sudo modprobe ath9k_htc
Just to clarify that all this is just an exercise in understanding. All my gear is working fine.
« Last Edit: August 30, 2024, 08:34:56 AM by GNUser »

Offline GNUser

  • Wiki Author
  • Hero Member
  • *****
  • Posts: 1461
Re: how do firmware extensions work without startup script?
« Reply #12 on: August 30, 2024, 08:38:43 AM »
So there's no way for a firmware extension loaded after boot to help the user short of manually disconnecting and reconnecting the relevant device?
« Last Edit: August 30, 2024, 08:52:31 AM by GNUser »

Offline Paul_123

  • Administrator
  • Hero Member
  • *****
  • Posts: 1229
Re: how do firmware extensions work without startup script?
« Reply #13 on: August 30, 2024, 09:20:54 AM »
Don’t load the device drivers at boot either (in this case wireless-KERNEL.tcz).

Offline Rich

  • Administrator
  • Hero Member
  • *****
  • Posts: 11518
Re: how do firmware extensions work without startup script?
« Reply #14 on: August 30, 2024, 09:38:29 AM »
Hi GNUser
Despite the firmware failure, does  lsmod  show that the
driver is still loaded? If it does, see if this works:
Code: [Select]
sudo modprobe -r ath9k_htc
tce-load -i firmware-atheros
sudo udevadm trigger