firmware proposals to use /lib/firmware for the Dev Team

[aus9]

This is my attempt to convince the Dev Team members the merit vs the status quo
of allowing me to update firmware to build to pathway /lib/firmware.

Short History
##########
About a year ago, I had the exact same hardware that I have now. A tower which has no internal wifi hardware.
I have ethernet and am happy with it
In the move to 16x kernel on x86_64 I had internet on ethernet but could not get wifi to work with my usb wlan network adaptor.
A question was posed to me that asked could I explain this development. I could not and still can not.
Instead I gave dmesg snippets and a web browser link to a asciinema movie showing that

Code: [Select]

inxi -Nxxx hanging or not showing the correct information
We found out later from another member that the firmware for some devices now needed /lib/firmware
A decision was made that instead of allowing me to submit against /lib/firmware that rootfs was modified to allow a sym link
so that /lib/firmware links back to /usr/local/lib/firmware

inxi FYI
######
For those not familar with inxi here is my ethernet result

Code: [Select]

inxi -Nxxx
Network:
  Device-1: Realtek RTL8111/8168/8211/8411 PCI Express Gigabit Ethernet
    vendor: Micro-Star MSI driver: r8169 v: kernel pcie: speed: 2.5 GT/s
    lanes: 1 port: f000 bus-ID: 22:00.0 chip-ID: 10ec:8168 class-ID: 0200inxi AFAIK never reveals the firmware only the kernel module also known as the driver.
If I run

Code: [Select]

modinfo r8169 | grep firmware
modinfo r8169 | grep 10EC | grep 8168 I get multiple hits for firmware and one hit for my device, as expected.

Code: [Select]

dmesg | grep firmware and {error} can also be informative YMMV

PROPOSAL (1)
#############
I am not asking for any change to 16x members. While x86_64 and maybe the others in Beta testing I propose

A) Allow me to update and upload to drop box the huge updates for the 40 odd TCEs built hopefully correctly with byte size so that that part of my script that makes the TCEs reads in part

Code: [Select]

mksquashfs -b 16384  blah blahThe dropbox link will be sent by pm unless Dev Team prefer I broadcast it.

B) The various Dev Team members that need to think about modifying rootfs sym link need to in house test my updates against their hardware. Naturally I will not dare upload if my in house testing fails.
That means I will mod my own 17x rootfs for inhouse testing.

C) If all appears OK we do a post to 17x members not to update unless they are on the latest test release with root fs changed as noted.

PROPOSAL (2)
############

Subject to (1) succeeding

A short history. We have had to split some packages in the past as they were getting too big.
I am aware that experienced members can make their own decisions to have some firmware in their backup
or make their personal TCE YMMV. I am aware that if I convince the Dev Team, some packages  without
splitting are huge. I prefer not split any that use the same kernel module.
The sizes I am referring to is my unsubmitted build script for 2025. I expect sizes to increase again.

However I am scared to split any package that gives members graphic firmware.
eg amdgpu i915 nvidia (with a possible exception)

nvidia is going to over 155Mb but has sub Dir with allegely the same kernel module
It is possible with a forum post we could split this....and I propose we do not have firmware-nvidia to confuse people if they are not already confused
If you look at Apps list something like firmware-nvida-ga (a group of ga Dirs) looks ugly I know

Looking at existing Apps and size on disk consider TCEs like
atheros rounded 30M expect that to double contains multiple kernel modules that could be split
intel rounded 20M expect to double contains multiple kernel modules that could be split
marvel already at 80M contains multiple kernel modules that could be split

CONTIGENCY PLAN
###############
Its good to plan for the future. I can not forecast the future but I am a grumpy old man and have a history of spitting the dummy.

If my proposal does not succeed for what ever reasons....the way I built the build script is to move dirs out of the unpacked source. That leaves you to decide where to move any new leftovers And you decide that by reading whence...I am showing the raw link its easy to read
https://gitlab.com/kernel-firmware/linux-firmware/-/raw/main/WHENCE?ref_type=heads

So whence tells you the name of kernel module and the dir (if any) and the firmware names
Then you find a match then read what kind of device it is and make a decison to move it somewhere.
The hard work has already been done. So if I spit the dummy....I allege it will not be hard for someone to take over

Thanks for reading

[Paul_123]

I agree that quite a few devices want to find the firmware in /lib/firmware.  And I have devices that want to see it there.

I have not seen any issues with a symlink  /lib/firmware -> ../usr/local/lib/firmware

Firmware extensions by default are all symlinked anyway.

Code: [Select]

$ readlink /lib/firmware/rtl_nic/rtl8107e-2.fw
/tmp/tcloop/firmware-rtl_nic/usr/local/lib/firmware/rtl_nic/rtl8168h-2.fw
r8169 is a kernel driver for realtek pci express chips.  Mine behaves just fine with firmware at the above paths. The kernel almost never reports success on firmware loads, but if I remove the firmware-rtl_nic.tcz extension, I get

Code: [Select]

tc@box:~$ dmesg | grep firmware
[    0.024216] raspberrypi-firmware soc:firmware: Attached to firmware from 2025-05-14T12:23:36, variant start
[    0.028222] raspberrypi-firmware soc:firmware: Firmware hash is 17084b403fb60475b8ee2641c26049a7d54bf153
[    7.556942] r8169 0000:01:00.0: Direct firmware load for rtl_nic/rtl8168h-2.fw failed with error -2
[    7.566560] r8169 0000:01:00.0: Unable to load firmware rtl_nic/rtl8168h-2.fw (-2)


Code: [Select]

tc@box:~$ lspci
00:00.0 PCI bridge: Broadcom Inc. and subsidiaries BCM2711 PCIe Bridge (rev 20)
01:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller (rev 15)
tc@box:~$ inxi -Nxxx
Network:
  Device-1: Realtek RTL8111/8168/8411 PCI Express Gigabit Ethernet
    driver: r8169 v: kernel pcie: speed: 2.5 GT/s lanes: 1 port: <unassigned>
    bus-ID: 01:00.0 chip-ID: 10ec:8168 class-ID: 0200
  Device-2: bcm2835-mmc driver: mmc_bcm2835 v: N/A port: N/A bus-ID: N/A
    chip-ID: brcm:fe300000 class-ID: mmcnr
  Device-3: bcm2711-genet-v5 driver: bcmgenet v: N/A port: N/A bus-ID: N/A
    chip-ID: brcm:fd580000 class-ID: ethernet

tc@box:~$ dmesg  | grep 10ec
[    0.665579] pci 0000:01:00.0: [10ec:8168] type 00 class 0x020000 PCIe Endpoint

Sure this is a Raspberry Pi Compute module 4 vs a x86 PC, but there are no modifications of the kernel to change firmware paths.

[Rich]

Hi aus9
Talk about how to deal with this issue began back in February 2025.

It was decided to link /lib/firmware->/usr/local/lib/firmware in June 2025:
https://forum.tinycorelinux.net/index.php/topic,27682.0.html

The choice was made because it works, it solves the issue cleanly, and
it minimized impacting extensions.

On that last point, I will add this quote (bold highlighting added by me):

* symlink added for /lib/firmware due to problems loading a few firmware blobs

Hi Juanito. IIRC tce-load is only able to load extensions if destination directory is a real directory (not a symlink).

So the  /lib/firmware  link in 16.1 will break any extension that contains  /lib/firmware/foo  . Fortunately there are only two such extensions: firmware.mediatek.tcz and wireless-regdb.tcz.

In these two extensions, the files in  /lib/firmware  should be moved to /usr/local/lib/firmware.

Only 2 extensions were impacted.

When making design changes, we try to keep those discussions public.
This allows for comments that may highlight something we may have
missed. While that may slow down the process of change, it also reduces
the chances of rushing a change that creates other problems.

While I understand you were trying to fix an issue involving firmware
paths, I feel you should have started a thread asking for some guidance
before implementing such a change.


I feel splitting some of the larger firmware files would be a great idea.
If you want to split some of the larger firmware files and would like some
help deciding where to split them, please start a thread indicating such.

[aus9]

Rich
I do not know who changed my mediatek submission. I have checked my 15x build script and nothing was moved or created by an install script to /lib/firmware

http://tinycorelinux.net/15.x/x86_64/tcz/src/firmware/build-firmware.txt
AG=mediatek
ULLF=usr/local/lib/firmware
there is no part of my build script that created or moved anything to /lib/firmware
So only a Dev Team member could have changed that
.....and done so with disrespect to me by not bothering to put anything the changelog.

(2) wireless-regd was rebuilt to not use any sym link it a clean mkdir but this was done until Oct 2025

#!/bin/sh
P=wireless-regdb
SRC=/usr/local/share/$P/files

# regulatory domain config
##############################
FILE=/etc/modprobe.d/regdom.conf
[ -d /etc/modprobe.d ] || mkdir /etc/modprobe.d
[ -f $FILE ] || cp $SRC/regdom.conf /etc/modprobe.d
chown root:root $FILE
chmod 644       $FILE

# firmware setup for any arch- may result is duplicate file
########################################################
SRC=/usr/local/share/$P/files
L=/lib/firmware
U=/usr/local/lib/firmware
# do not test for -w on dir just try both pathways
# may result in Rpi have 2 files
[ -d $L ] || mkdir -p $L
[ -d $U ] || mkdir -p $U
[ -f $L/regulatory.db ] || cp $SRC/regulatory.db $L
[ -f $U/regulatory.db ] || cp $SRC/regulatory.db $U

anyhow it only needed one Dev member to express their opinion they prefer the status quo.

I am done Do what you like. The build script should move licences out of unpack to make it easier
Or make up your own.