WelcomeWelcome | FAQFAQ | DownloadsDownloads | WikiWiki

Author Topic: how do firmware extensions work without startup script?  (Read 2616 times)

Offline curaga

  • Administrator
  • Hero Member
  • *****
  • Posts: 11045
Re: how do firmware extensions work without startup script?
« Reply #15 on: August 30, 2024, 09:41:41 AM »
modprobe -r sometimes fails if the driver is in use. Your chain of modprobe -r and modprobe should normally work. I suppose it could also be another driver that loads the firmware, IIRC ath9k had multiple component drivers.
The only barriers that can stop you are the ones you create yourself.

Offline GNUser

  • Wiki Author
  • Hero Member
  • *****
  • Posts: 1512
Re: how do firmware extensions work without startup script?
« Reply #16 on: August 30, 2024, 02:54:43 PM »
Hi GNUser
Despite the firmware failure, does  lsmod  show that the
driver is still loaded? If it does, see if this works:
Code: [Select]
sudo modprobe -r ath9k_htc
tce-load -i firmware-atheros
sudo udevadm trigger
Hi Rich. Yes  lsmod  shows the driver still loaded. The above commands do not cause the firmware to get loaded.

Hi curaga. I tried modprobe -r followed by modprobe using all of the ath component drivers, no luck.

I'm guessing that in the particular case of usb devices it's the usb system that's requesting the driver to load the firmware?

Offline GNUser

  • Wiki Author
  • Hero Member
  • *****
  • Posts: 1512
Re: how do firmware extensions work without startup script?
« Reply #17 on: August 30, 2024, 04:16:12 PM »
cuaraga, I can confirm that modprobe -r is removing the modules as expected (according to lsmod).

Offline GNUser

  • Wiki Author
  • Hero Member
  • *****
  • Posts: 1512
Re: how do firmware extensions work without startup script?
« Reply #18 on: August 30, 2024, 11:56:26 PM »
For pci devices, the modprobe -r <driver>; modprobe <driver> strategy works just fine for firmware loaded after boot. It seems the modprobe strategy only fails in the specific case of usb devices.

In the special case of usb devices, user needs to either a) load firmware during boot (via onboot.lst) or b) disconnect+reconnect the device after loading the necessary firmware.

While it would be nice to simulate usb device disconnect+reconnect at software level, it seems non-trivial and more trouble than it's worth.

Thread is solved. Thank you all.
« Last Edit: August 30, 2024, 11:57:59 PM by GNUser »

Offline CentralWare

  • Retired Admins
  • Hero Member
  • *****
  • Posts: 765
Re: how do firmware extensions work without startup script?
« Reply #19 on: August 30, 2024, 11:57:18 PM »
@GNUser:

   Take a peek at /usr/local/tce.installed/firmware-atheros and see if there are additional "instructions" which may not being launched when the extension is loaded (-i) manually

   The firmware package is crammed with a good amount of compiler/source debris (looking at 14.x64 file listing) however, you may need to simply probe the device chipset (sudo modprobe ar9271) if it's not already showing in lsmod (just a guess with limited intel) before you launch udevadm.  It's my guess that when you bring ar9271 online...  it likely sees ath#k as a dependency and brings that online as well.  See what happens?!  Again...  it's a total guess...  but looks like /usr/local/lib/firmware/carl9170fw contains most everything you'd need to trace it out yourself! :D

Offline Rich

  • Administrator
  • Hero Member
  • *****
  • Posts: 11639
Re: how do firmware extensions work without startup script?
« Reply #20 on: August 31, 2024, 11:01:16 AM »
Hi GNUser
I did some playing around and came up with this:
Code: [Select]
#!/bin/sh

Debug="test"
# Uncomment the next line to turn on debugging messages.
Debug="echo"

# Vendor ID and product ID (8644:8003).
VID="8644"
PID="8003"

# Find the  modalias  file that contains our VID and PID.
MODALIS="$(find /sys -name modalias | xargs  grep "v""$VID""p""$PID" 2> /dev/null)"

if [ -n "$MODALIS" ]
then
"$Debug" "$MODALIS"
# Grab the path to modalias.
UEVENT="${MODALIS%/*}"
# Grab the device designation from the end of the path.
DEVICE="${UEVENT##*/}"
# Complete path to uevent file.
UEVENT="$UEVENT""/uevent"
"$Debug" "DEVICE=$DEVICE"
"$Debug" "UEVENT=$UEVENT"
fi

if [ -f "$UEVENT" ]
then
# Grab the driver name from uevent file.
DRIVER="$(grep "DRIVER" $UEVENT | cut -d '=' -f2)"
"$Debug" "DRIVER=$DRIVER"
fi

if [ -n "$DRIVER" ]
then
# Find the directory named for the driver.
BINDPATH="$(find /sys -type d -name "$DRIVER")"
# That directory should contain the unbind and bind files.
BINDFILE="$BINDPATH""/bind"
UNBINDFILE="$BINDPATH""/unbind"
"$Debug" "BINDPATH=$BINDPATH"
"$Debug" "BINDFILE=$BINDFILE"
"$Debug" "UNBINDFILE=$UNBINDFILE"
fi

if [ -e "$UNBINDFILE" ]
then
# Remove device.
echo "$DEVICE" | sudo tee "$UNBINDFILE" 1> /dev/null
fi

"$Debug" "Sleep ..."
sleep 3
"$Debug" "... Wake up"


if [ -e "$BINDFILE" ]
then
# Add device back.
echo "$DEVICE" | sudo tee "$BINDFILE" 1> /dev/null
fi
I don't have any USB Wifi devices, but I tested it with a thumb drive.
I did observe remove and add events using  udevadm monitor.
Set the   VID  and  PID  variables to match your wifi device.
A copy of the script is attached. Ask questions if something is not clear.
« Last Edit: August 31, 2024, 02:15:44 PM by Rich »

Offline GNUser

  • Wiki Author
  • Hero Member
  • *****
  • Posts: 1512
Re: how do firmware extensions work without startup script?
« Reply #21 on: September 01, 2024, 03:55:46 PM »
Hi, Rich. Thank you very much for the script.

Here is  lsusb  output for context:

Code: [Select]
bruno@x230:~$ lsusb
...
Bus 002 Device 003: ID 040d:3801 VIA Technologies, Inc.
...

Unfortunately, the script fails here:

Code: [Select]
MODALIS="$(find /sys -name modalias | xargs  grep "v""$VID""p""$PID" 2> /dev/null)"
Because the two relevant files for my usb device are these (notice no "modalias" in the name)...

Code: [Select]
/sys/devices/pci0000:00/0000:00:1a.0/usb2/2-1/2-1.2/uevent
/sys/devices/pci0000:00/0000:00:1a.0/usb2/2-1/2-1.2/idProduct

...and they do not contain anything like v$VIDp$PID:

Code: [Select]
$ sudo cat /sys/devices/pci0000:00/0000:00:1a.0/usb2/2-1/2-1.2/uevent
MAJOR=189
MINOR=130
DEVNAME=bus/usb/002/003
DEVTYPE=usb_device
DRIVER=usb
PRODUCT=40d/3801/108
TYPE=255/255/255
BUSNUM=002
DEVNUM=003

$ sudo cat /sys/devices/pci0000:00/0000:00:1a.0/usb2/2-1/2-1.2/idProduct
3801
Also, notice how VID is 040d but the uevent file contains 40d (missing leading zero). Why, oh why?

Anyway, inspired by your script, I created this variation...

Code: [Select]
#!/bin/sh
productId=3801 # get this from lsusb
productFile=$(find /sys -name 'idProduct' | xargs grep -l "$productId" 2>/dev/null)
device=$(basename $(dirname $productFile)) # e.g., device=2-1.2
sudo sh -c "echo $device >/sys/bus/usb/drivers/usb/unbind 2>/dev/null"
sudo sh -c "echo $device >/sys/bus/usb/drivers/usb/bind"
...and it's working great:

Code: [Select]
$ ifconfig -a wlan1
ifconfig: wlan1: error fetching interface information: Device not found
$ tce-load -i firmware-atheros
firmware-atheros.tcz: OK
$ ./USBReset.sh # instead of manually detaching+reattaching the device
$ ifconfig -a wlan1
wlan1     Link encap:Ethernet  HWaddr 00:12:7B:20:6B:A0 
          BROADCAST MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)
Thanks for pointing me in the right direction with this!
« Last Edit: September 01, 2024, 03:59:01 PM by GNUser »

Offline Rich

  • Administrator
  • Hero Member
  • *****
  • Posts: 11639
Re: how do firmware extensions work without startup script?
« Reply #22 on: September 02, 2024, 12:06:28 AM »
Hi GNUser
Thanks for testing and the feedback.

... (notice no "modalias" in the name)...
I guess not all devices have aliases assign to them.

Quote
... Also, notice how VID is 040d but the uevent file contains 40d (missing leading zero). Why, oh why? ...
Now that you mention it, I recall running into that a couple of years ago.

Quote
... Anyway, inspired by your script, I created this variation...
A couple of notes:
1. idProduct is not unique if not coupled with idVendor.
2. Your search of the /sys dir may not be deep enough, possibly
   resetting the hub instead of the port.

Quote
... and it's working great: ...
Hey, no fair. That's what I was shooting for. ;D

Anyway, I reworked my script:
Code: [Select]
#!/bin/sh

Debug="test"
# Uncomment the next line to turn on debugging messages.
Debug="echo"

# Vendor ID and product ID (8644:8003).
VID="8644"
PID="8003"

# Strips leading zeros from variable. If variable is all
# zeros, variable will be set to a single zero.
while [ ${#VID} -gt 1 ] # ${#VID} returns string length of variable.
do
# If first character of variable is not "0", end loop.
[ ${VID:0:1} != "0" ] && break
# Remove first character from variable.
VID="${VID:1}"
done

# Strips leading zeros from variable. If variable is all
# zeros, variable will be set to a single zero.
while [ ${#PID} -gt 1 ] # ${#PID} returns string length of variable.
do
# If first character of variable is not "0", end loop.
[ ${PID:0:1} != "0" ] && break
# Remove first character from variable.
PID="${PID:1}"
done

# Find the  modalias  file that contains our VID and PID.
UEVENTS="$(find /sys -name uevent | xargs  grep "$VID""/""$PID" 2> /dev/null)"

if [ -n "$UEVENTS" ]
then
# Grab the last entry from $UEVENTS.
for UEVENT in $UEVENTS
do
UEVENT="$UEVENT"
done

"$Debug" "UEVENT=$UEVENT"
# Complete path to uevent file after stripping from :PRODUCT to end of line.
UEVENT="${UEVENT%:PRODUCT*}"
# Strip uevent file name from path.
DEVICE="${UEVENT%/*}"
"$Debug" "DEVICE=$DEVICE"
# Grab the device designation from the end of the path.
DEVICE="${DEVICE##*/}"
"$Debug" "UEVENT=$UEVENT"
"$Debug" "DEVICE=$DEVICE"
fi

if [ -f "$UEVENT" ]
then
# Grab the driver name from uevent file.
DRIVER="$(grep "DRIVER" $UEVENT | cut -d '=' -f2)"
"$Debug" "DRIVER=$DRIVER"
fi

if [ -n "$DRIVER" ]
then
# Find the directory named for the driver.
BINDPATH="$(find /sys -type d -name "$DRIVER")"
# That directory should contain the unbind and bind files.
BINDFILE="$BINDPATH""/bind"
UNBINDFILE="$BINDPATH""/unbind"
"$Debug" "BINDPATH=$BINDPATH"
"$Debug" "BINDFILE=$BINDFILE"
"$Debug" "UNBINDFILE=$UNBINDFILE"
fi

if [ -e "$UNBINDFILE" ]
then
# Remove device.
echo "$DEVICE" | sudo tee "$UNBINDFILE" 1> /dev/null
fi

"$Debug" "Sleep ..."
sleep 3
"$Debug" "... Wake up"


if [ -e "$BINDFILE" ]
then
# Add device back.
echo "$DEVICE" | sudo tee "$BINDFILE" 1> /dev/null
fi

I'd appreciate if you would take this version for a spin to
see if it works any better. I've attached a copy to this post.

Quote
... Thanks for pointing me in the right direction with this!
Thanks for providing an interesting puzzle. :)

Offline GNUser

  • Wiki Author
  • Hero Member
  • *****
  • Posts: 1512
Re: how do firmware extensions work without startup script?
« Reply #23 on: September 02, 2024, 08:17:28 AM »
Thanks for providing an interesting puzzle. :)
Only a hacker would say such a thing :)

Your revised script gets farther but fails to set DRIVER, without which it cannot figure out the crucial UNBINDFILE and BINDFILE:

Code: [Select]
$ ifconfig -a wlan1
ifconfig: wlan1: error fetching interface information: Device not found
$ tce-load -i firmware-atheros
firmware-atheros.tcz: OK
$ ./USBreset.sh
UEVENT=/sys/devices/pci0000:00/0000:00:1a.0/usb1/1-1/1-1.2/uevent:PRODUCT=40d/3801/108
DEVICE=/sys/devices/pci0000:00/0000:00:1a.0/usb1/1-1/1-1.2
UEVENT=/sys/devices/pci0000:00/0000:00:1a.0/usb1/1-1/1-1.2/uevent
DEVICE=1-1.2
DRIVER=
Sleep ...
... Wake up
$ ifconfig -a wlan1
ifconfig: wlan1: error fetching interface information: Device not found

Offline GNUser

  • Wiki Author
  • Hero Member
  • *****
  • Posts: 1512
Re: how do firmware extensions work without startup script?
« Reply #24 on: September 02, 2024, 01:00:53 PM »
Hi Rich. Why not unbind+bind using the generic  /sys/bus/usb/drivers/usb/unbind  and   /sys/bus/usb/drivers/usb/bind?

That simplifies the script because once we have DEVICE we're basically done. Here is what the script could look like with this logic plus passing VID and PID as arguments:

Code: [Select]
#!/bin/sh

# Syntax: $ USBReset.sh <vendorId> <productId>
# Usage example: $ USBReset.sh

Debug="test"
# Uncomment the next line to turn on debugging messages.
Debug="echo"

# Vendor ID and product ID (e.g., 8644 8003).
VID="$1"
PID="$2"

# Strips leading zeros from variable. If variable is all
# zeros, variable will be set to a single zero.
while [ ${#VID} -gt 1 ] # ${#VID} returns string length of variable.
do
# If first character of variable is not "0", end loop.
[ ${VID:0:1} != "0" ] && break
# Remove first character from variable.
VID="${VID:1}"
done

# Strips leading zeros from variable. If variable is all
# zeros, variable will be set to a single zero.
while [ ${#PID} -gt 1 ] # ${#PID} returns string length of variable.
do
# If first character of variable is not "0", end loop.
[ ${PID:0:1} != "0" ] && break
# Remove first character from variable.
PID="${PID:1}"
done

# Find the  modalias  file that contains our VID and PID.
UEVENTS="$(find /sys -name uevent | xargs  grep "$VID""/""$PID" 2> /dev/null)"

if [ -n "$UEVENTS" ]
then
# Grab the last entry from $UEVENTS.
for UEVENT in $UEVENTS
do
UEVENT="$UEVENT"
done

"$Debug" "UEVENT=$UEVENT"
# Complete path to uevent file after stripping from :PRODUCT to end of line.
UEVENT="${UEVENT%:PRODUCT*}"
# Strip uevent file name from path.
DEVICE="${UEVENT%/*}"
"$Debug" "DEVICE=$DEVICE"
# Grab the device designation from the end of the path.
DEVICE="${DEVICE##*/}"
"$Debug" "UEVENT=$UEVENT"
"$Debug" "DEVICE=$DEVICE"
fi

# Remove device.
echo "$DEVICE" | sudo tee /sys/bus/usb/drivers/usb/unbind >/dev/null 2>&1

"$Debug" "Sleep ..."
sleep 3
"$Debug" "... Wake up"

# Add device back.
echo "$DEVICE" | sudo tee /sys/bus/usb/drivers/usb/bind >/dev/null

It works well:
Code: [Select]
$ ifconfig -a wlan1
ifconfig: wlan1: error fetching interface information: Device not found
$ tce-load -i firmware-atheros
firmware-atheros.tcz: OK
$ ./USBreset.sh 040d 3801
UEVENT=/sys/devices/pci0000:00/0000:00:1a.0/usb2/2-1/2-1.2/uevent:PRODUCT=40d/3801/108
DEVICE=/sys/devices/pci0000:00/0000:00:1a.0/usb2/2-1/2-1.2
UEVENT=/sys/devices/pci0000:00/0000:00:1a.0/usb2/2-1/2-1.2/uevent
DEVICE=2-1.2
Sleep ...
... Wake up
$ ifconfig -a wlan1
wlan1     Link encap:Ethernet  HWaddr 00:12:7B:20:6B:A0 
          BROADCAST MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)

Offline Rich

  • Administrator
  • Hero Member
  • *****
  • Posts: 11639
Re: how do firmware extensions work without startup script?
« Reply #25 on: September 02, 2024, 10:24:54 PM »
Hi GNUser
Hi Rich. Why not unbind+bind using the generic  /sys/bus/usb/drivers/usb/unbind  and   /sys/bus/usb/drivers/usb/bind? ...
There are a bunch of directories under  /sys/bus/usb/drivers/usb/.
I don't want to guess and reset something too high in the chain of
devices, like a hub, and resetting everything connected to it.

I changed my tactics and rewrote the script:
1. I'm using udevadm to find device path(s) for VID and PID.
2. Then use udevadm to find the driver name.
3. Find the driver name in /sys/.
      That provides  BINDPATH (plus BINDFILE and UNBINDFILE).
4. Sort the links in  BINDPATH  from longest to shortest.
5. Check the links to see which one points back to our device.
6. Strip the leading path information from the link (USBPORT).
7. Use  USBPORT , BINDFILE , and  UNBINDFILE  to reset the port.

Code: [Select]
#!/bin/sh

Debug="test"
# Uncomment the next line to turn on debugging messages.
Debug="echo"

# Vendor ID and product ID (8644:8003).
VID="8644"
PID="8003"

# Notes
# This is the driver property we are looking for.
# ID_USB_DRIVER=usb-storage
#
# This won't work because it searches for "ID_VENDOR_ID=0644 OR ID_MODEL_ID=0200"
# udevadm trigger -v -n -t devices -p ID_VENDOR_ID=0644 -p ID_MODEL_ID=0200
#
# Returns properties for the device listed in the path.
# udevadm info -q property -p /sys/devices/pci0000:00/0000:00:1d.7/usb1/1-7

udevadm trigger -v -n -t devices -p ID_VENDOR_ID="$VID" | sort > VIDs.txt
udevadm trigger -v -n -t devices -p ID_MODEL_ID="$PID" | sort > PIDs.txt

# Since we had to search for VID and PID separately, we check which
# results are common to both lists for an exact match.
DEVICES="$(comm -12 VIDs.txt PIDs.txt)"

[ -z "$DEVICES" ] && echo "No DEVICES found" && exit 1

for DEVICE in $DEVICES
do
ID_USB_DRIVER="$(udevadm info -q property -p "$DEVICE" | grep "ID_USB_DRIVER=" | cut -d = -f2)"
"$Debug" "DEVICE=$DEVICE    ID_USB_DRIVER=$ID_USB_DRIVER"
[ -n "$ID_USB_DRIVER" ] && break
done

[ -z "$ID_USB_DRIVER" ] && echo "No ID_USB_DRIVER found" && exit 1

BINDPATH="$(find /sys/bus/usb/drivers/ -type d -name "$ID_USB_DRIVER")"
[ -z "$BINDPATH" ] && echo "No BINDPATH found" && exit 1
# That directory should contain the unbind and bind files.
BINDFILE="$BINDPATH""/bind"
UNBINDFILE="$BINDPATH""/unbind"
"$Debug" "BINDPATH=$BINDPATH"
"$Debug" "BINDFILE=$BINDFILE"
"$Debug" "UNBINDFILE=$UNBINDFILE"

# Needed for embedding a newline character in "for ITEM in ..." loop.
# Without it, sort and cut won't be applied to all ITEMs in LIST.
NL="
"

# BINDPATH directory contains links to devices using that driver.
# ${#ITEM} returns string length for sorting later on.
# "/sys/${ITEM##*../}" converts ../../../../devices/... to /sys/devices/...
for ITEM in $(readlink "$BINDPATH"/*)
do
LIST=$LIST$(printf "%04d %s\n" ${#ITEM} "/sys/${ITEM##*../}")$NL
done

"$Debug" "LIST=$LIST"

# We want to sort longest to shortest and remove the string lengths.
CHOICES="$(echo "$LIST" | sort -r | cut -c 6-)"

"$Debug" "CHOICES=$CHOICES"

# Yhis tries to find and remove needle (CHOICE) from haystack (DEVICE).
# If it succeeds, we found the link to our device.
for CHOICE in $CHOICES
do
if [ "$DEVICE" != "${DEVICE/$CHOICE/}" ]
then
# Remove leading path information.
USBPORT="${CHOICE##*/}"
"$Debug" "USBPORT=$USBPORT"
break
fi
done


if [ -e "$UNBINDFILE" ]
then
# Remove device.
echo "$USBPORT" | sudo tee "$UNBINDFILE" 1> /dev/null
fi

"$Debug" "Sleep ..."
sleep 3
"$Debug" "... Wake up"


if [ -e "$BINDFILE" ]
then
# Add device back.
echo "$USBPORT" | sudo tee "$BINDFILE" 1> /dev/null
fi

# Clean up.
rm -rf VIDs.txt
rm -rf PIDs.txt

Hopefully, third times the charm.
« Last Edit: September 04, 2024, 02:02:47 PM by Rich »

Offline GNUser

  • Wiki Author
  • Hero Member
  • *****
  • Posts: 1512
Re: how do firmware extensions work without startup script?
« Reply #26 on: September 03, 2024, 08:25:13 AM »
Hi Rich. I tested your rewrite of USBReset.sh:
Code: [Select]
$ ./USBreset.sh
DEVICE=/sys/devices/pci0000:00/0000:00:1a.0/usb2/2-1/2-1.2    ID_USB_DRIVER=
No ID_USB_DRIVER found
...it would be nice to simulate usb device disconnect+reconnect at software level, it seems non-trivial...
I was on to something ;)

Offline mocore

  • Hero Member
  • *****
  • Posts: 645
  • ~.~
Re: how do firmware extensions work without startup script?
« Reply #27 on: September 03, 2024, 08:55:17 AM »
While it would be nice to simulate usb device disconnect+reconnect at software level, it seems non-trivial and more trouble than it's worth.

 a bit of a tangent , idk how the lines are drawn (tbh) between "software level" and hw , or if you are all ready aware of this option 

any way this *  https://github.com/mvp/uhubctl -  uhubctl - USB hub per-port power control
 seams relevant (as a workaround with hw power off ctrl-ed via sw)

*(all though iv not had much luck finding hw(versions :() supporting this feature )
... perhaps if your lucky ur usb chip supports the feature !?!

or the issues provide some insight into relevant parts of linux usb system structure?! (longshoot)
« Last Edit: September 03, 2024, 08:57:19 AM by mocore »

Offline GNUser

  • Wiki Author
  • Hero Member
  • *****
  • Posts: 1512
Re: how do firmware extensions work without startup script?
« Reply #28 on: September 03, 2024, 10:40:44 AM »
a bit of a tangent , idk how the lines are drawn (tbh) between "software level" and hw
By "software level" I mean anything other than manually pulling out the usb device and pushing it back into the usb port.

Offline Rich

  • Administrator
  • Hero Member
  • *****
  • Posts: 11639
Re: how do firmware extensions work without startup script?
« Reply #29 on: September 03, 2024, 11:01:38 AM »
Hi GNUser
Could you please post the result of this command:
Code: [Select]
udevadm info -q property -p /sys/devices/pci0000:00/0000:00:1a.0/usb2/2-1/2-1.2