Intel 10GBASE-T X520/X540 driver support?

Tiny Core Extensions > TCE Corepure64

(1/2) > >>

CentralWare:
I have a pair of SuperMicro network cards (10-gig 2-port) I'm trying to pair between two linux machines, the TCL box and a Free/TrueNAS box.
TrueNAS sees the cards without effort; TCL I'm uncertain whether we have the same driver support?
Intel's Link and Read Me for ixg* Base Driver (which I have not ventured toward yet! :) )
SourceForge Link for Intel e1000 which I believe includes X5xx

--- Code: ---lspci
...
01:00.0 Ethernet controller: Intel Corporation Ethernet Controller 10-Gigabit X540-AT2 (rev 01)
01:00.1 Ethernet controller: Intel Corporation Ethernet Controller 10-Gigabit X540-AT2 (rev 01)
...

--- End code ---
I've tried everything I could think of (including i2c-KERNEL) for everything firmware-intel we do support, any ideas of what I might have overlooked?

Rich:
Hi CentralWare
Does lsmod show that the ixgbe driver is loaded?

I believe your card is 8086:1528:

--- Code: ---tc@E310:~$ modinfo ixgbe
filename: /lib/modules/4.19.10-tinycore/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko.gz
author: Intel Corporation, <linux.nics@intel.com>
description: Intel(R) 10 Gigabit PCI Express Network Driver
license: GPL
parm: debug:Debug level (0=none,...,16=all)
parm: allow_unsupported_sfp:Allow unsupported and untested SFP+ modules on 82599-based adapters
parm: max_vfs:Maximum number of virtual functions to allocate per physical function - default is zero and maximum value is 63. (Deprecated)
version: 5.1.0-k
----- Snip -----
alias: pci:v00008086d00001528sv*sd*bc*sc*i*
----- Snip -----
srcversion: 47E398AEA90A36B7EE58800
depends: mdio,ptp
intree: Y
vermagic: 4.19.10-tinycore SMP mod_unload 486
tc@E310:~$
--- End code ---

You could also check dmesg:

--- Code: ---dmesg | grep ixgbe
--- End code ---

If the driver was not loaded, try modprobing it first, maybe with
the drivers debug param.

It might also be useful to look at dmesg for your Free/TrueNAS box.
It might reveal a bootcode, firmware, or driver parameter being used.

CentralWare:
@Rich,

Thanks! This should be a fun mystery to figure out!

--- Quote ---Does lsmod show that the ixgbe driver is loaded?
--- End quote ---
Yes, it is/was in fact doing exactly what it was supposed to be doing.
It's loaded, modprobe was unnecessary as dmesg showed me unplugging and reconnecting a cable... so it's there... and it says ETH3 was unplugged.

Here's the entire output from lspci:

--- Code: ---00:00.0 Host bridge: Intel Corporation 4 Series Chipset DRAM Controller (rev 03)
00:01.0 PCI bridge: Intel Corporation 4 Series Chipset PCI Express Root Port (rev 03)
00:06.0 PCI bridge: Intel Corporation 4 Series Chipset PCI Express Root Port (rev 03)
00:1a.0 USB controller: Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #4
00:1a.1 USB controller: Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #5
00:1a.2 USB controller: Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #6
00:1a.7 USB controller: Intel Corporation 82801JI (ICH10 Family) USB2 EHCI Controller #2
00:1c.0 PCI bridge: Intel Corporation 82801JI (ICH10 Family) PCI Express Root Port 1
00:1c.3 PCI bridge: Intel Corporation 82801JI (ICH10 Family) PCI Express Root Port 4
00:1c.4 PCI bridge: Intel Corporation 82801JI (ICH10 Family) PCI Express Root Port 5
00:1c.5 PCI bridge: Intel Corporation 82801JI (ICH10 Family) PCI Express Root Port 6
00:1d.0 USB controller: Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #1
00:1d.1 USB controller: Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #2
00:1d.2 USB controller: Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #3
00:1d.7 USB controller: Intel Corporation 82801JI (ICH10 Family) USB2 EHCI Controller #1
00:1e.0 PCI bridge: Intel Corporation 82801 PCI Bridge (rev 90)
00:1f.0 ISA bridge: Intel Corporation 82801JIR (ICH10R) LPC Interface Controller
00:1f.2 IDE interface: Intel Corporation 82801JI (ICH10 Family) 4 port SATA IDE Controller #1
00:1f.3 SMBus: Intel Corporation 82801JI (ICH10 Family) SMBus Controller
00:1f.5 IDE interface: Intel Corporation 82801JI (ICH10 Family) 2 port SATA IDE Controller #2
01:00.0 Ethernet controller: Intel Corporation Ethernet Controller 10-Gigabit X540-AT2 (rev 01)
01:00.1 Ethernet controller: Intel Corporation Ethernet Controller 10-Gigabit X540-AT2 (rev 01)
02:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Caicos PRO [Radeon HD 7450]
02:00.1 Audio device: Advanced Micro Devices, Inc. [AMD/ATI] Caicos HDMI Audio [Radeon HD 6450 / 7450/8450/8490 OEM / R5 230/235/235X OEM]
03:00.0 USB controller: VIA Technologies, Inc. VL805 USB 3.0 Host Controller (rev 01)
04:00.0 SATA controller: JMicron Technology Corp. JMB363 SATA/IDE Controller (rev 02)
04:00.1 IDE interface: JMicron Technology Corp. JMB363 SATA/IDE Controller (rev 02)
05:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller (rev 02)
06:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller (rev 02)
07:00.0 RAID bus controller: Silicon Image, Inc. SiI 3114 [SATALink/SATARaid] Serial ATA Controller (rev 02)

--- End code ---

Here's where the fun starts...
1. As shown above, INTEL dual port is seen (twice, of course)
2. 05 and 06 are showing Realtek RTL 8xxx. The motherboard has two onboard... PCI32 #2 is also a two channel Realtek 8xxx.

ETH0/1 are onboard NICs bonded to 10.0.0.124/22
ETH2/3 are PCI32 NIC ports bonded to 10.0.4.1/22
Up to this point, everything worked exactly as expected... then:
INTEL card was installed, anticipating an ETH4 and 5. [Scratches Head, Instead]

If I pull either cable from ETH0/1 there's NO system message. If I pull BOTH cables, I'm kicked from SSH (expected) but still no system message.
If I do the same with ETH2/3 there are system messages; connectivity yet to be tested as it's a different network than this workstation has immediate access to.
If I pull either cable from INTEL there are system messages using the identifiers (ETH2/3) which the PCI32 card had been initially assigned.

I'm guessing there's an identifier overlap going on, but it doesn't explain why lspci doesn't list BOTH Realtek networks - which may be the cause of the overlap.
If memory serves, there's an app "ifrename" or similar which given a MAC address allows the identifier to be rewritten to MickeyMouse0 instead of ETH0. If I pull one of the cards and get MACs from the cards accordingly, that should band-aid the problem, but I'm intrigued what the cause might be!

AMMENDMENT:
As suggested, the TrueNAS box was looked into (yet to look into boot codes) but half-way down (6 seconds into boot) it does what I mentioned above...
renames ETHx to ENP2S0F0/F1. It also shows me pulling the cable a few times for testing the TCL box.

--- Code: ---root@TrueNAS1[~]# dmesg | grep ixg
[ 1.810957] ixgbe: Intel(R) 10 Gigabit PCI Express Network Driver
[ 1.811086] ixgbe: Copyright (c) 1999-2016 Intel Corporation.
[ 2.091115] ixgbe 0000:02:00.0: Multiqueue Enabled: Rx Queue count = 8, Tx Queue count = 8 XDP Queue count = 0
[ 2.175922] ixgbe 0000:02:00.0: 32.000 Gb/s available PCIe bandwidth (5.0 GT/s PCIe x8 link)
[ 2.200365] ixgbe 0000:02:00.0: MAC: 3, PHY: 0, PBA No: 030000-000
[ 2.200491] ixgbe 0000:02:00.0: 0c:c4:7a:58:e2:22
[ 2.348132] ixgbe 0000:02:00.0: Intel(R) 10 Gigabit Network Connection
[ 2.631088] ixgbe 0000:02:00.1: Multiqueue Enabled: Rx Queue count = 8, Tx Queue count = 8 XDP Queue count = 0
[ 2.717675] ixgbe 0000:02:00.1: 32.000 Gb/s available PCIe bandwidth (5.0 GT/s PCIe x8 link)
[ 2.742522] ixgbe 0000:02:00.1: MAC: 3, PHY: 0, PBA No: 030000-000
[ 2.742654] ixgbe 0000:02:00.1: 0c:c4:7a:58:e2:23
[ 2.894423] ixgbe 0000:02:00.1: Intel(R) 10 Gigabit Network Connection
[ 6.083030] ixgbe 0000:02:00.0 enp2s0f0: renamed from eth2
[ 6.115132] ixgbe 0000:02:00.1 enp2s0f1: renamed from eth4
[ 46.175464] ixgbe 0000:02:00.1: registered PHC device on enp2s0f1
[ 46.488261] ixgbe 0000:02:00.0: registered PHC device on enp2s0f0
[ 50.814429] ixgbe 0000:02:00.1 enp2s0f1: NIC Link is Up 10 Gbps, Flow Control: RX/TX
[ 239.779626] ixgbe 0000:02:00.1 enp2s0f1: NIC Link is Down
[ 282.429763] ixgbe 0000:02:00.1 enp2s0f1: NIC Link is Up 10 Gbps, Flow Control: None
[ 403.601818] ixgbe 0000:02:00.1 enp2s0f1: NIC Link is Down
[ 408.378783] ixgbe 0000:02:00.1 enp2s0f1: NIC Link is Up 10 Gbps, Flow Control: None
[ 467.311084] ixgbe 0000:02:00.1 enp2s0f1: NIC Link is Down
[ 471.657811] ixgbe 0000:02:00.1 enp2s0f1: NIC Link is Up 10 Gbps, Flow Control: RX/TX

--- End code ---

AMMENDMENT:

WHAT? IfRename isn't a part of ETHtools? It's WIFI???
tce-ab... find ifrename... install wireless... copy ifrename binary to /opt... uninstall wireless/wireless-KERNEL... all this for a 23kB file :)

Rich:
Hi CentralWare
Maybe your Free/TrueNAS box is using udev to rename
the network device:
https://www.baeldung.com/linux/rename-network-interface#permanent-renaming-network-interface-using-udev-rules

CentralWare:
Quite possible, especially considering how early in the boot process they get named. I'll add it to my research list!

UPDATE:
After playing musical "cards" I was able to gather the six MACs needed to make this recipe.
I changed out both of the dual cards with identical models JUST to make sure we weren't dealing with a hardware oddity -- same results.
During the swaps, MACs were recorded while they were "reliably" available.

ifrenme -p probes drivers; this is a "just in case."
-t (takeover) is likely more for when this patch gets added to our tc-config

We then create our config file /etc/iftab where we "name" our "eth#" interfaces with our MickeyMouse# replacements:

--- Code: ---mickey0 mac AA:BB:CC:DD:EE:F1
mickey1 mac AA:BB:CC:DD:EE:F2
mickey2 mac AA:BB:CC:DD:EE:F3
mickey3 mac AA:BB:CC:DD:EE:F4
mickey4 mac AA:BB:CC:DD:EE:F5
mickey5 mac AA:BB:CC:DD:EE:F6

--- End code ---

Afterward, in bootsync we launch ifrename (with no parameters) which does the heavy lifting making our MickeyMouse names a reality.
Finally, in bootlocal we can revert Mickey## back into ETH## by adding:
ifrename -i mickey0 -n eth0
(One line for each interface.)

For sake of covering all bases, it's probably best to ifconfig eth# down after probing, but before reading the config as I remember reading somewhere that rename will choke on an active device.

Now for the WHY...
My "best guess" was decided when I rebooted the server and took a photo of the IRQ assignments BIOS had designated.
The onboard NICs were assigned to 11 and 15.
The INTEL card... ALSO assigned to 15 and 11 (just in reverse order.)
The PCI32 RTL8211 card was assigned 3 and 10.
lspci must have been seeing the PCI32 card and the INTEL card was likely the last device read on those IRQs possibly disregarding the Realtek that was already on the same IRQs. Hooks assigned for messages from each were also likely given to the "last mac standing." How ETH2/ETH3 were "re-purposed" onto the INTEL leads to speculation... it's almost midnight, so I'll leave that for another day!

UPDATE:
MickeyMouse# above worked nicely.
Will look into adding it to uDev tomorrow once I read up on TC's take/rules/etc. on uDev.
Quite Redundant ( eth0 --> mickey0 --> eth0 ) but it gets us past the glitch!

--- Code: ---tc@TinyNAS1:~$ sudo ethtool eth5
Settings for eth5:
Supported ports: [ TP ]
Supported link modes: 100baseT/Full
1000baseT/Full
10000baseT/Full

--- End code ---

Navigation

[0] Message Index

[#] Next page

Go to full version