I have a Mellanox ConnectX-2 on my TrueNAS machine, connected to a mikrotik switch, and it works, mostly, fine.
I have the exactly same nic on my desktop, connected to the exact same switch, using the exact same AOC, which works completely fine.
I decided to try a pair of FS SFP+ optical transceivers on my TrueNAS machine to see if it fixes those issue (the link not coming up after a power failure unless I unplug and replug the SFP+ connector from the NIC, this does not happen on my desktop).
But for some reason it refuses to work properly.
I can ping it, access it through SSH, and list the contents of the network shares (incredibly slowly), but the web interface won't load, it gets stuck on the blinking/fading TrueNAS logo.
I tried the transceivers on my desktop and they worked just fine, no issues at all.
Rebooting the TrueNAS machine does nothing, the issue persists, only using the previous AOC "fixes" the issue.
I had similar issues previously, I don't remember the details, if it was swapping between a DAC and an AOC, it was quite some time ago.
I tested the optical transceiver on the second port of the NIC, configuring a different IP from a different network, added an address from that network to my desktop, and everything worked just fine, was able to saturate the link, I checked the traffic both on the web interface, looking at the stats on the port, and also on the switch, where I connected the second port, to make sure the traffic was going through the correct interface.
I'm on TrueNAS 25.04.2.4, the motherboard is an Intel Z270.
What could be going on here? It clearly has connectivity, but for some reason it's limited, looking at the stats of the port on the switch I can see it successfully negotiated at 10G, at the proper speeds, but connectivity is seriously impaired.