pfSense on Sophos XG125w - "no carrier" on all eth interfaces
-
Greetings,
I got my hands on a Sophos XG125w (rev. 3) appliance over the weekend, and managed to install pfSense 2.6.0 on it successfully.
However, connecting to it via the ethernet ports wasn't so much fun (yet)...
Problem
All eth ports are showing status as "no carrier". Hence cannot connect to the appliance via the eth ports.Context
- Appliance Hardware: Sophos XG125w (rev. 3)
- pfSense image: pfSense-CE-memstick-2.6.0-RELEASE-amd64.img.gz
- Client hardware/OS: Macbook Pro/MacOS
- Ethernet LED status on appliance (when eth cable is connected): Both lights ON (Solid Amber and Green, refer to photo below)
- DHCP never worked (as expected...), and I have manually assigned the IP address on client.
Attempts for Resolution
I have tried the following:- Use different eth cables (Cat 5E and Cat 6)
- Use different laptops/clients to connect
- Tried various negotiation settings (Auto-selection, manual base1000T Full-duplex, base100TX Full-duplex, etc) on both client and appliance ports
- Tried configuring different eth ports for LAN on the appliance (igb0, igb1, ix0, ix1, etc)
- Tried flashing different versions of pfSense image (2.6.0, 2.52, 2.51, 2.45)
- Flash SFOS (Sophos OS), and confirmed that NICs and all ports are functional.
However, nothing worked; so reaching out for some help here. Any ideas/thoughts are much appreciated.
Some photos for reference.
-
Default LAN WAN interface
-
LAN WAN ifconfig
-
List of eth ports/interfaces
-
Appliance eth port LED status (pls don't get confused by the label; cable is connected to the LAN port)
Thanks in advance.
-
D dkzsys referenced this topic on
-
D dkzsys referenced this topic on
-
D dkzsys referenced this topic on
-
All 9 NICs always show as no carrier in ifconfig? Even though they are linked?
-
@stephenw10 said in pfSense on Sophos XG125w - "no carrier" on all eth interfaces:
All 9 NICs always show as no carrier in ifconfig? Even though they are linked?
I've tried all 8 eth ports individually, and yes, all "no carrier" in ifconfig. (9th (igb4) is an SFP connection.)
I've also tried using "auto-detection" in Assign Interfaces. Always "No link-up detected."
-
Hmm, maybe they are using a modified driver. My first thought was that they are using a switch in there but they wouldn't have 9 NICs if so. What CPU is that? One with the 4 ix NICs onboard?
-
@stephenw10 said in pfSense on Sophos XG125w - "no carrier" on all eth interfaces:
Hmm, maybe they are using a modified driver. My first thought was that they are using a switch in there but they wouldn't have 9 NICs if so. What CPU is that? One with the 4 ix NICs onboard?
The CPU onboard is then Intel Atom C3508 (@ 1.6GHz).
One (pretty weird) thing worth mentioning - When I first tried to set my client/Mac eth interface to manual base1000X full-duplex, it did connect to the appliance LAN port (on pfSense v2.5.2), ONLY ONCE (I could even open the web GUI interface)... Then it just failed consistently, no matter what I did, on all pfSense image versions I flashed (including v2.5.2).
-
Have you tried putting a switch in between?
-
@stephenw10 said in pfSense on Sophos XG125w - "no carrier" on all eth interfaces:
Have you tried putting a switch in between?
Let me see if I can find an old switch, or an openWRT router laying somewhere.
-
Hard to imagine it would make any difference given that the ports show link LEDs but worth trying.
-
FWIW I have the XG115 Rev 3 and by default the Wan interface was igb0, I had to change the Wan interface to ibg2 to get it to work..
-
@stephenw10 said in pfSense on Sophos XG125w - "no carrier" on all eth interfaces:
Hard to imagine it would make any difference given that the ports show link LEDs but worth trying.
Got my old Airport Extreme out and set it up as a bridge/switch. Still no luck.
-
@s762 said in pfSense on Sophos XG125w - "no carrier" on all eth interfaces:
FWIW I have the XG115 Rev 3 and by default the Wan interface was igb0, I had to change the Wan interface to ibg2 to get it to work..
Yep - tried played around with different ports (including LAN and WAN), unfortunately not much luck here.
-
Mmm, that seems like it must be some custom PHY if the LEDs show link but the driver is not passed that info. Or at least something not supported in the FreeBSD driver.
Very odd that it would apply to both igb and ix though...
-
J jimp moved this topic from Problems Installing or Upgrading pfSense Software on
-
An update
I have tried a couple of ways to install drivers, but didn't have much luck. So I installed pfSense CE v2.7, hoping that the FreeBSD 14 may have newer drivers and might fix the issue. Unfortunately no luck there either.I can see drivers are missing for these hardware (excluding the wireless adaptor)...
- Atom Processor C3000 Series SMBus controller
- Atom Processor C3000 Series SMBus controller - Host
- Atom Processor C3000 Series SPI Controller
- Atom Processor C3000 Series QuickAssist Technology
- Atom Processor C3000 Series Root Complex Event Controller
- Aton Processor C3000 Series ME HECI 1
- Atom Processor C3000 Series Power Management Controller
Screenshots (pciconf) for reference:
-
The only time I've seen behaviour like that is when the NICs are arranged in bypass pairs. I don't see the ports labeled like that but you should check if there are bypass relays on the board.
-
S stephenw10 referenced this topic on
-
S stephenw10 referenced this topic on
-
I have the same problem and wondering if you managed to find a fix for this. It's so disappointing to see when you have a hardware where you shouldn't have problem with the drivers since the NICs have been around for ages and work great on regular headless PCs, but has problem with Sophos boxes.
-
@distengr Yes - I figured out what happened... and it was another silly human error... (Nevertheless, learnt a few things along the way, so consider it a relatively low return for the few nights play-around)
TL;DR:
- The issue was a false assumption on eth port allocation, and the failing auto-detection "Assign Interfaces".
- The resolution is to a) set static IP address on client/PC interface and b) connect to the correct eth interface.
The long version of the story
As per picture above, since 1) the auto-detection in "Assign Interfaces" didn't work, 2) the auto assigned WAN and LAN in pfSense are on igb0 and igb1, and 3) on the Sophos appliance, the WAN and LAN are labelled are the top NIC card, so I made a killer assumption that igb0 and igb1 are port 1 and port 2 of the top NIC card respectively.
It turns out that igb0 and igb1 actually correspond to port 1 and port 2 of the lower NIC card.
There is one more prerequisite to bring up the interface (to Active) - that is to set a static IP on the client side interface (i.e. your PC/Mac, etc).
Now I have a fully functional pfSense on the Sophos appliance (installed CE 2.6 and upgraded to pfSense Plus 23.01); and I am continuing my nights of effort playing around with pfSense features:)
I'd like to thank @stephenw10 for chipping in on the ideas! Very much appreciated!
BTW, @stephenw10, I've been messing around with squid transparent proxy and SSL bump (SSL/TLS filtering); as well as pfBlocker-NG DNSBL. Spent a few nights on them but couldn't find an elegant solution for implementation. Any pointers of reference or guide please?
Specific challenges below:
Squid Transparent Proxy and SSL Bump (SSL/TLS filtering)
With the default squid v3.5 branch from the pfSense package manager, I have challenges to- ACL destination domain (ssl_bump none), in order to workaround websites/apps that have implemented server cert pinning (I understand that if the DNS resolution for the domain resolves to multiple IPs, ACL dstdomain doesn't work/support it. Is this true?)
- wss WebSocket doesn't work. (I'll need http_upgrade_request_protocols from Squid v5??)
As a workaround for the 2 challenges above, I had to use ACL dst IP address... Is this really my only option?
pfBlocker-NG DNSBL
I am trying use DNSBL to block/not-resolve subdomains using regex (in DNSBL list), and it doesn't seem to work. Am I over-complicating it?
Is my other option to use "Wildcard Blocking (TLD)"? Any other options?
I found that information are pretty scattered; and most posts on these topic seem to be outdated. Would be great to have some advise from you, @stephenw10. Thanks in advance!
-
Personally I use DNS-BL with TLD widlcards enabled and it works well.
I would advise you don't use Squid at all unless you are forced to. And generally that means in a deployment where you must log all sites visited for example.
-
@stephenw10 said in pfSense on Sophos XG125w - "no carrier" on all eth interfaces:
Personally I use DNS-BL with TLD widlcards enabled and it works well.
An implementation question, let's say if you want to block all the subdomains for roblox.com, in your list do you define it as ".roblox.com" as a line entry?
I would advise you don't use Squid at all unless you are forced to. And generally that means in a deployment where you must log all sites visited for example.
I'm using it mainly to track my children's internet usage (till they figure out ways to workaround it..), so yes, I need all visited sites logged. I have a solution to segregate other devices, based on DHCP subnet, so other devices in the network are exempted from ssl bumping. Any thoughts/ideas on overcoming the 2 challenges?
-
I have a custom list with TLDs in I want to block that gets parsed with the fetched adblock lists. It just contains the TLDs.
How are you using the ACLs in Squid?
I've never tried to use anything special for websockets. Not a problem I've seen or tried to work past before.
Steve
-
@stephenw10 Thanks for sharing your implementation of custom list under DNSBL Groups. Nice and easy way to turn them on and off.
I am using pfSsense DNS Resolver (via DHCP), and DoT to external/public DNS service providers (Cloudflare and Google). Clients (browsers, and MacOS) seem to go directly to the public DNS service providers, via port 443 (DoH) or port 853 (DoT). I am testing with some firewall rules to block/reject them, to ensure all DNS traffic goes through pfSense DNS Resolver. Still not 100% clear how effective this implementation is.
WRT ACL, my implementation is relatively straightforward.
- acl bypass_ssl src IP address (DHCP subnet and specific hosts)
- acl bypass_ssl_dst dst IP address (to domains that use WebSockets or enforce server cert pinning)
- I've read through the squid config reference; and am about to try "acl broken_sites ssl::server_name .example.com" when I get home tonight.