Steelhead CX770 Port issues
-
Yeah I mean this really seems like it's the bypass NICs at work to me. Except that you confirmed pfSense sees igb5 get and lose link when you connect to it?
If so then I would run a packet capture on igb5 to see if anything is actually coming in on it.
The fact you can connect to the gui but only the gui on igb1 pretty much has to be a firewall rule issue.
-
@stephenw10, since I'll be switching interfaces, I'll also do a full reset of the firewall so I can reenter all rules. I'll also run Wireshark on the new firewall to see how things flow (or not!) through the ports. It really is a shame that these ports are not working as expected because the hardware is rock solid for a pfSense install. At least I can feel good that I purchased these two CX770s brand new as a bundle for only $100 on eBay. They came from a government surplus site in their original box and all the accessories.
Martin M. Mune
US Army Combat Veteran
Operation Iraqi FreedomVolunteer Soldier
International Legion for the Defense of UkraineСлава Україні!
Героям Слава! -
Mmm, all the by-pass ports I've ever encountered disconnect the internal interfaces when they are in by-pass mode. But I could imagine (maybe) a mode where it loops back the data lines while the MII connections from the PHY still report the link status. But for that to happen you would need both NICs in the by-pass pair to be linked.
-
@stephenw10, just finished switching all VLANs to the igb0 port. Guess what? ! It all works without any issues. Something is very wrong with the ports. It's not just the bypass ports; the AUX (igb5) which is controlled separately doesn't work. At this point I'm out of ideas. I also ran Wireshark and all traffic stops at the igb5 port. ICMP and HTTP traffic successfully reach the IP address on igb5 but nothing else goes through. I rechecked the BIOS settings to make sure the igb0-3 are not in bypass mode.
Anyone who has gotten a CX770 to work successfully, I would appreciate any assistance you can give. This is truly bizarre!
Martin M. Mune
US Army Combat Veteran
Operation Iraqi FreedomVolunteer Soldier
International Legion for the Defense of UkraineСлава Україні!
Героям Слава! -
@martimun said in Steelhead CX770 Port issues:
I also ran Wireshark and all traffic stops at the igb5 port. ICMP and HTTP traffic successfully reach the IP address on igb5 but nothing else goes through
When testing from where?
-
@stephenw10, everything was working and when I returned to the laptop directly connected to the CX770, nothing worked anymore. When I looked at the NAT setting on the working firewall, I noticed a couple rules that were created automatically. On the new firewall (pfSense CE 2.8.1), even though it was set to automatically create NAT rules, it wasn't creating them. As soon as I put NAT rules in place for all the nets everything started working! I'm streaming a series and watched three episodes without any interruptions. Now I'm happy.
The firewall that was in place and running on pfSense+, I need to downgrade. The original plan was to use igb4 for the WAN (which works) and igb4 for the LAN. I'll create the NAT rules manually so I can, hopefully, rule out the NAT config.
Is it possible that there may be a glitch with pfSense CE and automatic NAT rules? pfSense+ did not have this issue.
Stephen, let me finish rebuilding the firewall so I can post the results here. Once all is clear this subject can be closed. Thank you so much for all the feedback and assistance.
Martin M. Mune
US Army Combat Veteran
Operation Iraqi FreedomVolunteer Soldier
International Legion for the Defense of UkraineСлава Україні!
Героям Слава! -
@martimun said in Steelhead CX770 Port issues:
Is it possible that there may be a glitch with pfSense CE and automatic NAT rules?
Very unlikely. We would have thousands of complaining threads. The code there is almost all the same between CE and Plus
Much more likely is that what triggers the automatic outbound NAT rules is not in place. And that is that pfSense will add NAT rules on every WAN for all internal subnets but to determine which is a 'WAN' it checks for a gateway on the interface config. If you add a WAN gateway separately, in System > Routing > Gateways, it will work as a gateway but will not be on the interface directly and hence you won't get any OBN rules for it. I've seen that mistake many, many times!
-
@stephenw10, thank you for the clarification! Now I see exactly what happened! On the new firewall I actually created the WAN gateway to reflect the circuit ID from Spectrum Business whereas on the old firewall I left it untouched. But still it doesn't explain why any inter-VLAN/internal traffic was not moving which could still point to a physical port issue.
I still don't trust those ports though. Today, I will rebuild the old firewall to see if I can use all ports as intended. I'll post the results as soon as I finish. Again, thank you for your input and guidance!
Martin M. Mune
US Army Combat Veteran
Operation Iraqi FreedomVolunteer Soldier
International Legion for the Defense of UkraineСлава Україні!
Героям Слава! -
Indeed the lack of outbound NAT rules would not prevent access to other local subnets as long as firewall rules allow it. And assuming the other hosts allow access from a different subnet.
-
Looks like all is in good shape. I rebuilt the firewall and all ports are fully functional. I also re-checked all the BIOS settings. Everything looks good as long as the bypass interfaces are set to "No Bypass" in the BIOS. The good thing about the CX770s is that the bypass control is in the BIOS. So far performance is perfect! Looking forward to many years out of my modest investment on these CX770 devices. For anyone interested in these devices, they can be purchased on eBay for around $70-$100.
I would like to thank Stephen for all the assistance and input. This one soldier is an incredibly happy customer!
Martin M. Mune
US Army Combat Veteran
Operation Iraqi FreedomVolunteer Soldier
International Legion for the Defense of UkraineСлава Україні!
Героям Слава!
