Steelhead CX770 Port issues
-
@martimun said in Steelhead CX770 Port issues:
I'm not able to access the webui from any VLAN on igb5
Ah OK. Can you ping any IP on pfSense even? Do clients on those VLANs get a DHCP lease in the correct subnet? (assuming dhcp is in use).
Or do clients on LAN_AUX work? So using igb5 but without VLAN tagging?
-
@stephenw10, on the firewall where the VLANs are on port igb0 yes, I can. When connecting my laptop to LAN_AUX, I assign an IP address in the range of the IP address on LAN_AUX. The gateway is the .254 IP address that LAN_AUX has assigned. The only thing I can do is connect to the webui interface. I have tried every single possible combination imaginable. I even configured a Dell Force10 S55T spare with the same config as my production switch just to make sure there is nothing wrong with the switch config.
Just for testing purposes, I'll switch all VLANs on the new firewall to port igb0 (like in the firewall in production) just to confirm that the new firewall works as intended. If that works, that means there is something really weird going on with the igb5, igb1, igb2, and igb3 on the actual hardware. That's why I posted in this forum; since this particular Steelhead device is fairly popular as a pfSense firewall/router and knowing that people got them to work successfully, I'd like to see if someone here can sort me out.
Martin M. Mune
US Army Combat Veteran
Operation Iraqi FreedomVolunteer Soldier
International Legion for the Defense of UkraineСлава Україні!
Героям Слава! -
Hmm, sure seems like something low level not working with that port.
So you are not using pfSense as the DHCP server for those subnets?
-
@stephenw10, no. I have a pair of Windows domain controllers doing DHCP duty. On the production firewall, all nodes in every VLAN can get a DHCP lease except for VLAN 101. That's the server VLAN and every node has its own static IP address. I have to agree; there is something really wrong with the ports at the hardware level. And, it's not just one port; it's four of them and in two different units. Too much of a coincidence if you ask me. Yesterday, I haven't had time to transfer the VLANs to the igb0 port on the firewall that I'm bringing online. Later today, I'll do the transfer and report back with the results.
Martin M. Mune
US Army Combat Veteran
Operation Iraqi FreedomVolunteer Soldier
International Legion for the Defense of UkraineСлава Україні!
Героям Слава! -
Yeah I mean this really seems like it's the bypass NICs at work to me. Except that you confirmed pfSense sees igb5 get and lose link when you connect to it?
If so then I would run a packet capture on igb5 to see if anything is actually coming in on it.
The fact you can connect to the gui but only the gui on igb1 pretty much has to be a firewall rule issue.
-
@stephenw10, since I'll be switching interfaces, I'll also do a full reset of the firewall so I can reenter all rules. I'll also run Wireshark on the new firewall to see how things flow (or not!) through the ports. It really is a shame that these ports are not working as expected because the hardware is rock solid for a pfSense install. At least I can feel good that I purchased these two CX770s brand new as a bundle for only $100 on eBay. They came from a government surplus site in their original box and all the accessories.
Martin M. Mune
US Army Combat Veteran
Operation Iraqi FreedomVolunteer Soldier
International Legion for the Defense of UkraineСлава Україні!
Героям Слава! -
Mmm, all the by-pass ports I've ever encountered disconnect the internal interfaces when they are in by-pass mode. But I could imagine (maybe) a mode where it loops back the data lines while the MII connections from the PHY still report the link status. But for that to happen you would need both NICs in the by-pass pair to be linked.
-
@stephenw10, just finished switching all VLANs to the igb0 port. Guess what? ! It all works without any issues. Something is very wrong with the ports. It's not just the bypass ports; the AUX (igb5) which is controlled separately doesn't work. At this point I'm out of ideas. I also ran Wireshark and all traffic stops at the igb5 port. ICMP and HTTP traffic successfully reach the IP address on igb5 but nothing else goes through. I rechecked the BIOS settings to make sure the igb0-3 are not in bypass mode.
Anyone who has gotten a CX770 to work successfully, I would appreciate any assistance you can give. This is truly bizarre!
Martin M. Mune
US Army Combat Veteran
Operation Iraqi FreedomVolunteer Soldier
International Legion for the Defense of UkraineСлава Україні!
Героям Слава! -
@martimun said in Steelhead CX770 Port issues:
I also ran Wireshark and all traffic stops at the igb5 port. ICMP and HTTP traffic successfully reach the IP address on igb5 but nothing else goes through
When testing from where?
-
@stephenw10, everything was working and when I returned to the laptop directly connected to the CX770, nothing worked anymore. When I looked at the NAT setting on the working firewall, I noticed a couple rules that were created automatically. On the new firewall (pfSense CE 2.8.1), even though it was set to automatically create NAT rules, it wasn't creating them. As soon as I put NAT rules in place for all the nets everything started working! I'm streaming a series and watched three episodes without any interruptions. Now I'm happy.
The firewall that was in place and running on pfSense+, I need to downgrade. The original plan was to use igb4 for the WAN (which works) and igb4 for the LAN. I'll create the NAT rules manually so I can, hopefully, rule out the NAT config.
Is it possible that there may be a glitch with pfSense CE and automatic NAT rules? pfSense+ did not have this issue.
Stephen, let me finish rebuilding the firewall so I can post the results here. Once all is clear this subject can be closed. Thank you so much for all the feedback and assistance.
Martin M. Mune
US Army Combat Veteran
Operation Iraqi FreedomVolunteer Soldier
International Legion for the Defense of UkraineСлава Україні!
Героям Слава! -
@martimun said in Steelhead CX770 Port issues:
Is it possible that there may be a glitch with pfSense CE and automatic NAT rules?
Very unlikely. We would have thousands of complaining threads. The code there is almost all the same between CE and Plus
Much more likely is that what triggers the automatic outbound NAT rules is not in place. And that is that pfSense will add NAT rules on every WAN for all internal subnets but to determine which is a 'WAN' it checks for a gateway on the interface config. If you add a WAN gateway separately, in System > Routing > Gateways, it will work as a gateway but will not be on the interface directly and hence you won't get any OBN rules for it. I've seen that mistake many, many times!
-
@stephenw10, thank you for the clarification! Now I see exactly what happened! On the new firewall I actually created the WAN gateway to reflect the circuit ID from Spectrum Business whereas on the old firewall I left it untouched. But still it doesn't explain why any inter-VLAN/internal traffic was not moving which could still point to a physical port issue.
I still don't trust those ports though. Today, I will rebuild the old firewall to see if I can use all ports as intended. I'll post the results as soon as I finish. Again, thank you for your input and guidance!
Martin M. Mune
US Army Combat Veteran
Operation Iraqi FreedomVolunteer Soldier
International Legion for the Defense of UkraineСлава Україні!
Героям Слава!
