PfSense on a Riverbed Steelhead
-
@pantigon I've not tried a CXA-255, but based on the chassis, it might have BIOS control of the bypass NICs like 570/770 boxes. Meaning you won't have to fiddle with smbus settings. Take a look through the BIOS options and search for bypass NIC settings. I don't remember for sure, but I think they need to be set to "disable".
-
@pantigon I should have clarified. The LAN/WAN NICs should be enabled, but then disable the "bypass" feature.
As a reminder, the bypass feature causes the LAN/WAN ports to act like a wired crossover coupler when the box is powered off.
-
@pantigon I was poking around with my CX-770 today, and the BIOS setting for the bypass NICS should be set to "No Bypass". Your's might be the same.
-
@okijames Hi, in my CX-570, I have done the following settings in the bios:
but it seem to not work in proxmox 7.x
Could you help me. -
@anonsaber What's not working? Proxmox itself or pfsense inside Proxmox? FWIW installing Proxmox is a pain to install without a video card. What was your process?
-
@okijames I have connected GT710 with PCIE extension cable, booted from USB and finished the installation of Proxmox.
It showed that the former two ports worked in proxmox, but the last 4 ports was unavailable . (I just connected my worksation with CX-570 and executed ifup <interface name> on each nic in pve host.)
Then according to this post, I modified the BIOS, re-entered proxmox, but unfortunately these ports were still out of action.
PS: psfence was not been installed or used.
-
Quick Q for moderators, if you'd prefer we move this conversation to Proxmox forums, I'd be happy to.
@anonsaber Under Proxmox the NICs have interesting names of enpXs0. Going left to right, starting with the Pri port on the front of the 570/770, the NICs are...
enp2s0, enp3s0, enp1s0f1, enp1s0f2, enp1s0f3, enp1s0f4
These should all show up in dmesg. I enabled the NICs (by setting "No Bypass" like you did) before installing Proxmox. I have no idea how/if Proxmox will recognize them when you enable them post-install.
I only use one NIC (enp1s0f1) and I don't think I had to do an if up. Just edit /etc/network/interfaces. Mine is pasted below, and note that I modified from static IP to DHCP because prefer to use DHCP static assignments via DHCP.
cat /etc/network/interfaces # network interface settings; autogenerated # Please do NOT modify this file directly, unless you know what # you're doing. # # If you want to manage parts of the network configuration manually, # please utilize the 'source' or 'source-directory' directives to do # so. # PVE will preserve these directives, but will NOT read its network # configuration from sourced files, so do not attempt to move any of # the PVE managed interfaces into external files! auto lo iface lo inet loopback iface enp1s0f1 inet manual iface enp1s0f2 inet manual iface enp1s0f3 inet manual iface enp1s0f4 inet manual iface enp2s0 inet manual iface enp3s0 inet manual auto vmbr0 iface vmbr0 inet dhcp bridge-ports enp1s0f1 bridge-stp off bridge-fd 0Also note Proxmox uses the bridge interfaces "vmbrX" rather than assigning IP addresses to the physical NICs. My "ip a" output looks like this...
ip a 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2: enp1s0f1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master vmbr0 state UP group default qlen 1000 link/ether 00:0e:b6:b2:33:f0 brd ff:ff:ff:ff:ff:ff 3: enp1s0f2: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000 link/ether 00:0e:b6:b2:33:f1 brd ff:ff:ff:ff:ff:ff 4: enp1s0f3: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000 link/ether 00:0e:b6:b2:33:f2 brd ff:ff:ff:ff:ff:ff 5: enp1s0f4: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000 link/ether 00:0e:b6:b2:33:f3 brd ff:ff:ff:ff:ff:ff 6: enp2s0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000 link/ether 00:0e:b6:78:06:b0 brd ff:ff:ff:ff:ff:ff 7: enp3s0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000 link/ether 00:0e:b6:78:06:b1 brd ff:ff:ff:ff:ff:ff 8: vmbr0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000 link/ether 00:0e:b6:b2:33:f0 brd ff:ff:ff:ff:ff:ff inet 10.0.0.10/24 brd 10.0.0.255 scope global dynamic vmbr0 valid_lft 7065sec preferred_lft 7065sec inet6 fe80::20e:b6ff:feb2:33f0/64 scope link valid_lft forever preferred_lft forever -
@okijames said in PfSense on a Riverbed Steelhead:
Quick Q for moderators, if you'd prefer we move this conversation to Proxmox forums, I'd be happy to.
Probably more relevant there. But this thread doesn't see much action, it's not like everyone has to read it. Not really a problem IMO.
-
@okijames At first I didn't to do an if up, but when I connect to the network nothing happens.
These is my interface status:
and these is my network config file:# network interface settings; autogenerated # Please do NOT modify this file directly, unless you know what # you're doing. # # If you want to manage parts of the network configuration manually, # please utilize the 'source' or 'source-directory' directives to do # so. # PVE will preserve these directives, but will NOT read its network # configuration from sourced files, so do not attempt to move any of # the PVE managed interfaces into external files! auto lo iface lo inet loopback auto enp3s0 iface enp3s0 inet manual auto enp1s0f1 iface enp1s0f1 inet manual auto enp1s0f2 iface enp1s0f2 inet manual auto enp1s0f3 iface enp1s0f3 inet manual auto enp1s0f4 iface enp1s0f4 inet manual auto enp4s0 iface enp4s0 inet manual auto bond0 iface bond0 inet manual bond-slaves enp3s0 enp4s0 bond-miimon 100 bond-mode balance-rr auto vmbr0 iface vmbr0 inet static address 100.64.10.1/24 bridge-ports bond0 bridge-stp off bridge-fd 0 auto vmbr1 iface vmbr1 inet static address 192.168.1.1/24 bridge-ports enp1s0f1 bridge-stp off bridge-fd 0and there is my dmesg logfile:
dmesg.txt -
@lemon-k I must be misinterpreting something. It appears to me both your nodes are connected in a cluster (therefore using a functional network), and you have VMs running on each node. This is significantly different from "nothing happens" so I don't understand the issue.
I suggest trying the Proxmox forums and detailing what you're trying to accomplish and what is and is not working.
-
@stephenw10 Hi gonna jump on this my self, Picked up a CX-255 my self intending on Using it as a PfSense Box. i see a part number labeled 450-00556-03 on the bottom of my case.
I have My Onboard LAN Bypass Settings in my bios set to No Bypass.
They are showing up in pfsense but dont work still, is there any other commands and such i may need to run.i can provide any other info if needed like outputs from commands and such
-
Note: The setup wizard's link up/down auto port detection for identifying ports is not reliable. OS level link up/down detection works fine, but not in the wizard for some reason. So your NICs are probably working fine, but not for the wizard's auto port detection. You pretty much have to manually figure out which port = which igb NIC, then assign them manually to LAN, WAN, etc.
IIRC the mapping on my CX-570 is...
Pri = ign4
Aux = igb5
LAN/WAN ports are igb0 through igb3 -
@okijames During the setup i manually picked the nics and igb0-1 just seem to be passing through to each other only one that seems to work is igb2 which is the nic labeled PRI
Had picked Igb0 for WAN and igb1 and 2 for my lans
Had been planning on trying to use this as my main router instead of the crappy ISP one.I know there's a passthrough happening as even though igb1 is set to have the range 192.168.1.0/24, the device plugged into that port is is picking up an IP from the ISP router which is my wan for testing.
-
@theorangefloof Ok a couple things...
First, just out of an abundance of caution, I would not use a pair of ports capable of bypass (Ex: LAN_0 and WAN_0) as pfsense LAN/WAN ports. There's just too much potential for them to be set to bypass, where they behave as a physical crossover coupler. Use Pri and Aux instead.
Second, many Steelheads have IPMI/BMC available on the Pri port, so it's best to use it as a pfsense LAN port. Use Aux as the WAN port and Pri as the LAN port because they have no bypass capability between them, and the Aux port does not have IPMI/BMC.
On CX-570/770 the IPMI/BMC board is removable if 100% disabling of this capability is desired. Removing the board would allow safe use of Pri as a WAN port without potentially exposing IPMI/BMC to the world.
Oops, just did a little searching and...
In your case with a CX-255, which has no Aux port and no mention IPMI/BMC so, I'd use the Pri as WAN, and the bypass ports for your two LANs.
If you're still seeing bypass behavior between LAN ports, try some of the other BIOS options for the bypass NICs. I don't have a CX-255 or I'd check for you.
FWIW, the bypass feature (two ports acting like a cross-over coupler) should only engage when the Steelead is powered down. When power is applied, and the NICs are up, they should behave like two normal NICs.
-
@okijames Okay i tried the Lan bypass setting in the bios as any of the 3 values, still getting the bypass behavior, and the lights on igb0-1 are always showing orange no matter the setting picked in the Bios, also emailed to the seller who i brought this off he linked me to post 56 in this exact thread. so haven't gotten anywhere further yet
-
@theorangefloof FYI the post you were referred to is probably not correct for your model. That was for the older CX-250/550 32bit machines with no BIOS control of the bypass function.
If your BIOS looks like the screen shots below, all you needed to do was set the bypass NICs to "No Bypass".
Behavior of the bypass NICs should be...
-Orange/Amber NIC lights immediately after power up using the toggle switch on the back of the unit.
-After a couple seconds, you should hear a distinct click sound, then no NIC lights. The Power and HDD LEDs should also turn on after the click.You might also try dropping to shell after boot, and issue ifup commands for both bypass NICs.
If you followed post 56 and issued the smbmsg commands, I have no idea what state your machine is in. The bypass NICs might be unusable. Sorry if that's the case. I'll edit the old post with a warning.
-
This is what my bios looks like, didn't run the commands other than kldload and smbmsg -p, the addresses i have were completely different to that post anyway.
ran ifconfig igb0 up and ifconfig igb1 up, nothing no relay clicks that i could hear.
also haven't heard anything other than the startup/shutdown beeps and the fans.
Also the nic lights don't go off after i run those commands, i also looked at the status from ifconfig it says no Carrier on igb0 and 1. -
@theorangefloof Hate to say it, but you might just have some bad hardware.
-
@okijames You are right. I found a cx-770.
Apparently, it should be able to hear the relay when turning the machine on, but was not heard on my previous cx-570. After I replaced the relays, those Nic worked fine.
This has nothing to do with proxmox or pfsense.
-
Nice troubleshooting!
