PfSense on a Riverbed Steelhead



  • I successfully installed pfSense on an old Riverbed Steelhead appliance. You can see how, with pictures of the unit, here.

    Unfortunately, I ran into the same problems with the LAN bypass ports from this thread (though mine is a newer model). Of course, the original poster in that thread leaves without specifying how it was fixed.

    Regardless, it is still a useful appliance for pfSense, even with just two ports. It'll even fit on the shelf in my basement better than the Watchguard Firebox I have.



  • The synopsis of the installation is basically:
    1. Remove the flash drive and hard drive from the appliance
    2. Plug the flash drive into a USB header on your PCs motherboard
    3. Write the pfSense image to it
    4. Replace the flash drive, set kern.cam.boot_delay=10000
    5. Enjoy

    em0 and em1 are the bypass ports and don't work. em2 and em3 are the aux/primary ports and do work.


  • Netgate Administrator

    Yes, disappointing he didn't come back. Anyway reading between the lines I would guess he disabled or changed the mode of the LAN-bypass in the BIOS. As I said there you often use the TAB key to enter the BIOS setup via serial console. Can you access the BIOS on your box?

    Steve


  • Netgate Administrator

    You could probably disable the bypass manually with a small program anyway. Is there any evidence that board is made by Axoimtech?

    For example they give source code for their LAN-bypass control for various products:

    /* Use GPIO Control LAN by-pass*/
    /**************************************************************/
    int _8a811_lan_by_pass_enable(void)
    {
    	iopl(3);
    //	_gpio_use_sel(BIT0);
    //	_gp_io_sel(BIT12, GP_IO_SEL_OUT);
    	_gp_lvl(34, Low);	//control by ICH GP34
    	_gp_lvl(23, Low);	//control by ICH GP23
    	return 0;
    }
    /*******************************************/
    int _8a811_lan_by_pass_disable(void)
    {
    	iopl(3);
    //	_gpio_use_sel(BIT0);
    //	_gp_io_sel(BIT12, GP_IO_SEL_OUT);
    	_gp_lvl(34, High);	//control by ICH GP34
    	_gp_lvl(23, High);	//control by ICH GP23
    	return 0;
    }
    
    

    It would be easy to test something similar given sufficient clues.

    Steve



  • I did something stupid somewhere that is keeping from getting back into the BIOS, but when I was able to get into the BIOS, there were no options for LAN bypass.

    I'll have to look on the main board when I get home to see what markings there are and if any indicate what model the board is.



  • The board is a Jabil Circuits MNPBA000698C. I'm not having any luck finding much documentation on the board, but I can buy one for less than $30 on eBay: http://www.ebay.com/itm/JABIL-CIRCUIT-MOTHERBOARD-MAINBOARD-SBC-NEW-/300300470392



  • Yes, those are LAN bypass ports.  Looking at your pics, the 4 big white things on the MB behind the ports are relays; these control if the active lines on each port are connected to the MB or to each other.

    I'm not clear if the GPIO code, like Steve posted or in the bios, will directly control the state (bypassed or not), or if the GPIO simply allows or prevents bypass on power fail.  I suspect the latter.

    If you don't need the bypass feature, you may be able to do some hardware hacking to energize the relays directly from somewhere else on the board.  I see a big 1F supercap there, and there's also an LED on the front panel to indicate bypassed condition that may provide some clues.


  • Netgate Administrator

    The way that LAN bypass usually works is that the relays are powered via the output from a watchdog timer. This can be in the SuperIO chip or in the southbridge chip. The design of the system is such that if the OS crashes and stops resetting the watchdog then the LAN-bypass automatically kicks in. It's usually possible to control the properties of the watchdog by setting various registers in which ever chip is running it. Alternatively it might be possible to drive it via some separate GPIO pins in parallel.

    I see in your photo the board is labelled: ETON ET866 94V-0
    That seems to bring up references to graphics cards though.  :-\ Jabil Circuits seems to be a pcb assembly company rather than a motherboard designer so not much help as you found.

    I notice that the connector in the top right of the photo is labelled J20. This implies there are are more jumpers! Are there any that we can't see in the photo?

    Steve



  • There does not appear to be any other jumpers on the board. I've looked for any other J# labels as well (perhaps the pins were removed) and I don't see any.



  • Where you ever able to recover the bios password? I have the same issue, trying to get into the bios…



  • You could force bypass off permanently by shorting the control pins to either ground or 1v8/3v3/5v


  • Netgate Administrator

    You have identical hardware?
    https://shoup.io/project-steelwall.html

    it's a little out of date these days. No need to mount RW if you're running Nano as it's always mounted RW.

    Always use /boot/loader.conf.local

    That hardware appears to be 32bit which means no 2.4.

    Steve



  • hi

    i also interested to get the two left NICs working.

    someone been successful?

    Thanks


  • Netgate Administrator

    You have access to the BIOS?

    Any lan bypass or watchdog functions available there?

    Otherwise you will need to switch the relays manually by flipping the control registers. Or by changing the circuit that drives them.
    Are you up for a challenge?  ;)

    Steve



  • @stephenw10:

    You have access to the BIOS?

    Any lan bypass or watchdog functions available there?

    Otherwise you will need to switch the relays manually by flipping the control registers. Or by changing the circuit that drives them.
    Are you up for a challenge?  ;)

    Steve

    Unfortunately there is no option in the BIOS to activate the relays manually.

    I read a few posts on other Websites about bridge the relays power feedpin to an Mosfet to power them from boot.

    iam a noob in things like this. But i really want to get it working. do you have some information, maybe some pictures how i had to modify the relais circuit ?

    here is a picture

    thank you so far


  • Netgate Administrator

    Ok, so to be clear there no bypass OR watchdog settings in the BIOS?

    And there are no jumpers on the PCB? I can't make out any from your photo but it's not very high resolution.

    You have two choices. Electrically bridge the relays by powering them from somewhere. Or, more fun IMO, try to find the GPIO that controls the relays and set it in software.

    There will be typically two places that have GPIOs that could have been used, the ICH and the SuperIO chip. They may have used the parallel port but that's crude, unlikely for relays that are on the main board.

    Steve



  • I'd go the fun route. It's way more fun.  ;D


  • Netgate Administrator

    Yup, immensely more satisfying when (if) you get it to work.  ;D

    Steve



  • I took some pictures in better  resolution

    maybe you can see something on it



  • I'm afraid the real tracks are on the other side of the board. On top of that, GPIO control is done in software, so you'd have more luck poking around on the GPIO's on the shell.


  • Netgate Administrator

    Mmm, with no jumpers and nothing in the BIOS it's time to start poking GPIOs.

    You might want to read this thread for some ideas how to do that: https://forum.pfsense.org/index.php?topic=81292.0

    Steve



  • Have anyone enable this bypass ports?



  • @HarryH:

    Have anyone enable this bypass ports?

    Poke the GPIOs and you'll know :-)


  • Netgate Administrator

    Yup, you'll have to start poking GPIO registers. Tedious but fun when it works!

    I can probably offer assistance as time allows.

    Steve



  • @stephenw10

    If you reinstall the original Steelhead code you can toggle the bypass NIC to "fail-to-block" i.e. keep both NICs up all the time. See CLI commands below. The interface name is "inpath0_0". You can do a "show run" CLI command to see all the settings and interface names.

    Once you set "fail-to-block", the HW seems to remember the setting (it must be flipping a hidden BIOS setting) so you can install pfsense and have the additional two NICs.

    Fail-to-Block CLI commands:
    • no interface <interface-name> fail-to-bypass enable: Sets the interface to block when there is a failure.
    • interface <interface-name> fail-to-bypass enable: Sets the interface to bypass when there is a failure.



  • Hello everyone,

    I recently got a Rivebed Steelhead 250L identical to the on the image.
    There is a jumper that (J20) located beside the connection between the power supply and the motherboard.
    Just tried it now and it resets the bios settings :( not the bypass ports.

    I am trying to get ESXI or PFsense or FreeNAS to work on it now.


  • Netgate Administrator

    So you also have ports stuck in by-pass mode? No BIOS access?

    Steve



  • @stephenw10 Thanks for the reply. Yes they are not turning on. I do have bios access ( pressing del on startup) but no option there to change any setting regarding network.

    Tried:
    Fail-to-Block CLI commands:
    • no interface <interface-name> fail-to-bypass enable: Sets the interface to block when there is a failure.
    • interface <interface-name> fail-to-bypass enable: Sets the interface to bypass when there is a failure.

    Curious that i hear no clicks when i execute the commands, maybe power is already applied.

    When running on RIOS, these commands do turn the lights off/on on the ports. After reboot and boot from pfsense on usb, the ports remain off. Pfsense detects all ports but only AUX and Primary usable.

    I have tried also to install ESXI / Freenas on this appliance with no success even booting.


  • Netgate Administrator

    Ok, so unless you can see any jumpers that might set them then, as I said two years ago, you will have to start poking at GPIOs to try to find what controls them. You might find a clue in the RIOS boot logs or filesystem.

    Steve



  • Regarding GPIOs, rather than poking around thi maybe this can be of use. Using ipmitool under Linux, these commands are said to enable inpath nics. Never tried it myself.

    ipmitool -v raw 0x3e 0x20 0x80 0x7f 0x00 0x00
    ipmitool -v raw 0x3e 0x20 0x80 0x7f 0x01 0x00

    I think the above alters some i2c registers to do the trick. Try it under Linux first. If it works, try a FreeBSD method for setting such registers.

    I took a look at the, again Linux based, approach used by rbMode on github. That guy's script refers to SMBUS and I can't figure how how SMBUS addresses relate to the ipmitool addresses.



  • Quick note: Riverbed used Silicom bypass nics in their larger appliances. Silicom has FreeBSD drivers with source code on their web site. The source code might provide insight into methods for enabling your bypass nics. URL for Silicom 2port bypass nic here....

    https://www.silicom-usa.com/pr/server-adapters/networking-bypass-adapters/gigabit-ethernet-bypass-networking-server-adapters/pe2g2bpi80-bypass-card/


  • Netgate Administrator

    Hmm, yeah that's interesting. It could almost certainly be done just using shell script then. Just need to figure out the smbus addresses......

    ipmitool is already installed though and takes the same input as Linux so should would if that data in correct. Though it still requires an IPMI device of some kind in raw mode.
    Anyone able to test it?



  • @Okijames said in PfSense on a Riverbed Steelhead:

    ipmitool -v raw 0x3e 0x20 0x80 0x7f 0x01 0x00

    Just tried it on my riverbed 250 and unfortunetly it did not work. Error is:

    Could not open device at /dev/ipmi0 or /dev/ipmi/0 or /dev/ipmidev/0: No such file or directory

    There is a python script that i have tested to work but only on linux, Debian in my case. Pfsense's FreeBSD no luck so far.
    Maybe the script can help : Script


  • Netgate Administrator

    What happened when you ran rbmode in pfSense? What error?

    You probably have to load the smbus drivers for that to work:

    kldload ichsmb
    kldload smb
    

    Steve



  • @stephenw10 said in PfSense on a Riverbed Steelhead:

    kldload smb

    Hello,

    Thank you stephenw10 for your help.
    First the script imports smbus module but cant find it. I had to install pip and then smbus2 and change the script. That error went away.
    I executed the kldload commands with "success".

    After that rbmode gave this error:

    [2.3.5-RELEASE][root@pfSense.localdomain]/usr/local/sbin: rbmode u
    Setting mode to universal

    Traceback (most recent call last):
    File "/usr/local/sbin/rbmode", line 78, in <module>
    main()
    File "/usr/local/sbin/rbmode", line 70, in main
    setMode(args.mode)
    File "/usr/local/sbin/rbmode", line 38, in setMode
    s = smbus2.SMBus(0)
    File "/usr/local/lib/python2.7/site-packages/smbus2/smbus2.py", line 279, in init
    self.open(bus)
    File "/usr/local/lib/python2.7/site-packages/smbus2/smbus2.py", line 308, in open
    self.fd = os.open(filepath, os.O_RDWR)
    OSError: [Errno 2] No such file or directory: '/dev/i2c-0'

    Indeed there is no /dev/i2x-0 but there is a /dev/smb0 so i replaced the reference in smbus2.py from i2c to smb but then another error occurred after re-executing rbmode:

    IOError: [Errno 25] Inappropriate ioctl for device

    Note: I got this script to work on Debian flavour of linux with success.

    My knowledge of python is very very low as is FreeBSD so i am stuck at the moment...



  • FWIWI tried compiling the Silicom drivers on a FreeBSD 10.3 (to match pfSense 2.3.5) and turns out the bpmod.ko kernel module (note this is pre-built, NOT actually available as source code in the driver package) won't work, it complains about not finding a Silicom card, as expected I guess...


  • Netgate Administrator

    With those smb modules loaded try running: smbmsg -p

    That should return all the available smbus devices available. But it also might hang the bus or even the whole box so be ready to power cycle it. It will probably list some things though. Hopefully including 0x24.

    The smbus python tools appear to be more like raw i2c tools. The actual output command in rbmode appears to write to a device at 0x24 with subaddress (or smbus command) 0x55 and whatever the output of that checksum function is. My python sucks! 😉

    Steve

    You can probably do that easily enough with smbmsg.



  • root@:~ # kldload ichsmb
    ichsmb0: <Intel 631xESB/6321ESB (ESB2) SMBus controller> port 0x540-0x55f irq 19 at device 31.3 on pci0
    smbus0: <System Management Bus> on ichsmb0
    root@:~ # kldload smb
    smb0: <SMBus generic I/O> on smbus0
    root@:~ # smbmsg -p
    Probing for devices on /dev/smb0:
    Device @0x10: w
    Device @0x48: rw
    Device @0x5c: rw
    Device @0x60: rw
    Device @0x64: rw
    Device @0x66: rw
    ichsmb0: device timeout, status=0x41
    ichsmb0: device timeout, status=0x41
    ichsmb0: device timeout, status=0x41
    ichsmb0: device timeout, status=0x41



  • That's what I get, and then the "device timeout" messages continue forever. Rerunning smbmsg -p shows nothing but the timeout messages, so pretty much need to reboot after running once. No x24 addresses of course! curses...


  • Netgate Administrator

    Hmm. Have you also seen that script work in Linux?

    I could imagine the device might come up at a different address in FreeBSD perhaps...


Log in to reply