Banana Pi

toomeek · Jul 6, 2014, 5:47 PM

Few more issues later..
The point is to avoid modifing original pfSense OS, since things may mess up during later upgrades.
So I'm just modifing host OS on Banana Pi to allow booting pfSense (As far I know there is no way to run it on bare metal).

1. Printer port issue

add Parallel device with file pointing to, as example: /dev/printer0.ps
2. VGA card stuck
during booting OS stuck again, so replace currius driver with vmvga video type in QEMU settings
3. Slow install
my microSD card is REALLY SLOW, only 15MB/s read and 1MB/s write, so for installation I've used cache mode "writeback" for IDE disk to speed it up during writes (note this is UNSAFE if power fails!),
it was even so slow that guest OS reported couple LBA timeouts (!)
4. Enable NIC offload features on host
/etc/rc.local

#Enable offload for onboard Gigabit NIC
ethtool -K eth0 rx on tx on sg on gso on gro on
#tcp-segmentation-offload and large-receive-offload are not supported for this hardware!

5. Enable other "HA solutions"
/etc/rc.local

#Reboot on kernel panic
echo "1" > /proc/sys/kernel/panic

6. Create bridge for QEMU
/etc/network/interfaces

auto lo

iface lo inet loopback

#Manual mode for Gigabit NIC
auto eth0
iface eth0 inet manual

# Bridge for VM traffic
auto br0
iface br0 inet static
    address 10.20.20.222
    netmask 255.255.255.0
    network 10.20.20.0
    broadcast 10.20.20.255
    gateway 10.20.20.1
    dns-nameservers 10.20.20.1 8.8.8.8
    bridge-ports  eth0
    bridge-fd     9
    bridge-hello  2
    bridge-maxage 12
    bridge-stp    off

allow-hotplug wlan0
iface wlan0 inet manual
wpa-roam /etc/wpa_supplicant/wpa_supplicant.conf
iface default inet dhcp

7. Not booting after install
Boot loader just shows:

F1 pfSense
F6 PXE
Boot: F1 #
Read error

so I've installed i386 version on another host, then gzip -> copy via SCP -> gunzip
this didn't change anything, same results
answer was found here:
http://www.irbs.net/FreeBSD/FAQ/book.html
http://www.unixguide.net/freebsd/faq/05.28.shtml

4.26. Why does FreeBSD's boot loader display ``Read error'' and stop after the BIOS screen?
FreeBSD's boot loader is incorrectly recognizing the hard drive's geometry. This must be manually set within fdisk when creating or modifying FreeBSD's slice.
The correct drive geometry values can be found within the machine's BIOS. Look for the number of cylinders, heads and sectors for the particular drive.
Within sysinstall(8)'s fdisk, hit G to set the drive geometry.
A dialog will pop up requesting the number of cylinders, heads and sectors. Type the numbers found from the BIOS separates by forward slashes.
5000 cylinders, 250 sectors and 60 sectors would be entered as 5000/250/60
Press enter to set the values, and hit W to write the new partition table to the drive.

so I've run fdisk on host and it shows:

Disk /var/lib/libvirt/images/pfSense.img: 8589 MB, 8589934592 bytes
16 heads, 63 sectors/track, 16644 cylinders, total 16777216 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes

and compared this to fresh image:
qemu-img create -f raw test.img 8G
size match!
so that pointed me there must be something else.. regarding VM BIOS.
So, I've switched to different Boot Loader - GAG
http://gag.sourceforge.net/
after running it under Linux I was still not able to start it.. GAG stucks at "1".
And I'm stuck there, again.

DiskWizard · Jul 12, 2014, 12:44 PM

Erm, well - about non-bootable image. I will try to explain from datarecovery point of view.

Exact answer is - "Wrong" geometry of image.

Bootloader treats image considering it has 48bit LBA address translation, but since the image has 8Gb of data (am I correct about sizing?) the blocks are allocated unexpectedly for boot loader to see the data.

My recommendation is

Stripe down HDD size to the one SD card uses utilizing HPA. (reversible change)

repeat your steps once more.

toomeek · Aug 2, 2014, 4:15 PM

Hello, sorry for very long response.
The problem was indeed in VM BIOS.
I've created VM with only available option "Other OS" instead of "FreeBSD 8.x" in creator, this caused unbootable state.
Raspbian is missing some important packages, while Lubuntu even has virt-manager available, so I was able to use it to manage my VM. Newer packages also contains more options specific for booting.
However, pfSense stuck at VGA again (changing model doesn't help), it starts only in Fail-safe mode.
When I try to access GUI on this VM all I get is: login screen and after typing password:

CSRF check failed. Either your session has expired, this page has been inactive too long, or you need to enable cookies.
Debug: sid:1bb48302acef38e0c9b4d5d97244925cbc514c18,1410555529

I was looking into different BSD projects, some of them are for ARM (NetBSD), however doesn't seem to support Allwinner CPUs.. so it doesn't seem to run pfSense on Banana PI any way :(

But to let You know how Banana Pi performs networking here is result from iperf (Banana as server, connected by crossover cable to Gigabit client). Networking setup as described before (offload engines enabled, bridged to onboard NIC, etc..)

iperf-2.0.5-2-win32>iperf -c 192.168.137.222

------------------------------------------------------------
Client connecting to 192.168.137.222, TCP port 5001
TCP window size: 64.0 KByte (default)
------------------------------------------------------------
[  3] local 192.168.137.1 port 55946 connected with 192.168.137.222 port 5001
[ ID] Interval       Transfer     Bandwidth
[  3]  0.0-10.0 sec   429 MBytes   360 Mbits/sec

iperf-2.0.5-2-win32>iperf -c 192.168.137.222 -P 10

------------------------------------------------------------
Client connecting to 192.168.137.222, TCP port 5001
TCP window size: 64.0 KByte (default)
------------------------------------------------------------
[ 12] local 192.168.137.1 port 55957 connected with 192.168.137.222 port 5001
[  9] local 192.168.137.1 port 55954 connected with 192.168.137.222 port 5001
[ 11] local 192.168.137.1 port 55956 connected with 192.168.137.222 port 5001
[  8] local 192.168.137.1 port 55953 connected with 192.168.137.222 port 5001
[ 10] local 192.168.137.1 port 55955 connected with 192.168.137.222 port 5001
[  4] local 192.168.137.1 port 55949 connected with 192.168.137.222 port 5001
[  7] local 192.168.137.1 port 55952 connected with 192.168.137.222 port 5001
[  6] local 192.168.137.1 port 55951 connected with 192.168.137.222 port 5001
[  5] local 192.168.137.1 port 55950 connected with 192.168.137.222 port 5001
[  3] local 192.168.137.1 port 55948 connected with 192.168.137.222 port 5001
[ ID] Interval       Transfer     Bandwidth
[  9]  0.0-10.0 sec  94.2 MBytes  79.0 Mbits/sec
[ 11]  0.0-10.0 sec   103 MBytes  86.3 Mbits/sec
[  8]  0.0-10.0 sec   115 MBytes  96.6 Mbits/sec
[  4]  0.0-10.0 sec   111 MBytes  93.0 Mbits/sec
[  6]  0.0-10.0 sec   122 MBytes   102 Mbits/sec
[  5]  0.0-10.0 sec   106 MBytes  89.1 Mbits/sec
[  3]  0.0-10.0 sec   113 MBytes  94.6 Mbits/sec
[ 12]  0.0-10.0 sec   119 MBytes  99.3 Mbits/sec
[ 10]  0.0-10.0 sec  97.0 MBytes  81.1 Mbits/sec
[  7]  0.0-10.0 sec   126 MBytes   105 Mbits/sec
[SUM]  0.0-10.0 sec  1.08 GBytes   925 Mbits/sec

It just looks so promising for a low power router.. shame :(
EDIT: Banana Pi seems to better network device even than HummingBoards, if You read carefuly at:
http://www.solid-run.com/products/hummingboard/linux-sbc-specifications/

(*) Note that due to internal i.MX6 buses the 1000Mbps interface speed is limited to 470Mbps.

And if You see the screenshots there is no such network bandwidth limit on Banana Pi.

toomeek · Aug 3, 2014, 5:21 PM

I'm doing some progress with support pfSense on Banana Pi.
Current Lubuntu from Lemarker's image runs kernel version 3.4.90.
Latest available development version is 3.16-rc7.

Please see attached screenshots from kernel config menu.
After enabling LPAE new options in Virtualization menu are available for.. ARM arch.
One of them is: "KVM host support for ARM cpus". Sounds like what we need!

Right now my Banana is compiling kernel as .DEB package, will see if it works..

EDIT: keeping You updated.
Right now Banana Pi is running kernel 3.4.90, latest available is 3.4.93. However, KVM on ARM feature has been added at least in 3.9 branch, latest available is 3.16-rc7.
http://kernelnewbies.org/Linux_3.9#head-a914c539db462d70e9119fc436e97351070337d9
Since pfSense requires fully supported KVM to run optimal I have to wait until patches supporting this platform will become available.

Guest · Aug 4, 2014, 2:00 AM

@TooMeeK:

I'm doing some progress with support pfSense on Banana Pi.

[…]

Since pfSense requires fully supported KVM to run optimal I have to wait until patches supporting this platform will become available.

you can't call the result "pfSense", since it's not.
we are interested in the ARM devices market. Netgate is about to pick-up the Beaglebone Black.

toomeek · Sep 2, 2014, 12:19 AM

I have no access to the sources.. if I would have then I could do cross-compilation..
but still don't know is kernel supporting Allwinner SoCs. So far it's not:
http://www.freebsd.org/relnotes/CURRENT/hardware/proc.html
but there is limited information about Allwinner A20:
https://wiki.freebsd.org/FreeBSD/arm
Anyway, Cubieboard's kernel will not work, it probably will boot without networking support since it needs GMAC:
https://wiki.freebsd.org/FreeBSD/arm/Cubieboard

UPDATE:
Cubieboard's 2 image starts on Banana, however stuck at kernel load, there is no error message, just 2 penquens - Linux logo.
So I belive it FreeBSD image built for Cubieboard 2 won't work too.

UPDATE2:
As I see:
2.1.4-RELEASE runs FreeBSD 8.3-RELEASE-p16
while
2.2-ALPHA (amd64) runs FreeBSD 10.0-STABLE
I'll focus on Allwinner config supporting kernel from 10.0.

Where I can find pfSense kernel config file?
According to this:
http://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/kernelconfig-config.html
it should be at /usr/src/sys/amd64/conf
but there is no such directory, also no similar config locations.

UPDATE3:
Found ARM images for Beaglebone boards:
ftp://ftp.freebsd.org/pub/FreeBSD/snapshots/arm/armv6/ISO-IMAGES/10.0

UPDATE 2014-09-02:
I have good news and bad news.
Good is FreeBSD 10.0 runs on Banana Pi (reached login prompt), so it's likely able to run bare-metal pfSense.
Bad news is that there is no support for:

SD card reader as storage for main OS, only for booting
no HDMI output
and the worst: no 1Gbit driver available (GMAC).
So in this stage it's useless for base box for pfSense.

Additionally, there is no ARM-hardware Virtualization support in current Linux kernel (3.4.x), since it's available at least from 3.9.x branch.
So current versions of Bananian, Lubuntu or Raspbian have no capabilities to provide satisfactory performance.

pbk8530 · Oct 7, 2014, 9:49 AM

Hello,

do you see the new upcomming Banana Pi Router board?

With 5 Gbit Ethernet and Wifi + SATA support…

http://www.banana-pi.com/eacp_view.asp?id=64

It would be great to see this board with pfsense support!

KR

Guest · Oct 7, 2014, 1:21 PM

no price.

stephenw10 · Oct 7, 2014, 3:02 PM

Looks interesting though, as Jim says, without a price it's hard to judge.
I can't see how the network ports are connected, via a switch IC SOHO router style?

Steve

Guest · Oct 9, 2014, 3:02 PM

..arround 70 bugs

http://www.golem.de/news/banana-pi-embedded-router-board-1410-109743.html

toomeek · Oct 10, 2014, 4:45 PM

@stephenw10:

via a switch IC SOHO router style?

I belive here is Your answer:
Diese sind über einen Broadcom BCM53125 miteinander verbunden. = These are interconnected by a Broadcom BCM53125.
http://www.broadcom.com/products/Switching/Home-and-Small-Business/BCM53125

Guest · Oct 17, 2014, 2:39 PM

Yes, it's a Broadcom switch, Realtek radio.

Sample price is $70 (FOB China, no case, etc.)

I'm guessing $99-ish by the time there is a case, power supply, microSD card, etc.

The question is if the community is interested at that price (or even a little more).

stephenw10 · Oct 18, 2014, 3:34 PM

I would have thought a market exists for a device that can replace a SOHO style router directly but runs pfSense. There have to be loads of admins out there who would like to use such a device for telecommuters.
It would have to have usefully functioning wifi and something to make setting up a VPN very easy (even easier!). You might have to use some new install type though I guess there are plenty of devices that would meet the requirements for Nano these days.
I'm not sure this is it though. Do you really need an HDMI port on your router?

Steve

Guest · Oct 18, 2014, 4:24 PM

@stephenw10:

… Do you really need an HDMI port on your router?
Steve

Steve,

It's funny that you've mentioned that! Recently I was prospecting SoC specs for a new project and I was marvelling how far the ARM technology came. This A20 is a real powerhouse… but for video and graphics! Almost 3/4 of the die is for video bitmap, compression/decompression handling. 90% of its microcode is for h264.

Why on earth anyone would want to build a network router around A20 is beyond me. The worst part is that all design optimizations and new low level code is for video/graphics. Not a bit of new code was developed for basic, general purpose computational tasks which in the case of products like routers remain the cornerstone.

So, for all intents and purposes running pfSense on an A20 would be like running it on a Zilog Z80 of the 1970's except for the clock speed. And no, even the 8/16/32 bit aspect would be irrelevant! Algorithmics-wise they are the same generation.

How about porting pfSense on CP/M (ok for good measure let's say MP/M) and putting it back on an Intel 8008 (ok a bit more generous, how about 8085)? Any taker? (Ok, I am taking out my Sinclair ZX-80 and dusting it off ;D )

It's amazing how people do (crazy) things just because hardware is now dirt cheap... :-X

Halea

stephenw10 · Oct 18, 2014, 6:12 PM

Ha! If you get an IP stack running running on your ZX80 let me know and I'll dust off my ZX81. Pretty sure I have the RAM expansion pack here too so it should be no problem!
Your comments are interesting though, I guess the phone/tablet market is driving a lot of IC development. There are plenty of specific 'network processors' that are presumably far more suitable. Are they more expensive though.
I wouldn't want to discourage anyone from trying to make a thing run some OS it didn't before. I've spent many happy hours failing to get OpenWRT running on various hardware and leant a lot along the way. I had to laugh at that guy who got Doom running on his printer. :)

Don't let us put you off TooMeeK. Also apologies for somewhat hijacking your thread.

Steve

Guest · Oct 21, 2014, 7:29 PM

@stephenw10:

I would have thought a market exists for a device that can replace a SOHO style router directly but runs pfSense. There have to be loads of admins out there who would like to use such a device for telecommuters.
It would have to have usefully functioning wifi and something to make setting up a VPN very easy (even easier!). You might have to use some new install type though I guess there are plenty of devices that would meet the requirements for Nano these days.
I'm not sure this is it though. Do you really need an HDMI port on your router?

Steve

No, you don't need (nor want) an HDMI port on your router. The issue is that most of the cost of having that port (and the audio port, and camera port) have already been paid for (all the gates are already in the SoC). The BPI router makes me grouchy because it has a 10/100 port and a 10/100/1000 port (guess which one is attached to the switch!). Even, then, I've ordered a couple.

We had the Hackathon last week, and have most of pfSense working on the Beaglebone Black. It's not a big stretch to make it work on the BPI router. There are a lot of reasons for my push onto FreeBSD 10 (pfSense 2.2 will be based on FreeBSD 10.1-RELEASE), and getting to more platforms is one.

Guest · Oct 21, 2014, 7:55 PM

@gonzopancho:

… have most of pfSense working on the Beaglebone Black. ...

That's awesome news. I've been running RasPBX on BBB at home for a couple of months now. Prior to that it was an Raspberry Pi for more than a year (maybe two).
I would love to put my hands on a cookbook to run pfSense on BBB. Any attempts I made with a couple of colleagues to run it on BBB miserably failed although at one point we got FreeBSD to work for a while (then we messed something and we couldn't go back on our steps).
So, is there any notes you can share?
Halea

Guest · Oct 28, 2014, 4:49 AM

when it's ready

toomeek · Nov 26, 2014, 7:24 PM

Erm.. well.. long time since first post.
I see You're digging my topic :>

Anyway Beaglebone Black runs 100Mbit Ethernet, while BPi is 1Gbit which can be saturated easily,
but there is no GMAC driver for it in FreeBSD head yet.
My dream would be working tagged VLANs over GMAC driver that can handle at least 250Mbit up/down speeds on this small device..

FlashPan · Nov 26, 2014, 9:16 PM

I just found this:

5 port banana

http://www.aliexpress.com/store/product/Newest-arrive-BPI-R1-Opensource-Router-Original-Banana-pi/302756_2045261784.html