Motherboard ideas

dotdash

Ok, some tests. My methodology with this was to grab two machines- approx C1.8, 512mb, and drop in an Intel Pro/1000 GT nic. Loaded FreeBSD 6.2 on them, pretty much stock, and installed iperf 1.7 from source. All tests were done with a simple iperf -s on the target machine and iperf -c x.x.x.x on the client. The client was behind the LAN and the server behind the WAN. Results varied a bit, so I ran three tests and averaged.
Test boxes via crossover cable 472 Mbits/sec
Test boxes via GB switch 478 Mbits/sec

Tyan 3095 running 1.2 beta1. em0=lan em1=wan crossover cables to boxes.
Standard config, except for changing LAN and WAN addresses.
398 Mbits/sec. CPU didn't go above 4% during the tests. Disabled the firewall and re-tested, no change in speed.

While I had the lab setup, I did a couple of other tests just for grins. I substituted a Cel550 test box
for the 3095.
With onboard fxp0 and add-in fxp1: 92.5 Mbits/sec. CPU got up to around 30%
Two Netgear cards, sis0 and sis1: 93 Mbits/sec. CPU usage was similar but lasted 2-3 times longer.
Two cheap realtek cards, rl0 and rl1:78.7 Mbits/sec. CPU peaked for slightly longer than the sis.

Standard disclaimer: Your results may vary, my test-ology is not perfect and I could have screwed up a number of things. I did not account for various factors including: moon phase, ambient temperature, humidity, solar flares, backround music, etc.

fwuser07

Thanks for testing. So the performance of the 3095 is not bad.
And I think the throughput will be even a bit higher if the two machines generating the traffic would have the lan interface also connected via PCIe and not via slow 32bit PCI bus.

Interessting is the very low cpu usage. Have you also monitored the momery usage?

dotdash

Box was headless and connected via crossover, so I wasn't watching it live. I plugged a laptop into LAN after the test to check the RRD graphs to see what the CPU did.

fwuser07

Ok no problem. I will order the same board soon and when I've done some performance tests I'll post them.

dotdash

Here are the things you should know before getting the board:

Update the BIOS. Initial version has flakey fan sensor readings.
Board needs an EPS-12V PSU
Board does not have standard P4 heatsink mounts, you have to get a Pentium M heatsink.

fwuser07

Ok thanks for these hints.

tedced

I also need a system with a lot of Nics.
2 WAN, 1 LAN, 1 VOIPLan, 1 DMZ, 1 CARP, 1 WIFI,= 7 Nics

I am planning to use the Intel INTEL PRO/1000 PT PCI-E adapter as well as the INTEL PRO/1000MT PCI adapter.
I've considered the two Asus boards below, mostly because of the high number of PCI-E slots.
I only buy Asus boards…

Asus M2N-SLI Deluxe (AMD AM2) 4 pci-E and 3 pci + 2 onboard gbe
and the
Asus P5N-SLI (Intel LGA775) 5 pci-E and 2 pci + 1 onboard gbe

I bought the m2n but havn't been able to get the onboard nics to work yet.
I've posted that tread here: http://forum.pfsense.org/index.php/topic,4902.0.html

fwuser07

I can't help you with your nvidia nics but my impression is that you should use intel nics for a stable environment to have less trouble.

And in my opinion it would be better to work with vlans instead of using so many nics for your internal lans. To do so, your switches must also have vlan support, of cource.

dotdash

I have not heard good things about Nvidia chipset NICs and FreeBSD. Asus generally uses Broadcom on their server class motherboards, I think Broadcom NICs would be preferable to the Nvidia ones. I was considering an ASUS board myself, but couldn't find a small form factor board with Intel nics… Also, consider alternatives to using a PCI or PCIe slot for everything. For example, the Tyan board I was talking about earlier only has 1 PCI and 1 PCIe slot, but I could do everything you described using the 3 onboard NICs, an add-in PCI and PCIe nic, the onboard firewire for CARP, and a mini-pci wireless card. For WANs, you could use a cheap dual-port fxp card in one PCI slot, and for faster interfaces, a dual-port PCIe Intel card. So you might be able to do this without needing five PCIe slots.

fwuser07

@dotdash: I ordered all componentes except for the heatsink. Please tell me what heatsink you used in yout 1U case.

dotdash

It was a Dynatron I31 active cooler. They also have a passive heat sink I51 that fits.

tedced

… the onboard firewire for CARP...

can you use firewire for CARP?

sai

ethernet over firewire doesn't seem to work with pfSense. the fwe driver has been taken out of the kernel some months ago

dotdash

I got CARP running over firewire using fwip, not fwe (which seems to be non-standard).
I have some notes here:
http://forum.pfsense.org/index.php/topic,3604.0.html