[ Show your pfSenses! ] - Thread - (bandwidth warning!)

fLoo

I'd like to raise a pfSense show-off thread. I'm gonna start off with my just build up replacement for my virtual pfSense (moved from ESXi back to hardware)

Intel D2500CC (2x 1,8 Ghz)
4 GB Ram
3x Gbit NIC
128 GB SSD
75W power supply
pfSense 2.1 (pfSense-memstick-2.1-BETA1-amd64-20121221-0526.img.gz)

extide

Ok, here is my setup:

Cable Modem (Cisco/Linksys DPC-3008) sitting on top of machine.

Hardware:
Intel Core i3 3220 - 22nm Ivy Bridge - Dual Core 3.3Ghz - HT Disabled
ASRock Z77 Extreme 3
2x2GB DDR3 1333
64GB OCZ Summit SSD (Samsung Controller)
2x Intel PCI-E Gigabit Ethernet Adapters
Plextor DVD-RW
300W Seasonic 80+ Bronze PSU
Generic Case

Guts:

Below is a shot of the whole setup.
NOTE: The large Compaq server (8-way P3 Xeon) AND the Disk Array sittong on top of it ARE NOT IN USE. They are functioning ONLY AS A SHELF!

NetGear GS108P PoE Switch
2x Dlink DAP-2553 Wifi AP's using PoE from the GS108P (one for 2.4Ghz, the other for 5Ghz)
And the grey rectangle with green sticker on it sitting next to the monitor is the DirecTV DECA adapter. (Connects the DirecTV DECA network, which is ethernet over COAX that co-exists with the sat signals, to the rest of my network.) This way my DVR's all grab IP's from my pfSense box and have full internet access.

The monitor and keyboard are for when I need to manually go in there and work on something, which is pretty rare. :)

fLoo

Oh damn - you want the people totally show off? I see - lets continue with me:

My complete setup (2 Wi-Fi-APs missing here)

Cable Modem (Cisco EPC-3212)
TP-WR1043ND (Public AP - Routing traffic through vpntunnel.se)

You can see (or its hidden):

24 Port Management Switch
TP-WR1043ND (SamKnows bandwidth measurement)
RIPE Atlas node (Network measurement)
Raspberry PI + DVB-C Stick to monitor network segment usage of my local internet provider
mini-LCD Monitor to monitor network statistics

ESXi 5.1 WhiteBox (Core i5, 16 GB Ram / 2x 3 TB + 2x 64 GB SSD)
12 TB Raid-5 Firewire-NAS (hidden on the right)
Yes i need to clean the ESXi :)

extide

Hey, I just posted my router ;) I have a Ubuntu Server running ZFS, a windows server, and a dedicated linux folding@home box as well. :) I can put more pics later.

Tell me more about the "Raspberry PI + DVB-C Stick to monitor network segment usage of my local internet provider" that is pretty cool… What kinda stats can you see?

fLoo

@extide:

Hey, I just posted my router ;) I have a Ubuntu Server running ZFS, a windows server, and a dedicated linux folding@home box as well. :) I can put more pics later.

Tell me more about the "Raspberry PI + DVB-C Stick to monitor network segment usage of my local internet provider" that is pretty cool… What kinda stats can you see?

Because i'm using internet via a cable modem you can monitor the network usage of your network segment. The reason is because cable is a shared medium and every single customer in your segment can see every single bit (multicast), although its encrypted. So what u can do is the following:

Take a DVB-C-Stick (i prefer sundtek.de because of its full linux support), then u tune your dvb-c stick to the same frequency as your cable modem (in europe thats usally 546 - 602 mhz), toggle modulation and use dvbsnoop to read off the statistics of the frequency-channel.

Result:

extide

Yes, I understand how cablemodems and DOCSIS networks work, but I had no idea you could get a device to monitor the stuff like that! I am totally going to get one of those and set it up on one of my linux boxes!

Cool, looks like I can get a PCI-E one also. I may have to rig up some stuff to read the current DS/US channels from the cable modem and then feed that to dvbsnoop to get the infos. How are you making that graph? Are you manually doing it with RRDTOOL or is there some software out there for doing this specifically?

Thanks for the info BTW!

fLoo

@extide:

Yes, I understand how cablemodems and DOCSIS networks work, but I had no idea you could get a device to monitor the stuff like that! I am totally going to get one of those and set it up on one of my linux boxes!

Cool, waiting for your stats. If you need assitance (complete ready scripts for Cisco-Modems + RRDTools) just msg me. Oh and - dont forget to monitor your signal to noise :)

extide

Ok, sweet, I have a Cisco DPC 3008 so hopefully I wont need to modify the scripts much to grab the stats. It's funny, in another thread I posted on here today I asked if there was some software to grab this info from the cable modem and insert it into a DB so you can graph it over time. So, yes, I would love those scripts, thanks!

So, are you just using a coax splitter, standard -3.5dB one? Going to the DVB-C stick and the other side to the modem ?

fLoo

No wait - its a Coax splitter for your TV-Signal, not the one for the modem. You should have a multimedia-splitter already at your wall (1x modem, 1x tv, 1x radio) you split the tv one! And yes, its a standarf 3.5dB splitter, so you should keep care of your stats. Maybe some adjustment of your cablenetwork is needed (increase the power on your cable amplifier and use an -3,5 attenuator for your cable modem).
For me it worked just out of the box, without modifying anything, because its just - 3.5dB :)

IF YOU MODIFY YOUR AMPLIFIER - KEEP AN EYE ON THE SLOPE :)

So u got

PIN_
–---O-----
----/------
LOG___TV

Regarding your question: Using RRDTool with complete custom (self written) templates and also own scripts to feed the RRD (which i can supply).

Photo:

gderf

Cray XD1 with 11,000 interfaces running pfsense 2.1 snapshot.

cmb

Good idea for a thread. We're going to gather pictures from a variety of threads like these in the future and create some kind of micro-site showing off people's deployments. In the mean time, might as well get another thread going. :)

Here's our primary colocation facility, where this site and most of our others run, as well as the snapshot and release build servers. The firewalls are virtual in ESX, a HA pair with primary on one ESX server and secondary on another.

fLoo

Sweeeeeeet :o Thats what i'm trying to accomplish but i still need more money (although my setup is already too fat for home networking) ;)

stephenw10

Well I can't pass up this opportunity.

Yes, I have a problem. I'm trying to cut down. ;)

Steve

cmb

@stephenw10:

Well I can't pass up this opportunity.

Yes, I have a problem. I'm trying to cut down. ;)

haha Those aren't running the Watchguard software I presume? ;D

stephenw10

I'd love to say none of them are but the X-edge boxes at the top are Arm based and I haven't managed to hack a workable OpenWRT image onto them. Yet. ;)

Steve

kilthro

Well I wish I had a fancy setup. Here is a a pic of my pfSense box.

SUPERMICRO SYS-5015A-EHF-D525 1U Intel Atom D525 Dual Gigabit LAN w/ IPMI Server
4 gig ram
Intel EXPI9402PT 10/ 100/ 1000Mbps PCI-Express PRO/1000 PT Dual Port Server Adapter
two 250gig WD black 2.5 inch drives mirrored
1500AV UPS (not shown) for 1.5 hours of backup time.
And for the heck of it a Crystalfontz display. :-D
Have gigabit wireless access point as well hooked up. (not shown)

Don't make fun of my makeshift fan bracket (zip ties in vent). lol The system is "passively cooled" and I didn't like the 65°C so i put a case fan in and now its 40°C. Not exactly where i would want it but much better.

extide

@fLoo:

No wait - its a Coax splitter for your TV-Signal, not the one for the modem. You should have a multimedia-splitter already at your wall (1x modem, 1x tv, 1x radio) you split the tv one! And yes, its a standarf 3.5dB splitter, so you should keep care of your stats. Maybe some adjustment of your cablenetwork is needed (increase the power on your cable amplifier and use an -3,5 attenuator for your cable modem).
For me it worked just out of the box, without modifying anything, because its just - 3.5dB :)

IF YOU MODIFY YOUR AMPLIFIER - KEEP AN EYE ON THE SLOPE :)

So u got

PIN_
–---O-----
----/------
LOG___TV

Regarding your question: Using RRDTool with complete custom (self written) templates and also own scripts to feed the RRD (which i can supply).

Photo:

Heh, sounds like you guys have a slightly different cable setup over there. I have no cable amplifier, and I do not use the cable for my TV's (I use DirecTV for TV), so my coax is one single lin from the drop to the modem, no splitters no amps, nothing, its just raw right into it. So, yeah, I will be using a little simple 2-way slitter. But, my wife will be pissed if I buy the DVB-C card/stick right now so it will be a little bit. I WILL be working on making a scraper for my modem stat page, and then use that data to feed into the RRD graphs, though.

syro

Colocation setup:

pfSense 2.1-DEV based cluster running on two TYAN Transport servers ;D

Basic information:

1x CPU: Intel(R) Pentium(R) 4 CPU 3.00GHz (2992.51-MHz 686-class CPU)
2x Intel(R) PRO/1000 Legacy Network Connection (1.0.3)
1x 3Com 3c905B-FX/SC Fast Etherlink XL (PFSYNC, XMLRPC via Fiber Optical)
real memory = 4294967296 (4096 MB)

The switching stack was built using two NETGEAR GS724TS (stacked via HDMI). The firmware is a pure pain but when tamed it somehow gets the job done. It's the cheapest stack I found and using LACP accross multiple physical units is definitely a big plus! Our provider runs LACP for our uplink as well so even the wan connection survives a dead unit :)

The entire setup is considered stable and runs for almost 13 months now w/o any service interruptions (e.g. crashes or similar).

@Home:

1x PCEngines Alix 2D13, 4GB CF
1x HP Switch 1900-8G (formely known as 3com OfficeConnect 3CDSG8)
1x Linksys WRT54G running DD-WRT (Kamikaze)
1x WRT54G running DD-WRT (some release)
1x APC SmartUPS 500VA (in the back)

stephenw10

Nice. :)

@syro:

1x Linksys WRT54G running DD-WRT (Kamikaze)

Isn't 'Kamikaze' an OpenWRT code name?

Steve

syro

@stephenw10:

Nice. :)

@syro:

1x Linksys WRT54G running DD-WRT (Kamikaze)

Isn't 'Kamikaze' an OpenWRT code name?

Steve

Yeah, damn straight ;)

Fixed it, thanks!