Filtering a SPAN session from a switch
-
Hi, I was considering if pfSense can be used in my setup using (maybe) transparent firewalling or other features.
I have a big switch (Cat 6500) and a device that must analyse traffic flowing through it. The problem is the device only has 1 Gbit ports and the traffic is much higher on the VLAN I would like to monitor (cannot split VLANs further than that).
So basically I would like a setup where I receive the SPAN session from the switch on several Etherchannel/LACP/Plain interfaces*, filter them, and put the filtered traffic on the single 1gbit output interface (traffic will be unidirectional, being the source a mirror port).
The obvious answer "put ACLs on the SPAN session on the Cat" is not viable, since the Cat dies trying to filter the amount of traffic().
Also I cannot just put the output on a Gigabit interface and plug it to the device, because there are lots of dropped packets and the analysis does not work. If filtering the not interesting hosts on the source, the 1 gbit traffic will be enough for the analysis.
To sum it up I need a multi gigabit input -> ACLS -> output on a single interface. A sort of "filtered bridging".
(*) When used as an output virtual interface, aggregated interface should not use any aggregation protocol according to Cisco, so it seems they behave just like a plain bunch of interfaces where mirrored traffic is spread. Maybe this helps a bit.
() Yes, I suspect I'll need some serious computing power on my pfSense.Thank you for any suggestion.
-
Dude if your switch dies trying to put a ACL on your span.. Sounds like you need heftier switch;)
If your moving more than 1 gig of traffic that you need to work with… Use 10ge.. So bigger Analysis box as well.. This attempt at some sort of MacGyver hack with lagg ports is just behind nuts!!
-
Dude if your switch dies trying to put a ACL on your span.. Sounds like you need heftier switch;)
If your moving more than 1 gig of traffic that you need to work with… Use 10ge.. So bigger Analysis box as well.. This attempt at some sort of MacGyver hack with lagg ports is just behind nuts!!
I can surely do what you said, just lack half a million Euro to revamp my architecture while blocking production activities. Do you have some spare change ;) ?
I know I'm looking for an hack, and that is why I'm posting here (and yes, I love McGyver :)). In your view I can also replace my pfSense boxes with $BIGBRAND hardware doing transparent filtering on 10g fibers and solve my challenge.
I'm the first to suspect that if a device with ASICs devoted to filtering crashes under this load, the same when done using CPU power will not work very well but hey, that's the sort of things that make IT world interesting.
BTW moving SPAN sessions over aggregated ports is a perfectly valid solution in Cisco parlance and not a stupid one when not using aggregation protocols like LACP. If you have a 1.8 gbit traffic to send to your appliance you can use 2 ports on a low cost 1 gbit module, instead of buying a uber costly 10ge module and 10ge appliance.
For the sake of experimenting, cut all the numbers by 10: can this design work? I'm not sure where to create bridges between what. -
So just to clarify, you've got more than 1 Gbps of traffic, in a SPAN/RSPAN, being mirrored to a 1GBE port?
Even if filtering on the receive side to send to the analysis host in question, you're still going to be missing packets/frames/whatever you're spanning.
In theory, yes. You could filter non interesting host traffic before its sent to the analysis box. In practice, you might as well not even include the pfsense box, and on the (assuming linux) analysis box just tcpdump the traffic and do the filtering you need, then import the packet cap file and run analysis.
IMHO, the better option would be to filter down as much as possible on the cisco side. Remove some of the vlans that you've got set up in your RSPAN, in order to reduce the need for the ACLs, or atleast to the point where the ACLs are able to tackle the problem youre throwing at them.
good luck with the span and let us know how it goes! You're right though, this is what makes IT interesting ;)
-
So just to clarify, you've got more than 1 Gbps of traffic, in a SPAN/RSPAN, being mirrored to a 1GBE port?
Even if filtering on the receive side to send to the analysis host in question, you're still going to be missing packets/frames/whatever you're spanning.
In theory, yes. You could filter non interesting host traffic before its sent to the analysis box. In practice, you might as well not even include the pfsense box, and on the (assuming linux) analysis box just tcpdump the traffic and do the filtering you need, then import the packet cap file and run analysis.
IMHO, the better option would be to filter down as much as possible on the cisco side. Remove some of the vlans that you've got set up in your RSPAN, in order to reduce the need for the ACLs, or atleast to the point where the ACLs are able to tackle the problem youre throwing at them.
good luck with the span and let us know how it goes! You're right though, this is what makes IT interesting ;)
I have more then 1 gbit source traffic but using a portchannel I can direct it to several 1 gbit (R)SPAN ports, so nothing gets lost on the switch side. The problem is the receiving end that only has 1 network adapter at 1 gbit. I cannot dump and analyse the traffic, it is in real time. And as I wrote I cannot split the traffic anymore, it is a single VLAN with >1gbit.
Inside the single VLAN I know there is some traffic that I can filter out to reduce to <1 Gbit. This is where the pfSense comes in. Doing the same on the switch with VACLS just crashes it.
So I'm basically outsourcing the VLAN filtering to the pfSense device. -
My apologies, I misunderstood. You can accomplish this by setting some routing rules so the traffic coming in is routed through, and filter the traffic with firewall rules. The setup would have pfsense plugged into your span port, and your analyzing box plugged into pfsense. Assuming the traffic coming in is from the same subnet this should be pretty easy.
-
My apologies, I misunderstood. You can accomplish this by setting some routing rules so the traffic coming in is routed through, and filter the traffic with firewall rules. The setup would have pfsense plugged into your span port, and your analyzing box plugged into pfsense. Assuming the traffic coming in is from the same subnet this should be pretty easy.
This is where the thinsgs gets tricky and why I asked about the transparent firewall thing. The SPAN ports contains traffic that is basically duplicated production traffic, it is not the kind of traffic that flows through a firewall because it is routed to it. Also subnets are different. Ti give you an example in the mirrored VLAN I have 3 servers A,B and C. Hosts are in the 172.30.2.0/24 subnet (maybe in different subnets also but we'll leave this for another moment) and talk to C.
So I see 3-way handshakes and traffic between the machines, in both directions. My goal is ro receive all of this traffic and block any packet for server B (for example), so that the analysis machines only gets A<->C. Traffic will continue to flow between A<->B in the production VLAN, but the analysis machine will not receive it. Being B an high volume machine, removing that will reduce traffic that goes to the analysis machine via the 1 gbit interface. -
Maybe this Paint masterpiece helps
-
My apologies, I misunderstood. You can accomplish this by setting some routing rules so the traffic coming in is routed through, and filter the traffic with firewall rules. The setup would have pfsense plugged into your span port, and your analyzing box plugged into pfsense. Assuming the traffic coming in is from the same subnet this should be pretty easy.
This is where the thinsgs gets tricky and why I asked about the transparent firewall thing. The SPAN ports contains traffic that is basically duplicated production traffic, it is not the kind of traffic that flows through a firewall because it is routed to it. Also subnets are different. Ti give you an example in the mirrored VLAN I have 3 servers A,B and C. Hosts are in the 172.30.2.0/24 subnet (maybe in different subnets also but we'll leave this for another moment) and talk to C.
So I see 3-way handshakes and traffic between the machines, in both directions. My goal is ro receive all of this traffic and block any packet for server B (for example), so that the analysis machines only gets A<->C. Traffic will continue to flow between A<->B in the production VLAN, but the analysis machine will not receive it. Being B an high volume machine, removing that will reduce traffic that goes to the analysis machine via the 1 gbit interface.You set pfsense to route it. for example, set the WAN interface to go to span and give it a random ip outside of your private range. set a static route to forward traffic to LAN, and have the route encompass the network range for the equipment being tested (next hop to 172.30.2.0/24 is inside LAN), and have your analysis box on the lan port. Set firewall rules to pass ALL traffic on wan, and above that set filter rules to weed out HOST B. Also disable 'block bogon networks'.
this would essentially turn pfsense mostly into a router. PFsense thinks its forwarding traffic correctly. you could even go firther to give analysis box an ip on 172.16.1.0/30 subnet, and pfsense the other. then set that firewall rule to route traffic MATCHING the any rule to gateway "Analysis" and just define the analysis box as an upstream gateway. this would open up policy based routing for example.
-
What paint masterpiece?
I don't know how you think a layer 3 CPU will do better at filtering than a layer 2 switch.
If nothing else you should try filtering though another switch. Maybe something up to the task-at-hand.
-
What paint masterpiece?
I don't know how you think a layer 3 CPU will do better at filtering than a layer 2 switch.
If nothing else you should try filtering though another switch. Maybe something up to the task-at-hand.
he already said it wasnt in budget, hence abandoning my argument of replacing the cisco switch with a newer/more powerful model.
-
Maybe this Paint masterpiece helps
i dont have a fancy picture but youre going to do something like this with the topology:
hosts A/B/C in vlan XYZ get SPAN to 2gbps port -> pfsense wan (or opt interface, doesnt matter) set with static routes -> PFSense LAN connected to anaylisis box.
-
He doesn't need to replace the 6500. Just get something that can trim the traffic he needs trimmed. I mean if a $35 3750 off ebay can't do it, he's going to spend a helluva lot more on Xeons to get pfSense to do it.
-
He doesn't need to replace the 6500. Just get something that can trim the traffic he needs trimmed. I mean if a $35 3750 off ebay can't do it, he's going to spend a helluva lot more on Xeons to get pfSense to do it.
depends on what he has available. he expects his current pfsense hardware can handle it, so why not try it? My current setup is handling 10gbps throughput just fine, and im not sure an old 65k from cisco would be up to the task given the filterin requirements i have. thats besides the point though, point being that its possible he already has hardware able to do it except the said hardware isnt a switch.
EDIT:
wouldnt 166Mpps be well within most pfsense boxes anyway? thats the maximum throughput hes talking about filtering… -
Sigh
-
-
166Mpps. No.
-
so youre saying pfsense CANT handle theoretical throughput of 2gbps?
166 million packets isnt much, even for a firewall on non-asic hardware.
EDIT:
the 'medium business' hardware package being sold on the pfsense webstore can handle that just fine. and assuming its the 2015 cpu id say most pfsense boxes being built in a corporate test lab are going to be that age if not newer. OP is clearly not in a home lab. -
166Mpps at 64-byte packets is enough to saturate about 85 gigabit/sec, bro.
Oh. I get it. You think a packet is a byte. We will have to come to terms so we can communicate effectively. Might I suggest Comer.
-
166Mpps at 64-byte packets is enough to saturate about 85 gigabit/sec, bro.
Oh. I get it. You think a packet is a byte. We will have to come to terms so we can communicate effectively. Might I suggest Comer.
no, a packet isnt a bit. that would be a byte.
2gbps / 8 (bit to byte conversion) = 250MBps (bytes).
MTU of 1500 BYTES (since thats the typical mtu, but can vary.) 250,000,000 / 1500 = 166,666 packets. since most packets arent full, and average 576 bytes, that would be 434,027 packets per second. Since thats the theoretical throughput, you can actually adjust that number depending on actual averages. but if you wanted to run it your way and misinterpret a packet to be a byte, that would be 250Mpps ON A SINGLE LINK. Correlating that with the iperf tests i literally just ran 1gbps at 8% cpu utilization, and im not on top end equipment. So I'd say its fair to say a pfsense install in a corporate lab environment is going to be capable of pushing 166Mpps across its 'backplane'. Since, you know, packets per second are theoretical measurements used to descript the capability of a backplane, and not a sing link, bro.
i do appreciate the snide remarks though. keeps me on my toes :)
