Pfsense strange issue Flooding the network when Multi-Wan
-
i faced strange issue when i install pfsense with 2 GW and 1 LAN , to able to give LB between the 2 GW .
sometimes when i start to edit the FW rules for any of the GWs interfaces i face huge traffic going to the SW from the Pfsense server and it flood the network . and i cant ping or do any analyze with any tool over the network , and in seconds the SW get fully flooded . i can see that later with prtg ( i see that all interfaces on the SW spike to the max and never get down till i remove pfsesne server )
I installed latest version of pfsense on vmware esxi 5.5 u1
sometimes this issue happen when installing squid with the same above scenario. and i cant fix it unless i reinstall pfsense .
i dont know what really happen , i will remove that server ( esxi ) from the network then reboot it . then fast logging to it and then disable the cards on the pfsense before it boot up . then access it using the esxi console and access pfsense shell .
but what commands can i use to debug that issue?
Thanks
-
Sounds like you're somehow creating a neverending loop of traffic. What type of WANs, static, DHCP, PPPoE, …? What is your network like in general? How is your gateway group configured?
It's difficult, but theoretically possible, to create a never ending routing loop with route-to (firewall rules specifying a gateway or gateway group) because it doesn't decrement the TTL. I'm guessing you're somehow doing something similar to that.
but what commands can i use to debug that issue?
Things I would check, via console when it's broken:
pfctl -si
primarily interested in State Table, current entries.
State Table Total Rate current entries 375
To see how many active sessions there are. Knowing whether it's a small or large number of connections helps troubleshoot from there.
pftop shows some potentially useful stats, and may be enough to find the specific traffic in question.
pfctl -ss | more
to dump the state table, page by page. Glancing through a few pages may help.
And most useful, tcpdump. Grab some traffic from each interface to see what it's seeing. Pay attention to the source and destination MAC addresses as well.
tcpdump -nei em0 -c 1000 tcpdump -nei em1 -c 1000 tcpdump -nei em2 -c 1000
Replace emX with vmxX or whatever your NICs are. Those will capture 1000 packets (good idea to put a count when capturing a flood), disable name resolution so it's easier to read and doesn't have delays, and show layer 2 info (including src and dst MACs).
-
Thanks for your reply
well both GW are configured with static IPs
Group configured simply as tier 1 both and packet loss or high latency , then lan configured with that new grouped GW
this is a test network so it is very simple no vlans . just Normal Manged Switches without any network configurations.
both GWs are aDSL 20 MB
all SWs are Gb speed per each port
today i reinstalled pfsense and just configured the group and nothing more ( didnot install any packages ) and didnot create any FW rules , and it works fine .
btw if i didnot add any packages or didnot change any FW rules , it will work without any issue .
once i start creating FW rules or install packages , it will loopi will try to recreate the same scenario and try to get more results.
i will update here later