Pfsense strange issue Flooding the network when Multi-Wan



  • i faced strange issue when i install pfsense with 2 GW and 1 LAN , to able to give LB between the 2 GW .

    sometimes when i start to edit the FW rules for any of the GWs interfaces i face huge traffic going to the SW from the Pfsense server  and it flood the network . and i cant  ping or do any analyze with any tool  over the network , and in seconds the SW get fully flooded .  i can see that later with prtg ( i see that all interfaces on the SW spike to the max and never get down till i remove pfsesne server )

    I installed latest version of pfsense on vmware esxi 5.5 u1

    sometimes this issue happen when installing squid with the same above scenario.  and i cant fix it unless i reinstall pfsense .

    i dont know what really happen , i will remove that server ( esxi ) from the network then  reboot it . then fast logging to it and then disable the cards on the pfsense before it boot up . then access it using the esxi console and access pfsense shell .

    but what commands can i use to debug that issue?

    Thanks



  • Sounds like you're somehow creating a neverending loop of traffic. What type of WANs, static, DHCP, PPPoE, …? What is your network like in general? How is your gateway group configured?

    It's difficult, but theoretically possible, to create a never ending routing loop with route-to (firewall rules specifying a gateway or gateway group) because it doesn't decrement the TTL. I'm guessing you're somehow doing something similar to that.

    @Electrone:

    but what commands can i use to debug that issue?

    Things I would check, via console when it's broken:

    pfctl -si 
    

    primarily interested in State Table, current entries.

    State Table                          Total             Rate
      current entries                      375     
    

    To see how many active sessions there are. Knowing whether it's a small or large number of connections helps troubleshoot from there.

    pftop shows some potentially useful stats, and may be enough to find the specific traffic in question.

    pfctl -ss | more 
    

    to dump the state table, page by page. Glancing through a few pages may help.

    And most useful, tcpdump. Grab some traffic from each interface to see what it's seeing. Pay attention to the source and destination MAC addresses as well.

    tcpdump -nei em0 -c 1000 
    tcpdump -nei em1 -c 1000
    tcpdump -nei em2 -c 1000  
    

    Replace emX with vmxX or whatever your NICs are. Those will capture 1000 packets (good idea to put a count when capturing a flood), disable name resolution so it's easier to read and doesn't have delays, and show layer 2 info (including src and dst MACs).



  • Thanks for your reply

    well both GW are configured with static IPs

    Group configured simply  as tier 1 both and packet loss or high latency  , then lan configured with that new grouped GW

    this is a test network so it is very  simple no vlans . just Normal Manged Switches without any network configurations.

    both GWs are aDSL 20 MB

    all SWs are Gb speed per each port

    today i reinstalled  pfsense and just configured the group and nothing more ( didnot install any packages ) and didnot create any FW rules , and it works fine .

    btw if i didnot add any packages or didnot change any FW rules , it will work without any issue .
    once i start creating FW rules or install packages , it will loop

    i will try to recreate the same scenario and try to get more results.

    i will update here later


Log in to reply