Found a similar issue here: https://github.com/opnsense/apinger/issues/3 The problem I'm seeing is exactly the same as this post - " reetp commented on Aug 10 " I also found more wan failover / recovery issues on the day I posted this.  I didn't save the links though.  No solutions there anyway.  Maybe old/known problem. Also described here.  Odd that DHCP on the WAN causes problems, but static wan ip doesn't.  http://serverfault.com/questions/611664/pfsense-dropping-gateway-interface-randomly

Yes, packet capture led to a not-yet-powered-off AP that for some reason still was cabled to the network and had the same IP as the gateway. From time to time the IP collision was won by that AP and the redirects began. All hail Wireshark, but i had to wait until today to get time to do some real packet capture… Thanks for the help.

I forgot to mention, but of course this is enabled too: [image: pEZhVS.jpg]

Not sure what your trying to route? So on your lan interface rules set the gateway you want these clients to use.  On your opt2 (lan2) rules set specific gateway to use. You going to want to have a rule above these rules that call specific gateway to allow traffic between your lan and opt2 network - if you want to allow this traffic. Also your going to want to make sure your outbound nat is setup correctly.. So I assume these pfsense wan address are made up.. since one is rfc1918 space and the other is public space at 11.11.11, etc.  So are both public or is one actually 10.10.10 and you made up the other one that is public to hide it?

Oh thanks. That's exactly what I did: I used different monitoring IPs for both gateways.

Can't comment. My ISP provides "Internet services needed by TV or anything connected to it" through IPTV's gateway.. Browse internet through TV settop box or whatever..

If you don't have a switch that supports vlans then get one.  Create a new vlan for the wireless and set your unifi AP to the vlan you set on the firewall and switch.  Create a gateway group for the 2 ATT links and set the gateway for that vlan to the group for ATT. You will need to ad a rule or maybe 2 to allow traffic from the wifi vlan to get to the other respective networks before the primary rule with the gateway group

Thanx Mark for your reply, the problem was in the gateway menu, i should have put a value , other than "0" in "data payload" field. But now I've got another question: why cant i acces web gui from wan using the second ip (static) ? It works only with the first wan ip (pppoe) !

huh??? Makes no sense…  So you want to replace your current router(s) with pfsense?  These routers you have at 172.28.64.2 and another one at 192.168.1.1, I assume you have different ISPs??  Why not just use pfsense with dual wan and use 1 network behind and then let pfsense load balance or failover or use policy routing to determine what host uses what wan connection based up whatever criteria you want to use, etc.

i've never watched anything pfSense related on youtube, so i don't know

Can anyone tell me how to port this feature in FreeBSD 10.3? For me as soon as I use route-to in the PF ruleset, it breaks the pfil  ordering. For me, the input chain is ipfw –> pfil and output chain is pfil --> ipfw. When I do not use the route-to rule, everything works fine, however as soon as I use the route-to feature in PF, it breaks the order.

you sure seem to have a lot of ports for no real reason ;)  And like to use them up via lagg that seem to just be there to use up ports not for any sort of real load balancing or failover need, etc. What routing are you doing that you need downstream layer 3?  I doubt your pfsense box can route/firewall at 10ge - but what what sort of traffic would be going between segments that would need/use 10ge? Why can you not just use your pfsense box as your router/firewall between all your segments and just use a switch be it the juniper or the other in layer 2 mode?  If you want line speed between say clients and your servers that are on different segments at 10ge then sure your going to need something that can do that as downstream router. I love the 10ge and am a bit jealous to be sure.. But can you even leverage it?  What sort of speeds can you get out of your storage?

@DanC: For switches - I'm using Ubiquiti Unifi 48 port switches.  It seems they do not have bandwidth limiting available.  My connection is a symmetrical gig.  I'd like to limit each tenant to 100 mbps (10 total in the building) and deny P2P.  Are you saying that pfSense is not capable of doing that (or at least not efficiently) across 10 VLANs No, not at all. For that scenario I would use limiters on VLAN interfaces. That will be fine. I'll talk with my boss and my ISP about increasing our subnet size.  I really only see this working if I'm supplying at least /30's to each tenant.  As you said - do it right the first time.  Is there anything stopping me from breaking up a subnet into mixed sizes, or is that just poor form? For 10 customers you need at least a /26 to give them each a /30. No, making different size subnets is fine. /31s are your friend here. You might want to leave The way I'm planning on setting up the firewall - does that expose anything for me?  Is there a better configuration for that?  I need to make sure there are no security vulnerabilities as the LAN on that pfsense has the building's access control on it.  I also don't want to expose access to pfsense itself. On the customer interfaces, pass anything on the firewall they need access to like DNS, then block any any any to This firewall and any management or private LANs, then pass all traffic.

Hmm ok yes I was able to log dpinger triggering rc.gateway_alarm during WAN UP events as well but it wasn't consistent.  I believe as Denny said, sometimes other processes or scripts are killing dpinger and restarting it and thus it doesn't trigger the call to rc.gateway_alarm. I haven't had a chance to review the code in your PR but I will look at it. I know Renato wants to do things "right" - of course that is always best but sometimes when the SHTF you gotta do what you gotta do.

Bumping this one more time. Any ideeas?

Hi @Skid, This kind of setup really requires a good understanding of VLANs, how they work and how to configure them.  I get the impression you are not so familiar?  Go online, read up on access ports and trunk ports, tagged and untagged, VLAN IDs - different vendors vary the terminology a bit but it's all the same stuff! I've just returned from doing a temporary event with a very similar setup - only five ADSL connections on the WAN side but they were dotted all over site and had to pass through multiple switches to get to the router (a pfsense VM on a DL380). You need to define a few bits first: 1. Assign a VLAN ID to each WAN (eg. 51, 52 … 60). 2. Create untagged (access) ports on the cisco switch which connect to each modem. 3. Create a trunk (tagged) port on the cisco switch which passes all those VLAN IDs (ie. 51..60).  Connect that port to you r pfsense router and configure each VLAN on it's own interface in pfsense. 4. Don't use DHCP of PPPoE on the WAN connections, I had major issues doing it this way when a connection went offline.  Configure them all in their own subnets as you describe and set a static IP address for each WAN interface in pfsense. 5. Configure load balancing / traffic shaping in the pfsense router. You also need to create and configure a LAN connection - ideally via a physically separate network port but this could be a VLAN too, of course you'll need a suitably sized subnet and DHCP scope to cope with the number of users. What's your location?  I might be happy to help you with this.