So I added "any" for protocol in OPT1 rule, and the ping started to respond! When adding a new rule, it defaults to IPv4 protocol TCP. With a rule that just permits TCP you do not get ICMP (ping) or UDP or… That is a common trick for new players :)

Why would switching WAN connections affect my internal LAN/vLAN communications? Because it does.  :) Policy routing changes the way you need to treat internal traffic.

That's going to be problematic because you'd need to route traffic out the same interface it came in on. If you have, say: 192.168.1.1/24 pfSense LAN 192.168.1.25 Host 192.168.1.254 VPN device If you set the default gateway on 192.168.1.25 to 192.168.1.1, then make a rule on LAN sending traffic from 192.168.1.25 to 192.168.1.254 it's going to get weird. Why not just set the default gateway of the host to 192.168.1.254 if that's how you want it to behave anyway?  Then there's almost no possibility of leakage. What does the VPN device do that pfSense can't do itself?

@bbfrankopan: can second pfsense box be aware that wan1 on first pfsense is down,  and use different gateway group? Assuming that is L3 connection between them (OSPF).

@arduino, Were you able to get this working? I'm using pfSense on a 175/175 Bell Fibe connection as well and I am able to get internet passed through while bypassing the Home Hub 2000 entirely, but I'm not sure how to configure the TV. If you've figured it out, please share how you managed to do it. Thanks! Robert

Thanks a lot for reply. wow, nice feature, I never heard about this in pfsense, thanx for info but unusable in this case - I don't know why, but all packets from member interface to internet, bypass member interface firewall rule. For example, if I set deny rule for ICMP to 8.8.8.8 in member interface, ping still working, and in firewall log is sourece iface LAN_BRIDGE. I logging all rules now on member iface and LAN_BRIDGE, and seems like all internal traffic beteween LANs has source iface LAN_VYT or LAN_INTERNAL, but if it's traffic to internet, source iface in log is LAN_BRIDGE. For example 2 - if i delete any-any-any pass rule from member iface, cannot access form LAN_INTERNAL to LAN_VYT and vice versa. But still I can access to internet.. :( Thanx Rob EDIT: And yes, I have net.link.bridge.pfil_member and net.link.bridge.pfil_bridge both set to 1

No. That would be aggregation, not load balancing. IF you have four DSL lines from the same provider and that provider supports Multi-Link PPP (MLPPP) then you can get true bonding/aggregation which looks like one large pipe with one IP address that has the sum total bandwidth of all lines. But support for that is rare at the ISP level.

For what it's worth, to config devices like that I use a blank VLAN (no pfSense interface at all.) Say my blank bench VLAN is 1200.  I have untagged ports on VLAN 1200 on the bench and I create a VLAN interface on my workstation (a mac).  I can then set that VLAN interface to whatever IP network I need to access devices out-of-the-box.  This doesn't disrupt normal network traffic since my main LAN is also tagged to my workstation.  I don't have to worry about devices having DHCP servers enabled by default or anything since it's isolated from everyone.  The workstation has a tftp server for firmware/config files, etc.

Okay so i got it to work 8) Switch config: Port 1 ISP/FTTH box -> Tagged vlan 40 Port 2 Pfsense -> Tagged vlan 40,50 Port 3 - 8 -> Untagged vlan 50 pFsense console: Wan: em0_vlan40 Lan: em0_vlan50 i hope it is the right way to do it :)

Hello everyone , Yep , I am clinging to my topic !!!  :P My latest tests and my conclusion : I have been testing various protocol and it turns out they do not react the same way . As I have said above , ICMP is left of those below : With a ping, ICMP packets do well by the designated interface on the firewall 1 ICMP rule to the firewall 2 via the well VPN link and come on site 2, then arrive at the destination machine , leave and resume … well take the route specified in the rule of pfSense 2 Site 2 and finally arrive destination on the site 1 has taking good VPN link ! (remember, for all that is not ICMP, the return is ALWAYS ON DEFAULT GATEWAY on the distant PFSENSE !). ICMP -> Layer 3 of the OSI model max -> no no FLAG TCP ACK !!!! In fact : pfSense returns the default interface ALL PACKTES marked ACK ! ( when a packet traverses a rule it is tagged by pfSense ACK). So new question for advanced users : how to solve my problem knowing that, without mounting a gas plant with floatings rules and proxy ... ??? Thank's for your help and interest. Hope some people will be interest by the challenge !

@viragomann: Are your outbound NAT rules working correctly? You have to set up manual outbound NAT rules to tell pfSense to translate packets from PBX to WAN2 address. Otherwise the packets get the WAN address, which is a private one and will not be routed in the WAN2 net. These are the current outbound manual rules… I "think" these are right? [image: out.png] [image: out.png_thumb]

I don't think you will ever be happy with 4 USB NICs on a USB hub. sometime I get internet access and sometimes I don't. Yup.  Sounds familiar.  USB NICs suck.  There's your advice.  Get real hardware (like the Cisco you're trying to replace.)