<![CDATA[Two Bridges, two WAN and firewall rules problem]]>Hi i have two bridges, so 4 ifaces, 1 bridge is used for one ISP, the second for the other ISP.
i have servers behind this with a public IP all in a switch and i use rules to block some traffic.
I have problems when one server of the ISP1 trying to UPLOAD files to the ISP2, the firewall probably drop packets because if i disable firewall rules all work well, but when i active the rules the problem start, i check the firewall rules and y put all in GREEN, pass all traffic in all the 4 interfaces but the problem still there. How can i solve this problem? i try bypass firewall rules, i try other various options.

NOTE: The clients don't have any problem, they can upload files because the connection start in bridge1 and pass to bridge1 again, but when the packets start in bridge1 and enter in bridge2 then is blocked by "ghost" i don't know.

Thanks

]]>https://forum.netgate.com/topic/45313/two-bridges-two-wan-and-firewall-rules-problemRSS for NodeSat, 11 Apr 2026 09:06:45 GMTFri, 25 May 2012 14:44:28 GMT60<![CDATA[Reply to Two Bridges, two WAN and firewall rules problem on Tue, 07 Aug 2012 15:19:53 GMT]]>Ran across this today:

http://forum.pfsense.org/index.php/topic,50711.0/topicseen.html

See step number 2. Did you setup this advanced option:
2. VERY IMPORTANT: As mentioned at http://forum.pfsense.org/index.php?topic=30653.0, go to the 'System -> Advanced -> System Tunables'  and set net.link.bridge.pfil_bridge from 'default' to '1'

]]>https://forum.netgate.com/post/350418https://forum.netgate.com/post/350418Tue, 07 Aug 2012 15:19:53 GMT<![CDATA[Reply to Two Bridges, two WAN and firewall rules problem on Fri, 03 Aug 2012 19:15:19 GMT]]>I would do a tcpdump at each NIC to see where the packets are getting to before they are dropped. You might want to also log dropped packets. Are you seeing any drops or other errors in the system logs?

]]>https://forum.netgate.com/post/349984https://forum.netgate.com/post/349984Fri, 03 Aug 2012 19:15:19 GMT<![CDATA[Reply to Two Bridges, two WAN and firewall rules problem on Fri, 03 Aug 2012 17:54:21 GMT]]>Now states are in keep states, but i try with sloppy and not work, when i try with NONE i lose conection at all in the bridge.

Some idea?

]]>https://forum.netgate.com/post/349975https://forum.netgate.com/post/349975Fri, 03 Aug 2012 17:54:21 GMT<![CDATA[Reply to Two Bridges, two WAN and firewall rules problem on Fri, 03 Aug 2012 16:57:20 GMT]]>On GTD_1 and ENTEL_1 on that allow everything rule at the top, is keep state on or off in the advanced options?

]]>https://forum.netgate.com/post/349971https://forum.netgate.com/post/349971Fri, 03 Aug 2012 16:57:20 GMT<![CDATA[Reply to Two Bridges, two WAN and firewall rules problem on Fri, 03 Aug 2012 14:24:59 GMT]]>I don't have any nat i use the firewall like a filter bridge of my optic fibers.

In the servers i have 1 or 2 nics not all servers have two providers connected, but if server 1 is connected to GTD and server 2 connected with ENTEL if i send mails or something work but if i attach files to mail and send, or transfer more than ~48Kb the pfsense block the traffic.

this is my map of connections

SERVER1 –->SWITCH<--- SERVER2
                      |        |
                      |        | FO (ENTEL AND GTD)
                    PFSENSE  <---- HERE IS THE BRIDGES one bridge for entel and other for GTD providers...
                      |        |  <--FO external providers...

this is the nat screenshot now...

Pantallazo-9.png
Pantallazo-9.png_thumb

]]>https://forum.netgate.com/post/349937https://forum.netgate.com/post/349937Fri, 03 Aug 2012 14:24:59 GMT<![CDATA[Reply to Two Bridges, two WAN and firewall rules problem on Fri, 03 Aug 2012 13:51:48 GMT]]>I saw your firewall rules. You have an allow all at the top, so anything should be able to pass. What I am saying is that if you have 2 NIC in each server, one on either WAN subnet, then pfSense should not be consulted when trying to go from server to server (WAN1 to WAN2). It should just use the NIC in that subnet. If you are doing something weird with the server routing, you might be preventing it from working as it should.
What do you have set for your outbound NAT? Please make sure it is set to manual (not auto) and there are no rules except for the LAN subnet. When you turn off firewalling, it disables NAT also.

]]>https://forum.netgate.com/post/349928https://forum.netgate.com/post/349928Fri, 03 Aug 2012 13:51:48 GMT<![CDATA[Reply to Two Bridges, two WAN and firewall rules problem on Fri, 03 Aug 2012 13:10:13 GMT]]>No the IPTABLES of the servers was flushed, i can see the packets pass thoug the firewall usign the packet capture function, if i go to system->advanced->firewall/NAT->disable all packet filtering. and check this option all work fine but i don't have firewall rules working :/ so the servers was blocked by the firewall

]]>https://forum.netgate.com/post/349912https://forum.netgate.com/post/349912Fri, 03 Aug 2012 13:10:13 GMT<![CDATA[Reply to Two Bridges, two WAN and firewall rules problem on Thu, 02 Aug 2012 20:36:48 GMT]]>So the servers have 2 NICs one in each providers IP subnet? If that is the case, then pfSense should never be reached, it should just the NIC in the correct subnet. Are you just creating routing rules on the servers themselves? Are you using firewall rules on the servers themselves?

]]>https://forum.netgate.com/post/349836https://forum.netgate.com/post/349836Thu, 02 Aug 2012 20:36:48 GMT<![CDATA[Reply to Two Bridges, two WAN and firewall rules problem on Thu, 02 Aug 2012 19:56:25 GMT]]>@podilarius:

What kind of NAT are you doing? I think I will setup a test with bridging to see …

This might be just a routing problem. I am guessing that your 2 ISPs are in seperate subnets.
I think it should be routing to the ISP default gateway and then over to your second bridge through the second ISP. If you have told it not to NAT then the system will not know how to route back.
I think we are going to need more information here. NAT rules, route rules and such ... the firewall rules themselves seem to be fine. Just don't understand how it is working with firewall disabled.

there is no NAT i have Dell poweredge r710 with 6 NICs
and TWO ISP
GTD and ENTEL

i have several Ip blocks in every ISPs comes from 2 fiber optics mostly /29 nets

the connection is some like this
GTD  –-> to pfsense(nic1)  (bridge_gtd here) (nic3) -->SWITCH WAN
ENTEL --> to pfsense(nic2)  (bridge_entel here) (nic4) -->SWITCH WAN
pfsense(nic5) ---> SWITCH LAN (i use this nic with 192.168.1.250 ip to control pfsense webadministrator

so i don't have any NAT configured just 4 NIC cards  2 joined with a bridge for GTD provider and the other two with other bridge for the other ISP.

i configure my servers to route default for one or other with this script:

the script purpose if when the packet come from entel provider (190.151.x.x) go with default gateway to the corresponding default gateway of the providers.

ip route add table T1 default via 190.196.32.89
ip route add table T2 default via 190.151.71.161

ip rule add from 190.196.32.93 table T1
ip rule add to 190.196.32.93 table T1

ip rule add from 190.151.71.171 table T2
ip rule add from 190.151.71.171 table T2

ip route flush cache

the script works i can probe with a traceroute here.
[root@correo ~]# traceroute -s 190.151.71.171 8.8.8.8
traceroute to 8.8.8.8 (8.8.8.8), 30 hops max, 40 byte packets
1  200.111.166.121 (200.111.166.121)  1.790 ms  1.625 ms  1.837 ms  <–-- DEFAULT GATEWAY OF ENTEL provider
2  192.168.90.173 (192.168.90.173)  20.690 ms  20.612 ms  21.075 ms

traceroute to 8.8.8.8 (8.8.8.8), 30 hops max, 40 byte packets
1  190.196.32.89 (190.196.32.89)  2.491 ms  2.457 ms  2.448 ms  <--- DEFAULT GATEWAY OF GTD provider
2  190.196.126.126 (190.196.126.126)  2.452 ms  2.446 ms  2.435 ms
3  cn1.ge0-0-3.str2.gtdinternet.com (201.238.238.69)  3.742 ms  3.735 ms  3.798 ms

so all servers are configured the same with their respective providers the script work

when a client OUTSIDE MY NETWORK try to upload files if the packet come from entel is responsed to entel gateway
and if the IP packet come from GTD the server response from GTD. The system works fine no problem

so in my diagram
(CLIENT HERE) GTD  ---> to pfsense(nic1)  (bridge_gtd here) (nic3) -->SWITCH WAN <-->(server here) THIS WORK
(or client here) ENTEL --> to pfsense(nic2)  (bridge_entel here) (nic4) -->SWITCH WAN <--> (server here) THIS WORK

all work fine if the client start the connection

but if i have server1 and server2 in MY NETWORK and try to connect one with other PFSENSE block the trafic

(REACH GTD ROUTER) GTD  ---> to pfsense(nic1)  (bridge_gtd here) (nic3) <-->SWITCH WAN  <--- (IF TRAFFIC START HERE trying to go out to server2)
(back from entel) -->ENTEL --> to pfsense(nic2)  (bridge_entel here) (nic4) -->SWITCH WAN  ????? BLOCKED after ~48KBs of transfer....

so the problem if here i think if a routing problem maybe, pfsense don't understand why one packet goes out from one bridge and enter from the other bridge and is confused...

some ideas ????
i will provide some screenshots of my CFGs





thanks

]]>https://forum.netgate.com/post/349827https://forum.netgate.com/post/349827Thu, 02 Aug 2012 19:56:25 GMT<![CDATA[Reply to Two Bridges, two WAN and firewall rules problem on Thu, 02 Aug 2012 19:50:45 GMT]]>@podilarius:

What kind of NAT are you doing? I think I will setup a test with bridging to see …

This might be just a routing problem. I am guessing that your 2 ISPs are in seperate subnets.
I think it should be routing to the ISP default gateway and then over to your second bridge through the second ISP. If you have told it not to NAT then the system will not know how to route back.
I think we are going to need more information here. NAT rules, route rules and such ... the firewall rules themselves seem to be fine. Just don't understand how it is working with firewall disabled.

there is no NAT i have Dell poweredge r710 with 6 NICs
and TWO ISP
GTD and ENTEL

i have several Ip blocks in every ISPs comes from 2 fiber optics mostly /29 nets

the connection is some like this
GTD  –-> to pfsense(nic1)  (bridge_gtd here) (nic3) -->SWITCH WAN
ENTEL --> to pfsense(nic2)  (bridge_entel here) (nic4) -->SWITCH WAN
pfsense(nic5) ---> SWITCH LAN (i use this nic with 192.168.1.250 ip to control pfsense webadministrator

so i don't have any NAT configured just 4 NIC cards  2 joined with a bridge for GTD provider and the other two with other bridge for the other ISP.

i configure my servers to route default for one or other with this script:

the script purpose if when the packet come from entel provider (190.151.x.x) go with default gateway to the corresponding default gateway of the providers.

ip route add table T1 default via 190.196.32.89
ip route add table T2 default via 190.151.71.161

ip rule add from 190.196.32.93 table T1
ip rule add to 190.196.32.93 table T1

ip rule add from 190.151.71.171 table T2
ip rule add from 190.151.71.171 table T2

ip route flush cache

the script works i can probe with a traceroute here.
[root@correo ~]# traceroute -s 190.151.71.171 8.8.8.8
traceroute to 8.8.8.8 (8.8.8.8), 30 hops max, 40 byte packets
1  200.111.166.121 (200.111.166.121)  1.790 ms  1.625 ms  1.837 ms  <–-- DEFAULT GATEWAY OF ENTEL provider
2  192.168.90.173 (192.168.90.173)  20.690 ms  20.612 ms  21.075 ms

traceroute to 8.8.8.8 (8.8.8.8), 30 hops max, 40 byte packets
1  190.196.32.89 (190.196.32.89)  2.491 ms  2.457 ms  2.448 ms  <--- DEFAULT GATEWAY OF GTD provider
2  190.196.126.126 (190.196.126.126)  2.452 ms  2.446 ms  2.435 ms
3  cn1.ge0-0-3.str2.gtdinternet.com (201.238.238.69)  3.742 ms  3.735 ms  3.798 ms

so all servers are configured the same with their respective providers the script work

when a client OUTSIDE MY NETWORK try to upload files if the packet come from entel is responsed to entel gateway
and if the IP packet come from GTD the server response from GTD. The system works fine no problem

so in my diagram
(CLIENT HERE) GTD  ---> to pfsense(nic1)  (bridge_gtd here) (nic3) -->SWITCH WAN <-->(server here) THIS WORK
(or client here) ENTEL --> to pfsense(nic2)  (bridge_entel here) (nic4) -->SWITCH WAN <--> (server here) THIS WORK

all work fine if the client start the connection

but if i have server1 and server2 in MY NETWORK and try to connect one with other PFSENSE block the trafic

(REACH GTD ROUTER) GTD  ---> to pfsense(nic1)  (bridge_gtd here) (nic3) <-->SWITCH WAN  <--- (IF TRAFFIC START HERE trying to go out to server2)
(back from entel) -->ENTEL --> to pfsense(nic2)  (bridge_entel here) (nic4) -->SWITCH WAN  ????? BLOCKED after ~48KBs of transfer....

so the problem if here i think if a routing problem maybe, pfsense don't understand why one packet goes out from one bridge and enter from the other bridge and is confused...

some ideas ????
i will provide some screenshots of my CFGs

thanks

Pantallazo.png
Pantallazo.png_thumb
Pantallazo-1.png
Pantallazo-1.png_thumb
Pantallazo-2.png
Pantallazo-2.png_thumb
Pantallazo-3.png
Pantallazo-3.png_thumb
Pantallazo-4.png
Pantallazo-4.png_thumb
Pantallazo-5.png
Pantallazo-5.png_thumb

]]>https://forum.netgate.com/post/349826https://forum.netgate.com/post/349826Thu, 02 Aug 2012 19:50:45 GMT<![CDATA[Reply to Two Bridges, two WAN and firewall rules problem on Thu, 02 Aug 2012 17:28:42 GMT]]>What kind of NAT are you doing? I think I will setup a test with bridging to see …

This might be just a routing problem. I am guessing that your 2 ISPs are in seperate subnets.
I think it should be routing to the ISP default gateway and then over to your second bridge through the second ISP. If you have told it not to NAT then the system will not know how to route back.
I think we are going to need more information here. NAT rules, route rules and such ... the firewall rules themselves seem to be fine. Just don't understand how it is working with firewall disabled.

]]>https://forum.netgate.com/post/349794https://forum.netgate.com/post/349794Thu, 02 Aug 2012 17:28:42 GMT<![CDATA[Reply to Two Bridges, two WAN and firewall rules problem on Thu, 02 Aug 2012 15:46:38 GMT]]>@podilarius:

Do you have keep states turned off?

I test with keep states, sloppy states and none    and NOT WORK :s

]]>https://forum.netgate.com/post/349780https://forum.netgate.com/post/349780Thu, 02 Aug 2012 15:46:38 GMT<![CDATA[Reply to Two Bridges, two WAN and firewall rules problem on Tue, 29 May 2012 22:30:16 GMT]]>Do you have keep states turned off?

]]>https://forum.netgate.com/post/339033https://forum.netgate.com/post/339033Tue, 29 May 2012 22:30:16 GMT<![CDATA[Reply to Two Bridges, two WAN and firewall rules problem on Fri, 25 May 2012 15:59:33 GMT]]>@podilarius:

Sorry, I would also like the rules from the other bridge also.

Are the same of the image an all 4 bridge interfaces gtd_1 and 2 and entel_1 and 2

:/ all pass…

]]>https://forum.netgate.com/post/338472https://forum.netgate.com/post/338472Fri, 25 May 2012 15:59:33 GMT<![CDATA[Reply to Two Bridges, two WAN and firewall rules problem on Fri, 25 May 2012 15:50:29 GMT]]>Sorry, I would also like the rules from the other bridge also.

]]>https://forum.netgate.com/post/338467https://forum.netgate.com/post/338467Fri, 25 May 2012 15:50:29 GMT<![CDATA[Reply to Two Bridges, two WAN and firewall rules problem on Fri, 25 May 2012 15:06:42 GMT]]>I don't think if necesary :/ all interfaces was pass all trafic * * * for all protocols… so any difference.
but i will provide one if this solve something...
the screenshot is here... check
http://www.image-share.com/ipng-1502-122.html

Note: i'm try conservative, latency... and is not work :/ i'm try Clear invalid DF bits instead of dropping the packets
Disables the PF scrubbing option which can sometimes interfere with NFS and PPTP traffic.
well almost any option and combinations...

thanks

@podilarius:

I don't suppose that you could post a sanitized copy/screen shot of your rules?

]]>https://forum.netgate.com/post/338455https://forum.netgate.com/post/338455Fri, 25 May 2012 15:06:42 GMT<![CDATA[Reply to Two Bridges, two WAN and firewall rules problem on Fri, 25 May 2012 14:53:46 GMT]]>I don't suppose that you could post a sanitized copy/screen shot of your rules?

]]>https://forum.netgate.com/post/338449https://forum.netgate.com/post/338449Fri, 25 May 2012 14:53:46 GMT