Two Bridges, two WAN and firewall rules problem

morpheus747

Hi i have two bridges, so 4 ifaces, 1 bridge is used for one ISP, the second for the other ISP.
i have servers behind this with a public IP all in a switch and i use rules to block some traffic.
I have problems when one server of the ISP1 trying to UPLOAD files to the ISP2, the firewall probably drop packets because if i disable firewall rules all work well, but when i active the rules the problem start, i check the firewall rules and y put all in GREEN, pass all traffic in all the 4 interfaces but the problem still there. How can i solve this problem? i try bypass firewall rules, i try other various options.

NOTE: The clients don't have any problem, they can upload files because the connection start in bridge1 and pass to bridge1 again, but when the packets start in bridge1 and enter in bridge2 then is blocked by "ghost" i don't know.

Thanks

podilarius

I don't suppose that you could post a sanitized copy/screen shot of your rules?

morpheus747

I don't think if necesary :/ all interfaces was pass all trafic * * * for all protocols… so any difference.
but i will provide one if this solve something...
the screenshot is here... check
http://www.image-share.com/ipng-1502-122.html

Note: i'm try conservative, latency... and is not work :/ i'm try Clear invalid DF bits instead of dropping the packets
Disables the PF scrubbing option which can sometimes interfere with NFS and PPTP traffic.
well almost any option and combinations...

thanks

@podilarius:

I don't suppose that you could post a sanitized copy/screen shot of your rules?

podilarius

Sorry, I would also like the rules from the other bridge also.

morpheus747

@podilarius:

Sorry, I would also like the rules from the other bridge also.

Are the same of the image an all 4 bridge interfaces gtd_1 and 2 and entel_1 and 2

:/ all pass…

podilarius

Do you have keep states turned off?

morpheus747

@podilarius:

Do you have keep states turned off?

I test with keep states, sloppy states and none and NOT WORK :s

podilarius

What kind of NAT are you doing? I think I will setup a test with bridging to see …

This might be just a routing problem. I am guessing that your 2 ISPs are in seperate subnets.
I think it should be routing to the ISP default gateway and then over to your second bridge through the second ISP. If you have told it not to NAT then the system will not know how to route back.
I think we are going to need more information here. NAT rules, route rules and such ... the firewall rules themselves seem to be fine. Just don't understand how it is working with firewall disabled.

morpheus747

@podilarius:

What kind of NAT are you doing? I think I will setup a test with bridging to see …

This might be just a routing problem. I am guessing that your 2 ISPs are in seperate subnets.
I think it should be routing to the ISP default gateway and then over to your second bridge through the second ISP. If you have told it not to NAT then the system will not know how to route back.
I think we are going to need more information here. NAT rules, route rules and such ... the firewall rules themselves seem to be fine. Just don't understand how it is working with firewall disabled.

there is no NAT i have Dell poweredge r710 with 6 NICs
and TWO ISP
GTD and ENTEL

i have several Ip blocks in every ISPs comes from 2 fiber optics mostly /29 nets

the connection is some like this
GTD –-> to pfsense(nic1) (bridge_gtd here) (nic3) -->SWITCH WAN
ENTEL --> to pfsense(nic2) (bridge_entel here) (nic4) -->SWITCH WAN
pfsense(nic5) ---> SWITCH LAN (i use this nic with 192.168.1.250 ip to control pfsense webadministrator

so i don't have any NAT configured just 4 NIC cards 2 joined with a bridge for GTD provider and the other two with other bridge for the other ISP.

i configure my servers to route default for one or other with this script:

the script purpose if when the packet come from entel provider (190.151.x.x) go with default gateway to the corresponding default gateway of the providers.

ip route add table T1 default via 190.196.32.89
ip route add table T2 default via 190.151.71.161

ip rule add from 190.196.32.93 table T1
ip rule add to 190.196.32.93 table T1

ip rule add from 190.151.71.171 table T2
ip rule add from 190.151.71.171 table T2

ip route flush cache

the script works i can probe with a traceroute here.
[root@correo ~]# traceroute -s 190.151.71.171 8.8.8.8
traceroute to 8.8.8.8 (8.8.8.8), 30 hops max, 40 byte packets
1 200.111.166.121 (200.111.166.121) 1.790 ms 1.625 ms 1.837 ms <–-- DEFAULT GATEWAY OF ENTEL provider
2 192.168.90.173 (192.168.90.173) 20.690 ms 20.612 ms 21.075 ms

traceroute to 8.8.8.8 (8.8.8.8), 30 hops max, 40 byte packets
1 190.196.32.89 (190.196.32.89) 2.491 ms 2.457 ms 2.448 ms <--- DEFAULT GATEWAY OF GTD provider
2 190.196.126.126 (190.196.126.126) 2.452 ms 2.446 ms 2.435 ms
3 cn1.ge0-0-3.str2.gtdinternet.com (201.238.238.69) 3.742 ms 3.735 ms 3.798 ms

so all servers are configured the same with their respective providers the script work

when a client OUTSIDE MY NETWORK try to upload files if the packet come from entel is responsed to entel gateway
and if the IP packet come from GTD the server response from GTD. The system works fine no problem

so in my diagram
(CLIENT HERE) GTD ---> to pfsense(nic1) (bridge_gtd here) (nic3) -->SWITCH WAN <-->(server here) THIS WORK
(or client here) ENTEL --> to pfsense(nic2) (bridge_entel here) (nic4) -->SWITCH WAN <--> (server here) THIS WORK

all work fine if the client start the connection

but if i have server1 and server2 in MY NETWORK and try to connect one with other PFSENSE block the trafic

(REACH GTD ROUTER) GTD ---> to pfsense(nic1) (bridge_gtd here) (nic3) <-->SWITCH WAN <--- (IF TRAFFIC START HERE trying to go out to server2)
(back from entel) -->ENTEL --> to pfsense(nic2) (bridge_entel here) (nic4) -->SWITCH WAN ????? BLOCKED after ~48KBs of transfer....

so the problem if here i think if a routing problem maybe, pfsense don't understand why one packet goes out from one bridge and enter from the other bridge and is confused...

some ideas ????
i will provide some screenshots of my CFGs

thanks

Pantallazo.png_thumb

Pantallazo-1.png_thumb

Pantallazo-2.png_thumb

Pantallazo-3.png_thumb

Pantallazo-4.png_thumb

Pantallazo-5.png_thumb

morpheus747