Weird routing issue
-
We are trying to phase out some ubiquiti routers and our last untangle router.
Currently we have a web gateway sitting behind a ubiquti router, its a ts/citrix gateway to servers on the same vlan and on another vlan routed through the untangle router.
The web gateway's default gw is the ubnt router, the ubnt router has a static route to get to the other vlan via the untangle.
Pretty easy straight forward stuff.
When we swapped in the pfsense router, everything appeared to work properly, but we found the connection/routing would only stay alive for 10-30 seconds and then it wouldn't work anymore.
servers on the same vlan as the pfsense router would work fine, but it would not route traffic back through the untangle.
We have a lan firewall rule that is basically allow anything to anything on the lan interface.
but again it starts to work and then dies.
Let me know if you need anymore clarification, for the time being I think I get can around this by adding the static route on the webgateway to get to the other vlan, but I shouldn't have to do this as it works on the ubnt router just fine.
-
Draw it.
Sounds like asymmetry.
-
Some additional info.
I have another pfsense router that hosts my vpn tunnels.
From the across the vpn I can hit anything that uses the pfsense router for the vpn tunnels.
If I try to hit a machine that uses another pfsense router, the static route doesn't work.
I can hit the second pfsense router from across the vpn, so the routing is working, but I can't hit anything that uses the second pfsense router as a gateway.
I'm missing something stupid.
-
@Derelict It's just basic routing. But something is blocking traffic somewhere and i can't see it or i'm missing it.
-
A state disappearing (working for 30-ish seconds then stopping) indicates an asymmetric routing problem. The firewall state is not seeing all of the TCP handshake.
Like I said, diagram it out.
-
from the VPN side I can't get tcp packets in but ICMP works. Smells like a firewall rule but I have a allow ip any any rule to prevent that. I'll try and get a basic drawing for you.
But again this is pretty basic stuff, and worked with the Ubiquiti router.
I'm not a novice user, pretty advanced, ccnp ccdp etc . It just doesn't make any sense lol.
-
from the VPN side I can't get tcp packets in but ICMP works.
Exactly, that sounds like asymmetric routing too.
-
I have a computer using 10.0.0.2 as it's default gateway.
and a computer on 192.168.1.x vpn.
The computer using 10.0.0.2 as a gateway can get to the computer on 192.168.1.0
The computer on the vpn can not get back to the computer using 10.0.0.2.
But they can ping each other.
P2 has the simple default routes, but its like the route isn't obey'ed or it's trying to nat the traffic?
-
I think I got it figured out. I had to put in some no-nat rules in the outbound nat settings. Still testing, but it seems like I need an outbound nonat rule that matches my static routes.
-
@Derelict I dug into this some more this morning. and you were right. I enabled "Bypass firewall rules for traffic on the same interface" this appears to have resolved all of our issues.
-
@Kevin-S-Pare I would fix the asymmetric routing instead but that's probably just me.
-
I'm not sure how I can do that exactly right now. the only real way is to add static routes on all the various machines. unless i'm missing something obvious.
-
Put in a transit network to whatever is routing and get it off the same segment as other hosts.
-
A -> B -> C -> E - Traffic From Remote Location
E -> D -> C -> B -> A Traffic Going Back
A = Remote Server
B = Remote Cisco Router
C = PFSense Router 1
D = PFSense Router 2
E = Local ServerVPN Tunnel between B and C
E = Has services hosted via the wan connection on D, but needs to talk to a server over the VPN TunnelWhere would you put a transit network. That seems like it would complicate a rather simple configuration?
-
It's not simple. It's asymmetric. It breaks TCP through stateful firewalls as you have found out.
I can't make anything out of that "diagram". I'll need more detail. Like interface addresses, subnets, etc.
