TCP:SA Packets Blocked when Sending VoIP Traffic to alternate GW on Both Ends

Spiffster

Thanks for your response johnpoz!

I have policy based routes setup like so:

HQ:

Allow ALL on LAN from source: Avaya to ANY use MPLS GW (aka VOIPGROUP)

REMOTE OFFICES:

Allow ALL on LAN to dest: Avaya use MPLS GW (aka VOIPGROUP)

Screenshot of rules on both ends attached. Right now they are just set to route UDP traffic, and that does work obviously because UDP is stateless. If I change that to UDP/TCP the phones at the remote locations will disconnect because TCP:SA response is being blocked at HQ. Note: 192.168.5.12 is the Avaya IPO server

That said, I dont get why routes are asymmetric.

fw_VoIP.png_thumb

johnpoz

"Allow ALL on LAN from source: Avaya to ANY use MPLS GW"

and what order do you have these rules setup. Post your lan rules all of them.. Do you have any floating rules?

While udp is not like tcp, states will be created in pfsense that allow or block traffic, etc.

Rules are top down first rule to fire wins. So while I see you have connections being used on your remote office rule, I don't see any hits on your HQ rule. If there is an any any above that it would never be seen for example.

Spiffster

No floating rules in place. Attached rules for LAN on each fw. Thanks!

fw_Rules.png_thumb

Spiffster

What I see when I boot up the phones at remote office is they grab DHCP info from the local DHCP server, that will point them to grab their configs from the Avaya IPO server, they send a TCP:S to the server and never get the TCP:SA response because its blocked. Cant make any sense of it.

johnpoz

I don't see any hits on that rule for 192.168.5.12 as source.. Is that 192.168.5 a downstream network? Maybe the device has a different gateway set so it would never send to pfsense lan interface to be policy routed out?

What is the point of the openvpn rule on your lan - that would ever been seen since you have a lan net rule any any above it.

Spiffster

@johnpoz:

I don't see any hits on that rule for 192.168.5.12 as source.. Is that 192.168.5 a downstream network? Maybe the device has a different gateway set so it would never send to pfsense lan interface to be policy routed out?

Its a pretty basic setup. Every device on the HQ network (192.168.5.x) has this pfSense box as its GW. From what I understand is pfSense matches ANY traffic from that IP(192.168.5.12) and redirects it through the MPLS gateway… and on the other end (192.168.2.x) that pfSense box redirects any traffic to that IP(192.168.5.12) through the MPLS as well.

I dont see how this is asymmetric, but its apparently blocking the TCP:AS packets for that reason.

@johnpoz:

What is the point of the openvpn rule on your lan - that would ever been seen since you have a lan net rule any any above it.

That was put there by some wizard I ran a while back. I can remove it.

johnpoz

"From what I understand is pfSense matches ANY traffic from that IP(192.168.5.12) and redirects it through the MPLS gateway"

But its not or there is no traffic.. Your rule shows NO hits that 0 currently, and total its only moved a whole 12KB..

How are these connected? Are you natting at the mpls connections at both ends?

Spiffster

Yes, and I am probably missing something stupid but I dont know what… that traffic happily traverses the VPN from HQ to remote sites.

Normally:

192.168.5.12(phone server) <--> 192.168.5.1 <--> OpenVPN <--> 192.168.2.1 <--> 192.168.2.47(phone)

I would like OpenVPN to be replaced with MPLS... seems like this should be pretty straight forward.

I could do this entirely without pfSense by manually setting the phone server GW to 192.168.5.254(MPLS GW) and manually setting all remote phone GW to their local MPLS router eg: 192.168.2.254:

192.168.5.12(phone server) <--> 192.168.5.254 <--> MPLS <--> 192.168.2.254 <--> 192.168.2.47(phone)

But this is not feasible and would not allow for failover.

johnpoz

Mpls is not a tunnel. there has to be routes that say hey to get to 192.168.x go here

192.168.5.12(phone server) <–> 192.168.5.254 pfsense mpls_IP?? <--> MPLS <--> mpls_IP? pfsense 192.168.2.254 <--> 192.168.2.47(phone)

Spiffster

I do realize these are quite different technologies (private circuit vs internet VPN), but I can connect to each remote office through either. Its nice because I can work on the pfSense boxes remotely without the risk of losing my connection. I use MPLS as a backdoor of sorts, so if I hose a pfSense box I dont have to drive out to the remote office, I can just use a static route to a remote VM that has the MPLS router as its default GW. One doesnt rely on the other in this setup, I just want to re-route VoIP traffic over MPLS via pfSense.

Not sure if its worth mentioning, but the pfSense system at HQ is a CARP setup. Each remote location just have a single pfSense VM.

johnpoz

What part do you not understand about the mpls IPs?? And your natting to them.. WHAT are the mpls IPs - how does the mpls network know where 192.168.5 is?? Does it?

I would assume your natting to this mpls network??

Spiffster

I apologize for leaving information out but I am not natting anything on MPLS… Each of our four sites have an MPLS router (Adtran 908e) managed by Windstream. As far as I know (and obviously I dont know shit about MPLS) the MPLS routers are connected to one another via a private circuit. Our pfSense at each location just list MPLS as an additional GW. Under System-> Routing I have:

Comcast (default)
Windstream
MPLS

Here are some tracert output from my pc on the 192.168.5.x (HQ) network:
Using default GW: (pfSense OpenVPN)

tracert 192.168.2.20
1 <1 ms <1 ms <1 ms 192.168.5.2
2 16 ms 14 ms 14 ms 10.5.2.2
3 17 ms 14 ms 14 ms 192.168.2.20

Using GW 192.168.5.254 (Adtran Router MPLS)

tracert 192.168.2.55
1 1 ms <1 ms <1 ms 192.168.5.254
2 3 ms 3 ms 3 ms 216.43.148.141
3 * * * Request timed out.
4 3 ms 3 ms 3 ms 63-253-248-153.mcleodusa.net [63.253.248.153]
5 15 ms 10 ms 12 ms 63-253-248-154.ip.mcleodusa.net [63.253.248.154]
6 14 ms 13 ms 13 ms 192.168.2.55

I hope this helps to clarify things. Sorry if im being a PITA.

johnpoz

So yeah your natting… What does the IP address of your mpls interface say is its IP? And post up your outbound NAT page..

Spiffster

@johnpoz:

So yeah your natting… What does the IP address of your mpls interface say is its IP? And post up your outbound NAT page..

FWIW, looking at the router config for that MPLS circuit I do see an MPLS VPN setup. I dont see a NAT config.

Anyway, I do appreciate your help but I think the solution in my case would be to just change the Avaya IPO Server's gateway to the 192.168.5.254 (MPLS Router) from 192.168.5.1 (pfSense FW) which will eliminate the asymmetric route issue.