NAT Port Forwarding to Internal host UDP port 5060 not working as expected
-
Hey guys saw this is a recent thread and I am having the exact same issue. I tried upgrading to 2.1 but still having the issue. I've tried port forwarding, ive tried static outgoing nat, set the firewall to conservative, still no dice. I am having an issue with the FreeSWITCH pbx behind the firewall where incoming calls drop after exactly 32 seconds. Doing packet caps, I traced it to the firewall not forwarding SIP 'ACK' packets along to the pbx. I've attached the flow of 3 packet caps - one on the pfsense lan, one on wan, and one on the pbx. I have phones registered directly to the ip trunk carrier now as a workaround since my pbx is not fully functional due to this issue. In the packet caps the phones all ring for about a half second then the PBX auto-answers and routes to the IVR menu so the phones stop ringing after half a ring. You can see in the packet caps that the phones receive the incoming ACK packets fine, and note that the phones are using random ports, while the PBX is always using 5080. Maybe this has something to do with it?
Please note that the LAN capture and WAN capture were done for different calls, as I did not know how to do both at once with pfSense. The PBX capture was done at the same time as the pfSense WAN capture IIRC.Edit: Removed the images as they are not really relevant to this thread. We somewhat solved our issue as you can see below, but I plan to do some more testing with the static nat options and such to see if my install exhibits the same behavior as the OP's.
-
Just to let everyone know, we resolved our issue. Here is a summary of what we are working with:
pfSense 2.1 box on a Verizon FIOS connection
FreeSWITCH PBX on internal LAN, using SIP registration. All endpoints (phones) are on the internal lan as well.To fix this issue, we needed to modify the way our PBX behind the NAT formed its SIP packets. It was formatting the Contact field in the SIP headers with the external WAN IP and the PBX external port. When the UDP packets passed out from the pbx and through pfSense, pfSense was randomizing the source port. The voice carrier's systems seem to work in a way where if the IP in the contact field matches the source IP of the packet, it chooses to use that return IP along with the port shown in the contact field. Since the port was set as the PBX's external port, and pfSense was randomizing the port, the return traffic from the voice provider was addressed to that pbx port, not the pfsense port, and the traffic was dropped. When we modified the PBX configuration to use its local LAN ip in the contact field, the voice provider's systems chose to disregard the contact field, and would return the udp packets to the source port that pfSense set, and everything would work ok.
So I will try to document how this stuff works so anyone out there struggling with the same issue can understand how the symmetric NAT works with SIP and VOIP. Now there are two main elements to VOIP:
SIP Signalling (controls the flow of the call, can contain DTMF signalling and controls the beginning, ending, routing etc of the call)
RTP Voice Stream (voice and possibly video stream, also contains DTMF signalling in some cases - inband or oob)The signalling is a separate channel from the rtp data. The rtp data is always UDP. The SIP signalling can be either UDP OR TCP, depending on how your solution is configured.
RFC 4961, section 4 states:
There are two specific instances where symmetric RTP and symmetric
RTCP are REQUIRED:The first instance is NATs that lack integrated Application Layer
Gateway (ALG) functionality. Such NATs require that endpoints use
symmetric UDP ports to establish bidirectional traffic. This
requirement exists for all types of NATs described in Section 4 of
[RFC4787]. ALGs are defined in Section 4.4 of [RFC3022].Other UDP-based protocols can also benefit from common local transmit
and receive ports.There are no known cases where symmetric RTP or symmetric RTCP are
harmful.For these reasons, it is RECOMMENDED that symmetric RTP and symmetric
RTCP always be used for bidirectional RTP media streams.I have observed pfSense using the same port internally as it does externally for RTP streams. However, it does not use the same port as the remote server does.
As for SIP signalling over TCP and UDP, so far I have used purely UDP, although I plan to experiment with TCP. With non-rtp UDP traffic, I don't believe RFC says anything about ports needing to stay the same. In my experience, pfSense uses a random external port with non-rtp udp traffic. I believe this is normal functionality. According to a tech at our voice provider, with SIP signalling the ports on both ends SHOULD BE be the same for proper functionality. I am not sure whether the issue we had is common to all ip voip trunking providers, I would have to see if there is a spec saying anything about the contact field in the sip header. But it seems we worked around the issue of ports not being the same by setting an internal ip for the contact field so that the remote server uses the originating ip/port to return signalling traffic to. So from what I understand, it is good practice to keep the port that the PBX behind the firewall is using for signalling exactly the same on the outside of the firewall. I believe this is known as 'port preservation' in a NAT system.
Now for TCP SIP signalling, I have not had a chance to experiment, but I am assuming that the same functionality will be exhibited with a random port being used on the WAN interface.
Now I am going to attempt to figure out the best way to keep the ports symmetric in pfSense and I will post back. The outgoing static nat settings in pfSense may be the answer, but when I previously tested it, the ports seemed to still be asymmetric. But I now realize that I may need to clear the state table before the change will take effect, so I will try that next.
To work with a NAT device, a voip device on the LAN needs to either:
A. Set the SIP header contact field to its LOCAL LAN IP & PORT
B. Set the SIP header contact field to the EXTERNAL WAN IP & PORTNow there is a big caveat with B - the VOIP device MUST KNOW ITS EXTERNAL PORT. With a symmetric nat, this becomes tricky as the external port will be randomized. In this case, it is best to set the contact field to the internal lan ip and port, because some voip providers will mistakenly try to use the info from the contact field if the external IP matches but the port is not correct because of symmetric nat. In my case, I had the voip device set to use auto-nat. Some devices also use STUN servers to discover their external ip. It seems that these methods discovered the external IP, but did NOT discover the external port properly or at all (it was still using the internal port in the contact field). Even if the external server did discover the external port, the symmetric nat would use a different random port for two different connections so it may not work properly.
Seems like the moral of the story is that VOIP is pretty tricky sometimes.
Anyone else that has info, please chime in. I am not an expert, still learning this stuff. -
…
Now this is all fine and dandy, but in the case of pfSense, some clearing of the air is needed. I read somewhere (unsure of the accuracy of this statement), that pfSense did NOT contain any ALG functionality. However, according to a tech at the voice provider, with SIP the ports on both ends MUST be the same.
...
This leads me to believe that pfSense does indeed have ALG functionality.
I'd like to add a few comments here. What you are describing here, where pfSense randomizes the source port on outbound traffic, is not ALG functionality. ALG, in this case SIP ALG, means manipulating the SIP payload. That would for example mean manipulating the IP address and/port number in the contact header, or manipulating IP addresses and/or port numbers within the SDP of a SIP INVITE request.
You describe that your SIP provider will send return traffic to the port specified in the contact header if the IP address in the contact header matches the IP address for which the SIP request was received from. This is, as far as my knowledge goes, not a behavior which is specified in RFC3261 or other SIP standards, but rather a "proprietary" behavior.
From what you describe about how you managed to get this working, it may seem as your SIP provider supports rport (http://www.ietf.org/rfc/rfc3581.txt).
I believe that if you use manual outbound NAT in pfSense, so that pfSense does not randomize the outgoing source port, but rather uses a port specified by you (Normally 5060), that this would also work since the response from your SIP provider would then come back to a port which you have pre-defined.
Also, when you are quoting RFC 4961, that applies to RTP and RTCP, which is the actual audio/video media packets, which is not related to SIP signaling itself, which was the issue here.
Just my $0.02.
- Andreas
-
…
Now this is all fine and dandy, but in the case of pfSense, some clearing of the air is needed. I read somewhere (unsure of the accuracy of this statement), that pfSense did NOT contain any ALG functionality. However, according to a tech at the voice provider, with SIP the ports on both ends MUST be the same.
...
This leads me to believe that pfSense does indeed have ALG functionality.
I'd like to add a few comments here. What you are describing here, where pfSense randomizes the source port on outbound traffic, is not ALG functionality. ALG, in this case SIP ALG, means manipulating the SIP payload. That would for example mean manipulating the IP address and/port number in the contact header, or manipulating IP addresses and/or port numbers within the SDP of a SIP INVITE request.
You describe that your SIP provider will send return traffic to the port specified in the contact header if the IP address in the contact header matches the IP address for which the SIP request was received from. This is, as far as my knowledge goes, not a behavior which is specified in RFC3261 or other SIP standards, but rather a "proprietary" behavior.
From what you describe about how you managed to get this working, it may seem as your SIP provider supports rport (http://www.ietf.org/rfc/rfc3581.txt).
I believe that if you use manual outbound NAT in pfSense, so that pfSense does not randomize the outgoing source port, but rather uses a port specified by you (Normally 5060), that this would also work since the response from your SIP provider would then come back to a port which you have pre-defined.
Also, when you are quoting RFC 4961, that applies to RTP and RTCP, which is the actual audio/video media packets, which is not related to SIP signaling itself, which was the issue here.
Just my $0.02.
- Andreas
Andreas, thanks for your input. Regarding the misinterpretation of the RFC, I realized my mistake and was in the middle of editing my post when you posted. Please see my revised post. As for the manual outbound NAT, my testing did not work, BUT i feel that may be because the state table needed to be cleared after that change. It seemed that the port that was set in the state table was still being used the whole time. I also checked out the spec for RPORT that you linked - there is no RPORT appearing in the sip headers that I captured.
Here someone basically sums up what I have experienced:
http://qutecom.org/ticket/27It's easy: some providers (as Ekiga) don't allow private address in Contact, this means that the provider doesn't help when the client is behind NAT. This meants that the provider doesn't fix the signalling (by using received public address instead of private "Contact") but that the provider doesn't offer a RTP tunnel (proxy RTP).
In this case the only solution is in the client side which could be:
a) STUN support: If the client is behind a non symmetric NAT router and supports STUN, it can solve by itlsef the signalling and media (by setting public addresses in "Contact" and SDP (retrieved using STUN).
b) Manual port forwarding in the router and setting these values manually in the signalling and SDP sent by the client.In my case, setting the contact field to the private address disqualified it in the providers system, whereas with a publically routable address, the provider attemped to route the return packets to it. It may not be 'if the ip in the contact field matches the source ip' as I stated earlier, and they may just be looking for a private or public ip in there, since IIRC there are certain ip ranges that are always reserved for private network addressing.
I am realizing more and more that this is not really pfSense specific, it's more of a snafu when dealing with SIP and symmetric NAT.
-
Hi All,
I'm definitely getting replies from my voip service provider (type udp, dest = my wan ip:5060) they just aren't getting forwarded by my simple port forward rule (posted previously). I can't see a reason why the pfSense is dropping them. The packet trace (also posted previously) clearly shows the arriving packets on wan but nothing forwarded to lan. Can someone tell me what possible reasons a port forward wouldn't work? I've put firewall rules in place to allow everything on every interface. It looks to me like the forward rule is being ignored for some reason but the packet matches it.
Regards,
Tony -
Having the same problem as the OP. Decided to try upgrading 1.2.3 to 2.0.1. Outbound calls through pfSense work fine, inbound fails with 5060 forwarded to FreePBX server. No point wasting any time. Went back to 1.2.3 as soon as I found this thread.
-
Just to confirm that I was seeing a similar issue with 2.0.1 not forwarding UDP packets on port 5060 to my pbx, but I was only having problem with outbound calls whereby far end can never hangup the call properly (call still connected on my pbx leg until it times out). My provider correctly sends 200 OK for the call to the randomised port, but sends the BYE to port 5060, and pfsense just seems to ignore the latter, so pbx never saw the BYE.
If I were to go to diagnostic>states, and manually delete the firewall state entry (UDP) for an ongoing call, I found that disconnecting the call from far-end then worked fine, so this seems like port forwarding doesn't work if the firewall already have an existing state/connection for the same internal host ip/port.
As pointed out by the OP, creating a static manual outbound rule to 5060 for just my PBX seems to workaround this problem.
.
-
I'm having the same issue, with 2.01 and 2.03. Would be great if this bug could be fixed in the next release.
-
I don't think that the way SIP port is handled in pfsense is considered a bug by the developers. I think they consider it a security feature.
I could be wrong on that but I think they consider "static" port "Bad". Perhaps a button click on the outbound NAT menu to enable "static" on any outbound port 5060, 5061 and 500 without actually having to set up Manual Outbound NAT would be a nice happy middle ground? -
I don't think that the way SIP port is handled in pfsense is considered a bug by the developers.
It isn't.
I think they consider it a security feature.
Not really. More of a "breaks more than it helps" setup. Most phones these days do not need static 5060 on the way out to a remote PBX, only PBXs need that to trunks, and those can break in a lot more ways than just 5060, you really need manual outbound NAT to do static port for all UDP from the local PBX, or 1:1 NAT if you can.
I could be wrong on that but I think they consider "static" port "Bad".
It isn't bad, it just breaks more setups than it helps now. On 1.2.3 it was the other way because the majority needed it back then.
Perhaps a button click on the outbound NAT menu to enable "static" on any outbound port 5060, 5061 and 500 without actually having to set up Manual Outbound NAT would be a nice happy middle ground?
A better compromise is on my todo list for 2.2: http://redmine.pfsense.org/issues/2416
-
That is really nice, and it sounds like it was exactly what he was wanting.
-
Most weird. I'm using pfSense 2.03 since I'm in a production environment and 2.1 is a RC0 version. For me, the port forwarding for SIP & RTP inbound through my VoIP interface has always worked perfectly. The VoIP interface is configured with my public IP & Gateway to my SIP provider. My issue is the exact opposite: no one can make outbound calls. Launch3's posts may hold the key to what's wrong here. Using his two scenarios, my setup would be the one he refers to as "B".
-
Hmmmm…
For me, This is the way I see it.
For all devices inside the LAN that the Asterisk server is on, they don't need to know anything at all about the state of the connection further than the Asterisk server they are connected to. So, they get DNS inside the LAN and they get the LAN (private) IP of the Asterisk server and nothing more.
The Asterisk server needs to know its Behind NAT and it needs to know its private IP as well as its public IP.
Totally different than what the SIP phones need to know.So, on my Asterisk server here at home, it gets:
NAT - YES (This one is behind NAT with a dynamic IP)
Dynamic Host - mydynamicdns.domain.com (If you have purchased a static IP put it here)
Local Networks:
192.168.32.0 / 255.255.255.0 (network the asterisk server is on)
10.159.29.0 / 255.255.255.0
10.159.30.0 / 255.255.255.0
10.159.31.0 / 255.255.255.0 Long list of other local subnets behind my pfsense
10.159.32.0 / 255.255.255.0 Including any VPN subnets I want phones to work from
10.159.33.0 / 255.255.255.0
10.159.34.0 / 255.255.255.0No re-invite
No Jitter BuffersThe only thing I tell the SIP devices is the private IP address of the LAN side of the SIP server, username and password and that includes clients connecting in through VPNs.
Works for incoming and outgoing calls. Has for years.
P.S. Since RTP will always get broken when two layers of NAT are involved anyway, the only port I forward is 5060. Thats all.
I don't bother forwarding 10000 RTP ports hoping that a sip device outside my network will somehow magically work through NAT.
The Reason this works at all is because my SIP server is REGISTERED to a NON-NATed Trunk.
That doesn't work in reverse. If a SIP phone outside my network registers to my server without using VPN audio will be broken.
(I can't wait for IPV6! Can't get here soon enough for me so we can stop worrying all this NAT crap.)I wonder how many "network professionals" internet that just works will un-employ?
-
kejianshi: Please explain how you accomplished this:
I don't bother forwarding 10000 RTP ports hoping that a sip device outside my network will somehow magically work through NAT.
The Reason this works at all is because my SIP server is REGISTERED to a NON-NATed Trunk. -
At the trunks I register with, where the numbers are configured, I have all the incoming calls directed at port 5060, which is the only sip related port I have open, along with the extension/DID of the inbound call. Those hit asterisk which either sends the incoming call to IVR, the phone being called if the inbound DID matches an extension (Could be either SIP or IAX2 extension), or drops the call if its no match on my network. This works well for me because at least my trunk providers are either non-NATed or has a better NAT solution than me. Audio is good for me.
Actually, I'd love to have separate public IPs for every whim I get. It would make things easier, but being poor makes one creative.
-
I just got a PM from somebody having this same problem.
I've not been here in forever! but I am back.
I still believe it's a serious bug.
Like a number of people have stated, it's a braindead portforward and it's not working as expected and as other portforward rules are working.It should not EVER matter what in the doggone SIP headers, I don't know why that was even being discussed.
At the end of the day I think it's this "outbound port randomizer thing" that's causing the headaches and for whatever reason it's only seems to be happening
with UDP5060 SIP.It's not working like most people would expect it to work.
I did get around it as stated earlier by following the early suggestions.-Steve
-
Like a number of people have stated, it's a braindead portforward and it's not working as expected and as other portforward rules are working.
This ^^
At the end of the day I think it's this "outbound port randomizer thing" that's causing the headaches and for whatever reason it's only seems to be happening with UDP5060 SIP.
And this ^^
Are completely unrelated.
The port forward may work fine, but that's inbound NAT. Outbound port randomization/changing is outbound NAT. A port forward does nothing for outbound NAT, and outbound NAT does not control port forwarding.
If you want both in one rule, you use 1:1 NAT, otherwise you use manual outbound NAT and setup static port for the outbound SIP traffic in combination with a port forward.
It should not EVER matter what in the doggone SIP headers, I don't know why that was even being discussed.
Should not, yes, but it does in common setups. Some SIP trunks will send traffic to where the VIA headers say, not trusting the actual source IP/port of the packet. With those setups is where you need static port outbound NAT for your PBX for sure (or 1:1 NAT)
I talked to two people this week that had to have the same thing setup because if you watched a packet capture, the far side SIP trunk was sending the traffic back where it believed it should go, not where it actually came from.
-
Still didn't see a clear answer.
Is this the way the outbound nat rule should look like?WAN 10.0.10.0/24 * * 5060 WAN address * YES SIP - LAN to WAN
My voip server is on the 10.0.10.0 network.
Eric
-
no one ??
-
I must share the frustration expressed by others with this issue.
NAT rule to forward all UDP traffic from the VOIP provider to the Trixbox(Asterisk). Outbound calls from the Trixbox work fine. Inbound trunk calls all fail.
Wireshark monitoring shows SIP INVITEs coming into the WAN interface and NOTHING going out the LAN interface. The packets are being eaten by pfsense. No port rewrites. No nothing.
Tried the suggestions in this thread to no effect. If I had hair…
