Sip invite packets dropped after random time(fixed by reboot of pfsense)
-
Not a solution but a confirmation of the problem.
I too have INVITEs being dropped after the firewall has supposedly passed them.
WAN tcpdump shows packets arriving, firewall rule is logged and shows a PASS for each one, LAN interface on pfsense and on my asterisk server show no trace of the packets.
Curiously, normal SIP OPTIONS and OK traffic seems to work while all this is going on.
Like you I would welcome help doing some debugging. Clearing the filters sorts it out for a while then it happens again. Most annoying!
Jon
-
Can post all rules on the box?
pfctl -sr pfctl -sn -
I have put stars in my public ip address but it starts with 70...*
pfctl -sr
scrub in on xl0 all fragment reassemble
scrub in on xl1 all fragment reassemble
anchor "relayd/" all
block drop in log all label "Default deny rule"
block drop out log all label "Default deny rule"
block drop in quick inet6 all
block drop out quick inet6 all
block drop quick proto tcp from any port = 0 to any
block drop quick proto tcp from any to any port = 0
block drop quick proto udp from any port = 0 to any
block drop quick proto udp from any to any port = 0
block drop quick from <snort2c>to any label "Block snort2c hosts"
block drop quick from any to <snort2c>label "Block snort2c hosts"
block drop quick from <pfsnortsamout>to any label "Block pfSnortSamOut hosts"
block drop quick from any to <pfsnortsamin>label "Block pfSnortSamIn hosts"
block drop in log quick proto carp from (self) to any
pass quick proto carp all keep state
pass quick proto pfsync all keep state
block drop in log quick proto tcp from <sshlockout>to any port = ssh label "sshlockout"
block drop in log quick proto tcp from <webconfiguratorlockout>to any port = https label "webConfiguratorlockout"
block drop in quick from <virusprot>to any label "virusprot overload table"
block drop in on ! xl0 inet from 192.168.100.0/24 to any
block drop in inet from 192.168.100.1 to any
block drop in on xl0 inet6 from fe80::250:daff:fecc:fee5 to any
pass in on xl0 inet proto udp from any port = bootpc to 255.255.255.255 port = bootps keep state label "allow access to DHCP server"
pass in on xl0 inet proto udp from any port = bootpc to 192.168.100.1 port = bootps keep state label "allow access to DHCP server"
pass out on xl0 inet proto udp from 192.168.100.1 port = bootps to any port = bootpc keep state label "allow access to DHCP server"
block drop in log quick on xl1 from <bogons>to any label "block bogon networks from WAN"
block drop in on ! xl1 inet from 70.../29 to any
block drop in inet from 70...* to any
block drop in on xl1 inet6 from fe80::2b0:d0ff:fe72:f90b to any
block drop in log quick on xl1 inet from 10.0.0.0/8 to any label "block private networks from wan block 10/8"
block drop in log quick on xl1 inet from 127.0.0.0/8 to any label "block private networks from wan block 127/8"
block drop in log quick on xl1 inet from 172.16.0.0/12 to any label "block private networks from wan block 172.16/12"
block drop in log quick on xl1 inet from 192.168.0.0/16 to any label "block private networks from wan block 192.168/16"
pass in on lo0 all flags S/SA keep state label "pass loopback"
pass out on lo0 all flags S/SA keep state label "pass loopback"
pass out all flags S/SA keep state allow-opts label "let out anything from firewall host itself"
pass out route-to (xl1 70...) inet from 70... to ! 70.../29 flags S/SA keep state allow-opts label "let out anything from firewall host itself"
pass in quick on xl0 proto tcp from any to (xl0) port = http flags S/SA keep state label "anti-lockout rule"
pass in quick on xl0 proto tcp from any to (xl0) port = https flags S/SA keep state label "anti-lockout rule"
pass in quick on xl0 proto tcp from any to (xl0) port = ssh flags S/SA keep state label "anti-lockout rule"
block drop in quick on xl1 reply-to (xl1 70...) inet from 212.117.179.211 to any label "USER_RULE: someone was brute forcing me."
pass in quick on xl0 inet from 192.168.100.0/24 to any flags S/SA keep state label "USER_RULE: Default LAN -> any"
pass in quick on xl0 inet proto tcp from any to 192.168.100.5 port = domain flags S/SA keep state label "USER_RULE: NAT internal dns"
pass in quick on xl0 inet proto udp from any to 192.168.100.5 port = domain keep state label "USER_RULE: NAT internal dns"
anchor "tftp-proxy/*" all
anchor "miniupnpd" allpfctl -sn
nat-anchor "natearly/" all
nat-anchor "natrules/" all
nat on xl1 inet from 192.168.100.0/24 to any -> 70...* port 1024:65535
rdr-anchor "relayd/" all
rdr-anchor "tftp-proxy/" all
rdr-anchor "miniupnpd" all</bogons></virusprot></webconfiguratorlockout></sshlockout></pfsnortsamin></pfsnortsamout></snort2c></snort2c> -
I don't see any SIP related filter/nat rules. Can you point them out for me please?
-
Ahh, I took those in the middle of dropping and restoring the rules.
rdr on xl1 inet proto tcp from any to 70...* port 5060:5080 -> 192.168.100.20
rdr on xl1 inet proto udp from any to 70...* port 5060:5080 -> 192.168.100.20 -
Could you please post full tables? just output of mentioned above commands without any editing… thanks.
-
Unedited rules sent as a PM.
-
As I understand your SIP gateway (or whatever you call it) is 192.168.100.20. I see rules for both UDP and TCP, when you see this issue with SIP packets - what protocol is used? do you see a pause in coming packets (let's say no traffic at all for some time) that would expire state? Can you send me a capture anyway?
Thanks. -
The sip traffic is coming in as only UDP. The TCP rule was for testing. When the traffic gets dropped there is no network outage, the only thing that I have seen stop working is the sip traffic.
-
Can you post output of the next when you do not have problem and when you do```
pfctl -ss | grep 5060 -
I had the same filters exactly when it did work and when it did not work. I have a quick rule that I add and delete to bounce the firewall to fix the issue.
-
The reason I asked about filter: when troubleshoot things like this capture everything (even if it is quite a traffic) then when you open the file apply necessary filter rules if needed - this way you can see events that might indirectly affect SIP traffic, for example packets go through but nat does not happen properly or carp switches or … Even if examples look silly to you trust me very often capturing all traffic helps understand what is really going on.
So, to troubleshoot further I'd like to have two things
1. all packets captures from LAN and WAN for 2 minutes when it is not working + output of```
pfctl -sr
pfctl -sn
pfpfctl -ss2\. Add/delete a rule to fix the problem and again: all packets captures from LAN and WAN for 2 minutes when it is working + output of``` pfctl -sr pfctl -sn pfpfctl -ssit's up to you whether you take this way or not -)
Cheers! -
Forgot to ask - is there a reason you use CARP IP to terminate SIP traffic? I do not see many interfaces…
-
There is no particular reason I am using carp ip.
I will make the captures you asked for the next time I see the issue.
-
Is there a way to capture from both lan and wan from the command line on the pfsense router itself?
-
You can capture on both by having two open ssh sessions and doing one capture on each interface in each session.
-
You can capture on both by having two open ssh sessions and doing one capture on each interface in each session.
… or running something like
tcpdump -ni <lan> -s0 -w lan.cap & tcpdump -ni <wan> -s0 -w wan.cap</wan></lan>After you are done don't forget to kill the first session.
-
That works too, but I've found that separate windows makes it a bit easier logically to follow what's being done.
Also, you could use "fg" to get back to the backgrounded tcpdump in your example and then ^C it, rather than finding and killing the process.
-
Alright, I have been able to capture the issue. I am working on getting the info ready.
-
Alright, I have been able to capture the issue. I am working on getting the info ready.
I am sorry, this is what I asked some time ago:
@Evgeny:So, to troubleshoot further I'd like to have two things
1. all packets captures from LAN and WAN for 2 minutes when it is not working + output of```
pfctl -sr
pfctl -sn
pfpfctl -ss2\. Add/delete a rule to fix the problem and again: all packets captures from LAN and WAN for 2 minutes when it is working + output of``` pfctl -sr pfctl -sn pfpfctl -ssYou gave me just two captures from WAN interface (no captures from LAN) 15 minutes apart from each other. It is impossible to say anything here except that you are using CARP and probably at that moment there was a problem with Active node (switchover did not occur or whatever) -(
Can you provide full details (see above) and plusifconfig
