Port forward through OpenVPN tunnel

[svarto]

Dear Netgate users,

I have been banging my head against this for awhile now, I am trying to Port Forward the ports for email from a VPS, through an OpenVPN tunnel, to a Virtual Machine in one of my home LAN subnets. Visualizing my problem:

Basically, I can ping back and forth from the Digital Ocean VPS OpenVPN server and to the Mailcow VM. I can access the internet from the Mailcow VM, and also surf to it from outside my own LAN. All that works fine.

The problem is that the port forwarding I have made on the Digital Ocean VPS OpenVPN server does not completely work, I can see through Packet Capture that the requests come to the Mailcow VM, they are returned, but once they are sent out of the Mailcow VM then they just disappear.

What is important for me is that the source ip can be seen by the Mailcow VM, so I can't inbound NAT from the OpenVPN server, I have to keep the source IP in the request to Mailcow VM.

I would be immensely grateful for any help you can give me to get me going in the right direction - sorry for the long post!!!

Using Port 25 as an example, see below TCPDUMP & Packet Capture, iptables, Firewall rules, Outbound NAT (I have none):

Digital Ocean VPS OpenVPN server tun0 interface (i.e. they go in, but nothing comes out):

15:41:41.523972 IP requesting_ip.21412 > 172.16.30.99.25: Flags [SEW], seq 322544945, win 8192, options [mss 1460,nop,wscale 8,nop,nop,sackOK], length 0
15:41:44.524096 IP requesting_ip.21412 > 172.16.30.99.25: Flags [SEW], seq 322544945, win 8192, options [mss 1460,nop,wscale 8,nop,nop,sackOK], length 0
15:41:50.524108 IP requesting_ip.21412 > 172.16.30.99.25: Flags [S], seq 322544945, win 8192, options [mss 1460,nop,nop,sackOK], length 0

pfSense OpenVPN Gateway / extra tab, the one that is automatically created (i.e. it only goes in):

17:41:41.543330 IP requesting_ip.21412 > 172.16.30.99.25: tcp 0
17:41:44.543434 IP requesting_ip.21412 > 172.16.30.99.25: tcp 0
17:41:50.543385 IP requesting_ip.21412 > 172.16.30.99.25: tcp 0

pfSense OpenVPN interface, SVARTO_OPENVPN (i.e. it only goes in):

17:41:41.543330 IP requesting_ip.21412 > 172.16.30.99.25: tcp 0
17:41:44.543434 IP requesting_ip.21412 > 172.16.30.99.25: tcp 0
17:41:50.543385 IP requesting_ip.21412 > 172.16.30.99.25: tcp 0

Mailcow VM tcpdump (i.e. it goes in, and a response is sent out):

17:41:41.537727 IP requesting_ip.21412 > 172.16.30.99.25: Flags [SEW], seq 322544945, win 8192, options [mss 1357,nop,wscale 8,nop,nop,sackOK], length 0
17:41:41.537859 IP 172.16.30.99.25 > requesting_ip.21412: Flags [S.], seq 1075244014, ack 322544946, win 29200, options [mss 1460,nop,nop,sackOK,nop,wscale 7], length 0
17:41:42.544857 IP 172.16.30.99.25 > requesting_ip.21412: Flags [S.], seq 1075244014, ack 322544946, win 29200, options [mss 1460,nop,nop,sackOK,nop,wscale 7], length 0
17:41:44.537756 IP requesting_ip.21412 > 172.16.30.99.25: Flags [SEW], seq 322544945, win 8192, options [mss 1357,nop,wscale 8,nop,nop,sackOK], length 0
17:41:44.537873 IP 172.16.30.99.25 > requesting_ip.21412: Flags [S.], seq 1075244014, ack 322544946, win 29200, options [mss 1460,nop,nop,sackOK,nop,wscale 7], length 0
17:41:46.544887 IP 172.16.30.99.25 > requesting_ip.21412: Flags [S.], seq 1075244014, ack 322544946, win 29200, options [mss 1460,nop,nop,sackOK,nop,wscale 7], length 0
17:41:50.537770 IP requesting_ip.21412 > 172.16.30.99.25: Flags [S], seq 322544945, win 8192, options [mss 1357,nop,nop,sackOK], length 0
17:41:50.537900 IP 172.16.30.99.25 > requesting_ip.21412: Flags [S.], seq 1075244014, ack 322544946, win 29200, options [mss 1460,nop,nop,sackOK,nop,wscale 7], length 0
17:41:54.640968 IP 172.16.30.99.25 > requesting_ip.21412: Flags [S.], seq 1075244014, ack 322544946, win 29200, options [mss 1460,nop,nop,sackOK,nop,wscale 7], length 0
17:42:02.832832 IP 172.16.30.99.25 > requesting_ip.21412: Flags [S.], seq 1075244014, ack 322544946, win 29200, options [mss 1460,nop,nop,sackOK,nop,wscale 7], length 0

Digital Ocean VPS OpenVPN server iptables-save: DO_VPS_iptables-save.txt

Mailcow VM iptables-save: Mailcow_VM_iptables-save.txt

pfSense Outbound NAT - I have no Outbound NAT rules for the Mailcow subnet.

pfSense OpenVPN firewall tab:

pfSense SVARTO_OPENVPN interface firewalls:

pfSense Mailcow interface firewalls:

[Derelict]

The firewall rules on the Home side that pass the port forwarded traffic MUST NOT match a rule on the OpenVPN tab and MUST match a rule on the SVARTO_OPENVPN tab.

[svarto]

@Derelict Thanks a lot for your reply!

I did as you asked, I changed the firewall rules of OpenVPN to not match the incoming port 25 request, and I made the SVARTO_OPENVPN have a rule that matched this request. Unfortunately now it does not reach the Mailcow VM any longer.

OpenVPN tab (nothing matching the port 25 request):

SVARTO_OPENVPN tab (matching the port 25 request):

Now I only see it go to the SVARTO_VPN interface through Packet Capture, but nothing hits the Mailcow VM any longer...

Digital Ocean VPS OpenVPN server tun0 interface:

18:28:54.349965 IP requesting_ip.57556 > 172.16.30.99.25: Flags [SEW], seq 2501216746, win 8192, options [mss 1460,nop,wscale 8,nop,nop,sackOK], length 0
18:28:57.350166 IP requesting_ip.57556 > 172.16.30.99.25: Flags [SEW], seq 2501216746, win 8192, options [mss 1460,nop,wscale 8,nop,nop,sackOK], length 0
18:29:03.350169 IP requesting_ip.57556 > 172.16.30.99.25: Flags [S], seq 2501216746, win 8192, options [mss 1460,nop,nop,sackOK], length 0

pfSense SVARTO_VPN interface:

20:28:54.368054 IP requesting_ip.57556 > 172.16.30.99.25: tcp 0
20:28:57.368268 IP requesting_ip.57556 > 172.16.30.99.25: tcp 0
20:29:03.368235 IP requesting_ip.57556 > 172.16.30.99.25: tcp 0

And nothing on the Mailcow VM tcpdump for port 25...

[Derelict]

Those rules still have zero matches.

[svarto]

@Derelict apologies for being a beginner here, but I don't understand. The request on the SVARTO_VPN interface is:

20:28:54.368054 IP requesting_ip.57556 > 172.16.30.99.25: tcp 0

In other words, Source * to Mailcow Net on Port 25 on any Gateway. This is the first rule that I have on SVARTO_OPENVPN:

I don't understand, I also tried it with specifying the Gateway to be the OpenVPN gateway:

Still no luck..

[Derelict]

That tells me something else is matching those connections. I am probably the wrong person to help you because I simply despise "blocking" traffic with pass ! LOCAL_SUBNETS type rules.

If the traffic is passed from remote OpenVPN sources sourced from arbitrary IP addresses (like those on the internet in general) the connection MUST be passed by a rule on an assigned interface tab. If the traffic is passed by a rule on the OpenVPN interface group tab the connection will not have reply-to on it and reply traffic from the server to which the connection is forwarded will be routed according to the routing table (probably out the WAN that is local to the server being forwarded to.)ng"

[svarto]

@Derelict thank you so much, you gave me the clue to figure this out. Interestingly enough, when a connection comes through VPN it runs first through ALL the rules in the OpenVPN tab. If nothing there matches, then it goes on to the SVARTO_OPENVPN interface tab.

Because I had a catch-all block IPv4 and IPv6 rule in the OpenVPN tab in the Firewalls, it always blocked it. When I disabled those ones, it got to the SVARTO_OPENVPN tab and the reply-to started to work.

Thanks a lot!!

PS. Quick question, how do you write your "Internet Access" rules? Or do you more define what you DENY access to rather than ALLOW access to in your rules? I am still learning so very keen to hear how others do it.

[Derelict]

it is not interesting at all. It is the known and documented rule processing order.

1. Floating rules
2. Interface group rules (of which OpenVPN is one)
3. Interface rules

[Derelict]

In that case I would BLOCK LOCAL_SUBNETS then PASS ANY