Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    Routing specific LAN segment via OpenVPN tunnel

    Scheduled Pinned Locked Moved Routing and Multi WAN
    35 Posts 5 Posters 4.5k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • K
      kroem
      last edited by

      @marvosa:

      Ok, OVPN_99 is a VLAN, gotcha.

      Just curious, what is in "pfB ET comprblock Spamhaus drop" alias?

      That's a pfBlocker list. I only recently added it, it was the same without it.

      @marvosa:

      Something's not adding up… because according to what I'm looking at, there are no rules allowing outbound traffic to the internet at all, so I'm not sure how ANY traffic sourced from this VLAN (99) is making it anywhere.  It should all be blocked by the firewall.

      Also, if you only want this VLAN accessible and routed thru your VPN, then why are there rules configured with the default gateway on this tab?

      But there is, actually any traffic from the network 10.99.1.0/24 is allowed out to Internet via OVPN Gateway. I've tried a any/any (source any) rule going to the OVPN gateway and it's not working either.
      The local nets rule is to access it from my other networks.

      @marvosa:

      Another mystery, rule #3 (at the bottom), reads… allow traffic sourced from VLAN99 destined to other LAN interfaces and route that traffic over the VPN?  I'm not sure why that's there..... why would you route local traffic between two interfaces on the same machine over a VPN tunnel?

      No, that's sourced from VLAN99 not to other local nets. I.e. LAN traffic.

      @marvosa:

      I know you've already stated this, but I'm going to ask this specific question anyway because I hate to assume anything… is there any chance the rules you've listed are actually configured on the interface assigned to the tunnel instead of the VLAN interface?  Please post the rules from both rules from both the VLAN interface and the interface assigned to the tunnel.

      No, it's the VLAN interface :(

      I tried adding the OVPN gateway on the tunnel interface rule - no difference (as a test). The OVPN rules are attached.

      @marvosa:

      I also just noticed that your "LOCAL NETS" alias has vlan99 listed at 10.99.0.0/24, but your NAT mappings show "OVPN_99" lists the source as 10.99.1.0/24.  Is there a simple typo in your alias?  Or is there another reason for this discrepancy?

      Typo from me! godfuckingdammit - that's like the 5th time  :-[ :-[ Anyways, it's the same with the correct subnet.


      As a reference - here's the tcpdump from OpenVPN interface, VLAN and WAN interface. First. I'm trying to establish a iperf session towards my VM1 host, then I ping out from VM1 to the server which tried to establish connection:

      OVPN_99 interface, traffic flowing "correctly"
      [code]

      [2.3.3-RELEASE][root@fw-pf.kroem.eu]/root: tcpdump -i igb1_vlan99 -nn host 80.239.147.29
      tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
      listening on igb1_vlan99, link-type EN10MB (Ethernet), capture size 65535 bytes
      22:10:55.944376 IP 80.239.147.29.42200 > 10.99.1.201.59362: Flags ~~, seq 3526785419, win 29200, options [mss 1349,sackOK,TS val 34928838 ecr 0,nop,wscale 7], length 0
      22:10:55.944659 IP 10.99.1.201.59362 > 80.239.147.29.42200: Flags [S.], seq 2629731937, ack 3526785420, win 28960, options [mss 1460,sackOK,TS val 33974369 ecr 34928838,nop,wscale 7], length 0
      22:10:56.943438 IP 80.239.147.29.42200 > 10.99.1.201.59362: Flags ~~, seq 3526785419, win 29200, options [mss 1349,sackOK,TS val 34929088 ecr 0,nop,wscale 7], length 0
      22:10:56.943652 IP 10.99.1.201.59362 > 80.239.147.29.42200: Flags [S.], seq 2629731937, ack 3526785420, win 28960, options [mss 1460,sackOK,TS val 33974619 ecr 34928838,nop,wscale 7], length 0
      22:10:58.141497 IP 10.99.1.201.59362 > 80.239.147.29.42200: Flags [S.], seq 2629731937, ack 3526785420, win 28960, options [mss 1460,sackOK,TS val 33974919 ecr 34928838,nop,wscale 7], length 0
      22:10:58.947546 IP 80.239.147.29.42200 > 10.99.1.201.59362: Flags ~~, seq 3526785419, win 29200, options [mss 1349,sackOK,TS val 34929589 ecr 0,nop,wscale 7], length 0
      22:10:58.947812 IP 10.99.1.201.59362 > 80.239.147.29.42200: Flags [S.], seq 2629731937, ack 3526785420, win 28960, options [mss 1460,sackOK,TS val 33975120 ecr 34928838,nop,wscale 7], length 0
      22:11:01.341273 IP 10.99.1.201.59362 > 80.239.147.29.42200: Flags [S.], seq 2629731937, ack 3526785420, win 28960, options [mss 1460,sackOK,TS val 33975719 ecr 34928838,nop,wscale 7], length 0
      22:11:02.955574 IP 10.99.1.201 > 80.239.147.29: ICMP echo request, id 2099, seq 1, length 64
      22:11:02.977335 IP 80.239.147.29 > 10.99.1.201: ICMP echo reply, id 2099, seq 1, length 64
      22:11:03.956783 IP 10.99.1.201 > 80.239.147.29: ICMP echo request, id 2099, seq 2, length 64
      22:11:03.978472 IP 80.239.147.29 > 10.99.1.201: ICMP echo reply, id 2099, seq 2, length 64
      22:11:04.957965 IP 10.99.1.201 > 80.239.147.29: ICMP echo request, id 2099, seq 3, length 64
      22:11:04.979579 IP 80.239.147.29 > 10.99.1.201: ICMP echo reply, id 2099, seq 3, length 64
      22:11:05.959203 IP 10.99.1.201 > 80.239.147.29: ICMP echo request, id 2099, seq 4, length 64
      22:11:05.980780 IP 80.239.147.29 > 10.99.1.201: ICMP echo reply, id 2099, seq 4, length 64

      OpenVPN tunnel interface, you see the traffic incoming from the remote host trying to establish connection, but no replies. The ping request and replies are both here:

      
      [2.3.3-RELEASE][root@fw-pf.kroem.eu]/root: tcpdump -i ovpnc3 -nn host 80.239.147.29
      tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
      listening on ovpnc3, link-type NULL (BSD loopback), capture size 65535 bytes
      22:10:55.944344 IP 80.239.147.29.42200 > 10.128.164.2.59362: Flags [s], seq 3526785419, win 29200, options [mss 1349,sackOK,TS val 34928838 ecr 0,nop,wscale 7], length 0
      22:10:56.943427 IP 80.239.147.29.42200 > 10.128.164.2.59362: Flags [s], seq 3526785419, win 29200, options [mss 1349,sackOK,TS val 34929088 ecr 0,nop,wscale 7], length 0
      22:10:58.947523 IP 80.239.147.29.42200 > 10.128.164.2.59362: Flags [s], seq 3526785419, win 29200, options [mss 1349,sackOK,TS val 34929589 ecr 0,nop,wscale 7], length 0
      22:11:02.955619 IP 10.128.164.2 > 80.239.147.29: ICMP echo request, id 31135, seq 1, length 64
      22:11:02.977311 IP 80.239.147.29 > 10.128.164.2: ICMP echo reply, id 31135, seq 1, length 64
      22:11:03.956792 IP 10.128.164.2 > 80.239.147.29: ICMP echo request, id 31135, seq 2, length 64
      22:11:03.978450 IP 80.239.147.29 > 10.128.164.2: ICMP echo reply, id 31135, seq 2, length 64
      22:11:04.957975 IP 10.128.164.2 > 80.239.147.29: ICMP echo request, id 31135, seq 3, length 64
      22:11:04.979558 IP 80.239.147.29 > 10.128.164.2: ICMP echo reply, id 31135, seq 3, length 64
      22:11:05.959207 IP 10.128.164.2 > 80.239.147.29: ICMP echo request, id 31135, seq 4, length 64
      22:11:05.980760 IP 80.239.147.29 > 10.128.164.2: ICMP echo reply, id 31135, seq 4, length 64
      
      WAN interface, only the replies to the iperf session is here...
      [code]
      [2.3.3-RELEASE][root@fw-pf.kroem.eu]/root: tcpdump -i igb0 -nn host 80.239.147.29 and port not ssh
      tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
      listening on igb0, link-type EN10MB (Ethernet), capture size 65535 bytes
      22:10:55.944666 IP 10.128.164.2.59362 > 80.239.147.29.42200: Flags [S.], seq 2629731937, ack 3526785420, win 28960, options [mss 1460,sackOK,TS val 33974369 ecr 34928838,nop,wscale 7], length 0
      22:10:56.943658 IP 10.128.164.2.59362 > 80.239.147.29.42200: Flags [S.], seq 2629731937, ack 3526785420, win 28960, options [mss 1460,sackOK,TS val 33974619 ecr 34928838,nop,wscale 7], length 0
      22:10:58.141509 IP 10.128.164.2.59362 > 80.239.147.29.42200: Flags [S.], seq 2629731937, ack 3526785420, win 28960, options [mss 1460,sackOK,TS val 33974919 ecr 34928838,nop,wscale 7], length 0
      22:10:58.947823 IP 10.128.164.2.59362 > 80.239.147.29.42200: Flags [S.], seq 2629731937, ack 3526785420, win 28960, options [mss 1460,sackOK,TS val 33975120 ecr 34928838,nop,wscale 7], length 0
      22:11:01.341281 IP 10.128.164.2.59362 > 80.239.147.29.42200: Flags [S.], seq 2629731937, ack 3526785420, win 28960, options [mss 1460,sackOK,TS val 33975719 ecr 34928838,nop,wscale 7], length 0
      ^C
      [/code]
      
      It does look strange...
      
      ![Skärmklipp 2017-05-25 22.02.41.png](/public/_imported_attachments_/1/Skärmklipp 2017-05-25 22.02.41.png)
      ![Skärmklipp 2017-05-25 22.02.41.png_thumb](/public/_imported_attachments_/1/Skärmklipp 2017-05-25 22.02.41.png_thumb)[/s][/s][/s]
      ```~~~~~~
      1 Reply Last reply Reply Quote 0
      • K
        kroem
        last edited by

        @CuteBoi:

        https://forum.pfsense.org/index.php?topic=131074.0

        This smells like the same exact issue I'm having.  Word for word, port forwarding, the packets are attempting to go out my WAN, while the 2 hosts on the LAN that have rules assigning the VPN gateway work as intended.

        I'll follow this thread as well.

        Seems fishy - however, I'm on 2.3.3, if that really makes a difference. Dod you setup work before you upgraded?

        1 Reply Last reply Reply Quote 0
        • johnpozJ
          johnpoz LAYER 8 Global Moderator
          last edited by

          Why are you doing a pfblocker rule on your vpn connection… In what scenario would you have traffic you don't want coming from some spam source via your vpn connection??

          I am confused on how your seeing traffic on your vpn tunnel with that private address 80.x.x.x

          So your using using public IP space on one end of this tunnel?  Your vpn sniff on that interface should only show your internal IPs
          15:24:06.293864 IP 10.0.8.100 > 192.168.9.100: ICMP echo request, id 1, seq 1086, length 40
          15:24:06.294602 IP 192.168.9.100 > 10.0.8.100: ICMP echo reply, id 1, seq 1086, length 40
          15:24:07.282752 IP 10.0.8.100 > 192.168.9.100: ICMP echo request, id 1, seq 1087, length 40
          15:24:07.283160 IP 192.168.9.100 > 10.0.8.100: ICMP echo reply, id 1, seq 1087, length 40
          15:24:08.289218 IP 10.0.8.100 > 192.168.9.100: ICMP echo request, id 1, seq 1088, length 40
          15:24:08.289814 IP 192.168.9.100 > 10.0.8.100: ICMP echo reply, id 1, seq 1088, length 40
          15:24:09.295496 IP 10.0.8.100 > 192.168.9.100: ICMP echo request, id 1, seq 1089, length 40
          15:24:09.295787 IP 192.168.9.100 > 10.0.8.100: ICMP echo reply, id 1, seq 1089, length 40

          An intelligent man is sometimes forced to be drunk to spend time with his fools
          If you get confused: Listen to the Music Play
          Please don't Chat/PM me for help, unless mod related
          SG-4860 24.11 | Lab VMs 2.8, 24.11

          1 Reply Last reply Reply Quote 0
          • K
            kroem
            last edited by

            @johnpoz:

            Why are you doing a pfblocker rule on your vpn connection… In what scenario would you have traffic you don't want coming from some spam source via your vpn connection??

            I am confused on how your seeing traffic on your vpn tunnel with that private address 80.x.x.x

            So your using using public IP space on one end of this tunnel?  Your vpn sniff on that interface should only show your internal IPs
            15:24:06.293864 IP 10.0.8.100 > 192.168.9.100: ICMP echo request, id 1, seq 1086, length 40
            15:24:06.294602 IP 192.168.9.100 > 10.0.8.100: ICMP echo reply, id 1, seq 1086, length 40
            15:24:07.282752 IP 10.0.8.100 > 192.168.9.100: ICMP echo request, id 1, seq 1087, length 40
            15:24:07.283160 IP 192.168.9.100 > 10.0.8.100: ICMP echo reply, id 1, seq 1087, length 40
            15:24:08.289218 IP 10.0.8.100 > 192.168.9.100: ICMP echo request, id 1, seq 1088, length 40
            15:24:08.289814 IP 192.168.9.100 > 10.0.8.100: ICMP echo reply, id 1, seq 1088, length 40
            15:24:09.295496 IP 10.0.8.100 > 192.168.9.100: ICMP echo request, id 1, seq 1089, length 40
            15:24:09.295787 IP 192.168.9.100 > 10.0.8.100: ICMP echo reply, id 1, seq 1089, length 40

            Well it's not a site to site vpn, it's a vpn tunnel wan service, so basically a privacy service. The openvpn server is not hosted on the machine I'm establishing a connection from, it's hosted in a DC in the middle.

            Regarding the pfblockerng rule, since it's a WAN connection, I want the same ruleset for this interface as any other. Anyways I've tried with and without it - no difference.

            1 Reply Last reply Reply Quote 0
            • M
              marvosa
              last edited by

              Unless this is a bug (which it very well may be), my guess is the firewall rules are just over complicated and not being followed like you'd expect.

              What VPN service are you using?  I'd like to test the same scenario on my end.

              1 Reply Last reply Reply Quote 0
              • K
                kroem
                last edited by

                @marvosa:

                Unless this is a bug (which it very well may be), my guess is the firewall rules are just over complicated and not being followed like you'd expect.

                What VPN service are you using?  I'd like to test the same scenario on my end.

                Yes, but I need to know how the packet forwarding differs and what rules are being looked at differently when traffic is established from the outside versus inside.

                The service is OVPN (https://www.ovpn.com/en) you can get a test account for a few hours I believe…

                1 Reply Last reply Reply Quote 0
                • johnpozJ
                  johnpoz LAYER 8 Global Moderator
                  last edited by

                  "it's a vpn tunnel wan service"

                  So in that case you would be natting to what the exit point is on that vpn service.  If your not seeing return traffic..

                  "if I try to access a service running on a VM within VLAN99 externally"

                  This would have to be done via a port forward on the vpn service.. Your asking for unsolicited traffic to be sent down their tunnel.. Which you would then have to port forward on pfsense to what IP you want it to go to..  With a vpn privacy service your going to be behind a double nat..

                  So your client

                  192.168.1.100 –-> pfsense privateTunnelIP ---- vpn ----> VPNservice --- VPNpublicIP ----> internet

                  So to the internet you look like VPNpublicIP, answers get sent back to it, vpnservice state tables says oh that is privateTunnelIP so gets sent back to pfsense, pfsense says oh that is 192.168.1.100

                  What your asking for is unsolicated traffic..

                  Im on the internet and I talk to VPNpublicIP on port X.. That service has to have a port forward that says hey send that to privatedTunnelIP on pfsense.  Pfsense has to have a port forward that says oh traffic hitting me on my privateTunnelIP on port X gets forwarded to IP 192.168.1.?

                  An intelligent man is sometimes forced to be drunk to spend time with his fools
                  If you get confused: Listen to the Music Play
                  Please don't Chat/PM me for help, unless mod related
                  SG-4860 24.11 | Lab VMs 2.8, 24.11

                  1 Reply Last reply Reply Quote 0
                  • K
                    kroem
                    last edited by

                    @johnpoz:

                    "it's a vpn tunnel wan service"

                    So in that case you would be natting to what the exit point is on that vpn service.  If your not seeing return traffic..

                    "if I try to access a service running on a VM within VLAN99 externally"

                    This would have to be done via a port forward on the vpn service.. Your asking for unsolicited traffic to be sent down their tunnel.. Which you would then have to port forward on pfsense to what IP you want it to go to..  With a vpn privacy service your going to be behind a double nat..

                    So your client

                    192.168.1.100 –-> pfsense privateTunnelIP ---- vpn ----> VPNservice --- VPNpublicIP ----> internet

                    So to the internet you look like VPNpublicIP, answers get sent back to it, vpnservice state tables says oh that is privateTunnelIP so gets sent back to pfsense, pfsense says oh that is 192.168.1.100

                    What your asking for is unsolicated traffic..

                    Im on the internet and I talk to VPNpublicIP on port X.. That service has to have a port forward that says hey send that to privatedTunnelIP on pfsense.  Pfsense has to have a port forward that says oh traffic hitting me on my privateTunnelIP on port X gets forwarded to IP 192.168.1.?

                    Yes, the traffic is port forwarded on the VPN service provider ingress, it's routed via my tunnel and port forwarded to the local host.

                    All seems fine here, since I do see traffic incoming (as per the tcpdumps).

                    The same for an established session from local host.

                    1 Reply Last reply Reply Quote 0
                    • C
                      CuteBoi
                      last edited by

                      Per traffic in the tcpdump, the traffic IS arriving to the VPN ip, and arrives at the private IP in his vlan 99 group, and even responds, but the response is attempting to go out through the WAN, instead of the VPN gateway.

                      Yet he has the rules right there to use the gateway.

                      1 Reply Last reply Reply Quote 0
                      • johnpozJ
                        johnpoz LAYER 8 Global Moderator
                        last edited by

                        "but the response is attempting to go out through the WAN, instead of the VPN gateway."

                        Where are you seeing that???  The tcpdumps do not show that..

                        This is the unsolicited traffic, the way I see it

                        
                        22:10:55.944344 IP 80.239.147.29.42200 > 10.128.164.2.59362: Flags [s], seq 3526785419, win 29200, options [mss 1349,sackOK,TS val 34928838 ecr 0,nop,wscale 7], length 0
                        22:10:56.943427 IP 80.239.147.29.42200 > 10.128.164.2.59362: Flags [s], seq 3526785419, win 29200, options [mss 1349,sackOK,TS val 34929088 ecr 0,nop,wscale 7], length 0
                        22:10:58.947523 IP 80.239.147.29.42200 > 10.128.164.2.59362: Flags [s], seq 3526785419, win 29200, options [mss 1349,sackOK,TS val 34929589 ecr 0,nop,wscale 7], length 
                        
                        So where is his port forward on 10.128.164.2 (his vpn interface?) to send port 59362 his inside box on vlan99
                        
                        "and port forwarded to the local host. "
                        
                        Where is your port forward?  And where is your sniff on your vlan99 interface showing this traffic being forwarded to whatever IP on the inside??[/s][/s][/s]
                        

                        An intelligent man is sometimes forced to be drunk to spend time with his fools
                        If you get confused: Listen to the Music Play
                        Please don't Chat/PM me for help, unless mod related
                        SG-4860 24.11 | Lab VMs 2.8, 24.11

                        1 Reply Last reply Reply Quote 0
                        • K
                          kroem
                          last edited by

                          @johnpoz:

                          "but the response is attempting to go out through the WAN, instead of the VPN gateway."

                          Where are you seeing that???  The tcpdumps do not show that..

                          This is the unsolicited traffic, the way I see it

                          
                          22:10:55.944344 IP 80.239.147.29.42200 > 10.128.164.2.59362: Flags [s], seq 3526785419, win 29200, options [mss 1349,sackOK,TS val 34928838 ecr 0,nop,wscale 7], length 0
                          22:10:56.943427 IP 80.239.147.29.42200 > 10.128.164.2.59362: Flags [s], seq 3526785419, win 29200, options [mss 1349,sackOK,TS val 34929088 ecr 0,nop,wscale 7], length 0
                          22:10:58.947523 IP 80.239.147.29.42200 > 10.128.164.2.59362: Flags [s], seq 3526785419, win 29200, options [mss 1349,sackOK,TS val 34929589 ecr 0,nop,wscale 7], length 
                          
                          So where is his port forward on 10.128.164.2 (his vpn interface?) to send port 59362 his inside box on vlan99
                          
                          That's the incoming traffic, 80.239.147.29 is the external host. 10.128.164.2 is the OpenVPN tunnel network's IP.
                          
                          Could you explain what do you mean by unsolicited traffic? The traffic 80.239.147.29.42200 > 10.128.164.2.59362 is what I want, I just want the return path to go the same way... [/s][/s][/s]
                          
                          1 Reply Last reply Reply Quote 0
                          • C
                            CuteBoi
                            last edited by

                            @kroem:

                            WAN interface, only the replies to the iperf session is here…

                            
                            [2.3.3-RELEASE][root@fw-pf.kroem.eu]/root: tcpdump -i igb0 -nn host 80.239.147.29 and port not ssh
                            tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
                            listening on igb0, link-type EN10MB (Ethernet), capture size 65535 bytes
                            22:10:55.944666 IP 10.128.164.2.59362 > 80.239.147.29.42200: Flags [S.], seq 2629731937, ack 3526785420, win 28960, options [mss 1460,sackOK,TS val 33974369 ecr 34928838,nop,wscale 7], length 0
                            22:10:56.943658 IP 10.128.164.2.59362 > 80.239.147.29.42200: Flags [S.], seq 2629731937, ack 3526785420, win 28960, options [mss 1460,sackOK,TS val 33974619 ecr 34928838,nop,wscale 7], length 0
                            22:10:58.141509 IP 10.128.164.2.59362 > 80.239.147.29.42200: Flags [S.], seq 2629731937, ack 3526785420, win 28960, options [mss 1460,sackOK,TS val 33974919 ecr 34928838,nop,wscale 7], length 0
                            22:10:58.947823 IP 10.128.164.2.59362 > 80.239.147.29.42200: Flags [S.], seq 2629731937, ack 3526785420, win 28960, options [mss 1460,sackOK,TS val 33975120 ecr 34928838,nop,wscale 7], length 0
                            22:11:01.341281 IP 10.128.164.2.59362 > 80.239.147.29.42200: Flags [S.], seq 2629731937, ack 3526785420, win 28960, options [mss 1460,sackOK,TS val 33975719 ecr 34928838,nop,wscale 7], length 0
                            ^C
                            
                            

                            This is his WAN interface tcpdump.

                            1 Reply Last reply Reply Quote 0
                            • johnpozJ
                              johnpoz LAYER 8 Global Moderator
                              last edited by

                              "The traffic 80.239.147.29.42200 > 10.128.164.2.59362"

                              Yes this coming into your vpn interface.

                              So sniff your vlan99 interface when you do that, is your port forward sending it out the vlan99 interface to your client and is your client answering back to pfsense?

                              You could have an asymmetrical issue where pfsense sends it on to your host on your vlan99..  But vlan99 host is sending it to a different IP as its gateway?

                              Also it seems you have multiple threads about the same topic??  You should have the mods merge them!!!

                              Have you posted your port forwards for your vpn interface?

                              An intelligent man is sometimes forced to be drunk to spend time with his fools
                              If you get confused: Listen to the Music Play
                              Please don't Chat/PM me for help, unless mod related
                              SG-4860 24.11 | Lab VMs 2.8, 24.11

                              1 Reply Last reply Reply Quote 0
                              • M
                                marvosa
                                last edited by

                                I think it would also be helpful to see the rules from the "OPT" interface assigned to the tunnel as well.

                                1 Reply Last reply Reply Quote 0
                                • K
                                  kroem
                                  last edited by

                                  @johnpoz:

                                  "The traffic 80.239.147.29.42200 > 10.128.164.2.59362"

                                  Yes this coming into your vpn interface.

                                  So sniff your vlan99 interface when you do that, is your port forward sending it out the vlan99 interface to your client and is your client answering back to pfsense?

                                  I appreciate your help - but really, that information is already in the thread :) I have a TCPdump of my VLAN99 interface above, you can see that packets are being sent between the same IP's when it's "working" (ping test outbound) and not working (Iperf session inbound)

                                  [2.3.3-RELEASE][root@fw-pf.kroem.eu]/root: tcpdump -i igb1_vlan99 -nn host 80.239.147.29
                                  tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
                                  listening on igb1_vlan99, link-type EN10MB (Ethernet), capture size 65535 bytes
                                  22:10:55.944376 IP 80.239.147.29.42200 > 10.99.1.201.59362: Flags [s], seq 3526785419, win 29200, options [mss 1349,sackOK,TS val 34928838 ecr 0,nop,wscale 7], length 0
                                  22:10:55.944659 IP 10.99.1.201.59362 > 80.239.147.29.42200: Flags [S.], seq 2629731937, ack 3526785420, win 28960, options [mss 1460,sackOK,TS val 33974369 ecr 34928838,nop,wscale 7], length 0
                                  22:10:56.943438 IP 80.239.147.29.42200 > 10.99.1.201.59362: Flags [s], seq 3526785419, win 29200, options [mss 1349,sackOK,TS val 34929088 ecr 0,nop,wscale 7], length 0
                                  22:10:56.943652 IP 10.99.1.201.59362 > 80.239.147.29.42200: Flags [S.], seq 2629731937, ack 3526785420, win 28960, options [mss 1460,sackOK,TS val 33974619 ecr 34928838,nop,wscale 7], length 0
                                  22:10:58.141497 IP 10.99.1.201.59362 > 80.239.147.29.42200: Flags [S.], seq 2629731937, ack 3526785420, win 28960, options [mss 1460,sackOK,TS val 33974919 ecr 34928838,nop,wscale 7], length 0
                                  22:10:58.947546 IP 80.239.147.29.42200 > 10.99.1.201.59362: Flags [s], seq 3526785419, win 29200, options [mss 1349,sackOK,TS val 34929589 ecr 0,nop,wscale 7], length 0
                                  22:10:58.947812 IP 10.99.1.201.59362 > 80.239.147.29.42200: Flags [S.], seq 2629731937, ack 3526785420, win 28960, options [mss 1460,sackOK,TS val 33975120 ecr 34928838,nop,wscale 7], length 0
                                  22:11:01.341273 IP 10.99.1.201.59362 > 80.239.147.29.42200: Flags [S.], seq 2629731937, ack 3526785420, win 28960, options [mss 1460,sackOK,TS val 33975719 ecr 34928838,nop,wscale 7], length 0
                                  22:11:02.955574 IP 10.99.1.201 > 80.239.147.29: ICMP echo request, id 2099, seq 1, length 64
                                  22:11:02.977335 IP 80.239.147.29 > 10.99.1.201: ICMP echo reply, id 2099, seq 1, length 64
                                  22:11:03.956783 IP 10.99.1.201 > 80.239.147.29: ICMP echo request, id 2099, seq 2, length 64
                                  22:11:03.978472 IP 80.239.147.29 > 10.99.1.201: ICMP echo reply, id 2099, seq 2, length 64
                                  22:11:04.957965 IP 10.99.1.201 > 80.239.147.29: ICMP echo request, id 2099, seq 3, length 64
                                  22:11:04.979579 IP 80.239.147.29 > 10.99.1.201: ICMP echo reply, id 2099, seq 3, length 64
                                  22:11:05.959203 IP 10.99.1.201 > 80.239.147.29: ICMP echo request, id 2099, seq 4, length 64
                                  22:11:05.980780 IP 80.239.147.29 > 10.99.1.201: ICMP echo reply, id 2099, seq 4, length 64
                                  
                                  Here is the same test from the actual VM, you see the inbound connection attempt and a reply. Then I run a ping test, which is successful: 
                                  [code]root@vm1:~# tcpdump -i eth1 -nn host 80.239.147.29
                                  tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
                                  listening on eth1, link-type EN10MB (Ethernet), capture size 262144 bytes
                                  21:41:02.315127 IP 80.239.147.29.59236 > 10.99.1.201.59362: Flags [s], seq 2689225177, win 29200, options [mss 1349,sackOK,TS val 56078236 ecr 0,nop,wscale 7], length 0
                                  21:41:02.315179 IP 10.99.1.201.59362 > 80.239.147.29.59236: Flags [S.], seq 2659676728, ack 2689225178, win 28960, options [mss 1460,sackOK,TS val 55124643 ecr 56078236,nop,wscale 7], length 0
                                  21:41:03.312092 IP 80.239.147.29.59236 > 10.99.1.201.59362: Flags [s], seq 2689225177, win 29200, options [mss 1349,sackOK,TS val 56078486 ecr 0,nop,wscale 7], length 0
                                  21:41:03.312126 IP 10.99.1.201.59362 > 80.239.147.29.59236: Flags [S.], seq 2659676728, ack 2689225178, win 28960, options [mss 1460,sackOK,TS val 55124893 ecr 56078236,nop,wscale 7], length 0
                                  21:41:04.712097 IP 10.99.1.201.59362 > 80.239.147.29.59236: Flags [S.], seq 2659676728, ack 2689225178, win 28960, options [mss 1460,sackOK,TS val 55125243 ecr 56078236,nop,wscale 7], length 0
                                  21:41:06.604111 IP 10.99.1.201 > 80.239.147.29: ICMP echo request, id 3025, seq 1, length 64
                                  21:41:06.626188 IP 80.239.147.29 > 10.99.1.201: ICMP echo reply, id 3025, seq 1, length 64
                                  21:41:07.605414 IP 10.99.1.201 > 80.239.147.29: ICMP echo request, id 3025, seq 2, length 64
                                  21:41:07.627132 IP 80.239.147.29 > 10.99.1.201: ICMP echo reply, id 3025, seq 2, length 64
                                  
                                  The commands, for ref:
                                  [code]root@vm1:~# iperf3 -s -p 59362
                                  -----------------------------------------------------------
                                  Server listening on 59362
                                  -----------------------------------------------------------
                                  ^Ciperf3: interrupt - the server has terminated
                                  root@vm1:~# ping 80.239.147.29
                                  PING 80.239.147.29 (80.239.147.29) 56(84) bytes of data.
                                  64 bytes from 80.239.147.29: icmp_seq=1 ttl=55 time=22.0 ms
                                  64 bytes from 80.239.147.29: icmp_seq=2 ttl=55 time=21.7 ms
                                  ^C
                                  --- 80.239.147.29 ping statistics ---
                                  2 packets transmitted, 2 received, 0% packet loss, time 1001ms
                                  rtt min/avg/max/mdev = 21.741/21.912/22.084/0.226 ms[/code]
                                  
                                  [quote]
                                  You could have an asymmetrical issue where pfsense sends it on to your host on your vlan99..  But vlan99 host is sending it to a different IP as its gateway?
                                  [/quote]
                                  Yes, that is what is happening, but only when traffic is established from outside. I'm not surte what I'm doing wrong, or pfsense doing wrong or how to fix it
                                  
                                  [quote]Also it seems you have multiple threads about the same topic??  You should have the mods merge them!!!
                                  [/quote]
                                  
                                  I appreciate your help - but really, that information is already in the thread :) I have a TCPdump of my VLAN99 interface above, you can see that packets are being sent between the same IP's when it's "working" (ping test outbound) and not working (Iperf session inbound) 
                                  
                                  I do not have multiple threads, but I cant help it if there's other that see about the same issue :) 
                                  
                                  [quote]
                                  Have you posted your port forwards for your vpn interface?
                                  [/quote]
                                  Attaching it below.
                                  
                                  [quote]
                                  I think it would also be helpful to see the rules from the "OPT" interface assigned to the tunnel as well.
                                  [/quote]
                                  They are in my post above, when you asked if the rules where on the wrong interface. "OVPN" interface is the OPT interface assigned to the tunnel (on my pfsense box that is, I do not have access to the server which is hosted).
                                  
                                  Did you get a chance to see if you can hit the scenario yourself? This is getting interesting, some real troubleshooting :) 
                                  
                                  ![Skärmklipp 2017-05-26 21.37.12.png](/public/_imported_attachments_/1/Skärmklipp 2017-05-26 21.37.12.png)
                                  ![Skärmklipp 2017-05-26 21.37.12.png_thumb](/public/_imported_attachments_/1/Skärmklipp 2017-05-26 21.37.12.png_thumb)[/s][/s][/code][/s][/s][/s]
                                  
                                  1 Reply Last reply Reply Quote 0
                                  • C
                                    CuteBoi
                                    last edited by

                                    My problem seems to be the same as  Kroem's.  I had made mine around the same time he did.

                                    He's on 2.3.3, I'm on 2.3.4

                                    1 Reply Last reply Reply Quote 0
                                    • johnpozJ
                                      johnpoz LAYER 8 Global Moderator
                                      last edited by

                                      Ok - so I signed up for trial… Bing bang zoom working!!

                                      I created client connection per their instructions - which are nice btw..

                                      I then setup policy routing to send my client 192.168.9.100 out the vpn
                                      Checked my IP!  Yup going out the vpn
                                      Setup the port forward to send that port to the same port on my box 192.168.9.100
                                      Fired up HFS to listen on 58703
                                      Checked if port was open on canyouseeme.org - boom Works!

                                      You can see the packet capture on the ovpn interface created, and then sniff on my lan and you can see my 192.168.9.100 box respond..

                                      So I suggest you post up the same screenshots I posted..

                                      workshere.png
                                      workshere.png_thumb

                                      An intelligent man is sometimes forced to be drunk to spend time with his fools
                                      If you get confused: Listen to the Music Play
                                      Please don't Chat/PM me for help, unless mod related
                                      SG-4860 24.11 | Lab VMs 2.8, 24.11

                                      1 Reply Last reply Reply Quote 0
                                      • C
                                        CuteBoi
                                        last edited by

                                        What was your "default gateway"?

                                        Because ours isn't the VPN gateway, we only want select systems to use that tunnel.

                                        I can make port forwarding work flawlessly if I change my default gateway.

                                        1 Reply Last reply Reply Quote 0
                                        • K
                                          kroem
                                          last edited by

                                          @johnpoz:

                                          Ok - so I signed up for trial… Bing bang zoom working!!

                                          I created client connection per their instructions - which are nice btw..

                                          I then setup policy routing to send my client 192.168.9.100 out the vpn
                                          Checked my IP!  Yup going out the vpn
                                          Setup the port forward to send that port to the same port on my box 192.168.9.100
                                          Fired up HFS to listen on 58703
                                          Checked if port was open on canyouseeme.org - boom Works!

                                          You can see the packet capture on the ovpn interface created, and then sniff on my lan and you can see my 192.168.9.100 box respond..

                                          So I suggest you post up the same screenshots I posted..

                                          As CuteBoi said, if it's just a default gateway there is no problem…

                                          1 Reply Last reply Reply Quote 0
                                          • C
                                            CuteBoi
                                            last edited by

                                            I just tried something, for fun.

                                            If this works for you, I would ask WHY.  I would not want to allow an any to any rule on a VPN interface.

                                            Add an Any to any rule in your VPN interface at the TOP, over your port forward entries.

                                            Image attached.

                                            ![Screenshot from 2017-05-27 02-40-35.png](/public/imported_attachments/1/Screenshot from 2017-05-27 02-40-35.png)
                                            ![Screenshot from 2017-05-27 02-40-35.png_thumb](/public/imported_attachments/1/Screenshot from 2017-05-27 02-40-35.png_thumb)

                                            1 Reply Last reply Reply Quote 0
                                            • First post
                                              Last post
                                            Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.