Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    offloading OpenVPN using external gateway

    Scheduled Pinned Locked Moved Routing and Multi WAN
    70 Posts 2 Posters 13.7k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • bingo600B
      bingo600 @chrispazz
      last edited by

      @chrispazz said in offloading OpenVPN using external gateway:

      @bingo600 if I try a trace to www.google.it IP address as host I get this:

      18:25:31.846958 IP 192.168.5.1.4704 > 172.217.168.35.443: UDP, length 1350
      18:25:31.883433 IP 172.217.168.35.443 > 192.168.5.1.4704: UDP, length 1350

      so it is using 192.168.5.1 as gateway....

      Nope
      It's using 5.1 as source IP (Consistent with your WAN NAT)

      If you enable logging on the Policy rule , do you see any log lines ?

      Now you are not doing the traceroute on the pfSense box ... Are you ?

      What PC ip address are you doing the traceroute from , and are you 100% sure you have that ip in the OPENVPN Alias , and a policy route rule on the pfSense interface that that PC is connected to ?

      If you find my answer useful - Please give the post a šŸ‘ - "thumbs up"

      pfSense+ 23.05.1 (ZFS)

      QOTOM-Q355G4 Quad Lan.
      CPUĀ  : Core i5 5250U, Ram : 8GB Kingston DDR3LV 1600
      LANĀ  : 4 x Intel 211, DiskĀ  : 240G SAMSUNG MZ7L3240HCHQ SSD

      C 1 Reply Last reply Reply Quote 0
      • C
        chrispazz @bingo600
        last edited by chrispazz

        @bingo600 said in offloading OpenVPN using external gateway:

        If you enable logging on the Policy rule , do you see any log lines ?

        Yes, it is the only rules that has logging

        Now you are not doing the traceroute on the pfSense box ... Are you ?

        No, I am doing it from iMac client (192.168.20.110)

        What PC ip address are you doing the traceroute from , and are you 100% sure you have that ip in the OPENVPN Alias , and a policy route rule on the pfSense interface that that PC is connected to ?

        I am using a firewall rule like this on the same interface of the client (192.168.20):

        2.jpg

        and NVPNGTW is the gateway I added to pfsense in the gateway section.

        bingo600B 1 Reply Last reply Reply Quote 0
        • bingo600B
          bingo600 @chrispazz
          last edited by

          @chrispazz said in offloading OpenVPN using external gateway:

          @bingo600 said in offloading OpenVPN using external gateway:

          If you enable logging on the Policy rule , do you see any log lines ?

          Yes, it is the only rules that has logging

          Now you are not doing the traceroute on the pfSense box ... Are you ?

          No, I am doing it from iMac client (192.168.20.110)

          What PC ip address are you doing the traceroute from , and are you 100% sure you have that ip in the OPENVPN Alias , and a policy route rule on the pfSense interface that that PC is connected to ?

          I am using a firewall rule like this on the same interface of the client (192.168.20):

          2.jpg

          and NVPNGTW is the gateway I added to pfsense in the gateway section.

          That's a (for me strange rule) why does it have WAN all the way to the right ?
          Are you using floating rules ?

          If you find my answer useful - Please give the post a šŸ‘ - "thumbs up"

          pfSense+ 23.05.1 (ZFS)

          QOTOM-Q355G4 Quad Lan.
          CPUĀ  : Core i5 5250U, Ram : 8GB Kingston DDR3LV 1600
          LANĀ  : 4 x Intel 211, DiskĀ  : 240G SAMSUNG MZ7L3240HCHQ SSD

          C 1 Reply Last reply Reply Quote 0
          • C
            chrispazz @bingo600
            last edited by

            @bingo600 no no, sorry, it is only the description. Ignore it.

            bingo600B 1 Reply Last reply Reply Quote 0
            • bingo600B
              bingo600 @chrispazz
              last edited by

              @chrispazz

              Is that rule all the way at the top ?

              If you find my answer useful - Please give the post a šŸ‘ - "thumbs up"

              pfSense+ 23.05.1 (ZFS)

              QOTOM-Q355G4 Quad Lan.
              CPUĀ  : Core i5 5250U, Ram : 8GB Kingston DDR3LV 1600
              LANĀ  : 4 x Intel 211, DiskĀ  : 240G SAMSUNG MZ7L3240HCHQ SSD

              C 1 Reply Last reply Reply Quote 0
              • C
                chrispazz @bingo600
                last edited by

                @bingo600 yes, sure

                bingo600B 2 Replies Last reply Reply Quote 0
                • bingo600B
                  bingo600 @chrispazz
                  last edited by

                  @chrispazz
                  I have to pull in the tuff guyzz now.

                  @stephenw10
                  Any ideas here ??

                  If you find my answer useful - Please give the post a šŸ‘ - "thumbs up"

                  pfSense+ 23.05.1 (ZFS)

                  QOTOM-Q355G4 Quad Lan.
                  CPUĀ  : Core i5 5250U, Ram : 8GB Kingston DDR3LV 1600
                  LANĀ  : 4 x Intel 211, DiskĀ  : 240G SAMSUNG MZ7L3240HCHQ SSD

                  1 Reply Last reply Reply Quote 1
                  • bingo600B
                    bingo600 @chrispazz
                    last edited by bingo600

                    @chrispazz

                    But just to summarize :

                    You say if you put a PC on the 192.168.5.x lan , and set 192.168.5.99 (RasPI) as def-gw , the PC can use the RasPI & OpenVPN tunnel ok ?

                    You have put all desired OPENVPN PC's in a IP HOST Alias
                    You made direct host entries in the rule, and made Policy based rules with the RasPI as Gateway , on all the interfaces where a "OPENVPN" PC could be connected ?

                    But the pfSense won't match the source ip's, and therefore won't policy route t the RasPI. This is also confirmed by , enabling logging in the Policy rule. And no loglines appear.

                    Q:
                    We did a trace (the first one above) , where we succesfully matched a ping to google.it , and you got both loglines , and we saw the policy routing in effect in the pfSense packet trace.

                    What has changed in the pfSense config since that ?

                    Was that test also made with the iMAC ?

                    Q:
                    You do use either static IP's or DHCP MAC Lock (Reservation) , in order to ensure that the PC's used for test & OPN always get the same ip ?.

                    If you find my answer useful - Please give the post a šŸ‘ - "thumbs up"

                    pfSense+ 23.05.1 (ZFS)

                    QOTOM-Q355G4 Quad Lan.
                    CPUĀ  : Core i5 5250U, Ram : 8GB Kingston DDR3LV 1600
                    LANĀ  : 4 x Intel 211, DiskĀ  : 240G SAMSUNG MZ7L3240HCHQ SSD

                    C 1 Reply Last reply Reply Quote 0
                    • C
                      chrispazz @bingo600
                      last edited by chrispazz

                      @bingo600 said in offloading OpenVPN using external gateway:

                      @chrispazz

                      But just to summarize :

                      You say if you put a PC on the 192.168.5.x lan , and set 192.168.5.99 (RasPI) as def-gw , the PC can use the RasPI & OpenVPN tunnel ok ?

                      Yes, correct.

                      You have put all desired OPENVPN PC's in a IP HOST Alias
                      You made direct host entries in the rule, and made Policy based rules with the RasPI as Gateway , on all the interfaces where a "OPENVPN" PC could be connected ?

                      Yes.

                      But the pfSense won't match the source ip's, and therefore won't policy route t the RasPI. This is also confirmed by , enabling logging in the Policy rule. And no loglines appear.

                      No. The rule is correctly engaged and it is logged in firewall log but it seems to ignore the Raspi used as gateway.

                      Q:
                      We did a trace (the first one above) , where we succesfully matched a ping to google.it , and you got both loglines , and we saw the policy routing in effect in the pfSense packet trace.

                      Currently, using 192.168.5.9 (Raspi gateway) as host, trace display the following when doing a ping to www.google.it:

                      19:32:03.251588 IP 192.168.5.1 > 192.168.5.9: ICMP echo request, id 43267, seq 26591, length 9
                      19:32:03.252028 IP 192.168.5.9 > 192.168.5.1: ICMP echo reply, id 43267, seq 26591, length 9

                      Doing packet trace using google IP as host gives the following:

                      19:33:38.627397 IP 192.168.5.1 > 172.217.168.3: ICMP echo request, id 10411, seq 0, length 64
                      19:33:38.642607 IP 172.217.168.3 > 192.168.5.1: ICMP echo reply, id 10411, seq 0, length 64

                      Was that test also made with the iMAC ?

                      The packet tracing is done pinging from Imac to www.google.it

                      Q:
                      You do use either static IP's or DHCP MAC Lock (Reservation) , in order to ensure that the PC's used for test & OPN always get the same ip ?.

                      I have IP reservation for iMac and static IP for Pfsense.

                      Just to give you more confirmation about the rule catched, consider that without that new rule, the connection has to go via the pfsense VPN client.
                      If I deactivate the rule, the connection go with pfsense VPN client. if I enable the rule, the connection exit with ISP gateway (no raspi VPN).

                      bingo600B 1 Reply Last reply Reply Quote 0
                      • bingo600B
                        bingo600 @chrispazz
                        last edited by

                        @chrispazz said in offloading OpenVPN using external gateway:
                        packet trace.

                        Currently, using 192.168.5.9 (Raspi gateway) as host, trace display the following when doing a ping to www.google.it:

                        What does that mean ?
                        Currently, using 192.168.5.9 (Raspi gateway) as host

                        Wasnt the RasPI 192.168.5.99 ?

                        What does as host mean (you mean you ping the raspi) ?

                        The below 2 traces both show a req & a reply

                        19:32:03.251588 IP 192.168.5.1 > 192.168.5.9: ICMP echo request, id 43267, seq 26591, length 9
                        19:32:03.252028 IP 192.168.5.9 > 192.168.5.1: ICMP echo reply, id 43267, seq 26591, length 9

                        Doing packet trace using google IP as host gives the following:

                        19:33:38.627397 IP 192.168.5.1 > 172.217.168.3: ICMP echo request, id 10411, seq 0, length 64
                        19:33:38.642607 IP 172.217.168.3 > 192.168.5.1: ICMP echo reply, id 10411, seq 0, length 64

                        In the Packet capture try to set detail to high or full

                        2cedcbc3-da8f-40b9-92a1-e384cc29ec8f-image.png

                        We need to see the mac addresses in those packets.

                        And compare to the RasPI mac address

                        If you find my answer useful - Please give the post a šŸ‘ - "thumbs up"

                        pfSense+ 23.05.1 (ZFS)

                        QOTOM-Q355G4 Quad Lan.
                        CPUĀ  : Core i5 5250U, Ram : 8GB Kingston DDR3LV 1600
                        LANĀ  : 4 x Intel 211, DiskĀ  : 240G SAMSUNG MZ7L3240HCHQ SSD

                        C 1 Reply Last reply Reply Quote 0
                        • C
                          chrispazz @bingo600
                          last edited by

                          @bingo600 said in offloading OpenVPN using external gateway:

                          What does as host mean (you mean you ping the raspi) ?

                          HOST is the captured Host address

                          We need to see the mac addresses in those packets.

                          b8:27:eb:8e:00:43 is Raspi MAC Address
                          02:11:32:25:4e:f9 is pfsense WAN interface

                          Full detailed trace is the following:

                          192.168.5.1 > 192.168.5.99: ICMP echo request, id 43267, seq 28884, length 9
                          19:51:42.213831 b8:27:eb:8e:00:43 > 02:11:32:25:4e:f9, ethertype IPv4 (0x0800), length 60: (tos 0x0, ttl 64, id 59852, offset 0, flags [none], proto ICMP (1), length 29)
                          192.168.5.99 > 192.168.5.1: ICMP echo reply, id 43267, seq 28884, length 9
                          19:51:42.723672 02:11:32:25:4e:f9 > b8:27:eb:8e:00:43, ethertype IPv4 (0x0800), length 43: (tos 0x0, ttl 64, id 65165, offset 0, flags [none], proto ICMP (1), length 29)

                          bingo600B 2 Replies Last reply Reply Quote 0
                          • bingo600B
                            bingo600 @chrispazz
                            last edited by bingo600

                            @chrispazz

                            Now do the same trace with the google ip as the destination

                            Your iMAC isn't multihomed ?
                            I mean both cable & WiFi

                            If you find my answer useful - Please give the post a šŸ‘ - "thumbs up"

                            pfSense+ 23.05.1 (ZFS)

                            QOTOM-Q355G4 Quad Lan.
                            CPUĀ  : Core i5 5250U, Ram : 8GB Kingston DDR3LV 1600
                            LANĀ  : 4 x Intel 211, DiskĀ  : 240G SAMSUNG MZ7L3240HCHQ SSD

                            C 1 Reply Last reply Reply Quote 0
                            • C
                              chrispazz @bingo600
                              last edited by chrispazz

                              @bingo600 here it is:

                              192.168.5.1 > 216.58.215.227: ICMP echo request, id 61288, seq 0, length 64
                              20:09:20.262578 dc:39:6f:f2:4d:e4 > 02:11:32:25:4e:f9, ethertype IPv4 (0x0800), length 98: (tos 0x60, ttl 113, id 0, offset 0, flags [none], proto ICMP (1), length 84)
                              216.58.215.227 > 192.168.5.1: ICMP echo reply, id 61288, seq 0, length 64
                              20:09:21.238644 02:11:32:25:4e:f9 > dc:39:6f:f2:4d:e4, ethertype IPv4 (0x0800), length 98: (tos 0x0, ttl 64, id 58651, offset 0, flags [none], proto ICMP (1), length 84)

                              Trying to understand who is dc:39:6f:f2:4d:e4....

                              Currently iMac is connected only with wifi so no multiple ip

                              bingo600B 1 Reply Last reply Reply Quote 0
                              • bingo600B
                                bingo600 @chrispazz
                                last edited by

                                @chrispazz said in offloading OpenVPN using external gateway:

                                Trying to understand who is dc:39:6f:f2:4d:e4....

                                My money is on the ISP router 5.254

                                pfSense Diagnostic -> arp table will reveal

                                If you find my answer useful - Please give the post a šŸ‘ - "thumbs up"

                                pfSense+ 23.05.1 (ZFS)

                                QOTOM-Q355G4 Quad Lan.
                                CPUĀ  : Core i5 5250U, Ram : 8GB Kingston DDR3LV 1600
                                LANĀ  : 4 x Intel 211, DiskĀ  : 240G SAMSUNG MZ7L3240HCHQ SSD

                                C 1 Reply Last reply Reply Quote 0
                                • C
                                  chrispazz @bingo600
                                  last edited by

                                  @bingo600 you are right!

                                  bingo600B 1 Reply Last reply Reply Quote 0
                                  • bingo600B
                                    bingo600 @chrispazz
                                    last edited by

                                    @chrispazz

                                    In the capture , strange it matched ....
                                    didn't you set 192.168.5.99 as ip match ?

                                    If you find my answer useful - Please give the post a šŸ‘ - "thumbs up"

                                    pfSense+ 23.05.1 (ZFS)

                                    QOTOM-Q355G4 Quad Lan.
                                    CPUĀ  : Core i5 5250U, Ram : 8GB Kingston DDR3LV 1600
                                    LANĀ  : 4 x Intel 211, DiskĀ  : 240G SAMSUNG MZ7L3240HCHQ SSD

                                    C 1 Reply Last reply Reply Quote 0
                                    • C
                                      chrispazz @bingo600
                                      last edited by

                                      @bingo600 latest try was with google IP as you asked....am I wrong?

                                      1 Reply Last reply Reply Quote 0
                                      • bingo600B
                                        bingo600 @chrispazz
                                        last edited by

                                        @chrispazz said in offloading OpenVPN using external gateway:

                                        @bingo600 said in offloading OpenVPN using external gateway:

                                        What does as host mean (you mean you ping the raspi) ?

                                        HOST is the captured Host address

                                        We need to see the mac addresses in those packets.

                                        b8:27:eb:8e:00:43 is Raspi MAC Address
                                        02:11:32:25:4e:f9 is pfsense WAN interface

                                        Full detailed trace is the following:

                                        192.168.5.1 > 192.168.5.99: ICMP echo request, id 43267, seq 28884, length 9
                                        19:51:42.213831 b8:27:eb:8e:00:43 > 02:11:32:25:4e:f9, ethertype IPv4 (0x0800), length 60: (tos 0x0, ttl 64, id 59852, offset 0, flags [none], proto ICMP (1), length 29)
                                        192.168.5.99 > 192.168.5.1: ICMP echo reply, id 43267, seq 28884, length 9
                                        19:51:42.723672 02:11:32:25:4e:f9 > b8:27:eb:8e:00:43, ethertype IPv4 (0x0800), length 43: (tos 0x0, ttl 64, id 65165, offset 0, flags [none], proto ICMP (1), length 29)

                                        What command did you use on the iMAC to make this trace , where it goes to the Raspi (but raspi as destination)

                                        If you find my answer useful - Please give the post a šŸ‘ - "thumbs up"

                                        pfSense+ 23.05.1 (ZFS)

                                        QOTOM-Q355G4 Quad Lan.
                                        CPUĀ  : Core i5 5250U, Ram : 8GB Kingston DDR3LV 1600
                                        LANĀ  : 4 x Intel 211, DiskĀ  : 240G SAMSUNG MZ7L3240HCHQ SSD

                                        C 1 Reply Last reply Reply Quote 0
                                        • C
                                          chrispazz @bingo600
                                          last edited by

                                          @bingo600 only "ping www.google.it".

                                          But in first trace I used the raspi as Host capture.

                                          C bingo600B 2 Replies Last reply Reply Quote 0
                                          • C
                                            chrispazz @chrispazz
                                            last edited by chrispazz

                                            Ok this is interesting:

                                            I added a floating rule with no filters but "out" direction and in advanced settings with the Raspi Gateway.

                                            With this, the connection go correctly thru the Raspi VPN.

                                            So, the external gateway thur pfsense is working correctly.
                                            I have to understand why when using a firewall rule in a specific interface is not using Raspi gateway

                                            1 Reply Last reply Reply Quote 0
                                            • First post
                                              Last post
                                            Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.