• Categories
  • Recent
  • Tags
  • Popular
  • Users
  • Search
  • Register
  • Login
Netgate Discussion Forum
  • Categories
  • Recent
  • Tags
  • Popular
  • Users
  • Search
  • Register
  • Login

[Solved] Client subnet not accessible (and no internet)

Scheduled Pinned Locked Moved WireGuard
33 Posts 3 Posters 5.5k Views
Loading More Posts
  • Oldest to Newest
  • Newest to Oldest
  • Most Votes
Reply
  • Reply as topic
Log in to reply
This topic has been deleted. Only users with topic management privileges can see it.
  • A
    arrmo
    last edited by arrmo Feb 7, 2021, 3:17 AM Jan 30, 2021, 8:36 PM

    First off - my apologies for saying client and server, vs peers ... I know that's the official naming, but client / server makes it easier to explain (and in my head ... LOL!).

    I see some routing items noted, I just can't tell if they are the same, so let me try to explain my setup.

    "Client" - OpenWrt based router. WG configured, to send all traffic up to the server (pfSense box, and on the client the allowed IPs is 0.0.0.0/0).

    "Server" - pfSense ... my firewall, configured for allowed IP's back to the client - with the client IP address (192.168.253.3/32), also the DHCP subnet on the LAN side of the OpenWrt box (192.168.0.1/24).

    So - WG connects, I see the link on both ends, and can get to and fro using WG addresses 192.168.253.x. ssh'ing in to the OpenWrt router, I can get to the internet (traceroute, get my internet IP address, etc.). So all OK. But ... if I try to get to the internet from any of the DHCP assigned clients behind the OpenWrt (client) router ... no internet. I can get to boxes inside the pfSense firewall (i.e. on the server subnet), but not out to the internet. I also tried pinging from pfSense to anything behind the OpenWrt router (192.168.0.x) ... nope, no joy. So it's like traffic is not being routed back, even though this is set as an allowed IP on the server end of WG.

    Making sense? 😜 Sorry if not, just yell. But any thoughts how to route that traffic back over WG (and how to allow the "clients on the client end" to get to the internet)?

    Thanks!!

    1 Reply Last reply Reply Quote 0
    • A
      AB5G
      last edited by Jan 31, 2021, 12:47 AM

      Most likely a NAT issue - check that

      • the OpenWrt box is natting all OpenWrt LAN IP's to the tunnel IP
      • Wireguard interfacce rule allows WG net any any protocol any
      A 1 Reply Last reply Jan 31, 2021, 2:05 AM Reply Quote 0
      • A
        arrmo @AB5G
        last edited by Jan 31, 2021, 2:05 AM

        @ab5g said in Client subnet not accessible (and no internet):

        Most likely a NAT issue - check that

        Thanks for the suggestions!

        • the OpenWrt box is natting all OpenWrt LAN IP's to the tunnel IP

        Ummm ... can it? When WG is set for 0.0.0.0/0, all traffic is sent out directly, not going through LAN => WAN (so no NAT). No?

        • Wireguard interfacce rule allows WG net any any protocol any

        Yep!

        1 Reply Last reply Reply Quote 0
        • A
          AB5G
          last edited by AB5G Jan 31, 2021, 3:59 AM Jan 31, 2021, 3:58 AM

          @arrmo
          You need to set the OpenWrt box to NAT all traffic from OpenWrt LAN going to pfSense with the WG tunnel IP at the OpenWrt end (192.168.253.3). The LAN of Openwrt is possibly able to reach the pfsense , but the pfsense does not know how to route back( reach the Openwrt LAN.)

          Try running a tcpdump on the Wireguard interface of pfsense like
          tcpdump -i wg0 host <openwrt LAN host ip>
          tcpdump -i wg0 host 192.168.253.3

          If you setup the NAT correctly on Openwrt then you should see traffic coming from 192.168.253.3/32 , otherwise you will see traffic coming from openwrt LAN IP (in which case you'll need to setup NAT at openwrt).

          A 1 Reply Last reply Jan 31, 2021, 4:31 AM Reply Quote 0
          • A
            arrmo @AB5G
            last edited by Jan 31, 2021, 4:31 AM

            @ab5g said in Client subnet not accessible (and no internet):

            If you setup the NAT correctly on Openwrt then you should see traffic coming from 192.168.253.3/32 , otherwise you will see traffic coming from openwrt LAN IP (in which case you'll need to setup NAT at openwrt).

            Thanks makes sense, thanks! I was wondering, but when I asked OpenWrt folks, they said to assign WG to the LAN zone. But rather, sounds like I need to forward the LAN to WG, and masquerade ... right? So the only IP that comes across to pfSense is 192.168.253.3 ... correct? And of course then, I only need one allowed IP, given NAT - agreed?

            Thanks again!

            A 1 Reply Last reply Jan 31, 2021, 4:52 AM Reply Quote 0
            • A
              AB5G @arrmo
              last edited by Jan 31, 2021, 4:52 AM

              @arrmo That is correct

              A 1 Reply Last reply Feb 1, 2021, 2:07 PM Reply Quote 1
              • A
                arrmo @AB5G
                last edited by Feb 1, 2021, 2:07 PM

                @ab5g Still digging ... LOL. Starting to think this may be a pfSense thing after all (but I may be wrong!!!). Here is what I see though - observations that confuse me, make me wonder if it's pfSense routing (tables).

                1. pinging from pfSense directly, to a machine behind OpenWrt (i.e. DHCP client on the WG "client" (peer) side). A bit odd, but seems OK to me.
                $ ping 192.168.0.220
                PING 192.168.0.220 (192.168.0.220): 56 data bytes
                92 bytes from 192.168.253.1: Redirect Host(New addr: 192.168.0.220)
                Vr HL TOS  Len   ID Flg  off TTL Pro  cks      Src      Dst
                 4  5  00 0054 931f   0 0000  3e  01 6a5b 192.168.253.1  192.168.0.220
                
                1. pinging from a Linux box on the pfSense subnet, to the same machine behind OpenWrt (i.e. DHCP client on the WG "client" (peer) side). OK, this seems broken?
                $ ping 192.168.0.220
                PING 192.168.0.220 (192.168.0.220) 56(84) bytes of data.
                From 192.168.253.1 icmp_seq=1 Redirect Host(New nexthop: 0.0.0.0)
                From 192.168.253.1 icmp_seq=1 Redirect Host(New nexthop: 0.0.0.0)
                From 192.168.253.1 icmp_seq=1 Redirect Host(New nexthop: 0.0.0.0)
                From 192.168.253.1 icmp_seq=1 Redirect Host(New nexthop: 0.0.0.0)
                From 192.168.253.1 icmp_seq=1 Redirect Host(New nexthop: 0.0.0.0)
                From 192.168.253.1 icmp_seq=1 Redirect Host(New nexthop: 0.0.0.0)
                From 192.168.253.1 icmp_seq=1 Redirect Host(New nexthop: 0.0.0.0)
                From 192.168.253.1 icmp_seq=1 Redirect Host(New nexthop: 0.0.0.0)
                From 192.168.253.1 icmp_seq=1 Redirect Host(New nexthop: 0.0.0.0)
                From 192.168.253.1 icmp_seq=1 Redirect Host(New nexthop: 0.0.0.0)
                From 192.168.253.1 icmp_seq=1 Redirect Host(New nexthop: 0.0.0.0)
                From 192.168.253.1 icmp_seq=1 Redirect Host(New nexthop: 0.0.0.0)
                From 192.168.253.1 icmp_seq=1 Redirect Host(New nexthop: 0.0.0.0)
                From 192.168.253.1 icmp_seq=1 Redirect Host(New nexthop: 0.0.0.0)
                ^C
                --- 192.168.0.220 ping statistics ---
                1 packets transmitted, 0 received, +14 errors, 100% packet loss, time 0ms
                
                1. Trying to use a web browser, on the pfSense subnet, to get to a box on the OpenWrt (WG "client") subnet ... times out. But, this subnet is in the AllowedIPs list, so pfSense should route me over there, no?

                Thoughts?

                Thanks!

                1 Reply Last reply Reply Quote 0
                • A
                  AB5G
                  last edited by Feb 2, 2021, 1:08 AM

                  @arrmo
                  Both your pings are not working. I suspect it is because the gateway is not set correctly.
                  System > Routing > Gateways > Set that to WAN_DHCP (or whatever your gateway is) ; don't leave that to automatic.

                  If that doesn't work, you'll need to provide more info

                  • OpenWrt Side - LAN 192.168.0.1/24, Tunnel IP. 192.168.253.3 , Test PC - 192.168.0.220
                  • pfSense Side - LAN ? , Tunnel IP 192.168.253.1 , Test PC - ?
                    Routing tables, and wireguard config
                  A 1 Reply Last reply Feb 2, 2021, 1:31 AM Reply Quote 1
                  • A
                    arrmo @AB5G
                    last edited by Feb 2, 2021, 1:31 AM

                    @ab5g said in Client subnet not accessible (and no internet):

                    System > Routing > Gateways > Set that to WAN_DHCP (or whatever your gateway is) ; don't leave that to automatic.

                    It is set to automatic, but there is only one option (WAN_DHCP) ... and that one shows the world symbol. So OK, agreed?

                    @ab5g said in Client subnet not accessible (and no internet):

                    If that doesn't work, you'll need to provide more info

                    OpenWrt Side - LAN 192.168.0.1/24, Tunnel IP. 192.168.253.3 , Test PC - 192.168.0.220
                    pfSense Side - LAN ? , Tunnel IP 192.168.253.1 , Test PC - ?
                    Routing tables, and wireguard config

                    Absolutely! Let me try to clarify - yell if this is not clear.

                    1. pfSense - subnet is 192.168.2.x, WAN (to cable modem) is 192.168.1.x.
                    2. OpenWrt - subnet is 192.168.0.x. I can get to the internet from ssh (I think it comes from the WG address then?), but not from machines on the .0.x subnet. That's the core issue, .0.x machines not being able to get out. But they can get to machines on the .2.x (pfSense) subnet. Make sense?

                    AllowedIP's, for WG

                    1. pfSense: 192.168.253.3/32, 192.168.0.0/24
                    2. OpenWrt: 192.168.253.0/24, 216.115.184.69/32, 192.168.1.0/24, 192.168.2.0/24, 192.168.0.0/24 (last one seems to be needed to get ping from OpenWrt to pfSense to work?).

                    I also can't get from the pfSense subnet (..2.x) to OpenWrt subnet (.0.x).

                    Thoughts?

                    Thanks!

                    A 1 Reply Last reply Feb 2, 2021, 1:47 AM Reply Quote 0
                    • A
                      AB5G @arrmo
                      last edited by Feb 2, 2021, 1:47 AM

                      @arrmo
                      Let's tackle it one by one. First off "can't get from the pfSense subnet (..2.x) to OpenWrt subnet (.0.x)."

                      Thats because there's only 1 gateway in your pfsense which is the WAN_DHCP and so all packets coming from pf LAN are being sent out through that GW. The GW doesn't know how to route them so it doesn't work. To solve this, you need to create a gateway (corresponding to wireguard tunnel) and route packets destined to 192.168.0.0/24 to that gateway. Makes sense ?

                      To create the GW, you need to create the interface first. After the WG tunnel comes up, you need to goto Interface/Assignments and assign wg0 (the tunnel) a new Interface say "WG". Enable the interface and restart the tunnel. This should create the GW for you (System > Routing > Gateways).
                      Now select WAN_DHCP as your default gateway. Don't leave that to auto.

                      Next, you'll notice that the under firewall you'll see the interface entry for 'WG'. To keep things simple. Put a rule Allow any proto from src WG net to any dest. This will allow any inbound traffic coming from OpenWrt to reach your pf LAN.

                      Finally to allow pfLAN to reach OW LAN - create a LAN rule - proto any, src LAN net, dest 192.168.0.0/24 , advanced, set gateway to WG_GW.

                      A 2 Replies Last reply Feb 3, 2021, 2:40 AM Reply Quote 1
                      • A
                        arrmo @AB5G
                        last edited by Feb 3, 2021, 2:40 AM

                        @ab5g I THINK this is working 😆. Just a bit worried yet, I have fooled myself before - but cautiously optimistic. Let me roll with it for a day or two, see if it really is solid. It didn't seem to work initially, but did after I made some other WG changes ... so perhaps also needed a WG (interface) restart of sorts? I admit, not quite sure how to force that to happen (reloading and taking the interface down and back up I assume)?

                        Thanks for the pointers!

                        J 1 Reply Last reply Feb 3, 2021, 1:21 PM Reply Quote 1
                        • J
                          jimp Rebel Alliance Developer Netgate @arrmo
                          last edited by Feb 3, 2021, 1:21 PM

                          @arrmo said in Client subnet not accessible (and no internet):

                          I admit, not quite sure how to force that to happen

                          To refresh the WireGuard interface config, edit the WireGuard tunnel on VPN > WireGuard and save it there. That will redo the interface config and so on.

                          Remember: Upvote with the 👍 button for any user/post you find to be helpful, informative, or deserving of recognition!

                          Need help fast? Netgate Global Support!

                          Do not Chat/PM for help!

                          A 1 Reply Last reply Feb 3, 2021, 7:02 PM Reply Quote 2
                          • A
                            arrmo @jimp
                            last edited by Feb 3, 2021, 7:02 PM

                            @jimp said in Client subnet not accessible (and no internet):

                            To refresh the WireGuard interface config, edit the WireGuard tunnel on VPN > WireGuard and save it there. That will redo the interface config and so on.

                            Perfect, thanks! I assume this is similar to / the same as ifdown then ifup? Was thinking there may be a button on the main screen, but definitely not a biggie 😄

                            Thanks again.

                            1 Reply Last reply Reply Quote 0
                            • A
                              arrmo @AB5G
                              last edited by Feb 3, 2021, 7:07 PM

                              @ab5g said in Client subnet not accessible (and no internet):

                              Let's tackle it one by one. First off "can't get from the pfSense subnet (..2.x) to OpenWrt subnet (.0.x)."

                              OK, this one seems solid - so far at least 😆. Thanks again, really appreciate it!

                              So the one that is open yet - I can't seem to get from the OpenWrt subnet (.0.x) out to the internet. No issue getting to the pfSense subnet (.2.x), but from there ... I can't get "out the door". I can get from the OpenWrt "console" itself (to the internet), but pretty sure that's showing a source address of .253.3 (i.e. WG address). Where it falls down is coming from .0.x, trying to get out the Gateway (on pfSense). Perhaps a routing setting that is missing? Or because .0.x is being routed back to OpenWrt, vs. out the door?

                              Thanks!

                              A 1 Reply Last reply Feb 4, 2021, 1:31 AM Reply Quote 0
                              • A
                                AB5G @arrmo
                                last edited by Feb 4, 2021, 1:31 AM

                                @arrmo

                                For the second one, you'd need a NAT rule. The packets from OW LAN are Natted by OpenWrt to 192.168.253.3/32 and sent to pf WG interface. The interface FW rule is set to allow WG NET (192.168.253.0/24) to any > which means the traffic can now enter and therefore you can reach the pf LAN. To exit using the internet at pf site, you'd need to tell pf to allow the 192.168.253.0/24 (WG NET) to be natted to the WAN interface IP to exit.
                                Goto firewall > NAT > Manual Outbound > Create new > Int WAN > Src 192.168.253.0/24 > src port any dest any destport any, NAT address WAN address.
                                Save and restart the wireguard tunnel.

                                A 1 Reply Last reply Feb 4, 2021, 2:41 AM Reply Quote 1
                                • A
                                  arrmo @AB5G
                                  last edited by Feb 4, 2021, 2:41 AM

                                  @ab5g said in Client subnet not accessible (and no internet):

                                  Goto firewall > NAT > Manual Outbound > Create new > Int WAN > Src 192.168.253.0/24 > src port any dest any destport any, NAT address WAN address.
                                  Save and restart the wireguard tunnel.

                                  Thanks! Tried that, but I don't think it's working (may be me though!). Here is the rule I created,
                                  155f2313-fd44-4fc2-897f-56d23dc8da29-image.png

                                  And the auto one that also already exists (second one)?
                                  63df5d18-173f-4c48-a29a-0f8ce8db1ef5-image.png

                                  But, watching tcpdump across the WG interface, I get this for a ping (and no response ... traffic coming across from OpenWrt subnet),

                                  20:34:04.690815 IP 192.168.0.243 > 216.115.184.69: ICMP echo request, id 1, seq 441, length 40
                                  20:34:09.327358 IP 192.168.0.243 > 216.115.184.69: ICMP echo request, id 1, seq 442, length 40
                                  

                                  I did try changing the NAT rule you mentioned, to .0.0/24, but no joy still.

                                  Thoughts?

                                  Thanks!!!

                                  A 1 Reply Last reply Feb 4, 2021, 2:47 AM Reply Quote 0
                                  • A
                                    AB5G @arrmo
                                    last edited by AB5G Feb 4, 2021, 2:55 AM Feb 4, 2021, 2:47 AM

                                    @arrmo
                                    You should not see packets coming from 192.168.0.243 on pf. You should be natting OW LAN to 192.168.253.3/32 on the OpenWrt box. With this your NAT rule of 192.168.253.0/24 will work.

                                    In case you do not want to NAT on the OpenWrt side then on the WG interface (firewall rules) in pf you need to allow 192.168.0.0/24 segment (right now you are only allowing 192.168.253.0/24 segment). Then you'd need a NAT rule for 192.168.0.0/24.

                                    Don't forget to restart the tunnel after the NAT rule change

                                    A 1 Reply Last reply Feb 4, 2021, 3:48 AM Reply Quote 1
                                    • A
                                      arrmo @AB5G
                                      last edited by Feb 4, 2021, 3:48 AM

                                      @ab5g said in Client subnet not accessible (and no internet):

                                      In case you do not want to NAT on the OpenWrt side then on the WG interface (firewall rules) in pf you need to allow 192.168.0.0/24 segment (right now you are only allowing 192.168.253.0/24 segment). Then you'd need a NAT rule for 192.168.0.0/24.

                                      That makes sense - thanks! It seems like the interfaces all get auto-rules generated (even in Manual mode) ... is that right? But as you say, I'll add the .0.x rule, see if I get that working correctly.

                                      BTW, I seem to have a rule for "Wireguard" (group?) - thinking that was from different things I was trying. Assuming that's not needed (i.e. no rules under that Group), right?

                                      Thanks again!!

                                      A 1 Reply Last reply Feb 4, 2021, 3:50 AM Reply Quote 0
                                      • A
                                        AB5G @arrmo
                                        last edited by AB5G Feb 4, 2021, 3:51 AM Feb 4, 2021, 3:50 AM

                                        @arrmo said in Client subnet not accessible (and no internet):

                                        It seems like the interfaces all get auto-rules generated (even in Manual mode) ... is that right?

                                        Yes

                                        "Wireguard" (group?)

                                        Yeah you can ignore that - create all rules under WG Interface that you created.

                                        A 1 Reply Last reply Feb 4, 2021, 10:06 PM Reply Quote 1
                                        • A
                                          arrmo @AB5G
                                          last edited by Feb 4, 2021, 10:06 PM

                                          Perfect, that makes sense (auto rules to manual list). Thanks!

                                          @ab5g said in Client subnet not accessible (and no internet):

                                          Yeah you can ignore that - create all rules under WG Interface that you created.

                                          Well ... 😉. I did that, and broke things as a result. Still trying to wrap my head around it, but I seem to need the WireGuard (tunnel / group?) rule in place. I was thinking that something wasn't getting through the WG (interface) rule, but it may not be getting there perhaps? It's not quite clear to my why yet, sorry! What's also interesting ... with both rules in place, I only see states showing up in WireGuard (tunnel), not WG (interface) ... is that because somehow once the tunnel passes it the interface doesn't see it?

                                          Sorry, just a bit confused yet. But that's my head, I know 🤣.

                                          Thanks!

                                          A 1 Reply Last reply Feb 5, 2021, 1:32 AM Reply Quote 0
                                          20 out of 33
                                          • First post
                                            20/33
                                            Last post
                                          Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.
                                            This community forum collects and processes your personal information.
                                            consent.not_received