Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    Subnets can't communicate

    Scheduled Pinned Locked Moved Routing and Multi WAN
    29 Posts 5 Posters 3.4k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • M
      marvosa
      last edited by marvosa

      Personally, I'd add a 2nd NIC to each box, then create a transit network between the two LAN interfaces and adjust your static routes accordingly.

      1 Reply Last reply Reply Quote 1
      • johnpozJ
        johnpoz LAYER 8 Global Moderator
        last edited by

        ^ exactly... if your going to do downstream router there needs to be a transit network.. NO HOSTS.. If your going to leverage a network with hosts on it, then you would need to be natting at the downstream router. Or you would need to put routes on every host in this network your using as transit. Or you will have asymmetrical routing issues.

        An intelligent man is sometimes forced to be drunk to spend time with his fools
        If you get confused: Listen to the Music Play
        Please don't Chat/PM me for help, unless mod related
        SG-4860 24.11 | Lab VMs 2.8, 24.11

        1 Reply Last reply Reply Quote 0
        • W
          WisceBIat
          last edited by WisceBIat

          Annotation 2020-07-04 172846.jpg

          I've made some changes like adding extra interfaces. How would I go about setting static routes now?

          1 Reply Last reply Reply Quote 0
          • V
            viragomann
            last edited by

            System > Routing

            First go to the gateway tab and add a new gateway, select the proper interface, enter a name and the WAN IP of pfSense 2.
            Then go to the Static Routes tab and add a route. Type in 192.168.2.0/24 into the Destination network box and select the gateway you've added before.

            If the OPT1 IP of pfSense 1 is the default gateway on pfSense 2 there will be nothing to do here. However, in event ExpressVPN is the default GW you have also to add a static route to pfSense 2 in the same way with the respective other network and GW IP.

            W 1 Reply Last reply Reply Quote 0
            • W
              WisceBIat @viragomann
              last edited by

              @viragomann What would be the proper interface for the default gateway ?

              1 Reply Last reply Reply Quote 0
              • W
                WisceBIat
                last edited by WisceBIat

                This still isn't working.

                on pfSense 1 I created another default gateway, using OPT1 interface and giving it the IP of WAN attached to pfSense 2. Then a static route pointing to 192.168.2.0/24 and using the newly created default gateway.

                Next I went onto pfSense 2 and created another default gateway on WAN, and gave it the IP of OPT1 attached to pfSense 1. Then a static route pointing to 192.168.1.0 using the newly created gateway.

                1 Reply Last reply Reply Quote 0
                • johnpozJ
                  johnpoz LAYER 8 Global Moderator
                  last edited by johnpoz

                  Are you natting on pfsense2, to use pfsense as downstream router you have little use of natting. Especially if your connecting them via a transit network like you have shown.

                  BTW - why are you wanting to do this? As a learning experience? Why can you not just connect this 192.168.2 directly to pfsense1?

                  Also this is all running on virtual - your naming interfaces vmbrX

                  Is this the same VM host for both pfsense1 and 2? Oh just notice your bigger proxmox box.. so you have this all setup in virtual...

                  An intelligent man is sometimes forced to be drunk to spend time with his fools
                  If you get confused: Listen to the Music Play
                  Please don't Chat/PM me for help, unless mod related
                  SG-4860 24.11 | Lab VMs 2.8, 24.11

                  1 Reply Last reply Reply Quote 0
                  • W
                    WisceBIat
                    last edited by

                    pfSense 2 is doing NAT because I followed a guide to make all traffic go through ExpressVPN.
                    All this is done on a proxmox machine and I followed a guide to setup pfsense and expressvpn from their website

                    1 Reply Last reply Reply Quote 0
                    • johnpozJ
                      johnpoz LAYER 8 Global Moderator
                      last edited by

                      @WisceBIat said in Subnets can't communicate:

                      I followed a guide to setup pfsense and expressvpn from their website

                      Well that is problem 1 ;)

                      If you want expressvpn to be used by devices on your network, then just set it up on your edge pfsense.. And policy route whatever traffic or vlans you want to use that.. Is that the only reason your trying to setup a downstream pfsense?

                      An intelligent man is sometimes forced to be drunk to spend time with his fools
                      If you get confused: Listen to the Music Play
                      Please don't Chat/PM me for help, unless mod related
                      SG-4860 24.11 | Lab VMs 2.8, 24.11

                      W 1 Reply Last reply Reply Quote 0
                      • W
                        WisceBIat @johnpoz
                        last edited by

                        @johnpoz said in Subnets can't communicate:

                        @WisceBIat said in Subnets can't communicate:

                        I followed a guide to setup pfsense and expressvpn from their website

                        Well that is problem 1 ;)

                        If you want expressvpn to be used by devices on your network, then just set it up on your edge pfsense.. And policy route whatever traffic or vlans you want to use that.. Is that the only reason your trying to setup a downstream pfsense?

                        I was actually trying to nest, or chain pfSense VPNs together. The expressvpn part works perfectly. I just thought it would be easy to get the hosts on both lan's communicating, but so far it's been a 2 month ordeal that I can't figure out.

                        1 Reply Last reply Reply Quote 0
                        • johnpozJ
                          johnpoz LAYER 8 Global Moderator
                          last edited by johnpoz

                          Well if your forcing traffic out some vpn on pfsense 2, how do you think it would ever talk to anything connected off pfsense1?

                          And if your natting traffic on pfsense 2, you would have to port forward on pfsense 2 if you want anything on pfsense to start a conversation with with something behind pfsense 2.

                          What I would suggest is turn off all the vpn stuff, and just setup your 2 pfsense to talk to each other via your transit network.. This should take all of about 2 minutes.

                          Once you have that working, then you can play with whatever you want to play with vpn access. But there is zero reason to be natting on that 2nd pfsense.. Since any traffic from it, be it vpn traffic or normal traffic would be natted at your edge pfsense.

                          Pfsense will auto create outbound nats for you for downstream networks once you create a route to them.. Unless you have turned off auto outbound nat for some reason - like I don't know following some shit vpn service guide ;) Sorry but have yet to see one written by anyone with clue one.. But yeah trust all your traffic with a service that doesn't know the basics of networking.. /rant on vpn service guides ;)

                          An intelligent man is sometimes forced to be drunk to spend time with his fools
                          If you get confused: Listen to the Music Play
                          Please don't Chat/PM me for help, unless mod related
                          SG-4860 24.11 | Lab VMs 2.8, 24.11

                          W 1 Reply Last reply Reply Quote 0
                          • W
                            WisceBIat @johnpoz
                            last edited by

                            @johnpoz said in Subnets can't communicate:

                            Well if your forcing traffic out some vpn on pfsense 2, how do you think it would ever talk to anything connected off pfsense1?

                            And if your natting traffic on pfsense 2, you would have to port forward on pfsense 2 if you want anything on pfsense to start a conversation with with something behind pfsense 2.

                            What I would suggest is turn off all the vpn stuff, and just setup your 2 pfsense to talk to each other via your transit network.. This should take all of about 2 minutes.

                            Once you have that working, then you can play with whatever you want to play with vpn access. But there is zero reason to be natting on that 2nd pfsense.. Since any traffic from it, be it vpn traffic or normal traffic would be natted at your edge pfsense.

                            Pfsense will auto create outbound nats for you for downstream networks once you create a route to them.. Unless you have turned off auto outbound nat for some reason - like I don't know following some shit vpn service guide ;) Sorry but have yet to see one written by anyone with clue one.. But yeah trust all your traffic with a service that doesn't know the basics of networking.. /rant on vpn service guides ;)

                            oddly enough, even with NAT disabled and expressvpn disconnected, I still can't get both subnets to communicate

                            1 Reply Last reply Reply Quote 0
                            • johnpozJ
                              johnpoz LAYER 8 Global Moderator
                              last edited by johnpoz

                              Well your going to have to spell out what you have done if you want help what your missing or what you did wrong.

                              You have a transit network setup from your drawing..

                              So without any routing or gateways created can client on pfsense lan ping the IP of the pfsense 2 transit interface? 192.168.3.X from your drawing... vmbr4? .1 or .2? Use of a /30 where would make it clearer or that is a transit..

                              so

                              pfsense1 - 192.168.3.1 -- 192.168.3/30 --- 192.168.3.2 -- pfsense 2

                              Now client on 192.168.1.x should be able to ping 3.2, as long as you have allowed that on pfsense2 transit interface. Assuming pfs1 lan is default any any rules, and your not forcing traffic out some vpn on pfsense1

                              Once you can ping 3.2, then allow on your pfsense2 transit interface traffic that you want to get to 192.168.2/24

                              Setup a gateway on pfs1 that points to 3.2
                              Setup a route on pfs1 that points 192.168.2/24 to 3.2

                              If this transit network on pfs2 is wan, then its default gateway should point to 3.1 and that is all you would have to do. lan on both pfsense would be able to talk to each other.. And pfs2 lan should be able to get to the internet. Even with nat turned off on pfsense2

                              You will have to allow traffic on pfs1 transit interface for the downstream networks and the transit network if you want pfs2 to be able to get to the internet or pfs1 lan.

                              Again the auto outbound nat would create a outbound nat for your downstream network when you create a route to it.

                              An intelligent man is sometimes forced to be drunk to spend time with his fools
                              If you get confused: Listen to the Music Play
                              Please don't Chat/PM me for help, unless mod related
                              SG-4860 24.11 | Lab VMs 2.8, 24.11

                              1 Reply Last reply Reply Quote 0
                              • W
                                WisceBIat
                                last edited by

                                a0fca7b0-7924-4d9d-aa6d-d75456f56cfc-image.png

                                small changes made. Went from /24 to /30 in the transit network. Also added some IP information.

                                Right now PC 1 can ping 192.168.2.1
                                PC 2 can ping 192.168.1.2

                                PC 1 can't ping PC 2 and vice versa

                                1 Reply Last reply Reply Quote 0
                                • W
                                  WisceBIat
                                  last edited by

                                  Anyone know what could be preventing the PCs from pinging each other? They can ping the interfaces of both pfSense boxes, so maybe it's a firewall issue or something on windows?

                                  johnpozJ 1 Reply Last reply Reply Quote 0
                                  • A
                                    akuma1x
                                    last edited by akuma1x

                                    To pass ping traffic, you need specific rules - not TCP, not UDP, but it's specifically ICMP traffic. Over a transit network, like you've got setup, I'm pretty sure you still need the rules for the interfaces. Maybe one of the pros can comment on that for sure.

                                    Also, since these are "different" subnets, and you said "Windows", you need to open the internal firewall rules on the computers to allow this traffic from the other subnets. Windows considers traffic from machines NOT on the same subnet to be hostile. I know in your situation they aren't hostile, but Windows is programmed that way.

                                    Jeff

                                    1 Reply Last reply Reply Quote 0
                                    • johnpozJ
                                      johnpoz LAYER 8 Global Moderator @WisceBIat
                                      last edited by johnpoz

                                      @WisceBIat said in Subnets can't communicate:

                                      Anyone know what could be preventing the PCs from pinging each other?

                                      The firewalls on the devices themselves.. If you can ping pfsense interface in the network ie your 2.1 and 1.2 address it screams your PCs firewalls.

                                      Can tell you for sure that windows out of the box firewall does not allow pings from remote networks..

                                      Simple way to validate that.. While your pinging pc2 from pc1, sniff on pfsense2 interface 2.1 - do you see it sending the pings on to pc2?

                                      This is a very common mistake users make when they start to segment.. They forget about host firewalls and security software they are running, which might allow traffic of xyz type from the same network the device is on, but blocks when its not a local network.. Like what your doing pc1 is a different network than pc2.. Its not going to allow pinging unless you tell its firewall to allow it, or turn it off, etc.

                                      An intelligent man is sometimes forced to be drunk to spend time with his fools
                                      If you get confused: Listen to the Music Play
                                      Please don't Chat/PM me for help, unless mod related
                                      SG-4860 24.11 | Lab VMs 2.8, 24.11

                                      1 Reply Last reply Reply Quote 0
                                      • W
                                        WisceBIat
                                        last edited by

                                        Ughh it was the firewalls on windows 10 the entire time!!! I had allowed ipv4 echo requests, but it looks like that wasn't enough because I just fully disabled it and now it works

                                        1 Reply Last reply Reply Quote 0
                                        • johnpozJ
                                          johnpoz LAYER 8 Global Moderator
                                          last edited by johnpoz

                                          Great now that you have connectivity working between your 2 networks via your transit network.

                                          I assume pc2 has internet access as well?

                                          Now you can start playing with vpn stuff if you so desire. You have disabled nat I take it on pfs2 I take it..

                                          If you need help with the vpn stuff just ask.. happy to help.

                                          I really do not see the point of pfs2 in your setup to be honest, other than a learning experience it serves no real purpose that I can see.

                                          An intelligent man is sometimes forced to be drunk to spend time with his fools
                                          If you get confused: Listen to the Music Play
                                          Please don't Chat/PM me for help, unless mod related
                                          SG-4860 24.11 | Lab VMs 2.8, 24.11

                                          W 1 Reply Last reply Reply Quote 0
                                          • W
                                            WisceBIat @johnpoz
                                            last edited by WisceBIat

                                            @johnpoz

                                            Internet is working as well! I will now try to re-connect the VPN and re-enable the NAT. Technically all I need is port 22/SSH open because that's the only way I'll be communicating between both LANs. I will see if I can just do port forwarding SSH and that will be good enough. Thanks for all your help so far

                                            1 Reply Last reply Reply Quote 0
                                            • First post
                                              Last post
                                            Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.