Navigation

    Netgate Discussion Forum
    • Register
    • Login
    • Search
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search

    pfsense / HE tunnel / client vm problem

    IPv6
    3
    29
    314
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • C
      capt_scurrrvy last edited by johnpoz

      Hi folks,

      I have pfsense 2.4.5 running with HE tunnel configured. This runs as a VM on Proxmox 6.x. Two VMs behind pfsense have different working abilities.

      Proxmox has three bridges:
      vmbr0 - public (connected to nic)
      vmbr1 - VM's I want to be on 10.x.x.x subnet
      vmbr2 - VM's I want to be on the ipv6 tunnel

      pfsense has two nics, the WAN connected to vmbr0 and the LAN connected to vmbr2.
      Following the guides, created WANv6 and the HE tunnel. Pfsense can ping6 sites like www.yahoo.com. DHCPv6 Router Advertisements set to Managed.

      Two VMs, one Centos 8.x and one Windows 8.x, both connected to Proxmox's vmbr2.

      The Windows 8.x was formerly connected to vmbr1 but moved to vmbr2 to test. The Windows VM works fine after disabling ipv4. Web browsing to known ipv6 sites worked.

      The Centos VM only works partially. If the net manager is set to automatic, ipv6 address gets assigned and ping6 / dns lookups work, but browsing does not. If set to automatic-DHCP only, the address gets assigned but ping6 and dns lookups fail.

      Not sure how to trouble shoot as one client vm works while the other does not. I also tried with a debian based vm and had the same experience as the Centos VM.

      Thank you.

      Gertjan 1 Reply Last reply Reply Quote 0
      • Gertjan
        Gertjan @capt_scurrrvy last edited by Gertjan

        @capt_scurrrvy said in psfense / HE tunnel / client vm problem:

        If set to automatic-DHCP only

        DHCP is what one could consider as "DHCPv4". Your "DHCP only" means to me : IPv4 only.
        DHCP6 exists for obtaining an IPv6. You've set that up ? The IPv6 gets assigned ? Does your Centos request one ? Answers are in your DHCP(4+6) pfSense log.

        I'm using myself a pfSense running in a Hyper-V VM.
        Two hardware NIC's assigned for WAN - and one for LAN - the Hyper-V host - a W10-Pro - is assigned to the bridge and uses VM "LAN".

        I've a vanilla Debian 10 VM installed on the same host, installed from scratch : it gets a IPv4 and an IPv6 from pfSense, as the DHCP (v4) client and DHCPv6 clients are activated by default.... I guess. Don't remember if I had to edit /etc/network/interfaces to make IPv6 work.

        Btw : never used Centos - and it looks like I will never have a chance to see it in action.

        C 1 Reply Last reply Reply Quote 0
        • C
          capt_scurrrvy @Gertjan last edited by capt_scurrrvy

          @gertjan
          Sorry for not being more specific. The Centos VM, configured via the gui Net Manager has ipv4 disabled. The "automatic-dhcp only" I mentioned is under the ipv6 tab. On that tab is "Automatic", "Automatic-DHCP only", "Manual", "Disable". If I switch to the "Automatic" option, ping6 and DNS works but not browsing. "Automatic-DHCP only" selected, ping6 and DNS does not work. Both modes, a valid ipv6 address gets assigned and the /etc/resolv.conf is updated.

          The VM gets a ipv6 address. This is shown in the gui and on the command line. The DHCPv6 lease list on pfsense shows the entry too.

          I have DHCPv4 server disabled on pfsense. I am trying to go ipv6 only for this learning adventure.

          Gertjan johnpoz 2 Replies Last reply Reply Quote 0
          • Gertjan
            Gertjan @capt_scurrrvy last edited by Gertjan

            @capt_scurrrvy said in psfense / HE tunnel / client vm problem:

            The "automatic-dhcp only" I mentioned is under the ipv6 tab. On that tab is "Automatic", "Automatic-DHCP only", "Manual", "Disable". If I switch to the "Automatic" option, ping6 and DNS works but not browsing. "Automatic-DHCP only" selected, ping6 and DNS does not work. Both modes, a valid ipv6 address gets assigned and the /etc/resolv.conf is updated.

            You're talking about GUI interfaces and OS's that most of us have never seen.
            What about the common language - the one that works best and has a small bit foot-print :

            route -6
            
            ifconfig
            

            They will show you that you've got an lease IPv6. It should be in the IPv6 pool you set up on pfSense.
            That you have a DNS assigned - should be the IPv6 of pfSense on that LAN.
            And an IPv6 gateway assigned - should be the IPv6 of pfSense on that LAN.

            If a Win10 VM instance manages to pull, using DHCPv6, a gateway, network, DNS and IP from your pfSense, knowing that Microsoft didn't invent all this (they probably borrowed the IPv6 from 'elsewhere'), I'm pretty confident that your pfSense v6 setup is correct.

            Still, IPv6 with no IPv4 on a network, that's hardcore.

            C 1 Reply Last reply Reply Quote 0
            • C
              capt_scurrrvy @Gertjan last edited by

              @gertjan

              Below are both of the VM's output.

              Centos when configured for DHCPV6C=yes does not appear to set a gateway. When configured for IPV6_AUTOCONF=yes, route -6 would show [::]/0 _gateway UG.

              Centos 8 Config
              Relevant ipv6 entries in /etc/sysconfig/network-scripts/ifcfg-ens18
              IPV6INIT=yes
              IPV6_AUTOCONF=no
              IPV6_DEFROUTE=yes
              IPV6_FAILURE_FATAL=no
              IPV6_ADDR_GEN_MODE=stable-privacy
              IPV6_PRIVACY=no
              DHCPV6C=yes

              route -6
              Kernel IPv6 routing table
              Destination                    Next Hop                   Flag Met Ref Use If
              localhost/128                  [::]                       U    256 2      0 lo
              ipv6test/128                   [::]                       U    100 1      0 ens18
              fe80::/64                      [::]                       U    100 2      0 ens18
              [::]/0                         [::]                       !n   -1  1      0 lo
              localhost/128                  [::]                       Un   0   4      0 lo
              ipv6test/128                   [::]                       Un   0   3      0 ens18
              ipv6test/128                   [::]                       Un   0   3      0 ens18
              ff00::/8                       [::]                       U    256 3      0 ens18
              [::]/0                         [::]                       !n   -1  1      0 lo
              
              ifconfig
              ens18: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
                      inet6 2001:470:1f1d:bc::fff9  prefixlen 128  scopeid 0x0<global>
                      inet6 fe80::1252:9cd4:4670:46f  prefixlen 64  scopeid 0x20<link>
                      ether 1e:e3:a1:a6:55:f3  txqueuelen 1000  (Ethernet)
                      RX packets 1038  bytes 156850 (153.1 KiB)
                      RX errors 0  dropped 0  overruns 0  frame 0
                      TX packets 829  bytes 95666 (93.4 KiB)
                      TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
              
              lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536
                      inet 127.0.0.1  netmask 255.0.0.0
                      inet6 ::1  prefixlen 128  scopeid 0x10<host>
                      loop  txqueuelen 1000  (Local Loopback)
                      RX packets 292  bytes 31706 (30.9 KiB)
                      RX errors 0  dropped 0  overruns 0  frame 0
                      TX packets 292  bytes 31706 (30.9 KiB)
                      TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
              

              Windows 8 config

              IPv6 Route Table
              Active Routes:
               If Metric Network Destination      Gateway
                4    266 ::/0                     fe80::1:1
                1    306 ::1/128                  On-link
                4    266 2001:470:1f1d:bc::/64    On-link
                4    266 2001:470:1f1d:bc::ff40/128   On-link
                4    266 fe80::/64                On-link
                4    266 fe80::6d5d:6129:be0e:81e1/128  On-link
                1    306 ff00::/8                 On-link
                4    266 ff00::/8                 On-link
              
              Persistent Routes:
                None
              
              Ethernet adapter Ethernet 2:
              
                 Connection-specific DNS Suffix  . :
                 Description . . . . . . . . . . . . : Intel(R) PRO/1000 MT Network Connection
                 Physical Address. . . . . . . . . : 96-E2-5B-D3-F5-E2
                 DHCP Enabled. . . . . . . . . . . : Yes
                 Autoconfiguration Enabled . . . . : Yes
                 IPv6 Address. . . . . . . . . . . : 2001:470:1f1d:bc::ff40(Preferred)
                 Lease Obtained. . . . . . . . . . : Friday, January 22, 2021 06:48:03
                 Lease Expires . . . . . . . . . . : Friday, January 22, 2021 08:48:04
                 Link-local IPv6 Address . . . . . : fe80::6d5d:6129:be0e:81e1%4(Preferred)
                 Default Gateway . . . . . . . . . : fe80::1:1%4
                 DHCPv6 IAID . . . . . . . . . . . : 110551643
                 DHCPv6 Client DUID. . . . . . . . : 00-01-00-01-1E-3B-2A-C5-96-E2-5B-D3-F5-E2
              
                 DNS Servers . . . . . . . . . . . . : 2001:470:1f1d:bc::1
                 NetBIOS over Tcpip. . . . . . . . : Disabled
              
              Gertjan 1 Reply Last reply Reply Quote 0
              • Gertjan
                Gertjan @capt_scurrrvy last edited by Gertjan

                Your Centos :

                @capt_scurrrvy said in psfense / HE tunnel / client vm problem:

                inet6 2001:4x0:1f1d:bc::fff9

                That looks like a Ipv6 from he.net to me ^^

                But the gateway -a local link from pfSense - isn't set.

                C 1 Reply Last reply Reply Quote 0
                • C
                  capt_scurrrvy @Gertjan last edited by

                  @gertjan

                  That looks like a Ipv6 from he.net to me ^^

                  That is correct. As my subject line shows.

                  I don't understand why the gateway isn't being set. Shouldn't that be set by pfsense? Perhaps I have DHCPv6 or router advertisement configured incorrectly? This doesn't explain why the Windows client is getting the gateway set.

                  I followed "Configuring IPv6 Through A Tunnel Broker Service" in the docs.

                  Should I manually set the Centos 8 vm gateway using IPV6_DEFAULTGW in the ifcfg- file?

                  Gertjan 1 Reply Last reply Reply Quote 0
                  • Gertjan
                    Gertjan @capt_scurrrvy last edited by

                    @capt_scurrrvy said in psfense / HE tunnel / client vm problem:

                    Shouldn't that be set by pfsense?

                    Set any of these :

                    20cf6e0d-4256-499d-a39b-a25ae4c6d9d1-image.png

                    except "Disabled" and pfSense should signal devices that it is a router = gateway.

                    Your OS should react on it.
                    Can't say more, as I don't know Centos at all.

                    C 1 Reply Last reply Reply Quote 0
                    • C
                      capt_scurrrvy @Gertjan last edited by

                      @gertjan
                      Pfsense has been set to Managed from the start.

                      I've tried Centos, Debian, and Devuan. None of them get the gateway set. I'll try FreeBSD next. Frustratingly only Windows has worked. Maybe I'll try ReactOS for fun.

                      Thanks for trying.

                      Gertjan 1 Reply Last reply Reply Quote 0
                      • Gertjan
                        Gertjan @capt_scurrrvy last edited by Gertjan

                        @capt_scurrrvy said in psfense / HE tunnel / client vm problem:

                        Debian

                        I just created a HyperV instance from scratch, attached a Debian Buster ISO, mapped the VM instance to the "LAN" interface already used by my pfSense instance, and fired up debian, the installer started, I choose the minimum options (no graphics, just a SSH access).
                        I did not have to login, the debian VM got a IPv4 and a IPv6 from the 'dhcp server pfSense':

                        e2e43b26-3500-4bca-b880-7681edab7120-image.png

                        1 Reply Last reply Reply Quote 0
                        • johnpoz
                          johnpoz LAYER 8 Global Moderator @capt_scurrrvy last edited by

                          @capt_scurrrvy said in psfense / HE tunnel / client vm problem:

                          I am trying to go ipv6 only for this learning adventure.

                          That is going to be very problematic - since the VAST majority of stuff on the net is not IPv6.. Your going to be very limited to where you can actually go ;)

                          C 1 Reply Last reply Reply Quote 0
                          • C
                            capt_scurrrvy @johnpoz last edited by

                            @johnpoz
                            As it is for learning, the number of available site does not bother me.

                            Frustratingly, I have now failed to get FreeBSD to work. So Linux and FreeBSD fail to browse the web.

                            Windows 8 still working with no issue. Browses the web no problem.

                            There is something I am not seeing. To have one VM that works and the others not is disappointing. I also don't understand how the linux machines can get an address but cannot browse.

                            Gertjan 1 Reply Last reply Reply Quote 0
                            • Gertjan
                              Gertjan @capt_scurrrvy last edited by

                              @capt_scurrrvy said in psfense / HE tunnel / client vm problem:

                              .... cannot browse.

                              That's far to generic as a fault description.
                              These two words can be easily broke down into :
                              Gateway not known -no route to gateway.
                              DNS not set => resolving doesn't work.

                              When I installed the "Debian Buster 10.7 ISO", I had it's 'network interface' attached to the "virtual LAN switch", created by Hyper-V, the one that pfSense, also a VM, uses. It is also attached to a physical NIC called LAN, so I can use more physical devices on the LAN.
                              During the install, the Debian started to do a massive apt-get to update and install most needed packages.
                              I've added a

                              iface 
                                   iface eth0 inet6 dhcp
                              

                              To the /etc/networking/interfaces to kick off IPv6 using the DHCP6 server on pfSense's LAN.

                              You have the single default firewall rule on your pfSense, right ?

                              C 1 Reply Last reply Reply Quote 0
                              • C
                                capt_scurrrvy @Gertjan last edited by

                                @gertjan

                                Thank you for trying to solve this.

                                Promox is the host with a bridge vmbr2 configured. Pfsense vm LAN nic and the test vm's nic are connected to vmbr2.

                                I am currently trying Debian 10's live CD.

                                Shortening my full ipv6 address to just the last part.

                                ip address shows for ens18:
                                ::fffa/128 (fffa being inside the range of ff00 to ffff I configured for dhcp6 server on pfsense)
                                fe80::a518/64

                                ip -6 route show
                                ::fffa
                                ::/64
                                fe80::/64
                                default via fe80::1:1

                                DNS and ping6 work. I ping6 -c 2 www.yahoo.com and get a successful dns lookup of an ipv6 address and two successful pings.

                                By browser, I mean web browser, specifically Firefox. Type www.google.com in address bar and eventually get error server not found or time out. I don't remember which error.

                                I then tried apt-get update. It displays the ipv6 address of where it is trying to go but eventually times out.

                                johnpoz 1 Reply Last reply Reply Quote 0
                                • johnpoz
                                  johnpoz LAYER 8 Global Moderator @capt_scurrrvy last edited by

                                  @capt_scurrrvy said in psfense / HE tunnel / client vm problem:

                                  fe80::a518/64

                                  You understand that is link-local, you would not be able to get past your local network with such an address. Ie while you could look up stuff via local dns, you wouldn't be able to get to the internet, just like you ca not get to the internet with a rfc1918 address. Or a 169.254.x.x address

                                  C 1 Reply Last reply Reply Quote 0
                                  • C
                                    capt_scurrrvy @johnpoz last edited by

                                    @johnpoz
                                    Understood. fe80 is local-link.
                                    Why is pfsense not providing the GW?
                                    Why does ping6 work? DNS works. The ipv6 address for DNS is the LAN interface on pfsense.

                                    I don't like Windows (not wanting to start a comment war), but it works. I only turned off ipv4 in the properties. It is set to get ipv6 automatically.

                                    Next is to try setting the default route manually. Would that be the address of the LAN nic on pfsense?

                                    1 Reply Last reply Reply Quote 0
                                    • C
                                      capt_scurrrvy last edited by

                                      Side question:
                                      Is there a way to edit the title of this thread? I noticed today I typed psfense instead of pfsense.

                                      johnpoz 1 Reply Last reply Reply Quote 0
                                      • johnpoz
                                        johnpoz LAYER 8 Global Moderator @capt_scurrrvy last edited by johnpoz

                                        Um so windows got a global address? If all your linux has a link-local, still not going anywhere even if it uses a link-local gateway. Unless you natted your link-local ipv6 to a global ipv6. Which isn't how it works ;)

                                        Lets see the output of windows ipconfig /all

                                        edit: I edited your title for you.

                                        Also show us the ifconfig for you linux.. Here is my linux box with working IPv6

                                        global.png

                                        See how it has a global address, along with its link local address.

                                        C 1 Reply Last reply Reply Quote 0
                                        • C
                                          capt_scurrrvy @johnpoz last edited by

                                          @johnpoz
                                          I believe what you are asking for is in the post above. Under Windows 8 config.
                                          I don't remember if that was ipconfig or ipconfig /all. The routes are there too.

                                          Thanks.

                                          johnpoz 1 Reply Last reply Reply Quote 0
                                          • johnpoz
                                            johnpoz LAYER 8 Global Moderator @capt_scurrrvy last edited by johnpoz

                                            Sorry I missed that - yeah it has a global address.. Does your linux?

                                            edit: Oh missed that too - it does.. But your routing table is not correct..

                                            Here for example is my default route on actual interface

                                            route.png

                                            It can point to the link-local of your gateway. But not going to work on your lo interface.

                                            C 1 Reply Last reply Reply Quote 0
                                            • C
                                              capt_scurrrvy @johnpoz last edited by

                                              @johnpoz
                                              In that same post is my test Centos 8 vm. It did get assigned an address, but from the route list, doesn't look like the default route gets set.

                                              johnpoz 1 Reply Last reply Reply Quote 0
                                              • johnpoz
                                                johnpoz LAYER 8 Global Moderator @capt_scurrrvy last edited by

                                                Your default is set, but only for lo - which never going to work.

                                                C 1 Reply Last reply Reply Quote 0
                                                • C
                                                  capt_scurrrvy @johnpoz last edited by

                                                  @johnpoz
                                                  So is that Centos failing to request a route from the pfsense DHCP6 server or similarly failing to get the pfsense router advertisement?

                                                  johnpoz 1 Reply Last reply Reply Quote 0
                                                  • johnpoz
                                                    johnpoz LAYER 8 Global Moderator @capt_scurrrvy last edited by

                                                    Clients do not get "routes" from dhcpv6.. Not how IPv6 works. RA is used..

                                                    C 2 Replies Last reply Reply Quote 0
                                                    • C
                                                      capt_scurrrvy @johnpoz last edited by

                                                      @johnpoz
                                                      Does that mean pfsense's router advertisement is not sending the information?

                                                      1 Reply Last reply Reply Quote 0
                                                      • C
                                                        capt_scurrrvy last edited by

                                                        I noticed the fe80 address of the LAN nic of pfsense is being set automatically as the default route on the linux box.

                                                        [root@localhost ~]# ip -6 route
                                                        unreachable ::/96 dev lo metric 1024 error -113 pref medium
                                                        unreachable ::ffff:0.0.0.0/96 dev lo metric 1024 error -113 pref medium
                                                        2001:470:1f1d:bc::ff04 dev eth1 proto kernel metric 100 pref medium
                                                        2001:470:1f1d:bc::/64 dev eth1 proto ra metric 100 pref medium
                                                        unreachable 2002:a00::/24 dev lo metric 1024 error -113 pref medium
                                                        unreachable 2002:7f00::/24 dev lo metric 1024 error -113 pref medium
                                                        unreachable 2002:a9fe::/32 dev lo metric 1024 error -113 pref medium
                                                        unreachable 2002:ac10::/28 dev lo metric 1024 error -113 pref medium
                                                        unreachable 2002:c0a8::/32 dev lo metric 1024 error -113 pref medium
                                                        unreachable 2002:e000::/19 dev lo metric 1024 error -113 pref medium
                                                        unreachable 3ffe:ffff::/32 dev lo metric 1024 error -113 pref medium
                                                        fe80::/64 dev eth1 proto kernel metric 100 pref medium
                                                        fe80::/64 dev eth0 proto kernel metric 256 pref medium
                                                        default via fe80::8802:71ff:fe34:93cb dev eth1 proto ra metric 100 pref medium
                                                        

                                                        I don't understand why that would happen.

                                                        The ifcg_eth1 file

                                                        DEFROUTE=yes
                                                        IPV4_FAILURE_FATAL=no
                                                        IPV6INIT=yes
                                                        IPV6_AUTOCONF=yes
                                                        IPV6_DEFROUTE=yes
                                                        IPV6_FAILURE_FATAL=no
                                                        IPV6_ADDR_GEN_MODE=stable-privacy
                                                        
                                                        1 Reply Last reply Reply Quote 0
                                                        • C
                                                          capt_scurrrvy last edited by

                                                          Still trying to figure out why I can't get non Windows machines to communication via ipv6.

                                                          On pfsense, I have disabled DHCPv6 and Router Advertisements.

                                                          On the Centos vm, I statically set the ipv6 address and the gateway.

                                                          Just as previously, the VM can:
                                                          ping6
                                                          (this even does the correct DNS lookup)

                                                          traceroute6 and tracepath6 both show going out the Hurricane Electric tunnel.

                                                          Just as previously, the VM cannot:
                                                          Use a browser to view cnn, google, redhat.

                                                          I even put an allow any type rule on the pfsense WAN, LAN, and WANv6.
                                                          I disabled the firewall running on Centos.

                                                          It is as if TCP is not working. However, tcpdump shows traffic.

                                                          1 Reply Last reply Reply Quote 0
                                                          • C
                                                            capt_scurrrvy @johnpoz last edited by

                                                            @johnpoz
                                                            Upgraded to 2.5.0. Had to manually add the default route for my ipv4 gateway on the WAN. See Default gateway can not be created with GUI

                                                            Surprise, Firefox on CentOS VM started to get to ipv6 web sites. This is with a manually assigned ipv6 for CentOS. Not sure what has changed between 2.4.5 and 2.5.

                                                            Gertjan 1 Reply Last reply Reply Quote 0
                                                            • Gertjan
                                                              Gertjan @capt_scurrrvy last edited by

                                                              I upgraded this morning my main 'company' pfSense to 2.5.0.
                                                              I'm using he.net for my my IPv6 'needs'.

                                                              I had nothing to do.

                                                              Everything came up and was working fine.
                                                              ( + captive portal using FreeRadius - OpenVPN server for my remote access).

                                                              Even a non-native package I installed many years ago was upgraded and kept on running.

                                                              1 Reply Last reply Reply Quote 0
                                                              • First post
                                                                Last post

                                                              Products

                                                              • Platform Overview
                                                              • TNSR
                                                              • pfSense Plus
                                                              • Appliances

                                                              Services

                                                              • Training
                                                              • Professional Services

                                                              Support

                                                              • Subscription Plans
                                                              • Contact Support
                                                              • Product Lifecycle
                                                              • Documentation

                                                              News

                                                              • Media Coverage
                                                              • Press
                                                              • Events

                                                              Resources

                                                              • Blog
                                                              • FAQ
                                                              • Find a Partner
                                                              • Resource Library
                                                              • Security Information

                                                              Company

                                                              • About Us
                                                              • Careers
                                                              • Partners
                                                              • Contact Us
                                                              • Legal
                                                              Our Mission

                                                              We provide leading-edge network security at a fair price - regardless of organizational size or network sophistication. We believe that an open-source security model offers disruptive pricing along with the agility required to quickly address emerging threats.

                                                              Subscribe to our Newsletter

                                                              Product information, software announcements, and special offers. See our newsletter archive to sign up for future newsletters and to read past announcements.

                                                              © 2021 Rubicon Communications, LLC | Privacy Policy