pfsense / HE tunnel / client vm problem
-
Hi folks,
I have pfsense 2.4.5 running with HE tunnel configured. This runs as a VM on Proxmox 6.x. Two VMs behind pfsense have different working abilities.
Proxmox has three bridges:
vmbr0 - public (connected to nic)
vmbr1 - VM's I want to be on 10.x.x.x subnet
vmbr2 - VM's I want to be on the ipv6 tunnelpfsense has two nics, the WAN connected to vmbr0 and the LAN connected to vmbr2.
Following the guides, created WANv6 and the HE tunnel. Pfsense can ping6 sites like www.yahoo.com. DHCPv6 Router Advertisements set to Managed.Two VMs, one Centos 8.x and one Windows 8.x, both connected to Proxmox's vmbr2.
The Windows 8.x was formerly connected to vmbr1 but moved to vmbr2 to test. The Windows VM works fine after disabling ipv4. Web browsing to known ipv6 sites worked.
The Centos VM only works partially. If the net manager is set to automatic, ipv6 address gets assigned and ping6 / dns lookups work, but browsing does not. If set to automatic-DHCP only, the address gets assigned but ping6 and dns lookups fail.
Not sure how to trouble shoot as one client vm works while the other does not. I also tried with a debian based vm and had the same experience as the Centos VM.
Thank you.
-
@capt_scurrrvy said in psfense / HE tunnel / client vm problem:
If set to automatic-DHCP only
DHCP is what one could consider as "DHCPv4". Your "DHCP only" means to me : IPv4 only.
DHCP6 exists for obtaining an IPv6. You've set that up ? The IPv6 gets assigned ? Does your Centos request one ? Answers are in your DHCP(4+6) pfSense log.I'm using myself a pfSense running in a Hyper-V VM.
Two hardware NIC's assigned for WAN - and one for LAN - the Hyper-V host - a W10-Pro - is assigned to the bridge and uses VM "LAN".I've a vanilla Debian 10 VM installed on the same host, installed from scratch : it gets a IPv4 and an IPv6 from pfSense, as the DHCP (v4) client and DHCPv6 clients are activated by default.... I guess. Don't remember if I had to edit /etc/network/interfaces to make IPv6 work.
Btw : never used Centos - and it looks like I will never have a chance to see it in action.
-
@gertjan
Sorry for not being more specific. The Centos VM, configured via the gui Net Manager has ipv4 disabled. The "automatic-dhcp only" I mentioned is under the ipv6 tab. On that tab is "Automatic", "Automatic-DHCP only", "Manual", "Disable". If I switch to the "Automatic" option, ping6 and DNS works but not browsing. "Automatic-DHCP only" selected, ping6 and DNS does not work. Both modes, a valid ipv6 address gets assigned and the /etc/resolv.conf is updated.The VM gets a ipv6 address. This is shown in the gui and on the command line. The DHCPv6 lease list on pfsense shows the entry too.
I have DHCPv4 server disabled on pfsense. I am trying to go ipv6 only for this learning adventure.
-
@capt_scurrrvy said in psfense / HE tunnel / client vm problem:
The "automatic-dhcp only" I mentioned is under the ipv6 tab. On that tab is "Automatic", "Automatic-DHCP only", "Manual", "Disable". If I switch to the "Automatic" option, ping6 and DNS works but not browsing. "Automatic-DHCP only" selected, ping6 and DNS does not work. Both modes, a valid ipv6 address gets assigned and the /etc/resolv.conf is updated.
You're talking about GUI interfaces and OS's that most of us have never seen.
What about the common language - the one that works best and has a small bit foot-print :route -6ifconfigThey will show you that you've got an lease IPv6. It should be in the IPv6 pool you set up on pfSense.
That you have a DNS assigned - should be the IPv6 of pfSense on that LAN.
And an IPv6 gateway assigned - should be the IPv6 of pfSense on that LAN.If a Win10 VM instance manages to pull, using DHCPv6, a gateway, network, DNS and IP from your pfSense, knowing that Microsoft didn't invent all this (they probably borrowed the IPv6 from 'elsewhere'), I'm pretty confident that your pfSense v6 setup is correct.
Still, IPv6 with no IPv4 on a network, that's hardcore.
-
Below are both of the VM's output.
Centos when configured for DHCPV6C=yes does not appear to set a gateway. When configured for IPV6_AUTOCONF=yes, route -6 would show [::]/0 _gateway UG.
Centos 8 Config
Relevant ipv6 entries in /etc/sysconfig/network-scripts/ifcfg-ens18
IPV6INIT=yes
IPV6_AUTOCONF=no
IPV6_DEFROUTE=yes
IPV6_FAILURE_FATAL=no
IPV6_ADDR_GEN_MODE=stable-privacy
IPV6_PRIVACY=no
DHCPV6C=yesroute -6 Kernel IPv6 routing table Destination Next Hop Flag Met Ref Use If localhost/128 [::] U 256 2 0 lo ipv6test/128 [::] U 100 1 0 ens18 fe80::/64 [::] U 100 2 0 ens18 [::]/0 [::] !n -1 1 0 lo localhost/128 [::] Un 0 4 0 lo ipv6test/128 [::] Un 0 3 0 ens18 ipv6test/128 [::] Un 0 3 0 ens18 ff00::/8 [::] U 256 3 0 ens18 [::]/0 [::] !n -1 1 0 lo ifconfig ens18: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500 inet6 2001:470:1f1d:bc::fff9 prefixlen 128 scopeid 0x0<global> inet6 fe80::1252:9cd4:4670:46f prefixlen 64 scopeid 0x20<link> ether 1e:e3:a1:a6:55:f3 txqueuelen 1000 (Ethernet) RX packets 1038 bytes 156850 (153.1 KiB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 829 bytes 95666 (93.4 KiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 lo: flags=73<UP,LOOPBACK,RUNNING> mtu 65536 inet 127.0.0.1 netmask 255.0.0.0 inet6 ::1 prefixlen 128 scopeid 0x10<host> loop txqueuelen 1000 (Local Loopback) RX packets 292 bytes 31706 (30.9 KiB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 292 bytes 31706 (30.9 KiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0Windows 8 config
IPv6 Route Table Active Routes: If Metric Network Destination Gateway 4 266 ::/0 fe80::1:1 1 306 ::1/128 On-link 4 266 2001:470:1f1d:bc::/64 On-link 4 266 2001:470:1f1d:bc::ff40/128 On-link 4 266 fe80::/64 On-link 4 266 fe80::6d5d:6129:be0e:81e1/128 On-link 1 306 ff00::/8 On-link 4 266 ff00::/8 On-link Persistent Routes: None Ethernet adapter Ethernet 2: Connection-specific DNS Suffix . : Description . . . . . . . . . . . . : Intel(R) PRO/1000 MT Network Connection Physical Address. . . . . . . . . : 96-E2-5B-D3-F5-E2 DHCP Enabled. . . . . . . . . . . : Yes Autoconfiguration Enabled . . . . : Yes IPv6 Address. . . . . . . . . . . : 2001:470:1f1d:bc::ff40(Preferred) Lease Obtained. . . . . . . . . . : Friday, January 22, 2021 06:48:03 Lease Expires . . . . . . . . . . : Friday, January 22, 2021 08:48:04 Link-local IPv6 Address . . . . . : fe80::6d5d:6129:be0e:81e1%4(Preferred) Default Gateway . . . . . . . . . : fe80::1:1%4 DHCPv6 IAID . . . . . . . . . . . : 110551643 DHCPv6 Client DUID. . . . . . . . : 00-01-00-01-1E-3B-2A-C5-96-E2-5B-D3-F5-E2 DNS Servers . . . . . . . . . . . . : 2001:470:1f1d:bc::1 NetBIOS over Tcpip. . . . . . . . : Disabled -
Your Centos :
@capt_scurrrvy said in psfense / HE tunnel / client vm problem:
inet6 2001:4x0:1f1d:bc::fff9
That looks like a Ipv6 from he.net to me ^^
But the gateway -a local link from pfSense - isn't set.
-
That looks like a Ipv6 from he.net to me ^^
That is correct. As my subject line shows.
I don't understand why the gateway isn't being set. Shouldn't that be set by pfsense? Perhaps I have DHCPv6 or router advertisement configured incorrectly? This doesn't explain why the Windows client is getting the gateway set.
I followed "Configuring IPv6 Through A Tunnel Broker Service" in the docs.
Should I manually set the Centos 8 vm gateway using IPV6_DEFAULTGW in the ifcfg- file?
-
@capt_scurrrvy said in psfense / HE tunnel / client vm problem:
Shouldn't that be set by pfsense?
Set any of these :
except "Disabled" and pfSense should signal devices that it is a router = gateway.
Your OS should react on it.
Can't say more, as I don't know Centos at all. -
@gertjan
Pfsense has been set to Managed from the start.I've tried Centos, Debian, and Devuan. None of them get the gateway set. I'll try FreeBSD next. Frustratingly only Windows has worked. Maybe I'll try ReactOS for fun.
Thanks for trying.
-
@capt_scurrrvy said in psfense / HE tunnel / client vm problem:
Debian
I just created a HyperV instance from scratch, attached a Debian Buster ISO, mapped the VM instance to the "LAN" interface already used by my pfSense instance, and fired up debian, the installer started, I choose the minimum options (no graphics, just a SSH access).
I did not have to login, the debian VM got a IPv4 and a IPv6 from the 'dhcp server pfSense':
-
@capt_scurrrvy said in psfense / HE tunnel / client vm problem:
I am trying to go ipv6 only for this learning adventure.
That is going to be very problematic - since the VAST majority of stuff on the net is not IPv6.. Your going to be very limited to where you can actually go ;)
-
@johnpoz
As it is for learning, the number of available site does not bother me.Frustratingly, I have now failed to get FreeBSD to work. So Linux and FreeBSD fail to browse the web.
Windows 8 still working with no issue. Browses the web no problem.
There is something I am not seeing. To have one VM that works and the others not is disappointing. I also don't understand how the linux machines can get an address but cannot browse.
-
@capt_scurrrvy said in psfense / HE tunnel / client vm problem:
.... cannot browse.
That's far to generic as a fault description.
These two words can be easily broke down into :
Gateway not known -no route to gateway.
DNS not set => resolving doesn't work.When I installed the "Debian Buster 10.7 ISO", I had it's 'network interface' attached to the "virtual LAN switch", created by Hyper-V, the one that pfSense, also a VM, uses. It is also attached to a physical NIC called LAN, so I can use more physical devices on the LAN.
During the install, the Debian started to do a massive apt-get to update and install most needed packages.
I've added aiface iface eth0 inet6 dhcpTo the /etc/networking/interfaces to kick off IPv6 using the DHCP6 server on pfSense's LAN.
You have the single default firewall rule on your pfSense, right ?
-
Thank you for trying to solve this.
Promox is the host with a bridge vmbr2 configured. Pfsense vm LAN nic and the test vm's nic are connected to vmbr2.
I am currently trying Debian 10's live CD.
Shortening my full ipv6 address to just the last part.
ip address shows for ens18:
::fffa/128 (fffa being inside the range of ff00 to ffff I configured for dhcp6 server on pfsense)
fe80::a518/64ip -6 route show
::fffa
::/64
fe80::/64
default via fe80::1:1DNS and ping6 work. I ping6 -c 2 www.yahoo.com and get a successful dns lookup of an ipv6 address and two successful pings.
By browser, I mean web browser, specifically Firefox. Type www.google.com in address bar and eventually get error server not found or time out. I don't remember which error.
I then tried apt-get update. It displays the ipv6 address of where it is trying to go but eventually times out.
-
@capt_scurrrvy said in psfense / HE tunnel / client vm problem:
fe80::a518/64
You understand that is link-local, you would not be able to get past your local network with such an address. Ie while you could look up stuff via local dns, you wouldn't be able to get to the internet, just like you ca not get to the internet with a rfc1918 address. Or a 169.254.x.x address
-
@johnpoz
Understood. fe80 is local-link.
Why is pfsense not providing the GW?
Why does ping6 work? DNS works. The ipv6 address for DNS is the LAN interface on pfsense.I don't like Windows (not wanting to start a comment war), but it works. I only turned off ipv4 in the properties. It is set to get ipv6 automatically.
Next is to try setting the default route manually. Would that be the address of the LAN nic on pfsense?
-
Side question:
Is there a way to edit the title of this thread? I noticed today I typed psfense instead of pfsense. -
Um so windows got a global address? If all your linux has a link-local, still not going anywhere even if it uses a link-local gateway. Unless you natted your link-local ipv6 to a global ipv6. Which isn't how it works ;)
Lets see the output of windows ipconfig /all
edit: I edited your title for you.
Also show us the ifconfig for you linux.. Here is my linux box with working IPv6
See how it has a global address, along with its link local address.
-
@johnpoz
I believe what you are asking for is in the post above. Under Windows 8 config.
I don't remember if that was ipconfig or ipconfig /all. The routes are there too.Thanks.
-
Sorry I missed that - yeah it has a global address.. Does your linux?
edit: Oh missed that too - it does.. But your routing table is not correct..
Here for example is my default route on actual interface
It can point to the link-local of your gateway. But not going to work on your lo interface.
-
@johnpoz
In that same post is my test Centos 8 vm. It did get assigned an address, but from the route list, doesn't look like the default route gets set. -
Your default is set, but only for lo - which never going to work.
-
@johnpoz
So is that Centos failing to request a route from the pfsense DHCP6 server or similarly failing to get the pfsense router advertisement? -
Clients do not get "routes" from dhcpv6.. Not how IPv6 works. RA is used..
-
@johnpoz
Does that mean pfsense's router advertisement is not sending the information? -
I noticed the fe80 address of the LAN nic of pfsense is being set automatically as the default route on the linux box.
[root@localhost ~]# ip -6 route unreachable ::/96 dev lo metric 1024 error -113 pref medium unreachable ::ffff:0.0.0.0/96 dev lo metric 1024 error -113 pref medium 2001:470:1f1d:bc::ff04 dev eth1 proto kernel metric 100 pref medium 2001:470:1f1d:bc::/64 dev eth1 proto ra metric 100 pref medium unreachable 2002:a00::/24 dev lo metric 1024 error -113 pref medium unreachable 2002:7f00::/24 dev lo metric 1024 error -113 pref medium unreachable 2002:a9fe::/32 dev lo metric 1024 error -113 pref medium unreachable 2002:ac10::/28 dev lo metric 1024 error -113 pref medium unreachable 2002:c0a8::/32 dev lo metric 1024 error -113 pref medium unreachable 2002:e000::/19 dev lo metric 1024 error -113 pref medium unreachable 3ffe:ffff::/32 dev lo metric 1024 error -113 pref medium fe80::/64 dev eth1 proto kernel metric 100 pref medium fe80::/64 dev eth0 proto kernel metric 256 pref medium default via fe80::8802:71ff:fe34:93cb dev eth1 proto ra metric 100 pref mediumI don't understand why that would happen.
The ifcg_eth1 file
DEFROUTE=yes IPV4_FAILURE_FATAL=no IPV6INIT=yes IPV6_AUTOCONF=yes IPV6_DEFROUTE=yes IPV6_FAILURE_FATAL=no IPV6_ADDR_GEN_MODE=stable-privacy -
Still trying to figure out why I can't get non Windows machines to communication via ipv6.
On pfsense, I have disabled DHCPv6 and Router Advertisements.
On the Centos vm, I statically set the ipv6 address and the gateway.
Just as previously, the VM can:
ping6
(this even does the correct DNS lookup)traceroute6 and tracepath6 both show going out the Hurricane Electric tunnel.
Just as previously, the VM cannot:
Use a browser to view cnn, google, redhat.I even put an allow any type rule on the pfsense WAN, LAN, and WANv6.
I disabled the firewall running on Centos.It is as if TCP is not working. However, tcpdump shows traffic.
-
@johnpoz
Upgraded to 2.5.0. Had to manually add the default route for my ipv4 gateway on the WAN. See Default gateway can not be created with GUISurprise, Firefox on CentOS VM started to get to ipv6 web sites. This is with a manually assigned ipv6 for CentOS. Not sure what has changed between 2.4.5 and 2.5.
-
I upgraded this morning my main 'company' pfSense to 2.5.0.
I'm using he.net for my my IPv6 'needs'.I had nothing to do.
Everything came up and was working fine.
( + captive portal using FreeRadius - OpenVPN server for my remote access).Even a non-native package I installed many years ago was upgraded and kept on running.
