Possible routing issue, almost there?
-
Hi!
First time poster.Im trying to set up and learn ipv6 in our small server hall.
I'd like to have routeable static addresses on our servers behind the firewall.tldr: cant ping from lan to outside.
For now they can be slaac or manual, (sidenote: slaac seems nice automagic, but what if I migrate a service to a new machine, then other machines have to be repointed to new address? Maybe static is better then?)
Simple setup: ISP - pfsense with 4 NIC - switch - servers
ISP has given me a /48.
They said no gateway needed since it is normally advertised via RA to all devices connected on their CPA.
If gateway was an absolute must then use the link local address.Defenition: PF - prefix, in my case something like 2001:xxxx:xxxx
I was about to set wan statically as PF::1 when I noticed something answering ping there, so I chose PF::2/64.
I suspect that is my gateway. Traceroute6 shows that as my routers first hop. No gateway set.I then set LAN address as PF:abba::1/64, no gateway since LAN.
- From my pfsense box i can ping anything, servers on LAN on both link local and global, my presumed gateway and google.
- From my servers I
cancould ping both lan and wan addresses of router, and other servers on LAN, but nothing else. True for both pinging from link local address and global slaac address. Edit: I changed something, I suspect with RA settings, I can no longer ping WAN address from servers on LAN. I can only ping router LAN address. - From outside I can ping router WAN.
- If i try to ping router LAN addr from outside I get Destination unreachable: unknown code 6. This reply comes from a server that only shares the first 16 bits of prefix.
Firewall rules added: WAN: IPv6 ICMP any, * * * * * none. Same rule added on LAN tab.
RA tab, WAN: router only
Subnets advertised: PF::/64, PF:abba::/64
Lan: Unmanaged. Subnets advertised: PF:abba::/64, PF::/64I cannot see any blocks happening in the firewall logs, nor passes related to these pings.
Routes on pfsense show:
default fe80::1%bce0 PF:: (some addr I dont recognize with same prefix:0:slaac. Does not show up in traceroute, lets call this addrA) PF::/64 link#1 PF::2 link#1 addrA link#1 PF:abba:: PF::abba::1 PF:abba::/64 link#2 PF:abba::1 link#2
along with link local entries
Routes in servers:
One of my servers, Cent OS 8.1, show this in response to ip -6 route list::1 dev lo proto kernel metric 256 pref medium PF::/64 dev eno1 proto ra metric 100 pref medium PF:abba::/64 dev eno1 proto ra metric 100 pref medium fe80::/64 dev eno1 proto kernel metric 100 pref medium default via fe80::1:1 dev eno1 proto ra metric 100 pref medium
Another server, ubuntu 20, show:
PF::/64 dev eno1 proto ra metric 1024 pref medium PF:abba::/64 dev eno1 proto kernel metric 256 pref medium PF:abba::/64 dev eno1 proto ra metric 1024 pref medium fe80::/64 dev eno1 proto kernel metric 256 pref medium default via fe80::1:1 dev eno1 proto ra metric 1024 mtu 1500 pref medium
It seems they know that the way out is through my pfsense box, right?
I was about to try and set some static routes but was discouraged by the docs stating:
Static routes are used when hosts or networks are reachable through a router other than the default gateway. pfSense knows about the networks directly attached to it, and reaches all other networks as directed by its routing table - https://docs.netgate.com/pfsense/en/latest/book/routing/static-routes.htmlWhat am I missing/what have I misunderstood?
Kind regards
Robert -
@RobertTheSwede said in Possible routing issue, almost there?:
ISP has given me a /48.
They said no gateway needed since it is normally advertised via RA to all devices connected on their CPA.
If gateway was an absolute must then use the link local address.WTF???? They expect you to use a flat /48???? You can't do that. You need a router, such as pfSense to split that into /64s. LANs are always /64.
Link local addresses are often used for routing in IPv6. That's certainly the case with my ISP, though my WAN interface also gets an address that is not used for routing and has nothing to do with my prefix. My ISP uses DHCPv6-PD to provide my /56 prefix.
-
Yeah!
This is a snippet of the info they sent me:
Assigned IP-addresses: PF::/48 # PF is same prefix as in OP above. Default gateway *
But shouldn't I be able to split that /48 into several /64 networks myself?
Thats why I set up the router to have a PF::2/64.I've played with using PF::1/48 or PF::2/48 on the WAN address, but I don't know how to create subnets after that since pfsense then says:
IPv6 address PF:abba::1/64 is being used by or overlaps with: WAN (PF::2/48) The gateway address PF::2 does not lie within one of the chosen interface's subnets.
So I am a bit stumped on how to proceed from here.
-
You need them to provide DHCPv6-PD or specify manual config info. As mentioned, you can use the link local address for their gateway, assuming they're routing that /48 to you and not just giving it to you as is. If they expect anyone to use a /48, without routing they don't know what they're talking about. As an experiment, connect pfSense to the modem (I assume it's in bridge mode) and use Packet Capture to filter on DHCPv6 and see what turns up. You can also do that for ICMP6. Post the capture files here here.
-
Ah, so it isn't me who is totally lost here!
I will start with contacting my ISP, they usually have great support, I just don't want to waste their time. This way the times I do have to call them they have time to spend.If that comes up negative, I will try the packet dump.
The pfsense machine is actually right now directly connected to their modem!It is what handles our setup over IPv4 now, which is all routed through the same modem.
Thank you so much for the help, I will report back!
-
Do you have an IPv6 address on the WAN interface?
-
I called the support from my ISP, and the issue is solved!
Turns out that unless requested they throw away all traffic not destined for the first subnet/64.
They also told me that it is still very uncommon for customers to actually want all of this tech and no standard procedures exist (in their company, yet) for how to route more than the first subnet/64.
He then asked if I wanted the entire 48 routed to my router in the first subnet, I said yes and now it seems it all works as expected!
My subnets can be opened or blocked in pfsense and devices reach wan from lan and outside clients can reach lan clients if I allow it in pfsense.Happy networking!
-
WOW, that is dumb. I have a /56 and have used prefix ID ff for my VPN without issue. Works fine. Their policy should be everything in that /48 prefix should be forwarded to the customer. Also, it's extremely unlikely there would be any traffic for an unused prefix, as there is nothing to trigger it.