Bug ? 2.1 ipv6 policy based routing not working ?

Robstar

I currently have two IPV6 tunnels working:

One from Sixxs
One from he.net

I also have routed subnets from both. Both work from the GW.

(ips/domains changed)


Password:
*** Welcome to pfSense 2.1-BETA0-pfSense (amd64) on pfsense1 ***

 ISP_WAN4 (wan) -> em1        -> 173.1.1.2/28	NONE/NONE 
 ISP_LAN4 (lan) -> em0        -> 192.168.75.254/24	2001:eee:ffff::1/48 
 HE_WAN (opt1)   -> gif0       -> NONE/NONE	2001:aaa:bbbb:3c5::2/64 
 SIXXS (opt2)    -> gif1       -> NONE/NONE	2001:cccc:d:56f::2/64 

 0) Logout (SSH only)                  8) Shell
 1) Assign Interfaces                  9) pfTop
 2) Set interface(s) IP address       10) Filter Logs
 3) Reset webConfigurator password    11) Restart webConfigurator
 4) Reset to factory defaults         12) pfSense Developer Shell
 5) Reboot system                     13) Upgrade from console
 6) Halt system                       14) Disable Secure Shell (sshd)
 7) Ping host                         15) Restore recent configuration

Enter an option: 8

[2.1-BETA0][admin@pfsense1.example.com]/root(1): uname -a
FreeBSD pfsense1.example.com 8.3-RELEASE-p2 FreeBSD 8.3-RELEASE-p2 #1: Tue Jun  5 06:16:07 EDT 2012     root@FreeBSD_8.3_pfSense_2.1.snaps.pfsense.org:/usr/obj./usr/pfSensesrc/src/sys/pfSense_SMP.8  amd64
[2.1-BETA0][admin@pfsense1.example.com]/root(2): ifconfig gif0
gif0: flags=8051 <up,pointopoint,running,multicast>metric 0 mtu 1280
	tunnel inet 173.1.1.2 –> 209.51.181.2
	inet6 fe80::989b:ae60:7a84:7a9d%gif0 prefixlen 64 scopeid 0x9 
	inet6 2001:aaa:bbbb:3c5::2 prefixlen 64 
	nd6 options=3 <performnud,accept_rtadv>options=1 <accept_rev_ethip_ver>[2.1-BETA0][admin@pfsense1.example.com]/root(3): ifconfig gif1
gif1: flags=8051 <up,pointopoint,running,multicast>metric 0 mtu 1280
	tunnel inet 173.1.1.2 –> 216.14.98.22
	inet6 fe80::989b:ae60:7a84:7a9d%gif1 prefixlen 64 scopeid 0xa 
	inet6 2001:cccc:d:56f::2 prefixlen 64 
	nd6 options=3 <performnud,accept_rtadv>options=1 <accept_rev_ethip_ver>[2.1-BETA0][admin@pfsense1.example.com]/root(4): ping6 2001:cccc:d:56f::1
PING6(56=40+8+8 bytes) 2001:cccc:d:56f::2 –> 2001:cccc:d:56f::1
16 bytes from 2001:cccc:d:56f::1, icmp_seq=0 hlim=64 time=17.766 ms
16 bytes from 2001:cccc:d:56f::1, icmp_seq=1 hlim=64 time=13.352 ms
16 bytes from 2001:cccc:d:56f::1, icmp_seq=2 hlim=64 time=12.363 ms
^C
--- 2001:cccc:d:56f::1 ping6 statistics ---
3 packets transmitted, 3 packets received, 0.0% packet loss
round-trip min/avg/max/std-dev = 12.363/14.494/17.766/2.349 ms

[2.1-BETA0][admin@pfsense1.example.com]/root(5): ping6 2001:aaa:bbbb:3c5::1
PING6(56=40+8+8 bytes) 2001:aaa:bbbb:3c5::2 –> 2001:aaa:bbbb:3c5::1
16 bytes from 2001:aaa:bbbb:3c5::1, icmp_seq=0 hlim=64 time=39.749 ms
16 bytes from 2001:aaa:bbbb:3c5::1, icmp_seq=1 hlim=64 time=33.304 ms
16 bytes from 2001:aaa:bbbb:3c5::1, icmp_seq=2 hlim=64 time=36.184 ms
16 bytes from 2001:aaa:bbbb:3c5::1, icmp_seq=3 hlim=64 time=34.149 ms
^C
--- 2001:aaa:bbbb:3c5::1 ping6 statistics ---
4 packets transmitted, 4 packets received, 0.0% packet loss
round-trip min/avg/max/std-dev = 33.304/35.846/39.749/2.484 ms
[2.1-BETA0][admin@pfsense1.hendelman.net]/root(6): ifconfig em0
em0: flags=8843 <up,broadcast,running,simplex,multicast>metric 0 mtu 1500
	options=2098 <vlan_mtu,vlan_hwtagging,vlan_hwcsum,wol_magic>ether 16:12:f4:5a:db:75
	inet 192.168.75.254 netmask 0xffffff00 broadcast 192.168.75.255
	inet6 fe80::1412:f4ff:fe5a:db75%em0 prefixlen 64 scopeid 0x2 
	inet6 2001:a:b::1 prefixlen 48 
	nd6 options=1 <performnud>media: Ethernet autoselect (1000baseT <full-duplex>)
	status: active
[2.1-BETA0][admin@pfsense1.hendelman.net]/root(7): netstat -rn | grep -A2 Internet6
Internet6:
Destination                       Gateway                       Flags      Netif Expire
default                            2001:cccc:d:56f::1            UGS        gif1</full-duplex></performnud></vlan_mtu,vlan_hwtagging,vlan_hwcsum,wol_magic></up,broadcast,running,simplex,multicast></accept_rev_ethip_ver></performnud,accept_rtadv></up,pointopoint,running,multicast></accept_rev_ethip_ver></performnud,accept_rtadv></up,pointopoint,running,multicast>

So my tunnels work.

I have a client on 2001b::2

The client can ping 2001b::1 (lan ipv6 interface)
The client can ping 2001:aaa:bbbb:3c5::2 (wan ipv6 tunnel, my side)
The client can ping 2001:aaa:bbbb:3c5::1 (wan ipv6 tunnel ISP side).


rob-desktop rob2 # ifconfig br0
br0       Link encap:Ethernet  HWaddr 00:07:e9:16:b2:cf  
          inet addr:192.168.75.35  Bcast:192.168.75.255  Mask:255.255.255.0
          inet6 addr: 2001:a:b:999/48 Scope:Global
          inet6 addr: fe80::207:e9ff:fe16:b2cf/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:9304068 errors:0 dropped:40 overruns:0 frame:0
          TX packets:8382030 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:47278998769 (44.0 GiB)  TX bytes:1000987164 (954.6 MiB)

rob-desktop rob2 # ping6 2001:a:b:1
PING 2001:a:b:1(2001:a:b:1) 56 data bytes
64 bytes from 2001:a:b:1: icmp_seq=1 ttl=64 time=0.236 ms
64 bytes from 2001:a:b:1: icmp_seq=2 ttl=64 time=0.305 ms
64 bytes from 2001:a:b:1: icmp_seq=3 ttl=64 time=0.282 ms
^C
–- 2001:a:b:1 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 1998ms
rtt min/avg/max/mdev = 0.236/0.274/0.305/0.031 ms
rob-desktop rob2 # ping6 2001:aaa:bbbb:3c5::2
PING 2001:aaa:bbbb:3c5::2(2001:aaa:bbbb:3c5::2) 56 data bytes
64 bytes from 2001:aaa:bbbb:3c5::2: icmp_seq=1 ttl=64 time=0.259 ms
64 bytes from 2001:aaa:bbbb:3c5::2: icmp_seq=2 ttl=64 time=0.281 ms
64 bytes from 2001:aaa:bbbb:3c5::2: icmp_seq=3 ttl=64 time=0.298 ms
^C
--- 2001:aaa:bbbb:3c5::2 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 1998ms
rtt min/avg/max/mdev = 0.259/0.279/0.298/0.021 ms
rob-desktop rob2 # ping6 2001:aaa:bbbb:3c5::1
PING 2001:aaa:bbbb:3c5::1(2001:aaa:bbbb:3c5::1) 56 data bytes
64 bytes from 2001:aaa:bbbb:3c5::1: icmp_seq=1 ttl=63 time=40.2 ms
64 bytes from 2001:aaa:bbbb:3c5::1: icmp_seq=2 ttl=63 time=33.9 ms
64 bytes from 2001:aaa:bbbb:3c5::1: icmp_seq=3 ttl=63 time=48.8 ms
^C
--- 2001:aaa:bbbb:3c5::1 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2001ms
rtt min/avg/max/mdev = 33.949/41.005/48.841/6.106 ms

I've added (ipv6 only) rules to enable these 3.

I then add a 4th rule to allow pinging anywhere & specify the advanced gateway option as 2001:aaa:bbbb:3c5::1.

I can't ping anything from the client…


rob-desktop rob2 # ping6  2607:f8b0:4004:800::1011
PING 2607:f8b0:4004:800::1011(2607:f8b0:4004:800::1011) 56 data bytes
^C
–- 2607:f8b0:4004:800::1011 ping statistics ---
3 packets transmitted, 0 received, 100% packet loss, time 2000ms

My policy based rule to route over the non-default gw:


pass in quick on em0 route-to (opt1 2001:aaa:bbbb:3c5::1) inet6 proto ipv6-icmp from 2001:a:b::999 to any keep state label "USER_RULE: ping ipv6 world"

Anyone have any idea why this isn't working? Am I missing something obvious?

databeestje

have you setup NAT66 (NPt) for the correct WAN?
If you are pursuing multiwan with IPv6 you need this document.
http://doc.pfsense.org/index.php/Multi-WAN_for_IPv6

You will have to do so manually, these will not automatically be setup like the v4 automatic outbound NAT.

If you meant here that you want policy routing for traffic originating from prefix-a to only go out gateway-a and prefix-b to only go out gateway-b then that is a valid case that I would need to look into.

Robstar

Yes, there is no NAT involved.

tunnelbroker.net (Hurricane Electric) routes a /48 for me. I need the pfsense box to throw that out of the HE gateway
sixxs.net routes a /64 for me. I need pfsense to throw that traffic out of the sixxs gw (default gw, so it should work).

Thanks for looking into this.

Rob

Robstar

Do you need any more information from me to investigate this?