Cannot NAT trough OPT1 interface on multiwan



  • i've setup pfsense 2.2 and found that i cant NAT my webserver via OPT1 interfaces.

    my configuration are :

    WAN –>202.14.x.x
    LAN --> 172.20.200.x/24
    OPT1 -->114.31.x.x/24

    when i NAT (port forward) one of my web server from LAN to WAN interfaces, i can access it from the internet eg:

    172.20.200.28 Port Forward HTTP 80 NAT to 202.14.13.5 --> it works !

    but when i tried using the other WAN interfaces (OPT1)

    172.20.200.28 Port Forward HTTP 80 NAT to 114.31.242.50 --> cannot access it from outside

    i started the packet capture and its show :

    12:51:11.337184 IP 120.161.1.109.20241 > 172.20.200.28.80: tcp 0
    12:51:11.337872 IP 172.20.200.28.80 > 120.161.1.109.20241: tcp 0
    12:51:11.337904 IP 172.20.200.1 > 172.20.200.28: ICMP host 120.161.1.109 unreachable, length 72
    12:51:12.363936 IP 120.161.1.109.20241 > 172.20.200.28.80: tcp 0
    12:51:13.352929 IP 120.161.1.109.20241 > 172.20.200.28.80: tcp 0
    12:51:13.883907 IP 172.20.200.28.80 > 120.161.1.109.20241: tcp 0
    12:51:13.883942 IP 172.20.200.1 > 172.20.200.28: ICMP host 120.161.1.109 unreachable, length 72
    12:51:14.372410 IP 120.161.1.109.20241 > 172.20.200.28.80: tcp 0
    12:51:15.370767 IP 120.161.1.109.20241 > 172.20.200.28.80: tcp 0
    12:51:16.384578 IP 120.161.1.109.20241 > 172.20.200.28.80: tcp 0
    12:51:18.400488 IP 120.161.1.109.20241 > 172.20.200.28.80: tcp 0
    12:51:20.446202 IP 172.20.200.28.80 > 120.161.1.109.20241: tcp 0
    12:51:20.446237 IP 172.20.200.1 > 172.20.200.28: ICMP host 120.161.1.109 unreachable, length 72
    12:51:22.415551 IP 120.161.1.109.20241 > 172.20.200.28.80: tcp 0

    please help.



  • Oups… I have just posted the same. It's a bug. If you set default gateway yout opt it will work but then NAT on WAN will not work.  :( Nat works only on default gw interface. I think this bug also existed on 2.1 alpha versions.



  • Hooray I'm not the only one.



  • @apant:

    Oups… I have just posted the same. It's a bug. If you set default gateway yout opt it will work but then NAT on WAN will not work.  :( Nat works only on default gw interface. I think this bug also existed on 2.1 alpha versions.

    Not strictly true as you can setup manual NAT rules on the interface that work just fine.
    Not sure I'd call it a bug more a feature as most people do not require NAT on more than one interface.



  • What do you mean manual nat work just fine ?



  • I'm not alone ;D



  • Still not working.
    How can manually adjust?



  • perhaps this is why i can not access my modem any more? (using these instructions from the wiki)
    https://doc.pfsense.org/index.php/Accessing_modem_from_inside_firewall

    If so, is there a simple alternative?



  • im rolling back to 2.1.5, the reason why i do NAT on 2 interface because i have 2 different  WAN, which both have an public ip, and i use them to publish my website on both wan interface.

    is there any update about this problem? is it solve yet in 2.2?



  • I have 3 wan outputs with public IP. NAT does not work, inbound one WAN and exits on the second - the default GW. I went rolling back to 2.1.5 also.



  • Rebel Alliance Developer Netgate

    Anyone having this problem, if you could post your ruleset (/tmp/rules.debug) it may help. Feel free to sanitize the IPs but don't erase them (change the numbers to something else consistent).

    Looks like there is something with reply-to and/or route-to that isn't quite right. There is at least one similar ticket – https://redmine.pfsense.org/issues/3760 -- which may need commented on and reopened if it is still broken on current snapshots.



  • set optimization normal
    set timeout { adaptive.start 0, adaptive.end 0 }
    set limit states 60000
    set limit src-nodes 60000

    #System aliases

    loopback = "{ lo0 }"
    WAN = "{ pppoe0 }"
    LAN = "{ hn1 }"
    OPT1 = "{ pppoe1 }"

    #SSH Lockout Table
    table <sshlockout>persist
    table <webconfiguratorlockout>persist
    #Snort tables
    table <snort2c>table <virusprot>table <bogons>persist file "/etc/bogons"
    table <negate_networks># User Aliases

    Gateways

    GWOPT1_PPPOE = " route-to ( pppoe1 222.92.225.13 ) "
    GWWAN_PPPOE = " route-to ( pppoe0 117.83.20.1 ) "

    set loginterface hn1

    set skip on pfsync0

    scrub on $WAN all    fragment reassemble
    scrub on $LAN all    fragment reassemble
    scrub on $OPT1 all    fragment reassemble

    no nat proto carp
    no rdr proto carp
    nat-anchor "natearly/"
    nat-anchor "natrules/
    "

    Outbound NAT rules (automatic)

    Subnets to NAT

    tonatsubnets = "{ 127.0.0.0/8 10.10.10.0/24 }"
    nat on $WAN  from $tonatsubnets to any port 500 -> 117.83.21.147/32  static-port
    nat on $WAN  from $tonatsubnets to any -> 117.83.21.147/32 port 1024:65535 
    nat on $OPT1  from $tonatsubnets to any port 500 -> 49.84.243.228/32  static-port
    nat on $OPT1  from $tonatsubnets to any -> 49.84.243.228/32 port 1024:65535

    Load balancing anchor

    rdr-anchor "relayd/*"

    TFTP proxy

    rdr-anchor "tftp-proxy/*"

    NAT Inbound Redirects

    rdr on pppoe0 proto tcp from any to 117.83.21.147 port 8000 -> 10.10.10.10

    Reflection redirects

    rdr on hn1 proto tcp from any to 117.83.21.147 port 8000 tag PFREFLECT -> 127.0.0.1 port 19000

    rdr on pppoe1 proto tcp from any to 49.84.243.228 port 8000 -> 10.10.10.10

    Reflection redirects

    rdr on hn1 proto tcp from any to 49.84.243.228 port 8000 tag PFREFLECT -> 127.0.0.1 port 19001

    UPnPd rdr anchor

    rdr-anchor "miniupnpd"

    anchor "relayd/"
    anchor "openvpn/
    "
    anchor "ipsec/*"

    Block all IPv6

    block in log quick inet6 all tracker 1000000001 label "Block all IPv6"
    block out log quick inet6 all tracker 1000000002 label "Block all IPv6"
    #–-------------------------------------------------------------------------

    default deny rules

    #---------------------------------------------------------------------------
    block in log inet all tracker 1000000101 label "Default deny rule IPv4"
    block out log inet all tracker 1000000102 label "Default deny rule IPv4"
    block in log inet6 all tracker 1000000103 label "Default deny rule IPv6"
    block out log inet6 all tracker 1000000104 label "Default deny rule IPv6"

    IPv6 ICMP is not auxilary, it is required for operation

    See man icmp6(4)

    1    unreach        Destination unreachable

    2    toobig          Packet too big

    128  echoreq        Echo service request

    129  echorep        Echo service reply

    133  routersol      Router solicitation

    134  routeradv      Router advertisement

    135  neighbrsol      Neighbor solicitation

    136  neighbradv      Neighbor advertisement

    pass  quick inet6 proto ipv6-icmp from any to any icmp6-type {1,2,135,136} tracker 1000000105 keep state

    Allow only bare essential icmpv6 packets (NS, NA, and RA, echoreq, echorep)

    pass out  quick inet6 proto ipv6-icmp from fe80::/10 to fe80::/10 icmp6-type {129,133,134,135,136} tracker 1000000106 keep state
    pass out  quick inet6 proto ipv6-icmp from fe80::/10 to ff02::/16 icmp6-type {129,133,134,135,136} tracker 1000000107 keep state
    pass in  quick inet6 proto ipv6-icmp from fe80::/10 to fe80::/10 icmp6-type {128,133,134,135,136} tracker 1000000108 keep state
    pass in  quick inet6 proto ipv6-icmp from ff02::/16 to fe80::/10 icmp6-type {128,133,134,135,136} tracker 1000000109 keep state
    pass in  quick inet6 proto ipv6-icmp from fe80::/10 to ff02::/16 icmp6-type {128,133,134,135,136} tracker 1000000110 keep state

    We use the mighty pf, we cannot be fooled.

    block log quick inet proto { tcp, udp } from any port = 0 to any tracker 1000000111
    block log quick inet proto { tcp, udp } from any to any port = 0 tracker 1000000112
    block log quick inet6 proto { tcp, udp } from any port = 0 to any tracker 1000000113
    block log quick inet6 proto { tcp, udp } from any to any port = 0 tracker 1000000114

    Snort package

    block log quick from <snort2c>to any tracker 1000000115 label "Block snort2c hosts"
    block log quick from any to <snort2c>tracker 1000000116 label "Block snort2c hosts"

    SSH lockout

    block in log quick proto tcp from <sshlockout>to (self) port 22 tracker 1000000301 label "sshlockout"

    webConfigurator lockout

    block in log quick proto tcp from <webconfiguratorlockout>to (self) port 443 tracker 1000000351 label "webConfiguratorlockout"
    block in log quick from <virusprot>to any tracker 1000000400 label "virusprot overload table"
    antispoof log for $WAN tracker 1000001570
    antispoof log for $LAN tracker 1000002620

    allow access to DHCPv6 server on LAN

    We need inet6 icmp for stateless autoconfig and dhcpv6

    pass  quick on $LAN inet6 proto udp from fe80::/10 to fe80::/10 port = 546 tracker 1000002651 label "allow access to DHCPv6 server"
    pass  quick on $LAN inet6 proto udp from fe80::/10 to ff02::/16 port = 546 tracker 1000002652 label "allow access to DHCPv6 server"
    pass  quick on $LAN inet6 proto udp from fe80::/10 to ff02::/16 port = 547 tracker 1000002653 label "allow access to DHCPv6 server"
    pass  quick on $LAN inet6 proto udp from ff02::/16 to fe80::/10 port = 547 tracker 1000002654 label "allow access to DHCPv6 server"
    antispoof log for $OPT1 tracker 1000003670

    loopback

    pass in  on $loopback inet all tracker 1000003711 label "pass IPv4 loopback"
    pass out  on $loopback inet all tracker 1000003712 label "pass IPv4 loopback"
    pass in  on $loopback inet6 all tracker 1000003713 label "pass IPv6 loopback"
    pass out  on $loopback inet6 all tracker 1000003714 label "pass IPv6 loopback"

    let out anything from the firewall host itself and decrypted IPsec traffic

    pass out  inet all keep state allow-opts tracker 1000003715 label "let out anything IPv4 from firewall host itself"
    pass out  inet6 all keep state allow-opts tracker 1000003716 label "let out anything IPv6 from firewall host itself"
    pass out  route-to ( pppoe0 117.83.20.1 ) from 117.83.21.147 to !117.83.21.147/32 tracker 1000003811 keep state allow-opts label "let out anything from firewall host itself"
    pass out  route-to ( pppoe1 222.92.225.13 ) from 49.84.243.228 to !49.84.243.228/32 tracker 1000003812 keep state allow-opts label "let out anything from firewall host itself"

    make sure the user cannot lock himself out of the webConfigurator or SSH

    pass in  quick on hn1 proto tcp from any to (hn1) port { 443 80 } tracker 1000004121 keep state label "anti-lockout rule"

    NAT Reflection rules

    pass in  inet tagged PFREFLECT tracker 1000004141 keep state label "NAT REFLECT: Allow traffic to localhost"

    User-defined rules follow

    anchor "userrules/*"
    pass  in  quick  on $WAN reply-to ( pppoe0 117.83.20.1 )  proto tcp  from any to 10.10.10.10 port 8000 flags S/SA keep state  label "USER_RULE: NAT "
    pass  in  quick  on $WAN reply-to ( pppoe0 117.83.20.1 ) inet proto tcp  from any to 117.83.21.147 port 443 tracker 1410767372 flags S/SA keep state  label "USER_RULE"
    pass  in  quick  on $LAN inet proto tcp  from 10.10.10.10  to <negate_networks>tracker 1410768766 flags S/SA keep state  label "NEGATE_ROUTE: Negate policy routing for destination"
    pass  in  quick  on $LAN  $GWWAN_PPPOE inet proto tcp  from 10.10.10.10 to any tracker 1410768766 flags S/SA keep state  label "USER_RULE: Default allow LAN to any rule"
    pass  in  quick  on $LAN inet proto tcp  from 10.10.10.0/24  to <negate_networks>port 443 tracker 1410768578 flags S/SA keep state  label "NEGATE_ROUTE: Negate policy routing for destination"
    pass  in  quick  on $LAN  $GWWAN_PPPOE inet proto tcp  from 10.10.10.0/24 to any port 443 tracker 1410768578 flags S/SA keep state  label "USER_RULE: Default allow LAN to any rule"
    pass  in  quick  on $LAN inet from 10.10.10.0/24 to any tracker 0100000101 keep state  label "USER_RULE: Default allow LAN to any rule"

    at the break! label "USER_RULE: Default allow LAN IPv6 to any rule"

    pass  in  quick  on $OPT1  $GWOPT1_PPPOE inet proto tcp  from any to 49.84.243.228 port 443 tracker 1410767990 flags S/SA keep state  label "USER_RULE"
    pass  in  quick  on $OPT1  $GWOPT1_PPPOE inet proto tcp  from any to 10.10.10.10 port 8000 tracker 1411286824 flags S/SA keep state  label "USER_RULE: NAT "
    pass  in  quick  on $OPT1 reply-to ( pppoe1 222.92.225.13 ) inet proto icmp  from any to 49.84.243.228 tracker 1410843035 keep state  label "USER_RULE"

    VPN Rules

    anchor "tftp-proxy/*"</negate_networks></negate_networks></virusprot></webconfiguratorlockout></sshlockout></snort2c></snort2c></negate_networks></bogons></virusprot></snort2c></webconfiguratorlockout></sshlockout>



  • i cannot give the debug because I've already revert back to 2.1.5, and since pf is my only fw, and is being use in production site, so i hope someone kindly test it in 2.2 latest snapshot to give the answer is this problem still exist or not.

    thank you.



  • @villavai:

    i cannot give the debug because I've already revert back to 2.1.5, and since pf is my only fw, and is being use in production site, so i hope someone kindly test it in 2.2 latest snapshot to give the answer is this problem still exist or not.

    thank you.

    Unfortunately, it's still there.



  • Hi everybody,

    this is not only a NAT problem.
    It seems that the routing in general does not work properly with multiwan. (in my case WAN_KD=cable router, WAN_TO=PPPoE modem)
    I'm running the Oct 03 10:14:50 CDT 2014 BETA, but also had the same problem with a previous version (Oct 01).

    I can't even access the pfS Webgui from the WAN_TO side, although in the firewall log the request is shown as PASS.

    My config:
    set optimization normal
    set timeout { adaptive.start 0, adaptive.end 0 }
    set limit states 198000
    set limit src-nodes 198000

    #System aliases

    loopback = "{ lo0 }"
    WAN_KD = "{ re0 }"
    LAN = "{ re2 }"
    WAN_TO = "{ pppoe0 }"
    OpenVPN = "{ openvpn }"

    #SSH Lockout Table
    table <sshlockout>persist
    table <webconfiguratorlockout>persist
    #Snort tables
    table <snort2c>table <virusprot>table <bogons>persist file "/etc/bogons"
    table <vpn_networks>{ 192.168.254.0/24 }
    table <negate_networks>{ 192.168.254.0/24 }

    User Aliases

    admin_ports = "{  456  4562 }"
    ovpn_ports = "{  1194  1199 }"
    table <ovpn_remote_networks>{  192.168.254.0/24  192.168.222.0/24 }
    OVPN_Remote_Networks = "<ovpn_remote_networks>"
    table <rfc1918_private_networks>{  10.0.0.0/8  172.16.0.0/12  192.168.0.0/16 }
    RFC1918_Private_Networks = "<rfc1918_private_networks>"
    table <server>{  172.16.100.100 }
    Server = "<server>"

    Gateways

    GWWAN_KD_GW = " route-to ( re0 192.168.0.1 ) "
    GWWAN_TO_PPPOE = " route-to ( pppoe0 XXX.WAN.TO.GW ) "
    GWGWG_LAN = "  route-to { ( pppoe0 XXX.WAN.TO.GW )  }  "

    set loginterface re2

    set skip on pfsync0

    scrub on $WAN_KD all    fragment reassemble
    scrub on $LAN all    fragment reassemble
    scrub on $WAN_TO all    fragment reassemble

    no nat proto carp
    no rdr proto carp
    nat-anchor "natearly/"
    nat-anchor "natrules/
    "

    Outbound NAT rules (automatic)

    Subnets to NAT

    tonatsubnets = "{ 127.0.0.0/8 172.16.0.0/16 192.168.254.0/24 }"
    nat on $WAN_KD  from $tonatsubnets to any port 500 -> 192.168.0.2/32  static-port
    nat on $WAN_KD  from $tonatsubnets to any -> 192.168.0.2/32 port 1024:65535 
    nat on $WAN_TO  from $tonatsubnets to any port 500 -> XXX.WAN.TO.IP/32  static-port
    nat on $WAN_TO  from $tonatsubnets to any -> XXX.WAN.TO.IP/32 port 1024:65535

    Load balancing anchor

    rdr-anchor "relayd/*"

    TFTP proxy

    rdr-anchor "tftp-proxy/*"

    NAT Inbound Redirects

    rdr on re0 proto tcp from any to 192.168.0.2 port 443 -> $Server
    rdr on re0 proto tcp from any to 192.168.0.2 port 8182 -> $Server port 5900

    UPnPd rdr anchor

    rdr-anchor "miniupnpd"

    anchor "relayd/"
    anchor "openvpn/
    "
    anchor "ipsec/*"

    Block all IPv6

    block in log quick inet6 all tracker 1000000001 label "Block all IPv6"
    block out log quick inet6 all tracker 1000000002 label "Block all IPv6"
    #–-------------------------------------------------------------------------

    default deny rules

    #---------------------------------------------------------------------------
    block in log inet all tracker 1000000101 label "Default deny rule IPv4"
    block out log inet all tracker 1000000102 label "Default deny rule IPv4"
    block in log inet6 all tracker 1000000103 label "Default deny rule IPv6"
    block out log inet6 all tracker 1000000104 label "Default deny rule IPv6"

    IPv6 ICMP is not auxilary, it is required for operation

    See man icmp6(4)

    1    unreach        Destination unreachable

    2    toobig          Packet too big

    128  echoreq        Echo service request

    129  echorep        Echo service reply

    133  routersol      Router solicitation

    134  routeradv      Router advertisement

    135  neighbrsol      Neighbor solicitation

    136  neighbradv      Neighbor advertisement

    pass  quick inet6 proto ipv6-icmp from any to any icmp6-type {1,2,135,136} tracker 1000000105 keep state

    Allow only bare essential icmpv6 packets (NS, NA, and RA, echoreq, echorep)

    pass out  quick inet6 proto ipv6-icmp from fe80::/10 to fe80::/10 icmp6-type {129,133,134,135,136} tracker 1000000106 keep state
    pass out  quick inet6 proto ipv6-icmp from fe80::/10 to ff02::/16 icmp6-type {129,133,134,135,136} tracker 1000000107 keep state
    pass in  quick inet6 proto ipv6-icmp from fe80::/10 to fe80::/10 icmp6-type {128,133,134,135,136} tracker 1000000108 keep state
    pass in  quick inet6 proto ipv6-icmp from ff02::/16 to fe80::/10 icmp6-type {128,133,134,135,136} tracker 1000000109 keep state
    pass in  quick inet6 proto ipv6-icmp from fe80::/10 to ff02::/16 icmp6-type {128,133,134,135,136} tracker 1000000110 keep state

    We use the mighty pf, we cannot be fooled.

    block log quick inet proto { tcp, udp } from any port = 0 to any tracker 1000000111
    block log quick inet proto { tcp, udp } from any to any port = 0 tracker 1000000112
    block log quick inet6 proto { tcp, udp } from any port = 0 to any tracker 1000000113
    block log quick inet6 proto { tcp, udp } from any to any port = 0 tracker 1000000114

    Snort package

    block log quick from <snort2c>to any tracker 1000000115 label "Block snort2c hosts"
    block log quick from any to <snort2c>tracker 1000000116 label "Block snort2c hosts"

    SSH lockout

    block in log quick proto tcp from <sshlockout>to (self) port 4562 tracker 1000000301 label "sshlockout"

    webConfigurator lockout

    block in log quick proto tcp from <webconfiguratorlockout>to (self) port 456 tracker 1000000351 label "webConfiguratorlockout"
    block in log quick from <virusprot>to any tracker 1000000400 label "virusprot overload table"

    block bogon networks (IPv4)

    http://www.cymru.com/Documents/bogon-bn-nonagg.txt

    block in log quick on $WAN_KD from <bogons>to any tracker 1000001551 label "block bogon IPv4 networks from WAN_KD"
    antispoof log for $WAN_KD tracker 1000001570
    antispoof log for $LAN tracker 1000002620
    antispoof log for $WAN_TO tracker 1000003670

    loopback

    pass in  on $loopback inet all tracker 1000004761 label "pass IPv4 loopback"
    pass out  on $loopback inet all tracker 1000004762 label "pass IPv4 loopback"
    pass in  on $loopback inet6 all tracker 1000004763 label "pass IPv6 loopback"
    pass out  on $loopback inet6 all tracker 1000004764 label "pass IPv6 loopback"

    let out anything from the firewall host itself and decrypted IPsec traffic

    pass out  inet all keep state allow-opts tracker 1000004765 label "let out anything IPv4 from firewall host itself"
    pass out  inet6 all keep state allow-opts tracker 1000004766 label "let out anything IPv6 from firewall host itself"
    pass out  route-to ( re0 192.168.0.1 ) from 192.168.0.2 to !192.168.0.0/24 tracker 1000004861 keep state allow-opts label "let out anything from firewall host itself"
    pass out  route-to ( pppoe0 XXX.WAN.TO.GW ) from XXX.WAN.TO.IP to !XXX.WAN.TO.IP/32 tracker 1000004862 keep state allow-opts label "let out anything from firewall host itself"

    User-defined rules follow

    anchor "userrules/*"
    pass  in  quick  on $OpenVPN inet from $OVPN_Remote_Networks to 172.16.0.0/16 tracker 1412417430 keep state  label "USER_RULE"
    pass  in  quick  on $WAN_KD reply-to ( re0 192.168.0.1 ) inet proto tcp  from any to 192.168.0.2 port $admin_ports tracker 1412198572 flags S/SA keep state  label "USER_RULE"
    pass  in  quick  on $WAN_KD reply-to ( re0 192.168.0.1 ) inet proto udp  from any to 192.168.0.2 port $ovpn_ports tracker 1412415560 keep state  label "USER_RULE"
    pass  in  quick  on $WAN_KD reply-to ( re0 192.168.0.1 )  proto tcp  from any to $Server port 443 flags S/SA keep state  label "USER_RULE: NAT "
    pass  in  quick  on $WAN_KD reply-to ( re0 192.168.0.1 )  proto tcp  from any to $Server port 5900 flags S/SA keep state  label "USER_RULE: NAT "
    pass  in  quick  on $LAN inet proto tcp  from any to 172.16.100.1 port $admin_ports tracker 1412207357 flags S/SA keep state  label "USER_RULE"
    pass  in  quick  on $LAN inet from 172.16.0.0/16  to <negate_networks>tracker 0100000101 keep state  label "NEGATE_ROUTE: Negate policy routing for destination"
    pass  in  quick  on $LAN  $GWGWG_LAN inet from 172.16.0.0/16 to any tracker 0100000101 keep state  label "USER_RULE: Default allow LAN to any rule"
    pass  in  quick  on $WAN_TO reply-to ( pppoe0 XXX.WAN.TO.GW ) inet proto tcp  from any to XXX.WAN.TO.IP port $admin_ports tracker 1412448206 flags S/SA keep state  label "USER_RULE"

    VPN Rules

    anchor "tftp-proxy/*"

    Any news on the issue?

    Thanks,
    Harry</negate_networks></bogons></virusprot></webconfiguratorlockout></sshlockout></snort2c></snort2c></server></server></rfc1918_private_networks></rfc1918_private_networks></ovpn_remote_networks></ovpn_remote_networks></negate_networks></vpn_networks></bogons></virusprot></snort2c></webconfiguratorlockout></sshlockout>



  • I can confirm, I am having the same issue using the Oct 4th 64bit build. I can provide configurations if needed but I know I have configured everything correctly and the firewall blocks show no blocks. Downgrading back to 2.1.5 fixes the issues with the exact same configuration file. Just to give an over view of what I did.

    created a rule under lan to have my plex machine use the second WAN.
    Created a NAT (port Forward for TCP 32400) to plex machine
    Rule was automatically generated.

    Some fixes that I have found:

    Changing default gateways to second WAN (Puts everything on second WAN), Not the desired behavior
    Downgrading to 2.1.5 (everything works as expected)

    I have this running on my home router so I can flash firmware again if Needed.



  • Actually seems that the one that have posted here have some issues in their configuration.

    They have set a generic policy routing rule on their lan which overrides the reply-to(NAT of their OPTx) tha would make things work.



  • Hi Ermal,

    on my Dual WAN (WAN1 = Cable Router, WAN2 = PPPoE Modem) installation I successfully can ping each WAN IF.
    But I only can reach pfSenses WebGui or anything else on the current Default Gateway.
    By current I mean, that if the default Default Gateway (WAN1) is marked as down (unfortunately still way too often false alarm by apinger) then I am able to reach the WebGui via WAN2.

    I followed your suggestions, switched to manual outbound and added the propsed rule on top of everything, but that doesn't change anything.

    Has anybody had success with this workaround?
    Any other suggestions to make it work?

    Thanks,
    Harry

    Edit:
    Ermal in my case, you're right. I've policy routing on lan to a failover gateway group. That same scenario used to work fine with 2.15



  • Excuse my ignorance, but I can't quite figure out if there is a resolution to this yet? I see from Gloom this should be working if you set manual NAT rules on the OPT1 interface (not sure exactly how to, or what this means?).

    Also, ermal suggests that the problem is that we're using a "generic policy routing rule" - and this is overriding the reply-to. Again, I'm not exactly sure what is meant by "generic policy routing rule" and if there's anything we can do to work around this?

    I have run Wireshark on the NAT target machine, and whilst I don't understand what I'm looking at, you can clearly see that the http request I am making to the NAT forwarded port (from an external network) is hitting the target machine, so it does seem to be the reply that is getting lost somewhere.

    My setup

    WAN = DHCP client
    OPT1 = PIA VPN

    I have set-up forwarding via Firewall > NAT and manually created a floating rule for TCP/UDP traffic on OPT1 coming in the my target IP.

    I have two LAN rules, the first is for target machine and sets the default gateway as OPT1. The second is for all devices and has the default gateway as the router default (which would be WAN).

    The floating rule seems to make no difference, I see the traffic within Wireshark whether or not the floating rule is enabled.



  • It's not working, none of the workarounds are worth trying (short of for experimentation purposes if you want). This is the bug:
    https://redmine.pfsense.org/issues/3760



  • Thanks cmb - That saves me spending any more time on this for now!



  • Please try a next coming snapshot it should be fixed.



  • Hi Ermal,

    thanks a lot for addressing the issue!
    Is the fix already in the Wed Oct 15 14:37:51 CDT 2014 Build?

    Cheers,
    Harry



  • Yeah it should be on that build or newer.



  • Unfortunately, this still doesn't seem to be working for me. I've tried used the automatically generated linked rule and manually creating a floating rule, but nomatter what I try it is still not working for me on the October 16th 09:56:37 version.

    Is there anything else I should be looking at?



  • @mattbunce:

    Unfortunately, this still doesn't seem to be working for me. I've tried used the automatically generated linked rule and manually creating a floating rule, but nomatter what I try it is still not working for me on the October 16th 09:56:37 version.

    Is there anything else I should be looking at?

    Follow the bug report also: https://redmine.pfsense.org/issues/3760
    On that, cmb indicates he has done some testing and it is still not fixed.



  • Cheers Phil

    I have already commented on the bug report too, I just didn't want to get too 'chatty' on there, because it seems the folks on there have done some pretty advanced diagnostics and I didn't want to distract from the good work they were doing :)



  • Another round of fixes.
    Next snapshot should have the missing piece for fixing this.



  • I still can't seem to get this working on the 17th October, 11:27:45 version.

    Have you had a chance to try it out ermal? Perhaps I am not configuring the Nat and Rules pages correctly?

    M



  • This is not working for me and I'm using 2.2-BETA (amd64) built on Fri Oct 17 20:02:23 CDT 2014 FreeBSD 10.1-RC2

    I have attached my LAN, NAT (Port Forwrd), Manual Outbound NAT Rules.

    I have also captured packets on both my WAN, WAN2 and LAN interfaces. On my WAN2 and LAN I can see the Syn, and the Syn Ack packets however I don't see the ack packet. Also there are some retransmissions. When I capture packets on my WAN and filter for port 32400 the capture is blank which tells me that the packet is not being sent out the default gateway. I have attached the .pcap files here in the form of a .txt file if you would like to look at them in wireshark the extension just needs to be changed back to .pcap . Hope this helps

    Thanks,

    P.S. Not sure why I was trying to embed these images from my onedrive folder but it didn't seem to work so I had to add attachments.

    ![LAN Rule.png](/public/imported_attachments/1/LAN Rule.png)
    ![LAN Rule.png_thumb](/public/imported_attachments/1/LAN Rule.png_thumb)
    ![Nat Port forward.png](/public/imported_attachments/1/Nat Port forward.png)
    ![Nat Port forward.png_thumb](/public/imported_attachments/1/Nat Port forward.png_thumb)
    ![outbound Nat.png](/public/imported_attachments/1/outbound Nat.png)
    ![outbound Nat.png_thumb](/public/imported_attachments/1/outbound Nat.png_thumb)
    [capture on LAN.txt](/public/imported_attachments/1/capture on LAN.txt)
    [capture on wan2.txt](/public/imported_attachments/1/capture on wan2.txt)



  • I think my problem is the same.
    If someone has some spare time to read: https://forum.pfsense.org/index.php?topic=82944.0

    I am running:

    Version 2.2-BETA (amd64)
    built on Fri Oct 17 20:02:23 CDT 2014
    FreeBSD 10.1-RC2

    I am available to test any solution and report back.



  • It's a known bug and the developers are working on it. Until then if this is a deal breaker you can try going back to 2.1.5 until the issue is resolved. That is what I'm doing.



  • I'm seeing the same issues on the current snapshot – I've worked around the issue with some sourcerouting and having an upstream system on one of the WAN links handle NATting for now. That works (firewall rules forcing a gateway, then having manual NAT rules disabling NAT).

    It looked like using firewall rules to force the alternate gateway would make the multiWAN NAT rules start working correctly, but I didn't test that yet. It definitely adds another layer of config management to do this, so I eagerly await having this one squished :)



  • Don't change anything on your system trying to fix this, it's part of the packet filter that's currently broken in snapshots. It's slightly more correct now, as reply-to does at least return route correctly. But it's sending traffic with broken checksums, which still leaves it broken. Ermal's heading home from our hackathon in a few hours. He'll fix that when he gets back home later in the week.



  • @cmb:

    Don't change anything on your system trying to fix this, it's part of the packet filter that's currently broken in snapshots. It's slightly more correct now, as reply-to does at least return route correctly. But it's sending traffic with broken checksums, which still leaves it broken. Ermal's heading home from our hackathon in a few hours. He'll fix that when he gets back home later in the week.

    It's just a little portion of the old home network, and thankfully the changes are easy backed out to test any fixes :)

    Thanks a ton for the feedback that it's being worked on, aside from this particular snag, 2.2 is looking stellar for my usage.



  • @stewgoin:

    Thanks a ton for the feedback that it's being worked on, aside from this particular snag, 2.2 is looking stellar for my usage.

    Agreed - I'm new to pfSense, but I'm very impressed with the speed of development and the "get it right in the GUI" approach. I've been using OpenWRT for a few years and there is a real tendency to push any advanced configuration to the command line, leaving much of your intricate configuration hidden within the GUI. I think the approach pfSense takes seems much more sensible - right down for the meaningful descriptions of all parameters in within the GUI



  • @cmb:

    Ermal's heading home from our hackathon in a few hours. He'll fix that when he gets back home later in the week.

    Hi Ermal - have yo had a chance to take a look at this yet?

    M



  • Please test next coming snapshot.



  • This works for me on IPv4 now with a kernel Ermal built with the fix that'll be in the next round of snapshots. Hopefully next snapshot run will be good (first one from the 30th should have it).



  • Thanks Ermal - Everything seems to be working great now :)


Log in to reply