Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    AWS lan to wan problem

    Scheduled Pinned Locked Moved NAT
    1 Posts 1 Posters 2.1k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • P
      psilt
      last edited by

      I tried to read all forums posts of matter, and try all known solutions before posting, so I would not create unneccessary rtfm post. Unfortunately, I've been stuck couple of days implementing pfsense as NAT instance in AWS :P

      But I cannot get out from the private subnet (ping 8.8.8.8, apt-get update). I have outbound nat rule but things are odd. I have tried to disable all AWS security features for all hosts, so eveything is in allow everything security group.

      I created port forwarding rule from public ip/port 8000 to forward ssh to debian running in VPC private subnet. It works fine. I can see the traffic passed in tcpdump properly.

      I have very plain AWS VPC with CIDR 10.0.0.0/16

      public subnet (10.0.0.0/24) has routes
      10.0.0.0/16 local
      0.0.0.0/0 aws-provided internet-gateway

      private subnet (10.0.1.0/24) has routes
      10.0.0.0/16 local
      0.0.0.0/0 interface-id of LAN facing interface

      Public subnet has Pfsense 2.3.2 running with two interfaces, (10.0.0.20 public, 10.0.1.140 private) and elastic ip attached. One patch is installed to pfsense for chrome netblock problem fix.

      I can connect to elasticIP, and reach admin console and ssh to fw with no problem.

      AWS gives my private lan ip 10.0.1.140, pinging it is ok as tcpdump is concerned

      ping 10.0.1.1 results in lan:
      64 bytes from 10.0.1.1: icmp_seq=1 ttl=64 time=0.414 ms
      64 bytes from 10.0.1.1: icmp_seq=2 ttl=64 time=0.445 ms
      64 bytes from 10.0.1.1: icmp_seq=3 ttl=64 time=0.456 ms

      tcpdump -vv -i xn1 on pfsense

      23:37:41.796911 IP (tos 0x0, ttl 64, id 14383, offset 0, flags [none], proto ICMP (1), length 28)
          pfSense.localdomain > 10.0.1.1: ICMP echo request, id 56379, seq 48849, length 8
      23:37:41.797212 IP (tos 0x0, ttl 64, id 39768, offset 0, flags [none], proto ICMP (1), length 28)
          10.0.1.1 > pfSense.localdomain: ICMP echo reply, id 56379, seq 48849, length 8

      this is great - but then, wget www.google.com on client, and it's all quiet on tcpdump, cant get anything by filterin on source or so. Tried to put full logging on all filters, but can't get apt working inside LAN yet.

      interface configurations are as follows

      pflog0: flags=100 <promisc>metric 0 mtu 33160
      pfsync0: flags=0<> metric 0 mtu 1500
              syncpeer: 224.0.0.240 maxupd: 128 defer: on
              syncok: 1
      enc0: flags=0<> metric 0 mtu 1536
              nd6 options=21 <performnud,auto_linklocal>lo0: flags=8049 <up,loopback,running,multicast>metric 0 mtu 16384
              options=600003 <rxcsum,txcsum,rxcsum_ipv6,txcsum_ipv6>inet 127.0.0.1 netmask 0xff000000
              inet6 ::1 prefixlen 128
              inet6 fe80::1%lo0 prefixlen 64 scopeid 0x4
              inet 172.19.0.1 netmask 0xffffff00
              nd6 options=21 <performnud,auto_linklocal>xn0: flags=8843 <up,broadcast,running,simplex,multicast>metric 0 mtu 1500
              options=3 <rxcsum,txcsum>ether 02:4e:d2:8b:33:7f
              inet6 fe80::4e:d2ff:fe8b:337f%xn0 prefixlen 64 scopeid 0x5
              inet 10.0.0.20 netmask 0xffffff00 broadcast 10.0.0.255
              nd6 options=21 <performnud,auto_linklocal>media: Ethernet manual
              status: active
      xn1: flags=8843 <up,broadcast,running,simplex,multicast>metric 0 mtu 1500
              options=3 <rxcsum,txcsum>ether 02:e5:c2:30:2b:85
              inet6 fe80::e5:c2ff:fe30:2b85%xn1 prefixlen 64 scopeid 0x6
              inet 10.0.1.140 netmask 0xffffff00 broadcast 10.0.1.255
              nd6 options=21 <performnud,auto_linklocal>media: Ethernet manual
              status: active

      and the inside VPC debian has like this

      eth0      Link encap:Ethernet  HWaddr 02:75:9b:08:e5:2b
                inet addr:10.0.1.34  Bcast:10.0.1.255  Mask:255.255.255.0
                inet6 addr: fe80::75:9bff:fe08:e52b/64 Scope:Link
                UP BROADCAST RUNNING MULTICAST  MTU:9001  Metric:1
                RX packets:1949 errors:0 dropped:0 overruns:0 frame:0
                TX packets:2088 errors:0 dropped:0 overruns:0 carrier:0
                collisions:0 txqueuelen:1000
                RX bytes:128910 (125.8 KiB)  TX bytes:217848 (212.7 KiB)

      lo        Link encap:Local Loopback
                inet addr:127.0.0.1  Mask:255.0.0.0
                inet6 addr: ::1/128 Scope:Host
                UP LOOPBACK RUNNING  MTU:65536  Metric:1
                RX packets:0 errors:0 dropped:0 overruns:0 frame:0
                TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
                collisions:0 txqueuelen:0
                RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)

      with routes as this

      root@ip-10-0-1-34:~# route
      Kernel IP routing table
      Destination    Gateway        Genmask        Flags Metric Ref    Use Iface
      default        10.0.1.1        0.0.0.0        UG    0      0        0 eth0
      10.0.1.0        *              255.255.255.0  U    0      0        0 eth0

      here are all nat rules from pfsense:

      no nat proto carp all
      nat-anchor "natearly/" all
      nat-anchor "natrules/
      " all
      nat on xn0 inet from <networks_to_nat>to any -> 10.0.0.20 port 1024:65535
      nat on xn1 inet from <networks_to_nat>to any -> 10.0.1.140 port 1024:65535
      nat on xn0 inet from 127.0.0.0/8 to any port = isakmp -> 10.0.0.20 static-port
      nat on xn0 inet from 127.0.0.0/8 to any -> 10.0.0.20 port 1024:65535
      nat on xn1 inet from 127.0.0.0/8 to any port = isakmp -> 10.0.1.140 static-port
      nat on xn1 inet from 127.0.0.0/8 to any -> 10.0.1.140 port 1024:65535
      nat on xn0 inet from 172.25.53.0/24 to any port = isakmp -> 10.0.0.20 static-port
      nat on xn0 inet from 172.25.53.0/24 to any -> 10.0.0.20 port 1024:65535
      nat on xn1 inet from 172.25.53.0/24 to any port = isakmp -> 10.0.1.140 static-port
      nat on xn1 inet from 172.25.53.0/24 to any -> 10.0.1.140 port 1024:65535
      no rdr proto carp all
      rdr-anchor "relayd/" all
      rdr-anchor "tftp-proxy/
      " all
      rdr on xn0 inet proto tcp from any to 10.0.0.20 port = 8000 -> 10.0.1.34 port 22
      rdr-anchor "miniupnpd" all

      related fw rules

      pass out inet all flags S/SA keep state allow-opts label "let out anything IPv4 from firewall host itself"
      pass out inet6 all flags S/SA keep state allow-opts label "let out anything IPv6 from firewall host itself"
      pass out route-to (xn0 10.0.0.1) inet from 10.0.0.20 to ! 10.0.0.0/24 flags S/SA keep state allow-opts label "let out anything from firewall host itself"
      pass out route-to (xn1 10.0.1.1) inet from 10.0.1.140 to ! 10.0.1.0/24 flags S/SA keep state allow-opts label "let out anything from firewall host itself"
      anchor "userrules/" all
      pass in log quick on xn0 reply-to (xn0 10.0.0.1) inet proto icmp from any to 10.0.0.20 keep state label "USER_RULE: Default ICMP rule"
      pass in quick on xn0 reply-to (xn0 10.0.0.1) inet proto tcp from any to 10.0.0.20 port = ssh flags S/SA keep state label "USER_RULE: Default SSH rule replace_src_with_mgmtnet"
      pass in log quick on xn0 reply-to (xn0 10.0.0.1) inet proto tcp from any to 10.0.0.20 port = https flags S/SA keep state label "USER_RULE: Default HTTPS rule replace_src_with_mgmtnet"
      pass in log quick on xn0 reply-to (xn0 10.0.0.1) inet proto tcp from any to 10.0.0.20 port = http flags S/SA keep state label "USER_RULE: Default HTTP rule replace_src_with_mgmtnet"
      pass in quick on xn0 reply-to (xn0 10.0.0.1) inet proto tcp from any to 10.0.1.34 port = ssh flags S/SA keep state label "USER_RULE: NAT 8000 to zabbix 22"
      pass in log quick on xn1 reply-to (xn1 10.0.1.1) inet from 10.0.1.0/24 to any flags S/SA keep state label "USER_RULE: Default allow LAN to any rule"
      pass in log quick on xn1 reply-to (xn1 10.0.1.1) inet from 10.0.1.1 to 10.0.1.140 flags S/SA keep state label "USER_RULE: Easy Rule: Passed from Firewall Log View"
      anchor "tftp-proxy/
      " all</networks_to_nat></networks_to_nat></performnud,auto_linklocal></rxcsum,txcsum></up,broadcast,running,simplex,multicast></performnud,auto_linklocal></rxcsum,txcsum></up,broadcast,running,simplex,multicast></performnud,auto_linklocal></rxcsum,txcsum,rxcsum_ipv6,txcsum_ipv6></up,loopback,running,multicast></performnud,auto_linklocal></promisc>

      1 Reply Last reply Reply Quote 0
      • First post
        Last post
      Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.