Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    My pfSense keeps breaking (novel inside…)

    Scheduled Pinned Locked Moved General pfSense Questions
    46 Posts 5 Posters 19.8k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • C
      cmb
      last edited by

      You need to set a default gateway even if you're policy routing your egress traffic. And uncheck the default gateway switching.

      What type of WANs?

      1 Reply Last reply Reply Quote 0
      • S
        soteriologist
        last edited by

        I just rechecked to have just that default DSL line checked as the "Default Gateway" and unchecked "Allow default gateway switching" under "System >> Advanced >> Miscellaneous >> Load Balancing".
        Still no ping response beyond the router/modem it's plugged into.
        Stil no internet connection.
        Still nothing in state/firewall logs.

        Re-ran traceroute and this is what I have now:

        netstat -rn -f inet;  traceroute -n 8.8.8.8

        Routing tables

        Internet:
        Destination        Gateway            Flags    Refs      Use  Netif Expire
        default            192.168.2.2        US          0       51    em2
        127.0.0.1          link#12            UH          0     3524    lo0
        192.168.2.0/24     link#7             U           0    19094    em2
        192.168.2.2        link#7             UHS         0        0    lo0
        192.168.168.0/24   link#5             U           0      341    em0
        192.168.168.1      link#5             UHS         0        0    lo0
        traceroute to 8.8.8.8 (8.8.8.8), 64 hops max, 52 byte packets

        1 Reply Last reply Reply Quote 0
        • S
          soteriologist
          last edited by

          The three WANs that I have are:
          One DSL connection through a Verizon router/modem
          One T1 through an AdTran DSU/CSU
          One T1 through a Cisco DSU/CSU

          AT THE MOMENT I'm ONLY using the DSL for testing.  Just to simplify things and because the entire company is actively using the two T1s at office.  But when I had everything plugged in during off hours, they were all working fine until… well... everything stopped working.  So I had to put everything back they way I had it in the very late hours of the night before everyone came back in the next day and BACK TO THE DRAWING BOARD!

          1 Reply Last reply Reply Quote 0
          • W
            wallabybob
            last edited by

            @soteriologist:

            Re-ran traceroute and this is what I have now:

            Your traceroute output is incomplete.

            1 Reply Last reply Reply Quote 0
            • S
              soteriologist
              last edited by

              @wallabybob:

              @soteriologist:

              Re-ran traceroute and this is what I have now:

              Your traceroute output is incomplete.

              Ya… just realized that I hadn't copied everything, SORRY!

              Here we go:

              netstat -rn -f inet;  t                                                                            raceroute -n 8.8.8.8
              Routing tables

              Internet:
              Destination        Gateway            Flags    Refs      Use  Netif Expire
              default            192.168.2.2        US          0      201    em2
              127.0.0.1          link#12            UH          0    3568    lo0
              192.168.2.0/24    link#7            U          0    22223    em2
              192.168.2.2        link#7            UHS        0        0    lo0
              192.168.168.0/24  link#5            U          0      341    em0
              192.168.168.1      link#5            UHS        0        0    lo0
              traceroute to 8.8.8.8 (8.8.8.8), 64 hops max, 52 byte packets
              1  * * *
              2  * *traceroute: sendto: Host is down
              traceroute: wrote 8.8.8.8 52 chars, ret=-1
              *
              traceroute: sendto: Host is down
              3 traceroute: wrote 8.8.8.8 52 chars, ret=-1
              *traceroute: sendto: Host is down
              traceroute: wrote 8.8.8.8 52 chars, ret=-1
              *traceroute: sendto: Host is down
              traceroute: wrote 8.8.8.8 52 chars, ret=-1
              *
              traceroute: sendto: Host is down
              4 traceroute: wrote 8.8.8.8 52 chars, ret=-1
              *traceroute: sendto: Host is down
              traceroute: wrote 8.8.8.8 52 chars, ret=-1
              *traceroute: sendto: Host is down
              traceroute: wrote 8.8.8.8 52 chars, ret=-1
              *
              traceroute: sendto: Host is down
              5 traceroute: wrote 8.8.8.8 52 chars, ret=-1
              *traceroute: sendto: Host is down
              traceroute: wrote 8.8.8.8 52 chars, ret=-1
              *traceroute: sendto: Host is down
              traceroute: wrote 8.8.8.8 52 chars, ret=-1
              *
              traceroute: sendto: Host is down
              6 traceroute: wrote 8.8.8.8 52 chars, ret=-1
              *traceroute: sendto: Host is down
              traceroute: wrote 8.8.8.8 52 chars, ret=-1
              *traceroute: sendto: Host is down
              traceroute: wrote 8.8.8.8 52 chars, ret=-1
              *
              traceroute: sendto: Host is down
              7 traceroute: wrote 8.8.8.8 52 chars, ret=-1
              ^C

              1 Reply Last reply Reply Quote 0
              • W
                wallabybob
                last edited by

                @soteriologist:

                traceroute to 8.8.8.8 (8.8.8.8), 64 hops max, 52 byte packets
                1  * * *
                2  * *traceroute: sendto: Host is down
                traceroute: wrote 8.8.8.8 52 chars, ret=-1
                *
                traceroute: sendto: Host is down

                1    looks like your DSL router doesn't reply to the traceroute probes - that's allowed
                2    "Host is down" suggests the DSL router's WAN link is down or for some other reason (also lost its default route?) it doesn't know where to forward the traceroute probes.

                1 Reply Last reply Reply Quote 0
                • S
                  soteriologist
                  last edited by

                  I'm posting this connected through the DSL router.  So I know for a fact the router is working great.

                  Here's traceroute from my laptop attached to one of the other ports on the DSL router:

                  C:\Users\Administrator>tracert 8.8.8.8

                  Tracing route to google-public-dns-a.google.com [8.8.8.8]
                  over a maximum of 30 hops:

                  1    <1 ms    <1 ms    <1 ms  192.168.2.1
                   2    45 ms    45 ms    43 ms  10.39.5.1
                   3    50 ms    50 ms    49 ms  at-3-2-0-1715.LAX01-CORE-RTR2.verizon-gni.net [1
                  30.81.194.2]
                   4    50 ms    51 ms   125 ms  so-1-1-1-0.LAX01-BB-RTR2.verizon-gni.net [130.81
                  .16.130]
                   5    54 ms    51 ms    51 ms  0.so-6-0-0.XT2.LAX9.ALTER.NET [152.63.10.157]
                   6   122 ms   121 ms   121 ms  0.so-1-0-0.XT2.NYC4.ALTER.NET [152.63.64.126]
                   7   128 ms   133 ms   137 ms  TenGigE0-7-0-0.GW8.NYC4.ALTER.NET [152.63.22.45]

                  8   128 ms   127 ms   129 ms  Internet-gw.customer.alter.net [152.179.72.66]
                   9   128 ms   129 ms   129 ms  72.14.238.232
                  10   122 ms   119 ms   133 ms  209.85.252.2
                  11   131 ms   132 ms   130 ms  72.14.239.93
                  12   126 ms   127 ms   129 ms  72.14.236.200
                  13   142 ms   143 ms   129 ms  216.239.49.145
                  14   129 ms   129 ms   128 ms  google-public-dns-a.google.com [8.8.8.8]

                  Trace complete.

                  1 Reply Last reply Reply Quote 0
                  • chpalmerC
                    chpalmer
                    last edited by

                    Can you show a picture of your WAN interface configuration page?  The one connected to your DSL modem??

                    Triggering snowflakes one by one..
                    Intel(R) Core(TM) i5-4590T CPU @ 2.00GHz on an M400 WG box.

                    1 Reply Last reply Reply Quote 0
                    • S
                      soteriologist
                      last edited by

                      Here are printscreens of both my Interface and Gateway setup for the WAN.

                      GatewaySetup.png
                      GatewaySetup.png_thumb
                      WAN.png
                      WAN.png_thumb

                      1 Reply Last reply Reply Quote 0
                      • C
                        cmb
                        last edited by

                        Your gateway cannot be the same as the IP on its own interface.

                        1 Reply Last reply Reply Quote 0
                        • S
                          soteriologist
                          last edited by

                          SORRY ABOUT THAT!  Slight oversight the last time I reset to defaults.  >_<

                          I've set it to the proper 192.168.2.1 now.  (which it always has been except for the very last time I was setting it back up in a hurry and didn't pay attention).  =/

                          Ok, so with that properly in place this is the situation:

                          Still can't get a connection THROUGH pfSense.  (even though it seems to be FULLY communicating over both the WAN port and the LAN port).

                          When I run a traceroute from PFSENSE this is what I get:

                          netstat -rn -f inet; traceroute -n 8.8.8.8
                          Routing tables

                          Internet:
                          Destination        Gateway            Flags    Refs      Use  Netif Expire
                          default            192.168.2.1        UGS         0    19379    em2
                          4.2.2.2            192.168.2.2        UHS         0    14429    em2
                          10.39.5.1          192.168.2.1        UGHS        0   172195    em2
                          127.0.0.1          link#12            UH          0    10979    lo0
                          192.168.2.0/24     link#7             U           0     1989    em2
                          192.168.2.2        link#7             UHS         0        0    lo0
                          192.168.168.0/24   link#5             U           0    18742    em0
                          192.168.168.1      link#5             UHS         0        0    lo0
                          traceroute to 8.8.8.8 (8.8.8.8), 64 hops max, 52 byte packets
                          1  192.168.2.1  1.178 ms  0.298 ms  0.274 ms
                          2  10.39.5.1  41.901 ms  40.120 ms  39.915 ms
                          3  130.81.194.2  48.084 ms  48.009 ms  47.786 ms
                          4  130.81.16.130  52.014 ms  47.993 ms  48.036 ms
                          5  152.63.112.49  48.072 ms  48.021 ms  48.001 ms
                          6  152.63.64.126  113.913 ms  113.986 ms  114.065 ms
                          7  152.63.21.125  125.921 ms
                             152.63.16.125  134.030 ms
                             152.63.21.65  116.058 ms
                          8  152.179.72.66  127.841 ms  126.109 ms  127.908 ms
                          9  209.85.255.68  114.056 ms  116.028 ms
                             72.14.232.244  118.041 ms
                          10  209.85.251.37  127.852 ms
                             209.85.252.2  130.120 ms  130.018 ms
                          11  72.14.239.93  127.940 ms  128.042 ms  127.902 ms
                          12  72.14.236.200  126.165 ms  123.836 ms  125.851 ms
                          13  216.239.49.145  126.229 ms  125.828 ms  125.908 ms
                          14  8.8.8.8  124.284 ms  125.769 ms  128.827 ms

                          When I run a ping from pfSense this is what I get:

                          Ping output:
                          PING 8.8.8.8 (8.8.8.8) from 192.168.2.2: 56 data bytes
                          64 bytes from 8.8.8.8: icmp_seq=0 ttl=54 time=125.634 ms
                          64 bytes from 8.8.8.8: icmp_seq=1 ttl=54 time=124.157 ms
                          64 bytes from 8.8.8.8: icmp_seq=2 ttl=54 time=125.198 ms

                          –- 8.8.8.8 ping statistics ---
                          3 packets transmitted, 3 packets received, 0.0% packet loss
                          round-trip min/avg/max/stddev = 124.157/124.996/125.634/0.620 ms

                          But when I run a traceroute from my COMPTUER  I get:
                          C:\Users\Administrator>tracert 8.8.8.8

                          Tracing route to 8.8.8.8 over a maximum of 30 hops

                          1    <1 ms    <1 ms    <1 ms  192.168.168.1
                           2     *        *        *     Request timed out.
                           3     *        *        *     Request timed out.
                           4     *        *        *     Request timed out.
                           5     *        *        *     Request timed out.
                           6     *        *        *     Request timed out.
                           7     *     ^C

                          I also can't ping, or make any other connection to the outside world.

                          And AGAIN still not showing ANY states or ANYTHING in the firewall logs (even with logging turned on for the default rule.  It won't even show what it's NOT blocking.  It's just empty as always.)

                          1 Reply Last reply Reply Quote 0
                          • S
                            soteriologist
                            last edited by

                            Ok, so check this out as well…

                            I'm connect to the LAN port, so I'm testing pings on my laptop THROUGH pfSense and this is what I get:

                            C:\Users\Administrator>ping 192.168.2.1

                            Pinging 192.168.2.1 with 32 bytes of data:
                            Request timed out.
                            Request timed out.
                            Request timed out.
                            Request timed out.

                            Ping statistics for 192.168.2.1:
                               Packets: Sent = 4, Received = 0, Lost = 4 (100% loss),

                            C:\Users\Administrator>ping 192.168.2.2

                            Pinging 192.168.2.2 with 32 bytes of data:
                            Reply from 192.168.2.2: bytes=32 time<1ms TTL=64
                            Reply from 192.168.2.2: bytes=32 time<1ms TTL=64
                            Reply from 192.168.2.2: bytes=32 time<1ms TTL=64
                            Reply from 192.168.2.2: bytes=32 time<1ms TTL=64

                            Ping statistics for 192.168.2.2:
                               Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
                            Approximate round trip times in milli-seconds:
                               Minimum = 0ms, Maximum = 0ms, Average = 0ms

                            C:\Users\Administrator>ping 192.168.168.1

                            Pinging 192.168.168.1 with 32 bytes of data:
                            Reply from 192.168.168.1: bytes=32 time<1ms TTL=64
                            Reply from 192.168.168.1: bytes=32 time<1ms TTL=64
                            Reply from 192.168.168.1: bytes=32 time<1ms TTL=64
                            Reply from 192.168.168.1: bytes=32 time<1ms TTL=64

                            Ping statistics for 192.168.168.1:
                               Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
                            Approximate round trip times in milli-seconds:
                               Minimum = 0ms, Maximum = 0ms, Average = 0ms

                            C:\Users\Administrator>ping 8.8.8.8

                            Pinging 8.8.8.8 with 32 bytes of data:
                            Request timed out.
                            Request timed out.
                            Request timed out.
                            Request timed out.

                            Ping statistics for 8.8.8.8:
                               Packets: Sent = 4, Received = 0, Lost = 4 (100% loss),

                            So it'll respond to a ping on the port I'm directly connected to AND pass the ping along to the WAN interface, BUT it won't let anything past it.  =/

                            1 Reply Last reply Reply Quote 0
                            • chpalmerC
                              chpalmer
                              last edited by

                              I skimmed your posts but didn't see so hope this isn't redundant…

                              What do your outbound LAN rules and outbound port rules look like?

                              Triggering snowflakes one by one..
                              Intel(R) Core(TM) i5-4590T CPU @ 2.00GHz on an M400 WG box.

                              1 Reply Last reply Reply Quote 0
                              • S
                                soteriologist
                                last edited by

                                They're the factory defaults for right now.  Since I've reset everything.

                                The ONLY thing I've changed on the rules is that I turned logging on for the "Default allow LAN to any rule".

                                Attached are some printscreens.

                                LAN.png
                                LAN.png_thumb
                                Floating.png
                                Floating.png_thumb
                                WAN.png
                                WAN.png_thumb

                                1 Reply Last reply Reply Quote 0
                                • chpalmerC
                                  chpalmer
                                  last edited by

                                  On the    /firewall_nat_out.php  page-

                                  Try choosing "Manual Outbound NAT rule generation"    save and see what results you get…

                                  Assuming also that you have unchecked the last two boxes on your WAN interface page...    "Block Private Networks" and "Block bogon networks"      Gotta ask...    ;)

                                  Triggering snowflakes one by one..
                                  Intel(R) Core(TM) i5-4590T CPU @ 2.00GHz on an M400 WG box.

                                  1 Reply Last reply Reply Quote 0
                                  • S
                                    soteriologist
                                    last edited by

                                    Had just blocking of Bogons checked, so I unchecked it.

                                    Blocking of local addresses was already UNCHECKED cause I know that with these DSL connections they're using local IPs so I wouldn't want that checked.

                                    I turned on manual NAT out as you suggested and attached is a printscreen of what that looks like now.

                                    I'm still having the same results.

                                    NATOut.png
                                    NATOut.png_thumb

                                    1 Reply Last reply Reply Quote 0
                                    • W
                                      wallabybob
                                      last edited by

                                      My next step would be to verify that (say) a ping from LAN client computer to 8.8.8.8 was getting to the pfSense LAN interface. (Packet capture on the LAN interface, displaying just traffic to selected destination.)

                                      If you don't see the traffic in the packet capture then I would look at the IP configuration of the client: Does it have the correct IP address for its default gateway? (Should be the IP address of the pfSense LAN interface.) Does it have the correct MAC address for that IP address?

                                      1 Reply Last reply Reply Quote 0
                                      • S
                                        soteriologist
                                        last edited by

                                        Packet captured pings from laptop through the LAN interface (which all came back as "Request timed out." on my laptop) and this is what the capture looks like:

                                        16:38:32.824386 00:1b:38:62:d0:1a (oui Unknown) > 54:04:a6:f3:2b:05 (oui Unknown), ethertype IPv4 (0x0800), length 74: (tos 0x0, ttl 128, id 29038, offset 0, flags [none], proto ICMP (1), length 60)
                                            192.168.168.100 > google-public-dns-a.google.com: ICMP echo request, id 1, seq 117, length 40
                                        16:38:37.490298 00:1b:38:62:d0:1a (oui Unknown) > 54:04:a6:f3:2b:05 (oui Unknown), ethertype IPv4 (0x0800), length 74: (tos 0x0, ttl 128, id 29085, offset 0, flags [none], proto ICMP (1), length 60)
                                            192.168.168.100 > google-public-dns-a.google.com: ICMP echo request, id 1, seq 118, length 40
                                        16:38:42.490628 00:1b:38:62:d0:1a (oui Unknown) > 54:04:a6:f3:2b:05 (oui Unknown), ethertype IPv4 (0x0800), length 74: (tos 0x0, ttl 128, id 29122, offset 0, flags [none], proto ICMP (1), length 60)
                                            192.168.168.100 > google-public-dns-a.google.com: ICMP echo request, id 1, seq 119, length 40
                                        16:38:47.490840 00:1b:38:62:d0:1a (oui Unknown) > 54:04:a6:f3:2b:05 (oui Unknown), ethertype IPv4 (0x0800), length 74: (tos 0x0, ttl 128, id 29150, offset 0, flags [none], proto ICMP (1), length 60)
                                            192.168.168.100 > google-public-dns-a.google.com: ICMP echo request, id 1, seq 120, length 40

                                        As for the MAC addresses: I checked my arp tables on my laptop and all of the correct MAC addresses show as being associated with the correct IP addresses for both the WAN and LAN interfaces.  So everything is good there.

                                        1 Reply Last reply Reply Quote 0
                                        • W
                                          wallabybob
                                          last edited by

                                          So now with the same ping running, does a packet capture on the WAN interface show the ping leaving the pfSense box? If so, what is the source IP address?

                                          1 Reply Last reply Reply Quote 0
                                          • S
                                            soteriologist
                                            last edited by

                                            Interface set to WAN, Host set to 8.8.8.8, Full detail, Reverse DNS Lookup turned OFF:

                                            18:01:55.094629 54:04:a6:f3:2b:07 > 00:26:62:1b:09:87, ethertype IPv4 (0x0800), length 74: (tos 0x0, ttl 127, id 4003, offset 0, flags [none], proto ICMP (1), length 60)
                                                192.168.168.100 > 8.8.8.8: ICMP echo request, id 1, seq 133, length 40
                                            18:01:59.796664 54:04:a6:f3:2b:07 > 00:26:62:1b:09:87, ethertype IPv4 (0x0800), length 74: (tos 0x0, ttl 127, id 4006, offset 0, flags [none], proto ICMP (1), length 60)
                                                192.168.168.100 > 8.8.8.8: ICMP echo request, id 1, seq 134, length 40
                                            18:02:04.795955 54:04:a6:f3:2b:07 > 00:26:62:1b:09:87, ethertype IPv4 (0x0800), length 74: (tos 0x0, ttl 127, id 4009, offset 0, flags [none], proto ICMP (1), length 60)
                                                192.168.168.100 > 8.8.8.8: ICMP echo request, id 1, seq 135, length 40
                                            18:02:09.795179 54:04:a6:f3:2b:07 > 00:26:62:1b:09:87, ethertype IPv4 (0x0800), length 74: (tos 0x0, ttl 127, id 4013, offset 0, flags [none], proto ICMP (1), length 60)
                                                192.168.168.100 > 8.8.8.8: ICMP echo request, id 1, seq 136, length 40

                                            Interface set to WAN, Host set to 8.8.8.8, Full detail, Reverse DNS Lookup turned ON:

                                            18:03:26.746456 54:04:a6:f3:2b:07 (oui Unknown) > 00:26:62:1b:09:87 (oui Unknown), ethertype IPv4 (0x0800), length 74: (tos 0x0, ttl 127, id 4104, offset 0, flags [none], proto ICMP (1), length 60)
                                                192.168.168.100 > google-public-dns-a.google.com: ICMP echo request, id 1, seq 137, length 40
                                            18:03:31.294265 54:04:a6:f3:2b:07 (oui Unknown) > 00:26:62:1b:09:87 (oui Unknown), ethertype IPv4 (0x0800), length 74: (tos 0x0, ttl 127, id 4109, offset 0, flags [none], proto ICMP (1), length 60)
                                                192.168.168.100 > google-public-dns-a.google.com: ICMP echo request, id 1, seq 138, length 40
                                            18:03:36.293619 54:04:a6:f3:2b:07 (oui Unknown) > 00:26:62:1b:09:87 (oui Unknown), ethertype IPv4 (0x0800), length 74: (tos 0x0, ttl 127, id 4115, offset 0, flags [none], proto ICMP (1), length 60)
                                                192.168.168.100 > google-public-dns-a.google.com: ICMP echo request, id 1, seq 139, length 40
                                            18:03:41.293819 54:04:a6:f3:2b:07 (oui Unknown) > 00:26:62:1b:09:87 (oui Unknown), ethertype IPv4 (0x0800), length 74: (tos 0x0, ttl 127, id 4118, offset 0, flags [none], proto ICMP (1), length 60)
                                                192.168.168.100 > google-public-dns-a.google.com: ICMP echo request, id 1, seq 140, length 40

                                            192.168.168.100 is of course my laptop's IP address (where the pings are coming from).

                                            1 Reply Last reply Reply Quote 0
                                            • First post
                                              Last post
                                            Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.