Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    Dual (2)WAN / Multi (9)LAN Routing Issue with Public IP's

    Scheduled Pinned Locked Moved Routing and Multi WAN
    38 Posts 2 Posters 5.0k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • DerelictD Offline
      Derelict LAYER 8 Netgate
      last edited by

      can pfSense do this or will i need to remove nat'ed clients subnets to another pfsense box to allow route only platform?

      It will only NAT if there are NAT rules.  You will absolutely have to enable manual outbound NAT and only have rules for the networks/interfaces you want to NAT for.

      Any possibility you can start smaller during some dead time and work one WAN/one LAN at a time until get get more familiar with what pfSense needs?

      Chattanooga, Tennessee, USA
      A comprehensive network diagram is worth 10,000 words and 15 conference calls.
      DO NOT set a source address/port in a port forward or firewall rule unless you KNOW you need it!
      Do Not Chat For Help! NO_WAN_EGRESS(TM)

      1 Reply Last reply Reply Quote 0
      • D Offline
        Disturbed1
        last edited by

        i will give a try at 4/5am to add nat rules & switch to manual nat… (production enviro)

        before this setup i had...

        isp2 i've had on pfsense hardware before, everything was great, then tried the edgemax...  worked but missed features pfsense had...

        isp1 i had two pfsense vm's, first was route only platform, second used for nat'n clients behind single ip from /29 pool...

        surffered hvac failure in room, edgemax baked, vm mechine toasted raid card... etc etc etc...

        so as a last minute get everything back online i decided to go multi wan single hardware... i figured i missed something, thx for pointing out the manual outbound....

        will post update in morning....

        1 Reply Last reply Reply Quote 0
        • DerelictD Offline
          Derelict LAYER 8 Netgate
          last edited by

          Ok.  You will do it in the opposite order though - enable manual outbound NAT then tweak the rules.

          I am pretty sure you can enable manual outbound any time.  All of the rules that are placed there by the automatic process will be there so there should be no  change in behavior until you start tweaking rules.  Also keep in mind that you can just disable the ones you don't want instead of deleting them.

          You should see two rules for each LAN interface.  One with a static port for IPsec and one for everything else.  I'd just duplicate those for interfaces you want to NAT for.

          Chattanooga, Tennessee, USA
          A comprehensive network diagram is worth 10,000 words and 15 conference calls.
          DO NOT set a source address/port in a port forward or firewall rule unless you KNOW you need it!
          Do Not Chat For Help! NO_WAN_EGRESS(TM)

          1 Reply Last reply Reply Quote 0
          • D Offline
            Disturbed1
            last edited by

            good morning…

            so it's 5am here... (5:30am by time i post this)

            i set manual out bound, set nat rules...

            connected to pink ip pool, googled whats my ip, still came up with pink gw ip.... when connected to blue ip pools (tested three subnets) proper ip showed...

            :(

            so since i was here, i checked route only platform... and two things happened... google showed the ip from pink ip pool, and blue subnets could talk to each other... but nat'ed clients no access to net....

            so i'm a little baffled here....

            edit: also rebooted just incase... same resault after....

            1 Reply Last reply Reply Quote 0
            • DerelictD Offline
              Derelict LAYER 8 Netgate
              last edited by

              Screen shots of Firewall->NAT->Outbound and Firewall->Rules (Interfaces) please.  You have something not right.  Routing Only Platform is not what you want.

              Chattanooga, Tennessee, USA
              A comprehensive network diagram is worth 10,000 words and 15 conference calls.
              DO NOT set a source address/port in a port forward or firewall rule unless you KNOW you need it!
              Do Not Chat For Help! NO_WAN_EGRESS(TM)

              1 Reply Last reply Reply Quote 0
              • D Offline
                Disturbed1
                last edited by

                ka… snapshoots attached..... firewall rules same for both nat'ed subnets...

                natoutbound.png
                natoutbound.png_thumb
                firewallrules.png
                firewallrules.png_thumb

                1 Reply Last reply Reply Quote 0
                • DerelictD Offline
                  Derelict LAYER 8 Netgate
                  last edited by

                  Why are you natting your public IPs?

                  Chattanooga, Tennessee, USA
                  A comprehensive network diagram is worth 10,000 words and 15 conference calls.
                  DO NOT set a source address/port in a port forward or firewall rule unless you KNOW you need it!
                  Do Not Chat For Help! NO_WAN_EGRESS(TM)

                  1 Reply Last reply Reply Quote 0
                  • D Offline
                    Disturbed1
                    last edited by

                    no reason, forgot to remove it…. trail and error trying different things.... cleared it out now... just 172.16.5.0/172.16.10.0 in there now...

                    1 Reply Last reply Reply Quote 0
                    • DerelictD Offline
                      Derelict LAYER 8 Netgate
                      last edited by

                      OK.  And what, specifically, isn't working now?  Let's work one LAN/WAN interface at a time.

                      Chattanooga, Tennessee, USA
                      A comprehensive network diagram is worth 10,000 words and 15 conference calls.
                      DO NOT set a source address/port in a port forward or firewall rule unless you KNOW you need it!
                      Do Not Chat For Help! NO_WAN_EGRESS(TM)

                      1 Reply Last reply Reply Quote 0
                      • D Offline
                        Disturbed1
                        last edited by

                        first problem…

                        isp1,

                        204.101.*.208/29 subnet...

                        when connected to ip in that pool, i goto google and search whats my ip.... should show 204.101..209, but shows isp1 pfsense gw ip 67.69..254....

                        lan_bell = 204.101.*.208/29 subnet

                        VIP.png
                        VIP.png_thumb
                        lan_bell.png
                        lan_bell.png_thumb
                        gateways.png
                        gateways.png_thumb

                        1 Reply Last reply Reply Quote 0
                        • DerelictD Offline
                          Derelict LAYER 8 Netgate
                          last edited by

                          With those NAT rules it would do that.  Without them it should not.  Did you clear states after deleting the NAT rules?  You can clear only the states in question by filtering on 204.101.*.209.

                          I wouldn't have the Proxy ARP VIPs.  I'd have type Other - if you need any at all.  Out of curiosity, what is the IP address of the LAN_BELL interface?

                          Chattanooga, Tennessee, USA
                          A comprehensive network diagram is worth 10,000 words and 15 conference calls.
                          DO NOT set a source address/port in a port forward or firewall rule unless you KNOW you need it!
                          Do Not Chat For Help! NO_WAN_EGRESS(TM)

                          1 Reply Last reply Reply Quote 0
                          • D Offline
                            Disturbed1
                            last edited by

                            @Derelict:

                            With those NAT rules it would do that.  Without them it should not.  Did you clear states after deleting the NAT rules?  You can clear only the states in question by filtering on 204.101.*.209.

                            I wouldn't have the Proxy ARP VIPs.  I'd have type Other - if you need any at all.  Out of curiosity, what is the IP address of the LAN_BELL interface?

                            my bad for typo… 204.101..209 should have been 204.101..208/29 meaning a ip from that pool....

                            ka i changed VIP to other... checked and is now working.....  THANK YOU….  i swear i tried that once but must have over looked....

                            i will try changing the other isp2 vip setting in a minute and see if that changes the block between isp2's smaller subnets....

                            1 Reply Last reply Reply Quote 0
                            • D Offline
                              Disturbed1
                              last edited by

                              okay, so both VIPs are set to Other, and seem to be okay…. connected to different pools and whats my ip was correct in all tests from all pools....

                              while i was connected to pool 216.185..192/26 i tried to access email server in 216.185..160/27 pool with no success.... while connected to a pool fed from isp1, i could access with no prob... when tried from a pool feed by isp2 only time i could access email was while i was inside same pool as server....

                              here attached are the screenshots of firewall rules for both...

                              192.png
                              192.png_thumb
                              160.png
                              160.png_thumb

                              1 Reply Last reply Reply Quote 0
                              • DerelictD Offline
                                Derelict LAYER 8 Netgate
                                last edited by

                                Check all your netmasks and gateways.  What happens when it fails?  Anything in your firewall logs?

                                Chattanooga, Tennessee, USA
                                A comprehensive network diagram is worth 10,000 words and 15 conference calls.
                                DO NOT set a source address/port in a port forward or firewall rule unless you KNOW you need it!
                                Do Not Chat For Help! NO_WAN_EGRESS(TM)

                                1 Reply Last reply Reply Quote 0
                                • D Offline
                                  Disturbed1
                                  last edited by

                                  all netmasks correct… doubled checked with online subnet calculator...

                                  here ping resaults from 216.185..201/26 pool pinging 216.185.166/27

                                  C:\Users\chrism>ping 216.185.*.166

Pinging 216.185.*.166 with 32 bytes of data:
Reply from 216.185.*.1: TTL expired in transit.
Reply from 216.185.*.1: TTL expired in transit.
Reply from 216.185.*.1: TTL expired in transit.
Reply from 216.185.*.1: TTL expired in transit.

Ping statistics for 216.185.*.166:
    Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),

                                  packet capture:

                                  216.185.*.1 > 216.185.*.201: ICMP time exceeded in-transit, length 36
	(tos 0x0, ttl 1, id 6454, offset 0, flags [none], proto UDP (17), length 56)

                                  checked system logs / firewall, tried both source and Destination with ip 216.185.*.201, clear, nothing there…..

                                  1 Reply Last reply Reply Quote 0
                                  • DerelictD Offline
                                    Derelict LAYER 8 Netgate
                                    last edited by

                                    What routes have you put in System->Routing ??  You probably want to get rid of everything.

                                    Chattanooga, Tennessee, USA
                                    A comprehensive network diagram is worth 10,000 words and 15 conference calls.
                                    DO NOT set a source address/port in a port forward or firewall rule unless you KNOW you need it!
                                    Do Not Chat For Help! NO_WAN_EGRESS(TM)

                                    1 Reply Last reply Reply Quote 0
                                    • D Offline
                                      Disturbed1
                                      last edited by

                                      zero….

                                      screen shot of gateways in post #10

                                      staticroutes.png
                                      staticroutes.png_thumb

                                      1 Reply Last reply Reply Quote 0
                                      • DerelictD Offline
                                        Derelict LAYER 8 Netgate
                                        last edited by

                                        When you traceroute it what IPs is it bouncing between?

                                        Chattanooga, Tennessee, USA
                                        A comprehensive network diagram is worth 10,000 words and 15 conference calls.
                                        DO NOT set a source address/port in a port forward or firewall rule unless you KNOW you need it!
                                        Do Not Chat For Help! NO_WAN_EGRESS(TM)

                                        1 Reply Last reply Reply Quote 0
                                        • D Offline
                                          Disturbed1
                                          last edited by

                                          now this is where it gets funny….

                                          216.185.*.201 is from Canada(CA) in region North America

TraceRoute from Network-Tools.com to 216.185.*.201 [*************]
Hop	(ms)	(ms)	(ms)		     IP Address	Host name
1 	  Timed out 	  Timed out 	  Timed out 	     	  -  
2 	  19 	  22 	  20 	     4.69.158.145	 ae-205-3605.edge4.chicago2.level3.net  
3 	  25 	  19 	  19 	     4.69.158.145	 ae-205-3605.edge4.chicago2.level3.net  
4 	  24 	  24 	  24 	     4.28.68.22	  -  
5 	  49 	  49 	  49 	     199.212.168.186	 ge8-2.hcap7-tor.bb.allstream.net  
6 	  41 	  41 	  41 	     216.13.105.170	 216-13-105-170.dedicated.allstream.net  
7 	  42 	  42 	  42 	     66.207.112.74	 bb1-core-bra-kaa-g11-v3983.fibrewired.ca  
8 	  46 	  45 	  48 	     216.185.*.110	 mercuri.ca  
9 	  39 	  40 	  40 	     67.69.*.254	  -  
10 	  Timed out 	  Timed out 	  Timed out 	     	  -  
11 	  Timed out 	  Timed out 	  Timed out 	     	  - 

                                          216.185..110 don't belong to me, but belongs to my isp2 upstream… 67.69..254 is my pfsense isp1 gw.....

                                          used http://network-tools dot com to get this

                                          isp2 should have went 216.185..110 then to 216.185..1 which is the isp2 gw i connect to... then 216.185.*.2 which is my pfsense box

                                          1 Reply Last reply Reply Quote 0
                                          • DerelictD Offline
                                            Derelict LAYER 8 Netgate
                                            last edited by

                                            No.  I meant from inside.  Usually when TTLs expire in your situation you have a routing loop.

                                            Traceroute to .166 from .201

                                            Chattanooga, Tennessee, USA
                                            A comprehensive network diagram is worth 10,000 words and 15 conference calls.
                                            DO NOT set a source address/port in a port forward or firewall rule unless you KNOW you need it!
                                            Do Not Chat For Help! NO_WAN_EGRESS(TM)

                                            1 Reply Last reply Reply Quote 0
                                            • First post
                                              Last post
                                            Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.