2 LAN + 1 WAN - LAN's cannot reach eachother.



  • Seeking help getting LAN & OPT1 to route traffic to eachother.  Both can already see the internet via NAT to WAN, so that works fine.  But no matter the rules I try, and I've googled and tried for days now, I cannot get clients on LAN to talk to clients on OPT1, or vice versa.  I need this LAN <-> OPT1 routing so I can simulate different bandwidth profiles between clients and a server, and to more closely mimic a production environment.

    PFSense version:
    2.0.3-RELEASE (i386)
    built on Fri Apr 12 10:22:57 EDT 2013
    FreeBSD 8.1-RELEASE-p13

    Interface Details:

    LAN
    –-----------
    Type: Static
    Network: 10.100.0.0/24
    IP: 10.100.0.254
    Gateway: None
    Block private networks: no
    Block bogon networks: no

    OPT1

    Type: Static
    Network: 10.100.1.0/24
    IP: 10.100.1.254
    Gateway: None
    Block private networks: no
    Block bogon networks: no

    Firewall Rules:
    LAN

    ID: (blank)
    Proto: *
    Source *
    Port: *
    Dest: LAN Address
    Gateway: *
    Queue: *
    Schedule: (blank)
    Description: Anti-Lockout Rule

    [Following added by me after googling, as the global rule after this one hadn't worked for this.  I am betting this one isn't needed.]
    ID: (blank)
    Proto: *
    Source: LAN net
    Port: *
    Dest: OPT1 net
    Gateway: *
    Queue: none
    Schedule: (blank)

    ID: (blank)
    Proto: *
    Source: *
    Port: *
    Dest: *
    Gateway: *
    Queue: none
    Schedule: (blank)

    OPT1
    –------
    _ID: (blank)
    Proto: *
    Source: OPT1 net
    Port: *
    Dest: LAN net
    Gateway: *
    Queue: none
    Schedule: (blank)

    ID: (blank)
    Proto: *
    Source: *
    Port: *
    Dest: *
    Gateway: *
    Queue: none
    Schedule: (blank)

    What is trying to ping what:

    Ping / Traceroute fails…
    10.100.0.20 <-> 10.100.1.10

    10.100.0.20, Netmask 255.255.255.0, Gateway 10.100.0.254
    10.100.1.10, Netmask 255.255.255.0, Gateway 10.100.1.254

    Ping / Traceroute succeeds:
    10.100.0.20(Lan network client) <-> 10.100.1.254 (Opt1 gateway)
    10.100.1.10(Opt1 network client) <-> 10.100.0.254 (Lan gateway)

    So they can see each other's gateways, but not traverse them.

    What on earth am I doing wrong?_



  • Ping / Traceroute fails…
    10.100.0.20 <-> 10.100.1.10

    Does the traceroute show the 1st hop to the router correctly?
    Does ping to the clients work from pfSense?
    Do the clients respond to ping at all? (i.e. from another machine on the local LAN)
    It would be a shame to be testing to a Windows client that has the firewall on and doesn't respond to ping.
    Your network and rules look good and simple - it should work.



  • From PFSense's shell I can ping 10.100.0.20.

    I can also ping 10.100.1.10.

    Both are linux systems with IPTables disabled.

    Traceroute 10.100.1.10 -> 10.100.0.20:

    traceroute to 10.100.0.20 (10.100.0.20), 30 hops max, 60 byte packets
    1  10.100.1.254  0.116 ms  0.119 ms  0.109 ms
    2  * * *
    Gives up after 30.

    Interesting…  Traceroute the other way 10.100.0.20 -> 10.100.1.10:

    traceroute to 10.100.1.10 (10.100.1.10), 30 hops max, 60 byte packets
    1  10.100.0.20  3000.666 ms !H  3000.664 ms !H  3000.659 ms !H

    Hrmmmm.  Thoughts?



  • Traceroute the other way 10.100.0.20 -> 10.100.1.10:
    traceroute to 10.100.1.10 (10.100.1.10), 30 hops max, 60 byte packets
    1  10.100.0.20  3000.666 ms !H  3000.664 ms !H  3000.659 ms !H

    Hrmmmm.  Thoughts?

    10.100.0.20 works normally on its own subnet, where it does not have to use its default gateway or routing table.
    It seems to think it is:
    a) its own default gateway, (in this case it would not reach the internet either) or
    b) it has a route to 10.100.1.0/24 that points to itself.
    and it took 3 seconds to send the ICMP packet to itself. Something wrong with its routing configuration???

    Try another "ordinary" system in 10.100.0.0/24 and confirm it is working, then you can work on 10.100.0.20



  • If you want LAN and OPT1 to be able to talk to each other.

    Simply set up:

    Firewall Rules:

    LAN
    –------
    ID: (blank)
    Proto: *
    Source LAN net
    Port: *
    Dest: * 
    Gateway: *
    Queue: *
    Schedule: (blank)
    Description: Allow LAN to ALL

    OPT1

    ID: (blank)
    Proto: *
    Source OPT1 net
    Port: *
    Dest: * 
    Gateway: *
    Queue: *
    Schedule: (blank)
    Description: Allow OPT1 to ALL

    With no other rules listed above these, the LAN and OPT1 will be able to communicate with each other and the WAN.
    If you have blocking rules listed above these rules, all bets are off.



  • Yep, that last one didn't work, either. :(



  • No one is going to like this suggestion, but its times like this where I wipe the drive, reinstall and try again.  Because 1 WAN and 2 LAN is so simple.  Should work fast as you add the interfaces, subnets, IP, DHCP and firewall rules to allow.



  • Certainly a possibility, since this one was built, configured, changed, changed, backed up, corrupted, restored, and upgraded…



  • I'd start fresh if you don't have a complex config.



  • The firewall rules are simple.  The lengthy part will be setting back up all the dns entries in forwarder, the HAProxy plugin and some other stuff.



  • I would expect to have to goto System Routing and setup a Gateway named something like LAN1_OPT1GW assigning the LAN1 interface and having a gateway and monitor IP matching the IP of OPT1.  OPT1 of course being on a different subnet than LAN1. This should create a static route automatically. Then goto Status, Gateways to ensure the gateway link is established. At this point you should be able to ping devices in OPT1 subnet from the LAN1 subnet. Rules could also be added to define specific traffic to pass from Lan1 via the LAN1_OPT1GW gateway.