Subnets Routing Behind Layer 3 switch



  • Hi,

    From a subnet behind a Layer 3 switch I can't access internet or ping to lan IF of pfsense (172.27.110.155) , but it works from PFSENSE to user PC.

    I can' find what's missing, I've configured routes on switch, routes and rules on Pfsense

    Thanks for your Help

    *********** From User PC 10.164.56.85 ******************
    ping 172.27.110.155
    Haciendo ping a 172.27.110.155 con 32 bytes de datos:
    Tiempo de espera agotado para esta solicitud.
    …...
    Estadísticas de ping para 172.27.110.155:
        Paquetes: enviados = 4, recibidos = 0, perdidos = 4
        (100% perdidos), (100% packet loss)


    *********** From PFsense 172.27.110.155 ******************
    ping 10.164.56.85
    PING 10.164.56.85 (10.164.56.85): 56 data bytes
    64 bytes from 10.164.56.85: icmp_seq=0 ttl=127 time=6.247 ms
    64 bytes from 10.164.56.85: icmp_seq=1 ttl=127 time=5.861 ms
    …..
    10 packets transmitted, 10 packets received, 0.0% packet loss
    round-trip min/avg/max/stddev = 1.197/6.640/24.165/7.543 ms


    The network configuration is as follows:
    PC (10.164.56.85)
        |
    subnet
    10.164.56.0
        |
        |
    Layer 3 Switch (10.164.56.1)
    (ip route 0.0.0.0 0.0.0.0 172.27.110.155)
        |
    172.27.110.252
        |
        |
    Pfsense (172.27.110.155)
    route:("Network"10.164.56.0/24 "Gateway"LANGW - 172.27.110.252 "Interface"LAN)
    rule: IPv4 TCP/UDP 10.164.56.0/24 * * * DefaultGW(internet)
        |
    10.10.155.10 DefaultGW(10.10.155.1)
        |
        |
    ADSL Router (10.10.155.1)
        |
    Internet
    www.google.com


  • LAYER 8 Global Moderator

    "Gateway"LANGW - 172.27.110.252  "Interface"LAN)"

    So that is IP of your layer 3 switch transit network.. What is the mask on this network?

    What rules did you put on the pfsense interface in this transit?
    Just this
    "rule: IPv4 TCP/UDP  10.164.56.0/24  *  *  *  DefaultGW(internet)"

    That would not allow you to ping (icmp) pfsense or anything past pfsense. Did you modify your outbound nat to allow for this downstream network to be natted?



  • Many thanks for your soon answer:

    Regarding your questions:
    The IP/masks are:

    (PFSENSE LAN) 172.27.110.155/22 –----- (Layer 3 routing Switch 172.27.110.252/22--- 10.164.56.1/24) ---- (User PC 10.164.56.85/24)

    I didn't modify outbound nat, could you guide me how to do it?

    Thanks


  • LAYER 8 Global Moderator

    /22 as a transit network??  Why??

    You have to tell pfsense to nat the downstream network to your wan IP..  Just like it automatically nats your connected networks.



  • Thanks again for your help,

    I added a ICMP rule for lan, now ping is working to pfsense from subnet, but still  can't rout the subnets to internet

    /22 mask is because a lot of equipment are in this network.

    For the outbound nat, these are the automatically created rules, what else I need?

    WAN 10.164.56.0/24 **  500 WAN addressAuto created rule for ISAKMP - static route to WAN
    WAN 10.164.56.0/24 *** WAN address
    Auto created rule - static route to WAN
    LAN 10.164.56.0/24  **  500LAN addressAuto created rule for ISAKMP - static route to LAN
    LAN 10.164.56.0/24  *** LAN address
    Auto created rule - static route to LAN

    and added this outbound nat rule:
    Interface Source Source Port    Destination         Destination Port         NAT Address NAT Port Static Port Description Actions
    LAN         any           *               10.164.56.0/24       *                 LAN address *


  • LAYER 8 Global Moderator

    "/22 mask is because a lot of equipment are in this network."

    WRONG - completely BORKED.. there is not suppose to be equipment in a transit network!!  Your going to have asymmetrical routing problems with this.. Why does nobody seem to understand what a freaking transit network is..  I understand there are lot of new users here, new to routing, etc. etc..  But it just freaking amazes the shit out of me how a transit network seems to be completely greek to everyone.  This is basic routing 101…  Arrrggghhhh ;)

    "WAN 10.164.56.0/24 *** WAN address*Auto created rule - static route to WAN"

    So this was auto created.. Hmmm good.. So see my example you have this - please post a picture.  Where are you seeing this "auto created rule"  There is no comment in the outbound nat created.

    Here I simulated a downstream network via a gateway I created to 192.168.9.200, then created a route to 10.200.200/24 via this gateway and it added this to my auto outbound nats.




  • Thanks for answering my stupid questions, as you allready noticed I'm not a networking expert

    I mean there is a lot of equipment in the 172.27.108.0/22 network, that I can't control, and there are other subnets.

    :-\ And you are totally right I don't understand what is "transit network" :-\

    Please find attached my network configuration, The traffic you can see  from firewall rules must be from SQUID that is configured in pfesense, and perfectlly working

    Thanks

    ![net conf.jpg](/public/imported_attachments/1/net conf.jpg)
    ![net conf.jpg_thumb](/public/imported_attachments/1/net conf.jpg_thumb)
    ![net conf 1.jpg](/public/imported_attachments/1/net conf 1.jpg)
    ![net conf 1.jpg_thumb](/public/imported_attachments/1/net conf 1.jpg_thumb)


  • LAYER 8 Global Moderator

    Well how are you devices to access squid running on pfsense (which you didn't mention) before when your rule forces all traffic out your wan gateway.  That rule does not allow access to anything on pfsense like squid.. Just sends it out the wan gateway.

    So does stuff on your 10.164.56 network need to talk to stuff on your transit?  Which all have a gateway of pfsense 10.27.110.155??

    This causes asymmetrical routing




  • I am running a layer 3 switch behind pfsense.  There is a thread on this site under installation where I setup my configuration.  Looking at your config the only thing I see different is on the firewall rules under LAN is I have asterisk instead of default gateway name.  Probably won't make a difference.  I ran a /24 mask to start with in the transit network and it worked fine. I now use a 30 bit mask instead.  I am using a Cisco SG300-28 switch.

    I assume you are using an access port on the layer 3 switch and not a trunk port.



  • Thanks for your comments.

    I've just noticed a curious thing, the routing is working well to "outlook.office365.com", "www.cnn.com",  but not to "www.google.com", "www.ibm.com", I don't have any special rule for this. From Pfsense webconsole, al pings are 100% OK

    ping outlook.office365.com

    Haciendo ping a outlook.ms-acdc.office.com [40.102.35.114] con 32 bytes de datos:
    Respuesta desde 40.102.35.114: bytes=32 tiempo=236ms TTL=236
    Respuesta desde 40.102.35.114: bytes=32 tiempo=227ms TTL=236
    Respuesta desde 40.102.35.114: bytes=32 tiempo=227ms TTL=236
    Respuesta desde 40.102.35.114: bytes=32 tiempo=234ms TTL=236

    ping www.google.com

    Haciendo ping a www.google.com [172.217.28.228] con 32 bytes de datos:
    Tiempo de espera agotado para esta solicitud.
    Tiempo de espera agotado para esta solicitud.
    Tiempo de espera agotado para esta solicitud.
    Tiempo de espera agotado para esta solicitud.

    Estadísticas de ping para 172.217.28.228:
        Paquetes: enviados = 4, recibidos = 0, perdidos = 4
        (100% perdidos),



  • Hi,

    Thanks for your help, finally I found the problem, it was a bad defined rule in the switch router, at least I´ve learned a lesson about routing in PFsense.

    Thanks


  • LAYER 8 Global Moderator

    You still have a asymmetrical issue if devices on your transit need to be accessed from the downstream network(s) or the stuff on the transit access them.

    Simple to fix with just bringing up an actual transit between your L3 and pfsense vs using a host network as a transit network.



  • John they way I handled the asymmetrical issue is to let the layer 3 switch handle all the local routing.  In effect the layer 3 switch is the gateway for all local traffic and pfsense is the gateway for all internet traffic.  It worked fine this way.

    I decided I wanted my router in a VLAN by it self so I did move to using a 30 mask for the router VLAN.  But the layer 3 switch is still the gateway for all local LAN traffic and pfsense is the gateway for all internet traffic.


  • LAYER 8 Global Moderator

    "I decided I wanted my router in a VLAN by it self so I did move to using a 30 mask for the router VLAN"

    So you created a transit ;) between the layer 3 and pfsense..



  • Yes I did move over to a /30 mask but not right away.  I ran a /24 mask for a couple of months with no problems.  The reason I moved to a /30 mask was to eliminate all the chattiness from the work stations slowing down the router to where all traffic going to the router VLAN is destined for the internet.  When workstations start talking to each other behind the scenes this causes the router to wait because it is on the same network.  By isolating the router and allowing the layer 3 switch to switch local traffic I have effectively removed all local bottle necks for the router.


  • LAYER 8 Global Moderator

    "The reason I moved to a /30 mask was to eliminate all the chattiness from the work stations slowing down the router to where all traffic going to the router VLAN is destined for the internet."

    Huh ???

    At a complete loss to why would there be workstations on a transit?  And why would devices on a network talking to each other have anything to do with your router - are you using a HUB?



  • @johnpoz:

    "The reason I moved to a /30 mask was to eliminate all the chattiness from the work stations slowing down the router to where all traffic going to the router VLAN is destined for the internet."

    Huh ???

    At a complete loss to why would there be workstations on a transit?  And why would devices on a network talking to each other have anything to do with your router - are you using a HUB?

    What we are talking about John is all because you said asymmetrical routing does not work.  I said it does if you set it up right.  I used it for a while with workstations on the same network.  I later then changed for other reasons other then it did not work.


  • LAYER 8 Global Moderator

    "asymmetrical routing does not work.  I said it does if you set it up right."

    Yeah you can make it work with host routing, or source natting.  Or allowing your firewall out of state traffic.. etc. etc..

    Does not matter if you "can make it work".. The point it is setting it up in the first place is just plain BORKED!!  If you condone or promote anyone running a asymmetrical network.. You for sure should not be in the networking biz that is for damn freaking sure!  Sorry that is not ment as personal attack in anyway.. Its just stating my honest to goodness opinion.



  • Maybe with pfsense it is hard but it is easy to setup using a layer 3 switch.  All you have to do is point the local traffic to the layer 3 switch.  It knows where everything is and will route or switch to the device.  Nothing hard.  It is a good way to bring a layer 3 switch into the fold without disrupting normal operations.


Log in to reply